All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
@ 2013-02-12 16:28 ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Hello,

This series of patches introduces PCIe support for the Marvell Armada
370 and Armada XP.

This PATCHv3 follows:
 * PATCHv2, sent on January, 28th 2013
 * RFCv1, sent on December, 7th 2012

Thanks to all the people who discussed on the previous version of the
patch set. The discussions have been long and complicated, but
certainly very useful.

In order to make progress on this, and help reducing the size of the
patch set, I would ask if it would be possible to merge patches 1 to
17 from the series for 3.9 (only preparation work), keeping the rest
for 3.10. The patches in question are PCI-related, ARM-related, and
mvebu/orion-related.

Changes between v2 and v3:

 * Use of_irq_map_pci() instead of of_irq_map_raw(), as suggested by
   Andrew Murray. In order to do this, we moved the interrupt-map and
   interrupt-map-mask DT properties from the main PCIe controller node
   to the DT subnodes representing each PCIe interface.

 * Remove the usage of the emulated host bridge.

 * Move the emulated PCI-to-PCI bridge code into the Marvell PCI
   driver itself, in order to allow a tighter integration. Suggested
   by Bjorn Helgaas and Jason Gunthorpe.

 * Make the allocation of address decoding windows dynamic: it's when
   memory accesses or I/O accesses are enabled at the PCI-to-PCI
   bridge level that we allocate and setup the corresponding address
   decoding window. Requested by Bjorn Helgaas.

 * Fixed the implementation of I/O accesses to use I/O addresses that
   fall within the normal IO_SPACE_LIMIT. This required using the
   "remap" functionality of address decoding windows, and therefore
   some changes in the address decoding window allocator. Follows a
   long discussion about I/O accesses.

 * Set up a correct bus number in the configuration of the PCIe
   interfaces so that we don't have to fake bus numbers
   anymore. Requested by Jason Gunthorpe.

 * Fix the of_pci_get_devfn() implementation according to Stephen
   Warren's comment.

 * Use CFLAGS_ instead of ccflags to add the mach-mvebu and plat-orion
   include paths when building the pci-mvebu driver. This ensures that
   the include paths are only added when building this specific
   driver. Requested by Stephen Warren.

 * Fix the ->resource_align() to only apply on bus 0 (the one on which
   the emulated PCI-to-PCI bridges sit), and to request an alignment
   on the size of the window (and not only 64 KB for I/O windows and 1
   MB for memory windows).

 * Clarified the commit log of "clk: mvebu: create parent-child
   relation for PCIe clocks on Armada 370"

A quick description of the patches:

 * Patches 1 to 3 add PCI-related Device Tree parsing functions. Those
   patches are common with the Nvidia Tegra PCIe patch set from
   Thierry Redding. They are included in this series so that it can be
   tested easily.

 * Patch 4 extends the ARM PCI core to store a per-controller private
   data pointer. This patch is common with the Nvidia Tegra PCIe patch
   set from Thierry Redding. It is included in this series so that it
   can be tested easily.

 * Patch 5 fixes a problem in lib/devres.c that prevents certain
   PCI-related functions from being visible on NO_IOPORT platforms. I
   know this patch isn't acceptable by itself, but the discussion
   about this has been so huge and went in so many directions that in
   the end, I don't know what is the correct way of fixing this. If an
   agreement is found on how to fix this properly, I'm willing to work
   on it if needed.

 * Patch 6 extends the ARM PCI core with an additional hook that a PCI
   controller driver can register and get called to realign PCI
   ressource addresses. This is needed for the support of Marvell PCIe
   interfaces because the address decoding windows for I/O ranges have
   a granularity of 64 KB, while the PCI standard requires only a 4 KB
   alignement. See the patch itself for details.

 * Patch 7 fixes a mistake in the interrupt controller node of the
   Armada 370/XP Device Tree, which was invisible until we started
   using the of_irq_map_raw() function, needed in our PCIe support.

 * Patches 8 and 9 fix some issues in the Armada 370/XP clock gating
   driver, related to PCIe interfaces.

 * Patches 10 and 11 are cleanup/refactoring of the common plat-orion
   address decoding code, in preparation for further changes related
   to PCIe.

 * Patches 12 to 17 introduce a ORION_ADDR_MAP_NO_REMAP define that is
   used by existing Marvell SoC code to say "I don't need this window
   to remap anything". Previously a -1 value was used as the remap
   address to communicate the fact that no remap is needed, but this
   prevents any remap address higher than 2 GB.

 * Patch 18 removes __init from a few address window decoding
   functions that are now needed after boot.

 * Patch 19 introduces in the common plat-orion address decoding code
   functions to allocate/free an address decoding window. Until now,
   the address decoding windows were configured statically. With
   Armada XP having up to 10 PCIe interfaces, we don't want to
   allocate useless address decoding windows statically, so we move to
   a more dynamic model in which address decoding windows are
   configured only for the PCIe interfaces that are actually in use.

 * Patch 20 removes __init from a few PCIe functions that are now
   needed after boot.

 * Patch 21 improves the Armada 370/XP specific address decoding code
   to provide functions that add and remove an address decoding window
   for a given PCIe interface. It relies on the common functions added
   in patch 19.

 * Patch 22 makes the common plat-orion PCIe code available on
   PLAT_ORION platforms such as ARCH_MVEBU.

 * Patch 23 creates the drivers/pci/host directory and makes the
   related minimal changes to Kconfig/Makefile. This patch will
   trivially conflict with the NVidia Tegra PCIe support posted by
   Thierry Redding, which also creates the drivers/pci/host directory.

 * Patch 24 contains the Armada 370/XP PCIe driver itself, that
   implements the necessary operations required by the ARM PCI core,
   and configures the address decoding windows as needed. This driver
   relies on a Device Tree description of the PCIe interfaces.

 * Patch 25 marks the ARCH_MVEBU platform has having PCI available,
   which allows the compilation of the PCIe support.

 * Patches 26 and 27 add the SoC-level Device Tree informations
   related to PCIe for Armada 370 and Armada XP.

 * Patch 28 to 31 add the board-level Device Tree informations related
   to PCIe for the Armada XP DB, Armada 370 DB, PlatHome OpenBlocks
   AX3-4 and GlobalScale Mirabox boards.

 * Patch 32 updates mvebu_defconfig with PCI and USB support.

This patch set applies on top of v3.8-rc7, and has been pushed
at:

  git://github.com/MISL-EBU-System-SW/mainline-public.git marvell-pcie-v3

Thanks,

Thomas

---
Output of lspci -vvv:

00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:03.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: 00010000-00010fff
	Memory behind bridge: c1000000-c10fffff
	Prefetchable memory behind bridge: c1100000-c11fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:04.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:05.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
	I/O behind bridge: 00020000-00020fff
	Memory behind bridge: c1200000-c12fffff
	Prefetchable memory behind bridge: c1300000-c13fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:06.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

03:00.0 SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 PCI-e 4-port SATA-II (rev 02)
	Subsystem: Marvell Technology Group Ltd. Device 11ab
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 105
	Region 0: Memory at c1000000 (64-bit, non-prefetchable) [size=1M]
	Region 2: I/O ports at 10000 [size=256]
	[virtual] Expansion ROM at c1100000 [disabled] [size=512K]
	Capabilities: [40] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000012345678  Data: 0000
	Capabilities: [60] Express (v1) Legacy Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <256ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <256ns, L1 unlimited
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Kernel driver in use: sata_mv

05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
	Subsystem: Intel Corporation PRO/1000 PT Server Adapter
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 106
	Region 0: Memory at c1200000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at c1220000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at 20000 [disabled] [size=32]
	[virtual] Expansion ROM at c1300000 [disabled] [size=128K]
	Capabilities: [c8] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [e0] Express (v1) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-c1-c4-fe
	Kernel driver in use: e1000e

Boot messages from the PCI subsystem:

mvebu-pcie pcie-controller.1: PCIe0.0: link down
mvebu-pcie pcie-controller.1: PCIe0.1: link down
mvebu-pcie pcie-controller.1: PCIe0.2: link up
mvebu-pcie pcie-controller.1: PCIe0.3: link down
mvebu-pcie pcie-controller.1: PCIe2.0: link up
mvebu-pcie pcie-controller.1: PCIe3.0: link down
mvebu-pcie pcie-controller.1: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x0000-0xfffff]
pci_bus 0000:00: root bus resource [mem 0xc1000000-0xc8ffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
PCI: bus0: Fast back to back transfers disabled
pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:06.0: bridge configuration invalid ([bus 00-00]), reconfiguring
PCI: bus1: Fast back to back transfers enabled
PCI: bus2: Fast back to back transfers enabled
PCI: bus3: Fast back to back transfers disabled
PCI: bus4: Fast back to back transfers enabled
PCI: bus5: Fast back to back transfers disabled
PCI: bus6: Fast back to back transfers enabled
pci 0000:00:03.0: BAR 8: assigned [mem 0xc1000000-0xc10fffff]
pci 0000:00:03.0: BAR 9: assigned [mem 0xc1100000-0xc11fffff pref]
pci 0000:00:05.0: BAR 8: assigned [mem 0xc1200000-0xc12fffff]
pci 0000:00:05.0: BAR 9: assigned [mem 0xc1300000-0xc13fffff pref]
pci 0000:00:03.0: BAR 7: assigned [io  0x10000-0x10fff]
pci 0000:00:05.0: BAR 7: assigned [io  0x20000-0x20fff]
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:00:02.0: PCI bridge to [bus 02]
pci 0000:03:00.0: BAR 0: assigned [mem 0xc1000000-0xc10fffff 64bit]
pci 0000:03:00.0: BAR 6: assigned [mem 0xc1100000-0xc117ffff pref]
pci 0000:03:00.0: BAR 2: assigned [io  0x10000-0x100ff]
pci 0000:00:03.0: PCI bridge to [bus 03]
pci 0000:00:03.0:   bridge window [io  0x10000-0x10fff]
pci 0000:00:03.0:   bridge window [mem 0xc1000000-0xc10fffff]
pci 0000:00:03.0:   bridge window [mem 0xc1100000-0xc11fffff pref]
pci 0000:00:04.0: PCI bridge to [bus 04]
pci 0000:05:00.0: BAR 0: assigned [mem 0xc1200000-0xc121ffff]
pci 0000:05:00.0: BAR 1: assigned [mem 0xc1220000-0xc123ffff]
pci 0000:05:00.0: BAR 6: assigned [mem 0xc1300000-0xc131ffff pref]
pci 0000:05:00.0: BAR 2: assigned [io  0x20000-0x2001f]
pci 0000:00:05.0: PCI bridge to [bus 05]
pci 0000:00:05.0:   bridge window [io  0x20000-0x20fff]
pci 0000:00:05.0:   bridge window [mem 0xc1200000-0xc12fffff]
pci 0000:00:05.0:   bridge window [mem 0xc1300000-0xc13fffff pref]
pci 0000:00:06.0: PCI bridge to [bus 06]
PCI: enabling device 0000:00:01.0 (0140 -> 0143)
PCI: enabling device 0000:00:02.0 (0140 -> 0143)
PCI: enabling device 0000:00:03.0 (0140 -> 0143)
PCI: enabling device 0000:00:04.0 (0140 -> 0143)
PCI: enabling device 0000:00:05.0 (0140 -> 0143)
PCI: enabling device 0000:00:06.0 (0140 -> 0143)


^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
@ 2013-02-12 16:28 ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Hello,

This series of patches introduces PCIe support for the Marvell Armada
370 and Armada XP.

This PATCHv3 follows:
 * PATCHv2, sent on January, 28th 2013
 * RFCv1, sent on December, 7th 2012

Thanks to all the people who discussed on the previous version of the
patch set. The discussions have been long and complicated, but
certainly very useful.

In order to make progress on this, and help reducing the size of the
patch set, I would ask if it would be possible to merge patches 1 to
17 from the series for 3.9 (only preparation work), keeping the rest
for 3.10. The patches in question are PCI-related, ARM-related, and
mvebu/orion-related.

Changes between v2 and v3:

 * Use of_irq_map_pci() instead of of_irq_map_raw(), as suggested by
   Andrew Murray. In order to do this, we moved the interrupt-map and
   interrupt-map-mask DT properties from the main PCIe controller node
   to the DT subnodes representing each PCIe interface.

 * Remove the usage of the emulated host bridge.

 * Move the emulated PCI-to-PCI bridge code into the Marvell PCI
   driver itself, in order to allow a tighter integration. Suggested
   by Bjorn Helgaas and Jason Gunthorpe.

 * Make the allocation of address decoding windows dynamic: it's when
   memory accesses or I/O accesses are enabled at the PCI-to-PCI
   bridge level that we allocate and setup the corresponding address
   decoding window. Requested by Bjorn Helgaas.

 * Fixed the implementation of I/O accesses to use I/O addresses that
   fall within the normal IO_SPACE_LIMIT. This required using the
   "remap" functionality of address decoding windows, and therefore
   some changes in the address decoding window allocator. Follows a
   long discussion about I/O accesses.

 * Set up a correct bus number in the configuration of the PCIe
   interfaces so that we don't have to fake bus numbers
   anymore. Requested by Jason Gunthorpe.

 * Fix the of_pci_get_devfn() implementation according to Stephen
   Warren's comment.

 * Use CFLAGS_ instead of ccflags to add the mach-mvebu and plat-orion
   include paths when building the pci-mvebu driver. This ensures that
   the include paths are only added when building this specific
   driver. Requested by Stephen Warren.

 * Fix the ->resource_align() to only apply on bus 0 (the one on which
   the emulated PCI-to-PCI bridges sit), and to request an alignment
   on the size of the window (and not only 64 KB for I/O windows and 1
   MB for memory windows).

 * Clarified the commit log of "clk: mvebu: create parent-child
   relation for PCIe clocks on Armada 370"

A quick description of the patches:

 * Patches 1 to 3 add PCI-related Device Tree parsing functions. Those
   patches are common with the Nvidia Tegra PCIe patch set from
   Thierry Redding. They are included in this series so that it can be
   tested easily.

 * Patch 4 extends the ARM PCI core to store a per-controller private
   data pointer. This patch is common with the Nvidia Tegra PCIe patch
   set from Thierry Redding. It is included in this series so that it
   can be tested easily.

 * Patch 5 fixes a problem in lib/devres.c that prevents certain
   PCI-related functions from being visible on NO_IOPORT platforms. I
   know this patch isn't acceptable by itself, but the discussion
   about this has been so huge and went in so many directions that in
   the end, I don't know what is the correct way of fixing this. If an
   agreement is found on how to fix this properly, I'm willing to work
   on it if needed.

 * Patch 6 extends the ARM PCI core with an additional hook that a PCI
   controller driver can register and get called to realign PCI
   ressource addresses. This is needed for the support of Marvell PCIe
   interfaces because the address decoding windows for I/O ranges have
   a granularity of 64 KB, while the PCI standard requires only a 4 KB
   alignement. See the patch itself for details.

 * Patch 7 fixes a mistake in the interrupt controller node of the
   Armada 370/XP Device Tree, which was invisible until we started
   using the of_irq_map_raw() function, needed in our PCIe support.

 * Patches 8 and 9 fix some issues in the Armada 370/XP clock gating
   driver, related to PCIe interfaces.

 * Patches 10 and 11 are cleanup/refactoring of the common plat-orion
   address decoding code, in preparation for further changes related
   to PCIe.

 * Patches 12 to 17 introduce a ORION_ADDR_MAP_NO_REMAP define that is
   used by existing Marvell SoC code to say "I don't need this window
   to remap anything". Previously a -1 value was used as the remap
   address to communicate the fact that no remap is needed, but this
   prevents any remap address higher than 2 GB.

 * Patch 18 removes __init from a few address window decoding
   functions that are now needed after boot.

 * Patch 19 introduces in the common plat-orion address decoding code
   functions to allocate/free an address decoding window. Until now,
   the address decoding windows were configured statically. With
   Armada XP having up to 10 PCIe interfaces, we don't want to
   allocate useless address decoding windows statically, so we move to
   a more dynamic model in which address decoding windows are
   configured only for the PCIe interfaces that are actually in use.

 * Patch 20 removes __init from a few PCIe functions that are now
   needed after boot.

 * Patch 21 improves the Armada 370/XP specific address decoding code
   to provide functions that add and remove an address decoding window
   for a given PCIe interface. It relies on the common functions added
   in patch 19.

 * Patch 22 makes the common plat-orion PCIe code available on
   PLAT_ORION platforms such as ARCH_MVEBU.

 * Patch 23 creates the drivers/pci/host directory and makes the
   related minimal changes to Kconfig/Makefile. This patch will
   trivially conflict with the NVidia Tegra PCIe support posted by
   Thierry Redding, which also creates the drivers/pci/host directory.

 * Patch 24 contains the Armada 370/XP PCIe driver itself, that
   implements the necessary operations required by the ARM PCI core,
   and configures the address decoding windows as needed. This driver
   relies on a Device Tree description of the PCIe interfaces.

 * Patch 25 marks the ARCH_MVEBU platform has having PCI available,
   which allows the compilation of the PCIe support.

 * Patches 26 and 27 add the SoC-level Device Tree informations
   related to PCIe for Armada 370 and Armada XP.

 * Patch 28 to 31 add the board-level Device Tree informations related
   to PCIe for the Armada XP DB, Armada 370 DB, PlatHome OpenBlocks
   AX3-4 and GlobalScale Mirabox boards.

 * Patch 32 updates mvebu_defconfig with PCI and USB support.

This patch set applies on top of v3.8-rc7, and has been pushed
at:

  git://github.com/MISL-EBU-System-SW/mainline-public.git marvell-pcie-v3

Thanks,

Thomas

---
Output of lspci -vvv:

00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:03.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: 00010000-00010fff
	Memory behind bridge: c1000000-c10fffff
	Prefetchable memory behind bridge: c1100000-c11fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:04.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:05.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
	I/O behind bridge: 00020000-00020fff
	Memory behind bridge: c1200000-c12fffff
	Prefetchable memory behind bridge: c1300000-c13fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

00:06.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
	Capabilities: [fc] <chain broken>

03:00.0 SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 PCI-e 4-port SATA-II (rev 02)
	Subsystem: Marvell Technology Group Ltd. Device 11ab
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 105
	Region 0: Memory at c1000000 (64-bit, non-prefetchable) [size=1M]
	Region 2: I/O ports at 10000 [size=256]
	[virtual] Expansion ROM@c1100000 [disabled] [size=512K]
	Capabilities: [40] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000012345678  Data: 0000
	Capabilities: [60] Express (v1) Legacy Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <256ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <256ns, L1 unlimited
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Kernel driver in use: sata_mv

05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
	Subsystem: Intel Corporation PRO/1000 PT Server Adapter
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 106
	Region 0: Memory at c1200000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at c1220000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at 20000 [disabled] [size=32]
	[virtual] Expansion ROM@c1300000 [disabled] [size=128K]
	Capabilities: [c8] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [e0] Express (v1) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-c1-c4-fe
	Kernel driver in use: e1000e

Boot messages from the PCI subsystem:

mvebu-pcie pcie-controller.1: PCIe0.0: link down
mvebu-pcie pcie-controller.1: PCIe0.1: link down
mvebu-pcie pcie-controller.1: PCIe0.2: link up
mvebu-pcie pcie-controller.1: PCIe0.3: link down
mvebu-pcie pcie-controller.1: PCIe2.0: link up
mvebu-pcie pcie-controller.1: PCIe3.0: link down
mvebu-pcie pcie-controller.1: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x0000-0xfffff]
pci_bus 0000:00: root bus resource [mem 0xc1000000-0xc8ffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
PCI: bus0: Fast back to back transfers disabled
pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:06.0: bridge configuration invalid ([bus 00-00]), reconfiguring
PCI: bus1: Fast back to back transfers enabled
PCI: bus2: Fast back to back transfers enabled
PCI: bus3: Fast back to back transfers disabled
PCI: bus4: Fast back to back transfers enabled
PCI: bus5: Fast back to back transfers disabled
PCI: bus6: Fast back to back transfers enabled
pci 0000:00:03.0: BAR 8: assigned [mem 0xc1000000-0xc10fffff]
pci 0000:00:03.0: BAR 9: assigned [mem 0xc1100000-0xc11fffff pref]
pci 0000:00:05.0: BAR 8: assigned [mem 0xc1200000-0xc12fffff]
pci 0000:00:05.0: BAR 9: assigned [mem 0xc1300000-0xc13fffff pref]
pci 0000:00:03.0: BAR 7: assigned [io  0x10000-0x10fff]
pci 0000:00:05.0: BAR 7: assigned [io  0x20000-0x20fff]
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:00:02.0: PCI bridge to [bus 02]
pci 0000:03:00.0: BAR 0: assigned [mem 0xc1000000-0xc10fffff 64bit]
pci 0000:03:00.0: BAR 6: assigned [mem 0xc1100000-0xc117ffff pref]
pci 0000:03:00.0: BAR 2: assigned [io  0x10000-0x100ff]
pci 0000:00:03.0: PCI bridge to [bus 03]
pci 0000:00:03.0:   bridge window [io  0x10000-0x10fff]
pci 0000:00:03.0:   bridge window [mem 0xc1000000-0xc10fffff]
pci 0000:00:03.0:   bridge window [mem 0xc1100000-0xc11fffff pref]
pci 0000:00:04.0: PCI bridge to [bus 04]
pci 0000:05:00.0: BAR 0: assigned [mem 0xc1200000-0xc121ffff]
pci 0000:05:00.0: BAR 1: assigned [mem 0xc1220000-0xc123ffff]
pci 0000:05:00.0: BAR 6: assigned [mem 0xc1300000-0xc131ffff pref]
pci 0000:05:00.0: BAR 2: assigned [io  0x20000-0x2001f]
pci 0000:00:05.0: PCI bridge to [bus 05]
pci 0000:00:05.0:   bridge window [io  0x20000-0x20fff]
pci 0000:00:05.0:   bridge window [mem 0xc1200000-0xc12fffff]
pci 0000:00:05.0:   bridge window [mem 0xc1300000-0xc13fffff pref]
pci 0000:00:06.0: PCI bridge to [bus 06]
PCI: enabling device 0000:00:01.0 (0140 -> 0143)
PCI: enabling device 0000:00:02.0 (0140 -> 0143)
PCI: enabling device 0000:00:03.0 (0140 -> 0143)
PCI: enabling device 0000:00:04.0 (0140 -> 0143)
PCI: enabling device 0000:00:05.0 (0140 -> 0143)
PCI: enabling device 0000:00:06.0 (0140 -> 0143)

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 01/32] of/pci: Provide support for parsing PCI DT ranges property
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk, Andrew Murray,
	Andrew Murray, Liviu Dudau

From: Andrew Murray <andrew.murray@arm.com>

DT bindings for PCI host bridges often use the ranges property to describe
memory and IO ranges - this binding tends to be the same across architectures
yet several parsing implementations exist, e.g. arch/mips/pci/pci.c,
arch/powerpc/kernel/pci-common.c, arch/sparc/kernel/pci.c and
arch/microblaze/pci/pci-common.c (clone of PPC). Some of these duplicate
functionality provided by drivers/of/address.c.

This patch provides a common iterator-based parser for the ranges property, it
is hoped this will reduce DT representation differences between architectures
and that architectures will migrate in part to this new parser.

It is also hoped (and the motativation for the patch) that this patch will
reduce duplication of code when writing host bridge drivers that are supported
by multiple architectures.

This patch provides struct resources from a device tree node, e.g.:

	u32 *last = NULL;
	struct resource res;
	while ((last = of_pci_process_ranges(np, res, last))) {
		//do something with res
	}

Platforms with quirks can then do what they like with the resource or migrate
common quirk handling to the parser. In an ideal world drivers can just request
the obtained resources and pass them on (e.g. pci_add_resource_offset).

Signed-off-by: Andrew Murray <Andrew.Murray@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/of/address.c       |   63 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/of_address.h |    9 +++++++
 2 files changed, 72 insertions(+)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 0125524..d659527 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -13,6 +13,7 @@
 #define OF_CHECK_COUNTS(na, ns)	(OF_CHECK_ADDR_COUNT(na) && (ns) > 0)
 
 static struct of_bus *of_match_bus(struct device_node *np);
+static struct of_bus *of_find_bus(const char *name);
 static int __of_address_to_resource(struct device_node *dev,
 		const __be32 *addrp, u64 size, unsigned int flags,
 		const char *name, struct resource *r);
@@ -227,6 +228,57 @@ int of_pci_address_to_resource(struct device_node *dev, int bar,
 	return __of_address_to_resource(dev, addrp, size, flags, NULL, r);
 }
 EXPORT_SYMBOL_GPL(of_pci_address_to_resource);
+
+const __be32 *of_pci_process_ranges(struct device_node *node,
+				    struct resource *res, const __be32 *from)
+{
+	const __be32 *start, *end;
+	int na, ns, np, pna;
+	int rlen;
+	struct of_bus *bus;
+
+	WARN_ON(!res);
+
+	bus = of_find_bus("pci");
+	bus->count_cells(node, &na, &ns);
+	if (!OF_CHECK_COUNTS(na, ns)) {
+		pr_err("Bad cell count for %s\n", node->full_name);
+		return NULL;
+	}
+
+	pna = of_n_addr_cells(node);
+	np = pna + na + ns;
+
+	start = of_get_property(node, "ranges", &rlen);
+	if (start == NULL)
+		return NULL;
+
+	end = start + rlen / sizeof(__be32);
+
+	if (!from)
+		from = start;
+
+	while (from + np <= end) {
+		u64 cpu_addr, size;
+
+		cpu_addr = of_translate_address(node, from + na);
+		size = of_read_number(from + na + pna, ns);
+		res->flags = bus->get_flags(from);
+		from += np;
+
+		if (cpu_addr == OF_BAD_ADDR || size == 0)
+			continue;
+
+		res->name = node->full_name;
+		res->start = cpu_addr;
+		res->end = res->start + size - 1;
+		res->parent = res->child = res->sibling = NULL;
+		return from;
+	}
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(of_pci_process_ranges);
 #endif /* CONFIG_PCI */
 
 /*
@@ -337,6 +389,17 @@ static struct of_bus *of_match_bus(struct device_node *np)
 	return NULL;
 }
 
+static struct of_bus *of_find_bus(const char *name)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(of_busses); i++)
+		if (strcmp(name, of_busses[i].name) == 0)
+			return &of_busses[i];
+
+	return NULL;
+}
+
 static int of_translate_one(struct device_node *parent, struct of_bus *bus,
 			    struct of_bus *pbus, __be32 *addr,
 			    int na, int ns, int pna, const char *rprop)
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index 0506eb5..751e889 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -27,6 +27,8 @@ static inline unsigned long pci_address_to_pio(phys_addr_t addr) { return -1; }
 #define pci_address_to_pio pci_address_to_pio
 #endif
 
+const __be32 *of_pci_process_ranges(struct device_node *node,
+				    struct resource *res, const __be32 *from);
 #else /* CONFIG_OF_ADDRESS */
 #ifndef of_address_to_resource
 static inline int of_address_to_resource(struct device_node *dev, int index,
@@ -53,6 +55,13 @@ static inline const __be32 *of_get_address(struct device_node *dev, int index,
 {
 	return NULL;
 }
+
+static inline const __be32 *of_pci_process_ranges(struct device_node *node,
+						  struct resource *res,
+						  const __be32 *from)
+{
+	return NULL;
+}
 #endif /* CONFIG_OF_ADDRESS */
 
 
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 01/32] of/pci: Provide support for parsing PCI DT ranges property
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

From: Andrew Murray <andrew.murray@arm.com>

DT bindings for PCI host bridges often use the ranges property to describe
memory and IO ranges - this binding tends to be the same across architectures
yet several parsing implementations exist, e.g. arch/mips/pci/pci.c,
arch/powerpc/kernel/pci-common.c, arch/sparc/kernel/pci.c and
arch/microblaze/pci/pci-common.c (clone of PPC). Some of these duplicate
functionality provided by drivers/of/address.c.

This patch provides a common iterator-based parser for the ranges property, it
is hoped this will reduce DT representation differences between architectures
and that architectures will migrate in part to this new parser.

It is also hoped (and the motativation for the patch) that this patch will
reduce duplication of code when writing host bridge drivers that are supported
by multiple architectures.

This patch provides struct resources from a device tree node, e.g.:

	u32 *last = NULL;
	struct resource res;
	while ((last = of_pci_process_ranges(np, res, last))) {
		//do something with res
	}

Platforms with quirks can then do what they like with the resource or migrate
common quirk handling to the parser. In an ideal world drivers can just request
the obtained resources and pass them on (e.g. pci_add_resource_offset).

Signed-off-by: Andrew Murray <Andrew.Murray@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/of/address.c       |   63 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/of_address.h |    9 +++++++
 2 files changed, 72 insertions(+)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 0125524..d659527 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -13,6 +13,7 @@
 #define OF_CHECK_COUNTS(na, ns)	(OF_CHECK_ADDR_COUNT(na) && (ns) > 0)
 
 static struct of_bus *of_match_bus(struct device_node *np);
+static struct of_bus *of_find_bus(const char *name);
 static int __of_address_to_resource(struct device_node *dev,
 		const __be32 *addrp, u64 size, unsigned int flags,
 		const char *name, struct resource *r);
@@ -227,6 +228,57 @@ int of_pci_address_to_resource(struct device_node *dev, int bar,
 	return __of_address_to_resource(dev, addrp, size, flags, NULL, r);
 }
 EXPORT_SYMBOL_GPL(of_pci_address_to_resource);
+
+const __be32 *of_pci_process_ranges(struct device_node *node,
+				    struct resource *res, const __be32 *from)
+{
+	const __be32 *start, *end;
+	int na, ns, np, pna;
+	int rlen;
+	struct of_bus *bus;
+
+	WARN_ON(!res);
+
+	bus = of_find_bus("pci");
+	bus->count_cells(node, &na, &ns);
+	if (!OF_CHECK_COUNTS(na, ns)) {
+		pr_err("Bad cell count for %s\n", node->full_name);
+		return NULL;
+	}
+
+	pna = of_n_addr_cells(node);
+	np = pna + na + ns;
+
+	start = of_get_property(node, "ranges", &rlen);
+	if (start == NULL)
+		return NULL;
+
+	end = start + rlen / sizeof(__be32);
+
+	if (!from)
+		from = start;
+
+	while (from + np <= end) {
+		u64 cpu_addr, size;
+
+		cpu_addr = of_translate_address(node, from + na);
+		size = of_read_number(from + na + pna, ns);
+		res->flags = bus->get_flags(from);
+		from += np;
+
+		if (cpu_addr == OF_BAD_ADDR || size == 0)
+			continue;
+
+		res->name = node->full_name;
+		res->start = cpu_addr;
+		res->end = res->start + size - 1;
+		res->parent = res->child = res->sibling = NULL;
+		return from;
+	}
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(of_pci_process_ranges);
 #endif /* CONFIG_PCI */
 
 /*
@@ -337,6 +389,17 @@ static struct of_bus *of_match_bus(struct device_node *np)
 	return NULL;
 }
 
+static struct of_bus *of_find_bus(const char *name)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(of_busses); i++)
+		if (strcmp(name, of_busses[i].name) == 0)
+			return &of_busses[i];
+
+	return NULL;
+}
+
 static int of_translate_one(struct device_node *parent, struct of_bus *bus,
 			    struct of_bus *pbus, __be32 *addr,
 			    int na, int ns, int pna, const char *rprop)
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index 0506eb5..751e889 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -27,6 +27,8 @@ static inline unsigned long pci_address_to_pio(phys_addr_t addr) { return -1; }
 #define pci_address_to_pio pci_address_to_pio
 #endif
 
+const __be32 *of_pci_process_ranges(struct device_node *node,
+				    struct resource *res, const __be32 *from);
 #else /* CONFIG_OF_ADDRESS */
 #ifndef of_address_to_resource
 static inline int of_address_to_resource(struct device_node *dev, int index,
@@ -53,6 +55,13 @@ static inline const __be32 *of_get_address(struct device_node *dev, int index,
 {
 	return NULL;
 }
+
+static inline const __be32 *of_pci_process_ranges(struct device_node *node,
+						  struct resource *res,
+						  const __be32 *from)
+{
+	return NULL;
+}
 #endif /* CONFIG_OF_ADDRESS */
 
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 02/32] of/pci: Add of_pci_get_devfn() function
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

From: Thierry Reding <thierry.reding@avionic-design.de>

This function can be used to parse the device and function number from a
standard 5-cell PCI resource. PCI_SLOT() and PCI_FUNC() can be used on
the returned value obtain the device and function numbers respectively.

Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
v2 -> v2:
As suggested by Stephen Warren, rename the 'devfn' argument to 'data',
and use a local variable named 'devfn' instead of 'err', which was a
quite confusing name indeed.
---
 drivers/of/of_pci.c    |   34 +++++++++++++++++++++++++++++-----
 include/linux/of_pci.h |    1 +
 2 files changed, 30 insertions(+), 5 deletions(-)

diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
index 13e37e2..8c10757 100644
--- a/drivers/of/of_pci.c
+++ b/drivers/of/of_pci.c
@@ -5,14 +5,15 @@
 #include <asm/prom.h>
 
 static inline int __of_pci_pci_compare(struct device_node *node,
-				       unsigned int devfn)
+				       unsigned int data)
 {
-	unsigned int size;
-	const __be32 *reg = of_get_property(node, "reg", &size);
+	int devfn;
 
-	if (!reg || size < 5 * sizeof(__be32))
+	devfn = of_pci_get_devfn(node);
+	if (devfn < 0)
 		return 0;
-	return ((be32_to_cpup(&reg[0]) >> 8) & 0xff) == devfn;
+
+	return data == devfn;
 }
 
 struct device_node *of_pci_find_child_device(struct device_node *parent,
@@ -40,3 +41,26 @@ struct device_node *of_pci_find_child_device(struct device_node *parent,
 	return NULL;
 }
 EXPORT_SYMBOL_GPL(of_pci_find_child_device);
+
+/**
+ * of_pci_get_devfn() - Get device and function numbers for a device node
+ * @np: device node
+ *
+ * Parses a standard 5-cell PCI resource and returns an 8-bit value that can
+ * be passed to the PCI_SLOT() and PCI_FUNC() macros to extract the device
+ * and function numbers respectively. On error a negative error code is
+ * returned.
+ */
+int of_pci_get_devfn(struct device_node *np)
+{
+	unsigned int size;
+	const __be32 *reg;
+
+	reg = of_get_property(np, "reg", &size);
+
+	if (!reg || size < 5 * sizeof(__be32))
+		return -EINVAL;
+
+	return (be32_to_cpup(reg) >> 8) & 0xff;
+}
+EXPORT_SYMBOL_GPL(of_pci_get_devfn);
diff --git a/include/linux/of_pci.h b/include/linux/of_pci.h
index bb115de..91ec484 100644
--- a/include/linux/of_pci.h
+++ b/include/linux/of_pci.h
@@ -10,5 +10,6 @@ int of_irq_map_pci(const struct pci_dev *pdev, struct of_irq *out_irq);
 struct device_node;
 struct device_node *of_pci_find_child_device(struct device_node *parent,
 					     unsigned int devfn);
+int of_pci_get_devfn(struct device_node *np);
 
 #endif
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 02/32] of/pci: Add of_pci_get_devfn() function
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

From: Thierry Reding <thierry.reding@avionic-design.de>

This function can be used to parse the device and function number from a
standard 5-cell PCI resource. PCI_SLOT() and PCI_FUNC() can be used on
the returned value obtain the device and function numbers respectively.

Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
v2 -> v2:
As suggested by Stephen Warren, rename the 'devfn' argument to 'data',
and use a local variable named 'devfn' instead of 'err', which was a
quite confusing name indeed.
---
 drivers/of/of_pci.c    |   34 +++++++++++++++++++++++++++++-----
 include/linux/of_pci.h |    1 +
 2 files changed, 30 insertions(+), 5 deletions(-)

diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
index 13e37e2..8c10757 100644
--- a/drivers/of/of_pci.c
+++ b/drivers/of/of_pci.c
@@ -5,14 +5,15 @@
 #include <asm/prom.h>
 
 static inline int __of_pci_pci_compare(struct device_node *node,
-				       unsigned int devfn)
+				       unsigned int data)
 {
-	unsigned int size;
-	const __be32 *reg = of_get_property(node, "reg", &size);
+	int devfn;
 
-	if (!reg || size < 5 * sizeof(__be32))
+	devfn = of_pci_get_devfn(node);
+	if (devfn < 0)
 		return 0;
-	return ((be32_to_cpup(&reg[0]) >> 8) & 0xff) == devfn;
+
+	return data == devfn;
 }
 
 struct device_node *of_pci_find_child_device(struct device_node *parent,
@@ -40,3 +41,26 @@ struct device_node *of_pci_find_child_device(struct device_node *parent,
 	return NULL;
 }
 EXPORT_SYMBOL_GPL(of_pci_find_child_device);
+
+/**
+ * of_pci_get_devfn() - Get device and function numbers for a device node
+ * @np: device node
+ *
+ * Parses a standard 5-cell PCI resource and returns an 8-bit value that can
+ * be passed to the PCI_SLOT() and PCI_FUNC() macros to extract the device
+ * and function numbers respectively. On error a negative error code is
+ * returned.
+ */
+int of_pci_get_devfn(struct device_node *np)
+{
+	unsigned int size;
+	const __be32 *reg;
+
+	reg = of_get_property(np, "reg", &size);
+
+	if (!reg || size < 5 * sizeof(__be32))
+		return -EINVAL;
+
+	return (be32_to_cpup(reg) >> 8) & 0xff;
+}
+EXPORT_SYMBOL_GPL(of_pci_get_devfn);
diff --git a/include/linux/of_pci.h b/include/linux/of_pci.h
index bb115de..91ec484 100644
--- a/include/linux/of_pci.h
+++ b/include/linux/of_pci.h
@@ -10,5 +10,6 @@ int of_irq_map_pci(const struct pci_dev *pdev, struct of_irq *out_irq);
 struct device_node;
 struct device_node *of_pci_find_child_device(struct device_node *parent,
 					     unsigned int devfn);
+int of_pci_get_devfn(struct device_node *np);
 
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 03/32] of/pci: Add of_pci_parse_bus_range() function
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

From: Thierry Reding <thierry.reding@avionic-design.de>

This function can be used to parse a bus-range property as specified by
device nodes representing PCI bridges.

Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/of/of_pci.c    |   25 +++++++++++++++++++++++++
 include/linux/of_pci.h |    1 +
 2 files changed, 26 insertions(+)

diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
index 8c10757..a2c6f2c 100644
--- a/drivers/of/of_pci.c
+++ b/drivers/of/of_pci.c
@@ -64,3 +64,28 @@ int of_pci_get_devfn(struct device_node *np)
 	return (be32_to_cpup(reg) >> 8) & 0xff;
 }
 EXPORT_SYMBOL_GPL(of_pci_get_devfn);
+
+/**
+ * of_pci_parse_bus_range() - parse the bus-range property of a PCI device
+ * @node: device node
+ * @res: address to a struct resource to return the bus-range
+ *
+ * Returns 0 on success or a negative error-code on failure.
+ */
+int of_pci_parse_bus_range(struct device_node *node, struct resource *res)
+{
+	const __be32 *values;
+	int len;
+
+	values = of_get_property(node, "bus-range", &len);
+	if (!values || len < sizeof(*values) * 2)
+		return -EINVAL;
+
+	res->name = node->name;
+	res->start = be32_to_cpup(values++);
+	res->end = be32_to_cpup(values);
+	res->flags = IORESOURCE_BUS;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(of_pci_parse_bus_range);
diff --git a/include/linux/of_pci.h b/include/linux/of_pci.h
index 91ec484..7a04826 100644
--- a/include/linux/of_pci.h
+++ b/include/linux/of_pci.h
@@ -11,5 +11,6 @@ struct device_node;
 struct device_node *of_pci_find_child_device(struct device_node *parent,
 					     unsigned int devfn);
 int of_pci_get_devfn(struct device_node *np);
+int of_pci_parse_bus_range(struct device_node *node, struct resource *res);
 
 #endif
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 03/32] of/pci: Add of_pci_parse_bus_range() function
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

From: Thierry Reding <thierry.reding@avionic-design.de>

This function can be used to parse a bus-range property as specified by
device nodes representing PCI bridges.

Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/of/of_pci.c    |   25 +++++++++++++++++++++++++
 include/linux/of_pci.h |    1 +
 2 files changed, 26 insertions(+)

diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c
index 8c10757..a2c6f2c 100644
--- a/drivers/of/of_pci.c
+++ b/drivers/of/of_pci.c
@@ -64,3 +64,28 @@ int of_pci_get_devfn(struct device_node *np)
 	return (be32_to_cpup(reg) >> 8) & 0xff;
 }
 EXPORT_SYMBOL_GPL(of_pci_get_devfn);
+
+/**
+ * of_pci_parse_bus_range() - parse the bus-range property of a PCI device
+ * @node: device node
+ * @res: address to a struct resource to return the bus-range
+ *
+ * Returns 0 on success or a negative error-code on failure.
+ */
+int of_pci_parse_bus_range(struct device_node *node, struct resource *res)
+{
+	const __be32 *values;
+	int len;
+
+	values = of_get_property(node, "bus-range", &len);
+	if (!values || len < sizeof(*values) * 2)
+		return -EINVAL;
+
+	res->name = node->name;
+	res->start = be32_to_cpup(values++);
+	res->end = be32_to_cpup(values);
+	res->flags = IORESOURCE_BUS;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(of_pci_parse_bus_range);
diff --git a/include/linux/of_pci.h b/include/linux/of_pci.h
index 91ec484..7a04826 100644
--- a/include/linux/of_pci.h
+++ b/include/linux/of_pci.h
@@ -11,5 +11,6 @@ struct device_node;
 struct device_node *of_pci_find_child_device(struct device_node *parent,
 					     unsigned int devfn);
 int of_pci_get_devfn(struct device_node *np);
+int of_pci_parse_bus_range(struct device_node *node, struct resource *res);
 
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 04/32] ARM: pci: Allow passing per-controller private data
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

From: Thierry Reding <thierry.reding@avionic-design.de>

In order to allow drivers to specify private data for each controller,
this commit adds a private_data field to the struct hw_pci. This field
is an array of nr_controllers pointers that will be used to initialize
the private_data field of the corresponding controller's pci_sys_data
structure.

Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
---
 arch/arm/include/asm/mach/pci.h |    1 +
 arch/arm/kernel/bios32.c        |    3 +++
 2 files changed, 4 insertions(+)

diff --git a/arch/arm/include/asm/mach/pci.h b/arch/arm/include/asm/mach/pci.h
index db9fedb..5cf2e97 100644
--- a/arch/arm/include/asm/mach/pci.h
+++ b/arch/arm/include/asm/mach/pci.h
@@ -23,6 +23,7 @@ struct hw_pci {
 #endif
 	struct pci_ops	*ops;
 	int		nr_controllers;
+	void		**private_data;
 	int		(*setup)(int nr, struct pci_sys_data *);
 	struct pci_bus *(*scan)(int nr, struct pci_sys_data *);
 	void		(*preinit)(void);
diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index 379cf32..5401645 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -464,6 +464,9 @@ static void __init pcibios_init_hw(struct hw_pci *hw, struct list_head *head)
 		sys->map_irq = hw->map_irq;
 		INIT_LIST_HEAD(&sys->resources);
 
+		if (hw->private_data)
+			sys->private_data = hw->private_data[nr];
+
 		ret = hw->setup(nr, sys);
 
 		if (ret > 0) {
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 04/32] ARM: pci: Allow passing per-controller private data
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

From: Thierry Reding <thierry.reding@avionic-design.de>

In order to allow drivers to specify private data for each controller,
this commit adds a private_data field to the struct hw_pci. This field
is an array of nr_controllers pointers that will be used to initialize
the private_data field of the corresponding controller's pci_sys_data
structure.

Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
---
 arch/arm/include/asm/mach/pci.h |    1 +
 arch/arm/kernel/bios32.c        |    3 +++
 2 files changed, 4 insertions(+)

diff --git a/arch/arm/include/asm/mach/pci.h b/arch/arm/include/asm/mach/pci.h
index db9fedb..5cf2e97 100644
--- a/arch/arm/include/asm/mach/pci.h
+++ b/arch/arm/include/asm/mach/pci.h
@@ -23,6 +23,7 @@ struct hw_pci {
 #endif
 	struct pci_ops	*ops;
 	int		nr_controllers;
+	void		**private_data;
 	int		(*setup)(int nr, struct pci_sys_data *);
 	struct pci_bus *(*scan)(int nr, struct pci_sys_data *);
 	void		(*preinit)(void);
diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index 379cf32..5401645 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -464,6 +464,9 @@ static void __init pcibios_init_hw(struct hw_pci *hw, struct list_head *head)
 		sys->map_irq = hw->map_irq;
 		INIT_LIST_HEAD(&sys->resources);
 
+		if (hw->private_data)
+			sys->private_data = hw->private_data[nr];
+
 		ret = hw->setup(nr, sys);
 
 		if (ret > 0) {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk, Paul Gortmaker,
	Jesse Barnes, Yinghai Lu, linux-kernel

The pcim_*() functions are used by the libata-sff subsystem, and this
subsystem is used for many SATA drivers on ARM platforms that do not
necessarily have I/O ports.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: linux-kernel@vger.kernel.org
---
 lib/devres.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/devres.c b/lib/devres.c
index 80b9c76..5639c3e 100644
--- a/lib/devres.c
+++ b/lib/devres.c
@@ -195,6 +195,7 @@ void devm_ioport_unmap(struct device *dev, void __iomem *addr)
 			       devm_ioport_map_match, (void *)addr));
 }
 EXPORT_SYMBOL(devm_ioport_unmap);
+#endif /* CONFIG_HAS_IOPORT */
 
 #ifdef CONFIG_PCI
 /*
@@ -400,4 +401,3 @@ void pcim_iounmap_regions(struct pci_dev *pdev, int mask)
 }
 EXPORT_SYMBOL(pcim_iounmap_regions);
 #endif /* CONFIG_PCI */
-#endif /* CONFIG_HAS_IOPORT */
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

The pcim_*() functions are used by the libata-sff subsystem, and this
subsystem is used for many SATA drivers on ARM platforms that do not
necessarily have I/O ports.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: linux-kernel at vger.kernel.org
---
 lib/devres.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/devres.c b/lib/devres.c
index 80b9c76..5639c3e 100644
--- a/lib/devres.c
+++ b/lib/devres.c
@@ -195,6 +195,7 @@ void devm_ioport_unmap(struct device *dev, void __iomem *addr)
 			       devm_ioport_map_match, (void *)addr));
 }
 EXPORT_SYMBOL(devm_ioport_unmap);
+#endif /* CONFIG_HAS_IOPORT */
 
 #ifdef CONFIG_PCI
 /*
@@ -400,4 +401,3 @@ void pcim_iounmap_regions(struct pci_dev *pdev, int mask)
 }
 EXPORT_SYMBOL(pcim_iounmap_regions);
 #endif /* CONFIG_PCI */
-#endif /* CONFIG_HAS_IOPORT */
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 06/32] arm: pci: add a align_resource hook
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The PCI specifications says that an I/O region must be aligned on a 4
KB boundary, and a memory region aligned on a 1 MB boundary.

However, the Marvell PCIe interfaces rely on address decoding windows
(which allow to associate a range of physical addresses with a given
device), and those have special requirements compared to the standard
PCI-to-PCI bridge specifications.

PCIe memory windows must have a power of two size (which matches the
PCI spec), be at least of 1 MB (which matches the PCI spec), and be
aligned on their size (which does not match the PCI spec, that
requires only a 1 MB alignment).

PCIe I/O windows must have a power of two size (matches the PCI spec),
be at least of 64 KB (doesn't match the PCI spec, which requires only
a size of 4 KB), and be aligned on their size (doesn't match the PCI
spec).

The PCI core already calls pcibios_align_resource() in the ARM PCI
core, specifically for such purposes. So this patch extends the ARM
PCI core so that it calls a ->align_resource() hook registered by the
PCI driver, exactly like the existing ->map_irq() and ->swizzle()
hooks.

A particular PCI driver can register a align_resource() hook, and do
its own specific alignement, depending on the specific constraints of
the underlying hardware.

Following the earlier reviews of this patch, a few comments:

 * Using numerical values is not possible since the alignment value is
   not absolute, but depends on the size of the memory window or I/O
   window. We therefore really need a function hook.

 * Using window_alignment() doesn't work, because it doesn't allow to
   adjust the starting address of the memory or I/O window, and we
   don't have access to the size. The comment was that
   ->resource_align() would apply to all resources on all busses, but
   in the new implementation of the Marvell-specific
   ->resource_align() hook we are careful to only apply the alignment
   constraints on the specific bus that hosts the emulated PCI-to-PCI
   bridges needed to operate the Marvell PCIe interfaces.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Russell King <linux@arm.linux.org.uk>
---
 arch/arm/include/asm/mach/pci.h |   11 +++++++++++
 arch/arm/kernel/bios32.c        |    6 ++++++
 2 files changed, 17 insertions(+)

diff --git a/arch/arm/include/asm/mach/pci.h b/arch/arm/include/asm/mach/pci.h
index 5cf2e97..7d2c3c8 100644
--- a/arch/arm/include/asm/mach/pci.h
+++ b/arch/arm/include/asm/mach/pci.h
@@ -30,6 +30,11 @@ struct hw_pci {
 	void		(*postinit)(void);
 	u8		(*swizzle)(struct pci_dev *dev, u8 *pin);
 	int		(*map_irq)(const struct pci_dev *dev, u8 slot, u8 pin);
+	resource_size_t (*align_resource)(struct pci_dev *dev,
+					  const struct resource *res,
+					  resource_size_t start,
+					  resource_size_t size,
+					  resource_size_t align);
 };
 
 /*
@@ -51,6 +56,12 @@ struct pci_sys_data {
 	u8		(*swizzle)(struct pci_dev *, u8 *);
 					/* IRQ mapping				*/
 	int		(*map_irq)(const struct pci_dev *, u8, u8);
+					/* Resource alignement requirements	*/
+	resource_size_t (*align_resource)(struct pci_dev *dev,
+					  const struct resource *res,
+					  resource_size_t start,
+					  resource_size_t size,
+					  resource_size_t align);
 	void		*private_data;	/* platform controller private data	*/
 };
 
diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index 5401645..be2e6c9 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -462,6 +462,7 @@ static void __init pcibios_init_hw(struct hw_pci *hw, struct list_head *head)
 		sys->busnr   = busnr;
 		sys->swizzle = hw->swizzle;
 		sys->map_irq = hw->map_irq;
+		sys->align_resource = hw->align_resource;
 		INIT_LIST_HEAD(&sys->resources);
 
 		if (hw->private_data)
@@ -574,6 +575,8 @@ char * __init pcibios_setup(char *str)
 resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 				resource_size_t size, resource_size_t align)
 {
+	struct pci_dev *dev = data;
+	struct pci_sys_data *sys = dev->sysdata;
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO && start & 0x300)
@@ -581,6 +584,9 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 
 	start = (start + align - 1) & ~(align - 1);
 
+	if (sys->align_resource)
+		return sys->align_resource(dev, res, start, size, align);
+
 	return start;
 }
 
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 06/32] arm: pci: add a align_resource hook
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

The PCI specifications says that an I/O region must be aligned on a 4
KB boundary, and a memory region aligned on a 1 MB boundary.

However, the Marvell PCIe interfaces rely on address decoding windows
(which allow to associate a range of physical addresses with a given
device), and those have special requirements compared to the standard
PCI-to-PCI bridge specifications.

PCIe memory windows must have a power of two size (which matches the
PCI spec), be at least of 1 MB (which matches the PCI spec), and be
aligned on their size (which does not match the PCI spec, that
requires only a 1 MB alignment).

PCIe I/O windows must have a power of two size (matches the PCI spec),
be at least of 64 KB (doesn't match the PCI spec, which requires only
a size of 4 KB), and be aligned on their size (doesn't match the PCI
spec).

The PCI core already calls pcibios_align_resource() in the ARM PCI
core, specifically for such purposes. So this patch extends the ARM
PCI core so that it calls a ->align_resource() hook registered by the
PCI driver, exactly like the existing ->map_irq() and ->swizzle()
hooks.

A particular PCI driver can register a align_resource() hook, and do
its own specific alignement, depending on the specific constraints of
the underlying hardware.

Following the earlier reviews of this patch, a few comments:

 * Using numerical values is not possible since the alignment value is
   not absolute, but depends on the size of the memory window or I/O
   window. We therefore really need a function hook.

 * Using window_alignment() doesn't work, because it doesn't allow to
   adjust the starting address of the memory or I/O window, and we
   don't have access to the size. The comment was that
   ->resource_align() would apply to all resources on all busses, but
   in the new implementation of the Marvell-specific
   ->resource_align() hook we are careful to only apply the alignment
   constraints on the specific bus that hosts the emulated PCI-to-PCI
   bridges needed to operate the Marvell PCIe interfaces.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Russell King <linux@arm.linux.org.uk>
---
 arch/arm/include/asm/mach/pci.h |   11 +++++++++++
 arch/arm/kernel/bios32.c        |    6 ++++++
 2 files changed, 17 insertions(+)

diff --git a/arch/arm/include/asm/mach/pci.h b/arch/arm/include/asm/mach/pci.h
index 5cf2e97..7d2c3c8 100644
--- a/arch/arm/include/asm/mach/pci.h
+++ b/arch/arm/include/asm/mach/pci.h
@@ -30,6 +30,11 @@ struct hw_pci {
 	void		(*postinit)(void);
 	u8		(*swizzle)(struct pci_dev *dev, u8 *pin);
 	int		(*map_irq)(const struct pci_dev *dev, u8 slot, u8 pin);
+	resource_size_t (*align_resource)(struct pci_dev *dev,
+					  const struct resource *res,
+					  resource_size_t start,
+					  resource_size_t size,
+					  resource_size_t align);
 };
 
 /*
@@ -51,6 +56,12 @@ struct pci_sys_data {
 	u8		(*swizzle)(struct pci_dev *, u8 *);
 					/* IRQ mapping				*/
 	int		(*map_irq)(const struct pci_dev *, u8, u8);
+					/* Resource alignement requirements	*/
+	resource_size_t (*align_resource)(struct pci_dev *dev,
+					  const struct resource *res,
+					  resource_size_t start,
+					  resource_size_t size,
+					  resource_size_t align);
 	void		*private_data;	/* platform controller private data	*/
 };
 
diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index 5401645..be2e6c9 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -462,6 +462,7 @@ static void __init pcibios_init_hw(struct hw_pci *hw, struct list_head *head)
 		sys->busnr   = busnr;
 		sys->swizzle = hw->swizzle;
 		sys->map_irq = hw->map_irq;
+		sys->align_resource = hw->align_resource;
 		INIT_LIST_HEAD(&sys->resources);
 
 		if (hw->private_data)
@@ -574,6 +575,8 @@ char * __init pcibios_setup(char *str)
 resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 				resource_size_t size, resource_size_t align)
 {
+	struct pci_dev *dev = data;
+	struct pci_sys_data *sys = dev->sysdata;
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO && start & 0x300)
@@ -581,6 +584,9 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 
 	start = (start + align - 1) & ~(align - 1);
 
+	if (sys->align_resource)
+		return sys->align_resource(dev, res, start, size, align);
+
 	return start;
 }
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 07/32] arm: mvebu: fix address-cells in mpic DT node
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

There is no need to have a #address-cells property in the MPIC Device
Tree node, and more than that, having it confuses the of_irq_map_raw()
logic, which will be used by the Marvell PCIe driver.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-370-xp.dtsi |    1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/arm/boot/dts/armada-370-xp.dtsi b/arch/arm/boot/dts/armada-370-xp.dtsi
index 4c0abe8..1dcdae9 100644
--- a/arch/arm/boot/dts/armada-370-xp.dtsi
+++ b/arch/arm/boot/dts/armada-370-xp.dtsi
@@ -31,7 +31,6 @@
 	mpic: interrupt-controller@d0020000 {
 	      compatible = "marvell,mpic";
 	      #interrupt-cells = <1>;
-	      #address-cells = <1>;
 	      #size-cells = <1>;
 	      interrupt-controller;
 	};
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 07/32] arm: mvebu: fix address-cells in mpic DT node
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

There is no need to have a #address-cells property in the MPIC Device
Tree node, and more than that, having it confuses the of_irq_map_raw()
logic, which will be used by the Marvell PCIe driver.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-370-xp.dtsi |    1 -
 1 file changed, 1 deletion(-)

diff --git a/arch/arm/boot/dts/armada-370-xp.dtsi b/arch/arm/boot/dts/armada-370-xp.dtsi
index 4c0abe8..1dcdae9 100644
--- a/arch/arm/boot/dts/armada-370-xp.dtsi
+++ b/arch/arm/boot/dts/armada-370-xp.dtsi
@@ -31,7 +31,6 @@
 	mpic: interrupt-controller at d0020000 {
 	      compatible = "marvell,mpic";
 	      #interrupt-cells = <1>;
-	      #address-cells = <1>;
 	      #size-cells = <1>;
 	      interrupt-controller;
 	};
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 08/32] clk: mvebu: create parent-child relation for PCIe clocks on Armada 370
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk, Mike Turquette

The Armada 370 has two gatable clocks for each PCIe interface: pexY
and pexY_en. In order for a PCIe interface to be functional, we need
both of them to be enabled. Since there are no conditions in which
having one clock enabled and not the other would be useful, we simply
make those two clocks have a child-parent relationship.

So, the pex0 clock becomes a child of pex0_en (for the PCIe interface
0), and pex1 becomes a child of pex1_en (for the PCIe interface
1). This way, the DT node for the PCIe interface 0 only needs to grab
a reference to pex0, and the DT node for the PCIe interface 1 only
needs to grab a reference to pex1.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Mike Turquette <mturquette@linaro.org>
---
 drivers/clk/mvebu/clk-gating-ctrl.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/clk/mvebu/clk-gating-ctrl.c b/drivers/clk/mvebu/clk-gating-ctrl.c
index 8fa5408..fd52b5f 100644
--- a/drivers/clk/mvebu/clk-gating-ctrl.c
+++ b/drivers/clk/mvebu/clk-gating-ctrl.c
@@ -119,8 +119,8 @@ static const struct mvebu_soc_descr __initconst armada_370_gating_descr[] = {
 	{ "pex1_en", NULL,  2 },
 	{ "ge1", NULL, 3 },
 	{ "ge0", NULL, 4 },
-	{ "pex0", NULL, 5 },
-	{ "pex1", NULL, 9 },
+	{ "pex0", "pex0_en", 5 },
+	{ "pex1", "pex1_en", 9 },
 	{ "sata0", NULL, 15 },
 	{ "sdio", NULL, 17 },
 	{ "tdm", NULL, 25 },
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 08/32] clk: mvebu: create parent-child relation for PCIe clocks on Armada 370
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

The Armada 370 has two gatable clocks for each PCIe interface: pexY
and pexY_en. In order for a PCIe interface to be functional, we need
both of them to be enabled. Since there are no conditions in which
having one clock enabled and not the other would be useful, we simply
make those two clocks have a child-parent relationship.

So, the pex0 clock becomes a child of pex0_en (for the PCIe interface
0), and pex1 becomes a child of pex1_en (for the PCIe interface
1). This way, the DT node for the PCIe interface 0 only needs to grab
a reference to pex0, and the DT node for the PCIe interface 1 only
needs to grab a reference to pex1.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Mike Turquette <mturquette@linaro.org>
---
 drivers/clk/mvebu/clk-gating-ctrl.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/clk/mvebu/clk-gating-ctrl.c b/drivers/clk/mvebu/clk-gating-ctrl.c
index 8fa5408..fd52b5f 100644
--- a/drivers/clk/mvebu/clk-gating-ctrl.c
+++ b/drivers/clk/mvebu/clk-gating-ctrl.c
@@ -119,8 +119,8 @@ static const struct mvebu_soc_descr __initconst armada_370_gating_descr[] = {
 	{ "pex1_en", NULL,  2 },
 	{ "ge1", NULL, 3 },
 	{ "ge0", NULL, 4 },
-	{ "pex0", NULL, 5 },
-	{ "pex1", NULL, 9 },
+	{ "pex0", "pex0_en", 5 },
+	{ "pex1", "pex1_en", 9 },
 	{ "sata0", NULL, 15 },
 	{ "sdio", NULL, 17 },
 	{ "tdm", NULL, 25 },
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 09/32] clk: mvebu: add more PCIe clocks for Armada XP
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The current revision of the datasheet only mentions the gatable clocks
for the PCIe 0.0, 0.1, 0.2 and 0.3 interfaces, and forgot to mention
the ones for the PCIe 1.0, 1.1, 1.2, 1.3, 2.0 and 3.0
interfaces. After confirmation with Marvell engineers, this patch adds
the missing gatable clocks for those PCIe interfaces.

It also changes the name of the previously existing PCIe gatable
clocks, in order to match the naming using the datasheets.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/clk/mvebu/clk-gating-ctrl.c |   14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/clk/mvebu/clk-gating-ctrl.c b/drivers/clk/mvebu/clk-gating-ctrl.c
index fd52b5f..24137f8 100644
--- a/drivers/clk/mvebu/clk-gating-ctrl.c
+++ b/drivers/clk/mvebu/clk-gating-ctrl.c
@@ -137,10 +137,14 @@ static const struct mvebu_soc_descr __initconst armada_xp_gating_descr[] = {
 	{ "ge2", NULL,  2 },
 	{ "ge1", NULL, 3 },
 	{ "ge0", NULL, 4 },
-	{ "pex0", NULL, 5 },
-	{ "pex1", NULL, 6 },
-	{ "pex2", NULL, 7 },
-	{ "pex3", NULL, 8 },
+	{ "pex00", NULL, 5 },
+	{ "pex01", NULL, 6 },
+	{ "pex02", NULL, 7 },
+	{ "pex03", NULL, 8 },
+	{ "pex10", NULL, 9 },
+	{ "pex11", NULL, 10 },
+	{ "pex12", NULL, 11 },
+	{ "pex13", NULL, 12 },
 	{ "bp", NULL, 13 },
 	{ "sata0lnk", NULL, 14 },
 	{ "sata0", "sata0lnk", 15 },
@@ -152,6 +156,8 @@ static const struct mvebu_soc_descr __initconst armada_xp_gating_descr[] = {
 	{ "xor0", NULL, 22 },
 	{ "crypto", NULL, 23 },
 	{ "tdm", NULL, 25 },
+	{ "pex20", NULL, 26 },
+	{ "pex30", NULL, 27 },
 	{ "xor1", NULL, 28 },
 	{ "sata1lnk", NULL, 29 },
 	{ "sata1", "sata1lnk", 30 },
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 09/32] clk: mvebu: add more PCIe clocks for Armada XP
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

The current revision of the datasheet only mentions the gatable clocks
for the PCIe 0.0, 0.1, 0.2 and 0.3 interfaces, and forgot to mention
the ones for the PCIe 1.0, 1.1, 1.2, 1.3, 2.0 and 3.0
interfaces. After confirmation with Marvell engineers, this patch adds
the missing gatable clocks for those PCIe interfaces.

It also changes the name of the previously existing PCIe gatable
clocks, in order to match the naming using the datasheets.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/clk/mvebu/clk-gating-ctrl.c |   14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/clk/mvebu/clk-gating-ctrl.c b/drivers/clk/mvebu/clk-gating-ctrl.c
index fd52b5f..24137f8 100644
--- a/drivers/clk/mvebu/clk-gating-ctrl.c
+++ b/drivers/clk/mvebu/clk-gating-ctrl.c
@@ -137,10 +137,14 @@ static const struct mvebu_soc_descr __initconst armada_xp_gating_descr[] = {
 	{ "ge2", NULL,  2 },
 	{ "ge1", NULL, 3 },
 	{ "ge0", NULL, 4 },
-	{ "pex0", NULL, 5 },
-	{ "pex1", NULL, 6 },
-	{ "pex2", NULL, 7 },
-	{ "pex3", NULL, 8 },
+	{ "pex00", NULL, 5 },
+	{ "pex01", NULL, 6 },
+	{ "pex02", NULL, 7 },
+	{ "pex03", NULL, 8 },
+	{ "pex10", NULL, 9 },
+	{ "pex11", NULL, 10 },
+	{ "pex12", NULL, 11 },
+	{ "pex13", NULL, 12 },
 	{ "bp", NULL, 13 },
 	{ "sata0lnk", NULL, 14 },
 	{ "sata0", "sata0lnk", 15 },
@@ -152,6 +156,8 @@ static const struct mvebu_soc_descr __initconst armada_xp_gating_descr[] = {
 	{ "xor0", NULL, 22 },
 	{ "crypto", NULL, 23 },
 	{ "tdm", NULL, 25 },
+	{ "pex20", NULL, 26 },
+	{ "pex30", NULL, 27 },
 	{ "xor1", NULL, 28 },
 	{ "sata1lnk", NULL, 29 },
 	{ "sata1", "sata1lnk", 30 },
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 10/32] arm: plat-orion: introduce WIN_CTRL_ENABLE in address mapping code
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Instead of hardcoding "1" as being the bit value to enable an address
decoding window, introduce and use a WIN_CTRL_ENABLE definition.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index febe386..4dec3db 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -38,6 +38,7 @@ EXPORT_SYMBOL_GPL(mv_mbus_dram_info);
  * CPU Address Decode Windows registers
  */
 #define WIN_CTRL_OFF		0x0000
+#define   WIN_CTRL_ENABLE       BIT(0)
 #define WIN_BASE_OFF		0x0004
 #define WIN_REMAP_LO_OFF	0x0008
 #define WIN_REMAP_HI_OFF	0x000c
@@ -79,7 +80,8 @@ void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 	}
 
 	base_high = base & 0xffff0000;
-	ctrl = ((size - 1) & 0xffff0000) | (attr << 8) | (target << 4) | 1;
+	ctrl = ((size - 1) & 0xffff0000) | (attr << 8) | (target << 4) |
+		WIN_CTRL_ENABLE;
 
 	writel(base_high, addr + WIN_BASE_OFF);
 	writel(ctrl, addr + WIN_CTRL_OFF);
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 10/32] arm: plat-orion: introduce WIN_CTRL_ENABLE in address mapping code
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Instead of hardcoding "1" as being the bit value to enable an address
decoding window, introduce and use a WIN_CTRL_ENABLE definition.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index febe386..4dec3db 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -38,6 +38,7 @@ EXPORT_SYMBOL_GPL(mv_mbus_dram_info);
  * CPU Address Decode Windows registers
  */
 #define WIN_CTRL_OFF		0x0000
+#define   WIN_CTRL_ENABLE       BIT(0)
 #define WIN_BASE_OFF		0x0004
 #define WIN_REMAP_LO_OFF	0x0008
 #define WIN_REMAP_HI_OFF	0x000c
@@ -79,7 +80,8 @@ void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 	}
 
 	base_high = base & 0xffff0000;
-	ctrl = ((size - 1) & 0xffff0000) | (attr << 8) | (target << 4) | 1;
+	ctrl = ((size - 1) & 0xffff0000) | (attr << 8) | (target << 4) |
+		WIN_CTRL_ENABLE;
 
 	writel(base_high, addr + WIN_BASE_OFF);
 	writel(ctrl, addr + WIN_CTRL_OFF);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 11/32] arm: plat-orion: refactor the orion_disable_wins() function
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

In the address decoding code, the orion_disable_wins() function is
used at boot time to disable all address decoding windows, before
configuring only the ones that are needed. This allows to make sure
that no configuration is left from the bootloader.

As a preparation for the introduction of address decoding window
allocation/deallocation function, we refactor this function into an
orion_disable_cpu_win() which disables a single window.

The orion_config_wins() function is changed to call
orion_disable_cpu_win() in a loop, to preserve an identical behavior.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c |   35 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 18 deletions(-)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index 4dec3db..dd98638 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -95,6 +95,19 @@ void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 	}
 }
 
+static void __init orion_disable_cpu_win(const struct orion_addr_map_cfg *cfg,
+					 const int win)
+{
+	void __iomem *addr = cfg->win_cfg_base(cfg, win);
+
+	writel(0, addr + WIN_BASE_OFF);
+	writel(0, addr + WIN_CTRL_OFF);
+	if (cfg->cpu_win_can_remap(cfg, win)) {
+		writel(0, addr + WIN_REMAP_LO_OFF);
+		writel(0, addr + WIN_REMAP_HI_OFF);
+	}
+}
+
 /*
  * Configure a number of windows.
  */
@@ -108,36 +121,22 @@ static void __init orion_setup_cpu_wins(const struct orion_addr_map_cfg * cfg,
 	}
 }
 
-static void __init orion_disable_wins(const struct orion_addr_map_cfg * cfg)
-{
-	void __iomem *addr;
-	int i;
-
-	for (i = 0; i < cfg->num_wins; i++) {
-		addr = cfg->win_cfg_base(cfg, i);
-
-		writel(0, addr + WIN_BASE_OFF);
-		writel(0, addr + WIN_CTRL_OFF);
-		if (cfg->cpu_win_can_remap(cfg, i)) {
-			writel(0, addr + WIN_REMAP_LO_OFF);
-			writel(0, addr + WIN_REMAP_HI_OFF);
-		}
-	}
-}
-
 /*
  * Disable, clear and configure windows.
  */
 void __init orion_config_wins(struct orion_addr_map_cfg * cfg,
 			      const struct orion_addr_map_info *info)
 {
+	int win;
+
 	if (!cfg->cpu_win_can_remap)
 		cfg->cpu_win_can_remap = orion_cpu_win_can_remap;
 
 	if (!cfg->win_cfg_base)
 		cfg->win_cfg_base = orion_win_cfg_base;
 
-	orion_disable_wins(cfg);
+	for (win = 0; win < cfg->num_wins; win++)
+		orion_disable_cpu_win(cfg, win);
 
 	if (info)
 		orion_setup_cpu_wins(cfg, info);
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 11/32] arm: plat-orion: refactor the orion_disable_wins() function
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

In the address decoding code, the orion_disable_wins() function is
used at boot time to disable all address decoding windows, before
configuring only the ones that are needed. This allows to make sure
that no configuration is left from the bootloader.

As a preparation for the introduction of address decoding window
allocation/deallocation function, we refactor this function into an
orion_disable_cpu_win() which disables a single window.

The orion_config_wins() function is changed to call
orion_disable_cpu_win() in a loop, to preserve an identical behavior.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c |   35 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 18 deletions(-)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index 4dec3db..dd98638 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -95,6 +95,19 @@ void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 	}
 }
 
+static void __init orion_disable_cpu_win(const struct orion_addr_map_cfg *cfg,
+					 const int win)
+{
+	void __iomem *addr = cfg->win_cfg_base(cfg, win);
+
+	writel(0, addr + WIN_BASE_OFF);
+	writel(0, addr + WIN_CTRL_OFF);
+	if (cfg->cpu_win_can_remap(cfg, win)) {
+		writel(0, addr + WIN_REMAP_LO_OFF);
+		writel(0, addr + WIN_REMAP_HI_OFF);
+	}
+}
+
 /*
  * Configure a number of windows.
  */
@@ -108,36 +121,22 @@ static void __init orion_setup_cpu_wins(const struct orion_addr_map_cfg * cfg,
 	}
 }
 
-static void __init orion_disable_wins(const struct orion_addr_map_cfg * cfg)
-{
-	void __iomem *addr;
-	int i;
-
-	for (i = 0; i < cfg->num_wins; i++) {
-		addr = cfg->win_cfg_base(cfg, i);
-
-		writel(0, addr + WIN_BASE_OFF);
-		writel(0, addr + WIN_CTRL_OFF);
-		if (cfg->cpu_win_can_remap(cfg, i)) {
-			writel(0, addr + WIN_REMAP_LO_OFF);
-			writel(0, addr + WIN_REMAP_HI_OFF);
-		}
-	}
-}
-
 /*
  * Disable, clear and configure windows.
  */
 void __init orion_config_wins(struct orion_addr_map_cfg * cfg,
 			      const struct orion_addr_map_info *info)
 {
+	int win;
+
 	if (!cfg->cpu_win_can_remap)
 		cfg->cpu_win_can_remap = orion_cpu_win_can_remap;
 
 	if (!cfg->win_cfg_base)
 		cfg->win_cfg_base = orion_win_cfg_base;
 
-	orion_disable_wins(cfg);
+	for (win = 0; win < cfg->num_wins; win++)
+		orion_disable_cpu_win(cfg, win);
 
 	if (info)
 		orion_setup_cpu_wins(cfg, info);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 12/32] plat-orion: introduce ORION_ADDR_MAP_NO_REMAP
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

As a first step to fix this, we introduce ORION_ADDR_MAP_NO_REMAP
define, that will be the "remap" value that SoC-specific code should
pass to not have a remapping.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/include/plat/addr-map.h |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/plat-orion/include/plat/addr-map.h b/arch/arm/plat-orion/include/plat/addr-map.h
index b76c065..0861b8c 100644
--- a/arch/arm/plat-orion/include/plat/addr-map.h
+++ b/arch/arm/plat-orion/include/plat/addr-map.h
@@ -41,6 +41,8 @@ struct orion_addr_map_info {
 	const int remap;
 };
 
+#define ORION_ADDR_MAP_NO_REMAP (-1)
+
 void __init orion_config_wins(struct orion_addr_map_cfg *cfg,
 			      const struct orion_addr_map_info *info);
 
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 12/32] plat-orion: introduce ORION_ADDR_MAP_NO_REMAP
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

As a first step to fix this, we introduce ORION_ADDR_MAP_NO_REMAP
define, that will be the "remap" value that SoC-specific code should
pass to not have a remapping.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/include/plat/addr-map.h |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/plat-orion/include/plat/addr-map.h b/arch/arm/plat-orion/include/plat/addr-map.h
index b76c065..0861b8c 100644
--- a/arch/arm/plat-orion/include/plat/addr-map.h
+++ b/arch/arm/plat-orion/include/plat/addr-map.h
@@ -41,6 +41,8 @@ struct orion_addr_map_info {
 	const int remap;
 };
 
+#define ORION_ADDR_MAP_NO_REMAP (-1)
+
 void __init orion_config_wins(struct orion_addr_map_cfg *cfg,
 			      const struct orion_addr_map_info *info);
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 13/32] arm: mach-dove: use ORION_ADDR_MAP_NO_REMAP
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

In this commit, we convert mach-dove addr_map_info definition to use
ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-dove/addr-map.c |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/arm/mach-dove/addr-map.c b/arch/arm/mach-dove/addr-map.c
index 2a06c01..febb4dc 100644
--- a/arch/arm/mach-dove/addr-map.c
+++ b/arch/arm/mach-dove/addr-map.c
@@ -61,28 +61,28 @@ static const struct __initdata orion_addr_map_info addr_map_info[] = {
 	  TARGET_PCIE1, ATTR_PCIE_IO, DOVE_PCIE1_IO_BUS_BASE
 	},
 	{ 2, DOVE_PCIE0_MEM_PHYS_BASE, DOVE_PCIE0_MEM_SIZE,
-	  TARGET_PCIE0, ATTR_PCIE_MEM, -1
+	  TARGET_PCIE0, ATTR_PCIE_MEM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	{ 3, DOVE_PCIE1_MEM_PHYS_BASE, DOVE_PCIE1_MEM_SIZE,
-	  TARGET_PCIE1, ATTR_PCIE_MEM, -1
+	  TARGET_PCIE1, ATTR_PCIE_MEM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/*
 	 * Window for CESA engine.
 	 */
 	{ 4, DOVE_CESA_PHYS_BASE, DOVE_CESA_SIZE,
-	  TARGET_CESA, ATTR_CESA, -1
+	  TARGET_CESA, ATTR_CESA, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/*
 	 * Window to the BootROM for Standby and Sleep Resume
 	 */
 	{ 5, DOVE_BOOTROM_PHYS_BASE, DOVE_BOOTROM_SIZE,
-	  TARGET_BOOTROM, ATTR_BOOTROM, -1
+	  TARGET_BOOTROM, ATTR_BOOTROM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/*
 	 * Window to the PMU Scratch Pad space
 	 */
 	{ 6, DOVE_SCRATCHPAD_PHYS_BASE, DOVE_SCRATCHPAD_SIZE,
-	  TARGET_SCRATCHPAD, ATTR_SCRATCHPAD, -1
+	  TARGET_SCRATCHPAD, ATTR_SCRATCHPAD, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/* End marker */
 	{ -1, 0, 0, 0, 0, 0 }
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 13/32] arm: mach-dove: use ORION_ADDR_MAP_NO_REMAP
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

In this commit, we convert mach-dove addr_map_info definition to use
ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-dove/addr-map.c |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/arm/mach-dove/addr-map.c b/arch/arm/mach-dove/addr-map.c
index 2a06c01..febb4dc 100644
--- a/arch/arm/mach-dove/addr-map.c
+++ b/arch/arm/mach-dove/addr-map.c
@@ -61,28 +61,28 @@ static const struct __initdata orion_addr_map_info addr_map_info[] = {
 	  TARGET_PCIE1, ATTR_PCIE_IO, DOVE_PCIE1_IO_BUS_BASE
 	},
 	{ 2, DOVE_PCIE0_MEM_PHYS_BASE, DOVE_PCIE0_MEM_SIZE,
-	  TARGET_PCIE0, ATTR_PCIE_MEM, -1
+	  TARGET_PCIE0, ATTR_PCIE_MEM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	{ 3, DOVE_PCIE1_MEM_PHYS_BASE, DOVE_PCIE1_MEM_SIZE,
-	  TARGET_PCIE1, ATTR_PCIE_MEM, -1
+	  TARGET_PCIE1, ATTR_PCIE_MEM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/*
 	 * Window for CESA engine.
 	 */
 	{ 4, DOVE_CESA_PHYS_BASE, DOVE_CESA_SIZE,
-	  TARGET_CESA, ATTR_CESA, -1
+	  TARGET_CESA, ATTR_CESA, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/*
 	 * Window to the BootROM for Standby and Sleep Resume
 	 */
 	{ 5, DOVE_BOOTROM_PHYS_BASE, DOVE_BOOTROM_SIZE,
-	  TARGET_BOOTROM, ATTR_BOOTROM, -1
+	  TARGET_BOOTROM, ATTR_BOOTROM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/*
 	 * Window to the PMU Scratch Pad space
 	 */
 	{ 6, DOVE_SCRATCHPAD_PHYS_BASE, DOVE_SCRATCHPAD_SIZE,
-	  TARGET_SCRATCHPAD, ATTR_SCRATCHPAD, -1
+	  TARGET_SCRATCHPAD, ATTR_SCRATCHPAD, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/* End marker */
 	{ -1, 0, 0, 0, 0, 0 }
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 14/32] arm: mach-kirkwood: use ORION_ADDR_MAP_NO_REMAP
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

In this commit, we convert mach-kirkwood addr_map_info definition to
use ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-kirkwood/addr-map.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/mach-kirkwood/addr-map.c b/arch/arm/mach-kirkwood/addr-map.c
index 8f0d162..dc3a6d9 100644
--- a/arch/arm/mach-kirkwood/addr-map.c
+++ b/arch/arm/mach-kirkwood/addr-map.c
@@ -64,13 +64,13 @@ static const struct __initdata orion_addr_map_info addr_map_info[] = {
 	 * Window for NAND controller.
 	 */
 	{ 4, KIRKWOOD_NAND_MEM_PHYS_BASE, KIRKWOOD_NAND_MEM_SIZE,
-	  TARGET_DEV_BUS, ATTR_DEV_NAND, -1
+	  TARGET_DEV_BUS, ATTR_DEV_NAND, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/*
 	 * Window for SRAM.
 	 */
 	{ 5, KIRKWOOD_SRAM_PHYS_BASE, KIRKWOOD_SRAM_SIZE,
-	  TARGET_SRAM, ATTR_SRAM, -1
+	  TARGET_SRAM, ATTR_SRAM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/* End marker */
 	{ -1, 0, 0, 0, 0, 0 }
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 14/32] arm: mach-kirkwood: use ORION_ADDR_MAP_NO_REMAP
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

In this commit, we convert mach-kirkwood addr_map_info definition to
use ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-kirkwood/addr-map.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/mach-kirkwood/addr-map.c b/arch/arm/mach-kirkwood/addr-map.c
index 8f0d162..dc3a6d9 100644
--- a/arch/arm/mach-kirkwood/addr-map.c
+++ b/arch/arm/mach-kirkwood/addr-map.c
@@ -64,13 +64,13 @@ static const struct __initdata orion_addr_map_info addr_map_info[] = {
 	 * Window for NAND controller.
 	 */
 	{ 4, KIRKWOOD_NAND_MEM_PHYS_BASE, KIRKWOOD_NAND_MEM_SIZE,
-	  TARGET_DEV_BUS, ATTR_DEV_NAND, -1
+	  TARGET_DEV_BUS, ATTR_DEV_NAND, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/*
 	 * Window for SRAM.
 	 */
 	{ 5, KIRKWOOD_SRAM_PHYS_BASE, KIRKWOOD_SRAM_SIZE,
-	  TARGET_SRAM, ATTR_SRAM, -1
+	  TARGET_SRAM, ATTR_SRAM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/* End marker */
 	{ -1, 0, 0, 0, 0, 0 }
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 15/32] arm: mach-mvebu: use ORION_ADDR_MAP_NO_REMAP
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

In this commit, we convert mach-mvebu addr_map_info definition to use
ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/addr-map.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/mach-mvebu/addr-map.c b/arch/arm/mach-mvebu/addr-map.c
index ab9b3bd..77c807b 100644
--- a/arch/arm/mach-mvebu/addr-map.c
+++ b/arch/arm/mach-mvebu/addr-map.c
@@ -42,7 +42,7 @@ armada_xp_addr_map_info[] = {
 	 * Window for the BootROM, needed for SMP on Armada XP
 	 */
 	{ 0, 0xfff00000, SZ_1M, ARMADA_XP_TARGET_DEV_BUS,
-	  ARMADA_XP_ATTR_DEV_BOOTROM, -1 },
+	  ARMADA_XP_ATTR_DEV_BOOTROM, ORION_ADDR_MAP_NO_REMAP },
 	/* End marker */
 	{ -1, 0, 0, 0, 0, 0 },
 };
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 15/32] arm: mach-mvebu: use ORION_ADDR_MAP_NO_REMAP
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

In this commit, we convert mach-mvebu addr_map_info definition to use
ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/addr-map.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/mach-mvebu/addr-map.c b/arch/arm/mach-mvebu/addr-map.c
index ab9b3bd..77c807b 100644
--- a/arch/arm/mach-mvebu/addr-map.c
+++ b/arch/arm/mach-mvebu/addr-map.c
@@ -42,7 +42,7 @@ armada_xp_addr_map_info[] = {
 	 * Window for the BootROM, needed for SMP on Armada XP
 	 */
 	{ 0, 0xfff00000, SZ_1M, ARMADA_XP_TARGET_DEV_BUS,
-	  ARMADA_XP_ATTR_DEV_BOOTROM, -1 },
+	  ARMADA_XP_ATTR_DEV_BOOTROM, ORION_ADDR_MAP_NO_REMAP },
 	/* End marker */
 	{ -1, 0, 0, 0, 0, 0 },
 };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 16/32] arm: mach-orion5x: use ORION_ADDR_MAP_NO_REMAP
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

In this commit, we convert mach-orion5x addr_map_info definition to
use ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-orion5x/addr-map.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/mach-orion5x/addr-map.c b/arch/arm/mach-orion5x/addr-map.c
index b5efc0f..4b80206 100644
--- a/arch/arm/mach-orion5x/addr-map.c
+++ b/arch/arm/mach-orion5x/addr-map.c
@@ -93,10 +93,10 @@ static const struct __initdata orion_addr_map_info addr_map_info[] = {
 	  TARGET_PCI, ATTR_PCI_IO, ORION5X_PCI_IO_BUS_BASE
 	},
 	{ 2, ORION5X_PCIE_MEM_PHYS_BASE, ORION5X_PCIE_MEM_SIZE,
-	  TARGET_PCIE, ATTR_PCIE_MEM, -1
+	  TARGET_PCIE, ATTR_PCIE_MEM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	{ 3, ORION5X_PCI_MEM_PHYS_BASE, ORION5X_PCI_MEM_SIZE,
-	  TARGET_PCI, ATTR_PCI_MEM, -1
+	  TARGET_PCI, ATTR_PCI_MEM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/* End marker */
 	{ -1, 0, 0, 0, 0, 0 }
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 16/32] arm: mach-orion5x: use ORION_ADDR_MAP_NO_REMAP
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Currently, the addr_map_info structure uses an "int remap" to allow
SoC-specific code to specify whether a given address decoding window
should be remapped, and if so, the remapping address. To do so, it
tests if the int is less than zero (no remap) or greater than
zero. Unfortunately, this prevents any remapping address that is
higher than 2 GB.

In this commit, we convert mach-orion5x addr_map_info definition to
use ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-orion5x/addr-map.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/mach-orion5x/addr-map.c b/arch/arm/mach-orion5x/addr-map.c
index b5efc0f..4b80206 100644
--- a/arch/arm/mach-orion5x/addr-map.c
+++ b/arch/arm/mach-orion5x/addr-map.c
@@ -93,10 +93,10 @@ static const struct __initdata orion_addr_map_info addr_map_info[] = {
 	  TARGET_PCI, ATTR_PCI_IO, ORION5X_PCI_IO_BUS_BASE
 	},
 	{ 2, ORION5X_PCIE_MEM_PHYS_BASE, ORION5X_PCIE_MEM_SIZE,
-	  TARGET_PCIE, ATTR_PCIE_MEM, -1
+	  TARGET_PCIE, ATTR_PCIE_MEM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	{ 3, ORION5X_PCI_MEM_PHYS_BASE, ORION5X_PCI_MEM_SIZE,
-	  TARGET_PCI, ATTR_PCI_MEM, -1
+	  TARGET_PCI, ATTR_PCI_MEM, ORION_ADDR_MAP_NO_REMAP,
 	},
 	/* End marker */
 	{ -1, 0, 0, 0, 0, 0 }
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 17/32] arm: plat-orion: convert 'int remap' to 'u32 remap'
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Now that we have ORION_ADDR_MAP_NO_REMAP in place, we can easily
convert the 'int remap' field to a 'u32 remap' field, and accept any
remapping address, except the magical ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c              |    4 ++--
 arch/arm/plat-orion/include/plat/addr-map.h |    6 +++---
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index dd98638..b1d4806 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -69,7 +69,7 @@ static int __init orion_cpu_win_can_remap(const struct orion_addr_map_cfg *cfg,
 void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 				const int win, const u32 base,
 				const u32 size, const u8 target,
-				const u8 attr, const int remap)
+				const u8 attr, const u32 remap)
 {
 	void __iomem *addr = cfg->win_cfg_base(cfg, win);
 	u32 ctrl, base_high, remap_addr;
@@ -86,7 +86,7 @@ void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 	writel(base_high, addr + WIN_BASE_OFF);
 	writel(ctrl, addr + WIN_CTRL_OFF);
 	if (cfg->cpu_win_can_remap(cfg, win)) {
-		if (remap < 0)
+		if (remap == ORION_ADDR_MAP_NO_REMAP)
 			remap_addr = base;
 		else
 			remap_addr = remap;
diff --git a/arch/arm/plat-orion/include/plat/addr-map.h b/arch/arm/plat-orion/include/plat/addr-map.h
index 0861b8c..6533c47 100644
--- a/arch/arm/plat-orion/include/plat/addr-map.h
+++ b/arch/arm/plat-orion/include/plat/addr-map.h
@@ -38,10 +38,10 @@ struct orion_addr_map_info {
 	const u32 size;
 	const u8 target;
 	const u8 attr;
-	const int remap;
+	const u32 remap;
 };
 
-#define ORION_ADDR_MAP_NO_REMAP (-1)
+#define ORION_ADDR_MAP_NO_REMAP (0xFFFFFFFF)
 
 void __init orion_config_wins(struct orion_addr_map_cfg *cfg,
 			      const struct orion_addr_map_info *info);
@@ -49,7 +49,7 @@ void __init orion_config_wins(struct orion_addr_map_cfg *cfg,
 void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 				const int win, const u32 base,
 				const u32 size, const u8 target,
-				const u8 attr, const int remap);
+				const u8 attr, const u32 remap);
 
 void __init orion_setup_cpu_mbus_target(const struct orion_addr_map_cfg *cfg,
 					const void __iomem *ddr_window_cpu_base);
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 17/32] arm: plat-orion: convert 'int remap' to 'u32 remap'
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Now that we have ORION_ADDR_MAP_NO_REMAP in place, we can easily
convert the 'int remap' field to a 'u32 remap' field, and accept any
remapping address, except the magical ORION_ADDR_MAP_NO_REMAP.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c              |    4 ++--
 arch/arm/plat-orion/include/plat/addr-map.h |    6 +++---
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index dd98638..b1d4806 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -69,7 +69,7 @@ static int __init orion_cpu_win_can_remap(const struct orion_addr_map_cfg *cfg,
 void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 				const int win, const u32 base,
 				const u32 size, const u8 target,
-				const u8 attr, const int remap)
+				const u8 attr, const u32 remap)
 {
 	void __iomem *addr = cfg->win_cfg_base(cfg, win);
 	u32 ctrl, base_high, remap_addr;
@@ -86,7 +86,7 @@ void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 	writel(base_high, addr + WIN_BASE_OFF);
 	writel(ctrl, addr + WIN_CTRL_OFF);
 	if (cfg->cpu_win_can_remap(cfg, win)) {
-		if (remap < 0)
+		if (remap == ORION_ADDR_MAP_NO_REMAP)
 			remap_addr = base;
 		else
 			remap_addr = remap;
diff --git a/arch/arm/plat-orion/include/plat/addr-map.h b/arch/arm/plat-orion/include/plat/addr-map.h
index 0861b8c..6533c47 100644
--- a/arch/arm/plat-orion/include/plat/addr-map.h
+++ b/arch/arm/plat-orion/include/plat/addr-map.h
@@ -38,10 +38,10 @@ struct orion_addr_map_info {
 	const u32 size;
 	const u8 target;
 	const u8 attr;
-	const int remap;
+	const u32 remap;
 };
 
-#define ORION_ADDR_MAP_NO_REMAP (-1)
+#define ORION_ADDR_MAP_NO_REMAP (0xFFFFFFFF)
 
 void __init orion_config_wins(struct orion_addr_map_cfg *cfg,
 			      const struct orion_addr_map_info *info);
@@ -49,7 +49,7 @@ void __init orion_config_wins(struct orion_addr_map_cfg *cfg,
 void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 				const int win, const u32 base,
 				const u32 size, const u8 target,
-				const u8 attr, const int remap);
+				const u8 attr, const u32 remap);
 
 void __init orion_setup_cpu_mbus_target(const struct orion_addr_map_cfg *cfg,
 					const void __iomem *ddr_window_cpu_base);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 18/32] arm: plat-orion: remove __init from addr-map functions needed after boot time
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The I/O and memory windows of PCIe interfaces will now be configurable
dynamically, while the system runs, so we need to keep some the
plat-orion/addr-map.c functions around even after the system
initialization has completed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c              |   18 +++++++++---------
 arch/arm/plat-orion/include/plat/addr-map.h |    8 ++++----
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index b1d4806..a187524 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -48,7 +48,7 @@ EXPORT_SYMBOL_GPL(mv_mbus_dram_info);
 /*
  * Default implementation
  */
-static void __init __iomem *
+static void __iomem *
 orion_win_cfg_base(const struct orion_addr_map_cfg *cfg, int win)
 {
 	return cfg->bridge_virt_base + (win << 4);
@@ -57,8 +57,8 @@ orion_win_cfg_base(const struct orion_addr_map_cfg *cfg, int win)
 /*
  * Default implementation
  */
-static int __init orion_cpu_win_can_remap(const struct orion_addr_map_cfg *cfg,
-					  const int win)
+static int orion_cpu_win_can_remap(const struct orion_addr_map_cfg *cfg,
+				   const int win)
 {
 	if (win < cfg->remappable_wins)
 		return 1;
@@ -66,10 +66,10 @@ static int __init orion_cpu_win_can_remap(const struct orion_addr_map_cfg *cfg,
 	return 0;
 }
 
-void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
-				const int win, const u32 base,
-				const u32 size, const u8 target,
-				const u8 attr, const u32 remap)
+void orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
+			 const int win, const u32 base,
+			 const u32 size, const u8 target,
+			 const u8 attr, const u32 remap)
 {
 	void __iomem *addr = cfg->win_cfg_base(cfg, win);
 	u32 ctrl, base_high, remap_addr;
@@ -95,8 +95,8 @@ void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 	}
 }
 
-static void __init orion_disable_cpu_win(const struct orion_addr_map_cfg *cfg,
-					 const int win)
+static void orion_disable_cpu_win(const struct orion_addr_map_cfg *cfg,
+				  const int win)
 {
 	void __iomem *addr = cfg->win_cfg_base(cfg, win);
 
diff --git a/arch/arm/plat-orion/include/plat/addr-map.h b/arch/arm/plat-orion/include/plat/addr-map.h
index 6533c47..a5b21a9 100644
--- a/arch/arm/plat-orion/include/plat/addr-map.h
+++ b/arch/arm/plat-orion/include/plat/addr-map.h
@@ -46,10 +46,10 @@ struct orion_addr_map_info {
 void __init orion_config_wins(struct orion_addr_map_cfg *cfg,
 			      const struct orion_addr_map_info *info);
 
-void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
-				const int win, const u32 base,
-				const u32 size, const u8 target,
-				const u8 attr, const u32 remap);
+void orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
+			 const int win, const u32 base,
+			 const u32 size, const u8 target,
+			 const u8 attr, const u32 remap);
 
 void __init orion_setup_cpu_mbus_target(const struct orion_addr_map_cfg *cfg,
 					const void __iomem *ddr_window_cpu_base);
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 18/32] arm: plat-orion: remove __init from addr-map functions needed after boot time
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

The I/O and memory windows of PCIe interfaces will now be configurable
dynamically, while the system runs, so we need to keep some the
plat-orion/addr-map.c functions around even after the system
initialization has completed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c              |   18 +++++++++---------
 arch/arm/plat-orion/include/plat/addr-map.h |    8 ++++----
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index b1d4806..a187524 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -48,7 +48,7 @@ EXPORT_SYMBOL_GPL(mv_mbus_dram_info);
 /*
  * Default implementation
  */
-static void __init __iomem *
+static void __iomem *
 orion_win_cfg_base(const struct orion_addr_map_cfg *cfg, int win)
 {
 	return cfg->bridge_virt_base + (win << 4);
@@ -57,8 +57,8 @@ orion_win_cfg_base(const struct orion_addr_map_cfg *cfg, int win)
 /*
  * Default implementation
  */
-static int __init orion_cpu_win_can_remap(const struct orion_addr_map_cfg *cfg,
-					  const int win)
+static int orion_cpu_win_can_remap(const struct orion_addr_map_cfg *cfg,
+				   const int win)
 {
 	if (win < cfg->remappable_wins)
 		return 1;
@@ -66,10 +66,10 @@ static int __init orion_cpu_win_can_remap(const struct orion_addr_map_cfg *cfg,
 	return 0;
 }
 
-void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
-				const int win, const u32 base,
-				const u32 size, const u8 target,
-				const u8 attr, const u32 remap)
+void orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
+			 const int win, const u32 base,
+			 const u32 size, const u8 target,
+			 const u8 attr, const u32 remap)
 {
 	void __iomem *addr = cfg->win_cfg_base(cfg, win);
 	u32 ctrl, base_high, remap_addr;
@@ -95,8 +95,8 @@ void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 	}
 }
 
-static void __init orion_disable_cpu_win(const struct orion_addr_map_cfg *cfg,
-					 const int win)
+static void orion_disable_cpu_win(const struct orion_addr_map_cfg *cfg,
+				  const int win)
 {
 	void __iomem *addr = cfg->win_cfg_base(cfg, win);
 
diff --git a/arch/arm/plat-orion/include/plat/addr-map.h b/arch/arm/plat-orion/include/plat/addr-map.h
index 6533c47..a5b21a9 100644
--- a/arch/arm/plat-orion/include/plat/addr-map.h
+++ b/arch/arm/plat-orion/include/plat/addr-map.h
@@ -46,10 +46,10 @@ struct orion_addr_map_info {
 void __init orion_config_wins(struct orion_addr_map_cfg *cfg,
 			      const struct orion_addr_map_info *info);
 
-void __init orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
-				const int win, const u32 base,
-				const u32 size, const u8 target,
-				const u8 attr, const u32 remap);
+void orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
+			 const int win, const u32 base,
+			 const u32 size, const u8 target,
+			 const u8 attr, const u32 remap);
 
 void __init orion_setup_cpu_mbus_target(const struct orion_addr_map_cfg *cfg,
 					const void __iomem *ddr_window_cpu_base);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 19/32] arm: plat-orion: introduce orion_{alloc,free}_cpu_win() functions
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

In the address decoding code, we implement two new functions:
orion_alloc_cpu_win() and orion_free_cpu_win(). The first function
finds an unused address decoding window, and configures it according
to the given arguments (in terms of base address, size, target,
attributes). The second function frees an address decoding window,
given a physical base address.

Those two new functions will be used by the PCIe code, which needs to
dynamically register address decoding windows depending on the PCIe
devices that are detected.

The orion_free_cpu_win() function is only here to handle error cases
in the PCIe devices initialization, in the normal case, address
decoding windows are never freed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c              |   75 +++++++++++++++++++++++++++
 arch/arm/plat-orion/include/plat/addr-map.h |    7 +++
 2 files changed, 82 insertions(+)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index a187524..a7bdc21 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -108,6 +108,81 @@ static void orion_disable_cpu_win(const struct orion_addr_map_cfg *cfg,
 	}
 }
 
+static int orion_window_is_free(const struct orion_addr_map_cfg *cfg,
+				const int win)
+{
+	void __iomem *addr = cfg->win_cfg_base(cfg, win);
+	u32 ctrl = readl(addr + WIN_CTRL_OFF);
+	return !(ctrl & WIN_CTRL_ENABLE);
+}
+
+/*
+ * Find an unused address decoding window, and enable it according to
+ * the arguments passed (base, size, target, attributes, remap).
+ */
+int orion_alloc_cpu_win(const struct orion_addr_map_cfg *cfg,
+			const u32 base, const u32 size,
+			const u8 target, const u8 attr, const u32 remap)
+{
+	int win;
+
+	/*
+	 * If we don't need to remap, try to first allocate a window
+	 * that doesn't have remapping capabilities, since those are
+	 * pretty limited in number.
+	 */
+	if (remap == ORION_ADDR_MAP_NO_REMAP) {
+		for (win = cfg->remappable_wins; win < cfg->num_wins; win++)
+			if (orion_window_is_free(cfg, win))
+				break;
+
+		if (win < cfg->num_wins) {
+			orion_setup_cpu_win(cfg, win, base, size, target, attr, remap);
+			return 0;
+		}
+	}
+
+	/*
+	 * We need a remap window, or we haven't found a non-remap
+	 * capable window.
+	 */
+	for (win = 0; win < cfg->num_wins; win++)
+		if (orion_window_is_free(cfg, win))
+			break;
+
+	if (win < cfg->num_wins) {
+		orion_setup_cpu_win(cfg, win, base, size, target, attr, remap);
+		return 0;
+	}
+
+	/* No window found */
+	return -ENOMEM;
+}
+
+/*
+ * Free an address decoding window, given its base address.
+ */
+int orion_free_cpu_win(const struct orion_addr_map_cfg *cfg,
+		       const u32 base)
+{
+	int win;
+
+	for (win = 0; win < cfg->num_wins; win++) {
+		void __iomem *addr = cfg->win_cfg_base(cfg, win);
+		u32 winbase = readl(addr + WIN_BASE_OFF);
+
+		if (orion_window_is_free(cfg, win))
+			continue;
+
+		if (winbase == (base & 0xffff0000)) {
+			orion_disable_cpu_win(cfg, win);
+			return 0;
+		}
+	}
+
+	return -EINVAL;
+}
+
 /*
  * Configure a number of windows.
  */
diff --git a/arch/arm/plat-orion/include/plat/addr-map.h b/arch/arm/plat-orion/include/plat/addr-map.h
index a5b21a9..fc46878 100644
--- a/arch/arm/plat-orion/include/plat/addr-map.h
+++ b/arch/arm/plat-orion/include/plat/addr-map.h
@@ -51,6 +51,13 @@ void orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 			 const u32 size, const u8 target,
 			 const u8 attr, const u32 remap);
 
+int orion_alloc_cpu_win(const struct orion_addr_map_cfg *cfg,
+			const u32 base, const u32 size,
+			const u8 target, const u8 attr, const u32 remap);
+
+int orion_free_cpu_win(const struct orion_addr_map_cfg *cfg,
+		       const u32 base);
+
 void __init orion_setup_cpu_mbus_target(const struct orion_addr_map_cfg *cfg,
 					const void __iomem *ddr_window_cpu_base);
 #endif
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 19/32] arm: plat-orion: introduce orion_{alloc, free}_cpu_win() functions
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

In the address decoding code, we implement two new functions:
orion_alloc_cpu_win() and orion_free_cpu_win(). The first function
finds an unused address decoding window, and configures it according
to the given arguments (in terms of base address, size, target,
attributes). The second function frees an address decoding window,
given a physical base address.

Those two new functions will be used by the PCIe code, which needs to
dynamically register address decoding windows depending on the PCIe
devices that are detected.

The orion_free_cpu_win() function is only here to handle error cases
in the PCIe devices initialization, in the normal case, address
decoding windows are never freed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/addr-map.c              |   75 +++++++++++++++++++++++++++
 arch/arm/plat-orion/include/plat/addr-map.h |    7 +++
 2 files changed, 82 insertions(+)

diff --git a/arch/arm/plat-orion/addr-map.c b/arch/arm/plat-orion/addr-map.c
index a187524..a7bdc21 100644
--- a/arch/arm/plat-orion/addr-map.c
+++ b/arch/arm/plat-orion/addr-map.c
@@ -108,6 +108,81 @@ static void orion_disable_cpu_win(const struct orion_addr_map_cfg *cfg,
 	}
 }
 
+static int orion_window_is_free(const struct orion_addr_map_cfg *cfg,
+				const int win)
+{
+	void __iomem *addr = cfg->win_cfg_base(cfg, win);
+	u32 ctrl = readl(addr + WIN_CTRL_OFF);
+	return !(ctrl & WIN_CTRL_ENABLE);
+}
+
+/*
+ * Find an unused address decoding window, and enable it according to
+ * the arguments passed (base, size, target, attributes, remap).
+ */
+int orion_alloc_cpu_win(const struct orion_addr_map_cfg *cfg,
+			const u32 base, const u32 size,
+			const u8 target, const u8 attr, const u32 remap)
+{
+	int win;
+
+	/*
+	 * If we don't need to remap, try to first allocate a window
+	 * that doesn't have remapping capabilities, since those are
+	 * pretty limited in number.
+	 */
+	if (remap == ORION_ADDR_MAP_NO_REMAP) {
+		for (win = cfg->remappable_wins; win < cfg->num_wins; win++)
+			if (orion_window_is_free(cfg, win))
+				break;
+
+		if (win < cfg->num_wins) {
+			orion_setup_cpu_win(cfg, win, base, size, target, attr, remap);
+			return 0;
+		}
+	}
+
+	/*
+	 * We need a remap window, or we haven't found a non-remap
+	 * capable window.
+	 */
+	for (win = 0; win < cfg->num_wins; win++)
+		if (orion_window_is_free(cfg, win))
+			break;
+
+	if (win < cfg->num_wins) {
+		orion_setup_cpu_win(cfg, win, base, size, target, attr, remap);
+		return 0;
+	}
+
+	/* No window found */
+	return -ENOMEM;
+}
+
+/*
+ * Free an address decoding window, given its base address.
+ */
+int orion_free_cpu_win(const struct orion_addr_map_cfg *cfg,
+		       const u32 base)
+{
+	int win;
+
+	for (win = 0; win < cfg->num_wins; win++) {
+		void __iomem *addr = cfg->win_cfg_base(cfg, win);
+		u32 winbase = readl(addr + WIN_BASE_OFF);
+
+		if (orion_window_is_free(cfg, win))
+			continue;
+
+		if (winbase == (base & 0xffff0000)) {
+			orion_disable_cpu_win(cfg, win);
+			return 0;
+		}
+	}
+
+	return -EINVAL;
+}
+
 /*
  * Configure a number of windows.
  */
diff --git a/arch/arm/plat-orion/include/plat/addr-map.h b/arch/arm/plat-orion/include/plat/addr-map.h
index a5b21a9..fc46878 100644
--- a/arch/arm/plat-orion/include/plat/addr-map.h
+++ b/arch/arm/plat-orion/include/plat/addr-map.h
@@ -51,6 +51,13 @@ void orion_setup_cpu_win(const struct orion_addr_map_cfg *cfg,
 			 const u32 size, const u8 target,
 			 const u8 attr, const u32 remap);
 
+int orion_alloc_cpu_win(const struct orion_addr_map_cfg *cfg,
+			const u32 base, const u32 size,
+			const u8 target, const u8 attr, const u32 remap);
+
+int orion_free_cpu_win(const struct orion_addr_map_cfg *cfg,
+		       const u32 base);
+
 void __init orion_setup_cpu_mbus_target(const struct orion_addr_map_cfg *cfg,
 					const void __iomem *ddr_window_cpu_base);
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 20/32] arm: plat-orion: remove __init from PCIe functions needed after boot time
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The orion_pcie_set_local_bus_nr() function will be used by the Marvell
PCIe driver in its PCI-to-PCI bridge emulation. This means that this
function can now get called at any point during the system execution,
and no longer at boot time. For example, if PCIe devices
appear/disappear during the system execution (either through hotplug,
or manipulation of PCIe devices using /sys).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/pcie.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/plat-orion/pcie.c b/arch/arm/plat-orion/pcie.c
index f20a321..da921e7 100644
--- a/arch/arm/plat-orion/pcie.c
+++ b/arch/arm/plat-orion/pcie.c
@@ -79,7 +79,7 @@ int orion_pcie_get_local_bus_nr(void __iomem *base)
 	return (stat >> PCIE_STAT_BUS_OFFS) & PCIE_STAT_BUS_MASK;
 }
 
-void __init orion_pcie_set_local_bus_nr(void __iomem *base, int nr)
+void orion_pcie_set_local_bus_nr(void __iomem *base, int nr)
 {
 	u32 stat;
 
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 20/32] arm: plat-orion: remove __init from PCIe functions needed after boot time
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

The orion_pcie_set_local_bus_nr() function will be used by the Marvell
PCIe driver in its PCI-to-PCI bridge emulation. This means that this
function can now get called at any point during the system execution,
and no longer at boot time. For example, if PCIe devices
appear/disappear during the system execution (either through hotplug,
or manipulation of PCIe devices using /sys).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/pcie.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/plat-orion/pcie.c b/arch/arm/plat-orion/pcie.c
index f20a321..da921e7 100644
--- a/arch/arm/plat-orion/pcie.c
+++ b/arch/arm/plat-orion/pcie.c
@@ -79,7 +79,7 @@ int orion_pcie_get_local_bus_nr(void __iomem *base)
 	return (stat >> PCIE_STAT_BUS_OFFS) & PCIE_STAT_BUS_MASK;
 }
 
-void __init orion_pcie_set_local_bus_nr(void __iomem *base, int nr)
+void orion_pcie_set_local_bus_nr(void __iomem *base, int nr)
 {
 	u32 stat;
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 21/32] arm: mvebu: add functions to alloc/free PCIe decoding windows
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

This commit adds two functions armada_370_xp_alloc_pcie_window() and
armada_370_xp_free_pcie_window() that respectively allocate and free
an address decoding window pointing to either a memory or I/O region
of a PCIe device.

Those functions will be used by the PCIe driver to create and remove
those regions depending on the PCIe devices that are detected.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/addr-map.c              |   87 +++++++++++++++++++++++++--
 arch/arm/mach-mvebu/common.h                |    1 +
 arch/arm/mach-mvebu/include/mach/addr-map.h |   10 +++
 3 files changed, 94 insertions(+), 4 deletions(-)
 create mode 100644 arch/arm/mach-mvebu/include/mach/addr-map.h

diff --git a/arch/arm/mach-mvebu/addr-map.c b/arch/arm/mach-mvebu/addr-map.c
index 77c807b..6f9dda7 100644
--- a/arch/arm/mach-mvebu/addr-map.c
+++ b/arch/arm/mach-mvebu/addr-map.c
@@ -24,14 +24,10 @@
 #define ARMADA_XP_TARGET_DEV_BUS	1
 #define   ARMADA_XP_ATTR_DEV_BOOTROM    0x1D
 #define ARMADA_XP_TARGET_ETH1		3
-#define ARMADA_XP_TARGET_PCIE_0_2	4
 #define ARMADA_XP_TARGET_ETH0		7
-#define ARMADA_XP_TARGET_PCIE_1_3	8
 
 #define ARMADA_370_TARGET_DEV_BUS       1
 #define   ARMADA_370_ATTR_DEV_BOOTROM   0x1D
-#define ARMADA_370_TARGET_PCIE_0        4
-#define ARMADA_370_TARGET_PCIE_1        8
 
 #define ARMADA_WINDOW_8_PLUS_OFFSET       0x90
 #define ARMADA_SDRAM_ADDR_DECODING_OFFSET 0x180
@@ -89,6 +85,89 @@ static struct __initdata orion_addr_map_cfg addr_map_cfg = {
 	.win_cfg_base = armada_cfg_base,
 };
 
+#ifdef CONFIG_PCI
+/*
+ * This structure and the following arrays allow to map a PCIe (port,
+ * lane) tuple to the corresponding (target, attribute) tuple needed
+ * to configure an address decoding window for the given PCIe (port,
+ * lane).
+ */
+struct pcie_mapping {
+	int port;
+	int lane;
+	u8  target;
+	u8  attr;
+};
+
+struct pcie_mapping armada_xp_pcie_mappings[] = {
+	{ .port = 0, .lane = 0, .target = 4, .attr = 0xE0 },
+	{ .port = 0, .lane = 1, .target = 4, .attr = 0xD0 },
+	{ .port = 0, .lane = 2, .target = 4, .attr = 0xB0 },
+	{ .port = 0, .lane = 3, .target = 4, .attr = 0x70 },
+	{ .port = 1, .lane = 0, .target = 8, .attr = 0xE0 },
+	{ .port = 1, .lane = 1, .target = 8, .attr = 0xD0 },
+	{ .port = 1, .lane = 2, .target = 8, .attr = 0xB0 },
+	{ .port = 1, .lane = 3, .target = 8, .attr = 0x70 },
+	{ .port = 2, .lane = 0, .target = 4, .attr = 0xF0 },
+	{ .port = 3, .lane = 0, .target = 8, .attr = 0xF0 },
+	{ .port = -1 },
+};
+
+struct pcie_mapping armada_370_pcie_mappings[] = {
+	{ .port = 0, .lane = 0, .target = 4, .attr = 0xE0 },
+	{ .port = 1, .lane = 0, .target = 8, .attr = 0xE0 },
+	{ .port = -1 },
+};
+
+/*
+ * This function sets up a new address decoding window at the given
+ * base address, pointing to the given PCIe interface (through
+ * pcie_port and pcie_lane).
+ */
+int armada_370_xp_alloc_pcie_window(int pcie_port, int pcie_lane,
+				    u32 base, u32 remap, u32 size,
+				    int type)
+{
+	struct pcie_mapping *mapping, *mappings;
+	u8 target, attr;
+
+	if (of_machine_is_compatible("marvell,armadaxp"))
+		mappings = armada_xp_pcie_mappings;
+	else if (of_machine_is_compatible("marvell,armada370"))
+		mappings = armada_370_pcie_mappings;
+	else
+		return -ENODEV;
+
+	for (mapping = mappings; mapping->port != -1; mapping++)
+		if (mapping->port == pcie_port && mapping->lane == pcie_lane)
+			break;
+
+	if (mapping->port == -1)
+		return -ENODEV;
+
+	target = mapping->target;
+	attr = mapping->attr;
+
+	/*
+	 * Bit 3 of the attributes indicates that it is a
+	 * memory region, as opposed to an I/O region
+	 */
+	if (type == IORESOURCE_MEM)
+		attr |= (1 << 3);
+
+	return orion_alloc_cpu_win(&addr_map_cfg, base, size, target, attr, remap);
+}
+
+/*
+ * Frees an address decoding window previously set up by
+ * armada_370_xp_setup_pcie_window().
+ */
+int armada_370_xp_free_pcie_window(u32 base)
+{
+	return orion_free_cpu_win(&addr_map_cfg, base);
+}
+#endif
+
 static int __init armada_setup_cpu_mbus(void)
 {
 	struct device_node *np;
diff --git a/arch/arm/mach-mvebu/common.h b/arch/arm/mach-mvebu/common.h
index aa27bc2..77c078c 100644
--- a/arch/arm/mach-mvebu/common.h
+++ b/arch/arm/mach-mvebu/common.h
@@ -25,4 +25,5 @@ int armada_370_xp_coherency_init(void);
 int armada_370_xp_pmsu_init(void);
 void armada_xp_secondary_startup(void);
 extern struct smp_operations armada_xp_smp_ops;
+
 #endif
diff --git a/arch/arm/mach-mvebu/include/mach/addr-map.h b/arch/arm/mach-mvebu/include/mach/addr-map.h
new file mode 100644
index 0000000..e2d29cd
--- /dev/null
+++ b/arch/arm/mach-mvebu/include/mach/addr-map.h
@@ -0,0 +1,10 @@
+#ifndef MVEBU_ADDR_MAP_H
+#define MVEBU_ADDR_MAP_H
+
+#include <plat/addr-map.h>
+
+int armada_370_xp_alloc_pcie_window(int pcie_port, int pcie_lane,
+				    u32 base, u32 remap, u32 size, int type);
+int armada_370_xp_free_pcie_window(unsigned long base);
+
+#endif /* MVEBU_ADDR_MAP_H */
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 21/32] arm: mvebu: add functions to alloc/free PCIe decoding windows
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

This commit adds two functions armada_370_xp_alloc_pcie_window() and
armada_370_xp_free_pcie_window() that respectively allocate and free
an address decoding window pointing to either a memory or I/O region
of a PCIe device.

Those functions will be used by the PCIe driver to create and remove
those regions depending on the PCIe devices that are detected.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/addr-map.c              |   87 +++++++++++++++++++++++++--
 arch/arm/mach-mvebu/common.h                |    1 +
 arch/arm/mach-mvebu/include/mach/addr-map.h |   10 +++
 3 files changed, 94 insertions(+), 4 deletions(-)
 create mode 100644 arch/arm/mach-mvebu/include/mach/addr-map.h

diff --git a/arch/arm/mach-mvebu/addr-map.c b/arch/arm/mach-mvebu/addr-map.c
index 77c807b..6f9dda7 100644
--- a/arch/arm/mach-mvebu/addr-map.c
+++ b/arch/arm/mach-mvebu/addr-map.c
@@ -24,14 +24,10 @@
 #define ARMADA_XP_TARGET_DEV_BUS	1
 #define   ARMADA_XP_ATTR_DEV_BOOTROM    0x1D
 #define ARMADA_XP_TARGET_ETH1		3
-#define ARMADA_XP_TARGET_PCIE_0_2	4
 #define ARMADA_XP_TARGET_ETH0		7
-#define ARMADA_XP_TARGET_PCIE_1_3	8
 
 #define ARMADA_370_TARGET_DEV_BUS       1
 #define   ARMADA_370_ATTR_DEV_BOOTROM   0x1D
-#define ARMADA_370_TARGET_PCIE_0        4
-#define ARMADA_370_TARGET_PCIE_1        8
 
 #define ARMADA_WINDOW_8_PLUS_OFFSET       0x90
 #define ARMADA_SDRAM_ADDR_DECODING_OFFSET 0x180
@@ -89,6 +85,89 @@ static struct __initdata orion_addr_map_cfg addr_map_cfg = {
 	.win_cfg_base = armada_cfg_base,
 };
 
+#ifdef CONFIG_PCI
+/*
+ * This structure and the following arrays allow to map a PCIe (port,
+ * lane) tuple to the corresponding (target, attribute) tuple needed
+ * to configure an address decoding window for the given PCIe (port,
+ * lane).
+ */
+struct pcie_mapping {
+	int port;
+	int lane;
+	u8  target;
+	u8  attr;
+};
+
+struct pcie_mapping armada_xp_pcie_mappings[] = {
+	{ .port = 0, .lane = 0, .target = 4, .attr = 0xE0 },
+	{ .port = 0, .lane = 1, .target = 4, .attr = 0xD0 },
+	{ .port = 0, .lane = 2, .target = 4, .attr = 0xB0 },
+	{ .port = 0, .lane = 3, .target = 4, .attr = 0x70 },
+	{ .port = 1, .lane = 0, .target = 8, .attr = 0xE0 },
+	{ .port = 1, .lane = 1, .target = 8, .attr = 0xD0 },
+	{ .port = 1, .lane = 2, .target = 8, .attr = 0xB0 },
+	{ .port = 1, .lane = 3, .target = 8, .attr = 0x70 },
+	{ .port = 2, .lane = 0, .target = 4, .attr = 0xF0 },
+	{ .port = 3, .lane = 0, .target = 8, .attr = 0xF0 },
+	{ .port = -1 },
+};
+
+struct pcie_mapping armada_370_pcie_mappings[] = {
+	{ .port = 0, .lane = 0, .target = 4, .attr = 0xE0 },
+	{ .port = 1, .lane = 0, .target = 8, .attr = 0xE0 },
+	{ .port = -1 },
+};
+
+/*
+ * This function sets up a new address decoding window at the given
+ * base address, pointing to the given PCIe interface (through
+ * pcie_port and pcie_lane).
+ */
+int armada_370_xp_alloc_pcie_window(int pcie_port, int pcie_lane,
+				    u32 base, u32 remap, u32 size,
+				    int type)
+{
+	struct pcie_mapping *mapping, *mappings;
+	u8 target, attr;
+
+	if (of_machine_is_compatible("marvell,armadaxp"))
+		mappings = armada_xp_pcie_mappings;
+	else if (of_machine_is_compatible("marvell,armada370"))
+		mappings = armada_370_pcie_mappings;
+	else
+		return -ENODEV;
+
+	for (mapping = mappings; mapping->port != -1; mapping++)
+		if (mapping->port == pcie_port && mapping->lane == pcie_lane)
+			break;
+
+	if (mapping->port == -1)
+		return -ENODEV;
+
+	target = mapping->target;
+	attr = mapping->attr;
+
+	/*
+	 * Bit 3 of the attributes indicates that it is a
+	 * memory region, as opposed to an I/O region
+	 */
+	if (type == IORESOURCE_MEM)
+		attr |= (1 << 3);
+
+	return orion_alloc_cpu_win(&addr_map_cfg, base, size, target, attr, remap);
+}
+
+/*
+ * Frees an address decoding window previously set up by
+ * armada_370_xp_setup_pcie_window().
+ */
+int armada_370_xp_free_pcie_window(u32 base)
+{
+	return orion_free_cpu_win(&addr_map_cfg, base);
+}
+#endif
+
 static int __init armada_setup_cpu_mbus(void)
 {
 	struct device_node *np;
diff --git a/arch/arm/mach-mvebu/common.h b/arch/arm/mach-mvebu/common.h
index aa27bc2..77c078c 100644
--- a/arch/arm/mach-mvebu/common.h
+++ b/arch/arm/mach-mvebu/common.h
@@ -25,4 +25,5 @@ int armada_370_xp_coherency_init(void);
 int armada_370_xp_pmsu_init(void);
 void armada_xp_secondary_startup(void);
 extern struct smp_operations armada_xp_smp_ops;
+
 #endif
diff --git a/arch/arm/mach-mvebu/include/mach/addr-map.h b/arch/arm/mach-mvebu/include/mach/addr-map.h
new file mode 100644
index 0000000..e2d29cd
--- /dev/null
+++ b/arch/arm/mach-mvebu/include/mach/addr-map.h
@@ -0,0 +1,10 @@
+#ifndef MVEBU_ADDR_MAP_H
+#define MVEBU_ADDR_MAP_H
+
+#include <plat/addr-map.h>
+
+int armada_370_xp_alloc_pcie_window(int pcie_port, int pcie_lane,
+				    u32 base, u32 remap, u32 size, int type);
+int armada_370_xp_free_pcie_window(unsigned long base);
+
+#endif /* MVEBU_ADDR_MAP_H */
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 22/32] arm: plat-orion: make common PCIe code usable on mvebu
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

mvebu is a new-style Orion platform, so it only selects PLAT_ORION,
but not PLAT_ORION_LEGACY. It will however need the common PCIe code
from plat-orion, so make this code available for PLAT_ORION platforms
as a whole, and not only PLAT_ORION_LEGACY platforms.

We also take this opportunity to build the PCIe code only when
CONFIG_PCI is enabled.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/Makefile |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm/plat-orion/Makefile b/arch/arm/plat-orion/Makefile
index a82cecb..1aca22b 100644
--- a/arch/arm/plat-orion/Makefile
+++ b/arch/arm/plat-orion/Makefile
@@ -4,7 +4,8 @@
 ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include
 
 obj-y                             += addr-map.o
+obj-$(CONFIG_PCI)                 += pcie.o
 
 orion-gpio-$(CONFIG_GENERIC_GPIO) += gpio.o
-obj-$(CONFIG_PLAT_ORION_LEGACY)   += irq.o pcie.o time.o common.o mpp.o
+obj-$(CONFIG_PLAT_ORION_LEGACY)   += irq.o time.o common.o mpp.o
 obj-$(CONFIG_PLAT_ORION_LEGACY)   += $(orion-gpio-y)
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 22/32] arm: plat-orion: make common PCIe code usable on mvebu
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

mvebu is a new-style Orion platform, so it only selects PLAT_ORION,
but not PLAT_ORION_LEGACY. It will however need the common PCIe code
from plat-orion, so make this code available for PLAT_ORION platforms
as a whole, and not only PLAT_ORION_LEGACY platforms.

We also take this opportunity to build the PCIe code only when
CONFIG_PCI is enabled.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/plat-orion/Makefile |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/arm/plat-orion/Makefile b/arch/arm/plat-orion/Makefile
index a82cecb..1aca22b 100644
--- a/arch/arm/plat-orion/Makefile
+++ b/arch/arm/plat-orion/Makefile
@@ -4,7 +4,8 @@
 ccflags-$(CONFIG_ARCH_MULTIPLATFORM) := -I$(srctree)/$(src)/include
 
 obj-y                             += addr-map.o
+obj-$(CONFIG_PCI)                 += pcie.o
 
 orion-gpio-$(CONFIG_GENERIC_GPIO) += gpio.o
-obj-$(CONFIG_PLAT_ORION_LEGACY)   += irq.o pcie.o time.o common.o mpp.o
+obj-$(CONFIG_PLAT_ORION_LEGACY)   += irq.o time.o common.o mpp.o
 obj-$(CONFIG_PLAT_ORION_LEGACY)   += $(orion-gpio-y)
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 23/32] pci: infrastructure to add drivers in drivers/pci/host
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

As agreed by the community, PCI host drivers will now be stored in
drivers/pci/host. This commit adds this directory and the related
Kconfig/Makefile changes to allow new drivers to be added in this
directory.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/pci/Kconfig      |    2 ++
 drivers/pci/Makefile     |    3 +++
 drivers/pci/host/Kconfig |    4 ++++
 3 files changed, 9 insertions(+)
 create mode 100644 drivers/pci/host/Kconfig

diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 6d51aa6..ac45398 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -119,3 +119,5 @@ config PCI_IOAPIC
 config PCI_LABEL
 	def_bool y if (DMI || ACPI)
 	select NLS
+
+source "drivers/pci/host/Kconfig"
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 0c3efcf..6ebf5bf 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -67,3 +67,6 @@ obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
 obj-$(CONFIG_OF) += of.o
 
 ccflags-$(CONFIG_PCI_DEBUG) := -DDEBUG
+
+# PCI host controller drivers
+obj-y += host/
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
new file mode 100644
index 0000000..cc3a1af
--- /dev/null
+++ b/drivers/pci/host/Kconfig
@@ -0,0 +1,4 @@
+menu "PCI host controller drivers"
+	depends on PCI
+
+endmenu
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 23/32] pci: infrastructure to add drivers in drivers/pci/host
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

As agreed by the community, PCI host drivers will now be stored in
drivers/pci/host. This commit adds this directory and the related
Kconfig/Makefile changes to allow new drivers to be added in this
directory.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/pci/Kconfig      |    2 ++
 drivers/pci/Makefile     |    3 +++
 drivers/pci/host/Kconfig |    4 ++++
 3 files changed, 9 insertions(+)
 create mode 100644 drivers/pci/host/Kconfig

diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 6d51aa6..ac45398 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -119,3 +119,5 @@ config PCI_IOAPIC
 config PCI_LABEL
 	def_bool y if (DMI || ACPI)
 	select NLS
+
+source "drivers/pci/host/Kconfig"
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 0c3efcf..6ebf5bf 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -67,3 +67,6 @@ obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
 obj-$(CONFIG_OF) += of.o
 
 ccflags-$(CONFIG_PCI_DEBUG) := -DDEBUG
+
+# PCI host controller drivers
+obj-y += host/
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
new file mode 100644
index 0000000..cc3a1af
--- /dev/null
+++ b/drivers/pci/host/Kconfig
@@ -0,0 +1,4 @@
+menu "PCI host controller drivers"
+	depends on PCI
+
+endmenu
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

This driver implements the support for the PCIe interfaces on the
Marvell Armada 370/XP ARM SoCs. In the future, it might be extended to
cover earlier families of Marvell SoCs, such as Dove, Orion and
Kirkwood.

The driver implements the hw_pci operations needed by the core ARM PCI
code to setup PCI devices and get their corresponding IRQs, and the
pci_ops operations that are used by the PCI core to read/write the
configuration space of PCI devices.

Since the PCIe interfaces of Marvell SoCs are completely separate and
not linked together in a bus, this driver sets up an emulated PCI host
bridge, with one PCI-to-PCI bridge as child for each hardware PCIe
interface.

In addition, this driver enumerates the different PCIe slots, and for
those having a device plugged in, it sets up the necessary address
decoding windows, using the new armada_370_xp_alloc_pcie_window()
function from mach-mvebu/addr-map.c.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 .../devicetree/bindings/pci/mvebu-pci.txt          |  174 +++++
 drivers/pci/host/Kconfig                           |    6 +
 drivers/pci/host/Makefile                          |    4 +
 drivers/pci/host/pci-mvebu.c                       |  688 ++++++++++++++++++++
 4 files changed, 872 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/mvebu-pci.txt
 create mode 100644 drivers/pci/host/Makefile
 create mode 100644 drivers/pci/host/pci-mvebu.c

diff --git a/Documentation/devicetree/bindings/pci/mvebu-pci.txt b/Documentation/devicetree/bindings/pci/mvebu-pci.txt
new file mode 100644
index 0000000..bd621c8
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/mvebu-pci.txt
@@ -0,0 +1,174 @@
+* Marvell EBU PCIe interfaces
+
+Mandatory properties:
+- compatible: for now only "marvell,armada-370-xp-pcie" is supported
+- #address-cells, set to <3>
+- #size-cells, set to <2>
+- #interrupt-cells, set to <1>
+- bus-range: PCI bus numbers covered
+- ranges: standard PCI-style address ranges, describing the PCIe
+  registers for each PCIe interface, and then ranges for the PCI
+  memory and I/O regions.
+- interrupt-map-mask and interrupt-map are standard PCI Device Tree
+  properties to describe the interrupts associated to each PCI
+  interface.
+
+In addition, the Device Tree node must have sub-nodes describing each
+PCIe interface, having the following mandatory properties:
+- reg: the address and size of the PCIe registers (translated
+  addresses according to the ranges property of the parent)
+- clocks: the clock associated to this PCIe interface
+- marvell,pcie-port: the physical PCIe port number
+- status: either "disabled" or "okay"
+
+and the following optional properties:
+- marvell,pcie-lane: the physical PCIe lane number, for ports having
+  multiple lanes. If this property is not found, we assume that the
+  value is 0.
+
+Example:
+
+pcie-controller {
+	compatible = "marvell,armada-370-xp-pcie";
+	status = "disabled";
+
+	#address-cells = <3>;
+	#size-cells = <2>;
+
+	bus-range = <0x00 0xff>;
+
+	ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+	          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
+	          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
+	          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
+	          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
+		  0x00002800 0 0xd0080000 0xd0080000 0 0x00002000   /* port 1.0 registers */
+	          0x00005000 0 0xd0082000 0xd0082000 0 0x00002000   /* port 3.0 registers */
+		  0x00003000 0 0xd0084000 0xd0084000 0 0x00002000   /* port 1.1 registers */
+		  0x00003800 0 0xd0088000 0xd0088000 0 0x00002000   /* port 1.2 registers */
+		  0x00004000 0 0xd008C000 0xd008C000 0 0x00002000   /* port 1.3 registers */
+		  0x81000000 0 0	  0xc0000000 0 0x00100000   /* downstream I/O */
+		  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+	#interrupt-cells = <1>;
+	interrupt-map-mask = <0xf800 0 0 1>;
+	interrupt-map = <0x0800 0 0 1 &mpic 58
+		         0x1000 0 0 1 &mpic 59
+			 0x1800 0 0 1 &mpic 60
+			 0x2000 0 0 1 &mpic 61
+			 0x2800 0 0 1 &mpic 62
+		         0x3000 0 0 1 &mpic 63
+			 0x3800 0 0 1 &mpic 64
+			 0x4000 0 0 1 &mpic 65
+			 0x4800 0 0 1 &mpic 99
+			 0x5000 0 0 1 &mpic 103>;
+
+	pcie@0,0 {
+		device_type = "pciex";
+		reg = <0x0800 0 0xd0040000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <0>;
+		marvell,pcie-lane = <0>;
+		clocks = <&gateclk 5>;
+		status = "disabled";
+	};
+
+	pcie@0,1 {
+		device_type = "pciex";
+		reg = <0x1000 0 0xd0044000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <0>;
+		marvell,pcie-lane = <1>;
+		clocks = <&gateclk 6>;
+		status = "disabled";
+	};
+
+	pcie@0,2 {
+		device_type = "pciex";
+		reg = <0x1800 0 0xd0048000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <0>;
+		marvell,pcie-lane = <2>;
+		clocks = <&gateclk 7>;
+		status = "disabled";
+	};
+
+	pcie@0,3 {
+		device_type = "pciex";
+		reg = <0x2000 0 0xd004C000 0 0xC000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <0>;
+		marvell,pcie-lane = <3>;
+		clocks = <&gateclk 8>;
+		status = "disabled";
+	};
+
+	pcie@1,0 {
+		device_type = "pciex";
+		reg = <0x2800 0 0xd0080000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <1>;
+		marvell,pcie-lane = <0>;
+		clocks = <&gateclk 9>;
+		status = "disabled";
+	};
+
+	pcie@1,1 {
+		device_type = "pciex";
+		reg = <0x3000 0 0xd0084000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <1>;
+		marvell,pcie-lane = <1>;
+		clocks = <&gateclk 10>;
+		status = "disabled";
+	};
+
+	pcie@1,2 {
+		device_type = "pciex";
+		reg = <0x3800 0 0xd0088000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <1>;
+		marvell,pcie-lane = <2>;
+		clocks = <&gateclk 11>;
+		status = "disabled";
+	};
+
+	pcie@1,3 {
+		device_type = "pciex";
+		reg = <0x4000 0 0xd008C000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <1>;
+		marvell,pcie-lane = <3>;
+		clocks = <&gateclk 12>;
+		status = "disabled";
+	};
+	pcie@2,0 {
+		device_type = "pciex";
+		reg = <0x4800 0 0xd0042000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <2>;
+		marvell,pcie-lane = <0>;
+		clocks = <&gateclk 26>;
+		status = "disabled";
+	};
+
+	pcie@3,0 {
+		device_type = "pciex";
+		reg = <0x5000 0 0xd0082000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <3>;
+		marvell,pcie-lane = <0>;
+		clocks = <&gateclk 27>;
+		status = "disabled";
+	};
+};
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index cc3a1af..03e15e7 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -1,4 +1,10 @@
 menu "PCI host controller drivers"
 	depends on PCI
 
+config PCI_MVEBU
+	bool "Marvell EBU PCIe controller"
+	depends on ARCH_MVEBU
+	select PCI_SW_HOST_BRIDGE
+	select PCI_SW_PCI_PCI_BRIDGE
+
 endmenu
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
new file mode 100644
index 0000000..3ad563f
--- /dev/null
+++ b/drivers/pci/host/Makefile
@@ -0,0 +1,4 @@
+obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
+CFLAGS_pci-mvebu.o += \
+	-I$(srctree)/arch/arm/plat-orion/include \
+	-I$(srctree)/arch/arm/mach-mvebu/include
diff --git a/drivers/pci/host/pci-mvebu.c b/drivers/pci/host/pci-mvebu.c
new file mode 100644
index 0000000..82d13da
--- /dev/null
+++ b/drivers/pci/host/pci-mvebu.c
@@ -0,0 +1,688 @@
+/*
+ * PCIe driver for Marvell Armada 370 and Armada XP SoCs
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/platform_device.h>
+#include <linux/of_address.h>
+#include <linux/of_pci.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <plat/pcie.h>
+#include <mach/addr-map.h>
+
+/*
+ * This product ID is registered by Marvell, and used when the Marvell
+ * SoC is not the root complex, but an endpoint on the PCIe bus. It is
+ * therefore safe to re-use this PCI ID for our emulated PCI-to-PCI
+ * bridge.
+ */
+#define MARVELL_EMULATED_PCI_PCI_BRIDGE_ID 0x7846
+
+/* PCI configuration space of a PCI-to-PCI bridge */
+struct mvebu_sw_pci_bridge {
+	u16 vendor;
+	u16 device;
+	u16 command;
+	u16 status;
+	u16 class;
+	u8 interface;
+	u8 revision;
+	u8 bist;
+	u8 header_type;
+	u8 latency_timer;
+	u8 cache_line_size;
+	u32 bar[2];
+	u8 primary_bus;
+	u8 secondary_bus;
+	u8 subordinate_bus;
+	u8 secondary_latency_timer;
+	u8 iobase;
+	u8 iolimit;
+	u16 secondary_status;
+	u16 membase;
+	u16 memlimit;
+	u16 prefmembase;
+	u16 prefmemlimit;
+	u32 prefbaseupper;
+	u32 preflimitupper;
+	u16 iobaseupper;
+	u16 iolimitupper;
+	u8 cappointer;
+	u8 reserved1;
+	u16 reserved2;
+	u32 romaddr;
+	u8 intline;
+	u8 intpin;
+	u16 bridgectrl;
+};
+
+struct mvebu_pcie_port;
+
+/* Structure representing all PCIe interfaces */
+struct mvebu_pcie {
+	struct platform_device *pdev;
+	struct mvebu_pcie_port *ports;
+	struct resource io;
+	struct resource realio;
+	struct resource mem;
+	struct resource busn;
+	int nports;
+};
+
+/* Structure representing one PCIe interface */
+struct mvebu_pcie_port {
+	void __iomem *base;
+	spinlock_t conf_lock;
+	int haslink;
+	u32 port;
+	u32 lane;
+	int devfn;
+	struct clk *clk;
+	struct mvebu_sw_pci_bridge bridge;
+	struct device_node *dn;
+	struct mvebu_pcie *pcie;
+};
+
+static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port *port,
+				       int enable)
+{
+	unsigned long iobase, iolimit;
+
+	if (port->bridge.iolimit < port->bridge.iobase)
+		return;
+
+	iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
+		(port->bridge.iolimitupper << 16);
+	iobase = ((port->bridge.iobase & 0xF0) << 8) |
+		(port->bridge.iobaseupper << 16);
+
+	if (enable) {
+		unsigned long physbase = port->pcie->io.start + iobase;
+		armada_370_xp_alloc_pcie_window(port->port, port->lane,
+						physbase, iobase,
+						iolimit-iobase,
+						IORESOURCE_IO);
+		pci_ioremap_io(iobase, physbase);
+	}
+	else
+		armada_370_xp_free_pcie_window(iobase);
+}
+
+static void mvebu_pcie_setup_mem_window(struct mvebu_pcie_port *port,
+					int enable)
+{
+	unsigned long membase, memlimit;
+
+	if (port->bridge.memlimit < port->bridge.membase)
+		return;
+
+	membase = ((port->bridge.membase & 0xFFF0) << 16);
+	memlimit = ((port->bridge.memlimit & 0xFFF0) << 16) | 0xFFFFF;
+
+	if (enable)
+		armada_370_xp_alloc_pcie_window(port->port, port->lane,
+						membase, ORION_ADDR_MAP_NO_REMAP,
+						memlimit-membase,
+						IORESOURCE_MEM);
+	else
+		armada_370_xp_free_pcie_window(membase);
+}
+
+static void mvebu_handle_pcie_command(struct mvebu_pcie_port *port, u16 old,
+				      u16 new)
+{
+	/* Enabling an I/O window ? */
+	if (!(old & PCI_COMMAND_IO) && (new & PCI_COMMAND_IO))
+		mvebu_pcie_setup_io_window(port, 1);
+
+	/* Disabling an I/O window ? */
+	if ((old & PCI_COMMAND_IO) && !(new & PCI_COMMAND_IO))
+		mvebu_pcie_setup_io_window(port, 0);
+
+	/* Enabling a memory window ? */
+	if (!(old & PCI_COMMAND_MEMORY) && (new & PCI_COMMAND_MEMORY))
+		mvebu_pcie_setup_mem_window(port, 1);
+
+	/* Disabling a memory window ? */
+	if ((old & PCI_COMMAND_MEMORY) && !(new & PCI_COMMAND_MEMORY))
+		mvebu_pcie_setup_mem_window(port, 0);
+}
+
+/*
+ * Initialize the configuration space of the PCI-to-PCI bridge
+ * associated with the given PCIe interface.
+ */
+static void mvebu_sw_pci_bridge_init(struct mvebu_pcie_port *port)
+{
+	struct mvebu_sw_pci_bridge *bridge = &port->bridge;
+
+	memset(bridge, 0, sizeof(struct mvebu_sw_pci_bridge));
+
+	bridge->status = PCI_STATUS_CAP_LIST;
+	bridge->class = PCI_CLASS_BRIDGE_PCI;
+	bridge->vendor = PCI_VENDOR_ID_MARVELL;
+	bridge->device = MARVELL_EMULATED_PCI_PCI_BRIDGE_ID;
+	bridge->header_type = PCI_HEADER_TYPE_BRIDGE;
+	bridge->cache_line_size = 0x10;
+
+	/* We support 32 bits I/O addressing */
+	bridge->iobase = PCI_IO_RANGE_TYPE_32;
+	bridge->iolimit = PCI_IO_RANGE_TYPE_32;
+}
+
+/*
+ * Read the configuration space of the PCI-to-PCI bridge associated to
+ * the given PCIe interface.
+ */
+static int mvebu_sw_pci_bridge_read(struct mvebu_pcie_port *port,
+				  unsigned int where, int size, u32 *value)
+{
+	struct mvebu_sw_pci_bridge *bridge = &port->bridge;
+
+	switch (where & ~3) {
+	case PCI_VENDOR_ID:
+		*value = bridge->device << 16 | bridge->vendor;
+		break;
+
+	case PCI_COMMAND:
+		*value = bridge->status << 16 | bridge->command;
+		break;
+
+	case PCI_CLASS_REVISION:
+		*value = bridge->class << 16 | bridge->interface << 8 |
+			 bridge->revision;
+		break;
+
+	case PCI_CACHE_LINE_SIZE:
+		*value = bridge->bist << 24 | bridge->header_type << 16 |
+			 bridge->latency_timer << 8 | bridge->cache_line_size;
+		break;
+
+	case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_1:
+		*value = bridge->bar[((where & ~3) - PCI_BASE_ADDRESS_0) / 4];
+		break;
+
+	case PCI_PRIMARY_BUS:
+		*value = (bridge->secondary_latency_timer << 24 |
+			  bridge->subordinate_bus         << 16 |
+			  bridge->secondary_bus           <<  8 |
+			  bridge->primary_bus);
+		break;
+
+	case PCI_IO_BASE:
+		*value = (bridge->secondary_status << 16 |
+			  bridge->iolimit          <<  8 |
+			  bridge->iobase);
+		break;
+
+	case PCI_MEMORY_BASE:
+		*value = (bridge->memlimit << 16 | bridge->membase);
+		break;
+
+	case PCI_PREF_MEMORY_BASE:
+		*value = (bridge->prefmemlimit << 16 | bridge->prefmembase);
+		break;
+
+	case PCI_PREF_BASE_UPPER32:
+		*value = bridge->prefbaseupper;
+		break;
+
+	case PCI_PREF_LIMIT_UPPER32:
+		*value = bridge->preflimitupper;
+		break;
+
+	case PCI_IO_BASE_UPPER16:
+		*value = (bridge->iolimitupper << 16 | bridge->iobaseupper);
+		break;
+
+	case PCI_ROM_ADDRESS1:
+		*value = 0;
+		break;
+
+	default:
+		*value = 0xffffffff;
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+	}
+
+	if (size == 2)
+		*value = (*value >> (8 * (where & 3))) & 0xffff;
+	else if (size == 1)
+		*value = (*value >> (8 * (where & 3))) & 0xff;
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+/* Write to the PCI-to-PCI bridge configuration space */
+static int mvebu_sw_pci_bridge_write(struct mvebu_pcie_port *port,
+				     unsigned int where, int size, u32 value)
+{
+	struct mvebu_sw_pci_bridge *bridge = &port->bridge;
+	u32 mask, reg;
+	int err;
+
+	if (size == 4)
+		mask = 0x0;
+	else if (size == 2)
+		mask = ~(0xffff << ((where & 3) * 8));
+	else if (size == 1)
+		mask = ~(0xff << ((where & 3) * 8));
+	else
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+
+	err = mvebu_sw_pci_bridge_read(port, where & ~3, 4, &reg);
+	if (err)
+		return err;
+
+	value = (reg & mask) | value << ((where & 3) * 8);
+
+	switch (where & ~3) {
+	case PCI_COMMAND:
+		mvebu_handle_pcie_command(port, bridge->command,
+					  value & 0xffff);
+		bridge->command = value & 0xffff;
+		bridge->status = value >> 16;
+		break;
+
+	case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_1:
+		bridge->bar[((where & ~3) - PCI_BASE_ADDRESS_0) / 4] = value;
+		break;
+
+	case PCI_IO_BASE:
+		/*
+		 * We also keep bit 1 set, it is a read-only bit that
+		 * indicates we support 32 bits addressing for the
+		 * I/O
+		 */
+		bridge->iobase = (value & 0xff) | PCI_IO_RANGE_TYPE_32;
+		bridge->iolimit = ((value >> 8) & 0xff) | PCI_IO_RANGE_TYPE_32;
+		bridge->secondary_status = value >> 16;
+		break;
+
+	case PCI_MEMORY_BASE:
+		bridge->membase = value & 0xffff;
+		bridge->memlimit = value >> 16;
+		break;
+
+	case PCI_PREF_MEMORY_BASE:
+		bridge->prefmembase = value & 0xffff;
+		bridge->prefmemlimit = value >> 16;
+		break;
+
+	case PCI_PREF_BASE_UPPER32:
+		bridge->prefbaseupper = value;
+		break;
+
+	case PCI_PREF_LIMIT_UPPER32:
+		bridge->preflimitupper = value;
+		break;
+
+	case PCI_IO_BASE_UPPER16:
+		bridge->iobaseupper = value & 0xffff;
+		bridge->iolimitupper = value >> 16;
+		break;
+
+	case PCI_PRIMARY_BUS:
+		bridge->primary_bus             = value & 0xff;
+		bridge->secondary_bus           = (value >> 8) & 0xff;
+		bridge->subordinate_bus         = (value >> 16) & 0xff;
+		bridge->secondary_latency_timer = (value >> 24) & 0xff;
+		orion_pcie_set_local_bus_nr(port->base, bridge->secondary_bus);
+		break;
+
+	default:
+		break;
+	}
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+static inline struct mvebu_pcie *sys_to_pcie(struct pci_sys_data *sys)
+{
+	return sys->private_data;
+}
+
+/* Find the PCIe interface that corresponds to the given bus */
+static struct mvebu_pcie_port *mvebu_find_port_from_bus(struct mvebu_pcie *pcie,
+							int bus)
+{
+	int porti;
+
+	for (porti = 0; porti < pcie->nports; porti++)
+		if (pcie->ports[porti].bridge.secondary_bus == bus)
+			return &pcie->ports[porti];
+
+	return NULL;
+}
+
+/* PCI configuration space write function */
+static int mvebu_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
+			      int where, int size, u32 val)
+{
+	struct mvebu_pcie *pcie = sys_to_pcie(bus->sysdata);
+
+	if (bus->number != 0) {
+		/*
+		 * Accessing a real PCIe interface.
+		 */
+		struct mvebu_pcie_port *port;
+		unsigned long flags;
+		int ret;
+
+		port = mvebu_find_port_from_bus(pcie, bus->number);
+		if (!port)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+
+		if (!port->haslink)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+
+		if (PCI_SLOT(devfn) != 0)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+
+		spin_lock_irqsave(&port->conf_lock, flags);
+		ret = orion_pcie_wr_conf(port->base, bus,
+					 PCI_DEVFN(1, PCI_FUNC(devfn)),
+					 where, size, val);
+		spin_unlock_irqrestore(&port->conf_lock, flags);
+
+		return ret;
+	} else {
+		/*
+		 * Access the emulated PCI-to-PCI bridges.
+		 */
+		if (PCI_SLOT(devfn) >= 1 &&
+		    PCI_SLOT(devfn) <= pcie->nports) {
+			struct mvebu_pcie_port *port;
+			int porti = PCI_SLOT(devfn) - 1;
+			port = &pcie->ports[porti];
+			return mvebu_sw_pci_bridge_write(port, where, size, val);
+		} else {
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		}
+	}
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+/* PCI configuration space read function */
+static int mvebu_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
+			      int size, u32 *val)
+{
+	struct mvebu_pcie *pcie = sys_to_pcie(bus->sysdata);
+
+	if (bus->number != 0) {
+		/*
+		 * Accessing a real PCIe interface.
+		 */
+		struct mvebu_pcie_port *port;
+		unsigned long flags;
+		int ret;
+
+		port = mvebu_find_port_from_bus(pcie, bus->number);
+		if (!port) {
+			*val = 0xffffffff;
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		}
+
+		if (!port->haslink || PCI_SLOT(devfn) != 0) {
+			*val = 0xffffffff;
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		}
+
+		spin_lock_irqsave(&port->conf_lock, flags);
+		ret = orion_pcie_rd_conf(port->base, bus,
+					 PCI_DEVFN(1, PCI_FUNC(devfn)),
+					 where, size, val);
+		spin_unlock_irqrestore(&port->conf_lock, flags);
+
+		return ret;
+	} else {
+		/*
+		 * Access the emulated PCI-to-PCI bridges.
+		 */
+		if (PCI_SLOT(devfn) >= 1 &&
+		    PCI_SLOT(devfn) <= pcie->nports) {
+			struct mvebu_pcie_port *port;
+			int porti = PCI_SLOT(devfn) - 1;
+			port = &pcie->ports[porti];
+			return mvebu_sw_pci_bridge_read(port, where, size, val);
+		} else {
+			*val = 0xffffffff;
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		}
+	}
+}
+
+static struct pci_ops mvebu_pcie_ops = {
+	.read = mvebu_pcie_rd_conf,
+	.write = mvebu_pcie_wr_conf,
+};
+
+static int __init mvebu_pcie_setup(int nr, struct pci_sys_data *sys)
+{
+	struct mvebu_pcie *pcie = sys_to_pcie(sys);
+	int i;
+
+	pci_add_resource_offset(&sys->resources, &pcie->realio, sys->io_offset);
+	pci_add_resource_offset(&sys->resources, &pcie->mem, sys->mem_offset);
+	pci_add_resource(&sys->resources, &pcie->busn);
+
+	for (i = 0; i < pcie->nports; i++) {
+		struct mvebu_pcie_port *port = &pcie->ports[i];
+		orion_pcie_setup(port->base);
+	}
+
+	return 1;
+}
+
+static int __init mvebu_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
+{
+	struct of_irq oirq;
+	int ret;
+
+	ret = of_irq_map_pci(dev, &oirq);
+	if (ret) {
+		return ret;
+	}
+
+	return irq_create_of_mapping(oirq.controller, oirq.specifier,
+				     oirq.size);
+}
+
+static struct pci_bus *mvebu_pcie_scan_bus(int nr, struct pci_sys_data *sys)
+{
+	struct mvebu_pcie *pcie = sys_to_pcie(sys);
+	return pci_scan_root_bus(&pcie->pdev->dev, sys->busnr,
+				 &mvebu_pcie_ops, sys, &sys->resources);
+}
+
+resource_size_t mvebu_pcie_align_resource(struct pci_dev *dev,
+					  const struct resource *res,
+					  resource_size_t start,
+					  resource_size_t size,
+					  resource_size_t align)
+{
+	if (dev->bus->number != 0)
+		return start;
+
+	/*
+	 * On the PCI-to-PCI bridge side, the I/O windows must have at
+	 * least a 64 KB size and be aligned on their size, and the
+	 * memory windows must have at least a 1 MB size and be
+	 * aligned on their size
+	 */
+	if (res->flags & IORESOURCE_IO)
+		return round_up(start, max((resource_size_t)SZ_64K, size));
+	else if (res->flags & IORESOURCE_MEM)
+		return round_up(start, max((resource_size_t)SZ_1M, size));
+	else
+		return start;
+}
+
+static void mvebu_pcie_enable(struct mvebu_pcie *pcie)
+{
+	struct hw_pci hw;
+
+	memset(&hw, 0, sizeof(hw));
+
+	hw.nr_controllers = 1;
+	hw.private_data   = (void **)&pcie;
+	hw.setup          = mvebu_pcie_setup;
+	hw.scan           = mvebu_pcie_scan_bus;
+	hw.map_irq        = mvebu_pcie_map_irq;
+	hw.ops            = &mvebu_pcie_ops;
+	hw.align_resource = mvebu_pcie_align_resource;
+
+	pci_common_init(&hw);
+}
+
+static int __init mvebu_pcie_probe(struct platform_device *pdev)
+{
+	struct mvebu_pcie *pcie;
+	struct device_node *np = pdev->dev.of_node;
+	struct device_node *child;
+	const __be32 *range = NULL;
+	struct resource res;
+	int i, ret;
+
+	pcie = devm_kzalloc(&pdev->dev, sizeof(struct mvebu_pcie),
+			    GFP_KERNEL);
+	if (!pcie)
+		return -ENOMEM;
+
+	pcie->pdev = pdev;
+
+	/* Get the I/O and memory ranges from DT */
+	while ((range = of_pci_process_ranges(np, &res, range)) != NULL) {
+		if (resource_type(&res) == IORESOURCE_IO) {
+			memcpy(&pcie->io, &res, sizeof(res));
+			memcpy(&pcie->realio, &res, sizeof(res));
+			pcie->io.name = "I/O";
+			pcie->realio.start &= 0xFFFFF;
+			pcie->realio.end   &= 0xFFFFF;
+		}
+		if (resource_type(&res) == IORESOURCE_MEM) {
+			memcpy(&pcie->mem, &res, sizeof(res));
+			pcie->mem.name = "MEM";
+		}
+	}
+
+	/* Get the bus range */
+	ret = of_pci_parse_bus_range(np, &pcie->busn);
+	if (ret) {
+		dev_err(&pdev->dev, "failed to parse bus-range property: %d\n",
+			ret);
+		return ret;
+	}
+
+	for_each_child_of_node(pdev->dev.of_node, child) {
+		if (!of_device_is_available(child))
+			continue;
+		pcie->nports++;
+	}
+
+	pcie->ports = devm_kzalloc(&pdev->dev, pcie->nports *
+				   sizeof(struct mvebu_pcie_port),
+				   GFP_KERNEL);
+	if (!pcie->ports)
+		return -ENOMEM;
+
+	i = 0;
+	for_each_child_of_node(pdev->dev.of_node, child) {
+		struct mvebu_pcie_port *port = &pcie->ports[i];
+
+		if (!of_device_is_available(child))
+			continue;
+
+		port->pcie = pcie;
+
+		if (of_property_read_u32(child, "marvell,pcie-port",
+					 &port->port)) {
+			dev_warn(&pdev->dev,
+				 "ignoring PCIe DT node, missing pcie-port property\n");
+			continue;
+		}
+
+		if (of_property_read_u32(child, "marvell,pcie-lane",
+					 &port->lane))
+			port->lane = 0;
+
+		port->devfn = of_pci_get_devfn(child);
+		if (port->devfn < 0)
+			continue;
+
+		port->base = of_iomap(child, 0);
+		if (!port->base) {
+			dev_err(&pdev->dev, "PCIe%d.%d: cannot map registers\n",
+				port->port, port->lane);
+			continue;
+		}
+
+		if (orion_pcie_link_up(port->base)) {
+			port->haslink = 1;
+			dev_info(&pdev->dev, "PCIe%d.%d: link up\n",
+				 port->port, port->lane);
+		} else {
+			port->haslink = 0;
+			dev_info(&pdev->dev, "PCIe%d.%d: link down\n",
+				 port->port, port->lane);
+		}
+
+		port->clk = of_clk_get_by_name(child, NULL);
+		if (!port->clk) {
+			dev_err(&pdev->dev, "PCIe%d.%d: cannot get clock\n",
+			       port->port, port->lane);
+			iounmap(port->base);
+			port->haslink = 0;
+			continue;
+		}
+
+		port->dn = child;
+
+		clk_prepare_enable(port->clk);
+		spin_lock_init(&port->conf_lock);
+
+		mvebu_sw_pci_bridge_init(port);
+
+		i++;
+	}
+
+	mvebu_pcie_enable(pcie);
+
+	return 0;
+}
+
+static const struct of_device_id mvebu_pcie_of_match_table[] = {
+	{ .compatible = "marvell,armada-370-xp-pcie", },
+	{},
+};
+MODULE_DEVICE_TABLE(of, mvebu_pcie_of_match_table);
+
+static struct platform_driver mvebu_pcie_driver = {
+	.driver = {
+		.owner = THIS_MODULE,
+		.name = "mvebu-pcie",
+		.of_match_table =
+		   of_match_ptr(mvebu_pcie_of_match_table),
+	},
+};
+
+static int mvebu_pcie_init(void)
+{
+	return platform_driver_probe(&mvebu_pcie_driver,
+				     mvebu_pcie_probe);
+}
+
+subsys_initcall(mvebu_pcie_init);
+
+MODULE_AUTHOR("Thomas Petazzoni <thomas.petazzoni@free-electrons.com>");
+MODULE_DESCRIPTION("Marvell EBU PCIe driver");
+MODULE_LICENSE("GPLv2");
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

This driver implements the support for the PCIe interfaces on the
Marvell Armada 370/XP ARM SoCs. In the future, it might be extended to
cover earlier families of Marvell SoCs, such as Dove, Orion and
Kirkwood.

The driver implements the hw_pci operations needed by the core ARM PCI
code to setup PCI devices and get their corresponding IRQs, and the
pci_ops operations that are used by the PCI core to read/write the
configuration space of PCI devices.

Since the PCIe interfaces of Marvell SoCs are completely separate and
not linked together in a bus, this driver sets up an emulated PCI host
bridge, with one PCI-to-PCI bridge as child for each hardware PCIe
interface.

In addition, this driver enumerates the different PCIe slots, and for
those having a device plugged in, it sets up the necessary address
decoding windows, using the new armada_370_xp_alloc_pcie_window()
function from mach-mvebu/addr-map.c.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 .../devicetree/bindings/pci/mvebu-pci.txt          |  174 +++++
 drivers/pci/host/Kconfig                           |    6 +
 drivers/pci/host/Makefile                          |    4 +
 drivers/pci/host/pci-mvebu.c                       |  688 ++++++++++++++++++++
 4 files changed, 872 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/pci/mvebu-pci.txt
 create mode 100644 drivers/pci/host/Makefile
 create mode 100644 drivers/pci/host/pci-mvebu.c

diff --git a/Documentation/devicetree/bindings/pci/mvebu-pci.txt b/Documentation/devicetree/bindings/pci/mvebu-pci.txt
new file mode 100644
index 0000000..bd621c8
--- /dev/null
+++ b/Documentation/devicetree/bindings/pci/mvebu-pci.txt
@@ -0,0 +1,174 @@
+* Marvell EBU PCIe interfaces
+
+Mandatory properties:
+- compatible: for now only "marvell,armada-370-xp-pcie" is supported
+- #address-cells, set to <3>
+- #size-cells, set to <2>
+- #interrupt-cells, set to <1>
+- bus-range: PCI bus numbers covered
+- ranges: standard PCI-style address ranges, describing the PCIe
+  registers for each PCIe interface, and then ranges for the PCI
+  memory and I/O regions.
+- interrupt-map-mask and interrupt-map are standard PCI Device Tree
+  properties to describe the interrupts associated to each PCI
+  interface.
+
+In addition, the Device Tree node must have sub-nodes describing each
+PCIe interface, having the following mandatory properties:
+- reg: the address and size of the PCIe registers (translated
+  addresses according to the ranges property of the parent)
+- clocks: the clock associated to this PCIe interface
+- marvell,pcie-port: the physical PCIe port number
+- status: either "disabled" or "okay"
+
+and the following optional properties:
+- marvell,pcie-lane: the physical PCIe lane number, for ports having
+  multiple lanes. If this property is not found, we assume that the
+  value is 0.
+
+Example:
+
+pcie-controller {
+	compatible = "marvell,armada-370-xp-pcie";
+	status = "disabled";
+
+	#address-cells = <3>;
+	#size-cells = <2>;
+
+	bus-range = <0x00 0xff>;
+
+	ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+	          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
+	          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
+	          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
+	          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
+		  0x00002800 0 0xd0080000 0xd0080000 0 0x00002000   /* port 1.0 registers */
+	          0x00005000 0 0xd0082000 0xd0082000 0 0x00002000   /* port 3.0 registers */
+		  0x00003000 0 0xd0084000 0xd0084000 0 0x00002000   /* port 1.1 registers */
+		  0x00003800 0 0xd0088000 0xd0088000 0 0x00002000   /* port 1.2 registers */
+		  0x00004000 0 0xd008C000 0xd008C000 0 0x00002000   /* port 1.3 registers */
+		  0x81000000 0 0	  0xc0000000 0 0x00100000   /* downstream I/O */
+		  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+	#interrupt-cells = <1>;
+	interrupt-map-mask = <0xf800 0 0 1>;
+	interrupt-map = <0x0800 0 0 1 &mpic 58
+		         0x1000 0 0 1 &mpic 59
+			 0x1800 0 0 1 &mpic 60
+			 0x2000 0 0 1 &mpic 61
+			 0x2800 0 0 1 &mpic 62
+		         0x3000 0 0 1 &mpic 63
+			 0x3800 0 0 1 &mpic 64
+			 0x4000 0 0 1 &mpic 65
+			 0x4800 0 0 1 &mpic 99
+			 0x5000 0 0 1 &mpic 103>;
+
+	pcie at 0,0 {
+		device_type = "pciex";
+		reg = <0x0800 0 0xd0040000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <0>;
+		marvell,pcie-lane = <0>;
+		clocks = <&gateclk 5>;
+		status = "disabled";
+	};
+
+	pcie at 0,1 {
+		device_type = "pciex";
+		reg = <0x1000 0 0xd0044000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <0>;
+		marvell,pcie-lane = <1>;
+		clocks = <&gateclk 6>;
+		status = "disabled";
+	};
+
+	pcie at 0,2 {
+		device_type = "pciex";
+		reg = <0x1800 0 0xd0048000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <0>;
+		marvell,pcie-lane = <2>;
+		clocks = <&gateclk 7>;
+		status = "disabled";
+	};
+
+	pcie at 0,3 {
+		device_type = "pciex";
+		reg = <0x2000 0 0xd004C000 0 0xC000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <0>;
+		marvell,pcie-lane = <3>;
+		clocks = <&gateclk 8>;
+		status = "disabled";
+	};
+
+	pcie at 1,0 {
+		device_type = "pciex";
+		reg = <0x2800 0 0xd0080000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <1>;
+		marvell,pcie-lane = <0>;
+		clocks = <&gateclk 9>;
+		status = "disabled";
+	};
+
+	pcie at 1,1 {
+		device_type = "pciex";
+		reg = <0x3000 0 0xd0084000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <1>;
+		marvell,pcie-lane = <1>;
+		clocks = <&gateclk 10>;
+		status = "disabled";
+	};
+
+	pcie at 1,2 {
+		device_type = "pciex";
+		reg = <0x3800 0 0xd0088000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <1>;
+		marvell,pcie-lane = <2>;
+		clocks = <&gateclk 11>;
+		status = "disabled";
+	};
+
+	pcie at 1,3 {
+		device_type = "pciex";
+		reg = <0x4000 0 0xd008C000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <1>;
+		marvell,pcie-lane = <3>;
+		clocks = <&gateclk 12>;
+		status = "disabled";
+	};
+	pcie at 2,0 {
+		device_type = "pciex";
+		reg = <0x4800 0 0xd0042000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <2>;
+		marvell,pcie-lane = <0>;
+		clocks = <&gateclk 26>;
+		status = "disabled";
+	};
+
+	pcie at 3,0 {
+		device_type = "pciex";
+		reg = <0x5000 0 0xd0082000 0 0x2000>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+		marvell,pcie-port = <3>;
+		marvell,pcie-lane = <0>;
+		clocks = <&gateclk 27>;
+		status = "disabled";
+	};
+};
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index cc3a1af..03e15e7 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -1,4 +1,10 @@
 menu "PCI host controller drivers"
 	depends on PCI
 
+config PCI_MVEBU
+	bool "Marvell EBU PCIe controller"
+	depends on ARCH_MVEBU
+	select PCI_SW_HOST_BRIDGE
+	select PCI_SW_PCI_PCI_BRIDGE
+
 endmenu
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
new file mode 100644
index 0000000..3ad563f
--- /dev/null
+++ b/drivers/pci/host/Makefile
@@ -0,0 +1,4 @@
+obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
+CFLAGS_pci-mvebu.o += \
+	-I$(srctree)/arch/arm/plat-orion/include \
+	-I$(srctree)/arch/arm/mach-mvebu/include
diff --git a/drivers/pci/host/pci-mvebu.c b/drivers/pci/host/pci-mvebu.c
new file mode 100644
index 0000000..82d13da
--- /dev/null
+++ b/drivers/pci/host/pci-mvebu.c
@@ -0,0 +1,688 @@
+/*
+ * PCIe driver for Marvell Armada 370 and Armada XP SoCs
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2.  This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/platform_device.h>
+#include <linux/of_address.h>
+#include <linux/of_pci.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <plat/pcie.h>
+#include <mach/addr-map.h>
+
+/*
+ * This product ID is registered by Marvell, and used when the Marvell
+ * SoC is not the root complex, but an endpoint on the PCIe bus. It is
+ * therefore safe to re-use this PCI ID for our emulated PCI-to-PCI
+ * bridge.
+ */
+#define MARVELL_EMULATED_PCI_PCI_BRIDGE_ID 0x7846
+
+/* PCI configuration space of a PCI-to-PCI bridge */
+struct mvebu_sw_pci_bridge {
+	u16 vendor;
+	u16 device;
+	u16 command;
+	u16 status;
+	u16 class;
+	u8 interface;
+	u8 revision;
+	u8 bist;
+	u8 header_type;
+	u8 latency_timer;
+	u8 cache_line_size;
+	u32 bar[2];
+	u8 primary_bus;
+	u8 secondary_bus;
+	u8 subordinate_bus;
+	u8 secondary_latency_timer;
+	u8 iobase;
+	u8 iolimit;
+	u16 secondary_status;
+	u16 membase;
+	u16 memlimit;
+	u16 prefmembase;
+	u16 prefmemlimit;
+	u32 prefbaseupper;
+	u32 preflimitupper;
+	u16 iobaseupper;
+	u16 iolimitupper;
+	u8 cappointer;
+	u8 reserved1;
+	u16 reserved2;
+	u32 romaddr;
+	u8 intline;
+	u8 intpin;
+	u16 bridgectrl;
+};
+
+struct mvebu_pcie_port;
+
+/* Structure representing all PCIe interfaces */
+struct mvebu_pcie {
+	struct platform_device *pdev;
+	struct mvebu_pcie_port *ports;
+	struct resource io;
+	struct resource realio;
+	struct resource mem;
+	struct resource busn;
+	int nports;
+};
+
+/* Structure representing one PCIe interface */
+struct mvebu_pcie_port {
+	void __iomem *base;
+	spinlock_t conf_lock;
+	int haslink;
+	u32 port;
+	u32 lane;
+	int devfn;
+	struct clk *clk;
+	struct mvebu_sw_pci_bridge bridge;
+	struct device_node *dn;
+	struct mvebu_pcie *pcie;
+};
+
+static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port *port,
+				       int enable)
+{
+	unsigned long iobase, iolimit;
+
+	if (port->bridge.iolimit < port->bridge.iobase)
+		return;
+
+	iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
+		(port->bridge.iolimitupper << 16);
+	iobase = ((port->bridge.iobase & 0xF0) << 8) |
+		(port->bridge.iobaseupper << 16);
+
+	if (enable) {
+		unsigned long physbase = port->pcie->io.start + iobase;
+		armada_370_xp_alloc_pcie_window(port->port, port->lane,
+						physbase, iobase,
+						iolimit-iobase,
+						IORESOURCE_IO);
+		pci_ioremap_io(iobase, physbase);
+	}
+	else
+		armada_370_xp_free_pcie_window(iobase);
+}
+
+static void mvebu_pcie_setup_mem_window(struct mvebu_pcie_port *port,
+					int enable)
+{
+	unsigned long membase, memlimit;
+
+	if (port->bridge.memlimit < port->bridge.membase)
+		return;
+
+	membase = ((port->bridge.membase & 0xFFF0) << 16);
+	memlimit = ((port->bridge.memlimit & 0xFFF0) << 16) | 0xFFFFF;
+
+	if (enable)
+		armada_370_xp_alloc_pcie_window(port->port, port->lane,
+						membase, ORION_ADDR_MAP_NO_REMAP,
+						memlimit-membase,
+						IORESOURCE_MEM);
+	else
+		armada_370_xp_free_pcie_window(membase);
+}
+
+static void mvebu_handle_pcie_command(struct mvebu_pcie_port *port, u16 old,
+				      u16 new)
+{
+	/* Enabling an I/O window ? */
+	if (!(old & PCI_COMMAND_IO) && (new & PCI_COMMAND_IO))
+		mvebu_pcie_setup_io_window(port, 1);
+
+	/* Disabling an I/O window ? */
+	if ((old & PCI_COMMAND_IO) && !(new & PCI_COMMAND_IO))
+		mvebu_pcie_setup_io_window(port, 0);
+
+	/* Enabling a memory window ? */
+	if (!(old & PCI_COMMAND_MEMORY) && (new & PCI_COMMAND_MEMORY))
+		mvebu_pcie_setup_mem_window(port, 1);
+
+	/* Disabling a memory window ? */
+	if ((old & PCI_COMMAND_MEMORY) && !(new & PCI_COMMAND_MEMORY))
+		mvebu_pcie_setup_mem_window(port, 0);
+}
+
+/*
+ * Initialize the configuration space of the PCI-to-PCI bridge
+ * associated with the given PCIe interface.
+ */
+static void mvebu_sw_pci_bridge_init(struct mvebu_pcie_port *port)
+{
+	struct mvebu_sw_pci_bridge *bridge = &port->bridge;
+
+	memset(bridge, 0, sizeof(struct mvebu_sw_pci_bridge));
+
+	bridge->status = PCI_STATUS_CAP_LIST;
+	bridge->class = PCI_CLASS_BRIDGE_PCI;
+	bridge->vendor = PCI_VENDOR_ID_MARVELL;
+	bridge->device = MARVELL_EMULATED_PCI_PCI_BRIDGE_ID;
+	bridge->header_type = PCI_HEADER_TYPE_BRIDGE;
+	bridge->cache_line_size = 0x10;
+
+	/* We support 32 bits I/O addressing */
+	bridge->iobase = PCI_IO_RANGE_TYPE_32;
+	bridge->iolimit = PCI_IO_RANGE_TYPE_32;
+}
+
+/*
+ * Read the configuration space of the PCI-to-PCI bridge associated to
+ * the given PCIe interface.
+ */
+static int mvebu_sw_pci_bridge_read(struct mvebu_pcie_port *port,
+				  unsigned int where, int size, u32 *value)
+{
+	struct mvebu_sw_pci_bridge *bridge = &port->bridge;
+
+	switch (where & ~3) {
+	case PCI_VENDOR_ID:
+		*value = bridge->device << 16 | bridge->vendor;
+		break;
+
+	case PCI_COMMAND:
+		*value = bridge->status << 16 | bridge->command;
+		break;
+
+	case PCI_CLASS_REVISION:
+		*value = bridge->class << 16 | bridge->interface << 8 |
+			 bridge->revision;
+		break;
+
+	case PCI_CACHE_LINE_SIZE:
+		*value = bridge->bist << 24 | bridge->header_type << 16 |
+			 bridge->latency_timer << 8 | bridge->cache_line_size;
+		break;
+
+	case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_1:
+		*value = bridge->bar[((where & ~3) - PCI_BASE_ADDRESS_0) / 4];
+		break;
+
+	case PCI_PRIMARY_BUS:
+		*value = (bridge->secondary_latency_timer << 24 |
+			  bridge->subordinate_bus         << 16 |
+			  bridge->secondary_bus           <<  8 |
+			  bridge->primary_bus);
+		break;
+
+	case PCI_IO_BASE:
+		*value = (bridge->secondary_status << 16 |
+			  bridge->iolimit          <<  8 |
+			  bridge->iobase);
+		break;
+
+	case PCI_MEMORY_BASE:
+		*value = (bridge->memlimit << 16 | bridge->membase);
+		break;
+
+	case PCI_PREF_MEMORY_BASE:
+		*value = (bridge->prefmemlimit << 16 | bridge->prefmembase);
+		break;
+
+	case PCI_PREF_BASE_UPPER32:
+		*value = bridge->prefbaseupper;
+		break;
+
+	case PCI_PREF_LIMIT_UPPER32:
+		*value = bridge->preflimitupper;
+		break;
+
+	case PCI_IO_BASE_UPPER16:
+		*value = (bridge->iolimitupper << 16 | bridge->iobaseupper);
+		break;
+
+	case PCI_ROM_ADDRESS1:
+		*value = 0;
+		break;
+
+	default:
+		*value = 0xffffffff;
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+	}
+
+	if (size == 2)
+		*value = (*value >> (8 * (where & 3))) & 0xffff;
+	else if (size == 1)
+		*value = (*value >> (8 * (where & 3))) & 0xff;
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+/* Write to the PCI-to-PCI bridge configuration space */
+static int mvebu_sw_pci_bridge_write(struct mvebu_pcie_port *port,
+				     unsigned int where, int size, u32 value)
+{
+	struct mvebu_sw_pci_bridge *bridge = &port->bridge;
+	u32 mask, reg;
+	int err;
+
+	if (size == 4)
+		mask = 0x0;
+	else if (size == 2)
+		mask = ~(0xffff << ((where & 3) * 8));
+	else if (size == 1)
+		mask = ~(0xff << ((where & 3) * 8));
+	else
+		return PCIBIOS_BAD_REGISTER_NUMBER;
+
+	err = mvebu_sw_pci_bridge_read(port, where & ~3, 4, &reg);
+	if (err)
+		return err;
+
+	value = (reg & mask) | value << ((where & 3) * 8);
+
+	switch (where & ~3) {
+	case PCI_COMMAND:
+		mvebu_handle_pcie_command(port, bridge->command,
+					  value & 0xffff);
+		bridge->command = value & 0xffff;
+		bridge->status = value >> 16;
+		break;
+
+	case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_1:
+		bridge->bar[((where & ~3) - PCI_BASE_ADDRESS_0) / 4] = value;
+		break;
+
+	case PCI_IO_BASE:
+		/*
+		 * We also keep bit 1 set, it is a read-only bit that
+		 * indicates we support 32 bits addressing for the
+		 * I/O
+		 */
+		bridge->iobase = (value & 0xff) | PCI_IO_RANGE_TYPE_32;
+		bridge->iolimit = ((value >> 8) & 0xff) | PCI_IO_RANGE_TYPE_32;
+		bridge->secondary_status = value >> 16;
+		break;
+
+	case PCI_MEMORY_BASE:
+		bridge->membase = value & 0xffff;
+		bridge->memlimit = value >> 16;
+		break;
+
+	case PCI_PREF_MEMORY_BASE:
+		bridge->prefmembase = value & 0xffff;
+		bridge->prefmemlimit = value >> 16;
+		break;
+
+	case PCI_PREF_BASE_UPPER32:
+		bridge->prefbaseupper = value;
+		break;
+
+	case PCI_PREF_LIMIT_UPPER32:
+		bridge->preflimitupper = value;
+		break;
+
+	case PCI_IO_BASE_UPPER16:
+		bridge->iobaseupper = value & 0xffff;
+		bridge->iolimitupper = value >> 16;
+		break;
+
+	case PCI_PRIMARY_BUS:
+		bridge->primary_bus             = value & 0xff;
+		bridge->secondary_bus           = (value >> 8) & 0xff;
+		bridge->subordinate_bus         = (value >> 16) & 0xff;
+		bridge->secondary_latency_timer = (value >> 24) & 0xff;
+		orion_pcie_set_local_bus_nr(port->base, bridge->secondary_bus);
+		break;
+
+	default:
+		break;
+	}
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+static inline struct mvebu_pcie *sys_to_pcie(struct pci_sys_data *sys)
+{
+	return sys->private_data;
+}
+
+/* Find the PCIe interface that corresponds to the given bus */
+static struct mvebu_pcie_port *mvebu_find_port_from_bus(struct mvebu_pcie *pcie,
+							int bus)
+{
+	int porti;
+
+	for (porti = 0; porti < pcie->nports; porti++)
+		if (pcie->ports[porti].bridge.secondary_bus == bus)
+			return &pcie->ports[porti];
+
+	return NULL;
+}
+
+/* PCI configuration space write function */
+static int mvebu_pcie_wr_conf(struct pci_bus *bus, u32 devfn,
+			      int where, int size, u32 val)
+{
+	struct mvebu_pcie *pcie = sys_to_pcie(bus->sysdata);
+
+	if (bus->number != 0) {
+		/*
+		 * Accessing a real PCIe interface.
+		 */
+		struct mvebu_pcie_port *port;
+		unsigned long flags;
+		int ret;
+
+		port = mvebu_find_port_from_bus(pcie, bus->number);
+		if (!port)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+
+		if (!port->haslink)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+
+		if (PCI_SLOT(devfn) != 0)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+
+		spin_lock_irqsave(&port->conf_lock, flags);
+		ret = orion_pcie_wr_conf(port->base, bus,
+					 PCI_DEVFN(1, PCI_FUNC(devfn)),
+					 where, size, val);
+		spin_unlock_irqrestore(&port->conf_lock, flags);
+
+		return ret;
+	} else {
+		/*
+		 * Access the emulated PCI-to-PCI bridges.
+		 */
+		if (PCI_SLOT(devfn) >= 1 &&
+		    PCI_SLOT(devfn) <= pcie->nports) {
+			struct mvebu_pcie_port *port;
+			int porti = PCI_SLOT(devfn) - 1;
+			port = &pcie->ports[porti];
+			return mvebu_sw_pci_bridge_write(port, where, size, val);
+		} else {
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		}
+	}
+
+	return PCIBIOS_SUCCESSFUL;
+}
+
+/* PCI configuration space read function */
+static int mvebu_pcie_rd_conf(struct pci_bus *bus, u32 devfn, int where,
+			      int size, u32 *val)
+{
+	struct mvebu_pcie *pcie = sys_to_pcie(bus->sysdata);
+
+	if (bus->number != 0) {
+		/*
+		 * Accessing a real PCIe interface.
+		 */
+		struct mvebu_pcie_port *port;
+		unsigned long flags;
+		int ret;
+
+		port = mvebu_find_port_from_bus(pcie, bus->number);
+		if (!port) {
+			*val = 0xffffffff;
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		}
+
+		if (!port->haslink || PCI_SLOT(devfn) != 0) {
+			*val = 0xffffffff;
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		}
+
+		spin_lock_irqsave(&port->conf_lock, flags);
+		ret = orion_pcie_rd_conf(port->base, bus,
+					 PCI_DEVFN(1, PCI_FUNC(devfn)),
+					 where, size, val);
+		spin_unlock_irqrestore(&port->conf_lock, flags);
+
+		return ret;
+	} else {
+		/*
+		 * Access the emulated PCI-to-PCI bridges.
+		 */
+		if (PCI_SLOT(devfn) >= 1 &&
+		    PCI_SLOT(devfn) <= pcie->nports) {
+			struct mvebu_pcie_port *port;
+			int porti = PCI_SLOT(devfn) - 1;
+			port = &pcie->ports[porti];
+			return mvebu_sw_pci_bridge_read(port, where, size, val);
+		} else {
+			*val = 0xffffffff;
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		}
+	}
+}
+
+static struct pci_ops mvebu_pcie_ops = {
+	.read = mvebu_pcie_rd_conf,
+	.write = mvebu_pcie_wr_conf,
+};
+
+static int __init mvebu_pcie_setup(int nr, struct pci_sys_data *sys)
+{
+	struct mvebu_pcie *pcie = sys_to_pcie(sys);
+	int i;
+
+	pci_add_resource_offset(&sys->resources, &pcie->realio, sys->io_offset);
+	pci_add_resource_offset(&sys->resources, &pcie->mem, sys->mem_offset);
+	pci_add_resource(&sys->resources, &pcie->busn);
+
+	for (i = 0; i < pcie->nports; i++) {
+		struct mvebu_pcie_port *port = &pcie->ports[i];
+		orion_pcie_setup(port->base);
+	}
+
+	return 1;
+}
+
+static int __init mvebu_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
+{
+	struct of_irq oirq;
+	int ret;
+
+	ret = of_irq_map_pci(dev, &oirq);
+	if (ret) {
+		return ret;
+	}
+
+	return irq_create_of_mapping(oirq.controller, oirq.specifier,
+				     oirq.size);
+}
+
+static struct pci_bus *mvebu_pcie_scan_bus(int nr, struct pci_sys_data *sys)
+{
+	struct mvebu_pcie *pcie = sys_to_pcie(sys);
+	return pci_scan_root_bus(&pcie->pdev->dev, sys->busnr,
+				 &mvebu_pcie_ops, sys, &sys->resources);
+}
+
+resource_size_t mvebu_pcie_align_resource(struct pci_dev *dev,
+					  const struct resource *res,
+					  resource_size_t start,
+					  resource_size_t size,
+					  resource_size_t align)
+{
+	if (dev->bus->number != 0)
+		return start;
+
+	/*
+	 * On the PCI-to-PCI bridge side, the I/O windows must have at
+	 * least a 64 KB size and be aligned on their size, and the
+	 * memory windows must have at least a 1 MB size and be
+	 * aligned on their size
+	 */
+	if (res->flags & IORESOURCE_IO)
+		return round_up(start, max((resource_size_t)SZ_64K, size));
+	else if (res->flags & IORESOURCE_MEM)
+		return round_up(start, max((resource_size_t)SZ_1M, size));
+	else
+		return start;
+}
+
+static void mvebu_pcie_enable(struct mvebu_pcie *pcie)
+{
+	struct hw_pci hw;
+
+	memset(&hw, 0, sizeof(hw));
+
+	hw.nr_controllers = 1;
+	hw.private_data   = (void **)&pcie;
+	hw.setup          = mvebu_pcie_setup;
+	hw.scan           = mvebu_pcie_scan_bus;
+	hw.map_irq        = mvebu_pcie_map_irq;
+	hw.ops            = &mvebu_pcie_ops;
+	hw.align_resource = mvebu_pcie_align_resource;
+
+	pci_common_init(&hw);
+}
+
+static int __init mvebu_pcie_probe(struct platform_device *pdev)
+{
+	struct mvebu_pcie *pcie;
+	struct device_node *np = pdev->dev.of_node;
+	struct device_node *child;
+	const __be32 *range = NULL;
+	struct resource res;
+	int i, ret;
+
+	pcie = devm_kzalloc(&pdev->dev, sizeof(struct mvebu_pcie),
+			    GFP_KERNEL);
+	if (!pcie)
+		return -ENOMEM;
+
+	pcie->pdev = pdev;
+
+	/* Get the I/O and memory ranges from DT */
+	while ((range = of_pci_process_ranges(np, &res, range)) != NULL) {
+		if (resource_type(&res) == IORESOURCE_IO) {
+			memcpy(&pcie->io, &res, sizeof(res));
+			memcpy(&pcie->realio, &res, sizeof(res));
+			pcie->io.name = "I/O";
+			pcie->realio.start &= 0xFFFFF;
+			pcie->realio.end   &= 0xFFFFF;
+		}
+		if (resource_type(&res) == IORESOURCE_MEM) {
+			memcpy(&pcie->mem, &res, sizeof(res));
+			pcie->mem.name = "MEM";
+		}
+	}
+
+	/* Get the bus range */
+	ret = of_pci_parse_bus_range(np, &pcie->busn);
+	if (ret) {
+		dev_err(&pdev->dev, "failed to parse bus-range property: %d\n",
+			ret);
+		return ret;
+	}
+
+	for_each_child_of_node(pdev->dev.of_node, child) {
+		if (!of_device_is_available(child))
+			continue;
+		pcie->nports++;
+	}
+
+	pcie->ports = devm_kzalloc(&pdev->dev, pcie->nports *
+				   sizeof(struct mvebu_pcie_port),
+				   GFP_KERNEL);
+	if (!pcie->ports)
+		return -ENOMEM;
+
+	i = 0;
+	for_each_child_of_node(pdev->dev.of_node, child) {
+		struct mvebu_pcie_port *port = &pcie->ports[i];
+
+		if (!of_device_is_available(child))
+			continue;
+
+		port->pcie = pcie;
+
+		if (of_property_read_u32(child, "marvell,pcie-port",
+					 &port->port)) {
+			dev_warn(&pdev->dev,
+				 "ignoring PCIe DT node, missing pcie-port property\n");
+			continue;
+		}
+
+		if (of_property_read_u32(child, "marvell,pcie-lane",
+					 &port->lane))
+			port->lane = 0;
+
+		port->devfn = of_pci_get_devfn(child);
+		if (port->devfn < 0)
+			continue;
+
+		port->base = of_iomap(child, 0);
+		if (!port->base) {
+			dev_err(&pdev->dev, "PCIe%d.%d: cannot map registers\n",
+				port->port, port->lane);
+			continue;
+		}
+
+		if (orion_pcie_link_up(port->base)) {
+			port->haslink = 1;
+			dev_info(&pdev->dev, "PCIe%d.%d: link up\n",
+				 port->port, port->lane);
+		} else {
+			port->haslink = 0;
+			dev_info(&pdev->dev, "PCIe%d.%d: link down\n",
+				 port->port, port->lane);
+		}
+
+		port->clk = of_clk_get_by_name(child, NULL);
+		if (!port->clk) {
+			dev_err(&pdev->dev, "PCIe%d.%d: cannot get clock\n",
+			       port->port, port->lane);
+			iounmap(port->base);
+			port->haslink = 0;
+			continue;
+		}
+
+		port->dn = child;
+
+		clk_prepare_enable(port->clk);
+		spin_lock_init(&port->conf_lock);
+
+		mvebu_sw_pci_bridge_init(port);
+
+		i++;
+	}
+
+	mvebu_pcie_enable(pcie);
+
+	return 0;
+}
+
+static const struct of_device_id mvebu_pcie_of_match_table[] = {
+	{ .compatible = "marvell,armada-370-xp-pcie", },
+	{},
+};
+MODULE_DEVICE_TABLE(of, mvebu_pcie_of_match_table);
+
+static struct platform_driver mvebu_pcie_driver = {
+	.driver = {
+		.owner = THIS_MODULE,
+		.name = "mvebu-pcie",
+		.of_match_table =
+		   of_match_ptr(mvebu_pcie_of_match_table),
+	},
+};
+
+static int mvebu_pcie_init(void)
+{
+	return platform_driver_probe(&mvebu_pcie_driver,
+				     mvebu_pcie_probe);
+}
+
+subsys_initcall(mvebu_pcie_init);
+
+MODULE_AUTHOR("Thomas Petazzoni <thomas.petazzoni@free-electrons.com>");
+MODULE_DESCRIPTION("Marvell EBU PCIe driver");
+MODULE_LICENSE("GPLv2");
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 25/32] arm: mvebu: PCIe support is now available on mvebu
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:28   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Now that the PCIe driver for mvebu has been integrated and all its
relevant dependencies, we can mark the ARCH_MVEBU platform has
MIGHT_HAVE_PCI, which allows to select the PCI bus support if needed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/Kconfig |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/mach-mvebu/Kconfig b/arch/arm/mach-mvebu/Kconfig
index 440b13e..f12e475 100644
--- a/arch/arm/mach-mvebu/Kconfig
+++ b/arch/arm/mach-mvebu/Kconfig
@@ -13,6 +13,8 @@ config ARCH_MVEBU
 	select MVEBU_CLK_CORE
 	select MVEBU_CLK_CPU
 	select MVEBU_CLK_GATING
+	select MIGHT_HAVE_PCI
+	select PCI_QUIRKS if PCI
 
 if ARCH_MVEBU
 
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 25/32] arm: mvebu: PCIe support is now available on mvebu
@ 2013-02-12 16:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Now that the PCIe driver for mvebu has been integrated and all its
relevant dependencies, we can mark the ARCH_MVEBU platform has
MIGHT_HAVE_PCI, which allows to select the PCI bus support if needed.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/Kconfig |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm/mach-mvebu/Kconfig b/arch/arm/mach-mvebu/Kconfig
index 440b13e..f12e475 100644
--- a/arch/arm/mach-mvebu/Kconfig
+++ b/arch/arm/mach-mvebu/Kconfig
@@ -13,6 +13,8 @@ config ARCH_MVEBU
 	select MVEBU_CLK_CORE
 	select MVEBU_CLK_CPU
 	select MVEBU_CLK_GATING
+	select MIGHT_HAVE_PCI
+	select PCI_QUIRKS if PCI
 
 if ARCH_MVEBU
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 26/32] arm: mvebu: add PCIe Device Tree informations for Armada 370
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:29   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The Armada 370 SoC has two 1x PCIe 2.0 interfaces, so we add the
necessary Device Tree informations to make these interfaces availabel.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-370.dtsi |   43 +++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/arch/arm/boot/dts/armada-370.dtsi b/arch/arm/boot/dts/armada-370.dtsi
index 636cf7d..39055ba 100644
--- a/arch/arm/boot/dts/armada-370.dtsi
+++ b/arch/arm/boot/dts/armada-370.dtsi
@@ -132,5 +132,48 @@
 				dmacap,memset;
 			};
 		};
+
+		pcie-controller {
+			compatible = "marvell,armada-370-xp-pcie";
+			status = "disabled";
+
+			#address-cells = <3>;
+			#size-cells = <2>;
+
+			bus-range = <0x00 0xff>;
+
+			ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+				  0x00001000 0 0xd0080000 0xd0080000 0 0x00002000   /* port 1.0 registers */
+				  0x81000000 0 0	  0xc0000000 0 0x00010000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+			pcie@0,0 {
+				device_type = "pciex";
+				reg = <0x0800 0 0xd0040000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 58>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 5>;
+				status = "disabled";
+			};
+
+			pcie@1,0 {
+				device_type = "pciex";
+				reg = <0x1000 0 0xd0080000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 62>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 9>;
+				status = "disabled";
+			};
+		};
 	};
 };
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 26/32] arm: mvebu: add PCIe Device Tree informations for Armada 370
@ 2013-02-12 16:29   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: linux-arm-kernel

The Armada 370 SoC has two 1x PCIe 2.0 interfaces, so we add the
necessary Device Tree informations to make these interfaces availabel.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-370.dtsi |   43 +++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/arch/arm/boot/dts/armada-370.dtsi b/arch/arm/boot/dts/armada-370.dtsi
index 636cf7d..39055ba 100644
--- a/arch/arm/boot/dts/armada-370.dtsi
+++ b/arch/arm/boot/dts/armada-370.dtsi
@@ -132,5 +132,48 @@
 				dmacap,memset;
 			};
 		};
+
+		pcie-controller {
+			compatible = "marvell,armada-370-xp-pcie";
+			status = "disabled";
+
+			#address-cells = <3>;
+			#size-cells = <2>;
+
+			bus-range = <0x00 0xff>;
+
+			ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+				  0x00001000 0 0xd0080000 0xd0080000 0 0x00002000   /* port 1.0 registers */
+				  0x81000000 0 0	  0xc0000000 0 0x00010000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+			pcie at 0,0 {
+				device_type = "pciex";
+				reg = <0x0800 0 0xd0040000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 58>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 5>;
+				status = "disabled";
+			};
+
+			pcie at 1,0 {
+				device_type = "pciex";
+				reg = <0x1000 0 0xd0080000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 62>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 9>;
+				status = "disabled";
+			};
+		};
 	};
 };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 27/32] arm: mvebu: add PCIe Device Tree informations for Armada XP
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:29   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The Armada XP SoCs have multiple PCIe interfaces. The MV78230 has 2
PCIe units (one 4x or quad 1x, the other 1x only), the MV78260 has 3
PCIe units (two 4x or quad 1x and one 4x/1x), the MV78460 has 4 PCIe
units (two 4x or quad 1x and two 4x/1x). We therefore add the
necessary Device Tree informations to make those PCIe interfaces
usable.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-xp-mv78230.dtsi |   99 ++++++++++++++++++
 arch/arm/boot/dts/armada-xp-mv78260.dtsi |  114 ++++++++++++++++++++
 arch/arm/boot/dts/armada-xp-mv78460.dtsi |  167 ++++++++++++++++++++++++++++++
 3 files changed, 380 insertions(+)

diff --git a/arch/arm/boot/dts/armada-xp-mv78230.dtsi b/arch/arm/boot/dts/armada-xp-mv78230.dtsi
index e041f42..39344d1 100644
--- a/arch/arm/boot/dts/armada-xp-mv78230.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78230.dtsi
@@ -70,5 +70,104 @@
 			#interrupts-cells = <2>;
 			interrupts = <87>, <88>, <89>;
 		};
+
+		/*
+		 * MV78230 has 2 PCIe units Gen2.0: One unit can be
+		 * configured as x4 or quad x1 lanes. One unit is
+		 * x4/x1.
+		 */
+		pcie-controller {
+			compatible = "marvell,armada-370-xp-pcie";
+			status = "disabled";
+
+			#address-cells = <3>;
+			#size-cells = <2>;
+
+
+			bus-range = <0x00 0xff>;
+
+			ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+			          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
+			          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
+			          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
+			          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
+				  0x81000000 0 0	  0xc0000000 0 0x00010000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+			pcie@0,0 {
+				device_type = "pciex";
+				reg = <0x0800 0 0xd0040000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 58>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 5>;
+				status = "disabled";
+			};
+
+			pcie@0,1 {
+				device_type = "pciex";
+				reg = <0x1000 0 0xd0044000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 59>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <1>;
+				interrupts = <1>;
+				clocks = <&gateclk 6>;
+				status = "disabled";
+			};
+
+			pcie@0,2 {
+				device_type = "pciex";
+				reg = <0x1800 0 0xd0048000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 60>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <2>;
+				interrupts = <1>;
+				clocks = <&gateclk 7>;
+				status = "disabled";
+			};
+
+			pcie@0,3 {
+				device_type = "pciex";
+				reg = <0x2000 0 0xd004C000 0 0xC000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 61>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <3>;
+				interrupts = <1>;
+				clocks = <&gateclk 8>;
+				status = "disabled";
+			};
+
+			pcie@2,0 {
+				device_type = "pciex";
+				reg = <0x4800 0 0xd0042000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 99>;
+				marvell,pcie-port = <2>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 26>;
+				status = "disabled";
+			};
+		};
 	};
 };
diff --git a/arch/arm/boot/dts/armada-xp-mv78260.dtsi b/arch/arm/boot/dts/armada-xp-mv78260.dtsi
index 9e23bd8..1b4db26 100644
--- a/arch/arm/boot/dts/armada-xp-mv78260.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78260.dtsi
@@ -90,5 +90,119 @@
 				clocks = <&gateclk 1>;
 				status = "disabled";
 		};
+
+		/*
+		 * MV78260 has 3 PCIe units Gen2.0: Two units can be
+		 * configured as x4 or quad x1 lanes. One unit is
+		 * x4/x1.
+		 */
+		pcie-controller {
+			compatible = "marvell,armada-370-xp-pcie";
+			status = "disabled";
+
+			#address-cells = <3>;
+			#size-cells = <2>;
+
+			bus-range = <0x00 0xff>;
+
+			ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+			          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
+			          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
+			          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
+			          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
+			          0x00005000 0 0xd0082000 0xd0082000 0 0x00002000   /* port 3.0 registers */
+				  0x81000000 0 0	  0xc0000000 0 0x00010000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+			pcie@0,0 {
+				device_type = "pciex";
+				reg = <0x0800 0 0xd0040000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 58>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 5>;
+				status = "disabled";
+			};
+
+			pcie@0,1 {
+				device_type = "pciex";
+				reg = <0x1000 0 0xd0044000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 59>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <1>;
+				interrupts = <1>;
+				clocks = <&gateclk 6>;
+				status = "disabled";
+			};
+
+			pcie@0,2 {
+				device_type = "pciex";
+				reg = <0x1800 0 0xd0048000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 60>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <2>;
+				interrupts = <1>;
+				clocks = <&gateclk 7>;
+				status = "disabled";
+			};
+
+			pcie@0,3 {
+				device_type = "pciex";
+				reg = <0x2000 0 0xd004C000 0 0xC000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 61>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <3>;
+				interrupts = <1>;
+				clocks = <&gateclk 8>;
+				status = "disabled";
+			};
+
+			pcie@2,0 {
+				device_type = "pciex";
+				reg = <0x4800 0 0xd0042000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 99>;
+				marvell,pcie-port = <2>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 26>;
+				status = "disabled";
+			};
+
+			pcie@3,0 {
+				device_type = "pciex";
+				reg = <0x5000 0 0xd0082000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 103>;
+				marvell,pcie-port = <3>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 27>;
+				status = "disabled";
+			};
+		};
 	};
 };
diff --git a/arch/arm/boot/dts/armada-xp-mv78460.dtsi b/arch/arm/boot/dts/armada-xp-mv78460.dtsi
index 9659661..0673f77 100644
--- a/arch/arm/boot/dts/armada-xp-mv78460.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78460.dtsi
@@ -105,5 +105,172 @@
 				clocks = <&gateclk 1>;
 				status = "disabled";
 		};
+
+		/*
+		 * MV78460 has 4 PCIe units Gen2.0: Two units can be
+		 * configured as x4 or quad x1 lanes. Two units are
+		 * x4/x1.
+		 */
+		pcie-controller {
+			compatible = "marvell,armada-370-xp-pcie";
+			status = "disabled";
+
+			#address-cells = <3>;
+			#size-cells = <2>;
+
+			bus-range = <0x00 0xff>;
+
+			ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+			          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
+			          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
+			          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
+			          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
+				  0x00002800 0 0xd0080000 0xd0080000 0 0x00002000   /* port 1.0 registers */
+			          0x00005000 0 0xd0082000 0xd0082000 0 0x00002000   /* port 3.0 registers */
+				  0x00003000 0 0xd0084000 0xd0084000 0 0x00002000   /* port 1.1 registers */
+				  0x00003800 0 0xd0088000 0xd0088000 0 0x00002000   /* port 1.2 registers */
+				  0x00004000 0 0xd008C000 0xd008C000 0 0x00002000   /* port 1.3 registers */
+				  0x81000000 0 0	  0xc0000000 0 0x00100000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+			pcie@0,0 {
+				device_type = "pciex";
+				reg = <0x0800 0 0xd0040000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 58>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 5>;
+				status = "disabled";
+			};
+
+			pcie@0,1 {
+				device_type = "pciex";
+				reg = <0x1000 0 0xd0044000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 59>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <1>;
+				clocks = <&gateclk 6>;
+				status = "disabled";
+			};
+
+			pcie@0,2 {
+				device_type = "pciex";
+				reg = <0x1800 0 0xd0048000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 60>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <2>;
+				clocks = <&gateclk 7>;
+				status = "disabled";
+			};
+
+			pcie@0,3 {
+				device_type = "pciex";
+				reg = <0x2000 0 0xd004C000 0 0xC000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 61>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <3>;
+				clocks = <&gateclk 8>;
+				status = "disabled";
+			};
+
+			pcie@1,0 {
+				device_type = "pciex";
+				reg = <0x2800 0 0xd0080000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 62>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 9>;
+				status = "disabled";
+			};
+
+			pcie@1,1 {
+				device_type = "pciex";
+				reg = <0x3000 0 0xd0084000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 63>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <1>;
+				clocks = <&gateclk 10>;
+				status = "disabled";
+			};
+
+			pcie@1,2 {
+				device_type = "pciex";
+				reg = <0x3800 0 0xd0088000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 64>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <2>;
+				clocks = <&gateclk 11>;
+				status = "disabled";
+			};
+
+			pcie@1,3 {
+				device_type = "pciex";
+				reg = <0x4000 0 0xd008C000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 65>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <3>;
+				clocks = <&gateclk 12>;
+				status = "disabled";
+			};
+			pcie@2,0 {
+				device_type = "pciex";
+				reg = <0x4800 0 0xd0042000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 99>;
+				marvell,pcie-port = <2>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 26>;
+				status = "disabled";
+			};
+
+			pcie@3,0 {
+				device_type = "pciex";
+				reg = <0x5000 0 0xd0082000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 103>;
+				marvell,pcie-port = <3>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 27>;
+				status = "disabled";
+			};
+		};
 	};
  };
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 27/32] arm: mvebu: add PCIe Device Tree informations for Armada XP
@ 2013-02-12 16:29   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: linux-arm-kernel

The Armada XP SoCs have multiple PCIe interfaces. The MV78230 has 2
PCIe units (one 4x or quad 1x, the other 1x only), the MV78260 has 3
PCIe units (two 4x or quad 1x and one 4x/1x), the MV78460 has 4 PCIe
units (two 4x or quad 1x and two 4x/1x). We therefore add the
necessary Device Tree informations to make those PCIe interfaces
usable.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-xp-mv78230.dtsi |   99 ++++++++++++++++++
 arch/arm/boot/dts/armada-xp-mv78260.dtsi |  114 ++++++++++++++++++++
 arch/arm/boot/dts/armada-xp-mv78460.dtsi |  167 ++++++++++++++++++++++++++++++
 3 files changed, 380 insertions(+)

diff --git a/arch/arm/boot/dts/armada-xp-mv78230.dtsi b/arch/arm/boot/dts/armada-xp-mv78230.dtsi
index e041f42..39344d1 100644
--- a/arch/arm/boot/dts/armada-xp-mv78230.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78230.dtsi
@@ -70,5 +70,104 @@
 			#interrupts-cells = <2>;
 			interrupts = <87>, <88>, <89>;
 		};
+
+		/*
+		 * MV78230 has 2 PCIe units Gen2.0: One unit can be
+		 * configured as x4 or quad x1 lanes. One unit is
+		 * x4/x1.
+		 */
+		pcie-controller {
+			compatible = "marvell,armada-370-xp-pcie";
+			status = "disabled";
+
+			#address-cells = <3>;
+			#size-cells = <2>;
+
+
+			bus-range = <0x00 0xff>;
+
+			ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+			          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
+			          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
+			          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
+			          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
+				  0x81000000 0 0	  0xc0000000 0 0x00010000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+			pcie at 0,0 {
+				device_type = "pciex";
+				reg = <0x0800 0 0xd0040000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 58>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 5>;
+				status = "disabled";
+			};
+
+			pcie at 0,1 {
+				device_type = "pciex";
+				reg = <0x1000 0 0xd0044000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 59>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <1>;
+				interrupts = <1>;
+				clocks = <&gateclk 6>;
+				status = "disabled";
+			};
+
+			pcie at 0,2 {
+				device_type = "pciex";
+				reg = <0x1800 0 0xd0048000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 60>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <2>;
+				interrupts = <1>;
+				clocks = <&gateclk 7>;
+				status = "disabled";
+			};
+
+			pcie at 0,3 {
+				device_type = "pciex";
+				reg = <0x2000 0 0xd004C000 0 0xC000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 61>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <3>;
+				interrupts = <1>;
+				clocks = <&gateclk 8>;
+				status = "disabled";
+			};
+
+			pcie at 2,0 {
+				device_type = "pciex";
+				reg = <0x4800 0 0xd0042000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 99>;
+				marvell,pcie-port = <2>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 26>;
+				status = "disabled";
+			};
+		};
 	};
 };
diff --git a/arch/arm/boot/dts/armada-xp-mv78260.dtsi b/arch/arm/boot/dts/armada-xp-mv78260.dtsi
index 9e23bd8..1b4db26 100644
--- a/arch/arm/boot/dts/armada-xp-mv78260.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78260.dtsi
@@ -90,5 +90,119 @@
 				clocks = <&gateclk 1>;
 				status = "disabled";
 		};
+
+		/*
+		 * MV78260 has 3 PCIe units Gen2.0: Two units can be
+		 * configured as x4 or quad x1 lanes. One unit is
+		 * x4/x1.
+		 */
+		pcie-controller {
+			compatible = "marvell,armada-370-xp-pcie";
+			status = "disabled";
+
+			#address-cells = <3>;
+			#size-cells = <2>;
+
+			bus-range = <0x00 0xff>;
+
+			ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+			          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
+			          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
+			          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
+			          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
+			          0x00005000 0 0xd0082000 0xd0082000 0 0x00002000   /* port 3.0 registers */
+				  0x81000000 0 0	  0xc0000000 0 0x00010000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+			pcie at 0,0 {
+				device_type = "pciex";
+				reg = <0x0800 0 0xd0040000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 58>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 5>;
+				status = "disabled";
+			};
+
+			pcie at 0,1 {
+				device_type = "pciex";
+				reg = <0x1000 0 0xd0044000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 59>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <1>;
+				interrupts = <1>;
+				clocks = <&gateclk 6>;
+				status = "disabled";
+			};
+
+			pcie at 0,2 {
+				device_type = "pciex";
+				reg = <0x1800 0 0xd0048000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 60>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <2>;
+				interrupts = <1>;
+				clocks = <&gateclk 7>;
+				status = "disabled";
+			};
+
+			pcie at 0,3 {
+				device_type = "pciex";
+				reg = <0x2000 0 0xd004C000 0 0xC000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 61>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <3>;
+				interrupts = <1>;
+				clocks = <&gateclk 8>;
+				status = "disabled";
+			};
+
+			pcie at 2,0 {
+				device_type = "pciex";
+				reg = <0x4800 0 0xd0042000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 99>;
+				marvell,pcie-port = <2>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 26>;
+				status = "disabled";
+			};
+
+			pcie at 3,0 {
+				device_type = "pciex";
+				reg = <0x5000 0 0xd0082000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 103>;
+				marvell,pcie-port = <3>;
+				marvell,pcie-lane = <0>;
+				interrupts = <1>;
+				clocks = <&gateclk 27>;
+				status = "disabled";
+			};
+		};
 	};
 };
diff --git a/arch/arm/boot/dts/armada-xp-mv78460.dtsi b/arch/arm/boot/dts/armada-xp-mv78460.dtsi
index 9659661..0673f77 100644
--- a/arch/arm/boot/dts/armada-xp-mv78460.dtsi
+++ b/arch/arm/boot/dts/armada-xp-mv78460.dtsi
@@ -105,5 +105,172 @@
 				clocks = <&gateclk 1>;
 				status = "disabled";
 		};
+
+		/*
+		 * MV78460 has 4 PCIe units Gen2.0: Two units can be
+		 * configured as x4 or quad x1 lanes. Two units are
+		 * x4/x1.
+		 */
+		pcie-controller {
+			compatible = "marvell,armada-370-xp-pcie";
+			status = "disabled";
+
+			#address-cells = <3>;
+			#size-cells = <2>;
+
+			bus-range = <0x00 0xff>;
+
+			ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
+			          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
+			          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
+			          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
+			          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
+				  0x00002800 0 0xd0080000 0xd0080000 0 0x00002000   /* port 1.0 registers */
+			          0x00005000 0 0xd0082000 0xd0082000 0 0x00002000   /* port 3.0 registers */
+				  0x00003000 0 0xd0084000 0xd0084000 0 0x00002000   /* port 1.1 registers */
+				  0x00003800 0 0xd0088000 0xd0088000 0 0x00002000   /* port 1.2 registers */
+				  0x00004000 0 0xd008C000 0xd008C000 0 0x00002000   /* port 1.3 registers */
+				  0x81000000 0 0	  0xc0000000 0 0x00100000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
+
+			pcie at 0,0 {
+				device_type = "pciex";
+				reg = <0x0800 0 0xd0040000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 58>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 5>;
+				status = "disabled";
+			};
+
+			pcie at 0,1 {
+				device_type = "pciex";
+				reg = <0x1000 0 0xd0044000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 59>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <1>;
+				clocks = <&gateclk 6>;
+				status = "disabled";
+			};
+
+			pcie at 0,2 {
+				device_type = "pciex";
+				reg = <0x1800 0 0xd0048000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 60>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <2>;
+				clocks = <&gateclk 7>;
+				status = "disabled";
+			};
+
+			pcie at 0,3 {
+				device_type = "pciex";
+				reg = <0x2000 0 0xd004C000 0 0xC000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 61>;
+				marvell,pcie-port = <0>;
+				marvell,pcie-lane = <3>;
+				clocks = <&gateclk 8>;
+				status = "disabled";
+			};
+
+			pcie at 1,0 {
+				device_type = "pciex";
+				reg = <0x2800 0 0xd0080000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 62>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 9>;
+				status = "disabled";
+			};
+
+			pcie at 1,1 {
+				device_type = "pciex";
+				reg = <0x3000 0 0xd0084000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 63>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <1>;
+				clocks = <&gateclk 10>;
+				status = "disabled";
+			};
+
+			pcie at 1,2 {
+				device_type = "pciex";
+				reg = <0x3800 0 0xd0088000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 64>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <2>;
+				clocks = <&gateclk 11>;
+				status = "disabled";
+			};
+
+			pcie at 1,3 {
+				device_type = "pciex";
+				reg = <0x4000 0 0xd008C000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 65>;
+				marvell,pcie-port = <1>;
+				marvell,pcie-lane = <3>;
+				clocks = <&gateclk 12>;
+				status = "disabled";
+			};
+			pcie at 2,0 {
+				device_type = "pciex";
+				reg = <0x4800 0 0xd0042000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 99>;
+				marvell,pcie-port = <2>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 26>;
+				status = "disabled";
+			};
+
+			pcie at 3,0 {
+				device_type = "pciex";
+				reg = <0x5000 0 0xd0082000 0 0x2000>;
+				#address-cells = <3>;
+				#size-cells = <2>;
+				#interrupt-cells = <1>;
+				interrupt-map-mask = <0 0 0 0>;
+				interrupt-map = <0 0 0 0 &mpic 103>;
+				marvell,pcie-port = <3>;
+				marvell,pcie-lane = <0>;
+				clocks = <&gateclk 27>;
+				status = "disabled";
+			};
+		};
 	};
  };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 28/32] arm: mvebu: PCIe Device Tree informations for OpenBlocks AX3-4
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:29   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The PlatHome OpenBlocks AX3-4 has an internal mini-PCIe slot that can
be used to plug mini-PCIe devices. We therefore enable the PCIe
interface that corresponds to this slot.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts b/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
index b42652f..67fcaaa 100644
--- a/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
+++ b/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
@@ -121,5 +121,12 @@
 			nr-ports = <2>;
 			status = "okay";
 		};
+		pcie-controller {
+			status = "okay";
+			/* Internal mini-PCIe connector */
+			pcie@0,0 {
+				status = "okay";
+			};
+		};
 	};
 };
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 28/32] arm: mvebu: PCIe Device Tree informations for OpenBlocks AX3-4
@ 2013-02-12 16:29   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: linux-arm-kernel

The PlatHome OpenBlocks AX3-4 has an internal mini-PCIe slot that can
be used to plug mini-PCIe devices. We therefore enable the PCIe
interface that corresponds to this slot.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts |    7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts b/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
index b42652f..67fcaaa 100644
--- a/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
+++ b/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
@@ -121,5 +121,12 @@
 			nr-ports = <2>;
 			status = "okay";
 		};
+		pcie-controller {
+			status = "okay";
+			/* Internal mini-PCIe connector */
+			pcie at 0,0 {
+				status = "okay";
+			};
+		};
 	};
 };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 29/32] arm: mvebu: PCIe Device Tree informations for Armada XP DB
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:29   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The Marvell evaluation board (DB) for the Armada XP SoC has 6
physicals full-size PCIe slots, so we enable the corresponding PCIe
interfaces in the Device Tree.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-xp-db.dts |   27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/arch/arm/boot/dts/armada-xp-db.dts b/arch/arm/boot/dts/armada-xp-db.dts
index 8e53b25..7dcc36c 100644
--- a/arch/arm/boot/dts/armada-xp-db.dts
+++ b/arch/arm/boot/dts/armada-xp-db.dts
@@ -90,5 +90,32 @@
 			phy = <&phy3>;
 			phy-mode = "sgmii";
 		};
+
+		pcie-controller {
+			status = "okay";
+
+			/*
+			 * All 6 slots are physically present as
+			 * standard PCIe slots on the board.
+			 */
+			pcie@0,0 {
+				status = "okay";
+			};
+			pcie@0,1 {
+				status = "okay";
+			};
+			pcie@0,2 {
+				status = "okay";
+			};
+			pcie@0,3 {
+				status = "okay";
+			};
+			pcie@2,0 {
+				status = "okay";
+			};
+			pcie@3,0 {
+				status = "okay";
+			};
+		};
 	};
 };
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 29/32] arm: mvebu: PCIe Device Tree informations for Armada XP DB
@ 2013-02-12 16:29   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: linux-arm-kernel

The Marvell evaluation board (DB) for the Armada XP SoC has 6
physicals full-size PCIe slots, so we enable the corresponding PCIe
interfaces in the Device Tree.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-xp-db.dts |   27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/arch/arm/boot/dts/armada-xp-db.dts b/arch/arm/boot/dts/armada-xp-db.dts
index 8e53b25..7dcc36c 100644
--- a/arch/arm/boot/dts/armada-xp-db.dts
+++ b/arch/arm/boot/dts/armada-xp-db.dts
@@ -90,5 +90,32 @@
 			phy = <&phy3>;
 			phy-mode = "sgmii";
 		};
+
+		pcie-controller {
+			status = "okay";
+
+			/*
+			 * All 6 slots are physically present as
+			 * standard PCIe slots on the board.
+			 */
+			pcie at 0,0 {
+				status = "okay";
+			};
+			pcie at 0,1 {
+				status = "okay";
+			};
+			pcie at 0,2 {
+				status = "okay";
+			};
+			pcie at 0,3 {
+				status = "okay";
+			};
+			pcie at 2,0 {
+				status = "okay";
+			};
+			pcie at 3,0 {
+				status = "okay";
+			};
+		};
 	};
 };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 30/32] arm: mvebu: PCIe Device Tree informations for Armada 370 Mirabox
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:29   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The Globalscale Mirabox platform uses one PCIe interface for an
available mini-PCIe slot, and the other PCIe interface for an internal
USB 3.0 controller. We add the necessary Device Tree informations to
enable those two interfaces.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-370-mirabox.dts |   14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/arm/boot/dts/armada-370-mirabox.dts b/arch/arm/boot/dts/armada-370-mirabox.dts
index 3b40713..591068a 100644
--- a/arch/arm/boot/dts/armada-370-mirabox.dts
+++ b/arch/arm/boot/dts/armada-370-mirabox.dts
@@ -52,5 +52,19 @@
 			phy = <&phy1>;
 			phy-mode = "rgmii-id";
 		};
+
+		pcie-controller {
+			status = "okay";
+
+			/* Internal mini-PCIe connector */
+			pcie@0,0 {
+				status = "okay";
+			};
+
+			/* Connected on the PCB to a USB 3.0 XHCI controller */
+			pcie@1,0 {
+				status = "okay";
+			};
+		};
 	};
 };
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 30/32] arm: mvebu: PCIe Device Tree informations for Armada 370 Mirabox
@ 2013-02-12 16:29   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: linux-arm-kernel

The Globalscale Mirabox platform uses one PCIe interface for an
available mini-PCIe slot, and the other PCIe interface for an internal
USB 3.0 controller. We add the necessary Device Tree informations to
enable those two interfaces.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-370-mirabox.dts |   14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/arm/boot/dts/armada-370-mirabox.dts b/arch/arm/boot/dts/armada-370-mirabox.dts
index 3b40713..591068a 100644
--- a/arch/arm/boot/dts/armada-370-mirabox.dts
+++ b/arch/arm/boot/dts/armada-370-mirabox.dts
@@ -52,5 +52,19 @@
 			phy = <&phy1>;
 			phy-mode = "rgmii-id";
 		};
+
+		pcie-controller {
+			status = "okay";
+
+			/* Internal mini-PCIe connector */
+			pcie at 0,0 {
+				status = "okay";
+			};
+
+			/* Connected on the PCB to a USB 3.0 XHCI controller */
+			pcie at 1,0 {
+				status = "okay";
+			};
+		};
 	};
 };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 31/32] arm: mvebu: PCIe Device Tree informations for Armada 370 DB
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:29   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

The Marvell evaluation board (DB) for the Armada 370 SoC has 2
physical full-size PCIe slots, so we enable the corresponding PCIe
interfaces in the Device Tree.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-370-db.dts |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/arch/arm/boot/dts/armada-370-db.dts b/arch/arm/boot/dts/armada-370-db.dts
index 9b82fac..fba3e8e 100644
--- a/arch/arm/boot/dts/armada-370-db.dts
+++ b/arch/arm/boot/dts/armada-370-db.dts
@@ -59,5 +59,20 @@
 			phy = <&phy1>;
 			phy-mode = "rgmii-id";
 		};
+
+		pcie-controller {
+			status = "okay";
+			/*
+			 * The two PCIe units are accessible through
+			 * both standard PCIe slots and mini-PCIe
+			 * slots on the board.
+			 */
+			pcie@0,0 {
+				status = "okay";
+			};
+			pcie@1,0 {
+				status = "okay";
+			};
+		};
 	};
 };
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 31/32] arm: mvebu: PCIe Device Tree informations for Armada 370 DB
@ 2013-02-12 16:29   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: linux-arm-kernel

The Marvell evaluation board (DB) for the Armada 370 SoC has 2
physical full-size PCIe slots, so we enable the corresponding PCIe
interfaces in the Device Tree.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-370-db.dts |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/arch/arm/boot/dts/armada-370-db.dts b/arch/arm/boot/dts/armada-370-db.dts
index 9b82fac..fba3e8e 100644
--- a/arch/arm/boot/dts/armada-370-db.dts
+++ b/arch/arm/boot/dts/armada-370-db.dts
@@ -59,5 +59,20 @@
 			phy = <&phy1>;
 			phy-mode = "rgmii-id";
 		};
+
+		pcie-controller {
+			status = "okay";
+			/*
+			 * The two PCIe units are accessible through
+			 * both standard PCIe slots and mini-PCIe
+			 * slots on the board.
+			 */
+			pcie at 0,0 {
+				status = "okay";
+			};
+			pcie at 1,0 {
+				status = "okay";
+			};
+		};
 	};
 };
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 32/32] arm: mvebu: update defconfig with PCI and USB support
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 16:29   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-pci, linux-arm-kernel
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, Thierry Reding,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

Now that we have the necessary drivers and Device Tree informations to
support PCIe on Armada 370 and Armada XP, enable the CONFIG_PCI
option.

Also, since the Armada 370 Mirabox has a built-in USB XHCI controller
connected on the PCIe bus, enable the corresponding options as well.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/configs/mvebu_defconfig |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm/configs/mvebu_defconfig b/arch/arm/configs/mvebu_defconfig
index b5bc96c..68ef50b 100644
--- a/arch/arm/configs/mvebu_defconfig
+++ b/arch/arm/configs/mvebu_defconfig
@@ -13,6 +13,7 @@ CONFIG_MACH_ARMADA_370=y
 CONFIG_MACH_ARMADA_XP=y
 # CONFIG_CACHE_L2X0 is not set
 # CONFIG_SWP_EMULATE is not set
+CONFIG_PCI=y
 CONFIG_SMP=y
 # CONFIG_LOCAL_TIMERS is not set
 CONFIG_AEABI=y
@@ -36,7 +37,8 @@ CONFIG_SERIAL_8250_CONSOLE=y
 CONFIG_SERIAL_8250_DW=y
 CONFIG_GPIOLIB=y
 CONFIG_GPIO_SYSFS=y
-# CONFIG_USB_SUPPORT is not set
+CONFIG_USB=y
+CONFIG_USB_XHCI_HCD=y
 CONFIG_RTC_CLASS=y
 CONFIG_RTC_DRV_S35390A=y
 CONFIG_DMADEVICES=y
-- 
1.7.9.5


^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 32/32] arm: mvebu: update defconfig with PCI and USB support
@ 2013-02-12 16:29   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 16:29 UTC (permalink / raw)
  To: linux-arm-kernel

Now that we have the necessary drivers and Device Tree informations to
support PCIe on Armada 370 and Armada XP, enable the CONFIG_PCI
option.

Also, since the Armada 370 Mirabox has a built-in USB XHCI controller
connected on the PCIe bus, enable the corresponding options as well.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/configs/mvebu_defconfig |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm/configs/mvebu_defconfig b/arch/arm/configs/mvebu_defconfig
index b5bc96c..68ef50b 100644
--- a/arch/arm/configs/mvebu_defconfig
+++ b/arch/arm/configs/mvebu_defconfig
@@ -13,6 +13,7 @@ CONFIG_MACH_ARMADA_370=y
 CONFIG_MACH_ARMADA_XP=y
 # CONFIG_CACHE_L2X0 is not set
 # CONFIG_SWP_EMULATE is not set
+CONFIG_PCI=y
 CONFIG_SMP=y
 # CONFIG_LOCAL_TIMERS is not set
 CONFIG_AEABI=y
@@ -36,7 +37,8 @@ CONFIG_SERIAL_8250_CONSOLE=y
 CONFIG_SERIAL_8250_DW=y
 CONFIG_GPIOLIB=y
 CONFIG_GPIO_SYSFS=y
-# CONFIG_USB_SUPPORT is not set
+CONFIG_USB=y
+CONFIG_USB_XHCI_HCD=y
 CONFIG_RTC_CLASS=y
 CONFIG_RTC_DRV_S35390A=y
 CONFIG_DMADEVICES=y
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* Re: [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
  2013-02-12 16:28   ` Thomas Petazzoni
@ 2013-02-12 18:00     ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 18:00 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk, Paul Gortmaker, Jesse Barnes, Yinghai Lu,
	linux-kernel

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> The pcim_*() functions are used by the libata-sff subsystem, and this
> subsystem is used for many SATA drivers on ARM platforms that do not
> necessarily have I/O ports.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
> Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
> Cc: Yinghai Lu <yinghai@kernel.org>
> Cc: linux-kernel@vger.kernel.org

Sorry, but this patch is still incorrect. Any driver that requires a linear
mapping of I/O ports to __iomem pointers must depend CONFIG_HAS_IOPORT
with the current definition of that symbol (as mentioned before, we
should really rename that to CONFIG_HAS_IOPORT_MAP). Having these
functions not defined is a compile time check that is necessary to
ensure that all drivers have the correct annotation.

If a platform has no support for I/O ports at all, it should
probably not set CONFIG_NO_IOPORT at this point.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
@ 2013-02-12 18:00     ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 18:00 UTC (permalink / raw)
  To: linux-arm-kernel

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> The pcim_*() functions are used by the libata-sff subsystem, and this
> subsystem is used for many SATA drivers on ARM platforms that do not
> necessarily have I/O ports.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
> Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
> Cc: Yinghai Lu <yinghai@kernel.org>
> Cc: linux-kernel at vger.kernel.org

Sorry, but this patch is still incorrect. Any driver that requires a linear
mapping of I/O ports to __iomem pointers must depend CONFIG_HAS_IOPORT
with the current definition of that symbol (as mentioned before, we
should really rename that to CONFIG_HAS_IOPORT_MAP). Having these
functions not defined is a compile time check that is necessary to
ensure that all drivers have the correct annotation.

If a platform has no support for I/O ports at all, it should
probably not set CONFIG_NO_IOPORT at this point.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 06/32] arm: pci: add a align_resource hook
  2013-02-12 16:28   ` Thomas Petazzoni
@ 2013-02-12 18:03     ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 18:03 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> The PCI specifications says that an I/O region must be aligned on a 4
> KB boundary, and a memory region aligned on a 1 MB boundary.
> 
> However, the Marvell PCIe interfaces rely on address decoding windows
> (which allow to associate a range of physical addresses with a given
> device), and those have special requirements compared to the standard
> PCI-to-PCI bridge specifications.

I'm not convince that we should add this complexity yet, until everyone
agrees on the basic approach taken.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 06/32] arm: pci: add a align_resource hook
@ 2013-02-12 18:03     ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 18:03 UTC (permalink / raw)
  To: linux-arm-kernel

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> The PCI specifications says that an I/O region must be aligned on a 4
> KB boundary, and a memory region aligned on a 1 MB boundary.
> 
> However, the Marvell PCIe interfaces rely on address decoding windows
> (which allow to associate a range of physical addresses with a given
> device), and those have special requirements compared to the standard
> PCI-to-PCI bridge specifications.

I'm not convince that we should add this complexity yet, until everyone
agrees on the basic approach taken.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-12 18:12   ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 18:12 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> This series of patches introduces PCIe support for the Marvell Armada
> 370 and Armada XP.
> 
> This PATCHv3 follows:
>  * PATCHv2, sent on January, 28th 2013
>  * RFCv1, sent on December, 7th 2012
> 
> Thanks to all the people who discussed on the previous version of the
> patch set. The discussions have been long and complicated, but
> certainly very useful.
> 
> In order to make progress on this, and help reducing the size of the
> patch set, I would ask if it would be possible to merge patches 1 to
> 17 from the series for 3.9 (only preparation work), keeping the rest
> for 3.10. The patches in question are PCI-related, ARM-related, and
> mvebu/orion-related.

You can add my 'Reviewed-by' tag on all paches except 5, 6, and 24.

I'm fine with adding the other ones to 3.9, as well as adding
the binding document if you split that out from patch 24. Sorry for
making this so hard for you.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
@ 2013-02-12 18:12   ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 18:12 UTC (permalink / raw)
  To: linux-arm-kernel

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> This series of patches introduces PCIe support for the Marvell Armada
> 370 and Armada XP.
> 
> This PATCHv3 follows:
>  * PATCHv2, sent on January, 28th 2013
>  * RFCv1, sent on December, 7th 2012
> 
> Thanks to all the people who discussed on the previous version of the
> patch set. The discussions have been long and complicated, but
> certainly very useful.
> 
> In order to make progress on this, and help reducing the size of the
> patch set, I would ask if it would be possible to merge patches 1 to
> 17 from the series for 3.9 (only preparation work), keeping the rest
> for 3.10. The patches in question are PCI-related, ARM-related, and
> mvebu/orion-related.

You can add my 'Reviewed-by' tag on all paches except 5, 6, and 24.

I'm fine with adding the other ones to 3.9, as well as adding
the binding document if you split that out from patch 24. Sorry for
making this so hard for you.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 16:28   ` Thomas Petazzoni
@ 2013-02-12 18:30     ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 18:30 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> new file mode 100644
> index 0000000..3ad563f
> --- /dev/null
> +++ b/drivers/pci/host/Makefile
> @@ -0,0 +1,4 @@
> +obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
> +CFLAGS_pci-mvebu.o += \
> +	-I$(srctree)/arch/arm/plat-orion/include \
> +	-I$(srctree)/arch/arm/mach-mvebu/include

This does not seem like a good idea to me. We should not include
architecture specific directories from a driver directory.

What are the header files you need here? 

> +/*
> + * This product ID is registered by Marvell, and used when the Marvell
> + * SoC is not the root complex, but an endpoint on the PCIe bus. It is
> + * therefore safe to re-use this PCI ID for our emulated PCI-to-PCI
> + * bridge.
> + */
> +#define MARVELL_EMULATED_PCI_PCI_BRIDGE_ID 0x7846

Just a side note: What happens if you have two of these systems and connect
them over PCIe, putting one of them into host mode and the other into
endpoint mode?

> +static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port *port,
> +				       int enable)
> +{
> +	unsigned long iobase, iolimit;
> +
> +	if (port->bridge.iolimit < port->bridge.iobase)
> +		return;
> +
> +	iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
> +		(port->bridge.iolimitupper << 16);
> +	iobase = ((port->bridge.iobase & 0xF0) << 8) |
> +		(port->bridge.iobaseupper << 16);

I don't understand this code with the masks and shifts. Could you
add a comment here for readers like me?

> +
> +/*
> + * Initialize the configuration space of the PCI-to-PCI bridge
> + * associated with the given PCIe interface.
> + */
> +static void mvebu_sw_pci_bridge_init(struct mvebu_pcie_port *port)
> +{

As mentioned, I'm still skeptical of the sw_pci_bridge approach,
so I'm not commenting on the details of your implementations
(they seem fine on a first look though)

> +	/* Get the I/O and memory ranges from DT */
> +	while ((range = of_pci_process_ranges(np, &res, range)) != NULL) {
> +		if (resource_type(&res) == IORESOURCE_IO) {
> +			memcpy(&pcie->io, &res, sizeof(res));
> +			memcpy(&pcie->realio, &res, sizeof(res));
> +			pcie->io.name = "I/O";
> +			pcie->realio.start &= 0xFFFFF;
> +			pcie->realio.end   &= 0xFFFFF;
> +		}

The bit masking seems fishy here. What exactly are you doing,
does this just assume you have a 1MB window at most?

Maybe something like

	pcie->realio.start = 0;
	pcie->realio.end = pcie->io.end - pcie->io.start;

I suppose you also need to fix up pcie->io to be in IORESOURCE_MEM
space instead of IORESOURCE_IO, or fix the of_pci_process_ranges
function to return it in a different way.

> +static int mvebu_pcie_init(void)
> +{
> +	return platform_driver_probe(&mvebu_pcie_driver,
> +				     mvebu_pcie_probe);
> +}
> +
> +subsys_initcall(mvebu_pcie_init);

You don't have to do it, but I wonder if this could be a module
with unload support instead.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-12 18:30     ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 18:30 UTC (permalink / raw)
  To: linux-arm-kernel

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> new file mode 100644
> index 0000000..3ad563f
> --- /dev/null
> +++ b/drivers/pci/host/Makefile
> @@ -0,0 +1,4 @@
> +obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
> +CFLAGS_pci-mvebu.o += \
> +	-I$(srctree)/arch/arm/plat-orion/include \
> +	-I$(srctree)/arch/arm/mach-mvebu/include

This does not seem like a good idea to me. We should not include
architecture specific directories from a driver directory.

What are the header files you need here? 

> +/*
> + * This product ID is registered by Marvell, and used when the Marvell
> + * SoC is not the root complex, but an endpoint on the PCIe bus. It is
> + * therefore safe to re-use this PCI ID for our emulated PCI-to-PCI
> + * bridge.
> + */
> +#define MARVELL_EMULATED_PCI_PCI_BRIDGE_ID 0x7846

Just a side note: What happens if you have two of these systems and connect
them over PCIe, putting one of them into host mode and the other into
endpoint mode?

> +static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port *port,
> +				       int enable)
> +{
> +	unsigned long iobase, iolimit;
> +
> +	if (port->bridge.iolimit < port->bridge.iobase)
> +		return;
> +
> +	iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
> +		(port->bridge.iolimitupper << 16);
> +	iobase = ((port->bridge.iobase & 0xF0) << 8) |
> +		(port->bridge.iobaseupper << 16);

I don't understand this code with the masks and shifts. Could you
add a comment here for readers like me?

> +
> +/*
> + * Initialize the configuration space of the PCI-to-PCI bridge
> + * associated with the given PCIe interface.
> + */
> +static void mvebu_sw_pci_bridge_init(struct mvebu_pcie_port *port)
> +{

As mentioned, I'm still skeptical of the sw_pci_bridge approach,
so I'm not commenting on the details of your implementations
(they seem fine on a first look though)

> +	/* Get the I/O and memory ranges from DT */
> +	while ((range = of_pci_process_ranges(np, &res, range)) != NULL) {
> +		if (resource_type(&res) == IORESOURCE_IO) {
> +			memcpy(&pcie->io, &res, sizeof(res));
> +			memcpy(&pcie->realio, &res, sizeof(res));
> +			pcie->io.name = "I/O";
> +			pcie->realio.start &= 0xFFFFF;
> +			pcie->realio.end   &= 0xFFFFF;
> +		}

The bit masking seems fishy here. What exactly are you doing,
does this just assume you have a 1MB window at most?

Maybe something like

	pcie->realio.start = 0;
	pcie->realio.end = pcie->io.end - pcie->io.start;

I suppose you also need to fix up pcie->io to be in IORESOURCE_MEM
space instead of IORESOURCE_IO, or fix the of_pci_process_ranges
function to return it in a different way.

> +static int mvebu_pcie_init(void)
> +{
> +	return platform_driver_probe(&mvebu_pcie_driver,
> +				     mvebu_pcie_probe);
> +}
> +
> +subsys_initcall(mvebu_pcie_init);

You don't have to do it, but I wonder if this could be a module
with unload support instead.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
  2013-02-12 18:00     ` Arnd Bergmann
@ 2013-02-12 18:58       ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 18:58 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk, Paul Gortmaker, Jesse Barnes, Yinghai Lu,
	linux-kernel

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:00:48 +0000, Arnd Bergmann wrote:
> On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > The pcim_*() functions are used by the libata-sff subsystem, and
> > this subsystem is used for many SATA drivers on ARM platforms that
> > do not necessarily have I/O ports.
> > 
> > Signed-off-by: Thomas Petazzoni
> > <thomas.petazzoni@free-electrons.com> Cc: Paul Gortmaker
> > <paul.gortmaker@windriver.com> Cc: Jesse Barnes
> > <jbarnes@virtuousgeek.org> Cc: Yinghai Lu <yinghai@kernel.org>
> > Cc: linux-kernel@vger.kernel.org
> 
> Sorry, but this patch is still incorrect.

I know, but the discussion was so huge on the first posting that it was
basically impossible to draw a conclusion out of it.

> Any driver that requires a
> linear mapping of I/O ports to __iomem pointers must depend
> CONFIG_HAS_IOPORT with the current definition of that symbol (as
> mentioned before, we should really rename that to
> CONFIG_HAS_IOPORT_MAP). Having these functions not defined is a
> compile time check that is necessary to ensure that all drivers have
> the correct annotation.

I have the feeling that the problem is more complex than that. My
understanding is that the pcim_iomap_regions() function used by
drivers/ata/libata-sff.c can perfectly be used to map memory BARs, and
not necessarily I/O BARs. Therefore, this driver can perfectly be used
in an architecture where CONFIG_NO_IOPORT is selected.

The thing is that pcim_iomap_regions() transparently allows to remap an
I/O BAR is such a BAR is passed as argument, or a memory BAR if such a
BAR is passed as argument.

Therefore, I continue to believe that the pcim_*() functions are useful
even if the platform doesn't have CONFIG_HAS_IOPORT.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
@ 2013-02-12 18:58       ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 18:58 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:00:48 +0000, Arnd Bergmann wrote:
> On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > The pcim_*() functions are used by the libata-sff subsystem, and
> > this subsystem is used for many SATA drivers on ARM platforms that
> > do not necessarily have I/O ports.
> > 
> > Signed-off-by: Thomas Petazzoni
> > <thomas.petazzoni@free-electrons.com> Cc: Paul Gortmaker
> > <paul.gortmaker@windriver.com> Cc: Jesse Barnes
> > <jbarnes@virtuousgeek.org> Cc: Yinghai Lu <yinghai@kernel.org>
> > Cc: linux-kernel at vger.kernel.org
> 
> Sorry, but this patch is still incorrect.

I know, but the discussion was so huge on the first posting that it was
basically impossible to draw a conclusion out of it.

> Any driver that requires a
> linear mapping of I/O ports to __iomem pointers must depend
> CONFIG_HAS_IOPORT with the current definition of that symbol (as
> mentioned before, we should really rename that to
> CONFIG_HAS_IOPORT_MAP). Having these functions not defined is a
> compile time check that is necessary to ensure that all drivers have
> the correct annotation.

I have the feeling that the problem is more complex than that. My
understanding is that the pcim_iomap_regions() function used by
drivers/ata/libata-sff.c can perfectly be used to map memory BARs, and
not necessarily I/O BARs. Therefore, this driver can perfectly be used
in an architecture where CONFIG_NO_IOPORT is selected.

The thing is that pcim_iomap_regions() transparently allows to remap an
I/O BAR is such a BAR is passed as argument, or a memory BAR if such a
BAR is passed as argument.

Therefore, I continue to believe that the pcim_*() functions are useful
even if the platform doesn't have CONFIG_HAS_IOPORT.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 06/32] arm: pci: add a align_resource hook
  2013-02-12 18:03     ` Arnd Bergmann
@ 2013-02-12 19:01       ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 19:01 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:03:12 +0000, Arnd Bergmann wrote:
> On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > The PCI specifications says that an I/O region must be aligned on a
> > 4 KB boundary, and a memory region aligned on a 1 MB boundary.
> > 
> > However, the Marvell PCIe interfaces rely on address decoding
> > windows (which allow to associate a range of physical addresses
> > with a given device), and those have special requirements compared
> > to the standard PCI-to-PCI bridge specifications.
> 
> I'm not convince that we should add this complexity yet, until
> everyone agrees on the basic approach taken.

Regardless of whether we choose to have the emulated PCI-to-PCI bridges
or not, we still need this align_resource() hook.

In the solution you propose, where each PCIe interface is represented
as a separate PCIe domain, we still need the kernel to dynamically
assign ranges of address to each memory BAR and I/O BAR of each PCIe
device. And those range of address must comply with the address
decoding windows requirements, otherwise, we don't be able to create
those address decoding windows, and the devices will be unaccessible.

Of course, if you have an alternate solution to do a _dynamic_
assignment of address ranges to the different PCIe devices, I'm
entirely open to suggestions.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 06/32] arm: pci: add a align_resource hook
@ 2013-02-12 19:01       ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 19:01 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:03:12 +0000, Arnd Bergmann wrote:
> On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > The PCI specifications says that an I/O region must be aligned on a
> > 4 KB boundary, and a memory region aligned on a 1 MB boundary.
> > 
> > However, the Marvell PCIe interfaces rely on address decoding
> > windows (which allow to associate a range of physical addresses
> > with a given device), and those have special requirements compared
> > to the standard PCI-to-PCI bridge specifications.
> 
> I'm not convince that we should add this complexity yet, until
> everyone agrees on the basic approach taken.

Regardless of whether we choose to have the emulated PCI-to-PCI bridges
or not, we still need this align_resource() hook.

In the solution you propose, where each PCIe interface is represented
as a separate PCIe domain, we still need the kernel to dynamically
assign ranges of address to each memory BAR and I/O BAR of each PCIe
device. And those range of address must comply with the address
decoding windows requirements, otherwise, we don't be able to create
those address decoding windows, and the devices will be unaccessible.

Of course, if you have an alternate solution to do a _dynamic_
assignment of address ranges to the different PCIe devices, I'm
entirely open to suggestions.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
  2013-02-12 18:12   ` Arnd Bergmann
@ 2013-02-12 19:04     ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 19:04 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:12:11 +0000, Arnd Bergmann wrote:

> > In order to make progress on this, and help reducing the size of the
> > patch set, I would ask if it would be possible to merge patches 1 to
> > 17 from the series for 3.9 (only preparation work), keeping the rest
> > for 3.10. The patches in question are PCI-related, ARM-related, and
> > mvebu/orion-related.
> 
> You can add my 'Reviewed-by' tag on all paches except 5, 6, and 24.

Thanks. I would already be wonderful if I could merge many of those
preparations patches for 3.9.

> I'm fine with adding the other ones to 3.9, as well as adding
> the binding document if you split that out from patch 24. Sorry for
> making this so hard for you.

That's fine, the problem is difficult, so it's expected to be difficult
to find the right solution.

However, I'd like to get serious alternate proposals when the current
proposal isn't considered appropriate. A serious proposal needs to
provide a mechanism to dynamically assign range of addresses to the
different BARs of the different PCIe devices, and those range of
addresses must comply with the special alignment requirements of the
Marvell address decoding windows. So far, I haven't seen any alternate
proposal that has those characteristics. The only alternate proposals
were based on static assignments of addresses to each PCIe interface,
which is unacceptable.

Thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
@ 2013-02-12 19:04     ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 19:04 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:12:11 +0000, Arnd Bergmann wrote:

> > In order to make progress on this, and help reducing the size of the
> > patch set, I would ask if it would be possible to merge patches 1 to
> > 17 from the series for 3.9 (only preparation work), keeping the rest
> > for 3.10. The patches in question are PCI-related, ARM-related, and
> > mvebu/orion-related.
> 
> You can add my 'Reviewed-by' tag on all paches except 5, 6, and 24.

Thanks. I would already be wonderful if I could merge many of those
preparations patches for 3.9.

> I'm fine with adding the other ones to 3.9, as well as adding
> the binding document if you split that out from patch 24. Sorry for
> making this so hard for you.

That's fine, the problem is difficult, so it's expected to be difficult
to find the right solution.

However, I'd like to get serious alternate proposals when the current
proposal isn't considered appropriate. A serious proposal needs to
provide a mechanism to dynamically assign range of addresses to the
different BARs of the different PCIe devices, and those range of
addresses must comply with the special alignment requirements of the
Marvell address decoding windows. So far, I haven't seen any alternate
proposal that has those characteristics. The only alternate proposals
were based on static assignments of addresses to each PCIe interface,
which is unacceptable.

Thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 18:30     ` Arnd Bergmann
@ 2013-02-12 19:22       ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 19:22 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:30:11 +0000, Arnd Bergmann wrote:
> On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> > new file mode 100644
> > index 0000000..3ad563f
> > --- /dev/null
> > +++ b/drivers/pci/host/Makefile
> > @@ -0,0 +1,4 @@
> > +obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
> > +CFLAGS_pci-mvebu.o += \
> > +	-I$(srctree)/arch/arm/plat-orion/include \
> > +	-I$(srctree)/arch/arm/mach-mvebu/include
> 
> This does not seem like a good idea to me. We should not include
> architecture specific directories from a driver directory.
> 
> What are the header files you need here? 

>From the patch itself:

+#include <plat/pcie.h>
+#include <mach/addr-map.h>

<plat/pcie.h> is needed for a few PCIe functions shared with earlier
families of Marvell SoC. My plan is that once this PCI driver gets
accepted, I work on migrating the earlier Marvell SoC families to using
this PCI driver, and therefore those functions would ultimately move in
the driver in drivers/pci/host/, which would remove the <plat/pcie.h>.

The <mach/addr-map.h> is here to access the address decoding windows
allocation/free API. And for this, there is no other long term plan
than having an API provided by the platform code in arch/arm/, and used
by drivers. Some other drivers may have to use this API as well in the
future.

I think that completely preventing <mach/> and <plat/> includes from
drivers is not possible. Some sub-architectures will also have some
bizarre mechanism to handle (in our case the address decoding windows),
for which there is no kernel-wide API and kernel-wide subsystem to
handle it. In such cases, a sub-architecture specific solution is
really the only reasonable way, and in this case, we have to include
the sub-architecture headers.

Note that I have been careful to use CFLAGS_pci-mvebu.o, so that those
include paths only apply to *this* driver. I added a separate dummy
driver in drivers/pci/host/, and verified that those include paths are
not used when building this other driver. So those special CFLAGS are
still compatible with the multiplatform kernel.

> > +/*
> > + * This product ID is registered by Marvell, and used when the
> > Marvell
> > + * SoC is not the root complex, but an endpoint on the PCIe bus.
> > It is
> > + * therefore safe to re-use this PCI ID for our emulated PCI-to-PCI
> > + * bridge.
> > + */
> > +#define MARVELL_EMULATED_PCI_PCI_BRIDGE_ID 0x7846
> 
> Just a side note: What happens if you have two of these systems and
> connect them over PCIe, putting one of them into host mode and the
> other into endpoint mode?

I am not a PCI expert, but I don't think it would cause issues. Maybe
Jason Gunthorpe can comment on this, as he originally suggested to
re-use this PCI ID.

> > +static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port
> > *port,
> > +				       int enable)
> > +{
> > +	unsigned long iobase, iolimit;
> > +
> > +	if (port->bridge.iolimit < port->bridge.iobase)
> > +		return;
> > +
> > +	iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
> > +		(port->bridge.iolimitupper << 16);
> > +	iobase = ((port->bridge.iobase & 0xF0) << 8) |
> > +		(port->bridge.iobaseupper << 16);
> 
> I don't understand this code with the masks and shifts. Could you
> add a comment here for readers like me?

Sure, will do.

It basically comes from the PCI-to-PCI bridge specification, which
explains how the I/O address and I/O limit is split into two 16 bits
registers, with those bizarre shifts and hardcoded values. I'll put a
reference to the relevant section of the PCI-to-PCI bridge
specification here.

> > +
> > +/*
> > + * Initialize the configuration space of the PCI-to-PCI bridge
> > + * associated with the given PCIe interface.
> > + */
> > +static void mvebu_sw_pci_bridge_init(struct mvebu_pcie_port *port)
> > +{
> 
> As mentioned, I'm still skeptical of the sw_pci_bridge approach,
> so I'm not commenting on the details of your implementations
> (they seem fine on a first look though)

Yes, I understood your were still skeptical. But as I've mentioned in
other e-mails, I still haven't seen any other serious alternate
proposal that takes into account the need for dynamic assignment of
addresses.

> > +	/* Get the I/O and memory ranges from DT */
> > +	while ((range = of_pci_process_ranges(np, &res, range)) !=
> > NULL) {
> > +		if (resource_type(&res) == IORESOURCE_IO) {
> > +			memcpy(&pcie->io, &res, sizeof(res));
> > +			memcpy(&pcie->realio, &res, sizeof(res));
> > +			pcie->io.name = "I/O";
> > +			pcie->realio.start &= 0xFFFFF;
> > +			pcie->realio.end   &= 0xFFFFF;
> > +		}
> 
> The bit masking seems fishy here. What exactly are you doing,
> does this just assume you have a 1MB window at most?

Basically, I have two resources for the I/O:

 * One described in the DT, from 0xC0000000 to 0xC00FFFFF which will be
   used to create the address decoding windows for the I/O regions of
   the different PCIe interfaces. The PCI I/O virtual address 0xffe00000
   will be mapped to those physical addresses. Those address decoding
   windows are configured with the special "remap" mechanism that
   ensures that if an access is made at 0xC0000000 + offset, it will
   appear on the PCI bus as an I/O access at address "offset".

 * One covering the low addresses 0x0 -> 0xFFFFF (pcie->realio), which
   is used to tell the Linux PCI subsystem from which address range it
   should assign I/O addresses.

> Maybe something like
> 
> 	pcie->realio.start = 0;
> 	pcie->realio.end = pcie->io.end - pcie->io.start;

Indeed, that would result in the same values. If you find it clearer,
I'm fine with it.

> I suppose you also need to fix up pcie->io to be in IORESOURCE_MEM
> space instead of IORESOURCE_IO, or fix the of_pci_process_ranges
> function to return it in a different way.

Ok.

> > +static int mvebu_pcie_init(void)
> > +{
> > +	return platform_driver_probe(&mvebu_pcie_driver,
> > +				     mvebu_pcie_probe);
> > +}
> > +
> > +subsys_initcall(mvebu_pcie_init);
> 
> You don't have to do it, but I wonder if this could be a module
> with unload support instead.

This has already been discussed in the review of PATCHv2. Please see
http://lists.infradead.org/pipermail/linux-arm-kernel/2013-January/145580.html.

Basically, doing a module_init() initialization fails, because the XHCI
USB quirks are executed before we have the chance to create the address
decoding windows, which crashes the kernel at boot time (and we have
one platform where an USB 3.0 XHCI controller sits on the PCIe bus).
Bjorn Helgaas has acknowledged the problem in
http://lists.infradead.org/pipermail/linux-arm-kernel/2013-February/148292.html:

"""
This is not really a problem in your code; it's a generic PCI core
problem.  pci_scan_root_bus() does everything including creating the
root bus, scanning it, and adding the devices we find.  At the point
where we add a device (pci_bus_add_device()), it should be ready for a
driver to claim it -- all resource assignment should already be done.

I don't think it's completely trivial to fix this in the PCI core yet
(but we're moving in that direction) because we have some boot-time
ordering issues, e.g., x86 scans the root buses before we know about
the address space consumed by ACPI devices, so we can't just assign
the resources when we scan the bus.
"""

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-12 19:22       ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-12 19:22 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:30:11 +0000, Arnd Bergmann wrote:
> On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> > new file mode 100644
> > index 0000000..3ad563f
> > --- /dev/null
> > +++ b/drivers/pci/host/Makefile
> > @@ -0,0 +1,4 @@
> > +obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
> > +CFLAGS_pci-mvebu.o += \
> > +	-I$(srctree)/arch/arm/plat-orion/include \
> > +	-I$(srctree)/arch/arm/mach-mvebu/include
> 
> This does not seem like a good idea to me. We should not include
> architecture specific directories from a driver directory.
> 
> What are the header files you need here? 

>From the patch itself:

+#include <plat/pcie.h>
+#include <mach/addr-map.h>

<plat/pcie.h> is needed for a few PCIe functions shared with earlier
families of Marvell SoC. My plan is that once this PCI driver gets
accepted, I work on migrating the earlier Marvell SoC families to using
this PCI driver, and therefore those functions would ultimately move in
the driver in drivers/pci/host/, which would remove the <plat/pcie.h>.

The <mach/addr-map.h> is here to access the address decoding windows
allocation/free API. And for this, there is no other long term plan
than having an API provided by the platform code in arch/arm/, and used
by drivers. Some other drivers may have to use this API as well in the
future.

I think that completely preventing <mach/> and <plat/> includes from
drivers is not possible. Some sub-architectures will also have some
bizarre mechanism to handle (in our case the address decoding windows),
for which there is no kernel-wide API and kernel-wide subsystem to
handle it. In such cases, a sub-architecture specific solution is
really the only reasonable way, and in this case, we have to include
the sub-architecture headers.

Note that I have been careful to use CFLAGS_pci-mvebu.o, so that those
include paths only apply to *this* driver. I added a separate dummy
driver in drivers/pci/host/, and verified that those include paths are
not used when building this other driver. So those special CFLAGS are
still compatible with the multiplatform kernel.

> > +/*
> > + * This product ID is registered by Marvell, and used when the
> > Marvell
> > + * SoC is not the root complex, but an endpoint on the PCIe bus.
> > It is
> > + * therefore safe to re-use this PCI ID for our emulated PCI-to-PCI
> > + * bridge.
> > + */
> > +#define MARVELL_EMULATED_PCI_PCI_BRIDGE_ID 0x7846
> 
> Just a side note: What happens if you have two of these systems and
> connect them over PCIe, putting one of them into host mode and the
> other into endpoint mode?

I am not a PCI expert, but I don't think it would cause issues. Maybe
Jason Gunthorpe can comment on this, as he originally suggested to
re-use this PCI ID.

> > +static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port
> > *port,
> > +				       int enable)
> > +{
> > +	unsigned long iobase, iolimit;
> > +
> > +	if (port->bridge.iolimit < port->bridge.iobase)
> > +		return;
> > +
> > +	iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
> > +		(port->bridge.iolimitupper << 16);
> > +	iobase = ((port->bridge.iobase & 0xF0) << 8) |
> > +		(port->bridge.iobaseupper << 16);
> 
> I don't understand this code with the masks and shifts. Could you
> add a comment here for readers like me?

Sure, will do.

It basically comes from the PCI-to-PCI bridge specification, which
explains how the I/O address and I/O limit is split into two 16 bits
registers, with those bizarre shifts and hardcoded values. I'll put a
reference to the relevant section of the PCI-to-PCI bridge
specification here.

> > +
> > +/*
> > + * Initialize the configuration space of the PCI-to-PCI bridge
> > + * associated with the given PCIe interface.
> > + */
> > +static void mvebu_sw_pci_bridge_init(struct mvebu_pcie_port *port)
> > +{
> 
> As mentioned, I'm still skeptical of the sw_pci_bridge approach,
> so I'm not commenting on the details of your implementations
> (they seem fine on a first look though)

Yes, I understood your were still skeptical. But as I've mentioned in
other e-mails, I still haven't seen any other serious alternate
proposal that takes into account the need for dynamic assignment of
addresses.

> > +	/* Get the I/O and memory ranges from DT */
> > +	while ((range = of_pci_process_ranges(np, &res, range)) !=
> > NULL) {
> > +		if (resource_type(&res) == IORESOURCE_IO) {
> > +			memcpy(&pcie->io, &res, sizeof(res));
> > +			memcpy(&pcie->realio, &res, sizeof(res));
> > +			pcie->io.name = "I/O";
> > +			pcie->realio.start &= 0xFFFFF;
> > +			pcie->realio.end   &= 0xFFFFF;
> > +		}
> 
> The bit masking seems fishy here. What exactly are you doing,
> does this just assume you have a 1MB window at most?

Basically, I have two resources for the I/O:

 * One described in the DT, from 0xC0000000 to 0xC00FFFFF which will be
   used to create the address decoding windows for the I/O regions of
   the different PCIe interfaces. The PCI I/O virtual address 0xffe00000
   will be mapped to those physical addresses. Those address decoding
   windows are configured with the special "remap" mechanism that
   ensures that if an access is made at 0xC0000000 + offset, it will
   appear on the PCI bus as an I/O access at address "offset".

 * One covering the low addresses 0x0 -> 0xFFFFF (pcie->realio), which
   is used to tell the Linux PCI subsystem from which address range it
   should assign I/O addresses.

> Maybe something like
> 
> 	pcie->realio.start = 0;
> 	pcie->realio.end = pcie->io.end - pcie->io.start;

Indeed, that would result in the same values. If you find it clearer,
I'm fine with it.

> I suppose you also need to fix up pcie->io to be in IORESOURCE_MEM
> space instead of IORESOURCE_IO, or fix the of_pci_process_ranges
> function to return it in a different way.

Ok.

> > +static int mvebu_pcie_init(void)
> > +{
> > +	return platform_driver_probe(&mvebu_pcie_driver,
> > +				     mvebu_pcie_probe);
> > +}
> > +
> > +subsys_initcall(mvebu_pcie_init);
> 
> You don't have to do it, but I wonder if this could be a module
> with unload support instead.

This has already been discussed in the review of PATCHv2. Please see
http://lists.infradead.org/pipermail/linux-arm-kernel/2013-January/145580.html.

Basically, doing a module_init() initialization fails, because the XHCI
USB quirks are executed before we have the chance to create the address
decoding windows, which crashes the kernel at boot time (and we have
one platform where an USB 3.0 XHCI controller sits on the PCIe bus).
Bjorn Helgaas has acknowledged the problem in
http://lists.infradead.org/pipermail/linux-arm-kernel/2013-February/148292.html:

"""
This is not really a problem in your code; it's a generic PCI core
problem.  pci_scan_root_bus() does everything including creating the
root bus, scanning it, and adding the devices we find.  At the point
where we add a device (pci_bus_add_device()), it should be ready for a
driver to claim it -- all resource assignment should already be done.

I don't think it's completely trivial to fix this in the PCI core yet
(but we're moving in that direction) because we have some boot-time
ordering issues, e.g., x86 scans the root buses before we know about
the address space consumed by ACPI devices, so we can't just assign
the resources when we scan the bus.
"""

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 06/32] arm: pci: add a align_resource hook
  2013-02-12 19:01       ` Thomas Petazzoni
@ 2013-02-12 19:49         ` Russell King - ARM Linux
  -1 siblings, 0 replies; 291+ messages in thread
From: Russell King - ARM Linux @ 2013-02-12 19:49 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Arnd Bergmann, Bjorn Helgaas, linux-pci, linux-arm-kernel,
	Lior Amsalem, Andrew Lunn, Jason Cooper, Stephen Warren,
	Thierry Reding, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Gregory Clement, Jason Gunthorpe, Tawfik Bayouk

On Tue, Feb 12, 2013 at 08:01:18PM +0100, Thomas Petazzoni wrote:
> Dear Arnd Bergmann,
> 
> On Tue, 12 Feb 2013 18:03:12 +0000, Arnd Bergmann wrote:
> > On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > > The PCI specifications says that an I/O region must be aligned on a
> > > 4 KB boundary, and a memory region aligned on a 1 MB boundary.
> > > 
> > > However, the Marvell PCIe interfaces rely on address decoding
> > > windows (which allow to associate a range of physical addresses
> > > with a given device), and those have special requirements compared
> > > to the standard PCI-to-PCI bridge specifications.
> > 
> > I'm not convince that we should add this complexity yet, until
> > everyone agrees on the basic approach taken.
> 
> Regardless of whether we choose to have the emulated PCI-to-PCI bridges
> or not, we still need this align_resource() hook.
> 
> In the solution you propose, where each PCIe interface is represented
> as a separate PCIe domain, we still need the kernel to dynamically
> assign ranges of address to each memory BAR and I/O BAR of each PCIe
> device. And those range of address must comply with the address
> decoding windows requirements, otherwise, we don't be able to create
> those address decoding windows, and the devices will be unaccessible.

I think what Arnd is suggesting is that you treat each of your busses
as a separate root bus.

This then means that you register each bus separately via
pci_common_init() (this can take 0..N root buses and scan them.)

We can then size each bus (which means, totalling up the amount of space
each bus requires).

At that point, we know how much memory and IO space each bus requires,
and platform code can then setup the appropriate bridge windows.  The
alignment of the bases is something that platform code can deal with
in its allocation mechanism, and the resulting allocated resources
can then be assigned to each PCI bus.

Once that's done, we can then call pci_bus_assign_resources() to setup
the resources for each bus, which will use our allocated resources.

There's a downside to this though: the pci core doesn't size root buses,
because:
(a) it assumes that they're fixed size.
(b) root buses don't have a P2P 'bridge' to be sized.

I'm wondering if we could come up with a way to refactor
__pci_bus_size_bridges() so that we can get this information out of it.
That's more a question for PCI people though.

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 06/32] arm: pci: add a align_resource hook
@ 2013-02-12 19:49         ` Russell King - ARM Linux
  0 siblings, 0 replies; 291+ messages in thread
From: Russell King - ARM Linux @ 2013-02-12 19:49 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Feb 12, 2013 at 08:01:18PM +0100, Thomas Petazzoni wrote:
> Dear Arnd Bergmann,
> 
> On Tue, 12 Feb 2013 18:03:12 +0000, Arnd Bergmann wrote:
> > On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > > The PCI specifications says that an I/O region must be aligned on a
> > > 4 KB boundary, and a memory region aligned on a 1 MB boundary.
> > > 
> > > However, the Marvell PCIe interfaces rely on address decoding
> > > windows (which allow to associate a range of physical addresses
> > > with a given device), and those have special requirements compared
> > > to the standard PCI-to-PCI bridge specifications.
> > 
> > I'm not convince that we should add this complexity yet, until
> > everyone agrees on the basic approach taken.
> 
> Regardless of whether we choose to have the emulated PCI-to-PCI bridges
> or not, we still need this align_resource() hook.
> 
> In the solution you propose, where each PCIe interface is represented
> as a separate PCIe domain, we still need the kernel to dynamically
> assign ranges of address to each memory BAR and I/O BAR of each PCIe
> device. And those range of address must comply with the address
> decoding windows requirements, otherwise, we don't be able to create
> those address decoding windows, and the devices will be unaccessible.

I think what Arnd is suggesting is that you treat each of your busses
as a separate root bus.

This then means that you register each bus separately via
pci_common_init() (this can take 0..N root buses and scan them.)

We can then size each bus (which means, totalling up the amount of space
each bus requires).

At that point, we know how much memory and IO space each bus requires,
and platform code can then setup the appropriate bridge windows.  The
alignment of the bases is something that platform code can deal with
in its allocation mechanism, and the resulting allocated resources
can then be assigned to each PCI bus.

Once that's done, we can then call pci_bus_assign_resources() to setup
the resources for each bus, which will use our allocated resources.

There's a downside to this though: the pci core doesn't size root buses,
because:
(a) it assumes that they're fixed size.
(b) root buses don't have a P2P 'bridge' to be sized.

I'm wondering if we could come up with a way to refactor
__pci_bus_size_bridges() so that we can get this information out of it.
That's more a question for PCI people though.

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 19:22       ` Thomas Petazzoni
@ 2013-02-12 19:49         ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-12 19:49 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Arnd Bergmann, Bjorn Helgaas, linux-pci, linux-arm-kernel,
	Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk

On Tue, Feb 12, 2013 at 08:22:52PM +0100, Thomas Petazzoni wrote:

> > > +/*
> > > + * This product ID is registered by Marvell, and used when the
> > > Marvell
> > > + * SoC is not the root complex, but an endpoint on the PCIe bus.
> > > It is
> > > + * therefore safe to re-use this PCI ID for our emulated PCI-to-PCI
> > > + * bridge.
> > > + */
> > > +#define MARVELL_EMULATED_PCI_PCI_BRIDGE_ID 0x7846
> > 
> > Just a side note: What happens if you have two of these systems and
> > connect them over PCIe, putting one of them into host mode and the
> > other into endpoint mode?
> 
> I am not a PCI expert, but I don't think it would cause issues. Maybe
> Jason Gunthorpe can comment on this, as he originally suggested to
> re-use this PCI ID.

The answer is a bit complex..

- Yes the Marvell SOC, or related, can be used as an end point, and
  you could pair two SOCs together over PCI

- When in end point mode the SOC *probably* should have its ID
  reassigned. The ID is supposed to reflect the function of the
  device, not the HW vendor. In end point mode the ARM code controls
  how the device appears on PCIe.

  So, if someone made an add-in card based around one of these Marvell
  SOCs then it still won't expose this ID to the kernel.

- Even if the ID is reused there is a PCI-PCI bridge version and a
  end device version, so drivers could still tell them apart by class
  matching.

FWIW, I would fetch the ID from the HW, as different SOC varients may
vary the ID, though that is minor.

Basically, there is a theoretical problem here, but it is trivially
solvable in two ways by whomever decides to actually do this.

> > I suppose you also need to fix up pcie->io to be in IORESOURCE_MEM
> > space instead of IORESOURCE_IO, or fix the of_pci_process_ranges
> > function to return it in a different way.
> 
> Ok.

What does /proc/iomem say with your driver, are all the memory regions
properly requested?

FWIW, the orion mbus window driver should also get an update to
request memory regions, and IMHO, some of that code is better placed
there than in the PCI driver. Specifically, I think it should:

- Request regions for the DDR ('System RAM') and internal regs
- Request regions for every mbus window that was programmed in,
  orion_addr_map_cfg - ideally updated to decode the target specifier.
- Request and unrequest regions for the PCI driver's dynamic requests.

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-12 19:49         ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-12 19:49 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Feb 12, 2013 at 08:22:52PM +0100, Thomas Petazzoni wrote:

> > > +/*
> > > + * This product ID is registered by Marvell, and used when the
> > > Marvell
> > > + * SoC is not the root complex, but an endpoint on the PCIe bus.
> > > It is
> > > + * therefore safe to re-use this PCI ID for our emulated PCI-to-PCI
> > > + * bridge.
> > > + */
> > > +#define MARVELL_EMULATED_PCI_PCI_BRIDGE_ID 0x7846
> > 
> > Just a side note: What happens if you have two of these systems and
> > connect them over PCIe, putting one of them into host mode and the
> > other into endpoint mode?
> 
> I am not a PCI expert, but I don't think it would cause issues. Maybe
> Jason Gunthorpe can comment on this, as he originally suggested to
> re-use this PCI ID.

The answer is a bit complex..

- Yes the Marvell SOC, or related, can be used as an end point, and
  you could pair two SOCs together over PCI

- When in end point mode the SOC *probably* should have its ID
  reassigned. The ID is supposed to reflect the function of the
  device, not the HW vendor. In end point mode the ARM code controls
  how the device appears on PCIe.

  So, if someone made an add-in card based around one of these Marvell
  SOCs then it still won't expose this ID to the kernel.

- Even if the ID is reused there is a PCI-PCI bridge version and a
  end device version, so drivers could still tell them apart by class
  matching.

FWIW, I would fetch the ID from the HW, as different SOC varients may
vary the ID, though that is minor.

Basically, there is a theoretical problem here, but it is trivially
solvable in two ways by whomever decides to actually do this.

> > I suppose you also need to fix up pcie->io to be in IORESOURCE_MEM
> > space instead of IORESOURCE_IO, or fix the of_pci_process_ranges
> > function to return it in a different way.
> 
> Ok.

What does /proc/iomem say with your driver, are all the memory regions
properly requested?

FWIW, the orion mbus window driver should also get an update to
request memory regions, and IMHO, some of that code is better placed
there than in the PCI driver. Specifically, I think it should:

- Request regions for the DDR ('System RAM') and internal regs
- Request regions for every mbus window that was programmed in,
  orion_addr_map_cfg - ideally updated to decode the target specifier.
- Request and unrequest regions for the PCI driver's dynamic requests.

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 16:28   ` Thomas Petazzoni
@ 2013-02-12 22:35     ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-12 22:35 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Arnd Bergmann, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk

On Tue, Feb 12, 2013 at 05:28:58PM +0100, Thomas Petazzoni wrote:
> +++ b/Documentation/devicetree/bindings/pci/mvebu-pci.txt
[..]
> +pcie-controller {
> +	compatible = "marvell,armada-370-xp-pcie";
> +	status = "disabled";
> +
> +	#address-cells = <3>;
> +	#size-cells = <2>;
> +
> +	bus-range = <0x00 0xff>;
> +
> +	ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
> +	          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
> +	          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
> +	          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
> +	          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
> +		  0x00002800 0 0xd0080000 0xd0080000 0 0x00002000   /* port 1.0 registers */
> +	          0x00005000 0 0xd0082000 0xd0082000 0 0x00002000   /* port 3.0 registers */
> +		  0x00003000 0 0xd0084000 0xd0084000 0 0x00002000   /* port 1.1 registers */
> +		  0x00003800 0 0xd0088000 0xd0088000 0 0x00002000   /* port 1.2 registers */
> +		  0x00004000 0 0xd008C000 0xd008C000 0 0x00002000   /* port 1.3 registers */
> +		  0x81000000 0 0	  0xc0000000 0 0x00100000   /* downstream I/O */
> +		  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
> +
> +	#interrupt-cells = <1>;
> +	interrupt-map-mask = <0xf800 0 0 1>;
> +	interrupt-map = <0x0800 0 0 1 &mpic 58
> +		         0x1000 0 0 1 &mpic 59
> +			 0x1800 0 0 1 &mpic 60
> +			 0x2000 0 0 1 &mpic 61
> +			 0x2800 0 0 1 &mpic 62
> +		         0x3000 0 0 1 &mpic 63
> +			 0x3800 0 0 1 &mpic 64
> +			 0x4000 0 0 1 &mpic 65
> +			 0x4800 0 0 1 &mpic 99
> +			 0x5000 0 0 1 &mpic 103>;

This is out of date now?

> +	pcie@0,0 {
> +		device_type = "pciex";
> +		reg = <0x0800 0 0xd0040000 0 0x2000>;

It would be great to get this sorted as per my prior comments.. Maybe
like this is easy?

pcie-controller {
 compatible = "marvell,armada-370-xp-pcie";

 // Index by marvell,pcie-port ?
 regs = <0xd0040000 0x00002000
         0xd0080000 0x00002000>;

 ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
           0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */

 pcie@0,0 {
      device_type = "pci";
      reg = <0x0800 0 0 0>; // 00:01.0  (????)
      marvell,pcie-port = <0>;
 };
}

It is abusive to map the device internal per-port registers through
'0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
the OF spec.

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-12 22:35     ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-12 22:35 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Feb 12, 2013 at 05:28:58PM +0100, Thomas Petazzoni wrote:
> +++ b/Documentation/devicetree/bindings/pci/mvebu-pci.txt
[..]
> +pcie-controller {
> +	compatible = "marvell,armada-370-xp-pcie";
> +	status = "disabled";
> +
> +	#address-cells = <3>;
> +	#size-cells = <2>;
> +
> +	bus-range = <0x00 0xff>;
> +
> +	ranges = <0x00000800 0 0xd0040000 0xd0040000 0 0x00002000   /* port 0.0 registers */
> +	          0x00004800 0 0xd0042000 0xd0042000 0 0x00002000   /* port 2.0 registers */
> +	          0x00001000 0 0xd0044000 0xd0044000 0 0x00002000   /* port 0.1 registers */
> +	          0x00001800 0 0xd0048000 0xd0048000 0 0x00002000   /* port 0.2 registers */
> +	          0x00002000 0 0xd004C000 0xd004C000 0 0x00002000   /* port 0.3 registers */
> +		  0x00002800 0 0xd0080000 0xd0080000 0 0x00002000   /* port 1.0 registers */
> +	          0x00005000 0 0xd0082000 0xd0082000 0 0x00002000   /* port 3.0 registers */
> +		  0x00003000 0 0xd0084000 0xd0084000 0 0x00002000   /* port 1.1 registers */
> +		  0x00003800 0 0xd0088000 0xd0088000 0 0x00002000   /* port 1.2 registers */
> +		  0x00004000 0 0xd008C000 0xd008C000 0 0x00002000   /* port 1.3 registers */
> +		  0x81000000 0 0	  0xc0000000 0 0x00100000   /* downstream I/O */
> +		  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */
> +
> +	#interrupt-cells = <1>;
> +	interrupt-map-mask = <0xf800 0 0 1>;
> +	interrupt-map = <0x0800 0 0 1 &mpic 58
> +		         0x1000 0 0 1 &mpic 59
> +			 0x1800 0 0 1 &mpic 60
> +			 0x2000 0 0 1 &mpic 61
> +			 0x2800 0 0 1 &mpic 62
> +		         0x3000 0 0 1 &mpic 63
> +			 0x3800 0 0 1 &mpic 64
> +			 0x4000 0 0 1 &mpic 65
> +			 0x4800 0 0 1 &mpic 99
> +			 0x5000 0 0 1 &mpic 103>;

This is out of date now?

> +	pcie at 0,0 {
> +		device_type = "pciex";
> +		reg = <0x0800 0 0xd0040000 0 0x2000>;

It would be great to get this sorted as per my prior comments.. Maybe
like this is easy?

pcie-controller {
 compatible = "marvell,armada-370-xp-pcie";

 // Index by marvell,pcie-port ?
 regs = <0xd0040000 0x00002000
         0xd0080000 0x00002000>;

 ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
           0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */

 pcie at 0,0 {
      device_type = "pci";
      reg = <0x0800 0 0 0>; // 00:01.0  (????)
      marvell,pcie-port = <0>;
 };
}

It is abusive to map the device internal per-port registers through
'0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
the OF spec.

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
  2013-02-12 18:58       ` Thomas Petazzoni
@ 2013-02-12 22:36         ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 22:36 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk, Paul Gortmaker, Jesse Barnes, Yinghai Lu,
	linux-kernel

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > Any driver that requires a
> > linear mapping of I/O ports to __iomem pointers must depend
> > CONFIG_HAS_IOPORT with the current definition of that symbol (as
> > mentioned before, we should really rename that to
> > CONFIG_HAS_IOPORT_MAP). Having these functions not defined is a
> > compile time check that is necessary to ensure that all drivers have
> > the correct annotation.
> 
> I have the feeling that the problem is more complex than that. My
> understanding is that the pcim_iomap_regions() function used by
> drivers/ata/libata-sff.c can perfectly be used to map memory BARs, and
> not necessarily I/O BARs. Therefore, this driver can perfectly be used
> in an architecture where CONFIG_NO_IOPORT is selected.

That is correct.

> The thing is that pcim_iomap_regions() transparently allows to remap an
> I/O BAR is such a BAR is passed as argument, or a memory BAR if such a
> BAR is passed as argument.
> 
> Therefore, I continue to believe that the pcim_*() functions are useful
> even if the platform doesn't have CONFIG_HAS_IOPORT.

Yes, the pcim_ functions are useful in principle, but it falls back
to the __pci_ioport_map() for IORESOURCE_IO, and that needs to
return an error if CONFIG_HAS_IOPORT is not set.
I think it would be correct if you add this hunk:

diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c
index 0d83ea8..f9b6387 100644
--- a/lib/pci_iomap.c
+++ b/lib/pci_iomap.c
@@ -33,7 +33,7 @@ void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long maxlen)
                return NULL;
        if (maxlen && len > maxlen)
                len = maxlen;
-       if (flags & IORESOURCE_IO)
+       if (IS_ENABLED(CONFIG_HAS_IOPORT) && (flags & IORESOURCE_IO))
                return __pci_ioport_map(dev, start, len);
        if (flags & IORESOURCE_MEM) {
                if (flags & IORESOURCE_CACHEABLE)

in order to prevent a link error when CONFIG_HAS_IOPORT is unset.

	Arnd

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
@ 2013-02-12 22:36         ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 22:36 UTC (permalink / raw)
  To: linux-arm-kernel

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > Any driver that requires a
> > linear mapping of I/O ports to __iomem pointers must depend
> > CONFIG_HAS_IOPORT with the current definition of that symbol (as
> > mentioned before, we should really rename that to
> > CONFIG_HAS_IOPORT_MAP). Having these functions not defined is a
> > compile time check that is necessary to ensure that all drivers have
> > the correct annotation.
> 
> I have the feeling that the problem is more complex than that. My
> understanding is that the pcim_iomap_regions() function used by
> drivers/ata/libata-sff.c can perfectly be used to map memory BARs, and
> not necessarily I/O BARs. Therefore, this driver can perfectly be used
> in an architecture where CONFIG_NO_IOPORT is selected.

That is correct.

> The thing is that pcim_iomap_regions() transparently allows to remap an
> I/O BAR is such a BAR is passed as argument, or a memory BAR if such a
> BAR is passed as argument.
> 
> Therefore, I continue to believe that the pcim_*() functions are useful
> even if the platform doesn't have CONFIG_HAS_IOPORT.

Yes, the pcim_ functions are useful in principle, but it falls back
to the __pci_ioport_map() for IORESOURCE_IO, and that needs to
return an error if CONFIG_HAS_IOPORT is not set.
I think it would be correct if you add this hunk:

diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c
index 0d83ea8..f9b6387 100644
--- a/lib/pci_iomap.c
+++ b/lib/pci_iomap.c
@@ -33,7 +33,7 @@ void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long maxlen)
                return NULL;
        if (maxlen && len > maxlen)
                len = maxlen;
-       if (flags & IORESOURCE_IO)
+       if (IS_ENABLED(CONFIG_HAS_IOPORT) && (flags & IORESOURCE_IO))
                return __pci_ioport_map(dev, start, len);
        if (flags & IORESOURCE_MEM) {
                if (flags & IORESOURCE_CACHEABLE)

in order to prevent a link error when CONFIG_HAS_IOPORT is unset.

	Arnd

^ permalink raw reply related	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 19:22       ` Thomas Petazzoni
@ 2013-02-12 22:59         ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 22:59 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> On Tue, 12 Feb 2013 18:30:11 +0000, Arnd Bergmann wrote:
> > On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > > diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> > > new file mode 100644
> > > index 0000000..3ad563f
> > > --- /dev/null
> > > +++ b/drivers/pci/host/Makefile
> > > @@ -0,0 +1,4 @@
> > > +obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
> > > +CFLAGS_pci-mvebu.o += \
> > > +	-I$(srctree)/arch/arm/plat-orion/include \
> > > +	-I$(srctree)/arch/arm/mach-mvebu/include
> > 
> > This does not seem like a good idea to me. We should not include
> > architecture specific directories from a driver directory.
> > 
> > What are the header files you need here? 
> 
> From the patch itself:
> 
> +#include <plat/pcie.h>
> +#include <mach/addr-map.h>
> 
> <plat/pcie.h> is needed for a few PCIe functions shared with earlier
> families of Marvell SoC. My plan is that once this PCI driver gets
> accepted, I work on migrating the earlier Marvell SoC families to using
> this PCI driver, and therefore those functions would ultimately move in
> the driver in drivers/pci/host/, which would remove the <plat/pcie.h>.

Hmm, although it is a bit unusual, I would actually propose duplicating
that code for now, and merging a copy into the new driver right away.
This gets rid of the header file dependency and lets you just delete
the old file when the other platforms are converted.

> The <mach/addr-map.h> is here to access the address decoding windows
> allocation/free API. And for this, there is no other long term plan
> than having an API provided by the platform code in arch/arm/, and used
> by drivers. Some other drivers may have to use this API as well in the
> future.

This is harder to do, but I'm sure we can find a solution. At least
the addr-map.c code has no other dependencies than the plat/addr-map.h
header, so we are fairly free to move it elsehwere.

I can think of several approaches that I'd prefer over your approach,
although I have to admit that none of them makes me really happy:

a) arch/arm/common/mv-addr-map.c and arch/arm/include/asm/mach/mv-addr-map.h
I assume that Russell will object to this one, but I let him speak for
himself

b) drivers/bus/<something>
This would make a lot more sense if we followed the scheme I explained
in my discussion to Jason Gunthorpe, where we basically treat this
bus as a parent node in the device tree for anything that can get remapped.
Without that change, it feels a little misplaced

c) drivers/misc/
I usually object to anything put in here and would certainly prefer the
previous one over this.

> Note that I have been careful to use CFLAGS_pci-mvebu.o, so that those
> include paths only apply to *this* driver. I added a separate dummy
> driver in drivers/pci/host/, and verified that those include paths are
> not used when building this other driver. So those special CFLAGS are
> still compatible with the multiplatform kernel.

I understand this, but I think it would be better not to set a precedent
for doing this.

> 
> > > +static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port
> > > *port,
> > > +				       int enable)
> > > +{
> > > +	unsigned long iobase, iolimit;
> > > +
> > > +	if (port->bridge.iolimit < port->bridge.iobase)
> > > +		return;
> > > +
> > > +	iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
> > > +		(port->bridge.iolimitupper << 16);
> > > +	iobase = ((port->bridge.iobase & 0xF0) << 8) |
> > > +		(port->bridge.iobaseupper << 16);
> > 
> > I don't understand this code with the masks and shifts. Could you
> > add a comment here for readers like me?
> 
> Sure, will do.
> 
> It basically comes from the PCI-to-PCI bridge specification, which
> explains how the I/O address and I/O limit is split into two 16 bits
> registers, with those bizarre shifts and hardcoded values. I'll put a
> reference to the relevant section of the PCI-to-PCI bridge
> specification here.

Ok, I see. Thanks for the explanation.
 
> > > +	/* Get the I/O and memory ranges from DT */
> > > +	while ((range = of_pci_process_ranges(np, &res, range)) !=
> > > NULL) {
> > > +		if (resource_type(&res) == IORESOURCE_IO) {
> > > +			memcpy(&pcie->io, &res, sizeof(res));
> > > +			memcpy(&pcie->realio, &res, sizeof(res));
> > > +			pcie->io.name = "I/O";
> > > +			pcie->realio.start &= 0xFFFFF;
> > > +			pcie->realio.end   &= 0xFFFFF;
> > > +		}
> > 
> > The bit masking seems fishy here. What exactly are you doing,
> > does this just assume you have a 1MB window at most?
> 
> Basically, I have two resources for the I/O:
> 
>  * One described in the DT, from 0xC0000000 to 0xC00FFFFF which will be
>    used to create the address decoding windows for the I/O regions of
>    the different PCIe interfaces. The PCI I/O virtual address 0xffe00000
>    will be mapped to those physical addresses. Those address decoding
>    windows are configured with the special "remap" mechanism that
>    ensures that if an access is made at 0xC0000000 + offset, it will
>    appear on the PCI bus as an I/O access at address "offset".

Right, I got this from reading the code. Unfortunately the of_pci_process_ranges
functions from your earlier patch makes this a little confusing as it
marks this resource as IORESOURCE_IO when in reality it is the IORESOURCE_MEM
resource that describes where in MMIO space the I/O window is.

>  * One covering the low addresses 0x0 -> 0xFFFFF (pcie->realio), which
>    is used to tell the Linux PCI subsystem from which address range it
>    should assign I/O addresses.
> 
> > Maybe something like
> > 
> > 	pcie->realio.start = 0;
> > 	pcie->realio.end = pcie->io.end - pcie->io.start;
> 
> Indeed, that would result in the same values. If you find it clearer,
> I'm fine with it.

Ideally we would read that from the ranges property as well, since it
does not necessarily start at zero, although any other value would
be a bit silly.

I definitely prefer the version I suggested over your version. On second
thought I would make this

	pcie->realio.start = PCIBIOS_MIN_IO;
	pcie->realio.end = min(pcie->io.end - pcie->io.start, IO_SPACE_LIMIT);

to ensure that we are inside of the limits.

> 
> > > +static int mvebu_pcie_init(void)
> > > +{
> > > +	return platform_driver_probe(&mvebu_pcie_driver,
> > > +				     mvebu_pcie_probe);
> > > +}
> > > +
> > > +subsys_initcall(mvebu_pcie_init);
> > 
> > You don't have to do it, but I wonder if this could be a module
> > with unload support instead.
> 
> This has already been discussed in the review of PATCHv2. Please see
> http://lists.infradead.org/pipermail/linux-arm-kernel/2013-January/145580.html.

Ok.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-12 22:59         ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-12 22:59 UTC (permalink / raw)
  To: linux-arm-kernel

On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> On Tue, 12 Feb 2013 18:30:11 +0000, Arnd Bergmann wrote:
> > On Tuesday 12 February 2013, Thomas Petazzoni wrote:
> > > diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> > > new file mode 100644
> > > index 0000000..3ad563f
> > > --- /dev/null
> > > +++ b/drivers/pci/host/Makefile
> > > @@ -0,0 +1,4 @@
> > > +obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
> > > +CFLAGS_pci-mvebu.o += \
> > > +	-I$(srctree)/arch/arm/plat-orion/include \
> > > +	-I$(srctree)/arch/arm/mach-mvebu/include
> > 
> > This does not seem like a good idea to me. We should not include
> > architecture specific directories from a driver directory.
> > 
> > What are the header files you need here? 
> 
> From the patch itself:
> 
> +#include <plat/pcie.h>
> +#include <mach/addr-map.h>
> 
> <plat/pcie.h> is needed for a few PCIe functions shared with earlier
> families of Marvell SoC. My plan is that once this PCI driver gets
> accepted, I work on migrating the earlier Marvell SoC families to using
> this PCI driver, and therefore those functions would ultimately move in
> the driver in drivers/pci/host/, which would remove the <plat/pcie.h>.

Hmm, although it is a bit unusual, I would actually propose duplicating
that code for now, and merging a copy into the new driver right away.
This gets rid of the header file dependency and lets you just delete
the old file when the other platforms are converted.

> The <mach/addr-map.h> is here to access the address decoding windows
> allocation/free API. And for this, there is no other long term plan
> than having an API provided by the platform code in arch/arm/, and used
> by drivers. Some other drivers may have to use this API as well in the
> future.

This is harder to do, but I'm sure we can find a solution. At least
the addr-map.c code has no other dependencies than the plat/addr-map.h
header, so we are fairly free to move it elsehwere.

I can think of several approaches that I'd prefer over your approach,
although I have to admit that none of them makes me really happy:

a) arch/arm/common/mv-addr-map.c and arch/arm/include/asm/mach/mv-addr-map.h
I assume that Russell will object to this one, but I let him speak for
himself

b) drivers/bus/<something>
This would make a lot more sense if we followed the scheme I explained
in my discussion to Jason Gunthorpe, where we basically treat this
bus as a parent node in the device tree for anything that can get remapped.
Without that change, it feels a little misplaced

c) drivers/misc/
I usually object to anything put in here and would certainly prefer the
previous one over this.

> Note that I have been careful to use CFLAGS_pci-mvebu.o, so that those
> include paths only apply to *this* driver. I added a separate dummy
> driver in drivers/pci/host/, and verified that those include paths are
> not used when building this other driver. So those special CFLAGS are
> still compatible with the multiplatform kernel.

I understand this, but I think it would be better not to set a precedent
for doing this.

> 
> > > +static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port
> > > *port,
> > > +				       int enable)
> > > +{
> > > +	unsigned long iobase, iolimit;
> > > +
> > > +	if (port->bridge.iolimit < port->bridge.iobase)
> > > +		return;
> > > +
> > > +	iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
> > > +		(port->bridge.iolimitupper << 16);
> > > +	iobase = ((port->bridge.iobase & 0xF0) << 8) |
> > > +		(port->bridge.iobaseupper << 16);
> > 
> > I don't understand this code with the masks and shifts. Could you
> > add a comment here for readers like me?
> 
> Sure, will do.
> 
> It basically comes from the PCI-to-PCI bridge specification, which
> explains how the I/O address and I/O limit is split into two 16 bits
> registers, with those bizarre shifts and hardcoded values. I'll put a
> reference to the relevant section of the PCI-to-PCI bridge
> specification here.

Ok, I see. Thanks for the explanation.
 
> > > +	/* Get the I/O and memory ranges from DT */
> > > +	while ((range = of_pci_process_ranges(np, &res, range)) !=
> > > NULL) {
> > > +		if (resource_type(&res) == IORESOURCE_IO) {
> > > +			memcpy(&pcie->io, &res, sizeof(res));
> > > +			memcpy(&pcie->realio, &res, sizeof(res));
> > > +			pcie->io.name = "I/O";
> > > +			pcie->realio.start &= 0xFFFFF;
> > > +			pcie->realio.end   &= 0xFFFFF;
> > > +		}
> > 
> > The bit masking seems fishy here. What exactly are you doing,
> > does this just assume you have a 1MB window at most?
> 
> Basically, I have two resources for the I/O:
> 
>  * One described in the DT, from 0xC0000000 to 0xC00FFFFF which will be
>    used to create the address decoding windows for the I/O regions of
>    the different PCIe interfaces. The PCI I/O virtual address 0xffe00000
>    will be mapped to those physical addresses. Those address decoding
>    windows are configured with the special "remap" mechanism that
>    ensures that if an access is made at 0xC0000000 + offset, it will
>    appear on the PCI bus as an I/O access at address "offset".

Right, I got this from reading the code. Unfortunately the of_pci_process_ranges
functions from your earlier patch makes this a little confusing as it
marks this resource as IORESOURCE_IO when in reality it is the IORESOURCE_MEM
resource that describes where in MMIO space the I/O window is.

>  * One covering the low addresses 0x0 -> 0xFFFFF (pcie->realio), which
>    is used to tell the Linux PCI subsystem from which address range it
>    should assign I/O addresses.
> 
> > Maybe something like
> > 
> > 	pcie->realio.start = 0;
> > 	pcie->realio.end = pcie->io.end - pcie->io.start;
> 
> Indeed, that would result in the same values. If you find it clearer,
> I'm fine with it.

Ideally we would read that from the ranges property as well, since it
does not necessarily start at zero, although any other value would
be a bit silly.

I definitely prefer the version I suggested over your version. On second
thought I would make this

	pcie->realio.start = PCIBIOS_MIN_IO;
	pcie->realio.end = min(pcie->io.end - pcie->io.start, IO_SPACE_LIMIT);

to ensure that we are inside of the limits.

> 
> > > +static int mvebu_pcie_init(void)
> > > +{
> > > +	return platform_driver_probe(&mvebu_pcie_driver,
> > > +				     mvebu_pcie_probe);
> > > +}
> > > +
> > > +subsys_initcall(mvebu_pcie_init);
> > 
> > You don't have to do it, but I wonder if this could be a module
> > with unload support instead.
> 
> This has already been discussed in the review of PATCHv2. Please see
> http://lists.infradead.org/pipermail/linux-arm-kernel/2013-January/145580.html.

Ok.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 22:59         ` Arnd Bergmann
@ 2013-02-13  0:41           ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-13  0:41 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Thomas Petazzoni, Bjorn Helgaas, linux-pci, linux-arm-kernel,
	Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk

On Tue, Feb 12, 2013 at 10:59:53PM +0000, Arnd Bergmann wrote:
 
> b) drivers/bus/<something>
> This would make a lot more sense if we followed the scheme I explained
> in my discussion to Jason Gunthorpe, where we basically treat this
> bus as a parent node in the device tree for anything that can get remapped.
> Without that change, it feels a little misplaced

Also, FWIW, recall this related discussion and other possible DT
binding:

http://www.spinics.net/lists/arm-kernel/msg219992.html

drivers/bus/orion-mbus.c feels like the right option, but when I
looked at it, getting the DT binding, and full dynamicness setup
seemed like it would be best done after non-DT support was purged, and
that is somewhat contigent on getting the irqchip and timer stuff
sorted (see my first attempt at that):

https://patchwork.kernel.org/patch/1852011/

Guess it depends where you want to draw the line on cleanups before
something can be accepted..

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13  0:41           ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-13  0:41 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Feb 12, 2013 at 10:59:53PM +0000, Arnd Bergmann wrote:
 
> b) drivers/bus/<something>
> This would make a lot more sense if we followed the scheme I explained
> in my discussion to Jason Gunthorpe, where we basically treat this
> bus as a parent node in the device tree for anything that can get remapped.
> Without that change, it feels a little misplaced

Also, FWIW, recall this related discussion and other possible DT
binding:

http://www.spinics.net/lists/arm-kernel/msg219992.html

drivers/bus/orion-mbus.c feels like the right option, but when I
looked at it, getting the DT binding, and full dynamicness setup
seemed like it would be best done after non-DT support was purged, and
that is somewhat contigent on getting the irqchip and timer stuff
sorted (see my first attempt at that):

https://patchwork.kernel.org/patch/1852011/

Guess it depends where you want to draw the line on cleanups before
something can be accepted..

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 22:59         ` Arnd Bergmann
@ 2013-02-13  8:23           ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  8:23 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 22:59:53 +0000, Arnd Bergmann wrote:

> > <plat/pcie.h> is needed for a few PCIe functions shared with earlier
> > families of Marvell SoC. My plan is that once this PCI driver gets
> > accepted, I work on migrating the earlier Marvell SoC families to using
> > this PCI driver, and therefore those functions would ultimately move in
> > the driver in drivers/pci/host/, which would remove the <plat/pcie.h>.
> 
> Hmm, although it is a bit unusual, I would actually propose duplicating
> that code for now, and merging a copy into the new driver right away.
> This gets rid of the header file dependency and lets you just delete
> the old file when the other platforms are converted.

Hum, why not, but I would definitely prefer to wait for the conversion
of older platforms instead of duplicating this code. But if you feel
like it's the right solution, I'll do it.

> > The <mach/addr-map.h> is here to access the address decoding windows
> > allocation/free API. And for this, there is no other long term plan
> > than having an API provided by the platform code in arch/arm/, and used
> > by drivers. Some other drivers may have to use this API as well in the
> > future.
> 
> This is harder to do, but I'm sure we can find a solution. At least
> the addr-map.c code has no other dependencies than the plat/addr-map.h
> header, so we are fairly free to move it elsehwere.

The arch/arm/mach-mvebu/addr-map.c depends on
arch/arm/plat-orion/addr-map.c, so any change on this will affect
mach-kirkwood, mach-orion5x, mach-dove and mach-mv78xx0. As you can
see, we have to take into account the existing code, and I don't think
it's realistic to have a perfect solution immediately.

This address decoding code will continue to change for two reasons:

 *) We are going to work on NOR/NAND support for mach-mvebu, and that
    will also involve more interaction with the address decoding code.

 *) As we are moving the earlier Marvell SoC families (mach-kirkwood,
    mach-orion5x, mach-dove, mach-mv78xx0) to the Device Tree, this
    address decoding code will also evolve.

mach-mvebu is not a standalone new platform like mach-highbank for
example, it relies on a lot of existing code from ealier platforms. It
is nice to share existing code, but it also means that cleanups and
refactoring take a lot more time.

I think we need to leave time for all these platforms to gradually
converge and cleanup their infrastructure. It's not going to happen
overnight.

> I can think of several approaches that I'd prefer over your approach,
> although I have to admit that none of them makes me really happy:
> 
> a) arch/arm/common/mv-addr-map.c and arch/arm/include/asm/mach/mv-addr-map.h
> I assume that Russell will object to this one, but I let him speak for
> himself
> 
> b) drivers/bus/<something>
> This would make a lot more sense if we followed the scheme I explained
> in my discussion to Jason Gunthorpe, where we basically treat this
> bus as a parent node in the device tree for anything that can get remapped.
> Without that change, it feels a little misplaced
> 
> c) drivers/misc/
> I usually object to anything put in here and would certainly prefer the
> previous one over this.

All of those approaches require a huge amount of work to convert the
existing SoC families. Certainly, it will be done as time goes, and as
older SoC families are converted to the DT and gradually converge
inside mach-mvebu/, but if we have to wait for all of this to happen to
get the PCIe support merged, it's not going to happen before several
months (the PCIe stuff was originally posted on December, 7th,
initially with the hope of targeting 3.9, the review and rework has
taken a long time, so I'm now targeting 3.10 for the PCIe stuff, but
I'd prefer not to have to postpone this to 3.11 and even 3.12 due to
the heavy dependencies on address decoding rework).

> > > I don't understand this code with the masks and shifts. Could you
> > > add a comment here for readers like me?
> > 
> > Sure, will do.
> > 
> > It basically comes from the PCI-to-PCI bridge specification, which
> > explains how the I/O address and I/O limit is split into two 16 bits
> > registers, with those bizarre shifts and hardcoded values. I'll put a
> > reference to the relevant section of the PCI-to-PCI bridge
> > specification here.
> 
> Ok, I see. Thanks for the explanation.

I'll add a comment anyway, because it's true that it's a bit of magic
going on here.

> > > > +	/* Get the I/O and memory ranges from DT */
> > > > +	while ((range = of_pci_process_ranges(np, &res, range)) !=
> > > > NULL) {
> > > > +		if (resource_type(&res) == IORESOURCE_IO) {
> > > > +			memcpy(&pcie->io, &res, sizeof(res));
> > > > +			memcpy(&pcie->realio, &res, sizeof(res));
> > > > +			pcie->io.name = "I/O";
> > > > +			pcie->realio.start &= 0xFFFFF;
> > > > +			pcie->realio.end   &= 0xFFFFF;
> > > > +		}
> > > 
> > > The bit masking seems fishy here. What exactly are you doing,
> > > does this just assume you have a 1MB window at most?
> > 
> > Basically, I have two resources for the I/O:
> > 
> >  * One described in the DT, from 0xC0000000 to 0xC00FFFFF which will be
> >    used to create the address decoding windows for the I/O regions of
> >    the different PCIe interfaces. The PCI I/O virtual address 0xffe00000
> >    will be mapped to those physical addresses. Those address decoding
> >    windows are configured with the special "remap" mechanism that
> >    ensures that if an access is made at 0xC0000000 + offset, it will
> >    appear on the PCI bus as an I/O access at address "offset".
> 
> Right, I got this from reading the code. Unfortunately the of_pci_process_ranges
> functions from your earlier patch makes this a little confusing as it
> marks this resource as IORESOURCE_IO when in reality it is the IORESOURCE_MEM
> resource that describes where in MMIO space the I/O window is.

Indeed. So maybe I should mark this resource as being IORESOURCE_MEM
in the DT.

> >  * One covering the low addresses 0x0 -> 0xFFFFF (pcie->realio), which
> >    is used to tell the Linux PCI subsystem from which address range it
> >    should assign I/O addresses.
> > 
> > > Maybe something like
> > > 
> > > 	pcie->realio.start = 0;
> > > 	pcie->realio.end = pcie->io.end - pcie->io.start;
> > 
> > Indeed, that would result in the same values. If you find it clearer,
> > I'm fine with it.
> 
> Ideally we would read that from the ranges property as well, since it
> does not necessarily start at zero, although any other value would
> be a bit silly.
> 
> I definitely prefer the version I suggested over your version. On second
> thought I would make this
> 
> 	pcie->realio.start = PCIBIOS_MIN_IO;
> 	pcie->realio.end = min(pcie->io.end - pcie->io.start, IO_SPACE_LIMIT);
> 
> to ensure that we are inside of the limits.

Ok, thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13  8:23           ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  8:23 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 22:59:53 +0000, Arnd Bergmann wrote:

> > <plat/pcie.h> is needed for a few PCIe functions shared with earlier
> > families of Marvell SoC. My plan is that once this PCI driver gets
> > accepted, I work on migrating the earlier Marvell SoC families to using
> > this PCI driver, and therefore those functions would ultimately move in
> > the driver in drivers/pci/host/, which would remove the <plat/pcie.h>.
> 
> Hmm, although it is a bit unusual, I would actually propose duplicating
> that code for now, and merging a copy into the new driver right away.
> This gets rid of the header file dependency and lets you just delete
> the old file when the other platforms are converted.

Hum, why not, but I would definitely prefer to wait for the conversion
of older platforms instead of duplicating this code. But if you feel
like it's the right solution, I'll do it.

> > The <mach/addr-map.h> is here to access the address decoding windows
> > allocation/free API. And for this, there is no other long term plan
> > than having an API provided by the platform code in arch/arm/, and used
> > by drivers. Some other drivers may have to use this API as well in the
> > future.
> 
> This is harder to do, but I'm sure we can find a solution. At least
> the addr-map.c code has no other dependencies than the plat/addr-map.h
> header, so we are fairly free to move it elsehwere.

The arch/arm/mach-mvebu/addr-map.c depends on
arch/arm/plat-orion/addr-map.c, so any change on this will affect
mach-kirkwood, mach-orion5x, mach-dove and mach-mv78xx0. As you can
see, we have to take into account the existing code, and I don't think
it's realistic to have a perfect solution immediately.

This address decoding code will continue to change for two reasons:

 *) We are going to work on NOR/NAND support for mach-mvebu, and that
    will also involve more interaction with the address decoding code.

 *) As we are moving the earlier Marvell SoC families (mach-kirkwood,
    mach-orion5x, mach-dove, mach-mv78xx0) to the Device Tree, this
    address decoding code will also evolve.

mach-mvebu is not a standalone new platform like mach-highbank for
example, it relies on a lot of existing code from ealier platforms. It
is nice to share existing code, but it also means that cleanups and
refactoring take a lot more time.

I think we need to leave time for all these platforms to gradually
converge and cleanup their infrastructure. It's not going to happen
overnight.

> I can think of several approaches that I'd prefer over your approach,
> although I have to admit that none of them makes me really happy:
> 
> a) arch/arm/common/mv-addr-map.c and arch/arm/include/asm/mach/mv-addr-map.h
> I assume that Russell will object to this one, but I let him speak for
> himself
> 
> b) drivers/bus/<something>
> This would make a lot more sense if we followed the scheme I explained
> in my discussion to Jason Gunthorpe, where we basically treat this
> bus as a parent node in the device tree for anything that can get remapped.
> Without that change, it feels a little misplaced
> 
> c) drivers/misc/
> I usually object to anything put in here and would certainly prefer the
> previous one over this.

All of those approaches require a huge amount of work to convert the
existing SoC families. Certainly, it will be done as time goes, and as
older SoC families are converted to the DT and gradually converge
inside mach-mvebu/, but if we have to wait for all of this to happen to
get the PCIe support merged, it's not going to happen before several
months (the PCIe stuff was originally posted on December, 7th,
initially with the hope of targeting 3.9, the review and rework has
taken a long time, so I'm now targeting 3.10 for the PCIe stuff, but
I'd prefer not to have to postpone this to 3.11 and even 3.12 due to
the heavy dependencies on address decoding rework).

> > > I don't understand this code with the masks and shifts. Could you
> > > add a comment here for readers like me?
> > 
> > Sure, will do.
> > 
> > It basically comes from the PCI-to-PCI bridge specification, which
> > explains how the I/O address and I/O limit is split into two 16 bits
> > registers, with those bizarre shifts and hardcoded values. I'll put a
> > reference to the relevant section of the PCI-to-PCI bridge
> > specification here.
> 
> Ok, I see. Thanks for the explanation.

I'll add a comment anyway, because it's true that it's a bit of magic
going on here.

> > > > +	/* Get the I/O and memory ranges from DT */
> > > > +	while ((range = of_pci_process_ranges(np, &res, range)) !=
> > > > NULL) {
> > > > +		if (resource_type(&res) == IORESOURCE_IO) {
> > > > +			memcpy(&pcie->io, &res, sizeof(res));
> > > > +			memcpy(&pcie->realio, &res, sizeof(res));
> > > > +			pcie->io.name = "I/O";
> > > > +			pcie->realio.start &= 0xFFFFF;
> > > > +			pcie->realio.end   &= 0xFFFFF;
> > > > +		}
> > > 
> > > The bit masking seems fishy here. What exactly are you doing,
> > > does this just assume you have a 1MB window at most?
> > 
> > Basically, I have two resources for the I/O:
> > 
> >  * One described in the DT, from 0xC0000000 to 0xC00FFFFF which will be
> >    used to create the address decoding windows for the I/O regions of
> >    the different PCIe interfaces. The PCI I/O virtual address 0xffe00000
> >    will be mapped to those physical addresses. Those address decoding
> >    windows are configured with the special "remap" mechanism that
> >    ensures that if an access is made at 0xC0000000 + offset, it will
> >    appear on the PCI bus as an I/O access at address "offset".
> 
> Right, I got this from reading the code. Unfortunately the of_pci_process_ranges
> functions from your earlier patch makes this a little confusing as it
> marks this resource as IORESOURCE_IO when in reality it is the IORESOURCE_MEM
> resource that describes where in MMIO space the I/O window is.

Indeed. So maybe I should mark this resource as being IORESOURCE_MEM
in the DT.

> >  * One covering the low addresses 0x0 -> 0xFFFFF (pcie->realio), which
> >    is used to tell the Linux PCI subsystem from which address range it
> >    should assign I/O addresses.
> > 
> > > Maybe something like
> > > 
> > > 	pcie->realio.start = 0;
> > > 	pcie->realio.end = pcie->io.end - pcie->io.start;
> > 
> > Indeed, that would result in the same values. If you find it clearer,
> > I'm fine with it.
> 
> Ideally we would read that from the ranges property as well, since it
> does not necessarily start at zero, although any other value would
> be a bit silly.
> 
> I definitely prefer the version I suggested over your version. On second
> thought I would make this
> 
> 	pcie->realio.start = PCIBIOS_MIN_IO;
> 	pcie->realio.end = min(pcie->io.end - pcie->io.start, IO_SPACE_LIMIT);
> 
> to ensure that we are inside of the limits.

Ok, thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
  2013-02-12 18:12   ` Arnd Bergmann
@ 2013-02-13  8:50     ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  8:50 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:12:11 +0000, Arnd Bergmann wrote:

> You can add my 'Reviewed-by' tag on all paches except 5, 6, and 24.
> 
> I'm fine with adding the other ones to 3.9, as well as adding
> the binding document if you split that out from patch 24. Sorry for
> making this so hard for you.

Is this really true that some more code can be taken for 3.9 ? It
somewhat contradicts the statement that Olof has made on LAKML just a
few hours ago "arm-soc: No new material from 3.9, fixes only".

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
@ 2013-02-13  8:50     ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  8:50 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 18:12:11 +0000, Arnd Bergmann wrote:

> You can add my 'Reviewed-by' tag on all paches except 5, 6, and 24.
> 
> I'm fine with adding the other ones to 3.9, as well as adding
> the binding document if you split that out from patch 24. Sorry for
> making this so hard for you.

Is this really true that some more code can be taken for 3.9 ? It
somewhat contradicts the statement that Olof has made on LAKML just a
few hours ago "arm-soc: No new material from 3.9, fixes only".

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 22:35     ` Jason Gunthorpe
@ 2013-02-13  8:57       ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  8:57 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, linux-pci,
	Thierry Reding, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, Gregory Clement, Tawfik Bayouk,
	linux-arm-kernel

Dear Jason Gunthorpe,

On Tue, 12 Feb 2013 15:35:11 -0700, Jason Gunthorpe wrote:

> This is out of date now?

Yes, will fix.

> > +	pcie@0,0 {
> > +		device_type = "pciex";
> > +		reg = <0x0800 0 0xd0040000 0 0x2000>;
> 
> It would be great to get this sorted as per my prior comments.. Maybe
> like this is easy?
> 
> pcie-controller {
>  compatible = "marvell,armada-370-xp-pcie";
> 
>  // Index by marvell,pcie-port ?
>  regs = <0xd0040000 0x00002000
>          0xd0080000 0x00002000>;
> 
>  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
>            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> 
>  pcie@0,0 {
>       device_type = "pci";
>       reg = <0x0800 0 0 0>; // 00:01.0  (????)
>       marvell,pcie-port = <0>;
>  };
> }
> 
> It is abusive to map the device internal per-port registers through
> '0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
> the OF spec.

I am not sure to understand how this would work. Given a pcie@X,Y node,
how would I find the address of the internal registers (i.e the ones at
0xd0040000, 0xd0080000) ?

You seem to propose a global regs = <...> property under
pcie-controller, but indexing using marvell,pcie-port cannot work. PCIe
interfaces are identified by two values (port,lane), so we have 0.0,
0.1, 0.2, 0.3, 1.0, 1.1, 1.2, 1.3, 2.0 and 3.0 on MV78460. I really
would like to avoid having bizarre computations to find which entry in
this big regs = <...> array correspond to a given PCIe interface.

Could you give a more detailed example, matching the PCIe DT data of
the MV78460, which has many PCIe interfaces ?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13  8:57       ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  8:57 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Jason Gunthorpe,

On Tue, 12 Feb 2013 15:35:11 -0700, Jason Gunthorpe wrote:

> This is out of date now?

Yes, will fix.

> > +	pcie at 0,0 {
> > +		device_type = "pciex";
> > +		reg = <0x0800 0 0xd0040000 0 0x2000>;
> 
> It would be great to get this sorted as per my prior comments.. Maybe
> like this is easy?
> 
> pcie-controller {
>  compatible = "marvell,armada-370-xp-pcie";
> 
>  // Index by marvell,pcie-port ?
>  regs = <0xd0040000 0x00002000
>          0xd0080000 0x00002000>;
> 
>  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
>            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> 
>  pcie at 0,0 {
>       device_type = "pci";
>       reg = <0x0800 0 0 0>; // 00:01.0  (????)
>       marvell,pcie-port = <0>;
>  };
> }
> 
> It is abusive to map the device internal per-port registers through
> '0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
> the OF spec.

I am not sure to understand how this would work. Given a pcie at X,Y node,
how would I find the address of the internal registers (i.e the ones at
0xd0040000, 0xd0080000) ?

You seem to propose a global regs = <...> property under
pcie-controller, but indexing using marvell,pcie-port cannot work. PCIe
interfaces are identified by two values (port,lane), so we have 0.0,
0.1, 0.2, 0.3, 1.0, 1.1, 1.2, 1.3, 2.0 and 3.0 on MV78460. I really
would like to avoid having bizarre computations to find which entry in
this big regs = <...> array correspond to a given PCIe interface.

Could you give a more detailed example, matching the PCIe DT data of
the MV78460, which has many PCIe interfaces ?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13  0:41           ` Jason Gunthorpe
@ 2013-02-13  9:18             ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13  9:18 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thomas Petazzoni, Bjorn Helgaas, linux-pci, linux-arm-kernel,
	Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk

On Wednesday 13 February 2013, Jason Gunthorpe wrote:
> , Arnd Bergmann wrote:
>  
> > b) drivers/bus/<something>
> > This would make a lot more sense if we followed the scheme I explained
> > in my discussion to Jason Gunthorpe, where we basically treat this
> > bus as a parent node in the device tree for anything that can get remapped.
> > Without that change, it feels a little misplaced
> 
> Also, FWIW, recall this related discussion and other possible DT
> binding:
> 
> http://www.spinics.net/lists/arm-kernel/msg219992.html
> 
> drivers/bus/orion-mbus.c feels like the right option, but when I
> looked at it, getting the DT binding, and full dynamicness setup
> seemed like it would be best done after non-DT support was purged, and
> that is somewhat contigent on getting the irqchip and timer stuff
> sorted (see my first attempt at that):
> 
> https://patchwork.kernel.org/patch/1852011/
> 
> Guess it depends where you want to draw the line on cleanups before
> something can be accepted..

I guess as long as we agree on where we are headed with the
address translation, it's ok to just move the code now and
change the code later. I would like to be strict about the
include path stuff though.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13  9:18             ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13  9:18 UTC (permalink / raw)
  To: linux-arm-kernel

On Wednesday 13 February 2013, Jason Gunthorpe wrote:
> , Arnd Bergmann wrote:
>  
> > b) drivers/bus/<something>
> > This would make a lot more sense if we followed the scheme I explained
> > in my discussion to Jason Gunthorpe, where we basically treat this
> > bus as a parent node in the device tree for anything that can get remapped.
> > Without that change, it feels a little misplaced
> 
> Also, FWIW, recall this related discussion and other possible DT
> binding:
> 
> http://www.spinics.net/lists/arm-kernel/msg219992.html
> 
> drivers/bus/orion-mbus.c feels like the right option, but when I
> looked at it, getting the DT binding, and full dynamicness setup
> seemed like it would be best done after non-DT support was purged, and
> that is somewhat contigent on getting the irqchip and timer stuff
> sorted (see my first attempt at that):
> 
> https://patchwork.kernel.org/patch/1852011/
> 
> Guess it depends where you want to draw the line on cleanups before
> something can be accepted..

I guess as long as we agree on where we are headed with the
address translation, it's ok to just move the code now and
change the code later. I would like to be strict about the
include path stuff though.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13  8:23           ` Thomas Petazzoni
@ 2013-02-13  9:29             ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13  9:29 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Wednesday 13 February 2013, Thomas Petazzoni wrote:
> Dear Arnd Bergmann,
> 
> On Tue, 12 Feb 2013 22:59:53 +0000, Arnd Bergmann wrote:
> 
> > > <plat/pcie.h> is needed for a few PCIe functions shared with earlier
> > > families of Marvell SoC. My plan is that once this PCI driver gets
> > > accepted, I work on migrating the earlier Marvell SoC families to using
> > > this PCI driver, and therefore those functions would ultimately move in
> > > the driver in drivers/pci/host/, which would remove the <plat/pcie.h>.
> > 
> > Hmm, although it is a bit unusual, I would actually propose duplicating
> > that code for now, and merging a copy into the new driver right away.
> > This gets rid of the header file dependency and lets you just delete
> > the old file when the other platforms are converted.
> 
> Hum, why not, but I would definitely prefer to wait for the conversion
> of older platforms instead of duplicating this code. But if you feel
> like it's the right solution, I'll do it.

It's not something we do a lot, but in this case, I think it's better
if it lets us avoid adding the platform specific include path, which I
really want to avoid here.

> > > The <mach/addr-map.h> is here to access the address decoding windows
> > > allocation/free API. And for this, there is no other long term plan
> > > than having an API provided by the platform code in arch/arm/, and used
> > > by drivers. Some other drivers may have to use this API as well in the
> > > future.
> > 
> > This is harder to do, but I'm sure we can find a solution. At least
> > the addr-map.c code has no other dependencies than the plat/addr-map.h
> > header, so we are fairly free to move it elsehwere.
> 
> The arch/arm/mach-mvebu/addr-map.c depends on
> arch/arm/plat-orion/addr-map.c, so any change on this will affect
> mach-kirkwood, mach-orion5x, mach-dove and mach-mv78xx0. As you can
> see, we have to take into account the existing code, and I don't think
> it's realistic to have a perfect solution immediately.

Yes, I realize this. I was thinking we would move all at least the
file from plat-orion, and the header file. I don't care much whether
we also move the platform specific setup from mach-*/addr-map.c,
it works either way.
 
> This address decoding code will continue to change for two reasons:
> 
>  *) We are going to work on NOR/NAND support for mach-mvebu, and that
>     will also involve more interaction with the address decoding code.
> 
>  *) As we are moving the earlier Marvell SoC families (mach-kirkwood,
>     mach-orion5x, mach-dove, mach-mv78xx0) to the Device Tree, this
>     address decoding code will also evolve.
> 
> mach-mvebu is not a standalone new platform like mach-highbank for
> example, it relies on a lot of existing code from ealier platforms. It
> is nice to share existing code, but it also means that cleanups and
> refactoring take a lot more time.
> 
> I think we need to leave time for all these platforms to gradually
> converge and cleanup their infrastructure. It's not going to happen
> overnight.

We don't need to do a complete overhaul of that code right now, but
if we agree on a place where it can go, then I think we should
move it now as just one extra patch to get rid of the header
dependency. In the worst case, moving just the header file to
include/linux would work, too.

> > > 
> > >  * One described in the DT, from 0xC0000000 to 0xC00FFFFF which will be
> > >    used to create the address decoding windows for the I/O regions of
> > >    the different PCIe interfaces. The PCI I/O virtual address 0xffe00000
> > >    will be mapped to those physical addresses. Those address decoding
> > >    windows are configured with the special "remap" mechanism that
> > >    ensures that if an access is made at 0xC0000000 + offset, it will
> > >    appear on the PCI bus as an I/O access at address "offset".
> > 
> > Right, I got this from reading the code. Unfortunately the of_pci_process_ranges
> > functions from your earlier patch makes this a little confusing as it
> > marks this resource as IORESOURCE_IO when in reality it is the IORESOURCE_MEM
> > resource that describes where in MMIO space the I/O window is.
> 
> Indeed. So maybe I should mark this resource as being IORESOURCE_MEM
> in the DT.

The DT seems fine here, just the code that interprets it is a little
unusual. Maybe you can change the calling convention of that function
to pass the type of resource you want as an argument?


	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13  9:29             ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13  9:29 UTC (permalink / raw)
  To: linux-arm-kernel

On Wednesday 13 February 2013, Thomas Petazzoni wrote:
> Dear Arnd Bergmann,
> 
> On Tue, 12 Feb 2013 22:59:53 +0000, Arnd Bergmann wrote:
> 
> > > <plat/pcie.h> is needed for a few PCIe functions shared with earlier
> > > families of Marvell SoC. My plan is that once this PCI driver gets
> > > accepted, I work on migrating the earlier Marvell SoC families to using
> > > this PCI driver, and therefore those functions would ultimately move in
> > > the driver in drivers/pci/host/, which would remove the <plat/pcie.h>.
> > 
> > Hmm, although it is a bit unusual, I would actually propose duplicating
> > that code for now, and merging a copy into the new driver right away.
> > This gets rid of the header file dependency and lets you just delete
> > the old file when the other platforms are converted.
> 
> Hum, why not, but I would definitely prefer to wait for the conversion
> of older platforms instead of duplicating this code. But if you feel
> like it's the right solution, I'll do it.

It's not something we do a lot, but in this case, I think it's better
if it lets us avoid adding the platform specific include path, which I
really want to avoid here.

> > > The <mach/addr-map.h> is here to access the address decoding windows
> > > allocation/free API. And for this, there is no other long term plan
> > > than having an API provided by the platform code in arch/arm/, and used
> > > by drivers. Some other drivers may have to use this API as well in the
> > > future.
> > 
> > This is harder to do, but I'm sure we can find a solution. At least
> > the addr-map.c code has no other dependencies than the plat/addr-map.h
> > header, so we are fairly free to move it elsehwere.
> 
> The arch/arm/mach-mvebu/addr-map.c depends on
> arch/arm/plat-orion/addr-map.c, so any change on this will affect
> mach-kirkwood, mach-orion5x, mach-dove and mach-mv78xx0. As you can
> see, we have to take into account the existing code, and I don't think
> it's realistic to have a perfect solution immediately.

Yes, I realize this. I was thinking we would move all at least the
file from plat-orion, and the header file. I don't care much whether
we also move the platform specific setup from mach-*/addr-map.c,
it works either way.
 
> This address decoding code will continue to change for two reasons:
> 
>  *) We are going to work on NOR/NAND support for mach-mvebu, and that
>     will also involve more interaction with the address decoding code.
> 
>  *) As we are moving the earlier Marvell SoC families (mach-kirkwood,
>     mach-orion5x, mach-dove, mach-mv78xx0) to the Device Tree, this
>     address decoding code will also evolve.
> 
> mach-mvebu is not a standalone new platform like mach-highbank for
> example, it relies on a lot of existing code from ealier platforms. It
> is nice to share existing code, but it also means that cleanups and
> refactoring take a lot more time.
> 
> I think we need to leave time for all these platforms to gradually
> converge and cleanup their infrastructure. It's not going to happen
> overnight.

We don't need to do a complete overhaul of that code right now, but
if we agree on a place where it can go, then I think we should
move it now as just one extra patch to get rid of the header
dependency. In the worst case, moving just the header file to
include/linux would work, too.

> > > 
> > >  * One described in the DT, from 0xC0000000 to 0xC00FFFFF which will be
> > >    used to create the address decoding windows for the I/O regions of
> > >    the different PCIe interfaces. The PCI I/O virtual address 0xffe00000
> > >    will be mapped to those physical addresses. Those address decoding
> > >    windows are configured with the special "remap" mechanism that
> > >    ensures that if an access is made at 0xC0000000 + offset, it will
> > >    appear on the PCI bus as an I/O access at address "offset".
> > 
> > Right, I got this from reading the code. Unfortunately the of_pci_process_ranges
> > functions from your earlier patch makes this a little confusing as it
> > marks this resource as IORESOURCE_IO when in reality it is the IORESOURCE_MEM
> > resource that describes where in MMIO space the I/O window is.
> 
> Indeed. So maybe I should mark this resource as being IORESOURCE_MEM
> in the DT.

The DT seems fine here, just the code that interprets it is a little
unusual. Maybe you can change the calling convention of that function
to pass the type of resource you want as an argument?


	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13  9:18             ` Arnd Bergmann
@ 2013-02-13  9:31               ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  9:31 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Jason Gunthorpe, Bjorn Helgaas, linux-pci, linux-arm-kernel,
	Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk

Dear Arnd Bergmann,

On Wed, 13 Feb 2013 09:18:56 +0000, Arnd Bergmann wrote:

> > Also, FWIW, recall this related discussion and other possible DT
> > binding:
> > 
> > http://www.spinics.net/lists/arm-kernel/msg219992.html
> > 
> > drivers/bus/orion-mbus.c feels like the right option, but when I
> > looked at it, getting the DT binding, and full dynamicness setup
> > seemed like it would be best done after non-DT support was purged, and
> > that is somewhat contigent on getting the irqchip and timer stuff
> > sorted (see my first attempt at that):
> > 
> > https://patchwork.kernel.org/patch/1852011/
> > 
> > Guess it depends where you want to draw the line on cleanups before
> > something can be accepted..
> 
> I guess as long as we agree on where we are headed with the
> address translation, it's ok to just move the code now and
> change the code later. I would like to be strict about the
> include path stuff though.

Ok, so I could move the existing address decoding code into say
drivers/bus/orion-mbus.c. Would that be ok for you?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13  9:31               ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  9:31 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Wed, 13 Feb 2013 09:18:56 +0000, Arnd Bergmann wrote:

> > Also, FWIW, recall this related discussion and other possible DT
> > binding:
> > 
> > http://www.spinics.net/lists/arm-kernel/msg219992.html
> > 
> > drivers/bus/orion-mbus.c feels like the right option, but when I
> > looked at it, getting the DT binding, and full dynamicness setup
> > seemed like it would be best done after non-DT support was purged, and
> > that is somewhat contigent on getting the irqchip and timer stuff
> > sorted (see my first attempt at that):
> > 
> > https://patchwork.kernel.org/patch/1852011/
> > 
> > Guess it depends where you want to draw the line on cleanups before
> > something can be accepted..
> 
> I guess as long as we agree on where we are headed with the
> address translation, it's ok to just move the code now and
> change the code later. I would like to be strict about the
> include path stuff though.

Ok, so I could move the existing address decoding code into say
drivers/bus/orion-mbus.c. Would that be ok for you?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
  2013-02-13  8:50     ` Thomas Petazzoni
@ 2013-02-13  9:37       ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13  9:37 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Wednesday 13 February 2013, Thomas Petazzoni wrote:
> On Tue, 12 Feb 2013 18:12:11 +0000, Arnd Bergmann wrote:
> 
> > You can add my 'Reviewed-by' tag on all paches except 5, 6, and 24.
> > 
> > I'm fine with adding the other ones to 3.9, as well as adding
> > the binding document if you split that out from patch 24. Sorry for
> > making this so hard for you.
> 
> Is this really true that some more code can be taken for 3.9 ? It
> somewhat contradicts the statement that Olof has made on LAKML just a
> few hours ago "arm-soc: No new material from 3.9, fixes only".

Sorry, that was bad communiation between Olof and me. Olof has
been handling basically all of the pulls for this release
and his decision is final. The only part of the series that
I believe we should take is patch 7, since that is a bug fix.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
@ 2013-02-13  9:37       ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13  9:37 UTC (permalink / raw)
  To: linux-arm-kernel

On Wednesday 13 February 2013, Thomas Petazzoni wrote:
> On Tue, 12 Feb 2013 18:12:11 +0000, Arnd Bergmann wrote:
> 
> > You can add my 'Reviewed-by' tag on all paches except 5, 6, and 24.
> > 
> > I'm fine with adding the other ones to 3.9, as well as adding
> > the binding document if you split that out from patch 24. Sorry for
> > making this so hard for you.
> 
> Is this really true that some more code can be taken for 3.9 ? It
> somewhat contradicts the statement that Olof has made on LAKML just a
> few hours ago "arm-soc: No new material from 3.9, fixes only".

Sorry, that was bad communiation between Olof and me. Olof has
been handling basically all of the pulls for this release
and his decision is final. The only part of the series that
I believe we should take is patch 7, since that is a bug fix.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13  9:29             ` Arnd Bergmann
@ 2013-02-13  9:40               ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  9:40 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

Dear Arnd Bergmann,

On Wed, 13 Feb 2013 09:29:21 +0000, Arnd Bergmann wrote:

> > Hum, why not, but I would definitely prefer to wait for the conversion
> > of older platforms instead of duplicating this code. But if you feel
> > like it's the right solution, I'll do it.
> 
> It's not something we do a lot, but in this case, I think it's better
> if it lets us avoid adding the platform specific include path, which I
> really want to avoid here.

Ok.

> > The arch/arm/mach-mvebu/addr-map.c depends on
> > arch/arm/plat-orion/addr-map.c, so any change on this will affect
> > mach-kirkwood, mach-orion5x, mach-dove and mach-mv78xx0. As you can
> > see, we have to take into account the existing code, and I don't think
> > it's realistic to have a perfect solution immediately.
> 
> Yes, I realize this. I was thinking we would move all at least the
> file from plat-orion, and the header file. I don't care much whether
> we also move the platform specific setup from mach-*/addr-map.c,
> it works either way.

Ok.


> We don't need to do a complete overhaul of that code right now, but
> if we agree on a place where it can go, then I think we should
> move it now as just one extra patch to get rid of the header
> dependency. In the worst case, moving just the header file to
> include/linux would work, too.

I'll try to cook something for this.

> > Indeed. So maybe I should mark this resource as being IORESOURCE_MEM
> > in the DT.
> 
> The DT seems fine here, just the code that interprets it is a little
> unusual. Maybe you can change the calling convention of that function
> to pass the type of resource you want as an argument?

Erm? The type of the resource is encoded into the DT:

+				  0x81000000 0 0	  0xc0000000 0 0x00010000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */

>From http://devicetree.org/Device_Tree_Usage#PCI_Host_Bridge:

     phys.hi cell: npt000ss bbbbbbbb dddddfff rrrrrrrr
    phys.mid cell: hhhhhhhh hhhhhhhh hhhhhhhh hhhhhhhh
    phys.low cell: llllllll llllllll llllllll llllllll 

 ss: space code

    00: configuration space
    01: I/O space
    10: 32 bit memory space
    11: 64 bit memory space 

So the 0x81 at the beginning of the first line means I/O space, the
0x82 at the beginning of the second line means 32 bits memory space.
The of_pci_process_ranges() function simply decodes those informations
and fills the struct resource it returns with the appropriate resource
type.

So passing the resource type as argument to of_pci_process_ranges()
doesn't make much sense to me.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13  9:40               ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13  9:40 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Wed, 13 Feb 2013 09:29:21 +0000, Arnd Bergmann wrote:

> > Hum, why not, but I would definitely prefer to wait for the conversion
> > of older platforms instead of duplicating this code. But if you feel
> > like it's the right solution, I'll do it.
> 
> It's not something we do a lot, but in this case, I think it's better
> if it lets us avoid adding the platform specific include path, which I
> really want to avoid here.

Ok.

> > The arch/arm/mach-mvebu/addr-map.c depends on
> > arch/arm/plat-orion/addr-map.c, so any change on this will affect
> > mach-kirkwood, mach-orion5x, mach-dove and mach-mv78xx0. As you can
> > see, we have to take into account the existing code, and I don't think
> > it's realistic to have a perfect solution immediately.
> 
> Yes, I realize this. I was thinking we would move all at least the
> file from plat-orion, and the header file. I don't care much whether
> we also move the platform specific setup from mach-*/addr-map.c,
> it works either way.

Ok.


> We don't need to do a complete overhaul of that code right now, but
> if we agree on a place where it can go, then I think we should
> move it now as just one extra patch to get rid of the header
> dependency. In the worst case, moving just the header file to
> include/linux would work, too.

I'll try to cook something for this.

> > Indeed. So maybe I should mark this resource as being IORESOURCE_MEM
> > in the DT.
> 
> The DT seems fine here, just the code that interprets it is a little
> unusual. Maybe you can change the calling convention of that function
> to pass the type of resource you want as an argument?

Erm? The type of the resource is encoded into the DT:

+				  0x81000000 0 0	  0xc0000000 0 0x00010000   /* downstream I/O */
+				  0x82000000 0 0	  0xc1000000 0 0x08000000>; /* non-prefetchable memory */

>From http://devicetree.org/Device_Tree_Usage#PCI_Host_Bridge:

     phys.hi cell: npt000ss bbbbbbbb dddddfff rrrrrrrr
    phys.mid cell: hhhhhhhh hhhhhhhh hhhhhhhh hhhhhhhh
    phys.low cell: llllllll llllllll llllllll llllllll 

 ss: space code

    00: configuration space
    01: I/O space
    10: 32 bit memory space
    11: 64 bit memory space 

So the 0x81 at the beginning of the first line means I/O space, the
0x82 at the beginning of the second line means 32 bits memory space.
The of_pci_process_ranges() function simply decodes those informations
and fills the struct resource it returns with the appropriate resource
type.

So passing the resource type as argument to of_pci_process_ranges()
doesn't make much sense to me.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13  9:31               ` Thomas Petazzoni
@ 2013-02-13 10:23                 ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13 10:23 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Jason Gunthorpe, Bjorn Helgaas, linux-pci, linux-arm-kernel,
	Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk

On Wednesday 13 February 2013, Thomas Petazzoni wrote:
> > I guess as long as we agree on where we are headed with the
> > address translation, it's ok to just move the code now and
> > change the code later. I would like to be strict about the
> > include path stuff though.
> 
> Ok, so I could move the existing address decoding code into say
> drivers/bus/orion-mbus.c. Would that be ok for you?

Sure, sounds good to me.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13 10:23                 ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13 10:23 UTC (permalink / raw)
  To: linux-arm-kernel

On Wednesday 13 February 2013, Thomas Petazzoni wrote:
> > I guess as long as we agree on where we are headed with the
> > address translation, it's ok to just move the code now and
> > change the code later. I would like to be strict about the
> > include path stuff though.
> 
> Ok, so I could move the existing address decoding code into say
> drivers/bus/orion-mbus.c. Would that be ok for you?

Sure, sounds good to me.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13  9:40               ` Thomas Petazzoni
@ 2013-02-13 10:37                 ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13 10:37 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Wednesday 13 February 2013, Thomas Petazzoni wrote:
> > > Indeed. So maybe I should mark this resource as being IORESOURCE_MEM
> > > in the DT.
> > 
> > The DT seems fine here, just the code that interprets it is a little
> > unusual. Maybe you can change the calling convention of that function
> > to pass the type of resource you want as an argument?
> 
> Erm? The type of the resource is encoded into the DT:
> 
> +                                 0x81000000 0 0          0xc0000000 0 0x00010000   /* downstream I/O */
> +                                 0x82000000 0 0          0xc1000000 0 0x08000000>; /* non-prefetchable memory */
> 
> From http://devicetree.org/Device_Tree_Usage#PCI_Host_Bridge:
> 
>      phys.hi cell: npt000ss bbbbbbbb dddddfff rrrrrrrr
>     phys.mid cell: hhhhhhhh hhhhhhhh hhhhhhhh hhhhhhhh
>     phys.low cell: llllllll llllllll llllllll llllllll 
> 
>  ss: space code
> 
>     00: configuration space
>     01: I/O space
>     10: 32 bit memory space
>     11: 64 bit memory space 
> 
> So the 0x81 at the beginning of the first line means I/O space, the
> 0x82 at the beginning of the second line means 32 bits memory space.
> The of_pci_process_ranges() function simply decodes those informations
> and fills the struct resource it returns with the appropriate resource
> type.

Yes, of course. And the ranges property tells you how to turn the first
address space into the second address space. So the above property
defines that the PCI bus I/O space range from 0 to 0x10000 gets converted
into the host MMIO range 0xc0000000 to 0xc0010000 on the host, and
the PCI bus memory space range from 0 to 0x08000000 gets converted to
the host MMIO range 0xc1000000 to 0xc9000000.

The output of your of_pci_process_ranges() function is the host MMIO
range, not the range in the bus address space, so it has to be
IORESOURCE_MEM.

Note that the memory range you have given in the example is a little
unconventional. Usually we try to have an identity mapping between
PCI bus memory space addresses and host MMIO space, which would
result in a zero sys->mem_offset value in addition to the sys->io_offset
value that is already zero.

Is there a limitation with the way that your address translation
windows are set up? What I would hope you could do is a memory window
like 

          <0x82000000 0 0xc1000000   0xc1000000 0 0x08000000>; /* non-prefetchable memory */

that translates the host MMIO range 0xc1000000-0xc9000000 into  bus
address 0xc1000000-0xc9000000 instead of bus address 0x00000000-0x08000000.
Would it help if you switch the addresses around, so that the physical
location of the memory space window is naturally aligned at
0xc0000000-0xc8000000 and the I/O space window is at 0xc8000000-0xc8100000?

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13 10:37                 ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13 10:37 UTC (permalink / raw)
  To: linux-arm-kernel

On Wednesday 13 February 2013, Thomas Petazzoni wrote:
> > > Indeed. So maybe I should mark this resource as being IORESOURCE_MEM
> > > in the DT.
> > 
> > The DT seems fine here, just the code that interprets it is a little
> > unusual. Maybe you can change the calling convention of that function
> > to pass the type of resource you want as an argument?
> 
> Erm? The type of the resource is encoded into the DT:
> 
> +                                 0x81000000 0 0          0xc0000000 0 0x00010000   /* downstream I/O */
> +                                 0x82000000 0 0          0xc1000000 0 0x08000000>; /* non-prefetchable memory */
> 
> From http://devicetree.org/Device_Tree_Usage#PCI_Host_Bridge:
> 
>      phys.hi cell: npt000ss bbbbbbbb dddddfff rrrrrrrr
>     phys.mid cell: hhhhhhhh hhhhhhhh hhhhhhhh hhhhhhhh
>     phys.low cell: llllllll llllllll llllllll llllllll 
> 
>  ss: space code
> 
>     00: configuration space
>     01: I/O space
>     10: 32 bit memory space
>     11: 64 bit memory space 
> 
> So the 0x81 at the beginning of the first line means I/O space, the
> 0x82 at the beginning of the second line means 32 bits memory space.
> The of_pci_process_ranges() function simply decodes those informations
> and fills the struct resource it returns with the appropriate resource
> type.

Yes, of course. And the ranges property tells you how to turn the first
address space into the second address space. So the above property
defines that the PCI bus I/O space range from 0 to 0x10000 gets converted
into the host MMIO range 0xc0000000 to 0xc0010000 on the host, and
the PCI bus memory space range from 0 to 0x08000000 gets converted to
the host MMIO range 0xc1000000 to 0xc9000000.

The output of your of_pci_process_ranges() function is the host MMIO
range, not the range in the bus address space, so it has to be
IORESOURCE_MEM.

Note that the memory range you have given in the example is a little
unconventional. Usually we try to have an identity mapping between
PCI bus memory space addresses and host MMIO space, which would
result in a zero sys->mem_offset value in addition to the sys->io_offset
value that is already zero.

Is there a limitation with the way that your address translation
windows are set up? What I would hope you could do is a memory window
like 

          <0x82000000 0 0xc1000000   0xc1000000 0 0x08000000>; /* non-prefetchable memory */

that translates the host MMIO range 0xc1000000-0xc9000000 into  bus
address 0xc1000000-0xc9000000 instead of bus address 0x00000000-0x08000000.
Would it help if you switch the addresses around, so that the physical
location of the memory space window is naturally aligned at
0xc0000000-0xc8000000 and the I/O space window is at 0xc8000000-0xc8100000?

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
  2013-02-12 16:28 ` Thomas Petazzoni
@ 2013-02-13 15:27   ` Christophe Vu-Brugier
  -1 siblings, 0 replies; 291+ messages in thread
From: Christophe Vu-Brugier @ 2013-02-13 15:27 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Arnd Bergmann, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk, Jason Gunthorpe

Hi Thomas,

On Tue, 12 Feb 2013 17:28:34 +0100, Thomas Petazzoni wrote :
> This series of patches introduces PCIe support for the Marvell Armada
> 370 and Armada XP.

Tested on a Marvell DB-MV784MP-GP evaluation board with a Marvell 9220
SATA controller. Disks attached to this controller are working.

Tested-by: Christophe Vu-Brugier <cvubrugier@lacie.com>

  # /usr/sbin/lspci -v -s 01:00.0
  01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9220 (rev 10) (prog-if 01 [AHCI 1.0])
          Subsystem: Marvell Technology Group Ltd. Device 9220
          Flags: bus master, fast devsel, latency 0, IRQ 105
          I/O ports at 10020 [size=8]
          I/O ports at 10030 [size=4]
          I/O ports at 10028 [size=8]
          I/O ports at 10034 [size=4]
          I/O ports at 10000 [size=32]
          Memory at c1000000 (32-bit, non-prefetchable) [size=2K]
          Expansion ROM at c1100000 [disabled] [size=64K]
          Capabilities: [40] Power Management version 3
          Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
          Capabilities: [70] Express Legacy Endpoint, MSI 00
          Capabilities: [e0] SATA HBA v0.0
          Capabilities: [100] Advanced Error Reporting
          Kernel driver in use: ahci


> 
> This PATCHv3 follows:
>  * PATCHv2, sent on January, 28th 2013
>  * RFCv1, sent on December, 7th 2012
> 
> Thanks to all the people who discussed on the previous version of the
> patch set. The discussions have been long and complicated, but
> certainly very useful.
> 
> In order to make progress on this, and help reducing the size of the
> patch set, I would ask if it would be possible to merge patches 1 to
> 17 from the series for 3.9 (only preparation work), keeping the rest
> for 3.10. The patches in question are PCI-related, ARM-related, and
> mvebu/orion-related.
> 
> Changes between v2 and v3:
> 
>  * Use of_irq_map_pci() instead of of_irq_map_raw(), as suggested by
>    Andrew Murray. In order to do this, we moved the interrupt-map and
>    interrupt-map-mask DT properties from the main PCIe controller node
>    to the DT subnodes representing each PCIe interface.
> 
>  * Remove the usage of the emulated host bridge.
> 
>  * Move the emulated PCI-to-PCI bridge code into the Marvell PCI
>    driver itself, in order to allow a tighter integration. Suggested
>    by Bjorn Helgaas and Jason Gunthorpe.
> 
>  * Make the allocation of address decoding windows dynamic: it's when
>    memory accesses or I/O accesses are enabled at the PCI-to-PCI
>    bridge level that we allocate and setup the corresponding address
>    decoding window. Requested by Bjorn Helgaas.
> 
>  * Fixed the implementation of I/O accesses to use I/O addresses that
>    fall within the normal IO_SPACE_LIMIT. This required using the
>    "remap" functionality of address decoding windows, and therefore
>    some changes in the address decoding window allocator. Follows a
>    long discussion about I/O accesses.
> 
>  * Set up a correct bus number in the configuration of the PCIe
>    interfaces so that we don't have to fake bus numbers
>    anymore. Requested by Jason Gunthorpe.
> 
>  * Fix the of_pci_get_devfn() implementation according to Stephen
>    Warren's comment.
> 
>  * Use CFLAGS_ instead of ccflags to add the mach-mvebu and plat-orion
>    include paths when building the pci-mvebu driver. This ensures that
>    the include paths are only added when building this specific
>    driver. Requested by Stephen Warren.
> 
>  * Fix the ->resource_align() to only apply on bus 0 (the one on which
>    the emulated PCI-to-PCI bridges sit), and to request an alignment
>    on the size of the window (and not only 64 KB for I/O windows and 1
>    MB for memory windows).
> 
>  * Clarified the commit log of "clk: mvebu: create parent-child
>    relation for PCIe clocks on Armada 370"
> 
> A quick description of the patches:
> 
>  * Patches 1 to 3 add PCI-related Device Tree parsing functions. Those
>    patches are common with the Nvidia Tegra PCIe patch set from
>    Thierry Redding. They are included in this series so that it can be
>    tested easily.
> 
>  * Patch 4 extends the ARM PCI core to store a per-controller private
>    data pointer. This patch is common with the Nvidia Tegra PCIe patch
>    set from Thierry Redding. It is included in this series so that it
>    can be tested easily.
> 
>  * Patch 5 fixes a problem in lib/devres.c that prevents certain
>    PCI-related functions from being visible on NO_IOPORT platforms. I
>    know this patch isn't acceptable by itself, but the discussion
>    about this has been so huge and went in so many directions that in
>    the end, I don't know what is the correct way of fixing this. If an
>    agreement is found on how to fix this properly, I'm willing to work
>    on it if needed.
> 
>  * Patch 6 extends the ARM PCI core with an additional hook that a PCI
>    controller driver can register and get called to realign PCI
>    ressource addresses. This is needed for the support of Marvell PCIe
>    interfaces because the address decoding windows for I/O ranges have
>    a granularity of 64 KB, while the PCI standard requires only a 4 KB
>    alignement. See the patch itself for details.
> 
>  * Patch 7 fixes a mistake in the interrupt controller node of the
>    Armada 370/XP Device Tree, which was invisible until we started
>    using the of_irq_map_raw() function, needed in our PCIe support.
> 
>  * Patches 8 and 9 fix some issues in the Armada 370/XP clock gating
>    driver, related to PCIe interfaces.
> 
>  * Patches 10 and 11 are cleanup/refactoring of the common plat-orion
>    address decoding code, in preparation for further changes related
>    to PCIe.
> 
>  * Patches 12 to 17 introduce a ORION_ADDR_MAP_NO_REMAP define that is
>    used by existing Marvell SoC code to say "I don't need this window
>    to remap anything". Previously a -1 value was used as the remap
>    address to communicate the fact that no remap is needed, but this
>    prevents any remap address higher than 2 GB.
> 
>  * Patch 18 removes __init from a few address window decoding
>    functions that are now needed after boot.
> 
>  * Patch 19 introduces in the common plat-orion address decoding code
>    functions to allocate/free an address decoding window. Until now,
>    the address decoding windows were configured statically. With
>    Armada XP having up to 10 PCIe interfaces, we don't want to
>    allocate useless address decoding windows statically, so we move to
>    a more dynamic model in which address decoding windows are
>    configured only for the PCIe interfaces that are actually in use.
> 
>  * Patch 20 removes __init from a few PCIe functions that are now
>    needed after boot.
> 
>  * Patch 21 improves the Armada 370/XP specific address decoding code
>    to provide functions that add and remove an address decoding window
>    for a given PCIe interface. It relies on the common functions added
>    in patch 19.
> 
>  * Patch 22 makes the common plat-orion PCIe code available on
>    PLAT_ORION platforms such as ARCH_MVEBU.
> 
>  * Patch 23 creates the drivers/pci/host directory and makes the
>    related minimal changes to Kconfig/Makefile. This patch will
>    trivially conflict with the NVidia Tegra PCIe support posted by
>    Thierry Redding, which also creates the drivers/pci/host directory.
> 
>  * Patch 24 contains the Armada 370/XP PCIe driver itself, that
>    implements the necessary operations required by the ARM PCI core,
>    and configures the address decoding windows as needed. This driver
>    relies on a Device Tree description of the PCIe interfaces.
> 
>  * Patch 25 marks the ARCH_MVEBU platform has having PCI available,
>    which allows the compilation of the PCIe support.
> 
>  * Patches 26 and 27 add the SoC-level Device Tree informations
>    related to PCIe for Armada 370 and Armada XP.
> 
>  * Patch 28 to 31 add the board-level Device Tree informations related
>    to PCIe for the Armada XP DB, Armada 370 DB, PlatHome OpenBlocks
>    AX3-4 and GlobalScale Mirabox boards.
> 
>  * Patch 32 updates mvebu_defconfig with PCI and USB support.
> 
> This patch set applies on top of v3.8-rc7, and has been pushed
> at:
> 
>   git://github.com/MISL-EBU-System-SW/mainline-public.git marvell-pcie-v3
> 
> Thanks,
> 
> Thomas
> 
> ---
> Output of lspci -vvv:
> 
> 00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
> 	I/O behind bridge: 0000f000-00000fff
> 	Memory behind bridge: fff00000-000fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
> 	I/O behind bridge: 0000f000-00000fff
> 	Memory behind bridge: fff00000-000fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:03.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
> 	I/O behind bridge: 00010000-00010fff
> 	Memory behind bridge: c1000000-c10fffff
> 	Prefetchable memory behind bridge: c1100000-c11fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:04.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
> 	I/O behind bridge: 0000f000-00000fff
> 	Memory behind bridge: fff00000-000fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:05.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
> 	I/O behind bridge: 00020000-00020fff
> 	Memory behind bridge: c1200000-c12fffff
> 	Prefetchable memory behind bridge: c1300000-c13fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:06.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
> 	I/O behind bridge: 0000f000-00000fff
> 	Memory behind bridge: fff00000-000fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 03:00.0 SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 PCI-e 4-port SATA-II (rev 02)
> 	Subsystem: Marvell Technology Group Ltd. Device 11ab
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Interrupt: pin A routed to IRQ 105
> 	Region 0: Memory at c1000000 (64-bit, non-prefetchable) [size=1M]
> 	Region 2: I/O ports at 10000 [size=256]
> 	[virtual] Expansion ROM at c1100000 [disabled] [size=512K]
> 	Capabilities: [40] Power Management version 2
> 		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
> 		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> 	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
> 		Address: 0000000012345678  Data: 0000
> 	Capabilities: [60] Express (v1) Legacy Endpoint, MSI 00
> 		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <256ns, L1 <1us
> 			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
> 		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> 			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
> 			MaxPayload 128 bytes, MaxReadReq 512 bytes
> 		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
> 		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <256ns, L1 unlimited
> 			ClockPM- Surprise- LLActRep- BwNot-
> 		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
> 			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> 		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> 	Capabilities: [100 v1] Advanced Error Reporting
> 		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> 		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> 		UESvrt:	DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
> 		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
> 		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
> 	Kernel driver in use: sata_mv
> 
> 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
> 	Subsystem: Intel Corporation PRO/1000 PT Server Adapter
> 	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Interrupt: pin A routed to IRQ 106
> 	Region 0: Memory at c1200000 (32-bit, non-prefetchable) [size=128K]
> 	Region 1: Memory at c1220000 (32-bit, non-prefetchable) [size=128K]
> 	Region 2: I/O ports at 20000 [disabled] [size=32]
> 	[virtual] Expansion ROM at c1300000 [disabled] [size=128K]
> 	Capabilities: [c8] Power Management version 2
> 		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
> 		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
> 	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
> 		Address: 0000000000000000  Data: 0000
> 	Capabilities: [e0] Express (v1) Endpoint, MSI 00
> 		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
> 			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
> 		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> 			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> 			MaxPayload 128 bytes, MaxReadReq 512 bytes
> 		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
> 		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us
> 			ClockPM- Surprise- LLActRep- BwNot-
> 		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
> 			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> 		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> 	Capabilities: [100 v1] Advanced Error Reporting
> 		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> 		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> 		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
> 		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
> 		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
> 	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-c1-c4-fe
> 	Kernel driver in use: e1000e
> 
> Boot messages from the PCI subsystem:
> 
> mvebu-pcie pcie-controller.1: PCIe0.0: link down
> mvebu-pcie pcie-controller.1: PCIe0.1: link down
> mvebu-pcie pcie-controller.1: PCIe0.2: link up
> mvebu-pcie pcie-controller.1: PCIe0.3: link down
> mvebu-pcie pcie-controller.1: PCIe2.0: link up
> mvebu-pcie pcie-controller.1: PCIe3.0: link down
> mvebu-pcie pcie-controller.1: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0x0000-0xfffff]
> pci_bus 0000:00: root bus resource [mem 0xc1000000-0xc8ffffff]
> pci_bus 0000:00: root bus resource [bus 00-ff]
> PCI: bus0: Fast back to back transfers disabled
> pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:06.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> PCI: bus1: Fast back to back transfers enabled
> PCI: bus2: Fast back to back transfers enabled
> PCI: bus3: Fast back to back transfers disabled
> PCI: bus4: Fast back to back transfers enabled
> PCI: bus5: Fast back to back transfers disabled
> PCI: bus6: Fast back to back transfers enabled
> pci 0000:00:03.0: BAR 8: assigned [mem 0xc1000000-0xc10fffff]
> pci 0000:00:03.0: BAR 9: assigned [mem 0xc1100000-0xc11fffff pref]
> pci 0000:00:05.0: BAR 8: assigned [mem 0xc1200000-0xc12fffff]
> pci 0000:00:05.0: BAR 9: assigned [mem 0xc1300000-0xc13fffff pref]
> pci 0000:00:03.0: BAR 7: assigned [io  0x10000-0x10fff]
> pci 0000:00:05.0: BAR 7: assigned [io  0x20000-0x20fff]
> pci 0000:00:01.0: PCI bridge to [bus 01]
> pci 0000:00:02.0: PCI bridge to [bus 02]
> pci 0000:03:00.0: BAR 0: assigned [mem 0xc1000000-0xc10fffff 64bit]
> pci 0000:03:00.0: BAR 6: assigned [mem 0xc1100000-0xc117ffff pref]
> pci 0000:03:00.0: BAR 2: assigned [io  0x10000-0x100ff]
> pci 0000:00:03.0: PCI bridge to [bus 03]
> pci 0000:00:03.0:   bridge window [io  0x10000-0x10fff]
> pci 0000:00:03.0:   bridge window [mem 0xc1000000-0xc10fffff]
> pci 0000:00:03.0:   bridge window [mem 0xc1100000-0xc11fffff pref]
> pci 0000:00:04.0: PCI bridge to [bus 04]
> pci 0000:05:00.0: BAR 0: assigned [mem 0xc1200000-0xc121ffff]
> pci 0000:05:00.0: BAR 1: assigned [mem 0xc1220000-0xc123ffff]
> pci 0000:05:00.0: BAR 6: assigned [mem 0xc1300000-0xc131ffff pref]
> pci 0000:05:00.0: BAR 2: assigned [io  0x20000-0x2001f]
> pci 0000:00:05.0: PCI bridge to [bus 05]
> pci 0000:00:05.0:   bridge window [io  0x20000-0x20fff]
> pci 0000:00:05.0:   bridge window [mem 0xc1200000-0xc12fffff]
> pci 0000:00:05.0:   bridge window [mem 0xc1300000-0xc13fffff pref]
> pci 0000:00:06.0: PCI bridge to [bus 06]
> PCI: enabling device 0000:00:01.0 (0140 -> 0143)
> PCI: enabling device 0000:00:02.0 (0140 -> 0143)
> PCI: enabling device 0000:00:03.0 (0140 -> 0143)
> PCI: enabling device 0000:00:04.0 (0140 -> 0143)
> PCI: enabling device 0000:00:05.0 (0140 -> 0143)
> PCI: enabling device 0000:00:06.0 (0140 -> 0143)
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


-- 
Christophe Vu-Brugier

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
@ 2013-02-13 15:27   ` Christophe Vu-Brugier
  0 siblings, 0 replies; 291+ messages in thread
From: Christophe Vu-Brugier @ 2013-02-13 15:27 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Thomas,

On Tue, 12 Feb 2013 17:28:34 +0100, Thomas Petazzoni wrote :
> This series of patches introduces PCIe support for the Marvell Armada
> 370 and Armada XP.

Tested on a Marvell DB-MV784MP-GP evaluation board with a Marvell 9220
SATA controller. Disks attached to this controller are working.

Tested-by: Christophe Vu-Brugier <cvubrugier@lacie.com>

  # /usr/sbin/lspci -v -s 01:00.0
  01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9220 (rev 10) (prog-if 01 [AHCI 1.0])
          Subsystem: Marvell Technology Group Ltd. Device 9220
          Flags: bus master, fast devsel, latency 0, IRQ 105
          I/O ports at 10020 [size=8]
          I/O ports at 10030 [size=4]
          I/O ports at 10028 [size=8]
          I/O ports at 10034 [size=4]
          I/O ports at 10000 [size=32]
          Memory at c1000000 (32-bit, non-prefetchable) [size=2K]
          Expansion ROM at c1100000 [disabled] [size=64K]
          Capabilities: [40] Power Management version 3
          Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
          Capabilities: [70] Express Legacy Endpoint, MSI 00
          Capabilities: [e0] SATA HBA v0.0
          Capabilities: [100] Advanced Error Reporting
          Kernel driver in use: ahci


> 
> This PATCHv3 follows:
>  * PATCHv2, sent on January, 28th 2013
>  * RFCv1, sent on December, 7th 2012
> 
> Thanks to all the people who discussed on the previous version of the
> patch set. The discussions have been long and complicated, but
> certainly very useful.
> 
> In order to make progress on this, and help reducing the size of the
> patch set, I would ask if it would be possible to merge patches 1 to
> 17 from the series for 3.9 (only preparation work), keeping the rest
> for 3.10. The patches in question are PCI-related, ARM-related, and
> mvebu/orion-related.
> 
> Changes between v2 and v3:
> 
>  * Use of_irq_map_pci() instead of of_irq_map_raw(), as suggested by
>    Andrew Murray. In order to do this, we moved the interrupt-map and
>    interrupt-map-mask DT properties from the main PCIe controller node
>    to the DT subnodes representing each PCIe interface.
> 
>  * Remove the usage of the emulated host bridge.
> 
>  * Move the emulated PCI-to-PCI bridge code into the Marvell PCI
>    driver itself, in order to allow a tighter integration. Suggested
>    by Bjorn Helgaas and Jason Gunthorpe.
> 
>  * Make the allocation of address decoding windows dynamic: it's when
>    memory accesses or I/O accesses are enabled at the PCI-to-PCI
>    bridge level that we allocate and setup the corresponding address
>    decoding window. Requested by Bjorn Helgaas.
> 
>  * Fixed the implementation of I/O accesses to use I/O addresses that
>    fall within the normal IO_SPACE_LIMIT. This required using the
>    "remap" functionality of address decoding windows, and therefore
>    some changes in the address decoding window allocator. Follows a
>    long discussion about I/O accesses.
> 
>  * Set up a correct bus number in the configuration of the PCIe
>    interfaces so that we don't have to fake bus numbers
>    anymore. Requested by Jason Gunthorpe.
> 
>  * Fix the of_pci_get_devfn() implementation according to Stephen
>    Warren's comment.
> 
>  * Use CFLAGS_ instead of ccflags to add the mach-mvebu and plat-orion
>    include paths when building the pci-mvebu driver. This ensures that
>    the include paths are only added when building this specific
>    driver. Requested by Stephen Warren.
> 
>  * Fix the ->resource_align() to only apply on bus 0 (the one on which
>    the emulated PCI-to-PCI bridges sit), and to request an alignment
>    on the size of the window (and not only 64 KB for I/O windows and 1
>    MB for memory windows).
> 
>  * Clarified the commit log of "clk: mvebu: create parent-child
>    relation for PCIe clocks on Armada 370"
> 
> A quick description of the patches:
> 
>  * Patches 1 to 3 add PCI-related Device Tree parsing functions. Those
>    patches are common with the Nvidia Tegra PCIe patch set from
>    Thierry Redding. They are included in this series so that it can be
>    tested easily.
> 
>  * Patch 4 extends the ARM PCI core to store a per-controller private
>    data pointer. This patch is common with the Nvidia Tegra PCIe patch
>    set from Thierry Redding. It is included in this series so that it
>    can be tested easily.
> 
>  * Patch 5 fixes a problem in lib/devres.c that prevents certain
>    PCI-related functions from being visible on NO_IOPORT platforms. I
>    know this patch isn't acceptable by itself, but the discussion
>    about this has been so huge and went in so many directions that in
>    the end, I don't know what is the correct way of fixing this. If an
>    agreement is found on how to fix this properly, I'm willing to work
>    on it if needed.
> 
>  * Patch 6 extends the ARM PCI core with an additional hook that a PCI
>    controller driver can register and get called to realign PCI
>    ressource addresses. This is needed for the support of Marvell PCIe
>    interfaces because the address decoding windows for I/O ranges have
>    a granularity of 64 KB, while the PCI standard requires only a 4 KB
>    alignement. See the patch itself for details.
> 
>  * Patch 7 fixes a mistake in the interrupt controller node of the
>    Armada 370/XP Device Tree, which was invisible until we started
>    using the of_irq_map_raw() function, needed in our PCIe support.
> 
>  * Patches 8 and 9 fix some issues in the Armada 370/XP clock gating
>    driver, related to PCIe interfaces.
> 
>  * Patches 10 and 11 are cleanup/refactoring of the common plat-orion
>    address decoding code, in preparation for further changes related
>    to PCIe.
> 
>  * Patches 12 to 17 introduce a ORION_ADDR_MAP_NO_REMAP define that is
>    used by existing Marvell SoC code to say "I don't need this window
>    to remap anything". Previously a -1 value was used as the remap
>    address to communicate the fact that no remap is needed, but this
>    prevents any remap address higher than 2 GB.
> 
>  * Patch 18 removes __init from a few address window decoding
>    functions that are now needed after boot.
> 
>  * Patch 19 introduces in the common plat-orion address decoding code
>    functions to allocate/free an address decoding window. Until now,
>    the address decoding windows were configured statically. With
>    Armada XP having up to 10 PCIe interfaces, we don't want to
>    allocate useless address decoding windows statically, so we move to
>    a more dynamic model in which address decoding windows are
>    configured only for the PCIe interfaces that are actually in use.
> 
>  * Patch 20 removes __init from a few PCIe functions that are now
>    needed after boot.
> 
>  * Patch 21 improves the Armada 370/XP specific address decoding code
>    to provide functions that add and remove an address decoding window
>    for a given PCIe interface. It relies on the common functions added
>    in patch 19.
> 
>  * Patch 22 makes the common plat-orion PCIe code available on
>    PLAT_ORION platforms such as ARCH_MVEBU.
> 
>  * Patch 23 creates the drivers/pci/host directory and makes the
>    related minimal changes to Kconfig/Makefile. This patch will
>    trivially conflict with the NVidia Tegra PCIe support posted by
>    Thierry Redding, which also creates the drivers/pci/host directory.
> 
>  * Patch 24 contains the Armada 370/XP PCIe driver itself, that
>    implements the necessary operations required by the ARM PCI core,
>    and configures the address decoding windows as needed. This driver
>    relies on a Device Tree description of the PCIe interfaces.
> 
>  * Patch 25 marks the ARCH_MVEBU platform has having PCI available,
>    which allows the compilation of the PCIe support.
> 
>  * Patches 26 and 27 add the SoC-level Device Tree informations
>    related to PCIe for Armada 370 and Armada XP.
> 
>  * Patch 28 to 31 add the board-level Device Tree informations related
>    to PCIe for the Armada XP DB, Armada 370 DB, PlatHome OpenBlocks
>    AX3-4 and GlobalScale Mirabox boards.
> 
>  * Patch 32 updates mvebu_defconfig with PCI and USB support.
> 
> This patch set applies on top of v3.8-rc7, and has been pushed
> at:
> 
>   git://github.com/MISL-EBU-System-SW/mainline-public.git marvell-pcie-v3
> 
> Thanks,
> 
> Thomas
> 
> ---
> Output of lspci -vvv:
> 
> 00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
> 	I/O behind bridge: 0000f000-00000fff
> 	Memory behind bridge: fff00000-000fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
> 	I/O behind bridge: 0000f000-00000fff
> 	Memory behind bridge: fff00000-000fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:03.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
> 	I/O behind bridge: 00010000-00010fff
> 	Memory behind bridge: c1000000-c10fffff
> 	Prefetchable memory behind bridge: c1100000-c11fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:04.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
> 	I/O behind bridge: 0000f000-00000fff
> 	Memory behind bridge: fff00000-000fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:05.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
> 	I/O behind bridge: 00020000-00020fff
> 	Memory behind bridge: c1200000-c12fffff
> 	Prefetchable memory behind bridge: c1300000-c13fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 00:06.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode])
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
> 	I/O behind bridge: 0000f000-00000fff
> 	Memory behind bridge: fff00000-000fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> 	BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+
> 		PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+
> 	Capabilities: [fc] <chain broken>
> 
> 03:00.0 SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 PCI-e 4-port SATA-II (rev 02)
> 	Subsystem: Marvell Technology Group Ltd. Device 11ab
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Interrupt: pin A routed to IRQ 105
> 	Region 0: Memory at c1000000 (64-bit, non-prefetchable) [size=1M]
> 	Region 2: I/O ports at 10000 [size=256]
> 	[virtual] Expansion ROM at c1100000 [disabled] [size=512K]
> 	Capabilities: [40] Power Management version 2
> 		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
> 		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> 	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
> 		Address: 0000000012345678  Data: 0000
> 	Capabilities: [60] Express (v1) Legacy Endpoint, MSI 00
> 		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <256ns, L1 <1us
> 			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
> 		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> 			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
> 			MaxPayload 128 bytes, MaxReadReq 512 bytes
> 		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
> 		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <256ns, L1 unlimited
> 			ClockPM- Surprise- LLActRep- BwNot-
> 		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
> 			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> 		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> 	Capabilities: [100 v1] Advanced Error Reporting
> 		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> 		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> 		UESvrt:	DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
> 		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
> 		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
> 	Kernel driver in use: sata_mv
> 
> 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
> 	Subsystem: Intel Corporation PRO/1000 PT Server Adapter
> 	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 0, Cache Line Size: 64 bytes
> 	Interrupt: pin A routed to IRQ 106
> 	Region 0: Memory at c1200000 (32-bit, non-prefetchable) [size=128K]
> 	Region 1: Memory at c1220000 (32-bit, non-prefetchable) [size=128K]
> 	Region 2: I/O ports at 20000 [disabled] [size=32]
> 	[virtual] Expansion ROM at c1300000 [disabled] [size=128K]
> 	Capabilities: [c8] Power Management version 2
> 		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
> 		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
> 	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
> 		Address: 0000000000000000  Data: 0000
> 	Capabilities: [e0] Express (v1) Endpoint, MSI 00
> 		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
> 			ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
> 		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> 			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> 			MaxPayload 128 bytes, MaxReadReq 512 bytes
> 		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
> 		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us
> 			ClockPM- Surprise- LLActRep- BwNot-
> 		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
> 			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> 		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> 	Capabilities: [100 v1] Advanced Error Reporting
> 		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> 		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
> 		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
> 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
> 		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
> 		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
> 	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-c1-c4-fe
> 	Kernel driver in use: e1000e
> 
> Boot messages from the PCI subsystem:
> 
> mvebu-pcie pcie-controller.1: PCIe0.0: link down
> mvebu-pcie pcie-controller.1: PCIe0.1: link down
> mvebu-pcie pcie-controller.1: PCIe0.2: link up
> mvebu-pcie pcie-controller.1: PCIe0.3: link down
> mvebu-pcie pcie-controller.1: PCIe2.0: link up
> mvebu-pcie pcie-controller.1: PCIe3.0: link down
> mvebu-pcie pcie-controller.1: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0x0000-0xfffff]
> pci_bus 0000:00: root bus resource [mem 0xc1000000-0xc8ffffff]
> pci_bus 0000:00: root bus resource [bus 00-ff]
> PCI: bus0: Fast back to back transfers disabled
> pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:06.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> PCI: bus1: Fast back to back transfers enabled
> PCI: bus2: Fast back to back transfers enabled
> PCI: bus3: Fast back to back transfers disabled
> PCI: bus4: Fast back to back transfers enabled
> PCI: bus5: Fast back to back transfers disabled
> PCI: bus6: Fast back to back transfers enabled
> pci 0000:00:03.0: BAR 8: assigned [mem 0xc1000000-0xc10fffff]
> pci 0000:00:03.0: BAR 9: assigned [mem 0xc1100000-0xc11fffff pref]
> pci 0000:00:05.0: BAR 8: assigned [mem 0xc1200000-0xc12fffff]
> pci 0000:00:05.0: BAR 9: assigned [mem 0xc1300000-0xc13fffff pref]
> pci 0000:00:03.0: BAR 7: assigned [io  0x10000-0x10fff]
> pci 0000:00:05.0: BAR 7: assigned [io  0x20000-0x20fff]
> pci 0000:00:01.0: PCI bridge to [bus 01]
> pci 0000:00:02.0: PCI bridge to [bus 02]
> pci 0000:03:00.0: BAR 0: assigned [mem 0xc1000000-0xc10fffff 64bit]
> pci 0000:03:00.0: BAR 6: assigned [mem 0xc1100000-0xc117ffff pref]
> pci 0000:03:00.0: BAR 2: assigned [io  0x10000-0x100ff]
> pci 0000:00:03.0: PCI bridge to [bus 03]
> pci 0000:00:03.0:   bridge window [io  0x10000-0x10fff]
> pci 0000:00:03.0:   bridge window [mem 0xc1000000-0xc10fffff]
> pci 0000:00:03.0:   bridge window [mem 0xc1100000-0xc11fffff pref]
> pci 0000:00:04.0: PCI bridge to [bus 04]
> pci 0000:05:00.0: BAR 0: assigned [mem 0xc1200000-0xc121ffff]
> pci 0000:05:00.0: BAR 1: assigned [mem 0xc1220000-0xc123ffff]
> pci 0000:05:00.0: BAR 6: assigned [mem 0xc1300000-0xc131ffff pref]
> pci 0000:05:00.0: BAR 2: assigned [io  0x20000-0x2001f]
> pci 0000:00:05.0: PCI bridge to [bus 05]
> pci 0000:00:05.0:   bridge window [io  0x20000-0x20fff]
> pci 0000:00:05.0:   bridge window [mem 0xc1200000-0xc12fffff]
> pci 0000:00:05.0:   bridge window [mem 0xc1300000-0xc13fffff pref]
> pci 0000:00:06.0: PCI bridge to [bus 06]
> PCI: enabling device 0000:00:01.0 (0140 -> 0143)
> PCI: enabling device 0000:00:02.0 (0140 -> 0143)
> PCI: enabling device 0000:00:03.0 (0140 -> 0143)
> PCI: enabling device 0000:00:04.0 (0140 -> 0143)
> PCI: enabling device 0000:00:05.0 (0140 -> 0143)
> PCI: enabling device 0000:00:06.0 (0140 -> 0143)
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


-- 
Christophe Vu-Brugier

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
  2013-02-13 15:27   ` Christophe Vu-Brugier
@ 2013-02-13 15:30     ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13 15:30 UTC (permalink / raw)
  To: Christophe Vu-Brugier
  Cc: Bjorn Helgaas, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Arnd Bergmann, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk, Jason Gunthorpe

Dear Christophe Vu-Brugier,

On Wed, 13 Feb 2013 16:27:50 +0100, Christophe Vu-Brugier wrote:
> Hi Thomas,
> 
> On Tue, 12 Feb 2013 17:28:34 +0100, Thomas Petazzoni wrote :
> > This series of patches introduces PCIe support for the Marvell Armada
> > 370 and Armada XP.
> 
> Tested on a Marvell DB-MV784MP-GP evaluation board with a Marvell 9220
> SATA controller. Disks attached to this controller are working.
> 
> Tested-by: Christophe Vu-Brugier <cvubrugier@lacie.com>

Awesome, thanks a lot again Christophe for having taken the time to
test this patch series!

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs
@ 2013-02-13 15:30     ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-13 15:30 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Christophe Vu-Brugier,

On Wed, 13 Feb 2013 16:27:50 +0100, Christophe Vu-Brugier wrote:
> Hi Thomas,
> 
> On Tue, 12 Feb 2013 17:28:34 +0100, Thomas Petazzoni wrote :
> > This series of patches introduces PCIe support for the Marvell Armada
> > 370 and Armada XP.
> 
> Tested on a Marvell DB-MV784MP-GP evaluation board with a Marvell 9220
> SATA controller. Disks attached to this controller are working.
> 
> Tested-by: Christophe Vu-Brugier <cvubrugier@lacie.com>

Awesome, thanks a lot again Christophe for having taken the time to
test this patch series!

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13  8:57       ` Thomas Petazzoni
@ 2013-02-13 18:04         ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-13 18:04 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, linux-pci,
	Thierry Reding, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, Gregory Clement, Tawfik Bayouk,
	linux-arm-kernel

On Wed, Feb 13, 2013 at 09:57:01AM +0100, Thomas Petazzoni wrote:

> > > +	pcie@0,0 {
> > > +		device_type = "pciex";
> > > +		reg = <0x0800 0 0xd0040000 0 0x2000>;
> > 
> > It would be great to get this sorted as per my prior comments.. Maybe
> > like this is easy?
> > 
> > pcie-controller {
> >  compatible = "marvell,armada-370-xp-pcie";
> > 
> >  // Index by marvell,pcie-port ?
> >  regs = <0xd0040000 0x00002000
> >          0xd0080000 0x00002000>;
> > 
> >  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
> >            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> > 
> >  pcie@0,0 {
> >       device_type = "pci";
> >       reg = <0x0800 0 0 0>; // 00:01.0  (????)
> >       marvell,pcie-port = <0>;
> >  };
> > }
> > 
> > It is abusive to map the device internal per-port registers through
> > '0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
> > the OF spec.
> 
> I am not sure to understand how this would work. Given a pcie@X,Y node,
> how would I find the address of the internal registers (i.e the ones at
> 0xd0040000, 0xd0080000) ?

> You seem to propose a global regs = <...> property under
> pcie-controller, but indexing using marvell,pcie-port cannot work. PCIe
> interfaces are identified by two values (port,lane), so we have 0.0,
> 0.1, 0.2, 0.3, 1.0, 1.1, 1.2, 1.3, 2.0 and 3.0 on MV78460. I really
> would like to avoid having bizarre computations to find which entry in
> this big regs = <...> array correspond to a given PCIe interface.

Well, using the 'global' regs is an easy way to avoid the complexity with
address translation and 'assigned-addresses'. So some way to translate
between link # and regs index is necessary. How about using the PCI
device number? Device number 1 is regs entry 0, 2 is regs entry 1, etc.

The code flow would be then to iterate over DT children, extract the
bus/dev/function from 'reg', ensure bus ==0, and index the global regs
by device-1.

The other option is to try to make assigned-addresses work, Thierry
had some comments on that, it is a bit convoluted too, IMHO.

Also note setting the device_type = "pci";

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13 18:04         ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-13 18:04 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 13, 2013 at 09:57:01AM +0100, Thomas Petazzoni wrote:

> > > +	pcie at 0,0 {
> > > +		device_type = "pciex";
> > > +		reg = <0x0800 0 0xd0040000 0 0x2000>;
> > 
> > It would be great to get this sorted as per my prior comments.. Maybe
> > like this is easy?
> > 
> > pcie-controller {
> >  compatible = "marvell,armada-370-xp-pcie";
> > 
> >  // Index by marvell,pcie-port ?
> >  regs = <0xd0040000 0x00002000
> >          0xd0080000 0x00002000>;
> > 
> >  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
> >            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> > 
> >  pcie at 0,0 {
> >       device_type = "pci";
> >       reg = <0x0800 0 0 0>; // 00:01.0  (????)
> >       marvell,pcie-port = <0>;
> >  };
> > }
> > 
> > It is abusive to map the device internal per-port registers through
> > '0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
> > the OF spec.
> 
> I am not sure to understand how this would work. Given a pcie at X,Y node,
> how would I find the address of the internal registers (i.e the ones at
> 0xd0040000, 0xd0080000) ?

> You seem to propose a global regs = <...> property under
> pcie-controller, but indexing using marvell,pcie-port cannot work. PCIe
> interfaces are identified by two values (port,lane), so we have 0.0,
> 0.1, 0.2, 0.3, 1.0, 1.1, 1.2, 1.3, 2.0 and 3.0 on MV78460. I really
> would like to avoid having bizarre computations to find which entry in
> this big regs = <...> array correspond to a given PCIe interface.

Well, using the 'global' regs is an easy way to avoid the complexity with
address translation and 'assigned-addresses'. So some way to translate
between link # and regs index is necessary. How about using the PCI
device number? Device number 1 is regs entry 0, 2 is regs entry 1, etc.

The code flow would be then to iterate over DT children, extract the
bus/dev/function from 'reg', ensure bus ==0, and index the global regs
by device-1.

The other option is to try to make assigned-addresses work, Thierry
had some comments on that, it is a bit convoluted too, IMHO.

Also note setting the device_type = "pci";

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13 18:04         ` Jason Gunthorpe
@ 2013-02-13 19:33           ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13 19:33 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Stephen Warren,
	linux-pci, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, linux-arm-kernel

On Wednesday 13 February 2013, Jason Gunthorpe wrote:
> > You seem to propose a global regs = <...> property under
> > pcie-controller, but indexing using marvell,pcie-port cannot work. PCIe
> > interfaces are identified by two values (port,lane), so we have 0.0,
> > 0.1, 0.2, 0.3, 1.0, 1.1, 1.2, 1.3, 2.0 and 3.0 on MV78460. I really
> > would like to avoid having bizarre computations to find which entry in
> > this big regs = <...> array correspond to a given PCIe interface.
> 
> Well, using the 'global' regs is an easy way to avoid the complexity with
> address translation and 'assigned-addresses'. So some way to translate
> between link # and regs index is necessary. How about using the PCI
> device number? Device number 1 is regs entry 0, 2 is regs entry 1, etc.

Or you could use the "reg-names" property.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-13 19:33           ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-02-13 19:33 UTC (permalink / raw)
  To: linux-arm-kernel

On Wednesday 13 February 2013, Jason Gunthorpe wrote:
> > You seem to propose a global regs = <...> property under
> > pcie-controller, but indexing using marvell,pcie-port cannot work. PCIe
> > interfaces are identified by two values (port,lane), so we have 0.0,
> > 0.1, 0.2, 0.3, 1.0, 1.1, 1.2, 1.3, 2.0 and 3.0 on MV78460. I really
> > would like to avoid having bizarre computations to find which entry in
> > this big regs = <...> array correspond to a given PCIe interface.
> 
> Well, using the 'global' regs is an easy way to avoid the complexity with
> address translation and 'assigned-addresses'. So some way to translate
> between link # and regs index is necessary. How about using the PCI
> device number? Device number 1 is regs entry 0, 2 is regs entry 1, etc.

Or you could use the "reg-names" property.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 16:28   ` Thomas Petazzoni
@ 2013-02-15  0:36     ` Bjorn Helgaas
  -1 siblings, 0 replies; 291+ messages in thread
From: Bjorn Helgaas @ 2013-02-15  0:36 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: linux-pci, linux-arm-kernel, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Tue, Feb 12, 2013 at 9:28 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:

> --- a/drivers/pci/host/Kconfig
> +++ b/drivers/pci/host/Kconfig
> @@ -1,4 +1,10 @@
>  menu "PCI host controller drivers"
>         depends on PCI
>
> +config PCI_MVEBU
> +       bool "Marvell EBU PCIe controller"
> +       depends on ARCH_MVEBU
> +       select PCI_SW_HOST_BRIDGE
> +       select PCI_SW_PCI_PCI_BRIDGE

I think PCI_SW_HOST_BRIDGE and PCI_SW_PCI_PCI_BRIDGE are obsolete and
can be removed, right?

> --- /dev/null
> +++ b/drivers/pci/host/pci-mvebu.c
> ...
> +static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port *port,
> +                                      int enable)
> +{
> +       unsigned long iobase, iolimit;
> +
> +       if (port->bridge.iolimit < port->bridge.iobase)
> +               return;
> +
> +       iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
> +               (port->bridge.iolimitupper << 16);
> +       iobase = ((port->bridge.iobase & 0xF0) << 8) |
> +               (port->bridge.iobaseupper << 16);
> +
> +       if (enable) {
> +               unsigned long physbase = port->pcie->io.start + iobase;
> +               armada_370_xp_alloc_pcie_window(port->port, port->lane,
> +                                               physbase, iobase,
> +                                               iolimit-iobase,
> +                                               IORESOURCE_IO);
> +               pci_ioremap_io(iobase, physbase);
> +       }
> +       else
> +               armada_370_xp_free_pcie_window(iobase);
> +}
> +
> +static void mvebu_pcie_setup_mem_window(struct mvebu_pcie_port *port,
> +                                       int enable)
> +{
> +       unsigned long membase, memlimit;
> +
> +       if (port->bridge.memlimit < port->bridge.membase)
> +               return;
> +
> +       membase = ((port->bridge.membase & 0xFFF0) << 16);
> +       memlimit = ((port->bridge.memlimit & 0xFFF0) << 16) | 0xFFFFF;
> +
> +       if (enable)
> +               armada_370_xp_alloc_pcie_window(port->port, port->lane,
> +                                               membase, ORION_ADDR_MAP_NO_REMAP,
> +                                               memlimit-membase,
> +                                               IORESOURCE_MEM);
> +       else
> +               armada_370_xp_free_pcie_window(membase);
> +}
> +
> +static void mvebu_handle_pcie_command(struct mvebu_pcie_port *port, u16 old,
> +                                     u16 new)
> +{
> +       /* Enabling an I/O window ? */
> +       if (!(old & PCI_COMMAND_IO) && (new & PCI_COMMAND_IO))
> +               mvebu_pcie_setup_io_window(port, 1);
> +
> +       /* Disabling an I/O window ? */
> +       if ((old & PCI_COMMAND_IO) && !(new & PCI_COMMAND_IO))
> +               mvebu_pcie_setup_io_window(port, 0);
> +
> +       /* Enabling a memory window ? */
> +       if (!(old & PCI_COMMAND_MEMORY) && (new & PCI_COMMAND_MEMORY))
> +               mvebu_pcie_setup_mem_window(port, 1);
> +
> +       /* Disabling a memory window ? */
> +       if ((old & PCI_COMMAND_MEMORY) && !(new & PCI_COMMAND_MEMORY))
> +               mvebu_pcie_setup_mem_window(port, 0);
> +}
> ...
> +/* Write to the PCI-to-PCI bridge configuration space */
> +static int mvebu_sw_pci_bridge_write(struct mvebu_pcie_port *port,
> +                                    unsigned int where, int size, u32 value)
> +{
> +       struct mvebu_sw_pci_bridge *bridge = &port->bridge;
> +       u32 mask, reg;
> +       int err;
> +
> +       if (size == 4)
> +               mask = 0x0;
> +       else if (size == 2)
> +               mask = ~(0xffff << ((where & 3) * 8));
> +       else if (size == 1)
> +               mask = ~(0xff << ((where & 3) * 8));
> +       else
> +               return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> +       err = mvebu_sw_pci_bridge_read(port, where & ~3, 4, &reg);
> +       if (err)
> +               return err;
> +
> +       value = (reg & mask) | value << ((where & 3) * 8);
> +
> +       switch (where & ~3) {
> +       case PCI_COMMAND:
> +               mvebu_handle_pcie_command(port, bridge->command,
> +                                         value & 0xffff);
> +               bridge->command = value & 0xffff;
> +               bridge->status = value >> 16;
> +               break;
> +
> +       case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_1:
> +               bridge->bar[((where & ~3) - PCI_BASE_ADDRESS_0) / 4] = value;
> +               break;
> +
> +       case PCI_IO_BASE:
> +               /*
> +                * We also keep bit 1 set, it is a read-only bit that
> +                * indicates we support 32 bits addressing for the
> +                * I/O
> +                */
> +               bridge->iobase = (value & 0xff) | PCI_IO_RANGE_TYPE_32;
> +               bridge->iolimit = ((value >> 8) & 0xff) | PCI_IO_RANGE_TYPE_32;
> +               bridge->secondary_status = value >> 16;
> +               break;
> +
> +       case PCI_MEMORY_BASE:
> +               bridge->membase = value & 0xffff;
> +               bridge->memlimit = value >> 16;
> +               break;
> +
> +       case PCI_PREF_MEMORY_BASE:
> +               bridge->prefmembase = value & 0xffff;
> +               bridge->prefmemlimit = value >> 16;
> +               break;
> +
> +       case PCI_PREF_BASE_UPPER32:
> +               bridge->prefbaseupper = value;
> +               break;
> +
> +       case PCI_PREF_LIMIT_UPPER32:
> +               bridge->preflimitupper = value;
> +               break;

You're relying on a subsequent PCI_COMMAND write to set PCI_COMMAND_IO
and/or PCI_COMMAND_MEMORY, and you program the bridge windows at that
time.  It might be a good idea if the PCI core did clear those bits
while updating the windows, but I'm not sure we do.  In any case,
delaying the update is a difference from a standard P2P bridge that
could cause issues later.

> +       case PCI_IO_BASE_UPPER16:
> +               bridge->iobaseupper = value & 0xffff;
> +               bridge->iolimitupper = value >> 16;
> +               break;
> +
> +       case PCI_PRIMARY_BUS:
> +               bridge->primary_bus             = value & 0xff;
> +               bridge->secondary_bus           = (value >> 8) & 0xff;
> +               bridge->subordinate_bus         = (value >> 16) & 0xff;
> +               bridge->secondary_latency_timer = (value >> 24) & 0xff;
> +               orion_pcie_set_local_bus_nr(port->base, bridge->secondary_bus);
> +               break;
> +
> +       default:
> +               break;
> +       }
> +
> +       return PCIBIOS_SUCCESSFUL;
> +}

Bjorn

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-15  0:36     ` Bjorn Helgaas
  0 siblings, 0 replies; 291+ messages in thread
From: Bjorn Helgaas @ 2013-02-15  0:36 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Feb 12, 2013 at 9:28 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:

> --- a/drivers/pci/host/Kconfig
> +++ b/drivers/pci/host/Kconfig
> @@ -1,4 +1,10 @@
>  menu "PCI host controller drivers"
>         depends on PCI
>
> +config PCI_MVEBU
> +       bool "Marvell EBU PCIe controller"
> +       depends on ARCH_MVEBU
> +       select PCI_SW_HOST_BRIDGE
> +       select PCI_SW_PCI_PCI_BRIDGE

I think PCI_SW_HOST_BRIDGE and PCI_SW_PCI_PCI_BRIDGE are obsolete and
can be removed, right?

> --- /dev/null
> +++ b/drivers/pci/host/pci-mvebu.c
> ...
> +static void mvebu_pcie_setup_io_window(struct mvebu_pcie_port *port,
> +                                      int enable)
> +{
> +       unsigned long iobase, iolimit;
> +
> +       if (port->bridge.iolimit < port->bridge.iobase)
> +               return;
> +
> +       iolimit = 0xFFF | ((port->bridge.iolimit & 0xF0) << 8) |
> +               (port->bridge.iolimitupper << 16);
> +       iobase = ((port->bridge.iobase & 0xF0) << 8) |
> +               (port->bridge.iobaseupper << 16);
> +
> +       if (enable) {
> +               unsigned long physbase = port->pcie->io.start + iobase;
> +               armada_370_xp_alloc_pcie_window(port->port, port->lane,
> +                                               physbase, iobase,
> +                                               iolimit-iobase,
> +                                               IORESOURCE_IO);
> +               pci_ioremap_io(iobase, physbase);
> +       }
> +       else
> +               armada_370_xp_free_pcie_window(iobase);
> +}
> +
> +static void mvebu_pcie_setup_mem_window(struct mvebu_pcie_port *port,
> +                                       int enable)
> +{
> +       unsigned long membase, memlimit;
> +
> +       if (port->bridge.memlimit < port->bridge.membase)
> +               return;
> +
> +       membase = ((port->bridge.membase & 0xFFF0) << 16);
> +       memlimit = ((port->bridge.memlimit & 0xFFF0) << 16) | 0xFFFFF;
> +
> +       if (enable)
> +               armada_370_xp_alloc_pcie_window(port->port, port->lane,
> +                                               membase, ORION_ADDR_MAP_NO_REMAP,
> +                                               memlimit-membase,
> +                                               IORESOURCE_MEM);
> +       else
> +               armada_370_xp_free_pcie_window(membase);
> +}
> +
> +static void mvebu_handle_pcie_command(struct mvebu_pcie_port *port, u16 old,
> +                                     u16 new)
> +{
> +       /* Enabling an I/O window ? */
> +       if (!(old & PCI_COMMAND_IO) && (new & PCI_COMMAND_IO))
> +               mvebu_pcie_setup_io_window(port, 1);
> +
> +       /* Disabling an I/O window ? */
> +       if ((old & PCI_COMMAND_IO) && !(new & PCI_COMMAND_IO))
> +               mvebu_pcie_setup_io_window(port, 0);
> +
> +       /* Enabling a memory window ? */
> +       if (!(old & PCI_COMMAND_MEMORY) && (new & PCI_COMMAND_MEMORY))
> +               mvebu_pcie_setup_mem_window(port, 1);
> +
> +       /* Disabling a memory window ? */
> +       if ((old & PCI_COMMAND_MEMORY) && !(new & PCI_COMMAND_MEMORY))
> +               mvebu_pcie_setup_mem_window(port, 0);
> +}
> ...
> +/* Write to the PCI-to-PCI bridge configuration space */
> +static int mvebu_sw_pci_bridge_write(struct mvebu_pcie_port *port,
> +                                    unsigned int where, int size, u32 value)
> +{
> +       struct mvebu_sw_pci_bridge *bridge = &port->bridge;
> +       u32 mask, reg;
> +       int err;
> +
> +       if (size == 4)
> +               mask = 0x0;
> +       else if (size == 2)
> +               mask = ~(0xffff << ((where & 3) * 8));
> +       else if (size == 1)
> +               mask = ~(0xff << ((where & 3) * 8));
> +       else
> +               return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> +       err = mvebu_sw_pci_bridge_read(port, where & ~3, 4, &reg);
> +       if (err)
> +               return err;
> +
> +       value = (reg & mask) | value << ((where & 3) * 8);
> +
> +       switch (where & ~3) {
> +       case PCI_COMMAND:
> +               mvebu_handle_pcie_command(port, bridge->command,
> +                                         value & 0xffff);
> +               bridge->command = value & 0xffff;
> +               bridge->status = value >> 16;
> +               break;
> +
> +       case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_1:
> +               bridge->bar[((where & ~3) - PCI_BASE_ADDRESS_0) / 4] = value;
> +               break;
> +
> +       case PCI_IO_BASE:
> +               /*
> +                * We also keep bit 1 set, it is a read-only bit that
> +                * indicates we support 32 bits addressing for the
> +                * I/O
> +                */
> +               bridge->iobase = (value & 0xff) | PCI_IO_RANGE_TYPE_32;
> +               bridge->iolimit = ((value >> 8) & 0xff) | PCI_IO_RANGE_TYPE_32;
> +               bridge->secondary_status = value >> 16;
> +               break;
> +
> +       case PCI_MEMORY_BASE:
> +               bridge->membase = value & 0xffff;
> +               bridge->memlimit = value >> 16;
> +               break;
> +
> +       case PCI_PREF_MEMORY_BASE:
> +               bridge->prefmembase = value & 0xffff;
> +               bridge->prefmemlimit = value >> 16;
> +               break;
> +
> +       case PCI_PREF_BASE_UPPER32:
> +               bridge->prefbaseupper = value;
> +               break;
> +
> +       case PCI_PREF_LIMIT_UPPER32:
> +               bridge->preflimitupper = value;
> +               break;

You're relying on a subsequent PCI_COMMAND write to set PCI_COMMAND_IO
and/or PCI_COMMAND_MEMORY, and you program the bridge windows at that
time.  It might be a good idea if the PCI core did clear those bits
while updating the windows, but I'm not sure we do.  In any case,
delaying the update is a difference from a standard P2P bridge that
could cause issues later.

> +       case PCI_IO_BASE_UPPER16:
> +               bridge->iobaseupper = value & 0xffff;
> +               bridge->iolimitupper = value >> 16;
> +               break;
> +
> +       case PCI_PRIMARY_BUS:
> +               bridge->primary_bus             = value & 0xff;
> +               bridge->secondary_bus           = (value >> 8) & 0xff;
> +               bridge->subordinate_bus         = (value >> 16) & 0xff;
> +               bridge->secondary_latency_timer = (value >> 24) & 0xff;
> +               orion_pcie_set_local_bus_nr(port->base, bridge->secondary_bus);
> +               break;
> +
> +       default:
> +               break;
> +       }
> +
> +       return PCIBIOS_SUCCESSFUL;
> +}

Bjorn

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-15  0:36     ` Bjorn Helgaas
@ 2013-02-15  5:06       ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-15  5:06 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: linux-pci, linux-arm-kernel, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

Dear Bjorn Helgaas,

On Thu, 14 Feb 2013 17:36:39 -0700, Bjorn Helgaas wrote:

> > +config PCI_MVEBU
> > +       bool "Marvell EBU PCIe controller"
> > +       depends on ARCH_MVEBU
> > +       select PCI_SW_HOST_BRIDGE
> > +       select PCI_SW_PCI_PCI_BRIDGE
> 
> I think PCI_SW_HOST_BRIDGE and PCI_SW_PCI_PCI_BRIDGE are obsolete and
> can be removed, right?

Indeed, will fix.


> > +       case PCI_PREF_LIMIT_UPPER32:
> > +               bridge->preflimitupper = value;
> > +               break;
> 
> You're relying on a subsequent PCI_COMMAND write to set PCI_COMMAND_IO
> and/or PCI_COMMAND_MEMORY, and you program the bridge windows at that
> time.

Correct.

>  It might be a good idea if the PCI core did clear those bits
> while updating the windows, but I'm not sure we do.  In any case,
> delaying the update is a difference from a standard P2P bridge that
> could cause issues later.

When would you want the window assignment to occur? Directly when the
registers containing the base/limit are being written to? There are
multiple registers for base/limit, so I'll have to figure out a way to
validate when the base/limit informations are valid.

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-15  5:06       ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-02-15  5:06 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Bjorn Helgaas,

On Thu, 14 Feb 2013 17:36:39 -0700, Bjorn Helgaas wrote:

> > +config PCI_MVEBU
> > +       bool "Marvell EBU PCIe controller"
> > +       depends on ARCH_MVEBU
> > +       select PCI_SW_HOST_BRIDGE
> > +       select PCI_SW_PCI_PCI_BRIDGE
> 
> I think PCI_SW_HOST_BRIDGE and PCI_SW_PCI_PCI_BRIDGE are obsolete and
> can be removed, right?

Indeed, will fix.


> > +       case PCI_PREF_LIMIT_UPPER32:
> > +               bridge->preflimitupper = value;
> > +               break;
> 
> You're relying on a subsequent PCI_COMMAND write to set PCI_COMMAND_IO
> and/or PCI_COMMAND_MEMORY, and you program the bridge windows at that
> time.

Correct.

>  It might be a good idea if the PCI core did clear those bits
> while updating the windows, but I'm not sure we do.  In any case,
> delaying the update is a difference from a standard P2P bridge that
> could cause issues later.

When would you want the window assignment to occur? Directly when the
registers containing the base/limit are being written to? There are
multiple registers for base/limit, so I'll have to figure out a way to
validate when the base/limit informations are valid.

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-15  5:06       ` Thomas Petazzoni
@ 2013-02-15 16:26         ` Bjorn Helgaas
  -1 siblings, 0 replies; 291+ messages in thread
From: Bjorn Helgaas @ 2013-02-15 16:26 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: linux-pci, linux-arm-kernel, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, Thierry Reding, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Gregory Clement, Jason Gunthorpe,
	Tawfik Bayouk

On Thu, Feb 14, 2013 at 10:06 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:

>> > +       case PCI_PREF_LIMIT_UPPER32:
>> > +               bridge->preflimitupper = value;
>> > +               break;
>>
>> You're relying on a subsequent PCI_COMMAND write to set PCI_COMMAND_IO
>> and/or PCI_COMMAND_MEMORY, and you program the bridge windows at that
>> time.
>
> Correct.
>
>>  It might be a good idea if the PCI core did clear those bits
>> while updating the windows, but I'm not sure we do.  In any case,
>> delaying the update is a difference from a standard P2P bridge that
>> could cause issues later.
>
> When would you want the window assignment to occur? Directly when the
> registers containing the base/limit are being written to? There are
> multiple registers for base/limit, so I'll have to figure out a way to
> validate when the base/limit informations are valid.

For a standard bridge, the window change occurs immediately upon
writing the base/limit, so that would be the best time.

Here's the scenario I have in mind: We have a bridge X leading to
several devices A and B.  Drivers are bound to A and B, and they are
operating.  A hotplug event requires us to expand or shrink one of X's
windows, but the new window is still sufficient to operate A and B.
The question is how we do the window update.  If we turn off
PCI_COMMAND_MEMORY for the update, then we have to coordinate with the
drivers for A and B.  But I think it is possible to do some updates
safely even with the window decoding enabled, without any effect on A
and B, and these are the ones that would cause trouble with your
current scheme.

I don't think the PCI core does any updates like this today, but it's
something we might like to do in the future.

Bjorn

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-15 16:26         ` Bjorn Helgaas
  0 siblings, 0 replies; 291+ messages in thread
From: Bjorn Helgaas @ 2013-02-15 16:26 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Feb 14, 2013 at 10:06 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:

>> > +       case PCI_PREF_LIMIT_UPPER32:
>> > +               bridge->preflimitupper = value;
>> > +               break;
>>
>> You're relying on a subsequent PCI_COMMAND write to set PCI_COMMAND_IO
>> and/or PCI_COMMAND_MEMORY, and you program the bridge windows at that
>> time.
>
> Correct.
>
>>  It might be a good idea if the PCI core did clear those bits
>> while updating the windows, but I'm not sure we do.  In any case,
>> delaying the update is a difference from a standard P2P bridge that
>> could cause issues later.
>
> When would you want the window assignment to occur? Directly when the
> registers containing the base/limit are being written to? There are
> multiple registers for base/limit, so I'll have to figure out a way to
> validate when the base/limit informations are valid.

For a standard bridge, the window change occurs immediately upon
writing the base/limit, so that would be the best time.

Here's the scenario I have in mind: We have a bridge X leading to
several devices A and B.  Drivers are bound to A and B, and they are
operating.  A hotplug event requires us to expand or shrink one of X's
windows, but the new window is still sufficient to operate A and B.
The question is how we do the window update.  If we turn off
PCI_COMMAND_MEMORY for the update, then we have to coordinate with the
drivers for A and B.  But I think it is possible to do some updates
safely even with the window decoding enabled, without any effect on A
and B, and these are the ones that would cause trouble with your
current scheme.

I don't think the PCI core does any updates like this today, but it's
something we might like to do in the future.

Bjorn

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-15 16:26         ` Bjorn Helgaas
@ 2013-02-15 16:44           ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-15 16:44 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Thomas Petazzoni, linux-pci, linux-arm-kernel, Lior Amsalem,
	Andrew Lunn, Russell King - ARM Linux, Jason Cooper,
	Arnd Bergmann, Stephen Warren, Thierry Reding, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Gregory Clement,
	Tawfik Bayouk

On Fri, Feb 15, 2013 at 09:26:09AM -0700, Bjorn Helgaas wrote:
> On Thu, Feb 14, 2013 at 10:06 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> 
> >> > +       case PCI_PREF_LIMIT_UPPER32:
> >> > +               bridge->preflimitupper = value;
> >> > +               break;
> >>
> >> You're relying on a subsequent PCI_COMMAND write to set PCI_COMMAND_IO
> >> and/or PCI_COMMAND_MEMORY, and you program the bridge windows at that
> >> time.
> >
> > Correct.
> >
> >>  It might be a good idea if the PCI core did clear those bits
> >> while updating the windows, but I'm not sure we do.  In any case,
> >> delaying the update is a difference from a standard P2P bridge that
> >> could cause issues later.
> >
> > When would you want the window assignment to occur? Directly when the
> > registers containing the base/limit are being written to? There are
> > multiple registers for base/limit, so I'll have to figure out a way to
> > validate when the base/limit informations are valid.
> 
> For a standard bridge, the window change occurs immediately upon
> writing the base/limit, so that would be the best time.

The standard says a window where limit <= base is invalid/disabled.

So the write mechanism from the PCI core should be

base = 0xFFFFFFFF
limit = true value
base = true value

In the driver the window should be synchronized after any base/limit
update where base < limit. The initial values (ignoring fixed bits)
should be

base = 0xFFFFFFFF
limit = 0

It would be optimal to use the resource framework to ensure the region
is available, but that is something for the mbus driver to take care
of, not PCI-E...

> drivers for A and B.  But I think it is possible to do some updates
> safely even with the window decoding enabled, without any effect on A
> and B, and these are the ones that would cause trouble with your
> current scheme.

A resize update should just write the true values in any order, the
bridge window will expand without interrupting existing traffic.

However the core code must ensure that at no time do two bridges
decode the same window. That could crash the machine..

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-02-15 16:44           ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-02-15 16:44 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Feb 15, 2013 at 09:26:09AM -0700, Bjorn Helgaas wrote:
> On Thu, Feb 14, 2013 at 10:06 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> 
> >> > +       case PCI_PREF_LIMIT_UPPER32:
> >> > +               bridge->preflimitupper = value;
> >> > +               break;
> >>
> >> You're relying on a subsequent PCI_COMMAND write to set PCI_COMMAND_IO
> >> and/or PCI_COMMAND_MEMORY, and you program the bridge windows at that
> >> time.
> >
> > Correct.
> >
> >>  It might be a good idea if the PCI core did clear those bits
> >> while updating the windows, but I'm not sure we do.  In any case,
> >> delaying the update is a difference from a standard P2P bridge that
> >> could cause issues later.
> >
> > When would you want the window assignment to occur? Directly when the
> > registers containing the base/limit are being written to? There are
> > multiple registers for base/limit, so I'll have to figure out a way to
> > validate when the base/limit informations are valid.
> 
> For a standard bridge, the window change occurs immediately upon
> writing the base/limit, so that would be the best time.

The standard says a window where limit <= base is invalid/disabled.

So the write mechanism from the PCI core should be

base = 0xFFFFFFFF
limit = true value
base = true value

In the driver the window should be synchronized after any base/limit
update where base < limit. The initial values (ignoring fixed bits)
should be

base = 0xFFFFFFFF
limit = 0

It would be optimal to use the resource framework to ensure the region
is available, but that is something for the mbus driver to take care
of, not PCI-E...

> drivers for A and B.  But I think it is possible to do some updates
> safely even with the window decoding enabled, without any effect on A
> and B, and these are the ones that would cause trouble with your
> current scheme.

A resize update should just write the true values in any order, the
bridge window will expand without interrupting existing traffic.

However the core code must ensure that at no time do two bridges
decode the same window. That could crash the machine..

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
  2013-02-12 22:36         ` Arnd Bergmann
@ 2013-03-04 16:28           ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-04 16:28 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, Stephen Warren, linux-pci,
	Thierry Reding, Paul Gortmaker, linux-kernel, Jesse Barnes,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Bjorn Helgaas, Gregory Clement, Yinghai Lu, linux-arm-kernel,
	Jason Gunthorpe

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 22:36:37 +0000, Arnd Bergmann wrote:

> > I have the feeling that the problem is more complex than that. My
> > understanding is that the pcim_iomap_regions() function used by
> > drivers/ata/libata-sff.c can perfectly be used to map memory BARs, and
> > not necessarily I/O BARs. Therefore, this driver can perfectly be used
> > in an architecture where CONFIG_NO_IOPORT is selected.
> 
> That is correct.
> 
> > The thing is that pcim_iomap_regions() transparently allows to remap an
> > I/O BAR is such a BAR is passed as argument, or a memory BAR if such a
> > BAR is passed as argument.
> > 
> > Therefore, I continue to believe that the pcim_*() functions are useful
> > even if the platform doesn't have CONFIG_HAS_IOPORT.
> 
> Yes, the pcim_ functions are useful in principle, but it falls back
> to the __pci_ioport_map() for IORESOURCE_IO, and that needs to
> return an error if CONFIG_HAS_IOPORT is not set.
> I think it would be correct if you add this hunk:
> 
> diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c
> index 0d83ea8..f9b6387 100644
> --- a/lib/pci_iomap.c
> +++ b/lib/pci_iomap.c
> @@ -33,7 +33,7 @@ void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long maxlen)
>                 return NULL;
>         if (maxlen && len > maxlen)
>                 len = maxlen;
> -       if (flags & IORESOURCE_IO)
> +       if (IS_ENABLED(CONFIG_HAS_IOPORT) && (flags & IORESOURCE_IO))
>                 return __pci_ioport_map(dev, start, len);
>         if (flags & IORESOURCE_MEM) {
>                 if (flags & IORESOURCE_CACHEABLE)
> 
> in order to prevent a link error when CONFIG_HAS_IOPORT is unset.

FWIW, a patch that is doing what I was initially proposing has been
merged for 3.9, and it doesn't contain the
IS_ENABLED(CONFIG_HAS_IOPORT) test you were proposing (and which I
think was correct). See:

commit 9ed8a30f3471347c1b763bd062fa78ae80f18eae
Author: Jingoo Han <jg1.han@samsung.com>
Date:   Wed Feb 27 17:02:42 2013 -0800

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
@ 2013-03-04 16:28           ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-04 16:28 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Tue, 12 Feb 2013 22:36:37 +0000, Arnd Bergmann wrote:

> > I have the feeling that the problem is more complex than that. My
> > understanding is that the pcim_iomap_regions() function used by
> > drivers/ata/libata-sff.c can perfectly be used to map memory BARs, and
> > not necessarily I/O BARs. Therefore, this driver can perfectly be used
> > in an architecture where CONFIG_NO_IOPORT is selected.
> 
> That is correct.
> 
> > The thing is that pcim_iomap_regions() transparently allows to remap an
> > I/O BAR is such a BAR is passed as argument, or a memory BAR if such a
> > BAR is passed as argument.
> > 
> > Therefore, I continue to believe that the pcim_*() functions are useful
> > even if the platform doesn't have CONFIG_HAS_IOPORT.
> 
> Yes, the pcim_ functions are useful in principle, but it falls back
> to the __pci_ioport_map() for IORESOURCE_IO, and that needs to
> return an error if CONFIG_HAS_IOPORT is not set.
> I think it would be correct if you add this hunk:
> 
> diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c
> index 0d83ea8..f9b6387 100644
> --- a/lib/pci_iomap.c
> +++ b/lib/pci_iomap.c
> @@ -33,7 +33,7 @@ void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long maxlen)
>                 return NULL;
>         if (maxlen && len > maxlen)
>                 len = maxlen;
> -       if (flags & IORESOURCE_IO)
> +       if (IS_ENABLED(CONFIG_HAS_IOPORT) && (flags & IORESOURCE_IO))
>                 return __pci_ioport_map(dev, start, len);
>         if (flags & IORESOURCE_MEM) {
>                 if (flags & IORESOURCE_CACHEABLE)
> 
> in order to prevent a link error when CONFIG_HAS_IOPORT is unset.

FWIW, a patch that is doing what I was initially proposing has been
merged for 3.9, and it doesn't contain the
IS_ENABLED(CONFIG_HAS_IOPORT) test you were proposing (and which I
think was correct). See:

commit 9ed8a30f3471347c1b763bd062fa78ae80f18eae
Author: Jingoo Han <jg1.han@samsung.com>
Date:   Wed Feb 27 17:02:42 2013 -0800

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
  2013-03-04 16:28           ` Thomas Petazzoni
@ 2013-03-04 20:30             ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-03-04 20:30 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, Stephen Warren, linux-pci,
	Thierry Reding, Paul Gortmaker, linux-kernel, Jesse Barnes,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Bjorn Helgaas, Gregory Clement, Yinghai Lu, linux-arm-kernel,
	Jason Gunthorpe

On Monday 04 March 2013, Thomas Petazzoni wrote:
> FWIW, a patch that is doing what I was initially proposing has been
> merged for 3.9, and it doesn't contain the
> IS_ENABLED(CONFIG_HAS_IOPORT) test you were proposing (and which I
> think was correct). See:
> 
> commit 9ed8a30f3471347c1b763bd062fa78ae80f18eae
> Author: Jingoo Han <jg1.han@samsung.com>
> Date:   Wed Feb 27 17:02:42 2013 -0800
> 

Sigh.

I'll take it as an additional incentive to finally clean up the logic behind
CONFIG_HAS_IOPORT by introducing a CONFIG_HAS_IOPORT_MAP symbol to replace it.

Thanks for the heads up.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT
@ 2013-03-04 20:30             ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-03-04 20:30 UTC (permalink / raw)
  To: linux-arm-kernel

On Monday 04 March 2013, Thomas Petazzoni wrote:
> FWIW, a patch that is doing what I was initially proposing has been
> merged for 3.9, and it doesn't contain the
> IS_ENABLED(CONFIG_HAS_IOPORT) test you were proposing (and which I
> think was correct). See:
> 
> commit 9ed8a30f3471347c1b763bd062fa78ae80f18eae
> Author: Jingoo Han <jg1.han@samsung.com>
> Date:   Wed Feb 27 17:02:42 2013 -0800
> 

Sigh.

I'll take it as an additional incentive to finally clean up the logic behind
CONFIG_HAS_IOPORT by introducing a CONFIG_HAS_IOPORT_MAP symbol to replace it.

Thanks for the heads up.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-13 10:37                 ` Arnd Bergmann
@ 2013-03-06  9:50                   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-06  9:50 UTC (permalink / raw)
  To: Arnd Bergmann
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, Stephen Warren, linux-pci,
	Thierry Reding, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, Gregory Clement, linux-arm-kernel,
	Jason Gunthorpe

Dear Arnd Bergmann,

On Wed, 13 Feb 2013 10:37:02 +0000, Arnd Bergmann wrote:

> Yes, of course. And the ranges property tells you how to turn the
> first address space into the second address space. So the above
> property defines that the PCI bus I/O space range from 0 to 0x10000
> gets converted into the host MMIO range 0xc0000000 to 0xc0010000 on
> the host, and the PCI bus memory space range from 0 to 0x08000000
> gets converted to the host MMIO range 0xc1000000 to 0xc9000000.
> 
> The output of your of_pci_process_ranges() function is the host MMIO
> range, not the range in the bus address space, so it has to be
> IORESOURCE_MEM.

I am sorry, but I don't get how this can work. My code currently relies
on the DT encoding one I/O resource and one MEM resource to find which
is one to use.

Right now, in the ranges property of my DT, I have:

          0x81000000 0 0          0xc0000000 0 0x00010000   /* downstream I/O */
          0x82000000 0 0          0xc1000000 0 0x08000000>; /* non-prefetchable memory */

And the code parsing this does:

        /* Get the I/O and memory ranges from DT */
        while ((range = of_pci_process_ranges(np, &res, range)) != NULL) {
                if (resource_type(&res) == IORESOURCE_IO) {
                        memcpy(&pcie->io, &res, sizeof(res));
                        memcpy(&pcie->realio, &res, sizeof(res));
                        pcie->io.name = "I/O";
                        pcie->realio.start &= 0xFFFFF;
                        pcie->realio.end   &= 0xFFFFF;
                }
                if (resource_type(&res) == IORESOURCE_MEM) {
                        memcpy(&pcie->mem, &res, sizeof(res));
                        pcie->mem.name = "MEM";
                }
        }

As you can see, it relies on one of the two ranges being for I/O
(address starts with 0x81), and another range being for MEM (address
starts with 0x82).

If I make both ranges start with 0x82, how can my code differentiate
between the range used for I/O and the range used for MEM?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-06  9:50                   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-06  9:50 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Arnd Bergmann,

On Wed, 13 Feb 2013 10:37:02 +0000, Arnd Bergmann wrote:

> Yes, of course. And the ranges property tells you how to turn the
> first address space into the second address space. So the above
> property defines that the PCI bus I/O space range from 0 to 0x10000
> gets converted into the host MMIO range 0xc0000000 to 0xc0010000 on
> the host, and the PCI bus memory space range from 0 to 0x08000000
> gets converted to the host MMIO range 0xc1000000 to 0xc9000000.
> 
> The output of your of_pci_process_ranges() function is the host MMIO
> range, not the range in the bus address space, so it has to be
> IORESOURCE_MEM.

I am sorry, but I don't get how this can work. My code currently relies
on the DT encoding one I/O resource and one MEM resource to find which
is one to use.

Right now, in the ranges property of my DT, I have:

          0x81000000 0 0          0xc0000000 0 0x00010000   /* downstream I/O */
          0x82000000 0 0          0xc1000000 0 0x08000000>; /* non-prefetchable memory */

And the code parsing this does:

        /* Get the I/O and memory ranges from DT */
        while ((range = of_pci_process_ranges(np, &res, range)) != NULL) {
                if (resource_type(&res) == IORESOURCE_IO) {
                        memcpy(&pcie->io, &res, sizeof(res));
                        memcpy(&pcie->realio, &res, sizeof(res));
                        pcie->io.name = "I/O";
                        pcie->realio.start &= 0xFFFFF;
                        pcie->realio.end   &= 0xFFFFF;
                }
                if (resource_type(&res) == IORESOURCE_MEM) {
                        memcpy(&pcie->mem, &res, sizeof(res));
                        pcie->mem.name = "MEM";
                }
        }

As you can see, it relies on one of the two ranges being for I/O
(address starts with 0x81), and another range being for MEM (address
starts with 0x82).

If I make both ranges start with 0x82, how can my code differentiate
between the range used for I/O and the range used for MEM?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-02-12 22:35     ` Jason Gunthorpe
@ 2013-03-06  9:54       ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-06  9:54 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Arnd Bergmann, Stephen Warren, linux-pci,
	Thierry Reding, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, Gregory Clement, Tawfik Bayouk,
	linux-arm-kernel

Dear Jason Gunthorpe,

On Tue, 12 Feb 2013 15:35:11 -0700, Jason Gunthorpe wrote:

> > +	pcie@0,0 {
> > +		device_type = "pciex";
> > +		reg = <0x0800 0 0xd0040000 0 0x2000>;
> 
> It would be great to get this sorted as per my prior comments.. Maybe
> like this is easy?
> 
> pcie-controller {
>  compatible = "marvell,armada-370-xp-pcie";
> 
>  // Index by marvell,pcie-port ?
>  regs = <0xd0040000 0x00002000
>          0xd0080000 0x00002000>;
> 
>  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
>            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> 
>  pcie@0,0 {
>       device_type = "pci";
>       reg = <0x0800 0 0 0>; // 00:01.0  (????)
>       marvell,pcie-port = <0>;
>  };
> }
> 
> It is abusive to map the device internal per-port registers through
> '0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
> the OF spec.

The Device Tree would really look odd. We have one register range for
each PCIe interface, but instead of nicely putting them inside the
pcie@X,Y subnodes, we have a global regs = <..> property at the
pcie-controller level? I can do that if you want, but it really sounds
like the standard PCI DT bindings are horrible. Those register ranges
are *per* PCIe interface, so any logical person would expect them
inside the pcie@X,Y node...

But ok, if that's the way things should be, so be it.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-06  9:54       ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-06  9:54 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Jason Gunthorpe,

On Tue, 12 Feb 2013 15:35:11 -0700, Jason Gunthorpe wrote:

> > +	pcie at 0,0 {
> > +		device_type = "pciex";
> > +		reg = <0x0800 0 0xd0040000 0 0x2000>;
> 
> It would be great to get this sorted as per my prior comments.. Maybe
> like this is easy?
> 
> pcie-controller {
>  compatible = "marvell,armada-370-xp-pcie";
> 
>  // Index by marvell,pcie-port ?
>  regs = <0xd0040000 0x00002000
>          0xd0080000 0x00002000>;
> 
>  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
>            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> 
>  pcie at 0,0 {
>       device_type = "pci";
>       reg = <0x0800 0 0 0>; // 00:01.0  (????)
>       marvell,pcie-port = <0>;
>  };
> }
> 
> It is abusive to map the device internal per-port registers through
> '0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
> the OF spec.

The Device Tree would really look odd. We have one register range for
each PCIe interface, but instead of nicely putting them inside the
pcie at X,Y subnodes, we have a global regs = <..> property at the
pcie-controller level? I can do that if you want, but it really sounds
like the standard PCI DT bindings are horrible. Those register ranges
are *per* PCIe interface, so any logical person would expect them
inside the pcie at X,Y node...

But ok, if that's the way things should be, so be it.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-06  9:50                   ` Thomas Petazzoni
@ 2013-03-06 10:43                     ` Arnd Bergmann
  -1 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-03-06 10:43 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, Stephen Warren, linux-pci,
	Thierry Reding, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, Gregory Clement, linux-arm-kernel,
	Jason Gunthorpe

On Wednesday 06 March 2013, Thomas Petazzoni wrote:
> Dear Arnd Bergmann,
> 
> On Wed, 13 Feb 2013 10:37:02 +0000, Arnd Bergmann wrote:
> 
> > Yes, of course. And the ranges property tells you how to turn the
> > first address space into the second address space. So the above
> > property defines that the PCI bus I/O space range from 0 to 0x10000
> > gets converted into the host MMIO range 0xc0000000 to 0xc0010000 on
> > the host, and the PCI bus memory space range from 0 to 0x08000000
> > gets converted to the host MMIO range 0xc1000000 to 0xc9000000.
> > 
> > The output of your of_pci_process_ranges() function is the host MMIO
> > range, not the range in the bus address space, so it has to be
> > IORESOURCE_MEM.
> 
> I am sorry, but I don't get how this can work. My code currently relies
> on the DT encoding one I/O resource and one MEM resource to find which
> is one to use.
> 
> Right now, in the ranges property of my DT, I have:
> 
>           0x81000000 0 0          0xc0000000 0 0x00010000   /* downstream I/O */
>           0x82000000 0 0          0xc1000000 0 0x08000000>; /* non-prefetchable memory */

This looks correct, yes, although as I suggested earlier it would be more
conventional to use

          0x81000000 0 0          0xc0000000 0 0x00010000   /* downstream I/O */
          0x82000000 0 0xc1000000 0xc1000000 0 0x08000000>; /* non-prefetchable memory */


> And the code parsing this does:
> 
>         /* Get the I/O and memory ranges from DT */
>         while ((range = of_pci_process_ranges(np, &res, range)) != NULL) {
>                 if (resource_type(&res) == IORESOURCE_IO) {
>                         memcpy(&pcie->io, &res, sizeof(res));
>                         memcpy(&pcie->realio, &res, sizeof(res));
>                         pcie->io.name = "I/O";
>                         pcie->realio.start &= 0xFFFFF;
>                         pcie->realio.end   &= 0xFFFFF;
>                 }
>                 if (resource_type(&res) == IORESOURCE_MEM) {
>                         memcpy(&pcie->mem, &res, sizeof(res));
>                         pcie->mem.name = "MEM";
>                 }
>         }
> 
> As you can see, it relies on one of the two ranges being for I/O
> (address starts with 0x81), and another range being for MEM (address
> starts with 0x82).
> 
> If I make both ranges start with 0x82, how can my code differentiate
> between the range used for I/O and the range used for MEM?

No, I mean the parser is wrong ;-)

I think the proper solution would be to change the parser to return both
the host side resource, and another resource as the bus view. In the example
above, the resources you get should be:

io_host  = { .flags = IORESOURCE_MEM, .start = 0xc0000000, .end = 0xc000ffff };
io_bus   = { .flags = IORESOURCE_IO,  .start = 0,          .end =     0xffff };
mem_host = { .flags = IORESOURCE_MEM, .start = 0xc1000000, .end = 0xc9ffffff };
mem_bus  = { .flags = IORESOURCE_MEM, .start = 0,          .end =  0x7ffffff };

indicating that the PCI I/O space range 0-0xffff is mapped to the host physical
address space at 0xc0000000-0xc000ffff, and that the PCI memory space range 
0-0x7ffffff is mapped to host physical address space 0xc1000000-0xc9ffffff.

You really need all four resources to set up the mapping. For I/O space, you
should do

	pci_ioremap_io(io_bus->start, io_host->start);

and for the memory space, you need both to set up the correct mem_offset:

	pci_add_resource_offset(&sys->resources, mem_host, mem_host->start - mem_bus->start);

If you do as I suggested above and set the memory window to an identity
map at bus address 0xc1000000, the second argument to pci_add_resource_offset
becomes zero, indicating that any access to the window results in a bus
access at the same address.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-06 10:43                     ` Arnd Bergmann
  0 siblings, 0 replies; 291+ messages in thread
From: Arnd Bergmann @ 2013-03-06 10:43 UTC (permalink / raw)
  To: linux-arm-kernel

On Wednesday 06 March 2013, Thomas Petazzoni wrote:
> Dear Arnd Bergmann,
> 
> On Wed, 13 Feb 2013 10:37:02 +0000, Arnd Bergmann wrote:
> 
> > Yes, of course. And the ranges property tells you how to turn the
> > first address space into the second address space. So the above
> > property defines that the PCI bus I/O space range from 0 to 0x10000
> > gets converted into the host MMIO range 0xc0000000 to 0xc0010000 on
> > the host, and the PCI bus memory space range from 0 to 0x08000000
> > gets converted to the host MMIO range 0xc1000000 to 0xc9000000.
> > 
> > The output of your of_pci_process_ranges() function is the host MMIO
> > range, not the range in the bus address space, so it has to be
> > IORESOURCE_MEM.
> 
> I am sorry, but I don't get how this can work. My code currently relies
> on the DT encoding one I/O resource and one MEM resource to find which
> is one to use.
> 
> Right now, in the ranges property of my DT, I have:
> 
>           0x81000000 0 0          0xc0000000 0 0x00010000   /* downstream I/O */
>           0x82000000 0 0          0xc1000000 0 0x08000000>; /* non-prefetchable memory */

This looks correct, yes, although as I suggested earlier it would be more
conventional to use

          0x81000000 0 0          0xc0000000 0 0x00010000   /* downstream I/O */
          0x82000000 0 0xc1000000 0xc1000000 0 0x08000000>; /* non-prefetchable memory */


> And the code parsing this does:
> 
>         /* Get the I/O and memory ranges from DT */
>         while ((range = of_pci_process_ranges(np, &res, range)) != NULL) {
>                 if (resource_type(&res) == IORESOURCE_IO) {
>                         memcpy(&pcie->io, &res, sizeof(res));
>                         memcpy(&pcie->realio, &res, sizeof(res));
>                         pcie->io.name = "I/O";
>                         pcie->realio.start &= 0xFFFFF;
>                         pcie->realio.end   &= 0xFFFFF;
>                 }
>                 if (resource_type(&res) == IORESOURCE_MEM) {
>                         memcpy(&pcie->mem, &res, sizeof(res));
>                         pcie->mem.name = "MEM";
>                 }
>         }
> 
> As you can see, it relies on one of the two ranges being for I/O
> (address starts with 0x81), and another range being for MEM (address
> starts with 0x82).
> 
> If I make both ranges start with 0x82, how can my code differentiate
> between the range used for I/O and the range used for MEM?

No, I mean the parser is wrong ;-)

I think the proper solution would be to change the parser to return both
the host side resource, and another resource as the bus view. In the example
above, the resources you get should be:

io_host  = { .flags = IORESOURCE_MEM, .start = 0xc0000000, .end = 0xc000ffff };
io_bus   = { .flags = IORESOURCE_IO,  .start = 0,          .end =     0xffff };
mem_host = { .flags = IORESOURCE_MEM, .start = 0xc1000000, .end = 0xc9ffffff };
mem_bus  = { .flags = IORESOURCE_MEM, .start = 0,          .end =  0x7ffffff };

indicating that the PCI I/O space range 0-0xffff is mapped to the host physical
address space at 0xc0000000-0xc000ffff, and that the PCI memory space range 
0-0x7ffffff is mapped to host physical address space 0xc1000000-0xc9ffffff.

You really need all four resources to set up the mapping. For I/O space, you
should do

	pci_ioremap_io(io_bus->start, io_host->start);

and for the memory space, you need both to set up the correct mem_offset:

	pci_add_resource_offset(&sys->resources, mem_host, mem_host->start - mem_bus->start);

If you do as I suggested above and set the memory window to an identity
map at bus address 0xc1000000, the second argument to pci_add_resource_offset
becomes zero, indicating that any access to the window results in a bus
access at the same address.

	Arnd

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-06  9:54       ` Thomas Petazzoni
@ 2013-03-06 12:11         ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-06 12:11 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Jason Gunthorpe, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 3067 bytes --]

On Wed, Mar 06, 2013 at 10:54:41AM +0100, Thomas Petazzoni wrote:
> Dear Jason Gunthorpe,
> 
> On Tue, 12 Feb 2013 15:35:11 -0700, Jason Gunthorpe wrote:
> 
> > > +	pcie@0,0 {
> > > +		device_type = "pciex";
> > > +		reg = <0x0800 0 0xd0040000 0 0x2000>;
> > 
> > It would be great to get this sorted as per my prior comments.. Maybe
> > like this is easy?
> > 
> > pcie-controller {
> >  compatible = "marvell,armada-370-xp-pcie";
> > 
> >  // Index by marvell,pcie-port ?
> >  regs = <0xd0040000 0x00002000
> >          0xd0080000 0x00002000>;
> > 
> >  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
> >            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> > 
> >  pcie@0,0 {
> >       device_type = "pci";
> >       reg = <0x0800 0 0 0>; // 00:01.0  (????)
> >       marvell,pcie-port = <0>;
> >  };
> > }
> > 
> > It is abusive to map the device internal per-port registers through
> > '0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
> > the OF spec.
> 
> The Device Tree would really look odd. We have one register range for
> each PCIe interface, but instead of nicely putting them inside the
> pcie@X,Y subnodes, we have a global regs = <..> property at the
> pcie-controller level? I can do that if you want, but it really sounds
> like the standard PCI DT bindings are horrible. Those register ranges
> are *per* PCIe interface, so any logical person would expect them
> inside the pcie@X,Y node...

I agree here. It seems that back at the time when the PCI DT bindings
were devised no hardware existed with these properties. So I think that
maybe it needs to be revised to take into account some of these new
requirements. Just adding a bunch of properties to work around
shortcomings in the specification doesn't seem very desirable to me.

That said, on Tegra there's an additional argument in favour of this
because the addresses mapped through the ranges/reg properties are
actually used to access the device's configuration space, which the
'ss' field in the first cell specifies. Quoting the DTS:

	pcie-controller {
		...
		ranges = <0x00000800 0 0x80000000 0x80000000 0 0x00001000   /* port 0 registers */
			  0x00001000 0 0x80001000 0x80001000 0 0x00001000   /* port 1 registers */
			  0x81000000 0 0          0x82000000 0 0x00010000   /* downstream I/O */
			  0x82000000 0 0          0xa0000000 0 0x10000000   /* non-prefetchable memory */
			  0xc2000000 0 0          0xb0000000 0 0x10000000>; /* prefetchable memory */
		...
		pci@1,0 {
			...
			reg = <0x000800 0 0x80000000 0 0x1000>;
			...
		};

		pci@2,0 {
			...
			reg = <0x001000 0 0x80001000 0 0x1000>;
			...
		};
	};

So we're actually mapping the configuration space of pci@1,0 and pci@2,0
to the 0x80000000 and 0x80001000 addresses of the CPU physical address
space. According to the PCI DT specification this isn't allowed, but it
works on Linux and quite frankly I don't see why it shouldn't be
allowed.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-06 12:11         ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-06 12:11 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 06, 2013 at 10:54:41AM +0100, Thomas Petazzoni wrote:
> Dear Jason Gunthorpe,
> 
> On Tue, 12 Feb 2013 15:35:11 -0700, Jason Gunthorpe wrote:
> 
> > > +	pcie at 0,0 {
> > > +		device_type = "pciex";
> > > +		reg = <0x0800 0 0xd0040000 0 0x2000>;
> > 
> > It would be great to get this sorted as per my prior comments.. Maybe
> > like this is easy?
> > 
> > pcie-controller {
> >  compatible = "marvell,armada-370-xp-pcie";
> > 
> >  // Index by marvell,pcie-port ?
> >  regs = <0xd0040000 0x00002000
> >          0xd0080000 0x00002000>;
> > 
> >  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
> >            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> > 
> >  pcie at 0,0 {
> >       device_type = "pci";
> >       reg = <0x0800 0 0 0>; // 00:01.0  (????)
> >       marvell,pcie-port = <0>;
> >  };
> > }
> > 
> > It is abusive to map the device internal per-port registers through
> > '0x00000800 0 0xd0040000' and 'reg' - that is not really the intent of
> > the OF spec.
> 
> The Device Tree would really look odd. We have one register range for
> each PCIe interface, but instead of nicely putting them inside the
> pcie at X,Y subnodes, we have a global regs = <..> property at the
> pcie-controller level? I can do that if you want, but it really sounds
> like the standard PCI DT bindings are horrible. Those register ranges
> are *per* PCIe interface, so any logical person would expect them
> inside the pcie at X,Y node...

I agree here. It seems that back at the time when the PCI DT bindings
were devised no hardware existed with these properties. So I think that
maybe it needs to be revised to take into account some of these new
requirements. Just adding a bunch of properties to work around
shortcomings in the specification doesn't seem very desirable to me.

That said, on Tegra there's an additional argument in favour of this
because the addresses mapped through the ranges/reg properties are
actually used to access the device's configuration space, which the
'ss' field in the first cell specifies. Quoting the DTS:

	pcie-controller {
		...
		ranges = <0x00000800 0 0x80000000 0x80000000 0 0x00001000   /* port 0 registers */
			  0x00001000 0 0x80001000 0x80001000 0 0x00001000   /* port 1 registers */
			  0x81000000 0 0          0x82000000 0 0x00010000   /* downstream I/O */
			  0x82000000 0 0          0xa0000000 0 0x10000000   /* non-prefetchable memory */
			  0xc2000000 0 0          0xb0000000 0 0x10000000>; /* prefetchable memory */
		...
		pci at 1,0 {
			...
			reg = <0x000800 0 0x80000000 0 0x1000>;
			...
		};

		pci at 2,0 {
			...
			reg = <0x001000 0 0x80001000 0 0x1000>;
			...
		};
	};

So we're actually mapping the configuration space of pci at 1,0 and pci at 2,0
to the 0x80000000 and 0x80001000 addresses of the CPU physical address
space. According to the PCI DT specification this isn't allowed, but it
works on Linux and quite frankly I don't see why it shouldn't be
allowed.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130306/543b0f8a/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-06 12:11         ` Thierry Reding
@ 2013-03-06 18:09           ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-06 18:09 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, linux-arm-kernel

On Wed, Mar 06, 2013 at 01:11:19PM +0100, Thierry Reding wrote:

> I agree here. It seems that back at the time when the PCI DT bindings
> were devised no hardware existed with these properties. So I think that
> maybe it needs to be revised to take into account some of these new
> requirements. Just adding a bunch of properties to work around
> shortcomings in the specification doesn't seem very desirable to me.

Well, all the PCI hosts should use the same mechanism, and it should
make sense in the context of the rest of the spec. So I think some
approval/comment on this specific issue by Grant/Rob/Arnd is important
at this point.

The way the spec is written, 'reg' in a 'device_type = pci' context is
the device's config address - it is like 'reg' in a CPU block, or I2C
bus - the value has nothing to do with CPU mappable memory.

> space. According to the PCI DT specification this isn't allowed, but it
> works on Linux and quite frankly I don't see why it shouldn't be
> allowed.

AFAIK, this only works because you are not using 'device_type = pci'.

IMHO, 'device_type = pci' is not optional, so you need to come up with
something that works under those conditions, and it changes how the OF
core handles 'reg' and the like.

As I said before, the two best options to me seem to be:

pcie-controller {
 compatible = "marvell,armada-370-xp-pcie";

  // Index by marvell,pcie-port/reg-names/etc
  regs = <0xd0040000 0x00002000
          0xd0080000 0x00002000>;

  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */

  pcie@0,0 {
       device_type = "pci";
       reg = <0x0800 0 0 0>; // 00:01.0  (????)
       marvell,pcie-port = <0>;
  };
}

or:

pcie-controller {
  compatible = "marvell,armada-370-xp-pcie";

  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
            0x82000000 0 0  0xc1000000  0 0x08000000   /* non-prefetchable memory */
            // Mapping for per-port host controller registers
            0x82000000 0x10000000 0xd0040000  0xd0040000  0 0x2000>;

  pcie@0,0 {
       device_type = "pci";
       reg = <0x0800 0 0 0>; // 00:01.0  (????)
       marvell,pcie-port = <0>;
       
       assigned-addresses = <0 0 0  0 0 // BAR 0 of the bridge, unused
       			     0 0 0  0 0 // BAR 1 of the bridge, unused
                             0x82000000 0x10000000 0xd0040000  0 0x2000>; // Extended

  };
}

Both have various problems, but I think I prefer the first one as it
doesn't conflate the contoller registers and host apertures in a
single ranges..

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-06 18:09           ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-06 18:09 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 06, 2013 at 01:11:19PM +0100, Thierry Reding wrote:

> I agree here. It seems that back at the time when the PCI DT bindings
> were devised no hardware existed with these properties. So I think that
> maybe it needs to be revised to take into account some of these new
> requirements. Just adding a bunch of properties to work around
> shortcomings in the specification doesn't seem very desirable to me.

Well, all the PCI hosts should use the same mechanism, and it should
make sense in the context of the rest of the spec. So I think some
approval/comment on this specific issue by Grant/Rob/Arnd is important
at this point.

The way the spec is written, 'reg' in a 'device_type = pci' context is
the device's config address - it is like 'reg' in a CPU block, or I2C
bus - the value has nothing to do with CPU mappable memory.

> space. According to the PCI DT specification this isn't allowed, but it
> works on Linux and quite frankly I don't see why it shouldn't be
> allowed.

AFAIK, this only works because you are not using 'device_type = pci'.

IMHO, 'device_type = pci' is not optional, so you need to come up with
something that works under those conditions, and it changes how the OF
core handles 'reg' and the like.

As I said before, the two best options to me seem to be:

pcie-controller {
 compatible = "marvell,armada-370-xp-pcie";

  // Index by marvell,pcie-port/reg-names/etc
  regs = <0xd0040000 0x00002000
          0xd0080000 0x00002000>;

  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
            0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */

  pcie at 0,0 {
       device_type = "pci";
       reg = <0x0800 0 0 0>; // 00:01.0  (????)
       marvell,pcie-port = <0>;
  };
}

or:

pcie-controller {
  compatible = "marvell,armada-370-xp-pcie";

  ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
            0x82000000 0 0  0xc1000000  0 0x08000000   /* non-prefetchable memory */
            // Mapping for per-port host controller registers
            0x82000000 0x10000000 0xd0040000  0xd0040000  0 0x2000>;

  pcie at 0,0 {
       device_type = "pci";
       reg = <0x0800 0 0 0>; // 00:01.0  (????)
       marvell,pcie-port = <0>;
       
       assigned-addresses = <0 0 0  0 0 // BAR 0 of the bridge, unused
       			     0 0 0  0 0 // BAR 1 of the bridge, unused
                             0x82000000 0x10000000 0xd0040000  0 0x2000>; // Extended

  };
}

Both have various problems, but I think I prefer the first one as it
doesn't conflate the contoller registers and host apertures in a
single ranges..

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-06 18:09           ` Jason Gunthorpe
@ 2013-03-07  8:08             ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-07  8:08 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 4264 bytes --]

On Wed, Mar 06, 2013 at 11:09:46AM -0700, Jason Gunthorpe wrote:
> On Wed, Mar 06, 2013 at 01:11:19PM +0100, Thierry Reding wrote:
> 
> > I agree here. It seems that back at the time when the PCI DT bindings
> > were devised no hardware existed with these properties. So I think that
> > maybe it needs to be revised to take into account some of these new
> > requirements. Just adding a bunch of properties to work around
> > shortcomings in the specification doesn't seem very desirable to me.
> 
> Well, all the PCI hosts should use the same mechanism, and it should
> make sense in the context of the rest of the spec. So I think some
> approval/comment on this specific issue by Grant/Rob/Arnd is important
> at this point.
> 
> The way the spec is written, 'reg' in a 'device_type = pci' context is
> the device's config address - it is like 'reg' in a CPU block, or I2C
> bus - the value has nothing to do with CPU mappable memory.
> 
> > space. According to the PCI DT specification this isn't allowed, but it
> > works on Linux and quite frankly I don't see why it shouldn't be
> > allowed.
> 
> AFAIK, this only works because you are not using 'device_type = pci'.
> 
> IMHO, 'device_type = pci' is not optional, so you need to come up with
> something that works under those conditions, and it changes how the OF
> core handles 'reg' and the like.

Actually I do use device_type = "pciex" for the pci@1,0 and pci@2,0
nodes which was suggested back when I first posted this series. Yet the
address translation still works properly. And it should since they are
translated to the parent bus which is used to access the configuration
space window for each root port.

Looking at the code it looks indeed like the OF core only matches on
"pci" (and "vci"), but not "pciex". Perhaps I should change both nodes
to device_type = "pci" and see what the results are.

> As I said before, the two best options to me seem to be:
> 
> pcie-controller {
>  compatible = "marvell,armada-370-xp-pcie";
> 
>   // Index by marvell,pcie-port/reg-names/etc
>   regs = <0xd0040000 0x00002000
>           0xd0080000 0x00002000>;
> 
>   ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
>             0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> 
>   pcie@0,0 {
>        device_type = "pci";
>        reg = <0x0800 0 0 0>; // 00:01.0  (????)
>        marvell,pcie-port = <0>;
>   };
> }
> 
> or:
> 
> pcie-controller {
>   compatible = "marvell,armada-370-xp-pcie";
> 
>   ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
>             0x82000000 0 0  0xc1000000  0 0x08000000   /* non-prefetchable memory */
>             // Mapping for per-port host controller registers
>             0x82000000 0x10000000 0xd0040000  0xd0040000  0 0x2000>;
> 
>   pcie@0,0 {
>        device_type = "pci";
>        reg = <0x0800 0 0 0>; // 00:01.0  (????)
>        marvell,pcie-port = <0>;
>        
>        assigned-addresses = <0 0 0  0 0 // BAR 0 of the bridge, unused
>        			     0 0 0  0 0 // BAR 1 of the bridge, unused
>                              0x82000000 0x10000000 0xd0040000  0 0x2000>; // Extended
> 
>   };
> }
> 
> Both have various problems, but I think I prefer the first one as it
> doesn't conflate the contoller registers and host apertures in a
> single ranges..

I think a better alternative would be (and this matches what Thomas has
said elsewhere) to use something like the first alternative but move the
regs property into the pcie@0,X nodes. That would save us from having to
index a property in the parent. At least from a DT point of view I find
that to be a more consistent representation.

However that would probably not work out-of-the-box with the current OF
core because of_address_to_resource() won't know how to find the new
property. The assigned-addresses alternative seems to be the only one
that would work on the current OF core and achieve proper address
translation. But it doesn't seem like a good solution either since it
repurposes the meaning of the property and therefore isn't any better
than encoding the same information in the reg property.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-07  8:08             ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-07  8:08 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 06, 2013 at 11:09:46AM -0700, Jason Gunthorpe wrote:
> On Wed, Mar 06, 2013 at 01:11:19PM +0100, Thierry Reding wrote:
> 
> > I agree here. It seems that back at the time when the PCI DT bindings
> > were devised no hardware existed with these properties. So I think that
> > maybe it needs to be revised to take into account some of these new
> > requirements. Just adding a bunch of properties to work around
> > shortcomings in the specification doesn't seem very desirable to me.
> 
> Well, all the PCI hosts should use the same mechanism, and it should
> make sense in the context of the rest of the spec. So I think some
> approval/comment on this specific issue by Grant/Rob/Arnd is important
> at this point.
> 
> The way the spec is written, 'reg' in a 'device_type = pci' context is
> the device's config address - it is like 'reg' in a CPU block, or I2C
> bus - the value has nothing to do with CPU mappable memory.
> 
> > space. According to the PCI DT specification this isn't allowed, but it
> > works on Linux and quite frankly I don't see why it shouldn't be
> > allowed.
> 
> AFAIK, this only works because you are not using 'device_type = pci'.
> 
> IMHO, 'device_type = pci' is not optional, so you need to come up with
> something that works under those conditions, and it changes how the OF
> core handles 'reg' and the like.

Actually I do use device_type = "pciex" for the pci at 1,0 and pci at 2,0
nodes which was suggested back when I first posted this series. Yet the
address translation still works properly. And it should since they are
translated to the parent bus which is used to access the configuration
space window for each root port.

Looking at the code it looks indeed like the OF core only matches on
"pci" (and "vci"), but not "pciex". Perhaps I should change both nodes
to device_type = "pci" and see what the results are.

> As I said before, the two best options to me seem to be:
> 
> pcie-controller {
>  compatible = "marvell,armada-370-xp-pcie";
> 
>   // Index by marvell,pcie-port/reg-names/etc
>   regs = <0xd0040000 0x00002000
>           0xd0080000 0x00002000>;
> 
>   ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
>             0x82000000 0 0  0xc1000000  0 0x08000000>; /* non-prefetchable memory */
> 
>   pcie at 0,0 {
>        device_type = "pci";
>        reg = <0x0800 0 0 0>; // 00:01.0  (????)
>        marvell,pcie-port = <0>;
>   };
> }
> 
> or:
> 
> pcie-controller {
>   compatible = "marvell,armada-370-xp-pcie";
> 
>   ranges = <0x81000000 0 0  0xc0000000  0 0x00010000   /* downstream I/O */
>             0x82000000 0 0  0xc1000000  0 0x08000000   /* non-prefetchable memory */
>             // Mapping for per-port host controller registers
>             0x82000000 0x10000000 0xd0040000  0xd0040000  0 0x2000>;
> 
>   pcie at 0,0 {
>        device_type = "pci";
>        reg = <0x0800 0 0 0>; // 00:01.0  (????)
>        marvell,pcie-port = <0>;
>        
>        assigned-addresses = <0 0 0  0 0 // BAR 0 of the bridge, unused
>        			     0 0 0  0 0 // BAR 1 of the bridge, unused
>                              0x82000000 0x10000000 0xd0040000  0 0x2000>; // Extended
> 
>   };
> }
> 
> Both have various problems, but I think I prefer the first one as it
> doesn't conflate the contoller registers and host apertures in a
> single ranges..

I think a better alternative would be (and this matches what Thomas has
said elsewhere) to use something like the first alternative but move the
regs property into the pcie at 0,X nodes. That would save us from having to
index a property in the parent. At least from a DT point of view I find
that to be a more consistent representation.

However that would probably not work out-of-the-box with the current OF
core because of_address_to_resource() won't know how to find the new
property. The assigned-addresses alternative seems to be the only one
that would work on the current OF core and achieve proper address
translation. But it doesn't seem like a good solution either since it
repurposes the meaning of the property and therefore isn't any better
than encoding the same information in the reg property.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130307/989c145c/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-07  8:08             ` Thierry Reding
@ 2013-03-07 17:49               ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-07 17:49 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, linux-arm-kernel

On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:

> Actually I do use device_type = "pciex" for the pci@1,0 and pci@2,0
> nodes which was suggested back when I first posted this series. Yet the
> address translation still works properly. And it should since they are
> translated to the parent bus which is used to access the configuration
> space window for each root port.

Right - pciex != pci - as you note:

> Looking at the code it looks indeed like the OF core only matches on
> "pci" (and "vci"), but not "pciex". Perhaps I should change both nodes
> to device_type = "pci" and see what the results are.

The string 'pci' engages the special behavior, pciex isn't a valid
keyword.

This is why I keep saying these bindings need to use 'device_type = "pci"' :)

> > Both have various problems, but I think I prefer the first one as it
> > doesn't conflate the contoller registers and host apertures in a
> > single ranges..
> 
> I think a better alternative would be (and this matches what Thomas has
> said elsewhere) to use something like the first alternative but move the
> regs property into the pcie@0,X nodes. That would save us from having to
> index a property in the parent. At least from a DT point of view I find
> that to be a more consistent representation.

You are thinking a new property 'host-controller-regs' or the like?

> However that would probably not work out-of-the-box with the current OF
> core because of_address_to_resource() won't know how to find the new
> property. The assigned-addresses alternative seems to be the only one
> that would work on the current OF core and achieve proper address
> translation. But it doesn't seem like a good solution either since it
> repurposes the meaning of the property and therefore isn't any better
> than encoding the same information in the reg property.

Except that reg is specifically not supposed to be handling CPU bus
addresses and assigned-addresses is.

Adding a hidden non-standard BAR to model the hidden non-standard
memory region associated with the bridge is a stretch, but it is not a
very big stretch...

In any event, regs should not be used, and something needs to be
decided!

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-07 17:49               ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-07 17:49 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:

> Actually I do use device_type = "pciex" for the pci at 1,0 and pci at 2,0
> nodes which was suggested back when I first posted this series. Yet the
> address translation still works properly. And it should since they are
> translated to the parent bus which is used to access the configuration
> space window for each root port.

Right - pciex != pci - as you note:

> Looking at the code it looks indeed like the OF core only matches on
> "pci" (and "vci"), but not "pciex". Perhaps I should change both nodes
> to device_type = "pci" and see what the results are.

The string 'pci' engages the special behavior, pciex isn't a valid
keyword.

This is why I keep saying these bindings need to use 'device_type = "pci"' :)

> > Both have various problems, but I think I prefer the first one as it
> > doesn't conflate the contoller registers and host apertures in a
> > single ranges..
> 
> I think a better alternative would be (and this matches what Thomas has
> said elsewhere) to use something like the first alternative but move the
> regs property into the pcie at 0,X nodes. That would save us from having to
> index a property in the parent. At least from a DT point of view I find
> that to be a more consistent representation.

You are thinking a new property 'host-controller-regs' or the like?

> However that would probably not work out-of-the-box with the current OF
> core because of_address_to_resource() won't know how to find the new
> property. The assigned-addresses alternative seems to be the only one
> that would work on the current OF core and achieve proper address
> translation. But it doesn't seem like a good solution either since it
> repurposes the meaning of the property and therefore isn't any better
> than encoding the same information in the reg property.

Except that reg is specifically not supposed to be handling CPU bus
addresses and assigned-addresses is.

Adding a hidden non-standard BAR to model the hidden non-standard
memory region associated with the bridge is a stretch, but it is not a
very big stretch...

In any event, regs should not be used, and something needs to be
decided!

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-07 17:49               ` Jason Gunthorpe
@ 2013-03-07 19:48                 ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-07 19:48 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 3236 bytes --]

On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
> On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
[...]
> > > Both have various problems, but I think I prefer the first one as it
> > > doesn't conflate the contoller registers and host apertures in a
> > > single ranges..
> > 
> > I think a better alternative would be (and this matches what Thomas has
> > said elsewhere) to use something like the first alternative but move the
> > regs property into the pcie@0,X nodes. That would save us from having to
> > index a property in the parent. At least from a DT point of view I find
> > that to be a more consistent representation.
> 
> You are thinking a new property 'host-controller-regs' or the like?

Well, something shorter would be nice, but that's the general idea, yes.
As I mentioned before, for Tegra these registers aren't actually any
controller specific registers but rather a window to access the PCI
configuration space for the root ports.

Going via the FPCI mapping of the configuration space doesn't generate
transactions on bus 0, which is why there are extra windows for each
root port. That's why I was arguing about reusing the reg property in
the first place because it actually represents the configuration space.

But this is likely to be unique to Tegra and Marvell hardware doesn't
behave this way, but instead needs the registers for a different
purpose. Something like host-controller-regs might be more appropriate.

> > However that would probably not work out-of-the-box with the current OF
> > core because of_address_to_resource() won't know how to find the new
> > property. The assigned-addresses alternative seems to be the only one
> > that would work on the current OF core and achieve proper address
> > translation. But it doesn't seem like a good solution either since it
> > repurposes the meaning of the property and therefore isn't any better
> > than encoding the same information in the reg property.
> 
> Except that reg is specifically not supposed to be handling CPU bus
> addresses and assigned-addresses is.
> 
> Adding a hidden non-standard BAR to model the hidden non-standard
> memory region associated with the bridge is a stretch, but it is not a
> very big stretch...
> 
> In any event, regs should not be used, and something needs to be
> decided!

I don't think assigned-addresses is a good fit either. The PCI binding
document is equally specific about it as it is about the reg property.
So in my opinion a separate property would be a better choice. The only
big obstacle is that it needs to be somehow hooked up with the OF core
so that proper address translation can be performed.

One possible solution that wouldn't be too hard to implement is to
provide a new function (say of_get_named_address()) similar to
of_get_address() which doesn't get the name of the register property
from the struct of_bus but from a parameter and call that function from
another new function similar to of_address_to_resource() that also gets
the property name from a parameter. I can't think of a better name for
the latter than of_named_address_to_resource(), which is rather long.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-07 19:48                 ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-07 19:48 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
> On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
[...]
> > > Both have various problems, but I think I prefer the first one as it
> > > doesn't conflate the contoller registers and host apertures in a
> > > single ranges..
> > 
> > I think a better alternative would be (and this matches what Thomas has
> > said elsewhere) to use something like the first alternative but move the
> > regs property into the pcie at 0,X nodes. That would save us from having to
> > index a property in the parent. At least from a DT point of view I find
> > that to be a more consistent representation.
> 
> You are thinking a new property 'host-controller-regs' or the like?

Well, something shorter would be nice, but that's the general idea, yes.
As I mentioned before, for Tegra these registers aren't actually any
controller specific registers but rather a window to access the PCI
configuration space for the root ports.

Going via the FPCI mapping of the configuration space doesn't generate
transactions on bus 0, which is why there are extra windows for each
root port. That's why I was arguing about reusing the reg property in
the first place because it actually represents the configuration space.

But this is likely to be unique to Tegra and Marvell hardware doesn't
behave this way, but instead needs the registers for a different
purpose. Something like host-controller-regs might be more appropriate.

> > However that would probably not work out-of-the-box with the current OF
> > core because of_address_to_resource() won't know how to find the new
> > property. The assigned-addresses alternative seems to be the only one
> > that would work on the current OF core and achieve proper address
> > translation. But it doesn't seem like a good solution either since it
> > repurposes the meaning of the property and therefore isn't any better
> > than encoding the same information in the reg property.
> 
> Except that reg is specifically not supposed to be handling CPU bus
> addresses and assigned-addresses is.
> 
> Adding a hidden non-standard BAR to model the hidden non-standard
> memory region associated with the bridge is a stretch, but it is not a
> very big stretch...
> 
> In any event, regs should not be used, and something needs to be
> decided!

I don't think assigned-addresses is a good fit either. The PCI binding
document is equally specific about it as it is about the reg property.
So in my opinion a separate property would be a better choice. The only
big obstacle is that it needs to be somehow hooked up with the OF core
so that proper address translation can be performed.

One possible solution that wouldn't be too hard to implement is to
provide a new function (say of_get_named_address()) similar to
of_get_address() which doesn't get the name of the register property
from the struct of_bus but from a parameter and call that function from
another new function similar to of_address_to_resource() that also gets
the property name from a parameter. I can't think of a better name for
the latter than of_named_address_to_resource(), which is rather long.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130307/44e758d7/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-07 19:48                 ` Thierry Reding
@ 2013-03-07 20:02                   ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-07 20:02 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, linux-arm-kernel

On Thu, Mar 07, 2013 at 08:48:30PM +0100, Thierry Reding wrote:
> On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
> > On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
> [...]
> > > > Both have various problems, but I think I prefer the first one as it
> > > > doesn't conflate the contoller registers and host apertures in a
> > > > single ranges..
> > > 
> > > I think a better alternative would be (and this matches what Thomas has
> > > said elsewhere) to use something like the first alternative but move the
> > > regs property into the pcie@0,X nodes. That would save us from having to
> > > index a property in the parent. At least from a DT point of view I find
> > > that to be a more consistent representation.
> > 
> > You are thinking a new property 'host-controller-regs' or the like?
> 
> Well, something shorter would be nice, but that's the general idea, yes.
> As I mentioned before, for Tegra these registers aren't actually any
> controller specific registers but rather a window to access the PCI
> configuration space for the root ports.

Yes, I understand - but in this DT model configuration space access is
a host controller function, not a PCI-device function. Anyhow I was
also thinking that by the choice of the name it could do translation
from the host-controller scope, not from the bridge scope - so the
extra elements in ranges could be avoided as well. Hence the name..

> I don't think assigned-addresses is a good fit either. The PCI binding
> document is equally specific about it as it is about the reg property.
> So in my opinion a separate property would be a better choice. The only
> big obstacle is that it needs to be somehow hooked up with the OF core
> so that proper address translation can be performed.

Yes, agreed.

My suggestion is to get the OF experts like GKL/Rob H/etc to weigh in
on a preferred approach to this problem with the goal of standardizing
across all PCI host drivers. Seems like there are 2 main options
(outside regs + regnames/etc or new 'regs' in the bridge) and 1 hacky
one (assigned addresses)

> One possible solution that wouldn't be too hard to implement is to
> provide a new function (say of_get_named_address()) similar to
> of_get_address() which doesn't get the name of the register property
> from the struct of_bus but from a parameter and call that function from
> another new function similar to of_address_to_resource() that also gets
> the property name from a parameter. I can't think of a better name for
> the latter than of_named_address_to_resource(), which is rather long.

Seems like a reasonable API, maybe pass in a be32*/length pointer
instead of a name to be more flexible?

Cheers,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-07 20:02                   ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-07 20:02 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 07, 2013 at 08:48:30PM +0100, Thierry Reding wrote:
> On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
> > On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
> [...]
> > > > Both have various problems, but I think I prefer the first one as it
> > > > doesn't conflate the contoller registers and host apertures in a
> > > > single ranges..
> > > 
> > > I think a better alternative would be (and this matches what Thomas has
> > > said elsewhere) to use something like the first alternative but move the
> > > regs property into the pcie at 0,X nodes. That would save us from having to
> > > index a property in the parent. At least from a DT point of view I find
> > > that to be a more consistent representation.
> > 
> > You are thinking a new property 'host-controller-regs' or the like?
> 
> Well, something shorter would be nice, but that's the general idea, yes.
> As I mentioned before, for Tegra these registers aren't actually any
> controller specific registers but rather a window to access the PCI
> configuration space for the root ports.

Yes, I understand - but in this DT model configuration space access is
a host controller function, not a PCI-device function. Anyhow I was
also thinking that by the choice of the name it could do translation
from the host-controller scope, not from the bridge scope - so the
extra elements in ranges could be avoided as well. Hence the name..

> I don't think assigned-addresses is a good fit either. The PCI binding
> document is equally specific about it as it is about the reg property.
> So in my opinion a separate property would be a better choice. The only
> big obstacle is that it needs to be somehow hooked up with the OF core
> so that proper address translation can be performed.

Yes, agreed.

My suggestion is to get the OF experts like GKL/Rob H/etc to weigh in
on a preferred approach to this problem with the goal of standardizing
across all PCI host drivers. Seems like there are 2 main options
(outside regs + regnames/etc or new 'regs' in the bridge) and 1 hacky
one (assigned addresses)

> One possible solution that wouldn't be too hard to implement is to
> provide a new function (say of_get_named_address()) similar to
> of_get_address() which doesn't get the name of the register property
> from the struct of_bus but from a parameter and call that function from
> another new function similar to of_address_to_resource() that also gets
> the property name from a parameter. I can't think of a better name for
> the latter than of_named_address_to_resource(), which is rather long.

Seems like a reasonable API, maybe pass in a be32*/length pointer
instead of a name to be more flexible?

Cheers,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-07 20:02                   ` Jason Gunthorpe
  (?)
@ 2013-03-07 20:47                       ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-07 20:47 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Russell King - ARM Linux, Jason Cooper,
	Andrew Lunn, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Rob Herring,
	Bjorn Helgaas, Tawfik Bayouk,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r


[-- Attachment #1.1: Type: text/plain, Size: 4650 bytes --]

On Thu, Mar 07, 2013 at 01:02:35PM -0700, Jason Gunthorpe wrote:
> On Thu, Mar 07, 2013 at 08:48:30PM +0100, Thierry Reding wrote:
> > On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
> > > On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
> > [...]
> > > > > Both have various problems, but I think I prefer the first one as it
> > > > > doesn't conflate the contoller registers and host apertures in a
> > > > > single ranges..
> > > > 
> > > > I think a better alternative would be (and this matches what Thomas has
> > > > said elsewhere) to use something like the first alternative but move the
> > > > regs property into the pcie@0,X nodes. That would save us from having to
> > > > index a property in the parent. At least from a DT point of view I find
> > > > that to be a more consistent representation.
> > > 
> > > You are thinking a new property 'host-controller-regs' or the like?
> > 
> > Well, something shorter would be nice, but that's the general idea, yes.
> > As I mentioned before, for Tegra these registers aren't actually any
> > controller specific registers but rather a window to access the PCI
> > configuration space for the root ports.
> 
> Yes, I understand - but in this DT model configuration space access is
> a host controller function, not a PCI-device function. Anyhow I was
> also thinking that by the choice of the name it could do translation
> from the host-controller scope, not from the bridge scope - so the
> extra elements in ranges could be avoided as well. Hence the name..
> 
> > I don't think assigned-addresses is a good fit either. The PCI binding
> > document is equally specific about it as it is about the reg property.
> > So in my opinion a separate property would be a better choice. The only
> > big obstacle is that it needs to be somehow hooked up with the OF core
> > so that proper address translation can be performed.
> 
> Yes, agreed.
> 
> My suggestion is to get the OF experts like GKL/Rob H/etc to weigh in
> on a preferred approach to this problem with the goal of standardizing
> across all PCI host drivers. Seems like there are 2 main options
> (outside regs + regnames/etc or new 'regs' in the bridge) and 1 hacky
> one (assigned addresses)

Arnd is already Cc'ed on this thread, adding Grant, Rob and the
devicetree-discuss ML.

In a nutshell (since some of the context isn't quoted anymore) the
problem that we're trying to solve is that some of the embedded SoCs
require per-root-port registers for configuration. The PCI DT
specification doesn't make any provisions for this. A few alternatives
have been discussed so far:

	1) Use a "regs" property outside of the root port nodes with
	   some mechanism to index into them from within the root port
	   nodes. Conceptually somewhat like this:

		pcie-controller {
			...
			regs = <0x80000000 0x00001000
			        0x80001000 0x00001000>;

			pci@0,1 {
				...
				port-index = <0>;
			};

			pci@0,2 {
				...
				port-index = <1>;
			};
		};

	2) Use a "regs" property inside of the root part nodes, along
	   the following lines:

		pcie-controller {
			...
			pci@0,1 {
				...
				reg = <0x00000800 0 0 0 0>;

				regs = <0x80000000 0x00001000>;
			};

			pci@0,2 {
				...
				reg = <0x00001000 0 0 0 0>;

				regs = <0x80001000 0x00001000>;
			};
		};

	3) Repurpose the "assigned-addresses" property to achieve the
	   same. This should work out-of-the-box but isn't a good fit
	   because it conflicts with the OF PCI specification which
	   defines this property to contain the addresses assigned to
	   the base address registers.

Options 1 and 2 above require changes to the OF core to allow proper
address translation, but the changes shouldn't be very big.

> > One possible solution that wouldn't be too hard to implement is to
> > provide a new function (say of_get_named_address()) similar to
> > of_get_address() which doesn't get the name of the register property
> > from the struct of_bus but from a parameter and call that function from
> > another new function similar to of_address_to_resource() that also gets
> > the property name from a parameter. I can't think of a better name for
> > the latter than of_named_address_to_resource(), which is rather long.
> 
> Seems like a reasonable API, maybe pass in a be32*/length pointer
> instead of a name to be more flexible?

There's already __of_address_to_resource() which takes a be32 * and size
but I thought it might be easier to wrap that to make it easier on the
drivers to use the API.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-07 20:47                       ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-07 20:47 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, Grant Likely, Rob Herring, linux-arm-kernel,
	devicetree-discuss

[-- Attachment #1: Type: text/plain, Size: 4650 bytes --]

On Thu, Mar 07, 2013 at 01:02:35PM -0700, Jason Gunthorpe wrote:
> On Thu, Mar 07, 2013 at 08:48:30PM +0100, Thierry Reding wrote:
> > On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
> > > On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
> > [...]
> > > > > Both have various problems, but I think I prefer the first one as it
> > > > > doesn't conflate the contoller registers and host apertures in a
> > > > > single ranges..
> > > > 
> > > > I think a better alternative would be (and this matches what Thomas has
> > > > said elsewhere) to use something like the first alternative but move the
> > > > regs property into the pcie@0,X nodes. That would save us from having to
> > > > index a property in the parent. At least from a DT point of view I find
> > > > that to be a more consistent representation.
> > > 
> > > You are thinking a new property 'host-controller-regs' or the like?
> > 
> > Well, something shorter would be nice, but that's the general idea, yes.
> > As I mentioned before, for Tegra these registers aren't actually any
> > controller specific registers but rather a window to access the PCI
> > configuration space for the root ports.
> 
> Yes, I understand - but in this DT model configuration space access is
> a host controller function, not a PCI-device function. Anyhow I was
> also thinking that by the choice of the name it could do translation
> from the host-controller scope, not from the bridge scope - so the
> extra elements in ranges could be avoided as well. Hence the name..
> 
> > I don't think assigned-addresses is a good fit either. The PCI binding
> > document is equally specific about it as it is about the reg property.
> > So in my opinion a separate property would be a better choice. The only
> > big obstacle is that it needs to be somehow hooked up with the OF core
> > so that proper address translation can be performed.
> 
> Yes, agreed.
> 
> My suggestion is to get the OF experts like GKL/Rob H/etc to weigh in
> on a preferred approach to this problem with the goal of standardizing
> across all PCI host drivers. Seems like there are 2 main options
> (outside regs + regnames/etc or new 'regs' in the bridge) and 1 hacky
> one (assigned addresses)

Arnd is already Cc'ed on this thread, adding Grant, Rob and the
devicetree-discuss ML.

In a nutshell (since some of the context isn't quoted anymore) the
problem that we're trying to solve is that some of the embedded SoCs
require per-root-port registers for configuration. The PCI DT
specification doesn't make any provisions for this. A few alternatives
have been discussed so far:

	1) Use a "regs" property outside of the root port nodes with
	   some mechanism to index into them from within the root port
	   nodes. Conceptually somewhat like this:

		pcie-controller {
			...
			regs = <0x80000000 0x00001000
			        0x80001000 0x00001000>;

			pci@0,1 {
				...
				port-index = <0>;
			};

			pci@0,2 {
				...
				port-index = <1>;
			};
		};

	2) Use a "regs" property inside of the root part nodes, along
	   the following lines:

		pcie-controller {
			...
			pci@0,1 {
				...
				reg = <0x00000800 0 0 0 0>;

				regs = <0x80000000 0x00001000>;
			};

			pci@0,2 {
				...
				reg = <0x00001000 0 0 0 0>;

				regs = <0x80001000 0x00001000>;
			};
		};

	3) Repurpose the "assigned-addresses" property to achieve the
	   same. This should work out-of-the-box but isn't a good fit
	   because it conflicts with the OF PCI specification which
	   defines this property to contain the addresses assigned to
	   the base address registers.

Options 1 and 2 above require changes to the OF core to allow proper
address translation, but the changes shouldn't be very big.

> > One possible solution that wouldn't be too hard to implement is to
> > provide a new function (say of_get_named_address()) similar to
> > of_get_address() which doesn't get the name of the register property
> > from the struct of_bus but from a parameter and call that function from
> > another new function similar to of_address_to_resource() that also gets
> > the property name from a parameter. I can't think of a better name for
> > the latter than of_named_address_to_resource(), which is rather long.
> 
> Seems like a reasonable API, maybe pass in a be32*/length pointer
> instead of a name to be more flexible?

There's already __of_address_to_resource() which takes a be32 * and size
but I thought it might be easier to wrap that to make it easier on the
drivers to use the API.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-07 20:47                       ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-07 20:47 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 07, 2013 at 01:02:35PM -0700, Jason Gunthorpe wrote:
> On Thu, Mar 07, 2013 at 08:48:30PM +0100, Thierry Reding wrote:
> > On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
> > > On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
> > [...]
> > > > > Both have various problems, but I think I prefer the first one as it
> > > > > doesn't conflate the contoller registers and host apertures in a
> > > > > single ranges..
> > > > 
> > > > I think a better alternative would be (and this matches what Thomas has
> > > > said elsewhere) to use something like the first alternative but move the
> > > > regs property into the pcie at 0,X nodes. That would save us from having to
> > > > index a property in the parent. At least from a DT point of view I find
> > > > that to be a more consistent representation.
> > > 
> > > You are thinking a new property 'host-controller-regs' or the like?
> > 
> > Well, something shorter would be nice, but that's the general idea, yes.
> > As I mentioned before, for Tegra these registers aren't actually any
> > controller specific registers but rather a window to access the PCI
> > configuration space for the root ports.
> 
> Yes, I understand - but in this DT model configuration space access is
> a host controller function, not a PCI-device function. Anyhow I was
> also thinking that by the choice of the name it could do translation
> from the host-controller scope, not from the bridge scope - so the
> extra elements in ranges could be avoided as well. Hence the name..
> 
> > I don't think assigned-addresses is a good fit either. The PCI binding
> > document is equally specific about it as it is about the reg property.
> > So in my opinion a separate property would be a better choice. The only
> > big obstacle is that it needs to be somehow hooked up with the OF core
> > so that proper address translation can be performed.
> 
> Yes, agreed.
> 
> My suggestion is to get the OF experts like GKL/Rob H/etc to weigh in
> on a preferred approach to this problem with the goal of standardizing
> across all PCI host drivers. Seems like there are 2 main options
> (outside regs + regnames/etc or new 'regs' in the bridge) and 1 hacky
> one (assigned addresses)

Arnd is already Cc'ed on this thread, adding Grant, Rob and the
devicetree-discuss ML.

In a nutshell (since some of the context isn't quoted anymore) the
problem that we're trying to solve is that some of the embedded SoCs
require per-root-port registers for configuration. The PCI DT
specification doesn't make any provisions for this. A few alternatives
have been discussed so far:

	1) Use a "regs" property outside of the root port nodes with
	   some mechanism to index into them from within the root port
	   nodes. Conceptually somewhat like this:

		pcie-controller {
			...
			regs = <0x80000000 0x00001000
			        0x80001000 0x00001000>;

			pci at 0,1 {
				...
				port-index = <0>;
			};

			pci at 0,2 {
				...
				port-index = <1>;
			};
		};

	2) Use a "regs" property inside of the root part nodes, along
	   the following lines:

		pcie-controller {
			...
			pci at 0,1 {
				...
				reg = <0x00000800 0 0 0 0>;

				regs = <0x80000000 0x00001000>;
			};

			pci at 0,2 {
				...
				reg = <0x00001000 0 0 0 0>;

				regs = <0x80001000 0x00001000>;
			};
		};

	3) Repurpose the "assigned-addresses" property to achieve the
	   same. This should work out-of-the-box but isn't a good fit
	   because it conflicts with the OF PCI specification which
	   defines this property to contain the addresses assigned to
	   the base address registers.

Options 1 and 2 above require changes to the OF core to allow proper
address translation, but the changes shouldn't be very big.

> > One possible solution that wouldn't be too hard to implement is to
> > provide a new function (say of_get_named_address()) similar to
> > of_get_address() which doesn't get the name of the register property
> > from the struct of_bus but from a parameter and call that function from
> > another new function similar to of_address_to_resource() that also gets
> > the property name from a parameter. I can't think of a better name for
> > the latter than of_named_address_to_resource(), which is rather long.
> 
> Seems like a reasonable API, maybe pass in a be32*/length pointer
> instead of a name to be more flexible?

There's already __of_address_to_resource() which takes a be32 * and size
but I thought it might be easier to wrap that to make it easier on the
drivers to use the API.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130307/2df62bd0/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-07 20:47                       ` Thierry Reding
@ 2013-03-08  0:05                         ` Rob Herring
  -1 siblings, 0 replies; 291+ messages in thread
From: Rob Herring @ 2013-03-08  0:05 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, Grant Likely, linux-arm-kernel,
	devicetree-discuss

On 03/07/2013 02:47 PM, Thierry Reding wrote:
> On Thu, Mar 07, 2013 at 01:02:35PM -0700, Jason Gunthorpe wrote:
>> On Thu, Mar 07, 2013 at 08:48:30PM +0100, Thierry Reding wrote:
>>> On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
>>>> On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
>>> [...]
>>>>>> Both have various problems, but I think I prefer the first one as it
>>>>>> doesn't conflate the contoller registers and host apertures in a
>>>>>> single ranges..
>>>>>
>>>>> I think a better alternative would be (and this matches what Thomas has
>>>>> said elsewhere) to use something like the first alternative but move the
>>>>> regs property into the pcie@0,X nodes. That would save us from having to
>>>>> index a property in the parent. At least from a DT point of view I find
>>>>> that to be a more consistent representation.
>>>>
>>>> You are thinking a new property 'host-controller-regs' or the like?
>>>
>>> Well, something shorter would be nice, but that's the general idea, yes.
>>> As I mentioned before, for Tegra these registers aren't actually any
>>> controller specific registers but rather a window to access the PCI
>>> configuration space for the root ports.
>>
>> Yes, I understand - but in this DT model configuration space access is
>> a host controller function, not a PCI-device function. Anyhow I was
>> also thinking that by the choice of the name it could do translation
>> from the host-controller scope, not from the bridge scope - so the
>> extra elements in ranges could be avoided as well. Hence the name..
>>
>>> I don't think assigned-addresses is a good fit either. The PCI binding
>>> document is equally specific about it as it is about the reg property.
>>> So in my opinion a separate property would be a better choice. The only
>>> big obstacle is that it needs to be somehow hooked up with the OF core
>>> so that proper address translation can be performed.
>>
>> Yes, agreed.
>>
>> My suggestion is to get the OF experts like GKL/Rob H/etc to weigh in
>> on a preferred approach to this problem with the goal of standardizing
>> across all PCI host drivers. Seems like there are 2 main options
>> (outside regs + regnames/etc or new 'regs' in the bridge) and 1 hacky
>> one (assigned addresses)
> 
> Arnd is already Cc'ed on this thread, adding Grant, Rob and the
> devicetree-discuss ML.
> 
> In a nutshell (since some of the context isn't quoted anymore) the
> problem that we're trying to solve is that some of the embedded SoCs
> require per-root-port registers for configuration. The PCI DT
> specification doesn't make any provisions for this. A few alternatives
> have been discussed so far:

I'm not sure I follow. This is different than the host controller
registers? Why would this not just be multiple entries in the reg property?

Rob

> 
> 	1) Use a "regs" property outside of the root port nodes with
> 	   some mechanism to index into them from within the root port
> 	   nodes. Conceptually somewhat like this:
> 
> 		pcie-controller {
> 			...
> 			regs = <0x80000000 0x00001000
> 			        0x80001000 0x00001000>;
> 
> 			pci@0,1 {
> 				...
> 				port-index = <0>;
> 			};
> 
> 			pci@0,2 {
> 				...
> 				port-index = <1>;
> 			};
> 		};
> 
> 	2) Use a "regs" property inside of the root part nodes, along
> 	   the following lines:
> 
> 		pcie-controller {
> 			...
> 			pci@0,1 {
> 				...
> 				reg = <0x00000800 0 0 0 0>;
> 
> 				regs = <0x80000000 0x00001000>;
> 			};
> 
> 			pci@0,2 {
> 				...
> 				reg = <0x00001000 0 0 0 0>;
> 
> 				regs = <0x80001000 0x00001000>;
> 			};
> 		};
> 
> 	3) Repurpose the "assigned-addresses" property to achieve the
> 	   same. This should work out-of-the-box but isn't a good fit
> 	   because it conflicts with the OF PCI specification which
> 	   defines this property to contain the addresses assigned to
> 	   the base address registers.
> 
> Options 1 and 2 above require changes to the OF core to allow proper
> address translation, but the changes shouldn't be very big.
> 
>>> One possible solution that wouldn't be too hard to implement is to
>>> provide a new function (say of_get_named_address()) similar to
>>> of_get_address() which doesn't get the name of the register property
>>> from the struct of_bus but from a parameter and call that function from
>>> another new function similar to of_address_to_resource() that also gets
>>> the property name from a parameter. I can't think of a better name for
>>> the latter than of_named_address_to_resource(), which is rather long.
>>
>> Seems like a reasonable API, maybe pass in a be32*/length pointer
>> instead of a name to be more flexible?
> 
> There's already __of_address_to_resource() which takes a be32 * and size
> but I thought it might be easier to wrap that to make it easier on the
> drivers to use the API.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08  0:05                         ` Rob Herring
  0 siblings, 0 replies; 291+ messages in thread
From: Rob Herring @ 2013-03-08  0:05 UTC (permalink / raw)
  To: linux-arm-kernel

On 03/07/2013 02:47 PM, Thierry Reding wrote:
> On Thu, Mar 07, 2013 at 01:02:35PM -0700, Jason Gunthorpe wrote:
>> On Thu, Mar 07, 2013 at 08:48:30PM +0100, Thierry Reding wrote:
>>> On Thu, Mar 07, 2013 at 10:49:55AM -0700, Jason Gunthorpe wrote:
>>>> On Thu, Mar 07, 2013 at 09:08:32AM +0100, Thierry Reding wrote:
>>> [...]
>>>>>> Both have various problems, but I think I prefer the first one as it
>>>>>> doesn't conflate the contoller registers and host apertures in a
>>>>>> single ranges..
>>>>>
>>>>> I think a better alternative would be (and this matches what Thomas has
>>>>> said elsewhere) to use something like the first alternative but move the
>>>>> regs property into the pcie at 0,X nodes. That would save us from having to
>>>>> index a property in the parent. At least from a DT point of view I find
>>>>> that to be a more consistent representation.
>>>>
>>>> You are thinking a new property 'host-controller-regs' or the like?
>>>
>>> Well, something shorter would be nice, but that's the general idea, yes.
>>> As I mentioned before, for Tegra these registers aren't actually any
>>> controller specific registers but rather a window to access the PCI
>>> configuration space for the root ports.
>>
>> Yes, I understand - but in this DT model configuration space access is
>> a host controller function, not a PCI-device function. Anyhow I was
>> also thinking that by the choice of the name it could do translation
>> from the host-controller scope, not from the bridge scope - so the
>> extra elements in ranges could be avoided as well. Hence the name..
>>
>>> I don't think assigned-addresses is a good fit either. The PCI binding
>>> document is equally specific about it as it is about the reg property.
>>> So in my opinion a separate property would be a better choice. The only
>>> big obstacle is that it needs to be somehow hooked up with the OF core
>>> so that proper address translation can be performed.
>>
>> Yes, agreed.
>>
>> My suggestion is to get the OF experts like GKL/Rob H/etc to weigh in
>> on a preferred approach to this problem with the goal of standardizing
>> across all PCI host drivers. Seems like there are 2 main options
>> (outside regs + regnames/etc or new 'regs' in the bridge) and 1 hacky
>> one (assigned addresses)
> 
> Arnd is already Cc'ed on this thread, adding Grant, Rob and the
> devicetree-discuss ML.
> 
> In a nutshell (since some of the context isn't quoted anymore) the
> problem that we're trying to solve is that some of the embedded SoCs
> require per-root-port registers for configuration. The PCI DT
> specification doesn't make any provisions for this. A few alternatives
> have been discussed so far:

I'm not sure I follow. This is different than the host controller
registers? Why would this not just be multiple entries in the reg property?

Rob

> 
> 	1) Use a "regs" property outside of the root port nodes with
> 	   some mechanism to index into them from within the root port
> 	   nodes. Conceptually somewhat like this:
> 
> 		pcie-controller {
> 			...
> 			regs = <0x80000000 0x00001000
> 			        0x80001000 0x00001000>;
> 
> 			pci at 0,1 {
> 				...
> 				port-index = <0>;
> 			};
> 
> 			pci at 0,2 {
> 				...
> 				port-index = <1>;
> 			};
> 		};
> 
> 	2) Use a "regs" property inside of the root part nodes, along
> 	   the following lines:
> 
> 		pcie-controller {
> 			...
> 			pci at 0,1 {
> 				...
> 				reg = <0x00000800 0 0 0 0>;
> 
> 				regs = <0x80000000 0x00001000>;
> 			};
> 
> 			pci at 0,2 {
> 				...
> 				reg = <0x00001000 0 0 0 0>;
> 
> 				regs = <0x80001000 0x00001000>;
> 			};
> 		};
> 
> 	3) Repurpose the "assigned-addresses" property to achieve the
> 	   same. This should work out-of-the-box but isn't a good fit
> 	   because it conflicts with the OF PCI specification which
> 	   defines this property to contain the addresses assigned to
> 	   the base address registers.
> 
> Options 1 and 2 above require changes to the OF core to allow proper
> address translation, but the changes shouldn't be very big.
> 
>>> One possible solution that wouldn't be too hard to implement is to
>>> provide a new function (say of_get_named_address()) similar to
>>> of_get_address() which doesn't get the name of the register property
>>> from the struct of_bus but from a parameter and call that function from
>>> another new function similar to of_address_to_resource() that also gets
>>> the property name from a parameter. I can't think of a better name for
>>> the latter than of_named_address_to_resource(), which is rather long.
>>
>> Seems like a reasonable API, maybe pass in a be32*/length pointer
>> instead of a name to be more flexible?
> 
> There's already __of_address_to_resource() which takes a be32 * and size
> but I thought it might be easier to wrap that to make it easier on the
> drivers to use the API.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08  0:05                         ` Rob Herring
@ 2013-03-08  7:14                           ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-08  7:14 UTC (permalink / raw)
  To: Rob Herring
  Cc: Jason Gunthorpe, Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, Grant Likely, linux-arm-kernel,
	devicetree-discuss

[-- Attachment #1: Type: text/plain, Size: 2525 bytes --]

On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> On 03/07/2013 02:47 PM, Thierry Reding wrote:
[...]
> > In a nutshell (since some of the context isn't quoted anymore) the
> > problem that we're trying to solve is that some of the embedded SoCs
> > require per-root-port registers for configuration. The PCI DT
> > specification doesn't make any provisions for this. A few alternatives
> > have been discussed so far:
> 
> I'm not sure I follow. This is different than the host controller
> registers? Why would this not just be multiple entries in the reg property?

Well the register regions are per root-port. On Tegra20 there's 2 of
them, Tegra30 has 3 and if I understand correctly Marvell can have up to
10 (!). Adding all of them to the reg property of the host controller
could work but it needs some way to match the reg entry to the root port
similar to option 1 below.

Adding a property in the root port nodes seems like a cleaner and more
accurate representation of the hardware to me, but if that's not
acceptable perhaps we need to bite the bullet and add the code to look
the registers up from the parent's reg property.

Thierry

> > 	1) Use a "regs" property outside of the root port nodes with
> > 	   some mechanism to index into them from within the root port
> > 	   nodes. Conceptually somewhat like this:
> > 
> > 		pcie-controller {
> > 			...
> > 			regs = <0x80000000 0x00001000
> > 			        0x80001000 0x00001000>;
> > 
> > 			pci@0,1 {
> > 				...
> > 				port-index = <0>;
> > 			};
> > 
> > 			pci@0,2 {
> > 				...
> > 				port-index = <1>;
> > 			};
> > 		};
> > 
> > 	2) Use a "regs" property inside of the root part nodes, along
> > 	   the following lines:
> > 
> > 		pcie-controller {
> > 			...
> > 			pci@0,1 {
> > 				...
> > 				reg = <0x00000800 0 0 0 0>;
> > 
> > 				regs = <0x80000000 0x00001000>;
> > 			};
> > 
> > 			pci@0,2 {
> > 				...
> > 				reg = <0x00001000 0 0 0 0>;
> > 
> > 				regs = <0x80001000 0x00001000>;
> > 			};
> > 		};
> > 
> > 	3) Repurpose the "assigned-addresses" property to achieve the
> > 	   same. This should work out-of-the-box but isn't a good fit
> > 	   because it conflicts with the OF PCI specification which
> > 	   defines this property to contain the addresses assigned to
> > 	   the base address registers.
> > 
> > Options 1 and 2 above require changes to the OF core to allow proper
> > address translation, but the changes shouldn't be very big.

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08  7:14                           ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-08  7:14 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> On 03/07/2013 02:47 PM, Thierry Reding wrote:
[...]
> > In a nutshell (since some of the context isn't quoted anymore) the
> > problem that we're trying to solve is that some of the embedded SoCs
> > require per-root-port registers for configuration. The PCI DT
> > specification doesn't make any provisions for this. A few alternatives
> > have been discussed so far:
> 
> I'm not sure I follow. This is different than the host controller
> registers? Why would this not just be multiple entries in the reg property?

Well the register regions are per root-port. On Tegra20 there's 2 of
them, Tegra30 has 3 and if I understand correctly Marvell can have up to
10 (!). Adding all of them to the reg property of the host controller
could work but it needs some way to match the reg entry to the root port
similar to option 1 below.

Adding a property in the root port nodes seems like a cleaner and more
accurate representation of the hardware to me, but if that's not
acceptable perhaps we need to bite the bullet and add the code to look
the registers up from the parent's reg property.

Thierry

> > 	1) Use a "regs" property outside of the root port nodes with
> > 	   some mechanism to index into them from within the root port
> > 	   nodes. Conceptually somewhat like this:
> > 
> > 		pcie-controller {
> > 			...
> > 			regs = <0x80000000 0x00001000
> > 			        0x80001000 0x00001000>;
> > 
> > 			pci at 0,1 {
> > 				...
> > 				port-index = <0>;
> > 			};
> > 
> > 			pci at 0,2 {
> > 				...
> > 				port-index = <1>;
> > 			};
> > 		};
> > 
> > 	2) Use a "regs" property inside of the root part nodes, along
> > 	   the following lines:
> > 
> > 		pcie-controller {
> > 			...
> > 			pci at 0,1 {
> > 				...
> > 				reg = <0x00000800 0 0 0 0>;
> > 
> > 				regs = <0x80000000 0x00001000>;
> > 			};
> > 
> > 			pci at 0,2 {
> > 				...
> > 				reg = <0x00001000 0 0 0 0>;
> > 
> > 				regs = <0x80001000 0x00001000>;
> > 			};
> > 		};
> > 
> > 	3) Repurpose the "assigned-addresses" property to achieve the
> > 	   same. This should work out-of-the-box but isn't a good fit
> > 	   because it conflicts with the OF PCI specification which
> > 	   defines this property to contain the addresses assigned to
> > 	   the base address registers.
> > 
> > Options 1 and 2 above require changes to the OF core to allow proper
> > address translation, but the changes shouldn't be very big.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130308/17196dd1/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08  7:14                           ` Thierry Reding
@ 2013-03-08 16:52                             ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-08 16:52 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Rob Herring, Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, Grant Likely, linux-arm-kernel,
	devicetree-discuss

On Fri, Mar 08, 2013 at 08:14:44AM +0100, Thierry Reding wrote:
> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> > On 03/07/2013 02:47 PM, Thierry Reding wrote:
> [...]
> > > In a nutshell (since some of the context isn't quoted anymore) the
> > > problem that we're trying to solve is that some of the embedded SoCs
> > > require per-root-port registers for configuration. The PCI DT
> > > specification doesn't make any provisions for this. A few alternatives
> > > have been discussed so far:
> > 
> > I'm not sure I follow. This is different than the host controller
> > registers? Why would this not just be multiple entries in the reg property?
> 
> Well the register regions are per root-port. On Tegra20 there's 2 of
> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> 10 (!). Adding all of them to the reg property of the host controller
> could work but it needs some way to match the reg entry to the root port
> similar to option 1 below.

Thomas just posted an implementation like this, please see:

http://www.spinics.net/lists/arm-kernel/msg228749.html

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 16:52                             ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-08 16:52 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 08, 2013 at 08:14:44AM +0100, Thierry Reding wrote:
> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> > On 03/07/2013 02:47 PM, Thierry Reding wrote:
> [...]
> > > In a nutshell (since some of the context isn't quoted anymore) the
> > > problem that we're trying to solve is that some of the embedded SoCs
> > > require per-root-port registers for configuration. The PCI DT
> > > specification doesn't make any provisions for this. A few alternatives
> > > have been discussed so far:
> > 
> > I'm not sure I follow. This is different than the host controller
> > registers? Why would this not just be multiple entries in the reg property?
> 
> Well the register regions are per root-port. On Tegra20 there's 2 of
> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> 10 (!). Adding all of them to the reg property of the host controller
> could work but it needs some way to match the reg entry to the root port
> similar to option 1 below.

Thomas just posted an implementation like this, please see:

http://www.spinics.net/lists/arm-kernel/msg228749.html

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 16:52                             ` Jason Gunthorpe
@ 2013-03-08 19:12                               ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-08 19:12 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Rob Herring, Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, Grant Likely, linux-arm-kernel,
	devicetree-discuss

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

On Fri, Mar 08, 2013 at 09:52:28AM -0700, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 08:14:44AM +0100, Thierry Reding wrote:
> > On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> > > On 03/07/2013 02:47 PM, Thierry Reding wrote:
> > [...]
> > > > In a nutshell (since some of the context isn't quoted anymore) the
> > > > problem that we're trying to solve is that some of the embedded SoCs
> > > > require per-root-port registers for configuration. The PCI DT
> > > > specification doesn't make any provisions for this. A few alternatives
> > > > have been discussed so far:
> > > 
> > > I'm not sure I follow. This is different than the host controller
> > > registers? Why would this not just be multiple entries in the reg property?
> > 
> > Well the register regions are per root-port. On Tegra20 there's 2 of
> > them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> > 10 (!). Adding all of them to the reg property of the host controller
> > could work but it needs some way to match the reg entry to the root port
> > similar to option 1 below.
> 
> Thomas just posted an implementation like this, please see:
> 
> http://www.spinics.net/lists/arm-kernel/msg228749.html

Oh well. I don't like it, but if that's the way it has to be, then so be
it. Any reason why the reg-names can't match the root port's node name
so that it can be used directly instead of going through hoops to
construct a string from extra parameters? Like below:

	pcie-controller {
		regs = <...>;
		reg-names = ..., "pcie@1,0", "pcie@2,0";

		pcie@1,0 {
			...
		};

		pcie@2,0 {
			...
		};
	};

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 19:12                               ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-08 19:12 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 08, 2013 at 09:52:28AM -0700, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 08:14:44AM +0100, Thierry Reding wrote:
> > On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> > > On 03/07/2013 02:47 PM, Thierry Reding wrote:
> > [...]
> > > > In a nutshell (since some of the context isn't quoted anymore) the
> > > > problem that we're trying to solve is that some of the embedded SoCs
> > > > require per-root-port registers for configuration. The PCI DT
> > > > specification doesn't make any provisions for this. A few alternatives
> > > > have been discussed so far:
> > > 
> > > I'm not sure I follow. This is different than the host controller
> > > registers? Why would this not just be multiple entries in the reg property?
> > 
> > Well the register regions are per root-port. On Tegra20 there's 2 of
> > them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> > 10 (!). Adding all of them to the reg property of the host controller
> > could work but it needs some way to match the reg entry to the root port
> > similar to option 1 below.
> 
> Thomas just posted an implementation like this, please see:
> 
> http://www.spinics.net/lists/arm-kernel/msg228749.html

Oh well. I don't like it, but if that's the way it has to be, then so be
it. Any reason why the reg-names can't match the root port's node name
so that it can be used directly instead of going through hoops to
construct a string from extra parameters? Like below:

	pcie-controller {
		regs = <...>;
		reg-names = ..., "pcie at 1,0", "pcie at 2,0";

		pcie at 1,0 {
			...
		};

		pcie at 2,0 {
			...
		};
	};

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130308/c48fa649/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 19:12                               ` Thierry Reding
  (?)
@ 2013-03-08 19:43                                   ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-08 19:43 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Jason Gunthorpe, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	Tawfik Bayouk, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/8/2013 9:12 AM, Thierry Reding wrote:
> On Fri, Mar 08, 2013 at 09:52:28AM -0700, Jason Gunthorpe wrote:
>> On Fri, Mar 08, 2013 at 08:14:44AM +0100, Thierry Reding wrote:
>>> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
>>>> On 03/07/2013 02:47 PM, Thierry Reding wrote:
>>> [...]
>>>>> In a nutshell (since some of the context isn't quoted anymore) the
>>>>> problem that we're trying to solve is that some of the embedded SoCs
>>>>> require per-root-port registers for configuration. The PCI DT
>>>>> specification doesn't make any provisions for this. A few alternatives
>>>>> have been discussed so far:
>>>>
>>>> I'm not sure I follow. This is different than the host controller
>>>> registers? Why would this not just be multiple entries in the reg property?
>>>
>>> Well the register regions are per root-port. On Tegra20 there's 2 of
>>> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
>>> 10 (!). Adding all of them to the reg property of the host controller
>>> could work but it needs some way to match the reg entry to the root port
>>> similar to option 1 below.
>>
>> Thomas just posted an implementation like this, please see:
>>
>> http://www.spinics.net/lists/arm-kernel/msg228749.html

The example in that posting looks messed up to me.

1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
address in the address space defined by its parent - because the form of
the parent's ranges property indicates that it's a PCI-style address
form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
the type (I/O, memory, etc).

2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
<0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.

What is address space represented by the pcie-controller node?  reg
properties in child nodes must contain proper addresses in some
well-defined address space.  It looks like this example is trying to put
half of the PCI bus abstraction in the pcie-controller node and the rest
in the pcie sub-nodes, with some handwaving address-space splicing
between the two.  That's not kosher.  Each level of the tree needs a
coherent address space definition, otherwise the address space
"calculus" breaks.

> 
> Oh well. I don't like it, but if that's the way it has to be, then so be
> it. Any reason why the reg-names can't match the root port's node name
> so that it can be used directly instead of going through hoops to
> construct a string from extra parameters? Like below:
> 
> 	pcie-controller {
> 		regs = <...>;
> 		reg-names = ..., "pcie@1,0", "pcie@2,0";
> 
> 		pcie@1,0 {
> 			...
> 		};
> 
> 		pcie@2,0 {
> 			...
> 		};
> 	};
> 
> Thierry
> 
> 
> 
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 19:43                                   ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-08 19:43 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/8/2013 9:12 AM, Thierry Reding wrote:
> On Fri, Mar 08, 2013 at 09:52:28AM -0700, Jason Gunthorpe wrote:
>> On Fri, Mar 08, 2013 at 08:14:44AM +0100, Thierry Reding wrote:
>>> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
>>>> On 03/07/2013 02:47 PM, Thierry Reding wrote:
>>> [...]
>>>>> In a nutshell (since some of the context isn't quoted anymore) the
>>>>> problem that we're trying to solve is that some of the embedded SoCs
>>>>> require per-root-port registers for configuration. The PCI DT
>>>>> specification doesn't make any provisions for this. A few alternatives
>>>>> have been discussed so far:
>>>>
>>>> I'm not sure I follow. This is different than the host controller
>>>> registers? Why would this not just be multiple entries in the reg property?
>>>
>>> Well the register regions are per root-port. On Tegra20 there's 2 of
>>> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
>>> 10 (!). Adding all of them to the reg property of the host controller
>>> could work but it needs some way to match the reg entry to the root port
>>> similar to option 1 below.
>>
>> Thomas just posted an implementation like this, please see:
>>
>> http://www.spinics.net/lists/arm-kernel/msg228749.html

The example in that posting looks messed up to me.

1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
address in the address space defined by its parent - because the form of
the parent's ranges property indicates that it's a PCI-style address
form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
the type (I/O, memory, etc).

2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
<0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.

What is address space represented by the pcie-controller node?  reg
properties in child nodes must contain proper addresses in some
well-defined address space.  It looks like this example is trying to put
half of the PCI bus abstraction in the pcie-controller node and the rest
in the pcie sub-nodes, with some handwaving address-space splicing
between the two.  That's not kosher.  Each level of the tree needs a
coherent address space definition, otherwise the address space
"calculus" breaks.

> 
> Oh well. I don't like it, but if that's the way it has to be, then so be
> it. Any reason why the reg-names can't match the root port's node name
> so that it can be used directly instead of going through hoops to
> construct a string from extra parameters? Like below:
> 
> 	pcie-controller {
> 		regs = <...>;
> 		reg-names = ..., "pcie@1,0", "pcie@2,0";
> 
> 		pcie@1,0 {
> 			...
> 		};
> 
> 		pcie@2,0 {
> 			...
> 		};
> 	};
> 
> Thierry
> 
> 
> 
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 19:43                                   ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-08 19:43 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/8/2013 9:12 AM, Thierry Reding wrote:
> On Fri, Mar 08, 2013 at 09:52:28AM -0700, Jason Gunthorpe wrote:
>> On Fri, Mar 08, 2013 at 08:14:44AM +0100, Thierry Reding wrote:
>>> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
>>>> On 03/07/2013 02:47 PM, Thierry Reding wrote:
>>> [...]
>>>>> In a nutshell (since some of the context isn't quoted anymore) the
>>>>> problem that we're trying to solve is that some of the embedded SoCs
>>>>> require per-root-port registers for configuration. The PCI DT
>>>>> specification doesn't make any provisions for this. A few alternatives
>>>>> have been discussed so far:
>>>>
>>>> I'm not sure I follow. This is different than the host controller
>>>> registers? Why would this not just be multiple entries in the reg property?
>>>
>>> Well the register regions are per root-port. On Tegra20 there's 2 of
>>> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
>>> 10 (!). Adding all of them to the reg property of the host controller
>>> could work but it needs some way to match the reg entry to the root port
>>> similar to option 1 below.
>>
>> Thomas just posted an implementation like this, please see:
>>
>> http://www.spinics.net/lists/arm-kernel/msg228749.html

The example in that posting looks messed up to me.

1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
address in the address space defined by its parent - because the form of
the parent's ranges property indicates that it's a PCI-style address
form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
the type (I/O, memory, etc).

2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
<0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.

What is address space represented by the pcie-controller node?  reg
properties in child nodes must contain proper addresses in some
well-defined address space.  It looks like this example is trying to put
half of the PCI bus abstraction in the pcie-controller node and the rest
in the pcie sub-nodes, with some handwaving address-space splicing
between the two.  That's not kosher.  Each level of the tree needs a
coherent address space definition, otherwise the address space
"calculus" breaks.

> 
> Oh well. I don't like it, but if that's the way it has to be, then so be
> it. Any reason why the reg-names can't match the root port's node name
> so that it can be used directly instead of going through hoops to
> construct a string from extra parameters? Like below:
> 
> 	pcie-controller {
> 		regs = <...>;
> 		reg-names = ..., "pcie at 1,0", "pcie at 2,0";
> 
> 		pcie at 1,0 {
> 			...
> 		};
> 
> 		pcie at 2,0 {
> 			...
> 		};
> 	};
> 
> Thierry
> 
> 
> 
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss at lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 19:43                                   ` Mitch Bradley
@ 2013-03-08 20:02                                     ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-08 20:02 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:

> >> http://www.spinics.net/lists/arm-kernel/msg228749.html
> 
> The example in that posting looks messed up to me.
> 
> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
> address in the address space defined by its parent - because the form of
> the parent's ranges property indicates that it's a PCI-style address
> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
> the type (I/O, memory, etc).

You need to review the OF PCI bindings to make sense of this.  The
subnodes are PCI devices. Those PCI devices show up in the
configuration space (ie lspci). They are the PCI root port bridges.

The reg value follows the OF PCI spec and has the configuration
address of the bridge. For the first port's bridge this address in
lspci format is 00:01.0 which encodes to <0x800 0 0>

The Linux OF PCI core uses the reg value in this format to match the
OF node in the DT to the PCI device node as it does PCI discovery.

> 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.

@0,1,0 (bus,device,fn) could be more appropriate, but that is
cosmetic?

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 20:02                                     ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-08 20:02 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:

> >> http://www.spinics.net/lists/arm-kernel/msg228749.html
> 
> The example in that posting looks messed up to me.
> 
> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
> address in the address space defined by its parent - because the form of
> the parent's ranges property indicates that it's a PCI-style address
> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
> the type (I/O, memory, etc).

You need to review the OF PCI bindings to make sense of this.  The
subnodes are PCI devices. Those PCI devices show up in the
configuration space (ie lspci). They are the PCI root port bridges.

The reg value follows the OF PCI spec and has the configuration
address of the bridge. For the first port's bridge this address in
lspci format is 00:01.0 which encodes to <0x800 0 0>

The Linux OF PCI core uses the reg value in this format to match the
OF node in the DT to the PCI device node as it does PCI discovery.

> 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.

@0,1,0 (bus,device,fn) could be more appropriate, but that is
cosmetic?

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 20:02                                     ` Jason Gunthorpe
  (?)
@ 2013-03-08 20:13                                         ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-08 20:13 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r


[-- Attachment #1.1: Type: text/plain, Size: 535 bytes --]

On Fri, Mar 08, 2013 at 01:02:46PM -0700, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
[...]
> > 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> > <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> 
> @0,1,0 (bus,device,fn) could be more appropriate, but that is
> cosmetic?

The OF PCI specification is pretty strict about this as well. It says in
section 2.2.1.3. that only the DD and DD,FF forms can appear in a device
path.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 20:13                                         ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-08 20:13 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

On Fri, Mar 08, 2013 at 01:02:46PM -0700, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
[...]
> > 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> > <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> 
> @0,1,0 (bus,device,fn) could be more appropriate, but that is
> cosmetic?

The OF PCI specification is pretty strict about this as well. It says in
section 2.2.1.3. that only the DD and DD,FF forms can appear in a device
path.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 20:13                                         ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-08 20:13 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 08, 2013 at 01:02:46PM -0700, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
[...]
> > 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> > <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> 
> @0,1,0 (bus,device,fn) could be more appropriate, but that is
> cosmetic?

The OF PCI specification is pretty strict about this as well. It says in
section 2.2.1.3. that only the DD and DD,FF forms can appear in a device
path.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130308/82f997bf/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08  7:14                           ` Thierry Reding
@ 2013-03-08 23:12                             ` Rob Herring
  -1 siblings, 0 replies; 291+ messages in thread
From: Rob Herring @ 2013-03-08 23:12 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, Grant Likely, linux-arm-kernel,
	devicetree-discuss

On 03/08/2013 01:14 AM, Thierry Reding wrote:
> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
>> On 03/07/2013 02:47 PM, Thierry Reding wrote:
> [...]
>>> In a nutshell (since some of the context isn't quoted anymore) the
>>> problem that we're trying to solve is that some of the embedded SoCs
>>> require per-root-port registers for configuration. The PCI DT
>>> specification doesn't make any provisions for this. A few alternatives
>>> have been discussed so far:
>>
>> I'm not sure I follow. This is different than the host controller
>> registers? Why would this not just be multiple entries in the reg property?
> 
> Well the register regions are per root-port. On Tegra20 there's 2 of
> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> 10 (!). Adding all of them to the reg property of the host controller
> could work but it needs some way to match the reg entry to the root port
> similar to option 1 below.

The compatible property of the PCI host controller can imply what each
index of the reg property entries is for.

> 
> Adding a property in the root port nodes seems like a cleaner and more
> accurate representation of the hardware to me, but if that's not
> acceptable perhaps we need to bite the bullet and add the code to look
> the registers up from the parent's reg property.

What I don't like is a new property defined to describe mmio addresses.
We already have a property for that and it is "reg". But I think I'm
still missing something:

>>> 			pci@0,1 {
>>> 				...
>>> 				reg = <0x00000800 0 0 0 0>;

Is this a PCI bus address?

>>> 				regs = <0x80000000 0x00001000>;
>>> 			};

Rob

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 23:12                             ` Rob Herring
  0 siblings, 0 replies; 291+ messages in thread
From: Rob Herring @ 2013-03-08 23:12 UTC (permalink / raw)
  To: linux-arm-kernel

On 03/08/2013 01:14 AM, Thierry Reding wrote:
> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
>> On 03/07/2013 02:47 PM, Thierry Reding wrote:
> [...]
>>> In a nutshell (since some of the context isn't quoted anymore) the
>>> problem that we're trying to solve is that some of the embedded SoCs
>>> require per-root-port registers for configuration. The PCI DT
>>> specification doesn't make any provisions for this. A few alternatives
>>> have been discussed so far:
>>
>> I'm not sure I follow. This is different than the host controller
>> registers? Why would this not just be multiple entries in the reg property?
> 
> Well the register regions are per root-port. On Tegra20 there's 2 of
> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> 10 (!). Adding all of them to the reg property of the host controller
> could work but it needs some way to match the reg entry to the root port
> similar to option 1 below.

The compatible property of the PCI host controller can imply what each
index of the reg property entries is for.

> 
> Adding a property in the root port nodes seems like a cleaner and more
> accurate representation of the hardware to me, but if that's not
> acceptable perhaps we need to bite the bullet and add the code to look
> the registers up from the parent's reg property.

What I don't like is a new property defined to describe mmio addresses.
We already have a property for that and it is "reg". But I think I'm
still missing something:

>>> 			pci at 0,1 {
>>> 				...
>>> 				reg = <0x00000800 0 0 0 0>;

Is this a PCI bus address?

>>> 				regs = <0x80000000 0x00001000>;
>>> 			};

Rob

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 20:02                                     ` Jason Gunthorpe
  (?)
@ 2013-03-08 23:46                                         ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-08 23:46 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/8/2013 10:02 AM, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> 
>>>> http://www.spinics.net/lists/arm-kernel/msg228749.html
>>
>> The example in that posting looks messed up to me.
>>
>> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
>> address in the address space defined by its parent - because the form of
>> the parent's ranges property indicates that it's a PCI-style address
>> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
>> the type (I/O, memory, etc).
> 
> You need to review the OF PCI bindings to make sense of this.  The
> subnodes are PCI devices. Those PCI devices show up in the
> configuration space (ie lspci). They are the PCI root port bridges.

As it turns out, I wrote those bindings, almost 20 years ago.

Having dug through old versions of that patch, I think I see the source
of the confusion.

In a very early version of the patch -
  http://www.spinics.net/lists/arm-kernel/msg211839.html
the pcie-controller address space had 1 address cell, essentially
propagating the CPU address down to the subordinate nodes, and the
subordinate pcie nodes had no child address specification.  Somebody
correctly observed that PCI buses need to export 3/2 address/size cells
to their children.

But that stipulation applies only to the subordinate pcie nodes, not
necessarily to the enclosing pcie-controller node.

A later version -
   http://permalink.gmane.org/gmane.linux.kernel.pci/20358
applied 3/2 PCI addressing at both levels.  That is somewhat sensible if
you treat the top level as a "PCI root complex" and if the subordinate
port control registers are really PCI config header blocks.

But look at the ranges entries therein.  PCI config address <0x800 ..>
maps to CPU address 0xd004000 size 0x2000 and <0x1000 ..> maps to
0xd0044000 size 0x2000.  0x800 size 0x2000 encroaches into the <0x1000
..> range.  That can't be right.

That, plus the first version of the patch, makes me think that these
root-port control registers might not really be PCI config registers, in
which case the use of PCI addressing at the top level is a fiction.

Furthermore, we have the fact that pcie@0,0 corresponds to
marvell,pcie-port = <0> and marvell,pcie-lane = <0>, and similarly for
pcie@0,1 and pcie@0,2.  That correspondence still appears in the recent
patch.

So it looks like the "@0,0" addressing attempts to represent lane,port
instead of device,function PCI config space addressing.  0,0 is not the
right PCI device/function number for reg=<0x800 ...> - 0x800 is device
1, not device 0.

Another problem is the device_type values.  If the top level is to be
interpreted as a PCI addressing domain, it should have a device_type=pci
property.

One solution is for the top node to use 1-cell addresses, passing the
CPU address to the subordinates.  The subordinate nodes use the standard
PCI address representation.  The subordinate reg properties just list
the CPU address of the control registers.  Each subordinate has a ranges
properties to translate PCI to CPU addresses and a bus-range property
for PCI bus numbers (unless those are determined dynamically).

What you lose with that is the ability to refer to the nodes by
port,lane.  If that is important, then the top address domain needs an
additional cell to say whether a given address is a CPU address or
port,lane.  The top node would need a ranges to map the various forms
into CPU addresses.  The child reg properties could use the port,lane
form, and marvell,pcie-{port,lane} would disappear.

In neither case would you need the controversial "reg-names" thing.

The tradeoff:

Existing proposed patch: Violates addressing rules, requires funny new
"reg-names" mechanism, pretends to create a PCI bus level that does not
exist.

One-cell top address: Follows addressing rules, may be able to use
existing "simple-bus" address translation code, does not permit the
pretty @lane,port human-readable form.

Two-cell top address: Follows addressing rules, possibly requires new
code to translate this particular 2-address-cell format, permits
@lane,port representation.

> 
> The reg value follows the OF PCI spec and has the configuration
> address of the bridge. For the first port's bridge this address in
> lspci format is 00:01.0 which encodes to <0x800 0 0>
> 
> The Linux OF PCI core uses the reg value in this format to match the
> OF node in the DT to the PCI device node as it does PCI discovery.
> 
>> 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
>> <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> 
> @0,1,0 (bus,device,fn) could be more appropriate, but that is
> cosmetic?

Except that the @0,0 in this case seems to represent port,lane and not
device,function , as argued above.

> 
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 23:46                                         ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-08 23:46 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/8/2013 10:02 AM, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> 
>>>> http://www.spinics.net/lists/arm-kernel/msg228749.html
>>
>> The example in that posting looks messed up to me.
>>
>> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
>> address in the address space defined by its parent - because the form of
>> the parent's ranges property indicates that it's a PCI-style address
>> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
>> the type (I/O, memory, etc).
> 
> You need to review the OF PCI bindings to make sense of this.  The
> subnodes are PCI devices. Those PCI devices show up in the
> configuration space (ie lspci). They are the PCI root port bridges.

As it turns out, I wrote those bindings, almost 20 years ago.

Having dug through old versions of that patch, I think I see the source
of the confusion.

In a very early version of the patch -
  http://www.spinics.net/lists/arm-kernel/msg211839.html
the pcie-controller address space had 1 address cell, essentially
propagating the CPU address down to the subordinate nodes, and the
subordinate pcie nodes had no child address specification.  Somebody
correctly observed that PCI buses need to export 3/2 address/size cells
to their children.

But that stipulation applies only to the subordinate pcie nodes, not
necessarily to the enclosing pcie-controller node.

A later version -
   http://permalink.gmane.org/gmane.linux.kernel.pci/20358
applied 3/2 PCI addressing at both levels.  That is somewhat sensible if
you treat the top level as a "PCI root complex" and if the subordinate
port control registers are really PCI config header blocks.

But look at the ranges entries therein.  PCI config address <0x800 ..>
maps to CPU address 0xd004000 size 0x2000 and <0x1000 ..> maps to
0xd0044000 size 0x2000.  0x800 size 0x2000 encroaches into the <0x1000
..> range.  That can't be right.

That, plus the first version of the patch, makes me think that these
root-port control registers might not really be PCI config registers, in
which case the use of PCI addressing at the top level is a fiction.

Furthermore, we have the fact that pcie@0,0 corresponds to
marvell,pcie-port = <0> and marvell,pcie-lane = <0>, and similarly for
pcie@0,1 and pcie@0,2.  That correspondence still appears in the recent
patch.

So it looks like the "@0,0" addressing attempts to represent lane,port
instead of device,function PCI config space addressing.  0,0 is not the
right PCI device/function number for reg=<0x800 ...> - 0x800 is device
1, not device 0.

Another problem is the device_type values.  If the top level is to be
interpreted as a PCI addressing domain, it should have a device_type=pci
property.

One solution is for the top node to use 1-cell addresses, passing the
CPU address to the subordinates.  The subordinate nodes use the standard
PCI address representation.  The subordinate reg properties just list
the CPU address of the control registers.  Each subordinate has a ranges
properties to translate PCI to CPU addresses and a bus-range property
for PCI bus numbers (unless those are determined dynamically).

What you lose with that is the ability to refer to the nodes by
port,lane.  If that is important, then the top address domain needs an
additional cell to say whether a given address is a CPU address or
port,lane.  The top node would need a ranges to map the various forms
into CPU addresses.  The child reg properties could use the port,lane
form, and marvell,pcie-{port,lane} would disappear.

In neither case would you need the controversial "reg-names" thing.

The tradeoff:

Existing proposed patch: Violates addressing rules, requires funny new
"reg-names" mechanism, pretends to create a PCI bus level that does not
exist.

One-cell top address: Follows addressing rules, may be able to use
existing "simple-bus" address translation code, does not permit the
pretty @lane,port human-readable form.

Two-cell top address: Follows addressing rules, possibly requires new
code to translate this particular 2-address-cell format, permits
@lane,port representation.

> 
> The reg value follows the OF PCI spec and has the configuration
> address of the bridge. For the first port's bridge this address in
> lspci format is 00:01.0 which encodes to <0x800 0 0>
> 
> The Linux OF PCI core uses the reg value in this format to match the
> OF node in the DT to the PCI device node as it does PCI discovery.
> 
>> 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
>> <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> 
> @0,1,0 (bus,device,fn) could be more appropriate, but that is
> cosmetic?

Except that the @0,0 in this case seems to represent port,lane and not
device,function , as argued above.

> 
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-08 23:46                                         ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-08 23:46 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/8/2013 10:02 AM, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> 
>>>> http://www.spinics.net/lists/arm-kernel/msg228749.html
>>
>> The example in that posting looks messed up to me.
>>
>> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
>> address in the address space defined by its parent - because the form of
>> the parent's ranges property indicates that it's a PCI-style address
>> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
>> the type (I/O, memory, etc).
> 
> You need to review the OF PCI bindings to make sense of this.  The
> subnodes are PCI devices. Those PCI devices show up in the
> configuration space (ie lspci). They are the PCI root port bridges.

As it turns out, I wrote those bindings, almost 20 years ago.

Having dug through old versions of that patch, I think I see the source
of the confusion.

In a very early version of the patch -
  http://www.spinics.net/lists/arm-kernel/msg211839.html
the pcie-controller address space had 1 address cell, essentially
propagating the CPU address down to the subordinate nodes, and the
subordinate pcie nodes had no child address specification.  Somebody
correctly observed that PCI buses need to export 3/2 address/size cells
to their children.

But that stipulation applies only to the subordinate pcie nodes, not
necessarily to the enclosing pcie-controller node.

A later version -
   http://permalink.gmane.org/gmane.linux.kernel.pci/20358
applied 3/2 PCI addressing at both levels.  That is somewhat sensible if
you treat the top level as a "PCI root complex" and if the subordinate
port control registers are really PCI config header blocks.

But look at the ranges entries therein.  PCI config address <0x800 ..>
maps to CPU address 0xd004000 size 0x2000 and <0x1000 ..> maps to
0xd0044000 size 0x2000.  0x800 size 0x2000 encroaches into the <0x1000
..> range.  That can't be right.

That, plus the first version of the patch, makes me think that these
root-port control registers might not really be PCI config registers, in
which case the use of PCI addressing at the top level is a fiction.

Furthermore, we have the fact that pcie at 0,0 corresponds to
marvell,pcie-port = <0> and marvell,pcie-lane = <0>, and similarly for
pcie at 0,1 and pcie at 0,2.  That correspondence still appears in the recent
patch.

So it looks like the "@0,0" addressing attempts to represent lane,port
instead of device,function PCI config space addressing.  0,0 is not the
right PCI device/function number for reg=<0x800 ...> - 0x800 is device
1, not device 0.

Another problem is the device_type values.  If the top level is to be
interpreted as a PCI addressing domain, it should have a device_type=pci
property.

One solution is for the top node to use 1-cell addresses, passing the
CPU address to the subordinates.  The subordinate nodes use the standard
PCI address representation.  The subordinate reg properties just list
the CPU address of the control registers.  Each subordinate has a ranges
properties to translate PCI to CPU addresses and a bus-range property
for PCI bus numbers (unless those are determined dynamically).

What you lose with that is the ability to refer to the nodes by
port,lane.  If that is important, then the top address domain needs an
additional cell to say whether a given address is a CPU address or
port,lane.  The top node would need a ranges to map the various forms
into CPU addresses.  The child reg properties could use the port,lane
form, and marvell,pcie-{port,lane} would disappear.

In neither case would you need the controversial "reg-names" thing.

The tradeoff:

Existing proposed patch: Violates addressing rules, requires funny new
"reg-names" mechanism, pretends to create a PCI bus level that does not
exist.

One-cell top address: Follows addressing rules, may be able to use
existing "simple-bus" address translation code, does not permit the
pretty @lane,port human-readable form.

Two-cell top address: Follows addressing rules, possibly requires new
code to translate this particular 2-address-cell format, permits
@lane,port representation.

> 
> The reg value follows the OF PCI spec and has the configuration
> address of the bridge. For the first port's bridge this address in
> lspci format is 00:01.0 which encodes to <0x800 0 0>
> 
> The Linux OF PCI core uses the reg value in this format to match the
> OF node in the DT to the PCI device node as it does PCI discovery.
> 
>> 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
>> <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> 
> @0,1,0 (bus,device,fn) could be more appropriate, but that is
> cosmetic?

Except that the @0,0 in this case seems to represent port,lane and not
device,function , as argued above.

> 
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 23:46                                         ` Mitch Bradley
@ 2013-03-09  1:31                                           ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-09  1:31 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Fri, Mar 08, 2013 at 01:46:13PM -1000, Mitch Bradley wrote:
> On 3/8/2013 10:02 AM, Jason Gunthorpe wrote:
> > On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> > 
> >>>> http://www.spinics.net/lists/arm-kernel/msg228749.html
> >>
> >> The example in that posting looks messed up to me.
> >>
> >> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
> >> address in the address space defined by its parent - because the form of
> >> the parent's ranges property indicates that it's a PCI-style address
> >> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
> >> the type (I/O, memory, etc).
> > 
> > You need to review the OF PCI bindings to make sense of this.  The
> > subnodes are PCI devices. Those PCI devices show up in the
> > configuration space (ie lspci). They are the PCI root port bridges.
> 
> As it turns out, I wrote those bindings, almost 20 years ago.
> 
> Having dug through old versions of that patch, I think I see the source
> of the confusion.

Not sure, I think this has gone into a weird tangent, your conclusions
don't match how the PCI-E root complex is presented to the kernel.

Lets just go back to the start?

The pci-controller node is a root complex.

The children of that node are elements in the root complex - they are
the root port bridges. They are all on PCI bus 0.

Each root port bridge responds to configuration cycles, and has a PCI
configuration space. They show up in lspci.

The host controller device driver knows how to issue configuration
cycles, and it knows how to deliver configuration cycles on bus 0 to
the bridges.

This one:
+			pcie@0,0 {
+				device_type = "pci";
+				reg = <0x0800 0 0 0 0>;

Responds to bus 0, device 1, function 0.

Each root port bridge has an associated non-PCI MMIO address
space. The child above is associated with 0xd0040000. This MMIO
address space is needed by the host controller driver to operate the
port.

The text after the @ is a mistake, Linux doesn't enforce anything
about this text so nobody noticed till now, it should be corrected to
@1,0, similarly for the second one.

Also, from your comments the top level should gain a 'device_type =
"pci"'.

So, from that point, does the DT start to make sense? Are there other
problems?

The question at hand is how do you associate the non-PCI MMIO space
0xd0040000 with the root port bridge 'pcie@1,0' ? The patch proposes
to use a reg array in the controller plus reg-names to do this.

I have trouble making sense of your suggestions to switch away from
5dw addressing. It is critical that the DT child stanza be associated
with the PCI discovered root port bridge. By my understanding Linux
does this based on the configuration space format 'reg' value. How is
that association possible if you switch things away from 5dw
addressing?

Also, this is for firmware-less embedded. Linux is required to scan
the bus and do full address assignment of all PCI devies, including
the root port bridges. The ranges property on the top node specifies
the addressing apertures that are available for this process. This is
common practice, there are many examples of this in kernel dts for PCI
controllers.

It is impossible to specify any detail for the bridges (be it address
ranges or bus number ranges) because we have no idea what discovery
will find, or what values Linux will choose to assign.

> In neither case would you need the controversial "reg-names" thing.

reg-names has been a standard feature in the Linux kernel for a bit
already..

Thanks a lot for looking at this,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-09  1:31                                           ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-09  1:31 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 08, 2013 at 01:46:13PM -1000, Mitch Bradley wrote:
> On 3/8/2013 10:02 AM, Jason Gunthorpe wrote:
> > On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> > 
> >>>> http://www.spinics.net/lists/arm-kernel/msg228749.html
> >>
> >> The example in that posting looks messed up to me.
> >>
> >> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
> >> address in the address space defined by its parent - because the form of
> >> the parent's ranges property indicates that it's a PCI-style address
> >> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
> >> the type (I/O, memory, etc).
> > 
> > You need to review the OF PCI bindings to make sense of this.  The
> > subnodes are PCI devices. Those PCI devices show up in the
> > configuration space (ie lspci). They are the PCI root port bridges.
> 
> As it turns out, I wrote those bindings, almost 20 years ago.
> 
> Having dug through old versions of that patch, I think I see the source
> of the confusion.

Not sure, I think this has gone into a weird tangent, your conclusions
don't match how the PCI-E root complex is presented to the kernel.

Lets just go back to the start?

The pci-controller node is a root complex.

The children of that node are elements in the root complex - they are
the root port bridges. They are all on PCI bus 0.

Each root port bridge responds to configuration cycles, and has a PCI
configuration space. They show up in lspci.

The host controller device driver knows how to issue configuration
cycles, and it knows how to deliver configuration cycles on bus 0 to
the bridges.

This one:
+			pcie at 0,0 {
+				device_type = "pci";
+				reg = <0x0800 0 0 0 0>;

Responds to bus 0, device 1, function 0.

Each root port bridge has an associated non-PCI MMIO address
space. The child above is associated with 0xd0040000. This MMIO
address space is needed by the host controller driver to operate the
port.

The text after the @ is a mistake, Linux doesn't enforce anything
about this text so nobody noticed till now, it should be corrected to
@1,0, similarly for the second one.

Also, from your comments the top level should gain a 'device_type =
"pci"'.

So, from that point, does the DT start to make sense? Are there other
problems?

The question at hand is how do you associate the non-PCI MMIO space
0xd0040000 with the root port bridge 'pcie at 1,0' ? The patch proposes
to use a reg array in the controller plus reg-names to do this.

I have trouble making sense of your suggestions to switch away from
5dw addressing. It is critical that the DT child stanza be associated
with the PCI discovered root port bridge. By my understanding Linux
does this based on the configuration space format 'reg' value. How is
that association possible if you switch things away from 5dw
addressing?

Also, this is for firmware-less embedded. Linux is required to scan
the bus and do full address assignment of all PCI devies, including
the root port bridges. The ranges property on the top node specifies
the addressing apertures that are available for this process. This is
common practice, there are many examples of this in kernel dts for PCI
controllers.

It is impossible to specify any detail for the bridges (be it address
ranges or bus number ranges) because we have no idea what discovery
will find, or what values Linux will choose to assign.

> In neither case would you need the controversial "reg-names" thing.

reg-names has been a standard feature in the Linux kernel for a bit
already..

Thanks a lot for looking at this,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 19:12                               ` Thierry Reding
@ 2013-03-09  8:58                                 ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-09  8:58 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, Rob Herring, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, Grant Likely, linux-arm-kernel,
	devicetree-discuss

Dear Thierry Reding,

On Fri, 8 Mar 2013 20:12:27 +0100, Thierry Reding wrote:

> Oh well. I don't like it, but if that's the way it has to be, then so be
> it.

I don't like it too much either, but it's not a big problem either. It
works, and satisfies the DT requirements reasonably.

> Any reason why the reg-names can't match the root port's node name
> so that it can be used directly instead of going through hoops to
> construct a string from extra parameters? Like below:
> 
> 	pcie-controller {
> 		regs = <...>;
> 		reg-names = ..., "pcie@1,0", "pcie@2,0";
> 
> 		pcie@1,0 {
> 			...
> 		};
> 
> 		pcie@2,0 {
> 			...
> 		};
> 	};

Yes, I thought about doing this, but in my driver, I also use the
"pcie0.0" string to request the address decoding window from the
mvebu-mbus driver. So I would anyway have to construct this "pciX.Y"
string. But if you feel like using "pcie@X,Y" for the reg-names is
better even if I still need to construct the "pcieX.Y" string, then I
will be perfectly ok with making this change.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-09  8:58                                 ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-09  8:58 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Thierry Reding,

On Fri, 8 Mar 2013 20:12:27 +0100, Thierry Reding wrote:

> Oh well. I don't like it, but if that's the way it has to be, then so be
> it.

I don't like it too much either, but it's not a big problem either. It
works, and satisfies the DT requirements reasonably.

> Any reason why the reg-names can't match the root port's node name
> so that it can be used directly instead of going through hoops to
> construct a string from extra parameters? Like below:
> 
> 	pcie-controller {
> 		regs = <...>;
> 		reg-names = ..., "pcie at 1,0", "pcie at 2,0";
> 
> 		pcie at 1,0 {
> 			...
> 		};
> 
> 		pcie at 2,0 {
> 			...
> 		};
> 	};

Yes, I thought about doing this, but in my driver, I also use the
"pcie0.0" string to request the address decoding window from the
mvebu-mbus driver. So I would anyway have to construct this "pciX.Y"
string. But if you feel like using "pcie at X,Y" for the reg-names is
better even if I still need to construct the "pcieX.Y" string, then I
will be perfectly ok with making this change.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 23:12                             ` Rob Herring
  (?)
@ 2013-03-09 11:10                                 ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-09 11:10 UTC (permalink / raw)
  To: Rob Herring
  Cc: Lior Amsalem, Russell King - ARM Linux, Jason Cooper,
	Andrew Lunn, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Jason Gunthorpe, Maen Suleiman, Nadav Haklai, Shadi Ammouri,
	Bjorn Helgaas, Tawfik Bayouk,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r


[-- Attachment #1.1: Type: text/plain, Size: 2654 bytes --]

On Fri, Mar 08, 2013 at 05:12:04PM -0600, Rob Herring wrote:
> On 03/08/2013 01:14 AM, Thierry Reding wrote:
> > On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> >> On 03/07/2013 02:47 PM, Thierry Reding wrote:
> > [...]
> >>> In a nutshell (since some of the context isn't quoted anymore) the
> >>> problem that we're trying to solve is that some of the embedded SoCs
> >>> require per-root-port registers for configuration. The PCI DT
> >>> specification doesn't make any provisions for this. A few alternatives
> >>> have been discussed so far:
> >>
> >> I'm not sure I follow. This is different than the host controller
> >> registers? Why would this not just be multiple entries in the reg property?
> > 
> > Well the register regions are per root-port. On Tegra20 there's 2 of
> > them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> > 10 (!). Adding all of them to the reg property of the host controller
> > could work but it needs some way to match the reg entry to the root port
> > similar to option 1 below.
> 
> The compatible property of the PCI host controller can imply what each
> index of the reg property entries is for.
> 
> > 
> > Adding a property in the root port nodes seems like a cleaner and more
> > accurate representation of the hardware to me, but if that's not
> > acceptable perhaps we need to bite the bullet and add the code to look
> > the registers up from the parent's reg property.
> 
> What I don't like is a new property defined to describe mmio addresses.
> We already have a property for that and it is "reg".

Okay, understood.

> But I think I'm still missing something:
> 
> >>> 			pci@0,1 {
> >>> 				...
> >>> 				reg = <0x00000800 0 0 0 0>;
> 
> Is this a PCI bus address?

Yes. The example is slightly wrong. pci@0,1 should actually be pci@1,0.
That means it is device 1, function 0 on the bus (implicitly 0 in this
case). That matches the values in the reg property, whose first cell is
defined as follows, quoting the PCI OF specification:

	npt000ss bbbbbbbb dddddfff rrrrrrrr

Where:

	n         is 0 if the address is relocatable, 1 otherwise
	p         is 1 if the addressable region is "prefetchable", 0
	          otherwise
	t         is 1 if the address is aliased (for non-relocatable
	          I/O), below 1 MB (for Memory), or below 64 KB (for
	          relocatable I/O).
	ss        is the space code, denoting the address space
	bbbbbbbb  is the 8-bit Bus Number
	ddddd     is the 5-bit Device Number
	fff       is the 3-bit Function Number
	rrrrrrrr  is the 8-bit Register Number

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-09 11:10                                 ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-09 11:10 UTC (permalink / raw)
  To: Rob Herring
  Cc: Jason Gunthorpe, Thomas Petazzoni, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Arnd Bergmann,
	Stephen Warren, linux-pci, Eran Ben-Avi, Nadav Haklai,
	Maen Suleiman, Shadi Ammouri, Bjorn Helgaas, Gregory Clement,
	Tawfik Bayouk, Grant Likely, linux-arm-kernel,
	devicetree-discuss

[-- Attachment #1: Type: text/plain, Size: 2654 bytes --]

On Fri, Mar 08, 2013 at 05:12:04PM -0600, Rob Herring wrote:
> On 03/08/2013 01:14 AM, Thierry Reding wrote:
> > On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> >> On 03/07/2013 02:47 PM, Thierry Reding wrote:
> > [...]
> >>> In a nutshell (since some of the context isn't quoted anymore) the
> >>> problem that we're trying to solve is that some of the embedded SoCs
> >>> require per-root-port registers for configuration. The PCI DT
> >>> specification doesn't make any provisions for this. A few alternatives
> >>> have been discussed so far:
> >>
> >> I'm not sure I follow. This is different than the host controller
> >> registers? Why would this not just be multiple entries in the reg property?
> > 
> > Well the register regions are per root-port. On Tegra20 there's 2 of
> > them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> > 10 (!). Adding all of them to the reg property of the host controller
> > could work but it needs some way to match the reg entry to the root port
> > similar to option 1 below.
> 
> The compatible property of the PCI host controller can imply what each
> index of the reg property entries is for.
> 
> > 
> > Adding a property in the root port nodes seems like a cleaner and more
> > accurate representation of the hardware to me, but if that's not
> > acceptable perhaps we need to bite the bullet and add the code to look
> > the registers up from the parent's reg property.
> 
> What I don't like is a new property defined to describe mmio addresses.
> We already have a property for that and it is "reg".

Okay, understood.

> But I think I'm still missing something:
> 
> >>> 			pci@0,1 {
> >>> 				...
> >>> 				reg = <0x00000800 0 0 0 0>;
> 
> Is this a PCI bus address?

Yes. The example is slightly wrong. pci@0,1 should actually be pci@1,0.
That means it is device 1, function 0 on the bus (implicitly 0 in this
case). That matches the values in the reg property, whose first cell is
defined as follows, quoting the PCI OF specification:

	npt000ss bbbbbbbb dddddfff rrrrrrrr

Where:

	n         is 0 if the address is relocatable, 1 otherwise
	p         is 1 if the addressable region is "prefetchable", 0
	          otherwise
	t         is 1 if the address is aliased (for non-relocatable
	          I/O), below 1 MB (for Memory), or below 64 KB (for
	          relocatable I/O).
	ss        is the space code, denoting the address space
	bbbbbbbb  is the 8-bit Bus Number
	ddddd     is the 5-bit Device Number
	fff       is the 3-bit Function Number
	rrrrrrrr  is the 8-bit Register Number

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-09 11:10                                 ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-09 11:10 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 08, 2013 at 05:12:04PM -0600, Rob Herring wrote:
> On 03/08/2013 01:14 AM, Thierry Reding wrote:
> > On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
> >> On 03/07/2013 02:47 PM, Thierry Reding wrote:
> > [...]
> >>> In a nutshell (since some of the context isn't quoted anymore) the
> >>> problem that we're trying to solve is that some of the embedded SoCs
> >>> require per-root-port registers for configuration. The PCI DT
> >>> specification doesn't make any provisions for this. A few alternatives
> >>> have been discussed so far:
> >>
> >> I'm not sure I follow. This is different than the host controller
> >> registers? Why would this not just be multiple entries in the reg property?
> > 
> > Well the register regions are per root-port. On Tegra20 there's 2 of
> > them, Tegra30 has 3 and if I understand correctly Marvell can have up to
> > 10 (!). Adding all of them to the reg property of the host controller
> > could work but it needs some way to match the reg entry to the root port
> > similar to option 1 below.
> 
> The compatible property of the PCI host controller can imply what each
> index of the reg property entries is for.
> 
> > 
> > Adding a property in the root port nodes seems like a cleaner and more
> > accurate representation of the hardware to me, but if that's not
> > acceptable perhaps we need to bite the bullet and add the code to look
> > the registers up from the parent's reg property.
> 
> What I don't like is a new property defined to describe mmio addresses.
> We already have a property for that and it is "reg".

Okay, understood.

> But I think I'm still missing something:
> 
> >>> 			pci at 0,1 {
> >>> 				...
> >>> 				reg = <0x00000800 0 0 0 0>;
> 
> Is this a PCI bus address?

Yes. The example is slightly wrong. pci at 0,1 should actually be pci at 1,0.
That means it is device 1, function 0 on the bus (implicitly 0 in this
case). That matches the values in the reg property, whose first cell is
defined as follows, quoting the PCI OF specification:

	npt000ss bbbbbbbb dddddfff rrrrrrrr

Where:

	n         is 0 if the address is relocatable, 1 otherwise
	p         is 1 if the addressable region is "prefetchable", 0
	          otherwise
	t         is 1 if the address is aliased (for non-relocatable
	          I/O), below 1 MB (for Memory), or below 64 KB (for
	          relocatable I/O).
	ss        is the space code, denoting the address space
	bbbbbbbb  is the 8-bit Bus Number
	ddddd     is the 5-bit Device Number
	fff       is the 3-bit Function Number
	rrrrrrrr  is the 8-bit Register Number

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130309/5685c82c/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-09  1:31                                           ` Jason Gunthorpe
  (?)
@ 2013-03-10  4:52                                               ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10  4:52 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/8/2013 3:31 PM, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 01:46:13PM -1000, Mitch Bradley wrote:
>> On 3/8/2013 10:02 AM, Jason Gunthorpe wrote:
>>> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
>>>
>>>>>> http://www.spinics.net/lists/arm-kernel/msg228749.html
>>>>
>>>> The example in that posting looks messed up to me.
>>>>
>>>> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
>>>> address in the address space defined by its parent - because the
form of
>>>> the parent's ranges property indicates that it's a PCI-style address
>>>> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
>>>> the type (I/O, memory, etc).
>>>
>>> You need to review the OF PCI bindings to make sense of this.  The
>>> subnodes are PCI devices. Those PCI devices show up in the
>>> configuration space (ie lspci). They are the PCI root port bridges.
>>
>> As it turns out, I wrote those bindings, almost 20 years ago.
>>
>> Having dug through old versions of that patch, I think I see the source
>> of the confusion.
>
> Not sure, I think this has gone into a weird tangent, your conclusions
> don't match how the PCI-E root complex is presented to the kernel.
>
> Lets just go back to the start?
>
> The pci-controller node is a root complex.
>
> The children of that node are elements in the root complex - they are
> the root port bridges. They are all on PCI bus 0.
>
> Each root port bridge responds to configuration cycles, and has a PCI
> configuration space. They show up in lspci.
>
> The host controller device driver knows how to issue configuration
> cycles, and it knows how to deliver configuration cycles on bus 0 to
> the bridges.
>
> This one:
> +			pcie@0,0 {
> +				device_type = "pci";
> +				reg = <0x0800 0 0 0 0>;
>
> Responds to bus 0, device 1, function 0.
>
> Each root port bridge has an associated non-PCI MMIO address
> space. The child above is associated with 0xd0040000. This MMIO
> address space is needed by the host controller driver to operate the
> port.
>
> The text after the @ is a mistake, Linux doesn't enforce anything
> about this text so nobody noticed till now, it should be corrected to
> @1,0, similarly for the second one.
>
> Also, from your comments the top level should gain a 'device_type =
> "pci"'.
>
> So, from that point, does the DT start to make sense? Are there other
> problems?
>
> The question at hand is how do you associate the non-PCI MMIO space
> 0xd0040000 with the root port bridge 'pcie@1,0' ? The patch proposes
> to use a reg array in the controller plus reg-names to do this.
>
> I have trouble making sense of your suggestions to switch away from
> 5dw addressing. It is critical that the DT child stanza be associated
> with the PCI discovered root port bridge. By my understanding Linux
> does this based on the configuration space format 'reg' value. How is
> that association possible if you switch things away from 5dw
> addressing?
>
> Also, this is for firmware-less embedded. Linux is required to scan
> the bus and do full address assignment of all PCI devies, including
> the root port bridges. The ranges property on the top node specifies
> the addressing apertures that are available for this process. This is
> common practice, there are many examples of this in kernel dts for PCI
> controllers.

Okay, I think I finally get it.  The Marvell root port bridge setup
registers look like standard config headers, even though they aren't
really in config space (because you need a different access method for
them, compared to real config accesses to downstream devices).

So, in order for the common code that enumerates the PCI bus to work,
you need to "spoof" the config accesses so that when you try to access
those particular "pseudo config headers", it uses the MMIO method
instead of the real config access method.

If that is indeed the case, them I would vote for a slight modification
of the intermediate patch that I cited earlier - the one in which the
ranges property includes translations from those special config
addresses into CPU addresses.  The modification is to fix the sizes,
changing 0x2000 to 0x800, so those ranges entries do not overlap in the
child address domain.

With that change, plus fixing the "@port,lane" strings to be "@dev,fun",
the DT would not raise all the red flags that led me down this rathole.

The controller driver would still have to implement the magic spoofing
of those funky config headers, but at least it would be doing it based
on well-established representation principles, specifically ranges
entries and well-formed child addresses, without having to resort to the
new sideband "port and lane" address representation.

Here's what it would look like:

pcie-controller {
	compatible = "marvell,armada-370-pcie";
	status = "disabled";

	#address-cells = <3>;
	#size-cells = <2>;

	bus-range = <0x00 0xff>;

	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
port config header */
	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
port config header */
		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
non-prefetchable memory */
	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
downstream I/O */

	pcie@1,0 {
		device_type = "pci";
		reg = <0x0800 0 0 0 0>;
		#address-cells = <3>;
		#size-cells = <2>;
		#interrupt-cells = <1>;
		interrupt-map-mask = <0 0 0 0>;
		interrupt-map = <0 0 0 0 &mpic 58>;
		marvell,pcie-port = <0>;
		marvell,pcie-lane = <0>;
		clocks = <&gateclk 5>;
		status = "disabled";
	};

	pcie@2,0 {
		device_type = "pci";
		reg = <0x1000 0 0 0 0>;
		#address-cells = <3>;
		#size-cells = <2>;
		#interrupt-cells = <1>;
		interrupt-map-mask = <0 0 0 0>;
		interrupt-map = <0 0 0 0 &mpic 62>;
		marvell,pcie-port = <1>;
		marvell,pcie-lane = <0>;
		clocks = <&gateclk 9>;
		status = "disabled";
	};
};


>
> It is impossible to specify any detail for the bridges (be it address
> ranges or bus number ranges) because we have no idea what discovery
> will find, or what values Linux will choose to assign.
>
>> In neither case would you need the controversial "reg-names" thing.
>
> reg-names has been a standard feature in the Linux kernel for a bit
> already..
>
> Thanks a lot for looking at this,
> Jason
>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10  4:52                                               ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10  4:52 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/8/2013 3:31 PM, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 01:46:13PM -1000, Mitch Bradley wrote:
>> On 3/8/2013 10:02 AM, Jason Gunthorpe wrote:
>>> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
>>>
>>>>>> http://www.spinics.net/lists/arm-kernel/msg228749.html
>>>>
>>>> The example in that posting looks messed up to me.
>>>>
>>>> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
>>>> address in the address space defined by its parent - because the
form of
>>>> the parent's ranges property indicates that it's a PCI-style address
>>>> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
>>>> the type (I/O, memory, etc).
>>>
>>> You need to review the OF PCI bindings to make sense of this.  The
>>> subnodes are PCI devices. Those PCI devices show up in the
>>> configuration space (ie lspci). They are the PCI root port bridges.
>>
>> As it turns out, I wrote those bindings, almost 20 years ago.
>>
>> Having dug through old versions of that patch, I think I see the source
>> of the confusion.
>
> Not sure, I think this has gone into a weird tangent, your conclusions
> don't match how the PCI-E root complex is presented to the kernel.
>
> Lets just go back to the start?
>
> The pci-controller node is a root complex.
>
> The children of that node are elements in the root complex - they are
> the root port bridges. They are all on PCI bus 0.
>
> Each root port bridge responds to configuration cycles, and has a PCI
> configuration space. They show up in lspci.
>
> The host controller device driver knows how to issue configuration
> cycles, and it knows how to deliver configuration cycles on bus 0 to
> the bridges.
>
> This one:
> +			pcie@0,0 {
> +				device_type = "pci";
> +				reg = <0x0800 0 0 0 0>;
>
> Responds to bus 0, device 1, function 0.
>
> Each root port bridge has an associated non-PCI MMIO address
> space. The child above is associated with 0xd0040000. This MMIO
> address space is needed by the host controller driver to operate the
> port.
>
> The text after the @ is a mistake, Linux doesn't enforce anything
> about this text so nobody noticed till now, it should be corrected to
> @1,0, similarly for the second one.
>
> Also, from your comments the top level should gain a 'device_type =
> "pci"'.
>
> So, from that point, does the DT start to make sense? Are there other
> problems?
>
> The question at hand is how do you associate the non-PCI MMIO space
> 0xd0040000 with the root port bridge 'pcie@1,0' ? The patch proposes
> to use a reg array in the controller plus reg-names to do this.
>
> I have trouble making sense of your suggestions to switch away from
> 5dw addressing. It is critical that the DT child stanza be associated
> with the PCI discovered root port bridge. By my understanding Linux
> does this based on the configuration space format 'reg' value. How is
> that association possible if you switch things away from 5dw
> addressing?
>
> Also, this is for firmware-less embedded. Linux is required to scan
> the bus and do full address assignment of all PCI devies, including
> the root port bridges. The ranges property on the top node specifies
> the addressing apertures that are available for this process. This is
> common practice, there are many examples of this in kernel dts for PCI
> controllers.

Okay, I think I finally get it.  The Marvell root port bridge setup
registers look like standard config headers, even though they aren't
really in config space (because you need a different access method for
them, compared to real config accesses to downstream devices).

So, in order for the common code that enumerates the PCI bus to work,
you need to "spoof" the config accesses so that when you try to access
those particular "pseudo config headers", it uses the MMIO method
instead of the real config access method.

If that is indeed the case, them I would vote for a slight modification
of the intermediate patch that I cited earlier - the one in which the
ranges property includes translations from those special config
addresses into CPU addresses.  The modification is to fix the sizes,
changing 0x2000 to 0x800, so those ranges entries do not overlap in the
child address domain.

With that change, plus fixing the "@port,lane" strings to be "@dev,fun",
the DT would not raise all the red flags that led me down this rathole.

The controller driver would still have to implement the magic spoofing
of those funky config headers, but at least it would be doing it based
on well-established representation principles, specifically ranges
entries and well-formed child addresses, without having to resort to the
new sideband "port and lane" address representation.

Here's what it would look like:

pcie-controller {
	compatible = "marvell,armada-370-pcie";
	status = "disabled";

	#address-cells = <3>;
	#size-cells = <2>;

	bus-range = <0x00 0xff>;

	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
port config header */
	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
port config header */
		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
non-prefetchable memory */
	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
downstream I/O */

	pcie@1,0 {
		device_type = "pci";
		reg = <0x0800 0 0 0 0>;
		#address-cells = <3>;
		#size-cells = <2>;
		#interrupt-cells = <1>;
		interrupt-map-mask = <0 0 0 0>;
		interrupt-map = <0 0 0 0 &mpic 58>;
		marvell,pcie-port = <0>;
		marvell,pcie-lane = <0>;
		clocks = <&gateclk 5>;
		status = "disabled";
	};

	pcie@2,0 {
		device_type = "pci";
		reg = <0x1000 0 0 0 0>;
		#address-cells = <3>;
		#size-cells = <2>;
		#interrupt-cells = <1>;
		interrupt-map-mask = <0 0 0 0>;
		interrupt-map = <0 0 0 0 &mpic 62>;
		marvell,pcie-port = <1>;
		marvell,pcie-lane = <0>;
		clocks = <&gateclk 9>;
		status = "disabled";
	};
};


>
> It is impossible to specify any detail for the bridges (be it address
> ranges or bus number ranges) because we have no idea what discovery
> will find, or what values Linux will choose to assign.
>
>> In neither case would you need the controversial "reg-names" thing.
>
> reg-names has been a standard feature in the Linux kernel for a bit
> already..
>
> Thanks a lot for looking at this,
> Jason
>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10  4:52                                               ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10  4:52 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/8/2013 3:31 PM, Jason Gunthorpe wrote:
> On Fri, Mar 08, 2013 at 01:46:13PM -1000, Mitch Bradley wrote:
>> On 3/8/2013 10:02 AM, Jason Gunthorpe wrote:
>>> On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
>>>
>>>>>> http://www.spinics.net/lists/arm-kernel/msg228749.html
>>>>
>>>> The example in that posting looks messed up to me.
>>>>
>>>> 1) It has "reg = <0x0800 0 0 0 0>", but 0x0800 0 0  is not a valid
>>>> address in the address space defined by its parent - because the
form of
>>>> the parent's ranges property indicates that it's a PCI-style address
>>>> form.  0x0800 0 0 lacks the top bits that indicate non-relocatable and
>>>> the type (I/O, memory, etc).
>>>
>>> You need to review the OF PCI bindings to make sense of this.  The
>>> subnodes are PCI devices. Those PCI devices show up in the
>>> configuration space (ie lspci). They are the PCI root port bridges.
>>
>> As it turns out, I wrote those bindings, almost 20 years ago.
>>
>> Having dug through old versions of that patch, I think I see the source
>> of the confusion.
>
> Not sure, I think this has gone into a weird tangent, your conclusions
> don't match how the PCI-E root complex is presented to the kernel.
>
> Lets just go back to the start?
>
> The pci-controller node is a root complex.
>
> The children of that node are elements in the root complex - they are
> the root port bridges. They are all on PCI bus 0.
>
> Each root port bridge responds to configuration cycles, and has a PCI
> configuration space. They show up in lspci.
>
> The host controller device driver knows how to issue configuration
> cycles, and it knows how to deliver configuration cycles on bus 0 to
> the bridges.
>
> This one:
> +			pcie at 0,0 {
> +				device_type = "pci";
> +				reg = <0x0800 0 0 0 0>;
>
> Responds to bus 0, device 1, function 0.
>
> Each root port bridge has an associated non-PCI MMIO address
> space. The child above is associated with 0xd0040000. This MMIO
> address space is needed by the host controller driver to operate the
> port.
>
> The text after the @ is a mistake, Linux doesn't enforce anything
> about this text so nobody noticed till now, it should be corrected to
> @1,0, similarly for the second one.
>
> Also, from your comments the top level should gain a 'device_type =
> "pci"'.
>
> So, from that point, does the DT start to make sense? Are there other
> problems?
>
> The question at hand is how do you associate the non-PCI MMIO space
> 0xd0040000 with the root port bridge 'pcie at 1,0' ? The patch proposes
> to use a reg array in the controller plus reg-names to do this.
>
> I have trouble making sense of your suggestions to switch away from
> 5dw addressing. It is critical that the DT child stanza be associated
> with the PCI discovered root port bridge. By my understanding Linux
> does this based on the configuration space format 'reg' value. How is
> that association possible if you switch things away from 5dw
> addressing?
>
> Also, this is for firmware-less embedded. Linux is required to scan
> the bus and do full address assignment of all PCI devies, including
> the root port bridges. The ranges property on the top node specifies
> the addressing apertures that are available for this process. This is
> common practice, there are many examples of this in kernel dts for PCI
> controllers.

Okay, I think I finally get it.  The Marvell root port bridge setup
registers look like standard config headers, even though they aren't
really in config space (because you need a different access method for
them, compared to real config accesses to downstream devices).

So, in order for the common code that enumerates the PCI bus to work,
you need to "spoof" the config accesses so that when you try to access
those particular "pseudo config headers", it uses the MMIO method
instead of the real config access method.

If that is indeed the case, them I would vote for a slight modification
of the intermediate patch that I cited earlier - the one in which the
ranges property includes translations from those special config
addresses into CPU addresses.  The modification is to fix the sizes,
changing 0x2000 to 0x800, so those ranges entries do not overlap in the
child address domain.

With that change, plus fixing the "@port,lane" strings to be "@dev,fun",
the DT would not raise all the red flags that led me down this rathole.

The controller driver would still have to implement the magic spoofing
of those funky config headers, but at least it would be doing it based
on well-established representation principles, specifically ranges
entries and well-formed child addresses, without having to resort to the
new sideband "port and lane" address representation.

Here's what it would look like:

pcie-controller {
	compatible = "marvell,armada-370-pcie";
	status = "disabled";

	#address-cells = <3>;
	#size-cells = <2>;

	bus-range = <0x00 0xff>;

	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
port config header */
	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
port config header */
		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
non-prefetchable memory */
	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
downstream I/O */

	pcie at 1,0 {
		device_type = "pci";
		reg = <0x0800 0 0 0 0>;
		#address-cells = <3>;
		#size-cells = <2>;
		#interrupt-cells = <1>;
		interrupt-map-mask = <0 0 0 0>;
		interrupt-map = <0 0 0 0 &mpic 58>;
		marvell,pcie-port = <0>;
		marvell,pcie-lane = <0>;
		clocks = <&gateclk 5>;
		status = "disabled";
	};

	pcie at 2,0 {
		device_type = "pci";
		reg = <0x1000 0 0 0 0>;
		#address-cells = <3>;
		#size-cells = <2>;
		#interrupt-cells = <1>;
		interrupt-map-mask = <0 0 0 0>;
		interrupt-map = <0 0 0 0 &mpic 62>;
		marvell,pcie-port = <1>;
		marvell,pcie-lane = <0>;
		clocks = <&gateclk 9>;
		status = "disabled";
	};
};


>
> It is impossible to specify any detail for the bridges (be it address
> ranges or bus number ranges) because we have no idea what discovery
> will find, or what values Linux will choose to assign.
>
>> In neither case would you need the controversial "reg-names" thing.
>
> reg-names has been a standard feature in the Linux kernel for a bit
> already..
>
> Thanks a lot for looking at this,
> Jason
>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 23:12                             ` Rob Herring
  (?)
@ 2013-03-10  5:04                                 ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10  5:04 UTC (permalink / raw)
  To: Rob Herring
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Jason Gunthorpe, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	Tawfik Bayouk, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/8/2013 1:12 PM, Rob Herring wrote:
> On 03/08/2013 01:14 AM, Thierry Reding wrote:
>> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
>>> On 03/07/2013 02:47 PM, Thierry Reding wrote:
>> [...]
>>>> In a nutshell (since some of the context isn't quoted anymore) the
>>>> problem that we're trying to solve is that some of the embedded SoCs
>>>> require per-root-port registers for configuration. The PCI DT
>>>> specification doesn't make any provisions for this. A few alternatives
>>>> have been discussed so far:
>>>
>>> I'm not sure I follow. This is different than the host controller
>>> registers? Why would this not just be multiple entries in the reg property?
>>
>> Well the register regions are per root-port. On Tegra20 there's 2 of
>> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
>> 10 (!). Adding all of them to the reg property of the host controller
>> could work but it needs some way to match the reg entry to the root port
>> similar to option 1 below.
> 
> The compatible property of the PCI host controller can imply what each
> index of the reg property entries is for.
> 
>>
>> Adding a property in the root port nodes seems like a cleaner and more
>> accurate representation of the hardware to me, but if that's not
>> acceptable perhaps we need to bite the bullet and add the code to look
>> the registers up from the parent's reg property.
> 
> What I don't like is a new property defined to describe mmio addresses.
> We already have a property for that and it is "reg". But I think I'm
> still missing something:


As stated in my recent reply to Jason, I thing the correct property is
"ranges".  "Ranges" translates mappable child address space addresses
into parent addresses, and that is exactly what is going on.  A specific
subset of config addresses is mappable into parent MMIO space.

When done that way, there is no need for the multi-entry reg property at
the top level, and the correspondence between specific root port DT
nodes and their MMIO addresses is easily determined by matching child
reg properties to parent ranges entries.

> 
>>>> 			pci@0,1 {
>>>> 				...
>>>> 				reg = <0x00000800 0 0 0 0>;
> 
> Is this a PCI bus address?
> 
>>>> 				regs = <0x80000000 0x00001000>;
>>>> 			};
> 
> Rob
> 
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10  5:04                                 ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10  5:04 UTC (permalink / raw)
  To: Rob Herring
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Jason Gunthorpe, Maen Suleiman, Nadav Haklai,
	Shadi Ammouri, Bjorn Helgaas, Tawfik Bayouk, linux-arm-kernel

On 3/8/2013 1:12 PM, Rob Herring wrote:
> On 03/08/2013 01:14 AM, Thierry Reding wrote:
>> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
>>> On 03/07/2013 02:47 PM, Thierry Reding wrote:
>> [...]
>>>> In a nutshell (since some of the context isn't quoted anymore) the
>>>> problem that we're trying to solve is that some of the embedded SoCs
>>>> require per-root-port registers for configuration. The PCI DT
>>>> specification doesn't make any provisions for this. A few alternatives
>>>> have been discussed so far:
>>>
>>> I'm not sure I follow. This is different than the host controller
>>> registers? Why would this not just be multiple entries in the reg property?
>>
>> Well the register regions are per root-port. On Tegra20 there's 2 of
>> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
>> 10 (!). Adding all of them to the reg property of the host controller
>> could work but it needs some way to match the reg entry to the root port
>> similar to option 1 below.
> 
> The compatible property of the PCI host controller can imply what each
> index of the reg property entries is for.
> 
>>
>> Adding a property in the root port nodes seems like a cleaner and more
>> accurate representation of the hardware to me, but if that's not
>> acceptable perhaps we need to bite the bullet and add the code to look
>> the registers up from the parent's reg property.
> 
> What I don't like is a new property defined to describe mmio addresses.
> We already have a property for that and it is "reg". But I think I'm
> still missing something:


As stated in my recent reply to Jason, I thing the correct property is
"ranges".  "Ranges" translates mappable child address space addresses
into parent addresses, and that is exactly what is going on.  A specific
subset of config addresses is mappable into parent MMIO space.

When done that way, there is no need for the multi-entry reg property at
the top level, and the correspondence between specific root port DT
nodes and their MMIO addresses is easily determined by matching child
reg properties to parent ranges entries.

> 
>>>> 			pci@0,1 {
>>>> 				...
>>>> 				reg = <0x00000800 0 0 0 0>;
> 
> Is this a PCI bus address?
> 
>>>> 				regs = <0x80000000 0x00001000>;
>>>> 			};
> 
> Rob
> 
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10  5:04                                 ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10  5:04 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/8/2013 1:12 PM, Rob Herring wrote:
> On 03/08/2013 01:14 AM, Thierry Reding wrote:
>> On Thu, Mar 07, 2013 at 06:05:33PM -0600, Rob Herring wrote:
>>> On 03/07/2013 02:47 PM, Thierry Reding wrote:
>> [...]
>>>> In a nutshell (since some of the context isn't quoted anymore) the
>>>> problem that we're trying to solve is that some of the embedded SoCs
>>>> require per-root-port registers for configuration. The PCI DT
>>>> specification doesn't make any provisions for this. A few alternatives
>>>> have been discussed so far:
>>>
>>> I'm not sure I follow. This is different than the host controller
>>> registers? Why would this not just be multiple entries in the reg property?
>>
>> Well the register regions are per root-port. On Tegra20 there's 2 of
>> them, Tegra30 has 3 and if I understand correctly Marvell can have up to
>> 10 (!). Adding all of them to the reg property of the host controller
>> could work but it needs some way to match the reg entry to the root port
>> similar to option 1 below.
> 
> The compatible property of the PCI host controller can imply what each
> index of the reg property entries is for.
> 
>>
>> Adding a property in the root port nodes seems like a cleaner and more
>> accurate representation of the hardware to me, but if that's not
>> acceptable perhaps we need to bite the bullet and add the code to look
>> the registers up from the parent's reg property.
> 
> What I don't like is a new property defined to describe mmio addresses.
> We already have a property for that and it is "reg". But I think I'm
> still missing something:


As stated in my recent reply to Jason, I thing the correct property is
"ranges".  "Ranges" translates mappable child address space addresses
into parent addresses, and that is exactly what is going on.  A specific
subset of config addresses is mappable into parent MMIO space.

When done that way, there is no need for the multi-entry reg property at
the top level, and the correspondence between specific root port DT
nodes and their MMIO addresses is easily determined by matching child
reg properties to parent ranges entries.

> 
>>>> 			pci at 0,1 {
>>>> 				...
>>>> 				reg = <0x00000800 0 0 0 0>;
> 
> Is this a PCI bus address?
> 
>>>> 				regs = <0x80000000 0x00001000>;
>>>> 			};
> 
> Rob
> 
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss at lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-10  4:52                                               ` Mitch Bradley
@ 2013-03-10  6:55                                                 ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-10  6:55 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:

> Okay, I think I finally get it.  The Marvell root port bridge setup
> registers look like standard config headers, even though they aren't
> really in config space (because you need a different access method for
> them, compared to real config accesses to downstream devices).

Well, thats pretty close. On Marvell the MMIO space has a lot of
stuff, some of it is config related, lots of it isn't. In fact most of
it isn't. Tegra is different, the MMIO region is exactly the config
space of the root port bridge.

Tegra could probably happily and sanely do as you've described (well,
see below), but it is wonky for Marvell. It looks like there are more
drivers coming that have this need - so I had hoped for a general
answer to the 'per-port MMIO space' problem that is unrelated to the
MMIO spaces role in config access.

> So, in order for the common code that enumerates the PCI bus to work,
> you need to "spoof" the config accesses so that when you try to access
> those particular "pseudo config headers", it uses the MMIO method
> instead of the real config access method.

There are rules for config access in both Tegra and Marvell that are
not trivial. Again, the driver takes are of this and all Linux sees is
a nice compliant config space.

> If that is indeed the case, them I would vote for a slight modification
> of the intermediate patch that I cited earlier - the one in which the
> ranges property includes translations from those special config
> addresses into CPU addresses.  The modification is to fix the sizes,
> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
> child address domain.

On Marvell the size of the MMIO space is 0x2000, there are imporant
registers there beyond index 0x800 (todays driver may not touch them
but HW has them). Reducing the size doesn't seem appropriate.

> pcie-controller {
> 	compatible = "marvell,armada-370-pcie";
> 	status = "disabled";

        device_type = "pci";                                                                    

Here as well, as you noted before?

> 	#address-cells = <3>;
> 	#size-cells = <2>;
> 
> 	bus-range = <0x00 0xff>;
> 
> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
> port config header */
> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
> port config header */
> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
> non-prefetchable memory */
> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
> downstream I/O */
> 
> 	pcie@1,0 {
> 		device_type = "pci";
> 		reg = <0x0800 0 0 0 0>;

Okay, this was Thierry's original idea, and it was my note that this
didn't seem like a good choice. I'm not wedded to that, but I'll
explain my reasoning a bit.

Two things bothered me
  - Describing a CPU MMIO mapping with a config address space seems
    wonky
  - Linux's OF core doesn't parse a 'reg' property under
    'device_type = "pci"' as something CPU mappable, it only looks
    to assigned-addresses for that. So the OF core would need to learn
    how to handle this case, and it would have to be well defined..

    Thierry's original patch avoided this problem by not using
    device_type = "pci"..

Also, did you mean to have a 0 size for the 'reg'? How will things
know how big the MMIO region is? Can't just assume 0x800..

Again, thanks!
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10  6:55                                                 ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-10  6:55 UTC (permalink / raw)
  To: linux-arm-kernel

On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:

> Okay, I think I finally get it.  The Marvell root port bridge setup
> registers look like standard config headers, even though they aren't
> really in config space (because you need a different access method for
> them, compared to real config accesses to downstream devices).

Well, thats pretty close. On Marvell the MMIO space has a lot of
stuff, some of it is config related, lots of it isn't. In fact most of
it isn't. Tegra is different, the MMIO region is exactly the config
space of the root port bridge.

Tegra could probably happily and sanely do as you've described (well,
see below), but it is wonky for Marvell. It looks like there are more
drivers coming that have this need - so I had hoped for a general
answer to the 'per-port MMIO space' problem that is unrelated to the
MMIO spaces role in config access.

> So, in order for the common code that enumerates the PCI bus to work,
> you need to "spoof" the config accesses so that when you try to access
> those particular "pseudo config headers", it uses the MMIO method
> instead of the real config access method.

There are rules for config access in both Tegra and Marvell that are
not trivial. Again, the driver takes are of this and all Linux sees is
a nice compliant config space.

> If that is indeed the case, them I would vote for a slight modification
> of the intermediate patch that I cited earlier - the one in which the
> ranges property includes translations from those special config
> addresses into CPU addresses.  The modification is to fix the sizes,
> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
> child address domain.

On Marvell the size of the MMIO space is 0x2000, there are imporant
registers there beyond index 0x800 (todays driver may not touch them
but HW has them). Reducing the size doesn't seem appropriate.

> pcie-controller {
> 	compatible = "marvell,armada-370-pcie";
> 	status = "disabled";

        device_type = "pci";                                                                    

Here as well, as you noted before?

> 	#address-cells = <3>;
> 	#size-cells = <2>;
> 
> 	bus-range = <0x00 0xff>;
> 
> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
> port config header */
> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
> port config header */
> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
> non-prefetchable memory */
> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
> downstream I/O */
> 
> 	pcie at 1,0 {
> 		device_type = "pci";
> 		reg = <0x0800 0 0 0 0>;

Okay, this was Thierry's original idea, and it was my note that this
didn't seem like a good choice. I'm not wedded to that, but I'll
explain my reasoning a bit.

Two things bothered me
  - Describing a CPU MMIO mapping with a config address space seems
    wonky
  - Linux's OF core doesn't parse a 'reg' property under
    'device_type = "pci"' as something CPU mappable, it only looks
    to assigned-addresses for that. So the OF core would need to learn
    how to handle this case, and it would have to be well defined..

    Thierry's original patch avoided this problem by not using
    device_type = "pci"..

Also, did you mean to have a 0 size for the 'reg'? How will things
know how big the MMIO region is? Can't just assume 0x800..

Again, thanks!
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-10  5:04                                 ` Mitch Bradley
@ 2013-03-10 15:06                                   ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-10 15:06 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Rob Herring, Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Jason Gunthorpe, Maen Suleiman, Shadi Ammouri,
	Bjorn Helgaas, Tawfik Bayouk, linux-arm-kernel

Dear Mitch Bradley,

On Sat, 09 Mar 2013 19:04:51 -1000, Mitch Bradley wrote:
 
> As stated in my recent reply to Jason, I thing the correct property is
> "ranges".  "Ranges" translates mappable child address space addresses
> into parent addresses, and that is exactly what is going on.  A specific
> subset of config addresses is mappable into parent MMIO space.

The PCI configuration space is *not* mapped in the MMIO space on
Marvell hardware. In the MMIO space of each PCIe interface, there are
many registers, only *two* of which are dedicated to accessing the PCI
configuration space:

 * One register to set the offset in the PCI configuration space.

 * One register to read or write a value in the PCI configuration, at
   the offset specified in the first register.

See the implementation of mvebu_pcie_hw_rd_conf() and
mvebu_pcie_hw_wr_conf() in the driver.

So really, the values specified in the reg = <...> property are *not*
the PCI configuration spaces mapped in the MMIO space. They represent a
bunch of per PCIe interface registers used to configure them, get the
status of the link... and access, through an indirect mechanism, the
PCI configuration space.

Does this helps?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10 15:06                                   ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-10 15:06 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Mitch Bradley,

On Sat, 09 Mar 2013 19:04:51 -1000, Mitch Bradley wrote:
 
> As stated in my recent reply to Jason, I thing the correct property is
> "ranges".  "Ranges" translates mappable child address space addresses
> into parent addresses, and that is exactly what is going on.  A specific
> subset of config addresses is mappable into parent MMIO space.

The PCI configuration space is *not* mapped in the MMIO space on
Marvell hardware. In the MMIO space of each PCIe interface, there are
many registers, only *two* of which are dedicated to accessing the PCI
configuration space:

 * One register to set the offset in the PCI configuration space.

 * One register to read or write a value in the PCI configuration, at
   the offset specified in the first register.

See the implementation of mvebu_pcie_hw_rd_conf() and
mvebu_pcie_hw_wr_conf() in the driver.

So really, the values specified in the reg = <...> property are *not*
the PCI configuration spaces mapped in the MMIO space. They represent a
bunch of per PCIe interface registers used to configure them, get the
status of the link... and access, through an indirect mechanism, the
PCI configuration space.

Does this helps?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-08 20:13                                         ` Thierry Reding
@ 2013-03-10 15:09                                           ` Thomas Petazzoni
  -1 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-10 15:09 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Tawfik Bayouk, linux-pci,
	devicetree-discuss, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, linux-arm-kernel

Dear Thierry Reding,

On Fri, 8 Mar 2013 21:13:40 +0100, Thierry Reding wrote:
> On Fri, Mar 08, 2013 at 01:02:46PM -0700, Jason Gunthorpe wrote:
> > On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> [...]
> > > 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> > > <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> > 
> > @0,1,0 (bus,device,fn) could be more appropriate, but that is
> > cosmetic?
> 
> The OF PCI specification is pretty strict about this as well. It says in
> section 2.2.1.3. that only the DD and DD,FF forms can appear in a device
> path.

Note that in the case of my driver, the @X,Y represent the port and
lane of the PCIe interface. Not sure if it is correct, I can change the
names to whatever is appropriate, those names aren't used anywhere in
the driver.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10 15:09                                           ` Thomas Petazzoni
  0 siblings, 0 replies; 291+ messages in thread
From: Thomas Petazzoni @ 2013-03-10 15:09 UTC (permalink / raw)
  To: linux-arm-kernel

Dear Thierry Reding,

On Fri, 8 Mar 2013 21:13:40 +0100, Thierry Reding wrote:
> On Fri, Mar 08, 2013 at 01:02:46PM -0700, Jason Gunthorpe wrote:
> > On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> [...]
> > > 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> > > <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> > 
> > @0,1,0 (bus,device,fn) could be more appropriate, but that is
> > cosmetic?
> 
> The OF PCI specification is pretty strict about this as well. It says in
> section 2.2.1.3. that only the DD and DD,FF forms can appear in a device
> path.

Note that in the case of my driver, the @X,Y represent the port and
lane of the PCIe interface. Not sure if it is correct, I can change the
names to whatever is appropriate, those names aren't used anywhere in
the driver.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-10 15:06                                   ` Thomas Petazzoni
  (?)
@ 2013-03-10 18:33                                     ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10 18:33 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Bjorn Helgaas, Shadi Ammouri,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
	Jason Gunthorpe

On 3/10/2013 5:06 AM, Thomas Petazzoni wrote:
> Dear Mitch Bradley,
> 
> On Sat, 09 Mar 2013 19:04:51 -1000, Mitch Bradley wrote:
>  
>> As stated in my recent reply to Jason, I thing the correct property is
>> "ranges".  "Ranges" translates mappable child address space addresses
>> into parent addresses, and that is exactly what is going on.  A specific
>> subset of config addresses is mappable into parent MMIO space.
> 
> The PCI configuration space is *not* mapped in the MMIO space on
> Marvell hardware. In the MMIO space of each PCIe interface, there are
> many registers, only *two* of which are dedicated to accessing the PCI
> configuration space:
> 
>  * One register to set the offset in the PCI configuration space.
> 
>  * One register to read or write a value in the PCI configuration, at
>    the offset specified in the first register.
> 
> See the implementation of mvebu_pcie_hw_rd_conf() and
> mvebu_pcie_hw_wr_conf() in the driver.
> 
> So really, the values specified in the reg = <...> property are *not*
> the PCI configuration spaces mapped in the MMIO space. They represent a
> bunch of per PCIe interface registers used to configure them, get the
> status of the link... and access, through an indirect mechanism, the
> PCI configuration space.
> 
> Does this helps?

I agree that PCI config space accesses to *downstream* devices is via an
indirect-access register pair.

The question is, does that indirect-access mechanism apply also to the
internal config headers for the root port bridges?

According to section 20.15 of the MV78230 functional spec that I am
looking at, the configuration header registers are mapped to the
internal memory space.  That section is unclear about whether those
registers are CPU-accessible via indirect-access config transactions.
When the PCIe hardware is configured for endpoint mode, the internal
headers can be accessed via PCIe config transactions from an external
port, but in root complex mode, the possibility of indirect access from
the CPU is not mentioned.

The manual is a little vague in some respects, but it does say quite
clearly that MMIO access to the root port bridge config header is
possible, and it lists the MMIO addresses thereof (section A.10).

So we all agree that access to external (downstream) config registers is
via the indirect register pair.  The unclear thing is whether the
internal config registers for the root port bridges can be indirectly
accessed.

Do we have empirical evidence that indirect-access works to the internal
root port bridge config headers?

> 
> Thanks,
> 
> Thomas
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10 18:33                                     ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10 18:33 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Rob Herring, Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Jason Gunthorpe, Maen Suleiman, Shadi Ammouri,
	Bjorn Helgaas, Tawfik Bayouk, linux-arm-kernel

On 3/10/2013 5:06 AM, Thomas Petazzoni wrote:
> Dear Mitch Bradley,
> 
> On Sat, 09 Mar 2013 19:04:51 -1000, Mitch Bradley wrote:
>  
>> As stated in my recent reply to Jason, I thing the correct property is
>> "ranges".  "Ranges" translates mappable child address space addresses
>> into parent addresses, and that is exactly what is going on.  A specific
>> subset of config addresses is mappable into parent MMIO space.
> 
> The PCI configuration space is *not* mapped in the MMIO space on
> Marvell hardware. In the MMIO space of each PCIe interface, there are
> many registers, only *two* of which are dedicated to accessing the PCI
> configuration space:
> 
>  * One register to set the offset in the PCI configuration space.
> 
>  * One register to read or write a value in the PCI configuration, at
>    the offset specified in the first register.
> 
> See the implementation of mvebu_pcie_hw_rd_conf() and
> mvebu_pcie_hw_wr_conf() in the driver.
> 
> So really, the values specified in the reg = <...> property are *not*
> the PCI configuration spaces mapped in the MMIO space. They represent a
> bunch of per PCIe interface registers used to configure them, get the
> status of the link... and access, through an indirect mechanism, the
> PCI configuration space.
> 
> Does this helps?

I agree that PCI config space accesses to *downstream* devices is via an
indirect-access register pair.

The question is, does that indirect-access mechanism apply also to the
internal config headers for the root port bridges?

According to section 20.15 of the MV78230 functional spec that I am
looking at, the configuration header registers are mapped to the
internal memory space.  That section is unclear about whether those
registers are CPU-accessible via indirect-access config transactions.
When the PCIe hardware is configured for endpoint mode, the internal
headers can be accessed via PCIe config transactions from an external
port, but in root complex mode, the possibility of indirect access from
the CPU is not mentioned.

The manual is a little vague in some respects, but it does say quite
clearly that MMIO access to the root port bridge config header is
possible, and it lists the MMIO addresses thereof (section A.10).

So we all agree that access to external (downstream) config registers is
via the indirect register pair.  The unclear thing is whether the
internal config registers for the root port bridges can be indirectly
accessed.

Do we have empirical evidence that indirect-access works to the internal
root port bridge config headers?

> 
> Thanks,
> 
> Thomas
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-10 18:33                                     ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-10 18:33 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/10/2013 5:06 AM, Thomas Petazzoni wrote:
> Dear Mitch Bradley,
> 
> On Sat, 09 Mar 2013 19:04:51 -1000, Mitch Bradley wrote:
>  
>> As stated in my recent reply to Jason, I thing the correct property is
>> "ranges".  "Ranges" translates mappable child address space addresses
>> into parent addresses, and that is exactly what is going on.  A specific
>> subset of config addresses is mappable into parent MMIO space.
> 
> The PCI configuration space is *not* mapped in the MMIO space on
> Marvell hardware. In the MMIO space of each PCIe interface, there are
> many registers, only *two* of which are dedicated to accessing the PCI
> configuration space:
> 
>  * One register to set the offset in the PCI configuration space.
> 
>  * One register to read or write a value in the PCI configuration, at
>    the offset specified in the first register.
> 
> See the implementation of mvebu_pcie_hw_rd_conf() and
> mvebu_pcie_hw_wr_conf() in the driver.
> 
> So really, the values specified in the reg = <...> property are *not*
> the PCI configuration spaces mapped in the MMIO space. They represent a
> bunch of per PCIe interface registers used to configure them, get the
> status of the link... and access, through an indirect mechanism, the
> PCI configuration space.
> 
> Does this helps?

I agree that PCI config space accesses to *downstream* devices is via an
indirect-access register pair.

The question is, does that indirect-access mechanism apply also to the
internal config headers for the root port bridges?

According to section 20.15 of the MV78230 functional spec that I am
looking at, the configuration header registers are mapped to the
internal memory space.  That section is unclear about whether those
registers are CPU-accessible via indirect-access config transactions.
When the PCIe hardware is configured for endpoint mode, the internal
headers can be accessed via PCIe config transactions from an external
port, but in root complex mode, the possibility of indirect access from
the CPU is not mentioned.

The manual is a little vague in some respects, but it does say quite
clearly that MMIO access to the root port bridge config header is
possible, and it lists the MMIO addresses thereof (section A.10).

So we all agree that access to external (downstream) config registers is
via the indirect register pair.  The unclear thing is whether the
internal config registers for the root port bridges can be indirectly
accessed.

Do we have empirical evidence that indirect-access works to the internal
root port bridge config headers?

> 
> Thanks,
> 
> Thomas
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-10  6:55                                                 ` Jason Gunthorpe
  (?)
@ 2013-03-11  5:46                                                     ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11  5:46 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
> On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
> 
>> Okay, I think I finally get it.  The Marvell root port bridge setup
>> registers look like standard config headers, even though they aren't
>> really in config space (because you need a different access method for
>> them, compared to real config accesses to downstream devices).
> 
> Well, thats pretty close. On Marvell the MMIO space has a lot of
> stuff, some of it is config related, lots of it isn't. In fact most of
> it isn't. Tegra is different, the MMIO region is exactly the config
> space of the root port bridge.
> 
> Tegra could probably happily and sanely do as you've described (well,
> see below), but it is wonky for Marvell. It looks like there are more
> drivers coming that have this need - so I had hoped for a general
> answer to the 'per-port MMIO space' problem that is unrelated to the
> MMIO spaces role in config access.
> 
>> So, in order for the common code that enumerates the PCI bus to work,
>> you need to "spoof" the config accesses so that when you try to access
>> those particular "pseudo config headers", it uses the MMIO method
>> instead of the real config access method.
> 
> There are rules for config access in both Tegra and Marvell that are
> not trivial. Again, the driver takes are of this and all Linux sees is
> a nice compliant config space.

Thanks for explaining that.

So it seems that we are faced with two requirements that are somewhat at
odds with one another.

1) Some of the root port bridge registers have to be accessed via config
space access functions so common PCI enumeration code will work.  To
represent this with the usual DT structure, the top root-complex needs
to define a 3/2 address space so its children
can have standard PCI reg properties.  Presumably, if those registers
are being programmed by common code, Marvell-specific code would
restrict its role to setting up config-space access functions, leaving
the actual touching of the registers to the common code.

2) Marvell chips have additional non-standard per-root-port registers
that generic PCI code would not understand.  These registers would be
touched only by Marvell-specific code.

The two kinds of registers are adjacent in MMIO space.  However, unless
I am misunderstanding this MV78230 manual, the highest "config header"
register index is 0x134, well below the 0x1000 size limit of a PCIe
config header.  Some of the extra registers are at 0x8xx, and others are
above 0x1800.

That suggests the following:

For the "config header" registers that use generic PCI code, use a
ranges entry to associate (pass through) MMIO addresses to the config
header portion of the register block.  This parameterizes the code that
sets up the special-case PCI config access.  That specification
technique is general and could be used not only for the Marvell case,
but also for any other chip that has some number of direct-mapped config
headers.

For requirement (2), the top node has a reg property listing the
portions of the address space that are consumed by the driver at the top
level instead of being passed through to the PCI addressing domain.  E.g.

  reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;

Thus we have accurately described the two aspects of the true situation.
 The PCI-compliant "config header" registers are passed through to the
child nodes where they can be dealt with in the normal PCI fashion.  The
non-PCI-compliant register footprint lives within the Marvell-specific
root-complex driver.

The root-complex driver presumably needs associate non-compliant
register blocks with specific child nodes.  That can be done by
requiring that the reg entries are in the same order as the config-space
ranges entries.

Anticipating the possible objection that ARM's 0x1000-byte page size
does not permit virtual-to-physical mapping at 0x800-byte granularity:
The device tree does not guarantee that reg entries are page-aligned; it
simply tries to describe the reality, even though it might be messy.

> 
>> If that is indeed the case, them I would vote for a slight modification
>> of the intermediate patch that I cited earlier - the one in which the
>> ranges property includes translations from those special config
>> addresses into CPU addresses.  The modification is to fix the sizes,
>> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
>> child address domain.

BTW I have completely changed my mind about the overlap thing.

I said that it was bogus to use size=0x2000 for a config space header.
 That was based on an interpretation - which I now dislike - of the
meaning of 3-cell config addresses.  By that old interpretation,
size=0x800 would also be bogus, because bits 10-8 of the config address
are for the function number.

Consider the following question, which I have never previously
considered, at least not explicitly:

Q: What would be the 3-cell representation of the Command/Status
register address (offset 4) in device 1, function 1?

One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
because the value 0x80000904 is what you put in the cf8 indirect access
address register.  But it is weak because it unnecessarily restricts the
config header size, and because it works differently than offsets
applied to memory and I/O space addresses.  Furthermore, coupling the
offset to the cf8 register is dodgy because of the funniness where you
have to but the byte-selector bits 1..0 not in the address register cf8,
but rather in the data register cfc.

The better answer is  <0x00000900 0 4>, using the third cell for the
offset, as with IO and memory space offsets.

Under this better interpretation, config space sizes larger than 0x100
are no problem.  The first cell is just a selector.  The size is "added"
to the third cell, so it does not encroach into the first cell's
device,function bits.

This better interpretation easily handles PCIe's 0x1000-byte extended
config headers, and trivially accomodates even larger sizes should a
future PCI variant further increase the header size. as described way
down below.

> 
> On Marvell the size of the MMIO space is 0x2000, there are imporant
> registers there beyond index 0x800 (todays driver may not touch them
> but HW has them). Reducing the size doesn't seem appropriate.

Per the above, the current idea is not to reduce the size, but rather to
split the 0x2000 into two pieces, accurately reflecting the fact that
part of it is PCI-compliant and can be handled by generic PCI drivers,
while the rest must be handled differently, outside of standard PCI
land.  Commingling the two seems dubious.

> 
>> pcie-controller {
>> 	compatible = "marvell,armada-370-pcie";
>> 	status = "disabled";
> 
>         device_type = "pci";                                                                    
> 
> Here as well, as you noted before?
> 
>> 	#address-cells = <3>;
>> 	#size-cells = <2>;
>>
>> 	bus-range = <0x00 0xff>;
>>
>> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
>> port config header */
>> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
>> port config header */
>> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
>> non-prefetchable memory */
>> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
>> downstream I/O */
>>
>> 	pcie@1,0 {
>> 		device_type = "pci";
>> 		reg = <0x0800 0 0 0 0>;
> 
> Okay, this was Thierry's original idea, and it was my note that this
> didn't seem like a good choice. I'm not wedded to that, but I'll
> explain my reasoning a bit.
> 
> Two things bothered me
>   - Describing a CPU MMIO mapping with a config address space seems
>     wonky

I agree that mapping config space is sort of a jarring concept, but I
think that's because PCs have polluted the mindspace, not because there
is anything inherently bad about it.  The original PCI spec
fundamentally treated config space as just another (rather segmented)
linear address space.  It specified the cf8/cfc indirect access
mechanism not as a deep semantic feature, but rather as an
implementation hack to work around addressing limitations of PC chipsets
and the addressing-deficient popular OS of the time (Windows 3.1 which
was a veneer over DOS).  The PCI spec defined configuration mechanism #2
as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
Alpha, Power PC) often direct-mapped config space into some form of
load/store space.

But PCs soon encroached on the RISC market and RISC system vendors
started to "leverage" PC I/O chipsets.  The indirect access "config
mechanism #1" became part of the mindset.  Indirect access was hardcoded
in enough software that chipset designers would provide indirect access
even if direct-mapping was possible.

Fundamentally, a config header is just a block of registers, addressable
as an index relative to some base.  It might require some specific
access path - but that is true with any block of registers.  Even
direct-mapped MMIO registers often require specific semantics such as
non-cached, non-write-buffered, or special address space ids.

The PCI SIG expected PCIe extended config space to be direct-mapped -
see slide 25 of

http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf

So mappability of config space was intended from the get-go.  The real
wonky thing is the "indirect access zombie that cannot die".

>   - Linux's OF core doesn't parse a 'reg' property under
>     'device_type = "pci"' as something CPU mappable, it only looks
>     to assigned-addresses for that. So the OF core would need to learn
>     how to handle this case, and it would have to be well defined..

The way I envisioned it working is that the root-complex node defines
its own config space access functions (the "ops" argument to
pci_scan_root_bus).  For config addresses listed in ranges, the
functions do MMIO.  For others, they use the chipset's indirect-access
registers.  The OF core is not involved.

> 
>     Thierry's original patch avoided this problem by not using
>     device_type = "pci"..
> 
> Also, did you mean to have a 0 size for the 'reg'? How will things
> know how big the MMIO region is? Can't just assume 0x800..

Sorry, that was a typo.

> 
> Again, thanks!
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11  5:46                                                     ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11  5:46 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
> On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
> 
>> Okay, I think I finally get it.  The Marvell root port bridge setup
>> registers look like standard config headers, even though they aren't
>> really in config space (because you need a different access method for
>> them, compared to real config accesses to downstream devices).
> 
> Well, thats pretty close. On Marvell the MMIO space has a lot of
> stuff, some of it is config related, lots of it isn't. In fact most of
> it isn't. Tegra is different, the MMIO region is exactly the config
> space of the root port bridge.
> 
> Tegra could probably happily and sanely do as you've described (well,
> see below), but it is wonky for Marvell. It looks like there are more
> drivers coming that have this need - so I had hoped for a general
> answer to the 'per-port MMIO space' problem that is unrelated to the
> MMIO spaces role in config access.
> 
>> So, in order for the common code that enumerates the PCI bus to work,
>> you need to "spoof" the config accesses so that when you try to access
>> those particular "pseudo config headers", it uses the MMIO method
>> instead of the real config access method.
> 
> There are rules for config access in both Tegra and Marvell that are
> not trivial. Again, the driver takes are of this and all Linux sees is
> a nice compliant config space.

Thanks for explaining that.

So it seems that we are faced with two requirements that are somewhat at
odds with one another.

1) Some of the root port bridge registers have to be accessed via config
space access functions so common PCI enumeration code will work.  To
represent this with the usual DT structure, the top root-complex needs
to define a 3/2 address space so its children
can have standard PCI reg properties.  Presumably, if those registers
are being programmed by common code, Marvell-specific code would
restrict its role to setting up config-space access functions, leaving
the actual touching of the registers to the common code.

2) Marvell chips have additional non-standard per-root-port registers
that generic PCI code would not understand.  These registers would be
touched only by Marvell-specific code.

The two kinds of registers are adjacent in MMIO space.  However, unless
I am misunderstanding this MV78230 manual, the highest "config header"
register index is 0x134, well below the 0x1000 size limit of a PCIe
config header.  Some of the extra registers are at 0x8xx, and others are
above 0x1800.

That suggests the following:

For the "config header" registers that use generic PCI code, use a
ranges entry to associate (pass through) MMIO addresses to the config
header portion of the register block.  This parameterizes the code that
sets up the special-case PCI config access.  That specification
technique is general and could be used not only for the Marvell case,
but also for any other chip that has some number of direct-mapped config
headers.

For requirement (2), the top node has a reg property listing the
portions of the address space that are consumed by the driver at the top
level instead of being passed through to the PCI addressing domain.  E.g.

  reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;

Thus we have accurately described the two aspects of the true situation.
 The PCI-compliant "config header" registers are passed through to the
child nodes where they can be dealt with in the normal PCI fashion.  The
non-PCI-compliant register footprint lives within the Marvell-specific
root-complex driver.

The root-complex driver presumably needs associate non-compliant
register blocks with specific child nodes.  That can be done by
requiring that the reg entries are in the same order as the config-space
ranges entries.

Anticipating the possible objection that ARM's 0x1000-byte page size
does not permit virtual-to-physical mapping at 0x800-byte granularity:
The device tree does not guarantee that reg entries are page-aligned; it
simply tries to describe the reality, even though it might be messy.

> 
>> If that is indeed the case, them I would vote for a slight modification
>> of the intermediate patch that I cited earlier - the one in which the
>> ranges property includes translations from those special config
>> addresses into CPU addresses.  The modification is to fix the sizes,
>> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
>> child address domain.

BTW I have completely changed my mind about the overlap thing.

I said that it was bogus to use size=0x2000 for a config space header.
 That was based on an interpretation - which I now dislike - of the
meaning of 3-cell config addresses.  By that old interpretation,
size=0x800 would also be bogus, because bits 10-8 of the config address
are for the function number.

Consider the following question, which I have never previously
considered, at least not explicitly:

Q: What would be the 3-cell representation of the Command/Status
register address (offset 4) in device 1, function 1?

One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
because the value 0x80000904 is what you put in the cf8 indirect access
address register.  But it is weak because it unnecessarily restricts the
config header size, and because it works differently than offsets
applied to memory and I/O space addresses.  Furthermore, coupling the
offset to the cf8 register is dodgy because of the funniness where you
have to but the byte-selector bits 1..0 not in the address register cf8,
but rather in the data register cfc.

The better answer is  <0x00000900 0 4>, using the third cell for the
offset, as with IO and memory space offsets.

Under this better interpretation, config space sizes larger than 0x100
are no problem.  The first cell is just a selector.  The size is "added"
to the third cell, so it does not encroach into the first cell's
device,function bits.

This better interpretation easily handles PCIe's 0x1000-byte extended
config headers, and trivially accomodates even larger sizes should a
future PCI variant further increase the header size. as described way
down below.

> 
> On Marvell the size of the MMIO space is 0x2000, there are imporant
> registers there beyond index 0x800 (todays driver may not touch them
> but HW has them). Reducing the size doesn't seem appropriate.

Per the above, the current idea is not to reduce the size, but rather to
split the 0x2000 into two pieces, accurately reflecting the fact that
part of it is PCI-compliant and can be handled by generic PCI drivers,
while the rest must be handled differently, outside of standard PCI
land.  Commingling the two seems dubious.

> 
>> pcie-controller {
>> 	compatible = "marvell,armada-370-pcie";
>> 	status = "disabled";
> 
>         device_type = "pci";                                                                    
> 
> Here as well, as you noted before?
> 
>> 	#address-cells = <3>;
>> 	#size-cells = <2>;
>>
>> 	bus-range = <0x00 0xff>;
>>
>> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
>> port config header */
>> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
>> port config header */
>> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
>> non-prefetchable memory */
>> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
>> downstream I/O */
>>
>> 	pcie@1,0 {
>> 		device_type = "pci";
>> 		reg = <0x0800 0 0 0 0>;
> 
> Okay, this was Thierry's original idea, and it was my note that this
> didn't seem like a good choice. I'm not wedded to that, but I'll
> explain my reasoning a bit.
> 
> Two things bothered me
>   - Describing a CPU MMIO mapping with a config address space seems
>     wonky

I agree that mapping config space is sort of a jarring concept, but I
think that's because PCs have polluted the mindspace, not because there
is anything inherently bad about it.  The original PCI spec
fundamentally treated config space as just another (rather segmented)
linear address space.  It specified the cf8/cfc indirect access
mechanism not as a deep semantic feature, but rather as an
implementation hack to work around addressing limitations of PC chipsets
and the addressing-deficient popular OS of the time (Windows 3.1 which
was a veneer over DOS).  The PCI spec defined configuration mechanism #2
as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
Alpha, Power PC) often direct-mapped config space into some form of
load/store space.

But PCs soon encroached on the RISC market and RISC system vendors
started to "leverage" PC I/O chipsets.  The indirect access "config
mechanism #1" became part of the mindset.  Indirect access was hardcoded
in enough software that chipset designers would provide indirect access
even if direct-mapping was possible.

Fundamentally, a config header is just a block of registers, addressable
as an index relative to some base.  It might require some specific
access path - but that is true with any block of registers.  Even
direct-mapped MMIO registers often require specific semantics such as
non-cached, non-write-buffered, or special address space ids.

The PCI SIG expected PCIe extended config space to be direct-mapped -
see slide 25 of

http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf

So mappability of config space was intended from the get-go.  The real
wonky thing is the "indirect access zombie that cannot die".

>   - Linux's OF core doesn't parse a 'reg' property under
>     'device_type = "pci"' as something CPU mappable, it only looks
>     to assigned-addresses for that. So the OF core would need to learn
>     how to handle this case, and it would have to be well defined..

The way I envisioned it working is that the root-complex node defines
its own config space access functions (the "ops" argument to
pci_scan_root_bus).  For config addresses listed in ranges, the
functions do MMIO.  For others, they use the chipset's indirect-access
registers.  The OF core is not involved.

> 
>     Thierry's original patch avoided this problem by not using
>     device_type = "pci"..
> 
> Also, did you mean to have a 0 size for the 'reg'? How will things
> know how big the MMIO region is? Can't just assume 0x800..

Sorry, that was a typo.

> 
> Again, thanks!
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11  5:46                                                     ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11  5:46 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
> On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
> 
>> Okay, I think I finally get it.  The Marvell root port bridge setup
>> registers look like standard config headers, even though they aren't
>> really in config space (because you need a different access method for
>> them, compared to real config accesses to downstream devices).
> 
> Well, thats pretty close. On Marvell the MMIO space has a lot of
> stuff, some of it is config related, lots of it isn't. In fact most of
> it isn't. Tegra is different, the MMIO region is exactly the config
> space of the root port bridge.
> 
> Tegra could probably happily and sanely do as you've described (well,
> see below), but it is wonky for Marvell. It looks like there are more
> drivers coming that have this need - so I had hoped for a general
> answer to the 'per-port MMIO space' problem that is unrelated to the
> MMIO spaces role in config access.
> 
>> So, in order for the common code that enumerates the PCI bus to work,
>> you need to "spoof" the config accesses so that when you try to access
>> those particular "pseudo config headers", it uses the MMIO method
>> instead of the real config access method.
> 
> There are rules for config access in both Tegra and Marvell that are
> not trivial. Again, the driver takes are of this and all Linux sees is
> a nice compliant config space.

Thanks for explaining that.

So it seems that we are faced with two requirements that are somewhat at
odds with one another.

1) Some of the root port bridge registers have to be accessed via config
space access functions so common PCI enumeration code will work.  To
represent this with the usual DT structure, the top root-complex needs
to define a 3/2 address space so its children
can have standard PCI reg properties.  Presumably, if those registers
are being programmed by common code, Marvell-specific code would
restrict its role to setting up config-space access functions, leaving
the actual touching of the registers to the common code.

2) Marvell chips have additional non-standard per-root-port registers
that generic PCI code would not understand.  These registers would be
touched only by Marvell-specific code.

The two kinds of registers are adjacent in MMIO space.  However, unless
I am misunderstanding this MV78230 manual, the highest "config header"
register index is 0x134, well below the 0x1000 size limit of a PCIe
config header.  Some of the extra registers are at 0x8xx, and others are
above 0x1800.

That suggests the following:

For the "config header" registers that use generic PCI code, use a
ranges entry to associate (pass through) MMIO addresses to the config
header portion of the register block.  This parameterizes the code that
sets up the special-case PCI config access.  That specification
technique is general and could be used not only for the Marvell case,
but also for any other chip that has some number of direct-mapped config
headers.

For requirement (2), the top node has a reg property listing the
portions of the address space that are consumed by the driver at the top
level instead of being passed through to the PCI addressing domain.  E.g.

  reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;

Thus we have accurately described the two aspects of the true situation.
 The PCI-compliant "config header" registers are passed through to the
child nodes where they can be dealt with in the normal PCI fashion.  The
non-PCI-compliant register footprint lives within the Marvell-specific
root-complex driver.

The root-complex driver presumably needs associate non-compliant
register blocks with specific child nodes.  That can be done by
requiring that the reg entries are in the same order as the config-space
ranges entries.

Anticipating the possible objection that ARM's 0x1000-byte page size
does not permit virtual-to-physical mapping at 0x800-byte granularity:
The device tree does not guarantee that reg entries are page-aligned; it
simply tries to describe the reality, even though it might be messy.

> 
>> If that is indeed the case, them I would vote for a slight modification
>> of the intermediate patch that I cited earlier - the one in which the
>> ranges property includes translations from those special config
>> addresses into CPU addresses.  The modification is to fix the sizes,
>> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
>> child address domain.

BTW I have completely changed my mind about the overlap thing.

I said that it was bogus to use size=0x2000 for a config space header.
 That was based on an interpretation - which I now dislike - of the
meaning of 3-cell config addresses.  By that old interpretation,
size=0x800 would also be bogus, because bits 10-8 of the config address
are for the function number.

Consider the following question, which I have never previously
considered, at least not explicitly:

Q: What would be the 3-cell representation of the Command/Status
register address (offset 4) in device 1, function 1?

One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
because the value 0x80000904 is what you put in the cf8 indirect access
address register.  But it is weak because it unnecessarily restricts the
config header size, and because it works differently than offsets
applied to memory and I/O space addresses.  Furthermore, coupling the
offset to the cf8 register is dodgy because of the funniness where you
have to but the byte-selector bits 1..0 not in the address register cf8,
but rather in the data register cfc.

The better answer is  <0x00000900 0 4>, using the third cell for the
offset, as with IO and memory space offsets.

Under this better interpretation, config space sizes larger than 0x100
are no problem.  The first cell is just a selector.  The size is "added"
to the third cell, so it does not encroach into the first cell's
device,function bits.

This better interpretation easily handles PCIe's 0x1000-byte extended
config headers, and trivially accomodates even larger sizes should a
future PCI variant further increase the header size. as described way
down below.

> 
> On Marvell the size of the MMIO space is 0x2000, there are imporant
> registers there beyond index 0x800 (todays driver may not touch them
> but HW has them). Reducing the size doesn't seem appropriate.

Per the above, the current idea is not to reduce the size, but rather to
split the 0x2000 into two pieces, accurately reflecting the fact that
part of it is PCI-compliant and can be handled by generic PCI drivers,
while the rest must be handled differently, outside of standard PCI
land.  Commingling the two seems dubious.

> 
>> pcie-controller {
>> 	compatible = "marvell,armada-370-pcie";
>> 	status = "disabled";
> 
>         device_type = "pci";                                                                    
> 
> Here as well, as you noted before?
> 
>> 	#address-cells = <3>;
>> 	#size-cells = <2>;
>>
>> 	bus-range = <0x00 0xff>;
>>
>> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
>> port config header */
>> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
>> port config header */
>> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
>> non-prefetchable memory */
>> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
>> downstream I/O */
>>
>> 	pcie at 1,0 {
>> 		device_type = "pci";
>> 		reg = <0x0800 0 0 0 0>;
> 
> Okay, this was Thierry's original idea, and it was my note that this
> didn't seem like a good choice. I'm not wedded to that, but I'll
> explain my reasoning a bit.
> 
> Two things bothered me
>   - Describing a CPU MMIO mapping with a config address space seems
>     wonky

I agree that mapping config space is sort of a jarring concept, but I
think that's because PCs have polluted the mindspace, not because there
is anything inherently bad about it.  The original PCI spec
fundamentally treated config space as just another (rather segmented)
linear address space.  It specified the cf8/cfc indirect access
mechanism not as a deep semantic feature, but rather as an
implementation hack to work around addressing limitations of PC chipsets
and the addressing-deficient popular OS of the time (Windows 3.1 which
was a veneer over DOS).  The PCI spec defined configuration mechanism #2
as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
Alpha, Power PC) often direct-mapped config space into some form of
load/store space.

But PCs soon encroached on the RISC market and RISC system vendors
started to "leverage" PC I/O chipsets.  The indirect access "config
mechanism #1" became part of the mindset.  Indirect access was hardcoded
in enough software that chipset designers would provide indirect access
even if direct-mapping was possible.

Fundamentally, a config header is just a block of registers, addressable
as an index relative to some base.  It might require some specific
access path - but that is true with any block of registers.  Even
direct-mapped MMIO registers often require specific semantics such as
non-cached, non-write-buffered, or special address space ids.

The PCI SIG expected PCIe extended config space to be direct-mapped -
see slide 25 of

http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf

So mappability of config space was intended from the get-go.  The real
wonky thing is the "indirect access zombie that cannot die".

>   - Linux's OF core doesn't parse a 'reg' property under
>     'device_type = "pci"' as something CPU mappable, it only looks
>     to assigned-addresses for that. So the OF core would need to learn
>     how to handle this case, and it would have to be well defined..

The way I envisioned it working is that the root-complex node defines
its own config space access functions (the "ops" argument to
pci_scan_root_bus).  For config addresses listed in ranges, the
functions do MMIO.  For others, they use the chipset's indirect-access
registers.  The OF core is not involved.

> 
>     Thierry's original patch avoided this problem by not using
>     device_type = "pci"..
> 
> Also, did you mean to have a 0 size for the 'reg'? How will things
> know how big the MMIO region is? Can't just assume 0x800..

Sorry, that was a typo.

> 
> Again, thanks!
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11  5:46                                                     ` Mitch Bradley
  (?)
@ 2013-03-11  7:46                                                         ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-11  7:46 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Jason Gunthorpe, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	Tawfik Bayouk, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r


[-- Attachment #1.1: Type: text/plain, Size: 11049 bytes --]

On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
> On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
> > On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
> > 
> >> Okay, I think I finally get it.  The Marvell root port bridge setup
> >> registers look like standard config headers, even though they aren't
> >> really in config space (because you need a different access method for
> >> them, compared to real config accesses to downstream devices).
> > 
> > Well, thats pretty close. On Marvell the MMIO space has a lot of
> > stuff, some of it is config related, lots of it isn't. In fact most of
> > it isn't. Tegra is different, the MMIO region is exactly the config
> > space of the root port bridge.
> > 
> > Tegra could probably happily and sanely do as you've described (well,
> > see below), but it is wonky for Marvell. It looks like there are more
> > drivers coming that have this need - so I had hoped for a general
> > answer to the 'per-port MMIO space' problem that is unrelated to the
> > MMIO spaces role in config access.
> > 
> >> So, in order for the common code that enumerates the PCI bus to work,
> >> you need to "spoof" the config accesses so that when you try to access
> >> those particular "pseudo config headers", it uses the MMIO method
> >> instead of the real config access method.
> > 
> > There are rules for config access in both Tegra and Marvell that are
> > not trivial. Again, the driver takes are of this and all Linux sees is
> > a nice compliant config space.
> 
> Thanks for explaining that.
> 
> So it seems that we are faced with two requirements that are somewhat at
> odds with one another.
> 
> 1) Some of the root port bridge registers have to be accessed via config
> space access functions so common PCI enumeration code will work.  To
> represent this with the usual DT structure, the top root-complex needs
> to define a 3/2 address space so its children
> can have standard PCI reg properties.  Presumably, if those registers
> are being programmed by common code, Marvell-specific code would
> restrict its role to setting up config-space access functions, leaving
> the actual touching of the registers to the common code.
> 
> 2) Marvell chips have additional non-standard per-root-port registers
> that generic PCI code would not understand.  These registers would be
> touched only by Marvell-specific code.
> 
> The two kinds of registers are adjacent in MMIO space.  However, unless
> I am misunderstanding this MV78230 manual, the highest "config header"
> register index is 0x134, well below the 0x1000 size limit of a PCIe
> config header.  Some of the extra registers are at 0x8xx, and others are
> above 0x1800.
> 
> That suggests the following:
> 
> For the "config header" registers that use generic PCI code, use a
> ranges entry to associate (pass through) MMIO addresses to the config
> header portion of the register block.  This parameterizes the code that
> sets up the special-case PCI config access.  That specification
> technique is general and could be used not only for the Marvell case,
> but also for any other chip that has some number of direct-mapped config
> headers.
> 
> For requirement (2), the top node has a reg property listing the
> portions of the address space that are consumed by the driver at the top
> level instead of being passed through to the PCI addressing domain.  E.g.
> 
>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
> 
> Thus we have accurately described the two aspects of the true situation.
>  The PCI-compliant "config header" registers are passed through to the
> child nodes where they can be dealt with in the normal PCI fashion.  The
> non-PCI-compliant register footprint lives within the Marvell-specific
> root-complex driver.
> 
> The root-complex driver presumably needs associate non-compliant
> register blocks with specific child nodes.  That can be done by
> requiring that the reg entries are in the same order as the config-space
> ranges entries.
> 
> Anticipating the possible objection that ARM's 0x1000-byte page size
> does not permit virtual-to-physical mapping at 0x800-byte granularity:
> The device tree does not guarantee that reg entries are page-aligned; it
> simply tries to describe the reality, even though it might be messy.
> 
> > 
> >> If that is indeed the case, them I would vote for a slight modification
> >> of the intermediate patch that I cited earlier - the one in which the
> >> ranges property includes translations from those special config
> >> addresses into CPU addresses.  The modification is to fix the sizes,
> >> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
> >> child address domain.
> 
> BTW I have completely changed my mind about the overlap thing.
> 
> I said that it was bogus to use size=0x2000 for a config space header.
>  That was based on an interpretation - which I now dislike - of the
> meaning of 3-cell config addresses.  By that old interpretation,
> size=0x800 would also be bogus, because bits 10-8 of the config address
> are for the function number.
> 
> Consider the following question, which I have never previously
> considered, at least not explicitly:
> 
> Q: What would be the 3-cell representation of the Command/Status
> register address (offset 4) in device 1, function 1?
> 
> One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
> because the value 0x80000904 is what you put in the cf8 indirect access
> address register.  But it is weak because it unnecessarily restricts the
> config header size, and because it works differently than offsets
> applied to memory and I/O space addresses.  Furthermore, coupling the
> offset to the cf8 register is dodgy because of the funniness where you
> have to but the byte-selector bits 1..0 not in the address register cf8,
> but rather in the data register cfc.
> 
> The better answer is  <0x00000900 0 4>, using the third cell for the
> offset, as with IO and memory space offsets.
> 
> Under this better interpretation, config space sizes larger than 0x100
> are no problem.  The first cell is just a selector.  The size is "added"
> to the third cell, so it does not encroach into the first cell's
> device,function bits.
> 
> This better interpretation easily handles PCIe's 0x1000-byte extended
> config headers, and trivially accomodates even larger sizes should a
> future PCI variant further increase the header size. as described way
> down below.

That makes a lot of sense. It also mirrors parts of a discussion we
previously had when we first discussed a DT binding for Tegra PCIe.

Thanks for taking the time to go over this in so much detail. I very
much appreciate it.

> >> 	#address-cells = <3>;
> >> 	#size-cells = <2>;
> >>
> >> 	bus-range = <0x00 0xff>;
> >>
> >> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
> >> port config header */
> >> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
> >> port config header */
> >> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
> >> non-prefetchable memory */
> >> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
> >> downstream I/O */
> >>
> >> 	pcie@1,0 {
> >> 		device_type = "pci";
> >> 		reg = <0x0800 0 0 0 0>;
> > 
> > Okay, this was Thierry's original idea, and it was my note that this
> > didn't seem like a good choice. I'm not wedded to that, but I'll
> > explain my reasoning a bit.
> > 
> > Two things bothered me
> >   - Describing a CPU MMIO mapping with a config address space seems
> >     wonky
> 
> I agree that mapping config space is sort of a jarring concept, but I
> think that's because PCs have polluted the mindspace, not because there
> is anything inherently bad about it.  The original PCI spec
> fundamentally treated config space as just another (rather segmented)
> linear address space.  It specified the cf8/cfc indirect access
> mechanism not as a deep semantic feature, but rather as an
> implementation hack to work around addressing limitations of PC chipsets
> and the addressing-deficient popular OS of the time (Windows 3.1 which
> was a veneer over DOS).  The PCI spec defined configuration mechanism #2
> as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
> Alpha, Power PC) often direct-mapped config space into some form of
> load/store space.
> 
> But PCs soon encroached on the RISC market and RISC system vendors
> started to "leverage" PC I/O chipsets.  The indirect access "config
> mechanism #1" became part of the mindset.  Indirect access was hardcoded
> in enough software that chipset designers would provide indirect access
> even if direct-mapping was possible.
> 
> Fundamentally, a config header is just a block of registers, addressable
> as an index relative to some base.  It might require some specific
> access path - but that is true with any block of registers.  Even
> direct-mapped MMIO registers often require specific semantics such as
> non-cached, non-write-buffered, or special address space ids.
> 
> The PCI SIG expected PCIe extended config space to be direct-mapped -
> see slide 25 of
> 
> http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf
> 
> So mappability of config space was intended from the get-go.  The real
> wonky thing is the "indirect access zombie that cannot die".
> 
> >   - Linux's OF core doesn't parse a 'reg' property under
> >     'device_type = "pci"' as something CPU mappable, it only looks
> >     to assigned-addresses for that. So the OF core would need to learn
> >     how to handle this case, and it would have to be well defined..
> 
> The way I envisioned it working is that the root-complex node defines
> its own config space access functions (the "ops" argument to
> pci_scan_root_bus).  For config addresses listed in ranges, the
> functions do MMIO.  For others, they use the chipset's indirect-access
> registers.  The OF core is not involved.
> 
> > 
> >     Thierry's original patch avoided this problem by not using
> >     device_type = "pci"..

I think the OF core still needs to be involved in order to translate the
reg = <0x0800 0 0 0 0>; entry into a resource that can be ioremap()'ed.
As Jason already said, the OF core only does that for assigned-addresses
when device_type = "pci".

Otherwise the pci_ops implementation still couldn't access the MMIO
registers. Aiming for a driver-specific solution doesn't seem like a
good idea but if the functionality is common enough to be required by
two or more drivers perhaps a new helper could be created for exactly
this purpose.

Perhaps another alternative would be to extend the OF core to translate
the entries in the reg property as well. Or maybe I misunderstood what
you said.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11  7:46                                                         ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-11  7:46 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 11049 bytes --]

On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
> On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
> > On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
> > 
> >> Okay, I think I finally get it.  The Marvell root port bridge setup
> >> registers look like standard config headers, even though they aren't
> >> really in config space (because you need a different access method for
> >> them, compared to real config accesses to downstream devices).
> > 
> > Well, thats pretty close. On Marvell the MMIO space has a lot of
> > stuff, some of it is config related, lots of it isn't. In fact most of
> > it isn't. Tegra is different, the MMIO region is exactly the config
> > space of the root port bridge.
> > 
> > Tegra could probably happily and sanely do as you've described (well,
> > see below), but it is wonky for Marvell. It looks like there are more
> > drivers coming that have this need - so I had hoped for a general
> > answer to the 'per-port MMIO space' problem that is unrelated to the
> > MMIO spaces role in config access.
> > 
> >> So, in order for the common code that enumerates the PCI bus to work,
> >> you need to "spoof" the config accesses so that when you try to access
> >> those particular "pseudo config headers", it uses the MMIO method
> >> instead of the real config access method.
> > 
> > There are rules for config access in both Tegra and Marvell that are
> > not trivial. Again, the driver takes are of this and all Linux sees is
> > a nice compliant config space.
> 
> Thanks for explaining that.
> 
> So it seems that we are faced with two requirements that are somewhat at
> odds with one another.
> 
> 1) Some of the root port bridge registers have to be accessed via config
> space access functions so common PCI enumeration code will work.  To
> represent this with the usual DT structure, the top root-complex needs
> to define a 3/2 address space so its children
> can have standard PCI reg properties.  Presumably, if those registers
> are being programmed by common code, Marvell-specific code would
> restrict its role to setting up config-space access functions, leaving
> the actual touching of the registers to the common code.
> 
> 2) Marvell chips have additional non-standard per-root-port registers
> that generic PCI code would not understand.  These registers would be
> touched only by Marvell-specific code.
> 
> The two kinds of registers are adjacent in MMIO space.  However, unless
> I am misunderstanding this MV78230 manual, the highest "config header"
> register index is 0x134, well below the 0x1000 size limit of a PCIe
> config header.  Some of the extra registers are at 0x8xx, and others are
> above 0x1800.
> 
> That suggests the following:
> 
> For the "config header" registers that use generic PCI code, use a
> ranges entry to associate (pass through) MMIO addresses to the config
> header portion of the register block.  This parameterizes the code that
> sets up the special-case PCI config access.  That specification
> technique is general and could be used not only for the Marvell case,
> but also for any other chip that has some number of direct-mapped config
> headers.
> 
> For requirement (2), the top node has a reg property listing the
> portions of the address space that are consumed by the driver at the top
> level instead of being passed through to the PCI addressing domain.  E.g.
> 
>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
> 
> Thus we have accurately described the two aspects of the true situation.
>  The PCI-compliant "config header" registers are passed through to the
> child nodes where they can be dealt with in the normal PCI fashion.  The
> non-PCI-compliant register footprint lives within the Marvell-specific
> root-complex driver.
> 
> The root-complex driver presumably needs associate non-compliant
> register blocks with specific child nodes.  That can be done by
> requiring that the reg entries are in the same order as the config-space
> ranges entries.
> 
> Anticipating the possible objection that ARM's 0x1000-byte page size
> does not permit virtual-to-physical mapping at 0x800-byte granularity:
> The device tree does not guarantee that reg entries are page-aligned; it
> simply tries to describe the reality, even though it might be messy.
> 
> > 
> >> If that is indeed the case, them I would vote for a slight modification
> >> of the intermediate patch that I cited earlier - the one in which the
> >> ranges property includes translations from those special config
> >> addresses into CPU addresses.  The modification is to fix the sizes,
> >> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
> >> child address domain.
> 
> BTW I have completely changed my mind about the overlap thing.
> 
> I said that it was bogus to use size=0x2000 for a config space header.
>  That was based on an interpretation - which I now dislike - of the
> meaning of 3-cell config addresses.  By that old interpretation,
> size=0x800 would also be bogus, because bits 10-8 of the config address
> are for the function number.
> 
> Consider the following question, which I have never previously
> considered, at least not explicitly:
> 
> Q: What would be the 3-cell representation of the Command/Status
> register address (offset 4) in device 1, function 1?
> 
> One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
> because the value 0x80000904 is what you put in the cf8 indirect access
> address register.  But it is weak because it unnecessarily restricts the
> config header size, and because it works differently than offsets
> applied to memory and I/O space addresses.  Furthermore, coupling the
> offset to the cf8 register is dodgy because of the funniness where you
> have to but the byte-selector bits 1..0 not in the address register cf8,
> but rather in the data register cfc.
> 
> The better answer is  <0x00000900 0 4>, using the third cell for the
> offset, as with IO and memory space offsets.
> 
> Under this better interpretation, config space sizes larger than 0x100
> are no problem.  The first cell is just a selector.  The size is "added"
> to the third cell, so it does not encroach into the first cell's
> device,function bits.
> 
> This better interpretation easily handles PCIe's 0x1000-byte extended
> config headers, and trivially accomodates even larger sizes should a
> future PCI variant further increase the header size. as described way
> down below.

That makes a lot of sense. It also mirrors parts of a discussion we
previously had when we first discussed a DT binding for Tegra PCIe.

Thanks for taking the time to go over this in so much detail. I very
much appreciate it.

> >> 	#address-cells = <3>;
> >> 	#size-cells = <2>;
> >>
> >> 	bus-range = <0x00 0xff>;
> >>
> >> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
> >> port config header */
> >> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
> >> port config header */
> >> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
> >> non-prefetchable memory */
> >> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
> >> downstream I/O */
> >>
> >> 	pcie@1,0 {
> >> 		device_type = "pci";
> >> 		reg = <0x0800 0 0 0 0>;
> > 
> > Okay, this was Thierry's original idea, and it was my note that this
> > didn't seem like a good choice. I'm not wedded to that, but I'll
> > explain my reasoning a bit.
> > 
> > Two things bothered me
> >   - Describing a CPU MMIO mapping with a config address space seems
> >     wonky
> 
> I agree that mapping config space is sort of a jarring concept, but I
> think that's because PCs have polluted the mindspace, not because there
> is anything inherently bad about it.  The original PCI spec
> fundamentally treated config space as just another (rather segmented)
> linear address space.  It specified the cf8/cfc indirect access
> mechanism not as a deep semantic feature, but rather as an
> implementation hack to work around addressing limitations of PC chipsets
> and the addressing-deficient popular OS of the time (Windows 3.1 which
> was a veneer over DOS).  The PCI spec defined configuration mechanism #2
> as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
> Alpha, Power PC) often direct-mapped config space into some form of
> load/store space.
> 
> But PCs soon encroached on the RISC market and RISC system vendors
> started to "leverage" PC I/O chipsets.  The indirect access "config
> mechanism #1" became part of the mindset.  Indirect access was hardcoded
> in enough software that chipset designers would provide indirect access
> even if direct-mapping was possible.
> 
> Fundamentally, a config header is just a block of registers, addressable
> as an index relative to some base.  It might require some specific
> access path - but that is true with any block of registers.  Even
> direct-mapped MMIO registers often require specific semantics such as
> non-cached, non-write-buffered, or special address space ids.
> 
> The PCI SIG expected PCIe extended config space to be direct-mapped -
> see slide 25 of
> 
> http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf
> 
> So mappability of config space was intended from the get-go.  The real
> wonky thing is the "indirect access zombie that cannot die".
> 
> >   - Linux's OF core doesn't parse a 'reg' property under
> >     'device_type = "pci"' as something CPU mappable, it only looks
> >     to assigned-addresses for that. So the OF core would need to learn
> >     how to handle this case, and it would have to be well defined..
> 
> The way I envisioned it working is that the root-complex node defines
> its own config space access functions (the "ops" argument to
> pci_scan_root_bus).  For config addresses listed in ranges, the
> functions do MMIO.  For others, they use the chipset's indirect-access
> registers.  The OF core is not involved.
> 
> > 
> >     Thierry's original patch avoided this problem by not using
> >     device_type = "pci"..

I think the OF core still needs to be involved in order to translate the
reg = <0x0800 0 0 0 0>; entry into a resource that can be ioremap()'ed.
As Jason already said, the OF core only does that for assigned-addresses
when device_type = "pci".

Otherwise the pci_ops implementation still couldn't access the MMIO
registers. Aiming for a driver-specific solution doesn't seem like a
good idea but if the functionality is common enough to be required by
two or more drivers perhaps a new helper could be created for exactly
this purpose.

Perhaps another alternative would be to extend the OF core to translate
the entries in the reg property as well. Or maybe I misunderstood what
you said.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11  7:46                                                         ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-11  7:46 UTC (permalink / raw)
  To: linux-arm-kernel

On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
> On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
> > On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
> > 
> >> Okay, I think I finally get it.  The Marvell root port bridge setup
> >> registers look like standard config headers, even though they aren't
> >> really in config space (because you need a different access method for
> >> them, compared to real config accesses to downstream devices).
> > 
> > Well, thats pretty close. On Marvell the MMIO space has a lot of
> > stuff, some of it is config related, lots of it isn't. In fact most of
> > it isn't. Tegra is different, the MMIO region is exactly the config
> > space of the root port bridge.
> > 
> > Tegra could probably happily and sanely do as you've described (well,
> > see below), but it is wonky for Marvell. It looks like there are more
> > drivers coming that have this need - so I had hoped for a general
> > answer to the 'per-port MMIO space' problem that is unrelated to the
> > MMIO spaces role in config access.
> > 
> >> So, in order for the common code that enumerates the PCI bus to work,
> >> you need to "spoof" the config accesses so that when you try to access
> >> those particular "pseudo config headers", it uses the MMIO method
> >> instead of the real config access method.
> > 
> > There are rules for config access in both Tegra and Marvell that are
> > not trivial. Again, the driver takes are of this and all Linux sees is
> > a nice compliant config space.
> 
> Thanks for explaining that.
> 
> So it seems that we are faced with two requirements that are somewhat at
> odds with one another.
> 
> 1) Some of the root port bridge registers have to be accessed via config
> space access functions so common PCI enumeration code will work.  To
> represent this with the usual DT structure, the top root-complex needs
> to define a 3/2 address space so its children
> can have standard PCI reg properties.  Presumably, if those registers
> are being programmed by common code, Marvell-specific code would
> restrict its role to setting up config-space access functions, leaving
> the actual touching of the registers to the common code.
> 
> 2) Marvell chips have additional non-standard per-root-port registers
> that generic PCI code would not understand.  These registers would be
> touched only by Marvell-specific code.
> 
> The two kinds of registers are adjacent in MMIO space.  However, unless
> I am misunderstanding this MV78230 manual, the highest "config header"
> register index is 0x134, well below the 0x1000 size limit of a PCIe
> config header.  Some of the extra registers are at 0x8xx, and others are
> above 0x1800.
> 
> That suggests the following:
> 
> For the "config header" registers that use generic PCI code, use a
> ranges entry to associate (pass through) MMIO addresses to the config
> header portion of the register block.  This parameterizes the code that
> sets up the special-case PCI config access.  That specification
> technique is general and could be used not only for the Marvell case,
> but also for any other chip that has some number of direct-mapped config
> headers.
> 
> For requirement (2), the top node has a reg property listing the
> portions of the address space that are consumed by the driver at the top
> level instead of being passed through to the PCI addressing domain.  E.g.
> 
>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
> 
> Thus we have accurately described the two aspects of the true situation.
>  The PCI-compliant "config header" registers are passed through to the
> child nodes where they can be dealt with in the normal PCI fashion.  The
> non-PCI-compliant register footprint lives within the Marvell-specific
> root-complex driver.
> 
> The root-complex driver presumably needs associate non-compliant
> register blocks with specific child nodes.  That can be done by
> requiring that the reg entries are in the same order as the config-space
> ranges entries.
> 
> Anticipating the possible objection that ARM's 0x1000-byte page size
> does not permit virtual-to-physical mapping at 0x800-byte granularity:
> The device tree does not guarantee that reg entries are page-aligned; it
> simply tries to describe the reality, even though it might be messy.
> 
> > 
> >> If that is indeed the case, them I would vote for a slight modification
> >> of the intermediate patch that I cited earlier - the one in which the
> >> ranges property includes translations from those special config
> >> addresses into CPU addresses.  The modification is to fix the sizes,
> >> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
> >> child address domain.
> 
> BTW I have completely changed my mind about the overlap thing.
> 
> I said that it was bogus to use size=0x2000 for a config space header.
>  That was based on an interpretation - which I now dislike - of the
> meaning of 3-cell config addresses.  By that old interpretation,
> size=0x800 would also be bogus, because bits 10-8 of the config address
> are for the function number.
> 
> Consider the following question, which I have never previously
> considered, at least not explicitly:
> 
> Q: What would be the 3-cell representation of the Command/Status
> register address (offset 4) in device 1, function 1?
> 
> One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
> because the value 0x80000904 is what you put in the cf8 indirect access
> address register.  But it is weak because it unnecessarily restricts the
> config header size, and because it works differently than offsets
> applied to memory and I/O space addresses.  Furthermore, coupling the
> offset to the cf8 register is dodgy because of the funniness where you
> have to but the byte-selector bits 1..0 not in the address register cf8,
> but rather in the data register cfc.
> 
> The better answer is  <0x00000900 0 4>, using the third cell for the
> offset, as with IO and memory space offsets.
> 
> Under this better interpretation, config space sizes larger than 0x100
> are no problem.  The first cell is just a selector.  The size is "added"
> to the third cell, so it does not encroach into the first cell's
> device,function bits.
> 
> This better interpretation easily handles PCIe's 0x1000-byte extended
> config headers, and trivially accomodates even larger sizes should a
> future PCI variant further increase the header size. as described way
> down below.

That makes a lot of sense. It also mirrors parts of a discussion we
previously had when we first discussed a DT binding for Tegra PCIe.

Thanks for taking the time to go over this in so much detail. I very
much appreciate it.

> >> 	#address-cells = <3>;
> >> 	#size-cells = <2>;
> >>
> >> 	bus-range = <0x00 0xff>;
> >>
> >> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
> >> port config header */
> >> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
> >> port config header */
> >> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
> >> non-prefetchable memory */
> >> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
> >> downstream I/O */
> >>
> >> 	pcie at 1,0 {
> >> 		device_type = "pci";
> >> 		reg = <0x0800 0 0 0 0>;
> > 
> > Okay, this was Thierry's original idea, and it was my note that this
> > didn't seem like a good choice. I'm not wedded to that, but I'll
> > explain my reasoning a bit.
> > 
> > Two things bothered me
> >   - Describing a CPU MMIO mapping with a config address space seems
> >     wonky
> 
> I agree that mapping config space is sort of a jarring concept, but I
> think that's because PCs have polluted the mindspace, not because there
> is anything inherently bad about it.  The original PCI spec
> fundamentally treated config space as just another (rather segmented)
> linear address space.  It specified the cf8/cfc indirect access
> mechanism not as a deep semantic feature, but rather as an
> implementation hack to work around addressing limitations of PC chipsets
> and the addressing-deficient popular OS of the time (Windows 3.1 which
> was a veneer over DOS).  The PCI spec defined configuration mechanism #2
> as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
> Alpha, Power PC) often direct-mapped config space into some form of
> load/store space.
> 
> But PCs soon encroached on the RISC market and RISC system vendors
> started to "leverage" PC I/O chipsets.  The indirect access "config
> mechanism #1" became part of the mindset.  Indirect access was hardcoded
> in enough software that chipset designers would provide indirect access
> even if direct-mapping was possible.
> 
> Fundamentally, a config header is just a block of registers, addressable
> as an index relative to some base.  It might require some specific
> access path - but that is true with any block of registers.  Even
> direct-mapped MMIO registers often require specific semantics such as
> non-cached, non-write-buffered, or special address space ids.
> 
> The PCI SIG expected PCIe extended config space to be direct-mapped -
> see slide 25 of
> 
> http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf
> 
> So mappability of config space was intended from the get-go.  The real
> wonky thing is the "indirect access zombie that cannot die".
> 
> >   - Linux's OF core doesn't parse a 'reg' property under
> >     'device_type = "pci"' as something CPU mappable, it only looks
> >     to assigned-addresses for that. So the OF core would need to learn
> >     how to handle this case, and it would have to be well defined..
> 
> The way I envisioned it working is that the root-complex node defines
> its own config space access functions (the "ops" argument to
> pci_scan_root_bus).  For config addresses listed in ranges, the
> functions do MMIO.  For others, they use the chipset's indirect-access
> registers.  The OF core is not involved.
> 
> > 
> >     Thierry's original patch avoided this problem by not using
> >     device_type = "pci"..

I think the OF core still needs to be involved in order to translate the
reg = <0x0800 0 0 0 0>; entry into a resource that can be ioremap()'ed.
As Jason already said, the OF core only does that for assigned-addresses
when device_type = "pci".

Otherwise the pci_ops implementation still couldn't access the MMIO
registers. Aiming for a driver-specific solution doesn't seem like a
good idea but if the functionality is common enough to be required by
two or more drivers perhaps a new helper could be created for exactly
this purpose.

Perhaps another alternative would be to extend the OF core to translate
the entries in the reg property as well. Or maybe I misunderstood what
you said.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130311/4e7dd462/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-10 15:09                                           ` Thomas Petazzoni
@ 2013-03-11  8:08                                             ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-11  8:08 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Jason Gunthorpe, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Tawfik Bayouk, linux-pci,
	devicetree-discuss, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]

On Sun, Mar 10, 2013 at 04:09:37PM +0100, Thomas Petazzoni wrote:
> Dear Thierry Reding,
> 
> On Fri, 8 Mar 2013 21:13:40 +0100, Thierry Reding wrote:
> > On Fri, Mar 08, 2013 at 01:02:46PM -0700, Jason Gunthorpe wrote:
> > > On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> > [...]
> > > > 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> > > > <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> > > 
> > > @0,1,0 (bus,device,fn) could be more appropriate, but that is
> > > cosmetic?
> > 
> > The OF PCI specification is pretty strict about this as well. It says in
> > section 2.2.1.3. that only the DD and DD,FF forms can appear in a device
> > path.
> 
> Note that in the case of my driver, the @X,Y represent the port and
> lane of the PCIe interface. Not sure if it is correct, I can change the
> names to whatever is appropriate, those names aren't used anywhere in
> the driver.

I think they need to be changed to be @dev,fn in order to conform to the
OF PCI binding.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11  8:08                                             ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-11  8:08 UTC (permalink / raw)
  To: linux-arm-kernel

On Sun, Mar 10, 2013 at 04:09:37PM +0100, Thomas Petazzoni wrote:
> Dear Thierry Reding,
> 
> On Fri, 8 Mar 2013 21:13:40 +0100, Thierry Reding wrote:
> > On Fri, Mar 08, 2013 at 01:02:46PM -0700, Jason Gunthorpe wrote:
> > > On Fri, Mar 08, 2013 at 09:43:11AM -1000, Mitch Bradley wrote:
> > [...]
> > > > 2) The "@0,0" and "@1,0" suffixes do not correspond to the reg values
> > > > <0x0800 0 0 0 0> and <0x1000 0 0 0 0> using any rule that I know.
> > > 
> > > @0,1,0 (bus,device,fn) could be more appropriate, but that is
> > > cosmetic?
> > 
> > The OF PCI specification is pretty strict about this as well. It says in
> > section 2.2.1.3. that only the DD and DD,FF forms can appear in a device
> > path.
> 
> Note that in the case of my driver, the @X,Y represent the port and
> lane of the PCIe interface. Not sure if it is correct, I can change the
> names to whatever is appropriate, those names aren't used anywhere in
> the driver.

I think they need to be changed to be @dev,fn in order to conform to the
OF PCI binding.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130311/40ace6a4/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11  7:46                                                         ` Thierry Reding
  (?)
@ 2013-03-11 18:04                                                             ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 18:04 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Jason Gunthorpe, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	Tawfik Bayouk, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/10/2013 9:46 PM, Thierry Reding wrote:
> On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
>> On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
>>> On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
>>>
>>>> Okay, I think I finally get it.  The Marvell root port bridge setup
>>>> registers look like standard config headers, even though they aren't
>>>> really in config space (because you need a different access method for
>>>> them, compared to real config accesses to downstream devices).
>>>
>>> Well, thats pretty close. On Marvell the MMIO space has a lot of
>>> stuff, some of it is config related, lots of it isn't. In fact most of
>>> it isn't. Tegra is different, the MMIO region is exactly the config
>>> space of the root port bridge.
>>>
>>> Tegra could probably happily and sanely do as you've described (well,
>>> see below), but it is wonky for Marvell. It looks like there are more
>>> drivers coming that have this need - so I had hoped for a general
>>> answer to the 'per-port MMIO space' problem that is unrelated to the
>>> MMIO spaces role in config access.
>>>
>>>> So, in order for the common code that enumerates the PCI bus to work,
>>>> you need to "spoof" the config accesses so that when you try to access
>>>> those particular "pseudo config headers", it uses the MMIO method
>>>> instead of the real config access method.
>>>
>>> There are rules for config access in both Tegra and Marvell that are
>>> not trivial. Again, the driver takes are of this and all Linux sees is
>>> a nice compliant config space.
>>
>> Thanks for explaining that.
>>
>> So it seems that we are faced with two requirements that are somewhat at
>> odds with one another.
>>
>> 1) Some of the root port bridge registers have to be accessed via config
>> space access functions so common PCI enumeration code will work.  To
>> represent this with the usual DT structure, the top root-complex needs
>> to define a 3/2 address space so its children
>> can have standard PCI reg properties.  Presumably, if those registers
>> are being programmed by common code, Marvell-specific code would
>> restrict its role to setting up config-space access functions, leaving
>> the actual touching of the registers to the common code.
>>
>> 2) Marvell chips have additional non-standard per-root-port registers
>> that generic PCI code would not understand.  These registers would be
>> touched only by Marvell-specific code.
>>
>> The two kinds of registers are adjacent in MMIO space.  However, unless
>> I am misunderstanding this MV78230 manual, the highest "config header"
>> register index is 0x134, well below the 0x1000 size limit of a PCIe
>> config header.  Some of the extra registers are at 0x8xx, and others are
>> above 0x1800.
>>
>> That suggests the following:
>>
>> For the "config header" registers that use generic PCI code, use a
>> ranges entry to associate (pass through) MMIO addresses to the config
>> header portion of the register block.  This parameterizes the code that
>> sets up the special-case PCI config access.  That specification
>> technique is general and could be used not only for the Marvell case,
>> but also for any other chip that has some number of direct-mapped config
>> headers.
>>
>> For requirement (2), the top node has a reg property listing the
>> portions of the address space that are consumed by the driver at the top
>> level instead of being passed through to the PCI addressing domain.  E.g.
>>
>>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
>>
>> Thus we have accurately described the two aspects of the true situation.
>>  The PCI-compliant "config header" registers are passed through to the
>> child nodes where they can be dealt with in the normal PCI fashion.  The
>> non-PCI-compliant register footprint lives within the Marvell-specific
>> root-complex driver.
>>
>> The root-complex driver presumably needs associate non-compliant
>> register blocks with specific child nodes.  That can be done by
>> requiring that the reg entries are in the same order as the config-space
>> ranges entries.
>>
>> Anticipating the possible objection that ARM's 0x1000-byte page size
>> does not permit virtual-to-physical mapping at 0x800-byte granularity:
>> The device tree does not guarantee that reg entries are page-aligned; it
>> simply tries to describe the reality, even though it might be messy.
>>
>>>
>>>> If that is indeed the case, them I would vote for a slight modification
>>>> of the intermediate patch that I cited earlier - the one in which the
>>>> ranges property includes translations from those special config
>>>> addresses into CPU addresses.  The modification is to fix the sizes,
>>>> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
>>>> child address domain.
>>
>> BTW I have completely changed my mind about the overlap thing.
>>
>> I said that it was bogus to use size=0x2000 for a config space header.
>>  That was based on an interpretation - which I now dislike - of the
>> meaning of 3-cell config addresses.  By that old interpretation,
>> size=0x800 would also be bogus, because bits 10-8 of the config address
>> are for the function number.
>>
>> Consider the following question, which I have never previously
>> considered, at least not explicitly:
>>
>> Q: What would be the 3-cell representation of the Command/Status
>> register address (offset 4) in device 1, function 1?
>>
>> One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
>> because the value 0x80000904 is what you put in the cf8 indirect access
>> address register.  But it is weak because it unnecessarily restricts the
>> config header size, and because it works differently than offsets
>> applied to memory and I/O space addresses.  Furthermore, coupling the
>> offset to the cf8 register is dodgy because of the funniness where you
>> have to but the byte-selector bits 1..0 not in the address register cf8,
>> but rather in the data register cfc.
>>
>> The better answer is  <0x00000900 0 4>, using the third cell for the
>> offset, as with IO and memory space offsets.
>>
>> Under this better interpretation, config space sizes larger than 0x100
>> are no problem.  The first cell is just a selector.  The size is "added"
>> to the third cell, so it does not encroach into the first cell's
>> device,function bits.
>>
>> This better interpretation easily handles PCIe's 0x1000-byte extended
>> config headers, and trivially accomodates even larger sizes should a
>> future PCI variant further increase the header size. as described way
>> down below.
> 
> That makes a lot of sense. It also mirrors parts of a discussion we
> previously had when we first discussed a DT binding for Tegra PCIe.
> 
> Thanks for taking the time to go over this in so much detail. I very
> much appreciate it.
> 
>>>> 	#address-cells = <3>;
>>>> 	#size-cells = <2>;
>>>>
>>>> 	bus-range = <0x00 0xff>;
>>>>
>>>> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
>>>> port config header */
>>>> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
>>>> port config header */
>>>> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
>>>> non-prefetchable memory */
>>>> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
>>>> downstream I/O */
>>>>
>>>> 	pcie@1,0 {
>>>> 		device_type = "pci";
>>>> 		reg = <0x0800 0 0 0 0>;
>>>
>>> Okay, this was Thierry's original idea, and it was my note that this
>>> didn't seem like a good choice. I'm not wedded to that, but I'll
>>> explain my reasoning a bit.
>>>
>>> Two things bothered me
>>>   - Describing a CPU MMIO mapping with a config address space seems
>>>     wonky
>>
>> I agree that mapping config space is sort of a jarring concept, but I
>> think that's because PCs have polluted the mindspace, not because there
>> is anything inherently bad about it.  The original PCI spec
>> fundamentally treated config space as just another (rather segmented)
>> linear address space.  It specified the cf8/cfc indirect access
>> mechanism not as a deep semantic feature, but rather as an
>> implementation hack to work around addressing limitations of PC chipsets
>> and the addressing-deficient popular OS of the time (Windows 3.1 which
>> was a veneer over DOS).  The PCI spec defined configuration mechanism #2
>> as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
>> Alpha, Power PC) often direct-mapped config space into some form of
>> load/store space.
>>
>> But PCs soon encroached on the RISC market and RISC system vendors
>> started to "leverage" PC I/O chipsets.  The indirect access "config
>> mechanism #1" became part of the mindset.  Indirect access was hardcoded
>> in enough software that chipset designers would provide indirect access
>> even if direct-mapping was possible.
>>
>> Fundamentally, a config header is just a block of registers, addressable
>> as an index relative to some base.  It might require some specific
>> access path - but that is true with any block of registers.  Even
>> direct-mapped MMIO registers often require specific semantics such as
>> non-cached, non-write-buffered, or special address space ids.
>>
>> The PCI SIG expected PCIe extended config space to be direct-mapped -
>> see slide 25 of
>>
>> http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf
>>
>> So mappability of config space was intended from the get-go.  The real
>> wonky thing is the "indirect access zombie that cannot die".
>>
>>>   - Linux's OF core doesn't parse a 'reg' property under
>>>     'device_type = "pci"' as something CPU mappable, it only looks
>>>     to assigned-addresses for that. So the OF core would need to learn
>>>     how to handle this case, and it would have to be well defined..
>>
>> The way I envisioned it working is that the root-complex node defines
>> its own config space access functions (the "ops" argument to
>> pci_scan_root_bus).  For config addresses listed in ranges, the
>> functions do MMIO.  For others, they use the chipset's indirect-access
>> registers.  The OF core is not involved.
>>
>>>
>>>     Thierry's original patch avoided this problem by not using
>>>     device_type = "pci"..
> 
> I think the OF core still needs to be involved in order to translate the
> reg = <0x0800 0 0 0 0>; entry into a resource that can be ioremap()'ed.
> As Jason already said, the OF core only does that for assigned-addresses
> when device_type = "pci".
> 
> Otherwise the pci_ops implementation still couldn't access the MMIO
> registers. 

I'm not sure I understand your point that <0x0800 ...> needs to be
translated into a resource.

(a) Existing code that needs to match a child nodes to a discovered PCI
hardware device uses the existing of_pci_find_child_device(parent,
devfn) from of_pci.c .  That routine "just works", because the reg =
<0x0800 ..> entry is standard.

(b) The discovery/enumeration code needs to access config space via
pci_ops.  The root complex driver implements pci_ops based on a trivial
parsing of ranges (omitting irrelevant details):

pci_op_read(devfn, pos) {
   loop_over_ranges_entries {
      if (to_devfn(ranges_entry.child) == devfn) {
         return mmio_read(ranges_entry.parent + pos);
   }
   return indirect_read(devfn, pos);
}

The actual implementation might setup different data structures, but the
pseudocode captures the essential points:  The input argument is devfn,
and a simple scan of ranges controls whether to use MMIO or indirect
access for a given devfn value.  Every aspect of the scanning loop is
trivial.  The loop itself is just a stride-5 walk over an array of integers.

"ranges_entry.parent" is be32_to_cpu(ranges_entry[3])

and

"to_devfn(ranges_entry.child)" is "be32_to_cpup(ranges_entry[0]) >> 8"

(you don't want to mask the result in this case because not masking
prevents non-config-space ranges entries from matching devfn.)

> Aiming for a driver-specific solution doesn't seem like a
> good idea but if the functionality is common enough to be required by
> two or more drivers perhaps a new helper could be created for exactly
> this purpose.

Indeed - but it is often best to wait and see rather than guessing about
what sort of common helper is needed.  I usually guess wrong in such cases.

> 
> Perhaps another alternative would be to extend the OF core to translate
> the entries in the reg property as well. Or maybe I misunderstood what
> you said.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 18:04                                                             ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 18:04 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/10/2013 9:46 PM, Thierry Reding wrote:
> On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
>> On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
>>> On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
>>>
>>>> Okay, I think I finally get it.  The Marvell root port bridge setup
>>>> registers look like standard config headers, even though they aren't
>>>> really in config space (because you need a different access method for
>>>> them, compared to real config accesses to downstream devices).
>>>
>>> Well, thats pretty close. On Marvell the MMIO space has a lot of
>>> stuff, some of it is config related, lots of it isn't. In fact most of
>>> it isn't. Tegra is different, the MMIO region is exactly the config
>>> space of the root port bridge.
>>>
>>> Tegra could probably happily and sanely do as you've described (well,
>>> see below), but it is wonky for Marvell. It looks like there are more
>>> drivers coming that have this need - so I had hoped for a general
>>> answer to the 'per-port MMIO space' problem that is unrelated to the
>>> MMIO spaces role in config access.
>>>
>>>> So, in order for the common code that enumerates the PCI bus to work,
>>>> you need to "spoof" the config accesses so that when you try to access
>>>> those particular "pseudo config headers", it uses the MMIO method
>>>> instead of the real config access method.
>>>
>>> There are rules for config access in both Tegra and Marvell that are
>>> not trivial. Again, the driver takes are of this and all Linux sees is
>>> a nice compliant config space.
>>
>> Thanks for explaining that.
>>
>> So it seems that we are faced with two requirements that are somewhat at
>> odds with one another.
>>
>> 1) Some of the root port bridge registers have to be accessed via config
>> space access functions so common PCI enumeration code will work.  To
>> represent this with the usual DT structure, the top root-complex needs
>> to define a 3/2 address space so its children
>> can have standard PCI reg properties.  Presumably, if those registers
>> are being programmed by common code, Marvell-specific code would
>> restrict its role to setting up config-space access functions, leaving
>> the actual touching of the registers to the common code.
>>
>> 2) Marvell chips have additional non-standard per-root-port registers
>> that generic PCI code would not understand.  These registers would be
>> touched only by Marvell-specific code.
>>
>> The two kinds of registers are adjacent in MMIO space.  However, unless
>> I am misunderstanding this MV78230 manual, the highest "config header"
>> register index is 0x134, well below the 0x1000 size limit of a PCIe
>> config header.  Some of the extra registers are at 0x8xx, and others are
>> above 0x1800.
>>
>> That suggests the following:
>>
>> For the "config header" registers that use generic PCI code, use a
>> ranges entry to associate (pass through) MMIO addresses to the config
>> header portion of the register block.  This parameterizes the code that
>> sets up the special-case PCI config access.  That specification
>> technique is general and could be used not only for the Marvell case,
>> but also for any other chip that has some number of direct-mapped config
>> headers.
>>
>> For requirement (2), the top node has a reg property listing the
>> portions of the address space that are consumed by the driver at the top
>> level instead of being passed through to the PCI addressing domain.  E.g.
>>
>>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
>>
>> Thus we have accurately described the two aspects of the true situation.
>>  The PCI-compliant "config header" registers are passed through to the
>> child nodes where they can be dealt with in the normal PCI fashion.  The
>> non-PCI-compliant register footprint lives within the Marvell-specific
>> root-complex driver.
>>
>> The root-complex driver presumably needs associate non-compliant
>> register blocks with specific child nodes.  That can be done by
>> requiring that the reg entries are in the same order as the config-space
>> ranges entries.
>>
>> Anticipating the possible objection that ARM's 0x1000-byte page size
>> does not permit virtual-to-physical mapping at 0x800-byte granularity:
>> The device tree does not guarantee that reg entries are page-aligned; it
>> simply tries to describe the reality, even though it might be messy.
>>
>>>
>>>> If that is indeed the case, them I would vote for a slight modification
>>>> of the intermediate patch that I cited earlier - the one in which the
>>>> ranges property includes translations from those special config
>>>> addresses into CPU addresses.  The modification is to fix the sizes,
>>>> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
>>>> child address domain.
>>
>> BTW I have completely changed my mind about the overlap thing.
>>
>> I said that it was bogus to use size=0x2000 for a config space header.
>>  That was based on an interpretation - which I now dislike - of the
>> meaning of 3-cell config addresses.  By that old interpretation,
>> size=0x800 would also be bogus, because bits 10-8 of the config address
>> are for the function number.
>>
>> Consider the following question, which I have never previously
>> considered, at least not explicitly:
>>
>> Q: What would be the 3-cell representation of the Command/Status
>> register address (offset 4) in device 1, function 1?
>>
>> One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
>> because the value 0x80000904 is what you put in the cf8 indirect access
>> address register.  But it is weak because it unnecessarily restricts the
>> config header size, and because it works differently than offsets
>> applied to memory and I/O space addresses.  Furthermore, coupling the
>> offset to the cf8 register is dodgy because of the funniness where you
>> have to but the byte-selector bits 1..0 not in the address register cf8,
>> but rather in the data register cfc.
>>
>> The better answer is  <0x00000900 0 4>, using the third cell for the
>> offset, as with IO and memory space offsets.
>>
>> Under this better interpretation, config space sizes larger than 0x100
>> are no problem.  The first cell is just a selector.  The size is "added"
>> to the third cell, so it does not encroach into the first cell's
>> device,function bits.
>>
>> This better interpretation easily handles PCIe's 0x1000-byte extended
>> config headers, and trivially accomodates even larger sizes should a
>> future PCI variant further increase the header size. as described way
>> down below.
> 
> That makes a lot of sense. It also mirrors parts of a discussion we
> previously had when we first discussed a DT binding for Tegra PCIe.
> 
> Thanks for taking the time to go over this in so much detail. I very
> much appreciate it.
> 
>>>> 	#address-cells = <3>;
>>>> 	#size-cells = <2>;
>>>>
>>>> 	bus-range = <0x00 0xff>;
>>>>
>>>> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
>>>> port config header */
>>>> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
>>>> port config header */
>>>> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
>>>> non-prefetchable memory */
>>>> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
>>>> downstream I/O */
>>>>
>>>> 	pcie@1,0 {
>>>> 		device_type = "pci";
>>>> 		reg = <0x0800 0 0 0 0>;
>>>
>>> Okay, this was Thierry's original idea, and it was my note that this
>>> didn't seem like a good choice. I'm not wedded to that, but I'll
>>> explain my reasoning a bit.
>>>
>>> Two things bothered me
>>>   - Describing a CPU MMIO mapping with a config address space seems
>>>     wonky
>>
>> I agree that mapping config space is sort of a jarring concept, but I
>> think that's because PCs have polluted the mindspace, not because there
>> is anything inherently bad about it.  The original PCI spec
>> fundamentally treated config space as just another (rather segmented)
>> linear address space.  It specified the cf8/cfc indirect access
>> mechanism not as a deep semantic feature, but rather as an
>> implementation hack to work around addressing limitations of PC chipsets
>> and the addressing-deficient popular OS of the time (Windows 3.1 which
>> was a veneer over DOS).  The PCI spec defined configuration mechanism #2
>> as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
>> Alpha, Power PC) often direct-mapped config space into some form of
>> load/store space.
>>
>> But PCs soon encroached on the RISC market and RISC system vendors
>> started to "leverage" PC I/O chipsets.  The indirect access "config
>> mechanism #1" became part of the mindset.  Indirect access was hardcoded
>> in enough software that chipset designers would provide indirect access
>> even if direct-mapping was possible.
>>
>> Fundamentally, a config header is just a block of registers, addressable
>> as an index relative to some base.  It might require some specific
>> access path - but that is true with any block of registers.  Even
>> direct-mapped MMIO registers often require specific semantics such as
>> non-cached, non-write-buffered, or special address space ids.
>>
>> The PCI SIG expected PCIe extended config space to be direct-mapped -
>> see slide 25 of
>>
>> http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf
>>
>> So mappability of config space was intended from the get-go.  The real
>> wonky thing is the "indirect access zombie that cannot die".
>>
>>>   - Linux's OF core doesn't parse a 'reg' property under
>>>     'device_type = "pci"' as something CPU mappable, it only looks
>>>     to assigned-addresses for that. So the OF core would need to learn
>>>     how to handle this case, and it would have to be well defined..
>>
>> The way I envisioned it working is that the root-complex node defines
>> its own config space access functions (the "ops" argument to
>> pci_scan_root_bus).  For config addresses listed in ranges, the
>> functions do MMIO.  For others, they use the chipset's indirect-access
>> registers.  The OF core is not involved.
>>
>>>
>>>     Thierry's original patch avoided this problem by not using
>>>     device_type = "pci"..
> 
> I think the OF core still needs to be involved in order to translate the
> reg = <0x0800 0 0 0 0>; entry into a resource that can be ioremap()'ed.
> As Jason already said, the OF core only does that for assigned-addresses
> when device_type = "pci".
> 
> Otherwise the pci_ops implementation still couldn't access the MMIO
> registers. 

I'm not sure I understand your point that <0x0800 ...> needs to be
translated into a resource.

(a) Existing code that needs to match a child nodes to a discovered PCI
hardware device uses the existing of_pci_find_child_device(parent,
devfn) from of_pci.c .  That routine "just works", because the reg =
<0x0800 ..> entry is standard.

(b) The discovery/enumeration code needs to access config space via
pci_ops.  The root complex driver implements pci_ops based on a trivial
parsing of ranges (omitting irrelevant details):

pci_op_read(devfn, pos) {
   loop_over_ranges_entries {
      if (to_devfn(ranges_entry.child) == devfn) {
         return mmio_read(ranges_entry.parent + pos);
   }
   return indirect_read(devfn, pos);
}

The actual implementation might setup different data structures, but the
pseudocode captures the essential points:  The input argument is devfn,
and a simple scan of ranges controls whether to use MMIO or indirect
access for a given devfn value.  Every aspect of the scanning loop is
trivial.  The loop itself is just a stride-5 walk over an array of integers.

"ranges_entry.parent" is be32_to_cpu(ranges_entry[3])

and

"to_devfn(ranges_entry.child)" is "be32_to_cpup(ranges_entry[0]) >> 8"

(you don't want to mask the result in this case because not masking
prevents non-config-space ranges entries from matching devfn.)

> Aiming for a driver-specific solution doesn't seem like a
> good idea but if the functionality is common enough to be required by
> two or more drivers perhaps a new helper could be created for exactly
> this purpose.

Indeed - but it is often best to wait and see rather than guessing about
what sort of common helper is needed.  I usually guess wrong in such cases.

> 
> Perhaps another alternative would be to extend the OF core to translate
> the entries in the reg property as well. Or maybe I misunderstood what
> you said.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 18:04                                                             ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 18:04 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/10/2013 9:46 PM, Thierry Reding wrote:
> On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
>> On 3/9/2013 8:55 PM, Jason Gunthorpe wrote:
>>> On Sat, Mar 09, 2013 at 06:52:13PM -1000, Mitch Bradley wrote:
>>>
>>>> Okay, I think I finally get it.  The Marvell root port bridge setup
>>>> registers look like standard config headers, even though they aren't
>>>> really in config space (because you need a different access method for
>>>> them, compared to real config accesses to downstream devices).
>>>
>>> Well, thats pretty close. On Marvell the MMIO space has a lot of
>>> stuff, some of it is config related, lots of it isn't. In fact most of
>>> it isn't. Tegra is different, the MMIO region is exactly the config
>>> space of the root port bridge.
>>>
>>> Tegra could probably happily and sanely do as you've described (well,
>>> see below), but it is wonky for Marvell. It looks like there are more
>>> drivers coming that have this need - so I had hoped for a general
>>> answer to the 'per-port MMIO space' problem that is unrelated to the
>>> MMIO spaces role in config access.
>>>
>>>> So, in order for the common code that enumerates the PCI bus to work,
>>>> you need to "spoof" the config accesses so that when you try to access
>>>> those particular "pseudo config headers", it uses the MMIO method
>>>> instead of the real config access method.
>>>
>>> There are rules for config access in both Tegra and Marvell that are
>>> not trivial. Again, the driver takes are of this and all Linux sees is
>>> a nice compliant config space.
>>
>> Thanks for explaining that.
>>
>> So it seems that we are faced with two requirements that are somewhat at
>> odds with one another.
>>
>> 1) Some of the root port bridge registers have to be accessed via config
>> space access functions so common PCI enumeration code will work.  To
>> represent this with the usual DT structure, the top root-complex needs
>> to define a 3/2 address space so its children
>> can have standard PCI reg properties.  Presumably, if those registers
>> are being programmed by common code, Marvell-specific code would
>> restrict its role to setting up config-space access functions, leaving
>> the actual touching of the registers to the common code.
>>
>> 2) Marvell chips have additional non-standard per-root-port registers
>> that generic PCI code would not understand.  These registers would be
>> touched only by Marvell-specific code.
>>
>> The two kinds of registers are adjacent in MMIO space.  However, unless
>> I am misunderstanding this MV78230 manual, the highest "config header"
>> register index is 0x134, well below the 0x1000 size limit of a PCIe
>> config header.  Some of the extra registers are at 0x8xx, and others are
>> above 0x1800.
>>
>> That suggests the following:
>>
>> For the "config header" registers that use generic PCI code, use a
>> ranges entry to associate (pass through) MMIO addresses to the config
>> header portion of the register block.  This parameterizes the code that
>> sets up the special-case PCI config access.  That specification
>> technique is general and could be used not only for the Marvell case,
>> but also for any other chip that has some number of direct-mapped config
>> headers.
>>
>> For requirement (2), the top node has a reg property listing the
>> portions of the address space that are consumed by the driver at the top
>> level instead of being passed through to the PCI addressing domain.  E.g.
>>
>>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
>>
>> Thus we have accurately described the two aspects of the true situation.
>>  The PCI-compliant "config header" registers are passed through to the
>> child nodes where they can be dealt with in the normal PCI fashion.  The
>> non-PCI-compliant register footprint lives within the Marvell-specific
>> root-complex driver.
>>
>> The root-complex driver presumably needs associate non-compliant
>> register blocks with specific child nodes.  That can be done by
>> requiring that the reg entries are in the same order as the config-space
>> ranges entries.
>>
>> Anticipating the possible objection that ARM's 0x1000-byte page size
>> does not permit virtual-to-physical mapping at 0x800-byte granularity:
>> The device tree does not guarantee that reg entries are page-aligned; it
>> simply tries to describe the reality, even though it might be messy.
>>
>>>
>>>> If that is indeed the case, them I would vote for a slight modification
>>>> of the intermediate patch that I cited earlier - the one in which the
>>>> ranges property includes translations from those special config
>>>> addresses into CPU addresses.  The modification is to fix the sizes,
>>>> changing 0x2000 to 0x800, so those ranges entries do not overlap in the
>>>> child address domain.
>>
>> BTW I have completely changed my mind about the overlap thing.
>>
>> I said that it was bogus to use size=0x2000 for a config space header.
>>  That was based on an interpretation - which I now dislike - of the
>> meaning of 3-cell config addresses.  By that old interpretation,
>> size=0x800 would also be bogus, because bits 10-8 of the config address
>> are for the function number.
>>
>> Consider the following question, which I have never previously
>> considered, at least not explicitly:
>>
>> Q: What would be the 3-cell representation of the Command/Status
>> register address (offset 4) in device 1, function 1?
>>
>> One obvious - but weak - answer is  <0x00000904 0 0> .  It's obvious
>> because the value 0x80000904 is what you put in the cf8 indirect access
>> address register.  But it is weak because it unnecessarily restricts the
>> config header size, and because it works differently than offsets
>> applied to memory and I/O space addresses.  Furthermore, coupling the
>> offset to the cf8 register is dodgy because of the funniness where you
>> have to but the byte-selector bits 1..0 not in the address register cf8,
>> but rather in the data register cfc.
>>
>> The better answer is  <0x00000900 0 4>, using the third cell for the
>> offset, as with IO and memory space offsets.
>>
>> Under this better interpretation, config space sizes larger than 0x100
>> are no problem.  The first cell is just a selector.  The size is "added"
>> to the third cell, so it does not encroach into the first cell's
>> device,function bits.
>>
>> This better interpretation easily handles PCIe's 0x1000-byte extended
>> config headers, and trivially accomodates even larger sizes should a
>> future PCI variant further increase the header size. as described way
>> down below.
> 
> That makes a lot of sense. It also mirrors parts of a discussion we
> previously had when we first discussed a DT binding for Tegra PCIe.
> 
> Thanks for taking the time to go over this in so much detail. I very
> much appreciate it.
> 
>>>> 	#address-cells = <3>;
>>>> 	#size-cells = <2>;
>>>>
>>>> 	bus-range = <0x00 0xff>;
>>>>
>>>> 	ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root
>>>> port config header */
>>>> 	          0x00001000 0 0          0xd4008000 0 0x00000800   /* Root
>>>> port config header */
>>>> 		  0x82000000 0 0xe0000000 0xe0000000 0 0x08000000   /*
>>>> non-prefetchable memory */
>>>> 	          0x81000000 0 0          0xe8000000 0 0x00100000>; /*
>>>> downstream I/O */
>>>>
>>>> 	pcie at 1,0 {
>>>> 		device_type = "pci";
>>>> 		reg = <0x0800 0 0 0 0>;
>>>
>>> Okay, this was Thierry's original idea, and it was my note that this
>>> didn't seem like a good choice. I'm not wedded to that, but I'll
>>> explain my reasoning a bit.
>>>
>>> Two things bothered me
>>>   - Describing a CPU MMIO mapping with a config address space seems
>>>     wonky
>>
>> I agree that mapping config space is sort of a jarring concept, but I
>> think that's because PCs have polluted the mindspace, not because there
>> is anything inherently bad about it.  The original PCI spec
>> fundamentally treated config space as just another (rather segmented)
>> linear address space.  It specified the cf8/cfc indirect access
>> mechanism not as a deep semantic feature, but rather as an
>> implementation hack to work around addressing limitations of PC chipsets
>> and the addressing-deficient popular OS of the time (Windows 3.1 which
>> was a veneer over DOS).  The PCI spec defined configuration mechanism #2
>> as a direct map into PC I/O space.  RISC chipsets of that era (e.g.
>> Alpha, Power PC) often direct-mapped config space into some form of
>> load/store space.
>>
>> But PCs soon encroached on the RISC market and RISC system vendors
>> started to "leverage" PC I/O chipsets.  The indirect access "config
>> mechanism #1" became part of the mindset.  Indirect access was hardcoded
>> in enough software that chipset designers would provide indirect access
>> even if direct-mapping was possible.
>>
>> Fundamentally, a config header is just a block of registers, addressable
>> as an index relative to some base.  It might require some specific
>> access path - but that is true with any block of registers.  Even
>> direct-mapped MMIO registers often require specific semantics such as
>> non-cached, non-write-buffered, or special address space ids.
>>
>> The PCI SIG expected PCIe extended config space to be direct-mapped -
>> see slide 25 of
>>
>> http://www.pcisig.com/specifications/pciexpress/technical_library/dev_con_09_02/specifications/pciexpress/PCIExpress_SoftwareandConfigurationModel.pdf
>>
>> So mappability of config space was intended from the get-go.  The real
>> wonky thing is the "indirect access zombie that cannot die".
>>
>>>   - Linux's OF core doesn't parse a 'reg' property under
>>>     'device_type = "pci"' as something CPU mappable, it only looks
>>>     to assigned-addresses for that. So the OF core would need to learn
>>>     how to handle this case, and it would have to be well defined..
>>
>> The way I envisioned it working is that the root-complex node defines
>> its own config space access functions (the "ops" argument to
>> pci_scan_root_bus).  For config addresses listed in ranges, the
>> functions do MMIO.  For others, they use the chipset's indirect-access
>> registers.  The OF core is not involved.
>>
>>>
>>>     Thierry's original patch avoided this problem by not using
>>>     device_type = "pci"..
> 
> I think the OF core still needs to be involved in order to translate the
> reg = <0x0800 0 0 0 0>; entry into a resource that can be ioremap()'ed.
> As Jason already said, the OF core only does that for assigned-addresses
> when device_type = "pci".
> 
> Otherwise the pci_ops implementation still couldn't access the MMIO
> registers. 

I'm not sure I understand your point that <0x0800 ...> needs to be
translated into a resource.

(a) Existing code that needs to match a child nodes to a discovered PCI
hardware device uses the existing of_pci_find_child_device(parent,
devfn) from of_pci.c .  That routine "just works", because the reg =
<0x0800 ..> entry is standard.

(b) The discovery/enumeration code needs to access config space via
pci_ops.  The root complex driver implements pci_ops based on a trivial
parsing of ranges (omitting irrelevant details):

pci_op_read(devfn, pos) {
   loop_over_ranges_entries {
      if (to_devfn(ranges_entry.child) == devfn) {
         return mmio_read(ranges_entry.parent + pos);
   }
   return indirect_read(devfn, pos);
}

The actual implementation might setup different data structures, but the
pseudocode captures the essential points:  The input argument is devfn,
and a simple scan of ranges controls whether to use MMIO or indirect
access for a given devfn value.  Every aspect of the scanning loop is
trivial.  The loop itself is just a stride-5 walk over an array of integers.

"ranges_entry.parent" is be32_to_cpu(ranges_entry[3])

and

"to_devfn(ranges_entry.child)" is "be32_to_cpup(ranges_entry[0]) >> 8"

(you don't want to mask the result in this case because not masking
prevents non-config-space ranges entries from matching devfn.)

> Aiming for a driver-specific solution doesn't seem like a
> good idea but if the functionality is common enough to be required by
> two or more drivers perhaps a new helper could be created for exactly
> this purpose.

Indeed - but it is often best to wait and see rather than guessing about
what sort of common helper is needed.  I usually guess wrong in such cases.

> 
> Perhaps another alternative would be to extend the OF core to translate
> the entries in the reg property as well. Or maybe I misunderstood what
> you said.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11  5:46                                                     ` Mitch Bradley
@ 2013-03-11 18:15                                                       ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-11 18:15 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:

> So it seems that we are faced with two requirements that are somewhat at
> odds with one another.
> 
> 1) Some of the root port bridge registers have to be accessed via config
> space access functions so common PCI enumeration code will work.  To
> represent this with the usual DT structure, the top root-complex needs
> to define a 3/2 address space so its children
> can have standard PCI reg properties.  Presumably, if those registers
> are being programmed by common code, Marvell-specific code would
> restrict its role to setting up config-space access functions, leaving
> the actual touching of the registers to the common code.
> 
> 2) Marvell chips have additional non-standard per-root-port registers
> that generic PCI code would not understand.  These registers would be
> touched only by Marvell-specific code.
> 
> The two kinds of registers are adjacent in MMIO space.  However, unless
> I am misunderstanding this MV78230 manual, the highest "config header"
> register index is 0x134, well below the 0x1000 size limit of a PCIe
> config header.  Some of the extra registers are at 0x8xx, and others are
> above 0x1800.

The register block you are looking at is for the cores 'target end
port mode', and the MMIO version is for internal use only, to allow
the CPU to configure RO bits, and other tasks. The target core will
allow access to that space via config cycles, either internally
(through the indirection register) or externally (from the off-chip
root complex) generated.

However - the driver runs the core in a 'root port bridge mode' where
the config header register block you are looking at is inhibited. The
Marvell IP block requires software support to run in bridge mode. So
Marvell really has only (2), while Tegra has only (1).

> For requirement (2), the top node has a reg property listing the
> portions of the address space that are consumed by the driver at the top
> level instead of being passed through to the PCI addressing domain.  E.g.
> 
>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;

Okay, so this is agreeing with what Thomas already has:

http://www.spinics.net/lists/arm-kernel/msg228749.html

With your two notes:
 - Correct the pcie@x,y to match the OF spec
 - Add a 'device_type = "pci"' to the pcie-controller node

Is that right?
 
> I said that it was bogus to use size=0x2000 for a config space header.
>  That was based on an interpretation - which I now dislike - of the
> meaning of 3-cell config addresses.  By that old interpretation,
> size=0x800 would also be bogus, because bits 10-8 of the config address
> are for the function number.

Hum, I thought the spec was pretty clear on this point:

 00 denotes Configuration Space, in which case:
  [..]
     bbbbbbbb,ddddd,fff,rrrrrrrrr is the Configuration Space address
     hh...hh,ll...ll must be zero

And also the text at 2.1.4.4..

So you get an 8 bit register offset, placed in the highest DW..

Can you read things another way?

Is there an updated spec that supports PCI-E extended config space?

> Consider the following question, which I have never previously
> considered, at least not explicitly:
>
> Q: What would be the 3-cell representation of the Command/Status
> register address (offset 4) in device 1, function 1?

Well, see section 11.1.2, where it provides an example
for the 'Expansion ROM base address register (0x30)' as being:

   02xxxx30 00000000 00000000 

Also the f-code bindings say:

 The data type 'config-addr' refers to the 'phys.hi' cell of the
 numerical representation of a Configuration Space address.

So there is an strong convention to use the 'r' bits as the
offset..

Further, review section 12 about how ranges should be treated - it
specifically says that the b,d,f bits in ranges should be 0, and the
child address should have those bits masked prior to searching the
ranges.

Section 12 would seem to forbid this:

      ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
                0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */

Are you reading that differently?

> > Two things bothered me
> >   - Describing a CPU MMIO mapping with a config address space seems
> >     wonky
> 
> I agree that mapping config space is sort of a jarring concept, but I
> think that's because PCs have polluted the mindspace, not because
> there

I'm well aware of MMIO config space acces, that isn't so jarring. What
is so jarring is the idea that the OF translation would work like:

<0x00000900 0 0> -> 0xd4004000
<0x00000901 0 0> -> 0xd4004001

Which is completely unlike any ranges translation I've ever seen.

Basically, I look at how the register offset is encoded in the higest
dword plus the statement in section 12, and and conclude the there
wasn't an intention to model a memory map'd config space through OF.

What am I missing here?

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 18:15                                                       ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-11 18:15 UTC (permalink / raw)
  To: linux-arm-kernel

On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:

> So it seems that we are faced with two requirements that are somewhat at
> odds with one another.
> 
> 1) Some of the root port bridge registers have to be accessed via config
> space access functions so common PCI enumeration code will work.  To
> represent this with the usual DT structure, the top root-complex needs
> to define a 3/2 address space so its children
> can have standard PCI reg properties.  Presumably, if those registers
> are being programmed by common code, Marvell-specific code would
> restrict its role to setting up config-space access functions, leaving
> the actual touching of the registers to the common code.
> 
> 2) Marvell chips have additional non-standard per-root-port registers
> that generic PCI code would not understand.  These registers would be
> touched only by Marvell-specific code.
> 
> The two kinds of registers are adjacent in MMIO space.  However, unless
> I am misunderstanding this MV78230 manual, the highest "config header"
> register index is 0x134, well below the 0x1000 size limit of a PCIe
> config header.  Some of the extra registers are at 0x8xx, and others are
> above 0x1800.

The register block you are looking at is for the cores 'target end
port mode', and the MMIO version is for internal use only, to allow
the CPU to configure RO bits, and other tasks. The target core will
allow access to that space via config cycles, either internally
(through the indirection register) or externally (from the off-chip
root complex) generated.

However - the driver runs the core in a 'root port bridge mode' where
the config header register block you are looking at is inhibited. The
Marvell IP block requires software support to run in bridge mode. So
Marvell really has only (2), while Tegra has only (1).

> For requirement (2), the top node has a reg property listing the
> portions of the address space that are consumed by the driver at the top
> level instead of being passed through to the PCI addressing domain.  E.g.
> 
>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;

Okay, so this is agreeing with what Thomas already has:

http://www.spinics.net/lists/arm-kernel/msg228749.html

With your two notes:
 - Correct the pcie at x,y to match the OF spec
 - Add a 'device_type = "pci"' to the pcie-controller node

Is that right?
 
> I said that it was bogus to use size=0x2000 for a config space header.
>  That was based on an interpretation - which I now dislike - of the
> meaning of 3-cell config addresses.  By that old interpretation,
> size=0x800 would also be bogus, because bits 10-8 of the config address
> are for the function number.

Hum, I thought the spec was pretty clear on this point:

 00 denotes Configuration Space, in which case:
  [..]
     bbbbbbbb,ddddd,fff,rrrrrrrrr is the Configuration Space address
     hh...hh,ll...ll must be zero

And also the text at 2.1.4.4..

So you get an 8 bit register offset, placed in the highest DW..

Can you read things another way?

Is there an updated spec that supports PCI-E extended config space?

> Consider the following question, which I have never previously
> considered, at least not explicitly:
>
> Q: What would be the 3-cell representation of the Command/Status
> register address (offset 4) in device 1, function 1?

Well, see section 11.1.2, where it provides an example
for the 'Expansion ROM base address register (0x30)' as being:

   02xxxx30 00000000 00000000 

Also the f-code bindings say:

 The data type 'config-addr' refers to the 'phys.hi' cell of the
 numerical representation of a Configuration Space address.

So there is an strong convention to use the 'r' bits as the
offset..

Further, review section 12 about how ranges should be treated - it
specifically says that the b,d,f bits in ranges should be 0, and the
child address should have those bits masked prior to searching the
ranges.

Section 12 would seem to forbid this:

      ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
                0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */

Are you reading that differently?

> > Two things bothered me
> >   - Describing a CPU MMIO mapping with a config address space seems
> >     wonky
> 
> I agree that mapping config space is sort of a jarring concept, but I
> think that's because PCs have polluted the mindspace, not because
> there

I'm well aware of MMIO config space acces, that isn't so jarring. What
is so jarring is the idea that the OF translation would work like:

<0x00000900 0 0> -> 0xd4004000
<0x00000901 0 0> -> 0xd4004001

Which is completely unlike any ranges translation I've ever seen.

Basically, I look at how the register offset is encoded in the higest
dword plus the statement in section 12, and and conclude the there
wasn't an intention to model a memory map'd config space through OF.

What am I missing here?

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11 18:04                                                             ` Mitch Bradley
@ 2013-03-11 18:23                                                               ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-11 18:23 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

> (b) The discovery/enumeration code needs to access config space via
> pci_ops.  The root complex driver implements pci_ops based on a trivial
> parsing of ranges (omitting irrelevant details):
> 
> pci_op_read(devfn, pos) {
>    loop_over_ranges_entries {
>       if (to_devfn(ranges_entry.child) == devfn) {
>          return mmio_read(ranges_entry.parent + pos);
                            ^^^^^^^^^^^^^^^^^^^

This has to be converted through all enclosing node's ranges prior to
being used as a CPU address - the OF core has all the code to do this,
but a new entry point would be needed for this specific application...

Cheers,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 18:23                                                               ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-11 18:23 UTC (permalink / raw)
  To: linux-arm-kernel

> (b) The discovery/enumeration code needs to access config space via
> pci_ops.  The root complex driver implements pci_ops based on a trivial
> parsing of ranges (omitting irrelevant details):
> 
> pci_op_read(devfn, pos) {
>    loop_over_ranges_entries {
>       if (to_devfn(ranges_entry.child) == devfn) {
>          return mmio_read(ranges_entry.parent + pos);
                            ^^^^^^^^^^^^^^^^^^^

This has to be converted through all enclosing node's ranges prior to
being used as a CPU address - the OF core has all the code to do this,
but a new entry point would be needed for this specific application...

Cheers,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11 18:23                                                               ` Jason Gunthorpe
  (?)
@ 2013-03-11 19:49                                                                   ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 19:49 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/11/2013 8:23 AM, Jason Gunthorpe wrote:
>> (b) The discovery/enumeration code needs to access config space via
>> pci_ops.  The root complex driver implements pci_ops based on a trivial
>> parsing of ranges (omitting irrelevant details):
>>
>> pci_op_read(devfn, pos) {
>>    loop_over_ranges_entries {
>>       if (to_devfn(ranges_entry.child) == devfn) {
>>          return mmio_read(ranges_entry.parent + pos);
>                             ^^^^^^^^^^^^^^^^^^^
> 
> This has to be converted through all enclosing node's ranges prior to
> being used as a CPU address - the OF core has all the code to do this,
> but a new entry point would be needed for this specific application...

Agreed.  But I was considering that detail to be outside the scope of
this discussion because the same thing must happen for any physical
address at this level of the tree.

> 
> Cheers,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 19:49                                                                   ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 19:49 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/11/2013 8:23 AM, Jason Gunthorpe wrote:
>> (b) The discovery/enumeration code needs to access config space via
>> pci_ops.  The root complex driver implements pci_ops based on a trivial
>> parsing of ranges (omitting irrelevant details):
>>
>> pci_op_read(devfn, pos) {
>>    loop_over_ranges_entries {
>>       if (to_devfn(ranges_entry.child) == devfn) {
>>          return mmio_read(ranges_entry.parent + pos);
>                             ^^^^^^^^^^^^^^^^^^^
> 
> This has to be converted through all enclosing node's ranges prior to
> being used as a CPU address - the OF core has all the code to do this,
> but a new entry point would be needed for this specific application...

Agreed.  But I was considering that detail to be outside the scope of
this discussion because the same thing must happen for any physical
address at this level of the tree.

> 
> Cheers,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 19:49                                                                   ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 19:49 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/11/2013 8:23 AM, Jason Gunthorpe wrote:
>> (b) The discovery/enumeration code needs to access config space via
>> pci_ops.  The root complex driver implements pci_ops based on a trivial
>> parsing of ranges (omitting irrelevant details):
>>
>> pci_op_read(devfn, pos) {
>>    loop_over_ranges_entries {
>>       if (to_devfn(ranges_entry.child) == devfn) {
>>          return mmio_read(ranges_entry.parent + pos);
>                             ^^^^^^^^^^^^^^^^^^^
> 
> This has to be converted through all enclosing node's ranges prior to
> being used as a CPU address - the OF core has all the code to do this,
> but a new entry point would be needed for this specific application...

Agreed.  But I was considering that detail to be outside the scope of
this discussion because the same thing must happen for any physical
address at this level of the tree.

> 
> Cheers,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11 18:15                                                       ` Jason Gunthorpe
  (?)
@ 2013-03-11 21:50                                                           ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 21:50 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/11/2013 8:15 AM, Jason Gunthorpe wrote:
> On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
> 
>> So it seems that we are faced with two requirements that are somewhat at
>> odds with one another.
>>
>> 1) Some of the root port bridge registers have to be accessed via config
>> space access functions so common PCI enumeration code will work.  To
>> represent this with the usual DT structure, the top root-complex needs
>> to define a 3/2 address space so its children
>> can have standard PCI reg properties.  Presumably, if those registers
>> are being programmed by common code, Marvell-specific code would
>> restrict its role to setting up config-space access functions, leaving
>> the actual touching of the registers to the common code.
>>
>> 2) Marvell chips have additional non-standard per-root-port registers
>> that generic PCI code would not understand.  These registers would be
>> touched only by Marvell-specific code.
>>
>> The two kinds of registers are adjacent in MMIO space.  However, unless
>> I am misunderstanding this MV78230 manual, the highest "config header"
>> register index is 0x134, well below the 0x1000 size limit of a PCIe
>> config header.  Some of the extra registers are at 0x8xx, and others are
>> above 0x1800.
> 
> The register block you are looking at is for the cores 'target end
> port mode', and the MMIO version is for internal use only, to allow
> the CPU to configure RO bits, and other tasks. The target core will
> allow access to that space via config cycles, either internally
> (through the indirection register) or externally (from the off-chip
> root complex) generated.
> 
> However - the driver runs the core in a 'root port bridge mode' where
> the config header register block you are looking at is inhibited. The
> Marvell IP block requires software support to run in bridge mode. So
> Marvell really has only (2), while Tegra has only (1).

Interesting.

For Marvell, if Marvell has only (2), does that imply that standard PCI
discovery and enumeration code cannot find the root port bridges, and
also there is no way to auto-configure the bridges with common pci-pci
bridge code?

> 
>> For requirement (2), the top node has a reg property listing the
>> portions of the address space that are consumed by the driver at the top
>> level instead of being passed through to the PCI addressing domain.  E.g.
>>
>>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
> 
> Okay, so this is agreeing with what Thomas already has:
> 
> http://www.spinics.net/lists/arm-kernel/msg228749.html
> 
> With your two notes:
>  - Correct the pcie@x,y to match the OF spec
>  - Add a 'device_type = "pci"' to the pcie-controller node
> 
> Is that right?

At this point I am confused again.  There was talk of the need to use
standard PCI enumeration code to deal with the bridges, thus the need
for 3/2 to describe the interface between root-complex and the child
root-port nodes.  But now I think I'm hearing that these particular root
ports bear no resemblance to standard pcie bridges.  So I am back to
"not sure why we are using 3/2 PCI addresses across an interface
boundary that has no relation to PCI addressing".  Sorry if I'm being
obtuse.

For Tegra, I think it all makes sense, and I still like the ranges
representation to control the config-space spoofing.  But for Marvell,
it seems like there is neither the need for nor the possibility of
spoofing, nor for PCI-style addressing at the complex/port boundary.

>  
>> I said that it was bogus to use size=0x2000 for a config space header.
>>  That was based on an interpretation - which I now dislike - of the
>> meaning of 3-cell config addresses.  By that old interpretation,
>> size=0x800 would also be bogus, because bits 10-8 of the config address
>> are for the function number.
> 
> Hum, I thought the spec was pretty clear on this point:
> 
>  00 denotes Configuration Space, in which case:
>   [..]
>      bbbbbbbb,ddddd,fff,rrrrrrrrr is the Configuration Space address
>      hh...hh,ll...ll must be zero
> 
> And also the text at 2.1.4.4..
> 
> So you get an 8 bit register offset, placed in the highest DW..
> 
> Can you read things another way?

The reading is clear, but there is a crucial point that is missing.

I don't recall ever having seen anything but 0 in rrrrrrrr.  PCI reg
properties always list the base address of config header.  There is
never a need for a reg entry to point into the middle of a config
header.  The granularity is always at the dev,function level.  So the
fact that the spec puts the offset in those bits is moot, because the
offset is never used.  (I will address your counterargument below.)

I didn't anticipate that fact when I developed the representation, nor
did I anticipate that the rather-rigidly structured config space would
be extended years later.

With my "new interpretation", you still never actually set any of the
"offset bits".  The only thing you get from it is permission to use
larger size values without creating algebraic nonsense.  You never
actually do arithmetic on that representation.

> 
> Is there an updated spec that supports PCI-E extended config space?

http://www.openfirmware.org/ofwg/bindings/pci/pci-express.txt

It puts the extended config address bits in 27:24 of the high word - but
that said, I doubt that those bits have ever been set to nonzero.  Again
because you never need to point to an individual register using this
representation.  I will call David Kahn, who lives down the street from
me, later today and see what he thinks.


> 
>> Consider the following question, which I have never previously
>> considered, at least not explicitly:
>>
>> Q: What would be the 3-cell representation of the Command/Status
>> register address (offset 4) in device 1, function 1?
> 
> Well, see section 11.1.2, where it provides an example
> for the 'Expansion ROM base address register (0x30)' as being:
> 
>    02xxxx30 00000000 00000000 
> 
> Also the f-code bindings say:
> 
>  The data type 'config-addr' refers to the 'phys.hi' cell of the
>  numerical representation of a Configuration Space address.
> 
> So there is an strong convention to use the 'r' bits as the
> offset..

You're right, the spec does indeed say that.  But in practice, it never
actually happened that way.  Expansion ROMs, in practice, had to be
copied into memory because the tended to disappear under certain
settings of various enable bits.  And even that copying had to be done
very carefully, because expansion ROM access had an annoying tendency to
hang some bus interfaces.

So, yeah, the spec implies the convention, but I don't think it was ever
used.

> 
> Further, review section 12 about how ranges should be treated - it
> specifically says that the b,d,f bits in ranges should be 0, and the
> child address should have those bits masked prior to searching the
> ranges.
> 
> Section 12 would seem to forbid this:
> 
>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
> 
> Are you reading that differently?

I wasn't reading it at all, until you pointed it out to me :-)  I was
just reasoning based on my mental model.

Again, I agree with your reading of the spec, but I can't think of any
reason why relaxing that restriction to permit config space mappings
would be a problem.

> 
>>> Two things bothered me
>>>   - Describing a CPU MMIO mapping with a config address space seems
>>>     wonky
>>
>> I agree that mapping config space is sort of a jarring concept, but I
>> think that's because PCs have polluted the mindspace, not because
>> there
> 
> I'm well aware of MMIO config space acces, that isn't so jarring. What
> is so jarring is the idea that the OF translation would work like:
> 
> <0x00000900 0 0> -> 0xd4004000
> <0x00000901 0 0> -> 0xd4004001
> 
> Which is completely unlike any ranges translation I've ever seen.

You never actually say <0x901 0 0> in the DT.  What you say "there is a
config header based at <0x900 0 0>".  The offset is maintained
separately, not added directly to the DT representation of the config
header base address.

The most useful interpretation of a ranges entry is that the tuples
(child_base,offset) and (parent_base,offset) correspond for 0 <= offset
< size.  In some cases, you might be able to use arithmetic to merge
base and offset; in other cases you can't.

> 
> Basically, I look at how the register offset is encoded in the higest
> dword plus the statement in section 12, and and conclude the there
> wasn't an intention to model a memory map'd config space through OF.
> 
> What am I missing here?

The real point of section 12 is that you have to attend to the fact that
PCI a structured address space, not just a single linear space.

As you say, the text does not convey the intention of mapping config
space, but I can say categorically that I certainly did not intent "thou
shalt not map config space".


> 
> Regards,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 21:50                                                           ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 21:50 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/11/2013 8:15 AM, Jason Gunthorpe wrote:
> On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
> 
>> So it seems that we are faced with two requirements that are somewhat at
>> odds with one another.
>>
>> 1) Some of the root port bridge registers have to be accessed via config
>> space access functions so common PCI enumeration code will work.  To
>> represent this with the usual DT structure, the top root-complex needs
>> to define a 3/2 address space so its children
>> can have standard PCI reg properties.  Presumably, if those registers
>> are being programmed by common code, Marvell-specific code would
>> restrict its role to setting up config-space access functions, leaving
>> the actual touching of the registers to the common code.
>>
>> 2) Marvell chips have additional non-standard per-root-port registers
>> that generic PCI code would not understand.  These registers would be
>> touched only by Marvell-specific code.
>>
>> The two kinds of registers are adjacent in MMIO space.  However, unless
>> I am misunderstanding this MV78230 manual, the highest "config header"
>> register index is 0x134, well below the 0x1000 size limit of a PCIe
>> config header.  Some of the extra registers are at 0x8xx, and others are
>> above 0x1800.
> 
> The register block you are looking at is for the cores 'target end
> port mode', and the MMIO version is for internal use only, to allow
> the CPU to configure RO bits, and other tasks. The target core will
> allow access to that space via config cycles, either internally
> (through the indirection register) or externally (from the off-chip
> root complex) generated.
> 
> However - the driver runs the core in a 'root port bridge mode' where
> the config header register block you are looking at is inhibited. The
> Marvell IP block requires software support to run in bridge mode. So
> Marvell really has only (2), while Tegra has only (1).

Interesting.

For Marvell, if Marvell has only (2), does that imply that standard PCI
discovery and enumeration code cannot find the root port bridges, and
also there is no way to auto-configure the bridges with common pci-pci
bridge code?

> 
>> For requirement (2), the top node has a reg property listing the
>> portions of the address space that are consumed by the driver at the top
>> level instead of being passed through to the PCI addressing domain.  E.g.
>>
>>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
> 
> Okay, so this is agreeing with what Thomas already has:
> 
> http://www.spinics.net/lists/arm-kernel/msg228749.html
> 
> With your two notes:
>  - Correct the pcie@x,y to match the OF spec
>  - Add a 'device_type = "pci"' to the pcie-controller node
> 
> Is that right?

At this point I am confused again.  There was talk of the need to use
standard PCI enumeration code to deal with the bridges, thus the need
for 3/2 to describe the interface between root-complex and the child
root-port nodes.  But now I think I'm hearing that these particular root
ports bear no resemblance to standard pcie bridges.  So I am back to
"not sure why we are using 3/2 PCI addresses across an interface
boundary that has no relation to PCI addressing".  Sorry if I'm being
obtuse.

For Tegra, I think it all makes sense, and I still like the ranges
representation to control the config-space spoofing.  But for Marvell,
it seems like there is neither the need for nor the possibility of
spoofing, nor for PCI-style addressing at the complex/port boundary.

>  
>> I said that it was bogus to use size=0x2000 for a config space header.
>>  That was based on an interpretation - which I now dislike - of the
>> meaning of 3-cell config addresses.  By that old interpretation,
>> size=0x800 would also be bogus, because bits 10-8 of the config address
>> are for the function number.
> 
> Hum, I thought the spec was pretty clear on this point:
> 
>  00 denotes Configuration Space, in which case:
>   [..]
>      bbbbbbbb,ddddd,fff,rrrrrrrrr is the Configuration Space address
>      hh...hh,ll...ll must be zero
> 
> And also the text at 2.1.4.4..
> 
> So you get an 8 bit register offset, placed in the highest DW..
> 
> Can you read things another way?

The reading is clear, but there is a crucial point that is missing.

I don't recall ever having seen anything but 0 in rrrrrrrr.  PCI reg
properties always list the base address of config header.  There is
never a need for a reg entry to point into the middle of a config
header.  The granularity is always at the dev,function level.  So the
fact that the spec puts the offset in those bits is moot, because the
offset is never used.  (I will address your counterargument below.)

I didn't anticipate that fact when I developed the representation, nor
did I anticipate that the rather-rigidly structured config space would
be extended years later.

With my "new interpretation", you still never actually set any of the
"offset bits".  The only thing you get from it is permission to use
larger size values without creating algebraic nonsense.  You never
actually do arithmetic on that representation.

> 
> Is there an updated spec that supports PCI-E extended config space?

http://www.openfirmware.org/ofwg/bindings/pci/pci-express.txt

It puts the extended config address bits in 27:24 of the high word - but
that said, I doubt that those bits have ever been set to nonzero.  Again
because you never need to point to an individual register using this
representation.  I will call David Kahn, who lives down the street from
me, later today and see what he thinks.


> 
>> Consider the following question, which I have never previously
>> considered, at least not explicitly:
>>
>> Q: What would be the 3-cell representation of the Command/Status
>> register address (offset 4) in device 1, function 1?
> 
> Well, see section 11.1.2, where it provides an example
> for the 'Expansion ROM base address register (0x30)' as being:
> 
>    02xxxx30 00000000 00000000 
> 
> Also the f-code bindings say:
> 
>  The data type 'config-addr' refers to the 'phys.hi' cell of the
>  numerical representation of a Configuration Space address.
> 
> So there is an strong convention to use the 'r' bits as the
> offset..

You're right, the spec does indeed say that.  But in practice, it never
actually happened that way.  Expansion ROMs, in practice, had to be
copied into memory because the tended to disappear under certain
settings of various enable bits.  And even that copying had to be done
very carefully, because expansion ROM access had an annoying tendency to
hang some bus interfaces.

So, yeah, the spec implies the convention, but I don't think it was ever
used.

> 
> Further, review section 12 about how ranges should be treated - it
> specifically says that the b,d,f bits in ranges should be 0, and the
> child address should have those bits masked prior to searching the
> ranges.
> 
> Section 12 would seem to forbid this:
> 
>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
> 
> Are you reading that differently?

I wasn't reading it at all, until you pointed it out to me :-)  I was
just reasoning based on my mental model.

Again, I agree with your reading of the spec, but I can't think of any
reason why relaxing that restriction to permit config space mappings
would be a problem.

> 
>>> Two things bothered me
>>>   - Describing a CPU MMIO mapping with a config address space seems
>>>     wonky
>>
>> I agree that mapping config space is sort of a jarring concept, but I
>> think that's because PCs have polluted the mindspace, not because
>> there
> 
> I'm well aware of MMIO config space acces, that isn't so jarring. What
> is so jarring is the idea that the OF translation would work like:
> 
> <0x00000900 0 0> -> 0xd4004000
> <0x00000901 0 0> -> 0xd4004001
> 
> Which is completely unlike any ranges translation I've ever seen.

You never actually say <0x901 0 0> in the DT.  What you say "there is a
config header based at <0x900 0 0>".  The offset is maintained
separately, not added directly to the DT representation of the config
header base address.

The most useful interpretation of a ranges entry is that the tuples
(child_base,offset) and (parent_base,offset) correspond for 0 <= offset
< size.  In some cases, you might be able to use arithmetic to merge
base and offset; in other cases you can't.

> 
> Basically, I look at how the register offset is encoded in the higest
> dword plus the statement in section 12, and and conclude the there
> wasn't an intention to model a memory map'd config space through OF.
> 
> What am I missing here?

The real point of section 12 is that you have to attend to the fact that
PCI a structured address space, not just a single linear space.

As you say, the text does not convey the intention of mapping config
space, but I can say categorically that I certainly did not intent "thou
shalt not map config space".


> 
> Regards,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 21:50                                                           ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 21:50 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/11/2013 8:15 AM, Jason Gunthorpe wrote:
> On Sun, Mar 10, 2013 at 07:46:04PM -1000, Mitch Bradley wrote:
> 
>> So it seems that we are faced with two requirements that are somewhat at
>> odds with one another.
>>
>> 1) Some of the root port bridge registers have to be accessed via config
>> space access functions so common PCI enumeration code will work.  To
>> represent this with the usual DT structure, the top root-complex needs
>> to define a 3/2 address space so its children
>> can have standard PCI reg properties.  Presumably, if those registers
>> are being programmed by common code, Marvell-specific code would
>> restrict its role to setting up config-space access functions, leaving
>> the actual touching of the registers to the common code.
>>
>> 2) Marvell chips have additional non-standard per-root-port registers
>> that generic PCI code would not understand.  These registers would be
>> touched only by Marvell-specific code.
>>
>> The two kinds of registers are adjacent in MMIO space.  However, unless
>> I am misunderstanding this MV78230 manual, the highest "config header"
>> register index is 0x134, well below the 0x1000 size limit of a PCIe
>> config header.  Some of the extra registers are at 0x8xx, and others are
>> above 0x1800.
> 
> The register block you are looking at is for the cores 'target end
> port mode', and the MMIO version is for internal use only, to allow
> the CPU to configure RO bits, and other tasks. The target core will
> allow access to that space via config cycles, either internally
> (through the indirection register) or externally (from the off-chip
> root complex) generated.
> 
> However - the driver runs the core in a 'root port bridge mode' where
> the config header register block you are looking at is inhibited. The
> Marvell IP block requires software support to run in bridge mode. So
> Marvell really has only (2), while Tegra has only (1).

Interesting.

For Marvell, if Marvell has only (2), does that imply that standard PCI
discovery and enumeration code cannot find the root port bridges, and
also there is no way to auto-configure the bridges with common pci-pci
bridge code?

> 
>> For requirement (2), the top node has a reg property listing the
>> portions of the address space that are consumed by the driver at the top
>> level instead of being passed through to the PCI addressing domain.  E.g.
>>
>>   reg = <0xd0040800 0x1800>, <0xd0080800 0x1800>;
> 
> Okay, so this is agreeing with what Thomas already has:
> 
> http://www.spinics.net/lists/arm-kernel/msg228749.html
> 
> With your two notes:
>  - Correct the pcie at x,y to match the OF spec
>  - Add a 'device_type = "pci"' to the pcie-controller node
> 
> Is that right?

At this point I am confused again.  There was talk of the need to use
standard PCI enumeration code to deal with the bridges, thus the need
for 3/2 to describe the interface between root-complex and the child
root-port nodes.  But now I think I'm hearing that these particular root
ports bear no resemblance to standard pcie bridges.  So I am back to
"not sure why we are using 3/2 PCI addresses across an interface
boundary that has no relation to PCI addressing".  Sorry if I'm being
obtuse.

For Tegra, I think it all makes sense, and I still like the ranges
representation to control the config-space spoofing.  But for Marvell,
it seems like there is neither the need for nor the possibility of
spoofing, nor for PCI-style addressing at the complex/port boundary.

>  
>> I said that it was bogus to use size=0x2000 for a config space header.
>>  That was based on an interpretation - which I now dislike - of the
>> meaning of 3-cell config addresses.  By that old interpretation,
>> size=0x800 would also be bogus, because bits 10-8 of the config address
>> are for the function number.
> 
> Hum, I thought the spec was pretty clear on this point:
> 
>  00 denotes Configuration Space, in which case:
>   [..]
>      bbbbbbbb,ddddd,fff,rrrrrrrrr is the Configuration Space address
>      hh...hh,ll...ll must be zero
> 
> And also the text at 2.1.4.4..
> 
> So you get an 8 bit register offset, placed in the highest DW..
> 
> Can you read things another way?

The reading is clear, but there is a crucial point that is missing.

I don't recall ever having seen anything but 0 in rrrrrrrr.  PCI reg
properties always list the base address of config header.  There is
never a need for a reg entry to point into the middle of a config
header.  The granularity is always at the dev,function level.  So the
fact that the spec puts the offset in those bits is moot, because the
offset is never used.  (I will address your counterargument below.)

I didn't anticipate that fact when I developed the representation, nor
did I anticipate that the rather-rigidly structured config space would
be extended years later.

With my "new interpretation", you still never actually set any of the
"offset bits".  The only thing you get from it is permission to use
larger size values without creating algebraic nonsense.  You never
actually do arithmetic on that representation.

> 
> Is there an updated spec that supports PCI-E extended config space?

http://www.openfirmware.org/ofwg/bindings/pci/pci-express.txt

It puts the extended config address bits in 27:24 of the high word - but
that said, I doubt that those bits have ever been set to nonzero.  Again
because you never need to point to an individual register using this
representation.  I will call David Kahn, who lives down the street from
me, later today and see what he thinks.


> 
>> Consider the following question, which I have never previously
>> considered, at least not explicitly:
>>
>> Q: What would be the 3-cell representation of the Command/Status
>> register address (offset 4) in device 1, function 1?
> 
> Well, see section 11.1.2, where it provides an example
> for the 'Expansion ROM base address register (0x30)' as being:
> 
>    02xxxx30 00000000 00000000 
> 
> Also the f-code bindings say:
> 
>  The data type 'config-addr' refers to the 'phys.hi' cell of the
>  numerical representation of a Configuration Space address.
> 
> So there is an strong convention to use the 'r' bits as the
> offset..

You're right, the spec does indeed say that.  But in practice, it never
actually happened that way.  Expansion ROMs, in practice, had to be
copied into memory because the tended to disappear under certain
settings of various enable bits.  And even that copying had to be done
very carefully, because expansion ROM access had an annoying tendency to
hang some bus interfaces.

So, yeah, the spec implies the convention, but I don't think it was ever
used.

> 
> Further, review section 12 about how ranges should be treated - it
> specifically says that the b,d,f bits in ranges should be 0, and the
> child address should have those bits masked prior to searching the
> ranges.
> 
> Section 12 would seem to forbid this:
> 
>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
> 
> Are you reading that differently?

I wasn't reading it at all, until you pointed it out to me :-)  I was
just reasoning based on my mental model.

Again, I agree with your reading of the spec, but I can't think of any
reason why relaxing that restriction to permit config space mappings
would be a problem.

> 
>>> Two things bothered me
>>>   - Describing a CPU MMIO mapping with a config address space seems
>>>     wonky
>>
>> I agree that mapping config space is sort of a jarring concept, but I
>> think that's because PCs have polluted the mindspace, not because
>> there
> 
> I'm well aware of MMIO config space acces, that isn't so jarring. What
> is so jarring is the idea that the OF translation would work like:
> 
> <0x00000900 0 0> -> 0xd4004000
> <0x00000901 0 0> -> 0xd4004001
> 
> Which is completely unlike any ranges translation I've ever seen.

You never actually say <0x901 0 0> in the DT.  What you say "there is a
config header based at <0x900 0 0>".  The offset is maintained
separately, not added directly to the DT representation of the config
header base address.

The most useful interpretation of a ranges entry is that the tuples
(child_base,offset) and (parent_base,offset) correspond for 0 <= offset
< size.  In some cases, you might be able to use arithmetic to merge
base and offset; in other cases you can't.

> 
> Basically, I look at how the register offset is encoded in the higest
> dword plus the statement in section 12, and and conclude the there
> wasn't an intention to model a memory map'd config space through OF.
> 
> What am I missing here?

The real point of section 12 is that you have to attend to the fact that
PCI a structured address space, not just a single linear space.

As you say, the text does not convey the intention of mapping config
space, but I can say categorically that I certainly did not intent "thou
shalt not map config space".


> 
> Regards,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11 21:50                                                           ` Mitch Bradley
@ 2013-03-11 23:25                                                             ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-11 23:25 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Mon, Mar 11, 2013 at 11:50:19AM -1000, Mitch Bradley wrote:

> > However - the driver runs the core in a 'root port bridge mode' where
> > the config header register block you are looking at is inhibited. The
> > Marvell IP block requires software support to run in bridge mode. So
> > Marvell really has only (2), while Tegra has only (1).
> 
> Interesting.
> 
> For Marvell, if Marvell has only (2), does that imply that standard PCI
> discovery and enumeration code cannot find the root port bridges, and
> also there is no way to auto-configure the bridges with common pci-pci
> bridge code?

The driver is required to take care of all of this. From the common
code's perspective there is a bridge config header, and writing to it
controls the device, as the PCI spec intends. But the driver doesn't
copy those read/writes 1:1 to any HW register block, it distributes
them around the device's register space as necessary to create the
PCI bridge config header.

The DT is still modeling the HW, there really are root port bridge(s),
that are completely conformant at the TLP level, but the IP designers
choose to leave the detail of the bridge config space up to a SW
implementation.

> At this point I am confused again.  There was talk of the need to
> use standard PCI enumeration code to deal with the bridges, thus the
> need for 3/2 to describe the interface between root-complex and the
> child root-port nodes.

I think your confusion is coming from your expectation that the PCI-E
IP block would be an entirely self contained root port, rather than
'tools to build a root port, with the right driver SW'.

> > Further, review section 12 about how ranges should be treated - it
> > specifically says that the b,d,f bits in ranges should be 0, and the
> > child address should have those bits masked prior to searching the
> > ranges.
> > 
> > Section 12 would seem to forbid this:
> > 
> >       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
> >                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
> > 
> > Are you reading that differently?
> 
> I wasn't reading it at all, until you pointed it out to me :-)  I was
> just reasoning based on my mental model.
> 
> Again, I agree with your reading of the spec, but I can't think of any
> reason why relaxing that restriction to permit config space mappings
> would be a problem.

Okay, just wanted to confirm that, I think I follow you now. :)

So you are basically proposing a small update to Section 12 that would
read something like:

'When matching a config space address, the b,d,f bits are valid in the
 ranges and left unmasked in the child. The r bits in the ranges
 should be 0, and the r bits from the child are copied in the last DW
 and masked off in the first DW.'

Which, as you note, would not conflict with any existing usage..

Though, I can already see that the above language is not good enough,
it doesn't elegantly represent standard ECAM, for instance.. How would
you even represent ECAM with ranges??

Another wrinkle is that tegra not only has the per-bridge memory
mapped config space, bit it also has 4 'all port' ECAM-like memory
mapped config spaces. The rules around them are sufficiently
specialized that I'm not sure a ranges mapping would ever be
practical.. So right out of the gate we'd fail to capture memory
mapped config access via ranges on at least one SOC.

So, to me, this seems like overkill. Only ECAM has a hope of having a
common implementation, everything else will be host driver specific,
so why go to the trouble to revise the OF PCI specification for config
space?

Particularly since there are patches waiting on this question being
resolved :)

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 23:25                                                             ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-11 23:25 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, Mar 11, 2013 at 11:50:19AM -1000, Mitch Bradley wrote:

> > However - the driver runs the core in a 'root port bridge mode' where
> > the config header register block you are looking at is inhibited. The
> > Marvell IP block requires software support to run in bridge mode. So
> > Marvell really has only (2), while Tegra has only (1).
> 
> Interesting.
> 
> For Marvell, if Marvell has only (2), does that imply that standard PCI
> discovery and enumeration code cannot find the root port bridges, and
> also there is no way to auto-configure the bridges with common pci-pci
> bridge code?

The driver is required to take care of all of this. From the common
code's perspective there is a bridge config header, and writing to it
controls the device, as the PCI spec intends. But the driver doesn't
copy those read/writes 1:1 to any HW register block, it distributes
them around the device's register space as necessary to create the
PCI bridge config header.

The DT is still modeling the HW, there really are root port bridge(s),
that are completely conformant at the TLP level, but the IP designers
choose to leave the detail of the bridge config space up to a SW
implementation.

> At this point I am confused again.  There was talk of the need to
> use standard PCI enumeration code to deal with the bridges, thus the
> need for 3/2 to describe the interface between root-complex and the
> child root-port nodes.

I think your confusion is coming from your expectation that the PCI-E
IP block would be an entirely self contained root port, rather than
'tools to build a root port, with the right driver SW'.

> > Further, review section 12 about how ranges should be treated - it
> > specifically says that the b,d,f bits in ranges should be 0, and the
> > child address should have those bits masked prior to searching the
> > ranges.
> > 
> > Section 12 would seem to forbid this:
> > 
> >       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
> >                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
> > 
> > Are you reading that differently?
> 
> I wasn't reading it at all, until you pointed it out to me :-)  I was
> just reasoning based on my mental model.
> 
> Again, I agree with your reading of the spec, but I can't think of any
> reason why relaxing that restriction to permit config space mappings
> would be a problem.

Okay, just wanted to confirm that, I think I follow you now. :)

So you are basically proposing a small update to Section 12 that would
read something like:

'When matching a config space address, the b,d,f bits are valid in the
 ranges and left unmasked in the child. The r bits in the ranges
 should be 0, and the r bits from the child are copied in the last DW
 and masked off in the first DW.'

Which, as you note, would not conflict with any existing usage..

Though, I can already see that the above language is not good enough,
it doesn't elegantly represent standard ECAM, for instance.. How would
you even represent ECAM with ranges??

Another wrinkle is that tegra not only has the per-bridge memory
mapped config space, bit it also has 4 'all port' ECAM-like memory
mapped config spaces. The rules around them are sufficiently
specialized that I'm not sure a ranges mapping would ever be
practical.. So right out of the gate we'd fail to capture memory
mapped config access via ranges on at least one SOC.

So, to me, this seems like overkill. Only ECAM has a hope of having a
common implementation, everything else will be host driver specific,
so why go to the trouble to revise the OF PCI specification for config
space?

Particularly since there are patches waiting on this question being
resolved :)

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11 23:25                                                             ` Jason Gunthorpe
  (?)
@ 2013-03-11 23:38                                                                 ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 23:38 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r

On 3/11/2013 1:25 PM, Jason Gunthorpe wrote:
> On Mon, Mar 11, 2013 at 11:50:19AM -1000, Mitch Bradley wrote:
> 
>>> However - the driver runs the core in a 'root port bridge mode' where
>>> the config header register block you are looking at is inhibited. The
>>> Marvell IP block requires software support to run in bridge mode. So
>>> Marvell really has only (2), while Tegra has only (1).
>>
>> Interesting.
>>
>> For Marvell, if Marvell has only (2), does that imply that standard PCI
>> discovery and enumeration code cannot find the root port bridges, and
>> also there is no way to auto-configure the bridges with common pci-pci
>> bridge code?
> 
> The driver is required to take care of all of this. From the common
> code's perspective there is a bridge config header, and writing to it
> controls the device, as the PCI spec intends. But the driver doesn't
> copy those read/writes 1:1 to any HW register block, it distributes
> them around the device's register space as necessary to create the
> PCI bridge config header.
> 
> The DT is still modeling the HW, there really are root port bridge(s),
> that are completely conformant at the TLP level, but the IP designers
> choose to leave the detail of the bridge config space up to a SW
> implementation.

Oh, ouch, deep spoofing.  I had to do that in system management mode
firmware for a Geode chip.  What a pain that was.

> 
>> At this point I am confused again.  There was talk of the need to
>> use standard PCI enumeration code to deal with the bridges, thus the
>> need for 3/2 to describe the interface between root-complex and the
>> child root-port nodes.
> 
> I think your confusion is coming from your expectation that the PCI-E
> IP block would be an entirely self contained root port, rather than
> 'tools to build a root port, with the right driver SW'.
> 
>>> Further, review section 12 about how ranges should be treated - it
>>> specifically says that the b,d,f bits in ranges should be 0, and the
>>> child address should have those bits masked prior to searching the
>>> ranges.
>>>
>>> Section 12 would seem to forbid this:
>>>
>>>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
>>>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
>>>
>>> Are you reading that differently?
>>
>> I wasn't reading it at all, until you pointed it out to me :-)  I was
>> just reasoning based on my mental model.
>>
>> Again, I agree with your reading of the spec, but I can't think of any
>> reason why relaxing that restriction to permit config space mappings
>> would be a problem.
> 
> Okay, just wanted to confirm that, I think I follow you now. :)
> 
> So you are basically proposing a small update to Section 12 that would
> read something like:
> 
> 'When matching a config space address, the b,d,f bits are valid in the
>  ranges and left unmasked in the child. The r bits in the ranges
>  should be 0, and the r bits from the child are copied in the last DW
>  and masked off in the first DW.'
> 
> Which, as you note, would not conflict with any existing usage..
> 
> Though, I can already see that the above language is not good enough,
> it doesn't elegantly represent standard ECAM, for instance.. How would
> you even represent ECAM with ranges??
> 
> Another wrinkle is that tegra not only has the per-bridge memory
> mapped config space, bit it also has 4 'all port' ECAM-like memory
> mapped config spaces. The rules around them are sufficiently
> specialized that I'm not sure a ranges mapping would ever be
> practical.. So right out of the gate we'd fail to capture memory
> mapped config access via ranges on at least one SOC.
> 
> So, to me, this seems like overkill. Only ECAM has a hope of having a
> common implementation, everything else will be host driver specific,
> so why go to the trouble to revise the OF PCI specification for config
> space?
> 
> Particularly since there are patches waiting on this question being
> resolved :)

Okay, so I'll just bow out.  Do what you have to do.

> 
> Regards,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 23:38                                                                 ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 23:38 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/11/2013 1:25 PM, Jason Gunthorpe wrote:
> On Mon, Mar 11, 2013 at 11:50:19AM -1000, Mitch Bradley wrote:
> 
>>> However - the driver runs the core in a 'root port bridge mode' where
>>> the config header register block you are looking at is inhibited. The
>>> Marvell IP block requires software support to run in bridge mode. So
>>> Marvell really has only (2), while Tegra has only (1).
>>
>> Interesting.
>>
>> For Marvell, if Marvell has only (2), does that imply that standard PCI
>> discovery and enumeration code cannot find the root port bridges, and
>> also there is no way to auto-configure the bridges with common pci-pci
>> bridge code?
> 
> The driver is required to take care of all of this. From the common
> code's perspective there is a bridge config header, and writing to it
> controls the device, as the PCI spec intends. But the driver doesn't
> copy those read/writes 1:1 to any HW register block, it distributes
> them around the device's register space as necessary to create the
> PCI bridge config header.
> 
> The DT is still modeling the HW, there really are root port bridge(s),
> that are completely conformant at the TLP level, but the IP designers
> choose to leave the detail of the bridge config space up to a SW
> implementation.

Oh, ouch, deep spoofing.  I had to do that in system management mode
firmware for a Geode chip.  What a pain that was.

> 
>> At this point I am confused again.  There was talk of the need to
>> use standard PCI enumeration code to deal with the bridges, thus the
>> need for 3/2 to describe the interface between root-complex and the
>> child root-port nodes.
> 
> I think your confusion is coming from your expectation that the PCI-E
> IP block would be an entirely self contained root port, rather than
> 'tools to build a root port, with the right driver SW'.
> 
>>> Further, review section 12 about how ranges should be treated - it
>>> specifically says that the b,d,f bits in ranges should be 0, and the
>>> child address should have those bits masked prior to searching the
>>> ranges.
>>>
>>> Section 12 would seem to forbid this:
>>>
>>>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
>>>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
>>>
>>> Are you reading that differently?
>>
>> I wasn't reading it at all, until you pointed it out to me :-)  I was
>> just reasoning based on my mental model.
>>
>> Again, I agree with your reading of the spec, but I can't think of any
>> reason why relaxing that restriction to permit config space mappings
>> would be a problem.
> 
> Okay, just wanted to confirm that, I think I follow you now. :)
> 
> So you are basically proposing a small update to Section 12 that would
> read something like:
> 
> 'When matching a config space address, the b,d,f bits are valid in the
>  ranges and left unmasked in the child. The r bits in the ranges
>  should be 0, and the r bits from the child are copied in the last DW
>  and masked off in the first DW.'
> 
> Which, as you note, would not conflict with any existing usage..
> 
> Though, I can already see that the above language is not good enough,
> it doesn't elegantly represent standard ECAM, for instance.. How would
> you even represent ECAM with ranges??
> 
> Another wrinkle is that tegra not only has the per-bridge memory
> mapped config space, bit it also has 4 'all port' ECAM-like memory
> mapped config spaces. The rules around them are sufficiently
> specialized that I'm not sure a ranges mapping would ever be
> practical.. So right out of the gate we'd fail to capture memory
> mapped config access via ranges on at least one SOC.
> 
> So, to me, this seems like overkill. Only ECAM has a hope of having a
> common implementation, everything else will be host driver specific,
> so why go to the trouble to revise the OF PCI specification for config
> space?
> 
> Particularly since there are patches waiting on this question being
> resolved :)

Okay, so I'll just bow out.  Do what you have to do.

> 
> Regards,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-11 23:38                                                                 ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-11 23:38 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/11/2013 1:25 PM, Jason Gunthorpe wrote:
> On Mon, Mar 11, 2013 at 11:50:19AM -1000, Mitch Bradley wrote:
> 
>>> However - the driver runs the core in a 'root port bridge mode' where
>>> the config header register block you are looking at is inhibited. The
>>> Marvell IP block requires software support to run in bridge mode. So
>>> Marvell really has only (2), while Tegra has only (1).
>>
>> Interesting.
>>
>> For Marvell, if Marvell has only (2), does that imply that standard PCI
>> discovery and enumeration code cannot find the root port bridges, and
>> also there is no way to auto-configure the bridges with common pci-pci
>> bridge code?
> 
> The driver is required to take care of all of this. From the common
> code's perspective there is a bridge config header, and writing to it
> controls the device, as the PCI spec intends. But the driver doesn't
> copy those read/writes 1:1 to any HW register block, it distributes
> them around the device's register space as necessary to create the
> PCI bridge config header.
> 
> The DT is still modeling the HW, there really are root port bridge(s),
> that are completely conformant at the TLP level, but the IP designers
> choose to leave the detail of the bridge config space up to a SW
> implementation.

Oh, ouch, deep spoofing.  I had to do that in system management mode
firmware for a Geode chip.  What a pain that was.

> 
>> At this point I am confused again.  There was talk of the need to
>> use standard PCI enumeration code to deal with the bridges, thus the
>> need for 3/2 to describe the interface between root-complex and the
>> child root-port nodes.
> 
> I think your confusion is coming from your expectation that the PCI-E
> IP block would be an entirely self contained root port, rather than
> 'tools to build a root port, with the right driver SW'.
> 
>>> Further, review section 12 about how ranges should be treated - it
>>> specifically says that the b,d,f bits in ranges should be 0, and the
>>> child address should have those bits masked prior to searching the
>>> ranges.
>>>
>>> Section 12 would seem to forbid this:
>>>
>>>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
>>>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
>>>
>>> Are you reading that differently?
>>
>> I wasn't reading it at all, until you pointed it out to me :-)  I was
>> just reasoning based on my mental model.
>>
>> Again, I agree with your reading of the spec, but I can't think of any
>> reason why relaxing that restriction to permit config space mappings
>> would be a problem.
> 
> Okay, just wanted to confirm that, I think I follow you now. :)
> 
> So you are basically proposing a small update to Section 12 that would
> read something like:
> 
> 'When matching a config space address, the b,d,f bits are valid in the
>  ranges and left unmasked in the child. The r bits in the ranges
>  should be 0, and the r bits from the child are copied in the last DW
>  and masked off in the first DW.'
> 
> Which, as you note, would not conflict with any existing usage..
> 
> Though, I can already see that the above language is not good enough,
> it doesn't elegantly represent standard ECAM, for instance.. How would
> you even represent ECAM with ranges??
> 
> Another wrinkle is that tegra not only has the per-bridge memory
> mapped config space, bit it also has 4 'all port' ECAM-like memory
> mapped config spaces. The rules around them are sufficiently
> specialized that I'm not sure a ranges mapping would ever be
> practical.. So right out of the gate we'd fail to capture memory
> mapped config access via ranges on at least one SOC.
> 
> So, to me, this seems like overkill. Only ECAM has a hope of having a
> common implementation, everything else will be host driver specific,
> so why go to the trouble to revise the OF PCI specification for config
> space?
> 
> Particularly since there are patches waiting on this question being
> resolved :)

Okay, so I'll just bow out.  Do what you have to do.

> 
> Regards,
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-11 23:38                                                                 ` Mitch Bradley
  (?)
@ 2013-03-12  7:08                                                                     ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-12  7:08 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Jason Gunthorpe, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	Tawfik Bayouk, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r


[-- Attachment #1.1: Type: text/plain, Size: 4828 bytes --]

On Mon, Mar 11, 2013 at 01:38:38PM -1000, Mitch Bradley wrote:
> On 3/11/2013 1:25 PM, Jason Gunthorpe wrote:
> > On Mon, Mar 11, 2013 at 11:50:19AM -1000, Mitch Bradley wrote:
> > 
> >>> However - the driver runs the core in a 'root port bridge mode' where
> >>> the config header register block you are looking at is inhibited. The
> >>> Marvell IP block requires software support to run in bridge mode. So
> >>> Marvell really has only (2), while Tegra has only (1).
> >>
> >> Interesting.
> >>
> >> For Marvell, if Marvell has only (2), does that imply that standard PCI
> >> discovery and enumeration code cannot find the root port bridges, and
> >> also there is no way to auto-configure the bridges with common pci-pci
> >> bridge code?
> > 
> > The driver is required to take care of all of this. From the common
> > code's perspective there is a bridge config header, and writing to it
> > controls the device, as the PCI spec intends. But the driver doesn't
> > copy those read/writes 1:1 to any HW register block, it distributes
> > them around the device's register space as necessary to create the
> > PCI bridge config header.
> > 
> > The DT is still modeling the HW, there really are root port bridge(s),
> > that are completely conformant at the TLP level, but the IP designers
> > choose to leave the detail of the bridge config space up to a SW
> > implementation.
> 
> Oh, ouch, deep spoofing.  I had to do that in system management mode
> firmware for a Geode chip.  What a pain that was.
> 
> > 
> >> At this point I am confused again.  There was talk of the need to
> >> use standard PCI enumeration code to deal with the bridges, thus the
> >> need for 3/2 to describe the interface between root-complex and the
> >> child root-port nodes.
> > 
> > I think your confusion is coming from your expectation that the PCI-E
> > IP block would be an entirely self contained root port, rather than
> > 'tools to build a root port, with the right driver SW'.
> > 
> >>> Further, review section 12 about how ranges should be treated - it
> >>> specifically says that the b,d,f bits in ranges should be 0, and the
> >>> child address should have those bits masked prior to searching the
> >>> ranges.
> >>>
> >>> Section 12 would seem to forbid this:
> >>>
> >>>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
> >>>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
> >>>
> >>> Are you reading that differently?
> >>
> >> I wasn't reading it at all, until you pointed it out to me :-)  I was
> >> just reasoning based on my mental model.
> >>
> >> Again, I agree with your reading of the spec, but I can't think of any
> >> reason why relaxing that restriction to permit config space mappings
> >> would be a problem.
> > 
> > Okay, just wanted to confirm that, I think I follow you now. :)
> > 
> > So you are basically proposing a small update to Section 12 that would
> > read something like:
> > 
> > 'When matching a config space address, the b,d,f bits are valid in the
> >  ranges and left unmasked in the child. The r bits in the ranges
> >  should be 0, and the r bits from the child are copied in the last DW
> >  and masked off in the first DW.'
> > 
> > Which, as you note, would not conflict with any existing usage..
> > 
> > Though, I can already see that the above language is not good enough,
> > it doesn't elegantly represent standard ECAM, for instance.. How would
> > you even represent ECAM with ranges??
> > 
> > Another wrinkle is that tegra not only has the per-bridge memory
> > mapped config space, bit it also has 4 'all port' ECAM-like memory
> > mapped config spaces. The rules around them are sufficiently
> > specialized that I'm not sure a ranges mapping would ever be
> > practical.. So right out of the gate we'd fail to capture memory
> > mapped config access via ranges on at least one SOC.
> > 
> > So, to me, this seems like overkill. Only ECAM has a hope of having a
> > common implementation, everything else will be host driver specific,
> > so why go to the trouble to revise the OF PCI specification for config
> > space?
> > 
> > Particularly since there are patches waiting on this question being
> > resolved :)
> 
> Okay, so I'll just bow out.  Do what you have to do.

So to recapitulate, we agree that configuration space can be translated
through the "ranges" property. That means the only missing link is a new
function to translate not only "assigned-addresses" but also the "reg"
property for PCI devices. Is that it?

And for Marvell the non-configuration-space per-root-port registers need
to be mapped separately from the configuration space windows?

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12  7:08                                                                     ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-12  7:08 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 4828 bytes --]

On Mon, Mar 11, 2013 at 01:38:38PM -1000, Mitch Bradley wrote:
> On 3/11/2013 1:25 PM, Jason Gunthorpe wrote:
> > On Mon, Mar 11, 2013 at 11:50:19AM -1000, Mitch Bradley wrote:
> > 
> >>> However - the driver runs the core in a 'root port bridge mode' where
> >>> the config header register block you are looking at is inhibited. The
> >>> Marvell IP block requires software support to run in bridge mode. So
> >>> Marvell really has only (2), while Tegra has only (1).
> >>
> >> Interesting.
> >>
> >> For Marvell, if Marvell has only (2), does that imply that standard PCI
> >> discovery and enumeration code cannot find the root port bridges, and
> >> also there is no way to auto-configure the bridges with common pci-pci
> >> bridge code?
> > 
> > The driver is required to take care of all of this. From the common
> > code's perspective there is a bridge config header, and writing to it
> > controls the device, as the PCI spec intends. But the driver doesn't
> > copy those read/writes 1:1 to any HW register block, it distributes
> > them around the device's register space as necessary to create the
> > PCI bridge config header.
> > 
> > The DT is still modeling the HW, there really are root port bridge(s),
> > that are completely conformant at the TLP level, but the IP designers
> > choose to leave the detail of the bridge config space up to a SW
> > implementation.
> 
> Oh, ouch, deep spoofing.  I had to do that in system management mode
> firmware for a Geode chip.  What a pain that was.
> 
> > 
> >> At this point I am confused again.  There was talk of the need to
> >> use standard PCI enumeration code to deal with the bridges, thus the
> >> need for 3/2 to describe the interface between root-complex and the
> >> child root-port nodes.
> > 
> > I think your confusion is coming from your expectation that the PCI-E
> > IP block would be an entirely self contained root port, rather than
> > 'tools to build a root port, with the right driver SW'.
> > 
> >>> Further, review section 12 about how ranges should be treated - it
> >>> specifically says that the b,d,f bits in ranges should be 0, and the
> >>> child address should have those bits masked prior to searching the
> >>> ranges.
> >>>
> >>> Section 12 would seem to forbid this:
> >>>
> >>>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
> >>>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
> >>>
> >>> Are you reading that differently?
> >>
> >> I wasn't reading it at all, until you pointed it out to me :-)  I was
> >> just reasoning based on my mental model.
> >>
> >> Again, I agree with your reading of the spec, but I can't think of any
> >> reason why relaxing that restriction to permit config space mappings
> >> would be a problem.
> > 
> > Okay, just wanted to confirm that, I think I follow you now. :)
> > 
> > So you are basically proposing a small update to Section 12 that would
> > read something like:
> > 
> > 'When matching a config space address, the b,d,f bits are valid in the
> >  ranges and left unmasked in the child. The r bits in the ranges
> >  should be 0, and the r bits from the child are copied in the last DW
> >  and masked off in the first DW.'
> > 
> > Which, as you note, would not conflict with any existing usage..
> > 
> > Though, I can already see that the above language is not good enough,
> > it doesn't elegantly represent standard ECAM, for instance.. How would
> > you even represent ECAM with ranges??
> > 
> > Another wrinkle is that tegra not only has the per-bridge memory
> > mapped config space, bit it also has 4 'all port' ECAM-like memory
> > mapped config spaces. The rules around them are sufficiently
> > specialized that I'm not sure a ranges mapping would ever be
> > practical.. So right out of the gate we'd fail to capture memory
> > mapped config access via ranges on at least one SOC.
> > 
> > So, to me, this seems like overkill. Only ECAM has a hope of having a
> > common implementation, everything else will be host driver specific,
> > so why go to the trouble to revise the OF PCI specification for config
> > space?
> > 
> > Particularly since there are patches waiting on this question being
> > resolved :)
> 
> Okay, so I'll just bow out.  Do what you have to do.

So to recapitulate, we agree that configuration space can be translated
through the "ranges" property. That means the only missing link is a new
function to translate not only "assigned-addresses" but also the "reg"
property for PCI devices. Is that it?

And for Marvell the non-configuration-space per-root-port registers need
to be mapped separately from the configuration space windows?

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12  7:08                                                                     ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-12  7:08 UTC (permalink / raw)
  To: linux-arm-kernel

On Mon, Mar 11, 2013 at 01:38:38PM -1000, Mitch Bradley wrote:
> On 3/11/2013 1:25 PM, Jason Gunthorpe wrote:
> > On Mon, Mar 11, 2013 at 11:50:19AM -1000, Mitch Bradley wrote:
> > 
> >>> However - the driver runs the core in a 'root port bridge mode' where
> >>> the config header register block you are looking at is inhibited. The
> >>> Marvell IP block requires software support to run in bridge mode. So
> >>> Marvell really has only (2), while Tegra has only (1).
> >>
> >> Interesting.
> >>
> >> For Marvell, if Marvell has only (2), does that imply that standard PCI
> >> discovery and enumeration code cannot find the root port bridges, and
> >> also there is no way to auto-configure the bridges with common pci-pci
> >> bridge code?
> > 
> > The driver is required to take care of all of this. From the common
> > code's perspective there is a bridge config header, and writing to it
> > controls the device, as the PCI spec intends. But the driver doesn't
> > copy those read/writes 1:1 to any HW register block, it distributes
> > them around the device's register space as necessary to create the
> > PCI bridge config header.
> > 
> > The DT is still modeling the HW, there really are root port bridge(s),
> > that are completely conformant at the TLP level, but the IP designers
> > choose to leave the detail of the bridge config space up to a SW
> > implementation.
> 
> Oh, ouch, deep spoofing.  I had to do that in system management mode
> firmware for a Geode chip.  What a pain that was.
> 
> > 
> >> At this point I am confused again.  There was talk of the need to
> >> use standard PCI enumeration code to deal with the bridges, thus the
> >> need for 3/2 to describe the interface between root-complex and the
> >> child root-port nodes.
> > 
> > I think your confusion is coming from your expectation that the PCI-E
> > IP block would be an entirely self contained root port, rather than
> > 'tools to build a root port, with the right driver SW'.
> > 
> >>> Further, review section 12 about how ranges should be treated - it
> >>> specifically says that the b,d,f bits in ranges should be 0, and the
> >>> child address should have those bits masked prior to searching the
> >>> ranges.
> >>>
> >>> Section 12 would seem to forbid this:
> >>>
> >>>       ranges = <0x00000800 0 0          0xd4004000 0 0x00000800   /* Root port config header */
> >>>                 0x00001000 0 0          0xd4008000 0 0x00000800   /* Root port config header */
> >>>
> >>> Are you reading that differently?
> >>
> >> I wasn't reading it at all, until you pointed it out to me :-)  I was
> >> just reasoning based on my mental model.
> >>
> >> Again, I agree with your reading of the spec, but I can't think of any
> >> reason why relaxing that restriction to permit config space mappings
> >> would be a problem.
> > 
> > Okay, just wanted to confirm that, I think I follow you now. :)
> > 
> > So you are basically proposing a small update to Section 12 that would
> > read something like:
> > 
> > 'When matching a config space address, the b,d,f bits are valid in the
> >  ranges and left unmasked in the child. The r bits in the ranges
> >  should be 0, and the r bits from the child are copied in the last DW
> >  and masked off in the first DW.'
> > 
> > Which, as you note, would not conflict with any existing usage..
> > 
> > Though, I can already see that the above language is not good enough,
> > it doesn't elegantly represent standard ECAM, for instance.. How would
> > you even represent ECAM with ranges??
> > 
> > Another wrinkle is that tegra not only has the per-bridge memory
> > mapped config space, bit it also has 4 'all port' ECAM-like memory
> > mapped config spaces. The rules around them are sufficiently
> > specialized that I'm not sure a ranges mapping would ever be
> > practical.. So right out of the gate we'd fail to capture memory
> > mapped config access via ranges on at least one SOC.
> > 
> > So, to me, this seems like overkill. Only ECAM has a hope of having a
> > common implementation, everything else will be host driver specific,
> > so why go to the trouble to revise the OF PCI specification for config
> > space?
> > 
> > Particularly since there are patches waiting on this question being
> > resolved :)
> 
> Okay, so I'll just bow out.  Do what you have to do.

So to recapitulate, we agree that configuration space can be translated
through the "ranges" property. That means the only missing link is a new
function to translate not only "assigned-addresses" but also the "reg"
property for PCI devices. Is that it?

And for Marvell the non-configuration-space per-root-port registers need
to be mapped separately from the configuration space windows?

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130312/1908ebed/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-12  7:08                                                                     ` Thierry Reding
@ 2013-03-12 15:57                                                                       ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-12 15:57 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Tue, Mar 12, 2013 at 08:08:52AM +0100, Thierry Reding wrote:

> So to recapitulate, we agree that configuration space can be translated
> through the "ranges" property. That means the only missing link is a new
> function to translate not only "assigned-addresses" but also the "reg"
> property for PCI devices. Is that it?

No, the first conclusion is that placing a config space in ranges is
against the language in the current spec (see section 12).

The second conclusion is that there is probably a way forward to
update the spec in a backwards compatible way to model ECAM, but would
require more analysis.

Finally, there was no objections to the approach in Thomas's patch,
other than the note to fix the '@1,0' and the extra device_type="pci"
..

> And for Marvell the non-configuration-space per-root-port registers need
> to be mapped separately from the configuration space windows?

No, there are no configuration space windows on Marvell, so only one mapping.

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12 15:57                                                                       ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-12 15:57 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Mar 12, 2013 at 08:08:52AM +0100, Thierry Reding wrote:

> So to recapitulate, we agree that configuration space can be translated
> through the "ranges" property. That means the only missing link is a new
> function to translate not only "assigned-addresses" but also the "reg"
> property for PCI devices. Is that it?

No, the first conclusion is that placing a config space in ranges is
against the language in the current spec (see section 12).

The second conclusion is that there is probably a way forward to
update the spec in a backwards compatible way to model ECAM, but would
require more analysis.

Finally, there was no objections to the approach in Thomas's patch,
other than the note to fix the '@1,0' and the extra device_type="pci"
..

> And for Marvell the non-configuration-space per-root-port registers need
> to be mapped separately from the configuration space windows?

No, there are no configuration space windows on Marvell, so only one mapping.

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-12 15:57                                                                       ` Jason Gunthorpe
@ 2013-03-12 20:38                                                                         ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-12 20:38 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 3072 bytes --]

On Tue, Mar 12, 2013 at 09:57:49AM -0600, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 08:08:52AM +0100, Thierry Reding wrote:
> 
> > So to recapitulate, we agree that configuration space can be translated
> > through the "ranges" property. That means the only missing link is a new
> > function to translate not only "assigned-addresses" but also the "reg"
> > property for PCI devices. Is that it?
> 
> No, the first conclusion is that placing a config space in ranges is
> against the language in the current spec (see section 12).
> 
> The second conclusion is that there is probably a way forward to
> update the spec in a backwards compatible way to model ECAM, but would
> require more analysis.
> 
> Finally, there was no objections to the approach in Thomas's patch,
> other than the note to fix the '@1,0' and the extra device_type="pci"

That's not what I deduced from Mitch's comments. He said explicitly that
he liked the idea to represent mapped configuration space using the
ranges property. Mitch, correct me if I misinterpret what you said.

I've also verified that things actually do work with the binding that I
proposed for Tegra, even if I add device_type = "pci" to the root port
nodes. The reason is that the OF core looks up the type of the *parent*
node to find a matching bus. This happens to be the pcie-controller node
which is not a PCI device and therefore the address translation works
properly.

So let me quote the latest binding that I'm using:

	pcie-controller {
		compatible = "nvidia,tegra20-pcie";
		reg = <0x80003000 0x00000800   /* PADS registers */
		       0x80003800 0x00000200   /* AFI registers */
		       0x90000000 0x10000000>; /* configuration space */
		reg-names = "pads", "afi", "cs";
		interrupts = <0 98 0x04   /* controller interrupt */
		              0 99 0x04>; /* MSI interrupt */
		interrupt-names = "intr", "msi";

		bus-range = <0x00 0xff>;
		#address-cells = <3>;
		#size-cells = <2>;

		ranges = <0x00000800 0 0 0x80000000 0 0x00001000   /* port 0 configuration space */
			  0x00001000 0 0 0x80001000 0 0x00001000   /* port 1 configuration space */
			  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
			  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
			  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

		clocks = <&tegra_car 70>, <&tegra_car 72>, <&tegra_car 74>,
			 <&tegra_car 118>;
		clock-names = "pex", "afi", "pcie_xclk", "pll_e";
		status = "disabled";

		pci@1,0 {
			device_type = "pci";
			reg = <0x000800 0 0 0 0x1000>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};

		pci@2,0 {
			device_type = "pci";
			reg = <0x001000 0 0 0 0x1000>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};
	};

I can't find anything wrong in the above. As far as I can tell, there's
nothing in it that's in conflict with the specification.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12 20:38                                                                         ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-12 20:38 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Mar 12, 2013 at 09:57:49AM -0600, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 08:08:52AM +0100, Thierry Reding wrote:
> 
> > So to recapitulate, we agree that configuration space can be translated
> > through the "ranges" property. That means the only missing link is a new
> > function to translate not only "assigned-addresses" but also the "reg"
> > property for PCI devices. Is that it?
> 
> No, the first conclusion is that placing a config space in ranges is
> against the language in the current spec (see section 12).
> 
> The second conclusion is that there is probably a way forward to
> update the spec in a backwards compatible way to model ECAM, but would
> require more analysis.
> 
> Finally, there was no objections to the approach in Thomas's patch,
> other than the note to fix the '@1,0' and the extra device_type="pci"

That's not what I deduced from Mitch's comments. He said explicitly that
he liked the idea to represent mapped configuration space using the
ranges property. Mitch, correct me if I misinterpret what you said.

I've also verified that things actually do work with the binding that I
proposed for Tegra, even if I add device_type = "pci" to the root port
nodes. The reason is that the OF core looks up the type of the *parent*
node to find a matching bus. This happens to be the pcie-controller node
which is not a PCI device and therefore the address translation works
properly.

So let me quote the latest binding that I'm using:

	pcie-controller {
		compatible = "nvidia,tegra20-pcie";
		reg = <0x80003000 0x00000800   /* PADS registers */
		       0x80003800 0x00000200   /* AFI registers */
		       0x90000000 0x10000000>; /* configuration space */
		reg-names = "pads", "afi", "cs";
		interrupts = <0 98 0x04   /* controller interrupt */
		              0 99 0x04>; /* MSI interrupt */
		interrupt-names = "intr", "msi";

		bus-range = <0x00 0xff>;
		#address-cells = <3>;
		#size-cells = <2>;

		ranges = <0x00000800 0 0 0x80000000 0 0x00001000   /* port 0 configuration space */
			  0x00001000 0 0 0x80001000 0 0x00001000   /* port 1 configuration space */
			  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
			  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
			  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

		clocks = <&tegra_car 70>, <&tegra_car 72>, <&tegra_car 74>,
			 <&tegra_car 118>;
		clock-names = "pex", "afi", "pcie_xclk", "pll_e";
		status = "disabled";

		pci at 1,0 {
			device_type = "pci";
			reg = <0x000800 0 0 0 0x1000>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};

		pci at 2,0 {
			device_type = "pci";
			reg = <0x001000 0 0 0 0x1000>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};
	};

I can't find anything wrong in the above. As far as I can tell, there's
nothing in it that's in conflict with the specification.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130312/c5981d10/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-12 20:38                                                                         ` Thierry Reding
@ 2013-03-12 21:03                                                                           ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-12 21:03 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Tue, Mar 12, 2013 at 09:38:19PM +0100, Thierry Reding wrote:

> I've also verified that things actually do work with the binding that I
> proposed for Tegra, even if I add device_type = "pci" to the root port
> nodes. The reason is that the OF core looks up the type of the *parent*
> node to find a matching bus. This happens to be the pcie-controller node
> which is not a PCI device and therefore the address translation works
> properly.

Mitch pointed out early on that the parent node needs a 'device_type =
"pci"' as well.

The pcie-controller node would be classified as a 'bus node' according
to the spec.

Not going down the of_pci_* code paths for address translation at the
root port bridge nodes is certainly not right.

> ranges = <0x00000800 0 0 0x80000000 0 0x00001000   /* port 0 configuration space */
>           0x00001000 0 0 0x80001000 0 0x00001000   /* port 1 configuration space */

Section 12 says that the 'b,d,f' bits are 0 in ranges, so the above is
in conflict with that language:

 In particular, the phys.hi fields of the child address spaces in the
 "ranges" property for PCI does not contain the same information as
 "reg" property entries within PCI nodes. The only information that is
 present in "ranges" phys.hi entries are the non-relocatable,
 prefetchable and the PCI address space bits for which the entry
 applies. I.e., only the n, p and ss bits are present; the bbbbbbbb,
 ddddd, fff and rrrrrrrr fields are 0.

And the further requirement:

 When an address is to be mapped through a PCI bus bridge node, the
 phys.hi value of the address to be mapped and the child field of a
 "ranges" entry should be masked so that only the ss bits are compared.

Which is embodied by the code in of_bus_pci_map, meaning the OF core
won't even look at the bits in the high DW when searching ranges.

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12 21:03                                                                           ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-12 21:03 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Mar 12, 2013 at 09:38:19PM +0100, Thierry Reding wrote:

> I've also verified that things actually do work with the binding that I
> proposed for Tegra, even if I add device_type = "pci" to the root port
> nodes. The reason is that the OF core looks up the type of the *parent*
> node to find a matching bus. This happens to be the pcie-controller node
> which is not a PCI device and therefore the address translation works
> properly.

Mitch pointed out early on that the parent node needs a 'device_type =
"pci"' as well.

The pcie-controller node would be classified as a 'bus node' according
to the spec.

Not going down the of_pci_* code paths for address translation at the
root port bridge nodes is certainly not right.

> ranges = <0x00000800 0 0 0x80000000 0 0x00001000   /* port 0 configuration space */
>           0x00001000 0 0 0x80001000 0 0x00001000   /* port 1 configuration space */

Section 12 says that the 'b,d,f' bits are 0 in ranges, so the above is
in conflict with that language:

 In particular, the phys.hi fields of the child address spaces in the
 "ranges" property for PCI does not contain the same information as
 "reg" property entries within PCI nodes. The only information that is
 present in "ranges" phys.hi entries are the non-relocatable,
 prefetchable and the PCI address space bits for which the entry
 applies. I.e., only the n, p and ss bits are present; the bbbbbbbb,
 ddddd, fff and rrrrrrrr fields are 0.

And the further requirement:

 When an address is to be mapped through a PCI bus bridge node, the
 phys.hi value of the address to be mapped and the child field of a
 "ranges" entry should be masked so that only the ss bits are compared.

Which is embodied by the code in of_bus_pci_map, meaning the OF core
won't even look at the bits in the high DW when searching ranges.

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-12 21:03                                                                           ` Jason Gunthorpe
@ 2013-03-12 21:30                                                                             ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-12 21:30 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 2918 bytes --]

On Tue, Mar 12, 2013 at 03:03:28PM -0600, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 09:38:19PM +0100, Thierry Reding wrote:
> 
> > I've also verified that things actually do work with the binding that I
> > proposed for Tegra, even if I add device_type = "pci" to the root port
> > nodes. The reason is that the OF core looks up the type of the *parent*
> > node to find a matching bus. This happens to be the pcie-controller node
> > which is not a PCI device and therefore the address translation works
> > properly.
> 
> Mitch pointed out early on that the parent node needs a 'device_type =
> "pci"' as well.
> 
> The pcie-controller node would be classified as a 'bus node' according
> to the spec.
> 
> Not going down the of_pci_* code paths for address translation at the
> root port bridge nodes is certainly not right.

I'm not so sure. Why should the pcie-controller be a PCI device? It
really isn't. Instead it is a device on the processor/SoC/platform bus
that bridges to the PCI bus. As such its children should be PCI devices
to reflect that the translation happens within the pcie-controller node.

In the above it is logical to not set the pcie-controller's device_type
property to "pci".

> > ranges = <0x00000800 0 0 0x80000000 0 0x00001000   /* port 0 configuration space */
> >           0x00001000 0 0 0x80001000 0 0x00001000   /* port 1 configuration space */
> 
> Section 12 says that the 'b,d,f' bits are 0 in ranges, so the above is
> in conflict with that language:
> 
>  In particular, the phys.hi fields of the child address spaces in the
>  "ranges" property for PCI does not contain the same information as
>  "reg" property entries within PCI nodes. The only information that is
>  present in "ranges" phys.hi entries are the non-relocatable,
>  prefetchable and the PCI address space bits for which the entry
>  applies. I.e., only the n, p and ss bits are present; the bbbbbbbb,
>  ddddd, fff and rrrrrrrr fields are 0.
> 
> And the further requirement:
> 
>  When an address is to be mapped through a PCI bus bridge node, the
>  phys.hi value of the address to be mapped and the child field of a
>  "ranges" entry should be masked so that only the ss bits are compared.
> 
> Which is embodied by the code in of_bus_pci_map, meaning the OF core
> won't even look at the bits in the high DW when searching ranges.

Okay, so the problem with that is that we won't be able to include more
entries for configuration space within a ranges property because we
can't uniquely map them (they both have the same ss field values). One
possibility would be to differentiate using the phys.mid and phys.low
cells but they need to be 0 according to the spec.

The other alternative would be to amend the specification. Besides the
fact that the specification says so I don't see any reason why this
shouldn't be allowed.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12 21:30                                                                             ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-12 21:30 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Mar 12, 2013 at 03:03:28PM -0600, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 09:38:19PM +0100, Thierry Reding wrote:
> 
> > I've also verified that things actually do work with the binding that I
> > proposed for Tegra, even if I add device_type = "pci" to the root port
> > nodes. The reason is that the OF core looks up the type of the *parent*
> > node to find a matching bus. This happens to be the pcie-controller node
> > which is not a PCI device and therefore the address translation works
> > properly.
> 
> Mitch pointed out early on that the parent node needs a 'device_type =
> "pci"' as well.
> 
> The pcie-controller node would be classified as a 'bus node' according
> to the spec.
> 
> Not going down the of_pci_* code paths for address translation at the
> root port bridge nodes is certainly not right.

I'm not so sure. Why should the pcie-controller be a PCI device? It
really isn't. Instead it is a device on the processor/SoC/platform bus
that bridges to the PCI bus. As such its children should be PCI devices
to reflect that the translation happens within the pcie-controller node.

In the above it is logical to not set the pcie-controller's device_type
property to "pci".

> > ranges = <0x00000800 0 0 0x80000000 0 0x00001000   /* port 0 configuration space */
> >           0x00001000 0 0 0x80001000 0 0x00001000   /* port 1 configuration space */
> 
> Section 12 says that the 'b,d,f' bits are 0 in ranges, so the above is
> in conflict with that language:
> 
>  In particular, the phys.hi fields of the child address spaces in the
>  "ranges" property for PCI does not contain the same information as
>  "reg" property entries within PCI nodes. The only information that is
>  present in "ranges" phys.hi entries are the non-relocatable,
>  prefetchable and the PCI address space bits for which the entry
>  applies. I.e., only the n, p and ss bits are present; the bbbbbbbb,
>  ddddd, fff and rrrrrrrr fields are 0.
> 
> And the further requirement:
> 
>  When an address is to be mapped through a PCI bus bridge node, the
>  phys.hi value of the address to be mapped and the child field of a
>  "ranges" entry should be masked so that only the ss bits are compared.
> 
> Which is embodied by the code in of_bus_pci_map, meaning the OF core
> won't even look at the bits in the high DW when searching ranges.

Okay, so the problem with that is that we won't be able to include more
entries for configuration space within a ranges property because we
can't uniquely map them (they both have the same ss field values). One
possibility would be to differentiate using the phys.mid and phys.low
cells but they need to be 0 according to the spec.

The other alternative would be to amend the specification. Besides the
fact that the specification says so I don't see any reason why this
shouldn't be allowed.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130312/ca54ca56/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-12 21:30                                                                             ` Thierry Reding
@ 2013-03-12 22:08                                                                               ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-12 22:08 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Tue, Mar 12, 2013 at 10:30:06PM +0100, Thierry Reding wrote:

> > Not going down the of_pci_* code paths for address translation at the
> > root port bridge nodes is certainly not right.
> 
> I'm not so sure. Why should the pcie-controller be a PCI device?

The spec is clear on that point as well:

 device_type
  A Standard Package conforming to this specification and corresponding
  to a device that implements a PCI bus shall implement this property
  with the string value "pci"

The children of a pcie-controller node are PCI devices, thus the
pcie-controller node 'implements a PCI bus'. Or, as you say 'device on
the processor/SoC/platform bus that bridges to the PCI bus'

Think of device_type as also meaning bus_type and it makes more
logical sense, the name is terrible, but its usage is governed by the
OF docs..

> The other alternative would be to amend the specification. Besides the
> fact that the specification says so I don't see any reason why this
> shouldn't be allowed.

'the specification says so' *IS* the reason. DT isn't a free for all
where you get to do whatever you want, or whatever 'feels' right. It
is supposed to be a stable, OS agnostic ABI. That means bindings have
to follow the specs (when available).

Maybe a future revision will support PCI-E ECAM, but we don't know
what that will look like, and I'm pretty sure you don't want to hold
up your patches until an IEEE committee gets around to deciding
something ;)

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12 22:08                                                                               ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-12 22:08 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Mar 12, 2013 at 10:30:06PM +0100, Thierry Reding wrote:

> > Not going down the of_pci_* code paths for address translation at the
> > root port bridge nodes is certainly not right.
> 
> I'm not so sure. Why should the pcie-controller be a PCI device?

The spec is clear on that point as well:

 device_type
  A Standard Package conforming to this specification and corresponding
  to a device that implements a PCI bus shall implement this property
  with the string value "pci"

The children of a pcie-controller node are PCI devices, thus the
pcie-controller node 'implements a PCI bus'. Or, as you say 'device on
the processor/SoC/platform bus that bridges to the PCI bus'

Think of device_type as also meaning bus_type and it makes more
logical sense, the name is terrible, but its usage is governed by the
OF docs..

> The other alternative would be to amend the specification. Besides the
> fact that the specification says so I don't see any reason why this
> shouldn't be allowed.

'the specification says so' *IS* the reason. DT isn't a free for all
where you get to do whatever you want, or whatever 'feels' right. It
is supposed to be a stable, OS agnostic ABI. That means bindings have
to follow the specs (when available).

Maybe a future revision will support PCI-E ECAM, but we don't know
what that will look like, and I'm pretty sure you don't want to hold
up your patches until an IEEE committee gets around to deciding
something ;)

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-12 22:08                                                                               ` Jason Gunthorpe
  (?)
@ 2013-03-12 23:25                                                                                 ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-12 23:25 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci, devicetree-discuss,
	Thierry Reding, Eran Ben-Avi, Nadav Haklai, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, linux-arm-kernel

On 3/12/2013 12:08 PM, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 10:30:06PM +0100, Thierry Reding wrote:
> 
>>> Not going down the of_pci_* code paths for address translation at the
>>> root port bridge nodes is certainly not right.
>>
>> I'm not so sure. Why should the pcie-controller be a PCI device?
> 
> The spec is clear on that point as well:
> 
>  device_type
>   A Standard Package conforming to this specification and corresponding
>   to a device that implements a PCI bus shall implement this property
>   with the string value "pci"
> 
> The children of a pcie-controller node are PCI devices, thus the
> pcie-controller node 'implements a PCI bus'. Or, as you say 'device on
> the processor/SoC/platform bus that bridges to the PCI bus'
> 
> Think of device_type as also meaning bus_type and it makes more
> logical sense, the name is terrible, but its usage is governed by the
> OF docs..

I beg to differ about the name.  device_type means "what kind of
functionality is implemented by this device", not "what interface
connects this device to the rest of the system".   The use of
"device_type" for that meaning is perfectly reasonable in the English
language.

The DT answers the "what interface connects me to the system" question
by looking at the parent.

> 
>> The other alternative would be to amend the specification. Besides the
>> fact that the specification says so I don't see any reason why this
>> shouldn't be allowed.
> 
> 'the specification says so' *IS* the reason. DT isn't a free for all
> where you get to do whatever you want, or whatever 'feels' right. It
> is supposed to be a stable, OS agnostic ABI. That means bindings have
> to follow the specs (when available).
> 
> Maybe a future revision will support PCI-E ECAM, but we don't know
> what that will look like, and I'm pretty sure you don't want to hold
> up your patches until an IEEE committee gets around to deciding
> something ;)

The Open Firmware Working Group quit doing things under the IEEE banner
a long long time ago, precisely for the reason that the IEEE process was
too slow, cumbersome, and expensive.  The group thereafter published
"Recommended Practices" documents under its own non-authority instead of
going through the IEEE ratification process.  The group's only
"authority" arose from the fact that the members had expertise and
worked for companies that needed to use the results.

The Working Group is now inactive.  I am in contact with many of the
former members, but by and large they are no longer interested in
firmware standardization.  Many of them work for companies that now hold
their cards closely, that seemingly being the only way to make enough
money to stay in business.

In other words, there is no committee.

New versions of the document in question, or of any document for that
matter, can be issued by the simple expedient of interested parties
coming to agreement and writing down that agreement.  In practice, that
is what happens in Linux land for nearly every issue.

At this time, the primary "interest group" for device tree issues
revolves around the Linux kernel - the devicetree-discuss list is where
all the action is.  I suppose that Oracle still cares about Open
Firmware on its SPARC systems, but it is likely that Oracle is going to
do whatever they want no matter what Linux people say or do.  In
principle, the DT is supposed to be OS-agnostic, but in practice, which
other OS both uses the DT and has enough users to matter?  So it would
be entirely reasonable for a group of Linux kernel people to publish
revised documents, as has already been done in the form of
Documentation/ files setting down usage rules.


> 
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12 23:25                                                                                 ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-12 23:25 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/12/2013 12:08 PM, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 10:30:06PM +0100, Thierry Reding wrote:
> 
>>> Not going down the of_pci_* code paths for address translation at the
>>> root port bridge nodes is certainly not right.
>>
>> I'm not so sure. Why should the pcie-controller be a PCI device?
> 
> The spec is clear on that point as well:
> 
>  device_type
>   A Standard Package conforming to this specification and corresponding
>   to a device that implements a PCI bus shall implement this property
>   with the string value "pci"
> 
> The children of a pcie-controller node are PCI devices, thus the
> pcie-controller node 'implements a PCI bus'. Or, as you say 'device on
> the processor/SoC/platform bus that bridges to the PCI bus'
> 
> Think of device_type as also meaning bus_type and it makes more
> logical sense, the name is terrible, but its usage is governed by the
> OF docs..

I beg to differ about the name.  device_type means "what kind of
functionality is implemented by this device", not "what interface
connects this device to the rest of the system".   The use of
"device_type" for that meaning is perfectly reasonable in the English
language.

The DT answers the "what interface connects me to the system" question
by looking at the parent.

> 
>> The other alternative would be to amend the specification. Besides the
>> fact that the specification says so I don't see any reason why this
>> shouldn't be allowed.
> 
> 'the specification says so' *IS* the reason. DT isn't a free for all
> where you get to do whatever you want, or whatever 'feels' right. It
> is supposed to be a stable, OS agnostic ABI. That means bindings have
> to follow the specs (when available).
> 
> Maybe a future revision will support PCI-E ECAM, but we don't know
> what that will look like, and I'm pretty sure you don't want to hold
> up your patches until an IEEE committee gets around to deciding
> something ;)

The Open Firmware Working Group quit doing things under the IEEE banner
a long long time ago, precisely for the reason that the IEEE process was
too slow, cumbersome, and expensive.  The group thereafter published
"Recommended Practices" documents under its own non-authority instead of
going through the IEEE ratification process.  The group's only
"authority" arose from the fact that the members had expertise and
worked for companies that needed to use the results.

The Working Group is now inactive.  I am in contact with many of the
former members, but by and large they are no longer interested in
firmware standardization.  Many of them work for companies that now hold
their cards closely, that seemingly being the only way to make enough
money to stay in business.

In other words, there is no committee.

New versions of the document in question, or of any document for that
matter, can be issued by the simple expedient of interested parties
coming to agreement and writing down that agreement.  In practice, that
is what happens in Linux land for nearly every issue.

At this time, the primary "interest group" for device tree issues
revolves around the Linux kernel - the devicetree-discuss list is where
all the action is.  I suppose that Oracle still cares about Open
Firmware on its SPARC systems, but it is likely that Oracle is going to
do whatever they want no matter what Linux people say or do.  In
principle, the DT is supposed to be OS-agnostic, but in practice, which
other OS both uses the DT and has enough users to matter?  So it would
be entirely reasonable for a group of Linux kernel people to publish
revised documents, as has already been done in the form of
Documentation/ files setting down usage rules.


> 
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-12 23:25                                                                                 ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-12 23:25 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/12/2013 12:08 PM, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 10:30:06PM +0100, Thierry Reding wrote:
> 
>>> Not going down the of_pci_* code paths for address translation at the
>>> root port bridge nodes is certainly not right.
>>
>> I'm not so sure. Why should the pcie-controller be a PCI device?
> 
> The spec is clear on that point as well:
> 
>  device_type
>   A Standard Package conforming to this specification and corresponding
>   to a device that implements a PCI bus shall implement this property
>   with the string value "pci"
> 
> The children of a pcie-controller node are PCI devices, thus the
> pcie-controller node 'implements a PCI bus'. Or, as you say 'device on
> the processor/SoC/platform bus that bridges to the PCI bus'
> 
> Think of device_type as also meaning bus_type and it makes more
> logical sense, the name is terrible, but its usage is governed by the
> OF docs..

I beg to differ about the name.  device_type means "what kind of
functionality is implemented by this device", not "what interface
connects this device to the rest of the system".   The use of
"device_type" for that meaning is perfectly reasonable in the English
language.

The DT answers the "what interface connects me to the system" question
by looking at the parent.

> 
>> The other alternative would be to amend the specification. Besides the
>> fact that the specification says so I don't see any reason why this
>> shouldn't be allowed.
> 
> 'the specification says so' *IS* the reason. DT isn't a free for all
> where you get to do whatever you want, or whatever 'feels' right. It
> is supposed to be a stable, OS agnostic ABI. That means bindings have
> to follow the specs (when available).
> 
> Maybe a future revision will support PCI-E ECAM, but we don't know
> what that will look like, and I'm pretty sure you don't want to hold
> up your patches until an IEEE committee gets around to deciding
> something ;)

The Open Firmware Working Group quit doing things under the IEEE banner
a long long time ago, precisely for the reason that the IEEE process was
too slow, cumbersome, and expensive.  The group thereafter published
"Recommended Practices" documents under its own non-authority instead of
going through the IEEE ratification process.  The group's only
"authority" arose from the fact that the members had expertise and
worked for companies that needed to use the results.

The Working Group is now inactive.  I am in contact with many of the
former members, but by and large they are no longer interested in
firmware standardization.  Many of them work for companies that now hold
their cards closely, that seemingly being the only way to make enough
money to stay in business.

In other words, there is no committee.

New versions of the document in question, or of any document for that
matter, can be issued by the simple expedient of interested parties
coming to agreement and writing down that agreement.  In practice, that
is what happens in Linux land for nearly every issue.

At this time, the primary "interest group" for device tree issues
revolves around the Linux kernel - the devicetree-discuss list is where
all the action is.  I suppose that Oracle still cares about Open
Firmware on its SPARC systems, but it is likely that Oracle is going to
do whatever they want no matter what Linux people say or do.  In
principle, the DT is supposed to be OS-agnostic, but in practice, which
other OS both uses the DT and has enough users to matter?  So it would
be entirely reasonable for a group of Linux kernel people to publish
revised documents, as has already been done in the form of
Documentation/ files setting down usage rules.


> 
> Jason
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-12 22:08                                                                               ` Jason Gunthorpe
  (?)
@ 2013-03-13  8:18                                                                                 ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13  8:18 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Mitch Bradley, Bjorn Helgaas, linux-arm-kernel


[-- Attachment #1.1: Type: text/plain, Size: 2206 bytes --]

On Tue, Mar 12, 2013 at 04:08:54PM -0600, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 10:30:06PM +0100, Thierry Reding wrote:
> 
> > > Not going down the of_pci_* code paths for address translation at the
> > > root port bridge nodes is certainly not right.
> > 
> > I'm not so sure. Why should the pcie-controller be a PCI device?
> 
> The spec is clear on that point as well:
> 
>  device_type
>   A Standard Package conforming to this specification and corresponding
>   to a device that implements a PCI bus shall implement this property
>   with the string value "pci"
> 
> The children of a pcie-controller node are PCI devices, thus the
> pcie-controller node 'implements a PCI bus'. Or, as you say 'device on
> the processor/SoC/platform bus that bridges to the PCI bus'
> 
> Think of device_type as also meaning bus_type and it makes more
> logical sense, the name is terrible, but its usage is governed by the
> OF docs..
> 
> > The other alternative would be to amend the specification. Besides the
> > fact that the specification says so I don't see any reason why this
> > shouldn't be allowed.
> 
> 'the specification says so' *IS* the reason. DT isn't a free for all
> where you get to do whatever you want, or whatever 'feels' right. It
> is supposed to be a stable, OS agnostic ABI. That means bindings have
> to follow the specs (when available).
> 
> Maybe a future revision will support PCI-E ECAM, but we don't know
> what that will look like, and I'm pretty sure you don't want to hold
> up your patches until an IEEE committee gets around to deciding
> something ;)

Mitch already answered this. The specification is now almost 15 years
old and it couldn't possibly have foreseen all of the future use-cases.
If the specification is too restrictive and Mitch gives his blessing to
remove some of the restrictions, I don't see any reason why we shouldn't
do so if it lets us represent the reality of hardware more accurately in
DT.

Furthermore we're not discussing radical changes. None of the changes
will be backwards-incompatible, but they will allow recent hardware to
be represented more correctly or conveniently.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13  8:18                                                                                 ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13  8:18 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 2206 bytes --]

On Tue, Mar 12, 2013 at 04:08:54PM -0600, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 10:30:06PM +0100, Thierry Reding wrote:
> 
> > > Not going down the of_pci_* code paths for address translation at the
> > > root port bridge nodes is certainly not right.
> > 
> > I'm not so sure. Why should the pcie-controller be a PCI device?
> 
> The spec is clear on that point as well:
> 
>  device_type
>   A Standard Package conforming to this specification and corresponding
>   to a device that implements a PCI bus shall implement this property
>   with the string value "pci"
> 
> The children of a pcie-controller node are PCI devices, thus the
> pcie-controller node 'implements a PCI bus'. Or, as you say 'device on
> the processor/SoC/platform bus that bridges to the PCI bus'
> 
> Think of device_type as also meaning bus_type and it makes more
> logical sense, the name is terrible, but its usage is governed by the
> OF docs..
> 
> > The other alternative would be to amend the specification. Besides the
> > fact that the specification says so I don't see any reason why this
> > shouldn't be allowed.
> 
> 'the specification says so' *IS* the reason. DT isn't a free for all
> where you get to do whatever you want, or whatever 'feels' right. It
> is supposed to be a stable, OS agnostic ABI. That means bindings have
> to follow the specs (when available).
> 
> Maybe a future revision will support PCI-E ECAM, but we don't know
> what that will look like, and I'm pretty sure you don't want to hold
> up your patches until an IEEE committee gets around to deciding
> something ;)

Mitch already answered this. The specification is now almost 15 years
old and it couldn't possibly have foreseen all of the future use-cases.
If the specification is too restrictive and Mitch gives his blessing to
remove some of the restrictions, I don't see any reason why we shouldn't
do so if it lets us represent the reality of hardware more accurately in
DT.

Furthermore we're not discussing radical changes. None of the changes
will be backwards-incompatible, but they will allow recent hardware to
be represented more correctly or conveniently.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13  8:18                                                                                 ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13  8:18 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Mar 12, 2013 at 04:08:54PM -0600, Jason Gunthorpe wrote:
> On Tue, Mar 12, 2013 at 10:30:06PM +0100, Thierry Reding wrote:
> 
> > > Not going down the of_pci_* code paths for address translation at the
> > > root port bridge nodes is certainly not right.
> > 
> > I'm not so sure. Why should the pcie-controller be a PCI device?
> 
> The spec is clear on that point as well:
> 
>  device_type
>   A Standard Package conforming to this specification and corresponding
>   to a device that implements a PCI bus shall implement this property
>   with the string value "pci"
> 
> The children of a pcie-controller node are PCI devices, thus the
> pcie-controller node 'implements a PCI bus'. Or, as you say 'device on
> the processor/SoC/platform bus that bridges to the PCI bus'
> 
> Think of device_type as also meaning bus_type and it makes more
> logical sense, the name is terrible, but its usage is governed by the
> OF docs..
> 
> > The other alternative would be to amend the specification. Besides the
> > fact that the specification says so I don't see any reason why this
> > shouldn't be allowed.
> 
> 'the specification says so' *IS* the reason. DT isn't a free for all
> where you get to do whatever you want, or whatever 'feels' right. It
> is supposed to be a stable, OS agnostic ABI. That means bindings have
> to follow the specs (when available).
> 
> Maybe a future revision will support PCI-E ECAM, but we don't know
> what that will look like, and I'm pretty sure you don't want to hold
> up your patches until an IEEE committee gets around to deciding
> something ;)

Mitch already answered this. The specification is now almost 15 years
old and it couldn't possibly have foreseen all of the future use-cases.
If the specification is too restrictive and Mitch gives his blessing to
remove some of the restrictions, I don't see any reason why we shouldn't
do so if it lets us represent the reality of hardware more accurately in
DT.

Furthermore we're not discussing radical changes. None of the changes
will be backwards-incompatible, but they will allow recent hardware to
be represented more correctly or conveniently.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130313/901fb399/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13  8:18                                                                                 ` Thierry Reding
@ 2013-03-13 17:02                                                                                   ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-13 17:02 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Wed, Mar 13, 2013 at 09:18:15AM +0100, Thierry Reding wrote:

> Mitch already answered this. The specification is now almost 15 years
> old and it couldn't possibly have foreseen all of the future use-cases.
> If the specification is too restrictive and Mitch gives his blessing to
> remove some of the restrictions, I don't see any reason why we shouldn't
> do so if it lets us represent the reality of hardware more accurately in
> DT.

I understand the spec is old, and I have no problem with making a
Linux specific revision, but do *that* - don't bury some random
deviation inside the bindings for a driver. I even suggested some
language, but I feel more thought is needed to consider how to model
the standardized ECAM mechanism..

> Furthermore we're not discussing radical changes. None of the changes
> will be backwards-incompatible, but they will allow recent hardware to
> be represented more correctly or conveniently.

Sure, but it is still inconsistent, one MMIO config mechansim is in
ranges, one is in regs. Plus I don't think tegra's case is a great
starting point to design a spec update, it's config access mechanism
is especially strange...

Anyhow, I think this has been hashed to death, as long as your final
binding has the 'device_type = pci' on the pcie-controller node I
think it will be fine.

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 17:02                                                                                   ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-13 17:02 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 09:18:15AM +0100, Thierry Reding wrote:

> Mitch already answered this. The specification is now almost 15 years
> old and it couldn't possibly have foreseen all of the future use-cases.
> If the specification is too restrictive and Mitch gives his blessing to
> remove some of the restrictions, I don't see any reason why we shouldn't
> do so if it lets us represent the reality of hardware more accurately in
> DT.

I understand the spec is old, and I have no problem with making a
Linux specific revision, but do *that* - don't bury some random
deviation inside the bindings for a driver. I even suggested some
language, but I feel more thought is needed to consider how to model
the standardized ECAM mechanism..

> Furthermore we're not discussing radical changes. None of the changes
> will be backwards-incompatible, but they will allow recent hardware to
> be represented more correctly or conveniently.

Sure, but it is still inconsistent, one MMIO config mechansim is in
ranges, one is in regs. Plus I don't think tegra's case is a great
starting point to design a spec update, it's config access mechanism
is especially strange...

Anyhow, I think this has been hashed to death, as long as your final
binding has the 'device_type = pci' on the pcie-controller node I
think it will be fine.

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 17:02                                                                                   ` Jason Gunthorpe
  (?)
@ 2013-03-13 19:26                                                                                       ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 19:26 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Nadav Haklai, Maen Suleiman, Shadi Ammouri, Mitch Bradley,
	Bjorn Helgaas, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r


[-- Attachment #1.1: Type: text/plain, Size: 2725 bytes --]

On Wed, Mar 13, 2013 at 11:02:05AM -0600, Jason Gunthorpe wrote:
> On Wed, Mar 13, 2013 at 09:18:15AM +0100, Thierry Reding wrote:
> 
> > Mitch already answered this. The specification is now almost 15 years
> > old and it couldn't possibly have foreseen all of the future use-cases.
> > If the specification is too restrictive and Mitch gives his blessing to
> > remove some of the restrictions, I don't see any reason why we shouldn't
> > do so if it lets us represent the reality of hardware more accurately in
> > DT.
> 
> I understand the spec is old, and I have no problem with making a
> Linux specific revision, but do *that* - don't bury some random
> deviation inside the bindings for a driver. I even suggested some
> language, but I feel more thought is needed to consider how to model
> the standardized ECAM mechanism..

As Mitch already said there's very little chance that the specification
update will be ratified through IEEE, so I think that we might just as
well put a corresponding text somewhere below Documentation/devicetree.

Also note that this has absolutely nothing to do with ECAM. All I'm
proposing is to allow the reg property of a root port to be translated
into a CPU addressable memory region through the ranges property. This
turns out to actually work properly with the current OF core in Linux.

> > Furthermore we're not discussing radical changes. None of the changes
> > will be backwards-incompatible, but they will allow recent hardware to
> > be represented more correctly or conveniently.
> 
> Sure, but it is still inconsistent, one MMIO config mechansim is in
> ranges, one is in regs. Plus I don't think tegra's case is a great
> starting point to design a spec update, it's config access mechanism
> is especially strange...

Again, it is still the most accurate way to describe the hardware. I
know it's not terribly beautiful and doesn't match with what you'd
expect from PC hardware. However it is still a reality and something the
kernel will have to deal with. Marvell hardware isn't very pretty either
but that shouldn't exclude it from being supported by Linux.

> Anyhow, I think this has been hashed to death, as long as your final
> binding has the 'device_type = pci' on the pcie-controller node I
> think it will be fine.

No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
It is the instance that translates from the platform to the PCI bus. Why
should it be device_type = "pci"? And we've also been over this before,
making that change stops the proposed binding from working properly
because the entry in the reg property can't be translated into a CPU
addressable memory region.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 19:26                                                                                       ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 19:26 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 2725 bytes --]

On Wed, Mar 13, 2013 at 11:02:05AM -0600, Jason Gunthorpe wrote:
> On Wed, Mar 13, 2013 at 09:18:15AM +0100, Thierry Reding wrote:
> 
> > Mitch already answered this. The specification is now almost 15 years
> > old and it couldn't possibly have foreseen all of the future use-cases.
> > If the specification is too restrictive and Mitch gives his blessing to
> > remove some of the restrictions, I don't see any reason why we shouldn't
> > do so if it lets us represent the reality of hardware more accurately in
> > DT.
> 
> I understand the spec is old, and I have no problem with making a
> Linux specific revision, but do *that* - don't bury some random
> deviation inside the bindings for a driver. I even suggested some
> language, but I feel more thought is needed to consider how to model
> the standardized ECAM mechanism..

As Mitch already said there's very little chance that the specification
update will be ratified through IEEE, so I think that we might just as
well put a corresponding text somewhere below Documentation/devicetree.

Also note that this has absolutely nothing to do with ECAM. All I'm
proposing is to allow the reg property of a root port to be translated
into a CPU addressable memory region through the ranges property. This
turns out to actually work properly with the current OF core in Linux.

> > Furthermore we're not discussing radical changes. None of the changes
> > will be backwards-incompatible, but they will allow recent hardware to
> > be represented more correctly or conveniently.
> 
> Sure, but it is still inconsistent, one MMIO config mechansim is in
> ranges, one is in regs. Plus I don't think tegra's case is a great
> starting point to design a spec update, it's config access mechanism
> is especially strange...

Again, it is still the most accurate way to describe the hardware. I
know it's not terribly beautiful and doesn't match with what you'd
expect from PC hardware. However it is still a reality and something the
kernel will have to deal with. Marvell hardware isn't very pretty either
but that shouldn't exclude it from being supported by Linux.

> Anyhow, I think this has been hashed to death, as long as your final
> binding has the 'device_type = pci' on the pcie-controller node I
> think it will be fine.

No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
It is the instance that translates from the platform to the PCI bus. Why
should it be device_type = "pci"? And we've also been over this before,
making that change stops the proposed binding from working properly
because the entry in the reg property can't be translated into a CPU
addressable memory region.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 19:26                                                                                       ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 19:26 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 11:02:05AM -0600, Jason Gunthorpe wrote:
> On Wed, Mar 13, 2013 at 09:18:15AM +0100, Thierry Reding wrote:
> 
> > Mitch already answered this. The specification is now almost 15 years
> > old and it couldn't possibly have foreseen all of the future use-cases.
> > If the specification is too restrictive and Mitch gives his blessing to
> > remove some of the restrictions, I don't see any reason why we shouldn't
> > do so if it lets us represent the reality of hardware more accurately in
> > DT.
> 
> I understand the spec is old, and I have no problem with making a
> Linux specific revision, but do *that* - don't bury some random
> deviation inside the bindings for a driver. I even suggested some
> language, but I feel more thought is needed to consider how to model
> the standardized ECAM mechanism..

As Mitch already said there's very little chance that the specification
update will be ratified through IEEE, so I think that we might just as
well put a corresponding text somewhere below Documentation/devicetree.

Also note that this has absolutely nothing to do with ECAM. All I'm
proposing is to allow the reg property of a root port to be translated
into a CPU addressable memory region through the ranges property. This
turns out to actually work properly with the current OF core in Linux.

> > Furthermore we're not discussing radical changes. None of the changes
> > will be backwards-incompatible, but they will allow recent hardware to
> > be represented more correctly or conveniently.
> 
> Sure, but it is still inconsistent, one MMIO config mechansim is in
> ranges, one is in regs. Plus I don't think tegra's case is a great
> starting point to design a spec update, it's config access mechanism
> is especially strange...

Again, it is still the most accurate way to describe the hardware. I
know it's not terribly beautiful and doesn't match with what you'd
expect from PC hardware. However it is still a reality and something the
kernel will have to deal with. Marvell hardware isn't very pretty either
but that shouldn't exclude it from being supported by Linux.

> Anyhow, I think this has been hashed to death, as long as your final
> binding has the 'device_type = pci' on the pcie-controller node I
> think it will be fine.

No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
It is the instance that translates from the platform to the PCI bus. Why
should it be device_type = "pci"? And we've also been over this before,
making that change stops the proposed binding from working properly
because the entry in the reg property can't be translated into a CPU
addressable memory region.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130313/89f7970f/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 19:26                                                                                       ` Thierry Reding
@ 2013-03-13 19:59                                                                                         ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-13 19:59 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Wed, Mar 13, 2013 at 08:26:28PM +0100, Thierry Reding wrote:

> As Mitch already said there's very little chance that the specification
> update will be ratified through IEEE, so I think that we might just as
> well put a corresponding text somewhere below Documentation/devicetree.

Sure, I'm fine with that.
 
> Also note that this has absolutely nothing to do with ECAM. All I'm
> proposing is to allow the reg property of a root port to be translated
> into a CPU addressable memory region through the ranges property. This
> turns out to actually work properly with the current OF core in Linux.

ECAM is the only standard based mechanism that could possibly use this
new config space ranges concept. Whatever you do for tegra shouldn't
stomp on the possibility of making that work in the future. So at
least think about how ECAM could be modeled for a few mins before
going ahead.

Tegra is the non-standard unusual case here, an update to the PCI OF
binding should be of general use.

> > Anyhow, I think this has been hashed to death, as long as your final
> > binding has the 'device_type = pci' on the pcie-controller node I
> > think it will be fine.
> 
> No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
> It is the instance that translates from the platform to the PCI bus. Why
> should it be device_type = "pci"? 

Spec says so, Mitch said so, other PCI host bridge bindings in the
tree work like this. Do a grep in the powerpc tree, for instance.

arch/x86/platform/ce4100/falconfalls.dts is a pretty comprehensive
example.

> And we've also been over this before, making that change stops the
> proposed binding from working properly because the entry in the reg
> property can't be translated into a CPU addressable memory region.

Sigh, that's because the change makes things follow the OF PCI
spec. The Linux OF core follows the spec. Config space ranges binding
is specifically *defined to not work* in the spec. Why should it work
today?

You have to deal with this problem somehow, and omitting the top level
device_type is not a solution.

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 19:59                                                                                         ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-13 19:59 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 08:26:28PM +0100, Thierry Reding wrote:

> As Mitch already said there's very little chance that the specification
> update will be ratified through IEEE, so I think that we might just as
> well put a corresponding text somewhere below Documentation/devicetree.

Sure, I'm fine with that.
 
> Also note that this has absolutely nothing to do with ECAM. All I'm
> proposing is to allow the reg property of a root port to be translated
> into a CPU addressable memory region through the ranges property. This
> turns out to actually work properly with the current OF core in Linux.

ECAM is the only standard based mechanism that could possibly use this
new config space ranges concept. Whatever you do for tegra shouldn't
stomp on the possibility of making that work in the future. So at
least think about how ECAM could be modeled for a few mins before
going ahead.

Tegra is the non-standard unusual case here, an update to the PCI OF
binding should be of general use.

> > Anyhow, I think this has been hashed to death, as long as your final
> > binding has the 'device_type = pci' on the pcie-controller node I
> > think it will be fine.
> 
> No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
> It is the instance that translates from the platform to the PCI bus. Why
> should it be device_type = "pci"? 

Spec says so, Mitch said so, other PCI host bridge bindings in the
tree work like this. Do a grep in the powerpc tree, for instance.

arch/x86/platform/ce4100/falconfalls.dts is a pretty comprehensive
example.

> And we've also been over this before, making that change stops the
> proposed binding from working properly because the entry in the reg
> property can't be translated into a CPU addressable memory region.

Sigh, that's because the change makes things follow the OF PCI
spec. The Linux OF core follows the spec. Config space ranges binding
is specifically *defined to not work* in the spec. Why should it work
today?

You have to deal with this problem somehow, and omitting the top level
device_type is not a solution.

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 19:59                                                                                         ` Jason Gunthorpe
@ 2013-03-13 20:54                                                                                           ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 20:54 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 4250 bytes --]

On Wed, Mar 13, 2013 at 01:59:01PM -0600, Jason Gunthorpe wrote:
> On Wed, Mar 13, 2013 at 08:26:28PM +0100, Thierry Reding wrote:
> 
> > As Mitch already said there's very little chance that the specification
> > update will be ratified through IEEE, so I think that we might just as
> > well put a corresponding text somewhere below Documentation/devicetree.
> 
> Sure, I'm fine with that.
>  
> > Also note that this has absolutely nothing to do with ECAM. All I'm
> > proposing is to allow the reg property of a root port to be translated
> > into a CPU addressable memory region through the ranges property. This
> > turns out to actually work properly with the current OF core in Linux.
> 
> ECAM is the only standard based mechanism that could possibly use this
> new config space ranges concept. Whatever you do for tegra shouldn't
> stomp on the possibility of making that work in the future. So at
> least think about how ECAM could be modeled for a few mins before
> going ahead.
> 
> Tegra is the non-standard unusual case here, an update to the PCI OF
> binding should be of general use.

But I'm not trying to encode ECAM in the DT. ECAM isn't the problem at
all. The only problem is that each root port has a separate window in
the CPU address space that is used to access that port's configuration
space.

ECAM only comes into play when accessing the configuration space of
subordinate devices (via type 1 transactions). But at that point the
device tree doesn't need to know about it. The only thing that needs to
be in DT is the memory region that the controller should write to in
order to generate type 1 transactions. And that's a problem that has
long been solved.

> > > Anyhow, I think this has been hashed to death, as long as your final
> > > binding has the 'device_type = pci' on the pcie-controller node I
> > > think it will be fine.
> > 
> > No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
> > It is the instance that translates from the platform to the PCI bus. Why
> > should it be device_type = "pci"? 
> 
> Spec says so, Mitch said so, other PCI host bridge bindings in the
> tree work like this. Do a grep in the powerpc tree, for instance.
> 
> arch/x86/platform/ce4100/falconfalls.dts is a pretty comprehensive
> example.

Yes, yes. We all know by now what the spec says. The whole point is
that maybe the spec needs to be updated and this requirement lifted.
What I keep trying to say is that in case of Tegra the PCIe controller
does not provide the PCI bus. It doesn't appear as a device on the PCI
bus and it doesn't have configuration space itself.

Only the PCI root ports provide access to the subordinate busses. And
only they appear on the PCI bus.

> > And we've also been over this before, making that change stops the
> > proposed binding from working properly because the entry in the reg
> > property can't be translated into a CPU addressable memory region.
> 
> Sigh, that's because the change makes things follow the OF PCI
> spec. The Linux OF core follows the spec. Config space ranges binding
> is specifically *defined to not work* in the spec. Why should it work
> today?

Okay. I keep understanding what you're saying. But the whole point of
this discussion is that maybe the spec should be amended to allow this
to work. In fact it does work with the current implementation in the
Linux kernel. And what I've been trying to explain is that I think this
to be the most accurate description of the Tegra hardware.

> You have to deal with this problem somehow, and omitting the top level
> device_type is not a solution.

So we've already agreed that the specification might be too restrictive,
and I'm trying to solve this problem by finding an accurate description
in DT to represent the hardware without following the spec to the letter
because it wasn't written with this kind of hardware in mind.

Tegra's PCIe controller doesn't actually implement the bus. In fact we
have to jump through hoops to make both root ports appear on the root
bus specifically because there's no common access to the configuration
space that includes the host bridge or the root ports.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 20:54                                                                                           ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 20:54 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 01:59:01PM -0600, Jason Gunthorpe wrote:
> On Wed, Mar 13, 2013 at 08:26:28PM +0100, Thierry Reding wrote:
> 
> > As Mitch already said there's very little chance that the specification
> > update will be ratified through IEEE, so I think that we might just as
> > well put a corresponding text somewhere below Documentation/devicetree.
> 
> Sure, I'm fine with that.
>  
> > Also note that this has absolutely nothing to do with ECAM. All I'm
> > proposing is to allow the reg property of a root port to be translated
> > into a CPU addressable memory region through the ranges property. This
> > turns out to actually work properly with the current OF core in Linux.
> 
> ECAM is the only standard based mechanism that could possibly use this
> new config space ranges concept. Whatever you do for tegra shouldn't
> stomp on the possibility of making that work in the future. So at
> least think about how ECAM could be modeled for a few mins before
> going ahead.
> 
> Tegra is the non-standard unusual case here, an update to the PCI OF
> binding should be of general use.

But I'm not trying to encode ECAM in the DT. ECAM isn't the problem at
all. The only problem is that each root port has a separate window in
the CPU address space that is used to access that port's configuration
space.

ECAM only comes into play when accessing the configuration space of
subordinate devices (via type 1 transactions). But at that point the
device tree doesn't need to know about it. The only thing that needs to
be in DT is the memory region that the controller should write to in
order to generate type 1 transactions. And that's a problem that has
long been solved.

> > > Anyhow, I think this has been hashed to death, as long as your final
> > > binding has the 'device_type = pci' on the pcie-controller node I
> > > think it will be fine.
> > 
> > No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
> > It is the instance that translates from the platform to the PCI bus. Why
> > should it be device_type = "pci"? 
> 
> Spec says so, Mitch said so, other PCI host bridge bindings in the
> tree work like this. Do a grep in the powerpc tree, for instance.
> 
> arch/x86/platform/ce4100/falconfalls.dts is a pretty comprehensive
> example.

Yes, yes. We all know by now what the spec says. The whole point is
that maybe the spec needs to be updated and this requirement lifted.
What I keep trying to say is that in case of Tegra the PCIe controller
does not provide the PCI bus. It doesn't appear as a device on the PCI
bus and it doesn't have configuration space itself.

Only the PCI root ports provide access to the subordinate busses. And
only they appear on the PCI bus.

> > And we've also been over this before, making that change stops the
> > proposed binding from working properly because the entry in the reg
> > property can't be translated into a CPU addressable memory region.
> 
> Sigh, that's because the change makes things follow the OF PCI
> spec. The Linux OF core follows the spec. Config space ranges binding
> is specifically *defined to not work* in the spec. Why should it work
> today?

Okay. I keep understanding what you're saying. But the whole point of
this discussion is that maybe the spec should be amended to allow this
to work. In fact it does work with the current implementation in the
Linux kernel. And what I've been trying to explain is that I think this
to be the most accurate description of the Tegra hardware.

> You have to deal with this problem somehow, and omitting the top level
> device_type is not a solution.

So we've already agreed that the specification might be too restrictive,
and I'm trying to solve this problem by finding an accurate description
in DT to represent the hardware without following the spec to the letter
because it wasn't written with this kind of hardware in mind.

Tegra's PCIe controller doesn't actually implement the bus. In fact we
have to jump through hoops to make both root ports appear on the root
bus specifically because there's no common access to the configuration
space that includes the host bridge or the root ports.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130313/0239c411/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 19:26                                                                                       ` Thierry Reding
  (?)
@ 2013-03-13 20:58                                                                                         ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-13 20:58 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Jason Gunthorpe, Maen Suleiman, Shadi Ammouri,
	Bjorn Helgaas, Tawfik Bayouk, linux-arm-kernel

On 3/13/2013 9:26 AM, Thierry Reding wrote:
> On Wed, Mar 13, 2013 at 11:02:05AM -0600, Jason Gunthorpe wrote:
>> On Wed, Mar 13, 2013 at 09:18:15AM +0100, Thierry Reding wrote:
>>
>>> Mitch already answered this. The specification is now almost 15 years
>>> old and it couldn't possibly have foreseen all of the future use-cases.
>>> If the specification is too restrictive and Mitch gives his blessing to
>>> remove some of the restrictions, I don't see any reason why we shouldn't
>>> do so if it lets us represent the reality of hardware more accurately in
>>> DT.
>>
>> I understand the spec is old, and I have no problem with making a
>> Linux specific revision, but do *that* - don't bury some random
>> deviation inside the bindings for a driver. I even suggested some
>> language, but I feel more thought is needed to consider how to model
>> the standardized ECAM mechanism..
> 
> As Mitch already said there's very little chance that the specification
> update will be ratified through IEEE, so I think that we might just as
> well put a corresponding text somewhere below Documentation/devicetree.
> 
> Also note that this has absolutely nothing to do with ECAM. All I'm
> proposing is to allow the reg property of a root port to be translated
> into a CPU addressable memory region through the ranges property. This
> turns out to actually work properly with the current OF core in Linux.
> 
>>> Furthermore we're not discussing radical changes. None of the changes
>>> will be backwards-incompatible, but they will allow recent hardware to
>>> be represented more correctly or conveniently.
>>
>> Sure, but it is still inconsistent, one MMIO config mechansim is in
>> ranges, one is in regs. Plus I don't think tegra's case is a great
>> starting point to design a spec update, it's config access mechanism
>> is especially strange...
> 
> Again, it is still the most accurate way to describe the hardware. I
> know it's not terribly beautiful and doesn't match with what you'd
> expect from PC hardware. However it is still a reality and something the
> kernel will have to deal with. Marvell hardware isn't very pretty either
> but that shouldn't exclude it from being supported by Linux.
> 
>> Anyhow, I think this has been hashed to death, as long as your final
>> binding has the 'device_type = pci' on the pcie-controller node I
>> think it will be fine.
> 
> No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
> It is the instance that translates from the platform to the PCI bus. Why
> should it be device_type = "pci"? 

device_type="pci" means that this device implements PCI functionality,
which means that it is either a PCI host bridge or a PCI-to-PCI bridge.

"device_type" answers the question "What does this device do?".  The
alternative question "What bus does this device plug into?" is answered
by looking at the parent.

In this case, the answer to "what does pcie_controller do?" is "it
implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.

Having just re-read the code in drivers/of/address.c, I think I have a
deeper understanding of a few practical issues:

In order to translate a 3/2 PCI address via of_get_pci_address(), the
parent device (i.e. root complex / pcie-controller) must have both
'device_type = "pci"' (else of_match_bus() won't find the pci version of
"struct of_bus") and also 'name = "pci"' (else strcmp(bus->name, "pci")
will fail).

So it would seem that, if you want to use address.c verbatim (for the
interface between pcie-controller and its direct children), not only do
you need device_type = pci, but you also need to rename
"pcie-controller" to "pci".

But then you run into the problem that the pci variant of struct of_bus
uses "assigned-addresses" instead of "reg".  So it still doesn't work
as-is.  To make it work, you would need to add an "assigned-addresses"
property to each root port node.  That property value could be in
non-relocatable memory space, translatable via normal rules, in which
case the "map config space to MMIO via ranges" discussion is moot for
Marvell.

Which re-raises a question for which I have not yet heard a compelling
answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
its root ports (with their unusual programming model)?  The Marvell
hardware address structure between controller and root ports is
definitely not 3/2.  Certainly you need 3/2 addressing below
(implemented by) the root ports, but I just don't see the utility of
pretending 3/2 addressing at the complex-to-port interface.  If the root
port hardware used the common programming model, with real config
headers and stuff, 3/2 would be good because you could use existing
drivers.  But since you need a special root-port driver anyway, why go
to the trouble of emulating non-existent addressing?



And we've also been over this before,
> making that change stops the proposed binding from working properly
> because the entry in the reg property can't be translated into a CPU
> addressable memory region.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 20:58                                                                                         ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-13 20:58 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/13/2013 9:26 AM, Thierry Reding wrote:
> On Wed, Mar 13, 2013 at 11:02:05AM -0600, Jason Gunthorpe wrote:
>> On Wed, Mar 13, 2013 at 09:18:15AM +0100, Thierry Reding wrote:
>>
>>> Mitch already answered this. The specification is now almost 15 years
>>> old and it couldn't possibly have foreseen all of the future use-cases.
>>> If the specification is too restrictive and Mitch gives his blessing to
>>> remove some of the restrictions, I don't see any reason why we shouldn't
>>> do so if it lets us represent the reality of hardware more accurately in
>>> DT.
>>
>> I understand the spec is old, and I have no problem with making a
>> Linux specific revision, but do *that* - don't bury some random
>> deviation inside the bindings for a driver. I even suggested some
>> language, but I feel more thought is needed to consider how to model
>> the standardized ECAM mechanism..
> 
> As Mitch already said there's very little chance that the specification
> update will be ratified through IEEE, so I think that we might just as
> well put a corresponding text somewhere below Documentation/devicetree.
> 
> Also note that this has absolutely nothing to do with ECAM. All I'm
> proposing is to allow the reg property of a root port to be translated
> into a CPU addressable memory region through the ranges property. This
> turns out to actually work properly with the current OF core in Linux.
> 
>>> Furthermore we're not discussing radical changes. None of the changes
>>> will be backwards-incompatible, but they will allow recent hardware to
>>> be represented more correctly or conveniently.
>>
>> Sure, but it is still inconsistent, one MMIO config mechansim is in
>> ranges, one is in regs. Plus I don't think tegra's case is a great
>> starting point to design a spec update, it's config access mechanism
>> is especially strange...
> 
> Again, it is still the most accurate way to describe the hardware. I
> know it's not terribly beautiful and doesn't match with what you'd
> expect from PC hardware. However it is still a reality and something the
> kernel will have to deal with. Marvell hardware isn't very pretty either
> but that shouldn't exclude it from being supported by Linux.
> 
>> Anyhow, I think this has been hashed to death, as long as your final
>> binding has the 'device_type = pci' on the pcie-controller node I
>> think it will be fine.
> 
> No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
> It is the instance that translates from the platform to the PCI bus. Why
> should it be device_type = "pci"? 

device_type="pci" means that this device implements PCI functionality,
which means that it is either a PCI host bridge or a PCI-to-PCI bridge.

"device_type" answers the question "What does this device do?".  The
alternative question "What bus does this device plug into?" is answered
by looking at the parent.

In this case, the answer to "what does pcie_controller do?" is "it
implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.

Having just re-read the code in drivers/of/address.c, I think I have a
deeper understanding of a few practical issues:

In order to translate a 3/2 PCI address via of_get_pci_address(), the
parent device (i.e. root complex / pcie-controller) must have both
'device_type = "pci"' (else of_match_bus() won't find the pci version of
"struct of_bus") and also 'name = "pci"' (else strcmp(bus->name, "pci")
will fail).

So it would seem that, if you want to use address.c verbatim (for the
interface between pcie-controller and its direct children), not only do
you need device_type = pci, but you also need to rename
"pcie-controller" to "pci".

But then you run into the problem that the pci variant of struct of_bus
uses "assigned-addresses" instead of "reg".  So it still doesn't work
as-is.  To make it work, you would need to add an "assigned-addresses"
property to each root port node.  That property value could be in
non-relocatable memory space, translatable via normal rules, in which
case the "map config space to MMIO via ranges" discussion is moot for
Marvell.

Which re-raises a question for which I have not yet heard a compelling
answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
its root ports (with their unusual programming model)?  The Marvell
hardware address structure between controller and root ports is
definitely not 3/2.  Certainly you need 3/2 addressing below
(implemented by) the root ports, but I just don't see the utility of
pretending 3/2 addressing at the complex-to-port interface.  If the root
port hardware used the common programming model, with real config
headers and stuff, 3/2 would be good because you could use existing
drivers.  But since you need a special root-port driver anyway, why go
to the trouble of emulating non-existent addressing?



And we've also been over this before,
> making that change stops the proposed binding from working properly
> because the entry in the reg property can't be translated into a CPU
> addressable memory region.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 20:58                                                                                         ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-13 20:58 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/13/2013 9:26 AM, Thierry Reding wrote:
> On Wed, Mar 13, 2013 at 11:02:05AM -0600, Jason Gunthorpe wrote:
>> On Wed, Mar 13, 2013 at 09:18:15AM +0100, Thierry Reding wrote:
>>
>>> Mitch already answered this. The specification is now almost 15 years
>>> old and it couldn't possibly have foreseen all of the future use-cases.
>>> If the specification is too restrictive and Mitch gives his blessing to
>>> remove some of the restrictions, I don't see any reason why we shouldn't
>>> do so if it lets us represent the reality of hardware more accurately in
>>> DT.
>>
>> I understand the spec is old, and I have no problem with making a
>> Linux specific revision, but do *that* - don't bury some random
>> deviation inside the bindings for a driver. I even suggested some
>> language, but I feel more thought is needed to consider how to model
>> the standardized ECAM mechanism..
> 
> As Mitch already said there's very little chance that the specification
> update will be ratified through IEEE, so I think that we might just as
> well put a corresponding text somewhere below Documentation/devicetree.
> 
> Also note that this has absolutely nothing to do with ECAM. All I'm
> proposing is to allow the reg property of a root port to be translated
> into a CPU addressable memory region through the ranges property. This
> turns out to actually work properly with the current OF core in Linux.
> 
>>> Furthermore we're not discussing radical changes. None of the changes
>>> will be backwards-incompatible, but they will allow recent hardware to
>>> be represented more correctly or conveniently.
>>
>> Sure, but it is still inconsistent, one MMIO config mechansim is in
>> ranges, one is in regs. Plus I don't think tegra's case is a great
>> starting point to design a spec update, it's config access mechanism
>> is especially strange...
> 
> Again, it is still the most accurate way to describe the hardware. I
> know it's not terribly beautiful and doesn't match with what you'd
> expect from PC hardware. However it is still a reality and something the
> kernel will have to deal with. Marvell hardware isn't very pretty either
> but that shouldn't exclude it from being supported by Linux.
> 
>> Anyhow, I think this has been hashed to death, as long as your final
>> binding has the 'device_type = pci' on the pcie-controller node I
>> think it will be fine.
> 
> No. The pcie-controller is *not* a PCI device. It is a PCI host bridge.
> It is the instance that translates from the platform to the PCI bus. Why
> should it be device_type = "pci"? 

device_type="pci" means that this device implements PCI functionality,
which means that it is either a PCI host bridge or a PCI-to-PCI bridge.

"device_type" answers the question "What does this device do?".  The
alternative question "What bus does this device plug into?" is answered
by looking at the parent.

In this case, the answer to "what does pcie_controller do?" is "it
implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.

Having just re-read the code in drivers/of/address.c, I think I have a
deeper understanding of a few practical issues:

In order to translate a 3/2 PCI address via of_get_pci_address(), the
parent device (i.e. root complex / pcie-controller) must have both
'device_type = "pci"' (else of_match_bus() won't find the pci version of
"struct of_bus") and also 'name = "pci"' (else strcmp(bus->name, "pci")
will fail).

So it would seem that, if you want to use address.c verbatim (for the
interface between pcie-controller and its direct children), not only do
you need device_type = pci, but you also need to rename
"pcie-controller" to "pci".

But then you run into the problem that the pci variant of struct of_bus
uses "assigned-addresses" instead of "reg".  So it still doesn't work
as-is.  To make it work, you would need to add an "assigned-addresses"
property to each root port node.  That property value could be in
non-relocatable memory space, translatable via normal rules, in which
case the "map config space to MMIO via ranges" discussion is moot for
Marvell.

Which re-raises a question for which I have not yet heard a compelling
answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
its root ports (with their unusual programming model)?  The Marvell
hardware address structure between controller and root ports is
definitely not 3/2.  Certainly you need 3/2 addressing below
(implemented by) the root ports, but I just don't see the utility of
pretending 3/2 addressing at the complex-to-port interface.  If the root
port hardware used the common programming model, with real config
headers and stuff, 3/2 would be good because you could use existing
drivers.  But since you need a special root-port driver anyway, why go
to the trouble of emulating non-existent addressing?



And we've also been over this before,
> making that change stops the proposed binding from working properly
> because the entry in the reg property can't be translated into a CPU
> addressable memory region.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 20:58                                                                                         ` Mitch Bradley
@ 2013-03-13 21:33                                                                                           ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 21:33 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 3079 bytes --]

On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
[...]
> In this case, the answer to "what does pcie_controller do?" is "it
> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.

Alright, that's 2 against 1. I don't have much of a choice but to yield.

> Having just re-read the code in drivers/of/address.c, I think I have a
> deeper understanding of a few practical issues:
> 
> In order to translate a 3/2 PCI address via of_get_pci_address(), the
> parent device (i.e. root complex / pcie-controller) must have both
> 'device_type = "pci"' (else of_match_bus() won't find the pci version of
> "struct of_bus") and also 'name = "pci"' (else strcmp(bus->name, "pci")
> will fail).
> 
> So it would seem that, if you want to use address.c verbatim (for the
> interface between pcie-controller and its direct children), not only do
> you need device_type = pci, but you also need to rename
> "pcie-controller" to "pci".

bus->name refers to that of the of_bus structure, not the node name of
the controller.

> But then you run into the problem that the pci variant of struct of_bus
> uses "assigned-addresses" instead of "reg".  So it still doesn't work
> as-is.  To make it work, you would need to add an "assigned-addresses"
> property to each root port node.  That property value could be in
> non-relocatable memory space, translatable via normal rules, in which
> case the "map config space to MMIO via ranges" discussion is moot for
> Marvell.

At the risk of quoting the spec: it says (4.1.2) that the entries in the
"assigned-addresses" property correspond to the function's configuration
space BARs. In this case however that's no longer true. But if everybody
thinks that repurposing the "assigned-addresses" property is more
acceptable than leaving out the device_type = "pci" property from the
pcie-controller node, then so be it.

> Which re-raises a question for which I have not yet heard a compelling
> answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
> its root ports (with their unusual programming model)?  The Marvell
> hardware address structure between controller and root ports is
> definitely not 3/2.  Certainly you need 3/2 addressing below
> (implemented by) the root ports, but I just don't see the utility of
> pretending 3/2 addressing at the complex-to-port interface.  If the root
> port hardware used the common programming model, with real config
> headers and stuff, 3/2 would be good because you could use existing
> drivers.  But since you need a special root-port driver anyway, why go
> to the trouble of emulating non-existent addressing?

I think the reason is that reg needs to be in the standard format for
the device node matching code to work. There are various checks to make
sure the reg property has entries which are 3/2 each.

There's also the issue that the pcie-controller's ranges property
contains the mapping of the I/O and memory windows and therefore needs
the 3/2 addressing to encode the type.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 21:33                                                                                           ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 21:33 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
[...]
> In this case, the answer to "what does pcie_controller do?" is "it
> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.

Alright, that's 2 against 1. I don't have much of a choice but to yield.

> Having just re-read the code in drivers/of/address.c, I think I have a
> deeper understanding of a few practical issues:
> 
> In order to translate a 3/2 PCI address via of_get_pci_address(), the
> parent device (i.e. root complex / pcie-controller) must have both
> 'device_type = "pci"' (else of_match_bus() won't find the pci version of
> "struct of_bus") and also 'name = "pci"' (else strcmp(bus->name, "pci")
> will fail).
> 
> So it would seem that, if you want to use address.c verbatim (for the
> interface between pcie-controller and its direct children), not only do
> you need device_type = pci, but you also need to rename
> "pcie-controller" to "pci".

bus->name refers to that of the of_bus structure, not the node name of
the controller.

> But then you run into the problem that the pci variant of struct of_bus
> uses "assigned-addresses" instead of "reg".  So it still doesn't work
> as-is.  To make it work, you would need to add an "assigned-addresses"
> property to each root port node.  That property value could be in
> non-relocatable memory space, translatable via normal rules, in which
> case the "map config space to MMIO via ranges" discussion is moot for
> Marvell.

At the risk of quoting the spec: it says (4.1.2) that the entries in the
"assigned-addresses" property correspond to the function's configuration
space BARs. In this case however that's no longer true. But if everybody
thinks that repurposing the "assigned-addresses" property is more
acceptable than leaving out the device_type = "pci" property from the
pcie-controller node, then so be it.

> Which re-raises a question for which I have not yet heard a compelling
> answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
> its root ports (with their unusual programming model)?  The Marvell
> hardware address structure between controller and root ports is
> definitely not 3/2.  Certainly you need 3/2 addressing below
> (implemented by) the root ports, but I just don't see the utility of
> pretending 3/2 addressing at the complex-to-port interface.  If the root
> port hardware used the common programming model, with real config
> headers and stuff, 3/2 would be good because you could use existing
> drivers.  But since you need a special root-port driver anyway, why go
> to the trouble of emulating non-existent addressing?

I think the reason is that reg needs to be in the standard format for
the device node matching code to work. There are various checks to make
sure the reg property has entries which are 3/2 each.

There's also the issue that the pcie-controller's ranges property
contains the mapping of the I/O and memory windows and therefore needs
the 3/2 addressing to encode the type.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130313/8bd653a8/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 20:58                                                                                         ` Mitch Bradley
  (?)
@ 2013-03-13 22:02                                                                                             ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 22:02 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci-u79uwXL29TY76Z2rM5mHXA,
	devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ, Eran Ben-Avi,
	Jason Gunthorpe, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	Tawfik Bayouk, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r


[-- Attachment #1.1: Type: text/plain, Size: 3248 bytes --]

On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
[...]
> But then you run into the problem that the pci variant of struct of_bus
> uses "assigned-addresses" instead of "reg".  So it still doesn't work
> as-is.  To make it work, you would need to add an "assigned-addresses"
> property to each root port node.  That property value could be in
> non-relocatable memory space, translatable via normal rules, in which
> case the "map config space to MMIO via ranges" discussion is moot for
> Marvell.

So if I understand all of the above correctly, the Tegra binding should
look like this:

	pcie-controller {
		compatible = "nvidia,tegra20-pcie";
		device_type = "pci";
		reg = <0x80003000 0x00000800   /* PADS registers */
		       0x80003800 0x00000200   /* AFI registers */
		       0x90000000 0x10000000>; /* configuration space */
		reg-names = "pads", "afi", "cs";
		interrupts = <0 98 0x04   /* controller interrupt */
		              0 99 0x04>; /* MSI interrupt */
		interrupt-names = "intr", "msi";

		bus-range = <0x00 0xff>;
		#address-cells = <3>;
		#size-cells = <2>;

		ranges = <0x82000800 0 0 0x80000000 0 0x00001000   /* port 0 registers */
			  0x82001000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
			  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
			  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
			  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

		clocks = <&tegra_car 70>, <&tegra_car 72>, <&tegra_car 74>,
			 <&tegra_car 118>;
		clock-names = "pex", "afi", "pcie_xclk", "pll_e";
		status = "disabled";

		pci@1,0 {
			device_type = "pci";
			assigned-addresses = <0x82000800 0 0 0x80000000 0x1000>;
			reg = <0x000800 0 0 0 0>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};

		pci@2,0 {
			device_type = "pci";
			assigned-addresses = <0x82001000 0 0 0x80001000 0x1000>;
			reg = <0x001000 0 0 0 0>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};
	};

Does that look about correct?

One problem I see is that now the ranges property contains three entries
for 32-bit memory space:

	0x82000800 0 0 0x80000000 0 0x00001000   /* port 0 registers */
	0x82001000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
	0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */

But we used to rely on the ss field to parse the ranges property and
configure the windows in the controller properly so that accesses to
these regions will generate the correct transactions.

Now the code will actually match the first entry and assume that it
represents the non-prefetchable 32-bit memory space and program the
controller to forward accesses to the 0x80000000-0x80000fff region
as PCI memory write transactions.

I suppose we could add a check to verify that devfn == 0. However does
the above ranges property not imply that the brige on bus 0, device 1
has a 32-bit memory space from 0x80000000 to 0x80000fff and the bridge
on bus 0, device 2 has a 32-bit memory space from 0x80001000 to
0x80001fff? Which matches with the wording in the spec that the
"assigned-addresses" correspond to the bridge's BARs.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 22:02                                                                                             ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 22:02 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 3248 bytes --]

On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
[...]
> But then you run into the problem that the pci variant of struct of_bus
> uses "assigned-addresses" instead of "reg".  So it still doesn't work
> as-is.  To make it work, you would need to add an "assigned-addresses"
> property to each root port node.  That property value could be in
> non-relocatable memory space, translatable via normal rules, in which
> case the "map config space to MMIO via ranges" discussion is moot for
> Marvell.

So if I understand all of the above correctly, the Tegra binding should
look like this:

	pcie-controller {
		compatible = "nvidia,tegra20-pcie";
		device_type = "pci";
		reg = <0x80003000 0x00000800   /* PADS registers */
		       0x80003800 0x00000200   /* AFI registers */
		       0x90000000 0x10000000>; /* configuration space */
		reg-names = "pads", "afi", "cs";
		interrupts = <0 98 0x04   /* controller interrupt */
		              0 99 0x04>; /* MSI interrupt */
		interrupt-names = "intr", "msi";

		bus-range = <0x00 0xff>;
		#address-cells = <3>;
		#size-cells = <2>;

		ranges = <0x82000800 0 0 0x80000000 0 0x00001000   /* port 0 registers */
			  0x82001000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
			  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
			  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
			  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

		clocks = <&tegra_car 70>, <&tegra_car 72>, <&tegra_car 74>,
			 <&tegra_car 118>;
		clock-names = "pex", "afi", "pcie_xclk", "pll_e";
		status = "disabled";

		pci@1,0 {
			device_type = "pci";
			assigned-addresses = <0x82000800 0 0 0x80000000 0x1000>;
			reg = <0x000800 0 0 0 0>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};

		pci@2,0 {
			device_type = "pci";
			assigned-addresses = <0x82001000 0 0 0x80001000 0x1000>;
			reg = <0x001000 0 0 0 0>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};
	};

Does that look about correct?

One problem I see is that now the ranges property contains three entries
for 32-bit memory space:

	0x82000800 0 0 0x80000000 0 0x00001000   /* port 0 registers */
	0x82001000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
	0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */

But we used to rely on the ss field to parse the ranges property and
configure the windows in the controller properly so that accesses to
these regions will generate the correct transactions.

Now the code will actually match the first entry and assume that it
represents the non-prefetchable 32-bit memory space and program the
controller to forward accesses to the 0x80000000-0x80000fff region
as PCI memory write transactions.

I suppose we could add a check to verify that devfn == 0. However does
the above ranges property not imply that the brige on bus 0, device 1
has a 32-bit memory space from 0x80000000 to 0x80000fff and the bridge
on bus 0, device 2 has a 32-bit memory space from 0x80001000 to
0x80001fff? Which matches with the wording in the spec that the
"assigned-addresses" correspond to the bridge's BARs.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 22:02                                                                                             ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-13 22:02 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
[...]
> But then you run into the problem that the pci variant of struct of_bus
> uses "assigned-addresses" instead of "reg".  So it still doesn't work
> as-is.  To make it work, you would need to add an "assigned-addresses"
> property to each root port node.  That property value could be in
> non-relocatable memory space, translatable via normal rules, in which
> case the "map config space to MMIO via ranges" discussion is moot for
> Marvell.

So if I understand all of the above correctly, the Tegra binding should
look like this:

	pcie-controller {
		compatible = "nvidia,tegra20-pcie";
		device_type = "pci";
		reg = <0x80003000 0x00000800   /* PADS registers */
		       0x80003800 0x00000200   /* AFI registers */
		       0x90000000 0x10000000>; /* configuration space */
		reg-names = "pads", "afi", "cs";
		interrupts = <0 98 0x04   /* controller interrupt */
		              0 99 0x04>; /* MSI interrupt */
		interrupt-names = "intr", "msi";

		bus-range = <0x00 0xff>;
		#address-cells = <3>;
		#size-cells = <2>;

		ranges = <0x82000800 0 0 0x80000000 0 0x00001000   /* port 0 registers */
			  0x82001000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
			  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
			  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
			  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

		clocks = <&tegra_car 70>, <&tegra_car 72>, <&tegra_car 74>,
			 <&tegra_car 118>;
		clock-names = "pex", "afi", "pcie_xclk", "pll_e";
		status = "disabled";

		pci at 1,0 {
			device_type = "pci";
			assigned-addresses = <0x82000800 0 0 0x80000000 0x1000>;
			reg = <0x000800 0 0 0 0>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};

		pci at 2,0 {
			device_type = "pci";
			assigned-addresses = <0x82001000 0 0 0x80001000 0x1000>;
			reg = <0x001000 0 0 0 0>;
			status = "disabled";

			#address-cells = <3>;
			#size-cells = <2>;

			nvidia,num-lanes = <2>;
		};
	};

Does that look about correct?

One problem I see is that now the ranges property contains three entries
for 32-bit memory space:

	0x82000800 0 0 0x80000000 0 0x00001000   /* port 0 registers */
	0x82001000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
	0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */

But we used to rely on the ss field to parse the ranges property and
configure the windows in the controller properly so that accesses to
these regions will generate the correct transactions.

Now the code will actually match the first entry and assume that it
represents the non-prefetchable 32-bit memory space and program the
controller to forward accesses to the 0x80000000-0x80000fff region
as PCI memory write transactions.

I suppose we could add a check to verify that devfn == 0. However does
the above ranges property not imply that the brige on bus 0, device 1
has a 32-bit memory space from 0x80000000 to 0x80000fff and the bridge
on bus 0, device 2 has a 32-bit memory space from 0x80001000 to
0x80001fff? Which matches with the wording in the spec that the
"assigned-addresses" correspond to the bridge's BARs.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130313/0a8d2cfa/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 22:02                                                                                             ` Thierry Reding
@ 2013-03-13 22:21                                                                                               ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-13 22:21 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Wed, Mar 13, 2013 at 11:02:35PM +0100, Thierry Reding wrote:
 
> Does that look about correct?

By my reading of the spec the entries in ranges should not
have the b,d,f bits set..

Although it is not a big stretch to include them..

> Now the code will actually match the first entry and assume that it
> represents the non-prefetchable 32-bit memory space and program the
> controller to forward accesses to the 0x80000000-0x80000fff region
> as PCI memory write transactions.

Yah, it makes a mess of determining the host bridge aperture. Require
the aperture to be first/last?

Make use of the 2nd dword?

Go back to the idea of the top level reg?

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 22:21                                                                                               ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-13 22:21 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 11:02:35PM +0100, Thierry Reding wrote:
 
> Does that look about correct?

By my reading of the spec the entries in ranges should not
have the b,d,f bits set..

Although it is not a big stretch to include them..

> Now the code will actually match the first entry and assume that it
> represents the non-prefetchable 32-bit memory space and program the
> controller to forward accesses to the 0x80000000-0x80000fff region
> as PCI memory write transactions.

Yah, it makes a mess of determining the host bridge aperture. Require
the aperture to be first/last?

Make use of the 2nd dword?

Go back to the idea of the top level reg?

Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 20:58                                                                                         ` Mitch Bradley
@ 2013-03-13 22:22                                                                                           ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-13 22:22 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Thierry Reding, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:

> port hardware used the common programming model, with real config
> headers and stuff, 3/2 would be good because you could use existing
> drivers.  But since you need a special root-port driver anyway, why go
> to the trouble of emulating non-existent addressing?

Yes, I'm sorry, I've tried not to explore the complexity behind this
reasoning here.. Those discussions went on for a long time and I'm
tired of rehashing them :)

It isn't existing drivers that are valuable, it is the entire common
PCI infrastructure for dealing with discovery and address assignment
that is has value.

The common infrastructure can only allocate addresses within fixed
regions assigned to each PCI domain - and it divides those addreses
regions between ports in the root complex via the standard PCI-E root
port bridge memory windows.

Now, the largest SOC has 10 of the PCI-E ports in it, with 32 bit
addressing. It is essential that a small region of address space be
set aside for PCI-E use, and thus it is critical that resource
assignment be done across all ports, drawing on a common pool of
address space.

Thus two choices:
 1) Keep the PCI core the same and let the host bridge driver
    provide the missing configuration elements of the root
    complex
 2) Upgrade the PCI common core to deal with dynamic cross
    domain resource allocation, including hot plug.

Based on discussion/analysis #1 was chosen. Lots of reasons.

>From that choice flows the DT bindings, which are now required to
model the PCI bus 0.

Does that help explain the rational?

Cheers,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 22:22                                                                                           ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-13 22:22 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:

> port hardware used the common programming model, with real config
> headers and stuff, 3/2 would be good because you could use existing
> drivers.  But since you need a special root-port driver anyway, why go
> to the trouble of emulating non-existent addressing?

Yes, I'm sorry, I've tried not to explore the complexity behind this
reasoning here.. Those discussions went on for a long time and I'm
tired of rehashing them :)

It isn't existing drivers that are valuable, it is the entire common
PCI infrastructure for dealing with discovery and address assignment
that is has value.

The common infrastructure can only allocate addresses within fixed
regions assigned to each PCI domain - and it divides those addreses
regions between ports in the root complex via the standard PCI-E root
port bridge memory windows.

Now, the largest SOC has 10 of the PCI-E ports in it, with 32 bit
addressing. It is essential that a small region of address space be
set aside for PCI-E use, and thus it is critical that resource
assignment be done across all ports, drawing on a common pool of
address space.

Thus two choices:
 1) Keep the PCI core the same and let the host bridge driver
    provide the missing configuration elements of the root
    complex
 2) Upgrade the PCI common core to deal with dynamic cross
    domain resource allocation, including hot plug.

Based on discussion/analysis #1 was chosen. Lots of reasons.

>From that choice flows the DT bindings, which are now required to
model the PCI bus 0.

Does that help explain the rational?

Cheers,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 21:33                                                                                           ` Thierry Reding
@ 2013-03-13 22:48                                                                                             ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-13 22:48 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

On 3/13/2013 11:33 AM, Thierry Reding wrote:
> On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
> [...]
>> In this case, the answer to "what does pcie_controller do?" is "it
>> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.
> 
> Alright, that's 2 against 1. I don't have much of a choice but to yield.

All issues of "voting" aside, 'device_type = pci' is what tells
of_get_pci_address() to use the 3/2 interpretation.  So if you want a
node to implement 3/2 addresses, it must say device_type = pci, unless
you do address translation some other way.

> 
>> Having just re-read the code in drivers/of/address.c, I think I have a
>> deeper understanding of a few practical issues:
>>
>> In order to translate a 3/2 PCI address via of_get_pci_address(), the
>> parent device (i.e. root complex / pcie-controller) must have both
>> 'device_type = "pci"' (else of_match_bus() won't find the pci version of
>> "struct of_bus") and also 'name = "pci"' (else strcmp(bus->name, "pci")
>> will fail).
>>
>> So it would seem that, if you want to use address.c verbatim (for the
>> interface between pcie-controller and its direct children), not only do
>> you need device_type = pci, but you also need to rename
>> "pcie-controller" to "pci".
> 
> bus->name refers to that of the of_bus structure, not the node name of
> the controller.

D'oh!  Completely correct.  That's good; I didn't like the idea of
needing both device_type and name = pci.

> 
>> But then you run into the problem that the pci variant of struct of_bus
>> uses "assigned-addresses" instead of "reg".  So it still doesn't work
>> as-is.  To make it work, you would need to add an "assigned-addresses"
>> property to each root port node.  That property value could be in
>> non-relocatable memory space, translatable via normal rules, in which
>> case the "map config space to MMIO via ranges" discussion is moot for
>> Marvell.
> 
> At the risk of quoting the spec: it says (4.1.2) that the entries in the
> "assigned-addresses" property correspond to the function's configuration
> space BARs. In this case however that's no longer true.

Right, but we got into this mess by pretending that a PCI domain exists
despite the fact that the hardware is not PCI at that level, in order to
accommodate some existing Linux code.  Since we are already "faking it
to deal with existing code", it doesn't seem particularly strange to
take advantage of how existing code works, in order to accomplish the
forgery.

> But if everybody
> thinks that repurposing the "assigned-addresses" property is more
> acceptable than leaving out the device_type = "pci" property from the
> pcie-controller node, then so be it.

I'm confused again.  Why does leaving out device_type = pci make it work?

> 
>> Which re-raises a question for which I have not yet heard a compelling
>> answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
>> its root ports (with their unusual programming model)?  The Marvell
>> hardware address structure between controller and root ports is
>> definitely not 3/2.  Certainly you need 3/2 addressing below
>> (implemented by) the root ports, but I just don't see the utility of
>> pretending 3/2 addressing at the complex-to-port interface.  If the root
>> port hardware used the common programming model, with real config
>> headers and stuff, 3/2 would be good because you could use existing
>> drivers.  But since you need a special root-port driver anyway, why go
>> to the trouble of emulating non-existent addressing?
> 
> I think the reason is that reg needs to be in the standard format for
> the device node matching code to work. There are various checks to make
> sure the reg property has entries which are 3/2 each.

Could you point me to some of those checks so I can understand this better?

> 
> There's also the issue that the pcie-controller's ranges property
> contains the mapping of the I/O and memory windows and therefore needs
> the 3/2 addressing to encode the type.

Why can't that be done in each of the root port nodes?

> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-13 22:48                                                                                             ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-13 22:48 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/13/2013 11:33 AM, Thierry Reding wrote:
> On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
> [...]
>> In this case, the answer to "what does pcie_controller do?" is "it
>> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.
> 
> Alright, that's 2 against 1. I don't have much of a choice but to yield.

All issues of "voting" aside, 'device_type = pci' is what tells
of_get_pci_address() to use the 3/2 interpretation.  So if you want a
node to implement 3/2 addresses, it must say device_type = pci, unless
you do address translation some other way.

> 
>> Having just re-read the code in drivers/of/address.c, I think I have a
>> deeper understanding of a few practical issues:
>>
>> In order to translate a 3/2 PCI address via of_get_pci_address(), the
>> parent device (i.e. root complex / pcie-controller) must have both
>> 'device_type = "pci"' (else of_match_bus() won't find the pci version of
>> "struct of_bus") and also 'name = "pci"' (else strcmp(bus->name, "pci")
>> will fail).
>>
>> So it would seem that, if you want to use address.c verbatim (for the
>> interface between pcie-controller and its direct children), not only do
>> you need device_type = pci, but you also need to rename
>> "pcie-controller" to "pci".
> 
> bus->name refers to that of the of_bus structure, not the node name of
> the controller.

D'oh!  Completely correct.  That's good; I didn't like the idea of
needing both device_type and name = pci.

> 
>> But then you run into the problem that the pci variant of struct of_bus
>> uses "assigned-addresses" instead of "reg".  So it still doesn't work
>> as-is.  To make it work, you would need to add an "assigned-addresses"
>> property to each root port node.  That property value could be in
>> non-relocatable memory space, translatable via normal rules, in which
>> case the "map config space to MMIO via ranges" discussion is moot for
>> Marvell.
> 
> At the risk of quoting the spec: it says (4.1.2) that the entries in the
> "assigned-addresses" property correspond to the function's configuration
> space BARs. In this case however that's no longer true.

Right, but we got into this mess by pretending that a PCI domain exists
despite the fact that the hardware is not PCI at that level, in order to
accommodate some existing Linux code.  Since we are already "faking it
to deal with existing code", it doesn't seem particularly strange to
take advantage of how existing code works, in order to accomplish the
forgery.

> But if everybody
> thinks that repurposing the "assigned-addresses" property is more
> acceptable than leaving out the device_type = "pci" property from the
> pcie-controller node, then so be it.

I'm confused again.  Why does leaving out device_type = pci make it work?

> 
>> Which re-raises a question for which I have not yet heard a compelling
>> answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
>> its root ports (with their unusual programming model)?  The Marvell
>> hardware address structure between controller and root ports is
>> definitely not 3/2.  Certainly you need 3/2 addressing below
>> (implemented by) the root ports, but I just don't see the utility of
>> pretending 3/2 addressing at the complex-to-port interface.  If the root
>> port hardware used the common programming model, with real config
>> headers and stuff, 3/2 would be good because you could use existing
>> drivers.  But since you need a special root-port driver anyway, why go
>> to the trouble of emulating non-existent addressing?
> 
> I think the reason is that reg needs to be in the standard format for
> the device node matching code to work. There are various checks to make
> sure the reg property has entries which are 3/2 each.

Could you point me to some of those checks so I can understand this better?

> 
> There's also the issue that the pcie-controller's ranges property
> contains the mapping of the I/O and memory windows and therefore needs
> the 3/2 addressing to encode the type.

Why can't that be done in each of the root port nodes?

> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 22:48                                                                                             ` Mitch Bradley
@ 2013-03-14  0:43                                                                                               ` Rob Herring
  -1 siblings, 0 replies; 291+ messages in thread
From: Rob Herring @ 2013-03-14  0:43 UTC (permalink / raw)
  To: Mitch Bradley, Thierry Reding, Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Nadav Haklai, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Maen Suleiman, Shadi Ammouri, Bjorn Helgaas,
	Tawfik Bayouk, linux-arm-kernel

On 03/13/2013 05:48 PM, Mitch Bradley wrote:
> On 3/13/2013 11:33 AM, Thierry Reding wrote:
>> On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
>> [...]
>>> In this case, the answer to "what does pcie_controller do?" is "it
>>> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.
>>
>> Alright, that's 2 against 1. I don't have much of a choice but to yield.
> 
> All issues of "voting" aside, 'device_type = pci' is what tells
> of_get_pci_address() to use the 3/2 interpretation.  So if you want a
> node to implement 3/2 addresses, it must say device_type = pci, unless
> you do address translation some other way.

I should note that device_type is used for OF, but is supposed to not be
used for FDT as matching against compatible properties is preferred. I
don't have a good reason as to why, but Mitch may know the history.
However, there are numerous exceptions to that for compatibility and to
work with existing s/w. So this may be one of those cases.

Rob

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14  0:43                                                                                               ` Rob Herring
  0 siblings, 0 replies; 291+ messages in thread
From: Rob Herring @ 2013-03-14  0:43 UTC (permalink / raw)
  To: linux-arm-kernel

On 03/13/2013 05:48 PM, Mitch Bradley wrote:
> On 3/13/2013 11:33 AM, Thierry Reding wrote:
>> On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
>> [...]
>>> In this case, the answer to "what does pcie_controller do?" is "it
>>> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.
>>
>> Alright, that's 2 against 1. I don't have much of a choice but to yield.
> 
> All issues of "voting" aside, 'device_type = pci' is what tells
> of_get_pci_address() to use the 3/2 interpretation.  So if you want a
> node to implement 3/2 addresses, it must say device_type = pci, unless
> you do address translation some other way.

I should note that device_type is used for OF, but is supposed to not be
used for FDT as matching against compatible properties is preferred. I
don't have a good reason as to why, but Mitch may know the history.
However, there are numerous exceptions to that for compatibility and to
work with existing s/w. So this may be one of those cases.

Rob

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-14  0:43                                                                                               ` Rob Herring
@ 2013-03-14  1:20                                                                                                 ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-14  1:20 UTC (permalink / raw)
  To: Rob Herring
  Cc: Thierry Reding, Jason Gunthorpe, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Nadav Haklai, linux-pci,
	devicetree-discuss, Eran Ben-Avi, Maen Suleiman, Shadi Ammouri,
	Bjorn Helgaas, Tawfik Bayouk, linux-arm-kernel

On 3/13/2013 2:43 PM, Rob Herring wrote:
> On 03/13/2013 05:48 PM, Mitch Bradley wrote:
>> On 3/13/2013 11:33 AM, Thierry Reding wrote:
>>> On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
>>> [...]
>>>> In this case, the answer to "what does pcie_controller do?" is "it
>>>> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.
>>>
>>> Alright, that's 2 against 1. I don't have much of a choice but to yield.
>>
>> All issues of "voting" aside, 'device_type = pci' is what tells
>> of_get_pci_address() to use the 3/2 interpretation.  So if you want a
>> node to implement 3/2 addresses, it must say device_type = pci, unless
>> you do address translation some other way.
> 
> I should note that device_type is used for OF, but is supposed to not be
> used for FDT as matching against compatible properties is preferred. I
> don't have a good reason as to why, but Mitch may know the history.
> However, there are numerous exceptions to that for compatibility and to
> work with existing s/w. So this may be one of those cases.

Indeed.  The history is that, originally, the "name" property was
supposed to be a very precise name, like what is now the most specific
part of "compatible".  And "device_type" was "what, in general, does
this device do".

But with rule, pathnames quickly became unusable to humans:

  /marvell,mv87230/marvell,svxyz/adaptec,12345/wd,8876

What the heck is that?  Model names are like hashes; they have no rhyme
or reason and humans cannot remember them for long.  In order for a
random person to have a clue about what something was, it was much
better for pathname components to represent the generic function:

  /pci/pci/scsi/disk

Now it's clear that we are looking at a bridged PCI domain hosting a
SCSI disk. That's very useful for a system administrator; the
gobbledygook name is not.

Furthermore, in light of the facts that model numbers change much more
rapidly than programming interfaces, companies clone each others
programming interfaces, and companies change their names, a much more
capable way of describing programming models was needed.

So the "generic names" recommended practice changed things to make
"name" mean what "device_type" used to vaguely mean, "device_type" was
deprecated (it was never very precisely defined, and
mutually-conflicting usage patterns had developed), and "compatible" in
its present form was introduced.  "Generic names" was quite
controversial at the time - I think David Kahn wanted to strangle me at
one point - but it was the right thing.  (Not to mention the
embarrassing mistake of the underscore in device_type, mea culpa.)

That being the case, I personally would like for device_type to
disappear forever.  But the fact exists that address.c currently uses it
(and fdt.c duly parses it), which I why I said that, if you want to use
of_get_pci_address() in its present form, device_type=pci must be
present in the bus node that implements the 3/2 address domain.

So, yeah, it's compability with existing s/w, namely the PCI support in
address.c .

> 
> Rob
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14  1:20                                                                                                 ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-14  1:20 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/13/2013 2:43 PM, Rob Herring wrote:
> On 03/13/2013 05:48 PM, Mitch Bradley wrote:
>> On 3/13/2013 11:33 AM, Thierry Reding wrote:
>>> On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
>>> [...]
>>>> In this case, the answer to "what does pcie_controller do?" is "it
>>>> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.
>>>
>>> Alright, that's 2 against 1. I don't have much of a choice but to yield.
>>
>> All issues of "voting" aside, 'device_type = pci' is what tells
>> of_get_pci_address() to use the 3/2 interpretation.  So if you want a
>> node to implement 3/2 addresses, it must say device_type = pci, unless
>> you do address translation some other way.
> 
> I should note that device_type is used for OF, but is supposed to not be
> used for FDT as matching against compatible properties is preferred. I
> don't have a good reason as to why, but Mitch may know the history.
> However, there are numerous exceptions to that for compatibility and to
> work with existing s/w. So this may be one of those cases.

Indeed.  The history is that, originally, the "name" property was
supposed to be a very precise name, like what is now the most specific
part of "compatible".  And "device_type" was "what, in general, does
this device do".

But with rule, pathnames quickly became unusable to humans:

  /marvell,mv87230/marvell,svxyz/adaptec,12345/wd,8876

What the heck is that?  Model names are like hashes; they have no rhyme
or reason and humans cannot remember them for long.  In order for a
random person to have a clue about what something was, it was much
better for pathname components to represent the generic function:

  /pci/pci/scsi/disk

Now it's clear that we are looking at a bridged PCI domain hosting a
SCSI disk. That's very useful for a system administrator; the
gobbledygook name is not.

Furthermore, in light of the facts that model numbers change much more
rapidly than programming interfaces, companies clone each others
programming interfaces, and companies change their names, a much more
capable way of describing programming models was needed.

So the "generic names" recommended practice changed things to make
"name" mean what "device_type" used to vaguely mean, "device_type" was
deprecated (it was never very precisely defined, and
mutually-conflicting usage patterns had developed), and "compatible" in
its present form was introduced.  "Generic names" was quite
controversial at the time - I think David Kahn wanted to strangle me at
one point - but it was the right thing.  (Not to mention the
embarrassing mistake of the underscore in device_type, mea culpa.)

That being the case, I personally would like for device_type to
disappear forever.  But the fact exists that address.c currently uses it
(and fdt.c duly parses it), which I why I said that, if you want to use
of_get_pci_address() in its present form, device_type=pci must be
present in the bus node that implements the 3/2 address domain.

So, yeah, it's compability with existing s/w, namely the PCI support in
address.c .

> 
> Rob
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 21:33                                                                                           ` Thierry Reding
@ 2013-03-14  4:56                                                                                             ` Stephen Warren
  -1 siblings, 0 replies; 291+ messages in thread
From: Stephen Warren @ 2013-03-14  4:56 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, Lior Amsalem, Andrew Lunn,
	Russell King - ARM Linux, Jason Cooper, Nadav Haklai, linux-pci,
	devicetree-discuss, Eran Ben-Avi, Jason Gunthorpe, Maen Suleiman,
	Shadi Ammouri, Bjorn Helgaas, Tawfik Bayouk, linux-arm-kernel

On 03/13/2013 03:33 PM, Thierry Reding wrote:
> On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote: 
> [...]
>> In this case, the answer to "what does pcie_controller do?" is
>> "it implements a PCI bus" below.  So 'device_type = "pci"' is
>> appropriate.
> 
> Alright, that's 2 against 1. I don't have much of a choice but to
> yield.

Just one note here though re: how the Tegra HW works:

The Tegra "PCIe controller" HW translates from a SoC-internal bus to
another SoC-internal bus. The "PCIe root ports" translate from that
second SoC-internal bus to a PCIe bus. That's exactly why the PCIe
root port configuration registers don't show up via type 0 PCIe
configuration transactions. Thus, from a HW perspective, it really is
true that the PCIe root ports are PCI devices, but the "PCIe
controller" really isn't anything to do with PCIe.

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14  4:56                                                                                             ` Stephen Warren
  0 siblings, 0 replies; 291+ messages in thread
From: Stephen Warren @ 2013-03-14  4:56 UTC (permalink / raw)
  To: linux-arm-kernel

On 03/13/2013 03:33 PM, Thierry Reding wrote:
> On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote: 
> [...]
>> In this case, the answer to "what does pcie_controller do?" is
>> "it implements a PCI bus" below.  So 'device_type = "pci"' is
>> appropriate.
> 
> Alright, that's 2 against 1. I don't have much of a choice but to
> yield.

Just one note here though re: how the Tegra HW works:

The Tegra "PCIe controller" HW translates from a SoC-internal bus to
another SoC-internal bus. The "PCIe root ports" translate from that
second SoC-internal bus to a PCIe bus. That's exactly why the PCIe
root port configuration registers don't show up via type 0 PCIe
configuration transactions. Thus, from a HW perspective, it really is
true that the PCIe root ports are PCI devices, but the "PCIe
controller" really isn't anything to do with PCIe.

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 22:48                                                                                             ` Mitch Bradley
@ 2013-03-14  7:11                                                                                               ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14  7:11 UTC (permalink / raw)
  To: Mitch Bradley
  Cc: Jason Gunthorpe, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 6625 bytes --]

On Wed, Mar 13, 2013 at 12:48:28PM -1000, Mitch Bradley wrote:
> On 3/13/2013 11:33 AM, Thierry Reding wrote:
> > On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
> > [...]
> >> In this case, the answer to "what does pcie_controller do?" is "it
> >> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.
> > 
> > Alright, that's 2 against 1. I don't have much of a choice but to yield.
> 
> All issues of "voting" aside, 'device_type = pci' is what tells
> of_get_pci_address() to use the 3/2 interpretation.  So if you want a
> node to implement 3/2 addresses, it must say device_type = pci, unless
> you do address translation some other way.

of_bus_default_map() can also handle #address-cells > 2. We explicitly
added that special case back when the bindings still looked very
different and we needed to match the exact entry.

This is the reason why it actually works to not use device_type = "pci"
for the pcie-controller. For subordinate device address translation the
code will look at the parent bus, see that it isn't a PCI device and use
the default bus, which in turn will simply memcmp() the first 3 cells of
the "reg" property entry against those in the "ranges" property.

> >> But then you run into the problem that the pci variant of struct of_bus
> >> uses "assigned-addresses" instead of "reg".  So it still doesn't work
> >> as-is.  To make it work, you would need to add an "assigned-addresses"
> >> property to each root port node.  That property value could be in
> >> non-relocatable memory space, translatable via normal rules, in which
> >> case the "map config space to MMIO via ranges" discussion is moot for
> >> Marvell.
> > 
> > At the risk of quoting the spec: it says (4.1.2) that the entries in the
> > "assigned-addresses" property correspond to the function's configuration
> > space BARs. In this case however that's no longer true.
> 
> Right, but we got into this mess by pretending that a PCI domain exists
> despite the fact that the hardware is not PCI at that level, in order to
> accommodate some existing Linux code.

Actually at that level the hardware is indeed PCI. These entries are for
the root port nodes, which are in fact the bridges to the PCI bus. This
is what I've been trying to say all along. The PCIe controller is just a
collection of registers that allow CPU address space to be bridged to
an internal bus (FPCI, which is in fact a relic from the HyperTransport
roots of the controller IIUC) and setup/handle MSI.

The transition to the PCI fabric happens in the root ports. There is no
PCI host bridge.

> Since we are already "faking it to deal with existing code", it doesn't
> seem particularly strange to take advantage of how existing code works,
> in order to accomplish the forgery.

So we're back to square one. How is this kind of faking (repurposing the
"assigned-addresses" property) any better than translating the "reg"
property through an "ss = 00" entry in the "ranges" property?

> > But if everybody
> > thinks that repurposing the "assigned-addresses" property is more
> > acceptable than leaving out the device_type = "pci" property from the
> > pcie-controller node, then so be it.
> 
> I'm confused again.  Why does leaving out device_type = pci make it work?

I've already said this above. The default bus implementation handles the
case where #address-cells > 2 by comparing the first three cells, but
not go through the additional range matching that would otherwise need
to take place. Instead it treats the address cells as some kind of
identifier to match between the "reg" and "ranges" entries. That's very
much the way that the PCI bus implementation works, except for some
extra PCI-specific checks.

> >> Which re-raises a question for which I have not yet heard a compelling
> >> answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
> >> its root ports (with their unusual programming model)?  The Marvell
> >> hardware address structure between controller and root ports is
> >> definitely not 3/2.  Certainly you need 3/2 addressing below
> >> (implemented by) the root ports, but I just don't see the utility of
> >> pretending 3/2 addressing at the complex-to-port interface.  If the root
> >> port hardware used the common programming model, with real config
> >> headers and stuff, 3/2 would be good because you could use existing
> >> drivers.  But since you need a special root-port driver anyway, why go
> >> to the trouble of emulating non-existent addressing?
> > 
> > I think the reason is that reg needs to be in the standard format for
> > the device node matching code to work. There are various checks to make
> > sure the reg property has entries which are 3/2 each.
> 
> Could you point me to some of those checks so I can understand this better?

The code for that is in drivers/of/of_pci.c. The __of_pci_pci_compare()
function looks like this:

	static inline int __of_pci_pci_compare(struct device_node *node,
					       unsigned int devfn)
	{
		unsigned int size;
		const __be32 *reg = of_get_property(node, "reg", &size);

		if (!reg || size < 5 * sizeof(__be32))
			return 0;
		return ((be32_to_cpup(&reg[0]) >> 8) & 0xff) == devfn;
	}

This is the function that's called for each node to check whether it
corresponds to a given PCI device by matching the devfn to the value
found in the first cell of a device node's "reg" property.

> > There's also the issue that the pcie-controller's ranges property
> > contains the mapping of the I/O and memory windows and therefore needs
> > the 3/2 addressing to encode the type.
> 
> Why can't that be done in each of the root port nodes?

It could possibly be done, but we'd be lying. The way the hardware works
is that we configure a single window to map to each of the FPCI regions
for I/O and (non-)prefetchable memory spaces. So the pcie-controller
node's "ranges" property contains three entries, one for each of those
spaces which the driver uses to program the corresponding registers. The
same information is passed to the resource assignment code of the PCI
core so that it can partition these windows as appropriate and allocate
sub-windows that are assigned to the BARs of the root-ports.

Note that the programming of the root-port BARs is vital here. If memory
within one of the FPCI-mapped windows is accessed, the hardware will
look for a matching address in the root port BARs in order to determine
the port through which the transactions should be sent.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14  7:11                                                                                               ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14  7:11 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 12:48:28PM -1000, Mitch Bradley wrote:
> On 3/13/2013 11:33 AM, Thierry Reding wrote:
> > On Wed, Mar 13, 2013 at 10:58:02AM -1000, Mitch Bradley wrote:
> > [...]
> >> In this case, the answer to "what does pcie_controller do?" is "it
> >> implements a PCI bus" below.  So 'device_type = "pci"' is appropriate.
> > 
> > Alright, that's 2 against 1. I don't have much of a choice but to yield.
> 
> All issues of "voting" aside, 'device_type = pci' is what tells
> of_get_pci_address() to use the 3/2 interpretation.  So if you want a
> node to implement 3/2 addresses, it must say device_type = pci, unless
> you do address translation some other way.

of_bus_default_map() can also handle #address-cells > 2. We explicitly
added that special case back when the bindings still looked very
different and we needed to match the exact entry.

This is the reason why it actually works to not use device_type = "pci"
for the pcie-controller. For subordinate device address translation the
code will look at the parent bus, see that it isn't a PCI device and use
the default bus, which in turn will simply memcmp() the first 3 cells of
the "reg" property entry against those in the "ranges" property.

> >> But then you run into the problem that the pci variant of struct of_bus
> >> uses "assigned-addresses" instead of "reg".  So it still doesn't work
> >> as-is.  To make it work, you would need to add an "assigned-addresses"
> >> property to each root port node.  That property value could be in
> >> non-relocatable memory space, translatable via normal rules, in which
> >> case the "map config space to MMIO via ranges" discussion is moot for
> >> Marvell.
> > 
> > At the risk of quoting the spec: it says (4.1.2) that the entries in the
> > "assigned-addresses" property correspond to the function's configuration
> > space BARs. In this case however that's no longer true.
> 
> Right, but we got into this mess by pretending that a PCI domain exists
> despite the fact that the hardware is not PCI at that level, in order to
> accommodate some existing Linux code.

Actually at that level the hardware is indeed PCI. These entries are for
the root port nodes, which are in fact the bridges to the PCI bus. This
is what I've been trying to say all along. The PCIe controller is just a
collection of registers that allow CPU address space to be bridged to
an internal bus (FPCI, which is in fact a relic from the HyperTransport
roots of the controller IIUC) and setup/handle MSI.

The transition to the PCI fabric happens in the root ports. There is no
PCI host bridge.

> Since we are already "faking it to deal with existing code", it doesn't
> seem particularly strange to take advantage of how existing code works,
> in order to accomplish the forgery.

So we're back to square one. How is this kind of faking (repurposing the
"assigned-addresses" property) any better than translating the "reg"
property through an "ss = 00" entry in the "ranges" property?

> > But if everybody
> > thinks that repurposing the "assigned-addresses" property is more
> > acceptable than leaving out the device_type = "pci" property from the
> > pcie-controller node, then so be it.
> 
> I'm confused again.  Why does leaving out device_type = pci make it work?

I've already said this above. The default bus implementation handles the
case where #address-cells > 2 by comparing the first three cells, but
not go through the additional range matching that would otherwise need
to take place. Instead it treats the address cells as some kind of
identifier to match between the "reg" and "ranges" entries. That's very
much the way that the PCI bus implementation works, except for some
extra PCI-specific checks.

> >> Which re-raises a question for which I have not yet heard a compelling
> >> answer:  Why use 3/2 PCI addressing between Marvell pcie-controller and
> >> its root ports (with their unusual programming model)?  The Marvell
> >> hardware address structure between controller and root ports is
> >> definitely not 3/2.  Certainly you need 3/2 addressing below
> >> (implemented by) the root ports, but I just don't see the utility of
> >> pretending 3/2 addressing at the complex-to-port interface.  If the root
> >> port hardware used the common programming model, with real config
> >> headers and stuff, 3/2 would be good because you could use existing
> >> drivers.  But since you need a special root-port driver anyway, why go
> >> to the trouble of emulating non-existent addressing?
> > 
> > I think the reason is that reg needs to be in the standard format for
> > the device node matching code to work. There are various checks to make
> > sure the reg property has entries which are 3/2 each.
> 
> Could you point me to some of those checks so I can understand this better?

The code for that is in drivers/of/of_pci.c. The __of_pci_pci_compare()
function looks like this:

	static inline int __of_pci_pci_compare(struct device_node *node,
					       unsigned int devfn)
	{
		unsigned int size;
		const __be32 *reg = of_get_property(node, "reg", &size);

		if (!reg || size < 5 * sizeof(__be32))
			return 0;
		return ((be32_to_cpup(&reg[0]) >> 8) & 0xff) == devfn;
	}

This is the function that's called for each node to check whether it
corresponds to a given PCI device by matching the devfn to the value
found in the first cell of a device node's "reg" property.

> > There's also the issue that the pcie-controller's ranges property
> > contains the mapping of the I/O and memory windows and therefore needs
> > the 3/2 addressing to encode the type.
> 
> Why can't that be done in each of the root port nodes?

It could possibly be done, but we'd be lying. The way the hardware works
is that we configure a single window to map to each of the FPCI regions
for I/O and (non-)prefetchable memory spaces. So the pcie-controller
node's "ranges" property contains three entries, one for each of those
spaces which the driver uses to program the corresponding registers. The
same information is passed to the resource assignment code of the PCI
core so that it can partition these windows as appropriate and allocate
sub-windows that are assigned to the BARs of the root-ports.

Note that the programming of the root-port BARs is vital here. If memory
within one of the FPCI-mapped windows is accessed, the hardware will
look for a matching address in the root port BARs in order to determine
the port through which the transactions should be sent.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130314/52f7d3ec/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-13 22:21                                                                                               ` Jason Gunthorpe
  (?)
@ 2013-03-14  9:01                                                                                                 ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14  9:01 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Lior Amsalem, Andrew Lunn, Russell King - ARM Linux,
	Jason Cooper, Tawfik Bayouk, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Shadi Ammouri,
	Mitch Bradley, Bjorn Helgaas, linux-arm-kernel


[-- Attachment #1.1: Type: text/plain, Size: 2386 bytes --]

On Wed, Mar 13, 2013 at 04:21:02PM -0600, Jason Gunthorpe wrote:
> On Wed, Mar 13, 2013 at 11:02:35PM +0100, Thierry Reding wrote:
>  
> > Does that look about correct?
> 
> By my reading of the spec the entries in ranges should not
> have the b,d,f bits set..
> 
> Although it is not a big stretch to include them..
> 
> > Now the code will actually match the first entry and assume that it
> > represents the non-prefetchable 32-bit memory space and program the
> > controller to forward accesses to the 0x80000000-0x80000fff region
> > as PCI memory write transactions.
> 
> Yah, it makes a mess of determining the host bridge aperture. Require
> the aperture to be first/last?
> 
> Make use of the 2nd dword?
> 
> Go back to the idea of the top level reg?

It turns out that this works with the Tegra driver because it uses the
new of_pci_process_ranges() function and simply overwrites earlier
matches by subsequent ones.

	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

So the second entry would overwrite the non-prefetchable memory region
as established by the first entry. Finally that will also be overwritten
by the fourth entry.

So the above will actually work along with the corresponding root-port
"assigned-addresses" properties. I still don't like it much because I
don't think it accurately reflects the hardware. Furthermore it has the
same kludgy, non-spec conformant smack that my original proposal had
because it uses assigned-addresses for something it wasn't intended to.
Also the mapping is actually not correct, because the first two entries
don't refer to 32-bit memory space. They really map configuration space
and therefore the ss field is wrong IMO (it should be 00, but that won't
work with the OF core). One of the very early versions of my Tegra PCIe
patch series actually introduced an IORESOURCE_CS type to account for
this but it didn't go anywhere and became obsolete when I switched to
the new binding which side-steps this issue by not using the PCI bus'
translation mechanism.

Thierry

[-- Attachment #1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14  9:01                                                                                                 ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14  9:01 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, Lior Amsalem, Russell King - ARM Linux,
	Jason Cooper, Andrew Lunn, linux-pci, devicetree-discuss,
	Eran Ben-Avi, Nadav Haklai, Maen Suleiman, Bjorn Helgaas,
	Shadi Ammouri, Tawfik Bayouk, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 2386 bytes --]

On Wed, Mar 13, 2013 at 04:21:02PM -0600, Jason Gunthorpe wrote:
> On Wed, Mar 13, 2013 at 11:02:35PM +0100, Thierry Reding wrote:
>  
> > Does that look about correct?
> 
> By my reading of the spec the entries in ranges should not
> have the b,d,f bits set..
> 
> Although it is not a big stretch to include them..
> 
> > Now the code will actually match the first entry and assume that it
> > represents the non-prefetchable 32-bit memory space and program the
> > controller to forward accesses to the 0x80000000-0x80000fff region
> > as PCI memory write transactions.
> 
> Yah, it makes a mess of determining the host bridge aperture. Require
> the aperture to be first/last?
> 
> Make use of the 2nd dword?
> 
> Go back to the idea of the top level reg?

It turns out that this works with the Tegra driver because it uses the
new of_pci_process_ranges() function and simply overwrites earlier
matches by subsequent ones.

	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

So the second entry would overwrite the non-prefetchable memory region
as established by the first entry. Finally that will also be overwritten
by the fourth entry.

So the above will actually work along with the corresponding root-port
"assigned-addresses" properties. I still don't like it much because I
don't think it accurately reflects the hardware. Furthermore it has the
same kludgy, non-spec conformant smack that my original proposal had
because it uses assigned-addresses for something it wasn't intended to.
Also the mapping is actually not correct, because the first two entries
don't refer to 32-bit memory space. They really map configuration space
and therefore the ss field is wrong IMO (it should be 00, but that won't
work with the OF core). One of the very early versions of my Tegra PCIe
patch series actually introduced an IORESOURCE_CS type to account for
this but it didn't go anywhere and became obsolete when I switched to
the new binding which side-steps this issue by not using the PCI bus'
translation mechanism.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14  9:01                                                                                                 ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14  9:01 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Mar 13, 2013 at 04:21:02PM -0600, Jason Gunthorpe wrote:
> On Wed, Mar 13, 2013 at 11:02:35PM +0100, Thierry Reding wrote:
>  
> > Does that look about correct?
> 
> By my reading of the spec the entries in ranges should not
> have the b,d,f bits set..
> 
> Although it is not a big stretch to include them..
> 
> > Now the code will actually match the first entry and assume that it
> > represents the non-prefetchable 32-bit memory space and program the
> > controller to forward accesses to the 0x80000000-0x80000fff region
> > as PCI memory write transactions.
> 
> Yah, it makes a mess of determining the host bridge aperture. Require
> the aperture to be first/last?
> 
> Make use of the 2nd dword?
> 
> Go back to the idea of the top level reg?

It turns out that this works with the Tegra driver because it uses the
new of_pci_process_ranges() function and simply overwrites earlier
matches by subsequent ones.

	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

So the second entry would overwrite the non-prefetchable memory region
as established by the first entry. Finally that will also be overwritten
by the fourth entry.

So the above will actually work along with the corresponding root-port
"assigned-addresses" properties. I still don't like it much because I
don't think it accurately reflects the hardware. Furthermore it has the
same kludgy, non-spec conformant smack that my original proposal had
because it uses assigned-addresses for something it wasn't intended to.
Also the mapping is actually not correct, because the first two entries
don't refer to 32-bit memory space. They really map configuration space
and therefore the ss field is wrong IMO (it should be 00, but that won't
work with the OF core). One of the very early versions of my Tegra PCIe
patch series actually introduced an IORESOURCE_CS type to account for
this but it didn't go anywhere and became obsolete when I switched to
the new binding which side-steps this issue by not using the PCI bus'
translation mechanism.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130314/dd760e71/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-14  9:01                                                                                                 ` Thierry Reding
@ 2013-03-14 17:25                                                                                                   ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-14 17:25 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, linux-pci, devicetree-discuss, linux-arm-kernel

[trimm'd the cc list]

On Thu, Mar 14, 2013 at 10:01:20AM +0100, Thierry Reding wrote:

> It turns out that this works with the Tegra driver because it uses the
> new of_pci_process_ranges() function and simply overwrites earlier
> matches by subsequent ones.
> 
> 	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
> 		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
> 		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
> 		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
> 		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

Okay.. There is still something funny here, the 3rd dword of the child
address should not be 0 in every line and there shouldn't be overlaps
in the child address space.

I'm assuming 0x80000000, 0xa0000000 and 0xb0000000 are real CPU physical
addresses?

Then it should probably look like:

ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */

Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
address 0xa0000000' - this is the 'normal' case, I assume that is what
happens on tegra?

It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
with address 0' - this translation is something Linux typically
expects..

Then you'd go on to have:

               pci@1,0 {
                        device_type = "pci";
                        assigned-addresses = <0x82000000 0 0x80000000  0 0x1000>;
                        reg = <0x000800 0 0 0 0>;
               }
               pci@2,0 {
                        device_type = "pci";
                        assigned-addresses = <0x82000000 0 0x80001000  0 0x1000>;
                        reg = <0x001000 0 0 0 0>;
               }

Notice I've made the upper dw of assigned-addresses's size 0 and
included the full 3dw from the appropriate ranges line.

> So the above will actually work along with the corresponding root-port
> "assigned-addresses" properties. I still don't like it much because I
> don't think it accurately reflects the hardware. 

There are lots of valid ways to model the same HW :(

Bear in mind, for the PCI case - the OF PCI bindings model the HW
through the eyes of the abstractions in the PCI specification. That is
to say, they are not supposed to be an exact representation of the on
chip architecture.

Perhaps this would be clearer if you used 'pcie-root-complex' as the
name of the top level node?

> same kludgy, non-spec conformant smack that my original proposal had
> because it uses assigned-addresses for something it wasn't intended
> to.

Yes, only the top level 'reg' method avoids going outside any specs.

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14 17:25                                                                                                   ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-14 17:25 UTC (permalink / raw)
  To: linux-arm-kernel

[trimm'd the cc list]

On Thu, Mar 14, 2013 at 10:01:20AM +0100, Thierry Reding wrote:

> It turns out that this works with the Tegra driver because it uses the
> new of_pci_process_ranges() function and simply overwrites earlier
> matches by subsequent ones.
> 
> 	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
> 		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
> 		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
> 		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
> 		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */

Okay.. There is still something funny here, the 3rd dword of the child
address should not be 0 in every line and there shouldn't be overlaps
in the child address space.

I'm assuming 0x80000000, 0xa0000000 and 0xb0000000 are real CPU physical
addresses?

Then it should probably look like:

ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */

Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
address 0xa0000000' - this is the 'normal' case, I assume that is what
happens on tegra?

It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
with address 0' - this translation is something Linux typically
expects..

Then you'd go on to have:

               pci at 1,0 {
                        device_type = "pci";
                        assigned-addresses = <0x82000000 0 0x80000000  0 0x1000>;
                        reg = <0x000800 0 0 0 0>;
               }
               pci at 2,0 {
                        device_type = "pci";
                        assigned-addresses = <0x82000000 0 0x80001000  0 0x1000>;
                        reg = <0x001000 0 0 0 0>;
               }

Notice I've made the upper dw of assigned-addresses's size 0 and
included the full 3dw from the appropriate ranges line.

> So the above will actually work along with the corresponding root-port
> "assigned-addresses" properties. I still don't like it much because I
> don't think it accurately reflects the hardware. 

There are lots of valid ways to model the same HW :(

Bear in mind, for the PCI case - the OF PCI bindings model the HW
through the eyes of the abstractions in the PCI specification. That is
to say, they are not supposed to be an exact representation of the on
chip architecture.

Perhaps this would be clearer if you used 'pcie-root-complex' as the
name of the top level node?

> same kludgy, non-spec conformant smack that my original proposal had
> because it uses assigned-addresses for something it wasn't intended
> to.

Yes, only the top level 'reg' method avoids going outside any specs.

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-14 17:25                                                                                                   ` Jason Gunthorpe
@ 2013-03-14 20:38                                                                                                     ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14 20:38 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, linux-pci, devicetree-discuss, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 3852 bytes --]

On Thu, Mar 14, 2013 at 11:25:55AM -0600, Jason Gunthorpe wrote:
> [trimm'd the cc list]
> 
> On Thu, Mar 14, 2013 at 10:01:20AM +0100, Thierry Reding wrote:
> 
> > It turns out that this works with the Tegra driver because it uses the
> > new of_pci_process_ranges() function and simply overwrites earlier
> > matches by subsequent ones.
> > 
> > 	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
> > 		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
> > 		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
> > 		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
> > 		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */
> 
> Okay.. There is still something funny here, the 3rd dword of the child
> address should not be 0 in every line and there shouldn't be overlaps
> in the child address space.
> 
> I'm assuming 0x80000000, 0xa0000000 and 0xb0000000 are real CPU physical
> addresses?

Yes.

> Then it should probably look like:
> 
> ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
> 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
> 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
> 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
> 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
> 
> Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
> address 0xa0000000' - this is the 'normal' case, I assume that is what
> happens on tegra?
> 
> It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
> with address 0' - this translation is something Linux typically
> expects..
> 
> Then you'd go on to have:
> 
>                pci@1,0 {
>                         device_type = "pci";
>                         assigned-addresses = <0x82000000 0 0x80000000  0 0x1000>;
>                         reg = <0x000800 0 0 0 0>;
>                }
>                pci@2,0 {
>                         device_type = "pci";
>                         assigned-addresses = <0x82000000 0 0x80001000  0 0x1000>;
>                         reg = <0x001000 0 0 0 0>;
>                }
> 
> Notice I've made the upper dw of assigned-addresses's size 0 and
> included the full 3dw from the appropriate ranges line.

Okay, that all makes sense.

> > So the above will actually work along with the corresponding root-port
> > "assigned-addresses" properties. I still don't like it much because I
> > don't think it accurately reflects the hardware. 
> 
> There are lots of valid ways to model the same HW :(
> 
> Bear in mind, for the PCI case - the OF PCI bindings model the HW
> through the eyes of the abstractions in the PCI specification. That is
> to say, they are not supposed to be an exact representation of the on
> chip architecture.

That seems to be at odds with most other uses of DT that I've come
across. Generally the guideline seems to be to describe hardware
irrespective of the underlying implementation and leave it up to the
driver to translate the DT description into something the OS can use.

> Perhaps this would be clearer if you used 'pcie-root-complex' as the
> name of the top level node?

Perhaps. I'm not sure.

> > same kludgy, non-spec conformant smack that my original proposal had
> > because it uses assigned-addresses for something it wasn't intended
> > to.
> 
> Yes, only the top level 'reg' method avoids going outside any specs.

Yes. It has a couple of other disadvantages, though, so if what we've
been discussing here is in any way acceptable I'd rather go with that
solution, even if I'm not entirely happy about it either.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14 20:38                                                                                                     ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14 20:38 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 14, 2013 at 11:25:55AM -0600, Jason Gunthorpe wrote:
> [trimm'd the cc list]
> 
> On Thu, Mar 14, 2013 at 10:01:20AM +0100, Thierry Reding wrote:
> 
> > It turns out that this works with the Tegra driver because it uses the
> > new of_pci_process_ranges() function and simply overwrites earlier
> > matches by subsequent ones.
> > 
> > 	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
> > 		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
> > 		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
> > 		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
> > 		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */
> 
> Okay.. There is still something funny here, the 3rd dword of the child
> address should not be 0 in every line and there shouldn't be overlaps
> in the child address space.
> 
> I'm assuming 0x80000000, 0xa0000000 and 0xb0000000 are real CPU physical
> addresses?

Yes.

> Then it should probably look like:
> 
> ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
> 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
> 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
> 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
> 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
> 
> Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
> address 0xa0000000' - this is the 'normal' case, I assume that is what
> happens on tegra?
> 
> It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
> with address 0' - this translation is something Linux typically
> expects..
> 
> Then you'd go on to have:
> 
>                pci at 1,0 {
>                         device_type = "pci";
>                         assigned-addresses = <0x82000000 0 0x80000000  0 0x1000>;
>                         reg = <0x000800 0 0 0 0>;
>                }
>                pci at 2,0 {
>                         device_type = "pci";
>                         assigned-addresses = <0x82000000 0 0x80001000  0 0x1000>;
>                         reg = <0x001000 0 0 0 0>;
>                }
> 
> Notice I've made the upper dw of assigned-addresses's size 0 and
> included the full 3dw from the appropriate ranges line.

Okay, that all makes sense.

> > So the above will actually work along with the corresponding root-port
> > "assigned-addresses" properties. I still don't like it much because I
> > don't think it accurately reflects the hardware. 
> 
> There are lots of valid ways to model the same HW :(
> 
> Bear in mind, for the PCI case - the OF PCI bindings model the HW
> through the eyes of the abstractions in the PCI specification. That is
> to say, they are not supposed to be an exact representation of the on
> chip architecture.

That seems to be at odds with most other uses of DT that I've come
across. Generally the guideline seems to be to describe hardware
irrespective of the underlying implementation and leave it up to the
driver to translate the DT description into something the OS can use.

> Perhaps this would be clearer if you used 'pcie-root-complex' as the
> name of the top level node?

Perhaps. I'm not sure.

> > same kludgy, non-spec conformant smack that my original proposal had
> > because it uses assigned-addresses for something it wasn't intended
> > to.
> 
> Yes, only the top level 'reg' method avoids going outside any specs.

Yes. It has a couple of other disadvantages, though, so if what we've
been discussing here is in any way acceptable I'd rather go with that
solution, even if I'm not entirely happy about it either.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130314/b7e57fbb/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-14 20:38                                                                                                     ` Thierry Reding
@ 2013-03-14 21:05                                                                                                       ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-14 21:05 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, linux-pci, devicetree-discuss, linux-arm-kernel

On Thu, Mar 14, 2013 at 09:38:58PM +0100, Thierry Reding wrote:
> >                pci@1,0 {
> >                         device_type = "pci";
> >                         assigned-addresses = <0x82000000 0 0x80000000  0 0x1000>;

Sorry, I missed this. The b,d,f bits should be set:

                        assigned-addresses = <0x82000800 0 0x80000000  0 0x1000>;


> >                         reg = <0x000800 0 0 0 0>;
> >                }
> >                pci@2,0 {
> >                         device_type = "pci";
> >                         assigned-addresses = <0x82000000 0 0x80001000  0 0x1000>;

                         assigned-addresses = <0x82001000 0 0x80001000  0 0x1000>;


The 'ranges' stays the same, the PCI 3/2 parse will ignore the b,d,f
bits when comparing against ranges.

The use of r == 0 for assigned-addresses should be enough to
disambiguate this case from any future use of assigned-addresses - the
normal case has r == offset of the associated BAR != 0.

Cheers,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14 21:05                                                                                                       ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-14 21:05 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 14, 2013 at 09:38:58PM +0100, Thierry Reding wrote:
> >                pci at 1,0 {
> >                         device_type = "pci";
> >                         assigned-addresses = <0x82000000 0 0x80000000  0 0x1000>;

Sorry, I missed this. The b,d,f bits should be set:

                        assigned-addresses = <0x82000800 0 0x80000000  0 0x1000>;


> >                         reg = <0x000800 0 0 0 0>;
> >                }
> >                pci at 2,0 {
> >                         device_type = "pci";
> >                         assigned-addresses = <0x82000000 0 0x80001000  0 0x1000>;

                         assigned-addresses = <0x82001000 0 0x80001000  0 0x1000>;


The 'ranges' stays the same, the PCI 3/2 parse will ignore the b,d,f
bits when comparing against ranges.

The use of r == 0 for assigned-addresses should be enough to
disambiguate this case from any future use of assigned-addresses - the
normal case has r == offset of the associated BAR != 0.

Cheers,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-14 17:25                                                                                                   ` Jason Gunthorpe
@ 2013-03-14 21:09                                                                                                     ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14 21:09 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, linux-pci, devicetree-discuss, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

On Thu, Mar 14, 2013 at 11:25:55AM -0600, Jason Gunthorpe wrote:
[...]
> I'm assuming 0x80000000, 0xa0000000 and 0xb0000000 are real CPU physical
> addresses?
> 
> Then it should probably look like:
> 
> ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
> 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
> 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
> 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
> 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
> 
> Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
> address 0xa0000000' - this is the 'normal' case, I assume that is what
> happens on tegra?
> 
> It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
> with address 0' - this translation is something Linux typically
> expects..

Both of the above paragraphs are true. However accesses to the windows
at 0x80000000 and 0x80001000 don't generate memory TLPs but type 0
configuration space TLPs.

So my first instinct was to make the first cell of the first two entries
0, but that doesn't work, since the OF core expects to find either
memory or I/O spaces. ss == 2 isn't quite right here.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14 21:09                                                                                                     ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14 21:09 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 14, 2013 at 11:25:55AM -0600, Jason Gunthorpe wrote:
[...]
> I'm assuming 0x80000000, 0xa0000000 and 0xb0000000 are real CPU physical
> addresses?
> 
> Then it should probably look like:
> 
> ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
> 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
> 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
> 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
> 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
> 
> Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
> address 0xa0000000' - this is the 'normal' case, I assume that is what
> happens on tegra?
> 
> It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
> with address 0' - this translation is something Linux typically
> expects..

Both of the above paragraphs are true. However accesses to the windows
at 0x80000000 and 0x80001000 don't generate memory TLPs but type 0
configuration space TLPs.

So my first instinct was to make the first cell of the first two entries
0, but that doesn't work, since the OF core expects to find either
memory or I/O spaces. ss == 2 isn't quite right here.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130314/43cd68c5/attachment.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-14 20:38                                                                                                     ` Thierry Reding
@ 2013-03-14 21:10                                                                                                       ` Mitch Bradley
  -1 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-14 21:10 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Jason Gunthorpe, linux-pci, devicetree-discuss, linux-arm-kernel

On 3/14/2013 10:38 AM, Thierry Reding wrote:
> On Thu, Mar 14, 2013 at 11:25:55AM -0600, Jason Gunthorpe wrote:
>> [trimm'd the cc list]
>>
>> On Thu, Mar 14, 2013 at 10:01:20AM +0100, Thierry Reding wrote:
>>
>>> It turns out that this works with the Tegra driver because it uses the
>>> new of_pci_process_ranges() function and simply overwrites earlier
>>> matches by subsequent ones.
>>>
>>> 	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
>>> 		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
>>> 		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
>>> 		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
>>> 		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */
>>
>> Okay.. There is still something funny here, the 3rd dword of the child
>> address should not be 0 in every line and there shouldn't be overlaps
>> in the child address space.
>>
>> I'm assuming 0x80000000, 0xa0000000 and 0xb0000000 are real CPU physical
>> addresses?
> 
> Yes.
> 
>> Then it should probably look like:
>>
>> ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
>> 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
>> 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
>> 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
>> 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
>>
>> Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
>> address 0xa0000000' - this is the 'normal' case, I assume that is what
>> happens on tegra?
>>
>> It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
>> with address 0' - this translation is something Linux typically
>> expects..
>>
>> Then you'd go on to have:
>>
>>                pci@1,0 {
>>                         device_type = "pci";
>>                         assigned-addresses = <0x82000000 0 0x80000000  0 0x1000>;
>>                         reg = <0x000800 0 0 0 0>;
>>                }
>>                pci@2,0 {
>>                         device_type = "pci";
>>                         assigned-addresses = <0x82000000 0 0x80001000  0 0x1000>;
>>                         reg = <0x001000 0 0 0 0>;
>>                }
>>
>> Notice I've made the upper dw of assigned-addresses's size 0 and
>> included the full 3dw from the appropriate ranges line.
> 
> Okay, that all makes sense.
> 
>>> So the above will actually work along with the corresponding root-port
>>> "assigned-addresses" properties. I still don't like it much because I
>>> don't think it accurately reflects the hardware. 
>>
>> There are lots of valid ways to model the same HW :(
>>
>> Bear in mind, for the PCI case - the OF PCI bindings model the HW
>> through the eyes of the abstractions in the PCI specification. That is
>> to say, they are not supposed to be an exact representation of the on
>> chip architecture.
> 
> That seems to be at odds with most other uses of DT that I've come
> across. Generally the guideline seems to be to describe hardware
> irrespective of the underlying implementation and leave it up to the
> driver to translate the DT description into something the OS can use.

This can be a gray area where one just has to make a judgement call.
You can think of the hardware at various levels of abstraction and
sometimes it is not obvious where to draw the line.  "Tell the truth"
tends to be more stable in the long run, but can be more work in the
short run, and it can be hard to predict the future well enough to
choose the optimum approach.

> 
>> Perhaps this would be clearer if you used 'pcie-root-complex' as the
>> name of the top level node?
> 
> Perhaps. I'm not sure.
> 
>>> same kludgy, non-spec conformant smack that my original proposal had
>>> because it uses assigned-addresses for something it wasn't intended
>>> to.
>>
>> Yes, only the top level 'reg' method avoids going outside any specs.
> 
> Yes. It has a couple of other disadvantages, though, so if what we've
> been discussing here is in any way acceptable I'd rather go with that
> solution, even if I'm not entirely happy about it either.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14 21:10                                                                                                       ` Mitch Bradley
  0 siblings, 0 replies; 291+ messages in thread
From: Mitch Bradley @ 2013-03-14 21:10 UTC (permalink / raw)
  To: linux-arm-kernel

On 3/14/2013 10:38 AM, Thierry Reding wrote:
> On Thu, Mar 14, 2013 at 11:25:55AM -0600, Jason Gunthorpe wrote:
>> [trimm'd the cc list]
>>
>> On Thu, Mar 14, 2013 at 10:01:20AM +0100, Thierry Reding wrote:
>>
>>> It turns out that this works with the Tegra driver because it uses the
>>> new of_pci_process_ranges() function and simply overwrites earlier
>>> matches by subsequent ones.
>>>
>>> 	ranges = <0x82000000 0 0 0x80000000 0 0x00001000   /* port 0 registers */
>>> 		  0x82000000 0 0 0x80001000 0 0x00001000   /* port 1 registers */
>>> 		  0x81000000 0 0 0x82000000 0 0x00010000   /* downstream I/O */
>>> 		  0x82000000 0 0 0xa0000000 0 0x10000000   /* non-prefetchable memory */
>>> 		  0xc2000000 0 0 0xb0000000 0 0x10000000>; /* prefetchable memory */
>>
>> Okay.. There is still something funny here, the 3rd dword of the child
>> address should not be 0 in every line and there shouldn't be overlaps
>> in the child address space.
>>
>> I'm assuming 0x80000000, 0xa0000000 and 0xb0000000 are real CPU physical
>> addresses?
> 
> Yes.
> 
>> Then it should probably look like:
>>
>> ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
>> 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
>> 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
>> 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
>> 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
>>
>> Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
>> address 0xa0000000' - this is the 'normal' case, I assume that is what
>> happens on tegra?
>>
>> It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
>> with address 0' - this translation is something Linux typically
>> expects..
>>
>> Then you'd go on to have:
>>
>>                pci at 1,0 {
>>                         device_type = "pci";
>>                         assigned-addresses = <0x82000000 0 0x80000000  0 0x1000>;
>>                         reg = <0x000800 0 0 0 0>;
>>                }
>>                pci at 2,0 {
>>                         device_type = "pci";
>>                         assigned-addresses = <0x82000000 0 0x80001000  0 0x1000>;
>>                         reg = <0x001000 0 0 0 0>;
>>                }
>>
>> Notice I've made the upper dw of assigned-addresses's size 0 and
>> included the full 3dw from the appropriate ranges line.
> 
> Okay, that all makes sense.
> 
>>> So the above will actually work along with the corresponding root-port
>>> "assigned-addresses" properties. I still don't like it much because I
>>> don't think it accurately reflects the hardware. 
>>
>> There are lots of valid ways to model the same HW :(
>>
>> Bear in mind, for the PCI case - the OF PCI bindings model the HW
>> through the eyes of the abstractions in the PCI specification. That is
>> to say, they are not supposed to be an exact representation of the on
>> chip architecture.
> 
> That seems to be at odds with most other uses of DT that I've come
> across. Generally the guideline seems to be to describe hardware
> irrespective of the underlying implementation and leave it up to the
> driver to translate the DT description into something the OS can use.

This can be a gray area where one just has to make a judgement call.
You can think of the hardware at various levels of abstraction and
sometimes it is not obvious where to draw the line.  "Tell the truth"
tends to be more stable in the long run, but can be more work in the
short run, and it can be hard to predict the future well enough to
choose the optimum approach.

> 
>> Perhaps this would be clearer if you used 'pcie-root-complex' as the
>> name of the top level node?
> 
> Perhaps. I'm not sure.
> 
>>> same kludgy, non-spec conformant smack that my original proposal had
>>> because it uses assigned-addresses for something it wasn't intended
>>> to.
>>
>> Yes, only the top level 'reg' method avoids going outside any specs.
> 
> Yes. It has a couple of other disadvantages, though, so if what we've
> been discussing here is in any way acceptable I'd rather go with that
> solution, even if I'm not entirely happy about it either.
> 
> Thierry
> 

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-14 21:09                                                                                                     ` Thierry Reding
@ 2013-03-14 21:29                                                                                                       ` Jason Gunthorpe
  -1 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-14 21:29 UTC (permalink / raw)
  To: Thierry Reding
  Cc: Mitch Bradley, linux-pci, devicetree-discuss, linux-arm-kernel

On Thu, Mar 14, 2013 at 10:09:26PM +0100, Thierry Reding wrote:

> > ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
> > 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
> > 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
> > 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
> > 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
> > 
> > Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
> > address 0xa0000000' - this is the 'normal' case, I assume that is what
> > happens on tegra?
> > 
> > It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
> > with address 0' - this translation is something Linux typically
> > expects..
> 
> Both of the above paragraphs are true. However accesses to the windows
> at 0x80000000 and 0x80001000 don't generate memory TLPs but type 0
> configuration space TLPs.

By my understanding access to 0x80000000/0x80001000 doesn't generate
any externally visible TLPs?

IHMO modeling this register space as a controller-internal MMIO region
associated with the bridge is reasonable... After all, you are
iomapping it and accessing it with readl/writel - those are MMIO
functions..

> So my first instinct was to make the first cell of the first two entries
> 0, but that doesn't work, since the OF core expects to find either
> memory or I/O spaces. 

Right, there is no specified way to translate config ss through
ranges.

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14 21:29                                                                                                       ` Jason Gunthorpe
  0 siblings, 0 replies; 291+ messages in thread
From: Jason Gunthorpe @ 2013-03-14 21:29 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 14, 2013 at 10:09:26PM +0100, Thierry Reding wrote:

> > ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
> > 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
> > 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
> > 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
> > 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
> > 
> > Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
> > address 0xa0000000' - this is the 'normal' case, I assume that is what
> > happens on tegra?
> > 
> > It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
> > with address 0' - this translation is something Linux typically
> > expects..
> 
> Both of the above paragraphs are true. However accesses to the windows
> at 0x80000000 and 0x80001000 don't generate memory TLPs but type 0
> configuration space TLPs.

By my understanding access to 0x80000000/0x80001000 doesn't generate
any externally visible TLPs?

IHMO modeling this register space as a controller-internal MMIO region
associated with the bridge is reasonable... After all, you are
iomapping it and accessing it with readl/writel - those are MMIO
functions..

> So my first instinct was to make the first cell of the first two entries
> 0, but that doesn't work, since the OF core expects to find either
> memory or I/O spaces. 

Right, there is no specified way to translate config ss through
ranges.

Regards,
Jason

^ permalink raw reply	[flat|nested] 291+ messages in thread

* Re: [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
  2013-03-14 21:29                                                                                                       ` Jason Gunthorpe
@ 2013-03-14 21:37                                                                                                         ` Thierry Reding
  -1 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14 21:37 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Mitch Bradley, linux-pci, devicetree-discuss, linux-arm-kernel

[-- Attachment #1: Type: text/plain, Size: 1566 bytes --]

On Thu, Mar 14, 2013 at 03:29:21PM -0600, Jason Gunthorpe wrote:
> On Thu, Mar 14, 2013 at 10:09:26PM +0100, Thierry Reding wrote:
> 
> > > ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
> > > 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
> > > 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
> > > 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
> > > 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
> > > 
> > > Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
> > > address 0xa0000000' - this is the 'normal' case, I assume that is what
> > > happens on tegra?
> > > 
> > > It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
> > > with address 0' - this translation is something Linux typically
> > > expects..
> > 
> > Both of the above paragraphs are true. However accesses to the windows
> > at 0x80000000 and 0x80001000 don't generate memory TLPs but type 0
> > configuration space TLPs.
> 
> By my understanding access to 0x80000000/0x80001000 doesn't generate
> any externally visible TLPs?

Now that you mention it, that's probably correct, yes.

> IHMO modeling this register space as a controller-internal MMIO region
> associated with the bridge is reasonable... After all, you are
> iomapping it and accessing it with readl/writel - those are MMIO
> functions..

Yes, I think that'd be okay.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 291+ messages in thread

* [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems
@ 2013-03-14 21:37                                                                                                         ` Thierry Reding
  0 siblings, 0 replies; 291+ messages in thread
From: Thierry Reding @ 2013-03-14 21:37 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 14, 2013 at 03:29:21PM -0600, Jason Gunthorpe wrote:
> On Thu, Mar 14, 2013 at 10:09:26PM +0100, Thierry Reding wrote:
> 
> > > ranges = <0x82000000 0 0x80000000  0x80000000  0 0x00001000   /* port 0 registers */
> > > 	  0x82000000 0 0x80001000  0x80001000  0 0x00001000   /* port 1 registers */
> > > 	  0x81000000 0          0  0x82000000  0 0x00010000   /* downstream I/O */
> > > 	  0x82000000 0 0xa0000000  0xa0000000  0 0x10000000   /* non-prefetchable memory */
> > > 	  0xc2000000 0 0xb0000000  0xb0000000  0 0x10000000>; /* prefetchable memory */
> > > 
> > > Which says 'access to CPU address 0xa0000000 produces a PCI-E memory TLP with
> > > address 0xa0000000' - this is the 'normal' case, I assume that is what
> > > happens on tegra?
> > > 
> > > It also says 'access to CPU address 0x82000000 produces a PCI-E IO TLP
> > > with address 0' - this translation is something Linux typically
> > > expects..
> > 
> > Both of the above paragraphs are true. However accesses to the windows
> > at 0x80000000 and 0x80001000 don't generate memory TLPs but type 0
> > configuration space TLPs.
> 
> By my understanding access to 0x80000000/0x80001000 doesn't generate
> any externally visible TLPs?

Now that you mention it, that's probably correct, yes.

> IHMO modeling this register space as a controller-internal MMIO region
> associated with the bridge is reasonable... After all, you are
> iomapping it and accessing it with readl/writel - those are MMIO
> functions..

Yes, I think that'd be okay.

Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20130314/6ed5fe50/attachment-0001.sig>

^ permalink raw reply	[flat|nested] 291+ messages in thread

end of thread, other threads:[~2013-03-14 21:37 UTC | newest]

Thread overview: 291+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-02-12 16:28 [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs Thomas Petazzoni
2013-02-12 16:28 ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 01/32] of/pci: Provide support for parsing PCI DT ranges property Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 02/32] of/pci: Add of_pci_get_devfn() function Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 03/32] of/pci: Add of_pci_parse_bus_range() function Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 04/32] ARM: pci: Allow passing per-controller private data Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 05/32] lib: devres: don't enclose pcim_*() functions in CONFIG_HAS_IOPORT Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 18:00   ` Arnd Bergmann
2013-02-12 18:00     ` Arnd Bergmann
2013-02-12 18:58     ` Thomas Petazzoni
2013-02-12 18:58       ` Thomas Petazzoni
2013-02-12 22:36       ` Arnd Bergmann
2013-02-12 22:36         ` Arnd Bergmann
2013-03-04 16:28         ` Thomas Petazzoni
2013-03-04 16:28           ` Thomas Petazzoni
2013-03-04 20:30           ` Arnd Bergmann
2013-03-04 20:30             ` Arnd Bergmann
2013-02-12 16:28 ` [PATCH 06/32] arm: pci: add a align_resource hook Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 18:03   ` Arnd Bergmann
2013-02-12 18:03     ` Arnd Bergmann
2013-02-12 19:01     ` Thomas Petazzoni
2013-02-12 19:01       ` Thomas Petazzoni
2013-02-12 19:49       ` Russell King - ARM Linux
2013-02-12 19:49         ` Russell King - ARM Linux
2013-02-12 16:28 ` [PATCH 07/32] arm: mvebu: fix address-cells in mpic DT node Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 08/32] clk: mvebu: create parent-child relation for PCIe clocks on Armada 370 Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 09/32] clk: mvebu: add more PCIe clocks for Armada XP Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 10/32] arm: plat-orion: introduce WIN_CTRL_ENABLE in address mapping code Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 11/32] arm: plat-orion: refactor the orion_disable_wins() function Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 12/32] plat-orion: introduce ORION_ADDR_MAP_NO_REMAP Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 13/32] arm: mach-dove: use ORION_ADDR_MAP_NO_REMAP Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 14/32] arm: mach-kirkwood: " Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 15/32] arm: mach-mvebu: " Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 16/32] arm: mach-orion5x: " Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 17/32] arm: plat-orion: convert 'int remap' to 'u32 remap' Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 18/32] arm: plat-orion: remove __init from addr-map functions needed after boot time Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 19/32] arm: plat-orion: introduce orion_{alloc,free}_cpu_win() functions Thomas Petazzoni
2013-02-12 16:28   ` [PATCH 19/32] arm: plat-orion: introduce orion_{alloc, free}_cpu_win() functions Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 20/32] arm: plat-orion: remove __init from PCIe functions needed after boot time Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 21/32] arm: mvebu: add functions to alloc/free PCIe decoding windows Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 22/32] arm: plat-orion: make common PCIe code usable on mvebu Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 23/32] pci: infrastructure to add drivers in drivers/pci/host Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:28 ` [PATCH 24/32] pci: PCIe driver for Marvell Armada 370/XP systems Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 18:30   ` Arnd Bergmann
2013-02-12 18:30     ` Arnd Bergmann
2013-02-12 19:22     ` Thomas Petazzoni
2013-02-12 19:22       ` Thomas Petazzoni
2013-02-12 19:49       ` Jason Gunthorpe
2013-02-12 19:49         ` Jason Gunthorpe
2013-02-12 22:59       ` Arnd Bergmann
2013-02-12 22:59         ` Arnd Bergmann
2013-02-13  0:41         ` Jason Gunthorpe
2013-02-13  0:41           ` Jason Gunthorpe
2013-02-13  9:18           ` Arnd Bergmann
2013-02-13  9:18             ` Arnd Bergmann
2013-02-13  9:31             ` Thomas Petazzoni
2013-02-13  9:31               ` Thomas Petazzoni
2013-02-13 10:23               ` Arnd Bergmann
2013-02-13 10:23                 ` Arnd Bergmann
2013-02-13  8:23         ` Thomas Petazzoni
2013-02-13  8:23           ` Thomas Petazzoni
2013-02-13  9:29           ` Arnd Bergmann
2013-02-13  9:29             ` Arnd Bergmann
2013-02-13  9:40             ` Thomas Petazzoni
2013-02-13  9:40               ` Thomas Petazzoni
2013-02-13 10:37               ` Arnd Bergmann
2013-02-13 10:37                 ` Arnd Bergmann
2013-03-06  9:50                 ` Thomas Petazzoni
2013-03-06  9:50                   ` Thomas Petazzoni
2013-03-06 10:43                   ` Arnd Bergmann
2013-03-06 10:43                     ` Arnd Bergmann
2013-02-12 22:35   ` Jason Gunthorpe
2013-02-12 22:35     ` Jason Gunthorpe
2013-02-13  8:57     ` Thomas Petazzoni
2013-02-13  8:57       ` Thomas Petazzoni
2013-02-13 18:04       ` Jason Gunthorpe
2013-02-13 18:04         ` Jason Gunthorpe
2013-02-13 19:33         ` Arnd Bergmann
2013-02-13 19:33           ` Arnd Bergmann
2013-03-06  9:54     ` Thomas Petazzoni
2013-03-06  9:54       ` Thomas Petazzoni
2013-03-06 12:11       ` Thierry Reding
2013-03-06 12:11         ` Thierry Reding
2013-03-06 18:09         ` Jason Gunthorpe
2013-03-06 18:09           ` Jason Gunthorpe
2013-03-07  8:08           ` Thierry Reding
2013-03-07  8:08             ` Thierry Reding
2013-03-07 17:49             ` Jason Gunthorpe
2013-03-07 17:49               ` Jason Gunthorpe
2013-03-07 19:48               ` Thierry Reding
2013-03-07 19:48                 ` Thierry Reding
2013-03-07 20:02                 ` Jason Gunthorpe
2013-03-07 20:02                   ` Jason Gunthorpe
     [not found]                   ` <20130307200235.GB20695-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2013-03-07 20:47                     ` Thierry Reding
2013-03-07 20:47                       ` Thierry Reding
2013-03-07 20:47                       ` Thierry Reding
2013-03-08  0:05                       ` Rob Herring
2013-03-08  0:05                         ` Rob Herring
2013-03-08  7:14                         ` Thierry Reding
2013-03-08  7:14                           ` Thierry Reding
2013-03-08 16:52                           ` Jason Gunthorpe
2013-03-08 16:52                             ` Jason Gunthorpe
2013-03-08 19:12                             ` Thierry Reding
2013-03-08 19:12                               ` Thierry Reding
     [not found]                               ` <20130308191227.GA6551-RM9K5IK7kjIQXX3q8xo1gnVAuStQJXxyR5q1nwbD4aMs9pC9oP6+/A@public.gmane.org>
2013-03-08 19:43                                 ` Mitch Bradley
2013-03-08 19:43                                   ` Mitch Bradley
2013-03-08 19:43                                   ` Mitch Bradley
2013-03-08 20:02                                   ` Jason Gunthorpe
2013-03-08 20:02                                     ` Jason Gunthorpe
     [not found]                                     ` <20130308200245.GC29435-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2013-03-08 20:13                                       ` Thierry Reding
2013-03-08 20:13                                         ` Thierry Reding
2013-03-08 20:13                                         ` Thierry Reding
2013-03-10 15:09                                         ` Thomas Petazzoni
2013-03-10 15:09                                           ` Thomas Petazzoni
2013-03-11  8:08                                           ` Thierry Reding
2013-03-11  8:08                                             ` Thierry Reding
2013-03-08 23:46                                       ` Mitch Bradley
2013-03-08 23:46                                         ` Mitch Bradley
2013-03-08 23:46                                         ` Mitch Bradley
2013-03-09  1:31                                         ` Jason Gunthorpe
2013-03-09  1:31                                           ` Jason Gunthorpe
     [not found]                                           ` <20130309013152.GA3883-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2013-03-10  4:52                                             ` Mitch Bradley
2013-03-10  4:52                                               ` Mitch Bradley
2013-03-10  4:52                                               ` Mitch Bradley
2013-03-10  6:55                                               ` Jason Gunthorpe
2013-03-10  6:55                                                 ` Jason Gunthorpe
     [not found]                                                 ` <20130310065539.GA14704-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2013-03-11  5:46                                                   ` Mitch Bradley
2013-03-11  5:46                                                     ` Mitch Bradley
2013-03-11  5:46                                                     ` Mitch Bradley
     [not found]                                                     ` <513D6F9C.9000100-D5eQfiDGL7eakBO8gow8eQ@public.gmane.org>
2013-03-11  7:46                                                       ` Thierry Reding
2013-03-11  7:46                                                         ` Thierry Reding
2013-03-11  7:46                                                         ` Thierry Reding
     [not found]                                                         ` <20130311074615.GA6365-RM9K5IK7kjIQXX3q8xo1gnVAuStQJXxyR5q1nwbD4aMs9pC9oP6+/A@public.gmane.org>
2013-03-11 18:04                                                           ` Mitch Bradley
2013-03-11 18:04                                                             ` Mitch Bradley
2013-03-11 18:04                                                             ` Mitch Bradley
2013-03-11 18:23                                                             ` Jason Gunthorpe
2013-03-11 18:23                                                               ` Jason Gunthorpe
     [not found]                                                               ` <20130311182339.GB10992-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2013-03-11 19:49                                                                 ` Mitch Bradley
2013-03-11 19:49                                                                   ` Mitch Bradley
2013-03-11 19:49                                                                   ` Mitch Bradley
2013-03-11 18:15                                                     ` Jason Gunthorpe
2013-03-11 18:15                                                       ` Jason Gunthorpe
     [not found]                                                       ` <20130311181554.GA10992-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2013-03-11 21:50                                                         ` Mitch Bradley
2013-03-11 21:50                                                           ` Mitch Bradley
2013-03-11 21:50                                                           ` Mitch Bradley
2013-03-11 23:25                                                           ` Jason Gunthorpe
2013-03-11 23:25                                                             ` Jason Gunthorpe
     [not found]                                                             ` <20130311232516.GA13873-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2013-03-11 23:38                                                               ` Mitch Bradley
2013-03-11 23:38                                                                 ` Mitch Bradley
2013-03-11 23:38                                                                 ` Mitch Bradley
     [not found]                                                                 ` <513E6AFE.3090304-D5eQfiDGL7eakBO8gow8eQ@public.gmane.org>
2013-03-12  7:08                                                                   ` Thierry Reding
2013-03-12  7:08                                                                     ` Thierry Reding
2013-03-12  7:08                                                                     ` Thierry Reding
2013-03-12 15:57                                                                     ` Jason Gunthorpe
2013-03-12 15:57                                                                       ` Jason Gunthorpe
2013-03-12 20:38                                                                       ` Thierry Reding
2013-03-12 20:38                                                                         ` Thierry Reding
2013-03-12 21:03                                                                         ` Jason Gunthorpe
2013-03-12 21:03                                                                           ` Jason Gunthorpe
2013-03-12 21:30                                                                           ` Thierry Reding
2013-03-12 21:30                                                                             ` Thierry Reding
2013-03-12 22:08                                                                             ` Jason Gunthorpe
2013-03-12 22:08                                                                               ` Jason Gunthorpe
2013-03-12 23:25                                                                               ` Mitch Bradley
2013-03-12 23:25                                                                                 ` Mitch Bradley
2013-03-12 23:25                                                                                 ` Mitch Bradley
2013-03-13  8:18                                                                               ` Thierry Reding
2013-03-13  8:18                                                                                 ` Thierry Reding
2013-03-13  8:18                                                                                 ` Thierry Reding
2013-03-13 17:02                                                                                 ` Jason Gunthorpe
2013-03-13 17:02                                                                                   ` Jason Gunthorpe
     [not found]                                                                                   ` <20130313170205.GB24042-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2013-03-13 19:26                                                                                     ` Thierry Reding
2013-03-13 19:26                                                                                       ` Thierry Reding
2013-03-13 19:26                                                                                       ` Thierry Reding
2013-03-13 19:59                                                                                       ` Jason Gunthorpe
2013-03-13 19:59                                                                                         ` Jason Gunthorpe
2013-03-13 20:54                                                                                         ` Thierry Reding
2013-03-13 20:54                                                                                           ` Thierry Reding
2013-03-13 20:58                                                                                       ` Mitch Bradley
2013-03-13 20:58                                                                                         ` Mitch Bradley
2013-03-13 20:58                                                                                         ` Mitch Bradley
2013-03-13 21:33                                                                                         ` Thierry Reding
2013-03-13 21:33                                                                                           ` Thierry Reding
2013-03-13 22:48                                                                                           ` Mitch Bradley
2013-03-13 22:48                                                                                             ` Mitch Bradley
2013-03-14  0:43                                                                                             ` Rob Herring
2013-03-14  0:43                                                                                               ` Rob Herring
2013-03-14  1:20                                                                                               ` Mitch Bradley
2013-03-14  1:20                                                                                                 ` Mitch Bradley
2013-03-14  7:11                                                                                             ` Thierry Reding
2013-03-14  7:11                                                                                               ` Thierry Reding
2013-03-14  4:56                                                                                           ` Stephen Warren
2013-03-14  4:56                                                                                             ` Stephen Warren
     [not found]                                                                                         ` <5140E85A.3040900-D5eQfiDGL7eakBO8gow8eQ@public.gmane.org>
2013-03-13 22:02                                                                                           ` Thierry Reding
2013-03-13 22:02                                                                                             ` Thierry Reding
2013-03-13 22:02                                                                                             ` Thierry Reding
2013-03-13 22:21                                                                                             ` Jason Gunthorpe
2013-03-13 22:21                                                                                               ` Jason Gunthorpe
2013-03-14  9:01                                                                                               ` Thierry Reding
2013-03-14  9:01                                                                                                 ` Thierry Reding
2013-03-14  9:01                                                                                                 ` Thierry Reding
2013-03-14 17:25                                                                                                 ` Jason Gunthorpe
2013-03-14 17:25                                                                                                   ` Jason Gunthorpe
2013-03-14 20:38                                                                                                   ` Thierry Reding
2013-03-14 20:38                                                                                                     ` Thierry Reding
2013-03-14 21:05                                                                                                     ` Jason Gunthorpe
2013-03-14 21:05                                                                                                       ` Jason Gunthorpe
2013-03-14 21:10                                                                                                     ` Mitch Bradley
2013-03-14 21:10                                                                                                       ` Mitch Bradley
2013-03-14 21:09                                                                                                   ` Thierry Reding
2013-03-14 21:09                                                                                                     ` Thierry Reding
2013-03-14 21:29                                                                                                     ` Jason Gunthorpe
2013-03-14 21:29                                                                                                       ` Jason Gunthorpe
2013-03-14 21:37                                                                                                       ` Thierry Reding
2013-03-14 21:37                                                                                                         ` Thierry Reding
2013-03-13 22:22                                                                                         ` Jason Gunthorpe
2013-03-13 22:22                                                                                           ` Jason Gunthorpe
2013-03-09  8:58                               ` Thomas Petazzoni
2013-03-09  8:58                                 ` Thomas Petazzoni
2013-03-08 23:12                           ` Rob Herring
2013-03-08 23:12                             ` Rob Herring
     [not found]                             ` <513A7044.1020700-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-03-09 11:10                               ` Thierry Reding
2013-03-09 11:10                                 ` Thierry Reding
2013-03-09 11:10                                 ` Thierry Reding
2013-03-10  5:04                               ` Mitch Bradley
2013-03-10  5:04                                 ` Mitch Bradley
2013-03-10  5:04                                 ` Mitch Bradley
2013-03-10 15:06                                 ` Thomas Petazzoni
2013-03-10 15:06                                   ` Thomas Petazzoni
2013-03-10 18:33                                   ` Mitch Bradley
2013-03-10 18:33                                     ` Mitch Bradley
2013-03-10 18:33                                     ` Mitch Bradley
2013-02-15  0:36   ` Bjorn Helgaas
2013-02-15  0:36     ` Bjorn Helgaas
2013-02-15  5:06     ` Thomas Petazzoni
2013-02-15  5:06       ` Thomas Petazzoni
2013-02-15 16:26       ` Bjorn Helgaas
2013-02-15 16:26         ` Bjorn Helgaas
2013-02-15 16:44         ` Jason Gunthorpe
2013-02-15 16:44           ` Jason Gunthorpe
2013-02-12 16:28 ` [PATCH 25/32] arm: mvebu: PCIe support is now available on mvebu Thomas Petazzoni
2013-02-12 16:28   ` Thomas Petazzoni
2013-02-12 16:29 ` [PATCH 26/32] arm: mvebu: add PCIe Device Tree informations for Armada 370 Thomas Petazzoni
2013-02-12 16:29   ` Thomas Petazzoni
2013-02-12 16:29 ` [PATCH 27/32] arm: mvebu: add PCIe Device Tree informations for Armada XP Thomas Petazzoni
2013-02-12 16:29   ` Thomas Petazzoni
2013-02-12 16:29 ` [PATCH 28/32] arm: mvebu: PCIe Device Tree informations for OpenBlocks AX3-4 Thomas Petazzoni
2013-02-12 16:29   ` Thomas Petazzoni
2013-02-12 16:29 ` [PATCH 29/32] arm: mvebu: PCIe Device Tree informations for Armada XP DB Thomas Petazzoni
2013-02-12 16:29   ` Thomas Petazzoni
2013-02-12 16:29 ` [PATCH 30/32] arm: mvebu: PCIe Device Tree informations for Armada 370 Mirabox Thomas Petazzoni
2013-02-12 16:29   ` Thomas Petazzoni
2013-02-12 16:29 ` [PATCH 31/32] arm: mvebu: PCIe Device Tree informations for Armada 370 DB Thomas Petazzoni
2013-02-12 16:29   ` Thomas Petazzoni
2013-02-12 16:29 ` [PATCH 32/32] arm: mvebu: update defconfig with PCI and USB support Thomas Petazzoni
2013-02-12 16:29   ` Thomas Petazzoni
2013-02-12 18:12 ` [PATCH v3] PCIe support for the Armada 370 and Armada XP SoCs Arnd Bergmann
2013-02-12 18:12   ` Arnd Bergmann
2013-02-12 19:04   ` Thomas Petazzoni
2013-02-12 19:04     ` Thomas Petazzoni
2013-02-13  8:50   ` Thomas Petazzoni
2013-02-13  8:50     ` Thomas Petazzoni
2013-02-13  9:37     ` Arnd Bergmann
2013-02-13  9:37       ` Arnd Bergmann
2013-02-13 15:27 ` Christophe Vu-Brugier
2013-02-13 15:27   ` Christophe Vu-Brugier
2013-02-13 15:30   ` Thomas Petazzoni
2013-02-13 15:30     ` Thomas Petazzoni

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.