Re: [PATCH V2 00/23] MMCONFIG refactoring and support for ARM64 PCI hostbridge init based on ACPI

[okaya@codeaurora.org]

> On 17.12.2015 22:24, Sinan Kaya wrote:
>> Hi Tomasz,
>>
>> On 12/16/2015 10:16 AM, Tomasz Nowicki wrote:
>>>  From the functionality point of view this series might be split into
>>> the
>>> following logic parts:
>>> 1. Make MMCONFIG code arch-agnostic which allows all architectures to
>>> collect
>>>     PCI config regions and used when necessary.
>>> 2. Move non-arch specific bits to the core code.
>>> 3. Use MMCONFIG code and implement generic ACPI based PCI host
>>>     controller driver.
>>> 4. Enable above driver on ARM64
>>>
>>> Patches has been built on top of 4.4-rc4 and can be found here:
>>> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v2)
>>>
>>> NOTE, this patch set depends on Matthew's patches:
>>> http://www.spinics.net/lists/linux-pci/msg45950.html
>>> https://github.com/Vality/linux/tree/pci-fixes
>>>
>>> This has been tested on Cavium ThunderX 1 socket server and QEMU.
>>> Any help in reviewing and testing is very appreciated.
>>>
>>> v1 -> v2
>>> - moved non-arch specific piece of code to dirver/acpi/ directory
>>> - fixed IO resource handling
>>> - introduced PCI config accessors quirks matching
>>> - moved ACPI_COMPANION_SET to generic code
>>>
>>
>> Just tested your series. I'm seeing a resource assignment problem below.
>> The bus addresses show as memory addresses and memory addresses show as
>> bus addresses and IO resource did not show up.
>>
>>
>> Tomasz V2
>>
>> [    2.520852] ACPI: PCI Interrupt Link [LN1C] (IRQs *238)
>> [    2.535472] ACPI: PCI Interrupt Link [LN1D] (IRQs *239)
>> [    2.550562] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 00-1f])
>> [    2.567813] acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig ASPM
>> ClockPM Segments MSI]
>> [    2.591270] acpi PNP0A08:02: _OSC: platform does not support
>> [PCIeHotplug]
>> [    2.611144] acpi PNP0A08:02: _OSC: OS now controls [PME AER
>> PCIeCapability]
>> [    2.630299] ACPI: IORT: can't find node related to (null) device
>> [    2.647184]_acpi_PNP0A08:02:_PCI_host_bridge_to_bus_0002:00
>> [    2.662663] pci_bus 0002:00: root bus resource [mem
>> 0x00100000-0x3fffffff window] (bus address
>> [0xfffff5ff00100000-0xfffff5ff3fffffff])
>> [    2.703561] pci_bus 0002:00: root bus resource [mem
>> 0x40000000-0x7fffffff window] (bus address
>> [0xfffff5fe80000000-0xfffff5febfffffff])
>> [    2.737737] pci_bus 0002:00: root bus resource [mem
>> 0x80000000-0xffffffff window] (bus address
>> [0xfffff5fe00000000-0xfffff5fe7fffffff])
>> [    2.794961] pci_bus 0002:00: root bus resource [bus 00-1f]
>>
>> Mark Salter's patches
>>
>> [    2.730011] ACPI: PCI Interrupt Link [LN1C] (IRQs *238)
>> [    2.744648] ACPI: PCI Interrupt Link [LN1D] (IRQs *239)
>> [    2.759330] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 00-1f])
>> [    2.783295] acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig ASPM
>> ClockPM Segments MSI]
>> [    2.806726] acpi PNP0A08:02: _OSC: platform does not support
>> [PCIeHotplug]
>> [    2.826005] acpi PNP0A08:02: _OSC: OS now controls [PME AER
>> PCIeCapability]
>> [    2.845361] PCI host bridge to bus 0002:00
>> [    2.856719]_pci_bus_0002:00:_root_bus_resource_[bus_00-1f]
>> [    2.872056] pci_bus 0002:00: root bus resource [mem
>> 0xa0100100000-0xa013fffffff] (bus address [0x00100000-0x3fffffff])
>> [    2.902008] pci_bus 0002:00: root bus resource [mem
>> 0xa0200000000-0xa023fffffff] (bus address [0x40000000-0x7fffffff])
>> [    2.932396] pci_bus 0002:00: root bus resource [mem
>> 0xa0300000000-0xa037fffffff] (bus address [0x80000000-0xffffffff])
>> [    2.983827] pci_bus 0002:00: root bus resource [io  0x0000-0xffff]
>>
>> Here is how the ACPI table looks like:
>>
>> 		QWORDMemory(			// Consumed-And-prodced resource(all of memory space)
>> 			ResourceProducer,	// bit 0 of general flags is 0
>> 			PosDecode,		// positive Decode:  		_DEC
>> 			MinFixed,		// Range is fixed:   		_MIF
>> 			MaxFixed,		// Range is fixed:   		_MAF
>> 			NonCacheable,		// 				_MEM
>> 			ReadWrite,		// 				_RW
>> 			0x00000000,		// Granularity: 	   	_GRA
>> 			0x00100000,		// Min - PCI Memory start: 	_MIN
>> 			0x3FFFFFFF,		// Max - PCI Memory end:   	_MAX
>> 			0xA0100000000,		// Translation: 		_TRA
>> 			0x3FF00000,		// Range Length: 		_LEN
>> 			,			// Optional field left blank
>> 			,			// Optional field left blank
>> 			MEM0,			// Name declaration for this descriptor
>> 			AddressRangeMemory,
>> 			TypeStatic
>> 			)
>>
>>
>> Any thoughts?
>>
>>
>
> Yes, this is because of:
> [PATCH V2 20/23] ACPI, PCI: Refine the way to handle translation_offset
> for ACPI resources
> which should have RFC tag. I posted this patch to re-trigger discussion
> on this.
>
> The patch does not add Translation offset to the MMIO type resource
> start address and for acpi_pci_probe_root_resources(ci) causes problems
> like that. Indeed MMIO has to be fixed.

OK. I assume you'll post a patch for this soon similar to what Liu Jiang
is doing in IA64 directory (arch/ia64/pci/pci.c) as I can't proceed with
my testing without this bugfix.
>
> But IO resource type is more problematic. Actually, how
> acpi_decode_space() should parse resources and which ACPI IO descriptor
> should be used for ARM64: QWORDIO (offset == 0 vs offset != 0), DWordIO
> (TypeStatic vs TypeTranslation) + backward compatibility with IA64...
>
> Please refer to:
> https://lkml.org/lkml/2015/11/5/581
>
> As Lorenzo pointed out, we *all* need to agree upon the IO resource ACPI
> descriptor and its parsing method.

Here is what I have as an IO resource.

		QWORDIO(			//Consumed-And-produced resource
			ResourceProducer,	// bit 0 of general flags is 0
			MinFixed,		// Range is fixed
			MaxFixed,		// Range is fixed
			PosDecode,
			EntireRange,
			0x0000,			// Granularity
			0x1000,		// Min, 0 is not accepted
			0x10FFF,		// Max
			0x8FFFFFEF000,	// Translation
			0x10000,		// Range Length
			,, PI00
		 )

I don't have any type specified.

I agree with Lorenzo's assessment. The min and max values represent the
PCI IO bus addresses. The translation offset is added to these values to
figure out the CPU view of the PCI IO range.

The endpoints BAR addresses are programmed with IO addresses ranging
between 0x1000 and 0x10FFF for this example above.

Here is another question. Chris Covington and I asked this question on a
private email to you but we didn't hear back.

We were referring to a Linaro IO hack patch as we were not sure whether
this was a limitation of the hack or a general expectation for ARM64 PCI
in general.

I'll repeat it here.

I have multiple root ports with the same IO port configuration in the
current ACPI table.

Root port 0 = IO range 0x1000-0x10FFF
Root port 1 = IO range 0x1000-0x10FFF
Root port 2 = IO range 0x1000-0x10FFF

Each root port can have the same IO address range configuration, are we
expecting IO port numbers to be unique across the whole system for ARM64?

Something like

Root port 0 = IO range 0x1000-0x10FFF
Root port 1 = IO range 0x11000-0x20FFF
Root port 2 = IO range 0x21000-0x30FFF

since the IO addresses are being remapped into PCI IO range printed during
boot.

PCI I/O : 0xffff7ffffae00000 - 0xffff7ffffbe00000   (    16 MB)

and each root port would remap to 64k of the 16MB range.

>
> Any comments are very appreciated!
>
> Tomasz
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>