[PATCH V6 00/13] Support for generic ACPI based PCI host controller

[PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Tomasz Nowicki]

>From the functionality point of view this series might be split into the
following logic parts:
1. Necessary fixes as the preparation for using driver on ARM64.
2. New ECAM API and update for users of the pci-host-common API
3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
4. Enable above driver on ARM64

Patches has been built on top of 4.6-rc2 and can be found here:
git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)

This has been tested on Cavium ThunderX server. Any help in reviewing and
testing is very appreciated.

v5 -> v6
- dropped idea of x86 MMCONFIG code refactoring
- integrated JC's patches which introduce new ECAM API:
  https://lkml.org/lkml/2016/4/11/907
  git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
- integrated Sinan's fix for releasing IO resources, see patch [06/13]
- added ACPI support for ThunderX ECAM and PEM drivers
- rebased to 4.6-rc2

v4 -> v5
- dropped MCFG refactoring group patches 1-6 from series v4 and integrated Jayachandran's patch
  https://patchwork.ozlabs.org/patch/575525/
- rewrite PCI legacy IRQs allocation
- squashed two patches 11 and 12 from series v4, fixed bisection issue
- changelog improvements
- rebased to 4.5-rc3

v3 -> v4
- dropped Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
- added Lorenzo's fix patch 19/24
- ACPI PCI bus domain number assigning cleanup
- changed resource management, we now claim and reassign resources
- improvements for applying quirks
- dropped Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
- rebased to 4.5-rc1

v2 -> v3
- fix legacy IRQ assigning and IO ports registration
- remove reference to arch specific companion device for ia64
- move ACPI PCI host controller driver to pci_root.c
- drop generic domain assignment for x86 and ia64 as I am not
  able to run all necessary test variants
- drop patch which cleaned legacy IRQ assignment since it belongs to
  Mathew's series:
  https://patchwork.ozlabs.org/patch/557504/
- extend MCFG quirk code
- rebased to 4.4

v1 -> v2
- moved non-arch specific piece of code to dirver/acpi/ directory
- fixed IO resource handling
- introduced PCI config accessors quirks matching
- moved ACPI_COMPANION_SET to generic code

v1 - https://lkml.org/lkml/2015/10/27/504
v2 - https://lkml.org/lkml/2015/12/16/246
v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
v4 - https://lkml.org/lkml/2016/2/4/646
v5 - https://lkml.org/lkml/2016/2/16/426

Jayachandran C (2):
  PCI: Provide common functions for ECAM mapping
  PCI: generic, thunder: update to use generic ECAM API

Tomasz Nowicki (11):
  pci, acpi, x86, ia64: Move ACPI host bridge device companion
    assignment to core code.
  pci, acpi: Provide generic way to assign bus domain number.
  x86, ia64: Include acpi_pci_{add|remove}_bus to the default
    pcibios_{add|remove}_bus implementation.
  pci, of: Move the PCI I/O space management to PCI core code.
  acpi, pci: Support IO resources when parsing PCI host bridge
    resources.
  arm64, pci, acpi: ACPI support for legacy IRQs parsing and
    consolidation with DT code.
  pci, acpi: Support for ACPI based generic PCI host controller
  arm64, pci, acpi: Start using ACPI based PCI host controller driver
    for ARM64.
  pci, acpi: Match PCI config space accessors against platfrom specific
    quirks.
  pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.
  pci, pci-thunder-pem: Add ACPI support for ThunderX PEM.

 arch/arm64/Kconfig                  |  15 +++
 arch/arm64/include/asm/cpufeature.h |   3 +-
 arch/arm64/kernel/cpu_errata.c      |   8 ++
 arch/arm64/kernel/pci.c             |  35 ++---
 arch/ia64/hp/common/sba_iommu.c     |   2 +-
 arch/ia64/include/asm/pci.h         |   1 -
 arch/ia64/pci/pci.c                 |  26 ----
 arch/ia64/sn/kernel/io_acpi_init.c  |   4 +-
 arch/x86/include/asm/pci.h          |   3 -
 arch/x86/pci/acpi.c                 |  17 ---
 arch/x86/pci/common.c               |  10 --
 drivers/acpi/Kconfig                |   8 ++
 drivers/acpi/Makefile               |   1 +
 drivers/acpi/bus.c                  |   1 +
 drivers/acpi/pci_gen_host.c         | 259 ++++++++++++++++++++++++++++++++++++
 drivers/acpi/pci_root.c             |  58 +++++++-
 drivers/of/address.c                | 116 +---------------
 drivers/pci/Kconfig                 |   3 +
 drivers/pci/Makefile                |   2 +
 drivers/pci/ecam.c                  | 137 +++++++++++++++++++
 drivers/pci/ecam.h                  |  66 +++++++++
 drivers/pci/host/Kconfig            |   1 +
 drivers/pci/host/pci-host-common.c  | 119 ++++++++---------
 drivers/pci/host/pci-host-common.h  |  47 -------
 drivers/pci/host/pci-host-generic.c |  52 ++------
 drivers/pci/host/pci-thunder-ecam.c |  70 ++++++----
 drivers/pci/host/pci-thunder-pem.c  | 215 ++++++++++++++++++++++--------
 drivers/pci/pci.c                   | 150 ++++++++++++++++++++-
 drivers/pci/probe.c                 |   5 +
 include/asm-generic/vmlinux.lds.h   |   7 +
 include/linux/of_address.h          |   9 --
 include/linux/pci-acpi.h            |  20 +++
 include/linux/pci.h                 |  12 ++
 33 files changed, 1029 insertions(+), 453 deletions(-)
 create mode 100644 drivers/acpi/pci_gen_host.c
 create mode 100644 drivers/pci/ecam.c
 create mode 100644 drivers/pci/ecam.h
 delete mode 100644 drivers/pci/host/pci-host-common.h

-- 
1.9.1

[PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Tomasz Nowicki]

Currently we have two platforms (x86 & ia64) capable of PCI ACPI host
bridge initialization. They both use arch-specific sysdata to pass down
parent device reference and both rely on NULL parent in pci_create_root_bus()
to validate sysdata content.

It looks hacky and prevents us from getting some firmware specific
info for PCI host controller based on its acpi_device structure
in generic pci_create_root_bus() function. However, we overcome that
blocker by passing down parent device via pci_create_root_bus parameter
(as the ACPI device type). Then we use ACPI_COMPANION_SET in core code
for ACPI boot method only. ACPI_COMPANION_SET is safe to run for all
cases DT, ACPI and DT&ACPI.

Since now PCI core code is setting ACPI companion device for us,
x86 & ia64 specific ACPI companion device setting turns out to be dead now.
We can get rid of it, including related companion reference from
PCI sysdata structure. Aslo, PCI_CONTROLLER macro cannot return valid
companion device anymore. Therefore we need to convert its usage to
ACPI_COMPANION.

Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Duc Dang <dhdang@apm.com>
Tested-by: Dongdong Liu <liudongdong3@huawei.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
Tested-by: Sinan Kaya <okaya@codeaurora.org>
---
 arch/ia64/hp/common/sba_iommu.c    |  2 +-
 arch/ia64/include/asm/pci.h        |  1 -
 arch/ia64/pci/pci.c                | 16 ----------------
 arch/ia64/sn/kernel/io_acpi_init.c |  4 ++--
 arch/x86/include/asm/pci.h         |  3 ---
 arch/x86/pci/acpi.c                | 17 -----------------
 drivers/acpi/pci_root.c            |  7 ++++++-
 drivers/pci/probe.c                |  2 ++
 8 files changed, 11 insertions(+), 41 deletions(-)

diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
index a6d6190..78e4444 100644
--- a/arch/ia64/hp/common/sba_iommu.c
+++ b/arch/ia64/hp/common/sba_iommu.c
@@ -1981,7 +1981,7 @@ sba_connect_bus(struct pci_bus *bus)
 	if (PCI_CONTROLLER(bus)->iommu)
 		return;
 
-	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
+	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
 	if (!handle)
 		return;
 
diff --git a/arch/ia64/include/asm/pci.h b/arch/ia64/include/asm/pci.h
index c0835b0..12423f4 100644
--- a/arch/ia64/include/asm/pci.h
+++ b/arch/ia64/include/asm/pci.h
@@ -63,7 +63,6 @@ extern int pci_mmap_legacy_page_range(struct pci_bus *bus,
 #define pci_legacy_write platform_pci_legacy_write
 
 struct pci_controller {
-	struct acpi_device *companion;
 	void *iommu;
 	int segment;
 	int node;		/* nearest node with memory or NUMA_NO_NODE for global allocation */
diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
index 8f6ac2f..978d6af 100644
--- a/arch/ia64/pci/pci.c
+++ b/arch/ia64/pci/pci.c
@@ -301,28 +301,12 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 	}
 
 	info->controller.segment = root->segment;
-	info->controller.companion = device;
 	info->controller.node = acpi_get_node(device->handle);
 	INIT_LIST_HEAD(&info->io_resources);
 	return acpi_pci_root_create(root, &pci_acpi_root_ops,
 				    &info->common, &info->controller);
 }
 
-int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
-{
-	/*
-	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
-	 * here, pci_create_root_bus() has been called by someone else and
-	 * sysdata is likely to be different from what we expect.  Let it go in
-	 * that case.
-	 */
-	if (!bridge->dev.parent) {
-		struct pci_controller *controller = bridge->bus->sysdata;
-		ACPI_COMPANION_SET(&bridge->dev, controller->companion);
-	}
-	return 0;
-}
-
 void pcibios_fixup_device_resources(struct pci_dev *dev)
 {
 	int idx;
diff --git a/arch/ia64/sn/kernel/io_acpi_init.c b/arch/ia64/sn/kernel/io_acpi_init.c
index 231234c..e454492 100644
--- a/arch/ia64/sn/kernel/io_acpi_init.c
+++ b/arch/ia64/sn/kernel/io_acpi_init.c
@@ -132,7 +132,7 @@ sn_get_bussoft_ptr(struct pci_bus *bus)
 	struct acpi_resource_vendor_typed *vendor;
 
 
-	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
+	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
 	status = acpi_get_vendor_resource(handle, METHOD_NAME__CRS,
 					  &sn_uuid, &buffer);
 	if (ACPI_FAILURE(status)) {
@@ -360,7 +360,7 @@ sn_acpi_get_pcidev_info(struct pci_dev *dev, struct pcidev_info **pcidev_info,
 	acpi_status status;
 	struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
 
-	rootbus_handle = acpi_device_handle(PCI_CONTROLLER(dev)->companion);
+	rootbus_handle = acpi_device_handle(ACPI_COMPANION(dev->bus->bridge));
         status = acpi_evaluate_integer(rootbus_handle, METHOD_NAME__SEG, NULL,
                                        &segment);
         if (ACPI_SUCCESS(status)) {
diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
index 9ab7507..24de07d 100644
--- a/arch/x86/include/asm/pci.h
+++ b/arch/x86/include/asm/pci.h
@@ -14,9 +14,6 @@
 struct pci_sysdata {
 	int		domain;		/* PCI domain */
 	int		node;		/* NUMA node */
-#ifdef CONFIG_ACPI
-	struct acpi_device *companion;	/* ACPI companion device */
-#endif
 #ifdef CONFIG_X86_64
 	void		*iommu;		/* IOMMU private data */
 #endif
diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
index 3cd6983..f4ca17a 100644
--- a/arch/x86/pci/acpi.c
+++ b/arch/x86/pci/acpi.c
@@ -340,7 +340,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 		struct pci_sysdata sd = {
 			.domain = domain,
 			.node = node,
-			.companion = root->device
 		};
 
 		memcpy(bus->sysdata, &sd, sizeof(sd));
@@ -355,7 +354,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 		else {
 			info->sd.domain = domain;
 			info->sd.node = node;
-			info->sd.companion = root->device;
 			bus = acpi_pci_root_create(root, &acpi_pci_root_ops,
 						   &info->common, &info->sd);
 		}
@@ -373,21 +371,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 	return bus;
 }
 
-int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
-{
-	/*
-	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
-	 * here, pci_create_root_bus() has been called by someone else and
-	 * sysdata is likely to be different from what we expect.  Let it go in
-	 * that case.
-	 */
-	if (!bridge->dev.parent) {
-		struct pci_sysdata *sd = bridge->bus->sysdata;
-		ACPI_COMPANION_SET(&bridge->dev, sd->companion);
-	}
-	return 0;
-}
-
 int __init pci_acpi_init(void)
 {
 	struct pci_dev *dev = NULL;
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index ae3fe4e..4581e0e 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device,
 		}
 	}
 
+	/*
+	 * pci_create_root_bus() needs to detect the parent device type,
+	 * so initialize its companion data accordingly.
+	 */
+	ACPI_COMPANION_SET(&device->dev, device);
 	root->device = device;
 	root->segment = segment & 0xFFFF;
 	strcpy(acpi_device_name(device), ACPI_PCI_ROOT_DEVICE_NAME);
@@ -846,7 +851,7 @@ struct pci_bus *acpi_pci_root_create(struct acpi_pci_root *root,
 
 	pci_acpi_root_add_resources(info);
 	pci_add_resource(&info->resources, &root->secondary);
-	bus = pci_create_root_bus(NULL, busnum, ops->pci_ops,
+	bus = pci_create_root_bus(&device->dev, busnum, ops->pci_ops,
 				  sysdata, &info->resources);
 	if (!bus)
 		goto out_release_info;
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..8087297 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2141,6 +2141,8 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
 	bridge->dev.parent = parent;
 	bridge->dev.release = pci_release_host_bridge_dev;
 	dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+	if (parent)
+		ACPI_COMPANION_SET(&bridge->dev, ACPI_COMPANION(parent));
 	error = pcibios_root_bridge_prepare(bridge);
 	if (error) {
 		kfree(bridge);
-- 
1.9.1

[PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Tomasz Nowicki]

As we now have valid PCI host bridge device reference we can
introduce code that is going to find its bus domain number using
ACPI _SEG method.

Note that _SEG method is optional, therefore _SEG absence means
that all PCI buses belong to domain 0.

While at it, for the sake of code clarity we put ACPI and DT domain
assign methods into the corresponding helpers.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Reviewed-by: Liviu Dudau <Liviu.Dudau@arm.com>
Tested-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Tested-by: Jeremy Linton <jeremy.linton@arm.com>
Tested-by: Duc Dang <dhdang@apm.com>
Tested-by: Dongdong Liu <liudongdong3@huawei.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
Tested-by: Sinan Kaya <okaya@codeaurora.org>
---
 drivers/acpi/pci_root.c  | 18 ++++++++++++++++++
 drivers/pci/pci.c        | 11 +++++++++--
 include/linux/pci-acpi.h |  2 ++
 3 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index 4581e0e..d9a70c4 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -419,6 +419,24 @@ out:
 }
 EXPORT_SYMBOL(acpi_pci_osc_control_set);
 
+int acpi_pci_bus_domain_nr(struct device *parent)
+{
+	struct acpi_device *acpi_dev = to_acpi_device(parent);
+	unsigned long long segment = 0;
+	acpi_status status;
+
+	/*
+	 * If _SEG method does not exist, following ACPI spec (6.5.6)
+	 * all PCI buses belong to domain 0.
+	 */
+	status = acpi_evaluate_integer(acpi_dev->handle, METHOD_NAME__SEG, NULL,
+				       &segment);
+	if (ACPI_FAILURE(status) && status != AE_NOT_FOUND)
+		dev_err(&acpi_dev->dev, "can't evaluate _SEG\n");
+
+	return segment;
+}
+
 static void negotiate_os_control(struct acpi_pci_root *root, int *no_aspm)
 {
 	u32 support, control, requested;
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 25e0327..1a74e87 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -19,6 +19,7 @@
 #include <linux/spinlock.h>
 #include <linux/string.h>
 #include <linux/log2.h>
+#include <linux/pci-acpi.h>
 #include <linux/pci-aspm.h>
 #include <linux/pm_wakeup.h>
 #include <linux/interrupt.h>
@@ -4779,7 +4780,7 @@ int pci_get_new_domain_nr(void)
 }
 
 #ifdef CONFIG_PCI_DOMAINS_GENERIC
-void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
+static int of_pci_bus_domain_nr(struct device *parent)
 {
 	static int use_dt_domains = -1;
 	int domain = -1;
@@ -4823,7 +4824,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
 		domain = -1;
 	}
 
-	bus->domain_nr = domain;
+	return domain;
+}
+
+void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
+{
+	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
+					 acpi_pci_bus_domain_nr(parent);
 }
 #endif
 #endif
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 89ab057..a72e22d 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -22,6 +22,7 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
 {
 	return acpi_remove_pm_notifier(dev);
 }
+extern int acpi_pci_bus_domain_nr(struct device *parent);
 extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
 
 static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
@@ -109,6 +110,7 @@ extern const u8 pci_acpi_dsm_uuid[];
 #else	/* CONFIG_ACPI */
 static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
 static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
+static inline int acpi_pci_bus_domain_nr(struct device *parent) { return -1; }
 #endif	/* CONFIG_ACPI */
 
 #ifdef CONFIG_ACPI_APEI
-- 
1.9.1

[PATCH V6 03/13] x86, ia64: Include acpi_pci_{add|remove}_bus to the default pcibios_{add|remove}_bus implementation.

[Tomasz Nowicki]

x86 and ia64 are the only arches that implement pcibios_{add|remove}_bus hooks
and implement them in the same way. Moreover ARM64 is going to do the same.
So it seems that acpi_pci_{add|remove}_bus is generic enough to be default
option for pcibios_{add|remove}_bus hooks. Also, it is always safe to run
acpi_pci_{add|remove}_bus as they have empty stubs for !ACPI case and
return if ACPI has been switched off in run time.

After all we can remove x86 and ia64 pcibios_{add|remove}_bus
implementation.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Duc Dang <dhdang@apm.com>
Tested-by: Dongdong Liu <liudongdong3@huawei.com>
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
Tested-by: Sinan Kaya <okaya@codeaurora.org>
---
 arch/ia64/pci/pci.c   | 10 ----------
 arch/x86/pci/common.c | 10 ----------
 drivers/pci/probe.c   |  3 +++
 3 files changed, 3 insertions(+), 20 deletions(-)

diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
index 978d6af..be4c9ef 100644
--- a/arch/ia64/pci/pci.c
+++ b/arch/ia64/pci/pci.c
@@ -358,16 +358,6 @@ void pcibios_fixup_bus(struct pci_bus *b)
 	platform_pci_fixup_bus(b);
 }
 
-void pcibios_add_bus(struct pci_bus *bus)
-{
-	acpi_pci_add_bus(bus);
-}
-
-void pcibios_remove_bus(struct pci_bus *bus)
-{
-	acpi_pci_remove_bus(bus);
-}
-
 void pcibios_set_master (struct pci_dev *dev)
 {
 	/* No special bus mastering setup handling */
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 381a43c..7763a84 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -170,16 +170,6 @@ void pcibios_fixup_bus(struct pci_bus *b)
 		pcibios_fixup_device_resources(dev);
 }
 
-void pcibios_add_bus(struct pci_bus *bus)
-{
-	acpi_pci_add_bus(bus);
-}
-
-void pcibios_remove_bus(struct pci_bus *bus)
-{
-	acpi_pci_remove_bus(bus);
-}
-
 /*
  * Only use DMI information to set this if nothing was passed
  * on the kernel command line (which was parsed earlier).
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8087297..ef569e8 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
 #include <linux/slab.h>
 #include <linux/module.h>
 #include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
 #include <linux/pci-aspm.h>
 #include <linux/aer.h>
 #include <linux/acpi.h>
@@ -2101,10 +2102,12 @@ int __weak pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
 
 void __weak pcibios_add_bus(struct pci_bus *bus)
 {
+	acpi_pci_add_bus(bus);
 }
 
 void __weak pcibios_remove_bus(struct pci_bus *bus)
 {
+	acpi_pci_remove_bus(bus);
 }
 
 struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
-- 
1.9.1

[PATCH V6 04/13] pci, of: Move the PCI I/O space management to PCI core code.

[Tomasz Nowicki]

No functional changes in this patch.

PCI I/O space mapping code does not depend on OF, therefore it can be
moved to PCI core code. This way we will be able to use it
e.g. in ACPI PCI code.

Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
CC: Arnd Bergmann <arnd@arndb.de>
CC: Liviu Dudau <Liviu.Dudau@arm.com>
CC: Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
---
 drivers/of/address.c       | 116 +--------------------------------------------
 drivers/pci/pci.c          | 115 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/of_address.h |   9 ----
 include/linux/pci.h        |   5 ++
 4 files changed, 121 insertions(+), 124 deletions(-)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 91a469d..0a553c0 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -4,6 +4,7 @@
 #include <linux/ioport.h>
 #include <linux/module.h>
 #include <linux/of_address.h>
+#include <linux/pci.h>
 #include <linux/pci_regs.h>
 #include <linux/sizes.h>
 #include <linux/slab.h>
@@ -673,121 +674,6 @@ const __be32 *of_get_address(struct device_node *dev, int index, u64 *size,
 }
 EXPORT_SYMBOL(of_get_address);
 
-#ifdef PCI_IOBASE
-struct io_range {
-	struct list_head list;
-	phys_addr_t start;
-	resource_size_t size;
-};
-
-static LIST_HEAD(io_range_list);
-static DEFINE_SPINLOCK(io_range_lock);
-#endif
-
-/*
- * Record the PCI IO range (expressed as CPU physical address + size).
- * Return a negative value if an error has occured, zero otherwise
- */
-int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
-{
-	int err = 0;
-
-#ifdef PCI_IOBASE
-	struct io_range *range;
-	resource_size_t allocated_size = 0;
-
-	/* check if the range hasn't been previously recorded */
-	spin_lock(&io_range_lock);
-	list_for_each_entry(range, &io_range_list, list) {
-		if (addr >= range->start && addr + size <= range->start + size) {
-			/* range already registered, bail out */
-			goto end_register;
-		}
-		allocated_size += range->size;
-	}
-
-	/* range not registed yet, check for available space */
-	if (allocated_size + size - 1 > IO_SPACE_LIMIT) {
-		/* if it's too big check if 64K space can be reserved */
-		if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {
-			err = -E2BIG;
-			goto end_register;
-		}
-
-		size = SZ_64K;
-		pr_warn("Requested IO range too big, new size set to 64K\n");
-	}
-
-	/* add the range to the list */
-	range = kzalloc(sizeof(*range), GFP_ATOMIC);
-	if (!range) {
-		err = -ENOMEM;
-		goto end_register;
-	}
-
-	range->start = addr;
-	range->size = size;
-
-	list_add_tail(&range->list, &io_range_list);
-
-end_register:
-	spin_unlock(&io_range_lock);
-#endif
-
-	return err;
-}
-
-phys_addr_t pci_pio_to_address(unsigned long pio)
-{
-	phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;
-
-#ifdef PCI_IOBASE
-	struct io_range *range;
-	resource_size_t allocated_size = 0;
-
-	if (pio > IO_SPACE_LIMIT)
-		return address;
-
-	spin_lock(&io_range_lock);
-	list_for_each_entry(range, &io_range_list, list) {
-		if (pio >= allocated_size && pio < allocated_size + range->size) {
-			address = range->start + pio - allocated_size;
-			break;
-		}
-		allocated_size += range->size;
-	}
-	spin_unlock(&io_range_lock);
-#endif
-
-	return address;
-}
-
-unsigned long __weak pci_address_to_pio(phys_addr_t address)
-{
-#ifdef PCI_IOBASE
-	struct io_range *res;
-	resource_size_t offset = 0;
-	unsigned long addr = -1;
-
-	spin_lock(&io_range_lock);
-	list_for_each_entry(res, &io_range_list, list) {
-		if (address >= res->start && address < res->start + res->size) {
-			addr = address - res->start + offset;
-			break;
-		}
-		offset += res->size;
-	}
-	spin_unlock(&io_range_lock);
-
-	return addr;
-#else
-	if (address > IO_SPACE_LIMIT)
-		return (unsigned long)-1;
-
-	return (unsigned long) address;
-#endif
-}
-
 static int __of_address_to_resource(struct device_node *dev,
 		const __be32 *addrp, u64 size, unsigned int flags,
 		const char *name, struct resource *r)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 1a74e87..89e9996 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3022,6 +3022,121 @@ int pci_request_regions_exclusive(struct pci_dev *pdev, const char *res_name)
 }
 EXPORT_SYMBOL(pci_request_regions_exclusive);
 
+#ifdef PCI_IOBASE
+struct io_range {
+	struct list_head list;
+	phys_addr_t start;
+	resource_size_t size;
+};
+
+static LIST_HEAD(io_range_list);
+static DEFINE_SPINLOCK(io_range_lock);
+#endif
+
+/*
+ * Record the PCI IO range (expressed as CPU physical address + size).
+ * Return a negative value if an error has occured, zero otherwise
+ */
+int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
+{
+	int err = 0;
+
+#ifdef PCI_IOBASE
+	struct io_range *range;
+	resource_size_t allocated_size = 0;
+
+	/* check if the range hasn't been previously recorded */
+	spin_lock(&io_range_lock);
+	list_for_each_entry(range, &io_range_list, list) {
+		if (addr >= range->start && addr + size <= range->start + size) {
+			/* range already registered, bail out */
+			goto end_register;
+		}
+		allocated_size += range->size;
+	}
+
+	/* range not registed yet, check for available space */
+	if (allocated_size + size - 1 > IO_SPACE_LIMIT) {
+		/* if it's too big check if 64K space can be reserved */
+		if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {
+			err = -E2BIG;
+			goto end_register;
+		}
+
+		size = SZ_64K;
+		pr_warn("Requested IO range too big, new size set to 64K\n");
+	}
+
+	/* add the range to the list */
+	range = kzalloc(sizeof(*range), GFP_ATOMIC);
+	if (!range) {
+		err = -ENOMEM;
+		goto end_register;
+	}
+
+	range->start = addr;
+	range->size = size;
+
+	list_add_tail(&range->list, &io_range_list);
+
+end_register:
+	spin_unlock(&io_range_lock);
+#endif
+
+	return err;
+}
+
+phys_addr_t pci_pio_to_address(unsigned long pio)
+{
+	phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;
+
+#ifdef PCI_IOBASE
+	struct io_range *range;
+	resource_size_t allocated_size = 0;
+
+	if (pio > IO_SPACE_LIMIT)
+		return address;
+
+	spin_lock(&io_range_lock);
+	list_for_each_entry(range, &io_range_list, list) {
+		if (pio >= allocated_size && pio < allocated_size + range->size) {
+			address = range->start + pio - allocated_size;
+			break;
+		}
+		allocated_size += range->size;
+	}
+	spin_unlock(&io_range_lock);
+#endif
+
+	return address;
+}
+
+unsigned long __weak pci_address_to_pio(phys_addr_t address)
+{
+#ifdef PCI_IOBASE
+	struct io_range *res;
+	resource_size_t offset = 0;
+	unsigned long addr = -1;
+
+	spin_lock(&io_range_lock);
+	list_for_each_entry(res, &io_range_list, list) {
+		if (address >= res->start && address < res->start + res->size) {
+			addr = address - res->start + offset;
+			break;
+		}
+		offset += res->size;
+	}
+	spin_unlock(&io_range_lock);
+
+	return addr;
+#else
+	if (address > IO_SPACE_LIMIT)
+		return (unsigned long)-1;
+
+	return (unsigned long) address;
+#endif
+}
+
 /**
  *	pci_remap_iospace - Remap the memory mapped I/O space
  *	@res: Resource describing the I/O space
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index 01c0a55..3786473 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -47,10 +47,6 @@ void __iomem *of_io_request_and_map(struct device_node *device,
 extern const __be32 *of_get_address(struct device_node *dev, int index,
 			   u64 *size, unsigned int *flags);
 
-extern int pci_register_io_range(phys_addr_t addr, resource_size_t size);
-extern unsigned long pci_address_to_pio(phys_addr_t addr);
-extern phys_addr_t pci_pio_to_address(unsigned long pio);
-
 extern int of_pci_range_parser_init(struct of_pci_range_parser *parser,
 			struct device_node *node);
 extern struct of_pci_range *of_pci_range_parser_one(
@@ -86,11 +82,6 @@ static inline const __be32 *of_get_address(struct device_node *dev, int index,
 	return NULL;
 }
 
-static inline phys_addr_t pci_pio_to_address(unsigned long pio)
-{
-	return 0;
-}
-
 static inline int of_pci_range_parser_init(struct of_pci_range_parser *parser,
 			struct device_node *node)
 {
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 9b31d48..c28adb4 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1164,6 +1164,9 @@ int __must_check pci_bus_alloc_resource(struct pci_bus *bus,
 			void *alignf_data);
 
 
+int pci_register_io_range(phys_addr_t addr, resource_size_t size);
+unsigned long pci_address_to_pio(phys_addr_t addr);
+phys_addr_t pci_pio_to_address(unsigned long pio);
 int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
 
 static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
@@ -1480,6 +1483,8 @@ static inline int pci_request_regions(struct pci_dev *dev, const char *res_name)
 { return -EIO; }
 static inline void pci_release_regions(struct pci_dev *dev) { }
 
+static inline unsigned long pci_address_to_pio(phys_addr_t addr) { return -1; }
+
 static inline void pci_block_cfg_access(struct pci_dev *dev) { }
 static inline int pci_block_cfg_access_in_atomic(struct pci_dev *dev)
 { return 0; }
-- 
1.9.1

[PATCH V6 05/13] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Tomasz Nowicki]

Platforms that have memory mapped IO port (such as ARM64) need special
handling for PCI I/O resources. For host bridge's resource probing case
these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.

Furthermore, the same I/O resources need to be released after hotplug
removal so that it can be re-added back by the pci_remap_iospace
function during insertion. Therefore we implement new pci_unmap_iospace call
which unmaps I/O space as the symmetry to pci_remap_iospace.

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
---
 drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++
 drivers/pci/pci.c       | 24 ++++++++++++++++++++++++
 include/linux/pci.h     |  1 +
 3 files changed, 58 insertions(+)

diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index d9a70c4..815b6ca 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -742,6 +742,34 @@ next:
 			resource_list_add_tail(entry, resources);
 	}
 }
+static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
+{
+#ifdef PCI_IOBASE
+	struct resource *res = entry->res;
+	resource_size_t cpu_addr = res->start;
+	resource_size_t pci_addr = cpu_addr - entry->offset;
+	resource_size_t length = resource_size(res);
+	unsigned long port;
+
+	if (pci_register_io_range(cpu_addr, length))
+		goto err;
+
+	port = pci_address_to_pio(cpu_addr);
+	if (port == (unsigned long)-1)
+		goto err;
+
+	res->start = port;
+	res->end = port + length - 1;
+	entry->offset = port - pci_addr;
+
+	if (pci_remap_iospace(res, cpu_addr) < 0)
+		goto err;
+	pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
+	return;
+err:
+	res->flags |= IORESOURCE_DISABLED;
+#endif
+}
 
 int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
 {
@@ -763,6 +791,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
 			"no IO and memory resources present in _CRS\n");
 	else {
 		resource_list_for_each_entry_safe(entry, tmp, list) {
+			if (entry->res->flags & IORESOURCE_IO)
+				acpi_pci_root_remap_iospace(entry);
+
 			if (entry->res->flags & IORESOURCE_DISABLED)
 				resource_list_destroy_entry(entry);
 			else
@@ -834,6 +865,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
 
 	resource_list_for_each_entry(entry, &bridge->windows) {
 		res = entry->res;
+		if (res->flags & IORESOURCE_IO)
+			pci_unmap_iospace(res);
 		if (res->parent &&
 		    (res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
 			release_resource(res);
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 89e9996..c0f8a4e 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -26,6 +26,7 @@
 #include <linux/device.h>
 #include <linux/pm_runtime.h>
 #include <linux/pci_hotplug.h>
+#include <linux/vmalloc.h>
 #include <asm/setup.h>
 #include <linux/aer.h>
 #include "pci.h"
@@ -3168,6 +3169,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
 #endif
 }
 
+/**
+ *	pci_unmap_iospace - Unmap the memory mapped I/O space
+ *	@res: resource to be unmapped
+ *
+ *	Unmap the CPU virtual address @res from virtual address space.
+ *	Only architectures that have memory mapped IO functions defined
+ *	(and the PCI_IOBASE value defined) should call this function.
+ */
+void  pci_unmap_iospace(struct resource *res)
+{
+#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
+	unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
+
+	unmap_kernel_range(vaddr, resource_size(res));
+#else
+	/*
+	 * This architecture does not have memory mapped I/O space,
+	 * so this function should never be called.
+	 */
+	WARN_ONCE(1, "This architecture does not support memory mapped I/O\n");
+#endif
+}
+
 static void __pci_set_master(struct pci_dev *dev, bool enable)
 {
 	u16 old_cmd, cmd;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index c28adb4..df1f33d 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1168,6 +1168,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size);
 unsigned long pci_address_to_pio(phys_addr_t addr);
 phys_addr_t pci_pio_to_address(unsigned long pio);
 int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
+void pci_unmap_iospace(struct resource *res);
 
 static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
 {
-- 
1.9.1

[PATCH V6 06/13] arm64, pci, acpi: ACPI support for legacy IRQs parsing and consolidation with DT code.

[Tomasz Nowicki]

To enable PCI legacy IRQs on platforms booting with ACPI, arch code
should include ACPI specific callbacks that parse and set-up the
device IRQ number, equivalent to the DT boot path. Owing to the current
ACPI core scan handlers implementation, ACPI PCI legacy IRQs bindings
cannot be parsed at device add time, since that would trigger ACPI scan
handlers ordering issues depending on how the ACPI tables are defined.

To solve this problem and consolidate FW PCI legacy IRQs parsing in
one single pcibios callback (pending final removal), this patch moves
DT PCI IRQ parsing to the pcibios_alloc_irq() callback (called by
PCI core code at device probe time) and adds ACPI PCI legacy IRQs
parsing to the same callback too, so that FW PCI legacy IRQs parsing
is confined in one single arch callback that can be easily removed
when code parsing PCI legacy IRQs is consolidated and moved to core
PCI code.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
---
 arch/arm64/kernel/pci.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index c72de66..15109c11 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -50,11 +50,16 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
 }
 
 /*
- * Try to assign the IRQ number from DT when adding a new device
+ * Try to assign the IRQ number when probing a new device
  */
-int pcibios_add_device(struct pci_dev *dev)
+int pcibios_alloc_irq(struct pci_dev *dev)
 {
-	dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
+	if (acpi_disabled)
+		dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
+#ifdef CONFIG_ACPI
+	else
+		return acpi_pci_irq_enable(dev);
+#endif
 
 	return 0;
 }
-- 
1.9.1

[PATCH V6 07/13] PCI: Provide common functions for ECAM mapping

[Tomasz Nowicki]

From: Jayachandran C <jchandra@broadcom.com>

Add config option PCI_GENERIC_ECAM and file drivers/pci/ecam.c to
provide generic functions for accessing memory mapped PCI config space.

The API is defined in drivers/pci/ecam.h and is written to replace the
API in drivers/pci/host/pci-host-common.h. The file defines a new
'struct pci_config_window' to hold the information related to a PCI
config area and its mapping. This structure is expected to be used as
sysdata for controllers that have ECAM based mapping.

Helper functions are provided to setup the mapping, free the mapping
and to implement the map_bus method in 'struct pci_ops'

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 drivers/pci/Kconfig  |   3 ++
 drivers/pci/Makefile |   2 +
 drivers/pci/ecam.c   | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/pci/ecam.h   |  61 +++++++++++++++++++++++
 4 files changed, 203 insertions(+)
 create mode 100644 drivers/pci/ecam.c
 create mode 100644 drivers/pci/ecam.h

diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 209292e..e930d62 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -83,6 +83,9 @@ config HT_IRQ
 config PCI_ATS
 	bool
 
+config PCI_GENERIC_ECAM
+	bool
+
 config PCI_IOV
 	bool "PCI IOV support"
 	depends on PCI
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 2154092..810aec8 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -55,6 +55,8 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o
 
 obj-$(CONFIG_PCI_STUB) += pci-stub.o
 
+obj-$(CONFIG_PCI_GENERIC_ECAM) += ecam.o
+
 obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
 
 obj-$(CONFIG_OF) += of.o
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
new file mode 100644
index 0000000..ff04c01
--- /dev/null
+++ b/drivers/pci/ecam.c
@@ -0,0 +1,137 @@
+/*
+ * Copyright 2016 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+
+#include "ecam.h"
+
+/*
+ * On 64 bit systems, we do a single ioremap for the whole config space
+ * since we have enough virtual address range available. On 32 bit, do an
+ * ioremap per bus.
+ */
+static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
+
+/*
+ * Create a PCI config space window
+ *  - reserve mem region
+ *  - alloc struct pci_config_window with space for all mappings
+ *  - ioremap the config space
+ */
+struct pci_config_window *pci_generic_ecam_create(struct device *dev,
+				phys_addr_t addr, u8 bus_start, u8 bus_end,
+				struct pci_generic_ecam_ops *ops)
+{
+	struct pci_config_window *cfg;
+	unsigned int bus_shift, bus_range, bsz, mapsz;
+	int i, nidx;
+	int err = -ENOMEM;
+
+	if (bus_end < bus_start)
+		return ERR_PTR(-EINVAL);
+
+	bus_shift = ops->bus_shift;
+	bus_range = bus_end - bus_start + 1;
+	bsz = 1 << bus_shift;
+	nidx = per_bus_mapping ? bus_range : 1;
+	mapsz = per_bus_mapping ? bsz : bus_range * bsz;
+	cfg = kzalloc(sizeof(*cfg) + nidx * sizeof(cfg->win[0]), GFP_KERNEL);
+	if (!cfg)
+		return ERR_PTR(-ENOMEM);
+
+	cfg->bus_start = bus_start;
+	cfg->bus_end = bus_end;
+	cfg->ops = ops;
+
+	if (!request_mem_region(addr, bus_range * bsz, "Configuration Space"))
+		goto err_exit;
+
+	/* cfgaddr has to be set after request_mem_region */
+	cfg->cfgaddr = addr;
+
+	for (i = 0; i < nidx; i++) {
+		cfg->win[i] = ioremap(addr + i * mapsz, mapsz);
+		if (!cfg->win[i])
+			goto err_exit;
+	}
+
+	if (cfg->ops->init) {
+		err = cfg->ops->init(dev, cfg);
+		if (err)
+			goto err_exit;
+	}
+	return cfg;
+
+err_exit:
+	pci_generic_ecam_free(cfg);
+	return ERR_PTR(err);
+}
+
+/*
+ * Free a config space mapping
+ */
+void pci_generic_ecam_free(struct pci_config_window *cfg)
+{
+	unsigned int bus_range;
+	int i, nidx;
+
+	bus_range = cfg->bus_end - cfg->bus_start + 1;
+	nidx = per_bus_mapping ? bus_range : 1;
+	for (i = 0; i < nidx; i++)
+		if (cfg->win[i])
+			iounmap(cfg->win[i]);
+	if (cfg->cfgaddr)
+		release_mem_region(cfg->cfgaddr,
+				   bus_range << cfg->ops->bus_shift);
+	kfree(cfg);
+}
+
+/*
+ * Function to implement the pci_ops ->map_bus method
+ */
+void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
+				       int where)
+{
+	struct pci_config_window *cfg = bus->sysdata;
+	unsigned int devfn_shift = cfg->ops->bus_shift - 8;
+	unsigned int busn = bus->number;
+	void __iomem *base;
+
+	if (busn < cfg->bus_start || busn > cfg->bus_end)
+		return NULL;
+
+	busn -= cfg->bus_start;
+	if (per_bus_mapping)
+		base = cfg->win[busn];
+	else
+		base = cfg->win[0] + (busn << cfg->ops->bus_shift);
+	return base + (devfn << devfn_shift) + where;
+}
+
+/* default ECAM ops */
+struct pci_generic_ecam_ops pci_generic_ecam_default_ops = {
+	.bus_shift	= 20,
+	.pci_ops		= {
+		.map_bus	= pci_generic_ecam_map_bus,
+		.read		= pci_generic_config_read,
+		.write		= pci_generic_config_write,
+	}
+};
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
new file mode 100644
index 0000000..34c0aba
--- /dev/null
+++ b/drivers/pci/ecam.h
@@ -0,0 +1,61 @@
+/*
+ * Copyright 2016 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#ifndef DRIVERS_PCI_ECAM_H
+#define DRIVERS_PCI_ECAM_H
+
+#include <linux/kernel.h>
+#include <linux/platform_device.h>
+
+/*
+ * struct to hold pci ops and bus shift of the config window
+ * for a PCI controller.
+ */
+struct pci_config_window;
+struct pci_generic_ecam_ops {
+	unsigned int			bus_shift;
+	struct pci_ops			pci_ops;
+	int				(*init)(struct device *,
+						struct pci_config_window *);
+};
+
+/*
+ * struct to hold the mappings of a config space window. This
+ * will be allocated with enough entries in win[] to hold all
+ * the mappings for the bus range.
+ */
+struct pci_config_window {
+	phys_addr_t			cfgaddr;
+	u16				domain;
+	u8				bus_start;
+	u8				bus_end;
+	void				*priv;
+	struct pci_generic_ecam_ops	*ops;
+	void __iomem			*win[0];
+};
+
+/* create and free for pci_config_window */
+struct pci_config_window *pci_generic_ecam_create(struct device *dev,
+				phys_addr_t addr, u8 bus_start, u8 bus_end,
+				struct pci_generic_ecam_ops *ops);
+void pci_generic_ecam_free(struct pci_config_window *cfg);
+
+/* map_bus when ->sysdata is an instance of pci_config_window */
+void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
+				       int where);
+/* default ECAM ops, bus shift 20, generic read and write */
+extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
+
+#endif
-- 
1.9.1

[PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Tomasz Nowicki]

From: Jayachandran C <jchandra@broadcom.com>

Use functions provided by drivers/pci/ecam.h for mapping the config
space in drivers/pci/host/pci-host-common.c, and update its users to
use 'struct pci_config_window' and 'struct pci_generic_ecam_ops'

The changes are mostly to use 'struct pci_config_window' in place of
'struct gen_pci'. Some of the fields of gen_pci were only used
temporarily and can be eliminated by using local variables or function
arguments, these are not carried over to struct pci_config_window.

pci-thunder-ecam.c and pci-thunder-pem.c are the only users of the
pci_host_common_probe function and the gen_pci structure, these have
been updated to use the new API as well.

The patch does not introduce any functional changes other than a very
minor one: with the new code, on 64-bit platforms, we do just a single
ioremap for the whole config space.

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 drivers/pci/ecam.h                  |   5 ++
 drivers/pci/host/Kconfig            |   1 +
 drivers/pci/host/pci-host-common.c  | 119 ++++++++++++++++--------------------
 drivers/pci/host/pci-host-common.h  |  47 --------------
 drivers/pci/host/pci-host-generic.c |  52 +++-------------
 drivers/pci/host/pci-thunder-ecam.c |  39 +++---------
 drivers/pci/host/pci-thunder-pem.c  |  88 ++++++++++++--------------
 7 files changed, 115 insertions(+), 236 deletions(-)
 delete mode 100644 drivers/pci/host/pci-host-common.h

diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 34c0aba..706621a 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -58,4 +58,9 @@ void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
 /* default ECAM ops, bus shift 20, generic read and write */
 extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
 
+#ifdef CONFIG_PCI_HOST_GENERIC
+/* for DT based pci controllers that support ECAM */
+int pci_host_common_probe(struct platform_device *pdev,
+			  struct pci_generic_ecam_ops *ops);
+#endif
 #endif
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index 7a0780d..31d6eb5 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -82,6 +82,7 @@ config PCI_HOST_GENERIC
 	bool "Generic PCI host controller"
 	depends on (ARM || ARM64) && OF
 	select PCI_HOST_COMMON
+	select PCI_GENERIC_ECAM
 	help
 	  Say Y here if you want to support a simple generic PCI host
 	  controller, such as the one emulated by kvmtool.
diff --git a/drivers/pci/host/pci-host-common.c b/drivers/pci/host/pci-host-common.c
index e9f850f..99d99b3 100644
--- a/drivers/pci/host/pci-host-common.c
+++ b/drivers/pci/host/pci-host-common.c
@@ -22,27 +22,21 @@
 #include <linux/of_pci.h>
 #include <linux/platform_device.h>
 
-#include "pci-host-common.h"
+#include "../ecam.h"
 
-static void gen_pci_release_of_pci_ranges(struct gen_pci *pci)
-{
-	pci_free_resource_list(&pci->resources);
-}
-
-static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci)
+static int gen_pci_parse_request_of_pci_ranges(struct device *dev,
+		       struct list_head *resources, struct resource **bus_range)
 {
 	int err, res_valid = 0;
-	struct device *dev = pci->host.dev.parent;
 	struct device_node *np = dev->of_node;
 	resource_size_t iobase;
 	struct resource_entry *win;
 
-	err = of_pci_get_host_bridge_resources(np, 0, 0xff, &pci->resources,
-					       &iobase);
+	err = of_pci_get_host_bridge_resources(np, 0, 0xff, resources, &iobase);
 	if (err)
 		return err;
 
-	resource_list_for_each_entry(win, &pci->resources) {
+	resource_list_for_each_entry(win, resources) {
 		struct resource *parent, *res = win->res;
 
 		switch (resource_type(res)) {
@@ -60,7 +54,7 @@ static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci)
 			res_valid |= !(res->flags & IORESOURCE_PREFETCH);
 			break;
 		case IORESOURCE_BUS:
-			pci->cfg.bus_range = res;
+			*bus_range = res;
 		default:
 			continue;
 		}
@@ -79,65 +73,67 @@ static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci)
 	return 0;
 
 out_release_res:
-	gen_pci_release_of_pci_ranges(pci);
 	return err;
 }
 
-static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)
+static void gen_pci_generic_unmap_cfg(void *ptr)
+{
+	pci_generic_ecam_free((struct pci_config_window *)ptr);
+}
+
+static struct pci_config_window *gen_pci_init(struct device *dev,
+		struct list_head *resources, struct pci_generic_ecam_ops *ops)
 {
 	int err;
-	u8 bus_max;
-	resource_size_t busn;
-	struct resource *bus_range;
-	struct device *dev = pci->host.dev.parent;
-	struct device_node *np = dev->of_node;
-	u32 sz = 1 << pci->cfg.ops->bus_shift;
+	struct resource cfgres;
+	struct resource *bus_range = NULL;
+	struct pci_config_window *cfg;
+	unsigned int bus_shift = ops->bus_shift;
 
-	err = of_address_to_resource(np, 0, &pci->cfg.res);
+	/* Parse our PCI ranges and request their resources */
+	err = gen_pci_parse_request_of_pci_ranges(dev, resources, &bus_range);
+	if (err)
+		goto err_out;
+
+	err = of_address_to_resource(dev->of_node, 0, &cfgres);
 	if (err) {
 		dev_err(dev, "missing \"reg\" property\n");
-		return err;
+		goto err_out;
 	}
 
 	/* Limit the bus-range to fit within reg */
-	bus_max = pci->cfg.bus_range->start +
-		  (resource_size(&pci->cfg.res) >> pci->cfg.ops->bus_shift) - 1;
-	pci->cfg.bus_range->end = min_t(resource_size_t,
-					pci->cfg.bus_range->end, bus_max);
-
-	pci->cfg.win = devm_kcalloc(dev, resource_size(pci->cfg.bus_range),
-				    sizeof(*pci->cfg.win), GFP_KERNEL);
-	if (!pci->cfg.win)
-		return -ENOMEM;
-
-	/* Map our Configuration Space windows */
-	if (!devm_request_mem_region(dev, pci->cfg.res.start,
-				     resource_size(&pci->cfg.res),
-				     "Configuration Space"))
-		return -ENOMEM;
-
-	bus_range = pci->cfg.bus_range;
-	for (busn = bus_range->start; busn <= bus_range->end; ++busn) {
-		u32 idx = busn - bus_range->start;
-
-		pci->cfg.win[idx] = devm_ioremap(dev,
-						 pci->cfg.res.start + idx * sz,
-						 sz);
-		if (!pci->cfg.win[idx])
-			return -ENOMEM;
+	bus_range->end = min(bus_range->end,
+		bus_range->start + (resource_size(&cfgres) >> bus_shift) - 1);
+
+	cfg = pci_generic_ecam_create(dev, cfgres.start, bus_range->start,
+				      bus_range->end, ops);
+	if (IS_ERR(cfg)) {
+		err = PTR_ERR(cfg);
+		goto err_out;
 	}
 
-	return 0;
+	err = devm_add_action(dev, gen_pci_generic_unmap_cfg, cfg);
+	if (err) {
+		gen_pci_generic_unmap_cfg(cfg);
+		goto err_out;
+	}
+	return cfg;
+
+err_out:
+	pci_free_resource_list(resources);
+	return ERR_PTR(err);
 }
 
 int pci_host_common_probe(struct platform_device *pdev,
-			  struct gen_pci *pci)
+			  struct pci_generic_ecam_ops *ops)
+
 {
-	int err;
 	const char *type;
 	struct device *dev = &pdev->dev;
 	struct device_node *np = dev->of_node;
 	struct pci_bus *bus, *child;
+	struct pci_config_window *cfg;
+	struct list_head resources;
 
 	type = of_get_property(np, "device_type", NULL);
 	if (!type || strcmp(type, "pci")) {
@@ -147,29 +143,18 @@ int pci_host_common_probe(struct platform_device *pdev,
 
 	of_pci_check_probe_only();
 
-	pci->host.dev.parent = dev;
-	INIT_LIST_HEAD(&pci->host.windows);
-	INIT_LIST_HEAD(&pci->resources);
-
-	/* Parse our PCI ranges and request their resources */
-	err = gen_pci_parse_request_of_pci_ranges(pci);
-	if (err)
-		return err;
-
 	/* Parse and map our Configuration Space windows */
-	err = gen_pci_parse_map_cfg_windows(pci);
-	if (err) {
-		gen_pci_release_of_pci_ranges(pci);
-		return err;
-	}
+	INIT_LIST_HEAD(&resources);
+	cfg = gen_pci_init(dev, &resources, ops);
+	if (IS_ERR(cfg))
+		return PTR_ERR(cfg);
 
 	/* Do not reassign resources if probe only */
 	if (!pci_has_flag(PCI_PROBE_ONLY))
 		pci_add_flags(PCI_REASSIGN_ALL_RSRC | PCI_REASSIGN_ALL_BUS);
 
-
-	bus = pci_scan_root_bus(dev, pci->cfg.bus_range->start,
-				&pci->cfg.ops->ops, pci, &pci->resources);
+	bus = pci_scan_root_bus(dev, cfg->bus_start, &ops->pci_ops, cfg,
+				&resources);
 	if (!bus) {
 		dev_err(dev, "Scanning rootbus failed");
 		return -ENODEV;
diff --git a/drivers/pci/host/pci-host-common.h b/drivers/pci/host/pci-host-common.h
deleted file mode 100644
index 09f3fa0..0000000
--- a/drivers/pci/host/pci-host-common.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- *
- * Copyright (C) 2014 ARM Limited
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#ifndef _PCI_HOST_COMMON_H
-#define _PCI_HOST_COMMON_H
-
-#include <linux/kernel.h>
-#include <linux/platform_device.h>
-
-struct gen_pci_cfg_bus_ops {
-	u32 bus_shift;
-	struct pci_ops ops;
-};
-
-struct gen_pci_cfg_windows {
-	struct resource				res;
-	struct resource				*bus_range;
-	void __iomem				**win;
-
-	struct gen_pci_cfg_bus_ops		*ops;
-};
-
-struct gen_pci {
-	struct pci_host_bridge			host;
-	struct gen_pci_cfg_windows		cfg;
-	struct list_head			resources;
-};
-
-int pci_host_common_probe(struct platform_device *pdev,
-			  struct gen_pci *pci);
-
-#endif /* _PCI_HOST_COMMON_H */
diff --git a/drivers/pci/host/pci-host-generic.c b/drivers/pci/host/pci-host-generic.c
index e8aa78f..0150a62 100644
--- a/drivers/pci/host/pci-host-generic.c
+++ b/drivers/pci/host/pci-host-generic.c
@@ -25,41 +25,12 @@
 #include <linux/of_pci.h>
 #include <linux/platform_device.h>
 
-#include "pci-host-common.h"
+#include "../ecam.h"
 
-static void __iomem *gen_pci_map_cfg_bus_cam(struct pci_bus *bus,
-					     unsigned int devfn,
-					     int where)
-{
-	struct gen_pci *pci = bus->sysdata;
-	resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
-	return pci->cfg.win[idx] + ((devfn << 8) | where);
-}
-
-static struct gen_pci_cfg_bus_ops gen_pci_cfg_cam_bus_ops = {
+static struct pci_generic_ecam_ops gen_pci_cfg_cam_bus_ops = {
 	.bus_shift	= 16,
-	.ops		= {
-		.map_bus	= gen_pci_map_cfg_bus_cam,
-		.read		= pci_generic_config_read,
-		.write		= pci_generic_config_write,
-	}
-};
-
-static void __iomem *gen_pci_map_cfg_bus_ecam(struct pci_bus *bus,
-					      unsigned int devfn,
-					      int where)
-{
-	struct gen_pci *pci = bus->sysdata;
-	resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
-	return pci->cfg.win[idx] + ((devfn << 12) | where);
-}
-
-static struct gen_pci_cfg_bus_ops gen_pci_cfg_ecam_bus_ops = {
-	.bus_shift	= 20,
-	.ops		= {
-		.map_bus	= gen_pci_map_cfg_bus_ecam,
+	.pci_ops	= {
+		.map_bus	= pci_generic_ecam_map_bus,
 		.read		= pci_generic_config_read,
 		.write		= pci_generic_config_write,
 	}
@@ -70,25 +41,22 @@ static const struct of_device_id gen_pci_of_match[] = {
 	  .data = &gen_pci_cfg_cam_bus_ops },
 
 	{ .compatible = "pci-host-ecam-generic",
-	  .data = &gen_pci_cfg_ecam_bus_ops },
+	  .data = &pci_generic_ecam_default_ops },
 
 	{ },
 };
+
 MODULE_DEVICE_TABLE(of, gen_pci_of_match);
 
 static int gen_pci_probe(struct platform_device *pdev)
 {
-	struct device *dev = &pdev->dev;
 	const struct of_device_id *of_id;
-	struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL);
-
-	if (!pci)
-		return -ENOMEM;
+	struct pci_generic_ecam_ops *ops;
 
-	of_id = of_match_node(gen_pci_of_match, dev->of_node);
-	pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
+	of_id = of_match_node(gen_pci_of_match, pdev->dev.of_node);
+	ops = (struct pci_generic_ecam_ops *)of_id->data;
 
-	return pci_host_common_probe(pdev, pci);
+	return pci_host_common_probe(pdev, ops);
 }
 
 static struct platform_driver gen_pci_driver = {
diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
index d71935cb..f67c6d7 100644
--- a/drivers/pci/host/pci-thunder-ecam.c
+++ b/drivers/pci/host/pci-thunder-ecam.c
@@ -13,18 +13,7 @@
 #include <linux/of.h>
 #include <linux/platform_device.h>
 
-#include "pci-host-common.h"
-
-/* Mapping is standard ECAM */
-static void __iomem *thunder_ecam_map_bus(struct pci_bus *bus,
-					  unsigned int devfn,
-					  int where)
-{
-	struct gen_pci *pci = bus->sysdata;
-	resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
-	return pci->cfg.win[idx] + ((devfn << 12) | where);
-}
+#include "../ecam.h"
 
 static void set_val(u32 v, int where, int size, u32 *val)
 {
@@ -99,7 +88,7 @@ static int handle_ea_bar(u32 e0, int bar, struct pci_bus *bus,
 static int thunder_ecam_p2_config_read(struct pci_bus *bus, unsigned int devfn,
 				       int where, int size, u32 *val)
 {
-	struct gen_pci *pci = bus->sysdata;
+	struct pci_config_window *cfg = bus->sysdata;
 	int where_a = where & ~3;
 	void __iomem *addr;
 	u32 node_bits;
@@ -129,7 +118,7 @@ static int thunder_ecam_p2_config_read(struct pci_bus *bus, unsigned int devfn,
 	 * the config space access window.  Since we are working with
 	 * the high-order 32 bits, shift everything down by 32 bits.
 	 */
-	node_bits = (pci->cfg.res.start >> 32) & (1 << 12);
+	node_bits = (cfg->cfgaddr >> 32) & (1 << 12);
 
 	v |= node_bits;
 	set_val(v, where, size, val);
@@ -358,36 +347,24 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn,
 	return pci_generic_config_write(bus, devfn, where, size, val);
 }
 
-static struct gen_pci_cfg_bus_ops thunder_ecam_bus_ops = {
+static struct pci_generic_ecam_ops pci_thunder_ecam_ops = {
 	.bus_shift	= 20,
-	.ops		= {
-		.map_bus        = thunder_ecam_map_bus,
+	.pci_ops	= {
+		.map_bus        = pci_generic_ecam_map_bus,
 		.read           = thunder_ecam_config_read,
 		.write          = thunder_ecam_config_write,
 	}
 };
 
 static const struct of_device_id thunder_ecam_of_match[] = {
-	{ .compatible = "cavium,pci-host-thunder-ecam",
-	  .data = &thunder_ecam_bus_ops },
-
+	{ .compatible = "cavium,pci-host-thunder-ecam" },
 	{ },
 };
 MODULE_DEVICE_TABLE(of, thunder_ecam_of_match);
 
 static int thunder_ecam_probe(struct platform_device *pdev)
 {
-	struct device *dev = &pdev->dev;
-	const struct of_device_id *of_id;
-	struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL);
-
-	if (!pci)
-		return -ENOMEM;
-
-	of_id = of_match_node(thunder_ecam_of_match, dev->of_node);
-	pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
-
-	return pci_host_common_probe(pdev, pci);
+	return pci_host_common_probe(pdev, &pci_thunder_ecam_ops);
 }
 
 static struct platform_driver thunder_ecam_driver = {
diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index cabb92a..91cfeb9 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -20,34 +20,22 @@
 #include <linux/of_pci.h>
 #include <linux/platform_device.h>
 
-#include "pci-host-common.h"
+#include "../ecam.h"
 
 #define PEM_CFG_WR 0x28
 #define PEM_CFG_RD 0x30
 
 struct thunder_pem_pci {
-	struct gen_pci	gen_pci;
 	u32		ea_entry[3];
 	void __iomem	*pem_reg_base;
 };
 
-static void __iomem *thunder_pem_map_bus(struct pci_bus *bus,
-					 unsigned int devfn, int where)
-{
-	struct gen_pci *pci = bus->sysdata;
-	resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
-	return pci->cfg.win[idx] + ((devfn << 16) | where);
-}
-
 static int thunder_pem_bridge_read(struct pci_bus *bus, unsigned int devfn,
 				   int where, int size, u32 *val)
 {
 	u64 read_val;
-	struct thunder_pem_pci *pem_pci;
-	struct gen_pci *pci = bus->sysdata;
-
-	pem_pci = container_of(pci, struct thunder_pem_pci, gen_pci);
+	struct pci_config_window *cfg = bus->sysdata;
+	struct thunder_pem_pci *pem_pci = (struct thunder_pem_pci *)cfg->priv;
 
 	if (devfn != 0 || where >= 2048) {
 		*val = ~0;
@@ -132,17 +120,17 @@ static int thunder_pem_bridge_read(struct pci_bus *bus, unsigned int devfn,
 static int thunder_pem_config_read(struct pci_bus *bus, unsigned int devfn,
 				   int where, int size, u32 *val)
 {
-	struct gen_pci *pci = bus->sysdata;
+	struct pci_config_window *cfg = bus->sysdata;
 
-	if (bus->number < pci->cfg.bus_range->start ||
-	    bus->number > pci->cfg.bus_range->end)
+	if (bus->number < cfg->bus_start ||
+	    bus->number > cfg->bus_end)
 		return PCIBIOS_DEVICE_NOT_FOUND;
 
 	/*
 	 * The first device on the bus is the PEM PCIe bridge.
 	 * Special case its config access.
 	 */
-	if (bus->number == pci->cfg.bus_range->start)
+	if (bus->number == cfg->bus_start)
 		return thunder_pem_bridge_read(bus, devfn, where, size, val);
 
 	return pci_generic_config_read(bus, devfn, where, size, val);
@@ -187,12 +175,11 @@ static u32 thunder_pem_bridge_w1c_bits(int where)
 static int thunder_pem_bridge_write(struct pci_bus *bus, unsigned int devfn,
 				    int where, int size, u32 val)
 {
-	struct gen_pci *pci = bus->sysdata;
-	struct thunder_pem_pci *pem_pci;
+	struct pci_config_window *cfg = bus->sysdata;
+	struct thunder_pem_pci *pem_pci = (struct thunder_pem_pci *)cfg->priv;
 	u64 write_val, read_val;
 	u32 mask = 0;
 
-	pem_pci = container_of(pci, struct thunder_pem_pci, gen_pci);
 
 	if (devfn != 0 || where >= 2048)
 		return PCIBIOS_DEVICE_NOT_FOUND;
@@ -256,53 +243,34 @@ static int thunder_pem_bridge_write(struct pci_bus *bus, unsigned int devfn,
 static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn,
 				    int where, int size, u32 val)
 {
-	struct gen_pci *pci = bus->sysdata;
+	struct pci_config_window *cfg = bus->sysdata;
 
-	if (bus->number < pci->cfg.bus_range->start ||
-	    bus->number > pci->cfg.bus_range->end)
+	if (bus->number < cfg->bus_start ||
+	    bus->number > cfg->bus_end)
 		return PCIBIOS_DEVICE_NOT_FOUND;
 	/*
 	 * The first device on the bus is the PEM PCIe bridge.
 	 * Special case its config access.
 	 */
-	if (bus->number == pci->cfg.bus_range->start)
+	if (bus->number == cfg->bus_start)
 		return thunder_pem_bridge_write(bus, devfn, where, size, val);
 
 
 	return pci_generic_config_write(bus, devfn, where, size, val);
 }
 
-static struct gen_pci_cfg_bus_ops thunder_pem_bus_ops = {
-	.bus_shift	= 24,
-	.ops		= {
-		.map_bus	= thunder_pem_map_bus,
-		.read		= thunder_pem_config_read,
-		.write		= thunder_pem_config_write,
-	}
-};
-
-static const struct of_device_id thunder_pem_of_match[] = {
-	{ .compatible = "cavium,pci-host-thunder-pem",
-	  .data = &thunder_pem_bus_ops },
-
-	{ },
-};
-MODULE_DEVICE_TABLE(of, thunder_pem_of_match);
-
-static int thunder_pem_probe(struct platform_device *pdev)
+static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
 {
-	struct device *dev = &pdev->dev;
-	const struct of_device_id *of_id;
 	resource_size_t bar4_start;
 	struct resource *res_pem;
 	struct thunder_pem_pci *pem_pci;
+	struct platform_device *pdev;
 
 	pem_pci = devm_kzalloc(dev, sizeof(*pem_pci), GFP_KERNEL);
 	if (!pem_pci)
 		return -ENOMEM;
 
-	of_id = of_match_node(thunder_pem_of_match, dev->of_node);
-	pem_pci->gen_pci.cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
+	pdev = to_platform_device(dev);
 
 	/*
 	 * The second register range is the PEM bridge to the PCIe
@@ -330,7 +298,29 @@ static int thunder_pem_probe(struct platform_device *pdev)
 	pem_pci->ea_entry[1] = (u32)(res_pem->end - bar4_start) & ~3u;
 	pem_pci->ea_entry[2] = (u32)(bar4_start >> 32);
 
-	return pci_host_common_probe(pdev, &pem_pci->gen_pci);
+	cfg->priv = pem_pci;
+	return 0;
+}
+
+static struct pci_generic_ecam_ops pci_thunder_pem_ops = {
+	.bus_shift	= 24,
+	.init		= thunder_pem_init,
+	.pci_ops	= {
+		.map_bus	= pci_generic_ecam_map_bus,
+		.read		= thunder_pem_config_read,
+		.write		= thunder_pem_config_write,
+	}
+};
+
+static const struct of_device_id thunder_pem_of_match[] = {
+	{ .compatible = "cavium,pci-host-thunder-pem" },
+	{ },
+};
+MODULE_DEVICE_TABLE(of, thunder_pem_of_match);
+
+static int thunder_pem_probe(struct platform_device *pdev)
+{
+	return pci_host_common_probe(pdev, &pci_thunder_pem_ops);
 }
 
 static struct platform_driver thunder_pem_driver = {
-- 
1.9.1

[PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Tomasz Nowicki]

This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.

All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.

In order to handle PCI config space regions properly, we define new
MCFG interface which parses MCFG table and keep its entries
in a list. New pci_mcfg_init call is defined so that we do not depend
on PCI_MMCONFIG. Regions are not mapped until host bridge ask for it.

The implementation of pci_acpi_scan_root() looks up the saved MCFG entries
and sets up a new mapping. Generic PCI functions are used for
accessing config space. Driver selects PCI_GENERIC_ECAM and uses functions
from drivers/pci/ecam.h to create and access ECAM mappings.

As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.

This patch is heavily based on the updated version from Jayachandran C:
https://lkml.org/lkml/2016/4/11/908
git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 drivers/acpi/Kconfig        |   8 ++
 drivers/acpi/Makefile       |   1 +
 drivers/acpi/bus.c          |   1 +
 drivers/acpi/pci_gen_host.c | 231 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h         |   6 ++
 5 files changed, 247 insertions(+)
 create mode 100644 drivers/acpi/pci_gen_host.c

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 183ffa3..70272c5 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
 	  i.e., segment/bus/device/function tuples, with physical slots in
 	  the system.  If you are unsure, say N.
 
+config ACPI_PCI_HOST_GENERIC
+	bool
+	select PCI_GENERIC_ECAM
+	help
+	  Select this config option from the architecture Kconfig,
+	  if it is preferred to enable ACPI PCI host controller driver which
+	  has no arch-specific assumptions.
+
 config X86_PM_TIMER
 	bool "Power Management Timer Support" if EXPERT
 	depends on X86
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index 81e5cbc..b12fa64 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
 acpi-y				+= ec.o
 acpi-$(CONFIG_ACPI_DOCK)	+= dock.o
 acpi-y				+= pci_root.o pci_link.o pci_irq.o
+obj-$(CONFIG_ACPI_PCI_HOST_GENERIC)	+= pci_gen_host.o
 acpi-y				+= acpi_lpss.o acpi_apd.o
 acpi-y				+= acpi_platform.o
 acpi-y				+= acpi_pnp.o
diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
index c068c82..803a1d7 100644
--- a/drivers/acpi/bus.c
+++ b/drivers/acpi/bus.c
@@ -1107,6 +1107,7 @@ static int __init acpi_init(void)
 	}
 
 	pci_mmcfg_late_init();
+	pci_mcfg_init();
 	acpi_scan_init();
 	acpi_ec_init();
 	acpi_debugfs_init();
diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
new file mode 100644
index 0000000..fd360b5
--- /dev/null
+++ b/drivers/acpi/pci_gen_host.c
@@ -0,0 +1,231 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+#include <linux/sfi_acpi.h>
+#include <linux/slab.h>
+
+#include "../pci/ecam.h"
+
+#define PREFIX	"ACPI: "
+
+/* Structure to hold entries from the MCFG table */
+struct mcfg_entry {
+	struct list_head	list;
+	phys_addr_t		addr;
+	u16			segment;
+	u8			bus_start;
+	u8			bus_end;
+};
+
+/* List to save mcfg entries */
+static LIST_HEAD(pci_mcfg_list);
+static DEFINE_MUTEX(pci_mcfg_lock);
+
+/* ACPI info for generic ACPI PCI controller */
+struct acpi_pci_generic_root_info {
+	struct acpi_pci_root_info	common;
+	struct pci_config_window	*cfg;	/* config space mapping */
+};
+
+/* Find the entry in mcfg list which contains range bus_start */
+static struct mcfg_entry *pci_mcfg_lookup(u16 seg, u8 bus_start)
+{
+	struct mcfg_entry *e;
+
+	list_for_each_entry(e, &pci_mcfg_list, list) {
+		if (e->segment == seg &&
+		    e->bus_start <= bus_start && bus_start <= e->bus_end)
+			return e;
+	}
+
+	return NULL;
+}
+
+
+/*
+ * Lookup the bus range for the domain in MCFG, and set up config space
+ * mapping.
+ */
+static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
+				       struct acpi_pci_generic_root_info *ri)
+{
+	u16 seg = root->segment;
+	u8 bus_start = root->secondary.start;
+	u8 bus_end = root->secondary.end;
+	struct pci_config_window *cfg;
+	struct mcfg_entry *e;
+	phys_addr_t addr;
+	int err = 0;
+
+	mutex_lock(&pci_mcfg_lock);
+	e = pci_mcfg_lookup(seg, bus_start);
+	if (!e) {
+		addr = acpi_pci_root_get_mcfg_addr(root->device->handle);
+		if (addr == 0) {
+			pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
+			       seg, bus_start, bus_end);
+			err = -ENOENT;
+			goto err_out;
+		}
+	} else {
+		if (bus_start != e->bus_start) {
+			pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
+			       seg, bus_start, bus_end, e->bus_start);
+			err = -EINVAL;
+			goto err_out;
+		} else if (bus_end != e->bus_end) {
+			pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
+				seg, bus_start, bus_end, e->bus_end);
+			bus_end = min(bus_end, e->bus_end);
+		}
+		addr = e->addr;
+	}
+
+	cfg = pci_generic_ecam_create(&root->device->dev, addr, bus_start,
+				      bus_end, &pci_generic_ecam_default_ops);
+	if (IS_ERR(cfg)) {
+		err = PTR_ERR(cfg);
+		pr_err("%04x:%02x-%02x error %d mapping CAM\n", seg,
+			bus_start, bus_end, err);
+		goto err_out;
+	}
+
+	cfg->domain = seg;
+	ri->cfg = cfg;
+err_out:
+	mutex_unlock(&pci_mcfg_lock);
+	return err;
+}
+
+/* release_info: free resrouces allocated by init_info */
+static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
+{
+	struct acpi_pci_generic_root_info *ri;
+
+	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
+	pci_generic_ecam_free(ri->cfg);
+	kfree(ri);
+}
+
+static struct acpi_pci_root_ops acpi_pci_root_ops = {
+	.release_info = pci_acpi_generic_release_info,
+};
+
+/* Interface called from ACPI code to setup PCI host controller */
+struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
+{
+	int node = acpi_get_node(root->device->handle);
+	struct acpi_pci_generic_root_info *ri;
+	struct pci_bus *bus, *child;
+	int err;
+
+	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
+	if (!ri)
+		return NULL;
+
+	err = pci_acpi_setup_ecam_mapping(root, ri);
+	if (err)
+		return NULL;
+
+	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
+	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
+				   ri->cfg);
+	if (!bus)
+		return NULL;
+
+	pci_bus_size_bridges(bus);
+	pci_bus_assign_resources(bus);
+
+	list_for_each_entry(child, &bus->children, node)
+		pcie_bus_configure_settings(child);
+
+	return bus;
+}
+
+/* handle MCFG table entries */
+static __init int pci_mcfg_parse(struct acpi_table_header *header)
+{
+	struct acpi_table_mcfg *mcfg;
+	struct acpi_mcfg_allocation *mptr;
+	struct mcfg_entry *e, *arr;
+	int i, n;
+
+	if (!header)
+		return -EINVAL;
+
+	mcfg = (struct acpi_table_mcfg *)header;
+	mptr = (struct acpi_mcfg_allocation *) &mcfg[1];
+	n = (header->length - sizeof(*mcfg)) / sizeof(*mptr);
+	if (n <= 0 || n > 255) {
+		pr_err(PREFIX " MCFG has incorrect entries (%d).\n", n);
+		return -EINVAL;
+	}
+
+	arr = kcalloc(n, sizeof(*arr), GFP_KERNEL);
+	if (!arr)
+		return -ENOMEM;
+
+	for (i = 0, e = arr; i < n; i++, mptr++, e++) {
+		e->segment = mptr->pci_segment;
+		e->addr =  mptr->address;
+		e->bus_start = mptr->start_bus_number;
+		e->bus_end = mptr->end_bus_number;
+		list_add(&e->list, &pci_mcfg_list);
+		pr_info(PREFIX
+			"MCFG entry for domain %04x [bus %02x-%02x] (base %pa)\n",
+			e->segment, e->bus_start, e->bus_end, &e->addr);
+	}
+
+	return 0;
+}
+
+/* Interface called by ACPI - parse and save MCFG table */
+void __init pci_mcfg_init(void)
+{
+	int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
+	if (err)
+		pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);
+	else if (list_empty(&pci_mcfg_list))
+		pr_info(PREFIX "No valid entries in MCFG table.\n");
+	else {
+		struct mcfg_entry *e;
+		int i = 0;
+		list_for_each_entry(e, &pci_mcfg_list, list)
+			i++;
+		pr_info(PREFIX "MCFG table loaded, %d entries\n", i);
+	}
+}
+
+/* Raw operations, works only for MCFG entries with an associated bus */
+int raw_pci_read(unsigned int domain, unsigned int busn, unsigned int devfn,
+		 int reg, int len, u32 *val)
+{
+	struct pci_bus *bus = pci_find_bus(domain, busn);
+
+	if (!bus)
+		return PCIBIOS_DEVICE_NOT_FOUND;
+	return bus->ops->read(bus, devfn, reg, len, val);
+}
+
+int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int devfn,
+		  int reg, int len, u32 val)
+{
+	struct pci_bus *bus = pci_find_bus(domain, busn);
+
+	if (!bus)
+		return PCIBIOS_DEVICE_NOT_FOUND;
+	return bus->ops->write(bus, devfn, reg, len, val);
+}
diff --git a/include/linux/pci.h b/include/linux/pci.h
index df1f33d..c0422ea 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1729,6 +1729,12 @@ static inline void pci_mmcfg_early_init(void) { }
 static inline void pci_mmcfg_late_init(void) { }
 #endif
 
+#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
+void __init pci_mcfg_init(void);
+#else
+static inline void pci_mcfg_init(void) { return; }
+#endif
+
 int pci_ext_cfg_avail(void);
 
 void __iomem *pci_ioremap_bar(struct pci_dev *pdev, int bar);
-- 
1.9.1

[PATCH V6 10/13] arm64, pci, acpi: Start using ACPI based PCI host controller driver for ARM64.

[Tomasz Nowicki]

It is perfectly fine to use ACPI_PCI_HOST_GENERIC for ARM64,
so lets get rid of PCI init and RAW ACPI accessor empty stubs
and go with full-blown PCI host controller driver.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
To: Catalin Marinas <catalin.marinas@arm.com>
To: Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
To: Will Deacon <will.deacon@arm.com>
To: Arnd Bergmann <arnd@arndb.de>
Cc: Liviu Dudau <Liviu.Dudau@arm.com>
---
 arch/arm64/Kconfig      |  1 +
 arch/arm64/kernel/pci.c | 24 ------------------------
 2 files changed, 1 insertion(+), 24 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 4f43622..1bded87 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -2,6 +2,7 @@ config ARM64
 	def_bool y
 	select ACPI_CCA_REQUIRED if ACPI
 	select ACPI_GENERIC_GSI if ACPI
+	select ACPI_PCI_HOST_GENERIC if ACPI
 	select ACPI_REDUCED_HARDWARE_ONLY if ACPI
 	select ARCH_HAS_DEVMEM_IS_ALLOWED
 	select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 15109c11..9a8e6f7 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -63,27 +63,3 @@ int pcibios_alloc_irq(struct pci_dev *dev)
 
 	return 0;
 }
-
-/*
- * raw_pci_read/write - Platform-specific PCI config space access.
- */
-int raw_pci_read(unsigned int domain, unsigned int bus,
-		  unsigned int devfn, int reg, int len, u32 *val)
-{
-	return -ENXIO;
-}
-
-int raw_pci_write(unsigned int domain, unsigned int bus,
-		unsigned int devfn, int reg, int len, u32 val)
-{
-	return -ENXIO;
-}
-
-#ifdef CONFIG_ACPI
-/* Root bridge scanning */
-struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
-{
-	/* TODO: Should be revisited when implementing PCI on ACPI */
-	return NULL;
-}
-#endif
-- 
1.9.1

[PATCH V6 11/13] pci, acpi: Match PCI config space accessors against platfrom specific quirks.

[Tomasz Nowicki]

Some platforms may not be fully compliant with generic set of PCI config
accessors. For these cases we implement the way to overwrite accessors
set prior to PCI buses enumeration. Algorithm traverses available quirk
list, matches against <DMI ID (optional), domain, bus number> tuple and
an extra match call and returns corresponding PCI config ops.
All quirks can be defined using:
DECLARE_ACPI_MCFG_FIXUP() macro and kept self contained. Example:

/* Additional DMI platform identification (optional) */
static const struct dmi_system_id foo_dmi[] = {
        {
                .ident = "<Platform ident string>",
                .matches = {
                        DMI_MATCH(DMI_SYS_VENDOR, "<system vendor>"),
                        DMI_MATCH(DMI_PRODUCT_NAME, "<product name>"),
                        DMI_MATCH(DMI_PRODUCT_VERSION, "product version"),
                },
        },
        { }
};

/* Custom PCI config ops */
static struct pci_generic_ecam_ops foo_pci_ops = {
	.bus_shift	= 24,
	.pci_ops = {
		.map_bus = pci_mcfg_dev_base,
		.read = foo_ecam_config_read,
		.write = foo_ecam_config_write,
	}
};

static int foo_match(struct pci_mcfg_fixup *fixup, struct acpi_pci_root *root)
{
	if (additional platform identification)
	        return true;
	return false;
}

DECLARE_ACPI_MCFG_FIXUP(foo_dmi, foo_init, &foo_root_ops, <domain_nr>, <bus_nr>);

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
---
 drivers/acpi/pci_gen_host.c       | 30 +++++++++++++++++++++++++++++-
 include/asm-generic/vmlinux.lds.h |  7 +++++++
 include/linux/pci-acpi.h          | 18 ++++++++++++++++++
 3 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
index fd360b5..e55dfca 100644
--- a/drivers/acpi/pci_gen_host.c
+++ b/drivers/acpi/pci_gen_host.c
@@ -11,6 +11,8 @@
  * You should have received a copy of the GNU General Public License
  * version 2 (GPLv2) along with this source code.
  */
+
+#include <linux/dmi.h>
 #include <linux/kernel.h>
 #include <linux/pci.h>
 #include <linux/pci-acpi.h>
@@ -54,6 +56,32 @@ static struct mcfg_entry *pci_mcfg_lookup(u16 seg, u8 bus_start)
 	return NULL;
 }
 
+extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
+extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
+
+static struct pci_generic_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
+{
+	int bus_num = root->secondary.start;
+	int domain = root->segment;
+	struct pci_cfg_fixup *f;
+
+	/*
+	 * Match against platform specific quirks and return corresponding
+	 * CAM ops.
+	 *
+	 * First match against PCI topology <domain:bus> then use DMI or
+	 * custom match handler.
+	 */
+	for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
+		if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
+		    (f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
+		    (f->system ? dmi_check_system(f->system) : 1) &&
+		    (f->match ? f->match(f, root) : 1))
+			return f->ops;
+	}
+	/* No quirks, use ECAM */
+	return &pci_generic_ecam_default_ops;
+}
 
 /*
  * Lookup the bus range for the domain in MCFG, and set up config space
@@ -95,7 +123,7 @@ static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
 	}
 
 	cfg = pci_generic_ecam_create(&root->device->dev, addr, bus_start,
-				      bus_end, &pci_generic_ecam_default_ops);
+				      bus_end, pci_acpi_get_ops(root));
 	if (IS_ERR(cfg)) {
 		err = PTR_ERR(cfg);
 		pr_err("%04x:%02x-%02x error %d mapping CAM\n", seg,
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 339125b..c53b6b7 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -298,6 +298,13 @@
 		VMLINUX_SYMBOL(__end_pci_fixups_suspend_late) = .;	\
 	}								\
 									\
+	/* ACPI MCFG quirks */						\
+	.acpi_fixup        : AT(ADDR(.acpi_fixup) - LOAD_OFFSET) {	\
+		VMLINUX_SYMBOL(__start_acpi_mcfg_fixups) = .;		\
+		*(.acpi_fixup_mcfg)					\
+		VMLINUX_SYMBOL(__end_acpi_mcfg_fixups) = .;		\
+	}								\
+									\
 	/* Built-in firmware blobs */					\
 	.builtin_fw        : AT(ADDR(.builtin_fw) - LOAD_OFFSET) {	\
 		VMLINUX_SYMBOL(__start_builtin_fw) = .;			\
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index a72e22d..9545988 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -71,6 +71,24 @@ struct acpi_pci_root_ops {
 	int (*prepare_resources)(struct acpi_pci_root_info *info);
 };
 
+struct pci_cfg_fixup {
+	const struct dmi_system_id *system;
+	bool (*match)(struct pci_cfg_fixup *, struct acpi_pci_root *);
+	struct pci_generic_ecam_ops *ops;
+	int domain;
+	int bus_num;
+};
+
+#define PCI_MCFG_DOMAIN_ANY	-1
+#define PCI_MCFG_BUS_ANY	-1
+
+/* Designate a routine to fix up buggy MCFG */
+#define DECLARE_ACPI_MCFG_FIXUP(system, match, ops, dom, bus)		\
+	static const struct pci_cfg_fixup __mcfg_fixup_##system##dom##bus\
+	 __used	__attribute__((__section__(".acpi_fixup_mcfg"),		\
+				aligned((sizeof(void *))))) =		\
+	{ system, match, ops, dom, bus };
+
 extern int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info);
 extern struct pci_bus *acpi_pci_root_create(struct acpi_pci_root *root,
 					    struct acpi_pci_root_ops *ops,
-- 
1.9.1

[PATCH V6 12/13] pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.

[Tomasz Nowicki]

Passes 1.x miss PCI enhanced allocation (EA) header for fixed-BARs,
thus these passes should use Cavium-specific config access functions
that synthesize the missing EA capabilities.

We already have DT driver which addresses errata requirements and
allows to use special PCI config accessors. Currently this driver uses
compatible = "cavium,pci-host-thunder-ecam" to mach against the errata.
For ACPI case we need explicit errata number and corresponding
DECLARE_ACPI_MCFG_FIXUP fixup code.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
---
 arch/arm64/Kconfig                  | 14 ++++++++++++++
 arch/arm64/include/asm/cpufeature.h |  3 ++-
 arch/arm64/kernel/cpu_errata.c      |  8 ++++++++
 drivers/pci/host/pci-thunder-ecam.c | 35 +++++++++++++++++++++++++++++++++++
 4 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 1bded87..b7614b8 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -445,6 +445,20 @@ config CAVIUM_ERRATUM_27456
 
 	  If unsure, say Y.
 
+config CAVIUM_ERRATUM_24575
+	bool "Cavium erratum 24575: Enhanced Allocation (EA) emualaion"
+	default y
+	help
+	  Enable workaround for erratum 24575.
+
+	  Early versions of the Cavium Thunder CN88XX processor are missing
+	  Enhanced Allocation (EA) capabilities for the fixed BAR addresses used
+	  by the on-SoC hardware blocks. The erratum adds config access
+	  functions that synthesize the missing EA capabilities for versions
+	  that are missing that information.
+
+	  If unsure, say Y.
+
 endmenu
 
 
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index b9b6494..a78364e 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -35,8 +35,9 @@
 #define ARM64_ALT_PAN_NOT_UAO			10
 #define ARM64_HAS_VIRT_HOST_EXTN		11
 #define ARM64_WORKAROUND_CAVIUM_27456		12
+#define ARM64_WORKAROUND_CAVIUM_24575		13
 
-#define ARM64_NCAPS				13
+#define ARM64_NCAPS				14
 
 #ifndef __ASSEMBLY__
 
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 06afd04..89c13d7 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -97,6 +97,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 			   (1 << MIDR_VARIANT_SHIFT) | 1),
 	},
 #endif
+#ifdef CONFIG_CAVIUM_ERRATUM_24575
+	{
+	/* Cavium ThunderX, pass 1.x */
+		.desc = "Cavium erratum 24575",
+		.capability = ARM64_WORKAROUND_CAVIUM_24575,
+		MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
+	},
+#endif
 	{
 	}
 };
diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
index f67c6d7..01de697 100644
--- a/drivers/pci/host/pci-thunder-ecam.c
+++ b/drivers/pci/host/pci-thunder-ecam.c
@@ -11,10 +11,14 @@
 #include <linux/ioport.h>
 #include <linux/of_pci.h>
 #include <linux/of.h>
+#include <linux/pci-acpi.h>
 #include <linux/platform_device.h>
 
+#include <asm/virt.h>
+
 #include "../ecam.h"
 
+
 static void set_val(u32 v, int where, int size, u32 *val)
 {
 	int shift = (where & 3) * 8;
@@ -376,5 +380,36 @@ static struct platform_driver thunder_ecam_driver = {
 };
 module_platform_driver(thunder_ecam_driver);
 
+#ifdef CONFIG_ACPI
+
+static bool needs_cavium_erratum_24575(struct pci_cfg_fixup *fixup,
+				      struct acpi_pci_root *root)
+{
+	/*
+	 * We must match errata code and be hypervisor, quirk does not apply
+	 * for virtual machines.
+	 */
+	return cpus_have_cap(ARM64_WORKAROUND_CAVIUM_24575) &&
+	       is_hyp_mode_available();
+}
+
+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
+			0, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
+			1, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
+			2, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
+			3, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
+			10, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
+			11, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
+			12, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
+			13, PCI_MCFG_BUS_ANY);
+#endif
+
 MODULE_DESCRIPTION("Thunder ECAM PCI host driver");
 MODULE_LICENSE("GPL v2");
-- 
1.9.1

[PATCH V6 13/13] pci, pci-thunder-pem: Add ACPI support for ThunderX PEM.

[Tomasz Nowicki]

This patch uses DECLARE_ACPI_MCFG_FIXUP to overwrite PCI config accessors.
Also, it provides alternative way to find additional configuration region:
thunder_pem_get_acpi_res is looking for host bridge's child (_HID "THRX0001")
which contains mentioned configuration region description.
See example below:

Device (PEM0) {
    Name (_HID, EISAID ("PNP0A08"))
    Name (_CID, EISAID ("PNP0A03"))

    [...]

    Device (CFG0)
    {
      Name (_HID, "THRX0001") // PEM configuration space resources
      Name (_CRS, ResourceTemplate () {
        QWordMemory(ResourceConsumer, PosDecode, MinFixed, MaxFixed,
          NonCacheable, ReadWrite, 0, 0x87e0c5000000, 0x87E0C5FFFFFF,
          0, 0x01000000)
      })
    }
}

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
---
 drivers/pci/host/pci-thunder-pem.c | 137 ++++++++++++++++++++++++++++++++++---
 1 file changed, 128 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index 91cfeb9..685cd79 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -18,6 +18,7 @@
 #include <linux/module.h>
 #include <linux/of_address.h>
 #include <linux/of_pci.h>
+#include <linux/pci-acpi.h>
 #include <linux/platform_device.h>
 
 #include "../ecam.h"
@@ -259,6 +260,83 @@ static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn,
 	return pci_generic_config_write(bus, devfn, where, size, val);
 }
 
+#ifdef CONFIG_ACPI
+
+struct pem_acpi_res {
+	struct resource	resource;
+	int found;
+};
+
+static acpi_status
+thunder_pem_cfg(struct acpi_resource *resource, void *ctx)
+{
+	struct pem_acpi_res *pem_ctx = ctx;
+	struct resource *res = &pem_ctx->resource;
+
+	if ((resource->type != ACPI_RESOURCE_TYPE_ADDRESS64) ||
+	    (resource->data.address32.resource_type != ACPI_MEMORY_RANGE))
+		return AE_OK;
+
+	res->start = resource->data.address64.address.minimum;
+	res->end = resource->data.address64.address.maximum;
+	res->flags = IORESOURCE_MEM;
+
+	pem_ctx->found++;
+	return AE_OK;
+}
+
+static acpi_status
+thunder_pem_find_dev(acpi_handle handle, u32 level, void *ctx, void **ret)
+{
+	struct pem_acpi_res *pem_ctx = ctx;
+	struct acpi_device_info *info;
+	acpi_status status = AE_OK;
+
+	status = acpi_get_object_info(handle, &info);
+	if (ACPI_FAILURE(status))
+		return AE_OK;
+
+	if (strncmp(info->hardware_id.string, "THRX0001", 8) != 0)
+		goto out;
+
+	pem_ctx->found = 0;
+	status = acpi_walk_resources(handle, METHOD_NAME__CRS, thunder_pem_cfg,
+				     pem_ctx);
+	if (ACPI_FAILURE(status))
+		goto out;
+
+	if (pem_ctx->found)
+		status = AE_CTRL_TERMINATE;
+out:
+	kfree(info);
+	return status;
+}
+
+static struct resource *thunder_pem_get_acpi_res(struct device *dev)
+{
+	struct acpi_device *adev = ACPI_COMPANION(dev);
+	acpi_handle handle = acpi_device_handle(adev);
+	struct pem_acpi_res *pem_ctx;
+	acpi_status status;
+
+	pem_ctx = devm_kzalloc(dev, sizeof(*pem_ctx), GFP_KERNEL);
+	if (!pem_ctx)
+		return NULL;
+
+	status = acpi_walk_namespace(ACPI_TYPE_DEVICE, handle, 1,
+				     thunder_pem_find_dev, NULL, pem_ctx, NULL);
+	if (ACPI_FAILURE(status) || !pem_ctx->found)
+		return NULL;
+
+	return &pem_ctx->resource;
+}
+#else
+static struct resource *thunder_pem_get_acpi_res(struct device *dev)
+{
+	return NULL;
+}
+#endif
+
 static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
 {
 	resource_size_t bar4_start;
@@ -270,16 +348,20 @@ static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
 	if (!pem_pci)
 		return -ENOMEM;
 
-	pdev = to_platform_device(dev);
-
-	/*
-	 * The second register range is the PEM bridge to the PCIe
-	 * bus.  It has a different config access method than those
-	 * devices behind the bridge.
-	 */
-	res_pem = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+	if (acpi_disabled) {
+		pdev = to_platform_device(dev);
+
+		/*
+		 * The second register range is the PEM bridge to the PCIe
+		 * bus.  It has a different config access method than those
+		 * devices behind the bridge.
+		 */
+		res_pem = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+	} else {
+		res_pem = thunder_pem_get_acpi_res(dev);
+	}
 	if (!res_pem) {
-		dev_err(dev, "missing \"reg[1]\"property\n");
+		dev_err(dev, "missing configuration region\n");
 		return -EINVAL;
 	}
 
@@ -332,5 +414,42 @@ static struct platform_driver thunder_pem_driver = {
 };
 module_platform_driver(thunder_pem_driver);
 
+#ifdef CONFIG_ACPI
+
+static bool thunder_pem_acpi_init(struct pci_cfg_fixup *fixup,
+				 struct acpi_pci_root *root)
+{
+	u32 midr = read_cpuid_id();
+
+	return (MIDR_IMPLEMENTOR(midr) == ARM_CPU_IMP_CAVIUM) &&
+	    (MIDR_PARTNUM(midr) == CAVIUM_CPU_PART_THUNDERX);
+}
+
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			4, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			5, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			6, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			7, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			8, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			9, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			14, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			15, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			16, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			17, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			18, PCI_MCFG_BUS_ANY);
+DECLARE_ACPI_MCFG_FIXUP(NULL, thunder_pem_acpi_init, &pci_thunder_pem_ops,
+			19, PCI_MCFG_BUS_ANY);
+#endif
+
 MODULE_DESCRIPTION("Thunder PEM PCIe host driver");
 MODULE_LICENSE("GPL v2");
-- 
1.9.1

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Jon Masters]

On 04/15/2016 01:06 PM, Tomasz Nowicki wrote:
> From the functionality point of view this series might be split into the
> following logic parts:
> 1. Necessary fixes as the preparation for using driver on ARM64.
> 2. New ECAM API and update for users of the pci-host-common API
> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
> 4. Enable above driver on ARM64
> 
> Patches has been built on top of 4.6-rc2 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
> 
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.
> 
> v5 -> v6
> - dropped idea of x86 MMCONFIG code refactoring
> - integrated JC's patches which introduce new ECAM API:
>   https://lkml.org/lkml/2016/4/11/907
>   git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
> - added ACPI support for ThunderX ECAM and PEM drivers
> - rebased to 4.6-rc2

JC: can you explicitly confirm that you're ok with letting Tomasz drive
this? We would like to see one driver. Either that is Tomasz, or
Lorenzo, or it is you. But we need to have one overall cooordinated
effort to get this enablement into upstream as quickly as possible.

Some of the Enterprise folks are going to otherwise end up in a very
nasty situation of supporting the previous non-upstream patches for many
years, which is absolutely something we want to avoid...

Jon.

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Arnd Bergmann]

On Friday 15 April 2016 19:06:43 Tomasz Nowicki wrote:
> -MODULE_DEVICE_TABLE(of, thunder_pem_of_match);
> -
> -static int thunder_pem_probe(struct platform_device *pdev)
> +static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
>  {
> -       struct device *dev = &pdev->dev;
> -       const struct of_device_id *of_id;
>         resource_size_t bar4_start;
>         struct resource *res_pem;
>         struct thunder_pem_pci *pem_pci;
> +       struct platform_device *pdev;
>  
>         pem_pci = devm_kzalloc(dev, sizeof(*pem_pci), GFP_KERNEL);
>         if (!pem_pci)
>                 return -ENOMEM;
>  
> -       of_id = of_match_node(thunder_pem_of_match, dev->of_node);
> -       pem_pci->gen_pci.cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
> +       pdev = to_platform_device(dev);
>  
>         /*
>          * The second register range is the PEM bridge to the PCIe
> @@ -330,7 +298,29 @@ static int thunder_pem_probe(struct platform_device *pdev)
>         pem_pci->ea_entry[1] = (u32)(res_pem->end - bar4_start) & ~3u;
>         pem_pci->ea_entry[2] = (u32)(bar4_start >> 32);
>  
> -       return pci_host_common_probe(pdev, &pem_pci->gen_pci);
> +       cfg->priv = pem_pci;
> +       return 0;
> +}
> +
> 

I still think it would be better to keep the loadable PCI host drivers
separate from the ACPI PCI infrastructure. There are a number of
simplifications that we want to do to the DT based drivers in the long
run, so it's better if that code is not shared at this level. Abstracting
out the ECAM code is fine, but at that point you should be able to just
call it from the ACPI layer.

	Arnd

Re: [PATCH V6 07/13] PCI: Provide common functions for ECAM mapping

[Arnd Bergmann]

On Friday 15 April 2016 19:06:42 Tomasz Nowicki wrote:
> diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
> new file mode 100644
> index 0000000..34c0aba
> --- /dev/null
> +++ b/drivers/pci/ecam.h
> 

You are including this file from device drivers and potentially from ACPI 
code, so I think this needs to go into include/linux/pci*.h

	Arnd

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2847 bytes --]

Hi Tomasz,

[auto build test ERROR on pci/next]
[also build test ERROR on v4.6-rc3 next-20160415]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Tomasz-Nowicki/Support-for-generic-ACPI-based-PCI-host-controller/20160416-011935
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: ia64-allmodconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=ia64 

All errors (new ones prefixed by >>):

   drivers/pci/hotplug/sgi_hotplug.c: In function 'enable_slot':
>> drivers/pci/hotplug/sgi_hotplug.c:412:61: error: 'struct pci_controller' has no member named 'companion'
      phandle = acpi_device_handle(PCI_CONTROLLER(slot->pci_bus)->companion);
                                                                ^
   drivers/pci/hotplug/sgi_hotplug.c: In function 'disable_slot':
   drivers/pci/hotplug/sgi_hotplug.c:491:32: error: 'struct pci_controller' has no member named 'companion'
      PCI_CONTROLLER(slot->pci_bus)->companion) {
                                   ^
   drivers/pci/hotplug/sgi_hotplug.c:500:61: error: 'struct pci_controller' has no member named 'companion'
      phandle = acpi_device_handle(PCI_CONTROLLER(slot->pci_bus)->companion);
                                                                ^

vim +412 drivers/pci/hotplug/sgi_hotplug.c

3e643e77 John Keller       2007-01-30  406  		struct acpi_device *pdevice;
3e643e77 John Keller       2007-01-30  407  		acpi_handle phandle;
3e643e77 John Keller       2007-01-30  408  		acpi_handle chandle = NULL;
3e643e77 John Keller       2007-01-30  409  		acpi_handle rethandle;
3e643e77 John Keller       2007-01-30  410  		acpi_status ret;
3e643e77 John Keller       2007-01-30  411  
7b199811 Rafael J. Wysocki 2013-11-11 @412  		phandle = acpi_device_handle(PCI_CONTROLLER(slot->pci_bus)->companion);
3e643e77 John Keller       2007-01-30  413  
3e643e77 John Keller       2007-01-30  414  		if (acpi_bus_get_device(phandle, &pdevice)) {
227f0647 Ryan Desfosses    2014-04-18  415  			dev_dbg(&slot->pci_bus->self->dev, "no parent device, assuming NULL\n");

:::::: The code at line 412 was first introduced by commit
:::::: 7b1998116bbb2f3e5dd6cb9a8ee6db479b0b50a9 ACPI / driver core: Store an ACPI device pointer in struct acpi_dev_node

:::::: TO: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
:::::: CC: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 43194 bytes --]

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Jayachandran C]

On Sat, Apr 16, 2016 at 12:09 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Friday 15 April 2016 19:06:43 Tomasz Nowicki wrote:
>> -MODULE_DEVICE_TABLE(of, thunder_pem_of_match);
>> -
>> -static int thunder_pem_probe(struct platform_device *pdev)
>> +static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
>>  {
>> -       struct device *dev = &pdev->dev;
>> -       const struct of_device_id *of_id;
>>         resource_size_t bar4_start;
>>         struct resource *res_pem;
>>         struct thunder_pem_pci *pem_pci;
>> +       struct platform_device *pdev;
>>
>>         pem_pci = devm_kzalloc(dev, sizeof(*pem_pci), GFP_KERNEL);
>>         if (!pem_pci)
>>                 return -ENOMEM;
>>
>> -       of_id = of_match_node(thunder_pem_of_match, dev->of_node);
>> -       pem_pci->gen_pci.cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
>> +       pdev = to_platform_device(dev);
>>
>>         /*
>>          * The second register range is the PEM bridge to the PCIe
>> @@ -330,7 +298,29 @@ static int thunder_pem_probe(struct platform_device *pdev)
>>         pem_pci->ea_entry[1] = (u32)(res_pem->end - bar4_start) & ~3u;
>>         pem_pci->ea_entry[2] = (u32)(bar4_start >> 32);
>>
>> -       return pci_host_common_probe(pdev, &pem_pci->gen_pci);
>> +       cfg->priv = pem_pci;
>> +       return 0;
>> +}
>> +
>>
>
> I still think it would be better to keep the loadable PCI host drivers
> separate from the ACPI PCI infrastructure. There are a number of
> simplifications that we want to do to the DT based drivers in the long
> run, so it's better if that code is not shared at this level. Abstracting
> out the ECAM code is fine, but at that point you should be able to just
> call it from the ACPI layer.

The issue is not with this patch (in my opinion). This patch is just
re-arranging how thunder specific data is maintained. Earlier it was
a container_of gen_pci, now it is ->priv of pci_config_window.

I can see the issue in patches 12 and 13 of this patchset which adds
ACPI fixups into the thunder OF driver.

The simple approach when doing modular PCI drivers would be to make
pci-thunder-*.c like pci-host-common.c, to be compiled in if configured.
The fie will contain all the Thunder quirks and can export
pci_thunder_ecam_ops.

Then the OF driver part will be trivial and can be merged into
pci-host-generic.c which can be a module. The ACPI hooks can be
moved to the ACPI PCI host driver file.

Would appreciate any suggestions on the way forward.

Thanks,
JC.

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Arnd Bergmann]

On Saturday 16 April 2016 12:50:13 Jayachandran C wrote:
> >
> > I still think it would be better to keep the loadable PCI host drivers
> > separate from the ACPI PCI infrastructure. There are a number of
> > simplifications that we want to do to the DT based drivers in the long
> > run, so it's better if that code is not shared at this level. Abstracting
> > out the ECAM code is fine, but at that point you should be able to just
> > call it from the ACPI layer.
> 
> The issue is not with this patch (in my opinion). This patch is just
> re-arranging how thunder specific data is maintained. Earlier it was
> a container_of gen_pci, now it is ->priv of pci_config_window.
> 
> I can see the issue in patches 12 and 13 of this patchset which adds
> ACPI fixups into the thunder OF driver.

Right, I commented on this one, because it seems to rearrange the code
in order to do the later one.

> The simple approach when doing modular PCI drivers would be to make
> pci-thunder-*.c like pci-host-common.c, to be compiled in if configured.
> The fie will contain all the Thunder quirks and can export
> pci_thunder_ecam_ops.

I would argue that we should not export anything from drivers/pci/host,
those should really be standalone drivers that do not interact with other
subsystems.

How much code would you need to duplicate from thunder-ecam to have
the same functionality available in ACPI? My expectation is that it's
not really that much more compared to the code you need for sharing
a single implementation, but you get a lower complexity here, which
makes it easier to understand and to rework.

	Arnd

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Jayachandran C]

On Sat, Apr 16, 2016 at 1:01 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Saturday 16 April 2016 12:50:13 Jayachandran C wrote:
>> >
>> > I still think it would be better to keep the loadable PCI host drivers
>> > separate from the ACPI PCI infrastructure. There are a number of
>> > simplifications that we want to do to the DT based drivers in the long
>> > run, so it's better if that code is not shared at this level. Abstracting
>> > out the ECAM code is fine, but at that point you should be able to just
>> > call it from the ACPI layer.
>>
>> The issue is not with this patch (in my opinion). This patch is just
>> re-arranging how thunder specific data is maintained. Earlier it was
>> a container_of gen_pci, now it is ->priv of pci_config_window.
>>
>> I can see the issue in patches 12 and 13 of this patchset which adds
>> ACPI fixups into the thunder OF driver.
>
> Right, I commented on this one, because it seems to rearrange the code
> in order to do the later one.

Patches 11- 13 are not from me, and I am not completely on board
on the approach of adding the sections.We can look at reworking this.

>> The simple approach when doing modular PCI drivers would be to make
>> pci-thunder-*.c like pci-host-common.c, to be compiled in if configured.
>> The fie will contain all the Thunder quirks and can export
>> pci_thunder_ecam_ops.
>
> I would argue that we should not export anything from drivers/pci/host,
> those should really be standalone drivers that do not interact with other
> subsystems.

pci-host-common.c goes against being standalone. The files calling
pci_host_common_probe() are expected to have custom ECAM ops
the way it is written now. We need to have a reasonable way to share
those ECAM ops if needed by ACPI.

> How much code would you need to duplicate from thunder-ecam to have
> the same functionality available in ACPI? My expectation is that it's
> not really that much more compared to the code you need for sharing
> a single implementation, but you get a lower complexity here, which
> makes it easier to understand and to rework.

Like I wrote above, the sharing is really simple because both generic
ACPI and pci-host-common.c have been written for "ECAM with quirks".

The whole pci-thunder-*.c is to support thunder PCI quirks since the
generic OF is handled by pci-host-common.c and generic ECAM is now
separated -  duplicating the whole file for ACPI will be bad.

Any suggestions on how to do this better would be really welcome.

Thanks,
JC.

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Jayachandran C]

On Fri, Apr 15, 2016 at 11:49 PM, Jon Masters <jcm@redhat.com> wrote:
> On 04/15/2016 01:06 PM, Tomasz Nowicki wrote:
>> From the functionality point of view this series might be split into the
>> following logic parts:
>> 1. Necessary fixes as the preparation for using driver on ARM64.
>> 2. New ECAM API and update for users of the pci-host-common API
>> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
>> 4. Enable above driver on ARM64
>>
>> Patches has been built on top of 4.6-rc2 and can be found here:
>> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
>>
>> This has been tested on Cavium ThunderX server. Any help in reviewing and
>> testing is very appreciated.
>>
>> v5 -> v6
>> - dropped idea of x86 MMCONFIG code refactoring
>> - integrated JC's patches which introduce new ECAM API:
>>   https://lkml.org/lkml/2016/4/11/907
>>   git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
>> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
>> - added ACPI support for ThunderX ECAM and PEM drivers
>> - rebased to 4.6-rc2
>
> JC: can you explicitly confirm that you're ok with letting Tomasz drive
> this? We would like to see one driver. Either that is Tomasz, or
> Lorenzo, or it is you. But we need to have one overall cooordinated
> effort to get this enablement into upstream as quickly as possible.

I have been concentrating on the ECAM code and ECAM based ACPI
host controller, the rest of the code is from Tomasz original patchset.

I am not happy with the way the ACPI quirk handling is done in Tomasz's
current patchset. I believe that it has to be done in a separate patchset
with another set of discussions. It introduces additional complexity and
mixing that discussion with the ECAM one will not help in making progress.

I hope things will be clearer when more maintainers chime in. When
there is clarity on if the ECAM split is fine, I think we can look at how
to drive the whole patchset forward, otherwise we are back to square one.

> Some of the Enterprise folks are going to otherwise end up in a very
> nasty situation of supporting the previous non-upstream patches for many
> years, which is absolutely something we want to avoid...

Not having ACPI/PCI support upstream is a huge problem for us too...

JC.

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Duc Dang]

On Fri, Apr 15, 2016 at 10:06 AM, Tomasz Nowicki <tn@semihalf.com> wrote:
> From the functionality point of view this series might be split into the
> following logic parts:
> 1. Necessary fixes as the preparation for using driver on ARM64.
> 2. New ECAM API and update for users of the pci-host-common API
> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
> 4. Enable above driver on ARM64
>
> Patches has been built on top of 4.6-rc2 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
>
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.

Hi Tomasz,

I changed X-Gene ECAM fixup code to follow the new ECAM APIs and my
X-Gene PCIe on Mustang board works fine with this series.

Thanks and regards,
Duc Dang.
>
> v5 -> v6
> - dropped idea of x86 MMCONFIG code refactoring
> - integrated JC's patches which introduce new ECAM API:
>   https://lkml.org/lkml/2016/4/11/907
>   git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
> - added ACPI support for ThunderX ECAM and PEM drivers
> - rebased to 4.6-rc2
>
> v4 -> v5
> - dropped MCFG refactoring group patches 1-6 from series v4 and integrated Jayachandran's patch
>   https://patchwork.ozlabs.org/patch/575525/
> - rewrite PCI legacy IRQs allocation
> - squashed two patches 11 and 12 from series v4, fixed bisection issue
> - changelog improvements
> - rebased to 4.5-rc3
>
> v3 -> v4
> - dropped Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> - added Lorenzo's fix patch 19/24
> - ACPI PCI bus domain number assigning cleanup
> - changed resource management, we now claim and reassign resources
> - improvements for applying quirks
> - dropped Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
> - rebased to 4.5-rc1
>
> v2 -> v3
> - fix legacy IRQ assigning and IO ports registration
> - remove reference to arch specific companion device for ia64
> - move ACPI PCI host controller driver to pci_root.c
> - drop generic domain assignment for x86 and ia64 as I am not
>   able to run all necessary test variants
> - drop patch which cleaned legacy IRQ assignment since it belongs to
>   Mathew's series:
>   https://patchwork.ozlabs.org/patch/557504/
> - extend MCFG quirk code
> - rebased to 4.4
>
> v1 -> v2
> - moved non-arch specific piece of code to dirver/acpi/ directory
> - fixed IO resource handling
> - introduced PCI config accessors quirks matching
> - moved ACPI_COMPANION_SET to generic code
>
> v1 - https://lkml.org/lkml/2015/10/27/504
> v2 - https://lkml.org/lkml/2015/12/16/246
> v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
> v4 - https://lkml.org/lkml/2016/2/4/646
> v5 - https://lkml.org/lkml/2016/2/16/426
>
> Jayachandran C (2):
>   PCI: Provide common functions for ECAM mapping
>   PCI: generic, thunder: update to use generic ECAM API
>
> Tomasz Nowicki (11):
>   pci, acpi, x86, ia64: Move ACPI host bridge device companion
>     assignment to core code.
>   pci, acpi: Provide generic way to assign bus domain number.
>   x86, ia64: Include acpi_pci_{add|remove}_bus to the default
>     pcibios_{add|remove}_bus implementation.
>   pci, of: Move the PCI I/O space management to PCI core code.
>   acpi, pci: Support IO resources when parsing PCI host bridge
>     resources.
>   arm64, pci, acpi: ACPI support for legacy IRQs parsing and
>     consolidation with DT code.
>   pci, acpi: Support for ACPI based generic PCI host controller
>   arm64, pci, acpi: Start using ACPI based PCI host controller driver
>     for ARM64.
>   pci, acpi: Match PCI config space accessors against platfrom specific
>     quirks.
>   pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.
>   pci, pci-thunder-pem: Add ACPI support for ThunderX PEM.
>
>  arch/arm64/Kconfig                  |  15 +++
>  arch/arm64/include/asm/cpufeature.h |   3 +-
>  arch/arm64/kernel/cpu_errata.c      |   8 ++
>  arch/arm64/kernel/pci.c             |  35 ++---
>  arch/ia64/hp/common/sba_iommu.c     |   2 +-
>  arch/ia64/include/asm/pci.h         |   1 -
>  arch/ia64/pci/pci.c                 |  26 ----
>  arch/ia64/sn/kernel/io_acpi_init.c  |   4 +-
>  arch/x86/include/asm/pci.h          |   3 -
>  arch/x86/pci/acpi.c                 |  17 ---
>  arch/x86/pci/common.c               |  10 --
>  drivers/acpi/Kconfig                |   8 ++
>  drivers/acpi/Makefile               |   1 +
>  drivers/acpi/bus.c                  |   1 +
>  drivers/acpi/pci_gen_host.c         | 259 ++++++++++++++++++++++++++++++++++++
>  drivers/acpi/pci_root.c             |  58 +++++++-
>  drivers/of/address.c                | 116 +---------------
>  drivers/pci/Kconfig                 |   3 +
>  drivers/pci/Makefile                |   2 +
>  drivers/pci/ecam.c                  | 137 +++++++++++++++++++
>  drivers/pci/ecam.h                  |  66 +++++++++
>  drivers/pci/host/Kconfig            |   1 +
>  drivers/pci/host/pci-host-common.c  | 119 ++++++++---------
>  drivers/pci/host/pci-host-common.h  |  47 -------
>  drivers/pci/host/pci-host-generic.c |  52 ++------
>  drivers/pci/host/pci-thunder-ecam.c |  70 ++++++----
>  drivers/pci/host/pci-thunder-pem.c  | 215 ++++++++++++++++++++++--------
>  drivers/pci/pci.c                   | 150 ++++++++++++++++++++-
>  drivers/pci/probe.c                 |   5 +
>  include/asm-generic/vmlinux.lds.h   |   7 +
>  include/linux/of_address.h          |   9 --
>  include/linux/pci-acpi.h            |  20 +++
>  include/linux/pci.h                 |  12 ++
>  33 files changed, 1029 insertions(+), 453 deletions(-)
>  create mode 100644 drivers/acpi/pci_gen_host.c
>  create mode 100644 drivers/pci/ecam.c
>  create mode 100644 drivers/pci/ecam.h
>  delete mode 100644 drivers/pci/host/pci-host-common.h
>
> --
> 1.9.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Sinan Kaya]

On 4/15/2016 1:06 PM, Tomasz Nowicki wrote:
>>From the functionality point of view this series might be split into the
> following logic parts:
> 1. Necessary fixes as the preparation for using driver on ARM64.
> 2. New ECAM API and update for users of the pci-host-common API
> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
> 4. Enable above driver on ARM64
> 
> Patches has been built on top of 4.6-rc2 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
> 
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.
> 
> v5 -> v6
> - dropped idea of x86 MMCONFIG code refactoring
> - integrated JC's patches which introduce new ECAM API:
>   https://lkml.org/lkml/2016/4/11/907
>   git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
> - added ACPI support for ThunderX ECAM and PEM drivers
> - rebased to 4.6-rc2
> 
> v4 -> v5
> - dropped MCFG refactoring group patches 1-6 from series v4 and integrated Jayachandran's patch
>   https://patchwork.ozlabs.org/patch/575525/
> - rewrite PCI legacy IRQs allocation
> - squashed two patches 11 and 12 from series v4, fixed bisection issue
> - changelog improvements
> - rebased to 4.5-rc3
> 
> v3 -> v4
> - dropped Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> - added Lorenzo's fix patch 19/24
> - ACPI PCI bus domain number assigning cleanup
> - changed resource management, we now claim and reassign resources
> - improvements for applying quirks
> - dropped Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
> - rebased to 4.5-rc1
> 
> v2 -> v3
> - fix legacy IRQ assigning and IO ports registration
> - remove reference to arch specific companion device for ia64
> - move ACPI PCI host controller driver to pci_root.c
> - drop generic domain assignment for x86 and ia64 as I am not
>   able to run all necessary test variants
> - drop patch which cleaned legacy IRQ assignment since it belongs to
>   Mathew's series:
>   https://patchwork.ozlabs.org/patch/557504/
> - extend MCFG quirk code
> - rebased to 4.4
> 
> v1 -> v2
> - moved non-arch specific piece of code to dirver/acpi/ directory
> - fixed IO resource handling
> - introduced PCI config accessors quirks matching
> - moved ACPI_COMPANION_SET to generic code
> 
> v1 - https://lkml.org/lkml/2015/10/27/504
> v2 - https://lkml.org/lkml/2015/12/16/246
> v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
> v4 - https://lkml.org/lkml/2016/2/4/646
> v5 - https://lkml.org/lkml/2016/2/16/426
> 
> Jayachandran C (2):
>   PCI: Provide common functions for ECAM mapping
>   PCI: generic, thunder: update to use generic ECAM API
> 
> Tomasz Nowicki (11):
>   pci, acpi, x86, ia64: Move ACPI host bridge device companion
>     assignment to core code.
>   pci, acpi: Provide generic way to assign bus domain number.
>   x86, ia64: Include acpi_pci_{add|remove}_bus to the default
>     pcibios_{add|remove}_bus implementation.
>   pci, of: Move the PCI I/O space management to PCI core code.
>   acpi, pci: Support IO resources when parsing PCI host bridge
>     resources.
>   arm64, pci, acpi: ACPI support for legacy IRQs parsing and
>     consolidation with DT code.
>   pci, acpi: Support for ACPI based generic PCI host controller
>   arm64, pci, acpi: Start using ACPI based PCI host controller driver
>     for ARM64.
>   pci, acpi: Match PCI config space accessors against platfrom specific
>     quirks.
>   pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.
>   pci, pci-thunder-pem: Add ACPI support for ThunderX PEM.
> 
>  arch/arm64/Kconfig                  |  15 +++
>  arch/arm64/include/asm/cpufeature.h |   3 +-
>  arch/arm64/kernel/cpu_errata.c      |   8 ++
>  arch/arm64/kernel/pci.c             |  35 ++---
>  arch/ia64/hp/common/sba_iommu.c     |   2 +-
>  arch/ia64/include/asm/pci.h         |   1 -
>  arch/ia64/pci/pci.c                 |  26 ----
>  arch/ia64/sn/kernel/io_acpi_init.c  |   4 +-
>  arch/x86/include/asm/pci.h          |   3 -
>  arch/x86/pci/acpi.c                 |  17 ---
>  arch/x86/pci/common.c               |  10 --
>  drivers/acpi/Kconfig                |   8 ++
>  drivers/acpi/Makefile               |   1 +
>  drivers/acpi/bus.c                  |   1 +
>  drivers/acpi/pci_gen_host.c         | 259 ++++++++++++++++++++++++++++++++++++
>  drivers/acpi/pci_root.c             |  58 +++++++-
>  drivers/of/address.c                | 116 +---------------
>  drivers/pci/Kconfig                 |   3 +
>  drivers/pci/Makefile                |   2 +
>  drivers/pci/ecam.c                  | 137 +++++++++++++++++++
>  drivers/pci/ecam.h                  |  66 +++++++++
>  drivers/pci/host/Kconfig            |   1 +
>  drivers/pci/host/pci-host-common.c  | 119 ++++++++---------
>  drivers/pci/host/pci-host-common.h  |  47 -------
>  drivers/pci/host/pci-host-generic.c |  52 ++------
>  drivers/pci/host/pci-thunder-ecam.c |  70 ++++++----
>  drivers/pci/host/pci-thunder-pem.c  | 215 ++++++++++++++++++++++--------
>  drivers/pci/pci.c                   | 150 ++++++++++++++++++++-
>  drivers/pci/probe.c                 |   5 +
>  include/asm-generic/vmlinux.lds.h   |   7 +
>  include/linux/of_address.h          |   9 --
>  include/linux/pci-acpi.h            |  20 +++
>  include/linux/pci.h                 |  12 ++
>  33 files changed, 1029 insertions(+), 453 deletions(-)
>  create mode 100644 drivers/acpi/pci_gen_host.c
>  create mode 100644 drivers/pci/ecam.c
>  create mode 100644 drivers/pci/ecam.h
>  delete mode 100644 drivers/pci/host/pci-host-common.h
> 

+tested by for the v6.

Tested-by: Sinan Kaya <okaya@codeaurora.org>

using the Qualcomm Technologies QDF2XXX server.


-- 
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Martinez Kristofer]

On Sat, Apr 16, 2016 at 2:19 AM, Jon Masters <jcm@redhat.com> wrote:
> On 04/15/2016 01:06 PM, Tomasz Nowicki wrote:
>> From the functionality point of view this series might be split into the
>> following logic parts:
>> 1. Necessary fixes as the preparation for using driver on ARM64.
>> 2. New ECAM API and update for users of the pci-host-common API
>> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
>> 4. Enable above driver on ARM64
>>
>> Patches has been built on top of 4.6-rc2 and can be found here:
>> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
>>
>> This has been tested on Cavium ThunderX server. Any help in reviewing and
>> testing is very appreciated.
>>
>> v5 -> v6
>> - dropped idea of x86 MMCONFIG code refactoring
>> - integrated JC's patches which introduce new ECAM API:
>>   https://lkml.org/lkml/2016/4/11/907
>>   git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
>> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
>> - added ACPI support for ThunderX ECAM and PEM drivers
>> - rebased to 4.6-rc2
>
> JC: can you explicitly confirm that you're ok with letting Tomasz drive
> this? We would like to see one driver. Either that is Tomasz, or
> Lorenzo, or it is you. But we need to have one overall cooordinated
> effort to get this enablement into upstream as quickly as possible.
>
> Some of the Enterprise folks are going to otherwise end up in a very
> nasty situation of supporting the previous non-upstream patches for many
> years, which is absolutely something we want to avoid...
>
Indeed, we need only one driver for this. But given there're already
others except this
one exist, and I believe no one want to give up their efforts. Also I
think it's not exactly the
same feauture provided by those drivers, so can we make some trade-off
to combine
those drivers into one and add the Signed-off-by for all the anthors?
One agreement for all of us is to upstream the ACPI PCI for ARM64 asap!

M.K.

>
> Jon.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V6 11/13] pci, acpi: Match PCI config space accessors against platfrom specific quirks.

[liudongdong (C)]

Hi Tomasz

I merged my patchset to branch topci-acpi-v6.
The patchset is used for Hisilicon DO2 PCIe ACPI support.
I found some compile errors. The log as below.
drivers/pci/host/pcie-hisi-acpi.c: In function 'hisi_pcie_init':
drivers/pci/host/pcie-hisi-acpi.c:130:6: error: 'struct acpi_pci_root' has no member named 'sysdata'
   root->sysdata = reg_base;

In your PATCH V5, add "sysdata" for strcut acpi_pci_root, but PATCH V6 has not add it.
In my patch, I used root->sysdata which will be available along read/write accessor.
I want to know the reason this v6 patchset does not add "sysdata". I need this.

diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index 14362a8..0fc6f13 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -556,6 +556,7 @@ struct acpi_pci_root {
  	struct pci_bus *bus;
  	u16 segment;
  	struct resource secondary;	/* downstream bus range */
+	void *sysdata;

  	u32 osc_support_set;	/* _OSC state of support bits */
  	u32 osc_control_set;	/* _OSC state of control bits */

Thanks
Dongdong

在 2016/4/16 1:06, Tomasz Nowicki 写道:
> Some platforms may not be fully compliant with generic set of PCI config
> accessors. For these cases we implement the way to overwrite accessors
> set prior to PCI buses enumeration. Algorithm traverses available quirk
> list, matches against <DMI ID (optional), domain, bus number> tuple and
> an extra match call and returns corresponding PCI config ops.
> All quirks can be defined using:
> DECLARE_ACPI_MCFG_FIXUP() macro and kept self contained. Example:
>
> /* Additional DMI platform identification (optional) */
> static const struct dmi_system_id foo_dmi[] = {
>          {
>                  .ident = "<Platform ident string>",
>                  .matches = {
>                          DMI_MATCH(DMI_SYS_VENDOR, "<system vendor>"),
>                          DMI_MATCH(DMI_PRODUCT_NAME, "<product name>"),
>                          DMI_MATCH(DMI_PRODUCT_VERSION, "product version"),
>                  },
>          },
>          { }
> };
>
> /* Custom PCI config ops */
> static struct pci_generic_ecam_ops foo_pci_ops = {
> 	.bus_shift	= 24,
> 	.pci_ops = {
> 		.map_bus = pci_mcfg_dev_base,
> 		.read = foo_ecam_config_read,
> 		.write = foo_ecam_config_write,
> 	}
> };
>
> static int foo_match(struct pci_mcfg_fixup *fixup, struct acpi_pci_root *root)
> {
> 	if (additional platform identification)
> 	        return true;
> 	return false;
> }
>
> DECLARE_ACPI_MCFG_FIXUP(foo_dmi, foo_init, &foo_root_ops, <domain_nr>, <bus_nr>);
>
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> ---
>   drivers/acpi/pci_gen_host.c       | 30 +++++++++++++++++++++++++++++-
>   include/asm-generic/vmlinux.lds.h |  7 +++++++
>   include/linux/pci-acpi.h          | 18 ++++++++++++++++++
>   3 files changed, 54 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
> index fd360b5..e55dfca 100644
> --- a/drivers/acpi/pci_gen_host.c
> +++ b/drivers/acpi/pci_gen_host.c
> @@ -11,6 +11,8 @@
>    * You should have received a copy of the GNU General Public License
>    * version 2 (GPLv2) along with this source code.
>    */
> +
> +#include <linux/dmi.h>
>   #include <linux/kernel.h>
>   #include <linux/pci.h>
>   #include <linux/pci-acpi.h>
> @@ -54,6 +56,32 @@ static struct mcfg_entry *pci_mcfg_lookup(u16 seg, u8 bus_start)
>   	return NULL;
>   }
>
> +extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
> +extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
> +
> +static struct pci_generic_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
> +{
> +	int bus_num = root->secondary.start;
> +	int domain = root->segment;
> +	struct pci_cfg_fixup *f;
> +
> +	/*
> +	 * Match against platform specific quirks and return corresponding
> +	 * CAM ops.
> +	 *
> +	 * First match against PCI topology <domain:bus> then use DMI or
> +	 * custom match handler.
> +	 */
> +	for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
> +		if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
> +		    (f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
> +		    (f->system ? dmi_check_system(f->system) : 1) &&
> +		    (f->match ? f->match(f, root) : 1))
> +			return f->ops;
> +	}
> +	/* No quirks, use ECAM */
> +	return &pci_generic_ecam_default_ops;
> +}
>
>   /*
>    * Lookup the bus range for the domain in MCFG, and set up config space
> @@ -95,7 +123,7 @@ static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
>   	}
>
>   	cfg = pci_generic_ecam_create(&root->device->dev, addr, bus_start,
> -				      bus_end, &pci_generic_ecam_default_ops);
> +				      bus_end, pci_acpi_get_ops(root));
>   	if (IS_ERR(cfg)) {
>   		err = PTR_ERR(cfg);
>   		pr_err("%04x:%02x-%02x error %d mapping CAM\n", seg,
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index 339125b..c53b6b7 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -298,6 +298,13 @@
>   		VMLINUX_SYMBOL(__end_pci_fixups_suspend_late) = .;	\
>   	}								\
>   									\
> +	/* ACPI MCFG quirks */						\
> +	.acpi_fixup        : AT(ADDR(.acpi_fixup) - LOAD_OFFSET) {	\
> +		VMLINUX_SYMBOL(__start_acpi_mcfg_fixups) = .;		\
> +		*(.acpi_fixup_mcfg)					\
> +		VMLINUX_SYMBOL(__end_acpi_mcfg_fixups) = .;		\
> +	}								\
> +									\
>   	/* Built-in firmware blobs */					\
>   	.builtin_fw        : AT(ADDR(.builtin_fw) - LOAD_OFFSET) {	\
>   		VMLINUX_SYMBOL(__start_builtin_fw) = .;			\
> diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> index a72e22d..9545988 100644
> --- a/include/linux/pci-acpi.h
> +++ b/include/linux/pci-acpi.h
> @@ -71,6 +71,24 @@ struct acpi_pci_root_ops {
>   	int (*prepare_resources)(struct acpi_pci_root_info *info);
>   };
>
> +struct pci_cfg_fixup {
> +	const struct dmi_system_id *system;
> +	bool (*match)(struct pci_cfg_fixup *, struct acpi_pci_root *);
> +	struct pci_generic_ecam_ops *ops;
> +	int domain;
> +	int bus_num;
> +};
> +
> +#define PCI_MCFG_DOMAIN_ANY	-1
> +#define PCI_MCFG_BUS_ANY	-1
> +
> +/* Designate a routine to fix up buggy MCFG */
> +#define DECLARE_ACPI_MCFG_FIXUP(system, match, ops, dom, bus)		\
> +	static const struct pci_cfg_fixup __mcfg_fixup_##system##dom##bus\
> +	 __used	__attribute__((__section__(".acpi_fixup_mcfg"),		\
> +				aligned((sizeof(void *))))) =		\
> +	{ system, match, ops, dom, bus };
> +
>   extern int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info);
>   extern struct pci_bus *acpi_pci_root_create(struct acpi_pci_root *root,
>   					    struct acpi_pci_root_ops *ops,
>

Re: [PATCH V6 11/13] pci, acpi: Match PCI config space accessors against platfrom specific quirks.

[Tomasz Nowicki]

On 18.04.2016 13:37, liudongdong (C) wrote:
> Hi Tomasz
>
> I merged my patchset to branch topci-acpi-v6.
> The patchset is used for Hisilicon DO2 PCIe ACPI support.
> I found some compile errors. The log as below.
> drivers/pci/host/pcie-hisi-acpi.c: In function 'hisi_pcie_init':
> drivers/pci/host/pcie-hisi-acpi.c:130:6: error: 'struct acpi_pci_root'
> has no member named 'sysdata'
>    root->sysdata = reg_base;
>
> In your PATCH V5, add "sysdata" for strcut acpi_pci_root, but PATCH V6
> has not add it.
> In my patch, I used root->sysdata which will be available along
> read/write accessor.
> I want to know the reason this v6 patchset does not add "sysdata". I
> need this.

We are handling this different way. You can now use "struct 
pci_config_window" -> priv, see pci-thunder-pem.c driver.

Tomasz

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Tomasz Nowicki]

On 16.04.2016 16:36, Jayachandran C wrote:
> On Sat, Apr 16, 2016 at 1:01 PM, Arnd Bergmann <arnd@arndb.de> wrote:
>> On Saturday 16 April 2016 12:50:13 Jayachandran C wrote:
>>>>
>>>> I still think it would be better to keep the loadable PCI host drivers
>>>> separate from the ACPI PCI infrastructure. There are a number of
>>>> simplifications that we want to do to the DT based drivers in the long
>>>> run, so it's better if that code is not shared at this level. Abstracting
>>>> out the ECAM code is fine, but at that point you should be able to just
>>>> call it from the ACPI layer.
>>>
>>> The issue is not with this patch (in my opinion). This patch is just
>>> re-arranging how thunder specific data is maintained. Earlier it was
>>> a container_of gen_pci, now it is ->priv of pci_config_window.
>>>
>>> I can see the issue in patches 12 and 13 of this patchset which adds
>>> ACPI fixups into the thunder OF driver.
>>
>> Right, I commented on this one, because it seems to rearrange the code
>> in order to do the later one.
>
> Patches 11- 13 are not from me, and I am not completely on board
> on the approach of adding the sections.We can look at reworking this.
>
>>> The simple approach when doing modular PCI drivers would be to make
>>> pci-thunder-*.c like pci-host-common.c, to be compiled in if configured.
>>> The fie will contain all the Thunder quirks and can export
>>> pci_thunder_ecam_ops.
>>
>> I would argue that we should not export anything from drivers/pci/host,
>> those should really be standalone drivers that do not interact with other
>> subsystems.
>
> pci-host-common.c goes against being standalone. The files calling
> pci_host_common_probe() are expected to have custom ECAM ops
> the way it is written now. We need to have a reasonable way to share
> those ECAM ops if needed by ACPI.
>
>> How much code would you need to duplicate from thunder-ecam to have
>> the same functionality available in ACPI? My expectation is that it's
>> not really that much more compared to the code you need for sharing
>> a single implementation, but you get a lower complexity here, which
>> makes it easier to understand and to rework.
>
> Like I wrote above, the sharing is really simple because both generic
> ACPI and pci-host-common.c have been written for "ECAM with quirks".
>
> The whole pci-thunder-*.c is to support thunder PCI quirks since the
> generic OF is handled by pci-host-common.c and generic ECAM is now
> separated -  duplicating the whole file for ACPI will be bad.

Yes, it would be too much code duplication. Also, we already know 
drivers which need quirks.

We really need to agree on best approach here. Here are requirements 
which came up (please correct me if misunderstood sth):

Arnd:
1. Initial DT driver should be standalone [Arnd]
2. No exported symbols [Arnd]
3. Duplicate necessary code to ACPI framework.

JC:
1. Adding linker section is wrong.
2. Quirks should be exported (pci_thunder_ecam_ops), then no need for 
adding linker section
3. To much duplication to copy code into the ACPI framework.

My opinion:
1. I like linker section because it is easy to maintain and no need to 
export symbols.
2. We need more sophisticated algorithm for matching quirks (DMI is not 
enough and not only for ThunderX drivers). Of course I am open to any 
new suggestions.
3. To much duplication to copy code into the ACPI framework.

Thanks in advance for any pointers.

Thanks,
Tomasz

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Tomasz Nowicki]

On 16.04.2016 17:31, Jayachandran C wrote:
> On Fri, Apr 15, 2016 at 11:49 PM, Jon Masters <jcm@redhat.com> wrote:
>> On 04/15/2016 01:06 PM, Tomasz Nowicki wrote:
>>>  From the functionality point of view this series might be split into the
>>> following logic parts:
>>> 1. Necessary fixes as the preparation for using driver on ARM64.
>>> 2. New ECAM API and update for users of the pci-host-common API
>>> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
>>> 4. Enable above driver on ARM64
>>>
>>> Patches has been built on top of 4.6-rc2 and can be found here:
>>> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
>>>
>>> This has been tested on Cavium ThunderX server. Any help in reviewing and
>>> testing is very appreciated.
>>>
>>> v5 -> v6
>>> - dropped idea of x86 MMCONFIG code refactoring
>>> - integrated JC's patches which introduce new ECAM API:
>>>    https://lkml.org/lkml/2016/4/11/907
>>>    git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
>>> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
>>> - added ACPI support for ThunderX ECAM and PEM drivers
>>> - rebased to 4.6-rc2
>>
>> JC: can you explicitly confirm that you're ok with letting Tomasz drive
>> this? We would like to see one driver. Either that is Tomasz, or
>> Lorenzo, or it is you. But we need to have one overall cooordinated
>> effort to get this enablement into upstream as quickly as possible.
>
> I have been concentrating on the ECAM code and ECAM based ACPI
> host controller, the rest of the code is from Tomasz original patchset.
>
> I am not happy with the way the ACPI quirk handling is done in Tomasz's
> current patchset. I believe that it has to be done in a separate patchset
> with another set of discussions. It introduces additional complexity and
> mixing that discussion with the ECAM one will not help in making progress.

Of course we can split discussion into the two topics:
1. ECAM based ACPI host controller - patches [1-10]
2. Quirks handling and examples.

IMO, it is very helpful for reviewers to go with one unified patch set 
and see the whole picture. Also, as you can see, quirks handling allows 
people to test it easily with their servers (not only QEMU but real HW).

Thanks,
Tomasz

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Arnd Bergmann]

On Monday 18 April 2016 15:33:24 Tomasz Nowicki wrote:
> 
> Of course we can split discussion into the two topics:
> 1. ECAM based ACPI host controller - patches [1-10]
> 2. Quirks handling and examples.
> 
> IMO, it is very helpful for reviewers to go with one unified patch set 
> and see the whole picture. Also, as you can see, quirks handling allows 
> people to test it easily with their servers (not only QEMU but real HW).

I think splitting the two would help tremendously. The regular
PCI support should just get merged (it should have been completed
years ago when ACPI for ARM64 was first implemented), while the quirks
handling contains all ugly nonstandard hacks we have to be careful
about.

	Arnd

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Arnd Bergmann]

On Monday 18 April 2016 15:03:51 Tomasz Nowicki wrote:
> On 16.04.2016 16:36, Jayachandran C wrote:
> > On Sat, Apr 16, 2016 at 1:01 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> >> On Saturday 16 April 2016 12:50:13 Jayachandran C wrote:
> > The whole pci-thunder-*.c is to support thunder PCI quirks since the
> > generic OF is handled by pci-host-common.c and generic ECAM is now
> > separated -  duplicating the whole file for ACPI will be bad.
> 
> Yes, it would be too much code duplication. Also, we already know 
> drivers which need quirks.
> 
> We really need to agree on best approach here. Here are requirements 
> which came up (please correct me if misunderstood sth):
> 
> Arnd:
> 1. Initial DT driver should be standalone [Arnd]
> 2. No exported symbols [Arnd]
> 3. Duplicate necessary code to ACPI framework.

Correct.

> JC:
> 1. Adding linker section is wrong.
> 2. Quirks should be exported (pci_thunder_ecam_ops), then no need for 
> adding linker section
> 3. To much duplication to copy code into the ACPI framework.
> 
> My opinion:
> 1. I like linker section because it is easy to maintain and no need to 
> export symbols.
> 2. We need more sophisticated algorithm for matching quirks (DMI is not 
> enough and not only for ThunderX drivers). Of course I am open to any 
> new suggestions.

Agreed.

> 3. To much duplication to copy code into the ACPI framework.
> 
> Thanks in advance for any pointers.

Can you be more specific about what code actually would need to
be duplicated? Anything besides the config space operations?

	Arnd

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Tomasz Nowicki]

On 18.04.2016 16:38, Arnd Bergmann wrote:
> On Monday 18 April 2016 15:33:24 Tomasz Nowicki wrote:
>>
>> Of course we can split discussion into the two topics:
>> 1. ECAM based ACPI host controller - patches [1-10]
>> 2. Quirks handling and examples.
>>
>> IMO, it is very helpful for reviewers to go with one unified patch set
>> and see the whole picture. Also, as you can see, quirks handling allows
>> people to test it easily with their servers (not only QEMU but real HW).
>
> I think splitting the two would help tremendously. The regular
> PCI support should just get merged (it should have been completed
> years ago when ACPI for ARM64 was first implemented), while the quirks
> handling contains all ugly nonstandard hacks we have to be careful
> about.
>

OK, so for those who want to review just "ECAM based ACPI host 
controller" lets consider only patches [1-10]. Patches 11-13 are well 
isolated and do not affect previous one. Is that ok for this series?

Thanks,
Tomasz

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Tomasz Nowicki]

On 18.04.2016 16:44, Arnd Bergmann wrote:
> On Monday 18 April 2016 15:03:51 Tomasz Nowicki wrote:
>> On 16.04.2016 16:36, Jayachandran C wrote:
>>> On Sat, Apr 16, 2016 at 1:01 PM, Arnd Bergmann <arnd@arndb.de> wrote:
>>>> On Saturday 16 April 2016 12:50:13 Jayachandran C wrote:
>>> The whole pci-thunder-*.c is to support thunder PCI quirks since the
>>> generic OF is handled by pci-host-common.c and generic ECAM is now
>>> separated -  duplicating the whole file for ACPI will be bad.
>>
>> Yes, it would be too much code duplication. Also, we already know
>> drivers which need quirks.
>>
>> We really need to agree on best approach here. Here are requirements
>> which came up (please correct me if misunderstood sth):
>>
>> Arnd:
>> 1. Initial DT driver should be standalone [Arnd]
>> 2. No exported symbols [Arnd]
>> 3. Duplicate necessary code to ACPI framework.
>
> Correct.
>
>> JC:
>> 1. Adding linker section is wrong.
>> 2. Quirks should be exported (pci_thunder_ecam_ops), then no need for
>> adding linker section
>> 3. To much duplication to copy code into the ACPI framework.
>>
>> My opinion:
>> 1. I like linker section because it is easy to maintain and no need to
>> export symbols.
>> 2. We need more sophisticated algorithm for matching quirks (DMI is not
>> enough and not only for ThunderX drivers). Of course I am open to any
>> new suggestions.
>
> Agreed.
>
>> 3. To much duplication to copy code into the ACPI framework.
>>
>> Thanks in advance for any pointers.
>
> Can you be more specific about what code actually would need to
> be duplicated? Anything besides the config space operations?
>

Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.

pci-thunder-ecam.c contains config space accessors. Similar for 
pci-thunder-pem.c but it also has extra init call (it is now called 
thunder_pem_init) which finds and maps related registers.

Thanks,
Tomasz

Re: [PATCH V6 12/13] pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.

[Tomasz Nowicki]

On 15.04.2016 19:06, Tomasz Nowicki wrote:
> Passes 1.x miss PCI enhanced allocation (EA) header for fixed-BARs,
> thus these passes should use Cavium-specific config access functions
> that synthesize the missing EA capabilities.
>
> We already have DT driver which addresses errata requirements and
> allows to use special PCI config accessors. Currently this driver uses
> compatible = "cavium,pci-host-thunder-ecam" to mach against the errata.
> For ACPI case we need explicit errata number and corresponding
> DECLARE_ACPI_MCFG_FIXUP fixup code.
>
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> ---
>   arch/arm64/Kconfig                  | 14 ++++++++++++++
>   arch/arm64/include/asm/cpufeature.h |  3 ++-
>   arch/arm64/kernel/cpu_errata.c      |  8 ++++++++
>   drivers/pci/host/pci-thunder-ecam.c | 35 +++++++++++++++++++++++++++++++++++
>   4 files changed, 59 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 1bded87..b7614b8 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -445,6 +445,20 @@ config CAVIUM_ERRATUM_27456
>
>   	  If unsure, say Y.
>
> +config CAVIUM_ERRATUM_24575
> +	bool "Cavium erratum 24575: Enhanced Allocation (EA) emualaion"
> +	default y
> +	help
> +	  Enable workaround for erratum 24575.
> +
> +	  Early versions of the Cavium Thunder CN88XX processor are missing
> +	  Enhanced Allocation (EA) capabilities for the fixed BAR addresses used
> +	  by the on-SoC hardware blocks. The erratum adds config access
> +	  functions that synthesize the missing EA capabilities for versions
> +	  that are missing that information.
> +
> +	  If unsure, say Y.
> +
>   endmenu
>
>
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index b9b6494..a78364e 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -35,8 +35,9 @@
>   #define ARM64_ALT_PAN_NOT_UAO			10
>   #define ARM64_HAS_VIRT_HOST_EXTN		11
>   #define ARM64_WORKAROUND_CAVIUM_27456		12
> +#define ARM64_WORKAROUND_CAVIUM_24575		13
>
> -#define ARM64_NCAPS				13
> +#define ARM64_NCAPS				14
>
>   #ifndef __ASSEMBLY__
>
> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> index 06afd04..89c13d7 100644
> --- a/arch/arm64/kernel/cpu_errata.c
> +++ b/arch/arm64/kernel/cpu_errata.c
> @@ -97,6 +97,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
>   			   (1 << MIDR_VARIANT_SHIFT) | 1),
>   	},
>   #endif
> +#ifdef CONFIG_CAVIUM_ERRATUM_24575
> +	{
> +	/* Cavium ThunderX, pass 1.x */
> +		.desc = "Cavium erratum 24575",
> +		.capability = ARM64_WORKAROUND_CAVIUM_24575,
> +		MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
> +	},
> +#endif
>   	{
>   	}
>   };
> diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
> index f67c6d7..01de697 100644
> --- a/drivers/pci/host/pci-thunder-ecam.c
> +++ b/drivers/pci/host/pci-thunder-ecam.c
> @@ -11,10 +11,14 @@
>   #include <linux/ioport.h>
>   #include <linux/of_pci.h>
>   #include <linux/of.h>
> +#include <linux/pci-acpi.h>
>   #include <linux/platform_device.h>
>
> +#include <asm/virt.h>
> +
>   #include "../ecam.h"
>
> +
>   static void set_val(u32 v, int where, int size, u32 *val)
>   {
>   	int shift = (where & 3) * 8;
> @@ -376,5 +380,36 @@ static struct platform_driver thunder_ecam_driver = {
>   };
>   module_platform_driver(thunder_ecam_driver);
>
> +#ifdef CONFIG_ACPI
> +
> +static bool needs_cavium_erratum_24575(struct pci_cfg_fixup *fixup,
> +				      struct acpi_pci_root *root)
> +{
> +	/*
> +	 * We must match errata code and be hypervisor, quirk does not apply
> +	 * for virtual machines.
> +	 */
> +	return cpus_have_cap(ARM64_WORKAROUND_CAVIUM_24575) &&
> +	       is_hyp_mode_available();
> +}
> +
> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
> +			0, PCI_MCFG_BUS_ANY);
> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
> +			1, PCI_MCFG_BUS_ANY);
> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
> +			2, PCI_MCFG_BUS_ANY);
> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
> +			3, PCI_MCFG_BUS_ANY);
> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
> +			10, PCI_MCFG_BUS_ANY);
> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
> +			11, PCI_MCFG_BUS_ANY);
> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
> +			12, PCI_MCFG_BUS_ANY);
> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575, &pci_thunder_ecam_ops,
> +			13, PCI_MCFG_BUS_ANY);
> +#endif
> +

I wonder if we can identify quirk based on _DSD properties, like that:

Name (_DSD, Package () {
	ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
	Package () {
		Package (2) {"cavium,pci-host-thunder-ecam", 1},
	}
})

similar for PEM driver:

Name (_DSD, Package () {
	ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
	Package () {
		Package (2) {"cavium,pci-host-thunder-pem", 1},
	}
})

Do you think it is right thing to do?

Tomasz

Re: [Linaro-acpi] [PATCH V6 12/13] pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.

[G Gregory]

On 19 April 2016 at 11:26, Tomasz Nowicki <tn@semihalf.com> wrote:
> On 15.04.2016 19:06, Tomasz Nowicki wrote:
>>
>> Passes 1.x miss PCI enhanced allocation (EA) header for fixed-BARs,
>> thus these passes should use Cavium-specific config access functions
>> that synthesize the missing EA capabilities.
>>
>> We already have DT driver which addresses errata requirements and
>> allows to use special PCI config accessors. Currently this driver uses
>> compatible = "cavium,pci-host-thunder-ecam" to mach against the errata.
>> For ACPI case we need explicit errata number and corresponding
>> DECLARE_ACPI_MCFG_FIXUP fixup code.
>>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> ---
>>   arch/arm64/Kconfig                  | 14 ++++++++++++++
>>   arch/arm64/include/asm/cpufeature.h |  3 ++-
>>   arch/arm64/kernel/cpu_errata.c      |  8 ++++++++
>>   drivers/pci/host/pci-thunder-ecam.c | 35
>> +++++++++++++++++++++++++++++++++++
>>   4 files changed, 59 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index 1bded87..b7614b8 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -445,6 +445,20 @@ config CAVIUM_ERRATUM_27456
>>
>>           If unsure, say Y.
>>
>> +config CAVIUM_ERRATUM_24575
>> +       bool "Cavium erratum 24575: Enhanced Allocation (EA) emualaion"
>> +       default y
>> +       help
>> +         Enable workaround for erratum 24575.
>> +
>> +         Early versions of the Cavium Thunder CN88XX processor are
>> missing
>> +         Enhanced Allocation (EA) capabilities for the fixed BAR
>> addresses used
>> +         by the on-SoC hardware blocks. The erratum adds config access
>> +         functions that synthesize the missing EA capabilities for
>> versions
>> +         that are missing that information.
>> +
>> +         If unsure, say Y.
>> +
>>   endmenu
>>
>>
>> diff --git a/arch/arm64/include/asm/cpufeature.h
>> b/arch/arm64/include/asm/cpufeature.h
>> index b9b6494..a78364e 100644
>> --- a/arch/arm64/include/asm/cpufeature.h
>> +++ b/arch/arm64/include/asm/cpufeature.h
>> @@ -35,8 +35,9 @@
>>   #define ARM64_ALT_PAN_NOT_UAO                 10
>>   #define ARM64_HAS_VIRT_HOST_EXTN              11
>>   #define ARM64_WORKAROUND_CAVIUM_27456         12
>> +#define ARM64_WORKAROUND_CAVIUM_24575          13
>>
>> -#define ARM64_NCAPS                            13
>> +#define ARM64_NCAPS                            14
>>
>>   #ifndef __ASSEMBLY__
>>
>> diff --git a/arch/arm64/kernel/cpu_errata.c
>> b/arch/arm64/kernel/cpu_errata.c
>> index 06afd04..89c13d7 100644
>> --- a/arch/arm64/kernel/cpu_errata.c
>> +++ b/arch/arm64/kernel/cpu_errata.c
>> @@ -97,6 +97,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
>>                            (1 << MIDR_VARIANT_SHIFT) | 1),
>>         },
>>   #endif
>> +#ifdef CONFIG_CAVIUM_ERRATUM_24575
>> +       {
>> +       /* Cavium ThunderX, pass 1.x */
>> +               .desc = "Cavium erratum 24575",
>> +               .capability = ARM64_WORKAROUND_CAVIUM_24575,
>> +               MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
>> +       },
>> +#endif
>>         {
>>         }
>>   };
>> diff --git a/drivers/pci/host/pci-thunder-ecam.c
>> b/drivers/pci/host/pci-thunder-ecam.c
>> index f67c6d7..01de697 100644
>> --- a/drivers/pci/host/pci-thunder-ecam.c
>> +++ b/drivers/pci/host/pci-thunder-ecam.c
>> @@ -11,10 +11,14 @@
>>   #include <linux/ioport.h>
>>   #include <linux/of_pci.h>
>>   #include <linux/of.h>
>> +#include <linux/pci-acpi.h>
>>   #include <linux/platform_device.h>
>>
>> +#include <asm/virt.h>
>> +
>>   #include "../ecam.h"
>>
>> +
>>   static void set_val(u32 v, int where, int size, u32 *val)
>>   {
>>         int shift = (where & 3) * 8;
>> @@ -376,5 +380,36 @@ static struct platform_driver thunder_ecam_driver = {
>>   };
>>   module_platform_driver(thunder_ecam_driver);
>>
>> +#ifdef CONFIG_ACPI
>> +
>> +static bool needs_cavium_erratum_24575(struct pci_cfg_fixup *fixup,
>> +                                     struct acpi_pci_root *root)
>> +{
>> +       /*
>> +        * We must match errata code and be hypervisor, quirk does not
>> apply
>> +        * for virtual machines.
>> +        */
>> +       return cpus_have_cap(ARM64_WORKAROUND_CAVIUM_24575) &&
>> +              is_hyp_mode_available();
>> +}
>> +
>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>> &pci_thunder_ecam_ops,
>> +                       0, PCI_MCFG_BUS_ANY);
>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>> &pci_thunder_ecam_ops,
>> +                       1, PCI_MCFG_BUS_ANY);
>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>> &pci_thunder_ecam_ops,
>> +                       2, PCI_MCFG_BUS_ANY);
>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>> &pci_thunder_ecam_ops,
>> +                       3, PCI_MCFG_BUS_ANY);
>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>> &pci_thunder_ecam_ops,
>> +                       10, PCI_MCFG_BUS_ANY);
>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>> &pci_thunder_ecam_ops,
>> +                       11, PCI_MCFG_BUS_ANY);
>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>> &pci_thunder_ecam_ops,
>> +                       12, PCI_MCFG_BUS_ANY);
>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>> &pci_thunder_ecam_ops,
>> +                       13, PCI_MCFG_BUS_ANY);
>> +#endif
>> +
>
>
> I wonder if we can identify quirk based on _DSD properties, like that:
>
> Name (_DSD, Package () {
>         ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
>         Package () {
>                 Package (2) {"cavium,pci-host-thunder-ecam", 1},
>         }
> })
>
> similar for PEM driver:
>
> Name (_DSD, Package () {
>         ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
>         Package () {
>                 Package (2) {"cavium,pci-host-thunder-pem", 1},
>         }
> })
>
> Do you think it is right thing to do?
>

Well if your adding new properties why not add a Cavium specific _CID?

Graeme

Re: [Linaro-acpi] [PATCH V6 12/13] pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.

[Graeme Gregory]

On Tue, Apr 19, 2016 at 11:41:29AM +0100, G Gregory wrote:
> On 19 April 2016 at 11:26, Tomasz Nowicki <tn@semihalf.com> wrote:
> > On 15.04.2016 19:06, Tomasz Nowicki wrote:
> >>
> >> Passes 1.x miss PCI enhanced allocation (EA) header for fixed-BARs,
> >> thus these passes should use Cavium-specific config access functions
> >> that synthesize the missing EA capabilities.
> >>
> >> We already have DT driver which addresses errata requirements and
> >> allows to use special PCI config accessors. Currently this driver uses
> >> compatible = "cavium,pci-host-thunder-ecam" to mach against the errata.
> >> For ACPI case we need explicit errata number and corresponding
> >> DECLARE_ACPI_MCFG_FIXUP fixup code.
> >>
> >> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> >> ---
> >>   arch/arm64/Kconfig                  | 14 ++++++++++++++
> >>   arch/arm64/include/asm/cpufeature.h |  3 ++-
> >>   arch/arm64/kernel/cpu_errata.c      |  8 ++++++++
> >>   drivers/pci/host/pci-thunder-ecam.c | 35
> >> +++++++++++++++++++++++++++++++++++
> >>   4 files changed, 59 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> >> index 1bded87..b7614b8 100644
> >> --- a/arch/arm64/Kconfig
> >> +++ b/arch/arm64/Kconfig
> >> @@ -445,6 +445,20 @@ config CAVIUM_ERRATUM_27456
> >>
> >>           If unsure, say Y.
> >>
> >> +config CAVIUM_ERRATUM_24575
> >> +       bool "Cavium erratum 24575: Enhanced Allocation (EA) emualaion"
> >> +       default y
> >> +       help
> >> +         Enable workaround for erratum 24575.
> >> +
> >> +         Early versions of the Cavium Thunder CN88XX processor are
> >> missing
> >> +         Enhanced Allocation (EA) capabilities for the fixed BAR
> >> addresses used
> >> +         by the on-SoC hardware blocks. The erratum adds config access
> >> +         functions that synthesize the missing EA capabilities for
> >> versions
> >> +         that are missing that information.
> >> +
> >> +         If unsure, say Y.
> >> +
> >>   endmenu
> >>
> >>
> >> diff --git a/arch/arm64/include/asm/cpufeature.h
> >> b/arch/arm64/include/asm/cpufeature.h
> >> index b9b6494..a78364e 100644
> >> --- a/arch/arm64/include/asm/cpufeature.h
> >> +++ b/arch/arm64/include/asm/cpufeature.h
> >> @@ -35,8 +35,9 @@
> >>   #define ARM64_ALT_PAN_NOT_UAO                 10
> >>   #define ARM64_HAS_VIRT_HOST_EXTN              11
> >>   #define ARM64_WORKAROUND_CAVIUM_27456         12
> >> +#define ARM64_WORKAROUND_CAVIUM_24575          13
> >>
> >> -#define ARM64_NCAPS                            13
> >> +#define ARM64_NCAPS                            14
> >>
> >>   #ifndef __ASSEMBLY__
> >>
> >> diff --git a/arch/arm64/kernel/cpu_errata.c
> >> b/arch/arm64/kernel/cpu_errata.c
> >> index 06afd04..89c13d7 100644
> >> --- a/arch/arm64/kernel/cpu_errata.c
> >> +++ b/arch/arm64/kernel/cpu_errata.c
> >> @@ -97,6 +97,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
> >>                            (1 << MIDR_VARIANT_SHIFT) | 1),
> >>         },
> >>   #endif
> >> +#ifdef CONFIG_CAVIUM_ERRATUM_24575
> >> +       {
> >> +       /* Cavium ThunderX, pass 1.x */
> >> +               .desc = "Cavium erratum 24575",
> >> +               .capability = ARM64_WORKAROUND_CAVIUM_24575,
> >> +               MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
> >> +       },
> >> +#endif
> >>         {
> >>         }
> >>   };
> >> diff --git a/drivers/pci/host/pci-thunder-ecam.c
> >> b/drivers/pci/host/pci-thunder-ecam.c
> >> index f67c6d7..01de697 100644
> >> --- a/drivers/pci/host/pci-thunder-ecam.c
> >> +++ b/drivers/pci/host/pci-thunder-ecam.c
> >> @@ -11,10 +11,14 @@
> >>   #include <linux/ioport.h>
> >>   #include <linux/of_pci.h>
> >>   #include <linux/of.h>
> >> +#include <linux/pci-acpi.h>
> >>   #include <linux/platform_device.h>
> >>
> >> +#include <asm/virt.h>
> >> +
> >>   #include "../ecam.h"
> >>
> >> +
> >>   static void set_val(u32 v, int where, int size, u32 *val)
> >>   {
> >>         int shift = (where & 3) * 8;
> >> @@ -376,5 +380,36 @@ static struct platform_driver thunder_ecam_driver = {
> >>   };
> >>   module_platform_driver(thunder_ecam_driver);
> >>
> >> +#ifdef CONFIG_ACPI
> >> +
> >> +static bool needs_cavium_erratum_24575(struct pci_cfg_fixup *fixup,
> >> +                                     struct acpi_pci_root *root)
> >> +{
> >> +       /*
> >> +        * We must match errata code and be hypervisor, quirk does not
> >> apply
> >> +        * for virtual machines.
> >> +        */
> >> +       return cpus_have_cap(ARM64_WORKAROUND_CAVIUM_24575) &&
> >> +              is_hyp_mode_available();
> >> +}
> >> +
> >> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >> &pci_thunder_ecam_ops,
> >> +                       0, PCI_MCFG_BUS_ANY);
> >> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >> &pci_thunder_ecam_ops,
> >> +                       1, PCI_MCFG_BUS_ANY);
> >> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >> &pci_thunder_ecam_ops,
> >> +                       2, PCI_MCFG_BUS_ANY);
> >> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >> &pci_thunder_ecam_ops,
> >> +                       3, PCI_MCFG_BUS_ANY);
> >> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >> &pci_thunder_ecam_ops,
> >> +                       10, PCI_MCFG_BUS_ANY);
> >> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >> &pci_thunder_ecam_ops,
> >> +                       11, PCI_MCFG_BUS_ANY);
> >> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >> &pci_thunder_ecam_ops,
> >> +                       12, PCI_MCFG_BUS_ANY);
> >> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >> &pci_thunder_ecam_ops,
> >> +                       13, PCI_MCFG_BUS_ANY);
> >> +#endif
> >> +
> >
> >
> > I wonder if we can identify quirk based on _DSD properties, like that:
> >
> > Name (_DSD, Package () {
> >         ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
> >         Package () {
> >                 Package (2) {"cavium,pci-host-thunder-ecam", 1},
> >         }
> > })
> >
> > similar for PEM driver:
> >
> > Name (_DSD, Package () {
> >         ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
> >         Package () {
> >                 Package (2) {"cavium,pci-host-thunder-pem", 1},
> >         }
> > })
> >
> > Do you think it is right thing to do?
> >
> 
> Well if your adding new properties why not add a Cavium specific _CID?
> 

For clarity what I mean is something like this allowed.

Device (PCI0)
{
    Name (_HID, EisaId ("PNP0A08"))
    Name (_CID, EisaId ("PNP0A03"))
    Name (_HID, "CAV0666")) // Cavium Quirky Hardware

    ....

}

Graeme

Re: [Linaro-acpi] [PATCH V6 12/13] pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.

[Tomasz Nowicki]

On 19.04.2016 13:12, Graeme Gregory wrote:
> On Tue, Apr 19, 2016 at 11:41:29AM +0100, G Gregory wrote:
>> On 19 April 2016 at 11:26, Tomasz Nowicki <tn@semihalf.com> wrote:
>>> On 15.04.2016 19:06, Tomasz Nowicki wrote:
>>>>
>>>> Passes 1.x miss PCI enhanced allocation (EA) header for fixed-BARs,
>>>> thus these passes should use Cavium-specific config access functions
>>>> that synthesize the missing EA capabilities.
>>>>
>>>> We already have DT driver which addresses errata requirements and
>>>> allows to use special PCI config accessors. Currently this driver uses
>>>> compatible = "cavium,pci-host-thunder-ecam" to mach against the errata.
>>>> For ACPI case we need explicit errata number and corresponding
>>>> DECLARE_ACPI_MCFG_FIXUP fixup code.
>>>>
>>>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>>>> ---
>>>>    arch/arm64/Kconfig                  | 14 ++++++++++++++
>>>>    arch/arm64/include/asm/cpufeature.h |  3 ++-
>>>>    arch/arm64/kernel/cpu_errata.c      |  8 ++++++++
>>>>    drivers/pci/host/pci-thunder-ecam.c | 35
>>>> +++++++++++++++++++++++++++++++++++
>>>>    4 files changed, 59 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>>>> index 1bded87..b7614b8 100644
>>>> --- a/arch/arm64/Kconfig
>>>> +++ b/arch/arm64/Kconfig
>>>> @@ -445,6 +445,20 @@ config CAVIUM_ERRATUM_27456
>>>>
>>>>            If unsure, say Y.
>>>>
>>>> +config CAVIUM_ERRATUM_24575
>>>> +       bool "Cavium erratum 24575: Enhanced Allocation (EA) emualaion"
>>>> +       default y
>>>> +       help
>>>> +         Enable workaround for erratum 24575.
>>>> +
>>>> +         Early versions of the Cavium Thunder CN88XX processor are
>>>> missing
>>>> +         Enhanced Allocation (EA) capabilities for the fixed BAR
>>>> addresses used
>>>> +         by the on-SoC hardware blocks. The erratum adds config access
>>>> +         functions that synthesize the missing EA capabilities for
>>>> versions
>>>> +         that are missing that information.
>>>> +
>>>> +         If unsure, say Y.
>>>> +
>>>>    endmenu
>>>>
>>>>
>>>> diff --git a/arch/arm64/include/asm/cpufeature.h
>>>> b/arch/arm64/include/asm/cpufeature.h
>>>> index b9b6494..a78364e 100644
>>>> --- a/arch/arm64/include/asm/cpufeature.h
>>>> +++ b/arch/arm64/include/asm/cpufeature.h
>>>> @@ -35,8 +35,9 @@
>>>>    #define ARM64_ALT_PAN_NOT_UAO                 10
>>>>    #define ARM64_HAS_VIRT_HOST_EXTN              11
>>>>    #define ARM64_WORKAROUND_CAVIUM_27456         12
>>>> +#define ARM64_WORKAROUND_CAVIUM_24575          13
>>>>
>>>> -#define ARM64_NCAPS                            13
>>>> +#define ARM64_NCAPS                            14
>>>>
>>>>    #ifndef __ASSEMBLY__
>>>>
>>>> diff --git a/arch/arm64/kernel/cpu_errata.c
>>>> b/arch/arm64/kernel/cpu_errata.c
>>>> index 06afd04..89c13d7 100644
>>>> --- a/arch/arm64/kernel/cpu_errata.c
>>>> +++ b/arch/arm64/kernel/cpu_errata.c
>>>> @@ -97,6 +97,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
>>>>                             (1 << MIDR_VARIANT_SHIFT) | 1),
>>>>          },
>>>>    #endif
>>>> +#ifdef CONFIG_CAVIUM_ERRATUM_24575
>>>> +       {
>>>> +       /* Cavium ThunderX, pass 1.x */
>>>> +               .desc = "Cavium erratum 24575",
>>>> +               .capability = ARM64_WORKAROUND_CAVIUM_24575,
>>>> +               MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
>>>> +       },
>>>> +#endif
>>>>          {
>>>>          }
>>>>    };
>>>> diff --git a/drivers/pci/host/pci-thunder-ecam.c
>>>> b/drivers/pci/host/pci-thunder-ecam.c
>>>> index f67c6d7..01de697 100644
>>>> --- a/drivers/pci/host/pci-thunder-ecam.c
>>>> +++ b/drivers/pci/host/pci-thunder-ecam.c
>>>> @@ -11,10 +11,14 @@
>>>>    #include <linux/ioport.h>
>>>>    #include <linux/of_pci.h>
>>>>    #include <linux/of.h>
>>>> +#include <linux/pci-acpi.h>
>>>>    #include <linux/platform_device.h>
>>>>
>>>> +#include <asm/virt.h>
>>>> +
>>>>    #include "../ecam.h"
>>>>
>>>> +
>>>>    static void set_val(u32 v, int where, int size, u32 *val)
>>>>    {
>>>>          int shift = (where & 3) * 8;
>>>> @@ -376,5 +380,36 @@ static struct platform_driver thunder_ecam_driver = {
>>>>    };
>>>>    module_platform_driver(thunder_ecam_driver);
>>>>
>>>> +#ifdef CONFIG_ACPI
>>>> +
>>>> +static bool needs_cavium_erratum_24575(struct pci_cfg_fixup *fixup,
>>>> +                                     struct acpi_pci_root *root)
>>>> +{
>>>> +       /*
>>>> +        * We must match errata code and be hypervisor, quirk does not
>>>> apply
>>>> +        * for virtual machines.
>>>> +        */
>>>> +       return cpus_have_cap(ARM64_WORKAROUND_CAVIUM_24575) &&
>>>> +              is_hyp_mode_available();
>>>> +}
>>>> +
>>>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>>>> &pci_thunder_ecam_ops,
>>>> +                       0, PCI_MCFG_BUS_ANY);
>>>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>>>> &pci_thunder_ecam_ops,
>>>> +                       1, PCI_MCFG_BUS_ANY);
>>>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>>>> &pci_thunder_ecam_ops,
>>>> +                       2, PCI_MCFG_BUS_ANY);
>>>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>>>> &pci_thunder_ecam_ops,
>>>> +                       3, PCI_MCFG_BUS_ANY);
>>>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>>>> &pci_thunder_ecam_ops,
>>>> +                       10, PCI_MCFG_BUS_ANY);
>>>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>>>> &pci_thunder_ecam_ops,
>>>> +                       11, PCI_MCFG_BUS_ANY);
>>>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>>>> &pci_thunder_ecam_ops,
>>>> +                       12, PCI_MCFG_BUS_ANY);
>>>> +DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
>>>> &pci_thunder_ecam_ops,
>>>> +                       13, PCI_MCFG_BUS_ANY);
>>>> +#endif
>>>> +
>>>
>>>
>>> I wonder if we can identify quirk based on _DSD properties, like that:
>>>
>>> Name (_DSD, Package () {
>>>          ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
>>>          Package () {
>>>                  Package (2) {"cavium,pci-host-thunder-ecam", 1},
>>>          }
>>> })
>>>
>>> similar for PEM driver:
>>>
>>> Name (_DSD, Package () {
>>>          ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
>>>          Package () {
>>>                  Package (2) {"cavium,pci-host-thunder-pem", 1},
>>>          }
>>> })
>>>
>>> Do you think it is right thing to do?
>>>
>>
>> Well if your adding new properties why not add a Cavium specific _CID?
>>
>
> For clarity what I mean is something like this allowed.
>
> Device (PCI0)
> {
>      Name (_HID, EisaId ("PNP0A08"))
>      Name (_CID, EisaId ("PNP0A03"))
>      Name (_HID, "CAV0666")) // Cavium Quirky Hardware
>
>      ....
>
> }
>

Rafael, Bjorn, what do you think?

Tomasz

Re: [Linaro-acpi] [PATCH V6 12/13] pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.

[G Gregory]

On Tue, Apr 19, 2016 at 01:22:29PM +0200, Tomasz Nowicki wrote:
> 
> 
> On 19.04.2016 13:12, Graeme Gregory wrote:
> >On Tue, Apr 19, 2016 at 11:41:29AM +0100, G Gregory wrote:
> >>On 19 April 2016 at 11:26, Tomasz Nowicki <tn@semihalf.com> wrote:
> >>>On 15.04.2016 19:06, Tomasz Nowicki wrote:
> >>>>
> >>>>Passes 1.x miss PCI enhanced allocation (EA) header for fixed-BARs,
> >>>>thus these passes should use Cavium-specific config access functions
> >>>>that synthesize the missing EA capabilities.
> >>>>
> >>>>We already have DT driver which addresses errata requirements and
> >>>>allows to use special PCI config accessors. Currently this driver uses
> >>>>compatible = "cavium,pci-host-thunder-ecam" to mach against the errata.
> >>>>For ACPI case we need explicit errata number and corresponding
> >>>>DECLARE_ACPI_MCFG_FIXUP fixup code.
> >>>>
> >>>>Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> >>>>---
> >>>>   arch/arm64/Kconfig                  | 14 ++++++++++++++
> >>>>   arch/arm64/include/asm/cpufeature.h |  3 ++-
> >>>>   arch/arm64/kernel/cpu_errata.c      |  8 ++++++++
> >>>>   drivers/pci/host/pci-thunder-ecam.c | 35
> >>>>+++++++++++++++++++++++++++++++++++
> >>>>   4 files changed, 59 insertions(+), 1 deletion(-)
> >>>>
> >>>>diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> >>>>index 1bded87..b7614b8 100644
> >>>>--- a/arch/arm64/Kconfig
> >>>>+++ b/arch/arm64/Kconfig
> >>>>@@ -445,6 +445,20 @@ config CAVIUM_ERRATUM_27456
> >>>>
> >>>>           If unsure, say Y.
> >>>>
> >>>>+config CAVIUM_ERRATUM_24575
> >>>>+       bool "Cavium erratum 24575: Enhanced Allocation (EA) emualaion"
> >>>>+       default y
> >>>>+       help
> >>>>+         Enable workaround for erratum 24575.
> >>>>+
> >>>>+         Early versions of the Cavium Thunder CN88XX processor are
> >>>>missing
> >>>>+         Enhanced Allocation (EA) capabilities for the fixed BAR
> >>>>addresses used
> >>>>+         by the on-SoC hardware blocks. The erratum adds config access
> >>>>+         functions that synthesize the missing EA capabilities for
> >>>>versions
> >>>>+         that are missing that information.
> >>>>+
> >>>>+         If unsure, say Y.
> >>>>+
> >>>>   endmenu
> >>>>
> >>>>
> >>>>diff --git a/arch/arm64/include/asm/cpufeature.h
> >>>>b/arch/arm64/include/asm/cpufeature.h
> >>>>index b9b6494..a78364e 100644
> >>>>--- a/arch/arm64/include/asm/cpufeature.h
> >>>>+++ b/arch/arm64/include/asm/cpufeature.h
> >>>>@@ -35,8 +35,9 @@
> >>>>   #define ARM64_ALT_PAN_NOT_UAO                 10
> >>>>   #define ARM64_HAS_VIRT_HOST_EXTN              11
> >>>>   #define ARM64_WORKAROUND_CAVIUM_27456         12
> >>>>+#define ARM64_WORKAROUND_CAVIUM_24575          13
> >>>>
> >>>>-#define ARM64_NCAPS                            13
> >>>>+#define ARM64_NCAPS                            14
> >>>>
> >>>>   #ifndef __ASSEMBLY__
> >>>>
> >>>>diff --git a/arch/arm64/kernel/cpu_errata.c
> >>>>b/arch/arm64/kernel/cpu_errata.c
> >>>>index 06afd04..89c13d7 100644
> >>>>--- a/arch/arm64/kernel/cpu_errata.c
> >>>>+++ b/arch/arm64/kernel/cpu_errata.c
> >>>>@@ -97,6 +97,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
> >>>>                            (1 << MIDR_VARIANT_SHIFT) | 1),
> >>>>         },
> >>>>   #endif
> >>>>+#ifdef CONFIG_CAVIUM_ERRATUM_24575
> >>>>+       {
> >>>>+       /* Cavium ThunderX, pass 1.x */
> >>>>+               .desc = "Cavium erratum 24575",
> >>>>+               .capability = ARM64_WORKAROUND_CAVIUM_24575,
> >>>>+               MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
> >>>>+       },
> >>>>+#endif
> >>>>         {
> >>>>         }
> >>>>   };
> >>>>diff --git a/drivers/pci/host/pci-thunder-ecam.c
> >>>>b/drivers/pci/host/pci-thunder-ecam.c
> >>>>index f67c6d7..01de697 100644
> >>>>--- a/drivers/pci/host/pci-thunder-ecam.c
> >>>>+++ b/drivers/pci/host/pci-thunder-ecam.c
> >>>>@@ -11,10 +11,14 @@
> >>>>   #include <linux/ioport.h>
> >>>>   #include <linux/of_pci.h>
> >>>>   #include <linux/of.h>
> >>>>+#include <linux/pci-acpi.h>
> >>>>   #include <linux/platform_device.h>
> >>>>
> >>>>+#include <asm/virt.h>
> >>>>+
> >>>>   #include "../ecam.h"
> >>>>
> >>>>+
> >>>>   static void set_val(u32 v, int where, int size, u32 *val)
> >>>>   {
> >>>>         int shift = (where & 3) * 8;
> >>>>@@ -376,5 +380,36 @@ static struct platform_driver thunder_ecam_driver = {
> >>>>   };
> >>>>   module_platform_driver(thunder_ecam_driver);
> >>>>
> >>>>+#ifdef CONFIG_ACPI
> >>>>+
> >>>>+static bool needs_cavium_erratum_24575(struct pci_cfg_fixup *fixup,
> >>>>+                                     struct acpi_pci_root *root)
> >>>>+{
> >>>>+       /*
> >>>>+        * We must match errata code and be hypervisor, quirk does not
> >>>>apply
> >>>>+        * for virtual machines.
> >>>>+        */
> >>>>+       return cpus_have_cap(ARM64_WORKAROUND_CAVIUM_24575) &&
> >>>>+              is_hyp_mode_available();
> >>>>+}
> >>>>+
> >>>>+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >>>>&pci_thunder_ecam_ops,
> >>>>+                       0, PCI_MCFG_BUS_ANY);
> >>>>+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >>>>&pci_thunder_ecam_ops,
> >>>>+                       1, PCI_MCFG_BUS_ANY);
> >>>>+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >>>>&pci_thunder_ecam_ops,
> >>>>+                       2, PCI_MCFG_BUS_ANY);
> >>>>+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >>>>&pci_thunder_ecam_ops,
> >>>>+                       3, PCI_MCFG_BUS_ANY);
> >>>>+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >>>>&pci_thunder_ecam_ops,
> >>>>+                       10, PCI_MCFG_BUS_ANY);
> >>>>+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >>>>&pci_thunder_ecam_ops,
> >>>>+                       11, PCI_MCFG_BUS_ANY);
> >>>>+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >>>>&pci_thunder_ecam_ops,
> >>>>+                       12, PCI_MCFG_BUS_ANY);
> >>>>+DECLARE_ACPI_MCFG_FIXUP(NULL, needs_cavium_erratum_24575,
> >>>>&pci_thunder_ecam_ops,
> >>>>+                       13, PCI_MCFG_BUS_ANY);
> >>>>+#endif
> >>>>+
> >>>
> >>>
> >>>I wonder if we can identify quirk based on _DSD properties, like that:
> >>>
> >>>Name (_DSD, Package () {
> >>>         ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
> >>>         Package () {
> >>>                 Package (2) {"cavium,pci-host-thunder-ecam", 1},
> >>>         }
> >>>})
> >>>
> >>>similar for PEM driver:
> >>>
> >>>Name (_DSD, Package () {
> >>>         ToUUID("daffd814-6eba-4d8c-8a91-bc9bbf4aa301"),
> >>>         Package () {
> >>>                 Package (2) {"cavium,pci-host-thunder-pem", 1},
> >>>         }
> >>>})
> >>>
> >>>Do you think it is right thing to do?
> >>>
> >>
> >>Well if your adding new properties why not add a Cavium specific _CID?
> >>
> >
> >For clarity what I mean is something like this allowed.
> >
> >Device (PCI0)
> >{
> >     Name (_HID, EisaId ("PNP0A08"))
> >     Name (_CID, EisaId ("PNP0A03"))
> >     Name (_HID, "CAV0666")) // Cavium Quirky Hardware
> >
> >     ....
> >
> >}
> >
> 
> Rafael, Bjorn, what do you think?
> 

Another option as has been pointed out to me is that devices that are
not compliant don't advertise the IDs PNP0A03/8 and should use hardware
specific IDs.

Obviously in linux the drivers for them would re-use 90% of the generic
code with just their added accessors.

I do not know though what knock-on effects that would have and if there
is stuff that relies on the host bridge being IDed as the standard one.

Graeme

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Arnd Bergmann]

On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
> 
> Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.
> 
> pci-thunder-ecam.c contains config space accessors. Similar for 
> pci-thunder-pem.c but it also has extra init call (it is now called 
> thunder_pem_init) which finds and maps related registers.

They seem to do much more than just override the accessors, they actually
change the contents of the config space as well. Is that really necessary
on ACPI based systems as well?

Another idea: how about moving all of this logic into ACPI and calling
some AML method to access the config space if the devices are that
far out of spec.

	Arnd

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Arnd Bergmann]

On Friday 15 April 2016 19:06:43 Tomasz Nowicki wrote:
> From: Jayachandran C <jchandra@broadcom.com>
>
> Add config option PCI_GENERIC_ECAM and file drivers/pci/ecam.c to
> provide generic functions for accessing memory mapped PCI config space.
>
> The API is defined in drivers/pci/ecam.h and is written to replace the
> API in drivers/pci/host/pci-host-common.h. The file defines a new
> 'struct pci_config_window' to hold the information related to a PCI
> config area and its mapping. This structure is expected to be used as
> sysdata for controllers that have ECAM based mapping.
>
> Helper functions are provided to setup the mapping, free the mapping
> and to implement the map_bus method in 'struct pci_ops'
>
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>

I've taken a fresh look now at what is going on here.

> @@ -58,4 +58,9 @@ void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>  /* default ECAM ops, bus shift 20, generic read and write */
>  extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
>  
> +#ifdef CONFIG_PCI_HOST_GENERIC
> +/* for DT based pci controllers that support ECAM */
> +int pci_host_common_probe(struct platform_device *pdev,
> +			  struct pci_generic_ecam_ops *ops);
> +#endif
>  #endif

This doesn't seem to belong here: just leave the declaration
in the existing file. 

> diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
> index 7a0780d..31d6eb5 100644
> --- a/drivers/pci/host/Kconfig
> +++ b/drivers/pci/host/Kconfig
> @@ -82,6 +82,7 @@ config PCI_HOST_GENERIC
>  	bool "Generic PCI host controller"
>  	depends on (ARM || ARM64) && OF
>  	select PCI_HOST_COMMON
> +	select PCI_GENERIC_ECAM
>  	help
>  	  Say Y here if you want to support a simple generic PCI host
>  	  controller, such as the one emulated by kvmtool.
> diff --git a/drivers/pci/host/pci-host-common.c b/drivers/pci/host/pci-host-common.c
> index e9f850f..99d99b3 100644
> --- a/drivers/pci/host/pci-host-common.c
> +++ b/drivers/pci/host/pci-host-common.c
> @@ -22,27 +22,21 @@
>  #include <linux/of_pci.h>
>  #include <linux/platform_device.h>
>  
> -#include "pci-host-common.h"
> +#include "../ecam.h"

As mentioned, don't use headers from parent directories, anything
that needs to be shared must go into include/linux, while the parts
that are only needed in one directory should be declared there.

> -static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)
> +static void gen_pci_generic_unmap_cfg(void *ptr)
> +{
> +	pci_generic_ecam_free((struct pci_config_window *)ptr);
> +}

Why the void pointer?

> +static struct pci_generic_ecam_ops pci_thunder_pem_ops = {
> +	.bus_shift	= 24,
> +	.init		= thunder_pem_init,
> +	.pci_ops	= {
> +		.map_bus	= pci_generic_ecam_map_bus,
> +		.read		= thunder_pem_config_read,
> +		.write		= thunder_pem_config_write,
> +	}
> +};

Adding the callback pointer for init here and yet another structure
pci_config_window really seems to go too far with the number of
abstraction levels.

I think here it makes much more sense to just implement ECAM pci_ops
in ACPI separately, as the implementation really trivial to start with,
and all the complexity comes just from trying to share it with other
stuff. Doesn't ACPI already have an ECAM implementation for x86
that you could simply use?

	Arnd

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Jayachandran C]

On Wed, Apr 20, 2016 at 3:10 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Friday 15 April 2016 19:06:43 Tomasz Nowicki wrote:
>> From: Jayachandran C <jchandra@broadcom.com>
>>
>> Add config option PCI_GENERIC_ECAM and file drivers/pci/ecam.c to
>> provide generic functions for accessing memory mapped PCI config space.
>>
>> The API is defined in drivers/pci/ecam.h and is written to replace the
>> API in drivers/pci/host/pci-host-common.h. The file defines a new
>> 'struct pci_config_window' to hold the information related to a PCI
>> config area and its mapping. This structure is expected to be used as
>> sysdata for controllers that have ECAM based mapping.
>>
>> Helper functions are provided to setup the mapping, free the mapping
>> and to implement the map_bus method in 'struct pci_ops'
>>
>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>
> I've taken a fresh look now at what is going on here.
>
>> @@ -58,4 +58,9 @@ void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>>  /* default ECAM ops, bus shift 20, generic read and write */
>>  extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
>>
>> +#ifdef CONFIG_PCI_HOST_GENERIC
>> +/* for DT based pci controllers that support ECAM */
>> +int pci_host_common_probe(struct platform_device *pdev,
>> +                       struct pci_generic_ecam_ops *ops);
>> +#endif
>>  #endif
>
> This doesn't seem to belong here: just leave the declaration
> in the existing file.

This can be done, the file would just have one line so I thought
it made sense to move it to ecam.h where the struct is defined.

>> diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
>> index 7a0780d..31d6eb5 100644
>> --- a/drivers/pci/host/Kconfig
>> +++ b/drivers/pci/host/Kconfig
>> @@ -82,6 +82,7 @@ config PCI_HOST_GENERIC
>>       bool "Generic PCI host controller"
>>       depends on (ARM || ARM64) && OF
>>       select PCI_HOST_COMMON
>> +     select PCI_GENERIC_ECAM
>>       help
>>         Say Y here if you want to support a simple generic PCI host
>>         controller, such as the one emulated by kvmtool.
>> diff --git a/drivers/pci/host/pci-host-common.c b/drivers/pci/host/pci-host-common.c
>> index e9f850f..99d99b3 100644
>> --- a/drivers/pci/host/pci-host-common.c
>> +++ b/drivers/pci/host/pci-host-common.c
>> @@ -22,27 +22,21 @@
>>  #include <linux/of_pci.h>
>>  #include <linux/platform_device.h>
>>
>> -#include "pci-host-common.h"
>> +#include "../ecam.h"
>
> As mentioned, don't use headers from parent directories, anything
> that needs to be shared must go into include/linux, while the parts
> that are only needed in one directory should be declared there.

This is also ok - It can either go to pci.h or a separate pci-ecam.h

>> -static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)
>> +static void gen_pci_generic_unmap_cfg(void *ptr)
>> +{
>> +     pci_generic_ecam_free((struct pci_config_window *)ptr);
>> +}
>
> Why the void pointer?

devm_add_action() needs it.

>> +static struct pci_generic_ecam_ops pci_thunder_pem_ops = {
>> +     .bus_shift      = 24,
>> +     .init           = thunder_pem_init,
>> +     .pci_ops        = {
>> +             .map_bus        = pci_generic_ecam_map_bus,
>> +             .read           = thunder_pem_config_read,
>> +             .write          = thunder_pem_config_write,
>> +     }
>> +};
>
> Adding the callback pointer for init here and yet another structure
> pci_config_window really seems to go too far with the number of
> abstraction levels.

The abstraction was already there in pci-host-common.h for
ops for ECAM/CAM based controllers. It made sense to move it
to ecam.h and use it for ECAM based ACPI [1].

We need to pass pci_ops, bus_shift and an additional pointer
for quirks for ECAM based host controllers. Having it as a
structure pci_generic_ecam_ops reduces the function arguments,
and also keeps most of the older API.

> I think here it makes much more sense to just implement ECAM pci_ops
> in ACPI separately, as the implementation really trivial to start with,
> and all the complexity comes just from trying to share it with other
> stuff. Doesn't ACPI already have an ECAM implementation for x86
> that you could simply use?

The implementation is extremely trivial on 64 bit, and slightly more
complex in 32bit (pci-host-common.c per bus mapping and set_pte
based mapping on  x86). The generic ACPI on 64 bit is very simple
if there are no quirks,I have already posted that [2] some time back.

ACPI on x86 also has a 32 bit and a 64 bit version
(arch/x86/pci/mmconfig_{32,64}.c}. The code there is a bit messed
up and it does not make sense to share or reuse that.

There has been suggestions earlier from Bjorn on sharing the
ECAM implementation[1], which was the starting point of
doing this patch.

Overall, this patch improves config window mapping for
pci-host-common.c based drivers on 64 bit and deletes
quite a bit of duplicated code. I would argue that this makes
sense even without ACPI.

JC.

[1] https://lkml.org/lkml/2016/3/3/921
[2] http://article.gmane.org/gmane.linux.kernel.pci/47753

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Jayachandran C]

On Fri, Apr 15, 2016 at 10:36 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
> This patch is going to implement generic PCI host controller for
> ACPI world, similar to what pci-host-generic.c driver does for DT world.
>
> All such drivers, which we have seen so far, were implemented within
> arch/ directory since they had some arch assumptions (x86 and ia64).
> However, they all are doing similar thing, so it makes sense to find
> some common code and abstract it into the generic driver.
>
> In order to handle PCI config space regions properly, we define new
> MCFG interface which parses MCFG table and keep its entries
> in a list. New pci_mcfg_init call is defined so that we do not depend
> on PCI_MMCONFIG. Regions are not mapped until host bridge ask for it.
>
> The implementation of pci_acpi_scan_root() looks up the saved MCFG entries
> and sets up a new mapping. Generic PCI functions are used for
> accessing config space. Driver selects PCI_GENERIC_ECAM and uses functions
> from drivers/pci/ecam.h to create and access ECAM mappings.
>
> As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
> should be made on a per-architecture basis.
>
> This patch is heavily based on the updated version from Jayachandran C:
> https://lkml.org/lkml/2016/4/11/908
> git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)

This is a little bit unusual because I had not posted the v3 patch
to the mailing list yet, but you posted a variant of it The git repository
should not be in the commit comment because it is a temporary location.

There are some changes here I don't agree with. I think it will be
better if you can post a version without the quirk handling and with
some of the suggestions below.

> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> ---
>  drivers/acpi/Kconfig        |   8 ++
>  drivers/acpi/Makefile       |   1 +
>  drivers/acpi/bus.c          |   1 +
>  drivers/acpi/pci_gen_host.c | 231 ++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/pci.h         |   6 ++
>  5 files changed, 247 insertions(+)
>  create mode 100644 drivers/acpi/pci_gen_host.c
>
> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
> index 183ffa3..70272c5 100644
> --- a/drivers/acpi/Kconfig
> +++ b/drivers/acpi/Kconfig
> @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
>           i.e., segment/bus/device/function tuples, with physical slots in
>           the system.  If you are unsure, say N.
>
> +config ACPI_PCI_HOST_GENERIC
> +       bool
> +       select PCI_GENERIC_ECAM
> +       help
> +         Select this config option from the architecture Kconfig,
> +         if it is preferred to enable ACPI PCI host controller driver which
> +         has no arch-specific assumptions.
> +
>  config X86_PM_TIMER
>         bool "Power Management Timer Support" if EXPERT
>         depends on X86
> diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
> index 81e5cbc..b12fa64 100644
> --- a/drivers/acpi/Makefile
> +++ b/drivers/acpi/Makefile
> @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
>  acpi-y                         += ec.o
>  acpi-$(CONFIG_ACPI_DOCK)       += dock.o
>  acpi-y                         += pci_root.o pci_link.o pci_irq.o
> +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC)    += pci_gen_host.o
>  acpi-y                         += acpi_lpss.o acpi_apd.o
>  acpi-y                         += acpi_platform.o
>  acpi-y                         += acpi_pnp.o
> diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
> index c068c82..803a1d7 100644
> --- a/drivers/acpi/bus.c
> +++ b/drivers/acpi/bus.c
> @@ -1107,6 +1107,7 @@ static int __init acpi_init(void)
>         }
>
>         pci_mmcfg_late_init();
> +       pci_mcfg_init();

Please see below.

>         acpi_scan_init();
>         acpi_ec_init();
>         acpi_debugfs_init();
> diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
> new file mode 100644
> index 0000000..fd360b5
> --- /dev/null
> +++ b/drivers/acpi/pci_gen_host.c
> @@ -0,0 +1,231 @@
> +/*

You seem to have removed the copyright line, this is not proper, you
should probably add your copyright line if you think your changes are
significant.

> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation (the "GPL").
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 (GPLv2) for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 (GPLv2) along with this source code.
> + */
> +#include <linux/kernel.h>
> +#include <linux/pci.h>
> +#include <linux/pci-acpi.h>
> +#include <linux/sfi_acpi.h>
> +#include <linux/slab.h>
> +
> +#include "../pci/ecam.h"
> +
> +#define PREFIX "ACPI: "
> +
> +/* Structure to hold entries from the MCFG table */
> +struct mcfg_entry {
> +       struct list_head        list;
> +       phys_addr_t             addr;
> +       u16                     segment;
> +       u8                      bus_start;
> +       u8                      bus_end;
> +};
> +
> +/* List to save mcfg entries */
> +static LIST_HEAD(pci_mcfg_list);
> +static DEFINE_MUTEX(pci_mcfg_lock);

There is no need to use a list or lock here, I had used an
array and that is sufficient since it is not modified after it
is filled initially.

> +/* ACPI info for generic ACPI PCI controller */
> +struct acpi_pci_generic_root_info {
> +       struct acpi_pci_root_info       common;
> +       struct pci_config_window        *cfg;   /* config space mapping */
> +};
> +
> +/* Find the entry in mcfg list which contains range bus_start */
> +static struct mcfg_entry *pci_mcfg_lookup(u16 seg, u8 bus_start)
> +{
> +       struct mcfg_entry *e;
> +
> +       list_for_each_entry(e, &pci_mcfg_list, list) {
> +               if (e->segment == seg &&
> +                   e->bus_start <= bus_start && bus_start <= e->bus_end)
> +                       return e;
> +       }
> +
> +       return NULL;
> +}
> +
> +
> +/*
> + * Lookup the bus range for the domain in MCFG, and set up config space
> + * mapping.
> + */
> +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
> +                                      struct acpi_pci_generic_root_info *ri)
> +{
> +       u16 seg = root->segment;
> +       u8 bus_start = root->secondary.start;
> +       u8 bus_end = root->secondary.end;
> +       struct pci_config_window *cfg;
> +       struct mcfg_entry *e;
> +       phys_addr_t addr;
> +       int err = 0;
> +
> +       mutex_lock(&pci_mcfg_lock);
> +       e = pci_mcfg_lookup(seg, bus_start);
> +       if (!e) {
> +               addr = acpi_pci_root_get_mcfg_addr(root->device->handle);

The acpi_pci_root_get_mcfg_addr() is already called in pci_root.c, doing
it again here is unnecessary.

I think you can have a function to pick up addr, bus_start, bus_end given
a domain from either MCFG or using _CBA method, but I think that
should be done in pci_root.c in a separate patch.

> +               if (addr == 0) {
> +                       pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
> +                              seg, bus_start, bus_end);
> +                       err = -ENOENT;
> +                       goto err_out;
> +               }
> +       } else {
> +               if (bus_start != e->bus_start) {
> +                       pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
> +                              seg, bus_start, bus_end, e->bus_start);
> +                       err = -EINVAL;
> +                       goto err_out;
> +               } else if (bus_end != e->bus_end) {
> +                       pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
> +                               seg, bus_start, bus_end, e->bus_end);
> +                       bus_end = min(bus_end, e->bus_end);
> +               }
> +               addr = e->addr;
> +       }
> +
> +       cfg = pci_generic_ecam_create(&root->device->dev, addr, bus_start,
> +                                     bus_end, &pci_generic_ecam_default_ops);
> +       if (IS_ERR(cfg)) {
> +               err = PTR_ERR(cfg);
> +               pr_err("%04x:%02x-%02x error %d mapping CAM\n", seg,
> +                       bus_start, bus_end, err);
> +               goto err_out;
> +       }

You seem to have moved all the config space mapping to this
point. Intel seems to do the mapping when they read MCFG for the
entries, and I had followed that model, and that avoids having another
array/list to save the values.

> +       cfg->domain = seg;
> +       ri->cfg = cfg;
> +err_out:
> +       mutex_unlock(&pci_mcfg_lock);
> +       return err;
> +}
> +
> +/* release_info: free resrouces allocated by init_info */
> +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
> +{
> +       struct acpi_pci_generic_root_info *ri;
> +
> +       ri = container_of(ci, struct acpi_pci_generic_root_info, common);
> +       pci_generic_ecam_free(ri->cfg);
> +       kfree(ri);
> +}
> +
> +static struct acpi_pci_root_ops acpi_pci_root_ops = {
> +       .release_info = pci_acpi_generic_release_info,
> +};
> +
> +/* Interface called from ACPI code to setup PCI host controller */
> +struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
> +{
> +       int node = acpi_get_node(root->device->handle);
> +       struct acpi_pci_generic_root_info *ri;
> +       struct pci_bus *bus, *child;
> +       int err;
> +
> +       ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
> +       if (!ri)
> +               return NULL;
> +
> +       err = pci_acpi_setup_ecam_mapping(root, ri);
> +       if (err)
> +               return NULL;
> +
> +       acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
> +       bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
> +                                  ri->cfg);
> +       if (!bus)
> +               return NULL;
> +
> +       pci_bus_size_bridges(bus);
> +       pci_bus_assign_resources(bus);
> +
> +       list_for_each_entry(child, &bus->children, node)
> +               pcie_bus_configure_settings(child);
> +
> +       return bus;
> +}
> +
> +/* handle MCFG table entries */
> +static __init int pci_mcfg_parse(struct acpi_table_header *header)
> +{
> +       struct acpi_table_mcfg *mcfg;
> +       struct acpi_mcfg_allocation *mptr;
> +       struct mcfg_entry *e, *arr;
> +       int i, n;
> +
> +       if (!header)
> +               return -EINVAL;
> +
> +       mcfg = (struct acpi_table_mcfg *)header;
> +       mptr = (struct acpi_mcfg_allocation *) &mcfg[1];
> +       n = (header->length - sizeof(*mcfg)) / sizeof(*mptr);
> +       if (n <= 0 || n > 255) {
> +               pr_err(PREFIX " MCFG has incorrect entries (%d).\n", n);
> +               return -EINVAL;
> +       }
> +
> +       arr = kcalloc(n, sizeof(*arr), GFP_KERNEL);
> +       if (!arr)
> +               return -ENOMEM;

Here you already have an array which is also connected as a linked
list which is unnecessary.

> +       for (i = 0, e = arr; i < n; i++, mptr++, e++) {
> +               e->segment = mptr->pci_segment;
> +               e->addr =  mptr->address;
> +               e->bus_start = mptr->start_bus_number;
> +               e->bus_end = mptr->end_bus_number;
> +               list_add(&e->list, &pci_mcfg_list);
> +               pr_info(PREFIX
> +                       "MCFG entry for domain %04x [bus %02x-%02x] (base %pa)\n",
> +                       e->segment, e->bus_start, e->bus_end, &e->addr);
> +       }
> +
> +       return 0;
> +}
> +
> +/* Interface called by ACPI - parse and save MCFG table */
> +void __init pci_mcfg_init(void)
> +{
> +       int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
> +       if (err)
> +               pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);
> +       else if (list_empty(&pci_mcfg_list))
> +               pr_info(PREFIX "No valid entries in MCFG table.\n");
> +       else {
> +               struct mcfg_entry *e;
> +               int i = 0;
> +               list_for_each_entry(e, &pci_mcfg_list, list)
> +                       i++;
> +               pr_info(PREFIX "MCFG table loaded, %d entries\n", i);
> +       }
> +}
> +
> +/* Raw operations, works only for MCFG entries with an associated bus */
> +int raw_pci_read(unsigned int domain, unsigned int busn, unsigned int devfn,
> +                int reg, int len, u32 *val)
> +{
> +       struct pci_bus *bus = pci_find_bus(domain, busn);
> +
> +       if (!bus)
> +               return PCIBIOS_DEVICE_NOT_FOUND;
> +       return bus->ops->read(bus, devfn, reg, len, val);
> +}
> +
> +int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int devfn,
> +                 int reg, int len, u32 val)
> +{
> +       struct pci_bus *bus = pci_find_bus(domain, busn);
> +
> +       if (!bus)
> +               return PCIBIOS_DEVICE_NOT_FOUND;
> +       return bus->ops->write(bus, devfn, reg, len, val);
> +}
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index df1f33d..c0422ea 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1729,6 +1729,12 @@ static inline void pci_mmcfg_early_init(void) { }
>  static inline void pci_mmcfg_late_init(void) { }
>  #endif
>
> +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
> +void __init pci_mcfg_init(void);
> +#else
> +static inline void pci_mcfg_init(void) { return; }
> +#endif

You can still use the function pci_mmcfg_late_init() if
PCI_MMCONFIG or ACPI_PCI_HOST_GENERIC is defined

> +
>  int pci_ext_cfg_avail(void);
>
>  void __iomem *pci_ioremap_bar(struct pci_dev *pdev, int bar);


JC.

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Tomasz Nowicki]

On 20.04.2016 21:12, Jayachandran C wrote:
> On Fri, Apr 15, 2016 at 10:36 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
>> This patch is going to implement generic PCI host controller for
>> ACPI world, similar to what pci-host-generic.c driver does for DT world.
>>
>> All such drivers, which we have seen so far, were implemented within
>> arch/ directory since they had some arch assumptions (x86 and ia64).
>> However, they all are doing similar thing, so it makes sense to find
>> some common code and abstract it into the generic driver.
>>
>> In order to handle PCI config space regions properly, we define new
>> MCFG interface which parses MCFG table and keep its entries
>> in a list. New pci_mcfg_init call is defined so that we do not depend
>> on PCI_MMCONFIG. Regions are not mapped until host bridge ask for it.
>>
>> The implementation of pci_acpi_scan_root() looks up the saved MCFG entries
>> and sets up a new mapping. Generic PCI functions are used for
>> accessing config space. Driver selects PCI_GENERIC_ECAM and uses functions
>> from drivers/pci/ecam.h to create and access ECAM mappings.
>>
>> As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
>> should be made on a per-architecture basis.
>>
>> This patch is heavily based on the updated version from Jayachandran C:
>> https://lkml.org/lkml/2016/4/11/908
>> git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
>
> This is a little bit unusual because I had not posted the v3 patch
> to the mailing list yet, but you posted a variant of it The git repository
> should not be in the commit comment because it is a temporary location.

We all agree this too important for everybody to delay this series. So 
main motivation is to keep all discussion&patches within one unified 
series. I would like to finally find direction we need to go. Stating 
another discussion based on my previous patch set v5 confused people, 
they do no know who is driving this. Again, lets cooperate to move it 
forward within one patch set.

I agree with you we need maintainers to join this discussion.

>
> There are some changes here I don't agree with. I think it will be
> better if you can post a version without the quirk handling and with
> some of the suggestions below.

The next version will not have quirk handling part. Regarding your 
comments, please see below.

>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>> ---
>>   drivers/acpi/Kconfig        |   8 ++
>>   drivers/acpi/Makefile       |   1 +
>>   drivers/acpi/bus.c          |   1 +
>>   drivers/acpi/pci_gen_host.c | 231 ++++++++++++++++++++++++++++++++++++++++++++
>>   include/linux/pci.h         |   6 ++
>>   5 files changed, 247 insertions(+)
>>   create mode 100644 drivers/acpi/pci_gen_host.c
>>
>> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
>> index 183ffa3..70272c5 100644
>> --- a/drivers/acpi/Kconfig
>> +++ b/drivers/acpi/Kconfig
>> @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
>>            i.e., segment/bus/device/function tuples, with physical slots in
>>            the system.  If you are unsure, say N.
>>
>> +config ACPI_PCI_HOST_GENERIC
>> +       bool
>> +       select PCI_GENERIC_ECAM
>> +       help
>> +         Select this config option from the architecture Kconfig,
>> +         if it is preferred to enable ACPI PCI host controller driver which
>> +         has no arch-specific assumptions.
>> +
>>   config X86_PM_TIMER
>>          bool "Power Management Timer Support" if EXPERT
>>          depends on X86
>> diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
>> index 81e5cbc..b12fa64 100644
>> --- a/drivers/acpi/Makefile
>> +++ b/drivers/acpi/Makefile
>> @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
>>   acpi-y                         += ec.o
>>   acpi-$(CONFIG_ACPI_DOCK)       += dock.o
>>   acpi-y                         += pci_root.o pci_link.o pci_irq.o
>> +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC)    += pci_gen_host.o
>>   acpi-y                         += acpi_lpss.o acpi_apd.o
>>   acpi-y                         += acpi_platform.o
>>   acpi-y                         += acpi_pnp.o
>> diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
>> index c068c82..803a1d7 100644
>> --- a/drivers/acpi/bus.c
>> +++ b/drivers/acpi/bus.c
>> @@ -1107,6 +1107,7 @@ static int __init acpi_init(void)
>>          }
>>
>>          pci_mmcfg_late_init();
>> +       pci_mcfg_init();
>
> Please see below.
>
>>          acpi_scan_init();
>>          acpi_ec_init();
>>          acpi_debugfs_init();
>> diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
>> new file mode 100644
>> index 0000000..fd360b5
>> --- /dev/null
>> +++ b/drivers/acpi/pci_gen_host.c
>> @@ -0,0 +1,231 @@
>> +/*
>
> You seem to have removed the copyright line, this is not proper, you
> should probably add your copyright line if you think your changes are
> significant.

I rather forgot to add copyright here, I will fix it.

>
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License, version 2, as
>> + * published by the Free Software Foundation (the "GPL").
>> + *
>> + * This program is distributed in the hope that it will be useful, but
>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License version 2 (GPLv2) for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * version 2 (GPLv2) along with this source code.
>> + */
>> +#include <linux/kernel.h>
>> +#include <linux/pci.h>
>> +#include <linux/pci-acpi.h>
>> +#include <linux/sfi_acpi.h>
>> +#include <linux/slab.h>
>> +
>> +#include "../pci/ecam.h"
>> +
>> +#define PREFIX "ACPI: "
>> +
>> +/* Structure to hold entries from the MCFG table */
>> +struct mcfg_entry {
>> +       struct list_head        list;
>> +       phys_addr_t             addr;
>> +       u16                     segment;
>> +       u8                      bus_start;
>> +       u8                      bus_end;
>> +};
>> +
>> +/* List to save mcfg entries */
>> +static LIST_HEAD(pci_mcfg_list);
>> +static DEFINE_MUTEX(pci_mcfg_lock);
>
> There is no need to use a list or lock here, I had used an
> array and that is sufficient since it is not modified after it
> is filled initially.

ACPI PCI driver supports hot plug/removal, I want to avoid races using 
this lock. I decided to use list because I do ECAM mapping on demand. 
See below for more details.

>
>> +/* ACPI info for generic ACPI PCI controller */
>> +struct acpi_pci_generic_root_info {
>> +       struct acpi_pci_root_info       common;
>> +       struct pci_config_window        *cfg;   /* config space mapping */
>> +};
>> +
>> +/* Find the entry in mcfg list which contains range bus_start */
>> +static struct mcfg_entry *pci_mcfg_lookup(u16 seg, u8 bus_start)
>> +{
>> +       struct mcfg_entry *e;
>> +
>> +       list_for_each_entry(e, &pci_mcfg_list, list) {
>> +               if (e->segment == seg &&
>> +                   e->bus_start <= bus_start && bus_start <= e->bus_end)
>> +                       return e;
>> +       }
>> +
>> +       return NULL;
>> +}
>> +
>> +
>> +/*
>> + * Lookup the bus range for the domain in MCFG, and set up config space
>> + * mapping.
>> + */
>> +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
>> +                                      struct acpi_pci_generic_root_info *ri)
>> +{
>> +       u16 seg = root->segment;
>> +       u8 bus_start = root->secondary.start;
>> +       u8 bus_end = root->secondary.end;
>> +       struct pci_config_window *cfg;
>> +       struct mcfg_entry *e;
>> +       phys_addr_t addr;
>> +       int err = 0;
>> +
>> +       mutex_lock(&pci_mcfg_lock);
>> +       e = pci_mcfg_lookup(seg, bus_start);
>> +       if (!e) {
>> +               addr = acpi_pci_root_get_mcfg_addr(root->device->handle);
>
> The acpi_pci_root_get_mcfg_addr() is already called in pci_root.c, doing
> it again here is unnecessary.

I put it here as per Bjorn's request, see:
https://lkml.org/lkml/2016/3/4/946

If this is not valid any more, I can easily remove it.

>
> I think you can have a function to pick up addr, bus_start, bus_end given
> a domain from either MCFG or using _CBA method, but I think that
> should be done in pci_root.c in a separate patch.
>
>> +               if (addr == 0) {
>> +                       pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
>> +                              seg, bus_start, bus_end);
>> +                       err = -ENOENT;
>> +                       goto err_out;
>> +               }
>> +       } else {
>> +               if (bus_start != e->bus_start) {
>> +                       pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
>> +                              seg, bus_start, bus_end, e->bus_start);
>> +                       err = -EINVAL;
>> +                       goto err_out;
>> +               } else if (bus_end != e->bus_end) {
>> +                       pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
>> +                               seg, bus_start, bus_end, e->bus_end);
>> +                       bus_end = min(bus_end, e->bus_end);
>> +               }
>> +               addr = e->addr;
>> +       }
>> +
>> +       cfg = pci_generic_ecam_create(&root->device->dev, addr, bus_start,
>> +                                     bus_end, &pci_generic_ecam_default_ops);
>> +       if (IS_ERR(cfg)) {
>> +               err = PTR_ERR(cfg);
>> +               pr_err("%04x:%02x-%02x error %d mapping CAM\n", seg,
>> +                       bus_start, bus_end, err);
>> +               goto err_out;
>> +       }
>
> You seem to have moved all the config space mapping to this
> point. Intel seems to do the mapping when they read MCFG for the
> entries, and I had followed that model, and that avoids having another
> array/list to save the values.

See:
https://lkml.org/lkml/2016/3/3/921

I agree with Bjorn here, we should map it whenever we need this instead 
of mapping all MCFG entries just in case. Also, I see other advantages:
1. We can always use valid "dev" for pci_generic_ecam_create call, what 
you can't do when you use pci_generic_ecam_create during MCFG parsing.
2. No need for special handling for entries coming from _CBA, the path 
is the same for MCFG and _CBA. We do not have to remember that only _CBA 
related entries have to be unmapped.

>
>> +       cfg->domain = seg;
>> +       ri->cfg = cfg;
>> +err_out:
>> +       mutex_unlock(&pci_mcfg_lock);
>> +       return err;
>> +}
>> +
>> +/* release_info: free resrouces allocated by init_info */
>> +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
>> +{
>> +       struct acpi_pci_generic_root_info *ri;
>> +
>> +       ri = container_of(ci, struct acpi_pci_generic_root_info, common);
>> +       pci_generic_ecam_free(ri->cfg);
>> +       kfree(ri);
>> +}
>> +
>> +static struct acpi_pci_root_ops acpi_pci_root_ops = {
>> +       .release_info = pci_acpi_generic_release_info,
>> +};
>> +
>> +/* Interface called from ACPI code to setup PCI host controller */
>> +struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>> +{
>> +       int node = acpi_get_node(root->device->handle);
>> +       struct acpi_pci_generic_root_info *ri;
>> +       struct pci_bus *bus, *child;
>> +       int err;
>> +
>> +       ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
>> +       if (!ri)
>> +               return NULL;
>> +
>> +       err = pci_acpi_setup_ecam_mapping(root, ri);
>> +       if (err)
>> +               return NULL;
>> +
>> +       acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
>> +       bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
>> +                                  ri->cfg);
>> +       if (!bus)
>> +               return NULL;
>> +
>> +       pci_bus_size_bridges(bus);
>> +       pci_bus_assign_resources(bus);
>> +
>> +       list_for_each_entry(child, &bus->children, node)
>> +               pcie_bus_configure_settings(child);
>> +
>> +       return bus;
>> +}
>> +
>> +/* handle MCFG table entries */
>> +static __init int pci_mcfg_parse(struct acpi_table_header *header)
>> +{
>> +       struct acpi_table_mcfg *mcfg;
>> +       struct acpi_mcfg_allocation *mptr;
>> +       struct mcfg_entry *e, *arr;
>> +       int i, n;
>> +
>> +       if (!header)
>> +               return -EINVAL;
>> +
>> +       mcfg = (struct acpi_table_mcfg *)header;
>> +       mptr = (struct acpi_mcfg_allocation *) &mcfg[1];
>> +       n = (header->length - sizeof(*mcfg)) / sizeof(*mptr);
>> +       if (n <= 0 || n > 255) {
>> +               pr_err(PREFIX " MCFG has incorrect entries (%d).\n", n);
>> +               return -EINVAL;
>> +       }
>> +
>> +       arr = kcalloc(n, sizeof(*arr), GFP_KERNEL);
>> +       if (!arr)
>> +               return -ENOMEM;
>
> Here you already have an array which is also connected as a linked
> list which is unnecessary.

The array is to avoid complicated error handling in case the allocation 
for some of element would failed. I can rework this to allocate each 
element separately or to use array but the code is simpler now. As 
explained above (on demand mapping), I would like to keep list/array 
with MCFG entries (I preferred list).

>
>> +       for (i = 0, e = arr; i < n; i++, mptr++, e++) {
>> +               e->segment = mptr->pci_segment;
>> +               e->addr =  mptr->address;
>> +               e->bus_start = mptr->start_bus_number;
>> +               e->bus_end = mptr->end_bus_number;
>> +               list_add(&e->list, &pci_mcfg_list);
>> +               pr_info(PREFIX
>> +                       "MCFG entry for domain %04x [bus %02x-%02x] (base %pa)\n",
>> +                       e->segment, e->bus_start, e->bus_end, &e->addr);
>> +       }
>> +
>> +       return 0;
>> +}
>> +
>> +/* Interface called by ACPI - parse and save MCFG table */
>> +void __init pci_mcfg_init(void)
>> +{
>> +       int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
>> +       if (err)
>> +               pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);
>> +       else if (list_empty(&pci_mcfg_list))
>> +               pr_info(PREFIX "No valid entries in MCFG table.\n");
>> +       else {
>> +               struct mcfg_entry *e;
>> +               int i = 0;
>> +               list_for_each_entry(e, &pci_mcfg_list, list)
>> +                       i++;
>> +               pr_info(PREFIX "MCFG table loaded, %d entries\n", i);
>> +       }
>> +}
>> +
>> +/* Raw operations, works only for MCFG entries with an associated bus */
>> +int raw_pci_read(unsigned int domain, unsigned int busn, unsigned int devfn,
>> +                int reg, int len, u32 *val)
>> +{
>> +       struct pci_bus *bus = pci_find_bus(domain, busn);
>> +
>> +       if (!bus)
>> +               return PCIBIOS_DEVICE_NOT_FOUND;
>> +       return bus->ops->read(bus, devfn, reg, len, val);
>> +}
>> +
>> +int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int devfn,
>> +                 int reg, int len, u32 val)
>> +{
>> +       struct pci_bus *bus = pci_find_bus(domain, busn);
>> +
>> +       if (!bus)
>> +               return PCIBIOS_DEVICE_NOT_FOUND;
>> +       return bus->ops->write(bus, devfn, reg, len, val);
>> +}
>> diff --git a/include/linux/pci.h b/include/linux/pci.h
>> index df1f33d..c0422ea 100644
>> --- a/include/linux/pci.h
>> +++ b/include/linux/pci.h
>> @@ -1729,6 +1729,12 @@ static inline void pci_mmcfg_early_init(void) { }
>>   static inline void pci_mmcfg_late_init(void) { }
>>   #endif
>>
>> +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
>> +void __init pci_mcfg_init(void);
>> +#else
>> +static inline void pci_mcfg_init(void) { return; }
>> +#endif
>
> You can still use the function pci_mmcfg_late_init() if
> PCI_MMCONFIG or ACPI_PCI_HOST_GENERIC is defined
>

OK

-#ifdef CONFIG_PCI_MMCONFIG
+#if defined(CONFIG_PCI_MMCONFIG) || defined(ACPI_PCI_HOST_GENERIC)
void __init pci_mmcfg_early_init(void);
void __init pci_mmcfg_late_init(void);
#else
static inline void pci_mmcfg_early_init(void) { }
static inline void pci_mmcfg_late_init(void) { }
#endif

is that what you mean?

Tomasz

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Tomasz Nowicki]

On 19.04.2016 15:06, Arnd Bergmann wrote:
> On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
>>
>> Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.
>>
>> pci-thunder-ecam.c contains config space accessors. Similar for
>> pci-thunder-pem.c but it also has extra init call (it is now called
>> thunder_pem_init) which finds and maps related registers.
>
> They seem to do much more than just override the accessors, they actually
> change the contents of the config space as well. Is that really necessary
> on ACPI based systems as well?

Yes, the pci-thunder-ecam.c accessors are meant to emulate config space 
capabilities. They are necessary to synthesize EA capabilities (fixed 
PCI BARs), it wont work without this, for ACPI boot as well.

>
> Another idea: how about moving all of this logic into ACPI and calling
> some AML method to access the config space if the devices are that
> far out of spec.

Do you mean Linux specific way to call non-standard config space 
accessors? Then non-standard accessors are going to AML methods which 
are called from common code which handles quirks via unified API ?

Thanks,
Tomasz

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Arnd Bergmann]

On Thursday 21 April 2016 11:28:15 Tomasz Nowicki wrote:
> On 19.04.2016 15:06, Arnd Bergmann wrote:
> > On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
> >>
> >> Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.
> >>
> >> pci-thunder-ecam.c contains config space accessors. Similar for
> >> pci-thunder-pem.c but it also has extra init call (it is now called
> >> thunder_pem_init) which finds and maps related registers.
> >
> > They seem to do much more than just override the accessors, they actually
> > change the contents of the config space as well. Is that really necessary
> > on ACPI based systems as well?
> 
> Yes, the pci-thunder-ecam.c accessors are meant to emulate config space 
> capabilities. They are necessary to synthesize EA capabilities (fixed 
> PCI BARs), it wont work without this, for ACPI boot as well.

Why is that? I thought the BARs never get reassigned when using ACPI,
so I'm surprised it's actually needed. Maybe I misunderstood what
you mean by fixed PCI BARs.

> > Another idea: how about moving all of this logic into ACPI and calling
> > some AML method to access the config space if the devices are that
> > far out of spec.
> 
> Do you mean Linux specific way to call non-standard config space 
> accessors? Then non-standard accessors are going to AML methods which 
> are called from common code which handles quirks via unified API ?

What I really meant was a standardized way to do handle hardware that
is in some way or another not compliant with PNP0A08: We could have
a different hardware ID for this and let all the first-generation
ARM servers and also anything else using ACPI with nonstandard PCI
use the same method across operating systems.

	Arnd

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Tomasz Nowicki]

On 21.04.2016 11:36, Arnd Bergmann wrote:
> On Thursday 21 April 2016 11:28:15 Tomasz Nowicki wrote:
>> On 19.04.2016 15:06, Arnd Bergmann wrote:
>>> On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
>>>>
>>>> Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.
>>>>
>>>> pci-thunder-ecam.c contains config space accessors. Similar for
>>>> pci-thunder-pem.c but it also has extra init call (it is now called
>>>> thunder_pem_init) which finds and maps related registers.
>>>
>>> They seem to do much more than just override the accessors, they actually
>>> change the contents of the config space as well. Is that really necessary
>>> on ACPI based systems as well?
>>
>> Yes, the pci-thunder-ecam.c accessors are meant to emulate config space
>> capabilities. They are necessary to synthesize EA capabilities (fixed
>> PCI BARs), it wont work without this, for ACPI boot as well.
>
> Why is that? I thought the BARs never get reassigned when using ACPI,
> so I'm surprised it's actually needed. Maybe I misunderstood what
> you mean by fixed PCI BARs.

Yes, I meant something else. ThunderX has non-programmable PCI BAR 
addresses. So it uses PCI EA (Extended allocation) capabilities to get 
know PCI BARs addresses. But the early implementation (pass1.x) misses 
EA capabilities hence we need to emulate it in config space accessors.

Tomasz

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Jayachandran C]

On Thu, Apr 21, 2016 at 2:36 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
> On 20.04.2016 21:12, Jayachandran C wrote:
>>
>> On Fri, Apr 15, 2016 at 10:36 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
>>>
>>> This patch is going to implement generic PCI host controller for
>>> ACPI world, similar to what pci-host-generic.c driver does for DT world.
>>>
>>> All such drivers, which we have seen so far, were implemented within
>>> arch/ directory since they had some arch assumptions (x86 and ia64).
>>> However, they all are doing similar thing, so it makes sense to find
>>> some common code and abstract it into the generic driver.
>>>
>>> In order to handle PCI config space regions properly, we define new
>>> MCFG interface which parses MCFG table and keep its entries
>>> in a list. New pci_mcfg_init call is defined so that we do not depend
>>> on PCI_MMCONFIG. Regions are not mapped until host bridge ask for it.
>>>
>>> The implementation of pci_acpi_scan_root() looks up the saved MCFG
>>> entries
>>> and sets up a new mapping. Generic PCI functions are used for
>>> accessing config space. Driver selects PCI_GENERIC_ECAM and uses
>>> functions
>>> from drivers/pci/ecam.h to create and access ECAM mappings.
>>>
>>> As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
>>> should be made on a per-architecture basis.
>>>
>>> This patch is heavily based on the updated version from Jayachandran C:
>>> https://lkml.org/lkml/2016/4/11/908
>>> git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
>>
>>
>> This is a little bit unusual because I had not posted the v3 patch
>> to the mailing list yet, but you posted a variant of it The git repository
>> should not be in the commit comment because it is a temporary location.
>
>
> We all agree this too important for everybody to delay this series. So main
> motivation is to keep all discussion&patches within one unified series. I
> would like to finally find direction we need to go. Stating another
> discussion based on my previous patch set v5 confused people, they do no
> know who is driving this. Again, lets cooperate to move it forward within
> one patch set.
>
> I agree with you we need maintainers to join this discussion.
>
>>
>> There are some changes here I don't agree with. I think it will be
>> better if you can post a version without the quirk handling and with
>> some of the suggestions below.
>
>
> The next version will not have quirk handling part. Regarding your comments,
> please see below.
>
>
>>
>>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>>> ---
>>>   drivers/acpi/Kconfig        |   8 ++
>>>   drivers/acpi/Makefile       |   1 +
>>>   drivers/acpi/bus.c          |   1 +
>>>   drivers/acpi/pci_gen_host.c | 231
>>> ++++++++++++++++++++++++++++++++++++++++++++
>>>   include/linux/pci.h         |   6 ++
>>>   5 files changed, 247 insertions(+)
>>>   create mode 100644 drivers/acpi/pci_gen_host.c
>>>
>>> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
>>> index 183ffa3..70272c5 100644
>>> --- a/drivers/acpi/Kconfig
>>> +++ b/drivers/acpi/Kconfig
>>> @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
>>>            i.e., segment/bus/device/function tuples, with physical slots
>>> in
>>>            the system.  If you are unsure, say N.
>>>
>>> +config ACPI_PCI_HOST_GENERIC
>>> +       bool
>>> +       select PCI_GENERIC_ECAM
>>> +       help
>>> +         Select this config option from the architecture Kconfig,
>>> +         if it is preferred to enable ACPI PCI host controller driver
>>> which
>>> +         has no arch-specific assumptions.
>>> +
>>>   config X86_PM_TIMER
>>>          bool "Power Management Timer Support" if EXPERT
>>>          depends on X86
>>> diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
>>> index 81e5cbc..b12fa64 100644
>>> --- a/drivers/acpi/Makefile
>>> +++ b/drivers/acpi/Makefile
>>> @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) +=
>>> processor_pdc.o
>>>   acpi-y                         += ec.o
>>>   acpi-$(CONFIG_ACPI_DOCK)       += dock.o
>>>   acpi-y                         += pci_root.o pci_link.o pci_irq.o
>>> +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC)    += pci_gen_host.o
>>>   acpi-y                         += acpi_lpss.o acpi_apd.o
>>>   acpi-y                         += acpi_platform.o
>>>   acpi-y                         += acpi_pnp.o
>>> diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
>>> index c068c82..803a1d7 100644
>>> --- a/drivers/acpi/bus.c
>>> +++ b/drivers/acpi/bus.c
>>> @@ -1107,6 +1107,7 @@ static int __init acpi_init(void)
>>>          }
>>>
>>>          pci_mmcfg_late_init();
>>> +       pci_mcfg_init();
>>
>>
>> Please see below.
>>
>>>          acpi_scan_init();
>>>          acpi_ec_init();
>>>          acpi_debugfs_init();
>>> diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
>>> new file mode 100644
>>> index 0000000..fd360b5
>>> --- /dev/null
>>> +++ b/drivers/acpi/pci_gen_host.c
>>> @@ -0,0 +1,231 @@
>>> +/*
>>
>>
>> You seem to have removed the copyright line, this is not proper, you
>> should probably add your copyright line if you think your changes are
>> significant.
>
>
> I rather forgot to add copyright here, I will fix it.
>
>
>>
>>> + * This program is free software; you can redistribute it and/or modify
>>> + * it under the terms of the GNU General Public License, version 2, as
>>> + * published by the Free Software Foundation (the "GPL").
>>> + *
>>> + * This program is distributed in the hope that it will be useful, but
>>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>> + * General Public License version 2 (GPLv2) for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> + * version 2 (GPLv2) along with this source code.
>>> + */
>>> +#include <linux/kernel.h>
>>> +#include <linux/pci.h>
>>> +#include <linux/pci-acpi.h>
>>> +#include <linux/sfi_acpi.h>
>>> +#include <linux/slab.h>
>>> +
>>> +#include "../pci/ecam.h"
>>> +
>>> +#define PREFIX "ACPI: "
>>> +
>>> +/* Structure to hold entries from the MCFG table */
>>> +struct mcfg_entry {
>>> +       struct list_head        list;
>>> +       phys_addr_t             addr;
>>> +       u16                     segment;
>>> +       u8                      bus_start;
>>> +       u8                      bus_end;
>>> +};
>>> +
>>> +/* List to save mcfg entries */
>>> +static LIST_HEAD(pci_mcfg_list);
>>> +static DEFINE_MUTEX(pci_mcfg_lock);
>>
>>
>> There is no need to use a list or lock here, I had used an
>> array and that is sufficient since it is not modified after it
>> is filled initially.
>
>
> ACPI PCI driver supports hot plug/removal, I want to avoid races using this
> lock. I decided to use list because I do ECAM mapping on demand. See below
> for more details.

Yes, there is hotplug. but there is no change to the saved MCFG entries
after you create the array (or list). And, the ECAM mapping is on demand
but it also just does a lookup and does not modify the array/list, so the
locking is not needed.

There may be a locking issue for hotplug in raw_pci_read/write between
find_bus and bus->ops->read/write, but this does not solve that. My
expectation is that since both are ACPI originated this may not be
an issue.

>>
>>> +/* ACPI info for generic ACPI PCI controller */
>>> +struct acpi_pci_generic_root_info {
>>> +       struct acpi_pci_root_info       common;
>>> +       struct pci_config_window        *cfg;   /* config space mapping
>>> */
>>> +};
>>> +
>>> +/* Find the entry in mcfg list which contains range bus_start */
>>> +static struct mcfg_entry *pci_mcfg_lookup(u16 seg, u8 bus_start)
>>> +{
>>> +       struct mcfg_entry *e;
>>> +
>>> +       list_for_each_entry(e, &pci_mcfg_list, list) {
>>> +               if (e->segment == seg &&
>>> +                   e->bus_start <= bus_start && bus_start <= e->bus_end)
>>> +                       return e;
>>> +       }
>>> +
>>> +       return NULL;
>>> +}
>>> +
>>> +
>>> +/*
>>> + * Lookup the bus range for the domain in MCFG, and set up config space
>>> + * mapping.
>>> + */
>>> +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
>>> +                                      struct acpi_pci_generic_root_info
>>> *ri)
>>> +{
>>> +       u16 seg = root->segment;
>>> +       u8 bus_start = root->secondary.start;
>>> +       u8 bus_end = root->secondary.end;
>>> +       struct pci_config_window *cfg;
>>> +       struct mcfg_entry *e;
>>> +       phys_addr_t addr;
>>> +       int err = 0;
>>> +
>>> +       mutex_lock(&pci_mcfg_lock);
>>> +       e = pci_mcfg_lookup(seg, bus_start);
>>> +       if (!e) {
>>> +               addr = acpi_pci_root_get_mcfg_addr(root->device->handle);
>>
>>
>> The acpi_pci_root_get_mcfg_addr() is already called in pci_root.c, doing
>> it again here is unnecessary.
>
>
> I put it here as per Bjorn's request, see:
> https://lkml.org/lkml/2016/3/4/946
>
> If this is not valid any more, I can easily remove it.

There are multiple ways in which acpi_pci_root_add() tries get the
bus range and mcfg address for a root bus. I think Bjorn's suggestion
was to add looking up the MCFG there too, and I think it would be
better if it was done in a separate patchset.

>>
>> I think you can have a function to pick up addr, bus_start, bus_end given
>> a domain from either MCFG or using _CBA method, but I think that
>> should be done in pci_root.c in a separate patch.
>>
>>> +               if (addr == 0) {
>>> +                       pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
>>> +                              seg, bus_start, bus_end);
>>> +                       err = -ENOENT;
>>> +                       goto err_out;
>>> +               }
>>> +       } else {
>>> +               if (bus_start != e->bus_start) {
>>> +                       pr_err("%04x:%02x-%02x bus range mismatch
>>> %02x\n",
>>> +                              seg, bus_start, bus_end, e->bus_start);
>>> +                       err = -EINVAL;
>>> +                       goto err_out;
>>> +               } else if (bus_end != e->bus_end) {
>>> +                       pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
>>> +                               seg, bus_start, bus_end, e->bus_end);
>>> +                       bus_end = min(bus_end, e->bus_end);
>>> +               }
>>> +               addr = e->addr;
>>> +       }
>>> +
>>> +       cfg = pci_generic_ecam_create(&root->device->dev, addr,
>>> bus_start,
>>> +                                     bus_end,
>>> &pci_generic_ecam_default_ops);
>>> +       if (IS_ERR(cfg)) {
>>> +               err = PTR_ERR(cfg);
>>> +               pr_err("%04x:%02x-%02x error %d mapping CAM\n", seg,
>>> +                       bus_start, bus_end, err);
>>> +               goto err_out;
>>> +       }
>>
>>
>> You seem to have moved all the config space mapping to this
>> point. Intel seems to do the mapping when they read MCFG for the
>> entries, and I had followed that model, and that avoids having another
>> array/list to save the values.
>
>
> See:
> https://lkml.org/lkml/2016/3/3/921
>
> I agree with Bjorn here, we should map it whenever we need this instead of
> mapping all MCFG entries just in case. Also, I see other advantages:
> 1. We can always use valid "dev" for pci_generic_ecam_create call, what you
> can't do when you use pci_generic_ecam_create during MCFG parsing.
> 2. No need for special handling for entries coming from _CBA, the path is
> the same for MCFG and _CBA. We do not have to remember that only _CBA
> related entries have to be unmapped.
>
>
>>
>>> +       cfg->domain = seg;
>>> +       ri->cfg = cfg;
>>> +err_out:
>>> +       mutex_unlock(&pci_mcfg_lock);
>>> +       return err;
>>> +}
>>> +
>>> +/* release_info: free resrouces allocated by init_info */
>>> +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
>>> +{
>>> +       struct acpi_pci_generic_root_info *ri;
>>> +
>>> +       ri = container_of(ci, struct acpi_pci_generic_root_info, common);
>>> +       pci_generic_ecam_free(ri->cfg);
>>> +       kfree(ri);
>>> +}
>>> +
>>> +static struct acpi_pci_root_ops acpi_pci_root_ops = {
>>> +       .release_info = pci_acpi_generic_release_info,
>>> +};
>>> +
>>> +/* Interface called from ACPI code to setup PCI host controller */
>>> +struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>>> +{
>>> +       int node = acpi_get_node(root->device->handle);
>>> +       struct acpi_pci_generic_root_info *ri;
>>> +       struct pci_bus *bus, *child;
>>> +       int err;
>>> +
>>> +       ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
>>> +       if (!ri)
>>> +               return NULL;
>>> +
>>> +       err = pci_acpi_setup_ecam_mapping(root, ri);
>>> +       if (err)
>>> +               return NULL;
>>> +
>>> +       acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
>>> +       bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
>>> +                                  ri->cfg);
>>> +       if (!bus)
>>> +               return NULL;
>>> +
>>> +       pci_bus_size_bridges(bus);
>>> +       pci_bus_assign_resources(bus);
>>> +
>>> +       list_for_each_entry(child, &bus->children, node)
>>> +               pcie_bus_configure_settings(child);
>>> +
>>> +       return bus;
>>> +}
>>> +
>>> +/* handle MCFG table entries */
>>> +static __init int pci_mcfg_parse(struct acpi_table_header *header)
>>> +{
>>> +       struct acpi_table_mcfg *mcfg;
>>> +       struct acpi_mcfg_allocation *mptr;
>>> +       struct mcfg_entry *e, *arr;
>>> +       int i, n;
>>> +
>>> +       if (!header)
>>> +               return -EINVAL;
>>> +
>>> +       mcfg = (struct acpi_table_mcfg *)header;
>>> +       mptr = (struct acpi_mcfg_allocation *) &mcfg[1];
>>> +       n = (header->length - sizeof(*mcfg)) / sizeof(*mptr);
>>> +       if (n <= 0 || n > 255) {
>>> +               pr_err(PREFIX " MCFG has incorrect entries (%d).\n", n);
>>> +               return -EINVAL;
>>> +       }
>>> +
>>> +       arr = kcalloc(n, sizeof(*arr), GFP_KERNEL);
>>> +       if (!arr)
>>> +               return -ENOMEM;
>>
>>
>> Here you already have an array which is also connected as a linked
>> list which is unnecessary.
>
>
> The array is to avoid complicated error handling in case the allocation for
> some of element would failed. I can rework this to allocate each element
> separately or to use array but the code is simpler now. As explained above
> (on demand mapping), I would like to keep list/array with MCFG entries (I
> preferred list).
>
>
>>
>>> +       for (i = 0, e = arr; i < n; i++, mptr++, e++) {
>>> +               e->segment = mptr->pci_segment;
>>> +               e->addr =  mptr->address;
>>> +               e->bus_start = mptr->start_bus_number;
>>> +               e->bus_end = mptr->end_bus_number;
>>> +               list_add(&e->list, &pci_mcfg_list);
>>> +               pr_info(PREFIX
>>> +                       "MCFG entry for domain %04x [bus %02x-%02x] (base
>>> %pa)\n",
>>> +                       e->segment, e->bus_start, e->bus_end, &e->addr);
>>> +       }
>>> +
>>> +       return 0;
>>> +}
>>> +
>>> +/* Interface called by ACPI - parse and save MCFG table */
>>> +void __init pci_mcfg_init(void)
>>> +{
>>> +       int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
>>> +       if (err)
>>> +               pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);
>>> +       else if (list_empty(&pci_mcfg_list))
>>> +               pr_info(PREFIX "No valid entries in MCFG table.\n");
>>> +       else {
>>> +               struct mcfg_entry *e;
>>> +               int i = 0;
>>> +               list_for_each_entry(e, &pci_mcfg_list, list)
>>> +                       i++;
>>> +               pr_info(PREFIX "MCFG table loaded, %d entries\n", i);
>>> +       }
>>> +}
>>> +
>>> +/* Raw operations, works only for MCFG entries with an associated bus */
>>> +int raw_pci_read(unsigned int domain, unsigned int busn, unsigned int
>>> devfn,
>>> +                int reg, int len, u32 *val)
>>> +{
>>> +       struct pci_bus *bus = pci_find_bus(domain, busn);
>>> +
>>> +       if (!bus)
>>> +               return PCIBIOS_DEVICE_NOT_FOUND;
>>> +       return bus->ops->read(bus, devfn, reg, len, val);
>>> +}
>>> +
>>> +int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int
>>> devfn,
>>> +                 int reg, int len, u32 val)
>>> +{
>>> +       struct pci_bus *bus = pci_find_bus(domain, busn);
>>> +
>>> +       if (!bus)
>>> +               return PCIBIOS_DEVICE_NOT_FOUND;
>>> +       return bus->ops->write(bus, devfn, reg, len, val);
>>> +}
>>> diff --git a/include/linux/pci.h b/include/linux/pci.h
>>> index df1f33d..c0422ea 100644
>>> --- a/include/linux/pci.h
>>> +++ b/include/linux/pci.h
>>> @@ -1729,6 +1729,12 @@ static inline void pci_mmcfg_early_init(void) { }
>>>   static inline void pci_mmcfg_late_init(void) { }
>>>   #endif
>>>
>>> +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
>>> +void __init pci_mcfg_init(void);
>>> +#else
>>> +static inline void pci_mcfg_init(void) { return; }
>>> +#endif
>>
>>
>> You can still use the function pci_mmcfg_late_init() if
>> PCI_MMCONFIG or ACPI_PCI_HOST_GENERIC is defined
>>
>
> OK
>
> -#ifdef CONFIG_PCI_MMCONFIG
> +#if defined(CONFIG_PCI_MMCONFIG) || defined(ACPI_PCI_HOST_GENERIC)
> void __init pci_mmcfg_early_init(void);
> void __init pci_mmcfg_late_init(void);
> #else
> static inline void pci_mmcfg_early_init(void) { }
> static inline void pci_mmcfg_late_init(void) { }
> #endif
>
> is that what you mean?

Yes - this should be sufficient instead of a new function...

JC.

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Jon Masters]

On 04/21/2016 06:08 AM, Tomasz Nowicki wrote:
> On 21.04.2016 11:36, Arnd Bergmann wrote:
>> On Thursday 21 April 2016 11:28:15 Tomasz Nowicki wrote:
>>> On 19.04.2016 15:06, Arnd Bergmann wrote:
>>>> On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
>>>>>
>>>>> Basically the whole content of pci-thunder-ecam.c and
>>>>> pci-thunder-pem.c.
>>>>>
>>>>> pci-thunder-ecam.c contains config space accessors. Similar for
>>>>> pci-thunder-pem.c but it also has extra init call (it is now called
>>>>> thunder_pem_init) which finds and maps related registers.
>>>>
>>>> They seem to do much more than just override the accessors, they
>>>> actually
>>>> change the contents of the config space as well. Is that really
>>>> necessary
>>>> on ACPI based systems as well?
>>>
>>> Yes, the pci-thunder-ecam.c accessors are meant to emulate config space
>>> capabilities. They are necessary to synthesize EA capabilities (fixed
>>> PCI BARs), it wont work without this, for ACPI boot as well.
>>
>> Why is that? I thought the BARs never get reassigned when using ACPI,
>> so I'm surprised it's actually needed. Maybe I misunderstood what
>> you mean by fixed PCI BARs.
> 
> Yes, I meant something else. ThunderX has non-programmable PCI BAR
> addresses. So it uses PCI EA (Extended allocation) capabilities to get
> know PCI BARs addresses. But the early implementation (pass1.x) misses
> EA capabilities hence we need to emulate it in config space accessors.

Aside: In case it's helpful, at least one enterprise vendor I know of is
only supporting later silicon as a result of this. So IMO there's no
need to worry about this issue on the early preproduction chips.

-- 
Computer Architect | Sent from my Fedora powered laptop

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Jon Masters]

On 04/21/2016 05:06 AM, Tomasz Nowicki wrote:
> On 20.04.2016 21:12, Jayachandran C wrote:
>> On Fri, Apr 15, 2016 at 10:36 PM, Tomasz Nowicki <tn@semihalf.com> wrote:

>>> This patch is heavily based on the updated version from Jayachandran C:
>>> https://lkml.org/lkml/2016/4/11/908
>>> git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
>>
>> This is a little bit unusual because I had not posted the v3 patch
>> to the mailing list yet, but you posted a variant of it The git
>> repository
>> should not be in the commit comment because it is a temporary location.
> 
> We all agree this too important for everybody to delay this series. So
> main motivation is to keep all discussion&patches within one unified
> series. I would like to finally find direction we need to go. Stating
> another discussion based on my previous patch set v5 confused people,
> they do no know who is driving this. Again, lets cooperate to move it
> forward within one patch set.

We need one person in the driver's seat here for this patch series. I
believe the intention is that this is Tomasz, with others cooperating
and assisting. The previous alternative patch series did serve to cause
confusion, and worse, they made it look like the ARM vendors can't work
together. That ends. Right now. I've raised this individually with each
of you (and with all of the other vendors), as well as inside Linaro.
There will be one person driving this, and everyone else will help.

> I agree with you we need maintainers to join this discussion.

Please. I really want to lend support to the sense of urgency of getting
something merged that provides the basic ACPI based ECAM functionality
on ARMv8 systems. Until that happens, there are vendors who are going to
have to delay various other activities around ARM server until it is
done (because of the lack of an upstream patch is a critical blocker -
this isn't embedded, and we don't work that way).

This is incredibly critical and central to the successful adoption of
larger ARMv8 servers over the next year or so, all of which will rely
upon PCIe, using ACPI. We therefore need all hands on deck, and all
vendors getting overwhelmingly excited about making this patch series a
success. As an aside, to help the engineering orgs within these vendors
understand how much I need motion on this thread, I'm buzzing in their
ears multiple times per week specifically on engagement on this thread.
We need this done. If Bjorn or anyone else needs something, tell us.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[David Daney]

On 04/22/2016 07:30 AM, Jon Masters wrote:
> On 04/21/2016 06:08 AM, Tomasz Nowicki wrote:
>> On 21.04.2016 11:36, Arnd Bergmann wrote:
>>> On Thursday 21 April 2016 11:28:15 Tomasz Nowicki wrote:
>>>> On 19.04.2016 15:06, Arnd Bergmann wrote:
>>>>> On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
>>>>>>
>>>>>> Basically the whole content of pci-thunder-ecam.c and
>>>>>> pci-thunder-pem.c.
>>>>>>
>>>>>> pci-thunder-ecam.c contains config space accessors. Similar for
>>>>>> pci-thunder-pem.c but it also has extra init call (it is now called
>>>>>> thunder_pem_init) which finds and maps related registers.
>>>>>
>>>>> They seem to do much more than just override the accessors, they
>>>>> actually
>>>>> change the contents of the config space as well. Is that really
>>>>> necessary
>>>>> on ACPI based systems as well?
>>>>
>>>> Yes, the pci-thunder-ecam.c accessors are meant to emulate config space
>>>> capabilities. They are necessary to synthesize EA capabilities (fixed
>>>> PCI BARs), it wont work without this, for ACPI boot as well.
>>>
>>> Why is that? I thought the BARs never get reassigned when using ACPI,
>>> so I'm surprised it's actually needed. Maybe I misunderstood what
>>> you mean by fixed PCI BARs.
>>
>> Yes, I meant something else. ThunderX has non-programmable PCI BAR
>> addresses. So it uses PCI EA (Extended allocation) capabilities to get
>> know PCI BARs addresses. But the early implementation (pass1.x) misses
>> EA capabilities hence we need to emulate it in config space accessors.
>
> Aside: In case it's helpful, at least one enterprise vendor I know of is
> only supporting later silicon as a result of this. So IMO there's no
> need to worry about this issue on the early preproduction chips.
>

There are two separate issues that make fixing up the ECAM space necessary:

1) As Jon mentioned, preproduction silicon lacks EA capabilities.

2) On 2-node NUMA systems, the EA capabilities of some devices may be 
incorrect even in production silicon.

In general, the strategy we use for dealing with both of these is to 
hook into the ECAM access methods, and supply corrected config space 
data.  For the case of device-tree provisioned ECAM access, the fix ups 
are done in pci-thunder-ecam.c.  This code is already present and seems 
to be working well.

As we consider ACPI support, supporting case #2 above will be desirable. 
  If we reuse the code in pci-thunder-ecam.c for this, we will probably 
get support for #1 for free.

David Daney

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Robert Richter]

On 15.04.16 19:06:35, Tomasz Nowicki wrote:
> From the functionality point of view this series might be split into the
> following logic parts:
> 1. Necessary fixes as the preparation for using driver on ARM64.
> 2. New ECAM API and update for users of the pci-host-common API
> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
> 4. Enable above driver on ARM64
> 
> Patches has been built on top of 4.6-rc2 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
> 
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.

For the whole series:

Tested-by: Robert Richter <rrichter@cavium.com>
Acked-by: Robert Richter <rrichter@cavium.com>

-Robert

> v5 -> v6
> - dropped idea of x86 MMCONFIG code refactoring
> - integrated JC's patches which introduce new ECAM API:
>   https://lkml.org/lkml/2016/4/11/907
>   git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
> - added ACPI support for ThunderX ECAM and PEM drivers
> - rebased to 4.6-rc2
> 
> v4 -> v5
> - dropped MCFG refactoring group patches 1-6 from series v4 and integrated Jayachandran's patch
>   https://patchwork.ozlabs.org/patch/575525/
> - rewrite PCI legacy IRQs allocation
> - squashed two patches 11 and 12 from series v4, fixed bisection issue
> - changelog improvements
> - rebased to 4.5-rc3
> 
> v3 -> v4
> - dropped Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> - added Lorenzo's fix patch 19/24
> - ACPI PCI bus domain number assigning cleanup
> - changed resource management, we now claim and reassign resources
> - improvements for applying quirks
> - dropped Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
> - rebased to 4.5-rc1
> 
> v2 -> v3
> - fix legacy IRQ assigning and IO ports registration
> - remove reference to arch specific companion device for ia64
> - move ACPI PCI host controller driver to pci_root.c
> - drop generic domain assignment for x86 and ia64 as I am not
>   able to run all necessary test variants
> - drop patch which cleaned legacy IRQ assignment since it belongs to
>   Mathew's series:
>   https://patchwork.ozlabs.org/patch/557504/
> - extend MCFG quirk code
> - rebased to 4.4
> 
> v1 -> v2
> - moved non-arch specific piece of code to dirver/acpi/ directory
> - fixed IO resource handling
> - introduced PCI config accessors quirks matching
> - moved ACPI_COMPANION_SET to generic code
> 
> v1 - https://lkml.org/lkml/2015/10/27/504
> v2 - https://lkml.org/lkml/2015/12/16/246
> v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
> v4 - https://lkml.org/lkml/2016/2/4/646
> v5 - https://lkml.org/lkml/2016/2/16/426

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Suravee Suthikulpanit]

On 04/15/2016 12:06 PM, Tomasz Nowicki wrote:
>  From the functionality point of view this series might be split into the
> following logic parts:
> 1. Necessary fixes as the preparation for using driver on ARM64.
> 2. New ECAM API and update for users of the pci-host-common API
> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
> 4. Enable above driver on ARM64
>
> Patches has been built on top of 4.6-rc2 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
>
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.

For the whole series (On AMD Seattle):

Tested-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>

Thanks,
Suravee

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Jon Masters]

On 04/22/2016 10:40 AM, Jon Masters wrote:
> On 04/21/2016 05:06 AM, Tomasz Nowicki wrote:
>> On 20.04.2016 21:12, Jayachandran C wrote:
>>> On Fri, Apr 15, 2016 at 10:36 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
> 
>>>> This patch is heavily based on the updated version from Jayachandran C:
>>>> https://lkml.org/lkml/2016/4/11/908
>>>> git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
>>>
>>> This is a little bit unusual because I had not posted the v3 patch
>>> to the mailing list yet, but you posted a variant of it The git
>>> repository
>>> should not be in the commit comment because it is a temporary location.
>>
>> We all agree this too important for everybody to delay this series. So
>> main motivation is to keep all discussion&patches within one unified
>> series. I would like to finally find direction we need to go. Stating
>> another discussion based on my previous patch set v5 confused people,
>> they do no know who is driving this. Again, lets cooperate to move it
>> forward within one patch set.
> 
> We need one person in the driver's seat here for this patch series. I
> believe the intention is that this is Tomasz, with others cooperating
> and assisting. The previous alternative patch series did serve to cause
> confusion, and worse, they made it look like the ARM vendors can't work
> together. That ends. Right now. I've raised this individually with each
> of you (and with all of the other vendors), as well as inside Linaro.
> There will be one person driving this, and everyone else will help.

As a quick update, since yesterday I have confirmed that several
different microarchitecture implementations (different PCIe) have tested
and validated this patch series. Those minimally include:

* Cavium Networks ThunderX
* Qualcomm Technologies Inc QDF2XXX
* AMD A1100 ("Seattle")

Another is working on testing over the weekend. Still waiting for an ARM
tested-by on Juno I think. I will personally be testing this and future
releases on all of the above mentioned hw.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[Jeremy Linton]

On 04/15/2016 12:06 PM, Tomasz Nowicki wrote:
>  From the functionality point of view this series might be split into the
> following logic parts:
> 1. Necessary fixes as the preparation for using driver on ARM64.
> 2. New ECAM API and update for users of the pci-host-common API
> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
> 4. Enable above driver on ARM64
>
> Patches has been built on top of 4.6-rc2 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
>
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.

I did some basic testing of this series on ARM's JunoR2 platform. 
Everything seems to work as expected, the on-board SATA/Ethernet work 
correctly as do a couple of boards plugged into the slots.

Tested-by: Jeremy Linton <jeremy.linton@arm.com>

Thanks!


>
> v5 -> v6
> - dropped idea of x86 MMCONFIG code refactoring
> - integrated JC's patches which introduce new ECAM API:
>    https://lkml.org/lkml/2016/4/11/907
>    git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
> - added ACPI support for ThunderX ECAM and PEM drivers
> - rebased to 4.6-rc2
>
> v4 -> v5
> - dropped MCFG refactoring group patches 1-6 from series v4 and integrated Jayachandran's patch
>    https://patchwork.ozlabs.org/patch/575525/
> - rewrite PCI legacy IRQs allocation
> - squashed two patches 11 and 12 from series v4, fixed bisection issue
> - changelog improvements
> - rebased to 4.5-rc3
>
> v3 -> v4
> - dropped Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> - added Lorenzo's fix patch 19/24
> - ACPI PCI bus domain number assigning cleanup
> - changed resource management, we now claim and reassign resources
> - improvements for applying quirks
> - dropped Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
> - rebased to 4.5-rc1
>
> v2 -> v3
> - fix legacy IRQ assigning and IO ports registration
> - remove reference to arch specific companion device for ia64
> - move ACPI PCI host controller driver to pci_root.c
> - drop generic domain assignment for x86 and ia64 as I am not
>    able to run all necessary test variants
> - drop patch which cleaned legacy IRQ assignment since it belongs to
>    Mathew's series:
>    https://patchwork.ozlabs.org/patch/557504/
> - extend MCFG quirk code
> - rebased to 4.4
>
> v1 -> v2
> - moved non-arch specific piece of code to dirver/acpi/ directory
> - fixed IO resource handling
> - introduced PCI config accessors quirks matching
> - moved ACPI_COMPANION_SET to generic code
>
> v1 - https://lkml.org/lkml/2015/10/27/504
> v2 - https://lkml.org/lkml/2015/12/16/246
> v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
> v4 - https://lkml.org/lkml/2016/2/4/646
> v5 - https://lkml.org/lkml/2016/2/16/426
>
> Jayachandran C (2):
>    PCI: Provide common functions for ECAM mapping
>    PCI: generic, thunder: update to use generic ECAM API
>
> Tomasz Nowicki (11):
>    pci, acpi, x86, ia64: Move ACPI host bridge device companion
>      assignment to core code.
>    pci, acpi: Provide generic way to assign bus domain number.
>    x86, ia64: Include acpi_pci_{add|remove}_bus to the default
>      pcibios_{add|remove}_bus implementation.
>    pci, of: Move the PCI I/O space management to PCI core code.
>    acpi, pci: Support IO resources when parsing PCI host bridge
>      resources.
>    arm64, pci, acpi: ACPI support for legacy IRQs parsing and
>      consolidation with DT code.
>    pci, acpi: Support for ACPI based generic PCI host controller
>    arm64, pci, acpi: Start using ACPI based PCI host controller driver
>      for ARM64.
>    pci, acpi: Match PCI config space accessors against platfrom specific
>      quirks.
>    pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.
>    pci, pci-thunder-pem: Add ACPI support for ThunderX PEM.
>
>   arch/arm64/Kconfig                  |  15 +++
>   arch/arm64/include/asm/cpufeature.h |   3 +-
>   arch/arm64/kernel/cpu_errata.c      |   8 ++
>   arch/arm64/kernel/pci.c             |  35 ++---
>   arch/ia64/hp/common/sba_iommu.c     |   2 +-
>   arch/ia64/include/asm/pci.h         |   1 -
>   arch/ia64/pci/pci.c                 |  26 ----
>   arch/ia64/sn/kernel/io_acpi_init.c  |   4 +-
>   arch/x86/include/asm/pci.h          |   3 -
>   arch/x86/pci/acpi.c                 |  17 ---
>   arch/x86/pci/common.c               |  10 --
>   drivers/acpi/Kconfig                |   8 ++
>   drivers/acpi/Makefile               |   1 +
>   drivers/acpi/bus.c                  |   1 +
>   drivers/acpi/pci_gen_host.c         | 259 ++++++++++++++++++++++++++++++++++++
>   drivers/acpi/pci_root.c             |  58 +++++++-
>   drivers/of/address.c                | 116 +---------------
>   drivers/pci/Kconfig                 |   3 +
>   drivers/pci/Makefile                |   2 +
>   drivers/pci/ecam.c                  | 137 +++++++++++++++++++
>   drivers/pci/ecam.h                  |  66 +++++++++
>   drivers/pci/host/Kconfig            |   1 +
>   drivers/pci/host/pci-host-common.c  | 119 ++++++++---------
>   drivers/pci/host/pci-host-common.h  |  47 -------
>   drivers/pci/host/pci-host-generic.c |  52 ++------
>   drivers/pci/host/pci-thunder-ecam.c |  70 ++++++----
>   drivers/pci/host/pci-thunder-pem.c  | 215 ++++++++++++++++++++++--------
>   drivers/pci/pci.c                   | 150 ++++++++++++++++++++-
>   drivers/pci/probe.c                 |   5 +
>   include/asm-generic/vmlinux.lds.h   |   7 +
>   include/linux/of_address.h          |   9 --
>   include/linux/pci-acpi.h            |  20 +++
>   include/linux/pci.h                 |  12 ++
>   33 files changed, 1029 insertions(+), 453 deletions(-)
>   create mode 100644 drivers/acpi/pci_gen_host.c
>   create mode 100644 drivers/pci/ecam.c
>   create mode 100644 drivers/pci/ecam.h
>   delete mode 100644 drivers/pci/host/pci-host-common.h
>

Re: [PATCH V6 00/13] Support for generic ACPI based PCI host controller

[liudongdong (C)]

Based on the patchset and add the patch(Add ACPI support for HiSilicon PCIe Host Controllers).
Tested on the HiSilicon ARM64 D02 board.
it can work ok with Intel 82599 networking card.
This is the bootup log which contains PCIe host and Intel 82599 networking card part.

Tested-by: Dongdong Liu <liudongdong3@huawei.com>

Loading driver at 0x0006D771000 EntryPoint=0x0006E17E100
Loading driver at 0x0006D771000 EntryPoint=0x0006E17E100
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
GMAC ExitBootServicesEvent
SMMU ExitBootServicesEvent
[    0.000000] Booting Linux on physical CPU 0x20000
[    0.000000] Linux version 4.6.0-rc1+ (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #27 SMP PREEMPT Tue Apr 26 16:24:06 CST 2016
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000080300000 (options '')
[    0.000000] bootconsole [uart8250] enabled
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] EFI v2.50 by EDK II
[    0.000000] efi:  SMBIOS=0x7a650000  SMBIOS 3.0=0x7a630000  ACPI=0x7aba0000  ACPI 2.0=0x7aba0014
[    0.000000] cma: Reserved 16 MiB at 0x000000007e800000
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x000000007ABA0014 000024 (v02 HISI  )
[    0.000000] ACPI: XSDT 0x000000007A7000E8 00005C (v01 HISI   HISI-D02 20140727      01000013)
[    0.000000] ACPI: FACP 0x000000007A5F0000 00010C (v05 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: DSDT 0x000000007A5A0000 001694 (v01 HISI   HISI-D02 20140727 INTL 20150619)
[    0.000000] ACPI: DBG2 0x000000007A610000 00005A (v00 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: GTDT 0x000000007A5E0000 000060 (v02 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: APIC 0x000000007A5D0000 000564 (v01 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: MCFG 0x000000007A5C0000 00004C (v01 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: SPCR 0x000000007A5B0000 000050 (v02 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: IORT 0x000000007A590000 0001FC (v00 INTEL  TEMPLATE 00000000 INTL 20150619)
[    0.000000] ACPI: SPCR: console: uart,mmio,0x80300000,115200
[    0.000000] psci: probing for conduit method from ACPI.
NOTICE:  [psci_smc_handler]:[347L] PSCI_VERSION CALL
NOTICE:  [psci_version]:[99L] PSCI_MAJOR_VER: 10000: PSCI_MINOR_VER: 0

0808?844
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs

0808?844
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.

0808?844

0808?844
[    0.000000] percpu: Embedded 20 pages/cpu @ffffffd1ffe7e000 s43008 r8192 d30720 u81920
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] CPU features: enabling workaround for ARM erratum 834220
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2063376
[    0.000000] Kernel command line: console=ttyS0,115200 earlycon=uart8250,mmio32,0x80300000 initrd=filesystem.cpio.gz acpi=force pcie_aspm=off
[    0.000000] PCIe ASPM is disabled
[    0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 61440 bytes
[    0.000000] log_buf_len min size: 16384 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 13088(79%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] software IO TLB [mem 0x764f0000-0x7a4f0000] (64MB) mapped at [ffffffc0764f0000-ffffffc07a4effff]
[    0.000000] Memory: 8110360K/8384512K available (7240K kernel code, 632K rwdata, 3028K rodata, 840K init, 247K bss, 257768K reserved, 16384K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbdbfff0000   (   246 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008790000   (  7232 KB)
[    0.000000]     .rodata : 0xffffff8008790000 - 0xffffff8008a89000   (  3044 KB)
[    0.000000]       .init : 0xffffff8008a89000 - 0xffffff8008b5b000   (   840 KB)
[    0.000000]       .data : 0xffffff8008b5b000 - 0xffffff8008bf9200   (   633 KB)
[    0.000000]     vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000   (     8 GB maximum)
[    0.000000]               0xffffffbdc0000000 - 0xffffffbe08000000   (  1152 MB actual)
[    0.000000]     fixed   : 0xffffffbffe7fd000 - 0xffffffbffec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbffee00000 - 0xffffffbfffe00000   (    16 MB)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffd200000000   ( 73728 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=16.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=16
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] ITS@0x8c000000
[    0.000000] ITS: allocated 65536 Devices @11f6c80000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Virtual CPUs @11f6c0c000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Interrupt Collections @11f6c0d000 (psz 4K, shr 1)
[    0.000000] ITS@0xc6000000
[    0.000000] ITS: allocated 65536 Devices @11f6d00000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Virtual CPUs @11f6c0e000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Interrupt Collections @11f6c0f000 (psz 4K, shr 1)
[    0.000000] ITS@0xa3000000
[    0.000000] ITS: allocated 65536 Devices @11f6d80000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Virtual CPUs @11f6c31000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Interrupt Collections @11f6c32000 (psz 4K, shr 1)
[    0.000000] ITS@0xb7000000
[    0.000000] ITS: allocated 65536 Devices @11f6e00000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Virtual CPUs @11f6c33000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Interrupt Collections @11f6c34000 (psz 4K, shr 1)
[    0.000000] GIC: using LPI property table @0x00000011f6c60000
[    0.000000] ITS: Allocated 1792 chunks for LPIs
[    0.000000] CPU0: found redistributor 20000 region 0:0x000000008d100000
[    0.000000] CPU0: using LPI pending table @0x00000011f6c70000
[    0.000000] Unable to get hardware information used for virtualization
[    0.000000] GTDT: No Platform Timer structures.
[    0.000000] arch_timer: Can't find GT Block.
[    0.000000] Architected cp15 and mmio timer(s) running at 50.00MHz (phys/phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[    0.000001] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
[    0.008027] Console: colour dummy device 80x25
[    0.012461] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000)
[    0.022853] pid_max: default: 32768 minimum: 301
[    0.027463] ACPI: Core revision 20160108
[    0.032014] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    0.038345]
[    0.039856] Security Framework initialized
[    0.043942] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.050782] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.058482] ASID allocator initialised with 65536 entries
[    0.064125] PCI/MSI: ITS@0x8c000000 domain created
[    0.068894] PCI/MSI: ITS@0xc6000000 domain created
[    0.073660] PCI/MSI: ITS@0xa3000000 domain created
[    0.078425] PCI/MSI: ITS@0xb7000000 domain created
[    0.083196] Platform MSI: irqchip@000000008c000000 domain created
[    0.089258] Platform MSI: irqchip@00000000c6000000 domain created
[    0.095319] Platform MSI: irqchip@00000000a3000000 domain created
[    0.101381] Platform MSI: irqchip@00000000b7000000 domain created
[    0.107485] Remapping and enabling EFI services.
[    0.112101]   EFI remap 0x000000007a4f0000 => 0000000020000000
[    0.117917]   EFI remap 0x000000007a540000 => 0000000020050000
[    0.123740]   EFI remap 0x000000007a620000 => 00000000200a0000
[    0.129555]   EFI remap 0x000000007a6b0000 => 0000000020130000
[    0.135369]   EFI remap 0x000000007a710000 => 0000000020180000
[    0.141187]   EFI remap 0x000000007a760000 => 00000000201d0000
[    0.147002]   EFI remap 0x000000007a7b0000 => 0000000020220000
[    0.152816]   EFI remap 0x000000007a800000 => 0000000020270000
[    0.158630]   EFI remap 0x000000007a850000 => 00000000202c0000
[    0.164445]   EFI remap 0x000000007a8a0000 => 0000000020310000
[    0.170259]   EFI remap 0x000000007a8f0000 => 0000000020360000
[    0.176073]   EFI remap 0x000000007a940000 => 00000000203b0000
[    0.181904]   EFI remap 0x000000007a990000 => 0000000020400000
[    0.187718]   EFI remap 0x000000007aa00000 => 0000000020470000
[    0.193533]   EFI remap 0x000000007aa50000 => 00000000204c0000
[    0.199347]   EFI remap 0x000000007aaa0000 => 0000000020510000
[    0.205162]   EFI remap 0x000000007aaf0000 => 0000000020560000
[    0.210976]   EFI remap 0x000000007ab40000 => 00000000205b0000
[    0.216789]   EFI remap 0x000000007fbb0000 => 0000000020600000
[    0.222595]   EFI remap 0x0000000080300000 => 0000000020630000
[    0.228409]   EFI remap 0x00000000a00f0000 => 0000000020640000
[    0.234213]   EFI remap 0x00000000a4000000 => 0000000020800000
[    0.240022]   EFI remap 0x00000000a6000000 => 0000000021800000
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20001 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c080
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d190
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20002 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?8A4B4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c100
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d3a0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20003 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c180
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d5b0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20100 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c200
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d7c0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20101 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c280
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d9d0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20102 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c300
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3dbe0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20103 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c380
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ddf0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20200 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c400
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e000
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20201 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c480
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e210
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20202 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c500
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e420
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20203 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c580
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e630
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20300 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c600
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e840
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20301 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c680
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ea50
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20302 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c700
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ec60
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20303 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?84AB4
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c780
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ee70
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
[    0.287781] Detected PIPT I-cache on CPU1
[    0.287793] CPU1: found redistributor 20001 region 0:0x000000008d130000
[    0.287813] CPU1: using LPI pending table @0x00000011f6410000
[    0.287871] CPU1: Booted secondary processor [411fd071]
[    0.330893] Detected PIPT I-cache on CPU2
[    0.330899] CPU2: found redistributor 20002 region 0:0x000000008d160000
[    0.330919] CPU2: using LPI pending table @0x00000011f6440000
[    0.330966] CPU2: Booted secondary processor [411fd071]
[    0.374006] Detected PIPT I-cache on CPU3
[    0.374013] CPU3: found redistributor 20003 region 0:0x000000008d190000
[    0.374033] CPU3: using LPI pending table @0x00000011f6480000
[    0.374077] CPU3: Booted secondary processor [411fd071]
[    0.417121] Detected PIPT I-cache on CPU4
[    0.417129] CPU4: found redistributor 20100 region 0:0x000000008d1c0000
[    0.417149] CPU4: using LPI pending table @0x00000011f64c0000
[    0.417197] CPU4: Booted secondary processor [411fd071]
[    0.460234] Detected PIPT I-cache on CPU5
[    0.460241] CPU5: found redistributor 20101 region 0:0x000000008d1f0000
[    0.460260] CPU5: using LPI pending table @0x00000011f64f0000
[    0.460307] CPU5: Booted secondary processor [411fd071]
[    0.503347] Detected PIPT I-cache on CPU6
[    0.503354] CPU6: found redistributor 20102 region 0:0x000000008d220000
[    0.503374] CPU6: using LPI pending table @0x00000011f6530000
[    0.503419] CPU6: Booted secondary processor [411fd071]
[    0.546461] Detected PIPT I-cache on CPU7
[    0.546468] CPU7: found redistributor 20103 region 0:0x000000008d250000
[    0.546488] CPU7: using LPI pending table @0x00000011f6560000
[    0.546533] CPU7: Booted secondary processor [411fd071]
[    0.589576] Detected PIPT I-cache on CPU8
[    0.589585] CPU8: found redistributor 20200 region 0:0x000000008d280000
[    0.589606] CPU8: using LPI pending table @0x00000011f65a0000
[    0.589657] CPU8: Booted secondary processor [411fd071]
[    0.632688] Detected PIPT I-cache on CPU9
[    0.632695] CPU9: found redistributor 20201 region 0:0x000000008d2b0000
[    0.632716] CPU9: using LPI pending table @0x00000011f65e0000
[    0.632763] CPU9: Booted secondary processor [411fd071]
[    0.675802] Detected PIPT I-cache on CPU10
[    0.675809] CPU10: found redistributor 20202 region 0:0x000000008d2e0000
[    0.675830] CPU10: using LPI pending table @0x00000011f6610000
[    0.675875] CPU10: Booted secondary processor [411fd071]
[    0.718915] Detected PIPT I-cache on CPU11
[    0.718922] CPU11: found redistributor 20203 region 0:0x000000008d310000
[    0.718943] CPU11: using LPI pending table @0x00000011f6650000
[    0.718988] CPU11: Booted secondary processor [411fd071]
[    0.762029] Detected PIPT I-cache on CPU12
[    0.762038] CPU12: found redistributor 20300 region 0:0x000000008d340000
[    0.762060] CPU12: using LPI pending table @0x00000011f6680000
[    0.762110] CPU12: Booted secondary processor [411fd071]
[    0.805142] Detected PIPT I-cache on CPU13
[    0.805149] CPU13: found redistributor 20301 region 0:0x000000008d370000
[    0.805169] CPU13: using LPI pending table @0x00000011f66c0000
[    0.805215] CPU13: Booted secondary processor [411fd071]
[    0.848255] Detected PIPT I-cache on CPU14
[    0.848262] CPU14: found redistributor 20302 region 0:0x000000008d3a0000
[    0.848282] CPU14: using LPI pending table @0x00000011f6700000
[    0.848327] CPU14: Booted secondary processor [411fd071]
[    0.891369] Detected PIPT I-cache on CPU15
[    0.891376] CPU15: found redistributor 20303 region 0:0x000000008d3d0000
[    0.891396] CPU15: using LPI pending table @0x00000011f6730000
[    0.891443] CPU15: Booted secondary processor [411fd071]
[    0.891474] Brought up 16 CPUs
[    1.220030] SMP: Total of 16 processors activated.
[    1.224797] CPU features: detected feature: GIC system register CPU interface
[    1.231896] CPU: All CPU(s) started at EL2
[    1.235995] alternatives: patching kernel code
[    1.243242] devtmpfs: initialized
[    1.246777] SMBIOS 3.0.0 present.
[    1.250186] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.260128] pinctrl core: initialized pinctrl subsystem
[    1.265705] NET: Registered protocol family 16
[    1.282139] cpuidle: using governor menu
[    1.286084] vdso: 2 pages (1 code @ ffffff8008796000, 1 data @ ffffff8008b60000)
[    1.293459] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    1.300550] DMA: preallocated 256 KiB pool for atomic allocations
[    1.306685] ACPI: bus type PCI registered
[    1.310732] Serial: AMBA PL011 UART driver
[    1.326982] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    1.333704] ACPI: Added _OSI(Module Device)
[    1.337868] ACPI: Added _OSI(Processor Device)
[    1.342288] ACPI: Added _OSI(3.0 _SCP Extensions)
[    1.346969] ACPI: Added _OSI(Processor Aggregator Device)
[    1.353554] ACPI: Interpreter enabled
[    1.357201] ACPI: Using GIC for interrupt routing
[    1.361902] ACPI: MCFG entry for domain 0001 [bus 40-7f] (base 0x0000022004000000)
[    1.369435] ACPI: MCFG entry for domain 0002 [bus 80-bf] (base 0x0000024008000000)
[    1.376974] ACPI: MCFG table loaded, 2 entries
[    1.384442] Hisilicon MBIGEN-V1 HISI0151:00: Allocated 256 MSIs
[    1.390414] Hisilicon MBIGEN-V1 HISI0151:01: Allocated 640 MSIs
[    1.396377] Hisilicon MBIGEN-V1 HISI0151:02: Allocated 256 MSIs
[    1.402342] Hisilicon MBIGEN-V1 HISI0151:03: Allocated 640 MSIs
[    1.408462] ACPI: IORT: can't find node related to (null) device
[    1.414536] ACPI: IORT: can't find node related to (null) device
[    1.420572] ACPI: IORT: can't find node related to (null) device
[    1.426595] ACPI: IORT: can't find node related to (null) device
[    1.432626] ACPI: IORT: can't find node related to (null) device
[    1.438897] ACPI: IORT: can't find node related to (null) device
[    1.445175] ACPI: IORT: can't find node related to (null) device
[    1.451211] ACPI: IORT: can't find node related to (null) device
[    1.459089] ACPI: IORT: can't find node related to (null) device
[    1.465123] ACPI: IORT: can't find node related to (null) device
[    1.471138] ACPI: IORT: can't find node related to (null) device
[    1.477150] ACPI: IORT: can't find node related to (null) device
[    1.483165] ACPI: IORT: can't find node related to (null) device
[    1.489205] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus 40-7f])
[    1.495360] acpi HISI0080:00: _OSC: OS supports [ExtendedConfig Segments MSI]
[    1.502464] acpi HISI0080:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    1.509180] Remapped I/O 0x000002200fff0000 to [io  0x0000-0xffff window]
[    1.516016] acpi HISI0080:00: PCI host bridge to bus 0001:40
[    1.521651] pci_bus 0001:40: root bus resource [mem 0x22008000000-0x2200ffeffff window] (bus address [0xb0000000-0xb7feffff])
[    1.532906] pci_bus 0001:40: root bus resource [io  0x0000-0xffff window]
[    1.539661] pci_bus 0001:40: root bus resource [bus 40-7f]
[    1.545136] pci 0001:40:00.0: ignoring class 0x000000 (doesn't match header type 01)
[    1.553059] pci 0001:40:00.0: not setting up bridge for bus 0001:41
[    1.559335] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-bf])
[    1.565488] acpi HISI0080:01: _OSC: OS supports [ExtendedConfig Segments MSI]
[    1.572591] acpi HISI0080:01: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    1.579299] Remapped I/O 0x000002400fff0000 to [io  0x10000-0x1ffff window]
[    1.586302] acpi HISI0080:01: PCI host bridge to bus 0002:80
[    1.591936] pci_bus 0002:80: root bus resource [mem 0x2400c000000-0x2400ffeffff window] (bus address [0xc0000000-0xc3feffff])
[    1.603183] pci_bus 0002:80: root bus resource [io  0x10000-0x1ffff window] (bus address [0x0000-0xffff])
[    1.612703] pci_bus 0002:80: root bus resource [bus 80-bf]
[    1.623076] pci 0002:81:00.0: VF(n) BAR0 space: [mem 0x2400ce08000-0x2400cf07fff 64bit pref] (contains BAR0 for 64 VFs)
[    1.634312] pci 0002:81:00.0: VF(n) BAR3 space: [mem 0x2400cf08000-0x2400d007fff 64bit pref] (contains BAR3 for 64 VFs)
[    1.657152] pci 0002:81:00.1: VF(n) BAR0 space: [mem 0x2400cc04000-0x2400cd03fff 64bit pref] (contains BAR0 for 64 VFs)
[    1.668374] pci 0002:81:00.1: VF(n) BAR3 space: [mem 0x2400cd04000-0x2400ce03fff 64bit pref] (contains BAR3 for 64 VFs)
[    1.686652] pci 0002:80:00.0: BAR 15: assigned [mem 0x2400c000000-0x2400d5fffff pref]
[    1.694447] pci 0002:80:00.0: BAR 13: assigned [io  0x10000-0x10fff]
[    1.700775] pci 0002:81:00.0: BAR 0: assigned [mem 0x2400c000000-0x2400c3fffff 64bit pref]
[    1.709249] pci 0002:81:00.0: BAR 6: assigned [mem 0x2400c400000-0x2400c7fffff pref]
[    1.716956] pci 0002:81:00.1: BAR 0: assigned [mem 0x2400c800000-0x2400cbfffff 64bit pref]
[    1.725425] pci 0002:81:00.1: BAR 6: assigned [mem 0x2400cc00000-0x2400cffffff pref]
[    1.733133] pci 0002:81:00.0: BAR 4: assigned [mem 0x2400d000000-0x2400d003fff 64bit pref]
[    1.741611] pci 0002:81:00.0: BAR 7: assigned [mem 0x2400d004000-0x2400d103fff 64bit pref]
[    1.750081] pci 0002:81:00.0: BAR 10: assigned [mem 0x2400d104000-0x2400d203fff 64bit pref]
[    1.758639] pci 0002:81:00.1: BAR 4: assigned [mem 0x2400d204000-0x2400d207fff 64bit pref]
[    1.767109] pci 0002:81:00.1: BAR 7: assigned [mem 0x2400d208000-0x2400d307fff 64bit pref]
[    1.775578] pci 0002:81:00.1: BAR 10: assigned [mem 0x2400d308000-0x2400d407fff 64bit pref]
[    1.784133] pci 0002:81:00.0: BAR 2: assigned [io  0x10000-0x1001f]
[    1.790443] pci 0002:81:00.1: BAR 2: assigned [io  0x10020-0x1003f]
[    1.796753] pci 0002:80:00.0: PCI bridge to [bus 81-82]
[    1.801953] pci 0002:80:00.0:   bridge window [io  0x10000-0x10fff]
[    1.808224] pci 0002:80:00.0:   bridge window [mem 0x2400c000000-0x2400d5fffff pref]
[    1.816152] vgaarb: loaded
[    1.818950] SCSI subsystem initialized
[    1.822813] ACPI: bus type USB registered
[    1.826841] usbcore: registered new interface driver usbfs
[    1.832316] usbcore: registered new interface driver hub
[    1.837642] usbcore: registered new device driver usb
[    1.842725] pps_core: LinuxPPS API ver. 1 registered
[    1.847666] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    1.856759] PTP clock support registered
[    1.860780] Advanced Linux Sound Architecture Driver Initialized.
[    1.867087] clocksource: Switched to clocksource arch_sys_counter
[    1.873199] VFS: Disk quotas dquot_6.6.0
[    1.877120] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.884093] pnp: PnP ACPI init
[    1.887279] system 00:00: [mem 0xb0080000-0xb008ffff] has been reserved
[    1.893917] system 00:01: [mem 0xb0090000-0xb009ffff] has been reserved
[    1.900506] pnp: PnP ACPI: found 2 devices
[    1.907077] NET: Registered protocol family 2
[    1.911636] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[    1.918961] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    1.925949] TCP: Hash tables configured (established 65536 bind 65536)
[    1.932596] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[    1.938726] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[    1.945437] NET: Registered protocol family 1
[    1.949937] RPC: Registered named UNIX socket transport module.
[    1.955918] RPC: Registered udp transport module.
[    1.960668] RPC: Registered tcp transport module.
[    1.965431] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.972119] Unpacking initramfs...
[    2.337992] Freeing initrd memory: 27492K (ffffffc01e520000 - ffffffc01fff9000)
[    2.345883] kvm [1]: 8-bit VMID
[    2.349049] kvm [1]: Hyp mode initialized successfully
[    2.354260] kvm [1]: error: KVM vGIC probing failed
[    2.359209] kvm [1]: virtual timer IRQ3
[    2.364173] ACPI: IORT: can't find node related to (null) device
[    2.370636] futex hash table entries: 4096 (order: 7, 524288 bytes)
[    2.377097] audit: initializing netlink subsys (disabled)
[    2.382570] audit: type=2000 audit(1.864:1): initialized
[    2.388157] workingset: timestamp_bits=44 max_order=21 bucket_order=0
[    2.398177] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.404332] NFS: Registering the id_resolver key type
[    2.409442] Key type id_resolver registered
[    2.413657] Key type id_legacy registered
[    2.417776] fuse init (API version 7.24)
[    2.421909] 9p: Installing v9fs 9p2000 file system support
[    2.428256] io scheduler noop registered
[    2.432295] io scheduler cfq registered (default)
[    2.437480] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    2.443970] pcieport 0002:80:00.0: PCI INT A: no GSI
[    2.449726] xenfs: not registering filesystem on non-xen platform
[    2.456932] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    2.463834] console [ttyS0] disabled
[    2.467457] APMC0D08:00: ttyS0 at MMIO 0x80300000 (irq = 5, base_baud = 12500000) is a 16550A
[    2.476041] console [ttyS0] enabled
[    2.476041] console [ttyS0] enabled
[    2.483090] bootconsole [uart8250] disabled
[    2.483090] bootconsole [uart8250] disabled
[    2.491669] SuperH (H)SCI(F) driver initialized
[    2.496261] msm_serial: driver initialized
[    2.500585] Failed to find cpu0 device node
[    2.504787] Unable to detect cache hierarchy from DT for CPU 0
[    2.513098] loop: module loaded
[    2.516658] tun: Universal TUN/TAP device driver, 1.6
[    2.521735] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    2.528048] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    2.533908] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    2.539879] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k
[    2.546871] igb: Copyright (c) 2007-2014 Intel Corporation.
[    2.552493] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.0.2-k
[    2.560358] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    2.566323] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.2.1-k
[    2.574013] ixgbe: Copyright (c) 1999-2015 Intel Corporation.
[    2.579828] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    2.586299] ixgbe 0002:81:00.0: PCI INT A: no GSI
[    2.591152] ixgbe 0002:81:00.0: enabling device (0000 -> 0002)
[    3.730644] ixgbe 0002:81:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16
[    3.739317] ixgbe 0002:81:00.0: PCI Express bandwidth of 32GT/s available
[    3.746137] ixgbe 0002:81:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[    3.753381] ixgbe 0002:81:00.0: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF
[    3.759590] ixgbe 0002:81:00.0: 68:a8:28:2e:c9:10
[    3.768837] ixgbe 0002:81:00.0: Intel(R) 10 Gigabit Network Connection
[    3.775419] pcieport 0002:80:00.0: can't derive routing for PCI INT B
[    3.781891] ixgbe 0002:81:00.1: PCI INT B: no GSI
[    3.786709] ixgbe 0002:81:00.1: enabling device (0000 -> 0002)
[    3.950600] ixgbe 0002:81:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16
[    3.959271] ixgbe 0002:81:00.1: PCI Express bandwidth of 32GT/s available
[    3.966092] ixgbe 0002:81:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[    3.973337] ixgbe 0002:81:00.1: MAC: 2, PHY: 17, SFP+: 6, PBA No: FFFFFF-0FF
[    3.980416] ixgbe 0002:81:00.1: 68:a8:28:2e:c9:11
[    3.989652] ixgbe 0002:81:00.1: Intel(R) 10 Gigabit Network Connection
[    3.996242] sky2: driver version 1.30
[    4.000065] VFIO - User Level meta-driver version: 0.3
[    4.005749] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    4.012310] ehci-pci: EHCI PCI platform driver
[    4.016791] ehci-platform: EHCI generic platform driver
[    4.022075] ehci-platform PNP0D20:00: EHCI Host Controller
[    4.027594] ehci-platform PNP0D20:00: new USB bus registered, assigned bus number 1
[    4.035417] ehci-platform PNP0D20:00: irq 6, io mem 0xa1000000
[    4.051095] ehci-platform PNP0D20:00: USB 2.0 started, EHCI 1.00
[    4.057361] hub 1-0:1.0: USB hub found
[    4.061137] hub 1-0:1.0: 1 port detected
[    4.065225] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[    4.070846] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    4.077058] ohci-pci: OHCI PCI platform driver
[    4.081542] ohci-platform: OHCI generic platform driver
[    4.086870] usbcore: registered new interface driver usb-storage
[    4.093131] mousedev: PS/2 mouse device common for all mice
[    4.162242] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[    4.169358] i2c /dev entries driver
[    4.173098] sdhci: Secure Digital Host Controller Interface driver
[    4.179305] sdhci: Copyright(c) Pierre Ossman
[    4.183696] Synopsys Designware Multimedia Card Interface Driver
[    4.189782] sdhci-pltfm: SDHCI platform and OF driver helper
[    4.195546] ledtrig-cpu: registered to indicate activity on CPUs
[    4.201794] usbcore: registered new interface driver usbhid
[    4.207392] usbhid: USB HID core driver
[    4.211413] ACPI: IORT: can't find node related to (null) device
[    4.217683] NET: Registered protocol family 17
[    4.222183] 9pnet: Installing 9P2000 support
[    4.226502] Key type dns_resolver registered
[    4.231001] registered taskstats version 1
[    4.298696] rtc-efi rtc-efi: setting system clock to 2100-01-04 02:55:42 UTC (4102714542)
[    4.307002] ALSA device list:
[    4.309987]   No soundcards found.
[    4.313486] ttyS0 - failed to request DMA
[    4.317789] Freeing unused kernel memory: 840K (ffffff8008a89000 - ffffff8008b5b000)
root@(none)$ ifconfig eh th1 192.168.20.188
[   27.272687] ixgbe 0002:81:00.1: registered PHC device on eth1
root@(none)$ [   27.443193] ixgbe 0002:81:00.1 eth1: detected SFP+: 6
[   27.683130] ixgbe 0002:81:00.1 eth1: NIC Link is Up 10 Gbps, Flow Control: RX/TX

root@(none)$ ping 192.168.20.4
PING 192.168.20.4 (192.168.20.4): 56 data bytes

--IPOP Time:2016/4/26_16:54:51--
64 bytes from 192.168.20.4: seq=19 ttl=128 time=999.570 ms
64 bytes from 192.168.20.4: seq=20 ttl=128 time=0.278 ms
64 bytes from 192.168.20.4: seq=21 ttl=128 time=0.304 ms
64 bytes from 192.168.20.4: seq=22 ttl=128 time=0.311 ms
64 bytes from 192.168.20.4: seq=23 ttl=128 time=0.287 ms
64 bytes from 192.168.20.4: seq=24 ttl=128 time=0.282 ms
64 bytes from 192.168.20.4: seq=25 ttl=128 time=0.312 ms
64 bytes from 192.168.20.4: seq=26 ttl=128 time=0.295 ms
64 bytes from 192.168.20.4: seq=27 ttl=128 time=0.334 ms
64 bytes from 192.168.20.4: seq=28 ttl=128 time=0.306 ms
64 bytes from 192.168.20.4: seq=29 ttl=128 time=0.254 ms
64 bytes from 192.168.20.4: seq=30 ttl=128 time=0.297 ms
64 bytes from 192.168.20.4: seq=31 ttl=128 time=0.307 ms
[   67.963287] random: nonblocking pool is initialized
64 bytes from 192.168.20.4: seq=32 ttl=128 time=0.302 ms
64 bytes from 192.168.20.4: seq=33 ttl=128 time=0.327 ms
64 bytes from 192.168.20.4: seq=34 ttl=128 time=0.283 ms
64 bytes from 192.168.20.4: seq=35 ttl=128 time=0.325 ms
64 bytes from 192.168.20.4: seq=36 ttl=128 time=0.308 ms
64 bytes from 192.168.20.4: seq=37 ttl=128 time=0.336 ms
64 bytes from 192.168.20.4: seq=38 ttl=128 time=0.243 ms
64 bytes from 192.168.20.4: seq=39 ttl=128 time=0.239 ms
64 bytes from 192.168.20.4: seq=40 ttl=128 time=0.261 ms
64 bytes from 192.168.20.4: seq=41 ttl=128 time=0.298 ms
64 bytes from 192.168.20.4: seq=42 ttl=128 time=0.340 ms
64 bytes from 192.168.20.4: seq=43 ttl=128 time=0.283 ms
64 bytes from 192.168.20.4: seq=44 ttl=128 time=0.341 ms
64 bytes from 192.168.20.4: seq=45 ttl=128 time=0.368 ms
64 bytes from 192.168.20.4: seq=46 ttl=128 time=0.336 ms
64 bytes from 192.168.20.4: seq=47 ttl=128 time=0.373 ms
64 bytes from 192.168.20.4: seq=48 ttl=128 time=0.373 ms
64 bytes from 192.168.20.4: seq=49 ttl=128 time=0.361 ms
64 bytes from 192.168.20.4: seq=50 ttl=128 time=0.404 ms
64 bytes from 192.168.20.4: seq=51 ttl=128 time=0.357 ms
64 bytes from 192.168.20.4: seq=52 ttl=128 time=0.348 ms
64 bytes from 192.168.20.4: seq=53 ttl=128 time=0.362 ms
64 bytes from 192.168.20.4: seq=54 ttl=128 time=0.323 ms
64 bytes from 192.168.20.4: seq=55 ttl=128 time=0.309 ms
64 bytes from 192.168.20.4: seq=56 ttl=128 time=0.340 ms
64 bytes from 192.168.20.4: seq=57 ttl=128 time=0.325 ms
64 bytes from 192.168.20.4: seq=58 ttl=128 time=0.313 ms
64 bytes from 192.168.20.4: seq=59 ttl=128 time=0.327 ms

在 2016/4/16 1:06, Tomasz Nowicki 写道:
>>From the functionality point of view this series might be split into the
> following logic parts:
> 1. Necessary fixes as the preparation for using driver on ARM64.
> 2. New ECAM API and update for users of the pci-host-common API
> 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
> 4. Enable above driver on ARM64
>
> Patches has been built on top of 4.6-rc2 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v6)
>
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.
>
> v5 -> v6
> - dropped idea of x86 MMCONFIG code refactoring
> - integrated JC's patches which introduce new ECAM API:
>    https://lkml.org/lkml/2016/4/11/907
>    git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> - integrated Sinan's fix for releasing IO resources, see patch [06/13]
> - added ACPI support for ThunderX ECAM and PEM drivers
> - rebased to 4.6-rc2
>
> v4 -> v5
> - dropped MCFG refactoring group patches 1-6 from series v4 and integrated Jayachandran's patch
>    https://patchwork.ozlabs.org/patch/575525/
> - rewrite PCI legacy IRQs allocation
> - squashed two patches 11 and 12 from series v4, fixed bisection issue
> - changelog improvements
> - rebased to 4.5-rc3
>
> v3 -> v4
> - dropped Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> - added Lorenzo's fix patch 19/24
> - ACPI PCI bus domain number assigning cleanup
> - changed resource management, we now claim and reassign resources
> - improvements for applying quirks
> - dropped Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
> - rebased to 4.5-rc1
>
> v2 -> v3
> - fix legacy IRQ assigning and IO ports registration
> - remove reference to arch specific companion device for ia64
> - move ACPI PCI host controller driver to pci_root.c
> - drop generic domain assignment for x86 and ia64 as I am not
>    able to run all necessary test variants
> - drop patch which cleaned legacy IRQ assignment since it belongs to
>    Mathew's series:
>    https://patchwork.ozlabs.org/patch/557504/
> - extend MCFG quirk code
> - rebased to 4.4
>
> v1 -> v2
> - moved non-arch specific piece of code to dirver/acpi/ directory
> - fixed IO resource handling
> - introduced PCI config accessors quirks matching
> - moved ACPI_COMPANION_SET to generic code
>
> v1 - https://lkml.org/lkml/2015/10/27/504
> v2 - https://lkml.org/lkml/2015/12/16/246
> v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
> v4 - https://lkml.org/lkml/2016/2/4/646
> v5 - https://lkml.org/lkml/2016/2/16/426
>
> Jayachandran C (2):
>    PCI: Provide common functions for ECAM mapping
>    PCI: generic, thunder: update to use generic ECAM API
>
> Tomasz Nowicki (11):
>    pci, acpi, x86, ia64: Move ACPI host bridge device companion
>      assignment to core code.
>    pci, acpi: Provide generic way to assign bus domain number.
>    x86, ia64: Include acpi_pci_{add|remove}_bus to the default
>      pcibios_{add|remove}_bus implementation.
>    pci, of: Move the PCI I/O space management to PCI core code.
>    acpi, pci: Support IO resources when parsing PCI host bridge
>      resources.
>    arm64, pci, acpi: ACPI support for legacy IRQs parsing and
>      consolidation with DT code.
>    pci, acpi: Support for ACPI based generic PCI host controller
>    arm64, pci, acpi: Start using ACPI based PCI host controller driver
>      for ARM64.
>    pci, acpi: Match PCI config space accessors against platfrom specific
>      quirks.
>    pci, pci-thunder-ecam: Add ACPI support for ThunderX ECAM.
>    pci, pci-thunder-pem: Add ACPI support for ThunderX PEM.
>
>   arch/arm64/Kconfig                  |  15 +++
>   arch/arm64/include/asm/cpufeature.h |   3 +-
>   arch/arm64/kernel/cpu_errata.c      |   8 ++
>   arch/arm64/kernel/pci.c             |  35 ++---
>   arch/ia64/hp/common/sba_iommu.c     |   2 +-
>   arch/ia64/include/asm/pci.h         |   1 -
>   arch/ia64/pci/pci.c                 |  26 ----
>   arch/ia64/sn/kernel/io_acpi_init.c  |   4 +-
>   arch/x86/include/asm/pci.h          |   3 -
>   arch/x86/pci/acpi.c                 |  17 ---
>   arch/x86/pci/common.c               |  10 --
>   drivers/acpi/Kconfig                |   8 ++
>   drivers/acpi/Makefile               |   1 +
>   drivers/acpi/bus.c                  |   1 +
>   drivers/acpi/pci_gen_host.c         | 259 ++++++++++++++++++++++++++++++++++++
>   drivers/acpi/pci_root.c             |  58 +++++++-
>   drivers/of/address.c                | 116 +---------------
>   drivers/pci/Kconfig                 |   3 +
>   drivers/pci/Makefile                |   2 +
>   drivers/pci/ecam.c                  | 137 +++++++++++++++++++
>   drivers/pci/ecam.h                  |  66 +++++++++
>   drivers/pci/host/Kconfig            |   1 +
>   drivers/pci/host/pci-host-common.c  | 119 ++++++++---------
>   drivers/pci/host/pci-host-common.h  |  47 -------
>   drivers/pci/host/pci-host-generic.c |  52 ++------
>   drivers/pci/host/pci-thunder-ecam.c |  70 ++++++----
>   drivers/pci/host/pci-thunder-pem.c  | 215 ++++++++++++++++++++++--------
>   drivers/pci/pci.c                   | 150 ++++++++++++++++++++-
>   drivers/pci/probe.c                 |   5 +
>   include/asm-generic/vmlinux.lds.h   |   7 +
>   include/linux/of_address.h          |   9 --
>   include/linux/pci-acpi.h            |  20 +++
>   include/linux/pci.h                 |  12 ++
>   33 files changed, 1029 insertions(+), 453 deletions(-)
>   create mode 100644 drivers/acpi/pci_gen_host.c
>   create mode 100644 drivers/pci/ecam.c
>   create mode 100644 drivers/pci/ecam.h
>   delete mode 100644 drivers/pci/host/pci-host-common.h
>

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Bjorn Helgaas]

On Sat, Apr 16, 2016 at 04:41:45AM +0800, kbuild test robot wrote:
> Hi Tomasz,
> 
> [auto build test ERROR on pci/next]
> [also build test ERROR on v4.6-rc3 next-20160415]
> [if your patch is applied to the wrong git tree, please drop us a note to help improving the system]
> 
> url:    https://github.com/0day-ci/linux/commits/Tomasz-Nowicki/Support-for-generic-ACPI-based-PCI-host-controller/20160416-011935
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
> config: ia64-allmodconfig (attached as .config)
> reproduce:
>         wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         make.cross ARCH=ia64 
> 
> All errors (new ones prefixed by >>):
> 
>    drivers/pci/hotplug/sgi_hotplug.c: In function 'enable_slot':
> >> drivers/pci/hotplug/sgi_hotplug.c:412:61: error: 'struct pci_controller' has no member named 'companion'
>       phandle = acpi_device_handle(PCI_CONTROLLER(slot->pci_bus)->companion);
>                                                                 ^
>    drivers/pci/hotplug/sgi_hotplug.c: In function 'disable_slot':
>    drivers/pci/hotplug/sgi_hotplug.c:491:32: error: 'struct pci_controller' has no member named 'companion'
>       PCI_CONTROLLER(slot->pci_bus)->companion) {
>                                    ^
>    drivers/pci/hotplug/sgi_hotplug.c:500:61: error: 'struct pci_controller' has no member named 'companion'
>       phandle = acpi_device_handle(PCI_CONTROLLER(slot->pci_bus)->companion);
>                                                                 ^

I assume somebody is fixing this?

> 
> vim +412 drivers/pci/hotplug/sgi_hotplug.c
> 
> 3e643e77 John Keller       2007-01-30  406  		struct acpi_device *pdevice;
> 3e643e77 John Keller       2007-01-30  407  		acpi_handle phandle;
> 3e643e77 John Keller       2007-01-30  408  		acpi_handle chandle = NULL;
> 3e643e77 John Keller       2007-01-30  409  		acpi_handle rethandle;
> 3e643e77 John Keller       2007-01-30  410  		acpi_status ret;
> 3e643e77 John Keller       2007-01-30  411  
> 7b199811 Rafael J. Wysocki 2013-11-11 @412  		phandle = acpi_device_handle(PCI_CONTROLLER(slot->pci_bus)->companion);
> 3e643e77 John Keller       2007-01-30  413  
> 3e643e77 John Keller       2007-01-30  414  		if (acpi_bus_get_device(phandle, &pdevice)) {
> 227f0647 Ryan Desfosses    2014-04-18  415  			dev_dbg(&slot->pci_bus->self->dev, "no parent device, assuming NULL\n");
> 
> :::::: The code at line 412 was first introduced by commit
> :::::: 7b1998116bbb2f3e5dd6cb9a8ee6db479b0b50a9 ACPI / driver core: Store an ACPI device pointer in struct acpi_dev_node
> 
> :::::: TO: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> :::::: CC: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> 
> ---
> 0-DAY kernel test infrastructure                Open Source Technology Center
> https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Bjorn Helgaas]

On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
> As we now have valid PCI host bridge device reference we can
> introduce code that is going to find its bus domain number using
> ACPI _SEG method.
> 
> Note that _SEG method is optional, therefore _SEG absence means
> that all PCI buses belong to domain 0.
> 
> While at it, for the sake of code clarity we put ACPI and DT domain
> assign methods into the corresponding helpers.
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Reviewed-by: Liviu Dudau <Liviu.Dudau@arm.com>
> Tested-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> Tested-by: Jeremy Linton <jeremy.linton@arm.com>
> Tested-by: Duc Dang <dhdang@apm.com>
> Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> Tested-by: Sinan Kaya <okaya@codeaurora.org>
> ---
>  drivers/acpi/pci_root.c  | 18 ++++++++++++++++++
>  drivers/pci/pci.c        | 11 +++++++++--
>  include/linux/pci-acpi.h |  2 ++
>  3 files changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> index 4581e0e..d9a70c4 100644
> --- a/drivers/acpi/pci_root.c
> +++ b/drivers/acpi/pci_root.c
> @@ -419,6 +419,24 @@ out:
>  }
>  EXPORT_SYMBOL(acpi_pci_osc_control_set);
>  
> +int acpi_pci_bus_domain_nr(struct device *parent)
> +{
> +	struct acpi_device *acpi_dev = to_acpi_device(parent);
> +	unsigned long long segment = 0;
> +	acpi_status status;
> +
> +	/*
> +	 * If _SEG method does not exist, following ACPI spec (6.5.6)
> +	 * all PCI buses belong to domain 0.
> +	 */
> +	status = acpi_evaluate_integer(acpi_dev->handle, METHOD_NAME__SEG, NULL,
> +				       &segment);

We already have code in acpi_pci_root_add() to evaluate _SEG.  We
don't want to evaluate it *twice*, do we?

I was sort of expecting that if you added it here, we'd remove the
existing call, but it looks like you're keeping both?

> +	if (ACPI_FAILURE(status) && status != AE_NOT_FOUND)
> +		dev_err(&acpi_dev->dev, "can't evaluate _SEG\n");
> +
> +	return segment;
> +}
> +
>  static void negotiate_os_control(struct acpi_pci_root *root, int *no_aspm)
>  {
>  	u32 support, control, requested;
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 25e0327..1a74e87 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -19,6 +19,7 @@
>  #include <linux/spinlock.h>
>  #include <linux/string.h>
>  #include <linux/log2.h>
> +#include <linux/pci-acpi.h>
>  #include <linux/pci-aspm.h>
>  #include <linux/pm_wakeup.h>
>  #include <linux/interrupt.h>
> @@ -4779,7 +4780,7 @@ int pci_get_new_domain_nr(void)
>  }
>  
>  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> +static int of_pci_bus_domain_nr(struct device *parent)
>  {
>  	static int use_dt_domains = -1;
>  	int domain = -1;
> @@ -4823,7 +4824,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
>  		domain = -1;
>  	}
>  
> -	bus->domain_nr = domain;
> +	return domain;
> +}
> +
> +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> +{
> +	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
> +					 acpi_pci_bus_domain_nr(parent);
>  }
>  #endif
>  #endif
> diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> index 89ab057..a72e22d 100644
> --- a/include/linux/pci-acpi.h
> +++ b/include/linux/pci-acpi.h
> @@ -22,6 +22,7 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
>  {
>  	return acpi_remove_pm_notifier(dev);
>  }
> +extern int acpi_pci_bus_domain_nr(struct device *parent);
>  extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
>  
>  static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
> @@ -109,6 +110,7 @@ extern const u8 pci_acpi_dsm_uuid[];
>  #else	/* CONFIG_ACPI */
>  static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
>  static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
> +static inline int acpi_pci_bus_domain_nr(struct device *parent) { return -1; }
>  #endif	/* CONFIG_ACPI */
>  
>  #ifdef CONFIG_ACPI_APEI
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V6 03/13] x86, ia64: Include acpi_pci_{add|remove}_bus to the default pcibios_{add|remove}_bus implementation.

[Bjorn Helgaas]

On Fri, Apr 15, 2016 at 07:06:38PM +0200, Tomasz Nowicki wrote:
> x86 and ia64 are the only arches that implement pcibios_{add|remove}_bus hooks
> and implement them in the same way. Moreover ARM64 is going to do the same.
> So it seems that acpi_pci_{add|remove}_bus is generic enough to be default
> option for pcibios_{add|remove}_bus hooks. Also, it is always safe to run
> acpi_pci_{add|remove}_bus as they have empty stubs for !ACPI case and
> return if ACPI has been switched off in run time.
> 
> After all we can remove x86 and ia64 pcibios_{add|remove}_bus
> implementation.
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Tested-by: Duc Dang <dhdang@apm.com>
> Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> Tested-by: Sinan Kaya <okaya@codeaurora.org>
> ---
>  arch/ia64/pci/pci.c   | 10 ----------
>  arch/x86/pci/common.c | 10 ----------
>  drivers/pci/probe.c   |  3 +++
>  3 files changed, 3 insertions(+), 20 deletions(-)
> 
> diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
> index 978d6af..be4c9ef 100644
> --- a/arch/ia64/pci/pci.c
> +++ b/arch/ia64/pci/pci.c
> @@ -358,16 +358,6 @@ void pcibios_fixup_bus(struct pci_bus *b)
>  	platform_pci_fixup_bus(b);
>  }
>  
> -void pcibios_add_bus(struct pci_bus *bus)
> -{
> -	acpi_pci_add_bus(bus);
> -}
> -
> -void pcibios_remove_bus(struct pci_bus *bus)
> -{
> -	acpi_pci_remove_bus(bus);
> -}
> -
>  void pcibios_set_master (struct pci_dev *dev)
>  {
>  	/* No special bus mastering setup handling */
> diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
> index 381a43c..7763a84 100644
> --- a/arch/x86/pci/common.c
> +++ b/arch/x86/pci/common.c
> @@ -170,16 +170,6 @@ void pcibios_fixup_bus(struct pci_bus *b)
>  		pcibios_fixup_device_resources(dev);
>  }
>  
> -void pcibios_add_bus(struct pci_bus *bus)
> -{
> -	acpi_pci_add_bus(bus);
> -}
> -
> -void pcibios_remove_bus(struct pci_bus *bus)
> -{
> -	acpi_pci_remove_bus(bus);
> -}
> -
>  /*
>   * Only use DMI information to set this if nothing was passed
>   * on the kernel command line (which was parsed earlier).
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 8087297..ef569e8 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -12,6 +12,7 @@
>  #include <linux/slab.h>
>  #include <linux/module.h>
>  #include <linux/cpumask.h>
> +#include <linux/pci-acpi.h>
>  #include <linux/pci-aspm.h>
>  #include <linux/aer.h>
>  #include <linux/acpi.h>
> @@ -2101,10 +2102,12 @@ int __weak pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
>  
>  void __weak pcibios_add_bus(struct pci_bus *bus)
>  {
> +	acpi_pci_add_bus(bus);
>  }
>  
>  void __weak pcibios_remove_bus(struct pci_bus *bus)
>  {
> +	acpi_pci_remove_bus(bus);
>  }

Is this buying us something more than just getting rid of these
pcibios functions in the arches?  The arch-specific pcibios methods
by themselves don't seem too onerous, and I don't really want to add
#includes and calls to every firmware interface under the sun.

I admit it's a net removal of 17 lines, but I'm not sure it's a net
reduction in complexity for the reader, who now has to remember that
this ACPI stuff is a no-op on most arches.

As a tangent, some of the stuff in acpi_pci_add_bus() really belongs
elsewhere anyway.  For example, the _DSM stuff should probably be in
acpi_pci_root_create() since it's a one-per-host bridge kind of thing.

>  struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
> -- 
> 1.9.1
> 

Re: [PATCH V6 05/13] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Bjorn Helgaas]

On Fri, Apr 15, 2016 at 07:06:40PM +0200, Tomasz Nowicki wrote:
> Platforms that have memory mapped IO port (such as ARM64) need special
> handling for PCI I/O resources. For host bridge's resource probing case
> these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.

ia64 also has memory-mapped I/O port space.  It would be ideal to find
some way to handle ia64 and ARM64 similarly.  At the very least, we
have to make sure that this doesn't break ia64.  The ia64 dense/sparse
I/O spaces complicate things; I don't know if ARM64 has something
similar or not.

> Furthermore, the same I/O resources need to be released after hotplug
> removal so that it can be re-added back by the pci_remap_iospace
> function during insertion. Therefore we implement new pci_unmap_iospace call
> which unmaps I/O space as the symmetry to pci_remap_iospace.

"Furthermore" is a hint that we should check to see if this can be
split into two patches.

We already have a pci_remap_iospace(), and you're adding
pci_unmap_iospace(), which will be used for hotplug removal.  So let's 
add pci_unmap_iospace() first in a patch by itself because that's
potentially useful for other callers of pci_remap_iospace(), even if
they don't need the acpi_pci_root_remap_iospace() stuff.

> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> ---
>  drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++
>  drivers/pci/pci.c       | 24 ++++++++++++++++++++++++
>  include/linux/pci.h     |  1 +
>  3 files changed, 58 insertions(+)
> 
> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> index d9a70c4..815b6ca 100644
> --- a/drivers/acpi/pci_root.c
> +++ b/drivers/acpi/pci_root.c
> @@ -742,6 +742,34 @@ next:
>  			resource_list_add_tail(entry, resources);
>  	}
>  }
> +static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
> +{
> +#ifdef PCI_IOBASE
> +	struct resource *res = entry->res;
> +	resource_size_t cpu_addr = res->start;
> +	resource_size_t pci_addr = cpu_addr - entry->offset;
> +	resource_size_t length = resource_size(res);
> +	unsigned long port;
> +
> +	if (pci_register_io_range(cpu_addr, length))
> +		goto err;
> +
> +	port = pci_address_to_pio(cpu_addr);
> +	if (port == (unsigned long)-1)
> +		goto err;
> +
> +	res->start = port;
> +	res->end = port + length - 1;
> +	entry->offset = port - pci_addr;
> +
> +	if (pci_remap_iospace(res, cpu_addr) < 0)
> +		goto err;
> +	pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
> +	return;
> +err:
> +	res->flags |= IORESOURCE_DISABLED;
> +#endif
> +}
>  
>  int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
>  {
> @@ -763,6 +791,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
>  			"no IO and memory resources present in _CRS\n");
>  	else {
>  		resource_list_for_each_entry_safe(entry, tmp, list) {
> +			if (entry->res->flags & IORESOURCE_IO)
> +				acpi_pci_root_remap_iospace(entry);
> +
>  			if (entry->res->flags & IORESOURCE_DISABLED)
>  				resource_list_destroy_entry(entry);
>  			else
> @@ -834,6 +865,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
>  
>  	resource_list_for_each_entry(entry, &bridge->windows) {
>  		res = entry->res;
> +		if (res->flags & IORESOURCE_IO)
> +			pci_unmap_iospace(res);
>  		if (res->parent &&
>  		    (res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
>  			release_resource(res);
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 89e9996..c0f8a4e 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -26,6 +26,7 @@
>  #include <linux/device.h>
>  #include <linux/pm_runtime.h>
>  #include <linux/pci_hotplug.h>
> +#include <linux/vmalloc.h>
>  #include <asm/setup.h>
>  #include <linux/aer.h>
>  #include "pci.h"
> @@ -3168,6 +3169,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
>  #endif
>  }
>  
> +/**
> + *	pci_unmap_iospace - Unmap the memory mapped I/O space
> + *	@res: resource to be unmapped
> + *
> + *	Unmap the CPU virtual address @res from virtual address space.
> + *	Only architectures that have memory mapped IO functions defined
> + *	(and the PCI_IOBASE value defined) should call this function.
> + */
> +void  pci_unmap_iospace(struct resource *res)
> +{
> +#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
> +	unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
> +
> +	unmap_kernel_range(vaddr, resource_size(res));
> +#else
> +	/*
> +	 * This architecture does not have memory mapped I/O space,
> +	 * so this function should never be called.
> +	 */
> +	WARN_ONCE(1, "This architecture does not support memory mapped I/O\n");
> +#endif
> +}
> +
>  static void __pci_set_master(struct pci_dev *dev, bool enable)
>  {
>  	u16 old_cmd, cmd;
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index c28adb4..df1f33d 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1168,6 +1168,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size);
>  unsigned long pci_address_to_pio(phys_addr_t addr);
>  phys_addr_t pci_pio_to_address(unsigned long pio);
>  int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
> +void pci_unmap_iospace(struct resource *res);
>  
>  static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
>  {
> -- 
> 1.9.1
> 

Re: [PATCH V6 06/13] arm64, pci, acpi: ACPI support for legacy IRQs parsing and consolidation with DT code.

[Bjorn Helgaas]

On Fri, Apr 15, 2016 at 07:06:41PM +0200, Tomasz Nowicki wrote:
> To enable PCI legacy IRQs on platforms booting with ACPI, arch code
> should include ACPI specific callbacks that parse and set-up the
> device IRQ number, equivalent to the DT boot path. Owing to the current
> ACPI core scan handlers implementation, ACPI PCI legacy IRQs bindings
> cannot be parsed at device add time, since that would trigger ACPI scan
> handlers ordering issues depending on how the ACPI tables are defined.

Can you be a little more specific about the issue here?  I think you
mean pci_device_add()-time, because that's where we call
pcibios_add_device.  Which ACPI tables are involved?  _PRT?  Why is
that a problem?  We don't cache those tables any more after
181380b702ee ("PCI/ACPI: Don't cache _PRT, and don't associate them
with bus numbers").

x86 and ia64 both call acpi_pci_irq_enable() from
pcibios_enable_device().  Could you do the same on ARM64?
pcibios_enable_device() happens later than either pci_device_add() or
pci_device_probe().

> To solve this problem and consolidate FW PCI legacy IRQs parsing in
> one single pcibios callback (pending final removal), this patch moves
> DT PCI IRQ parsing to the pcibios_alloc_irq() callback (called by
> PCI core code at device probe time) and adds ACPI PCI legacy IRQs
> parsing to the same callback too, so that FW PCI legacy IRQs parsing
> is confined in one single arch callback that can be easily removed
> when code parsing PCI legacy IRQs is consolidated and moved to core
> PCI code.
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> ---
>  arch/arm64/kernel/pci.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> index c72de66..15109c11 100644
> --- a/arch/arm64/kernel/pci.c
> +++ b/arch/arm64/kernel/pci.c
> @@ -50,11 +50,16 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
>  }
>  
>  /*
> - * Try to assign the IRQ number from DT when adding a new device
> + * Try to assign the IRQ number when probing a new device
>   */
> -int pcibios_add_device(struct pci_dev *dev)
> +int pcibios_alloc_irq(struct pci_dev *dev)
>  {
> -	dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
> +	if (acpi_disabled)
> +		dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
> +#ifdef CONFIG_ACPI
> +	else
> +		return acpi_pci_irq_enable(dev);
> +#endif

Not your problem, but your patch makes it obvious: it's ugly that we
set dev->irq to the IRQ returned from of_irq_parse_and_map_pci(), but
acpi_pci_irq_enable() sets dev->irq internally.

x86 also has the situation of calling either acpi_pci_irq_enable() or
of_irq_parse_and_map_pci(), and it looks like they can even decide at
run-time as you can here.  If we're solving the same problem, can we
use a similar mechanism?  x86 sets a pcibios_enable_irq function
pointer.

>  	return 0;
>  }
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Bjorn Helgaas]

[question for Rafael below]

On Fri, Apr 15, 2016 at 07:06:36PM +0200, Tomasz Nowicki wrote:
> Currently we have two platforms (x86 & ia64) capable of PCI ACPI host
> bridge initialization. They both use arch-specific sysdata to pass down
> parent device reference and both rely on NULL parent in pci_create_root_bus()
> to validate sysdata content.
> 
> It looks hacky and prevents us from getting some firmware specific
> info for PCI host controller based on its acpi_device structure
> in generic pci_create_root_bus() function. However, we overcome that
> blocker by passing down parent device via pci_create_root_bus parameter
> (as the ACPI device type). Then we use ACPI_COMPANION_SET in core code
> for ACPI boot method only. ACPI_COMPANION_SET is safe to run for all
> cases DT, ACPI and DT&ACPI.
> 
> Since now PCI core code is setting ACPI companion device for us,
> x86 & ia64 specific ACPI companion device setting turns out to be dead now.
> We can get rid of it, including related companion reference from
> PCI sysdata structure. Aslo, PCI_CONTROLLER macro cannot return valid
> companion device anymore. Therefore we need to convert its usage to
> ACPI_COMPANION.
> 
> Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Tested-by: Duc Dang <dhdang@apm.com>
> Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> Tested-by: Sinan Kaya <okaya@codeaurora.org>
> ---
>  arch/ia64/hp/common/sba_iommu.c    |  2 +-
>  arch/ia64/include/asm/pci.h        |  1 -
>  arch/ia64/pci/pci.c                | 16 ----------------
>  arch/ia64/sn/kernel/io_acpi_init.c |  4 ++--
>  arch/x86/include/asm/pci.h         |  3 ---
>  arch/x86/pci/acpi.c                | 17 -----------------
>  drivers/acpi/pci_root.c            |  7 ++++++-
>  drivers/pci/probe.c                |  2 ++
>  8 files changed, 11 insertions(+), 41 deletions(-)
> 
> diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
> index a6d6190..78e4444 100644
> --- a/arch/ia64/hp/common/sba_iommu.c
> +++ b/arch/ia64/hp/common/sba_iommu.c
> @@ -1981,7 +1981,7 @@ sba_connect_bus(struct pci_bus *bus)
>  	if (PCI_CONTROLLER(bus)->iommu)
>  		return;
>  
> -	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
> +	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
>  	if (!handle)
>  		return;
>  
> diff --git a/arch/ia64/include/asm/pci.h b/arch/ia64/include/asm/pci.h
> index c0835b0..12423f4 100644
> --- a/arch/ia64/include/asm/pci.h
> +++ b/arch/ia64/include/asm/pci.h
> @@ -63,7 +63,6 @@ extern int pci_mmap_legacy_page_range(struct pci_bus *bus,
>  #define pci_legacy_write platform_pci_legacy_write
>  
>  struct pci_controller {
> -	struct acpi_device *companion;
>  	void *iommu;
>  	int segment;
>  	int node;		/* nearest node with memory or NUMA_NO_NODE for global allocation */
> diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
> index 8f6ac2f..978d6af 100644
> --- a/arch/ia64/pci/pci.c
> +++ b/arch/ia64/pci/pci.c
> @@ -301,28 +301,12 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  	}
>  
>  	info->controller.segment = root->segment;
> -	info->controller.companion = device;
>  	info->controller.node = acpi_get_node(device->handle);
>  	INIT_LIST_HEAD(&info->io_resources);
>  	return acpi_pci_root_create(root, &pci_acpi_root_ops,
>  				    &info->common, &info->controller);
>  }
>  
> -int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
> -{
> -	/*
> -	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
> -	 * here, pci_create_root_bus() has been called by someone else and
> -	 * sysdata is likely to be different from what we expect.  Let it go in
> -	 * that case.
> -	 */
> -	if (!bridge->dev.parent) {
> -		struct pci_controller *controller = bridge->bus->sysdata;
> -		ACPI_COMPANION_SET(&bridge->dev, controller->companion);
> -	}
> -	return 0;
> -}
> -
>  void pcibios_fixup_device_resources(struct pci_dev *dev)
>  {
>  	int idx;
> diff --git a/arch/ia64/sn/kernel/io_acpi_init.c b/arch/ia64/sn/kernel/io_acpi_init.c
> index 231234c..e454492 100644
> --- a/arch/ia64/sn/kernel/io_acpi_init.c
> +++ b/arch/ia64/sn/kernel/io_acpi_init.c
> @@ -132,7 +132,7 @@ sn_get_bussoft_ptr(struct pci_bus *bus)
>  	struct acpi_resource_vendor_typed *vendor;
>  
>  
> -	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
> +	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
>  	status = acpi_get_vendor_resource(handle, METHOD_NAME__CRS,
>  					  &sn_uuid, &buffer);
>  	if (ACPI_FAILURE(status)) {
> @@ -360,7 +360,7 @@ sn_acpi_get_pcidev_info(struct pci_dev *dev, struct pcidev_info **pcidev_info,
>  	acpi_status status;
>  	struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
>  
> -	rootbus_handle = acpi_device_handle(PCI_CONTROLLER(dev)->companion);
> +	rootbus_handle = acpi_device_handle(ACPI_COMPANION(dev->bus->bridge));
>          status = acpi_evaluate_integer(rootbus_handle, METHOD_NAME__SEG, NULL,
>                                         &segment);
>          if (ACPI_SUCCESS(status)) {
> diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
> index 9ab7507..24de07d 100644
> --- a/arch/x86/include/asm/pci.h
> +++ b/arch/x86/include/asm/pci.h
> @@ -14,9 +14,6 @@
>  struct pci_sysdata {
>  	int		domain;		/* PCI domain */
>  	int		node;		/* NUMA node */
> -#ifdef CONFIG_ACPI
> -	struct acpi_device *companion;	/* ACPI companion device */
> -#endif
>  #ifdef CONFIG_X86_64
>  	void		*iommu;		/* IOMMU private data */
>  #endif
> diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
> index 3cd6983..f4ca17a 100644
> --- a/arch/x86/pci/acpi.c
> +++ b/arch/x86/pci/acpi.c
> @@ -340,7 +340,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  		struct pci_sysdata sd = {
>  			.domain = domain,
>  			.node = node,
> -			.companion = root->device
>  		};
>  
>  		memcpy(bus->sysdata, &sd, sizeof(sd));
> @@ -355,7 +354,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  		else {
>  			info->sd.domain = domain;
>  			info->sd.node = node;
> -			info->sd.companion = root->device;
>  			bus = acpi_pci_root_create(root, &acpi_pci_root_ops,
>  						   &info->common, &info->sd);
>  		}
> @@ -373,21 +371,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  	return bus;
>  }
>  
> -int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
> -{
> -	/*
> -	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
> -	 * here, pci_create_root_bus() has been called by someone else and
> -	 * sysdata is likely to be different from what we expect.  Let it go in
> -	 * that case.
> -	 */
> -	if (!bridge->dev.parent) {
> -		struct pci_sysdata *sd = bridge->bus->sysdata;
> -		ACPI_COMPANION_SET(&bridge->dev, sd->companion);
> -	}
> -	return 0;
> -}
> -
>  int __init pci_acpi_init(void)
>  {
>  	struct pci_dev *dev = NULL;
> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> index ae3fe4e..4581e0e 100644
> --- a/drivers/acpi/pci_root.c
> +++ b/drivers/acpi/pci_root.c
> @@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device,
>  		}
>  	}
>  
> +	/*
> +	 * pci_create_root_bus() needs to detect the parent device type,
> +	 * so initialize its companion data accordingly.
> +	 */
> +	ACPI_COMPANION_SET(&device->dev, device);
>  	root->device = device;
>  	root->segment = segment & 0xFFFF;
>  	strcpy(acpi_device_name(device), ACPI_PCI_ROOT_DEVICE_NAME);
> @@ -846,7 +851,7 @@ struct pci_bus *acpi_pci_root_create(struct acpi_pci_root *root,
>  
>  	pci_acpi_root_add_resources(info);
>  	pci_add_resource(&info->resources, &root->secondary);
> -	bus = pci_create_root_bus(NULL, busnum, ops->pci_ops,
> +	bus = pci_create_root_bus(&device->dev, busnum, ops->pci_ops,
>  				  sysdata, &info->resources);

"device" here is a struct acpi_device *.  Rafael, is that the right
thing to do?  I dimly recall proposing something similar long ago and
that it turned out to be a bad idea.

>  	if (!bus)
>  		goto out_release_info;
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 8004f67..8087297 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -2141,6 +2141,8 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
>  	bridge->dev.parent = parent;
>  	bridge->dev.release = pci_release_host_bridge_dev;
>  	dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
> +	if (parent)
> +		ACPI_COMPANION_SET(&bridge->dev, ACPI_COMPANION(parent));
>  	error = pcibios_root_bridge_prepare(bridge);
>  	if (error) {
>  		kfree(bridge);
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V6 05/13] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Jon Masters]

On 04/26/2016 10:39 PM, Bjorn Helgaas wrote:
> On Fri, Apr 15, 2016 at 07:06:40PM +0200, Tomasz Nowicki wrote:
>> Platforms that have memory mapped IO port (such as ARM64) need special
>> handling for PCI I/O resources. For host bridge's resource probing case
>> these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
> 
> ia64 also has memory-mapped I/O port space.

The specific references of interest to anyone here are:

*). Volume 2, Part 1: Itanium® Architecture-based Operating System
Interaction Model with IA-32 Applications 2:267 section "10.7 I/O Port
Space Model" which describes how they can map 4 "legacy" IO ports on a
virtual page when operating in a "sparse" mode.

*). Page 378 of the ACPI6.1 specification Table 6-213 I/O Resource Flag
(Resource Type = 1) Definitions describes how a "sparse" translation can
exist depending upon bit _TRS. This seems to be implemented in Linux
using the ACPI_SPARSE_TRANSLATION types.

> It would be ideal to find
> some way to handle ia64 and ARM64 similarly.  At the very least, we
> have to make sure that this doesn't break ia64.  The ia64 dense/sparse
> I/O spaces complicate things; I don't know if ARM64 has something
> similar or not.

There's nothing directly similar - it's just regular MMIO.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Tomasz Nowicki]

On 27.04.2016 00:36, Bjorn Helgaas wrote:
> On Sat, Apr 16, 2016 at 04:41:45AM +0800, kbuild test robot wrote:
>> Hi Tomasz,
>>
>> [auto build test ERROR on pci/next]
>> [also build test ERROR on v4.6-rc3 next-20160415]
>> [if your patch is applied to the wrong git tree, please drop us a note to help improving the system]
>>
>> url:    https://github.com/0day-ci/linux/commits/Tomasz-Nowicki/Support-for-generic-ACPI-based-PCI-host-controller/20160416-011935
>> base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
>> config: ia64-allmodconfig (attached as .config)
>> reproduce:
>>          wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
>>          chmod +x ~/bin/make.cross
>>          # save the attached .config to linux build tree
>>          make.cross ARCH=ia64
>>
>> All errors (new ones prefixed by >>):
>>
>>     drivers/pci/hotplug/sgi_hotplug.c: In function 'enable_slot':
>>>> drivers/pci/hotplug/sgi_hotplug.c:412:61: error: 'struct pci_controller' has no member named 'companion'
>>        phandle = acpi_device_handle(PCI_CONTROLLER(slot->pci_bus)->companion);
>>                                                                  ^
>>     drivers/pci/hotplug/sgi_hotplug.c: In function 'disable_slot':
>>     drivers/pci/hotplug/sgi_hotplug.c:491:32: error: 'struct pci_controller' has no member named 'companion'
>>        PCI_CONTROLLER(slot->pci_bus)->companion) {
>>                                     ^
>>     drivers/pci/hotplug/sgi_hotplug.c:500:61: error: 'struct pci_controller' has no member named 'companion'
>>        phandle = acpi_device_handle(PCI_CONTROLLER(slot->pci_bus)->companion);
>>                                                                  ^
>
> I assume somebody is fixing this?

Yes, it will be fixed in next version also I will ask Hanjun to re-test 
it on IA64 as in the past.

Tomasz

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Lorenzo Pieralisi]

On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
> On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
> > As we now have valid PCI host bridge device reference we can
> > introduce code that is going to find its bus domain number using
> > ACPI _SEG method.
> > 
> > Note that _SEG method is optional, therefore _SEG absence means
> > that all PCI buses belong to domain 0.
> > 
> > While at it, for the sake of code clarity we put ACPI and DT domain
> > assign methods into the corresponding helpers.
> > 
> > Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> > Reviewed-by: Liviu Dudau <Liviu.Dudau@arm.com>
> > Tested-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> > Tested-by: Jeremy Linton <jeremy.linton@arm.com>
> > Tested-by: Duc Dang <dhdang@apm.com>
> > Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> > Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> > Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> > Tested-by: Sinan Kaya <okaya@codeaurora.org>
> > ---
> >  drivers/acpi/pci_root.c  | 18 ++++++++++++++++++
> >  drivers/pci/pci.c        | 11 +++++++++--
> >  include/linux/pci-acpi.h |  2 ++
> >  3 files changed, 29 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> > index 4581e0e..d9a70c4 100644
> > --- a/drivers/acpi/pci_root.c
> > +++ b/drivers/acpi/pci_root.c
> > @@ -419,6 +419,24 @@ out:
> >  }
> >  EXPORT_SYMBOL(acpi_pci_osc_control_set);
> >  
> > +int acpi_pci_bus_domain_nr(struct device *parent)
> > +{
> > +	struct acpi_device *acpi_dev = to_acpi_device(parent);
> > +	unsigned long long segment = 0;
> > +	acpi_status status;
> > +
> > +	/*
> > +	 * If _SEG method does not exist, following ACPI spec (6.5.6)
> > +	 * all PCI buses belong to domain 0.
> > +	 */
> > +	status = acpi_evaluate_integer(acpi_dev->handle, METHOD_NAME__SEG, NULL,
> > +				       &segment);
> 
> We already have code in acpi_pci_root_add() to evaluate _SEG.  We
> don't want to evaluate it *twice*, do we?
> 
> I was sort of expecting that if you added it here, we'd remove the
> existing call, but it looks like you're keeping both?

We can't remove the existing call, since it is used on X86 and IA64
to store the segment number that, in the process, is used in their
pci_domain_nr() arch specific callback to retrieve the domain nr.

On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
to retrieve the domain number that is not arch dependent, since
this is generic code, we can't rely on any bus->sysdata format (unless
we do something like JC did below), therefore the only way is to call
the _SEG method *again* here, which also forced Tomasz to go through
the ACPI_COMPANION setting song and dance and pass the parent pointer
to pci_create_root_bus() (see patch 1), which BTW is a source of
trouble on its own as you noticed.

JC solved it differently, via sysdata and pseudo-generic code:

http://www.spinics.net/lists/arm-kernel/msg478167.html
http://www.spinics.net/lists/arm-kernel/msg478169.html

I like neither, we need the lesser of two evils though.

Lorenzo

> > +	if (ACPI_FAILURE(status) && status != AE_NOT_FOUND)
> > +		dev_err(&acpi_dev->dev, "can't evaluate _SEG\n");
> > +
> > +	return segment;
> > +}
> > +
> >  static void negotiate_os_control(struct acpi_pci_root *root, int *no_aspm)
> >  {
> >  	u32 support, control, requested;
> > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > index 25e0327..1a74e87 100644
> > --- a/drivers/pci/pci.c
> > +++ b/drivers/pci/pci.c
> > @@ -19,6 +19,7 @@
> >  #include <linux/spinlock.h>
> >  #include <linux/string.h>
> >  #include <linux/log2.h>
> > +#include <linux/pci-acpi.h>
> >  #include <linux/pci-aspm.h>
> >  #include <linux/pm_wakeup.h>
> >  #include <linux/interrupt.h>
> > @@ -4779,7 +4780,7 @@ int pci_get_new_domain_nr(void)
> >  }
> >  
> >  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> > -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > +static int of_pci_bus_domain_nr(struct device *parent)
> >  {
> >  	static int use_dt_domains = -1;
> >  	int domain = -1;
> > @@ -4823,7 +4824,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> >  		domain = -1;
> >  	}
> >  
> > -	bus->domain_nr = domain;
> > +	return domain;
> > +}
> > +
> > +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > +{
> > +	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
> > +					 acpi_pci_bus_domain_nr(parent);
> >  }
> >  #endif
> >  #endif
> > diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> > index 89ab057..a72e22d 100644
> > --- a/include/linux/pci-acpi.h
> > +++ b/include/linux/pci-acpi.h
> > @@ -22,6 +22,7 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
> >  {
> >  	return acpi_remove_pm_notifier(dev);
> >  }
> > +extern int acpi_pci_bus_domain_nr(struct device *parent);
> >  extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
> >  
> >  static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
> > @@ -109,6 +110,7 @@ extern const u8 pci_acpi_dsm_uuid[];
> >  #else	/* CONFIG_ACPI */
> >  static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
> >  static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
> > +static inline int acpi_pci_bus_domain_nr(struct device *parent) { return -1; }
> >  #endif	/* CONFIG_ACPI */
> >  
> >  #ifdef CONFIG_ACPI_APEI
> > -- 
> > 1.9.1
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH V6 06/13] arm64, pci, acpi: ACPI support for legacy IRQs parsing and consolidation with DT code.

[Lorenzo Pieralisi]

On Tue, Apr 26, 2016 at 09:44:30PM -0500, Bjorn Helgaas wrote:
> On Fri, Apr 15, 2016 at 07:06:41PM +0200, Tomasz Nowicki wrote:
> > To enable PCI legacy IRQs on platforms booting with ACPI, arch code
> > should include ACPI specific callbacks that parse and set-up the
> > device IRQ number, equivalent to the DT boot path. Owing to the current
> > ACPI core scan handlers implementation, ACPI PCI legacy IRQs bindings
> > cannot be parsed at device add time, since that would trigger ACPI scan
> > handlers ordering issues depending on how the ACPI tables are defined.
> 
> Can you be a little more specific about the issue here?  I think you
> mean pci_device_add()-time, because that's where we call
> pcibios_add_device.  Which ACPI tables are involved?  _PRT?  Why is
> that a problem?  We don't cache those tables any more after
> 181380b702ee ("PCI/ACPI: Don't cache _PRT, and don't associate them
> with bus numbers").

https://lists.linaro.org/pipermail/linaro-acpi/2015-October/005944.html

I think it is a scan handler ordering issue and probably by caching
_PRT this problem would not exist but I have to read the commit above
in details to understand if that's the case.

> x86 and ia64 both call acpi_pci_irq_enable() from
> pcibios_enable_device().  Could you do the same on ARM64?
> pcibios_enable_device() happens later than either pci_device_add() or
> pci_device_probe().

We could in theory. In practice we have to see if that triggers DT
regressions on PCI host controllers that do not call pci_fixup_irqs(),
but rely on the legacy IRQ routing to be done in arm64
pcibios_add_device().

> > To solve this problem and consolidate FW PCI legacy IRQs parsing in
> > one single pcibios callback (pending final removal), this patch moves
> > DT PCI IRQ parsing to the pcibios_alloc_irq() callback (called by
> > PCI core code at device probe time) and adds ACPI PCI legacy IRQs
> > parsing to the same callback too, so that FW PCI legacy IRQs parsing
> > is confined in one single arch callback that can be easily removed
> > when code parsing PCI legacy IRQs is consolidated and moved to core
> > PCI code.
> > 
> > Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> > Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > ---
> >  arch/arm64/kernel/pci.c | 11 ++++++++---
> >  1 file changed, 8 insertions(+), 3 deletions(-)
> > 
> > diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> > index c72de66..15109c11 100644
> > --- a/arch/arm64/kernel/pci.c
> > +++ b/arch/arm64/kernel/pci.c
> > @@ -50,11 +50,16 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
> >  }
> >  
> >  /*
> > - * Try to assign the IRQ number from DT when adding a new device
> > + * Try to assign the IRQ number when probing a new device
> >   */
> > -int pcibios_add_device(struct pci_dev *dev)
> > +int pcibios_alloc_irq(struct pci_dev *dev)
> >  {
> > -	dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
> > +	if (acpi_disabled)
> > +		dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
> > +#ifdef CONFIG_ACPI
> > +	else
> > +		return acpi_pci_irq_enable(dev);
> > +#endif
> 
> Not your problem, but your patch makes it obvious: it's ugly that we
> set dev->irq to the IRQ returned from of_irq_parse_and_map_pci(), but
> acpi_pci_irq_enable() sets dev->irq internally.
> 
> x86 also has the situation of calling either acpi_pci_irq_enable() or
> of_irq_parse_and_map_pci(), and it looks like they can even decide at
> run-time as you can here.  If we're solving the same problem, can we
> use a similar mechanism?  x86 sets a pcibios_enable_irq function
> pointer.

Yes we could, but that's orthogonal to this patch, it's basically
rewriting this code in a different way and adding flexibility to the
function mapping irqs.

Thanks,
Lorenzo

> 
> >  	return 0;
> >  }
> > -- 
> > 1.9.1
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Tomasz Nowicki]

On 27.04.2016 04:26, Bjorn Helgaas wrote:
> On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
>> As we now have valid PCI host bridge device reference we can
>> introduce code that is going to find its bus domain number using
>> ACPI _SEG method.
>>
>> Note that _SEG method is optional, therefore _SEG absence means
>> that all PCI buses belong to domain 0.
>>
>> While at it, for the sake of code clarity we put ACPI and DT domain
>> assign methods into the corresponding helpers.
>>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Reviewed-by: Liviu Dudau <Liviu.Dudau@arm.com>
>> Tested-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>> Tested-by: Jeremy Linton <jeremy.linton@arm.com>
>> Tested-by: Duc Dang <dhdang@apm.com>
>> Tested-by: Dongdong Liu <liudongdong3@huawei.com>
>> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
>> Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
>> Tested-by: Sinan Kaya <okaya@codeaurora.org>
>> ---
>>   drivers/acpi/pci_root.c  | 18 ++++++++++++++++++
>>   drivers/pci/pci.c        | 11 +++++++++--
>>   include/linux/pci-acpi.h |  2 ++
>>   3 files changed, 29 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
>> index 4581e0e..d9a70c4 100644
>> --- a/drivers/acpi/pci_root.c
>> +++ b/drivers/acpi/pci_root.c
>> @@ -419,6 +419,24 @@ out:
>>   }
>>   EXPORT_SYMBOL(acpi_pci_osc_control_set);
>>
>> +int acpi_pci_bus_domain_nr(struct device *parent)
>> +{
>> +	struct acpi_device *acpi_dev = to_acpi_device(parent);
>> +	unsigned long long segment = 0;
>> +	acpi_status status;
>> +
>> +	/*
>> +	 * If _SEG method does not exist, following ACPI spec (6.5.6)
>> +	 * all PCI buses belong to domain 0.
>> +	 */
>> +	status = acpi_evaluate_integer(acpi_dev->handle, METHOD_NAME__SEG, NULL,
>> +				       &segment);
>
> We already have code in acpi_pci_root_add() to evaluate _SEG.  We
> don't want to evaluate it *twice*, do we?

Ideally we do not want.

The main intention here was to avoid using "void *sysdata" to retrieve 
domain number. sysdata means something different for each architectures. 
This would be common way for all arch using PCI_DOMAINS_GENERIC option 
such as ARM64.

>
> I was sort of expecting that if you added it here, we'd remove the
> existing call, but it looks like you're keeping both?

I leave _SEG evaluation in acpi_pci_root_add to keep support for IA64 
and x86 which are still using arch-specific sysdata (struct 
pci_sysdata->domain for x86 and struct pci_controller->segment for IA64) 
to retrieve domain number.

ARM64 uses PCI_DOMAINS_GENERIC for DT boot method, it would be 
consistent to keep it for ACPI too.

I am open to suggestions, do you think we should use sysdata for ARM64?

Thanks,
Tomasz

Re: [PATCH V6 03/13] x86, ia64: Include acpi_pci_{add|remove}_bus to the default pcibios_{add|remove}_bus implementation.

[Tomasz Nowicki]

On 27.04.2016 04:34, Bjorn Helgaas wrote:
> On Fri, Apr 15, 2016 at 07:06:38PM +0200, Tomasz Nowicki wrote:
>> x86 and ia64 are the only arches that implement pcibios_{add|remove}_bus hooks
>> and implement them in the same way. Moreover ARM64 is going to do the same.
>> So it seems that acpi_pci_{add|remove}_bus is generic enough to be default
>> option for pcibios_{add|remove}_bus hooks. Also, it is always safe to run
>> acpi_pci_{add|remove}_bus as they have empty stubs for !ACPI case and
>> return if ACPI has been switched off in run time.
>>
>> After all we can remove x86 and ia64 pcibios_{add|remove}_bus
>> implementation.
>>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
>> Tested-by: Duc Dang <dhdang@apm.com>
>> Tested-by: Dongdong Liu <liudongdong3@huawei.com>
>> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
>> Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
>> Tested-by: Sinan Kaya <okaya@codeaurora.org>
>> ---
>>   arch/ia64/pci/pci.c   | 10 ----------
>>   arch/x86/pci/common.c | 10 ----------
>>   drivers/pci/probe.c   |  3 +++
>>   3 files changed, 3 insertions(+), 20 deletions(-)
>>
>> diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
>> index 978d6af..be4c9ef 100644
>> --- a/arch/ia64/pci/pci.c
>> +++ b/arch/ia64/pci/pci.c
>> @@ -358,16 +358,6 @@ void pcibios_fixup_bus(struct pci_bus *b)
>>   	platform_pci_fixup_bus(b);
>>   }
>>
>> -void pcibios_add_bus(struct pci_bus *bus)
>> -{
>> -	acpi_pci_add_bus(bus);
>> -}
>> -
>> -void pcibios_remove_bus(struct pci_bus *bus)
>> -{
>> -	acpi_pci_remove_bus(bus);
>> -}
>> -
>>   void pcibios_set_master (struct pci_dev *dev)
>>   {
>>   	/* No special bus mastering setup handling */
>> diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
>> index 381a43c..7763a84 100644
>> --- a/arch/x86/pci/common.c
>> +++ b/arch/x86/pci/common.c
>> @@ -170,16 +170,6 @@ void pcibios_fixup_bus(struct pci_bus *b)
>>   		pcibios_fixup_device_resources(dev);
>>   }
>>
>> -void pcibios_add_bus(struct pci_bus *bus)
>> -{
>> -	acpi_pci_add_bus(bus);
>> -}
>> -
>> -void pcibios_remove_bus(struct pci_bus *bus)
>> -{
>> -	acpi_pci_remove_bus(bus);
>> -}
>> -
>>   /*
>>    * Only use DMI information to set this if nothing was passed
>>    * on the kernel command line (which was parsed earlier).
>> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
>> index 8087297..ef569e8 100644
>> --- a/drivers/pci/probe.c
>> +++ b/drivers/pci/probe.c
>> @@ -12,6 +12,7 @@
>>   #include <linux/slab.h>
>>   #include <linux/module.h>
>>   #include <linux/cpumask.h>
>> +#include <linux/pci-acpi.h>
>>   #include <linux/pci-aspm.h>
>>   #include <linux/aer.h>
>>   #include <linux/acpi.h>
>> @@ -2101,10 +2102,12 @@ int __weak pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
>>
>>   void __weak pcibios_add_bus(struct pci_bus *bus)
>>   {
>> +	acpi_pci_add_bus(bus);
>>   }
>>
>>   void __weak pcibios_remove_bus(struct pci_bus *bus)
>>   {
>> +	acpi_pci_remove_bus(bus);
>>   }
>
> Is this buying us something more than just getting rid of these
> pcibios functions in the arches?  The arch-specific pcibios methods
> by themselves don't seem too onerous, and I don't really want to add
> #includes and calls to every firmware interface under the sun.
>
> I admit it's a net removal of 17 lines, but I'm not sure it's a net
> reduction in complexity for the reader, who now has to remember that
> this ACPI stuff is a no-op on most arches.
>
> As a tangent, some of the stuff in acpi_pci_add_bus() really belongs
> elsewhere anyway.  For example, the _DSM stuff should probably be in
> acpi_pci_root_create() since it's a one-per-host bridge kind of thing.
>

OK, I will add pcibios_add_bus to ARM64 arch code and call 
acpi_pci_remove_bus(bus) from there.

Thanks,
Tomasz

Re: [PATCH V6 05/13] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Lorenzo Pieralisi]

On Tue, Apr 26, 2016 at 09:39:16PM -0500, Bjorn Helgaas wrote:
> On Fri, Apr 15, 2016 at 07:06:40PM +0200, Tomasz Nowicki wrote:
> > Platforms that have memory mapped IO port (such as ARM64) need special
> > handling for PCI I/O resources. For host bridge's resource probing case
> > these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
> 
> ia64 also has memory-mapped I/O port space.  It would be ideal to find
> some way to handle ia64 and ARM64 similarly.  At the very least, we
> have to make sure that this doesn't break ia64.  The ia64 dense/sparse
> I/O spaces complicate things; I don't know if ARM64 has something
> similar or not.

No it does not, and that's exactly the same problem we faced with
the DT generic version of of_pci_range_to_resource() which basically
relies on PCI_IOBASE to be defined to add code that creates IO port
resources out of the MMIO resource describing how IO port space is
mapped to MMIO (physical) address space.

IIRC everything hinges on PCI_IOBASE definition to make sure that
of_pci_range_to_resource() *works*, which means that if PCI_IOBASE is
not defined (ie IA64) that code - acpi_pci_root_remap_iospace() in this
case - does nothing.

So acpi_pci_root_remap_iospace() is of_pci_range_to_resource() ACPI
equivalent + the pci_remap_iospace() call (I have to dig into the
logs to check why Liviu did not add a call to pci_remap_iospace()
in of_pci_get_host_bridge_resources() - I want to do that actually).

The point here is: IO space (in DT and ACPI) handling is arch specific.

For DT, by relying on PCI_IOBASE, we left that code in drivers/of and
it works (well, with some niggles - see the thread with Murali on IO
space on TI keystone) for ARM/ARM64.

http://www.spinics.net/lists/linux-pci/msg49725.html

What are we going to do with the ACPI version ?

Do we want to add an arch specific call that takes the raw resource
describing IO space and creates an IO port resource (and the MMIO
equivalent - that's what add_io_space() does in IA64) and use that
in generic ACPI parsing code ?

Or we just do what Tomasz does, which is basically the approach we took
for DT ?

> > Furthermore, the same I/O resources need to be released after hotplug
> > removal so that it can be re-added back by the pci_remap_iospace
> > function during insertion. Therefore we implement new pci_unmap_iospace call
> > which unmaps I/O space as the symmetry to pci_remap_iospace.
> 
> "Furthermore" is a hint that we should check to see if this can be
> split into two patches.
> 
> We already have a pci_remap_iospace(), and you're adding
> pci_unmap_iospace(), which will be used for hotplug removal.  So let's 
> add pci_unmap_iospace() first in a patch by itself because that's
> potentially useful for other callers of pci_remap_iospace(), even if
> they don't need the acpi_pci_root_remap_iospace() stuff.

I agree.

Thanks,
Lorenzo

> > Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> > Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> > Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> > ---
> >  drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++
> >  drivers/pci/pci.c       | 24 ++++++++++++++++++++++++
> >  include/linux/pci.h     |  1 +
> >  3 files changed, 58 insertions(+)
> > 
> > diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> > index d9a70c4..815b6ca 100644
> > --- a/drivers/acpi/pci_root.c
> > +++ b/drivers/acpi/pci_root.c
> > @@ -742,6 +742,34 @@ next:
> >  			resource_list_add_tail(entry, resources);
> >  	}
> >  }
> > +static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
> > +{
> > +#ifdef PCI_IOBASE
> > +	struct resource *res = entry->res;
> > +	resource_size_t cpu_addr = res->start;
> > +	resource_size_t pci_addr = cpu_addr - entry->offset;
> > +	resource_size_t length = resource_size(res);
> > +	unsigned long port;
> > +
> > +	if (pci_register_io_range(cpu_addr, length))
> > +		goto err;
> > +
> > +	port = pci_address_to_pio(cpu_addr);
> > +	if (port == (unsigned long)-1)
> > +		goto err;
> > +
> > +	res->start = port;
> > +	res->end = port + length - 1;
> > +	entry->offset = port - pci_addr;
> > +
> > +	if (pci_remap_iospace(res, cpu_addr) < 0)
> > +		goto err;
> > +	pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
> > +	return;
> > +err:
> > +	res->flags |= IORESOURCE_DISABLED;
> > +#endif
> > +}
> >  
> >  int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
> >  {
> > @@ -763,6 +791,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
> >  			"no IO and memory resources present in _CRS\n");
> >  	else {
> >  		resource_list_for_each_entry_safe(entry, tmp, list) {
> > +			if (entry->res->flags & IORESOURCE_IO)
> > +				acpi_pci_root_remap_iospace(entry);
> > +
> >  			if (entry->res->flags & IORESOURCE_DISABLED)
> >  				resource_list_destroy_entry(entry);
> >  			else
> > @@ -834,6 +865,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
> >  
> >  	resource_list_for_each_entry(entry, &bridge->windows) {
> >  		res = entry->res;
> > +		if (res->flags & IORESOURCE_IO)
> > +			pci_unmap_iospace(res);
> >  		if (res->parent &&
> >  		    (res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
> >  			release_resource(res);
> > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > index 89e9996..c0f8a4e 100644
> > --- a/drivers/pci/pci.c
> > +++ b/drivers/pci/pci.c
> > @@ -26,6 +26,7 @@
> >  #include <linux/device.h>
> >  #include <linux/pm_runtime.h>
> >  #include <linux/pci_hotplug.h>
> > +#include <linux/vmalloc.h>
> >  #include <asm/setup.h>
> >  #include <linux/aer.h>
> >  #include "pci.h"
> > @@ -3168,6 +3169,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
> >  #endif
> >  }
> >  
> > +/**
> > + *	pci_unmap_iospace - Unmap the memory mapped I/O space
> > + *	@res: resource to be unmapped
> > + *
> > + *	Unmap the CPU virtual address @res from virtual address space.
> > + *	Only architectures that have memory mapped IO functions defined
> > + *	(and the PCI_IOBASE value defined) should call this function.
> > + */
> > +void  pci_unmap_iospace(struct resource *res)
> > +{
> > +#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
> > +	unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
> > +
> > +	unmap_kernel_range(vaddr, resource_size(res));
> > +#else
> > +	/*
> > +	 * This architecture does not have memory mapped I/O space,
> > +	 * so this function should never be called.
> > +	 */
> > +	WARN_ONCE(1, "This architecture does not support memory mapped I/O\n");
> > +#endif
> > +}
> > +
> >  static void __pci_set_master(struct pci_dev *dev, bool enable)
> >  {
> >  	u16 old_cmd, cmd;
> > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > index c28adb4..df1f33d 100644
> > --- a/include/linux/pci.h
> > +++ b/include/linux/pci.h
> > @@ -1168,6 +1168,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size);
> >  unsigned long pci_address_to_pio(phys_addr_t addr);
> >  phys_addr_t pci_pio_to_address(unsigned long pio);
> >  int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
> > +void pci_unmap_iospace(struct resource *res);
> >  
> >  static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
> >  {
> > -- 
> > 1.9.1
> > 
> 

Re: [PATCH V6 05/13] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Liviu.Dudau]

On Wed, Apr 27, 2016 at 03:26:59PM +0100, Lorenzo Pieralisi wrote:
> On Tue, Apr 26, 2016 at 09:39:16PM -0500, Bjorn Helgaas wrote:
> > On Fri, Apr 15, 2016 at 07:06:40PM +0200, Tomasz Nowicki wrote:
> > > Platforms that have memory mapped IO port (such as ARM64) need special
> > > handling for PCI I/O resources. For host bridge's resource probing case
> > > these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
> > 
> > ia64 also has memory-mapped I/O port space.  It would be ideal to find
> > some way to handle ia64 and ARM64 similarly.  At the very least, we
> > have to make sure that this doesn't break ia64.  The ia64 dense/sparse
> > I/O spaces complicate things; I don't know if ARM64 has something
> > similar or not.
> 
> No it does not, and that's exactly the same problem we faced with
> the DT generic version of of_pci_range_to_resource() which basically
> relies on PCI_IOBASE to be defined to add code that creates IO port
> resources out of the MMIO resource describing how IO port space is
> mapped to MMIO (physical) address space.
> 
> IIRC everything hinges on PCI_IOBASE definition to make sure that
> of_pci_range_to_resource() *works*, which means that if PCI_IOBASE is
> not defined (ie IA64) that code - acpi_pci_root_remap_iospace() in this
> case - does nothing.
> 
> So acpi_pci_root_remap_iospace() is of_pci_range_to_resource() ACPI
> equivalent + the pci_remap_iospace() call (I have to dig into the
> logs to check why Liviu did not add a call to pci_remap_iospace()
> in of_pci_get_host_bridge_resources() - I want to do that actually).

Because of_pci_get_host_bridge_resources() only gives you a list of resources,
it doesn't allocate them. An arch or platform could add further filtering
to that list before it gets requested (in our case it is done in pci-host-common.c)

Best regards,
Liviu

> 
> The point here is: IO space (in DT and ACPI) handling is arch specific.
> 
> For DT, by relying on PCI_IOBASE, we left that code in drivers/of and
> it works (well, with some niggles - see the thread with Murali on IO
> space on TI keystone) for ARM/ARM64.
> 
> http://www.spinics.net/lists/linux-pci/msg49725.html
> 
> What are we going to do with the ACPI version ?
> 
> Do we want to add an arch specific call that takes the raw resource
> describing IO space and creates an IO port resource (and the MMIO
> equivalent - that's what add_io_space() does in IA64) and use that
> in generic ACPI parsing code ?
> 
> Or we just do what Tomasz does, which is basically the approach we took
> for DT ?
> 
> > > Furthermore, the same I/O resources need to be released after hotplug
> > > removal so that it can be re-added back by the pci_remap_iospace
> > > function during insertion. Therefore we implement new pci_unmap_iospace call
> > > which unmaps I/O space as the symmetry to pci_remap_iospace.
> > 
> > "Furthermore" is a hint that we should check to see if this can be
> > split into two patches.
> > 
> > We already have a pci_remap_iospace(), and you're adding
> > pci_unmap_iospace(), which will be used for hotplug removal.  So let's 
> > add pci_unmap_iospace() first in a patch by itself because that's
> > potentially useful for other callers of pci_remap_iospace(), even if
> > they don't need the acpi_pci_root_remap_iospace() stuff.
> 
> I agree.
> 
> Thanks,
> Lorenzo
> 
> > > Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> > > Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> > > Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> > > ---
> > >  drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++
> > >  drivers/pci/pci.c       | 24 ++++++++++++++++++++++++
> > >  include/linux/pci.h     |  1 +
> > >  3 files changed, 58 insertions(+)
> > > 
> > > diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> > > index d9a70c4..815b6ca 100644
> > > --- a/drivers/acpi/pci_root.c
> > > +++ b/drivers/acpi/pci_root.c
> > > @@ -742,6 +742,34 @@ next:
> > >  			resource_list_add_tail(entry, resources);
> > >  	}
> > >  }
> > > +static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
> > > +{
> > > +#ifdef PCI_IOBASE
> > > +	struct resource *res = entry->res;
> > > +	resource_size_t cpu_addr = res->start;
> > > +	resource_size_t pci_addr = cpu_addr - entry->offset;
> > > +	resource_size_t length = resource_size(res);
> > > +	unsigned long port;
> > > +
> > > +	if (pci_register_io_range(cpu_addr, length))
> > > +		goto err;
> > > +
> > > +	port = pci_address_to_pio(cpu_addr);
> > > +	if (port == (unsigned long)-1)
> > > +		goto err;
> > > +
> > > +	res->start = port;
> > > +	res->end = port + length - 1;
> > > +	entry->offset = port - pci_addr;
> > > +
> > > +	if (pci_remap_iospace(res, cpu_addr) < 0)
> > > +		goto err;
> > > +	pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
> > > +	return;
> > > +err:
> > > +	res->flags |= IORESOURCE_DISABLED;
> > > +#endif
> > > +}
> > >  
> > >  int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
> > >  {
> > > @@ -763,6 +791,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
> > >  			"no IO and memory resources present in _CRS\n");
> > >  	else {
> > >  		resource_list_for_each_entry_safe(entry, tmp, list) {
> > > +			if (entry->res->flags & IORESOURCE_IO)
> > > +				acpi_pci_root_remap_iospace(entry);
> > > +
> > >  			if (entry->res->flags & IORESOURCE_DISABLED)
> > >  				resource_list_destroy_entry(entry);
> > >  			else
> > > @@ -834,6 +865,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
> > >  
> > >  	resource_list_for_each_entry(entry, &bridge->windows) {
> > >  		res = entry->res;
> > > +		if (res->flags & IORESOURCE_IO)
> > > +			pci_unmap_iospace(res);
> > >  		if (res->parent &&
> > >  		    (res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
> > >  			release_resource(res);
> > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > > index 89e9996..c0f8a4e 100644
> > > --- a/drivers/pci/pci.c
> > > +++ b/drivers/pci/pci.c
> > > @@ -26,6 +26,7 @@
> > >  #include <linux/device.h>
> > >  #include <linux/pm_runtime.h>
> > >  #include <linux/pci_hotplug.h>
> > > +#include <linux/vmalloc.h>
> > >  #include <asm/setup.h>
> > >  #include <linux/aer.h>
> > >  #include "pci.h"
> > > @@ -3168,6 +3169,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
> > >  #endif
> > >  }
> > >  
> > > +/**
> > > + *	pci_unmap_iospace - Unmap the memory mapped I/O space
> > > + *	@res: resource to be unmapped
> > > + *
> > > + *	Unmap the CPU virtual address @res from virtual address space.
> > > + *	Only architectures that have memory mapped IO functions defined
> > > + *	(and the PCI_IOBASE value defined) should call this function.
> > > + */
> > > +void  pci_unmap_iospace(struct resource *res)
> > > +{
> > > +#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
> > > +	unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
> > > +
> > > +	unmap_kernel_range(vaddr, resource_size(res));
> > > +#else
> > > +	/*
> > > +	 * This architecture does not have memory mapped I/O space,
> > > +	 * so this function should never be called.
> > > +	 */
> > > +	WARN_ONCE(1, "This architecture does not support memory mapped I/O\n");
> > > +#endif
> > > +}
> > > +
> > >  static void __pci_set_master(struct pci_dev *dev, bool enable)
> > >  {
> > >  	u16 old_cmd, cmd;
> > > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > > index c28adb4..df1f33d 100644
> > > --- a/include/linux/pci.h
> > > +++ b/include/linux/pci.h
> > > @@ -1168,6 +1168,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size);
> > >  unsigned long pci_address_to_pio(phys_addr_t addr);
> > >  phys_addr_t pci_pio_to_address(unsigned long pio);
> > >  int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
> > > +void pci_unmap_iospace(struct resource *res);
> > >  
> > >  static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
> > >  {
> > > -- 
> > > 1.9.1
> > > 
> > 
> 

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH V6 05/13] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Lorenzo Pieralisi]

On Wed, Apr 27, 2016 at 04:10:36PM +0100, Liviu.Dudau@arm.com wrote:
> On Wed, Apr 27, 2016 at 03:26:59PM +0100, Lorenzo Pieralisi wrote:
> > On Tue, Apr 26, 2016 at 09:39:16PM -0500, Bjorn Helgaas wrote:
> > > On Fri, Apr 15, 2016 at 07:06:40PM +0200, Tomasz Nowicki wrote:
> > > > Platforms that have memory mapped IO port (such as ARM64) need special
> > > > handling for PCI I/O resources. For host bridge's resource probing case
> > > > these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
> > > 
> > > ia64 also has memory-mapped I/O port space.  It would be ideal to find
> > > some way to handle ia64 and ARM64 similarly.  At the very least, we
> > > have to make sure that this doesn't break ia64.  The ia64 dense/sparse
> > > I/O spaces complicate things; I don't know if ARM64 has something
> > > similar or not.
> > 
> > No it does not, and that's exactly the same problem we faced with
> > the DT generic version of of_pci_range_to_resource() which basically
> > relies on PCI_IOBASE to be defined to add code that creates IO port
> > resources out of the MMIO resource describing how IO port space is
> > mapped to MMIO (physical) address space.
> > 
> > IIRC everything hinges on PCI_IOBASE definition to make sure that
> > of_pci_range_to_resource() *works*, which means that if PCI_IOBASE is
> > not defined (ie IA64) that code - acpi_pci_root_remap_iospace() in this
> > case - does nothing.
> > 
> > So acpi_pci_root_remap_iospace() is of_pci_range_to_resource() ACPI
> > equivalent + the pci_remap_iospace() call (I have to dig into the
> > logs to check why Liviu did not add a call to pci_remap_iospace()
> > in of_pci_get_host_bridge_resources() - I want to do that actually).
> 
> Because of_pci_get_host_bridge_resources() only gives you a list of
> resources, it doesn't allocate them. An arch or platform could add
> further filtering to that list before it gets requested (in our case
> it is done in pci-host-common.c)

Well, it does register the IO cpu physical address in pci_register_io_range()
though, if pci_remap_iospace() fails in arch/platform code we can delete the
resource but we must also unregister the corresponding cpu address from
the IO ranges otherwise we end up with stale entries in the io_range_list.

Anyway, it is not related to this thread, I will see what I can do
to improve that API from this standpoint.

Thanks !
Lorenzo

> 
> Best regards,
> Liviu
> 
> > 
> > The point here is: IO space (in DT and ACPI) handling is arch specific.
> > 
> > For DT, by relying on PCI_IOBASE, we left that code in drivers/of and
> > it works (well, with some niggles - see the thread with Murali on IO
> > space on TI keystone) for ARM/ARM64.
> > 
> > http://www.spinics.net/lists/linux-pci/msg49725.html
> > 
> > What are we going to do with the ACPI version ?
> > 
> > Do we want to add an arch specific call that takes the raw resource
> > describing IO space and creates an IO port resource (and the MMIO
> > equivalent - that's what add_io_space() does in IA64) and use that
> > in generic ACPI parsing code ?
> > 
> > Or we just do what Tomasz does, which is basically the approach we took
> > for DT ?
> > 
> > > > Furthermore, the same I/O resources need to be released after hotplug
> > > > removal so that it can be re-added back by the pci_remap_iospace
> > > > function during insertion. Therefore we implement new pci_unmap_iospace call
> > > > which unmaps I/O space as the symmetry to pci_remap_iospace.
> > > 
> > > "Furthermore" is a hint that we should check to see if this can be
> > > split into two patches.
> > > 
> > > We already have a pci_remap_iospace(), and you're adding
> > > pci_unmap_iospace(), which will be used for hotplug removal.  So let's 
> > > add pci_unmap_iospace() first in a patch by itself because that's
> > > potentially useful for other callers of pci_remap_iospace(), even if
> > > they don't need the acpi_pci_root_remap_iospace() stuff.
> > 
> > I agree.
> > 
> > Thanks,
> > Lorenzo
> > 
> > > > Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> > > > Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> > > > Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> > > > ---
> > > >  drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++
> > > >  drivers/pci/pci.c       | 24 ++++++++++++++++++++++++
> > > >  include/linux/pci.h     |  1 +
> > > >  3 files changed, 58 insertions(+)
> > > > 
> > > > diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> > > > index d9a70c4..815b6ca 100644
> > > > --- a/drivers/acpi/pci_root.c
> > > > +++ b/drivers/acpi/pci_root.c
> > > > @@ -742,6 +742,34 @@ next:
> > > >  			resource_list_add_tail(entry, resources);
> > > >  	}
> > > >  }
> > > > +static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
> > > > +{
> > > > +#ifdef PCI_IOBASE
> > > > +	struct resource *res = entry->res;
> > > > +	resource_size_t cpu_addr = res->start;
> > > > +	resource_size_t pci_addr = cpu_addr - entry->offset;
> > > > +	resource_size_t length = resource_size(res);
> > > > +	unsigned long port;
> > > > +
> > > > +	if (pci_register_io_range(cpu_addr, length))
> > > > +		goto err;
> > > > +
> > > > +	port = pci_address_to_pio(cpu_addr);
> > > > +	if (port == (unsigned long)-1)
> > > > +		goto err;
> > > > +
> > > > +	res->start = port;
> > > > +	res->end = port + length - 1;
> > > > +	entry->offset = port - pci_addr;
> > > > +
> > > > +	if (pci_remap_iospace(res, cpu_addr) < 0)
> > > > +		goto err;
> > > > +	pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
> > > > +	return;
> > > > +err:
> > > > +	res->flags |= IORESOURCE_DISABLED;
> > > > +#endif
> > > > +}
> > > >  
> > > >  int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
> > > >  {
> > > > @@ -763,6 +791,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
> > > >  			"no IO and memory resources present in _CRS\n");
> > > >  	else {
> > > >  		resource_list_for_each_entry_safe(entry, tmp, list) {
> > > > +			if (entry->res->flags & IORESOURCE_IO)
> > > > +				acpi_pci_root_remap_iospace(entry);
> > > > +
> > > >  			if (entry->res->flags & IORESOURCE_DISABLED)
> > > >  				resource_list_destroy_entry(entry);
> > > >  			else
> > > > @@ -834,6 +865,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
> > > >  
> > > >  	resource_list_for_each_entry(entry, &bridge->windows) {
> > > >  		res = entry->res;
> > > > +		if (res->flags & IORESOURCE_IO)
> > > > +			pci_unmap_iospace(res);
> > > >  		if (res->parent &&
> > > >  		    (res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
> > > >  			release_resource(res);
> > > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > > > index 89e9996..c0f8a4e 100644
> > > > --- a/drivers/pci/pci.c
> > > > +++ b/drivers/pci/pci.c
> > > > @@ -26,6 +26,7 @@
> > > >  #include <linux/device.h>
> > > >  #include <linux/pm_runtime.h>
> > > >  #include <linux/pci_hotplug.h>
> > > > +#include <linux/vmalloc.h>
> > > >  #include <asm/setup.h>
> > > >  #include <linux/aer.h>
> > > >  #include "pci.h"
> > > > @@ -3168,6 +3169,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
> > > >  #endif
> > > >  }
> > > >  
> > > > +/**
> > > > + *	pci_unmap_iospace - Unmap the memory mapped I/O space
> > > > + *	@res: resource to be unmapped
> > > > + *
> > > > + *	Unmap the CPU virtual address @res from virtual address space.
> > > > + *	Only architectures that have memory mapped IO functions defined
> > > > + *	(and the PCI_IOBASE value defined) should call this function.
> > > > + */
> > > > +void  pci_unmap_iospace(struct resource *res)
> > > > +{
> > > > +#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
> > > > +	unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
> > > > +
> > > > +	unmap_kernel_range(vaddr, resource_size(res));
> > > > +#else
> > > > +	/*
> > > > +	 * This architecture does not have memory mapped I/O space,
> > > > +	 * so this function should never be called.
> > > > +	 */
> > > > +	WARN_ONCE(1, "This architecture does not support memory mapped I/O\n");
> > > > +#endif
> > > > +}
> > > > +
> > > >  static void __pci_set_master(struct pci_dev *dev, bool enable)
> > > >  {
> > > >  	u16 old_cmd, cmd;
> > > > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > > > index c28adb4..df1f33d 100644
> > > > --- a/include/linux/pci.h
> > > > +++ b/include/linux/pci.h
> > > > @@ -1168,6 +1168,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size);
> > > >  unsigned long pci_address_to_pio(phys_addr_t addr);
> > > >  phys_addr_t pci_pio_to_address(unsigned long pio);
> > > >  int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
> > > > +void pci_unmap_iospace(struct resource *res);
> > > >  
> > > >  static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
> > > >  {
> > > > -- 
> > > > 1.9.1
> > > > 
> > > 
> > 
> 
> -- 
> ====================
> | I would like to |
> | fix the world,  |
> | but they're not |
> | giving me the   |
>  \ source code!  /
>   ---------------
>     ??\_(???)_/??

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Bjorn Helgaas]

On Wed, Apr 27, 2016 at 12:17:58PM +0100, Lorenzo Pieralisi wrote:
> On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
> > On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
> > > As we now have valid PCI host bridge device reference we can
> > > introduce code that is going to find its bus domain number using
> > > ACPI _SEG method.
> > > 
> > > Note that _SEG method is optional, therefore _SEG absence means
> > > that all PCI buses belong to domain 0.
> > > 
> > > While at it, for the sake of code clarity we put ACPI and DT domain
> > > assign methods into the corresponding helpers.
> > > 
> > > Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> > > Reviewed-by: Liviu Dudau <Liviu.Dudau@arm.com>
> > > Tested-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> > > Tested-by: Jeremy Linton <jeremy.linton@arm.com>
> > > Tested-by: Duc Dang <dhdang@apm.com>
> > > Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> > > Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> > > Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> > > Tested-by: Sinan Kaya <okaya@codeaurora.org>
> > > ---
> > >  drivers/acpi/pci_root.c  | 18 ++++++++++++++++++
> > >  drivers/pci/pci.c        | 11 +++++++++--
> > >  include/linux/pci-acpi.h |  2 ++
> > >  3 files changed, 29 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> > > index 4581e0e..d9a70c4 100644
> > > --- a/drivers/acpi/pci_root.c
> > > +++ b/drivers/acpi/pci_root.c
> > > @@ -419,6 +419,24 @@ out:
> > >  }
> > >  EXPORT_SYMBOL(acpi_pci_osc_control_set);
> > >  
> > > +int acpi_pci_bus_domain_nr(struct device *parent)

It looks like acpi_pci_bus_domain_nr() could be under #ifdef
CONFIG_PCI_DOMAINS_GENERIC, right?

> > > +{
> > > +	struct acpi_device *acpi_dev = to_acpi_device(parent);
> > > +	unsigned long long segment = 0;
> > > +	acpi_status status;
> > > +
> > > +	/*
> > > +	 * If _SEG method does not exist, following ACPI spec (6.5.6)
> > > +	 * all PCI buses belong to domain 0.
> > > +	 */
> > > +	status = acpi_evaluate_integer(acpi_dev->handle, METHOD_NAME__SEG, NULL,
> > > +				       &segment);
> > 
> > We already have code in acpi_pci_root_add() to evaluate _SEG.  We
> > don't want to evaluate it *twice*, do we?
> > 
> > I was sort of expecting that if you added it here, we'd remove the
> > existing call, but it looks like you're keeping both?
> 
> We can't remove the existing call, since it is used on X86 and IA64
> to store the segment number that, in the process, is used in their
> pci_domain_nr() arch specific callback to retrieve the domain nr.
> 
> On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
> to retrieve the domain number that is not arch dependent, since
> this is generic code, we can't rely on any bus->sysdata format (unless
> we do something like JC did below), therefore the only way is to call
> the _SEG method *again* here, which also forced Tomasz to go through
> the ACPI_COMPANION setting song and dance and pass the parent pointer
> to pci_create_root_bus() (see patch 1), which BTW is a source of
> trouble on its own as you noticed.
> 
> JC solved it differently, via sysdata and pseudo-generic code:
> 
> http://www.spinics.net/lists/arm-kernel/msg478167.html

The thing I don't like about this is the special case of checking
parent and parent->of_node to figure out whether we should use the
segment from ACPI and the fragility of depending on the fact that the
companion hasn't been set yet.

> http://www.spinics.net/lists/arm-kernel/msg478169.html
> 
> I like neither, we need the lesser of two evils though.

Today we call pci_bus_assign_domain_nr() from the PCI core (from
pci_create_root_bus()).  This is only implemented for
PCI_DOMAINS_GENERIC, but even so, it fiddles around to figure out
whether to get the domain from DT or to assign a new one.

That seems backwards to me.  The host bridge drivers already know
where the domain should come from (ACPI _SEG, DT, etc.) and in the
long term, I think they should be responsible for looking up or
assigning a domain number *before* they call pci_create_root_bus().

> > > +	if (ACPI_FAILURE(status) && status != AE_NOT_FOUND)
> > > +		dev_err(&acpi_dev->dev, "can't evaluate _SEG\n");
> > > +
> > > +	return segment;
> > > +}
> > > +
> > >  static void negotiate_os_control(struct acpi_pci_root *root, int *no_aspm)
> > >  {
> > >  	u32 support, control, requested;
> > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > > index 25e0327..1a74e87 100644
> > > --- a/drivers/pci/pci.c
> > > +++ b/drivers/pci/pci.c
> > > @@ -19,6 +19,7 @@
> > >  #include <linux/spinlock.h>
> > >  #include <linux/string.h>
> > >  #include <linux/log2.h>
> > > +#include <linux/pci-acpi.h>
> > >  #include <linux/pci-aspm.h>
> > >  #include <linux/pm_wakeup.h>
> > >  #include <linux/interrupt.h>
> > > @@ -4779,7 +4780,7 @@ int pci_get_new_domain_nr(void)
> > >  }
> > >  
> > >  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> > > -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > +static int of_pci_bus_domain_nr(struct device *parent)
> > >  {
> > >  	static int use_dt_domains = -1;
> > >  	int domain = -1;
> > > @@ -4823,7 +4824,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > >  		domain = -1;
> > >  	}
> > >  
> > > -	bus->domain_nr = domain;
> > > +	return domain;
> > > +}
> > > +
> > > +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > +{
> > > +	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
> > > +					 acpi_pci_bus_domain_nr(parent);

We have the pci_bus * here, so to_pci_host_bridge(bus->bridge) gives
us the struct pci_host_bridge.  I can't remember why we put domain_nr
in the struct pci_bus instead of in the struct pci_host_bridge.  It
seems like pci_host_bridge is the more logical place for it, because
every bus below the host bridge must have the same domain by
definition.

Would it be feasible to either (a) move domain_nr to the
pci_host_bridge, or (b) change acpi_pci_bus_domain_nr() so it uses the
struct pci_bus * or the struct device * to find the struct
acpi_pci_root where segment has already been stored by
acpi_pci_root_add()?

Another wrinkle is the quirk added by 1f09b09b4de0 ("x86/PCI: Ignore
_SEG on HP xw9300").  x86 doesn't use PCI_DOMAINS_GENERIC yet, so this
patch wouldn't break it, but I hope x86 can use PCI_DOMAINS_GENERIC in
the future, and then it will be a problem if we evaluate _SEG again.

> > >  }
> > >  #endif
> > >  #endif
> > > diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> > > index 89ab057..a72e22d 100644
> > > --- a/include/linux/pci-acpi.h
> > > +++ b/include/linux/pci-acpi.h
> > > @@ -22,6 +22,7 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
> > >  {
> > >  	return acpi_remove_pm_notifier(dev);
> > >  }
> > > +extern int acpi_pci_bus_domain_nr(struct device *parent);
> > >  extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
> > >  
> > >  static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
> > > @@ -109,6 +110,7 @@ extern const u8 pci_acpi_dsm_uuid[];
> > >  #else	/* CONFIG_ACPI */
> > >  static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
> > >  static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
> > > +static inline int acpi_pci_bus_domain_nr(struct device *parent) { return -1; }
> > >  #endif	/* CONFIG_ACPI */
> > >  
> > >  #ifdef CONFIG_ACPI_APEI
> > > -- 
> > > 1.9.1
> > > 
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Lorenzo Pieralisi]

On Wed, Apr 27, 2016 at 11:44:53AM -0500, Bjorn Helgaas wrote:
> On Wed, Apr 27, 2016 at 12:17:58PM +0100, Lorenzo Pieralisi wrote:
> > On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
> > > On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
> > > > As we now have valid PCI host bridge device reference we can
> > > > introduce code that is going to find its bus domain number using
> > > > ACPI _SEG method.
> > > > 
> > > > Note that _SEG method is optional, therefore _SEG absence means
> > > > that all PCI buses belong to domain 0.
> > > > 
> > > > While at it, for the sake of code clarity we put ACPI and DT domain
> > > > assign methods into the corresponding helpers.
> > > > 
> > > > Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> > > > Reviewed-by: Liviu Dudau <Liviu.Dudau@arm.com>
> > > > Tested-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> > > > Tested-by: Jeremy Linton <jeremy.linton@arm.com>
> > > > Tested-by: Duc Dang <dhdang@apm.com>
> > > > Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> > > > Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> > > > Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> > > > Tested-by: Sinan Kaya <okaya@codeaurora.org>
> > > > ---
> > > >  drivers/acpi/pci_root.c  | 18 ++++++++++++++++++
> > > >  drivers/pci/pci.c        | 11 +++++++++--
> > > >  include/linux/pci-acpi.h |  2 ++
> > > >  3 files changed, 29 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> > > > index 4581e0e..d9a70c4 100644
> > > > --- a/drivers/acpi/pci_root.c
> > > > +++ b/drivers/acpi/pci_root.c
> > > > @@ -419,6 +419,24 @@ out:
> > > >  }
> > > >  EXPORT_SYMBOL(acpi_pci_osc_control_set);
> > > >  
> > > > +int acpi_pci_bus_domain_nr(struct device *parent)
> 
> It looks like acpi_pci_bus_domain_nr() could be under #ifdef
> CONFIG_PCI_DOMAINS_GENERIC, right?

Yes it should.

> > > > +{
> > > > +	struct acpi_device *acpi_dev = to_acpi_device(parent);
> > > > +	unsigned long long segment = 0;
> > > > +	acpi_status status;
> > > > +
> > > > +	/*
> > > > +	 * If _SEG method does not exist, following ACPI spec (6.5.6)
> > > > +	 * all PCI buses belong to domain 0.
> > > > +	 */
> > > > +	status = acpi_evaluate_integer(acpi_dev->handle, METHOD_NAME__SEG, NULL,
> > > > +				       &segment);
> > > 
> > > We already have code in acpi_pci_root_add() to evaluate _SEG.  We
> > > don't want to evaluate it *twice*, do we?
> > > 
> > > I was sort of expecting that if you added it here, we'd remove the
> > > existing call, but it looks like you're keeping both?
> > 
> > We can't remove the existing call, since it is used on X86 and IA64
> > to store the segment number that, in the process, is used in their
> > pci_domain_nr() arch specific callback to retrieve the domain nr.
> > 
> > On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
> > to retrieve the domain number that is not arch dependent, since
> > this is generic code, we can't rely on any bus->sysdata format (unless
> > we do something like JC did below), therefore the only way is to call
> > the _SEG method *again* here, which also forced Tomasz to go through
> > the ACPI_COMPANION setting song and dance and pass the parent pointer
> > to pci_create_root_bus() (see patch 1), which BTW is a source of
> > trouble on its own as you noticed.
> > 
> > JC solved it differently, via sysdata and pseudo-generic code:
> > 
> > http://www.spinics.net/lists/arm-kernel/msg478167.html
> 
> The thing I don't like about this is the special case of checking
> parent and parent->of_node to figure out whether we should use the
> segment from ACPI and the fragility of depending on the fact that the
> companion hasn't been set yet.
> 
> > http://www.spinics.net/lists/arm-kernel/msg478169.html
> > 
> > I like neither, we need the lesser of two evils though.
> 
> Today we call pci_bus_assign_domain_nr() from the PCI core (from
> pci_create_root_bus()).  This is only implemented for
> PCI_DOMAINS_GENERIC, but even so, it fiddles around to figure out
> whether to get the domain from DT or to assign a new one.
> 
> That seems backwards to me.  The host bridge drivers already know
> where the domain should come from (ACPI _SEG, DT, etc.) and in the
> long term, I think they should be responsible for looking up or
> assigning a domain number *before* they call pci_create_root_bus().

Yes, the question still is how pci_create_root_bus() can get that
value (I am pretty certain this was heavily debated in the past, which
does not mean we can't give it another try).

> > > > +	if (ACPI_FAILURE(status) && status != AE_NOT_FOUND)
> > > > +		dev_err(&acpi_dev->dev, "can't evaluate _SEG\n");
> > > > +
> > > > +	return segment;
> > > > +}
> > > > +
> > > >  static void negotiate_os_control(struct acpi_pci_root *root, int *no_aspm)
> > > >  {
> > > >  	u32 support, control, requested;
> > > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > > > index 25e0327..1a74e87 100644
> > > > --- a/drivers/pci/pci.c
> > > > +++ b/drivers/pci/pci.c
> > > > @@ -19,6 +19,7 @@
> > > >  #include <linux/spinlock.h>
> > > >  #include <linux/string.h>
> > > >  #include <linux/log2.h>
> > > > +#include <linux/pci-acpi.h>
> > > >  #include <linux/pci-aspm.h>
> > > >  #include <linux/pm_wakeup.h>
> > > >  #include <linux/interrupt.h>
> > > > @@ -4779,7 +4780,7 @@ int pci_get_new_domain_nr(void)
> > > >  }
> > > >  
> > > >  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> > > > -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > > +static int of_pci_bus_domain_nr(struct device *parent)
> > > >  {
> > > >  	static int use_dt_domains = -1;
> > > >  	int domain = -1;
> > > > @@ -4823,7 +4824,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > >  		domain = -1;
> > > >  	}
> > > >  
> > > > -	bus->domain_nr = domain;
> > > > +	return domain;
> > > > +}
> > > > +
> > > > +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > > +{
> > > > +	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
> > > > +					 acpi_pci_bus_domain_nr(parent);
> 
> We have the pci_bus * here, so to_pci_host_bridge(bus->bridge) gives
> us the struct pci_host_bridge.  I can't remember why we put domain_nr
> in the struct pci_bus instead of in the struct pci_host_bridge.  It
> seems like pci_host_bridge is the more logical place for it, because
> every bus below the host bridge must have the same domain by
> definition.
> 
> Would it be feasible to either (a) move domain_nr to the
> pci_host_bridge, or (b) change acpi_pci_bus_domain_nr() so it uses the
> struct pci_bus * or the struct device * to find the struct
> acpi_pci_root where segment has already been stored by
> acpi_pci_root_add()?

(b) is what JC implemented even though it works differently for
different hosts since it all depends on what's in bus->sysdata.

It can certainly be done in a generic way (that works on X86 and IA64
too), let's give it more thought.

> Another wrinkle is the quirk added by 1f09b09b4de0 ("x86/PCI: Ignore
> _SEG on HP xw9300").  x86 doesn't use PCI_DOMAINS_GENERIC yet, so this
> patch wouldn't break it, but I hope x86 can use PCI_DOMAINS_GENERIC in
> the future, and then it will be a problem if we evaluate _SEG again.

Yes, I share your concern here and I thought about that, if that's the
end goal let's find a solution that works across arches (or we temporarily
use JC's code and we then generalize it).

Thanks,
Lorenzo

> 
> > > >  }
> > > >  #endif
> > > >  #endif
> > > > diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> > > > index 89ab057..a72e22d 100644
> > > > --- a/include/linux/pci-acpi.h
> > > > +++ b/include/linux/pci-acpi.h
> > > > @@ -22,6 +22,7 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
> > > >  {
> > > >  	return acpi_remove_pm_notifier(dev);
> > > >  }
> > > > +extern int acpi_pci_bus_domain_nr(struct device *parent);
> > > >  extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
> > > >  
> > > >  static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
> > > > @@ -109,6 +110,7 @@ extern const u8 pci_acpi_dsm_uuid[];
> > > >  #else	/* CONFIG_ACPI */
> > > >  static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
> > > >  static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
> > > > +static inline int acpi_pci_bus_domain_nr(struct device *parent) { return -1; }
> > > >  #endif	/* CONFIG_ACPI */
> > > >  
> > > >  #ifdef CONFIG_ACPI_APEI
> > > > -- 
> > > > 1.9.1
> > > > 
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> > > > the body of a message to majordomo@vger.kernel.org
> > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > 
> 

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Liviu.Dudau]

On Wed, Apr 27, 2016 at 06:31:29PM +0100, Lorenzo Pieralisi wrote:
> On Wed, Apr 27, 2016 at 11:44:53AM -0500, Bjorn Helgaas wrote:
> > On Wed, Apr 27, 2016 at 12:17:58PM +0100, Lorenzo Pieralisi wrote:
> > > On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
> > > > On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
> > > > > As we now have valid PCI host bridge device reference we can
> > > > > introduce code that is going to find its bus domain number using
> > > > > ACPI _SEG method.
> > > > > 
> > > > > Note that _SEG method is optional, therefore _SEG absence means
> > > > > that all PCI buses belong to domain 0.
> > > > > 
> > > > > While at it, for the sake of code clarity we put ACPI and DT domain
> > > > > assign methods into the corresponding helpers.
> > > > > 
> > > > > Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> > > > > Reviewed-by: Liviu Dudau <Liviu.Dudau@arm.com>
> > > > > Tested-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> > > > > Tested-by: Jeremy Linton <jeremy.linton@arm.com>
> > > > > Tested-by: Duc Dang <dhdang@apm.com>
> > > > > Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> > > > > Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> > > > > Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> > > > > Tested-by: Sinan Kaya <okaya@codeaurora.org>
> > > > > ---
> > > > >  drivers/acpi/pci_root.c  | 18 ++++++++++++++++++
> > > > >  drivers/pci/pci.c        | 11 +++++++++--
> > > > >  include/linux/pci-acpi.h |  2 ++
> > > > >  3 files changed, 29 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> > > > > index 4581e0e..d9a70c4 100644
> > > > > --- a/drivers/acpi/pci_root.c
> > > > > +++ b/drivers/acpi/pci_root.c
> > > > > @@ -419,6 +419,24 @@ out:
> > > > >  }
> > > > >  EXPORT_SYMBOL(acpi_pci_osc_control_set);
> > > > >  
> > > > > +int acpi_pci_bus_domain_nr(struct device *parent)
> > 
> > It looks like acpi_pci_bus_domain_nr() could be under #ifdef
> > CONFIG_PCI_DOMAINS_GENERIC, right?
> 
> Yes it should.
> 
> > > > > +{
> > > > > +	struct acpi_device *acpi_dev = to_acpi_device(parent);
> > > > > +	unsigned long long segment = 0;
> > > > > +	acpi_status status;
> > > > > +
> > > > > +	/*
> > > > > +	 * If _SEG method does not exist, following ACPI spec (6.5.6)
> > > > > +	 * all PCI buses belong to domain 0.
> > > > > +	 */
> > > > > +	status = acpi_evaluate_integer(acpi_dev->handle, METHOD_NAME__SEG, NULL,
> > > > > +				       &segment);
> > > > 
> > > > We already have code in acpi_pci_root_add() to evaluate _SEG.  We
> > > > don't want to evaluate it *twice*, do we?
> > > > 
> > > > I was sort of expecting that if you added it here, we'd remove the
> > > > existing call, but it looks like you're keeping both?
> > > 
> > > We can't remove the existing call, since it is used on X86 and IA64
> > > to store the segment number that, in the process, is used in their
> > > pci_domain_nr() arch specific callback to retrieve the domain nr.
> > > 
> > > On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
> > > to retrieve the domain number that is not arch dependent, since
> > > this is generic code, we can't rely on any bus->sysdata format (unless
> > > we do something like JC did below), therefore the only way is to call
> > > the _SEG method *again* here, which also forced Tomasz to go through
> > > the ACPI_COMPANION setting song and dance and pass the parent pointer
> > > to pci_create_root_bus() (see patch 1), which BTW is a source of
> > > trouble on its own as you noticed.
> > > 
> > > JC solved it differently, via sysdata and pseudo-generic code:
> > > 
> > > http://www.spinics.net/lists/arm-kernel/msg478167.html
> > 
> > The thing I don't like about this is the special case of checking
> > parent and parent->of_node to figure out whether we should use the
> > segment from ACPI and the fragility of depending on the fact that the
> > companion hasn't been set yet.
> > 
> > > http://www.spinics.net/lists/arm-kernel/msg478169.html
> > > 
> > > I like neither, we need the lesser of two evils though.
> > 
> > Today we call pci_bus_assign_domain_nr() from the PCI core (from
> > pci_create_root_bus()).  This is only implemented for
> > PCI_DOMAINS_GENERIC, but even so, it fiddles around to figure out
> > whether to get the domain from DT or to assign a new one.
> > 
> > That seems backwards to me.  The host bridge drivers already know
> > where the domain should come from (ACPI _SEG, DT, etc.) and in the
> > long term, I think they should be responsible for looking up or
> > assigning a domain number *before* they call pci_create_root_bus().
> 
> Yes, the question still is how pci_create_root_bus() can get that
> value (I am pretty certain this was heavily debated in the past, which
> does not mean we can't give it another try).

The main issue is that pci_create_root_bus() does a weird dance trying to
figure out if the root bus hasn't been already allocated. It allocates a
new bus, assigns a domain number and then it tries to find it in the list
of already allocated busses. Because pci_alloc_bus() does not pass any
additional information, pci_bus_assign_domain_nr() needs to try to guess
where the barely initialised bus should live and give you back a number.

Simplifying the creation of root busses to be the job of the host bridges
would greatly simplify the code as well.

Best regards,
Liviu

> 
> > > > > +	if (ACPI_FAILURE(status) && status != AE_NOT_FOUND)
> > > > > +		dev_err(&acpi_dev->dev, "can't evaluate _SEG\n");
> > > > > +
> > > > > +	return segment;
> > > > > +}
> > > > > +
> > > > >  static void negotiate_os_control(struct acpi_pci_root *root, int *no_aspm)
> > > > >  {
> > > > >  	u32 support, control, requested;
> > > > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > > > > index 25e0327..1a74e87 100644
> > > > > --- a/drivers/pci/pci.c
> > > > > +++ b/drivers/pci/pci.c
> > > > > @@ -19,6 +19,7 @@
> > > > >  #include <linux/spinlock.h>
> > > > >  #include <linux/string.h>
> > > > >  #include <linux/log2.h>
> > > > > +#include <linux/pci-acpi.h>
> > > > >  #include <linux/pci-aspm.h>
> > > > >  #include <linux/pm_wakeup.h>
> > > > >  #include <linux/interrupt.h>
> > > > > @@ -4779,7 +4780,7 @@ int pci_get_new_domain_nr(void)
> > > > >  }
> > > > >  
> > > > >  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> > > > > -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > > > +static int of_pci_bus_domain_nr(struct device *parent)
> > > > >  {
> > > > >  	static int use_dt_domains = -1;
> > > > >  	int domain = -1;
> > > > > @@ -4823,7 +4824,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > > >  		domain = -1;
> > > > >  	}
> > > > >  
> > > > > -	bus->domain_nr = domain;
> > > > > +	return domain;
> > > > > +}
> > > > > +
> > > > > +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > > > +{
> > > > > +	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
> > > > > +					 acpi_pci_bus_domain_nr(parent);
> > 
> > We have the pci_bus * here, so to_pci_host_bridge(bus->bridge) gives
> > us the struct pci_host_bridge.  I can't remember why we put domain_nr
> > in the struct pci_bus instead of in the struct pci_host_bridge.  It
> > seems like pci_host_bridge is the more logical place for it, because
> > every bus below the host bridge must have the same domain by
> > definition.
> > 
> > Would it be feasible to either (a) move domain_nr to the
> > pci_host_bridge, or (b) change acpi_pci_bus_domain_nr() so it uses the
> > struct pci_bus * or the struct device * to find the struct
> > acpi_pci_root where segment has already been stored by
> > acpi_pci_root_add()?
> 
> (b) is what JC implemented even though it works differently for
> different hosts since it all depends on what's in bus->sysdata.
> 
> It can certainly be done in a generic way (that works on X86 and IA64
> too), let's give it more thought.
> 
> > Another wrinkle is the quirk added by 1f09b09b4de0 ("x86/PCI: Ignore
> > _SEG on HP xw9300").  x86 doesn't use PCI_DOMAINS_GENERIC yet, so this
> > patch wouldn't break it, but I hope x86 can use PCI_DOMAINS_GENERIC in
> > the future, and then it will be a problem if we evaluate _SEG again.
> 
> Yes, I share your concern here and I thought about that, if that's the
> end goal let's find a solution that works across arches (or we temporarily
> use JC's code and we then generalize it).
> 
> Thanks,
> Lorenzo
> 
> > 
> > > > >  }
> > > > >  #endif
> > > > >  #endif
> > > > > diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> > > > > index 89ab057..a72e22d 100644
> > > > > --- a/include/linux/pci-acpi.h
> > > > > +++ b/include/linux/pci-acpi.h
> > > > > @@ -22,6 +22,7 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
> > > > >  {
> > > > >  	return acpi_remove_pm_notifier(dev);
> > > > >  }
> > > > > +extern int acpi_pci_bus_domain_nr(struct device *parent);
> > > > >  extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
> > > > >  
> > > > >  static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
> > > > > @@ -109,6 +110,7 @@ extern const u8 pci_acpi_dsm_uuid[];
> > > > >  #else	/* CONFIG_ACPI */
> > > > >  static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
> > > > >  static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
> > > > > +static inline int acpi_pci_bus_domain_nr(struct device *parent) { return -1; }
> > > > >  #endif	/* CONFIG_ACPI */
> > > > >  
> > > > >  #ifdef CONFIG_ACPI_APEI
> > > > > -- 
> > > > > 1.9.1
> > > > > 
> > > > > --
> > > > > To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> > > > > the body of a message to majordomo@vger.kernel.org
> > > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > > 
> > 
> 

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Bjorn Helgaas]

On Wed, Apr 27, 2016 at 06:31:29PM +0100, Lorenzo Pieralisi wrote:
> On Wed, Apr 27, 2016 at 11:44:53AM -0500, Bjorn Helgaas wrote:
> > On Wed, Apr 27, 2016 at 12:17:58PM +0100, Lorenzo Pieralisi wrote:
> > > On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
> > > > On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:


> > Today we call pci_bus_assign_domain_nr() from the PCI core (from
> > pci_create_root_bus()).  This is only implemented for
> > PCI_DOMAINS_GENERIC, but even so, it fiddles around to figure out
> > whether to get the domain from DT or to assign a new one.
> > 
> > That seems backwards to me.  The host bridge drivers already know
> > where the domain should come from (ACPI _SEG, DT, etc.) and in the
> > long term, I think they should be responsible for looking up or
> > assigning a domain number *before* they call pci_create_root_bus().
> 
> Yes, the question still is how pci_create_root_bus() can get that
> value (I am pretty certain this was heavily debated in the past, which
> does not mean we can't give it another try).

Right, we don't have a good mechanism for passing more info into
pci_create_root_bus().  Maybe the caller could fill in a struct so we
have a chance to extend it without having to change all the existing
callers.

I wonder if there's a design pattern we can copy, e.g., would
something like the scsi_host_alloc(), scsi_add_host(),
scsi_scan_host() model work here?

> > > > > +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > > > +{
> > > > > +	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
> > > > > +					 acpi_pci_bus_domain_nr(parent);
> > 
> > We have the pci_bus * here, so to_pci_host_bridge(bus->bridge) gives
> > us the struct pci_host_bridge.  I can't remember why we put domain_nr
> > in the struct pci_bus instead of in the struct pci_host_bridge.  It
> > seems like pci_host_bridge is the more logical place for it, because
> > every bus below the host bridge must have the same domain by
> > definition.
> > 
> > Would it be feasible to either (a) move domain_nr to the
> > pci_host_bridge, or (b) change acpi_pci_bus_domain_nr() so it uses the
> > struct pci_bus * or the struct device * to find the struct
> > acpi_pci_root where segment has already been stored by
> > acpi_pci_root_add()?
> 
> (b) is what JC implemented even though it works differently for
> different hosts since it all depends on what's in bus->sysdata.
> 
> It can certainly be done in a generic way (that works on X86 and IA64
> too), let's give it more thought.
> 
> > Another wrinkle is the quirk added by 1f09b09b4de0 ("x86/PCI: Ignore
> > _SEG on HP xw9300").  x86 doesn't use PCI_DOMAINS_GENERIC yet, so this
> > patch wouldn't break it, but I hope x86 can use PCI_DOMAINS_GENERIC in
> > the future, and then it will be a problem if we evaluate _SEG again.
> 
> Yes, I share your concern here and I thought about that, if that's the
> end goal let's find a solution that works across arches (or we temporarily
> use JC's code and we then generalize it).

I would ultimately like all arches to use PCI_DOMAINS_GENERIC, because
I don't think there's anything intrisically arch-specific about where
we store the domain number.  The means of discovering or assigning a
domain number might be arch-specific, but I think it would be cleanest
if the host bridge driver handled that.

Bjorn

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Arnd Bergmann]

On Thursday 28 April 2016 10:12:12 Bjorn Helgaas wrote:
> On Wed, Apr 27, 2016 at 06:31:29PM +0100, Lorenzo Pieralisi wrote:
> > On Wed, Apr 27, 2016 at 11:44:53AM -0500, Bjorn Helgaas wrote:
> > > On Wed, Apr 27, 2016 at 12:17:58PM +0100, Lorenzo Pieralisi wrote:
> > > > On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
> > > > > On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
> 
> 
> > > Today we call pci_bus_assign_domain_nr() from the PCI core (from
> > > pci_create_root_bus()).  This is only implemented for
> > > PCI_DOMAINS_GENERIC, but even so, it fiddles around to figure out
> > > whether to get the domain from DT or to assign a new one.
> > > 
> > > That seems backwards to me.  The host bridge drivers already know
> > > where the domain should come from (ACPI _SEG, DT, etc.) and in the
> > > long term, I think they should be responsible for looking up or
> > > assigning a domain number *before* they call pci_create_root_bus().
> > 
> > Yes, the question still is how pci_create_root_bus() can get that
> > value (I am pretty certain this was heavily debated in the past, which
> > does not mean we can't give it another try).
> 
> Right, we don't have a good mechanism for passing more info into
> pci_create_root_bus().  Maybe the caller could fill in a struct so we
> have a chance to extend it without having to change all the existing
> callers.
> 
> I wonder if there's a design pattern we can copy, e.g., would
> something like the scsi_host_alloc(), scsi_add_host(),
> scsi_scan_host() model work here?

Yes, I think that is a good idea in general. Especially
now that we have separate the ARM code from pci_common_init_dev
and pci_sys_data, that can help with cleanups in the other drivers
as well.

I see two common variations in other subsystems: some use a
special alloc() function that you pass the size of the private
data into, while others just expect you to embed a structure
inside of the driver specific one allocate that separately to
have the generic registration function fill out the common fields.

I have a slight preference for the second, but they are really
the same thing basically.

	Arnd

Re: [PATCH V6 05/13] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Bjorn Helgaas]

On Wed, Apr 27, 2016 at 03:26:59PM +0100, Lorenzo Pieralisi wrote:
> On Tue, Apr 26, 2016 at 09:39:16PM -0500, Bjorn Helgaas wrote:
> > On Fri, Apr 15, 2016 at 07:06:40PM +0200, Tomasz Nowicki wrote:
> > > Platforms that have memory mapped IO port (such as ARM64) need special
> > > handling for PCI I/O resources. For host bridge's resource probing case
> > > these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
> > 
> > ia64 also has memory-mapped I/O port space.  It would be ideal to find
> > some way to handle ia64 and ARM64 similarly.  At the very least, we
> > have to make sure that this doesn't break ia64.  The ia64 dense/sparse
> > I/O spaces complicate things; I don't know if ARM64 has something
> > similar or not.
> 
> No it does not, and that's exactly the same problem we faced with
> the DT generic version of of_pci_range_to_resource() which basically
> relies on PCI_IOBASE to be defined to add code that creates IO port
> resources out of the MMIO resource describing how IO port space is
> mapped to MMIO (physical) address space.

Mapping IO port space into MMIO space is pretty common since most
arches don't have inb/outb instructions like x86 does.  Several arches
(at least ia64 and parisc) have some sort of sparse mapping, but my
point is that the "dense" mapping (one byte MMIO space per IO port) is
pretty similar across arches.

There are differences in how we compute the MMIO address from the IO
port number, of course, e.g., on arm64 the CPU virtual MMIO address is
a simple offset from the IO port number, i.e., "vaddr = PCI_IOBASE +
res->start" in pci_remap_iospace(), while on ia64, that CPU address is
less constrained, i.e., "vaddr = space->mmio_base | port" in
__ia64_mk_io_addr().

> IIRC everything hinges on PCI_IOBASE definition to make sure that
> of_pci_range_to_resource() *works*, which means that if PCI_IOBASE is
> not defined (ie IA64) that code - acpi_pci_root_remap_iospace() in this
> case - does nothing.

OK.  That's confusing to read, but I see that it probably works.

> So acpi_pci_root_remap_iospace() is of_pci_range_to_resource() ACPI
> equivalent + the pci_remap_iospace() call (I have to dig into the
> logs to check why Liviu did not add a call to pci_remap_iospace()
> in of_pci_get_host_bridge_resources() - I want to do that actually).
> 
> The point here is: IO space (in DT and ACPI) handling is arch specific.
> 
> For DT, by relying on PCI_IOBASE, we left that code in drivers/of and
> it works (well, with some niggles - see the thread with Murali on IO
> space on TI keystone) for ARM/ARM64.
> 
> http://www.spinics.net/lists/linux-pci/msg49725.html
> 
> What are we going to do with the ACPI version ?
> 
> Do we want to add an arch specific call that takes the raw resource
> describing IO space and creates an IO port resource (and the MMIO
> equivalent - that's what add_io_space() does in IA64) and use that
> in generic ACPI parsing code ?

There's a lot of non-arch-specific stuff here, which is what's
bothering me.  The arch-specific parts are:

  - discovering IO port region (bus address start and size)
  - discovering MMIO mapping address and size (CPU "mem" resource)
  - discovering or assigning IO port base (CPU "io" resource)
  - setting up whatever structures arch uses to implement inb()

Once you have the CPU "mem" and "io" resources, the code to insert
them into iomem_resource and ioport_resource and ioremap() the MMIO
space should be pretty generic.  Right now that code is scattered
through the arches and most of them don't do it correctly, e.g.,
typically they don't request the "mem" resource.

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Bjorn Helgaas]

On Thu, Apr 21, 2016 at 11:36:53AM +0200, Arnd Bergmann wrote:
> On Thursday 21 April 2016 11:28:15 Tomasz Nowicki wrote:
> > On 19.04.2016 15:06, Arnd Bergmann wrote:
> > > On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
> > >>
> > >> Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.
> > >>
> > >> pci-thunder-ecam.c contains config space accessors. Similar for
> > >> pci-thunder-pem.c but it also has extra init call (it is now called
> > >> thunder_pem_init) which finds and maps related registers.
> > >
> > > They seem to do much more than just override the accessors, they actually
> > > change the contents of the config space as well. Is that really necessary
> > > on ACPI based systems as well?
> > 
> > Yes, the pci-thunder-ecam.c accessors are meant to emulate config space 
> > capabilities. They are necessary to synthesize EA capabilities (fixed 
> > PCI BARs), it wont work without this, for ACPI boot as well.
> 
> Why is that? I thought the BARs never get reassigned when using ACPI,

In general, there's no reason we can't reassign BARs, whether we're
using DT, ACPI, or whatever.  In many cases, systems with ACPI also
assign all the BARs in firmware, and Linux doesn't reassign them
unless it needs to.  But that's just a coincidence.  There's no
requirement that Linux leave BARs as firmware programmed them.

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Arnd Bergmann]

On Thursday 28 April 2016 15:14:39 Bjorn Helgaas wrote:
> On Thu, Apr 21, 2016 at 11:36:53AM +0200, Arnd Bergmann wrote:
> > On Thursday 21 April 2016 11:28:15 Tomasz Nowicki wrote:
> > > On 19.04.2016 15:06, Arnd Bergmann wrote:
> > > > On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
> > > >>
> > > >> Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.
> > > >>
> > > >> pci-thunder-ecam.c contains config space accessors. Similar for
> > > >> pci-thunder-pem.c but it also has extra init call (it is now called
> > > >> thunder_pem_init) which finds and maps related registers.
> > > >
> > > > They seem to do much more than just override the accessors, they actually
> > > > change the contents of the config space as well. Is that really necessary
> > > > on ACPI based systems as well?
> > > 
> > > Yes, the pci-thunder-ecam.c accessors are meant to emulate config space 
> > > capabilities. They are necessary to synthesize EA capabilities (fixed 
> > > PCI BARs), it wont work without this, for ACPI boot as well.
> > 
> > Why is that? I thought the BARs never get reassigned when using ACPI,
> 
> In general, there's no reason we can't reassign BARs, whether we're
> using DT, ACPI, or whatever.  In many cases, systems with ACPI also
> assign all the BARs in firmware, and Linux doesn't reassign them
> unless it needs to.  But that's just a coincidence.  There's no
> requirement that Linux leave BARs as firmware programmed them.

I'm thought I've seen systems in which the ACPI BIOS assumes that
certain PCI devices never move around, because it pokes the registers
from AML, and changing them would require never using the same device
through ACPI. It's likely that this is against some standard, but that
won't help you if you have to deal with the system anyway.

	Arnd

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Bjorn Helgaas]

On Thu, Apr 28, 2016 at 10:40:35PM +0200, Arnd Bergmann wrote:
> On Thursday 28 April 2016 15:14:39 Bjorn Helgaas wrote:
> > On Thu, Apr 21, 2016 at 11:36:53AM +0200, Arnd Bergmann wrote:
> > > On Thursday 21 April 2016 11:28:15 Tomasz Nowicki wrote:
> > > > On 19.04.2016 15:06, Arnd Bergmann wrote:
> > > > > On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
> > > > >>
> > > > >> Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.
> > > > >>
> > > > >> pci-thunder-ecam.c contains config space accessors. Similar for
> > > > >> pci-thunder-pem.c but it also has extra init call (it is now called
> > > > >> thunder_pem_init) which finds and maps related registers.
> > > > >
> > > > > They seem to do much more than just override the accessors, they actually
> > > > > change the contents of the config space as well. Is that really necessary
> > > > > on ACPI based systems as well?
> > > > 
> > > > Yes, the pci-thunder-ecam.c accessors are meant to emulate config space 
> > > > capabilities. They are necessary to synthesize EA capabilities (fixed 
> > > > PCI BARs), it wont work without this, for ACPI boot as well.
> > > 
> > > Why is that? I thought the BARs never get reassigned when using ACPI,
> > 
> > In general, there's no reason we can't reassign BARs, whether we're
> > using DT, ACPI, or whatever.  In many cases, systems with ACPI also
> > assign all the BARs in firmware, and Linux doesn't reassign them
> > unless it needs to.  But that's just a coincidence.  There's no
> > requirement that Linux leave BARs as firmware programmed them.
> 
> I'm thought I've seen systems in which the ACPI BIOS assumes that
> certain PCI devices never move around, because it pokes the registers
> from AML, and changing them would require never using the same device
> through ACPI. It's likely that this is against some standard, but that
> won't help you if you have to deal with the system anyway.

Yes, I'm pretty sure there are systems like that, e.g., I think SMM
code on some HP servers assumes the iLO address never changes.  I
think that is a firmware defect because I don't think there's any spec
that says firmware retains control over PCI BARs after handoff.  And
this particular case isn't really ACPI-specific.

But as you say, we have to deal with these systems anyway, even if we
consider that behavior broken.  My proposal has been to add quirks to
mark those devices as IORESOURCE_PCI_FIXED, but I don't think anybody
has gotten around to doing that.

Bjorn

[PATCH] acpi: pci: QDF2432 32 bit config space accessors

[Christopher Covington]

Qualcomm Technologies QDF2432 SoCs require 32 bit accessors to be used
for the PCI configuration space. Register the appropriate quirk.

Signed-off-by: Christopher Covington <cov@codeaurora.org>
---
Depends on Tomasz Nowicki's latest series:
[PATCH V6 00/13] Support for generic ACPI based PCI host controller
---
 drivers/acpi/pci_gen_host.c | 13 +++++++++++++
 drivers/pci/ecam.c          | 10 ++++++++++
 drivers/pci/ecam.h          |  3 +++
 3 files changed, 26 insertions(+)

diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
index e55dfca..daf8a86 100644
--- a/drivers/acpi/pci_gen_host.c
+++ b/drivers/acpi/pci_gen_host.c
@@ -257,3 +257,16 @@ int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int devfn,
 		return PCIBIOS_DEVICE_NOT_FOUND;
 	return bus->ops->write(bus, devfn, reg, len, val);
 }
+
+static const struct dmi_system_id qcom_qdf2432[] = {
+	{
+		.ident = "Qualcomm Technologies QDF2432",
+		.matches = {
+			DMI_MATCH(DMI_SYS_VENDOR, "Qualcomm"),
+			DMI_MATCH(DMI_PRODUCT_NAME, "QDF2432"),
+		},
+	},
+	{ }
+};
+
+DECLARE_ACPI_MCFG_FIXUP(qcom_qdf2432, NULL, &pci_generic_ecam_32b_ops, PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index ff04c01..720a038 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -135,3 +135,13 @@ struct pci_generic_ecam_ops pci_generic_ecam_default_ops = {
 		.write		= pci_generic_config_write,
 	}
 };
+
+/* ECAM ops with 32 bit config space access quirk */
+struct pci_generic_ecam_ops pci_generic_ecam_32b_ops = {
+	.bus_shift	= 20,
+	.pci_ops		= {
+		.map_bus	= pci_generic_ecam_map_bus,
+		.read		= pci_generic_config_read32,
+		.write		= pci_generic_config_write32,
+	}
+};
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 706621a..4af764f 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -58,6 +58,9 @@ void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
 /* default ECAM ops, bus shift 20, generic read and write */
 extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
 
+/* ECAM ops with 32 bit config space access quirk */
+extern struct pci_generic_ecam_ops pci_generic_ecam_32b_ops;
+
 #ifdef CONFIG_PCI_HOST_GENERIC
 /* for DT based pci controllers that support ECAM */
 int pci_host_common_probe(struct platform_device *pdev,
-- 
Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: [PATCH] acpi: pci: QDF2432 32 bit config space accessors

[Rafael J. Wysocki]

On Thu, Apr 28, 2016 at 11:27 PM, Christopher Covington
<cov@codeaurora.org> wrote:
> Qualcomm Technologies QDF2432 SoCs require 32 bit accessors to be used
> for the PCI configuration space. Register the appropriate quirk.
>
> Signed-off-by: Christopher Covington <cov@codeaurora.org>
> ---
> Depends on Tomasz Nowicki's latest series:
> [PATCH V6 00/13] Support for generic ACPI based PCI host controller

Please send this when the above patch has been put into linux-next at least.

Sending it earlier doesn't serve any purpose and is adding to noise.

Thanks,
Rafael

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Jon Masters]

Hi Bjorn, Arnd, all,

On 04/28/2016 05:18 PM, Bjorn Helgaas wrote:
> On Thu, Apr 28, 2016 at 10:40:35PM +0200, Arnd Bergmann wrote:
>> On Thursday 28 April 2016 15:14:39 Bjorn Helgaas wrote:
>>> On Thu, Apr 21, 2016 at 11:36:53AM +0200, Arnd Bergmann wrote:
>>>> On Thursday 21 April 2016 11:28:15 Tomasz Nowicki wrote:
>>>>> On 19.04.2016 15:06, Arnd Bergmann wrote:
>>>>>> On Monday 18 April 2016 21:31:54 Tomasz Nowicki wrote:
>>>>>>>
>>>>>>> Basically the whole content of pci-thunder-ecam.c and pci-thunder-pem.c.
>>>>>>>
>>>>>>> pci-thunder-ecam.c contains config space accessors. Similar for
>>>>>>> pci-thunder-pem.c but it also has extra init call (it is now called
>>>>>>> thunder_pem_init) which finds and maps related registers.
>>>>>>
>>>>>> They seem to do much more than just override the accessors, they actually
>>>>>> change the contents of the config space as well. Is that really necessary
>>>>>> on ACPI based systems as well?
>>>>>
>>>>> Yes, the pci-thunder-ecam.c accessors are meant to emulate config space 
>>>>> capabilities. They are necessary to synthesize EA capabilities (fixed 
>>>>> PCI BARs), it wont work without this, for ACPI boot as well.
>>>>
>>>> Why is that? I thought the BARs never get reassigned when using ACPI,

Just to specifically jump in here and clarify this piece, which only
pertains to the specific platform's special extra host driver (which
generally speaking I am encouraging all future platforms not to do). In
other words, the following has nothing to do with the rest of the patch
series and is entirely down to one specific SoC and its implementation.

ThunderX supports two different methods of PCIe configuration space for
on-chip devices: with EA and without EA (which is being phased out). EA
(Enhanced Allocation) is a fancy way of saying "read only BARs". Intel
did the spec change for that in PCI SIG, so it wasn't us folks in the
ARM community doing something weird. The good folks at Cavium desired a
means to express their on-SoC hardware using PCI so that it was nice and
enumerable, but without full boat PCI. EA fit the bill better than just
wiring BARs as write ignore or whatever. Again, it's happening in many
cases and there must be a reason Intel wanted to get it also.

I believe pci-thunder-ecam.c contains code to support the older devices
that don't do full EA by faking the EA capabilities, but they can
clarify. The point is, this is a specific and separate issue with the
way one vendor has chosen to implement on-SoC devices as PCIe
discoverable but using the newer PCI EA extension. And then the quirk is
to handle that not every device that's out there yet has real EA.

>>> In general, there's no reason we can't reassign BARs, whether we're
>>> using DT, ACPI, or whatever.  In many cases, systems with ACPI also
>>> assign all the BARs in firmware, and Linux doesn't reassign them
>>> unless it needs to.  But that's just a coincidence.  There's no
>>> requirement that Linux leave BARs as firmware programmed them.

There's no requirement, generally, that PCI compliant devices with ECAM
can't be programmed with different base addresses. There's this PCI
change called EA that is disjoint and some vendors have chosen to use
it. We didn't catch that early in the definition of the SBSA for ARM,
but just as an aside, I have already suggested we require future
generations of chips to not use EA and only support writeable BARs (even
for the decoders in the on-SoC platformish devices doing "PCI"). This
isn't Cavium's fault - they did the right thing with the data at hand
and nobody really considered the impact of PCI getting EA added. Again,
that's something that will likely happen on x86 at some point (maybe it
already is, I don't get any data about future Intel stuff).

On the rest of the quirks and hacks. Without going into too much detail,
some "concerned citizens" are chatting with various folks to ensure that
many of these common quirks aren't needed in future parts.

>> I'm thought I've seen systems in which the ACPI BIOS assumes that
>> certain PCI devices never move around, because it pokes the registers
>> from AML, and changing them would require never using the same device
>> through ACPI. It's likely that this is against some standard, but that
>> won't help you if you have to deal with the system anyway.

Right. This has happened, I think, and there you're no worse off on ARM
than you would be on x86 if you had AML poking at something underneath.

> Yes, I'm pretty sure there are systems like that, e.g., I think SMM
> code on some HP servers assumes the iLO address never changes.  I
> think that is a firmware defect because I don't think there's any spec
> that says firmware retains control over PCI BARs after handoff.  And
> this particular case isn't really ACPI-specific.

If you substitute SMM for EL3 on ARM we're bound to eventually have the
same kinds of things happening on some systems. It's just life.

> But as you say, we have to deal with these systems anyway, even if we
> consider that behavior broken.  My proposal has been to add quirks to
> mark those devices as IORESOURCE_PCI_FIXED, but I don't think anybody
> has gotten around to doing that.

Good to know.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

Re: [PATCH V6 07/13] PCI: Provide common functions for ECAM mapping

[Bjorn Helgaas]

On Fri, Apr 15, 2016 at 07:06:42PM +0200, Tomasz Nowicki wrote:
> From: Jayachandran C <jchandra@broadcom.com>
> 
> Add config option PCI_GENERIC_ECAM and file drivers/pci/ecam.c to
> provide generic functions for accessing memory mapped PCI config space.
> 
> The API is defined in drivers/pci/ecam.h and is written to replace the
> API in drivers/pci/host/pci-host-common.h. The file defines a new
> 'struct pci_config_window' to hold the information related to a PCI
> config area and its mapping. This structure is expected to be used as
> sysdata for controllers that have ECAM based mapping.
> 
> Helper functions are provided to setup the mapping, free the mapping
> and to implement the map_bus method in 'struct pci_ops'

Spec reference: PCI Express Base Specification, rev 3.0, sec 7.2.2.

> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> ---
>  drivers/pci/Kconfig  |   3 ++
>  drivers/pci/Makefile |   2 +
>  drivers/pci/ecam.c   | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  drivers/pci/ecam.h   |  61 +++++++++++++++++++++++
>  4 files changed, 203 insertions(+)
>  create mode 100644 drivers/pci/ecam.c
>  create mode 100644 drivers/pci/ecam.h
> 
> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
> index 209292e..e930d62 100644
> --- a/drivers/pci/Kconfig
> +++ b/drivers/pci/Kconfig
> @@ -83,6 +83,9 @@ config HT_IRQ
>  config PCI_ATS
>  	bool
>  
> +config PCI_GENERIC_ECAM
> +	bool

"PCI_ECAM" is enough, I think.  It's defined by and required by the
spec unless there's some arch-specific interface.  Plus, if I
understand correctly, this infrastructure supports non-generic ECAM
implementations as well, since the caller supplies "struct
pci_generic_ecam_ops *ops".

>  config PCI_IOV
>  	bool "PCI IOV support"
>  	depends on PCI
> diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
> index 2154092..810aec8 100644
> --- a/drivers/pci/Makefile
> +++ b/drivers/pci/Makefile
> @@ -55,6 +55,8 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o
>  
>  obj-$(CONFIG_PCI_STUB) += pci-stub.o
>  
> +obj-$(CONFIG_PCI_GENERIC_ECAM) += ecam.o
> +
>  obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
>  
>  obj-$(CONFIG_OF) += of.o
> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> new file mode 100644
> index 0000000..ff04c01
> --- /dev/null
> +++ b/drivers/pci/ecam.c
> @@ -0,0 +1,137 @@
> +/*
> + * Copyright 2016 Broadcom
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation (the "GPL").
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 (GPLv2) for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 (GPLv2) along with this source code.
> + */
> +
> +#include <linux/device.h>
> +#include <linux/io.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/pci.h>
> +#include <linux/slab.h>
> +
> +#include "ecam.h"
> +
> +/*
> + * On 64 bit systems, we do a single ioremap for the whole config space
> + * since we have enough virtual address range available. On 32 bit, do an
> + * ioremap per bus.
> + */
> +static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
> +
> +/*
> + * Create a PCI config space window
> + *  - reserve mem region
> + *  - alloc struct pci_config_window with space for all mappings
> + *  - ioremap the config space
> + */
> +struct pci_config_window *pci_generic_ecam_create(struct device *dev,
> +				phys_addr_t addr, u8 bus_start, u8 bus_end,

Can you take pointers to struct resources here instead of addr,
bus_start, and bus_end?  The caller probably has them already, and
then you could add a useful printk like:

  dev_info(dev, "ECAM for %pR at %pR\n", busn_res, mmio_res);

Would have to be careful about the struct resource lifetimes though.

If you had the MMIO resource here, you could also do the range
checking you currently have in gen_pci_init() here instead, so all
callers could benefit.

> +				struct pci_generic_ecam_ops *ops)
> +{
> +	struct pci_config_window *cfg;
> +	unsigned int bus_shift, bus_range, bsz, mapsz;
> +	int i, nidx;
> +	int err = -ENOMEM;
> +
> +	if (bus_end < bus_start)
> +		return ERR_PTR(-EINVAL);
> +
> +	bus_shift = ops->bus_shift;
> +	bus_range = bus_end - bus_start + 1;
> +	bsz = 1 << bus_shift;
> +	nidx = per_bus_mapping ? bus_range : 1;
> +	mapsz = per_bus_mapping ? bsz : bus_range * bsz;
> +	cfg = kzalloc(sizeof(*cfg) + nidx * sizeof(cfg->win[0]), GFP_KERNEL);
> +	if (!cfg)
> +		return ERR_PTR(-ENOMEM);
> +
> +	cfg->bus_start = bus_start;
> +	cfg->bus_end = bus_end;
> +	cfg->ops = ops;
> +
> +	if (!request_mem_region(addr, bus_range * bsz, "Configuration Space"))
> +		goto err_exit;
> +
> +	/* cfgaddr has to be set after request_mem_region */
> +	cfg->cfgaddr = addr;
> +
> +	for (i = 0; i < nidx; i++) {
> +		cfg->win[i] = ioremap(addr + i * mapsz, mapsz);
> +		if (!cfg->win[i])
> +			goto err_exit;
> +	}
> +
> +	if (cfg->ops->init) {
> +		err = cfg->ops->init(dev, cfg);
> +		if (err)
> +			goto err_exit;
> +	}
> +	return cfg;
> +
> +err_exit:
> +	pci_generic_ecam_free(cfg);
> +	return ERR_PTR(err);
> +}
> +
> +/*
> + * Free a config space mapping
> + */

Superfluous comment.

> +void pci_generic_ecam_free(struct pci_config_window *cfg)
> +{
> +	unsigned int bus_range;
> +	int i, nidx;
> +
> +	bus_range = cfg->bus_end - cfg->bus_start + 1;
> +	nidx = per_bus_mapping ? bus_range : 1;
> +	for (i = 0; i < nidx; i++)
> +		if (cfg->win[i])
> +			iounmap(cfg->win[i]);
> +	if (cfg->cfgaddr)
> +		release_mem_region(cfg->cfgaddr,
> +				   bus_range << cfg->ops->bus_shift);
> +	kfree(cfg);
> +}
> +
> +/*
> + * Function to implement the pci_ops ->map_bus method
> + */
> +void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
> +				       int where)
> +{
> +	struct pci_config_window *cfg = bus->sysdata;

I don't really like the use of bus->sysdata here, because sysdata is
explicitly arch-specific.

But I guess we're in a bind right now: it'd be nice to save the cfg
pointer in struct pci_host_bridge, but you have to call
pci_generic_ecam_create() before the struct pci_host_bridge has been
allocated, and you have to pass the pointer into pci_scan_root_bus(),
and there's no generic way to do that yet.

So I guess this will have to do for now.

> +	unsigned int devfn_shift = cfg->ops->bus_shift - 8;
> +	unsigned int busn = bus->number;
> +	void __iomem *base;
> +
> +	if (busn < cfg->bus_start || busn > cfg->bus_end)
> +		return NULL;
> +
> +	busn -= cfg->bus_start;
> +	if (per_bus_mapping)
> +		base = cfg->win[busn];
> +	else
> +		base = cfg->win[0] + (busn << cfg->ops->bus_shift);
> +	return base + (devfn << devfn_shift) + where;
> +}
> +
> +/* default ECAM ops */
> +struct pci_generic_ecam_ops pci_generic_ecam_default_ops = {

Using both "generic" and "default" seems overkill.  Maybe just:

  struct pci_ecam_ops pci_generic_ecam_ops = { ... ?

> +	.bus_shift	= 20,
> +	.pci_ops		= {
> +		.map_bus	= pci_generic_ecam_map_bus,
> +		.read		= pci_generic_config_read,
> +		.write		= pci_generic_config_write,
> +	}
> +};
> diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
> new file mode 100644
> index 0000000..34c0aba
> --- /dev/null
> +++ b/drivers/pci/ecam.h
> @@ -0,0 +1,61 @@
> +/*
> + * Copyright 2016 Broadcom
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation (the "GPL").
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 (GPLv2) for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 (GPLv2) along with this source code.
> + */
> +#ifndef DRIVERS_PCI_ECAM_H
> +#define DRIVERS_PCI_ECAM_H
> +
> +#include <linux/kernel.h>
> +#include <linux/platform_device.h>
> +
> +/*
> + * struct to hold pci ops and bus shift of the config window
> + * for a PCI controller.
> + */
> +struct pci_config_window;
> +struct pci_generic_ecam_ops {

"struct pci_ecam_ops"

> +	unsigned int			bus_shift;
> +	struct pci_ops			pci_ops;
> +	int				(*init)(struct device *,
> +						struct pci_config_window *);
> +};
> +
> +/*
> + * struct to hold the mappings of a config space window. This
> + * will be allocated with enough entries in win[] to hold all
> + * the mappings for the bus range.
> + */
> +struct pci_config_window {
> +	phys_addr_t			cfgaddr;
> +	u16				domain;
> +	u8				bus_start;
> +	u8				bus_end;
> +	void				*priv;
> +	struct pci_generic_ecam_ops	*ops;
> +	void __iomem			*win[0];
> +};
> +
> +/* create and free for pci_config_window */

Superfluous comment.

> +struct pci_config_window *pci_generic_ecam_create(struct device *dev,
> +				phys_addr_t addr, u8 bus_start, u8 bus_end,
> +				struct pci_generic_ecam_ops *ops);
> +void pci_generic_ecam_free(struct pci_config_window *cfg);

"pci_ecam_create" and "pci_ecam_free"?  I suspect you're going to call
these for flavors of ECAM that are definitely not "generic".

> +/* map_bus when ->sysdata is an instance of pci_config_window */
> +void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
> +				       int where);
> +/* default ECAM ops, bus shift 20, generic read and write */
> +extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
> +
> +#endif
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Bjorn Helgaas]

Hey, I really kind of like this.  I think this might work out well.

On Fri, Apr 15, 2016 at 07:06:44PM +0200, Tomasz Nowicki wrote:
> This patch is going to implement generic PCI host controller for
> ACPI world, similar to what pci-host-generic.c driver does for DT world.
> 
> All such drivers, which we have seen so far, were implemented within
> arch/ directory since they had some arch assumptions (x86 and ia64).
> However, they all are doing similar thing, so it makes sense to find
> some common code and abstract it into the generic driver.
> 
> In order to handle PCI config space regions properly, we define new
> MCFG interface which parses MCFG table and keep its entries
> in a list. New pci_mcfg_init call is defined so that we do not depend
> on PCI_MMCONFIG. Regions are not mapped until host bridge ask for it.
> 
> The implementation of pci_acpi_scan_root() looks up the saved MCFG entries
> and sets up a new mapping. Generic PCI functions are used for
> accessing config space. Driver selects PCI_GENERIC_ECAM and uses functions
> from drivers/pci/ecam.h to create and access ECAM mappings.
> 
> As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
> should be made on a per-architecture basis.
> 
> This patch is heavily based on the updated version from Jayachandran C:
> https://lkml.org/lkml/2016/4/11/908
> git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> ---
>  drivers/acpi/Kconfig        |   8 ++
>  drivers/acpi/Makefile       |   1 +
>  drivers/acpi/bus.c          |   1 +
>  drivers/acpi/pci_gen_host.c | 231 ++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/pci.h         |   6 ++
>  5 files changed, 247 insertions(+)
>  create mode 100644 drivers/acpi/pci_gen_host.c
> 
> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
> index 183ffa3..70272c5 100644
> --- a/drivers/acpi/Kconfig
> +++ b/drivers/acpi/Kconfig
> @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
>  	  i.e., segment/bus/device/function tuples, with physical slots in
>  	  the system.  If you are unsure, say N.
>  
> +config ACPI_PCI_HOST_GENERIC
> +	bool
> +	select PCI_GENERIC_ECAM
> +	help
> +	  Select this config option from the architecture Kconfig,
> +	  if it is preferred to enable ACPI PCI host controller driver which
> +	  has no arch-specific assumptions.
> +
>  config X86_PM_TIMER
>  	bool "Power Management Timer Support" if EXPERT
>  	depends on X86
> diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
> index 81e5cbc..b12fa64 100644
> --- a/drivers/acpi/Makefile
> +++ b/drivers/acpi/Makefile
> @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
>  acpi-y				+= ec.o
>  acpi-$(CONFIG_ACPI_DOCK)	+= dock.o
>  acpi-y				+= pci_root.o pci_link.o pci_irq.o
> +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC)	+= pci_gen_host.o

Maybe "pci_root_generic.c" so it shows up next to and is obviously
related to pci_root.c?

>  acpi-y				+= acpi_lpss.o acpi_apd.o
>  acpi-y				+= acpi_platform.o
>  acpi-y				+= acpi_pnp.o
> diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
> index c068c82..803a1d7 100644
> --- a/drivers/acpi/bus.c
> +++ b/drivers/acpi/bus.c
> @@ -1107,6 +1107,7 @@ static int __init acpi_init(void)
>  	}
>  
>  	pci_mmcfg_late_init();
> +	pci_mcfg_init();
>  	acpi_scan_init();
>  	acpi_ec_init();
>  	acpi_debugfs_init();
> diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
> new file mode 100644
> index 0000000..fd360b5
> --- /dev/null
> +++ b/drivers/acpi/pci_gen_host.c
> @@ -0,0 +1,231 @@
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation (the "GPL").
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 (GPLv2) for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 (GPLv2) along with this source code.
> + */
> +#include <linux/kernel.h>
> +#include <linux/pci.h>
> +#include <linux/pci-acpi.h>
> +#include <linux/sfi_acpi.h>
> +#include <linux/slab.h>
> +
> +#include "../pci/ecam.h"
> +
> +#define PREFIX	"ACPI: "
> +
> +/* Structure to hold entries from the MCFG table */
> +struct mcfg_entry {
> +	struct list_head	list;
> +	phys_addr_t		addr;
> +	u16			segment;
> +	u8			bus_start;
> +	u8			bus_end;
> +};
> +
> +/* List to save mcfg entries */
> +static LIST_HEAD(pci_mcfg_list);
> +static DEFINE_MUTEX(pci_mcfg_lock);
> +
> +/* ACPI info for generic ACPI PCI controller */
> +struct acpi_pci_generic_root_info {
> +	struct acpi_pci_root_info	common;
> +	struct pci_config_window	*cfg;	/* config space mapping */
> +};
> +
> +/* Find the entry in mcfg list which contains range bus_start */
> +static struct mcfg_entry *pci_mcfg_lookup(u16 seg, u8 bus_start)
> +{
> +	struct mcfg_entry *e;
> +
> +	list_for_each_entry(e, &pci_mcfg_list, list) {
> +		if (e->segment == seg &&
> +		    e->bus_start <= bus_start && bus_start <= e->bus_end)
> +			return e;
> +	}
> +
> +	return NULL;
> +}

Can you put the MCFG parsing, caching, and searching in a different
file, e.g., drivers/acpi/pci_mcfg.c?

> +/*
> + * Lookup the bus range for the domain in MCFG, and set up config space
> + * mapping.
> + */
> +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
> +				       struct acpi_pci_generic_root_info *ri)
> +{
> +	u16 seg = root->segment;
> +	u8 bus_start = root->secondary.start;
> +	u8 bus_end = root->secondary.end;
> +	struct pci_config_window *cfg;
> +	struct mcfg_entry *e;
> +	phys_addr_t addr;
> +	int err = 0;
> +
> +	mutex_lock(&pci_mcfg_lock);

What does this lock protect?  The pci_mcfg_list should already be
initialized by the time we get there, and it should be immutable for
the life of the system.  In fact, I would prefer if we could just
search the static table itself whenever we need it rather than caching
it in our own list.  But I don't think we can easily do that because
acpi_table_parse() is __init.

> +	e = pci_mcfg_lookup(seg, bus_start);

I would argue that we should check for _CBA first, and fall back to
MCFG if _CBA doesn't exist.

> +	if (!e) {
> +		addr = acpi_pci_root_get_mcfg_addr(root->device->handle);

IMO, acpi_pci_root_get_mcfg_addr() is misnamed.  It should be
acpi_pci_config_base_addr() or similar.  It definitely is not related
to MCFG.  Not your fault, obviously.

> +		if (addr == 0) {
> +			pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
> +			       seg, bus_start, bus_end);
> +			err = -ENOENT;
> +			goto err_out;
> +		}
> +	} else {
> +		if (bus_start != e->bus_start) {
> +			pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
> +			       seg, bus_start, bus_end, e->bus_start);
> +			err = -EINVAL;
> +			goto err_out;
> +		} else if (bus_end != e->bus_end) {
> +			pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
> +				seg, bus_start, bus_end, e->bus_end);
> +			bus_end = min(bus_end, e->bus_end);
> +		}
> +		addr = e->addr;
> +	}

I really don't think you need a lock around this, so you can factor
out the address lookup into something like:

  addr = acpi_pci_config_base_addr(...);
  if (addr)
    return addr;

  return acpi_pci_mcfg_lookup(seg, busn_res);

You can check inside acpi_pci_mcfg_lookup() to make sure the entry you
find covers the entire [busn_res.start-busn_res.end] range and return
failure if it doesn't.  At this point, I'm not sure it's worth it to
truncate the host bridge bus range to match something we find in MCFG.

If the MCFG entry covers *more* than the host bridge range from _CRS,
that's fine.  In any case, we have to be careful with the start address,
because the MCFG start address is always based on bus 0, but I think
pci_generic_ecam_create() expects the start address based on the
bus_start you pass to it.

> +
> +	cfg = pci_generic_ecam_create(&root->device->dev, addr, bus_start,
> +				      bus_end, &pci_generic_ecam_default_ops);
> +	if (IS_ERR(cfg)) {
> +		err = PTR_ERR(cfg);
> +		pr_err("%04x:%02x-%02x error %d mapping CAM\n", seg,
> +			bus_start, bus_end, err);
> +		goto err_out;
> +	}
> +
> +	cfg->domain = seg;
> +	ri->cfg = cfg;
> +err_out:
> +	mutex_unlock(&pci_mcfg_lock);
> +	return err;
> +}
> +
> +/* release_info: free resrouces allocated by init_info */
> +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
> +{
> +	struct acpi_pci_generic_root_info *ri;
> +
> +	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
> +	pci_generic_ecam_free(ri->cfg);
> +	kfree(ri);
> +}
> +
> +static struct acpi_pci_root_ops acpi_pci_root_ops = {
> +	.release_info = pci_acpi_generic_release_info,
> +};
> +
> +/* Interface called from ACPI code to setup PCI host controller */
> +struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
> +{
> +	int node = acpi_get_node(root->device->handle);
> +	struct acpi_pci_generic_root_info *ri;
> +	struct pci_bus *bus, *child;
> +	int err;
> +
> +	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
> +	if (!ri)
> +		return NULL;
> +
> +	err = pci_acpi_setup_ecam_mapping(root, ri);
> +	if (err)
> +		return NULL;
> +
> +	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
> +	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
> +				   ri->cfg);
> +	if (!bus)
> +		return NULL;
> +
> +	pci_bus_size_bridges(bus);
> +	pci_bus_assign_resources(bus);
> +
> +	list_for_each_entry(child, &bus->children, node)
> +		pcie_bus_configure_settings(child);
> +
> +	return bus;
> +}
> +
> +/* handle MCFG table entries */
> +static __init int pci_mcfg_parse(struct acpi_table_header *header)
> +{
> +	struct acpi_table_mcfg *mcfg;
> +	struct acpi_mcfg_allocation *mptr;
> +	struct mcfg_entry *e, *arr;
> +	int i, n;
> +
> +	if (!header)
> +		return -EINVAL;
> +
> +	mcfg = (struct acpi_table_mcfg *)header;
> +	mptr = (struct acpi_mcfg_allocation *) &mcfg[1];
> +	n = (header->length - sizeof(*mcfg)) / sizeof(*mptr);
> +	if (n <= 0 || n > 255) {
> +		pr_err(PREFIX " MCFG has incorrect entries (%d).\n", n);
> +		return -EINVAL;
> +	}
> +
> +	arr = kcalloc(n, sizeof(*arr), GFP_KERNEL);
> +	if (!arr)
> +		return -ENOMEM;
> +
> +	for (i = 0, e = arr; i < n; i++, mptr++, e++) {
> +		e->segment = mptr->pci_segment;
> +		e->addr =  mptr->address;
> +		e->bus_start = mptr->start_bus_number;
> +		e->bus_end = mptr->end_bus_number;
> +		list_add(&e->list, &pci_mcfg_list);
> +		pr_info(PREFIX
> +			"MCFG entry for domain %04x [bus %02x-%02x] (base %pa)\n",
> +			e->segment, e->bus_start, e->bus_end, &e->addr);

Ah, this is information similar to what I suggested we might print in
pci_generic_ecam_create().  Could go either way, but personally I
think I'd put it in pci_generic_ecam_create() instead, because then we
only print the info we're actually using.  And I think it'd be nice to
have the actual MMIO resource ("[mem 0x...-0x...]") instead of just
the base.

> +	}
> +
> +	return 0;
> +}
> +
> +/* Interface called by ACPI - parse and save MCFG table */
> +void __init pci_mcfg_init(void)
> +{
> +	int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
> +	if (err)
> +		pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);
> +	else if (list_empty(&pci_mcfg_list))
> +		pr_info(PREFIX "No valid entries in MCFG table.\n");
> +	else {
> +		struct mcfg_entry *e;
> +		int i = 0;
> +		list_for_each_entry(e, &pci_mcfg_list, list)
> +			i++;
> +		pr_info(PREFIX "MCFG table loaded, %d entries\n", i);
> +	}
> +}
> +
> +/* Raw operations, works only for MCFG entries with an associated bus */

Can you elaborate a little on the connection with MCFG?  I don't quite
see how they're related.  Obviously the device has to be on a bus
we've already enumerated and assigned bus->ops for, but it seems like
we don't really know or care what bus->ops actually is.  Given that
these are in this file, acpi_pci_root_ops is the only possibility,
since that's what we pass to acpi_pci_root_create(), but it doesn't
seem worth mentioning specifically in a comment.

> +int raw_pci_read(unsigned int domain, unsigned int busn, unsigned int devfn,
> +		 int reg, int len, u32 *val)
> +{
> +	struct pci_bus *bus = pci_find_bus(domain, busn);
> +
> +	if (!bus)
> +		return PCIBIOS_DEVICE_NOT_FOUND;
> +	return bus->ops->read(bus, devfn, reg, len, val);
> +}
> +
> +int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int devfn,
> +		  int reg, int len, u32 val)
> +{
> +	struct pci_bus *bus = pci_find_bus(domain, busn);
> +
> +	if (!bus)
> +		return PCIBIOS_DEVICE_NOT_FOUND;
> +	return bus->ops->write(bus, devfn, reg, len, val);
> +}
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index df1f33d..c0422ea 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1729,6 +1729,12 @@ static inline void pci_mmcfg_early_init(void) { }
>  static inline void pci_mmcfg_late_init(void) { }
>  #endif
>  
> +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
> +void __init pci_mcfg_init(void);
> +#else
> +static inline void pci_mcfg_init(void) { return; }
> +#endif
> +
>  int pci_ext_cfg_avail(void);
>  
>  void __iomem *pci_ioremap_bar(struct pci_dev *pdev, int bar);
> -- 
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V6 05/13] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Jon Masters]

On 04/27/2016 01:36 AM, Jon Masters wrote:
> On 04/26/2016 10:39 PM, Bjorn Helgaas wrote:

>> It would be ideal to find
>> some way to handle ia64 and ARM64 similarly.  At the very least, we
>> have to make sure that this doesn't break ia64.  The ia64 dense/sparse
>> I/O spaces complicate things; I don't know if ARM64 has something
>> similar or not.
> 
> There's nothing directly similar - it's just regular MMIO.

Just a footnote on the IA64 thing. I'm working on getting access to a
few Itanium systems and running V6 on these (even bought my first
Itanium system today so I can run this at home also). I'm also chatting
with the internal RH QE folks about testing on a few dozen x86 systems.
Will followup when we've any results on that testing.

Jon.

-- 
Computer Architect | Sent from my Fedora powered laptop

Re: [PATCH V6 07/13] PCI: Provide common functions for ECAM mapping

[Jayachandran C]

On Fri, Apr 29, 2016 at 3:17 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> On Fri, Apr 15, 2016 at 07:06:42PM +0200, Tomasz Nowicki wrote:
>> From: Jayachandran C <jchandra@broadcom.com>
>>
>> Add config option PCI_GENERIC_ECAM and file drivers/pci/ecam.c to
>> provide generic functions for accessing memory mapped PCI config space.
>>
>> The API is defined in drivers/pci/ecam.h and is written to replace the
>> API in drivers/pci/host/pci-host-common.h. The file defines a new
>> 'struct pci_config_window' to hold the information related to a PCI
>> config area and its mapping. This structure is expected to be used as
>> sysdata for controllers that have ECAM based mapping.
>>
>> Helper functions are provided to setup the mapping, free the mapping
>> and to implement the map_bus method in 'struct pci_ops'
>
> Spec reference: PCI Express Base Specification, rev 3.0, sec 7.2.2.
>
>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>> ---
>>  drivers/pci/Kconfig  |   3 ++
>>  drivers/pci/Makefile |   2 +
>>  drivers/pci/ecam.c   | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>  drivers/pci/ecam.h   |  61 +++++++++++++++++++++++
>>  4 files changed, 203 insertions(+)
>>  create mode 100644 drivers/pci/ecam.c
>>  create mode 100644 drivers/pci/ecam.h
>>
>> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
>> index 209292e..e930d62 100644
>> --- a/drivers/pci/Kconfig
>> +++ b/drivers/pci/Kconfig
>> @@ -83,6 +83,9 @@ config HT_IRQ
>>  config PCI_ATS
>>       bool
>>
>> +config PCI_GENERIC_ECAM
>> +     bool
>
> "PCI_ECAM" is enough, I think.  It's defined by and required by the
> spec unless there's some arch-specific interface.  Plus, if I

Ok. Looking at the comments I think I have to take out generic from
all the names - will do this in next version.

> understand correctly, this infrastructure supports non-generic ECAM
> implementations as well, since the caller supplies "struct
> pci_generic_ecam_ops *ops".

Yes, the idea was to support ECAM with quirks (and CAM) on both
32 and 64 bit, otherwise it would be too trivial to have a separate
implementation.

>>  config PCI_IOV
>>       bool "PCI IOV support"
>>       depends on PCI
>> diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
>> index 2154092..810aec8 100644
>> --- a/drivers/pci/Makefile
>> +++ b/drivers/pci/Makefile
>> @@ -55,6 +55,8 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o
>>
>>  obj-$(CONFIG_PCI_STUB) += pci-stub.o
>>
>> +obj-$(CONFIG_PCI_GENERIC_ECAM) += ecam.o
>> +
>>  obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
>>
>>  obj-$(CONFIG_OF) += of.o
>> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
>> new file mode 100644
>> index 0000000..ff04c01
>> --- /dev/null
>> +++ b/drivers/pci/ecam.c
>> @@ -0,0 +1,137 @@
>> +/*
>> + * Copyright 2016 Broadcom
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License, version 2, as
>> + * published by the Free Software Foundation (the "GPL").
>> + *
>> + * This program is distributed in the hope that it will be useful, but
>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License version 2 (GPLv2) for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * version 2 (GPLv2) along with this source code.
>> + */
>> +
>> +#include <linux/device.h>
>> +#include <linux/io.h>
>> +#include <linux/kernel.h>
>> +#include <linux/module.h>
>> +#include <linux/pci.h>
>> +#include <linux/slab.h>
>> +
>> +#include "ecam.h"
>> +
>> +/*
>> + * On 64 bit systems, we do a single ioremap for the whole config space
>> + * since we have enough virtual address range available. On 32 bit, do an
>> + * ioremap per bus.
>> + */
>> +static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
>> +
>> +/*
>> + * Create a PCI config space window
>> + *  - reserve mem region
>> + *  - alloc struct pci_config_window with space for all mappings
>> + *  - ioremap the config space
>> + */
>> +struct pci_config_window *pci_generic_ecam_create(struct device *dev,
>> +                             phys_addr_t addr, u8 bus_start, u8 bus_end,
>
> Can you take pointers to struct resources here instead of addr,
> bus_start, and bus_end?  The caller probably has them already, and
> then you could add a useful printk like:
>
>   dev_info(dev, "ECAM for %pR at %pR\n", busn_res, mmio_res);
>
> Would have to be careful about the struct resource lifetimes though.

Yes, I had thought of this. The reason I did not go down that path was
that we are using request_mem_region() which takes the address
and creates a resource .internally.

Beyond that, as you noted, the ownership and lifetime is slightly
more complex. Either the calling code has to allocate the
resource and handoff, or the ecam code has to make a copy of
the resource. I would go with copy since it is much more simple
to use.

Since resource based API seems to be preferred, I will switch
to passing bus and mmio resource and will use
request_resource_conflict() to allocate the ECAM mem region,

> If you had the MMIO resource here, you could also do the range
> checking you currently have in gen_pci_init() here instead, so all
> callers could benefit.

Ok.

>> +                             struct pci_generic_ecam_ops *ops)
>> +{
>> +     struct pci_config_window *cfg;
>> +     unsigned int bus_shift, bus_range, bsz, mapsz;
>> +     int i, nidx;
>> +     int err = -ENOMEM;
>> +
>> +     if (bus_end < bus_start)
>> +             return ERR_PTR(-EINVAL);
>> +
>> +     bus_shift = ops->bus_shift;
>> +     bus_range = bus_end - bus_start + 1;
>> +     bsz = 1 << bus_shift;
>> +     nidx = per_bus_mapping ? bus_range : 1;
>> +     mapsz = per_bus_mapping ? bsz : bus_range * bsz;
>> +     cfg = kzalloc(sizeof(*cfg) + nidx * sizeof(cfg->win[0]), GFP_KERNEL);
>> +     if (!cfg)
>> +             return ERR_PTR(-ENOMEM);
>> +
>> +     cfg->bus_start = bus_start;
>> +     cfg->bus_end = bus_end;
>> +     cfg->ops = ops;
>> +
>> +     if (!request_mem_region(addr, bus_range * bsz, "Configuration Space"))
>> +             goto err_exit;
>> +
>> +     /* cfgaddr has to be set after request_mem_region */
>> +     cfg->cfgaddr = addr;
>> +
>> +     for (i = 0; i < nidx; i++) {
>> +             cfg->win[i] = ioremap(addr + i * mapsz, mapsz);
>> +             if (!cfg->win[i])
>> +                     goto err_exit;
>> +     }
>> +
>> +     if (cfg->ops->init) {
>> +             err = cfg->ops->init(dev, cfg);
>> +             if (err)
>> +                     goto err_exit;
>> +     }
>> +     return cfg;
>> +
>> +err_exit:
>> +     pci_generic_ecam_free(cfg);
>> +     return ERR_PTR(err);
>> +}
>> +
>> +/*
>> + * Free a config space mapping
>> + */
>
> Superfluous comment.

Ok, will trim comments to keep the ones that are useful.

>> +void pci_generic_ecam_free(struct pci_config_window *cfg)
>> +{
>> +     unsigned int bus_range;
>> +     int i, nidx;
>> +
>> +     bus_range = cfg->bus_end - cfg->bus_start + 1;
>> +     nidx = per_bus_mapping ? bus_range : 1;
>> +     for (i = 0; i < nidx; i++)
>> +             if (cfg->win[i])
>> +                     iounmap(cfg->win[i]);
>> +     if (cfg->cfgaddr)
>> +             release_mem_region(cfg->cfgaddr,
>> +                                bus_range << cfg->ops->bus_shift);
>> +     kfree(cfg);
>> +}
>> +
>> +/*
>> + * Function to implement the pci_ops ->map_bus method
>> + */
>> +void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>> +                                    int where)
>> +{
>> +     struct pci_config_window *cfg = bus->sysdata;
>
> I don't really like the use of bus->sysdata here, because sysdata is
> explicitly arch-specific.
>
> But I guess we're in a bind right now: it'd be nice to save the cfg
> pointer in struct pci_host_bridge, but you have to call
> pci_generic_ecam_create() before the struct pci_host_bridge has been
> allocated, and you have to pass the pointer into pci_scan_root_bus(),
> and there's no generic way to do that yet.
>
> So I guess this will have to do for now.

I could call the structure pci_ecam_sysdata instead of pci_config_window
to make it clear that it is to be used as sysdata for ECAM based PCI host
controllers.

The current push (at least on ARM/ARM64) seems to be to make
bus->sysdata controller specific and avoid arch specific sysdata.

>> +     unsigned int devfn_shift = cfg->ops->bus_shift - 8;
>> +     unsigned int busn = bus->number;
>> +     void __iomem *base;
>> +
>> +     if (busn < cfg->bus_start || busn > cfg->bus_end)
>> +             return NULL;
>> +
>> +     busn -= cfg->bus_start;
>> +     if (per_bus_mapping)
>> +             base = cfg->win[busn];
>> +     else
>> +             base = cfg->win[0] + (busn << cfg->ops->bus_shift);
>> +     return base + (devfn << devfn_shift) + where;
>> +}
>> +
>> +/* default ECAM ops */
>> +struct pci_generic_ecam_ops pci_generic_ecam_default_ops = {
>
> Using both "generic" and "default" seems overkill.  Maybe just:
>
>   struct pci_ecam_ops pci_generic_ecam_ops = { ... ?

Ok.

>> +     .bus_shift      = 20,
>> +     .pci_ops                = {
>> +             .map_bus        = pci_generic_ecam_map_bus,
>> +             .read           = pci_generic_config_read,
>> +             .write          = pci_generic_config_write,
>> +     }
>> +};
>> diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
>> new file mode 100644
>> index 0000000..34c0aba
>> --- /dev/null
>> +++ b/drivers/pci/ecam.h
>> @@ -0,0 +1,61 @@
>> +/*
>> + * Copyright 2016 Broadcom
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License, version 2, as
>> + * published by the Free Software Foundation (the "GPL").
>> + *
>> + * This program is distributed in the hope that it will be useful, but
>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License version 2 (GPLv2) for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * version 2 (GPLv2) along with this source code.
>> + */
>> +#ifndef DRIVERS_PCI_ECAM_H
>> +#define DRIVERS_PCI_ECAM_H
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/platform_device.h>
>> +
>> +/*
>> + * struct to hold pci ops and bus shift of the config window
>> + * for a PCI controller.
>> + */
>> +struct pci_config_window;
>> +struct pci_generic_ecam_ops {
>
> "struct pci_ecam_ops"

Ok.

>> +     unsigned int                    bus_shift;
>> +     struct pci_ops                  pci_ops;
>> +     int                             (*init)(struct device *,
>> +                                             struct pci_config_window *);
>> +};
>> +
>> +/*
>> + * struct to hold the mappings of a config space window. This
>> + * will be allocated with enough entries in win[] to hold all
>> + * the mappings for the bus range.
>> + */
>> +struct pci_config_window {
>> +     phys_addr_t                     cfgaddr;
>> +     u16                             domain;
>> +     u8                              bus_start;
>> +     u8                              bus_end;
>> +     void                            *priv;
>> +     struct pci_generic_ecam_ops     *ops;
>> +     void __iomem                    *win[0];
>> +};
>> +
>> +/* create and free for pci_config_window */
>
> Superfluous comment.
>
>> +struct pci_config_window *pci_generic_ecam_create(struct device *dev,
>> +                             phys_addr_t addr, u8 bus_start, u8 bus_end,
>> +                             struct pci_generic_ecam_ops *ops);
>> +void pci_generic_ecam_free(struct pci_config_window *cfg);
>
> "pci_ecam_create" and "pci_ecam_free"?  I suspect you're going to call
> these for flavors of ECAM that are definitely not "generic".
>
>> +/* map_bus when ->sysdata is an instance of pci_config_window */
>> +void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>> +                                    int where);
>> +/* default ECAM ops, bus shift 20, generic read and write */
>> +extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
>> +
>> +#endif

Thanks for the review. The ECAM patchset can be merged separately
if you do not want to tie it the ACPI changes.

Please let me know how you want to proceed. Depending on that, I will
either post it as a separate patchset or ask Tomasz to pick up my
changes and post the ACPI patchset. again.

JC.

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Lorenzo Pieralisi]

On Thu, Apr 28, 2016 at 04:48:00PM -0500, Bjorn Helgaas wrote:

[...]

> > +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
> > +				       struct acpi_pci_generic_root_info *ri)
> > +{
> > +	u16 seg = root->segment;
> > +	u8 bus_start = root->secondary.start;
> > +	u8 bus_end = root->secondary.end;
> > +	struct pci_config_window *cfg;
> > +	struct mcfg_entry *e;
> > +	phys_addr_t addr;
> > +	int err = 0;
> > +
> > +	mutex_lock(&pci_mcfg_lock);
> 
> What does this lock protect?  The pci_mcfg_list should already be
> initialized by the time we get there, and it should be immutable for
> the life of the system.  In fact, I would prefer if we could just
> search the static table itself whenever we need it rather than caching
> it in our own list.  But I don't think we can easily do that because
> acpi_table_parse() is __init.
> 
> > +	e = pci_mcfg_lookup(seg, bus_start);
> 
> I would argue that we should check for _CBA first, and fall back to
> MCFG if _CBA doesn't exist.
> 
> > +	if (!e) {
> > +		addr = acpi_pci_root_get_mcfg_addr(root->device->handle);
> 
> IMO, acpi_pci_root_get_mcfg_addr() is misnamed.  It should be
> acpi_pci_config_base_addr() or similar.  It definitely is not related
> to MCFG.  Not your fault, obviously.
> 
> > +		if (addr == 0) {
> > +			pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
> > +			       seg, bus_start, bus_end);
> > +			err = -ENOENT;
> > +			goto err_out;
> > +		}
> > +	} else {
> > +		if (bus_start != e->bus_start) {
> > +			pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
> > +			       seg, bus_start, bus_end, e->bus_start);
> > +			err = -EINVAL;
> > +			goto err_out;
> > +		} else if (bus_end != e->bus_end) {
> > +			pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
> > +				seg, bus_start, bus_end, e->bus_end);
> > +			bus_end = min(bus_end, e->bus_end);
> > +		}
> > +		addr = e->addr;
> > +	}
> 
> I really don't think you need a lock around this, so you can factor
> out the address lookup into something like:
> 
>   addr = acpi_pci_config_base_addr(...);
>   if (addr)
>     return addr;
> 
>   return acpi_pci_mcfg_lookup(seg, busn_res);
> 
> You can check inside acpi_pci_mcfg_lookup() to make sure the entry you
> find covers the entire [busn_res.start-busn_res.end] range and return
> failure if it doesn't.  At this point, I'm not sure it's worth it to
> truncate the host bridge bus range to match something we find in MCFG.
> 
> If the MCFG entry covers *more* than the host bridge range from _CRS,
> that's fine.  In any case, we have to be careful with the start address,
> because the MCFG start address is always based on bus 0, but I think
> pci_generic_ecam_create() expects the start address based on the
> bus_start you pass to it.

Yes, I spotted this too, it is unfortunate but DT and MCFG handle
the ECAM regions differently. In DT the reg property is relative
to bus_start - ie reg MMIO region maps config space starting at
the first bus in bus-range:

Documentation/devicetree/bindings/pci/host-generic-pci.txt

in ACPI(MCFG) as you said it is always relative to bus 0, it is
unfortunate but the address to be mapped should be computed
differently in the ECAM layer.

Lorenzo

Re: [PATCH V6 08/13] PCI: generic, thunder: update to use generic ECAM API

[Lorenzo Pieralisi]

On Thu, Apr 28, 2016 at 05:47:15PM -0400, Jon Masters wrote:

[...]

> >>> In general, there's no reason we can't reassign BARs, whether we're
> >>> using DT, ACPI, or whatever.  In many cases, systems with ACPI also
> >>> assign all the BARs in firmware, and Linux doesn't reassign them
> >>> unless it needs to.  But that's just a coincidence.  There's no
> >>> requirement that Linux leave BARs as firmware programmed them.
> 
> There's no requirement, generally, that PCI compliant devices with ECAM
> can't be programmed with different base addresses. There's this PCI
> change called EA that is disjoint and some vendors have chosen to use
> it. We didn't catch that early in the definition of the SBSA for ARM,
> but just as an aside, I have already suggested we require future
> generations of chips to not use EA and only support writeable BARs (even
> for the decoders in the on-SoC platformish devices doing "PCI"). This
> isn't Cavium's fault - they did the right thing with the data at hand
> and nobody really considered the impact of PCI getting EA added. Again,
> that's something that will likely happen on x86 at some point (maybe it
> already is, I don't get any data about future Intel stuff).

PCI EA support in the kernel was implemented by Intel, for the records.

And I do not think anyone is questioning EA here (I mean implemented
through a real PCI capability, not config space quirks).

> On the rest of the quirks and hacks. Without going into too much detail,
> some "concerned citizens" are chatting with various folks to ensure that
> many of these common quirks aren't needed in future parts.
> 
> >> I'm thought I've seen systems in which the ACPI BIOS assumes that
> >> certain PCI devices never move around, because it pokes the registers
> >> from AML, and changing them would require never using the same device
> >> through ACPI. It's likely that this is against some standard, but that
> >> won't help you if you have to deal with the system anyway.
> 
> Right. This has happened, I think, and there you're no worse off on ARM
> than you would be on x86 if you had AML poking at something underneath.

Except that (if I read their code correctly - arch/x86/pci/i386.c,
see pcibios_resource_survey()) X86 claims the resources as
set-up by FW and thus does not reassign them, whereas on ARM we reassign
the whole PCI address space and we totally ignore the FW set-up (in DT
and ACPI alike), whether that's a problem or not time will tell,
as Bjorn mentioned I do not think that by the time FW hands over to
the OS there is any requirement whatsoever that prevents the OS
from reprogramming the PCI BARs set-up.

> > Yes, I'm pretty sure there are systems like that, e.g., I think SMM
> > code on some HP servers assumes the iLO address never changes.  I
> > think that is a firmware defect because I don't think there's any spec
> > that says firmware retains control over PCI BARs after handoff.  And
> > this particular case isn't really ACPI-specific.
> 
> If you substitute SMM for EL3 on ARM we're bound to eventually have the
> same kinds of things happening on some systems. It's just life.
> 
> > But as you say, we have to deal with these systems anyway, even if we
> > consider that behavior broken.  My proposal has been to add quirks to
> > mark those devices as IORESOURCE_PCI_FIXED, but I don't think anybody
> > has gotten around to doing that.

Well, that's going to be interesting. To me it is more something FW
should be able to communicate to the OS rather than a device specific
quirk, it is not that the device has fixed BARs, it is that the FW
expects them to be immutable (not saying that's the correct FW
behaviour - but it looks like a FW specific issue, not device specific).

I wonder whether this can be solved (at least in ACPI) through
a PCI BAR Target Operation Region (ACPI 6.0, 5.5.2.4.2), I will have
a look into that.

Lorenzo

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Jayachandran C]

On Fri, Apr 29, 2016 at 2:07 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
> On Thu, Apr 28, 2016 at 04:48:00PM -0500, Bjorn Helgaas wrote:
>
> [...]
>
>> > +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
>> > +                                  struct acpi_pci_generic_root_info *ri)
>> > +{
>> > +   u16 seg = root->segment;
>> > +   u8 bus_start = root->secondary.start;
>> > +   u8 bus_end = root->secondary.end;
>> > +   struct pci_config_window *cfg;
>> > +   struct mcfg_entry *e;
>> > +   phys_addr_t addr;
>> > +   int err = 0;
>> > +
>> > +   mutex_lock(&pci_mcfg_lock);
>>
>> What does this lock protect?  The pci_mcfg_list should already be
>> initialized by the time we get there, and it should be immutable for
>> the life of the system.  In fact, I would prefer if we could just
>> search the static table itself whenever we need it rather than caching
>> it in our own list.  But I don't think we can easily do that because
>> acpi_table_parse() is __init.
>>
>> > +   e = pci_mcfg_lookup(seg, bus_start);
>>
>> I would argue that we should check for _CBA first, and fall back to
>> MCFG if _CBA doesn't exist.
>>
>> > +   if (!e) {
>> > +           addr = acpi_pci_root_get_mcfg_addr(root->device->handle);
>>
>> IMO, acpi_pci_root_get_mcfg_addr() is misnamed.  It should be
>> acpi_pci_config_base_addr() or similar.  It definitely is not related
>> to MCFG.  Not your fault, obviously.
>>
>> > +           if (addr == 0) {
>> > +                   pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
>> > +                          seg, bus_start, bus_end);
>> > +                   err = -ENOENT;
>> > +                   goto err_out;
>> > +           }
>> > +   } else {
>> > +           if (bus_start != e->bus_start) {
>> > +                   pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
>> > +                          seg, bus_start, bus_end, e->bus_start);
>> > +                   err = -EINVAL;
>> > +                   goto err_out;
>> > +           } else if (bus_end != e->bus_end) {
>> > +                   pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
>> > +                           seg, bus_start, bus_end, e->bus_end);
>> > +                   bus_end = min(bus_end, e->bus_end);
>> > +           }
>> > +           addr = e->addr;
>> > +   }
>>
>> I really don't think you need a lock around this, so you can factor
>> out the address lookup into something like:
>>
>>   addr = acpi_pci_config_base_addr(...);
>>   if (addr)
>>     return addr;
>>
>>   return acpi_pci_mcfg_lookup(seg, busn_res);
>>
>> You can check inside acpi_pci_mcfg_lookup() to make sure the entry you
>> find covers the entire [busn_res.start-busn_res.end] range and return
>> failure if it doesn't.  At this point, I'm not sure it's worth it to
>> truncate the host bridge bus range to match something we find in MCFG.
>>
>> If the MCFG entry covers *more* than the host bridge range from _CRS,
>> that's fine.  In any case, we have to be careful with the start address,
>> because the MCFG start address is always based on bus 0, but I think
>> pci_generic_ecam_create() expects the start address based on the
>> bus_start you pass to it.
>
> Yes, I spotted this too, it is unfortunate but DT and MCFG handle
> the ECAM regions differently. In DT the reg property is relative
> to bus_start - ie reg MMIO region maps config space starting at
> the first bus in bus-range:
>
> Documentation/devicetree/bindings/pci/host-generic-pci.txt
>
> in ACPI(MCFG) as you said it is always relative to bus 0, it is
> unfortunate but the address to be mapped should be computed
> differently in the ECAM layer.

Can't this be handled by fixing up the address before passing to
pci_generic_ecam_create?

JC.

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Arnd Bergmann]

On Thursday 28 April 2016 17:34:10 Arnd Bergmann wrote:
> On Thursday 28 April 2016 10:12:12 Bjorn Helgaas wrote:
> > Right, we don't have a good mechanism for passing more info into
> > pci_create_root_bus().  Maybe the caller could fill in a struct so we
> > have a chance to extend it without having to change all the existing
> > callers.
> > 
> > I wonder if there's a design pattern we can copy, e.g., would
> > something like the scsi_host_alloc(), scsi_add_host(),
> > scsi_scan_host() model work here?
> 
> Yes, I think that is a good idea in general. Especially
> now that we have separate the ARM code from pci_common_init_dev
> and pci_sys_data, that can help with cleanups in the other drivers
> as well.
> 
> I see two common variations in other subsystems: some use a
> special alloc() function that you pass the size of the private
> data into, while others just expect you to embed a structure
> inside of the driver specific one allocate that separately to
> have the generic registration function fill out the common fields.
> 
> I have a slight preference for the second, but they are really
> the same thing basically.

I've tried this out now, and will follow up with a separate patch
series. Overall, I think it works out well, though I haven't gotten
to the point of actually saving code yet. I've converted two
drivers for demonstration.

	Arnd

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Tomasz Nowicki]

On 04/28/2016 11:48 PM, Bjorn Helgaas wrote:
> Hey, I really kind of like this.  I think this might work out well.
>
> On Fri, Apr 15, 2016 at 07:06:44PM +0200, Tomasz Nowicki wrote:
>> This patch is going to implement generic PCI host controller for
>> ACPI world, similar to what pci-host-generic.c driver does for DT world.
>>
>> All such drivers, which we have seen so far, were implemented within
>> arch/ directory since they had some arch assumptions (x86 and ia64).
>> However, they all are doing similar thing, so it makes sense to find
>> some common code and abstract it into the generic driver.
>>
>> In order to handle PCI config space regions properly, we define new
>> MCFG interface which parses MCFG table and keep its entries
>> in a list. New pci_mcfg_init call is defined so that we do not depend
>> on PCI_MMCONFIG. Regions are not mapped until host bridge ask for it.
>>
>> The implementation of pci_acpi_scan_root() looks up the saved MCFG entries
>> and sets up a new mapping. Generic PCI functions are used for
>> accessing config space. Driver selects PCI_GENERIC_ECAM and uses functions
>> from drivers/pci/ecam.h to create and access ECAM mappings.
>>
>> As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
>> should be made on a per-architecture basis.
>>
>> This patch is heavily based on the updated version from Jayachandran C:
>> https://lkml.org/lkml/2016/4/11/908
>> git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
>>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>> ---
>>   drivers/acpi/Kconfig        |   8 ++
>>   drivers/acpi/Makefile       |   1 +
>>   drivers/acpi/bus.c          |   1 +
>>   drivers/acpi/pci_gen_host.c | 231 ++++++++++++++++++++++++++++++++++++++++++++
>>   include/linux/pci.h         |   6 ++
>>   5 files changed, 247 insertions(+)
>>   create mode 100644 drivers/acpi/pci_gen_host.c
>>
>> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
>> index 183ffa3..70272c5 100644
>> --- a/drivers/acpi/Kconfig
>> +++ b/drivers/acpi/Kconfig
>> @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
>>   	  i.e., segment/bus/device/function tuples, with physical slots in
>>   	  the system.  If you are unsure, say N.
>>   
>> +config ACPI_PCI_HOST_GENERIC
>> +	bool
>> +	select PCI_GENERIC_ECAM
>> +	help
>> +	  Select this config option from the architecture Kconfig,
>> +	  if it is preferred to enable ACPI PCI host controller driver which
>> +	  has no arch-specific assumptions.
>> +
>>   config X86_PM_TIMER
>>   	bool "Power Management Timer Support" if EXPERT
>>   	depends on X86
>> diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
>> index 81e5cbc..b12fa64 100644
>> --- a/drivers/acpi/Makefile
>> +++ b/drivers/acpi/Makefile
>> @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
>>   acpi-y				+= ec.o
>>   acpi-$(CONFIG_ACPI_DOCK)	+= dock.o
>>   acpi-y				+= pci_root.o pci_link.o pci_irq.o
>> +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC)	+= pci_gen_host.o
> Maybe "pci_root_generic.c" so it shows up next to and is obviously
> related to pci_root.c?

Yes, this is good idea.

>
>>   acpi-y				+= acpi_lpss.o acpi_apd.o
>>   acpi-y				+= acpi_platform.o
>>   acpi-y				+= acpi_pnp.o
>> diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c
>> index c068c82..803a1d7 100644
>> --- a/drivers/acpi/bus.c
>> +++ b/drivers/acpi/bus.c
>> @@ -1107,6 +1107,7 @@ static int __init acpi_init(void)
>>   	}
>>   
>>   	pci_mmcfg_late_init();
>> +	pci_mcfg_init();
>>   	acpi_scan_init();
>>   	acpi_ec_init();
>>   	acpi_debugfs_init();
>> diff --git a/drivers/acpi/pci_gen_host.c b/drivers/acpi/pci_gen_host.c
>> new file mode 100644
>> index 0000000..fd360b5
>> --- /dev/null
>> +++ b/drivers/acpi/pci_gen_host.c
>> @@ -0,0 +1,231 @@
>> +/*
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License, version 2, as
>> + * published by the Free Software Foundation (the "GPL").
>> + *
>> + * This program is distributed in the hope that it will be useful, but
>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License version 2 (GPLv2) for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * version 2 (GPLv2) along with this source code.
>> + */
>> +#include <linux/kernel.h>
>> +#include <linux/pci.h>
>> +#include <linux/pci-acpi.h>
>> +#include <linux/sfi_acpi.h>
>> +#include <linux/slab.h>
>> +
>> +#include "../pci/ecam.h"
>> +
>> +#define PREFIX	"ACPI: "
>> +
>> +/* Structure to hold entries from the MCFG table */
>> +struct mcfg_entry {
>> +	struct list_head	list;
>> +	phys_addr_t		addr;
>> +	u16			segment;
>> +	u8			bus_start;
>> +	u8			bus_end;
>> +};
>> +
>> +/* List to save mcfg entries */
>> +static LIST_HEAD(pci_mcfg_list);
>> +static DEFINE_MUTEX(pci_mcfg_lock);
>> +
>> +/* ACPI info for generic ACPI PCI controller */
>> +struct acpi_pci_generic_root_info {
>> +	struct acpi_pci_root_info	common;
>> +	struct pci_config_window	*cfg;	/* config space mapping */
>> +};
>> +
>> +/* Find the entry in mcfg list which contains range bus_start */
>> +static struct mcfg_entry *pci_mcfg_lookup(u16 seg, u8 bus_start)
>> +{
>> +	struct mcfg_entry *e;
>> +
>> +	list_for_each_entry(e, &pci_mcfg_list, list) {
>> +		if (e->segment == seg &&
>> +		    e->bus_start <= bus_start && bus_start <= e->bus_end)
>> +			return e;
>> +	}
>> +
>> +	return NULL;
>> +}
> Can you put the MCFG parsing, caching, and searching in a different
> file, e.g., drivers/acpi/pci_mcfg.c?
I will.
>
>> +/*
>> + * Lookup the bus range for the domain in MCFG, and set up config space
>> + * mapping.
>> + */
>> +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
>> +				       struct acpi_pci_generic_root_info *ri)
>> +{
>> +	u16 seg = root->segment;
>> +	u8 bus_start = root->secondary.start;
>> +	u8 bus_end = root->secondary.end;
>> +	struct pci_config_window *cfg;
>> +	struct mcfg_entry *e;
>> +	phys_addr_t addr;
>> +	int err = 0;
>> +
>> +	mutex_lock(&pci_mcfg_lock);
> What does this lock protect?  The pci_mcfg_list should already be
> initialized by the time we get there, and it should be immutable for
> the life of the system.
Right, lock is useless.
>    In fact, I would prefer if we could just
> search the static table itself whenever we need it rather than caching
> it in our own list.  But I don't think we can easily do that because
> acpi_table_parse() is __init.

It is doable. We can implement our MCFG __init parsing handle to do 
necessary sanity checks and then store MCFG table root pointer in 
pci_mcfg.c. Then lookup call would use that pointer to traverse 
available entries on demand.

>
>> +	e = pci_mcfg_lookup(seg, bus_start);
> I would argue that we should check for _CBA first, and fall back to
> MCFG if _CBA doesn't exist.
Agree.
>
>> +	if (!e) {
>> +		addr = acpi_pci_root_get_mcfg_addr(root->device->handle);
> IMO, acpi_pci_root_get_mcfg_addr() is misnamed.  It should be
> acpi_pci_config_base_addr() or similar.  It definitely is not related
> to MCFG.  Not your fault, obviously.
>
>> +		if (addr == 0) {
>> +			pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
>> +			       seg, bus_start, bus_end);
>> +			err = -ENOENT;
>> +			goto err_out;
>> +		}
>> +	} else {
>> +		if (bus_start != e->bus_start) {
>> +			pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
>> +			       seg, bus_start, bus_end, e->bus_start);
>> +			err = -EINVAL;
>> +			goto err_out;
>> +		} else if (bus_end != e->bus_end) {
>> +			pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
>> +				seg, bus_start, bus_end, e->bus_end);
>> +			bus_end = min(bus_end, e->bus_end);
>> +		}
>> +		addr = e->addr;
>> +	}
> I really don't think you need a lock around this, so you can factor
> out the address lookup into something like:
>
>    addr = acpi_pci_config_base_addr(...);
>    if (addr)
>      return addr;
>
>    return acpi_pci_mcfg_lookup(seg, busn_res);
>
> You can check inside acpi_pci_mcfg_lookup() to make sure the entry you
> find covers the entire [busn_res.start-busn_res.end] range and return
> failure if it doesn't.  At this point, I'm not sure it's worth it to
> truncate the host bridge bus range to match something we find in MCFG.
>
> If the MCFG entry covers *more* than the host bridge range from _CRS,
> that's fine.
Makes sense for me.

>    In any case, we have to be careful with the start address,
> because the MCFG start address is always based on bus 0, but I think
> pci_generic_ecam_create() expects the start address based on the
> bus_start you pass to it.

We will have a look how to fix this.
>
>> +
>> +	cfg = pci_generic_ecam_create(&root->device->dev, addr, bus_start,
>> +				      bus_end, &pci_generic_ecam_default_ops);
>> +	if (IS_ERR(cfg)) {
>> +		err = PTR_ERR(cfg);
>> +		pr_err("%04x:%02x-%02x error %d mapping CAM\n", seg,
>> +			bus_start, bus_end, err);
>> +		goto err_out;
>> +	}
>> +
>> +	cfg->domain = seg;
>> +	ri->cfg = cfg;
>> +err_out:
>> +	mutex_unlock(&pci_mcfg_lock);
>> +	return err;
>> +}
>> +
>> +/* release_info: free resrouces allocated by init_info */
>> +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
>> +{
>> +	struct acpi_pci_generic_root_info *ri;
>> +
>> +	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
>> +	pci_generic_ecam_free(ri->cfg);
>> +	kfree(ri);
>> +}
>> +
>> +static struct acpi_pci_root_ops acpi_pci_root_ops = {
>> +	.release_info = pci_acpi_generic_release_info,
>> +};
>> +
>> +/* Interface called from ACPI code to setup PCI host controller */
>> +struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>> +{
>> +	int node = acpi_get_node(root->device->handle);
>> +	struct acpi_pci_generic_root_info *ri;
>> +	struct pci_bus *bus, *child;
>> +	int err;
>> +
>> +	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
>> +	if (!ri)
>> +		return NULL;
>> +
>> +	err = pci_acpi_setup_ecam_mapping(root, ri);
>> +	if (err)
>> +		return NULL;
>> +
>> +	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
>> +	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
>> +				   ri->cfg);
>> +	if (!bus)
>> +		return NULL;
>> +
>> +	pci_bus_size_bridges(bus);
>> +	pci_bus_assign_resources(bus);
>> +
>> +	list_for_each_entry(child, &bus->children, node)
>> +		pcie_bus_configure_settings(child);
>> +
>> +	return bus;
>> +}
>> +
>> +/* handle MCFG table entries */
>> +static __init int pci_mcfg_parse(struct acpi_table_header *header)
>> +{
>> +	struct acpi_table_mcfg *mcfg;
>> +	struct acpi_mcfg_allocation *mptr;
>> +	struct mcfg_entry *e, *arr;
>> +	int i, n;
>> +
>> +	if (!header)
>> +		return -EINVAL;
>> +
>> +	mcfg = (struct acpi_table_mcfg *)header;
>> +	mptr = (struct acpi_mcfg_allocation *) &mcfg[1];
>> +	n = (header->length - sizeof(*mcfg)) / sizeof(*mptr);
>> +	if (n <= 0 || n > 255) {
>> +		pr_err(PREFIX " MCFG has incorrect entries (%d).\n", n);
>> +		return -EINVAL;
>> +	}
>> +
>> +	arr = kcalloc(n, sizeof(*arr), GFP_KERNEL);
>> +	if (!arr)
>> +		return -ENOMEM;
>> +
>> +	for (i = 0, e = arr; i < n; i++, mptr++, e++) {
>> +		e->segment = mptr->pci_segment;
>> +		e->addr =  mptr->address;
>> +		e->bus_start = mptr->start_bus_number;
>> +		e->bus_end = mptr->end_bus_number;
>> +		list_add(&e->list, &pci_mcfg_list);
>> +		pr_info(PREFIX
>> +			"MCFG entry for domain %04x [bus %02x-%02x] (base %pa)\n",
>> +			e->segment, e->bus_start, e->bus_end, &e->addr);
> Ah, this is information similar to what I suggested we might print in
> pci_generic_ecam_create().  Could go either way, but personally I
> think I'd put it in pci_generic_ecam_create() instead, because then we
> only print the info we're actually using.  And I think it'd be nice to
> have the actual MMIO resource ("[mem 0x...-0x...]") instead of just
> the base.

Giving above that I will drop MCFG entries cache, this will be printed 
in pci_generic_ecam_create, as you suggested.
>
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +/* Interface called by ACPI - parse and save MCFG table */
>> +void __init pci_mcfg_init(void)
>> +{
>> +	int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
>> +	if (err)
>> +		pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);
>> +	else if (list_empty(&pci_mcfg_list))
>> +		pr_info(PREFIX "No valid entries in MCFG table.\n");
>> +	else {
>> +		struct mcfg_entry *e;
>> +		int i = 0;
>> +		list_for_each_entry(e, &pci_mcfg_list, list)
>> +			i++;
>> +		pr_info(PREFIX "MCFG table loaded, %d entries\n", i);
>> +	}
>> +}
>> +
>> +/* Raw operations, works only for MCFG entries with an associated bus */
> Can you elaborate a little on the connection with MCFG?  I don't quite
> see how they're related.  Obviously the device has to be on a bus
> we've already enumerated and assigned bus->ops for, but it seems like
> we don't really know or care what bus->ops actually is.  Given that
> these are in this file, acpi_pci_root_ops is the only possibility,
> since that's what we pass to acpi_pci_root_create(), but it doesn't
> seem worth mentioning specifically in a comment.

Yes, you are right. I will drop this comment.

>
>> +int raw_pci_read(unsigned int domain, unsigned int busn, unsigned int devfn,
>> +		 int reg, int len, u32 *val)
>> +{
>> +	struct pci_bus *bus = pci_find_bus(domain, busn);
>> +
>> +	if (!bus)
>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>> +	return bus->ops->read(bus, devfn, reg, len, val);
>> +}
>> +
>> +int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int devfn,
>> +		  int reg, int len, u32 val)
>> +{
>> +	struct pci_bus *bus = pci_find_bus(domain, busn);
>> +
>> +	if (!bus)
>> +		return PCIBIOS_DEVICE_NOT_FOUND;
>> +	return bus->ops->write(bus, devfn, reg, len, val);
>> +}
>> diff --git a/include/linux/pci.h b/include/linux/pci.h
>> index df1f33d..c0422ea 100644
>> --- a/include/linux/pci.h
>> +++ b/include/linux/pci.h
>> @@ -1729,6 +1729,12 @@ static inline void pci_mmcfg_early_init(void) { }
>>   static inline void pci_mmcfg_late_init(void) { }
>>   #endif
>>   
>> +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
>> +void __init pci_mcfg_init(void);
>> +#else
>> +static inline void pci_mcfg_init(void) { return; }
>> +#endif
>> +
>>   int pci_ext_cfg_avail(void);
>>   
>>   void __iomem *pci_ioremap_bar(struct pci_dev *pdev, int bar);
>> -- 
>> 1.9.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thanks,
Tomasz

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Tomasz Nowicki]

On 04/29/2016 07:35 PM, Jayachandran C wrote:
> On Fri, Apr 29, 2016 at 2:07 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
>> On Thu, Apr 28, 2016 at 04:48:00PM -0500, Bjorn Helgaas wrote:
>>
>> [...]
>>
>>>> +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
>>>> +                                  struct acpi_pci_generic_root_info *ri)
>>>> +{
>>>> +   u16 seg = root->segment;
>>>> +   u8 bus_start = root->secondary.start;
>>>> +   u8 bus_end = root->secondary.end;
>>>> +   struct pci_config_window *cfg;
>>>> +   struct mcfg_entry *e;
>>>> +   phys_addr_t addr;
>>>> +   int err = 0;
>>>> +
>>>> +   mutex_lock(&pci_mcfg_lock);
>>> What does this lock protect?  The pci_mcfg_list should already be
>>> initialized by the time we get there, and it should be immutable for
>>> the life of the system.  In fact, I would prefer if we could just
>>> search the static table itself whenever we need it rather than caching
>>> it in our own list.  But I don't think we can easily do that because
>>> acpi_table_parse() is __init.
>>>
>>>> +   e = pci_mcfg_lookup(seg, bus_start);
>>> I would argue that we should check for _CBA first, and fall back to
>>> MCFG if _CBA doesn't exist.
>>>
>>>> +   if (!e) {
>>>> +           addr = acpi_pci_root_get_mcfg_addr(root->device->handle);
>>> IMO, acpi_pci_root_get_mcfg_addr() is misnamed.  It should be
>>> acpi_pci_config_base_addr() or similar.  It definitely is not related
>>> to MCFG.  Not your fault, obviously.
>>>
>>>> +           if (addr == 0) {
>>>> +                   pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
>>>> +                          seg, bus_start, bus_end);
>>>> +                   err = -ENOENT;
>>>> +                   goto err_out;
>>>> +           }
>>>> +   } else {
>>>> +           if (bus_start != e->bus_start) {
>>>> +                   pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
>>>> +                          seg, bus_start, bus_end, e->bus_start);
>>>> +                   err = -EINVAL;
>>>> +                   goto err_out;
>>>> +           } else if (bus_end != e->bus_end) {
>>>> +                   pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
>>>> +                           seg, bus_start, bus_end, e->bus_end);
>>>> +                   bus_end = min(bus_end, e->bus_end);
>>>> +           }
>>>> +           addr = e->addr;
>>>> +   }
>>> I really don't think you need a lock around this, so you can factor
>>> out the address lookup into something like:
>>>
>>>    addr = acpi_pci_config_base_addr(...);
>>>    if (addr)
>>>      return addr;
>>>
>>>    return acpi_pci_mcfg_lookup(seg, busn_res);
>>>
>>> You can check inside acpi_pci_mcfg_lookup() to make sure the entry you
>>> find covers the entire [busn_res.start-busn_res.end] range and return
>>> failure if it doesn't.  At this point, I'm not sure it's worth it to
>>> truncate the host bridge bus range to match something we find in MCFG.
>>>
>>> If the MCFG entry covers *more* than the host bridge range from _CRS,
>>> that's fine.  In any case, we have to be careful with the start address,
>>> because the MCFG start address is always based on bus 0, but I think
>>> pci_generic_ecam_create() expects the start address based on the
>>> bus_start you pass to it.
>> Yes, I spotted this too, it is unfortunate but DT and MCFG handle
>> the ECAM regions differently. In DT the reg property is relative
>> to bus_start - ie reg MMIO region maps config space starting at
>> the first bus in bus-range:
>>
>> Documentation/devicetree/bindings/pci/host-generic-pci.txt
>>
>> in ACPI(MCFG) as you said it is always relative to bus 0, it is
>> unfortunate but the address to be mapped should be computed
>> differently in the ECAM layer.
> Can't this be handled by fixing up the address before passing to
> pci_generic_ecam_create?
>
I agree, this should work, IMO.

Tomasz

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Tomasz Nowicki]

On 04/27/2016 01:17 PM, Lorenzo Pieralisi wrote:
> On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
>> On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
[...]
>>   
>> +int acpi_pci_bus_domain_nr(struct device *parent)
>> +{
>> +	struct acpi_device *acpi_dev = to_acpi_device(parent);
>> +	unsigned long long segment = 0;
>> +	acpi_status status;
>> +
>> +	/*
>> +	 * If _SEG method does not exist, following ACPI spec (6.5.6)
>> +	 * all PCI buses belong to domain 0.
>> +	 */
>> +	status = acpi_evaluate_integer(acpi_dev->handle, METHOD_NAME__SEG, NULL,
>> +				       &segment);
>> We already have code in acpi_pci_root_add() to evaluate _SEG.  We
>> don't want to evaluate it *twice*, do we?
>>
>> I was sort of expecting that if you added it here, we'd remove the
>> existing call, but it looks like you're keeping both?
> We can't remove the existing call, since it is used on X86 and IA64
> to store the segment number that, in the process, is used in their
> pci_domain_nr() arch specific callback to retrieve the domain nr.
>
> On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
> to retrieve the domain number that is not arch dependent, since
> this is generic code, we can't rely on any bus->sysdata format (unless
> we do something like JC did below), therefore the only way is to call
> the _SEG method *again* here, which also forced Tomasz to go through
> the ACPI_COMPANION setting song and dance and pass the parent pointer
> to pci_create_root_bus() (see patch 1), which BTW is a source of
> trouble on its own as you noticed.
What trouble in patch 1 do you mean? I may miss something.

I agree that patch 1 is not necessary if we decide to use sysdata or 
rework root bus scanning to move domain to host bridge. Nevertheless, 
patch 1 is still a cleanup IMO.

Thanks,
Tomasz

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Jayachandran C]

On Mon, May 2, 2016 at 6:13 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
> On 04/27/2016 01:17 PM, Lorenzo Pieralisi wrote:
>>
>> On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
>>>
>>> On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
>
> [...]
>
>>>   +int acpi_pci_bus_domain_nr(struct device *parent)
>>> +{
>>> +       struct acpi_device *acpi_dev = to_acpi_device(parent);
>>> +       unsigned long long segment = 0;
>>> +       acpi_status status;
>>> +
>>> +       /*
>>> +        * If _SEG method does not exist, following ACPI spec (6.5.6)
>>> +        * all PCI buses belong to domain 0.
>>> +        */
>>> +       status = acpi_evaluate_integer(acpi_dev->handle,
>>> METHOD_NAME__SEG, NULL,
>>> +                                      &segment);
>>> We already have code in acpi_pci_root_add() to evaluate _SEG.  We
>>> don't want to evaluate it *twice*, do we?
>>>
>>> I was sort of expecting that if you added it here, we'd remove the
>>> existing call, but it looks like you're keeping both?
>>
>> We can't remove the existing call, since it is used on X86 and IA64
>> to store the segment number that, in the process, is used in their
>> pci_domain_nr() arch specific callback to retrieve the domain nr.
>>
>> On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
>> to retrieve the domain number that is not arch dependent, since
>> this is generic code, we can't rely on any bus->sysdata format (unless
>> we do something like JC did below), therefore the only way is to call
>> the _SEG method *again* here, which also forced Tomasz to go through
>> the ACPI_COMPANION setting song and dance and pass the parent pointer
>> to pci_create_root_bus() (see patch 1), which BTW is a source of
>> trouble on its own as you noticed.
>
> What trouble in patch 1 do you mean? I may miss something.
>
> I agree that patch 1 is not necessary if we decide to use sysdata or rework
> root bus scanning to move domain to host bridge. Nevertheless, patch 1 is
> still a cleanup IMO.

In this case, getting the domain should be trivial since the ACPI
companion on parent is already set, this should work

int acpi_pci_bus_domain_nr(struct device *parent)
{
        struct acpi_device *acpi_dev = to_acpi_device(parent);
        struct acpi_pci_root *root = acpi_dev->driver_data;

        return root->segment;
}

Or am I missing something here?

JC.

Re: [PATCH V6 09/13] pci, acpi: Support for ACPI based generic PCI host controller

[Lorenzo Pieralisi]

On Fri, Apr 29, 2016 at 11:05:34PM +0530, Jayachandran C wrote:
> On Fri, Apr 29, 2016 at 2:07 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> > On Thu, Apr 28, 2016 at 04:48:00PM -0500, Bjorn Helgaas wrote:
> >
> > [...]
> >
> >> > +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
> >> > +                                  struct acpi_pci_generic_root_info *ri)
> >> > +{
> >> > +   u16 seg = root->segment;
> >> > +   u8 bus_start = root->secondary.start;
> >> > +   u8 bus_end = root->secondary.end;
> >> > +   struct pci_config_window *cfg;
> >> > +   struct mcfg_entry *e;
> >> > +   phys_addr_t addr;
> >> > +   int err = 0;
> >> > +
> >> > +   mutex_lock(&pci_mcfg_lock);
> >>
> >> What does this lock protect?  The pci_mcfg_list should already be
> >> initialized by the time we get there, and it should be immutable for
> >> the life of the system.  In fact, I would prefer if we could just
> >> search the static table itself whenever we need it rather than caching
> >> it in our own list.  But I don't think we can easily do that because
> >> acpi_table_parse() is __init.
> >>
> >> > +   e = pci_mcfg_lookup(seg, bus_start);
> >>
> >> I would argue that we should check for _CBA first, and fall back to
> >> MCFG if _CBA doesn't exist.
> >>
> >> > +   if (!e) {
> >> > +           addr = acpi_pci_root_get_mcfg_addr(root->device->handle);
> >>
> >> IMO, acpi_pci_root_get_mcfg_addr() is misnamed.  It should be
> >> acpi_pci_config_base_addr() or similar.  It definitely is not related
> >> to MCFG.  Not your fault, obviously.
> >>
> >> > +           if (addr == 0) {
> >> > +                   pr_err(PREFIX"%04x:%02x-%02x bus range error\n",
> >> > +                          seg, bus_start, bus_end);
> >> > +                   err = -ENOENT;
> >> > +                   goto err_out;
> >> > +           }
> >> > +   } else {
> >> > +           if (bus_start != e->bus_start) {
> >> > +                   pr_err("%04x:%02x-%02x bus range mismatch %02x\n",
> >> > +                          seg, bus_start, bus_end, e->bus_start);
> >> > +                   err = -EINVAL;
> >> > +                   goto err_out;
> >> > +           } else if (bus_end != e->bus_end) {
> >> > +                   pr_warn("%04x:%02x-%02x bus end mismatch %02x\n",
> >> > +                           seg, bus_start, bus_end, e->bus_end);
> >> > +                   bus_end = min(bus_end, e->bus_end);
> >> > +           }
> >> > +           addr = e->addr;
> >> > +   }
> >>
> >> I really don't think you need a lock around this, so you can factor
> >> out the address lookup into something like:
> >>
> >>   addr = acpi_pci_config_base_addr(...);
> >>   if (addr)
> >>     return addr;
> >>
> >>   return acpi_pci_mcfg_lookup(seg, busn_res);
> >>
> >> You can check inside acpi_pci_mcfg_lookup() to make sure the entry you
> >> find covers the entire [busn_res.start-busn_res.end] range and return
> >> failure if it doesn't.  At this point, I'm not sure it's worth it to
> >> truncate the host bridge bus range to match something we find in MCFG.
> >>
> >> If the MCFG entry covers *more* than the host bridge range from _CRS,
> >> that's fine.  In any case, we have to be careful with the start address,
> >> because the MCFG start address is always based on bus 0, but I think
> >> pci_generic_ecam_create() expects the start address based on the
> >> bus_start you pass to it.
> >
> > Yes, I spotted this too, it is unfortunate but DT and MCFG handle
> > the ECAM regions differently. In DT the reg property is relative
> > to bus_start - ie reg MMIO region maps config space starting at
> > the first bus in bus-range:
> >
> > Documentation/devicetree/bindings/pci/host-generic-pci.txt
> >
> > in ACPI(MCFG) as you said it is always relative to bus 0, it is
> > unfortunate but the address to be mapped should be computed
> > differently in the ECAM layer.
> 
> Can't this be handled by fixing up the address before passing to
> pci_generic_ecam_create?

Yes it can, you just need to apply the bus shift, given that we know
it is ECAM anyway you can even add a macro in the ecam generic header to
compute it, anyway that's a minor detail, we just should not forget to
fix it.

Lorenzo

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Lorenzo Pieralisi]

On Mon, May 02, 2016 at 06:56:13PM +0530, Jayachandran C wrote:
> On Mon, May 2, 2016 at 6:13 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
> > On 04/27/2016 01:17 PM, Lorenzo Pieralisi wrote:
> >>
> >> On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
> >>>
> >>> On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
> >
> > [...]
> >
> >>>   +int acpi_pci_bus_domain_nr(struct device *parent)
> >>> +{
> >>> +       struct acpi_device *acpi_dev = to_acpi_device(parent);
> >>> +       unsigned long long segment = 0;
> >>> +       acpi_status status;
> >>> +
> >>> +       /*
> >>> +        * If _SEG method does not exist, following ACPI spec (6.5.6)
> >>> +        * all PCI buses belong to domain 0.
> >>> +        */
> >>> +       status = acpi_evaluate_integer(acpi_dev->handle,
> >>> METHOD_NAME__SEG, NULL,
> >>> +                                      &segment);
> >>> We already have code in acpi_pci_root_add() to evaluate _SEG.  We
> >>> don't want to evaluate it *twice*, do we?
> >>>
> >>> I was sort of expecting that if you added it here, we'd remove the
> >>> existing call, but it looks like you're keeping both?
> >>
> >> We can't remove the existing call, since it is used on X86 and IA64
> >> to store the segment number that, in the process, is used in their
> >> pci_domain_nr() arch specific callback to retrieve the domain nr.
> >>
> >> On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
> >> to retrieve the domain number that is not arch dependent, since
> >> this is generic code, we can't rely on any bus->sysdata format (unless
> >> we do something like JC did below), therefore the only way is to call
> >> the _SEG method *again* here, which also forced Tomasz to go through
> >> the ACPI_COMPANION setting song and dance and pass the parent pointer
> >> to pci_create_root_bus() (see patch 1), which BTW is a source of
> >> trouble on its own as you noticed.
> >
> > What trouble in patch 1 do you mean? I may miss something.
> >
> > I agree that patch 1 is not necessary if we decide to use sysdata or rework
> > root bus scanning to move domain to host bridge. Nevertheless, patch 1 is
> > still a cleanup IMO.
> 
> In this case, getting the domain should be trivial since the ACPI
> companion on parent is already set, this should work
> 
> int acpi_pci_bus_domain_nr(struct device *parent)
> {
>         struct acpi_device *acpi_dev = to_acpi_device(parent);
>         struct acpi_pci_root *root = acpi_dev->driver_data;
> 
>         return root->segment;
> }
> 
> Or am I missing something here?

Well, I thought that the whole idea behind this exercise was to move
the domain number into struct pci_host_bridge (Arnd did not do it with
his first set but this does not mean we can't add it as Bjorn suggested),
so that the domain number could be read from there straight away in an
arch (and FW) independent manner, right ?

Lorenzo

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Jayachandran C]

On Tue, May 3, 2016 at 4:32 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
> On Mon, May 02, 2016 at 06:56:13PM +0530, Jayachandran C wrote:
>> On Mon, May 2, 2016 at 6:13 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
>> > On 04/27/2016 01:17 PM, Lorenzo Pieralisi wrote:
>> >>
>> >> On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
>> >>>
>> >>> On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
>> >
>> > [...]
>> >
>> >>>   +int acpi_pci_bus_domain_nr(struct device *parent)
>> >>> +{
>> >>> +       struct acpi_device *acpi_dev = to_acpi_device(parent);
>> >>> +       unsigned long long segment = 0;
>> >>> +       acpi_status status;
>> >>> +
>> >>> +       /*
>> >>> +        * If _SEG method does not exist, following ACPI spec (6.5.6)
>> >>> +        * all PCI buses belong to domain 0.
>> >>> +        */
>> >>> +       status = acpi_evaluate_integer(acpi_dev->handle,
>> >>> METHOD_NAME__SEG, NULL,
>> >>> +                                      &segment);
>> >>> We already have code in acpi_pci_root_add() to evaluate _SEG.  We
>> >>> don't want to evaluate it *twice*, do we?
>> >>>
>> >>> I was sort of expecting that if you added it here, we'd remove the
>> >>> existing call, but it looks like you're keeping both?
>> >>
>> >> We can't remove the existing call, since it is used on X86 and IA64
>> >> to store the segment number that, in the process, is used in their
>> >> pci_domain_nr() arch specific callback to retrieve the domain nr.
>> >>
>> >> On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
>> >> to retrieve the domain number that is not arch dependent, since
>> >> this is generic code, we can't rely on any bus->sysdata format (unless
>> >> we do something like JC did below), therefore the only way is to call
>> >> the _SEG method *again* here, which also forced Tomasz to go through
>> >> the ACPI_COMPANION setting song and dance and pass the parent pointer
>> >> to pci_create_root_bus() (see patch 1), which BTW is a source of
>> >> trouble on its own as you noticed.
>> >
>> > What trouble in patch 1 do you mean? I may miss something.
>> >
>> > I agree that patch 1 is not necessary if we decide to use sysdata or rework
>> > root bus scanning to move domain to host bridge. Nevertheless, patch 1 is
>> > still a cleanup IMO.
>>
>> In this case, getting the domain should be trivial since the ACPI
>> companion on parent is already set, this should work
>>
>> int acpi_pci_bus_domain_nr(struct device *parent)
>> {
>>         struct acpi_device *acpi_dev = to_acpi_device(parent);
>>         struct acpi_pci_root *root = acpi_dev->driver_data;
>>
>>         return root->segment;
>> }
>>
>> Or am I missing something here?
>
> Well, I thought that the whole idea behind this exercise was to move
> the domain number into struct pci_host_bridge (Arnd did not do it with
> his first set but this does not mean we can't add it as Bjorn suggested),
> so that the domain number could be read from there straight away in an
> arch (and FW) independent manner, right ?

The original issue was using _SEG call again instead of using
the value from acpi_pci_root, and the solution for that problem
is very simple.

The pci host bridge work is of course very useful, but creating a
dependency with that work for an issue that can be so easily solved
is unnecessary.

JC.

Re: [PATCH V6 02/13] pci, acpi: Provide generic way to assign bus domain number.

[Lorenzo Pieralisi]

On Tue, May 03, 2016 at 07:52:52PM +0530, Jayachandran C wrote:
> On Tue, May 3, 2016 at 4:32 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> > On Mon, May 02, 2016 at 06:56:13PM +0530, Jayachandran C wrote:
> >> On Mon, May 2, 2016 at 6:13 PM, Tomasz Nowicki <tn@semihalf.com> wrote:
> >> > On 04/27/2016 01:17 PM, Lorenzo Pieralisi wrote:
> >> >>
> >> >> On Tue, Apr 26, 2016 at 09:26:49PM -0500, Bjorn Helgaas wrote:
> >> >>>
> >> >>> On Fri, Apr 15, 2016 at 07:06:37PM +0200, Tomasz Nowicki wrote:
> >> >
> >> > [...]
> >> >
> >> >>>   +int acpi_pci_bus_domain_nr(struct device *parent)
> >> >>> +{
> >> >>> +       struct acpi_device *acpi_dev = to_acpi_device(parent);
> >> >>> +       unsigned long long segment = 0;
> >> >>> +       acpi_status status;
> >> >>> +
> >> >>> +       /*
> >> >>> +        * If _SEG method does not exist, following ACPI spec (6.5.6)
> >> >>> +        * all PCI buses belong to domain 0.
> >> >>> +        */
> >> >>> +       status = acpi_evaluate_integer(acpi_dev->handle,
> >> >>> METHOD_NAME__SEG, NULL,
> >> >>> +                                      &segment);
> >> >>> We already have code in acpi_pci_root_add() to evaluate _SEG.  We
> >> >>> don't want to evaluate it *twice*, do we?
> >> >>>
> >> >>> I was sort of expecting that if you added it here, we'd remove the
> >> >>> existing call, but it looks like you're keeping both?
> >> >>
> >> >> We can't remove the existing call, since it is used on X86 and IA64
> >> >> to store the segment number that, in the process, is used in their
> >> >> pci_domain_nr() arch specific callback to retrieve the domain nr.
> >> >>
> >> >> On ARM64, that selects PCI_DOMAINS_GENERIC, we have to find a way
> >> >> to retrieve the domain number that is not arch dependent, since
> >> >> this is generic code, we can't rely on any bus->sysdata format (unless
> >> >> we do something like JC did below), therefore the only way is to call
> >> >> the _SEG method *again* here, which also forced Tomasz to go through
> >> >> the ACPI_COMPANION setting song and dance and pass the parent pointer
> >> >> to pci_create_root_bus() (see patch 1), which BTW is a source of
> >> >> trouble on its own as you noticed.
> >> >
> >> > What trouble in patch 1 do you mean? I may miss something.
> >> >
> >> > I agree that patch 1 is not necessary if we decide to use sysdata or rework
> >> > root bus scanning to move domain to host bridge. Nevertheless, patch 1 is
> >> > still a cleanup IMO.
> >>
> >> In this case, getting the domain should be trivial since the ACPI
> >> companion on parent is already set, this should work
> >>
> >> int acpi_pci_bus_domain_nr(struct device *parent)
> >> {
> >>         struct acpi_device *acpi_dev = to_acpi_device(parent);
> >>         struct acpi_pci_root *root = acpi_dev->driver_data;
> >>
> >>         return root->segment;
> >> }
> >>
> >> Or am I missing something here?
> >
> > Well, I thought that the whole idea behind this exercise was to move
> > the domain number into struct pci_host_bridge (Arnd did not do it with
> > his first set but this does not mean we can't add it as Bjorn suggested),
> > so that the domain number could be read from there straight away in an
> > arch (and FW) independent manner, right ?
> 
> The original issue was using _SEG call again instead of using
> the value from acpi_pci_root, and the solution for that problem
> is very simple.
> 
> The pci host bridge work is of course very useful, but creating a
> dependency with that work for an issue that can be so easily solved
> is unnecessary.

It can be easily solved if we do not drop patch 1 (I understood the
additional call to _SEG was only part of the problem). If we keep patch
1 the code above is fine by me, if we have to drop patch 1 (IIRC by
passing the parent pointer to pci_create_root_bus() we are affecting
IA64 and X86 code, if that's acceptable that's fine by me) I do not
think we can use the code above anymore, that's what my comment was
all about because Bjorn was concerned about the fragility of the code
setting the ACPI companion.

Lorenzo

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Tomasz Nowicki]

On 27.04.2016 04:45, Bjorn Helgaas wrote:
> [question for Rafael below]
>
> On Fri, Apr 15, 2016 at 07:06:36PM +0200, Tomasz Nowicki wrote:
>> Currently we have two platforms (x86 & ia64) capable of PCI ACPI host
>> bridge initialization. They both use arch-specific sysdata to pass down
>> parent device reference and both rely on NULL parent in pci_create_root_bus()
>> to validate sysdata content.
>>
>> It looks hacky and prevents us from getting some firmware specific
>> info for PCI host controller based on its acpi_device structure
>> in generic pci_create_root_bus() function. However, we overcome that
>> blocker by passing down parent device via pci_create_root_bus parameter
>> (as the ACPI device type). Then we use ACPI_COMPANION_SET in core code
>> for ACPI boot method only. ACPI_COMPANION_SET is safe to run for all
>> cases DT, ACPI and DT&ACPI.
>>
>> Since now PCI core code is setting ACPI companion device for us,
>> x86 & ia64 specific ACPI companion device setting turns out to be dead now.
>> We can get rid of it, including related companion reference from
>> PCI sysdata structure. Aslo, PCI_CONTROLLER macro cannot return valid
>> companion device anymore. Therefore we need to convert its usage to
>> ACPI_COMPANION.
>>
>> Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
>> Tested-by: Duc Dang <dhdang@apm.com>
>> Tested-by: Dongdong Liu <liudongdong3@huawei.com>
>> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
>> Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
>> Tested-by: Sinan Kaya <okaya@codeaurora.org>
>> ---
>>   arch/ia64/hp/common/sba_iommu.c    |  2 +-
>>   arch/ia64/include/asm/pci.h        |  1 -
>>   arch/ia64/pci/pci.c                | 16 ----------------
>>   arch/ia64/sn/kernel/io_acpi_init.c |  4 ++--
>>   arch/x86/include/asm/pci.h         |  3 ---
>>   arch/x86/pci/acpi.c                | 17 -----------------
>>   drivers/acpi/pci_root.c            |  7 ++++++-
>>   drivers/pci/probe.c                |  2 ++
>>   8 files changed, 11 insertions(+), 41 deletions(-)
>>
>> diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
>> index a6d6190..78e4444 100644
>> --- a/arch/ia64/hp/common/sba_iommu.c
>> +++ b/arch/ia64/hp/common/sba_iommu.c
>> @@ -1981,7 +1981,7 @@ sba_connect_bus(struct pci_bus *bus)
>>   	if (PCI_CONTROLLER(bus)->iommu)
>>   		return;
>>
>> -	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
>> +	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
>>   	if (!handle)
>>   		return;
>>
>> diff --git a/arch/ia64/include/asm/pci.h b/arch/ia64/include/asm/pci.h
>> index c0835b0..12423f4 100644
>> --- a/arch/ia64/include/asm/pci.h
>> +++ b/arch/ia64/include/asm/pci.h
>> @@ -63,7 +63,6 @@ extern int pci_mmap_legacy_page_range(struct pci_bus *bus,
>>   #define pci_legacy_write platform_pci_legacy_write
>>
>>   struct pci_controller {
>> -	struct acpi_device *companion;
>>   	void *iommu;
>>   	int segment;
>>   	int node;		/* nearest node with memory or NUMA_NO_NODE for global allocation */
>> diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
>> index 8f6ac2f..978d6af 100644
>> --- a/arch/ia64/pci/pci.c
>> +++ b/arch/ia64/pci/pci.c
>> @@ -301,28 +301,12 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>>   	}
>>
>>   	info->controller.segment = root->segment;
>> -	info->controller.companion = device;
>>   	info->controller.node = acpi_get_node(device->handle);
>>   	INIT_LIST_HEAD(&info->io_resources);
>>   	return acpi_pci_root_create(root, &pci_acpi_root_ops,
>>   				    &info->common, &info->controller);
>>   }
>>
>> -int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
>> -{
>> -	/*
>> -	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
>> -	 * here, pci_create_root_bus() has been called by someone else and
>> -	 * sysdata is likely to be different from what we expect.  Let it go in
>> -	 * that case.
>> -	 */
>> -	if (!bridge->dev.parent) {
>> -		struct pci_controller *controller = bridge->bus->sysdata;
>> -		ACPI_COMPANION_SET(&bridge->dev, controller->companion);
>> -	}
>> -	return 0;
>> -}
>> -
>>   void pcibios_fixup_device_resources(struct pci_dev *dev)
>>   {
>>   	int idx;
>> diff --git a/arch/ia64/sn/kernel/io_acpi_init.c b/arch/ia64/sn/kernel/io_acpi_init.c
>> index 231234c..e454492 100644
>> --- a/arch/ia64/sn/kernel/io_acpi_init.c
>> +++ b/arch/ia64/sn/kernel/io_acpi_init.c
>> @@ -132,7 +132,7 @@ sn_get_bussoft_ptr(struct pci_bus *bus)
>>   	struct acpi_resource_vendor_typed *vendor;
>>
>>
>> -	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
>> +	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
>>   	status = acpi_get_vendor_resource(handle, METHOD_NAME__CRS,
>>   					  &sn_uuid, &buffer);
>>   	if (ACPI_FAILURE(status)) {
>> @@ -360,7 +360,7 @@ sn_acpi_get_pcidev_info(struct pci_dev *dev, struct pcidev_info **pcidev_info,
>>   	acpi_status status;
>>   	struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
>>
>> -	rootbus_handle = acpi_device_handle(PCI_CONTROLLER(dev)->companion);
>> +	rootbus_handle = acpi_device_handle(ACPI_COMPANION(dev->bus->bridge));
>>           status = acpi_evaluate_integer(rootbus_handle, METHOD_NAME__SEG, NULL,
>>                                          &segment);
>>           if (ACPI_SUCCESS(status)) {
>> diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
>> index 9ab7507..24de07d 100644
>> --- a/arch/x86/include/asm/pci.h
>> +++ b/arch/x86/include/asm/pci.h
>> @@ -14,9 +14,6 @@
>>   struct pci_sysdata {
>>   	int		domain;		/* PCI domain */
>>   	int		node;		/* NUMA node */
>> -#ifdef CONFIG_ACPI
>> -	struct acpi_device *companion;	/* ACPI companion device */
>> -#endif
>>   #ifdef CONFIG_X86_64
>>   	void		*iommu;		/* IOMMU private data */
>>   #endif
>> diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
>> index 3cd6983..f4ca17a 100644
>> --- a/arch/x86/pci/acpi.c
>> +++ b/arch/x86/pci/acpi.c
>> @@ -340,7 +340,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>>   		struct pci_sysdata sd = {
>>   			.domain = domain,
>>   			.node = node,
>> -			.companion = root->device
>>   		};
>>
>>   		memcpy(bus->sysdata, &sd, sizeof(sd));
>> @@ -355,7 +354,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>>   		else {
>>   			info->sd.domain = domain;
>>   			info->sd.node = node;
>> -			info->sd.companion = root->device;
>>   			bus = acpi_pci_root_create(root, &acpi_pci_root_ops,
>>   						   &info->common, &info->sd);
>>   		}
>> @@ -373,21 +371,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>>   	return bus;
>>   }
>>
>> -int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
>> -{
>> -	/*
>> -	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
>> -	 * here, pci_create_root_bus() has been called by someone else and
>> -	 * sysdata is likely to be different from what we expect.  Let it go in
>> -	 * that case.
>> -	 */
>> -	if (!bridge->dev.parent) {
>> -		struct pci_sysdata *sd = bridge->bus->sysdata;
>> -		ACPI_COMPANION_SET(&bridge->dev, sd->companion);
>> -	}
>> -	return 0;
>> -}
>> -
>>   int __init pci_acpi_init(void)
>>   {
>>   	struct pci_dev *dev = NULL;
>> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
>> index ae3fe4e..4581e0e 100644
>> --- a/drivers/acpi/pci_root.c
>> +++ b/drivers/acpi/pci_root.c
>> @@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device,
>>   		}
>>   	}
>>
>> +	/*
>> +	 * pci_create_root_bus() needs to detect the parent device type,
>> +	 * so initialize its companion data accordingly.
>> +	 */
>> +	ACPI_COMPANION_SET(&device->dev, device);
>>   	root->device = device;
>>   	root->segment = segment & 0xFFFF;
>>   	strcpy(acpi_device_name(device), ACPI_PCI_ROOT_DEVICE_NAME);
>> @@ -846,7 +851,7 @@ struct pci_bus *acpi_pci_root_create(struct acpi_pci_root *root,
>>
>>   	pci_acpi_root_add_resources(info);
>>   	pci_add_resource(&info->resources, &root->secondary);
>> -	bus = pci_create_root_bus(NULL, busnum, ops->pci_ops,
>> +	bus = pci_create_root_bus(&device->dev, busnum, ops->pci_ops,
>>   				  sysdata, &info->resources);
>
> "device" here is a struct acpi_device *.  Rafael, is that the right
> thing to do?  I dimly recall proposing something similar long ago and
> that it turned out to be a bad idea.
>

Joining Bjorn's question. Rafael, do you recall what was the issue here 
from the past?

Thanks,
Tomasz

Re: [PATCH V6 07/13] PCI: Provide common functions for ECAM mapping

[Jayachandran C]

On Fri, Apr 29, 2016 at 1:31 PM, Jayachandran C <jchandra@broadcom.com> wrote:
> On Fri, Apr 29, 2016 at 3:17 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
>> On Fri, Apr 15, 2016 at 07:06:42PM +0200, Tomasz Nowicki wrote:
>>> From: Jayachandran C <jchandra@broadcom.com>
>>>
>>> Add config option PCI_GENERIC_ECAM and file drivers/pci/ecam.c to
>>> provide generic functions for accessing memory mapped PCI config space.
>>>
>>> The API is defined in drivers/pci/ecam.h and is written to replace the
>>> API in drivers/pci/host/pci-host-common.h. The file defines a new
>>> 'struct pci_config_window' to hold the information related to a PCI
>>> config area and its mapping. This structure is expected to be used as
>>> sysdata for controllers that have ECAM based mapping.
>>>
>>> Helper functions are provided to setup the mapping, free the mapping
>>> and to implement the map_bus method in 'struct pci_ops'
>>
>> Spec reference: PCI Express Base Specification, rev 3.0, sec 7.2.2.
>>
>>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>>> ---
>>>  drivers/pci/Kconfig  |   3 ++
>>>  drivers/pci/Makefile |   2 +
>>>  drivers/pci/ecam.c   | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>>  drivers/pci/ecam.h   |  61 +++++++++++++++++++++++
>>>  4 files changed, 203 insertions(+)
>>>  create mode 100644 drivers/pci/ecam.c
>>>  create mode 100644 drivers/pci/ecam.h
>>>
>>> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
>>> index 209292e..e930d62 100644
>>> --- a/drivers/pci/Kconfig
>>> +++ b/drivers/pci/Kconfig
>>> @@ -83,6 +83,9 @@ config HT_IRQ
>>>  config PCI_ATS
>>>       bool
>>>
>>> +config PCI_GENERIC_ECAM
>>> +     bool
>>
>> "PCI_ECAM" is enough, I think.  It's defined by and required by the
>> spec unless there's some arch-specific interface.  Plus, if I
>
> Ok. Looking at the comments I think I have to take out generic from
> all the names - will do this in next version.
>
>> understand correctly, this infrastructure supports non-generic ECAM
>> implementations as well, since the caller supplies "struct
>> pci_generic_ecam_ops *ops".
>
> Yes, the idea was to support ECAM with quirks (and CAM) on both
> 32 and 64 bit, otherwise it would be too trivial to have a separate
> implementation.
>
>>>  config PCI_IOV
>>>       bool "PCI IOV support"
>>>       depends on PCI
>>> diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
>>> index 2154092..810aec8 100644
>>> --- a/drivers/pci/Makefile
>>> +++ b/drivers/pci/Makefile
>>> @@ -55,6 +55,8 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o
>>>
>>>  obj-$(CONFIG_PCI_STUB) += pci-stub.o
>>>
>>> +obj-$(CONFIG_PCI_GENERIC_ECAM) += ecam.o
>>> +
>>>  obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
>>>
>>>  obj-$(CONFIG_OF) += of.o
>>> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
>>> new file mode 100644
>>> index 0000000..ff04c01
>>> --- /dev/null
>>> +++ b/drivers/pci/ecam.c
>>> @@ -0,0 +1,137 @@
>>> +/*
>>> + * Copyright 2016 Broadcom
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify
>>> + * it under the terms of the GNU General Public License, version 2, as
>>> + * published by the Free Software Foundation (the "GPL").
>>> + *
>>> + * This program is distributed in the hope that it will be useful, but
>>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>> + * General Public License version 2 (GPLv2) for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> + * version 2 (GPLv2) along with this source code.
>>> + */
>>> +
>>> +#include <linux/device.h>
>>> +#include <linux/io.h>
>>> +#include <linux/kernel.h>
>>> +#include <linux/module.h>
>>> +#include <linux/pci.h>
>>> +#include <linux/slab.h>
>>> +
>>> +#include "ecam.h"
>>> +
>>> +/*
>>> + * On 64 bit systems, we do a single ioremap for the whole config space
>>> + * since we have enough virtual address range available. On 32 bit, do an
>>> + * ioremap per bus.
>>> + */
>>> +static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
>>> +
>>> +/*
>>> + * Create a PCI config space window
>>> + *  - reserve mem region
>>> + *  - alloc struct pci_config_window with space for all mappings
>>> + *  - ioremap the config space
>>> + */
>>> +struct pci_config_window *pci_generic_ecam_create(struct device *dev,
>>> +                             phys_addr_t addr, u8 bus_start, u8 bus_end,
>>
>> Can you take pointers to struct resources here instead of addr,
>> bus_start, and bus_end?  The caller probably has them already, and
>> then you could add a useful printk like:
>>
>>   dev_info(dev, "ECAM for %pR at %pR\n", busn_res, mmio_res);
>>
>> Would have to be careful about the struct resource lifetimes though.
>
> Yes, I had thought of this. The reason I did not go down that path was
> that we are using request_mem_region() which takes the address
> and creates a resource .internally.
>
> Beyond that, as you noted, the ownership and lifetime is slightly
> more complex. Either the calling code has to allocate the
> resource and handoff, or the ecam code has to make a copy of
> the resource. I would go with copy since it is much more simple
> to use.
>
> Since resource based API seems to be preferred, I will switch
> to passing bus and mmio resource and will use
> request_resource_conflict() to allocate the ECAM mem region,
>
>> If you had the MMIO resource here, you could also do the range
>> checking you currently have in gen_pci_init() here instead, so all
>> callers could benefit.
>
> Ok.
>
>>> +                             struct pci_generic_ecam_ops *ops)
>>> +{
>>> +     struct pci_config_window *cfg;
>>> +     unsigned int bus_shift, bus_range, bsz, mapsz;
>>> +     int i, nidx;
>>> +     int err = -ENOMEM;
>>> +
>>> +     if (bus_end < bus_start)
>>> +             return ERR_PTR(-EINVAL);
>>> +
>>> +     bus_shift = ops->bus_shift;
>>> +     bus_range = bus_end - bus_start + 1;
>>> +     bsz = 1 << bus_shift;
>>> +     nidx = per_bus_mapping ? bus_range : 1;
>>> +     mapsz = per_bus_mapping ? bsz : bus_range * bsz;
>>> +     cfg = kzalloc(sizeof(*cfg) + nidx * sizeof(cfg->win[0]), GFP_KERNEL);
>>> +     if (!cfg)
>>> +             return ERR_PTR(-ENOMEM);
>>> +
>>> +     cfg->bus_start = bus_start;
>>> +     cfg->bus_end = bus_end;
>>> +     cfg->ops = ops;
>>> +
>>> +     if (!request_mem_region(addr, bus_range * bsz, "Configuration Space"))
>>> +             goto err_exit;
>>> +
>>> +     /* cfgaddr has to be set after request_mem_region */
>>> +     cfg->cfgaddr = addr;
>>> +
>>> +     for (i = 0; i < nidx; i++) {
>>> +             cfg->win[i] = ioremap(addr + i * mapsz, mapsz);
>>> +             if (!cfg->win[i])
>>> +                     goto err_exit;
>>> +     }
>>> +
>>> +     if (cfg->ops->init) {
>>> +             err = cfg->ops->init(dev, cfg);
>>> +             if (err)
>>> +                     goto err_exit;
>>> +     }
>>> +     return cfg;
>>> +
>>> +err_exit:
>>> +     pci_generic_ecam_free(cfg);
>>> +     return ERR_PTR(err);
>>> +}
>>> +
>>> +/*
>>> + * Free a config space mapping
>>> + */
>>
>> Superfluous comment.
>
> Ok, will trim comments to keep the ones that are useful.
>
>>> +void pci_generic_ecam_free(struct pci_config_window *cfg)
>>> +{
>>> +     unsigned int bus_range;
>>> +     int i, nidx;
>>> +
>>> +     bus_range = cfg->bus_end - cfg->bus_start + 1;
>>> +     nidx = per_bus_mapping ? bus_range : 1;
>>> +     for (i = 0; i < nidx; i++)
>>> +             if (cfg->win[i])
>>> +                     iounmap(cfg->win[i]);
>>> +     if (cfg->cfgaddr)
>>> +             release_mem_region(cfg->cfgaddr,
>>> +                                bus_range << cfg->ops->bus_shift);
>>> +     kfree(cfg);
>>> +}
>>> +
>>> +/*
>>> + * Function to implement the pci_ops ->map_bus method
>>> + */
>>> +void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>>> +                                    int where)
>>> +{
>>> +     struct pci_config_window *cfg = bus->sysdata;
>>
>> I don't really like the use of bus->sysdata here, because sysdata is
>> explicitly arch-specific.
>>
>> But I guess we're in a bind right now: it'd be nice to save the cfg
>> pointer in struct pci_host_bridge, but you have to call
>> pci_generic_ecam_create() before the struct pci_host_bridge has been
>> allocated, and you have to pass the pointer into pci_scan_root_bus(),
>> and there's no generic way to do that yet.
>>
>> So I guess this will have to do for now.
>
> I could call the structure pci_ecam_sysdata instead of pci_config_window
> to make it clear that it is to be used as sysdata for ECAM based PCI host
> controllers.
>
> The current push (at least on ARM/ARM64) seems to be to make
> bus->sysdata controller specific and avoid arch specific sysdata.
>
>>> +     unsigned int devfn_shift = cfg->ops->bus_shift - 8;
>>> +     unsigned int busn = bus->number;
>>> +     void __iomem *base;
>>> +
>>> +     if (busn < cfg->bus_start || busn > cfg->bus_end)
>>> +             return NULL;
>>> +
>>> +     busn -= cfg->bus_start;
>>> +     if (per_bus_mapping)
>>> +             base = cfg->win[busn];
>>> +     else
>>> +             base = cfg->win[0] + (busn << cfg->ops->bus_shift);
>>> +     return base + (devfn << devfn_shift) + where;
>>> +}
>>> +
>>> +/* default ECAM ops */
>>> +struct pci_generic_ecam_ops pci_generic_ecam_default_ops = {
>>
>> Using both "generic" and "default" seems overkill.  Maybe just:
>>
>>   struct pci_ecam_ops pci_generic_ecam_ops = { ... ?
>
> Ok.
>
>>> +     .bus_shift      = 20,
>>> +     .pci_ops                = {
>>> +             .map_bus        = pci_generic_ecam_map_bus,
>>> +             .read           = pci_generic_config_read,
>>> +             .write          = pci_generic_config_write,
>>> +     }
>>> +};
>>> diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
>>> new file mode 100644
>>> index 0000000..34c0aba
>>> --- /dev/null
>>> +++ b/drivers/pci/ecam.h
>>> @@ -0,0 +1,61 @@
>>> +/*
>>> + * Copyright 2016 Broadcom
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify
>>> + * it under the terms of the GNU General Public License, version 2, as
>>> + * published by the Free Software Foundation (the "GPL").
>>> + *
>>> + * This program is distributed in the hope that it will be useful, but
>>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>> + * General Public License version 2 (GPLv2) for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> + * version 2 (GPLv2) along with this source code.
>>> + */
>>> +#ifndef DRIVERS_PCI_ECAM_H
>>> +#define DRIVERS_PCI_ECAM_H
>>> +
>>> +#include <linux/kernel.h>
>>> +#include <linux/platform_device.h>
>>> +
>>> +/*
>>> + * struct to hold pci ops and bus shift of the config window
>>> + * for a PCI controller.
>>> + */
>>> +struct pci_config_window;
>>> +struct pci_generic_ecam_ops {
>>
>> "struct pci_ecam_ops"
>
> Ok.
>
>>> +     unsigned int                    bus_shift;
>>> +     struct pci_ops                  pci_ops;
>>> +     int                             (*init)(struct device *,
>>> +                                             struct pci_config_window *);
>>> +};
>>> +
>>> +/*
>>> + * struct to hold the mappings of a config space window. This
>>> + * will be allocated with enough entries in win[] to hold all
>>> + * the mappings for the bus range.
>>> + */
>>> +struct pci_config_window {
>>> +     phys_addr_t                     cfgaddr;
>>> +     u16                             domain;
>>> +     u8                              bus_start;
>>> +     u8                              bus_end;
>>> +     void                            *priv;
>>> +     struct pci_generic_ecam_ops     *ops;
>>> +     void __iomem                    *win[0];
>>> +};
>>> +
>>> +/* create and free for pci_config_window */
>>
>> Superfluous comment.
>>
>>> +struct pci_config_window *pci_generic_ecam_create(struct device *dev,
>>> +                             phys_addr_t addr, u8 bus_start, u8 bus_end,
>>> +                             struct pci_generic_ecam_ops *ops);
>>> +void pci_generic_ecam_free(struct pci_config_window *cfg);
>>
>> "pci_ecam_create" and "pci_ecam_free"?  I suspect you're going to call
>> these for flavors of ECAM that are definitely not "generic".
>>
>>> +/* map_bus when ->sysdata is an instance of pci_config_window */
>>> +void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>>> +                                    int where);
>>> +/* default ECAM ops, bus shift 20, generic read and write */
>>> +extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
>>> +
>>> +#endif
>
> Thanks for the review. The ECAM patchset can be merged separately
> if you do not want to tie it the ACPI changes.
>
> Please let me know how you want to proceed. Depending on that, I will
> either post it as a separate patchset or ask Tomasz to pick up my
> changes and post the ACPI patchset. again.

I have made the changes suggested here, and added one more
change to move from 'void __iomem *win[0]' to a union for
pci_config_window so that it can be embedded in other structures[1].

The changes are at https://github.com/jchandra-brcm/linux branch
arm64-acpi-pci-v4 so that Tomasz can add it to v7 of the ACPI/PCI
patches. I am not posting the changes here to avoid the impression
that there are clashing ACPI/PCI patchsets.

The branch also has an example patch for ACPI generic PCI root
using the new ECAM code. This also has the simpler domain
setting code for ACPI as well as fix for the ECAM address
calculation which came up during the earlier discussion.

JC.
[1] Minor API changes will be needed for embedding now.
    This change will make it easier to merge with Arnd's host bridge
    changes when that goes in.

Re: [PATCH V6 07/13] PCI: Provide common functions for ECAM mapping

[Tomasz Nowicki]

On 05.05.2016 11:24, Jayachandran C wrote:
> On Fri, Apr 29, 2016 at 1:31 PM, Jayachandran C <jchandra@broadcom.com> wrote:
>> On Fri, Apr 29, 2016 at 3:17 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
>>> On Fri, Apr 15, 2016 at 07:06:42PM +0200, Tomasz Nowicki wrote:
>>>> From: Jayachandran C <jchandra@broadcom.com>
>>>>
>>>> Add config option PCI_GENERIC_ECAM and file drivers/pci/ecam.c to
>>>> provide generic functions for accessing memory mapped PCI config space.
>>>>
>>>> The API is defined in drivers/pci/ecam.h and is written to replace the
>>>> API in drivers/pci/host/pci-host-common.h. The file defines a new
>>>> 'struct pci_config_window' to hold the information related to a PCI
>>>> config area and its mapping. This structure is expected to be used as
>>>> sysdata for controllers that have ECAM based mapping.
>>>>
>>>> Helper functions are provided to setup the mapping, free the mapping
>>>> and to implement the map_bus method in 'struct pci_ops'
>>>
>>> Spec reference: PCI Express Base Specification, rev 3.0, sec 7.2.2.
>>>
>>>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>>>> ---
>>>>   drivers/pci/Kconfig  |   3 ++
>>>>   drivers/pci/Makefile |   2 +
>>>>   drivers/pci/ecam.c   | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>   drivers/pci/ecam.h   |  61 +++++++++++++++++++++++
>>>>   4 files changed, 203 insertions(+)
>>>>   create mode 100644 drivers/pci/ecam.c
>>>>   create mode 100644 drivers/pci/ecam.h
>>>>
>>>> diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
>>>> index 209292e..e930d62 100644
>>>> --- a/drivers/pci/Kconfig
>>>> +++ b/drivers/pci/Kconfig
>>>> @@ -83,6 +83,9 @@ config HT_IRQ
>>>>   config PCI_ATS
>>>>        bool
>>>>
>>>> +config PCI_GENERIC_ECAM
>>>> +     bool
>>>
>>> "PCI_ECAM" is enough, I think.  It's defined by and required by the
>>> spec unless there's some arch-specific interface.  Plus, if I
>>
>> Ok. Looking at the comments I think I have to take out generic from
>> all the names - will do this in next version.
>>
>>> understand correctly, this infrastructure supports non-generic ECAM
>>> implementations as well, since the caller supplies "struct
>>> pci_generic_ecam_ops *ops".
>>
>> Yes, the idea was to support ECAM with quirks (and CAM) on both
>> 32 and 64 bit, otherwise it would be too trivial to have a separate
>> implementation.
>>
>>>>   config PCI_IOV
>>>>        bool "PCI IOV support"
>>>>        depends on PCI
>>>> diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
>>>> index 2154092..810aec8 100644
>>>> --- a/drivers/pci/Makefile
>>>> +++ b/drivers/pci/Makefile
>>>> @@ -55,6 +55,8 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o
>>>>
>>>>   obj-$(CONFIG_PCI_STUB) += pci-stub.o
>>>>
>>>> +obj-$(CONFIG_PCI_GENERIC_ECAM) += ecam.o
>>>> +
>>>>   obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
>>>>
>>>>   obj-$(CONFIG_OF) += of.o
>>>> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
>>>> new file mode 100644
>>>> index 0000000..ff04c01
>>>> --- /dev/null
>>>> +++ b/drivers/pci/ecam.c
>>>> @@ -0,0 +1,137 @@
>>>> +/*
>>>> + * Copyright 2016 Broadcom
>>>> + *
>>>> + * This program is free software; you can redistribute it and/or modify
>>>> + * it under the terms of the GNU General Public License, version 2, as
>>>> + * published by the Free Software Foundation (the "GPL").
>>>> + *
>>>> + * This program is distributed in the hope that it will be useful, but
>>>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>>> + * General Public License version 2 (GPLv2) for more details.
>>>> + *
>>>> + * You should have received a copy of the GNU General Public License
>>>> + * version 2 (GPLv2) along with this source code.
>>>> + */
>>>> +
>>>> +#include <linux/device.h>
>>>> +#include <linux/io.h>
>>>> +#include <linux/kernel.h>
>>>> +#include <linux/module.h>
>>>> +#include <linux/pci.h>
>>>> +#include <linux/slab.h>
>>>> +
>>>> +#include "ecam.h"
>>>> +
>>>> +/*
>>>> + * On 64 bit systems, we do a single ioremap for the whole config space
>>>> + * since we have enough virtual address range available. On 32 bit, do an
>>>> + * ioremap per bus.
>>>> + */
>>>> +static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
>>>> +
>>>> +/*
>>>> + * Create a PCI config space window
>>>> + *  - reserve mem region
>>>> + *  - alloc struct pci_config_window with space for all mappings
>>>> + *  - ioremap the config space
>>>> + */
>>>> +struct pci_config_window *pci_generic_ecam_create(struct device *dev,
>>>> +                             phys_addr_t addr, u8 bus_start, u8 bus_end,
>>>
>>> Can you take pointers to struct resources here instead of addr,
>>> bus_start, and bus_end?  The caller probably has them already, and
>>> then you could add a useful printk like:
>>>
>>>    dev_info(dev, "ECAM for %pR at %pR\n", busn_res, mmio_res);
>>>
>>> Would have to be careful about the struct resource lifetimes though.
>>
>> Yes, I had thought of this. The reason I did not go down that path was
>> that we are using request_mem_region() which takes the address
>> and creates a resource .internally.
>>
>> Beyond that, as you noted, the ownership and lifetime is slightly
>> more complex. Either the calling code has to allocate the
>> resource and handoff, or the ecam code has to make a copy of
>> the resource. I would go with copy since it is much more simple
>> to use.
>>
>> Since resource based API seems to be preferred, I will switch
>> to passing bus and mmio resource and will use
>> request_resource_conflict() to allocate the ECAM mem region,
>>
>>> If you had the MMIO resource here, you could also do the range
>>> checking you currently have in gen_pci_init() here instead, so all
>>> callers could benefit.
>>
>> Ok.
>>
>>>> +                             struct pci_generic_ecam_ops *ops)
>>>> +{
>>>> +     struct pci_config_window *cfg;
>>>> +     unsigned int bus_shift, bus_range, bsz, mapsz;
>>>> +     int i, nidx;
>>>> +     int err = -ENOMEM;
>>>> +
>>>> +     if (bus_end < bus_start)
>>>> +             return ERR_PTR(-EINVAL);
>>>> +
>>>> +     bus_shift = ops->bus_shift;
>>>> +     bus_range = bus_end - bus_start + 1;
>>>> +     bsz = 1 << bus_shift;
>>>> +     nidx = per_bus_mapping ? bus_range : 1;
>>>> +     mapsz = per_bus_mapping ? bsz : bus_range * bsz;
>>>> +     cfg = kzalloc(sizeof(*cfg) + nidx * sizeof(cfg->win[0]), GFP_KERNEL);
>>>> +     if (!cfg)
>>>> +             return ERR_PTR(-ENOMEM);
>>>> +
>>>> +     cfg->bus_start = bus_start;
>>>> +     cfg->bus_end = bus_end;
>>>> +     cfg->ops = ops;
>>>> +
>>>> +     if (!request_mem_region(addr, bus_range * bsz, "Configuration Space"))
>>>> +             goto err_exit;
>>>> +
>>>> +     /* cfgaddr has to be set after request_mem_region */
>>>> +     cfg->cfgaddr = addr;
>>>> +
>>>> +     for (i = 0; i < nidx; i++) {
>>>> +             cfg->win[i] = ioremap(addr + i * mapsz, mapsz);
>>>> +             if (!cfg->win[i])
>>>> +                     goto err_exit;
>>>> +     }
>>>> +
>>>> +     if (cfg->ops->init) {
>>>> +             err = cfg->ops->init(dev, cfg);
>>>> +             if (err)
>>>> +                     goto err_exit;
>>>> +     }
>>>> +     return cfg;
>>>> +
>>>> +err_exit:
>>>> +     pci_generic_ecam_free(cfg);
>>>> +     return ERR_PTR(err);
>>>> +}
>>>> +
>>>> +/*
>>>> + * Free a config space mapping
>>>> + */
>>>
>>> Superfluous comment.
>>
>> Ok, will trim comments to keep the ones that are useful.
>>
>>>> +void pci_generic_ecam_free(struct pci_config_window *cfg)
>>>> +{
>>>> +     unsigned int bus_range;
>>>> +     int i, nidx;
>>>> +
>>>> +     bus_range = cfg->bus_end - cfg->bus_start + 1;
>>>> +     nidx = per_bus_mapping ? bus_range : 1;
>>>> +     for (i = 0; i < nidx; i++)
>>>> +             if (cfg->win[i])
>>>> +                     iounmap(cfg->win[i]);
>>>> +     if (cfg->cfgaddr)
>>>> +             release_mem_region(cfg->cfgaddr,
>>>> +                                bus_range << cfg->ops->bus_shift);
>>>> +     kfree(cfg);
>>>> +}
>>>> +
>>>> +/*
>>>> + * Function to implement the pci_ops ->map_bus method
>>>> + */
>>>> +void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>>>> +                                    int where)
>>>> +{
>>>> +     struct pci_config_window *cfg = bus->sysdata;
>>>
>>> I don't really like the use of bus->sysdata here, because sysdata is
>>> explicitly arch-specific.
>>>
>>> But I guess we're in a bind right now: it'd be nice to save the cfg
>>> pointer in struct pci_host_bridge, but you have to call
>>> pci_generic_ecam_create() before the struct pci_host_bridge has been
>>> allocated, and you have to pass the pointer into pci_scan_root_bus(),
>>> and there's no generic way to do that yet.
>>>
>>> So I guess this will have to do for now.
>>
>> I could call the structure pci_ecam_sysdata instead of pci_config_window
>> to make it clear that it is to be used as sysdata for ECAM based PCI host
>> controllers.
>>
>> The current push (at least on ARM/ARM64) seems to be to make
>> bus->sysdata controller specific and avoid arch specific sysdata.
>>
>>>> +     unsigned int devfn_shift = cfg->ops->bus_shift - 8;
>>>> +     unsigned int busn = bus->number;
>>>> +     void __iomem *base;
>>>> +
>>>> +     if (busn < cfg->bus_start || busn > cfg->bus_end)
>>>> +             return NULL;
>>>> +
>>>> +     busn -= cfg->bus_start;
>>>> +     if (per_bus_mapping)
>>>> +             base = cfg->win[busn];
>>>> +     else
>>>> +             base = cfg->win[0] + (busn << cfg->ops->bus_shift);
>>>> +     return base + (devfn << devfn_shift) + where;
>>>> +}
>>>> +
>>>> +/* default ECAM ops */
>>>> +struct pci_generic_ecam_ops pci_generic_ecam_default_ops = {
>>>
>>> Using both "generic" and "default" seems overkill.  Maybe just:
>>>
>>>    struct pci_ecam_ops pci_generic_ecam_ops = { ... ?
>>
>> Ok.
>>
>>>> +     .bus_shift      = 20,
>>>> +     .pci_ops                = {
>>>> +             .map_bus        = pci_generic_ecam_map_bus,
>>>> +             .read           = pci_generic_config_read,
>>>> +             .write          = pci_generic_config_write,
>>>> +     }
>>>> +};
>>>> diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
>>>> new file mode 100644
>>>> index 0000000..34c0aba
>>>> --- /dev/null
>>>> +++ b/drivers/pci/ecam.h
>>>> @@ -0,0 +1,61 @@
>>>> +/*
>>>> + * Copyright 2016 Broadcom
>>>> + *
>>>> + * This program is free software; you can redistribute it and/or modify
>>>> + * it under the terms of the GNU General Public License, version 2, as
>>>> + * published by the Free Software Foundation (the "GPL").
>>>> + *
>>>> + * This program is distributed in the hope that it will be useful, but
>>>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>>>> + * General Public License version 2 (GPLv2) for more details.
>>>> + *
>>>> + * You should have received a copy of the GNU General Public License
>>>> + * version 2 (GPLv2) along with this source code.
>>>> + */
>>>> +#ifndef DRIVERS_PCI_ECAM_H
>>>> +#define DRIVERS_PCI_ECAM_H
>>>> +
>>>> +#include <linux/kernel.h>
>>>> +#include <linux/platform_device.h>
>>>> +
>>>> +/*
>>>> + * struct to hold pci ops and bus shift of the config window
>>>> + * for a PCI controller.
>>>> + */
>>>> +struct pci_config_window;
>>>> +struct pci_generic_ecam_ops {
>>>
>>> "struct pci_ecam_ops"
>>
>> Ok.
>>
>>>> +     unsigned int                    bus_shift;
>>>> +     struct pci_ops                  pci_ops;
>>>> +     int                             (*init)(struct device *,
>>>> +                                             struct pci_config_window *);
>>>> +};
>>>> +
>>>> +/*
>>>> + * struct to hold the mappings of a config space window. This
>>>> + * will be allocated with enough entries in win[] to hold all
>>>> + * the mappings for the bus range.
>>>> + */
>>>> +struct pci_config_window {
>>>> +     phys_addr_t                     cfgaddr;
>>>> +     u16                             domain;
>>>> +     u8                              bus_start;
>>>> +     u8                              bus_end;
>>>> +     void                            *priv;
>>>> +     struct pci_generic_ecam_ops     *ops;
>>>> +     void __iomem                    *win[0];
>>>> +};
>>>> +
>>>> +/* create and free for pci_config_window */
>>>
>>> Superfluous comment.
>>>
>>>> +struct pci_config_window *pci_generic_ecam_create(struct device *dev,
>>>> +                             phys_addr_t addr, u8 bus_start, u8 bus_end,
>>>> +                             struct pci_generic_ecam_ops *ops);
>>>> +void pci_generic_ecam_free(struct pci_config_window *cfg);
>>>
>>> "pci_ecam_create" and "pci_ecam_free"?  I suspect you're going to call
>>> these for flavors of ECAM that are definitely not "generic".
>>>
>>>> +/* map_bus when ->sysdata is an instance of pci_config_window */
>>>> +void __iomem *pci_generic_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
>>>> +                                    int where);
>>>> +/* default ECAM ops, bus shift 20, generic read and write */
>>>> +extern struct pci_generic_ecam_ops pci_generic_ecam_default_ops;
>>>> +
>>>> +#endif
>>
>> Thanks for the review. The ECAM patchset can be merged separately
>> if you do not want to tie it the ACPI changes.
>>
>> Please let me know how you want to proceed. Depending on that, I will
>> either post it as a separate patchset or ask Tomasz to pick up my
>> changes and post the ACPI patchset. again.
>
> I have made the changes suggested here, and added one more
> change to move from 'void __iomem *win[0]' to a union for
> pci_config_window so that it can be embedded in other structures[1].
>
> The changes are at https://github.com/jchandra-brcm/linux branch
> arm64-acpi-pci-v4 so that Tomasz can add it to v7 of the ACPI/PCI
> patches. I am not posting the changes here to avoid the impression
> that there are clashing ACPI/PCI patchsets.
>
> The branch also has an example patch for ACPI generic PCI root
> using the new ECAM code. This also has the simpler domain
> setting code for ACPI as well as fix for the ECAM address
> calculation which came up during the earlier discussion.
>
> JC.
> [1] Minor API changes will be needed for embedding now.
>      This change will make it easier to merge with Arnd's host bridge
>      changes when that goes in.
>

Thanks JC!

Tomasz

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Rafael J. Wysocki]

On Wednesday, May 04, 2016 10:10:58 AM Tomasz Nowicki wrote:
> On 27.04.2016 04:45, Bjorn Helgaas wrote:
> > [question for Rafael below]
> >
> > On Fri, Apr 15, 2016 at 07:06:36PM +0200, Tomasz Nowicki wrote:
> >> Currently we have two platforms (x86 & ia64) capable of PCI ACPI host
> >> bridge initialization. They both use arch-specific sysdata to pass down
> >> parent device reference and both rely on NULL parent in pci_create_root_bus()
> >> to validate sysdata content.
> >>
> >> It looks hacky and prevents us from getting some firmware specific
> >> info for PCI host controller based on its acpi_device structure
> >> in generic pci_create_root_bus() function. However, we overcome that
> >> blocker by passing down parent device via pci_create_root_bus parameter
> >> (as the ACPI device type). Then we use ACPI_COMPANION_SET in core code
> >> for ACPI boot method only. ACPI_COMPANION_SET is safe to run for all
> >> cases DT, ACPI and DT&ACPI.
> >>
> >> Since now PCI core code is setting ACPI companion device for us,
> >> x86 & ia64 specific ACPI companion device setting turns out to be dead now.
> >> We can get rid of it, including related companion reference from
> >> PCI sysdata structure. Aslo, PCI_CONTROLLER macro cannot return valid
> >> companion device anymore. Therefore we need to convert its usage to
> >> ACPI_COMPANION.
> >>
> >> Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> >> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> >> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> >> Tested-by: Duc Dang <dhdang@apm.com>
> >> Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> >> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> >> Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> >> Tested-by: Sinan Kaya <okaya@codeaurora.org>
> >> ---
> >>   arch/ia64/hp/common/sba_iommu.c    |  2 +-
> >>   arch/ia64/include/asm/pci.h        |  1 -
> >>   arch/ia64/pci/pci.c                | 16 ----------------
> >>   arch/ia64/sn/kernel/io_acpi_init.c |  4 ++--
> >>   arch/x86/include/asm/pci.h         |  3 ---
> >>   arch/x86/pci/acpi.c                | 17 -----------------
> >>   drivers/acpi/pci_root.c            |  7 ++++++-
> >>   drivers/pci/probe.c                |  2 ++
> >>   8 files changed, 11 insertions(+), 41 deletions(-)
> >>
> >> diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
> >> index a6d6190..78e4444 100644
> >> --- a/arch/ia64/hp/common/sba_iommu.c
> >> +++ b/arch/ia64/hp/common/sba_iommu.c
> >> @@ -1981,7 +1981,7 @@ sba_connect_bus(struct pci_bus *bus)
> >>   	if (PCI_CONTROLLER(bus)->iommu)
> >>   		return;
> >>
> >> -	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
> >> +	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
> >>   	if (!handle)
> >>   		return;
> >>
> >> diff --git a/arch/ia64/include/asm/pci.h b/arch/ia64/include/asm/pci.h
> >> index c0835b0..12423f4 100644
> >> --- a/arch/ia64/include/asm/pci.h
> >> +++ b/arch/ia64/include/asm/pci.h
> >> @@ -63,7 +63,6 @@ extern int pci_mmap_legacy_page_range(struct pci_bus *bus,
> >>   #define pci_legacy_write platform_pci_legacy_write
> >>
> >>   struct pci_controller {
> >> -	struct acpi_device *companion;
> >>   	void *iommu;
> >>   	int segment;
> >>   	int node;		/* nearest node with memory or NUMA_NO_NODE for global allocation */
> >> diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
> >> index 8f6ac2f..978d6af 100644
> >> --- a/arch/ia64/pci/pci.c
> >> +++ b/arch/ia64/pci/pci.c
> >> @@ -301,28 +301,12 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
> >>   	}
> >>
> >>   	info->controller.segment = root->segment;
> >> -	info->controller.companion = device;
> >>   	info->controller.node = acpi_get_node(device->handle);
> >>   	INIT_LIST_HEAD(&info->io_resources);
> >>   	return acpi_pci_root_create(root, &pci_acpi_root_ops,
> >>   				    &info->common, &info->controller);
> >>   }
> >>
> >> -int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
> >> -{
> >> -	/*
> >> -	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
> >> -	 * here, pci_create_root_bus() has been called by someone else and
> >> -	 * sysdata is likely to be different from what we expect.  Let it go in
> >> -	 * that case.
> >> -	 */
> >> -	if (!bridge->dev.parent) {
> >> -		struct pci_controller *controller = bridge->bus->sysdata;
> >> -		ACPI_COMPANION_SET(&bridge->dev, controller->companion);
> >> -	}
> >> -	return 0;
> >> -}
> >> -
> >>   void pcibios_fixup_device_resources(struct pci_dev *dev)
> >>   {
> >>   	int idx;
> >> diff --git a/arch/ia64/sn/kernel/io_acpi_init.c b/arch/ia64/sn/kernel/io_acpi_init.c
> >> index 231234c..e454492 100644
> >> --- a/arch/ia64/sn/kernel/io_acpi_init.c
> >> +++ b/arch/ia64/sn/kernel/io_acpi_init.c
> >> @@ -132,7 +132,7 @@ sn_get_bussoft_ptr(struct pci_bus *bus)
> >>   	struct acpi_resource_vendor_typed *vendor;
> >>
> >>
> >> -	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
> >> +	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
> >>   	status = acpi_get_vendor_resource(handle, METHOD_NAME__CRS,
> >>   					  &sn_uuid, &buffer);
> >>   	if (ACPI_FAILURE(status)) {
> >> @@ -360,7 +360,7 @@ sn_acpi_get_pcidev_info(struct pci_dev *dev, struct pcidev_info **pcidev_info,
> >>   	acpi_status status;
> >>   	struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
> >>
> >> -	rootbus_handle = acpi_device_handle(PCI_CONTROLLER(dev)->companion);
> >> +	rootbus_handle = acpi_device_handle(ACPI_COMPANION(dev->bus->bridge));
> >>           status = acpi_evaluate_integer(rootbus_handle, METHOD_NAME__SEG, NULL,
> >>                                          &segment);
> >>           if (ACPI_SUCCESS(status)) {
> >> diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
> >> index 9ab7507..24de07d 100644
> >> --- a/arch/x86/include/asm/pci.h
> >> +++ b/arch/x86/include/asm/pci.h
> >> @@ -14,9 +14,6 @@
> >>   struct pci_sysdata {
> >>   	int		domain;		/* PCI domain */
> >>   	int		node;		/* NUMA node */
> >> -#ifdef CONFIG_ACPI
> >> -	struct acpi_device *companion;	/* ACPI companion device */
> >> -#endif
> >>   #ifdef CONFIG_X86_64
> >>   	void		*iommu;		/* IOMMU private data */
> >>   #endif
> >> diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
> >> index 3cd6983..f4ca17a 100644
> >> --- a/arch/x86/pci/acpi.c
> >> +++ b/arch/x86/pci/acpi.c
> >> @@ -340,7 +340,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
> >>   		struct pci_sysdata sd = {
> >>   			.domain = domain,
> >>   			.node = node,
> >> -			.companion = root->device
> >>   		};
> >>
> >>   		memcpy(bus->sysdata, &sd, sizeof(sd));
> >> @@ -355,7 +354,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
> >>   		else {
> >>   			info->sd.domain = domain;
> >>   			info->sd.node = node;
> >> -			info->sd.companion = root->device;
> >>   			bus = acpi_pci_root_create(root, &acpi_pci_root_ops,
> >>   						   &info->common, &info->sd);
> >>   		}
> >> @@ -373,21 +371,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
> >>   	return bus;
> >>   }
> >>
> >> -int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
> >> -{
> >> -	/*
> >> -	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
> >> -	 * here, pci_create_root_bus() has been called by someone else and
> >> -	 * sysdata is likely to be different from what we expect.  Let it go in
> >> -	 * that case.
> >> -	 */
> >> -	if (!bridge->dev.parent) {
> >> -		struct pci_sysdata *sd = bridge->bus->sysdata;
> >> -		ACPI_COMPANION_SET(&bridge->dev, sd->companion);
> >> -	}
> >> -	return 0;
> >> -}
> >> -
> >>   int __init pci_acpi_init(void)
> >>   {
> >>   	struct pci_dev *dev = NULL;
> >> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> >> index ae3fe4e..4581e0e 100644
> >> --- a/drivers/acpi/pci_root.c
> >> +++ b/drivers/acpi/pci_root.c
> >> @@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device,
> >>   		}
> >>   	}
> >>
> >> +	/*
> >> +	 * pci_create_root_bus() needs to detect the parent device type,
> >> +	 * so initialize its companion data accordingly.
> >> +	 */
> >> +	ACPI_COMPANION_SET(&device->dev, device);
> >>   	root->device = device;
> >>   	root->segment = segment & 0xFFFF;
> >>   	strcpy(acpi_device_name(device), ACPI_PCI_ROOT_DEVICE_NAME);
> >> @@ -846,7 +851,7 @@ struct pci_bus *acpi_pci_root_create(struct acpi_pci_root *root,
> >>
> >>   	pci_acpi_root_add_resources(info);
> >>   	pci_add_resource(&info->resources, &root->secondary);
> >> -	bus = pci_create_root_bus(NULL, busnum, ops->pci_ops,
> >> +	bus = pci_create_root_bus(&device->dev, busnum, ops->pci_ops,
> >>   				  sysdata, &info->resources);
> >
> > "device" here is a struct acpi_device *.  Rafael, is that the right
> > thing to do?  I dimly recall proposing something similar long ago and
> > that it turned out to be a bad idea.
> >
> 
> Joining Bjorn's question. Rafael, do you recall what was the issue here 
> from the past?

Yes, I do.

Anything struct acpi_device doesn't represent a real device.  It represents
a firmware object that can be associated with a device.  IOW, nothing under
LNXSYSTM\:00/ should ever be used as the parent or sibling etc with respect to
anything outside of that directory.  In fact, the entire LNXSYSTM\:00/ should
be located under /sys/firmware/acpi/ and it was a mistake to put it under
/sys/devices/.

One immediate consequence of the above change, AFAICS, would be that the whole
PCI hierarchy would now hang under /sys/devices/LNXSYSTM\:00/LNXSYBUS\:00/PNP0A08\:00/
which would not make any sense whatever, because
/sys/devices/LNXSYSTM\:00/LNXSYBUS\:00/PNP0A08\:00/physical_node already points to
/sys/devices/pci0000\:00/.

IOW, both PNP0A08\:00/ and pci0000\:00/ already represent the same thing, ie. the
host bridge.

If you want to have a parent for pci0000\:00/, you need a physical_node for
LNXSYBUS\:00 and point to that as the parent from pci0000\:00/.  That at least
will lead to a sysfs hierarchy that makes sense, although it may break user
space tools I suppose (which may be assuming that pci0000\:00/ will always be
present directly under /sys/devices/).

Thanks,
Rafael

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Rafael J. Wysocki]

On Friday, April 15, 2016 07:06:36 PM Tomasz Nowicki wrote:
> Currently we have two platforms (x86 & ia64) capable of PCI ACPI host
> bridge initialization. They both use arch-specific sysdata to pass down
> parent device reference and both rely on NULL parent in pci_create_root_bus()
> to validate sysdata content.

No, this is not relied on for that.  It is just NULL, because we don't
have a physical parent device representation for the PCI host bridge.

> It looks hacky and prevents us from getting some firmware specific
> info for PCI host controller based on its acpi_device structure
> in generic pci_create_root_bus() function.

You can't get that info without adding ACPI-specific code to that function
anyway.

> However, we overcome that blocker by passing down parent device via
> pci_create_root_bus parameter (as the ACPI device type).

Which is completely wrong.

> Then we use ACPI_COMPANION_SET in core code
> for ACPI boot method only. ACPI_COMPANION_SET is safe to run for all
> cases DT, ACPI and DT&ACPI.
> 
> Since now PCI core code is setting ACPI companion device for us,
> x86 & ia64 specific ACPI companion device setting turns out to be dead now.
> We can get rid of it, including related companion reference from
> PCI sysdata structure. Aslo, PCI_CONTROLLER macro cannot return valid
> companion device anymore. Therefore we need to convert its usage to
> ACPI_COMPANION.
> 
> Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Tested-by: Duc Dang <dhdang@apm.com>
> Tested-by: Dongdong Liu <liudongdong3@huawei.com>
> Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
> Tested-by: Graeme Gregory <graeme.gregory@linaro.org>
> Tested-by: Sinan Kaya <okaya@codeaurora.org>
> ---
>  arch/ia64/hp/common/sba_iommu.c    |  2 +-
>  arch/ia64/include/asm/pci.h        |  1 -
>  arch/ia64/pci/pci.c                | 16 ----------------
>  arch/ia64/sn/kernel/io_acpi_init.c |  4 ++--
>  arch/x86/include/asm/pci.h         |  3 ---
>  arch/x86/pci/acpi.c                | 17 -----------------
>  drivers/acpi/pci_root.c            |  7 ++++++-
>  drivers/pci/probe.c                |  2 ++
>  8 files changed, 11 insertions(+), 41 deletions(-)
> 
> diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
> index a6d6190..78e4444 100644
> --- a/arch/ia64/hp/common/sba_iommu.c
> +++ b/arch/ia64/hp/common/sba_iommu.c
> @@ -1981,7 +1981,7 @@ sba_connect_bus(struct pci_bus *bus)
>  	if (PCI_CONTROLLER(bus)->iommu)
>  		return;
>  
> -	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
> +	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
>  	if (!handle)
>  		return;
>  
> diff --git a/arch/ia64/include/asm/pci.h b/arch/ia64/include/asm/pci.h
> index c0835b0..12423f4 100644
> --- a/arch/ia64/include/asm/pci.h
> +++ b/arch/ia64/include/asm/pci.h
> @@ -63,7 +63,6 @@ extern int pci_mmap_legacy_page_range(struct pci_bus *bus,
>  #define pci_legacy_write platform_pci_legacy_write
>  
>  struct pci_controller {
> -	struct acpi_device *companion;
>  	void *iommu;
>  	int segment;
>  	int node;		/* nearest node with memory or NUMA_NO_NODE for global allocation */
> diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
> index 8f6ac2f..978d6af 100644
> --- a/arch/ia64/pci/pci.c
> +++ b/arch/ia64/pci/pci.c
> @@ -301,28 +301,12 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  	}
>  
>  	info->controller.segment = root->segment;
> -	info->controller.companion = device;
>  	info->controller.node = acpi_get_node(device->handle);
>  	INIT_LIST_HEAD(&info->io_resources);
>  	return acpi_pci_root_create(root, &pci_acpi_root_ops,
>  				    &info->common, &info->controller);
>  }
>  
> -int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
> -{
> -	/*
> -	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
> -	 * here, pci_create_root_bus() has been called by someone else and
> -	 * sysdata is likely to be different from what we expect.  Let it go in
> -	 * that case.
> -	 */
> -	if (!bridge->dev.parent) {
> -		struct pci_controller *controller = bridge->bus->sysdata;
> -		ACPI_COMPANION_SET(&bridge->dev, controller->companion);
> -	}
> -	return 0;
> -}
> -
>  void pcibios_fixup_device_resources(struct pci_dev *dev)
>  {
>  	int idx;
> diff --git a/arch/ia64/sn/kernel/io_acpi_init.c b/arch/ia64/sn/kernel/io_acpi_init.c
> index 231234c..e454492 100644
> --- a/arch/ia64/sn/kernel/io_acpi_init.c
> +++ b/arch/ia64/sn/kernel/io_acpi_init.c
> @@ -132,7 +132,7 @@ sn_get_bussoft_ptr(struct pci_bus *bus)
>  	struct acpi_resource_vendor_typed *vendor;
>  
>  
> -	handle = acpi_device_handle(PCI_CONTROLLER(bus)->companion);
> +	handle = acpi_device_handle(ACPI_COMPANION(bus->bridge));
>  	status = acpi_get_vendor_resource(handle, METHOD_NAME__CRS,
>  					  &sn_uuid, &buffer);
>  	if (ACPI_FAILURE(status)) {
> @@ -360,7 +360,7 @@ sn_acpi_get_pcidev_info(struct pci_dev *dev, struct pcidev_info **pcidev_info,
>  	acpi_status status;
>  	struct acpi_buffer name_buffer = { ACPI_ALLOCATE_BUFFER, NULL };
>  
> -	rootbus_handle = acpi_device_handle(PCI_CONTROLLER(dev)->companion);
> +	rootbus_handle = acpi_device_handle(ACPI_COMPANION(dev->bus->bridge));
>          status = acpi_evaluate_integer(rootbus_handle, METHOD_NAME__SEG, NULL,
>                                         &segment);
>          if (ACPI_SUCCESS(status)) {
> diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h
> index 9ab7507..24de07d 100644
> --- a/arch/x86/include/asm/pci.h
> +++ b/arch/x86/include/asm/pci.h
> @@ -14,9 +14,6 @@
>  struct pci_sysdata {
>  	int		domain;		/* PCI domain */
>  	int		node;		/* NUMA node */
> -#ifdef CONFIG_ACPI
> -	struct acpi_device *companion;	/* ACPI companion device */
> -#endif
>  #ifdef CONFIG_X86_64
>  	void		*iommu;		/* IOMMU private data */
>  #endif
> diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
> index 3cd6983..f4ca17a 100644
> --- a/arch/x86/pci/acpi.c
> +++ b/arch/x86/pci/acpi.c
> @@ -340,7 +340,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  		struct pci_sysdata sd = {
>  			.domain = domain,
>  			.node = node,
> -			.companion = root->device
>  		};
>  
>  		memcpy(bus->sysdata, &sd, sizeof(sd));
> @@ -355,7 +354,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  		else {
>  			info->sd.domain = domain;
>  			info->sd.node = node;
> -			info->sd.companion = root->device;
>  			bus = acpi_pci_root_create(root, &acpi_pci_root_ops,
>  						   &info->common, &info->sd);
>  		}
> @@ -373,21 +371,6 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  	return bus;
>  }
>  
> -int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
> -{
> -	/*
> -	 * We pass NULL as parent to pci_create_root_bus(), so if it is not NULL
> -	 * here, pci_create_root_bus() has been called by someone else and
> -	 * sysdata is likely to be different from what we expect.  Let it go in
> -	 * that case.
> -	 */
> -	if (!bridge->dev.parent) {
> -		struct pci_sysdata *sd = bridge->bus->sysdata;
> -		ACPI_COMPANION_SET(&bridge->dev, sd->companion);
> -	}
> -	return 0;
> -}
> -
>  int __init pci_acpi_init(void)
>  {
>  	struct pci_dev *dev = NULL;
> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> index ae3fe4e..4581e0e 100644
> --- a/drivers/acpi/pci_root.c
> +++ b/drivers/acpi/pci_root.c
> @@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device,
>  		}
>  	}
>  
> +	/*
> +	 * pci_create_root_bus() needs to detect the parent device type,
> +	 * so initialize its companion data accordingly.
> +	 */
> +	ACPI_COMPANION_SET(&device->dev, device);

This is a complete nonsense, sorry.

This just means that the ACPI firmware object will now be an ACPI companion of
itself.

Thanks,
Rafael

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Bjorn Helgaas]

On Tue, May 10, 2016 at 12:56:37AM +0200, Rafael J. Wysocki wrote:
> On Friday, April 15, 2016 07:06:36 PM Tomasz Nowicki wrote:
> > Currently we have two platforms (x86 & ia64) capable of PCI ACPI host
> > bridge initialization. They both use arch-specific sysdata to pass down
> > parent device reference and both rely on NULL parent in pci_create_root_bus()
> > to validate sysdata content.
> 
> No, this is not relied on for that.  It is just NULL, because we don't
> have a physical parent device representation for the PCI host bridge.
> 
> > It looks hacky and prevents us from getting some firmware specific
> > info for PCI host controller based on its acpi_device structure
> > in generic pci_create_root_bus() function.
> 
> You can't get that info without adding ACPI-specific code to that function
> anyway.
> 
> > However, we overcome that blocker by passing down parent device via
> > pci_create_root_bus parameter (as the ACPI device type).
> 
> Which is completely wrong.
> 
> > Then we use ACPI_COMPANION_SET in core code
> > for ACPI boot method only. ACPI_COMPANION_SET is safe to run for all
> > cases DT, ACPI and DT&ACPI.
> > 
> > Since now PCI core code is setting ACPI companion device for us,
> > x86 & ia64 specific ACPI companion device setting turns out to be dead now.
> > We can get rid of it, including related companion reference from
> > PCI sysdata structure. Aslo, PCI_CONTROLLER macro cannot return valid
> > companion device anymore. Therefore we need to convert its usage to
> > ACPI_COMPANION.

I think getting rid of arch-specific sysdata is a good goal (at least
for things like the domain and ACPI companion that are really not
arch-specific).

But I'm afraid I've been guilty of allowing the perfect to be the
enemy of the good.  I think it will be more useful in the end if we
can get as much of this work in v4.7 as possible, even if there's more
polishing we'd like to do later.

What if we keep the x86 and ia64 handling of ACPI_COMPANION the way it
is today, and do it the same way for arm64?  Then later, after we
merge Arnd's work to make a generic pci_register_host() or similar, I
suspect we might be able to unify the ACPI_COMPANION stuff more
easily.  If the driver (acpi/pci_root.c) allocates the struct
pci_host_bridge, then maybe it will be able to fill in the domain and
set up the ACPI_COMPANION before it calls the scan interface.

Bjorn

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Lorenzo Pieralisi]

On Mon, May 09, 2016 at 08:53:36PM -0500, Bjorn Helgaas wrote:
> On Tue, May 10, 2016 at 12:56:37AM +0200, Rafael J. Wysocki wrote:
> > On Friday, April 15, 2016 07:06:36 PM Tomasz Nowicki wrote:
> > > Currently we have two platforms (x86 & ia64) capable of PCI ACPI host
> > > bridge initialization. They both use arch-specific sysdata to pass down
> > > parent device reference and both rely on NULL parent in pci_create_root_bus()
> > > to validate sysdata content.
> > 
> > No, this is not relied on for that.  It is just NULL, because we don't
> > have a physical parent device representation for the PCI host bridge.
> > 
> > > It looks hacky and prevents us from getting some firmware specific
> > > info for PCI host controller based on its acpi_device structure
> > > in generic pci_create_root_bus() function.
> > 
> > You can't get that info without adding ACPI-specific code to that function
> > anyway.
> > 
> > > However, we overcome that blocker by passing down parent device via
> > > pci_create_root_bus parameter (as the ACPI device type).
> > 
> > Which is completely wrong.
> > 
> > > Then we use ACPI_COMPANION_SET in core code
> > > for ACPI boot method only. ACPI_COMPANION_SET is safe to run for all
> > > cases DT, ACPI and DT&ACPI.
> > > 
> > > Since now PCI core code is setting ACPI companion device for us,
> > > x86 & ia64 specific ACPI companion device setting turns out to be dead now.
> > > We can get rid of it, including related companion reference from
> > > PCI sysdata structure. Aslo, PCI_CONTROLLER macro cannot return valid
> > > companion device anymore. Therefore we need to convert its usage to
> > > ACPI_COMPANION.
> 
> I think getting rid of arch-specific sysdata is a good goal (at least
> for things like the domain and ACPI companion that are really not
> arch-specific).
> 
> But I'm afraid I've been guilty of allowing the perfect to be the
> enemy of the good.  I think it will be more useful in the end if we
> can get as much of this work in v4.7 as possible, even if there's more
> polishing we'd like to do later.
> 
> What if we keep the x86 and ia64 handling of ACPI_COMPANION the way it
> is today, and do it the same way for arm64?  Then later, after we
> merge Arnd's work to make a generic pci_register_host() or similar, I
> suspect we might be able to unify the ACPI_COMPANION stuff more
> easily.  If the driver (acpi/pci_root.c) allocates the struct
> pci_host_bridge, then maybe it will be able to fill in the domain and
> set up the ACPI_COMPANION before it calls the scan interface.

Ok, JC already solved that problem like this:

http://www.spinics.net/lists/arm-kernel/msg478167.html
http://www.spinics.net/lists/arm-kernel/msg478169.html

Since ARM64 relies on PCI_DOMAINS_GENERIC (DT) I do not see any other
way of pulling it off quickly (and in a really generic way), sysdata
is the only way of handling it in ACPI unless we start clutching
at straws (which we already did with the ACPI companion hacks).

Thanks,
Lorenzo

Re: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.

[Lorenzo Pieralisi]

On Tue, May 10, 2016 at 12:18:59AM +0200, Rafael J. Wysocki wrote:

[...]

> > >> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> > >> index ae3fe4e..4581e0e 100644
> > >> --- a/drivers/acpi/pci_root.c
> > >> +++ b/drivers/acpi/pci_root.c
> > >> @@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device,
> > >>   		}
> > >>   	}
> > >>
> > >> +	/*
> > >> +	 * pci_create_root_bus() needs to detect the parent device type,
> > >> +	 * so initialize its companion data accordingly.
> > >> +	 */
> > >> +	ACPI_COMPANION_SET(&device->dev, device);
> > >>   	root->device = device;
> > >>   	root->segment = segment & 0xFFFF;
> > >>   	strcpy(acpi_device_name(device), ACPI_PCI_ROOT_DEVICE_NAME);
> > >> @@ -846,7 +851,7 @@ struct pci_bus *acpi_pci_root_create(struct acpi_pci_root *root,
> > >>
> > >>   	pci_acpi_root_add_resources(info);
> > >>   	pci_add_resource(&info->resources, &root->secondary);
> > >> -	bus = pci_create_root_bus(NULL, busnum, ops->pci_ops,
> > >> +	bus = pci_create_root_bus(&device->dev, busnum, ops->pci_ops,
> > >>   				  sysdata, &info->resources);
> > >
> > > "device" here is a struct acpi_device *.  Rafael, is that the right
> > > thing to do?  I dimly recall proposing something similar long ago and
> > > that it turned out to be a bad idea.
> > >
> > 
> > Joining Bjorn's question. Rafael, do you recall what was the issue here 
> > from the past?
> 
> Yes, I do.
> 
> Anything struct acpi_device doesn't represent a real device.  It
> represents a firmware object that can be associated with a device.
> IOW, nothing under LNXSYSTM\:00/ should ever be used as the parent or
> sibling etc with respect to anything outside of that directory.  In
> fact, the entire LNXSYSTM\:00/ should be located under
> /sys/firmware/acpi/ and it was a mistake to put it under
> /sys/devices/.
> 
> One immediate consequence of the above change, AFAICS, would be that
> the whole PCI hierarchy would now hang under
> /sys/devices/LNXSYSTM\:00/LNXSYBUS\:00/PNP0A08\:00/ which would not
> make any sense whatever, because
> /sys/devices/LNXSYSTM\:00/LNXSYBUS\:00/PNP0A08\:00/physical_node
> already points to /sys/devices/pci0000\:00/.
> 
> IOW, both PNP0A08\:00/ and pci0000\:00/ already represent the same
> thing, ie. the host bridge.
> 
> If you want to have a parent for pci0000\:00/, you need a
> physical_node for LNXSYBUS\:00 and point to that as the parent from
> pci0000\:00/.  That at least will lead to a sysfs hierarchy that makes
> sense, although it may break user space tools I suppose (which may be
> assuming that pci0000\:00/ will always be present directly under
> /sys/devices/).

Ok, I have a question though. As an example, DT based host controllers
(that pass the parent pointer to pci_create_root_bus()), are already
rooted at the respective host controller platform device sysfs path, so
if user space can't cope with that, that is a problem *now* on some
systems unless I am missing something.

Anyway, thanks for clarifying the companion handling mechanism, we
decidedly have to find a proper way to handle it instead of working
around it.

Lorenzo