[PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Tomasz Nowicki]

>From the functionality point of view this series may be split into the
following logic parts:
1. Export ECAM API and add parent device to pci_config_window
2. Add IO resources handling to PCI core code
3. Support for generic domain assignment based on ACPI
4. New MCFG driver
5. Implement ARM64 ACPI based PCI host controller driver under arch/arm64/

Patches has been built on top of 4.7-rc1 and can be found here:
git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v8)

This has been tested on Cavium ThunderX server. Any help in reviewing and
testing is very appreciated.

v7 -> v8
- move code from drivers/acpi/pci_root_generic.c to arch/arm64/kernel/pci.c
- minor changes around domain assignment
- pci_mcfg.c improvements for parsing MCFG tables and lookup its entries

v6 -> v7
- drop quirks handling
- changes for ACPI companion and domain number assignment approach
- implement arch pcibios_{add|remove}_bus and call acpi_pci_{add|remove}_bus from there
- cleanups around nomenclature
- use resources oriented API for ECAM
- fix for based address calculation before mapping ECAM region
- remove useless lock for MCFG lookup
- move MCFG stuff to separated file pci_mcfg.c
- drop MCFG entries caching
- rebase against 4.6-rc7

v5 -> v6
- drop idea of x86 MMCONFIG code refactoring
- integrate JC's patches which introduce new ECAM API:
  https://lkml.org/lkml/2016/4/11/907
  git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
- integrate Sinan's fix for releasing IO resources, see patch [06/13]
- added ACPI support for ThunderX ECAM and PEM drivers
- rebase against 4.6-rc2

v4 -> v5
- drop MCFG refactoring group patches 1-6 from series v4 and integrate Jayachandran's patch
  https://patchwork.ozlabs.org/patch/575525/
- rewrite PCI legacy IRQs allocation
- squash two patches 11 and 12 from series v4, fixed bisection issue
- changelog improvements
- rebase against 4.5-rc3

v3 -> v4
- drop Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
- add Lorenzo's fix patch 19/24
- ACPI PCI bus domain number assigning cleanup
- change resource management, we now claim and reassign resources
- improvements for applying quirks
- drop Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
- rebase against 4.5-rc1

v2 -> v3
- fix legacy IRQ assigning and IO ports registration
- remove reference to arch specific companion device for ia64
- move ACPI PCI host controller driver to pci_root.c
- drop generic domain assignment for x86 and ia64 as I am not
  able to run all necessary test variants
- drop patch which cleaned legacy IRQ assignment since it belongs to
  Mathew's series:
  https://patchwork.ozlabs.org/patch/557504/
- extend MCFG quirk code
- rebase against 4.4

v1 -> v2
- move non-arch specific piece of code to dirver/acpi/ directory
- fix IO resource handling
- introduce PCI config accessors quirks matching
- moved ACPI_COMPANION_SET to generic code

v1 - https://lkml.org/lkml/2015/10/27/504
v2 - https://lkml.org/lkml/2015/12/16/246
v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
v4 - https://lkml.org/lkml/2016/2/4/646
v5 - https://lkml.org/lkml/2016/2/16/426
v6 - https://lkml.org/lkml/2016/4/15/594

Jayachandran C (2):
  PCI: ecam: move ecam.h to linux/include/pci-ecam.h
  PCI: ecam: Add parent device field to pci_config_window

Tomasz Nowicki (7):
  pci: Add new function to unmap IO resources.
  acpi, pci: Support IO resources when parsing PCI host bridge
    resources.
  pci, acpi: add acpi hook to assign domain number.
  arm64, pci, acpi: ACPI support for legacy IRQs parsing and
    consolidation with DT code.
  acpi: Add generic MCFG table handling
  arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus
    enumeration.
  pci, acpi: ARM64 support for ACPI based generic PCI host controller

 arch/arm64/Kconfig                  |   2 +
 arch/arm64/kernel/pci.c             | 143 ++++++++++++++++++++++++++++++++++--
 drivers/acpi/Kconfig                |   3 +
 drivers/acpi/Makefile               |   1 +
 drivers/acpi/pci_mcfg.c             |  94 ++++++++++++++++++++++++
 drivers/acpi/pci_root.c             |  39 ++++++++++
 drivers/pci/ecam.c                  |   6 +-
 drivers/pci/ecam.h                  |  67 -----------------
 drivers/pci/host/pci-host-common.c  |   3 +-
 drivers/pci/host/pci-host-generic.c |   3 +-
 drivers/pci/host/pci-thunder-ecam.c |   3 +-
 drivers/pci/host/pci-thunder-pem.c  |   6 +-
 drivers/pci/pci.c                   |  29 +++++++-
 include/linux/pci-acpi.h            |   2 +
 include/linux/pci-ecam.h            |  67 +++++++++++++++++
 include/linux/pci.h                 |   9 ++-
 16 files changed, 387 insertions(+), 90 deletions(-)
 create mode 100644 drivers/acpi/pci_mcfg.c
 delete mode 100644 drivers/pci/ecam.h
 create mode 100644 include/linux/pci-ecam.h

-- 
1.9.1

[PATCH V8 1/9] PCI: ecam: move ecam.h to linux/include/pci-ecam.h

[Tomasz Nowicki]

From: Jayachandran C <jchandra@broadcom.com>

This header will be used from arch/arm64 for ACPI PCI implementation
so it needs to be moved out of drivers/pci.

Update users of the header file to use the new name. No functional
changes.

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 drivers/pci/ecam.c                  |  3 +-
 drivers/pci/ecam.h                  | 67 -------------------------------------
 drivers/pci/host/pci-host-common.c  |  3 +-
 drivers/pci/host/pci-host-generic.c |  3 +-
 drivers/pci/host/pci-thunder-ecam.c |  3 +-
 drivers/pci/host/pci-thunder-pem.c  |  3 +-
 include/linux/pci-ecam.h            | 67 +++++++++++++++++++++++++++++++++++++
 7 files changed, 72 insertions(+), 77 deletions(-)
 delete mode 100644 drivers/pci/ecam.h
 create mode 100644 include/linux/pci-ecam.h

diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index f9832ad..820e26b 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -19,10 +19,9 @@
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/slab.h>
 
-#include "ecam.h"
-
 /*
  * On 64-bit systems, we do a single ioremap for the whole config space
  * since we have enough virtual address range available.  On 32-bit, we
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
deleted file mode 100644
index 9878beb..0000000
--- a/drivers/pci/ecam.h
+++ /dev/null
@@ -1,67 +0,0 @@
-/*
- * Copyright 2016 Broadcom
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License, version 2, as
- * published by the Free Software Foundation (the "GPL").
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * General Public License version 2 (GPLv2) for more details.
- *
- * You should have received a copy of the GNU General Public License
- * version 2 (GPLv2) along with this source code.
- */
-#ifndef DRIVERS_PCI_ECAM_H
-#define DRIVERS_PCI_ECAM_H
-
-#include <linux/kernel.h>
-#include <linux/platform_device.h>
-
-/*
- * struct to hold pci ops and bus shift of the config window
- * for a PCI controller.
- */
-struct pci_config_window;
-struct pci_ecam_ops {
-	unsigned int			bus_shift;
-	struct pci_ops			pci_ops;
-	int				(*init)(struct device *,
-						struct pci_config_window *);
-};
-
-/*
- * struct to hold the mappings of a config space window. This
- * is expected to be used as sysdata for PCI controllers that
- * use ECAM.
- */
-struct pci_config_window {
-	struct resource			res;
-	struct resource			busr;
-	void				*priv;
-	struct pci_ecam_ops		*ops;
-	union {
-		void __iomem		*win;	/* 64-bit single mapping */
-		void __iomem		**winp; /* 32-bit per-bus mapping */
-	};
-};
-
-/* create and free pci_config_window */
-struct pci_config_window *pci_ecam_create(struct device *dev,
-		struct resource *cfgres, struct resource *busr,
-		struct pci_ecam_ops *ops);
-void pci_ecam_free(struct pci_config_window *cfg);
-
-/* map_bus when ->sysdata is an instance of pci_config_window */
-void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
-			       int where);
-/* default ECAM ops */
-extern struct pci_ecam_ops pci_generic_ecam_ops;
-
-#ifdef CONFIG_PCI_HOST_GENERIC
-/* for DT-based PCI controllers that support ECAM */
-int pci_host_common_probe(struct platform_device *pdev,
-			  struct pci_ecam_ops *ops);
-#endif
-#endif
diff --git a/drivers/pci/host/pci-host-common.c b/drivers/pci/host/pci-host-common.c
index 8cba7ab..c18b9e3 100644
--- a/drivers/pci/host/pci-host-common.c
+++ b/drivers/pci/host/pci-host-common.c
@@ -20,10 +20,9 @@
 #include <linux/module.h>
 #include <linux/of_address.h>
 #include <linux/of_pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 
-#include "../ecam.h"
-
 static int gen_pci_parse_request_of_pci_ranges(struct device *dev,
 		       struct list_head *resources, struct resource **bus_range)
 {
diff --git a/drivers/pci/host/pci-host-generic.c b/drivers/pci/host/pci-host-generic.c
index 6eaceab..f0ca6de 100644
--- a/drivers/pci/host/pci-host-generic.c
+++ b/drivers/pci/host/pci-host-generic.c
@@ -23,10 +23,9 @@
 #include <linux/module.h>
 #include <linux/of_address.h>
 #include <linux/of_pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 
-#include "../ecam.h"
-
 static struct pci_ecam_ops gen_pci_cfg_cam_bus_ops = {
 	.bus_shift	= 16,
 	.pci_ops	= {
diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
index 540d030..a9fc1c9 100644
--- a/drivers/pci/host/pci-thunder-ecam.c
+++ b/drivers/pci/host/pci-thunder-ecam.c
@@ -11,10 +11,9 @@
 #include <linux/ioport.h>
 #include <linux/of_pci.h>
 #include <linux/of.h>
+#include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 
-#include "../ecam.h"
-
 static void set_val(u32 v, int where, int size, u32 *val)
 {
 	int shift = (where & 3) * 8;
diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index 9b8ab94..5020d3d 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -18,10 +18,9 @@
 #include <linux/module.h>
 #include <linux/of_address.h>
 #include <linux/of_pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 
-#include "../ecam.h"
-
 #define PEM_CFG_WR 0x28
 #define PEM_CFG_RD 0x30
 
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
new file mode 100644
index 0000000..9878beb
--- /dev/null
+++ b/include/linux/pci-ecam.h
@@ -0,0 +1,67 @@
+/*
+ * Copyright 2016 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#ifndef DRIVERS_PCI_ECAM_H
+#define DRIVERS_PCI_ECAM_H
+
+#include <linux/kernel.h>
+#include <linux/platform_device.h>
+
+/*
+ * struct to hold pci ops and bus shift of the config window
+ * for a PCI controller.
+ */
+struct pci_config_window;
+struct pci_ecam_ops {
+	unsigned int			bus_shift;
+	struct pci_ops			pci_ops;
+	int				(*init)(struct device *,
+						struct pci_config_window *);
+};
+
+/*
+ * struct to hold the mappings of a config space window. This
+ * is expected to be used as sysdata for PCI controllers that
+ * use ECAM.
+ */
+struct pci_config_window {
+	struct resource			res;
+	struct resource			busr;
+	void				*priv;
+	struct pci_ecam_ops		*ops;
+	union {
+		void __iomem		*win;	/* 64-bit single mapping */
+		void __iomem		**winp; /* 32-bit per-bus mapping */
+	};
+};
+
+/* create and free pci_config_window */
+struct pci_config_window *pci_ecam_create(struct device *dev,
+		struct resource *cfgres, struct resource *busr,
+		struct pci_ecam_ops *ops);
+void pci_ecam_free(struct pci_config_window *cfg);
+
+/* map_bus when ->sysdata is an instance of pci_config_window */
+void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
+			       int where);
+/* default ECAM ops */
+extern struct pci_ecam_ops pci_generic_ecam_ops;
+
+#ifdef CONFIG_PCI_HOST_GENERIC
+/* for DT-based PCI controllers that support ECAM */
+int pci_host_common_probe(struct platform_device *pdev,
+			  struct pci_ecam_ops *ops);
+#endif
+#endif
-- 
1.9.1

[PATCH V8 2/9] PCI: ecam: Add parent device field to pci_config_window

[Tomasz Nowicki]

From: Jayachandran C <jchandra@broadcom.com>

Add a parent device field to struct pci_config_window. The parent
is not saved now, but will be useful to save it in some cases.
Specifically in case of ACPI for ARM64, it can be used to setup
ACPI companion and domain.

Since the parent dev is in struct pci_config_window now, we need
not pass it to he init function as a separate argument.

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 drivers/pci/ecam.c                 | 3 ++-
 drivers/pci/host/pci-thunder-pem.c | 3 ++-
 include/linux/pci-ecam.h           | 4 ++--
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 820e26b..66e0d71 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -51,6 +51,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
 	if (!cfg)
 		return ERR_PTR(-ENOMEM);
 
+	cfg->parent = dev;
 	cfg->ops = ops;
 	cfg->busr.start = busr->start;
 	cfg->busr.end = busr->end;
@@ -94,7 +95,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
 	}
 
 	if (ops->init) {
-		err = ops->init(dev, cfg);
+		err = ops->init(cfg);
 		if (err)
 			goto err_exit;
 	}
diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index 5020d3d..91f6fc6 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -284,8 +284,9 @@ static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn,
 	return pci_generic_config_write(bus, devfn, where, size, val);
 }
 
-static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
+static int thunder_pem_init(struct pci_config_window *cfg)
 {
+	struct device *dev = cfg->parent;
 	resource_size_t bar4_start;
 	struct resource *res_pem;
 	struct thunder_pem_pci *pem_pci;
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 9878beb..7adad20 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -27,8 +27,7 @@ struct pci_config_window;
 struct pci_ecam_ops {
 	unsigned int			bus_shift;
 	struct pci_ops			pci_ops;
-	int				(*init)(struct device *,
-						struct pci_config_window *);
+	int				(*init)(struct pci_config_window *);
 };
 
 /*
@@ -45,6 +44,7 @@ struct pci_config_window {
 		void __iomem		*win;	/* 64-bit single mapping */
 		void __iomem		**winp; /* 32-bit per-bus mapping */
 	};
+	struct device			*parent;/* ECAM res was from this dev */
 };
 
 /* create and free pci_config_window */
-- 
1.9.1

[PATCH V8 3/9] pci: Add new function to unmap IO resources.

[Tomasz Nowicki]

We need to release I/O resources so that the same I/O resources
can be allocated again (pci_remap_iospace), like in PCI hotplug removal
scenario. Therefore this patch implements new pci_unmap_iospace call which
unmaps I/O space as the symmetry to pci_remap_iospace.

Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
---
 drivers/pci/pci.c   | 18 ++++++++++++++++++
 include/linux/pci.h |  1 +
 2 files changed, 19 insertions(+)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index c8b4dbd..eb431b5 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -25,6 +25,7 @@
 #include <linux/device.h>
 #include <linux/pm_runtime.h>
 #include <linux/pci_hotplug.h>
+#include <linux/vmalloc.h>
 #include <asm/setup.h>
 #include <linux/aer.h>
 #include "pci.h"
@@ -3165,6 +3166,23 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
 #endif
 }
 
+/**
+ *	pci_unmap_iospace - Unmap the memory mapped I/O space
+ *	@res: resource to be unmapped
+ *
+ *	Unmap the CPU virtual address @res from virtual address space.
+ *	Only architectures that have memory mapped IO functions defined
+ *	(and the PCI_IOBASE value defined) should call this function.
+ */
+void pci_unmap_iospace(struct resource *res)
+{
+#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
+	unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
+
+	unmap_kernel_range(vaddr, resource_size(res));
+#endif
+}
+
 static void __pci_set_master(struct pci_dev *dev, bool enable)
 {
 	u16 old_cmd, cmd;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index b67e4df..12349de 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1167,6 +1167,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size);
 unsigned long pci_address_to_pio(phys_addr_t addr);
 phys_addr_t pci_pio_to_address(unsigned long pio);
 int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
+void pci_unmap_iospace(struct resource *res);
 
 static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
 {
-- 
1.9.1

[PATCH V8 4/9] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Tomasz Nowicki]

Platforms that have memory mapped IO port (such as ARM64) need special
handling for PCI I/O resources. For host bridge's resource probing case
these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.

The same I/O resources need to be released after hotplug
removal so that it can be re-added back by the pci_remap_iospace
function during insertion. As a consequence we unmap I/O resources
with pci_unmap_iospace when we release host bridge resources.

Signed-off-by: Jayachandran C <jchandra@broadcom.com>
Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
---
 drivers/acpi/pci_root.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index ae3fe4e..ee3b728 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -720,6 +720,40 @@ next:
 	}
 }
 
+#ifdef PCI_IOBASE
+static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
+{
+	struct resource *res = entry->res;
+	resource_size_t cpu_addr = res->start;
+	resource_size_t pci_addr = cpu_addr - entry->offset;
+	resource_size_t length = resource_size(res);
+	unsigned long port;
+
+	if (pci_register_io_range(cpu_addr, length))
+		goto err;
+
+	port = pci_address_to_pio(cpu_addr);
+	if (port == (unsigned long)-1)
+		goto err;
+
+	res->start = port;
+	res->end = port + length - 1;
+	entry->offset = port - pci_addr;
+
+	if (pci_remap_iospace(res, cpu_addr) < 0)
+		goto err;
+
+	pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
+	return;
+err:
+	res->flags |= IORESOURCE_DISABLED;
+}
+#else
+static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
+{
+}
+#endif
+
 int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
 {
 	int ret;
@@ -740,6 +774,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
 			"no IO and memory resources present in _CRS\n");
 	else {
 		resource_list_for_each_entry_safe(entry, tmp, list) {
+			if (entry->res->flags & IORESOURCE_IO)
+				acpi_pci_root_remap_iospace(entry);
+
 			if (entry->res->flags & IORESOURCE_DISABLED)
 				resource_list_destroy_entry(entry);
 			else
@@ -811,6 +848,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
 
 	resource_list_for_each_entry(entry, &bridge->windows) {
 		res = entry->res;
+		if (res->flags & IORESOURCE_IO)
+			pci_unmap_iospace(res);
 		if (res->parent &&
 		    (res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
 			release_resource(res);
-- 
1.9.1

[PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Tomasz Nowicki]

PCI core code provides a config option (CONFIG_PCI_DOMAINS_GENERIC)
that allows assigning the PCI bus domain number generically by
relying on device tree bindings, and falling back to a simple counter
when the respective DT properties (ie "linux,pci-domain") are not
specified in the host bridge device tree node.

In a similar way, when a system is booted through ACPI, architectures
that are selecting CONFIG_PCI_DOMAINS_GENERIC (ie ARM64) require kernel
hooks to retrieve the domain number so that the PCI bus domain number
set-up can be handled seamlessly with DT and ACPI in generic core code
when CONFIG_PCI_DOMAINS_GENERIC is selected.

Since currently it is not possible to retrieve a pointer to the PCI
host bridge ACPI device backing the host bridge from core PCI code
(which would allow retrieving the domain number in an arch agnostic
way through the ACPI _SEG method), an arch specific ACPI hook has to
be declared and implemented by all arches that rely on
CONFIG_PCI_DOMAINS_GENERIC to retrieve the domain number and set it
up in core PCI code.

For the aforementioned reasons, this patch introduces a dummy
acpi_pci_bus_domain_nr() hook in preparation for per-arch implementation
of the same to retrieve the domain number on a per-arch basis when
the system boots through ACPI.

For the sake of code clarity the current code implementing generic
domain number assignment (ie pci_bus_assign_domain_nr(), selected by
CONFIG_PCI_DOMAINS_GENERIC) is reshuffled so that the code implementing
the DT domain assignment function is stubbed out into a corresponding
helper, so that DT and ACPI functions are clearly separated in
preparation for arches acpi_pci_bus_domain_nr() implementations.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
---
 drivers/pci/pci.c   | 11 +++++++++--
 include/linux/pci.h |  1 +
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index eb431b5..2b52178 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -7,6 +7,7 @@
  *	Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
  */
 
+#include <linux/acpi.h>
 #include <linux/kernel.h>
 #include <linux/delay.h>
 #include <linux/init.h>
@@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
 }
 
 #ifdef CONFIG_PCI_DOMAINS_GENERIC
-void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
+static int of_pci_bus_domain_nr(struct device *parent)
 {
 	static int use_dt_domains = -1;
 	int domain = -1;
@@ -4985,7 +4986,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
 		domain = -1;
 	}
 
-	bus->domain_nr = domain;
+	return domain;
+}
+
+void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
+{
+	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
+					 acpi_pci_bus_domain_nr(bus);
 }
 #endif
 #endif
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 12349de..bba4053 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1390,6 +1390,7 @@ static inline int pci_domain_nr(struct pci_bus *bus)
 {
 	return bus->domain_nr;
 }
+static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return -1; }
 void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent);
 #else
 static inline void pci_bus_assign_domain_nr(struct pci_bus *bus,
-- 
1.9.1

[PATCH V8 6/9] arm64, pci, acpi: ACPI support for legacy IRQs parsing and consolidation with DT code.

[Tomasz Nowicki]

To enable PCI legacy IRQs on platforms booting with ACPI, arch code
should include ACPI specific callbacks that parse and set-up the
device IRQ number, equivalent to the DT boot path. Owing to the current
ACPI core scan handlers implementation, ACPI PCI legacy IRQs bindings
cannot be parsed at device add time, since that would trigger ACPI scan
handlers ordering issues depending on how the ACPI tables are defined.

To solve this problem and consolidate FW PCI legacy IRQs parsing in
one single pcibios callback (pending final removal), this patch moves
DT PCI IRQ parsing to the pcibios_alloc_irq() callback (called by
PCI core code at device probe time) and adds ACPI PCI legacy IRQs
parsing to the same callback too, so that FW PCI legacy IRQs parsing
is confined in one single arch callback that can be easily removed
when code parsing PCI legacy IRQs is consolidated and moved to core
PCI code.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Suggested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
---
 arch/arm64/kernel/pci.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 3c4e308..336deef 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -50,11 +50,16 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
 }
 
 /*
- * Try to assign the IRQ number from DT when adding a new device
+ * Try to assign the IRQ number when probing a new device
  */
-int pcibios_add_device(struct pci_dev *dev)
+int pcibios_alloc_irq(struct pci_dev *dev)
 {
-	dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
+	if (acpi_disabled)
+		dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
+#ifdef CONFIG_ACPI
+	else
+		return acpi_pci_irq_enable(dev);
+#endif
 
 	return 0;
 }
-- 
1.9.1

[PATCH V8 7/9] acpi: Add generic MCFG table handling

[Tomasz Nowicki]

In order to handle PCI config space regions properly in ACPI, new MCFG
interface is defined which does sanity checks on MCFG table and keeps its
root pointer. The user is able to lookup MCFG regions based on
host bridge root structure and domain:bus_start:bus_end touple.
Use pci_mmcfg_late_init old prototype to avoid another function name.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 drivers/acpi/Kconfig     |  3 ++
 drivers/acpi/Makefile    |  1 +
 drivers/acpi/pci_mcfg.c  | 94 ++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/pci-acpi.h |  2 ++
 include/linux/pci.h      |  2 +-
 5 files changed, 101 insertions(+), 1 deletion(-)
 create mode 100644 drivers/acpi/pci_mcfg.c

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index b7e2e77..f98c328 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -217,6 +217,9 @@ config ACPI_PROCESSOR_IDLE
 	bool
 	select CPU_IDLE
 
+config ACPI_MCFG
+	bool
+
 config ACPI_CPPC_LIB
 	bool
 	depends on ACPI_PROCESSOR
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index 251ce85..632e81f 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
 acpi-y				+= ec.o
 acpi-$(CONFIG_ACPI_DOCK)	+= dock.o
 acpi-y				+= pci_root.o pci_link.o pci_irq.o
+obj-$(CONFIG_ACPI_MCFG)		+= pci_mcfg.o
 acpi-y				+= acpi_lpss.o acpi_apd.o
 acpi-y				+= acpi_platform.o
 acpi-y				+= acpi_pnp.o
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
new file mode 100644
index 0000000..1847f74
--- /dev/null
+++ b/drivers/acpi/pci_mcfg.c
@@ -0,0 +1,94 @@
+/*
+ * Copyright (C) 2016 Broadcom
+ *	Author: Jayachandran C <jchandra@broadcom.com>
+ * Copyright (C) 2016 Semihalf
+ * 	Author: Tomasz Nowicki <tn@semihalf.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+
+#define pr_fmt(fmt) "ACPI: " fmt
+
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+
+/* Root pointer to the mapped MCFG table */
+static struct acpi_table_mcfg *mcfg_table;
+static int mcfg_entries;
+
+int pci_mcfg_lookup(struct acpi_pci_root *root)
+{
+	struct acpi_mcfg_allocation *mptr, *entry = NULL;
+	struct resource *bus_res = &root->secondary;
+	int i;
+
+	if (mcfg_table) {
+		mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
+		for (i = 0; i < mcfg_entries && !entry; i++, mptr++)
+			if (mptr->pci_segment == root->segment &&
+			    mptr->start_bus_number == bus_res->start)
+				entry = mptr;
+	}
+
+	/* not found, use _CBA if available, else error */
+	if (!entry) {
+		if (root->mcfg_addr)
+			return root->mcfg_addr;
+		pr_err("%04x:%pR MCFG lookup failed\n", root->segment, bus_res);
+		return -ENOENT;
+	} else if (root->mcfg_addr && entry->address != root->mcfg_addr) {
+		pr_warn("%04x:%pR CBA %pa != MCFG %lx, using CBA\n",
+			root->segment, bus_res, &root->mcfg_addr,
+			(unsigned long)entry->address);
+		return 0;
+	}
+
+	/* found matching entry, bus range check */
+	if (entry->end_bus_number != bus_res->end) {
+		resource_size_t bus_end = min_t(resource_size_t,
+					entry->end_bus_number, bus_res->end);
+		pr_warn("%04x:%pR bus end mismatch, using %02lx\n",
+			root->segment, bus_res, (unsigned long)bus_end);
+		bus_res->end = bus_end;
+	}
+
+	if (!root->mcfg_addr)
+		root->mcfg_addr = entry->address;
+	return 0;
+}
+
+static __init int pci_mcfg_parse(struct acpi_table_header *header)
+{
+	if (header->length < sizeof(struct acpi_table_mcfg))
+		return -EINVAL;
+
+	mcfg_entries = (header->length - sizeof(struct acpi_table_mcfg)) /
+					sizeof(struct acpi_mcfg_allocation);
+	if (mcfg_entries == 0) {
+		pr_err("MCFG has no entries\n");
+		return -EINVAL;
+	}
+
+	mcfg_table = (struct acpi_table_mcfg *)header;
+	pr_info("MCFG table detected, %d entries\n", mcfg_entries);
+	return 0;
+}
+
+/* Interface called by ACPI - parse and save MCFG table */
+void __init pci_mmcfg_late_init(void)
+{
+	int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
+	if (err)
+		pr_err("Failed to parse MCFG (%d)\n", err);
+}
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 89ab057..e0e6dfc 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -24,6 +24,8 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
 }
 extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
 
+extern int pci_mcfg_lookup(struct acpi_pci_root *root);
+
 static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
 {
 	struct pci_bus *pbus = pdev->bus;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index bba4053..9661c85 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1724,7 +1724,7 @@ void pcibios_free_irq(struct pci_dev *dev);
 extern struct dev_pm_ops pcibios_pm_ops;
 #endif
 
-#ifdef CONFIG_PCI_MMCONFIG
+#if defined(CONFIG_PCI_MMCONFIG) || defined(CONFIG_ACPI_MCFG)
 void __init pci_mmcfg_early_init(void);
 void __init pci_mmcfg_late_init(void);
 #else
-- 
1.9.1

[PATCH V8 8/9] arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus enumeration.

[Tomasz Nowicki]

ACPI requires to run acpi_pci_{add|remove}_bus while new PCI bus is created.
This allows to do some ACPI-specific additional configuration, like
PCI hotplug slot enumeration. In order to fulfill these requirements,
we implement arch-specific pcibios_{add|remove}_bus calls
and call acpi_pci_{add|remove}_bus from there.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
---
 arch/arm64/kernel/pci.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 336deef..3663be1 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -17,6 +17,7 @@
 #include <linux/mm.h>
 #include <linux/of_pci.h>
 #include <linux/of_platform.h>
+#include <linux/pci-acpi.h>
 #include <linux/slab.h>
 
 /*
@@ -96,4 +97,15 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 	/* TODO: Should be revisited when implementing PCI on ACPI */
 	return NULL;
 }
+
+void pcibios_add_bus(struct pci_bus *bus)
+{
+	acpi_pci_add_bus(bus);
+}
+
+void pcibios_remove_bus(struct pci_bus *bus)
+{
+	acpi_pci_remove_bus(bus);
+}
+
 #endif
-- 
1.9.1

[PATCH V8 9/9] pci, acpi: ARM64 support for ACPI based generic PCI host controller

[Tomasz Nowicki]

This patch implements pci_acpi_scan_root call so that ARM64 can start
using ACPI to setup and enumerate PCI buses.

The implementation of pci_acpi_scan_root() looks up config space regions
through MCFG interface. Then ECAM library is doing a new mapping
and attach generic ECAM ops which are used for accessing config space.

On ARM64, ACPI and DT can be enabled together, and in that case
we need to use generic domains. In order to do that we implement
ARM64 specific way of retrieving domain number from pci_config_window
structure.

Since we enable PCI for ACPI we need to implement raw_pci_{read|write}
at the same time. ARM64 provides RAW accessors as long as there is
correlated valid pci_bus structure, but not before.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Jayachandran C <jchandra@broadcom.com>
---
 arch/arm64/Kconfig      |   2 +
 arch/arm64/kernel/pci.c | 120 ++++++++++++++++++++++++++++++++++++++++++++++--
 include/linux/pci.h     |   5 ++
 3 files changed, 122 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 76747d9..87c48ad 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -3,6 +3,7 @@ config ARM64
 	select ACPI_CCA_REQUIRED if ACPI
 	select ACPI_GENERIC_GSI if ACPI
 	select ACPI_REDUCED_HARDWARE_ONLY if ACPI
+	select ACPI_MCFG if ACPI
 	select ARCH_HAS_DEVMEM_IS_ALLOWED
 	select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
 	select ARCH_HAS_ELF_RANDOMIZE
@@ -96,6 +97,7 @@ config ARM64
 	select OF_EARLY_FLATTREE
 	select OF_NUMA if NUMA && OF
 	select OF_RESERVED_MEM
+	select PCI_ECAM if ACPI
 	select PERF_USE_VMALLOC
 	select POWER_RESET
 	select POWER_SUPPLY
diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 3663be1..39f2a40 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -17,7 +17,9 @@
 #include <linux/mm.h>
 #include <linux/of_pci.h>
 #include <linux/of_platform.h>
+#include <linux/pci.h>
 #include <linux/pci-acpi.h>
+#include <linux/pci-ecam.h>
 #include <linux/slab.h>
 
 /*
@@ -71,13 +73,21 @@ int pcibios_alloc_irq(struct pci_dev *dev)
 int raw_pci_read(unsigned int domain, unsigned int bus,
 		  unsigned int devfn, int reg, int len, u32 *val)
 {
-	return -ENXIO;
+	struct pci_bus *b = pci_find_bus(domain, bus);
+
+	if (!b)
+		return PCIBIOS_DEVICE_NOT_FOUND;
+	return b->ops->read(b, devfn, reg, len, val);
 }
 
 int raw_pci_write(unsigned int domain, unsigned int bus,
 		unsigned int devfn, int reg, int len, u32 val)
 {
-	return -ENXIO;
+	struct pci_bus *b = pci_find_bus(domain, bus);
+
+	if (!b)
+		return PCIBIOS_DEVICE_NOT_FOUND;
+	return b->ops->write(b, devfn, reg, len, val);
 }
 
 #ifdef CONFIG_NUMA
@@ -91,11 +101,111 @@ EXPORT_SYMBOL(pcibus_to_node);
 #endif
 
 #ifdef CONFIG_ACPI
-/* Root bridge scanning */
+
+struct acpi_pci_generic_root_info {
+	struct acpi_pci_root_info	common;
+	struct pci_config_window	*cfg;	/* config space mapping */
+};
+
+int acpi_pci_bus_domain_nr(struct pci_bus *bus)
+{
+	struct pci_config_window *cfg = bus->sysdata;
+	struct acpi_device *adev = to_acpi_device(cfg->parent);
+	struct acpi_pci_root *root = acpi_driver_data(adev);
+
+	return root->segment;
+}
+
+int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
+{
+	if (!acpi_disabled) {
+		struct pci_config_window *cfg = bridge->bus->sysdata;
+		struct acpi_device *adev = to_acpi_device(cfg->parent);
+		ACPI_COMPANION_SET(&bridge->dev, adev);
+	}
+
+	return 0;
+}
+
+/*
+ * Lookup the bus range for the domain in MCFG, and set up config space
+ * mapping.
+ */
+static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
+				       struct acpi_pci_generic_root_info *ri)
+{
+	struct resource *bus_res = &root->secondary;
+	u16 seg = root->segment;
+	struct pci_config_window *cfg;
+	struct resource cfgres;
+	unsigned int bsz;
+	int err;
+
+	err = pci_mcfg_lookup(root);
+	if (err) {
+		pr_err("%04x:%pR MCFG region not found\n", seg, bus_res);
+		return err;
+	}
+
+	bsz = 1 << pci_generic_ecam_ops.bus_shift;
+	cfgres.start = root->mcfg_addr + bus_res->start * bsz;
+	cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
+	cfgres.flags = IORESOURCE_MEM;
+	cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res,
+			      &pci_generic_ecam_ops);
+	if (IS_ERR(cfg)) {
+		pr_err("%04x:%pR error %ld mapping CAM\n", seg, bus_res,
+		       PTR_ERR(cfg));
+		return PTR_ERR(cfg);
+	}
+
+	ri->cfg = cfg;
+	return 0;
+}
+
+/* release_info: free resrouces allocated by init_info */
+static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
+{
+	struct acpi_pci_generic_root_info *ri;
+
+	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
+	pci_ecam_free(ri->cfg);
+	kfree(ri);
+}
+
+static struct acpi_pci_root_ops acpi_pci_root_ops = {
+	.release_info = pci_acpi_generic_release_info,
+};
+
+/* Interface called from ACPI code to setup PCI host controller */
 struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 {
-	/* TODO: Should be revisited when implementing PCI on ACPI */
-	return NULL;
+	int node = acpi_get_node(root->device->handle);
+	struct acpi_pci_generic_root_info *ri;
+	struct pci_bus *bus, *child;
+	int err;
+
+	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
+	if (!ri)
+		return NULL;
+
+	err = pci_acpi_setup_ecam_mapping(root, ri);
+	if (err)
+		return NULL;
+
+	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
+	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
+				   ri->cfg);
+	if (!bus)
+		return NULL;
+
+	pci_bus_size_bridges(bus);
+	pci_bus_assign_resources(bus);
+
+	list_for_each_entry(child, &bus->children, node)
+		pcie_bus_configure_settings(child);
+
+	return bus;
 }
 
 void pcibios_add_bus(struct pci_bus *bus)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 9661c85..f66d188 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1390,7 +1390,12 @@ static inline int pci_domain_nr(struct pci_bus *bus)
 {
 	return bus->domain_nr;
 }
+/* Arch specific ACPI hook to set-up domain number */
+#ifdef CONFIG_ACPI
+int acpi_pci_bus_domain_nr(struct pci_bus *bus);
+#else
 static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return -1; }
+#endif
 void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent);
 #else
 static inline void pci_bus_assign_domain_nr(struct pci_bus *bus,
-- 
1.9.1

Re: [PATCH V8 9/9] pci, acpi: ARM64 support for ACPI based generic PCI host controller

[Arnd Bergmann]

On Monday, May 30, 2016 5:14:22 PM CEST Tomasz Nowicki wrote:
> +       bsz = 1 << pci_generic_ecam_ops.bus_shift;
> +       cfgres.start = root->mcfg_addr + bus_res->start * bsz;
> +       cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
> +       cfgres.flags = IORESOURCE_MEM;
> +       cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res,
> +                             &pci_generic_ecam_ops);
> +       if (IS_ERR(cfg)) {
> +               pr_err("%04x:%pR error %ld mapping CAM\n", seg, bus_res,
> +                      PTR_ERR(cfg));
> +               return PTR_ERR(cfg);
> +       }
> +
> +       ri->cfg = cfg;
> +       return 0;
> +}

I wonder if we could do this a little simpler and just put a pointer
to the mmconfig space into struct pci_host_bridge, and then have a trivial
map_bus function alongside pci_generic_config_read/pci_generic_config_write.

As this code is specific to arch/arm64, we don't need any of the complexity
of drivers/pci/ecam.c, which basically only exists to work around the
limited virtual address space of 32-bit machines.

	Arnd

Re: [PATCH V8 9/9] pci, acpi: ARM64 support for ACPI based generic PCI host controller

[Jayachandran C]

On Mon, May 30, 2016 at 9:08 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Monday, May 30, 2016 5:14:22 PM CEST Tomasz Nowicki wrote:
>> +       bsz = 1 << pci_generic_ecam_ops.bus_shift;
>> +       cfgres.start = root->mcfg_addr + bus_res->start * bsz;
>> +       cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
>> +       cfgres.flags = IORESOURCE_MEM;
>> +       cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res,
>> +                             &pci_generic_ecam_ops);
>> +       if (IS_ERR(cfg)) {
>> +               pr_err("%04x:%pR error %ld mapping CAM\n", seg, bus_res,
>> +                      PTR_ERR(cfg));
>> +               return PTR_ERR(cfg);
>> +       }
>> +
>> +       ri->cfg = cfg;
>> +       return 0;
>> +}
>
> I wonder if we could do this a little simpler and just put a pointer
> to the mmconfig space into struct pci_host_bridge, and then have a trivial
> map_bus function alongside pci_generic_config_read/pci_generic_config_write.
>
> As this code is specific to arch/arm64, we don't need any of the complexity
> of drivers/pci/ecam.c, which basically only exists to work around the
> limited virtual address space of 32-bit machines.

The map_bus provided by pci-ecam.h is already trivial in 64bit case,
it was written so that it could be optimized at compile time. The
pci_ecam_create is a useful to have here instead of redoing the
request_resource/ioremap sequence.

The struct pci_config_window can be moved into struct
pci_host_bridge (or a wrapper of pci_host_bridge) when we have
the setup to point bus->sysdata to something like that. The ECAM
API can be fixed up to handle that case as well. I did not make
any changes since the pci_host_bridge discussion is still going on.

JC.

RE: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Gabriele Paoloni]

Hi all

In v7 thread Lorenzo has summarized a roadmap to
get ACPI support for PCI controllers upstream.

The second point of the roadmap was
<< 2) In a real world (1) is not enough. Some ARM64 platforms, not entirely
      ECAM compliant, already shipped with the corresponding firmware that
      we can't update. HW has ECAM quirks and to work around it in the kernel
      we put forward many solutions to the problem, it is time we found a
      solution (when, of course, (1) is completed and upstream).
      Using the MCFG table OEMID matching floated around in this thread
      would work fine for most of the platforms (and cross-OS) that have
      shipped with HW ECAM quirks, so I think that's the starting point for
      our solution and that's how we can sort this out, _today_.

      The solution is a trivial look-up table:
      MCFG OEMID <-> PCI config space ops >>

I think maybe it is worth to post this quirk mechanism as RFC on
top of this v8 patchset, so that we can start to review it and
make some progress on the quirks.
 
If you agree I think Jon can tell who's the best person to
push the quirk RFC (as my understanding is that this mechanism
is currently used by some platforms deployed on the market...)

Thanks

Gab


> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org] On Behalf Of Tomasz Nowicki
> Sent: 30 May 2016 16:14
> To: helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com;
> catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org;
> Lorenzo.Pieralisi@arm.com; okaya@codeaurora.org; jchandra@broadcom.com
> Cc: robert.richter@caviumnetworks.com; mw@semihalf.com;
> Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; Wangyijing;
> Suravee.Suthikulpanit@amd.com; msalter@redhat.com; linux-
> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux-
> acpi@vger.kernel.org; linux-kernel@vger.kernel.org; linaro-
> acpi@lists.linaro.org; jcm@redhat.com; andrea.gallo@linaro.org;
> dhdang@apm.com; jeremy.linton@arm.com; liudongdong (C);
> cov@codeaurora.org; Tomasz Nowicki
> Subject: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host
> controller
> 
> From the functionality point of view this series may be split into the
> following logic parts:
> 1. Export ECAM API and add parent device to pci_config_window
> 2. Add IO resources handling to PCI core code
> 3. Support for generic domain assignment based on ACPI
> 4. New MCFG driver
> 5. Implement ARM64 ACPI based PCI host controller driver under
> arch/arm64/
> 
> Patches has been built on top of 4.7-rc1 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v8)
> 
> This has been tested on Cavium ThunderX server. Any help in reviewing
> and
> testing is very appreciated.
> 
> v7 -> v8
> - move code from drivers/acpi/pci_root_generic.c to
> arch/arm64/kernel/pci.c
> - minor changes around domain assignment
> - pci_mcfg.c improvements for parsing MCFG tables and lookup its
> entries
> 
> v6 -> v7
> - drop quirks handling
> - changes for ACPI companion and domain number assignment approach
> - implement arch pcibios_{add|remove}_bus and call
> acpi_pci_{add|remove}_bus from there
> - cleanups around nomenclature
> - use resources oriented API for ECAM
> - fix for based address calculation before mapping ECAM region
> - remove useless lock for MCFG lookup
> - move MCFG stuff to separated file pci_mcfg.c
> - drop MCFG entries caching
> - rebase against 4.6-rc7
> 
> v5 -> v6
> - drop idea of x86 MMCONFIG code refactoring
> - integrate JC's patches which introduce new ECAM API:
>   https://lkml.org/lkml/2016/4/11/907
>   git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> - integrate Sinan's fix for releasing IO resources, see patch [06/13]
> - added ACPI support for ThunderX ECAM and PEM drivers
> - rebase against 4.6-rc2
> 
> v4 -> v5
> - drop MCFG refactoring group patches 1-6 from series v4 and integrate
> Jayachandran's patch
>   https://patchwork.ozlabs.org/patch/575525/
> - rewrite PCI legacy IRQs allocation
> - squash two patches 11 and 12 from series v4, fixed bisection issue
> - changelog improvements
> - rebase against 4.5-rc3
> 
> v3 -> v4
> - drop Jiang's fix
> http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> - add Lorenzo's fix patch 19/24
> - ACPI PCI bus domain number assigning cleanup
> - change resource management, we now claim and reassign resources
> - improvements for applying quirks
> - drop Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html
> dependency
> - rebase against 4.5-rc1
> 
> v2 -> v3
> - fix legacy IRQ assigning and IO ports registration
> - remove reference to arch specific companion device for ia64
> - move ACPI PCI host controller driver to pci_root.c
> - drop generic domain assignment for x86 and ia64 as I am not
>   able to run all necessary test variants
> - drop patch which cleaned legacy IRQ assignment since it belongs to
>   Mathew's series:
>   https://patchwork.ozlabs.org/patch/557504/
> - extend MCFG quirk code
> - rebase against 4.4
> 
> v1 -> v2
> - move non-arch specific piece of code to dirver/acpi/ directory
> - fix IO resource handling
> - introduce PCI config accessors quirks matching
> - moved ACPI_COMPANION_SET to generic code
> 
> v1 - https://lkml.org/lkml/2015/10/27/504
> v2 - https://lkml.org/lkml/2015/12/16/246
> v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
> v4 - https://lkml.org/lkml/2016/2/4/646
> v5 - https://lkml.org/lkml/2016/2/16/426
> v6 - https://lkml.org/lkml/2016/4/15/594
> 
> Jayachandran C (2):
>   PCI: ecam: move ecam.h to linux/include/pci-ecam.h
>   PCI: ecam: Add parent device field to pci_config_window
> 
> Tomasz Nowicki (7):
>   pci: Add new function to unmap IO resources.
>   acpi, pci: Support IO resources when parsing PCI host bridge
>     resources.
>   pci, acpi: add acpi hook to assign domain number.
>   arm64, pci, acpi: ACPI support for legacy IRQs parsing and
>     consolidation with DT code.
>   acpi: Add generic MCFG table handling
>   arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus
>     enumeration.
>   pci, acpi: ARM64 support for ACPI based generic PCI host controller
> 
>  arch/arm64/Kconfig                  |   2 +
>  arch/arm64/kernel/pci.c             | 143
> ++++++++++++++++++++++++++++++++++--
>  drivers/acpi/Kconfig                |   3 +
>  drivers/acpi/Makefile               |   1 +
>  drivers/acpi/pci_mcfg.c             |  94 ++++++++++++++++++++++++
>  drivers/acpi/pci_root.c             |  39 ++++++++++
>  drivers/pci/ecam.c                  |   6 +-
>  drivers/pci/ecam.h                  |  67 -----------------
>  drivers/pci/host/pci-host-common.c  |   3 +-
>  drivers/pci/host/pci-host-generic.c |   3 +-
>  drivers/pci/host/pci-thunder-ecam.c |   3 +-
>  drivers/pci/host/pci-thunder-pem.c  |   6 +-
>  drivers/pci/pci.c                   |  29 +++++++-
>  include/linux/pci-acpi.h            |   2 +
>  include/linux/pci-ecam.h            |  67 +++++++++++++++++
>  include/linux/pci.h                 |   9 ++-
>  16 files changed, 387 insertions(+), 90 deletions(-)
>  create mode 100644 drivers/acpi/pci_mcfg.c
>  delete mode 100644 drivers/pci/ecam.h
>  create mode 100644 include/linux/pci-ecam.h
> 
> --
> 1.9.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Jon Masters]

On 06/01/2016 03:36 AM, Gabriele Paoloni wrote:

> If you agree I think Jon can tell who's the best person to
> push the quirk RFC (as my understanding is that this mechanism
> is currently used by some platforms deployed on the market...)

Let me ping Linaro folks to see who has that (quirks) ball. We can
certainly share the older OEM matching quirks Mark Salter did for
earlier RHEL(SA) internal versions as a seed for that[0] activity.

BUT...I don't think we should block this thread on the quirks. They're
separate (but important). I see Arnd's reply as well, but nobody else
has yet chimed in to this thread (and I am about to prod all of the
vendors to reply to this thread and ACK). Can I ask whether we can't
just stage v8 as-is for -next at this point? Can Arnd's (or other)
suggestions be handled as followup patches post-merge please? Bjorn?

Jon.

[0] RHEL(SA) has had three different PCIe enabled ACPI stacks maintained
independently so far since we need PCIe to boot most of the hundreds of
v8 systems we have internal to Red Hat - which all have PCI.

-- 
Computer Architect | Sent from my Fedora powered laptop

Re: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Jon Masters]

[-- Attachment #1: Type: text/plain, Size: 792 bytes --]

On 05/30/2016 11:14 AM, Tomasz Nowicki wrote:
> From the functionality point of view this series may be split into the
> following logic parts:
> 1. Export ECAM API and add parent device to pci_config_window
> 2. Add IO resources handling to PCI core code
> 3. Support for generic domain assignment based on ACPI
> 4. New MCFG driver
> 5. Implement ARM64 ACPI based PCI host controller driver under arch/arm64/
> 
> Patches has been built on top of 4.7-rc1 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v8)
> 
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.

Boot log from Intel Itanium server attached for existential proof of
non-interference with IA64 systems on latest patch revision.

Jon.

[-- Attachment #2: hamartia-4.7.0-rc1.jcm_tomasz_pciv8_test+_boot.txt --]
[-- Type: text/plain, Size: 22749 bytes --]

[    0.000000] Linux version 4.7.0-rc1.jcm_tomasz_pciv8_test+ (jcm@hamartia) (gcc version 4.4.5 (Debian 4.4.5-8) ) #3 SMP Thu Jun 2 04:34:49 EDT 2016
[    0.000000] EFI v1.10 by HP:
[    0.000000] efi:  SALsystab=0x3fefa000  ACPI 2.0=0x3fd5e000  SMBIOS=0x3fefc000  HCDP=0x3fd5c000 
[    0.000000] booting generic kernel on platform hpzx1
[    0.000000] PCDP: v0 at 0x3fd5c000
[    0.000000] earlycon: uart8250 at MMIO 0x00000000ff5e0000 (options '9600n8')
[    0.000000] bootconsole [uart8250] enabled
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x000000003FD5E000 000028 (v02 HP    )
[    0.000000] ACPI: XSDT 0x000000003FD5E02C 00008C (v01 HP     rx2620   00000000 HP   00000000)
[    0.000000] ACPI: FACP 0x000000003FD672A0 0000F4 (v03 HP     rx2620   00000000 HP   00000000)
[    0.000000] ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe0Block: 32/16 (20160422/tbfadt-624)
[    0.000000] ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe1Block: 32/16 (20160422/tbfadt-624)
[    0.000000] ACPI: DSDT 0x000000003FD5E100 005F3C (v01 HP     rx2620   00000007 INTL 02012044)
[    0.000000] ACPI: FACS 0x000000003FD67398 000040
[    0.000000] ACPI: SPCR 0x000000003FD673D8 000050 (v01 HP     rx2620   00000000 HP   00000000)
[    0.000000] ACPI: DBGP 0x000000003FD67428 000034 (v01 HP     rx2620   00000000 HP   00000000)
[    0.000000] ACPI: APIC 0x000000003FD67520 0000A4 (v01 HP     rx2620   00000000 HP   00000000)
[    0.000000] ACPI: SPMI 0x000000003FD67460 000050 (v04 HP     rx2620   00000000 HP   00000000)
[    0.000000] ACPI: CPEP 0x000000003FD674B0 000034 (v01 HP     rx2620   00000000 HP   00000000)
[    0.000000] ACPI: SSDT 0x000000003FD64040 0001D6 (v01 HP     rx2620   00000006 INTL 02012044)
[    0.000000] ACPI: SSDT 0x000000003FD64220 000702 (v01 HP     rx2620   00000006 INTL 02012044)
[    0.000000] ACPI: SSDT 0x000000003FD64930 000A16 (v01 HP     rx2620   00000006 INTL 02012044)
[    0.000000] ACPI: SSDT 0x000000003FD65350 000A16 (v01 HP     rx2620   00000006 INTL 02012044)
[    0.000000] ACPI: SSDT 0x000000003FD65D70 000A16 (v01 HP     rx2620   00000006 INTL 02012044)
[    0.000000] ACPI: SSDT 0x000000003FD66790 000A16 (v01 HP     rx2620   00000006 INTL 02012044)
[    0.000000] ACPI: SSDT 0x000000003FD671B0 0000EB (v01 HP     rx2620   00000006 INTL 02012044)
[    0.000000] ACPI: Local APIC address c0000000fee00000
[    0.000000] 1 CPUs available, 1 CPUs total
[    0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] warning: skipping physical page 0
[    0.000000] warning: skipping physical page 0
[    0.000000] warning: skipping physical page 0
[    0.000000] warning: skipping physical page 0
[    0.000000] Initial ramdisk at: 0xe00000003d4aa000 (17110715 bytes)
[    0.000000] SAL 3.1: HP version 3.15
[    0.000000] SAL Platform features: None
[    0.000000] SAL: AP wakeup using external interrupt vector 0xff
[    0.000000] MCA related initialization done
[    0.000000] warning: skipping physical page 0
[    0.000000] Virtual mem_map starts at 0xa0007fffffc80000
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000004000-0x00000000ffffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000004000-0x000000003f4b3fff]
[    0.000000]   node   0: [mem 0x000000003fc00000-0x000000003fc7bfff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000004000-0x000000003fc7bfff]
[    0.000000] Built 1 zonelists in Node order, mobility grouping off.  Total pages: 64619
[    0.000000] Policy zone: DMA
[    0.000000] Kernel command line: BOOT_IMAGE=scsi0:/EFI/debian/vmtest root=/dev/sda2  ro
[    0.000000] PID hash table entries: 4096 (order: 1, 32768 bytes)
[    0.000000] Sorting __ex_table...
[    0.000000] Memory: 989776K/1037488K available (12192K kernel code, 1251K rwdata, 1440K rodata, 816K init, 816K bss, 47712K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=1, Nodes=256
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=1
[    0.000000] NR_IRQS:1024
[    0.000000] ACPI: Local APIC address c0000000fee00000
[    0.000000] GSI 36 (level, low) -> CPU 0 (0x0000) vector 49
[    0.000000] clocksource: itc: mask: 0xffffffffffffffff max_cycles: 0x17101df767f, max_idle_ns: 440795286379 ns
[    0.000000] Console: colour dummy device 80x25
[    0.192000] Calibrating delay loop... 2390.01 BogoMIPS (lpj=4780032)
[    0.288000] pid_max: default: 32768 minimum: 301
[    0.348000] ACPI: Core revision 20160422
[    0.400000] Security Framework initialized
[    0.452000] SELinux:  Disabled at boot.
[    0.504000] Dentry cache hash table entries: 131072 (order: 6, 1048576 bytes)
[    0.596000] Inode-cache hash table entries: 65536 (order: 5, 524288 bytes)
[    0.688000] Mount-cache hash table entries: 2048 (order: 0, 16384 bytes)
[    0.780000] Mountpoint-cache hash table entries: 2048 (order: 0, 16384 bytes)
[    0.872000] Boot processor id 0x0/0x0
[    0.920000] Brought up 1 CPUs
[    0.960000] Total of 1 processors activated (2390.01 BogoMIPS).
[    1.040000] devtmpfs: initialized
[    1.084000] SMBIOS 2.3 present.
[    1.124000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.252000] NET: Registered protocol family 16
[    1.312000] ACPI: bus type PCI registered
[    1.364000] HugeTLB registered 256 MB page size, pre-allocated 0 pages
[    1.452000] ACPI: Added _OSI(Module Device)
[    1.508000] ACPI: Added _OSI(Processor Device)
[    1.564000] ACPI: Added _OSI(3.0 _SCP Extensions)
[    1.628000] ACPI: Added _OSI(Processor Aggregator Device)
[    1.704000] ACPI: 8 ACPI AML tables successfully acquired and loaded
[    1.784000] 
[    1.804000] kernel unaligned access to 0xc0000000ff5c1016, ip=0xa00000010059b1e0
[    1.904000] kernel unaligned access to 0xc0000000ff5c1016, ip=0xa00000010061da70
[    2.000000] kernel unaligned access to 0xc0000000ff5c1017, ip=0xa00000010059b1e0
[    2.100000] kernel unaligned access to 0xc0000000ff5c1017, ip=0xa00000010061da70
[    2.196000] kernel unaligned access to 0xc0000000ff5c1015, ip=0xa00000010059b1e0
[    2.304000] ACPI: Interpreter enabled
[    2.356000] ACPI: (supports S0 S5)
[    2.400000] ACPI: Using IOSAPIC for interrupt routing
[    2.608000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
[    2.688000] acpi HWP0002:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    2.796000] acpi HWP0002:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    2.896000] PCI host bridge to bus 0000:00
[    2.948000] pci_bus 0000:00: root bus resource [io  0x0000-0x1fff window]
[    3.040000] pci_bus 0000:00: root bus resource [mem 0x80000000-0x8fffffff window]
[    3.136000] pci_bus 0000:00: root bus resource [mem 0x80004000000-0x80103fffffe window]
[    3.244000] pci_bus 0000:00: root bus resource [bus 00-1f]
[    3.356000] ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 20-3f])
[    3.436000] acpi HWP0002:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    3.544000] acpi HWP0002:01: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    3.648000] PCI host bridge to bus 0000:20
[    3.700000] pci_bus 0000:20: root bus resource [io  0x2000-0x2fff window]
[    3.792000] pci_bus 0000:20: root bus resource [mem 0x90000000-0x97ffffff window]
[    3.888000] pci_bus 0000:20: root bus resource [mem 0x90004000000-0x90103fffffe window]
[    3.996000] pci_bus 0000:20: root bus resource [bus 20-3f]
[    4.108001] ACPI: PCI Root Bridge [PCI2] (domain 0000 [bus 40-5f])
[    4.192001] acpi HWP0002:02: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    4.300001] acpi HWP0002:02: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    4.396001] PCI host bridge to bus 0000:40
[    4.452001] pci_bus 0000:40: root bus resource [io  0x3000-0x5fff window]
[    4.540001] pci_bus 0000:40: root bus resource [mem 0x98000000-0xafffffff window]
[    4.640001] pci_bus 0000:40: root bus resource [mem 0xa0004000000-0xa0103fffffe window]
[    4.744001] pci_bus 0000:40: root bus resource [bus 40-5f]
[    4.840001] ACPI: PCI Root Bridge [PCI3] (domain 0000 [bus 60-7f])
[    4.920001] acpi HWP0002:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    5.028001] acpi HWP0002:03: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    5.128001] PCI host bridge to bus 0000:60
[    5.184001] pci_bus 0000:60: root bus resource [io  0x6000-0x7fff window]
[    5.272001] pci_bus 0000:60: root bus resource [mem 0xb0000000-0xc7ffffff window]
[    5.372001] pci_bus 0000:60: root bus resource [mem 0xb0004000000-0xb0103fffffe window]
[    5.476001] pci_bus 0000:60: root bus resource [bus 60-7f]
[    5.572001] ACPI: PCI Root Bridge [PCI4] (domain 0000 [bus 80-bf])
[    5.652001] acpi HWP0002:04: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    5.760001] acpi HWP0002:04: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    5.860001] PCI host bridge to bus 0000:80
[    5.912001] pci_bus 0000:80: root bus resource [io  0x8000-0xbfff window]
[    6.004001] pci_bus 0000:80: root bus resource [mem 0xc8000000-0xdfffffff window]
[    6.100001] pci_bus 0000:80: root bus resource [mem 0xc0004000000-0xc0103fffffe window]
[    6.208001] pci_bus 0000:80: root bus resource [bus 80-bf]
[    6.304001] ACPI: PCI Root Bridge [PCI6] (domain 0000 [bus c0-df])
[    6.384001] acpi HWP0002:05: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    6.492001] acpi HWP0002:05: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    6.592001] PCI host bridge to bus 0000:c0
[    6.644001] pci_bus 0000:c0: root bus resource [io  0xc000-0xdfff window]
[    6.732001] pci_bus 0000:c0: root bus resource [mem 0xe0000000-0xefffffff window]
[    6.832001] pci_bus 0000:c0: root bus resource [mem 0xe0004000000-0xe0103fffffe window]
[    6.940001] pci_bus 0000:c0: root bus resource [bus c0-df]
[    7.024001] ia64_handle_unaligned: 4 callbacks suppressed
[    7.024001] kernel unaligned access to 0xc0000000ff5c1016, ip=0xa00000010059b1e0
[    7.024001] kernel unaligned access to 0xc0000000ff5c1016, ip=0xa00000010059b1e0
[    7.024001] kernel unaligned access to 0xc0000000ff5c1016, ip=0xa00000010061da70
[    7.024001] ACPI: Enabled 1 GPEs in block 10 to 1F
[    7.448001] vgaarb: loaded
[    7.484001] SCSI subsystem initialized
[    7.536001] ACPI: bus type USB registered
[    7.588001] usbcore: registered new interface driver usbfs
[    7.660001] usbcore: registered new interface driver hub
[    7.728001] usbcore: registered new device driver usb
[    7.800001] IOC: zx1 2.3 HPA 0xfed01000 IOVA space 1024Mb at 0x40000000
[    7.936001] clocksource: Switched to clocksource itc
[    8.005385] VFS: Disk quotas dquot_6.6.0
[    8.057154] VFS: Dquot-cache hash table entries: 2048 (order 0, 16384 bytes)
[    8.150083] pnp: PnP ACPI init
[    8.190381] GSI 34 (level, low) -> CPU 0 (0x0000) vector 50
[    8.266957] GSI 35 (level, low) -> CPU 0 (0x0000) vector 51
[    8.382737] pnp: PnP ACPI: found 2 devices
[    8.439035] NET: Registered protocol family 2
[    8.498935] TCP established hash table entries: 8192 (order: 2, 65536 bytes)
[    8.592072] TCP bind hash table entries: 8192 (order: 3, 131072 bytes)
[    8.678109] TCP: Hash tables configured (established 8192 bind 8192)
[    8.761965] UDP hash table entries: 512 (order: 0, 16384 bytes)
[    8.839985] UDP-Lite hash table entries: 512 (order: 0, 16384 bytes)
[    8.923737] NET: Registered protocol family 1
[    8.981126] GSI 16 (level, low) -> CPU 0 (0x0000) vector 52
[    9.056135] GSI 16 (level, low) -> CPU 0 (0x0000) vector 52 unregistered
[    9.146035] GSI 17 (level, low) -> CPU 0 (0x0000) vector 52
[    9.221032] GSI 17 (level, low) -> CPU 0 (0x0000) vector 52 unregistered
[    9.310934] GSI 18 (level, low) -> CPU 0 (0x0000) vector 52
[    9.385929] GSI 18 (level, low) -> CPU 0 (0x0000) vector 52 unregistered
[    9.475832] Unpacking initramfs...
[   10.230738] Freeing initrd memory: 16688kB freed
[   10.294377] perfmon: version 2.0 IRQ 238
[   10.346357] perfmon: Itanium 2 PMU detected, 16 PMCs, 18 PMDs, 4 counters (47 bits)
[   10.447206] perfmon: added sampling format default_format
[   10.518758] perfmon_default_smpl: default_format v2.0 registered
[   10.597825] futex hash table entries: 256 (order: 1, 32768 bytes)
[   10.678242] audit: initializing netlink subsys (disabled)
[   10.749386] audit: type=2000 audit(1464856926.744:1): initialized
[   10.829702] workingset: timestamp_bits=52 max_order=16 bucket_order=0
[   10.918740] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[   11.016274] io scheduler noop registered
[   11.067929] io scheduler deadline registered
[   11.124077] io scheduler cfq registered (default)
[   11.186057] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[   11.283778] ACPI: Power Button [PWRF]
[   11.331993] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
[   11.429508] ACPI: Sleep Button [SLPF]
[   11.477619] thermal LNXTHERM:00: registered as thermal_zone0
[   11.552527] ACPI: Thermal Zone [THM0] (27 C)
[   11.608667] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[   11.711595] 00:00: ttyS0 at MMIO 0xff5e0000 (irq = 50, base_baud = 115200) is a 16550A
[   11.815907] console [ttyS0] enabled
[   11.815907] console [ttyS0] enabled
[   11.903612] bootconsole [uart8250] disabled
[   11.903612] bootconsole [uart8250] disabled
[   12.029184] 00:01: ttyS1 at MMIO 0xff5e2000 (irq = 51, base_baud = 115200) is a 16550A
[   12.124137] Linux agpgart interface v0.103
[   12.190737] loop: module loaded
[   12.230478] Uniform Multi-Platform E-IDE driver
[   12.284858] ide-cd driver 5.00
[   12.321519] GSI 21 (level, low) -> CPU 0 (0x0000) vector 52
[   12.390071] pata_cmd64x 0000:00:02.0: Secondary port is disabled
[   12.462264] scsi host0: pata_cmd64x
[   12.504649] scsi host1: pata_cmd64x
[   12.546629] ata1: PATA max UDMA/100 cmd 0xd18 ctl 0xd24 bmdma 0xd00 irq 55
[   12.629141] ata2: DUMMY
[   12.658395] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
[   12.743000] e1000: Copyright (c) 1999-2006 Intel Corporation.
[   12.811953] GSI 29 (level, low) -> CPU 0 (0x0000) vector 53
[   12.982737] ata1.00: ATAPI: DV-28E-N, E.6A, max UDMA/33
[   13.122737] ata1.00: configured for UDMA/33
[   13.230737] scsi 0:0:0:0: CD-ROM            TEAC     DV-28E-N         E.6A PQ: 0 ANSI: 5
[   13.386740] sr 0:0:0:0: [sr0] scsi3-mmc drive: 24x/24x cd/rw xa/form2 cdda tray
[   13.475091] cdrom: Uniform CD-ROM driver Revision: 3.20
[   13.546746] e1000 0000:20:02.0 eth0: (PCI-X:66MHz:64-bit) 00:17:08:7c:18:20
[   13.631138] e1000 0000:20:02.0 eth0: Intel(R) PRO/1000 Network Connection
[   13.712595] sr 0:0:0:0: Attached scsi generic sg0 type 5
[   13.776550] GSI 30 (level, low) -> CPU 0 (0x0000) vector 54
[   14.154741] e1000 0000:20:02.1 eth1: (PCI-X:66MHz:64-bit) 00:17:08:7c:18:21
[   14.240104] e1000 0000:20:02.1 eth1: Intel(R) PRO/1000 Network Connection
[   14.321559] Fusion MPT base driver 3.04.20
[   14.370719] Copyright (c) 1999-2008 LSI Corporation
[   14.429161] Fusion MPT SPI Host driver 3.04.20
[   14.482388] GSI 27 (level, low) -> CPU 0 (0x0000) vector 55
[   14.550625] mptbase: ioc0: Initiating bringup
[   15.082739] ioc0: LSI53C1030 C0: Capabilities={Initiator,Target}
[   15.826739] scsi host2: ioc0: LSI53C1030 C0, FwRev=01032341h, Ports=1, MaxQ=255, IRQ=58
[   16.558739] GSI 28 (level, low) -> CPU 0 (0x0000) vector 56
[   16.628260] mptbase: ioc1: Initiating bringup
[   17.158739] ioc1: LSI53C1030 C0: Capabilities={Initiator,Target}
[   17.902739] scsi host3: ioc1: LSI53C1030 C0, FwRev=01032341h, Ports=1, MaxQ=255, IRQ=59
[   18.001207] scsi 2:0:0:0: Direct-Access     HP 73.4G ST373454LC       HPC2 PQ: 0 ANSI: 3
[   18.099853] scsi target2:0:0: Beginning Domain Validation
[   18.782739] Fusion MPT FC Host driver 3.04.20
[   18.836205] Fusion MPT SAS Host driver 3.04.20
[   18.889538] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[   18.967884] ehci-pci: EHCI PCI platform driver
[   19.021207] GSI 18 (level, low) -> CPU 0 (0x0000) vector 57
[   19.089650] ehci-pci 0000:00:01.2: EHCI Host Controller
[   19.152462] ehci-pci 0000:00:01.2: new USB bus registered, assigned bus number 1
[   19.241220] ehci-pci 0000:00:01.2: irq 54, io mem 0x80000000
[   19.314741] scsi target2:0:0: Ending Domain Validation
[   19.378402] ehci-pci 0000:00:01.2: USB 2.0 started, EHCI 0.95
[   19.447367] scsi target2:0:0: FAST-160 WIDE SCSI 320.0 MB/s DT IU QAS RTI WRFLOW PCOMP (6.25 ns, offset 63)
[   19.564259] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[   19.645707] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   19.732378] usb usb1: Product: EHCI Host Controller
[   19.790910] usb usb1: Manufacturer: Linux 4.7.0-rc1.jcm_tomasz_pciv8_test+ ehci_hcd
[   19.882692] usb usb1: SerialNumber: 0000:00:01.2
[   19.938098] hub 1-0:1.0: USB hub found
[   19.983304] hub 1-0:1.0: 5 ports detected
[   20.031429] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[   20.105814] mousedev: PS/2 mouse device common for all mice
[   20.172792] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[   20.248314] EFI Variables Facility v0.08 2004-May-17
[   20.308105] NET: Registered protocol family 10
[   20.362164] NET: Registered protocol family 17
[   20.415919] registered taskstats version 1
[   20.465289] rtc-efi rtc-efi: setting system clock to 2016-06-02 08:42:17 UTC (1464856937)
[   23.666739] sd 2:0:0:0: Attached scsi generic sg1 type 0
[   23.732532] sd 2:0:0:0: [sda] 143374738 512-byte logical blocks: (73.4 GB/68.4 GiB)
[   23.824829] sd 2:0:0:0: [sda] Write Protect is off
[   23.883778] sd 2:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
[   24.010740]  sda: sda1 sda2 sda3
[   24.052839] sd 2:0:0:0: [sda] Attached SCSI disk
[   24.113158] Freeing unused kernel memory: 816K (e000000004e74000 - e000000004f40000)
[   24.206089] This architecture does not have kernel memory protection.
Loading, please wait...
[   24.302743] udev[112]: starting version 164
FATAL: Could not[   24.646740] EXT4-fs (sda2): mounting ext3 file system using the ext4 subsystem
 load /lib/modul[   24.750740] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
es/4.7.0-rc1.jcm_tomasz_pciv8_test+/modules.dep: No such file or directory
Begin: Loading essential drivers ... FATAL: Could not load /lib/modules/4.7.0-rc1.jcm_tomasz_pciv8_test+/modules.dep: No such file or directory
done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
FATAL: Could not load /lib/modules/4.7.0-rc1.jcm_tomasz_pciv8_test+/modules.dep: No such file or directory
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... done.
modprobe: FATAL: Could not load /lib/modules/4.7.0-rc1.jcm_tomasz_pciv8_test+/modules.dep: No such file or directory

\rINIT: version 2.88 booting
[   25.718740] random: nonblocking pool is initialized
Using makefile-style concurrent boot in runlevel S.
Starting the hotplug events dispatcher: udevd[   26.114739] udev[261]: starting version 164
.
Synthesizing the initial hotplug events...done.
Waiting for /dev to be fully populated...done.
Activating swap...[   26.894740] Adding 2000784k swap on /dev/sda3.  Priority:-1 extents:1 across:2000784k 
done.
[   26.993208] EXT4-fs (sda2): re-mounted. Opts: (null)
Checking root file system...fsck from util-linux-ng 2.17.2
/dev/sda2: clean[   27.194743] EXT4-fs (sda2): re-mounted. Opts: errors=remount-ro
, 95298/4349952 files, 1091056/17397216 blocks
done.
Cleaning up ifupdown....
Loading kernel modules...done.
Setting up networking....
Activating lvm and md swap...done.
Checking file systems...fsck from util-linux-ng 2.17.2
done.
Mounting local filesystems...done.
Activating swapfile swap...done.
Cleaning up temporary files....
Configuring network interfaces...done.
Starting portmap daemon....
Starting NFS common utilities: statd.
Cleaning up temporary files....
Setting console screen modes and fonts.
cannot (un)set powersave mode
^[[9;30]^[[14;30]Setting kernel variables ...done.
\rINIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting portmap daemon...Already running..
Starting NFS common utilities: statd.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting ACPI services....
Starting MTA:[   28.750744] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   28.824281] e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[   28.922747] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
 exim4.
Starting mpt-status monitor: mpt-statusd.
Starting OpenBSD Secure Shell server: sshd[   30.010739] sshd (1079): /proc/1079/oom_adj is deprecated, please use /proc/1079/oom_score_adj instead.
.

Debian GNU/Linux 6.0 hamartia ttyS0

hamartia login: root
Password: 
Last login: Thu Jun  2 04:30:46 EDT 2016 on ttyS0
Linux hamartia 4.7.0-rc1.jcm_tomasz_pciv8_test+ #3 SMP Thu Jun 2 04:34:49 EDT 2016 ia64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
root@hamartia:~# ping www.intel.com
PING a961.g.akamai.net (104.96.210.229) 56(84) bytes of data.
64 bytes from a104-96-210-229.deploy.static.akamaitechnologies.com (104.96.210.229): icmp_req=1 ttl=58 time=11.9 ms
64 bytes from a104-96-210-229.deploy.static.akamaitechnologies.com (104.96.210.229): icmp_req=2 ttl=58 time=10.2 ms
64 bytes from a104-96-210-229.deploy.static.akamaitechnologies.com (104.96.210.229): icmp_req=3 ttl=58 time=10.6 ms
^C
--- a961.g.akamai.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2005ms
rtt min/avg/max/mdev = 10.272/10.971/11.944/0.719 ms

Re: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Tomasz Nowicki]

Hi Gabriele,

On 01.06.2016 09:36, Gabriele Paoloni wrote:
> I think maybe it is worth to post this quirk mechanism as RFC on
> top of this v8 patchset, so that we can start to review it and
> make some progress on the quirks.

I've just posted my RFC patches, please see:
https://patchwork.ozlabs.org/patch/629121/

We can start review from there so that this series is not blocked.

Thanks,
Tomasz

Re: [PATCH V8 9/9] pci, acpi: ARM64 support for ACPI based generic PCI host controller

[Lorenzo Pieralisi]

On Mon, May 30, 2016 at 05:14:22PM +0200, Tomasz Nowicki wrote:
> This patch implements pci_acpi_scan_root call so that ARM64 can start
> using ACPI to setup and enumerate PCI buses.
> 
> The implementation of pci_acpi_scan_root() looks up config space regions
> through MCFG interface. Then ECAM library is doing a new mapping
> and attach generic ECAM ops which are used for accessing config space.
> 
> On ARM64, ACPI and DT can be enabled together, and in that case
> we need to use generic domains. In order to do that we implement
> ARM64 specific way of retrieving domain number from pci_config_window
> structure.
> 
> Since we enable PCI for ACPI we need to implement raw_pci_{read|write}
> at the same time. ARM64 provides RAW accessors as long as there is
> correlated valid pci_bus structure, but not before.
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>

[...]

> +/* release_info: free resrouces allocated by init_info */

Nit: s/resrouces/resources

> +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
> +{
> +	struct acpi_pci_generic_root_info *ri;
> +
> +	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
> +	pci_ecam_free(ri->cfg);
> +	kfree(ri);
> +}
> +
> +static struct acpi_pci_root_ops acpi_pci_root_ops = {
> +	.release_info = pci_acpi_generic_release_info,
> +};
> +
> +/* Interface called from ACPI code to setup PCI host controller */
>  struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  {
> -	/* TODO: Should be revisited when implementing PCI on ACPI */
> -	return NULL;
> +	int node = acpi_get_node(root->device->handle);
> +	struct acpi_pci_generic_root_info *ri;
> +	struct pci_bus *bus, *child;
> +	int err;
> +
> +	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
> +	if (!ri)
> +		return NULL;
> +
> +	err = pci_acpi_setup_ecam_mapping(root, ri);
> +	if (err)
> +		return NULL;
                ^
Leaking memory -> kfree(ri)

> +	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
> +	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
> +				   ri->cfg);
> +	if (!bus)
> +		return NULL;

Ditto.

> +
> +	pci_bus_size_bridges(bus);
> +	pci_bus_assign_resources(bus);
> +
> +	list_for_each_entry(child, &bus->children, node)
> +		pcie_bus_configure_settings(child);
> +
> +	return bus;
>  }
>  
>  void pcibios_add_bus(struct pci_bus *bus)
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 9661c85..f66d188 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1390,7 +1390,12 @@ static inline int pci_domain_nr(struct pci_bus *bus)
>  {
>  	return bus->domain_nr;
>  }
> +/* Arch specific ACPI hook to set-up domain number */
> +#ifdef CONFIG_ACPI
> +int acpi_pci_bus_domain_nr(struct pci_bus *bus);

Technically speaking, this should be introduced in a separate patch;
given that I know it is a temporary plaster and no other architecture will
have to implement it I reckon it is fine to leave it here, I will make
it disappear as fast as I can.

We can argue if it is best to move the ACPI bits into a separate file
(acpi-pci.c), I am fine with keeping DT/ACPI in one file if that's fine
with Catalin and Will.

You can send the fix-ups above in the final pull request which I hope
is nigh.

Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> +#else
>  static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return -1; }
> +#endif
>  void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent);
>  #else
>  static inline void pci_bus_assign_domain_nr(struct pci_bus *bus,

Re: [PATCH V8 9/9] pci, acpi: ARM64 support for ACPI based generic PCI host controller

[Tomasz Nowicki]

On 02.06.2016 11:35, Lorenzo Pieralisi wrote:
> On Mon, May 30, 2016 at 05:14:22PM +0200, Tomasz Nowicki wrote:
>> This patch implements pci_acpi_scan_root call so that ARM64 can start
>> using ACPI to setup and enumerate PCI buses.
>>
>> The implementation of pci_acpi_scan_root() looks up config space regions
>> through MCFG interface. Then ECAM library is doing a new mapping
>> and attach generic ECAM ops which are used for accessing config space.
>>
>> On ARM64, ACPI and DT can be enabled together, and in that case
>> we need to use generic domains. In order to do that we implement
>> ARM64 specific way of retrieving domain number from pci_config_window
>> structure.
>>
>> Since we enable PCI for ACPI we need to implement raw_pci_{read|write}
>> at the same time. ARM64 provides RAW accessors as long as there is
>> correlated valid pci_bus structure, but not before.
>>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>
> [...]
>
>> +/* release_info: free resrouces allocated by init_info */
>
> Nit: s/resrouces/resources
>
>> +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
>> +{
>> +	struct acpi_pci_generic_root_info *ri;
>> +
>> +	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
>> +	pci_ecam_free(ri->cfg);
>> +	kfree(ri);
>> +}
>> +
>> +static struct acpi_pci_root_ops acpi_pci_root_ops = {
>> +	.release_info = pci_acpi_generic_release_info,
>> +};
>> +
>> +/* Interface called from ACPI code to setup PCI host controller */
>>   struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>>   {
>> -	/* TODO: Should be revisited when implementing PCI on ACPI */
>> -	return NULL;
>> +	int node = acpi_get_node(root->device->handle);
>> +	struct acpi_pci_generic_root_info *ri;
>> +	struct pci_bus *bus, *child;
>> +	int err;
>> +
>> +	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
>> +	if (!ri)
>> +		return NULL;
>> +
>> +	err = pci_acpi_setup_ecam_mapping(root, ri);
>> +	if (err)
>> +		return NULL;
>                  ^
> Leaking memory -> kfree(ri)

I missed that. Will fix.

>
>> +	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
>> +	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
>> +				   ri->cfg);
>> +	if (!bus)
>> +		return NULL;
>
> Ditto.

Here kfree(ri) is not needed. __acpi_pci_root_release_info will free ri 
inside of acpi_pci_root_create.

>
>> +
>> +	pci_bus_size_bridges(bus);
>> +	pci_bus_assign_resources(bus);
>> +
>> +	list_for_each_entry(child, &bus->children, node)
>> +		pcie_bus_configure_settings(child);
>> +
>> +	return bus;
>>   }
>>
>>   void pcibios_add_bus(struct pci_bus *bus)
>> diff --git a/include/linux/pci.h b/include/linux/pci.h
>> index 9661c85..f66d188 100644
>> --- a/include/linux/pci.h
>> +++ b/include/linux/pci.h
>> @@ -1390,7 +1390,12 @@ static inline int pci_domain_nr(struct pci_bus *bus)
>>   {
>>   	return bus->domain_nr;
>>   }
>> +/* Arch specific ACPI hook to set-up domain number */
>> +#ifdef CONFIG_ACPI
>> +int acpi_pci_bus_domain_nr(struct pci_bus *bus);
>
> Technically speaking, this should be introduced in a separate patch;
> given that I know it is a temporary plaster and no other architecture will
> have to implement it I reckon it is fine to leave it here, I will make
> it disappear as fast as I can.
>
> We can argue if it is best to move the ACPI bits into a separate file
> (acpi-pci.c), I am fine with keeping DT/ACPI in one file if that's fine
> with Catalin and Will.
>
> You can send the fix-ups above in the final pull request which I hope
> is nigh.
>
> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Thanks,
Tomasz

Re: [PATCH V8 8/9] arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus enumeration.

[Lorenzo Pieralisi]

On Mon, May 30, 2016 at 05:14:21PM +0200, Tomasz Nowicki wrote:
> ACPI requires to run acpi_pci_{add|remove}_bus while new PCI bus is created.
> This allows to do some ACPI-specific additional configuration, like
> PCI hotplug slot enumeration. In order to fulfill these requirements,
> we implement arch-specific pcibios_{add|remove}_bus calls
> and call acpi_pci_{add|remove}_bus from there.
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> ---
>  arch/arm64/kernel/pci.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)

Is there a reason why this has to be a standalone patch ?

It is pretty much useless without patch 9 or I have stared at
this series for too long.

Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> index 336deef..3663be1 100644
> --- a/arch/arm64/kernel/pci.c
> +++ b/arch/arm64/kernel/pci.c
> @@ -17,6 +17,7 @@
>  #include <linux/mm.h>
>  #include <linux/of_pci.h>
>  #include <linux/of_platform.h>
> +#include <linux/pci-acpi.h>
>  #include <linux/slab.h>
>  
>  /*
> @@ -96,4 +97,15 @@ struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  	/* TODO: Should be revisited when implementing PCI on ACPI */
>  	return NULL;
>  }
> +
> +void pcibios_add_bus(struct pci_bus *bus)
> +{
> +	acpi_pci_add_bus(bus);
> +}
> +
> +void pcibios_remove_bus(struct pci_bus *bus)
> +{
> +	acpi_pci_remove_bus(bus);
> +}
> +
>  #endif
> -- 
> 1.9.1
> 

Re: [PATCH V8 1/9] PCI: ecam: move ecam.h to linux/include/pci-ecam.h

[Lorenzo Pieralisi]

On Mon, May 30, 2016 at 05:14:14PM +0200, Tomasz Nowicki wrote:
> From: Jayachandran C <jchandra@broadcom.com>
> 
> This header will be used from arch/arm64 for ACPI PCI implementation
> so it needs to be moved out of drivers/pci.
> 
> Update users of the header file to use the new name. No functional
> changes.
> 
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> ---
>  drivers/pci/ecam.c                  |  3 +-
>  drivers/pci/ecam.h                  | 67 -------------------------------------
>  drivers/pci/host/pci-host-common.c  |  3 +-
>  drivers/pci/host/pci-host-generic.c |  3 +-
>  drivers/pci/host/pci-thunder-ecam.c |  3 +-
>  drivers/pci/host/pci-thunder-pem.c  |  3 +-
>  include/linux/pci-ecam.h            | 67 +++++++++++++++++++++++++++++++++++++
>  7 files changed, 72 insertions(+), 77 deletions(-)
>  delete mode 100644 drivers/pci/ecam.h
>  create mode 100644 include/linux/pci-ecam.h

Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> 
> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> index f9832ad..820e26b 100644
> --- a/drivers/pci/ecam.c
> +++ b/drivers/pci/ecam.c
> @@ -19,10 +19,9 @@
>  #include <linux/kernel.h>
>  #include <linux/module.h>
>  #include <linux/pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/slab.h>
>  
> -#include "ecam.h"
> -
>  /*
>   * On 64-bit systems, we do a single ioremap for the whole config space
>   * since we have enough virtual address range available.  On 32-bit, we
> diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
> deleted file mode 100644
> index 9878beb..0000000
> --- a/drivers/pci/ecam.h
> +++ /dev/null
> @@ -1,67 +0,0 @@
> -/*
> - * Copyright 2016 Broadcom
> - *
> - * This program is free software; you can redistribute it and/or modify
> - * it under the terms of the GNU General Public License, version 2, as
> - * published by the Free Software Foundation (the "GPL").
> - *
> - * This program is distributed in the hope that it will be useful, but
> - * WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> - * General Public License version 2 (GPLv2) for more details.
> - *
> - * You should have received a copy of the GNU General Public License
> - * version 2 (GPLv2) along with this source code.
> - */
> -#ifndef DRIVERS_PCI_ECAM_H
> -#define DRIVERS_PCI_ECAM_H
> -
> -#include <linux/kernel.h>
> -#include <linux/platform_device.h>
> -
> -/*
> - * struct to hold pci ops and bus shift of the config window
> - * for a PCI controller.
> - */
> -struct pci_config_window;
> -struct pci_ecam_ops {
> -	unsigned int			bus_shift;
> -	struct pci_ops			pci_ops;
> -	int				(*init)(struct device *,
> -						struct pci_config_window *);
> -};
> -
> -/*
> - * struct to hold the mappings of a config space window. This
> - * is expected to be used as sysdata for PCI controllers that
> - * use ECAM.
> - */
> -struct pci_config_window {
> -	struct resource			res;
> -	struct resource			busr;
> -	void				*priv;
> -	struct pci_ecam_ops		*ops;
> -	union {
> -		void __iomem		*win;	/* 64-bit single mapping */
> -		void __iomem		**winp; /* 32-bit per-bus mapping */
> -	};
> -};
> -
> -/* create and free pci_config_window */
> -struct pci_config_window *pci_ecam_create(struct device *dev,
> -		struct resource *cfgres, struct resource *busr,
> -		struct pci_ecam_ops *ops);
> -void pci_ecam_free(struct pci_config_window *cfg);
> -
> -/* map_bus when ->sysdata is an instance of pci_config_window */
> -void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
> -			       int where);
> -/* default ECAM ops */
> -extern struct pci_ecam_ops pci_generic_ecam_ops;
> -
> -#ifdef CONFIG_PCI_HOST_GENERIC
> -/* for DT-based PCI controllers that support ECAM */
> -int pci_host_common_probe(struct platform_device *pdev,
> -			  struct pci_ecam_ops *ops);
> -#endif
> -#endif
> diff --git a/drivers/pci/host/pci-host-common.c b/drivers/pci/host/pci-host-common.c
> index 8cba7ab..c18b9e3 100644
> --- a/drivers/pci/host/pci-host-common.c
> +++ b/drivers/pci/host/pci-host-common.c
> @@ -20,10 +20,9 @@
>  #include <linux/module.h>
>  #include <linux/of_address.h>
>  #include <linux/of_pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>  
> -#include "../ecam.h"
> -
>  static int gen_pci_parse_request_of_pci_ranges(struct device *dev,
>  		       struct list_head *resources, struct resource **bus_range)
>  {
> diff --git a/drivers/pci/host/pci-host-generic.c b/drivers/pci/host/pci-host-generic.c
> index 6eaceab..f0ca6de 100644
> --- a/drivers/pci/host/pci-host-generic.c
> +++ b/drivers/pci/host/pci-host-generic.c
> @@ -23,10 +23,9 @@
>  #include <linux/module.h>
>  #include <linux/of_address.h>
>  #include <linux/of_pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>  
> -#include "../ecam.h"
> -
>  static struct pci_ecam_ops gen_pci_cfg_cam_bus_ops = {
>  	.bus_shift	= 16,
>  	.pci_ops	= {
> diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
> index 540d030..a9fc1c9 100644
> --- a/drivers/pci/host/pci-thunder-ecam.c
> +++ b/drivers/pci/host/pci-thunder-ecam.c
> @@ -11,10 +11,9 @@
>  #include <linux/ioport.h>
>  #include <linux/of_pci.h>
>  #include <linux/of.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>  
> -#include "../ecam.h"
> -
>  static void set_val(u32 v, int where, int size, u32 *val)
>  {
>  	int shift = (where & 3) * 8;
> diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
> index 9b8ab94..5020d3d 100644
> --- a/drivers/pci/host/pci-thunder-pem.c
> +++ b/drivers/pci/host/pci-thunder-pem.c
> @@ -18,10 +18,9 @@
>  #include <linux/module.h>
>  #include <linux/of_address.h>
>  #include <linux/of_pci.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>  
> -#include "../ecam.h"
> -
>  #define PEM_CFG_WR 0x28
>  #define PEM_CFG_RD 0x30
>  
> diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
> new file mode 100644
> index 0000000..9878beb
> --- /dev/null
> +++ b/include/linux/pci-ecam.h
> @@ -0,0 +1,67 @@
> +/*
> + * Copyright 2016 Broadcom
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation (the "GPL").
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 (GPLv2) for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 (GPLv2) along with this source code.
> + */
> +#ifndef DRIVERS_PCI_ECAM_H
> +#define DRIVERS_PCI_ECAM_H
> +
> +#include <linux/kernel.h>
> +#include <linux/platform_device.h>
> +
> +/*
> + * struct to hold pci ops and bus shift of the config window
> + * for a PCI controller.
> + */
> +struct pci_config_window;
> +struct pci_ecam_ops {
> +	unsigned int			bus_shift;
> +	struct pci_ops			pci_ops;
> +	int				(*init)(struct device *,
> +						struct pci_config_window *);
> +};
> +
> +/*
> + * struct to hold the mappings of a config space window. This
> + * is expected to be used as sysdata for PCI controllers that
> + * use ECAM.
> + */
> +struct pci_config_window {
> +	struct resource			res;
> +	struct resource			busr;
> +	void				*priv;
> +	struct pci_ecam_ops		*ops;
> +	union {
> +		void __iomem		*win;	/* 64-bit single mapping */
> +		void __iomem		**winp; /* 32-bit per-bus mapping */
> +	};
> +};
> +
> +/* create and free pci_config_window */
> +struct pci_config_window *pci_ecam_create(struct device *dev,
> +		struct resource *cfgres, struct resource *busr,
> +		struct pci_ecam_ops *ops);
> +void pci_ecam_free(struct pci_config_window *cfg);
> +
> +/* map_bus when ->sysdata is an instance of pci_config_window */
> +void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
> +			       int where);
> +/* default ECAM ops */
> +extern struct pci_ecam_ops pci_generic_ecam_ops;
> +
> +#ifdef CONFIG_PCI_HOST_GENERIC
> +/* for DT-based PCI controllers that support ECAM */
> +int pci_host_common_probe(struct platform_device *pdev,
> +			  struct pci_ecam_ops *ops);
> +#endif
> +#endif
> -- 
> 1.9.1
> 

Re: [PATCH V8 8/9] arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus enumeration.

[Tomasz Nowicki]

On 02.06.2016 11:45, Lorenzo Pieralisi wrote:
> On Mon, May 30, 2016 at 05:14:21PM +0200, Tomasz Nowicki wrote:
>> ACPI requires to run acpi_pci_{add|remove}_bus while new PCI bus is created.
>> This allows to do some ACPI-specific additional configuration, like
>> PCI hotplug slot enumeration. In order to fulfill these requirements,
>> we implement arch-specific pcibios_{add|remove}_bus calls
>> and call acpi_pci_{add|remove}_bus from there.
>>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> ---
>>   arch/arm64/kernel/pci.c | 12 ++++++++++++
>>   1 file changed, 12 insertions(+)
>
> Is there a reason why this has to be a standalone patch ?

No there is no good reason behind this I will squash it with 9.

>
> It is pretty much useless without patch 9 or I have stared at
> this series for too long.

Yes, it is useless w/o patch 9.

>
> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
>

Thanks,
Tomasz

RE: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Gabriele Paoloni]

Hi Tomasz

> -----Original Message-----
> From: Tomasz Nowicki [mailto:tn@semihalf.com]
> Sent: 02 June 2016 09:52
> To: Gabriele Paoloni; helgaas@kernel.org; arnd@arndb.de;
> will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org;
> hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com; okaya@codeaurora.org;
> jchandra@broadcom.com
> Cc: robert.richter@caviumnetworks.com; mw@semihalf.com;
> Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; Wangyijing;
> Suravee.Suthikulpanit@amd.com; msalter@redhat.com; linux-
> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux-
> acpi@vger.kernel.org; linux-kernel@vger.kernel.org; linaro-
> acpi@lists.linaro.org; jcm@redhat.com; andrea.gallo@linaro.org;
> dhdang@apm.com; jeremy.linton@arm.com; liudongdong (C);
> cov@codeaurora.org
> Subject: Re: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host
> controller
> 
> Hi Gabriele,
> 
> On 01.06.2016 09:36, Gabriele Paoloni wrote:
> > I think maybe it is worth to post this quirk mechanism as RFC on
> > top of this v8 patchset, so that we can start to review it and
> > make some progress on the quirks.
> 
> I've just posted my RFC patches, please see:
> https://patchwork.ozlabs.org/patch/629121/
> 
> We can start review from there so that this series is not blocked.

Many Thanks for this!

Sure let's start working on this.

Cheers

Gab

> 
> Thanks,
> Tomasz

RE: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Gabriele Paoloni]

Hi Jon

> -----Original Message-----
> From: Jon Masters [mailto:jcm@redhat.com]
> Sent: 02 June 2016 08:32
> To: Gabriele Paoloni; Tomasz Nowicki; helgaas@kernel.org;
> arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com;
> rafael@kernel.org; hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com;
> okaya@codeaurora.org; jchandra@broadcom.com
> Cc: robert.richter@caviumnetworks.com; mw@semihalf.com;
> Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; Wangyijing;
> Suravee.Suthikulpanit@amd.com; msalter@redhat.com; linux-
> pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux-
> acpi@vger.kernel.org; linux-kernel@vger.kernel.org; linaro-
> acpi@lists.linaro.org; jcm@redhat.com; andrea.gallo@linaro.org;
> dhdang@apm.com; jeremy.linton@arm.com
> Subject: Re: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host
> controller
> 
> On 06/01/2016 03:36 AM, Gabriele Paoloni wrote:
> 
> > If you agree I think Jon can tell who's the best person to
> > push the quirk RFC (as my understanding is that this mechanism
> > is currently used by some platforms deployed on the market...)
> 
> Let me ping Linaro folks to see who has that (quirks) ball. We can
> certainly share the older OEM matching quirks Mark Salter did for
> earlier RHEL(SA) internal versions as a seed for that[0] activity.

Tomasz has posted the RFC so we'll start to look at that

> 
> BUT...I don't think we should block this thread on the quirks. They're
> separate (but important). I see Arnd's reply as well, but nobody else
> has yet chimed in to this thread (and I am about to prod all of the
> vendors to reply to this thread and ACK). Can I ask whether we can't
> just stage v8 as-is for -next at this point? Can Arnd's (or other)
> suggestions be handled as followup patches post-merge please? Bjorn?

I will ask HiSilicon folks to test it so we can add Tested-by on this

Thanks

Gab

> 
> Jon.
> 
> [0] RHEL(SA) has had three different PCIe enabled ACPI stacks
> maintained
> independently so far since we need PCIe to boot most of the hundreds of
> v8 systems we have internal to Red Hat - which all have PCI.
> 
> --
> Computer Architect | Sent from my Fedora powered laptop

Re: [PATCH V8 2/9] PCI: ecam: Add parent device field to pci_config_window

[Lorenzo Pieralisi]

On Mon, May 30, 2016 at 05:14:15PM +0200, Tomasz Nowicki wrote:
> From: Jayachandran C <jchandra@broadcom.com>
> 
> Add a parent device field to struct pci_config_window. The parent
> is not saved now, but will be useful to save it in some cases.
> Specifically in case of ACPI for ARM64, it can be used to setup
> ACPI companion and domain.
> 
> Since the parent dev is in struct pci_config_window now, we need
> not pass it to he init function as a separate argument.

Nit: "the init function"

> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> ---
>  drivers/pci/ecam.c                 | 3 ++-
>  drivers/pci/host/pci-thunder-pem.c | 3 ++-
>  include/linux/pci-ecam.h           | 4 ++--
>  3 files changed, 6 insertions(+), 4 deletions(-)

Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> index 820e26b..66e0d71 100644
> --- a/drivers/pci/ecam.c
> +++ b/drivers/pci/ecam.c
> @@ -51,6 +51,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
>  	if (!cfg)
>  		return ERR_PTR(-ENOMEM);
>  
> +	cfg->parent = dev;
>  	cfg->ops = ops;
>  	cfg->busr.start = busr->start;
>  	cfg->busr.end = busr->end;
> @@ -94,7 +95,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
>  	}
>  
>  	if (ops->init) {
> -		err = ops->init(dev, cfg);
> +		err = ops->init(cfg);
>  		if (err)
>  			goto err_exit;
>  	}
> diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
> index 5020d3d..91f6fc6 100644
> --- a/drivers/pci/host/pci-thunder-pem.c
> +++ b/drivers/pci/host/pci-thunder-pem.c
> @@ -284,8 +284,9 @@ static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn,
>  	return pci_generic_config_write(bus, devfn, where, size, val);
>  }
>  
> -static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
> +static int thunder_pem_init(struct pci_config_window *cfg)
>  {
> +	struct device *dev = cfg->parent;
>  	resource_size_t bar4_start;
>  	struct resource *res_pem;
>  	struct thunder_pem_pci *pem_pci;
> diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
> index 9878beb..7adad20 100644
> --- a/include/linux/pci-ecam.h
> +++ b/include/linux/pci-ecam.h
> @@ -27,8 +27,7 @@ struct pci_config_window;
>  struct pci_ecam_ops {
>  	unsigned int			bus_shift;
>  	struct pci_ops			pci_ops;
> -	int				(*init)(struct device *,
> -						struct pci_config_window *);
> +	int				(*init)(struct pci_config_window *);
>  };
>  
>  /*
> @@ -45,6 +44,7 @@ struct pci_config_window {
>  		void __iomem		*win;	/* 64-bit single mapping */
>  		void __iomem		**winp; /* 32-bit per-bus mapping */
>  	};
> +	struct device			*parent;/* ECAM res was from this dev */
>  };
>  
>  /* create and free pci_config_window */
> -- 
> 1.9.1
> 

Re: [PATCH V8 3/9] pci: Add new function to unmap IO resources.

[Lorenzo Pieralisi]

On Mon, May 30, 2016 at 05:14:16PM +0200, Tomasz Nowicki wrote:
> We need to release I/O resources so that the same I/O resources
> can be allocated again (pci_remap_iospace), like in PCI hotplug removal
> scenario. Therefore this patch implements new pci_unmap_iospace call which
> unmaps I/O space as the symmetry to pci_remap_iospace.
> 
> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> ---
>  drivers/pci/pci.c   | 18 ++++++++++++++++++
>  include/linux/pci.h |  1 +
>  2 files changed, 19 insertions(+)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index c8b4dbd..eb431b5 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -25,6 +25,7 @@
>  #include <linux/device.h>
>  #include <linux/pm_runtime.h>
>  #include <linux/pci_hotplug.h>
> +#include <linux/vmalloc.h>
>  #include <asm/setup.h>
>  #include <linux/aer.h>
>  #include "pci.h"
> @@ -3165,6 +3166,23 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
>  #endif
>  }
>  
> +/**
> + *	pci_unmap_iospace - Unmap the memory mapped I/O space
> + *	@res: resource to be unmapped
> + *
> + *	Unmap the CPU virtual address @res from virtual address space.
> + *	Only architectures that have memory mapped IO functions defined
> + *	(and the PCI_IOBASE value defined) should call this function.
> + */
> +void pci_unmap_iospace(struct resource *res)
> +{
> +#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
> +	unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
> +
> +	unmap_kernel_range(vaddr, resource_size(res));
> +#endif
> +}

Rafael mentioned that, it would be cleaner to move the ifdef outside
the function and add an empty static inline (but that probably should
be done in one go along with pci_remap_iospace() and
pci_pio_to_address() and pci_address_to_pio()) it would be weird
to do it just for pci_unmap_iospace().

It is true that unmap_kernel_range() is not commonly used in generic
code, but to the best of my knowledge it does what's required here,
(ie clear page table entries and flush tlbs, given that it is used
in conjuction with ioremap_page_range() that maps a physical address
to a specific preallocated virtual address ie PCI_IOBASE), so:

Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> +
>  static void __pci_set_master(struct pci_dev *dev, bool enable)
>  {
>  	u16 old_cmd, cmd;
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index b67e4df..12349de 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1167,6 +1167,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size);
>  unsigned long pci_address_to_pio(phys_addr_t addr);
>  phys_addr_t pci_pio_to_address(unsigned long pio);
>  int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
> +void pci_unmap_iospace(struct resource *res);
>  
>  static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
>  {
> -- 
> 1.9.1
> 

Re: [PATCH V8 4/9] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Lorenzo Pieralisi]

On Mon, May 30, 2016 at 05:14:17PM +0200, Tomasz Nowicki wrote:
> Platforms that have memory mapped IO port (such as ARM64) need special
> handling for PCI I/O resources. For host bridge's resource probing case
> these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
> 
> The same I/O resources need to be released after hotplug
> removal so that it can be re-added back by the pci_remap_iospace
> function during insertion. As a consequence we unmap I/O resources
> with pci_unmap_iospace when we release host bridge resources.
> 
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> ---
>  drivers/acpi/pci_root.c | 39 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 39 insertions(+)

I will take a rain-check on ACPI PCI IO space consolidation between
IA64 and ARM64.

> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> index ae3fe4e..ee3b728 100644
> --- a/drivers/acpi/pci_root.c
> +++ b/drivers/acpi/pci_root.c
> @@ -720,6 +720,40 @@ next:
>  	}
>  }
>  
> +#ifdef PCI_IOBASE
> +static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
> +{
> +	struct resource *res = entry->res;
> +	resource_size_t cpu_addr = res->start;
> +	resource_size_t pci_addr = cpu_addr - entry->offset;
> +	resource_size_t length = resource_size(res);
> +	unsigned long port;
> +
> +	if (pci_register_io_range(cpu_addr, length))
> +		goto err;
> +
> +	port = pci_address_to_pio(cpu_addr);
> +	if (port == (unsigned long)-1)
> +		goto err;
> +
> +	res->start = port;
> +	res->end = port + length - 1;
> +	entry->offset = port - pci_addr;
> +
> +	if (pci_remap_iospace(res, cpu_addr) < 0)
> +		goto err;
> +
> +	pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);

Side note: we need to allocate and request a resource for the corresponding
memory resource (and do that for DT and ACPI alike), see above.

Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> +	return;
> +err:
> +	res->flags |= IORESOURCE_DISABLED;
> +}
> +#else
> +static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
> +{
> +}
> +#endif
> +
>  int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
>  {
>  	int ret;
> @@ -740,6 +774,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
>  			"no IO and memory resources present in _CRS\n");
>  	else {
>  		resource_list_for_each_entry_safe(entry, tmp, list) {
> +			if (entry->res->flags & IORESOURCE_IO)
> +				acpi_pci_root_remap_iospace(entry);
> +
>  			if (entry->res->flags & IORESOURCE_DISABLED)
>  				resource_list_destroy_entry(entry);
>  			else
> @@ -811,6 +848,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
>  
>  	resource_list_for_each_entry(entry, &bridge->windows) {
>  		res = entry->res;
> +		if (res->flags & IORESOURCE_IO)
> +			pci_unmap_iospace(res);
>  		if (res->parent &&
>  		    (res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
>  			release_resource(res);
> -- 
> 1.9.1
> 

Re: [PATCH V8 7/9] acpi: Add generic MCFG table handling

[Lorenzo Pieralisi]

On Mon, May 30, 2016 at 05:14:20PM +0200, Tomasz Nowicki wrote:
> In order to handle PCI config space regions properly in ACPI, new MCFG
> interface is defined which does sanity checks on MCFG table and keeps its
> root pointer. The user is able to lookup MCFG regions based on
> host bridge root structure and domain:bus_start:bus_end touple.
> Use pci_mmcfg_late_init old prototype to avoid another function name.

"According to PCI firmware specifications, on systems booting with ACPI,
PCI configuration for a host bridge must be set-up through the MCFG table
regions for non-hotpluggable bridges and _CBA method for hotpluggable ones.

Current MCFG table handling code, as implemented for x86, cannot be
easily generalized owing to x86 specific quirks handling and related
code, which makes it hard to reuse on other architectures.

In order to implement MCFG PCI configuration handling for new platforms
booting with ACPI (eg ARM64) this patch re-implements MCFG handling from
scratch in a streamlined fashion and provides (through a generic
interface available to all arches):

- Simplified MCFG table parsing (executed through the pci_mmcfg_late_init()
  hook as in current x86)
- MCFG regions look-up interface through domain:bus_start:bus_end tuple

The new MCFG regions handling interface is added to generic ACPI code
so that existing architectures (eg x86) can be moved over to it and
architectures relying on MCFG for ACPI PCI config space can rely on it
without having to resort to arch specific implementations."

> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> ---
>  drivers/acpi/Kconfig     |  3 ++
>  drivers/acpi/Makefile    |  1 +
>  drivers/acpi/pci_mcfg.c  | 94 ++++++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/pci-acpi.h |  2 ++
>  include/linux/pci.h      |  2 +-
>  5 files changed, 101 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/acpi/pci_mcfg.c
> 
> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
> index b7e2e77..f98c328 100644
> --- a/drivers/acpi/Kconfig
> +++ b/drivers/acpi/Kconfig
> @@ -217,6 +217,9 @@ config ACPI_PROCESSOR_IDLE
>  	bool
>  	select CPU_IDLE
>  
> +config ACPI_MCFG
> +	bool
> +
>  config ACPI_CPPC_LIB
>  	bool
>  	depends on ACPI_PROCESSOR
> diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
> index 251ce85..632e81f 100644
> --- a/drivers/acpi/Makefile
> +++ b/drivers/acpi/Makefile
> @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
>  acpi-y				+= ec.o
>  acpi-$(CONFIG_ACPI_DOCK)	+= dock.o
>  acpi-y				+= pci_root.o pci_link.o pci_irq.o
> +obj-$(CONFIG_ACPI_MCFG)		+= pci_mcfg.o
>  acpi-y				+= acpi_lpss.o acpi_apd.o
>  acpi-y				+= acpi_platform.o
>  acpi-y				+= acpi_pnp.o
> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> new file mode 100644
> index 0000000..1847f74
> --- /dev/null
> +++ b/drivers/acpi/pci_mcfg.c
> @@ -0,0 +1,94 @@
> +/*
> + * Copyright (C) 2016 Broadcom
> + *	Author: Jayachandran C <jchandra@broadcom.com>
> + * Copyright (C) 2016 Semihalf
> + * 	Author: Tomasz Nowicki <tn@semihalf.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation (the "GPL").
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 (GPLv2) for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 (GPLv2) along with this source code.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: " fmt
> +
> +#include <linux/kernel.h>
> +#include <linux/pci.h>
> +#include <linux/pci-acpi.h>
> +
> +/* Root pointer to the mapped MCFG table */
> +static struct acpi_table_mcfg *mcfg_table;
> +static int mcfg_entries;
> +
> +int pci_mcfg_lookup(struct acpi_pci_root *root)
> +{
> +	struct acpi_mcfg_allocation *mptr, *entry = NULL;
> +	struct resource *bus_res = &root->secondary;
> +	int i;
> +
> +	if (mcfg_table) {
> +		mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
> +		for (i = 0; i < mcfg_entries && !entry; i++, mptr++)
> +			if (mptr->pci_segment == root->segment &&
> +			    mptr->start_bus_number == bus_res->start)
> +				entry = mptr;
> +	}
> +
> +	/* not found, use _CBA if available, else error */
> +	if (!entry) {
> +		if (root->mcfg_addr)
> +			return root->mcfg_addr;
> +		pr_err("%04x:%pR MCFG lookup failed\n", root->segment, bus_res);
> +		return -ENOENT;
> +	} else if (root->mcfg_addr && entry->address != root->mcfg_addr) {
> +		pr_warn("%04x:%pR CBA %pa != MCFG %lx, using CBA\n",
> +			root->segment, bus_res, &root->mcfg_addr,
> +			(unsigned long)entry->address);
> +		return 0;
> +	}
> +
> +	/* found matching entry, bus range check */
> +	if (entry->end_bus_number != bus_res->end) {
> +		resource_size_t bus_end = min_t(resource_size_t,
> +					entry->end_bus_number, bus_res->end);
> +		pr_warn("%04x:%pR bus end mismatch, using %02lx\n",
> +			root->segment, bus_res, (unsigned long)bus_end);
> +		bus_res->end = bus_end;
> +	}
> +
> +	if (!root->mcfg_addr)
> +		root->mcfg_addr = entry->address;

Let's hope that no one will ever implement a hotplug bridge with config
space starting at physical address 0x0.

Nit: You should define what the return value means. For success, once you
return the _CBA address, once 0; this should be consistent.

Anyway, this function is not easy to read but it may well be fine, I will let
Bjorn decide what to do with corner cases:

a) _CBA is != 0 and you get a MCFG entry that matches its address (I am
    not sure that capping the _CRS bus numbers is PCI compliant in that case)
b) bus_end capping, either you leave code as-is (that caps also _CRS) or
   just warn and fail if the bus->end numbers mismatch

Pending Bjorn's opinion on the above (and commit log update):

Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

> +	return 0;
> +}
> +
> +static __init int pci_mcfg_parse(struct acpi_table_header *header)
> +{
> +	if (header->length < sizeof(struct acpi_table_mcfg))
> +		return -EINVAL;
> +
> +	mcfg_entries = (header->length - sizeof(struct acpi_table_mcfg)) /
> +					sizeof(struct acpi_mcfg_allocation);
> +	if (mcfg_entries == 0) {
> +		pr_err("MCFG has no entries\n");
> +		return -EINVAL;
> +	}
> +
> +	mcfg_table = (struct acpi_table_mcfg *)header;
> +	pr_info("MCFG table detected, %d entries\n", mcfg_entries);
> +	return 0;
> +}
> +
> +/* Interface called by ACPI - parse and save MCFG table */
> +void __init pci_mmcfg_late_init(void)
> +{
> +	int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
> +	if (err)
> +		pr_err("Failed to parse MCFG (%d)\n", err);
> +}
> diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> index 89ab057..e0e6dfc 100644
> --- a/include/linux/pci-acpi.h
> +++ b/include/linux/pci-acpi.h
> @@ -24,6 +24,8 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
>  }
>  extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
>  
> +extern int pci_mcfg_lookup(struct acpi_pci_root *root);
> +
>  static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
>  {
>  	struct pci_bus *pbus = pdev->bus;
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index bba4053..9661c85 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1724,7 +1724,7 @@ void pcibios_free_irq(struct pci_dev *dev);
>  extern struct dev_pm_ops pcibios_pm_ops;
>  #endif
>  
> -#ifdef CONFIG_PCI_MMCONFIG
> +#if defined(CONFIG_PCI_MMCONFIG) || defined(CONFIG_ACPI_MCFG)
>  void __init pci_mmcfg_early_init(void);
>  void __init pci_mmcfg_late_init(void);
>  #else
> -- 
> 1.9.1
> 

Re: [PATCH V8 7/9] acpi: Add generic MCFG table handling

[Tomasz Nowicki]

On 03.06.2016 13:38, Lorenzo Pieralisi wrote:
> On Mon, May 30, 2016 at 05:14:20PM +0200, Tomasz Nowicki wrote:
>> In order to handle PCI config space regions properly in ACPI, new MCFG
>> interface is defined which does sanity checks on MCFG table and keeps its
>> root pointer. The user is able to lookup MCFG regions based on
>> host bridge root structure and domain:bus_start:bus_end touple.
>> Use pci_mmcfg_late_init old prototype to avoid another function name.
>
> "According to PCI firmware specifications, on systems booting with ACPI,
> PCI configuration for a host bridge must be set-up through the MCFG table
> regions for non-hotpluggable bridges and _CBA method for hotpluggable ones.
>
> Current MCFG table handling code, as implemented for x86, cannot be
> easily generalized owing to x86 specific quirks handling and related
> code, which makes it hard to reuse on other architectures.
>
> In order to implement MCFG PCI configuration handling for new platforms
> booting with ACPI (eg ARM64) this patch re-implements MCFG handling from
> scratch in a streamlined fashion and provides (through a generic
> interface available to all arches):
>
> - Simplified MCFG table parsing (executed through the pci_mmcfg_late_init()
>    hook as in current x86)
> - MCFG regions look-up interface through domain:bus_start:bus_end tuple
>
> The new MCFG regions handling interface is added to generic ACPI code
> so that existing architectures (eg x86) can be moved over to it and
> architectures relying on MCFG for ACPI PCI config space can rely on it
> without having to resort to arch specific implementations."
>

[...]

>> +
>> +#include <linux/kernel.h>
>> +#include <linux/pci.h>
>> +#include <linux/pci-acpi.h>
>> +
>> +/* Root pointer to the mapped MCFG table */
>> +static struct acpi_table_mcfg *mcfg_table;
>> +static int mcfg_entries;
>> +
>> +int pci_mcfg_lookup(struct acpi_pci_root *root)
>> +{
>> +	struct acpi_mcfg_allocation *mptr, *entry = NULL;
>> +	struct resource *bus_res = &root->secondary;
>> +	int i;
>> +
>> +	if (mcfg_table) {
>> +		mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
>> +		for (i = 0; i < mcfg_entries && !entry; i++, mptr++)
>> +			if (mptr->pci_segment == root->segment &&
>> +			    mptr->start_bus_number == bus_res->start)
>> +				entry = mptr;
>> +	}
>> +
>> +	/* not found, use _CBA if available, else error */
>> +	if (!entry) {
>> +		if (root->mcfg_addr)
>> +			return root->mcfg_addr;
>> +		pr_err("%04x:%pR MCFG lookup failed\n", root->segment, bus_res);
>> +		return -ENOENT;
>> +	} else if (root->mcfg_addr && entry->address != root->mcfg_addr) {
>> +		pr_warn("%04x:%pR CBA %pa != MCFG %lx, using CBA\n",
>> +			root->segment, bus_res, &root->mcfg_addr,
>> +			(unsigned long)entry->address);
>> +		return 0;
>> +	}
>> +
>> +	/* found matching entry, bus range check */
>> +	if (entry->end_bus_number != bus_res->end) {
>> +		resource_size_t bus_end = min_t(resource_size_t,
>> +					entry->end_bus_number, bus_res->end);
>> +		pr_warn("%04x:%pR bus end mismatch, using %02lx\n",
>> +			root->segment, bus_res, (unsigned long)bus_end);
>> +		bus_res->end = bus_end;
>> +	}
>> +
>> +	if (!root->mcfg_addr)
>> +		root->mcfg_addr = entry->address;
>
> Let's hope that no one will ever implement a hotplug bridge with config
> space starting at physical address 0x0.
>
> Nit: You should define what the return value means. For success, once you
> return the _CBA address, once 0; this should be consistent.

As we decided to return CFG start address in root->mcfg_addr we should 
return 0 for the case (!entry) && (root->mcfg_addr). I'll fix it.

>
> Anyway, this function is not easy to read but it may well be fine, I will let
> Bjorn decide what to do with corner cases:
>
> a) _CBA is != 0 and you get a MCFG entry that matches its address (I am
>      not sure that capping the _CRS bus numbers is PCI compliant in that case)
> b) bus_end capping, either you leave code as-is (that caps also _CRS) or
>     just warn and fail if the bus->end numbers mismatch
>
> Pending Bjorn's opinion on the above (and commit log update):
>
> Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
>

Thanks,
Tomasz

Re: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Bjorn Helgaas]

Hi Tomasz,

On Mon, May 30, 2016 at 05:14:13PM +0200, Tomasz Nowicki wrote:
> From the functionality point of view this series may be split into the
> following logic parts:
> 1. Export ECAM API and add parent device to pci_config_window
> 2. Add IO resources handling to PCI core code
> 3. Support for generic domain assignment based on ACPI
> 4. New MCFG driver
> 5. Implement ARM64 ACPI based PCI host controller driver under arch/arm64/
> 
> Patches has been built on top of 4.7-rc1 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v8)

Trivial comments while I work on the rest:

  - Run "git log --oneline" on the files you change and make sure the
    subject lines match existing style.

  - Always capitalize "PCI" and "ACPI".

  - Use a capitalized verb as the first word after the "PCI: " prefix
    in a changelog subject.

  - Don't add a period at the end of a changelog subject.

  - Wrap changelogs so "git log" output fits in 80 columns.  I use
    "set textwidth=75" in vim.

  - Always use "()" after function names.

  - Don't write "This patch does ..." in a changelog.  It's obvious
    that the context is this patch, so write something like "Update
    users of the header file to use the new name."  I like it when
    there's a little bit of background followed by a short paragraph
    starting with a strong verb that tells me what the patch does.

I usually fix all this sort of stuff silently, but I'm slow and it scales
better if you do it.

Don't bother posting an update until I have a chance to look at the code
itself.  I'm sure (at least, I hope) I'll have more substantive comments
than these :)

Bjorn

Re: [PATCH V8 4/9] acpi, pci: Support IO resources when parsing PCI host bridge resources.

[Bjorn Helgaas]

On Mon, May 30, 2016 at 05:14:17PM +0200, Tomasz Nowicki wrote:
> Platforms that have memory mapped IO port (such as ARM64) need special
> handling for PCI I/O resources. For host bridge's resource probing case
> these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
> 
> The same I/O resources need to be released after hotplug
> removal so that it can be re-added back by the pci_remap_iospace
> function during insertion. As a consequence we unmap I/O resources
> with pci_unmap_iospace when we release host bridge resources.
> 
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> ---
>  drivers/acpi/pci_root.c | 39 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 39 insertions(+)
> 
> diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
> index ae3fe4e..ee3b728 100644
> --- a/drivers/acpi/pci_root.c
> +++ b/drivers/acpi/pci_root.c
> @@ -720,6 +720,40 @@ next:
>  	}
>  }
>  
> +#ifdef PCI_IOBASE
> +static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
> +{
> +	struct resource *res = entry->res;
> +	resource_size_t cpu_addr = res->start;
> +	resource_size_t pci_addr = cpu_addr - entry->offset;
> +	resource_size_t length = resource_size(res);
> +	unsigned long port;
> +
> +	if (pci_register_io_range(cpu_addr, length))
> +		goto err;
> +
> +	port = pci_address_to_pio(cpu_addr);
> +	if (port == (unsigned long)-1)
> +		goto err;
> +
> +	res->start = port;
> +	res->end = port + length - 1;
> +	entry->offset = port - pci_addr;
> +
> +	if (pci_remap_iospace(res, cpu_addr) < 0)
> +		goto err;
> +
> +	pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
> +	return;
> +err:
> +	res->flags |= IORESOURCE_DISABLED;

Trivial, but I might write this as:

    res->flags |= IORESOURCE_DISABLED;
    if (pci_register_io_range(cpu_addr, length))
      return;
    ...
    if (pci_remap_iospace(res, cpu_addr) < 0)
      return;

    res->flags &= ~IORESOURCE_DISABLED;
    pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
  }

to get rid of the gotos.  But I dunno, it's a little ugly to fiddle
with res->flags in the case where everything succeeds.

> +}
> +#else
> +static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
> +{
> +}
> +#endif

Rafael's code here, and I know he had an earlier comment about
in-function #ifdefs, so I don't know what he thinks here.  To me,
it would be simpler to move this ifdef inside so we don't have to
repeat the function signature.

Re: [PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Bjorn Helgaas]

Hi Tomasz,

On Mon, May 30, 2016 at 05:14:18PM +0200, Tomasz Nowicki wrote:
> PCI core code provides a config option (CONFIG_PCI_DOMAINS_GENERIC)
> that allows assigning the PCI bus domain number generically by
> relying on device tree bindings, and falling back to a simple counter
> when the respective DT properties (ie "linux,pci-domain") are not
> specified in the host bridge device tree node.
> 
> In a similar way, when a system is booted through ACPI, architectures
> that are selecting CONFIG_PCI_DOMAINS_GENERIC (ie ARM64) require kernel
> hooks to retrieve the domain number so that the PCI bus domain number
> set-up can be handled seamlessly with DT and ACPI in generic core code
> when CONFIG_PCI_DOMAINS_GENERIC is selected.
> 
> Since currently it is not possible to retrieve a pointer to the PCI
> host bridge ACPI device backing the host bridge from core PCI code
> (which would allow retrieving the domain number in an arch agnostic
> way through the ACPI _SEG method), an arch specific ACPI hook has to
> be declared and implemented by all arches that rely on
> CONFIG_PCI_DOMAINS_GENERIC to retrieve the domain number and set it
> up in core PCI code.
> 
> For the aforementioned reasons, this patch introduces a dummy
> acpi_pci_bus_domain_nr() hook in preparation for per-arch implementation
> of the same to retrieve the domain number on a per-arch basis when
> the system boots through ACPI.
> 
> For the sake of code clarity the current code implementing generic
> domain number assignment (ie pci_bus_assign_domain_nr(), selected by
> CONFIG_PCI_DOMAINS_GENERIC) is reshuffled so that the code implementing
> the DT domain assignment function is stubbed out into a corresponding
> helper, so that DT and ACPI functions are clearly separated in
> preparation for arches acpi_pci_bus_domain_nr() implementations.
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> ---
>  drivers/pci/pci.c   | 11 +++++++++--
>  include/linux/pci.h |  1 +
>  2 files changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index eb431b5..2b52178 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -7,6 +7,7 @@
>   *	Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
>   */
>  
> +#include <linux/acpi.h>
>  #include <linux/kernel.h>
>  #include <linux/delay.h>
>  #include <linux/init.h>
> @@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
>  }
>  
>  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> +static int of_pci_bus_domain_nr(struct device *parent)

Can we do a little cleanup before this patch?  

  - pci_bus_assign_domain_nr() is only used inside drivers/pci, so
    maybe we move the prototype to drivers/pci/pci.h?

  - I don't really like the style of calling a function that
    internally assigns bus->domain_nr.  Could we do something like
    this instead?

    int pci_bus_domain_nr(...)
    {
      ...
      return domain;
    }

    ... pci_create_root_bus(...)
    {
      ...
      b->domain_nr = pci_bus_domain_nr(...);

That would be two new patches, if this makes sense.

And this patch would only rename pci_bus_assign_domain_nr() to
of_pci_bus_domain_nr() and add the pci_bus_domain_nr() wrapper.

>  {
>  	static int use_dt_domains = -1;
>  	int domain = -1;
> @@ -4985,7 +4986,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
>  		domain = -1;
>  	}
>  
> -	bus->domain_nr = domain;
> +	return domain;
> +}
> +
> +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> +{
> +	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
> +					 acpi_pci_bus_domain_nr(bus);
>  }
>  #endif
>  #endif
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 12349de..bba4053 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1390,6 +1390,7 @@ static inline int pci_domain_nr(struct pci_bus *bus)
>  {
>  	return bus->domain_nr;
>  }
> +static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return -1; }

I would split the addition of acpi_pci_bus_domain_nr() to a separate
patch and include the ARM64 definition in that same patch.  That patch
would only add this stub definition, the ARM64 definition, and the new
call in pci_bus_domain_nr().

>  void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent);
>  #else
>  static inline void pci_bus_assign_domain_nr(struct pci_bus *bus,
> -- 
> 1.9.1
> 

Re: [PATCH V8 7/9] acpi: Add generic MCFG table handling

[Bjorn Helgaas]

On Mon, May 30, 2016 at 05:14:20PM +0200, Tomasz Nowicki wrote:
> In order to handle PCI config space regions properly in ACPI, new MCFG
> interface is defined which does sanity checks on MCFG table and keeps its
> root pointer. The user is able to lookup MCFG regions based on
> host bridge root structure and domain:bus_start:bus_end touple.
> Use pci_mmcfg_late_init old prototype to avoid another function name.
> 
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> ---
>  drivers/acpi/Kconfig     |  3 ++
>  drivers/acpi/Makefile    |  1 +
>  drivers/acpi/pci_mcfg.c  | 94 ++++++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/pci-acpi.h |  2 ++
>  include/linux/pci.h      |  2 +-
>  5 files changed, 101 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/acpi/pci_mcfg.c
> 
> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
> index b7e2e77..f98c328 100644
> --- a/drivers/acpi/Kconfig
> +++ b/drivers/acpi/Kconfig
> @@ -217,6 +217,9 @@ config ACPI_PROCESSOR_IDLE
>  	bool
>  	select CPU_IDLE
>  
> +config ACPI_MCFG
> +	bool
> +
>  config ACPI_CPPC_LIB
>  	bool
>  	depends on ACPI_PROCESSOR
> diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
> index 251ce85..632e81f 100644
> --- a/drivers/acpi/Makefile
> +++ b/drivers/acpi/Makefile
> @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
>  acpi-y				+= ec.o
>  acpi-$(CONFIG_ACPI_DOCK)	+= dock.o
>  acpi-y				+= pci_root.o pci_link.o pci_irq.o
> +obj-$(CONFIG_ACPI_MCFG)		+= pci_mcfg.o
>  acpi-y				+= acpi_lpss.o acpi_apd.o
>  acpi-y				+= acpi_platform.o
>  acpi-y				+= acpi_pnp.o
> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> new file mode 100644
> index 0000000..1847f74
> --- /dev/null
> +++ b/drivers/acpi/pci_mcfg.c
> @@ -0,0 +1,94 @@
> +/*
> + * Copyright (C) 2016 Broadcom
> + *	Author: Jayachandran C <jchandra@broadcom.com>
> + * Copyright (C) 2016 Semihalf
> + * 	Author: Tomasz Nowicki <tn@semihalf.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation (the "GPL").
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License version 2 (GPLv2) for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * version 2 (GPLv2) along with this source code.
> + */
> +
> +#define pr_fmt(fmt) "ACPI: " fmt
> +
> +#include <linux/kernel.h>
> +#include <linux/pci.h>
> +#include <linux/pci-acpi.h>
> +
> +/* Root pointer to the mapped MCFG table */
> +static struct acpi_table_mcfg *mcfg_table;
> +static int mcfg_entries;
> +
> +int pci_mcfg_lookup(struct acpi_pci_root *root)

I think this would be better if we passed in the domain and a pointer
to the bus range resource and returned the ECAM base address.  I don't
think we need to be connected to struct acpi_pci_root.

> +{
> +	struct acpi_mcfg_allocation *mptr, *entry = NULL;
> +	struct resource *bus_res = &root->secondary;
> +	int i;
> +
> +	if (mcfg_table) {
> +		mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
> +		for (i = 0; i < mcfg_entries && !entry; i++, mptr++)
> +			if (mptr->pci_segment == root->segment &&
> +			    mptr->start_bus_number == bus_res->start)
> +				entry = mptr;
> +	}
> +
> +	/* not found, use _CBA if available, else error */
> +	if (!entry) {
> +		if (root->mcfg_addr)
> +			return root->mcfg_addr;
> +		pr_err("%04x:%pR MCFG lookup failed\n", root->segment, bus_res);
> +		return -ENOENT;
> +	} else if (root->mcfg_addr && entry->address != root->mcfg_addr) {
> +		pr_warn("%04x:%pR CBA %pa != MCFG %lx, using CBA\n",
> +			root->segment, bus_res, &root->mcfg_addr,
> +			(unsigned long)entry->address);
> +		return 0;
> +	}

I keep looking at this code, trying to find where we "use _CBA", but I
can't find it.  Oh, I see, acpi_pci_root_add() calls
acpi_pci_root_get_mcfg_addr() (which evaluates _CBA), and sets
root->mcfg_addr to the result.

"root->mcfg_addr" is sort of an unfortunate name because "MCFG" is the
ACPI table name, we've used "ECAM" for the memory-mapped config space,
and this address can come from either the MCFG table or the _CBA
method.

In the case where we have both _CBA and an MCFG entry, I think we
should prefer _CBA, and I'm not sure it's even worth warning about it.
So I think you could simplify this function if you made
pci_acpi_setup_ecam_mapping() look like this:

  ... pci_acpi_setup_ecam_mapping(...)
  {
    ...
    if (!root->mcfg_addr)
      root->mcfg_addr = pci_mcfg_lookup(root);

    if (!root->mcfg_addr) {
      dev_err(..., "no ECAM region found\n");
      return -EINVAL;
    }

> +
> +	/* found matching entry, bus range check */
> +	if (entry->end_bus_number != bus_res->end) {
> +		resource_size_t bus_end = min_t(resource_size_t,
> +					entry->end_bus_number, bus_res->end);
> +		pr_warn("%04x:%pR bus end mismatch, using %02lx\n",
> +			root->segment, bus_res, (unsigned long)bus_end);
> +		bus_res->end = bus_end;
> +	}
> +
> +	if (!root->mcfg_addr)
> +		root->mcfg_addr = entry->address;

Please move the assignment to the caller (I think Lorenzo pointed this
out already).

> +	return 0;
> +}
> +
> +static __init int pci_mcfg_parse(struct acpi_table_header *header)
> +{
> +	if (header->length < sizeof(struct acpi_table_mcfg))
> +		return -EINVAL;
> +
> +	mcfg_entries = (header->length - sizeof(struct acpi_table_mcfg)) /
> +					sizeof(struct acpi_mcfg_allocation);
> +	if (mcfg_entries == 0) {
> +		pr_err("MCFG has no entries\n");

Include an address here?  I'm not really sure either of the messages
here is necessary.  Users (callers of pci_mcfg_lookup()) will notice
if we can't find any ECAM space and will probably complain there,
where the message can include more information, e.g., the affected
device.

> +		return -EINVAL;
> +	}
> +
> +	mcfg_table = (struct acpi_table_mcfg *)header;
> +	pr_info("MCFG table detected, %d entries\n", mcfg_entries);
> +	return 0;
> +}
> +
> +/* Interface called by ACPI - parse and save MCFG table */

I think we save a *pointer* to the MCFG table, not the table itself.

And acpi_table_parse() calls early_acpi_os_unmap_memory() immediately
after it calls pci_mcfg_parse(), so I'm doubtful that the pointer
remains valid.

> +void __init pci_mmcfg_late_init(void)
> +{
> +	int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
> +	if (err)
> +		pr_err("Failed to parse MCFG (%d)\n", err);
> +}
> diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
> index 89ab057..e0e6dfc 100644
> --- a/include/linux/pci-acpi.h
> +++ b/include/linux/pci-acpi.h
> @@ -24,6 +24,8 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
>  }
>  extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
>  
> +extern int pci_mcfg_lookup(struct acpi_pci_root *root);
> +
>  static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
>  {
>  	struct pci_bus *pbus = pdev->bus;
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index bba4053..9661c85 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1724,7 +1724,7 @@ void pcibios_free_irq(struct pci_dev *dev);
>  extern struct dev_pm_ops pcibios_pm_ops;
>  #endif
>  
> -#ifdef CONFIG_PCI_MMCONFIG
> +#if defined(CONFIG_PCI_MMCONFIG) || defined(CONFIG_ACPI_MCFG)
>  void __init pci_mmcfg_early_init(void);
>  void __init pci_mmcfg_late_init(void);
>  #else
> -- 
> 1.9.1
> 

Re: [PATCH V8 9/9] pci, acpi: ARM64 support for ACPI based generic PCI host controller

[Bjorn Helgaas]

On Mon, May 30, 2016 at 05:14:22PM +0200, Tomasz Nowicki wrote:
> This patch implements pci_acpi_scan_root call so that ARM64 can start
> using ACPI to setup and enumerate PCI buses.
> 
> The implementation of pci_acpi_scan_root() looks up config space regions
> through MCFG interface. Then ECAM library is doing a new mapping
> and attach generic ECAM ops which are used for accessing config space.
> 
> On ARM64, ACPI and DT can be enabled together, and in that case
> we need to use generic domains. In order to do that we implement
> ARM64 specific way of retrieving domain number from pci_config_window
> structure.
> 
> Since we enable PCI for ACPI we need to implement raw_pci_{read|write}
> at the same time. ARM64 provides RAW accessors as long as there is
> correlated valid pci_bus structure, but not before.

I think this is important and needs to be spelled out a little more
explicitly using the terms from the ACPI spec so it will make sense to
people like BIOS writers who might be bitten by this.

"raw_pci_read()" is a Linux term that won't mean anything to them.
But they might expect that AML can access PCI config space before the
Linux driver claims the host bridge, and I think you're saying that
will not work.

> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
> ---
>  arch/arm64/Kconfig      |   2 +
>  arch/arm64/kernel/pci.c | 120 ++++++++++++++++++++++++++++++++++++++++++++++--
>  include/linux/pci.h     |   5 ++
>  3 files changed, 122 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 76747d9..87c48ad 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -3,6 +3,7 @@ config ARM64
>  	select ACPI_CCA_REQUIRED if ACPI
>  	select ACPI_GENERIC_GSI if ACPI
>  	select ACPI_REDUCED_HARDWARE_ONLY if ACPI
> +	select ACPI_MCFG if ACPI
>  	select ARCH_HAS_DEVMEM_IS_ALLOWED
>  	select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
>  	select ARCH_HAS_ELF_RANDOMIZE
> @@ -96,6 +97,7 @@ config ARM64
>  	select OF_EARLY_FLATTREE
>  	select OF_NUMA if NUMA && OF
>  	select OF_RESERVED_MEM
> +	select PCI_ECAM if ACPI
>  	select PERF_USE_VMALLOC
>  	select POWER_RESET
>  	select POWER_SUPPLY
> diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> index 3663be1..39f2a40 100644
> --- a/arch/arm64/kernel/pci.c
> +++ b/arch/arm64/kernel/pci.c
> @@ -17,7 +17,9 @@
>  #include <linux/mm.h>
>  #include <linux/of_pci.h>
>  #include <linux/of_platform.h>
> +#include <linux/pci.h>
>  #include <linux/pci-acpi.h>
> +#include <linux/pci-ecam.h>
>  #include <linux/slab.h>
>  
>  /*
> @@ -71,13 +73,21 @@ int pcibios_alloc_irq(struct pci_dev *dev)
>  int raw_pci_read(unsigned int domain, unsigned int bus,
>  		  unsigned int devfn, int reg, int len, u32 *val)
>  {
> -	return -ENXIO;
> +	struct pci_bus *b = pci_find_bus(domain, bus);
> +
> +	if (!b)
> +		return PCIBIOS_DEVICE_NOT_FOUND;
> +	return b->ops->read(b, devfn, reg, len, val);
>  }
>  
>  int raw_pci_write(unsigned int domain, unsigned int bus,
>  		unsigned int devfn, int reg, int len, u32 val)
>  {
> -	return -ENXIO;
> +	struct pci_bus *b = pci_find_bus(domain, bus);
> +
> +	if (!b)
> +		return PCIBIOS_DEVICE_NOT_FOUND;
> +	return b->ops->write(b, devfn, reg, len, val);
>  }

I think these functions could be under #ifdef ACPI, couldn't they?
I see x86 and ia64 don't do that, and they do call them from non-ACPI
places.  So maybe there'd be no point in doing it differently here.

You could split this out into a separate patch, though, which would be
a nice way to highlight the issue of AML access before pci_root.c
claims the bridge.

>  #ifdef CONFIG_NUMA
> @@ -91,11 +101,111 @@ EXPORT_SYMBOL(pcibus_to_node);
>  #endif
>  
>  #ifdef CONFIG_ACPI
> -/* Root bridge scanning */
> +
> +struct acpi_pci_generic_root_info {
> +	struct acpi_pci_root_info	common;
> +	struct pci_config_window	*cfg;	/* config space mapping */
> +};
> +
> +int acpi_pci_bus_domain_nr(struct pci_bus *bus)
> +{
> +	struct pci_config_window *cfg = bus->sysdata;
> +	struct acpi_device *adev = to_acpi_device(cfg->parent);
> +	struct acpi_pci_root *root = acpi_driver_data(adev);
> +
> +	return root->segment;
> +}
> +
> +int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
> +{
> +	if (!acpi_disabled) {
> +		struct pci_config_window *cfg = bridge->bus->sysdata;
> +		struct acpi_device *adev = to_acpi_device(cfg->parent);
> +		ACPI_COMPANION_SET(&bridge->dev, adev);
> +	}
> +
> +	return 0;
> +}
> +
> +/*
> + * Lookup the bus range for the domain in MCFG, and set up config space
> + * mapping.
> + */
> +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
> +				       struct acpi_pci_generic_root_info *ri)

Maybe return the "struct pci_config_window *" and do the assignment in
the caller?  Then I don't think you'd have to pass in "ri".

> +{
> +	struct resource *bus_res = &root->secondary;
> +	u16 seg = root->segment;
> +	struct pci_config_window *cfg;
> +	struct resource cfgres;
> +	unsigned int bsz;
> +	int err;
> +
> +	err = pci_mcfg_lookup(root);
> +	if (err) {
> +		pr_err("%04x:%pR MCFG region not found\n", seg, bus_res);

dev_err()

> +		return err;
> +	}
> +
> +	bsz = 1 << pci_generic_ecam_ops.bus_shift;
> +	cfgres.start = root->mcfg_addr + bus_res->start * bsz;
> +	cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
> +	cfgres.flags = IORESOURCE_MEM;
> +	cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res,
> +			      &pci_generic_ecam_ops);
> +	if (IS_ERR(cfg)) {
> +		pr_err("%04x:%pR error %ld mapping CAM\n", seg, bus_res,
> +		       PTR_ERR(cfg));

dev_err()

> +		return PTR_ERR(cfg);
> +	}
> +
> +	ri->cfg = cfg;
> +	return 0;
> +}
> +
> +/* release_info: free resrouces allocated by init_info */
> +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
> +{
> +	struct acpi_pci_generic_root_info *ri;
> +
> +	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
> +	pci_ecam_free(ri->cfg);
> +	kfree(ri);
> +}
> +
> +static struct acpi_pci_root_ops acpi_pci_root_ops = {
> +	.release_info = pci_acpi_generic_release_info,
> +};
> +
> +/* Interface called from ACPI code to setup PCI host controller */
>  struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
>  {
> -	/* TODO: Should be revisited when implementing PCI on ACPI */
> -	return NULL;
> +	int node = acpi_get_node(root->device->handle);
> +	struct acpi_pci_generic_root_info *ri;
> +	struct pci_bus *bus, *child;
> +	int err;
> +
> +	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
> +	if (!ri)
> +		return NULL;
> +
> +	err = pci_acpi_setup_ecam_mapping(root, ri);
> +	if (err)
> +		return NULL;
> +
> +	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
> +	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
> +				   ri->cfg);
> +	if (!bus)
> +		return NULL;
> +
> +	pci_bus_size_bridges(bus);
> +	pci_bus_assign_resources(bus);
> +
> +	list_for_each_entry(child, &bus->children, node)
> +		pcie_bus_configure_settings(child);
> +
> +	return bus;
>  }
>  
>  void pcibios_add_bus(struct pci_bus *bus)
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 9661c85..f66d188 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1390,7 +1390,12 @@ static inline int pci_domain_nr(struct pci_bus *bus)
>  {
>  	return bus->domain_nr;
>  }
> +/* Arch specific ACPI hook to set-up domain number */
> +#ifdef CONFIG_ACPI
> +int acpi_pci_bus_domain_nr(struct pci_bus *bus);
> +#else
>  static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return -1; }
> +#endif
>  void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent);
>  #else
>  static inline void pci_bus_assign_domain_nr(struct pci_bus *bus,
> -- 
> 1.9.1
> 

Re: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Dongdong Liu]

在 2016/5/30 23:14, Tomasz Nowicki 写道:
>>From the functionality point of view this series may be split into the
> following logic parts:
> 1. Export ECAM API and add parent device to pci_config_window
> 2. Add IO resources handling to PCI core code
> 3. Support for generic domain assignment based on ACPI
> 4. New MCFG driver
> 5. Implement ARM64 ACPI based PCI host controller driver under arch/arm64/


Based on the patchset (with "[RFC PATCH 0/3] ECAM quirks handling for ARM64 platforms" added)
Tested on the HiSilicon ARM64 D02 board.
It worked ok with Intel 82599 networking card.
This is the bootup log which contains PCIe host and Intel 82599 networking card part.

Tested-by: Dongdong Liu <liudongdong3@huawei.com>

Loading driver at 0x0006D473000 EntryPoint=0x0006DE80100
Loading driver at 0x0006D473000 EntryPoint=0x0006DE80100
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
GMAC ExitBootServicesEvent
SMMU ExitBootServicesEvent
[    0.000000] Booting Linux on physical CPU 0x20000
[    0.000000] Linux version 4.6.0-rc1+ (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #54 SMP PREEMPT Mon Jun 6 16:47:28 CST 2016
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000080300000 (options '')
[    0.000000] bootconsole [uart8250] enabled
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] EFI v2.50 by EDK II
[    0.000000] efi:  SMBIOS=0x7a650000  SMBIOS 3.0=0x7a630000  ACPI=0x7aba0000  ACPI 2.0=0x7aba0014
[    0.000000] cma: Reserved 16 MiB at 0x000000007e800000
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x000000007ABA0014 000024 (v02 HISI  )
[    0.000000] ACPI: XSDT 0x000000007A7000E8 000064 (v01 HISI   HISI-D02 20140727      01000013)
[    0.000000] ACPI: FACP 0x000000007A5F0000 00010C (v05 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: DSDT 0x000000007A5A0000 001656 (v01 HISI   HISI-D02 20140727 INTL 20150619)
[    0.000000] ACPI: DBG2 0x000000007A610000 00005A (v00 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: GTDT 0x000000007A5E0000 000060 (v02 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: APIC 0x000000007A5D0000 000554 (v01 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: MCFG 0x000000007A5C0000 00004C (v01 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: SPCR 0x000000007A5B0000 000050 (v02 HISI   HISI-D02 20140727 HISI 00000099)
[    0.000000] ACPI: IORT 0x000000007A590000 0001FC (v00 INTEL  TEMPLATE 00000000 INTL 20150619)
[    0.000000] ACPI: SSDT 0x000000007A580000 00046E (v01 HISI   SAS0     20140727 INTL 20150619)
[    0.000000] ACPI: SPCR: console: uart,mmio,0x80300000,115200
[    0.000000] psci: probing for conduit method from ACPI.
NOTICE:  [psci_smc_handler]:[347L] PSCI_VERSION CALL
NOTICE:  [psci_version]:[99L] PSCI_MAJOR_VER: 10000: PSCI_MINOR_VER: 0

0808?844
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs

0808?844
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.

0808?844

0808?844
[    0.000000] percpu: Embedded 20 pages/cpu @ffffffd1ffe7e000 s43008 r8192 d30720 u81920
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] CPU features: enabling workaround for ARM erratum 834220
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2063376
[    0.000000] Kernel command line: console=ttyS0,115200 earlycon=uart8250,mmio32,0x80300000 initrd=filesystem.cpio.gz acpi=force pcie_aspm=off
[    0.000000] PCIe ASPM is disabled
[    0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 61440 bytes
[    0.000000] log_buf_len min size: 16384 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 12988(79%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] software IO TLB [mem 0x764e0000-0x7a4e0000] (64MB) mapped at [ffffffc0764e0000-ffffffc07a4dffff]
[    0.000000] Memory: 8110296K/8384512K available (7240K kernel code, 632K rwdata, 3028K rodata, 840K init, 247K bss, 257832K reserved, 16384K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbdbfff0000   (   246 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008790000   (  7232 KB)
[    0.000000]     .rodata : 0xffffff8008790000 - 0xffffff8008a89000   (  3044 KB)
[    0.000000]       .init : 0xffffff8008a89000 - 0xffffff8008b5b000   (   840 KB)
[    0.000000]       .data : 0xffffff8008b5b000 - 0xffffff8008bf9200   (   633 KB)
[    0.000000]     vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000   (     8 GB maximum)
[    0.000000]               0xffffffbdc0000000 - 0xffffffbe08000000   (  1152 MB actual)
[    0.000000]     fixed   : 0xffffffbffe7fd000 - 0xffffffbffec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbffee00000 - 0xffffffbfffe00000   (    16 MB)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffd200000000   ( 73728 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=16.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=16
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] ITS@0x8c000000
[    0.000000] ITS: allocated 65536 Devices @11f6c80000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Virtual CPUs @11f6c0f000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Interrupt Collections @11f6c20000 (psz 4K, shr 1)
[    0.000000] ITS@0xc6000000
[    0.000000] ITS: allocated 65536 Devices @11f6d00000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Virtual CPUs @11f6c21000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Interrupt Collections @11f6c22000 (psz 4K, shr 1)
[    0.000000] ITS@0xa3000000
[    0.000000] ITS: allocated 65536 Devices @11f6d80000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Virtual CPUs @11f6c24000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Interrupt Collections @11f6c25000 (psz 4K, shr 1)
[    0.000000] ITS@0xb7000000
[    0.000000] ITS: allocated 65536 Devices @11f6e00000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Virtual CPUs @11f6c26000 (psz 4K, shr 1)
[    0.000000] ITS: allocated 512 Interrupt Collections @11f6c27000 (psz 4K, shr 1)
[    0.000000] GIC: using LPI property table @0x00000011f6c60000
[    0.000000] ITS: Allocated 1792 chunks for LPIs
[    0.000000] CPU0: found redistributor 20000 region 0:0x000000008d100000
[    0.000000] CPU0: using LPI pending table @0x00000011f6c70000
[    0.000000] Unable to get hardware information used for virtualization
[    0.000000] GTDT: No Platform Timer structures.
[    0.000000] arch_timer: Can't find GT Block.
[    0.000000] Architected cp15 and mmio timer(s) running at 50.00MHz (phys/phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[    0.000001] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
[    0.008025] Console: colour dummy device 80x25
[    0.012456] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000)
[    0.022849] pid_max: default: 32768 minimum: 301
[    0.027449] ACPI: Core revision 20160108
[    0.032291] ACPI: 2 ACPI AML tables successfully acquired and loaded
[    0.038623]
[    0.040133] Security Framework initialized
[    0.044221] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.051061] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.058750] ASID allocator initialised with 65536 entries
[    0.064391] PCI/MSI: ITS@0x8c000000 domain created
[    0.069169] PCI/MSI: ITS@0xc6000000 domain created
[    0.073935] PCI/MSI: ITS@0xa3000000 domain created
[    0.078700] PCI/MSI: ITS@0xb7000000 domain created
[    0.083471] Platform MSI: irqchip@000000008c000000 domain created
[    0.089533] Platform MSI: irqchip@00000000c6000000 domain created
[    0.095603] Platform MSI: irqchip@00000000a3000000 domain created
[    0.101673] Platform MSI: irqchip@00000000b7000000 domain created
[    0.107778] Remapping and enabling EFI services.
[    0.112396]   EFI remap 0x000000007a4e0000 => 0000000020000000
[    0.118212]   EFI remap 0x000000007a530000 => 0000000020050000
[    0.124035]   EFI remap 0x000000007a620000 => 00000000200a0000
[    0.129850]   EFI remap 0x000000007a6b0000 => 0000000020130000
[    0.135664]   EFI remap 0x000000007a710000 => 0000000020180000
[    0.141491]   EFI remap 0x000000007a760000 => 00000000201d0000
[    0.147306]   EFI remap 0x000000007a7b0000 => 0000000020220000
[    0.153120]   EFI remap 0x000000007a800000 => 0000000020270000
[    0.158934]   EFI remap 0x000000007a850000 => 00000000202c0000
[    0.164748]   EFI remap 0x000000007a8a0000 => 0000000020310000
[    0.170571]   EFI remap 0x000000007a8f0000 => 0000000020360000
[    0.176385]   EFI remap 0x000000007a940000 => 00000000203b0000
[    0.182207]   EFI remap 0x000000007a990000 => 0000000020400000
[    0.188022]   EFI remap 0x000000007aa00000 => 0000000020470000
[    0.193837]   EFI remap 0x000000007aa50000 => 00000000204c0000
[    0.199651]   EFI remap 0x000000007aaa0000 => 0000000020510000
[    0.205466]   EFI remap 0x000000007aaf0000 => 0000000020560000
[    0.211280]   EFI remap 0x000000007ab40000 => 00000000205b0000
[    0.217093]   EFI remap 0x000000007fbb0000 => 0000000020600000
[    0.222899]   EFI remap 0x0000000080300000 => 0000000020630000
[    0.228712]   EFI remap 0x00000000a00f0000 => 0000000020640000
[    0.234516]   EFI remap 0x00000000a4000000 => 0000000020800000
[    0.240324]   EFI remap 0x00000000a6000000 => 0000000021800000
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20001 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c080
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d190
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20002 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c100
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d3a0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20003 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c180
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d5b0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20100 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c200
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d7c0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20101 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c280
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d9d0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20102 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c300
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3dbe0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20103 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c380
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ddf0
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20200 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c400
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e000
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20201 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c480
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e210
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20202 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c500
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e420
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20203 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c580
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e630
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20300 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c600
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e840
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20301 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c680
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ea50
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20302 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c700
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ec60
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE:  [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE:  [psci_smc_handler]:[409L] x1=0x20303 x2=0x82870 x3=0x0
NOTICE:  [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE:  [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?8AB44
NOTICE:  [psci_afflvl_power_on_finish]:[504L]
NOTICE:  [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c780
NOTICE:  [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ee70
NOTICE:  [psci_afflvl_power_on_finish]:[562L]

00082870
[    0.288086] Detected PIPT I-cache on CPU1
[    0.288098] CPU1: found redistributor 20001 region 1:0x000000008d130000
[    0.288118] CPU1: using LPI pending table @0x00000011f6410000
[    0.288178] CPU1: Booted secondary processor [411fd071]
[    0.331198] Detected PIPT I-cache on CPU2
[    0.331204] CPU2: found redistributor 20002 region 2:0x000000008d160000
[    0.331225] CPU2: using LPI pending table @0x00000011f6440000
[    0.331272] CPU2: Booted secondary processor [411fd071]
[    0.374311] Detected PIPT I-cache on CPU3
[    0.374318] CPU3: found redistributor 20003 region 3:0x000000008d190000
[    0.374338] CPU3: using LPI pending table @0x00000011f6480000
[    0.374383] CPU3: Booted secondary processor [411fd071]
[    0.417427] Detected PIPT I-cache on CPU4
[    0.417435] CPU4: found redistributor 20100 region 4:0x000000008d1c0000
[    0.417455] CPU4: using LPI pending table @0x00000011f64c0000
[    0.417502] CPU4: Booted secondary processor [411fd071]
[    0.460539] Detected PIPT I-cache on CPU5
[    0.460546] CPU5: found redistributor 20101 region 5:0x000000008d1f0000
[    0.460565] CPU5: using LPI pending table @0x00000011f64f0000
[    0.460612] CPU5: Booted secondary processor [411fd071]
[    0.503653] Detected PIPT I-cache on CPU6
[    0.503660] CPU6: found redistributor 20102 region 6:0x000000008d220000
[    0.503679] CPU6: using LPI pending table @0x00000011f6530000
[    0.503726] CPU6: Booted secondary processor [411fd071]
[    0.546766] Detected PIPT I-cache on CPU7
[    0.546773] CPU7: found redistributor 20103 region 7:0x000000008d250000
[    0.546793] CPU7: using LPI pending table @0x00000011f6560000
[    0.546838] CPU7: Booted secondary processor [411fd071]
[    0.589881] Detected PIPT I-cache on CPU8
[    0.589890] CPU8: found redistributor 20200 region 8:0x000000008d280000
[    0.589911] CPU8: using LPI pending table @0x00000011f65a0000
[    0.589963] CPU8: Booted secondary processor [411fd071]
[    0.632994] Detected PIPT I-cache on CPU9
[    0.633001] CPU9: found redistributor 20201 region 9:0x000000008d2b0000
[    0.633021] CPU9: using LPI pending table @0x00000011f65e0000
[    0.633068] CPU9: Booted secondary processor [411fd071]
[    0.676107] Detected PIPT I-cache on CPU10
[    0.676114] CPU10: found redistributor 20202 region 10:0x000000008d2e0000
[    0.676136] CPU10: using LPI pending table @0x00000011f6610000
[    0.676182] CPU10: Booted secondary processor [411fd071]
[    0.719221] Detected PIPT I-cache on CPU11
[    0.719228] CPU11: found redistributor 20203 region 11:0x000000008d310000
[    0.719249] CPU11: using LPI pending table @0x00000011f6650000
[    0.719295] CPU11: Booted secondary processor [411fd071]
[    0.762335] Detected PIPT I-cache on CPU12
[    0.762344] CPU12: found redistributor 20300 region 12:0x000000008d340000
[    0.762365] CPU12: using LPI pending table @0x00000011f6680000
[    0.762414] CPU12: Booted secondary processor [411fd071]
[    0.805447] Detected PIPT I-cache on CPU13
[    0.805454] CPU13: found redistributor 20301 region 13:0x000000008d370000
[    0.805475] CPU13: using LPI pending table @0x00000011f66c0000
[    0.805521] CPU13: Booted secondary processor [411fd071]
[    0.848561] Detected PIPT I-cache on CPU14
[    0.848568] CPU14: found redistributor 20302 region 14:0x000000008d3a0000
[    0.848588] CPU14: using LPI pending table @0x00000011f6700000
[    0.848633] CPU14: Booted secondary processor [411fd071]
[    0.891674] Detected PIPT I-cache on CPU15
[    0.891682] CPU15: found redistributor 20303 region 15:0x000000008d3d0000
[    0.891702] CPU15: using LPI pending table @0x00000011f6750000
[    0.891747] CPU15: Booted secondary processor [411fd071]
[    0.891777] Brought up 16 CPUs
[    1.220819] SMP: Total of 16 processors activated.
[    1.225585] CPU features: detected feature: GIC system register CPU interface
[    1.232685] CPU: All CPU(s) started at EL2
[    1.236783] alternatives: patching kernel code
[    1.244033] devtmpfs: initialized
[    1.247592] SMBIOS 3.0.0 present.
[    1.251009] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.260957] pinctrl core: initialized pinctrl subsystem
[    1.266535] NET: Registered protocol family 16
[    1.286979] cpuidle: using governor menu
[    1.290926] vdso: 2 pages (1 code @ ffffff8008796000, 1 data @ ffffff8008b60000)
[    1.298337] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    1.305478] DMA: preallocated 256 KiB pool for atomic allocations
[    1.311614] ACPI: bus type PCI registered
[    1.315661] Serial: AMBA PL011 UART driver
[    1.335915] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    1.342641] ACPI: Added _OSI(Module Device)
[    1.346823] ACPI: Added _OSI(Processor Device)
[    1.351261] ACPI: Added _OSI(3.0 _SCP Extensions)
[    1.355978] ACPI: Added _OSI(Processor Aggregator Device)
[    1.362839] ACPI: Interpreter enabled
[    1.366483] ACPI: Using GIC for interrupt routing
[    1.371216] ACPI: MCFG table detected, 2 entries
[    1.378990] Hisilicon MBIGEN-V1 HISI0151:00: Allocated 256 MSIs
[    1.384979] Hisilicon MBIGEN-V1 HISI0151:01: Allocated 640 MSIs
[    1.390943] Hisilicon MBIGEN-V1 HISI0151:02: Allocated 256 MSIs
[    1.396936] Hisilicon MBIGEN-V1 HISI0151:03: Allocated 640 MSIs
[    1.403099] ACPI: IORT: can't find node related to (null) device
[    1.409175] ACPI: IORT: can't find node related to (null) device
[    1.415242] ACPI: IORT: can't find node related to (null) device
[    1.421301] ACPI: IORT: can't find node related to (null) device
[    1.427347] ACPI: IORT: can't find node related to (null) device
[    1.433371] ACPI: IORT: can't find node related to (null) device
[    1.440631] ACPI: IORT: can't find node related to (null) device
[    1.446670] ACPI: IORT: can't find node related to (null) device
[    1.452704] ACPI: IORT: can't find node related to (null) device
[    1.458734] ACPI: IORT: can't find node related to (null) device
[    1.464747] ACPI: IORT: can't find node related to (null) device
[    1.470788] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus 40-7f])
[    1.476960] acpi HISI0080:00: _OSC: OS supports [ExtendedConfig Segments MSI]
[    1.484125] acpi HISI0080:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    1.490913] acpi HISI0080:00: ECAM at [mem 0x22004000000-0x22007ffffff] for [bus 40-7f]
[    1.498924] Remapped I/O 0x000002200fff0000 to [io  0x0000-0xffff window]
[    1.505769] PCI host bridge to bus 0001:40
[    1.509866] pci_bus 0001:40: root bus resource [mem 0x22008000000-0x2200ffeffff window] (bus address [0xb0000000-0xb7feffff])
[    1.521157] pci_bus 0001:40: root bus resource [io  0x0000-0xffff window]
[    1.527929] pci_bus 0001:40: root bus resource [bus 40-7f]
[    1.538205] pci 0001:41:00.0: VF(n) BAR0 space: [mem 0x22008e08000-0x22008f07fff 64bit pref] (contains BAR0 for 64 VFs)
[    1.549469] pci 0001:41:00.0: VF(n) BAR3 space: [mem 0x22008f08000-0x22009007fff 64bit pref] (contains BAR3 for 64 VFs)
[    1.572065] pci 0001:41:00.1: VF(n) BAR0 space: [mem 0x22008c04000-0x22008d03fff 64bit pref] (contains BAR0 for 64 VFs)
[    1.583295] pci 0001:41:00.1: VF(n) BAR3 space: [mem 0x22008d04000-0x22008e03fff 64bit pref] (contains BAR3 for 64 VFs)
[    1.601438] pci 0001:40:00.0: BAR 15: assigned [mem 0x22008000000-0x220095fffff pref]
[    1.609249] pci 0001:40:00.0: BAR 13: assigned [io  0x1000-0x1fff]
[    1.615446] pci 0001:41:00.0: BAR 0: assigned [mem 0x22008000000-0x220083fffff 64bit pref]
[    1.623961] pci 0001:41:00.0: BAR 6: assigned [mem 0x22008400000-0x220087fffff pref]
[    1.631668] pci 0001:41:00.1: BAR 0: assigned [mem 0x22008800000-0x22008bfffff 64bit pref]
[    1.640185] pci 0001:41:00.1: BAR 6: assigned [mem 0x22008c00000-0x22008ffffff pref]
[    1.647909] pci 0001:41:00.0: BAR 4: assigned [mem 0x22009000000-0x22009003fff 64bit pref]
[    1.656391] pci 0001:41:00.0: BAR 7: assigned [mem 0x22009004000-0x22009103fff 64bit pref]
[    1.664881] pci 0001:41:00.0: BAR 10: assigned [mem 0x22009104000-0x22009203fff 64bit pref]
[    1.673471] pci 0001:41:00.1: BAR 4: assigned [mem 0x22009204000-0x22009207fff 64bit pref]
[    1.681961] pci 0001:41:00.1: BAR 7: assigned [mem 0x22009208000-0x22009307fff 64bit pref]
[    1.690469] pci 0001:41:00.1: BAR 10: assigned [mem 0x22009308000-0x22009407fff 64bit pref]
[    1.699054] pci 0001:41:00.0: BAR 2: assigned [io  0x1000-0x101f]
[    1.705205] pci 0001:41:00.1: BAR 2: assigned [io  0x1020-0x103f]
[    1.711341] pci 0001:40:00.0: PCI bridge to [bus 41-42]
[    1.716550] pci 0001:40:00.0:   bridge window [io  0x1000-0x1fff]
[    1.722622] pci 0001:40:00.0:   bridge window [mem 0x22008000000-0x220095fffff pref]
[    1.730413] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-bf])
[    1.736573] acpi HISI0080:01: _OSC: OS supports [ExtendedConfig Segments MSI]
[    1.743744] acpi HISI0080:01: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    1.750498] acpi HISI0080:01: ECAM at [mem 0x24008000000-0x2400bffffff] for [bus 80-bf]
[    1.758501] Remapped I/O 0x000002400fff0000 to [io  0x10000-0x1ffff window]
[    1.765498] PCI host bridge to bus 0002:80
[    1.769603] pci_bus 0002:80: root bus resource [mem 0x2400c000000-0x2400ffeffff window] (bus address [0xc0000000-0xc3feffff])
[    1.780955] pci_bus 0002:80: root bus resource [io  0x10000-0x1ffff window] (bus address [0x0000-0xffff])
[    1.790543] pci_bus 0002:80: root bus resource [bus 80-bf]
[    1.800721] pci 0002:81:00.0: VF(n) BAR0 space: [mem 0x2400ce08000-0x2400cf07fff 64bit pref] (contains BAR0 for 64 VFs)
[    1.811963] pci 0002:81:00.0: VF(n) BAR3 space: [mem 0x2400cf08000-0x2400d007fff 64bit pref] (contains BAR3 for 64 VFs)
[    1.834499] pci 0002:81:00.1: VF(n) BAR0 space: [mem 0x2400cc04000-0x2400cd03fff 64bit pref] (contains BAR0 for 64 VFs)
[    1.845746] pci 0002:81:00.1: VF(n) BAR3 space: [mem 0x2400cd04000-0x2400ce03fff 64bit pref] (contains BAR3 for 64 VFs)
[    1.863882] pci 0002:80:00.0: BAR 15: assigned [mem 0x2400c000000-0x2400d5fffff pref]
[    1.871683] pci 0002:80:00.0: BAR 13: assigned [io  0x10000-0x10fff]
[    1.878053] pci 0002:81:00.0: BAR 0: assigned [mem 0x2400c000000-0x2400c3fffff 64bit pref]
[    1.886586] pci 0002:81:00.0: BAR 6: assigned [mem 0x2400c400000-0x2400c7fffff pref]
[    1.894303] pci 0002:81:00.1: BAR 0: assigned [mem 0x2400c800000-0x2400cbfffff 64bit pref]
[    1.902845] pci 0002:81:00.1: BAR 6: assigned [mem 0x2400cc00000-0x2400cffffff pref]
[    1.910553] pci 0002:81:00.0: BAR 4: assigned [mem 0x2400d000000-0x2400d003fff 64bit pref]
[    1.919017] pci 0002:81:00.0: BAR 7: assigned [mem 0x2400d004000-0x2400d103fff 64bit pref]
[    1.927491] pci 0002:81:00.0: BAR 10: assigned [mem 0x2400d104000-0x2400d203fff 64bit pref]
[    1.936070] pci 0002:81:00.1: BAR 4: assigned [mem 0x2400d204000-0x2400d207fff 64bit pref]
[    1.944577] pci 0002:81:00.1: BAR 7: assigned [mem 0x2400d208000-0x2400d307fff 64bit pref]
[    1.953050] pci 0002:81:00.1: BAR 10: assigned [mem 0x2400d308000-0x2400d407fff 64bit pref]
[    1.961616] pci 0002:81:00.0: BAR 2: assigned [io  0x10000-0x1001f]
[    1.967931] pci 0002:81:00.1: BAR 2: assigned [io  0x10020-0x1003f]
[    1.974247] pci 0002:80:00.0: PCI bridge to [bus 81-82]
[    1.979455] pci 0002:80:00.0:   bridge window [io  0x10000-0x10fff]
[    1.985752] pci 0002:80:00.0:   bridge window [mem 0x2400c000000-0x2400d5fffff pref]
[    1.993935] ACPI: IORT: can't find node related to (null) device
[    2.000280] ACPI: IORT: can't find node related to (null) device
[    2.006518] vgaarb: loaded
[    2.009305] SCSI subsystem initialized
[    2.013165] ACPI: bus type USB registered
[    2.017216] usbcore: registered new interface driver usbfs
[    2.022710] usbcore: registered new interface driver hub
[    2.028036] usbcore: registered new device driver usb
[    2.033133] pps_core: LinuxPPS API ver. 1 registered
[    2.038081] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    2.047210] PTP clock support registered
[    2.051242] Advanced Linux Sound Architecture Driver Initialized.
[    2.057588] clocksource: Switched to clocksource arch_sys_counter
[    2.063718] VFS: Disk quotas dquot_6.6.0
[    2.067668] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    2.074641] pnp: PnP ACPI init
[    2.077840] system 00:00: [mem 0xb0080000-0xb008ffff] has been reserved
[    2.084508] system 00:01: [mem 0xb0090000-0xb009ffff] has been reserved
[    2.091147] pnp: PnP ACPI: found 2 devices
[    2.097821] NET: Registered protocol family 2
[    2.102391] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[    2.109728] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    2.116726] TCP: Hash tables configured (established 65536 bind 65536)
[    2.123285] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[    2.129420] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[    2.136046] NET: Registered protocol family 1
[    2.140531] RPC: Registered named UNIX socket transport module.
[    2.146432] RPC: Registered udp transport module.
[    2.151156] RPC: Registered tcp transport module.
[    2.155853] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    2.162588] Unpacking initramfs...
[    2.524222] Freeing initrd memory: 27492K (ffffffc01e520000 - ffffffc01fff9000)
[    2.532046] kvm [1]: 8-bit VMID
[    2.535185] kvm [1]: Hyp mode initialized successfully
[    2.540318] kvm [1]: error: KVM vGIC probing failed
[    2.545231] kvm [1]: virtual timer IRQ3
[    2.550055] ACPI: IORT: can't find node related to (null) device
[    2.556473] futex hash table entries: 4096 (order: 7, 524288 bytes)
[    2.562819] audit: initializing netlink subsys (disabled)
[    2.568241] audit: type=2000 audit(2.048:1): initialized
[    2.573809] workingset: timestamp_bits=44 max_order=21 bucket_order=0
[    2.583793] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.589908] NFS: Registering the id_resolver key type
[    2.594986] Key type id_resolver registered
[    2.599192] Key type id_legacy registered
[    2.603280] fuse init (API version 7.24)
[    2.607352] 9p: Installing v9fs 9p2000 file system support
[    2.613675] io scheduler noop registered
[    2.617652] io scheduler cfq registered (default)
[    2.622760] pcieport 0001:40:00.0: can't derive routing for PCI INT A
[    2.629188] pcieport 0001:40:00.0: PCI INT A: no GSI
[    2.634240] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    2.640674] pcieport 0002:80:00.0: PCI INT A: no GSI
[    2.646398] xenfs: not registering filesystem on non-xen platform
[    2.653568] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    2.660401] console [ttyS0] disabled
[    2.663979] APMC0D08:00: ttyS0 at MMIO 0x80300000 (irq = 5, base_baud = 12500000) is a 16550A
[    2.672570] console [ttyS0] enabled
[    2.672570] console [ttyS0] enabled
[    2.679571] bootconsole [uart8250] disabled
[    2.679571] bootconsole [uart8250] disabled
[    2.688167] SuperH (H)SCI(F) driver initialized
[    2.692759] msm_serial: driver initialized
[    2.697087] Failed to find cpu0 device node
[    2.701290] Unable to detect cache hierarchy from DT for CPU 0
[    2.709537] loop: module loaded
[    2.713143] tun: Universal TUN/TAP device driver, 1.6
[    2.718221] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    2.724530] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    2.730390] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    2.736361] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k
[    2.743353] igb: Copyright (c) 2007-2014 Intel Corporation.
[    2.748974] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.0.2-k
[    2.756838] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    2.762804] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.2.1-k
[    2.770493] ixgbe: Copyright (c) 1999-2015 Intel Corporation.
[    2.776311] pcieport 0001:40:00.0: can't derive routing for PCI INT A
[    2.782783] ixgbe 0001:41:00.0: PCI INT A: no GSI
[    2.787630] ixgbe 0001:41:00.0: enabling device (0000 -> 0002)
[    2.949048] ixgbe 0001:41:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16
[    2.957721] ixgbe 0001:41:00.0: PCI Express bandwidth of 32GT/s available
[    2.964541] ixgbe 0001:41:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[    2.971785] ixgbe 0001:41:00.0: MAC: 2, PHY: 17, SFP+: 5, PBA No: FFFFFF-0FF
[    2.978864] ixgbe 0001:41:00.0: 68:a8:28:2e:c9:10
[    2.988081] ixgbe 0001:41:00.0: Intel(R) 10 Gigabit Network Connection
[    2.994671] pcieport 0001:40:00.0: can't derive routing for PCI INT B
[    3.001142] ixgbe 0001:41:00.1: PCI INT B: no GSI
[    3.005959] ixgbe 0001:41:00.1: enabling device (0000 -> 0002)
[    4.145034] ixgbe 0001:41:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16
[    4.153704] ixgbe 0001:41:00.1: PCI Express bandwidth of 32GT/s available
[    4.160523] ixgbe 0001:41:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[    4.167767] ixgbe 0001:41:00.1: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF
[    4.173973] ixgbe 0001:41:00.1: 68:a8:28:2e:c9:11
[    4.183147] ixgbe 0001:41:00.1: Intel(R) 10 Gigabit Network Connection
[    4.189727] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    4.196197] ixgbe 0002:81:00.0: PCI INT A: no GSI
[    4.201010] ixgbe 0002:81:00.0: enabling device (0000 -> 0002)
[    5.341097] ixgbe 0002:81:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16
[    5.349767] ixgbe 0002:81:00.0: PCI Express bandwidth of 32GT/s available
[    5.356587] ixgbe 0002:81:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[    5.363830] ixgbe 0002:81:00.0: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF
[    5.370036] ixgbe 0002:81:00.0: 9c:37:f4:90:cd:23
[    5.379230] ixgbe 0002:81:00.0: Intel(R) 10 Gigabit Network Connection
[    5.385810] pcieport 0002:80:00.0: can't derive routing for PCI INT B
[    5.392282] ixgbe 0002:81:00.1: PCI INT B: no GSI
[    5.397094] ixgbe 0002:81:00.1: enabling device (0000 -> 0002)
[    6.537144] ixgbe 0002:81:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16
[    6.545814] ixgbe 0002:81:00.1: PCI Express bandwidth of 32GT/s available
[    6.552635] ixgbe 0002:81:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[    6.559879] ixgbe 0002:81:00.1: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF
[    6.566087] ixgbe 0002:81:00.1: 9c:37:f4:90:cd:24
[    6.575284] ixgbe 0002:81:00.1: Intel(R) 10 Gigabit Network Connection
[    6.581876] sky2: driver version 1.30
[    6.585704] VFIO - User Level meta-driver version: 0.3
[    6.591381] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    6.597943] ehci-pci: EHCI PCI platform driver
[    6.602426] ehci-platform: EHCI generic platform driver
[    6.607706] ehci-platform PNP0D20:00: EHCI Host Controller
[    6.613226] ehci-platform PNP0D20:00: new USB bus registered, assigned bus number 1
[    6.621049] ehci-platform PNP0D20:00: irq 6, io mem 0xa1000000
[    6.637594] ehci-platform PNP0D20:00: USB 2.0 started, EHCI 1.00
[    6.643858] hub 1-0:1.0: USB hub found
[    6.647634] hub 1-0:1.0: 1 port detected
[    6.651722] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[    6.657341] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    6.663555] ohci-pci: OHCI PCI platform driver
[    6.668038] ohci-platform: OHCI generic platform driver
[    6.673371] usbcore: registered new interface driver usb-storage
[    6.679628] mousedev: PS/2 mouse device common for all mice
[    6.748745] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[    6.754943] i2c /dev entries driver
[    6.758679] sdhci: Secure Digital Host Controller Interface driver
[    6.764887] sdhci: Copyright(c) Pierre Ossman
[    6.769279] Synopsys Designware Multimedia Card Interface Driver
[    6.775364] sdhci-pltfm: SDHCI platform and OF driver helper
[    6.781131] ledtrig-cpu: registered to indicate activity on CPUs
[    6.787381] usbcore: registered new interface driver usbhid
[    6.792980] usbhid: USB HID core driver
[    6.797007] ACPI: IORT: can't find node related to (null) device
[    6.803269] NET: Registered protocol family 17
[    6.807771] 9pnet: Installing 9P2000 support
[    6.812084] Key type dns_resolver registered
[    6.816588] registered taskstats version 1
[    6.884354] rtc-efi rtc-efi: hctosys: unable to read the hardware clock
[    6.891089] ALSA device list:
[    6.894072]   No soundcards found.
[    6.897571] ttyS0 - failed to request DMA
[    6.901858] Freeing unused kernel memory: 840K (ffffff8008a89000 - ffffff8008b5b000)
root@(none)$ ifconfig -a
eth0      Link encap:Ethernet  HWaddr 68:a8:28:2e:c9:10
           BROADCAST MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth1      Link encap:Ethernet  HWaddr 68:a8:28:2e:c9:11
           BROADCAST MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth2      Link encap:Ethernet  HWaddr 9c:37:f4:90:cd:23
           BROADCAST MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth3      Link encap:Ethernet  HWaddr 9c:37:f4:90:cd:24
           BROADCAST MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback
           inet addr:127.0.0.1  Mask:255.0.0.0
           UP LOOPBACK RUNNING  MTU:65536  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1
           RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

root@(none)$ ifconfig eth0 192.168.20.188
[   87.949144] ixgbe 0001:41:00.0: registered PHC device on eth0
root@(none)$ [   88.122426] ixgbe 0001:41:00.0 eth0: detected SFP+: 5
[   88.361631] ixgbe 0001:41:00.0 eth0: NIC Link is Up 10 Gbps, Flow Control: RX/TX

root@(none)$ ping 192.168.20.4 188
PING 192.168.20.4 (192.168.20.4): 56 data bytes
64 bytes from 192.168.20.4: seq=6 ttl=128 time=1.434 ms
64 bytes from 192.168.20.4: seq=7 ttl=128 time=0.296 ms
64 bytes from 192.168.20.4: seq=8 ttl=128 time=0.232 ms
64 bytes from 192.168.20.4: seq=9 ttl=128 time=0.615 ms
64 bytes from 192.168.20.4: seq=10 ttl=128 time=1.197 ms
64 bytes from 192.168.20.4: seq=11 ttl=128 time=1.204 ms
64 bytes from 192.168.20.4: seq=12 ttl=128 time=0.305 ms
64 bytes from 192.168.20.4: seq=13 ttl=128 time=0.379 ms
64 bytes from 192.168.20.4: seq=14 ttl=128 time=0.256 ms
64 bytes from 192.168.20.4: seq=15 ttl=128 time=0.288 ms
64 bytes from 192.168.20.4: seq=16 ttl=128 time=0.343 ms
64 bytes from 192.168.20.4: seq=17 ttl=128 time=0.326 ms
64 bytes from 192.168.20.4: seq=18 ttl=128 time=0.350 ms
64 bytes from 192.168.20.4: seq=19 ttl=128 time=0.427 ms
64 bytes from 192.168.20.4: seq=20 ttl=128 time=0.313 ms
64 bytes from 192.168.20.4: seq=21 ttl=128 time=0.261 ms
64 bytes from 192.168.20.4: seq=22 ttl=128 time=0.376 ms
64 bytes from 192.168.20.4: seq=23 ttl=128 time=0.360 ms
64 bytes from 192.168.20.4: seq=24 ttl=128 time=0.263 ms
64 bytes from 192.168.20.4: seq=25 ttl=128 time=0.380 ms
64 bytes from 192.168.20.4: seq=26 ttl=128 time=0.294 ms
64 bytes from 192.168.20.4: seq=27 ttl=128 time=0.538 ms
64 bytes from 192.168.20.4: seq=28 ttl=128 time=0.321 ms
64 bytes from 192.168.20.4: seq=29 ttl=128 time=0.472 ms
64 bytes from 192.168.20.4: seq=30 ttl=128 time=0.288 ms
64 bytes from 192.168.20.4: seq=31 ttl=128 time=0.268 ms
64 bytes from 192.168.20.4: seq=32 ttl=128 time=0.335 ms
64 bytes from 192.168.20.4: seq=33 ttl=128 time=0.305 ms
64 bytes from 192.168.20.4: seq=34 ttl=128 time=0.697 ms
64 bytes from 192.168.20.4: seq=35 ttl=128 time=0.329 ms
64 bytes from 192.168.20.4: seq=36 ttl=128 time=0.304 ms
64 bytes from 192.168.20.4: seq=37 ttl=128 time=0.773 ms
64 bytes from 192.168.20.4: seq=38 ttl=128 time=0.340 ms
64 bytes from 192.168.20.4: seq=39 ttl=128 time=0.306 ms
64 bytes from 192.168.20.4: seq=40 ttl=128 time=0.821 ms
64 bytes from 192.168.20.4: seq=41 ttl=128 time=0.408 ms
64 bytes from 192.168.20.4: seq=42 ttl=128 time=0.235 ms
64 bytes from 192.168.20.4: seq=43 ttl=128 time=0.954 ms
64 bytes from 192.168.20.4: seq=44 ttl=128 time=0.294 ms
64 bytes from 192.168.20.4: seq=45 ttl=128 time=0.921 ms
64 bytes from 192.168.20.4: seq=46 ttl=128 time=0.974 ms
64 bytes from 192.168.20.4: seq=47 ttl=128 time=0.409 ms
64 bytes from 192.168.20.4: seq=48 ttl=128 time=0.990 ms
64 bytes from 192.168.20.4: seq=49 ttl=128 time=0.306 ms
64 bytes from 192.168.20.4: seq=50 ttl=128 time=1.021 ms
64 bytes from 192.168.20.4: seq=51 ttl=128 time=0.290 ms
64 bytes from 192.168.20.4: seq=52 ttl=128 time=0.304 ms
64 bytes from 192.168.20.4: seq=53 ttl=128 time=0.323 ms
64 bytes from 192.168.20.4: seq=54 ttl=128 time=0.305 ms
64 bytes from 192.168.20.4: seq=55 ttl=128 time=0.371 ms
64 bytes from 192.168.20.4: seq=56 ttl=128 time=0.329 ms
64 bytes from 192.168.20.4: seq=57 ttl=128 time=0.322 ms
64 bytes from 192.168.20.4: seq=58 ttl=128 time=0.320 ms
64 bytes

Re: [PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Tomasz Nowicki]

Hi Bjorn,

On 08.06.2016 02:15, Bjorn Helgaas wrote:
> Hi Tomasz,
>
> On Mon, May 30, 2016 at 05:14:18PM +0200, Tomasz Nowicki wrote:
>> PCI core code provides a config option (CONFIG_PCI_DOMAINS_GENERIC)
>> that allows assigning the PCI bus domain number generically by
>> relying on device tree bindings, and falling back to a simple counter
>> when the respective DT properties (ie "linux,pci-domain") are not
>> specified in the host bridge device tree node.
>>
>> In a similar way, when a system is booted through ACPI, architectures
>> that are selecting CONFIG_PCI_DOMAINS_GENERIC (ie ARM64) require kernel
>> hooks to retrieve the domain number so that the PCI bus domain number
>> set-up can be handled seamlessly with DT and ACPI in generic core code
>> when CONFIG_PCI_DOMAINS_GENERIC is selected.
>>
>> Since currently it is not possible to retrieve a pointer to the PCI
>> host bridge ACPI device backing the host bridge from core PCI code
>> (which would allow retrieving the domain number in an arch agnostic
>> way through the ACPI _SEG method), an arch specific ACPI hook has to
>> be declared and implemented by all arches that rely on
>> CONFIG_PCI_DOMAINS_GENERIC to retrieve the domain number and set it
>> up in core PCI code.
>>
>> For the aforementioned reasons, this patch introduces a dummy
>> acpi_pci_bus_domain_nr() hook in preparation for per-arch implementation
>> of the same to retrieve the domain number on a per-arch basis when
>> the system boots through ACPI.
>>
>> For the sake of code clarity the current code implementing generic
>> domain number assignment (ie pci_bus_assign_domain_nr(), selected by
>> CONFIG_PCI_DOMAINS_GENERIC) is reshuffled so that the code implementing
>> the DT domain assignment function is stubbed out into a corresponding
>> helper, so that DT and ACPI functions are clearly separated in
>> preparation for arches acpi_pci_bus_domain_nr() implementations.
>>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
>> ---
>>   drivers/pci/pci.c   | 11 +++++++++--
>>   include/linux/pci.h |  1 +
>>   2 files changed, 10 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
>> index eb431b5..2b52178 100644
>> --- a/drivers/pci/pci.c
>> +++ b/drivers/pci/pci.c
>> @@ -7,6 +7,7 @@
>>    *	Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
>>    */
>>
>> +#include <linux/acpi.h>
>>   #include <linux/kernel.h>
>>   #include <linux/delay.h>
>>   #include <linux/init.h>
>> @@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
>>   }
>>
>>   #ifdef CONFIG_PCI_DOMAINS_GENERIC
>> -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
>> +static int of_pci_bus_domain_nr(struct device *parent)
>
> Can we do a little cleanup before this patch?
>
>    - pci_bus_assign_domain_nr() is only used inside drivers/pci, so
>      maybe we move the prototype to drivers/pci/pci.h?

pci_bus_assign_domain_nr() goes with pci_domain_nr() as an option for 
CONFIG_PCI_DOMAINS_GENERIC. pci_domain_nr() is used commonly outside 
drivers/pci so we would need to split these calls then, thus personally 
I think it would be better to keep both in inclue/linux/pci.h

>
>    - I don't really like the style of calling a function that
>      internally assigns bus->domain_nr.  Could we do something like
>      this instead?
>
>      int pci_bus_domain_nr(...)
>      {
>        ...
>        return domain;
>      }
>
>      ... pci_create_root_bus(...)
>      {
>        ...
>        b->domain_nr = pci_bus_domain_nr(...);


We can. I do not see much difference between pci_bus_domain_nr() and 
pci_domain_nr() which we already have so how about calling it 
pci_bus_find_domain_nr instead? Lorenzo, any strong preference for it?

>
> That would be two new patches, if this makes sense.
>
> And this patch would only rename pci_bus_assign_domain_nr() to
> of_pci_bus_domain_nr() and add the pci_bus_domain_nr() wrapper.

Giving that we would keep prototypes in inclue/linux/pci.h

1. First patch would rename pci_bus_assign_domain_nr() to 
pci_bus_find_domain_nr() and it would return domain number so that we 
could do:
      ... pci_create_root_bus(...)
      {
        ...
        b->domain_nr = pci_bus_domain_nr(...);
        ...
      }

2. Second patch would transform pci_bus_find_domain_nr() to be the 
wrapper for of_pci_bus_domain_nr()

3. Third patch would add stub definition, the ARM64 definition and the 
new call acpi_pci_bus_domain_nr() in pci_bus_find_domain_nr() wrapper.

Does this plan sound reasonable?

>
>>   {
>>   	static int use_dt_domains = -1;
>>   	int domain = -1;
>> @@ -4985,7 +4986,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
>>   		domain = -1;
>>   	}
>>
>> -	bus->domain_nr = domain;
>> +	return domain;
>> +}
>> +
>> +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
>> +{
>> +	bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
>> +					 acpi_pci_bus_domain_nr(bus);
>>   }
>>   #endif
>>   #endif
>> diff --git a/include/linux/pci.h b/include/linux/pci.h
>> index 12349de..bba4053 100644
>> --- a/include/linux/pci.h
>> +++ b/include/linux/pci.h
>> @@ -1390,6 +1390,7 @@ static inline int pci_domain_nr(struct pci_bus *bus)
>>   {
>>   	return bus->domain_nr;
>>   }
>> +static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return -1; }
>
> I would split the addition of acpi_pci_bus_domain_nr() to a separate
> patch and include the ARM64 definition in that same patch.  That patch
> would only add this stub definition, the ARM64 definition, and the new
> call in pci_bus_domain_nr().

OK

Thanks,
Tomasz

Re: [PATCH V8 7/9] acpi: Add generic MCFG table handling

[Tomasz Nowicki]

On 08.06.2016 03:56, Bjorn Helgaas wrote:
> On Mon, May 30, 2016 at 05:14:20PM +0200, Tomasz Nowicki wrote:
>> In order to handle PCI config space regions properly in ACPI, new MCFG
>> interface is defined which does sanity checks on MCFG table and keeps its
>> root pointer. The user is able to lookup MCFG regions based on
>> host bridge root structure and domain:bus_start:bus_end touple.
>> Use pci_mmcfg_late_init old prototype to avoid another function name.
>>
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Signed-off-by: Jayachandran C <jchandra@broadcom.com>
>> ---
>>   drivers/acpi/Kconfig     |  3 ++
>>   drivers/acpi/Makefile    |  1 +
>>   drivers/acpi/pci_mcfg.c  | 94 ++++++++++++++++++++++++++++++++++++++++++++++++
>>   include/linux/pci-acpi.h |  2 ++
>>   include/linux/pci.h      |  2 +-
>>   5 files changed, 101 insertions(+), 1 deletion(-)
>>   create mode 100644 drivers/acpi/pci_mcfg.c
>>
>> diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
>> index b7e2e77..f98c328 100644
>> --- a/drivers/acpi/Kconfig
>> +++ b/drivers/acpi/Kconfig
>> @@ -217,6 +217,9 @@ config ACPI_PROCESSOR_IDLE
>>   	bool
>>   	select CPU_IDLE
>>
>> +config ACPI_MCFG
>> +	bool
>> +
>>   config ACPI_CPPC_LIB
>>   	bool
>>   	depends on ACPI_PROCESSOR
>> diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
>> index 251ce85..632e81f 100644
>> --- a/drivers/acpi/Makefile
>> +++ b/drivers/acpi/Makefile
>> @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
>>   acpi-y				+= ec.o
>>   acpi-$(CONFIG_ACPI_DOCK)	+= dock.o
>>   acpi-y				+= pci_root.o pci_link.o pci_irq.o
>> +obj-$(CONFIG_ACPI_MCFG)		+= pci_mcfg.o
>>   acpi-y				+= acpi_lpss.o acpi_apd.o
>>   acpi-y				+= acpi_platform.o
>>   acpi-y				+= acpi_pnp.o
>> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
>> new file mode 100644
>> index 0000000..1847f74
>> --- /dev/null
>> +++ b/drivers/acpi/pci_mcfg.c
>> @@ -0,0 +1,94 @@
>> +/*
>> + * Copyright (C) 2016 Broadcom
>> + *	Author: Jayachandran C <jchandra@broadcom.com>
>> + * Copyright (C) 2016 Semihalf
>> + * 	Author: Tomasz Nowicki <tn@semihalf.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License, version 2, as
>> + * published by the Free Software Foundation (the "GPL").
>> + *
>> + * This program is distributed in the hope that it will be useful, but
>> + * WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License version 2 (GPLv2) for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * version 2 (GPLv2) along with this source code.
>> + */
>> +
>> +#define pr_fmt(fmt) "ACPI: " fmt
>> +
>> +#include <linux/kernel.h>
>> +#include <linux/pci.h>
>> +#include <linux/pci-acpi.h>
>> +
>> +/* Root pointer to the mapped MCFG table */
>> +static struct acpi_table_mcfg *mcfg_table;
>> +static int mcfg_entries;
>> +
>> +int pci_mcfg_lookup(struct acpi_pci_root *root)
>
> I think this would be better if we passed in the domain and a pointer
> to the bus range resource and returned the ECAM base address.  I don't
> think we need to be connected to struct acpi_pci_root.

I will use domain and bus range resource as you suggested.

>
>> +{
>> +	struct acpi_mcfg_allocation *mptr, *entry = NULL;
>> +	struct resource *bus_res = &root->secondary;
>> +	int i;
>> +
>> +	if (mcfg_table) {
>> +		mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
>> +		for (i = 0; i < mcfg_entries && !entry; i++, mptr++)
>> +			if (mptr->pci_segment == root->segment &&
>> +			    mptr->start_bus_number == bus_res->start)
>> +				entry = mptr;
>> +	}
>> +
>> +	/* not found, use _CBA if available, else error */
>> +	if (!entry) {
>> +		if (root->mcfg_addr)
>> +			return root->mcfg_addr;
>> +		pr_err("%04x:%pR MCFG lookup failed\n", root->segment, bus_res);
>> +		return -ENOENT;
>> +	} else if (root->mcfg_addr && entry->address != root->mcfg_addr) {
>> +		pr_warn("%04x:%pR CBA %pa != MCFG %lx, using CBA\n",
>> +			root->segment, bus_res, &root->mcfg_addr,
>> +			(unsigned long)entry->address);
>> +		return 0;
>> +	}
>
> I keep looking at this code, trying to find where we "use _CBA", but I
> can't find it.  Oh, I see, acpi_pci_root_add() calls
> acpi_pci_root_get_mcfg_addr() (which evaluates _CBA), and sets
> root->mcfg_addr to the result.
>
> "root->mcfg_addr" is sort of an unfortunate name because "MCFG" is the
> ACPI table name, we've used "ECAM" for the memory-mapped config space,
> and this address can come from either the MCFG table or the _CBA
> method.
>
> In the case where we have both _CBA and an MCFG entry, I think we
> should prefer _CBA, and I'm not sure it's even worth warning about it.
> So I think you could simplify this function if you made
> pci_acpi_setup_ecam_mapping() look like this:
>
>    ... pci_acpi_setup_ecam_mapping(...)
>    {
>      ...
>      if (!root->mcfg_addr)
>        root->mcfg_addr = pci_mcfg_lookup(root);
>
>      if (!root->mcfg_addr) {
>        dev_err(..., "no ECAM region found\n");
>        return -EINVAL;
>      }

OK

>
>> +
>> +	/* found matching entry, bus range check */
>> +	if (entry->end_bus_number != bus_res->end) {
>> +		resource_size_t bus_end = min_t(resource_size_t,
>> +					entry->end_bus_number, bus_res->end);
>> +		pr_warn("%04x:%pR bus end mismatch, using %02lx\n",
>> +			root->segment, bus_res, (unsigned long)bus_end);
>> +		bus_res->end = bus_end;
>> +	}

What about bus end mismatch case? Should we trim the host bridge bus 
range or expect MCFG entry covers that range? Sometimes we get _BBN-0xFF 
bus range, not from _CRS.

>> +
>> +	if (!root->mcfg_addr)
>> +		root->mcfg_addr = entry->address;
>
> Please move the assignment to the caller (I think Lorenzo pointed this
> out already).
>
>> +	return 0;
>> +}
>> +
>> +static __init int pci_mcfg_parse(struct acpi_table_header *header)
>> +{
>> +	if (header->length < sizeof(struct acpi_table_mcfg))
>> +		return -EINVAL;
>> +
>> +	mcfg_entries = (header->length - sizeof(struct acpi_table_mcfg)) /
>> +					sizeof(struct acpi_mcfg_allocation);
>> +	if (mcfg_entries == 0) {
>> +		pr_err("MCFG has no entries\n");
>
> Include an address here?  I'm not really sure either of the messages
> here is necessary.  Users (callers of pci_mcfg_lookup()) will notice
> if we can't find any ECAM space and will probably complain there,
> where the message can include more information, e.g., the affected
> device.

I would keep message about how many entries we found here. It would be 
valuable information IMO.

>
>> +		return -EINVAL;
>> +	}
>> +
>> +	mcfg_table = (struct acpi_table_mcfg *)header;
>> +	pr_info("MCFG table detected, %d entries\n", mcfg_entries);
>> +	return 0;
>> +}
>> +
>> +/* Interface called by ACPI - parse and save MCFG table */
>
> I think we save a *pointer* to the MCFG table, not the table itself.

Right, the comment is broken.

>
> And acpi_table_parse() calls early_acpi_os_unmap_memory() immediately
> after it calls pci_mcfg_parse(), so I'm doubtful that the pointer
> remains valid.

At this stage early_acpi_os_unmap_memory() is doing nothing since 
acpi_early_init() set acpi_gbl_permanent_mmap to 1 way before. The 
pointer is fine then.

Thanks,
Tomasz

Re: [PATCH V8 7/9] acpi: Add generic MCFG table handling

[Bjorn Helgaas]

On Wed, Jun 08, 2016 at 02:21:30PM +0200, Tomasz Nowicki wrote:
> On 08.06.2016 03:56, Bjorn Helgaas wrote:
> >On Mon, May 30, 2016 at 05:14:20PM +0200, Tomasz Nowicki wrote:
> >>In order to handle PCI config space regions properly in ACPI, new MCFG
> >>interface is defined which does sanity checks on MCFG table and keeps its
> >>root pointer. The user is able to lookup MCFG regions based on
> >>host bridge root structure and domain:bus_start:bus_end touple.
> >>Use pci_mmcfg_late_init old prototype to avoid another function name.

> >>+	/* found matching entry, bus range check */
> >>+	if (entry->end_bus_number != bus_res->end) {
> >>+		resource_size_t bus_end = min_t(resource_size_t,
> >>+					entry->end_bus_number, bus_res->end);
> >>+		pr_warn("%04x:%pR bus end mismatch, using %02lx\n",
> >>+			root->segment, bus_res, (unsigned long)bus_end);
> >>+		bus_res->end = bus_end;
> >>+	}
> 
> What about bus end mismatch case? Should we trim the host bridge bus
> range or expect MCFG entry covers that range? Sometimes we get
> _BBN-0xFF bus range, not from _CRS.

Lack of a bus range in _CRS is a firmware defect.  There's a comment
about this in acpi_pci_root_add().  On x86, we probably had to live
with firmware in the field that had this defect.  I think we should
expect all ARM64 systems to provide a bus number range in _CRS, and
fail the attach if it's not there.

I don't think we should warn about an MCFG entry that covers more than
the _CRS bus range.  On x86, it's common to have something like:

  ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f])
  ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 80-ff])

with a single MCFG entry that covers [bus 00-ff].  That seems
reasonable and I don't think it's worth warning about it.

If the MCFG entry doesn't cover all of a _CRS bus range, we should
just fail so we can find and fix broken firmware.

> >>+/* Interface called by ACPI - parse and save MCFG table */
> >
> >I think we save a *pointer* to the MCFG table, not the table itself.
> 
> Right, the comment is broken.
> 
> >And acpi_table_parse() calls early_acpi_os_unmap_memory() immediately
> >after it calls pci_mcfg_parse(), so I'm doubtful that the pointer
> >remains valid.
> 
> At this stage early_acpi_os_unmap_memory() is doing nothing since
> acpi_early_init() set acpi_gbl_permanent_mmap to 1 way before. The
> pointer is fine then.

Hmmm...  I see your argument, but this is a problem waiting to happen.
We should not depend on the internal implementation of
early_acpi_os_unmap_memory().  The pattern of:

  y = x;
  unmap(x);
  z = *y;

is just broken and we shouldn't expect readers to recognize that "oh,
unmap() isn't really unmapping anything in this special case, so this
looks wrong but is really fine."

Bjorn

Re: [PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Bjorn Helgaas]

On Wed, Jun 08, 2016 at 12:21:19PM +0200, Tomasz Nowicki wrote:
> On 08.06.2016 02:15, Bjorn Helgaas wrote:
> >On Mon, May 30, 2016 at 05:14:18PM +0200, Tomasz Nowicki wrote:
> >>PCI core code provides a config option (CONFIG_PCI_DOMAINS_GENERIC)
> >>that allows assigning the PCI bus domain number generically by
> >>relying on device tree bindings, and falling back to a simple counter
> >>when the respective DT properties (ie "linux,pci-domain") are not
> >>specified in the host bridge device tree node.
> >>
> >>In a similar way, when a system is booted through ACPI, architectures
> >>that are selecting CONFIG_PCI_DOMAINS_GENERIC (ie ARM64) require kernel
> >>hooks to retrieve the domain number so that the PCI bus domain number
> >>set-up can be handled seamlessly with DT and ACPI in generic core code
> >>when CONFIG_PCI_DOMAINS_GENERIC is selected.
> >>
> >>Since currently it is not possible to retrieve a pointer to the PCI
> >>host bridge ACPI device backing the host bridge from core PCI code
> >>(which would allow retrieving the domain number in an arch agnostic
> >>way through the ACPI _SEG method), an arch specific ACPI hook has to
> >>be declared and implemented by all arches that rely on
> >>CONFIG_PCI_DOMAINS_GENERIC to retrieve the domain number and set it
> >>up in core PCI code.
> >>
> >>For the aforementioned reasons, this patch introduces a dummy
> >>acpi_pci_bus_domain_nr() hook in preparation for per-arch implementation
> >>of the same to retrieve the domain number on a per-arch basis when
> >>the system boots through ACPI.
> >>
> >>For the sake of code clarity the current code implementing generic
> >>domain number assignment (ie pci_bus_assign_domain_nr(), selected by
> >>CONFIG_PCI_DOMAINS_GENERIC) is reshuffled so that the code implementing
> >>the DT domain assignment function is stubbed out into a corresponding
> >>helper, so that DT and ACPI functions are clearly separated in
> >>preparation for arches acpi_pci_bus_domain_nr() implementations.
> >>
> >>Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> >>Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> >>---
> >>  drivers/pci/pci.c   | 11 +++++++++--
> >>  include/linux/pci.h |  1 +
> >>  2 files changed, 10 insertions(+), 2 deletions(-)
> >>
> >>diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> >>index eb431b5..2b52178 100644
> >>--- a/drivers/pci/pci.c
> >>+++ b/drivers/pci/pci.c
> >>@@ -7,6 +7,7 @@
> >>   *	Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
> >>   */
> >>
> >>+#include <linux/acpi.h>
> >>  #include <linux/kernel.h>
> >>  #include <linux/delay.h>
> >>  #include <linux/init.h>
> >>@@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
> >>  }
> >>
> >>  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> >>-void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> >>+static int of_pci_bus_domain_nr(struct device *parent)
> >
> >Can we do a little cleanup before this patch?
> >
> >   - pci_bus_assign_domain_nr() is only used inside drivers/pci, so
> >     maybe we move the prototype to drivers/pci/pci.h?
> 
> pci_bus_assign_domain_nr() goes with pci_domain_nr() as an option
> for CONFIG_PCI_DOMAINS_GENERIC. pci_domain_nr() is used commonly
> outside drivers/pci so we would need to split these calls then, thus
> personally I think it would be better to keep both in
> inclue/linux/pci.h

OK.

> >   - I don't really like the style of calling a function that
> >     internally assigns bus->domain_nr.  Could we do something like
> >     this instead?
> >
> >     int pci_bus_domain_nr(...)
> >     {
> >       ...
> >       return domain;
> >     }
> >
> >     ... pci_create_root_bus(...)
> >     {
> >       ...
> >       b->domain_nr = pci_bus_domain_nr(...);
> 
> 
> We can. I do not see much difference between pci_bus_domain_nr() and
> pci_domain_nr() which we already have so how about calling it
> pci_bus_find_domain_nr instead? Lorenzo, any strong preference for
> it?

I suggested that to make them all parallel:

  pci_bus_domain_nr()
  of_pci_bus_domain_nr()
  acpi_pci_bus_domain_nr()

If you want to make them all *pci_bus_find_domain_nr() that would be
OK with me, but I think it is worth keeping them the same except for
the prefix (none/"of_"/"acpi_").  I think that's what you suggested
below.

> >That would be two new patches, if this makes sense.
> >
> >And this patch would only rename pci_bus_assign_domain_nr() to
> >of_pci_bus_domain_nr() and add the pci_bus_domain_nr() wrapper.
> 
> Giving that we would keep prototypes in inclue/linux/pci.h
> 
> 1. First patch would rename pci_bus_assign_domain_nr() to
> pci_bus_find_domain_nr() and it would return domain number so that
> we could do:
>      ... pci_create_root_bus(...)
>      {
>        ...
>        b->domain_nr = pci_bus_domain_nr(...);
>        ...
>      }
> 
> 2. Second patch would transform pci_bus_find_domain_nr() to be the
> wrapper for of_pci_bus_domain_nr()
> 
> 3. Third patch would add stub definition, the ARM64 definition and
> the new call acpi_pci_bus_domain_nr() in pci_bus_find_domain_nr()
> wrapper.
> 
> Does this plan sound reasonable?

Sounds perfect!

Re: [PATCH V8 7/9] acpi: Add generic MCFG table handling

[Tomasz Nowicki]

On 08.06.2016 15:17, Bjorn Helgaas wrote:
> On Wed, Jun 08, 2016 at 02:21:30PM +0200, Tomasz Nowicki wrote:
>> On 08.06.2016 03:56, Bjorn Helgaas wrote:
>>> On Mon, May 30, 2016 at 05:14:20PM +0200, Tomasz Nowicki wrote:
>>>> In order to handle PCI config space regions properly in ACPI, new MCFG
>>>> interface is defined which does sanity checks on MCFG table and keeps its
>>>> root pointer. The user is able to lookup MCFG regions based on
>>>> host bridge root structure and domain:bus_start:bus_end touple.
>>>> Use pci_mmcfg_late_init old prototype to avoid another function name.
>
>>>> +	/* found matching entry, bus range check */
>>>> +	if (entry->end_bus_number != bus_res->end) {
>>>> +		resource_size_t bus_end = min_t(resource_size_t,
>>>> +					entry->end_bus_number, bus_res->end);
>>>> +		pr_warn("%04x:%pR bus end mismatch, using %02lx\n",
>>>> +			root->segment, bus_res, (unsigned long)bus_end);
>>>> +		bus_res->end = bus_end;
>>>> +	}
>>
>> What about bus end mismatch case? Should we trim the host bridge bus
>> range or expect MCFG entry covers that range? Sometimes we get
>> _BBN-0xFF bus range, not from _CRS.
>
> Lack of a bus range in _CRS is a firmware defect.  There's a comment
> about this in acpi_pci_root_add().  On x86, we probably had to live
> with firmware in the field that had this defect.  I think we should
> expect all ARM64 systems to provide a bus number range in _CRS, and
> fail the attach if it's not there.
>
> I don't think we should warn about an MCFG entry that covers more than
> the _CRS bus range.  On x86, it's common to have something like:
>
>    ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f])
>    ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 80-ff])
>
> with a single MCFG entry that covers [bus 00-ff].  That seems
> reasonable and I don't think it's worth warning about it.
>
> If the MCFG entry doesn't cover all of a _CRS bus range, we should
> just fail so we can find and fix broken firmware.

Make sense to me.

>
>>>> +/* Interface called by ACPI - parse and save MCFG table */
>>>
>>> I think we save a *pointer* to the MCFG table, not the table itself.
>>
>> Right, the comment is broken.
>>
>>> And acpi_table_parse() calls early_acpi_os_unmap_memory() immediately
>>> after it calls pci_mcfg_parse(), so I'm doubtful that the pointer
>>> remains valid.
>>
>> At this stage early_acpi_os_unmap_memory() is doing nothing since
>> acpi_early_init() set acpi_gbl_permanent_mmap to 1 way before. The
>> pointer is fine then.
>
> Hmmm...  I see your argument, but this is a problem waiting to happen.
> We should not depend on the internal implementation of
> early_acpi_os_unmap_memory().  The pattern of:
>
>    y = x;
>    unmap(x);
>    z = *y;
>
> is just broken and we shouldn't expect readers to recognize that "oh,
> unmap() isn't really unmapping anything in this special case, so this
> looks wrong but is really fine."
>

Right, so we are back to MCFG cache.

Thanks,
Tomasz

Re: [PATCH V8 0/9] Support for ARM64 ACPI based PCI host controller

[Suravee Suthikulanit]

On 5/30/2016 10:14 AM, Tomasz Nowicki wrote:
> From the functionality point of view this series may be split into the
> following logic parts:
> 1. Export ECAM API and add parent device to pci_config_window
> 2. Add IO resources handling to PCI core code
> 3. Support for generic domain assignment based on ACPI
> 4. New MCFG driver
> 5. Implement ARM64 ACPI based PCI host controller driver under arch/arm64/
>
> Patches has been built on top of 4.7-rc1 and can be found here:
> git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v8)
>
> This has been tested on Cavium ThunderX server. Any help in reviewing and
> testing is very appreciated.
>
> v7 -> v8
> - move code from drivers/acpi/pci_root_generic.c to arch/arm64/kernel/pci.c
> - minor changes around domain assignment
> - pci_mcfg.c improvements for parsing MCFG tables and lookup its entries
>
> v6 -> v7
> - drop quirks handling
> - changes for ACPI companion and domain number assignment approach
> - implement arch pcibios_{add|remove}_bus and call acpi_pci_{add|remove}_bus from there
> - cleanups around nomenclature
> - use resources oriented API for ECAM
> - fix for based address calculation before mapping ECAM region
> - remove useless lock for MCFG lookup
> - move MCFG stuff to separated file pci_mcfg.c
> - drop MCFG entries caching
> - rebase against 4.6-rc7
>
> v5 -> v6
> - drop idea of x86 MMCONFIG code refactoring
> - integrate JC's patches which introduce new ECAM API:
>   https://lkml.org/lkml/2016/4/11/907
>   git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
> - integrate Sinan's fix for releasing IO resources, see patch [06/13]
> - added ACPI support for ThunderX ECAM and PEM drivers
> - rebase against 4.6-rc2
>
> v4 -> v5
> - drop MCFG refactoring group patches 1-6 from series v4 and integrate Jayachandran's patch
>   https://patchwork.ozlabs.org/patch/575525/
> - rewrite PCI legacy IRQs allocation
> - squash two patches 11 and 12 from series v4, fixed bisection issue
> - changelog improvements
> - rebase against 4.5-rc3
>
> v3 -> v4
> - drop Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
> - add Lorenzo's fix patch 19/24
> - ACPI PCI bus domain number assigning cleanup
> - change resource management, we now claim and reassign resources
> - improvements for applying quirks
> - drop Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
> - rebase against 4.5-rc1
>
> v2 -> v3
> - fix legacy IRQ assigning and IO ports registration
> - remove reference to arch specific companion device for ia64
> - move ACPI PCI host controller driver to pci_root.c
> - drop generic domain assignment for x86 and ia64 as I am not
>   able to run all necessary test variants
> - drop patch which cleaned legacy IRQ assignment since it belongs to
>   Mathew's series:
>   https://patchwork.ozlabs.org/patch/557504/
> - extend MCFG quirk code
> - rebase against 4.4
>
> v1 -> v2
> - move non-arch specific piece of code to dirver/acpi/ directory
> - fix IO resource handling
> - introduce PCI config accessors quirks matching
> - moved ACPI_COMPANION_SET to generic code
>
> v1 - https://lkml.org/lkml/2015/10/27/504
> v2 - https://lkml.org/lkml/2015/12/16/246
> v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
> v4 - https://lkml.org/lkml/2016/2/4/646
> v5 - https://lkml.org/lkml/2016/2/16/426
> v6 - https://lkml.org/lkml/2016/4/15/594
>
> Jayachandran C (2):
>   PCI: ecam: move ecam.h to linux/include/pci-ecam.h
>   PCI: ecam: Add parent device field to pci_config_window
>
> Tomasz Nowicki (7):
>   pci: Add new function to unmap IO resources.
>   acpi, pci: Support IO resources when parsing PCI host bridge
>     resources.
>   pci, acpi: add acpi hook to assign domain number.
>   arm64, pci, acpi: ACPI support for legacy IRQs parsing and
>     consolidation with DT code.
>   acpi: Add generic MCFG table handling
>   arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus
>     enumeration.
>   pci, acpi: ARM64 support for ACPI based generic PCI host controller
>
>  arch/arm64/Kconfig                  |   2 +
>  arch/arm64/kernel/pci.c             | 143 ++++++++++++++++++++++++++++++++++--
>  drivers/acpi/Kconfig                |   3 +
>  drivers/acpi/Makefile               |   1 +
>  drivers/acpi/pci_mcfg.c             |  94 ++++++++++++++++++++++++
>  drivers/acpi/pci_root.c             |  39 ++++++++++
>  drivers/pci/ecam.c                  |   6 +-
>  drivers/pci/ecam.h                  |  67 -----------------
>  drivers/pci/host/pci-host-common.c  |   3 +-
>  drivers/pci/host/pci-host-generic.c |   3 +-
>  drivers/pci/host/pci-thunder-ecam.c |   3 +-
>  drivers/pci/host/pci-thunder-pem.c  |   6 +-
>  drivers/pci/pci.c                   |  29 +++++++-
>  include/linux/pci-acpi.h            |   2 +
>  include/linux/pci-ecam.h            |  67 +++++++++++++++++
>  include/linux/pci.h                 |   9 ++-
>  16 files changed, 387 insertions(+), 90 deletions(-)
>  create mode 100644 drivers/acpi/pci_mcfg.c
>  delete mode 100644 drivers/pci/ecam.h
>  create mode 100644 include/linux/pci-ecam.h
>

For the series,

Tested-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>

Thanks,
Suravee

Re: [PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Lorenzo Pieralisi]

Hi Bjorn, Tomasz,

On Tue, Jun 07, 2016 at 07:15:59PM -0500, Bjorn Helgaas wrote:

[...]

> > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > index eb431b5..2b52178 100644
> > --- a/drivers/pci/pci.c
> > +++ b/drivers/pci/pci.c
> > @@ -7,6 +7,7 @@
> >   *	Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
> >   */
> >  
> > +#include <linux/acpi.h>
> >  #include <linux/kernel.h>
> >  #include <linux/delay.h>
> >  #include <linux/init.h>
> > @@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
> >  }
> >  
> >  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> > -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > +static int of_pci_bus_domain_nr(struct device *parent)
> 
> Can we do a little cleanup before this patch?  
> 
>   - pci_bus_assign_domain_nr() is only used inside drivers/pci, so
>     maybe we move the prototype to drivers/pci/pci.h?
> 
>   - I don't really like the style of calling a function that
>     internally assigns bus->domain_nr.  Could we do something like
>     this instead?
> 
>     int pci_bus_domain_nr(...)
>     {
>       ...
>       return domain;
>     }
> 
>     ... pci_create_root_bus(...)
>     {
>       ...
>       b->domain_nr = pci_bus_domain_nr(...);

We noticed while preparing v9, that this would force us to
write an empty pci_bus_domain_nr() prototype for
!PCI_DOMAINS_GENERIC (ie every arch but ARM/ARM64) that should
return 0 to keep current behaviour unchanged.

That's why pci_bus_assign_domain_nr() was there, so that it
was compiled out on !PCI_DOMAINS_GENERIC.

I really would like v9 to be final so let's fix it before posting it
shortly please.

For the above we have three options:

1) Leave code as-is in v8

2) in pci_create_root_bus():
#ifdef CONFIG_PCI_DOMAINS_GENERIC
       b->domain_nr = pci_bus_domain_nr(...);
#endif

+ other changes requested above

3) in pci_create_root_bus()

b->domain_nr = pci_bus_domain_nr(...);

unguarded and a stub:

#ifndef CONFIG_PCI_DOMAINS_GENERIC
static inline int pci_bus_domain_nr() { return 0; }
#endif

+ other changes requested above

I think it is a matter of details and frankly I am not sure (2) or (3)
make things much cleaner than they are (given that as you know with
Arnd's changes we will rewrite this code anyway), so please let's
choose an option so that we can post v9 and hopefully this series is
ready to go.

Thanks !
Lorenzo

Re: [PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Lorenzo Pieralisi]

On Fri, Jun 10, 2016 at 04:14:58PM +0100, Lorenzo Pieralisi wrote:
> Hi Bjorn, Tomasz,
> 
> On Tue, Jun 07, 2016 at 07:15:59PM -0500, Bjorn Helgaas wrote:
> 
> [...]
> 
> > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > > index eb431b5..2b52178 100644
> > > --- a/drivers/pci/pci.c
> > > +++ b/drivers/pci/pci.c
> > > @@ -7,6 +7,7 @@
> > >   *	Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
> > >   */
> > >  
> > > +#include <linux/acpi.h>
> > >  #include <linux/kernel.h>
> > >  #include <linux/delay.h>
> > >  #include <linux/init.h>
> > > @@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
> > >  }
> > >  
> > >  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> > > -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> > > +static int of_pci_bus_domain_nr(struct device *parent)
> > 
> > Can we do a little cleanup before this patch?  
> > 
> >   - pci_bus_assign_domain_nr() is only used inside drivers/pci, so
> >     maybe we move the prototype to drivers/pci/pci.h?
> > 
> >   - I don't really like the style of calling a function that
> >     internally assigns bus->domain_nr.  Could we do something like
> >     this instead?
> > 
> >     int pci_bus_domain_nr(...)
> >     {
> >       ...
> >       return domain;
> >     }
> > 
> >     ... pci_create_root_bus(...)
> >     {
> >       ...
> >       b->domain_nr = pci_bus_domain_nr(...);
> 
> We noticed while preparing v9, that this would force us to
> write an empty pci_bus_domain_nr() prototype for
> !PCI_DOMAINS_GENERIC (ie every arch but ARM/ARM64) that should
> return 0 to keep current behaviour unchanged.
> 
> That's why pci_bus_assign_domain_nr() was there, so that it
> was compiled out on !PCI_DOMAINS_GENERIC.
> 
> I really would like v9 to be final so let's fix it before posting it
> shortly please.
> 
> For the above we have three options:
> 
> 1) Leave code as-is in v8
> 
> 2) in pci_create_root_bus():
> #ifdef CONFIG_PCI_DOMAINS_GENERIC
>        b->domain_nr = pci_bus_domain_nr(...);
> #endif
> 
> + other changes requested above
> 
> 3) in pci_create_root_bus()
> 
> b->domain_nr = pci_bus_domain_nr(...);
> 
> unguarded and a stub:
> 
> #ifndef CONFIG_PCI_DOMAINS_GENERIC
> static inline int pci_bus_domain_nr() { return 0; }
> #endif
> 
> + other changes requested above

Actually, Tomasz made me notice that pci_bus.domain_nr is
only declared for CONFIG_PCI_DOMAINS_GENERIC so (3) is not
even an option and IMO (2) is not much nicer than code in
v8 as-is with an ifdef in the middle of pci_create_root_bus().

Thanks,
Lorenzo

Re: [PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Tomasz Nowicki]

On 10.06.2016 17:49, Lorenzo Pieralisi wrote:
> On Fri, Jun 10, 2016 at 04:14:58PM +0100, Lorenzo Pieralisi wrote:
>> Hi Bjorn, Tomasz,
>>
>> On Tue, Jun 07, 2016 at 07:15:59PM -0500, Bjorn Helgaas wrote:
>>
>> [...]
>>
>>>> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
>>>> index eb431b5..2b52178 100644
>>>> --- a/drivers/pci/pci.c
>>>> +++ b/drivers/pci/pci.c
>>>> @@ -7,6 +7,7 @@
>>>>    *	Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
>>>>    */
>>>>
>>>> +#include <linux/acpi.h>
>>>>   #include <linux/kernel.h>
>>>>   #include <linux/delay.h>
>>>>   #include <linux/init.h>
>>>> @@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
>>>>   }
>>>>
>>>>   #ifdef CONFIG_PCI_DOMAINS_GENERIC
>>>> -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
>>>> +static int of_pci_bus_domain_nr(struct device *parent)
>>>
>>> Can we do a little cleanup before this patch?
>>>
>>>    - pci_bus_assign_domain_nr() is only used inside drivers/pci, so
>>>      maybe we move the prototype to drivers/pci/pci.h?
>>>
>>>    - I don't really like the style of calling a function that
>>>      internally assigns bus->domain_nr.  Could we do something like
>>>      this instead?
>>>
>>>      int pci_bus_domain_nr(...)
>>>      {
>>>        ...
>>>        return domain;
>>>      }
>>>
>>>      ... pci_create_root_bus(...)
>>>      {
>>>        ...
>>>        b->domain_nr = pci_bus_domain_nr(...);
>>
>> We noticed while preparing v9, that this would force us to
>> write an empty pci_bus_domain_nr() prototype for
>> !PCI_DOMAINS_GENERIC (ie every arch but ARM/ARM64) that should
>> return 0 to keep current behaviour unchanged.
>>
>> That's why pci_bus_assign_domain_nr() was there, so that it
>> was compiled out on !PCI_DOMAINS_GENERIC.
>>
>> I really would like v9 to be final so let's fix it before posting it
>> shortly please.
>>
>> For the above we have three options:
>>
>> 1) Leave code as-is in v8
>>
>> 2) in pci_create_root_bus():
>> #ifdef CONFIG_PCI_DOMAINS_GENERIC
>>         b->domain_nr = pci_bus_domain_nr(...);
>> #endif
>>
>> + other changes requested above
>>
>> 3) in pci_create_root_bus()
>>
>> b->domain_nr = pci_bus_domain_nr(...);
>>
>> unguarded and a stub:
>>
>> #ifndef CONFIG_PCI_DOMAINS_GENERIC
>> static inline int pci_bus_domain_nr() { return 0; }
>> #endif
>>
>> + other changes requested above
>
> Actually, Tomasz made me notice that pci_bus.domain_nr is
> only declared for CONFIG_PCI_DOMAINS_GENERIC so (3) is not
> even an option and IMO (2) is not much nicer than code in
> v8 as-is with an ifdef in the middle of pci_create_root_bus().
>

To me (1) is nicer too. Bjorn what is your take on this? This is last 
bit before sending v9.

Thanks,
Tomasz

Re: [PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Bjorn Helgaas]

On Fri, Jun 10, 2016 at 06:49:32PM +0200, Tomasz Nowicki wrote:
> On 10.06.2016 17:49, Lorenzo Pieralisi wrote:
> >On Fri, Jun 10, 2016 at 04:14:58PM +0100, Lorenzo Pieralisi wrote:
> >>Hi Bjorn, Tomasz,
> >>
> >>On Tue, Jun 07, 2016 at 07:15:59PM -0500, Bjorn Helgaas wrote:
> >>
> >>[...]
> >>
> >>>>diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> >>>>index eb431b5..2b52178 100644
> >>>>--- a/drivers/pci/pci.c
> >>>>+++ b/drivers/pci/pci.c
> >>>>@@ -7,6 +7,7 @@
> >>>>   *	Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
> >>>>   */
> >>>>
> >>>>+#include <linux/acpi.h>
> >>>>  #include <linux/kernel.h>
> >>>>  #include <linux/delay.h>
> >>>>  #include <linux/init.h>
> >>>>@@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
> >>>>  }
> >>>>
> >>>>  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> >>>>-void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
> >>>>+static int of_pci_bus_domain_nr(struct device *parent)
> >>>
> >>>Can we do a little cleanup before this patch?
> >>>
> >>>   - pci_bus_assign_domain_nr() is only used inside drivers/pci, so
> >>>     maybe we move the prototype to drivers/pci/pci.h?
> >>>
> >>>   - I don't really like the style of calling a function that
> >>>     internally assigns bus->domain_nr.  Could we do something like
> >>>     this instead?
> >>>
> >>>     int pci_bus_domain_nr(...)
> >>>     {
> >>>       ...
> >>>       return domain;
> >>>     }
> >>>
> >>>     ... pci_create_root_bus(...)
> >>>     {
> >>>       ...
> >>>       b->domain_nr = pci_bus_domain_nr(...);
> >>
> >>We noticed while preparing v9, that this would force us to
> >>write an empty pci_bus_domain_nr() prototype for
> >>!PCI_DOMAINS_GENERIC (ie every arch but ARM/ARM64) that should
> >>return 0 to keep current behaviour unchanged.
> >>
> >>That's why pci_bus_assign_domain_nr() was there, so that it
> >>was compiled out on !PCI_DOMAINS_GENERIC.
> >>
> >>I really would like v9 to be final so let's fix it before posting it
> >>shortly please.
> >>
> >>For the above we have three options:
> >>
> >>1) Leave code as-is in v8
> >>
> >>2) in pci_create_root_bus():
> >>#ifdef CONFIG_PCI_DOMAINS_GENERIC
> >>        b->domain_nr = pci_bus_domain_nr(...);
> >>#endif
> >>
> >>+ other changes requested above
> >>
> >>3) in pci_create_root_bus()
> >>
> >>b->domain_nr = pci_bus_domain_nr(...);
> >>
> >>unguarded and a stub:
> >>
> >>#ifndef CONFIG_PCI_DOMAINS_GENERIC
> >>static inline int pci_bus_domain_nr() { return 0; }
> >>#endif
> >>
> >>+ other changes requested above
> >
> >Actually, Tomasz made me notice that pci_bus.domain_nr is
> >only declared for CONFIG_PCI_DOMAINS_GENERIC so (3) is not
> >even an option and IMO (2) is not much nicer than code in
> >v8 as-is with an ifdef in the middle of pci_create_root_bus().
> >
> 
> To me (1) is nicer too. Bjorn what is your take on this? This is
> last bit before sending v9.

My preference is (2).  The ifdef in pci_create_root_bus() is a little
ugly, but I like it better because it will fit nicely into Arnd's
idea of having the native drivers allocate and fill in a host bridge
structure before calling the PCI core.  The domain is one thing those
drivers could fill in.  I like that model much better than having the
PCI core make callbacks to get information that we should have passed
in to begin with.

The current code suggests that assigning the domain is the PCI core's
responsibility, and that's not really the case -- for ACPI it's
totally up to pci_root.c, for other drivers it comes from the DT, and
for others it might depend on the driver's knowledge of the hardware
(I'm thinking of parisc, where, I think we currently put all the
bridges in the same domain, but IIRC they *could* each be in their own
domain with a full [bus 00-ff] range for each bridge because each
bridge has its own config space access mechanism).

But it's not that big a deal either way -- we could do this bit of
restructuring later, too.

Bjorn

Re: [PATCH V8 5/9] pci, acpi: add acpi hook to assign domain number.

[Jon Masters]

Hi Bjorn,

Sorry for top posting while on the road. If this refactoring can happen later, is it possible to merge now (well, -next anyway) and explore other work as next steps?

Jon.

-- 
Computer Architect | Sent from my 64-bit #ARM Powered phone

> On Jun 10, 2016, at 13:18, Bjorn Helgaas <helgaas@kernel.org> wrote:
> 
>> On Fri, Jun 10, 2016 at 06:49:32PM +0200, Tomasz Nowicki wrote:
>>> On 10.06.2016 17:49, Lorenzo Pieralisi wrote:
>>>> On Fri, Jun 10, 2016 at 04:14:58PM +0100, Lorenzo Pieralisi wrote:
>>>> Hi Bjorn, Tomasz,
>>>> 
>>>> On Tue, Jun 07, 2016 at 07:15:59PM -0500, Bjorn Helgaas wrote:
>>>> 
>>>> [...]
>>>> 
>>>>>> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
>>>>>> index eb431b5..2b52178 100644
>>>>>> --- a/drivers/pci/pci.c
>>>>>> +++ b/drivers/pci/pci.c
>>>>>> @@ -7,6 +7,7 @@
>>>>>>  *    Copyright 1997 -- 2000 Martin Mares <mj@ucw.cz>
>>>>>>  */
>>>>>> 
>>>>>> +#include <linux/acpi.h>
>>>>>> #include <linux/kernel.h>
>>>>>> #include <linux/delay.h>
>>>>>> #include <linux/init.h>
>>>>>> @@ -4941,7 +4942,7 @@ int pci_get_new_domain_nr(void)
>>>>>> }
>>>>>> 
>>>>>> #ifdef CONFIG_PCI_DOMAINS_GENERIC
>>>>>> -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
>>>>>> +static int of_pci_bus_domain_nr(struct device *parent)
>>>>> 
>>>>> Can we do a little cleanup before this patch?
>>>>> 
>>>>>  - pci_bus_assign_domain_nr() is only used inside drivers/pci, so
>>>>>    maybe we move the prototype to drivers/pci/pci.h?
>>>>> 
>>>>>  - I don't really like the style of calling a function that
>>>>>    internally assigns bus->domain_nr.  Could we do something like
>>>>>    this instead?
>>>>> 
>>>>>    int pci_bus_domain_nr(...)
>>>>>    {
>>>>>      ...
>>>>>      return domain;
>>>>>    }
>>>>> 
>>>>>    ... pci_create_root_bus(...)
>>>>>    {
>>>>>      ...
>>>>>      b->domain_nr = pci_bus_domain_nr(...);
>>>> 
>>>> We noticed while preparing v9, that this would force us to
>>>> write an empty pci_bus_domain_nr() prototype for
>>>> !PCI_DOMAINS_GENERIC (ie every arch but ARM/ARM64) that should
>>>> return 0 to keep current behaviour unchanged.
>>>> 
>>>> That's why pci_bus_assign_domain_nr() was there, so that it
>>>> was compiled out on !PCI_DOMAINS_GENERIC.
>>>> 
>>>> I really would like v9 to be final so let's fix it before posting it
>>>> shortly please.
>>>> 
>>>> For the above we have three options:
>>>> 
>>>> 1) Leave code as-is in v8
>>>> 
>>>> 2) in pci_create_root_bus():
>>>> #ifdef CONFIG_PCI_DOMAINS_GENERIC
>>>>       b->domain_nr = pci_bus_domain_nr(...);
>>>> #endif
>>>> 
>>>> + other changes requested above
>>>> 
>>>> 3) in pci_create_root_bus()
>>>> 
>>>> b->domain_nr = pci_bus_domain_nr(...);
>>>> 
>>>> unguarded and a stub:
>>>> 
>>>> #ifndef CONFIG_PCI_DOMAINS_GENERIC
>>>> static inline int pci_bus_domain_nr() { return 0; }
>>>> #endif
>>>> 
>>>> + other changes requested above
>>> 
>>> Actually, Tomasz made me notice that pci_bus.domain_nr is
>>> only declared for CONFIG_PCI_DOMAINS_GENERIC so (3) is not
>>> even an option and IMO (2) is not much nicer than code in
>>> v8 as-is with an ifdef in the middle of pci_create_root_bus().
>> 
>> To me (1) is nicer too. Bjorn what is your take on this? This is
>> last bit before sending v9.
> 
> My preference is (2).  The ifdef in pci_create_root_bus() is a little
> ugly, but I like it better because it will fit nicely into Arnd's
> idea of having the native drivers allocate and fill in a host bridge
> structure before calling the PCI core.  The domain is one thing those
> drivers could fill in.  I like that model much better than having the
> PCI core make callbacks to get information that we should have passed
> in to begin with.
> 
> The current code suggests that assigning the domain is the PCI core's
> responsibility, and that's not really the case -- for ACPI it's
> totally up to pci_root.c, for other drivers it comes from the DT, and
> for others it might depend on the driver's knowledge of the hardware
> (I'm thinking of parisc, where, I think we currently put all the
> bridges in the same domain, but IIRC they *could* each be in their own
> domain with a full [bus 00-ff] range for each bridge because each
> bridge has its own config space access mechanism).
> 
> But it's not that big a deal either way -- we could do this bit of
> restructuring later, too.
> 
> Bjorn