[PATCH v10 00/12] PCI: ARM64 ECAM quirks

[PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Bjorn Helgaas]

This is what I've collected on my pci/ecam branch so far.  This is
preliminary and probably doesn't even build since I made several
changes, but at least it's a strawman.  I called it v10 to keep it
separate from the previous postings of the various pieces.

  ACPI: Add acpi_resource_consumer() to find device that claims a resource
  PCI: Search ACPI namespace to ensure ECAM space is reserved
  x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG

    acpi_resource_consumer() takes a resource and returns an
    acpi_device that has _CRS that contains the resource.  The idea is
    that we call this on a resource from MCFG.  We *should* find a
    PNP0C02 or other device that reserves that resource.  On x86 we
    already do this in a more hand-coded way.  I probably wouldn't
    merge the x86 patch for v4.10 but I included it here as an
    example.

    I don't think it's worth trying to fabricate ACPI devices or _CRS
    resources to compensate for firmware that doesn't describe
    everything.  This check will emit a warning if the MCFG region is
    not reserved, and that's probably enough to motivate firmware
    fixes for the next round of hardware.  The ECAM code itself does
    reserve the region, so that part will be safe from other users.
    Unreported non-ECAM register space is still a landmine and there's
    no real way to look for it.

  arm64: PCI: Manage controller-specific data on per-controller basis

    Tomasz's fix for pci_acpi_scan_root(), unchanged.

  PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors
  PCI/ACPI: Check for platform-specific MCFG quirks

    Tomasz's quirk infrastructure.  I put this under
    CONFIG_PCI_QUIRKS.

  PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform

    Dongdong's interface to look up a _HID with a _UID matching the
    segment.  I don't have an opinion on using _UID yet.  It's
    encapsulated so it could be changed easily.  I put this under
    CONFIG_PCI_QUIRKS && CONFIG_ARM64.

  PCI: Add MCFG quirks for Qualcomm QDF2432 host controller

    Christopher's Qualcomm quirks.  Basically unchanged except to put
    the quirk ops under CONFIG_ACPI and CONFIG_PCI_QUIRKS.

  PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers

    Dongdong's HiSilicon quirks.  Here's where it gets interesting.  I
    moved this to the existing pcie-hisi.c instead of adding
    pcie-hisi-acpi.c.  I changed the Makefile so we always build
    pcie-hisi.c on ARM64.  I added ifdefs so we get the quirk code if
    CONFIG_ACPI and CONFIG_PCI_QUIRKS and we get the original platform
    driver if CONFIG_PCI_HISI.  It's possible to have both, and if we
    process the MCFG quirk we get the quirk code.

    I'm confused about why the quirk accessors are so much different
    than the original accessors.  hisi_pcie_acpi_rd_conf() looks much
    different than hisi_pcie_cfg_read().  The original driver claims
    Hipxx only supports 32-bit config accesses, but the quirk
    accessors don't enforce that.

  PCI: thunder-pem: Factor out resource lookup
  PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller
  PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller

    Tomasz's ThunderX quirks.  I restructured the PEM init I could do
    the same ifdef tricks as in hisi.  The quirks make a lot more
    sense here -- they use the same accessors as the original platform
    driver.  Only the initialization is different.

The Makefile looks a little strange:

  obj-$(CONFIG_ARM64) += pcie-hisi.o
  obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
  obj-$(CONFIG_ARM64) += pci-thunder-pem.o

and the ifdefs inside those files are a little unwieldy.  But I think
they do accomplish the goals of:

  - Adding no new files,

  - Including the MCFG quirks whenever CONFIG_ACPI and
    CONFIG_PCI_QUIRKS are enabled, regardless of the driver-specific
    config,

  - Including the original platform drivers when they are specifically
    enabled, and

  - Sharing the same code for the platform driver and the quirks
    (except for HiSi, which I don't understand)

I started looking at the Duc's X-Gene quirks, but I haven't wrapped my
head around those yet.  I'm going to push this branch to get build
testing and so you can comment on it.

If by some miracle it builds and you test it, please collect the dmesg
log and /proc/iomem contents.

---

Bjorn Helgaas (5):
      ACPI: Add acpi_resource_consumer() to find device that claims a resource
      PCI: Search ACPI namespace to ensure ECAM space is reserved
      x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG
      PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers
      PCI: thunder-pem: Factor out resource lookup

Christopher Covington (1):
      PCI: Add MCFG quirks for Qualcomm QDF2432 host controller

Dongdong Liu (1):
      PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform

Tomasz Nowicki (5):
      arm64: PCI: Manage controller-specific data on per-controller basis
      PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors
      PCI/ACPI: Check for platform-specific MCFG quirks
      PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller
      PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller


 arch/arm64/kernel/pci.c             |   34 +++----
 arch/x86/pci/mmconfig-shared.c      |   69 ++-------------
 drivers/acpi/pci_mcfg.c             |  165 ++++++++++++++++++++++++++++++++++-
 drivers/acpi/resource.c             |   57 ++++++++++++
 drivers/pci/ecam.c                  |   31 +++++++
 drivers/pci/host/Makefile           |    6 +
 drivers/pci/host/pci-thunder-ecam.c |    9 ++
 drivers/pci/host/pci-thunder-pem.c  |   88 ++++++++++++++-----
 drivers/pci/host/pcie-hisi.c        |   95 ++++++++++++++++++++
 drivers/pci/pci-acpi.c              |   71 +++++++++++++++
 drivers/pci/pci.h                   |    4 +
 include/linux/acpi.h                |    7 +
 include/linux/pci-acpi.h            |    4 +
 include/linux/pci-ecam.h            |    7 +
 14 files changed, 537 insertions(+), 110 deletions(-)

[PATCH v10 01/12] ACPI: Add acpi_resource_consumer() to find device that claims a resource

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

Add acpi_resource_consumer().  This takes a struct resource and searches
the ACPI namespace for a device whose current resource settings (_CRS)
includes the resource.  It returns the device if it exists, or NULL if no
device uses the resource.

If more than one device uses the resource (this may happen in the case of
bridges), acpi_resource_consumer() returns the first one found by
acpi_get_devices() in its modified depth-first walk of the namespace.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/resource.c |   57 +++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/acpi.h    |    7 ++++++
 2 files changed, 64 insertions(+)

diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c
index 56241eb..cb57962 100644
--- a/drivers/acpi/resource.c
+++ b/drivers/acpi/resource.c
@@ -664,3 +664,60 @@ int acpi_dev_filter_resource_type(struct acpi_resource *ares,
 	return (type & types) ? 0 : 1;
 }
 EXPORT_SYMBOL_GPL(acpi_dev_filter_resource_type);
+
+static int acpi_dev_consumes_res(struct acpi_device *adev, struct resource *res)
+{
+	struct list_head resource_list;
+	struct resource_entry *rentry;
+	int ret, found = 0;
+
+	INIT_LIST_HEAD(&resource_list);
+	ret = acpi_dev_get_resources(adev, &resource_list, NULL, NULL);
+	if (ret < 0)
+		return 0;
+
+	list_for_each_entry(rentry, &resource_list, node) {
+		if (resource_contains(rentry->res, res)) {
+			found = 1;
+			break;
+		}
+
+	}
+
+	acpi_dev_free_resource_list(&resource_list);
+	return found;
+}
+
+static acpi_status acpi_res_consumer_cb(acpi_handle handle, u32 depth,
+					 void *context, void **ret)
+{
+	struct resource *res = context;
+	struct acpi_device **consumer = (struct acpi_device **) ret;
+	struct acpi_device *adev;
+
+	if (acpi_bus_get_device(handle, &adev))
+		return AE_OK;
+
+	if (acpi_dev_consumes_res(adev, res)) {
+		*consumer = adev;
+		return AE_CTRL_TERMINATE;
+	}
+
+	return AE_OK;
+}
+
+/**
+ * acpi_resource_consumer - Find the ACPI device that consumes @res.
+ * @res: Resource to search for.
+ *
+ * Search the current resource settings (_CRS) of every ACPI device node
+ * for @res.  If we find an ACPI device whose _CRS includes @res, return
+ * it.  Otherwise, return NULL.
+ */
+struct acpi_device *acpi_resource_consumer(struct resource *res)
+{
+	struct acpi_device *consumer = NULL;
+
+	acpi_get_devices(NULL, acpi_res_consumer_cb, res, (void **) &consumer);
+	return consumer;
+}
diff --git a/include/linux/acpi.h b/include/linux/acpi.h
index ddbeda6..b00ad73 100644
--- a/include/linux/acpi.h
+++ b/include/linux/acpi.h
@@ -419,6 +419,8 @@ static inline int acpi_dev_filter_resource_type_cb(struct acpi_resource *ares,
 	return acpi_dev_filter_resource_type(ares, (unsigned long)arg);
 }
 
+struct acpi_device *acpi_resource_consumer(struct resource *res);
+
 int acpi_check_resource_conflict(const struct resource *res);
 
 int acpi_check_region(resource_size_t start, resource_size_t n,
@@ -762,6 +764,11 @@ static inline int acpi_reconfig_notifier_unregister(struct notifier_block *nb)
 	return -EINVAL;
 }
 
+static inline struct acpi_device *acpi_resource_consumer(struct resource *res)
+{
+	return NULL;
+}
+
 #endif	/* !CONFIG_ACPI */
 
 #ifdef CONFIG_ACPI_HOTPLUG_IOAPIC

Re: [PATCH v10 01/12] ACPI: Add acpi_resource_consumer() to find device that claims a resource

[Rafael J. Wysocki]

On Thu, Dec 1, 2016 at 9:29 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> From: Bjorn Helgaas <bhelgaas@google.com>
>
> Add acpi_resource_consumer().  This takes a struct resource and searches
> the ACPI namespace for a device whose current resource settings (_CRS)
> includes the resource.  It returns the device if it exists, or NULL if no
> device uses the resource.
>
> If more than one device uses the resource (this may happen in the case of
> bridges), acpi_resource_consumer() returns the first one found by
> acpi_get_devices() in its modified depth-first walk of the namespace.
>
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

(this patch should have been CCed to linux-acpi too, however).

> ---
>  drivers/acpi/resource.c |   57 +++++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/acpi.h    |    7 ++++++
>  2 files changed, 64 insertions(+)
>
> diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c
> index 56241eb..cb57962 100644
> --- a/drivers/acpi/resource.c
> +++ b/drivers/acpi/resource.c
> @@ -664,3 +664,60 @@ int acpi_dev_filter_resource_type(struct acpi_resource *ares,
>         return (type & types) ? 0 : 1;
>  }
>  EXPORT_SYMBOL_GPL(acpi_dev_filter_resource_type);
> +
> +static int acpi_dev_consumes_res(struct acpi_device *adev, struct resource *res)
> +{
> +       struct list_head resource_list;
> +       struct resource_entry *rentry;
> +       int ret, found = 0;
> +
> +       INIT_LIST_HEAD(&resource_list);
> +       ret = acpi_dev_get_resources(adev, &resource_list, NULL, NULL);
> +       if (ret < 0)
> +               return 0;
> +
> +       list_for_each_entry(rentry, &resource_list, node) {
> +               if (resource_contains(rentry->res, res)) {
> +                       found = 1;
> +                       break;
> +               }
> +
> +       }
> +
> +       acpi_dev_free_resource_list(&resource_list);
> +       return found;
> +}
> +
> +static acpi_status acpi_res_consumer_cb(acpi_handle handle, u32 depth,
> +                                        void *context, void **ret)
> +{
> +       struct resource *res = context;
> +       struct acpi_device **consumer = (struct acpi_device **) ret;
> +       struct acpi_device *adev;
> +
> +       if (acpi_bus_get_device(handle, &adev))
> +               return AE_OK;
> +
> +       if (acpi_dev_consumes_res(adev, res)) {
> +               *consumer = adev;
> +               return AE_CTRL_TERMINATE;
> +       }
> +
> +       return AE_OK;
> +}
> +
> +/**
> + * acpi_resource_consumer - Find the ACPI device that consumes @res.
> + * @res: Resource to search for.
> + *
> + * Search the current resource settings (_CRS) of every ACPI device node
> + * for @res.  If we find an ACPI device whose _CRS includes @res, return
> + * it.  Otherwise, return NULL.
> + */
> +struct acpi_device *acpi_resource_consumer(struct resource *res)
> +{
> +       struct acpi_device *consumer = NULL;
> +
> +       acpi_get_devices(NULL, acpi_res_consumer_cb, res, (void **) &consumer);
> +       return consumer;
> +}
> diff --git a/include/linux/acpi.h b/include/linux/acpi.h
> index ddbeda6..b00ad73 100644
> --- a/include/linux/acpi.h
> +++ b/include/linux/acpi.h
> @@ -419,6 +419,8 @@ static inline int acpi_dev_filter_resource_type_cb(struct acpi_resource *ares,
>         return acpi_dev_filter_resource_type(ares, (unsigned long)arg);
>  }
>
> +struct acpi_device *acpi_resource_consumer(struct resource *res);
> +
>  int acpi_check_resource_conflict(const struct resource *res);
>
>  int acpi_check_region(resource_size_t start, resource_size_t n,
> @@ -762,6 +764,11 @@ static inline int acpi_reconfig_notifier_unregister(struct notifier_block *nb)
>         return -EINVAL;
>  }
>
> +static inline struct acpi_device *acpi_resource_consumer(struct resource *res)
> +{
> +       return NULL;
> +}
> +
>  #endif /* !CONFIG_ACPI */
>
>  #ifdef CONFIG_ACPI_HOTPLUG_IOAPIC
>

Re: [PATCH v10 01/12] ACPI: Add acpi_resource_consumer() to find device that claims a resource

[Bjorn Helgaas]

On Thu, Dec 01, 2016 at 02:17:20PM +0100, Rafael J. Wysocki wrote:
> On Thu, Dec 1, 2016 at 9:29 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
> > From: Bjorn Helgaas <bhelgaas@google.com>
> >
> > Add acpi_resource_consumer().  This takes a struct resource and searches
> > the ACPI namespace for a device whose current resource settings (_CRS)
> > includes the resource.  It returns the device if it exists, or NULL if no
> > device uses the resource.
> >
> > If more than one device uses the resource (this may happen in the case of
> > bridges), acpi_resource_consumer() returns the first one found by
> > acpi_get_devices() in its modified depth-first walk of the namespace.
> >
> > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> 
> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> 
> (this patch should have been CCed to linux-acpi too, however).

Thanks for taking a look!  I didn't CC linux-acpi because this was mostly
to get build testing and I didn't want linux-acpi to get spammed by
buildbot errors.  I'll add linux-acpi on future postings.

Bjorn

[PATCH v10 02/12] PCI: Search ACPI namespace to ensure ECAM space is reserved

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

The static MCFG table tells us the base of ECAM space, but it does not
reserve the space -- the reservation should be done via a device in the
ACPI namespace whose _CRS includes the ECAM region.

Add pci_ecam_verify_reservation() to check whether the ECAM space is
reserved by an ACPI namespace device.  If it is, emit a message showing
which device reserves it.  If not, emit a "[Firmware Bug]" warning.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/ecam.c |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 43ed08d..3805122 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -14,6 +14,7 @@
  * version 2 (GPLv2) along with this source code.
  */
 
+#include <linux/acpi.h>
 #include <linux/device.h>
 #include <linux/io.h>
 #include <linux/kernel.h>
@@ -29,6 +30,24 @@
  */
 static const bool per_bus_mapping = !IS_ENABLED(CONFIG_64BIT);
 
+static void pci_ecam_verify_reservation(struct device *dev,
+					struct resource *ecam)
+{
+#ifdef CONFIG_ACPI
+	struct acpi_device *adev;
+
+	adev = acpi_resource_consumer(ecam);
+	if (!adev) {
+		dev_warn(dev, FW_BUG "ECAM area %pR not reserved in ACPI namespace\n",
+			 ecam);
+		return;
+	}
+
+	dev_info(dev, "ECAM area %pR reserved by %s\n", ecam,
+		 dev_name(&adev->dev));
+#endif
+}
+
 /*
  * Create a PCI config space window
  *  - reserve mem region
@@ -51,6 +70,8 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
 	if (!cfg)
 		return ERR_PTR(-ENOMEM);
 
+	pci_ecam_verify_reservation(dev, cfgres);
+
 	cfg->parent = dev;
 	cfg->ops = ops;
 	cfg->busr.start = busr->start;

Re: [PATCH v10 02/12] PCI: Search ACPI namespace to ensure ECAM space is reserved

[Lorenzo Pieralisi]

Hi Bjorn,

Thank you very much for putting this series together !

On Thu, Dec 01, 2016 at 02:29:39AM -0600, Bjorn Helgaas wrote:
> From: Bjorn Helgaas <bhelgaas@google.com>
> 
> The static MCFG table tells us the base of ECAM space, but it does not
> reserve the space -- the reservation should be done via a device in the
> ACPI namespace whose _CRS includes the ECAM region.
> 
> Add pci_ecam_verify_reservation() to check whether the ECAM space is
> reserved by an ACPI namespace device.  If it is, emit a message showing
> which device reserves it.  If not, emit a "[Firmware Bug]" warning.

I have a question: do we want to carry out a search on a restricted set
of _HIDs here ? I am not sure we should consider an ECAM region matching
with a resource in a device _CRS whose _HID/_CID is not part of
{PNP0c02, PNP0A03, PNP0A08} as a success, actually I think that would be
a FW bug, right ?

Thanks,
Lorenzo

> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
>  drivers/pci/ecam.c |   21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
> 
> diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> index 43ed08d..3805122 100644
> --- a/drivers/pci/ecam.c
> +++ b/drivers/pci/ecam.c
> @@ -14,6 +14,7 @@
>   * version 2 (GPLv2) along with this source code.
>   */
>  
> +#include <linux/acpi.h>
>  #include <linux/device.h>
>  #include <linux/io.h>
>  #include <linux/kernel.h>
> @@ -29,6 +30,24 @@
>   */
>  static const bool per_bus_mapping = !IS_ENABLED(CONFIG_64BIT);
>  
> +static void pci_ecam_verify_reservation(struct device *dev,
> +					struct resource *ecam)
> +{
> +#ifdef CONFIG_ACPI
> +	struct acpi_device *adev;
> +
> +	adev = acpi_resource_consumer(ecam);
> +	if (!adev) {
> +		dev_warn(dev, FW_BUG "ECAM area %pR not reserved in ACPI namespace\n",
> +			 ecam);
> +		return;
> +	}
> +
> +	dev_info(dev, "ECAM area %pR reserved by %s\n", ecam,
> +		 dev_name(&adev->dev));
> +#endif
> +}
> +
>  /*
>   * Create a PCI config space window
>   *  - reserve mem region
> @@ -51,6 +70,8 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
>  	if (!cfg)
>  		return ERR_PTR(-ENOMEM);
>  
> +	pci_ecam_verify_reservation(dev, cfgres);
> +
>  	cfg->parent = dev;
>  	cfg->ops = ops;
>  	cfg->busr.start = busr->start;
> 

Re: [PATCH v10 02/12] PCI: Search ACPI namespace to ensure ECAM space is reserved

[Bjorn Helgaas]

On Thu, Dec 01, 2016 at 02:17:29PM +0000, Lorenzo Pieralisi wrote:
> Hi Bjorn,
> 
> Thank you very much for putting this series together !
> 
> On Thu, Dec 01, 2016 at 02:29:39AM -0600, Bjorn Helgaas wrote:
> > From: Bjorn Helgaas <bhelgaas@google.com>
> > 
> > The static MCFG table tells us the base of ECAM space, but it does not
> > reserve the space -- the reservation should be done via a device in the
> > ACPI namespace whose _CRS includes the ECAM region.
> > 
> > Add pci_ecam_verify_reservation() to check whether the ECAM space is
> > reserved by an ACPI namespace device.  If it is, emit a message showing
> > which device reserves it.  If not, emit a "[Firmware Bug]" warning.
> 
> I have a question: do we want to carry out a search on a restricted set
> of _HIDs here ? I am not sure we should consider an ECAM region matching
> with a resource in a device _CRS whose _HID/_CID is not part of
> {PNP0c02, PNP0A03, PNP0A08} as a success, actually I think that would be
> a FW bug, right ?

That's a good question.  I did it the simplest way I could for now,
but we could extend that if we decide it's necessary.  Sometimes I've
been overly zealous about looking for problems and ended up
overengineering things or shooting myself in the foot.

My main goal is to make sure the space is described *somewhere*, which
is enough to keep it from being a land mine.

> > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> > ---
> >  drivers/pci/ecam.c |   21 +++++++++++++++++++++
> >  1 file changed, 21 insertions(+)
> > 
> > diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
> > index 43ed08d..3805122 100644
> > --- a/drivers/pci/ecam.c
> > +++ b/drivers/pci/ecam.c
> > @@ -14,6 +14,7 @@
> >   * version 2 (GPLv2) along with this source code.
> >   */
> >  
> > +#include <linux/acpi.h>
> >  #include <linux/device.h>
> >  #include <linux/io.h>
> >  #include <linux/kernel.h>
> > @@ -29,6 +30,24 @@
> >   */
> >  static const bool per_bus_mapping = !IS_ENABLED(CONFIG_64BIT);
> >  
> > +static void pci_ecam_verify_reservation(struct device *dev,
> > +					struct resource *ecam)
> > +{
> > +#ifdef CONFIG_ACPI
> > +	struct acpi_device *adev;
> > +
> > +	adev = acpi_resource_consumer(ecam);
> > +	if (!adev) {
> > +		dev_warn(dev, FW_BUG "ECAM area %pR not reserved in ACPI namespace\n",
> > +			 ecam);
> > +		return;
> > +	}
> > +
> > +	dev_info(dev, "ECAM area %pR reserved by %s\n", ecam,
> > +		 dev_name(&adev->dev));
> > +#endif
> > +}
> > +
> >  /*
> >   * Create a PCI config space window
> >   *  - reserve mem region
> > @@ -51,6 +70,8 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
> >  	if (!cfg)
> >  		return ERR_PTR(-ENOMEM);
> >  
> > +	pci_ecam_verify_reservation(dev, cfgres);
> > +
> >  	cfg->parent = dev;
> >  	cfg->ops = ops;
> >  	cfg->busr.start = busr->start;
> > 

[PATCH v10 03/12] x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

The static MCFG table tells us the base of MMCFG space, but it does not
reserve the space -- the reservation should be done via a device in the
ACPI namespace whose _CRS includes the MMCFG region.

Replace find_mboard_resource(), which already searches for such a device,
with the new acpi_resource_consumer() interface that does essentially the
same thing.

acpi_resource_consumer() is not as strict as find_mboard_resource() was:
find_mboard_resource() only looks at PNP0C01 and PNP0C02 devices and the
following _CRS descriptor types:

  ACPI_RESOURCE_TYPE_FIXED_MEMORY32
  ACPI_RESOURCE_TYPE_ADDRESS32
  ACPI_RESOURCE_TYPE_ADDRESS64

but acpi_resource_consumer() looks at *all* devices in the namespace and
the following descriptor types:

  ACPI_RESOURCE_TYPE_MEMORY24
  ACPI_RESOURCE_TYPE_MEMORY32
  ACPI_RESOURCE_TYPE_FIXED_MEMORY32
  ACPI_RESOURCE_TYPE_ADDRESS16
  ACPI_RESOURCE_TYPE_ADDRESS32
  ACPI_RESOURCE_TYPE_ADDRESS64
  ACPI_RESOURCE_TYPE_EXTENDED_ADDRESS64

I think it is correct to accept the larger set of descriptor types.

Note 2 to Table 4-2 of the PCI Firmware spec r3.2 does suggest that the
MMCFG region should be declared by a PNP0C02 device.  I don't know why it
calls out PNP0C02 specifically, unless it's related to the fact that the
Consumer/Producer issues make it hard to use the PNP0A03 device itself.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/x86/pci/mmconfig-shared.c |   69 +++++-----------------------------------
 1 file changed, 9 insertions(+), 60 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index dd30b7e..f62ffe7 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -373,71 +373,20 @@ static int __init pci_mmcfg_check_hostbridge(void)
 	return !list_empty(&pci_mmcfg_list);
 }
 
-static acpi_status check_mcfg_resource(struct acpi_resource *res, void *data)
-{
-	struct resource *mcfg_res = data;
-	struct acpi_resource_address64 address;
-	acpi_status status;
-
-	if (res->type == ACPI_RESOURCE_TYPE_FIXED_MEMORY32) {
-		struct acpi_resource_fixed_memory32 *fixmem32 =
-			&res->data.fixed_memory32;
-		if (!fixmem32)
-			return AE_OK;
-		if ((mcfg_res->start >= fixmem32->address) &&
-		    (mcfg_res->end < (fixmem32->address +
-				      fixmem32->address_length))) {
-			mcfg_res->flags = 1;
-			return AE_CTRL_TERMINATE;
-		}
-	}
-	if ((res->type != ACPI_RESOURCE_TYPE_ADDRESS32) &&
-	    (res->type != ACPI_RESOURCE_TYPE_ADDRESS64))
-		return AE_OK;
-
-	status = acpi_resource_to_address64(res, &address);
-	if (ACPI_FAILURE(status) ||
-	   (address.address.address_length <= 0) ||
-	   (address.resource_type != ACPI_MEMORY_RANGE))
-		return AE_OK;
-
-	if ((mcfg_res->start >= address.address.minimum) &&
-	    (mcfg_res->end < (address.address.minimum + address.address.address_length))) {
-		mcfg_res->flags = 1;
-		return AE_CTRL_TERMINATE;
-	}
-	return AE_OK;
-}
-
-static acpi_status find_mboard_resource(acpi_handle handle, u32 lvl,
-					void *context, void **rv)
-{
-	struct resource *mcfg_res = context;
-
-	acpi_walk_resources(handle, METHOD_NAME__CRS,
-			    check_mcfg_resource, context);
-
-	if (mcfg_res->flags)
-		return AE_CTRL_TERMINATE;
-
-	return AE_OK;
-}
-
 static int is_acpi_reserved(u64 start, u64 end, unsigned not_used)
 {
 	struct resource mcfg_res;
+	struct acpi_device *adev;
 
 	mcfg_res.start = start;
 	mcfg_res.end = end - 1;
-	mcfg_res.flags = 0;
+	mcfg_res.flags = IORESOURCE_MEM;
 
-	acpi_get_devices("PNP0C01", find_mboard_resource, &mcfg_res, NULL);
-
-	if (!mcfg_res.flags)
-		acpi_get_devices("PNP0C02", find_mboard_resource, &mcfg_res,
-				 NULL);
+	adev = acpi_resource_consumer(&mcfg_res);
+	if (adev)
+		return 1;
 
-	return mcfg_res.flags;
+	return 0;
 }
 
 typedef int (*check_reserved_t)(u64 start, u64 end, unsigned type);
@@ -450,7 +399,7 @@ static int __ref is_mmconf_reserved(check_reserved_t is_reserved,
 	u64 size = resource_size(&cfg->res);
 	u64 old_size = size;
 	int num_buses;
-	char *method = with_e820 ? "E820" : "ACPI motherboard resources";
+	char *method = with_e820 ? "E820" : "ACPI namespace";
 
 	while (!is_reserved(addr, addr + size, E820_RESERVED)) {
 		size >>= 1;
@@ -504,12 +453,12 @@ static int __ref pci_mmcfg_check_reserved(struct device *dev,
 		if (dev)
 			dev_info(dev, FW_INFO
 				 "MMCONFIG at %pR not reserved in "
-				 "ACPI motherboard resources\n",
+				 "ACPI namespace\n",
 				 &cfg->res);
 		else
 			pr_info(FW_INFO PREFIX
 			       "MMCONFIG at %pR not reserved in "
-			       "ACPI motherboard resources\n",
+			       "ACPI namespace\n",
 			       &cfg->res);
 	}
 

[PATCH v10 04/12] arm64: PCI: Manage controller-specific data on per-controller basis

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

Currently we use one shared global acpi_pci_root_ops structure to keep
controller-specific ops. We pass its pointer to acpi_pci_root_create() and
associate it with a host bridge instance for good.  Such a design implies
serious drawback. Any potential manipulation on the single system-wide
acpi_pci_root_ops leads to kernel crash. The structure content is not
really changing even across multiple host bridges creation; thus it was not
an issue so far.

In preparation for adding ECAM quirks mechanism (where controller-specific
PCI ops may be different for each host bridge) allocate new
acpi_pci_root_ops and fill in with data for each bridge. Now it is safe to
have different controller-specific info. As a consequence free
acpi_pci_root_ops when host bridge is released.

No functional changes in this patch.

Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/arm64/kernel/pci.c |   17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index acf3872..95e0bb4 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -159,33 +159,36 @@ static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
 
 	ri = container_of(ci, struct acpi_pci_generic_root_info, common);
 	pci_ecam_free(ri->cfg);
+	kfree(ci->ops);
 	kfree(ri);
 }
 
-static struct acpi_pci_root_ops acpi_pci_root_ops = {
-	.release_info = pci_acpi_generic_release_info,
-};
-
 /* Interface called from ACPI code to setup PCI host controller */
 struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
 {
 	int node = acpi_get_node(root->device->handle);
 	struct acpi_pci_generic_root_info *ri;
 	struct pci_bus *bus, *child;
+	struct acpi_pci_root_ops *root_ops;
 
 	ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
 	if (!ri)
 		return NULL;
 
+	root_ops = kzalloc_node(sizeof(*root_ops), GFP_KERNEL, node);
+	if (!root_ops)
+		return NULL;
+
 	ri->cfg = pci_acpi_setup_ecam_mapping(root);
 	if (!ri->cfg) {
 		kfree(ri);
+		kfree(root_ops);
 		return NULL;
 	}
 
-	acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
-	bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
-				   ri->cfg);
+	root_ops->release_info = pci_acpi_generic_release_info;
+	root_ops->pci_ops = &ri->cfg->ops->pci_ops;
+	bus = acpi_pci_root_create(root, root_ops, &ri->common, ri->cfg);
 	if (!bus)
 		return NULL;
 

[PATCH v10 05/12] PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

pci_mcfg_lookup() is the external interface to the generic MCFG code.
Previously it merely looked up the ECAM base address for a given domain and
bus range.  We want a way to add MCFG quirks, some of which may require
special config accessors and adjustments to the ECAM address range.

Extend pci_mcfg_lookup() so it can return a pointer to a pci_ecam_ops
structure and a struct resource for the ECAM address space.  For now, it
always returns &pci_generic_ecam_ops (the standard accessor) and the
resource described by the MCFG.

No functional changes intended.

[bhelgaas: changelog]
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/arm64/kernel/pci.c  |   17 +++++------------
 drivers/acpi/pci_mcfg.c  |   28 +++++++++++++++++++++++++---
 include/linux/pci-acpi.h |    4 +++-
 3 files changed, 33 insertions(+), 16 deletions(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 95e0bb4..5c08baf 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -125,24 +125,17 @@ pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root)
 	u16 seg = root->segment;
 	struct pci_config_window *cfg;
 	struct resource cfgres;
-	unsigned int bsz;
+	struct pci_ecam_ops *ecam_ops;
+	int ret;
 
-	/* Use address from _CBA if present, otherwise lookup MCFG */
-	if (!root->mcfg_addr)
-		root->mcfg_addr = pci_mcfg_lookup(seg, bus_res);
-
-	if (!root->mcfg_addr) {
+	ret = pci_mcfg_lookup(root, &cfgres, &ecam_ops);
+	if (ret) {
 		dev_err(&root->device->dev, "%04x:%pR ECAM region not found\n",
 			seg, bus_res);
 		return NULL;
 	}
 
-	bsz = 1 << pci_generic_ecam_ops.bus_shift;
-	cfgres.start = root->mcfg_addr + bus_res->start * bsz;
-	cfgres.end = cfgres.start + resource_size(bus_res) * bsz - 1;
-	cfgres.flags = IORESOURCE_MEM;
-	cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res,
-			      &pci_generic_ecam_ops);
+	cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res, ecam_ops);
 	if (IS_ERR(cfg)) {
 		dev_err(&root->device->dev, "%04x:%pR error %ld mapping ECAM\n",
 			seg, bus_res, PTR_ERR(cfg));
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index b5b376e..ffcc651 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -22,6 +22,7 @@
 #include <linux/kernel.h>
 #include <linux/pci.h>
 #include <linux/pci-acpi.h>
+#include <linux/pci-ecam.h>
 
 /* Structure to hold entries from the MCFG table */
 struct mcfg_entry {
@@ -35,9 +36,18 @@ struct mcfg_entry {
 /* List to save MCFG entries */
 static LIST_HEAD(pci_mcfg_list);
 
-phys_addr_t pci_mcfg_lookup(u16 seg, struct resource *bus_res)
+int pci_mcfg_lookup(struct acpi_pci_root *root, struct resource *cfgres,
+		    struct pci_ecam_ops **ecam_ops)
 {
+	struct pci_ecam_ops *ops = &pci_generic_ecam_ops;
+	struct resource *bus_res = &root->secondary;
+	u16 seg = root->segment;
 	struct mcfg_entry *e;
+	struct resource res;
+
+	/* Use address from _CBA if present, otherwise lookup MCFG */
+	if (root->mcfg_addr)
+		goto skip_lookup;
 
 	/*
 	 * We expect exact match, unless MCFG entry end bus covers more than
@@ -45,10 +55,22 @@ phys_addr_t pci_mcfg_lookup(u16 seg, struct resource *bus_res)
 	 */
 	list_for_each_entry(e, &pci_mcfg_list, list) {
 		if (e->segment == seg && e->bus_start == bus_res->start &&
-		    e->bus_end >= bus_res->end)
-			return e->addr;
+		    e->bus_end >= bus_res->end) {
+			root->mcfg_addr = e->addr;
+		}
+
 	}
 
+	if (!root->mcfg_addr)
+		return -ENXIO;
+
+skip_lookup:
+	memset(&res, 0, sizeof(res));
+	res.start = root->mcfg_addr + (bus_res->start << 20);
+	res.end = res.start + (resource_size(bus_res) << 20) - 1;
+	res.flags = IORESOURCE_MEM;
+	*cfgres = res;
+	*ecam_ops = ops;
 	return 0;
 }
 
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 7d63a66..7a4e83a 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -24,7 +24,9 @@ static inline acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
 }
 extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle);
 
-extern phys_addr_t pci_mcfg_lookup(u16 domain, struct resource *bus_res);
+struct pci_ecam_ops;
+extern int pci_mcfg_lookup(struct acpi_pci_root *root, struct resource *cfgres,
+			   struct pci_ecam_ops **ecam_ops);
 
 static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
 {

[PATCH v10 06/12] PCI/ACPI: Check for platform-specific MCFG quirks

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

The PCIe spec (r3.0, sec 7.2.2) specifies an "Enhanced Configuration Access
Mechanism" (ECAM) for memory-mapped access to configuration space.  ECAM is
required for PCIe systems unless there's a standard firmware interface for
config access.

In the absence of a firmware interface, we use pci_generic_ecam_ops, and on
ACPI systems, we discover the ECAM space via the MCFG table and/or the _CBA
method.

Unfortunately some systems provide MCFG but don't implement ECAM according
to spec, so we need a mechanism for quirks to make those systems work.

Add an MCFG quirk mechanism to override the config accessor functions
and/or the memory-mapped address space.

A quirk is selected if it matches all of the following:

  - OEM ID
  - OEM Table ID
  - OEM Revision
  - PCI segment (from _SEG)
  - PCI bus number range (from _CRS, wildcard allowed)

If the quirk specifies config accessor functions or a memory-mapped address
range, these override the defaults.

[bhelgaas: changelog, reorder quirk matching, fix oem_revision typo per
Duc, add under #ifdef CONFIG_PCI_QUIRKS]
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
Signed-off-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c |   92 ++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 86 insertions(+), 6 deletions(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index ffcc651..1ef7285 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -33,6 +33,69 @@ struct mcfg_entry {
 	u8			bus_end;
 };
 
+#ifdef CONFIG_PCI_QUIRKS
+struct mcfg_fixup {
+	char oem_id[ACPI_OEM_ID_SIZE + 1];
+	char oem_table_id[ACPI_OEM_TABLE_ID_SIZE + 1];
+	u32 oem_revision;
+	u16 segment;
+	struct resource bus_range;
+	struct pci_ecam_ops *ops;
+	struct resource cfgres;
+};
+
+#define MCFG_BUS_RANGE(start, end)	DEFINE_RES_NAMED((start),	\
+						((end) - (start) + 1),	\
+						NULL, IORESOURCE_BUS)
+#define MCFG_BUS_ANY			MCFG_BUS_RANGE(0x0, 0xff)
+
+static struct mcfg_fixup mcfg_quirks[] = {
+/*	{ OEM_ID, OEM_TABLE_ID, REV, SEGMENT, BUS_RANGE, ops, cfgres }, */
+};
+
+static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
+static char mcfg_oem_table_id[ACPI_OEM_TABLE_ID_SIZE];
+static u32 mcfg_oem_revision;
+
+static int pci_mcfg_quirk_matches(struct mcfg_fixup *f, u16 segment,
+				  struct resource *bus_range)
+{
+	if (!memcmp(f->oem_id, mcfg_oem_id, ACPI_OEM_ID_SIZE) &&
+	    !memcmp(f->oem_table_id, mcfg_oem_table_id,
+	            ACPI_OEM_TABLE_ID_SIZE) &&
+	    f->oem_revision == mcfg_oem_revision &&
+	    f->segment == segment &&
+	    resource_contains(&f->bus_range, bus_range))
+		return 1;
+
+	return 0;
+}
+#endif
+
+static void pci_mcfg_apply_quirks(struct acpi_pci_root *root,
+				  struct resource *cfgres,
+				  struct pci_ecam_ops **ecam_ops)
+{
+#ifdef CONFIG_PCI_QUIRKS
+	u16 segment = root->segment;
+	struct resource *bus_range = &root->secondary;
+	struct mcfg_fixup *f;
+	int i;
+
+	for (i = 0, f = mcfg_quirks; i < ARRAY_SIZE(mcfg_quirks); i++, f++) {
+		if (pci_mcfg_quirk_matches(f, segment, bus_range)) {
+			if (f->cfgres.start)
+				*cfgres = f->cfgres;
+			if (f->ops)
+				*ecam_ops =  f->ops;
+			dev_info(&root->device->dev, "MCFG quirk: ECAM at %pR for %pR with %ps\n",
+				 cfgres, bus_range, *ecam_ops);
+			return;
+		}
+	}
+#endif
+}
+
 /* List to save MCFG entries */
 static LIST_HEAD(pci_mcfg_list);
 
@@ -61,14 +124,24 @@ int pci_mcfg_lookup(struct acpi_pci_root *root, struct resource *cfgres,
 
 	}
 
-	if (!root->mcfg_addr)
-		return -ENXIO;
-
 skip_lookup:
 	memset(&res, 0, sizeof(res));
-	res.start = root->mcfg_addr + (bus_res->start << 20);
-	res.end = res.start + (resource_size(bus_res) << 20) - 1;
-	res.flags = IORESOURCE_MEM;
+	if (root->mcfg_addr) {
+		res.start = root->mcfg_addr + (bus_res->start << 20);
+		res.end = res.start + (resource_size(bus_res) << 20) - 1;
+		res.flags = IORESOURCE_MEM;
+	}
+
+	/*
+	 * Allow quirks to override default ECAM ops and CFG resource
+	 * range.  This may even fabricate a CFG resource range in case
+	 * MCFG does not have it.  Invalid CFG start address means MCFG
+	 * firmware bug or we need another quirk in array.
+	 */
+	pci_mcfg_apply_quirks(root, &res, &ops);
+	if (!res.start)
+		return -ENXIO;
+
 	*cfgres = res;
 	*ecam_ops = ops;
 	return 0;
@@ -101,6 +174,13 @@ static __init int pci_mcfg_parse(struct acpi_table_header *header)
 		list_add(&e->list, &pci_mcfg_list);
 	}
 
+#ifdef CONFIG_PCI_QUIRKS
+	/* Save MCFG IDs and revision for quirks matching */
+	memcpy(mcfg_oem_id, header->oem_id, ACPI_OEM_ID_SIZE);
+	memcpy(mcfg_oem_table_id, header->oem_table_id, ACPI_OEM_TABLE_ID_SIZE);
+	mcfg_oem_revision = header->oem_revision;
+#endif
+
 	pr_info("MCFG table detected, %d entries\n", n);
 	return 0;
 }

[PATCH v10 07/12] PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform

[Bjorn Helgaas]

From: Dongdong Liu <liudongdong3@huawei.com>

The acpi_get_rc_resources() is used to get the RC register address that can
not be described in MCFG.  It takes the _HID & segment to look for and
outputs the RC address resource.  Use PNP0C02 devices to describe such RC
address resource.  Use _UID to match segment to tell which root bus the
PNP0C02 resource belongs to.

[bhelgaas: wrap in #ifdef CONFIG_PCI_QUIRKS]
Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/pci-acpi.c |   71 ++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.h      |    4 +++
 2 files changed, 75 insertions(+)

diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index d966d47..cfab024 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -29,6 +29,77 @@ const u8 pci_acpi_dsm_uuid[] = {
 	0x91, 0x17, 0xea, 0x4d, 0x19, 0xc3, 0x43, 0x4d
 };
 
+#if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_ARM64)
+static int acpi_get_rc_addr(struct acpi_device *adev, struct resource *res)
+{
+	struct resource_entry *entry;
+	struct list_head list;
+	unsigned long flags;
+	int ret;
+
+	INIT_LIST_HEAD(&list);
+	flags = IORESOURCE_MEM;
+	ret = acpi_dev_get_resources(adev, &list,
+				     acpi_dev_filter_resource_type_cb,
+				     (void *) flags);
+	if (ret < 0) {
+		dev_err(&adev->dev,
+			"failed to parse _CRS method, error code %d\n", ret);
+		return ret;
+	} else if (ret == 0) {
+		dev_err(&adev->dev,
+			"no IO and memory resources present in _CRS\n");
+		return -EINVAL;
+	}
+
+	entry = list_first_entry(&list, struct resource_entry, node);
+	*res = *entry->res;
+	acpi_dev_free_resource_list(&list);
+	return 0;
+}
+
+static acpi_status acpi_match_rc(acpi_handle handle, u32 lvl, void *context,
+				 void **retval)
+{
+	u16 *segment = context;
+	unsigned long long uid;
+	acpi_status status;
+
+	status = acpi_evaluate_integer(handle, "_UID", NULL, &uid);
+	if (ACPI_FAILURE(status) || uid != *segment)
+		return AE_CTRL_DEPTH;
+
+	*(acpi_handle *)retval = handle;
+	return AE_CTRL_TERMINATE;
+}
+
+int acpi_get_rc_resources(const char *hid, u16 segment, struct resource *res)
+{
+	struct acpi_device *adev;
+	acpi_status status;
+	acpi_handle handle;
+	int ret;
+
+	status = acpi_get_devices(hid, acpi_match_rc, &segment, &handle);
+	if (ACPI_FAILURE(status)) {
+		pr_err("Can't find _HID %s device\n", hid);
+		return -ENODEV;
+	}
+
+	ret = acpi_bus_get_device(handle, &adev);
+	if (ret)
+		return ret;
+
+	ret = acpi_get_rc_addr(adev, res);
+	if (ret) {
+		dev_err(&adev->dev, "can't get RC resource\n");
+		return ret;
+	}
+
+	return 0;
+}
+#endif
+
 phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle)
 {
 	acpi_status status = AE_NOT_EXIST;
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 4518562..c27b0fc 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -356,4 +356,8 @@ static inline int pci_dev_specific_reset(struct pci_dev *dev, int probe)
 }
 #endif
 
+#if defined(CONFIG_PCI_QUIRKS) && defined(CONFIG_ARM64)
+int acpi_get_rc_resources(const char *hid, u16 segment, struct resource *res);
+#endif
+
 #endif /* DRIVERS_PCI_H */

[PATCH v10 08/12] PCI: Add MCFG quirks for Qualcomm QDF2432 host controller

[Bjorn Helgaas]

From: Christopher Covington <cov@codeaurora.org>

The Qualcomm Technologies QDF2432 SoC does not support accesses smaller
than 32 bits to the PCI configuration space.  Register the appropriate
quirk.

[bhelgaas: add QCOM_ECAM32 macro, ifdef for ACPI and PCI_QUIRKS]
Signed-off-by: Christopher Covington <cov@codeaurora.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c  |   11 +++++++++++
 drivers/pci/ecam.c       |   12 ++++++++++++
 include/linux/pci-ecam.h |    4 ++++
 3 files changed, 27 insertions(+)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index 1ef7285..c74c054 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -51,6 +51,17 @@ struct mcfg_fixup {
 
 static struct mcfg_fixup mcfg_quirks[] = {
 /*	{ OEM_ID, OEM_TABLE_ID, REV, SEGMENT, BUS_RANGE, ops, cfgres }, */
+
+#define QCOM_ECAM32(seg) \
+	{ "QCOM  ", "QDF2432 ", 1, seg, MCFG_BUS_ANY, &pci_32b_ops },
+	QCOM_ECAM32(0),
+	QCOM_ECAM32(1),
+	QCOM_ECAM32(2),
+	QCOM_ECAM32(3),
+	QCOM_ECAM32(4),
+	QCOM_ECAM32(5),
+	QCOM_ECAM32(6),
+	QCOM_ECAM32(7),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 3805122..fc0baa5 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -183,3 +183,15 @@ struct pci_ecam_ops pci_generic_ecam_ops = {
 		.write		= pci_generic_config_write,
 	}
 };
+
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+/* ECAM ops for 32-bit access only (non-compliant) */
+struct pci_ecam_ops pci_32b_ops = {
+	.bus_shift	= 20,
+	.pci_ops	= {
+		.map_bus	= pci_ecam_map_bus,
+		.read		= pci_generic_config_read32,
+		.write		= pci_generic_config_write32,
+	}
+};
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 7adad20..739d233 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -59,6 +59,10 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
 /* default ECAM ops */
 extern struct pci_ecam_ops pci_generic_ecam_ops;
 
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
+#endif
+
 #ifdef CONFIG_PCI_HOST_GENERIC
 /* for DT-based PCI controllers that support ECAM */
 int pci_host_common_probe(struct platform_device *pdev,

[PATCH v10 09/12] PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers

[Bjorn Helgaas]

From: Dongdong Liu <liudongdong3@huawei.com>

The PCIe controller in Hip05/Hip06/Hip07 SoCs is not completely
ECAM-compliant.  It is non-ECAM only for the RC bus config space; for any
other bus underneath the root bus it does support ECAM access.

Add specific quirks for PCI config space accessors.  This involves:
1. New initialization call hisi_pcie_init() to obtain RC base
addresses from PNP0C02 at the root of the ACPI namespace (under \_SB).
2. New entry in common quirk array.

[bhelgaas: move to pcie-hisi.c and change Makefile/ifdefs so quirk doesn't
depend on CONFIG_PCI_HISI]
Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
Signed-off-by: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c      |   12 +++++
 drivers/pci/host/Makefile    |    2 -
 drivers/pci/host/pcie-hisi.c |   95 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/pci-ecam.h     |    1 
 4 files changed, 109 insertions(+), 1 deletion(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index c74c054..c6a6949 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -62,6 +62,18 @@ static struct mcfg_fixup mcfg_quirks[] = {
 	QCOM_ECAM32(5),
 	QCOM_ECAM32(6),
 	QCOM_ECAM32(7),
+
+#define HISI_QUAD_DOM(table_id, seg, ops) \
+	{ "HISI  ", table_id, 0, seg + 0, MCFG_BUS_ANY, ops }, \
+	{ "HISI  ", table_id, 0, seg + 1, MCFG_BUS_ANY, ops }, \
+	{ "HISI  ", table_id, 0, seg + 2, MCFG_BUS_ANY, ops }, \
+	{ "HISI  ", table_id, 0, seg + 3, MCFG_BUS_ANY, ops }
+	HISI_QUAD_DOM("HIP05   ",  0, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP06   ",  0, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP07   ",  0, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP07   ",  4, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP07   ",  8, &hisi_pcie_ops),
+	HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 084cb49..64845f0 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -25,7 +25,7 @@ obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
 obj-$(CONFIG_PCIE_IPROC_BCMA) += pcie-iproc-bcma.o
 obj-$(CONFIG_PCIE_ALTERA) += pcie-altera.o
 obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
-obj-$(CONFIG_PCI_HISI) += pcie-hisi.o
+obj-$(CONFIG_ARM64) += pcie-hisi.o
 obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
 obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
 obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
diff --git a/drivers/pci/host/pcie-hisi.c b/drivers/pci/host/pcie-hisi.c
index 56154c2..ee378c2 100644
--- a/drivers/pci/host/pcie-hisi.c
+++ b/drivers/pci/host/pcie-hisi.c
@@ -18,8 +18,101 @@
 #include <linux/of_pci.h>
 #include <linux/platform_device.h>
 #include <linux/of_device.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+#include <linux/pci-ecam.h>
 #include <linux/regmap.h>
 
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+
+static int hisi_pcie_acpi_rd_conf(struct pci_bus *bus, u32 devfn, int where,
+				  int size, u32 *val)
+{
+	struct pci_config_window *cfg = bus->sysdata;
+	int dev = PCI_SLOT(devfn);
+
+	if (bus->number == cfg->busr.start) {
+		/* access only one slot on each root port */
+		if (dev > 0)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		else
+			return pci_generic_config_read32(bus, devfn, where,
+							 size, val);
+	}
+
+	return pci_generic_config_read(bus, devfn, where, size, val);
+}
+
+static int hisi_pcie_acpi_wr_conf(struct pci_bus *bus, u32 devfn,
+				  int where, int size, u32 val)
+{
+	struct pci_config_window *cfg = bus->sysdata;
+	int dev = PCI_SLOT(devfn);
+
+	if (bus->number == cfg->busr.start) {
+		/* access only one slot on each root port */
+		if (dev > 0)
+			return PCIBIOS_DEVICE_NOT_FOUND;
+		else
+			return pci_generic_config_write32(bus, devfn, where,
+							  size, val);
+	}
+
+	return pci_generic_config_write(bus, devfn, where, size, val);
+}
+
+static void __iomem *hisi_pcie_map_bus(struct pci_bus *bus, unsigned int devfn,
+				       int where)
+{
+	struct pci_config_window *cfg = bus->sysdata;
+	void __iomem *reg_base = cfg->priv;
+
+	if (bus->number == cfg->busr.start)
+		return reg_base + where;
+	else
+		return pci_ecam_map_bus(bus, devfn, where);
+}
+
+static int hisi_pcie_init(struct pci_config_window *cfg)
+{
+	struct device *dev = cfg->parent;
+	struct acpi_device *adev = to_acpi_device(dev);
+	struct acpi_pci_root *root = acpi_driver_data(adev);
+	struct resource *res;
+	void __iomem *reg_base;
+
+	/*
+	 * Retrieve RC base and size from a HISI0081 device with _UID
+	 * matching our segment.
+	 */
+	res = acpi_get_rc_resources("HISI0081", root->segment);
+	if (!res) {
+		dev_err(dev, "can't get rc base address\n");
+		return -ENOMEM;
+	}
+
+	reg_base = devm_ioremap(dev, res->start, resource_size(res));
+	if (!reg_base)
+		return -ENOMEM;
+
+	cfg->priv = reg_base;
+	return 0;
+}
+
+struct pci_ecam_ops hisi_pcie_ops = {
+	.bus_shift    = 20,
+	.init         =  hisi_pcie_init,
+	.pci_ops      = {
+		.map_bus    = hisi_pcie_map_bus,
+		.read       = hisi_pcie_acpi_rd_conf,
+		.write      = hisi_pcie_acpi_wr_conf,
+	}
+};
+
+#endif
+
+#ifdef CONFIG_PCI_HISI
+
 #include "pcie-designware.h"
 
 #define PCIE_SUBCTRL_SYS_STATE4_REG		0x6818
@@ -227,3 +320,5 @@ static struct platform_driver hisi_pcie_driver = {
 	},
 };
 builtin_platform_driver(hisi_pcie_driver);
+
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 739d233..bdacbc8 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -61,6 +61,7 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
 
 #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
 extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
+extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
 #endif
 
 #ifdef CONFIG_PCI_HOST_GENERIC

[PATCH v10 10/12] PCI: thunder-pem: Factor out resource lookup

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

Pull the register resource lookup out of thunder_pem_init() so we can
easily add a corresponding lookup using ACPI.  No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/host/pci-thunder-pem.c |   50 +++++++++++++++++++-----------------
 1 file changed, 27 insertions(+), 23 deletions(-)

diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index 6abaf80..c3276ee 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -284,35 +284,16 @@ static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn,
 	return pci_generic_config_write(bus, devfn, where, size, val);
 }
 
-static int thunder_pem_init(struct pci_config_window *cfg)
+static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg,
+			    struct resource *res_pem)
 {
-	struct device *dev = cfg->parent;
-	resource_size_t bar4_start;
-	struct resource *res_pem;
 	struct thunder_pem_pci *pem_pci;
-	struct platform_device *pdev;
-
-	/* Only OF support for now */
-	if (!dev->of_node)
-		return -EINVAL;
+	resource_size_t bar4_start;
 
 	pem_pci = devm_kzalloc(dev, sizeof(*pem_pci), GFP_KERNEL);
 	if (!pem_pci)
 		return -ENOMEM;
 
-	pdev = to_platform_device(dev);
-
-	/*
-	 * The second register range is the PEM bridge to the PCIe
-	 * bus.  It has a different config access method than those
-	 * devices behind the bridge.
-	 */
-	res_pem = platform_get_resource(pdev, IORESOURCE_MEM, 1);
-	if (!res_pem) {
-		dev_err(dev, "missing \"reg[1]\"property\n");
-		return -EINVAL;
-	}
-
 	pem_pci->pem_reg_base = devm_ioremap(dev, res_pem->start, 0x10000);
 	if (!pem_pci->pem_reg_base)
 		return -ENOMEM;
@@ -332,9 +313,32 @@ static int thunder_pem_init(struct pci_config_window *cfg)
 	return 0;
 }
 
+static int thunder_pem_platform_init(struct pci_config_window *cfg)
+{
+	struct device *dev = cfg->parent;
+	struct platform_device *pdev = to_platform_device(dev);
+	struct resource *res_pem;
+
+	if (!dev->of_node)
+		return -EINVAL;
+
+	/*
+	 * The second register range is the PEM bridge to the PCIe
+	 * bus.  It has a different config access method than those
+	 * devices behind the bridge.
+	 */
+	res_pem = platform_get_resource(pdev, IORESOURCE_MEM, 1);
+	if (!res_pem) {
+		dev_err(dev, "missing \"reg[1]\"property\n");
+		return -EINVAL;
+	}
+
+	return thunder_pem_init(dev, cfg, res_pem);
+}
+
 static struct pci_ecam_ops pci_thunder_pem_ops = {
 	.bus_shift	= 24,
-	.init		= thunder_pem_init,
+	.init		= thunder_pem_platform_init,
 	.pci_ops	= {
 		.map_bus	= pci_ecam_map_bus,
 		.read		= thunder_pem_config_read,

[PATCH v10 11/12] PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

ThunderX PCIe controller to off-chip devices (so-called PEM) is not fully
compliant with ECAM standard. It uses non-standard configuration space
accessors (see thunder_pem_ecam_ops) and custom configuration space
granulation (see bus_shift = 24). In order to access configuration space
and probe PEM as ACPI-based PCI host controller we need to add MCFG quirk
infrastructure. This involves:
1. A new thunder_pem_acpi_init() init function to locate PEM-specific
   register ranges using ACPI.
2. Export PEM thunder_pem_ecam_ops structure so it is visible to MCFG quirk
   code.
3. New quirk entries for each PEM segment. Each contains platform IDs,
   mentioned thunder_pem_ecam_ops and CFG resources.

Quirk is considered for ThunderX silicon pass2.x only which is identified
via MCFG revision 1.

[bhelgaas: adapt to use acpi_get_rc_resources(), update Makefile/ifdefs so
quirk doesn't depend on CONFIG_PCI_HOST_THUNDER_PEM]
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c            |   19 ++++++++++++++++++
 drivers/pci/host/Makefile          |    2 +-
 drivers/pci/host/pci-thunder-pem.c |   38 ++++++++++++++++++++++++++++++++++++
 include/linux/pci-ecam.h           |    1 +
 4 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index c6a6949..a48b508 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -74,6 +74,25 @@ static struct mcfg_fixup mcfg_quirks[] = {
 	HISI_QUAD_DOM("HIP07   ",  4, &hisi_pcie_ops),
 	HISI_QUAD_DOM("HIP07   ",  8, &hisi_pcie_ops),
 	HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),
+
+#define THUNDER_PEM_RES(addr, node) \
+	DEFINE_RES_MEM(addr + (node << 44), 0x39 * SZ_16M)
+#define THUNDER_PEM_QUIRK(rev, node) \
+	{ "CAVIUM", "THUNDERX", rev, 4 + (10 * node), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88001f000000UL, node) }, \
+	{ "CAVIUM", "THUNDERX", rev, 5 + (10 * node), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x884057000000UL, node) }, \
+	{ "CAVIUM", "THUNDERX", rev, 6 + (10 * node), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88808f000000UL, node) }, \
+	{ "CAVIUM", "THUNDERX", rev, 7 + (10 * node), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89001f000000UL, node) }, \
+	{ "CAVIUM", "THUNDERX", rev, 8 + (10 * node), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x894057000000UL, node) }, \
+	{ "CAVIUM", "THUNDERX", rev, 9 + (10 * node), MCFG_BUS_ANY,	    \
+	  &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89808f000000UL, node) }
+	/* SoC pass2.x */
+	THUNDER_PEM_QUIRK(1, 0UL),
+	THUNDER_PEM_QUIRK(1, 1UL),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 64845f0..97e6bfc 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -28,7 +28,7 @@ obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
 obj-$(CONFIG_ARM64) += pcie-hisi.o
 obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
 obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
-obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
+obj-$(CONFIG_ARM64) += pci-thunder-pem.o
 obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
 obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
 obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index c3276ee..7b03939 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -18,9 +18,12 @@
 #include <linux/init.h>
 #include <linux/of_address.h>
 #include <linux/of_pci.h>
+#include <linux/pci-acpi.h>
 #include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 
+#if defined(CONFIG_PCI_HOST_THUNDER_PEM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
+
 #define PEM_CFG_WR 0x28
 #define PEM_CFG_RD 0x30
 
@@ -313,6 +316,38 @@ static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg,
 	return 0;
 }
 
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
+
+static int thunder_pem_acpi_init(struct pci_config_window *cfg)
+{
+	struct device *dev = cfg->parent;
+	struct acpi_device *adev = to_acpi_device(dev);
+	struct acpi_pci_root *root = acpi_driver_data(adev);
+	struct resource *res_pem;
+
+	ret = acpi_get_rc_resources("THRX0002", root->segment, res_pem);
+	if (ret) {
+		dev_err(dev, "can't get rc base address\n");
+		return ret;
+	}
+
+	return thunder_pem_init(dev, cfg, res_pem);
+}
+
+struct pci_ecam_ops thunder_pem_ecam_ops = {
+	.bus_shift	= 24,
+	.init		= thunder_pem_acpi_init,
+	.pci_ops	= {
+		.map_bus	= pci_ecam_map_bus,
+		.read		= thunder_pem_config_read,
+		.write		= thunder_pem_config_write,
+	}
+};
+
+#endif
+
+#ifdef CONFIG_PCI_HOST_THUNDER_PEM
+
 static int thunder_pem_platform_init(struct pci_config_window *cfg)
 {
 	struct device *dev = cfg->parent;
@@ -364,3 +399,6 @@ static struct platform_driver thunder_pem_driver = {
 	.probe = thunder_pem_probe,
 };
 builtin_platform_driver(thunder_pem_driver);
+
+#endif
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index bdacbc8..e88d7db 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -62,6 +62,7 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
 #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
 extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
 extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
+extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 2.x */
 #endif
 
 #ifdef CONFIG_PCI_HOST_GENERIC

Re: [PATCH v10 11/12] PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller

[Bjorn Helgaas]

On Thu, Dec 01, 2016 at 02:30:58AM -0600, Bjorn Helgaas wrote:
> From: Tomasz Nowicki <tn@semihalf.com>
> 
> ThunderX PCIe controller to off-chip devices (so-called PEM) is not fully
> compliant with ECAM standard. It uses non-standard configuration space
> accessors (see thunder_pem_ecam_ops) and custom configuration space
> granulation (see bus_shift = 24). In order to access configuration space
> and probe PEM as ACPI-based PCI host controller we need to add MCFG quirk
> infrastructure. This involves:
> 1. A new thunder_pem_acpi_init() init function to locate PEM-specific
>    register ranges using ACPI.
> 2. Export PEM thunder_pem_ecam_ops structure so it is visible to MCFG quirk
>    code.
> 3. New quirk entries for each PEM segment. Each contains platform IDs,
>    mentioned thunder_pem_ecam_ops and CFG resources.
> 
> Quirk is considered for ThunderX silicon pass2.x only which is identified
> via MCFG revision 1.
> 
> [bhelgaas: adapt to use acpi_get_rc_resources(), update Makefile/ifdefs so
> quirk doesn't depend on CONFIG_PCI_HOST_THUNDER_PEM]
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
>  drivers/acpi/pci_mcfg.c            |   19 ++++++++++++++++++
>  drivers/pci/host/Makefile          |    2 +-
>  drivers/pci/host/pci-thunder-pem.c |   38 ++++++++++++++++++++++++++++++++++++
>  include/linux/pci-ecam.h           |    1 +
>  4 files changed, 59 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> index c6a6949..a48b508 100644
> --- a/drivers/acpi/pci_mcfg.c
> +++ b/drivers/acpi/pci_mcfg.c
> @@ -74,6 +74,25 @@ static struct mcfg_fixup mcfg_quirks[] = {
>  	HISI_QUAD_DOM("HIP07   ",  4, &hisi_pcie_ops),
>  	HISI_QUAD_DOM("HIP07   ",  8, &hisi_pcie_ops),
>  	HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),
> +
> +#define THUNDER_PEM_RES(addr, node) \
> +	DEFINE_RES_MEM(addr + (node << 44), 0x39 * SZ_16M)

I assume we should do s/node/seg/ here as well?  I already did that for
THUNDER_ECAM_QUIRK in the next patch.

Re: [PATCH v10 11/12] PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller

[Tomasz Nowicki]

On 02.12.2016 18:35, Bjorn Helgaas wrote:
> On Thu, Dec 01, 2016 at 02:30:58AM -0600, Bjorn Helgaas wrote:
>> From: Tomasz Nowicki <tn@semihalf.com>
>>
>> ThunderX PCIe controller to off-chip devices (so-called PEM) is not fully
>> compliant with ECAM standard. It uses non-standard configuration space
>> accessors (see thunder_pem_ecam_ops) and custom configuration space
>> granulation (see bus_shift = 24). In order to access configuration space
>> and probe PEM as ACPI-based PCI host controller we need to add MCFG quirk
>> infrastructure. This involves:
>> 1. A new thunder_pem_acpi_init() init function to locate PEM-specific
>>    register ranges using ACPI.
>> 2. Export PEM thunder_pem_ecam_ops structure so it is visible to MCFG quirk
>>    code.
>> 3. New quirk entries for each PEM segment. Each contains platform IDs,
>>    mentioned thunder_pem_ecam_ops and CFG resources.
>>
>> Quirk is considered for ThunderX silicon pass2.x only which is identified
>> via MCFG revision 1.
>>
>> [bhelgaas: adapt to use acpi_get_rc_resources(), update Makefile/ifdefs so
>> quirk doesn't depend on CONFIG_PCI_HOST_THUNDER_PEM]
>> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
>> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>> ---
>>  drivers/acpi/pci_mcfg.c            |   19 ++++++++++++++++++
>>  drivers/pci/host/Makefile          |    2 +-
>>  drivers/pci/host/pci-thunder-pem.c |   38 ++++++++++++++++++++++++++++++++++++
>>  include/linux/pci-ecam.h           |    1 +
>>  4 files changed, 59 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
>> index c6a6949..a48b508 100644
>> --- a/drivers/acpi/pci_mcfg.c
>> +++ b/drivers/acpi/pci_mcfg.c
>> @@ -74,6 +74,25 @@ static struct mcfg_fixup mcfg_quirks[] = {
>>  	HISI_QUAD_DOM("HIP07   ",  4, &hisi_pcie_ops),
>>  	HISI_QUAD_DOM("HIP07   ",  8, &hisi_pcie_ops),
>>  	HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),
>> +
>> +#define THUNDER_PEM_RES(addr, node) \
>> +	DEFINE_RES_MEM(addr + (node << 44), 0x39 * SZ_16M)
>
> I assume we should do s/node/seg/ here as well?  I already did that for
> THUNDER_ECAM_QUIRK in the next patch.
>

Here node means NUMA node number so it is fine.

Thanks,
Tomasz

[PATCH v10 12/12] PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller

[Bjorn Helgaas]

From: Tomasz Nowicki <tn@semihalf.com>

ThunderX pass1.x requires to emulate the EA headers for on-chip devices
hence it has to use custom pci_thunder_ecam_ops for accessing PCI config
space (pci-thuner-ecam.c). Add new entries to MCFG quirk array where it can
be applied while probing ACPI based PCI host controller.

ThunderX pass1.x is using the same way for accessing off-chip devices
(so-called PEM) as silicon pass-2.x so we need to add PEM quirk entries
too.

Quirk is considered for ThunderX silicon pass1.x only which is identified
via MCFG revision 2.

[bhelgaas: change Makefile/ifdefs so quirk doesn't depend on
CONFIG_PCI_HOST_THUNDER_ECAM]
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/acpi/pci_mcfg.c             |   15 +++++++++++++++
 drivers/pci/host/Makefile           |    2 +-
 drivers/pci/host/pci-thunder-ecam.c |    9 ++++++++-
 include/linux/pci-ecam.h            |    1 +
 4 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index a48b508..cdceaf5 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -93,6 +93,21 @@ static struct mcfg_fixup mcfg_quirks[] = {
 	/* SoC pass2.x */
 	THUNDER_PEM_QUIRK(1, 0UL),
 	THUNDER_PEM_QUIRK(1, 1UL),
+
+#define THUNDER_ECAM_QUIRK(rev, node)					\
+	{ "CAVIUM", "THUNDERX", rev, node, MCFG_BUS_ANY,		\
+	&pci_thunder_ecam_ops },
+	/* SoC pass1.x */
+	THUNDER_PEM_QUIRK(2,  0),	/* off-chip devices */
+	THUNDER_PEM_QUIRK(2,  1),	/* off-chip devices */
+	THUNDER_ECAM_QUIRK(2,  0),
+	THUNDER_ECAM_QUIRK(2,  1),
+	THUNDER_ECAM_QUIRK(2,  2),
+	THUNDER_ECAM_QUIRK(2,  3),
+	THUNDER_ECAM_QUIRK(2, 10),
+	THUNDER_ECAM_QUIRK(2, 11),
+	THUNDER_ECAM_QUIRK(2, 12),
+	THUNDER_ECAM_QUIRK(2, 13),
 };
 
 static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
index 97e6bfc..639494a 100644
--- a/drivers/pci/host/Makefile
+++ b/drivers/pci/host/Makefile
@@ -27,7 +27,7 @@ obj-$(CONFIG_PCIE_ALTERA) += pcie-altera.o
 obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
 obj-$(CONFIG_ARM64) += pcie-hisi.o
 obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
-obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
+obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
 obj-$(CONFIG_ARM64) += pci-thunder-pem.o
 obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
 obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
index d50a3dc..3f54a43 100644
--- a/drivers/pci/host/pci-thunder-ecam.c
+++ b/drivers/pci/host/pci-thunder-ecam.c
@@ -14,6 +14,8 @@
 #include <linux/pci-ecam.h>
 #include <linux/platform_device.h>
 
+#if defined(CONFIG_PCI_HOST_THUNDER_ECAM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
+
 static void set_val(u32 v, int where, int size, u32 *val)
 {
 	int shift = (where & 3) * 8;
@@ -346,7 +348,7 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn,
 	return pci_generic_config_write(bus, devfn, where, size, val);
 }
 
-static struct pci_ecam_ops pci_thunder_ecam_ops = {
+struct pci_ecam_ops pci_thunder_ecam_ops = {
 	.bus_shift	= 20,
 	.pci_ops	= {
 		.map_bus        = pci_ecam_map_bus,
@@ -355,6 +357,8 @@ static struct pci_ecam_ops pci_thunder_ecam_ops = {
 	}
 };
 
+#ifdef CONFIG_PCI_HOST_THUNDER_ECAM
+
 static const struct of_device_id thunder_ecam_of_match[] = {
 	{ .compatible = "cavium,pci-host-thunder-ecam" },
 	{ },
@@ -373,3 +377,6 @@ static struct platform_driver thunder_ecam_driver = {
 	.probe = thunder_ecam_probe,
 };
 builtin_platform_driver(thunder_ecam_driver);
+
+#endif
+#endif
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index e88d7db..2afa70b 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -63,6 +63,7 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
 extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
 extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
 extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 2.x */
+extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */
 #endif
 
 #ifdef CONFIG_PCI_HOST_GENERIC

Re: [PATCH v10 12/12] PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller

[Tomasz Nowicki]

On 01.12.2016 09:31, Bjorn Helgaas wrote:
> From: Tomasz Nowicki <tn@semihalf.com>
>
> ThunderX pass1.x requires to emulate the EA headers for on-chip devices
> hence it has to use custom pci_thunder_ecam_ops for accessing PCI config
> space (pci-thuner-ecam.c). Add new entries to MCFG quirk array where it can
> be applied while probing ACPI based PCI host controller.
>
> ThunderX pass1.x is using the same way for accessing off-chip devices
> (so-called PEM) as silicon pass-2.x so we need to add PEM quirk entries
> too.
>
> Quirk is considered for ThunderX silicon pass1.x only which is identified
> via MCFG revision 2.
>
> [bhelgaas: change Makefile/ifdefs so quirk doesn't depend on
> CONFIG_PCI_HOST_THUNDER_ECAM]
> Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
>  drivers/acpi/pci_mcfg.c             |   15 +++++++++++++++
>  drivers/pci/host/Makefile           |    2 +-
>  drivers/pci/host/pci-thunder-ecam.c |    9 ++++++++-
>  include/linux/pci-ecam.h            |    1 +
>  4 files changed, 25 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> index a48b508..cdceaf5 100644
> --- a/drivers/acpi/pci_mcfg.c
> +++ b/drivers/acpi/pci_mcfg.c
> @@ -93,6 +93,21 @@ static struct mcfg_fixup mcfg_quirks[] = {
>  	/* SoC pass2.x */
>  	THUNDER_PEM_QUIRK(1, 0UL),
>  	THUNDER_PEM_QUIRK(1, 1UL),
> +
> +#define THUNDER_ECAM_QUIRK(rev, node)					\
> +	{ "CAVIUM", "THUNDERX", rev, node, MCFG_BUS_ANY,		\
> +	&pci_thunder_ecam_ops },

Nit: 0-3 and 10-11 are segment ranges:

+#define THUNDER_ECAM_QUIRK(rev, seg)			\
+	{ "CAVIUM", "THUNDERX", rev, seg, MCFG_BUS_ANY,	\
+	&pci_thunder_ecam_ops },

> +	/* SoC pass1.x */
> +	THUNDER_PEM_QUIRK(2,  0),	/* off-chip devices */
> +	THUNDER_PEM_QUIRK(2,  1),	/* off-chip devices */
> +	THUNDER_ECAM_QUIRK(2,  0),
> +	THUNDER_ECAM_QUIRK(2,  1),
> +	THUNDER_ECAM_QUIRK(2,  2),
> +	THUNDER_ECAM_QUIRK(2,  3),
> +	THUNDER_ECAM_QUIRK(2, 10),
> +	THUNDER_ECAM_QUIRK(2, 11),
> +	THUNDER_ECAM_QUIRK(2, 12),
> +	THUNDER_ECAM_QUIRK(2, 13),
>  };
>
>  static char mcfg_oem_id[ACPI_OEM_ID_SIZE];
> diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile
> index 97e6bfc..639494a 100644
> --- a/drivers/pci/host/Makefile
> +++ b/drivers/pci/host/Makefile
> @@ -27,7 +27,7 @@ obj-$(CONFIG_PCIE_ALTERA) += pcie-altera.o
>  obj-$(CONFIG_PCIE_ALTERA_MSI) += pcie-altera-msi.o
>  obj-$(CONFIG_ARM64) += pcie-hisi.o
>  obj-$(CONFIG_PCIE_QCOM) += pcie-qcom.o
> -obj-$(CONFIG_PCI_HOST_THUNDER_ECAM) += pci-thunder-ecam.o
> +obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
>  obj-$(CONFIG_ARM64) += pci-thunder-pem.o
>  obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
>  obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
> diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
> index d50a3dc..3f54a43 100644
> --- a/drivers/pci/host/pci-thunder-ecam.c
> +++ b/drivers/pci/host/pci-thunder-ecam.c
> @@ -14,6 +14,8 @@
>  #include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
>
> +#if defined(CONFIG_PCI_HOST_THUNDER_ECAM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
> +
>  static void set_val(u32 v, int where, int size, u32 *val)
>  {
>  	int shift = (where & 3) * 8;
> @@ -346,7 +348,7 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn,
>  	return pci_generic_config_write(bus, devfn, where, size, val);
>  }
>
> -static struct pci_ecam_ops pci_thunder_ecam_ops = {
> +struct pci_ecam_ops pci_thunder_ecam_ops = {
>  	.bus_shift	= 20,
>  	.pci_ops	= {
>  		.map_bus        = pci_ecam_map_bus,
> @@ -355,6 +357,8 @@ static struct pci_ecam_ops pci_thunder_ecam_ops = {
>  	}
>  };
>
> +#ifdef CONFIG_PCI_HOST_THUNDER_ECAM
> +
>  static const struct of_device_id thunder_ecam_of_match[] = {
>  	{ .compatible = "cavium,pci-host-thunder-ecam" },
>  	{ },
> @@ -373,3 +377,6 @@ static struct platform_driver thunder_ecam_driver = {
>  	.probe = thunder_ecam_probe,
>  };
>  builtin_platform_driver(thunder_ecam_driver);
> +
> +#endif
> +#endif
> diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
> index e88d7db..2afa70b 100644
> --- a/include/linux/pci-ecam.h
> +++ b/include/linux/pci-ecam.h
> @@ -63,6 +63,7 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
>  extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
>  extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
>  extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 2.x */
> +extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */

nit: thunder_pem_ecam_ops is also valid for ThunderX pass1.x too. We add 
relevant entries to mcfg_quirks array above. Also we use passY.X for SoC 
version.

extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 
pass2.x and pass1.x */
extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 
pass1.x */

Thanks for this patch set Bjorn!
Tomasz

Re: [PATCH v10 12/12] PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller

[Bjorn Helgaas]

On Thu, Dec 01, 2016 at 08:02:29PM +0100, Tomasz Nowicki wrote:
> On 01.12.2016 09:31, Bjorn Helgaas wrote:
> >From: Tomasz Nowicki <tn@semihalf.com>
> >
> >ThunderX pass1.x requires to emulate the EA headers for on-chip devices
> >hence it has to use custom pci_thunder_ecam_ops for accessing PCI config
> >space (pci-thuner-ecam.c). Add new entries to MCFG quirk array where it can
> >be applied while probing ACPI based PCI host controller.
> >
> >ThunderX pass1.x is using the same way for accessing off-chip devices
> >(so-called PEM) as silicon pass-2.x so we need to add PEM quirk entries
> >too.
> >
> >Quirk is considered for ThunderX silicon pass1.x only which is identified
> >via MCFG revision 2.
> >
> >[bhelgaas: change Makefile/ifdefs so quirk doesn't depend on
> >CONFIG_PCI_HOST_THUNDER_ECAM]
> >Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
> >Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> >---
> > drivers/acpi/pci_mcfg.c             |   15 +++++++++++++++
> > drivers/pci/host/Makefile           |    2 +-
> > drivers/pci/host/pci-thunder-ecam.c |    9 ++++++++-
> > include/linux/pci-ecam.h            |    1 +
> > 4 files changed, 25 insertions(+), 2 deletions(-)
> >
> >diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> >index a48b508..cdceaf5 100644
> >--- a/drivers/acpi/pci_mcfg.c
> >+++ b/drivers/acpi/pci_mcfg.c
> >@@ -93,6 +93,21 @@ static struct mcfg_fixup mcfg_quirks[] = {
> > 	/* SoC pass2.x */
> > 	THUNDER_PEM_QUIRK(1, 0UL),
> > 	THUNDER_PEM_QUIRK(1, 1UL),
> >+
> >+#define THUNDER_ECAM_QUIRK(rev, node)					\
> >+	{ "CAVIUM", "THUNDERX", rev, node, MCFG_BUS_ANY,		\
> >+	&pci_thunder_ecam_ops },
> 
> Nit: 0-3 and 10-11 are segment ranges:
> 
> +#define THUNDER_ECAM_QUIRK(rev, seg)			\
> +	{ "CAVIUM", "THUNDERX", rev, seg, MCFG_BUS_ANY,	\
> +	&pci_thunder_ecam_ops },

> > extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 2.x */
> >+extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */
> 
> nit: thunder_pem_ecam_ops is also valid for ThunderX pass1.x too. We
> add relevant entries to mcfg_quirks array above. Also we use passY.X
> for SoC version.
> 
> extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX
> pass2.x and pass1.x */
> extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX
> pass1.x */

I updated these with the following incremental diff.  I hope this is what
you meant!

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index f4e9c1c..d34d196 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -95,7 +95,7 @@ static struct mcfg_fixup mcfg_quirks[] = {
 	THUNDER_PEM_QUIRK(1, 1UL),
 
 #define THUNDER_ECAM_QUIRK(rev, seg)					\
-	{ "CAVIUM", "THUNDERX", rev, seg, MCFG_BUS_ANY,		\
+	{ "CAVIUM", "THUNDERX", rev, seg, MCFG_BUS_ANY,			\
 	&pci_thunder_ecam_ops }
 	/* SoC pass1.x */
 	THUNDER_PEM_QUIRK(2,  0),	/* off-chip devices */
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 2afa70b..00eb8eb 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -62,7 +62,7 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
 #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
 extern struct pci_ecam_ops pci_32b_ops;		/* 32-bit accesses only */
 extern struct pci_ecam_ops hisi_pcie_ops;	/* HiSilicon */
-extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 2.x */
+extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 1.x & 2.x */
 extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */
 #endif
 

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Dongdong Liu]

Hi Bjorn

Thank you for reworking the patchset.
I found some complie errors and can fix the errors by the below modification.

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index cdceaf5..c84330a 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -53,7 +53,7 @@ struct mcfg_fixup {
  /*     { OEM_ID, OEM_TABLE_ID, REV, SEGMENT, BUS_RANGE, ops, cfgres }, */

  #define QCOM_ECAM32(seg) \
-   { "QCOM  ", "QDF2432 ", 1, seg, MCFG_BUS_ANY, &pci_32b_ops },
+ { "QCOM  ", "QDF2432 ", 1, seg, MCFG_BUS_ANY, &pci_32b_ops }
         QCOM_ECAM32(0),
         QCOM_ECAM32(1),
         QCOM_ECAM32(2),
@@ -96,7 +96,7 @@ struct mcfg_fixup {

  #define THUNDER_ECAM_QUIRK(rev, node)                                  \
         { "CAVIUM", "THUNDERX", rev, node, MCFG_BUS_ANY,                \
-   &pci_thunder_ecam_ops },
+ &pci_thunder_ecam_ops }
         /* SoC pass1.x */
         THUNDER_PEM_QUIRK(2,  0),       /* off-chip devices */
         THUNDER_PEM_QUIRK(2,  1),       /* off-chip devices */
diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index 7b03939..2066410 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -21,6 +21,7 @@
  #include <linux/pci-acpi.h>
  #include <linux/pci-ecam.h>
  #include <linux/platform_device.h>
+#include "../pci.h"

  #if defined(CONFIG_PCI_HOST_THUNDER_PEM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))

@@ -324,6 +325,11 @@ static int thunder_pem_acpi_init(struct pci_config_window *cfg)
         struct acpi_device *adev = to_acpi_device(dev);
         struct acpi_pci_root *root = acpi_driver_data(adev);
         struct resource *res_pem;
+ int ret;
+
+ res_pem = devm_kzalloc(&adev->dev, sizeof(*res_pem), GFP_KERNEL);
+ if (!res_pem)
+         return -ENOMEM;

         ret = acpi_get_rc_resources("THRX0002", root->segment, res_pem);
         if (ret) {
diff --git a/drivers/pci/host/pcie-hisi.c b/drivers/pci/host/pcie-hisi.c
index ee378c2..6b39939 100644
--- a/drivers/pci/host/pcie-hisi.c
+++ b/drivers/pci/host/pcie-hisi.c
@@ -22,6 +22,7 @@
  #include <linux/pci-acpi.h>
  #include <linux/pci-ecam.h>
  #include <linux/regmap.h>
+#include "../pci.h"

  #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)

@@ -80,15 +81,20 @@ static int hisi_pcie_init(struct pci_config_window *cfg)
         struct acpi_pci_root *root = acpi_driver_data(adev);
         struct resource *res;
         void __iomem *reg_base;
+ int ret;
+
+ res = devm_kzalloc(&adev->dev, sizeof(*res), GFP_KERNEL);
+ if (!res)
+         return -ENOMEM;

         /*
          * Retrieve RC base and size from a HISI0081 device with _UID
          * matching our segment.
          */
-   res = acpi_get_rc_resources("HISI0081", root->segment);
-   if (!res) {
-           dev_err(dev, "can't get rc base address\n");
-           return -ENOMEM;
+ ret = acpi_get_rc_resources("HISI0081", root->segment, res);
+ if (ret) {
+         dev_err(&adev->dev, "can't get rc base address");
+         return ret;
         }

         reg_base = devm_ioremap(dev, res->start, resource_size(res));


Thanks,
Dongdong
在 2016/12/1 16:29, Bjorn Helgaas 写道:
> This is what I've collected on my pci/ecam branch so far.  This is
> preliminary and probably doesn't even build since I made several
> changes, but at least it's a strawman.  I called it v10 to keep it
> separate from the previous postings of the various pieces.
>
>   ACPI: Add acpi_resource_consumer() to find device that claims a resource
>   PCI: Search ACPI namespace to ensure ECAM space is reserved
>   x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG
>
>     acpi_resource_consumer() takes a resource and returns an
>     acpi_device that has _CRS that contains the resource.  The idea is
>     that we call this on a resource from MCFG.  We *should* find a
>     PNP0C02 or other device that reserves that resource.  On x86 we
>     already do this in a more hand-coded way.  I probably wouldn't
>     merge the x86 patch for v4.10 but I included it here as an
>     example.
>
>     I don't think it's worth trying to fabricate ACPI devices or _CRS
>     resources to compensate for firmware that doesn't describe
>     everything.  This check will emit a warning if the MCFG region is
>     not reserved, and that's probably enough to motivate firmware
>     fixes for the next round of hardware.  The ECAM code itself does
>     reserve the region, so that part will be safe from other users.
>     Unreported non-ECAM register space is still a landmine and there's
>     no real way to look for it.
>
>   arm64: PCI: Manage controller-specific data on per-controller basis
>
>     Tomasz's fix for pci_acpi_scan_root(), unchanged.
>
>   PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors
>   PCI/ACPI: Check for platform-specific MCFG quirks
>
>     Tomasz's quirk infrastructure.  I put this under
>     CONFIG_PCI_QUIRKS.
>
>   PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform
>
>     Dongdong's interface to look up a _HID with a _UID matching the
>     segment.  I don't have an opinion on using _UID yet.  It's
>     encapsulated so it could be changed easily.  I put this under
>     CONFIG_PCI_QUIRKS && CONFIG_ARM64.
>
>   PCI: Add MCFG quirks for Qualcomm QDF2432 host controller
>
>     Christopher's Qualcomm quirks.  Basically unchanged except to put
>     the quirk ops under CONFIG_ACPI and CONFIG_PCI_QUIRKS.
>
>   PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers
>
>     Dongdong's HiSilicon quirks.  Here's where it gets interesting.  I
>     moved this to the existing pcie-hisi.c instead of adding
>     pcie-hisi-acpi.c.  I changed the Makefile so we always build
>     pcie-hisi.c on ARM64.  I added ifdefs so we get the quirk code if
>     CONFIG_ACPI and CONFIG_PCI_QUIRKS and we get the original platform
>     driver if CONFIG_PCI_HISI.  It's possible to have both, and if we
>     process the MCFG quirk we get the quirk code.
>
>     I'm confused about why the quirk accessors are so much different
>     than the original accessors.  hisi_pcie_acpi_rd_conf() looks much
>     different than hisi_pcie_cfg_read().  The original driver claims
>     Hipxx only supports 32-bit config accesses, but the quirk
>     accessors don't enforce that.
>
>   PCI: thunder-pem: Factor out resource lookup
>   PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller
>   PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller
>
>     Tomasz's ThunderX quirks.  I restructured the PEM init I could do
>     the same ifdef tricks as in hisi.  The quirks make a lot more
>     sense here -- they use the same accessors as the original platform
>     driver.  Only the initialization is different.
>
> The Makefile looks a little strange:
>
>   obj-$(CONFIG_ARM64) += pcie-hisi.o
>   obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
>   obj-$(CONFIG_ARM64) += pci-thunder-pem.o
>
> and the ifdefs inside those files are a little unwieldy.  But I think
> they do accomplish the goals of:
>
>   - Adding no new files,
>
>   - Including the MCFG quirks whenever CONFIG_ACPI and
>     CONFIG_PCI_QUIRKS are enabled, regardless of the driver-specific
>     config,
>
>   - Including the original platform drivers when they are specifically
>     enabled, and
>
>   - Sharing the same code for the platform driver and the quirks
>     (except for HiSi, which I don't understand)
>
> I started looking at the Duc's X-Gene quirks, but I haven't wrapped my
> head around those yet.  I'm going to push this branch to get build
> testing and so you can comment on it.
>
> If by some miracle it builds and you test it, please collect the dmesg
> log and /proc/iomem contents.
>
> ---
>
> Bjorn Helgaas (5):
>       ACPI: Add acpi_resource_consumer() to find device that claims a resource
>       PCI: Search ACPI namespace to ensure ECAM space is reserved
>       x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG
>       PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers
>       PCI: thunder-pem: Factor out resource lookup
>
> Christopher Covington (1):
>       PCI: Add MCFG quirks for Qualcomm QDF2432 host controller
>
> Dongdong Liu (1):
>       PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform
>
> Tomasz Nowicki (5):
>       arm64: PCI: Manage controller-specific data on per-controller basis
>       PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors
>       PCI/ACPI: Check for platform-specific MCFG quirks
>       PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller
>       PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller
>
>
>  arch/arm64/kernel/pci.c             |   34 +++----
>  arch/x86/pci/mmconfig-shared.c      |   69 ++-------------
>  drivers/acpi/pci_mcfg.c             |  165 ++++++++++++++++++++++++++++++++++-
>  drivers/acpi/resource.c             |   57 ++++++++++++
>  drivers/pci/ecam.c                  |   31 +++++++
>  drivers/pci/host/Makefile           |    6 +
>  drivers/pci/host/pci-thunder-ecam.c |    9 ++
>  drivers/pci/host/pci-thunder-pem.c  |   88 ++++++++++++++-----
>  drivers/pci/host/pcie-hisi.c        |   95 ++++++++++++++++++++
>  drivers/pci/pci-acpi.c              |   71 +++++++++++++++
>  drivers/pci/pci.h                   |    4 +
>  include/linux/acpi.h                |    7 +
>  include/linux/pci-acpi.h            |    4 +
>  include/linux/pci-ecam.h            |    7 +
>  14 files changed, 537 insertions(+), 110 deletions(-)
>
> .
>

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Dongdong Liu]

在 2016/12/1 21:02, Dongdong Liu 写道:
> Hi Bjorn
>
> Thank you for reworking the patchset.
> I found some complie errors and can fix the errors by the below modification.
>
> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> index cdceaf5..c84330a 100644
> --- a/drivers/acpi/pci_mcfg.c
> +++ b/drivers/acpi/pci_mcfg.c
> @@ -53,7 +53,7 @@ struct mcfg_fixup {
>  /*     { OEM_ID, OEM_TABLE_ID, REV, SEGMENT, BUS_RANGE, ops, cfgres }, */
>
>  #define QCOM_ECAM32(seg) \
> -   { "QCOM  ", "QDF2432 ", 1, seg, MCFG_BUS_ANY, &pci_32b_ops },
> + { "QCOM  ", "QDF2432 ", 1, seg, MCFG_BUS_ANY, &pci_32b_ops }
>         QCOM_ECAM32(0),
>         QCOM_ECAM32(1),
>         QCOM_ECAM32(2),
> @@ -96,7 +96,7 @@ struct mcfg_fixup {
>
>  #define THUNDER_ECAM_QUIRK(rev, node)                                  \
>         { "CAVIUM", "THUNDERX", rev, node, MCFG_BUS_ANY,                \
> -   &pci_thunder_ecam_ops },
> + &pci_thunder_ecam_ops }
>         /* SoC pass1.x */
>         THUNDER_PEM_QUIRK(2,  0),       /* off-chip devices */
>         THUNDER_PEM_QUIRK(2,  1),       /* off-chip devices */
> diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
> index 7b03939..2066410 100644
> --- a/drivers/pci/host/pci-thunder-pem.c
> +++ b/drivers/pci/host/pci-thunder-pem.c
> @@ -21,6 +21,7 @@
>  #include <linux/pci-acpi.h>
>  #include <linux/pci-ecam.h>
>  #include <linux/platform_device.h>
> +#include "../pci.h"
>
>  #if defined(CONFIG_PCI_HOST_THUNDER_PEM) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS))
>
> @@ -324,6 +325,11 @@ static int thunder_pem_acpi_init(struct pci_config_window *cfg)
>         struct acpi_device *adev = to_acpi_device(dev);
>         struct acpi_pci_root *root = acpi_driver_data(adev);
>         struct resource *res_pem;
> + int ret;
> +
> + res_pem = devm_kzalloc(&adev->dev, sizeof(*res_pem), GFP_KERNEL);
> + if (!res_pem)
> +         return -ENOMEM;
>
>         ret = acpi_get_rc_resources("THRX0002", root->segment, res_pem);
>         if (ret) {
> diff --git a/drivers/pci/host/pcie-hisi.c b/drivers/pci/host/pcie-hisi.c
> index ee378c2..6b39939 100644
> --- a/drivers/pci/host/pcie-hisi.c
> +++ b/drivers/pci/host/pcie-hisi.c
> @@ -22,6 +22,7 @@
>  #include <linux/pci-acpi.h>
>  #include <linux/pci-ecam.h>
>  #include <linux/regmap.h>
> +#include "../pci.h"
>
>  #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
>
> @@ -80,15 +81,20 @@ static int hisi_pcie_init(struct pci_config_window *cfg)
>         struct acpi_pci_root *root = acpi_driver_data(adev);
>         struct resource *res;
>         void __iomem *reg_base;
> + int ret;
> +
> + res = devm_kzalloc(&adev->dev, sizeof(*res), GFP_KERNEL);
> + if (!res)
> +         return -ENOMEM;
>
>         /*
>          * Retrieve RC base and size from a HISI0081 device with _UID
>          * matching our segment.
>          */
> -   res = acpi_get_rc_resources("HISI0081", root->segment);
> -   if (!res) {
> -           dev_err(dev, "can't get rc base address\n");
> -           return -ENOMEM;
> + ret = acpi_get_rc_resources("HISI0081", root->segment, res);
> + if (ret) {
> +         dev_err(&adev->dev, "can't get rc base address");
> +         return ret;
>         }
>
>         reg_base = devm_ioremap(dev, res->start, resource_size(res));
>
>
> Thanks,
> Dongdong

Aftet fixing the compile errors. I tested on HiSilicon D03 board. It works ok.
The dmesg is as below

root@(none)$ dmesg
[    0.000000] Booting Linux on physical CPU 0x10000
[    0.000000] Linux version 4.9.0-rc1-gaf05ef2-dirty (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #257 SMP PREEMPT Thu Dec 1 22:13:05 CST 2016
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] earlycon: hisilpcuart0 at MMIO 0x00000000a01b0000 (options '0,0x2f8')
[    0.000000] bootconsole [hisilpcuart0] enabled
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: EFI v2.60 by EDK II
[    0.000000] efi:  SMBIOS=0x3f110000  SMBIOS 3.0=0x39ce0000  ACPI=0x39db0000  ACPI 2.0=0x39db0014  MEMATTR=0x3c945018
[    0.000000] cma: Reserved 16 MiB at 0x000000003e000000
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x0000000039DB0014 000024 (v02 HISI  )
[    0.000000] ACPI: XSDT 0x0000000039DA00E8 00004C (v01 HISI   HIP06    00000000      01000013)
[    0.000000] ACPI: FACP 0x0000000039CA0000 00010C (v05 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: DSDT 0x0000000039C60000 0038CE (v01 HISI   HIP06    00000000 INTL 20150619)
[    0.000000] ACPI: MCFG 0x0000000039CC0000 00005C (v01 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: GTDT 0x0000000039C90000 000098 (v02 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: APIC 0x0000000039C80000 000518 (v01 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: IORT 0x0000000039C70000 0002E8 (v00 HISI   HIP06    00000000 INTL 20150619)
[    0.000000] ACPI: NUMA: Failed to initialise from firmware
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000027ffffffff]
[    0.000000] NUMA: Adding memblock [0x0 - 0x257ff] on node 0
[    0.000000] NUMA: Adding memblock [0x25800 - 0x319fffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31a00000 - 0x31a6ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31a70000 - 0x31b5ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b60000 - 0x31b6ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b70000 - 0x31b7dfff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b7e000 - 0x31b81fff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b82000 - 0x39c0ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x39c10000 - 0x3a11ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3a120000 - 0x3f10ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3f110000 - 0x3f13ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3f140000 - 0x3fbfffff] on node 0
[    0.000000] NUMA: Adding memblock [0x2040000000 - 0x27ffffffff] on node 0
[    0.000000] NUMA: Initmem setup node 0 [mem 0x00000000-0x27ffffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x27fffef500-0x27ffff0fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000027ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000000024fff]
[    0.000000]   node   0: [mem 0x0000000000026000-0x00000000319fffff]
[    0.000000]   node   0: [mem 0x0000000031a00000-0x0000000031a6ffff]
[    0.000000]   node   0: [mem 0x0000000031a70000-0x0000000031b5ffff]
[    0.000000]   node   0: [mem 0x0000000031b60000-0x0000000031b6ffff]
[    0.000000]   node   0: [mem 0x0000000031b70000-0x0000000031b7dfff]
[    0.000000]   node   0: [mem 0x0000000031b7e000-0x0000000031b81fff]
[    0.000000]   node   0: [mem 0x0000000031b82000-0x0000000039c0ffff]
[    0.000000]   node   0: [mem 0x0000000039c10000-0x000000003a11ffff]
[    0.000000]   node   0: [mem 0x000000003a120000-0x000000003f10ffff]
[    0.000000]   node   0: [mem 0x000000003f110000-0x000000003f13ffff]
[    0.000000]   node   0: [mem 0x000000003f140000-0x000000003fbfffff]
[    0.000000]   node   0: [mem 0x0000002040000000-0x00000027ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000027ffffffff]
[    0.000000] On node 0 totalpages: 8387583
[    0.000000]   DMA zone: 4080 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 261119 pages, LIFO batch:31
[    0.000000]   Normal zone: 126976 pages used for memmap
[    0.000000]   Normal zone: 8126464 pages, LIFO batch:31
[    0.000000] psci: probing for conduit method from ACPI.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] percpu: Embedded 21 pages/cpu @ffff8027ffe6c000 s48000 r8192 d29824 u86016
[    0.000000] pcpu-alloc: s48000 r8192 d29824 u86016 alloc=21*4096
[    0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07
[    0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] CPU features: enabling workaround for ARM erratum 834220
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 8256527
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: earlycon=hisilpcuart,mmio,0xa01b0000,0,0x2f8 console=ttyS0,115200 initrd=filesystem.cpio.gz acpi=force pcie_aspm=off pci=pcie_bus_perf
[    0.000000] PCIe ASPM is disabled
[    0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 61440 bytes
[    0.000000] log_buf_len min size: 16384 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 10700(65%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] software IO TLB [mem 0x35c10000-0x39c10000] (64MB) mapped at [ffff800035c10000-ffff800039c0ffff]
[    0.000000] Memory: 32894040K/33550332K available (8380K kernel code, 864K rwdata, 3680K rodata, 1024K init, 286K bss, 639908K reserved, 16384K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffff000000000000 - 0xffff000008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000   (129022 GB)
[    0.000000]       .text : 0xffff000008080000 - 0xffff0000088b0000   (  8384 KB)
[    0.000000]     .rodata : 0xffff0000088b0000 - 0xffff000008c50000   (  3712 KB)
[    0.000000]       .init : 0xffff000008c50000 - 0xffff000008d50000   (  1024 KB)
[    0.000000]       .data : 0xffff000008d50000 - 0xffff000008e28200   (   865 KB)
[    0.000000]        .bss : 0xffff000008e28200 - 0xffff000008e6fdc0   (   287 KB)
[    0.000000]     fixed   : 0xffff7dfffe7fd000 - 0xffff7dfffec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffff7e0000000000 - 0xffff800000000000   (  2048 GB maximum)
[    0.000000]               0xffff7e0000000000 - 0xffff7e00a0000000   (  2560 MB actual)
[    0.000000]     memory  : 0xffff800000000000 - 0xffff802800000000   (163840 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=16.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=16
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] ITS [mem 0xc6000000-0xc601ffff]
[    0.000000] ITS@0x00000000c6000000: allocated 524288 Devices @27df400000 (flat, esz 8, psz 16K, shr 1)
[    0.000000] ITS@0x00000000c6000000: allocated 2048 Virtual CPUs @27df814000 (flat, esz 8, psz 4K, shr 1)
[    0.000000] ITS@0x00000000c6000000: allocated 512 Interrupt Collections @27df811000 (flat, esz 8, psz 4K, shr 1)
[    0.000000] GIC: using LPI property table @0x00000027df830000
[    0.000000] ITS: Allocated 1792 chunks for LPIs
[    0.000000] GICv3: CPU0: found redistributor 10000 region 0:0x000000004d100000
[    0.000000] CPU0: using LPI pending table @0x00000027df840000
[    0.000000] arch_timer: cp15 timer(s) running at 50.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[    0.000002] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
[    0.009100] Console: colour dummy device 80x25
[    0.014154] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000)
[    0.025976] pid_max: default: 32768 minimum: 301
[    0.031197] ACPI: Core revision 20160831
[    0.036669] ACPI: 1 ACPI AML tables successfully acquired and loaded

[    0.045562] Security Framework initialized
[    0.054566] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
[    0.075798] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[    0.089463] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.097298] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.106386] ASID allocator initialised with 65536 entries
[    0.144537] PCI/MSI: ITS@0xc6000000 domain created
[    0.149957] Platform MSI: ITS@0xc6000000 domain created
[    0.155909] Remapping and enabling EFI services.
[    0.161174]   EFI remap 0x0000000031a00000 => 0000000020000000
[    0.167801]   EFI remap 0x0000000031b60000 => 0000000020070000
[    0.174434]   EFI remap 0x0000000039c10000 => 0000000020080000
[    0.181083]   EFI remap 0x0000000039c50000 => 00000000200c0000
[    0.187715]   EFI remap 0x0000000039cd0000 => 00000000200d0000
[    0.194361]   EFI remap 0x0000000039d50000 => 0000000020150000
[    0.200988]   EFI remap 0x0000000039dc0000 => 00000000201a0000
[    0.207641]   EFI remap 0x0000000039e10000 => 00000000201f0000
[    0.214262]   EFI remap 0x0000000039f20000 => 0000000020300000
[    0.220894]   EFI remap 0x0000000039f90000 => 0000000020370000
[    0.227539]   EFI remap 0x0000000039fe0000 => 00000000203c0000
[    0.234165]   EFI remap 0x000000003a030000 => 0000000020410000
[    0.240800]   EFI remap 0x000000003a080000 => 0000000020460000
[    0.247436]   EFI remap 0x000000003a0d0000 => 00000000204b0000
[    0.254075]   EFI remap 0x000000003f110000 => 0000000020500000
[    0.260717]   EFI remap 0x0000000078000000 => 0000000020530000
[    0.267344]   EFI remap 0x00000000a4000000 => 0000000020600000
[    0.273936]   EFI remap 0x00000000a6000000 => 0000000021600000
[    0.280556]   EFI remap 0x00000000d00e0000 => 0000000021610000
[    0.387646] Detected PIPT I-cache on CPU1
[    0.387662] GICv3: CPU1: found redistributor 10001 region 1:0x000000004d130000
[    0.387693] CPU1: using LPI pending table @0x00000027dc300000
[    0.387709] CPU1: Booted secondary processor [411fd071]
[    0.468326] Detected PIPT I-cache on CPU2
[    0.468334] GICv3: CPU2: found redistributor 10002 region 2:0x000000004d160000
[    0.468363] CPU2: using LPI pending table @0x00000027dc330000
[    0.468374] CPU2: Booted secondary processor [411fd071]
[    0.549016] Detected PIPT I-cache on CPU3
[    0.549024] GICv3: CPU3: found redistributor 10003 region 3:0x000000004d190000
[    0.549054] CPU3: using LPI pending table @0x00000027dc370000
[    0.549064] CPU3: Booted secondary processor [411fd071]
[    0.629722] Detected PIPT I-cache on CPU4
[    0.629735] GICv3: CPU4: found redistributor 10100 region 4:0x000000004d1c0000
[    0.629765] CPU4: using LPI pending table @0x00000027dc3a0000
[    0.629779] CPU4: Booted secondary processor [411fd071]
[    0.710402] Detected PIPT I-cache on CPU5
[    0.710410] GICv3: CPU5: found redistributor 10101 region 5:0x000000004d1f0000
[    0.710439] CPU5: using LPI pending table @0x00000027dc3d0000
[    0.710449] CPU5: Booted secondary processor [411fd071]
[    0.791125] Detected PIPT I-cache on CPU6
[    0.791133] GICv3: CPU6: found redistributor 10102 region 6:0x000000004d220000
[    0.791162] CPU6: using LPI pending table @0x00000027dbc20000
[    0.791172] CPU6: Booted secondary processor [411fd071]
[    0.871819] Detected PIPT I-cache on CPU7
[    0.871827] GICv3: CPU7: found redistributor 10103 region 7:0x000000004d250000
[    0.871856] CPU7: using LPI pending table @0x00000027dbc50000
[    0.871866] CPU7: Booted secondary processor [411fd071]
[    0.952607] Detected PIPT I-cache on CPU8
[    0.952620] GICv3: CPU8: found redistributor 10200 region 8:0x000000004d280000
[    0.952652] CPU8: using LPI pending table @0x00000027dbc80000
[    0.952666] CPU8: Booted secondary processor [411fd071]
[    1.033263] Detected PIPT I-cache on CPU9
[    1.033271] GICv3: CPU9: found redistributor 10201 region 9:0x000000004d2b0000
[    1.033301] CPU9: using LPI pending table @0x00000027dbcc0000
[    1.033311] CPU9: Booted secondary processor [411fd071]
[    1.113992] Detected PIPT I-cache on CPU10
[    1.114000] GICv3: CPU10: found redistributor 10202 region 10:0x000000004d2e0000
[    1.114030] CPU10: using LPI pending table @0x00000027dbcf0000
[    1.114040] CPU10: Booted secondary processor [411fd071]
[    1.194677] Detected PIPT I-cache on CPU11
[    1.194685] GICv3: CPU11: found redistributor 10203 region 11:0x000000004d310000
[    1.194716] CPU11: using LPI pending table @0x00000027dbd30000
[    1.194726] CPU11: Booted secondary processor [411fd071]
[    1.275373] Detected PIPT I-cache on CPU12
[    1.275387] GICv3: CPU12: found redistributor 10300 region 12:0x000000004d340000
[    1.275418] CPU12: using LPI pending table @0x00000027dbd60000
[    1.275431] CPU12: Booted secondary processor [411fd071]
[    1.356063] Detected PIPT I-cache on CPU13
[    1.356072] GICv3: CPU13: found redistributor 10301 region 13:0x000000004d370000
[    1.356102] CPU13: using LPI pending table @0x00000027dbdb0000
[    1.356111] CPU13: Booted secondary processor [411fd071]
[    1.436757] Detected PIPT I-cache on CPU14
[    1.436766] GICv3: CPU14: found redistributor 10302 region 14:0x000000004d3a0000
[    1.436796] CPU14: using LPI pending table @0x00000027dbde0000
[    1.436806] CPU14: Booted secondary processor [411fd071]
[    1.517449] Detected PIPT I-cache on CPU15
[    1.517458] GICv3: CPU15: found redistributor 10303 region 15:0x000000004d3d0000
[    1.517489] CPU15: using LPI pending table @0x00000027dbe10000
[    1.517499] CPU15: Booted secondary processor [411fd071]
[    1.517538] Brought up 16 CPUs
[    1.899942] SMP: Total of 16 processors activated.
[    1.905353] CPU features: detected feature: GIC system register CPU interface
[    1.913415] CPU features: detected feature: 32-bit EL0 Support
[    1.920062] CPU: All CPU(s) started at EL2
[    1.924754] alternatives: patching kernel code
[    1.934743] devtmpfs: initialized
[    1.938851] SMBIOS 3.0.0 present.
[    1.942710] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.953985] pinctrl core: initialized pinctrl subsystem
[    1.960338] NET: Registered protocol family 16
[    1.998505] cpuidle: using governor menu
[    2.003034] vdso: 2 pages (1 code @ ffff0000088b7000, 1 data @ ffff000008d54000)
[    2.011473] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    2.019756] DMA: preallocated 256 KiB pool for atomic allocations
[    2.026887] ACPI: bus type PCI registered
[    2.031550] Serial: AMBA PL011 UART driver
[    2.068489] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    2.076311] ACPI: Added _OSI(Module Device)
[    2.081138] ACPI: Added _OSI(Processor Device)
[    2.086243] ACPI: Added _OSI(3.0 _SCP Extensions)
[    2.091662] ACPI: Added _OSI(Processor Aggregator Device)
[    2.099799] ACPI: Interpreter enabled
[    2.104019] ACPI: Using GIC for interrupt routing
[    2.109466] ACPI: MCFG table detected, 3 entries
[    2.132280] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
[    2.139416] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.147563] acpi PNP0A08:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    2.155225] acpi PNP0A08:00: MCFG quirk: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f] with hisi_pcie_ops
[    2.167558] acpi PNP0A08:00: [Firmware Bug]: ECAM area [mem 0xb0000000-0xb1ffffff] not reserved in ACPI namespace
[    2.179629] acpi PNP0A08:00: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f]
[    2.188199] Remapped I/O 0x00000000b7ff0000 to [io  0x0000-0xffff window]
[    2.196013] PCI host bridge to bus 0000:00
[    2.200679] pci_bus 0000:00: root bus resource [mem 0xb2000000-0xb7feffff window]
[    2.209233] pci_bus 0000:00: root bus resource [io  0x0000-0xffff window]
[    2.216972] pci_bus 0000:00: root bus resource [bus 00-1f]
[    2.223225] pci 0000:00:00.0: [19e5:1610] type 01 class 0x060400
[    2.223236] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.223273] pci 0000:00:00.0: supports D1 D2
[    2.223275] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.274630] pci 0000:00:00.0: BAR 0: assigned [mem 0xb2000000-0xb200ffff]
[    2.282374] pci 0000:00:00.0: PCI bridge to [bus 01]
[    2.288051] pci 0000:00:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.297714] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus e0-ff])
[    2.304770] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.312841] acpi PNP0A08:01: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    2.320388] acpi PNP0A08:01: MCFG quirk: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff] with hisi_pcie_ops
[    2.332547] acpi PNP0A08:01: [Firmware Bug]: ECAM area [mem 0xbe000000-0xbfffffff] not reserved in ACPI namespace
[    2.344507] acpi PNP0A08:01: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff]
[    2.352952] Remapped I/O 0x00000000bdff0000 to [io  0x10000-0x1ffff window]
[    2.360942] PCI host bridge to bus 0001:e0
[    2.365614] pci_bus 0001:e0: root bus resource [mem 0xb8000000-0xbdfeffff window]
[    2.374118] pci_bus 0001:e0: root bus resource [io  0x10000-0x1ffff window] (bus address [0x0000-0xffff])
[    2.385041] pci_bus 0001:e0: root bus resource [bus e0-ff]
[    2.391302] pci 0001:e0:00.0: [19e5:1610] type 01 class 0x060400
[    2.391310] pci 0001:e0:00.0: reg 0x10: [mem 0x00000000-0x00000fff]
[    2.391318] pci 0001:e0:00.0: reg 0x38: [mem 0xfffe0000-0xffffffff pref]
[    2.391345] pci 0001:e0:00.0: supports D1 D2
[    2.391347] pci 0001:e0:00.0: PME# supported from D0 D1 D3hot
[    2.391476] pci 0001:e1:00.0: [1000:005d] type 00 class 0x010400
[    2.391484] pci 0001:e1:00.0: reg 0x10: [io  0x10000-0x100ff]
[    2.391492] pci 0001:e1:00.0: reg 0x14: [mem 0xb8100000-0xb810ffff 64bit]
[    2.391499] pci 0001:e1:00.0: reg 0x1c: [mem 0xb8000000-0xb80fffff 64bit]
[    2.391506] pci 0001:e1:00.0: reg 0x30: [mem 0xfff00000-0xffffffff pref]
[    2.391556] pci 0001:e1:00.0: supports D1 D2
[    2.391637] pci 0001:e0:00.0: BAR 14: assigned [mem 0xb8000000-0xb82fffff]
[    2.399463] pci 0001:e0:00.0: BAR 6: assigned [mem 0xb8300000-0xb831ffff pref]
[    2.407702] pci 0001:e0:00.0: BAR 0: assigned [mem 0xb8320000-0xb8320fff]
[    2.415444] pci 0001:e0:00.0: BAR 13: assigned [io  0x10000-0x10fff]
[    2.422698] pci 0001:e1:00.0: BAR 3: assigned [mem 0xb8000000-0xb80fffff 64bit]
[    2.431020] pci 0001:e1:00.0: BAR 6: assigned [mem 0xb8100000-0xb81fffff pref]
[    2.439262] pci 0001:e1:00.0: BAR 1: assigned [mem 0xb8200000-0xb820ffff 64bit]
[    2.447623] pci 0001:e1:00.0: BAR 0: assigned [io  0x10000-0x100ff]
[    2.454741] pci 0001:e0:00.0: PCI bridge to [bus e1]
[    2.460383] pci 0001:e0:00.0:   bridge window [io  0x10000-0x10fff]
[    2.467533] pci 0001:e0:00.0:   bridge window [mem 0xb8000000-0xb82fffff]
[    2.475289] pci 0001:e0:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.484916] pci 0001:e1:00.0: Max Payload Size set to  512/4096 (was  128), Max Read Rq  512
[    2.494568] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-9f])
[    2.501617] acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.509658] acpi PNP0A08:02: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    2.517222] acpi PNP0A08:02: MCFG quirk: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f] with hisi_pcie_ops
[    2.529349] acpi PNP0A08:02: [Firmware Bug]: ECAM area [mem 0xa8000000-0xa9ffffff] not reserved in ACPI namespace
[    2.541330] acpi PNP0A08:02: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f]
[    2.549806] Remapped I/O 0x00000000afff0000 to [io  0x20000-0x2ffff window]
[    2.557801] PCI host bridge to bus 0002:80
[    2.562472] pci_bus 0002:80: root bus resource [mem 0xaa000000-0xaffeffff window]
[    2.570987] pci_bus 0002:80: root bus resource [io  0x20000-0x2ffff window] (bus address [0x0000-0xffff])
[    2.581895] pci_bus 0002:80: root bus resource [bus 80-9f]
[    2.588173] pci 0002:80:00.0: [19e5:1610] type 01 class 0x060400
[    2.588181] pci 0002:80:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.588210] pci 0002:80:00.0: supports D1 D2
[    2.588212] pci 0002:80:00.0: PME# supported from D0 D1 D3hot
[    2.588342] pci 0002:81:00.0: [10b5:8725] type 01 class 0x060400
[    2.588353] pci 0002:81:00.0: reg 0x10: [mem 0xaa200000-0xaa23ffff]
[    2.588425] pci 0002:81:00.0: PME# supported from D0 D3hot D3cold
[    2.588574] pci 0002:82:00.0: [10b5:8725] type 01 class 0x060400
[    2.588655] pci 0002:82:00.0: PME# supported from D0 D3hot D3cold
[    2.588751] pci 0002:82:01.0: [10b5:8725] type 01 class 0x060400
[    2.588832] pci 0002:82:01.0: PME# supported from D0 D3hot D3cold
[    2.588931] pci 0002:82:02.0: [10b5:8725] type 01 class 0x060400
[    2.589012] pci 0002:82:02.0: PME# supported from D0 D3hot D3cold
[    2.589111] pci 0002:82:08.0: [10b5:8725] type 01 class 0x060400
[    2.589192] pci 0002:82:08.0: PME# supported from D0 D3hot D3cold
[    2.589405] pci 0002:84:00.0: [19e5:1711] type 00 class 0x030000
[    2.589423] pci 0002:84:00.0: reg 0x10: [mem 0xac000000-0xadffffff pref]
[    2.589433] pci 0002:84:00.0: reg 0x14: [mem 0xaa000000-0xaa1fffff]
[    2.589533] pci 0002:84:00.0: supports D1
[    2.589535] pci 0002:84:00.0: PME# supported from D0 D1 D3hot
[    2.589762] pci 0002:80:00.0: BAR 14: assigned [mem 0xaa000000-0xae7fffff]
[    2.597573] pci 0002:80:00.0: BAR 15: assigned [mem 0xae800000-0xaebfffff 64bit pref]
[    2.606517] pci 0002:80:00.0: BAR 0: assigned [mem 0xaec00000-0xaec0ffff]
[    2.614273] pci 0002:80:00.0: BAR 13: assigned [io  0x20000-0x21fff]
[    2.621528] pci 0002:81:00.0: BAR 14: assigned [mem 0xaa000000-0xadffffff]
[    2.629351] pci 0002:81:00.0: BAR 15: assigned [mem 0xae800000-0xaebfffff 64bit pref]
[    2.638284] pci 0002:81:00.0: BAR 0: assigned [mem 0xae000000-0xae03ffff]
[    2.646020] pci 0002:81:00.0: BAR 13: assigned [io  0x20000-0x21fff]
[    2.653254] pci 0002:82:01.0: BAR 14: assigned [mem 0xaa000000-0xacffffff]
[    2.661085] pci 0002:82:00.0: BAR 14: assigned [mem 0xad000000-0xad1fffff]
[    2.668915] pci 0002:82:00.0: BAR 15: assigned [mem 0xae800000-0xae9fffff 64bit pref]
[    2.677838] pci 0002:82:01.0: BAR 15: assigned [mem 0xaea00000-0xaebfffff 64bit pref]
[    2.686772] pci 0002:82:00.0: BAR 13: assigned [io  0x20000-0x20fff]
[    2.693990] pci 0002:82:01.0: BAR 13: assigned [io  0x21000-0x21fff]
[    2.701240] pci 0002:82:00.0: PCI bridge to [bus 83]
[    2.706889] pci 0002:82:00.0:   bridge window [io  0x20000-0x20fff]
[    2.714036] pci 0002:82:00.0:   bridge window [mem 0xad000000-0xad1fffff]
[    2.721785] pci 0002:82:00.0:   bridge window [mem 0xae800000-0xae9fffff 64bit pref]
[    2.730622] pci 0002:84:00.0: BAR 0: assigned [mem 0xaa000000-0xabffffff pref]
[    2.738866] pci 0002:84:00.0: BAR 1: assigned [mem 0xac000000-0xac1fffff]
[    2.746601] pci 0002:82:01.0: PCI bridge to [bus 84]
[    2.752263] pci 0002:82:01.0:   bridge window [io  0x21000-0x21fff]
[    2.759402] pci 0002:82:01.0:   bridge window [mem 0xaa000000-0xacffffff]
[    2.767128] pci 0002:82:01.0:   bridge window [mem 0xaea00000-0xaebfffff 64bit pref]
[    2.775953] pci 0002:82:02.0: PCI bridge to [bus 85]
[    2.781617] pci 0002:82:08.0: PCI bridge to [bus 86]
[    2.787270] pci 0002:81:00.0: PCI bridge to [bus 82-86]
[    2.793225] pci 0002:81:00.0:   bridge window [io  0x20000-0x21fff]
[    2.800360] pci 0002:81:00.0:   bridge window [mem 0xaa000000-0xadffffff]
[    2.808112] pci 0002:81:00.0:   bridge window [mem 0xae800000-0xaebfffff 64bit pref]
[    2.816952] pci 0002:80:00.0: PCI bridge to [bus 81-86]
[    2.822902] pci 0002:80:00.0:   bridge window [io  0x20000-0x21fff]
[    2.830045] pci 0002:80:00.0:   bridge window [mem 0xaa000000-0xae7fffff]
[    2.837753] pci 0002:80:00.0:   bridge window [mem 0xae800000-0xaebfffff 64bit pref]
[    2.846608] pci 0002:80:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.856245] pci 0002:81:00.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.865870] pci 0002:82:00.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.875504] pci 0002:82:01.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.885114] pci 0002:84:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  256
[    2.894747] pci 0002:82:02.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.904377] pci 0002:82:08.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.914432] vgaarb: device added: PCI:0002:84:00.0,decodes=io+mem,owns=none,locks=none
[    2.923379] vgaarb: loaded
[    2.926430] vgaarb: bridge control possible 0002:84:00.0
[    2.932577] SCSI subsystem initialized
[    2.936899] libata version 3.00 loaded.
[    2.936975] ACPI: bus type USB registered
[    2.941550] usbcore: registered new interface driver usbfs
[    2.947775] usbcore: registered new interface driver hub
[    2.953827] usbcore: registered new device driver usb
[    2.959651] pps_core: LinuxPPS API ver. 1 registered
[    2.965259] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    2.975584] PTP clock support registered
[    2.980084] Registered efivars operations
[    2.984816] Advanced Linux Sound Architecture Driver Initialized.
[    2.992097] clocksource: Switched to clocksource arch_sys_counter
[    2.999063] VFS: Disk quotas dquot_6.6.0
[    3.003527] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    3.011433] pnp: PnP ACPI init
[    3.015076] system 00:00: [mem 0xa0090000-0xa009ffff] has been reserved
[    3.022595] system 00:00: [mem 0xb0000000-0xb01fffff] could not be reserved
[    3.030505] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.030567] system 00:01: [mem 0xa0200000-0xa020ffff] has been reserved
[    3.038058] system 00:01: [mem 0xbe000000-0xbe1fffff] could not be reserved
[    3.045967] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.046031] system 00:02: [mem 0xa00a0000-0xa00affff] has been reserved
[    3.053529] system 00:02: [mem 0xa8000000-0xa81fffff] could not be reserved
[    3.061424] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.061431] pnp: PnP ACPI: found 3 devices
[    3.068917] NET: Registered protocol family 2
[    3.074383] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
[    3.083534] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    3.091504] TCP: Hash tables configured (established 262144 bind 65536)
[    3.099135] UDP hash table entries: 16384 (order: 7, 524288 bytes)
[    3.106382] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
[    3.114144] NET: Registered protocol family 1
[    3.119275] RPC: Registered named UNIX socket transport module.
[    3.125985] RPC: Registered udp transport module.
[    3.131334] RPC: Registered tcp transport module.
[    3.136676] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    3.144004] pci 0001:e1:00.0: [Firmware Bug]: VPD access disabled
[    3.150929] PCI: CLS 0 bytes, default 128
[    3.151018] Unpacking initramfs...
[    3.537707] Freeing initrd memory: 27492K (ffff80001e520000 - ffff80001fff9000)
[    3.546265] kvm [1]: 8-bit VMID
[    3.549831] kvm [1]: IDMAP page: 89f000
[    3.554178] kvm [1]: HYP VA range: 800000000000:ffffffffffff
[    3.561314] kvm [1]: Hyp mode initialized successfully
[    3.567168] kvm [1]: vgic-v2@fe020000
[    3.571353] kvm [1]: GIC system register CPU interface enabled
[    3.578148] kvm [1]: vgic interrupt IRQ1
[    3.582627] kvm [1]: virtual timer IRQ4
[    3.588653] futex hash table entries: 4096 (order: 7, 524288 bytes)
[    3.595882] audit: initializing netlink subsys (disabled)
[    3.602047] audit: type=2000 audit(2.956:1): initialized
[    3.608314] workingset: timestamp_bits=44 max_order=23 bucket_order=0
[    3.619690] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    3.626680] NFS: Registering the id_resolver key type
[    3.632403] Key type id_resolver registered
[    3.637128] Key type id_legacy registered
[    3.641661] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    3.649298] 9p: Installing v9fs 9p2000 file system support
[    3.656725] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    3.665084] io scheduler noop registered
[    3.669570] io scheduler cfq registered (default)
[    3.675102] libphy: mdio_driver_register: phy-bcm-ns2-pci
[    3.681559] pcieport 0000:00:00.0: can't derive routing for PCI INT A
[    3.688840] pcieport 0000:00:00.0: PCI INT A: no GSI
[    3.694505] pcieport 0001:e0:00.0: can't derive routing for PCI INT A
[    3.701786] pcieport 0001:e0:00.0: PCI INT A: no GSI
[    3.707428] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.714707] pcieport 0002:80:00.0: PCI INT A: no GSI
[    3.720344] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.727624] pcieport 0002:81:00.0: PCI INT A: no GSI
[    3.733413] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.740694] pcieport 0002:82:00.0: PCI INT A: no GSI
[    3.746451] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.753735] pcieport 0002:82:01.0: PCI INT A: no GSI
[    3.759489] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.766772] pcieport 0002:82:02.0: PCI INT A: no GSI
[    3.772524] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.779807] pcieport 0002:82:08.0: PCI INT A: no GSI
[    3.785778] ipmi message handler version 39.2
[    3.790706] ipmi device interface
[    3.794487] IPMI System Interface driver.
[    3.799069] ipmi_si: probing via SMBIOS
[    3.803403] ipmi_si: SMBIOS: io 0xe4 regsize 1 spacing 1 irq 0
[    3.809993] ipmi_si: Adding SMBIOS-specified bt state machine
[    3.816501] ipmi_si: Trying SMBIOS-specified bt state machine at i/o address 0xe4, slave address 0x0, irq 0
[    3.884142] IPMI BT: using default values
[    3.888667] IPMI BT: req2rsp=5 secs retries=2
[    4.100235] ipmi_si ipmi_si.0: Found new BMC (man_id: 0x0007db, prod_id: 0x0001, dev_id: 0x01)
[    4.110025] ipmi_si ipmi_si.0: IPMI bt interface initialized
[    4.117260] xenfs: not registering filesystem on non-xen platform
[    4.125541] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    4.133352] console [ttyS0] disabled
[    4.158766] serial8250.10: ttyS0 at I/O 0x2f8 (irq = 0, base_baud = 115200) is a 16550A
[    4.167898] console [ttyS0] enabled
[    4.175338] bootconsole [hisilpcuart0] disabled
[    4.185185] SuperH (H)SCI(F) driver initialized
[    4.189785] msm_serial: driver initialized
[    4.194110] Failed to find cpu0 device node
[    4.198295] Unable to detect cache hierarchy from DT for CPU 0
[    4.207123] loop: module loaded
[    4.210478] hisi_sas: driver version v1.6
[    4.348649] scsi host0: hisi_sas
[    4.484723] scsi host1: hisi_sas
[    4.620658] scsi host2: hisi_sas
[    4.625341] libphy: Fixed MDIO Bus: probed
[    4.629582] tun: Universal TUN/TAP device driver, 1.6
[    4.634629] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    4.720102] libphy: Hisilicon MII Bus: probed
[    5.600184] netif_napi_add() called with weight 256 on device eth%d
[    5.696720] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    5.702548] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    5.708498] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    5.715445] igb: Copyright (c) 2007-2014 Intel Corporation.
[    5.721036] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[    5.728847] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    5.734778] sky2: driver version 1.30
[    5.738625] VFIO - User Level meta-driver version: 0.3
[    5.744591] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    5.751111] ehci-pci: EHCI PCI platform driver
[    5.755572] ehci-platform: EHCI generic platform driver
[    5.760845] ehci-platform PNP0D20:00: EHCI Host Controller
[    5.766332] ehci-platform PNP0D20:00: new USB bus registered, assigned bus number 1
[    5.774080] ehci-platform PNP0D20:00: irq 6, io mem 0xa7020000
[    5.792104] ehci-platform PNP0D20:00: USB 2.0 started, EHCI 1.00
[    5.798407] hub 1-0:1.0: USB hub found
[    5.802169] hub 1-0:1.0: 2 ports detected
[    5.806398] ehci-exynos: EHCI EXYNOS driver
[    5.810612] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[    5.816197] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    5.822371] ohci-pci: OHCI PCI platform driver
[    5.826836] ohci-platform: OHCI generic platform driver
[    5.832073] ohci-exynos: OHCI EXYNOS driver
[    5.836366] usbcore: registered new interface driver usb-storage
[    5.842629] mousedev: PS/2 mouse device common for all mice
[    5.974335] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[    5.980559] i2c /dev entries driver
[    5.984504] sdhci: Secure Digital Host Controller Interface driver
[    5.990672] sdhci: Copyright(c) Pierre Ossman
[    5.995083] Synopsys Designware Multimedia Card Interface Driver
[    6.001153] sdhci-pltfm: SDHCI platform and OF driver helper
[    6.007096] ledtrig-cpu: registered to indicate activity on CPUs
[    6.013424] usbcore: registered new interface driver usbhid
[    6.018991] usbhid: USB HID core driver
[    6.023298] NET: Registered protocol family 17
[    6.027786] 9pnet: Installing 9P2000 support
[    6.032082] Key type dns_resolver registered
[    6.036587] registered taskstats version 1
[    6.104037] rtc-efi rtc-efi: setting system clock to 2016-12-01 22:00:02 UTC (1480629602)
[    6.112251] ALSA device list:
[    6.115221]   No soundcards found.
[    6.119269] Freeing unused kernel memory: 1024K (ffff800000c50000 - ffff800000d50000)
[    6.160105] usb 1-1: new high-speed USB device number 2 using ehci-platform
[    6.312719] hub 1-1:1.0: USB hub found
[    6.316542] hub 1-1:1.0: 4 ports detected
[    6.440100] usb 1-2: new high-speed USB device number 3 using ehci-platform
[    6.592714] hub 1-2:1.0: USB hub found
[    6.596542] hub 1-2:1.0: 4 ports detected
[    6.888098] usb 1-2.1: new full-speed USB device number 4 using ehci-platform
[    7.004739] input: Keyboard/Mouse KVM 1.1.0 as /devices/platform/PNP0D20:00/usb1/1-2/1-2.1/1-2.1:1.0/0003:12D1:0003.0001/input/input0
[    7.076340] hid-generic 0003:12D1:0003.0001: input: USB HID v1.10 Keyboard [Keyboard/Mouse KVM 1.1.0] on usb-PNP0D20:00-2.1/input0
[    7.089213] input: Keyboard/Mouse KVM 1.1.0 as /devices/platform/PNP0D20:00/usb1/1-2/1-2.1/1-2.1:1.1/0003:12D1:0003.0002/input/input1
[    7.101270] hid-generic 0003:12D1:0003.0002: input: USB HID v1.10 Mouse [Keyboard/Mouse KVM 1.1.0] on usb-PNP0D20:00-2.1/input1

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Bjorn Helgaas]

On Thu, Dec 01, 2016 at 10:04:48PM +0800, Dongdong Liu wrote:
> ... 
> Aftet fixing the compile errors. I tested on HiSilicon D03 board. It works ok.
> The dmesg is as below

Thanks very much for testing this!  A few comments below.

> root@(none)$ dmesg
> [    0.000000] Booting Linux on physical CPU 0x10000
> [    0.000000] Linux version 4.9.0-rc1-gaf05ef2-dirty (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #257 SMP PREEMPT Thu Dec 1 22:13:05 CST 2016
> [    0.000000] Boot CPU: AArch64 Processor [411fd071]
> [    0.000000] earlycon: hisilpcuart0 at MMIO 0x00000000a01b0000 (options '0,0x2f8')
> [    0.000000] bootconsole [hisilpcuart0] enabled
> [    0.000000] efi: Getting EFI parameters from FDT:
> [    0.000000] efi: EFI v2.60 by EDK II
> [    0.000000] efi:  SMBIOS=0x3f110000  SMBIOS 3.0=0x39ce0000  ACPI=0x39db0000  ACPI 2.0=0x39db0014  MEMATTR=0x3c945018

On x86 we normally have info about the machine and BIOS, e.g.,

  DMI: LENOVO 20FCS12V03/20FCS12V03, BIOS N1FET40W (1.14 ) 04/18/2016

I think this comes from dmi_present(), and it looks like we should
call that via arm64_dmi_init().  Any idea why we don't see that info?
Is it just missing from the SMBIOS table?  I think it's useful to
have for debugging problems.

> [    2.109466] ACPI: MCFG table detected, 3 entries
> [    2.132280] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
> [    2.139416] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig Segments MSI]
> [    2.147563] acpi PNP0A08:00: _OSC failed (AE_NOT_FOUND); disabling ASPM

Per the PCI Firmware spec, r3.2, sec 4.5.1, _OSC is required for host
bridges that originate PCIe hierarchies.  So the lack of one here
looks like a firmware defect.

> [    2.155225] acpi PNP0A08:00: MCFG quirk: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f] with hisi_pcie_ops
> [    2.167558] acpi PNP0A08:00: [Firmware Bug]: ECAM area [mem 0xb0000000-0xb1ffffff] not reserved in ACPI namespace

This is what I expected.  Apparently your DSDT doesn't contain a
PNP0C02 device that reserves the ECAM space, so we warn about it.  I
expect that if you look at /proc/iomem, we should still see the
reservation done by pci_ecam_create().

> [    2.179629] acpi PNP0A08:00: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f]

> [    2.297714] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus e0-ff])
> [    2.320388] acpi PNP0A08:01: MCFG quirk: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff] with hisi_pcie_ops
> [    2.332547] acpi PNP0A08:01: [Firmware Bug]: ECAM area [mem 0xbe000000-0xbfffffff] not reserved in ACPI namespace
> [    2.344507] acpi PNP0A08:01: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff]

> [    2.494568] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-9f])
> [    2.517222] acpi PNP0A08:02: MCFG quirk: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f] with hisi_pcie_ops
> [    2.529349] acpi PNP0A08:02: [Firmware Bug]: ECAM area [mem 0xa8000000-0xa9ffffff] not reserved in ACPI namespace
> [    2.541330] acpi PNP0A08:02: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f]

> [    3.015076] system 00:00: [mem 0xa0090000-0xa009ffff] has been reserved
> [    3.022595] system 00:00: [mem 0xb0000000-0xb01fffff] could not be reserved
> [    3.030505] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)

Huh.  This PNP0C02 device reserves *part* of the PNP0A08:00 ECAM
space, but not all of it.  Is this just a typo in the PNP0C02 _CRS,
i.e., somebody wrote a size of "0x001fffff" instead of "0x01ffffff"?

> [    3.030567] system 00:01: [mem 0xa0200000-0xa020ffff] has been reserved
> [    3.038058] system 00:01: [mem 0xbe000000-0xbe1fffff] could not be reserved
> [    3.045967] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)

Same here?  This reserves part of the PNP0A08:01 ECAM space.

> [    3.046031] system 00:02: [mem 0xa00a0000-0xa00affff] has been reserved
> [    3.053529] system 00:02: [mem 0xa8000000-0xa81fffff] could not be reserved
> [    3.061424] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active)

And here for PNP0A08:02?

> [    3.681559] pcieport 0000:00:00.0: can't derive routing for PCI INT A
> [    3.688840] pcieport 0000:00:00.0: PCI INT A: no GSI

Is something wrong with your _PRT?

> [    3.694505] pcieport 0001:e0:00.0: can't derive routing for PCI INT A
> [    3.701786] pcieport 0001:e0:00.0: PCI INT A: no GSI
> [    3.707428] pcieport 0002:80:00.0: can't derive routing for PCI INT A
> [    3.714707] pcieport 0002:80:00.0: PCI INT A: no GSI
> [    3.720344] pcieport 0002:80:00.0: can't derive routing for PCI INT A
> [    3.727624] pcieport 0002:81:00.0: PCI INT A: no GSI
> [    3.733413] pcieport 0002:80:00.0: can't derive routing for PCI INT A
> [    3.740694] pcieport 0002:82:00.0: PCI INT A: no GSI
> [    3.746451] pcieport 0002:80:00.0: can't derive routing for PCI INT A
> [    3.753735] pcieport 0002:82:01.0: PCI INT A: no GSI
> [    3.759489] pcieport 0002:80:00.0: can't derive routing for PCI INT A
> [    3.766772] pcieport 0002:82:02.0: PCI INT A: no GSI
> [    3.772524] pcieport 0002:80:00.0: can't derive routing for PCI INT A
> [    3.779807] pcieport 0002:82:08.0: PCI INT A: no GSI

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Dongdong Liu]

Hi Bjorn

在 2016/12/2 0:31, Bjorn Helgaas 写道:
> On Thu, Dec 01, 2016 at 10:04:48PM +0800, Dongdong Liu wrote:
>> ...
>> Aftet fixing the compile errors. I tested on HiSilicon D03 board. It works ok.
>> The dmesg is as below
>
> Thanks very much for testing this!  A few comments below.
>
>> root@(none)$ dmesg
>> [    0.000000] Booting Linux on physical CPU 0x10000
>> [    0.000000] Linux version 4.9.0-rc1-gaf05ef2-dirty (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #257 SMP PREEMPT Thu Dec 1 22:13:05 CST 2016
>> [    0.000000] Boot CPU: AArch64 Processor [411fd071]
>> [    0.000000] earlycon: hisilpcuart0 at MMIO 0x00000000a01b0000 (options '0,0x2f8')
>> [    0.000000] bootconsole [hisilpcuart0] enabled
>> [    0.000000] efi: Getting EFI parameters from FDT:
>> [    0.000000] efi: EFI v2.60 by EDK II
>> [    0.000000] efi:  SMBIOS=0x3f110000  SMBIOS 3.0=0x39ce0000  ACPI=0x39db0000  ACPI 2.0=0x39db0014  MEMATTR=0x3c945018
>
> On x86 we normally have info about the machine and BIOS, e.g.,
>
>   DMI: LENOVO 20FCS12V03/20FCS12V03, BIOS N1FET40W (1.14 ) 04/18/2016
>
> I think this comes from dmi_present(), and it looks like we should
> call that via arm64_dmi_init().  Any idea why we don't see that info?
> Is it just missing from the SMBIOS table?  I think it's useful to
> have for debugging problems.

I need to ask my colleagues about this, then reply.

>
>> [    2.109466] ACPI: MCFG table detected, 3 entries
>> [    2.132280] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
>> [    2.139416] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig Segments MSI]
>> [    2.147563] acpi PNP0A08:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
>
> Per the PCI Firmware spec, r3.2, sec 4.5.1, _OSC is required for host
> bridges that originate PCIe hierarchies.  So the lack of one here
> looks like a firmware defect.
Thanks for pointing out this. will fix this in UEFI.
>
>> [    2.155225] acpi PNP0A08:00: MCFG quirk: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f] with hisi_pcie_ops
>> [    2.167558] acpi PNP0A08:00: [Firmware Bug]: ECAM area [mem 0xb0000000-0xb1ffffff] not reserved in ACPI namespace
>
> This is what I expected.  Apparently your DSDT doesn't contain a
> PNP0C02 device that reserves the ECAM space, so we warn about it.  I
> expect that if you look at /proc/iomem, we should still see the
> reservation done by pci_ecam_create().
>
>> [    2.179629] acpi PNP0A08:00: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f]
>
>> [    2.297714] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus e0-ff])
>> [    2.320388] acpi PNP0A08:01: MCFG quirk: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff] with hisi_pcie_ops
>> [    2.332547] acpi PNP0A08:01: [Firmware Bug]: ECAM area [mem 0xbe000000-0xbfffffff] not reserved in ACPI namespace
>> [    2.344507] acpi PNP0A08:01: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff]
>
>> [    2.494568] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-9f])
>> [    2.517222] acpi PNP0A08:02: MCFG quirk: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f] with hisi_pcie_ops
>> [    2.529349] acpi PNP0A08:02: [Firmware Bug]: ECAM area [mem 0xa8000000-0xa9ffffff] not reserved in ACPI namespace
>> [    2.541330] acpi PNP0A08:02: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f]
>
>> [    3.015076] system 00:00: [mem 0xa0090000-0xa009ffff] has been reserved
>> [    3.022595] system 00:00: [mem 0xb0000000-0xb01fffff] could not be reserved
>> [    3.030505] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)
>
> Huh.  This PNP0C02 device reserves *part* of the PNP0A08:00 ECAM
> space, but not all of it.  Is this just a typo in the PNP0C02 _CRS,
> i.e., somebody wrote a size of "0x001fffff" instead of "0x01ffffff"?

Good catch, will fix this in UEFI.

>
>> [    3.030567] system 00:01: [mem 0xa0200000-0xa020ffff] has been reserved
>> [    3.038058] system 00:01: [mem 0xbe000000-0xbe1fffff] could not be reserved
>> [    3.045967] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
>
> Same here?  This reserves part of the PNP0A08:01 ECAM space.
will fix this in UEFI.
>
>> [    3.046031] system 00:02: [mem 0xa00a0000-0xa00affff] has been reserved
>> [    3.053529] system 00:02: [mem 0xa8000000-0xa81fffff] could not be reserved
>> [    3.061424] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active)
>
> And here for PNP0A08:02?
will fix this in UEFI.
>
>> [    3.681559] pcieport 0000:00:00.0: can't derive routing for PCI INT A
>> [    3.688840] pcieport 0000:00:00.0: PCI INT A: no GSI
>
> Is something wrong with your _PRT?

Yes, we have not added _PRT in our PCI DSDT Table. Our platform can not use _PRT
to describle the INTX interrupts. we will send a RFC PATCH to discuss this.

>
>> [    3.694505] pcieport 0001:e0:00.0: can't derive routing for PCI INT A
>> [    3.701786] pcieport 0001:e0:00.0: PCI INT A: no GSI
>> [    3.707428] pcieport 0002:80:00.0: can't derive routing for PCI INT A
>> [    3.714707] pcieport 0002:80:00.0: PCI INT A: no GSI
>> [    3.720344] pcieport 0002:80:00.0: can't derive routing for PCI INT A
>> [    3.727624] pcieport 0002:81:00.0: PCI INT A: no GSI
>> [    3.733413] pcieport 0002:80:00.0: can't derive routing for PCI INT A
>> [    3.740694] pcieport 0002:82:00.0: PCI INT A: no GSI
>> [    3.746451] pcieport 0002:80:00.0: can't derive routing for PCI INT A
>> [    3.753735] pcieport 0002:82:01.0: PCI INT A: no GSI
>> [    3.759489] pcieport 0002:80:00.0: can't derive routing for PCI INT A
>> [    3.766772] pcieport 0002:82:02.0: PCI INT A: no GSI
>> [    3.772524] pcieport 0002:80:00.0: can't derive routing for PCI INT A
>> [    3.779807] pcieport 0002:82:08.0: PCI INT A: no GSI
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> .
>

After fix the length of the reserve resource in _CRS and add _OSC in UEFI, the new dmesg and /proc/iomem are as below.

root@(none)$ dmesg
[    0.000000] Booting Linux on physical CPU 0x10000
[    0.000000] Linux version 4.9.0-rc1-gaf05ef2-dirty (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #257 SMP PREEMPT Thu Dec 1 22:13:05 CST 2016
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] earlycon: hisilpcuart0 at MMIO 0x00000000a01b0000 (options '0,0x2f8')
[    0.000000] bootconsole [hisilpcuart0] enabled
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: EFI v2.60 by EDK II
[    0.000000] efi:  SMBIOS=0x3f110000  SMBIOS 3.0=0x39ce0000  ACPI=0x39db0000  ACPI 2.0=0x39db0014  MEMATTR=0x3ca44018
[    0.000000] cma: Reserved 16 MiB at 0x000000003e000000
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x0000000039DB0014 000024 (v02 HISI  )
[    0.000000] ACPI: XSDT 0x0000000039DA00E8 00004C (v01 HISI   HIP06    00000000      01000013)
[    0.000000] ACPI: FACP 0x0000000039CA0000 00010C (v05 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: DSDT 0x0000000039C60000 003B02 (v01 HISI   HIP06    00000000 INTL 20150619)
[    0.000000] ACPI: MCFG 0x0000000039CC0000 00005C (v01 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: GTDT 0x0000000039C90000 000098 (v02 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: APIC 0x0000000039C80000 000518 (v01 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: IORT 0x0000000039C70000 0002E8 (v00 HISI   HIP06    00000000 INTL 20150619)
[    0.000000] ACPI: NUMA: Failed to initialise from firmware
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000027ffffffff]
[    0.000000] NUMA: Adding memblock [0x0 - 0x257ff] on node 0
[    0.000000] NUMA: Adding memblock [0x25800 - 0x31a0ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31a10000 - 0x31a7ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31a80000 - 0x31b5ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b60000 - 0x31b6ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b70000 - 0x31b7dfff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b7e000 - 0x31b81fff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b82000 - 0x39c0ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x39c10000 - 0x3a11ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3a120000 - 0x3f10ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3f110000 - 0x3f13ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3f140000 - 0x3fbfffff] on node 0
[    0.000000] NUMA: Adding memblock [0x2040000000 - 0x27ffffffff] on node 0
[    0.000000] NUMA: Initmem setup node 0 [mem 0x00000000-0x27ffffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x27fffef500-0x27ffff0fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000027ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000000024fff]
[    0.000000]   node   0: [mem 0x0000000000026000-0x0000000031a0ffff]
[    0.000000]   node   0: [mem 0x0000000031a10000-0x0000000031a7ffff]
[    0.000000]   node   0: [mem 0x0000000031a80000-0x0000000031b5ffff]
[    0.000000]   node   0: [mem 0x0000000031b60000-0x0000000031b6ffff]
[    0.000000]   node   0: [mem 0x0000000031b70000-0x0000000031b7dfff]
[    0.000000]   node   0: [mem 0x0000000031b7e000-0x0000000031b81fff]
[    0.000000]   node   0: [mem 0x0000000031b82000-0x0000000039c0ffff]
[    0.000000]   node   0: [mem 0x0000000039c10000-0x000000003a11ffff]
[    0.000000]   node   0: [mem 0x000000003a120000-0x000000003f10ffff]
[    0.000000]   node   0: [mem 0x000000003f110000-0x000000003f13ffff]
[    0.000000]   node   0: [mem 0x000000003f140000-0x000000003fbfffff]
[    0.000000]   node   0: [mem 0x0000002040000000-0x00000027ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000027ffffffff]
[    0.000000] On node 0 totalpages: 8387583
[    0.000000]   DMA zone: 4080 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 261119 pages, LIFO batch:31
[    0.000000]   Normal zone: 126976 pages used for memmap
[    0.000000]   Normal zone: 8126464 pages, LIFO batch:31
[    0.000000] psci: probing for conduit method from ACPI.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] percpu: Embedded 21 pages/cpu @ffff8027ffe6c000 s48000 r8192 d29824 u86016
[    0.000000] pcpu-alloc: s48000 r8192 d29824 u86016 alloc=21*4096
[    0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07
[    0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] CPU features: enabling workaround for ARM erratum 834220
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 8256527
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: earlycon=hisilpcuart,mmio,0xa01b0000,0,0x2f8 console=ttyS0,115200 initrd=filesystem.cpio.gz acpi=force pcie_aspm=off pci=pcie_bus_perf
[    0.000000] PCIe ASPM is disabled
[    0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 61440 bytes
[    0.000000] log_buf_len min size: 16384 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 10700(65%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] software IO TLB [mem 0x35c10000-0x39c10000] (64MB) mapped at [ffff800035c10000-ffff800039c0ffff]
[    0.000000] Memory: 32894040K/33550332K available (8380K kernel code, 864K rwdata, 3680K rodata, 1024K init, 286K bss, 639908K reserved, 16384K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffff000000000000 - 0xffff000008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000   (129022 GB)
[    0.000000]       .text : 0xffff000008080000 - 0xffff0000088b0000   (  8384 KB)
[    0.000000]     .rodata : 0xffff0000088b0000 - 0xffff000008c50000   (  3712 KB)
[    0.000000]       .init : 0xffff000008c50000 - 0xffff000008d50000   (  1024 KB)
[    0.000000]       .data : 0xffff000008d50000 - 0xffff000008e28200   (   865 KB)
[    0.000000]        .bss : 0xffff000008e28200 - 0xffff000008e6fdc0   (   287 KB)
[    0.000000]     fixed   : 0xffff7dfffe7fd000 - 0xffff7dfffec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffff7e0000000000 - 0xffff800000000000   (  2048 GB maximum)
[    0.000000]               0xffff7e0000000000 - 0xffff7e00a0000000   (  2560 MB actual)
[    0.000000]     memory  : 0xffff800000000000 - 0xffff802800000000   (163840 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=16.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=16
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] ITS [mem 0xc6000000-0xc601ffff]
[    0.000000] ITS@0x00000000c6000000: allocated 524288 Devices @27df400000 (flat, esz 8, psz 16K, shr 1)
[    0.000000] ITS@0x00000000c6000000: allocated 2048 Virtual CPUs @27df814000 (flat, esz 8, psz 4K, shr 1)
[    0.000000] ITS@0x00000000c6000000: allocated 512 Interrupt Collections @27df811000 (flat, esz 8, psz 4K, shr 1)
[    0.000000] GIC: using LPI property table @0x00000027df830000
[    0.000000] ITS: Allocated 1792 chunks for LPIs
[    0.000000] GICv3: CPU0: found redistributor 10000 region 0:0x000000004d100000
[    0.000000] CPU0: using LPI pending table @0x00000027df840000
[    0.000000] arch_timer: cp15 timer(s) running at 50.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[    0.000002] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
[    0.009146] Console: colour dummy device 80x25
[    0.014187] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000)
[    0.025977] pid_max: default: 32768 minimum: 301
[    0.031217] ACPI: Core revision 20160831
[    0.036750] ACPI: 1 ACPI AML tables successfully acquired and loaded

[    0.045688] Security Framework initialized
[    0.054722] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
[    0.075995] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[    0.089710] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.097589] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.106741] ASID allocator initialised with 65536 entries
[    0.144929] PCI/MSI: ITS@0xc6000000 domain created
[    0.150379] Platform MSI: ITS@0xc6000000 domain created
[    0.156364] Remapping and enabling EFI services.
[    0.161601]   EFI remap 0x0000000031a10000 => 0000000020000000
[    0.168193]   EFI remap 0x0000000031b60000 => 0000000020070000
[    0.174784]   EFI remap 0x0000000039c10000 => 0000000020080000
[    0.181378]   EFI remap 0x0000000039c50000 => 00000000200c0000
[    0.187969]   EFI remap 0x0000000039cd0000 => 00000000200d0000
[    0.194562]   EFI remap 0x0000000039d50000 => 0000000020150000
[    0.201154]   EFI remap 0x0000000039dc0000 => 00000000201a0000
[    0.207749]   EFI remap 0x0000000039e10000 => 00000000201f0000
[    0.214340]   EFI remap 0x0000000039f20000 => 0000000020300000
[    0.220934]   EFI remap 0x0000000039f90000 => 0000000020370000
[    0.227530]   EFI remap 0x0000000039fe0000 => 00000000203c0000
[    0.234121]   EFI remap 0x000000003a030000 => 0000000020410000
[    0.240713]   EFI remap 0x000000003a080000 => 0000000020460000
[    0.247306]   EFI remap 0x000000003a0d0000 => 00000000204b0000
[    0.253897]   EFI remap 0x000000003f110000 => 0000000020500000
[    0.260490]   EFI remap 0x0000000078000000 => 0000000020530000
[    0.267082]   EFI remap 0x00000000a4000000 => 0000000020600000
[    0.273707]   EFI remap 0x00000000a6000000 => 0000000021600000
[    0.280299]   EFI remap 0x00000000d00e0000 => 0000000021610000
[    0.387360] Detected PIPT I-cache on CPU1
[    0.387375] GICv3: CPU1: found redistributor 10001 region 1:0x000000004d130000
[    0.387405] CPU1: using LPI pending table @0x00000027dc300000
[    0.387420] CPU1: Booted secondary processor [411fd071]
[    0.468061] Detected PIPT I-cache on CPU2
[    0.468069] GICv3: CPU2: found redistributor 10002 region 2:0x000000004d160000
[    0.468098] CPU2: using LPI pending table @0x00000027dc330000
[    0.468109] CPU2: Booted secondary processor [411fd071]
[    0.548781] Detected PIPT I-cache on CPU3
[    0.548789] GICv3: CPU3: found redistributor 10003 region 3:0x000000004d190000
[    0.548819] CPU3: using LPI pending table @0x00000027dc370000
[    0.548830] CPU3: Booted secondary processor [411fd071]
[    0.629484] Detected PIPT I-cache on CPU4
[    0.629496] GICv3: CPU4: found redistributor 10100 region 4:0x000000004d1c0000
[    0.629527] CPU4: using LPI pending table @0x00000027dc3a0000
[    0.629540] CPU4: Booted secondary processor [411fd071]
[    0.710225] Detected PIPT I-cache on CPU5
[    0.710233] GICv3: CPU5: found redistributor 10101 region 5:0x000000004d1f0000
[    0.710262] CPU5: using LPI pending table @0x00000027dc3d0000
[    0.710272] CPU5: Booted secondary processor [411fd071]
[    0.790947] Detected PIPT I-cache on CPU6
[    0.790954] GICv3: CPU6: found redistributor 10102 region 6:0x000000004d220000
[    0.790984] CPU6: using LPI pending table @0x00000027dbc20000
[    0.790995] CPU6: Booted secondary processor [411fd071]
[    0.871682] Detected PIPT I-cache on CPU7
[    0.871690] GICv3: CPU7: found redistributor 10103 region 7:0x000000004d250000
[    0.871719] CPU7: using LPI pending table @0x00000027dbc50000
[    0.871729] CPU7: Booted secondary processor [411fd071]
[    0.952477] Detected PIPT I-cache on CPU8
[    0.952490] GICv3: CPU8: found redistributor 10200 region 8:0x000000004d280000
[    0.952521] CPU8: using LPI pending table @0x00000027dbc80000
[    0.952535] CPU8: Booted secondary processor [411fd071]
[    1.033196] Detected PIPT I-cache on CPU9
[    1.033204] GICv3: CPU9: found redistributor 10201 region 9:0x000000004d2b0000
[    1.033234] CPU9: using LPI pending table @0x00000027dbcc0000
[    1.033244] CPU9: Booted secondary processor [411fd071]
[    1.113941] Detected PIPT I-cache on CPU10
[    1.113949] GICv3: CPU10: found redistributor 10202 region 10:0x000000004d2e0000
[    1.113979] CPU10: using LPI pending table @0x00000027dbcf0000
[    1.113989] CPU10: Booted secondary processor [411fd071]
[    1.194654] Detected PIPT I-cache on CPU11
[    1.194663] GICv3: CPU11: found redistributor 10203 region 11:0x000000004d310000
[    1.194692] CPU11: using LPI pending table @0x00000027dbd30000
[    1.194702] CPU11: Booted secondary processor [411fd071]
[    1.275406] Detected PIPT I-cache on CPU12
[    1.275420] GICv3: CPU12: found redistributor 10300 region 12:0x000000004d340000
[    1.275451] CPU12: using LPI pending table @0x00000027dbd60000
[    1.275464] CPU12: Booted secondary processor [411fd071]
[    1.356145] Detected PIPT I-cache on CPU13
[    1.356154] GICv3: CPU13: found redistributor 10301 region 13:0x000000004d370000
[    1.356184] CPU13: using LPI pending table @0x00000027dbdb0000
[    1.356193] CPU13: Booted secondary processor [411fd071]
[    1.436861] Detected PIPT I-cache on CPU14
[    1.436870] GICv3: CPU14: found redistributor 10302 region 14:0x000000004d3a0000
[    1.436899] CPU14: using LPI pending table @0x00000027dbde0000
[    1.436910] CPU14: Booted secondary processor [411fd071]
[    1.517556] Detected PIPT I-cache on CPU15
[    1.517565] GICv3: CPU15: found redistributor 10303 region 15:0x000000004d3d0000
[    1.517595] CPU15: using LPI pending table @0x00000027dbe10000
[    1.517605] CPU15: Booted secondary processor [411fd071]
[    1.517645] Brought up 16 CPUs
[    1.902431] SMP: Total of 16 processors activated.
[    1.907882] CPU features: detected feature: GIC system register CPU interface
[    1.915997] CPU features: detected feature: 32-bit EL0 Support
[    1.922681] CPU: All CPU(s) started at EL2
[    1.927341] alternatives: patching kernel code
[    1.937312] devtmpfs: initialized
[    1.941430] SMBIOS 3.0.0 present.
[    1.945313] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.956636] pinctrl core: initialized pinctrl subsystem
[    1.963030] NET: Registered protocol family 16
[    1.989072] cpuidle: using governor menu
[    1.993660] vdso: 2 pages (1 code @ ffff0000088b7000, 1 data @ ffff000008d54000)
[    2.002215] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    2.010555] DMA: preallocated 256 KiB pool for atomic allocations
[    2.017674] ACPI: bus type PCI registered
[    2.022352] Serial: AMBA PL011 UART driver
[    2.055313] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    2.063178] ACPI: Added _OSI(Module Device)
[    2.067949] ACPI: Added _OSI(Processor Device)
[    2.072994] ACPI: Added _OSI(3.0 _SCP Extensions)
[    2.078330] ACPI: Added _OSI(Processor Aggregator Device)
[    2.086411] ACPI: Interpreter enabled
[    2.090590] ACPI: Using GIC for interrupt routing
[    2.095976] ACPI: MCFG table detected, 3 entries
[    2.118730] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
[    2.125783] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.133904] acpi PNP0A08:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    2.145015] acpi PNP0A08:00: MCFG quirk: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f] with hisi_pcie_ops
[    2.157151] acpi PNP0A08:00: ECAM area [mem 0xb0000000-0xb1ffffff] reserved by HISI0081:00
[    2.166792] acpi PNP0A08:00: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f]
[    2.175253] Remapped I/O 0x00000000b7ff0000 to [io  0x0000-0xffff window]
[    2.183158] PCI host bridge to bus 0000:00
[    2.187889] pci_bus 0000:00: root bus resource [mem 0xb2000000-0xb7feffff window]
[    2.196523] pci_bus 0000:00: root bus resource [io  0x0000-0xffff window]
[    2.204362] pci_bus 0000:00: root bus resource [bus 00-1f]
[    2.210690] pci 0000:00:00.0: [19e5:1610] type 01 class 0x060400
[    2.210700] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.210738] pci 0000:00:00.0: supports D1 D2
[    2.210741] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.262093] pci 0000:00:00.0: BAR 0: assigned [mem 0xb2000000-0xb200ffff]
[    2.269936] pci 0000:00:00.0: PCI bridge to [bus 01]
[    2.275655] pci 0000:00:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.285430] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus e0-ff])
[    2.292570] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.300775] acpi PNP0A08:01: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    2.312037] acpi PNP0A08:01: MCFG quirk: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff] with hisi_pcie_ops
[    2.324304] acpi PNP0A08:01: ECAM area [mem 0xbe000000-0xbfffffff] reserved by HISI0081:01
[    2.334076] acpi PNP0A08:01: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff]
[    2.342631] Remapped I/O 0x00000000bdff0000 to [io  0x10000-0x1ffff window]
[    2.350737] PCI host bridge to bus 0001:e0
[    2.355457] pci_bus 0001:e0: root bus resource [mem 0xb8000000-0xbdfeffff window]
[    2.364093] pci_bus 0001:e0: root bus resource [io  0x10000-0x1ffff window] (bus address [0x0000-0xffff])
[    2.375126] pci_bus 0001:e0: root bus resource [bus e0-ff]
[    2.381456] pci 0001:e0:00.0: [19e5:1610] type 01 class 0x060400
[    2.381464] pci 0001:e0:00.0: reg 0x10: [mem 0x00000000-0x00000fff]
[    2.381471] pci 0001:e0:00.0: reg 0x38: [mem 0xfffe0000-0xffffffff pref]
[    2.381498] pci 0001:e0:00.0: supports D1 D2
[    2.381500] pci 0001:e0:00.0: PME# supported from D0 D1 D3hot
[    2.381627] pci 0001:e1:00.0: [1000:005d] type 00 class 0x010400
[    2.381636] pci 0001:e1:00.0: reg 0x10: [io  0x10000-0x100ff]
[    2.381643] pci 0001:e1:00.0: reg 0x14: [mem 0xb8100000-0xb810ffff 64bit]
[    2.381650] pci 0001:e1:00.0: reg 0x1c: [mem 0xb8000000-0xb80fffff 64bit]
[    2.381658] pci 0001:e1:00.0: reg 0x30: [mem 0xfff00000-0xffffffff pref]
[    2.381707] pci 0001:e1:00.0: supports D1 D2
[    2.381787] pci 0001:e0:00.0: BAR 14: assigned [mem 0xb8000000-0xb82fffff]
[    2.389710] pci 0001:e0:00.0: BAR 6: assigned [mem 0xb8300000-0xb831ffff pref]
[    2.398049] pci 0001:e0:00.0: BAR 0: assigned [mem 0xb8320000-0xb8320fff]
[    2.405886] pci 0001:e0:00.0: BAR 13: assigned [io  0x10000-0x10fff]
[    2.413222] pci 0001:e1:00.0: BAR 3: assigned [mem 0xb8000000-0xb80fffff 64bit]
[    2.421646] pci 0001:e1:00.0: BAR 6: assigned [mem 0xb8100000-0xb81fffff pref]
[    2.429974] pci 0001:e1:00.0: BAR 1: assigned [mem 0xb8200000-0xb820ffff 64bit]
[    2.438421] pci 0001:e1:00.0: BAR 0: assigned [io  0x10000-0x100ff]
[    2.445651] pci 0001:e0:00.0: PCI bridge to [bus e1]
[    2.451370] pci 0001:e0:00.0:   bridge window [io  0x10000-0x10fff]
[    2.458610] pci 0001:e0:00.0:   bridge window [mem 0xb8000000-0xb82fffff]
[    2.466436] pci 0001:e0:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.476176] pci 0001:e1:00.0: Max Payload Size set to  512/4096 (was  128), Max Read Rq  512
[    2.485944] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-9f])
[    2.493080] acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.501290] acpi PNP0A08:02: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    2.512537] acpi PNP0A08:02: MCFG quirk: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f] with hisi_pcie_ops
[    2.524803] acpi PNP0A08:02: ECAM area [mem 0xa8000000-0xa9ffffff] reserved by HISI0081:02
[    2.534585] acpi PNP0A08:02: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f]
[    2.543138] Remapped I/O 0x00000000afff0000 to [io  0x20000-0x2ffff window]
[    2.551225] PCI host bridge to bus 0002:80
[    2.555948] pci_bus 0002:80: root bus resource [mem 0xaa000000-0xaffeffff window]
[    2.564593] pci_bus 0002:80: root bus resource [io  0x20000-0x2ffff window] (bus address [0x0000-0xffff])
[    2.575642] pci_bus 0002:80: root bus resource [bus 80-9f]
[    2.581986] pci 0002:80:00.0: [19e5:1610] type 01 class 0x060400
[    2.581993] pci 0002:80:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.582023] pci 0002:80:00.0: supports D1 D2
[    2.582025] pci 0002:80:00.0: PME# supported from D0 D1 D3hot
[    2.582152] pci 0002:81:00.0: [10b5:8725] type 01 class 0x060400
[    2.582162] pci 0002:81:00.0: reg 0x10: [mem 0xaa200000-0xaa23ffff]
[    2.582234] pci 0002:81:00.0: PME# supported from D0 D3hot D3cold
[    2.582381] pci 0002:82:00.0: [10b5:8725] type 01 class 0x060400
[    2.582462] pci 0002:82:00.0: PME# supported from D0 D3hot D3cold
[    2.582560] pci 0002:82:01.0: [10b5:8725] type 01 class 0x060400
[    2.582640] pci 0002:82:01.0: PME# supported from D0 D3hot D3cold
[    2.582735] pci 0002:82:02.0: [10b5:8725] type 01 class 0x060400
[    2.582816] pci 0002:82:02.0: PME# supported from D0 D3hot D3cold
[    2.582914] pci 0002:82:08.0: [10b5:8725] type 01 class 0x060400
[    2.582995] pci 0002:82:08.0: PME# supported from D0 D3hot D3cold
[    2.583207] pci 0002:84:00.0: [19e5:1711] type 00 class 0x030000
[    2.583225] pci 0002:84:00.0: reg 0x10: [mem 0xac000000-0xadffffff pref]
[    2.583235] pci 0002:84:00.0: reg 0x14: [mem 0xaa000000-0xaa1fffff]
[    2.583335] pci 0002:84:00.0: supports D1
[    2.583336] pci 0002:84:00.0: PME# supported from D0 D1 D3hot
[    2.583560] pci 0002:80:00.0: BAR 14: assigned [mem 0xaa000000-0xae7fffff]
[    2.591498] pci 0002:80:00.0: BAR 15: assigned [mem 0xae800000-0xaebfffff 64bit pref]
[    2.600547] pci 0002:80:00.0: BAR 0: assigned [mem 0xaec00000-0xaec0ffff]
[    2.608375] pci 0002:80:00.0: BAR 13: assigned [io  0x20000-0x21fff]
[    2.615720] pci 0002:81:00.0: BAR 14: assigned [mem 0xaa000000-0xadffffff]
[    2.623640] pci 0002:81:00.0: BAR 15: assigned [mem 0xae800000-0xaebfffff 64bit pref]
[    2.632677] pci 0002:81:00.0: BAR 0: assigned [mem 0xae000000-0xae03ffff]
[    2.640507] pci 0002:81:00.0: BAR 13: assigned [io  0x20000-0x21fff]
[    2.647831] pci 0002:82:01.0: BAR 14: assigned [mem 0xaa000000-0xacffffff]
[    2.655751] pci 0002:82:00.0: BAR 14: assigned [mem 0xad000000-0xad1fffff]
[    2.663674] pci 0002:82:00.0: BAR 15: assigned [mem 0xae800000-0xae9fffff 64bit pref]
[    2.672702] pci 0002:82:01.0: BAR 15: assigned [mem 0xaea00000-0xaebfffff 64bit pref]
[    2.681724] pci 0002:82:00.0: BAR 13: assigned [io  0x20000-0x20fff]
[    2.689049] pci 0002:82:01.0: BAR 13: assigned [io  0x21000-0x21fff]
[    2.696372] pci 0002:82:00.0: PCI bridge to [bus 83]
[    2.702087] pci 0002:82:00.0:   bridge window [io  0x20000-0x20fff]
[    2.709295] pci 0002:82:00.0:   bridge window [mem 0xad000000-0xad1fffff]
[    2.717105] pci 0002:82:00.0:   bridge window [mem 0xae800000-0xae9fffff 64bit pref]
[    2.726030] pci 0002:84:00.0: BAR 0: assigned [mem 0xaa000000-0xabffffff pref]
[    2.734349] pci 0002:84:00.0: BAR 1: assigned [mem 0xac000000-0xac1fffff]
[    2.742173] pci 0002:82:01.0: PCI bridge to [bus 84]
[    2.747897] pci 0002:82:01.0:   bridge window [io  0x21000-0x21fff]
[    2.755111] pci 0002:82:01.0:   bridge window [mem 0xaa000000-0xacffffff]
[    2.762937] pci 0002:82:01.0:   bridge window [mem 0xaea00000-0xaebfffff 64bit pref]
[    2.771850] pci 0002:82:02.0: PCI bridge to [bus 85]
[    2.777572] pci 0002:82:08.0: PCI bridge to [bus 86]
[    2.783296] pci 0002:81:00.0: PCI bridge to [bus 82-86]
[    2.789316] pci 0002:81:00.0:   bridge window [io  0x20000-0x21fff]
[    2.796539] pci 0002:81:00.0:   bridge window [mem 0xaa000000-0xadffffff]
[    2.804365] pci 0002:81:00.0:   bridge window [mem 0xae800000-0xaebfffff 64bit pref]
[    2.813297] pci 0002:80:00.0: PCI bridge to [bus 81-86]
[    2.819315] pci 0002:80:00.0:   bridge window [io  0x20000-0x21fff]
[    2.826557] pci 0002:80:00.0:   bridge window [mem 0xaa000000-0xae7fffff]
[    2.834388] pci 0002:80:00.0:   bridge window [mem 0xae800000-0xaebfffff 64bit pref]
[    2.843323] pci 0002:80:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.853060] pci 0002:81:00.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.862794] pci 0002:82:00.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.872514] pci 0002:82:01.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.882246] pci 0002:84:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  256
[    2.891970] pci 0002:82:02.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.901703] pci 0002:82:08.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    2.911870] vgaarb: device added: PCI:0002:84:00.0,decodes=io+mem,owns=none,locks=none
[    2.920840] vgaarb: loaded
[    2.923904] vgaarb: bridge control possible 0002:84:00.0
[    2.930049] SCSI subsystem initialized
[    2.934393] libata version 3.00 loaded.
[    2.934467] ACPI: bus type USB registered
[    2.939057] usbcore: registered new interface driver usbfs
[    2.945280] usbcore: registered new interface driver hub
[    2.951340] usbcore: registered new device driver usb
[    2.957162] pps_core: LinuxPPS API ver. 1 registered
[    2.962785] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    2.973128] PTP clock support registered
[    2.977637] Registered efivars operations
[    2.982364] Advanced Linux Sound Architecture Driver Initialized.
[    2.989635] clocksource: Switched to clocksource arch_sys_counter
[    2.996604] VFS: Disk quotas dquot_6.6.0
[    3.001064] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    3.008968] pnp: PnP ACPI init
[    3.012598] system 00:00: [mem 0xa0090000-0xa009ffff] has been reserved
[    3.020087] system 00:00: [mem 0xb0000000-0xb1ffffff] could not be reserved
[    3.027988] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.028053] system 00:01: [mem 0xa0200000-0xa020ffff] has been reserved
[    3.035544] system 00:01: [mem 0xbe000000-0xbfffffff] could not be reserved
[    3.043426] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.043490] system 00:02: [mem 0xa00a0000-0xa00affff] has been reserved
[    3.050985] system 00:02: [mem 0xa8000000-0xa9ffffff] could not be reserved
[    3.058871] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.058877] pnp: PnP ACPI: found 3 devices
[    3.066430] NET: Registered protocol family 2
[    3.071880] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
[    3.081036] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    3.088972] TCP: Hash tables configured (established 262144 bind 65536)
[    3.096572] UDP hash table entries: 16384 (order: 7, 524288 bytes)
[    3.103786] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
[    3.111527] NET: Registered protocol family 1
[    3.116639] RPC: Registered named UNIX socket transport module.
[    3.123325] RPC: Registered udp transport module.
[    3.128635] RPC: Registered tcp transport module.
[    3.133960] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    3.141254] pci 0001:e1:00.0: [Firmware Bug]: VPD access disabled
[    3.148171] PCI: CLS 0 bytes, default 128
[    3.148266] Unpacking initramfs...
[    3.534560] Freeing initrd memory: 27492K (ffff80001e520000 - ffff80001fff9000)
[    3.543097] kvm [1]: 8-bit VMID
[    3.546649] kvm [1]: IDMAP page: 89f000
[    3.550983] kvm [1]: HYP VA range: 800000000000:ffffffffffff
[    3.558078] kvm [1]: Hyp mode initialized successfully
[    3.563910] kvm [1]: vgic-v2@fe020000
[    3.568074] kvm [1]: GIC system register CPU interface enabled
[    3.574880] kvm [1]: vgic interrupt IRQ1
[    3.579356] kvm [1]: virtual timer IRQ4
[    3.585377] futex hash table entries: 4096 (order: 7, 524288 bytes)
[    3.592592] audit: initializing netlink subsys (disabled)
[    3.598734] audit: type=2000 audit(2.952:1): initialized
[    3.604972] workingset: timestamp_bits=44 max_order=23 bucket_order=0
[    3.616133] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    3.623103] NFS: Registering the id_resolver key type
[    3.628831] Key type id_resolver registered
[    3.633561] Key type id_legacy registered
[    3.638094] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    3.645751] 9p: Installing v9fs 9p2000 file system support
[    3.653177] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    3.661550] io scheduler noop registered
[    3.666053] io scheduler cfq registered (default)
[    3.671602] libphy: mdio_driver_register: phy-bcm-ns2-pci
[    3.678064] pcieport 0000:00:00.0: can't derive routing for PCI INT A
[    3.685351] pcieport 0000:00:00.0: PCI INT A: no GSI
[    3.691018] pcieport 0001:e0:00.0: can't derive routing for PCI INT A
[    3.698318] pcieport 0001:e0:00.0: PCI INT A: no GSI
[    3.703965] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.711246] pcieport 0002:80:00.0: PCI INT A: no GSI
[    3.716886] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.724166] pcieport 0002:81:00.0: PCI INT A: no GSI
[    3.729961] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.737244] pcieport 0002:82:00.0: PCI INT A: no GSI
[    3.743007] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.750290] pcieport 0002:82:01.0: PCI INT A: no GSI
[    3.756053] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.763335] pcieport 0002:82:02.0: PCI INT A: no GSI
[    3.769092] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.776373] pcieport 0002:82:08.0: PCI INT A: no GSI
[    3.782349] ipmi message handler version 39.2
[    3.787286] ipmi device interface
[    3.791064] IPMI System Interface driver.
[    3.795645] ipmi_si: probing via SMBIOS
[    3.799993] ipmi_si: SMBIOS: io 0xe4 regsize 1 spacing 1 irq 0
[    3.806586] ipmi_si: Adding SMBIOS-specified bt state machine
[    3.813101] ipmi_si: Trying SMBIOS-specified bt state machine at i/o address 0xe4, slave address 0x0, irq 0
[    3.881680] IPMI BT: using default values
[    3.886214] IPMI BT: req2rsp=5 secs retries=2
[    4.113770] ipmi_si ipmi_si.0: Found new BMC (man_id: 0x0007db, prod_id: 0x0001, dev_id: 0x01)
[    4.123577] ipmi_si ipmi_si.0: IPMI bt interface initialized
[    4.130827] xenfs: not registering filesystem on non-xen platform
[    4.139096] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    4.146871] console [ttyS0] disabled
[    4.172274] serial8250.10: ttyS0 at I/O 0x2f8 (irq = 0, base_baud = 115200) is a 16550A
[    4.181363] console [ttyS0] enabled
[    4.188789] bootconsole [hisilpcuart0] disabled
[    4.198609] SuperH (H)SCI(F) driver initialized
[    4.203210] msm_serial: driver initialized
[    4.207530] Failed to find cpu0 device node
[    4.211714] Unable to detect cache hierarchy from DT for CPU 0
[    4.220570] loop: module loaded
[    4.223922] hisi_sas: driver version v1.6
[    4.362191] scsi host0: hisi_sas
[    4.498201] scsi host1: hisi_sas
[    4.634214] scsi host2: hisi_sas
[    4.638933] libphy: Fixed MDIO Bus: probed
[    4.643181] tun: Universal TUN/TAP device driver, 1.6
[    4.648232] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    4.733642] libphy: Hisilicon MII Bus: probed
[    5.613723] netif_napi_add() called with weight 256 on device eth%d
[    5.710164] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    5.715994] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    5.721947] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    5.728894] igb: Copyright (c) 2007-2014 Intel Corporation.
[    5.734485] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[    5.742293] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    5.748223] sky2: driver version 1.30
[    5.752082] VFIO - User Level meta-driver version: 0.3
[    5.757982] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    5.764500] ehci-pci: EHCI PCI platform driver
[    5.768992] ehci-platform: EHCI generic platform driver
[    5.774259] ehci-platform PNP0D20:00: EHCI Host Controller
[    5.779752] ehci-platform PNP0D20:00: new USB bus registered, assigned bus number 1
[    5.787498] ehci-platform PNP0D20:00: irq 6, io mem 0xa7020000
[    5.809639] ehci-platform PNP0D20:00: USB 2.0 started, EHCI 1.00
[    5.815920] hub 1-0:1.0: USB hub found
[    5.819682] hub 1-0:1.0: 2 ports detected
[    5.823910] ehci-exynos: EHCI EXYNOS driver
[    5.828121] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[    5.833714] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    5.839900] ohci-pci: OHCI PCI platform driver
[    5.844369] ohci-platform: OHCI generic platform driver
[    5.849607] ohci-exynos: OHCI EXYNOS driver
[    5.853894] usbcore: registered new interface driver usb-storage
[    5.860146] mousedev: PS/2 mouse device common for all mice
[    5.991841] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[    5.998075] i2c /dev entries driver
[    6.002024] sdhci: Secure Digital Host Controller Interface driver
[    6.008192] sdhci: Copyright(c) Pierre Ossman
[    6.012602] Synopsys Designware Multimedia Card Interface Driver
[    6.018673] sdhci-pltfm: SDHCI platform and OF driver helper
[    6.024619] ledtrig-cpu: registered to indicate activity on CPUs
[    6.030911] usbcore: registered new interface driver usbhid
[    6.036479] usbhid: USB HID core driver
[    6.040800] NET: Registered protocol family 17
[    6.045288] 9pnet: Installing 9P2000 support
[    6.049583] Key type dns_resolver registered
[    6.054105] registered taskstats version 1
[    6.121345] rtc-efi rtc-efi: setting system clock to 2016-12-02 11:17:35 UTC (1480677455)
[    6.129557] ALSA device list:
[    6.132530]   No soundcards found.
[    6.136579] Freeing unused kernel memory: 1024K (ffff800000c50000 - ffff800000d50000)
[    6.213643] usb 1-1: new high-speed USB device number 2 using ehci-platform
[    6.366266] hub 1-1:1.0: USB hub found
[    6.370087] hub 1-1:1.0: 4 ports detected
[    6.493638] usb 1-2: new high-speed USB device number 3 using ehci-platform
[    6.646262] hub 1-2:1.0: USB hub found
[    6.650087] hub 1-2:1.0: 4 ports detected
[    6.941637] usb 1-2.1: new full-speed USB device number 4 using ehci-platform
[    7.058410] input: Keyboard/Mouse KVM 1.1.0 as /devices/platform/PNP0D20:00/usb1/1-2/1-2.1/1-2.1:1.0/0003:12D1:0003.0001/input/input0
[    7.129870] hid-generic 0003:12D1:0003.0001: input: USB HID v1.10 Keyboard [Keyboard/Mouse KVM 1.1.0] on usb-PNP0D20:00-2.1/input0
[    7.142758] input: Keyboard/Mouse KVM 1.1.0 as /devices/platform/PNP0D20:00/usb1/1-2/1-2.1/1-2.1:1.1/0003:12D1:0003.0002/input/input1
[    7.154837] hid-generic 0003:12D1:0003.0002: input: USB HID v1.10 Mouse [Keyboard/Mouse KVM 1.1.0] on usb-PNP0D20:00-2.1/input1
root@(none)$ cat /proc/iomem
00000000-00024fff : reserved
00026000-31a0ffff : System RAM
   00080000-00c4ffff : Kernel code
   00d50000-00e75fff : Kernel data
31a10000-31a7ffff : reserved
31a80000-31b5ffff : System RAM
31b60000-31b6ffff : reserved
31b70000-31b7dfff : System RAM
31b7e000-31b81fff : reserved
31b82000-39c0ffff : System RAM
39c10000-3a11ffff : reserved
3a120000-3f10ffff : System RAM
3f110000-3f13ffff : reserved
3f140000-3fbfffff : System RAM
603c0000-603cffff : HISI0141:00
   603c0000-603cffff : HISI0141:00
a0080000-a008ffff : HISI0152:02
   a0080000-a008ffff : HISI0152:01
     a0080000-a008ffff : HISI0152:00
a0090000-a009ffff : pnp 00:00
a00a0000-a00affff : pnp 00:02
a01b0000-a01b0fff : HISI0191:00
   a01b0000-a01b0fff : HISI0191:00
a0200000-a020ffff : pnp 00:01
a2000000-a200ffff : HISI0162:01
   a2000000-a200ffff : HISI0162:01
a3000000-a300ffff : HISI0162:02
   a3000000-a300ffff : HISI0162:02
a7020000-a702ffff : PNP0D20:00
   a7020000-a702ffff : PNP0D20:00
a8000000-a9ffffff : PCI ECAM
aa000000-affeffff : PCI Bus 0002:80
   aa000000-ae7fffff : PCI Bus 0002:81
     aa000000-adffffff : PCI Bus 0002:82
       aa000000-acffffff : PCI Bus 0002:84
         aa000000-abffffff : 0002:84:00.0
         ac000000-ac1fffff : 0002:84:00.0
       ad000000-ad1fffff : PCI Bus 0002:83
     ae000000-ae03ffff : 0002:81:00.0
   ae800000-aebfffff : PCI Bus 0002:81
     ae800000-aebfffff : PCI Bus 0002:82
       ae800000-ae9fffff : PCI Bus 0002:83
       aea00000-aebfffff : PCI Bus 0002:84
   aec00000-aec0ffff : 0002:80:00.0
b0000000-b1ffffff : PCI ECAM
b2000000-b7feffff : PCI Bus 0000:00
   b2000000-b200ffff : 0000:00:00.0
b8000000-bdfeffff : PCI Bus 0001:e0
   b8000000-b82fffff : PCI Bus 0001:e1
     b8000000-b80fffff : 0001:e1:00.0
     b8100000-b81fffff : 0001:e1:00.0
     b8200000-b820ffff : 0001:e1:00.0
   b8300000-b831ffff : 0001:e0:00.0
   b8320000-b8320fff : 0001:e0:00.0
be000000-bfffffff : PCI ECAM
c0080000-c008ffff : HISI0152:05
   c0080000-c008ffff : HISI0152:04
     c0080000-c008ffff : HISI0152:03
c3000000-c300ffff : HISI0162:00
   c3000000-c300ffff : HISI0162:00
c4000000-c40fffff : HISI00D1:00
c5000000-c588ffff : HISI00B2:00
   c5000000-c588ffff : HISI00B2:00
c7000000-c705ffff : HISI00B2:00
   c7000000-c705ffff : HISI00B2:00
2040000000-27ffffffff : System RAM

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Hanjun Guo]

On 2016/12/2 11:46, Dongdong Liu wrote:
> Hi Bjorn
>
> 在 2016/12/2 0:31, Bjorn Helgaas 写道:
>> On Thu, Dec 01, 2016 at 10:04:48PM +0800, Dongdong Liu wrote:
>>> ...
>>> Aftet fixing the compile errors. I tested on HiSilicon D03 board. It works ok.
>>> The dmesg is as below
>>
>> Thanks very much for testing this!  A few comments below.
>>
>>> root@(none)$ dmesg
>>> [    0.000000] Booting Linux on physical CPU 0x10000
>>> [    0.000000] Linux version 4.9.0-rc1-gaf05ef2-dirty (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #257 SMP PREEMPT Thu Dec 1 22:13:05 CST 2016
>>> [    0.000000] Boot CPU: AArch64 Processor [411fd071]
>>> [    0.000000] earlycon: hisilpcuart0 at MMIO 0x00000000a01b0000 (options '0,0x2f8')
>>> [    0.000000] bootconsole [hisilpcuart0] enabled
>>> [    0.000000] efi: Getting EFI parameters from FDT:
>>> [    0.000000] efi: EFI v2.60 by EDK II
>>> [    0.000000] efi:  SMBIOS=0x3f110000  SMBIOS 3.0=0x39ce0000  ACPI=0x39db0000  ACPI 2.0=0x39db0014  MEMATTR=0x3c945018
>>
>> On x86 we normally have info about the machine and BIOS, e.g.,
>>
>>   DMI: LENOVO 20FCS12V03/20FCS12V03, BIOS N1FET40W (1.14 ) 04/18/2016
>>
>> I think this comes from dmi_present(), and it looks like we should
>> call that via arm64_dmi_init().  Any idea why we don't see that info?
>> Is it just missing from the SMBIOS table?  I think it's useful to
>> have for debugging problems.
>
> I need to ask my colleagues about this, then reply.

git grep -w "DMI:"
drivers/firmware/dmi_scan.c:                      printk(KERN_DEBUG "DMI: %s\n", dmi_ids_string);
drivers/firmware/dmi_scan.c:                      pr_debug("DMI: %s\n", dmi_ids_string);

It's a pr_debug information, seems we didn't enable the debug print?

Thanks
Hanjun

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Bjorn Helgaas]

On Fri, Dec 02, 2016 at 04:15:43PM +0800, Hanjun Guo wrote:
> On 2016/12/2 11:46, Dongdong Liu wrote:
> > Hi Bjorn
> >
> > 在 2016/12/2 0:31, Bjorn Helgaas 写道:
> >> On Thu, Dec 01, 2016 at 10:04:48PM +0800, Dongdong Liu wrote:
> >>> ...
> >>> Aftet fixing the compile errors. I tested on HiSilicon D03 board. It works ok.
> >>> The dmesg is as below
> >>
> >> Thanks very much for testing this!  A few comments below.
> >>
> >>> root@(none)$ dmesg
> >>> [    0.000000] Booting Linux on physical CPU 0x10000
> >>> [    0.000000] Linux version 4.9.0-rc1-gaf05ef2-dirty (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #257 SMP PREEMPT Thu Dec 1 22:13:05 CST 2016
> >>> [    0.000000] Boot CPU: AArch64 Processor [411fd071]
> >>> [    0.000000] earlycon: hisilpcuart0 at MMIO 0x00000000a01b0000 (options '0,0x2f8')
> >>> [    0.000000] bootconsole [hisilpcuart0] enabled
> >>> [    0.000000] efi: Getting EFI parameters from FDT:
> >>> [    0.000000] efi: EFI v2.60 by EDK II
> >>> [    0.000000] efi:  SMBIOS=0x3f110000  SMBIOS 3.0=0x39ce0000  ACPI=0x39db0000  ACPI 2.0=0x39db0014  MEMATTR=0x3c945018
> >>
> >> On x86 we normally have info about the machine and BIOS, e.g.,
> >>
> >>   DMI: LENOVO 20FCS12V03/20FCS12V03, BIOS N1FET40W (1.14 ) 04/18/2016
> >>
> >> I think this comes from dmi_present(), and it looks like we should
> >> call that via arm64_dmi_init().  Any idea why we don't see that info?
> >> Is it just missing from the SMBIOS table?  I think it's useful to
> >> have for debugging problems.
> >
> > I need to ask my colleagues about this, then reply.
> 
> git grep -w "DMI:"
> drivers/firmware/dmi_scan.c:                      printk(KERN_DEBUG "DMI: %s\n", dmi_ids_string);
> drivers/firmware/dmi_scan.c:                      pr_debug("DMI: %s\n", dmi_ids_string);
> 
> It's a pr_debug information, seems we didn't enable the debug print?

Yeah, probably.  Looks like somebody else was deceived into thinking that
pr_debug() is equivalent to printk(KERN_DEBUG).

Bjorn

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Bjorn Helgaas]

On Thu, Dec 01, 2016 at 02:29:22AM -0600, Bjorn Helgaas wrote:
> This is what I've collected on my pci/ecam branch so far.  This is
> preliminary and probably doesn't even build since I made several
> changes, but at least it's a strawman.  I called it v10 to keep it
> separate from the previous postings of the various pieces.

I don't want to spam the list with a zillion revisions, but I updated
the branch with the build and acpi_resource_consumer() fixes, thanks
to Dongdong, Tomasz, Rober, and Lorenzo.  Current head is:

http://git.kernel.org/cgit/linux/kernel/git/helgaas/pci.git/commit/?id=f80edf4d6c05

If you test this, please mention the SHA1 you're testing and collect
the dmesg log and /proc/iomem contents.

I'll post the full set again soon, hopefully including the X-Gene
quirks.

Bjorn

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Dongdong Liu]

Hi Bjorn

在 2016/12/2 2:01, Bjorn Helgaas 写道:
> On Thu, Dec 01, 2016 at 02:29:22AM -0600, Bjorn Helgaas wrote:
>> This is what I've collected on my pci/ecam branch so far.  This is
>> preliminary and probably doesn't even build since I made several
>> changes, but at least it's a strawman.  I called it v10 to keep it
>> separate from the previous postings of the various pieces.
>
> I don't want to spam the list with a zillion revisions, but I updated
> the branch with the build and acpi_resource_consumer() fixes, thanks
> to Dongdong, Tomasz, Rober, and Lorenzo.  Current head is:
>
> http://git.kernel.org/cgit/linux/kernel/git/helgaas/pci.git/commit/?id=f80edf4d6c05
>
> If you test this, please mention the SHA1 you're testing and collect
> the dmesg log and /proc/iomem contents.

I tested ok on HiSilicon D03 board except some compile warnings as below.
(commit 36532ac6c5253bcae6624e480b9d2467c2ae7dad)

drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
   THUNDER_PEM_QUIRK(2,  0), /* off-chip devices */
   ^
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[44].cfgres.start')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[44].cfgres.end')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[45].cfgres.start')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[45].cfgres.end')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[46].cfgres.start')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[46].cfgres.end')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[47].cfgres.start')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[47].cfgres.end')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[48].cfgres.start')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[48].cfgres.end')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[49].cfgres.start')
drivers/acpi/pci_mcfg.c:101:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:101:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:101:2: warning: (near initialization for 'mcfg_quirks[49].cfgres.end')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
   THUNDER_PEM_QUIRK(2,  1), /* off-chip devices */
   ^
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[50].cfgres.start')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[50].cfgres.end')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[51].cfgres.start')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[51].cfgres.end')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[52].cfgres.start')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[52].cfgres.end')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[53].cfgres.start')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[53].cfgres.end')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[54].cfgres.start')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[54].cfgres.end')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[55].cfgres.start')
drivers/acpi/pci_mcfg.c:102:2: warning: left shift count >= width of type
drivers/acpi/pci_mcfg.c:102:2: warning: initializer element is not a constant expression
drivers/acpi/pci_mcfg.c:102:2: warning: (near initialization for 'mcfg_quirks[55].cfgres.end')
>
> I'll post the full set again soon, hopefully including the X-Gene
> quirks.
>
> Bjorn
>
> .
>

The dmesg and /proc/iomem information are as below.
root@(none)$ dmesg
[    0.000000] Booting Linux on physical CPU 0x10000
[    0.000000] Linux version 4.9.0-rc1-g1f58b0b-dirty (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #259 SMP PREEMPT Fri Dec 2 19:39:39 CST 2016
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] earlycon: hisilpcuart0 at MMIO 0x00000000a01b0000 (options '0,0x2f8')
[    0.000000] bootconsole [hisilpcuart0] enabled
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: EFI v2.60 by EDK II
[    0.000000] efi:  SMBIOS=0x3f110000  SMBIOS 3.0=0x39ce0000  ACPI=0x39db0000  ACPI 2.0=0x39db0014  MEMATTR=0x3c942018
[    0.000000] cma: Reserved 16 MiB at 0x000000003e000000
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x0000000039DB0014 000024 (v02 HISI  )
[    0.000000] ACPI: XSDT 0x0000000039DA00E8 00004C (v01 HISI   HIP06    00000000      01000013)
[    0.000000] ACPI: FACP 0x0000000039CA0000 00010C (v05 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: DSDT 0x0000000039C60000 003B02 (v01 HISI   HIP06    00000000 INTL 20150619)
[    0.000000] ACPI: MCFG 0x0000000039CC0000 00005C (v01 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: GTDT 0x0000000039C90000 000098 (v02 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: APIC 0x0000000039C80000 000518 (v01 HISI   HIP06    00000000 INTL 20151124)
[    0.000000] ACPI: IORT 0x0000000039C70000 0002E8 (v00 HISI   HIP06    00000000 INTL 20150619)
[    0.000000] ACPI: NUMA: Failed to initialise from firmware
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000027ffffffff]
[    0.000000] NUMA: Adding memblock [0x0 - 0x257ff] on node 0
[    0.000000] NUMA: Adding memblock [0x25800 - 0x319fffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31a00000 - 0x31a6ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31a70000 - 0x31b5ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b60000 - 0x31b6ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b70000 - 0x31b7dfff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b7e000 - 0x31b81fff] on node 0
[    0.000000] NUMA: Adding memblock [0x31b82000 - 0x39c0ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x39c10000 - 0x3a11ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3a120000 - 0x3f10ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3f110000 - 0x3f13ffff] on node 0
[    0.000000] NUMA: Adding memblock [0x3f140000 - 0x3fbfffff] on node 0
[    0.000000] NUMA: Adding memblock [0x2040000000 - 0x27ffffffff] on node 0
[    0.000000] NUMA: Initmem setup node 0 [mem 0x00000000-0x27ffffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x27fffef500-0x27ffff0fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000027ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000000024fff]
[    0.000000]   node   0: [mem 0x0000000000026000-0x00000000319fffff]
[    0.000000]   node   0: [mem 0x0000000031a00000-0x0000000031a6ffff]
[    0.000000]   node   0: [mem 0x0000000031a70000-0x0000000031b5ffff]
[    0.000000]   node   0: [mem 0x0000000031b60000-0x0000000031b6ffff]
[    0.000000]   node   0: [mem 0x0000000031b70000-0x0000000031b7dfff]
[    0.000000]   node   0: [mem 0x0000000031b7e000-0x0000000031b81fff]
[    0.000000]   node   0: [mem 0x0000000031b82000-0x0000000039c0ffff]
[    0.000000]   node   0: [mem 0x0000000039c10000-0x000000003a11ffff]
[    0.000000]   node   0: [mem 0x000000003a120000-0x000000003f10ffff]
[    0.000000]   node   0: [mem 0x000000003f110000-0x000000003f13ffff]
[    0.000000]   node   0: [mem 0x000000003f140000-0x000000003fbfffff]
[    0.000000]   node   0: [mem 0x0000002040000000-0x00000027ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000027ffffffff]
[    0.000000] On node 0 totalpages: 8387583
[    0.000000]   DMA zone: 4080 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 261119 pages, LIFO batch:31
[    0.000000]   Normal zone: 126976 pages used for memmap
[    0.000000]   Normal zone: 8126464 pages, LIFO batch:31
[    0.000000] psci: probing for conduit method from ACPI.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] percpu: Embedded 21 pages/cpu @ffff8027ffe6c000 s48000 r8192 d29824 u86016
[    0.000000] pcpu-alloc: s48000 r8192 d29824 u86016 alloc=21*4096
[    0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07
[    0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] CPU features: enabling workaround for ARM erratum 834220
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 8256527
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: earlycon=hisilpcuart,mmio,0xa01b0000,0,0x2f8 console=ttyS0,115200 initrd=filesystem.cpio.gz acpi=force pcie_aspm=off pci=pcie_bus_perf
[    0.000000] PCIe ASPM is disabled
[    0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 61440 bytes
[    0.000000] log_buf_len min size: 16384 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 10700(65%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] software IO TLB [mem 0x35c10000-0x39c10000] (64MB) mapped at [ffff800035c10000-ffff800039c0ffff]
[    0.000000] Memory: 32894040K/33550332K available (8380K kernel code, 864K rwdata, 3680K rodata, 1024K init, 286K bss, 639908K reserved, 16384K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffff000000000000 - 0xffff000008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000   (129022 GB)
[    0.000000]       .text : 0xffff000008080000 - 0xffff0000088b0000   (  8384 KB)
[    0.000000]     .rodata : 0xffff0000088b0000 - 0xffff000008c50000   (  3712 KB)
[    0.000000]       .init : 0xffff000008c50000 - 0xffff000008d50000   (  1024 KB)
[    0.000000]       .data : 0xffff000008d50000 - 0xffff000008e28200   (   865 KB)
[    0.000000]        .bss : 0xffff000008e28200 - 0xffff000008e6fdc0   (   287 KB)
[    0.000000]     fixed   : 0xffff7dfffe7fd000 - 0xffff7dfffec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffff7e0000000000 - 0xffff800000000000   (  2048 GB maximum)
[    0.000000]               0xffff7e0000000000 - 0xffff7e00a0000000   (  2560 MB actual)
[    0.000000]     memory  : 0xffff800000000000 - 0xffff802800000000   (163840 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=16.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=16
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] ITS [mem 0xc6000000-0xc601ffff]
[    0.000000] ITS@0x00000000c6000000: allocated 524288 Devices @27df400000 (flat, esz 8, psz 16K, shr 1)
[    0.000000] ITS@0x00000000c6000000: allocated 2048 Virtual CPUs @27df814000 (flat, esz 8, psz 4K, shr 1)
[    0.000000] ITS@0x00000000c6000000: allocated 512 Interrupt Collections @27df811000 (flat, esz 8, psz 4K, shr 1)
[    0.000000] GIC: using LPI property table @0x00000027df830000
[    0.000000] ITS: Allocated 1792 chunks for LPIs
[    0.000000] GICv3: CPU0: found redistributor 10000 region 0:0x000000004d100000
[    0.000000] CPU0: using LPI pending table @0x00000027df840000
[    0.000000] arch_timer: cp15 timer(s) running at 50.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[    0.000002] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
[    0.009151] Console: colour dummy device 80x25
[    0.014188] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000)
[    0.025963] pid_max: default: 32768 minimum: 301
[    0.031229] ACPI: Core revision 20160831
[    0.036781] ACPI: 1 ACPI AML tables successfully acquired and loaded

[    0.045753] Security Framework initialized
[    0.054796] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
[    0.076108] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[    0.089846] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.097742] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.106894] ASID allocator initialised with 65536 entries
[    0.145095] PCI/MSI: ITS@0xc6000000 domain created
[    0.150546] Platform MSI: ITS@0xc6000000 domain created
[    0.156550] Remapping and enabling EFI services.
[    0.161792]   EFI remap 0x0000000031a00000 => 0000000020000000
[    0.168390]   EFI remap 0x0000000031b60000 => 0000000020070000
[    0.174988]   EFI remap 0x0000000039c10000 => 0000000020080000
[    0.181585]   EFI remap 0x0000000039c50000 => 00000000200c0000
[    0.188178]   EFI remap 0x0000000039cd0000 => 00000000200d0000
[    0.194776]   EFI remap 0x0000000039d50000 => 0000000020150000
[    0.201373]   EFI remap 0x0000000039dc0000 => 00000000201a0000
[    0.207971]   EFI remap 0x0000000039e10000 => 00000000201f0000
[    0.214564]   EFI remap 0x0000000039f20000 => 0000000020300000
[    0.221162]   EFI remap 0x0000000039f90000 => 0000000020370000
[    0.227762]   EFI remap 0x0000000039fe0000 => 00000000203c0000
[    0.234359]   EFI remap 0x000000003a030000 => 0000000020410000
[    0.240956]   EFI remap 0x000000003a080000 => 0000000020460000
[    0.247550]   EFI remap 0x000000003a0d0000 => 00000000204b0000
[    0.254143]   EFI remap 0x000000003f110000 => 0000000020500000
[    0.260741]   EFI remap 0x0000000078000000 => 0000000020530000
[    0.267339]   EFI remap 0x00000000a4000000 => 0000000020600000
[    0.273975]   EFI remap 0x00000000a6000000 => 0000000021600000
[    0.280568]   EFI remap 0x00000000d00e0000 => 0000000021610000
[    0.387658] Detected PIPT I-cache on CPU1
[    0.387673] GICv3: CPU1: found redistributor 10001 region 1:0x000000004d130000
[    0.387704] CPU1: using LPI pending table @0x00000027dc300000
[    0.387718] CPU1: Booted secondary processor [411fd071]
[    0.468344] Detected PIPT I-cache on CPU2
[    0.468352] GICv3: CPU2: found redistributor 10002 region 2:0x000000004d160000
[    0.468381] CPU2: using LPI pending table @0x00000027dc330000
[    0.468391] CPU2: Booted secondary processor [411fd071]
[    0.549078] Detected PIPT I-cache on CPU3
[    0.549086] GICv3: CPU3: found redistributor 10003 region 3:0x000000004d190000
[    0.549115] CPU3: using LPI pending table @0x00000027dc370000
[    0.549125] CPU3: Booted secondary processor [411fd071]
[    0.629744] Detected PIPT I-cache on CPU4
[    0.629755] GICv3: CPU4: found redistributor 10100 region 4:0x000000004d1c0000
[    0.629786] CPU4: using LPI pending table @0x00000027dc3a0000
[    0.629800] CPU4: Booted secondary processor [411fd071]
[    0.710450] Detected PIPT I-cache on CPU5
[    0.710458] GICv3: CPU5: found redistributor 10101 region 5:0x000000004d1f0000
[    0.710487] CPU5: using LPI pending table @0x00000027dc3d0000
[    0.710497] CPU5: Booted secondary processor [411fd071]
[    0.791173] Detected PIPT I-cache on CPU6
[    0.791181] GICv3: CPU6: found redistributor 10102 region 6:0x000000004d220000
[    0.791211] CPU6: using LPI pending table @0x00000027dbc20000
[    0.791221] CPU6: Booted secondary processor [411fd071]
[    0.871868] Detected PIPT I-cache on CPU7
[    0.871876] GICv3: CPU7: found redistributor 10103 region 7:0x000000004d250000
[    0.871904] CPU7: using LPI pending table @0x00000027dbc50000
[    0.871914] CPU7: Booted secondary processor [411fd071]
[    0.952671] Detected PIPT I-cache on CPU8
[    0.952683] GICv3: CPU8: found redistributor 10200 region 8:0x000000004d280000
[    0.952714] CPU8: using LPI pending table @0x00000027dbc80000
[    0.952728] CPU8: Booted secondary processor [411fd071]
[    1.033372] Detected PIPT I-cache on CPU9
[    1.033380] GICv3: CPU9: found redistributor 10201 region 9:0x000000004d2b0000
[    1.033410] CPU9: using LPI pending table @0x00000027dbcc0000
[    1.033420] CPU9: Booted secondary processor [411fd071]
[    1.114066] Detected PIPT I-cache on CPU10
[    1.114074] GICv3: CPU10: found redistributor 10202 region 10:0x000000004d2e0000
[    1.114103] CPU10: using LPI pending table @0x00000027dbcf0000
[    1.114113] CPU10: Booted secondary processor [411fd071]
[    1.194801] Detected PIPT I-cache on CPU11
[    1.194810] GICv3: CPU11: found redistributor 10203 region 11:0x000000004d310000
[    1.194840] CPU11: using LPI pending table @0x00000027dbd30000
[    1.194850] CPU11: Booted secondary processor [411fd071]
[    1.275527] Detected PIPT I-cache on CPU12
[    1.275540] GICv3: CPU12: found redistributor 10300 region 12:0x000000004d340000
[    1.275571] CPU12: using LPI pending table @0x00000027dbd60000
[    1.275585] CPU12: Booted secondary processor [411fd071]
[    1.356231] Detected PIPT I-cache on CPU13
[    1.356239] GICv3: CPU13: found redistributor 10301 region 13:0x000000004d370000
[    1.356268] CPU13: using LPI pending table @0x00000027dbdb0000
[    1.356278] CPU13: Booted secondary processor [411fd071]
[    1.436937] Detected PIPT I-cache on CPU14
[    1.436946] GICv3: CPU14: found redistributor 10302 region 14:0x000000004d3a0000
[    1.436975] CPU14: using LPI pending table @0x00000027dbde0000
[    1.436985] CPU14: Booted secondary processor [411fd071]
[    1.517656] Detected PIPT I-cache on CPU15
[    1.517664] GICv3: CPU15: found redistributor 10303 region 15:0x000000004d3d0000
[    1.517694] CPU15: using LPI pending table @0x00000027dbe10000
[    1.517704] CPU15: Booted secondary processor [411fd071]
[    1.517744] Brought up 16 CPUs
[    1.902893] SMP: Total of 16 processors activated.
[    1.908347] CPU features: detected feature: GIC system register CPU interface
[    1.916460] CPU features: detected feature: 32-bit EL0 Support
[    1.923155] CPU: All CPU(s) started at EL2
[    1.927814] alternatives: patching kernel code
[    1.937777] devtmpfs: initialized
[    1.941919] SMBIOS 3.0.0 present.
[    1.945799] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.957134] pinctrl core: initialized pinctrl subsystem
[    1.963526] NET: Registered protocol family 16
[    2.193616] cpuidle: using governor menu
[    2.198197] vdso: 2 pages (1 code @ ffff0000088b7000, 1 data @ ffff000008d54000)
[    2.206747] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    2.215097] DMA: preallocated 256 KiB pool for atomic allocations
[    2.222123] ACPI: bus type PCI registered
[    2.226733] Serial: AMBA PL011 UART driver
[    2.251622] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    2.259366] ACPI: Added _OSI(Module Device)
[    2.264179] ACPI: Added _OSI(Processor Device)
[    2.269297] ACPI: Added _OSI(3.0 _SCP Extensions)
[    2.274717] ACPI: Added _OSI(Processor Aggregator Device)
[    2.282835] ACPI: Interpreter enabled
[    2.287049] ACPI: Using GIC for interrupt routing
[    2.292504] ACPI: MCFG table detected, 3 entries
[    2.315378] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
[    2.322521] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.330752] acpi PNP0A08:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    2.342018] acpi PNP0A08:00: MCFG quirk: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f] with hisi_pcie_ops
[    2.354271] acpi PNP0A08:00: ECAM area [mem 0xb0000000-0xb1ffffff] reserved by HISI0081:00
[    2.363996] acpi PNP0A08:00: ECAM at [mem 0xb0000000-0xb1ffffff] for [bus 00-1f]
[    2.372554] Remapped I/O 0x00000000b7ff0000 to [io  0x0000-0xffff window]
[    2.380346] PCI host bridge to bus 0000:00
[    2.385008] pci_bus 0000:00: root bus resource [mem 0xb2000000-0xb7feffff window]
[    2.393531] pci_bus 0000:00: root bus resource [io  0x0000-0xffff window]
[    2.401260] pci_bus 0000:00: root bus resource [bus 00-1f]
[    2.407514] pci 0000:00:00.0: [19e5:1610] type 01 class 0x060400
[    2.407525] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.407564] pci 0000:00:00.0: supports D1 D2
[    2.407566] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.458916] pci 0000:00:00.0: BAR 0: assigned [mem 0xb2000000-0xb200ffff]
[    2.466641] pci 0000:00:00.0: PCI bridge to [bus 01]
[    2.472296] pci 0000:00:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.481939] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus e0-ff])
[    2.488972] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.497067] acpi PNP0A08:01: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    2.508154] acpi PNP0A08:01: MCFG quirk: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff] with hisi_pcie_ops
[    2.520271] acpi PNP0A08:01: ECAM area [mem 0xbe000000-0xbfffffff] reserved by HISI0081:01
[    2.529906] acpi PNP0A08:01: ECAM at [mem 0xbe000000-0xbfffffff] for [bus e0-ff]
[    2.538342] Remapped I/O 0x00000000bdff0000 to [io  0x10000-0x1ffff window]
[    2.546313] PCI host bridge to bus 0001:e0
[    2.550974] pci_bus 0001:e0: root bus resource [mem 0xb8000000-0xbdfeffff window]
[    2.559479] pci_bus 0001:e0: root bus resource [io  0x10000-0x1ffff window] (bus address [0x0000-0xffff])
[    2.570366] pci_bus 0001:e0: root bus resource [bus e0-ff]
[    2.576619] pci 0001:e0:00.0: [19e5:1610] type 01 class 0x060400
[    2.576627] pci 0001:e0:00.0: reg 0x10: [mem 0x00000000-0x00000fff]
[    2.576635] pci 0001:e0:00.0: reg 0x38: [mem 0xfffe0000-0xffffffff pref]
[    2.576662] pci 0001:e0:00.0: supports D1 D2
[    2.576664] pci 0001:e0:00.0: PME# supported from D0 D1 D3hot
[    2.576793] pci 0001:e1:00.0: [1000:005d] type 00 class 0x010400
[    2.576801] pci 0001:e1:00.0: reg 0x10: [io  0x10000-0x100ff]
[    2.576809] pci 0001:e1:00.0: reg 0x14: [mem 0xb8100000-0xb810ffff 64bit]
[    2.576816] pci 0001:e1:00.0: reg 0x1c: [mem 0xb8000000-0xb80fffff 64bit]
[    2.576824] pci 0001:e1:00.0: reg 0x30: [mem 0xfff00000-0xffffffff pref]
[    2.576874] pci 0001:e1:00.0: supports D1 D2
[    2.576954] pci 0001:e0:00.0: BAR 14: assigned [mem 0xb8000000-0xb82fffff]
[    2.584780] pci 0001:e0:00.0: BAR 6: assigned [mem 0xb8300000-0xb831ffff pref]
[    2.593001] pci 0001:e0:00.0: BAR 0: assigned [mem 0xb8320000-0xb8320fff]
[    2.600718] pci 0001:e0:00.0: BAR 13: assigned [io  0x10000-0x10fff]
[    2.607958] pci 0001:e1:00.0: BAR 3: assigned [mem 0xb8000000-0xb80fffff 64bit]
[    2.616272] pci 0001:e1:00.0: BAR 6: assigned [mem 0xb8100000-0xb81fffff pref]
[    2.624479] pci 0001:e1:00.0: BAR 1: assigned [mem 0xb8200000-0xb820ffff 64bit]
[    2.632798] pci 0001:e1:00.0: BAR 0: assigned [io  0x10000-0x100ff]
[    2.639943] pci 0001:e0:00.0: PCI bridge to [bus e1]
[    2.645584] pci 0001:e0:00.0:   bridge window [io  0x10000-0x10fff]
[    2.652711] pci 0001:e0:00.0:   bridge window [mem 0xb8000000-0xb82fffff]
[    2.660439] pci 0001:e0:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    2.670045] pci 0001:e1:00.0: Max Payload Size set to  512/4096 (was  128), Max Read Rq  512
[    2.679689] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-9f])
[    2.686723] acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig Segments MSI]
[    2.694818] acpi PNP0A08:02: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    2.705902] acpi PNP0A08:02: MCFG quirk: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f] with hisi_pcie_ops
[    2.717989] acpi PNP0A08:02: ECAM area [mem 0xa8000000-0xa9ffffff] reserved by HISI0081:02
[    2.727615] acpi PNP0A08:02: ECAM at [mem 0xa8000000-0xa9ffffff] for [bus 80-9f]
[    2.736049] Remapped I/O 0x00000000afff0000 to [io  0x20000-0x2ffff window]
[    2.744024] PCI host bridge to bus 0002:80
[    2.748685] pci_bus 0002:80: root bus resource [mem 0xaa000000-0xaffeffff window]
[    2.757205] pci_bus 0002:80: root bus resource [io  0x20000-0x2ffff window] (bus address [0x0000-0xffff])
[    2.768097] pci_bus 0002:80: root bus resource [bus 80-9f]
[    2.774344] pci 0002:80:00.0: [19e5:1610] type 01 class 0x060400
[    2.774351] pci 0002:80:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.774381] pci 0002:80:00.0: supports D1 D2
[    2.774383] pci 0002:80:00.0: PME# supported from D0 D1 D3hot
[    2.774514] pci 0002:81:00.0: [10b5:8725] type 01 class 0x060400
[    2.774525] pci 0002:81:00.0: reg 0x10: [mem 0xaa200000-0xaa23ffff]
[    2.774598] pci 0002:81:00.0: PME# supported from D0 D3hot D3cold
[    2.774749] pci 0002:82:00.0: [10b5:8725] type 01 class 0x060400
[    2.774831] pci 0002:82:00.0: PME# supported from D0 D3hot D3cold
[    2.774926] pci 0002:82:01.0: [10b5:8725] type 01 class 0x060400
[    2.775007] pci 0002:82:01.0: PME# supported from D0 D3hot D3cold
[    2.775103] pci 0002:82:02.0: [10b5:8725] type 01 class 0x060400
[    2.775184] pci 0002:82:02.0: PME# supported from D0 D3hot D3cold
[    2.775282] pci 0002:82:08.0: [10b5:8725] type 01 class 0x060400
[    2.775363] pci 0002:82:08.0: PME# supported from D0 D3hot D3cold
[    2.775574] pci 0002:84:00.0: [19e5:1711] type 00 class 0x030000
[    2.775591] pci 0002:84:00.0: reg 0x10: [mem 0xac000000-0xadffffff pref]
[    2.775602] pci 0002:84:00.0: reg 0x14: [mem 0xaa000000-0xaa1fffff]
[    2.775702] pci 0002:84:00.0: supports D1
[    2.775704] pci 0002:84:00.0: PME# supported from D0 D1 D3hot
[    2.775932] pci 0002:80:00.0: BAR 14: assigned [mem 0xaa000000-0xae7fffff]
[    2.783759] pci 0002:80:00.0: BAR 15: assigned [mem 0xae800000-0xaebfffff 64bit pref]
[    2.792668] pci 0002:80:00.0: BAR 0: assigned [mem 0xaec00000-0xaec0ffff]
[    2.800381] pci 0002:80:00.0: BAR 13: assigned [io  0x20000-0x21fff]
[    2.807611] pci 0002:81:00.0: BAR 14: assigned [mem 0xaa000000-0xadffffff]
[    2.815429] pci 0002:81:00.0: BAR 15: assigned [mem 0xae800000-0xaebfffff 64bit pref]
[    2.824331] pci 0002:81:00.0: BAR 0: assigned [mem 0xae000000-0xae03ffff]
[    2.832058] pci 0002:81:00.0: BAR 13: assigned [io  0x20000-0x21fff]
[    2.839281] pci 0002:82:01.0: BAR 14: assigned [mem 0xaa000000-0xacffffff]
[    2.847089] pci 0002:82:00.0: BAR 14: assigned [mem 0xad000000-0xad1fffff]
[    2.854919] pci 0002:82:00.0: BAR 15: assigned [mem 0xae800000-0xae9fffff 64bit pref]
[    2.863818] pci 0002:82:01.0: BAR 15: assigned [mem 0xaea00000-0xaebfffff 64bit pref]
[    2.872725] pci 0002:82:00.0: BAR 13: assigned [io  0x20000-0x20fff]
[    2.879955] pci 0002:82:01.0: BAR 13: assigned [io  0x21000-0x21fff]
[    2.887184] pci 0002:82:00.0: PCI bridge to [bus 83]
[    2.892832] pci 0002:82:00.0:   bridge window [io  0x20000-0x20fff]
[    2.899964] pci 0002:82:00.0:   bridge window [mem 0xad000000-0xad1fffff]
[    2.907674] pci 0002:82:00.0:   bridge window [mem 0xae800000-0xae9fffff 64bit pref]
[    2.916488] pci 0002:84:00.0: BAR 0: assigned [mem 0xaa000000-0xabffffff pref]
[    2.924703] pci 0002:84:00.0: BAR 1: assigned [mem 0xac000000-0xac1fffff]
[    2.932418] pci 0002:82:01.0: PCI bridge to [bus 84]
[    2.938055] pci 0002:82:01.0:   bridge window [io  0x21000-0x21fff]
[    2.945178] pci 0002:82:01.0:   bridge window [mem 0xaa000000-0xacffffff]
[    2.952907] pci 0002:82:01.0:   bridge window [mem 0xaea00000-0xaebfffff 64bit pref]
[    2.961707] pci 0002:82:02.0: PCI bridge to [bus 85]
[    2.967353] pci 0002:82:08.0: PCI bridge to [bus 86]
[    2.973005] pci 0002:81:00.0: PCI bridge to [bus 82-86]
[    2.978949] pci 0002:81:00.0:   bridge window [io  0x20000-0x21fff]
[    2.986071] pci 0002:81:00.0:   bridge window [mem 0xaa000000-0xadffffff]
[    2.993783] pci 0002:81:00.0:   bridge window [mem 0xae800000-0xaebfffff 64bit pref]
[    3.002589] pci 0002:80:00.0: PCI bridge to [bus 81-86]
[    3.008526] pci 0002:80:00.0:   bridge window [io  0x20000-0x21fff]
[    3.015647] pci 0002:80:00.0:   bridge window [mem 0xaa000000-0xae7fffff]
[    3.023364] pci 0002:80:00.0:   bridge window [mem 0xae800000-0xaebfffff 64bit pref]
[    3.032169] pci 0002:80:00.0: Max Payload Size set to  512/ 512 (was  128), Max Read Rq  512
[    3.041766] pci 0002:81:00.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    3.051365] pci 0002:82:00.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    3.060961] pci 0002:82:01.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    3.070563] pci 0002:84:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  256
[    3.080146] pci 0002:82:02.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    3.089748] pci 0002:82:08.0: Max Payload Size set to  512/1024 (was  128), Max Read Rq  128
[    3.099769] vgaarb: device added: PCI:0002:84:00.0,decodes=io+mem,owns=none,locks=none
[    3.108717] vgaarb: loaded
[    3.111768] vgaarb: bridge control possible 0002:84:00.0
[    3.117919] SCSI subsystem initialized
[    3.122244] libata version 3.00 loaded.
[    3.122322] ACPI: bus type USB registered
[    3.126926] usbcore: registered new interface driver usbfs
[    3.133179] usbcore: registered new interface driver hub
[    3.139270] usbcore: registered new device driver usb
[    3.145122] pps_core: LinuxPPS API ver. 1 registered
[    3.150754] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    3.161136] PTP clock support registered
[    3.165659] Registered efivars operations
[    3.170389] Advanced Linux Sound Architecture Driver Initialized.
[    3.177625] clocksource: Switched to clocksource arch_sys_counter
[    3.184576] VFS: Disk quotas dquot_6.6.0
[    3.189037] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    3.196917] pnp: PnP ACPI init
[    3.200549] system 00:00: [mem 0xa0090000-0xa009ffff] has been reserved
[    3.208026] system 00:00: [mem 0xb0000000-0xb1ffffff] could not be reserved
[    3.215895] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.215960] system 00:01: [mem 0xa0200000-0xa020ffff] has been reserved
[    3.223435] system 00:01: [mem 0xbe000000-0xbfffffff] could not be reserved
[    3.231303] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.231368] system 00:02: [mem 0xa00a0000-0xa00affff] has been reserved
[    3.238842] system 00:02: [mem 0xa8000000-0xa9ffffff] could not be reserved
[    3.246710] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active)
[    3.246717] pnp: PnP ACPI: found 3 devices
[    3.254213] NET: Registered protocol family 2
[    3.259664] TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
[    3.268853] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    3.276795] TCP: Hash tables configured (established 262144 bind 65536)
[    3.284389] UDP hash table entries: 16384 (order: 7, 524288 bytes)
[    3.291603] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
[    3.299348] NET: Registered protocol family 1
[    3.304456] RPC: Registered named UNIX socket transport module.
[    3.311145] RPC: Registered udp transport module.
[    3.316457] RPC: Registered tcp transport module.
[    3.321768] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    3.329061] pci 0001:e1:00.0: [Firmware Bug]: VPD access disabled
[    3.335960] PCI: CLS 0 bytes, default 128
[    3.336049] Unpacking initramfs...
[    3.722561] Freeing initrd memory: 27492K (ffff80001e520000 - ffff80001fff9000)
[    3.731095] kvm [1]: 8-bit VMID
[    3.734651] kvm [1]: IDMAP page: 89f000
[    3.738985] kvm [1]: HYP VA range: 800000000000:ffffffffffff
[    3.746090] kvm [1]: Hyp mode initialized successfully
[    3.751923] kvm [1]: vgic-v2@fe020000
[    3.756085] kvm [1]: GIC system register CPU interface enabled
[    3.762895] kvm [1]: vgic interrupt IRQ1
[    3.767338] kvm [1]: virtual timer IRQ4
[    3.773322] futex hash table entries: 4096 (order: 7, 524288 bytes)
[    3.780529] audit: initializing netlink subsys (disabled)
[    3.786666] audit: type=2000 audit(3.140:1): initialized
[    3.792914] workingset: timestamp_bits=44 max_order=23 bucket_order=0
[    3.804097] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    3.811051] NFS: Registering the id_resolver key type
[    3.816774] Key type id_resolver registered
[    3.821498] Key type id_legacy registered
[    3.826029] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    3.833672] 9p: Installing v9fs 9p2000 file system support
[    3.840990] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    3.849350] io scheduler noop registered
[    3.853838] io scheduler cfq registered (default)
[    3.859375] libphy: mdio_driver_register: phy-bcm-ns2-pci
[    3.865858] pcieport 0000:00:00.0: can't derive routing for PCI INT A
[    3.873136] pcieport 0000:00:00.0: PCI INT A: no GSI
[    3.878800] pcieport 0001:e0:00.0: can't derive routing for PCI INT A
[    3.886093] pcieport 0001:e0:00.0: PCI INT A: no GSI
[    3.891734] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.899012] pcieport 0002:80:00.0: PCI INT A: no GSI
[    3.904649] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.911924] pcieport 0002:81:00.0: PCI INT A: no GSI
[    3.917720] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.924996] pcieport 0002:82:00.0: PCI INT A: no GSI
[    3.930753] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.938030] pcieport 0002:82:01.0: PCI INT A: no GSI
[    3.943786] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.951063] pcieport 0002:82:02.0: PCI INT A: no GSI
[    3.956812] pcieport 0002:80:00.0: can't derive routing for PCI INT A
[    3.964090] pcieport 0002:82:08.0: PCI INT A: no GSI
[    3.970064] ipmi message handler version 39.2
[    3.974988] ipmi device interface
[    3.978761] IPMI System Interface driver.
[    3.983345] ipmi_si: probing via SMBIOS
[    3.987677] ipmi_si: SMBIOS: io 0xe4 regsize 1 spacing 1 irq 0
[    3.994265] ipmi_si: Adding SMBIOS-specified bt state machine
[    4.000766] ipmi_si: Trying SMBIOS-specified bt state machine at i/o address 0xe4, slave address 0x0, irq 0
[    4.069672] IPMI BT: using default values
[    4.074195] IPMI BT: req2rsp=5 secs retries=2
[    4.277762] ipmi_si ipmi_si.0: Found new BMC (man_id: 0x0007db, prod_id: 0x0001, dev_id: 0x01)
[    4.287559] ipmi_si ipmi_si.0: IPMI bt interface initialized
[    4.294811] xenfs: not registering filesystem on non-xen platform
[    4.303084] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    4.310866] console [ttyS0] disabled
[    4.336261] serial8250.10: ttyS0 at I/O 0x2f8 (irq = 0, base_baud = 115200) is a 16550A
[    4.345346] console [ttyS0] enabled
[    4.352768] bootconsole [hisilpcuart0] disabled
[    4.362587] SuperH (H)SCI(F) driver initialized
[    4.367188] msm_serial: driver initialized
[    4.371517] Failed to find cpu0 device node
[    4.375702] Unable to detect cache hierarchy from DT for CPU 0
[    4.384605] loop: module loaded
[    4.387964] hisi_sas: driver version v1.6
[    4.526172] scsi host0: hisi_sas
[    4.666170] scsi host1: hisi_sas
[    4.802186] scsi host2: hisi_sas
[    4.806904] libphy: Fixed MDIO Bus: probed
[    4.811147] tun: Universal TUN/TAP device driver, 1.6
[    4.816197] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    4.901635] libphy: Hisilicon MII Bus: probed
[    5.781718] netif_napi_add() called with weight 256 on device eth%d
[    5.878175] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    5.884003] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    5.889956] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    5.896901] igb: Copyright (c) 2007-2014 Intel Corporation.
[    5.902491] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[    5.910305] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    5.916235] sky2: driver version 1.30
[    5.920085] VFIO - User Level meta-driver version: 0.3
[    5.925956] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    5.932473] ehci-pci: EHCI PCI platform driver
[    5.936938] ehci-platform: EHCI generic platform driver
[    5.942204] ehci-platform PNP0D20:00: EHCI Host Controller
[    5.947692] ehci-platform PNP0D20:00: new USB bus registered, assigned bus number 1
[    5.955441] ehci-platform PNP0D20:00: irq 6, io mem 0xa7020000
[    5.973635] ehci-platform PNP0D20:00: USB 2.0 started, EHCI 1.00
[    5.979947] hub 1-0:1.0: USB hub found
[    5.983707] hub 1-0:1.0: 2 ports detected
[    5.987945] ehci-exynos: EHCI EXYNOS driver
[    5.992153] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[    5.997740] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    6.003917] ohci-pci: OHCI PCI platform driver
[    6.008384] ohci-platform: OHCI generic platform driver
[    6.013627] ohci-exynos: OHCI EXYNOS driver
[    6.017915] usbcore: registered new interface driver usb-storage
[    6.024184] mousedev: PS/2 mouse device common for all mice
[    6.155910] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[    6.162144] i2c /dev entries driver
[    6.166106] sdhci: Secure Digital Host Controller Interface driver
[    6.172274] sdhci: Copyright(c) Pierre Ossman
[    6.176688] Synopsys Designware Multimedia Card Interface Driver
[    6.182760] sdhci-pltfm: SDHCI platform and OF driver helper
[    6.188706] ledtrig-cpu: registered to indicate activity on CPUs
[    6.195002] usbcore: registered new interface driver usbhid
[    6.200569] usbhid: USB HID core driver
[    6.204901] NET: Registered protocol family 17
[    6.209389] 9pnet: Installing 9P2000 support
[    6.213693] Key type dns_resolver registered
[    6.218220] registered taskstats version 1
[    6.285444] rtc-efi rtc-efi: setting system clock to 2016-12-02 19:27:20 UTC (1480706840)
[    6.293660] ALSA device list:
[    6.296629]   No soundcards found.
[    6.300685] Freeing unused kernel memory: 1024K (ffff800000c50000 - ffff800000d50000)
[    6.313634] usb 1-1: new high-speed USB device number 2 using ehci-platform
[    6.466335] hub 1-1:1.0: USB hub found
[    6.470155] hub 1-1:1.0: 4 ports detected
[    6.593629] usb 1-2: new high-speed USB device number 3 using ehci-platform
[    6.746330] hub 1-2:1.0: USB hub found
[    6.750156] hub 1-2:1.0: 4 ports detected
[    7.041629] usb 1-2.1: new full-speed USB device number 4 using ehci-platform
[    7.158255] input: Keyboard/Mouse KVM 1.1.0 as /devices/platform/PNP0D20:00/usb1/1-2/1-2.1/1-2.1:1.0/0003:12D1:0003.0001/input/input0
[    7.229870] hid-generic 0003:12D1:0003.0001: input: USB HID v1.10 Keyboard [Keyboard/Mouse KVM 1.1.0] on usb-PNP0D20:00-2.1/input0
[    7.242699] input: Keyboard/Mouse KVM 1.1.0 as /devices/platform/PNP0D20:00/usb1/1-2/1-2.1/1-2.1:1.1/0003:12D1:0003.0002/input/input1
[    7.254754] hid-generic 0003:12D1:0003.0002: input: USB HID v1.10 Mouse [Keyboard/Mouse KVM 1.1.0] on usb-PNP0D20:00-2.1/input1
root@(none)$ cat /proc/iomem
00000000-00024fff : reserved
00026000-319fffff : System RAM
   00080000-00c4ffff : Kernel code
   00d50000-00e75fff : Kernel data
31a00000-31a6ffff : reserved
31a70000-31b5ffff : System RAM
31b60000-31b6ffff : reserved
31b70000-31b7dfff : System RAM
31b7e000-31b81fff : reserved
31b82000-39c0ffff : System RAM
39c10000-3a11ffff : reserved
3a120000-3f10ffff : System RAM
3f110000-3f13ffff : reserved
3f140000-3fbfffff : System RAM
603c0000-603cffff : HISI0141:00
   603c0000-603cffff : HISI0141:00
a0080000-a008ffff : HISI0152:02
   a0080000-a008ffff : HISI0152:01
     a0080000-a008ffff : HISI0152:00
a0090000-a009ffff : pnp 00:00
a00a0000-a00affff : pnp 00:02
a01b0000-a01b0fff : HISI0191:00
   a01b0000-a01b0fff : HISI0191:00
a0200000-a020ffff : pnp 00:01
a2000000-a200ffff : HISI0162:01
   a2000000-a200ffff : HISI0162:01
a3000000-a300ffff : HISI0162:02
   a3000000-a300ffff : HISI0162:02
a7020000-a702ffff : PNP0D20:00
   a7020000-a702ffff : PNP0D20:00
a8000000-a9ffffff : PCI ECAM
aa000000-affeffff : PCI Bus 0002:80
   aa000000-ae7fffff : PCI Bus 0002:81
     aa000000-adffffff : PCI Bus 0002:82
       aa000000-acffffff : PCI Bus 0002:84
         aa000000-abffffff : 0002:84:00.0
         ac000000-ac1fffff : 0002:84:00.0
       ad000000-ad1fffff : PCI Bus 0002:83
     ae000000-ae03ffff : 0002:81:00.0
   ae800000-aebfffff : PCI Bus 0002:81
     ae800000-aebfffff : PCI Bus 0002:82
       ae800000-ae9fffff : PCI Bus 0002:83
       aea00000-aebfffff : PCI Bus 0002:84
   aec00000-aec0ffff : 0002:80:00.0
b0000000-b1ffffff : PCI ECAM
b2000000-b7feffff : PCI Bus 0000:00
   b2000000-b200ffff : 0000:00:00.0
b8000000-bdfeffff : PCI Bus 0001:e0
   b8000000-b82fffff : PCI Bus 0001:e1
     b8000000-b80fffff : 0001:e1:00.0
     b8100000-b81fffff : 0001:e1:00.0
     b8200000-b820ffff : 0001:e1:00.0
   b8300000-b831ffff : 0001:e0:00.0
   b8320000-b8320fff : 0001:e0:00.0
be000000-bfffffff : PCI ECAM
c0080000-c008ffff : HISI0152:05
   c0080000-c008ffff : HISI0152:04
     c0080000-c008ffff : HISI0152:03
c3000000-c300ffff : HISI0162:00
   c3000000-c300ffff : HISI0162:00
c4000000-c40fffff : HISI00D1:00
c5000000-c588ffff : HISI00B2:00
   c5000000-c588ffff : HISI00B2:00
c7000000-c705ffff : HISI00B2:00
   c7000000-c705ffff : HISI00B2:00
2040000000-27ffffffff : System RAM


Thanks,
Dongdong

RE: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Gabriele Paoloni]

SGkgQmpvcm4NCg0KTWFueSBtYW55IHRoYW5rcyBmb3IgcHV0dGluZyB0aGlzIHRvZ2V0aGVyIQ0K
DQpbLi4uXQ0KDQo+IA0KPiAgIFBDSTogQWRkIE1DRkcgcXVpcmtzIGZvciBIaVNpbGljb24gSGlw
MDUvMDYvMDcgaG9zdCBjb250cm9sbGVycw0KPiANCj4gICAgIERvbmdkb25nJ3MgSGlTaWxpY29u
IHF1aXJrcy4gIEhlcmUncyB3aGVyZSBpdCBnZXRzIGludGVyZXN0aW5nLiAgSQ0KPiAgICAgbW92
ZWQgdGhpcyB0byB0aGUgZXhpc3RpbmcgcGNpZS1oaXNpLmMgaW5zdGVhZCBvZiBhZGRpbmcNCj4g
ICAgIHBjaWUtaGlzaS1hY3BpLmMuICBJIGNoYW5nZWQgdGhlIE1ha2VmaWxlIHNvIHdlIGFsd2F5
cyBidWlsZA0KPiAgICAgcGNpZS1oaXNpLmMgb24gQVJNNjQuICBJIGFkZGVkIGlmZGVmcyBzbyB3
ZSBnZXQgdGhlIHF1aXJrIGNvZGUgaWYNCj4gICAgIENPTkZJR19BQ1BJIGFuZCBDT05GSUdfUENJ
X1FVSVJLUyBhbmQgd2UgZ2V0IHRoZSBvcmlnaW5hbCBwbGF0Zm9ybQ0KPiAgICAgZHJpdmVyIGlm
IENPTkZJR19QQ0lfSElTSS4gIEl0J3MgcG9zc2libGUgdG8gaGF2ZSBib3RoLCBhbmQgaWYgd2UN
Cj4gICAgIHByb2Nlc3MgdGhlIE1DRkcgcXVpcmsgd2UgZ2V0IHRoZSBxdWlyayBjb2RlLg0KPiAN
Cj4gICAgIEknbSBjb25mdXNlZCBhYm91dCB3aHkgdGhlIHF1aXJrIGFjY2Vzc29ycyBhcmUgc28g
bXVjaCBkaWZmZXJlbnQNCj4gICAgIHRoYW4gdGhlIG9yaWdpbmFsIGFjY2Vzc29ycy4gIGhpc2lf
cGNpZV9hY3BpX3JkX2NvbmYoKSBsb29rcyBtdWNoDQo+ICAgICBkaWZmZXJlbnQgdGhhbiBoaXNp
X3BjaWVfY2ZnX3JlYWQoKS4gIFRoZSBvcmlnaW5hbCBkcml2ZXIgY2xhaW1zDQo+ICAgICBIaXB4
eCBvbmx5IHN1cHBvcnRzIDMyLWJpdCBjb25maWcgYWNjZXNzZXMsIGJ1dCB0aGUgcXVpcmsNCj4g
ICAgIGFjY2Vzc29ycyBkb24ndCBlbmZvcmNlIHRoYXQuDQoNCldlbGwgdGhlIG1haW4gcG9pbnQg
aXMgdGhhdCBoaXNpX3BjaWVfYWNwaV9yZF9jb25mKCkgYXNzaWduZWQgdG8NCnBjaV9vcHMucmVh
ZCB3aGVyZWFzIGhpc2lfcGNpZV9jZmdfcmVhZCgpIGlzIGFzc2lnbmVkIHRvDQpoaXNpX3BjaWVf
aG9zdF9vcHMucmRfb3duX2NvbmYuIFRoZSBmaXJzdCBvbmUgaXMgdXNlZCB0byBhY2Nlc3MgdGhl
DQp3aG9sZSBwY2kgY29uZmlndXJhdGlvbiBzcGFjZSwgd2hlcmVhcyB0aGUgc2Vjb25kIGlzIGRl
ZGljYXRlZCB0byB0aGUNCnJvb3QgY29tcGxleCAnb3duJyBjb25maWd1cmF0aW9uIHNwYWNlLg0K
QXMgeW91IGNhbiBzZWUgYWxzbyBpbiBoaXNpX3BjaWVfYWNwaV9yZF9jb25mKCkgd2UgZW5mb3Jj
ZSAzMmIgYWNjZXNzDQpmb3IgdGhlIHJvb3QgY29tcGxleC4NCg0KTm93IHRoZSBtYWluIGlzc3Vl
IGhlcmUgaXMgdGhhdCBhbGwgdGhlIGRlc2lnbndhcmUgYmFzZWQgZHJpdmVycyBjdXJyZW50bHkN
CnVzZSB0aGUgc2FtZSBpbnN0YW5jZSBvZiBwY2lfb3BzOg0KaHR0cDovL2x4ci5mcmVlLWVsZWN0
cm9ucy5jb20vc291cmNlL2RyaXZlcnMvcGNpL2hvc3QvcGNpZS1kZXNpZ253YXJlLmMjTDcxMw0K
DQpJbiBvcmRlciB0byBzb2x2ZSB0aGlzIHdlIHNob3VsZCB0dXJuIHRoZSBkZXNpZ253YXJlIHBj
aV9vcHMgaW50byBwZXINCmRyaXZlciBpbnN0YW5jZXM7IHNlZSBteSBjb21tZW50IGluIGEgcHJl
dmlvdXMgZGlzY3Vzc2lvbjoNCmh0dHA6Ly93d3cuc3Bpbmljcy5uZXQvbGlzdHMvbGludXgtcGNp
L21zZzU2MDk3Lmh0bWwNCg0KVGhhbmtzDQoNCkdhYg0KWy4uLl0NCg==

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Tomasz Nowicki]

On 01.12.2016 09:29, Bjorn Helgaas wrote:
> This is what I've collected on my pci/ecam branch so far.  This is
> preliminary and probably doesn't even build since I made several
> changes, but at least it's a strawman.  I called it v10 to keep it
> separate from the previous postings of the various pieces.
>
>   ACPI: Add acpi_resource_consumer() to find device that claims a resource
>   PCI: Search ACPI namespace to ensure ECAM space is reserved
>   x86/PCI: Use acpi_resource_consumer() to search ACPI namespace for MMCFG
>
>     acpi_resource_consumer() takes a resource and returns an
>     acpi_device that has _CRS that contains the resource.  The idea is
>     that we call this on a resource from MCFG.  We *should* find a
>     PNP0C02 or other device that reserves that resource.  On x86 we
>     already do this in a more hand-coded way.  I probably wouldn't
>     merge the x86 patch for v4.10 but I included it here as an
>     example.
>
>     I don't think it's worth trying to fabricate ACPI devices or _CRS
>     resources to compensate for firmware that doesn't describe
>     everything.  This check will emit a warning if the MCFG region is
>     not reserved, and that's probably enough to motivate firmware
>     fixes for the next round of hardware.  The ECAM code itself does
>     reserve the region, so that part will be safe from other users.
>     Unreported non-ECAM register space is still a landmine and there's
>     no real way to look for it.
>
>   arm64: PCI: Manage controller-specific data on per-controller basis
>
>     Tomasz's fix for pci_acpi_scan_root(), unchanged.
>
>   PCI/ACPI: Extend pci_mcfg_lookup() to return ECAM config accessors
>   PCI/ACPI: Check for platform-specific MCFG quirks
>
>     Tomasz's quirk infrastructure.  I put this under
>     CONFIG_PCI_QUIRKS.
>
>   PCI/ACPI: Provide acpi_get_rc_resources() for ARM64 platform
>
>     Dongdong's interface to look up a _HID with a _UID matching the
>     segment.  I don't have an opinion on using _UID yet.  It's
>     encapsulated so it could be changed easily.  I put this under
>     CONFIG_PCI_QUIRKS && CONFIG_ARM64.
>
>   PCI: Add MCFG quirks for Qualcomm QDF2432 host controller
>
>     Christopher's Qualcomm quirks.  Basically unchanged except to put
>     the quirk ops under CONFIG_ACPI and CONFIG_PCI_QUIRKS.
>
>   PCI: Add MCFG quirks for HiSilicon Hip05/06/07 host controllers
>
>     Dongdong's HiSilicon quirks.  Here's where it gets interesting.  I
>     moved this to the existing pcie-hisi.c instead of adding
>     pcie-hisi-acpi.c.  I changed the Makefile so we always build
>     pcie-hisi.c on ARM64.  I added ifdefs so we get the quirk code if
>     CONFIG_ACPI and CONFIG_PCI_QUIRKS and we get the original platform
>     driver if CONFIG_PCI_HISI.  It's possible to have both, and if we
>     process the MCFG quirk we get the quirk code.
>
>     I'm confused about why the quirk accessors are so much different
>     than the original accessors.  hisi_pcie_acpi_rd_conf() looks much
>     different than hisi_pcie_cfg_read().  The original driver claims
>     Hipxx only supports 32-bit config accesses, but the quirk
>     accessors don't enforce that.
>
>   PCI: thunder-pem: Factor out resource lookup
>   PCI: Add MCFG quirks for Cavium ThunderX pass2.x host controller
>   PCI: Add MCFG quirks for Cavium ThunderX pass1.x host controller
>
>     Tomasz's ThunderX quirks.  I restructured the PEM init I could do
>     the same ifdef tricks as in hisi.  The quirks make a lot more
>     sense here -- they use the same accessors as the original platform
>     driver.  Only the initialization is different.
>
> The Makefile looks a little strange:
>
>   obj-$(CONFIG_ARM64) += pcie-hisi.o
>   obj-$(CONFIG_ARM64) += pci-thunder-ecam.o
>   obj-$(CONFIG_ARM64) += pci-thunder-pem.o

Hi Bjorn:

dmesg from ThunderX pass2.0 (1 socket board) + small fix:

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index d34d196..9e8fa4e 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -98,8 +98,8 @@ static struct mcfg_fixup mcfg_quirks[] = {
         { "CAVIUM", "THUNDERX", rev, seg, MCFG_BUS_ANY,                 \
         &pci_thunder_ecam_ops }
         /* SoC pass1.x */
-       THUNDER_PEM_QUIRK(2,  0),       /* off-chip devices */
-       THUNDER_PEM_QUIRK(2,  1),       /* off-chip devices */
+       THUNDER_PEM_QUIRK(2,  0UL),     /* off-chip devices */
+       THUNDER_PEM_QUIRK(2,  1UL),     /* off-chip devices */
         THUNDER_ECAM_QUIRK(2,  0),
         THUNDER_ECAM_QUIRK(2,  1),
         THUNDER_ECAM_QUIRK(2,  2),


EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
Booting Linux on physical CPU 0x0
Linux version 4.9.0-rc7+ (tn@ThinkPad-Edge-E530) (gcc version 5.2.0 
(Cavium Inc. Version 0.99 build 401) ) #54 SMP PREEMPT Fri Dec 2 
12:56:56 CET 2016
Boot CPU: AArch64 Processor [431f0a10]
earlycon: pl11 at MMIO 0x000087e024000000 (options '')
bootconsole [pl11] enabled
efi: Getting EFI parameters from FDT:
efi: EFI v2.40 by Cavium Thunder cn88xx EFI 
ThunderX-Firmware-Release-1.22.12-build_04-2-g3ca4638 Nov 14 2016 09:57:1
efi:  ACPI=0xfffff000  ACPI 2.0=0xfffff014  SMBIOS 3.0=0x1ffaa58000
ACPI: Early table checksum verification disabled
ACPI: RSDP 0x00000000FFFFF014 000024 (v02 CAVIUM)
ACPI: XSDT 0x00000000FFFFE0E8 00006C (v01 CAVIUM THUNDERX 00000000 01000013)
ACPI: FACP 0x00000000FFFF7000 000114 (v06 CAVIUM THUNDERX 00000000 INTL 
20150717)
ACPI: DSDT 0x00000000FFFF8000 0020C5 (v02 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: APIC 0x00000000FFFFD000 000F68 (v03 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: SSDT 0x00000000FFFFC000 000158 (v02 CAVIUM CRB-1S   00000001 INTL 
20150717)
ACPI: DBG2 0x00000000FFFFB000 0000AA (v01 CAVIUM CN88XDBG 00000000 INTL 
20150717)
ACPI: GTDT 0x00000000FFFF5000 0000E0 (v02 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: IORT 0x00000000FFFF3000 0013D4 (v01 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: MCFG 0x00000000FFFF2000 00006C (v01 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: SLIT 0x00000000FFFF1000 000030 (v01 CAVIUM TEMPLATE 00000001 INTL 
20150717)
ACPI: SPCR 0x00000000FFFF0000 000050 (v02 CAVIUM THUNDERX 00000001 INTL 
20150717)
ACPI: SPCR: console: pl011,mmio,0x87e024000000,115200
ACPI: NUMA: Failed to initialise from firmware
NUMA: Faking a node at [mem 0x0000000000000000-0x0000001fffffffff]
NUMA: Adding memblock [0x1400000 - 0xfffeffff] on node 0
NUMA: Adding memblock [0xffff0000 - 0xffffffff] on node 0
NUMA: Adding memblock [0x100000000 - 0x1ffaa4ffff] on node 0
NUMA: Adding memblock [0x1ffaa50000 - 0x1ffaa7ffff] on node 0
NUMA: Adding memblock [0x1ffaa80000 - 0x1fffa3ffff] on node 0
NUMA: Adding memblock [0x1fffa40000 - 0x1fffa8ffff] on node 0
NUMA: Adding memblock [0x1fffa90000 - 0x1fffffffff] on node 0
NUMA: Initmem setup node 0 [mem 0x01400000-0x1fffffffff]
NUMA: NODE_DATA [mem 0x1fffffe200-0x1fffffffff]
Zone ranges:
   DMA      [mem 0x0000000001400000-0x00000000ffffffff]
   Normal   [mem 0x0000000100000000-0x0000001fffffffff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x0000000001400000-0x00000000fffeffff]
   node   0: [mem 0x00000000ffff0000-0x00000000ffffffff]
   node   0: [mem 0x0000000100000000-0x0000001ffaa4ffff]
   node   0: [mem 0x0000001ffaa50000-0x0000001ffaa7ffff]
   node   0: [mem 0x0000001ffaa80000-0x0000001fffa3ffff]
   node   0: [mem 0x0000001fffa40000-0x0000001fffa8ffff]
   node   0: [mem 0x0000001fffa90000-0x0000001fffffffff]
Initmem setup node 0 [mem 0x0000000001400000-0x0000001fffffffff]
psci: probing for conduit method from ACPI.
psci: PSCIv0.2 detected in firmware.
psci: Using standard PSCI v0.2 function IDs
psci: Trusted OS resident on physical CPU 0x0
percpu: Embedded 2 pages/cpu @ffff801ff76f0000 s48256 r8192 d74624 u131072
Detected VIPT I-cache on CPU0
CPU features: enabling workaround for Cavium erratum 27456
Built 1 zonelists in Node order, mobility grouping on.  Total pages: 2094784
Policy zone: Normal
Kernel command line: BOOT_IMAGE=(tftp)efi/Image_tn root=/dev/sda2 rw 
console=ttyAMA0 earlycon=pl011,0x87e024000000 kgdboc=ttyAMA0,115200 
uefi_debug acpi=force
log_buf_len individual max cpu contribution: 4096 bytes
log_buf_len total cpu_extra contributions: 192512 bytes
log_buf_len min size: 16384 bytes
log_buf_len: 262144 bytes
early log buf free: 10892(66%)
PID hash table entries: 4096 (order: -1, 32768 bytes)
software IO TLB [mem 0xfbfe0000-0xfffe0000] (64MB) mapped at 
[ffff8000fbfe0000-ffff8000fffdffff]
Memory: 133978304K/134197248K available (8572K kernel code, 722K rwdata, 
3136K rodata, 640K init, 647K bss, 218944K reserved, 0K cma-reserved)
Virtual kernel memory layout:
     modules : 0xffff000000000000 - 0xffff000008000000   (   128 MB)
     vmalloc : 0xffff000008000000 - 0xffff7bdfffff0000   (126847 GB)
       .text : 0xffff000008080000 - 0xffff0000088e0000   (  8576 KB)
     .rodata : 0xffff0000088e0000 - 0xffff000008c00000   (  3200 KB)
       .init : 0xffff000008c00000 - 0xffff000008ca0000   (   640 KB)
       .data : 0xffff000008ca0000 - 0xffff000008d54a00   (   723 KB)
        .bss : 0xffff000008d54a00 - 0xffff000008df6968   (   648 KB)
     fixed   : 0xffff7fdffe7d0000 - 0xffff7fdffec00000   (  4288 KB)
     PCI I/O : 0xffff7fdffee00000 - 0xffff7fdfffe00000   (    16 MB)
     vmemmap : 0xffff7fe000000000 - 0xffff800000000000   (   128 GB maximum)
               0xffff7fe000005000 - 0xffff7fe008000000   (   127 MB actual)
     memory  : 0xffff800001400000 - 0xffff802000000000   (131052 MB)
SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=48, Nodes=1
Preemptible hierarchical RCU implementation.
RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=48
NR_IRQS:64 nr_irqs:64 0
GICv3: GIC: Using split EOI/Deactivate mode
ITS [mem 0x801000020000-0x80100003ffff]
ITS@0x0000801000020000: allocated 2097152 Devices @1fc1000000 (flat, esz 
8, psz 64K, shr 1)
GIC: using LPI property table @0x0000001fc0060000
ITS: Allocated 32512 chunks for LPIs
GICv3: CPU0: found redistributor 0 region 0:0x0000801080000000
CPU0: using LPI pending table @0x0000001fc0070000
arm_arch_timer: Architected cp15 timer(s) running at 100.00MHz (phys).
clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 
0x171024e7e0, max_idle_ns: 440795205315 ns
sched_clock: 56 bits at 100MHz, resolution 10ns, wraps every 4398046511100ns
Console: colour dummy device 80x25
Calibrating delay loop (skipped), value calculated using timer 
frequency.. 200.00 BogoMIPS (lpj=400000)
pid_max: default: 49152 minimum: 384
ACPI: Core revision 20160831
ACPI: 2 ACPI AML tables successfully acquired and loaded
Security Framework initialized
Dentry cache hash table entries: 16777216 (order: 11, 134217728 bytes)
Inode-cache hash table entries: 8388608 (order: 10, 67108864 bytes)
Mount-cache hash table entries: 262144 (order: 5, 2097152 bytes)
Mountpoint-cache hash table entries: 262144 (order: 5, 2097152 bytes)
ASID allocator initialised with 65536 entries
PCI/MSI: ITS@0x801000020000 domain created
Remapping and enabling EFI services.
   EFI remap 0x0000001ffaa52000 => 0000000020002000
   EFI remap 0x0000001ffaa53000 => 0000000020003000
   EFI remap 0x0000001ffaa57000 => 0000000020007000
   EFI remap 0x0000001fffa42000 => 0000000020032000
   EFI remap 0x0000001fffa56000 => 0000000020046000
   EFI remap 0x0000804000001000 => 0000000020081000
   EFI remap 0x000087e0d2001000 => 0000000020091000
Detected VIPT I-cache on CPU1
GICv3: CPU1: found redistributor 1 region 0:0x0000801080020000
CPU1: using LPI pending table @0x0000001fcd880000
CPU1: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU2
GICv3: CPU2: found redistributor 2 region 0:0x0000801080040000
CPU2: using LPI pending table @0x0000001fcd8a0000
CPU2: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU3
GICv3: CPU3: found redistributor 3 region 0:0x0000801080060000
CPU3: using LPI pending table @0x0000001fcd8b0000
CPU3: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU4
GICv3: CPU4: found redistributor 4 region 0:0x0000801080080000
CPU4: using LPI pending table @0x0000001fcd8f0000
CPU4: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU5
GICv3: CPU5: found redistributor 5 region 0:0x00008010800a0000
CPU5: using LPI pending table @0x0000001fcd900000
CPU5: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU6
GICv3: CPU6: found redistributor 6 region 0:0x00008010800c0000
CPU6: using LPI pending table @0x0000001fcd930000
CPU6: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU7
GICv3: CPU7: found redistributor 7 region 0:0x00008010800e0000
CPU7: using LPI pending table @0x0000001fcd940000
CPU7: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU8
GICv3: CPU8: found redistributor 8 region 0:0x0000801080100000
CPU8: using LPI pending table @0x0000001fcd960000
CPU8: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU9
GICv3: CPU9: found redistributor 9 region 0:0x0000801080120000
CPU9: using LPI pending table @0x0000001fcda00000
CPU9: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU10
GICv3: CPU10: found redistributor a region 0:0x0000801080140000
CPU10: using LPI pending table @0x0000001fcda10000
CPU10: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU11
GICv3: CPU11: found redistributor b region 0:0x0000801080160000
CPU11: using LPI pending table @0x0000001fcda60000
CPU11: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU12
GICv3: CPU12: found redistributor c region 0:0x0000801080180000
CPU12: using LPI pending table @0x0000001fcda70000
CPU12: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU13
GICv3: CPU13: found redistributor d region 0:0x00008010801a0000
CPU13: using LPI pending table @0x0000001fcdb10000
CPU13: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU14
GICv3: CPU14: found redistributor e region 0:0x00008010801c0000
CPU14: using LPI pending table @0x0000001fcdb20000
CPU14: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU15
GICv3: CPU15: found redistributor f region 0:0x00008010801e0000
CPU15: using LPI pending table @0x0000001fcdb40000
CPU15: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU16
GICv3: CPU16: found redistributor 100 region 0:0x0000801080200000
CPU16: using LPI pending table @0x0000001fcdb60000
CPU16: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU17
GICv3: CPU17: found redistributor 101 region 0:0x0000801080220000
CPU17: using LPI pending table @0x0000001fcdc20000
CPU17: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU18
GICv3: CPU18: found redistributor 102 region 0:0x0000801080240000
CPU18: using LPI pending table @0x0000001fcdc30000
CPU18: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU19
GICv3: CPU19: found redistributor 103 region 0:0x0000801080260000
CPU19: using LPI pending table @0x0000001fcdc60000
CPU19: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU20
GICv3: CPU20: found redistributor 104 region 0:0x0000801080280000
CPU20: using LPI pending table @0x0000001fcdc70000
CPU20: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU21
GICv3: CPU21: found redistributor 105 region 0:0x00008010802a0000
CPU21: using LPI pending table @0x0000001fcdc90000
CPU21: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU22
GICv3: CPU22: found redistributor 106 region 0:0x00008010802c0000
CPU22: using LPI pending table @0x0000001fcdcb0000
CPU22: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU23
GICv3: CPU23: found redistributor 107 region 0:0x00008010802e0000
CPU23: using LPI pending table @0x0000001fcdcc0000
CPU23: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU24
GICv3: CPU24: found redistributor 108 region 0:0x0000801080300000
CPU24: using LPI pending table @0x0000001fcdd80000
CPU24: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU25
GICv3: CPU25: found redistributor 109 region 0:0x0000801080320000
CPU25: using LPI pending table @0x0000001fcdd90000
CPU25: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU26
GICv3: CPU26: found redistributor 10a region 0:0x0000801080340000
CPU26: using LPI pending table @0x0000001fcddc0000
CPU26: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU27
GICv3: CPU27: found redistributor 10b region 0:0x0000801080360000
CPU27: using LPI pending table @0x0000001fcddd0000
CPU27: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU28
GICv3: CPU28: found redistributor 10c region 0:0x0000801080380000
CPU28: using LPI pending table @0x0000001fcde80000
CPU28: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU29
GICv3: CPU29: found redistributor 10d region 0:0x00008010803a0000
CPU29: using LPI pending table @0x0000001fcde90000
CPU29: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU30
GICv3: CPU30: found redistributor 10e region 0:0x00008010803c0000
CPU30: using LPI pending table @0x0000001fcdeb0000
CPU30: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU31
GICv3: CPU31: found redistributor 10f region 0:0x00008010803e0000
CPU31: using LPI pending table @0x0000001fcdef0000
CPU31: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU32
GICv3: CPU32: found redistributor 200 region 0:0x0000801080400000
CPU32: using LPI pending table @0x0000001fcdf10000
CPU32: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU33
GICv3: CPU33: found redistributor 201 region 0:0x0000801080420000
CPU33: using LPI pending table @0x0000001fcdf40000
CPU33: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU34
GICv3: CPU34: found redistributor 202 region 0:0x0000801080440000
CPU34: using LPI pending table @0x0000001fcdf50000
CPU34: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU35
GICv3: CPU35: found redistributor 203 region 0:0x0000801080460000
CPU35: using LPI pending table @0x0000001fcdf70000
CPU35: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU36
GICv3: CPU36: found redistributor 204 region 0:0x0000801080480000
CPU36: using LPI pending table @0x0000001fce010000
CPU36: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU37
GICv3: CPU37: found redistributor 205 region 0:0x00008010804a0000
CPU37: using LPI pending table @0x0000001fce060000
CPU37: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU38
GICv3: CPU38: found redistributor 206 region 0:0x00008010804c0000
CPU38: using LPI pending table @0x0000001fce070000
CPU38: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU39
GICv3: CPU39: found redistributor 207 region 0:0x00008010804e0000
CPU39: using LPI pending table @0x0000001fce110000
CPU39: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU40
GICv3: CPU40: found redistributor 208 region 0:0x0000801080500000
CPU40: using LPI pending table @0x0000001fce120000
CPU40: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU41
GICv3: CPU41: found redistributor 209 region 0:0x0000801080520000
CPU41: using LPI pending table @0x0000001fce140000
CPU41: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU42
GICv3: CPU42: found redistributor 20a region 0:0x0000801080540000
CPU42: using LPI pending table @0x0000001fce160000
CPU42: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU43
GICv3: CPU43: found redistributor 20b region 0:0x0000801080560000
CPU43: using LPI pending table @0x0000001fce170000
CPU43: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU44
GICv3: CPU44: found redistributor 20c region 0:0x0000801080580000
CPU44: using LPI pending table @0x0000001fce230000
CPU44: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU45
GICv3: CPU45: found redistributor 20d region 0:0x00008010805a0000
CPU45: using LPI pending table @0x0000001fce240000
CPU45: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU46
GICv3: CPU46: found redistributor 20e region 0:0x00008010805c0000
CPU46: using LPI pending table @0x0000001fce270000
CPU46: Booted secondary processor [431f0a10]
Detected VIPT I-cache on CPU47
GICv3: CPU47: found redistributor 20f region 0:0x00008010805e0000
CPU47: using LPI pending table @0x0000001fce290000
CPU47: Booted secondary processor [431f0a10]
Brought up 48 CPUs
SMP: Total of 48 processors activated.
CPU features: detected feature: GIC system register CPU interface
CPU features: detected feature: Software prefetching using PRFM
CPU: All CPU(s) started at EL2
alternatives: patching kernel code
devtmpfs: initialized
SMBIOS 3.0.0 present.
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, 
max_idle_ns: 7645041785100000 ns
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
cpuidle: using governor ladder
cpuidle: using governor menu
vdso: 2 pages (1 code @ ffff000008900000, 1 data @ ffff000008cb0000)
hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
DMA: preallocated 256 KiB pool for atomic allocations
ACPI: bus type PCI registered
Serial: AMBA PL011 UART driver
HugeTLB registered 2 MB page size, pre-allocated 0 pages
HugeTLB registered 512 MB page size, pre-allocated 0 pages
ACPI: Added _OSI(Module Device)
ACPI: Added _OSI(Processor Device)
ACPI: Added _OSI(3.0 _SCP Extensions)
ACPI: Added _OSI(Processor Aggregator Device)
ACPI: Interpreter enabled
ACPI: Using GIC for interrupt routing
ACPI: MCFG table detected, 4 entries
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:00: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:00: ECAM area [mem 0x848000000000-0x848001ffffff] reserved 
by THRX0001:00
acpi PNP0A08:00: ECAM at [mem 0x848000000000-0x848001ffffff] for [bus 00-1f]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x801000000000-0x807fffffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x838000000000-0x841fffffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x846000000000-0x847fffffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x868000000000-0x87e023ffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x87e026000000-0x87e0bfffffff 
window]
pci_bus 0000:00: root bus resource [mem 0x87e0c6000000-0x87ffffffffff 
window]
pci_bus 0000:00: root bus resource [bus 00-1f]
pci 0000:00:09.0: VF(n) BAR0 space: [mem 0x840000800000-0x8400008fffff 
64bit] (contains BAR0 for 1 VFs)
pci 0000:01:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0000:04:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0000:00:01.0: PCI bridge to [bus 01]
pci 0000:00:14.0: PCI bridge to [bus 02]
pci 0000:00:15.0: PCI bridge to [bus 03]
pci 0000:00:16.0: PCI bridge to [bus 04]
ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus 00-1f])
acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:01: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:01: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:01: ECAM area [mem 0x849000000000-0x849001ffffff] reserved 
by THRX0001:01
acpi PNP0A08:01: ECAM at [mem 0x849000000000-0x849001ffffff] for [bus 00-1f]
PCI host bridge to bus 0001:00
pci_bus 0001:00: root bus resource [mem 0x810000000000-0x817fffffffff 
window]
pci_bus 0001:00: root bus resource [bus 00-1f]
ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 00-1f])
acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:02: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:02: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:02: ECAM area [mem 0x84a000000000-0x84a001ffffff] reserved 
by THRX0001:02
acpi PNP0A08:02: ECAM at [mem 0x84a000000000-0x84a001ffffff] for [bus 00-1f]
PCI host bridge to bus 0002:00
pci_bus 0002:00: root bus resource [mem 0x842000000000-0x843fffffffff 
window]
pci_bus 0002:00: root bus resource [bus 00-1f]
pci 0002:01:00.0: VF(n) BAR0 space: [mem 0x8430a0000000-0x8430afffffff 
64bit] (contains BAR0 for 128 VFs)
pci 0002:01:00.0: VF(n) BAR4 space: [mem 0x8430e0000000-0x8430efffffff 
64bit] (contains BAR4 for 128 VFs)
pci 0002:01:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0002:00:02.0: PCI bridge to [bus 01]
ACPI: PCI Root Bridge [PCI3] (domain 0003 [bus 00-1f])
acpi PNP0A08:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:03: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:03: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:03: ECAM area [mem 0x84b000000000-0x84b001ffffff] reserved 
by THRX0001:03
acpi PNP0A08:03: ECAM at [mem 0x84b000000000-0x84b001ffffff] for [bus 00-1f]
PCI host bridge to bus 0003:00
pci_bus 0003:00: root bus resource [mem 0x818000000000-0x81ffffffffff 
window]
pci_bus 0003:00: root bus resource [bus 00-1f]
ACPI: PCI Root Bridge [PEM0] (domain 0004 [bus 1f-57])
acpi PNP0A08:04: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:04: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:04: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:04: MCFG quirk: ECAM at [mem 0x88001f000000-0x880057ffffff] 
for [bus 1f-57] with thunder_pem_ecam_ops
acpi PNP0A08:04: ECAM area [mem 0x88001f000000-0x880057ffffff] reserved 
by THRX0002:00
acpi PNP0A08:04: ECAM at [mem 0x88001f000000-0x880057ffffff] for [bus 1f-57]
Remapped I/O 0x0000883000000000 to [io  0x0000-0xffff window]
PCI host bridge to bus 0004:1f
pci_bus 0004:1f: root bus resource [io  0x0000-0xffff window]
pci_bus 0004:1f: root bus resource [mem 0x881010000000-0x881fffffffff 
window] (bus address [0x10000000-0xfffffffff])
pci_bus 0004:1f: root bus resource [mem 0x882000000000-0x882fffffffff 
pref window] (bus address [0x1000000000-0x1fffffffff])
pci_bus 0004:1f: root bus resource [mem 0x87e0c0000000-0x87e0c0ffffff 
window]
pci_bus 0004:1f: root bus resource [bus 1f-57]
pci 0004:1f:00.0: Primary bus is hard wired to 0
pci 0004:20:00.0: can't set Max Payload Size to 256; if necessary, use 
"pci=pcie_bus_safe" and report a bug
pci 0004:20:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable 
it with 'pcie_aspm=force'
pci 0004:1f:00.0: Primary bus is hard wired to 0
pci 0004:1f:00.0: BAR 14: assigned [mem 0x881010000000-0x8810117fffff]
pci 0004:1f:00.0: BAR 6: assigned [mem 0x881011800000-0x88101180ffff pref]
pci 0004:1f:00.0: BAR 13: assigned [io  0x1000-0x1fff]
pci 0004:20:00.0: BAR 14: assigned [mem 0x881010000000-0x8810117fffff]
pci 0004:20:00.0: BAR 13: assigned [io  0x1000-0x1fff]
pci 0004:21:00.0: BAR 0: assigned [mem 0x881010000000-0x881010ffffff]
pci 0004:21:00.0: BAR 1: assigned [mem 0x881011000000-0x88101101ffff]
pci 0004:21:00.0: BAR 2: assigned [io  0x1000-0x107f]
pci 0004:20:00.0: PCI bridge to [bus 21]
pci 0004:20:00.0:   bridge window [io  0x1000-0x1fff]
pci 0004:20:00.0:   bridge window [mem 0x881010000000-0x8810117fffff]
pci 0004:1f:00.0: PCI bridge to [bus 20-57]
pci 0004:1f:00.0:   bridge window [io  0x1000-0x1fff]
pci 0004:1f:00.0:   bridge window [mem 0x881010000000-0x8810117fffff]
ACPI: PCI Interrupt Link [LN0A] (IRQs *48)
ACPI: PCI Interrupt Link [LN0B] (IRQs *49)
ACPI: PCI Interrupt Link [LN0C] (IRQs *50)
ACPI: PCI Interrupt Link [LN0D] (IRQs *51)
ACPI: PCI Root Bridge [PEM2] (domain 0006 [bus 8f-c7])
acpi PNP0A08:06: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments 
MSI]
acpi PNP0A08:06: _OSC: platform does not support [PCIeHotplug PME AER]
acpi PNP0A08:06: _OSC: OS now controls [PCIeCapability]
acpi PNP0A08:06: MCFG quirk: ECAM at [mem 0x88808f000000-0x8880c7ffffff] 
for [bus 8f-c7] with thunder_pem_ecam_ops
acpi PNP0A08:06: ECAM area [mem 0x88808f000000-0x8880c7ffffff] reserved 
by THRX0002:02
acpi PNP0A08:06: ECAM at [mem 0x88808f000000-0x8880c7ffffff] for [bus 8f-c7]
Remapped I/O 0x000088b000020000 to [io  0x10000-0x1ffff window]
PCI host bridge to bus 0006:8f
pci_bus 0006:8f: root bus resource [io  0x10000-0x1ffff window] (bus 
address [0x20000-0x2ffff])
pci_bus 0006:8f: root bus resource [mem 0x889010000000-0x889fffffffff 
window] (bus address [0x10000000-0xfffffffff])
pci_bus 0006:8f: root bus resource [mem 0x88a000000000-0x88afffffffff 
pref window] (bus address [0x1000000000-0x1fffffffff])
pci_bus 0006:8f: root bus resource [mem 0x87e0c2000000-0x87e0c2ffffff 
window]
pci_bus 0006:8f: root bus resource [bus 8f-c7]
pci 0006:8f:00.0: Primary bus is hard wired to 0
pci 0006:90:00.0: Max Payload Size set to 256 (was 128, max 512)
pci 0006:90:00.0: VF(n) BAR0 space: [mem 0x00000000-0x003fffff 64bit] 
(contains BAR0 for 64 VFs)
pci 0006:90:00.0: VF(n) BAR3 space: [mem 0x00000000-0x003fffff 64bit] 
(contains BAR3 for 64 VFs)
pci 0006:8f:00.0: Primary bus is hard wired to 0
pci 0006:8f:00.0: BAR 14: assigned [mem 0x889010000000-0x8890107fffff]
pci 0006:8f:00.0: BAR 15: assigned [mem 0x88a000000000-0x88a0000fffff 
64bit pref]
pci 0006:8f:00.0: BAR 6: assigned [mem 0x889010800000-0x88901080ffff pref]
pci 0006:8f:00.0: BAR 13: assigned [io  0x10000-0x10fff]
pci 0006:90:00.0: BAR 0: assigned [mem 0x88a000000000-0x88a00007ffff 
64bit pref]
pci 0006:90:00.0: BAR 7: assigned [mem 0x889010000000-0x8890103fffff 64bit]
pci 0006:90:00.0: BAR 10: assigned [mem 0x889010400000-0x8890107fffff 64bit]
pci 0006:90:00.0: BAR 4: assigned [mem 0x88a000080000-0x88a000083fff 
64bit pref]
pci 0006:90:00.0: BAR 2: assigned [io  0x10000-0x1001f]
pci 0006:8f:00.0: PCI bridge to [bus 90-c7]
pci 0006:8f:00.0:   bridge window [io  0x10000-0x10fff]
pci 0006:8f:00.0:   bridge window [mem 0x889010000000-0x8890107fffff]
pci 0006:8f:00.0:   bridge window [mem 0x88a000000000-0x88a0000fffff 
64bit pref]
ACPI: PCI Interrupt Link [LN0A] (IRQs *56)
ACPI: PCI Interrupt Link [LN0B] (IRQs *57)
ACPI: PCI Interrupt Link [LN0C] (IRQs *58)
ACPI: PCI Interrupt Link [LN0D] (IRQs *59)
ARMH0011:00: ttyAMA0 at MMIO 0x87e024000000 (irq = 10, base_baud = 0) is 
a SBSA
console [ttyAMA0] enabled
console [ttyAMA0] enabled
bootconsole [pl11] disabled
bootconsole [pl11] disabled
ARMH0011:01: ttyAMA1 at MMIO 0x87e025000000 (irq = 11, base_baud = 0) is 
a SBSA
vgaarb: device added: PCI:0004:21:00.0,decodes=io+mem,owns=none,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0004:21:00.0
SCSI subsystem initialized
ACPI: bus type USB registered
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti 
<giometti@linux.it>
PTP clock support registered
Registered efivars operations
clocksource: Switched to clocksource arch_sys_counter
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
pnp: PnP ACPI init
system 00:00: [mem 0x848000000000-0x848001ffffff] could not be reserved
system 00:01: [mem 0x849000000000-0x849001ffffff] could not be reserved
system 00:02: [mem 0x84a000000000-0x84a001ffffff] could not be reserved
system 00:03: [mem 0x84b000000000-0x84b001ffffff] could not be reserved
system 00:04: [mem 0x87e0c0000000-0x87e0c0ffffff] could not be reserved
system 00:04: [mem 0x88001f000000-0x880057ffffff] could not be reserved
system 00:05: [mem 0x87e0c2000000-0x87e0c2ffffff] could not be reserved
system 00:05: [mem 0x88808f000000-0x8880c7ffffff] could not be reserved
pnp: PnP ACPI: found 6 devices
NET: Registered protocol family 2
TCP established hash table entries: 524288 (order: 6, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 4, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
UDP hash table entries: 65536 (order: 5, 2097152 bytes)
UDP-Lite hash table entries: 65536 (order: 5, 2097152 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
kvm [1]: 16-bit VMID
kvm [1]: IDMAP page: 1cca000
kvm [1]: HYP VA range: 800000000000:ffffffffffff
kvm [1]: Hyp mode initialized successfully
kvm [1]: GICv3: no GICV resource entry
kvm [1]: disabling GICv2 emulation
kvm [1]: GIC system register CPU interface enabled
kvm [1]: vgic interrupt IRQ1
kvm [1]: Invalid trigger for IRQ4, assuming level low
kvm [1]: virtual timer IRQ4
futex hash table entries: 16384 (order: 5, 2097152 bytes)
audit: initializing netlink subsys (disabled)
audit: type=2000 audit(6.980:1): initialized
workingset: timestamp_bits=44 max_order=21 bucket_order=0
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
fuse init (API version 7.26)
9p: Installing v9fs 9p2000 file system support
io scheduler noop registered
io scheduler cfq registered (default)
ACPI: PCI Interrupt Link [LN0A] enabled at IRQ 48
pcieport 0004:1f:00.0: enabling device (0506 -> 0507)
ACPI: PCI Interrupt Link [LN0A] enabled at IRQ 56
pcieport 0006:8f:00.0: enabling device (0506 -> 0507)
acpi-ged ACPI0013:00: GED listening GSI 32 @ IRQ 12
input: Power Button as 
/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
msm_serial: driver initialized
Failed to find cpu0 device node
Unable to detect cache hierarchy from DT for CPU 0
loop: module loaded
ahci 0001:00:08.0: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
ahci 0001:00:08.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs 
pio slum part ccc apst
ahci 0001:00:08.0: port 0 is not capable of FBS
scsi host0: ahci
ata1: SATA max UDMA/133 abar m2097152@0x814000000000 port 0x814000000100 
irq 14
ahci 0001:00:09.0: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
ahci 0001:00:09.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs 
pio slum part ccc apst
ahci 0001:00:09.0: port 0 is not capable of FBS
scsi host1: ahci
ata2: SATA max UDMA/133 abar m2097152@0x815000000000 port 0x815000000100 
irq 15
ahci 0001:00:0a.0: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
ahci 0001:00:0a.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs 
pio slum part ccc apst
ahci 0001:00:0a.0: port 0 is not capable of FBS
scsi host2: ahci
ata3: SATA max UDMA/133 abar m2097152@0x816000000000 port 0x816000000100 
irq 16
ahci 0001:00:0b.0: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
ahci 0001:00:0b.0: flags: 64bit ncq sntf ilck pm led clo only pmp fbs 
pio slum part ccc apst
ahci 0001:00:0b.0: port 0 is not capable of FBS
scsi host3: ahci
ata4: SATA max UDMA/133 abar m2097152@0x817000000000 port 0x817000000100 
irq 17
libphy: Fixed MDIO Bus: probed
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
thunder-xcv, ver 1.0
thunder-BGX, ver 1.0
thunder-BGX 0000:01:10.0: BGX0 QLM mode: XLAUI
thunder-BGX 0000:01:10.0: MAC address set to: 40:8d:5c:ba:b8:ea
thunder-BGX 0000:01:10.1: BGX1 QLM mode: XFI
thunder-BGX 0000:01:10.1: MAC address set to: 40:8d:5c:ba:b8:eb
thunder-BGX 0000:01:10.1: MAC address set to: 40:8d:5c:ba:b8:ec
thunder-nic, ver 1.0
thunder-nic 0002:01:00.0: SRIOV enabled, number of VF available 36
thunder-nicvf, ver 1.0
thunder-nicvf 0002:01:00.1: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:00.2: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:00.3: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:00.4: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:00.5: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:00.6: enabling device (0004 -> 0006)
ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata1.00: ATA-8: WDC WD5003ABYZ-011FA0, 01.01S03, max UDMA/133
ata1.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      WDC WD5003ABYZ-0 1S03 PQ: 0 ANSI: 5
ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata2.00: ATA-9: WDC WD5000AZLX-00CL5A0, 01.01A01, max UDMA/133
ata2.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
ata2.00: configured for UDMA/133
sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/466 GiB)
scsi 1:0:0:0: Direct-Access     ATA      WDC WD5000AZLX-0 1A01 PQ: 0 ANSI: 5
ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata3.00: ATA-8: WDC WD5003ABYZ-011FA0, 01.01S03, max UDMA/133
ata3.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
ata3.00: configured for UDMA/133
thunder-nicvf 0002:01:00.7: enabling device (0004 -> 0006)
ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
sd 0:0:0:0: [sda] Write Protect is off
ata4.00: ATA-8: OCZ-VERTEX3, 2.25, max UDMA/133
ata4.00: 234441648 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
thunder-nicvf 0002:01:01.0: enabling device (0004 -> 0006)
ata4.00: configured for UDMA/133
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
sd 1:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/466 GiB)
scsi 2:0:0:0: Direct-Access     ATA      WDC WD5003ABYZ-0 1S03 PQ: 0 ANSI: 5
sd 1:0:0:0: [sdb] 4096-byte physical blocks
thunder-nicvf 0002:01:01.1: enabling device (0004 -> 0006)
sd 1:0:0:0: [sdb] Write Protect is off
  sda: sda1 sda2 sda3
sd 0:0:0:0: [sda] Attached SCSI disk
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
sd 2:0:0:0: [sdc] 976773168 512-byte logical blocks: (500 GB/466 GiB)
scsi 3:0:0:0: Direct-Access     ATA      OCZ-VERTEX3      2.25 PQ: 0 ANSI: 5
thunder-nicvf 0002:01:01.2: enabling device (0004 -> 0006)
sd 1:0:0:0: [sdb] Attached SCSI disk
sd 2:0:0:0: [sdc] Write Protect is off
sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
thunder-nicvf 0002:01:01.3: enabling device (0004 -> 0006)
sd 3:0:0:0: [sdd] 234441648 512-byte logical blocks: (120 GB/112 GiB)
sd 3:0:0:0: [sdd] Write Protect is off
sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't 
support DPO or FUA
  sdc: sdc1 sdc2 sdc3
  sdd: sdd1
sd 3:0:0:0: [sdd] Attached SCSI disk
sd 2:0:0:0: [sdc] Attached SCSI disk
thunder-nicvf 0002:01:01.4: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:01.5: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:01.6: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:01.7: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.0: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.1: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.2: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.3: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.4: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.5: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.6: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:02.7: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.0: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.1: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.2: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.3: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.4: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.5: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.6: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:03.7: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.0: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.1: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.2: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.3: enabling device (0004 -> 0006)
thunder-nicvf 0002:01:04.4: enabling device (0004 -> 0006)
dmfe: Davicom DM9xxx net driver, version 1.36.4 (2002-01-17)
v1.01-e (2.4 port) Sep-11-2006  Donald Becker <becker@scyld.com>
   http://www.scyld.com/network/drivers.html
uli526x: ULi M5261/M5263 net driver, version 0.9.3 (2005-7-29)
e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
e100: Copyright(c) 1999-2006 Intel Corporation
e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
e1000: Copyright (c) 1999-2006 Intel Corporation.
e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
igb: Copyright (c) 2007-2014 Intel Corporation.
igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.4.0-k
ixgbe: Copyright (c) 1999-2016 Intel Corporation.
ixgbe 0006:90:00.0: enabling device (0000 -> 0002)
ixgbe 0006:90:00.0: Multiqueue Enabled: Rx Queue count = 48, Tx Queue 
count = 48
ixgbe 0006:90:00.0: PCI Express bandwidth of 32GT/s available
ixgbe 0006:90:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
ixgbe 0006:90:00.0: MAC: 2, PHY: 18, SFP+: 5, PBA No: E68787-006
ixgbe 0006:90:00.0: 90:e2:ba:16:d1:86
ixgbe 0006:90:00.0: Intel(R) 10 Gigabit Network Connection
ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver 
- version 3.2.2-k
ixgbevf: Copyright (c) 2009 - 2015 Intel Corporation.
i40e: Intel(R) Ethernet Connection XL710 Network Driver - version 1.6.16-k
i40e: Copyright (c) 2013 - 2014 Intel Corporation.
ixgb: Intel(R) PRO/10GbE Network Driver - version 1.0.135-k2-NAPI
ixgb: Copyright (c) 1999-2008 Intel Corporation.
i40evf: Intel(R) 40-10 Gigabit Virtual Function Network Driver - version 
1.6.16-k
Copyright (c) 2013 - 2015 Intel Corporation.
Intel(R) Ethernet Switch Host Interface Driver - version 0.21.2-k
Copyright (c) 2013 - 2016 Intel Corporation.
sky2: driver version 1.30
usbcore: registered new interface driver asix
usbcore: registered new interface driver ax88179_178a
usbcore: registered new interface driver cdc_ether
usbcore: registered new interface driver net1080
usbcore: registered new interface driver cdc_subset
usbcore: registered new interface driver zaurus
usbcore: registered new interface driver cdc_ncm
VFIO - User Level meta-driver version: 0.3
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-platform: EHCI generic platform driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
ohci-platform: OHCI generic platform driver
uhci_hcd: USB Universal Host Controller Interface driver
xhci_hcd 0000:00:10.0: xHCI Host Controller
xhci_hcd 0000:00:10.0: new USB bus registered, assigned bus number 1
xhci_hcd 0000:00:10.0: hcc params 0x0220f665 hci version 0x100 quirks 
0x00000010
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 1 port detected
xhci_hcd 0000:00:10.0: xHCI Host Controller
xhci_hcd 0000:00:10.0: new USB bus registered, assigned bus number 2
usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 1 port detected
xhci_hcd 0000:00:11.0: xHCI Host Controller
xhci_hcd 0000:00:11.0: new USB bus registered, assigned bus number 3
xhci_hcd 0000:00:11.0: hcc params 0x0220f665 hci version 0x100 quirks 
0x00000010
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 1 port detected
xhci_hcd 0000:00:11.0: xHCI Host Controller
xhci_hcd 0000:00:11.0: new USB bus registered, assigned bus number 4
usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 1 port detected
usbcore: registered new interface driver usb-storage
mousedev: PS/2 mouse device common for all mice
device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: 
dm-devel@redhat.com
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
sdhci-pltfm: SDHCI platform and OF driver helper
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
Netfilter messages via NETLINK v0.30.
nfnl_acct: registering with nfnetlink.
nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
ip_set: protocol 6
IPVS: Registered protocols ()
IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
IPVS: Creating netns size=1336 id=0
IPVS: ipvs loaded.
ip_tables: (C) 2000-2006 Netfilter Core Team
arp_tables: arp_tables: (C) 2002 David S. Miller
NET: Registered protocol family 17
bridge: filtering via arp/ip/ip6tables is no longer available by 
default. Update your scripts to load br_netfilter if you need this.
9pnet: Installing 9P2000 support
Key type dns_resolver registered
registered taskstats version 1
usb 3-1: new high-speed USB device number 2 using xhci_hcd
hctosys: unable to open rtc device (rtc0)
EXT4-fs (sda2): couldn't mount as ext3 due to feature incompatibilities
hub 3-1:1.0: USB hub found
random: fast init done
hub 3-1:1.0: 4 ports detected
usb 3-1: selecting invalid altsetting 1
hub 3-1:1.0: Using single TT (err -22)
EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) on device 8:2.
devtmpfs: mounted
Freeing unused kernel memory: 640K (ffff800002000000 - ffff8000020a0000)
usb 4-1: new SuperSpeed USB device number 2 using xhci_hcd
hub 4-1:1.0: USB hub found
hub 4-1:1.0: 4 ports detected
systemd[1]: System time before build time, advancing clock.
usb 4-1.3: new SuperSpeed USB device number 3 using xhci_hcd
systemd[1]: systemd 229 running in system mode. (+PAM +AUDIT +SELINUX 
+IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS 
+ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN)
systemd[1]: Detected architecture arm64.

Welcome to Ubuntu 16.04.1 LTS!

root@ubuntu:~# cat /proc/iomem
01400000-fffeffff : System RAM
   01480000-01ffffff : Kernel code
   020a0000-0225ffff : Kernel data
ffff0000-ffffffff : reserved
100000000-1ffaa4ffff : System RAM
1ffaa50000-1ffaa7ffff : reserved
1ffaa80000-1fffa3ffff : System RAM
1fffa40000-1fffa8ffff : reserved
1fffa90000-1fffffffff : System RAM
801000000000-807fffffffff : PCI Bus 0000:00
810000000000-817fffffffff : PCI Bus 0001:00
   814000000000-8140001fffff : 0001:00:08.0
     814000000000-8140001fffff : ahci
   814000200000-8140002fffff : 0001:00:08.0
     814000200000-8140002fffff : ahci
   815000000000-8150001fffff : 0001:00:09.0
     815000000000-8150001fffff : ahci
   815000200000-8150002fffff : 0001:00:09.0
     815000200000-8150002fffff : ahci
   816000000000-8160001fffff : 0001:00:0a.0
     816000000000-8160001fffff : ahci
   816000200000-8160002fffff : 0001:00:0a.0
     816000200000-8160002fffff : ahci
   817000000000-8170001fffff : 0001:00:0b.0
     817000000000-8170001fffff : ahci
   817000200000-8170002fffff : 0001:00:0b.0
     817000200000-8170002fffff : ahci
818000000000-81ffffffffff : PCI Bus 0003:00
838000000000-841fffffffff : PCI Bus 0000:00
   838000000000-8380003fffff : 0000:03:00.0
   838000f00000-838000ffffff : 0000:03:00.0
   840000800000-8400008fffff : 0000:00:09.0
842000000000-843fffffffff : PCI Bus 0002:00
   842000000000-84200000ffff : 0002:00:03.0
   842000f00000-842000ffffff : 0002:00:03.0
   842040000000-84207fffffff : 0002:00:03.0
   843000000000-84303fffffff : 0002:01:00.0
     843000000000-84303fffffff : thunder-nic
   843060000000-8430600fffff : 0002:01:00.0
     843060000000-8430600fffff : thunder-nic
   8430a0000000-8430afffffff : 0002:01:00.0
     8430a0000000-8430a01fffff : 0002:01:00.1
       8430a0000000-8430a01fffff : thunder-nicvf
     8430a0200000-8430a03fffff : 0002:01:00.2
       8430a0200000-8430a03fffff : thunder-nicvf
     8430a0400000-8430a05fffff : 0002:01:00.3
       8430a0400000-8430a05fffff : thunder-nicvf
     8430a0600000-8430a07fffff : 0002:01:00.4
       8430a0600000-8430a07fffff : thunder-nicvf
     8430a0800000-8430a09fffff : 0002:01:00.5
       8430a0800000-8430a09fffff : thunder-nicvf
     8430a0a00000-8430a0bfffff : 0002:01:00.6
       8430a0a00000-8430a0bfffff : thunder-nicvf
     8430a0c00000-8430a0dfffff : 0002:01:00.7
       8430a0c00000-8430a0dfffff : thunder-nicvf
     8430a0e00000-8430a0ffffff : 0002:01:01.0
       8430a0e00000-8430a0ffffff : thunder-nicvf
     8430a1000000-8430a11fffff : 0002:01:01.1
       8430a1000000-8430a11fffff : thunder-nicvf
     8430a1200000-8430a13fffff : 0002:01:01.2
       8430a1200000-8430a13fffff : thunder-nicvf
     8430a1400000-8430a15fffff : 0002:01:01.3
       8430a1400000-8430a15fffff : thunder-nicvf
     8430a1600000-8430a17fffff : 0002:01:01.4
       8430a1600000-8430a17fffff : thunder-nicvf
     8430a1800000-8430a19fffff : 0002:01:01.5
       8430a1800000-8430a19fffff : thunder-nicvf
     8430a1a00000-8430a1bfffff : 0002:01:01.6
       8430a1a00000-8430a1bfffff : thunder-nicvf
     8430a1c00000-8430a1dfffff : 0002:01:01.7
       8430a1c00000-8430a1dfffff : thunder-nicvf
     8430a1e00000-8430a1ffffff : 0002:01:02.0
       8430a1e00000-8430a1ffffff : thunder-nicvf
     8430a2000000-8430a21fffff : 0002:01:02.1
       8430a2000000-8430a21fffff : thunder-nicvf
     8430a2200000-8430a23fffff : 0002:01:02.2
       8430a2200000-8430a23fffff : thunder-nicvf
     8430a2400000-8430a25fffff : 0002:01:02.3
       8430a2400000-8430a25fffff : thunder-nicvf
     8430a2600000-8430a27fffff : 0002:01:02.4
       8430a2600000-8430a27fffff : thunder-nicvf
     8430a2800000-8430a29fffff : 0002:01:02.5
       8430a2800000-8430a29fffff : thunder-nicvf
     8430a2a00000-8430a2bfffff : 0002:01:02.6
       8430a2a00000-8430a2bfffff : thunder-nicvf
     8430a2c00000-8430a2dfffff : 0002:01:02.7
       8430a2c00000-8430a2dfffff : thunder-nicvf
     8430a2e00000-8430a2ffffff : 0002:01:03.0
       8430a2e00000-8430a2ffffff : thunder-nicvf
     8430a3000000-8430a31fffff : 0002:01:03.1
       8430a3000000-8430a31fffff : thunder-nicvf
     8430a3200000-8430a33fffff : 0002:01:03.2
       8430a3200000-8430a33fffff : thunder-nicvf
     8430a3400000-8430a35fffff : 0002:01:03.3
       8430a3400000-8430a35fffff : thunder-nicvf
     8430a3600000-8430a37fffff : 0002:01:03.4
       8430a3600000-8430a37fffff : thunder-nicvf
     8430a3800000-8430a39fffff : 0002:01:03.5
       8430a3800000-8430a39fffff : thunder-nicvf
     8430a3a00000-8430a3bfffff : 0002:01:03.6
       8430a3a00000-8430a3bfffff : thunder-nicvf
     8430a3c00000-8430a3dfffff : 0002:01:03.7
       8430a3c00000-8430a3dfffff : thunder-nicvf
     8430a3e00000-8430a3ffffff : 0002:01:04.0
       8430a3e00000-8430a3ffffff : thunder-nicvf
     8430a4000000-8430a41fffff : 0002:01:04.1
       8430a4000000-8430a41fffff : thunder-nicvf
     8430a4200000-8430a43fffff : 0002:01:04.2
       8430a4200000-8430a43fffff : thunder-nicvf
     8430a4400000-8430a45fffff : 0002:01:04.3
       8430a4400000-8430a45fffff : thunder-nicvf
     8430a4600000-8430a47fffff : 0002:01:04.4
       8430a4600000-8430a47fffff : thunder-nicvf
   8430e0000000-8430efffffff : 0002:01:00.0
     8430e0000000-8430e01fffff : 0002:01:00.1
       8430e0000000-8430e01fffff : thunder-nicvf
     8430e0200000-8430e03fffff : 0002:01:00.2
       8430e0200000-8430e03fffff : thunder-nicvf
     8430e0400000-8430e05fffff : 0002:01:00.3
       8430e0400000-8430e05fffff : thunder-nicvf
     8430e0600000-8430e07fffff : 0002:01:00.4
       8430e0600000-8430e07fffff : thunder-nicvf
     8430e0800000-8430e09fffff : 0002:01:00.5
       8430e0800000-8430e09fffff : thunder-nicvf
     8430e0a00000-8430e0bfffff : 0002:01:00.6
       8430e0a00000-8430e0bfffff : thunder-nicvf
     8430e0c00000-8430e0dfffff : 0002:01:00.7
       8430e0c00000-8430e0dfffff : thunder-nicvf
     8430e0e00000-8430e0ffffff : 0002:01:01.0
       8430e0e00000-8430e0ffffff : thunder-nicvf
     8430e1000000-8430e11fffff : 0002:01:01.1
       8430e1000000-8430e11fffff : thunder-nicvf
     8430e1200000-8430e13fffff : 0002:01:01.2
       8430e1200000-8430e13fffff : thunder-nicvf
     8430e1400000-8430e15fffff : 0002:01:01.3
       8430e1400000-8430e15fffff : thunder-nicvf
     8430e1600000-8430e17fffff : 0002:01:01.4
       8430e1600000-8430e17fffff : thunder-nicvf
     8430e1800000-8430e19fffff : 0002:01:01.5
       8430e1800000-8430e19fffff : thunder-nicvf
     8430e1a00000-8430e1bfffff : 0002:01:01.6
       8430e1a00000-8430e1bfffff : thunder-nicvf
     8430e1c00000-8430e1dfffff : 0002:01:01.7
       8430e1c00000-8430e1dfffff : thunder-nicvf
     8430e1e00000-8430e1ffffff : 0002:01:02.0
       8430e1e00000-8430e1ffffff : thunder-nicvf
     8430e2000000-8430e21fffff : 0002:01:02.1
       8430e2000000-8430e21fffff : thunder-nicvf
     8430e2200000-8430e23fffff : 0002:01:02.2
       8430e2200000-8430e23fffff : thunder-nicvf
     8430e2400000-8430e25fffff : 0002:01:02.3
       8430e2400000-8430e25fffff : thunder-nicvf
     8430e2600000-8430e27fffff : 0002:01:02.4
       8430e2600000-8430e27fffff : thunder-nicvf
     8430e2800000-8430e29fffff : 0002:01:02.5
       8430e2800000-8430e29fffff : thunder-nicvf
     8430e2a00000-8430e2bfffff : 0002:01:02.6
       8430e2a00000-8430e2bfffff : thunder-nicvf
     8430e2c00000-8430e2dfffff : 0002:01:02.7
       8430e2c00000-8430e2dfffff : thunder-nicvf
     8430e2e00000-8430e2ffffff : 0002:01:03.0
       8430e2e00000-8430e2ffffff : thunder-nicvf
     8430e3000000-8430e31fffff : 0002:01:03.1
       8430e3000000-8430e31fffff : thunder-nicvf
     8430e3200000-8430e33fffff : 0002:01:03.2
       8430e3200000-8430e33fffff : thunder-nicvf
     8430e3400000-8430e35fffff : 0002:01:03.3
       8430e3400000-8430e35fffff : thunder-nicvf
     8430e3600000-8430e37fffff : 0002:01:03.4
       8430e3600000-8430e37fffff : thunder-nicvf
     8430e3800000-8430e39fffff : 0002:01:03.5
       8430e3800000-8430e39fffff : thunder-nicvf
     8430e3a00000-8430e3bfffff : 0002:01:03.6
       8430e3a00000-8430e3bfffff : thunder-nicvf
     8430e3c00000-8430e3dfffff : 0002:01:03.7
       8430e3c00000-8430e3dfffff : thunder-nicvf
     8430e3e00000-8430e3ffffff : 0002:01:04.0
       8430e3e00000-8430e3ffffff : thunder-nicvf
     8430e4000000-8430e41fffff : 0002:01:04.1
       8430e4000000-8430e41fffff : thunder-nicvf
     8430e4200000-8430e43fffff : 0002:01:04.2
       8430e4200000-8430e43fffff : thunder-nicvf
     8430e4400000-8430e45fffff : 0002:01:04.3
       8430e4400000-8430e45fffff : thunder-nicvf
     8430e4600000-8430e47fffff : 0002:01:04.4
       8430e4600000-8430e47fffff : thunder-nicvf
846000000000-847fffffffff : PCI Bus 0000:00
   846000000000-8467ffffffff : 0000:04:00.0
   846a00000000-846a000fffff : 0000:04:00.0
848000000000-848001ffffff : PCI ECAM
849000000000-849001ffffff : PCI ECAM
84a000000000-84a001ffffff : PCI ECAM
84b000000000-84b001ffffff : PCI ECAM
868000000000-87e023ffffff : PCI Bus 0000:00
   868000000000-8680001fffff : 0000:00:10.0
     868000000000-8680001fffff : xhci-hcd
   868000200000-8680002fffff : 0000:00:10.0
   869000000000-8690001fffff : 0000:00:11.0
     869000000000-8690001fffff : xhci-hcd
   869000200000-8690002fffff : 0000:00:11.0
   870000000000-8700007fffff : 0000:02:00.0
   870000f00000-870000ffffff : 0000:02:00.0
   87e005000000-87e0057fffff : 0000:01:01.3
     87e005000000-87e0057fffff : mdio_thunder
   87e006000000-87e0067fffff : 0000:01:00.1
   87e006f00000-87e006ffffff : 0000:01:00.1
   87e009000000-87e0097fffff : 0000:01:01.4
   87e009f00000-87e009ffffff : 0000:01:01.4
87e024000000-87e024000fff : ARMH0011:00
   87e024000000-87e024000fff : ARMH0011:00
87e025000000-87e025000fff : ARMH0011:01
   87e025000000-87e025000fff : ARMH0011:01
87e026000000-87e0bfffffff : PCI Bus 0000:00
   87e027000000-87e0277fffff : CAVA02A:00
   87e040000000-87e0400fffff : 0000:00:09.0
   87e050000000-87e0507fffff : 0000:01:06.0
   87e050f00000-87e050ffffff : 0000:01:06.0
   87e051000000-87e0517fffff : 0000:01:06.1
   87e051f00000-87e051ffffff : 0000:01:06.1
   87e052000000-87e0527fffff : 0000:01:06.2
   87e052f00000-87e052ffffff : 0000:01:06.2
   87e053000000-87e0537fffff : 0000:01:06.3
   87e053f00000-87e053ffffff : 0000:01:06.3
   87e054000000-87e0547fffff : 0000:01:06.4
   87e054f00000-87e054ffffff : 0000:01:06.4
   87e055000000-87e0557fffff : 0000:01:06.5
   87e055f00000-87e055ffffff : 0000:01:06.5
   87e056000000-87e0567fffff : 0000:01:06.6
   87e056f00000-87e056ffffff : 0000:01:06.6
   87e057000000-87e0577fffff : 0000:01:06.7
   87e057f00000-87e057ffffff : 0000:01:06.7
   87e058000000-87e0587fffff : 0000:01:07.0
   87e058f00000-87e058ffffff : 0000:01:07.0
   87e059000000-87e0597fffff : 0000:01:07.1
   87e059f00000-87e059ffffff : 0000:01:07.1
   87e05a000000-87e05a7fffff : 0000:01:07.2
   87e05af00000-87e05affffff : 0000:01:07.2
   87e05b000000-87e05b7fffff : 0000:01:07.3
   87e05bf00000-87e05bffffff : 0000:01:07.3
   87e05c000000-87e05c7fffff : 0000:01:07.4
   87e05cf00000-87e05cffffff : 0000:01:07.4
   87e05d000000-87e05d7fffff : 0000:01:07.5
   87e05df00000-87e05dffffff : 0000:01:07.5
   87e05e000000-87e05e7fffff : 0000:01:07.6
   87e05ef00000-87e05effffff : 0000:01:07.6
   87e05f000000-87e05f7fffff : 0000:01:07.7
   87e05ff00000-87e05fffffff : 0000:01:07.7
   87e088000000-87e0887fffff : 0000:01:0a.0
   87e088f00000-87e088ffffff : 0000:01:0a.0
   87e089000000-87e0897fffff : 0000:01:0a.1
   87e089f00000-87e089ffffff : 0000:01:0a.1
   87e08a000000-87e08a7fffff : 0000:01:0a.2
   87e08af00000-87e08affffff : 0000:01:0a.2
   87e08b000000-87e08b7fffff : 0000:01:0a.3
   87e08bf00000-87e08bffffff : 0000:01:0a.3
87e0c0000000-87e0c0ffffff : PCI Bus 0004:1f
   87e0c0f00000-87e0c0ffffff : 0004:1f:00.0
87e0c2000000-87e0c2ffffff : PCI Bus 0006:8f
   87e0c2f00000-87e0c2ffffff : 0006:8f:00.0
87e0c6000000-87ffffffffff : PCI Bus 0000:00
   87e0d0000000-87e0d07fffff : 0000:01:09.0
   87e0d0f00000-87e0d0ffffff : 0000:01:09.0
   87e0d1000000-87e0d17fffff : 0000:01:09.1
   87e0d1f00000-87e0d1ffffff : 0000:01:09.1
   87e0d2000000-87e0d27fffff : 0000:01:09.2
   87e0d2f00000-87e0d2ffffff : 0000:01:09.2
   87e0d3000000-87e0d37fffff : 0000:01:09.3
   87e0d3f00000-87e0d3ffffff : 0000:01:09.3
   87e0d4000000-87e0d47fffff : 0000:01:09.4
   87e0d4f00000-87e0d4ffffff : 0000:01:09.4
   87e0e0000000-87e0e03fffff : 0000:01:10.0
     87e0e0000000-87e0e03fffff : thunder-BGX
   87e0e0400000-87e0e07fffff : 0000:01:10.0
     87e0e0400000-87e0e07fffff : thunder-BGX
   87e0e1000000-87e0e13fffff : 0000:01:10.1
     87e0e1000000-87e0e13fffff : thunder-BGX
   87e0e1400000-87e0e17fffff : 0000:01:10.1
     87e0e1400000-87e0e17fffff : thunder-BGX
   87e0fc000000-87e0fc0fffff : 0000:01:00.0
   87e0fcf00000-87e0fcffffff : 0000:01:00.0
88001f000000-880057ffffff : PCI ECAM
881010000000-881fffffffff : PCI Bus 0004:1f
   881010000000-8810117fffff : PCI Bus 0004:20
     881010000000-8810117fffff : PCI Bus 0004:21
       881010000000-881010ffffff : 0004:21:00.0
       881011000000-88101101ffff : 0004:21:00.0
   881011800000-88101180ffff : 0004:1f:00.0
882000000000-882fffffffff : PCI Bus 0004:1f
88808f000000-8880c7ffffff : PCI ECAM
889010000000-889fffffffff : PCI Bus 0006:8f
   889010000000-8890107fffff : PCI Bus 0006:90
     889010000000-8890103fffff : 0006:90:00.0
     889010400000-8890107fffff : 0006:90:00.0
   889010800000-88901080ffff : 0006:8f:00.0
88a000000000-88afffffffff : PCI Bus 0006:8f
   88a000000000-88a0000fffff : PCI Bus 0006:90
     88a000000000-88a00007ffff : 0006:90:00.0
       88a000000000-88a00007ffff : ixgbe
     88a000080000-88a000083fff : 0006:90:00.0
       88a000080000-88a000083fff : ixgbe

Thanks,
Tomasz

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Bjorn Helgaas]

On Fri, Dec 02, 2016 at 03:20:28PM +0100, Tomasz Nowicki wrote:
> dmesg from ThunderX pass2.0 (1 socket board) + small fix:
> 
> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> -       THUNDER_PEM_QUIRK(2,  0),       /* off-chip devices */
> -       THUNDER_PEM_QUIRK(2,  1),       /* off-chip devices */
> +       THUNDER_PEM_QUIRK(2,  0UL),     /* off-chip devices */
> +       THUNDER_PEM_QUIRK(2,  1UL),     /* off-chip devices */

Thanks!  I folded this change into my branch (possibly to be updated
if Robert sends something better).

> ACPI: MCFG table detected, 4 entries
> ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
> acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM
> Segments MSI]
> acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME AER]
> acpi PNP0A08:00: _OSC: OS now controls [PCIeCapability]
> acpi PNP0A08:00: ECAM area [mem 0x848000000000-0x848001ffffff]
> reserved by THRX0001:00
> acpi PNP0A08:00: ECAM at [mem 0x848000000000-0x848001ffffff] for [bus 00-1f]

I guess we don't need MCFG quirks for these bridges, since I don't see
the "MCFG quirk" message?

> system 00:00: [mem 0x848000000000-0x848001ffffff] could not be reserved
> system 00:01: [mem 0x849000000000-0x849001ffffff] could not be reserved
> system 00:02: [mem 0x84a000000000-0x84a001ffffff] could not be reserved
> system 00:03: [mem 0x84b000000000-0x84b001ffffff] could not be reserved
> system 00:04: [mem 0x87e0c0000000-0x87e0c0ffffff] could not be reserved
> system 00:04: [mem 0x88001f000000-0x880057ffffff] could not be reserved
> system 00:05: [mem 0x87e0c2000000-0x87e0c2ffffff] could not be reserved
> system 00:05: [mem 0x88808f000000-0x8880c7ffffff] could not be reserved

Most of these match ECAM spaces, which is good.  00:04 and 00:05 each
have one ECAM space and one "other" space, which might be PEMx host
bridge registers?  That all seems good.

But I assume the other bridges (PCIx) also have register space in
addition to ECAM, and that should be reported also.

> root@ubuntu:~# cat /proc/iomem
> ...
> 838000000000-841fffffffff : PCI Bus 0000:00
>   838000000000-8380003fffff : 0000:03:00.0

Something's missing here: we should have a clue about how we got from
bus 00 to bus 03.  From your dmesg, 0000:00:15.0 is a bridge from bus
00 to bus 03, and its windows should appear here.  I'd expect
something like:

  838000000000-841fffffffff : PCI Bus 0000:00
    838000000000-838fffffffff : PCI Bus 0000:03  <- 00:15.0 window
      838000000000-8380003fffff : 0000:03:00.0

This window should be inserted by generic code, so I don't know how
this could be broken.  Your dmesg should also have something like
this:

  pci 0000:00:15.0: PCI bridge to [bus 03]
  pci 0000:00:15.0:   bridge window [mem 0x838000000000-0x838fffffffff]

I don't see that, maybe because this is actually a console log
collected without "ignore_loglevel"?  But that obviously doesn't
affect /proc/iomem, so I'm still puzzled about that.

> 87e024000000-87e024000fff : ARMH0011:00
>   87e024000000-87e024000fff : ARMH0011:00

This is interesting.  This must be a driver reserving these areas?
Normally a driver would use the driver name, not the device name.

Ideally, I think the core should reserve the region with the device
name, and the driver would request it using the driver name, like
this:

  843000000000-84303fffffff : 0002:01:00.0    <-- PCI core reservation
    843000000000-84303fffffff : thunder-nic   <-- driver request

The ACPI core doesn't actually do the reservation, so I assume the
ARMH0011:00 stuff is from the driver, and it's curious that it has the
same region twice.

> 87e026000000-87e0bfffffff : PCI Bus 0000:00
>   87e027000000-87e0277fffff : CAVA02A:00

This is interesting, too.  CAVA02A:00 looks like an ACPI device, but
apparently it consumes some of the space that we think is routed to
PCI bus 0000:00.  I think this happens on some x86 boxes, too, but it
is somewhat confusing.

Based on this, I don't see any problems with the ThunderX quirks.
I'd like to understand what's going on with the PCI-to-PCI bridge
windows in /proc/iomem, but I doubt that's related to your quirks.

Bjorn

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Tomasz Nowicki]

Hi Bjorn,

On 02.12.2016 22:57, Bjorn Helgaas wrote:
> On Fri, Dec 02, 2016 at 03:20:28PM +0100, Tomasz Nowicki wrote:
>> dmesg from ThunderX pass2.0 (1 socket board) + small fix:
>>
>> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
>> -       THUNDER_PEM_QUIRK(2,  0),       /* off-chip devices */
>> -       THUNDER_PEM_QUIRK(2,  1),       /* off-chip devices */
>> +       THUNDER_PEM_QUIRK(2,  0UL),     /* off-chip devices */
>> +       THUNDER_PEM_QUIRK(2,  1UL),     /* off-chip devices */
>
> Thanks!  I folded this change into my branch (possibly to be updated
> if Robert sends something better).

I believe Robert meant:
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index d34d196..8ca8ca8 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -76,7 +76,7 @@ static struct mcfg_fixup mcfg_quirks[] = {
         HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),

  #define THUNDER_PEM_RES(addr, node) \
-       DEFINE_RES_MEM(addr + (node << 44), 0x39 * SZ_16M)
+       DEFINE_RES_MEM(addr + ((u64)node << 44), 0x39 * SZ_16M)

Which means we can forget UL suffix for THUNDER_PEM_QUIRK macro:
@@ -91,8 +91,8 @@ static struct mcfg_fixup mcfg_quirks[] = {
         { "CAVIUM", "THUNDERX", rev, 9 + (10 * node), MCFG_BUS_ANY, 
     \
           &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89808f000000UL, node) }
         /* SoC pass2.x */
-       THUNDER_PEM_QUIRK(1, 0UL),
-       THUNDER_PEM_QUIRK(1, 1UL),
+       THUNDER_PEM_QUIRK(1, 0),
+       THUNDER_PEM_QUIRK(1, 1),

Also, my apologies, I should have fixed parentheses in this macro from 
the very beginning. For you convenience below incremental patch where I 
fixed both issues:

diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index d34d196..cf6c321 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -76,23 +76,23 @@ static struct mcfg_fixup mcfg_quirks[] = {
         HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),

  #define THUNDER_PEM_RES(addr, node) \
-       DEFINE_RES_MEM(addr + (node << 44), 0x39 * SZ_16M)
+       DEFINE_RES_MEM((addr) + ((u64)(node) << 44), 0x39 * SZ_16M)
  #define THUNDER_PEM_QUIRK(rev, node) \
-       { "CAVIUM", "THUNDERX", rev, 4 + (10 * node), MCFG_BUS_ANY, 
    \
-         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88001f000000UL, node) 
}, \
-       { "CAVIUM", "THUNDERX", rev, 5 + (10 * node), MCFG_BUS_ANY, 
    \
-         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x884057000000UL, node) 
}, \
-       { "CAVIUM", "THUNDERX", rev, 6 + (10 * node), MCFG_BUS_ANY, 
    \
-         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88808f000000UL, node) 
}, \
-       { "CAVIUM", "THUNDERX", rev, 7 + (10 * node), MCFG_BUS_ANY, 
    \
-         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89001f000000UL, node) 
}, \
-       { "CAVIUM", "THUNDERX", rev, 8 + (10 * node), MCFG_BUS_ANY, 
    \
-         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x894057000000UL, node) 
}, \
-       { "CAVIUM", "THUNDERX", rev, 9 + (10 * node), MCFG_BUS_ANY, 
    \
-         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89808f000000UL, node) }
+       { "CAVIUM", "THUNDERX", (rev), 4 + (10 * (node)), MCFG_BUS_ANY, 
      \
+         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88001f000000UL, 
(node)) }, \
+       { "CAVIUM", "THUNDERX", (rev), 5 + (10 * (node)), MCFG_BUS_ANY, 
      \
+         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x884057000000UL, 
(node)) }, \
+       { "CAVIUM", "THUNDERX", (rev), 6 + (10 * (node)), MCFG_BUS_ANY, 
      \
+         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88808f000000UL, 
(node)) }, \
+       { "CAVIUM", "THUNDERX", (rev), 7 + (10 * (node)), MCFG_BUS_ANY, 
      \
+         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89001f000000UL, 
(node)) }, \
+       { "CAVIUM", "THUNDERX", (rev), 8 + (10 * (node)), MCFG_BUS_ANY, 
      \
+         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x894057000000UL, 
(node)) }, \
+       { "CAVIUM", "THUNDERX", (rev), 9 + (10 * (node)), MCFG_BUS_ANY, 
      \
+         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89808f000000UL, (node)) }
         /* SoC pass2.x */
-       THUNDER_PEM_QUIRK(1, 0UL),
-       THUNDER_PEM_QUIRK(1, 1UL),
+       THUNDER_PEM_QUIRK(1, 0),
+       THUNDER_PEM_QUIRK(1, 1),

  #define THUNDER_ECAM_QUIRK(rev, seg)                                   \
         { "CAVIUM", "THUNDERX", rev, seg, MCFG_BUS_ANY,                 \
diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
index 00eb8eb..643c9e7 100644
--- a/include/linux/pci-ecam.h
+++ b/include/linux/pci-ecam.h
@@ -62,8 +62,9 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
  #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
  extern struct pci_ecam_ops pci_32b_ops;                /* 32-bit 
accesses only */
  extern struct pci_ecam_ops hisi_pcie_ops;      /* HiSilicon */
-extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 1.x 
& 2.x */
-extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */
+extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 
pass1.x &
+                                                   pass2.x */
+extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 
pass1.x */
  #endif

  #ifdef CONFIG_PCI_HOST_GENERIC

>
>> ACPI: MCFG table detected, 4 entries
>> ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
>> acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM
>> Segments MSI]
>> acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME AER]
>> acpi PNP0A08:00: _OSC: OS now controls [PCIeCapability]
>> acpi PNP0A08:00: ECAM area [mem 0x848000000000-0x848001ffffff]
>> reserved by THRX0001:00
>> acpi PNP0A08:00: ECAM at [mem 0x848000000000-0x848001ffffff] for [bus 00-1f]
>
> I guess we don't need MCFG quirks for these bridges, since I don't see
> the "MCFG quirk" message?

Yes, this is ThunderX 1-socket pass2.x so quirks are needed for 4-9 host 
bridges only. Below summary of where/how quirks should be applied:

ThunderX pass2.x
NUMA node -> segments -> ECAM compliant note
0 -> 0-3 -> ECAM compliant
0 -> 4-9 -> ECAM quirks (thunder_pem_ecam_ops)
1 (optionally) -> 10-13 -> ECAM compliant
1 (optionally) -> 14-19 -> ECAM quirks (thunder_pem_ecam_ops)

ThunderX pass1.x
NUMA node -> segments -> ECAM compliant note
0 -> 0-3 -> ECAM quirks (pci_thunder_ecam_ops)
0 -> 4-9 -> ECAM quirks (thunder_pem_ecam_ops)
1 (optionally) -> 10-13 -> ECAM quirks (pci_thunder_ecam_ops)
1 (optionally) -> 14-19 -> ECAM quirks (thunder_pem_ecam_ops)

>
>> system 00:00: [mem 0x848000000000-0x848001ffffff] could not be reserved
>> system 00:01: [mem 0x849000000000-0x849001ffffff] could not be reserved
>> system 00:02: [mem 0x84a000000000-0x84a001ffffff] could not be reserved
>> system 00:03: [mem 0x84b000000000-0x84b001ffffff] could not be reserved
>> system 00:04: [mem 0x87e0c0000000-0x87e0c0ffffff] could not be reserved
>> system 00:04: [mem 0x88001f000000-0x880057ffffff] could not be reserved
>> system 00:05: [mem 0x87e0c2000000-0x87e0c2ffffff] could not be reserved
>> system 00:05: [mem 0x88808f000000-0x8880c7ffffff] could not be reserved
>
> Most of these match ECAM spaces, which is good.  00:04 and 00:05 each
> have one ECAM space and one "other" space, which might be PEMx host
> bridge registers?  That all seems good.

Yes, "other" space is PEMx host bridge registers.

>
> But I assume the other bridges (PCIx) also have register space in
> addition to ECAM, and that should be reported also.

00:00, 00:01, 00:02, 00:03 are ECAM compliant so for those we need ECAM 
region only.

>
>> root@ubuntu:~# cat /proc/iomem
>> ...
>> 838000000000-841fffffffff : PCI Bus 0000:00
>>   838000000000-8380003fffff : 0000:03:00.0
>
> Something's missing here: we should have a clue about how we got from
> bus 00 to bus 03.  From your dmesg, 0000:00:15.0 is a bridge from bus
> 00 to bus 03, and its windows should appear here.  I'd expect
> something like:
>
>   838000000000-841fffffffff : PCI Bus 0000:00
>     838000000000-838fffffffff : PCI Bus 0000:03  <- 00:15.0 window
>       838000000000-8380003fffff : 0000:03:00.0
>
> This window should be inserted by generic code, so I don't know how
> this could be broken.  Your dmesg should also have something like
> this:
>
>   pci 0000:00:15.0: PCI bridge to [bus 03]
>   pci 0000:00:15.0:   bridge window [mem 0x838000000000-0x838fffffffff]
>
> I don't see that, maybe because this is actually a console log
> collected without "ignore_loglevel"?  But that obviously doesn't
> affect /proc/iomem, so I'm still puzzled about that.


Enhanced Allocation is used for devices but not for bridges which have 
invalid BARs in standard configuration header. Thus devices request 
resources from first valid parent bridge (host bridge in this case).

>
>> 87e024000000-87e024000fff : ARMH0011:00
>>   87e024000000-87e024000fff : ARMH0011:00
>
> This is interesting.  This must be a driver reserving these areas?
> Normally a driver would use the driver name, not the device name.
>
> Ideally, I think the core should reserve the region with the device
> name, and the driver would request it using the driver name, like
> this:
>
>   843000000000-84303fffffff : 0002:01:00.0    <-- PCI core reservation
>     843000000000-84303fffffff : thunder-nic   <-- driver request
>
> The ACPI core doesn't actually do the reservation, so I assume the
> ARMH0011:00 stuff is from the driver, and it's curious that it has the
> same region twice.
>
>> 87e026000000-87e0bfffffff : PCI Bus 0000:00
>>   87e027000000-87e0277fffff : CAVA02A:00
>
> This is interesting, too.  CAVA02A:00 looks like an ACPI device, but
> apparently it consumes some of the space that we think is routed to
> PCI bus 0000:00.  I think this happens on some x86 boxes, too, but it
> is somewhat confusing.

I will take a look.

>
> Based on this, I don't see any problems with the ThunderX quirks.
> I'd like to understand what's going on with the PCI-to-PCI bridge
> windows in /proc/iomem, but I doubt that's related to your quirks.

Yes it is not related to quirks.

However, CAVA02A:00 and ARMH0011 things should be investigated.

Thanks,
Tomasz

Re: [PATCH v10 00/12] PCI: ARM64 ECAM quirks

[Bjorn Helgaas]

On Mon, Dec 05, 2016 at 01:36:01PM +0100, Tomasz Nowicki wrote:
> On 02.12.2016 22:57, Bjorn Helgaas wrote:
> >On Fri, Dec 02, 2016 at 03:20:28PM +0100, Tomasz Nowicki wrote:
> >>dmesg from ThunderX pass2.0 (1 socket board) + small fix:

> I believe Robert meant:
> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> index d34d196..8ca8ca8 100644
> --- a/drivers/acpi/pci_mcfg.c
> +++ b/drivers/acpi/pci_mcfg.c
> @@ -76,7 +76,7 @@ static struct mcfg_fixup mcfg_quirks[] = {
>         HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),
> 
>  #define THUNDER_PEM_RES(addr, node) \
> -       DEFINE_RES_MEM(addr + (node << 44), 0x39 * SZ_16M)
> +       DEFINE_RES_MEM(addr + ((u64)node << 44), 0x39 * SZ_16M)
> 
> Which means we can forget UL suffix for THUNDER_PEM_QUIRK macro:
> @@ -91,8 +91,8 @@ static struct mcfg_fixup mcfg_quirks[] = {
>         { "CAVIUM", "THUNDERX", rev, 9 + (10 * node), MCFG_BUS_ANY,
> \
>           &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89808f000000UL, node) }
>         /* SoC pass2.x */
> -       THUNDER_PEM_QUIRK(1, 0UL),
> -       THUNDER_PEM_QUIRK(1, 1UL),
> +       THUNDER_PEM_QUIRK(1, 0),
> +       THUNDER_PEM_QUIRK(1, 1),
> 
> Also, my apologies, I should have fixed parentheses in this macro
> from the very beginning. For you convenience below incremental patch
> where I fixed both issues:
> 
> diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
> index d34d196..cf6c321 100644
> --- a/drivers/acpi/pci_mcfg.c
> +++ b/drivers/acpi/pci_mcfg.c
> @@ -76,23 +76,23 @@ static struct mcfg_fixup mcfg_quirks[] = {
>         HISI_QUAD_DOM("HIP07   ", 12, &hisi_pcie_ops),
> 
>  #define THUNDER_PEM_RES(addr, node) \
> -       DEFINE_RES_MEM(addr + (node << 44), 0x39 * SZ_16M)
> +       DEFINE_RES_MEM((addr) + ((u64)(node) << 44), 0x39 * SZ_16M)
>  #define THUNDER_PEM_QUIRK(rev, node) \
> -       { "CAVIUM", "THUNDERX", rev, 4 + (10 * node), MCFG_BUS_ANY,
> \
> -         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88001f000000UL,
> node) }, \
> -       { "CAVIUM", "THUNDERX", rev, 5 + (10 * node), MCFG_BUS_ANY,
> \
> -         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x884057000000UL,
> node) }, \
> -       { "CAVIUM", "THUNDERX", rev, 6 + (10 * node), MCFG_BUS_ANY,
> \
> -         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88808f000000UL,
> node) }, \
> -       { "CAVIUM", "THUNDERX", rev, 7 + (10 * node), MCFG_BUS_ANY,
> \
> -         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89001f000000UL,
> node) }, \
> -       { "CAVIUM", "THUNDERX", rev, 8 + (10 * node), MCFG_BUS_ANY,
> \
> -         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x894057000000UL,
> node) }, \
> -       { "CAVIUM", "THUNDERX", rev, 9 + (10 * node), MCFG_BUS_ANY,
> \
> -         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89808f000000UL, node) }
> +       { "CAVIUM", "THUNDERX", (rev), 4 + (10 * (node)),
> MCFG_BUS_ANY,      \
> +         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88001f000000UL,
> (node)) }, \
> +       { "CAVIUM", "THUNDERX", (rev), 5 + (10 * (node)),
> MCFG_BUS_ANY,      \
> +         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x884057000000UL,
> (node)) }, \
> +       { "CAVIUM", "THUNDERX", (rev), 6 + (10 * (node)),
> MCFG_BUS_ANY,      \
> +         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x88808f000000UL,
> (node)) }, \
> +       { "CAVIUM", "THUNDERX", (rev), 7 + (10 * (node)),
> MCFG_BUS_ANY,      \
> +         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89001f000000UL,
> (node)) }, \
> +       { "CAVIUM", "THUNDERX", (rev), 8 + (10 * (node)),
> MCFG_BUS_ANY,      \
> +         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x894057000000UL,
> (node)) }, \
> +       { "CAVIUM", "THUNDERX", (rev), 9 + (10 * (node)),
> MCFG_BUS_ANY,      \
> +         &thunder_pem_ecam_ops, THUNDER_PEM_RES(0x89808f000000UL, (node)) }
>         /* SoC pass2.x */
> -       THUNDER_PEM_QUIRK(1, 0UL),
> -       THUNDER_PEM_QUIRK(1, 1UL),
> +       THUNDER_PEM_QUIRK(1, 0),
> +       THUNDER_PEM_QUIRK(1, 1),
> 
>  #define THUNDER_ECAM_QUIRK(rev, seg)                                   \
>         { "CAVIUM", "THUNDERX", rev, seg, MCFG_BUS_ANY,                 \
> diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h
> index 00eb8eb..643c9e7 100644
> --- a/include/linux/pci-ecam.h
> +++ b/include/linux/pci-ecam.h
> @@ -62,8 +62,9 @@ extern struct pci_ecam_ops pci_generic_ecam_ops;
>  #if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)
>  extern struct pci_ecam_ops pci_32b_ops;                /* 32-bit
> accesses only */
>  extern struct pci_ecam_ops hisi_pcie_ops;      /* HiSilicon */
> -extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX
> 1.x & 2.x */
> -extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */
> +extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX
> pass1.x &
> +                                                   pass2.x */
> +extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX
> pass1.x */
>  #endif

I applied the above by hand, thanks!

> >>ACPI: MCFG table detected, 4 entries
> >>ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-1f])
> >>acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM
> >>Segments MSI]
> >>acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME AER]
> >>acpi PNP0A08:00: _OSC: OS now controls [PCIeCapability]
> >>acpi PNP0A08:00: ECAM area [mem 0x848000000000-0x848001ffffff]
> >>reserved by THRX0001:00
> >>acpi PNP0A08:00: ECAM at [mem 0x848000000000-0x848001ffffff] for [bus 00-1f]
> >
> >I guess we don't need MCFG quirks for these bridges, since I don't see
> >the "MCFG quirk" message?
> 
> Yes, this is ThunderX 1-socket pass2.x so quirks are needed for 4-9
> host bridges only. Below summary of where/how quirks should be
> applied:
> 
> ThunderX pass2.x
> NUMA node -> segments -> ECAM compliant note
> 0 -> 0-3 -> ECAM compliant
> 0 -> 4-9 -> ECAM quirks (thunder_pem_ecam_ops)
> 1 (optionally) -> 10-13 -> ECAM compliant
> 1 (optionally) -> 14-19 -> ECAM quirks (thunder_pem_ecam_ops)
> 
> ThunderX pass1.x
> NUMA node -> segments -> ECAM compliant note
> 0 -> 0-3 -> ECAM quirks (pci_thunder_ecam_ops)
> 0 -> 4-9 -> ECAM quirks (thunder_pem_ecam_ops)
> 1 (optionally) -> 10-13 -> ECAM quirks (pci_thunder_ecam_ops)
> 1 (optionally) -> 14-19 -> ECAM quirks (thunder_pem_ecam_ops)

I included the above in the appropriate changelogs.

> >>system 00:00: [mem 0x848000000000-0x848001ffffff] could not be reserved
> >>system 00:01: [mem 0x849000000000-0x849001ffffff] could not be reserved
> >>system 00:02: [mem 0x84a000000000-0x84a001ffffff] could not be reserved
> >>system 00:03: [mem 0x84b000000000-0x84b001ffffff] could not be reserved
> >>system 00:04: [mem 0x87e0c0000000-0x87e0c0ffffff] could not be reserved
> >>system 00:04: [mem 0x88001f000000-0x880057ffffff] could not be reserved
> >>system 00:05: [mem 0x87e0c2000000-0x87e0c2ffffff] could not be reserved
> >>system 00:05: [mem 0x88808f000000-0x8880c7ffffff] could not be reserved
> >
> >Most of these match ECAM spaces, which is good.  00:04 and 00:05 each
> >have one ECAM space and one "other" space, which might be PEMx host
> >bridge registers?  That all seems good.
> 
> Yes, "other" space is PEMx host bridge registers.
> 
> >But I assume the other bridges (PCIx) also have register space in
> >addition to ECAM, and that should be reported also.
> 
> 00:00, 00:01, 00:02, 00:03 are ECAM compliant so for those we need
> ECAM region only.

Even if they're ECAM compliant, I'm surprised that the bridges have no
other register space, e.g., for programming the range of bus numbers
below the bridge, root complex error reporting, etc.  This would all
be device-specific, of course, so in the ACPI model it would be used
by firmware only (either firmware init or by ASL).

> >>root@ubuntu:~# cat /proc/iomem
> >>...
> >>838000000000-841fffffffff : PCI Bus 0000:00
> >>  838000000000-8380003fffff : 0000:03:00.0
> >
> >Something's missing here: we should have a clue about how we got from
> >bus 00 to bus 03.  From your dmesg, 0000:00:15.0 is a bridge from bus
> >00 to bus 03, and its windows should appear here.  I'd expect
> >something like:
> >
> >  838000000000-841fffffffff : PCI Bus 0000:00
> >    838000000000-838fffffffff : PCI Bus 0000:03  <- 00:15.0 window
> >      838000000000-8380003fffff : 0000:03:00.0
> >
> >This window should be inserted by generic code, so I don't know how
> >this could be broken.  Your dmesg should also have something like
> >this:
> >
> >  pci 0000:00:15.0: PCI bridge to [bus 03]
> >  pci 0000:00:15.0:   bridge window [mem 0x838000000000-0x838fffffffff]
> >
> >I don't see that, maybe because this is actually a console log
> >collected without "ignore_loglevel"?  But that obviously doesn't
> >affect /proc/iomem, so I'm still puzzled about that.
> 
> Enhanced Allocation is used for devices but not for bridges which
> have invalid BARs in standard configuration header. Thus devices
> request resources from first valid parent bridge (host bridge in
> this case).

Hmm, OK.  I can tell EA is going to be a pain.  We might need to
improve our logging somehow to make this more visible.

For my own edification, does this mean the bridge itself has magic EA
functionality?  I was assuming EA would be used for non-configurable
endpoints.  If the bridges leading to those endpoints have extra
apertures not described by the standard bridge window registers, that
makes it much more complicated to understand the topology.

> >>87e024000000-87e024000fff : ARMH0011:00
> >>  87e024000000-87e024000fff : ARMH0011:00
> >
> >This is interesting.  This must be a driver reserving these areas?
> >Normally a driver would use the driver name, not the device name.
> >
> >Ideally, I think the core should reserve the region with the device
> >name, and the driver would request it using the driver name, like
> >this:
> >
> >  843000000000-84303fffffff : 0002:01:00.0    <-- PCI core reservation
> >    843000000000-84303fffffff : thunder-nic   <-- driver request
> >
> >The ACPI core doesn't actually do the reservation, so I assume the
> >ARMH0011:00 stuff is from the driver, and it's curious that it has the
> >same region twice.
> >
> >>87e026000000-87e0bfffffff : PCI Bus 0000:00
> >>  87e027000000-87e0277fffff : CAVA02A:00
> >
> >This is interesting, too.  CAVA02A:00 looks like an ACPI device, but
> >apparently it consumes some of the space that we think is routed to
> >PCI bus 0000:00.  I think this happens on some x86 boxes, too, but it
> >is somewhat confusing.
> 
> I will take a look.
> 
> >
> >Based on this, I don't see any problems with the ThunderX quirks.
> >I'd like to understand what's going on with the PCI-to-PCI bridge
> >windows in /proc/iomem, but I doubt that's related to your quirks.
> 
> Yes it is not related to quirks.
> 
> However, CAVA02A:00 and ARMH0011 things should be investigated.

Thanks!  This piece is more a curiosity thing on my part.

ARMH0011 looks like it's claimed by arm_sbsa_uart_platform_driver.
This path:

  sbsa_uart_probe
    pl011_setup_port
      devm_ioremap_resource
        devm_request_mem_region

explains one of the lines, still curious about the other.

Bjorn