[PATCH v6 0/9] PCI: brcmstb: have host-bridge turn on sub-device power

[PATCH v6 0/9] PCI: brcmstb: have host-bridge turn on sub-device power

[Jim Quinlan]

v6
     -- Dropped the idea of a placeholder regulator
        property (brcm-ep-a-supply). (MarkB)
     -- device_initialize() now called once.  Two
        commits were added for this.  (GKH)
     -- In two cases, separated a single function 
        into two or more functions (MarkB)
     -- "(void)foo();" => "foo()".  Note that although
        foo() returns an int, in this instance it is being
	invoked within a function returning void, and foo()
	already executes a dev_err() on error. (MarkB)
     -- Added a commit to correct PCIe interrupts in YAML.
     -- Removed "device_type = "pci";" for the EP node
        in the YAML example.
     -- Updated the URL related to the voltage regulator
        names on GitHub.  Note that I added vpciev3v3aux.

v5 [NOTE: It has been a while since v4.  Sorry]
     -- See "PCI: allow for callback to prepare nascent subdev"
        commit message for the cornerstone of this patchset
        and the reasons behind it.  This is a new commit.
     -- The RC driver now looks into its DT children and
        turns on regulators for a sub-device, and this occurs
	prior to PCIe link as it must.
     -- Dropped commits not related to the focus of this patchset.

v4 [NOTE: I'm not sure this fixes RobH and MarkB constraints but I'd
          like to use this pullreq as a basis for future discussion.]
   [Commit: Add bindings for ...]
     -- Fix syntax error in YAML bindings example (RobH)
     -- {vpcie12v,vpcie3v3}-supply props are back in root complex DT node
        (I believe RobH said this was okay)
   [Commit: Add control of ..]
     -- Do not do global search for regulator; now we look specifically
        for the property {vpcie12v,vpcie3v3}-supply in the root complex
	DT node and then call devm_regulator_bulk_get() (MarkB)
     -- Use devm_regulator_bulk_get() (Bjorn)
     -- s/EP/slot0 device/ (Bjorn)
     -- Spelling, capitalization (Bjorn)
     -- Have brcm_phy_stop() return a void (Bjorn)
   [Commit: Do not turn off ...]
     -- Capitalization (Bjorn)
   [Commit: Check return value ...]
     -- Commit message content (Bjorn)
     -- Move 6/6 hunk to 2/6 where it belongs (Bjorn)
     -- Move the rest of 6/6 before all other commits (Bjorn)

v3 -- Driver now searches for EP DT subnode for any regulators to turn on.
      If present, these regulators have the property names
      "vpcie12v-supply" and "vpcie3v3-supply".  The existence of these
      regulators in the EP subnode are currently pending as a pullreq
      in pci-bus.yaml at
      https://github.com/devicetree-org/dt-schema/pull/54
      (MarkB, RobH).
   -- Check return of brcm_set_regulators() (Florian)
   -- Specify one regulator string per line for easier update (Florian)
   -- Author/Committer/Signoff email changed from that of V2 from
      'james.quinlan@broadcom.com' to 'jim2101024@gmail.com'.

v2 -- Use regulator bulk API rather than multiple calls (MarkB).

v1 -- Bindings are added for fixed regulators that may power the EP device.
   -- The brcmstb RC driver is modified to control these regulators
      during probe, suspend, and resume.
   -- 7216 type SOCs have additional error reporting HW and a
      panic handler is added to dump its info.
   -- A missing return value check is added.

Jim Quinlan (9):
  dt-bindings: PCI: correct brcmstb interrupts, interrupt-map.
  dt-bindings: PCI: Add bindings for Brcmstb EP voltage regulators
  PCI: move pci_device_add() call
  PCI: separate device_initialize() from pci_device_add()
  PCI: allow for callback to prepare nascent subdev
  PCI: brcmstb: split brcm_pcie_setup() into two funcs
  PCI: brcmstb: Add control of subdevice voltage regulators
  PCI: brcmstb: Do not turn off regulators if EP can wake up
  PCI: brcmstb: change brcm_phy_stop() to return void

 .../bindings/pci/brcm,stb-pcie.yaml           |  35 ++-
 drivers/pci/controller/pcie-brcmstb.c         | 243 ++++++++++++++++--
 drivers/pci/iov.c                             |   1 +
 drivers/pci/probe.c                           |  61 +++--
 include/linux/pci.h                           |   4 +
 5 files changed, 308 insertions(+), 36 deletions(-)

-- 
2.17.1

[PATCH v6 1/9] dt-bindings: PCI: correct brcmstb interrupts, interrupt-map.

[Jim Quinlan]

The "pcie" and "msi" interrupts were given the same interrupt when they are
actually different.  Interrupt-map only had the INTA entry; the INTB, INTC,
and INTD entries are added.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
index b9589a0daa5c..508e5dce1282 100644
--- a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
@@ -142,11 +142,15 @@ examples:
                     #address-cells = <3>;
                     #size-cells = <2>;
                     #interrupt-cells = <1>;
-                    interrupts = <GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>,
+                    interrupts = <GIC_SPI 147 IRQ_TYPE_LEVEL_HIGH>,
                                  <GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>;
                     interrupt-names = "pcie", "msi";
                     interrupt-map-mask = <0x0 0x0 0x0 0x7>;
-                    interrupt-map = <0 0 0 1 &gicv2 GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
+                    interrupt-map = <0 0 0 1 &gicv2 GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH
+                                     0 0 0 2 &gicv2 GIC_SPI 144 IRQ_TYPE_LEVEL_HIGH
+                                     0 0 0 3 &gicv2 GIC_SPI 145 IRQ_TYPE_LEVEL_HIGH
+                                     0 0 0 4 &gicv2 GIC_SPI 146 IRQ_TYPE_LEVEL_HIGH>;
+
                     msi-parent = <&pcie0>;
                     msi-controller;
                     ranges = <0x02000000 0x0 0xf8000000 0x6 0x00000000 0x0 0x04000000>;
-- 
2.17.1

[PATCH v6 2/9] dt-bindings: PCI: Add bindings for Brcmstb EP voltage regulators

[Jim Quinlan]

Similar to the regulator bindings found in "rockchip-pcie-host.txt", this
allows optional regulators to be attached and controlled by the PCIe RC
driver.  That being said, this driver searches in the DT subnode (the EP
node, eg pci-ep@0,0) for the regulator property.

The use of a regulator property in the pcie EP subnode such as
"vpcie12v-supply" depends on a pending pullreq to the pci-bus.yaml
file at

https://github.com/devicetree-org/dt-schema/pull/63

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 .../bindings/pci/brcm,stb-pcie.yaml           | 27 +++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
index 508e5dce1282..d90d867deb5c 100644
--- a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
@@ -62,6 +62,10 @@ properties:
 
   aspm-no-l0s: true
 
+  vpcie12v-supply: true
+  vpcie3v3-supply: true
+  vpcie3v3aux-supply: true
+
   brcm,scb-sizes:
     description: u64 giving the 64bit PCIe memory
       viewport size of a memory controller.  There may be up to
@@ -158,5 +162,28 @@ examples:
                                  <0x42000000 0x1 0x80000000 0x3 0x00000000 0x0 0x80000000>;
                     brcm,enable-ssc;
                     brcm,scb-sizes =  <0x0000000080000000 0x0000000080000000>;
+
+                    /* PCIe bridge */
+                    pci@0,0 {
+                            #address-cells = <3>;
+                            #size-cells = <2>;
+                            reg = <0x0 0x0 0x0 0x0 0x0>;
+                            compatible = "pciclass,0604";
+                            device_type = "pci";
+                            ranges;
+
+                            /* PCIe endpoint */
+                            pci-ep@0,0 {
+                                    assigned-addresses =
+                                        <0x82010000 0x0 0xf8000000 0x6 0x00000000 0x0 0x2000>;
+                                    reg = <0x0 0x0 0x0 0x0 0x0>;
+                                    compatible = "pci14e4,1688";
+                                    vpcie3v3-supply = <&vreg7>;
+                                    #address-cells = <3>;
+                                    #size-cells = <2>;
+
+                                    ranges;
+                            };
+                    };
             };
     };
-- 
2.17.1

[PATCH v6 3/9] PCI: move pci_device_add() call

[Jim Quinlan]

Move call to pci_device_add() from pci_scan_single_device() to
pci_scan_device().  No other functions call pci_scan_device() so
this should be harmless.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 drivers/pci/probe.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index d9fc02a71baa..f3fc807b4fe8 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2391,6 +2391,7 @@ static struct pci_dev *pci_scan_device(struct pci_bus *bus, int devfn)
 		kfree(dev);
 		return NULL;
 	}
+	pci_device_add(dev, bus);
 
 	return dev;
 }
@@ -2540,13 +2541,7 @@ struct pci_dev *pci_scan_single_device(struct pci_bus *bus, int devfn)
 		return dev;
 	}
 
-	dev = pci_scan_device(bus, devfn);
-	if (!dev)
-		return NULL;
-
-	pci_device_add(dev, bus);
-
-	return dev;
+	return pci_scan_device(bus, devfn);
 }
 EXPORT_SYMBOL(pci_scan_single_device);
 
-- 
2.17.1

[PATCH v6 4/9] PCI: separate device_initialize() from pci_device_add()

[Jim Quinlan]

Do this so that device_initialize() may be called separately
(to prepare for an imminent commit).  This change does reverse
the invocation of these calls:

	pci_configure_device(dev);
	device_initialize(&dev->dev)
to
	device_initialize(&dev->dev)
	pci_configure_device(dev);

I reviewed this and didn't see any issue but it deserves mentioning.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 drivers/pci/iov.c   | 1 +
 drivers/pci/probe.c | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index dafdc652fcd0..baf5c1af47de 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -292,6 +292,7 @@ int pci_iov_add_virtfn(struct pci_dev *dev, int id)
 		BUG_ON(rc);
 	}
 
+	device_initialize(&virtfn->dev);
 	pci_device_add(virtfn, virtfn->bus);
 	rc = pci_iov_sysfs_link(dev, virtfn, id);
 	if (rc)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index f3fc807b4fe8..0f092882b33f 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2391,6 +2391,7 @@ static struct pci_dev *pci_scan_device(struct pci_bus *bus, int devfn)
 		kfree(dev);
 		return NULL;
 	}
+	device_initialize(&dev->dev);
 	pci_device_add(dev, bus);
 
 	return dev;
@@ -2491,7 +2492,6 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 
 	pci_configure_device(dev);
 
-	device_initialize(&dev->dev);
 	dev->dev.release = pci_release_dev;
 
 	set_dev_node(&dev->dev, pcibus_to_node(bus));
-- 
2.17.1

[PATCH v6 5/9] PCI: allow for callback to prepare nascent subdev

[Jim Quinlan]

We would like the Broadcom STB PCIe root complex driver to be able to turn
off/on regulators[1] that provide power to endpoint[2] devices.  Typically,
the drivers of these endpoint devices are stock Linux drivers that are not
aware that these regulator(s) exist and must be turned on for the driver to
be probed.  The simple solution of course is to turn these regulators on at
boot and keep them on.  However, this solution does not satisfy at least
three of our usage modes:

1. For example, one customer uses multiple PCIe controllers, but wants the
ability to, by script, turn any or all of them by and their subdevices off
to save power, e.g. when in battery mode.

2. Another example is when a watchdog script discovers that an endpoint
device is in an unresponsive state and would like to unbind, power toggle,
and re-bind just the PCIe endpoint and controller.

3. Of course we also want power turned off during suspend mode.  However,
some endpoint devices may be able to "wake" during suspend and we need to
recognise this case and veto the nominal act of turning off its regulator.
Such is the case with Wake-on-LAN and Wake-on-WLAN support where PCIe
end-point device needs to be kept powered on in order to receive network
packets and wake-up the system.

In all of these cases it is advantageous for the PCIe controller to govern
the turning off/on the regulators needed by the endpoint device.  The first
two cases can be done by simply unbinding and binding the PCIe controller,
if the controller has control of these regulators.

This commit solves the "chicken-and-egg" problem by providing a callback to
the RC driver when a downstream device is "discovered" by inspecting its
DT, and allowing said driver to allocate the device object prematurely in
order to get the regulator(s) and turn them on before the device's ID is
read.

[1] These regulators typically govern the actual power supply to the
    endpoint chip.  Sometimes they may be a the official PCIe socket
    power -- such as 3.3v or aux-3.3v.  Sometimes they are truly
    the regulator(s) that supply power to the EP chip.

[2] The 99% configuration of our boards is a single endpoint device
    attached to the PCIe controller.  I use the term endpoint but it could
    possible mean a switch as well.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 drivers/pci/probe.c | 52 ++++++++++++++++++++++++++++++++++-----------
 include/linux/pci.h |  4 ++++
 2 files changed, 44 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 0f092882b33f..8a58a9975ff6 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2372,29 +2372,57 @@ EXPORT_SYMBOL(pci_bus_read_dev_vendor_id);
  */
 static struct pci_dev *pci_scan_device(struct pci_bus *bus, int devfn)
 {
-	struct pci_dev *dev;
+	struct pci_host_bridge *hb = pci_find_host_bridge(bus);
+	struct pci_dev *dev = NULL;
+	bool device_initialize_called = false;
 	u32 l;
 
-	if (!pci_bus_read_dev_vendor_id(bus, devfn, &l, 60*1000))
-		return NULL;
+	/*
+	 * If the host bridge has a pci_subdev_can_prepare() function, call
+	 * it to see if it "cares" about this device.  If it does,
+	 * partially allocate some of the device and call
+	 * pci_subdev_prepare().  This tells it to allow the host bridge
+	 * driver to pre-allocate some resources such as voltage
+	 * regulators so that the pcie linkup can be established.
+	 */
+	if (hb->pci_subdev_can_prepare
+	    && hb->pci_subdev_can_prepare(bus, devfn)) {
+		dev = pci_alloc_dev(bus);
+		if (!dev)
+			return NULL;
 
-	dev = pci_alloc_dev(bus);
-	if (!dev)
-		return NULL;
+		dev->devfn = devfn;
+		device_initialize(&dev->dev);
+		device_initialize_called = true;
 
+		if (hb->pci_subdev_prepare(bus, devfn, hb, dev)
+		    || !pci_bus_read_dev_vendor_id(bus, devfn, &l, 60*1000))
+			goto err_out;
+	} else {
+		if (!pci_bus_read_dev_vendor_id(bus, devfn, &l, 60*1000))
+			return NULL;
+		dev = pci_alloc_dev(bus);
+		if (!dev)
+			return NULL;
+	}
 	dev->devfn = devfn;
 	dev->vendor = l & 0xffff;
 	dev->device = (l >> 16) & 0xffff;
 
-	if (pci_setup_device(dev)) {
-		pci_bus_put(dev->bus);
-		kfree(dev);
-		return NULL;
-	}
-	device_initialize(&dev->dev);
+	if (pci_setup_device(dev))
+		goto err_out;
+
+	if (!device_initialize_called)
+		device_initialize(&dev->dev);
+
 	pci_device_add(dev, bus);
 
 	return dev;
+
+err_out:
+	pci_bus_put(dev->bus);
+	kfree(dev);
+	return NULL;
 }
 
 void pcie_report_downtraining(struct pci_dev *dev)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index cd8aa6fce204..45ba24537ef2 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -552,6 +552,10 @@ struct pci_host_bridge {
 	u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
 	int (*map_irq)(const struct pci_dev *, u8, u8);
 	void (*release_fn)(struct pci_host_bridge *);
+	bool (*pci_subdev_can_prepare)(struct pci_bus *bus, int devfn);
+	int (*pci_subdev_prepare)(struct pci_bus *bus, int devfn,
+				  struct pci_host_bridge *hb,
+				  struct pci_dev *pdev);
 	void		*release_data;
 	unsigned int	ignore_reset_delay:1;	/* For entire hierarchy */
 	unsigned int	no_ext_tags:1;		/* No Extended Tags */
-- 
2.17.1

[PATCH v6 6/9] PCI: brcmstb: split brcm_pcie_setup() into two funcs

[Jim Quinlan]

We need to split a function in two so that the driver can take advantage of
the recently added function pci_subdev_prepare() which is a member
of struct pci_host_bridge.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 30 +++++++++++++++++++--------
 1 file changed, 21 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index cc30215f5a43..ba4d6daf312c 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -863,17 +863,10 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
 
 static int brcm_pcie_setup(struct brcm_pcie *pcie)
 {
-	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
 	u64 rc_bar2_offset, rc_bar2_size;
 	void __iomem *base = pcie->base;
-	struct device *dev = pcie->dev;
-	struct resource_entry *entry;
-	bool ssc_good = false;
-	struct resource *res;
-	int num_out_wins = 0;
-	u16 nlw, cls, lnksta;
-	int i, ret, memc;
-	u32 tmp, burst, aspm_support;
+	int ret, memc;
+	u32 tmp, burst;
 
 	/* Reset the bridge */
 	pcie->bridge_sw_init_set(pcie, 1);
@@ -956,6 +949,21 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 
 	if (pcie->gen)
 		brcm_pcie_set_gen(pcie, pcie->gen);
+	return 0;
+}
+
+static int brcm_pcie_linkup(struct brcm_pcie *pcie)
+{
+	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
+	struct device *dev = pcie->dev;
+	void __iomem *base = pcie->base;
+	struct resource_entry *entry;
+	struct resource *res;
+	int num_out_wins = 0;
+	u16 nlw, cls, lnksta;
+	bool ssc_good = false;
+	u32 aspm_support, tmp;
+	int ret, i;
 
 	/* Unassert the fundamental reset */
 	pcie->perst_set(pcie, 0);
@@ -1186,6 +1194,10 @@ static int brcm_pcie_resume(struct device *dev)
 	if (ret)
 		goto err_reset;
 
+	ret = brcm_pcie_linkup(pcie);
+	if (ret)
+		goto err_reset;
+
 	if (pcie->msi)
 		brcm_msi_set_regs(pcie->msi);
 
-- 
2.17.1

[PATCH v6 7/9] PCI: brcmstb: Add control of subdevice voltage regulators

[Jim Quinlan]

This Broadcom STB PCIe RC driver has one port and connects directly to one
device, be it a switch or an endpoint.  We want to be able to turn on/off
any regulators for that device.  Control of regulators is needed because of
the chicken-and-egg situation: although the regulator is "owned" by the
device and would be best handled by its driver, the device cannot be
discovered and probed unless its regulator is already turned on.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 164 +++++++++++++++++++++++++-
 1 file changed, 162 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index ba4d6daf312c..6635e143cfcb 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -24,6 +24,7 @@
 #include <linux/pci.h>
 #include <linux/pci-ecam.h>
 #include <linux/printk.h>
+#include <linux/regulator/consumer.h>
 #include <linux/reset.h>
 #include <linux/sizes.h>
 #include <linux/slab.h>
@@ -192,6 +193,12 @@ static inline void brcm_pcie_perst_set_4908(struct brcm_pcie *pcie, u32 val);
 static inline void brcm_pcie_perst_set_7278(struct brcm_pcie *pcie, u32 val);
 static inline void brcm_pcie_perst_set_generic(struct brcm_pcie *pcie, u32 val);
 
+static const char * const supplies[] = {
+	"vpcie3v3",
+	"vpcie3v3aux",
+	"vpcie12v",
+};
+
 enum {
 	RGR1_SW_INIT_1,
 	EXT_CFG_INDEX,
@@ -295,8 +302,38 @@ struct brcm_pcie {
 	u32			hw_rev;
 	void			(*perst_set)(struct brcm_pcie *pcie, u32 val);
 	void			(*bridge_sw_init_set)(struct brcm_pcie *pcie, u32 val);
+	struct regulator_bulk_data supplies[ARRAY_SIZE(supplies)];
+	unsigned int		num_supplies;
 };
 
+static int brcm_regulators_on(struct brcm_pcie *pcie)
+{
+	struct device *dev = pcie->dev;
+	int ret;
+
+	if (!pcie->num_supplies)
+		return 0;
+	ret = regulator_bulk_enable(pcie->num_supplies, pcie->supplies);
+	if (ret)
+		dev_err(dev, "failed to enable EP regulators\n");
+
+	return ret;
+}
+
+static int brcm_regulators_off(struct brcm_pcie *pcie)
+{
+	struct device *dev = pcie->dev;
+	int ret;
+
+	if (!pcie->num_supplies)
+		return 0;
+	ret = regulator_bulk_disable(pcie->num_supplies, pcie->supplies);
+	if (ret)
+		dev_err(dev, "failed to disable EP regulators\n");
+
+	return ret;
+}
+
 /*
  * This is to convert the size of the inbound "BAR" region to the
  * non-linear values of PCIE_X_MISC_RC_BAR[123]_CONFIG_LO.SIZE
@@ -1148,6 +1185,59 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
 	pcie->bridge_sw_init_set(pcie, 1);
 }
 
+static int brcm_pcie_get_regulators(struct brcm_pcie *pcie, struct pci_dev *dev)
+{
+	const unsigned int ns = ARRAY_SIZE(supplies);
+	struct device_node *dn;
+	struct property *pp;
+	unsigned int i;
+	int ret;
+
+	/* This is for Broadcom STB/CM chips only */
+	if (pcie->type == BCM2711)
+		return 0;
+
+	pci_set_of_node(dev);
+	dn = dev->dev.of_node;
+	if (!dn)
+		return 0;
+
+	for_each_property_of_node(dn, pp) {
+		for (i = 0; i < ns; i++) {
+			char prop_name[64]; /* 64 is max size of property name */
+
+			snprintf(prop_name, 64, "%s-supply", supplies[i]);
+			if (strcmp(prop_name, pp->name) == 0)
+				break;
+		}
+		if (i >= ns || pcie->num_supplies >= ARRAY_SIZE(supplies))
+			continue;
+
+		pcie->supplies[pcie->num_supplies++].supply = supplies[i];
+	}
+
+	if (pcie->num_supplies == 0)
+		return 0;
+
+	/*
+	 * We set the name ahead of time as the regulator core expects
+	 * it to exist when regulator_bulk_get() is called.
+	 */
+	dev_set_name(&dev->dev, "%04x:%02x:%02x.%d",
+		     pci_domain_nr(dev->bus),
+		     dev->bus->number, PCI_SLOT(dev->devfn),
+		     PCI_FUNC(dev->devfn));
+	/*
+	 * We cannot use devm_regulator_bulk_get() because the
+	 * downstream device may be removed w/o the regulator
+	 * first being disabled by the host bridge.
+	 */
+	ret = regulator_bulk_get(&dev->dev, pcie->num_supplies,
+				 pcie->supplies);
+
+	return ret;
+}
+
 static int brcm_pcie_suspend(struct device *dev)
 {
 	struct brcm_pcie *pcie = dev_get_drvdata(dev);
@@ -1158,7 +1248,7 @@ static int brcm_pcie_suspend(struct device *dev)
 	reset_control_rearm(pcie->rescal);
 	clk_disable_unprepare(pcie->clk);
 
-	return ret;
+	return brcm_regulators_off(pcie);
 }
 
 static int brcm_pcie_resume(struct device *dev)
@@ -1174,6 +1264,9 @@ static int brcm_pcie_resume(struct device *dev)
 	ret = reset_control_reset(pcie->rescal);
 	if (ret)
 		goto err_disable_clk;
+	ret = brcm_regulators_on(pcie);
+	if (ret)
+		goto err_reset;
 
 	ret = brcm_phy_start(pcie);
 	if (ret)
@@ -1217,6 +1310,10 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
 	brcm_phy_stop(pcie);
 	reset_control_rearm(pcie->rescal);
 	clk_disable_unprepare(pcie->clk);
+	if (pcie->num_supplies) {
+		brcm_regulators_off(pcie);
+		regulator_bulk_free(pcie->num_supplies, pcie->supplies);
+	}
 }
 
 static int brcm_pcie_remove(struct platform_device *pdev)
@@ -1241,6 +1338,57 @@ static const struct of_device_id brcm_pcie_match[] = {
 	{},
 };
 
+static bool brcm_pcie_pci_subdev_can_prepare(struct pci_bus *bus, int devfn)
+{
+	/*
+	 * We only care about a device that is directly connected
+	 * to the root complex, ie bus == 1 and slot == 0.
+	 */
+	return (bus->number == 1 && PCI_SLOT(devfn) == 0);
+}
+
+static int brcm_pcie_pci_subdev_prepare(struct pci_bus *bus, int devfn,
+					struct pci_host_bridge *bridge,
+					struct pci_dev *pdev)
+{
+	struct brcm_pcie *pcie;
+	int ret = 0;
+
+	pcie = (struct brcm_pcie *) bridge->sysdata;
+	ret = brcm_pcie_get_regulators(pcie, pdev);
+	if (ret) {
+		pcie->num_supplies = 0;
+		if (ret != -EPROBE_DEFER)
+			dev_err(pcie->dev, "failed to get regulators (err=%d)\n", ret);
+		return ret;
+	}
+
+	ret = brcm_regulators_on(pcie);
+	if (ret)
+		goto err_out0;
+
+	ret = brcm_pcie_linkup(pcie);
+	if (ret)
+		goto err_out1;
+
+	/*
+	 * dev_set_name() was called in brcm_get_regulators().  Free the
+	 * string it allocated as it will be called again when
+	 * pci_setup_device() is invoked.
+	 */
+	kfree_const(pdev->dev.kobj.name);
+	pdev->dev.kobj.name = NULL;
+
+	return 0;
+
+err_out1:
+	brcm_regulators_off(pcie);
+err_out0:
+	regulator_bulk_free(pcie->num_supplies, pcie->supplies);
+	pcie->num_supplies = 0;
+	return ret;
+}
+
 static int brcm_pcie_probe(struct platform_device *pdev)
 {
 	struct device_node *np = pdev->dev.of_node, *msi_np;
@@ -1327,12 +1475,24 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 		}
 	}
 
+	bridge->pci_subdev_can_prepare = brcm_pcie_pci_subdev_can_prepare;
+	bridge->pci_subdev_prepare = brcm_pcie_pci_subdev_prepare;
 	bridge->ops = &brcm_pcie_ops;
 	bridge->sysdata = pcie;
 
 	platform_set_drvdata(pdev, pcie);
 
-	return pci_host_probe(bridge);
+	ret = pci_host_probe(bridge);
+	if (!ret && !brcm_pcie_link_up(pcie))
+		ret = -ENODEV;
+
+	if (ret) {
+		brcm_pcie_remove(pdev);
+		return ret;
+	}
+
+	return 0;
+
 fail:
 	__brcm_pcie_remove(pcie);
 	return ret;
-- 
2.17.1

[PATCH v6 8/9] PCI: brcmstb: Do not turn off regulators if EP can wake up

[Jim Quinlan]

If any downstream device may wake up during S2/S3 suspend, we do not want
to turn off its power when suspending.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 49 ++++++++++++++++++++++++---
 1 file changed, 45 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 6635e143cfcb..18b9f7c97864 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -192,6 +192,7 @@ static inline void brcm_pcie_bridge_sw_init_set_generic(struct brcm_pcie *pcie,
 static inline void brcm_pcie_perst_set_4908(struct brcm_pcie *pcie, u32 val);
 static inline void brcm_pcie_perst_set_7278(struct brcm_pcie *pcie, u32 val);
 static inline void brcm_pcie_perst_set_generic(struct brcm_pcie *pcie, u32 val);
+static bool brcm_pcie_link_up(struct brcm_pcie *pcie);
 
 static const char * const supplies[] = {
 	"vpcie3v3",
@@ -304,8 +305,20 @@ struct brcm_pcie {
 	void			(*bridge_sw_init_set)(struct brcm_pcie *pcie, u32 val);
 	struct regulator_bulk_data supplies[ARRAY_SIZE(supplies)];
 	unsigned int		num_supplies;
+	bool			ep_wakeup_capable;
 };
 
+static int pci_dev_may_wakeup(struct pci_dev *dev, void *data)
+{
+	bool *ret = data;
+
+	if (device_may_wakeup(&dev->dev)) {
+		*ret = true;
+		dev_dbg(&dev->dev, "disable cancelled for wake-up device\n");
+	}
+	return (int) *ret;
+}
+
 static int brcm_regulators_on(struct brcm_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
@@ -313,6 +326,18 @@ static int brcm_regulators_on(struct brcm_pcie *pcie)
 
 	if (!pcie->num_supplies)
 		return 0;
+
+	if (pcie->ep_wakeup_capable) {
+		/*
+		 * We are resuming from a suspend.  In the previous suspend
+		 * we did not disable the power supplies, so there is no
+		 * need to enable them (and falsely increase their usage
+		 * count).
+		 */
+		pcie->ep_wakeup_capable = false;
+		return 0;
+	}
+
 	ret = regulator_bulk_enable(pcie->num_supplies, pcie->supplies);
 	if (ret)
 		dev_err(dev, "failed to enable EP regulators\n");
@@ -320,13 +345,28 @@ static int brcm_regulators_on(struct brcm_pcie *pcie)
 	return ret;
 }
 
-static int brcm_regulators_off(struct brcm_pcie *pcie)
+static int brcm_regulators_off(struct brcm_pcie *pcie, bool force)
 {
+	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
 	struct device *dev = pcie->dev;
 	int ret;
 
 	if (!pcie->num_supplies)
 		return 0;
+
+	pcie->ep_wakeup_capable = false;
+
+	if (!force && bridge->bus && brcm_pcie_link_up(pcie)) {
+		/*
+		 * If at least one device on this bus is enabled as a
+		 * wake-up source, do not turn off regulators.
+		 */
+		pci_walk_bus(bridge->bus, pci_dev_may_wakeup,
+			     &pcie->ep_wakeup_capable);
+		if (pcie->ep_wakeup_capable)
+			return 0;
+	}
+
 	ret = regulator_bulk_disable(pcie->num_supplies, pcie->supplies);
 	if (ret)
 		dev_err(dev, "failed to disable EP regulators\n");
@@ -1248,7 +1288,7 @@ static int brcm_pcie_suspend(struct device *dev)
 	reset_control_rearm(pcie->rescal);
 	clk_disable_unprepare(pcie->clk);
 
-	return brcm_regulators_off(pcie);
+	return brcm_regulators_off(pcie, false);
 }
 
 static int brcm_pcie_resume(struct device *dev)
@@ -1264,6 +1304,7 @@ static int brcm_pcie_resume(struct device *dev)
 	ret = reset_control_reset(pcie->rescal);
 	if (ret)
 		goto err_disable_clk;
+
 	ret = brcm_regulators_on(pcie);
 	if (ret)
 		goto err_reset;
@@ -1311,7 +1352,7 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
 	reset_control_rearm(pcie->rescal);
 	clk_disable_unprepare(pcie->clk);
 	if (pcie->num_supplies) {
-		brcm_regulators_off(pcie);
+		brcm_regulators_off(pcie, true);
 		regulator_bulk_free(pcie->num_supplies, pcie->supplies);
 	}
 }
@@ -1382,7 +1423,7 @@ static int brcm_pcie_pci_subdev_prepare(struct pci_bus *bus, int devfn,
 	return 0;
 
 err_out1:
-	brcm_regulators_off(pcie);
+	brcm_regulators_off(pcie, true);
 err_out0:
 	regulator_bulk_free(pcie->num_supplies, pcie->supplies);
 	pcie->num_supplies = 0;
-- 
2.17.1

[PATCH v6 9/9] PCI: brcmstb: change brcm_phy_stop() to return void

[Jim Quinlan]

We do not use the result of this function so make it void.

Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 18b9f7c97864..c1f8fdb89cec 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -1196,9 +1196,10 @@ static inline int brcm_phy_start(struct brcm_pcie *pcie)
 	return pcie->rescal ? brcm_phy_cntl(pcie, 1) : 0;
 }
 
-static inline int brcm_phy_stop(struct brcm_pcie *pcie)
+static inline void brcm_phy_stop(struct brcm_pcie *pcie)
 {
-	return pcie->rescal ? brcm_phy_cntl(pcie, 0) : 0;
+	if (pcie->rescal)
+		brcm_phy_cntl(pcie, 0);
 }
 
 static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
@@ -1281,10 +1282,9 @@ static int brcm_pcie_get_regulators(struct brcm_pcie *pcie, struct pci_dev *dev)
 static int brcm_pcie_suspend(struct device *dev)
 {
 	struct brcm_pcie *pcie = dev_get_drvdata(dev);
-	int ret;
 
 	brcm_pcie_turn_off(pcie);
-	ret = brcm_phy_stop(pcie);
+	brcm_phy_stop(pcie);
 	reset_control_rearm(pcie->rescal);
 	clk_disable_unprepare(pcie->clk);
 
-- 
2.17.1

Re: [PATCH v6 1/9] dt-bindings: PCI: correct brcmstb interrupts, interrupt-map.

[Florian Fainelli]

On 10/29/21 1:03 PM, Jim Quinlan wrote:
> The "pcie" and "msi" interrupts were given the same interrupt when they are
> actually different.  Interrupt-map only had the INTA entry; the INTB, INTC,
> and INTD entries are added.
> 
> Signed-off-by: Jim Quinlan <jim2101024@gmail.com>

Acked-by: Florian Fainelli <f.fainelli@gmail.com>

I will submit a fix for arch/arm/boot/dts/bcm2711.dtsi shortly, since it
has the same issues as the example you are fixing, thanks!
-- 
Florian

Re: [PATCH v6 7/9] PCI: brcmstb: Add control of subdevice voltage regulators

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]

Hi Jim,

I love your patch! Yet something to improve:

[auto build test ERROR on helgaas-pci/next]
[also build test ERROR on robh/for-next v5.15-rc7 next-20211029]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Jim-Quinlan/PCI-brcmstb-have-host-bridge-turn-on-sub-device-power/20211030-040531
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/0day-ci/linux/commit/00b5df640b36d92065547cd17d317a1b9f241124
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Jim-Quinlan/PCI-brcmstb-have-host-bridge-turn-on-sub-device-power/20211030-040531
        git checkout 00b5df640b36d92065547cd17d317a1b9f241124
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross ARCH=arm64 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>, old ones prefixed by <<):

>> ERROR: modpost: "pci_set_of_node" [drivers/pci/controller/pcie-brcmstb.ko] undefined!

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 55878 bytes --]

Re: [PATCH v6 7/9] PCI: brcmstb: Add control of subdevice voltage regulators

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 550 bytes --]

On Fri, Oct 29, 2021 at 04:03:15PM -0400, Jim Quinlan wrote:

> This Broadcom STB PCIe RC driver has one port and connects directly to one
> device, be it a switch or an endpoint.  We want to be able to turn on/off
> any regulators for that device.  Control of regulators is needed because of
> the chicken-and-egg situation: although the regulator is "owned" by the
> device and would be best handled by its driver, the device cannot be
> discovered and probed unless its regulator is already turned on.

Reviwed-by: Mark Brown <broonie@kernel.org>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v6 8/9] PCI: brcmstb: Do not turn off regulators if EP can wake up

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

On Fri, Oct 29, 2021 at 04:03:16PM -0400, Jim Quinlan wrote:

> If any downstream device may wake up during S2/S3 suspend, we do not want
> to turn off its power when suspending.

Reviwed-by: Mark Brown <broonie@kernel.org>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v6 1/9] dt-bindings: PCI: correct brcmstb interrupts, interrupt-map.

[Rob Herring]

On Fri, 29 Oct 2021 16:03:09 -0400, Jim Quinlan wrote:
> The "pcie" and "msi" interrupts were given the same interrupt when they are
> actually different.  Interrupt-map only had the INTA entry; the INTB, INTC,
> and INTD entries are added.
> 
> Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
> ---
>  Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 

Acked-by: Rob Herring <robh@kernel.org>

Re: [PATCH v6 2/9] dt-bindings: PCI: Add bindings for Brcmstb EP voltage regulators

[Rob Herring]

On Fri, Oct 29, 2021 at 04:03:10PM -0400, Jim Quinlan wrote:
> Similar to the regulator bindings found in "rockchip-pcie-host.txt", this
> allows optional regulators to be attached and controlled by the PCIe RC
> driver.  That being said, this driver searches in the DT subnode (the EP
> node, eg pci-ep@0,0) for the regulator property.
> 
> The use of a regulator property in the pcie EP subnode such as
> "vpcie12v-supply" depends on a pending pullreq to the pci-bus.yaml
> file at
> 
> https://github.com/devicetree-org/dt-schema/pull/63
> 
> Signed-off-by: Jim Quinlan <jim2101024@gmail.com>
> ---
>  .../bindings/pci/brcm,stb-pcie.yaml           | 27 +++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
> index 508e5dce1282..d90d867deb5c 100644
> --- a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
> +++ b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
> @@ -62,6 +62,10 @@ properties:
>  
>    aspm-no-l0s: true
>  
> +  vpcie12v-supply: true
> +  vpcie3v3-supply: true
> +  vpcie3v3aux-supply: true
> +

You've put these in the host bridge node and in *any* bridge node for 
pci-bus.yaml, but that's not where you have them in the example.

The question is where is the right place. Normally, I'd say that's in 
the node consuming or connected to the resource. That would be as you 
have the example. However, as we're talking about standard slots (or at 
least standard slot rails), we likely don't have node for the device in 
the slot and can't add one since it is likely unknown what the device 
is. In other cases, we'd define a connector or slot node, but there's 
not really a way to do that given the PCI bus topology is already 
defined. So I think the right place is the bridge node on the upstream 
side of the slot/connector. So the 'pci@0,0' node in your case.

Rob

>    brcm,scb-sizes:
>      description: u64 giving the 64bit PCIe memory
>        viewport size of a memory controller.  There may be up to
> @@ -158,5 +162,28 @@ examples:
>                                   <0x42000000 0x1 0x80000000 0x3 0x00000000 0x0 0x80000000>;
>                      brcm,enable-ssc;
>                      brcm,scb-sizes =  <0x0000000080000000 0x0000000080000000>;
> +
> +                    /* PCIe bridge */
> +                    pci@0,0 {
> +                            #address-cells = <3>;
> +                            #size-cells = <2>;
> +                            reg = <0x0 0x0 0x0 0x0 0x0>;
> +                            compatible = "pciclass,0604";
> +                            device_type = "pci";
> +                            ranges;
> +
> +                            /* PCIe endpoint */
> +                            pci-ep@0,0 {
> +                                    assigned-addresses =
> +                                        <0x82010000 0x0 0xf8000000 0x6 0x00000000 0x0 0x2000>;
> +                                    reg = <0x0 0x0 0x0 0x0 0x0>;
> +                                    compatible = "pci14e4,1688";
> +                                    vpcie3v3-supply = <&vreg7>;
> +                                    #address-cells = <3>;
> +                                    #size-cells = <2>;
> +
> +                                    ranges;
> +                            };
> +                    };
>              };
>      };
> -- 
> 2.17.1
> 
> 

Re: [PATCH v6 7/9] PCI: brcmstb: Add control of subdevice voltage regulators

[Rob Herring]

On Fri, Oct 29, 2021 at 04:03:15PM -0400, Jim Quinlan wrote:
> This Broadcom STB PCIe RC driver has one port and connects directly to one
> device, be it a switch or an endpoint.  We want to be able to turn on/off
> any regulators for that device.  Control of regulators is needed because of
> the chicken-and-egg situation: although the regulator is "owned" by the
> device and would be best handled by its driver, the device cannot be
> discovered and probed unless its regulator is already turned on.

I think this can be done in a much more simple way that avoids the 
prior patches using the pci_ops.add_bus() (and remove_bus()) hook. 
add_bus is called before the core scans a child bus. In the handler, you 
just need to get the bridge device, then the bridge DT node, and then 
get the regulators and enable.  

Given we're talking about standard properties in a standard (bridge) 
node, I think the implementation for .add_bus should be common 
(drivers/pci/of.c). It doesn't scale to be doing this in every host 
bridge driver.

Rob

Re: [PATCH v6 7/9] PCI: brcmstb: Add control of subdevice voltage regulators

[Jim Quinlan]

On Tue, Nov 2, 2021 at 12:00 PM Rob Herring <robh@kernel.org> wrote:
>
> On Fri, Oct 29, 2021 at 04:03:15PM -0400, Jim Quinlan wrote:
> > This Broadcom STB PCIe RC driver has one port and connects directly to one
> > device, be it a switch or an endpoint.  We want to be able to turn on/off
> > any regulators for that device.  Control of regulators is needed because of
> > the chicken-and-egg situation: although the regulator is "owned" by the
> > device and would be best handled by its driver, the device cannot be
> > discovered and probed unless its regulator is already turned on.
>
> I think this can be done in a much more simple way that avoids the
> prior patches using the pci_ops.add_bus() (and remove_bus()) hook.
> add_bus is called before the core scans a child bus. In the handler, you
> just need to get the bridge device, then the bridge DT node, and then
> get the regulators and enable.
Hi Rob,
In reply to my bindings commit you wanted to put the "xxx-supply"
property(s) under the
bridge node rather than under the pci-ep node.   This not only makes
sense but also removes
the burden of prematurely creating the struct device *ptr as the
bridge device has
already been created.

However, there is still an issue:  if  the pcie-link is not
successful, we want the bus enumeration
to stop and not read the vendor/dev id of the EP.  Our controller has
the disadvantage of causing
an abort when accessing config space when the link is not established.  Other
controllers kindly return 0xffffffff as the data.

Doing something like this gets around the issue:

static struct pci_bus *pci_alloc_child_bus(...)
{
        /* ... */
add_dev:
        /* ... */
        if (child->ops->add_bus) {
                ret = child->ops->add_bus(child);
+               if (ret == -ENOLINK)
+                       return NULL;
                if (WARN_ON(ret < 0))
                        dev_err(&child->dev, "failed to add bus: %d\n", ret);
        }

Is this acceptable?  Other suggestions?


>
> Given we're talking about standard properties in a standard (bridge)
> node, I think the implementation for .add_bus should be common
> (drivers/pci/of.c). It doesn't scale to be doing this in every host
> bridge driver.
Are you saying that the bridge DT node  should have a property such as
"get-and-turn-on-subdev-regulators;" which would invoke what I'm now
calling brcm_pcie_add_bus()?   The problem with this is that our host
bridge needs to be the agent freeing the regulators.  IIRC correctly, when
the regulators were freed by the EP device -- or now the bridge in this case
-- we got panics when doing unbinds.  I will go back and get the details
on this, but I'm wondering if our controller has arcane but necessary
requirements
outside of what a general mechanism could provide.

Thanks,
Jim

>
> Rob

Re: [PATCH v6 7/9] PCI: brcmstb: Add control of subdevice voltage regulators

[Rob Herring]

On Tue, Nov 2, 2021 at 5:36 PM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>
> On Tue, Nov 2, 2021 at 12:00 PM Rob Herring <robh@kernel.org> wrote:
> >
> > On Fri, Oct 29, 2021 at 04:03:15PM -0400, Jim Quinlan wrote:
> > > This Broadcom STB PCIe RC driver has one port and connects directly to one
> > > device, be it a switch or an endpoint.  We want to be able to turn on/off
> > > any regulators for that device.  Control of regulators is needed because of
> > > the chicken-and-egg situation: although the regulator is "owned" by the
> > > device and would be best handled by its driver, the device cannot be
> > > discovered and probed unless its regulator is already turned on.
> >
> > I think this can be done in a much more simple way that avoids the
> > prior patches using the pci_ops.add_bus() (and remove_bus()) hook.
> > add_bus is called before the core scans a child bus. In the handler, you
> > just need to get the bridge device, then the bridge DT node, and then
> > get the regulators and enable.
> Hi Rob,
> In reply to my bindings commit you wanted to put the "xxx-supply"
> property(s) under the
> bridge node rather than under the pci-ep node.   This not only makes
> sense but also removes
> the burden of prematurely creating the struct device *ptr as the
> bridge device has
> already been created.
>
> However, there is still an issue:  if  the pcie-link is not
> successful, we want the bus enumeration
> to stop and not read the vendor/dev id of the EP.  Our controller has
> the disadvantage of causing
> an abort when accessing config space when the link is not established.  Other
> controllers kindly return 0xffffffff as the data.
>
> Doing something like this gets around the issue:
>
> static struct pci_bus *pci_alloc_child_bus(...)
> {
>         /* ... */
> add_dev:
>         /* ... */
>         if (child->ops->add_bus) {
>                 ret = child->ops->add_bus(child);
> +               if (ret == -ENOLINK)
> +                       return NULL;
>                 if (WARN_ON(ret < 0))
>                         dev_err(&child->dev, "failed to add bus: %d\n", ret);
>         }
>
> Is this acceptable?  Other suggestions?

Acceptable yes once we agree on error code to return. I'd just do -ENODEV.

> > Given we're talking about standard properties in a standard (bridge)
> > node, I think the implementation for .add_bus should be common
> > (drivers/pci/of.c). It doesn't scale to be doing this in every host
> > bridge driver.
> Are you saying that the bridge DT node  should have a property such as
> "get-and-turn-on-subdev-regulators;" which would invoke what I'm now
> calling brcm_pcie_add_bus()?

No! Define a common function that host drivers can opt in to by
setting their .add_bus() hook to or calling from their own add_bus
function.

Ideally, it would work on Rockchip too as it's the same supplies.
However, that would require some reworking of the link initialization
and PERST handling. As Bjorn has mentioned, all that should be per RP,
not per host bridge anyways. I'm taking it one step further and saying
it should be per PCI bridge. Hikey for example needs PERST handling on
bridges behind a switch.

Rob