* [PATCH v6 0/5] powerpc/powernv/pci: Make hotplug self-sufficient, independent of FW and DT
@ 2019-08-16 16:16 Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 1/5] powerpc/pci: Access PCI config space directly w/o pci_dn Sergey Miroshnichenko
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Sergey Miroshnichenko @ 2019-08-16 16:16 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sam Bobroff, Sergey Miroshnichenko, Oliver O'Halloran, linux
Allow switching from the pnv_php module to the standard pciehp driver for
PowerNV, if the platform supports it: it can be a server working on top of
the skiboot with the [1] patchset applied.
Add the ability to discover hot-added devices which weren't added to the
Device Tree (by the pnv_php via an explicit OPAL call when a hotplug event
was intercepted) by direct access to the bus.
Sync the changes in PCIe topology (bus numbers and PEs) with the skiboot.
Tested on POWER8 PowerNV+PHB3 ppc64le (our Vesnin server) with:
- the pciehp driver active;
- the pnv_php driver disabled;
- the "pci=pcie_bus_peer2peer,realloc" kernel command line argument;
- controlled hotplug of a network card with SR-IOV works;
- activating of SR-IOV on a network card works;
- [with extra patches for movable BARs and bus numbers] manually initiated
(via sysfs) rescan has found and turned on a hotplugged bridge.
[1] https://lists.ozlabs.org/pipermail/skiboot/2019-August/015140.html
[Skiboot] [PATCH v3 0/5] core/pci: Track changes of topology by an OS
Change since v5:
- Activates on "ibm,supported-movable-bdfs" property in DT from skiboot
instead of the "pci=realloc" flag;
- Removed the code refactoring patches - will send them separately.
Changes since v4:
- Fixed failing build when EEH is disabled in a kernel config;
- Unfreeze the bus on EEH_IO_ERROR_VALUE(size), not only 0xffffffff;
- Replaced the 0xff magic constant with phb->ioda.reserved_pe_idx;
- Renamed create_pdn() -> pci_create_pdn_from_dev();
- Renamed add_one_dev_pci_data(..., vf_index, ...) -> pci_alloc_pdn();
- Renamed add_dev_pci_data() -> pci_create_vf_pdns();
- Renamed remove_dev_pci_data() -> pci_destroy_vf_pdns();
- Removed the patch fixing uninitialized IOMMU group - now it is fixed in
commit 8f5b27347e88 ("powerpc/powernv/sriov: Register IOMMU groups for
VFs")
Changes since v3:
- Subject changed;
- Don't disable EEH during rescan anymore - instead just unfreeze the
target buses deliberately;
- Add synchronization with the firmware when changing the PCIe topology;
- Fixed for VFs;
- Code cleanup.
Changes since v2:
- Don't reassign bus numbers on PowerNV by default (to retain the default
behavior), but only when pci=realloc is passed;
- Less code affected;
- pci_add_device_node_info is refactored with add_one_dev_pci_data;
- Minor code cleanup.
Changes since v1:
- Fixed build for ppc64le and ppc64be when CONFIG_PCI_IOV is disabled;
- Fixed build for ppc64e when CONFIG_EEH is disabled;
- Fixed code style warnings.
Sergey Miroshnichenko (5):
powerpc/pci: Access PCI config space directly w/o pci_dn
powerpc/powernv/pci: Suppress an EEH error when reading an empty slot
powerpc/pci: Create pci_dn on demand
powerpc/powernv/pci: Hook up the writes to PCI_SECONDARY_BUS register
powerpc/pci: Enable assigning bus numbers instead of reading them from
DT
arch/powerpc/include/asm/ppc-pci.h | 1 +
arch/powerpc/kernel/pci_dn.c | 95 +++++++--
arch/powerpc/kernel/rtas_pci.c | 97 ++++++---
arch/powerpc/platforms/powernv/eeh-powernv.c | 2 +-
arch/powerpc/platforms/powernv/pci.c | 205 +++++++++++++++++--
5 files changed, 331 insertions(+), 69 deletions(-)
--
2.21.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v6 1/5] powerpc/pci: Access PCI config space directly w/o pci_dn
2019-08-16 16:16 [PATCH v6 0/5] powerpc/powernv/pci: Make hotplug self-sufficient, independent of FW and DT Sergey Miroshnichenko
@ 2019-08-16 16:16 ` Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 2/5] powerpc/powernv/pci: Suppress an EEH error when reading an empty slot Sergey Miroshnichenko
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Sergey Miroshnichenko @ 2019-08-16 16:16 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sam Bobroff, Sergey Miroshnichenko, Oliver O'Halloran, linux
To fetch an updated DT for the newly hotplugged device, OS must explicitly
request it from the firmware via the pnv_php driver.
If pnv_php wasn't triggered/loaded, it is still possible to discover new
devices if PCIe I/O will not stop in absence of the pci_dn structure.
Reviewed-by: Oliver O'Halloran <oohall@gmail.com>
Signed-off-by: Sergey Miroshnichenko <s.miroshnichenko@yadro.com>
---
arch/powerpc/kernel/rtas_pci.c | 97 +++++++++++++++++++---------
arch/powerpc/platforms/powernv/pci.c | 64 ++++++++++++------
2 files changed, 109 insertions(+), 52 deletions(-)
diff --git a/arch/powerpc/kernel/rtas_pci.c b/arch/powerpc/kernel/rtas_pci.c
index ae5e43eaca48..912da28b3737 100644
--- a/arch/powerpc/kernel/rtas_pci.c
+++ b/arch/powerpc/kernel/rtas_pci.c
@@ -42,10 +42,26 @@ static inline int config_access_valid(struct pci_dn *dn, int where)
return 0;
}
-int rtas_read_config(struct pci_dn *pdn, int where, int size, u32 *val)
+static int rtas_read_raw_config(unsigned long buid, int busno, unsigned int devfn,
+ int where, int size, u32 *val)
{
int returnval = -1;
- unsigned long buid, addr;
+ unsigned long addr = rtas_config_addr(busno, devfn, where);
+ int ret;
+
+ if (buid) {
+ ret = rtas_call(ibm_read_pci_config, 4, 2, &returnval,
+ addr, BUID_HI(buid), BUID_LO(buid), size);
+ } else {
+ ret = rtas_call(read_pci_config, 2, 2, &returnval, addr, size);
+ }
+ *val = returnval;
+
+ return ret;
+}
+
+int rtas_read_config(struct pci_dn *pdn, int where, int size, u32 *val)
+{
int ret;
if (!pdn)
@@ -58,16 +74,8 @@ int rtas_read_config(struct pci_dn *pdn, int where, int size, u32 *val)
return PCIBIOS_SET_FAILED;
#endif
- addr = rtas_config_addr(pdn->busno, pdn->devfn, where);
- buid = pdn->phb->buid;
- if (buid) {
- ret = rtas_call(ibm_read_pci_config, 4, 2, &returnval,
- addr, BUID_HI(buid), BUID_LO(buid), size);
- } else {
- ret = rtas_call(read_pci_config, 2, 2, &returnval, addr, size);
- }
- *val = returnval;
-
+ ret = rtas_read_raw_config(pdn->phb->buid, pdn->busno, pdn->devfn,
+ where, size, val);
if (ret)
return PCIBIOS_DEVICE_NOT_FOUND;
@@ -85,18 +93,44 @@ static int rtas_pci_read_config(struct pci_bus *bus,
pdn = pci_get_pdn_by_devfn(bus, devfn);
- /* Validity of pdn is checked in here */
- ret = rtas_read_config(pdn, where, size, val);
- if (*val == EEH_IO_ERROR_VALUE(size) &&
- eeh_dev_check_failure(pdn_to_eeh_dev(pdn)))
- return PCIBIOS_DEVICE_NOT_FOUND;
+ if (pdn) {
+ /* Validity of pdn is checked in here */
+ ret = rtas_read_config(pdn, where, size, val);
+
+ if (*val == EEH_IO_ERROR_VALUE(size) &&
+ eeh_dev_check_failure(pdn_to_eeh_dev(pdn)))
+ ret = PCIBIOS_DEVICE_NOT_FOUND;
+ } else {
+ struct pci_controller *phb = pci_bus_to_host(bus);
+
+ ret = rtas_read_raw_config(phb->buid, bus->number, devfn,
+ where, size, val);
+ }
return ret;
}
+static int rtas_write_raw_config(unsigned long buid, int busno, unsigned int devfn,
+ int where, int size, u32 val)
+{
+ unsigned long addr = rtas_config_addr(busno, devfn, where);
+ int ret;
+
+ if (buid) {
+ ret = rtas_call(ibm_write_pci_config, 5, 1, NULL, addr,
+ BUID_HI(buid), BUID_LO(buid), size, (ulong)val);
+ } else {
+ ret = rtas_call(write_pci_config, 3, 1, NULL, addr, size, (ulong)val);
+ }
+
+ if (ret)
+ return PCIBIOS_DEVICE_NOT_FOUND;
+
+ return PCIBIOS_SUCCESSFUL;
+}
+
int rtas_write_config(struct pci_dn *pdn, int where, int size, u32 val)
{
- unsigned long buid, addr;
int ret;
if (!pdn)
@@ -109,15 +143,8 @@ int rtas_write_config(struct pci_dn *pdn, int where, int size, u32 val)
return PCIBIOS_SET_FAILED;
#endif
- addr = rtas_config_addr(pdn->busno, pdn->devfn, where);
- buid = pdn->phb->buid;
- if (buid) {
- ret = rtas_call(ibm_write_pci_config, 5, 1, NULL, addr,
- BUID_HI(buid), BUID_LO(buid), size, (ulong) val);
- } else {
- ret = rtas_call(write_pci_config, 3, 1, NULL, addr, size, (ulong)val);
- }
-
+ ret = rtas_write_raw_config(pdn->phb->buid, pdn->busno, pdn->devfn,
+ where, size, val);
if (ret)
return PCIBIOS_DEVICE_NOT_FOUND;
@@ -128,12 +155,20 @@ static int rtas_pci_write_config(struct pci_bus *bus,
unsigned int devfn,
int where, int size, u32 val)
{
- struct pci_dn *pdn;
+ struct pci_dn *pdn = pci_get_pdn_by_devfn(bus, devfn);
+ int ret;
- pdn = pci_get_pdn_by_devfn(bus, devfn);
+ if (pdn) {
+ /* Validity of pdn is checked in here. */
+ ret = rtas_write_config(pdn, where, size, val);
+ } else {
+ struct pci_controller *phb = pci_bus_to_host(bus);
- /* Validity of pdn is checked in here. */
- return rtas_write_config(pdn, where, size, val);
+ ret = rtas_write_raw_config(phb->buid, bus->number, devfn,
+ where, size, val);
+ }
+
+ return ret;
}
static struct pci_ops rtas_pci_ops = {
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 6104418c9ad5..8d6c094f074e 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -647,30 +647,29 @@ static void pnv_pci_config_check_eeh(struct pci_dn *pdn)
}
}
-int pnv_pci_cfg_read(struct pci_dn *pdn,
- int where, int size, u32 *val)
+static int pnv_pci_cfg_read_raw(u64 phb_id, int busno, unsigned int devfn,
+ int where, int size, u32 *val)
{
- struct pnv_phb *phb = pdn->phb->private_data;
- u32 bdfn = (pdn->busno << 8) | pdn->devfn;
+ u32 bdfn = (busno << 8) | devfn;
s64 rc;
switch (size) {
case 1: {
u8 v8;
- rc = opal_pci_config_read_byte(phb->opal_id, bdfn, where, &v8);
+ rc = opal_pci_config_read_byte(phb_id, bdfn, where, &v8);
*val = (rc == OPAL_SUCCESS) ? v8 : 0xff;
break;
}
case 2: {
__be16 v16;
- rc = opal_pci_config_read_half_word(phb->opal_id, bdfn, where,
- &v16);
+ rc = opal_pci_config_read_half_word(phb_id, bdfn, where,
+ &v16);
*val = (rc == OPAL_SUCCESS) ? be16_to_cpu(v16) : 0xffff;
break;
}
case 4: {
__be32 v32;
- rc = opal_pci_config_read_word(phb->opal_id, bdfn, where, &v32);
+ rc = opal_pci_config_read_word(phb_id, bdfn, where, &v32);
*val = (rc == OPAL_SUCCESS) ? be32_to_cpu(v32) : 0xffffffff;
break;
}
@@ -679,27 +678,28 @@ int pnv_pci_cfg_read(struct pci_dn *pdn,
}
pr_devel("%s: bus: %x devfn: %x +%x/%x -> %08x\n",
- __func__, pdn->busno, pdn->devfn, where, size, *val);
+ __func__, busno, devfn, where, size, *val);
+
return PCIBIOS_SUCCESSFUL;
}
-int pnv_pci_cfg_write(struct pci_dn *pdn,
- int where, int size, u32 val)
+static int pnv_pci_cfg_write_raw(u64 phb_id, int busno, unsigned int devfn,
+ int where, int size, u32 val)
{
- struct pnv_phb *phb = pdn->phb->private_data;
- u32 bdfn = (pdn->busno << 8) | pdn->devfn;
+ u32 bdfn = (busno << 8) | devfn;
pr_devel("%s: bus: %x devfn: %x +%x/%x -> %08x\n",
- __func__, pdn->busno, pdn->devfn, where, size, val);
+ __func__, busno, devfn, where, size, val);
+
switch (size) {
case 1:
- opal_pci_config_write_byte(phb->opal_id, bdfn, where, val);
+ opal_pci_config_write_byte(phb_id, bdfn, where, val);
break;
case 2:
- opal_pci_config_write_half_word(phb->opal_id, bdfn, where, val);
+ opal_pci_config_write_half_word(phb_id, bdfn, where, val);
break;
case 4:
- opal_pci_config_write_word(phb->opal_id, bdfn, where, val);
+ opal_pci_config_write_word(phb_id, bdfn, where, val);
break;
default:
return PCIBIOS_FUNC_NOT_SUPPORTED;
@@ -708,6 +708,24 @@ int pnv_pci_cfg_write(struct pci_dn *pdn,
return PCIBIOS_SUCCESSFUL;
}
+int pnv_pci_cfg_read(struct pci_dn *pdn,
+ int where, int size, u32 *val)
+{
+ struct pnv_phb *phb = pdn->phb->private_data;
+
+ return pnv_pci_cfg_read_raw(phb->opal_id, pdn->busno, pdn->devfn,
+ where, size, val);
+}
+
+int pnv_pci_cfg_write(struct pci_dn *pdn,
+ int where, int size, u32 val)
+{
+ struct pnv_phb *phb = pdn->phb->private_data;
+
+ return pnv_pci_cfg_write_raw(phb->opal_id, pdn->busno, pdn->devfn,
+ where, size, val);
+}
+
#if CONFIG_EEH
static bool pnv_pci_cfg_check(struct pci_dn *pdn)
{
@@ -743,13 +761,15 @@ static int pnv_pci_read_config(struct pci_bus *bus,
int where, int size, u32 *val)
{
struct pci_dn *pdn;
- struct pnv_phb *phb;
+ struct pci_controller *hose = pci_bus_to_host(bus);
+ struct pnv_phb *phb = hose->private_data;
int ret;
*val = 0xFFFFFFFF;
pdn = pci_get_pdn_by_devfn(bus, devfn);
if (!pdn)
- return PCIBIOS_DEVICE_NOT_FOUND;
+ return pnv_pci_cfg_read_raw(phb->opal_id, bus->number, devfn,
+ where, size, val);
if (!pnv_pci_cfg_check(pdn))
return PCIBIOS_DEVICE_NOT_FOUND;
@@ -772,12 +792,14 @@ static int pnv_pci_write_config(struct pci_bus *bus,
int where, int size, u32 val)
{
struct pci_dn *pdn;
- struct pnv_phb *phb;
+ struct pci_controller *hose = pci_bus_to_host(bus);
+ struct pnv_phb *phb = hose->private_data;
int ret;
pdn = pci_get_pdn_by_devfn(bus, devfn);
if (!pdn)
- return PCIBIOS_DEVICE_NOT_FOUND;
+ return pnv_pci_cfg_write_raw(phb->opal_id, bus->number, devfn,
+ where, size, val);
if (!pnv_pci_cfg_check(pdn))
return PCIBIOS_DEVICE_NOT_FOUND;
--
2.21.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v6 2/5] powerpc/powernv/pci: Suppress an EEH error when reading an empty slot
2019-08-16 16:16 [PATCH v6 0/5] powerpc/powernv/pci: Make hotplug self-sufficient, independent of FW and DT Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 1/5] powerpc/pci: Access PCI config space directly w/o pci_dn Sergey Miroshnichenko
@ 2019-08-16 16:16 ` Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 3/5] powerpc/pci: Create pci_dn on demand Sergey Miroshnichenko
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Sergey Miroshnichenko @ 2019-08-16 16:16 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sam Bobroff, Sergey Miroshnichenko, Oliver O'Halloran, linux
Reading an empty slot returns all ones, which triggers a false
EEH error event on PowerNV. This patch unfreezes the bus where
it has happened.
Reviewed-by: Oliver O'Halloran <oohall@gmail.com>
Signed-off-by: Sergey Miroshnichenko <s.miroshnichenko@yadro.com>
---
arch/powerpc/include/asm/ppc-pci.h | 1 +
arch/powerpc/kernel/pci_dn.c | 2 +-
arch/powerpc/platforms/powernv/pci.c | 31 +++++++++++++++++++++++++---
3 files changed, 30 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/ppc-pci.h b/arch/powerpc/include/asm/ppc-pci.h
index cec2d6409515..8b51c8577b94 100644
--- a/arch/powerpc/include/asm/ppc-pci.h
+++ b/arch/powerpc/include/asm/ppc-pci.h
@@ -36,6 +36,7 @@ void *traverse_pci_dn(struct pci_dn *root,
void *(*fn)(struct pci_dn *, void *),
void *data);
extern void pci_devs_phb_init_dynamic(struct pci_controller *phb);
+struct pci_dn *pci_bus_to_pdn(struct pci_bus *bus);
/* From rtas_pci.h */
extern void init_pci_config_tokens (void);
diff --git a/arch/powerpc/kernel/pci_dn.c b/arch/powerpc/kernel/pci_dn.c
index c4c8c237a106..e1a0ab2caafe 100644
--- a/arch/powerpc/kernel/pci_dn.c
+++ b/arch/powerpc/kernel/pci_dn.c
@@ -27,7 +27,7 @@
* one of PF's bridge. For other devices, their firmware
* data is linked to that of their bridge.
*/
-static struct pci_dn *pci_bus_to_pdn(struct pci_bus *bus)
+struct pci_dn *pci_bus_to_pdn(struct pci_bus *bus)
{
struct pci_bus *pbus;
struct device_node *dn;
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 8d6c094f074e..a5b04410c8b4 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -756,6 +756,21 @@ static inline pnv_pci_cfg_check(struct pci_dn *pdn)
}
#endif /* CONFIG_EEH */
+static int get_bus_pe_number(struct pci_bus *bus)
+{
+ struct pci_dn *pdn = pci_bus_to_pdn(bus);
+ struct pci_dn *child;
+
+ if (!pdn)
+ return IODA_INVALID_PE;
+
+ list_for_each_entry(child, &pdn->child_list, list)
+ if (child->pe_number != IODA_INVALID_PE)
+ return child->pe_number;
+
+ return IODA_INVALID_PE;
+}
+
static int pnv_pci_read_config(struct pci_bus *bus,
unsigned int devfn,
int where, int size, u32 *val)
@@ -767,9 +782,19 @@ static int pnv_pci_read_config(struct pci_bus *bus,
*val = 0xFFFFFFFF;
pdn = pci_get_pdn_by_devfn(bus, devfn);
- if (!pdn)
- return pnv_pci_cfg_read_raw(phb->opal_id, bus->number, devfn,
- where, size, val);
+ if (!pdn) {
+ int pe_number = get_bus_pe_number(bus);
+
+ ret = pnv_pci_cfg_read_raw(phb->opal_id, bus->number, devfn,
+ where, size, val);
+
+ if (!ret && (*val == EEH_IO_ERROR_VALUE(size)) && phb->unfreeze_pe)
+ phb->unfreeze_pe(phb, (pe_number == IODA_INVALID_PE) ?
+ phb->ioda.reserved_pe_idx : pe_number,
+ OPAL_EEH_ACTION_CLEAR_FREEZE_ALL);
+
+ return ret;
+ }
if (!pnv_pci_cfg_check(pdn))
return PCIBIOS_DEVICE_NOT_FOUND;
--
2.21.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v6 3/5] powerpc/pci: Create pci_dn on demand
2019-08-16 16:16 [PATCH v6 0/5] powerpc/powernv/pci: Make hotplug self-sufficient, independent of FW and DT Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 1/5] powerpc/pci: Access PCI config space directly w/o pci_dn Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 2/5] powerpc/powernv/pci: Suppress an EEH error when reading an empty slot Sergey Miroshnichenko
@ 2019-08-16 16:16 ` Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 4/5] powerpc/powernv/pci: Hook up the writes to PCI_SECONDARY_BUS register Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 5/5] powerpc/pci: Enable assigning bus numbers instead of reading them from DT Sergey Miroshnichenko
4 siblings, 0 replies; 6+ messages in thread
From: Sergey Miroshnichenko @ 2019-08-16 16:16 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sam Bobroff, Sergey Miroshnichenko, Oliver O'Halloran, linux
If a struct pci_dn hasn't yet been created for the PCIe device (there was
no DT node for it), allocate this structure and fill with info read from
the device directly.
Signed-off-by: Sergey Miroshnichenko <s.miroshnichenko@yadro.com>
---
arch/powerpc/kernel/pci_dn.c | 88 ++++++++++++++++++++++++++++++------
1 file changed, 74 insertions(+), 14 deletions(-)
diff --git a/arch/powerpc/kernel/pci_dn.c b/arch/powerpc/kernel/pci_dn.c
index e1a0ab2caafe..261d61460eac 100644
--- a/arch/powerpc/kernel/pci_dn.c
+++ b/arch/powerpc/kernel/pci_dn.c
@@ -20,6 +20,9 @@
#include <asm/firmware.h>
#include <asm/eeh.h>
+static struct pci_dn *pci_create_pdn_from_dev(struct pci_dev *pdev,
+ struct pci_dn *parent);
+
/*
* The function is used to find the firmware data of one
* specific PCI device, which is attached to the indicated
@@ -52,6 +55,9 @@ struct pci_dn *pci_bus_to_pdn(struct pci_bus *bus)
dn = pci_bus_to_OF_node(pbus);
pdn = dn ? PCI_DN(dn) : NULL;
+ if (!pdn && pbus->self)
+ pdn = pbus->self->dev.archdata.pci_data;
+
return pdn;
}
@@ -61,10 +67,13 @@ struct pci_dn *pci_get_pdn_by_devfn(struct pci_bus *bus,
struct device_node *dn = NULL;
struct pci_dn *parent, *pdn;
struct pci_dev *pdev = NULL;
+ bool pdev_found = false;
/* Fast path: fetch from PCI device */
list_for_each_entry(pdev, &bus->devices, bus_list) {
if (pdev->devfn == devfn) {
+ pdev_found = true;
+
if (pdev->dev.archdata.pci_data)
return pdev->dev.archdata.pci_data;
@@ -73,6 +82,9 @@ struct pci_dn *pci_get_pdn_by_devfn(struct pci_bus *bus,
}
}
+ if (!pdev_found)
+ pdev = NULL;
+
/* Fast path: fetch from device node */
pdn = dn ? PCI_DN(dn) : NULL;
if (pdn)
@@ -85,9 +97,12 @@ struct pci_dn *pci_get_pdn_by_devfn(struct pci_bus *bus,
list_for_each_entry(pdn, &parent->child_list, list) {
if (pdn->busno == bus->number &&
- pdn->devfn == devfn)
- return pdn;
- }
+ pdn->devfn == devfn) {
+ if (pdev)
+ pdev->dev.archdata.pci_data = pdn;
+ return pdn;
+ }
+ }
return NULL;
}
@@ -117,17 +132,17 @@ struct pci_dn *pci_get_pdn(struct pci_dev *pdev)
list_for_each_entry(pdn, &parent->child_list, list) {
if (pdn->busno == pdev->bus->number &&
- pdn->devfn == pdev->devfn)
+ pdn->devfn == pdev->devfn) {
+ pdev->dev.archdata.pci_data = pdn;
return pdn;
+ }
}
- return NULL;
+ return pci_create_pdn_from_dev(pdev, parent);
}
-#ifdef CONFIG_PCI_IOV
-static struct pci_dn *add_one_dev_pci_data(struct pci_dn *parent,
- int vf_index,
- int busno, int devfn)
+static struct pci_dn *pci_alloc_pdn(struct pci_dn *parent,
+ int busno, int devfn)
{
struct pci_dn *pdn;
@@ -143,7 +158,6 @@ static struct pci_dn *add_one_dev_pci_data(struct pci_dn *parent,
pdn->parent = parent;
pdn->busno = busno;
pdn->devfn = devfn;
- pdn->vf_index = vf_index;
pdn->pe_number = IODA_INVALID_PE;
INIT_LIST_HEAD(&pdn->child_list);
INIT_LIST_HEAD(&pdn->list);
@@ -151,7 +165,51 @@ static struct pci_dn *add_one_dev_pci_data(struct pci_dn *parent,
return pdn;
}
-#endif
+
+static struct pci_dn *pci_create_pdn_from_dev(struct pci_dev *pdev,
+ struct pci_dn *parent)
+{
+ struct pci_dn *pdn = NULL;
+ u32 class_code;
+ u16 device_id;
+ u16 vendor_id;
+
+ if (!parent)
+ return NULL;
+
+ pdn = pci_alloc_pdn(parent, pdev->bus->busn_res.start, pdev->devfn);
+ pci_info(pdev, "Create a new pdn for devfn %2x\n", pdev->devfn / 8);
+
+ if (!pdn) {
+ pci_err(pdev, "%s: Failed to allocate pdn\n", __func__);
+ return NULL;
+ }
+
+ #ifdef CONFIG_EEH
+ if (!eeh_dev_init(pdn)) {
+ kfree(pdn);
+ pci_err(pdev, "%s: Failed to allocate edev\n", __func__);
+ return NULL;
+ }
+ #endif /* CONFIG_EEH */
+
+ pci_bus_read_config_word(pdev->bus, pdev->devfn,
+ PCI_VENDOR_ID, &vendor_id);
+ pdn->vendor_id = vendor_id;
+
+ pci_bus_read_config_word(pdev->bus, pdev->devfn,
+ PCI_DEVICE_ID, &device_id);
+ pdn->device_id = device_id;
+
+ pci_bus_read_config_dword(pdev->bus, pdev->devfn,
+ PCI_CLASS_REVISION, &class_code);
+ class_code >>= 8;
+ pdn->class_code = class_code;
+
+ pdev->dev.archdata.pci_data = pdn;
+
+ return pdn;
+}
struct pci_dn *add_dev_pci_data(struct pci_dev *pdev)
{
@@ -176,15 +234,17 @@ struct pci_dn *add_dev_pci_data(struct pci_dev *pdev)
for (i = 0; i < pci_sriov_get_totalvfs(pdev); i++) {
struct eeh_dev *edev __maybe_unused;
- pdn = add_one_dev_pci_data(parent, i,
- pci_iov_virtfn_bus(pdev, i),
- pci_iov_virtfn_devfn(pdev, i));
+ pdn = pci_alloc_pdn(parent,
+ pci_iov_virtfn_bus(pdev, i),
+ pci_iov_virtfn_devfn(pdev, i));
if (!pdn) {
dev_warn(&pdev->dev, "%s: Cannot create firmware data for VF#%d\n",
__func__, i);
return NULL;
}
+ pdn->vf_index = i;
+
#ifdef CONFIG_EEH
/* Create the EEH device for the VF */
edev = eeh_dev_init(pdn);
--
2.21.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v6 4/5] powerpc/powernv/pci: Hook up the writes to PCI_SECONDARY_BUS register
2019-08-16 16:16 [PATCH v6 0/5] powerpc/powernv/pci: Make hotplug self-sufficient, independent of FW and DT Sergey Miroshnichenko
` (2 preceding siblings ...)
2019-08-16 16:16 ` [PATCH v6 3/5] powerpc/pci: Create pci_dn on demand Sergey Miroshnichenko
@ 2019-08-16 16:16 ` Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 5/5] powerpc/pci: Enable assigning bus numbers instead of reading them from DT Sergey Miroshnichenko
4 siblings, 0 replies; 6+ messages in thread
From: Sergey Miroshnichenko @ 2019-08-16 16:16 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sam Bobroff, Sergey Miroshnichenko, Oliver O'Halloran, linux
Writing a new value to the PCI_SECONDARY_BUS register of the bridge means
that its children will become addressable on another address (new B in BDF)
or even un-addressable if the secondary bus is set to zero.
On PowerNV, device PEs are heavily BDF-dependent, so they must be updated
on every such change of its address.
Signed-off-by: Sergey Miroshnichenko <s.miroshnichenko@yadro.com>
---
arch/powerpc/platforms/powernv/pci.c | 118 ++++++++++++++++++++++++++-
1 file changed, 116 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index a5b04410c8b4..e9b4ed0f97a3 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -717,13 +717,127 @@ int pnv_pci_cfg_read(struct pci_dn *pdn,
where, size, val);
}
+static void invalidate_children_pes(struct pci_dn *pdn)
+{
+ struct pnv_phb *phb = pdn->phb->private_data;
+ struct pci_dn *child;
+ bool found_pe = false;
+ int pe_num;
+ int pe_bus;
+
+ list_for_each_entry(child, &pdn->child_list, list) {
+ struct pnv_ioda_pe *pe = (child->pe_number != IODA_INVALID_PE) ?
+ &phb->ioda.pe_array[child->pe_number] :
+ NULL;
+
+ if (!child->busno)
+ continue;
+
+ if ((child->class_code >> 8) == PCI_CLASS_BRIDGE_PCI)
+ invalidate_children_pes(child);
+
+ if (pe) {
+ u8 rid_bus = (pe->rid >> 8) & 0xff;
+
+ if (rid_bus) {
+ pe_num = child->pe_number;
+ pe_bus = rid_bus;
+ found_pe = true;
+ }
+
+ pe->rid &= 0xff;
+ }
+
+ child->busno = 0;
+ }
+
+ if (found_pe) {
+ u16 rid = pe_bus << 8;
+
+ opal_pci_set_pe(phb->opal_id, pe_num, rid, 7, 0, 0, OPAL_UNMAP_PE);
+ }
+}
+
+static u8 pre_hook_new_sec_bus(struct pci_dn *pdn, u8 new_secondary_bus)
+{
+ u32 old_secondary_bus = 0;
+
+ if ((pdn->class_code >> 8) != PCI_CLASS_BRIDGE_PCI)
+ return 0;
+
+ pnv_pci_cfg_read(pdn, PCI_SECONDARY_BUS, 1, &old_secondary_bus);
+ old_secondary_bus &= 0xff;
+
+ if (old_secondary_bus != new_secondary_bus)
+ invalidate_children_pes(pdn);
+
+ return old_secondary_bus;
+}
+
+static void update_children_pes(struct pci_dn *pdn, u8 new_secondary_bus)
+{
+ struct pnv_phb *phb = pdn->phb->private_data;
+ struct pci_dn *child;
+ bool found_pe = false;
+ int pe_num;
+
+ if (!new_secondary_bus)
+ return;
+
+ list_for_each_entry(child, &pdn->child_list, list) {
+ struct pnv_ioda_pe *pe = (child->pe_number != IODA_INVALID_PE) ?
+ &phb->ioda.pe_array[child->pe_number] :
+ NULL;
+
+ if (child->busno)
+ continue;
+
+ child->busno = new_secondary_bus;
+
+ if (pe) {
+ pe->rid |= (child->busno << 8);
+ pe_num = child->pe_number;
+ found_pe = true;
+ }
+ }
+
+ if (found_pe) {
+ u16 rid = new_secondary_bus << 8;
+
+ opal_pci_set_pe(phb->opal_id, pe_num, rid, 7, 0, 0, OPAL_MAP_PE);
+ }
+}
+
+static void post_hook_new_sec_bus(struct pci_dn *pdn, u8 new_secondary_bus)
+{
+ if ((pdn->class_code >> 8) != PCI_CLASS_BRIDGE_PCI)
+ return;
+
+ update_children_pes(pdn, new_secondary_bus);
+}
+
int pnv_pci_cfg_write(struct pci_dn *pdn,
int where, int size, u32 val)
{
struct pnv_phb *phb = pdn->phb->private_data;
+ u8 old_secondary_bus = 0, new_secondary_bus = 0;
+ int rc;
+
+ if (where == PCI_SECONDARY_BUS) {
+ new_secondary_bus = val & 0xff;
+ old_secondary_bus = pre_hook_new_sec_bus(pdn, new_secondary_bus);
+ } else if (where == PCI_PRIMARY_BUS && size > 1) {
+ new_secondary_bus = (val >> 8) & 0xff;
+ old_secondary_bus = pre_hook_new_sec_bus(pdn, new_secondary_bus);
+ }
- return pnv_pci_cfg_write_raw(phb->opal_id, pdn->busno, pdn->devfn,
- where, size, val);
+ rc = pnv_pci_cfg_write_raw(phb->opal_id, pdn->busno, pdn->devfn,
+ where, size, val);
+
+ if (new_secondary_bus && old_secondary_bus != new_secondary_bus)
+ post_hook_new_sec_bus(pdn, new_secondary_bus);
+
+ return rc;
}
#if CONFIG_EEH
--
2.21.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v6 5/5] powerpc/pci: Enable assigning bus numbers instead of reading them from DT
2019-08-16 16:16 [PATCH v6 0/5] powerpc/powernv/pci: Make hotplug self-sufficient, independent of FW and DT Sergey Miroshnichenko
` (3 preceding siblings ...)
2019-08-16 16:16 ` [PATCH v6 4/5] powerpc/powernv/pci: Hook up the writes to PCI_SECONDARY_BUS register Sergey Miroshnichenko
@ 2019-08-16 16:16 ` Sergey Miroshnichenko
4 siblings, 0 replies; 6+ messages in thread
From: Sergey Miroshnichenko @ 2019-08-16 16:16 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sam Bobroff, Sergey Miroshnichenko, Oliver O'Halloran, linux
If the firmware indicates support of reassigning bus numbers via the PHB's
"ibm,supported-movable-bdfs" property in DT, PowerNV will not depend on PCI
topology info from DT anymore.
This makes possible to re-enumerate the fabric, assign the new bus numbers
and switch from the pnv_php module to the standard pciehp driver for PCI
hotplug functionality.
Signed-off-by: Sergey Miroshnichenko <s.miroshnichenko@yadro.com>
---
arch/powerpc/kernel/pci_dn.c | 5 +++++
arch/powerpc/platforms/powernv/eeh-powernv.c | 2 +-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/pci_dn.c b/arch/powerpc/kernel/pci_dn.c
index 261d61460eac..90f8d46550df 100644
--- a/arch/powerpc/kernel/pci_dn.c
+++ b/arch/powerpc/kernel/pci_dn.c
@@ -542,6 +542,11 @@ void pci_devs_phb_init_dynamic(struct pci_controller *phb)
phb->pci_data = pdn;
}
+ if (of_get_property(dn, "ibm,supported-movable-bdfs", NULL)) {
+ pci_add_flags(PCI_REASSIGN_ALL_BUS);
+ return;
+ }
+
/* Update dn->phb ptrs for new phb and children devices */
pci_traverse_device_nodes(dn, add_pdn, phb);
}
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 620a986209f5..eb01f16c4e60 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -41,7 +41,7 @@ void pnv_pcibios_bus_add_device(struct pci_dev *pdev)
{
struct pci_dn *pdn = pci_get_pdn(pdev);
- if (!pdev->is_virtfn)
+ if (!pdev->is_virtfn && !pci_has_flag(PCI_REASSIGN_ALL_BUS))
return;
/*
--
2.21.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-08-16 16:30 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-16 16:16 [PATCH v6 0/5] powerpc/powernv/pci: Make hotplug self-sufficient, independent of FW and DT Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 1/5] powerpc/pci: Access PCI config space directly w/o pci_dn Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 2/5] powerpc/powernv/pci: Suppress an EEH error when reading an empty slot Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 3/5] powerpc/pci: Create pci_dn on demand Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 4/5] powerpc/powernv/pci: Hook up the writes to PCI_SECONDARY_BUS register Sergey Miroshnichenko
2019-08-16 16:16 ` [PATCH v6 5/5] powerpc/pci: Enable assigning bus numbers instead of reading them from DT Sergey Miroshnichenko
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.