From: Frederic Barrat <fbarrat@linux.ibm.com>
To: linuxppc-dev@lists.ozlabs.org, andrew.donnellan@au1.ibm.com,
clombard@linux.ibm.com
Cc: aik@ozlabs.ru, Frederic Barrat <fbarrat@linux.ibm.com>,
oohall@gmail.com, groug@kaod.org, alastair@au1.ibm.com
Subject: [PATCH v2 01/11] powerpc/powernv/ioda: Fix ref count for devices with their own PE
Date: Thu, 21 Nov 2019 14:49:08 +0100 [thread overview]
Message-ID: <20191121134918.7155-2-fbarrat@linux.ibm.com> (raw)
In-Reply-To: <20191121134918.7155-1-fbarrat@linux.ibm.com>
The pci_dn structure used to store a pointer to the struct pci_dev, so
taking a reference on the device was required. However, the pci_dev
pointer was later removed from the pci_dn structure, but the reference
was kept for the npu device.
See commit 902bdc57451c ("powerpc/powernv/idoa: Remove unnecessary
pcidev from pci_dn").
We don't need to take a reference on the device when assigning the PE
as the struct pnv_ioda_pe is cleaned up at the same time as
the (physical) device is released. Doing so prevents the device from
being released, which is a problem for opencapi devices, since we want
to be able to remove them through PCI hotplug.
Now the ugly part: nvlink npu devices are not meant to be
released. Because of the above, we've always leaked a reference and
simply removing it now is dangerous and would likely require more
work. There's currently no release device callback for nvlink devices
for example. So to be safe, this patch leaks a reference on the npu
device, but only for nvlink and not opencapi.
CC: aik@ozlabs.ru
CC: oohall@gmail.com
Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com>
---
Changelog:
v2:
- clarify the ref counting (and leak) done on npu devices when
setting up the PE
- rework commit message
arch/powerpc/platforms/powernv/pci-ioda.c | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index c28d0d9b7ee0..155333872cb4 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1062,14 +1062,13 @@ static struct pnv_ioda_pe *pnv_ioda_setup_dev_PE(struct pci_dev *dev)
return NULL;
}
- /* NOTE: We get only one ref to the pci_dev for the pdn, not for the
- * pointer in the PE data structure, both should be destroyed at the
- * same time. However, this needs to be looked at more closely again
- * once we actually start removing things (Hotplug, SR-IOV, ...)
+ /* NOTE: We don't get a reference for the pointer in the PE
+ * data structure, both the device and PE structures should be
+ * destroyed at the same time. However, removing nvlink
+ * devices will need some work.
*
* At some point we want to remove the PDN completely anyways
*/
- pci_dev_get(dev);
pdn->pe_number = pe->pe_number;
pe->flags = PNV_IODA_PE_DEV;
pe->pdev = dev;
@@ -1084,7 +1083,6 @@ static struct pnv_ioda_pe *pnv_ioda_setup_dev_PE(struct pci_dev *dev)
pnv_ioda_free_pe(pe);
pdn->pe_number = IODA_INVALID_PE;
pe->pdev = NULL;
- pci_dev_put(dev);
return NULL;
}
@@ -1205,6 +1203,14 @@ static struct pnv_ioda_pe *pnv_ioda_setup_npu_PE(struct pci_dev *npu_pdev)
struct pci_controller *hose = pci_bus_to_host(npu_pdev->bus);
struct pnv_phb *phb = hose->private_data;
+ /*
+ * Intentionally leak a reference on the npu device (for
+ * nvlink only; this is not an opencapi path) to make sure it
+ * never goes away, as it's been the case all along and some
+ * work is needed otherwise.
+ */
+ pci_dev_get(npu_pdev);
+
/*
* Due to a hardware errata PE#0 on the NPU is reserved for
* error handling. This means we only have three PEs remaining
@@ -1228,7 +1234,6 @@ static struct pnv_ioda_pe *pnv_ioda_setup_npu_PE(struct pci_dev *npu_pdev)
*/
dev_info(&npu_pdev->dev,
"Associating to existing PE %x\n", pe_num);
- pci_dev_get(npu_pdev);
npu_pdn = pci_get_pdn(npu_pdev);
rid = npu_pdev->bus->number << 8 | npu_pdn->devfn;
npu_pdn->pe_number = pe_num;
--
2.21.0
next prev parent reply other threads:[~2019-11-21 13:53 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-21 13:49 [PATCH v2 00/11] opencapi: enable card reset and link retraining Frederic Barrat
2019-11-21 13:49 ` Frederic Barrat [this message]
2020-01-08 7:33 ` [PATCH v2 01/11] powerpc/powernv/ioda: Fix ref count for devices with their own PE Andrew Donnellan
2020-01-08 14:11 ` Frederic Barrat
2020-01-08 20:19 ` Andrew Donnellan
2020-01-29 5:17 ` Michael Ellerman
2019-11-21 13:49 ` [PATCH v2 02/11] powerpc/powernv/ioda: Protect PE list Frederic Barrat
2019-11-21 13:49 ` [PATCH v2 03/11] powerpc/powernv/ioda: set up PE on opencapi device when enabling Frederic Barrat
2019-11-21 13:49 ` [PATCH v2 04/11] powerpc/powernv/ioda: Release opencapi device Frederic Barrat
2019-11-22 5:22 ` Andrew Donnellan
2019-11-21 13:49 ` [PATCH v2 05/11] powerpc/powernv/ioda: Find opencapi slot for a device node Frederic Barrat
2019-11-22 4:27 ` Andrew Donnellan
2019-11-21 13:49 ` [PATCH v2 06/11] pci/hotplug/pnv-php: Remove erroneous warning Frederic Barrat
2019-11-21 13:49 ` [PATCH v2 07/11] pci/hotplug/pnv-php: Improve error msg on power state change failure Frederic Barrat
2019-11-21 13:49 ` [PATCH v2 08/11] pci/hotplug/pnv-php: Register opencapi slots Frederic Barrat
2019-11-22 4:33 ` Andrew Donnellan
2019-11-21 13:49 ` [PATCH v2 09/11] pci/hotplug/pnv-php: Relax check when disabling slot Frederic Barrat
2019-11-21 13:49 ` [PATCH v2 10/11] pci/hotplug/pnv-php: Wrap warnings in macro Frederic Barrat
2019-11-21 13:49 ` [PATCH v2 11/11] ocxl: Add PCI hotplug dependency to Kconfig Frederic Barrat
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191121134918.7155-2-fbarrat@linux.ibm.com \
--to=fbarrat@linux.ibm.com \
--cc=aik@ozlabs.ru \
--cc=alastair@au1.ibm.com \
--cc=andrew.donnellan@au1.ibm.com \
--cc=clombard@linux.ibm.com \
--cc=groug@kaod.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=oohall@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.