From: "Alastair D'Silva" <alastair@au1.ibm.com>
To: Frederic Barrat <fbarrat@linux.ibm.com>,
linuxppc-dev@lists.ozlabs.org, andrew.donnellan@au1.ibm.com,
clombard@linux.ibm.com
Cc: groug@kaod.org
Subject: Re: [PATCH 04/11] powerpc/powernv/ioda: Release opencapi device
Date: Tue, 10 Sep 2019 10:56:58 +1000 [thread overview]
Message-ID: <05ebc83c696cd3c5e65d64e76132371af2fb81d5.camel@au1.ibm.com> (raw)
In-Reply-To: <20190909154600.19917-5-fbarrat@linux.ibm.com>
On Mon, 2019-09-09 at 17:45 +0200, Frederic Barrat wrote:
> With hotplug, an opencapi device can now go away. It needs to be
> released, mostly to clean up its PE state. We were previously not
> defining any device callback. We can reuse the standard PCI release
> callback, it does a bit too much for an opencapi device, but it's
> harmless, and only needs minor tuning.
>
> Also separate the undo of the PELT-V code in a separate function, it
> is not needed for NPU devices and it improves a bit the readability
> of
> the code.
>
> Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com>
> ---
> arch/powerpc/platforms/powernv/pci-ioda.c | 59 +++++++++++++++----
> ----
> 1 file changed, 39 insertions(+), 20 deletions(-)
>
> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c
> b/arch/powerpc/platforms/powernv/pci-ioda.c
> index 06ce7ddaa0cf..e5895c05efae 100644
> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> @@ -188,7 +188,7 @@ static void pnv_ioda_free_pe(struct pnv_ioda_pe
> *pe)
> unsigned int pe_num = pe->pe_number;
>
> WARN_ON(pe->pdev);
> - WARN_ON(pe->npucomp); /* NPUs are not supposed to be freed */
> + WARN_ON(pe->npucomp); /* NPUs for nvlink are not supposed to be
> freed */
> kfree(pe->npucomp);
> memset(pe, 0, sizeof(struct pnv_ioda_pe));
> clear_bit(pe_num, phb->ioda.pe_alloc);
> @@ -777,6 +777,34 @@ static int pnv_ioda_set_peltv(struct pnv_phb
> *phb,
> return 0;
> }
>
> +static void pnv_ioda_unset_peltv(struct pnv_phb *phb,
> + struct pnv_ioda_pe *pe,
> + struct pci_dev *parent)
> +{
> + int64_t rc;
> +
> + while (parent) {
> + struct pci_dn *pdn = pci_get_pdn(parent);
> +
> + if (pdn && pdn->pe_number != IODA_INVALID_PE) {
> + rc = opal_pci_set_peltv(phb->opal_id, pdn-
> >pe_number,
> + pe->pe_number,
> + OPAL_REMOVE_PE_FROM_DOM
> AIN);
> + /* XXX What to do in case of error ? */
Can we take the opportunity to address this comment?
> + }
> + parent = parent->bus->self;
> + }
> +
> + opal_pci_eeh_freeze_clear(phb->opal_id, pe->pe_number,
> + OPAL_EEH_ACTION_CLEAR_FREEZE_ALL);
> +
> + /* Disassociate PE in PELT */
> + rc = opal_pci_set_peltv(phb->opal_id, pe->pe_number,
> + pe->pe_number,
> OPAL_REMOVE_PE_FROM_DOMAIN);
> + if (rc)
> + pe_warn(pe, "OPAL error %lld remove self from PELTV\n",
> rc);
> +}
> +
> static int pnv_ioda_deconfigure_pe(struct pnv_phb *phb, struct
> pnv_ioda_pe *pe)
> {
> struct pci_dev *parent;
> @@ -827,25 +855,13 @@ static int pnv_ioda_deconfigure_pe(struct
> pnv_phb *phb, struct pnv_ioda_pe *pe)
> for (rid = pe->rid; rid < rid_end; rid++)
> phb->ioda.pe_rmap[rid] = IODA_INVALID_PE;
>
> - /* Release from all parents PELT-V */
> - while (parent) {
> - struct pci_dn *pdn = pci_get_pdn(parent);
> - if (pdn && pdn->pe_number != IODA_INVALID_PE) {
> - rc = opal_pci_set_peltv(phb->opal_id, pdn-
> >pe_number,
> - pe->pe_number,
> OPAL_REMOVE_PE_FROM_DOMAIN);
> - /* XXX What to do in case of error ? */
> - }
> - parent = parent->bus->self;
> - }
> -
> - opal_pci_eeh_freeze_clear(phb->opal_id, pe->pe_number,
> - OPAL_EEH_ACTION_CLEAR_FREEZE_ALL);
> + /*
> + * Release from all parents PELT-V. NPUs don't have a PELTV
> + * table
> + */
> + if (phb->type != PNV_PHB_NPU_NVLINK && phb->type !=
> PNV_PHB_NPU_OCAPI)
> + pnv_ioda_unset_peltv(phb, pe, parent);
>
> - /* Disassociate PE in PELT */
> - rc = opal_pci_set_peltv(phb->opal_id, pe->pe_number,
> - pe->pe_number,
> OPAL_REMOVE_PE_FROM_DOMAIN);
> - if (rc)
> - pe_warn(pe, "OPAL error %lld remove self from PELTV\n",
> rc);
> rc = opal_pci_set_pe(phb->opal_id, pe->pe_number, pe->rid,
> bcomp, dcomp, fcomp, OPAL_UNMAP_PE);
> if (rc)
> @@ -3540,6 +3556,8 @@ static void pnv_ioda_release_pe(struct
> pnv_ioda_pe *pe)
> case PNV_PHB_IODA2:
> pnv_pci_ioda2_release_pe_dma(pe);
> break;
> + case PNV_PHB_NPU_OCAPI:
> + break;
> default:
> WARN_ON(1);
> }
> @@ -3592,7 +3610,7 @@ static void pnv_pci_release_device(struct
> pci_dev *pdev)
> pe = &phb->ioda.pe_array[pdn->pe_number];
> pdn->pe_number = IODA_INVALID_PE;
>
> - WARN_ON(--pe->device_count < 0);
> + WARN_ON((pe->flags != PNV_IODA_PE_DEV) && (--pe->device_count <
> 0));
> if (pe->device_count == 0)
> pnv_ioda_release_pe(pe);
> }
> @@ -3641,6 +3659,7 @@ static const struct pci_controller_ops
> pnv_npu_ioda_controller_ops = {
>
> static const struct pci_controller_ops
> pnv_npu_ocapi_ioda_controller_ops = {
> .enable_device_hook = pnv_ocapi_enable_device_hook,
> + .release_device = pnv_pci_release_device,
> .window_alignment = pnv_pci_window_alignment,
> .reset_secondary_bus = pnv_pci_reset_secondary_bus,
> .shutdown = pnv_pci_ioda_shutdown,
--
Alastair D'Silva
Open Source Developer
Linux Technology Centre, IBM Australia
mob: 0423 762 819
next prev parent reply other threads:[~2019-09-10 0:58 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-09 15:45 [PATCH 00/11] opencapi: enable card reset and link retraining Frederic Barrat
2019-09-09 15:45 ` [PATCH 01/11] powerpc/powernv/ioda: Fix ref count for devices with their own PE Frederic Barrat
2019-09-10 0:20 ` Alastair D'Silva
2019-09-26 16:44 ` Andrew Donnellan
2019-09-26 17:15 ` Frederic Barrat
2019-09-27 6:54 ` Alexey Kardashevskiy
2019-11-12 17:38 ` Frederic Barrat
2019-11-18 1:04 ` Alistair Popple
2019-11-18 1:24 ` Oliver O'Halloran
2019-11-18 2:36 ` Alistair Popple
2019-11-18 18:21 ` Frederic Barrat
2019-09-09 15:45 ` [PATCH 02/11] powerpc/powernv/ioda: Protect PE list Frederic Barrat
2019-09-10 0:34 ` Alastair D'Silva
2019-11-19 12:55 ` Frederic Barrat
2019-11-19 13:22 ` Oliver O'Halloran
2019-11-19 14:36 ` Frederic Barrat
2019-11-19 5:55 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 03/11] powerpc/powernv/ioda: set up PE on opencapi device when enabling Frederic Barrat
2019-09-10 0:38 ` Alastair D'Silva
2019-09-27 16:43 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 04/11] powerpc/powernv/ioda: Release opencapi device Frederic Barrat
2019-09-10 0:56 ` Alastair D'Silva [this message]
2019-11-19 17:32 ` Frederic Barrat
2019-11-19 7:08 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 05/11] powerpc/powernv/ioda: Find opencapi slot for a device node Frederic Barrat
2019-09-10 0:57 ` Alastair D'Silva
2019-11-19 1:26 ` Andrew Donnellan
2019-11-19 14:33 ` Frederic Barrat
2019-09-09 15:45 ` [PATCH 06/11] pci/hotplug/pnv-php: Remove erroneous warning Frederic Barrat
2019-09-10 0:58 ` Alastair D'Silva
2019-11-19 5:00 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 07/11] pci/hotplug/pnv-php: Improve error msg on power state change failure Frederic Barrat
2019-09-10 0:59 ` Alastair D'Silva
2019-11-19 2:23 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 08/11] pci/hotplug/pnv-php: Register opencapi slots Frederic Barrat
2019-09-10 1:00 ` Alastair D'Silva
2019-11-19 5:18 ` Andrew Donnellan
2019-11-19 15:15 ` Frederic Barrat
2019-09-09 15:45 ` [PATCH 09/11] pci/hotplug/pnv-php: Relax check when disabling slot Frederic Barrat
2019-09-10 1:00 ` Alastair D'Silva
2019-09-27 15:56 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 10/11] pci/hotplug/pnv-php: Wrap warnings in macro Frederic Barrat
2019-09-10 1:03 ` Alastair D'Silva
2019-09-26 17:18 ` Andrew Donnellan
2019-09-09 15:46 ` [PATCH 11/11] ocxl: Add PCI hotplug dependency to Kconfig Frederic Barrat
2019-09-10 1:03 ` Alastair D'Silva
2019-09-26 17:11 ` Andrew Donnellan
2019-09-24 4:24 ` [PATCH 00/11] opencapi: enable card reset and link retraining Alexey Kardashevskiy
2019-09-24 6:39 ` Frederic Barrat
2019-09-25 0:20 ` Alexey Kardashevskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=05ebc83c696cd3c5e65d64e76132371af2fb81d5.camel@au1.ibm.com \
--to=alastair@au1.ibm.com \
--cc=andrew.donnellan@au1.ibm.com \
--cc=clombard@linux.ibm.com \
--cc=fbarrat@linux.ibm.com \
--cc=groug@kaod.org \
--cc=linuxppc-dev@lists.ozlabs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).