Re: [PATCH 01/11] powerpc/powernv/ioda: Fix ref count for devices with their own PE

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 27/09/2019 03:15, Frederic Barrat wrote:
> 
> 
> Le 26/09/2019 à 18:44, Andrew Donnellan a écrit :
>> On 9/9/19 5:45 pm, Frederic Barrat wrote:
>>> Taking a reference on the pci_dev structure was required with initial
>>> commit 184cd4a3b962 ("powerpc/powernv: PCI support for p7IOC under
>>> OPAL v2"), where we were storing the pci_dev in the pci_dn structure.
>>>
>>> However, the pci_dev was later removed from the pci_dn structure, but
>>> the reference was kept. See commit 902bdc57451c ("powerpc/powernv/idoa:
>>> Remove unnecessary pcidev from pci_dn").
>>>
>>> The pnv_ioda_pe structure life cycle is the same as the pci_dev
>>> structure, the PE is freed when the device is released. So we don't
>>> need a reference for the pci_dev stored in the PE, otherwise the
>>> pci_dev will never be released. Which is not really a surprise as the
>>> comment (removed here as no longer needed) was stating as much.
>>>
>>> Fixes: 902bdc57451c ("powerpc/powernv/idoa: Remove unnecessary pcidev from pci_dn")
>>> Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com>
>>> ---
>>>   arch/powerpc/platforms/powernv/pci-ioda.c | 11 +----------
>>>   1 file changed, 1 insertion(+), 10 deletions(-)
>>>
>>> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
>>> index d8080558d020..92767f006f20 100644
>>> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
>>> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
>>> @@ -1062,14 +1062,6 @@ static struct pnv_ioda_pe *pnv_ioda_setup_dev_PE(struct pci_dev *dev)
>>>           return NULL;
>>>       }
>>> -    /* NOTE: We get only one ref to the pci_dev for the pdn, not for the
>>> -     * pointer in the PE data structure, both should be destroyed at the
>>> -     * same time. However, this needs to be looked at more closely again
>>> -     * once we actually start removing things (Hotplug, SR-IOV, ...)
>>> -     *
>>> -     * At some point we want to remove the PDN completely anyways
>>> -     */
>>> -    pci_dev_get(dev);
>>>       pdn->pe_number = pe->pe_number;
>>>       pe->flags = PNV_IODA_PE_DEV;
>>>       pe->pdev = dev;
>>> @@ -1084,7 +1076,6 @@ static struct pnv_ioda_pe *pnv_ioda_setup_dev_PE(struct pci_dev *dev)
>>>           pnv_ioda_free_pe(pe);
>>>           pdn->pe_number = IODA_INVALID_PE;
>>>           pe->pdev = NULL;
>>> -        pci_dev_put(dev);
>>>           return NULL;
>>>       }
>>> @@ -1228,7 +1219,7 @@ static struct pnv_ioda_pe *pnv_ioda_setup_npu_PE(struct pci_dev *npu_pdev)
>>>                */
>>>               dev_info(&npu_pdev->dev,
>>>                   "Associating to existing PE %x\n", pe_num);
>>> -            pci_dev_get(npu_pdev);
>>> +            pci_dev_get(npu_pdev); // still needed after 902bdc57451c ?
>>
>> Did you mean to leave that comment in?
> 
> Yes, I assumed the series wouldn't get in on the first try and a nvlink-minded developer would weigh in :)

I am looking at it and I am still not so sure what exactly guarantees that lifetime(@dev) == lifetime(@pe). For example,
sriov_disable() removes VFs first, and only then releases PEs so there is a window when we may have a stale pdev. Not sure.


> 
>   Fred
> 
> 
>>>               npu_pdn = pci_get_pdn(npu_pdev);
>>>               rid = npu_pdev->bus->number << 8 | npu_pdn->devfn;
>>>               npu_pdn->pe_number = pe_num;
>>>
>>
> 

-- 
Alexey