From: Frederic Barrat <fbarrat@linux.ibm.com>
To: Alistair Popple <alistair@popple.id.au>, linuxppc-dev@lists.ozlabs.org
Cc: Andrew Donnellan <ajd@linux.ibm.com>,
Alexey Kardashevskiy <aik@ozlabs.ru>, Greg Kurz <groug@kaod.org>,
Christophe Lombard <clombard@linux.ibm.com>,
Oliver O'Halloran <oohall@gmail.com>,
Andrew Donnellan <andrew.donnellan@au1.ibm.com>,
Reza Arbab <arbab@linux.ibm.com>,
Alastair D'Silva <alastair@au1.ibm.com>
Subject: Re: [PATCH 01/11] powerpc/powernv/ioda: Fix ref count for devices with their own PE
Date: Mon, 18 Nov 2019 19:21:08 +0100 [thread overview]
Message-ID: <c07ed277-b527-3b1c-9cf8-aecdb7145a5e@linux.ibm.com> (raw)
In-Reply-To: <2783297.m1VkgVQdWf@townsend>
Le 18/11/2019 à 03:36, Alistair Popple a écrit :
> On Monday, 18 November 2019 12:24:24 PM AEDT Oliver O'Halloran wrote:
>> On Mon, Nov 18, 2019 at 12:06 PM Alistair Popple <alistair@popple.id.au>
> wrote:
>>>
>>> On Wednesday, 13 November 2019 4:38:21 AM AEDT Frederic Barrat wrote:
>>>>
>>>> However, one question is whether this patch breaks nvlink and if nvlink
>>>> assumes the devices won’t go away because we explicitly take a reference
>>>> forever. In npu_dma.c, there are 2 functions which allow to find the GPU
>>>> associated to a npu device, and vice-versa. Both functions return a
>>>> pointer to a struct pci_dev, but they don’t take a reference on the
>>>> device being returned. So that seems dangerous. I’m probably missing
>>>> something.
>>>>
>>>> Alexey, Alistair: what, if anything, guarantees, that the npu or gpu
>>>> devices stay valid. Is it because we simply don’t provide any means to
>>>> get rid of them ? Otherwise, don’t we need the callers of
>>>> pnv_pci_get_gpu_dev() and pnv_pci_get_npu_dev() to worry about reference
>>>> counting ? I’ve started looking into it and the changes are scary, which
>>>> explains Greg’s related commit 02c5f5394918b.
>>>
>>> To be honest the reference counting looks like it has evolved into
> something
>>> quite suspect and I don't think you're missing anything. In practice
> though we
>>> likely haven't hit any issues because the original callers didn't store
>>> references to the pdev which would make the window quite small (although
> the
>>> pass through work may have changed that). And as you say there simply
> wasn't
>>> any means to get rid of them anyway (EEH, hotplug, etc. has never been
>>> implemented or supported for GPUs and all sorts of terrible things happen
> if
>>> you try).
>>
>> In other words: leaking a ref is the only safe thing to do.
>
> Correct.
>
>>> The best solution would likely be to review the reference counting and to
>>> teach callers of get_*_dev() to call pci_put_dev(), etc.
>>
>> The issue is that the two callers of get_pci_dev() are non-GPL
>> exported symbols so we don't know what's calling them or what their
>> expectations are. We be doing whatever makes sense for OpenCAPI and if
>> that happens to cause a problem for someone else, then they can deal
>> with it.
>
> The issue isn't that it's exported non-GPL vs. GPL, rather that there are
> probably out-of-tree modules like the NVIDIA driver using it. However as you
> say supporting out-of-tree modules is not generally a concern for kernel
> developers so we certainly shouldn't let that prevent us doing a fix.
Thanks Alistair and Oliver. I'll bite the bullet and try do the right
thing wrt ref counting in npu-dma.c
Fred
> - Alistair
>
>> Oliver
>
>
>
>
next prev parent reply other threads:[~2019-11-18 18:23 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-09 15:45 [PATCH 00/11] opencapi: enable card reset and link retraining Frederic Barrat
2019-09-09 15:45 ` [PATCH 01/11] powerpc/powernv/ioda: Fix ref count for devices with their own PE Frederic Barrat
2019-09-10 0:20 ` Alastair D'Silva
2019-09-26 16:44 ` Andrew Donnellan
2019-09-26 17:15 ` Frederic Barrat
2019-09-27 6:54 ` Alexey Kardashevskiy
2019-11-12 17:38 ` Frederic Barrat
2019-11-18 1:04 ` Alistair Popple
2019-11-18 1:24 ` Oliver O'Halloran
2019-11-18 2:36 ` Alistair Popple
2019-11-18 18:21 ` Frederic Barrat [this message]
2019-09-09 15:45 ` [PATCH 02/11] powerpc/powernv/ioda: Protect PE list Frederic Barrat
2019-09-10 0:34 ` Alastair D'Silva
2019-11-19 12:55 ` Frederic Barrat
2019-11-19 13:22 ` Oliver O'Halloran
2019-11-19 14:36 ` Frederic Barrat
2019-11-19 5:55 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 03/11] powerpc/powernv/ioda: set up PE on opencapi device when enabling Frederic Barrat
2019-09-10 0:38 ` Alastair D'Silva
2019-09-27 16:43 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 04/11] powerpc/powernv/ioda: Release opencapi device Frederic Barrat
2019-09-10 0:56 ` Alastair D'Silva
2019-11-19 17:32 ` Frederic Barrat
2019-11-19 7:08 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 05/11] powerpc/powernv/ioda: Find opencapi slot for a device node Frederic Barrat
2019-09-10 0:57 ` Alastair D'Silva
2019-11-19 1:26 ` Andrew Donnellan
2019-11-19 14:33 ` Frederic Barrat
2019-09-09 15:45 ` [PATCH 06/11] pci/hotplug/pnv-php: Remove erroneous warning Frederic Barrat
2019-09-10 0:58 ` Alastair D'Silva
2019-11-19 5:00 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 07/11] pci/hotplug/pnv-php: Improve error msg on power state change failure Frederic Barrat
2019-09-10 0:59 ` Alastair D'Silva
2019-11-19 2:23 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 08/11] pci/hotplug/pnv-php: Register opencapi slots Frederic Barrat
2019-09-10 1:00 ` Alastair D'Silva
2019-11-19 5:18 ` Andrew Donnellan
2019-11-19 15:15 ` Frederic Barrat
2019-09-09 15:45 ` [PATCH 09/11] pci/hotplug/pnv-php: Relax check when disabling slot Frederic Barrat
2019-09-10 1:00 ` Alastair D'Silva
2019-09-27 15:56 ` Andrew Donnellan
2019-09-09 15:45 ` [PATCH 10/11] pci/hotplug/pnv-php: Wrap warnings in macro Frederic Barrat
2019-09-10 1:03 ` Alastair D'Silva
2019-09-26 17:18 ` Andrew Donnellan
2019-09-09 15:46 ` [PATCH 11/11] ocxl: Add PCI hotplug dependency to Kconfig Frederic Barrat
2019-09-10 1:03 ` Alastair D'Silva
2019-09-26 17:11 ` Andrew Donnellan
2019-09-24 4:24 ` [PATCH 00/11] opencapi: enable card reset and link retraining Alexey Kardashevskiy
2019-09-24 6:39 ` Frederic Barrat
2019-09-25 0:20 ` Alexey Kardashevskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c07ed277-b527-3b1c-9cf8-aecdb7145a5e@linux.ibm.com \
--to=fbarrat@linux.ibm.com \
--cc=aik@ozlabs.ru \
--cc=ajd@linux.ibm.com \
--cc=alastair@au1.ibm.com \
--cc=alistair@popple.id.au \
--cc=andrew.donnellan@au1.ibm.com \
--cc=arbab@linux.ibm.com \
--cc=clombard@linux.ibm.com \
--cc=groug@kaod.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=oohall@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).