* [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]
@ 2020-02-06 3:17 Sam Bobroff
2020-02-06 4:23 ` Alexey Kardashevskiy
0 siblings, 1 reply; 4+ messages in thread
From: Sam Bobroff @ 2020-02-06 3:17 UTC (permalink / raw)
To: kvm, linuxppc-dev; +Cc: aik
Older versions of skiboot only provide a single value in the device
tree property "ibm,mmio-atsd", even when multiple Address Translation
Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
(other than the first) from being used with vfio-pci because vfio-pci
expects to be able to assign a dedicated ATSD register to each NVLink2
device.
However, ATSD registers can be shared among devices. This change
allows vfio-pci to fall back to sharing the register at index 0 if
necessary.
Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com>
---
drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
index f2983f0f84be..851ba673882b 100644
--- a/drivers/vfio/pci/vfio_pci_nvlink2.c
+++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
@@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
&mmio_atsd)) {
- dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
- mmio_atsd = 0;
+ dev_warn(&vdev->pdev->dev,
+ "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
+ nvlink_index);
+ if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
+ &mmio_atsd)) {
+ dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
+ mmio_atsd = 0;
+ } else {
+ dev_warn(&vdev->pdev->dev,
+ "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
+ }
}
if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
--
2.22.0.216.g00a2a96fc9
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]
2020-02-06 3:17 [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] Sam Bobroff
@ 2020-02-06 4:23 ` Alexey Kardashevskiy
2020-02-07 2:39 ` Sam Bobroff
0 siblings, 1 reply; 4+ messages in thread
From: Alexey Kardashevskiy @ 2020-02-06 4:23 UTC (permalink / raw)
To: Sam Bobroff, kvm, linuxppc-dev, Alex Williamson
On 06/02/2020 14:17, Sam Bobroff wrote:
> Older versions of skiboot only provide a single value in the device
> tree property "ibm,mmio-atsd", even when multiple Address Translation
> Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
> (other than the first) from being used with vfio-pci because vfio-pci
> expects to be able to assign a dedicated ATSD register to each NVLink2
> device.
>
> However, ATSD registers can be shared among devices. This change
> allows vfio-pci to fall back to sharing the register at index 0 if
> necessary.
>
> Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com>
> ---
> drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
> 1 file changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
> index f2983f0f84be..851ba673882b 100644
> --- a/drivers/vfio/pci/vfio_pci_nvlink2.c
> +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
> @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
>
> if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
> &mmio_atsd)) {
> - dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> - mmio_atsd = 0;
> + dev_warn(&vdev->pdev->dev,
> + "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
> + nvlink_index);
We do not really need this warning (nvlink_index doesn't matter that
much, we can work out from the device tree what happened), warnings
below are enough (if you really want, you can print nvlink_index there).
Either way,
Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
> + &mmio_atsd)) {
> + dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> + mmio_atsd = 0;
> + } else {
> + dev_warn(&vdev->pdev->dev,
> + "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
> + }
> }
>
> if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
>
--
Alexey
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]
2020-02-06 4:23 ` Alexey Kardashevskiy
@ 2020-02-07 2:39 ` Sam Bobroff
2020-02-07 4:43 ` Sam Bobroff
0 siblings, 1 reply; 4+ messages in thread
From: Sam Bobroff @ 2020-02-07 2:39 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: Alex Williamson, linuxppc-dev, kvm
[-- Attachment #1: Type: text/plain, Size: 2363 bytes --]
On Thu, Feb 06, 2020 at 03:23:03PM +1100, Alexey Kardashevskiy wrote:
>
>
> On 06/02/2020 14:17, Sam Bobroff wrote:
> > Older versions of skiboot only provide a single value in the device
> > tree property "ibm,mmio-atsd", even when multiple Address Translation
> > Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
> > (other than the first) from being used with vfio-pci because vfio-pci
> > expects to be able to assign a dedicated ATSD register to each NVLink2
> > device.
> >
> > However, ATSD registers can be shared among devices. This change
> > allows vfio-pci to fall back to sharing the register at index 0 if
> > necessary.
> >
> > Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com>
> > ---
> > drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
> > 1 file changed, 11 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
> > index f2983f0f84be..851ba673882b 100644
> > --- a/drivers/vfio/pci/vfio_pci_nvlink2.c
> > +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
> > @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
> >
> > if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
> > &mmio_atsd)) {
> > - dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> > - mmio_atsd = 0;
> > + dev_warn(&vdev->pdev->dev,
> > + "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
> > + nvlink_index);
>
>
> We do not really need this warning (nvlink_index doesn't matter that
> much, we can work out from the device tree what happened), warnings
> below are enough (if you really want, you can print nvlink_index there).
>
> Either way,
>
> Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>
Thanks,
I'll change it if there's some reason to do another version but
otherwise leave it as is.
Sam.
>
>
>
> > + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
> > + &mmio_atsd)) {
> > + dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> > + mmio_atsd = 0;
> > + } else {
> > + dev_warn(&vdev->pdev->dev,
> > + "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
> > + }
> > }
> >
> > if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
> >
>
> --
> Alexey
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0]
2020-02-07 2:39 ` Sam Bobroff
@ 2020-02-07 4:43 ` Sam Bobroff
0 siblings, 0 replies; 4+ messages in thread
From: Sam Bobroff @ 2020-02-07 4:43 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: Alex Williamson, linuxppc-dev, kvm
[-- Attachment #1: Type: text/plain, Size: 2710 bytes --]
On Fri, Feb 07, 2020 at 01:39:14PM +1100, Sam Bobroff wrote:
> On Thu, Feb 06, 2020 at 03:23:03PM +1100, Alexey Kardashevskiy wrote:
> >
> >
> > On 06/02/2020 14:17, Sam Bobroff wrote:
> > > Older versions of skiboot only provide a single value in the device
> > > tree property "ibm,mmio-atsd", even when multiple Address Translation
> > > Shoot Down (ATSD) registers are present. This prevents NVLink2 devices
> > > (other than the first) from being used with vfio-pci because vfio-pci
> > > expects to be able to assign a dedicated ATSD register to each NVLink2
> > > device.
> > >
> > > However, ATSD registers can be shared among devices. This change
> > > allows vfio-pci to fall back to sharing the register at index 0 if
> > > necessary.
> > >
> > > Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com>
> > > ---
> > > drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++--
> > > 1 file changed, 11 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c
> > > index f2983f0f84be..851ba673882b 100644
> > > --- a/drivers/vfio/pci/vfio_pci_nvlink2.c
> > > +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c
> > > @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
> > >
> > > if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index,
> > > &mmio_atsd)) {
> > > - dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> > > - mmio_atsd = 0;
> > > + dev_warn(&vdev->pdev->dev,
> > > + "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n",
> > > + nvlink_index);
> >
> >
> > We do not really need this warning (nvlink_index doesn't matter that
> > much, we can work out from the device tree what happened), warnings
> > below are enough (if you really want, you can print nvlink_index there).
> >
> > Either way,
> >
> > Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >
>
> Thanks,
>
> I'll change it if there's some reason to do another version but
> otherwise leave it as is.
>
> Sam.
Oh! I almost forgot: maybe this should carry a fixes tag?
Fixes: 7f92891778df ("vfio_pci: Add NVIDIA GV100GL [Tesla V100 SXM2] subdriver")
> >
> >
> >
> > > + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0,
> > > + &mmio_atsd)) {
> > > + dev_warn(&vdev->pdev->dev, "No available ATSD found\n");
> > > + mmio_atsd = 0;
> > > + } else {
> > > + dev_warn(&vdev->pdev->dev,
> > > + "Using fallback ibm,mmio-atsd[0] for ATSD.\n");
> > > + }
> > > }
> > >
> > > if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) {
> > >
> >
> > --
> > Alexey
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-02-07 4:44 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-06 3:17 [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] Sam Bobroff
2020-02-06 4:23 ` Alexey Kardashevskiy
2020-02-07 2:39 ` Sam Bobroff
2020-02-07 4:43 ` Sam Bobroff
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).