[PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Logan Gunthorpe]

The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
transactions between root ports and found to work. Therefore add it
to the list.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Huang Rui <ray.huang@amd.com>
Cc: Alex Deucher <alexdeucher@gmail.com>
---
 drivers/pci/p2pdma.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
index e8e444eeb1cd..3d67a1ee083e 100644
--- a/drivers/pci/p2pdma.c
+++ b/drivers/pci/p2pdma.c
@@ -284,6 +284,8 @@ static const struct pci_p2pdma_whitelist_entry {
 	{PCI_VENDOR_ID_AMD,	0x1450,	0},
 	{PCI_VENDOR_ID_AMD,	0x15d0,	0},
 	{PCI_VENDOR_ID_AMD,	0x1630,	0},
+	/* AMD ZEN 2 */
+	{PCI_VENDOR_ID_AMD,	0x1480,	0},
 
 	/* Intel Xeon E5/Core i7 */
 	{PCI_VENDOR_ID_INTEL,	0x3c00, REQ_SAME_HOST_BRIDGE},

base-commit: ba47d845d715a010f7b51f6f89bae32845e6acb7
-- 
2.20.1

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Alex Deucher]

On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
>
> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
> transactions between root ports and found to work. Therefore add it
> to the list.
>
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Christian König <christian.koenig@amd.com>
> Cc: Huang Rui <ray.huang@amd.com>
> Cc: Alex Deucher <alexdeucher@gmail.com>

Starting with Zen, all AMD platforms support P2P for reads and writes.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

> ---
>  drivers/pci/p2pdma.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
> index e8e444eeb1cd..3d67a1ee083e 100644
> --- a/drivers/pci/p2pdma.c
> +++ b/drivers/pci/p2pdma.c
> @@ -284,6 +284,8 @@ static const struct pci_p2pdma_whitelist_entry {
>         {PCI_VENDOR_ID_AMD,     0x1450, 0},
>         {PCI_VENDOR_ID_AMD,     0x15d0, 0},
>         {PCI_VENDOR_ID_AMD,     0x1630, 0},
> +       /* AMD ZEN 2 */
> +       {PCI_VENDOR_ID_AMD,     0x1480, 0},
>
>         /* Intel Xeon E5/Core i7 */
>         {PCI_VENDOR_ID_INTEL,   0x3c00, REQ_SAME_HOST_BRIDGE},
>
> base-commit: ba47d845d715a010f7b51f6f89bae32845e6acb7
> --
> 2.20.1
>

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Bjorn Helgaas]

[+cc Andrew, Armen, hpa]

On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:
> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
> >
> > The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
> > transactions between root ports and found to work. Therefore add it
> > to the list.
> >
> > Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> > Cc: Bjorn Helgaas <bhelgaas@google.com>
> > Cc: Christian König <christian.koenig@amd.com>
> > Cc: Huang Rui <ray.huang@amd.com>
> > Cc: Alex Deucher <alexdeucher@gmail.com>
> 
> Starting with Zen, all AMD platforms support P2P for reads and writes.

What's the plan for getting out of the cycle of "update this list for
every new chip"?  Any new _DSMs planned, for instance?

A continuous trickle of updates like this is not really appealing.  So
far we have:

  7d5b10fcb81e ("PCI/P2PDMA: Add AMD Zen Raven and Renoir Root Ports to whitelist")
  7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
  bc123a515cb7 ("PCI/P2PDMA: Add Intel SkyLake-E to the whitelist")
  494d63b0d5d0 ("PCI/P2PDMA: Whitelist some Intel host bridges")
  0f97da831026 ("PCI/P2PDMA: Allow P2P DMA between any devices under AMD ZEN Root Complex")

And that's just from the last year, not including this patch.

> > ---
> >  drivers/pci/p2pdma.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
> > index e8e444eeb1cd..3d67a1ee083e 100644
> > --- a/drivers/pci/p2pdma.c
> > +++ b/drivers/pci/p2pdma.c
> > @@ -284,6 +284,8 @@ static const struct pci_p2pdma_whitelist_entry {
> >         {PCI_VENDOR_ID_AMD,     0x1450, 0},
> >         {PCI_VENDOR_ID_AMD,     0x15d0, 0},
> >         {PCI_VENDOR_ID_AMD,     0x1630, 0},
> > +       /* AMD ZEN 2 */
> > +       {PCI_VENDOR_ID_AMD,     0x1480, 0},
> >
> >         /* Intel Xeon E5/Core i7 */
> >         {PCI_VENDOR_ID_INTEL,   0x3c00, REQ_SAME_HOST_BRIDGE},
> >
> > base-commit: ba47d845d715a010f7b51f6f89bae32845e6acb7
> > --
> > 2.20.1
> >

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Logan Gunthorpe]

On 2020-07-23 1:57 p.m., Bjorn Helgaas wrote:
> [+cc Andrew, Armen, hpa]
> 
> On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:
>> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
>>>
>>> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
>>> transactions between root ports and found to work. Therefore add it
>>> to the list.
>>>
>>> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
>>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>>> Cc: Christian König <christian.koenig@amd.com>
>>> Cc: Huang Rui <ray.huang@amd.com>
>>> Cc: Alex Deucher <alexdeucher@gmail.com>
>>
>> Starting with Zen, all AMD platforms support P2P for reads and writes.
> 
> What's the plan for getting out of the cycle of "update this list for
> every new chip"?  Any new _DSMs planned, for instance?

Well there was an effort to add capabilities in the PCI spec to describe
this but, as far as I know, they never got anywhere, and hardware still
doesn't self describe with this.

> A continuous trickle of updates like this is not really appealing.  So
> far we have:
> 
>   7d5b10fcb81e ("PCI/P2PDMA: Add AMD Zen Raven and Renoir Root Ports to whitelist")
>   7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
>   bc123a515cb7 ("PCI/P2PDMA: Add Intel SkyLake-E to the whitelist")
>   494d63b0d5d0 ("PCI/P2PDMA: Whitelist some Intel host bridges")
>   0f97da831026 ("PCI/P2PDMA: Allow P2P DMA between any devices under AMD ZEN Root Complex")
> 
> And that's just from the last year, not including this patch.

Yes, it's not ideal. But most of these are adding old devices as people
test and care about running on those platforms -- a lot of this is
bootstrapping the list. I'd expect this to slow down a bit as by now we
have hopefully got a lot of the existing platforms people care about.
But we'd still probably expect to be adding a new Intel and AMD devices
about once a year as they produce new hardware designs.

Unless, the Intel and AMD folks know of a way to detect this, or even to
query if a root complex is newer than a certain generation, I'm not sure
what else we can do here.

Logan

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Alex Deucher]

On Thu, Jul 23, 2020 at 4:11 PM Logan Gunthorpe <logang@deltatee.com> wrote:
>
>
>
> On 2020-07-23 1:57 p.m., Bjorn Helgaas wrote:
> > [+cc Andrew, Armen, hpa]
> >
> > On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:
> >> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
> >>>
> >>> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
> >>> transactions between root ports and found to work. Therefore add it
> >>> to the list.
> >>>
> >>> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> >>> Cc: Bjorn Helgaas <bhelgaas@google.com>
> >>> Cc: Christian König <christian.koenig@amd.com>
> >>> Cc: Huang Rui <ray.huang@amd.com>
> >>> Cc: Alex Deucher <alexdeucher@gmail.com>
> >>
> >> Starting with Zen, all AMD platforms support P2P for reads and writes.
> >
> > What's the plan for getting out of the cycle of "update this list for
> > every new chip"?  Any new _DSMs planned, for instance?
>
> Well there was an effort to add capabilities in the PCI spec to describe
> this but, as far as I know, they never got anywhere, and hardware still
> doesn't self describe with this.
>
> > A continuous trickle of updates like this is not really appealing.  So
> > far we have:
> >
> >   7d5b10fcb81e ("PCI/P2PDMA: Add AMD Zen Raven and Renoir Root Ports to whitelist")
> >   7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
> >   bc123a515cb7 ("PCI/P2PDMA: Add Intel SkyLake-E to the whitelist")
> >   494d63b0d5d0 ("PCI/P2PDMA: Whitelist some Intel host bridges")
> >   0f97da831026 ("PCI/P2PDMA: Allow P2P DMA between any devices under AMD ZEN Root Complex")
> >
> > And that's just from the last year, not including this patch.
>
> Yes, it's not ideal. But most of these are adding old devices as people
> test and care about running on those platforms -- a lot of this is
> bootstrapping the list. I'd expect this to slow down a bit as by now we
> have hopefully got a lot of the existing platforms people care about.
> But we'd still probably expect to be adding a new Intel and AMD devices
> about once a year as they produce new hardware designs.
>
> Unless, the Intel and AMD folks know of a way to detect this, or even to
> query if a root complex is newer than a certain generation, I'm not sure
> what else we can do here.

I started a thread internally to see if I can find a way.  FWIW,
pre-ZEN parts also support p2p DMA, but only for writes.  If I can get
a definitive list, maybe we could switch to a blacklist for the old
ones?

Alex

>
> Logan

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Logan Gunthorpe]

On 2020-07-23 2:18 p.m., Alex Deucher wrote:
> On Thu, Jul 23, 2020 at 4:11 PM Logan Gunthorpe <logang@deltatee.com> wrote:
>>
>>
>>
>> On 2020-07-23 1:57 p.m., Bjorn Helgaas wrote:
>>> [+cc Andrew, Armen, hpa]
>>>
>>> On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:
>>>> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
>>>>>
>>>>> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
>>>>> transactions between root ports and found to work. Therefore add it
>>>>> to the list.
>>>>>
>>>>> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
>>>>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>>>>> Cc: Christian König <christian.koenig@amd.com>
>>>>> Cc: Huang Rui <ray.huang@amd.com>
>>>>> Cc: Alex Deucher <alexdeucher@gmail.com>
>>>>
>>>> Starting with Zen, all AMD platforms support P2P for reads and writes.
>>>
>>> What's the plan for getting out of the cycle of "update this list for
>>> every new chip"?  Any new _DSMs planned, for instance?
>>
>> Well there was an effort to add capabilities in the PCI spec to describe
>> this but, as far as I know, they never got anywhere, and hardware still
>> doesn't self describe with this.
>>
>>> A continuous trickle of updates like this is not really appealing.  So
>>> far we have:
>>>
>>>   7d5b10fcb81e ("PCI/P2PDMA: Add AMD Zen Raven and Renoir Root Ports to whitelist")
>>>   7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
>>>   bc123a515cb7 ("PCI/P2PDMA: Add Intel SkyLake-E to the whitelist")
>>>   494d63b0d5d0 ("PCI/P2PDMA: Whitelist some Intel host bridges")
>>>   0f97da831026 ("PCI/P2PDMA: Allow P2P DMA between any devices under AMD ZEN Root Complex")
>>>
>>> And that's just from the last year, not including this patch.
>>
>> Yes, it's not ideal. But most of these are adding old devices as people
>> test and care about running on those platforms -- a lot of this is
>> bootstrapping the list. I'd expect this to slow down a bit as by now we
>> have hopefully got a lot of the existing platforms people care about.
>> But we'd still probably expect to be adding a new Intel and AMD devices
>> about once a year as they produce new hardware designs.
>>
>> Unless, the Intel and AMD folks know of a way to detect this, or even to
>> query if a root complex is newer than a certain generation, I'm not sure
>> what else we can do here.
> 
> I started a thread internally to see if I can find a way.  FWIW,
> pre-ZEN parts also support p2p DMA, but only for writes.  If I can get
> a definitive list, maybe we could switch to a blacklist for the old
> ones?

It would have to be an AMD specific list, falling back to the general
whitelist.... I suppose we can also mine the pci_ids database for Intel
root complexes and create a blacklist there too. But there are a lot
more root complexes outside of the x86 world...

Logan

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Bjorn Helgaas]

On Thu, Jul 23, 2020 at 02:10:52PM -0600, Logan Gunthorpe wrote:
> On 2020-07-23 1:57 p.m., Bjorn Helgaas wrote:
> > On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:
> >> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
> >>>
> >>> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
> >>> transactions between root ports and found to work. Therefore add it
> >>> to the list.
> >>>
> >>> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> >>> Cc: Bjorn Helgaas <bhelgaas@google.com>
> >>> Cc: Christian König <christian.koenig@amd.com>
> >>> Cc: Huang Rui <ray.huang@amd.com>
> >>> Cc: Alex Deucher <alexdeucher@gmail.com>
> >>
> >> Starting with Zen, all AMD platforms support P2P for reads and writes.
> > 
> > What's the plan for getting out of the cycle of "update this list for
> > every new chip"?  Any new _DSMs planned, for instance?
> 
> Well there was an effort to add capabilities in the PCI spec to describe
> this but, as far as I know, they never got anywhere, and hardware still
> doesn't self describe with this.

Any idea what happened?  Is there hope for the future?  I'm really not
happy about signing up for open-ended device-specific patches like
this.  It's certainly not in the plug and play spirit that has made
PCI successful.  I know, preaching to the choir here.

Bjorn

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Logan Gunthorpe]

[+cc Jonathan]

On 2020-07-24 9:06 a.m., Bjorn Helgaas wrote:
> On Thu, Jul 23, 2020 at 02:10:52PM -0600, Logan Gunthorpe wrote:
>> On 2020-07-23 1:57 p.m., Bjorn Helgaas wrote:
>>> On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:
>>>> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
>>>>>
>>>>> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
>>>>> transactions between root ports and found to work. Therefore add it
>>>>> to the list.
>>>>>
>>>>> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
>>>>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>>>>> Cc: Christian König <christian.koenig@amd.com>
>>>>> Cc: Huang Rui <ray.huang@amd.com>
>>>>> Cc: Alex Deucher <alexdeucher@gmail.com>
>>>>
>>>> Starting with Zen, all AMD platforms support P2P for reads and writes.
>>>
>>> What's the plan for getting out of the cycle of "update this list for
>>> every new chip"?  Any new _DSMs planned, for instance?
>>
>> Well there was an effort to add capabilities in the PCI spec to describe
>> this but, as far as I know, they never got anywhere, and hardware still
>> doesn't self describe with this.
> 
> Any idea what happened?  Is there hope for the future?  I'm really not
> happy about signing up for open-ended device-specific patches like
> this.  It's certainly not in the plug and play spirit that has made
> PCI successful.  I know, preaching to the choir here.

Agreed, though I'm not really hooked into the PCI SIG. The last email I
got about this was an RFC from Jonathan Cameron in late 2018. I've CC'd
him here, maybe he'll have a bit more insight.

Logan

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Alex Deucher]

On Thu, Jul 23, 2020 at 4:18 PM Alex Deucher <alexdeucher@gmail.com> wrote:
>
> On Thu, Jul 23, 2020 at 4:11 PM Logan Gunthorpe <logang@deltatee.com> wrote:
> >
> >
> >
> > On 2020-07-23 1:57 p.m., Bjorn Helgaas wrote:
> > > [+cc Andrew, Armen, hpa]
> > >
> > > On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:
> > >> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
> > >>>
> > >>> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
> > >>> transactions between root ports and found to work. Therefore add it
> > >>> to the list.
> > >>>
> > >>> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> > >>> Cc: Bjorn Helgaas <bhelgaas@google.com>
> > >>> Cc: Christian König <christian.koenig@amd.com>
> > >>> Cc: Huang Rui <ray.huang@amd.com>
> > >>> Cc: Alex Deucher <alexdeucher@gmail.com>
> > >>
> > >> Starting with Zen, all AMD platforms support P2P for reads and writes.
> > >
> > > What's the plan for getting out of the cycle of "update this list for
> > > every new chip"?  Any new _DSMs planned, for instance?
> >
> > Well there was an effort to add capabilities in the PCI spec to describe
> > this but, as far as I know, they never got anywhere, and hardware still
> > doesn't self describe with this.
> >
> > > A continuous trickle of updates like this is not really appealing.  So
> > > far we have:
> > >
> > >   7d5b10fcb81e ("PCI/P2PDMA: Add AMD Zen Raven and Renoir Root Ports to whitelist")
> > >   7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
> > >   bc123a515cb7 ("PCI/P2PDMA: Add Intel SkyLake-E to the whitelist")
> > >   494d63b0d5d0 ("PCI/P2PDMA: Whitelist some Intel host bridges")
> > >   0f97da831026 ("PCI/P2PDMA: Allow P2P DMA between any devices under AMD ZEN Root Complex")
> > >
> > > And that's just from the last year, not including this patch.
> >
> > Yes, it's not ideal. But most of these are adding old devices as people
> > test and care about running on those platforms -- a lot of this is
> > bootstrapping the list. I'd expect this to slow down a bit as by now we
> > have hopefully got a lot of the existing platforms people care about.
> > But we'd still probably expect to be adding a new Intel and AMD devices
> > about once a year as they produce new hardware designs.
> >
> > Unless, the Intel and AMD folks know of a way to detect this, or even to
> > query if a root complex is newer than a certain generation, I'm not sure
> > what else we can do here.
>
> I started a thread internally to see if I can find a way.  FWIW,
> pre-ZEN parts also support p2p DMA, but only for writes.  If I can get
> a definitive list, maybe we could switch to a blacklist for the old
> ones?

After talking with a few people internally, for AMD chips, it would
probably be easiest to just whitelist based on the CPU family id for
zen and newer (e.g., >= 0x17).

Alex

>
> Alex
>
> >
> > Logan

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Logan Gunthorpe]

On 2020-07-24 10:07 a.m., Alex Deucher wrote:
> On Thu, Jul 23, 2020 at 4:18 PM Alex Deucher <alexdeucher@gmail.com> wrote:
>>
>> On Thu, Jul 23, 2020 at 4:11 PM Logan Gunthorpe <logang@deltatee.com> wrote:
>>>
>>>
>>>
>>> On 2020-07-23 1:57 p.m., Bjorn Helgaas wrote:
>>>> [+cc Andrew, Armen, hpa]
>>>>
>>>> On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:
>>>>> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:
>>>>>>
>>>>>> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
>>>>>> transactions between root ports and found to work. Therefore add it
>>>>>> to the list.
>>>>>>
>>>>>> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
>>>>>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>>>>>> Cc: Christian König <christian.koenig@amd.com>
>>>>>> Cc: Huang Rui <ray.huang@amd.com>
>>>>>> Cc: Alex Deucher <alexdeucher@gmail.com>
>>>>>
>>>>> Starting with Zen, all AMD platforms support P2P for reads and writes.
>>>>
>>>> What's the plan for getting out of the cycle of "update this list for
>>>> every new chip"?  Any new _DSMs planned, for instance?
>>>
>>> Well there was an effort to add capabilities in the PCI spec to describe
>>> this but, as far as I know, they never got anywhere, and hardware still
>>> doesn't self describe with this.
>>>
>>>> A continuous trickle of updates like this is not really appealing.  So
>>>> far we have:
>>>>
>>>>   7d5b10fcb81e ("PCI/P2PDMA: Add AMD Zen Raven and Renoir Root Ports to whitelist")
>>>>   7b94b53db34f ("PCI/P2PDMA: Add Intel Sky Lake-E Root Ports B, C, D to the whitelist")
>>>>   bc123a515cb7 ("PCI/P2PDMA: Add Intel SkyLake-E to the whitelist")
>>>>   494d63b0d5d0 ("PCI/P2PDMA: Whitelist some Intel host bridges")
>>>>   0f97da831026 ("PCI/P2PDMA: Allow P2P DMA between any devices under AMD ZEN Root Complex")
>>>>
>>>> And that's just from the last year, not including this patch.
>>>
>>> Yes, it's not ideal. But most of these are adding old devices as people
>>> test and care about running on those platforms -- a lot of this is
>>> bootstrapping the list. I'd expect this to slow down a bit as by now we
>>> have hopefully got a lot of the existing platforms people care about.
>>> But we'd still probably expect to be adding a new Intel and AMD devices
>>> about once a year as they produce new hardware designs.
>>>
>>> Unless, the Intel and AMD folks know of a way to detect this, or even to
>>> query if a root complex is newer than a certain generation, I'm not sure
>>> what else we can do here.
>>
>> I started a thread internally to see if I can find a way.  FWIW,
>> pre-ZEN parts also support p2p DMA, but only for writes.  If I can get
>> a definitive list, maybe we could switch to a blacklist for the old
>> ones?
> 
> After talking with a few people internally, for AMD chips, it would
> probably be easiest to just whitelist based on the CPU family id for
> zen and newer (e.g., >= 0x17).

That seems sensible. I was trying to see if we could do something
similar for Intel, and just allow anything after Skylake. I found
this[1]. It seems they have been on family 6 for a long time, and I'm
not comfortable enabling the whole family. Their model numbers also
don't seem to increment in a favorable fashion, in that "small core"
atoms (which have different host bridges with very much unknown support)
have model numbers interspersed with regular "big core" CPUS.

So we might be stuck with the Intel white list for a while, but using
the AMD family number will at least cut the number of additions down a
fair amount. I can try to put a patch together in place of this one.

Logan

[1]
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/intel-family.h#L73

Re: [PATCH] PCI/P2PDMA: Add AMD Zen 2 root complex to the list of allowed bridges

[Jonathan Cameron]

On Fri, 24 Jul 2020 09:56:39 -0600
Logan Gunthorpe <logang@deltatee.com> wrote:

> [+cc Jonathan]
> 
> On 2020-07-24 9:06 a.m., Bjorn Helgaas wrote:
> > On Thu, Jul 23, 2020 at 02:10:52PM -0600, Logan Gunthorpe wrote:  
> >> On 2020-07-23 1:57 p.m., Bjorn Helgaas wrote:  
> >>> On Thu, Jul 23, 2020 at 02:01:17PM -0400, Alex Deucher wrote:  
> >>>> On Thu, Jul 23, 2020 at 1:43 PM Logan Gunthorpe <logang@deltatee.com> wrote:  
> >>>>>
> >>>>> The AMD Zen 2 root complex (Starship/Matisse) was tested for P2PDMA
> >>>>> transactions between root ports and found to work. Therefore add it
> >>>>> to the list.
> >>>>>
> >>>>> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> >>>>> Cc: Bjorn Helgaas <bhelgaas@google.com>
> >>>>> Cc: Christian König <christian.koenig@amd.com>
> >>>>> Cc: Huang Rui <ray.huang@amd.com>
> >>>>> Cc: Alex Deucher <alexdeucher@gmail.com>  
> >>>>
> >>>> Starting with Zen, all AMD platforms support P2P for reads and writes.  
> >>>
> >>> What's the plan for getting out of the cycle of "update this list for
> >>> every new chip"?  Any new _DSMs planned, for instance?  
> >>
> >> Well there was an effort to add capabilities in the PCI spec to describe
> >> this but, as far as I know, they never got anywhere, and hardware still
> >> doesn't self describe with this.  
> > 
> > Any idea what happened?  Is there hope for the future?  I'm really not
> > happy about signing up for open-ended device-specific patches like
> > this.  It's certainly not in the plug and play spirit that has made
> > PCI successful.  I know, preaching to the choir here.  
> 
> Agreed, though I'm not really hooked into the PCI SIG. The last email I
> got about this was an RFC from Jonathan Cameron in late 2018. I've CC'd
> him here, maybe he'll have a bit more insight.

For non technical reasons, you can probably figure out, that particular
ECR stalled. Unfortunately I can't directly provide info on any newer
discussions. Eric, could you perhaps find out if there is anything we can share?

This is the same question of trying to find a way to avoid white listing
root complexes that can do peer 2 peer that would have been covered by
your Advanced Peer to Peer Capabilities ECR.

Thanks,

Jonathan





> 
> Logan