Understanding P2P DMA related errors

[Ramesh Errabolu <Ramesh.Errabolu@amd.com>]

Logan,

Wanted to thank you for all the time you have given me. I now understand
the PCIe device tree better. The thing that was throwing me off is the
way I was looking at the "lspci -tv" output.

I realized that Root Complex should be understood to mean a set of special
devices hanging of a BUS. The most important member of this set is the device
that acts as "HOST BRIDGE". This becomes apparent when the device tree is sketched
out on a piece of paper. One can then circumscribe this set logically to form
a logical device - "ROOT COMPLEX". I wish I could share my sketch via email.

Including below lspci output of this device set:

    localhost:~ # 
    localhost:~ # lspci -vs 0000:16:00.0
    16:00.0 System peripheral: Intel Corporation Device 09a2 (rev 04)
        Subsystem: Intel Corporation Device 0000
        Flags: fast devsel, NUMA node 0
        Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

    localhost:~ # 
    localhost:~ # lspci -vs 0000:16:00.1
    16:00.1 System peripheral: Intel Corporation Device 09a4 (rev 04)
        Subsystem: Intel Corporation Device 0000
        Flags: fast devsel, NUMA node 0
        Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

    localhost:~ # 
    localhost:~ # lspci -vs 0000:16:00.2
    16:00.2 System peripheral: Intel Corporation Device 09a3 (rev 04)
        Subsystem: Intel Corporation Device 0000
        Flags: fast devsel, NUMA node 0
        Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

    localhost:~ # 
    localhost:~ # lspci -vs 0000:16:00.4
    16:00.4 Host bridge: Intel Corporation Device 0998
        Subsystem: Intel Corporation Device 0000
        Flags: fast devsel, NUMA node 0
        Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

    localhost:~ # 
    localhost:~ # 

In the above log one can see the device 0000:64:00.4 playing the role of
"HOST BRIDGE' while the remaining devices 0000:64:00.0/1/2 play the role
of devices that act as ENDPOINT. I suspect, not sure, these devices play
a role in Inter-Root complex transactions. If so the whitelist should have
all these devices to support P2P traffic.

Interestingly in a patch I could find I see only 0x09A2 being added. Perhaps
the intention was to support only those systems that have 0x09A2 and not these
other devices such as 0x09A3 and 0x09A4 which may be newer.

Regards,
Ramesh