PCI / PCIe Device Memory - Rationale for Choosing MMIO Over PMIO (and Visa-Versa)

PCI / PCIe Device Memory - Rationale for Choosing MMIO Over PMIO (and Visa-Versa)

[Jack Winch]

Hi all,

Over the last couple of months, I've been reading the hardware
documentation and Linux device driver source code for a range of
different PCI and PCIe devices.  Those examined range from
multi-function data acquisition cards through to avionics bus
interface devices.  In doing so, I have referenced numerous resources
(including the Third Edition of LDD - what a great book - and the
documentation available for the Linux PCI Bus Subsystem on
kernel.org).

One thing I'm still a little unclear on is why vendors might opt to
map PCI / PCIe device memory into the system memory map as either
Memory-Mapped I/O (MMIO) or Port-Mapped I/O (PMIO).  That is, for what
reasons would a device manufacturer choose to make use of one address
space over the other for regions of a PCI / PCIe device's memory?
Some of the general reasons are alluded to by the aforementioned
resources (e.g., more instruction cycles are required to access data
via PMIO, MMIO can be marked as prefetchable and handled as such,
etc).

Would anyone who's been engaged in the development of a PCI / PCIe
device talk about their experience and what factors led to one address
space being chosen over the other (for specific regions of a specific
device)?

Specific examples would really help me (and probably others)
understand what factors are involved in this decision and how a
suitable choice is made.  Reading the driver source code, for specific
devices, has been great for developing an initial understanding of the
different approaches taken by device manufacturers, but source code
and hardware documentation rarely provide any information on the
rationale for a chosen implementation.

Any specific technical accounts on this matter, etc, would be much appreciated.

Jack

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: PCI / PCIe Device Memory - Rationale for Choosing MMIO Over PMIO (and Visa-Versa)

[Greg KH]

On Thu, Nov 05, 2020 at 02:27:58PM +0000, Jack Winch wrote:
> Hi all,
> 
> Over the last couple of months, I've been reading the hardware
> documentation and Linux device driver source code for a range of
> different PCI and PCIe devices.  Those examined range from
> multi-function data acquisition cards through to avionics bus
> interface devices.  In doing so, I have referenced numerous resources
> (including the Third Edition of LDD - what a great book - and the
> documentation available for the Linux PCI Bus Subsystem on
> kernel.org).
> 
> One thing I'm still a little unclear on is why vendors might opt to
> map PCI / PCIe device memory into the system memory map as either
> Memory-Mapped I/O (MMIO) or Port-Mapped I/O (PMIO).  That is, for what
> reasons would a device manufacturer choose to make use of one address
> space over the other for regions of a PCI / PCIe device's memory?
> Some of the general reasons are alluded to by the aforementioned
> resources (e.g., more instruction cycles are required to access data
> via PMIO, MMIO can be marked as prefetchable and handled as such,
> etc).

I think you should be talking to hardware people about this, as this is
almost always due to hardware limitations/issues/design decisions.  The
PCI-SIG should have a bunch of resources about this, have you looked
into that?

good luck!

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: PCI / PCIe Device Memory - Rationale for Choosing MMIO Over PMIO (and Visa-Versa)

[Jack Winch]

Hey Greg,

> I think you should be talking to hardware people about this, as this is
> almost always due to hardware limitations/issues/design decisions.


To be honest, I anticipated a response along these lines (after
briefly speaking to another individual on this matter).  They
commented on how this decision was usually made and asserted by the
hardware engineer(s).

> The PCI-SIG should have a bunch of resources about this, have you looked
> into that?


I haven't as of yet, but I'll shift focus onto their resources next.
Do any specific resources of theirs spring to mind offhand?  No
worries if not - I'm not afraid to spend time searching them out.

Thanks for the quick response and your time. :)

Jack

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies