Re: Exploring the PCI EP Framework

[Kishon Vijay Abraham I <kishon@ti.com>]

+linux-pci

Hi,

On 12/11/19 5:18 PM, Ismael Luceno Cortes wrote:
> Hello Kishon and Lorenzo,
> 
> I'm in the process of evaluating options to implement virtual devices,
> and I have a doubt that I couldn't solve by a quick look at the code.
> 
> I have some prototype boards with general purpose cores that can
> synthesize devices (sadly a custom driver, but I plan to replace it if I
> get approval).
> 
> The cores are for the most part equal, even in function, so the problem
> is that for whichever takes the role of endpoint I need a lightweight
> mechanism to signal an incoming transfer.
> 
> I didn't see any way to implement doorbell registers; is there such a
> thing?

The endpoint relies on polling a shared memory to detect any events. However if
the HW supports some sort of memory signaled interrupt (similar to what is
usually available in systems with RC) then we could use it for interrupting the
endpoint system. All we have to do is map the MSI address to a EP BAR region so
that host can write to a mapped BAR in order to raise MSI interrupt.

Even if the HW supports, the EP framework doesn't support to use MSI. I'll have
to see how that can be added.
> 
> I realize it may not exist because it would imply not yet existing
> mechanisms. I even don't know if such a thing is feasible with a
> standard IOMMU, but I'm trying to figure out just that.

Does the PCI EP controller also wired to IOMMU?
> 
> I'm tempted to try to glue together the EP framework with VFIO for the
> purpose.

That would help to create a user space endpoint function driver. Not sure if
it'll help for the doorbell.
> 
> Are you aware of any efforts along those lines?

Most of the efforts in EP right now are towards binding the epf to virtio.
https://lore.kernel.org/linux-pci/20190905161516.2845-1-haotian.wang@sifive.com/T/#m4092f14a49852425a00a9a9afa80b4d3b1b836d1

Thanks
Kishon