* PCIe endpoint network driver? @ 2021-05-26 15:44 Tim Harvey 2021-05-26 16:28 ` Bjorn Helgaas 0 siblings, 1 reply; 4+ messages in thread From: Tim Harvey @ 2021-05-26 15:44 UTC (permalink / raw) To: linux-pci Greetings, Is there an existing driver to implement a network interface controller via a PCIe endpoint? I'm envisioning a system with a PCIe master and multiple endpoints that all have a network interface to communicate with each other. It looks like the PCI Endpoint Framework has a function library but I didn't see anything that looked like a network driver. Best regards, Tim ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: PCIe endpoint network driver? 2021-05-26 15:44 PCIe endpoint network driver? Tim Harvey @ 2021-05-26 16:28 ` Bjorn Helgaas 2021-05-26 17:01 ` Logan Gunthorpe 0 siblings, 1 reply; 4+ messages in thread From: Bjorn Helgaas @ 2021-05-26 16:28 UTC (permalink / raw) To: Tim Harvey, Kishon Vijay Abraham I, Jon Mason, Logan Gunthorpe; +Cc: linux-pci [+to Kishon, Jon, Logan, who might have more insight] On Wed, May 26, 2021 at 08:44:59AM -0700, Tim Harvey wrote: > Greetings, > > Is there an existing driver to implement a network interface > controller via a PCIe endpoint? I'm envisioning a system with a PCIe > master and multiple endpoints that all have a network interface to > communicate with each other. > > It looks like the PCI Endpoint Framework has a function library but I > didn't see anything that looked like a network driver. > > Best regards, > > Tim ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: PCIe endpoint network driver? 2021-05-26 16:28 ` Bjorn Helgaas @ 2021-05-26 17:01 ` Logan Gunthorpe 2021-05-28 7:24 ` Vidya Sagar 0 siblings, 1 reply; 4+ messages in thread From: Logan Gunthorpe @ 2021-05-26 17:01 UTC (permalink / raw) To: Bjorn Helgaas, Tim Harvey, Kishon Vijay Abraham I, Jon Mason; +Cc: linux-pci On 2021-05-26 10:28 a.m., Bjorn Helgaas wrote: > [+to Kishon, Jon, Logan, who might have more insight] > > On Wed, May 26, 2021 at 08:44:59AM -0700, Tim Harvey wrote: >> Greetings, >> >> Is there an existing driver to implement a network interface >> controller via a PCIe endpoint? I'm envisioning a system with a PCIe >> master and multiple endpoints that all have a network interface to >> communicate with each other. That sounds awfully similar to NTB. See ntb_netdev and ntb_transport. Though IMO NTB has proven to be a poor solution to the problem. Modern network cards with RDMA are pretty much superior in every way. Logan ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: PCIe endpoint network driver? 2021-05-26 17:01 ` Logan Gunthorpe @ 2021-05-28 7:24 ` Vidya Sagar 0 siblings, 0 replies; 4+ messages in thread From: Vidya Sagar @ 2021-05-28 7:24 UTC (permalink / raw) To: Logan Gunthorpe, Bjorn Helgaas, Tim Harvey, Kishon Vijay Abraham I, Jon Mason Cc: linux-pci I'm not sure if it is fine to give non Linux kernel references here, but Tegra194 has this implemented (not in an optimized way though) for Tegra194(RP) <-> Tegra194(EP) configuration. It does use Tegra194's proprietary syncpoint shim hardware to facilitate interrupt generation from RP to EP. (FWIW, regular MSIs are used from EP to RP). syncpoint shim hardware gets mapped to a portion of the BAR during initialization, and when RP does a write to this BAR region, it generates an interrupt to the local CPU (i.e. EP's local CPU). You can take a look at the code here EPF driver (on EP system): https://nv-tegra.nvidia.com/gitweb/?p=linux-nvidia.git;a=blob;f=drivers/pci/endpoint/functions/pci-epf-tegra-vnet.c;h=f55790f8c569368ad6012aeb9726b9a6c08c5304; hb=6dc57fec39c444e4c4448be61ddd19c55693daf1 EP's device driver (on RP system): https://nv-tegra.nvidia.com/gitweb/?p=linux-nvidia.git;a=blob;f=drivers/net/ethernet/nvidia/pcie/tegra_vnet.c;h=af74baae1452fea25c3c5292a36a4cd1d8f22e50;hb=6dc57fec39c444e4c4448be61ddd19c55693daf1 As I mentioned, this is not an optimized version and we are yet to upstream it (hence it may not be of upstream code quality). We get around 5Gbps throughput with this. - Vidya Sagar On 5/26/2021 10:31 PM, Logan Gunthorpe wrote: > External email: Use caution opening links or attachments > > > On 2021-05-26 10:28 a.m., Bjorn Helgaas wrote: >> [+to Kishon, Jon, Logan, who might have more insight] >> >> On Wed, May 26, 2021 at 08:44:59AM -0700, Tim Harvey wrote: >>> Greetings, >>> >>> Is there an existing driver to implement a network interface >>> controller via a PCIe endpoint? I'm envisioning a system with a PCIe >>> master and multiple endpoints that all have a network interface to >>> communicate with each other. > > That sounds awfully similar to NTB. See ntb_netdev and ntb_transport. > > Though IMO NTB has proven to be a poor solution to the problem. Modern > network cards with RDMA are pretty much superior in every way. > > Logan > ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-05-28 7:24 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-05-26 15:44 PCIe endpoint network driver? Tim Harvey 2021-05-26 16:28 ` Bjorn Helgaas 2021-05-26 17:01 ` Logan Gunthorpe 2021-05-28 7:24 ` Vidya Sagar
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).