Re: [EXT] [RFC PATCH 0/4] PCI: endpoint: Introduce a virtio-net EP function

[Shunsuke Mie <mie@igel.co.jp>]

On 2023/02/08 1:02, Frank Li wrote:
>> We project extending this module to support RDMA. The plan is based on
>> virtio-rdma[1].
>> It extends the virtio-net and we are plan to implement the proposed
>> spec based on this patch.
>> [1] virtio-rdma
>> - proposal:
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
>> ernel.org%2Fall%2F20220511095900.343-1-
>> xieyongji%40bytedance.com%2FT%2F&data=05%7C01%7Cfrank.li%40nxp.co
>> m%7C0ef2bd62eda945c413be08db08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5
>> c301635%7C0%7C0%7C638113625610341574%7CUnknown%7CTWFpbGZsb3d
>> 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
>> 3D%7C3000%7C%7C%7C&sdata=HyhpRTG8MNx%2BtfmWn6x3srmdBjHcZAo
>> 2qbxL9USph9o%3D&reserved=0
>> - presentation on kvm forum:
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fyout
>> u.be%2FQrhv6hC_YK4&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62
>> eda945c413be08db08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
>> 7C0%7C638113625610341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
>> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
>> C%7C%7C&sdata=ucOsGR1letTjxf0gKN6uls5y951CXaIspZtLGnASEC8%3D&res
>> erved=0
>>
> Sorry for our outlook client always change link.  This previous discussion.
> https://lore.kernel.org/imx/d098a631-9930-26d3-48f3-8f95386c8e50@ti.com/T/#t
>
> Look like Endpoint maintainer Kishon like endpoint side work as vhost.
> Previous  Haotian Wang submit similar patches, which just not use eDMA, just use memcpy.
> But overall idea is the same.
>
> I think your and haotian's method is more reasonable for PCI-RC EP connection.
>
> Kishon is not active recently.   Maybe need Lorenzo Pieralisi and Bjorn helgass's comments
> for overall directions.
I think so too. Thank you for your summarization. I've commented on the 
e-mail.
> Frank Li
>
>> Please feel free to comment and suggest.
>>> Frank Li
>>>
>>>> To realize the function, this patchset has few changes and introduces a
>>>> new APIs to PCI EP framework related to virtio. Furthermore, it device
>>>> depends on the some patchtes that is discussing. Those depended
>> patchset
>>>> are following:
>>>> - [PATCH 1/2] dmaengine: dw-edma: Fix to change for continuous
>> transfer
>>>> link:
>>>>
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
>> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
>> 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
>> 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
>> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
>> VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
>>>> ernel.org%2Fdmaengine%2F20221223022608.550697-1-
>>>>
>> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
>>>> 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
>>>>
>> 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
>> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
>> C%7C%7C&sdata=tIn0MHzEvrdxaC4KKTvTRvYXBzQ6MyrFa2GXpa3ePv0%3D&
>>>> reserved=0
>>>> - [RFC PATCH 0/3] Deal with alignment restriction on EP side
>>>> link:
>>>>
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
>> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
>> 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
>> 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
>> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
>> VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
>>>> ernel.org%2Flinux-pci%2F20230113090350.1103494-1-
>>>>
>> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
>>>> 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
>>>>
>> 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
>> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
>> C%7C%7C&sdata=RLpnDiLwfqQd5QMXdiQyPVCkfOj8q2AyVeZOwWHvlsM%3
>>>> D&reserved=0
>>>> - [RFC PATCH v2 0/7] Introduce a vringh accessor for IO memory
>>>> link:
>>>>
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
>> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
>> 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
>> 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
>> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
>> VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
>>>> ernel.org%2Fvirtualization%2F20230202090934.549556-1-
>>>>
>> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
>>>> 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
>>>>
>> 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
>> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
>> C%7C%7C&sdata=6jgY76BMSbvamb%2Fl3Urjt4Gcizeqon%2BZE5nPssc2kDA%
>>>> 3D&reserved=0
>>>>
>>>> About this patchset has 4 patches. The first of two patch is little changes
>>>> to virtio. The third patch add APIs to easily access virtio data structure
>>>> on PCIe Host side memory. The last one introduce a virtio-net EP device
>>>> function. Details are in commit respectively.
>>>>
>>>> Currently those network devices are testd using ping only. I'll add a
>>>> result of performance evaluation using iperf and etc to the future version
>>>> of this patchset.
>>>>
>>>> Shunsuke Mie (4):
>>>>    virtio_pci: add a definition of queue flag in ISR
>>>>    virtio_ring: remove const from vring getter
>>>>    PCI: endpoint: Introduce virtio library for EP functions
>>>>    PCI: endpoint: function: Add EP function driver to provide virtio net
>>>>      device
>>>>
>>>>   drivers/pci/endpoint/Kconfig                  |   7 +
>>>>   drivers/pci/endpoint/Makefile                 |   1 +
>>>>   drivers/pci/endpoint/functions/Kconfig        |  12 +
>>>>   drivers/pci/endpoint/functions/Makefile       |   1 +
>>>>   .../pci/endpoint/functions/pci-epf-vnet-ep.c  | 343 ++++++++++
>>>>   .../pci/endpoint/functions/pci-epf-vnet-rc.c  | 635
>> ++++++++++++++++++
>>>>   drivers/pci/endpoint/functions/pci-epf-vnet.c | 387 +++++++++++
>>>>   drivers/pci/endpoint/functions/pci-epf-vnet.h |  62 ++
>>>>   drivers/pci/endpoint/pci-epf-virtio.c         | 113 ++++
>>>>   drivers/virtio/virtio_ring.c                  |   2 +-
>>>>   include/linux/pci-epf-virtio.h                |  25 +
>>>>   include/linux/virtio.h                        |   2 +-
>>>>   include/uapi/linux/virtio_pci.h               |   2 +
>>>>   13 files changed, 1590 insertions(+), 2 deletions(-)
>>>>   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet-ep.c
>>>>   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet-rc.c
>>>>   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet.c
>>>>   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet.h
>>>>   create mode 100644 drivers/pci/endpoint/pci-epf-virtio.c
>>>>   create mode 100644 include/linux/pci-epf-virtio.h
>>>>
>>>> --
>>>> 2.25.1
>> Best,
>> Shunsuke

Best,

Shunsuke.