Re: [RFC] bulk zero copy transport

[Jason Gunthorpe <jgg@ziepe.ca>]

On Thu, Aug 19, 2021 at 03:09:02PM -0400, Dennis Dalessandro wrote:
> Just wanted to float an idea we are thinking about. It builds on the basic idea
> of what Intel submitted as their RV module [1]. This however does things a bit
> differently and is really all about bulk zero-copy using the kernel. It is a new
> ULP.
> 
> The major differences are that there will be no new cdev needed. We will make
> use of the existing HFI1 cdev where an FD is needed. We also propose to make use
> of IO-Uring (hence needing FD) to get requests into the kernel. The idea will be
> to not share Uverbs objects with the kernel. The kernel will maintain
> ownership of the qp, pd, mr, cq, etc.

I feel a lot of reluctance to see the API surface of the HFI1 cdev
expanded, especially to encompass an entire ULP

As you know I think that cdev is very much the wrong way to design
driver interfaces, and since all the work is now completed to do it
through verbs I'm not keen on any expansion.

But I'm confused how you are calling something a ULP but then talking
about the HFI (or uverbs even) cdev? That isn't a ULP.

A ULP is something like RDS that spawns its own cdevs and interworks
with the common RDMA stack.

I suppose I don't get what you are trying to sketch. Maybe you could
share the uAPI you envision in more detail?

Jason