From: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
To: Stefan Metzmacher <metze@samba.org>,
Linux RDMA <linux-rdma@vger.kernel.org>,
Jason Gunthorpe <jgg@ziepe.ca>,
Doug Ledford <dledford@redhat.com>
Cc: kaike.wan@intel.com, "Marciniszyn,
Mike" <mike.marciniszyn@cornelisnetworks.com>
Subject: Re: [RFC] bulk zero copy transport
Date: Fri, 20 Aug 2021 08:37:55 -0400 [thread overview]
Message-ID: <0b24d051-e601-a471-847e-7763115c2a33@cornelisnetworks.com> (raw)
In-Reply-To: <c40e7056-e903-623c-8413-67f5d63246e7@samba.org>
On 8/20/21 4:18 AM, Stefan Metzmacher wrote:
> Hi Dennis,
>
> just as a wild idea, would be an option to use the SMB-Direct [1] protocol defined here?
>
> It basically provides a stream/packet like transport based on IB_WR_SEND[_WITH_INV]
> and in addition it allows direct memory transfers with IB_WR_RDMA_READ or IB_WR_RDMA_WRITE.
>
> It's called SMB-Direct as it's currently used as a transport for the SMB3 protocol,
> but it could also be used as transport for other things.
>
> Over the last years I've been working on a PF_SMBDIRECT socket driver [2]
> as a hobby project in order to support it for Samba. It's not yet production ready
> and has known memory leaks, but the basics already work. The api [3] is based on
> sendmsg/recvmsg with using MSG_OOB with msg->msg_control for direct memory transfers.
> I'll actually use it with IORING_OP_SENDMSG and IORING_OP_RECVMSG, which allow msg->msg_control
> starting 5.12 kernels.
>
> metze
>
> [1] https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-SMBD/%5bMS-SMBD%5d.pdf
> [2] https://git.samba.org/?p=metze/linux/smbdirect.git;a=summary
> [3] https://git.samba.org/?p=metze/linux/smbdirect.git;a=blob;f=smbdirect.h;hb=refs/heads/smbdirect-work-in-progress
>
The problem with SMB-Direct is probably going to be the same as with things like
RDS in that there is extra overhead beyond what we are trying to achieve. I can
honestly say it's not something I've considered so I will definitely take some
time to read through the links you provided.
Thanks
-Denny
prev parent reply other threads:[~2021-08-20 12:38 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-19 19:09 [RFC] bulk zero copy transport Dennis Dalessandro
2021-08-19 23:01 ` Jason Gunthorpe
2021-08-20 12:55 ` Dennis Dalessandro
2021-08-20 8:18 ` Stefan Metzmacher
2021-08-20 12:37 ` Dennis Dalessandro [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0b24d051-e601-a471-847e-7763115c2a33@cornelisnetworks.com \
--to=dennis.dalessandro@cornelisnetworks.com \
--cc=dledford@redhat.com \
--cc=jgg@ziepe.ca \
--cc=kaike.wan@intel.com \
--cc=linux-rdma@vger.kernel.org \
--cc=metze@samba.org \
--cc=mike.marciniszyn@cornelisnetworks.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).