From mboxrd@z Thu Jan  1 00:00:00 1970
Content-Type: multipart/mixed; boundary="===============1037557351213232688=="
MIME-Version: 1.0
From: Mittal, Rishabh <rimittal at ebay.com>
Subject: [SPDK] NBD with SPDK
Date: Sun, 11 Aug 2019 01:08:49 +0000
Message-ID: <A31C369D-ED41-4C25-BE0E-C1963FB2F95D@ebay.com>
List-ID: <spdk@lists.01.org>
To: spdk@lists.01.org

--===============1037557351213232688==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

Hi,

We are trying to use NBD and SPDK on client side.  Data path looks like this

File System ----> NBD client ------>SPDK------->NVMEoF


Currently we are seeing a high latency in the order of 50 us by using this =
path. It seems like there is data buffer copy happening for write commands =
from kernel to user space when spdk nbd read data from the nbd socket.

I think that there could be two ways to prevent data copy .


  1.  Memory mapped the kernel buffers to spdk virtual space.  I am not sur=
e if it is possible to mmap a buffer. And what is the impact to call mmap f=
or each IO.
  2.  If NBD kernel give the physical address of a buffer and SPDK use that=
 to DMA it to NVMEoF. I think spdk must also be changing a virtual address =
to physical address before sending it to nvmeof.

Option 2 makes more sense to me. Please let me know if option 2 is feasible=
 in spdk

Thanks
Rishabh Mittal


--===============1037557351213232688==--