[Patch v7 00/22] CIFS: Implement SMB Direct protocol

[Patch v7 00/22] CIFS: Implement SMB Direct protocol

[Long Li]

From: Long Li <longli@microsoft.com>

Starting with SMB2 dialect 3.0, Microsoft introduced SMB Direct transport
protocol for transferring upper layer (SMB2) payload over RDMA via Infiniband,
RoCE or iWARP. The prococol is published in [MS-SMBD]
(https://msdn.microsoft.com/en-us/library/hh536346.aspx).

Change log:
v2:
Implemented RDMA read/write via memory registration.
Re-arranged patches for review [Christoph Hellwig <hch@infradead.org>].
Restructured the code and fixed bugs on protocol timer and keepalive
[Tom Talpey <ttalpey@microsoft.com>].

v3:
Improved performance by introducing an additional queue for handling
empty packets and reducing lock contention on IRQ path.
Added light weight profiling by reading TSC.
Improved the code for checking SMB versions when mounting with rdma option
[Leon Romanovsky <leon@kernel.org>]. 
Moved to use pages instead of buffers for passing I/O for RDMA
[Christoph Hellwig <hch@infradead.org>].
Removed redundant code and refactored I/O code paths
[Christoph Hellwig <hch@infradead.org>, Tom Talpey <ttalpey@microsoft.com>].

v4:
Fixed connectivity issues with iWAPR devices.
Exported configurable protocol parameters to /proc/fs/cifs
[Steve French <sfrench@samba.org>]
Re-arranged patches for review
[Pavel Shilovsky <piastryyy@gmail.com>].

v5:
Fixed compiling errors on ia64, i386 and when INFINIBAND is not
configured. [kbuild test robot]
Profiling is removed and will be introduced in a seperate patch.

v6:
Report internal code error via WARN_ON(). Change description in
Kconfig [Pavel Shilovsky <piastryyy@gmail.com>].

v7:
Removed the use of #ifdef CONFIG_CIFS_SMB_DIRECT in upper layer
code calling transport I/O. [Matthew Wilcox <mawilcox@microsoft.com>,
Tom Talpey <ttalpey@microsoft.com>]

Long Li (22):
  CIFS: SMBD: Add parameter rdata to smb2_new_read_req
  CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT
  CIFS: SMBD: Add rdma mount option
  CIFS: SMBD: Add SMB Direct protocol initial values and constants
  CIFS: SMBD: Establish SMB Direct connection
  CIFS: SMBD: export protocol initial values
  CIFS: SMBD: Implement function to create a SMB Direct connection
  CIFS: SMBD: Upper layer connects to SMBDirect session
  CIFS: SMBD: Implement function to reconnect to a SMB Direct transport
  CIFS: SMBD: Upper layer reconnects to SMB Direct session
  CIFS: SMBD: Implement function to destroy a SMB Direct connection
  CIFS: SMBD: Upper layer destroys SMB Direct session on shutdown or
    umount
  CIFS: SMBD: Set SMB Direct maximum read or write size for I/O
  CIFS: SMBD: Implement function to receive data via RDMA receive
  CIFS: SMBD: Upper layer receives data via RDMA receive
  CIFS: SMBD: Implement function to send data via RDMA send
  CIFS: SMBD: Upper layer sends data via RDMA send
  CIFS: SMBD: Implement RDMA memory registration
  CIFS: SMBD: Upper layer performs SMB write via RDMA read through
    memory registration
  CIFS: SMBD: Read correct returned data length for RDMA write (SMB
    read) I/O
  CIFS: SMBD: Upper layer performs SMB read via RDMA write through
    memory registration
  CIFS: SMBD: Add SMB Direct debug counters

 fs/cifs/Kconfig      |    8 +
 fs/cifs/Makefile     |    2 +
 fs/cifs/cifs_debug.c |  147 +++
 fs/cifs/cifsfs.c     |    2 +
 fs/cifs/cifsglob.h   |   21 +-
 fs/cifs/cifssmb.c    |   15 +-
 fs/cifs/connect.c    |   53 +-
 fs/cifs/file.c       |   17 +-
 fs/cifs/smb1ops.c    |    4 +-
 fs/cifs/smb2ops.c    |   24 +-
 fs/cifs/smb2pdu.c    |  127 ++-
 fs/cifs/smbdirect.c  | 2617 ++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h  |  330 +++++++
 fs/cifs/transport.c  |    8 +-
 14 files changed, 3344 insertions(+), 31 deletions(-)
 create mode 100644 fs/cifs/smbdirect.c
 create mode 100644 fs/cifs/smbdirect.h

-- 
2.7.4

[Patch v7 01/22] CIFS: SMBD: Add parameter rdata to smb2_new_read_req

[Long Li]

From: Long Li <longli@microsoft.com>

This patch is for preparing upper layer for doing SMB read via RDMA write.

When we assemble the SMB read packet header, we need to know the I/O layout
if this request is to use a RDMA write. rdata has all the information we need
for memory registration. Add rdata to smb2_new_read_req.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smb2pdu.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index bab3da6..32ad590 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -2350,18 +2350,21 @@ SMB2_flush(const unsigned int xid, struct cifs_tcon *tcon, u64 persistent_fid,
  */
 static int
 smb2_new_read_req(void **buf, unsigned int *total_len,
-		  struct cifs_io_parms *io_parms, unsigned int remaining_bytes,
-		  int request_type)
+	struct cifs_io_parms *io_parms, struct cifs_readdata *rdata,
+	unsigned int remaining_bytes, int request_type)
 {
 	int rc = -EACCES;
 	struct smb2_read_plain_req *req = NULL;
 	struct smb2_sync_hdr *shdr;
+	struct TCP_Server_Info *server;
 
 	rc = smb2_plain_req_init(SMB2_READ, io_parms->tcon, (void **) &req,
 				 total_len);
 	if (rc)
 		return rc;
-	if (io_parms->tcon->ses->server == NULL)
+
+	server = io_parms->tcon->ses->server;
+	if (server == NULL)
 		return -ECONNABORTED;
 
 	shdr = &req->sync_hdr;
@@ -2489,7 +2492,8 @@ smb2_async_readv(struct cifs_readdata *rdata)
 
 	server = io_parms.tcon->ses->server;
 
-	rc = smb2_new_read_req((void **) &buf, &total_len, &io_parms, 0, 0);
+	rc = smb2_new_read_req(
+		(void **) &buf, &total_len, &io_parms, rdata, 0, 0);
 	if (rc) {
 		if (rc == -EAGAIN && rdata->credits) {
 			/* credits was reset by reconnect */
@@ -2557,7 +2561,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
 	struct cifs_ses *ses = io_parms->tcon->ses;
 
 	*nbytes = 0;
-	rc = smb2_new_read_req((void **)&req, &total_len, io_parms, 0, 0);
+	rc = smb2_new_read_req((void **)&req, &total_len, io_parms, NULL, 0, 0);
 	if (rc)
 		return rc;
 
-- 
2.7.4

RE: [Patch v7 01/22] CIFS: SMBD: Add parameter rdata to smb2_new_read_req

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>:
> From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
>
> This patch is for preparing upper layer for doing SMB read via RDMA write.
>
> When we assemble the SMB read packet header, we need to know the I/O layout
> if this request is to use a RDMA write. rdata has all the information we need
> for memory registration. Add rdata to smb2_new_read_req.
>
> Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
> ---
>  fs/cifs/smb2pdu.c | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
> index bab3da6..32ad590 100644
> --- a/fs/cifs/smb2pdu.c
> +++ b/fs/cifs/smb2pdu.c
> @@ -2350,18 +2350,21 @@ SMB2_flush(const unsigned int xid, struct cifs_tcon *tcon, u64 persistent_fid,
>   */
>  static int
>  smb2_new_read_req(void **buf, unsigned int *total_len,
> -                 struct cifs_io_parms *io_parms, unsigned int remaining_bytes,
> -                 int request_type)
> +       struct cifs_io_parms *io_parms, struct cifs_readdata *rdata,
> +       unsigned int remaining_bytes, int request_type)
>  {
>         int rc = -EACCES;
>         struct smb2_read_plain_req *req = NULL;
>         struct smb2_sync_hdr *shdr;
> +       struct TCP_Server_Info *server;
>
>         rc = smb2_plain_req_init(SMB2_READ, io_parms->tcon, (void **) &req,
>                                  total_len);
>         if (rc)
>                 return rc;
> -       if (io_parms->tcon->ses->server == NULL)
> +
> +       server = io_parms->tcon->ses->server;
> +       if (server == NULL)
>                 return -ECONNABORTED;
>
>         shdr = &req->sync_hdr;
> @@ -2489,7 +2492,8 @@ smb2_async_readv(struct cifs_readdata *rdata)
>
>         server = io_parms.tcon->ses->server;
>
> -       rc = smb2_new_read_req((void **) &buf, &total_len, &io_parms, 0, 0);
> +       rc = smb2_new_read_req(
> +               (void **) &buf, &total_len, &io_parms, rdata, 0, 0);
>         if (rc) {
>                 if (rc == -EAGAIN && rdata->credits) {
>                         /* credits was reset by reconnect */
> @@ -2557,7 +2561,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
>         struct cifs_ses *ses = io_parms->tcon->ses;
>
>         *nbytes = 0;
> -       rc = smb2_new_read_req((void **)&req, &total_len, io_parms, 0, 0);
> +       rc = smb2_new_read_req((void **)&req, &total_len, io_parms, NULL, 0, 0);
>         if (rc)
>                 return rc;
>
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Looks good.

Reviewed-by: Pavel Shilovsky <pshilov-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

--
Best regards,
Pavel Shilovsky

RE: [Patch v7 01/22] CIFS: SMBD: Add parameter rdata to smb2_new_read_req

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli@exchange.microsoft.com>:
> From: Long Li <longli@microsoft.com>
>
> This patch is for preparing upper layer for doing SMB read via RDMA write.
>
> When we assemble the SMB read packet header, we need to know the I/O layout
> if this request is to use a RDMA write. rdata has all the information we need
> for memory registration. Add rdata to smb2_new_read_req.
>
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>  fs/cifs/smb2pdu.c | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
> index bab3da6..32ad590 100644
> --- a/fs/cifs/smb2pdu.c
> +++ b/fs/cifs/smb2pdu.c
> @@ -2350,18 +2350,21 @@ SMB2_flush(const unsigned int xid, struct cifs_tcon *tcon, u64 persistent_fid,
>   */
>  static int
>  smb2_new_read_req(void **buf, unsigned int *total_len,
> -                 struct cifs_io_parms *io_parms, unsigned int remaining_bytes,
> -                 int request_type)
> +       struct cifs_io_parms *io_parms, struct cifs_readdata *rdata,
> +       unsigned int remaining_bytes, int request_type)
>  {
>         int rc = -EACCES;
>         struct smb2_read_plain_req *req = NULL;
>         struct smb2_sync_hdr *shdr;
> +       struct TCP_Server_Info *server;
>
>         rc = smb2_plain_req_init(SMB2_READ, io_parms->tcon, (void **) &req,
>                                  total_len);
>         if (rc)
>                 return rc;
> -       if (io_parms->tcon->ses->server == NULL)
> +
> +       server = io_parms->tcon->ses->server;
> +       if (server == NULL)
>                 return -ECONNABORTED;
>
>         shdr = &req->sync_hdr;
> @@ -2489,7 +2492,8 @@ smb2_async_readv(struct cifs_readdata *rdata)
>
>         server = io_parms.tcon->ses->server;
>
> -       rc = smb2_new_read_req((void **) &buf, &total_len, &io_parms, 0, 0);
> +       rc = smb2_new_read_req(
> +               (void **) &buf, &total_len, &io_parms, rdata, 0, 0);
>         if (rc) {
>                 if (rc == -EAGAIN && rdata->credits) {
>                         /* credits was reset by reconnect */
> @@ -2557,7 +2561,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
>         struct cifs_ses *ses = io_parms->tcon->ses;
>
>         *nbytes = 0;
> -       rc = smb2_new_read_req((void **)&req, &total_len, io_parms, 0, 0);
> +       rc = smb2_new_read_req((void **)&req, &total_len, io_parms, NULL, 0, 0);
>         if (rc)
>                 return rc;
>
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Looks good.

Reviewed-by: Pavel Shilovsky <pshilov@microsoft.com>

--
Best regards,
Pavel Shilovsky

RE: [Patch v7 01/22] CIFS: SMBD: Add parameter rdata to smb2_new_read_req

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>:
> From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
>
> This patch is for preparing upper layer for doing SMB read via RDMA write.
>
> When we assemble the SMB read packet header, we need to know the I/O layout
> if this request is to use a RDMA write. rdata has all the information we need
> for memory registration. Add rdata to smb2_new_read_req.
>
> Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
> ---
>  fs/cifs/smb2pdu.c | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
> index bab3da6..32ad590 100644
> --- a/fs/cifs/smb2pdu.c
> +++ b/fs/cifs/smb2pdu.c
> @@ -2350,18 +2350,21 @@ SMB2_flush(const unsigned int xid, struct cifs_tcon *tcon, u64 persistent_fid,
>   */
>  static int
>  smb2_new_read_req(void **buf, unsigned int *total_len,
> -                 struct cifs_io_parms *io_parms, unsigned int remaining_bytes,
> -                 int request_type)
> +       struct cifs_io_parms *io_parms, struct cifs_readdata *rdata,
> +       unsigned int remaining_bytes, int request_type)
>  {
>         int rc = -EACCES;
>         struct smb2_read_plain_req *req = NULL;
>         struct smb2_sync_hdr *shdr;
> +       struct TCP_Server_Info *server;
>
>         rc = smb2_plain_req_init(SMB2_READ, io_parms->tcon, (void **) &req,
>                                  total_len);
>         if (rc)
>                 return rc;
> -       if (io_parms->tcon->ses->server == NULL)
> +
> +       server = io_parms->tcon->ses->server;
> +       if (server == NULL)
>                 return -ECONNABORTED;
>
>         shdr = &req->sync_hdr;
> @@ -2489,7 +2492,8 @@ smb2_async_readv(struct cifs_readdata *rdata)
>
>         server = io_parms.tcon->ses->server;
>
> -       rc = smb2_new_read_req((void **) &buf, &total_len, &io_parms, 0, 0);
> +       rc = smb2_new_read_req(
> +               (void **) &buf, &total_len, &io_parms, rdata, 0, 0);
>         if (rc) {
>                 if (rc == -EAGAIN && rdata->credits) {
>                         /* credits was reset by reconnect */
> @@ -2557,7 +2561,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
>         struct cifs_ses *ses = io_parms->tcon->ses;
>
>         *nbytes = 0;
> -       rc = smb2_new_read_req((void **)&req, &total_len, io_parms, 0, 0);
> +       rc = smb2_new_read_req((void **)&req, &total_len, io_parms, NULL, 0, 0);
>         if (rc)
>                 return rc;
>
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Looks good.

Reviewed-by: Pavel Shilovsky <pshilov-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

--
Best regards,
Pavel Shilovsky

Re: [Patch v7 01/22] CIFS: SMBD: Add parameter rdata to smb2_new_read_req

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

----- Original Message -----
From: "Long Li" <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>
To: "Steve French" <sfrench-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>, linux-cifs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, samba-technical-w/Ol4Ecudpl8XjKLYN78aQ@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "Christoph Hellwig" <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, "Tom Talpey" <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Matthew Wilcox" <mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Stephen Hemminger" <sthemmin-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Cc: "Long Li" <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Sent: Tuesday, 7 November, 2017 7:54:53 PM
Subject: [Patch v7 01/22] CIFS: SMBD: Add parameter rdata to smb2_new_read_req

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

This patch is for preparing upper layer for doing SMB read via RDMA write.

When we assemble the SMB read packet header, we need to know the I/O layout
if this request is to use a RDMA write. rdata has all the information we need
for memory registration. Add rdata to smb2_new_read_req.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/smb2pdu.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index bab3da6..32ad590 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -2350,18 +2350,21 @@ SMB2_flush(const unsigned int xid, struct cifs_tcon *tcon, u64 persistent_fid,
  */
 static int
 smb2_new_read_req(void **buf, unsigned int *total_len,
-		  struct cifs_io_parms *io_parms, unsigned int remaining_bytes,
-		  int request_type)
+	struct cifs_io_parms *io_parms, struct cifs_readdata *rdata,
+	unsigned int remaining_bytes, int request_type)
 {
 	int rc = -EACCES;
 	struct smb2_read_plain_req *req = NULL;
 	struct smb2_sync_hdr *shdr;
+	struct TCP_Server_Info *server;
 
 	rc = smb2_plain_req_init(SMB2_READ, io_parms->tcon, (void **) &req,
 				 total_len);
 	if (rc)
 		return rc;
-	if (io_parms->tcon->ses->server == NULL)
+
+	server = io_parms->tcon->ses->server;
+	if (server == NULL)
 		return -ECONNABORTED;
 
 	shdr = &req->sync_hdr;
@@ -2489,7 +2492,8 @@ smb2_async_readv(struct cifs_readdata *rdata)
 
 	server = io_parms.tcon->ses->server;
 
-	rc = smb2_new_read_req((void **) &buf, &total_len, &io_parms, 0, 0);
+	rc = smb2_new_read_req(
+		(void **) &buf, &total_len, &io_parms, rdata, 0, 0);
 	if (rc) {
 		if (rc == -EAGAIN && rdata->credits) {
 			/* credits was reset by reconnect */
@@ -2557,7 +2561,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
 	struct cifs_ses *ses = io_parms->tcon->ses;
 
 	*nbytes = 0;
-	rc = smb2_new_read_req((void **)&req, &total_len, io_parms, 0, 0);
+	rc = smb2_new_read_req((void **)&req, &total_len, io_parms, NULL, 0, 0);
 	if (rc)
 		return rc;
 
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 01/22] CIFS: SMBD: Add parameter rdata to smb2_new_read_req

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber@redhat.com>

----- Original Message -----
From: "Long Li" <longli@exchange.microsoft.com>
To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox" <mawilcox@microsoft.com>, "Stephen Hemminger" <sthemmin@microsoft.com>
Cc: "Long Li" <longli@microsoft.com>
Sent: Tuesday, 7 November, 2017 7:54:53 PM
Subject: [Patch v7 01/22] CIFS: SMBD: Add parameter rdata to smb2_new_read_req

From: Long Li <longli@microsoft.com>

This patch is for preparing upper layer for doing SMB read via RDMA write.

When we assemble the SMB read packet header, we need to know the I/O layout
if this request is to use a RDMA write. rdata has all the information we need
for memory registration. Add rdata to smb2_new_read_req.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smb2pdu.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index bab3da6..32ad590 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -2350,18 +2350,21 @@ SMB2_flush(const unsigned int xid, struct cifs_tcon *tcon, u64 persistent_fid,
  */
 static int
 smb2_new_read_req(void **buf, unsigned int *total_len,
-		  struct cifs_io_parms *io_parms, unsigned int remaining_bytes,
-		  int request_type)
+	struct cifs_io_parms *io_parms, struct cifs_readdata *rdata,
+	unsigned int remaining_bytes, int request_type)
 {
 	int rc = -EACCES;
 	struct smb2_read_plain_req *req = NULL;
 	struct smb2_sync_hdr *shdr;
+	struct TCP_Server_Info *server;
 
 	rc = smb2_plain_req_init(SMB2_READ, io_parms->tcon, (void **) &req,
 				 total_len);
 	if (rc)
 		return rc;
-	if (io_parms->tcon->ses->server == NULL)
+
+	server = io_parms->tcon->ses->server;
+	if (server == NULL)
 		return -ECONNABORTED;
 
 	shdr = &req->sync_hdr;
@@ -2489,7 +2492,8 @@ smb2_async_readv(struct cifs_readdata *rdata)
 
 	server = io_parms.tcon->ses->server;
 
-	rc = smb2_new_read_req((void **) &buf, &total_len, &io_parms, 0, 0);
+	rc = smb2_new_read_req(
+		(void **) &buf, &total_len, &io_parms, rdata, 0, 0);
 	if (rc) {
 		if (rc == -EAGAIN && rdata->credits) {
 			/* credits was reset by reconnect */
@@ -2557,7 +2561,7 @@ SMB2_read(const unsigned int xid, struct cifs_io_parms *io_parms,
 	struct cifs_ses *ses = io_parms->tcon->ses;
 
 	*nbytes = 0;
-	rc = smb2_new_read_req((void **)&req, &total_len, io_parms, 0, 0);
+	rc = smb2_new_read_req((void **)&req, &total_len, io_parms, NULL, 0, 0);
 	if (rc)
 		return rc;
 
-- 
2.7.4

[Patch v7 04/22] CIFS: SMBD: Add SMB Direct protocol initial values and constants

[Long Li]

From: Long Li <longli@microsoft.com>

To prepare for protocol implementation, add constants and user-configurable
values for the SMB Direct protocol.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smbdirect.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h | 21 +++++++++++++++
 2 files changed, 98 insertions(+)
 create mode 100644 fs/cifs/smbdirect.c
 create mode 100644 fs/cifs/smbdirect.h

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
new file mode 100644
index 0000000..d3c16f8
--- /dev/null
+++ b/fs/cifs/smbdirect.c
@@ -0,0 +1,77 @@
+/*
+ *   Copyright (C) 2017, Microsoft Corporation.
+ *
+ *   Author(s): Long Li <longli@microsoft.com>
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ */
+#include "smbdirect.h"
+
+/* SMBD version number */
+#define SMBD_V1	0x0100
+
+/* Port numbers for SMBD transport */
+#define SMB_PORT	445
+#define SMBD_PORT	5445
+
+/* Address lookup and resolve timeout in ms */
+#define RDMA_RESOLVE_TIMEOUT	5000
+
+/* SMBD negotiation timeout in seconds */
+#define SMBD_NEGOTIATE_TIMEOUT	120
+
+/* SMBD minimum receive size and fragmented sized defined in [MS-SMBD] */
+#define SMBD_MIN_RECEIVE_SIZE		128
+#define SMBD_MIN_FRAGMENTED_SIZE	131072
+
+/*
+ * Default maximum number of RDMA read/write outstanding on this connection
+ * This value is possibly decreased during QP creation on hardware limit
+ */
+#define SMBD_CM_RESPONDER_RESOURCES	32
+
+/* Maximum number of retries on data transfer operations */
+#define SMBD_CM_RETRY			6
+/* No need to retry on Receiver Not Ready since SMBD manages credits */
+#define SMBD_CM_RNR_RETRY		0
+
+/*
+ * User configurable initial values per SMBD transport connection
+ * as defined in [MS-SMBD] 3.1.1.1
+ * Those may change after a SMBD negotiation
+ */
+/* The local peer's maximum number of credits to grant to the peer */
+int smbd_receive_credit_max = 255;
+
+/* The remote peer's credit request of local peer */
+int smbd_send_credit_target = 255;
+
+/* The maximum single message size can be sent to remote peer */
+int smbd_max_send_size = 1364;
+
+/*  The maximum fragmented upper-layer payload receive size supported */
+int smbd_max_fragmented_recv_size = 1024 * 1024;
+
+/*  The maximum single-message size which can be received */
+int smbd_max_receive_size = 8192;
+
+/* The timeout to initiate send of a keepalive message on idle */
+int smbd_keep_alive_interval = 120;
+
+/*
+ * User configurable initial values for RDMA transport
+ * The actual values used may be lower and are limited to hardware capabilities
+ */
+/* Default maximum number of SGEs in a RDMA write/read */
+int smbd_max_frmr_depth = 2048;
+
+/* If payload is less than this byte, use RDMA send/recv not read/write */
+int rdma_readwrite_threshold = 4096;
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
new file mode 100644
index 0000000..c55f28b
--- /dev/null
+++ b/fs/cifs/smbdirect.h
@@ -0,0 +1,21 @@
+/*
+ *   Copyright (C) 2017, Microsoft Corporation.
+ *
+ *   Author(s): Long Li <longli@microsoft.com>
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ */
+#ifndef _SMBDIRECT_H
+#define _SMBDIRECT_H
+
+/* Default maximum number of SGEs in a RDMA send/recv */
+#define SMBDIRECT_MAX_SGE	16
+#endif
-- 
2.7.4

Re: [Patch v7 04/22] CIFS: SMBD: Add SMB Direct protocol initial values and constants

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>


----- Original Message -----
From: "Long Li" <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>
To: "Steve French" <sfrench-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>, linux-cifs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, samba-technical-w/Ol4Ecudpl8XjKLYN78aQ@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "Christoph Hellwig" <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, "Tom Talpey" <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Matthew Wilcox" <mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Stephen Hemminger" <sthemmin-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Cc: "Long Li" <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Sent: Tuesday, 7 November, 2017 7:54:56 PM
Subject: [Patch v7 04/22] CIFS: SMBD: Add SMB Direct protocol initial values and constants

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

To prepare for protocol implementation, add constants and user-configurable
values for the SMB Direct protocol.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/smbdirect.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h | 21 +++++++++++++++
 2 files changed, 98 insertions(+)
 create mode 100644 fs/cifs/smbdirect.c
 create mode 100644 fs/cifs/smbdirect.h

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
new file mode 100644
index 0000000..d3c16f8
--- /dev/null
+++ b/fs/cifs/smbdirect.c
@@ -0,0 +1,77 @@
+/*
+ *   Copyright (C) 2017, Microsoft Corporation.
+ *
+ *   Author(s): Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ */
+#include "smbdirect.h"
+
+/* SMBD version number */
+#define SMBD_V1	0x0100
+
+/* Port numbers for SMBD transport */
+#define SMB_PORT	445
+#define SMBD_PORT	5445
+
+/* Address lookup and resolve timeout in ms */
+#define RDMA_RESOLVE_TIMEOUT	5000
+
+/* SMBD negotiation timeout in seconds */
+#define SMBD_NEGOTIATE_TIMEOUT	120
+
+/* SMBD minimum receive size and fragmented sized defined in [MS-SMBD] */
+#define SMBD_MIN_RECEIVE_SIZE		128
+#define SMBD_MIN_FRAGMENTED_SIZE	131072
+
+/*
+ * Default maximum number of RDMA read/write outstanding on this connection
+ * This value is possibly decreased during QP creation on hardware limit
+ */
+#define SMBD_CM_RESPONDER_RESOURCES	32
+
+/* Maximum number of retries on data transfer operations */
+#define SMBD_CM_RETRY			6
+/* No need to retry on Receiver Not Ready since SMBD manages credits */
+#define SMBD_CM_RNR_RETRY		0
+
+/*
+ * User configurable initial values per SMBD transport connection
+ * as defined in [MS-SMBD] 3.1.1.1
+ * Those may change after a SMBD negotiation
+ */
+/* The local peer's maximum number of credits to grant to the peer */
+int smbd_receive_credit_max = 255;
+
+/* The remote peer's credit request of local peer */
+int smbd_send_credit_target = 255;
+
+/* The maximum single message size can be sent to remote peer */
+int smbd_max_send_size = 1364;
+
+/*  The maximum fragmented upper-layer payload receive size supported */
+int smbd_max_fragmented_recv_size = 1024 * 1024;
+
+/*  The maximum single-message size which can be received */
+int smbd_max_receive_size = 8192;
+
+/* The timeout to initiate send of a keepalive message on idle */
+int smbd_keep_alive_interval = 120;
+
+/*
+ * User configurable initial values for RDMA transport
+ * The actual values used may be lower and are limited to hardware capabilities
+ */
+/* Default maximum number of SGEs in a RDMA write/read */
+int smbd_max_frmr_depth = 2048;
+
+/* If payload is less than this byte, use RDMA send/recv not read/write */
+int rdma_readwrite_threshold = 4096;
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
new file mode 100644
index 0000000..c55f28b
--- /dev/null
+++ b/fs/cifs/smbdirect.h
@@ -0,0 +1,21 @@
+/*
+ *   Copyright (C) 2017, Microsoft Corporation.
+ *
+ *   Author(s): Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ */
+#ifndef _SMBDIRECT_H
+#define _SMBDIRECT_H
+
+/* Default maximum number of SGEs in a RDMA send/recv */
+#define SMBDIRECT_MAX_SGE	16
+#endif
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 04/22] CIFS: SMBD: Add SMB Direct protocol initial values and constants

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber@redhat.com>


----- Original Message -----
From: "Long Li" <longli@exchange.microsoft.com>
To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox" <mawilcox@microsoft.com>, "Stephen Hemminger" <sthemmin@microsoft.com>
Cc: "Long Li" <longli@microsoft.com>
Sent: Tuesday, 7 November, 2017 7:54:56 PM
Subject: [Patch v7 04/22] CIFS: SMBD: Add SMB Direct protocol initial values and constants

From: Long Li <longli@microsoft.com>

To prepare for protocol implementation, add constants and user-configurable
values for the SMB Direct protocol.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smbdirect.c | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h | 21 +++++++++++++++
 2 files changed, 98 insertions(+)
 create mode 100644 fs/cifs/smbdirect.c
 create mode 100644 fs/cifs/smbdirect.h

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
new file mode 100644
index 0000000..d3c16f8
--- /dev/null
+++ b/fs/cifs/smbdirect.c
@@ -0,0 +1,77 @@
+/*
+ *   Copyright (C) 2017, Microsoft Corporation.
+ *
+ *   Author(s): Long Li <longli@microsoft.com>
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ */
+#include "smbdirect.h"
+
+/* SMBD version number */
+#define SMBD_V1	0x0100
+
+/* Port numbers for SMBD transport */
+#define SMB_PORT	445
+#define SMBD_PORT	5445
+
+/* Address lookup and resolve timeout in ms */
+#define RDMA_RESOLVE_TIMEOUT	5000
+
+/* SMBD negotiation timeout in seconds */
+#define SMBD_NEGOTIATE_TIMEOUT	120
+
+/* SMBD minimum receive size and fragmented sized defined in [MS-SMBD] */
+#define SMBD_MIN_RECEIVE_SIZE		128
+#define SMBD_MIN_FRAGMENTED_SIZE	131072
+
+/*
+ * Default maximum number of RDMA read/write outstanding on this connection
+ * This value is possibly decreased during QP creation on hardware limit
+ */
+#define SMBD_CM_RESPONDER_RESOURCES	32
+
+/* Maximum number of retries on data transfer operations */
+#define SMBD_CM_RETRY			6
+/* No need to retry on Receiver Not Ready since SMBD manages credits */
+#define SMBD_CM_RNR_RETRY		0
+
+/*
+ * User configurable initial values per SMBD transport connection
+ * as defined in [MS-SMBD] 3.1.1.1
+ * Those may change after a SMBD negotiation
+ */
+/* The local peer's maximum number of credits to grant to the peer */
+int smbd_receive_credit_max = 255;
+
+/* The remote peer's credit request of local peer */
+int smbd_send_credit_target = 255;
+
+/* The maximum single message size can be sent to remote peer */
+int smbd_max_send_size = 1364;
+
+/*  The maximum fragmented upper-layer payload receive size supported */
+int smbd_max_fragmented_recv_size = 1024 * 1024;
+
+/*  The maximum single-message size which can be received */
+int smbd_max_receive_size = 8192;
+
+/* The timeout to initiate send of a keepalive message on idle */
+int smbd_keep_alive_interval = 120;
+
+/*
+ * User configurable initial values for RDMA transport
+ * The actual values used may be lower and are limited to hardware capabilities
+ */
+/* Default maximum number of SGEs in a RDMA write/read */
+int smbd_max_frmr_depth = 2048;
+
+/* If payload is less than this byte, use RDMA send/recv not read/write */
+int rdma_readwrite_threshold = 4096;
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
new file mode 100644
index 0000000..c55f28b
--- /dev/null
+++ b/fs/cifs/smbdirect.h
@@ -0,0 +1,21 @@
+/*
+ *   Copyright (C) 2017, Microsoft Corporation.
+ *
+ *   Author(s): Long Li <longli@microsoft.com>
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ */
+#ifndef _SMBDIRECT_H
+#define _SMBDIRECT_H
+
+/* Default maximum number of SGEs in a RDMA send/recv */
+#define SMBDIRECT_MAX_SGE	16
+#endif
-- 
2.7.4

[Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

[Long Li]

From: Long Li <longli@microsoft.com>

Add code to implement the core functions to establish a SMB Direct connection.

1. Establish an RDMA connection to SMB server.
2. Negotiate and setup SMB Direct protocol.
3. Implement idle connection timer and credit management.

SMB Direct is enabled by setting CONFIG_CIFS_SMB_DIRECT.

Add to Makefile to enable building SMB Direct.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/Makefile    |    2 +
 fs/cifs/smbdirect.c | 1576 +++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h |  280 +++++++++
 3 files changed, 1858 insertions(+)

diff --git a/fs/cifs/Makefile b/fs/cifs/Makefile
index 5e853a3..ad00873 100644
--- a/fs/cifs/Makefile
+++ b/fs/cifs/Makefile
@@ -18,3 +18,5 @@ cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
 cifs-$(CONFIG_CIFS_DFS_UPCALL) += dns_resolve.o cifs_dfs_ref.o
 
 cifs-$(CONFIG_CIFS_FSCACHE) += fscache.o cache.o
+
+cifs-$(CONFIG_CIFS_SMB_DIRECT) += smbdirect.o
diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index d3c16f8..021d527 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -13,7 +13,34 @@
  *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
  *   the GNU General Public License for more details.
  */
+#include <linux/module.h>
 #include "smbdirect.h"
+#include "cifs_debug.h"
+
+static struct smbd_response *get_empty_queue_buffer(
+		struct smbd_connection *info);
+static struct smbd_response *get_receive_buffer(
+		struct smbd_connection *info);
+static void put_receive_buffer(
+		struct smbd_connection *info,
+		struct smbd_response *response,
+		bool lock);
+static int allocate_receive_buffers(struct smbd_connection *info, int num_buf);
+static void destroy_receive_buffers(struct smbd_connection *info);
+
+static void put_empty_packet(
+		struct smbd_connection *info, struct smbd_response *response);
+static void enqueue_reassembly(
+		struct smbd_connection *info,
+		struct smbd_response *response, int data_length);
+static struct smbd_response *_get_first_reassembly(
+		struct smbd_connection *info);
+
+static int smbd_post_recv(
+		struct smbd_connection *info,
+		struct smbd_response *response);
+
+static int smbd_post_send_empty(struct smbd_connection *info);
 
 /* SMBD version number */
 #define SMBD_V1	0x0100
@@ -75,3 +102,1552 @@ int smbd_max_frmr_depth = 2048;
 
 /* If payload is less than this byte, use RDMA send/recv not read/write */
 int rdma_readwrite_threshold = 4096;
+
+/* Transport logging functions
+ * Logging are defined as classes. They can be OR'ed to define the actual
+ * logging level via module parameter smbd_logging_class
+ * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
+ * log_rdma_event()
+ */
+#define LOG_OUTGOING			0x1
+#define LOG_INCOMING			0x2
+#define LOG_READ			0x4
+#define LOG_WRITE			0x8
+#define LOG_RDMA_SEND			0x10
+#define LOG_RDMA_RECV			0x20
+#define LOG_KEEP_ALIVE			0x40
+#define LOG_RDMA_EVENT			0x80
+#define LOG_RDMA_MR			0x100
+static unsigned int smbd_logging_class = 0;
+module_param(smbd_logging_class, uint, 0644);
+MODULE_PARM_DESC(smbd_logging_class,
+	"Logging class for SMBD transport 0x0 to 0x100");
+
+#define ERR		0x0
+#define INFO		0x1
+static unsigned int smbd_logging_level = ERR;
+module_param(smbd_logging_level, uint, 0644);
+MODULE_PARM_DESC(smbd_logging_level,
+	"Logging level for SMBD transport, 0 (default): error, 1: info");
+
+#define log_rdma(level, class, fmt, args...)				\
+do {									\
+	if (level <= smbd_logging_level || class & smbd_logging_class)	\
+		cifs_dbg(VFS, "%s:%d " fmt, __func__, __LINE__, ##args);\
+} while (0)
+
+#define log_outgoing(level, fmt, args...) \
+		log_rdma(level, LOG_OUTGOING, fmt, ##args)
+#define log_incoming(level, fmt, args...) \
+		log_rdma(level, LOG_INCOMING, fmt, ##args)
+#define log_read(level, fmt, args...)	log_rdma(level, LOG_READ, fmt, ##args)
+#define log_write(level, fmt, args...)	log_rdma(level, LOG_WRITE, fmt, ##args)
+#define log_rdma_send(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_SEND, fmt, ##args)
+#define log_rdma_recv(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_RECV, fmt, ##args)
+#define log_keep_alive(level, fmt, args...) \
+		log_rdma(level, LOG_KEEP_ALIVE, fmt, ##args)
+#define log_rdma_event(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_EVENT, fmt, ##args)
+#define log_rdma_mr(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_MR, fmt, ##args)
+
+/*
+ * Destroy the transport and related RDMA and memory resources
+ * Need to go through all the pending counters and make sure on one is using
+ * the transport while it is destroyed
+ */
+static void smbd_destroy_rdma_work(struct work_struct *work)
+{
+	struct smbd_response *response;
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, destroy_work);
+	unsigned long flags;
+
+	log_rdma_event(INFO, "destroying qp\n");
+	ib_drain_qp(info->id->qp);
+	rdma_destroy_qp(info->id);
+
+	/* Unblock all I/O waiting on the send queue */
+	wake_up_interruptible_all(&info->wait_send_queue);
+
+	log_rdma_event(INFO, "cancelling idle timer\n");
+	cancel_delayed_work_sync(&info->idle_timer_work);
+	log_rdma_event(INFO, "cancelling send immediate work\n");
+	cancel_delayed_work_sync(&info->send_immediate_work);
+
+	log_rdma_event(INFO, "wait for all recv to finish\n");
+	wake_up_interruptible(&info->wait_reassembly_queue);
+
+	log_rdma_event(INFO, "wait for all send posted to IB to finish\n");
+	wait_event(info->wait_send_pending,
+		atomic_read(&info->send_pending) == 0);
+	wait_event(info->wait_send_payload_pending,
+		atomic_read(&info->send_payload_pending) == 0);
+
+	/* It's not posssible for upper layer to get to reassembly */
+	log_rdma_event(INFO, "drain the reassembly queue\n");
+	do {
+		spin_lock_irqsave(&info->reassembly_queue_lock, flags);
+		response = _get_first_reassembly(info);
+		if (response) {
+			list_del(&response->list);
+			spin_unlock_irqrestore(
+				&info->reassembly_queue_lock, flags);
+			put_receive_buffer(info, response, true);
+		}
+	} while (response);
+	spin_unlock_irqrestore(&info->reassembly_queue_lock, flags);
+	info->reassembly_data_length = 0;
+
+	log_rdma_event(INFO, "free receive buffers\n");
+	wait_event(info->wait_receive_queues,
+		info->count_receive_queue + info->count_empty_packet_queue
+			== info->receive_credit_max);
+	destroy_receive_buffers(info);
+
+	ib_free_cq(info->send_cq);
+	ib_free_cq(info->recv_cq);
+	ib_dealloc_pd(info->pd);
+	rdma_destroy_id(info->id);
+
+	/* free mempools */
+	mempool_destroy(info->request_mempool);
+	kmem_cache_destroy(info->request_cache);
+
+	mempool_destroy(info->response_mempool);
+	kmem_cache_destroy(info->response_cache);
+
+	info->transport_status = SMBD_DESTROYED;
+	wake_up_all(&info->wait_destroy);
+}
+
+static int smbd_process_disconnected(struct smbd_connection *info)
+{
+//	queue_work(info->workqueue, &info->destroy_work);
+	schedule_work(&info->destroy_work);
+	return 0;
+}
+
+static void smbd_disconnect_rdma_work(struct work_struct *work)
+{
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, disconnect_work);
+
+	if (info->transport_status == SMBD_CONNECTED) {
+		info->transport_status = SMBD_DISCONNECTING;
+		rdma_disconnect(info->id);
+	}
+}
+
+static void smbd_disconnect_rdma_connection(struct smbd_connection *info)
+{
+	queue_work(info->workqueue, &info->disconnect_work);
+}
+
+/* Upcall from RDMA CM */
+static int smbd_conn_upcall(
+		struct rdma_cm_id *id, struct rdma_cm_event *event)
+{
+	struct smbd_connection *info = id->context;
+
+	log_rdma_event(INFO, "event=%d status=%d\n",
+		event->event, event->status);
+
+	switch (event->event) {
+	case RDMA_CM_EVENT_ADDR_RESOLVED:
+	case RDMA_CM_EVENT_ROUTE_RESOLVED:
+		info->ri_rc = 0;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ADDR_ERROR:
+		info->ri_rc = -EHOSTUNREACH;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ROUTE_ERROR:
+		info->ri_rc = -ENETUNREACH;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ESTABLISHED:
+		log_rdma_event(INFO, "connected event=%d\n", event->event);
+		info->transport_status = SMBD_CONNECTED;
+		wake_up_interruptible(&info->conn_wait);
+		break;
+
+	case RDMA_CM_EVENT_CONNECT_ERROR:
+	case RDMA_CM_EVENT_UNREACHABLE:
+	case RDMA_CM_EVENT_REJECTED:
+		log_rdma_event(INFO, "connecting failed event=%d\n", event->event);
+		info->transport_status = SMBD_DISCONNECTED;
+		wake_up_interruptible(&info->conn_wait);
+		break;
+
+	case RDMA_CM_EVENT_DEVICE_REMOVAL:
+	case RDMA_CM_EVENT_DISCONNECTED:
+		/* This happenes when we fail the negotiation */
+		if (info->transport_status == SMBD_NEGOTIATE_FAILED) {
+			info->transport_status = SMBD_DISCONNECTED;
+			wake_up(&info->conn_wait);
+			break;
+		}
+
+		info->transport_status = SMBD_DISCONNECTED;
+		smbd_process_disconnected(info);
+		break;
+
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+/* Upcall from RDMA QP */
+static void
+smbd_qp_async_error_upcall(struct ib_event *event, void *context)
+{
+	struct smbd_connection *info = context;
+
+	log_rdma_event(ERR, "%s on device %s info %p\n",
+		ib_event_msg(event->event), event->device->name, info);
+
+	switch (event->event) {
+	case IB_EVENT_CQ_ERR:
+	case IB_EVENT_QP_FATAL:
+		smbd_disconnect_rdma_connection(info);
+
+	default:
+		break;
+	}
+}
+
+static inline void *smbd_request_payload(struct smbd_request *request)
+{
+	return (void *)request->packet;
+}
+
+static inline void *smbd_response_payload(struct smbd_response *response)
+{
+	return (void *)response->packet;
+}
+
+/* Called when a RDMA send is done */
+static void send_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+	int i;
+	struct smbd_request *request =
+		container_of(wc->wr_cqe, struct smbd_request, cqe);
+
+	log_rdma_send(INFO, "smbd_request %p completed wc->status=%d\n",
+		request, wc->status);
+
+	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_SEND) {
+		log_rdma_send(ERR, "wc->status=%d wc->opcode=%d\n",
+			wc->status, wc->opcode);
+		smbd_disconnect_rdma_connection(request->info);
+	}
+
+	for (i = 0; i < request->num_sge; i++)
+		ib_dma_unmap_single(request->info->id->device,
+			request->sge[i].addr,
+			request->sge[i].length,
+			DMA_TO_DEVICE);
+
+	if (request->has_payload) {
+		if (atomic_dec_and_test(&request->info->send_payload_pending))
+			wake_up(&request->info->wait_send_payload_pending);
+	} else {
+		if (atomic_dec_and_test(&request->info->send_pending))
+			wake_up(&request->info->wait_send_pending);
+	}
+
+	mempool_free(request, request->info->request_mempool);
+}
+
+static void dump_smbd_negotiate_resp(struct smbd_negotiate_resp *resp)
+{
+	log_rdma_event(INFO, "resp message min_version %u max_version %u "
+		"negotiated_version %u credits_requested %u "
+		"credits_granted %u status %u max_readwrite_size %u "
+		"preferred_send_size %u max_receive_size %u "
+		"max_fragmented_size %u\n",
+		resp->min_version, resp->max_version, resp->negotiated_version,
+		resp->credits_requested, resp->credits_granted, resp->status,
+		resp->max_readwrite_size, resp->preferred_send_size,
+		resp->max_receive_size, resp->max_fragmented_size);
+}
+
+/*
+ * Process a negotiation response message, according to [MS-SMBD]3.1.5.7
+ * response, packet_length: the negotiation response message
+ * return value: true if negotiation is a success, false if failed
+ */
+static bool process_negotiation_response(
+		struct smbd_response *response, int packet_length)
+{
+	struct smbd_connection *info = response->info;
+	struct smbd_negotiate_resp *packet = smbd_response_payload(response);
+
+	if (packet_length < sizeof(struct smbd_negotiate_resp)) {
+		log_rdma_event(ERR,
+			"error: packet_length=%d\n", packet_length);
+		return false;
+	}
+
+	if (le16_to_cpu(packet->negotiated_version) != SMBD_V1) {
+		log_rdma_event(ERR, "error: negotiated_version=%x\n",
+			le16_to_cpu(packet->negotiated_version));
+		return false;
+	}
+	info->protocol = le16_to_cpu(packet->negotiated_version);
+
+	if (packet->credits_requested == 0) {
+		log_rdma_event(ERR, "error: credits_requested==0\n");
+		return false;
+	}
+	info->receive_credit_target = le16_to_cpu(packet->credits_requested);
+
+	if (packet->credits_granted == 0) {
+		log_rdma_event(ERR, "error: credits_granted==0\n");
+		return false;
+	}
+	atomic_set(&info->send_credits, le16_to_cpu(packet->credits_granted));
+
+	atomic_set(&info->receive_credits, 0);
+
+	if (le32_to_cpu(packet->preferred_send_size) > info->max_receive_size) {
+		log_rdma_event(ERR, "error: preferred_send_size=%d\n",
+			le32_to_cpu(packet->preferred_send_size));
+		return false;
+	}
+	info->max_receive_size = le32_to_cpu(packet->preferred_send_size);
+
+	if (le32_to_cpu(packet->max_receive_size) < SMBD_MIN_RECEIVE_SIZE) {
+		log_rdma_event(ERR, "error: max_receive_size=%d\n",
+			le32_to_cpu(packet->max_receive_size));
+		return false;
+	}
+	info->max_send_size = min_t(int, info->max_send_size,
+					le32_to_cpu(packet->max_receive_size));
+
+	if (le32_to_cpu(packet->max_fragmented_size) <
+			SMBD_MIN_FRAGMENTED_SIZE) {
+		log_rdma_event(ERR, "error: max_fragmented_size=%d\n",
+			le32_to_cpu(packet->max_fragmented_size));
+		return false;
+	}
+	info->max_fragmented_send_size =
+		le32_to_cpu(packet->max_fragmented_size);
+
+	return true;
+}
+
+/*
+ * Check and schedule to send an immediate packet
+ * This is used to extend credtis to remote peer to keep the transport busy
+ */
+static void check_and_send_immediate(struct smbd_connection *info)
+{
+	if (info->transport_status != SMBD_CONNECTED)
+		return;
+
+	info->send_immediate = true;
+
+	/*
+	 * Promptly send a packet if our peer is running low on receive
+	 * credits
+	 */
+	if (atomic_read(&info->receive_credits) <
+		info->receive_credit_target - 1)
+		queue_delayed_work(
+			info->workqueue, &info->send_immediate_work, 0);
+}
+
+static void smbd_post_send_credits(struct work_struct *work)
+{
+	int ret = 0;
+	int use_receive_queue = 1;
+	int rc;
+	struct smbd_response *response;
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection,
+			post_send_credits_work);
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		wake_up(&info->wait_receive_queues);
+		return;
+	}
+
+	if (info->receive_credit_target >
+		atomic_read(&info->receive_credits)) {
+		while (true) {
+			if (use_receive_queue)
+				response = get_receive_buffer(info);
+			else
+				response = get_empty_queue_buffer(info);
+			if (!response) {
+				/* now switch to emtpy packet queue */
+				if (use_receive_queue) {
+					use_receive_queue = 0;
+					continue;
+				} else
+					break;
+			}
+
+			response->type = SMBD_TRANSFER_DATA;
+			response->first_segment = false;
+			rc = smbd_post_recv(info, response);
+			if (rc) {
+				log_rdma_recv(ERR,
+					"post_recv failed rc=%d\n", rc);
+				put_receive_buffer(info, response, true);
+				break;
+			}
+
+			ret++;
+		}
+	}
+
+	spin_lock(&info->lock_new_credits_offered);
+	info->new_credits_offered += ret;
+	spin_unlock(&info->lock_new_credits_offered);
+
+	atomic_add(ret, &info->receive_credits);
+
+	/* Check if we can post new receive and grant credits to peer */
+	check_and_send_immediate(info);
+}
+
+static void smbd_recv_done_work(struct work_struct *work)
+{
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, recv_done_work);
+
+	/*
+	 * We may have new send credits granted from remote peer
+	 * If any sender is blcoked on lack of credets, unblock it
+	 */
+	if (atomic_read(&info->send_credits))
+		wake_up_interruptible(&info->wait_send_queue);
+
+	/*
+	 * Check if we need to send something to remote peer to
+	 * grant more credits or respond to KEEP_ALIVE packet
+	 */
+	check_and_send_immediate(info);
+}
+
+/* Called from softirq, when recv is done */
+static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+	struct smbd_data_transfer *data_transfer;
+	struct smbd_response *response =
+		container_of(wc->wr_cqe, struct smbd_response, cqe);
+	struct smbd_connection *info = response->info;
+	int data_length = 0;
+
+	log_rdma_recv(INFO, "response=%p type=%d wc status=%d wc opcode %d "
+		      "byte_len=%d pkey_index=%x\n",
+		response, response->type, wc->status, wc->opcode,
+		wc->byte_len, wc->pkey_index);
+
+	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
+		log_rdma_recv(INFO, "wc->status=%d opcode=%d\n",
+			wc->status, wc->opcode);
+		smbd_disconnect_rdma_connection(info);
+		goto error;
+	}
+
+	ib_dma_sync_single_for_cpu(
+		wc->qp->device,
+		response->sge.addr,
+		response->sge.length,
+		DMA_FROM_DEVICE);
+
+	switch (response->type) {
+	/* SMBD negotiation response */
+	case SMBD_NEGOTIATE_RESP:
+		dump_smbd_negotiate_resp(smbd_response_payload(response));
+		info->full_packet_received = true;
+		info->negotiate_done =
+			process_negotiation_response(response, wc->byte_len);
+		complete(&info->negotiate_completion);
+		break;
+
+	/* SMBD data transfer packet */
+	case SMBD_TRANSFER_DATA:
+		data_transfer = smbd_response_payload(response);
+		data_length = le32_to_cpu(data_transfer->data_length);
+
+		/*
+		 * If this is a packet with data playload place the data in
+		 * reassembly queue and wake up the reading thread
+		 */
+		if (data_length) {
+			if (info->full_packet_received)
+				response->first_segment = true;
+
+			if (le32_to_cpu(data_transfer->remaining_data_length))
+				info->full_packet_received = false;
+			else
+				info->full_packet_received = true;
+
+			enqueue_reassembly(
+				info,
+				response,
+				data_length);
+		} else
+			put_empty_packet(info, response);
+
+		if (data_length)
+			wake_up_interruptible(&info->wait_reassembly_queue);
+
+		atomic_dec(&info->receive_credits);
+		info->receive_credit_target =
+			le16_to_cpu(data_transfer->credits_requested);
+		atomic_add(le16_to_cpu(data_transfer->credits_granted),
+			&info->send_credits);
+
+		log_incoming(INFO, "data flags %d data_offset %d "
+			"data_length %d remaining_data_length %d\n",
+			le16_to_cpu(data_transfer->flags),
+			le32_to_cpu(data_transfer->data_offset),
+			le32_to_cpu(data_transfer->data_length),
+			le32_to_cpu(data_transfer->remaining_data_length));
+
+		/* Send a KEEP_ALIVE response right away if requested */
+		info->keep_alive_requested = KEEP_ALIVE_NONE;
+		if (le16_to_cpu(data_transfer->flags) &
+				SMB_DIRECT_RESPONSE_REQUESTED) {
+			info->keep_alive_requested = KEEP_ALIVE_PENDING;
+		}
+
+		queue_work(info->workqueue, &info->recv_done_work);
+		return;
+
+	default:
+		log_rdma_recv(ERR,
+			"unexpected response type=%d\n", response->type);
+	}
+
+error:
+	put_receive_buffer(info, response, true);
+}
+
+static struct rdma_cm_id *smbd_create_id(
+		struct smbd_connection *info,
+		struct sockaddr *dstaddr, int port)
+{
+	struct rdma_cm_id *id;
+	int rc;
+	__be16 *sport;
+
+	id = rdma_create_id(&init_net, smbd_conn_upcall, info,
+		RDMA_PS_TCP, IB_QPT_RC);
+	if (IS_ERR(id)) {
+		rc = PTR_ERR(id);
+		log_rdma_event(ERR, "rdma_create_id() failed %i\n", rc);
+		return id;
+	}
+
+	if (dstaddr->sa_family == AF_INET6)
+		sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port;
+	else
+		sport = &((struct sockaddr_in *)dstaddr)->sin_port;
+
+	*sport = htons(port);
+
+	init_completion(&info->ri_done);
+	info->ri_rc = -ETIMEDOUT;
+
+	rc = rdma_resolve_addr(id, NULL, (struct sockaddr *)dstaddr,
+		RDMA_RESOLVE_TIMEOUT);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_addr() failed %i\n", rc);
+		goto out;
+	}
+	wait_for_completion_interruptible_timeout(
+		&info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
+	rc = info->ri_rc;
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_addr() completed %i\n", rc);
+		goto out;
+	}
+
+	info->ri_rc = -ETIMEDOUT;
+	rc = rdma_resolve_route(id, RDMA_RESOLVE_TIMEOUT);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_route() failed %i\n", rc);
+		goto out;
+	}
+	wait_for_completion_interruptible_timeout(
+		&info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
+	rc = info->ri_rc;
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_route() completed %i\n", rc);
+		goto out;
+	}
+
+	return id;
+
+out:
+	rdma_destroy_id(id);
+	return ERR_PTR(rc);
+}
+
+/*
+ * Test if FRWR (Fast Registration Work Requests) is supported on the device
+ * This implementation requries FRWR on RDMA read/write
+ * return value: true if it is supported
+ */
+static bool frwr_is_supported(struct ib_device_attr *attrs)
+{
+	if (!(attrs->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS))
+		return false;
+	if (attrs->max_fast_reg_page_list_len == 0)
+		return false;
+	return true;
+}
+
+static int smbd_ia_open(
+		struct smbd_connection *info,
+		struct sockaddr *dstaddr, int port)
+{
+	int rc;
+
+	info->id = smbd_create_id(info, dstaddr, port);
+	if (IS_ERR(info->id)) {
+		rc = PTR_ERR(info->id);
+		goto out1;
+	}
+
+	if (!frwr_is_supported(&info->id->device->attrs)) {
+		log_rdma_event(ERR,
+			"Fast Registration Work Requests "
+			"(FRWR) is not supported\n");
+		log_rdma_event(ERR,
+			"Device capability flags = %llx "
+			"max_fast_reg_page_list_len = %u\n",
+			info->id->device->attrs.device_cap_flags,
+			info->id->device->attrs.max_fast_reg_page_list_len);
+		rc = -EPROTONOSUPPORT;
+		goto out2;
+	}
+
+	info->pd = ib_alloc_pd(info->id->device, 0);
+	if (IS_ERR(info->pd)) {
+		rc = PTR_ERR(info->pd);
+		log_rdma_event(ERR, "ib_alloc_pd() returned %d\n", rc);
+		goto out2;
+	}
+
+	return 0;
+
+out2:
+	rdma_destroy_id(info->id);
+	info->id = NULL;
+
+out1:
+	return rc;
+}
+
+/*
+ * Send a negotiation request message to the peer
+ * The negotiation procedure is in [MS-SMBD] 3.1.5.2 and 3.1.5.3
+ * After negotiation, the transport is connected and ready for
+ * carrying upper layer SMB payload
+ */
+static int smbd_post_send_negotiate_req(struct smbd_connection *info)
+{
+	struct ib_send_wr send_wr, *send_wr_fail;
+	int rc = -ENOMEM;
+	struct smbd_request *request;
+	struct smbd_negotiate_req *packet;
+
+	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
+	if (!request)
+		return rc;
+
+	request->info = info;
+
+	packet = smbd_request_payload(request);
+	packet->min_version = cpu_to_le16(SMBD_V1);
+	packet->max_version = cpu_to_le16(SMBD_V1);
+	packet->reserved = 0;
+	packet->credits_requested = cpu_to_le16(info->send_credit_target);
+	packet->preferred_send_size = cpu_to_le32(info->max_send_size);
+	packet->max_receive_size = cpu_to_le32(info->max_receive_size);
+	packet->max_fragmented_size =
+		cpu_to_le32(info->max_fragmented_recv_size);
+
+	request->num_sge = 1;
+	request->sge[0].addr = ib_dma_map_single(
+				info->id->device, (void *)packet,
+				sizeof(*packet), DMA_TO_DEVICE);
+	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
+		rc = -EIO;
+		goto dma_mapping_failed;
+	}
+
+	request->sge[0].length = sizeof(*packet);
+	request->sge[0].lkey = info->pd->local_dma_lkey;
+
+	ib_dma_sync_single_for_device(
+		info->id->device, request->sge[0].addr,
+		request->sge[0].length, DMA_TO_DEVICE);
+
+	request->cqe.done = send_done;
+
+	send_wr.next = NULL;
+	send_wr.wr_cqe = &request->cqe;
+	send_wr.sg_list = request->sge;
+	send_wr.num_sge = request->num_sge;
+	send_wr.opcode = IB_WR_SEND;
+	send_wr.send_flags = IB_SEND_SIGNALED;
+
+	log_rdma_send(INFO, "sge addr=%llx length=%x lkey=%x\n",
+		request->sge[0].addr,
+		request->sge[0].length, request->sge[0].lkey);
+
+	request->has_payload = false;
+	atomic_inc(&info->send_pending);
+	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
+	if (!rc)
+		return 0;
+
+	/* if we reach here, post send failed */
+	log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
+	atomic_dec(&info->send_pending);
+	ib_dma_unmap_single(info->id->device, request->sge[0].addr,
+		request->sge[0].length, DMA_TO_DEVICE);
+
+dma_mapping_failed:
+	mempool_free(request, info->request_mempool);
+	return rc;
+}
+
+/*
+ * Extend the credits to remote peer
+ * This implements [MS-SMBD] 3.1.5.9
+ * The idea is that we should extend credits to remote peer as quickly as
+ * it's allowed, to maintain data flow. We allocate as much receive
+ * buffer as possible, and extend the receive credits to remote peer
+ * return value: the new credtis being granted.
+ */
+static int manage_credits_prior_sending(struct smbd_connection *info)
+{
+	int new_credits;
+
+	spin_lock(&info->lock_new_credits_offered);
+	new_credits = info->new_credits_offered;
+	info->new_credits_offered = 0;
+	spin_unlock(&info->lock_new_credits_offered);
+
+	return new_credits;
+}
+
+/*
+ * Check if we need to send a KEEP_ALIVE message
+ * The idle connection timer triggers a KEEP_ALIVE message when expires
+ * SMB_DIRECT_RESPONSE_REQUESTED is set in the message flag to have peer send
+ * back a response.
+ * return value:
+ * 1 if SMB_DIRECT_RESPONSE_REQUESTED needs to be set
+ * 0: otherwise
+ */
+static int manage_keep_alive_before_sending(struct smbd_connection *info)
+{
+	if (info->keep_alive_requested == KEEP_ALIVE_PENDING) {
+		info->keep_alive_requested = KEEP_ALIVE_SENT;
+		return 1;
+	}
+	return 0;
+}
+
+/*
+ * Build and prepare the SMBD packet header
+ * This function waits for avaialbe send credits and build a SMBD packet
+ * header. The caller then optional append payload to the packet after
+ * the header
+ * intput values
+ * size: the size of the payload
+ * remaining_data_length: remaining data to send if this is part of a
+ * fragmented packet
+ * output values
+ * request_out: the request allocated from this function
+ * return values: 0 on success, otherwise actual error code returned
+ */
+static int smbd_create_header(struct smbd_connection *info,
+		int size, int remaining_data_length,
+		struct smbd_request **request_out)
+{
+	struct smbd_request *request;
+	struct smbd_data_transfer *packet;
+	int header_length;
+	int rc;
+
+	/* Wait for send credits. A SMBD packet needs one credit */
+	rc = wait_event_interruptible(info->wait_send_queue,
+		atomic_read(&info->send_credits) > 0 ||
+		info->transport_status != SMBD_CONNECTED);
+	if (rc)
+		return rc;
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		log_outgoing(ERR, "disconnected not sending\n");
+		return -ENOENT;
+	}
+	atomic_dec(&info->send_credits);
+
+	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
+	if (!request) {
+		rc = -ENOMEM;
+		goto err;
+	}
+
+	request->info = info;
+
+	/* Fill in the packet header */
+	packet = smbd_request_payload(request);
+	packet->credits_requested = cpu_to_le16(info->send_credit_target);
+	packet->credits_granted =
+		cpu_to_le16(manage_credits_prior_sending(info));
+	info->send_immediate = false;
+
+	packet->flags = 0;
+	if (manage_keep_alive_before_sending(info))
+		packet->flags |= cpu_to_le16(SMB_DIRECT_RESPONSE_REQUESTED);
+
+	packet->reserved = 0;
+	if (!size)
+		packet->data_offset = 0;
+	else
+		packet->data_offset = cpu_to_le32(24);
+	packet->data_length = cpu_to_le32(size);
+	packet->remaining_data_length = cpu_to_le32(remaining_data_length);
+	packet->padding = 0;
+
+	log_outgoing(INFO, "credits_requested=%d credits_granted=%d "
+		"data_offset=%d data_length=%d remaining_data_length=%d\n",
+		le16_to_cpu(packet->credits_requested),
+		le16_to_cpu(packet->credits_granted),
+		le32_to_cpu(packet->data_offset),
+		le32_to_cpu(packet->data_length),
+		le32_to_cpu(packet->remaining_data_length));
+
+	/* Map the packet to DMA */
+	header_length = sizeof(struct smbd_data_transfer);
+	/* If this is a packet without payload, don't send padding */
+	if (!size)
+		header_length = offsetof(struct smbd_data_transfer, padding);
+
+	request->num_sge = 1;
+	request->sge[0].addr = ib_dma_map_single(info->id->device,
+						 (void *)packet,
+						 header_length,
+						 DMA_BIDIRECTIONAL);
+	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
+		mempool_free(request, info->request_mempool);
+		rc = -EIO;
+		goto err;
+	}
+
+	request->sge[0].length = header_length;
+	request->sge[0].lkey = info->pd->local_dma_lkey;
+
+	*request_out = request;
+	return 0;
+
+err:
+	atomic_inc(&info->send_credits);
+	return rc;
+}
+
+static void smbd_destroy_header(struct smbd_connection *info,
+		struct smbd_request *request)
+{
+
+	ib_dma_unmap_single(info->id->device,
+			    request->sge[0].addr,
+			    request->sge[0].length,
+			    DMA_TO_DEVICE);
+	mempool_free(request, info->request_mempool);
+	atomic_inc(&info->send_credits);
+}
+
+/* Post the send request */
+static int smbd_post_send(struct smbd_connection *info,
+		struct smbd_request *request, bool has_payload)
+{
+	struct ib_send_wr send_wr, *send_wr_fail;
+	int rc, i;
+
+	for (i = 0; i < request->num_sge; i++) {
+		log_rdma_send(INFO,
+			"rdma_request sge[%d] addr=%llu legnth=%u\n",
+			i, request->sge[0].addr, request->sge[0].length);
+		ib_dma_sync_single_for_device(
+			info->id->device,
+			request->sge[i].addr,
+			request->sge[i].length,
+			DMA_TO_DEVICE);
+	}
+
+	request->cqe.done = send_done;
+
+	send_wr.next = NULL;
+	send_wr.wr_cqe = &request->cqe;
+	send_wr.sg_list = request->sge;
+	send_wr.num_sge = request->num_sge;
+	send_wr.opcode = IB_WR_SEND;
+	send_wr.send_flags = IB_SEND_SIGNALED;
+
+	if (has_payload) {
+		request->has_payload = true;
+		atomic_inc(&info->send_payload_pending);
+	} else {
+		request->has_payload = false;
+		atomic_inc(&info->send_pending);
+	}
+
+	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
+	if (rc) {
+		log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
+		if (has_payload) {
+			if (atomic_dec_and_test(&info->send_payload_pending))
+				wake_up(&info->wait_send_payload_pending);
+		} else {
+			if (atomic_dec_and_test(&info->send_pending))
+				wake_up(&info->wait_send_pending);
+		}
+	} else
+		/* Reset timer for idle connection after packet is sent */
+		mod_delayed_work(info->workqueue, &info->idle_timer_work,
+			info->keep_alive_interval*HZ);
+
+	return rc;
+}
+
+static int smbd_post_send_sgl(struct smbd_connection *info,
+	struct scatterlist *sgl, int data_length, int remaining_data_length)
+{
+	int num_sgs;
+	int i, rc;
+	struct smbd_request *request;
+	struct scatterlist *sg;
+
+	rc = smbd_create_header(
+		info, data_length, remaining_data_length, &request);
+	if (rc)
+		return rc;
+
+	num_sgs = sgl ? sg_nents(sgl) : 0;
+	for_each_sg(sgl, sg, num_sgs, i) {
+		request->sge[i+1].addr =
+			ib_dma_map_page(info->id->device, sg_page(sg),
+			       sg->offset, sg->length, DMA_BIDIRECTIONAL);
+		if (ib_dma_mapping_error(
+				info->id->device, request->sge[i+1].addr)) {
+			rc = -EIO;
+			request->sge[i+1].addr = 0;
+			goto dma_mapping_failure;
+		}
+		request->sge[i+1].length = sg->length;
+		request->sge[i+1].lkey = info->pd->local_dma_lkey;
+		request->num_sge++;
+	}
+
+	rc = smbd_post_send(info, request, data_length);
+	if (!rc)
+		return 0;
+
+dma_mapping_failure:
+	for (i = 1; i < request->num_sge; i++)
+		if (request->sge[i].addr)
+			ib_dma_unmap_single(info->id->device,
+					    request->sge[i].addr,
+					    request->sge[i].length,
+					    DMA_TO_DEVICE);
+	smbd_destroy_header(info, request);
+	return rc;
+}
+
+/*
+ * Send an empty message
+ * Empty message is used to extend credits to peer to for keep live
+ * while there is no upper layer payload to send at the time
+ */
+static int smbd_post_send_empty(struct smbd_connection *info)
+{
+	info->count_send_empty++;
+	return smbd_post_send_sgl(info, NULL, 0, 0);
+}
+
+/*
+ * Post a receive request to the transport
+ * The remote peer can only send data when a receive request is posted
+ * The interaction is controlled by send/receive credit system
+ */
+static int smbd_post_recv(
+		struct smbd_connection *info, struct smbd_response *response)
+{
+	struct ib_recv_wr recv_wr, *recv_wr_fail = NULL;
+	int rc = -EIO;
+
+	response->sge.addr = ib_dma_map_single(
+				info->id->device, response->packet,
+				info->max_receive_size, DMA_FROM_DEVICE);
+	if (ib_dma_mapping_error(info->id->device, response->sge.addr))
+		return rc;
+
+	response->sge.length = info->max_receive_size;
+	response->sge.lkey = info->pd->local_dma_lkey;
+
+	response->cqe.done = recv_done;
+
+	recv_wr.wr_cqe = &response->cqe;
+	recv_wr.next = NULL;
+	recv_wr.sg_list = &response->sge;
+	recv_wr.num_sge = 1;
+
+	rc = ib_post_recv(info->id->qp, &recv_wr, &recv_wr_fail);
+	if (rc) {
+		ib_dma_unmap_single(info->id->device, response->sge.addr,
+				    response->sge.length, DMA_FROM_DEVICE);
+
+		log_rdma_recv(ERR, "ib_post_recv failed rc=%d\n", rc);
+	}
+
+	return rc;
+}
+
+/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */
+static int smbd_negotiate(struct smbd_connection *info)
+{
+	int rc;
+	struct smbd_response *response = get_receive_buffer(info);
+
+	response->type = SMBD_NEGOTIATE_RESP;
+	rc = smbd_post_recv(info, response);
+	log_rdma_event(INFO,
+		"smbd_post_recv rc=%d iov.addr=%llx iov.length=%x "
+		"iov.lkey=%x\n",
+		rc, response->sge.addr,
+		response->sge.length, response->sge.lkey);
+	if (rc)
+		return rc;
+
+	init_completion(&info->negotiate_completion);
+	info->negotiate_done = false;
+	rc = smbd_post_send_negotiate_req(info);
+	if (rc)
+		return rc;
+
+	rc = wait_for_completion_interruptible_timeout(
+		&info->negotiate_completion, SMBD_NEGOTIATE_TIMEOUT * HZ);
+	log_rdma_event(INFO, "wait_for_completion_timeout rc=%d\n", rc);
+
+	if (info->negotiate_done)
+		return 0;
+
+	if (rc == 0)
+		rc = -ETIMEDOUT;
+	else if (rc == -ERESTARTSYS)
+		rc = -EINTR;
+	else
+		rc = -ENOTCONN;
+
+	return rc;
+}
+
+static void put_empty_packet(
+		struct smbd_connection *info, struct smbd_response *response)
+{
+	spin_lock(&info->empty_packet_queue_lock);
+	list_add_tail(&response->list, &info->empty_packet_queue);
+	info->count_empty_packet_queue++;
+	spin_unlock(&info->empty_packet_queue_lock);
+
+	queue_work(info->workqueue, &info->post_send_credits_work);
+}
+
+/*
+ * Implement Connection.FragmentReassemblyBuffer defined in [MS-SMBD] 3.1.1.1
+ * This is a queue for reassembling upper layer payload and present to upper
+ * layer. All the inncoming payload go to the reassembly queue, regardless of
+ * if reassembly is required. The uuper layer code reads from the queue for all
+ * incoming payloads.
+ * Put a received packet to the reassembly queue
+ * response: the packet received
+ * data_length: the size of payload in this packet
+ */
+static void enqueue_reassembly(
+	struct smbd_connection *info,
+	struct smbd_response *response,
+	int data_length)
+{
+	spin_lock(&info->reassembly_queue_lock);
+	list_add_tail(&response->list, &info->reassembly_queue);
+	info->reassembly_queue_length++;
+	/*
+	 * Make sure reassembly_data_length is updated after list and
+	 * reassembly_queue_length are updated. On the dequeue side
+	 * reassembly_data_length is checked without a lock to determine
+	 * if reassembly_queue_length and list is up to date
+	 */
+	virt_wmb();
+	info->reassembly_data_length += data_length;
+	spin_unlock(&info->reassembly_queue_lock);
+	info->count_reassembly_queue++;
+	info->count_enqueue_reassembly_queue++;
+}
+
+/*
+ * Get the first entry at the front of reassembly queue
+ * Caller is responsible for locking
+ * return value: the first entry if any, NULL if queue is empty
+ */
+static struct smbd_response *_get_first_reassembly(struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+
+	if (!list_empty(&info->reassembly_queue)) {
+		ret = list_first_entry(
+			&info->reassembly_queue,
+			struct smbd_response, list);
+	}
+	return ret;
+}
+
+static struct smbd_response *get_empty_queue_buffer(
+		struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->empty_packet_queue_lock, flags);
+	if (!list_empty(&info->empty_packet_queue)) {
+		ret = list_first_entry(
+			&info->empty_packet_queue,
+			struct smbd_response, list);
+		list_del(&ret->list);
+		info->count_empty_packet_queue--;
+	}
+	spin_unlock_irqrestore(&info->empty_packet_queue_lock, flags);
+
+	return ret;
+}
+
+/*
+ * Get a receive buffer
+ * For each remote send, we need to post a receive. The receive buffers are
+ * pre-allocated in advance.
+ * return value: the receive buffer, NULL if none is available
+ */
+static struct smbd_response *get_receive_buffer(struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->receive_queue_lock, flags);
+	if (!list_empty(&info->receive_queue)) {
+		ret = list_first_entry(
+			&info->receive_queue,
+			struct smbd_response, list);
+		list_del(&ret->list);
+		info->count_receive_queue--;
+		info->count_get_receive_buffer++;
+	}
+	spin_unlock_irqrestore(&info->receive_queue_lock, flags);
+
+	return ret;
+}
+
+/*
+ * Return a receive buffer
+ * Upon returning of a receive buffer, we can post new receive and extend
+ * more receive credits to remote peer. This is done immediately after a
+ * receive buffer is returned.
+ */
+static void put_receive_buffer(
+	struct smbd_connection *info, struct smbd_response *response,
+	bool lock)
+{
+	unsigned long flags;
+
+	ib_dma_unmap_single(info->id->device, response->sge.addr,
+		response->sge.length, DMA_FROM_DEVICE);
+
+	if (lock)
+		spin_lock_irqsave(&info->receive_queue_lock, flags);
+	list_add_tail(&response->list, &info->receive_queue);
+	info->count_receive_queue++;
+	info->count_put_receive_buffer++;
+	if (lock)
+		spin_unlock_irqrestore(&info->receive_queue_lock, flags);
+
+	queue_work(info->workqueue, &info->post_send_credits_work);
+}
+
+/* Preallocate all receive buffer on transport establishment */
+static int allocate_receive_buffers(struct smbd_connection *info, int num_buf)
+{
+	int i;
+	struct smbd_response *response;
+
+	INIT_LIST_HEAD(&info->reassembly_queue);
+	spin_lock_init(&info->reassembly_queue_lock);
+	info->reassembly_data_length = 0;
+	info->reassembly_queue_length = 0;
+
+	INIT_LIST_HEAD(&info->receive_queue);
+	spin_lock_init(&info->receive_queue_lock);
+	info->count_receive_queue = 0;
+
+	INIT_LIST_HEAD(&info->empty_packet_queue);
+	spin_lock_init(&info->empty_packet_queue_lock);
+	info->count_empty_packet_queue = 0;
+
+	init_waitqueue_head(&info->wait_receive_queues);
+
+	for (i = 0; i < num_buf; i++) {
+		response = mempool_alloc(info->response_mempool, GFP_KERNEL);
+		if (!response)
+			goto allocate_failed;
+
+		response->info = info;
+		list_add_tail(&response->list, &info->receive_queue);
+		info->count_receive_queue++;
+	}
+
+	return 0;
+
+allocate_failed:
+	while (!list_empty(&info->receive_queue)) {
+		response = list_first_entry(
+				&info->receive_queue,
+				struct smbd_response, list);
+		list_del(&response->list);
+		info->count_receive_queue--;
+
+		mempool_free(response, info->response_mempool);
+	}
+	return -ENOMEM;
+}
+
+static void destroy_receive_buffers(struct smbd_connection *info)
+{
+	struct smbd_response *response;
+
+	while ((response = get_receive_buffer(info)))
+		mempool_free(response, info->response_mempool);
+
+	while ((response = get_empty_queue_buffer(info)))
+		mempool_free(response, info->response_mempool);
+}
+
+/*
+ * Check and send an immediate or keep alive packet
+ * The condition to send those packets are defined in [MS-SMBD] 3.1.1.1
+ * Connection.KeepaliveRequested and Connection.SendImmediate
+ * The idea is to extend credits to server as soon as it becomes available
+ */
+static void send_immediate_work(struct work_struct *work)
+{
+	struct smbd_connection *info = container_of(
+					work, struct smbd_connection,
+					send_immediate_work.work);
+
+	if (info->keep_alive_requested == KEEP_ALIVE_PENDING ||
+	    info->send_immediate) {
+		log_keep_alive(INFO, "send an empty message\n");
+		smbd_post_send_empty(info);
+	}
+}
+
+/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */
+static void idle_connection_timer(struct work_struct *work)
+{
+	struct smbd_connection *info = container_of(
+					work, struct smbd_connection,
+					idle_timer_work.work);
+
+	if (info->keep_alive_requested != KEEP_ALIVE_NONE) {
+		log_keep_alive(ERR,
+			"error status info->keep_alive_requested=%d\n",
+			info->keep_alive_requested);
+		smbd_disconnect_rdma_connection(info);
+		return;
+	}
+
+	log_keep_alive(INFO, "about to send an empty idle message\n");
+	smbd_post_send_empty(info);
+
+	/* Setup the next idle timeout work */
+	queue_delayed_work(info->workqueue, &info->idle_timer_work,
+			info->keep_alive_interval*HZ);
+}
+
+static void destroy_caches_and_workqueue(struct smbd_connection *info)
+{
+	destroy_receive_buffers(info);
+	destroy_workqueue(info->workqueue);
+	mempool_destroy(info->response_mempool);
+	kmem_cache_destroy(info->response_cache);
+	mempool_destroy(info->request_mempool);
+	kmem_cache_destroy(info->request_cache);
+}
+
+#define MAX_NAME_LEN	80
+static int allocate_caches_and_workqueue(struct smbd_connection *info)
+{
+	char name[MAX_NAME_LEN];
+	int rc;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_request_%p", info);
+	info->request_cache =
+		kmem_cache_create(
+			name,
+			sizeof(struct smbd_request) +
+				sizeof(struct smbd_data_transfer),
+			0, SLAB_HWCACHE_ALIGN, NULL);
+	if (!info->request_cache)
+		return -ENOMEM;
+
+	info->request_mempool =
+		mempool_create(info->send_credit_target, mempool_alloc_slab,
+			mempool_free_slab, info->request_cache);
+	if (!info->request_mempool)
+		goto out1;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_response_%p", info);
+	info->response_cache =
+		kmem_cache_create(
+			name,
+			sizeof(struct smbd_response) +
+				info->max_receive_size,
+			0, SLAB_HWCACHE_ALIGN, NULL);
+	if (!info->response_cache)
+		goto out2;
+
+	info->response_mempool =
+		mempool_create(info->receive_credit_max, mempool_alloc_slab,
+		       mempool_free_slab, info->response_cache);
+	if (!info->response_mempool)
+		goto out3;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_%p", info);
+	info->workqueue = create_workqueue(name);
+	if (!info->workqueue)
+		goto out4;
+
+	rc = allocate_receive_buffers(info, info->receive_credit_max);
+	if (rc) {
+		log_rdma_event(ERR, "failed to allocate receive buffers\n");
+		goto out5;
+	}
+
+	return 0;
+
+out5:
+	destroy_workqueue(info->workqueue);
+out4:
+	mempool_destroy(info->response_mempool);
+out3:
+	kmem_cache_destroy(info->response_cache);
+out2:
+	mempool_destroy(info->request_mempool);
+out1:
+	kmem_cache_destroy(info->request_cache);
+	return -ENOMEM;
+}
+
+/* Create a SMBD connection, called by upper layer */
+struct smbd_connection *_smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
+{
+	int rc;
+	struct smbd_connection *info;
+	struct rdma_conn_param conn_param;
+	struct ib_qp_init_attr qp_attr;
+	struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;
+
+	info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL);
+	if (!info)
+		return NULL;
+
+	info->transport_status = SMBD_CONNECTING;
+	rc = smbd_ia_open(info, dstaddr, port);
+	if (rc) {
+		log_rdma_event(INFO, "smbd_ia_open rc=%d\n", rc);
+		goto create_id_failed;
+	}
+
+	if (smbd_send_credit_target > info->id->device->attrs.max_cqe ||
+	    smbd_send_credit_target > info->id->device->attrs.max_qp_wr) {
+		log_rdma_event(ERR,
+			"consider lowering send_credit_target = %d. "
+			"Possible CQE overrun, device "
+			"reporting max_cpe %d max_qp_wr %d\n",
+			smbd_send_credit_target,
+			info->id->device->attrs.max_cqe,
+			info->id->device->attrs.max_qp_wr);
+		goto config_failed;
+	}
+
+	if (smbd_receive_credit_max > info->id->device->attrs.max_cqe ||
+	    smbd_receive_credit_max > info->id->device->attrs.max_qp_wr) {
+		log_rdma_event(ERR,
+			"consider lowering receive_credit_max = %d. "
+			"Possible CQE overrun, device "
+			"reporting max_cpe %d max_qp_wr %d\n",
+			smbd_receive_credit_max,
+			info->id->device->attrs.max_cqe,
+			info->id->device->attrs.max_qp_wr);
+		goto config_failed;
+	}
+
+	info->receive_credit_max = smbd_receive_credit_max;
+	info->send_credit_target = smbd_send_credit_target;
+	info->max_send_size = smbd_max_send_size;
+	info->max_fragmented_recv_size = smbd_max_fragmented_recv_size;
+	info->max_receive_size = smbd_max_receive_size;
+	info->keep_alive_interval = smbd_keep_alive_interval;
+
+	if (SMBDIRECT_MAX_SGE > info->id->device->attrs.max_sge) {
+		log_rdma_event(ERR, "warning: device max_sge = %d too small\n",
+			info->id->device->attrs.max_sge);
+		log_rdma_event(ERR, "Queue Pair creation may fail\n");
+	}
+
+	info->send_cq = NULL;
+	info->recv_cq = NULL;
+	info->send_cq = ib_alloc_cq(info->id->device, info,
+			info->send_credit_target, 0, IB_POLL_SOFTIRQ);
+	if (IS_ERR(info->send_cq)) {
+		info->send_cq = NULL;
+		goto alloc_cq_failed;
+	}
+
+	info->recv_cq = ib_alloc_cq(info->id->device, info,
+			info->receive_credit_max, 0, IB_POLL_SOFTIRQ);
+	if (IS_ERR(info->recv_cq)) {
+		info->recv_cq = NULL;
+		goto alloc_cq_failed;
+	}
+
+	memset(&qp_attr, 0, sizeof(qp_attr));
+	qp_attr.event_handler = smbd_qp_async_error_upcall;
+	qp_attr.qp_context = info;
+	qp_attr.cap.max_send_wr = info->send_credit_target;
+	qp_attr.cap.max_recv_wr = info->receive_credit_max;
+	qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
+	qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
+	qp_attr.cap.max_inline_data = 0;
+	qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
+	qp_attr.qp_type = IB_QPT_RC;
+	qp_attr.send_cq = info->send_cq;
+	qp_attr.recv_cq = info->recv_cq;
+	qp_attr.port_num = ~0;
+
+	rc = rdma_create_qp(info->id, info->pd, &qp_attr);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_create_qp failed %i\n", rc);
+		goto create_qp_failed;
+	}
+
+	memset(&conn_param, 0, sizeof(conn_param));
+	conn_param.initiator_depth = 0;
+
+	conn_param.retry_count = SMBD_CM_RETRY;
+	conn_param.rnr_retry_count = SMBD_CM_RNR_RETRY;
+	conn_param.flow_control = 0;
+	init_waitqueue_head(&info->wait_destroy);
+
+	log_rdma_event(INFO, "connecting to IP %pI4 port %d\n",
+		&addr_in->sin_addr, port);
+
+	init_waitqueue_head(&info->conn_wait);
+	rc = rdma_connect(info->id, &conn_param);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_connect() failed with %i\n", rc);
+		goto rdma_connect_failed;
+	}
+
+	wait_event_interruptible(
+		info->conn_wait, info->transport_status != SMBD_CONNECTING);
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		log_rdma_event(ERR, "rdma_connect failed port=%d\n", port);
+		goto rdma_connect_failed;
+	}
+
+	log_rdma_event(INFO, "rdma_connect connected\n");
+
+	rc = allocate_caches_and_workqueue(info);
+	if (rc) {
+		log_rdma_event(ERR, "cache allocation failed\n");
+		goto allocate_cache_failed;
+	}
+
+	init_waitqueue_head(&info->wait_send_queue);
+	init_waitqueue_head(&info->wait_reassembly_queue);
+
+	INIT_DELAYED_WORK(&info->idle_timer_work, idle_connection_timer);
+	INIT_DELAYED_WORK(&info->send_immediate_work, send_immediate_work);
+	queue_delayed_work(info->workqueue, &info->idle_timer_work,
+		info->keep_alive_interval*HZ);
+
+	init_waitqueue_head(&info->wait_send_pending);
+	atomic_set(&info->send_pending, 0);
+
+	init_waitqueue_head(&info->wait_send_payload_pending);
+	atomic_set(&info->send_payload_pending, 0);
+
+	INIT_WORK(&info->disconnect_work, smbd_disconnect_rdma_work);
+	INIT_WORK(&info->destroy_work, smbd_destroy_rdma_work);
+	INIT_WORK(&info->recv_done_work, smbd_recv_done_work);
+	INIT_WORK(&info->post_send_credits_work, smbd_post_send_credits);
+	info->new_credits_offered = 0;
+	spin_lock_init(&info->lock_new_credits_offered);
+
+	rc = smbd_negotiate(info);
+	if (rc) {
+		log_rdma_event(ERR, "smbd_negotiate rc=%d\n", rc);
+		goto negotiation_failed;
+	}
+
+	return info;
+
+negotiation_failed:
+	cancel_delayed_work_sync(&info->idle_timer_work);
+	destroy_caches_and_workqueue(info);
+	info->transport_status = SMBD_NEGOTIATE_FAILED;
+	init_waitqueue_head(&info->conn_wait);
+	rdma_disconnect(info->id);
+	wait_event(info->conn_wait,
+		info->transport_status == SMBD_DISCONNECTED);
+
+allocate_cache_failed:
+rdma_connect_failed:
+	rdma_destroy_qp(info->id);
+
+create_qp_failed:
+alloc_cq_failed:
+	if (info->send_cq)
+		ib_free_cq(info->send_cq);
+	if (info->recv_cq)
+		ib_free_cq(info->recv_cq);
+
+config_failed:
+	ib_dealloc_pd(info->pd);
+	rdma_destroy_id(info->id);
+
+create_id_failed:
+	kfree(info);
+	return NULL;
+}
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
index c55f28b..35bc25b 100644
--- a/fs/cifs/smbdirect.h
+++ b/fs/cifs/smbdirect.h
@@ -16,6 +16,286 @@
 #ifndef _SMBDIRECT_H
 #define _SMBDIRECT_H
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#define cifs_rdma_enabled(server)	((server)->rdma)
+
+#include "cifsglob.h"
+#include <rdma/ib_verbs.h>
+#include <rdma/rdma_cm.h>
+#include <linux/mempool.h>
+
+enum keep_alive_status {
+	KEEP_ALIVE_NONE,
+	KEEP_ALIVE_PENDING,
+	KEEP_ALIVE_SENT,
+};
+
+enum smbd_connection_status {
+	SMBD_CREATED,
+	SMBD_CONNECTING,
+	SMBD_CONNECTED,
+	SMBD_NEGOTIATE_FAILED,
+	SMBD_DISCONNECTING,
+	SMBD_DISCONNECTED,
+	SMBD_DESTROYED
+};
+
+/*
+ * The context for the SMBDirect transport
+ * Everything related to the transport is here. It has several logical parts
+ * 1. RDMA related structures
+ * 2. SMBDirect connection parameters
+ * 3. Memory registrations
+ * 4. Receive and reassembly queues for data receive path
+ * 5. mempools for allocating packets
+ */
+struct smbd_connection {
+	enum smbd_connection_status transport_status;
+
+	/* RDMA related */
+	struct rdma_cm_id *id;
+	struct ib_qp_init_attr qp_attr;
+	struct ib_pd *pd;
+	struct ib_cq *send_cq, *recv_cq;
+	struct ib_device_attr dev_attr;
+	int ri_rc;
+	struct completion ri_done;
+	wait_queue_head_t conn_wait;
+	wait_queue_head_t wait_destroy;
+
+	struct completion negotiate_completion;
+	bool negotiate_done;
+
+	struct work_struct destroy_work;
+	struct work_struct disconnect_work;
+	struct work_struct recv_done_work;
+	struct work_struct post_send_credits_work;
+
+	spinlock_t lock_new_credits_offered;
+	int new_credits_offered;
+
+	/* Connection parameters defined in [MS-SMBD] 3.1.1.1 */
+	int receive_credit_max;
+	int send_credit_target;
+	int max_send_size;
+	int max_fragmented_recv_size;
+	int max_fragmented_send_size;
+	int max_receive_size;
+	int keep_alive_interval;
+	int max_readwrite_size;
+	enum keep_alive_status keep_alive_requested;
+	int protocol;
+	atomic_t send_credits;
+	atomic_t receive_credits;
+	int receive_credit_target;
+	int fragment_reassembly_remaining;
+
+	/* Activity accoutning */
+
+	atomic_t send_pending;
+	wait_queue_head_t wait_send_pending;
+	atomic_t send_payload_pending;
+	wait_queue_head_t wait_send_payload_pending;
+
+	/* Receive queue */
+	struct list_head receive_queue;
+	int count_receive_queue;
+	spinlock_t receive_queue_lock;
+
+	struct list_head empty_packet_queue;
+	int count_empty_packet_queue;
+	spinlock_t empty_packet_queue_lock;
+
+	wait_queue_head_t wait_receive_queues;
+
+	/* Reassembly queue */
+	struct list_head reassembly_queue;
+	spinlock_t reassembly_queue_lock;
+	wait_queue_head_t wait_reassembly_queue;
+
+	/* total data length of reassembly queue */
+	int reassembly_data_length;
+	int reassembly_queue_length;
+	/* the offset to first buffer in reassembly queue */
+	int first_entry_offset;
+
+	bool send_immediate;
+
+	wait_queue_head_t wait_send_queue;
+
+	/*
+	 * Indicate if we have received a full packet on the connection
+	 * This is used to identify the first SMBD packet of a assembled
+	 * payload (SMB packet) in reassembly queue so we can return a
+	 * RFC1002 length to upper layer to indicate the length of the SMB
+	 * packet received
+	 */
+	bool full_packet_received;
+
+	struct workqueue_struct *workqueue;
+	struct delayed_work idle_timer_work;
+	struct delayed_work send_immediate_work;
+
+	/* Memory pool for preallocating buffers */
+	/* request pool for RDMA send */
+	struct kmem_cache *request_cache;
+	mempool_t *request_mempool;
+
+	/* response pool for RDMA receive */
+	struct kmem_cache *response_cache;
+	mempool_t *response_mempool;
+
+	/* for debug purposes */
+	unsigned int count_get_receive_buffer;
+	unsigned int count_put_receive_buffer;
+	unsigned int count_reassembly_queue;
+	unsigned int count_enqueue_reassembly_queue;
+	unsigned int count_dequeue_reassembly_queue;
+	unsigned int count_send_empty;
+};
+
+enum smbd_message_type {
+	SMBD_NEGOTIATE_RESP,
+	SMBD_TRANSFER_DATA,
+};
+
+#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
+
+/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */
+struct smbd_negotiate_req {
+	__le16 min_version;
+	__le16 max_version;
+	__le16 reserved;
+	__le16 credits_requested;
+	__le32 preferred_send_size;
+	__le32 max_receive_size;
+	__le32 max_fragmented_size;
+} __packed;
+
+/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */
+struct smbd_negotiate_resp {
+	__le16 min_version;
+	__le16 max_version;
+	__le16 negotiated_version;
+	__le16 reserved;
+	__le16 credits_requested;
+	__le16 credits_granted;
+	__le32 status;
+	__le32 max_readwrite_size;
+	__le32 preferred_send_size;
+	__le32 max_receive_size;
+	__le32 max_fragmented_size;
+} __packed;
+
+/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */
+struct smbd_data_transfer {
+	__le16 credits_requested;
+	__le16 credits_granted;
+	__le16 flags;
+	__le16 reserved;
+	__le32 remaining_data_length;
+	__le32 data_offset;
+	__le32 data_length;
+	__le32 padding;
+	__u8 buffer[];
+} __packed;
+
+/* The packet fields for a registered RDMA buffer */
+struct smbd_buffer_descriptor_v1 {
+	__le64 offset;
+	__le32 token;
+	__le32 length;
+} __packed;
+
 /* Default maximum number of SGEs in a RDMA send/recv */
 #define SMBDIRECT_MAX_SGE	16
+/* The context for a SMBD request */
+struct smbd_request {
+	struct smbd_connection *info;
+	struct ib_cqe cqe;
+
+	/* true if this request carries upper layer payload */
+	bool has_payload;
+
+	/* the SGE entries for this packet */
+	struct ib_sge sge[SMBDIRECT_MAX_SGE];
+	int num_sge;
+
+	/* SMBD packet header follows this structure */
+	u8 packet[];
+};
+
+/* The context for a SMBD response */
+struct smbd_response {
+	struct smbd_connection *info;
+	struct ib_cqe cqe;
+	struct ib_sge sge;
+
+	enum smbd_message_type type;
+
+	/* Link to receive queue or reassembly queue */
+	struct list_head list;
+
+	/* Indicate if this is the 1st packet of a payload */
+	bool first_segment;
+
+	/* SMBD packet header and payload follows this structure */
+	u8 packet[];
+};
+
+/* Create a SMBDirect session */
+struct smbd_connection *smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr);
+
+/* Reconnect SMBDirect session */
+int smbd_reconnect(struct TCP_Server_Info *server);
+
+/* Destroy SMBDirect session */
+void smbd_destroy(struct smbd_connection *info);
+
+/* Interface for carrying upper layer I/O through send/recv */
+int smbd_recv(struct smbd_connection *info, struct msghdr *msg);
+int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst);
+
+enum mr_state {
+	MR_READY,
+	MR_REGISTERED,
+	MR_INVALIDATED,
+	MR_ERROR
+};
+
+struct smbd_mr {
+	struct smbd_connection	*conn;
+	struct list_head	list;
+	enum mr_state		state;
+	struct ib_mr		*mr;
+	struct scatterlist	*sgl;
+	int			sgl_count;
+	enum dma_data_direction	dir;
+	union {
+		struct ib_reg_wr	wr;
+		struct ib_send_wr	inv_wr;
+	};
+	struct ib_cqe		cqe;
+	bool			need_invalidate;
+	struct completion	invalidate_done;
+};
+
+/* Interfaces to register and deregister MR for RDMA read/write */
+struct smbd_mr *smbd_register_mr(
+	struct smbd_connection *info, struct page *pages[], int num_pages,
+	int tailsz, bool writing, bool need_invalidate);
+int smbd_deregister_mr(struct smbd_mr *mr);
+
+#else
+#define cifs_rdma_enabled(server)	0
+struct smbd_connection{};
+static inline void *smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr) {return NULL;}
+static inline int smbd_reconnect(struct TCP_Server_Info *server) {return -1;}
+static inline void smbd_destroy(struct smbd_connection *info) {}
+static inline int smbd_recv(struct smbd_connection *info, struct msghdr *msg) {return -1;}
+static inline int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst) {return -1;}
+#endif
+
 #endif
-- 
2.7.4

Re: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

[ronnie sahlberg]

On Tue, Nov 7, 2017 at 6:54 PM, Long Li <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org> wrote:
> From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
>
> Add code to implement the core functions to establish a SMB Direct connection.
>
> 1. Establish an RDMA connection to SMB server.
> 2. Negotiate and setup SMB Direct protocol.
> 3. Implement idle connection timer and credit management.
>
> SMB Direct is enabled by setting CONFIG_CIFS_SMB_DIRECT.
>
> Add to Makefile to enable building SMB Direct.
>
> Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
> ---
>  fs/cifs/Makefile    |    2 +
>  fs/cifs/smbdirect.c | 1576 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  fs/cifs/smbdirect.h |  280 +++++++++
>  3 files changed, 1858 insertions(+)
>
> diff --git a/fs/cifs/Makefile b/fs/cifs/Makefile
> index 5e853a3..ad00873 100644
> --- a/fs/cifs/Makefile
> +++ b/fs/cifs/Makefile
> @@ -18,3 +18,5 @@ cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
>  cifs-$(CONFIG_CIFS_DFS_UPCALL) += dns_resolve.o cifs_dfs_ref.o
>
>  cifs-$(CONFIG_CIFS_FSCACHE) += fscache.o cache.o
> +
> +cifs-$(CONFIG_CIFS_SMB_DIRECT) += smbdirect.o
> diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
> index d3c16f8..021d527 100644
> --- a/fs/cifs/smbdirect.c
> +++ b/fs/cifs/smbdirect.c
> @@ -13,7 +13,34 @@
>   *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
>   *   the GNU General Public License for more details.
>   */
> +#include <linux/module.h>
>  #include "smbdirect.h"
> +#include "cifs_debug.h"
> +
> +static struct smbd_response *get_empty_queue_buffer(
> +               struct smbd_connection *info);
> +static struct smbd_response *get_receive_buffer(
> +               struct smbd_connection *info);
> +static void put_receive_buffer(
> +               struct smbd_connection *info,
> +               struct smbd_response *response,
> +               bool lock);
> +static int allocate_receive_buffers(struct smbd_connection *info, int num_buf);
> +static void destroy_receive_buffers(struct smbd_connection *info);
> +
> +static void put_empty_packet(
> +               struct smbd_connection *info, struct smbd_response *response);
> +static void enqueue_reassembly(
> +               struct smbd_connection *info,
> +               struct smbd_response *response, int data_length);
> +static struct smbd_response *_get_first_reassembly(
> +               struct smbd_connection *info);
> +
> +static int smbd_post_recv(
> +               struct smbd_connection *info,
> +               struct smbd_response *response);
> +
> +static int smbd_post_send_empty(struct smbd_connection *info);
>
>  /* SMBD version number */
>  #define SMBD_V1        0x0100
> @@ -75,3 +102,1552 @@ int smbd_max_frmr_depth = 2048;
>
>  /* If payload is less than this byte, use RDMA send/recv not read/write */
>  int rdma_readwrite_threshold = 4096;
> +
> +/* Transport logging functions
> + * Logging are defined as classes. They can be OR'ed to define the actual
> + * logging level via module parameter smbd_logging_class
> + * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
> + * log_rdma_event()

Should be 0xa0 ?

> + */
> +#define LOG_OUTGOING                   0x1
> +#define LOG_INCOMING                   0x2
> +#define LOG_READ                       0x4
> +#define LOG_WRITE                      0x8
> +#define LOG_RDMA_SEND                  0x10
> +#define LOG_RDMA_RECV                  0x20
> +#define LOG_KEEP_ALIVE                 0x40
> +#define LOG_RDMA_EVENT                 0x80
> +#define LOG_RDMA_MR                    0x100
> +static unsigned int smbd_logging_class = 0;
> +module_param(smbd_logging_class, uint, 0644);
> +MODULE_PARM_DESC(smbd_logging_class,
> +       "Logging class for SMBD transport 0x0 to 0x100");
> +
> +#define ERR            0x0
> +#define INFO           0x1
> +static unsigned int smbd_logging_level = ERR;
> +module_param(smbd_logging_level, uint, 0644);
> +MODULE_PARM_DESC(smbd_logging_level,
> +       "Logging level for SMBD transport, 0 (default): error, 1: info");
> +
> +#define log_rdma(level, class, fmt, args...)                           \
> +do {                                                                   \
> +       if (level <= smbd_logging_level || class & smbd_logging_class)  \
> +               cifs_dbg(VFS, "%s:%d " fmt, __func__, __LINE__, ##args);\
> +} while (0)
> +
> +#define log_outgoing(level, fmt, args...) \
> +               log_rdma(level, LOG_OUTGOING, fmt, ##args)
> +#define log_incoming(level, fmt, args...) \
> +               log_rdma(level, LOG_INCOMING, fmt, ##args)
> +#define log_read(level, fmt, args...)  log_rdma(level, LOG_READ, fmt, ##args)
> +#define log_write(level, fmt, args...) log_rdma(level, LOG_WRITE, fmt, ##args)
> +#define log_rdma_send(level, fmt, args...) \
> +               log_rdma(level, LOG_RDMA_SEND, fmt, ##args)
> +#define log_rdma_recv(level, fmt, args...) \
> +               log_rdma(level, LOG_RDMA_RECV, fmt, ##args)
> +#define log_keep_alive(level, fmt, args...) \
> +               log_rdma(level, LOG_KEEP_ALIVE, fmt, ##args)
> +#define log_rdma_event(level, fmt, args...) \
> +               log_rdma(level, LOG_RDMA_EVENT, fmt, ##args)
> +#define log_rdma_mr(level, fmt, args...) \
> +               log_rdma(level, LOG_RDMA_MR, fmt, ##args)
> +
> +/*
> + * Destroy the transport and related RDMA and memory resources
> + * Need to go through all the pending counters and make sure on one is using
> + * the transport while it is destroyed
> + */
> +static void smbd_destroy_rdma_work(struct work_struct *work)
> +{
> +       struct smbd_response *response;
> +       struct smbd_connection *info =
> +               container_of(work, struct smbd_connection, destroy_work);
> +       unsigned long flags;
> +
> +       log_rdma_event(INFO, "destroying qp\n");
> +       ib_drain_qp(info->id->qp);
> +       rdma_destroy_qp(info->id);
> +
> +       /* Unblock all I/O waiting on the send queue */
> +       wake_up_interruptible_all(&info->wait_send_queue);
> +
> +       log_rdma_event(INFO, "cancelling idle timer\n");
> +       cancel_delayed_work_sync(&info->idle_timer_work);
> +       log_rdma_event(INFO, "cancelling send immediate work\n");
> +       cancel_delayed_work_sync(&info->send_immediate_work);
> +
> +       log_rdma_event(INFO, "wait for all recv to finish\n");
> +       wake_up_interruptible(&info->wait_reassembly_queue);
> +
> +       log_rdma_event(INFO, "wait for all send posted to IB to finish\n");
> +       wait_event(info->wait_send_pending,
> +               atomic_read(&info->send_pending) == 0);
> +       wait_event(info->wait_send_payload_pending,
> +               atomic_read(&info->send_payload_pending) == 0);
> +
> +       /* It's not posssible for upper layer to get to reassembly */
> +       log_rdma_event(INFO, "drain the reassembly queue\n");
> +       do {
> +               spin_lock_irqsave(&info->reassembly_queue_lock, flags);
> +               response = _get_first_reassembly(info);
> +               if (response) {
> +                       list_del(&response->list);
> +                       spin_unlock_irqrestore(
> +                               &info->reassembly_queue_lock, flags);
> +                       put_receive_buffer(info, response, true);
> +               }
> +       } while (response);
> +       spin_unlock_irqrestore(&info->reassembly_queue_lock, flags);
> +       info->reassembly_data_length = 0;
> +
> +       log_rdma_event(INFO, "free receive buffers\n");
> +       wait_event(info->wait_receive_queues,
> +               info->count_receive_queue + info->count_empty_packet_queue
> +                       == info->receive_credit_max);
> +       destroy_receive_buffers(info);
> +
> +       ib_free_cq(info->send_cq);
> +       ib_free_cq(info->recv_cq);
> +       ib_dealloc_pd(info->pd);
> +       rdma_destroy_id(info->id);
> +
> +       /* free mempools */
> +       mempool_destroy(info->request_mempool);
> +       kmem_cache_destroy(info->request_cache);
> +
> +       mempool_destroy(info->response_mempool);
> +       kmem_cache_destroy(info->response_cache);
> +
> +       info->transport_status = SMBD_DESTROYED;
> +       wake_up_all(&info->wait_destroy);
> +}
> +
> +static int smbd_process_disconnected(struct smbd_connection *info)
> +{
> +//     queue_work(info->workqueue, &info->destroy_work);
> +       schedule_work(&info->destroy_work);
> +       return 0;
> +}
> +
> +static void smbd_disconnect_rdma_work(struct work_struct *work)
> +{
> +       struct smbd_connection *info =
> +               container_of(work, struct smbd_connection, disconnect_work);
> +
> +       if (info->transport_status == SMBD_CONNECTED) {
> +               info->transport_status = SMBD_DISCONNECTING;
> +               rdma_disconnect(info->id);
> +       }
> +}
> +
> +static void smbd_disconnect_rdma_connection(struct smbd_connection *info)
> +{
> +       queue_work(info->workqueue, &info->disconnect_work);
> +}
> +
> +/* Upcall from RDMA CM */
> +static int smbd_conn_upcall(
> +               struct rdma_cm_id *id, struct rdma_cm_event *event)
> +{
> +       struct smbd_connection *info = id->context;
> +
> +       log_rdma_event(INFO, "event=%d status=%d\n",
> +               event->event, event->status);
> +
> +       switch (event->event) {
> +       case RDMA_CM_EVENT_ADDR_RESOLVED:
> +       case RDMA_CM_EVENT_ROUTE_RESOLVED:
> +               info->ri_rc = 0;
> +               complete(&info->ri_done);
> +               break;
> +
> +       case RDMA_CM_EVENT_ADDR_ERROR:
> +               info->ri_rc = -EHOSTUNREACH;
> +               complete(&info->ri_done);
> +               break;
> +
> +       case RDMA_CM_EVENT_ROUTE_ERROR:
> +               info->ri_rc = -ENETUNREACH;
> +               complete(&info->ri_done);
> +               break;
> +
> +       case RDMA_CM_EVENT_ESTABLISHED:
> +               log_rdma_event(INFO, "connected event=%d\n", event->event);
> +               info->transport_status = SMBD_CONNECTED;
> +               wake_up_interruptible(&info->conn_wait);
> +               break;
> +
> +       case RDMA_CM_EVENT_CONNECT_ERROR:
> +       case RDMA_CM_EVENT_UNREACHABLE:
> +       case RDMA_CM_EVENT_REJECTED:
> +               log_rdma_event(INFO, "connecting failed event=%d\n", event->event);
> +               info->transport_status = SMBD_DISCONNECTED;
> +               wake_up_interruptible(&info->conn_wait);
> +               break;
> +
> +       case RDMA_CM_EVENT_DEVICE_REMOVAL:
> +       case RDMA_CM_EVENT_DISCONNECTED:
> +               /* This happenes when we fail the negotiation */
> +               if (info->transport_status == SMBD_NEGOTIATE_FAILED) {
> +                       info->transport_status = SMBD_DISCONNECTED;
> +                       wake_up(&info->conn_wait);
> +                       break;
> +               }
> +
> +               info->transport_status = SMBD_DISCONNECTED;
> +               smbd_process_disconnected(info);
> +               break;
> +
> +       default:
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
> +/* Upcall from RDMA QP */
> +static void
> +smbd_qp_async_error_upcall(struct ib_event *event, void *context)
> +{
> +       struct smbd_connection *info = context;
> +
> +       log_rdma_event(ERR, "%s on device %s info %p\n",
> +               ib_event_msg(event->event), event->device->name, info);
> +
> +       switch (event->event) {
> +       case IB_EVENT_CQ_ERR:
> +       case IB_EVENT_QP_FATAL:
> +               smbd_disconnect_rdma_connection(info);
> +
> +       default:
> +               break;
> +       }
> +}
> +
> +static inline void *smbd_request_payload(struct smbd_request *request)
> +{
> +       return (void *)request->packet;
> +}
> +
> +static inline void *smbd_response_payload(struct smbd_response *response)
> +{
> +       return (void *)response->packet;
> +}
> +
> +/* Called when a RDMA send is done */
> +static void send_done(struct ib_cq *cq, struct ib_wc *wc)
> +{
> +       int i;
> +       struct smbd_request *request =
> +               container_of(wc->wr_cqe, struct smbd_request, cqe);
> +
> +       log_rdma_send(INFO, "smbd_request %p completed wc->status=%d\n",
> +               request, wc->status);
> +
> +       if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_SEND) {
> +               log_rdma_send(ERR, "wc->status=%d wc->opcode=%d\n",
> +                       wc->status, wc->opcode);
> +               smbd_disconnect_rdma_connection(request->info);
> +       }
> +
> +       for (i = 0; i < request->num_sge; i++)
> +               ib_dma_unmap_single(request->info->id->device,
> +                       request->sge[i].addr,
> +                       request->sge[i].length,
> +                       DMA_TO_DEVICE);
> +
> +       if (request->has_payload) {
> +               if (atomic_dec_and_test(&request->info->send_payload_pending))
> +                       wake_up(&request->info->wait_send_payload_pending);
> +       } else {
> +               if (atomic_dec_and_test(&request->info->send_pending))
> +                       wake_up(&request->info->wait_send_pending);
> +       }
> +
> +       mempool_free(request, request->info->request_mempool);
> +}
> +
> +static void dump_smbd_negotiate_resp(struct smbd_negotiate_resp *resp)
> +{
> +       log_rdma_event(INFO, "resp message min_version %u max_version %u "
> +               "negotiated_version %u credits_requested %u "
> +               "credits_granted %u status %u max_readwrite_size %u "
> +               "preferred_send_size %u max_receive_size %u "
> +               "max_fragmented_size %u\n",
> +               resp->min_version, resp->max_version, resp->negotiated_version,
> +               resp->credits_requested, resp->credits_granted, resp->status,
> +               resp->max_readwrite_size, resp->preferred_send_size,
> +               resp->max_receive_size, resp->max_fragmented_size);
> +}
> +
> +/*
> + * Process a negotiation response message, according to [MS-SMBD]3.1.5.7
> + * response, packet_length: the negotiation response message
> + * return value: true if negotiation is a success, false if failed
> + */
> +static bool process_negotiation_response(
> +               struct smbd_response *response, int packet_length)
> +{
> +       struct smbd_connection *info = response->info;
> +       struct smbd_negotiate_resp *packet = smbd_response_payload(response);
> +
> +       if (packet_length < sizeof(struct smbd_negotiate_resp)) {
> +               log_rdma_event(ERR,
> +                       "error: packet_length=%d\n", packet_length);
> +               return false;
> +       }
> +
> +       if (le16_to_cpu(packet->negotiated_version) != SMBD_V1) {
> +               log_rdma_event(ERR, "error: negotiated_version=%x\n",
> +                       le16_to_cpu(packet->negotiated_version));
> +               return false;
> +       }
> +       info->protocol = le16_to_cpu(packet->negotiated_version);
> +
> +       if (packet->credits_requested == 0) {
> +               log_rdma_event(ERR, "error: credits_requested==0\n");
> +               return false;
> +       }
> +       info->receive_credit_target = le16_to_cpu(packet->credits_requested);
> +
> +       if (packet->credits_granted == 0) {
> +               log_rdma_event(ERR, "error: credits_granted==0\n");
> +               return false;
> +       }
> +       atomic_set(&info->send_credits, le16_to_cpu(packet->credits_granted));
> +
> +       atomic_set(&info->receive_credits, 0);
> +
> +       if (le32_to_cpu(packet->preferred_send_size) > info->max_receive_size) {
> +               log_rdma_event(ERR, "error: preferred_send_size=%d\n",
> +                       le32_to_cpu(packet->preferred_send_size));
> +               return false;
> +       }
> +       info->max_receive_size = le32_to_cpu(packet->preferred_send_size);
> +
> +       if (le32_to_cpu(packet->max_receive_size) < SMBD_MIN_RECEIVE_SIZE) {
> +               log_rdma_event(ERR, "error: max_receive_size=%d\n",
> +                       le32_to_cpu(packet->max_receive_size));
> +               return false;
> +       }
> +       info->max_send_size = min_t(int, info->max_send_size,
> +                                       le32_to_cpu(packet->max_receive_size));
> +
> +       if (le32_to_cpu(packet->max_fragmented_size) <
> +                       SMBD_MIN_FRAGMENTED_SIZE) {
> +               log_rdma_event(ERR, "error: max_fragmented_size=%d\n",
> +                       le32_to_cpu(packet->max_fragmented_size));
> +               return false;
> +       }
> +       info->max_fragmented_send_size =
> +               le32_to_cpu(packet->max_fragmented_size);
> +
> +       return true;
> +}
> +
> +/*
> + * Check and schedule to send an immediate packet
> + * This is used to extend credtis to remote peer to keep the transport busy
> + */
> +static void check_and_send_immediate(struct smbd_connection *info)
> +{
> +       if (info->transport_status != SMBD_CONNECTED)
> +               return;
> +
> +       info->send_immediate = true;
> +
> +       /*
> +        * Promptly send a packet if our peer is running low on receive
> +        * credits
> +        */
> +       if (atomic_read(&info->receive_credits) <
> +               info->receive_credit_target - 1)
> +               queue_delayed_work(
> +                       info->workqueue, &info->send_immediate_work, 0);
> +}
> +
> +static void smbd_post_send_credits(struct work_struct *work)
> +{
> +       int ret = 0;
> +       int use_receive_queue = 1;
> +       int rc;
> +       struct smbd_response *response;
> +       struct smbd_connection *info =
> +               container_of(work, struct smbd_connection,
> +                       post_send_credits_work);
> +
> +       if (info->transport_status != SMBD_CONNECTED) {
> +               wake_up(&info->wait_receive_queues);
> +               return;
> +       }
> +
> +       if (info->receive_credit_target >
> +               atomic_read(&info->receive_credits)) {
> +               while (true) {
> +                       if (use_receive_queue)
> +                               response = get_receive_buffer(info);
> +                       else
> +                               response = get_empty_queue_buffer(info);
> +                       if (!response) {
> +                               /* now switch to emtpy packet queue */
> +                               if (use_receive_queue) {
> +                                       use_receive_queue = 0;
> +                                       continue;
> +                               } else
> +                                       break;
> +                       }
> +
> +                       response->type = SMBD_TRANSFER_DATA;
> +                       response->first_segment = false;
> +                       rc = smbd_post_recv(info, response);
> +                       if (rc) {
> +                               log_rdma_recv(ERR,
> +                                       "post_recv failed rc=%d\n", rc);
> +                               put_receive_buffer(info, response, true);
> +                               break;
> +                       }
> +
> +                       ret++;
> +               }
> +       }
> +
> +       spin_lock(&info->lock_new_credits_offered);
> +       info->new_credits_offered += ret;
> +       spin_unlock(&info->lock_new_credits_offered);
> +
> +       atomic_add(ret, &info->receive_credits);
> +
> +       /* Check if we can post new receive and grant credits to peer */
> +       check_and_send_immediate(info);
> +}
> +
> +static void smbd_recv_done_work(struct work_struct *work)
> +{
> +       struct smbd_connection *info =
> +               container_of(work, struct smbd_connection, recv_done_work);
> +
> +       /*
> +        * We may have new send credits granted from remote peer
> +        * If any sender is blcoked on lack of credets, unblock it
> +        */
> +       if (atomic_read(&info->send_credits))
> +               wake_up_interruptible(&info->wait_send_queue);
> +
> +       /*
> +        * Check if we need to send something to remote peer to
> +        * grant more credits or respond to KEEP_ALIVE packet
> +        */
> +       check_and_send_immediate(info);
> +}
> +
> +/* Called from softirq, when recv is done */
> +static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> +{
> +       struct smbd_data_transfer *data_transfer;
> +       struct smbd_response *response =
> +               container_of(wc->wr_cqe, struct smbd_response, cqe);
> +       struct smbd_connection *info = response->info;
> +       int data_length = 0;
> +
> +       log_rdma_recv(INFO, "response=%p type=%d wc status=%d wc opcode %d "
> +                     "byte_len=%d pkey_index=%x\n",
> +               response, response->type, wc->status, wc->opcode,
> +               wc->byte_len, wc->pkey_index);
> +
> +       if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
> +               log_rdma_recv(INFO, "wc->status=%d opcode=%d\n",
> +                       wc->status, wc->opcode);
> +               smbd_disconnect_rdma_connection(info);
> +               goto error;
> +       }
> +
> +       ib_dma_sync_single_for_cpu(
> +               wc->qp->device,
> +               response->sge.addr,
> +               response->sge.length,
> +               DMA_FROM_DEVICE);
> +
> +       switch (response->type) {
> +       /* SMBD negotiation response */
> +       case SMBD_NEGOTIATE_RESP:
> +               dump_smbd_negotiate_resp(smbd_response_payload(response));
> +               info->full_packet_received = true;
> +               info->negotiate_done =
> +                       process_negotiation_response(response, wc->byte_len);
> +               complete(&info->negotiate_completion);
> +               break;
> +
> +       /* SMBD data transfer packet */
> +       case SMBD_TRANSFER_DATA:
> +               data_transfer = smbd_response_payload(response);
> +               data_length = le32_to_cpu(data_transfer->data_length);
> +
> +               /*
> +                * If this is a packet with data playload place the data in
> +                * reassembly queue and wake up the reading thread
> +                */
> +               if (data_length) {
> +                       if (info->full_packet_received)
> +                               response->first_segment = true;
> +
> +                       if (le32_to_cpu(data_transfer->remaining_data_length))
> +                               info->full_packet_received = false;
> +                       else
> +                               info->full_packet_received = true;
> +
> +                       enqueue_reassembly(
> +                               info,
> +                               response,
> +                               data_length);
> +               } else
> +                       put_empty_packet(info, response);
> +
> +               if (data_length)
> +                       wake_up_interruptible(&info->wait_reassembly_queue);
> +
> +               atomic_dec(&info->receive_credits);
> +               info->receive_credit_target =
> +                       le16_to_cpu(data_transfer->credits_requested);
> +               atomic_add(le16_to_cpu(data_transfer->credits_granted),
> +                       &info->send_credits);
> +
> +               log_incoming(INFO, "data flags %d data_offset %d "
> +                       "data_length %d remaining_data_length %d\n",
> +                       le16_to_cpu(data_transfer->flags),
> +                       le32_to_cpu(data_transfer->data_offset),
> +                       le32_to_cpu(data_transfer->data_length),
> +                       le32_to_cpu(data_transfer->remaining_data_length));
> +
> +               /* Send a KEEP_ALIVE response right away if requested */
> +               info->keep_alive_requested = KEEP_ALIVE_NONE;
> +               if (le16_to_cpu(data_transfer->flags) &
> +                               SMB_DIRECT_RESPONSE_REQUESTED) {
> +                       info->keep_alive_requested = KEEP_ALIVE_PENDING;
> +               }
> +
> +               queue_work(info->workqueue, &info->recv_done_work);
> +               return;
> +
> +       default:
> +               log_rdma_recv(ERR,
> +                       "unexpected response type=%d\n", response->type);
> +       }
> +
> +error:
> +       put_receive_buffer(info, response, true);
> +}
> +
> +static struct rdma_cm_id *smbd_create_id(
> +               struct smbd_connection *info,
> +               struct sockaddr *dstaddr, int port)
> +{
> +       struct rdma_cm_id *id;
> +       int rc;
> +       __be16 *sport;
> +
> +       id = rdma_create_id(&init_net, smbd_conn_upcall, info,
> +               RDMA_PS_TCP, IB_QPT_RC);
> +       if (IS_ERR(id)) {
> +               rc = PTR_ERR(id);
> +               log_rdma_event(ERR, "rdma_create_id() failed %i\n", rc);
> +               return id;
> +       }
> +
> +       if (dstaddr->sa_family == AF_INET6)
> +               sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port;
> +       else
> +               sport = &((struct sockaddr_in *)dstaddr)->sin_port;
> +
> +       *sport = htons(port);
> +
> +       init_completion(&info->ri_done);
> +       info->ri_rc = -ETIMEDOUT;
> +
> +       rc = rdma_resolve_addr(id, NULL, (struct sockaddr *)dstaddr,
> +               RDMA_RESOLVE_TIMEOUT);
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_resolve_addr() failed %i\n", rc);
> +               goto out;
> +       }
> +       wait_for_completion_interruptible_timeout(
> +               &info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
> +       rc = info->ri_rc;
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_resolve_addr() completed %i\n", rc);
> +               goto out;
> +       }
> +
> +       info->ri_rc = -ETIMEDOUT;
> +       rc = rdma_resolve_route(id, RDMA_RESOLVE_TIMEOUT);
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_resolve_route() failed %i\n", rc);
> +               goto out;
> +       }
> +       wait_for_completion_interruptible_timeout(
> +               &info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
> +       rc = info->ri_rc;
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_resolve_route() completed %i\n", rc);
> +               goto out;
> +       }
> +
> +       return id;
> +
> +out:
> +       rdma_destroy_id(id);
> +       return ERR_PTR(rc);
> +}
> +
> +/*
> + * Test if FRWR (Fast Registration Work Requests) is supported on the device
> + * This implementation requries FRWR on RDMA read/write
> + * return value: true if it is supported
> + */
> +static bool frwr_is_supported(struct ib_device_attr *attrs)
> +{
> +       if (!(attrs->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS))
> +               return false;
> +       if (attrs->max_fast_reg_page_list_len == 0)
> +               return false;
> +       return true;
> +}
> +
> +static int smbd_ia_open(
> +               struct smbd_connection *info,
> +               struct sockaddr *dstaddr, int port)
> +{
> +       int rc;
> +
> +       info->id = smbd_create_id(info, dstaddr, port);
> +       if (IS_ERR(info->id)) {
> +               rc = PTR_ERR(info->id);
> +               goto out1;
> +       }
> +
> +       if (!frwr_is_supported(&info->id->device->attrs)) {
> +               log_rdma_event(ERR,
> +                       "Fast Registration Work Requests "
> +                       "(FRWR) is not supported\n");
> +               log_rdma_event(ERR,
> +                       "Device capability flags = %llx "
> +                       "max_fast_reg_page_list_len = %u\n",
> +                       info->id->device->attrs.device_cap_flags,
> +                       info->id->device->attrs.max_fast_reg_page_list_len);
> +               rc = -EPROTONOSUPPORT;
> +               goto out2;
> +       }
> +
> +       info->pd = ib_alloc_pd(info->id->device, 0);
> +       if (IS_ERR(info->pd)) {
> +               rc = PTR_ERR(info->pd);
> +               log_rdma_event(ERR, "ib_alloc_pd() returned %d\n", rc);
> +               goto out2;
> +       }
> +
> +       return 0;
> +
> +out2:
> +       rdma_destroy_id(info->id);
> +       info->id = NULL;
> +
> +out1:
> +       return rc;
> +}
> +
> +/*
> + * Send a negotiation request message to the peer
> + * The negotiation procedure is in [MS-SMBD] 3.1.5.2 and 3.1.5.3
> + * After negotiation, the transport is connected and ready for
> + * carrying upper layer SMB payload
> + */
> +static int smbd_post_send_negotiate_req(struct smbd_connection *info)
> +{
> +       struct ib_send_wr send_wr, *send_wr_fail;
> +       int rc = -ENOMEM;
> +       struct smbd_request *request;
> +       struct smbd_negotiate_req *packet;
> +
> +       request = mempool_alloc(info->request_mempool, GFP_KERNEL);
> +       if (!request)
> +               return rc;
> +
> +       request->info = info;
> +
> +       packet = smbd_request_payload(request);
> +       packet->min_version = cpu_to_le16(SMBD_V1);
> +       packet->max_version = cpu_to_le16(SMBD_V1);
> +       packet->reserved = 0;
> +       packet->credits_requested = cpu_to_le16(info->send_credit_target);
> +       packet->preferred_send_size = cpu_to_le32(info->max_send_size);
> +       packet->max_receive_size = cpu_to_le32(info->max_receive_size);
> +       packet->max_fragmented_size =
> +               cpu_to_le32(info->max_fragmented_recv_size);
> +
> +       request->num_sge = 1;
> +       request->sge[0].addr = ib_dma_map_single(
> +                               info->id->device, (void *)packet,
> +                               sizeof(*packet), DMA_TO_DEVICE);
> +       if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
> +               rc = -EIO;
> +               goto dma_mapping_failed;
> +       }
> +
> +       request->sge[0].length = sizeof(*packet);
> +       request->sge[0].lkey = info->pd->local_dma_lkey;
> +
> +       ib_dma_sync_single_for_device(
> +               info->id->device, request->sge[0].addr,
> +               request->sge[0].length, DMA_TO_DEVICE);
> +
> +       request->cqe.done = send_done;
> +
> +       send_wr.next = NULL;
> +       send_wr.wr_cqe = &request->cqe;
> +       send_wr.sg_list = request->sge;
> +       send_wr.num_sge = request->num_sge;
> +       send_wr.opcode = IB_WR_SEND;
> +       send_wr.send_flags = IB_SEND_SIGNALED;
> +
> +       log_rdma_send(INFO, "sge addr=%llx length=%x lkey=%x\n",
> +               request->sge[0].addr,
> +               request->sge[0].length, request->sge[0].lkey);
> +
> +       request->has_payload = false;
> +       atomic_inc(&info->send_pending);
> +       rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
> +       if (!rc)
> +               return 0;
> +
> +       /* if we reach here, post send failed */
> +       log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
> +       atomic_dec(&info->send_pending);
> +       ib_dma_unmap_single(info->id->device, request->sge[0].addr,
> +               request->sge[0].length, DMA_TO_DEVICE);
> +
> +dma_mapping_failed:
> +       mempool_free(request, info->request_mempool);
> +       return rc;
> +}
> +
> +/*
> + * Extend the credits to remote peer
> + * This implements [MS-SMBD] 3.1.5.9
> + * The idea is that we should extend credits to remote peer as quickly as
> + * it's allowed, to maintain data flow. We allocate as much receive
> + * buffer as possible, and extend the receive credits to remote peer
> + * return value: the new credtis being granted.
> + */
> +static int manage_credits_prior_sending(struct smbd_connection *info)
> +{
> +       int new_credits;
> +
> +       spin_lock(&info->lock_new_credits_offered);
> +       new_credits = info->new_credits_offered;
> +       info->new_credits_offered = 0;
> +       spin_unlock(&info->lock_new_credits_offered);
> +
> +       return new_credits;
> +}
> +
> +/*
> + * Check if we need to send a KEEP_ALIVE message
> + * The idle connection timer triggers a KEEP_ALIVE message when expires
> + * SMB_DIRECT_RESPONSE_REQUESTED is set in the message flag to have peer send
> + * back a response.
> + * return value:
> + * 1 if SMB_DIRECT_RESPONSE_REQUESTED needs to be set
> + * 0: otherwise
> + */
> +static int manage_keep_alive_before_sending(struct smbd_connection *info)
> +{
> +       if (info->keep_alive_requested == KEEP_ALIVE_PENDING) {
> +               info->keep_alive_requested = KEEP_ALIVE_SENT;
> +               return 1;
> +       }
> +       return 0;
> +}
> +
> +/*
> + * Build and prepare the SMBD packet header
> + * This function waits for avaialbe send credits and build a SMBD packet
> + * header. The caller then optional append payload to the packet after
> + * the header
> + * intput values
> + * size: the size of the payload
> + * remaining_data_length: remaining data to send if this is part of a
> + * fragmented packet
> + * output values
> + * request_out: the request allocated from this function
> + * return values: 0 on success, otherwise actual error code returned
> + */
> +static int smbd_create_header(struct smbd_connection *info,
> +               int size, int remaining_data_length,
> +               struct smbd_request **request_out)
> +{
> +       struct smbd_request *request;
> +       struct smbd_data_transfer *packet;
> +       int header_length;
> +       int rc;
> +
> +       /* Wait for send credits. A SMBD packet needs one credit */
> +       rc = wait_event_interruptible(info->wait_send_queue,
> +               atomic_read(&info->send_credits) > 0 ||
> +               info->transport_status != SMBD_CONNECTED);
> +       if (rc)
> +               return rc;
> +
> +       if (info->transport_status != SMBD_CONNECTED) {
> +               log_outgoing(ERR, "disconnected not sending\n");
> +               return -ENOENT;
> +       }
> +       atomic_dec(&info->send_credits);
> +
> +       request = mempool_alloc(info->request_mempool, GFP_KERNEL);
> +       if (!request) {
> +               rc = -ENOMEM;
> +               goto err;
> +       }
> +
> +       request->info = info;
> +
> +       /* Fill in the packet header */
> +       packet = smbd_request_payload(request);
> +       packet->credits_requested = cpu_to_le16(info->send_credit_target);
> +       packet->credits_granted =
> +               cpu_to_le16(manage_credits_prior_sending(info));
> +       info->send_immediate = false;
> +
> +       packet->flags = 0;
> +       if (manage_keep_alive_before_sending(info))
> +               packet->flags |= cpu_to_le16(SMB_DIRECT_RESPONSE_REQUESTED);
> +
> +       packet->reserved = 0;
> +       if (!size)
> +               packet->data_offset = 0;
> +       else
> +               packet->data_offset = cpu_to_le32(24);
> +       packet->data_length = cpu_to_le32(size);
> +       packet->remaining_data_length = cpu_to_le32(remaining_data_length);
> +       packet->padding = 0;
> +
> +       log_outgoing(INFO, "credits_requested=%d credits_granted=%d "
> +               "data_offset=%d data_length=%d remaining_data_length=%d\n",
> +               le16_to_cpu(packet->credits_requested),
> +               le16_to_cpu(packet->credits_granted),
> +               le32_to_cpu(packet->data_offset),
> +               le32_to_cpu(packet->data_length),
> +               le32_to_cpu(packet->remaining_data_length));
> +
> +       /* Map the packet to DMA */
> +       header_length = sizeof(struct smbd_data_transfer);
> +       /* If this is a packet without payload, don't send padding */
> +       if (!size)
> +               header_length = offsetof(struct smbd_data_transfer, padding);
> +
> +       request->num_sge = 1;
> +       request->sge[0].addr = ib_dma_map_single(info->id->device,
> +                                                (void *)packet,
> +                                                header_length,
> +                                                DMA_BIDIRECTIONAL);
> +       if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
> +               mempool_free(request, info->request_mempool);
> +               rc = -EIO;
> +               goto err;
> +       }
> +
> +       request->sge[0].length = header_length;
> +       request->sge[0].lkey = info->pd->local_dma_lkey;
> +
> +       *request_out = request;
> +       return 0;
> +
> +err:
> +       atomic_inc(&info->send_credits);
> +       return rc;
> +}
> +
> +static void smbd_destroy_header(struct smbd_connection *info,
> +               struct smbd_request *request)
> +{
> +
> +       ib_dma_unmap_single(info->id->device,
> +                           request->sge[0].addr,
> +                           request->sge[0].length,
> +                           DMA_TO_DEVICE);
> +       mempool_free(request, info->request_mempool);
> +       atomic_inc(&info->send_credits);
> +}
> +
> +/* Post the send request */
> +static int smbd_post_send(struct smbd_connection *info,
> +               struct smbd_request *request, bool has_payload)
> +{
> +       struct ib_send_wr send_wr, *send_wr_fail;
> +       int rc, i;
> +
> +       for (i = 0; i < request->num_sge; i++) {
> +               log_rdma_send(INFO,
> +                       "rdma_request sge[%d] addr=%llu legnth=%u\n",
> +                       i, request->sge[0].addr, request->sge[0].length);
> +               ib_dma_sync_single_for_device(
> +                       info->id->device,
> +                       request->sge[i].addr,
> +                       request->sge[i].length,
> +                       DMA_TO_DEVICE);
> +       }
> +
> +       request->cqe.done = send_done;
> +
> +       send_wr.next = NULL;
> +       send_wr.wr_cqe = &request->cqe;
> +       send_wr.sg_list = request->sge;
> +       send_wr.num_sge = request->num_sge;
> +       send_wr.opcode = IB_WR_SEND;
> +       send_wr.send_flags = IB_SEND_SIGNALED;
> +
> +       if (has_payload) {
> +               request->has_payload = true;
> +               atomic_inc(&info->send_payload_pending);
> +       } else {
> +               request->has_payload = false;
> +               atomic_inc(&info->send_pending);
> +       }
> +
> +       rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
> +       if (rc) {
> +               log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
> +               if (has_payload) {
> +                       if (atomic_dec_and_test(&info->send_payload_pending))
> +                               wake_up(&info->wait_send_payload_pending);
> +               } else {
> +                       if (atomic_dec_and_test(&info->send_pending))
> +                               wake_up(&info->wait_send_pending);
> +               }
> +       } else
> +               /* Reset timer for idle connection after packet is sent */
> +               mod_delayed_work(info->workqueue, &info->idle_timer_work,
> +                       info->keep_alive_interval*HZ);
> +
> +       return rc;
> +}
> +
> +static int smbd_post_send_sgl(struct smbd_connection *info,
> +       struct scatterlist *sgl, int data_length, int remaining_data_length)
> +{
> +       int num_sgs;
> +       int i, rc;
> +       struct smbd_request *request;
> +       struct scatterlist *sg;
> +
> +       rc = smbd_create_header(
> +               info, data_length, remaining_data_length, &request);
> +       if (rc)
> +               return rc;
> +
> +       num_sgs = sgl ? sg_nents(sgl) : 0;
> +       for_each_sg(sgl, sg, num_sgs, i) {
> +               request->sge[i+1].addr =
> +                       ib_dma_map_page(info->id->device, sg_page(sg),
> +                              sg->offset, sg->length, DMA_BIDIRECTIONAL);
> +               if (ib_dma_mapping_error(
> +                               info->id->device, request->sge[i+1].addr)) {
> +                       rc = -EIO;
> +                       request->sge[i+1].addr = 0;
> +                       goto dma_mapping_failure;
> +               }
> +               request->sge[i+1].length = sg->length;
> +               request->sge[i+1].lkey = info->pd->local_dma_lkey;
> +               request->num_sge++;
> +       }
> +
> +       rc = smbd_post_send(info, request, data_length);
> +       if (!rc)
> +               return 0;
> +
> +dma_mapping_failure:
> +       for (i = 1; i < request->num_sge; i++)
> +               if (request->sge[i].addr)
> +                       ib_dma_unmap_single(info->id->device,
> +                                           request->sge[i].addr,
> +                                           request->sge[i].length,
> +                                           DMA_TO_DEVICE);
> +       smbd_destroy_header(info, request);
> +       return rc;
> +}
> +
> +/*
> + * Send an empty message
> + * Empty message is used to extend credits to peer to for keep live
> + * while there is no upper layer payload to send at the time
> + */
> +static int smbd_post_send_empty(struct smbd_connection *info)
> +{
> +       info->count_send_empty++;
> +       return smbd_post_send_sgl(info, NULL, 0, 0);
> +}
> +
> +/*
> + * Post a receive request to the transport
> + * The remote peer can only send data when a receive request is posted
> + * The interaction is controlled by send/receive credit system
> + */
> +static int smbd_post_recv(
> +               struct smbd_connection *info, struct smbd_response *response)
> +{
> +       struct ib_recv_wr recv_wr, *recv_wr_fail = NULL;
> +       int rc = -EIO;
> +
> +       response->sge.addr = ib_dma_map_single(
> +                               info->id->device, response->packet,
> +                               info->max_receive_size, DMA_FROM_DEVICE);
> +       if (ib_dma_mapping_error(info->id->device, response->sge.addr))
> +               return rc;
> +
> +       response->sge.length = info->max_receive_size;
> +       response->sge.lkey = info->pd->local_dma_lkey;
> +
> +       response->cqe.done = recv_done;
> +
> +       recv_wr.wr_cqe = &response->cqe;
> +       recv_wr.next = NULL;
> +       recv_wr.sg_list = &response->sge;
> +       recv_wr.num_sge = 1;
> +
> +       rc = ib_post_recv(info->id->qp, &recv_wr, &recv_wr_fail);
> +       if (rc) {
> +               ib_dma_unmap_single(info->id->device, response->sge.addr,
> +                                   response->sge.length, DMA_FROM_DEVICE);
> +
> +               log_rdma_recv(ERR, "ib_post_recv failed rc=%d\n", rc);
> +       }
> +
> +       return rc;
> +}
> +
> +/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */
> +static int smbd_negotiate(struct smbd_connection *info)
> +{
> +       int rc;
> +       struct smbd_response *response = get_receive_buffer(info);
> +
> +       response->type = SMBD_NEGOTIATE_RESP;
> +       rc = smbd_post_recv(info, response);
> +       log_rdma_event(INFO,
> +               "smbd_post_recv rc=%d iov.addr=%llx iov.length=%x "
> +               "iov.lkey=%x\n",
> +               rc, response->sge.addr,
> +               response->sge.length, response->sge.lkey);
> +       if (rc)
> +               return rc;
> +
> +       init_completion(&info->negotiate_completion);
> +       info->negotiate_done = false;
> +       rc = smbd_post_send_negotiate_req(info);
> +       if (rc)
> +               return rc;
> +
> +       rc = wait_for_completion_interruptible_timeout(
> +               &info->negotiate_completion, SMBD_NEGOTIATE_TIMEOUT * HZ);
> +       log_rdma_event(INFO, "wait_for_completion_timeout rc=%d\n", rc);
> +
> +       if (info->negotiate_done)
> +               return 0;
> +
> +       if (rc == 0)
> +               rc = -ETIMEDOUT;
> +       else if (rc == -ERESTARTSYS)
> +               rc = -EINTR;
> +       else
> +               rc = -ENOTCONN;
> +
> +       return rc;
> +}
> +
> +static void put_empty_packet(
> +               struct smbd_connection *info, struct smbd_response *response)
> +{
> +       spin_lock(&info->empty_packet_queue_lock);
> +       list_add_tail(&response->list, &info->empty_packet_queue);
> +       info->count_empty_packet_queue++;
> +       spin_unlock(&info->empty_packet_queue_lock);
> +
> +       queue_work(info->workqueue, &info->post_send_credits_work);
> +}
> +
> +/*
> + * Implement Connection.FragmentReassemblyBuffer defined in [MS-SMBD] 3.1.1.1
> + * This is a queue for reassembling upper layer payload and present to upper
> + * layer. All the inncoming payload go to the reassembly queue, regardless of
> + * if reassembly is required. The uuper layer code reads from the queue for all
> + * incoming payloads.
> + * Put a received packet to the reassembly queue
> + * response: the packet received
> + * data_length: the size of payload in this packet
> + */
> +static void enqueue_reassembly(
> +       struct smbd_connection *info,
> +       struct smbd_response *response,
> +       int data_length)
> +{
> +       spin_lock(&info->reassembly_queue_lock);
> +       list_add_tail(&response->list, &info->reassembly_queue);
> +       info->reassembly_queue_length++;
> +       /*
> +        * Make sure reassembly_data_length is updated after list and
> +        * reassembly_queue_length are updated. On the dequeue side
> +        * reassembly_data_length is checked without a lock to determine
> +        * if reassembly_queue_length and list is up to date
> +        */
> +       virt_wmb();
> +       info->reassembly_data_length += data_length;
> +       spin_unlock(&info->reassembly_queue_lock);
> +       info->count_reassembly_queue++;
> +       info->count_enqueue_reassembly_queue++;
> +}
> +
> +/*
> + * Get the first entry at the front of reassembly queue
> + * Caller is responsible for locking
> + * return value: the first entry if any, NULL if queue is empty
> + */
> +static struct smbd_response *_get_first_reassembly(struct smbd_connection *info)
> +{
> +       struct smbd_response *ret = NULL;
> +
> +       if (!list_empty(&info->reassembly_queue)) {
> +               ret = list_first_entry(
> +                       &info->reassembly_queue,
> +                       struct smbd_response, list);
> +       }
> +       return ret;
> +}
> +
> +static struct smbd_response *get_empty_queue_buffer(
> +               struct smbd_connection *info)
> +{
> +       struct smbd_response *ret = NULL;
> +       unsigned long flags;
> +
> +       spin_lock_irqsave(&info->empty_packet_queue_lock, flags);
> +       if (!list_empty(&info->empty_packet_queue)) {
> +               ret = list_first_entry(
> +                       &info->empty_packet_queue,
> +                       struct smbd_response, list);
> +               list_del(&ret->list);
> +               info->count_empty_packet_queue--;
> +       }
> +       spin_unlock_irqrestore(&info->empty_packet_queue_lock, flags);
> +
> +       return ret;
> +}
> +
> +/*
> + * Get a receive buffer
> + * For each remote send, we need to post a receive. The receive buffers are
> + * pre-allocated in advance.
> + * return value: the receive buffer, NULL if none is available
> + */
> +static struct smbd_response *get_receive_buffer(struct smbd_connection *info)
> +{
> +       struct smbd_response *ret = NULL;
> +       unsigned long flags;
> +
> +       spin_lock_irqsave(&info->receive_queue_lock, flags);
> +       if (!list_empty(&info->receive_queue)) {
> +               ret = list_first_entry(
> +                       &info->receive_queue,
> +                       struct smbd_response, list);
> +               list_del(&ret->list);
> +               info->count_receive_queue--;
> +               info->count_get_receive_buffer++;
> +       }
> +       spin_unlock_irqrestore(&info->receive_queue_lock, flags);
> +
> +       return ret;
> +}
> +
> +/*
> + * Return a receive buffer
> + * Upon returning of a receive buffer, we can post new receive and extend
> + * more receive credits to remote peer. This is done immediately after a
> + * receive buffer is returned.
> + */
> +static void put_receive_buffer(
> +       struct smbd_connection *info, struct smbd_response *response,
> +       bool lock)
> +{
> +       unsigned long flags;
> +
> +       ib_dma_unmap_single(info->id->device, response->sge.addr,
> +               response->sge.length, DMA_FROM_DEVICE);
> +
> +       if (lock)
> +               spin_lock_irqsave(&info->receive_queue_lock, flags);
> +       list_add_tail(&response->list, &info->receive_queue);
> +       info->count_receive_queue++;
> +       info->count_put_receive_buffer++;
> +       if (lock)
> +               spin_unlock_irqrestore(&info->receive_queue_lock, flags);
> +
> +       queue_work(info->workqueue, &info->post_send_credits_work);
> +}
> +
> +/* Preallocate all receive buffer on transport establishment */
> +static int allocate_receive_buffers(struct smbd_connection *info, int num_buf)
> +{
> +       int i;
> +       struct smbd_response *response;
> +
> +       INIT_LIST_HEAD(&info->reassembly_queue);
> +       spin_lock_init(&info->reassembly_queue_lock);
> +       info->reassembly_data_length = 0;
> +       info->reassembly_queue_length = 0;
> +
> +       INIT_LIST_HEAD(&info->receive_queue);
> +       spin_lock_init(&info->receive_queue_lock);
> +       info->count_receive_queue = 0;
> +
> +       INIT_LIST_HEAD(&info->empty_packet_queue);
> +       spin_lock_init(&info->empty_packet_queue_lock);
> +       info->count_empty_packet_queue = 0;
> +
> +       init_waitqueue_head(&info->wait_receive_queues);
> +
> +       for (i = 0; i < num_buf; i++) {
> +               response = mempool_alloc(info->response_mempool, GFP_KERNEL);
> +               if (!response)
> +                       goto allocate_failed;
> +
> +               response->info = info;
> +               list_add_tail(&response->list, &info->receive_queue);
> +               info->count_receive_queue++;
> +       }
> +
> +       return 0;
> +
> +allocate_failed:
> +       while (!list_empty(&info->receive_queue)) {
> +               response = list_first_entry(
> +                               &info->receive_queue,
> +                               struct smbd_response, list);
> +               list_del(&response->list);
> +               info->count_receive_queue--;
> +
> +               mempool_free(response, info->response_mempool);
> +       }
> +       return -ENOMEM;
> +}
> +
> +static void destroy_receive_buffers(struct smbd_connection *info)
> +{
> +       struct smbd_response *response;
> +
> +       while ((response = get_receive_buffer(info)))
> +               mempool_free(response, info->response_mempool);
> +
> +       while ((response = get_empty_queue_buffer(info)))
> +               mempool_free(response, info->response_mempool);
> +}
> +
> +/*
> + * Check and send an immediate or keep alive packet
> + * The condition to send those packets are defined in [MS-SMBD] 3.1.1.1
> + * Connection.KeepaliveRequested and Connection.SendImmediate
> + * The idea is to extend credits to server as soon as it becomes available
> + */
> +static void send_immediate_work(struct work_struct *work)
> +{
> +       struct smbd_connection *info = container_of(
> +                                       work, struct smbd_connection,
> +                                       send_immediate_work.work);
> +
> +       if (info->keep_alive_requested == KEEP_ALIVE_PENDING ||
> +           info->send_immediate) {
> +               log_keep_alive(INFO, "send an empty message\n");
> +               smbd_post_send_empty(info);
> +       }
> +}
> +
> +/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */
> +static void idle_connection_timer(struct work_struct *work)
> +{
> +       struct smbd_connection *info = container_of(
> +                                       work, struct smbd_connection,
> +                                       idle_timer_work.work);
> +
> +       if (info->keep_alive_requested != KEEP_ALIVE_NONE) {
> +               log_keep_alive(ERR,
> +                       "error status info->keep_alive_requested=%d\n",
> +                       info->keep_alive_requested);
> +               smbd_disconnect_rdma_connection(info);
> +               return;
> +       }
> +
> +       log_keep_alive(INFO, "about to send an empty idle message\n");
> +       smbd_post_send_empty(info);
> +
> +       /* Setup the next idle timeout work */
> +       queue_delayed_work(info->workqueue, &info->idle_timer_work,
> +                       info->keep_alive_interval*HZ);
> +}
> +
> +static void destroy_caches_and_workqueue(struct smbd_connection *info)
> +{
> +       destroy_receive_buffers(info);
> +       destroy_workqueue(info->workqueue);
> +       mempool_destroy(info->response_mempool);
> +       kmem_cache_destroy(info->response_cache);
> +       mempool_destroy(info->request_mempool);
> +       kmem_cache_destroy(info->request_cache);
> +}
> +
> +#define MAX_NAME_LEN   80
> +static int allocate_caches_and_workqueue(struct smbd_connection *info)
> +{
> +       char name[MAX_NAME_LEN];
> +       int rc;
> +
> +       snprintf(name, MAX_NAME_LEN, "smbd_request_%p", info);
> +       info->request_cache =
> +               kmem_cache_create(
> +                       name,
> +                       sizeof(struct smbd_request) +
> +                               sizeof(struct smbd_data_transfer),
> +                       0, SLAB_HWCACHE_ALIGN, NULL);
> +       if (!info->request_cache)
> +               return -ENOMEM;
> +
> +       info->request_mempool =
> +               mempool_create(info->send_credit_target, mempool_alloc_slab,
> +                       mempool_free_slab, info->request_cache);
> +       if (!info->request_mempool)
> +               goto out1;
> +
> +       snprintf(name, MAX_NAME_LEN, "smbd_response_%p", info);
> +       info->response_cache =
> +               kmem_cache_create(
> +                       name,
> +                       sizeof(struct smbd_response) +
> +                               info->max_receive_size,
> +                       0, SLAB_HWCACHE_ALIGN, NULL);
> +       if (!info->response_cache)
> +               goto out2;
> +
> +       info->response_mempool =
> +               mempool_create(info->receive_credit_max, mempool_alloc_slab,
> +                      mempool_free_slab, info->response_cache);
> +       if (!info->response_mempool)
> +               goto out3;
> +
> +       snprintf(name, MAX_NAME_LEN, "smbd_%p", info);
> +       info->workqueue = create_workqueue(name);
> +       if (!info->workqueue)
> +               goto out4;
> +
> +       rc = allocate_receive_buffers(info, info->receive_credit_max);
> +       if (rc) {
> +               log_rdma_event(ERR, "failed to allocate receive buffers\n");
> +               goto out5;
> +       }
> +
> +       return 0;
> +
> +out5:
> +       destroy_workqueue(info->workqueue);
> +out4:
> +       mempool_destroy(info->response_mempool);
> +out3:
> +       kmem_cache_destroy(info->response_cache);
> +out2:
> +       mempool_destroy(info->request_mempool);
> +out1:
> +       kmem_cache_destroy(info->request_cache);
> +       return -ENOMEM;
> +}
> +
> +/* Create a SMBD connection, called by upper layer */
> +struct smbd_connection *_smbd_get_connection(
> +       struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
> +{
> +       int rc;
> +       struct smbd_connection *info;
> +       struct rdma_conn_param conn_param;
> +       struct ib_qp_init_attr qp_attr;
> +       struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;
> +
> +       info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL);
> +       if (!info)
> +               return NULL;
> +
> +       info->transport_status = SMBD_CONNECTING;
> +       rc = smbd_ia_open(info, dstaddr, port);
> +       if (rc) {
> +               log_rdma_event(INFO, "smbd_ia_open rc=%d\n", rc);
> +               goto create_id_failed;
> +       }
> +
> +       if (smbd_send_credit_target > info->id->device->attrs.max_cqe ||
> +           smbd_send_credit_target > info->id->device->attrs.max_qp_wr) {
> +               log_rdma_event(ERR,
> +                       "consider lowering send_credit_target = %d. "
> +                       "Possible CQE overrun, device "
> +                       "reporting max_cpe %d max_qp_wr %d\n",
> +                       smbd_send_credit_target,
> +                       info->id->device->attrs.max_cqe,
> +                       info->id->device->attrs.max_qp_wr);
> +               goto config_failed;
> +       }
> +
> +       if (smbd_receive_credit_max > info->id->device->attrs.max_cqe ||
> +           smbd_receive_credit_max > info->id->device->attrs.max_qp_wr) {
> +               log_rdma_event(ERR,
> +                       "consider lowering receive_credit_max = %d. "
> +                       "Possible CQE overrun, device "
> +                       "reporting max_cpe %d max_qp_wr %d\n",
> +                       smbd_receive_credit_max,
> +                       info->id->device->attrs.max_cqe,
> +                       info->id->device->attrs.max_qp_wr);
> +               goto config_failed;
> +       }
> +
> +       info->receive_credit_max = smbd_receive_credit_max;
> +       info->send_credit_target = smbd_send_credit_target;
> +       info->max_send_size = smbd_max_send_size;
> +       info->max_fragmented_recv_size = smbd_max_fragmented_recv_size;
> +       info->max_receive_size = smbd_max_receive_size;
> +       info->keep_alive_interval = smbd_keep_alive_interval;
> +
> +       if (SMBDIRECT_MAX_SGE > info->id->device->attrs.max_sge) {
> +               log_rdma_event(ERR, "warning: device max_sge = %d too small\n",
> +                       info->id->device->attrs.max_sge);
> +               log_rdma_event(ERR, "Queue Pair creation may fail\n");
> +       }
> +
> +       info->send_cq = NULL;
> +       info->recv_cq = NULL;
> +       info->send_cq = ib_alloc_cq(info->id->device, info,
> +                       info->send_credit_target, 0, IB_POLL_SOFTIRQ);
> +       if (IS_ERR(info->send_cq)) {
> +               info->send_cq = NULL;
> +               goto alloc_cq_failed;
> +       }
> +
> +       info->recv_cq = ib_alloc_cq(info->id->device, info,
> +                       info->receive_credit_max, 0, IB_POLL_SOFTIRQ);
> +       if (IS_ERR(info->recv_cq)) {
> +               info->recv_cq = NULL;
> +               goto alloc_cq_failed;
> +       }
> +
> +       memset(&qp_attr, 0, sizeof(qp_attr));
> +       qp_attr.event_handler = smbd_qp_async_error_upcall;
> +       qp_attr.qp_context = info;
> +       qp_attr.cap.max_send_wr = info->send_credit_target;
> +       qp_attr.cap.max_recv_wr = info->receive_credit_max;
> +       qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
> +       qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
> +       qp_attr.cap.max_inline_data = 0;
> +       qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
> +       qp_attr.qp_type = IB_QPT_RC;
> +       qp_attr.send_cq = info->send_cq;
> +       qp_attr.recv_cq = info->recv_cq;
> +       qp_attr.port_num = ~0;
> +
> +       rc = rdma_create_qp(info->id, info->pd, &qp_attr);
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_create_qp failed %i\n", rc);
> +               goto create_qp_failed;
> +       }
> +
> +       memset(&conn_param, 0, sizeof(conn_param));
> +       conn_param.initiator_depth = 0;
> +
> +       conn_param.retry_count = SMBD_CM_RETRY;
> +       conn_param.rnr_retry_count = SMBD_CM_RNR_RETRY;
> +       conn_param.flow_control = 0;
> +       init_waitqueue_head(&info->wait_destroy);
> +
> +       log_rdma_event(INFO, "connecting to IP %pI4 port %d\n",
> +               &addr_in->sin_addr, port);
> +
> +       init_waitqueue_head(&info->conn_wait);
> +       rc = rdma_connect(info->id, &conn_param);
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_connect() failed with %i\n", rc);
> +               goto rdma_connect_failed;
> +       }
> +
> +       wait_event_interruptible(
> +               info->conn_wait, info->transport_status != SMBD_CONNECTING);
> +
> +       if (info->transport_status != SMBD_CONNECTED) {
> +               log_rdma_event(ERR, "rdma_connect failed port=%d\n", port);
> +               goto rdma_connect_failed;
> +       }
> +
> +       log_rdma_event(INFO, "rdma_connect connected\n");
> +
> +       rc = allocate_caches_and_workqueue(info);
> +       if (rc) {
> +               log_rdma_event(ERR, "cache allocation failed\n");
> +               goto allocate_cache_failed;
> +       }
> +
> +       init_waitqueue_head(&info->wait_send_queue);
> +       init_waitqueue_head(&info->wait_reassembly_queue);
> +
> +       INIT_DELAYED_WORK(&info->idle_timer_work, idle_connection_timer);
> +       INIT_DELAYED_WORK(&info->send_immediate_work, send_immediate_work);
> +       queue_delayed_work(info->workqueue, &info->idle_timer_work,
> +               info->keep_alive_interval*HZ);
> +
> +       init_waitqueue_head(&info->wait_send_pending);
> +       atomic_set(&info->send_pending, 0);
> +
> +       init_waitqueue_head(&info->wait_send_payload_pending);
> +       atomic_set(&info->send_payload_pending, 0);
> +
> +       INIT_WORK(&info->disconnect_work, smbd_disconnect_rdma_work);
> +       INIT_WORK(&info->destroy_work, smbd_destroy_rdma_work);
> +       INIT_WORK(&info->recv_done_work, smbd_recv_done_work);
> +       INIT_WORK(&info->post_send_credits_work, smbd_post_send_credits);
> +       info->new_credits_offered = 0;
> +       spin_lock_init(&info->lock_new_credits_offered);
> +
> +       rc = smbd_negotiate(info);
> +       if (rc) {
> +               log_rdma_event(ERR, "smbd_negotiate rc=%d\n", rc);
> +               goto negotiation_failed;
> +       }
> +
> +       return info;
> +
> +negotiation_failed:
> +       cancel_delayed_work_sync(&info->idle_timer_work);
> +       destroy_caches_and_workqueue(info);
> +       info->transport_status = SMBD_NEGOTIATE_FAILED;
> +       init_waitqueue_head(&info->conn_wait);
> +       rdma_disconnect(info->id);
> +       wait_event(info->conn_wait,
> +               info->transport_status == SMBD_DISCONNECTED);
> +
> +allocate_cache_failed:
> +rdma_connect_failed:
> +       rdma_destroy_qp(info->id);
> +
> +create_qp_failed:
> +alloc_cq_failed:
> +       if (info->send_cq)
> +               ib_free_cq(info->send_cq);
> +       if (info->recv_cq)
> +               ib_free_cq(info->recv_cq);
> +
> +config_failed:
> +       ib_dealloc_pd(info->pd);
> +       rdma_destroy_id(info->id);
> +
> +create_id_failed:
> +       kfree(info);
> +       return NULL;
> +}
> diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
> index c55f28b..35bc25b 100644
> --- a/fs/cifs/smbdirect.h
> +++ b/fs/cifs/smbdirect.h
> @@ -16,6 +16,286 @@
>  #ifndef _SMBDIRECT_H
>  #define _SMBDIRECT_H
>
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +#define cifs_rdma_enabled(server)      ((server)->rdma)
> +
> +#include "cifsglob.h"
> +#include <rdma/ib_verbs.h>
> +#include <rdma/rdma_cm.h>
> +#include <linux/mempool.h>
> +
> +enum keep_alive_status {
> +       KEEP_ALIVE_NONE,
> +       KEEP_ALIVE_PENDING,
> +       KEEP_ALIVE_SENT,
> +};
> +
> +enum smbd_connection_status {
> +       SMBD_CREATED,
> +       SMBD_CONNECTING,
> +       SMBD_CONNECTED,
> +       SMBD_NEGOTIATE_FAILED,
> +       SMBD_DISCONNECTING,
> +       SMBD_DISCONNECTED,
> +       SMBD_DESTROYED
> +};
> +
> +/*
> + * The context for the SMBDirect transport
> + * Everything related to the transport is here. It has several logical parts
> + * 1. RDMA related structures
> + * 2. SMBDirect connection parameters
> + * 3. Memory registrations
> + * 4. Receive and reassembly queues for data receive path
> + * 5. mempools for allocating packets
> + */
> +struct smbd_connection {
> +       enum smbd_connection_status transport_status;
> +
> +       /* RDMA related */
> +       struct rdma_cm_id *id;
> +       struct ib_qp_init_attr qp_attr;
> +       struct ib_pd *pd;
> +       struct ib_cq *send_cq, *recv_cq;
> +       struct ib_device_attr dev_attr;
> +       int ri_rc;
> +       struct completion ri_done;
> +       wait_queue_head_t conn_wait;
> +       wait_queue_head_t wait_destroy;
> +
> +       struct completion negotiate_completion;
> +       bool negotiate_done;
> +
> +       struct work_struct destroy_work;
> +       struct work_struct disconnect_work;
> +       struct work_struct recv_done_work;
> +       struct work_struct post_send_credits_work;
> +
> +       spinlock_t lock_new_credits_offered;
> +       int new_credits_offered;
> +
> +       /* Connection parameters defined in [MS-SMBD] 3.1.1.1 */
> +       int receive_credit_max;
> +       int send_credit_target;
> +       int max_send_size;
> +       int max_fragmented_recv_size;
> +       int max_fragmented_send_size;
> +       int max_receive_size;
> +       int keep_alive_interval;
> +       int max_readwrite_size;
> +       enum keep_alive_status keep_alive_requested;
> +       int protocol;
> +       atomic_t send_credits;
> +       atomic_t receive_credits;
> +       int receive_credit_target;
> +       int fragment_reassembly_remaining;
> +
> +       /* Activity accoutning */
> +
> +       atomic_t send_pending;
> +       wait_queue_head_t wait_send_pending;
> +       atomic_t send_payload_pending;
> +       wait_queue_head_t wait_send_payload_pending;
> +
> +       /* Receive queue */
> +       struct list_head receive_queue;
> +       int count_receive_queue;
> +       spinlock_t receive_queue_lock;
> +
> +       struct list_head empty_packet_queue;
> +       int count_empty_packet_queue;
> +       spinlock_t empty_packet_queue_lock;
> +
> +       wait_queue_head_t wait_receive_queues;
> +
> +       /* Reassembly queue */
> +       struct list_head reassembly_queue;
> +       spinlock_t reassembly_queue_lock;
> +       wait_queue_head_t wait_reassembly_queue;
> +
> +       /* total data length of reassembly queue */
> +       int reassembly_data_length;
> +       int reassembly_queue_length;
> +       /* the offset to first buffer in reassembly queue */
> +       int first_entry_offset;
> +
> +       bool send_immediate;
> +
> +       wait_queue_head_t wait_send_queue;
> +
> +       /*
> +        * Indicate if we have received a full packet on the connection
> +        * This is used to identify the first SMBD packet of a assembled
> +        * payload (SMB packet) in reassembly queue so we can return a
> +        * RFC1002 length to upper layer to indicate the length of the SMB
> +        * packet received
> +        */
> +       bool full_packet_received;
> +
> +       struct workqueue_struct *workqueue;
> +       struct delayed_work idle_timer_work;
> +       struct delayed_work send_immediate_work;
> +
> +       /* Memory pool for preallocating buffers */
> +       /* request pool for RDMA send */
> +       struct kmem_cache *request_cache;
> +       mempool_t *request_mempool;
> +
> +       /* response pool for RDMA receive */
> +       struct kmem_cache *response_cache;
> +       mempool_t *response_mempool;
> +
> +       /* for debug purposes */
> +       unsigned int count_get_receive_buffer;
> +       unsigned int count_put_receive_buffer;
> +       unsigned int count_reassembly_queue;
> +       unsigned int count_enqueue_reassembly_queue;
> +       unsigned int count_dequeue_reassembly_queue;
> +       unsigned int count_send_empty;
> +};
> +
> +enum smbd_message_type {
> +       SMBD_NEGOTIATE_RESP,
> +       SMBD_TRANSFER_DATA,
> +};
> +
> +#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
> +
> +/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */
> +struct smbd_negotiate_req {
> +       __le16 min_version;
> +       __le16 max_version;
> +       __le16 reserved;
> +       __le16 credits_requested;
> +       __le32 preferred_send_size;
> +       __le32 max_receive_size;
> +       __le32 max_fragmented_size;
> +} __packed;
> +
> +/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */
> +struct smbd_negotiate_resp {
> +       __le16 min_version;
> +       __le16 max_version;
> +       __le16 negotiated_version;
> +       __le16 reserved;
> +       __le16 credits_requested;
> +       __le16 credits_granted;
> +       __le32 status;
> +       __le32 max_readwrite_size;
> +       __le32 preferred_send_size;
> +       __le32 max_receive_size;
> +       __le32 max_fragmented_size;
> +} __packed;
> +
> +/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */
> +struct smbd_data_transfer {
> +       __le16 credits_requested;
> +       __le16 credits_granted;
> +       __le16 flags;
> +       __le16 reserved;
> +       __le32 remaining_data_length;
> +       __le32 data_offset;
> +       __le32 data_length;
> +       __le32 padding;
> +       __u8 buffer[];
> +} __packed;
> +
> +/* The packet fields for a registered RDMA buffer */
> +struct smbd_buffer_descriptor_v1 {
> +       __le64 offset;
> +       __le32 token;
> +       __le32 length;
> +} __packed;
> +
>  /* Default maximum number of SGEs in a RDMA send/recv */
>  #define SMBDIRECT_MAX_SGE      16
> +/* The context for a SMBD request */
> +struct smbd_request {
> +       struct smbd_connection *info;
> +       struct ib_cqe cqe;
> +
> +       /* true if this request carries upper layer payload */
> +       bool has_payload;
> +
> +       /* the SGE entries for this packet */
> +       struct ib_sge sge[SMBDIRECT_MAX_SGE];
> +       int num_sge;
> +
> +       /* SMBD packet header follows this structure */
> +       u8 packet[];
> +};
> +
> +/* The context for a SMBD response */
> +struct smbd_response {
> +       struct smbd_connection *info;
> +       struct ib_cqe cqe;
> +       struct ib_sge sge;
> +
> +       enum smbd_message_type type;
> +
> +       /* Link to receive queue or reassembly queue */
> +       struct list_head list;
> +
> +       /* Indicate if this is the 1st packet of a payload */
> +       bool first_segment;
> +
> +       /* SMBD packet header and payload follows this structure */
> +       u8 packet[];
> +};
> +
> +/* Create a SMBDirect session */
> +struct smbd_connection *smbd_get_connection(
> +       struct TCP_Server_Info *server, struct sockaddr *dstaddr);
> +
> +/* Reconnect SMBDirect session */
> +int smbd_reconnect(struct TCP_Server_Info *server);
> +
> +/* Destroy SMBDirect session */
> +void smbd_destroy(struct smbd_connection *info);
> +
> +/* Interface for carrying upper layer I/O through send/recv */
> +int smbd_recv(struct smbd_connection *info, struct msghdr *msg);
> +int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst);
> +
> +enum mr_state {
> +       MR_READY,
> +       MR_REGISTERED,
> +       MR_INVALIDATED,
> +       MR_ERROR
> +};
> +
> +struct smbd_mr {
> +       struct smbd_connection  *conn;
> +       struct list_head        list;
> +       enum mr_state           state;
> +       struct ib_mr            *mr;
> +       struct scatterlist      *sgl;
> +       int                     sgl_count;
> +       enum dma_data_direction dir;
> +       union {
> +               struct ib_reg_wr        wr;
> +               struct ib_send_wr       inv_wr;
> +       };
> +       struct ib_cqe           cqe;
> +       bool                    need_invalidate;
> +       struct completion       invalidate_done;
> +};
> +
> +/* Interfaces to register and deregister MR for RDMA read/write */
> +struct smbd_mr *smbd_register_mr(
> +       struct smbd_connection *info, struct page *pages[], int num_pages,
> +       int tailsz, bool writing, bool need_invalidate);
> +int smbd_deregister_mr(struct smbd_mr *mr);
> +
> +#else
> +#define cifs_rdma_enabled(server)      0
> +struct smbd_connection{};
> +static inline void *smbd_get_connection(
> +       struct TCP_Server_Info *server, struct sockaddr *dstaddr) {return NULL;}
> +static inline int smbd_reconnect(struct TCP_Server_Info *server) {return -1;}
> +static inline void smbd_destroy(struct smbd_connection *info) {}
> +static inline int smbd_recv(struct smbd_connection *info, struct msghdr *msg) {return -1;}
> +static inline int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst) {return -1;}
> +#endif
> +
>  #endif
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

[ronnie sahlberg]

On Tue, Nov 7, 2017 at 6:54 PM, Long Li <longli@exchange.microsoft.com> wrote:
> From: Long Li <longli@microsoft.com>
>
> Add code to implement the core functions to establish a SMB Direct connection.
>
> 1. Establish an RDMA connection to SMB server.
> 2. Negotiate and setup SMB Direct protocol.
> 3. Implement idle connection timer and credit management.
>
> SMB Direct is enabled by setting CONFIG_CIFS_SMB_DIRECT.
>
> Add to Makefile to enable building SMB Direct.
>
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>  fs/cifs/Makefile    |    2 +
>  fs/cifs/smbdirect.c | 1576 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  fs/cifs/smbdirect.h |  280 +++++++++
>  3 files changed, 1858 insertions(+)
>
> diff --git a/fs/cifs/Makefile b/fs/cifs/Makefile
> index 5e853a3..ad00873 100644
> --- a/fs/cifs/Makefile
> +++ b/fs/cifs/Makefile
> @@ -18,3 +18,5 @@ cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
>  cifs-$(CONFIG_CIFS_DFS_UPCALL) += dns_resolve.o cifs_dfs_ref.o
>
>  cifs-$(CONFIG_CIFS_FSCACHE) += fscache.o cache.o
> +
> +cifs-$(CONFIG_CIFS_SMB_DIRECT) += smbdirect.o
> diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
> index d3c16f8..021d527 100644
> --- a/fs/cifs/smbdirect.c
> +++ b/fs/cifs/smbdirect.c
> @@ -13,7 +13,34 @@
>   *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
>   *   the GNU General Public License for more details.
>   */
> +#include <linux/module.h>
>  #include "smbdirect.h"
> +#include "cifs_debug.h"
> +
> +static struct smbd_response *get_empty_queue_buffer(
> +               struct smbd_connection *info);
> +static struct smbd_response *get_receive_buffer(
> +               struct smbd_connection *info);
> +static void put_receive_buffer(
> +               struct smbd_connection *info,
> +               struct smbd_response *response,
> +               bool lock);
> +static int allocate_receive_buffers(struct smbd_connection *info, int num_buf);
> +static void destroy_receive_buffers(struct smbd_connection *info);
> +
> +static void put_empty_packet(
> +               struct smbd_connection *info, struct smbd_response *response);
> +static void enqueue_reassembly(
> +               struct smbd_connection *info,
> +               struct smbd_response *response, int data_length);
> +static struct smbd_response *_get_first_reassembly(
> +               struct smbd_connection *info);
> +
> +static int smbd_post_recv(
> +               struct smbd_connection *info,
> +               struct smbd_response *response);
> +
> +static int smbd_post_send_empty(struct smbd_connection *info);
>
>  /* SMBD version number */
>  #define SMBD_V1        0x0100
> @@ -75,3 +102,1552 @@ int smbd_max_frmr_depth = 2048;
>
>  /* If payload is less than this byte, use RDMA send/recv not read/write */
>  int rdma_readwrite_threshold = 4096;
> +
> +/* Transport logging functions
> + * Logging are defined as classes. They can be OR'ed to define the actual
> + * logging level via module parameter smbd_logging_class
> + * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
> + * log_rdma_event()

Should be 0xa0 ?

> + */
> +#define LOG_OUTGOING                   0x1
> +#define LOG_INCOMING                   0x2
> +#define LOG_READ                       0x4
> +#define LOG_WRITE                      0x8
> +#define LOG_RDMA_SEND                  0x10
> +#define LOG_RDMA_RECV                  0x20
> +#define LOG_KEEP_ALIVE                 0x40
> +#define LOG_RDMA_EVENT                 0x80
> +#define LOG_RDMA_MR                    0x100
> +static unsigned int smbd_logging_class = 0;
> +module_param(smbd_logging_class, uint, 0644);
> +MODULE_PARM_DESC(smbd_logging_class,
> +       "Logging class for SMBD transport 0x0 to 0x100");
> +
> +#define ERR            0x0
> +#define INFO           0x1
> +static unsigned int smbd_logging_level = ERR;
> +module_param(smbd_logging_level, uint, 0644);
> +MODULE_PARM_DESC(smbd_logging_level,
> +       "Logging level for SMBD transport, 0 (default): error, 1: info");
> +
> +#define log_rdma(level, class, fmt, args...)                           \
> +do {                                                                   \
> +       if (level <= smbd_logging_level || class & smbd_logging_class)  \
> +               cifs_dbg(VFS, "%s:%d " fmt, __func__, __LINE__, ##args);\
> +} while (0)
> +
> +#define log_outgoing(level, fmt, args...) \
> +               log_rdma(level, LOG_OUTGOING, fmt, ##args)
> +#define log_incoming(level, fmt, args...) \
> +               log_rdma(level, LOG_INCOMING, fmt, ##args)
> +#define log_read(level, fmt, args...)  log_rdma(level, LOG_READ, fmt, ##args)
> +#define log_write(level, fmt, args...) log_rdma(level, LOG_WRITE, fmt, ##args)
> +#define log_rdma_send(level, fmt, args...) \
> +               log_rdma(level, LOG_RDMA_SEND, fmt, ##args)
> +#define log_rdma_recv(level, fmt, args...) \
> +               log_rdma(level, LOG_RDMA_RECV, fmt, ##args)
> +#define log_keep_alive(level, fmt, args...) \
> +               log_rdma(level, LOG_KEEP_ALIVE, fmt, ##args)
> +#define log_rdma_event(level, fmt, args...) \
> +               log_rdma(level, LOG_RDMA_EVENT, fmt, ##args)
> +#define log_rdma_mr(level, fmt, args...) \
> +               log_rdma(level, LOG_RDMA_MR, fmt, ##args)
> +
> +/*
> + * Destroy the transport and related RDMA and memory resources
> + * Need to go through all the pending counters and make sure on one is using
> + * the transport while it is destroyed
> + */
> +static void smbd_destroy_rdma_work(struct work_struct *work)
> +{
> +       struct smbd_response *response;
> +       struct smbd_connection *info =
> +               container_of(work, struct smbd_connection, destroy_work);
> +       unsigned long flags;
> +
> +       log_rdma_event(INFO, "destroying qp\n");
> +       ib_drain_qp(info->id->qp);
> +       rdma_destroy_qp(info->id);
> +
> +       /* Unblock all I/O waiting on the send queue */
> +       wake_up_interruptible_all(&info->wait_send_queue);
> +
> +       log_rdma_event(INFO, "cancelling idle timer\n");
> +       cancel_delayed_work_sync(&info->idle_timer_work);
> +       log_rdma_event(INFO, "cancelling send immediate work\n");
> +       cancel_delayed_work_sync(&info->send_immediate_work);
> +
> +       log_rdma_event(INFO, "wait for all recv to finish\n");
> +       wake_up_interruptible(&info->wait_reassembly_queue);
> +
> +       log_rdma_event(INFO, "wait for all send posted to IB to finish\n");
> +       wait_event(info->wait_send_pending,
> +               atomic_read(&info->send_pending) == 0);
> +       wait_event(info->wait_send_payload_pending,
> +               atomic_read(&info->send_payload_pending) == 0);
> +
> +       /* It's not posssible for upper layer to get to reassembly */
> +       log_rdma_event(INFO, "drain the reassembly queue\n");
> +       do {
> +               spin_lock_irqsave(&info->reassembly_queue_lock, flags);
> +               response = _get_first_reassembly(info);
> +               if (response) {
> +                       list_del(&response->list);
> +                       spin_unlock_irqrestore(
> +                               &info->reassembly_queue_lock, flags);
> +                       put_receive_buffer(info, response, true);
> +               }
> +       } while (response);
> +       spin_unlock_irqrestore(&info->reassembly_queue_lock, flags);
> +       info->reassembly_data_length = 0;
> +
> +       log_rdma_event(INFO, "free receive buffers\n");
> +       wait_event(info->wait_receive_queues,
> +               info->count_receive_queue + info->count_empty_packet_queue
> +                       == info->receive_credit_max);
> +       destroy_receive_buffers(info);
> +
> +       ib_free_cq(info->send_cq);
> +       ib_free_cq(info->recv_cq);
> +       ib_dealloc_pd(info->pd);
> +       rdma_destroy_id(info->id);
> +
> +       /* free mempools */
> +       mempool_destroy(info->request_mempool);
> +       kmem_cache_destroy(info->request_cache);
> +
> +       mempool_destroy(info->response_mempool);
> +       kmem_cache_destroy(info->response_cache);
> +
> +       info->transport_status = SMBD_DESTROYED;
> +       wake_up_all(&info->wait_destroy);
> +}
> +
> +static int smbd_process_disconnected(struct smbd_connection *info)
> +{
> +//     queue_work(info->workqueue, &info->destroy_work);
> +       schedule_work(&info->destroy_work);
> +       return 0;
> +}
> +
> +static void smbd_disconnect_rdma_work(struct work_struct *work)
> +{
> +       struct smbd_connection *info =
> +               container_of(work, struct smbd_connection, disconnect_work);
> +
> +       if (info->transport_status == SMBD_CONNECTED) {
> +               info->transport_status = SMBD_DISCONNECTING;
> +               rdma_disconnect(info->id);
> +       }
> +}
> +
> +static void smbd_disconnect_rdma_connection(struct smbd_connection *info)
> +{
> +       queue_work(info->workqueue, &info->disconnect_work);
> +}
> +
> +/* Upcall from RDMA CM */
> +static int smbd_conn_upcall(
> +               struct rdma_cm_id *id, struct rdma_cm_event *event)
> +{
> +       struct smbd_connection *info = id->context;
> +
> +       log_rdma_event(INFO, "event=%d status=%d\n",
> +               event->event, event->status);
> +
> +       switch (event->event) {
> +       case RDMA_CM_EVENT_ADDR_RESOLVED:
> +       case RDMA_CM_EVENT_ROUTE_RESOLVED:
> +               info->ri_rc = 0;
> +               complete(&info->ri_done);
> +               break;
> +
> +       case RDMA_CM_EVENT_ADDR_ERROR:
> +               info->ri_rc = -EHOSTUNREACH;
> +               complete(&info->ri_done);
> +               break;
> +
> +       case RDMA_CM_EVENT_ROUTE_ERROR:
> +               info->ri_rc = -ENETUNREACH;
> +               complete(&info->ri_done);
> +               break;
> +
> +       case RDMA_CM_EVENT_ESTABLISHED:
> +               log_rdma_event(INFO, "connected event=%d\n", event->event);
> +               info->transport_status = SMBD_CONNECTED;
> +               wake_up_interruptible(&info->conn_wait);
> +               break;
> +
> +       case RDMA_CM_EVENT_CONNECT_ERROR:
> +       case RDMA_CM_EVENT_UNREACHABLE:
> +       case RDMA_CM_EVENT_REJECTED:
> +               log_rdma_event(INFO, "connecting failed event=%d\n", event->event);
> +               info->transport_status = SMBD_DISCONNECTED;
> +               wake_up_interruptible(&info->conn_wait);
> +               break;
> +
> +       case RDMA_CM_EVENT_DEVICE_REMOVAL:
> +       case RDMA_CM_EVENT_DISCONNECTED:
> +               /* This happenes when we fail the negotiation */
> +               if (info->transport_status == SMBD_NEGOTIATE_FAILED) {
> +                       info->transport_status = SMBD_DISCONNECTED;
> +                       wake_up(&info->conn_wait);
> +                       break;
> +               }
> +
> +               info->transport_status = SMBD_DISCONNECTED;
> +               smbd_process_disconnected(info);
> +               break;
> +
> +       default:
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
> +/* Upcall from RDMA QP */
> +static void
> +smbd_qp_async_error_upcall(struct ib_event *event, void *context)
> +{
> +       struct smbd_connection *info = context;
> +
> +       log_rdma_event(ERR, "%s on device %s info %p\n",
> +               ib_event_msg(event->event), event->device->name, info);
> +
> +       switch (event->event) {
> +       case IB_EVENT_CQ_ERR:
> +       case IB_EVENT_QP_FATAL:
> +               smbd_disconnect_rdma_connection(info);
> +
> +       default:
> +               break;
> +       }
> +}
> +
> +static inline void *smbd_request_payload(struct smbd_request *request)
> +{
> +       return (void *)request->packet;
> +}
> +
> +static inline void *smbd_response_payload(struct smbd_response *response)
> +{
> +       return (void *)response->packet;
> +}
> +
> +/* Called when a RDMA send is done */
> +static void send_done(struct ib_cq *cq, struct ib_wc *wc)
> +{
> +       int i;
> +       struct smbd_request *request =
> +               container_of(wc->wr_cqe, struct smbd_request, cqe);
> +
> +       log_rdma_send(INFO, "smbd_request %p completed wc->status=%d\n",
> +               request, wc->status);
> +
> +       if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_SEND) {
> +               log_rdma_send(ERR, "wc->status=%d wc->opcode=%d\n",
> +                       wc->status, wc->opcode);
> +               smbd_disconnect_rdma_connection(request->info);
> +       }
> +
> +       for (i = 0; i < request->num_sge; i++)
> +               ib_dma_unmap_single(request->info->id->device,
> +                       request->sge[i].addr,
> +                       request->sge[i].length,
> +                       DMA_TO_DEVICE);
> +
> +       if (request->has_payload) {
> +               if (atomic_dec_and_test(&request->info->send_payload_pending))
> +                       wake_up(&request->info->wait_send_payload_pending);
> +       } else {
> +               if (atomic_dec_and_test(&request->info->send_pending))
> +                       wake_up(&request->info->wait_send_pending);
> +       }
> +
> +       mempool_free(request, request->info->request_mempool);
> +}
> +
> +static void dump_smbd_negotiate_resp(struct smbd_negotiate_resp *resp)
> +{
> +       log_rdma_event(INFO, "resp message min_version %u max_version %u "
> +               "negotiated_version %u credits_requested %u "
> +               "credits_granted %u status %u max_readwrite_size %u "
> +               "preferred_send_size %u max_receive_size %u "
> +               "max_fragmented_size %u\n",
> +               resp->min_version, resp->max_version, resp->negotiated_version,
> +               resp->credits_requested, resp->credits_granted, resp->status,
> +               resp->max_readwrite_size, resp->preferred_send_size,
> +               resp->max_receive_size, resp->max_fragmented_size);
> +}
> +
> +/*
> + * Process a negotiation response message, according to [MS-SMBD]3.1.5.7
> + * response, packet_length: the negotiation response message
> + * return value: true if negotiation is a success, false if failed
> + */
> +static bool process_negotiation_response(
> +               struct smbd_response *response, int packet_length)
> +{
> +       struct smbd_connection *info = response->info;
> +       struct smbd_negotiate_resp *packet = smbd_response_payload(response);
> +
> +       if (packet_length < sizeof(struct smbd_negotiate_resp)) {
> +               log_rdma_event(ERR,
> +                       "error: packet_length=%d\n", packet_length);
> +               return false;
> +       }
> +
> +       if (le16_to_cpu(packet->negotiated_version) != SMBD_V1) {
> +               log_rdma_event(ERR, "error: negotiated_version=%x\n",
> +                       le16_to_cpu(packet->negotiated_version));
> +               return false;
> +       }
> +       info->protocol = le16_to_cpu(packet->negotiated_version);
> +
> +       if (packet->credits_requested == 0) {
> +               log_rdma_event(ERR, "error: credits_requested==0\n");
> +               return false;
> +       }
> +       info->receive_credit_target = le16_to_cpu(packet->credits_requested);
> +
> +       if (packet->credits_granted == 0) {
> +               log_rdma_event(ERR, "error: credits_granted==0\n");
> +               return false;
> +       }
> +       atomic_set(&info->send_credits, le16_to_cpu(packet->credits_granted));
> +
> +       atomic_set(&info->receive_credits, 0);
> +
> +       if (le32_to_cpu(packet->preferred_send_size) > info->max_receive_size) {
> +               log_rdma_event(ERR, "error: preferred_send_size=%d\n",
> +                       le32_to_cpu(packet->preferred_send_size));
> +               return false;
> +       }
> +       info->max_receive_size = le32_to_cpu(packet->preferred_send_size);
> +
> +       if (le32_to_cpu(packet->max_receive_size) < SMBD_MIN_RECEIVE_SIZE) {
> +               log_rdma_event(ERR, "error: max_receive_size=%d\n",
> +                       le32_to_cpu(packet->max_receive_size));
> +               return false;
> +       }
> +       info->max_send_size = min_t(int, info->max_send_size,
> +                                       le32_to_cpu(packet->max_receive_size));
> +
> +       if (le32_to_cpu(packet->max_fragmented_size) <
> +                       SMBD_MIN_FRAGMENTED_SIZE) {
> +               log_rdma_event(ERR, "error: max_fragmented_size=%d\n",
> +                       le32_to_cpu(packet->max_fragmented_size));
> +               return false;
> +       }
> +       info->max_fragmented_send_size =
> +               le32_to_cpu(packet->max_fragmented_size);
> +
> +       return true;
> +}
> +
> +/*
> + * Check and schedule to send an immediate packet
> + * This is used to extend credtis to remote peer to keep the transport busy
> + */
> +static void check_and_send_immediate(struct smbd_connection *info)
> +{
> +       if (info->transport_status != SMBD_CONNECTED)
> +               return;
> +
> +       info->send_immediate = true;
> +
> +       /*
> +        * Promptly send a packet if our peer is running low on receive
> +        * credits
> +        */
> +       if (atomic_read(&info->receive_credits) <
> +               info->receive_credit_target - 1)
> +               queue_delayed_work(
> +                       info->workqueue, &info->send_immediate_work, 0);
> +}
> +
> +static void smbd_post_send_credits(struct work_struct *work)
> +{
> +       int ret = 0;
> +       int use_receive_queue = 1;
> +       int rc;
> +       struct smbd_response *response;
> +       struct smbd_connection *info =
> +               container_of(work, struct smbd_connection,
> +                       post_send_credits_work);
> +
> +       if (info->transport_status != SMBD_CONNECTED) {
> +               wake_up(&info->wait_receive_queues);
> +               return;
> +       }
> +
> +       if (info->receive_credit_target >
> +               atomic_read(&info->receive_credits)) {
> +               while (true) {
> +                       if (use_receive_queue)
> +                               response = get_receive_buffer(info);
> +                       else
> +                               response = get_empty_queue_buffer(info);
> +                       if (!response) {
> +                               /* now switch to emtpy packet queue */
> +                               if (use_receive_queue) {
> +                                       use_receive_queue = 0;
> +                                       continue;
> +                               } else
> +                                       break;
> +                       }
> +
> +                       response->type = SMBD_TRANSFER_DATA;
> +                       response->first_segment = false;
> +                       rc = smbd_post_recv(info, response);
> +                       if (rc) {
> +                               log_rdma_recv(ERR,
> +                                       "post_recv failed rc=%d\n", rc);
> +                               put_receive_buffer(info, response, true);
> +                               break;
> +                       }
> +
> +                       ret++;
> +               }
> +       }
> +
> +       spin_lock(&info->lock_new_credits_offered);
> +       info->new_credits_offered += ret;
> +       spin_unlock(&info->lock_new_credits_offered);
> +
> +       atomic_add(ret, &info->receive_credits);
> +
> +       /* Check if we can post new receive and grant credits to peer */
> +       check_and_send_immediate(info);
> +}
> +
> +static void smbd_recv_done_work(struct work_struct *work)
> +{
> +       struct smbd_connection *info =
> +               container_of(work, struct smbd_connection, recv_done_work);
> +
> +       /*
> +        * We may have new send credits granted from remote peer
> +        * If any sender is blcoked on lack of credets, unblock it
> +        */
> +       if (atomic_read(&info->send_credits))
> +               wake_up_interruptible(&info->wait_send_queue);
> +
> +       /*
> +        * Check if we need to send something to remote peer to
> +        * grant more credits or respond to KEEP_ALIVE packet
> +        */
> +       check_and_send_immediate(info);
> +}
> +
> +/* Called from softirq, when recv is done */
> +static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> +{
> +       struct smbd_data_transfer *data_transfer;
> +       struct smbd_response *response =
> +               container_of(wc->wr_cqe, struct smbd_response, cqe);
> +       struct smbd_connection *info = response->info;
> +       int data_length = 0;
> +
> +       log_rdma_recv(INFO, "response=%p type=%d wc status=%d wc opcode %d "
> +                     "byte_len=%d pkey_index=%x\n",
> +               response, response->type, wc->status, wc->opcode,
> +               wc->byte_len, wc->pkey_index);
> +
> +       if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
> +               log_rdma_recv(INFO, "wc->status=%d opcode=%d\n",
> +                       wc->status, wc->opcode);
> +               smbd_disconnect_rdma_connection(info);
> +               goto error;
> +       }
> +
> +       ib_dma_sync_single_for_cpu(
> +               wc->qp->device,
> +               response->sge.addr,
> +               response->sge.length,
> +               DMA_FROM_DEVICE);
> +
> +       switch (response->type) {
> +       /* SMBD negotiation response */
> +       case SMBD_NEGOTIATE_RESP:
> +               dump_smbd_negotiate_resp(smbd_response_payload(response));
> +               info->full_packet_received = true;
> +               info->negotiate_done =
> +                       process_negotiation_response(response, wc->byte_len);
> +               complete(&info->negotiate_completion);
> +               break;
> +
> +       /* SMBD data transfer packet */
> +       case SMBD_TRANSFER_DATA:
> +               data_transfer = smbd_response_payload(response);
> +               data_length = le32_to_cpu(data_transfer->data_length);
> +
> +               /*
> +                * If this is a packet with data playload place the data in
> +                * reassembly queue and wake up the reading thread
> +                */
> +               if (data_length) {
> +                       if (info->full_packet_received)
> +                               response->first_segment = true;
> +
> +                       if (le32_to_cpu(data_transfer->remaining_data_length))
> +                               info->full_packet_received = false;
> +                       else
> +                               info->full_packet_received = true;
> +
> +                       enqueue_reassembly(
> +                               info,
> +                               response,
> +                               data_length);
> +               } else
> +                       put_empty_packet(info, response);
> +
> +               if (data_length)
> +                       wake_up_interruptible(&info->wait_reassembly_queue);
> +
> +               atomic_dec(&info->receive_credits);
> +               info->receive_credit_target =
> +                       le16_to_cpu(data_transfer->credits_requested);
> +               atomic_add(le16_to_cpu(data_transfer->credits_granted),
> +                       &info->send_credits);
> +
> +               log_incoming(INFO, "data flags %d data_offset %d "
> +                       "data_length %d remaining_data_length %d\n",
> +                       le16_to_cpu(data_transfer->flags),
> +                       le32_to_cpu(data_transfer->data_offset),
> +                       le32_to_cpu(data_transfer->data_length),
> +                       le32_to_cpu(data_transfer->remaining_data_length));
> +
> +               /* Send a KEEP_ALIVE response right away if requested */
> +               info->keep_alive_requested = KEEP_ALIVE_NONE;
> +               if (le16_to_cpu(data_transfer->flags) &
> +                               SMB_DIRECT_RESPONSE_REQUESTED) {
> +                       info->keep_alive_requested = KEEP_ALIVE_PENDING;
> +               }
> +
> +               queue_work(info->workqueue, &info->recv_done_work);
> +               return;
> +
> +       default:
> +               log_rdma_recv(ERR,
> +                       "unexpected response type=%d\n", response->type);
> +       }
> +
> +error:
> +       put_receive_buffer(info, response, true);
> +}
> +
> +static struct rdma_cm_id *smbd_create_id(
> +               struct smbd_connection *info,
> +               struct sockaddr *dstaddr, int port)
> +{
> +       struct rdma_cm_id *id;
> +       int rc;
> +       __be16 *sport;
> +
> +       id = rdma_create_id(&init_net, smbd_conn_upcall, info,
> +               RDMA_PS_TCP, IB_QPT_RC);
> +       if (IS_ERR(id)) {
> +               rc = PTR_ERR(id);
> +               log_rdma_event(ERR, "rdma_create_id() failed %i\n", rc);
> +               return id;
> +       }
> +
> +       if (dstaddr->sa_family == AF_INET6)
> +               sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port;
> +       else
> +               sport = &((struct sockaddr_in *)dstaddr)->sin_port;
> +
> +       *sport = htons(port);
> +
> +       init_completion(&info->ri_done);
> +       info->ri_rc = -ETIMEDOUT;
> +
> +       rc = rdma_resolve_addr(id, NULL, (struct sockaddr *)dstaddr,
> +               RDMA_RESOLVE_TIMEOUT);
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_resolve_addr() failed %i\n", rc);
> +               goto out;
> +       }
> +       wait_for_completion_interruptible_timeout(
> +               &info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
> +       rc = info->ri_rc;
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_resolve_addr() completed %i\n", rc);
> +               goto out;
> +       }
> +
> +       info->ri_rc = -ETIMEDOUT;
> +       rc = rdma_resolve_route(id, RDMA_RESOLVE_TIMEOUT);
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_resolve_route() failed %i\n", rc);
> +               goto out;
> +       }
> +       wait_for_completion_interruptible_timeout(
> +               &info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
> +       rc = info->ri_rc;
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_resolve_route() completed %i\n", rc);
> +               goto out;
> +       }
> +
> +       return id;
> +
> +out:
> +       rdma_destroy_id(id);
> +       return ERR_PTR(rc);
> +}
> +
> +/*
> + * Test if FRWR (Fast Registration Work Requests) is supported on the device
> + * This implementation requries FRWR on RDMA read/write
> + * return value: true if it is supported
> + */
> +static bool frwr_is_supported(struct ib_device_attr *attrs)
> +{
> +       if (!(attrs->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS))
> +               return false;
> +       if (attrs->max_fast_reg_page_list_len == 0)
> +               return false;
> +       return true;
> +}
> +
> +static int smbd_ia_open(
> +               struct smbd_connection *info,
> +               struct sockaddr *dstaddr, int port)
> +{
> +       int rc;
> +
> +       info->id = smbd_create_id(info, dstaddr, port);
> +       if (IS_ERR(info->id)) {
> +               rc = PTR_ERR(info->id);
> +               goto out1;
> +       }
> +
> +       if (!frwr_is_supported(&info->id->device->attrs)) {
> +               log_rdma_event(ERR,
> +                       "Fast Registration Work Requests "
> +                       "(FRWR) is not supported\n");
> +               log_rdma_event(ERR,
> +                       "Device capability flags = %llx "
> +                       "max_fast_reg_page_list_len = %u\n",
> +                       info->id->device->attrs.device_cap_flags,
> +                       info->id->device->attrs.max_fast_reg_page_list_len);
> +               rc = -EPROTONOSUPPORT;
> +               goto out2;
> +       }
> +
> +       info->pd = ib_alloc_pd(info->id->device, 0);
> +       if (IS_ERR(info->pd)) {
> +               rc = PTR_ERR(info->pd);
> +               log_rdma_event(ERR, "ib_alloc_pd() returned %d\n", rc);
> +               goto out2;
> +       }
> +
> +       return 0;
> +
> +out2:
> +       rdma_destroy_id(info->id);
> +       info->id = NULL;
> +
> +out1:
> +       return rc;
> +}
> +
> +/*
> + * Send a negotiation request message to the peer
> + * The negotiation procedure is in [MS-SMBD] 3.1.5.2 and 3.1.5.3
> + * After negotiation, the transport is connected and ready for
> + * carrying upper layer SMB payload
> + */
> +static int smbd_post_send_negotiate_req(struct smbd_connection *info)
> +{
> +       struct ib_send_wr send_wr, *send_wr_fail;
> +       int rc = -ENOMEM;
> +       struct smbd_request *request;
> +       struct smbd_negotiate_req *packet;
> +
> +       request = mempool_alloc(info->request_mempool, GFP_KERNEL);
> +       if (!request)
> +               return rc;
> +
> +       request->info = info;
> +
> +       packet = smbd_request_payload(request);
> +       packet->min_version = cpu_to_le16(SMBD_V1);
> +       packet->max_version = cpu_to_le16(SMBD_V1);
> +       packet->reserved = 0;
> +       packet->credits_requested = cpu_to_le16(info->send_credit_target);
> +       packet->preferred_send_size = cpu_to_le32(info->max_send_size);
> +       packet->max_receive_size = cpu_to_le32(info->max_receive_size);
> +       packet->max_fragmented_size =
> +               cpu_to_le32(info->max_fragmented_recv_size);
> +
> +       request->num_sge = 1;
> +       request->sge[0].addr = ib_dma_map_single(
> +                               info->id->device, (void *)packet,
> +                               sizeof(*packet), DMA_TO_DEVICE);
> +       if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
> +               rc = -EIO;
> +               goto dma_mapping_failed;
> +       }
> +
> +       request->sge[0].length = sizeof(*packet);
> +       request->sge[0].lkey = info->pd->local_dma_lkey;
> +
> +       ib_dma_sync_single_for_device(
> +               info->id->device, request->sge[0].addr,
> +               request->sge[0].length, DMA_TO_DEVICE);
> +
> +       request->cqe.done = send_done;
> +
> +       send_wr.next = NULL;
> +       send_wr.wr_cqe = &request->cqe;
> +       send_wr.sg_list = request->sge;
> +       send_wr.num_sge = request->num_sge;
> +       send_wr.opcode = IB_WR_SEND;
> +       send_wr.send_flags = IB_SEND_SIGNALED;
> +
> +       log_rdma_send(INFO, "sge addr=%llx length=%x lkey=%x\n",
> +               request->sge[0].addr,
> +               request->sge[0].length, request->sge[0].lkey);
> +
> +       request->has_payload = false;
> +       atomic_inc(&info->send_pending);
> +       rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
> +       if (!rc)
> +               return 0;
> +
> +       /* if we reach here, post send failed */
> +       log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
> +       atomic_dec(&info->send_pending);
> +       ib_dma_unmap_single(info->id->device, request->sge[0].addr,
> +               request->sge[0].length, DMA_TO_DEVICE);
> +
> +dma_mapping_failed:
> +       mempool_free(request, info->request_mempool);
> +       return rc;
> +}
> +
> +/*
> + * Extend the credits to remote peer
> + * This implements [MS-SMBD] 3.1.5.9
> + * The idea is that we should extend credits to remote peer as quickly as
> + * it's allowed, to maintain data flow. We allocate as much receive
> + * buffer as possible, and extend the receive credits to remote peer
> + * return value: the new credtis being granted.
> + */
> +static int manage_credits_prior_sending(struct smbd_connection *info)
> +{
> +       int new_credits;
> +
> +       spin_lock(&info->lock_new_credits_offered);
> +       new_credits = info->new_credits_offered;
> +       info->new_credits_offered = 0;
> +       spin_unlock(&info->lock_new_credits_offered);
> +
> +       return new_credits;
> +}
> +
> +/*
> + * Check if we need to send a KEEP_ALIVE message
> + * The idle connection timer triggers a KEEP_ALIVE message when expires
> + * SMB_DIRECT_RESPONSE_REQUESTED is set in the message flag to have peer send
> + * back a response.
> + * return value:
> + * 1 if SMB_DIRECT_RESPONSE_REQUESTED needs to be set
> + * 0: otherwise
> + */
> +static int manage_keep_alive_before_sending(struct smbd_connection *info)
> +{
> +       if (info->keep_alive_requested == KEEP_ALIVE_PENDING) {
> +               info->keep_alive_requested = KEEP_ALIVE_SENT;
> +               return 1;
> +       }
> +       return 0;
> +}
> +
> +/*
> + * Build and prepare the SMBD packet header
> + * This function waits for avaialbe send credits and build a SMBD packet
> + * header. The caller then optional append payload to the packet after
> + * the header
> + * intput values
> + * size: the size of the payload
> + * remaining_data_length: remaining data to send if this is part of a
> + * fragmented packet
> + * output values
> + * request_out: the request allocated from this function
> + * return values: 0 on success, otherwise actual error code returned
> + */
> +static int smbd_create_header(struct smbd_connection *info,
> +               int size, int remaining_data_length,
> +               struct smbd_request **request_out)
> +{
> +       struct smbd_request *request;
> +       struct smbd_data_transfer *packet;
> +       int header_length;
> +       int rc;
> +
> +       /* Wait for send credits. A SMBD packet needs one credit */
> +       rc = wait_event_interruptible(info->wait_send_queue,
> +               atomic_read(&info->send_credits) > 0 ||
> +               info->transport_status != SMBD_CONNECTED);
> +       if (rc)
> +               return rc;
> +
> +       if (info->transport_status != SMBD_CONNECTED) {
> +               log_outgoing(ERR, "disconnected not sending\n");
> +               return -ENOENT;
> +       }
> +       atomic_dec(&info->send_credits);
> +
> +       request = mempool_alloc(info->request_mempool, GFP_KERNEL);
> +       if (!request) {
> +               rc = -ENOMEM;
> +               goto err;
> +       }
> +
> +       request->info = info;
> +
> +       /* Fill in the packet header */
> +       packet = smbd_request_payload(request);
> +       packet->credits_requested = cpu_to_le16(info->send_credit_target);
> +       packet->credits_granted =
> +               cpu_to_le16(manage_credits_prior_sending(info));
> +       info->send_immediate = false;
> +
> +       packet->flags = 0;
> +       if (manage_keep_alive_before_sending(info))
> +               packet->flags |= cpu_to_le16(SMB_DIRECT_RESPONSE_REQUESTED);
> +
> +       packet->reserved = 0;
> +       if (!size)
> +               packet->data_offset = 0;
> +       else
> +               packet->data_offset = cpu_to_le32(24);
> +       packet->data_length = cpu_to_le32(size);
> +       packet->remaining_data_length = cpu_to_le32(remaining_data_length);
> +       packet->padding = 0;
> +
> +       log_outgoing(INFO, "credits_requested=%d credits_granted=%d "
> +               "data_offset=%d data_length=%d remaining_data_length=%d\n",
> +               le16_to_cpu(packet->credits_requested),
> +               le16_to_cpu(packet->credits_granted),
> +               le32_to_cpu(packet->data_offset),
> +               le32_to_cpu(packet->data_length),
> +               le32_to_cpu(packet->remaining_data_length));
> +
> +       /* Map the packet to DMA */
> +       header_length = sizeof(struct smbd_data_transfer);
> +       /* If this is a packet without payload, don't send padding */
> +       if (!size)
> +               header_length = offsetof(struct smbd_data_transfer, padding);
> +
> +       request->num_sge = 1;
> +       request->sge[0].addr = ib_dma_map_single(info->id->device,
> +                                                (void *)packet,
> +                                                header_length,
> +                                                DMA_BIDIRECTIONAL);
> +       if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
> +               mempool_free(request, info->request_mempool);
> +               rc = -EIO;
> +               goto err;
> +       }
> +
> +       request->sge[0].length = header_length;
> +       request->sge[0].lkey = info->pd->local_dma_lkey;
> +
> +       *request_out = request;
> +       return 0;
> +
> +err:
> +       atomic_inc(&info->send_credits);
> +       return rc;
> +}
> +
> +static void smbd_destroy_header(struct smbd_connection *info,
> +               struct smbd_request *request)
> +{
> +
> +       ib_dma_unmap_single(info->id->device,
> +                           request->sge[0].addr,
> +                           request->sge[0].length,
> +                           DMA_TO_DEVICE);
> +       mempool_free(request, info->request_mempool);
> +       atomic_inc(&info->send_credits);
> +}
> +
> +/* Post the send request */
> +static int smbd_post_send(struct smbd_connection *info,
> +               struct smbd_request *request, bool has_payload)
> +{
> +       struct ib_send_wr send_wr, *send_wr_fail;
> +       int rc, i;
> +
> +       for (i = 0; i < request->num_sge; i++) {
> +               log_rdma_send(INFO,
> +                       "rdma_request sge[%d] addr=%llu legnth=%u\n",
> +                       i, request->sge[0].addr, request->sge[0].length);
> +               ib_dma_sync_single_for_device(
> +                       info->id->device,
> +                       request->sge[i].addr,
> +                       request->sge[i].length,
> +                       DMA_TO_DEVICE);
> +       }
> +
> +       request->cqe.done = send_done;
> +
> +       send_wr.next = NULL;
> +       send_wr.wr_cqe = &request->cqe;
> +       send_wr.sg_list = request->sge;
> +       send_wr.num_sge = request->num_sge;
> +       send_wr.opcode = IB_WR_SEND;
> +       send_wr.send_flags = IB_SEND_SIGNALED;
> +
> +       if (has_payload) {
> +               request->has_payload = true;
> +               atomic_inc(&info->send_payload_pending);
> +       } else {
> +               request->has_payload = false;
> +               atomic_inc(&info->send_pending);
> +       }
> +
> +       rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
> +       if (rc) {
> +               log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
> +               if (has_payload) {
> +                       if (atomic_dec_and_test(&info->send_payload_pending))
> +                               wake_up(&info->wait_send_payload_pending);
> +               } else {
> +                       if (atomic_dec_and_test(&info->send_pending))
> +                               wake_up(&info->wait_send_pending);
> +               }
> +       } else
> +               /* Reset timer for idle connection after packet is sent */
> +               mod_delayed_work(info->workqueue, &info->idle_timer_work,
> +                       info->keep_alive_interval*HZ);
> +
> +       return rc;
> +}
> +
> +static int smbd_post_send_sgl(struct smbd_connection *info,
> +       struct scatterlist *sgl, int data_length, int remaining_data_length)
> +{
> +       int num_sgs;
> +       int i, rc;
> +       struct smbd_request *request;
> +       struct scatterlist *sg;
> +
> +       rc = smbd_create_header(
> +               info, data_length, remaining_data_length, &request);
> +       if (rc)
> +               return rc;
> +
> +       num_sgs = sgl ? sg_nents(sgl) : 0;
> +       for_each_sg(sgl, sg, num_sgs, i) {
> +               request->sge[i+1].addr =
> +                       ib_dma_map_page(info->id->device, sg_page(sg),
> +                              sg->offset, sg->length, DMA_BIDIRECTIONAL);
> +               if (ib_dma_mapping_error(
> +                               info->id->device, request->sge[i+1].addr)) {
> +                       rc = -EIO;
> +                       request->sge[i+1].addr = 0;
> +                       goto dma_mapping_failure;
> +               }
> +               request->sge[i+1].length = sg->length;
> +               request->sge[i+1].lkey = info->pd->local_dma_lkey;
> +               request->num_sge++;
> +       }
> +
> +       rc = smbd_post_send(info, request, data_length);
> +       if (!rc)
> +               return 0;
> +
> +dma_mapping_failure:
> +       for (i = 1; i < request->num_sge; i++)
> +               if (request->sge[i].addr)
> +                       ib_dma_unmap_single(info->id->device,
> +                                           request->sge[i].addr,
> +                                           request->sge[i].length,
> +                                           DMA_TO_DEVICE);
> +       smbd_destroy_header(info, request);
> +       return rc;
> +}
> +
> +/*
> + * Send an empty message
> + * Empty message is used to extend credits to peer to for keep live
> + * while there is no upper layer payload to send at the time
> + */
> +static int smbd_post_send_empty(struct smbd_connection *info)
> +{
> +       info->count_send_empty++;
> +       return smbd_post_send_sgl(info, NULL, 0, 0);
> +}
> +
> +/*
> + * Post a receive request to the transport
> + * The remote peer can only send data when a receive request is posted
> + * The interaction is controlled by send/receive credit system
> + */
> +static int smbd_post_recv(
> +               struct smbd_connection *info, struct smbd_response *response)
> +{
> +       struct ib_recv_wr recv_wr, *recv_wr_fail = NULL;
> +       int rc = -EIO;
> +
> +       response->sge.addr = ib_dma_map_single(
> +                               info->id->device, response->packet,
> +                               info->max_receive_size, DMA_FROM_DEVICE);
> +       if (ib_dma_mapping_error(info->id->device, response->sge.addr))
> +               return rc;
> +
> +       response->sge.length = info->max_receive_size;
> +       response->sge.lkey = info->pd->local_dma_lkey;
> +
> +       response->cqe.done = recv_done;
> +
> +       recv_wr.wr_cqe = &response->cqe;
> +       recv_wr.next = NULL;
> +       recv_wr.sg_list = &response->sge;
> +       recv_wr.num_sge = 1;
> +
> +       rc = ib_post_recv(info->id->qp, &recv_wr, &recv_wr_fail);
> +       if (rc) {
> +               ib_dma_unmap_single(info->id->device, response->sge.addr,
> +                                   response->sge.length, DMA_FROM_DEVICE);
> +
> +               log_rdma_recv(ERR, "ib_post_recv failed rc=%d\n", rc);
> +       }
> +
> +       return rc;
> +}
> +
> +/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */
> +static int smbd_negotiate(struct smbd_connection *info)
> +{
> +       int rc;
> +       struct smbd_response *response = get_receive_buffer(info);
> +
> +       response->type = SMBD_NEGOTIATE_RESP;
> +       rc = smbd_post_recv(info, response);
> +       log_rdma_event(INFO,
> +               "smbd_post_recv rc=%d iov.addr=%llx iov.length=%x "
> +               "iov.lkey=%x\n",
> +               rc, response->sge.addr,
> +               response->sge.length, response->sge.lkey);
> +       if (rc)
> +               return rc;
> +
> +       init_completion(&info->negotiate_completion);
> +       info->negotiate_done = false;
> +       rc = smbd_post_send_negotiate_req(info);
> +       if (rc)
> +               return rc;
> +
> +       rc = wait_for_completion_interruptible_timeout(
> +               &info->negotiate_completion, SMBD_NEGOTIATE_TIMEOUT * HZ);
> +       log_rdma_event(INFO, "wait_for_completion_timeout rc=%d\n", rc);
> +
> +       if (info->negotiate_done)
> +               return 0;
> +
> +       if (rc == 0)
> +               rc = -ETIMEDOUT;
> +       else if (rc == -ERESTARTSYS)
> +               rc = -EINTR;
> +       else
> +               rc = -ENOTCONN;
> +
> +       return rc;
> +}
> +
> +static void put_empty_packet(
> +               struct smbd_connection *info, struct smbd_response *response)
> +{
> +       spin_lock(&info->empty_packet_queue_lock);
> +       list_add_tail(&response->list, &info->empty_packet_queue);
> +       info->count_empty_packet_queue++;
> +       spin_unlock(&info->empty_packet_queue_lock);
> +
> +       queue_work(info->workqueue, &info->post_send_credits_work);
> +}
> +
> +/*
> + * Implement Connection.FragmentReassemblyBuffer defined in [MS-SMBD] 3.1.1.1
> + * This is a queue for reassembling upper layer payload and present to upper
> + * layer. All the inncoming payload go to the reassembly queue, regardless of
> + * if reassembly is required. The uuper layer code reads from the queue for all
> + * incoming payloads.
> + * Put a received packet to the reassembly queue
> + * response: the packet received
> + * data_length: the size of payload in this packet
> + */
> +static void enqueue_reassembly(
> +       struct smbd_connection *info,
> +       struct smbd_response *response,
> +       int data_length)
> +{
> +       spin_lock(&info->reassembly_queue_lock);
> +       list_add_tail(&response->list, &info->reassembly_queue);
> +       info->reassembly_queue_length++;
> +       /*
> +        * Make sure reassembly_data_length is updated after list and
> +        * reassembly_queue_length are updated. On the dequeue side
> +        * reassembly_data_length is checked without a lock to determine
> +        * if reassembly_queue_length and list is up to date
> +        */
> +       virt_wmb();
> +       info->reassembly_data_length += data_length;
> +       spin_unlock(&info->reassembly_queue_lock);
> +       info->count_reassembly_queue++;
> +       info->count_enqueue_reassembly_queue++;
> +}
> +
> +/*
> + * Get the first entry at the front of reassembly queue
> + * Caller is responsible for locking
> + * return value: the first entry if any, NULL if queue is empty
> + */
> +static struct smbd_response *_get_first_reassembly(struct smbd_connection *info)
> +{
> +       struct smbd_response *ret = NULL;
> +
> +       if (!list_empty(&info->reassembly_queue)) {
> +               ret = list_first_entry(
> +                       &info->reassembly_queue,
> +                       struct smbd_response, list);
> +       }
> +       return ret;
> +}
> +
> +static struct smbd_response *get_empty_queue_buffer(
> +               struct smbd_connection *info)
> +{
> +       struct smbd_response *ret = NULL;
> +       unsigned long flags;
> +
> +       spin_lock_irqsave(&info->empty_packet_queue_lock, flags);
> +       if (!list_empty(&info->empty_packet_queue)) {
> +               ret = list_first_entry(
> +                       &info->empty_packet_queue,
> +                       struct smbd_response, list);
> +               list_del(&ret->list);
> +               info->count_empty_packet_queue--;
> +       }
> +       spin_unlock_irqrestore(&info->empty_packet_queue_lock, flags);
> +
> +       return ret;
> +}
> +
> +/*
> + * Get a receive buffer
> + * For each remote send, we need to post a receive. The receive buffers are
> + * pre-allocated in advance.
> + * return value: the receive buffer, NULL if none is available
> + */
> +static struct smbd_response *get_receive_buffer(struct smbd_connection *info)
> +{
> +       struct smbd_response *ret = NULL;
> +       unsigned long flags;
> +
> +       spin_lock_irqsave(&info->receive_queue_lock, flags);
> +       if (!list_empty(&info->receive_queue)) {
> +               ret = list_first_entry(
> +                       &info->receive_queue,
> +                       struct smbd_response, list);
> +               list_del(&ret->list);
> +               info->count_receive_queue--;
> +               info->count_get_receive_buffer++;
> +       }
> +       spin_unlock_irqrestore(&info->receive_queue_lock, flags);
> +
> +       return ret;
> +}
> +
> +/*
> + * Return a receive buffer
> + * Upon returning of a receive buffer, we can post new receive and extend
> + * more receive credits to remote peer. This is done immediately after a
> + * receive buffer is returned.
> + */
> +static void put_receive_buffer(
> +       struct smbd_connection *info, struct smbd_response *response,
> +       bool lock)
> +{
> +       unsigned long flags;
> +
> +       ib_dma_unmap_single(info->id->device, response->sge.addr,
> +               response->sge.length, DMA_FROM_DEVICE);
> +
> +       if (lock)
> +               spin_lock_irqsave(&info->receive_queue_lock, flags);
> +       list_add_tail(&response->list, &info->receive_queue);
> +       info->count_receive_queue++;
> +       info->count_put_receive_buffer++;
> +       if (lock)
> +               spin_unlock_irqrestore(&info->receive_queue_lock, flags);
> +
> +       queue_work(info->workqueue, &info->post_send_credits_work);
> +}
> +
> +/* Preallocate all receive buffer on transport establishment */
> +static int allocate_receive_buffers(struct smbd_connection *info, int num_buf)
> +{
> +       int i;
> +       struct smbd_response *response;
> +
> +       INIT_LIST_HEAD(&info->reassembly_queue);
> +       spin_lock_init(&info->reassembly_queue_lock);
> +       info->reassembly_data_length = 0;
> +       info->reassembly_queue_length = 0;
> +
> +       INIT_LIST_HEAD(&info->receive_queue);
> +       spin_lock_init(&info->receive_queue_lock);
> +       info->count_receive_queue = 0;
> +
> +       INIT_LIST_HEAD(&info->empty_packet_queue);
> +       spin_lock_init(&info->empty_packet_queue_lock);
> +       info->count_empty_packet_queue = 0;
> +
> +       init_waitqueue_head(&info->wait_receive_queues);
> +
> +       for (i = 0; i < num_buf; i++) {
> +               response = mempool_alloc(info->response_mempool, GFP_KERNEL);
> +               if (!response)
> +                       goto allocate_failed;
> +
> +               response->info = info;
> +               list_add_tail(&response->list, &info->receive_queue);
> +               info->count_receive_queue++;
> +       }
> +
> +       return 0;
> +
> +allocate_failed:
> +       while (!list_empty(&info->receive_queue)) {
> +               response = list_first_entry(
> +                               &info->receive_queue,
> +                               struct smbd_response, list);
> +               list_del(&response->list);
> +               info->count_receive_queue--;
> +
> +               mempool_free(response, info->response_mempool);
> +       }
> +       return -ENOMEM;
> +}
> +
> +static void destroy_receive_buffers(struct smbd_connection *info)
> +{
> +       struct smbd_response *response;
> +
> +       while ((response = get_receive_buffer(info)))
> +               mempool_free(response, info->response_mempool);
> +
> +       while ((response = get_empty_queue_buffer(info)))
> +               mempool_free(response, info->response_mempool);
> +}
> +
> +/*
> + * Check and send an immediate or keep alive packet
> + * The condition to send those packets are defined in [MS-SMBD] 3.1.1.1
> + * Connection.KeepaliveRequested and Connection.SendImmediate
> + * The idea is to extend credits to server as soon as it becomes available
> + */
> +static void send_immediate_work(struct work_struct *work)
> +{
> +       struct smbd_connection *info = container_of(
> +                                       work, struct smbd_connection,
> +                                       send_immediate_work.work);
> +
> +       if (info->keep_alive_requested == KEEP_ALIVE_PENDING ||
> +           info->send_immediate) {
> +               log_keep_alive(INFO, "send an empty message\n");
> +               smbd_post_send_empty(info);
> +       }
> +}
> +
> +/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */
> +static void idle_connection_timer(struct work_struct *work)
> +{
> +       struct smbd_connection *info = container_of(
> +                                       work, struct smbd_connection,
> +                                       idle_timer_work.work);
> +
> +       if (info->keep_alive_requested != KEEP_ALIVE_NONE) {
> +               log_keep_alive(ERR,
> +                       "error status info->keep_alive_requested=%d\n",
> +                       info->keep_alive_requested);
> +               smbd_disconnect_rdma_connection(info);
> +               return;
> +       }
> +
> +       log_keep_alive(INFO, "about to send an empty idle message\n");
> +       smbd_post_send_empty(info);
> +
> +       /* Setup the next idle timeout work */
> +       queue_delayed_work(info->workqueue, &info->idle_timer_work,
> +                       info->keep_alive_interval*HZ);
> +}
> +
> +static void destroy_caches_and_workqueue(struct smbd_connection *info)
> +{
> +       destroy_receive_buffers(info);
> +       destroy_workqueue(info->workqueue);
> +       mempool_destroy(info->response_mempool);
> +       kmem_cache_destroy(info->response_cache);
> +       mempool_destroy(info->request_mempool);
> +       kmem_cache_destroy(info->request_cache);
> +}
> +
> +#define MAX_NAME_LEN   80
> +static int allocate_caches_and_workqueue(struct smbd_connection *info)
> +{
> +       char name[MAX_NAME_LEN];
> +       int rc;
> +
> +       snprintf(name, MAX_NAME_LEN, "smbd_request_%p", info);
> +       info->request_cache =
> +               kmem_cache_create(
> +                       name,
> +                       sizeof(struct smbd_request) +
> +                               sizeof(struct smbd_data_transfer),
> +                       0, SLAB_HWCACHE_ALIGN, NULL);
> +       if (!info->request_cache)
> +               return -ENOMEM;
> +
> +       info->request_mempool =
> +               mempool_create(info->send_credit_target, mempool_alloc_slab,
> +                       mempool_free_slab, info->request_cache);
> +       if (!info->request_mempool)
> +               goto out1;
> +
> +       snprintf(name, MAX_NAME_LEN, "smbd_response_%p", info);
> +       info->response_cache =
> +               kmem_cache_create(
> +                       name,
> +                       sizeof(struct smbd_response) +
> +                               info->max_receive_size,
> +                       0, SLAB_HWCACHE_ALIGN, NULL);
> +       if (!info->response_cache)
> +               goto out2;
> +
> +       info->response_mempool =
> +               mempool_create(info->receive_credit_max, mempool_alloc_slab,
> +                      mempool_free_slab, info->response_cache);
> +       if (!info->response_mempool)
> +               goto out3;
> +
> +       snprintf(name, MAX_NAME_LEN, "smbd_%p", info);
> +       info->workqueue = create_workqueue(name);
> +       if (!info->workqueue)
> +               goto out4;
> +
> +       rc = allocate_receive_buffers(info, info->receive_credit_max);
> +       if (rc) {
> +               log_rdma_event(ERR, "failed to allocate receive buffers\n");
> +               goto out5;
> +       }
> +
> +       return 0;
> +
> +out5:
> +       destroy_workqueue(info->workqueue);
> +out4:
> +       mempool_destroy(info->response_mempool);
> +out3:
> +       kmem_cache_destroy(info->response_cache);
> +out2:
> +       mempool_destroy(info->request_mempool);
> +out1:
> +       kmem_cache_destroy(info->request_cache);
> +       return -ENOMEM;
> +}
> +
> +/* Create a SMBD connection, called by upper layer */
> +struct smbd_connection *_smbd_get_connection(
> +       struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
> +{
> +       int rc;
> +       struct smbd_connection *info;
> +       struct rdma_conn_param conn_param;
> +       struct ib_qp_init_attr qp_attr;
> +       struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;
> +
> +       info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL);
> +       if (!info)
> +               return NULL;
> +
> +       info->transport_status = SMBD_CONNECTING;
> +       rc = smbd_ia_open(info, dstaddr, port);
> +       if (rc) {
> +               log_rdma_event(INFO, "smbd_ia_open rc=%d\n", rc);
> +               goto create_id_failed;
> +       }
> +
> +       if (smbd_send_credit_target > info->id->device->attrs.max_cqe ||
> +           smbd_send_credit_target > info->id->device->attrs.max_qp_wr) {
> +               log_rdma_event(ERR,
> +                       "consider lowering send_credit_target = %d. "
> +                       "Possible CQE overrun, device "
> +                       "reporting max_cpe %d max_qp_wr %d\n",
> +                       smbd_send_credit_target,
> +                       info->id->device->attrs.max_cqe,
> +                       info->id->device->attrs.max_qp_wr);
> +               goto config_failed;
> +       }
> +
> +       if (smbd_receive_credit_max > info->id->device->attrs.max_cqe ||
> +           smbd_receive_credit_max > info->id->device->attrs.max_qp_wr) {
> +               log_rdma_event(ERR,
> +                       "consider lowering receive_credit_max = %d. "
> +                       "Possible CQE overrun, device "
> +                       "reporting max_cpe %d max_qp_wr %d\n",
> +                       smbd_receive_credit_max,
> +                       info->id->device->attrs.max_cqe,
> +                       info->id->device->attrs.max_qp_wr);
> +               goto config_failed;
> +       }
> +
> +       info->receive_credit_max = smbd_receive_credit_max;
> +       info->send_credit_target = smbd_send_credit_target;
> +       info->max_send_size = smbd_max_send_size;
> +       info->max_fragmented_recv_size = smbd_max_fragmented_recv_size;
> +       info->max_receive_size = smbd_max_receive_size;
> +       info->keep_alive_interval = smbd_keep_alive_interval;
> +
> +       if (SMBDIRECT_MAX_SGE > info->id->device->attrs.max_sge) {
> +               log_rdma_event(ERR, "warning: device max_sge = %d too small\n",
> +                       info->id->device->attrs.max_sge);
> +               log_rdma_event(ERR, "Queue Pair creation may fail\n");
> +       }
> +
> +       info->send_cq = NULL;
> +       info->recv_cq = NULL;
> +       info->send_cq = ib_alloc_cq(info->id->device, info,
> +                       info->send_credit_target, 0, IB_POLL_SOFTIRQ);
> +       if (IS_ERR(info->send_cq)) {
> +               info->send_cq = NULL;
> +               goto alloc_cq_failed;
> +       }
> +
> +       info->recv_cq = ib_alloc_cq(info->id->device, info,
> +                       info->receive_credit_max, 0, IB_POLL_SOFTIRQ);
> +       if (IS_ERR(info->recv_cq)) {
> +               info->recv_cq = NULL;
> +               goto alloc_cq_failed;
> +       }
> +
> +       memset(&qp_attr, 0, sizeof(qp_attr));
> +       qp_attr.event_handler = smbd_qp_async_error_upcall;
> +       qp_attr.qp_context = info;
> +       qp_attr.cap.max_send_wr = info->send_credit_target;
> +       qp_attr.cap.max_recv_wr = info->receive_credit_max;
> +       qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
> +       qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
> +       qp_attr.cap.max_inline_data = 0;
> +       qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
> +       qp_attr.qp_type = IB_QPT_RC;
> +       qp_attr.send_cq = info->send_cq;
> +       qp_attr.recv_cq = info->recv_cq;
> +       qp_attr.port_num = ~0;
> +
> +       rc = rdma_create_qp(info->id, info->pd, &qp_attr);
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_create_qp failed %i\n", rc);
> +               goto create_qp_failed;
> +       }
> +
> +       memset(&conn_param, 0, sizeof(conn_param));
> +       conn_param.initiator_depth = 0;
> +
> +       conn_param.retry_count = SMBD_CM_RETRY;
> +       conn_param.rnr_retry_count = SMBD_CM_RNR_RETRY;
> +       conn_param.flow_control = 0;
> +       init_waitqueue_head(&info->wait_destroy);
> +
> +       log_rdma_event(INFO, "connecting to IP %pI4 port %d\n",
> +               &addr_in->sin_addr, port);
> +
> +       init_waitqueue_head(&info->conn_wait);
> +       rc = rdma_connect(info->id, &conn_param);
> +       if (rc) {
> +               log_rdma_event(ERR, "rdma_connect() failed with %i\n", rc);
> +               goto rdma_connect_failed;
> +       }
> +
> +       wait_event_interruptible(
> +               info->conn_wait, info->transport_status != SMBD_CONNECTING);
> +
> +       if (info->transport_status != SMBD_CONNECTED) {
> +               log_rdma_event(ERR, "rdma_connect failed port=%d\n", port);
> +               goto rdma_connect_failed;
> +       }
> +
> +       log_rdma_event(INFO, "rdma_connect connected\n");
> +
> +       rc = allocate_caches_and_workqueue(info);
> +       if (rc) {
> +               log_rdma_event(ERR, "cache allocation failed\n");
> +               goto allocate_cache_failed;
> +       }
> +
> +       init_waitqueue_head(&info->wait_send_queue);
> +       init_waitqueue_head(&info->wait_reassembly_queue);
> +
> +       INIT_DELAYED_WORK(&info->idle_timer_work, idle_connection_timer);
> +       INIT_DELAYED_WORK(&info->send_immediate_work, send_immediate_work);
> +       queue_delayed_work(info->workqueue, &info->idle_timer_work,
> +               info->keep_alive_interval*HZ);
> +
> +       init_waitqueue_head(&info->wait_send_pending);
> +       atomic_set(&info->send_pending, 0);
> +
> +       init_waitqueue_head(&info->wait_send_payload_pending);
> +       atomic_set(&info->send_payload_pending, 0);
> +
> +       INIT_WORK(&info->disconnect_work, smbd_disconnect_rdma_work);
> +       INIT_WORK(&info->destroy_work, smbd_destroy_rdma_work);
> +       INIT_WORK(&info->recv_done_work, smbd_recv_done_work);
> +       INIT_WORK(&info->post_send_credits_work, smbd_post_send_credits);
> +       info->new_credits_offered = 0;
> +       spin_lock_init(&info->lock_new_credits_offered);
> +
> +       rc = smbd_negotiate(info);
> +       if (rc) {
> +               log_rdma_event(ERR, "smbd_negotiate rc=%d\n", rc);
> +               goto negotiation_failed;
> +       }
> +
> +       return info;
> +
> +negotiation_failed:
> +       cancel_delayed_work_sync(&info->idle_timer_work);
> +       destroy_caches_and_workqueue(info);
> +       info->transport_status = SMBD_NEGOTIATE_FAILED;
> +       init_waitqueue_head(&info->conn_wait);
> +       rdma_disconnect(info->id);
> +       wait_event(info->conn_wait,
> +               info->transport_status == SMBD_DISCONNECTED);
> +
> +allocate_cache_failed:
> +rdma_connect_failed:
> +       rdma_destroy_qp(info->id);
> +
> +create_qp_failed:
> +alloc_cq_failed:
> +       if (info->send_cq)
> +               ib_free_cq(info->send_cq);
> +       if (info->recv_cq)
> +               ib_free_cq(info->recv_cq);
> +
> +config_failed:
> +       ib_dealloc_pd(info->pd);
> +       rdma_destroy_id(info->id);
> +
> +create_id_failed:
> +       kfree(info);
> +       return NULL;
> +}
> diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
> index c55f28b..35bc25b 100644
> --- a/fs/cifs/smbdirect.h
> +++ b/fs/cifs/smbdirect.h
> @@ -16,6 +16,286 @@
>  #ifndef _SMBDIRECT_H
>  #define _SMBDIRECT_H
>
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +#define cifs_rdma_enabled(server)      ((server)->rdma)
> +
> +#include "cifsglob.h"
> +#include <rdma/ib_verbs.h>
> +#include <rdma/rdma_cm.h>
> +#include <linux/mempool.h>
> +
> +enum keep_alive_status {
> +       KEEP_ALIVE_NONE,
> +       KEEP_ALIVE_PENDING,
> +       KEEP_ALIVE_SENT,
> +};
> +
> +enum smbd_connection_status {
> +       SMBD_CREATED,
> +       SMBD_CONNECTING,
> +       SMBD_CONNECTED,
> +       SMBD_NEGOTIATE_FAILED,
> +       SMBD_DISCONNECTING,
> +       SMBD_DISCONNECTED,
> +       SMBD_DESTROYED
> +};
> +
> +/*
> + * The context for the SMBDirect transport
> + * Everything related to the transport is here. It has several logical parts
> + * 1. RDMA related structures
> + * 2. SMBDirect connection parameters
> + * 3. Memory registrations
> + * 4. Receive and reassembly queues for data receive path
> + * 5. mempools for allocating packets
> + */
> +struct smbd_connection {
> +       enum smbd_connection_status transport_status;
> +
> +       /* RDMA related */
> +       struct rdma_cm_id *id;
> +       struct ib_qp_init_attr qp_attr;
> +       struct ib_pd *pd;
> +       struct ib_cq *send_cq, *recv_cq;
> +       struct ib_device_attr dev_attr;
> +       int ri_rc;
> +       struct completion ri_done;
> +       wait_queue_head_t conn_wait;
> +       wait_queue_head_t wait_destroy;
> +
> +       struct completion negotiate_completion;
> +       bool negotiate_done;
> +
> +       struct work_struct destroy_work;
> +       struct work_struct disconnect_work;
> +       struct work_struct recv_done_work;
> +       struct work_struct post_send_credits_work;
> +
> +       spinlock_t lock_new_credits_offered;
> +       int new_credits_offered;
> +
> +       /* Connection parameters defined in [MS-SMBD] 3.1.1.1 */
> +       int receive_credit_max;
> +       int send_credit_target;
> +       int max_send_size;
> +       int max_fragmented_recv_size;
> +       int max_fragmented_send_size;
> +       int max_receive_size;
> +       int keep_alive_interval;
> +       int max_readwrite_size;
> +       enum keep_alive_status keep_alive_requested;
> +       int protocol;
> +       atomic_t send_credits;
> +       atomic_t receive_credits;
> +       int receive_credit_target;
> +       int fragment_reassembly_remaining;
> +
> +       /* Activity accoutning */
> +
> +       atomic_t send_pending;
> +       wait_queue_head_t wait_send_pending;
> +       atomic_t send_payload_pending;
> +       wait_queue_head_t wait_send_payload_pending;
> +
> +       /* Receive queue */
> +       struct list_head receive_queue;
> +       int count_receive_queue;
> +       spinlock_t receive_queue_lock;
> +
> +       struct list_head empty_packet_queue;
> +       int count_empty_packet_queue;
> +       spinlock_t empty_packet_queue_lock;
> +
> +       wait_queue_head_t wait_receive_queues;
> +
> +       /* Reassembly queue */
> +       struct list_head reassembly_queue;
> +       spinlock_t reassembly_queue_lock;
> +       wait_queue_head_t wait_reassembly_queue;
> +
> +       /* total data length of reassembly queue */
> +       int reassembly_data_length;
> +       int reassembly_queue_length;
> +       /* the offset to first buffer in reassembly queue */
> +       int first_entry_offset;
> +
> +       bool send_immediate;
> +
> +       wait_queue_head_t wait_send_queue;
> +
> +       /*
> +        * Indicate if we have received a full packet on the connection
> +        * This is used to identify the first SMBD packet of a assembled
> +        * payload (SMB packet) in reassembly queue so we can return a
> +        * RFC1002 length to upper layer to indicate the length of the SMB
> +        * packet received
> +        */
> +       bool full_packet_received;
> +
> +       struct workqueue_struct *workqueue;
> +       struct delayed_work idle_timer_work;
> +       struct delayed_work send_immediate_work;
> +
> +       /* Memory pool for preallocating buffers */
> +       /* request pool for RDMA send */
> +       struct kmem_cache *request_cache;
> +       mempool_t *request_mempool;
> +
> +       /* response pool for RDMA receive */
> +       struct kmem_cache *response_cache;
> +       mempool_t *response_mempool;
> +
> +       /* for debug purposes */
> +       unsigned int count_get_receive_buffer;
> +       unsigned int count_put_receive_buffer;
> +       unsigned int count_reassembly_queue;
> +       unsigned int count_enqueue_reassembly_queue;
> +       unsigned int count_dequeue_reassembly_queue;
> +       unsigned int count_send_empty;
> +};
> +
> +enum smbd_message_type {
> +       SMBD_NEGOTIATE_RESP,
> +       SMBD_TRANSFER_DATA,
> +};
> +
> +#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
> +
> +/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */
> +struct smbd_negotiate_req {
> +       __le16 min_version;
> +       __le16 max_version;
> +       __le16 reserved;
> +       __le16 credits_requested;
> +       __le32 preferred_send_size;
> +       __le32 max_receive_size;
> +       __le32 max_fragmented_size;
> +} __packed;
> +
> +/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */
> +struct smbd_negotiate_resp {
> +       __le16 min_version;
> +       __le16 max_version;
> +       __le16 negotiated_version;
> +       __le16 reserved;
> +       __le16 credits_requested;
> +       __le16 credits_granted;
> +       __le32 status;
> +       __le32 max_readwrite_size;
> +       __le32 preferred_send_size;
> +       __le32 max_receive_size;
> +       __le32 max_fragmented_size;
> +} __packed;
> +
> +/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */
> +struct smbd_data_transfer {
> +       __le16 credits_requested;
> +       __le16 credits_granted;
> +       __le16 flags;
> +       __le16 reserved;
> +       __le32 remaining_data_length;
> +       __le32 data_offset;
> +       __le32 data_length;
> +       __le32 padding;
> +       __u8 buffer[];
> +} __packed;
> +
> +/* The packet fields for a registered RDMA buffer */
> +struct smbd_buffer_descriptor_v1 {
> +       __le64 offset;
> +       __le32 token;
> +       __le32 length;
> +} __packed;
> +
>  /* Default maximum number of SGEs in a RDMA send/recv */
>  #define SMBDIRECT_MAX_SGE      16
> +/* The context for a SMBD request */
> +struct smbd_request {
> +       struct smbd_connection *info;
> +       struct ib_cqe cqe;
> +
> +       /* true if this request carries upper layer payload */
> +       bool has_payload;
> +
> +       /* the SGE entries for this packet */
> +       struct ib_sge sge[SMBDIRECT_MAX_SGE];
> +       int num_sge;
> +
> +       /* SMBD packet header follows this structure */
> +       u8 packet[];
> +};
> +
> +/* The context for a SMBD response */
> +struct smbd_response {
> +       struct smbd_connection *info;
> +       struct ib_cqe cqe;
> +       struct ib_sge sge;
> +
> +       enum smbd_message_type type;
> +
> +       /* Link to receive queue or reassembly queue */
> +       struct list_head list;
> +
> +       /* Indicate if this is the 1st packet of a payload */
> +       bool first_segment;
> +
> +       /* SMBD packet header and payload follows this structure */
> +       u8 packet[];
> +};
> +
> +/* Create a SMBDirect session */
> +struct smbd_connection *smbd_get_connection(
> +       struct TCP_Server_Info *server, struct sockaddr *dstaddr);
> +
> +/* Reconnect SMBDirect session */
> +int smbd_reconnect(struct TCP_Server_Info *server);
> +
> +/* Destroy SMBDirect session */
> +void smbd_destroy(struct smbd_connection *info);
> +
> +/* Interface for carrying upper layer I/O through send/recv */
> +int smbd_recv(struct smbd_connection *info, struct msghdr *msg);
> +int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst);
> +
> +enum mr_state {
> +       MR_READY,
> +       MR_REGISTERED,
> +       MR_INVALIDATED,
> +       MR_ERROR
> +};
> +
> +struct smbd_mr {
> +       struct smbd_connection  *conn;
> +       struct list_head        list;
> +       enum mr_state           state;
> +       struct ib_mr            *mr;
> +       struct scatterlist      *sgl;
> +       int                     sgl_count;
> +       enum dma_data_direction dir;
> +       union {
> +               struct ib_reg_wr        wr;
> +               struct ib_send_wr       inv_wr;
> +       };
> +       struct ib_cqe           cqe;
> +       bool                    need_invalidate;
> +       struct completion       invalidate_done;
> +};
> +
> +/* Interfaces to register and deregister MR for RDMA read/write */
> +struct smbd_mr *smbd_register_mr(
> +       struct smbd_connection *info, struct page *pages[], int num_pages,
> +       int tailsz, bool writing, bool need_invalidate);
> +int smbd_deregister_mr(struct smbd_mr *mr);
> +
> +#else
> +#define cifs_rdma_enabled(server)      0
> +struct smbd_connection{};
> +static inline void *smbd_get_connection(
> +       struct TCP_Server_Info *server, struct sockaddr *dstaddr) {return NULL;}
> +static inline int smbd_reconnect(struct TCP_Server_Info *server) {return -1;}
> +static inline void smbd_destroy(struct smbd_connection *info) {}
> +static inline int smbd_recv(struct smbd_connection *info, struct msghdr *msg) {return -1;}
> +static inline int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst) {return -1;}
> +#endif
> +
>  #endif
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

But two tiny nits:
+ * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
Should be 0xa0

+//	queue_work(info->workqueue, &info->destroy_work);
Don't leave commented out code in there.


----- Original Message -----
From: "Long Li" <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>
To: "Steve French" <sfrench-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>, linux-cifs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, samba-technical-w/Ol4Ecudpl8XjKLYN78aQ@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "Christoph Hellwig" <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, "Tom Talpey" <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Matthew Wilcox" <mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Stephen Hemminger" <sthemmin-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Cc: "Long Li" <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Sent: Tuesday, 7 November, 2017 7:54:57 PM
Subject: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

Add code to implement the core functions to establish a SMB Direct connection.

1. Establish an RDMA connection to SMB server.
2. Negotiate and setup SMB Direct protocol.
3. Implement idle connection timer and credit management.

SMB Direct is enabled by setting CONFIG_CIFS_SMB_DIRECT.

Add to Makefile to enable building SMB Direct.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/Makefile    |    2 +
 fs/cifs/smbdirect.c | 1576 +++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h |  280 +++++++++
 3 files changed, 1858 insertions(+)

diff --git a/fs/cifs/Makefile b/fs/cifs/Makefile
index 5e853a3..ad00873 100644
--- a/fs/cifs/Makefile
+++ b/fs/cifs/Makefile
@@ -18,3 +18,5 @@ cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
 cifs-$(CONFIG_CIFS_DFS_UPCALL) += dns_resolve.o cifs_dfs_ref.o
 
 cifs-$(CONFIG_CIFS_FSCACHE) += fscache.o cache.o
+
+cifs-$(CONFIG_CIFS_SMB_DIRECT) += smbdirect.o
diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index d3c16f8..021d527 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -13,7 +13,34 @@
  *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
  *   the GNU General Public License for more details.
  */
+#include <linux/module.h>
 #include "smbdirect.h"
+#include "cifs_debug.h"
+
+static struct smbd_response *get_empty_queue_buffer(
+		struct smbd_connection *info);
+static struct smbd_response *get_receive_buffer(
+		struct smbd_connection *info);
+static void put_receive_buffer(
+		struct smbd_connection *info,
+		struct smbd_response *response,
+		bool lock);
+static int allocate_receive_buffers(struct smbd_connection *info, int num_buf);
+static void destroy_receive_buffers(struct smbd_connection *info);
+
+static void put_empty_packet(
+		struct smbd_connection *info, struct smbd_response *response);
+static void enqueue_reassembly(
+		struct smbd_connection *info,
+		struct smbd_response *response, int data_length);
+static struct smbd_response *_get_first_reassembly(
+		struct smbd_connection *info);
+
+static int smbd_post_recv(
+		struct smbd_connection *info,
+		struct smbd_response *response);
+
+static int smbd_post_send_empty(struct smbd_connection *info);
 
 /* SMBD version number */
 #define SMBD_V1	0x0100
@@ -75,3 +102,1552 @@ int smbd_max_frmr_depth = 2048;
 
 /* If payload is less than this byte, use RDMA send/recv not read/write */
 int rdma_readwrite_threshold = 4096;
+
+/* Transport logging functions
+ * Logging are defined as classes. They can be OR'ed to define the actual
+ * logging level via module parameter smbd_logging_class
+ * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
+ * log_rdma_event()
+ */
+#define LOG_OUTGOING			0x1
+#define LOG_INCOMING			0x2
+#define LOG_READ			0x4
+#define LOG_WRITE			0x8
+#define LOG_RDMA_SEND			0x10
+#define LOG_RDMA_RECV			0x20
+#define LOG_KEEP_ALIVE			0x40
+#define LOG_RDMA_EVENT			0x80
+#define LOG_RDMA_MR			0x100
+static unsigned int smbd_logging_class = 0;
+module_param(smbd_logging_class, uint, 0644);
+MODULE_PARM_DESC(smbd_logging_class,
+	"Logging class for SMBD transport 0x0 to 0x100");
+
+#define ERR		0x0
+#define INFO		0x1
+static unsigned int smbd_logging_level = ERR;
+module_param(smbd_logging_level, uint, 0644);
+MODULE_PARM_DESC(smbd_logging_level,
+	"Logging level for SMBD transport, 0 (default): error, 1: info");
+
+#define log_rdma(level, class, fmt, args...)				\
+do {									\
+	if (level <= smbd_logging_level || class & smbd_logging_class)	\
+		cifs_dbg(VFS, "%s:%d " fmt, __func__, __LINE__, ##args);\
+} while (0)
+
+#define log_outgoing(level, fmt, args...) \
+		log_rdma(level, LOG_OUTGOING, fmt, ##args)
+#define log_incoming(level, fmt, args...) \
+		log_rdma(level, LOG_INCOMING, fmt, ##args)
+#define log_read(level, fmt, args...)	log_rdma(level, LOG_READ, fmt, ##args)
+#define log_write(level, fmt, args...)	log_rdma(level, LOG_WRITE, fmt, ##args)
+#define log_rdma_send(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_SEND, fmt, ##args)
+#define log_rdma_recv(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_RECV, fmt, ##args)
+#define log_keep_alive(level, fmt, args...) \
+		log_rdma(level, LOG_KEEP_ALIVE, fmt, ##args)
+#define log_rdma_event(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_EVENT, fmt, ##args)
+#define log_rdma_mr(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_MR, fmt, ##args)
+
+/*
+ * Destroy the transport and related RDMA and memory resources
+ * Need to go through all the pending counters and make sure on one is using
+ * the transport while it is destroyed
+ */
+static void smbd_destroy_rdma_work(struct work_struct *work)
+{
+	struct smbd_response *response;
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, destroy_work);
+	unsigned long flags;
+
+	log_rdma_event(INFO, "destroying qp\n");
+	ib_drain_qp(info->id->qp);
+	rdma_destroy_qp(info->id);
+
+	/* Unblock all I/O waiting on the send queue */
+	wake_up_interruptible_all(&info->wait_send_queue);
+
+	log_rdma_event(INFO, "cancelling idle timer\n");
+	cancel_delayed_work_sync(&info->idle_timer_work);
+	log_rdma_event(INFO, "cancelling send immediate work\n");
+	cancel_delayed_work_sync(&info->send_immediate_work);
+
+	log_rdma_event(INFO, "wait for all recv to finish\n");
+	wake_up_interruptible(&info->wait_reassembly_queue);
+
+	log_rdma_event(INFO, "wait for all send posted to IB to finish\n");
+	wait_event(info->wait_send_pending,
+		atomic_read(&info->send_pending) == 0);
+	wait_event(info->wait_send_payload_pending,
+		atomic_read(&info->send_payload_pending) == 0);
+
+	/* It's not posssible for upper layer to get to reassembly */
+	log_rdma_event(INFO, "drain the reassembly queue\n");
+	do {
+		spin_lock_irqsave(&info->reassembly_queue_lock, flags);
+		response = _get_first_reassembly(info);
+		if (response) {
+			list_del(&response->list);
+			spin_unlock_irqrestore(
+				&info->reassembly_queue_lock, flags);
+			put_receive_buffer(info, response, true);
+		}
+	} while (response);
+	spin_unlock_irqrestore(&info->reassembly_queue_lock, flags);
+	info->reassembly_data_length = 0;
+
+	log_rdma_event(INFO, "free receive buffers\n");
+	wait_event(info->wait_receive_queues,
+		info->count_receive_queue + info->count_empty_packet_queue
+			== info->receive_credit_max);
+	destroy_receive_buffers(info);
+
+	ib_free_cq(info->send_cq);
+	ib_free_cq(info->recv_cq);
+	ib_dealloc_pd(info->pd);
+	rdma_destroy_id(info->id);
+
+	/* free mempools */
+	mempool_destroy(info->request_mempool);
+	kmem_cache_destroy(info->request_cache);
+
+	mempool_destroy(info->response_mempool);
+	kmem_cache_destroy(info->response_cache);
+
+	info->transport_status = SMBD_DESTROYED;
+	wake_up_all(&info->wait_destroy);
+}
+
+static int smbd_process_disconnected(struct smbd_connection *info)
+{
+//	queue_work(info->workqueue, &info->destroy_work);
+	schedule_work(&info->destroy_work);
+	return 0;
+}
+
+static void smbd_disconnect_rdma_work(struct work_struct *work)
+{
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, disconnect_work);
+
+	if (info->transport_status == SMBD_CONNECTED) {
+		info->transport_status = SMBD_DISCONNECTING;
+		rdma_disconnect(info->id);
+	}
+}
+
+static void smbd_disconnect_rdma_connection(struct smbd_connection *info)
+{
+	queue_work(info->workqueue, &info->disconnect_work);
+}
+
+/* Upcall from RDMA CM */
+static int smbd_conn_upcall(
+		struct rdma_cm_id *id, struct rdma_cm_event *event)
+{
+	struct smbd_connection *info = id->context;
+
+	log_rdma_event(INFO, "event=%d status=%d\n",
+		event->event, event->status);
+
+	switch (event->event) {
+	case RDMA_CM_EVENT_ADDR_RESOLVED:
+	case RDMA_CM_EVENT_ROUTE_RESOLVED:
+		info->ri_rc = 0;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ADDR_ERROR:
+		info->ri_rc = -EHOSTUNREACH;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ROUTE_ERROR:
+		info->ri_rc = -ENETUNREACH;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ESTABLISHED:
+		log_rdma_event(INFO, "connected event=%d\n", event->event);
+		info->transport_status = SMBD_CONNECTED;
+		wake_up_interruptible(&info->conn_wait);
+		break;
+
+	case RDMA_CM_EVENT_CONNECT_ERROR:
+	case RDMA_CM_EVENT_UNREACHABLE:
+	case RDMA_CM_EVENT_REJECTED:
+		log_rdma_event(INFO, "connecting failed event=%d\n", event->event);
+		info->transport_status = SMBD_DISCONNECTED;
+		wake_up_interruptible(&info->conn_wait);
+		break;
+
+	case RDMA_CM_EVENT_DEVICE_REMOVAL:
+	case RDMA_CM_EVENT_DISCONNECTED:
+		/* This happenes when we fail the negotiation */
+		if (info->transport_status == SMBD_NEGOTIATE_FAILED) {
+			info->transport_status = SMBD_DISCONNECTED;
+			wake_up(&info->conn_wait);
+			break;
+		}
+
+		info->transport_status = SMBD_DISCONNECTED;
+		smbd_process_disconnected(info);
+		break;
+
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+/* Upcall from RDMA QP */
+static void
+smbd_qp_async_error_upcall(struct ib_event *event, void *context)
+{
+	struct smbd_connection *info = context;
+
+	log_rdma_event(ERR, "%s on device %s info %p\n",
+		ib_event_msg(event->event), event->device->name, info);
+
+	switch (event->event) {
+	case IB_EVENT_CQ_ERR:
+	case IB_EVENT_QP_FATAL:
+		smbd_disconnect_rdma_connection(info);
+
+	default:
+		break;
+	}
+}
+
+static inline void *smbd_request_payload(struct smbd_request *request)
+{
+	return (void *)request->packet;
+}
+
+static inline void *smbd_response_payload(struct smbd_response *response)
+{
+	return (void *)response->packet;
+}
+
+/* Called when a RDMA send is done */
+static void send_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+	int i;
+	struct smbd_request *request =
+		container_of(wc->wr_cqe, struct smbd_request, cqe);
+
+	log_rdma_send(INFO, "smbd_request %p completed wc->status=%d\n",
+		request, wc->status);
+
+	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_SEND) {
+		log_rdma_send(ERR, "wc->status=%d wc->opcode=%d\n",
+			wc->status, wc->opcode);
+		smbd_disconnect_rdma_connection(request->info);
+	}
+
+	for (i = 0; i < request->num_sge; i++)
+		ib_dma_unmap_single(request->info->id->device,
+			request->sge[i].addr,
+			request->sge[i].length,
+			DMA_TO_DEVICE);
+
+	if (request->has_payload) {
+		if (atomic_dec_and_test(&request->info->send_payload_pending))
+			wake_up(&request->info->wait_send_payload_pending);
+	} else {
+		if (atomic_dec_and_test(&request->info->send_pending))
+			wake_up(&request->info->wait_send_pending);
+	}
+
+	mempool_free(request, request->info->request_mempool);
+}
+
+static void dump_smbd_negotiate_resp(struct smbd_negotiate_resp *resp)
+{
+	log_rdma_event(INFO, "resp message min_version %u max_version %u "
+		"negotiated_version %u credits_requested %u "
+		"credits_granted %u status %u max_readwrite_size %u "
+		"preferred_send_size %u max_receive_size %u "
+		"max_fragmented_size %u\n",
+		resp->min_version, resp->max_version, resp->negotiated_version,
+		resp->credits_requested, resp->credits_granted, resp->status,
+		resp->max_readwrite_size, resp->preferred_send_size,
+		resp->max_receive_size, resp->max_fragmented_size);
+}
+
+/*
+ * Process a negotiation response message, according to [MS-SMBD]3.1.5.7
+ * response, packet_length: the negotiation response message
+ * return value: true if negotiation is a success, false if failed
+ */
+static bool process_negotiation_response(
+		struct smbd_response *response, int packet_length)
+{
+	struct smbd_connection *info = response->info;
+	struct smbd_negotiate_resp *packet = smbd_response_payload(response);
+
+	if (packet_length < sizeof(struct smbd_negotiate_resp)) {
+		log_rdma_event(ERR,
+			"error: packet_length=%d\n", packet_length);
+		return false;
+	}
+
+	if (le16_to_cpu(packet->negotiated_version) != SMBD_V1) {
+		log_rdma_event(ERR, "error: negotiated_version=%x\n",
+			le16_to_cpu(packet->negotiated_version));
+		return false;
+	}
+	info->protocol = le16_to_cpu(packet->negotiated_version);
+
+	if (packet->credits_requested == 0) {
+		log_rdma_event(ERR, "error: credits_requested==0\n");
+		return false;
+	}
+	info->receive_credit_target = le16_to_cpu(packet->credits_requested);
+
+	if (packet->credits_granted == 0) {
+		log_rdma_event(ERR, "error: credits_granted==0\n");
+		return false;
+	}
+	atomic_set(&info->send_credits, le16_to_cpu(packet->credits_granted));
+
+	atomic_set(&info->receive_credits, 0);
+
+	if (le32_to_cpu(packet->preferred_send_size) > info->max_receive_size) {
+		log_rdma_event(ERR, "error: preferred_send_size=%d\n",
+			le32_to_cpu(packet->preferred_send_size));
+		return false;
+	}
+	info->max_receive_size = le32_to_cpu(packet->preferred_send_size);
+
+	if (le32_to_cpu(packet->max_receive_size) < SMBD_MIN_RECEIVE_SIZE) {
+		log_rdma_event(ERR, "error: max_receive_size=%d\n",
+			le32_to_cpu(packet->max_receive_size));
+		return false;
+	}
+	info->max_send_size = min_t(int, info->max_send_size,
+					le32_to_cpu(packet->max_receive_size));
+
+	if (le32_to_cpu(packet->max_fragmented_size) <
+			SMBD_MIN_FRAGMENTED_SIZE) {
+		log_rdma_event(ERR, "error: max_fragmented_size=%d\n",
+			le32_to_cpu(packet->max_fragmented_size));
+		return false;
+	}
+	info->max_fragmented_send_size =
+		le32_to_cpu(packet->max_fragmented_size);
+
+	return true;
+}
+
+/*
+ * Check and schedule to send an immediate packet
+ * This is used to extend credtis to remote peer to keep the transport busy
+ */
+static void check_and_send_immediate(struct smbd_connection *info)
+{
+	if (info->transport_status != SMBD_CONNECTED)
+		return;
+
+	info->send_immediate = true;
+
+	/*
+	 * Promptly send a packet if our peer is running low on receive
+	 * credits
+	 */
+	if (atomic_read(&info->receive_credits) <
+		info->receive_credit_target - 1)
+		queue_delayed_work(
+			info->workqueue, &info->send_immediate_work, 0);
+}
+
+static void smbd_post_send_credits(struct work_struct *work)
+{
+	int ret = 0;
+	int use_receive_queue = 1;
+	int rc;
+	struct smbd_response *response;
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection,
+			post_send_credits_work);
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		wake_up(&info->wait_receive_queues);
+		return;
+	}
+
+	if (info->receive_credit_target >
+		atomic_read(&info->receive_credits)) {
+		while (true) {
+			if (use_receive_queue)
+				response = get_receive_buffer(info);
+			else
+				response = get_empty_queue_buffer(info);
+			if (!response) {
+				/* now switch to emtpy packet queue */
+				if (use_receive_queue) {
+					use_receive_queue = 0;
+					continue;
+				} else
+					break;
+			}
+
+			response->type = SMBD_TRANSFER_DATA;
+			response->first_segment = false;
+			rc = smbd_post_recv(info, response);
+			if (rc) {
+				log_rdma_recv(ERR,
+					"post_recv failed rc=%d\n", rc);
+				put_receive_buffer(info, response, true);
+				break;
+			}
+
+			ret++;
+		}
+	}
+
+	spin_lock(&info->lock_new_credits_offered);
+	info->new_credits_offered += ret;
+	spin_unlock(&info->lock_new_credits_offered);
+
+	atomic_add(ret, &info->receive_credits);
+
+	/* Check if we can post new receive and grant credits to peer */
+	check_and_send_immediate(info);
+}
+
+static void smbd_recv_done_work(struct work_struct *work)
+{
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, recv_done_work);
+
+	/*
+	 * We may have new send credits granted from remote peer
+	 * If any sender is blcoked on lack of credets, unblock it
+	 */
+	if (atomic_read(&info->send_credits))
+		wake_up_interruptible(&info->wait_send_queue);
+
+	/*
+	 * Check if we need to send something to remote peer to
+	 * grant more credits or respond to KEEP_ALIVE packet
+	 */
+	check_and_send_immediate(info);
+}
+
+/* Called from softirq, when recv is done */
+static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+	struct smbd_data_transfer *data_transfer;
+	struct smbd_response *response =
+		container_of(wc->wr_cqe, struct smbd_response, cqe);
+	struct smbd_connection *info = response->info;
+	int data_length = 0;
+
+	log_rdma_recv(INFO, "response=%p type=%d wc status=%d wc opcode %d "
+		      "byte_len=%d pkey_index=%x\n",
+		response, response->type, wc->status, wc->opcode,
+		wc->byte_len, wc->pkey_index);
+
+	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
+		log_rdma_recv(INFO, "wc->status=%d opcode=%d\n",
+			wc->status, wc->opcode);
+		smbd_disconnect_rdma_connection(info);
+		goto error;
+	}
+
+	ib_dma_sync_single_for_cpu(
+		wc->qp->device,
+		response->sge.addr,
+		response->sge.length,
+		DMA_FROM_DEVICE);
+
+	switch (response->type) {
+	/* SMBD negotiation response */
+	case SMBD_NEGOTIATE_RESP:
+		dump_smbd_negotiate_resp(smbd_response_payload(response));
+		info->full_packet_received = true;
+		info->negotiate_done =
+			process_negotiation_response(response, wc->byte_len);
+		complete(&info->negotiate_completion);
+		break;
+
+	/* SMBD data transfer packet */
+	case SMBD_TRANSFER_DATA:
+		data_transfer = smbd_response_payload(response);
+		data_length = le32_to_cpu(data_transfer->data_length);
+
+		/*
+		 * If this is a packet with data playload place the data in
+		 * reassembly queue and wake up the reading thread
+		 */
+		if (data_length) {
+			if (info->full_packet_received)
+				response->first_segment = true;
+
+			if (le32_to_cpu(data_transfer->remaining_data_length))
+				info->full_packet_received = false;
+			else
+				info->full_packet_received = true;
+
+			enqueue_reassembly(
+				info,
+				response,
+				data_length);
+		} else
+			put_empty_packet(info, response);
+
+		if (data_length)
+			wake_up_interruptible(&info->wait_reassembly_queue);
+
+		atomic_dec(&info->receive_credits);
+		info->receive_credit_target =
+			le16_to_cpu(data_transfer->credits_requested);
+		atomic_add(le16_to_cpu(data_transfer->credits_granted),
+			&info->send_credits);
+
+		log_incoming(INFO, "data flags %d data_offset %d "
+			"data_length %d remaining_data_length %d\n",
+			le16_to_cpu(data_transfer->flags),
+			le32_to_cpu(data_transfer->data_offset),
+			le32_to_cpu(data_transfer->data_length),
+			le32_to_cpu(data_transfer->remaining_data_length));
+
+		/* Send a KEEP_ALIVE response right away if requested */
+		info->keep_alive_requested = KEEP_ALIVE_NONE;
+		if (le16_to_cpu(data_transfer->flags) &
+				SMB_DIRECT_RESPONSE_REQUESTED) {
+			info->keep_alive_requested = KEEP_ALIVE_PENDING;
+		}
+
+		queue_work(info->workqueue, &info->recv_done_work);
+		return;
+
+	default:
+		log_rdma_recv(ERR,
+			"unexpected response type=%d\n", response->type);
+	}
+
+error:
+	put_receive_buffer(info, response, true);
+}
+
+static struct rdma_cm_id *smbd_create_id(
+		struct smbd_connection *info,
+		struct sockaddr *dstaddr, int port)
+{
+	struct rdma_cm_id *id;
+	int rc;
+	__be16 *sport;
+
+	id = rdma_create_id(&init_net, smbd_conn_upcall, info,
+		RDMA_PS_TCP, IB_QPT_RC);
+	if (IS_ERR(id)) {
+		rc = PTR_ERR(id);
+		log_rdma_event(ERR, "rdma_create_id() failed %i\n", rc);
+		return id;
+	}
+
+	if (dstaddr->sa_family == AF_INET6)
+		sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port;
+	else
+		sport = &((struct sockaddr_in *)dstaddr)->sin_port;
+
+	*sport = htons(port);
+
+	init_completion(&info->ri_done);
+	info->ri_rc = -ETIMEDOUT;
+
+	rc = rdma_resolve_addr(id, NULL, (struct sockaddr *)dstaddr,
+		RDMA_RESOLVE_TIMEOUT);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_addr() failed %i\n", rc);
+		goto out;
+	}
+	wait_for_completion_interruptible_timeout(
+		&info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
+	rc = info->ri_rc;
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_addr() completed %i\n", rc);
+		goto out;
+	}
+
+	info->ri_rc = -ETIMEDOUT;
+	rc = rdma_resolve_route(id, RDMA_RESOLVE_TIMEOUT);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_route() failed %i\n", rc);
+		goto out;
+	}
+	wait_for_completion_interruptible_timeout(
+		&info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
+	rc = info->ri_rc;
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_route() completed %i\n", rc);
+		goto out;
+	}
+
+	return id;
+
+out:
+	rdma_destroy_id(id);
+	return ERR_PTR(rc);
+}
+
+/*
+ * Test if FRWR (Fast Registration Work Requests) is supported on the device
+ * This implementation requries FRWR on RDMA read/write
+ * return value: true if it is supported
+ */
+static bool frwr_is_supported(struct ib_device_attr *attrs)
+{
+	if (!(attrs->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS))
+		return false;
+	if (attrs->max_fast_reg_page_list_len == 0)
+		return false;
+	return true;
+}
+
+static int smbd_ia_open(
+		struct smbd_connection *info,
+		struct sockaddr *dstaddr, int port)
+{
+	int rc;
+
+	info->id = smbd_create_id(info, dstaddr, port);
+	if (IS_ERR(info->id)) {
+		rc = PTR_ERR(info->id);
+		goto out1;
+	}
+
+	if (!frwr_is_supported(&info->id->device->attrs)) {
+		log_rdma_event(ERR,
+			"Fast Registration Work Requests "
+			"(FRWR) is not supported\n");
+		log_rdma_event(ERR,
+			"Device capability flags = %llx "
+			"max_fast_reg_page_list_len = %u\n",
+			info->id->device->attrs.device_cap_flags,
+			info->id->device->attrs.max_fast_reg_page_list_len);
+		rc = -EPROTONOSUPPORT;
+		goto out2;
+	}
+
+	info->pd = ib_alloc_pd(info->id->device, 0);
+	if (IS_ERR(info->pd)) {
+		rc = PTR_ERR(info->pd);
+		log_rdma_event(ERR, "ib_alloc_pd() returned %d\n", rc);
+		goto out2;
+	}
+
+	return 0;
+
+out2:
+	rdma_destroy_id(info->id);
+	info->id = NULL;
+
+out1:
+	return rc;
+}
+
+/*
+ * Send a negotiation request message to the peer
+ * The negotiation procedure is in [MS-SMBD] 3.1.5.2 and 3.1.5.3
+ * After negotiation, the transport is connected and ready for
+ * carrying upper layer SMB payload
+ */
+static int smbd_post_send_negotiate_req(struct smbd_connection *info)
+{
+	struct ib_send_wr send_wr, *send_wr_fail;
+	int rc = -ENOMEM;
+	struct smbd_request *request;
+	struct smbd_negotiate_req *packet;
+
+	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
+	if (!request)
+		return rc;
+
+	request->info = info;
+
+	packet = smbd_request_payload(request);
+	packet->min_version = cpu_to_le16(SMBD_V1);
+	packet->max_version = cpu_to_le16(SMBD_V1);
+	packet->reserved = 0;
+	packet->credits_requested = cpu_to_le16(info->send_credit_target);
+	packet->preferred_send_size = cpu_to_le32(info->max_send_size);
+	packet->max_receive_size = cpu_to_le32(info->max_receive_size);
+	packet->max_fragmented_size =
+		cpu_to_le32(info->max_fragmented_recv_size);
+
+	request->num_sge = 1;
+	request->sge[0].addr = ib_dma_map_single(
+				info->id->device, (void *)packet,
+				sizeof(*packet), DMA_TO_DEVICE);
+	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
+		rc = -EIO;
+		goto dma_mapping_failed;
+	}
+
+	request->sge[0].length = sizeof(*packet);
+	request->sge[0].lkey = info->pd->local_dma_lkey;
+
+	ib_dma_sync_single_for_device(
+		info->id->device, request->sge[0].addr,
+		request->sge[0].length, DMA_TO_DEVICE);
+
+	request->cqe.done = send_done;
+
+	send_wr.next = NULL;
+	send_wr.wr_cqe = &request->cqe;
+	send_wr.sg_list = request->sge;
+	send_wr.num_sge = request->num_sge;
+	send_wr.opcode = IB_WR_SEND;
+	send_wr.send_flags = IB_SEND_SIGNALED;
+
+	log_rdma_send(INFO, "sge addr=%llx length=%x lkey=%x\n",
+		request->sge[0].addr,
+		request->sge[0].length, request->sge[0].lkey);
+
+	request->has_payload = false;
+	atomic_inc(&info->send_pending);
+	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
+	if (!rc)
+		return 0;
+
+	/* if we reach here, post send failed */
+	log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
+	atomic_dec(&info->send_pending);
+	ib_dma_unmap_single(info->id->device, request->sge[0].addr,
+		request->sge[0].length, DMA_TO_DEVICE);
+
+dma_mapping_failed:
+	mempool_free(request, info->request_mempool);
+	return rc;
+}
+
+/*
+ * Extend the credits to remote peer
+ * This implements [MS-SMBD] 3.1.5.9
+ * The idea is that we should extend credits to remote peer as quickly as
+ * it's allowed, to maintain data flow. We allocate as much receive
+ * buffer as possible, and extend the receive credits to remote peer
+ * return value: the new credtis being granted.
+ */
+static int manage_credits_prior_sending(struct smbd_connection *info)
+{
+	int new_credits;
+
+	spin_lock(&info->lock_new_credits_offered);
+	new_credits = info->new_credits_offered;
+	info->new_credits_offered = 0;
+	spin_unlock(&info->lock_new_credits_offered);
+
+	return new_credits;
+}
+
+/*
+ * Check if we need to send a KEEP_ALIVE message
+ * The idle connection timer triggers a KEEP_ALIVE message when expires
+ * SMB_DIRECT_RESPONSE_REQUESTED is set in the message flag to have peer send
+ * back a response.
+ * return value:
+ * 1 if SMB_DIRECT_RESPONSE_REQUESTED needs to be set
+ * 0: otherwise
+ */
+static int manage_keep_alive_before_sending(struct smbd_connection *info)
+{
+	if (info->keep_alive_requested == KEEP_ALIVE_PENDING) {
+		info->keep_alive_requested = KEEP_ALIVE_SENT;
+		return 1;
+	}
+	return 0;
+}
+
+/*
+ * Build and prepare the SMBD packet header
+ * This function waits for avaialbe send credits and build a SMBD packet
+ * header. The caller then optional append payload to the packet after
+ * the header
+ * intput values
+ * size: the size of the payload
+ * remaining_data_length: remaining data to send if this is part of a
+ * fragmented packet
+ * output values
+ * request_out: the request allocated from this function
+ * return values: 0 on success, otherwise actual error code returned
+ */
+static int smbd_create_header(struct smbd_connection *info,
+		int size, int remaining_data_length,
+		struct smbd_request **request_out)
+{
+	struct smbd_request *request;
+	struct smbd_data_transfer *packet;
+	int header_length;
+	int rc;
+
+	/* Wait for send credits. A SMBD packet needs one credit */
+	rc = wait_event_interruptible(info->wait_send_queue,
+		atomic_read(&info->send_credits) > 0 ||
+		info->transport_status != SMBD_CONNECTED);
+	if (rc)
+		return rc;
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		log_outgoing(ERR, "disconnected not sending\n");
+		return -ENOENT;
+	}
+	atomic_dec(&info->send_credits);
+
+	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
+	if (!request) {
+		rc = -ENOMEM;
+		goto err;
+	}
+
+	request->info = info;
+
+	/* Fill in the packet header */
+	packet = smbd_request_payload(request);
+	packet->credits_requested = cpu_to_le16(info->send_credit_target);
+	packet->credits_granted =
+		cpu_to_le16(manage_credits_prior_sending(info));
+	info->send_immediate = false;
+
+	packet->flags = 0;
+	if (manage_keep_alive_before_sending(info))
+		packet->flags |= cpu_to_le16(SMB_DIRECT_RESPONSE_REQUESTED);
+
+	packet->reserved = 0;
+	if (!size)
+		packet->data_offset = 0;
+	else
+		packet->data_offset = cpu_to_le32(24);
+	packet->data_length = cpu_to_le32(size);
+	packet->remaining_data_length = cpu_to_le32(remaining_data_length);
+	packet->padding = 0;
+
+	log_outgoing(INFO, "credits_requested=%d credits_granted=%d "
+		"data_offset=%d data_length=%d remaining_data_length=%d\n",
+		le16_to_cpu(packet->credits_requested),
+		le16_to_cpu(packet->credits_granted),
+		le32_to_cpu(packet->data_offset),
+		le32_to_cpu(packet->data_length),
+		le32_to_cpu(packet->remaining_data_length));
+
+	/* Map the packet to DMA */
+	header_length = sizeof(struct smbd_data_transfer);
+	/* If this is a packet without payload, don't send padding */
+	if (!size)
+		header_length = offsetof(struct smbd_data_transfer, padding);
+
+	request->num_sge = 1;
+	request->sge[0].addr = ib_dma_map_single(info->id->device,
+						 (void *)packet,
+						 header_length,
+						 DMA_BIDIRECTIONAL);
+	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
+		mempool_free(request, info->request_mempool);
+		rc = -EIO;
+		goto err;
+	}
+
+	request->sge[0].length = header_length;
+	request->sge[0].lkey = info->pd->local_dma_lkey;
+
+	*request_out = request;
+	return 0;
+
+err:
+	atomic_inc(&info->send_credits);
+	return rc;
+}
+
+static void smbd_destroy_header(struct smbd_connection *info,
+		struct smbd_request *request)
+{
+
+	ib_dma_unmap_single(info->id->device,
+			    request->sge[0].addr,
+			    request->sge[0].length,
+			    DMA_TO_DEVICE);
+	mempool_free(request, info->request_mempool);
+	atomic_inc(&info->send_credits);
+}
+
+/* Post the send request */
+static int smbd_post_send(struct smbd_connection *info,
+		struct smbd_request *request, bool has_payload)
+{
+	struct ib_send_wr send_wr, *send_wr_fail;
+	int rc, i;
+
+	for (i = 0; i < request->num_sge; i++) {
+		log_rdma_send(INFO,
+			"rdma_request sge[%d] addr=%llu legnth=%u\n",
+			i, request->sge[0].addr, request->sge[0].length);
+		ib_dma_sync_single_for_device(
+			info->id->device,
+			request->sge[i].addr,
+			request->sge[i].length,
+			DMA_TO_DEVICE);
+	}
+
+	request->cqe.done = send_done;
+
+	send_wr.next = NULL;
+	send_wr.wr_cqe = &request->cqe;
+	send_wr.sg_list = request->sge;
+	send_wr.num_sge = request->num_sge;
+	send_wr.opcode = IB_WR_SEND;
+	send_wr.send_flags = IB_SEND_SIGNALED;
+
+	if (has_payload) {
+		request->has_payload = true;
+		atomic_inc(&info->send_payload_pending);
+	} else {
+		request->has_payload = false;
+		atomic_inc(&info->send_pending);
+	}
+
+	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
+	if (rc) {
+		log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
+		if (has_payload) {
+			if (atomic_dec_and_test(&info->send_payload_pending))
+				wake_up(&info->wait_send_payload_pending);
+		} else {
+			if (atomic_dec_and_test(&info->send_pending))
+				wake_up(&info->wait_send_pending);
+		}
+	} else
+		/* Reset timer for idle connection after packet is sent */
+		mod_delayed_work(info->workqueue, &info->idle_timer_work,
+			info->keep_alive_interval*HZ);
+
+	return rc;
+}
+
+static int smbd_post_send_sgl(struct smbd_connection *info,
+	struct scatterlist *sgl, int data_length, int remaining_data_length)
+{
+	int num_sgs;
+	int i, rc;
+	struct smbd_request *request;
+	struct scatterlist *sg;
+
+	rc = smbd_create_header(
+		info, data_length, remaining_data_length, &request);
+	if (rc)
+		return rc;
+
+	num_sgs = sgl ? sg_nents(sgl) : 0;
+	for_each_sg(sgl, sg, num_sgs, i) {
+		request->sge[i+1].addr =
+			ib_dma_map_page(info->id->device, sg_page(sg),
+			       sg->offset, sg->length, DMA_BIDIRECTIONAL);
+		if (ib_dma_mapping_error(
+				info->id->device, request->sge[i+1].addr)) {
+			rc = -EIO;
+			request->sge[i+1].addr = 0;
+			goto dma_mapping_failure;
+		}
+		request->sge[i+1].length = sg->length;
+		request->sge[i+1].lkey = info->pd->local_dma_lkey;
+		request->num_sge++;
+	}
+
+	rc = smbd_post_send(info, request, data_length);
+	if (!rc)
+		return 0;
+
+dma_mapping_failure:
+	for (i = 1; i < request->num_sge; i++)
+		if (request->sge[i].addr)
+			ib_dma_unmap_single(info->id->device,
+					    request->sge[i].addr,
+					    request->sge[i].length,
+					    DMA_TO_DEVICE);
+	smbd_destroy_header(info, request);
+	return rc;
+}
+
+/*
+ * Send an empty message
+ * Empty message is used to extend credits to peer to for keep live
+ * while there is no upper layer payload to send at the time
+ */
+static int smbd_post_send_empty(struct smbd_connection *info)
+{
+	info->count_send_empty++;
+	return smbd_post_send_sgl(info, NULL, 0, 0);
+}
+
+/*
+ * Post a receive request to the transport
+ * The remote peer can only send data when a receive request is posted
+ * The interaction is controlled by send/receive credit system
+ */
+static int smbd_post_recv(
+		struct smbd_connection *info, struct smbd_response *response)
+{
+	struct ib_recv_wr recv_wr, *recv_wr_fail = NULL;
+	int rc = -EIO;
+
+	response->sge.addr = ib_dma_map_single(
+				info->id->device, response->packet,
+				info->max_receive_size, DMA_FROM_DEVICE);
+	if (ib_dma_mapping_error(info->id->device, response->sge.addr))
+		return rc;
+
+	response->sge.length = info->max_receive_size;
+	response->sge.lkey = info->pd->local_dma_lkey;
+
+	response->cqe.done = recv_done;
+
+	recv_wr.wr_cqe = &response->cqe;
+	recv_wr.next = NULL;
+	recv_wr.sg_list = &response->sge;
+	recv_wr.num_sge = 1;
+
+	rc = ib_post_recv(info->id->qp, &recv_wr, &recv_wr_fail);
+	if (rc) {
+		ib_dma_unmap_single(info->id->device, response->sge.addr,
+				    response->sge.length, DMA_FROM_DEVICE);
+
+		log_rdma_recv(ERR, "ib_post_recv failed rc=%d\n", rc);
+	}
+
+	return rc;
+}
+
+/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */
+static int smbd_negotiate(struct smbd_connection *info)
+{
+	int rc;
+	struct smbd_response *response = get_receive_buffer(info);
+
+	response->type = SMBD_NEGOTIATE_RESP;
+	rc = smbd_post_recv(info, response);
+	log_rdma_event(INFO,
+		"smbd_post_recv rc=%d iov.addr=%llx iov.length=%x "
+		"iov.lkey=%x\n",
+		rc, response->sge.addr,
+		response->sge.length, response->sge.lkey);
+	if (rc)
+		return rc;
+
+	init_completion(&info->negotiate_completion);
+	info->negotiate_done = false;
+	rc = smbd_post_send_negotiate_req(info);
+	if (rc)
+		return rc;
+
+	rc = wait_for_completion_interruptible_timeout(
+		&info->negotiate_completion, SMBD_NEGOTIATE_TIMEOUT * HZ);
+	log_rdma_event(INFO, "wait_for_completion_timeout rc=%d\n", rc);
+
+	if (info->negotiate_done)
+		return 0;
+
+	if (rc == 0)
+		rc = -ETIMEDOUT;
+	else if (rc == -ERESTARTSYS)
+		rc = -EINTR;
+	else
+		rc = -ENOTCONN;
+
+	return rc;
+}
+
+static void put_empty_packet(
+		struct smbd_connection *info, struct smbd_response *response)
+{
+	spin_lock(&info->empty_packet_queue_lock);
+	list_add_tail(&response->list, &info->empty_packet_queue);
+	info->count_empty_packet_queue++;
+	spin_unlock(&info->empty_packet_queue_lock);
+
+	queue_work(info->workqueue, &info->post_send_credits_work);
+}
+
+/*
+ * Implement Connection.FragmentReassemblyBuffer defined in [MS-SMBD] 3.1.1.1
+ * This is a queue for reassembling upper layer payload and present to upper
+ * layer. All the inncoming payload go to the reassembly queue, regardless of
+ * if reassembly is required. The uuper layer code reads from the queue for all
+ * incoming payloads.
+ * Put a received packet to the reassembly queue
+ * response: the packet received
+ * data_length: the size of payload in this packet
+ */
+static void enqueue_reassembly(
+	struct smbd_connection *info,
+	struct smbd_response *response,
+	int data_length)
+{
+	spin_lock(&info->reassembly_queue_lock);
+	list_add_tail(&response->list, &info->reassembly_queue);
+	info->reassembly_queue_length++;
+	/*
+	 * Make sure reassembly_data_length is updated after list and
+	 * reassembly_queue_length are updated. On the dequeue side
+	 * reassembly_data_length is checked without a lock to determine
+	 * if reassembly_queue_length and list is up to date
+	 */
+	virt_wmb();
+	info->reassembly_data_length += data_length;
+	spin_unlock(&info->reassembly_queue_lock);
+	info->count_reassembly_queue++;
+	info->count_enqueue_reassembly_queue++;
+}
+
+/*
+ * Get the first entry at the front of reassembly queue
+ * Caller is responsible for locking
+ * return value: the first entry if any, NULL if queue is empty
+ */
+static struct smbd_response *_get_first_reassembly(struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+
+	if (!list_empty(&info->reassembly_queue)) {
+		ret = list_first_entry(
+			&info->reassembly_queue,
+			struct smbd_response, list);
+	}
+	return ret;
+}
+
+static struct smbd_response *get_empty_queue_buffer(
+		struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->empty_packet_queue_lock, flags);
+	if (!list_empty(&info->empty_packet_queue)) {
+		ret = list_first_entry(
+			&info->empty_packet_queue,
+			struct smbd_response, list);
+		list_del(&ret->list);
+		info->count_empty_packet_queue--;
+	}
+	spin_unlock_irqrestore(&info->empty_packet_queue_lock, flags);
+
+	return ret;
+}
+
+/*
+ * Get a receive buffer
+ * For each remote send, we need to post a receive. The receive buffers are
+ * pre-allocated in advance.
+ * return value: the receive buffer, NULL if none is available
+ */
+static struct smbd_response *get_receive_buffer(struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->receive_queue_lock, flags);
+	if (!list_empty(&info->receive_queue)) {
+		ret = list_first_entry(
+			&info->receive_queue,
+			struct smbd_response, list);
+		list_del(&ret->list);
+		info->count_receive_queue--;
+		info->count_get_receive_buffer++;
+	}
+	spin_unlock_irqrestore(&info->receive_queue_lock, flags);
+
+	return ret;
+}
+
+/*
+ * Return a receive buffer
+ * Upon returning of a receive buffer, we can post new receive and extend
+ * more receive credits to remote peer. This is done immediately after a
+ * receive buffer is returned.
+ */
+static void put_receive_buffer(
+	struct smbd_connection *info, struct smbd_response *response,
+	bool lock)
+{
+	unsigned long flags;
+
+	ib_dma_unmap_single(info->id->device, response->sge.addr,
+		response->sge.length, DMA_FROM_DEVICE);
+
+	if (lock)
+		spin_lock_irqsave(&info->receive_queue_lock, flags);
+	list_add_tail(&response->list, &info->receive_queue);
+	info->count_receive_queue++;
+	info->count_put_receive_buffer++;
+	if (lock)
+		spin_unlock_irqrestore(&info->receive_queue_lock, flags);
+
+	queue_work(info->workqueue, &info->post_send_credits_work);
+}
+
+/* Preallocate all receive buffer on transport establishment */
+static int allocate_receive_buffers(struct smbd_connection *info, int num_buf)
+{
+	int i;
+	struct smbd_response *response;
+
+	INIT_LIST_HEAD(&info->reassembly_queue);
+	spin_lock_init(&info->reassembly_queue_lock);
+	info->reassembly_data_length = 0;
+	info->reassembly_queue_length = 0;
+
+	INIT_LIST_HEAD(&info->receive_queue);
+	spin_lock_init(&info->receive_queue_lock);
+	info->count_receive_queue = 0;
+
+	INIT_LIST_HEAD(&info->empty_packet_queue);
+	spin_lock_init(&info->empty_packet_queue_lock);
+	info->count_empty_packet_queue = 0;
+
+	init_waitqueue_head(&info->wait_receive_queues);
+
+	for (i = 0; i < num_buf; i++) {
+		response = mempool_alloc(info->response_mempool, GFP_KERNEL);
+		if (!response)
+			goto allocate_failed;
+
+		response->info = info;
+		list_add_tail(&response->list, &info->receive_queue);
+		info->count_receive_queue++;
+	}
+
+	return 0;
+
+allocate_failed:
+	while (!list_empty(&info->receive_queue)) {
+		response = list_first_entry(
+				&info->receive_queue,
+				struct smbd_response, list);
+		list_del(&response->list);
+		info->count_receive_queue--;
+
+		mempool_free(response, info->response_mempool);
+	}
+	return -ENOMEM;
+}
+
+static void destroy_receive_buffers(struct smbd_connection *info)
+{
+	struct smbd_response *response;
+
+	while ((response = get_receive_buffer(info)))
+		mempool_free(response, info->response_mempool);
+
+	while ((response = get_empty_queue_buffer(info)))
+		mempool_free(response, info->response_mempool);
+}
+
+/*
+ * Check and send an immediate or keep alive packet
+ * The condition to send those packets are defined in [MS-SMBD] 3.1.1.1
+ * Connection.KeepaliveRequested and Connection.SendImmediate
+ * The idea is to extend credits to server as soon as it becomes available
+ */
+static void send_immediate_work(struct work_struct *work)
+{
+	struct smbd_connection *info = container_of(
+					work, struct smbd_connection,
+					send_immediate_work.work);
+
+	if (info->keep_alive_requested == KEEP_ALIVE_PENDING ||
+	    info->send_immediate) {
+		log_keep_alive(INFO, "send an empty message\n");
+		smbd_post_send_empty(info);
+	}
+}
+
+/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */
+static void idle_connection_timer(struct work_struct *work)
+{
+	struct smbd_connection *info = container_of(
+					work, struct smbd_connection,
+					idle_timer_work.work);
+
+	if (info->keep_alive_requested != KEEP_ALIVE_NONE) {
+		log_keep_alive(ERR,
+			"error status info->keep_alive_requested=%d\n",
+			info->keep_alive_requested);
+		smbd_disconnect_rdma_connection(info);
+		return;
+	}
+
+	log_keep_alive(INFO, "about to send an empty idle message\n");
+	smbd_post_send_empty(info);
+
+	/* Setup the next idle timeout work */
+	queue_delayed_work(info->workqueue, &info->idle_timer_work,
+			info->keep_alive_interval*HZ);
+}
+
+static void destroy_caches_and_workqueue(struct smbd_connection *info)
+{
+	destroy_receive_buffers(info);
+	destroy_workqueue(info->workqueue);
+	mempool_destroy(info->response_mempool);
+	kmem_cache_destroy(info->response_cache);
+	mempool_destroy(info->request_mempool);
+	kmem_cache_destroy(info->request_cache);
+}
+
+#define MAX_NAME_LEN	80
+static int allocate_caches_and_workqueue(struct smbd_connection *info)
+{
+	char name[MAX_NAME_LEN];
+	int rc;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_request_%p", info);
+	info->request_cache =
+		kmem_cache_create(
+			name,
+			sizeof(struct smbd_request) +
+				sizeof(struct smbd_data_transfer),
+			0, SLAB_HWCACHE_ALIGN, NULL);
+	if (!info->request_cache)
+		return -ENOMEM;
+
+	info->request_mempool =
+		mempool_create(info->send_credit_target, mempool_alloc_slab,
+			mempool_free_slab, info->request_cache);
+	if (!info->request_mempool)
+		goto out1;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_response_%p", info);
+	info->response_cache =
+		kmem_cache_create(
+			name,
+			sizeof(struct smbd_response) +
+				info->max_receive_size,
+			0, SLAB_HWCACHE_ALIGN, NULL);
+	if (!info->response_cache)
+		goto out2;
+
+	info->response_mempool =
+		mempool_create(info->receive_credit_max, mempool_alloc_slab,
+		       mempool_free_slab, info->response_cache);
+	if (!info->response_mempool)
+		goto out3;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_%p", info);
+	info->workqueue = create_workqueue(name);
+	if (!info->workqueue)
+		goto out4;
+
+	rc = allocate_receive_buffers(info, info->receive_credit_max);
+	if (rc) {
+		log_rdma_event(ERR, "failed to allocate receive buffers\n");
+		goto out5;
+	}
+
+	return 0;
+
+out5:
+	destroy_workqueue(info->workqueue);
+out4:
+	mempool_destroy(info->response_mempool);
+out3:
+	kmem_cache_destroy(info->response_cache);
+out2:
+	mempool_destroy(info->request_mempool);
+out1:
+	kmem_cache_destroy(info->request_cache);
+	return -ENOMEM;
+}
+
+/* Create a SMBD connection, called by upper layer */
+struct smbd_connection *_smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
+{
+	int rc;
+	struct smbd_connection *info;
+	struct rdma_conn_param conn_param;
+	struct ib_qp_init_attr qp_attr;
+	struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;
+
+	info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL);
+	if (!info)
+		return NULL;
+
+	info->transport_status = SMBD_CONNECTING;
+	rc = smbd_ia_open(info, dstaddr, port);
+	if (rc) {
+		log_rdma_event(INFO, "smbd_ia_open rc=%d\n", rc);
+		goto create_id_failed;
+	}
+
+	if (smbd_send_credit_target > info->id->device->attrs.max_cqe ||
+	    smbd_send_credit_target > info->id->device->attrs.max_qp_wr) {
+		log_rdma_event(ERR,
+			"consider lowering send_credit_target = %d. "
+			"Possible CQE overrun, device "
+			"reporting max_cpe %d max_qp_wr %d\n",
+			smbd_send_credit_target,
+			info->id->device->attrs.max_cqe,
+			info->id->device->attrs.max_qp_wr);
+		goto config_failed;
+	}
+
+	if (smbd_receive_credit_max > info->id->device->attrs.max_cqe ||
+	    smbd_receive_credit_max > info->id->device->attrs.max_qp_wr) {
+		log_rdma_event(ERR,
+			"consider lowering receive_credit_max = %d. "
+			"Possible CQE overrun, device "
+			"reporting max_cpe %d max_qp_wr %d\n",
+			smbd_receive_credit_max,
+			info->id->device->attrs.max_cqe,
+			info->id->device->attrs.max_qp_wr);
+		goto config_failed;
+	}
+
+	info->receive_credit_max = smbd_receive_credit_max;
+	info->send_credit_target = smbd_send_credit_target;
+	info->max_send_size = smbd_max_send_size;
+	info->max_fragmented_recv_size = smbd_max_fragmented_recv_size;
+	info->max_receive_size = smbd_max_receive_size;
+	info->keep_alive_interval = smbd_keep_alive_interval;
+
+	if (SMBDIRECT_MAX_SGE > info->id->device->attrs.max_sge) {
+		log_rdma_event(ERR, "warning: device max_sge = %d too small\n",
+			info->id->device->attrs.max_sge);
+		log_rdma_event(ERR, "Queue Pair creation may fail\n");
+	}
+
+	info->send_cq = NULL;
+	info->recv_cq = NULL;
+	info->send_cq = ib_alloc_cq(info->id->device, info,
+			info->send_credit_target, 0, IB_POLL_SOFTIRQ);
+	if (IS_ERR(info->send_cq)) {
+		info->send_cq = NULL;
+		goto alloc_cq_failed;
+	}
+
+	info->recv_cq = ib_alloc_cq(info->id->device, info,
+			info->receive_credit_max, 0, IB_POLL_SOFTIRQ);
+	if (IS_ERR(info->recv_cq)) {
+		info->recv_cq = NULL;
+		goto alloc_cq_failed;
+	}
+
+	memset(&qp_attr, 0, sizeof(qp_attr));
+	qp_attr.event_handler = smbd_qp_async_error_upcall;
+	qp_attr.qp_context = info;
+	qp_attr.cap.max_send_wr = info->send_credit_target;
+	qp_attr.cap.max_recv_wr = info->receive_credit_max;
+	qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
+	qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
+	qp_attr.cap.max_inline_data = 0;
+	qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
+	qp_attr.qp_type = IB_QPT_RC;
+	qp_attr.send_cq = info->send_cq;
+	qp_attr.recv_cq = info->recv_cq;
+	qp_attr.port_num = ~0;
+
+	rc = rdma_create_qp(info->id, info->pd, &qp_attr);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_create_qp failed %i\n", rc);
+		goto create_qp_failed;
+	}
+
+	memset(&conn_param, 0, sizeof(conn_param));
+	conn_param.initiator_depth = 0;
+
+	conn_param.retry_count = SMBD_CM_RETRY;
+	conn_param.rnr_retry_count = SMBD_CM_RNR_RETRY;
+	conn_param.flow_control = 0;
+	init_waitqueue_head(&info->wait_destroy);
+
+	log_rdma_event(INFO, "connecting to IP %pI4 port %d\n",
+		&addr_in->sin_addr, port);
+
+	init_waitqueue_head(&info->conn_wait);
+	rc = rdma_connect(info->id, &conn_param);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_connect() failed with %i\n", rc);
+		goto rdma_connect_failed;
+	}
+
+	wait_event_interruptible(
+		info->conn_wait, info->transport_status != SMBD_CONNECTING);
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		log_rdma_event(ERR, "rdma_connect failed port=%d\n", port);
+		goto rdma_connect_failed;
+	}
+
+	log_rdma_event(INFO, "rdma_connect connected\n");
+
+	rc = allocate_caches_and_workqueue(info);
+	if (rc) {
+		log_rdma_event(ERR, "cache allocation failed\n");
+		goto allocate_cache_failed;
+	}
+
+	init_waitqueue_head(&info->wait_send_queue);
+	init_waitqueue_head(&info->wait_reassembly_queue);
+
+	INIT_DELAYED_WORK(&info->idle_timer_work, idle_connection_timer);
+	INIT_DELAYED_WORK(&info->send_immediate_work, send_immediate_work);
+	queue_delayed_work(info->workqueue, &info->idle_timer_work,
+		info->keep_alive_interval*HZ);
+
+	init_waitqueue_head(&info->wait_send_pending);
+	atomic_set(&info->send_pending, 0);
+
+	init_waitqueue_head(&info->wait_send_payload_pending);
+	atomic_set(&info->send_payload_pending, 0);
+
+	INIT_WORK(&info->disconnect_work, smbd_disconnect_rdma_work);
+	INIT_WORK(&info->destroy_work, smbd_destroy_rdma_work);
+	INIT_WORK(&info->recv_done_work, smbd_recv_done_work);
+	INIT_WORK(&info->post_send_credits_work, smbd_post_send_credits);
+	info->new_credits_offered = 0;
+	spin_lock_init(&info->lock_new_credits_offered);
+
+	rc = smbd_negotiate(info);
+	if (rc) {
+		log_rdma_event(ERR, "smbd_negotiate rc=%d\n", rc);
+		goto negotiation_failed;
+	}
+
+	return info;
+
+negotiation_failed:
+	cancel_delayed_work_sync(&info->idle_timer_work);
+	destroy_caches_and_workqueue(info);
+	info->transport_status = SMBD_NEGOTIATE_FAILED;
+	init_waitqueue_head(&info->conn_wait);
+	rdma_disconnect(info->id);
+	wait_event(info->conn_wait,
+		info->transport_status == SMBD_DISCONNECTED);
+
+allocate_cache_failed:
+rdma_connect_failed:
+	rdma_destroy_qp(info->id);
+
+create_qp_failed:
+alloc_cq_failed:
+	if (info->send_cq)
+		ib_free_cq(info->send_cq);
+	if (info->recv_cq)
+		ib_free_cq(info->recv_cq);
+
+config_failed:
+	ib_dealloc_pd(info->pd);
+	rdma_destroy_id(info->id);
+
+create_id_failed:
+	kfree(info);
+	return NULL;
+}
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
index c55f28b..35bc25b 100644
--- a/fs/cifs/smbdirect.h
+++ b/fs/cifs/smbdirect.h
@@ -16,6 +16,286 @@
 #ifndef _SMBDIRECT_H
 #define _SMBDIRECT_H
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#define cifs_rdma_enabled(server)	((server)->rdma)
+
+#include "cifsglob.h"
+#include <rdma/ib_verbs.h>
+#include <rdma/rdma_cm.h>
+#include <linux/mempool.h>
+
+enum keep_alive_status {
+	KEEP_ALIVE_NONE,
+	KEEP_ALIVE_PENDING,
+	KEEP_ALIVE_SENT,
+};
+
+enum smbd_connection_status {
+	SMBD_CREATED,
+	SMBD_CONNECTING,
+	SMBD_CONNECTED,
+	SMBD_NEGOTIATE_FAILED,
+	SMBD_DISCONNECTING,
+	SMBD_DISCONNECTED,
+	SMBD_DESTROYED
+};
+
+/*
+ * The context for the SMBDirect transport
+ * Everything related to the transport is here. It has several logical parts
+ * 1. RDMA related structures
+ * 2. SMBDirect connection parameters
+ * 3. Memory registrations
+ * 4. Receive and reassembly queues for data receive path
+ * 5. mempools for allocating packets
+ */
+struct smbd_connection {
+	enum smbd_connection_status transport_status;
+
+	/* RDMA related */
+	struct rdma_cm_id *id;
+	struct ib_qp_init_attr qp_attr;
+	struct ib_pd *pd;
+	struct ib_cq *send_cq, *recv_cq;
+	struct ib_device_attr dev_attr;
+	int ri_rc;
+	struct completion ri_done;
+	wait_queue_head_t conn_wait;
+	wait_queue_head_t wait_destroy;
+
+	struct completion negotiate_completion;
+	bool negotiate_done;
+
+	struct work_struct destroy_work;
+	struct work_struct disconnect_work;
+	struct work_struct recv_done_work;
+	struct work_struct post_send_credits_work;
+
+	spinlock_t lock_new_credits_offered;
+	int new_credits_offered;
+
+	/* Connection parameters defined in [MS-SMBD] 3.1.1.1 */
+	int receive_credit_max;
+	int send_credit_target;
+	int max_send_size;
+	int max_fragmented_recv_size;
+	int max_fragmented_send_size;
+	int max_receive_size;
+	int keep_alive_interval;
+	int max_readwrite_size;
+	enum keep_alive_status keep_alive_requested;
+	int protocol;
+	atomic_t send_credits;
+	atomic_t receive_credits;
+	int receive_credit_target;
+	int fragment_reassembly_remaining;
+
+	/* Activity accoutning */
+
+	atomic_t send_pending;
+	wait_queue_head_t wait_send_pending;
+	atomic_t send_payload_pending;
+	wait_queue_head_t wait_send_payload_pending;
+
+	/* Receive queue */
+	struct list_head receive_queue;
+	int count_receive_queue;
+	spinlock_t receive_queue_lock;
+
+	struct list_head empty_packet_queue;
+	int count_empty_packet_queue;
+	spinlock_t empty_packet_queue_lock;
+
+	wait_queue_head_t wait_receive_queues;
+
+	/* Reassembly queue */
+	struct list_head reassembly_queue;
+	spinlock_t reassembly_queue_lock;
+	wait_queue_head_t wait_reassembly_queue;
+
+	/* total data length of reassembly queue */
+	int reassembly_data_length;
+	int reassembly_queue_length;
+	/* the offset to first buffer in reassembly queue */
+	int first_entry_offset;
+
+	bool send_immediate;
+
+	wait_queue_head_t wait_send_queue;
+
+	/*
+	 * Indicate if we have received a full packet on the connection
+	 * This is used to identify the first SMBD packet of a assembled
+	 * payload (SMB packet) in reassembly queue so we can return a
+	 * RFC1002 length to upper layer to indicate the length of the SMB
+	 * packet received
+	 */
+	bool full_packet_received;
+
+	struct workqueue_struct *workqueue;
+	struct delayed_work idle_timer_work;
+	struct delayed_work send_immediate_work;
+
+	/* Memory pool for preallocating buffers */
+	/* request pool for RDMA send */
+	struct kmem_cache *request_cache;
+	mempool_t *request_mempool;
+
+	/* response pool for RDMA receive */
+	struct kmem_cache *response_cache;
+	mempool_t *response_mempool;
+
+	/* for debug purposes */
+	unsigned int count_get_receive_buffer;
+	unsigned int count_put_receive_buffer;
+	unsigned int count_reassembly_queue;
+	unsigned int count_enqueue_reassembly_queue;
+	unsigned int count_dequeue_reassembly_queue;
+	unsigned int count_send_empty;
+};
+
+enum smbd_message_type {
+	SMBD_NEGOTIATE_RESP,
+	SMBD_TRANSFER_DATA,
+};
+
+#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
+
+/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */
+struct smbd_negotiate_req {
+	__le16 min_version;
+	__le16 max_version;
+	__le16 reserved;
+	__le16 credits_requested;
+	__le32 preferred_send_size;
+	__le32 max_receive_size;
+	__le32 max_fragmented_size;
+} __packed;
+
+/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */
+struct smbd_negotiate_resp {
+	__le16 min_version;
+	__le16 max_version;
+	__le16 negotiated_version;
+	__le16 reserved;
+	__le16 credits_requested;
+	__le16 credits_granted;
+	__le32 status;
+	__le32 max_readwrite_size;
+	__le32 preferred_send_size;
+	__le32 max_receive_size;
+	__le32 max_fragmented_size;
+} __packed;
+
+/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */
+struct smbd_data_transfer {
+	__le16 credits_requested;
+	__le16 credits_granted;
+	__le16 flags;
+	__le16 reserved;
+	__le32 remaining_data_length;
+	__le32 data_offset;
+	__le32 data_length;
+	__le32 padding;
+	__u8 buffer[];
+} __packed;
+
+/* The packet fields for a registered RDMA buffer */
+struct smbd_buffer_descriptor_v1 {
+	__le64 offset;
+	__le32 token;
+	__le32 length;
+} __packed;
+
 /* Default maximum number of SGEs in a RDMA send/recv */
 #define SMBDIRECT_MAX_SGE	16
+/* The context for a SMBD request */
+struct smbd_request {
+	struct smbd_connection *info;
+	struct ib_cqe cqe;
+
+	/* true if this request carries upper layer payload */
+	bool has_payload;
+
+	/* the SGE entries for this packet */
+	struct ib_sge sge[SMBDIRECT_MAX_SGE];
+	int num_sge;
+
+	/* SMBD packet header follows this structure */
+	u8 packet[];
+};
+
+/* The context for a SMBD response */
+struct smbd_response {
+	struct smbd_connection *info;
+	struct ib_cqe cqe;
+	struct ib_sge sge;
+
+	enum smbd_message_type type;
+
+	/* Link to receive queue or reassembly queue */
+	struct list_head list;
+
+	/* Indicate if this is the 1st packet of a payload */
+	bool first_segment;
+
+	/* SMBD packet header and payload follows this structure */
+	u8 packet[];
+};
+
+/* Create a SMBDirect session */
+struct smbd_connection *smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr);
+
+/* Reconnect SMBDirect session */
+int smbd_reconnect(struct TCP_Server_Info *server);
+
+/* Destroy SMBDirect session */
+void smbd_destroy(struct smbd_connection *info);
+
+/* Interface for carrying upper layer I/O through send/recv */
+int smbd_recv(struct smbd_connection *info, struct msghdr *msg);
+int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst);
+
+enum mr_state {
+	MR_READY,
+	MR_REGISTERED,
+	MR_INVALIDATED,
+	MR_ERROR
+};
+
+struct smbd_mr {
+	struct smbd_connection	*conn;
+	struct list_head	list;
+	enum mr_state		state;
+	struct ib_mr		*mr;
+	struct scatterlist	*sgl;
+	int			sgl_count;
+	enum dma_data_direction	dir;
+	union {
+		struct ib_reg_wr	wr;
+		struct ib_send_wr	inv_wr;
+	};
+	struct ib_cqe		cqe;
+	bool			need_invalidate;
+	struct completion	invalidate_done;
+};
+
+/* Interfaces to register and deregister MR for RDMA read/write */
+struct smbd_mr *smbd_register_mr(
+	struct smbd_connection *info, struct page *pages[], int num_pages,
+	int tailsz, bool writing, bool need_invalidate);
+int smbd_deregister_mr(struct smbd_mr *mr);
+
+#else
+#define cifs_rdma_enabled(server)	0
+struct smbd_connection{};
+static inline void *smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr) {return NULL;}
+static inline int smbd_reconnect(struct TCP_Server_Info *server) {return -1;}
+static inline void smbd_destroy(struct smbd_connection *info) {}
+static inline int smbd_recv(struct smbd_connection *info, struct msghdr *msg) {return -1;}
+static inline int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst) {return -1;}
+#endif
+
 #endif
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber@gmail.com>

But two tiny nits:
+ * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
Should be 0xa0

+//	queue_work(info->workqueue, &info->destroy_work);
Don't leave commented out code in there.


----- Original Message -----
From: "Long Li" <longli@exchange.microsoft.com>
To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox" <mawilcox@microsoft.com>, "Stephen Hemminger" <sthemmin@microsoft.com>
Cc: "Long Li" <longli@microsoft.com>
Sent: Tuesday, 7 November, 2017 7:54:57 PM
Subject: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

From: Long Li <longli@microsoft.com>

Add code to implement the core functions to establish a SMB Direct connection.

1. Establish an RDMA connection to SMB server.
2. Negotiate and setup SMB Direct protocol.
3. Implement idle connection timer and credit management.

SMB Direct is enabled by setting CONFIG_CIFS_SMB_DIRECT.

Add to Makefile to enable building SMB Direct.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/Makefile    |    2 +
 fs/cifs/smbdirect.c | 1576 +++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h |  280 +++++++++
 3 files changed, 1858 insertions(+)

diff --git a/fs/cifs/Makefile b/fs/cifs/Makefile
index 5e853a3..ad00873 100644
--- a/fs/cifs/Makefile
+++ b/fs/cifs/Makefile
@@ -18,3 +18,5 @@ cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
 cifs-$(CONFIG_CIFS_DFS_UPCALL) += dns_resolve.o cifs_dfs_ref.o
 
 cifs-$(CONFIG_CIFS_FSCACHE) += fscache.o cache.o
+
+cifs-$(CONFIG_CIFS_SMB_DIRECT) += smbdirect.o
diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index d3c16f8..021d527 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -13,7 +13,34 @@
  *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
  *   the GNU General Public License for more details.
  */
+#include <linux/module.h>
 #include "smbdirect.h"
+#include "cifs_debug.h"
+
+static struct smbd_response *get_empty_queue_buffer(
+		struct smbd_connection *info);
+static struct smbd_response *get_receive_buffer(
+		struct smbd_connection *info);
+static void put_receive_buffer(
+		struct smbd_connection *info,
+		struct smbd_response *response,
+		bool lock);
+static int allocate_receive_buffers(struct smbd_connection *info, int num_buf);
+static void destroy_receive_buffers(struct smbd_connection *info);
+
+static void put_empty_packet(
+		struct smbd_connection *info, struct smbd_response *response);
+static void enqueue_reassembly(
+		struct smbd_connection *info,
+		struct smbd_response *response, int data_length);
+static struct smbd_response *_get_first_reassembly(
+		struct smbd_connection *info);
+
+static int smbd_post_recv(
+		struct smbd_connection *info,
+		struct smbd_response *response);
+
+static int smbd_post_send_empty(struct smbd_connection *info);
 
 /* SMBD version number */
 #define SMBD_V1	0x0100
@@ -75,3 +102,1552 @@ int smbd_max_frmr_depth = 2048;
 
 /* If payload is less than this byte, use RDMA send/recv not read/write */
 int rdma_readwrite_threshold = 4096;
+
+/* Transport logging functions
+ * Logging are defined as classes. They can be OR'ed to define the actual
+ * logging level via module parameter smbd_logging_class
+ * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
+ * log_rdma_event()
+ */
+#define LOG_OUTGOING			0x1
+#define LOG_INCOMING			0x2
+#define LOG_READ			0x4
+#define LOG_WRITE			0x8
+#define LOG_RDMA_SEND			0x10
+#define LOG_RDMA_RECV			0x20
+#define LOG_KEEP_ALIVE			0x40
+#define LOG_RDMA_EVENT			0x80
+#define LOG_RDMA_MR			0x100
+static unsigned int smbd_logging_class = 0;
+module_param(smbd_logging_class, uint, 0644);
+MODULE_PARM_DESC(smbd_logging_class,
+	"Logging class for SMBD transport 0x0 to 0x100");
+
+#define ERR		0x0
+#define INFO		0x1
+static unsigned int smbd_logging_level = ERR;
+module_param(smbd_logging_level, uint, 0644);
+MODULE_PARM_DESC(smbd_logging_level,
+	"Logging level for SMBD transport, 0 (default): error, 1: info");
+
+#define log_rdma(level, class, fmt, args...)				\
+do {									\
+	if (level <= smbd_logging_level || class & smbd_logging_class)	\
+		cifs_dbg(VFS, "%s:%d " fmt, __func__, __LINE__, ##args);\
+} while (0)
+
+#define log_outgoing(level, fmt, args...) \
+		log_rdma(level, LOG_OUTGOING, fmt, ##args)
+#define log_incoming(level, fmt, args...) \
+		log_rdma(level, LOG_INCOMING, fmt, ##args)
+#define log_read(level, fmt, args...)	log_rdma(level, LOG_READ, fmt, ##args)
+#define log_write(level, fmt, args...)	log_rdma(level, LOG_WRITE, fmt, ##args)
+#define log_rdma_send(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_SEND, fmt, ##args)
+#define log_rdma_recv(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_RECV, fmt, ##args)
+#define log_keep_alive(level, fmt, args...) \
+		log_rdma(level, LOG_KEEP_ALIVE, fmt, ##args)
+#define log_rdma_event(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_EVENT, fmt, ##args)
+#define log_rdma_mr(level, fmt, args...) \
+		log_rdma(level, LOG_RDMA_MR, fmt, ##args)
+
+/*
+ * Destroy the transport and related RDMA and memory resources
+ * Need to go through all the pending counters and make sure on one is using
+ * the transport while it is destroyed
+ */
+static void smbd_destroy_rdma_work(struct work_struct *work)
+{
+	struct smbd_response *response;
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, destroy_work);
+	unsigned long flags;
+
+	log_rdma_event(INFO, "destroying qp\n");
+	ib_drain_qp(info->id->qp);
+	rdma_destroy_qp(info->id);
+
+	/* Unblock all I/O waiting on the send queue */
+	wake_up_interruptible_all(&info->wait_send_queue);
+
+	log_rdma_event(INFO, "cancelling idle timer\n");
+	cancel_delayed_work_sync(&info->idle_timer_work);
+	log_rdma_event(INFO, "cancelling send immediate work\n");
+	cancel_delayed_work_sync(&info->send_immediate_work);
+
+	log_rdma_event(INFO, "wait for all recv to finish\n");
+	wake_up_interruptible(&info->wait_reassembly_queue);
+
+	log_rdma_event(INFO, "wait for all send posted to IB to finish\n");
+	wait_event(info->wait_send_pending,
+		atomic_read(&info->send_pending) == 0);
+	wait_event(info->wait_send_payload_pending,
+		atomic_read(&info->send_payload_pending) == 0);
+
+	/* It's not posssible for upper layer to get to reassembly */
+	log_rdma_event(INFO, "drain the reassembly queue\n");
+	do {
+		spin_lock_irqsave(&info->reassembly_queue_lock, flags);
+		response = _get_first_reassembly(info);
+		if (response) {
+			list_del(&response->list);
+			spin_unlock_irqrestore(
+				&info->reassembly_queue_lock, flags);
+			put_receive_buffer(info, response, true);
+		}
+	} while (response);
+	spin_unlock_irqrestore(&info->reassembly_queue_lock, flags);
+	info->reassembly_data_length = 0;
+
+	log_rdma_event(INFO, "free receive buffers\n");
+	wait_event(info->wait_receive_queues,
+		info->count_receive_queue + info->count_empty_packet_queue
+			== info->receive_credit_max);
+	destroy_receive_buffers(info);
+
+	ib_free_cq(info->send_cq);
+	ib_free_cq(info->recv_cq);
+	ib_dealloc_pd(info->pd);
+	rdma_destroy_id(info->id);
+
+	/* free mempools */
+	mempool_destroy(info->request_mempool);
+	kmem_cache_destroy(info->request_cache);
+
+	mempool_destroy(info->response_mempool);
+	kmem_cache_destroy(info->response_cache);
+
+	info->transport_status = SMBD_DESTROYED;
+	wake_up_all(&info->wait_destroy);
+}
+
+static int smbd_process_disconnected(struct smbd_connection *info)
+{
+//	queue_work(info->workqueue, &info->destroy_work);
+	schedule_work(&info->destroy_work);
+	return 0;
+}
+
+static void smbd_disconnect_rdma_work(struct work_struct *work)
+{
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, disconnect_work);
+
+	if (info->transport_status == SMBD_CONNECTED) {
+		info->transport_status = SMBD_DISCONNECTING;
+		rdma_disconnect(info->id);
+	}
+}
+
+static void smbd_disconnect_rdma_connection(struct smbd_connection *info)
+{
+	queue_work(info->workqueue, &info->disconnect_work);
+}
+
+/* Upcall from RDMA CM */
+static int smbd_conn_upcall(
+		struct rdma_cm_id *id, struct rdma_cm_event *event)
+{
+	struct smbd_connection *info = id->context;
+
+	log_rdma_event(INFO, "event=%d status=%d\n",
+		event->event, event->status);
+
+	switch (event->event) {
+	case RDMA_CM_EVENT_ADDR_RESOLVED:
+	case RDMA_CM_EVENT_ROUTE_RESOLVED:
+		info->ri_rc = 0;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ADDR_ERROR:
+		info->ri_rc = -EHOSTUNREACH;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ROUTE_ERROR:
+		info->ri_rc = -ENETUNREACH;
+		complete(&info->ri_done);
+		break;
+
+	case RDMA_CM_EVENT_ESTABLISHED:
+		log_rdma_event(INFO, "connected event=%d\n", event->event);
+		info->transport_status = SMBD_CONNECTED;
+		wake_up_interruptible(&info->conn_wait);
+		break;
+
+	case RDMA_CM_EVENT_CONNECT_ERROR:
+	case RDMA_CM_EVENT_UNREACHABLE:
+	case RDMA_CM_EVENT_REJECTED:
+		log_rdma_event(INFO, "connecting failed event=%d\n", event->event);
+		info->transport_status = SMBD_DISCONNECTED;
+		wake_up_interruptible(&info->conn_wait);
+		break;
+
+	case RDMA_CM_EVENT_DEVICE_REMOVAL:
+	case RDMA_CM_EVENT_DISCONNECTED:
+		/* This happenes when we fail the negotiation */
+		if (info->transport_status == SMBD_NEGOTIATE_FAILED) {
+			info->transport_status = SMBD_DISCONNECTED;
+			wake_up(&info->conn_wait);
+			break;
+		}
+
+		info->transport_status = SMBD_DISCONNECTED;
+		smbd_process_disconnected(info);
+		break;
+
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+/* Upcall from RDMA QP */
+static void
+smbd_qp_async_error_upcall(struct ib_event *event, void *context)
+{
+	struct smbd_connection *info = context;
+
+	log_rdma_event(ERR, "%s on device %s info %p\n",
+		ib_event_msg(event->event), event->device->name, info);
+
+	switch (event->event) {
+	case IB_EVENT_CQ_ERR:
+	case IB_EVENT_QP_FATAL:
+		smbd_disconnect_rdma_connection(info);
+
+	default:
+		break;
+	}
+}
+
+static inline void *smbd_request_payload(struct smbd_request *request)
+{
+	return (void *)request->packet;
+}
+
+static inline void *smbd_response_payload(struct smbd_response *response)
+{
+	return (void *)response->packet;
+}
+
+/* Called when a RDMA send is done */
+static void send_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+	int i;
+	struct smbd_request *request =
+		container_of(wc->wr_cqe, struct smbd_request, cqe);
+
+	log_rdma_send(INFO, "smbd_request %p completed wc->status=%d\n",
+		request, wc->status);
+
+	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_SEND) {
+		log_rdma_send(ERR, "wc->status=%d wc->opcode=%d\n",
+			wc->status, wc->opcode);
+		smbd_disconnect_rdma_connection(request->info);
+	}
+
+	for (i = 0; i < request->num_sge; i++)
+		ib_dma_unmap_single(request->info->id->device,
+			request->sge[i].addr,
+			request->sge[i].length,
+			DMA_TO_DEVICE);
+
+	if (request->has_payload) {
+		if (atomic_dec_and_test(&request->info->send_payload_pending))
+			wake_up(&request->info->wait_send_payload_pending);
+	} else {
+		if (atomic_dec_and_test(&request->info->send_pending))
+			wake_up(&request->info->wait_send_pending);
+	}
+
+	mempool_free(request, request->info->request_mempool);
+}
+
+static void dump_smbd_negotiate_resp(struct smbd_negotiate_resp *resp)
+{
+	log_rdma_event(INFO, "resp message min_version %u max_version %u "
+		"negotiated_version %u credits_requested %u "
+		"credits_granted %u status %u max_readwrite_size %u "
+		"preferred_send_size %u max_receive_size %u "
+		"max_fragmented_size %u\n",
+		resp->min_version, resp->max_version, resp->negotiated_version,
+		resp->credits_requested, resp->credits_granted, resp->status,
+		resp->max_readwrite_size, resp->preferred_send_size,
+		resp->max_receive_size, resp->max_fragmented_size);
+}
+
+/*
+ * Process a negotiation response message, according to [MS-SMBD]3.1.5.7
+ * response, packet_length: the negotiation response message
+ * return value: true if negotiation is a success, false if failed
+ */
+static bool process_negotiation_response(
+		struct smbd_response *response, int packet_length)
+{
+	struct smbd_connection *info = response->info;
+	struct smbd_negotiate_resp *packet = smbd_response_payload(response);
+
+	if (packet_length < sizeof(struct smbd_negotiate_resp)) {
+		log_rdma_event(ERR,
+			"error: packet_length=%d\n", packet_length);
+		return false;
+	}
+
+	if (le16_to_cpu(packet->negotiated_version) != SMBD_V1) {
+		log_rdma_event(ERR, "error: negotiated_version=%x\n",
+			le16_to_cpu(packet->negotiated_version));
+		return false;
+	}
+	info->protocol = le16_to_cpu(packet->negotiated_version);
+
+	if (packet->credits_requested == 0) {
+		log_rdma_event(ERR, "error: credits_requested==0\n");
+		return false;
+	}
+	info->receive_credit_target = le16_to_cpu(packet->credits_requested);
+
+	if (packet->credits_granted == 0) {
+		log_rdma_event(ERR, "error: credits_granted==0\n");
+		return false;
+	}
+	atomic_set(&info->send_credits, le16_to_cpu(packet->credits_granted));
+
+	atomic_set(&info->receive_credits, 0);
+
+	if (le32_to_cpu(packet->preferred_send_size) > info->max_receive_size) {
+		log_rdma_event(ERR, "error: preferred_send_size=%d\n",
+			le32_to_cpu(packet->preferred_send_size));
+		return false;
+	}
+	info->max_receive_size = le32_to_cpu(packet->preferred_send_size);
+
+	if (le32_to_cpu(packet->max_receive_size) < SMBD_MIN_RECEIVE_SIZE) {
+		log_rdma_event(ERR, "error: max_receive_size=%d\n",
+			le32_to_cpu(packet->max_receive_size));
+		return false;
+	}
+	info->max_send_size = min_t(int, info->max_send_size,
+					le32_to_cpu(packet->max_receive_size));
+
+	if (le32_to_cpu(packet->max_fragmented_size) <
+			SMBD_MIN_FRAGMENTED_SIZE) {
+		log_rdma_event(ERR, "error: max_fragmented_size=%d\n",
+			le32_to_cpu(packet->max_fragmented_size));
+		return false;
+	}
+	info->max_fragmented_send_size =
+		le32_to_cpu(packet->max_fragmented_size);
+
+	return true;
+}
+
+/*
+ * Check and schedule to send an immediate packet
+ * This is used to extend credtis to remote peer to keep the transport busy
+ */
+static void check_and_send_immediate(struct smbd_connection *info)
+{
+	if (info->transport_status != SMBD_CONNECTED)
+		return;
+
+	info->send_immediate = true;
+
+	/*
+	 * Promptly send a packet if our peer is running low on receive
+	 * credits
+	 */
+	if (atomic_read(&info->receive_credits) <
+		info->receive_credit_target - 1)
+		queue_delayed_work(
+			info->workqueue, &info->send_immediate_work, 0);
+}
+
+static void smbd_post_send_credits(struct work_struct *work)
+{
+	int ret = 0;
+	int use_receive_queue = 1;
+	int rc;
+	struct smbd_response *response;
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection,
+			post_send_credits_work);
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		wake_up(&info->wait_receive_queues);
+		return;
+	}
+
+	if (info->receive_credit_target >
+		atomic_read(&info->receive_credits)) {
+		while (true) {
+			if (use_receive_queue)
+				response = get_receive_buffer(info);
+			else
+				response = get_empty_queue_buffer(info);
+			if (!response) {
+				/* now switch to emtpy packet queue */
+				if (use_receive_queue) {
+					use_receive_queue = 0;
+					continue;
+				} else
+					break;
+			}
+
+			response->type = SMBD_TRANSFER_DATA;
+			response->first_segment = false;
+			rc = smbd_post_recv(info, response);
+			if (rc) {
+				log_rdma_recv(ERR,
+					"post_recv failed rc=%d\n", rc);
+				put_receive_buffer(info, response, true);
+				break;
+			}
+
+			ret++;
+		}
+	}
+
+	spin_lock(&info->lock_new_credits_offered);
+	info->new_credits_offered += ret;
+	spin_unlock(&info->lock_new_credits_offered);
+
+	atomic_add(ret, &info->receive_credits);
+
+	/* Check if we can post new receive and grant credits to peer */
+	check_and_send_immediate(info);
+}
+
+static void smbd_recv_done_work(struct work_struct *work)
+{
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, recv_done_work);
+
+	/*
+	 * We may have new send credits granted from remote peer
+	 * If any sender is blcoked on lack of credets, unblock it
+	 */
+	if (atomic_read(&info->send_credits))
+		wake_up_interruptible(&info->wait_send_queue);
+
+	/*
+	 * Check if we need to send something to remote peer to
+	 * grant more credits or respond to KEEP_ALIVE packet
+	 */
+	check_and_send_immediate(info);
+}
+
+/* Called from softirq, when recv is done */
+static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+	struct smbd_data_transfer *data_transfer;
+	struct smbd_response *response =
+		container_of(wc->wr_cqe, struct smbd_response, cqe);
+	struct smbd_connection *info = response->info;
+	int data_length = 0;
+
+	log_rdma_recv(INFO, "response=%p type=%d wc status=%d wc opcode %d "
+		      "byte_len=%d pkey_index=%x\n",
+		response, response->type, wc->status, wc->opcode,
+		wc->byte_len, wc->pkey_index);
+
+	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
+		log_rdma_recv(INFO, "wc->status=%d opcode=%d\n",
+			wc->status, wc->opcode);
+		smbd_disconnect_rdma_connection(info);
+		goto error;
+	}
+
+	ib_dma_sync_single_for_cpu(
+		wc->qp->device,
+		response->sge.addr,
+		response->sge.length,
+		DMA_FROM_DEVICE);
+
+	switch (response->type) {
+	/* SMBD negotiation response */
+	case SMBD_NEGOTIATE_RESP:
+		dump_smbd_negotiate_resp(smbd_response_payload(response));
+		info->full_packet_received = true;
+		info->negotiate_done =
+			process_negotiation_response(response, wc->byte_len);
+		complete(&info->negotiate_completion);
+		break;
+
+	/* SMBD data transfer packet */
+	case SMBD_TRANSFER_DATA:
+		data_transfer = smbd_response_payload(response);
+		data_length = le32_to_cpu(data_transfer->data_length);
+
+		/*
+		 * If this is a packet with data playload place the data in
+		 * reassembly queue and wake up the reading thread
+		 */
+		if (data_length) {
+			if (info->full_packet_received)
+				response->first_segment = true;
+
+			if (le32_to_cpu(data_transfer->remaining_data_length))
+				info->full_packet_received = false;
+			else
+				info->full_packet_received = true;
+
+			enqueue_reassembly(
+				info,
+				response,
+				data_length);
+		} else
+			put_empty_packet(info, response);
+
+		if (data_length)
+			wake_up_interruptible(&info->wait_reassembly_queue);
+
+		atomic_dec(&info->receive_credits);
+		info->receive_credit_target =
+			le16_to_cpu(data_transfer->credits_requested);
+		atomic_add(le16_to_cpu(data_transfer->credits_granted),
+			&info->send_credits);
+
+		log_incoming(INFO, "data flags %d data_offset %d "
+			"data_length %d remaining_data_length %d\n",
+			le16_to_cpu(data_transfer->flags),
+			le32_to_cpu(data_transfer->data_offset),
+			le32_to_cpu(data_transfer->data_length),
+			le32_to_cpu(data_transfer->remaining_data_length));
+
+		/* Send a KEEP_ALIVE response right away if requested */
+		info->keep_alive_requested = KEEP_ALIVE_NONE;
+		if (le16_to_cpu(data_transfer->flags) &
+				SMB_DIRECT_RESPONSE_REQUESTED) {
+			info->keep_alive_requested = KEEP_ALIVE_PENDING;
+		}
+
+		queue_work(info->workqueue, &info->recv_done_work);
+		return;
+
+	default:
+		log_rdma_recv(ERR,
+			"unexpected response type=%d\n", response->type);
+	}
+
+error:
+	put_receive_buffer(info, response, true);
+}
+
+static struct rdma_cm_id *smbd_create_id(
+		struct smbd_connection *info,
+		struct sockaddr *dstaddr, int port)
+{
+	struct rdma_cm_id *id;
+	int rc;
+	__be16 *sport;
+
+	id = rdma_create_id(&init_net, smbd_conn_upcall, info,
+		RDMA_PS_TCP, IB_QPT_RC);
+	if (IS_ERR(id)) {
+		rc = PTR_ERR(id);
+		log_rdma_event(ERR, "rdma_create_id() failed %i\n", rc);
+		return id;
+	}
+
+	if (dstaddr->sa_family == AF_INET6)
+		sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port;
+	else
+		sport = &((struct sockaddr_in *)dstaddr)->sin_port;
+
+	*sport = htons(port);
+
+	init_completion(&info->ri_done);
+	info->ri_rc = -ETIMEDOUT;
+
+	rc = rdma_resolve_addr(id, NULL, (struct sockaddr *)dstaddr,
+		RDMA_RESOLVE_TIMEOUT);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_addr() failed %i\n", rc);
+		goto out;
+	}
+	wait_for_completion_interruptible_timeout(
+		&info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
+	rc = info->ri_rc;
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_addr() completed %i\n", rc);
+		goto out;
+	}
+
+	info->ri_rc = -ETIMEDOUT;
+	rc = rdma_resolve_route(id, RDMA_RESOLVE_TIMEOUT);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_route() failed %i\n", rc);
+		goto out;
+	}
+	wait_for_completion_interruptible_timeout(
+		&info->ri_done, msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
+	rc = info->ri_rc;
+	if (rc) {
+		log_rdma_event(ERR, "rdma_resolve_route() completed %i\n", rc);
+		goto out;
+	}
+
+	return id;
+
+out:
+	rdma_destroy_id(id);
+	return ERR_PTR(rc);
+}
+
+/*
+ * Test if FRWR (Fast Registration Work Requests) is supported on the device
+ * This implementation requries FRWR on RDMA read/write
+ * return value: true if it is supported
+ */
+static bool frwr_is_supported(struct ib_device_attr *attrs)
+{
+	if (!(attrs->device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS))
+		return false;
+	if (attrs->max_fast_reg_page_list_len == 0)
+		return false;
+	return true;
+}
+
+static int smbd_ia_open(
+		struct smbd_connection *info,
+		struct sockaddr *dstaddr, int port)
+{
+	int rc;
+
+	info->id = smbd_create_id(info, dstaddr, port);
+	if (IS_ERR(info->id)) {
+		rc = PTR_ERR(info->id);
+		goto out1;
+	}
+
+	if (!frwr_is_supported(&info->id->device->attrs)) {
+		log_rdma_event(ERR,
+			"Fast Registration Work Requests "
+			"(FRWR) is not supported\n");
+		log_rdma_event(ERR,
+			"Device capability flags = %llx "
+			"max_fast_reg_page_list_len = %u\n",
+			info->id->device->attrs.device_cap_flags,
+			info->id->device->attrs.max_fast_reg_page_list_len);
+		rc = -EPROTONOSUPPORT;
+		goto out2;
+	}
+
+	info->pd = ib_alloc_pd(info->id->device, 0);
+	if (IS_ERR(info->pd)) {
+		rc = PTR_ERR(info->pd);
+		log_rdma_event(ERR, "ib_alloc_pd() returned %d\n", rc);
+		goto out2;
+	}
+
+	return 0;
+
+out2:
+	rdma_destroy_id(info->id);
+	info->id = NULL;
+
+out1:
+	return rc;
+}
+
+/*
+ * Send a negotiation request message to the peer
+ * The negotiation procedure is in [MS-SMBD] 3.1.5.2 and 3.1.5.3
+ * After negotiation, the transport is connected and ready for
+ * carrying upper layer SMB payload
+ */
+static int smbd_post_send_negotiate_req(struct smbd_connection *info)
+{
+	struct ib_send_wr send_wr, *send_wr_fail;
+	int rc = -ENOMEM;
+	struct smbd_request *request;
+	struct smbd_negotiate_req *packet;
+
+	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
+	if (!request)
+		return rc;
+
+	request->info = info;
+
+	packet = smbd_request_payload(request);
+	packet->min_version = cpu_to_le16(SMBD_V1);
+	packet->max_version = cpu_to_le16(SMBD_V1);
+	packet->reserved = 0;
+	packet->credits_requested = cpu_to_le16(info->send_credit_target);
+	packet->preferred_send_size = cpu_to_le32(info->max_send_size);
+	packet->max_receive_size = cpu_to_le32(info->max_receive_size);
+	packet->max_fragmented_size =
+		cpu_to_le32(info->max_fragmented_recv_size);
+
+	request->num_sge = 1;
+	request->sge[0].addr = ib_dma_map_single(
+				info->id->device, (void *)packet,
+				sizeof(*packet), DMA_TO_DEVICE);
+	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
+		rc = -EIO;
+		goto dma_mapping_failed;
+	}
+
+	request->sge[0].length = sizeof(*packet);
+	request->sge[0].lkey = info->pd->local_dma_lkey;
+
+	ib_dma_sync_single_for_device(
+		info->id->device, request->sge[0].addr,
+		request->sge[0].length, DMA_TO_DEVICE);
+
+	request->cqe.done = send_done;
+
+	send_wr.next = NULL;
+	send_wr.wr_cqe = &request->cqe;
+	send_wr.sg_list = request->sge;
+	send_wr.num_sge = request->num_sge;
+	send_wr.opcode = IB_WR_SEND;
+	send_wr.send_flags = IB_SEND_SIGNALED;
+
+	log_rdma_send(INFO, "sge addr=%llx length=%x lkey=%x\n",
+		request->sge[0].addr,
+		request->sge[0].length, request->sge[0].lkey);
+
+	request->has_payload = false;
+	atomic_inc(&info->send_pending);
+	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
+	if (!rc)
+		return 0;
+
+	/* if we reach here, post send failed */
+	log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
+	atomic_dec(&info->send_pending);
+	ib_dma_unmap_single(info->id->device, request->sge[0].addr,
+		request->sge[0].length, DMA_TO_DEVICE);
+
+dma_mapping_failed:
+	mempool_free(request, info->request_mempool);
+	return rc;
+}
+
+/*
+ * Extend the credits to remote peer
+ * This implements [MS-SMBD] 3.1.5.9
+ * The idea is that we should extend credits to remote peer as quickly as
+ * it's allowed, to maintain data flow. We allocate as much receive
+ * buffer as possible, and extend the receive credits to remote peer
+ * return value: the new credtis being granted.
+ */
+static int manage_credits_prior_sending(struct smbd_connection *info)
+{
+	int new_credits;
+
+	spin_lock(&info->lock_new_credits_offered);
+	new_credits = info->new_credits_offered;
+	info->new_credits_offered = 0;
+	spin_unlock(&info->lock_new_credits_offered);
+
+	return new_credits;
+}
+
+/*
+ * Check if we need to send a KEEP_ALIVE message
+ * The idle connection timer triggers a KEEP_ALIVE message when expires
+ * SMB_DIRECT_RESPONSE_REQUESTED is set in the message flag to have peer send
+ * back a response.
+ * return value:
+ * 1 if SMB_DIRECT_RESPONSE_REQUESTED needs to be set
+ * 0: otherwise
+ */
+static int manage_keep_alive_before_sending(struct smbd_connection *info)
+{
+	if (info->keep_alive_requested == KEEP_ALIVE_PENDING) {
+		info->keep_alive_requested = KEEP_ALIVE_SENT;
+		return 1;
+	}
+	return 0;
+}
+
+/*
+ * Build and prepare the SMBD packet header
+ * This function waits for avaialbe send credits and build a SMBD packet
+ * header. The caller then optional append payload to the packet after
+ * the header
+ * intput values
+ * size: the size of the payload
+ * remaining_data_length: remaining data to send if this is part of a
+ * fragmented packet
+ * output values
+ * request_out: the request allocated from this function
+ * return values: 0 on success, otherwise actual error code returned
+ */
+static int smbd_create_header(struct smbd_connection *info,
+		int size, int remaining_data_length,
+		struct smbd_request **request_out)
+{
+	struct smbd_request *request;
+	struct smbd_data_transfer *packet;
+	int header_length;
+	int rc;
+
+	/* Wait for send credits. A SMBD packet needs one credit */
+	rc = wait_event_interruptible(info->wait_send_queue,
+		atomic_read(&info->send_credits) > 0 ||
+		info->transport_status != SMBD_CONNECTED);
+	if (rc)
+		return rc;
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		log_outgoing(ERR, "disconnected not sending\n");
+		return -ENOENT;
+	}
+	atomic_dec(&info->send_credits);
+
+	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
+	if (!request) {
+		rc = -ENOMEM;
+		goto err;
+	}
+
+	request->info = info;
+
+	/* Fill in the packet header */
+	packet = smbd_request_payload(request);
+	packet->credits_requested = cpu_to_le16(info->send_credit_target);
+	packet->credits_granted =
+		cpu_to_le16(manage_credits_prior_sending(info));
+	info->send_immediate = false;
+
+	packet->flags = 0;
+	if (manage_keep_alive_before_sending(info))
+		packet->flags |= cpu_to_le16(SMB_DIRECT_RESPONSE_REQUESTED);
+
+	packet->reserved = 0;
+	if (!size)
+		packet->data_offset = 0;
+	else
+		packet->data_offset = cpu_to_le32(24);
+	packet->data_length = cpu_to_le32(size);
+	packet->remaining_data_length = cpu_to_le32(remaining_data_length);
+	packet->padding = 0;
+
+	log_outgoing(INFO, "credits_requested=%d credits_granted=%d "
+		"data_offset=%d data_length=%d remaining_data_length=%d\n",
+		le16_to_cpu(packet->credits_requested),
+		le16_to_cpu(packet->credits_granted),
+		le32_to_cpu(packet->data_offset),
+		le32_to_cpu(packet->data_length),
+		le32_to_cpu(packet->remaining_data_length));
+
+	/* Map the packet to DMA */
+	header_length = sizeof(struct smbd_data_transfer);
+	/* If this is a packet without payload, don't send padding */
+	if (!size)
+		header_length = offsetof(struct smbd_data_transfer, padding);
+
+	request->num_sge = 1;
+	request->sge[0].addr = ib_dma_map_single(info->id->device,
+						 (void *)packet,
+						 header_length,
+						 DMA_BIDIRECTIONAL);
+	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr)) {
+		mempool_free(request, info->request_mempool);
+		rc = -EIO;
+		goto err;
+	}
+
+	request->sge[0].length = header_length;
+	request->sge[0].lkey = info->pd->local_dma_lkey;
+
+	*request_out = request;
+	return 0;
+
+err:
+	atomic_inc(&info->send_credits);
+	return rc;
+}
+
+static void smbd_destroy_header(struct smbd_connection *info,
+		struct smbd_request *request)
+{
+
+	ib_dma_unmap_single(info->id->device,
+			    request->sge[0].addr,
+			    request->sge[0].length,
+			    DMA_TO_DEVICE);
+	mempool_free(request, info->request_mempool);
+	atomic_inc(&info->send_credits);
+}
+
+/* Post the send request */
+static int smbd_post_send(struct smbd_connection *info,
+		struct smbd_request *request, bool has_payload)
+{
+	struct ib_send_wr send_wr, *send_wr_fail;
+	int rc, i;
+
+	for (i = 0; i < request->num_sge; i++) {
+		log_rdma_send(INFO,
+			"rdma_request sge[%d] addr=%llu legnth=%u\n",
+			i, request->sge[0].addr, request->sge[0].length);
+		ib_dma_sync_single_for_device(
+			info->id->device,
+			request->sge[i].addr,
+			request->sge[i].length,
+			DMA_TO_DEVICE);
+	}
+
+	request->cqe.done = send_done;
+
+	send_wr.next = NULL;
+	send_wr.wr_cqe = &request->cqe;
+	send_wr.sg_list = request->sge;
+	send_wr.num_sge = request->num_sge;
+	send_wr.opcode = IB_WR_SEND;
+	send_wr.send_flags = IB_SEND_SIGNALED;
+
+	if (has_payload) {
+		request->has_payload = true;
+		atomic_inc(&info->send_payload_pending);
+	} else {
+		request->has_payload = false;
+		atomic_inc(&info->send_pending);
+	}
+
+	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
+	if (rc) {
+		log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
+		if (has_payload) {
+			if (atomic_dec_and_test(&info->send_payload_pending))
+				wake_up(&info->wait_send_payload_pending);
+		} else {
+			if (atomic_dec_and_test(&info->send_pending))
+				wake_up(&info->wait_send_pending);
+		}
+	} else
+		/* Reset timer for idle connection after packet is sent */
+		mod_delayed_work(info->workqueue, &info->idle_timer_work,
+			info->keep_alive_interval*HZ);
+
+	return rc;
+}
+
+static int smbd_post_send_sgl(struct smbd_connection *info,
+	struct scatterlist *sgl, int data_length, int remaining_data_length)
+{
+	int num_sgs;
+	int i, rc;
+	struct smbd_request *request;
+	struct scatterlist *sg;
+
+	rc = smbd_create_header(
+		info, data_length, remaining_data_length, &request);
+	if (rc)
+		return rc;
+
+	num_sgs = sgl ? sg_nents(sgl) : 0;
+	for_each_sg(sgl, sg, num_sgs, i) {
+		request->sge[i+1].addr =
+			ib_dma_map_page(info->id->device, sg_page(sg),
+			       sg->offset, sg->length, DMA_BIDIRECTIONAL);
+		if (ib_dma_mapping_error(
+				info->id->device, request->sge[i+1].addr)) {
+			rc = -EIO;
+			request->sge[i+1].addr = 0;
+			goto dma_mapping_failure;
+		}
+		request->sge[i+1].length = sg->length;
+		request->sge[i+1].lkey = info->pd->local_dma_lkey;
+		request->num_sge++;
+	}
+
+	rc = smbd_post_send(info, request, data_length);
+	if (!rc)
+		return 0;
+
+dma_mapping_failure:
+	for (i = 1; i < request->num_sge; i++)
+		if (request->sge[i].addr)
+			ib_dma_unmap_single(info->id->device,
+					    request->sge[i].addr,
+					    request->sge[i].length,
+					    DMA_TO_DEVICE);
+	smbd_destroy_header(info, request);
+	return rc;
+}
+
+/*
+ * Send an empty message
+ * Empty message is used to extend credits to peer to for keep live
+ * while there is no upper layer payload to send at the time
+ */
+static int smbd_post_send_empty(struct smbd_connection *info)
+{
+	info->count_send_empty++;
+	return smbd_post_send_sgl(info, NULL, 0, 0);
+}
+
+/*
+ * Post a receive request to the transport
+ * The remote peer can only send data when a receive request is posted
+ * The interaction is controlled by send/receive credit system
+ */
+static int smbd_post_recv(
+		struct smbd_connection *info, struct smbd_response *response)
+{
+	struct ib_recv_wr recv_wr, *recv_wr_fail = NULL;
+	int rc = -EIO;
+
+	response->sge.addr = ib_dma_map_single(
+				info->id->device, response->packet,
+				info->max_receive_size, DMA_FROM_DEVICE);
+	if (ib_dma_mapping_error(info->id->device, response->sge.addr))
+		return rc;
+
+	response->sge.length = info->max_receive_size;
+	response->sge.lkey = info->pd->local_dma_lkey;
+
+	response->cqe.done = recv_done;
+
+	recv_wr.wr_cqe = &response->cqe;
+	recv_wr.next = NULL;
+	recv_wr.sg_list = &response->sge;
+	recv_wr.num_sge = 1;
+
+	rc = ib_post_recv(info->id->qp, &recv_wr, &recv_wr_fail);
+	if (rc) {
+		ib_dma_unmap_single(info->id->device, response->sge.addr,
+				    response->sge.length, DMA_FROM_DEVICE);
+
+		log_rdma_recv(ERR, "ib_post_recv failed rc=%d\n", rc);
+	}
+
+	return rc;
+}
+
+/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */
+static int smbd_negotiate(struct smbd_connection *info)
+{
+	int rc;
+	struct smbd_response *response = get_receive_buffer(info);
+
+	response->type = SMBD_NEGOTIATE_RESP;
+	rc = smbd_post_recv(info, response);
+	log_rdma_event(INFO,
+		"smbd_post_recv rc=%d iov.addr=%llx iov.length=%x "
+		"iov.lkey=%x\n",
+		rc, response->sge.addr,
+		response->sge.length, response->sge.lkey);
+	if (rc)
+		return rc;
+
+	init_completion(&info->negotiate_completion);
+	info->negotiate_done = false;
+	rc = smbd_post_send_negotiate_req(info);
+	if (rc)
+		return rc;
+
+	rc = wait_for_completion_interruptible_timeout(
+		&info->negotiate_completion, SMBD_NEGOTIATE_TIMEOUT * HZ);
+	log_rdma_event(INFO, "wait_for_completion_timeout rc=%d\n", rc);
+
+	if (info->negotiate_done)
+		return 0;
+
+	if (rc == 0)
+		rc = -ETIMEDOUT;
+	else if (rc == -ERESTARTSYS)
+		rc = -EINTR;
+	else
+		rc = -ENOTCONN;
+
+	return rc;
+}
+
+static void put_empty_packet(
+		struct smbd_connection *info, struct smbd_response *response)
+{
+	spin_lock(&info->empty_packet_queue_lock);
+	list_add_tail(&response->list, &info->empty_packet_queue);
+	info->count_empty_packet_queue++;
+	spin_unlock(&info->empty_packet_queue_lock);
+
+	queue_work(info->workqueue, &info->post_send_credits_work);
+}
+
+/*
+ * Implement Connection.FragmentReassemblyBuffer defined in [MS-SMBD] 3.1.1.1
+ * This is a queue for reassembling upper layer payload and present to upper
+ * layer. All the inncoming payload go to the reassembly queue, regardless of
+ * if reassembly is required. The uuper layer code reads from the queue for all
+ * incoming payloads.
+ * Put a received packet to the reassembly queue
+ * response: the packet received
+ * data_length: the size of payload in this packet
+ */
+static void enqueue_reassembly(
+	struct smbd_connection *info,
+	struct smbd_response *response,
+	int data_length)
+{
+	spin_lock(&info->reassembly_queue_lock);
+	list_add_tail(&response->list, &info->reassembly_queue);
+	info->reassembly_queue_length++;
+	/*
+	 * Make sure reassembly_data_length is updated after list and
+	 * reassembly_queue_length are updated. On the dequeue side
+	 * reassembly_data_length is checked without a lock to determine
+	 * if reassembly_queue_length and list is up to date
+	 */
+	virt_wmb();
+	info->reassembly_data_length += data_length;
+	spin_unlock(&info->reassembly_queue_lock);
+	info->count_reassembly_queue++;
+	info->count_enqueue_reassembly_queue++;
+}
+
+/*
+ * Get the first entry at the front of reassembly queue
+ * Caller is responsible for locking
+ * return value: the first entry if any, NULL if queue is empty
+ */
+static struct smbd_response *_get_first_reassembly(struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+
+	if (!list_empty(&info->reassembly_queue)) {
+		ret = list_first_entry(
+			&info->reassembly_queue,
+			struct smbd_response, list);
+	}
+	return ret;
+}
+
+static struct smbd_response *get_empty_queue_buffer(
+		struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->empty_packet_queue_lock, flags);
+	if (!list_empty(&info->empty_packet_queue)) {
+		ret = list_first_entry(
+			&info->empty_packet_queue,
+			struct smbd_response, list);
+		list_del(&ret->list);
+		info->count_empty_packet_queue--;
+	}
+	spin_unlock_irqrestore(&info->empty_packet_queue_lock, flags);
+
+	return ret;
+}
+
+/*
+ * Get a receive buffer
+ * For each remote send, we need to post a receive. The receive buffers are
+ * pre-allocated in advance.
+ * return value: the receive buffer, NULL if none is available
+ */
+static struct smbd_response *get_receive_buffer(struct smbd_connection *info)
+{
+	struct smbd_response *ret = NULL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&info->receive_queue_lock, flags);
+	if (!list_empty(&info->receive_queue)) {
+		ret = list_first_entry(
+			&info->receive_queue,
+			struct smbd_response, list);
+		list_del(&ret->list);
+		info->count_receive_queue--;
+		info->count_get_receive_buffer++;
+	}
+	spin_unlock_irqrestore(&info->receive_queue_lock, flags);
+
+	return ret;
+}
+
+/*
+ * Return a receive buffer
+ * Upon returning of a receive buffer, we can post new receive and extend
+ * more receive credits to remote peer. This is done immediately after a
+ * receive buffer is returned.
+ */
+static void put_receive_buffer(
+	struct smbd_connection *info, struct smbd_response *response,
+	bool lock)
+{
+	unsigned long flags;
+
+	ib_dma_unmap_single(info->id->device, response->sge.addr,
+		response->sge.length, DMA_FROM_DEVICE);
+
+	if (lock)
+		spin_lock_irqsave(&info->receive_queue_lock, flags);
+	list_add_tail(&response->list, &info->receive_queue);
+	info->count_receive_queue++;
+	info->count_put_receive_buffer++;
+	if (lock)
+		spin_unlock_irqrestore(&info->receive_queue_lock, flags);
+
+	queue_work(info->workqueue, &info->post_send_credits_work);
+}
+
+/* Preallocate all receive buffer on transport establishment */
+static int allocate_receive_buffers(struct smbd_connection *info, int num_buf)
+{
+	int i;
+	struct smbd_response *response;
+
+	INIT_LIST_HEAD(&info->reassembly_queue);
+	spin_lock_init(&info->reassembly_queue_lock);
+	info->reassembly_data_length = 0;
+	info->reassembly_queue_length = 0;
+
+	INIT_LIST_HEAD(&info->receive_queue);
+	spin_lock_init(&info->receive_queue_lock);
+	info->count_receive_queue = 0;
+
+	INIT_LIST_HEAD(&info->empty_packet_queue);
+	spin_lock_init(&info->empty_packet_queue_lock);
+	info->count_empty_packet_queue = 0;
+
+	init_waitqueue_head(&info->wait_receive_queues);
+
+	for (i = 0; i < num_buf; i++) {
+		response = mempool_alloc(info->response_mempool, GFP_KERNEL);
+		if (!response)
+			goto allocate_failed;
+
+		response->info = info;
+		list_add_tail(&response->list, &info->receive_queue);
+		info->count_receive_queue++;
+	}
+
+	return 0;
+
+allocate_failed:
+	while (!list_empty(&info->receive_queue)) {
+		response = list_first_entry(
+				&info->receive_queue,
+				struct smbd_response, list);
+		list_del(&response->list);
+		info->count_receive_queue--;
+
+		mempool_free(response, info->response_mempool);
+	}
+	return -ENOMEM;
+}
+
+static void destroy_receive_buffers(struct smbd_connection *info)
+{
+	struct smbd_response *response;
+
+	while ((response = get_receive_buffer(info)))
+		mempool_free(response, info->response_mempool);
+
+	while ((response = get_empty_queue_buffer(info)))
+		mempool_free(response, info->response_mempool);
+}
+
+/*
+ * Check and send an immediate or keep alive packet
+ * The condition to send those packets are defined in [MS-SMBD] 3.1.1.1
+ * Connection.KeepaliveRequested and Connection.SendImmediate
+ * The idea is to extend credits to server as soon as it becomes available
+ */
+static void send_immediate_work(struct work_struct *work)
+{
+	struct smbd_connection *info = container_of(
+					work, struct smbd_connection,
+					send_immediate_work.work);
+
+	if (info->keep_alive_requested == KEEP_ALIVE_PENDING ||
+	    info->send_immediate) {
+		log_keep_alive(INFO, "send an empty message\n");
+		smbd_post_send_empty(info);
+	}
+}
+
+/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */
+static void idle_connection_timer(struct work_struct *work)
+{
+	struct smbd_connection *info = container_of(
+					work, struct smbd_connection,
+					idle_timer_work.work);
+
+	if (info->keep_alive_requested != KEEP_ALIVE_NONE) {
+		log_keep_alive(ERR,
+			"error status info->keep_alive_requested=%d\n",
+			info->keep_alive_requested);
+		smbd_disconnect_rdma_connection(info);
+		return;
+	}
+
+	log_keep_alive(INFO, "about to send an empty idle message\n");
+	smbd_post_send_empty(info);
+
+	/* Setup the next idle timeout work */
+	queue_delayed_work(info->workqueue, &info->idle_timer_work,
+			info->keep_alive_interval*HZ);
+}
+
+static void destroy_caches_and_workqueue(struct smbd_connection *info)
+{
+	destroy_receive_buffers(info);
+	destroy_workqueue(info->workqueue);
+	mempool_destroy(info->response_mempool);
+	kmem_cache_destroy(info->response_cache);
+	mempool_destroy(info->request_mempool);
+	kmem_cache_destroy(info->request_cache);
+}
+
+#define MAX_NAME_LEN	80
+static int allocate_caches_and_workqueue(struct smbd_connection *info)
+{
+	char name[MAX_NAME_LEN];
+	int rc;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_request_%p", info);
+	info->request_cache =
+		kmem_cache_create(
+			name,
+			sizeof(struct smbd_request) +
+				sizeof(struct smbd_data_transfer),
+			0, SLAB_HWCACHE_ALIGN, NULL);
+	if (!info->request_cache)
+		return -ENOMEM;
+
+	info->request_mempool =
+		mempool_create(info->send_credit_target, mempool_alloc_slab,
+			mempool_free_slab, info->request_cache);
+	if (!info->request_mempool)
+		goto out1;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_response_%p", info);
+	info->response_cache =
+		kmem_cache_create(
+			name,
+			sizeof(struct smbd_response) +
+				info->max_receive_size,
+			0, SLAB_HWCACHE_ALIGN, NULL);
+	if (!info->response_cache)
+		goto out2;
+
+	info->response_mempool =
+		mempool_create(info->receive_credit_max, mempool_alloc_slab,
+		       mempool_free_slab, info->response_cache);
+	if (!info->response_mempool)
+		goto out3;
+
+	snprintf(name, MAX_NAME_LEN, "smbd_%p", info);
+	info->workqueue = create_workqueue(name);
+	if (!info->workqueue)
+		goto out4;
+
+	rc = allocate_receive_buffers(info, info->receive_credit_max);
+	if (rc) {
+		log_rdma_event(ERR, "failed to allocate receive buffers\n");
+		goto out5;
+	}
+
+	return 0;
+
+out5:
+	destroy_workqueue(info->workqueue);
+out4:
+	mempool_destroy(info->response_mempool);
+out3:
+	kmem_cache_destroy(info->response_cache);
+out2:
+	mempool_destroy(info->request_mempool);
+out1:
+	kmem_cache_destroy(info->request_cache);
+	return -ENOMEM;
+}
+
+/* Create a SMBD connection, called by upper layer */
+struct smbd_connection *_smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
+{
+	int rc;
+	struct smbd_connection *info;
+	struct rdma_conn_param conn_param;
+	struct ib_qp_init_attr qp_attr;
+	struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;
+
+	info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL);
+	if (!info)
+		return NULL;
+
+	info->transport_status = SMBD_CONNECTING;
+	rc = smbd_ia_open(info, dstaddr, port);
+	if (rc) {
+		log_rdma_event(INFO, "smbd_ia_open rc=%d\n", rc);
+		goto create_id_failed;
+	}
+
+	if (smbd_send_credit_target > info->id->device->attrs.max_cqe ||
+	    smbd_send_credit_target > info->id->device->attrs.max_qp_wr) {
+		log_rdma_event(ERR,
+			"consider lowering send_credit_target = %d. "
+			"Possible CQE overrun, device "
+			"reporting max_cpe %d max_qp_wr %d\n",
+			smbd_send_credit_target,
+			info->id->device->attrs.max_cqe,
+			info->id->device->attrs.max_qp_wr);
+		goto config_failed;
+	}
+
+	if (smbd_receive_credit_max > info->id->device->attrs.max_cqe ||
+	    smbd_receive_credit_max > info->id->device->attrs.max_qp_wr) {
+		log_rdma_event(ERR,
+			"consider lowering receive_credit_max = %d. "
+			"Possible CQE overrun, device "
+			"reporting max_cpe %d max_qp_wr %d\n",
+			smbd_receive_credit_max,
+			info->id->device->attrs.max_cqe,
+			info->id->device->attrs.max_qp_wr);
+		goto config_failed;
+	}
+
+	info->receive_credit_max = smbd_receive_credit_max;
+	info->send_credit_target = smbd_send_credit_target;
+	info->max_send_size = smbd_max_send_size;
+	info->max_fragmented_recv_size = smbd_max_fragmented_recv_size;
+	info->max_receive_size = smbd_max_receive_size;
+	info->keep_alive_interval = smbd_keep_alive_interval;
+
+	if (SMBDIRECT_MAX_SGE > info->id->device->attrs.max_sge) {
+		log_rdma_event(ERR, "warning: device max_sge = %d too small\n",
+			info->id->device->attrs.max_sge);
+		log_rdma_event(ERR, "Queue Pair creation may fail\n");
+	}
+
+	info->send_cq = NULL;
+	info->recv_cq = NULL;
+	info->send_cq = ib_alloc_cq(info->id->device, info,
+			info->send_credit_target, 0, IB_POLL_SOFTIRQ);
+	if (IS_ERR(info->send_cq)) {
+		info->send_cq = NULL;
+		goto alloc_cq_failed;
+	}
+
+	info->recv_cq = ib_alloc_cq(info->id->device, info,
+			info->receive_credit_max, 0, IB_POLL_SOFTIRQ);
+	if (IS_ERR(info->recv_cq)) {
+		info->recv_cq = NULL;
+		goto alloc_cq_failed;
+	}
+
+	memset(&qp_attr, 0, sizeof(qp_attr));
+	qp_attr.event_handler = smbd_qp_async_error_upcall;
+	qp_attr.qp_context = info;
+	qp_attr.cap.max_send_wr = info->send_credit_target;
+	qp_attr.cap.max_recv_wr = info->receive_credit_max;
+	qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
+	qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
+	qp_attr.cap.max_inline_data = 0;
+	qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
+	qp_attr.qp_type = IB_QPT_RC;
+	qp_attr.send_cq = info->send_cq;
+	qp_attr.recv_cq = info->recv_cq;
+	qp_attr.port_num = ~0;
+
+	rc = rdma_create_qp(info->id, info->pd, &qp_attr);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_create_qp failed %i\n", rc);
+		goto create_qp_failed;
+	}
+
+	memset(&conn_param, 0, sizeof(conn_param));
+	conn_param.initiator_depth = 0;
+
+	conn_param.retry_count = SMBD_CM_RETRY;
+	conn_param.rnr_retry_count = SMBD_CM_RNR_RETRY;
+	conn_param.flow_control = 0;
+	init_waitqueue_head(&info->wait_destroy);
+
+	log_rdma_event(INFO, "connecting to IP %pI4 port %d\n",
+		&addr_in->sin_addr, port);
+
+	init_waitqueue_head(&info->conn_wait);
+	rc = rdma_connect(info->id, &conn_param);
+	if (rc) {
+		log_rdma_event(ERR, "rdma_connect() failed with %i\n", rc);
+		goto rdma_connect_failed;
+	}
+
+	wait_event_interruptible(
+		info->conn_wait, info->transport_status != SMBD_CONNECTING);
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		log_rdma_event(ERR, "rdma_connect failed port=%d\n", port);
+		goto rdma_connect_failed;
+	}
+
+	log_rdma_event(INFO, "rdma_connect connected\n");
+
+	rc = allocate_caches_and_workqueue(info);
+	if (rc) {
+		log_rdma_event(ERR, "cache allocation failed\n");
+		goto allocate_cache_failed;
+	}
+
+	init_waitqueue_head(&info->wait_send_queue);
+	init_waitqueue_head(&info->wait_reassembly_queue);
+
+	INIT_DELAYED_WORK(&info->idle_timer_work, idle_connection_timer);
+	INIT_DELAYED_WORK(&info->send_immediate_work, send_immediate_work);
+	queue_delayed_work(info->workqueue, &info->idle_timer_work,
+		info->keep_alive_interval*HZ);
+
+	init_waitqueue_head(&info->wait_send_pending);
+	atomic_set(&info->send_pending, 0);
+
+	init_waitqueue_head(&info->wait_send_payload_pending);
+	atomic_set(&info->send_payload_pending, 0);
+
+	INIT_WORK(&info->disconnect_work, smbd_disconnect_rdma_work);
+	INIT_WORK(&info->destroy_work, smbd_destroy_rdma_work);
+	INIT_WORK(&info->recv_done_work, smbd_recv_done_work);
+	INIT_WORK(&info->post_send_credits_work, smbd_post_send_credits);
+	info->new_credits_offered = 0;
+	spin_lock_init(&info->lock_new_credits_offered);
+
+	rc = smbd_negotiate(info);
+	if (rc) {
+		log_rdma_event(ERR, "smbd_negotiate rc=%d\n", rc);
+		goto negotiation_failed;
+	}
+
+	return info;
+
+negotiation_failed:
+	cancel_delayed_work_sync(&info->idle_timer_work);
+	destroy_caches_and_workqueue(info);
+	info->transport_status = SMBD_NEGOTIATE_FAILED;
+	init_waitqueue_head(&info->conn_wait);
+	rdma_disconnect(info->id);
+	wait_event(info->conn_wait,
+		info->transport_status == SMBD_DISCONNECTED);
+
+allocate_cache_failed:
+rdma_connect_failed:
+	rdma_destroy_qp(info->id);
+
+create_qp_failed:
+alloc_cq_failed:
+	if (info->send_cq)
+		ib_free_cq(info->send_cq);
+	if (info->recv_cq)
+		ib_free_cq(info->recv_cq);
+
+config_failed:
+	ib_dealloc_pd(info->pd);
+	rdma_destroy_id(info->id);
+
+create_id_failed:
+	kfree(info);
+	return NULL;
+}
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
index c55f28b..35bc25b 100644
--- a/fs/cifs/smbdirect.h
+++ b/fs/cifs/smbdirect.h
@@ -16,6 +16,286 @@
 #ifndef _SMBDIRECT_H
 #define _SMBDIRECT_H
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#define cifs_rdma_enabled(server)	((server)->rdma)
+
+#include "cifsglob.h"
+#include <rdma/ib_verbs.h>
+#include <rdma/rdma_cm.h>
+#include <linux/mempool.h>
+
+enum keep_alive_status {
+	KEEP_ALIVE_NONE,
+	KEEP_ALIVE_PENDING,
+	KEEP_ALIVE_SENT,
+};
+
+enum smbd_connection_status {
+	SMBD_CREATED,
+	SMBD_CONNECTING,
+	SMBD_CONNECTED,
+	SMBD_NEGOTIATE_FAILED,
+	SMBD_DISCONNECTING,
+	SMBD_DISCONNECTED,
+	SMBD_DESTROYED
+};
+
+/*
+ * The context for the SMBDirect transport
+ * Everything related to the transport is here. It has several logical parts
+ * 1. RDMA related structures
+ * 2. SMBDirect connection parameters
+ * 3. Memory registrations
+ * 4. Receive and reassembly queues for data receive path
+ * 5. mempools for allocating packets
+ */
+struct smbd_connection {
+	enum smbd_connection_status transport_status;
+
+	/* RDMA related */
+	struct rdma_cm_id *id;
+	struct ib_qp_init_attr qp_attr;
+	struct ib_pd *pd;
+	struct ib_cq *send_cq, *recv_cq;
+	struct ib_device_attr dev_attr;
+	int ri_rc;
+	struct completion ri_done;
+	wait_queue_head_t conn_wait;
+	wait_queue_head_t wait_destroy;
+
+	struct completion negotiate_completion;
+	bool negotiate_done;
+
+	struct work_struct destroy_work;
+	struct work_struct disconnect_work;
+	struct work_struct recv_done_work;
+	struct work_struct post_send_credits_work;
+
+	spinlock_t lock_new_credits_offered;
+	int new_credits_offered;
+
+	/* Connection parameters defined in [MS-SMBD] 3.1.1.1 */
+	int receive_credit_max;
+	int send_credit_target;
+	int max_send_size;
+	int max_fragmented_recv_size;
+	int max_fragmented_send_size;
+	int max_receive_size;
+	int keep_alive_interval;
+	int max_readwrite_size;
+	enum keep_alive_status keep_alive_requested;
+	int protocol;
+	atomic_t send_credits;
+	atomic_t receive_credits;
+	int receive_credit_target;
+	int fragment_reassembly_remaining;
+
+	/* Activity accoutning */
+
+	atomic_t send_pending;
+	wait_queue_head_t wait_send_pending;
+	atomic_t send_payload_pending;
+	wait_queue_head_t wait_send_payload_pending;
+
+	/* Receive queue */
+	struct list_head receive_queue;
+	int count_receive_queue;
+	spinlock_t receive_queue_lock;
+
+	struct list_head empty_packet_queue;
+	int count_empty_packet_queue;
+	spinlock_t empty_packet_queue_lock;
+
+	wait_queue_head_t wait_receive_queues;
+
+	/* Reassembly queue */
+	struct list_head reassembly_queue;
+	spinlock_t reassembly_queue_lock;
+	wait_queue_head_t wait_reassembly_queue;
+
+	/* total data length of reassembly queue */
+	int reassembly_data_length;
+	int reassembly_queue_length;
+	/* the offset to first buffer in reassembly queue */
+	int first_entry_offset;
+
+	bool send_immediate;
+
+	wait_queue_head_t wait_send_queue;
+
+	/*
+	 * Indicate if we have received a full packet on the connection
+	 * This is used to identify the first SMBD packet of a assembled
+	 * payload (SMB packet) in reassembly queue so we can return a
+	 * RFC1002 length to upper layer to indicate the length of the SMB
+	 * packet received
+	 */
+	bool full_packet_received;
+
+	struct workqueue_struct *workqueue;
+	struct delayed_work idle_timer_work;
+	struct delayed_work send_immediate_work;
+
+	/* Memory pool for preallocating buffers */
+	/* request pool for RDMA send */
+	struct kmem_cache *request_cache;
+	mempool_t *request_mempool;
+
+	/* response pool for RDMA receive */
+	struct kmem_cache *response_cache;
+	mempool_t *response_mempool;
+
+	/* for debug purposes */
+	unsigned int count_get_receive_buffer;
+	unsigned int count_put_receive_buffer;
+	unsigned int count_reassembly_queue;
+	unsigned int count_enqueue_reassembly_queue;
+	unsigned int count_dequeue_reassembly_queue;
+	unsigned int count_send_empty;
+};
+
+enum smbd_message_type {
+	SMBD_NEGOTIATE_RESP,
+	SMBD_TRANSFER_DATA,
+};
+
+#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
+
+/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */
+struct smbd_negotiate_req {
+	__le16 min_version;
+	__le16 max_version;
+	__le16 reserved;
+	__le16 credits_requested;
+	__le32 preferred_send_size;
+	__le32 max_receive_size;
+	__le32 max_fragmented_size;
+} __packed;
+
+/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */
+struct smbd_negotiate_resp {
+	__le16 min_version;
+	__le16 max_version;
+	__le16 negotiated_version;
+	__le16 reserved;
+	__le16 credits_requested;
+	__le16 credits_granted;
+	__le32 status;
+	__le32 max_readwrite_size;
+	__le32 preferred_send_size;
+	__le32 max_receive_size;
+	__le32 max_fragmented_size;
+} __packed;
+
+/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */
+struct smbd_data_transfer {
+	__le16 credits_requested;
+	__le16 credits_granted;
+	__le16 flags;
+	__le16 reserved;
+	__le32 remaining_data_length;
+	__le32 data_offset;
+	__le32 data_length;
+	__le32 padding;
+	__u8 buffer[];
+} __packed;
+
+/* The packet fields for a registered RDMA buffer */
+struct smbd_buffer_descriptor_v1 {
+	__le64 offset;
+	__le32 token;
+	__le32 length;
+} __packed;
+
 /* Default maximum number of SGEs in a RDMA send/recv */
 #define SMBDIRECT_MAX_SGE	16
+/* The context for a SMBD request */
+struct smbd_request {
+	struct smbd_connection *info;
+	struct ib_cqe cqe;
+
+	/* true if this request carries upper layer payload */
+	bool has_payload;
+
+	/* the SGE entries for this packet */
+	struct ib_sge sge[SMBDIRECT_MAX_SGE];
+	int num_sge;
+
+	/* SMBD packet header follows this structure */
+	u8 packet[];
+};
+
+/* The context for a SMBD response */
+struct smbd_response {
+	struct smbd_connection *info;
+	struct ib_cqe cqe;
+	struct ib_sge sge;
+
+	enum smbd_message_type type;
+
+	/* Link to receive queue or reassembly queue */
+	struct list_head list;
+
+	/* Indicate if this is the 1st packet of a payload */
+	bool first_segment;
+
+	/* SMBD packet header and payload follows this structure */
+	u8 packet[];
+};
+
+/* Create a SMBDirect session */
+struct smbd_connection *smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr);
+
+/* Reconnect SMBDirect session */
+int smbd_reconnect(struct TCP_Server_Info *server);
+
+/* Destroy SMBDirect session */
+void smbd_destroy(struct smbd_connection *info);
+
+/* Interface for carrying upper layer I/O through send/recv */
+int smbd_recv(struct smbd_connection *info, struct msghdr *msg);
+int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst);
+
+enum mr_state {
+	MR_READY,
+	MR_REGISTERED,
+	MR_INVALIDATED,
+	MR_ERROR
+};
+
+struct smbd_mr {
+	struct smbd_connection	*conn;
+	struct list_head	list;
+	enum mr_state		state;
+	struct ib_mr		*mr;
+	struct scatterlist	*sgl;
+	int			sgl_count;
+	enum dma_data_direction	dir;
+	union {
+		struct ib_reg_wr	wr;
+		struct ib_send_wr	inv_wr;
+	};
+	struct ib_cqe		cqe;
+	bool			need_invalidate;
+	struct completion	invalidate_done;
+};
+
+/* Interfaces to register and deregister MR for RDMA read/write */
+struct smbd_mr *smbd_register_mr(
+	struct smbd_connection *info, struct page *pages[], int num_pages,
+	int tailsz, bool writing, bool need_invalidate);
+int smbd_deregister_mr(struct smbd_mr *mr);
+
+#else
+#define cifs_rdma_enabled(server)	0
+struct smbd_connection{};
+static inline void *smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr) {return NULL;}
+static inline int smbd_reconnect(struct TCP_Server_Info *server) {return -1;}
+static inline void smbd_destroy(struct smbd_connection *info) {}
+static inline int smbd_recv(struct smbd_connection *info, struct msghdr *msg) {return -1;}
+static inline int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst) {return -1;}
+#endif
+
 #endif
-- 
2.7.4

RE: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

[Long Li]

> -----Original Message-----
> From: Leif Sahlberg [mailto:lsahlber@redhat.com]
> Sent: Sunday, November 19, 2017 9:47 PM
> To: Long Li <longli@microsoft.com>
> Cc: Steve French <sfrench@samba.org>; linux-cifs@vger.kernel.org; linux-
> kernel@vger.kernel.org; linux-rdma@vger.kernel.org; Christoph Hellwig
> <hch@infradead.org>; Tom Talpey <ttalpey@microsoft.com>; Matthew
> Wilcox <mawilcox@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; Long Li <longli@microsoft.com>
> Subject: Re: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection
> 
> Acked-by: Ronnie Sahlberg <lsahlber@gmail.com>
> 
> But two tiny nits:
> + * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
> Should be 0xa0
> 
> +//	queue_work(info->workqueue, &info->destroy_work);
> Don't leave commented out code in there.

I will address those. Thanks!

> 
> 
> ----- Original Message -----
> From: "Long Li" <longli@exchange.microsoft.com>
> To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org,
> samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-
> rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom
> Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox"
> <mawilcox@microsoft.com>, "Stephen Hemminger"
> <sthemmin@microsoft.com>
> Cc: "Long Li" <longli@microsoft.com>
> Sent: Tuesday, 7 November, 2017 7:54:57 PM
> Subject: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection
> 
> From: Long Li <longli@microsoft.com>
> 
> Add code to implement the core functions to establish a SMB Direct
> connection.
> 
> 1. Establish an RDMA connection to SMB server.
> 2. Negotiate and setup SMB Direct protocol.
> 3. Implement idle connection timer and credit management.
> 
> SMB Direct is enabled by setting CONFIG_CIFS_SMB_DIRECT.
> 
> Add to Makefile to enable building SMB Direct.
> 
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>  fs/cifs/Makefile    |    2 +
>  fs/cifs/smbdirect.c | 1576
> +++++++++++++++++++++++++++++++++++++++++++++++++++
>  fs/cifs/smbdirect.h |  280 +++++++++
>  3 files changed, 1858 insertions(+)
> 
> diff --git a/fs/cifs/Makefile b/fs/cifs/Makefile
> index 5e853a3..ad00873 100644
> --- a/fs/cifs/Makefile
> +++ b/fs/cifs/Makefile
> @@ -18,3 +18,5 @@ cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
>  cifs-$(CONFIG_CIFS_DFS_UPCALL) += dns_resolve.o cifs_dfs_ref.o
> 
>  cifs-$(CONFIG_CIFS_FSCACHE) += fscache.o cache.o
> +
> +cifs-$(CONFIG_CIFS_SMB_DIRECT) += smbdirect.o
> diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
> index d3c16f8..021d527 100644
> --- a/fs/cifs/smbdirect.c
> +++ b/fs/cifs/smbdirect.c
> @@ -13,7 +13,34 @@
>   *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
>   *   the GNU General Public License for more details.
>   */
> +#include <linux/module.h>
>  #include "smbdirect.h"
> +#include "cifs_debug.h"
> +
> +static struct smbd_response *get_empty_queue_buffer(
> +		struct smbd_connection *info);
> +static struct smbd_response *get_receive_buffer(
> +		struct smbd_connection *info);
> +static void put_receive_buffer(
> +		struct smbd_connection *info,
> +		struct smbd_response *response,
> +		bool lock);
> +static int allocate_receive_buffers(struct smbd_connection *info, int
> num_buf);
> +static void destroy_receive_buffers(struct smbd_connection *info);
> +
> +static void put_empty_packet(
> +		struct smbd_connection *info, struct smbd_response
> *response);
> +static void enqueue_reassembly(
> +		struct smbd_connection *info,
> +		struct smbd_response *response, int data_length);
> +static struct smbd_response *_get_first_reassembly(
> +		struct smbd_connection *info);
> +
> +static int smbd_post_recv(
> +		struct smbd_connection *info,
> +		struct smbd_response *response);
> +
> +static int smbd_post_send_empty(struct smbd_connection *info);
> 
>  /* SMBD version number */
>  #define SMBD_V1	0x0100
> @@ -75,3 +102,1552 @@ int smbd_max_frmr_depth = 2048;
> 
>  /* If payload is less than this byte, use RDMA send/recv not read/write */
>  int rdma_readwrite_threshold = 4096;
> +
> +/* Transport logging functions
> + * Logging are defined as classes. They can be OR'ed to define the actual
> + * logging level via module parameter smbd_logging_class
> + * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
> + * log_rdma_event()
> + */
> +#define LOG_OUTGOING			0x1
> +#define LOG_INCOMING			0x2
> +#define LOG_READ			0x4
> +#define LOG_WRITE			0x8
> +#define LOG_RDMA_SEND			0x10
> +#define LOG_RDMA_RECV			0x20
> +#define LOG_KEEP_ALIVE			0x40
> +#define LOG_RDMA_EVENT			0x80
> +#define LOG_RDMA_MR			0x100
> +static unsigned int smbd_logging_class = 0;
> +module_param(smbd_logging_class, uint, 0644);
> +MODULE_PARM_DESC(smbd_logging_class,
> +	"Logging class for SMBD transport 0x0 to 0x100");
> +
> +#define ERR		0x0
> +#define INFO		0x1
> +static unsigned int smbd_logging_level = ERR;
> +module_param(smbd_logging_level, uint, 0644);
> +MODULE_PARM_DESC(smbd_logging_level,
> +	"Logging level for SMBD transport, 0 (default): error, 1: info");
> +
> +#define log_rdma(level, class, fmt, args...)				\
> +do {									\
> +	if (level <= smbd_logging_level || class & smbd_logging_class)	\
> +		cifs_dbg(VFS, "%s:%d " fmt, __func__, __LINE__, ##args);\
> +} while (0)
> +
> +#define log_outgoing(level, fmt, args...) \
> +		log_rdma(level, LOG_OUTGOING, fmt, ##args)
> +#define log_incoming(level, fmt, args...) \
> +		log_rdma(level, LOG_INCOMING, fmt, ##args)
> +#define log_read(level, fmt, args...)	log_rdma(level, LOG_READ, fmt,
> ##args)
> +#define log_write(level, fmt, args...)	log_rdma(level, LOG_WRITE, fmt,
> ##args)
> +#define log_rdma_send(level, fmt, args...) \
> +		log_rdma(level, LOG_RDMA_SEND, fmt, ##args)
> +#define log_rdma_recv(level, fmt, args...) \
> +		log_rdma(level, LOG_RDMA_RECV, fmt, ##args)
> +#define log_keep_alive(level, fmt, args...) \
> +		log_rdma(level, LOG_KEEP_ALIVE, fmt, ##args)
> +#define log_rdma_event(level, fmt, args...) \
> +		log_rdma(level, LOG_RDMA_EVENT, fmt, ##args)
> +#define log_rdma_mr(level, fmt, args...) \
> +		log_rdma(level, LOG_RDMA_MR, fmt, ##args)
> +
> +/*
> + * Destroy the transport and related RDMA and memory resources
> + * Need to go through all the pending counters and make sure on one is
> using
> + * the transport while it is destroyed
> + */
> +static void smbd_destroy_rdma_work(struct work_struct *work)
> +{
> +	struct smbd_response *response;
> +	struct smbd_connection *info =
> +		container_of(work, struct smbd_connection, destroy_work);
> +	unsigned long flags;
> +
> +	log_rdma_event(INFO, "destroying qp\n");
> +	ib_drain_qp(info->id->qp);
> +	rdma_destroy_qp(info->id);
> +
> +	/* Unblock all I/O waiting on the send queue */
> +	wake_up_interruptible_all(&info->wait_send_queue);
> +
> +	log_rdma_event(INFO, "cancelling idle timer\n");
> +	cancel_delayed_work_sync(&info->idle_timer_work);
> +	log_rdma_event(INFO, "cancelling send immediate work\n");
> +	cancel_delayed_work_sync(&info->send_immediate_work);
> +
> +	log_rdma_event(INFO, "wait for all recv to finish\n");
> +	wake_up_interruptible(&info->wait_reassembly_queue);
> +
> +	log_rdma_event(INFO, "wait for all send posted to IB to finish\n");
> +	wait_event(info->wait_send_pending,
> +		atomic_read(&info->send_pending) == 0);
> +	wait_event(info->wait_send_payload_pending,
> +		atomic_read(&info->send_payload_pending) == 0);
> +
> +	/* It's not posssible for upper layer to get to reassembly */
> +	log_rdma_event(INFO, "drain the reassembly queue\n");
> +	do {
> +		spin_lock_irqsave(&info->reassembly_queue_lock, flags);
> +		response = _get_first_reassembly(info);
> +		if (response) {
> +			list_del(&response->list);
> +			spin_unlock_irqrestore(
> +				&info->reassembly_queue_lock, flags);
> +			put_receive_buffer(info, response, true);
> +		}
> +	} while (response);
> +	spin_unlock_irqrestore(&info->reassembly_queue_lock, flags);
> +	info->reassembly_data_length = 0;
> +
> +	log_rdma_event(INFO, "free receive buffers\n");
> +	wait_event(info->wait_receive_queues,
> +		info->count_receive_queue + info-
> >count_empty_packet_queue
> +			== info->receive_credit_max);
> +	destroy_receive_buffers(info);
> +
> +	ib_free_cq(info->send_cq);
> +	ib_free_cq(info->recv_cq);
> +	ib_dealloc_pd(info->pd);
> +	rdma_destroy_id(info->id);
> +
> +	/* free mempools */
> +	mempool_destroy(info->request_mempool);
> +	kmem_cache_destroy(info->request_cache);
> +
> +	mempool_destroy(info->response_mempool);
> +	kmem_cache_destroy(info->response_cache);
> +
> +	info->transport_status = SMBD_DESTROYED;
> +	wake_up_all(&info->wait_destroy);
> +}
> +
> +static int smbd_process_disconnected(struct smbd_connection *info)
> +{
> +//	queue_work(info->workqueue, &info->destroy_work);
> +	schedule_work(&info->destroy_work);
> +	return 0;
> +}
> +
> +static void smbd_disconnect_rdma_work(struct work_struct *work)
> +{
> +	struct smbd_connection *info =
> +		container_of(work, struct smbd_connection,
> disconnect_work);
> +
> +	if (info->transport_status == SMBD_CONNECTED) {
> +		info->transport_status = SMBD_DISCONNECTING;
> +		rdma_disconnect(info->id);
> +	}
> +}
> +
> +static void smbd_disconnect_rdma_connection(struct smbd_connection
> *info)
> +{
> +	queue_work(info->workqueue, &info->disconnect_work);
> +}
> +
> +/* Upcall from RDMA CM */
> +static int smbd_conn_upcall(
> +		struct rdma_cm_id *id, struct rdma_cm_event *event)
> +{
> +	struct smbd_connection *info = id->context;
> +
> +	log_rdma_event(INFO, "event=%d status=%d\n",
> +		event->event, event->status);
> +
> +	switch (event->event) {
> +	case RDMA_CM_EVENT_ADDR_RESOLVED:
> +	case RDMA_CM_EVENT_ROUTE_RESOLVED:
> +		info->ri_rc = 0;
> +		complete(&info->ri_done);
> +		break;
> +
> +	case RDMA_CM_EVENT_ADDR_ERROR:
> +		info->ri_rc = -EHOSTUNREACH;
> +		complete(&info->ri_done);
> +		break;
> +
> +	case RDMA_CM_EVENT_ROUTE_ERROR:
> +		info->ri_rc = -ENETUNREACH;
> +		complete(&info->ri_done);
> +		break;
> +
> +	case RDMA_CM_EVENT_ESTABLISHED:
> +		log_rdma_event(INFO, "connected event=%d\n", event-
> >event);
> +		info->transport_status = SMBD_CONNECTED;
> +		wake_up_interruptible(&info->conn_wait);
> +		break;
> +
> +	case RDMA_CM_EVENT_CONNECT_ERROR:
> +	case RDMA_CM_EVENT_UNREACHABLE:
> +	case RDMA_CM_EVENT_REJECTED:
> +		log_rdma_event(INFO, "connecting failed event=%d\n",
> event->event);
> +		info->transport_status = SMBD_DISCONNECTED;
> +		wake_up_interruptible(&info->conn_wait);
> +		break;
> +
> +	case RDMA_CM_EVENT_DEVICE_REMOVAL:
> +	case RDMA_CM_EVENT_DISCONNECTED:
> +		/* This happenes when we fail the negotiation */
> +		if (info->transport_status == SMBD_NEGOTIATE_FAILED) {
> +			info->transport_status = SMBD_DISCONNECTED;
> +			wake_up(&info->conn_wait);
> +			break;
> +		}
> +
> +		info->transport_status = SMBD_DISCONNECTED;
> +		smbd_process_disconnected(info);
> +		break;
> +
> +	default:
> +		break;
> +	}
> +
> +	return 0;
> +}
> +
> +/* Upcall from RDMA QP */
> +static void
> +smbd_qp_async_error_upcall(struct ib_event *event, void *context)
> +{
> +	struct smbd_connection *info = context;
> +
> +	log_rdma_event(ERR, "%s on device %s info %p\n",
> +		ib_event_msg(event->event), event->device->name, info);
> +
> +	switch (event->event) {
> +	case IB_EVENT_CQ_ERR:
> +	case IB_EVENT_QP_FATAL:
> +		smbd_disconnect_rdma_connection(info);
> +
> +	default:
> +		break;
> +	}
> +}
> +
> +static inline void *smbd_request_payload(struct smbd_request *request)
> +{
> +	return (void *)request->packet;
> +}
> +
> +static inline void *smbd_response_payload(struct smbd_response
> *response)
> +{
> +	return (void *)response->packet;
> +}
> +
> +/* Called when a RDMA send is done */
> +static void send_done(struct ib_cq *cq, struct ib_wc *wc)
> +{
> +	int i;
> +	struct smbd_request *request =
> +		container_of(wc->wr_cqe, struct smbd_request, cqe);
> +
> +	log_rdma_send(INFO, "smbd_request %p completed wc-
> >status=%d\n",
> +		request, wc->status);
> +
> +	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_SEND) {
> +		log_rdma_send(ERR, "wc->status=%d wc->opcode=%d\n",
> +			wc->status, wc->opcode);
> +		smbd_disconnect_rdma_connection(request->info);
> +	}
> +
> +	for (i = 0; i < request->num_sge; i++)
> +		ib_dma_unmap_single(request->info->id->device,
> +			request->sge[i].addr,
> +			request->sge[i].length,
> +			DMA_TO_DEVICE);
> +
> +	if (request->has_payload) {
> +		if (atomic_dec_and_test(&request->info-
> >send_payload_pending))
> +			wake_up(&request->info-
> >wait_send_payload_pending);
> +	} else {
> +		if (atomic_dec_and_test(&request->info->send_pending))
> +			wake_up(&request->info->wait_send_pending);
> +	}
> +
> +	mempool_free(request, request->info->request_mempool);
> +}
> +
> +static void dump_smbd_negotiate_resp(struct smbd_negotiate_resp
> *resp)
> +{
> +	log_rdma_event(INFO, "resp message min_version %u max_version
> %u "
> +		"negotiated_version %u credits_requested %u "
> +		"credits_granted %u status %u max_readwrite_size %u "
> +		"preferred_send_size %u max_receive_size %u "
> +		"max_fragmented_size %u\n",
> +		resp->min_version, resp->max_version, resp-
> >negotiated_version,
> +		resp->credits_requested, resp->credits_granted, resp-
> >status,
> +		resp->max_readwrite_size, resp->preferred_send_size,
> +		resp->max_receive_size, resp->max_fragmented_size);
> +}
> +
> +/*
> + * Process a negotiation response message, according to [MS-SMBD]3.1.5.7
> + * response, packet_length: the negotiation response message
> + * return value: true if negotiation is a success, false if failed
> + */
> +static bool process_negotiation_response(
> +		struct smbd_response *response, int packet_length)
> +{
> +	struct smbd_connection *info = response->info;
> +	struct smbd_negotiate_resp *packet =
> smbd_response_payload(response);
> +
> +	if (packet_length < sizeof(struct smbd_negotiate_resp)) {
> +		log_rdma_event(ERR,
> +			"error: packet_length=%d\n", packet_length);
> +		return false;
> +	}
> +
> +	if (le16_to_cpu(packet->negotiated_version) != SMBD_V1) {
> +		log_rdma_event(ERR, "error: negotiated_version=%x\n",
> +			le16_to_cpu(packet->negotiated_version));
> +		return false;
> +	}
> +	info->protocol = le16_to_cpu(packet->negotiated_version);
> +
> +	if (packet->credits_requested == 0) {
> +		log_rdma_event(ERR, "error: credits_requested==0\n");
> +		return false;
> +	}
> +	info->receive_credit_target = le16_to_cpu(packet-
> >credits_requested);
> +
> +	if (packet->credits_granted == 0) {
> +		log_rdma_event(ERR, "error: credits_granted==0\n");
> +		return false;
> +	}
> +	atomic_set(&info->send_credits, le16_to_cpu(packet-
> >credits_granted));
> +
> +	atomic_set(&info->receive_credits, 0);
> +
> +	if (le32_to_cpu(packet->preferred_send_size) > info-
> >max_receive_size) {
> +		log_rdma_event(ERR, "error: preferred_send_size=%d\n",
> +			le32_to_cpu(packet->preferred_send_size));
> +		return false;
> +	}
> +	info->max_receive_size = le32_to_cpu(packet-
> >preferred_send_size);
> +
> +	if (le32_to_cpu(packet->max_receive_size) <
> SMBD_MIN_RECEIVE_SIZE) {
> +		log_rdma_event(ERR, "error: max_receive_size=%d\n",
> +			le32_to_cpu(packet->max_receive_size));
> +		return false;
> +	}
> +	info->max_send_size = min_t(int, info->max_send_size,
> +					le32_to_cpu(packet-
> >max_receive_size));
> +
> +	if (le32_to_cpu(packet->max_fragmented_size) <
> +			SMBD_MIN_FRAGMENTED_SIZE) {
> +		log_rdma_event(ERR, "error: max_fragmented_size=%d\n",
> +			le32_to_cpu(packet->max_fragmented_size));
> +		return false;
> +	}
> +	info->max_fragmented_send_size =
> +		le32_to_cpu(packet->max_fragmented_size);
> +
> +	return true;
> +}
> +
> +/*
> + * Check and schedule to send an immediate packet
> + * This is used to extend credtis to remote peer to keep the transport busy
> + */
> +static void check_and_send_immediate(struct smbd_connection *info)
> +{
> +	if (info->transport_status != SMBD_CONNECTED)
> +		return;
> +
> +	info->send_immediate = true;
> +
> +	/*
> +	 * Promptly send a packet if our peer is running low on receive
> +	 * credits
> +	 */
> +	if (atomic_read(&info->receive_credits) <
> +		info->receive_credit_target - 1)
> +		queue_delayed_work(
> +			info->workqueue, &info->send_immediate_work,
> 0);
> +}
> +
> +static void smbd_post_send_credits(struct work_struct *work)
> +{
> +	int ret = 0;
> +	int use_receive_queue = 1;
> +	int rc;
> +	struct smbd_response *response;
> +	struct smbd_connection *info =
> +		container_of(work, struct smbd_connection,
> +			post_send_credits_work);
> +
> +	if (info->transport_status != SMBD_CONNECTED) {
> +		wake_up(&info->wait_receive_queues);
> +		return;
> +	}
> +
> +	if (info->receive_credit_target >
> +		atomic_read(&info->receive_credits)) {
> +		while (true) {
> +			if (use_receive_queue)
> +				response = get_receive_buffer(info);
> +			else
> +				response = get_empty_queue_buffer(info);
> +			if (!response) {
> +				/* now switch to emtpy packet queue */
> +				if (use_receive_queue) {
> +					use_receive_queue = 0;
> +					continue;
> +				} else
> +					break;
> +			}
> +
> +			response->type = SMBD_TRANSFER_DATA;
> +			response->first_segment = false;
> +			rc = smbd_post_recv(info, response);
> +			if (rc) {
> +				log_rdma_recv(ERR,
> +					"post_recv failed rc=%d\n", rc);
> +				put_receive_buffer(info, response, true);
> +				break;
> +			}
> +
> +			ret++;
> +		}
> +	}
> +
> +	spin_lock(&info->lock_new_credits_offered);
> +	info->new_credits_offered += ret;
> +	spin_unlock(&info->lock_new_credits_offered);
> +
> +	atomic_add(ret, &info->receive_credits);
> +
> +	/* Check if we can post new receive and grant credits to peer */
> +	check_and_send_immediate(info);
> +}
> +
> +static void smbd_recv_done_work(struct work_struct *work)
> +{
> +	struct smbd_connection *info =
> +		container_of(work, struct smbd_connection,
> recv_done_work);
> +
> +	/*
> +	 * We may have new send credits granted from remote peer
> +	 * If any sender is blcoked on lack of credets, unblock it
> +	 */
> +	if (atomic_read(&info->send_credits))
> +		wake_up_interruptible(&info->wait_send_queue);
> +
> +	/*
> +	 * Check if we need to send something to remote peer to
> +	 * grant more credits or respond to KEEP_ALIVE packet
> +	 */
> +	check_and_send_immediate(info);
> +}
> +
> +/* Called from softirq, when recv is done */
> +static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> +{
> +	struct smbd_data_transfer *data_transfer;
> +	struct smbd_response *response =
> +		container_of(wc->wr_cqe, struct smbd_response, cqe);
> +	struct smbd_connection *info = response->info;
> +	int data_length = 0;
> +
> +	log_rdma_recv(INFO, "response=%p type=%d wc status=%d wc
> opcode %d "
> +		      "byte_len=%d pkey_index=%x\n",
> +		response, response->type, wc->status, wc->opcode,
> +		wc->byte_len, wc->pkey_index);
> +
> +	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
> +		log_rdma_recv(INFO, "wc->status=%d opcode=%d\n",
> +			wc->status, wc->opcode);
> +		smbd_disconnect_rdma_connection(info);
> +		goto error;
> +	}
> +
> +	ib_dma_sync_single_for_cpu(
> +		wc->qp->device,
> +		response->sge.addr,
> +		response->sge.length,
> +		DMA_FROM_DEVICE);
> +
> +	switch (response->type) {
> +	/* SMBD negotiation response */
> +	case SMBD_NEGOTIATE_RESP:
> +
> 	dump_smbd_negotiate_resp(smbd_response_payload(response));
> +		info->full_packet_received = true;
> +		info->negotiate_done =
> +			process_negotiation_response(response, wc-
> >byte_len);
> +		complete(&info->negotiate_completion);
> +		break;
> +
> +	/* SMBD data transfer packet */
> +	case SMBD_TRANSFER_DATA:
> +		data_transfer = smbd_response_payload(response);
> +		data_length = le32_to_cpu(data_transfer->data_length);
> +
> +		/*
> +		 * If this is a packet with data playload place the data in
> +		 * reassembly queue and wake up the reading thread
> +		 */
> +		if (data_length) {
> +			if (info->full_packet_received)
> +				response->first_segment = true;
> +
> +			if (le32_to_cpu(data_transfer-
> >remaining_data_length))
> +				info->full_packet_received = false;
> +			else
> +				info->full_packet_received = true;
> +
> +			enqueue_reassembly(
> +				info,
> +				response,
> +				data_length);
> +		} else
> +			put_empty_packet(info, response);
> +
> +		if (data_length)
> +			wake_up_interruptible(&info-
> >wait_reassembly_queue);
> +
> +		atomic_dec(&info->receive_credits);
> +		info->receive_credit_target =
> +			le16_to_cpu(data_transfer->credits_requested);
> +		atomic_add(le16_to_cpu(data_transfer->credits_granted),
> +			&info->send_credits);
> +
> +		log_incoming(INFO, "data flags %d data_offset %d "
> +			"data_length %d remaining_data_length %d\n",
> +			le16_to_cpu(data_transfer->flags),
> +			le32_to_cpu(data_transfer->data_offset),
> +			le32_to_cpu(data_transfer->data_length),
> +			le32_to_cpu(data_transfer-
> >remaining_data_length));
> +
> +		/* Send a KEEP_ALIVE response right away if requested */
> +		info->keep_alive_requested = KEEP_ALIVE_NONE;
> +		if (le16_to_cpu(data_transfer->flags) &
> +				SMB_DIRECT_RESPONSE_REQUESTED) {
> +			info->keep_alive_requested =
> KEEP_ALIVE_PENDING;
> +		}
> +
> +		queue_work(info->workqueue, &info->recv_done_work);
> +		return;
> +
> +	default:
> +		log_rdma_recv(ERR,
> +			"unexpected response type=%d\n", response-
> >type);
> +	}
> +
> +error:
> +	put_receive_buffer(info, response, true);
> +}
> +
> +static struct rdma_cm_id *smbd_create_id(
> +		struct smbd_connection *info,
> +		struct sockaddr *dstaddr, int port)
> +{
> +	struct rdma_cm_id *id;
> +	int rc;
> +	__be16 *sport;
> +
> +	id = rdma_create_id(&init_net, smbd_conn_upcall, info,
> +		RDMA_PS_TCP, IB_QPT_RC);
> +	if (IS_ERR(id)) {
> +		rc = PTR_ERR(id);
> +		log_rdma_event(ERR, "rdma_create_id() failed %i\n", rc);
> +		return id;
> +	}
> +
> +	if (dstaddr->sa_family == AF_INET6)
> +		sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port;
> +	else
> +		sport = &((struct sockaddr_in *)dstaddr)->sin_port;
> +
> +	*sport = htons(port);
> +
> +	init_completion(&info->ri_done);
> +	info->ri_rc = -ETIMEDOUT;
> +
> +	rc = rdma_resolve_addr(id, NULL, (struct sockaddr *)dstaddr,
> +		RDMA_RESOLVE_TIMEOUT);
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_resolve_addr() failed %i\n",
> rc);
> +		goto out;
> +	}
> +	wait_for_completion_interruptible_timeout(
> +		&info->ri_done,
> msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
> +	rc = info->ri_rc;
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_resolve_addr() completed
> %i\n", rc);
> +		goto out;
> +	}
> +
> +	info->ri_rc = -ETIMEDOUT;
> +	rc = rdma_resolve_route(id, RDMA_RESOLVE_TIMEOUT);
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_resolve_route() failed %i\n",
> rc);
> +		goto out;
> +	}
> +	wait_for_completion_interruptible_timeout(
> +		&info->ri_done,
> msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
> +	rc = info->ri_rc;
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_resolve_route() completed
> %i\n", rc);
> +		goto out;
> +	}
> +
> +	return id;
> +
> +out:
> +	rdma_destroy_id(id);
> +	return ERR_PTR(rc);
> +}
> +
> +/*
> + * Test if FRWR (Fast Registration Work Requests) is supported on the
> device
> + * This implementation requries FRWR on RDMA read/write
> + * return value: true if it is supported
> + */
> +static bool frwr_is_supported(struct ib_device_attr *attrs)
> +{
> +	if (!(attrs->device_cap_flags &
> IB_DEVICE_MEM_MGT_EXTENSIONS))
> +		return false;
> +	if (attrs->max_fast_reg_page_list_len == 0)
> +		return false;
> +	return true;
> +}
> +
> +static int smbd_ia_open(
> +		struct smbd_connection *info,
> +		struct sockaddr *dstaddr, int port)
> +{
> +	int rc;
> +
> +	info->id = smbd_create_id(info, dstaddr, port);
> +	if (IS_ERR(info->id)) {
> +		rc = PTR_ERR(info->id);
> +		goto out1;
> +	}
> +
> +	if (!frwr_is_supported(&info->id->device->attrs)) {
> +		log_rdma_event(ERR,
> +			"Fast Registration Work Requests "
> +			"(FRWR) is not supported\n");
> +		log_rdma_event(ERR,
> +			"Device capability flags = %llx "
> +			"max_fast_reg_page_list_len = %u\n",
> +			info->id->device->attrs.device_cap_flags,
> +			info->id->device-
> >attrs.max_fast_reg_page_list_len);
> +		rc = -EPROTONOSUPPORT;
> +		goto out2;
> +	}
> +
> +	info->pd = ib_alloc_pd(info->id->device, 0);
> +	if (IS_ERR(info->pd)) {
> +		rc = PTR_ERR(info->pd);
> +		log_rdma_event(ERR, "ib_alloc_pd() returned %d\n", rc);
> +		goto out2;
> +	}
> +
> +	return 0;
> +
> +out2:
> +	rdma_destroy_id(info->id);
> +	info->id = NULL;
> +
> +out1:
> +	return rc;
> +}
> +
> +/*
> + * Send a negotiation request message to the peer
> + * The negotiation procedure is in [MS-SMBD] 3.1.5.2 and 3.1.5.3
> + * After negotiation, the transport is connected and ready for
> + * carrying upper layer SMB payload
> + */
> +static int smbd_post_send_negotiate_req(struct smbd_connection *info)
> +{
> +	struct ib_send_wr send_wr, *send_wr_fail;
> +	int rc = -ENOMEM;
> +	struct smbd_request *request;
> +	struct smbd_negotiate_req *packet;
> +
> +	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
> +	if (!request)
> +		return rc;
> +
> +	request->info = info;
> +
> +	packet = smbd_request_payload(request);
> +	packet->min_version = cpu_to_le16(SMBD_V1);
> +	packet->max_version = cpu_to_le16(SMBD_V1);
> +	packet->reserved = 0;
> +	packet->credits_requested = cpu_to_le16(info-
> >send_credit_target);
> +	packet->preferred_send_size = cpu_to_le32(info->max_send_size);
> +	packet->max_receive_size = cpu_to_le32(info->max_receive_size);
> +	packet->max_fragmented_size =
> +		cpu_to_le32(info->max_fragmented_recv_size);
> +
> +	request->num_sge = 1;
> +	request->sge[0].addr = ib_dma_map_single(
> +				info->id->device, (void *)packet,
> +				sizeof(*packet), DMA_TO_DEVICE);
> +	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr))
> {
> +		rc = -EIO;
> +		goto dma_mapping_failed;
> +	}
> +
> +	request->sge[0].length = sizeof(*packet);
> +	request->sge[0].lkey = info->pd->local_dma_lkey;
> +
> +	ib_dma_sync_single_for_device(
> +		info->id->device, request->sge[0].addr,
> +		request->sge[0].length, DMA_TO_DEVICE);
> +
> +	request->cqe.done = send_done;
> +
> +	send_wr.next = NULL;
> +	send_wr.wr_cqe = &request->cqe;
> +	send_wr.sg_list = request->sge;
> +	send_wr.num_sge = request->num_sge;
> +	send_wr.opcode = IB_WR_SEND;
> +	send_wr.send_flags = IB_SEND_SIGNALED;
> +
> +	log_rdma_send(INFO, "sge addr=%llx length=%x lkey=%x\n",
> +		request->sge[0].addr,
> +		request->sge[0].length, request->sge[0].lkey);
> +
> +	request->has_payload = false;
> +	atomic_inc(&info->send_pending);
> +	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
> +	if (!rc)
> +		return 0;
> +
> +	/* if we reach here, post send failed */
> +	log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
> +	atomic_dec(&info->send_pending);
> +	ib_dma_unmap_single(info->id->device, request->sge[0].addr,
> +		request->sge[0].length, DMA_TO_DEVICE);
> +
> +dma_mapping_failed:
> +	mempool_free(request, info->request_mempool);
> +	return rc;
> +}
> +
> +/*
> + * Extend the credits to remote peer
> + * This implements [MS-SMBD] 3.1.5.9
> + * The idea is that we should extend credits to remote peer as quickly as
> + * it's allowed, to maintain data flow. We allocate as much receive
> + * buffer as possible, and extend the receive credits to remote peer
> + * return value: the new credtis being granted.
> + */
> +static int manage_credits_prior_sending(struct smbd_connection *info)
> +{
> +	int new_credits;
> +
> +	spin_lock(&info->lock_new_credits_offered);
> +	new_credits = info->new_credits_offered;
> +	info->new_credits_offered = 0;
> +	spin_unlock(&info->lock_new_credits_offered);
> +
> +	return new_credits;
> +}
> +
> +/*
> + * Check if we need to send a KEEP_ALIVE message
> + * The idle connection timer triggers a KEEP_ALIVE message when expires
> + * SMB_DIRECT_RESPONSE_REQUESTED is set in the message flag to have
> peer send
> + * back a response.
> + * return value:
> + * 1 if SMB_DIRECT_RESPONSE_REQUESTED needs to be set
> + * 0: otherwise
> + */
> +static int manage_keep_alive_before_sending(struct smbd_connection
> *info)
> +{
> +	if (info->keep_alive_requested == KEEP_ALIVE_PENDING) {
> +		info->keep_alive_requested = KEEP_ALIVE_SENT;
> +		return 1;
> +	}
> +	return 0;
> +}
> +
> +/*
> + * Build and prepare the SMBD packet header
> + * This function waits for avaialbe send credits and build a SMBD packet
> + * header. The caller then optional append payload to the packet after
> + * the header
> + * intput values
> + * size: the size of the payload
> + * remaining_data_length: remaining data to send if this is part of a
> + * fragmented packet
> + * output values
> + * request_out: the request allocated from this function
> + * return values: 0 on success, otherwise actual error code returned
> + */
> +static int smbd_create_header(struct smbd_connection *info,
> +		int size, int remaining_data_length,
> +		struct smbd_request **request_out)
> +{
> +	struct smbd_request *request;
> +	struct smbd_data_transfer *packet;
> +	int header_length;
> +	int rc;
> +
> +	/* Wait for send credits. A SMBD packet needs one credit */
> +	rc = wait_event_interruptible(info->wait_send_queue,
> +		atomic_read(&info->send_credits) > 0 ||
> +		info->transport_status != SMBD_CONNECTED);
> +	if (rc)
> +		return rc;
> +
> +	if (info->transport_status != SMBD_CONNECTED) {
> +		log_outgoing(ERR, "disconnected not sending\n");
> +		return -ENOENT;
> +	}
> +	atomic_dec(&info->send_credits);
> +
> +	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
> +	if (!request) {
> +		rc = -ENOMEM;
> +		goto err;
> +	}
> +
> +	request->info = info;
> +
> +	/* Fill in the packet header */
> +	packet = smbd_request_payload(request);
> +	packet->credits_requested = cpu_to_le16(info-
> >send_credit_target);
> +	packet->credits_granted =
> +		cpu_to_le16(manage_credits_prior_sending(info));
> +	info->send_immediate = false;
> +
> +	packet->flags = 0;
> +	if (manage_keep_alive_before_sending(info))
> +		packet->flags |=
> cpu_to_le16(SMB_DIRECT_RESPONSE_REQUESTED);
> +
> +	packet->reserved = 0;
> +	if (!size)
> +		packet->data_offset = 0;
> +	else
> +		packet->data_offset = cpu_to_le32(24);
> +	packet->data_length = cpu_to_le32(size);
> +	packet->remaining_data_length =
> cpu_to_le32(remaining_data_length);
> +	packet->padding = 0;
> +
> +	log_outgoing(INFO, "credits_requested=%d credits_granted=%d "
> +		"data_offset=%d data_length=%d
> remaining_data_length=%d\n",
> +		le16_to_cpu(packet->credits_requested),
> +		le16_to_cpu(packet->credits_granted),
> +		le32_to_cpu(packet->data_offset),
> +		le32_to_cpu(packet->data_length),
> +		le32_to_cpu(packet->remaining_data_length));
> +
> +	/* Map the packet to DMA */
> +	header_length = sizeof(struct smbd_data_transfer);
> +	/* If this is a packet without payload, don't send padding */
> +	if (!size)
> +		header_length = offsetof(struct smbd_data_transfer,
> padding);
> +
> +	request->num_sge = 1;
> +	request->sge[0].addr = ib_dma_map_single(info->id->device,
> +						 (void *)packet,
> +						 header_length,
> +						 DMA_BIDIRECTIONAL);
> +	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr))
> {
> +		mempool_free(request, info->request_mempool);
> +		rc = -EIO;
> +		goto err;
> +	}
> +
> +	request->sge[0].length = header_length;
> +	request->sge[0].lkey = info->pd->local_dma_lkey;
> +
> +	*request_out = request;
> +	return 0;
> +
> +err:
> +	atomic_inc(&info->send_credits);
> +	return rc;
> +}
> +
> +static void smbd_destroy_header(struct smbd_connection *info,
> +		struct smbd_request *request)
> +{
> +
> +	ib_dma_unmap_single(info->id->device,
> +			    request->sge[0].addr,
> +			    request->sge[0].length,
> +			    DMA_TO_DEVICE);
> +	mempool_free(request, info->request_mempool);
> +	atomic_inc(&info->send_credits);
> +}
> +
> +/* Post the send request */
> +static int smbd_post_send(struct smbd_connection *info,
> +		struct smbd_request *request, bool has_payload)
> +{
> +	struct ib_send_wr send_wr, *send_wr_fail;
> +	int rc, i;
> +
> +	for (i = 0; i < request->num_sge; i++) {
> +		log_rdma_send(INFO,
> +			"rdma_request sge[%d] addr=%llu legnth=%u\n",
> +			i, request->sge[0].addr, request->sge[0].length);
> +		ib_dma_sync_single_for_device(
> +			info->id->device,
> +			request->sge[i].addr,
> +			request->sge[i].length,
> +			DMA_TO_DEVICE);
> +	}
> +
> +	request->cqe.done = send_done;
> +
> +	send_wr.next = NULL;
> +	send_wr.wr_cqe = &request->cqe;
> +	send_wr.sg_list = request->sge;
> +	send_wr.num_sge = request->num_sge;
> +	send_wr.opcode = IB_WR_SEND;
> +	send_wr.send_flags = IB_SEND_SIGNALED;
> +
> +	if (has_payload) {
> +		request->has_payload = true;
> +		atomic_inc(&info->send_payload_pending);
> +	} else {
> +		request->has_payload = false;
> +		atomic_inc(&info->send_pending);
> +	}
> +
> +	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
> +	if (rc) {
> +		log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
> +		if (has_payload) {
> +			if (atomic_dec_and_test(&info-
> >send_payload_pending))
> +				wake_up(&info-
> >wait_send_payload_pending);
> +		} else {
> +			if (atomic_dec_and_test(&info->send_pending))
> +				wake_up(&info->wait_send_pending);
> +		}
> +	} else
> +		/* Reset timer for idle connection after packet is sent */
> +		mod_delayed_work(info->workqueue, &info-
> >idle_timer_work,
> +			info->keep_alive_interval*HZ);
> +
> +	return rc;
> +}
> +
> +static int smbd_post_send_sgl(struct smbd_connection *info,
> +	struct scatterlist *sgl, int data_length, int remaining_data_length)
> +{
> +	int num_sgs;
> +	int i, rc;
> +	struct smbd_request *request;
> +	struct scatterlist *sg;
> +
> +	rc = smbd_create_header(
> +		info, data_length, remaining_data_length, &request);
> +	if (rc)
> +		return rc;
> +
> +	num_sgs = sgl ? sg_nents(sgl) : 0;
> +	for_each_sg(sgl, sg, num_sgs, i) {
> +		request->sge[i+1].addr =
> +			ib_dma_map_page(info->id->device, sg_page(sg),
> +			       sg->offset, sg->length, DMA_BIDIRECTIONAL);
> +		if (ib_dma_mapping_error(
> +				info->id->device, request->sge[i+1].addr)) {
> +			rc = -EIO;
> +			request->sge[i+1].addr = 0;
> +			goto dma_mapping_failure;
> +		}
> +		request->sge[i+1].length = sg->length;
> +		request->sge[i+1].lkey = info->pd->local_dma_lkey;
> +		request->num_sge++;
> +	}
> +
> +	rc = smbd_post_send(info, request, data_length);
> +	if (!rc)
> +		return 0;
> +
> +dma_mapping_failure:
> +	for (i = 1; i < request->num_sge; i++)
> +		if (request->sge[i].addr)
> +			ib_dma_unmap_single(info->id->device,
> +					    request->sge[i].addr,
> +					    request->sge[i].length,
> +					    DMA_TO_DEVICE);
> +	smbd_destroy_header(info, request);
> +	return rc;
> +}
> +
> +/*
> + * Send an empty message
> + * Empty message is used to extend credits to peer to for keep live
> + * while there is no upper layer payload to send at the time
> + */
> +static int smbd_post_send_empty(struct smbd_connection *info)
> +{
> +	info->count_send_empty++;
> +	return smbd_post_send_sgl(info, NULL, 0, 0);
> +}
> +
> +/*
> + * Post a receive request to the transport
> + * The remote peer can only send data when a receive request is posted
> + * The interaction is controlled by send/receive credit system
> + */
> +static int smbd_post_recv(
> +		struct smbd_connection *info, struct smbd_response
> *response)
> +{
> +	struct ib_recv_wr recv_wr, *recv_wr_fail = NULL;
> +	int rc = -EIO;
> +
> +	response->sge.addr = ib_dma_map_single(
> +				info->id->device, response->packet,
> +				info->max_receive_size,
> DMA_FROM_DEVICE);
> +	if (ib_dma_mapping_error(info->id->device, response->sge.addr))
> +		return rc;
> +
> +	response->sge.length = info->max_receive_size;
> +	response->sge.lkey = info->pd->local_dma_lkey;
> +
> +	response->cqe.done = recv_done;
> +
> +	recv_wr.wr_cqe = &response->cqe;
> +	recv_wr.next = NULL;
> +	recv_wr.sg_list = &response->sge;
> +	recv_wr.num_sge = 1;
> +
> +	rc = ib_post_recv(info->id->qp, &recv_wr, &recv_wr_fail);
> +	if (rc) {
> +		ib_dma_unmap_single(info->id->device, response-
> >sge.addr,
> +				    response->sge.length,
> DMA_FROM_DEVICE);
> +
> +		log_rdma_recv(ERR, "ib_post_recv failed rc=%d\n", rc);
> +	}
> +
> +	return rc;
> +}
> +
> +/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */
> +static int smbd_negotiate(struct smbd_connection *info)
> +{
> +	int rc;
> +	struct smbd_response *response = get_receive_buffer(info);
> +
> +	response->type = SMBD_NEGOTIATE_RESP;
> +	rc = smbd_post_recv(info, response);
> +	log_rdma_event(INFO,
> +		"smbd_post_recv rc=%d iov.addr=%llx iov.length=%x "
> +		"iov.lkey=%x\n",
> +		rc, response->sge.addr,
> +		response->sge.length, response->sge.lkey);
> +	if (rc)
> +		return rc;
> +
> +	init_completion(&info->negotiate_completion);
> +	info->negotiate_done = false;
> +	rc = smbd_post_send_negotiate_req(info);
> +	if (rc)
> +		return rc;
> +
> +	rc = wait_for_completion_interruptible_timeout(
> +		&info->negotiate_completion, SMBD_NEGOTIATE_TIMEOUT
> * HZ);
> +	log_rdma_event(INFO, "wait_for_completion_timeout rc=%d\n",
> rc);
> +
> +	if (info->negotiate_done)
> +		return 0;
> +
> +	if (rc == 0)
> +		rc = -ETIMEDOUT;
> +	else if (rc == -ERESTARTSYS)
> +		rc = -EINTR;
> +	else
> +		rc = -ENOTCONN;
> +
> +	return rc;
> +}
> +
> +static void put_empty_packet(
> +		struct smbd_connection *info, struct smbd_response
> *response)
> +{
> +	spin_lock(&info->empty_packet_queue_lock);
> +	list_add_tail(&response->list, &info->empty_packet_queue);
> +	info->count_empty_packet_queue++;
> +	spin_unlock(&info->empty_packet_queue_lock);
> +
> +	queue_work(info->workqueue, &info->post_send_credits_work);
> +}
> +
> +/*
> + * Implement Connection.FragmentReassemblyBuffer defined in [MS-
> SMBD] 3.1.1.1
> + * This is a queue for reassembling upper layer payload and present to
> upper
> + * layer. All the inncoming payload go to the reassembly queue, regardless
> of
> + * if reassembly is required. The uuper layer code reads from the queue for
> all
> + * incoming payloads.
> + * Put a received packet to the reassembly queue
> + * response: the packet received
> + * data_length: the size of payload in this packet
> + */
> +static void enqueue_reassembly(
> +	struct smbd_connection *info,
> +	struct smbd_response *response,
> +	int data_length)
> +{
> +	spin_lock(&info->reassembly_queue_lock);
> +	list_add_tail(&response->list, &info->reassembly_queue);
> +	info->reassembly_queue_length++;
> +	/*
> +	 * Make sure reassembly_data_length is updated after list and
> +	 * reassembly_queue_length are updated. On the dequeue side
> +	 * reassembly_data_length is checked without a lock to determine
> +	 * if reassembly_queue_length and list is up to date
> +	 */
> +	virt_wmb();
> +	info->reassembly_data_length += data_length;
> +	spin_unlock(&info->reassembly_queue_lock);
> +	info->count_reassembly_queue++;
> +	info->count_enqueue_reassembly_queue++;
> +}
> +
> +/*
> + * Get the first entry at the front of reassembly queue
> + * Caller is responsible for locking
> + * return value: the first entry if any, NULL if queue is empty
> + */
> +static struct smbd_response *_get_first_reassembly(struct
> smbd_connection *info)
> +{
> +	struct smbd_response *ret = NULL;
> +
> +	if (!list_empty(&info->reassembly_queue)) {
> +		ret = list_first_entry(
> +			&info->reassembly_queue,
> +			struct smbd_response, list);
> +	}
> +	return ret;
> +}
> +
> +static struct smbd_response *get_empty_queue_buffer(
> +		struct smbd_connection *info)
> +{
> +	struct smbd_response *ret = NULL;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&info->empty_packet_queue_lock, flags);
> +	if (!list_empty(&info->empty_packet_queue)) {
> +		ret = list_first_entry(
> +			&info->empty_packet_queue,
> +			struct smbd_response, list);
> +		list_del(&ret->list);
> +		info->count_empty_packet_queue--;
> +	}
> +	spin_unlock_irqrestore(&info->empty_packet_queue_lock, flags);
> +
> +	return ret;
> +}
> +
> +/*
> + * Get a receive buffer
> + * For each remote send, we need to post a receive. The receive buffers are
> + * pre-allocated in advance.
> + * return value: the receive buffer, NULL if none is available
> + */
> +static struct smbd_response *get_receive_buffer(struct smbd_connection
> *info)
> +{
> +	struct smbd_response *ret = NULL;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&info->receive_queue_lock, flags);
> +	if (!list_empty(&info->receive_queue)) {
> +		ret = list_first_entry(
> +			&info->receive_queue,
> +			struct smbd_response, list);
> +		list_del(&ret->list);
> +		info->count_receive_queue--;
> +		info->count_get_receive_buffer++;
> +	}
> +	spin_unlock_irqrestore(&info->receive_queue_lock, flags);
> +
> +	return ret;
> +}
> +
> +/*
> + * Return a receive buffer
> + * Upon returning of a receive buffer, we can post new receive and extend
> + * more receive credits to remote peer. This is done immediately after a
> + * receive buffer is returned.
> + */
> +static void put_receive_buffer(
> +	struct smbd_connection *info, struct smbd_response *response,
> +	bool lock)
> +{
> +	unsigned long flags;
> +
> +	ib_dma_unmap_single(info->id->device, response->sge.addr,
> +		response->sge.length, DMA_FROM_DEVICE);
> +
> +	if (lock)
> +		spin_lock_irqsave(&info->receive_queue_lock, flags);
> +	list_add_tail(&response->list, &info->receive_queue);
> +	info->count_receive_queue++;
> +	info->count_put_receive_buffer++;
> +	if (lock)
> +		spin_unlock_irqrestore(&info->receive_queue_lock, flags);
> +
> +	queue_work(info->workqueue, &info->post_send_credits_work);
> +}
> +
> +/* Preallocate all receive buffer on transport establishment */
> +static int allocate_receive_buffers(struct smbd_connection *info, int
> num_buf)
> +{
> +	int i;
> +	struct smbd_response *response;
> +
> +	INIT_LIST_HEAD(&info->reassembly_queue);
> +	spin_lock_init(&info->reassembly_queue_lock);
> +	info->reassembly_data_length = 0;
> +	info->reassembly_queue_length = 0;
> +
> +	INIT_LIST_HEAD(&info->receive_queue);
> +	spin_lock_init(&info->receive_queue_lock);
> +	info->count_receive_queue = 0;
> +
> +	INIT_LIST_HEAD(&info->empty_packet_queue);
> +	spin_lock_init(&info->empty_packet_queue_lock);
> +	info->count_empty_packet_queue = 0;
> +
> +	init_waitqueue_head(&info->wait_receive_queues);
> +
> +	for (i = 0; i < num_buf; i++) {
> +		response = mempool_alloc(info->response_mempool,
> GFP_KERNEL);
> +		if (!response)
> +			goto allocate_failed;
> +
> +		response->info = info;
> +		list_add_tail(&response->list, &info->receive_queue);
> +		info->count_receive_queue++;
> +	}
> +
> +	return 0;
> +
> +allocate_failed:
> +	while (!list_empty(&info->receive_queue)) {
> +		response = list_first_entry(
> +				&info->receive_queue,
> +				struct smbd_response, list);
> +		list_del(&response->list);
> +		info->count_receive_queue--;
> +
> +		mempool_free(response, info->response_mempool);
> +	}
> +	return -ENOMEM;
> +}
> +
> +static void destroy_receive_buffers(struct smbd_connection *info)
> +{
> +	struct smbd_response *response;
> +
> +	while ((response = get_receive_buffer(info)))
> +		mempool_free(response, info->response_mempool);
> +
> +	while ((response = get_empty_queue_buffer(info)))
> +		mempool_free(response, info->response_mempool);
> +}
> +
> +/*
> + * Check and send an immediate or keep alive packet
> + * The condition to send those packets are defined in [MS-SMBD] 3.1.1.1
> + * Connection.KeepaliveRequested and Connection.SendImmediate
> + * The idea is to extend credits to server as soon as it becomes available
> + */
> +static void send_immediate_work(struct work_struct *work)
> +{
> +	struct smbd_connection *info = container_of(
> +					work, struct smbd_connection,
> +					send_immediate_work.work);
> +
> +	if (info->keep_alive_requested == KEEP_ALIVE_PENDING ||
> +	    info->send_immediate) {
> +		log_keep_alive(INFO, "send an empty message\n");
> +		smbd_post_send_empty(info);
> +	}
> +}
> +
> +/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */
> +static void idle_connection_timer(struct work_struct *work)
> +{
> +	struct smbd_connection *info = container_of(
> +					work, struct smbd_connection,
> +					idle_timer_work.work);
> +
> +	if (info->keep_alive_requested != KEEP_ALIVE_NONE) {
> +		log_keep_alive(ERR,
> +			"error status info->keep_alive_requested=%d\n",
> +			info->keep_alive_requested);
> +		smbd_disconnect_rdma_connection(info);
> +		return;
> +	}
> +
> +	log_keep_alive(INFO, "about to send an empty idle message\n");
> +	smbd_post_send_empty(info);
> +
> +	/* Setup the next idle timeout work */
> +	queue_delayed_work(info->workqueue, &info->idle_timer_work,
> +			info->keep_alive_interval*HZ);
> +}
> +
> +static void destroy_caches_and_workqueue(struct smbd_connection
> *info)
> +{
> +	destroy_receive_buffers(info);
> +	destroy_workqueue(info->workqueue);
> +	mempool_destroy(info->response_mempool);
> +	kmem_cache_destroy(info->response_cache);
> +	mempool_destroy(info->request_mempool);
> +	kmem_cache_destroy(info->request_cache);
> +}
> +
> +#define MAX_NAME_LEN	80
> +static int allocate_caches_and_workqueue(struct smbd_connection *info)
> +{
> +	char name[MAX_NAME_LEN];
> +	int rc;
> +
> +	snprintf(name, MAX_NAME_LEN, "smbd_request_%p", info);
> +	info->request_cache =
> +		kmem_cache_create(
> +			name,
> +			sizeof(struct smbd_request) +
> +				sizeof(struct smbd_data_transfer),
> +			0, SLAB_HWCACHE_ALIGN, NULL);
> +	if (!info->request_cache)
> +		return -ENOMEM;
> +
> +	info->request_mempool =
> +		mempool_create(info->send_credit_target,
> mempool_alloc_slab,
> +			mempool_free_slab, info->request_cache);
> +	if (!info->request_mempool)
> +		goto out1;
> +
> +	snprintf(name, MAX_NAME_LEN, "smbd_response_%p", info);
> +	info->response_cache =
> +		kmem_cache_create(
> +			name,
> +			sizeof(struct smbd_response) +
> +				info->max_receive_size,
> +			0, SLAB_HWCACHE_ALIGN, NULL);
> +	if (!info->response_cache)
> +		goto out2;
> +
> +	info->response_mempool =
> +		mempool_create(info->receive_credit_max,
> mempool_alloc_slab,
> +		       mempool_free_slab, info->response_cache);
> +	if (!info->response_mempool)
> +		goto out3;
> +
> +	snprintf(name, MAX_NAME_LEN, "smbd_%p", info);
> +	info->workqueue = create_workqueue(name);
> +	if (!info->workqueue)
> +		goto out4;
> +
> +	rc = allocate_receive_buffers(info, info->receive_credit_max);
> +	if (rc) {
> +		log_rdma_event(ERR, "failed to allocate receive buffers\n");
> +		goto out5;
> +	}
> +
> +	return 0;
> +
> +out5:
> +	destroy_workqueue(info->workqueue);
> +out4:
> +	mempool_destroy(info->response_mempool);
> +out3:
> +	kmem_cache_destroy(info->response_cache);
> +out2:
> +	mempool_destroy(info->request_mempool);
> +out1:
> +	kmem_cache_destroy(info->request_cache);
> +	return -ENOMEM;
> +}
> +
> +/* Create a SMBD connection, called by upper layer */
> +struct smbd_connection *_smbd_get_connection(
> +	struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
> +{
> +	int rc;
> +	struct smbd_connection *info;
> +	struct rdma_conn_param conn_param;
> +	struct ib_qp_init_attr qp_attr;
> +	struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;
> +
> +	info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL);
> +	if (!info)
> +		return NULL;
> +
> +	info->transport_status = SMBD_CONNECTING;
> +	rc = smbd_ia_open(info, dstaddr, port);
> +	if (rc) {
> +		log_rdma_event(INFO, "smbd_ia_open rc=%d\n", rc);
> +		goto create_id_failed;
> +	}
> +
> +	if (smbd_send_credit_target > info->id->device->attrs.max_cqe ||
> +	    smbd_send_credit_target > info->id->device->attrs.max_qp_wr) {
> +		log_rdma_event(ERR,
> +			"consider lowering send_credit_target = %d. "
> +			"Possible CQE overrun, device "
> +			"reporting max_cpe %d max_qp_wr %d\n",
> +			smbd_send_credit_target,
> +			info->id->device->attrs.max_cqe,
> +			info->id->device->attrs.max_qp_wr);
> +		goto config_failed;
> +	}
> +
> +	if (smbd_receive_credit_max > info->id->device->attrs.max_cqe ||
> +	    smbd_receive_credit_max > info->id->device->attrs.max_qp_wr)
> {
> +		log_rdma_event(ERR,
> +			"consider lowering receive_credit_max = %d. "
> +			"Possible CQE overrun, device "
> +			"reporting max_cpe %d max_qp_wr %d\n",
> +			smbd_receive_credit_max,
> +			info->id->device->attrs.max_cqe,
> +			info->id->device->attrs.max_qp_wr);
> +		goto config_failed;
> +	}
> +
> +	info->receive_credit_max = smbd_receive_credit_max;
> +	info->send_credit_target = smbd_send_credit_target;
> +	info->max_send_size = smbd_max_send_size;
> +	info->max_fragmented_recv_size =
> smbd_max_fragmented_recv_size;
> +	info->max_receive_size = smbd_max_receive_size;
> +	info->keep_alive_interval = smbd_keep_alive_interval;
> +
> +	if (SMBDIRECT_MAX_SGE > info->id->device->attrs.max_sge) {
> +		log_rdma_event(ERR, "warning: device max_sge = %d too
> small\n",
> +			info->id->device->attrs.max_sge);
> +		log_rdma_event(ERR, "Queue Pair creation may fail\n");
> +	}
> +
> +	info->send_cq = NULL;
> +	info->recv_cq = NULL;
> +	info->send_cq = ib_alloc_cq(info->id->device, info,
> +			info->send_credit_target, 0, IB_POLL_SOFTIRQ);
> +	if (IS_ERR(info->send_cq)) {
> +		info->send_cq = NULL;
> +		goto alloc_cq_failed;
> +	}
> +
> +	info->recv_cq = ib_alloc_cq(info->id->device, info,
> +			info->receive_credit_max, 0, IB_POLL_SOFTIRQ);
> +	if (IS_ERR(info->recv_cq)) {
> +		info->recv_cq = NULL;
> +		goto alloc_cq_failed;
> +	}
> +
> +	memset(&qp_attr, 0, sizeof(qp_attr));
> +	qp_attr.event_handler = smbd_qp_async_error_upcall;
> +	qp_attr.qp_context = info;
> +	qp_attr.cap.max_send_wr = info->send_credit_target;
> +	qp_attr.cap.max_recv_wr = info->receive_credit_max;
> +	qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
> +	qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
> +	qp_attr.cap.max_inline_data = 0;
> +	qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
> +	qp_attr.qp_type = IB_QPT_RC;
> +	qp_attr.send_cq = info->send_cq;
> +	qp_attr.recv_cq = info->recv_cq;
> +	qp_attr.port_num = ~0;
> +
> +	rc = rdma_create_qp(info->id, info->pd, &qp_attr);
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_create_qp failed %i\n", rc);
> +		goto create_qp_failed;
> +	}
> +
> +	memset(&conn_param, 0, sizeof(conn_param));
> +	conn_param.initiator_depth = 0;
> +
> +	conn_param.retry_count = SMBD_CM_RETRY;
> +	conn_param.rnr_retry_count = SMBD_CM_RNR_RETRY;
> +	conn_param.flow_control = 0;
> +	init_waitqueue_head(&info->wait_destroy);
> +
> +	log_rdma_event(INFO, "connecting to IP %pI4 port %d\n",
> +		&addr_in->sin_addr, port);
> +
> +	init_waitqueue_head(&info->conn_wait);
> +	rc = rdma_connect(info->id, &conn_param);
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_connect() failed with %i\n", rc);
> +		goto rdma_connect_failed;
> +	}
> +
> +	wait_event_interruptible(
> +		info->conn_wait, info->transport_status !=
> SMBD_CONNECTING);
> +
> +	if (info->transport_status != SMBD_CONNECTED) {
> +		log_rdma_event(ERR, "rdma_connect failed port=%d\n",
> port);
> +		goto rdma_connect_failed;
> +	}
> +
> +	log_rdma_event(INFO, "rdma_connect connected\n");
> +
> +	rc = allocate_caches_and_workqueue(info);
> +	if (rc) {
> +		log_rdma_event(ERR, "cache allocation failed\n");
> +		goto allocate_cache_failed;
> +	}
> +
> +	init_waitqueue_head(&info->wait_send_queue);
> +	init_waitqueue_head(&info->wait_reassembly_queue);
> +
> +	INIT_DELAYED_WORK(&info->idle_timer_work,
> idle_connection_timer);
> +	INIT_DELAYED_WORK(&info->send_immediate_work,
> send_immediate_work);
> +	queue_delayed_work(info->workqueue, &info->idle_timer_work,
> +		info->keep_alive_interval*HZ);
> +
> +	init_waitqueue_head(&info->wait_send_pending);
> +	atomic_set(&info->send_pending, 0);
> +
> +	init_waitqueue_head(&info->wait_send_payload_pending);
> +	atomic_set(&info->send_payload_pending, 0);
> +
> +	INIT_WORK(&info->disconnect_work,
> smbd_disconnect_rdma_work);
> +	INIT_WORK(&info->destroy_work, smbd_destroy_rdma_work);
> +	INIT_WORK(&info->recv_done_work, smbd_recv_done_work);
> +	INIT_WORK(&info->post_send_credits_work,
> smbd_post_send_credits);
> +	info->new_credits_offered = 0;
> +	spin_lock_init(&info->lock_new_credits_offered);
> +
> +	rc = smbd_negotiate(info);
> +	if (rc) {
> +		log_rdma_event(ERR, "smbd_negotiate rc=%d\n", rc);
> +		goto negotiation_failed;
> +	}
> +
> +	return info;
> +
> +negotiation_failed:
> +	cancel_delayed_work_sync(&info->idle_timer_work);
> +	destroy_caches_and_workqueue(info);
> +	info->transport_status = SMBD_NEGOTIATE_FAILED;
> +	init_waitqueue_head(&info->conn_wait);
> +	rdma_disconnect(info->id);
> +	wait_event(info->conn_wait,
> +		info->transport_status == SMBD_DISCONNECTED);
> +
> +allocate_cache_failed:
> +rdma_connect_failed:
> +	rdma_destroy_qp(info->id);
> +
> +create_qp_failed:
> +alloc_cq_failed:
> +	if (info->send_cq)
> +		ib_free_cq(info->send_cq);
> +	if (info->recv_cq)
> +		ib_free_cq(info->recv_cq);
> +
> +config_failed:
> +	ib_dealloc_pd(info->pd);
> +	rdma_destroy_id(info->id);
> +
> +create_id_failed:
> +	kfree(info);
> +	return NULL;
> +}
> diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
> index c55f28b..35bc25b 100644
> --- a/fs/cifs/smbdirect.h
> +++ b/fs/cifs/smbdirect.h
> @@ -16,6 +16,286 @@
>  #ifndef _SMBDIRECT_H
>  #define _SMBDIRECT_H
> 
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +#define cifs_rdma_enabled(server)	((server)->rdma)
> +
> +#include "cifsglob.h"
> +#include <rdma/ib_verbs.h>
> +#include <rdma/rdma_cm.h>
> +#include <linux/mempool.h>
> +
> +enum keep_alive_status {
> +	KEEP_ALIVE_NONE,
> +	KEEP_ALIVE_PENDING,
> +	KEEP_ALIVE_SENT,
> +};
> +
> +enum smbd_connection_status {
> +	SMBD_CREATED,
> +	SMBD_CONNECTING,
> +	SMBD_CONNECTED,
> +	SMBD_NEGOTIATE_FAILED,
> +	SMBD_DISCONNECTING,
> +	SMBD_DISCONNECTED,
> +	SMBD_DESTROYED
> +};
> +
> +/*
> + * The context for the SMBDirect transport
> + * Everything related to the transport is here. It has several logical parts
> + * 1. RDMA related structures
> + * 2. SMBDirect connection parameters
> + * 3. Memory registrations
> + * 4. Receive and reassembly queues for data receive path
> + * 5. mempools for allocating packets
> + */
> +struct smbd_connection {
> +	enum smbd_connection_status transport_status;
> +
> +	/* RDMA related */
> +	struct rdma_cm_id *id;
> +	struct ib_qp_init_attr qp_attr;
> +	struct ib_pd *pd;
> +	struct ib_cq *send_cq, *recv_cq;
> +	struct ib_device_attr dev_attr;
> +	int ri_rc;
> +	struct completion ri_done;
> +	wait_queue_head_t conn_wait;
> +	wait_queue_head_t wait_destroy;
> +
> +	struct completion negotiate_completion;
> +	bool negotiate_done;
> +
> +	struct work_struct destroy_work;
> +	struct work_struct disconnect_work;
> +	struct work_struct recv_done_work;
> +	struct work_struct post_send_credits_work;
> +
> +	spinlock_t lock_new_credits_offered;
> +	int new_credits_offered;
> +
> +	/* Connection parameters defined in [MS-SMBD] 3.1.1.1 */
> +	int receive_credit_max;
> +	int send_credit_target;
> +	int max_send_size;
> +	int max_fragmented_recv_size;
> +	int max_fragmented_send_size;
> +	int max_receive_size;
> +	int keep_alive_interval;
> +	int max_readwrite_size;
> +	enum keep_alive_status keep_alive_requested;
> +	int protocol;
> +	atomic_t send_credits;
> +	atomic_t receive_credits;
> +	int receive_credit_target;
> +	int fragment_reassembly_remaining;
> +
> +	/* Activity accoutning */
> +
> +	atomic_t send_pending;
> +	wait_queue_head_t wait_send_pending;
> +	atomic_t send_payload_pending;
> +	wait_queue_head_t wait_send_payload_pending;
> +
> +	/* Receive queue */
> +	struct list_head receive_queue;
> +	int count_receive_queue;
> +	spinlock_t receive_queue_lock;
> +
> +	struct list_head empty_packet_queue;
> +	int count_empty_packet_queue;
> +	spinlock_t empty_packet_queue_lock;
> +
> +	wait_queue_head_t wait_receive_queues;
> +
> +	/* Reassembly queue */
> +	struct list_head reassembly_queue;
> +	spinlock_t reassembly_queue_lock;
> +	wait_queue_head_t wait_reassembly_queue;
> +
> +	/* total data length of reassembly queue */
> +	int reassembly_data_length;
> +	int reassembly_queue_length;
> +	/* the offset to first buffer in reassembly queue */
> +	int first_entry_offset;
> +
> +	bool send_immediate;
> +
> +	wait_queue_head_t wait_send_queue;
> +
> +	/*
> +	 * Indicate if we have received a full packet on the connection
> +	 * This is used to identify the first SMBD packet of a assembled
> +	 * payload (SMB packet) in reassembly queue so we can return a
> +	 * RFC1002 length to upper layer to indicate the length of the SMB
> +	 * packet received
> +	 */
> +	bool full_packet_received;
> +
> +	struct workqueue_struct *workqueue;
> +	struct delayed_work idle_timer_work;
> +	struct delayed_work send_immediate_work;
> +
> +	/* Memory pool for preallocating buffers */
> +	/* request pool for RDMA send */
> +	struct kmem_cache *request_cache;
> +	mempool_t *request_mempool;
> +
> +	/* response pool for RDMA receive */
> +	struct kmem_cache *response_cache;
> +	mempool_t *response_mempool;
> +
> +	/* for debug purposes */
> +	unsigned int count_get_receive_buffer;
> +	unsigned int count_put_receive_buffer;
> +	unsigned int count_reassembly_queue;
> +	unsigned int count_enqueue_reassembly_queue;
> +	unsigned int count_dequeue_reassembly_queue;
> +	unsigned int count_send_empty;
> +};
> +
> +enum smbd_message_type {
> +	SMBD_NEGOTIATE_RESP,
> +	SMBD_TRANSFER_DATA,
> +};
> +
> +#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
> +
> +/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */
> +struct smbd_negotiate_req {
> +	__le16 min_version;
> +	__le16 max_version;
> +	__le16 reserved;
> +	__le16 credits_requested;
> +	__le32 preferred_send_size;
> +	__le32 max_receive_size;
> +	__le32 max_fragmented_size;
> +} __packed;
> +
> +/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */
> +struct smbd_negotiate_resp {
> +	__le16 min_version;
> +	__le16 max_version;
> +	__le16 negotiated_version;
> +	__le16 reserved;
> +	__le16 credits_requested;
> +	__le16 credits_granted;
> +	__le32 status;
> +	__le32 max_readwrite_size;
> +	__le32 preferred_send_size;
> +	__le32 max_receive_size;
> +	__le32 max_fragmented_size;
> +} __packed;
> +
> +/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */
> +struct smbd_data_transfer {
> +	__le16 credits_requested;
> +	__le16 credits_granted;
> +	__le16 flags;
> +	__le16 reserved;
> +	__le32 remaining_data_length;
> +	__le32 data_offset;
> +	__le32 data_length;
> +	__le32 padding;
> +	__u8 buffer[];
> +} __packed;
> +
> +/* The packet fields for a registered RDMA buffer */
> +struct smbd_buffer_descriptor_v1 {
> +	__le64 offset;
> +	__le32 token;
> +	__le32 length;
> +} __packed;
> +
>  /* Default maximum number of SGEs in a RDMA send/recv */
>  #define SMBDIRECT_MAX_SGE	16
> +/* The context for a SMBD request */
> +struct smbd_request {
> +	struct smbd_connection *info;
> +	struct ib_cqe cqe;
> +
> +	/* true if this request carries upper layer payload */
> +	bool has_payload;
> +
> +	/* the SGE entries for this packet */
> +	struct ib_sge sge[SMBDIRECT_MAX_SGE];
> +	int num_sge;
> +
> +	/* SMBD packet header follows this structure */
> +	u8 packet[];
> +};
> +
> +/* The context for a SMBD response */
> +struct smbd_response {
> +	struct smbd_connection *info;
> +	struct ib_cqe cqe;
> +	struct ib_sge sge;
> +
> +	enum smbd_message_type type;
> +
> +	/* Link to receive queue or reassembly queue */
> +	struct list_head list;
> +
> +	/* Indicate if this is the 1st packet of a payload */
> +	bool first_segment;
> +
> +	/* SMBD packet header and payload follows this structure */
> +	u8 packet[];
> +};
> +
> +/* Create a SMBDirect session */
> +struct smbd_connection *smbd_get_connection(
> +	struct TCP_Server_Info *server, struct sockaddr *dstaddr);
> +
> +/* Reconnect SMBDirect session */
> +int smbd_reconnect(struct TCP_Server_Info *server);
> +
> +/* Destroy SMBDirect session */
> +void smbd_destroy(struct smbd_connection *info);
> +
> +/* Interface for carrying upper layer I/O through send/recv */
> +int smbd_recv(struct smbd_connection *info, struct msghdr *msg);
> +int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst);
> +
> +enum mr_state {
> +	MR_READY,
> +	MR_REGISTERED,
> +	MR_INVALIDATED,
> +	MR_ERROR
> +};
> +
> +struct smbd_mr {
> +	struct smbd_connection	*conn;
> +	struct list_head	list;
> +	enum mr_state		state;
> +	struct ib_mr		*mr;
> +	struct scatterlist	*sgl;
> +	int			sgl_count;
> +	enum dma_data_direction	dir;
> +	union {
> +		struct ib_reg_wr	wr;
> +		struct ib_send_wr	inv_wr;
> +	};
> +	struct ib_cqe		cqe;
> +	bool			need_invalidate;
> +	struct completion	invalidate_done;
> +};
> +
> +/* Interfaces to register and deregister MR for RDMA read/write */
> +struct smbd_mr *smbd_register_mr(
> +	struct smbd_connection *info, struct page *pages[], int num_pages,
> +	int tailsz, bool writing, bool need_invalidate);
> +int smbd_deregister_mr(struct smbd_mr *mr);
> +
> +#else
> +#define cifs_rdma_enabled(server)	0
> +struct smbd_connection{};
> +static inline void *smbd_get_connection(
> +	struct TCP_Server_Info *server, struct sockaddr *dstaddr) {return
> NULL;}
> +static inline int smbd_reconnect(struct TCP_Server_Info *server) {return -
> 1;}
> +static inline void smbd_destroy(struct smbd_connection *info) {}
> +static inline int smbd_recv(struct smbd_connection *info, struct msghdr
> *msg) {return -1;}
> +static inline int smbd_send(struct smbd_connection *info, struct smb_rqst
> *rqst) {return -1;}
> +#endif
> +
>  #endif
> --
> 2.7.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.ke
> rnel.org%2Fmajordomo-
> info.html&data=02%7C01%7Clongli%40microsoft.com%7Ce7da39e3ff8f409ea
> 29208d52fda17a4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636
> 467536124091212&sdata=eRCijuXR1Y%2FsFds3UZDrdK6AoDutBBLCQ0UZU5v
> 5XAc%3D&reserved=0

RE: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection

[Long Li]

> -----Original Message-----
> From: Leif Sahlberg [mailto:lsahlber@redhat.com]
> Sent: Sunday, November 19, 2017 9:47 PM
> To: Long Li <longli@microsoft.com>
> Cc: Steve French <sfrench@samba.org>; linux-cifs@vger.kernel.org; linux-
> kernel@vger.kernel.org; linux-rdma@vger.kernel.org; Christoph Hellwig
> <hch@infradead.org>; Tom Talpey <ttalpey@microsoft.com>; Matthew
> Wilcox <mawilcox@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; Long Li <longli@microsoft.com>
> Subject: Re: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection
> 
> Acked-by: Ronnie Sahlberg <lsahlber@gmail.com>
> 
> But two tiny nits:
> + * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
> Should be 0xa0
> 
> +//	queue_work(info->workqueue, &info->destroy_work);
> Don't leave commented out code in there.

I will address those. Thanks!

> 
> 
> ----- Original Message -----
> From: "Long Li" <longli@exchange.microsoft.com>
> To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org,
> samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-
> rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom
> Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox"
> <mawilcox@microsoft.com>, "Stephen Hemminger"
> <sthemmin@microsoft.com>
> Cc: "Long Li" <longli@microsoft.com>
> Sent: Tuesday, 7 November, 2017 7:54:57 PM
> Subject: [Patch v7 05/22] CIFS: SMBD: Establish SMB Direct connection
> 
> From: Long Li <longli@microsoft.com>
> 
> Add code to implement the core functions to establish a SMB Direct
> connection.
> 
> 1. Establish an RDMA connection to SMB server.
> 2. Negotiate and setup SMB Direct protocol.
> 3. Implement idle connection timer and credit management.
> 
> SMB Direct is enabled by setting CONFIG_CIFS_SMB_DIRECT.
> 
> Add to Makefile to enable building SMB Direct.
> 
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>  fs/cifs/Makefile    |    2 +
>  fs/cifs/smbdirect.c | 1576
> +++++++++++++++++++++++++++++++++++++++++++++++++++
>  fs/cifs/smbdirect.h |  280 +++++++++
>  3 files changed, 1858 insertions(+)
> 
> diff --git a/fs/cifs/Makefile b/fs/cifs/Makefile
> index 5e853a3..ad00873 100644
> --- a/fs/cifs/Makefile
> +++ b/fs/cifs/Makefile
> @@ -18,3 +18,5 @@ cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
>  cifs-$(CONFIG_CIFS_DFS_UPCALL) += dns_resolve.o cifs_dfs_ref.o
> 
>  cifs-$(CONFIG_CIFS_FSCACHE) += fscache.o cache.o
> +
> +cifs-$(CONFIG_CIFS_SMB_DIRECT) += smbdirect.o
> diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
> index d3c16f8..021d527 100644
> --- a/fs/cifs/smbdirect.c
> +++ b/fs/cifs/smbdirect.c
> @@ -13,7 +13,34 @@
>   *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
>   *   the GNU General Public License for more details.
>   */
> +#include <linux/module.h>
>  #include "smbdirect.h"
> +#include "cifs_debug.h"
> +
> +static struct smbd_response *get_empty_queue_buffer(
> +		struct smbd_connection *info);
> +static struct smbd_response *get_receive_buffer(
> +		struct smbd_connection *info);
> +static void put_receive_buffer(
> +		struct smbd_connection *info,
> +		struct smbd_response *response,
> +		bool lock);
> +static int allocate_receive_buffers(struct smbd_connection *info, int
> num_buf);
> +static void destroy_receive_buffers(struct smbd_connection *info);
> +
> +static void put_empty_packet(
> +		struct smbd_connection *info, struct smbd_response
> *response);
> +static void enqueue_reassembly(
> +		struct smbd_connection *info,
> +		struct smbd_response *response, int data_length);
> +static struct smbd_response *_get_first_reassembly(
> +		struct smbd_connection *info);
> +
> +static int smbd_post_recv(
> +		struct smbd_connection *info,
> +		struct smbd_response *response);
> +
> +static int smbd_post_send_empty(struct smbd_connection *info);
> 
>  /* SMBD version number */
>  #define SMBD_V1	0x0100
> @@ -75,3 +102,1552 @@ int smbd_max_frmr_depth = 2048;
> 
>  /* If payload is less than this byte, use RDMA send/recv not read/write */
>  int rdma_readwrite_threshold = 4096;
> +
> +/* Transport logging functions
> + * Logging are defined as classes. They can be OR'ed to define the actual
> + * logging level via module parameter smbd_logging_class
> + * e.g. cifs.smbd_logging_class=0x500 will log all log_rdma_recv() and
> + * log_rdma_event()
> + */
> +#define LOG_OUTGOING			0x1
> +#define LOG_INCOMING			0x2
> +#define LOG_READ			0x4
> +#define LOG_WRITE			0x8
> +#define LOG_RDMA_SEND			0x10
> +#define LOG_RDMA_RECV			0x20
> +#define LOG_KEEP_ALIVE			0x40
> +#define LOG_RDMA_EVENT			0x80
> +#define LOG_RDMA_MR			0x100
> +static unsigned int smbd_logging_class = 0;
> +module_param(smbd_logging_class, uint, 0644);
> +MODULE_PARM_DESC(smbd_logging_class,
> +	"Logging class for SMBD transport 0x0 to 0x100");
> +
> +#define ERR		0x0
> +#define INFO		0x1
> +static unsigned int smbd_logging_level = ERR;
> +module_param(smbd_logging_level, uint, 0644);
> +MODULE_PARM_DESC(smbd_logging_level,
> +	"Logging level for SMBD transport, 0 (default): error, 1: info");
> +
> +#define log_rdma(level, class, fmt, args...)				\
> +do {									\
> +	if (level <= smbd_logging_level || class & smbd_logging_class)	\
> +		cifs_dbg(VFS, "%s:%d " fmt, __func__, __LINE__, ##args);\
> +} while (0)
> +
> +#define log_outgoing(level, fmt, args...) \
> +		log_rdma(level, LOG_OUTGOING, fmt, ##args)
> +#define log_incoming(level, fmt, args...) \
> +		log_rdma(level, LOG_INCOMING, fmt, ##args)
> +#define log_read(level, fmt, args...)	log_rdma(level, LOG_READ, fmt,
> ##args)
> +#define log_write(level, fmt, args...)	log_rdma(level, LOG_WRITE, fmt,
> ##args)
> +#define log_rdma_send(level, fmt, args...) \
> +		log_rdma(level, LOG_RDMA_SEND, fmt, ##args)
> +#define log_rdma_recv(level, fmt, args...) \
> +		log_rdma(level, LOG_RDMA_RECV, fmt, ##args)
> +#define log_keep_alive(level, fmt, args...) \
> +		log_rdma(level, LOG_KEEP_ALIVE, fmt, ##args)
> +#define log_rdma_event(level, fmt, args...) \
> +		log_rdma(level, LOG_RDMA_EVENT, fmt, ##args)
> +#define log_rdma_mr(level, fmt, args...) \
> +		log_rdma(level, LOG_RDMA_MR, fmt, ##args)
> +
> +/*
> + * Destroy the transport and related RDMA and memory resources
> + * Need to go through all the pending counters and make sure on one is
> using
> + * the transport while it is destroyed
> + */
> +static void smbd_destroy_rdma_work(struct work_struct *work)
> +{
> +	struct smbd_response *response;
> +	struct smbd_connection *info =
> +		container_of(work, struct smbd_connection, destroy_work);
> +	unsigned long flags;
> +
> +	log_rdma_event(INFO, "destroying qp\n");
> +	ib_drain_qp(info->id->qp);
> +	rdma_destroy_qp(info->id);
> +
> +	/* Unblock all I/O waiting on the send queue */
> +	wake_up_interruptible_all(&info->wait_send_queue);
> +
> +	log_rdma_event(INFO, "cancelling idle timer\n");
> +	cancel_delayed_work_sync(&info->idle_timer_work);
> +	log_rdma_event(INFO, "cancelling send immediate work\n");
> +	cancel_delayed_work_sync(&info->send_immediate_work);
> +
> +	log_rdma_event(INFO, "wait for all recv to finish\n");
> +	wake_up_interruptible(&info->wait_reassembly_queue);
> +
> +	log_rdma_event(INFO, "wait for all send posted to IB to finish\n");
> +	wait_event(info->wait_send_pending,
> +		atomic_read(&info->send_pending) == 0);
> +	wait_event(info->wait_send_payload_pending,
> +		atomic_read(&info->send_payload_pending) == 0);
> +
> +	/* It's not posssible for upper layer to get to reassembly */
> +	log_rdma_event(INFO, "drain the reassembly queue\n");
> +	do {
> +		spin_lock_irqsave(&info->reassembly_queue_lock, flags);
> +		response = _get_first_reassembly(info);
> +		if (response) {
> +			list_del(&response->list);
> +			spin_unlock_irqrestore(
> +				&info->reassembly_queue_lock, flags);
> +			put_receive_buffer(info, response, true);
> +		}
> +	} while (response);
> +	spin_unlock_irqrestore(&info->reassembly_queue_lock, flags);
> +	info->reassembly_data_length = 0;
> +
> +	log_rdma_event(INFO, "free receive buffers\n");
> +	wait_event(info->wait_receive_queues,
> +		info->count_receive_queue + info-
> >count_empty_packet_queue
> +			== info->receive_credit_max);
> +	destroy_receive_buffers(info);
> +
> +	ib_free_cq(info->send_cq);
> +	ib_free_cq(info->recv_cq);
> +	ib_dealloc_pd(info->pd);
> +	rdma_destroy_id(info->id);
> +
> +	/* free mempools */
> +	mempool_destroy(info->request_mempool);
> +	kmem_cache_destroy(info->request_cache);
> +
> +	mempool_destroy(info->response_mempool);
> +	kmem_cache_destroy(info->response_cache);
> +
> +	info->transport_status = SMBD_DESTROYED;
> +	wake_up_all(&info->wait_destroy);
> +}
> +
> +static int smbd_process_disconnected(struct smbd_connection *info)
> +{
> +//	queue_work(info->workqueue, &info->destroy_work);
> +	schedule_work(&info->destroy_work);
> +	return 0;
> +}
> +
> +static void smbd_disconnect_rdma_work(struct work_struct *work)
> +{
> +	struct smbd_connection *info =
> +		container_of(work, struct smbd_connection,
> disconnect_work);
> +
> +	if (info->transport_status == SMBD_CONNECTED) {
> +		info->transport_status = SMBD_DISCONNECTING;
> +		rdma_disconnect(info->id);
> +	}
> +}
> +
> +static void smbd_disconnect_rdma_connection(struct smbd_connection
> *info)
> +{
> +	queue_work(info->workqueue, &info->disconnect_work);
> +}
> +
> +/* Upcall from RDMA CM */
> +static int smbd_conn_upcall(
> +		struct rdma_cm_id *id, struct rdma_cm_event *event)
> +{
> +	struct smbd_connection *info = id->context;
> +
> +	log_rdma_event(INFO, "event=%d status=%d\n",
> +		event->event, event->status);
> +
> +	switch (event->event) {
> +	case RDMA_CM_EVENT_ADDR_RESOLVED:
> +	case RDMA_CM_EVENT_ROUTE_RESOLVED:
> +		info->ri_rc = 0;
> +		complete(&info->ri_done);
> +		break;
> +
> +	case RDMA_CM_EVENT_ADDR_ERROR:
> +		info->ri_rc = -EHOSTUNREACH;
> +		complete(&info->ri_done);
> +		break;
> +
> +	case RDMA_CM_EVENT_ROUTE_ERROR:
> +		info->ri_rc = -ENETUNREACH;
> +		complete(&info->ri_done);
> +		break;
> +
> +	case RDMA_CM_EVENT_ESTABLISHED:
> +		log_rdma_event(INFO, "connected event=%d\n", event-
> >event);
> +		info->transport_status = SMBD_CONNECTED;
> +		wake_up_interruptible(&info->conn_wait);
> +		break;
> +
> +	case RDMA_CM_EVENT_CONNECT_ERROR:
> +	case RDMA_CM_EVENT_UNREACHABLE:
> +	case RDMA_CM_EVENT_REJECTED:
> +		log_rdma_event(INFO, "connecting failed event=%d\n",
> event->event);
> +		info->transport_status = SMBD_DISCONNECTED;
> +		wake_up_interruptible(&info->conn_wait);
> +		break;
> +
> +	case RDMA_CM_EVENT_DEVICE_REMOVAL:
> +	case RDMA_CM_EVENT_DISCONNECTED:
> +		/* This happenes when we fail the negotiation */
> +		if (info->transport_status == SMBD_NEGOTIATE_FAILED) {
> +			info->transport_status = SMBD_DISCONNECTED;
> +			wake_up(&info->conn_wait);
> +			break;
> +		}
> +
> +		info->transport_status = SMBD_DISCONNECTED;
> +		smbd_process_disconnected(info);
> +		break;
> +
> +	default:
> +		break;
> +	}
> +
> +	return 0;
> +}
> +
> +/* Upcall from RDMA QP */
> +static void
> +smbd_qp_async_error_upcall(struct ib_event *event, void *context)
> +{
> +	struct smbd_connection *info = context;
> +
> +	log_rdma_event(ERR, "%s on device %s info %p\n",
> +		ib_event_msg(event->event), event->device->name, info);
> +
> +	switch (event->event) {
> +	case IB_EVENT_CQ_ERR:
> +	case IB_EVENT_QP_FATAL:
> +		smbd_disconnect_rdma_connection(info);
> +
> +	default:
> +		break;
> +	}
> +}
> +
> +static inline void *smbd_request_payload(struct smbd_request *request)
> +{
> +	return (void *)request->packet;
> +}
> +
> +static inline void *smbd_response_payload(struct smbd_response
> *response)
> +{
> +	return (void *)response->packet;
> +}
> +
> +/* Called when a RDMA send is done */
> +static void send_done(struct ib_cq *cq, struct ib_wc *wc)
> +{
> +	int i;
> +	struct smbd_request *request =
> +		container_of(wc->wr_cqe, struct smbd_request, cqe);
> +
> +	log_rdma_send(INFO, "smbd_request %p completed wc-
> >status=%d\n",
> +		request, wc->status);
> +
> +	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_SEND) {
> +		log_rdma_send(ERR, "wc->status=%d wc->opcode=%d\n",
> +			wc->status, wc->opcode);
> +		smbd_disconnect_rdma_connection(request->info);
> +	}
> +
> +	for (i = 0; i < request->num_sge; i++)
> +		ib_dma_unmap_single(request->info->id->device,
> +			request->sge[i].addr,
> +			request->sge[i].length,
> +			DMA_TO_DEVICE);
> +
> +	if (request->has_payload) {
> +		if (atomic_dec_and_test(&request->info-
> >send_payload_pending))
> +			wake_up(&request->info-
> >wait_send_payload_pending);
> +	} else {
> +		if (atomic_dec_and_test(&request->info->send_pending))
> +			wake_up(&request->info->wait_send_pending);
> +	}
> +
> +	mempool_free(request, request->info->request_mempool);
> +}
> +
> +static void dump_smbd_negotiate_resp(struct smbd_negotiate_resp
> *resp)
> +{
> +	log_rdma_event(INFO, "resp message min_version %u max_version
> %u "
> +		"negotiated_version %u credits_requested %u "
> +		"credits_granted %u status %u max_readwrite_size %u "
> +		"preferred_send_size %u max_receive_size %u "
> +		"max_fragmented_size %u\n",
> +		resp->min_version, resp->max_version, resp-
> >negotiated_version,
> +		resp->credits_requested, resp->credits_granted, resp-
> >status,
> +		resp->max_readwrite_size, resp->preferred_send_size,
> +		resp->max_receive_size, resp->max_fragmented_size);
> +}
> +
> +/*
> + * Process a negotiation response message, according to [MS-SMBD]3.1.5.7
> + * response, packet_length: the negotiation response message
> + * return value: true if negotiation is a success, false if failed
> + */
> +static bool process_negotiation_response(
> +		struct smbd_response *response, int packet_length)
> +{
> +	struct smbd_connection *info = response->info;
> +	struct smbd_negotiate_resp *packet =
> smbd_response_payload(response);
> +
> +	if (packet_length < sizeof(struct smbd_negotiate_resp)) {
> +		log_rdma_event(ERR,
> +			"error: packet_length=%d\n", packet_length);
> +		return false;
> +	}
> +
> +	if (le16_to_cpu(packet->negotiated_version) != SMBD_V1) {
> +		log_rdma_event(ERR, "error: negotiated_version=%x\n",
> +			le16_to_cpu(packet->negotiated_version));
> +		return false;
> +	}
> +	info->protocol = le16_to_cpu(packet->negotiated_version);
> +
> +	if (packet->credits_requested == 0) {
> +		log_rdma_event(ERR, "error: credits_requested==0\n");
> +		return false;
> +	}
> +	info->receive_credit_target = le16_to_cpu(packet-
> >credits_requested);
> +
> +	if (packet->credits_granted == 0) {
> +		log_rdma_event(ERR, "error: credits_granted==0\n");
> +		return false;
> +	}
> +	atomic_set(&info->send_credits, le16_to_cpu(packet-
> >credits_granted));
> +
> +	atomic_set(&info->receive_credits, 0);
> +
> +	if (le32_to_cpu(packet->preferred_send_size) > info-
> >max_receive_size) {
> +		log_rdma_event(ERR, "error: preferred_send_size=%d\n",
> +			le32_to_cpu(packet->preferred_send_size));
> +		return false;
> +	}
> +	info->max_receive_size = le32_to_cpu(packet-
> >preferred_send_size);
> +
> +	if (le32_to_cpu(packet->max_receive_size) <
> SMBD_MIN_RECEIVE_SIZE) {
> +		log_rdma_event(ERR, "error: max_receive_size=%d\n",
> +			le32_to_cpu(packet->max_receive_size));
> +		return false;
> +	}
> +	info->max_send_size = min_t(int, info->max_send_size,
> +					le32_to_cpu(packet-
> >max_receive_size));
> +
> +	if (le32_to_cpu(packet->max_fragmented_size) <
> +			SMBD_MIN_FRAGMENTED_SIZE) {
> +		log_rdma_event(ERR, "error: max_fragmented_size=%d\n",
> +			le32_to_cpu(packet->max_fragmented_size));
> +		return false;
> +	}
> +	info->max_fragmented_send_size =
> +		le32_to_cpu(packet->max_fragmented_size);
> +
> +	return true;
> +}
> +
> +/*
> + * Check and schedule to send an immediate packet
> + * This is used to extend credtis to remote peer to keep the transport busy
> + */
> +static void check_and_send_immediate(struct smbd_connection *info)
> +{
> +	if (info->transport_status != SMBD_CONNECTED)
> +		return;
> +
> +	info->send_immediate = true;
> +
> +	/*
> +	 * Promptly send a packet if our peer is running low on receive
> +	 * credits
> +	 */
> +	if (atomic_read(&info->receive_credits) <
> +		info->receive_credit_target - 1)
> +		queue_delayed_work(
> +			info->workqueue, &info->send_immediate_work,
> 0);
> +}
> +
> +static void smbd_post_send_credits(struct work_struct *work)
> +{
> +	int ret = 0;
> +	int use_receive_queue = 1;
> +	int rc;
> +	struct smbd_response *response;
> +	struct smbd_connection *info =
> +		container_of(work, struct smbd_connection,
> +			post_send_credits_work);
> +
> +	if (info->transport_status != SMBD_CONNECTED) {
> +		wake_up(&info->wait_receive_queues);
> +		return;
> +	}
> +
> +	if (info->receive_credit_target >
> +		atomic_read(&info->receive_credits)) {
> +		while (true) {
> +			if (use_receive_queue)
> +				response = get_receive_buffer(info);
> +			else
> +				response = get_empty_queue_buffer(info);
> +			if (!response) {
> +				/* now switch to emtpy packet queue */
> +				if (use_receive_queue) {
> +					use_receive_queue = 0;
> +					continue;
> +				} else
> +					break;
> +			}
> +
> +			response->type = SMBD_TRANSFER_DATA;
> +			response->first_segment = false;
> +			rc = smbd_post_recv(info, response);
> +			if (rc) {
> +				log_rdma_recv(ERR,
> +					"post_recv failed rc=%d\n", rc);
> +				put_receive_buffer(info, response, true);
> +				break;
> +			}
> +
> +			ret++;
> +		}
> +	}
> +
> +	spin_lock(&info->lock_new_credits_offered);
> +	info->new_credits_offered += ret;
> +	spin_unlock(&info->lock_new_credits_offered);
> +
> +	atomic_add(ret, &info->receive_credits);
> +
> +	/* Check if we can post new receive and grant credits to peer */
> +	check_and_send_immediate(info);
> +}
> +
> +static void smbd_recv_done_work(struct work_struct *work)
> +{
> +	struct smbd_connection *info =
> +		container_of(work, struct smbd_connection,
> recv_done_work);
> +
> +	/*
> +	 * We may have new send credits granted from remote peer
> +	 * If any sender is blcoked on lack of credets, unblock it
> +	 */
> +	if (atomic_read(&info->send_credits))
> +		wake_up_interruptible(&info->wait_send_queue);
> +
> +	/*
> +	 * Check if we need to send something to remote peer to
> +	 * grant more credits or respond to KEEP_ALIVE packet
> +	 */
> +	check_and_send_immediate(info);
> +}
> +
> +/* Called from softirq, when recv is done */
> +static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> +{
> +	struct smbd_data_transfer *data_transfer;
> +	struct smbd_response *response =
> +		container_of(wc->wr_cqe, struct smbd_response, cqe);
> +	struct smbd_connection *info = response->info;
> +	int data_length = 0;
> +
> +	log_rdma_recv(INFO, "response=%p type=%d wc status=%d wc
> opcode %d "
> +		      "byte_len=%d pkey_index=%x\n",
> +		response, response->type, wc->status, wc->opcode,
> +		wc->byte_len, wc->pkey_index);
> +
> +	if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
> +		log_rdma_recv(INFO, "wc->status=%d opcode=%d\n",
> +			wc->status, wc->opcode);
> +		smbd_disconnect_rdma_connection(info);
> +		goto error;
> +	}
> +
> +	ib_dma_sync_single_for_cpu(
> +		wc->qp->device,
> +		response->sge.addr,
> +		response->sge.length,
> +		DMA_FROM_DEVICE);
> +
> +	switch (response->type) {
> +	/* SMBD negotiation response */
> +	case SMBD_NEGOTIATE_RESP:
> +
> 	dump_smbd_negotiate_resp(smbd_response_payload(response));
> +		info->full_packet_received = true;
> +		info->negotiate_done =
> +			process_negotiation_response(response, wc-
> >byte_len);
> +		complete(&info->negotiate_completion);
> +		break;
> +
> +	/* SMBD data transfer packet */
> +	case SMBD_TRANSFER_DATA:
> +		data_transfer = smbd_response_payload(response);
> +		data_length = le32_to_cpu(data_transfer->data_length);
> +
> +		/*
> +		 * If this is a packet with data playload place the data in
> +		 * reassembly queue and wake up the reading thread
> +		 */
> +		if (data_length) {
> +			if (info->full_packet_received)
> +				response->first_segment = true;
> +
> +			if (le32_to_cpu(data_transfer-
> >remaining_data_length))
> +				info->full_packet_received = false;
> +			else
> +				info->full_packet_received = true;
> +
> +			enqueue_reassembly(
> +				info,
> +				response,
> +				data_length);
> +		} else
> +			put_empty_packet(info, response);
> +
> +		if (data_length)
> +			wake_up_interruptible(&info-
> >wait_reassembly_queue);
> +
> +		atomic_dec(&info->receive_credits);
> +		info->receive_credit_target =
> +			le16_to_cpu(data_transfer->credits_requested);
> +		atomic_add(le16_to_cpu(data_transfer->credits_granted),
> +			&info->send_credits);
> +
> +		log_incoming(INFO, "data flags %d data_offset %d "
> +			"data_length %d remaining_data_length %d\n",
> +			le16_to_cpu(data_transfer->flags),
> +			le32_to_cpu(data_transfer->data_offset),
> +			le32_to_cpu(data_transfer->data_length),
> +			le32_to_cpu(data_transfer-
> >remaining_data_length));
> +
> +		/* Send a KEEP_ALIVE response right away if requested */
> +		info->keep_alive_requested = KEEP_ALIVE_NONE;
> +		if (le16_to_cpu(data_transfer->flags) &
> +				SMB_DIRECT_RESPONSE_REQUESTED) {
> +			info->keep_alive_requested =
> KEEP_ALIVE_PENDING;
> +		}
> +
> +		queue_work(info->workqueue, &info->recv_done_work);
> +		return;
> +
> +	default:
> +		log_rdma_recv(ERR,
> +			"unexpected response type=%d\n", response-
> >type);
> +	}
> +
> +error:
> +	put_receive_buffer(info, response, true);
> +}
> +
> +static struct rdma_cm_id *smbd_create_id(
> +		struct smbd_connection *info,
> +		struct sockaddr *dstaddr, int port)
> +{
> +	struct rdma_cm_id *id;
> +	int rc;
> +	__be16 *sport;
> +
> +	id = rdma_create_id(&init_net, smbd_conn_upcall, info,
> +		RDMA_PS_TCP, IB_QPT_RC);
> +	if (IS_ERR(id)) {
> +		rc = PTR_ERR(id);
> +		log_rdma_event(ERR, "rdma_create_id() failed %i\n", rc);
> +		return id;
> +	}
> +
> +	if (dstaddr->sa_family == AF_INET6)
> +		sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port;
> +	else
> +		sport = &((struct sockaddr_in *)dstaddr)->sin_port;
> +
> +	*sport = htons(port);
> +
> +	init_completion(&info->ri_done);
> +	info->ri_rc = -ETIMEDOUT;
> +
> +	rc = rdma_resolve_addr(id, NULL, (struct sockaddr *)dstaddr,
> +		RDMA_RESOLVE_TIMEOUT);
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_resolve_addr() failed %i\n",
> rc);
> +		goto out;
> +	}
> +	wait_for_completion_interruptible_timeout(
> +		&info->ri_done,
> msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
> +	rc = info->ri_rc;
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_resolve_addr() completed
> %i\n", rc);
> +		goto out;
> +	}
> +
> +	info->ri_rc = -ETIMEDOUT;
> +	rc = rdma_resolve_route(id, RDMA_RESOLVE_TIMEOUT);
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_resolve_route() failed %i\n",
> rc);
> +		goto out;
> +	}
> +	wait_for_completion_interruptible_timeout(
> +		&info->ri_done,
> msecs_to_jiffies(RDMA_RESOLVE_TIMEOUT));
> +	rc = info->ri_rc;
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_resolve_route() completed
> %i\n", rc);
> +		goto out;
> +	}
> +
> +	return id;
> +
> +out:
> +	rdma_destroy_id(id);
> +	return ERR_PTR(rc);
> +}
> +
> +/*
> + * Test if FRWR (Fast Registration Work Requests) is supported on the
> device
> + * This implementation requries FRWR on RDMA read/write
> + * return value: true if it is supported
> + */
> +static bool frwr_is_supported(struct ib_device_attr *attrs)
> +{
> +	if (!(attrs->device_cap_flags &
> IB_DEVICE_MEM_MGT_EXTENSIONS))
> +		return false;
> +	if (attrs->max_fast_reg_page_list_len == 0)
> +		return false;
> +	return true;
> +}
> +
> +static int smbd_ia_open(
> +		struct smbd_connection *info,
> +		struct sockaddr *dstaddr, int port)
> +{
> +	int rc;
> +
> +	info->id = smbd_create_id(info, dstaddr, port);
> +	if (IS_ERR(info->id)) {
> +		rc = PTR_ERR(info->id);
> +		goto out1;
> +	}
> +
> +	if (!frwr_is_supported(&info->id->device->attrs)) {
> +		log_rdma_event(ERR,
> +			"Fast Registration Work Requests "
> +			"(FRWR) is not supported\n");
> +		log_rdma_event(ERR,
> +			"Device capability flags = %llx "
> +			"max_fast_reg_page_list_len = %u\n",
> +			info->id->device->attrs.device_cap_flags,
> +			info->id->device-
> >attrs.max_fast_reg_page_list_len);
> +		rc = -EPROTONOSUPPORT;
> +		goto out2;
> +	}
> +
> +	info->pd = ib_alloc_pd(info->id->device, 0);
> +	if (IS_ERR(info->pd)) {
> +		rc = PTR_ERR(info->pd);
> +		log_rdma_event(ERR, "ib_alloc_pd() returned %d\n", rc);
> +		goto out2;
> +	}
> +
> +	return 0;
> +
> +out2:
> +	rdma_destroy_id(info->id);
> +	info->id = NULL;
> +
> +out1:
> +	return rc;
> +}
> +
> +/*
> + * Send a negotiation request message to the peer
> + * The negotiation procedure is in [MS-SMBD] 3.1.5.2 and 3.1.5.3
> + * After negotiation, the transport is connected and ready for
> + * carrying upper layer SMB payload
> + */
> +static int smbd_post_send_negotiate_req(struct smbd_connection *info)
> +{
> +	struct ib_send_wr send_wr, *send_wr_fail;
> +	int rc = -ENOMEM;
> +	struct smbd_request *request;
> +	struct smbd_negotiate_req *packet;
> +
> +	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
> +	if (!request)
> +		return rc;
> +
> +	request->info = info;
> +
> +	packet = smbd_request_payload(request);
> +	packet->min_version = cpu_to_le16(SMBD_V1);
> +	packet->max_version = cpu_to_le16(SMBD_V1);
> +	packet->reserved = 0;
> +	packet->credits_requested = cpu_to_le16(info-
> >send_credit_target);
> +	packet->preferred_send_size = cpu_to_le32(info->max_send_size);
> +	packet->max_receive_size = cpu_to_le32(info->max_receive_size);
> +	packet->max_fragmented_size =
> +		cpu_to_le32(info->max_fragmented_recv_size);
> +
> +	request->num_sge = 1;
> +	request->sge[0].addr = ib_dma_map_single(
> +				info->id->device, (void *)packet,
> +				sizeof(*packet), DMA_TO_DEVICE);
> +	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr))
> {
> +		rc = -EIO;
> +		goto dma_mapping_failed;
> +	}
> +
> +	request->sge[0].length = sizeof(*packet);
> +	request->sge[0].lkey = info->pd->local_dma_lkey;
> +
> +	ib_dma_sync_single_for_device(
> +		info->id->device, request->sge[0].addr,
> +		request->sge[0].length, DMA_TO_DEVICE);
> +
> +	request->cqe.done = send_done;
> +
> +	send_wr.next = NULL;
> +	send_wr.wr_cqe = &request->cqe;
> +	send_wr.sg_list = request->sge;
> +	send_wr.num_sge = request->num_sge;
> +	send_wr.opcode = IB_WR_SEND;
> +	send_wr.send_flags = IB_SEND_SIGNALED;
> +
> +	log_rdma_send(INFO, "sge addr=%llx length=%x lkey=%x\n",
> +		request->sge[0].addr,
> +		request->sge[0].length, request->sge[0].lkey);
> +
> +	request->has_payload = false;
> +	atomic_inc(&info->send_pending);
> +	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
> +	if (!rc)
> +		return 0;
> +
> +	/* if we reach here, post send failed */
> +	log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
> +	atomic_dec(&info->send_pending);
> +	ib_dma_unmap_single(info->id->device, request->sge[0].addr,
> +		request->sge[0].length, DMA_TO_DEVICE);
> +
> +dma_mapping_failed:
> +	mempool_free(request, info->request_mempool);
> +	return rc;
> +}
> +
> +/*
> + * Extend the credits to remote peer
> + * This implements [MS-SMBD] 3.1.5.9
> + * The idea is that we should extend credits to remote peer as quickly as
> + * it's allowed, to maintain data flow. We allocate as much receive
> + * buffer as possible, and extend the receive credits to remote peer
> + * return value: the new credtis being granted.
> + */
> +static int manage_credits_prior_sending(struct smbd_connection *info)
> +{
> +	int new_credits;
> +
> +	spin_lock(&info->lock_new_credits_offered);
> +	new_credits = info->new_credits_offered;
> +	info->new_credits_offered = 0;
> +	spin_unlock(&info->lock_new_credits_offered);
> +
> +	return new_credits;
> +}
> +
> +/*
> + * Check if we need to send a KEEP_ALIVE message
> + * The idle connection timer triggers a KEEP_ALIVE message when expires
> + * SMB_DIRECT_RESPONSE_REQUESTED is set in the message flag to have
> peer send
> + * back a response.
> + * return value:
> + * 1 if SMB_DIRECT_RESPONSE_REQUESTED needs to be set
> + * 0: otherwise
> + */
> +static int manage_keep_alive_before_sending(struct smbd_connection
> *info)
> +{
> +	if (info->keep_alive_requested == KEEP_ALIVE_PENDING) {
> +		info->keep_alive_requested = KEEP_ALIVE_SENT;
> +		return 1;
> +	}
> +	return 0;
> +}
> +
> +/*
> + * Build and prepare the SMBD packet header
> + * This function waits for avaialbe send credits and build a SMBD packet
> + * header. The caller then optional append payload to the packet after
> + * the header
> + * intput values
> + * size: the size of the payload
> + * remaining_data_length: remaining data to send if this is part of a
> + * fragmented packet
> + * output values
> + * request_out: the request allocated from this function
> + * return values: 0 on success, otherwise actual error code returned
> + */
> +static int smbd_create_header(struct smbd_connection *info,
> +		int size, int remaining_data_length,
> +		struct smbd_request **request_out)
> +{
> +	struct smbd_request *request;
> +	struct smbd_data_transfer *packet;
> +	int header_length;
> +	int rc;
> +
> +	/* Wait for send credits. A SMBD packet needs one credit */
> +	rc = wait_event_interruptible(info->wait_send_queue,
> +		atomic_read(&info->send_credits) > 0 ||
> +		info->transport_status != SMBD_CONNECTED);
> +	if (rc)
> +		return rc;
> +
> +	if (info->transport_status != SMBD_CONNECTED) {
> +		log_outgoing(ERR, "disconnected not sending\n");
> +		return -ENOENT;
> +	}
> +	atomic_dec(&info->send_credits);
> +
> +	request = mempool_alloc(info->request_mempool, GFP_KERNEL);
> +	if (!request) {
> +		rc = -ENOMEM;
> +		goto err;
> +	}
> +
> +	request->info = info;
> +
> +	/* Fill in the packet header */
> +	packet = smbd_request_payload(request);
> +	packet->credits_requested = cpu_to_le16(info-
> >send_credit_target);
> +	packet->credits_granted =
> +		cpu_to_le16(manage_credits_prior_sending(info));
> +	info->send_immediate = false;
> +
> +	packet->flags = 0;
> +	if (manage_keep_alive_before_sending(info))
> +		packet->flags |=
> cpu_to_le16(SMB_DIRECT_RESPONSE_REQUESTED);
> +
> +	packet->reserved = 0;
> +	if (!size)
> +		packet->data_offset = 0;
> +	else
> +		packet->data_offset = cpu_to_le32(24);
> +	packet->data_length = cpu_to_le32(size);
> +	packet->remaining_data_length =
> cpu_to_le32(remaining_data_length);
> +	packet->padding = 0;
> +
> +	log_outgoing(INFO, "credits_requested=%d credits_granted=%d "
> +		"data_offset=%d data_length=%d
> remaining_data_length=%d\n",
> +		le16_to_cpu(packet->credits_requested),
> +		le16_to_cpu(packet->credits_granted),
> +		le32_to_cpu(packet->data_offset),
> +		le32_to_cpu(packet->data_length),
> +		le32_to_cpu(packet->remaining_data_length));
> +
> +	/* Map the packet to DMA */
> +	header_length = sizeof(struct smbd_data_transfer);
> +	/* If this is a packet without payload, don't send padding */
> +	if (!size)
> +		header_length = offsetof(struct smbd_data_transfer,
> padding);
> +
> +	request->num_sge = 1;
> +	request->sge[0].addr = ib_dma_map_single(info->id->device,
> +						 (void *)packet,
> +						 header_length,
> +						 DMA_BIDIRECTIONAL);
> +	if (ib_dma_mapping_error(info->id->device, request->sge[0].addr))
> {
> +		mempool_free(request, info->request_mempool);
> +		rc = -EIO;
> +		goto err;
> +	}
> +
> +	request->sge[0].length = header_length;
> +	request->sge[0].lkey = info->pd->local_dma_lkey;
> +
> +	*request_out = request;
> +	return 0;
> +
> +err:
> +	atomic_inc(&info->send_credits);
> +	return rc;
> +}
> +
> +static void smbd_destroy_header(struct smbd_connection *info,
> +		struct smbd_request *request)
> +{
> +
> +	ib_dma_unmap_single(info->id->device,
> +			    request->sge[0].addr,
> +			    request->sge[0].length,
> +			    DMA_TO_DEVICE);
> +	mempool_free(request, info->request_mempool);
> +	atomic_inc(&info->send_credits);
> +}
> +
> +/* Post the send request */
> +static int smbd_post_send(struct smbd_connection *info,
> +		struct smbd_request *request, bool has_payload)
> +{
> +	struct ib_send_wr send_wr, *send_wr_fail;
> +	int rc, i;
> +
> +	for (i = 0; i < request->num_sge; i++) {
> +		log_rdma_send(INFO,
> +			"rdma_request sge[%d] addr=%llu legnth=%u\n",
> +			i, request->sge[0].addr, request->sge[0].length);
> +		ib_dma_sync_single_for_device(
> +			info->id->device,
> +			request->sge[i].addr,
> +			request->sge[i].length,
> +			DMA_TO_DEVICE);
> +	}
> +
> +	request->cqe.done = send_done;
> +
> +	send_wr.next = NULL;
> +	send_wr.wr_cqe = &request->cqe;
> +	send_wr.sg_list = request->sge;
> +	send_wr.num_sge = request->num_sge;
> +	send_wr.opcode = IB_WR_SEND;
> +	send_wr.send_flags = IB_SEND_SIGNALED;
> +
> +	if (has_payload) {
> +		request->has_payload = true;
> +		atomic_inc(&info->send_payload_pending);
> +	} else {
> +		request->has_payload = false;
> +		atomic_inc(&info->send_pending);
> +	}
> +
> +	rc = ib_post_send(info->id->qp, &send_wr, &send_wr_fail);
> +	if (rc) {
> +		log_rdma_send(ERR, "ib_post_send failed rc=%d\n", rc);
> +		if (has_payload) {
> +			if (atomic_dec_and_test(&info-
> >send_payload_pending))
> +				wake_up(&info-
> >wait_send_payload_pending);
> +		} else {
> +			if (atomic_dec_and_test(&info->send_pending))
> +				wake_up(&info->wait_send_pending);
> +		}
> +	} else
> +		/* Reset timer for idle connection after packet is sent */
> +		mod_delayed_work(info->workqueue, &info-
> >idle_timer_work,
> +			info->keep_alive_interval*HZ);
> +
> +	return rc;
> +}
> +
> +static int smbd_post_send_sgl(struct smbd_connection *info,
> +	struct scatterlist *sgl, int data_length, int remaining_data_length)
> +{
> +	int num_sgs;
> +	int i, rc;
> +	struct smbd_request *request;
> +	struct scatterlist *sg;
> +
> +	rc = smbd_create_header(
> +		info, data_length, remaining_data_length, &request);
> +	if (rc)
> +		return rc;
> +
> +	num_sgs = sgl ? sg_nents(sgl) : 0;
> +	for_each_sg(sgl, sg, num_sgs, i) {
> +		request->sge[i+1].addr =
> +			ib_dma_map_page(info->id->device, sg_page(sg),
> +			       sg->offset, sg->length, DMA_BIDIRECTIONAL);
> +		if (ib_dma_mapping_error(
> +				info->id->device, request->sge[i+1].addr)) {
> +			rc = -EIO;
> +			request->sge[i+1].addr = 0;
> +			goto dma_mapping_failure;
> +		}
> +		request->sge[i+1].length = sg->length;
> +		request->sge[i+1].lkey = info->pd->local_dma_lkey;
> +		request->num_sge++;
> +	}
> +
> +	rc = smbd_post_send(info, request, data_length);
> +	if (!rc)
> +		return 0;
> +
> +dma_mapping_failure:
> +	for (i = 1; i < request->num_sge; i++)
> +		if (request->sge[i].addr)
> +			ib_dma_unmap_single(info->id->device,
> +					    request->sge[i].addr,
> +					    request->sge[i].length,
> +					    DMA_TO_DEVICE);
> +	smbd_destroy_header(info, request);
> +	return rc;
> +}
> +
> +/*
> + * Send an empty message
> + * Empty message is used to extend credits to peer to for keep live
> + * while there is no upper layer payload to send at the time
> + */
> +static int smbd_post_send_empty(struct smbd_connection *info)
> +{
> +	info->count_send_empty++;
> +	return smbd_post_send_sgl(info, NULL, 0, 0);
> +}
> +
> +/*
> + * Post a receive request to the transport
> + * The remote peer can only send data when a receive request is posted
> + * The interaction is controlled by send/receive credit system
> + */
> +static int smbd_post_recv(
> +		struct smbd_connection *info, struct smbd_response
> *response)
> +{
> +	struct ib_recv_wr recv_wr, *recv_wr_fail = NULL;
> +	int rc = -EIO;
> +
> +	response->sge.addr = ib_dma_map_single(
> +				info->id->device, response->packet,
> +				info->max_receive_size,
> DMA_FROM_DEVICE);
> +	if (ib_dma_mapping_error(info->id->device, response->sge.addr))
> +		return rc;
> +
> +	response->sge.length = info->max_receive_size;
> +	response->sge.lkey = info->pd->local_dma_lkey;
> +
> +	response->cqe.done = recv_done;
> +
> +	recv_wr.wr_cqe = &response->cqe;
> +	recv_wr.next = NULL;
> +	recv_wr.sg_list = &response->sge;
> +	recv_wr.num_sge = 1;
> +
> +	rc = ib_post_recv(info->id->qp, &recv_wr, &recv_wr_fail);
> +	if (rc) {
> +		ib_dma_unmap_single(info->id->device, response-
> >sge.addr,
> +				    response->sge.length,
> DMA_FROM_DEVICE);
> +
> +		log_rdma_recv(ERR, "ib_post_recv failed rc=%d\n", rc);
> +	}
> +
> +	return rc;
> +}
> +
> +/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */
> +static int smbd_negotiate(struct smbd_connection *info)
> +{
> +	int rc;
> +	struct smbd_response *response = get_receive_buffer(info);
> +
> +	response->type = SMBD_NEGOTIATE_RESP;
> +	rc = smbd_post_recv(info, response);
> +	log_rdma_event(INFO,
> +		"smbd_post_recv rc=%d iov.addr=%llx iov.length=%x "
> +		"iov.lkey=%x\n",
> +		rc, response->sge.addr,
> +		response->sge.length, response->sge.lkey);
> +	if (rc)
> +		return rc;
> +
> +	init_completion(&info->negotiate_completion);
> +	info->negotiate_done = false;
> +	rc = smbd_post_send_negotiate_req(info);
> +	if (rc)
> +		return rc;
> +
> +	rc = wait_for_completion_interruptible_timeout(
> +		&info->negotiate_completion, SMBD_NEGOTIATE_TIMEOUT
> * HZ);
> +	log_rdma_event(INFO, "wait_for_completion_timeout rc=%d\n",
> rc);
> +
> +	if (info->negotiate_done)
> +		return 0;
> +
> +	if (rc == 0)
> +		rc = -ETIMEDOUT;
> +	else if (rc == -ERESTARTSYS)
> +		rc = -EINTR;
> +	else
> +		rc = -ENOTCONN;
> +
> +	return rc;
> +}
> +
> +static void put_empty_packet(
> +		struct smbd_connection *info, struct smbd_response
> *response)
> +{
> +	spin_lock(&info->empty_packet_queue_lock);
> +	list_add_tail(&response->list, &info->empty_packet_queue);
> +	info->count_empty_packet_queue++;
> +	spin_unlock(&info->empty_packet_queue_lock);
> +
> +	queue_work(info->workqueue, &info->post_send_credits_work);
> +}
> +
> +/*
> + * Implement Connection.FragmentReassemblyBuffer defined in [MS-
> SMBD] 3.1.1.1
> + * This is a queue for reassembling upper layer payload and present to
> upper
> + * layer. All the inncoming payload go to the reassembly queue, regardless
> of
> + * if reassembly is required. The uuper layer code reads from the queue for
> all
> + * incoming payloads.
> + * Put a received packet to the reassembly queue
> + * response: the packet received
> + * data_length: the size of payload in this packet
> + */
> +static void enqueue_reassembly(
> +	struct smbd_connection *info,
> +	struct smbd_response *response,
> +	int data_length)
> +{
> +	spin_lock(&info->reassembly_queue_lock);
> +	list_add_tail(&response->list, &info->reassembly_queue);
> +	info->reassembly_queue_length++;
> +	/*
> +	 * Make sure reassembly_data_length is updated after list and
> +	 * reassembly_queue_length are updated. On the dequeue side
> +	 * reassembly_data_length is checked without a lock to determine
> +	 * if reassembly_queue_length and list is up to date
> +	 */
> +	virt_wmb();
> +	info->reassembly_data_length += data_length;
> +	spin_unlock(&info->reassembly_queue_lock);
> +	info->count_reassembly_queue++;
> +	info->count_enqueue_reassembly_queue++;
> +}
> +
> +/*
> + * Get the first entry at the front of reassembly queue
> + * Caller is responsible for locking
> + * return value: the first entry if any, NULL if queue is empty
> + */
> +static struct smbd_response *_get_first_reassembly(struct
> smbd_connection *info)
> +{
> +	struct smbd_response *ret = NULL;
> +
> +	if (!list_empty(&info->reassembly_queue)) {
> +		ret = list_first_entry(
> +			&info->reassembly_queue,
> +			struct smbd_response, list);
> +	}
> +	return ret;
> +}
> +
> +static struct smbd_response *get_empty_queue_buffer(
> +		struct smbd_connection *info)
> +{
> +	struct smbd_response *ret = NULL;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&info->empty_packet_queue_lock, flags);
> +	if (!list_empty(&info->empty_packet_queue)) {
> +		ret = list_first_entry(
> +			&info->empty_packet_queue,
> +			struct smbd_response, list);
> +		list_del(&ret->list);
> +		info->count_empty_packet_queue--;
> +	}
> +	spin_unlock_irqrestore(&info->empty_packet_queue_lock, flags);
> +
> +	return ret;
> +}
> +
> +/*
> + * Get a receive buffer
> + * For each remote send, we need to post a receive. The receive buffers are
> + * pre-allocated in advance.
> + * return value: the receive buffer, NULL if none is available
> + */
> +static struct smbd_response *get_receive_buffer(struct smbd_connection
> *info)
> +{
> +	struct smbd_response *ret = NULL;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&info->receive_queue_lock, flags);
> +	if (!list_empty(&info->receive_queue)) {
> +		ret = list_first_entry(
> +			&info->receive_queue,
> +			struct smbd_response, list);
> +		list_del(&ret->list);
> +		info->count_receive_queue--;
> +		info->count_get_receive_buffer++;
> +	}
> +	spin_unlock_irqrestore(&info->receive_queue_lock, flags);
> +
> +	return ret;
> +}
> +
> +/*
> + * Return a receive buffer
> + * Upon returning of a receive buffer, we can post new receive and extend
> + * more receive credits to remote peer. This is done immediately after a
> + * receive buffer is returned.
> + */
> +static void put_receive_buffer(
> +	struct smbd_connection *info, struct smbd_response *response,
> +	bool lock)
> +{
> +	unsigned long flags;
> +
> +	ib_dma_unmap_single(info->id->device, response->sge.addr,
> +		response->sge.length, DMA_FROM_DEVICE);
> +
> +	if (lock)
> +		spin_lock_irqsave(&info->receive_queue_lock, flags);
> +	list_add_tail(&response->list, &info->receive_queue);
> +	info->count_receive_queue++;
> +	info->count_put_receive_buffer++;
> +	if (lock)
> +		spin_unlock_irqrestore(&info->receive_queue_lock, flags);
> +
> +	queue_work(info->workqueue, &info->post_send_credits_work);
> +}
> +
> +/* Preallocate all receive buffer on transport establishment */
> +static int allocate_receive_buffers(struct smbd_connection *info, int
> num_buf)
> +{
> +	int i;
> +	struct smbd_response *response;
> +
> +	INIT_LIST_HEAD(&info->reassembly_queue);
> +	spin_lock_init(&info->reassembly_queue_lock);
> +	info->reassembly_data_length = 0;
> +	info->reassembly_queue_length = 0;
> +
> +	INIT_LIST_HEAD(&info->receive_queue);
> +	spin_lock_init(&info->receive_queue_lock);
> +	info->count_receive_queue = 0;
> +
> +	INIT_LIST_HEAD(&info->empty_packet_queue);
> +	spin_lock_init(&info->empty_packet_queue_lock);
> +	info->count_empty_packet_queue = 0;
> +
> +	init_waitqueue_head(&info->wait_receive_queues);
> +
> +	for (i = 0; i < num_buf; i++) {
> +		response = mempool_alloc(info->response_mempool,
> GFP_KERNEL);
> +		if (!response)
> +			goto allocate_failed;
> +
> +		response->info = info;
> +		list_add_tail(&response->list, &info->receive_queue);
> +		info->count_receive_queue++;
> +	}
> +
> +	return 0;
> +
> +allocate_failed:
> +	while (!list_empty(&info->receive_queue)) {
> +		response = list_first_entry(
> +				&info->receive_queue,
> +				struct smbd_response, list);
> +		list_del(&response->list);
> +		info->count_receive_queue--;
> +
> +		mempool_free(response, info->response_mempool);
> +	}
> +	return -ENOMEM;
> +}
> +
> +static void destroy_receive_buffers(struct smbd_connection *info)
> +{
> +	struct smbd_response *response;
> +
> +	while ((response = get_receive_buffer(info)))
> +		mempool_free(response, info->response_mempool);
> +
> +	while ((response = get_empty_queue_buffer(info)))
> +		mempool_free(response, info->response_mempool);
> +}
> +
> +/*
> + * Check and send an immediate or keep alive packet
> + * The condition to send those packets are defined in [MS-SMBD] 3.1.1.1
> + * Connection.KeepaliveRequested and Connection.SendImmediate
> + * The idea is to extend credits to server as soon as it becomes available
> + */
> +static void send_immediate_work(struct work_struct *work)
> +{
> +	struct smbd_connection *info = container_of(
> +					work, struct smbd_connection,
> +					send_immediate_work.work);
> +
> +	if (info->keep_alive_requested == KEEP_ALIVE_PENDING ||
> +	    info->send_immediate) {
> +		log_keep_alive(INFO, "send an empty message\n");
> +		smbd_post_send_empty(info);
> +	}
> +}
> +
> +/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */
> +static void idle_connection_timer(struct work_struct *work)
> +{
> +	struct smbd_connection *info = container_of(
> +					work, struct smbd_connection,
> +					idle_timer_work.work);
> +
> +	if (info->keep_alive_requested != KEEP_ALIVE_NONE) {
> +		log_keep_alive(ERR,
> +			"error status info->keep_alive_requested=%d\n",
> +			info->keep_alive_requested);
> +		smbd_disconnect_rdma_connection(info);
> +		return;
> +	}
> +
> +	log_keep_alive(INFO, "about to send an empty idle message\n");
> +	smbd_post_send_empty(info);
> +
> +	/* Setup the next idle timeout work */
> +	queue_delayed_work(info->workqueue, &info->idle_timer_work,
> +			info->keep_alive_interval*HZ);
> +}
> +
> +static void destroy_caches_and_workqueue(struct smbd_connection
> *info)
> +{
> +	destroy_receive_buffers(info);
> +	destroy_workqueue(info->workqueue);
> +	mempool_destroy(info->response_mempool);
> +	kmem_cache_destroy(info->response_cache);
> +	mempool_destroy(info->request_mempool);
> +	kmem_cache_destroy(info->request_cache);
> +}
> +
> +#define MAX_NAME_LEN	80
> +static int allocate_caches_and_workqueue(struct smbd_connection *info)
> +{
> +	char name[MAX_NAME_LEN];
> +	int rc;
> +
> +	snprintf(name, MAX_NAME_LEN, "smbd_request_%p", info);
> +	info->request_cache =
> +		kmem_cache_create(
> +			name,
> +			sizeof(struct smbd_request) +
> +				sizeof(struct smbd_data_transfer),
> +			0, SLAB_HWCACHE_ALIGN, NULL);
> +	if (!info->request_cache)
> +		return -ENOMEM;
> +
> +	info->request_mempool =
> +		mempool_create(info->send_credit_target,
> mempool_alloc_slab,
> +			mempool_free_slab, info->request_cache);
> +	if (!info->request_mempool)
> +		goto out1;
> +
> +	snprintf(name, MAX_NAME_LEN, "smbd_response_%p", info);
> +	info->response_cache =
> +		kmem_cache_create(
> +			name,
> +			sizeof(struct smbd_response) +
> +				info->max_receive_size,
> +			0, SLAB_HWCACHE_ALIGN, NULL);
> +	if (!info->response_cache)
> +		goto out2;
> +
> +	info->response_mempool =
> +		mempool_create(info->receive_credit_max,
> mempool_alloc_slab,
> +		       mempool_free_slab, info->response_cache);
> +	if (!info->response_mempool)
> +		goto out3;
> +
> +	snprintf(name, MAX_NAME_LEN, "smbd_%p", info);
> +	info->workqueue = create_workqueue(name);
> +	if (!info->workqueue)
> +		goto out4;
> +
> +	rc = allocate_receive_buffers(info, info->receive_credit_max);
> +	if (rc) {
> +		log_rdma_event(ERR, "failed to allocate receive buffers\n");
> +		goto out5;
> +	}
> +
> +	return 0;
> +
> +out5:
> +	destroy_workqueue(info->workqueue);
> +out4:
> +	mempool_destroy(info->response_mempool);
> +out3:
> +	kmem_cache_destroy(info->response_cache);
> +out2:
> +	mempool_destroy(info->request_mempool);
> +out1:
> +	kmem_cache_destroy(info->request_cache);
> +	return -ENOMEM;
> +}
> +
> +/* Create a SMBD connection, called by upper layer */
> +struct smbd_connection *_smbd_get_connection(
> +	struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
> +{
> +	int rc;
> +	struct smbd_connection *info;
> +	struct rdma_conn_param conn_param;
> +	struct ib_qp_init_attr qp_attr;
> +	struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;
> +
> +	info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL);
> +	if (!info)
> +		return NULL;
> +
> +	info->transport_status = SMBD_CONNECTING;
> +	rc = smbd_ia_open(info, dstaddr, port);
> +	if (rc) {
> +		log_rdma_event(INFO, "smbd_ia_open rc=%d\n", rc);
> +		goto create_id_failed;
> +	}
> +
> +	if (smbd_send_credit_target > info->id->device->attrs.max_cqe ||
> +	    smbd_send_credit_target > info->id->device->attrs.max_qp_wr) {
> +		log_rdma_event(ERR,
> +			"consider lowering send_credit_target = %d. "
> +			"Possible CQE overrun, device "
> +			"reporting max_cpe %d max_qp_wr %d\n",
> +			smbd_send_credit_target,
> +			info->id->device->attrs.max_cqe,
> +			info->id->device->attrs.max_qp_wr);
> +		goto config_failed;
> +	}
> +
> +	if (smbd_receive_credit_max > info->id->device->attrs.max_cqe ||
> +	    smbd_receive_credit_max > info->id->device->attrs.max_qp_wr)
> {
> +		log_rdma_event(ERR,
> +			"consider lowering receive_credit_max = %d. "
> +			"Possible CQE overrun, device "
> +			"reporting max_cpe %d max_qp_wr %d\n",
> +			smbd_receive_credit_max,
> +			info->id->device->attrs.max_cqe,
> +			info->id->device->attrs.max_qp_wr);
> +		goto config_failed;
> +	}
> +
> +	info->receive_credit_max = smbd_receive_credit_max;
> +	info->send_credit_target = smbd_send_credit_target;
> +	info->max_send_size = smbd_max_send_size;
> +	info->max_fragmented_recv_size =
> smbd_max_fragmented_recv_size;
> +	info->max_receive_size = smbd_max_receive_size;
> +	info->keep_alive_interval = smbd_keep_alive_interval;
> +
> +	if (SMBDIRECT_MAX_SGE > info->id->device->attrs.max_sge) {
> +		log_rdma_event(ERR, "warning: device max_sge = %d too
> small\n",
> +			info->id->device->attrs.max_sge);
> +		log_rdma_event(ERR, "Queue Pair creation may fail\n");
> +	}
> +
> +	info->send_cq = NULL;
> +	info->recv_cq = NULL;
> +	info->send_cq = ib_alloc_cq(info->id->device, info,
> +			info->send_credit_target, 0, IB_POLL_SOFTIRQ);
> +	if (IS_ERR(info->send_cq)) {
> +		info->send_cq = NULL;
> +		goto alloc_cq_failed;
> +	}
> +
> +	info->recv_cq = ib_alloc_cq(info->id->device, info,
> +			info->receive_credit_max, 0, IB_POLL_SOFTIRQ);
> +	if (IS_ERR(info->recv_cq)) {
> +		info->recv_cq = NULL;
> +		goto alloc_cq_failed;
> +	}
> +
> +	memset(&qp_attr, 0, sizeof(qp_attr));
> +	qp_attr.event_handler = smbd_qp_async_error_upcall;
> +	qp_attr.qp_context = info;
> +	qp_attr.cap.max_send_wr = info->send_credit_target;
> +	qp_attr.cap.max_recv_wr = info->receive_credit_max;
> +	qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
> +	qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
> +	qp_attr.cap.max_inline_data = 0;
> +	qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
> +	qp_attr.qp_type = IB_QPT_RC;
> +	qp_attr.send_cq = info->send_cq;
> +	qp_attr.recv_cq = info->recv_cq;
> +	qp_attr.port_num = ~0;
> +
> +	rc = rdma_create_qp(info->id, info->pd, &qp_attr);
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_create_qp failed %i\n", rc);
> +		goto create_qp_failed;
> +	}
> +
> +	memset(&conn_param, 0, sizeof(conn_param));
> +	conn_param.initiator_depth = 0;
> +
> +	conn_param.retry_count = SMBD_CM_RETRY;
> +	conn_param.rnr_retry_count = SMBD_CM_RNR_RETRY;
> +	conn_param.flow_control = 0;
> +	init_waitqueue_head(&info->wait_destroy);
> +
> +	log_rdma_event(INFO, "connecting to IP %pI4 port %d\n",
> +		&addr_in->sin_addr, port);
> +
> +	init_waitqueue_head(&info->conn_wait);
> +	rc = rdma_connect(info->id, &conn_param);
> +	if (rc) {
> +		log_rdma_event(ERR, "rdma_connect() failed with %i\n", rc);
> +		goto rdma_connect_failed;
> +	}
> +
> +	wait_event_interruptible(
> +		info->conn_wait, info->transport_status !=
> SMBD_CONNECTING);
> +
> +	if (info->transport_status != SMBD_CONNECTED) {
> +		log_rdma_event(ERR, "rdma_connect failed port=%d\n",
> port);
> +		goto rdma_connect_failed;
> +	}
> +
> +	log_rdma_event(INFO, "rdma_connect connected\n");
> +
> +	rc = allocate_caches_and_workqueue(info);
> +	if (rc) {
> +		log_rdma_event(ERR, "cache allocation failed\n");
> +		goto allocate_cache_failed;
> +	}
> +
> +	init_waitqueue_head(&info->wait_send_queue);
> +	init_waitqueue_head(&info->wait_reassembly_queue);
> +
> +	INIT_DELAYED_WORK(&info->idle_timer_work,
> idle_connection_timer);
> +	INIT_DELAYED_WORK(&info->send_immediate_work,
> send_immediate_work);
> +	queue_delayed_work(info->workqueue, &info->idle_timer_work,
> +		info->keep_alive_interval*HZ);
> +
> +	init_waitqueue_head(&info->wait_send_pending);
> +	atomic_set(&info->send_pending, 0);
> +
> +	init_waitqueue_head(&info->wait_send_payload_pending);
> +	atomic_set(&info->send_payload_pending, 0);
> +
> +	INIT_WORK(&info->disconnect_work,
> smbd_disconnect_rdma_work);
> +	INIT_WORK(&info->destroy_work, smbd_destroy_rdma_work);
> +	INIT_WORK(&info->recv_done_work, smbd_recv_done_work);
> +	INIT_WORK(&info->post_send_credits_work,
> smbd_post_send_credits);
> +	info->new_credits_offered = 0;
> +	spin_lock_init(&info->lock_new_credits_offered);
> +
> +	rc = smbd_negotiate(info);
> +	if (rc) {
> +		log_rdma_event(ERR, "smbd_negotiate rc=%d\n", rc);
> +		goto negotiation_failed;
> +	}
> +
> +	return info;
> +
> +negotiation_failed:
> +	cancel_delayed_work_sync(&info->idle_timer_work);
> +	destroy_caches_and_workqueue(info);
> +	info->transport_status = SMBD_NEGOTIATE_FAILED;
> +	init_waitqueue_head(&info->conn_wait);
> +	rdma_disconnect(info->id);
> +	wait_event(info->conn_wait,
> +		info->transport_status == SMBD_DISCONNECTED);
> +
> +allocate_cache_failed:
> +rdma_connect_failed:
> +	rdma_destroy_qp(info->id);
> +
> +create_qp_failed:
> +alloc_cq_failed:
> +	if (info->send_cq)
> +		ib_free_cq(info->send_cq);
> +	if (info->recv_cq)
> +		ib_free_cq(info->recv_cq);
> +
> +config_failed:
> +	ib_dealloc_pd(info->pd);
> +	rdma_destroy_id(info->id);
> +
> +create_id_failed:
> +	kfree(info);
> +	return NULL;
> +}
> diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
> index c55f28b..35bc25b 100644
> --- a/fs/cifs/smbdirect.h
> +++ b/fs/cifs/smbdirect.h
> @@ -16,6 +16,286 @@
>  #ifndef _SMBDIRECT_H
>  #define _SMBDIRECT_H
> 
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +#define cifs_rdma_enabled(server)	((server)->rdma)
> +
> +#include "cifsglob.h"
> +#include <rdma/ib_verbs.h>
> +#include <rdma/rdma_cm.h>
> +#include <linux/mempool.h>
> +
> +enum keep_alive_status {
> +	KEEP_ALIVE_NONE,
> +	KEEP_ALIVE_PENDING,
> +	KEEP_ALIVE_SENT,
> +};
> +
> +enum smbd_connection_status {
> +	SMBD_CREATED,
> +	SMBD_CONNECTING,
> +	SMBD_CONNECTED,
> +	SMBD_NEGOTIATE_FAILED,
> +	SMBD_DISCONNECTING,
> +	SMBD_DISCONNECTED,
> +	SMBD_DESTROYED
> +};
> +
> +/*
> + * The context for the SMBDirect transport
> + * Everything related to the transport is here. It has several logical parts
> + * 1. RDMA related structures
> + * 2. SMBDirect connection parameters
> + * 3. Memory registrations
> + * 4. Receive and reassembly queues for data receive path
> + * 5. mempools for allocating packets
> + */
> +struct smbd_connection {
> +	enum smbd_connection_status transport_status;
> +
> +	/* RDMA related */
> +	struct rdma_cm_id *id;
> +	struct ib_qp_init_attr qp_attr;
> +	struct ib_pd *pd;
> +	struct ib_cq *send_cq, *recv_cq;
> +	struct ib_device_attr dev_attr;
> +	int ri_rc;
> +	struct completion ri_done;
> +	wait_queue_head_t conn_wait;
> +	wait_queue_head_t wait_destroy;
> +
> +	struct completion negotiate_completion;
> +	bool negotiate_done;
> +
> +	struct work_struct destroy_work;
> +	struct work_struct disconnect_work;
> +	struct work_struct recv_done_work;
> +	struct work_struct post_send_credits_work;
> +
> +	spinlock_t lock_new_credits_offered;
> +	int new_credits_offered;
> +
> +	/* Connection parameters defined in [MS-SMBD] 3.1.1.1 */
> +	int receive_credit_max;
> +	int send_credit_target;
> +	int max_send_size;
> +	int max_fragmented_recv_size;
> +	int max_fragmented_send_size;
> +	int max_receive_size;
> +	int keep_alive_interval;
> +	int max_readwrite_size;
> +	enum keep_alive_status keep_alive_requested;
> +	int protocol;
> +	atomic_t send_credits;
> +	atomic_t receive_credits;
> +	int receive_credit_target;
> +	int fragment_reassembly_remaining;
> +
> +	/* Activity accoutning */
> +
> +	atomic_t send_pending;
> +	wait_queue_head_t wait_send_pending;
> +	atomic_t send_payload_pending;
> +	wait_queue_head_t wait_send_payload_pending;
> +
> +	/* Receive queue */
> +	struct list_head receive_queue;
> +	int count_receive_queue;
> +	spinlock_t receive_queue_lock;
> +
> +	struct list_head empty_packet_queue;
> +	int count_empty_packet_queue;
> +	spinlock_t empty_packet_queue_lock;
> +
> +	wait_queue_head_t wait_receive_queues;
> +
> +	/* Reassembly queue */
> +	struct list_head reassembly_queue;
> +	spinlock_t reassembly_queue_lock;
> +	wait_queue_head_t wait_reassembly_queue;
> +
> +	/* total data length of reassembly queue */
> +	int reassembly_data_length;
> +	int reassembly_queue_length;
> +	/* the offset to first buffer in reassembly queue */
> +	int first_entry_offset;
> +
> +	bool send_immediate;
> +
> +	wait_queue_head_t wait_send_queue;
> +
> +	/*
> +	 * Indicate if we have received a full packet on the connection
> +	 * This is used to identify the first SMBD packet of a assembled
> +	 * payload (SMB packet) in reassembly queue so we can return a
> +	 * RFC1002 length to upper layer to indicate the length of the SMB
> +	 * packet received
> +	 */
> +	bool full_packet_received;
> +
> +	struct workqueue_struct *workqueue;
> +	struct delayed_work idle_timer_work;
> +	struct delayed_work send_immediate_work;
> +
> +	/* Memory pool for preallocating buffers */
> +	/* request pool for RDMA send */
> +	struct kmem_cache *request_cache;
> +	mempool_t *request_mempool;
> +
> +	/* response pool for RDMA receive */
> +	struct kmem_cache *response_cache;
> +	mempool_t *response_mempool;
> +
> +	/* for debug purposes */
> +	unsigned int count_get_receive_buffer;
> +	unsigned int count_put_receive_buffer;
> +	unsigned int count_reassembly_queue;
> +	unsigned int count_enqueue_reassembly_queue;
> +	unsigned int count_dequeue_reassembly_queue;
> +	unsigned int count_send_empty;
> +};
> +
> +enum smbd_message_type {
> +	SMBD_NEGOTIATE_RESP,
> +	SMBD_TRANSFER_DATA,
> +};
> +
> +#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
> +
> +/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */
> +struct smbd_negotiate_req {
> +	__le16 min_version;
> +	__le16 max_version;
> +	__le16 reserved;
> +	__le16 credits_requested;
> +	__le32 preferred_send_size;
> +	__le32 max_receive_size;
> +	__le32 max_fragmented_size;
> +} __packed;
> +
> +/* SMBD negotiation response packet [MS-SMBD] 2.2.2 */
> +struct smbd_negotiate_resp {
> +	__le16 min_version;
> +	__le16 max_version;
> +	__le16 negotiated_version;
> +	__le16 reserved;
> +	__le16 credits_requested;
> +	__le16 credits_granted;
> +	__le32 status;
> +	__le32 max_readwrite_size;
> +	__le32 preferred_send_size;
> +	__le32 max_receive_size;
> +	__le32 max_fragmented_size;
> +} __packed;
> +
> +/* SMBD data transfer packet with payload [MS-SMBD] 2.2.3 */
> +struct smbd_data_transfer {
> +	__le16 credits_requested;
> +	__le16 credits_granted;
> +	__le16 flags;
> +	__le16 reserved;
> +	__le32 remaining_data_length;
> +	__le32 data_offset;
> +	__le32 data_length;
> +	__le32 padding;
> +	__u8 buffer[];
> +} __packed;
> +
> +/* The packet fields for a registered RDMA buffer */
> +struct smbd_buffer_descriptor_v1 {
> +	__le64 offset;
> +	__le32 token;
> +	__le32 length;
> +} __packed;
> +
>  /* Default maximum number of SGEs in a RDMA send/recv */
>  #define SMBDIRECT_MAX_SGE	16
> +/* The context for a SMBD request */
> +struct smbd_request {
> +	struct smbd_connection *info;
> +	struct ib_cqe cqe;
> +
> +	/* true if this request carries upper layer payload */
> +	bool has_payload;
> +
> +	/* the SGE entries for this packet */
> +	struct ib_sge sge[SMBDIRECT_MAX_SGE];
> +	int num_sge;
> +
> +	/* SMBD packet header follows this structure */
> +	u8 packet[];
> +};
> +
> +/* The context for a SMBD response */
> +struct smbd_response {
> +	struct smbd_connection *info;
> +	struct ib_cqe cqe;
> +	struct ib_sge sge;
> +
> +	enum smbd_message_type type;
> +
> +	/* Link to receive queue or reassembly queue */
> +	struct list_head list;
> +
> +	/* Indicate if this is the 1st packet of a payload */
> +	bool first_segment;
> +
> +	/* SMBD packet header and payload follows this structure */
> +	u8 packet[];
> +};
> +
> +/* Create a SMBDirect session */
> +struct smbd_connection *smbd_get_connection(
> +	struct TCP_Server_Info *server, struct sockaddr *dstaddr);
> +
> +/* Reconnect SMBDirect session */
> +int smbd_reconnect(struct TCP_Server_Info *server);
> +
> +/* Destroy SMBDirect session */
> +void smbd_destroy(struct smbd_connection *info);
> +
> +/* Interface for carrying upper layer I/O through send/recv */
> +int smbd_recv(struct smbd_connection *info, struct msghdr *msg);
> +int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst);
> +
> +enum mr_state {
> +	MR_READY,
> +	MR_REGISTERED,
> +	MR_INVALIDATED,
> +	MR_ERROR
> +};
> +
> +struct smbd_mr {
> +	struct smbd_connection	*conn;
> +	struct list_head	list;
> +	enum mr_state		state;
> +	struct ib_mr		*mr;
> +	struct scatterlist	*sgl;
> +	int			sgl_count;
> +	enum dma_data_direction	dir;
> +	union {
> +		struct ib_reg_wr	wr;
> +		struct ib_send_wr	inv_wr;
> +	};
> +	struct ib_cqe		cqe;
> +	bool			need_invalidate;
> +	struct completion	invalidate_done;
> +};
> +
> +/* Interfaces to register and deregister MR for RDMA read/write */
> +struct smbd_mr *smbd_register_mr(
> +	struct smbd_connection *info, struct page *pages[], int num_pages,
> +	int tailsz, bool writing, bool need_invalidate);
> +int smbd_deregister_mr(struct smbd_mr *mr);
> +
> +#else
> +#define cifs_rdma_enabled(server)	0
> +struct smbd_connection{};
> +static inline void *smbd_get_connection(
> +	struct TCP_Server_Info *server, struct sockaddr *dstaddr) {return
> NULL;}
> +static inline int smbd_reconnect(struct TCP_Server_Info *server) {return -
> 1;}
> +static inline void smbd_destroy(struct smbd_connection *info) {}
> +static inline int smbd_recv(struct smbd_connection *info, struct msghdr
> *msg) {return -1;}
> +static inline int smbd_send(struct smbd_connection *info, struct smb_rqst
> *rqst) {return -1;}
> +#endif
> +
>  #endif
> --
> 2.7.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.ke
> rnel.org%2Fmajordomo-
> info.html&data=02%7C01%7Clongli%40microsoft.com%7Ce7da39e3ff8f409ea
> 29208d52fda17a4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636
> 467536124091212&sdata=eRCijuXR1Y%2FsFds3UZDrdK6AoDutBBLCQ0UZU5v
> 5XAc%3D&reserved=0

[Patch v7 07/22] CIFS: SMBD: Implement function to create a SMB Direct connection

[Long Li]

From: Long Li <longli@microsoft.com>

The upper layer calls this function to connect to peer through SMB Direct.
Each SMB Direct connection is based on a RDMA RC Queue Pair.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smbdirect.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index 021d527..1db39a34 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -1651,3 +1651,20 @@ struct smbd_connection *_smbd_get_connection(
 	kfree(info);
 	return NULL;
 }
+
+struct smbd_connection *smbd_get_connection(
+	struct TCP_Server_Info *server, struct sockaddr *dstaddr)
+{
+	struct smbd_connection *ret;
+	int port = SMBD_PORT;
+
+try_again:
+	ret = _smbd_get_connection(server, dstaddr, port);
+
+	/* Try SMB_PORT if SMBD_PORT doesn't work */
+	if (!ret && port == SMBD_PORT) {
+		port = SMB_PORT;
+		goto try_again;
+	}
+	return ret;
+}
-- 
2.7.4

[Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

[Long Li]

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

Build SMB Direct code when this option is set.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/Kconfig | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig
index f724361..8d05fff 100644
--- a/fs/cifs/Kconfig
+++ b/fs/cifs/Kconfig
@@ -191,6 +191,14 @@ config CIFS_SMB311
 	  This dialect includes improved security negotiation features.
 	  If unsure, say N
 
+config CIFS_SMB_DIRECT
+	bool "SMB Direct support (Experimental)"
+	depends on CIFS && INFINIBAND
+	help
+	  Enables SMB Direct experimental support for SMB 3.0, 3.02 and 3.1.1.
+	  SMB Direct allows transferring SMB packets over RDMA. If unsure,
+	  say N.
+
 config CIFS_FSCACHE
 	  bool "Provide CIFS client caching support"
 	  depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
-- 
2.7.4

[Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

[Long Li]

From: Long Li <longli@microsoft.com>

Build SMB Direct code when this option is set.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/Kconfig | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig
index f724361..8d05fff 100644
--- a/fs/cifs/Kconfig
+++ b/fs/cifs/Kconfig
@@ -191,6 +191,14 @@ config CIFS_SMB311
 	  This dialect includes improved security negotiation features.
 	  If unsure, say N
 
+config CIFS_SMB_DIRECT
+	bool "SMB Direct support (Experimental)"
+	depends on CIFS && INFINIBAND
+	help
+	  Enables SMB Direct experimental support for SMB 3.0, 3.02 and 3.1.1.
+	  SMB Direct allows transferring SMB packets over RDMA. If unsure,
+	  say N.
+
 config CIFS_FSCACHE
 	  bool "Provide CIFS client caching support"
 	  depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
-- 
2.7.4

RE: [Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>:
> From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
>
> Build SMB Direct code when this option is set.
>
> Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
> ---
>  fs/cifs/Kconfig | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig
> index f724361..8d05fff 100644
> --- a/fs/cifs/Kconfig
> +++ b/fs/cifs/Kconfig
> @@ -191,6 +191,14 @@ config CIFS_SMB311
>           This dialect includes improved security negotiation features.
>           If unsure, say N
>
> +config CIFS_SMB_DIRECT
> +       bool "SMB Direct support (Experimental)"
> +       depends on CIFS && INFINIBAND
> +       help
> +         Enables SMB Direct experimental support for SMB 3.0, 3.02 and 3.1.1.
> +         SMB Direct allows transferring SMB packets over RDMA. If unsure,
> +         say N.
> +
>  config CIFS_FSCACHE
>           bool "Provide CIFS client caching support"
>           depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reviewed-by: Pavel Shilovsky <pshilov-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

--
Best regards,
Pavel Shilovsky
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli@exchange.microsoft.com>:
> From: Long Li <longli@microsoft.com>
>
> Build SMB Direct code when this option is set.
>
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>  fs/cifs/Kconfig | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig
> index f724361..8d05fff 100644
> --- a/fs/cifs/Kconfig
> +++ b/fs/cifs/Kconfig
> @@ -191,6 +191,14 @@ config CIFS_SMB311
>           This dialect includes improved security negotiation features.
>           If unsure, say N
>
> +config CIFS_SMB_DIRECT
> +       bool "SMB Direct support (Experimental)"
> +       depends on CIFS && INFINIBAND
> +       help
> +         Enables SMB Direct experimental support for SMB 3.0, 3.02 and 3.1.1.
> +         SMB Direct allows transferring SMB packets over RDMA. If unsure,
> +         say N.
> +
>  config CIFS_FSCACHE
>           bool "Provide CIFS client caching support"
>           depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reviewed-by: Pavel Shilovsky <pshilov@microsoft.com>

--
Best regards,
Pavel Shilovsky

RE: [Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>:
> From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
>
> Build SMB Direct code when this option is set.
>
> Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
> ---
>  fs/cifs/Kconfig | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig
> index f724361..8d05fff 100644
> --- a/fs/cifs/Kconfig
> +++ b/fs/cifs/Kconfig
> @@ -191,6 +191,14 @@ config CIFS_SMB311
>           This dialect includes improved security negotiation features.
>           If unsure, say N
>
> +config CIFS_SMB_DIRECT
> +       bool "SMB Direct support (Experimental)"
> +       depends on CIFS && INFINIBAND
> +       help
> +         Enables SMB Direct experimental support for SMB 3.0, 3.02 and 3.1.1.
> +         SMB Direct allows transferring SMB packets over RDMA. If unsure,
> +         say N.
> +
>  config CIFS_FSCACHE
>           bool "Provide CIFS client caching support"
>           depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reviewed-by: Pavel Shilovsky <pshilov-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

--
Best regards,
Pavel Shilovsky
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>


----- Original Message -----
From: "Long Li" <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>
To: "Steve French" <sfrench-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>, linux-cifs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, samba-technical-w/Ol4Ecudpl8XjKLYN78aQ@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "Christoph Hellwig" <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, "Tom Talpey" <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Matthew Wilcox" <mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Stephen Hemminger" <sthemmin-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Cc: "Long Li" <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Sent: Tuesday, 7 November, 2017 7:54:54 PM
Subject: [Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

Build SMB Direct code when this option is set.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/Kconfig | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig
index f724361..8d05fff 100644
--- a/fs/cifs/Kconfig
+++ b/fs/cifs/Kconfig
@@ -191,6 +191,14 @@ config CIFS_SMB311
 	  This dialect includes improved security negotiation features.
 	  If unsure, say N
 
+config CIFS_SMB_DIRECT
+	bool "SMB Direct support (Experimental)"
+	depends on CIFS && INFINIBAND
+	help
+	  Enables SMB Direct experimental support for SMB 3.0, 3.02 and 3.1.1.
+	  SMB Direct allows transferring SMB packets over RDMA. If unsure,
+	  say N.
+
 config CIFS_FSCACHE
 	  bool "Provide CIFS client caching support"
 	  depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber@redhat.com>


----- Original Message -----
From: "Long Li" <longli@exchange.microsoft.com>
To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox" <mawilcox@microsoft.com>, "Stephen Hemminger" <sthemmin@microsoft.com>
Cc: "Long Li" <longli@microsoft.com>
Sent: Tuesday, 7 November, 2017 7:54:54 PM
Subject: [Patch v7 02/22] CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT

From: Long Li <longli@microsoft.com>

Build SMB Direct code when this option is set.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/Kconfig | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig
index f724361..8d05fff 100644
--- a/fs/cifs/Kconfig
+++ b/fs/cifs/Kconfig
@@ -191,6 +191,14 @@ config CIFS_SMB311
 	  This dialect includes improved security negotiation features.
 	  If unsure, say N
 
+config CIFS_SMB_DIRECT
+	bool "SMB Direct support (Experimental)"
+	depends on CIFS && INFINIBAND
+	help
+	  Enables SMB Direct experimental support for SMB 3.0, 3.02 and 3.1.1.
+	  SMB Direct allows transferring SMB packets over RDMA. If unsure,
+	  say N.
+
 config CIFS_FSCACHE
 	  bool "Provide CIFS client caching support"
 	  depends on CIFS=m && FSCACHE || CIFS=y && FSCACHE=y
-- 
2.7.4

[Patch v7 03/22] CIFS: SMBD: Add rdma mount option

[Long Li]

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

Add "rdma" to CIFS mount options to connect to SMB Direct.
Add checks to validate this is used on SMB 3.X dialects.

To connect to SMBDirect, use "mount.cifs -o rdma,vers=3.x".
At the time of this patch, 3.x can be 3.0, 3.02 or 3.1.1.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/cifs_debug.c |  2 ++
 fs/cifs/cifsfs.c     |  2 ++
 fs/cifs/cifsglob.h   |  5 +++++
 fs/cifs/connect.c    | 15 ++++++++++++++-
 4 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index 9727e1d..ba0870d 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -171,6 +171,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 				ses->ses_count, ses->serverOS, ses->serverNOS,
 				ses->capabilities, ses->status);
 			}
+			if (server->rdma)
+				seq_printf(m, "RDMA\n\t");
 			seq_printf(m, "TCP status: %d\n\tLocal Users To "
 				   "Server: %d SecMode: 0x%x Req On Wire: %d",
 				   server->tcpStatus, server->srv_count,
diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
index 180b335..e15fbf1 100644
--- a/fs/cifs/cifsfs.c
+++ b/fs/cifs/cifsfs.c
@@ -327,6 +327,8 @@ cifs_show_address(struct seq_file *s, struct TCP_Server_Info *server)
 	default:
 		seq_puts(s, "(unknown)");
 	}
+	if (server->rdma)
+		seq_puts(s, ",rdma");
 }
 
 static void
diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
index 808486c..5585516 100644
--- a/fs/cifs/cifsglob.h
+++ b/fs/cifs/cifsglob.h
@@ -530,6 +530,7 @@ struct smb_vol {
 	bool nopersistent:1;
 	bool resilient:1; /* noresilient not required since not fored for CA */
 	bool domainauto:1;
+	bool rdma:1;
 	unsigned int rsize;
 	unsigned int wsize;
 	bool sockopt_tcp_nodelay:1;
@@ -646,6 +647,10 @@ struct TCP_Server_Info {
 	bool	sec_kerberos;		/* supports plain Kerberos */
 	bool	sec_mskerberos;		/* supports legacy MS Kerberos */
 	bool	large_buf;		/* is current buffer large? */
+	/* use SMBD connection instead of socket */
+	bool	rdma;
+	/* point to the SMBD connection if RDMA is used instead of socket */
+	struct smbd_connection *smbd_conn;
 	struct delayed_work	echo; /* echo ping workqueue job */
 	char	*smallbuf;	/* pointer to current "small" buffer */
 	char	*bigbuf;	/* pointer to current "big" buffer */
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 59647eb..b5a575f 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -92,7 +92,7 @@ enum {
 	Opt_multiuser, Opt_sloppy, Opt_nosharesock,
 	Opt_persistent, Opt_nopersistent,
 	Opt_resilient, Opt_noresilient,
-	Opt_domainauto,
+	Opt_domainauto, Opt_rdma,
 
 	/* Mount options which take numeric value */
 	Opt_backupuid, Opt_backupgid, Opt_uid,
@@ -183,6 +183,7 @@ static const match_table_t cifs_mount_option_tokens = {
 	{ Opt_resilient, "resilienthandles"},
 	{ Opt_noresilient, "noresilienthandles"},
 	{ Opt_domainauto, "domainauto"},
+	{ Opt_rdma, "rdma"},
 
 	{ Opt_backupuid, "backupuid=%s" },
 	{ Opt_backupgid, "backupgid=%s" },
@@ -1538,6 +1539,9 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
 		case Opt_domainauto:
 			vol->domainauto = true;
 			break;
+		case Opt_rdma:
+			vol->rdma = true;
+			break;
 
 		/* Numeric Values */
 		case Opt_backupuid:
@@ -1928,6 +1932,11 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
 		goto cifs_parse_mount_err;
 	}
 
+	if (vol->rdma && vol->vals->protocol_id < SMB30_PROT_ID) {
+		cifs_dbg(VFS, "SMB Direct requires Version >=3.0\n");
+		goto cifs_parse_mount_err;
+	}
+
 #ifndef CONFIG_KEYS
 	/* Muliuser mounts require CONFIG_KEYS support */
 	if (vol->multiuser) {
@@ -2131,6 +2140,9 @@ static int match_server(struct TCP_Server_Info *server, struct smb_vol *vol)
 	if (server->echo_interval != vol->echo_interval * HZ)
 		return 0;
 
+	if (server->rdma != vol->rdma)
+		return 0;
+
 	return 1;
 }
 
@@ -2229,6 +2241,7 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
 	tcp_ses->noblocksnd = volume_info->noblocksnd;
 	tcp_ses->noautotune = volume_info->noautotune;
 	tcp_ses->tcp_nodelay = volume_info->sockopt_tcp_nodelay;
+	tcp_ses->rdma = volume_info->rdma;
 	tcp_ses->in_flight = 0;
 	tcp_ses->credits = 1;
 	init_waitqueue_head(&tcp_ses->response_q);
-- 
2.7.4

[Patch v7 03/22] CIFS: SMBD: Add rdma mount option

[Long Li]

From: Long Li <longli@microsoft.com>

Add "rdma" to CIFS mount options to connect to SMB Direct.
Add checks to validate this is used on SMB 3.X dialects.

To connect to SMBDirect, use "mount.cifs -o rdma,vers=3.x".
At the time of this patch, 3.x can be 3.0, 3.02 or 3.1.1.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/cifs_debug.c |  2 ++
 fs/cifs/cifsfs.c     |  2 ++
 fs/cifs/cifsglob.h   |  5 +++++
 fs/cifs/connect.c    | 15 ++++++++++++++-
 4 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index 9727e1d..ba0870d 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -171,6 +171,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 				ses->ses_count, ses->serverOS, ses->serverNOS,
 				ses->capabilities, ses->status);
 			}
+			if (server->rdma)
+				seq_printf(m, "RDMA\n\t");
 			seq_printf(m, "TCP status: %d\n\tLocal Users To "
 				   "Server: %d SecMode: 0x%x Req On Wire: %d",
 				   server->tcpStatus, server->srv_count,
diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
index 180b335..e15fbf1 100644
--- a/fs/cifs/cifsfs.c
+++ b/fs/cifs/cifsfs.c
@@ -327,6 +327,8 @@ cifs_show_address(struct seq_file *s, struct TCP_Server_Info *server)
 	default:
 		seq_puts(s, "(unknown)");
 	}
+	if (server->rdma)
+		seq_puts(s, ",rdma");
 }
 
 static void
diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
index 808486c..5585516 100644
--- a/fs/cifs/cifsglob.h
+++ b/fs/cifs/cifsglob.h
@@ -530,6 +530,7 @@ struct smb_vol {
 	bool nopersistent:1;
 	bool resilient:1; /* noresilient not required since not fored for CA */
 	bool domainauto:1;
+	bool rdma:1;
 	unsigned int rsize;
 	unsigned int wsize;
 	bool sockopt_tcp_nodelay:1;
@@ -646,6 +647,10 @@ struct TCP_Server_Info {
 	bool	sec_kerberos;		/* supports plain Kerberos */
 	bool	sec_mskerberos;		/* supports legacy MS Kerberos */
 	bool	large_buf;		/* is current buffer large? */
+	/* use SMBD connection instead of socket */
+	bool	rdma;
+	/* point to the SMBD connection if RDMA is used instead of socket */
+	struct smbd_connection *smbd_conn;
 	struct delayed_work	echo; /* echo ping workqueue job */
 	char	*smallbuf;	/* pointer to current "small" buffer */
 	char	*bigbuf;	/* pointer to current "big" buffer */
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 59647eb..b5a575f 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -92,7 +92,7 @@ enum {
 	Opt_multiuser, Opt_sloppy, Opt_nosharesock,
 	Opt_persistent, Opt_nopersistent,
 	Opt_resilient, Opt_noresilient,
-	Opt_domainauto,
+	Opt_domainauto, Opt_rdma,
 
 	/* Mount options which take numeric value */
 	Opt_backupuid, Opt_backupgid, Opt_uid,
@@ -183,6 +183,7 @@ static const match_table_t cifs_mount_option_tokens = {
 	{ Opt_resilient, "resilienthandles"},
 	{ Opt_noresilient, "noresilienthandles"},
 	{ Opt_domainauto, "domainauto"},
+	{ Opt_rdma, "rdma"},
 
 	{ Opt_backupuid, "backupuid=%s" },
 	{ Opt_backupgid, "backupgid=%s" },
@@ -1538,6 +1539,9 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
 		case Opt_domainauto:
 			vol->domainauto = true;
 			break;
+		case Opt_rdma:
+			vol->rdma = true;
+			break;
 
 		/* Numeric Values */
 		case Opt_backupuid:
@@ -1928,6 +1932,11 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
 		goto cifs_parse_mount_err;
 	}
 
+	if (vol->rdma && vol->vals->protocol_id < SMB30_PROT_ID) {
+		cifs_dbg(VFS, "SMB Direct requires Version >=3.0\n");
+		goto cifs_parse_mount_err;
+	}
+
 #ifndef CONFIG_KEYS
 	/* Muliuser mounts require CONFIG_KEYS support */
 	if (vol->multiuser) {
@@ -2131,6 +2140,9 @@ static int match_server(struct TCP_Server_Info *server, struct smb_vol *vol)
 	if (server->echo_interval != vol->echo_interval * HZ)
 		return 0;
 
+	if (server->rdma != vol->rdma)
+		return 0;
+
 	return 1;
 }
 
@@ -2229,6 +2241,7 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
 	tcp_ses->noblocksnd = volume_info->noblocksnd;
 	tcp_ses->noautotune = volume_info->noautotune;
 	tcp_ses->tcp_nodelay = volume_info->sockopt_tcp_nodelay;
+	tcp_ses->rdma = volume_info->rdma;
 	tcp_ses->in_flight = 0;
 	tcp_ses->credits = 1;
 	init_waitqueue_head(&tcp_ses->response_q);
-- 
2.7.4

RE: [Patch v7 03/22] CIFS: SMBD: Add rdma mount option

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>:
> From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
>
> Add "rdma" to CIFS mount options to connect to SMB Direct.
> Add checks to validate this is used on SMB 3.X dialects.
>
> To connect to SMBDirect, use "mount.cifs -o rdma,vers=3.x".
> At the time of this patch, 3.x can be 3.0, 3.02 or 3.1.1.
>
> Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
> ---
>  fs/cifs/cifs_debug.c |  2 ++
>  fs/cifs/cifsfs.c     |  2 ++
>  fs/cifs/cifsglob.h   |  5 +++++
>  fs/cifs/connect.c    | 15 ++++++++++++++-
>  4 files changed, 23 insertions(+), 1 deletion(-)
>
> diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
> index 9727e1d..ba0870d 100644
> --- a/fs/cifs/cifs_debug.c
> +++ b/fs/cifs/cifs_debug.c
> @@ -171,6 +171,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>                                 ses->ses_count, ses->serverOS, ses->serverNOS,
>                                 ses->capabilities, ses->status);
>                         }
> +                       if (server->rdma)
> +                               seq_printf(m, "RDMA\n\t");
>                         seq_printf(m, "TCP status: %d\n\tLocal Users To "
>                                    "Server: %d SecMode: 0x%x Req On Wire: %d",
>                                    server->tcpStatus, server->srv_count,
> diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
> index 180b335..e15fbf1 100644
> --- a/fs/cifs/cifsfs.c
> +++ b/fs/cifs/cifsfs.c
> @@ -327,6 +327,8 @@ cifs_show_address(struct seq_file *s, struct TCP_Server_Info *server)
>         default:
>                 seq_puts(s, "(unknown)");
>         }
> +       if (server->rdma)
> +               seq_puts(s, ",rdma");
>  }
>
>  static void
> diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
> index 808486c..5585516 100644
> --- a/fs/cifs/cifsglob.h
> +++ b/fs/cifs/cifsglob.h
> @@ -530,6 +530,7 @@ struct smb_vol {
>         bool nopersistent:1;
>         bool resilient:1; /* noresilient not required since not fored for CA */
>         bool domainauto:1;
> +       bool rdma:1;
>         unsigned int rsize;
>         unsigned int wsize;
>         bool sockopt_tcp_nodelay:1;
> @@ -646,6 +647,10 @@ struct TCP_Server_Info {
>         bool    sec_kerberos;           /* supports plain Kerberos */
>         bool    sec_mskerberos;         /* supports legacy MS Kerberos */
>         bool    large_buf;              /* is current buffer large? */
> +       /* use SMBD connection instead of socket */
> +       bool    rdma;
> +       /* point to the SMBD connection if RDMA is used instead of socket */
> +       struct smbd_connection *smbd_conn;
>         struct delayed_work     echo; /* echo ping workqueue job */
>         char    *smallbuf;      /* pointer to current "small" buffer */
>         char    *bigbuf;        /* pointer to current "big" buffer */
> diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
> index 59647eb..b5a575f 100644
> --- a/fs/cifs/connect.c
> +++ b/fs/cifs/connect.c
> @@ -92,7 +92,7 @@ enum {
>         Opt_multiuser, Opt_sloppy, Opt_nosharesock,
>         Opt_persistent, Opt_nopersistent,
>         Opt_resilient, Opt_noresilient,
> -       Opt_domainauto,
> +       Opt_domainauto, Opt_rdma,
>
>         /* Mount options which take numeric value */
>         Opt_backupuid, Opt_backupgid, Opt_uid,
> @@ -183,6 +183,7 @@ static const match_table_t cifs_mount_option_tokens = {
>         { Opt_resilient, "resilienthandles"},
>         { Opt_noresilient, "noresilienthandles"},
>         { Opt_domainauto, "domainauto"},
> +       { Opt_rdma, "rdma"},
>
>         { Opt_backupuid, "backupuid=%s" },
>         { Opt_backupgid, "backupgid=%s" },
> @@ -1538,6 +1539,9 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
>                 case Opt_domainauto:
>                         vol->domainauto = true;
>                         break;
> +               case Opt_rdma:
> +                       vol->rdma = true;
> +                       break;
>
>                 /* Numeric Values */
>                 case Opt_backupuid:
> @@ -1928,6 +1932,11 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
>                 goto cifs_parse_mount_err;
>         }
>
> +       if (vol->rdma && vol->vals->protocol_id < SMB30_PROT_ID) {
> +               cifs_dbg(VFS, "SMB Direct requires Version >=3.0\n");
> +               goto cifs_parse_mount_err;
> +       }
> +
>  #ifndef CONFIG_KEYS
>         /* Muliuser mounts require CONFIG_KEYS support */
>         if (vol->multiuser) {
> @@ -2131,6 +2140,9 @@ static int match_server(struct TCP_Server_Info *server, struct smb_vol *vol)
>         if (server->echo_interval != vol->echo_interval * HZ)
>                 return 0;
>
> +       if (server->rdma != vol->rdma)
> +               return 0;
> +
>         return 1;
>  }
>
> @@ -2229,6 +2241,7 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
>         tcp_ses->noblocksnd = volume_info->noblocksnd;
>         tcp_ses->noautotune = volume_info->noautotune;
>         tcp_ses->tcp_nodelay = volume_info->sockopt_tcp_nodelay;
> +       tcp_ses->rdma = volume_info->rdma;
>         tcp_ses->in_flight = 0;
>         tcp_ses->credits = 1;
>         init_waitqueue_head(&tcp_ses->response_q);
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reviewed-by: Pavel Shilovsky <pshilov-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

--
Best regards,
Pavel Shilovsky

RE: [Patch v7 03/22] CIFS: SMBD: Add rdma mount option

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli@exchange.microsoft.com>:
> From: Long Li <longli@microsoft.com>
>
> Add "rdma" to CIFS mount options to connect to SMB Direct.
> Add checks to validate this is used on SMB 3.X dialects.
>
> To connect to SMBDirect, use "mount.cifs -o rdma,vers=3.x".
> At the time of this patch, 3.x can be 3.0, 3.02 or 3.1.1.
>
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>  fs/cifs/cifs_debug.c |  2 ++
>  fs/cifs/cifsfs.c     |  2 ++
>  fs/cifs/cifsglob.h   |  5 +++++
>  fs/cifs/connect.c    | 15 ++++++++++++++-
>  4 files changed, 23 insertions(+), 1 deletion(-)
>
> diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
> index 9727e1d..ba0870d 100644
> --- a/fs/cifs/cifs_debug.c
> +++ b/fs/cifs/cifs_debug.c
> @@ -171,6 +171,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>                                 ses->ses_count, ses->serverOS, ses->serverNOS,
>                                 ses->capabilities, ses->status);
>                         }
> +                       if (server->rdma)
> +                               seq_printf(m, "RDMA\n\t");
>                         seq_printf(m, "TCP status: %d\n\tLocal Users To "
>                                    "Server: %d SecMode: 0x%x Req On Wire: %d",
>                                    server->tcpStatus, server->srv_count,
> diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
> index 180b335..e15fbf1 100644
> --- a/fs/cifs/cifsfs.c
> +++ b/fs/cifs/cifsfs.c
> @@ -327,6 +327,8 @@ cifs_show_address(struct seq_file *s, struct TCP_Server_Info *server)
>         default:
>                 seq_puts(s, "(unknown)");
>         }
> +       if (server->rdma)
> +               seq_puts(s, ",rdma");
>  }
>
>  static void
> diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
> index 808486c..5585516 100644
> --- a/fs/cifs/cifsglob.h
> +++ b/fs/cifs/cifsglob.h
> @@ -530,6 +530,7 @@ struct smb_vol {
>         bool nopersistent:1;
>         bool resilient:1; /* noresilient not required since not fored for CA */
>         bool domainauto:1;
> +       bool rdma:1;
>         unsigned int rsize;
>         unsigned int wsize;
>         bool sockopt_tcp_nodelay:1;
> @@ -646,6 +647,10 @@ struct TCP_Server_Info {
>         bool    sec_kerberos;           /* supports plain Kerberos */
>         bool    sec_mskerberos;         /* supports legacy MS Kerberos */
>         bool    large_buf;              /* is current buffer large? */
> +       /* use SMBD connection instead of socket */
> +       bool    rdma;
> +       /* point to the SMBD connection if RDMA is used instead of socket */
> +       struct smbd_connection *smbd_conn;
>         struct delayed_work     echo; /* echo ping workqueue job */
>         char    *smallbuf;      /* pointer to current "small" buffer */
>         char    *bigbuf;        /* pointer to current "big" buffer */
> diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
> index 59647eb..b5a575f 100644
> --- a/fs/cifs/connect.c
> +++ b/fs/cifs/connect.c
> @@ -92,7 +92,7 @@ enum {
>         Opt_multiuser, Opt_sloppy, Opt_nosharesock,
>         Opt_persistent, Opt_nopersistent,
>         Opt_resilient, Opt_noresilient,
> -       Opt_domainauto,
> +       Opt_domainauto, Opt_rdma,
>
>         /* Mount options which take numeric value */
>         Opt_backupuid, Opt_backupgid, Opt_uid,
> @@ -183,6 +183,7 @@ static const match_table_t cifs_mount_option_tokens = {
>         { Opt_resilient, "resilienthandles"},
>         { Opt_noresilient, "noresilienthandles"},
>         { Opt_domainauto, "domainauto"},
> +       { Opt_rdma, "rdma"},
>
>         { Opt_backupuid, "backupuid=%s" },
>         { Opt_backupgid, "backupgid=%s" },
> @@ -1538,6 +1539,9 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
>                 case Opt_domainauto:
>                         vol->domainauto = true;
>                         break;
> +               case Opt_rdma:
> +                       vol->rdma = true;
> +                       break;
>
>                 /* Numeric Values */
>                 case Opt_backupuid:
> @@ -1928,6 +1932,11 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
>                 goto cifs_parse_mount_err;
>         }
>
> +       if (vol->rdma && vol->vals->protocol_id < SMB30_PROT_ID) {
> +               cifs_dbg(VFS, "SMB Direct requires Version >=3.0\n");
> +               goto cifs_parse_mount_err;
> +       }
> +
>  #ifndef CONFIG_KEYS
>         /* Muliuser mounts require CONFIG_KEYS support */
>         if (vol->multiuser) {
> @@ -2131,6 +2140,9 @@ static int match_server(struct TCP_Server_Info *server, struct smb_vol *vol)
>         if (server->echo_interval != vol->echo_interval * HZ)
>                 return 0;
>
> +       if (server->rdma != vol->rdma)
> +               return 0;
> +
>         return 1;
>  }
>
> @@ -2229,6 +2241,7 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
>         tcp_ses->noblocksnd = volume_info->noblocksnd;
>         tcp_ses->noautotune = volume_info->noautotune;
>         tcp_ses->tcp_nodelay = volume_info->sockopt_tcp_nodelay;
> +       tcp_ses->rdma = volume_info->rdma;
>         tcp_ses->in_flight = 0;
>         tcp_ses->credits = 1;
>         init_waitqueue_head(&tcp_ses->response_q);
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reviewed-by: Pavel Shilovsky <pshilov@microsoft.com>

--
Best regards,
Pavel Shilovsky

RE: [Patch v7 03/22] CIFS: SMBD: Add rdma mount option

[Pavel Shilovskiy]

2017-11-07 0:54 GMT-08:00 Long Li <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>:
> From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
>
> Add "rdma" to CIFS mount options to connect to SMB Direct.
> Add checks to validate this is used on SMB 3.X dialects.
>
> To connect to SMBDirect, use "mount.cifs -o rdma,vers=3.x".
> At the time of this patch, 3.x can be 3.0, 3.02 or 3.1.1.
>
> Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
> ---
>  fs/cifs/cifs_debug.c |  2 ++
>  fs/cifs/cifsfs.c     |  2 ++
>  fs/cifs/cifsglob.h   |  5 +++++
>  fs/cifs/connect.c    | 15 ++++++++++++++-
>  4 files changed, 23 insertions(+), 1 deletion(-)
>
> diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
> index 9727e1d..ba0870d 100644
> --- a/fs/cifs/cifs_debug.c
> +++ b/fs/cifs/cifs_debug.c
> @@ -171,6 +171,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
>                                 ses->ses_count, ses->serverOS, ses->serverNOS,
>                                 ses->capabilities, ses->status);
>                         }
> +                       if (server->rdma)
> +                               seq_printf(m, "RDMA\n\t");
>                         seq_printf(m, "TCP status: %d\n\tLocal Users To "
>                                    "Server: %d SecMode: 0x%x Req On Wire: %d",
>                                    server->tcpStatus, server->srv_count,
> diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
> index 180b335..e15fbf1 100644
> --- a/fs/cifs/cifsfs.c
> +++ b/fs/cifs/cifsfs.c
> @@ -327,6 +327,8 @@ cifs_show_address(struct seq_file *s, struct TCP_Server_Info *server)
>         default:
>                 seq_puts(s, "(unknown)");
>         }
> +       if (server->rdma)
> +               seq_puts(s, ",rdma");
>  }
>
>  static void
> diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
> index 808486c..5585516 100644
> --- a/fs/cifs/cifsglob.h
> +++ b/fs/cifs/cifsglob.h
> @@ -530,6 +530,7 @@ struct smb_vol {
>         bool nopersistent:1;
>         bool resilient:1; /* noresilient not required since not fored for CA */
>         bool domainauto:1;
> +       bool rdma:1;
>         unsigned int rsize;
>         unsigned int wsize;
>         bool sockopt_tcp_nodelay:1;
> @@ -646,6 +647,10 @@ struct TCP_Server_Info {
>         bool    sec_kerberos;           /* supports plain Kerberos */
>         bool    sec_mskerberos;         /* supports legacy MS Kerberos */
>         bool    large_buf;              /* is current buffer large? */
> +       /* use SMBD connection instead of socket */
> +       bool    rdma;
> +       /* point to the SMBD connection if RDMA is used instead of socket */
> +       struct smbd_connection *smbd_conn;
>         struct delayed_work     echo; /* echo ping workqueue job */
>         char    *smallbuf;      /* pointer to current "small" buffer */
>         char    *bigbuf;        /* pointer to current "big" buffer */
> diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
> index 59647eb..b5a575f 100644
> --- a/fs/cifs/connect.c
> +++ b/fs/cifs/connect.c
> @@ -92,7 +92,7 @@ enum {
>         Opt_multiuser, Opt_sloppy, Opt_nosharesock,
>         Opt_persistent, Opt_nopersistent,
>         Opt_resilient, Opt_noresilient,
> -       Opt_domainauto,
> +       Opt_domainauto, Opt_rdma,
>
>         /* Mount options which take numeric value */
>         Opt_backupuid, Opt_backupgid, Opt_uid,
> @@ -183,6 +183,7 @@ static const match_table_t cifs_mount_option_tokens = {
>         { Opt_resilient, "resilienthandles"},
>         { Opt_noresilient, "noresilienthandles"},
>         { Opt_domainauto, "domainauto"},
> +       { Opt_rdma, "rdma"},
>
>         { Opt_backupuid, "backupuid=%s" },
>         { Opt_backupgid, "backupgid=%s" },
> @@ -1538,6 +1539,9 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
>                 case Opt_domainauto:
>                         vol->domainauto = true;
>                         break;
> +               case Opt_rdma:
> +                       vol->rdma = true;
> +                       break;
>
>                 /* Numeric Values */
>                 case Opt_backupuid:
> @@ -1928,6 +1932,11 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
>                 goto cifs_parse_mount_err;
>         }
>
> +       if (vol->rdma && vol->vals->protocol_id < SMB30_PROT_ID) {
> +               cifs_dbg(VFS, "SMB Direct requires Version >=3.0\n");
> +               goto cifs_parse_mount_err;
> +       }
> +
>  #ifndef CONFIG_KEYS
>         /* Muliuser mounts require CONFIG_KEYS support */
>         if (vol->multiuser) {
> @@ -2131,6 +2140,9 @@ static int match_server(struct TCP_Server_Info *server, struct smb_vol *vol)
>         if (server->echo_interval != vol->echo_interval * HZ)
>                 return 0;
>
> +       if (server->rdma != vol->rdma)
> +               return 0;
> +
>         return 1;
>  }
>
> @@ -2229,6 +2241,7 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
>         tcp_ses->noblocksnd = volume_info->noblocksnd;
>         tcp_ses->noautotune = volume_info->noautotune;
>         tcp_ses->tcp_nodelay = volume_info->sockopt_tcp_nodelay;
> +       tcp_ses->rdma = volume_info->rdma;
>         tcp_ses->in_flight = 0;
>         tcp_ses->credits = 1;
>         init_waitqueue_head(&tcp_ses->response_q);
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reviewed-by: Pavel Shilovsky <pshilov-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

--
Best regards,
Pavel Shilovsky

Re: [Patch v7 03/22] CIFS: SMBD: Add rdma mount option

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>


----- Original Message -----
From: "Long Li" <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>
To: "Steve French" <sfrench-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>, linux-cifs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, samba-technical-w/Ol4Ecudpl8XjKLYN78aQ@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "Christoph Hellwig" <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, "Tom Talpey" <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Matthew Wilcox" <mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Stephen Hemminger" <sthemmin-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Cc: "Long Li" <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Sent: Tuesday, 7 November, 2017 7:54:55 PM
Subject: [Patch v7 03/22] CIFS: SMBD: Add rdma mount option

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

Add "rdma" to CIFS mount options to connect to SMB Direct.
Add checks to validate this is used on SMB 3.X dialects.

To connect to SMBDirect, use "mount.cifs -o rdma,vers=3.x".
At the time of this patch, 3.x can be 3.0, 3.02 or 3.1.1.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/cifs_debug.c |  2 ++
 fs/cifs/cifsfs.c     |  2 ++
 fs/cifs/cifsglob.h   |  5 +++++
 fs/cifs/connect.c    | 15 ++++++++++++++-
 4 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index 9727e1d..ba0870d 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -171,6 +171,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 				ses->ses_count, ses->serverOS, ses->serverNOS,
 				ses->capabilities, ses->status);
 			}
+			if (server->rdma)
+				seq_printf(m, "RDMA\n\t");
 			seq_printf(m, "TCP status: %d\n\tLocal Users To "
 				   "Server: %d SecMode: 0x%x Req On Wire: %d",
 				   server->tcpStatus, server->srv_count,
diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
index 180b335..e15fbf1 100644
--- a/fs/cifs/cifsfs.c
+++ b/fs/cifs/cifsfs.c
@@ -327,6 +327,8 @@ cifs_show_address(struct seq_file *s, struct TCP_Server_Info *server)
 	default:
 		seq_puts(s, "(unknown)");
 	}
+	if (server->rdma)
+		seq_puts(s, ",rdma");
 }
 
 static void
diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
index 808486c..5585516 100644
--- a/fs/cifs/cifsglob.h
+++ b/fs/cifs/cifsglob.h
@@ -530,6 +530,7 @@ struct smb_vol {
 	bool nopersistent:1;
 	bool resilient:1; /* noresilient not required since not fored for CA */
 	bool domainauto:1;
+	bool rdma:1;
 	unsigned int rsize;
 	unsigned int wsize;
 	bool sockopt_tcp_nodelay:1;
@@ -646,6 +647,10 @@ struct TCP_Server_Info {
 	bool	sec_kerberos;		/* supports plain Kerberos */
 	bool	sec_mskerberos;		/* supports legacy MS Kerberos */
 	bool	large_buf;		/* is current buffer large? */
+	/* use SMBD connection instead of socket */
+	bool	rdma;
+	/* point to the SMBD connection if RDMA is used instead of socket */
+	struct smbd_connection *smbd_conn;
 	struct delayed_work	echo; /* echo ping workqueue job */
 	char	*smallbuf;	/* pointer to current "small" buffer */
 	char	*bigbuf;	/* pointer to current "big" buffer */
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 59647eb..b5a575f 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -92,7 +92,7 @@ enum {
 	Opt_multiuser, Opt_sloppy, Opt_nosharesock,
 	Opt_persistent, Opt_nopersistent,
 	Opt_resilient, Opt_noresilient,
-	Opt_domainauto,
+	Opt_domainauto, Opt_rdma,
 
 	/* Mount options which take numeric value */
 	Opt_backupuid, Opt_backupgid, Opt_uid,
@@ -183,6 +183,7 @@ static const match_table_t cifs_mount_option_tokens = {
 	{ Opt_resilient, "resilienthandles"},
 	{ Opt_noresilient, "noresilienthandles"},
 	{ Opt_domainauto, "domainauto"},
+	{ Opt_rdma, "rdma"},
 
 	{ Opt_backupuid, "backupuid=%s" },
 	{ Opt_backupgid, "backupgid=%s" },
@@ -1538,6 +1539,9 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
 		case Opt_domainauto:
 			vol->domainauto = true;
 			break;
+		case Opt_rdma:
+			vol->rdma = true;
+			break;
 
 		/* Numeric Values */
 		case Opt_backupuid:
@@ -1928,6 +1932,11 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
 		goto cifs_parse_mount_err;
 	}
 
+	if (vol->rdma && vol->vals->protocol_id < SMB30_PROT_ID) {
+		cifs_dbg(VFS, "SMB Direct requires Version >=3.0\n");
+		goto cifs_parse_mount_err;
+	}
+
 #ifndef CONFIG_KEYS
 	/* Muliuser mounts require CONFIG_KEYS support */
 	if (vol->multiuser) {
@@ -2131,6 +2140,9 @@ static int match_server(struct TCP_Server_Info *server, struct smb_vol *vol)
 	if (server->echo_interval != vol->echo_interval * HZ)
 		return 0;
 
+	if (server->rdma != vol->rdma)
+		return 0;
+
 	return 1;
 }
 
@@ -2229,6 +2241,7 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
 	tcp_ses->noblocksnd = volume_info->noblocksnd;
 	tcp_ses->noautotune = volume_info->noautotune;
 	tcp_ses->tcp_nodelay = volume_info->sockopt_tcp_nodelay;
+	tcp_ses->rdma = volume_info->rdma;
 	tcp_ses->in_flight = 0;
 	tcp_ses->credits = 1;
 	init_waitqueue_head(&tcp_ses->response_q);
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 03/22] CIFS: SMBD: Add rdma mount option

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber@redhat.com>


----- Original Message -----
From: "Long Li" <longli@exchange.microsoft.com>
To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox" <mawilcox@microsoft.com>, "Stephen Hemminger" <sthemmin@microsoft.com>
Cc: "Long Li" <longli@microsoft.com>
Sent: Tuesday, 7 November, 2017 7:54:55 PM
Subject: [Patch v7 03/22] CIFS: SMBD: Add rdma mount option

From: Long Li <longli@microsoft.com>

Add "rdma" to CIFS mount options to connect to SMB Direct.
Add checks to validate this is used on SMB 3.X dialects.

To connect to SMBDirect, use "mount.cifs -o rdma,vers=3.x".
At the time of this patch, 3.x can be 3.0, 3.02 or 3.1.1.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/cifs_debug.c |  2 ++
 fs/cifs/cifsfs.c     |  2 ++
 fs/cifs/cifsglob.h   |  5 +++++
 fs/cifs/connect.c    | 15 ++++++++++++++-
 4 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index 9727e1d..ba0870d 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -171,6 +171,8 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 				ses->ses_count, ses->serverOS, ses->serverNOS,
 				ses->capabilities, ses->status);
 			}
+			if (server->rdma)
+				seq_printf(m, "RDMA\n\t");
 			seq_printf(m, "TCP status: %d\n\tLocal Users To "
 				   "Server: %d SecMode: 0x%x Req On Wire: %d",
 				   server->tcpStatus, server->srv_count,
diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c
index 180b335..e15fbf1 100644
--- a/fs/cifs/cifsfs.c
+++ b/fs/cifs/cifsfs.c
@@ -327,6 +327,8 @@ cifs_show_address(struct seq_file *s, struct TCP_Server_Info *server)
 	default:
 		seq_puts(s, "(unknown)");
 	}
+	if (server->rdma)
+		seq_puts(s, ",rdma");
 }
 
 static void
diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
index 808486c..5585516 100644
--- a/fs/cifs/cifsglob.h
+++ b/fs/cifs/cifsglob.h
@@ -530,6 +530,7 @@ struct smb_vol {
 	bool nopersistent:1;
 	bool resilient:1; /* noresilient not required since not fored for CA */
 	bool domainauto:1;
+	bool rdma:1;
 	unsigned int rsize;
 	unsigned int wsize;
 	bool sockopt_tcp_nodelay:1;
@@ -646,6 +647,10 @@ struct TCP_Server_Info {
 	bool	sec_kerberos;		/* supports plain Kerberos */
 	bool	sec_mskerberos;		/* supports legacy MS Kerberos */
 	bool	large_buf;		/* is current buffer large? */
+	/* use SMBD connection instead of socket */
+	bool	rdma;
+	/* point to the SMBD connection if RDMA is used instead of socket */
+	struct smbd_connection *smbd_conn;
 	struct delayed_work	echo; /* echo ping workqueue job */
 	char	*smallbuf;	/* pointer to current "small" buffer */
 	char	*bigbuf;	/* pointer to current "big" buffer */
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 59647eb..b5a575f 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -92,7 +92,7 @@ enum {
 	Opt_multiuser, Opt_sloppy, Opt_nosharesock,
 	Opt_persistent, Opt_nopersistent,
 	Opt_resilient, Opt_noresilient,
-	Opt_domainauto,
+	Opt_domainauto, Opt_rdma,
 
 	/* Mount options which take numeric value */
 	Opt_backupuid, Opt_backupgid, Opt_uid,
@@ -183,6 +183,7 @@ static const match_table_t cifs_mount_option_tokens = {
 	{ Opt_resilient, "resilienthandles"},
 	{ Opt_noresilient, "noresilienthandles"},
 	{ Opt_domainauto, "domainauto"},
+	{ Opt_rdma, "rdma"},
 
 	{ Opt_backupuid, "backupuid=%s" },
 	{ Opt_backupgid, "backupgid=%s" },
@@ -1538,6 +1539,9 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
 		case Opt_domainauto:
 			vol->domainauto = true;
 			break;
+		case Opt_rdma:
+			vol->rdma = true;
+			break;
 
 		/* Numeric Values */
 		case Opt_backupuid:
@@ -1928,6 +1932,11 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
 		goto cifs_parse_mount_err;
 	}
 
+	if (vol->rdma && vol->vals->protocol_id < SMB30_PROT_ID) {
+		cifs_dbg(VFS, "SMB Direct requires Version >=3.0\n");
+		goto cifs_parse_mount_err;
+	}
+
 #ifndef CONFIG_KEYS
 	/* Muliuser mounts require CONFIG_KEYS support */
 	if (vol->multiuser) {
@@ -2131,6 +2140,9 @@ static int match_server(struct TCP_Server_Info *server, struct smb_vol *vol)
 	if (server->echo_interval != vol->echo_interval * HZ)
 		return 0;
 
+	if (server->rdma != vol->rdma)
+		return 0;
+
 	return 1;
 }
 
@@ -2229,6 +2241,7 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
 	tcp_ses->noblocksnd = volume_info->noblocksnd;
 	tcp_ses->noautotune = volume_info->noautotune;
 	tcp_ses->tcp_nodelay = volume_info->sockopt_tcp_nodelay;
+	tcp_ses->rdma = volume_info->rdma;
 	tcp_ses->in_flight = 0;
 	tcp_ses->credits = 1;
 	init_waitqueue_head(&tcp_ses->response_q);
-- 
2.7.4

[Patch v7 06/22] CIFS: SMBD: export protocol initial values

[Long Li]

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

For use-configurable SMB Direct protocol values, export them to /proc/fs/cifs.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/cifs_debug.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index ba0870d..7025d8d 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -30,6 +30,9 @@
 #include "cifsproto.h"
 #include "cifs_debug.h"
 #include "cifsfs.h"
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#include "smbdirect.h"
+#endif
 
 void
 cifs_dump_mem(char *label, void *data, int length)
@@ -371,6 +374,54 @@ static const struct file_operations cifs_stats_proc_fops = {
 };
 #endif /* STATS */
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#define PROC_FILE_DEFINE(name) \
+static ssize_t name##_write(struct file *file, const char __user *buffer, \
+	size_t count, loff_t *ppos) \
+{ \
+	int rc; \
+	rc = kstrtoint_from_user(buffer, count, 10, & name ); \
+	if (rc) \
+		return rc; \
+	return count; \
+} \
+static int name##_proc_show(struct seq_file *m, void *v) \
+{ \
+	seq_printf(m, "%d\n", name ); \
+	return 0; \
+} \
+static int name##_open(struct inode *inode, struct file *file) \
+{ \
+	return single_open(file, name##_proc_show, NULL); \
+} \
+\
+static const struct file_operations cifs_##name##_proc_fops = { \
+	.open		= name##_open, \
+	.read		= seq_read, \
+	.llseek		= seq_lseek, \
+	.release	= single_release, \
+	.write		= name##_write, \
+}
+
+extern int rdma_readwrite_threshold;
+extern int smbd_max_frmr_depth;
+extern int smbd_keep_alive_interval;
+extern int smbd_max_receive_size;
+extern int smbd_max_fragmented_recv_size;
+extern int smbd_max_send_size;
+extern int smbd_send_credit_target;
+extern int smbd_receive_credit_max;
+
+PROC_FILE_DEFINE(rdma_readwrite_threshold);
+PROC_FILE_DEFINE(smbd_max_frmr_depth);
+PROC_FILE_DEFINE(smbd_keep_alive_interval);
+PROC_FILE_DEFINE(smbd_max_receive_size);
+PROC_FILE_DEFINE(smbd_max_fragmented_recv_size);
+PROC_FILE_DEFINE(smbd_max_send_size);
+PROC_FILE_DEFINE(smbd_send_credit_target);
+PROC_FILE_DEFINE(smbd_receive_credit_max);
+#endif
+
 static struct proc_dir_entry *proc_fs_cifs;
 static const struct file_operations cifsFYI_proc_fops;
 static const struct file_operations cifs_lookup_cache_proc_fops;
@@ -398,6 +449,24 @@ cifs_proc_init(void)
 		    &cifs_security_flags_proc_fops);
 	proc_create("LookupCacheEnabled", 0, proc_fs_cifs,
 		    &cifs_lookup_cache_proc_fops);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	proc_create("rdma_readwrite_threshold", 0, proc_fs_cifs,
+		&cifs_rdma_readwrite_threshold_proc_fops);
+	proc_create("smbd_max_frmr_depth", 0, proc_fs_cifs,
+		&cifs_smbd_max_frmr_depth_proc_fops);
+	proc_create("smbd_keep_alive_interval", 0, proc_fs_cifs,
+		&cifs_smbd_keep_alive_interval_proc_fops);
+	proc_create("smbd_max_receive_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_receive_size_proc_fops);
+	proc_create("smbd_max_fragmented_recv_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_fragmented_recv_size_proc_fops);
+	proc_create("smbd_max_send_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_send_size_proc_fops);
+	proc_create("smbd_send_credit_target", 0, proc_fs_cifs,
+		&cifs_smbd_send_credit_target_proc_fops);
+	proc_create("smbd_receive_credit_max", 0, proc_fs_cifs,
+		&cifs_smbd_receive_credit_max_proc_fops);
+#endif
 }
 
 void
@@ -415,6 +484,16 @@ cifs_proc_clean(void)
 	remove_proc_entry("SecurityFlags", proc_fs_cifs);
 	remove_proc_entry("LinuxExtensionsEnabled", proc_fs_cifs);
 	remove_proc_entry("LookupCacheEnabled", proc_fs_cifs);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	remove_proc_entry("rdma_readwrite_threshold", proc_fs_cifs);
+	remove_proc_entry("smbd_max_frmr_depth", proc_fs_cifs);
+	remove_proc_entry("smbd_keep_alive_interval", proc_fs_cifs);
+	remove_proc_entry("smbd_max_receive_size", proc_fs_cifs);
+	remove_proc_entry("smbd_max_fragmented_recv_size", proc_fs_cifs);
+	remove_proc_entry("smbd_max_send_size", proc_fs_cifs);
+	remove_proc_entry("smbd_send_credit_target", proc_fs_cifs);
+	remove_proc_entry("smbd_receive_credit_max", proc_fs_cifs);
+#endif
 	remove_proc_entry("fs/cifs", NULL);
 }
 
-- 
2.7.4

[Patch v7 06/22] CIFS: SMBD: export protocol initial values

[Long Li]

From: Long Li <longli@microsoft.com>

For use-configurable SMB Direct protocol values, export them to /proc/fs/cifs.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/cifs_debug.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index ba0870d..7025d8d 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -30,6 +30,9 @@
 #include "cifsproto.h"
 #include "cifs_debug.h"
 #include "cifsfs.h"
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#include "smbdirect.h"
+#endif
 
 void
 cifs_dump_mem(char *label, void *data, int length)
@@ -371,6 +374,54 @@ static const struct file_operations cifs_stats_proc_fops = {
 };
 #endif /* STATS */
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#define PROC_FILE_DEFINE(name) \
+static ssize_t name##_write(struct file *file, const char __user *buffer, \
+	size_t count, loff_t *ppos) \
+{ \
+	int rc; \
+	rc = kstrtoint_from_user(buffer, count, 10, & name ); \
+	if (rc) \
+		return rc; \
+	return count; \
+} \
+static int name##_proc_show(struct seq_file *m, void *v) \
+{ \
+	seq_printf(m, "%d\n", name ); \
+	return 0; \
+} \
+static int name##_open(struct inode *inode, struct file *file) \
+{ \
+	return single_open(file, name##_proc_show, NULL); \
+} \
+\
+static const struct file_operations cifs_##name##_proc_fops = { \
+	.open		= name##_open, \
+	.read		= seq_read, \
+	.llseek		= seq_lseek, \
+	.release	= single_release, \
+	.write		= name##_write, \
+}
+
+extern int rdma_readwrite_threshold;
+extern int smbd_max_frmr_depth;
+extern int smbd_keep_alive_interval;
+extern int smbd_max_receive_size;
+extern int smbd_max_fragmented_recv_size;
+extern int smbd_max_send_size;
+extern int smbd_send_credit_target;
+extern int smbd_receive_credit_max;
+
+PROC_FILE_DEFINE(rdma_readwrite_threshold);
+PROC_FILE_DEFINE(smbd_max_frmr_depth);
+PROC_FILE_DEFINE(smbd_keep_alive_interval);
+PROC_FILE_DEFINE(smbd_max_receive_size);
+PROC_FILE_DEFINE(smbd_max_fragmented_recv_size);
+PROC_FILE_DEFINE(smbd_max_send_size);
+PROC_FILE_DEFINE(smbd_send_credit_target);
+PROC_FILE_DEFINE(smbd_receive_credit_max);
+#endif
+
 static struct proc_dir_entry *proc_fs_cifs;
 static const struct file_operations cifsFYI_proc_fops;
 static const struct file_operations cifs_lookup_cache_proc_fops;
@@ -398,6 +449,24 @@ cifs_proc_init(void)
 		    &cifs_security_flags_proc_fops);
 	proc_create("LookupCacheEnabled", 0, proc_fs_cifs,
 		    &cifs_lookup_cache_proc_fops);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	proc_create("rdma_readwrite_threshold", 0, proc_fs_cifs,
+		&cifs_rdma_readwrite_threshold_proc_fops);
+	proc_create("smbd_max_frmr_depth", 0, proc_fs_cifs,
+		&cifs_smbd_max_frmr_depth_proc_fops);
+	proc_create("smbd_keep_alive_interval", 0, proc_fs_cifs,
+		&cifs_smbd_keep_alive_interval_proc_fops);
+	proc_create("smbd_max_receive_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_receive_size_proc_fops);
+	proc_create("smbd_max_fragmented_recv_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_fragmented_recv_size_proc_fops);
+	proc_create("smbd_max_send_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_send_size_proc_fops);
+	proc_create("smbd_send_credit_target", 0, proc_fs_cifs,
+		&cifs_smbd_send_credit_target_proc_fops);
+	proc_create("smbd_receive_credit_max", 0, proc_fs_cifs,
+		&cifs_smbd_receive_credit_max_proc_fops);
+#endif
 }
 
 void
@@ -415,6 +484,16 @@ cifs_proc_clean(void)
 	remove_proc_entry("SecurityFlags", proc_fs_cifs);
 	remove_proc_entry("LinuxExtensionsEnabled", proc_fs_cifs);
 	remove_proc_entry("LookupCacheEnabled", proc_fs_cifs);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	remove_proc_entry("rdma_readwrite_threshold", proc_fs_cifs);
+	remove_proc_entry("smbd_max_frmr_depth", proc_fs_cifs);
+	remove_proc_entry("smbd_keep_alive_interval", proc_fs_cifs);
+	remove_proc_entry("smbd_max_receive_size", proc_fs_cifs);
+	remove_proc_entry("smbd_max_fragmented_recv_size", proc_fs_cifs);
+	remove_proc_entry("smbd_max_send_size", proc_fs_cifs);
+	remove_proc_entry("smbd_send_credit_target", proc_fs_cifs);
+	remove_proc_entry("smbd_receive_credit_max", proc_fs_cifs);
+#endif
 	remove_proc_entry("fs/cifs", NULL);
 }
 
-- 
2.7.4

Re: [Patch v7 06/22] CIFS: SMBD: export protocol initial values

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>


----- Original Message -----
From: "Long Li" <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org>
To: "Steve French" <sfrench-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>, linux-cifs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, samba-technical-w/Ol4Ecudpl8XjKLYN78aQ@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "Christoph Hellwig" <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, "Tom Talpey" <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Matthew Wilcox" <mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>, "Stephen Hemminger" <sthemmin-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Cc: "Long Li" <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Sent: Tuesday, 7 November, 2017 7:54:58 PM
Subject: [Patch v7 06/22] CIFS: SMBD: export protocol initial values

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

For use-configurable SMB Direct protocol values, export them to /proc/fs/cifs.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/cifs_debug.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index ba0870d..7025d8d 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -30,6 +30,9 @@
 #include "cifsproto.h"
 #include "cifs_debug.h"
 #include "cifsfs.h"
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#include "smbdirect.h"
+#endif
 
 void
 cifs_dump_mem(char *label, void *data, int length)
@@ -371,6 +374,54 @@ static const struct file_operations cifs_stats_proc_fops = {
 };
 #endif /* STATS */
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#define PROC_FILE_DEFINE(name) \
+static ssize_t name##_write(struct file *file, const char __user *buffer, \
+	size_t count, loff_t *ppos) \
+{ \
+	int rc; \
+	rc = kstrtoint_from_user(buffer, count, 10, & name ); \
+	if (rc) \
+		return rc; \
+	return count; \
+} \
+static int name##_proc_show(struct seq_file *m, void *v) \
+{ \
+	seq_printf(m, "%d\n", name ); \
+	return 0; \
+} \
+static int name##_open(struct inode *inode, struct file *file) \
+{ \
+	return single_open(file, name##_proc_show, NULL); \
+} \
+\
+static const struct file_operations cifs_##name##_proc_fops = { \
+	.open		= name##_open, \
+	.read		= seq_read, \
+	.llseek		= seq_lseek, \
+	.release	= single_release, \
+	.write		= name##_write, \
+}
+
+extern int rdma_readwrite_threshold;
+extern int smbd_max_frmr_depth;
+extern int smbd_keep_alive_interval;
+extern int smbd_max_receive_size;
+extern int smbd_max_fragmented_recv_size;
+extern int smbd_max_send_size;
+extern int smbd_send_credit_target;
+extern int smbd_receive_credit_max;
+
+PROC_FILE_DEFINE(rdma_readwrite_threshold);
+PROC_FILE_DEFINE(smbd_max_frmr_depth);
+PROC_FILE_DEFINE(smbd_keep_alive_interval);
+PROC_FILE_DEFINE(smbd_max_receive_size);
+PROC_FILE_DEFINE(smbd_max_fragmented_recv_size);
+PROC_FILE_DEFINE(smbd_max_send_size);
+PROC_FILE_DEFINE(smbd_send_credit_target);
+PROC_FILE_DEFINE(smbd_receive_credit_max);
+#endif
+
 static struct proc_dir_entry *proc_fs_cifs;
 static const struct file_operations cifsFYI_proc_fops;
 static const struct file_operations cifs_lookup_cache_proc_fops;
@@ -398,6 +449,24 @@ cifs_proc_init(void)
 		    &cifs_security_flags_proc_fops);
 	proc_create("LookupCacheEnabled", 0, proc_fs_cifs,
 		    &cifs_lookup_cache_proc_fops);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	proc_create("rdma_readwrite_threshold", 0, proc_fs_cifs,
+		&cifs_rdma_readwrite_threshold_proc_fops);
+	proc_create("smbd_max_frmr_depth", 0, proc_fs_cifs,
+		&cifs_smbd_max_frmr_depth_proc_fops);
+	proc_create("smbd_keep_alive_interval", 0, proc_fs_cifs,
+		&cifs_smbd_keep_alive_interval_proc_fops);
+	proc_create("smbd_max_receive_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_receive_size_proc_fops);
+	proc_create("smbd_max_fragmented_recv_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_fragmented_recv_size_proc_fops);
+	proc_create("smbd_max_send_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_send_size_proc_fops);
+	proc_create("smbd_send_credit_target", 0, proc_fs_cifs,
+		&cifs_smbd_send_credit_target_proc_fops);
+	proc_create("smbd_receive_credit_max", 0, proc_fs_cifs,
+		&cifs_smbd_receive_credit_max_proc_fops);
+#endif
 }
 
 void
@@ -415,6 +484,16 @@ cifs_proc_clean(void)
 	remove_proc_entry("SecurityFlags", proc_fs_cifs);
 	remove_proc_entry("LinuxExtensionsEnabled", proc_fs_cifs);
 	remove_proc_entry("LookupCacheEnabled", proc_fs_cifs);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	remove_proc_entry("rdma_readwrite_threshold", proc_fs_cifs);
+	remove_proc_entry("smbd_max_frmr_depth", proc_fs_cifs);
+	remove_proc_entry("smbd_keep_alive_interval", proc_fs_cifs);
+	remove_proc_entry("smbd_max_receive_size", proc_fs_cifs);
+	remove_proc_entry("smbd_max_fragmented_recv_size", proc_fs_cifs);
+	remove_proc_entry("smbd_max_send_size", proc_fs_cifs);
+	remove_proc_entry("smbd_send_credit_target", proc_fs_cifs);
+	remove_proc_entry("smbd_receive_credit_max", proc_fs_cifs);
+#endif
 	remove_proc_entry("fs/cifs", NULL);
 }
 
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 06/22] CIFS: SMBD: export protocol initial values

[Leif Sahlberg]

Acked-by: Ronnie Sahlberg <lsahlber@redhat.com>


----- Original Message -----
From: "Long Li" <longli@exchange.microsoft.com>
To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox" <mawilcox@microsoft.com>, "Stephen Hemminger" <sthemmin@microsoft.com>
Cc: "Long Li" <longli@microsoft.com>
Sent: Tuesday, 7 November, 2017 7:54:58 PM
Subject: [Patch v7 06/22] CIFS: SMBD: export protocol initial values

From: Long Li <longli@microsoft.com>

For use-configurable SMB Direct protocol values, export them to /proc/fs/cifs.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/cifs_debug.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index ba0870d..7025d8d 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -30,6 +30,9 @@
 #include "cifsproto.h"
 #include "cifs_debug.h"
 #include "cifsfs.h"
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#include "smbdirect.h"
+#endif
 
 void
 cifs_dump_mem(char *label, void *data, int length)
@@ -371,6 +374,54 @@ static const struct file_operations cifs_stats_proc_fops = {
 };
 #endif /* STATS */
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+#define PROC_FILE_DEFINE(name) \
+static ssize_t name##_write(struct file *file, const char __user *buffer, \
+	size_t count, loff_t *ppos) \
+{ \
+	int rc; \
+	rc = kstrtoint_from_user(buffer, count, 10, & name ); \
+	if (rc) \
+		return rc; \
+	return count; \
+} \
+static int name##_proc_show(struct seq_file *m, void *v) \
+{ \
+	seq_printf(m, "%d\n", name ); \
+	return 0; \
+} \
+static int name##_open(struct inode *inode, struct file *file) \
+{ \
+	return single_open(file, name##_proc_show, NULL); \
+} \
+\
+static const struct file_operations cifs_##name##_proc_fops = { \
+	.open		= name##_open, \
+	.read		= seq_read, \
+	.llseek		= seq_lseek, \
+	.release	= single_release, \
+	.write		= name##_write, \
+}
+
+extern int rdma_readwrite_threshold;
+extern int smbd_max_frmr_depth;
+extern int smbd_keep_alive_interval;
+extern int smbd_max_receive_size;
+extern int smbd_max_fragmented_recv_size;
+extern int smbd_max_send_size;
+extern int smbd_send_credit_target;
+extern int smbd_receive_credit_max;
+
+PROC_FILE_DEFINE(rdma_readwrite_threshold);
+PROC_FILE_DEFINE(smbd_max_frmr_depth);
+PROC_FILE_DEFINE(smbd_keep_alive_interval);
+PROC_FILE_DEFINE(smbd_max_receive_size);
+PROC_FILE_DEFINE(smbd_max_fragmented_recv_size);
+PROC_FILE_DEFINE(smbd_max_send_size);
+PROC_FILE_DEFINE(smbd_send_credit_target);
+PROC_FILE_DEFINE(smbd_receive_credit_max);
+#endif
+
 static struct proc_dir_entry *proc_fs_cifs;
 static const struct file_operations cifsFYI_proc_fops;
 static const struct file_operations cifs_lookup_cache_proc_fops;
@@ -398,6 +449,24 @@ cifs_proc_init(void)
 		    &cifs_security_flags_proc_fops);
 	proc_create("LookupCacheEnabled", 0, proc_fs_cifs,
 		    &cifs_lookup_cache_proc_fops);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	proc_create("rdma_readwrite_threshold", 0, proc_fs_cifs,
+		&cifs_rdma_readwrite_threshold_proc_fops);
+	proc_create("smbd_max_frmr_depth", 0, proc_fs_cifs,
+		&cifs_smbd_max_frmr_depth_proc_fops);
+	proc_create("smbd_keep_alive_interval", 0, proc_fs_cifs,
+		&cifs_smbd_keep_alive_interval_proc_fops);
+	proc_create("smbd_max_receive_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_receive_size_proc_fops);
+	proc_create("smbd_max_fragmented_recv_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_fragmented_recv_size_proc_fops);
+	proc_create("smbd_max_send_size", 0, proc_fs_cifs,
+		&cifs_smbd_max_send_size_proc_fops);
+	proc_create("smbd_send_credit_target", 0, proc_fs_cifs,
+		&cifs_smbd_send_credit_target_proc_fops);
+	proc_create("smbd_receive_credit_max", 0, proc_fs_cifs,
+		&cifs_smbd_receive_credit_max_proc_fops);
+#endif
 }
 
 void
@@ -415,6 +484,16 @@ cifs_proc_clean(void)
 	remove_proc_entry("SecurityFlags", proc_fs_cifs);
 	remove_proc_entry("LinuxExtensionsEnabled", proc_fs_cifs);
 	remove_proc_entry("LookupCacheEnabled", proc_fs_cifs);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	remove_proc_entry("rdma_readwrite_threshold", proc_fs_cifs);
+	remove_proc_entry("smbd_max_frmr_depth", proc_fs_cifs);
+	remove_proc_entry("smbd_keep_alive_interval", proc_fs_cifs);
+	remove_proc_entry("smbd_max_receive_size", proc_fs_cifs);
+	remove_proc_entry("smbd_max_fragmented_recv_size", proc_fs_cifs);
+	remove_proc_entry("smbd_max_send_size", proc_fs_cifs);
+	remove_proc_entry("smbd_send_credit_target", proc_fs_cifs);
+	remove_proc_entry("smbd_receive_credit_max", proc_fs_cifs);
+#endif
 	remove_proc_entry("fs/cifs", NULL);
 }
 
-- 
2.7.4

Re: [Patch v7 06/22] CIFS: SMBD: export protocol initial values

[Steve French]

merged into cifs-2.6.git for-next after cleaning up a couple checkpatch warnings

On Mon, Nov 20, 2017 at 1:37 AM, Leif Sahlberg <lsahlber@redhat.com> wrote:
> Acked-by: Ronnie Sahlberg <lsahlber@redhat.com>
>
>
> ----- Original Message -----
> From: "Long Li" <longli@exchange.microsoft.com>
> To: "Steve French" <sfrench@samba.org>, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, "Christoph Hellwig" <hch@infradead.org>, "Tom Talpey" <ttalpey@microsoft.com>, "Matthew Wilcox" <mawilcox@microsoft.com>, "Stephen Hemminger" <sthemmin@microsoft.com>
> Cc: "Long Li" <longli@microsoft.com>
> Sent: Tuesday, 7 November, 2017 7:54:58 PM
> Subject: [Patch v7 06/22] CIFS: SMBD: export protocol initial values
>
> From: Long Li <longli@microsoft.com>
>
> For use-configurable SMB Direct protocol values, export them to /proc/fs/cifs.
>
> Signed-off-by: Long Li <longli@microsoft.com>
> ---
>  fs/cifs/cifs_debug.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 79 insertions(+)
>
> diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
> index ba0870d..7025d8d 100644
> --- a/fs/cifs/cifs_debug.c
> +++ b/fs/cifs/cifs_debug.c
> @@ -30,6 +30,9 @@
>  #include "cifsproto.h"
>  #include "cifs_debug.h"
>  #include "cifsfs.h"
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +#include "smbdirect.h"
> +#endif
>
>  void
>  cifs_dump_mem(char *label, void *data, int length)
> @@ -371,6 +374,54 @@ static const struct file_operations cifs_stats_proc_fops = {
>  };
>  #endif /* STATS */
>
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +#define PROC_FILE_DEFINE(name) \
> +static ssize_t name##_write(struct file *file, const char __user *buffer, \
> +       size_t count, loff_t *ppos) \
> +{ \
> +       int rc; \
> +       rc = kstrtoint_from_user(buffer, count, 10, & name ); \
> +       if (rc) \
> +               return rc; \
> +       return count; \
> +} \
> +static int name##_proc_show(struct seq_file *m, void *v) \
> +{ \
> +       seq_printf(m, "%d\n", name ); \
> +       return 0; \
> +} \
> +static int name##_open(struct inode *inode, struct file *file) \
> +{ \
> +       return single_open(file, name##_proc_show, NULL); \
> +} \
> +\
> +static const struct file_operations cifs_##name##_proc_fops = { \
> +       .open           = name##_open, \
> +       .read           = seq_read, \
> +       .llseek         = seq_lseek, \
> +       .release        = single_release, \
> +       .write          = name##_write, \
> +}
> +
> +extern int rdma_readwrite_threshold;
> +extern int smbd_max_frmr_depth;
> +extern int smbd_keep_alive_interval;
> +extern int smbd_max_receive_size;
> +extern int smbd_max_fragmented_recv_size;
> +extern int smbd_max_send_size;
> +extern int smbd_send_credit_target;
> +extern int smbd_receive_credit_max;
> +
> +PROC_FILE_DEFINE(rdma_readwrite_threshold);
> +PROC_FILE_DEFINE(smbd_max_frmr_depth);
> +PROC_FILE_DEFINE(smbd_keep_alive_interval);
> +PROC_FILE_DEFINE(smbd_max_receive_size);
> +PROC_FILE_DEFINE(smbd_max_fragmented_recv_size);
> +PROC_FILE_DEFINE(smbd_max_send_size);
> +PROC_FILE_DEFINE(smbd_send_credit_target);
> +PROC_FILE_DEFINE(smbd_receive_credit_max);
> +#endif
> +
>  static struct proc_dir_entry *proc_fs_cifs;
>  static const struct file_operations cifsFYI_proc_fops;
>  static const struct file_operations cifs_lookup_cache_proc_fops;
> @@ -398,6 +449,24 @@ cifs_proc_init(void)
>                     &cifs_security_flags_proc_fops);
>         proc_create("LookupCacheEnabled", 0, proc_fs_cifs,
>                     &cifs_lookup_cache_proc_fops);
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +       proc_create("rdma_readwrite_threshold", 0, proc_fs_cifs,
> +               &cifs_rdma_readwrite_threshold_proc_fops);
> +       proc_create("smbd_max_frmr_depth", 0, proc_fs_cifs,
> +               &cifs_smbd_max_frmr_depth_proc_fops);
> +       proc_create("smbd_keep_alive_interval", 0, proc_fs_cifs,
> +               &cifs_smbd_keep_alive_interval_proc_fops);
> +       proc_create("smbd_max_receive_size", 0, proc_fs_cifs,
> +               &cifs_smbd_max_receive_size_proc_fops);
> +       proc_create("smbd_max_fragmented_recv_size", 0, proc_fs_cifs,
> +               &cifs_smbd_max_fragmented_recv_size_proc_fops);
> +       proc_create("smbd_max_send_size", 0, proc_fs_cifs,
> +               &cifs_smbd_max_send_size_proc_fops);
> +       proc_create("smbd_send_credit_target", 0, proc_fs_cifs,
> +               &cifs_smbd_send_credit_target_proc_fops);
> +       proc_create("smbd_receive_credit_max", 0, proc_fs_cifs,
> +               &cifs_smbd_receive_credit_max_proc_fops);
> +#endif
>  }
>
>  void
> @@ -415,6 +484,16 @@ cifs_proc_clean(void)
>         remove_proc_entry("SecurityFlags", proc_fs_cifs);
>         remove_proc_entry("LinuxExtensionsEnabled", proc_fs_cifs);
>         remove_proc_entry("LookupCacheEnabled", proc_fs_cifs);
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +       remove_proc_entry("rdma_readwrite_threshold", proc_fs_cifs);
> +       remove_proc_entry("smbd_max_frmr_depth", proc_fs_cifs);
> +       remove_proc_entry("smbd_keep_alive_interval", proc_fs_cifs);
> +       remove_proc_entry("smbd_max_receive_size", proc_fs_cifs);
> +       remove_proc_entry("smbd_max_fragmented_recv_size", proc_fs_cifs);
> +       remove_proc_entry("smbd_max_send_size", proc_fs_cifs);
> +       remove_proc_entry("smbd_send_credit_target", proc_fs_cifs);
> +       remove_proc_entry("smbd_receive_credit_max", proc_fs_cifs);
> +#endif
>         remove_proc_entry("fs/cifs", NULL);
>  }
>
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Thanks,

Steve

[Patch v7 08/22] CIFS: SMBD: Upper layer connects to SMBDirect session

[Long Li]

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

When "rdma" is specified in the mount option, make CIFS connect to
SMB Direct.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/connect.c | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index b5a575f..75eb50b 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -44,7 +44,6 @@
 #include <net/ipv6.h>
 #include <linux/parser.h>
 #include <linux/bvec.h>
-
 #include "cifspdu.h"
 #include "cifsglob.h"
 #include "cifsproto.h"
@@ -56,6 +55,7 @@
 #include "rfc1002pdu.h"
 #include "fscache.h"
 #include "smb2proto.h"
+#include "smbdirect.h"
 
 #define CIFS_PORT 445
 #define RFC1001_PORT 139
@@ -2279,13 +2279,29 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
 		tcp_ses->echo_interval = volume_info->echo_interval * HZ;
 	else
 		tcp_ses->echo_interval = SMB_ECHO_INTERVAL_DEFAULT * HZ;
-
+	if (tcp_ses->rdma) {
+#ifndef CONFIG_CIFS_SMB_DIRECT
+		cifs_dbg(VFS, "CONFIG_CIFS_SMB_DIRECT is not enabled\n");
+		rc = -ENOENT;
+		goto out_err_crypto_release;
+#endif
+		tcp_ses->smbd_conn = smbd_get_connection(
+			tcp_ses, (struct sockaddr *)&volume_info->dstaddr);
+		if (tcp_ses->smbd_conn) {
+			cifs_dbg(VFS, "RDMA transport established\n");
+			rc = 0;
+			goto smbd_connected;
+		} else {
+			rc = -ENOENT;
+			goto out_err_crypto_release;
+		}
+	}
 	rc = ip_connect(tcp_ses);
 	if (rc < 0) {
 		cifs_dbg(VFS, "Error connecting to socket. Aborting operation.\n");
 		goto out_err_crypto_release;
 	}
-
+smbd_connected:
 	/*
 	 * since we're in a cifs function already, we know that
 	 * this will succeed. No need for try_module_get().
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Patch v7 08/22] CIFS: SMBD: Upper layer connects to SMBDirect session

[Long Li]

From: Long Li <longli@microsoft.com>

When "rdma" is specified in the mount option, make CIFS connect to
SMB Direct.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/connect.c | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index b5a575f..75eb50b 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -44,7 +44,6 @@
 #include <net/ipv6.h>
 #include <linux/parser.h>
 #include <linux/bvec.h>
-
 #include "cifspdu.h"
 #include "cifsglob.h"
 #include "cifsproto.h"
@@ -56,6 +55,7 @@
 #include "rfc1002pdu.h"
 #include "fscache.h"
 #include "smb2proto.h"
+#include "smbdirect.h"
 
 #define CIFS_PORT 445
 #define RFC1001_PORT 139
@@ -2279,13 +2279,29 @@ cifs_get_tcp_session(struct smb_vol *volume_info)
 		tcp_ses->echo_interval = volume_info->echo_interval * HZ;
 	else
 		tcp_ses->echo_interval = SMB_ECHO_INTERVAL_DEFAULT * HZ;
-
+	if (tcp_ses->rdma) {
+#ifndef CONFIG_CIFS_SMB_DIRECT
+		cifs_dbg(VFS, "CONFIG_CIFS_SMB_DIRECT is not enabled\n");
+		rc = -ENOENT;
+		goto out_err_crypto_release;
+#endif
+		tcp_ses->smbd_conn = smbd_get_connection(
+			tcp_ses, (struct sockaddr *)&volume_info->dstaddr);
+		if (tcp_ses->smbd_conn) {
+			cifs_dbg(VFS, "RDMA transport established\n");
+			rc = 0;
+			goto smbd_connected;
+		} else {
+			rc = -ENOENT;
+			goto out_err_crypto_release;
+		}
+	}
 	rc = ip_connect(tcp_ses);
 	if (rc < 0) {
 		cifs_dbg(VFS, "Error connecting to socket. Aborting operation.\n");
 		goto out_err_crypto_release;
 	}
-
+smbd_connected:
 	/*
 	 * since we're in a cifs function already, we know that
 	 * this will succeed. No need for try_module_get().
-- 
2.7.4

[Patch v7 15/22] CIFS: SMBD: Upper layer receives data via RDMA receive

[Long Li]

From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>

With SMB Direct connected, use it for receiving data via RDMA receive.

Signed-off-by: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
---
 fs/cifs/connect.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 3d74983..fcd01df 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -539,8 +539,10 @@ cifs_readv_from_socket(struct TCP_Server_Info *server, struct msghdr *smb_msg)
 
 		if (server_unresponsive(server))
 			return -ECONNABORTED;
-
-		length = sock_recvmsg(server->ssocket, smb_msg, 0);
+		if (cifs_rdma_enabled(server) && server->smbd_conn)
+			length = smbd_recv(server->smbd_conn, smb_msg);
+		else
+			length = sock_recvmsg(server->ssocket, smb_msg, 0);
 
 		if (server->tcpStatus == CifsExiting)
 			return -ESHUTDOWN;
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Patch v7 15/22] CIFS: SMBD: Upper layer receives data via RDMA receive

[Long Li]

From: Long Li <longli@microsoft.com>

With SMB Direct connected, use it for receiving data via RDMA receive.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/connect.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 3d74983..fcd01df 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -539,8 +539,10 @@ cifs_readv_from_socket(struct TCP_Server_Info *server, struct msghdr *smb_msg)
 
 		if (server_unresponsive(server))
 			return -ECONNABORTED;
-
-		length = sock_recvmsg(server->ssocket, smb_msg, 0);
+		if (cifs_rdma_enabled(server) && server->smbd_conn)
+			length = smbd_recv(server->smbd_conn, smb_msg);
+		else
+			length = sock_recvmsg(server->ssocket, smb_msg, 0);
 
 		if (server->tcpStatus == CifsExiting)
 			return -ESHUTDOWN;
-- 
2.7.4

Re: [Patch v7 00/22] CIFS: Implement SMB Direct protocol

[Steve French]

Have remerged (to cifs-2.6.git for-next) the first seven in this
series - after he incorporated the fixes for the recent feedback on
the series to a few (and I cleaned up a few minor checkpatch nits).

On Tue, Nov 7, 2017 at 2:54 AM, Long Li <longli-Lp/cVzEoVyZiJJESP9tAQJZ3qXmFLfmx@public.gmane.org> wrote:
> From: Long Li <longli-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
>
> Starting with SMB2 dialect 3.0, Microsoft introduced SMB Direct transport
> protocol for transferring upper layer (SMB2) payload over RDMA via Infiniband,
> RoCE or iWARP. The prococol is published in [MS-SMBD]
> (https://msdn.microsoft.com/en-us/library/hh536346.aspx).
>
> Change log:
> v2:
> Implemented RDMA read/write via memory registration.
> Re-arranged patches for review [Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>].
> Restructured the code and fixed bugs on protocol timer and keepalive
> [Tom Talpey <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>].
>
> v3:
> Improved performance by introducing an additional queue for handling
> empty packets and reducing lock contention on IRQ path.
> Added light weight profiling by reading TSC.
> Improved the code for checking SMB versions when mounting with rdma option
> [Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>].
> Moved to use pages instead of buffers for passing I/O for RDMA
> [Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>].
> Removed redundant code and refactored I/O code paths
> [Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, Tom Talpey <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>].
>
> v4:
> Fixed connectivity issues with iWAPR devices.
> Exported configurable protocol parameters to /proc/fs/cifs
> [Steve French <sfrench-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>]
> Re-arranged patches for review
> [Pavel Shilovsky <piastryyy-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>].
>
> v5:
> Fixed compiling errors on ia64, i386 and when INFINIBAND is not
> configured. [kbuild test robot]
> Profiling is removed and will be introduced in a seperate patch.
>
> v6:
> Report internal code error via WARN_ON(). Change description in
> Kconfig [Pavel Shilovsky <piastryyy-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>].
>
> v7:
> Removed the use of #ifdef CONFIG_CIFS_SMB_DIRECT in upper layer
> code calling transport I/O. [Matthew Wilcox <mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>,
> Tom Talpey <ttalpey-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>]
>
> Long Li (22):
>   CIFS: SMBD: Add parameter rdata to smb2_new_read_req
>   CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT
>   CIFS: SMBD: Add rdma mount option
>   CIFS: SMBD: Add SMB Direct protocol initial values and constants
>   CIFS: SMBD: Establish SMB Direct connection
>   CIFS: SMBD: export protocol initial values
>   CIFS: SMBD: Implement function to create a SMB Direct connection
>   CIFS: SMBD: Upper layer connects to SMBDirect session
>   CIFS: SMBD: Implement function to reconnect to a SMB Direct transport
>   CIFS: SMBD: Upper layer reconnects to SMB Direct session
>   CIFS: SMBD: Implement function to destroy a SMB Direct connection
>   CIFS: SMBD: Upper layer destroys SMB Direct session on shutdown or
>     umount
>   CIFS: SMBD: Set SMB Direct maximum read or write size for I/O
>   CIFS: SMBD: Implement function to receive data via RDMA receive
>   CIFS: SMBD: Upper layer receives data via RDMA receive
>   CIFS: SMBD: Implement function to send data via RDMA send
>   CIFS: SMBD: Upper layer sends data via RDMA send
>   CIFS: SMBD: Implement RDMA memory registration
>   CIFS: SMBD: Upper layer performs SMB write via RDMA read through
>     memory registration
>   CIFS: SMBD: Read correct returned data length for RDMA write (SMB
>     read) I/O
>   CIFS: SMBD: Upper layer performs SMB read via RDMA write through
>     memory registration
>   CIFS: SMBD: Add SMB Direct debug counters
>
>  fs/cifs/Kconfig      |    8 +
>  fs/cifs/Makefile     |    2 +
>  fs/cifs/cifs_debug.c |  147 +++
>  fs/cifs/cifsfs.c     |    2 +
>  fs/cifs/cifsglob.h   |   21 +-
>  fs/cifs/cifssmb.c    |   15 +-
>  fs/cifs/connect.c    |   53 +-
>  fs/cifs/file.c       |   17 +-
>  fs/cifs/smb1ops.c    |    4 +-
>  fs/cifs/smb2ops.c    |   24 +-
>  fs/cifs/smb2pdu.c    |  127 ++-
>  fs/cifs/smbdirect.c  | 2617 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  fs/cifs/smbdirect.h  |  330 +++++++
>  fs/cifs/transport.c  |    8 +-
>  14 files changed, 3344 insertions(+), 31 deletions(-)
>  create mode 100644 fs/cifs/smbdirect.c
>  create mode 100644 fs/cifs/smbdirect.h
>
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Thanks,

Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Patch v7 00/22] CIFS: Implement SMB Direct protocol

[Steve French]

Have remerged (to cifs-2.6.git for-next) the first seven in this
series - after he incorporated the fixes for the recent feedback on
the series to a few (and I cleaned up a few minor checkpatch nits).

On Tue, Nov 7, 2017 at 2:54 AM, Long Li <longli@exchange.microsoft.com> wrote:
> From: Long Li <longli@microsoft.com>
>
> Starting with SMB2 dialect 3.0, Microsoft introduced SMB Direct transport
> protocol for transferring upper layer (SMB2) payload over RDMA via Infiniband,
> RoCE or iWARP. The prococol is published in [MS-SMBD]
> (https://msdn.microsoft.com/en-us/library/hh536346.aspx).
>
> Change log:
> v2:
> Implemented RDMA read/write via memory registration.
> Re-arranged patches for review [Christoph Hellwig <hch@infradead.org>].
> Restructured the code and fixed bugs on protocol timer and keepalive
> [Tom Talpey <ttalpey@microsoft.com>].
>
> v3:
> Improved performance by introducing an additional queue for handling
> empty packets and reducing lock contention on IRQ path.
> Added light weight profiling by reading TSC.
> Improved the code for checking SMB versions when mounting with rdma option
> [Leon Romanovsky <leon@kernel.org>].
> Moved to use pages instead of buffers for passing I/O for RDMA
> [Christoph Hellwig <hch@infradead.org>].
> Removed redundant code and refactored I/O code paths
> [Christoph Hellwig <hch@infradead.org>, Tom Talpey <ttalpey@microsoft.com>].
>
> v4:
> Fixed connectivity issues with iWAPR devices.
> Exported configurable protocol parameters to /proc/fs/cifs
> [Steve French <sfrench@samba.org>]
> Re-arranged patches for review
> [Pavel Shilovsky <piastryyy@gmail.com>].
>
> v5:
> Fixed compiling errors on ia64, i386 and when INFINIBAND is not
> configured. [kbuild test robot]
> Profiling is removed and will be introduced in a seperate patch.
>
> v6:
> Report internal code error via WARN_ON(). Change description in
> Kconfig [Pavel Shilovsky <piastryyy@gmail.com>].
>
> v7:
> Removed the use of #ifdef CONFIG_CIFS_SMB_DIRECT in upper layer
> code calling transport I/O. [Matthew Wilcox <mawilcox@microsoft.com>,
> Tom Talpey <ttalpey@microsoft.com>]
>
> Long Li (22):
>   CIFS: SMBD: Add parameter rdata to smb2_new_read_req
>   CIFS: SMBD: Introduce kernel config option CONFIG_CIFS_SMB_DIRECT
>   CIFS: SMBD: Add rdma mount option
>   CIFS: SMBD: Add SMB Direct protocol initial values and constants
>   CIFS: SMBD: Establish SMB Direct connection
>   CIFS: SMBD: export protocol initial values
>   CIFS: SMBD: Implement function to create a SMB Direct connection
>   CIFS: SMBD: Upper layer connects to SMBDirect session
>   CIFS: SMBD: Implement function to reconnect to a SMB Direct transport
>   CIFS: SMBD: Upper layer reconnects to SMB Direct session
>   CIFS: SMBD: Implement function to destroy a SMB Direct connection
>   CIFS: SMBD: Upper layer destroys SMB Direct session on shutdown or
>     umount
>   CIFS: SMBD: Set SMB Direct maximum read or write size for I/O
>   CIFS: SMBD: Implement function to receive data via RDMA receive
>   CIFS: SMBD: Upper layer receives data via RDMA receive
>   CIFS: SMBD: Implement function to send data via RDMA send
>   CIFS: SMBD: Upper layer sends data via RDMA send
>   CIFS: SMBD: Implement RDMA memory registration
>   CIFS: SMBD: Upper layer performs SMB write via RDMA read through
>     memory registration
>   CIFS: SMBD: Read correct returned data length for RDMA write (SMB
>     read) I/O
>   CIFS: SMBD: Upper layer performs SMB read via RDMA write through
>     memory registration
>   CIFS: SMBD: Add SMB Direct debug counters
>
>  fs/cifs/Kconfig      |    8 +
>  fs/cifs/Makefile     |    2 +
>  fs/cifs/cifs_debug.c |  147 +++
>  fs/cifs/cifsfs.c     |    2 +
>  fs/cifs/cifsglob.h   |   21 +-
>  fs/cifs/cifssmb.c    |   15 +-
>  fs/cifs/connect.c    |   53 +-
>  fs/cifs/file.c       |   17 +-
>  fs/cifs/smb1ops.c    |    4 +-
>  fs/cifs/smb2ops.c    |   24 +-
>  fs/cifs/smb2pdu.c    |  127 ++-
>  fs/cifs/smbdirect.c  | 2617 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  fs/cifs/smbdirect.h  |  330 +++++++
>  fs/cifs/transport.c  |    8 +-
>  14 files changed, 3344 insertions(+), 31 deletions(-)
>  create mode 100644 fs/cifs/smbdirect.c
>  create mode 100644 fs/cifs/smbdirect.h
>
> --
> 2.7.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Thanks,

Steve

[Patch v7 09/22] CIFS: SMBD: Implement function to reconnect to a SMB Direct transport

[Long Li]

From: Long Li <longli@microsoft.com>

Add function to implement a reconnect to SMB Direct. This involves tearing down
the current connection and establishing/negotiating a new connection.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smbdirect.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index 1db39a34..9f23330 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -1392,6 +1392,42 @@ static void idle_connection_timer(struct work_struct *work)
 			info->keep_alive_interval*HZ);
 }
 
+/*
+ * Reconnect this SMBD connection, called from upper layer
+ * return value: 0 on success, or actual error code
+ */
+int smbd_reconnect(struct TCP_Server_Info *server)
+{
+	log_rdma_event(INFO, "reconnecting rdma session\n");
+
+	if (!server->smbd_conn) {
+		log_rdma_event(ERR, "rdma session already destroyed\n");
+		return -EINVAL;
+	}
+
+	/*
+	 * This is possible if transport is disconnected and we haven't received
+	 * notification from RDMA, but upper layer has detected timeout
+	 */
+	if (server->smbd_conn->transport_status == SMBD_CONNECTED) {
+		log_rdma_event(INFO, "disconnecting transport\n");
+		smbd_disconnect_rdma_connection(server->smbd_conn);
+	}
+
+	/* wait until the transport is destroyed */
+	wait_event(server->smbd_conn->wait_destroy,
+		server->smbd_conn->transport_status == SMBD_DESTROYED);
+
+	destroy_workqueue(server->smbd_conn->workqueue);
+	kfree(server->smbd_conn);
+
+	log_rdma_event(INFO, "creating rdma session\n");
+	server->smbd_conn = smbd_get_connection(
+		server, (struct sockaddr *) &server->dstaddr);
+
+	return server->smbd_conn ? 0 : -ENOENT;
+}
+
 static void destroy_caches_and_workqueue(struct smbd_connection *info)
 {
 	destroy_receive_buffers(info);
-- 
2.7.4

[Patch v7 10/22] CIFS: SMBD: Upper layer reconnects to SMB Direct session

[Long Li]

From: Long Li <longli@microsoft.com>

Do a reconnect on SMB Direct when it is used as the connection. Reconnect can
happen for many reasons and it's mostly the decision of SMB2 upper layer.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/connect.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 75eb50b..3ccd703 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -404,7 +404,10 @@ cifs_reconnect(struct TCP_Server_Info *server)
 
 		/* we should try only the port we connected to before */
 		mutex_lock(&server->srv_mutex);
-		rc = generic_ip_connect(server);
+		if (cifs_rdma_enabled(server))
+			rc = smbd_reconnect(server);
+		else
+			rc = generic_ip_connect(server);
 		if (rc) {
 			cifs_dbg(FYI, "reconnect error %d\n", rc);
 			mutex_unlock(&server->srv_mutex);
-- 
2.7.4

[Patch v7 11/22] CIFS: SMBD: Implement function to destroy a SMB Direct connection

[Long Li]

From: Long Li <longli@microsoft.com>

Add function to tear down a SMB Direct connection. This is used by upper layer
to free all SMB Direct connection and transport resources.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smbdirect.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index 9f23330..a8e7239 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -1392,6 +1392,22 @@ static void idle_connection_timer(struct work_struct *work)
 			info->keep_alive_interval*HZ);
 }
 
+/* Destroy this SMBD connection, called from upper layer */
+void smbd_destroy(struct smbd_connection *info)
+{
+	log_rdma_event(INFO, "destroying rdma session\n");
+
+	/* Kick off the disconnection process */
+	smbd_disconnect_rdma_connection(info);
+
+	log_rdma_event(INFO, "wait for transport being destroyed\n");
+	wait_event(info->wait_destroy,
+		info->transport_status == SMBD_DESTROYED);
+
+	destroy_workqueue(info->workqueue);
+	kfree(info);
+}
+
 /*
  * Reconnect this SMBD connection, called from upper layer
  * return value: 0 on success, or actual error code
-- 
2.7.4

[Patch v7 12/22] CIFS: SMBD: Upper layer destroys SMB Direct session on shutdown or umount

[Long Li]

From: Long Li <longli@microsoft.com>

When upper layer wants to umount, make it call shutdown on transport when
SMB Direct is used.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/connect.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 3ccd703..3d74983 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -701,7 +701,10 @@ static void clean_demultiplex_info(struct TCP_Server_Info *server)
 	wake_up_all(&server->request_q);
 	/* give those requests time to exit */
 	msleep(125);
-
+	if (cifs_rdma_enabled(server) && server->smbd_conn) {
+		smbd_destroy(server->smbd_conn);
+		server->smbd_conn = NULL;
+	}
 	if (server->ssocket) {
 		sock_release(server->ssocket);
 		server->ssocket = NULL;
-- 
2.7.4

[Patch v7 13/22] CIFS: SMBD: Set SMB Direct maximum read or write size for I/O

[Long Li]

From: Long Li <longli@microsoft.com>

When connecting over SMB Direct, the transport negotiates its maximum I/O sizes
with the server and determines how to choose to do RDMA send/recv vs
read/write. Expose these maximum I/O sizes to upper layer so we will get the
correct sized payloads.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smb2ops.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/fs/cifs/smb2ops.c b/fs/cifs/smb2ops.c
index fb2934b..abc8bd5 100644
--- a/fs/cifs/smb2ops.c
+++ b/fs/cifs/smb2ops.c
@@ -32,6 +32,7 @@
 #include "smb2status.h"
 #include "smb2glob.h"
 #include "cifs_ioctl.h"
+#include "smbdirect.h"
 
 static int
 change_conf(struct TCP_Server_Info *server)
@@ -250,7 +251,11 @@ smb2_negotiate_wsize(struct cifs_tcon *tcon, struct smb_vol *volume_info)
 	/* start with specified wsize, or default */
 	wsize = volume_info->wsize ? volume_info->wsize : CIFS_DEFAULT_IOSIZE;
 	wsize = min_t(unsigned int, wsize, server->max_write);
-
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	if (server->rdma)
+		wsize = min_t(unsigned int,
+				wsize, server->smbd_conn->max_readwrite_size);
+#endif
 	if (!(server->capabilities & SMB2_GLOBAL_CAP_LARGE_MTU))
 		wsize = min_t(unsigned int, wsize, SMB2_MAX_BUFFER_SIZE);
 
@@ -266,6 +271,11 @@ smb2_negotiate_rsize(struct cifs_tcon *tcon, struct smb_vol *volume_info)
 	/* start with specified rsize, or default */
 	rsize = volume_info->rsize ? volume_info->rsize : CIFS_DEFAULT_IOSIZE;
 	rsize = min_t(unsigned int, rsize, server->max_read);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	if (server->rdma)
+		rsize = min_t(unsigned int,
+				rsize, server->smbd_conn->max_readwrite_size);
+#endif
 
 	if (!(server->capabilities & SMB2_GLOBAL_CAP_LARGE_MTU))
 		rsize = min_t(unsigned int, rsize, SMB2_MAX_BUFFER_SIZE);
-- 
2.7.4

[Patch v7 14/22] CIFS: SMBD: Implement function to receive data via RDMA receive

[Long Li]

From: Long Li <longli@microsoft.com>

On the receive path, the transport maintains receive buffers and a reassembly
queue for transferring payload via RDMA recv. There is data copy in the
transport on recv when it copies the payload to upper layer.

The transport recognizes the RFC1002 header length use in the SMB
upper layer payloads in CIFS. Because this length is mainly used for TCP and
not applicable to RDMA, it is handled as a out-of-band information and is
never sent over the wire, and the trasnport behaves like TCP to upper layer
by processing and exposing the length correctly on data payloads.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smbdirect.c | 228 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h |   3 +
 2 files changed, 231 insertions(+)

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index a8e7239..5a08015 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -14,6 +14,7 @@
  *   the GNU General Public License for more details.
  */
 #include <linux/module.h>
+#include <linux/highmem.h>
 #include "smbdirect.h"
 #include "cifs_debug.h"
 
@@ -179,6 +180,8 @@ static void smbd_destroy_rdma_work(struct work_struct *work)
 
 	log_rdma_event(INFO, "wait for all recv to finish\n");
 	wake_up_interruptible(&info->wait_reassembly_queue);
+	wait_event(info->wait_smbd_recv_pending,
+		info->smbd_recv_pending == 0);
 
 	log_rdma_event(INFO, "wait for all send posted to IB to finish\n");
 	wait_event(info->wait_send_pending,
@@ -1654,6 +1657,9 @@ struct smbd_connection *_smbd_get_connection(
 	queue_delayed_work(info->workqueue, &info->idle_timer_work,
 		info->keep_alive_interval*HZ);
 
+	init_waitqueue_head(&info->wait_smbd_recv_pending);
+	info->smbd_recv_pending = 0;
+
 	init_waitqueue_head(&info->wait_send_pending);
 	atomic_set(&info->send_pending, 0);
 
@@ -1720,3 +1726,225 @@ struct smbd_connection *smbd_get_connection(
 	}
 	return ret;
 }
+
+/*
+ * Receive data from receive reassembly queue
+ * All the incoming data packets are placed in reassembly queue
+ * buf: the buffer to read data into
+ * size: the length of data to read
+ * return value: actual data read
+ * Note: this implementation copies the data from reassebmly queue to receive
+ * buffers used by upper layer. This is not the optimal code path. A better way
+ * to do it is to not have upper layer allocate its receive buffers but rather
+ * borrow the buffer from reassembly queue, and return it after data is
+ * consumed. But this will require more changes to upper layer code, and also
+ * need to consider packet boundaries while they still being reassembled.
+ */
+int smbd_recv_buf(struct smbd_connection *info, char *buf, unsigned int size)
+{
+	struct smbd_response *response;
+	struct smbd_data_transfer *data_transfer;
+	int to_copy, to_read, data_read, offset;
+	u32 data_length, remaining_data_length, data_offset;
+	int rc;
+	unsigned long flags;
+
+again:
+	if (info->transport_status != SMBD_CONNECTED) {
+		log_read(ERR, "disconnected\n");
+		return -ENODEV;
+	}
+
+	/*
+	 * No need to hold the reassembly queue lock all the time as we are
+	 * the only one reading from the front of the queue. The transport
+	 * may add more entries to the back of the queeu at the same time
+	 */
+	log_read(INFO, "size=%d info->reassembly_data_length=%d\n", size,
+		info->reassembly_data_length);
+	if (info->reassembly_data_length >= size) {
+		int queue_length;
+		int queue_removed = 0;
+
+		/*
+		 * Need to make sure reassembly_data_length is read before
+		 * reading reassembly_queue_length and calling
+		 * _get_first_reassembly. This call is lock free
+		 * as we never read at the end of the queue which are being
+		 * updated in SOFTIRQ as more data is received
+		 */
+		virt_rmb();
+		queue_length = info->reassembly_queue_length;
+		data_read = 0;
+		to_read = size;
+		offset = info->first_entry_offset;
+		while (data_read < size) {
+			response = _get_first_reassembly(info);
+			data_transfer = smbd_response_payload(response);
+			data_length = le32_to_cpu(data_transfer->data_length);
+			remaining_data_length =
+				le32_to_cpu(
+					data_transfer->remaining_data_length);
+			data_offset = le32_to_cpu(data_transfer->data_offset);
+
+			/*
+			 * The upper layer expects RFC1002 length at the
+			 * beginning of the payload. Return it to indicate
+			 * the total length of the packet. This minimize the
+			 * change to upper layer packet processing logic. This
+			 * will be eventually remove when an intermediate
+			 * transport layer is added
+			 */
+			if (response->first_segment && size == 4) {
+				unsigned int rfc1002_len =
+					data_length + remaining_data_length;
+				*((__be32 *)buf) = cpu_to_be32(rfc1002_len);
+				data_read = 4;
+				response->first_segment = false;
+				log_read(INFO, "returning rfc1002 length %d\n",
+					rfc1002_len);
+				goto read_rfc1002_done;
+			}
+
+			to_copy = min_t(int, data_length - offset, to_read);
+			memcpy(
+				buf + data_read,
+				(char *)data_transfer + data_offset + offset,
+				to_copy);
+
+			/* move on to the next buffer? */
+			if (to_copy == data_length - offset) {
+				queue_length--;
+				/*
+				 * No need to lock if we are not at the
+				 * end of the queue
+				 */
+				if (!queue_length)
+					spin_lock_irqsave(
+						&info->reassembly_queue_lock,
+						flags);
+				list_del(&response->list);
+				queue_removed++;
+				if (!queue_length)
+					spin_unlock_irqrestore(
+						&info->reassembly_queue_lock,
+						flags);
+
+				info->count_reassembly_queue--;
+				info->count_dequeue_reassembly_queue++;
+				put_receive_buffer(info, response, true);
+				offset = 0;
+				log_read(INFO, "put_receive_buffer offset=0\n");
+			} else
+				offset += to_copy;
+
+			to_read -= to_copy;
+			data_read += to_copy;
+
+			log_read(INFO, "_get_first_reassembly memcpy %d bytes "
+				"data_transfer_length-offset=%d after that "
+				"to_read=%d data_read=%d offset=%d\n",
+				to_copy, data_length - offset,
+				to_read, data_read, offset);
+		}
+
+		spin_lock_irqsave(&info->reassembly_queue_lock, flags);
+		info->reassembly_data_length -= data_read;
+		info->reassembly_queue_length -= queue_removed;
+		spin_unlock_irqrestore(&info->reassembly_queue_lock, flags);
+
+		info->first_entry_offset = offset;
+		log_read(INFO, "returning to thread data_read=%d "
+			"reassembly_data_length=%d first_entry_offset=%d\n",
+			data_read, info->reassembly_data_length,
+			info->first_entry_offset);
+read_rfc1002_done:
+		return data_read;
+	}
+
+	log_read(INFO, "wait_event on more data\n");
+	rc = wait_event_interruptible(
+		info->wait_reassembly_queue,
+		info->reassembly_data_length >= size ||
+			info->transport_status != SMBD_CONNECTED);
+	/* Don't return any data if interrupted */
+	if (rc)
+		return -ENODEV;
+
+	goto again;
+}
+
+/*
+ * Receive a page from receive reassembly queue
+ * page: the page to read data into
+ * to_read: the length of data to read
+ * return value: actual data read
+ */
+int smbd_recv_page(struct smbd_connection *info,
+		struct page *page, unsigned int to_read)
+{
+	int ret;
+	char *to_address;
+
+	/* make sure we have the page ready for read */
+	ret = wait_event_interruptible(
+		info->wait_reassembly_queue,
+		info->reassembly_data_length >= to_read ||
+			info->transport_status != SMBD_CONNECTED);
+	if (ret)
+		return 0;
+
+	/* now we can read from reassembly queue and not sleep */
+	to_address = kmap_atomic(page);
+
+	log_read(INFO, "reading from page=%p address=%p to_read=%d\n",
+		page, to_address, to_read);
+
+	ret = smbd_recv_buf(info, to_address, to_read);
+	kunmap_atomic(to_address);
+
+	return ret;
+}
+
+/*
+ * Receive data from transport
+ * msg: a msghdr point to the buffer, can be ITER_KVEC or ITER_BVEC
+ * return: total bytes read, or 0. SMB Direct will not do partial read.
+ */
+int smbd_recv(struct smbd_connection *info, struct msghdr *msg)
+{
+	char *buf;
+	struct page *page;
+	unsigned int to_read;
+	int rc;
+
+	info->smbd_recv_pending++;
+
+	switch (msg->msg_iter.type) {
+	case READ | ITER_KVEC:
+		buf = msg->msg_iter.kvec->iov_base;
+		to_read = msg->msg_iter.kvec->iov_len;
+		rc = smbd_recv_buf(info, buf, to_read);
+		break;
+
+	case READ | ITER_BVEC:
+		page = msg->msg_iter.bvec->bv_page;
+		to_read = msg->msg_iter.bvec->bv_len;
+		rc = smbd_recv_page(info, page, to_read);
+		break;
+
+	default:
+		/* It's a bug in upper layer to get there */
+		cifs_dbg(VFS, "CIFS: invalid msg type %d\n",
+			msg->msg_iter.type);
+		rc = -EIO;
+	}
+
+	info->smbd_recv_pending--;
+	wake_up(&info->wait_smbd_recv_pending);
+
+	/* SMBDirect will read it all or nothing */
+	if (rc > 0)
+		msg->msg_iter.count = 0;
+	return rc;
+}
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
index 35bc25b..65a431f 100644
--- a/fs/cifs/smbdirect.h
+++ b/fs/cifs/smbdirect.h
@@ -91,6 +91,9 @@ struct smbd_connection {
 	int fragment_reassembly_remaining;
 
 	/* Activity accoutning */
+	/* Pending reqeusts issued from upper layer */
+	int smbd_recv_pending;
+	wait_queue_head_t wait_smbd_recv_pending;
 
 	atomic_t send_pending;
 	wait_queue_head_t wait_send_pending;
-- 
2.7.4

[Patch v7 16/22] CIFS: SMBD: Implement function to send data via RDMA send

[Long Li]

From: Long Li <longli@microsoft.com>

The transport doesn't maintain send buffers or send queue for transferring
payload via RDMA send. There is no data copy in the transport on send.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smbdirect.c | 246 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h |   3 +
 2 files changed, 249 insertions(+)

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index 5a08015..0705f49 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -42,6 +42,12 @@ static int smbd_post_recv(
 		struct smbd_response *response);
 
 static int smbd_post_send_empty(struct smbd_connection *info);
+static int smbd_post_send_data(
+		struct smbd_connection *info,
+		struct kvec *iov, int n_vec, int remaining_data_length);
+static int smbd_post_send_page(struct smbd_connection *info,
+		struct page *page, unsigned long offset,
+		size_t size, int remaining_data_length);
 
 /* SMBD version number */
 #define SMBD_V1	0x0100
@@ -178,6 +184,10 @@ static void smbd_destroy_rdma_work(struct work_struct *work)
 	log_rdma_event(INFO, "cancelling send immediate work\n");
 	cancel_delayed_work_sync(&info->send_immediate_work);
 
+	log_rdma_event(INFO, "wait for all send to finish\n");
+	wait_event(info->wait_smbd_send_pending,
+		info->smbd_send_pending == 0);
+
 	log_rdma_event(INFO, "wait for all recv to finish\n");
 	wake_up_interruptible(&info->wait_reassembly_queue);
 	wait_event(info->wait_smbd_recv_pending,
@@ -1080,6 +1090,24 @@ static int smbd_post_send_sgl(struct smbd_connection *info,
 }
 
 /*
+ * Send a page
+ * page: the page to send
+ * offset: offset in the page to send
+ * size: length in the page to send
+ * remaining_data_length: remaining data to send in this payload
+ */
+static int smbd_post_send_page(struct smbd_connection *info, struct page *page,
+		unsigned long offset, size_t size, int remaining_data_length)
+{
+	struct scatterlist sgl;
+
+	sg_init_table(&sgl, 1);
+	sg_set_page(&sgl, page, size, offset);
+
+	return smbd_post_send_sgl(info, &sgl, size, remaining_data_length);
+}
+
+/*
  * Send an empty message
  * Empty message is used to extend credits to peer to for keep live
  * while there is no upper layer payload to send at the time
@@ -1091,6 +1119,35 @@ static int smbd_post_send_empty(struct smbd_connection *info)
 }
 
 /*
+ * Send a data buffer
+ * iov: the iov array describing the data buffers
+ * n_vec: number of iov array
+ * remaining_data_length: remaining data to send following this packet
+ * in segmented SMBD packet
+ */
+static int smbd_post_send_data(
+	struct smbd_connection *info, struct kvec *iov, int n_vec,
+	int remaining_data_length)
+{
+	int i;
+	u32 data_length = 0;
+	struct scatterlist sgl[SMBDIRECT_MAX_SGE];
+
+	if (n_vec > SMBDIRECT_MAX_SGE) {
+		cifs_dbg(VFS, "Can't fit data to SGL, n_vec=%d\n", n_vec);
+		return -ENOMEM;
+	}
+
+	sg_init_table(sgl, n_vec);
+	for (i = 0; i < n_vec; i++) {
+		data_length += iov[i].iov_len;
+		sg_set_buf(&sgl[i], iov[i].iov_base, iov[i].iov_len);
+	}
+
+	return smbd_post_send_sgl(info, sgl, data_length, remaining_data_length);
+}
+
+/*
  * Post a receive request to the transport
  * The remote peer can only send data when a receive request is posted
  * The interaction is controlled by send/receive credit system
@@ -1657,6 +1714,9 @@ struct smbd_connection *_smbd_get_connection(
 	queue_delayed_work(info->workqueue, &info->idle_timer_work,
 		info->keep_alive_interval*HZ);
 
+	init_waitqueue_head(&info->wait_smbd_send_pending);
+	info->smbd_send_pending = 0;
+
 	init_waitqueue_head(&info->wait_smbd_recv_pending);
 	info->smbd_recv_pending = 0;
 
@@ -1948,3 +2008,189 @@ int smbd_recv(struct smbd_connection *info, struct msghdr *msg)
 		msg->msg_iter.count = 0;
 	return rc;
 }
+
+/*
+ * Send data to transport
+ * Each rqst is transported as a SMBDirect payload
+ * rqst: the data to write
+ * return value: 0 if successfully write, otherwise error code
+ */
+int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst)
+{
+	struct kvec vec;
+	int nvecs;
+	int size;
+	int buflen = 0, remaining_data_length;
+	int start, i, j;
+	int max_iov_size =
+		info->max_send_size - sizeof(struct smbd_data_transfer);
+	struct kvec iov[SMBDIRECT_MAX_SGE];
+	int rc;
+
+	info->smbd_send_pending++;
+	if (info->transport_status != SMBD_CONNECTED) {
+		rc = -ENODEV;
+		goto done;
+	}
+
+	/*
+	 * This usually means a configuration error
+	 * We use RDMA read/write for packet size > rdma_readwrite_threshold
+	 * as long as it's properly configured we should never get into this
+	 * situation
+	 */
+	if (rqst->rq_nvec + rqst->rq_npages > SMBDIRECT_MAX_SGE) {
+		log_write(ERR, "maximum send segment %x exceeding %x\n",
+			 rqst->rq_nvec + rqst->rq_npages, SMBDIRECT_MAX_SGE);
+		rc = -EINVAL;
+		goto done;
+	}
+
+	/*
+	 * Remove the RFC1002 length defined in MS-SMB2 section 2.1
+	 * It is used only for TCP transport
+	 * In future we may want to add a transport layer under protocol
+	 * layer so this will only be issued to TCP transport
+	 */
+	iov[0].iov_base = (char *)rqst->rq_iov[0].iov_base + 4;
+	iov[0].iov_len = rqst->rq_iov[0].iov_len - 4;
+	buflen += iov[0].iov_len;
+
+	/* total up iov array first */
+	for (i = 1; i < rqst->rq_nvec; i++) {
+		iov[i].iov_base = rqst->rq_iov[i].iov_base;
+		iov[i].iov_len = rqst->rq_iov[i].iov_len;
+		buflen += iov[i].iov_len;
+	}
+
+	/* add in the page array if there is one */
+	if (rqst->rq_npages) {
+		buflen += rqst->rq_pagesz * (rqst->rq_npages - 1);
+		buflen += rqst->rq_tailsz;
+	}
+
+	if (buflen + sizeof(struct smbd_data_transfer) >
+		info->max_fragmented_send_size) {
+		log_write(ERR, "payload size %d > max size %d\n",
+			buflen, info->max_fragmented_send_size);
+		rc = -EINVAL;
+		goto done;
+	}
+
+	remaining_data_length = buflen;
+
+	log_write(INFO, "rqst->rq_nvec=%d rqst->rq_npages=%d rq_pagesz=%d "
+		"rq_tailsz=%d buflen=%d\n",
+		rqst->rq_nvec, rqst->rq_npages, rqst->rq_pagesz,
+		rqst->rq_tailsz, buflen);
+
+	start = i = iov[0].iov_len ? 0 : 1;
+	buflen = 0;
+	while (true) {
+		buflen += iov[i].iov_len;
+		if (buflen > max_iov_size) {
+			if (i > start) {
+				remaining_data_length -=
+					(buflen-iov[i].iov_len);
+				log_write(INFO, "sending iov[] from start=%d "
+					"i=%d nvecs=%d "
+					"remaining_data_length=%d\n",
+					start, i, i-start,
+					remaining_data_length);
+				rc = smbd_post_send_data(
+					info, &iov[start], i-start,
+					remaining_data_length);
+				if (rc)
+					goto done;
+			} else {
+				/* iov[start] is too big, break it */
+				nvecs = (buflen+max_iov_size-1)/max_iov_size;
+				log_write(INFO, "iov[%d] iov_base=%p buflen=%d"
+					" break to %d vectors\n",
+					start, iov[start].iov_base,
+					buflen, nvecs);
+				for (j = 0; j < nvecs; j++) {
+					vec.iov_base =
+						(char *)iov[start].iov_base +
+						j*max_iov_size;
+					vec.iov_len = max_iov_size;
+					if (j == nvecs-1)
+						vec.iov_len =
+							buflen -
+							max_iov_size*(nvecs-1);
+					remaining_data_length -= vec.iov_len;
+					log_write(INFO,
+						"sending vec j=%d iov_base=%p"
+						" iov_len=%zu "
+						"remaining_data_length=%d\n",
+						j, vec.iov_base, vec.iov_len,
+						remaining_data_length);
+					rc = smbd_post_send_data(
+						info, &vec, 1,
+						remaining_data_length);
+					if (rc)
+						goto done;
+				}
+				i++;
+			}
+			start = i;
+			buflen = 0;
+		} else {
+			i++;
+			if (i == rqst->rq_nvec) {
+				/* send out all remaining vecs */
+				remaining_data_length -= buflen;
+				log_write(INFO,
+					"sending iov[] from start=%d i=%d "
+					"nvecs=%d remaining_data_length=%d\n",
+					start, i, i-start,
+					remaining_data_length);
+				rc = smbd_post_send_data(info, &iov[start],
+					i-start, remaining_data_length);
+				if (rc)
+					goto done;
+				break;
+			}
+		}
+		log_write(INFO, "looping i=%d buflen=%d\n", i, buflen);
+	}
+
+	/* now sending pages if there are any */
+	for (i = 0; i < rqst->rq_npages; i++) {
+		buflen = (i == rqst->rq_npages-1) ?
+			rqst->rq_tailsz : rqst->rq_pagesz;
+		nvecs = (buflen + max_iov_size - 1) / max_iov_size;
+		log_write(INFO, "sending pages buflen=%d nvecs=%d\n",
+			buflen, nvecs);
+		for (j = 0; j < nvecs; j++) {
+			size = max_iov_size;
+			if (j == nvecs-1)
+				size = buflen - j*max_iov_size;
+			remaining_data_length -= size;
+			log_write(INFO, "sending pages i=%d offset=%d size=%d"
+				" remaining_data_length=%d\n",
+				i, j*max_iov_size, size, remaining_data_length);
+			rc = smbd_post_send_page(
+				info, rqst->rq_pages[i], j*max_iov_size,
+				size, remaining_data_length);
+			if (rc)
+				goto done;
+		}
+	}
+
+done:
+	/*
+	 * As an optimization, we don't wait for individual I/O to finish
+	 * before sending the next one.
+	 * Send them all and wait for pending send count to get to 0
+	 * that means all the I/Os have been out and we are good to return
+	 */
+
+	wait_event(info->wait_send_payload_pending,
+		atomic_read(&info->send_payload_pending) == 0);
+
+	info->smbd_send_pending--;
+	wake_up(&info->wait_smbd_send_pending);
+
+	return rc;
+}
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
index 65a431f..e777d77 100644
--- a/fs/cifs/smbdirect.h
+++ b/fs/cifs/smbdirect.h
@@ -92,6 +92,9 @@ struct smbd_connection {
 
 	/* Activity accoutning */
 	/* Pending reqeusts issued from upper layer */
+	int smbd_send_pending;
+	wait_queue_head_t wait_smbd_send_pending;
+
 	int smbd_recv_pending;
 	wait_queue_head_t wait_smbd_recv_pending;
 
-- 
2.7.4

[Patch v7 17/22] CIFS: SMBD: Upper layer sends data via RDMA send

[Long Li]

From: Long Li <longli@microsoft.com>

With SMB Direct connected, use it for sending data via RDMA send.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/transport.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/fs/cifs/transport.c b/fs/cifs/transport.c
index 7efbab0..bb9ffdd 100644
--- a/fs/cifs/transport.c
+++ b/fs/cifs/transport.c
@@ -37,6 +37,7 @@
 #include "cifsglob.h"
 #include "cifsproto.h"
 #include "cifs_debug.h"
+#include "smbdirect.h"
 
 void
 cifs_wake_up_task(struct mid_q_entry *mid)
@@ -229,7 +230,10 @@ __smb_send_rqst(struct TCP_Server_Info *server, struct smb_rqst *rqst)
 	struct socket *ssocket = server->ssocket;
 	struct msghdr smb_msg;
 	int val = 1;
-
+	if (cifs_rdma_enabled(server) && server->smbd_conn) {
+		rc = smbd_send(server->smbd_conn, rqst);
+		goto smbd_done;
+	}
 	if (ssocket == NULL)
 		return -ENOTSOCK;
 
@@ -298,7 +302,7 @@ __smb_send_rqst(struct TCP_Server_Info *server, struct smb_rqst *rqst)
 		 */
 		server->tcpStatus = CifsNeedReconnect;
 	}
-
+smbd_done:
 	if (rc < 0 && rc != -EINTR)
 		cifs_dbg(VFS, "Error %d sending data on socket to server\n",
 			 rc);
-- 
2.7.4

[Patch v7 18/22] CIFS: SMBD: Implement RDMA memory registration

[Long Li]

From: Long Li <longli@microsoft.com>

Memory registration is used for transferring payload via RDMA read or write.
After I/O is done, memory registrations are recovered and reused. This
process can be time consuming and is done in a work queue.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/smbdirect.c | 421 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/cifs/smbdirect.h |  23 +++
 2 files changed, 444 insertions(+)

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index 0705f49..5c50c8a 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -49,6 +49,9 @@ static int smbd_post_send_page(struct smbd_connection *info,
 		struct page *page, unsigned long offset,
 		size_t size, int remaining_data_length);
 
+static void destroy_mr_list(struct smbd_connection *info);
+static int allocate_mr_list(struct smbd_connection *info);
+
 /* SMBD version number */
 #define SMBD_V1	0x0100
 
@@ -199,6 +202,12 @@ static void smbd_destroy_rdma_work(struct work_struct *work)
 	wait_event(info->wait_send_payload_pending,
 		atomic_read(&info->send_payload_pending) == 0);
 
+	log_rdma_event(INFO, "freeing mr list\n");
+	wake_up_interruptible_all(&info->wait_mr);
+	wait_event(info->wait_for_mr_cleanup,
+		atomic_read(&info->mr_used_count) == 0);
+	destroy_mr_list(info);
+
 	/* It's not posssible for upper layer to get to reassembly */
 	log_rdma_event(INFO, "drain the reassembly queue\n");
 	do {
@@ -455,6 +464,16 @@ static bool process_negotiation_response(
 	}
 	info->max_fragmented_send_size =
 		le32_to_cpu(packet->max_fragmented_size);
+	info->rdma_readwrite_threshold =
+		rdma_readwrite_threshold > info->max_fragmented_send_size ?
+		info->max_fragmented_send_size :
+		rdma_readwrite_threshold;
+
+
+	info->max_readwrite_size = min_t(u32,
+			le32_to_cpu(packet->max_readwrite_size),
+			info->max_frmr_depth * PAGE_SIZE);
+	info->max_frmr_depth = info->max_readwrite_size / PAGE_SIZE;
 
 	return true;
 }
@@ -750,6 +769,12 @@ static int smbd_ia_open(
 		rc = -EPROTONOSUPPORT;
 		goto out2;
 	}
+	info->max_frmr_depth = min_t(int,
+		smbd_max_frmr_depth,
+		info->id->device->attrs.max_fast_reg_page_list_len);
+	info->mr_type = IB_MR_TYPE_MEM_REG;
+	if (info->id->device->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG)
+		info->mr_type = IB_MR_TYPE_SG_GAPS;
 
 	info->pd = ib_alloc_pd(info->id->device, 0);
 	if (IS_ERR(info->pd)) {
@@ -1587,6 +1612,8 @@ struct smbd_connection *_smbd_get_connection(
 	struct rdma_conn_param conn_param;
 	struct ib_qp_init_attr qp_attr;
 	struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;
+	struct ib_port_immutable port_immutable;
+	u32 ird_ord_hdr[2];
 
 	info = kzalloc(sizeof(struct smbd_connection), GFP_KERNEL);
 	if (!info)
@@ -1675,6 +1702,28 @@ struct smbd_connection *_smbd_get_connection(
 	memset(&conn_param, 0, sizeof(conn_param));
 	conn_param.initiator_depth = 0;
 
+	conn_param.responder_resources =
+		info->id->device->attrs.max_qp_rd_atom
+			< SMBD_CM_RESPONDER_RESOURCES ?
+		info->id->device->attrs.max_qp_rd_atom :
+		SMBD_CM_RESPONDER_RESOURCES;
+	info->responder_resources = conn_param.responder_resources;
+	log_rdma_mr(INFO, "responder_resources=%d\n",
+		info->responder_resources);
+
+	/* Need to send IRD/ORD in private data for iWARP */
+	info->id->device->get_port_immutable(
+		info->id->device, info->id->port_num, &port_immutable);
+	if (port_immutable.core_cap_flags & RDMA_CORE_PORT_IWARP) {
+		ird_ord_hdr[0] = info->responder_resources;
+		ird_ord_hdr[1] = 1;
+		conn_param.private_data = ird_ord_hdr;
+		conn_param.private_data_len = sizeof(ird_ord_hdr);
+	} else {
+		conn_param.private_data = NULL;
+		conn_param.private_data_len = 0;
+	}
+
 	conn_param.retry_count = SMBD_CM_RETRY;
 	conn_param.rnr_retry_count = SMBD_CM_RNR_RETRY;
 	conn_param.flow_control = 0;
@@ -1739,8 +1788,19 @@ struct smbd_connection *_smbd_get_connection(
 		goto negotiation_failed;
 	}
 
+	rc = allocate_mr_list(info);
+	if (rc) {
+		log_rdma_mr(ERR, "memory registration allocation failed\n");
+		goto allocate_mr_failed;
+	}
+
 	return info;
 
+allocate_mr_failed:
+	/* At this point, need to a full transport shutdown */
+	smbd_destroy(info);
+	return NULL;
+
 negotiation_failed:
 	cancel_delayed_work_sync(&info->idle_timer_work);
 	destroy_caches_and_workqueue(info);
@@ -2194,3 +2254,364 @@ int smbd_send(struct smbd_connection *info, struct smb_rqst *rqst)
 
 	return rc;
 }
+
+static void register_mr_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+	struct smbd_mr *mr;
+	struct ib_cqe *cqe;
+
+	if (wc->status) {
+		log_rdma_mr(ERR, "status=%d\n", wc->status);
+		cqe = wc->wr_cqe;
+		mr = container_of(cqe, struct smbd_mr, cqe);
+		smbd_disconnect_rdma_connection(mr->conn);
+	}
+}
+
+/*
+ * The work queue function that recovers MRs
+ * We need to call ib_dereg_mr() and ib_alloc_mr() before this MR can be used
+ * again. Both calls are slow, so finish them in a workqueue. This will not
+ * block I/O path.
+ * There is one workqueue that recovers MRs, there is no need to lock as the
+ * I/O requests calling smbd_register_mr will never update the links in the
+ * mr_list.
+ */
+static void smbd_mr_recovery_work(struct work_struct *work)
+{
+	struct smbd_connection *info =
+		container_of(work, struct smbd_connection, mr_recovery_work);
+	struct smbd_mr *smbdirect_mr;
+	int rc;
+
+	list_for_each_entry(smbdirect_mr, &info->mr_list, list) {
+		if (smbdirect_mr->state == MR_INVALIDATED ||
+			smbdirect_mr->state == MR_ERROR) {
+
+			if (smbdirect_mr->state == MR_INVALIDATED) {
+				ib_dma_unmap_sg(
+					info->id->device, smbdirect_mr->sgl,
+					smbdirect_mr->sgl_count,
+					smbdirect_mr->dir);
+				smbdirect_mr->state = MR_READY;
+			} else if (smbdirect_mr->state == MR_ERROR) {
+
+				/* recover this MR entry */
+				rc = ib_dereg_mr(smbdirect_mr->mr);
+				if (rc) {
+					log_rdma_mr(ERR,
+						"ib_dereg_mr faield rc=%x\n",
+						rc);
+					smbd_disconnect_rdma_connection(info);
+				}
+
+				smbdirect_mr->mr = ib_alloc_mr(
+					info->pd, info->mr_type,
+					info->max_frmr_depth);
+				if (IS_ERR(smbdirect_mr->mr)) {
+					log_rdma_mr(ERR,
+						"ib_alloc_mr failed mr_type=%x "
+						"max_frmr_depth=%x\n",
+						info->mr_type,
+						info->max_frmr_depth);
+					smbd_disconnect_rdma_connection(info);
+				}
+
+				smbdirect_mr->state = MR_READY;
+			}
+			/* smbdirect_mr->state is updated by this function
+			 * and is read and updated by I/O issuing CPUs trying
+			 * to get a MR, the call to atomic_inc_return
+			 * implicates a memory barrier and guarantees this
+			 * value is updated before waking up any calls to
+			 * get_mr() from the I/O issuing CPUs
+			 */
+			if (atomic_inc_return(&info->mr_ready_count) == 1)
+				wake_up_interruptible(&info->wait_mr);
+		}
+	}
+}
+
+static void destroy_mr_list(struct smbd_connection *info)
+{
+	struct smbd_mr *mr, *tmp;
+
+	cancel_work_sync(&info->mr_recovery_work);
+	list_for_each_entry_safe(mr, tmp, &info->mr_list, list) {
+		if (mr->state == MR_INVALIDATED)
+			ib_dma_unmap_sg(info->id->device, mr->sgl,
+				mr->sgl_count, mr->dir);
+		ib_dereg_mr(mr->mr);
+		kfree(mr->sgl);
+		kfree(mr);
+	}
+}
+
+/*
+ * Allocate MRs used for RDMA read/write
+ * The number of MRs will not exceed hardware capability in responder_resources
+ * All MRs are kept in mr_list. The MR can be recovered after it's used
+ * Recovery is done in smbd_mr_recovery_work. The content of list entry changes
+ * as MRs are used and recovered for I/O, but the list links will not change
+ */
+static int allocate_mr_list(struct smbd_connection *info)
+{
+	int i;
+	struct smbd_mr *smbdirect_mr, *tmp;
+
+	INIT_LIST_HEAD(&info->mr_list);
+	init_waitqueue_head(&info->wait_mr);
+	spin_lock_init(&info->mr_list_lock);
+	atomic_set(&info->mr_ready_count, 0);
+	atomic_set(&info->mr_used_count, 0);
+	init_waitqueue_head(&info->wait_for_mr_cleanup);
+	/* Allocate more MRs (2x) than hardware responder_resources */
+	for (i = 0; i < info->responder_resources * 2; i++) {
+		smbdirect_mr = kzalloc(sizeof(*smbdirect_mr), GFP_KERNEL);
+		if (!smbdirect_mr)
+			goto out;
+		smbdirect_mr->mr = ib_alloc_mr(info->pd, info->mr_type,
+					info->max_frmr_depth);
+		if (IS_ERR(smbdirect_mr->mr)) {
+			log_rdma_mr(ERR, "ib_alloc_mr failed mr_type=%x "
+				"max_frmr_depth=%x\n",
+				info->mr_type, info->max_frmr_depth);
+			goto out;
+		}
+		smbdirect_mr->sgl = kcalloc(
+					info->max_frmr_depth,
+					sizeof(struct scatterlist),
+					GFP_KERNEL);
+		if (!smbdirect_mr->sgl) {
+			log_rdma_mr(ERR, "failed to allocate sgl\n");
+			ib_dereg_mr(smbdirect_mr->mr);
+			goto out;
+		}
+		smbdirect_mr->state = MR_READY;
+		smbdirect_mr->conn = info;
+
+		list_add_tail(&smbdirect_mr->list, &info->mr_list);
+		atomic_inc(&info->mr_ready_count);
+	}
+	INIT_WORK(&info->mr_recovery_work, smbd_mr_recovery_work);
+	return 0;
+
+out:
+	kfree(smbdirect_mr);
+
+	list_for_each_entry_safe(smbdirect_mr, tmp, &info->mr_list, list) {
+		ib_dereg_mr(smbdirect_mr->mr);
+		kfree(smbdirect_mr->sgl);
+		kfree(smbdirect_mr);
+	}
+	return -ENOMEM;
+}
+
+/*
+ * Get a MR from mr_list. This function waits until there is at least one
+ * MR available in the list. It may access the list while the
+ * smbd_mr_recovery_work is recovering the MR list. This doesn't need a lock
+ * as they never modify the same places. However, there may be several CPUs
+ * issueing I/O trying to get MR at the same time, mr_list_lock is used to
+ * protect this situation.
+ */
+static struct smbd_mr *get_mr(struct smbd_connection *info)
+{
+	struct smbd_mr *ret;
+	int rc;
+again:
+	rc = wait_event_interruptible(info->wait_mr,
+		atomic_read(&info->mr_ready_count) ||
+		info->transport_status != SMBD_CONNECTED);
+	if (rc) {
+		log_rdma_mr(ERR, "wait_event_interruptible rc=%x\n", rc);
+		return NULL;
+	}
+
+	if (info->transport_status != SMBD_CONNECTED) {
+		log_rdma_mr(ERR, "info->transport_status=%x\n",
+			info->transport_status);
+		return NULL;
+	}
+
+	spin_lock(&info->mr_list_lock);
+	list_for_each_entry(ret, &info->mr_list, list) {
+		if (ret->state == MR_READY) {
+			ret->state = MR_REGISTERED;
+			spin_unlock(&info->mr_list_lock);
+			atomic_dec(&info->mr_ready_count);
+			atomic_inc(&info->mr_used_count);
+			return ret;
+		}
+	}
+
+	spin_unlock(&info->mr_list_lock);
+	/*
+	 * It is possible that we can get a MR because other processes may try
+	 * to acquire a MR at the same time. If this is the case, retry it.
+	 */
+	goto again;
+}
+
+/*
+ * Register memory for RDMA read/write
+ * pages[]: the list of pages to register memory with
+ * num_pages: the number of pages to register
+ * tailsz: if non-zero, the bytes to register in the last page
+ * writing: true if this is a RDMA write (SMB read), false for RDMA read
+ * need_invalidate: true if this MR needs to be locally invalidated after I/O
+ * return value: the MR registered, NULL if failed.
+ */
+struct smbd_mr *smbd_register_mr(
+	struct smbd_connection *info, struct page *pages[], int num_pages,
+	int tailsz, bool writing, bool need_invalidate)
+{
+	struct smbd_mr *smbdirect_mr;
+	int rc, i;
+	enum dma_data_direction dir;
+	struct ib_reg_wr *reg_wr;
+	struct ib_send_wr *bad_wr;
+
+	if (num_pages > info->max_frmr_depth) {
+		log_rdma_mr(ERR, "num_pages=%d max_frmr_depth=%d\n",
+			num_pages, info->max_frmr_depth);
+		return NULL;
+	}
+
+	smbdirect_mr = get_mr(info);
+	if (!smbdirect_mr) {
+		log_rdma_mr(ERR, "get_mr returning NULL\n");
+		return NULL;
+	}
+	smbdirect_mr->need_invalidate = need_invalidate;
+	smbdirect_mr->sgl_count = num_pages;
+	sg_init_table(smbdirect_mr->sgl, num_pages);
+
+	for (i = 0; i < num_pages - 1; i++)
+		sg_set_page(&smbdirect_mr->sgl[i], pages[i], PAGE_SIZE, 0);
+
+	sg_set_page(&smbdirect_mr->sgl[i], pages[i],
+		tailsz ? tailsz : PAGE_SIZE, 0);
+
+	dir = writing ? DMA_FROM_DEVICE : DMA_TO_DEVICE;
+	smbdirect_mr->dir = dir;
+	rc = ib_dma_map_sg(info->id->device, smbdirect_mr->sgl, num_pages, dir);
+	if (!rc) {
+		log_rdma_mr(INFO, "ib_dma_map_sg num_pages=%x dir=%x rc=%x\n",
+			num_pages, dir, rc);
+		goto dma_map_error;
+	}
+
+	rc = ib_map_mr_sg(smbdirect_mr->mr, smbdirect_mr->sgl, num_pages,
+		NULL, PAGE_SIZE);
+	if (rc != num_pages) {
+		log_rdma_mr(INFO,
+			"ib_map_mr_sg failed rc = %x num_pages = %x\n",
+			rc, num_pages);
+		goto map_mr_error;
+	}
+
+	ib_update_fast_reg_key(smbdirect_mr->mr,
+		ib_inc_rkey(smbdirect_mr->mr->rkey));
+	reg_wr = &smbdirect_mr->wr;
+	reg_wr->wr.opcode = IB_WR_REG_MR;
+	smbdirect_mr->cqe.done = register_mr_done;
+	reg_wr->wr.wr_cqe = &smbdirect_mr->cqe;
+	reg_wr->wr.num_sge = 0;
+	reg_wr->wr.send_flags = IB_SEND_SIGNALED;
+	reg_wr->mr = smbdirect_mr->mr;
+	reg_wr->key = smbdirect_mr->mr->rkey;
+	reg_wr->access = writing ?
+			IB_ACCESS_REMOTE_WRITE | IB_ACCESS_LOCAL_WRITE :
+			IB_ACCESS_REMOTE_READ;
+
+	/*
+	 * There is no need for waiting for complemtion on ib_post_send
+	 * on IB_WR_REG_MR. Hardware enforces a barrier and order of execution
+	 * on the next ib_post_send when we actaully send I/O to remote peer
+	 */
+	rc = ib_post_send(info->id->qp, &reg_wr->wr, &bad_wr);
+	if (!rc)
+		return smbdirect_mr;
+
+	log_rdma_mr(ERR, "ib_post_send failed rc=%x reg_wr->key=%x\n",
+		rc, reg_wr->key);
+
+	/* If all failed, attempt to recover this MR by setting it MR_ERROR*/
+map_mr_error:
+	ib_dma_unmap_sg(info->id->device, smbdirect_mr->sgl,
+		smbdirect_mr->sgl_count, smbdirect_mr->dir);
+
+dma_map_error:
+	smbdirect_mr->state = MR_ERROR;
+	if (atomic_dec_and_test(&info->mr_used_count))
+		wake_up(&info->wait_for_mr_cleanup);
+
+	return NULL;
+}
+
+static void local_inv_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+	struct smbd_mr *smbdirect_mr;
+	struct ib_cqe *cqe;
+
+	cqe = wc->wr_cqe;
+	smbdirect_mr = container_of(cqe, struct smbd_mr, cqe);
+	smbdirect_mr->state = MR_INVALIDATED;
+	if (wc->status != IB_WC_SUCCESS) {
+		log_rdma_mr(ERR, "invalidate failed status=%x\n", wc->status);
+		smbdirect_mr->state = MR_ERROR;
+	}
+	complete(&smbdirect_mr->invalidate_done);
+}
+
+/*
+ * Deregister a MR after I/O is done
+ * This function may wait if remote invalidation is not used
+ * and we have to locally invalidate the buffer to prevent data is being
+ * modified by remote peer after upper layer consumes it
+ */
+int smbd_deregister_mr(struct smbd_mr *smbdirect_mr)
+{
+	struct ib_send_wr *wr, *bad_wr;
+	struct smbd_connection *info = smbdirect_mr->conn;
+	int rc = 0;
+
+	if (smbdirect_mr->need_invalidate) {
+		/* Need to finish local invalidation before returning */
+		wr = &smbdirect_mr->inv_wr;
+		wr->opcode = IB_WR_LOCAL_INV;
+		smbdirect_mr->cqe.done = local_inv_done;
+		wr->wr_cqe = &smbdirect_mr->cqe;
+		wr->num_sge = 0;
+		wr->ex.invalidate_rkey = smbdirect_mr->mr->rkey;
+		wr->send_flags = IB_SEND_SIGNALED;
+
+		init_completion(&smbdirect_mr->invalidate_done);
+		rc = ib_post_send(info->id->qp, wr, &bad_wr);
+		if (rc) {
+			log_rdma_mr(ERR, "ib_post_send failed rc=%x\n", rc);
+			smbd_disconnect_rdma_connection(info);
+			goto done;
+		}
+		wait_for_completion(&smbdirect_mr->invalidate_done);
+		smbdirect_mr->need_invalidate = false;
+	} else
+		/*
+		 * For remote invalidation, just set it to MR_INVALIDATED
+		 * and defer to mr_recovery_work to recover the MR for next use
+		 */
+		smbdirect_mr->state = MR_INVALIDATED;
+
+	/*
+	 * Schedule the work to do MR recovery for future I/Os
+	 * MR recovery is slow and we don't want it to block the current I/O
+	 */
+	queue_work(info->workqueue, &info->mr_recovery_work);
+
+done:
+	if (atomic_dec_and_test(&info->mr_used_count))
+		wake_up(&info->wait_for_mr_cleanup);
+
+	return rc;
+}
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
index e777d77..42c61d3 100644
--- a/fs/cifs/smbdirect.h
+++ b/fs/cifs/smbdirect.h
@@ -90,6 +90,29 @@ struct smbd_connection {
 	int receive_credit_target;
 	int fragment_reassembly_remaining;
 
+	/* Memory registrations */
+	/* Maximum number of RDMA read/write outstanding on this connection */
+	int responder_resources;
+	/* Maximum number of SGEs in a RDMA write/read */
+	int max_frmr_depth;
+	/*
+	 * If payload is less than or equal to the threshold,
+	 * use RDMA send/recv to send upper layer I/O.
+	 * If payload is more than the threshold,
+	 * use RDMA read/write through memory registration for I/O.
+	 */
+	int rdma_readwrite_threshold;
+	enum ib_mr_type mr_type;
+	struct list_head mr_list;
+	spinlock_t mr_list_lock;
+	/* The number of available MRs ready for memory registration */
+	atomic_t mr_ready_count;
+	atomic_t mr_used_count;
+	wait_queue_head_t wait_mr;
+	struct work_struct mr_recovery_work;
+	/* Used by transport to wait until all MRs are returned */
+	wait_queue_head_t wait_for_mr_cleanup;
+
 	/* Activity accoutning */
 	/* Pending reqeusts issued from upper layer */
 	int smbd_send_pending;
-- 
2.7.4

[Patch v7 19/22] CIFS: SMBD: Upper layer performs SMB write via RDMA read through memory registration

[Long Li]

From: Long Li <longli@microsoft.com>

When sending I/O, if size is larger than rdma_readwrite_threshold we prepare
to send SMB write packet for a RDMA read via memory registration. The actual
I/O is done by remote peer through local RDMA hardware. Modify the relevant
fields in the packet accordingly, and append a smbd_buffer_descriptor_v1 to
the end of the SMB write packet.

On write I/O finish, deregister the memory region if this was for a RDMA read.
If remote invalidation is not used, the call to smbd_deregister_mr will do
local invalidation and possibly wait. Memory region is normally deregistered
in MID callback as soon as it's used. There are situations where the MID may
not be created on I/O failure, under which memory region is deregistered when
write data context is released.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/cifsglob.h |  3 +++
 fs/cifs/cifssmb.c  |  7 ++++++
 fs/cifs/smb2pdu.c  | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 72 insertions(+), 4 deletions(-)

diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
index 5585516..66f210d 100644
--- a/fs/cifs/cifsglob.h
+++ b/fs/cifs/cifsglob.h
@@ -1168,6 +1168,9 @@ struct cifs_writedata {
 	pid_t				pid;
 	unsigned int			bytes;
 	int				result;
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	struct smbd_mr			*mr;
+#endif
 	unsigned int			pagesz;
 	unsigned int			tailsz;
 	unsigned int			credits;
diff --git a/fs/cifs/cifssmb.c b/fs/cifs/cifssmb.c
index 5857009..e012e3f 100644
--- a/fs/cifs/cifssmb.c
+++ b/fs/cifs/cifssmb.c
@@ -43,6 +43,7 @@
 #include "cifs_unicode.h"
 #include "cifs_debug.h"
 #include "fscache.h"
+#include "smbdirect.h"
 
 #ifdef CONFIG_CIFS_POSIX
 static struct {
@@ -1911,6 +1912,12 @@ cifs_writedata_release(struct kref *refcount)
 {
 	struct cifs_writedata *wdata = container_of(refcount,
 					struct cifs_writedata, refcount);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	if (wdata->mr) {
+		smbd_deregister_mr(wdata->mr);
+		wdata->mr = NULL;
+	}
+#endif
 
 	if (wdata->cfile)
 		cifsFileInfo_put(wdata->cfile);
diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index 32ad590..c8afb83 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -48,6 +48,7 @@
 #include "smb2glob.h"
 #include "cifspdu.h"
 #include "cifs_spnego.h"
+#include "smbdirect.h"
 
 /*
  *  The following table defines the expected "StructureSize" of SMB2 requests
@@ -2656,7 +2657,19 @@ smb2_writev_callback(struct mid_q_entry *mid)
 		wdata->result = -EIO;
 		break;
 	}
-
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	/*
+	 * If this wdata has a memory registered, the MR can be freed
+	 * The number of MRs available is limited, it's important to recover
+	 * used MR as soon as I/O is finished. Hold MR longer in the later
+	 * I/O process can possibly result in I/O deadlock due to lack of MR
+	 * to send request on I/O retry
+	 */
+	if (wdata->mr) {
+		smbd_deregister_mr(wdata->mr);
+		wdata->mr = NULL;
+	}
+#endif
 	if (wdata->result)
 		cifs_stats_fail_inc(tcon, SMB2_WRITE_HE);
 
@@ -2707,7 +2720,42 @@ smb2_async_writev(struct cifs_writedata *wdata,
 	req->DataOffset = cpu_to_le16(
 				offsetof(struct smb2_write_req, Buffer) - 4);
 	req->RemainingBytes = 0;
-
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	/*
+	 * If we want to do a server RDMA read, fill in and append
+	 * smbd_buffer_descriptor_v1 to the end of write request
+	 */
+	if (server->rdma && wdata->bytes >=
+		server->smbd_conn->rdma_readwrite_threshold) {
+
+		struct smbd_buffer_descriptor_v1 *v1;
+		bool need_invalidate = server->dialect == SMB30_PROT_ID;
+
+		wdata->mr = smbd_register_mr(
+				server->smbd_conn, wdata->pages,
+				wdata->nr_pages, wdata->tailsz,
+				false, need_invalidate);
+		if (!wdata->mr) {
+			rc = -ENOBUFS;
+			goto async_writev_out;
+		}
+		req->Length = 0;
+		req->DataOffset = 0;
+		req->RemainingBytes =
+			(wdata->nr_pages-1)*PAGE_SIZE + wdata->tailsz;
+		req->Channel = SMB2_CHANNEL_RDMA_V1_INVALIDATE;
+		if (need_invalidate)
+			req->Channel = SMB2_CHANNEL_RDMA_V1;
+		req->WriteChannelInfoOffset =
+			offsetof(struct smb2_write_req, Buffer) - 4;
+		req->WriteChannelInfoLength =
+			sizeof(struct smbd_buffer_descriptor_v1);
+		v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0];
+		v1->offset = wdata->mr->mr->iova;
+		v1->token = wdata->mr->mr->rkey;
+		v1->length = wdata->mr->mr->length;
+	}
+#endif
 	/* 4 for rfc1002 length field and 1 for Buffer */
 	iov[0].iov_len = 4;
 	iov[0].iov_base = req;
@@ -2720,12 +2768,22 @@ smb2_async_writev(struct cifs_writedata *wdata,
 	rqst.rq_npages = wdata->nr_pages;
 	rqst.rq_pagesz = wdata->pagesz;
 	rqst.rq_tailsz = wdata->tailsz;
-
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	if (wdata->mr) {
+		iov[1].iov_len += sizeof(struct smbd_buffer_descriptor_v1);
+		rqst.rq_npages = 0;
+	}
+#endif
 	cifs_dbg(FYI, "async write at %llu %u bytes\n",
 		 wdata->offset, wdata->bytes);
 
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	/* For RDMA read, I/O size is in RemainingBytes not in Length */
+	if (!wdata->mr)
+		req->Length = cpu_to_le32(wdata->bytes);
+#else
 	req->Length = cpu_to_le32(wdata->bytes);
-
+#endif
 	inc_rfc1001_len(&req->hdr, wdata->bytes - 1 /* Buffer */);
 
 	if (wdata->credits) {
-- 
2.7.4

[Patch v7 20/22] CIFS: SMBD: Read correct returned data length for RDMA write (SMB read) I/O

[Long Li]

From: Long Li <longli@microsoft.com>

This patch is for preparing upper layer doing SMB read via RDMA write.

When RDMA write is used for SMB read, the returned data length is in
DataRemaining in the response packet. Reading it properly by adding a
parameter to specifiy where the returned data length is.

Add the defition for memory registration to wdata and return the correct
length based on if RDMA write is used.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/cifsglob.h | 13 +++++++++++--
 fs/cifs/cifssmb.c  |  8 ++++++--
 fs/cifs/smb1ops.c  |  4 +++-
 fs/cifs/smb2ops.c  | 12 ++++++++++--
 4 files changed, 30 insertions(+), 7 deletions(-)

diff --git a/fs/cifs/cifsglob.h b/fs/cifs/cifsglob.h
index 66f210d..ab62755 100644
--- a/fs/cifs/cifsglob.h
+++ b/fs/cifs/cifsglob.h
@@ -228,8 +228,14 @@ struct smb_version_operations {
 	__u64 (*get_next_mid)(struct TCP_Server_Info *);
 	/* data offset from read response message */
 	unsigned int (*read_data_offset)(char *);
-	/* data length from read response message */
-	unsigned int (*read_data_length)(char *);
+	/*
+	 * Data length from read response message
+	 * When in_remaining is true, the returned data length is in
+	 * message field DataRemaining for out-of-band data read (e.g through
+	 * Memory Registration RDMA write in SMBD).
+	 * Otherwise, the returned data length is in message field DataLength.
+	 */
+	unsigned int (*read_data_length)(char *, bool in_remaining);
 	/* map smb to linux error */
 	int (*map_error)(char *, bool);
 	/* find mid corresponding to the response message */
@@ -1146,6 +1152,9 @@ struct cifs_readdata {
 				struct cifs_readdata *rdata,
 				struct iov_iter *iter);
 	struct kvec			iov[2];
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	struct smbd_mr			*mr;
+#endif
 	unsigned int			pagesz;
 	unsigned int			tailsz;
 	unsigned int			credits;
diff --git a/fs/cifs/cifssmb.c b/fs/cifs/cifssmb.c
index e012e3f..494ebb4 100644
--- a/fs/cifs/cifssmb.c
+++ b/fs/cifs/cifssmb.c
@@ -1443,6 +1443,7 @@ cifs_readv_receive(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 	struct cifs_readdata *rdata = mid->callback_data;
 	char *buf = server->smallbuf;
 	unsigned int buflen = get_rfc1002_length(buf) + 4;
+	bool use_rdma_mr = false;
 
 	cifs_dbg(FYI, "%s: mid=%llu offset=%llu bytes=%u\n",
 		 __func__, mid->mid, rdata->offset, rdata->bytes);
@@ -1531,8 +1532,11 @@ cifs_readv_receive(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 		 rdata->iov[0].iov_base, server->total_read);
 
 	/* how much data is in the response? */
-	data_len = server->ops->read_data_length(buf);
-	if (data_offset + data_len > buflen) {
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	use_rdma_mr = rdata->mr;
+#endif
+	data_len = server->ops->read_data_length(buf, use_rdma_mr);
+	if (!use_rdma_mr && (data_offset + data_len > buflen)) {
 		/* data_len is corrupt -- discard frame */
 		rdata->result = -EIO;
 		return cifs_readv_discard(server, mid);
diff --git a/fs/cifs/smb1ops.c b/fs/cifs/smb1ops.c
index a723df3..3d495e4 100644
--- a/fs/cifs/smb1ops.c
+++ b/fs/cifs/smb1ops.c
@@ -87,9 +87,11 @@ cifs_read_data_offset(char *buf)
 }
 
 static unsigned int
-cifs_read_data_length(char *buf)
+cifs_read_data_length(char *buf, bool in_remaining)
 {
 	READ_RSP *rsp = (READ_RSP *)buf;
+	/* It's a bug reading remaining data for SMB1 packets */
+	WARN_ON(in_remaining);
 	return (le16_to_cpu(rsp->DataLengthHigh) << 16) +
 	       le16_to_cpu(rsp->DataLength);
 }
diff --git a/fs/cifs/smb2ops.c b/fs/cifs/smb2ops.c
index abc8bd5..60273028 100644
--- a/fs/cifs/smb2ops.c
+++ b/fs/cifs/smb2ops.c
@@ -936,9 +936,13 @@ smb2_read_data_offset(char *buf)
 }
 
 static unsigned int
-smb2_read_data_length(char *buf)
+smb2_read_data_length(char *buf, bool in_remaining)
 {
 	struct smb2_read_rsp *rsp = (struct smb2_read_rsp *)buf;
+
+	if (in_remaining)
+		return le32_to_cpu(rsp->DataRemaining);
+
 	return le32_to_cpu(rsp->DataLength);
 }
 
@@ -2421,6 +2425,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 	struct iov_iter iter;
 	struct kvec iov;
 	int length;
+	bool use_rdma_mr = false;
 
 	if (shdr->Command != SMB2_READ) {
 		cifs_dbg(VFS, "only big read responses are supported\n");
@@ -2447,7 +2452,10 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 	}
 
 	data_offset = server->ops->read_data_offset(buf) + 4;
-	data_len = server->ops->read_data_length(buf);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	use_rdma_mr = rdata->mr;
+#endif
+	data_len = server->ops->read_data_length(buf, use_rdma_mr);
 
 	if (data_offset < server->vals->read_rsp_size) {
 		/*
-- 
2.7.4

[Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via RDMA write through memory registration

[Long Li]

From: Long Li <longli@microsoft.com>

If I/O size is larger than rdma_readwrite_threshold, use RDMA write for
SMB read by specifying channel SMB2_CHANNEL_RDMA_V1 or
SMB2_CHANNEL_RDMA_V1_INVALIDATE in the SMB packet, depending on SMB dialect
used. Append a smbd_buffer_descriptor_v1 to the end of the SMB packet and fill
in other values to indicate this SMB read uses RDMA write.

There is no need to read from the transport for incoming payload. At the time
SMB read response comes back, the data is already transfered and placed in the
pages by RDMA hardware.

When SMB read is finished, deregister the memory regions if RDMA write is used
for this SMB read. smbd_deregister_mr may need to do local invalidation and
sleep, if server remote invalidation is not used.

There are situations where the MID may not be created on I/O failure, under
which memory region is deregistered when read data context is released.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/file.c    | 17 +++++++++++++++--
 fs/cifs/smb2pdu.c | 45 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 59 insertions(+), 3 deletions(-)

diff --git a/fs/cifs/file.c b/fs/cifs/file.c
index 0786f19..464776a 100644
--- a/fs/cifs/file.c
+++ b/fs/cifs/file.c
@@ -42,7 +42,7 @@
 #include "cifs_debug.h"
 #include "cifs_fs_sb.h"
 #include "fscache.h"
-
+#include "smbdirect.h"
 
 static inline int cifs_convert_flags(unsigned int flags)
 {
@@ -2908,7 +2908,12 @@ cifs_readdata_release(struct kref *refcount)
 {
 	struct cifs_readdata *rdata = container_of(refcount,
 					struct cifs_readdata, refcount);
-
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	if (rdata->mr) {
+		smbd_deregister_mr(rdata->mr);
+		rdata->mr = NULL;
+	}
+#endif
 	if (rdata->cfile)
 		cifsFileInfo_put(rdata->cfile);
 
@@ -3037,6 +3042,10 @@ uncached_fill_pages(struct TCP_Server_Info *server,
 		}
 		if (iter)
 			result = copy_page_from_iter(page, 0, n, iter);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+		else if (rdata->mr)
+			result = n;
+#endif
 		else
 			result = cifs_read_page_from_socket(server, page, n);
 		if (result < 0)
@@ -3606,6 +3615,10 @@ readpages_fill_pages(struct TCP_Server_Info *server,
 
 		if (iter)
 			result = copy_page_from_iter(page, 0, n, iter);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+		else if (rdata->mr)
+			result = n;
+#endif
 		else
 			result = cifs_read_page_from_socket(server, page, n);
 		if (result < 0)
diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
index c8afb83..8a5ff90 100644
--- a/fs/cifs/smb2pdu.c
+++ b/fs/cifs/smb2pdu.c
@@ -2379,7 +2379,40 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
 	req->MinimumCount = 0;
 	req->Length = cpu_to_le32(io_parms->length);
 	req->Offset = cpu_to_le64(io_parms->offset);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	/*
+	 * If we want to do a RDMA write, fill in and append
+	 * smbd_buffer_descriptor_v1 to the end of read request
+	 */
+	if (server->rdma && rdata &&
+		rdata->bytes >= server->smbd_conn->rdma_readwrite_threshold) {
+
+		struct smbd_buffer_descriptor_v1 *v1;
+		bool need_invalidate =
+			io_parms->tcon->ses->server->dialect == SMB30_PROT_ID;
+
+		rdata->mr = smbd_register_mr(
+				server->smbd_conn, rdata->pages,
+				rdata->nr_pages, rdata->tailsz,
+				true, need_invalidate);
+		if (!rdata->mr)
+			return -ENOBUFS;
+
+		req->Channel = SMB2_CHANNEL_RDMA_V1_INVALIDATE;
+		if (need_invalidate)
+			req->Channel = SMB2_CHANNEL_RDMA_V1;
+		req->ReadChannelInfoOffset =
+			offsetof(struct smb2_read_plain_req, Buffer);
+		req->ReadChannelInfoLength =
+			sizeof(struct smbd_buffer_descriptor_v1);
+		v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0];
+		v1->offset = rdata->mr->mr->iova;
+		v1->token = rdata->mr->mr->rkey;
+		v1->length = rdata->mr->mr->length;
 
+		*total_len += sizeof(*v1) - 1;
+	}
+#endif
 	if (request_type & CHAINED_REQUEST) {
 		if (!(request_type & END_OF_CHAIN)) {
 			/* next 8-byte aligned request */
@@ -2458,7 +2491,17 @@ smb2_readv_callback(struct mid_q_entry *mid)
 		if (rdata->result != -ENODATA)
 			rdata->result = -EIO;
 	}
-
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	/*
+	 * If this rdata has a memmory registered, the MR can be freed
+	 * MR needs to be freed as soon as I/O finishes to prevent deadlock
+	 * because they have limited number and are used for future I/Os
+	 */
+	if (rdata->mr) {
+		smbd_deregister_mr(rdata->mr);
+		rdata->mr = NULL;
+	}
+#endif
 	if (rdata->result)
 		cifs_stats_fail_inc(tcon, SMB2_READ_HE);
 
-- 
2.7.4

Re: [Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via RDMA write through memory registration

[Tom Talpey]

Replying to a very old message, but it's something we
discussed today at the IOLab event so to capture it:

On 11/7/2017 12:55 AM, Long Li wrote:
> From: Long Li <longli@microsoft.com>
> 
> ---
>   fs/cifs/file.c    | 17 +++++++++++++++--
>   fs/cifs/smb2pdu.c | 45 ++++++++++++++++++++++++++++++++++++++++++++-
>   2 files changed, 59 insertions(+), 3 deletions(-)
> ...
> diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c
> index c8afb83..8a5ff90 100644
> --- a/fs/cifs/smb2pdu.c
> +++ b/fs/cifs/smb2pdu.c
> @@ -2379,7 +2379,40 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
>   	req->MinimumCount = 0;
>   	req->Length = cpu_to_le32(io_parms->length);
>   	req->Offset = cpu_to_le64(io_parms->offset);
> +#ifdef CONFIG_CIFS_SMB_DIRECT
> +	/*
> +	 * If we want to do a RDMA write, fill in and append
> +	 * smbd_buffer_descriptor_v1 to the end of read request
> +	 */
> +	if (server->rdma && rdata &&
> +		rdata->bytes >= server->smbd_conn->rdma_readwrite_threshold) {
> +
> +		struct smbd_buffer_descriptor_v1 *v1;
> +		bool need_invalidate =
> +			io_parms->tcon->ses->server->dialect == SMB30_PROT_ID;
> +
> +		rdata->mr = smbd_register_mr(
> +				server->smbd_conn, rdata->pages,
> +				rdata->nr_pages, rdata->tailsz,
> +				true, need_invalidate);
> +		if (!rdata->mr)
> +			return -ENOBUFS;
> +
> +		req->Channel = SMB2_CHANNEL_RDMA_V1_INVALIDATE;
> +		if (need_invalidate)
> +			req->Channel = SMB2_CHANNEL_RDMA_V1;
> +		req->ReadChannelInfoOffset =
> +			offsetof(struct smb2_read_plain_req, Buffer);
> +		req->ReadChannelInfoLength =
> +			sizeof(struct smbd_buffer_descriptor_v1);
> +		v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0];
> +		v1->offset = rdata->mr->mr->iova;

It's unnecessary, and possibly leaking kernel information, to use
the IOVA as the offset of a memory region which is registered using
an FRWR. Because such regions are based on the exact bytes targeted
by the memory handle, the offset can be set to any value, typically
zero, but nearly arbitrary. As long as the (offset + length) does
not wrap or otherwise overflow, offset can be set to anything
convenient.

Since SMB reads and writes range up to 8MB, I'd suggest zeroing the
least significant 23 bits, which should guarantee it. The other 41
bits, party on. You could randomize them, pass some clever identifier
such as MID sequence, whatever.

Tom.

> +		v1->token = rdata->mr->mr->rkey;
> +		v1->length = rdata->mr->mr->length;

RE: [Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via RDMA write through memory registration

[Long Li]

> Subject: Re: [Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via
> RDMA write through memory registration
> 
> Replying to a very old message, but it's something we discussed today at the
> IOLab event so to capture it:
> 
> On 11/7/2017 12:55 AM, Long Li wrote:
> > From: Long Li <longli@microsoft.com>
> >
> > ---
> >   fs/cifs/file.c    | 17 +++++++++++++++--
> >   fs/cifs/smb2pdu.c | 45
> ++++++++++++++++++++++++++++++++++++++++++++-
> >   2 files changed, 59 insertions(+), 3 deletions(-) ...
> > diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c index
> > c8afb83..8a5ff90 100644
> > --- a/fs/cifs/smb2pdu.c
> > +++ b/fs/cifs/smb2pdu.c
> > @@ -2379,7 +2379,40 @@ smb2_new_read_req(void **buf, unsigned int
> *total_len,
> >   	req->MinimumCount = 0;
> >   	req->Length = cpu_to_le32(io_parms->length);
> >   	req->Offset = cpu_to_le64(io_parms->offset);
> > +#ifdef CONFIG_CIFS_SMB_DIRECT
> > +	/*
> > +	 * If we want to do a RDMA write, fill in and append
> > +	 * smbd_buffer_descriptor_v1 to the end of read request
> > +	 */
> > +	if (server->rdma && rdata &&
> > +		rdata->bytes >= server->smbd_conn-
> >rdma_readwrite_threshold) {
> > +
> > +		struct smbd_buffer_descriptor_v1 *v1;
> > +		bool need_invalidate =
> > +			io_parms->tcon->ses->server->dialect ==
> SMB30_PROT_ID;
> > +
> > +		rdata->mr = smbd_register_mr(
> > +				server->smbd_conn, rdata->pages,
> > +				rdata->nr_pages, rdata->tailsz,
> > +				true, need_invalidate);
> > +		if (!rdata->mr)
> > +			return -ENOBUFS;
> > +
> > +		req->Channel = SMB2_CHANNEL_RDMA_V1_INVALIDATE;
> > +		if (need_invalidate)
> > +			req->Channel = SMB2_CHANNEL_RDMA_V1;
> > +		req->ReadChannelInfoOffset =
> > +			offsetof(struct smb2_read_plain_req, Buffer);
> > +		req->ReadChannelInfoLength =
> > +			sizeof(struct smbd_buffer_descriptor_v1);
> > +		v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0];
> > +		v1->offset = rdata->mr->mr->iova;
> 
> It's unnecessary, and possibly leaking kernel information, to use the IOVA as
> the offset of a memory region which is registered using an FRWR. Because
> such regions are based on the exact bytes targeted by the memory handle,
> the offset can be set to any value, typically zero, but nearly arbitrary. As long
> as the (offset + length) does not wrap or otherwise overflow, offset can be
> set to anything convenient.
> 
> Since SMB reads and writes range up to 8MB, I'd suggest zeroing the least
> significant 23 bits, which should guarantee it. The other 41 bits, party on. You
> could randomize them, pass some clever identifier such as MID sequence,
> whatever.
> 
> Tom.

Thanks Tom. I will fix this.

> 
> > +		v1->token = rdata->mr->mr->rkey;
> > +		v1->length = rdata->mr->mr->length;

Re: [Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via RDMA write through memory registration

[Stefan Metzmacher]

[-- Attachment #1.1: Type: text/plain, Size: 1429 bytes --]

Hi,

>> +        req->Channel = SMB2_CHANNEL_RDMA_V1_INVALIDATE;
>> +        if (need_invalidate)
>> +            req->Channel = SMB2_CHANNEL_RDMA_V1;
>> +        req->ReadChannelInfoOffset =
>> +            offsetof(struct smb2_read_plain_req, Buffer);
>> +        req->ReadChannelInfoLength =
>> +            sizeof(struct smbd_buffer_descriptor_v1);
>> +        v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0];
>> +        v1->offset = rdata->mr->mr->iova;
> 
> It's unnecessary, and possibly leaking kernel information, to use
> the IOVA as the offset of a memory region which is registered using
> an FRWR. Because such regions are based on the exact bytes targeted
> by the memory handle, the offset can be set to any value, typically
> zero, but nearly arbitrary. As long as the (offset + length) does
> not wrap or otherwise overflow, offset can be set to anything
> convenient.
> 
> Since SMB reads and writes range up to 8MB, I'd suggest zeroing the
> least significant 23 bits, which should guarantee it. The other 41
> bits, party on. You could randomize them, pass some clever identifier
> such as MID sequence, whatever.

I just tested that setting:

mr->iova &= (PAGE_SIZE - 1);
mr->iova |= 0xFFFFFFFF00000000;

after the ib_map_mr_sg() and before doing the IB_WR_REG_MR, seems to work.

metze


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via RDMA write through memory registration

[Tom Talpey]

On 9/21/2018 8:56 PM, Stefan Metzmacher wrote:
> Hi,
> 
>>> +        req->Channel = SMB2_CHANNEL_RDMA_V1_INVALIDATE;
>>> +        if (need_invalidate)
>>> +            req->Channel = SMB2_CHANNEL_RDMA_V1;
>>> +        req->ReadChannelInfoOffset =
>>> +            offsetof(struct smb2_read_plain_req, Buffer);
>>> +        req->ReadChannelInfoLength =
>>> +            sizeof(struct smbd_buffer_descriptor_v1);
>>> +        v1 = (struct smbd_buffer_descriptor_v1 *) &req->Buffer[0];
>>> +        v1->offset = rdata->mr->mr->iova;
>>
>> It's unnecessary, and possibly leaking kernel information, to use
>> the IOVA as the offset of a memory region which is registered using
>> an FRWR. Because such regions are based on the exact bytes targeted
>> by the memory handle, the offset can be set to any value, typically
>> zero, but nearly arbitrary. As long as the (offset + length) does
>> not wrap or otherwise overflow, offset can be set to anything
>> convenient.
>>
>> Since SMB reads and writes range up to 8MB, I'd suggest zeroing the
>> least significant 23 bits, which should guarantee it. The other 41
>> bits, party on. You could randomize them, pass some clever identifier
>> such as MID sequence, whatever.
> 
> I just tested that setting:
> 
> mr->iova &= (PAGE_SIZE - 1);
> mr->iova |= 0xFFFFFFFF00000000;
> 
> after the ib_map_mr_sg() and before doing the IB_WR_REG_MR, seems to work.

Good! As you know, we were concerned about it after seeing that
the ib_dma_map_sg() code was unconditionally setting it to the
dma_mapped address. By salting those FFFF's with varying data,
this should give your FRWR regions stronger integrity in addition
to not leaking kernel "addresses" to the wire.

Tom.

Re: [Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via RDMA write through memory registration

[Stefan Metzmacher]

[-- Attachment #1.1: Type: text/plain, Size: 593 bytes --]

Hi Tom,

>> I just tested that setting:
>>
>> mr->iova &= (PAGE_SIZE - 1);
>> mr->iova |= 0xFFFFFFFF00000000;
>>
>> after the ib_map_mr_sg() and before doing the IB_WR_REG_MR, seems to
>> work.
> 
> Good! As you know, we were concerned about it after seeing that
> the ib_dma_map_sg() code was unconditionally setting it to the
> dma_mapped address. By salting those FFFF's with varying data,
> this should give your FRWR regions stronger integrity in addition
> to not leaking kernel "addresses" to the wire.

Just wondering... Isn't the thing we use called FRMR?

metze


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via RDMA write through memory registration

[Tom Talpey]

On 9/23/2018 2:24 PM, Stefan Metzmacher wrote:
> Hi Tom,
> 
>>> I just tested that setting:
>>>
>>> mr->iova &= (PAGE_SIZE - 1);
>>> mr->iova |= 0xFFFFFFFF00000000;
>>>
>>> after the ib_map_mr_sg() and before doing the IB_WR_REG_MR, seems to
>>> work.
>>
>> Good! As you know, we were concerned about it after seeing that
>> the ib_dma_map_sg() code was unconditionally setting it to the
>> dma_mapped address. By salting those FFFF's with varying data,
>> this should give your FRWR regions stronger integrity in addition
>> to not leaking kernel "addresses" to the wire.
> 
> Just wondering... Isn't the thing we use called FRMR?

They're basically the same concept, it's a subtle difference.

FRMR = Fast Register Memory Region
FRWR = Fast Register Work Request

The memory region is the mr itself, this is created early on.

The work request is built when actually binding the physical
pages to the region, and setting the offset, length, etc, which
is what's happening in the routine that I made the comment on.

So, for this discussion I chose to say FRWR. Sorry for any
confusion!

Tom.

Re: [Patch v7 21/22] CIFS: SMBD: Upper layer performs SMB read via RDMA write through memory registration

[Stefan Metzmacher]

[-- Attachment #1.1: Type: text/plain, Size: 544 bytes --]

> They're basically the same concept, it's a subtle difference.
> 
> FRMR = Fast Register Memory Region
> FRWR = Fast Register Work Request
> 
> The memory region is the mr itself, this is created early on.
> 
> The work request is built when actually binding the physical
> pages to the region, and setting the offset, length, etc, which
> is what's happening in the routine that I made the comment on.
> 
> So, for this discussion I chose to say FRWR. Sorry for any
> confusion!

Ah, thanks! Confusion resolved:-)

metze



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[Patch v7 22/22] CIFS: SMBD: Add SMB Direct debug counters

[Long Li]

From: Long Li <longli@microsoft.com>

For debugging and troubleshooting, export SMBDirect debug counters to
/proc/fs/cifs/DebugData.

Signed-off-by: Long Li <longli@microsoft.com>
---
 fs/cifs/cifs_debug.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/fs/cifs/cifs_debug.c b/fs/cifs/cifs_debug.c
index 7025d8d..cd65759 100644
--- a/fs/cifs/cifs_debug.c
+++ b/fs/cifs/cifs_debug.c
@@ -155,6 +155,72 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)
 	list_for_each(tmp1, &cifs_tcp_ses_list) {
 		server = list_entry(tmp1, struct TCP_Server_Info,
 				    tcp_ses_list);
+
+#ifdef CONFIG_CIFS_SMB_DIRECT
+		if (!server->rdma)
+			goto skip_rdma;
+
+		seq_printf(m, "\nSMBDirect (in hex) protocol version: %x "
+			"transport status: %x",
+			server->smbd_conn->protocol,
+			server->smbd_conn->transport_status);
+		seq_printf(m, "\nConn receive_credit_max: %x "
+			"send_credit_target: %x max_send_size: %x",
+			server->smbd_conn->receive_credit_max,
+			server->smbd_conn->send_credit_target,
+			server->smbd_conn->max_send_size);
+		seq_printf(m, "\nConn max_fragmented_recv_size: %x "
+			"max_fragmented_send_size: %x max_receive_size:%x",
+			server->smbd_conn->max_fragmented_recv_size,
+			server->smbd_conn->max_fragmented_send_size,
+			server->smbd_conn->max_receive_size);
+		seq_printf(m, "\nConn keep_alive_interval: %x "
+			"max_readwrite_size: %x rdma_readwrite_threshold: %x",
+			server->smbd_conn->keep_alive_interval,
+			server->smbd_conn->max_readwrite_size,
+			server->smbd_conn->rdma_readwrite_threshold);
+		seq_printf(m, "\nDebug count_get_receive_buffer: %x "
+			"count_put_receive_buffer: %x count_send_empty: %x",
+			server->smbd_conn->count_get_receive_buffer,
+			server->smbd_conn->count_put_receive_buffer,
+			server->smbd_conn->count_send_empty);
+		seq_printf(m, "\nRead Queue count_reassembly_queue: %x "
+			"count_enqueue_reassembly_queue: %x "
+			"count_dequeue_reassembly_queue: %x "
+			"fragment_reassembly_remaining: %x "
+			"reassembly_data_length: %x "
+			"reassembly_queue_length: %x",
+			server->smbd_conn->count_reassembly_queue,
+			server->smbd_conn->count_enqueue_reassembly_queue,
+			server->smbd_conn->count_dequeue_reassembly_queue,
+			server->smbd_conn->fragment_reassembly_remaining,
+			server->smbd_conn->reassembly_data_length,
+			server->smbd_conn->reassembly_queue_length);
+		seq_printf(m, "\nCurrent Credits send_credits: %x "
+			"receive_credits: %x receive_credit_target: %x",
+			atomic_read(&server->smbd_conn->send_credits),
+			atomic_read(&server->smbd_conn->receive_credits),
+			server->smbd_conn->receive_credit_target);
+		seq_printf(m, "\nPending send_pending: %x send_payload_pending:"
+			" %x smbd_send_pending: %x smbd_recv_pending: %x",
+			atomic_read(&server->smbd_conn->send_pending),
+			atomic_read(&server->smbd_conn->send_payload_pending),
+			server->smbd_conn->smbd_send_pending,
+			server->smbd_conn->smbd_recv_pending);
+		seq_printf(m, "\nReceive buffers count_receive_queue: %x "
+			"count_empty_packet_queue: %x",
+			server->smbd_conn->count_receive_queue,
+			server->smbd_conn->count_empty_packet_queue);
+		seq_printf(m, "\nMR responder_resources: %x "
+			"max_frmr_depth: %x mr_type: %x",
+			server->smbd_conn->responder_resources,
+			server->smbd_conn->max_frmr_depth,
+			server->smbd_conn->mr_type);
+		seq_printf(m, "\nMR mr_ready_count: %x mr_used_count: %x",
+			atomic_read(&server->smbd_conn->mr_ready_count),
+			atomic_read(&server->smbd_conn->mr_used_count));
+skip_rdma:
+#endif
 		seq_printf(m, "\nNumber of credits: %d", server->credits);
 		i++;
 		list_for_each(tmp2, &server->smb_ses_list) {
-- 
2.7.4