[PATCH] nvme-tcp: fix possible req->offset corruption

[PATCH] nvme-tcp: fix possible req->offset corruption

[Varun Prakash]

With commit db5ad6b7f8cd ("nvme-tcp: try to send request in queue_rq
context") r2t and response PDU can get processed while send function
is executing.

Current data digest send code uses req->offset after kernel_sendmsg(),
this creates a race condition where req->offset gets reset before it
is used in send function.

This can happen in two cases -
1. Target sends r2t PDU which resets req->offset.
2. Target send response PDU which completes the req and then req is
   used for a new command, nvme_tcp_setup_cmd_pdu() resets req->offset.

Fix this by storing req->offset in a local variable and using
this local variable after kernel_sendmsg().

Fixes: db5ad6b7f8cd ("nvme-tcp: try to send request in queue_rq context")
Signed-off-by: Varun Prakash <varun@chelsio.com>
---
 drivers/nvme/host/tcp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 22da856..2217897 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1048,6 +1048,7 @@ static int nvme_tcp_try_send_data_pdu(struct nvme_tcp_request *req)
 static int nvme_tcp_try_send_ddgst(struct nvme_tcp_request *req)
 {
 	struct nvme_tcp_queue *queue = req->queue;
+	size_t offset = req->offset;
 	int ret;
 	struct msghdr msg = { .msg_flags = MSG_DONTWAIT };
 	struct kvec iov = {
@@ -1064,7 +1065,7 @@ static int nvme_tcp_try_send_ddgst(struct nvme_tcp_request *req)
 	if (unlikely(ret <= 0))
 		return ret;
 
-	if (req->offset + ret == NVME_TCP_DIGEST_LENGTH) {
+	if (offset + ret == NVME_TCP_DIGEST_LENGTH) {
 		nvme_tcp_done_send_req(queue);
 		return 1;
 	}
-- 
2.0.2

Re: [PATCH] nvme-tcp: fix possible req->offset corruption

[Keith Busch]

On Tue, Oct 26, 2021 at 07:01:55PM +0530, Varun Prakash wrote:
> With commit db5ad6b7f8cd ("nvme-tcp: try to send request in queue_rq
> context") r2t and response PDU can get processed while send function
> is executing.
> 
> Current data digest send code uses req->offset after kernel_sendmsg(),
> this creates a race condition where req->offset gets reset before it
> is used in send function.
> 
> This can happen in two cases -
> 1. Target sends r2t PDU which resets req->offset.
> 2. Target send response PDU which completes the req and then req is
>    used for a new command, nvme_tcp_setup_cmd_pdu() resets req->offset.
> 
> Fix this by storing req->offset in a local variable and using
> this local variable after kernel_sendmsg().

Nice catch.

Reviewed-by: Keith Busch <kbusch@kernel.org>
 
> Fixes: db5ad6b7f8cd ("nvme-tcp: try to send request in queue_rq context")
> Signed-off-by: Varun Prakash <varun@chelsio.com>
> ---
>  drivers/nvme/host/tcp.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 22da856..2217897 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -1048,6 +1048,7 @@ static int nvme_tcp_try_send_data_pdu(struct nvme_tcp_request *req)
>  static int nvme_tcp_try_send_ddgst(struct nvme_tcp_request *req)
>  {
>  	struct nvme_tcp_queue *queue = req->queue;
> +	size_t offset = req->offset;
>  	int ret;
>  	struct msghdr msg = { .msg_flags = MSG_DONTWAIT };
>  	struct kvec iov = {
> @@ -1064,7 +1065,7 @@ static int nvme_tcp_try_send_ddgst(struct nvme_tcp_request *req)
>  	if (unlikely(ret <= 0))
>  		return ret;
>  
> -	if (req->offset + ret == NVME_TCP_DIGEST_LENGTH) {
> +	if (offset + ret == NVME_TCP_DIGEST_LENGTH) {
>  		nvme_tcp_done_send_req(queue);
>  		return 1;
>  	}
> -- 
> 2.0.2
> 

Re: [PATCH] nvme-tcp: fix possible req->offset corruption

[Sagi Grimberg]

Reviewed-by: Sagi Grimberg <sagi@grimberg.me>

Re: [PATCH] nvme-tcp: fix possible req->offset corruption

[Christoph Hellwig]

Thanks,

applied to nvme-5.15.