[PATCH RESEND] nbd: Aovid double completion of a request

[PATCH RESEND] nbd: Aovid double completion of a request

[Xie Yongji]

There is a race between iterating over requests in
nbd_clear_que() and completing requests in recv_work(),
which can lead to double completion of a request.

To fix it, flush the recv worker before iterating over
the requests and don't abort the completed request
while iterating.

Fixes: 96d97e17828f ("nbd: clear_sock on netlink disconnect")
Reported-by: Jiang Yadong <jiangyadong@bytedance.com>
Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
---
 drivers/block/nbd.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index c38317979f74..19f5d5a8b16a 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -818,6 +818,10 @@ static bool nbd_clear_req(struct request *req, void *data, bool reserved)
 {
 	struct nbd_cmd *cmd = blk_mq_rq_to_pdu(req);
 
+	/* don't abort one completed request */
+	if (blk_mq_request_completed(req))
+		return true;
+
 	mutex_lock(&cmd->lock);
 	cmd->status = BLK_STS_IOERR;
 	mutex_unlock(&cmd->lock);
@@ -2004,15 +2008,19 @@ static void nbd_disconnect_and_put(struct nbd_device *nbd)
 {
 	mutex_lock(&nbd->config_lock);
 	nbd_disconnect(nbd);
-	nbd_clear_sock(nbd);
-	mutex_unlock(&nbd->config_lock);
+	sock_shutdown(nbd);
 	/*
 	 * Make sure recv thread has finished, so it does not drop the last
 	 * config ref and try to destroy the workqueue from inside the work
-	 * queue.
+	 * queue. And this also ensure that we can safely call nbd_clear_que()
+	 * to cancel the inflight I/Os.
 	 */
 	if (nbd->recv_workq)
 		flush_workqueue(nbd->recv_workq);
+	nbd_clear_que(nbd);
+	nbd->task_setup = NULL;
+	mutex_unlock(&nbd->config_lock);
+
 	if (test_and_clear_bit(NBD_RT_HAS_CONFIG_REF,
 			       &nbd->config->runtime_flags))
 		nbd_config_put(nbd);
-- 
2.11.0

Re: [PATCH RESEND] nbd: Aovid double completion of a request

[Josef Bacik]

On 8/13/21 11:13 AM, Xie Yongji wrote:
> There is a race between iterating over requests in
> nbd_clear_que() and completing requests in recv_work(),
> which can lead to double completion of a request.
> 
> To fix it, flush the recv worker before iterating over
> the requests and don't abort the completed request
> while iterating.
> 
> Fixes: 96d97e17828f ("nbd: clear_sock on netlink disconnect")
> Reported-by: Jiang Yadong <jiangyadong@bytedance.com>
> Signed-off-by: Xie Yongji <xieyongji@bytedance.com>

Thanks for resending this, my email client messed up.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>

Josef

Re: [PATCH RESEND] nbd: Aovid double completion of a request

[Jens Axboe]

On 8/13/21 9:13 AM, Xie Yongji wrote:
> There is a race between iterating over requests in
> nbd_clear_que() and completing requests in recv_work(),
> which can lead to double completion of a request.
> 
> To fix it, flush the recv worker before iterating over
> the requests and don't abort the completed request
> while iterating.

Applied, thanks.

-- 
Jens Axboe