Re: [PATCH V3 4/7] ublk_drv: requeue rqs with recovery feature enabled

From: Ming Lei <ming.lei@redhat.com>
To: Ziyang Zhang <ZiyangZhang@linux.alibaba.com>
Cc: axboe@kernel.dk, xiaoguang.wang@linux.alibaba.com,
	linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
	joseph.qi@linux.alibaba.com, ming.lei@redhat.com
Subject: Re: [PATCH V3 4/7] ublk_drv: requeue rqs with recovery feature enabled
Date: Mon, 19 Sep 2022 20:39:26 +0800	[thread overview]
Message-ID: <Yyhi/kavaq1aLAQY@T590> (raw)
In-Reply-To: <ff61718d-da2d-f754-5e56-b58a3e57820f@linux.alibaba.com>

On Mon, Sep 19, 2022 at 05:12:21PM +0800, Ziyang Zhang wrote:
> On 2022/9/19 11:55, Ming Lei wrote:
> > On Tue, Sep 13, 2022 at 12:17:04PM +0800, ZiyangZhang wrote:
> >> With recovery feature enabled, in ublk_queue_rq or task work
> >> (in exit_task_work or fallback wq), we requeue rqs instead of
> >> ending(aborting) them. Besides, No matter recovery feature is enabled
> >> or disabled, we schedule monitor_work immediately.
> >>
> >> Signed-off-by: ZiyangZhang <ZiyangZhang@linux.alibaba.com>
> >> ---
> >>  drivers/block/ublk_drv.c | 34 ++++++++++++++++++++++++++++++++--
> >>  1 file changed, 32 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
> >> index 23337bd7c105..b067f33a1913 100644
> >> --- a/drivers/block/ublk_drv.c
> >> +++ b/drivers/block/ublk_drv.c
> >> @@ -682,6 +682,21 @@ static void ubq_complete_io_cmd(struct ublk_io *io, int res)
> >>  
> >>  #define UBLK_REQUEUE_DELAY_MS	3
> >>  
> >> +static inline void __ublk_abort_rq_in_task_work(struct ublk_queue *ubq,
> >> +		struct request *rq)
> >> +{
> >> +	pr_devel("%s: %s q_id %d tag %d io_flags %x.\n", __func__,
> >> +			(ublk_queue_can_use_recovery(ubq)) ? "requeue" : "abort",
> >> +			ubq->q_id, rq->tag, ubq->ios[rq->tag].flags);
> >> +	/* We cannot process this rq so just requeue it. */
> >> +	if (ublk_queue_can_use_recovery(ubq)) {
> >> +		blk_mq_requeue_request(rq, false);
> >> +		blk_mq_delay_kick_requeue_list(rq->q, UBLK_REQUEUE_DELAY_MS);
> > 
> > Here you needn't to kick requeue list since we know it can't make
> > progress. And you can do that once before deleting gendisk
> > or the queue is recovered.
> 
> No, kicking rq here is necessary.
> 
> Consider USER_RECOVERY is enabled and everything goes well.
> User sends STOP_DEV, and we have kicked requeue list in
> ublk_stop_dev() and are going to call del_gendisk().
> However, a crash happens now. Then rqs may be still requeued
> by ublk_queue_rq() because ublk_queue_rq() sees a dying
> ubq_daemon. So del_gendisk() will hang because there are
> rqs leaving in requeue list and no one kicks them.

Why can't you kick requeue list before calling del_gendisk().

> 
> BTW, kicking requeue list after requeue rqs is really harmless
> since we schedule quiesce_work immediately after finding a
> dying ubq_daemon. So few rqs have chance to be re-dispatched.

Do you think it makes sense to kick requeue list when the queue
can't handle any request?

Thanks,
Ming