Re: [PATCH] block: fix null pointer dereference in blk_mq_rq_timed_out()

[Yufen Yu <yuyufen@huawei.com>]

On 2019/9/12 12:16, Ming Lei wrote:
> On Thu, Sep 12, 2019 at 11:29:18AM +0800, Yufen Yu wrote:
>>
>> On 2019/9/12 10:46, Ming Lei wrote:
>>> On Sat, Sep 07, 2019 at 06:24:50PM +0800, Yufen Yu wrote:
>>>> There is a race condition between timeout check and completion for
>>>> flush request as follow:
>>>>
>>>> timeout_work    issue flush      issue flush
>>>>                   blk_insert_flush
>>>>                                    blk_insert_flush
>>>> blk_mq_timeout_work
>>>>                   blk_kick_flush
>>>>
>>>> blk_mq_queue_tag_busy_iter
>>>> blk_mq_check_expired(flush_rq)
>>>>
>>>>                   __blk_mq_end_request
>>>>                  flush_end_io
>>>>                  blk_kick_flush
>>>>                  blk_rq_init(flush_rq)
>>>>                  memset(flush_rq, 0)
>>> Not see there is memset(flush_rq, 0) in block/blk-flush.c
>> Call path as follow:
>>
>> blk_kick_flush
>>      blk_rq_init
>>          memset(rq, 0, sizeof(*rq));
> Looks I miss this one in blk_rq_init(), sorry for that.
>
> Given there are only two users of blk_rq_init(), one simple fix could be
> not clearing queue in blk_rq_init(), something like below?
>
> diff --git a/block/blk-core.c b/block/blk-core.c
> index 77807a5d7f9e..25e6a045c821 100644
> --- a/block/blk-core.c
> +++ b/block/blk-core.c
> @@ -107,7 +107,9 @@ EXPORT_SYMBOL_GPL(blk_queue_flag_test_and_set);
>   
>   void blk_rq_init(struct request_queue *q, struct request *rq)
>   {
> -	memset(rq, 0, sizeof(*rq));
> +	const int offset = offsetof(struct request, q);
> +
> +	memset((void *)rq + offset, 0, sizeof(*rq) - offset);
>   
>   	INIT_LIST_HEAD(&rq->queuelist);
>   	rq->q = q;
> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
> index 1ac790178787..382e71b8787d 100644
> --- a/include/linux/blkdev.h
> +++ b/include/linux/blkdev.h
> @@ -130,7 +130,7 @@ enum mq_rq_state {
>    * especially blk_mq_rq_ctx_init() to take care of the added fields.
>    */
>   struct request {
> -	struct request_queue *q;
> +	struct request_queue *q;	/* Must be the 1st field */
>   	struct blk_mq_ctx *mq_ctx;
>   	struct blk_mq_hw_ctx *mq_hctx;

Not set req->q as '0' can just avoid BUG_ON for NULL pointer deference.

However, the root problem is that 'flush_rq' have been reused while
timeout function handle it currently. That means mq_ops->timeout() may
access old values remained by the last flush request and make the wrong 
decision.

Take the race condition in the patch as an example.

blk_mq_check_expired
     blk_mq_rq_timed_out
         req->q->mq_ops->timeout  // Driver timeout handle may read old data
     refcount_dec_and_test(&rq)
     __blk_mq_free_request   // If rq have been reset has '1' in 
blk_rq_init(), it will be free here.

So, I think we should solve this problem completely. Just like normal 
request,
we can prevent flush request to call end_io when timeout handle the request.

Thanks,
Yufen