archive mirror
 help / color / mirror / Atom feed
From: Jens Axboe <>
To: Bart Van Assche <>,
	Kashyap Desai <>,
	linux-block <>,
	linux-scsi <>
Cc: Ming Lei <>,
	Suganath Prabu Subramani  <>,
	Sreekanth Reddy <>,
	Sathya Prakash <>
Subject: Re: [PATCH V2] blk-mq: Set request mapping to NULL in blk_mq_put_driver_tag
Date: Tue, 18 Dec 2018 09:18:47 -0700	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On 12/18/18 9:15 AM, Bart Van Assche wrote:
> On Tue, 2018-12-18 at 12:38 +0530, Kashyap Desai wrote:
>> V1 -> V2
>> Added fix in __blk_mq_finish_request around blk_mq_put_tag() for
>> non-internal tags
>> Problem statement :
>> Whenever try to get outstanding request via scsi_host_find_tag,
>> block layer will return stale entries instead of actual outstanding
>> request. Kernel panic if stale entry is inaccessible or memory is reused.
>> Fix :
>> Undo request mapping in blk_mq_put_driver_tag  nce request is return.
>> More detail :
>> Whenever each SDEV entry is created, block layer allocate separate tags
>> and static requestis.Those requests are not valid after SDEV is deleted
>> from the system. On the fly, block layer maps static rqs to rqs as below
>> from blk_mq_get_driver_tag()
>> data.hctx->tags->rqs[rq->tag] = rq;
>> Above mapping is active in-used requests and it is the same mapping which
>> is referred in function scsi_host_find_tag().
>> After running some IOs, “data.hctx->tags->rqs[rq->tag]” will have some
>> entries which will never be reset in block layer.
>> There would be a kernel panic, If request pointing to
>> “data.hctx->tags->rqs[rq->tag]” is part of “sdev” which is removed
>> and as part of that all the memory allocation of request associated with
>> that sdev might be reused or inaccessible to the driver.
>> Kernel panic snippet -
>> BUG: unable to handle kernel paging request at ffffff8000000010
>> IP: [<ffffffffc048306c>] mpt3sas_scsih_scsi_lookup_get+0x6c/0xc0 [mpt3sas]
>> PGD aa4414067 PUD 0
>> Oops: 0000 [#1] SMP
>> Call Trace:
>>  [<ffffffffc046f72f>] mpt3sas_get_st_from_smid+0x1f/0x60 [mpt3sas]
>>  [<ffffffffc047e125>] scsih_shutdown+0x55/0x100 [mpt3sas]
> Other block drivers (e.g. ib_srp, skd) do not need this to work reliably.
> It has been explained to you that the bug that you reported can be fixed
> by modifying the mpt3sas driver. So why to fix this by modifying the block
> layer? Additionally, what prevents that a race condition occurs between
> the block layer clearing hctx->tags->rqs[rq->tag] and scsi_host_find_tag()
> reading that same array element? I'm afraid that this is an attempt to
> paper over a real problem instead of fixing the root cause.

I have to agree with Bart here, I just don't see how the mpt3sas use case
is special. The change will paper over the issue in any case.

Jens Axboe

  reply	other threads:[~2018-12-18 16:18 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-18  7:08 [PATCH V2] blk-mq: Set request mapping to NULL in blk_mq_put_driver_tag Kashyap Desai
2018-12-18  8:27 ` Hannes Reinecke
2018-12-18 16:15 ` Bart Van Assche
2018-12-18 16:18   ` Jens Axboe [this message]
2018-12-18 17:08     ` Kashyap Desai
2018-12-18 17:13       ` Jens Axboe
2018-12-18 17:48         ` Kashyap Desai
2018-12-18 17:50           ` Jens Axboe
2018-12-18 18:08             ` Kashyap Desai
2018-12-18 18:11               ` Jens Axboe
2018-12-18 18:22                 ` Kashyap Desai
2018-12-18 18:28                   ` Jens Axboe
2018-12-18 19:04                     ` Kashyap Desai

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).