Re: [PATCH V2] blk-mq: Set request mapping to NULL in blk_mq_put_driver_tag

[Jens Axboe <axboe@kernel.dk>]

On 12/18/18 11:22 AM, Kashyap Desai wrote:
>>>
>>> At the time of device removal,  it requires reverse traversing.  Find
>>> out if each requests associated with sdev is part of hctx->tags->rqs()
>>> and clear that entry.
>>> Not sure about atomic traverse if more than one device removal is
>>> happening in parallel.  May be more error prone. ?
>>>
>>> Just wondering both the way we will be removing invalid request from
>>> array.
>>> Are you suspecting any performance issue if we do it per IO ?
>>
>> It's an extra store, and it's a store to an area that's then now shared
>> between
>> issue and completion. Those are never a good idea. Besides, it's the kind
>> of
>> issue you solve in the SLOW path, not in the fast path. Since that's
>> doable, it
>> would be silly to do it for every IO.
>>
>> This might not matter on mpt3sas, but on more efficient hw it definitely
>> will.
> 
> Understood your primary concern is to avoid per IO and do it if no better
> way.
> 
>> I'm still trying to convince myself that this issue even exists. I can see
>> having
>> stale entries, but those should never be busy. Why are you finding them
>> with
>> the tag iteration? It must be because the tag is reused, and you are
>> finding it
>> before it's re-assigned?
> 
> 
> Stale entries will be forever if we remove scsi devices. It is not timing
> issue. If memory associated with request (freed due to device removal)
> reused, kernel panic occurs.
> We have 24 Drives behind Expander and follow expander reset which will
> remove all 24 drives and add it back. Add and removal of all the drives
> happens quickly.
> As part of Expander reset, <mpt3sas> driver process broadcast primitive
> event and that requires finding all outstanding scsi command.
> 
> In some cases, we need firmware restart and that path also requires tag
> iteration.

I actually took a look at scsi_host_find_tag() - what I think needs
fixing here is that it should not return a tag that isn't allocated.
You're just looking up random stuff, that is a recipe for disaster.
But even with that, there's no guarantee that the tag isn't going away.

The mpt3sas use case is crap. It's iterating every tag, just in case it
needs to do something to it.

My suggestion would be to scrap that bad implementation and have
something available for iterating busy tags instead. That'd be more
appropriate and a lot more efficient that a random loop from 0..depth.
If you are flushing running commands, looking up tags that aren't even
active is silly and counterproductive.

-- 
Jens Axboe