Re: [PATCH 13/17] scsi: push host_lock down into scsi_{host,target}_queue_ready

[Bart Van Assche <bvanassche@acm.org>]

On 02/10/14 21:09, Jens Axboe wrote:
> On Mon, Feb 10 2014, Christoph Hellwig wrote:
>>> I also think we should be getting more utility out of threading
>>> guarantees.  So, if there's only one thread active per device we don't
>>> need any device counters to be atomic.  Likewise, u32 read/write is an
>>> atomic operation, so we might be able to use sloppy counters for the
>>> target and host stuff (one per CPU that are incremented/decremented on
>>> that CPU ... this will only work using CPU locality ... completion on
>>> same CPU but that seems to be an element of a lot of stuff nowadays).
>>
>> The blk-mq code is aiming for CPU locality, but there are no hard
>> guarantees.  I'm also not sure always bouncing around the I/O submission
>> is a win, but it might be something to play around with at the block
>> layer.
>>
>> Jens, did you try something like this earlier?
> 
> Nope, I've always thought that if you needed to bounce submission
> around, you would already have lost. [ ... ]

This comment makes a lot of sense to me. The approach that has been
taken in the scsi-mq patches that have been posted on February 5 is to
associate one blk-mq device with each LUN. That blk-mq device has one
hctx with queue depth shost->cmd_per_lun. So if there are multiple LUNs
per SCSI host the combined hctx queue depth of its LUNs can exceed
shost->can_queue. I'm not sure whether it's possible to prevent this
without modifying the block layer. How about modifying the block layer
such that a single hctx can be shared by multiple block devices and
adding cmd_per_lun support in the block layer ? I think that would allow
to prevent having to bounce submission in the SCSI mid-layer.

Thanks,

Bart.