Re: [PATCH V7 3/3] blk-mq: don't deactivate hctx if managed irq isn't used

[Ming Lei <ming.lei@redhat.com>]

On Wed, Sep 15, 2021 at 06:14:59PM +0200, Daniel Wagner wrote:
> On Wed, Aug 18, 2021 at 10:44:28PM +0800, Ming Lei wrote:
> >  struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
> >  	unsigned int op, blk_mq_req_flags_t flags, unsigned int hctx_idx)
> >  {
> > @@ -468,7 +485,10 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
> >  	data.hctx = q->queue_hw_ctx[hctx_idx];
> >  	if (!blk_mq_hw_queue_mapped(data.hctx))
> >  		goto out_queue_exit;
> > -	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
> > +
> > +	WARN_ON_ONCE(blk_mq_hctx_use_managed_irq(data.hctx));
> > +
> > +	cpu = blk_mq_first_mapped_cpu(data.hctx);
> >  	data.ctx = __blk_mq_get_ctx(q, cpu);
> 
> I was pondering how we could address the issue that the qla2xxx driver
> is using managed IRQs which makes nvme-fc depending as class on managed
> IRQ.
> 
> blk_mq_alloc_request_hctx() is the only place where we really need to
> distinguish between managed and !managed IRQs. As far I undertand the
> situation, if all CPUs for a hctx are going offline, the driver wont use
> this context. So there is only the case we end up in this code path is
> when the driver tries to reconnect the queues, e.g. after
> devloss. Couldn't we in this case not just return an error and go into
> error recovery? Something like this:
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index a2db50886a26..52fc8592c72e 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -486,9 +486,13 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
>         if (!blk_mq_hw_queue_mapped(data.hctx))
>                 goto out_queue_exit;
>  
> -       WARN_ON_ONCE(blk_mq_hctx_use_managed_irq(data.hctx));
> -
> -       cpu = blk_mq_first_mapped_cpu(data.hctx);
> +       if (blk_mq_hctx_use_managed_irq(data.hctx)) {

Firstly, even with patches of 'qla2xxx - add nvme map_queues support',
the knowledge if managed irq is used in nvmef LLD is still missed, so
blk_mq_hctx_use_managed_irq() may always return false, but that
shouldn't be hard to solve.

The problem is that we still should make connect io queue completed
when all CPUs of this hctx is offline in case of managed irq.

One solution might be to use io polling for connecting io queue, but nvme fc
doesn't support polling, all the other nvme hosts do support it.



Thanks,
Ming