Re: [PATCH V4] nvme-fabrics: reject I/O to offline device

[Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>]

On 3/31/20 11:21 AM, Alex Lyakas wrote:
> Thank you for addressing this issue. I asked about this scenario in
> June 2019 in http://lists.infradead.org/pipermail/linux-nvme/2019-June/024766.html.
>
> I tested the patch on top of Mellanox OFED 4.7 with kernel 4.14. From
> my perspective, the direction is very good. But I think the problem is
> only partially addressed.
>
> When a controller enters a CONNECTING state and the fast_io_fail_tmo
> expires, all new IOs to this controller are failed immediately. This
> is great!
>
> However, in-flight IOs, i.e., those that were issued before the
> controller got disconnected, are still stuck until the controller
> succeeds to reconnect, or forever. I believe those IOs need to be
> errored out as well after fast_io_fail_tmo expires.
>
> I did some debugging to try to accomplish that. I thought that the
> crux is that in-flight IOs are failed, and retried due to non-zero
> nvme_max_retries parameter. And by this time the request queue is
> quiesced by blk_mq_quiesce_queue(), and that's why in-flight IOs get
> stuck. I indeed see that nvme_retry_req() is called for some IOs. But
> I also see that after that, the request queue is un-quiesced via:
> nvme_rdma_error_recovery_work() =>
>     nvme_rdma_teardown_io_queues() => nvme_stop_queues() // this
> quiesces the queue
>    nvme_start_queues() // this un-quiesces the queue
>
> Can anybody perhaps give a hint on the approach to error out the
> in-flight IOs? I can modify the patch and test.
>
> Thanks,
> Alex.


Thanks for the feedback. Let me see if I can generate the scenario and
have a test for this.

I'll send out the updated patch soon.


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme