Re: [PATCH for-next] RDMA/hns: Bugfix for flush cqe in case softirq and multi-process

From: Leon Romanovsky <leon@kernel.org>
To: "Liuyixian (Eason)" <liuyixian@huawei.com>
Cc: Weihang Li <liweihang@hisilicon.com>,
	linux-rdma@vger.kernel.org, jgg@ziepe.ca, dledford@redhat.com,
	linuxarm@huawei.com
Subject: Re: [PATCH for-next] RDMA/hns: Bugfix for flush cqe in case softirq and multi-process
Date: Tue, 10 Sep 2019 10:52:16 +0300	[thread overview]
Message-ID: <20190910075216.GX6601@unreal> (raw)
In-Reply-To: <f8f29a6a-b473-6c89-8ec7-092fd53aea16@huawei.com>

On Tue, Sep 10, 2019 at 02:40:20PM +0800, Liuyixian (Eason) wrote:
>
>
> On 2019/9/8 16:03, Leon Romanovsky wrote:
> > On Thu, Sep 05, 2019 at 08:31:11PM +0800, Weihang Li wrote:
> >> From: Yixian Liu <liuyixian@huawei.com>
> >>
> >> Hip08 has the feature flush cqe, which help to flush wqe in workqueue
> >> (sq and rq) when error happened by transmitting producer index with
> >> mailbox to hardware. Flush cqe is emplemented in post send and recv
> >> verbs. However, under NVMe cases, these verbs will be called under
> >> softirq context, and it will lead to following calltrace with
> >> current driver as mailbox used by flush cqe can go to sleep.
> >>
> >> This patch solves this problem by using workqueue to do flush cqe,
> >
> > Unbelievable, almost every bug in this driver is solved by introducing
> > workqueue. You should fix "sleep in flush path" issue and not by adding
> > new workqueue.
> >
> Hi Leon,
>
> Thanks for the comment.
> Up to now, for hip08, only one place use workqueue in hns_roce_hw_v2.c
> where for irq prints.

Thanks to our lack of desire to add more workqueues and previous patches
which removed extra workqueues from the driver.

>
> The solution for flush cqe in this patch is as follow:
> While flush cqe should be implement, the driver should modify qp to error state
> through mailbox with the newest product index of sq and rq, the hardware then
> can flush all outstanding wqes in sq and rq.
>
> That's the whole mechanism of flush cqe, also is the flush path. We can't
> change neither mailbox sleep attribute or flush cqe occurred in post send/recv.
> To avoid the calltrace of flush cqe in post verbs under NVMe softirq,
> use workqueue for flush cqe seems reasonable.
>
> As far as I know, there is no other alternative solution for this situation.
> I will be very grateful if you reminder me more information.

ib_drain_rq/ib_drain_sq/ib_drain_qp????

>
> Thanks
>
> > _______________________________________________
> > Linuxarm mailing list
> > Linuxarm@huawei.com
> > http://hulk.huawei.com/mailman/listinfo/linuxarm
> >
> >
>