From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bart Van Assche Subject: Re: [PATCH] IB/core: Don't drain the receive queue for srq attached queue-pair Date: Tue, 26 Apr 2016 10:27:01 -0700 Message-ID: <571FA4E5.2030501@sandisk.com> References: <1461682538-19647-1-git-send-email-sagi@grimberg.me> <571F841D.3010909@opengridcomputing.com> <20160426154328.GA12398@lst.de> <912C9E71-05E3-4ED9-9B41-137E131E3A71@oracle.com> <571F9084.2040506@opengridcomputing.com> <571F917C.40008@sandisk.com> <571F972B.6030904@opengridcomputing.com> Mime-Version: 1.0 Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <571F972B.6030904-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org> Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Steve Wise , Chuck Lever , Christoph Hellwig Cc: sagi grimberg , "linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" List-Id: linux-rdma@vger.kernel.org On 04/26/2016 09:28 AM, Steve Wise wrote: > On 4/26/2016 11:04 AM, Bart Van Assche wrote: >> On 04/26/2016 09:00 AM, Steve Wise wrote: >>> On 4/26/2016 10:55 AM, Chuck Lever wrote: >>>>> On Apr 26, 2016, at 11:43 AM, Christoph Hellwig wrote: >>>>> >>>>> Just curious: what takes care of draining SRQs when we unregister them >>>>> after all queues are gone? >>>> >>>> Conversely, I would think that a consumer that uses SRQ >>>> would want a similar drain mechanism to guarantee there >>>> are no more posted Receives for the associated QP, and >>>> thus it is safe to destroy it. >>> >>> I don't think there is anything now that handles draining an SRQ, nor >>> ensuring a QP's RQEs are completed/consumed from its SRQ when draining >>> the QP. Sounds like work is needed here. >>> >>> But are there any kernel SRQ consumers at this point? >> >> I think the following two: >> >> $ git grep -nHE '(ib|rdma)_create_srq' drivers/infiniband/ulp >> drivers/infiniband/ulp/ipoib/ipoib_cm.c:1521: priv->cm.srq = >> ib_create_srq(priv->pd, &srq_init_attr); >> drivers/infiniband/ulp/srpt/ib_srpt.c:2723: sdev->srq = >> ib_create_srq(sdev->pd, &srq_attr); > > If we make the requirement that the ib_drain_rq() caller must consume > all completions for all QPs attached to an SRQ if they are outstanding, > then I think we can modify ib_drain_rq() to post the drain recv WR to > the SRQ. It should work, right? At least the ib_srpt driver already guarantees that no further receive completions will be generated before ib_destroy_qp() is called. But posting an additional receive WR on the SRQ from inside ib_drain_rq() shouldn't hurt. Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html