From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sagi Grimberg Subject: Re: [PATCH 7/8] xprtrdma: Split the completion queue Date: Thu, 17 Apr 2014 10:06:07 +0300 Message-ID: <534F7D5F.1090908@dev.mellanox.co.il> References: <20140414220041.20646.63991.stgit@manet.1015granger.net> <20140414222323.20646.66946.stgit@manet.1015granger.net> <534E7C1C.5070407@dev.mellanox.co.il> <534E8608.8030801@opengridcomputing.com> <534EA06A.7090200@dev.mellanox.co.il> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: In-Reply-To: Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Chuck Lever Cc: Steve Wise , Linux NFS Mailing List , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-Id: linux-rdma@vger.kernel.org On 4/16/2014 9:21 PM, Chuck Lever wrote: > Passing a small array to ip_poll_cq() is actually easy to do, and is > exactly equivalent to a poll budget. The struct ib_wc should be taken > off the stack anyway, IMO. > > The only other example I see in 3.15 right now is IPoIB, which seems > to do exactly this. > > I=92m testing a patch now. I=92d like to start simple and make it mor= e > complex only if we need to. What array size are you using? Note that if you use a small array it ma= y=20 be an overkill since a lot more interrupts are invoked (-> more latency). I found that for a= =20 high workload a budget of 256/512/1024 keeps fairness and doesn't increase latency. Regardless, doing array-polling is a nice optimization reducing CQ=20 entrances. Sagi. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" i= n the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: linux-nfs-owner@vger.kernel.org Received: from mail-wg0-f47.google.com ([74.125.82.47]:35444 "EHLO mail-wg0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751019AbaDQHGL (ORCPT ); Thu, 17 Apr 2014 03:06:11 -0400 Received: by mail-wg0-f47.google.com with SMTP id x12so47220wgg.18 for ; Thu, 17 Apr 2014 00:06:09 -0700 (PDT) Message-ID: <534F7D5F.1090908@dev.mellanox.co.il> Date: Thu, 17 Apr 2014 10:06:07 +0300 From: Sagi Grimberg MIME-Version: 1.0 To: Chuck Lever CC: Steve Wise , Linux NFS Mailing List , linux-rdma@vger.kernel.org Subject: Re: [PATCH 7/8] xprtrdma: Split the completion queue References: <20140414220041.20646.63991.stgit@manet.1015granger.net> <20140414222323.20646.66946.stgit@manet.1015granger.net> <534E7C1C.5070407@dev.mellanox.co.il> <534E8608.8030801@opengridcomputing.com> <534EA06A.7090200@dev.mellanox.co.il> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Sender: linux-nfs-owner@vger.kernel.org List-ID: On 4/16/2014 9:21 PM, Chuck Lever wrote: > Passing a small array to ip_poll_cq() is actually easy to do, and is > exactly equivalent to a poll budget. The struct ib_wc should be taken > off the stack anyway, IMO. > > The only other example I see in 3.15 right now is IPoIB, which seems > to do exactly this. > > I’m testing a patch now. I’d like to start simple and make it more > complex only if we need to. What array size are you using? Note that if you use a small array it may be an overkill since a lot more interrupts are invoked (-> more latency). I found that for a high workload a budget of 256/512/1024 keeps fairness and doesn't increase latency. Regardless, doing array-polling is a nice optimization reducing CQ entrances. Sagi.