From mboxrd@z Thu Jan 1 00:00:00 1970 From: Roland Dreier Subject: Re: ibv_post_send/recv kernel path optimizations Date: Wed, 05 Jan 2011 10:16:57 -0800 Message-ID: References: <20101013091312.GB6060@bicker> <20101123071025.GI1522@bicker> <20101124221845.GH2369@obsidianresearch.com> <20101125041337.GA11049@obsidianresearch.com> <4CEE7A22.2040706@voltaire.com> <4CF60343.7050602@voltaire.com> <20101214181735.GA2506@obsidianresearch.com> <4D1888CB.2010101@Voltaire.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: In-Reply-To: (Miroslaw Walukiewicz's message of "Mon, 27 Dec 2010 15:18:08 +0000") Sender: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: "Walukiewicz, Miroslaw" Cc: Or Gerlitz , Jason Gunthorpe , "Hefty, Sean" , "linux-rdma@vger.kernel.org" List-Id: linux-rdma@vger.kernel.org > The patch for ofed-1.5.3 looks like below. I will try to push it to kernel.org after porting. > > Now an uverbs post_send/post_recv path is modified to make pre-lookup > for RAW_ETH QPs. When a RAW_ETH QP is found the driver specific path > is used for posting buffers. for example using a shared page approach in > cooperation with user-space library I don't quite see why a hash table helps performance much vs. an IDR. Is the actual IDR lookup a significant part of the cost? (By the way, instead of list_head you could use hlist_head to make your hash table denser and save cache footprint -- that way an 8-entry table on 64-bit systems fits in one cacheline) Also it seems that you get rid of all the locking on QPs when you look them up in your hash table. What protects against userspace posting a send in one thread and destroying the QP in another thread, and ending up having the destroy complete before the send is posted (leading to use-after-free in the kernel)? I would guess that your speedup is really coming from getting rid of locking that is actually required for correctness. Maybe I'm wrong though, I'm just guessing here. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html