Re: IPoIB performance

["Atchley, Scott" <atchleyes-1Heg1YXhbW8@public.gmane.org>]

On Sep 5, 2012, at 3:04 PM, Reeted wrote:

> On 09/05/12 19:59, Atchley, Scott wrote:
>> On Sep 5, 2012, at 1:50 PM, Reeted wrote:
>> 
>>> 
>>> I have read that with newer cards the datagram (unconnected) mode is
>>> faster at IPoIB than connected mode. Do you want to check?
>> I have read that the latency is lower (better) but the bandwidth is lower.
>> 
>> Using datagram mode limits the MTU to 2044 and the throughput to ~3 Gb/s on these machines/cards. Connected mode at the same MTU performs roughly the same. The win in connected mode comes with larger MTUs. With a 9000 MTU, I see ~6 Gb/s. Pushing the MTU to 655120 (the maximum for ipoib), I can get ~13 Gb/s.
>> 
> 
> Have a look at an old thread in this ML by Sebastien Dugue "IPoIB to 
> Ethernet routing performance"
> He had numbers much higher than yours on similar hardware, and was 
> suggested to use datagram to achieve offloading and even higher speeds.
> Keep me informed if you can fix this, I am interested but can't test 
> infiniband myself right now.

He claims 20 Gb/s and Or replies that one should also get near 20 Gb/s using datagram mode. I checked and datagram mode shows support via ethtool for more offloads. In my case, I still see better performance with connected mode.

Thanks,

Scott--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html