From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Herbert Subject: Re: [pull request][net-next 0/3] Mellanox, mlx5 GRE tunnel offloads Date: Wed, 6 Sep 2017 09:17:50 -0700 Message-ID: References: <20170830230409.15176-1-saeedm@mellanox.com> <87fuc7ong0.fsf@stressinduktion.org> <1504402349.3299394.1093467800.72B91372@webmail.messagingengine.com> <87pob6bmjv.fsf@stressinduktion.org> <8760cytnif.fsf@stressinduktion.org> <87k21ez6r9.fsf@stressinduktion.org> <87r2vls8hp.fsf@stressinduktion.org> <87fuc1q7en.fsf@stressinduktion.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Cc: Hannes Frederic Sowa , Saeed Mahameed , Saeed Mahameed , "David S. Miller" , Linux Netdev List To: Alexander Duyck Return-path: Received: from mail-qk0-f175.google.com ([209.85.220.175]:35224 "EHLO mail-qk0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752462AbdIFQRw (ORCPT ); Wed, 6 Sep 2017 12:17:52 -0400 Received: by mail-qk0-f175.google.com with SMTP id r141so20663402qke.2 for ; Wed, 06 Sep 2017 09:17:52 -0700 (PDT) In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: On Tue, Sep 5, 2017 at 8:06 PM, Alexander Duyck wrote: > On Tue, Sep 5, 2017 at 2:13 PM, Tom Herbert wrote: >>> The situation with encapsulation is even more complicated: >>> >>> We are basically only interested in the UDP/vxlan/Ethernet/IP/UDP >>> constellation. If we do the fragmentation inside the vxlan tunnel and >>> carry over the skb hash to all resulting UDP/vxlan packets source ports, >>> we are fine and reordering on the receiver NIC won't happen in this >>> case. If the fragmentation happens on the outer UDP header, this will >>> result in reordering of the inner L2 flow. Unfortunately this depends on >>> how the vxlan tunnel was set up, how other devices do that and (I >>> believe so) on the kernel version. >>> >> This really isn't that complicated. The assumption that an IP network >> always delivers packets in order is simply wrong. The inventors of >> VXLAN must have know full well that when you use IP, packets can and >> eventually will be delivered out of order. This isn't just because of >> fragmentation, there are many other reasons that packets can be >> delivered OOO. This also must have been known when IP/GRE and any >> other protocol that carries L2 over IP was invented. If OOO is an >> issue for these protocols then they need to be fixed-- this is not a >> concern with IP protocol nor the stack. >> >> Tom > > As far as a little background on the original patch I believe the > issue that was fixed by the patch was a video streaming application > that was sending/receiving a mix of fragmented and non-fragmented > packets. Receiving them out of order due to the fragmentation was > causing issues with stutters in the video and so we ended up disabling > UDP by default in the NICs listed. We decided to go that way as UDP > RSS was viewed as a performance optimization, while the out-of-order > problems were viewed as a functionality issue. > Hi Alex, Thanks for the details! Were you able to find the root cause for this? In particular, it would be interesting to know if it is the kernel or device that introduced the jitter, or if it's the application that doesn't handle OOO well... Tom