From: Eric Dumazet <eric.dumazet@gmail.com>
To: Simon Kirby <sim@hostway.ca>
Cc: netdev@vger.kernel.org, Simon Horman <horms@verge.net.au>
Subject: Re: TSO/GRO/LRO/somethingO breaks LVS on 2.6.36
Date: Fri, 03 Dec 2010 12:29:03 +0100 [thread overview]
Message-ID: <1291375743.2897.141.camel@edumazet-laptop> (raw)
In-Reply-To: <20101203103447.GA29714@hostway.ca>
Le vendredi 03 décembre 2010 à 02:34 -0800, Simon Kirby a écrit :
> Hello!
>
> We upgraded some LVS (DR) servers from 2.6.35 to 2.6.36 on tg3 cards
> (partno(BCM95721) rev 4201) with VLAN tags in use, to think that
> everything looked great, but in fact...
>
> LVS was receiving magically-merged TCP packets which it tried to forward
> on to the real server, only to get annoyed at itself for trying to
> forward a packet bigger than the device MTU:
>
> IP A.47376 > B.529: . 175488:176936(1448) ack 1 win 92 <nop,nop,timestamp 52737308 29656081>
> IP A.47376 > B.529: . 176936:179832(2896) ack 1 win 92 <nop,nop,timestamp 52737325 29656098>
> IP B > A: ICMP B unreachable - need to frag (mtu 1500), length 556
>
Hi Simon
This is a tcpdump on A ?
Could you take it also on B ?
tcpdump displays large buffers, but they should be split (of course)
when sent on wire.
> This caused packet loss for any merged frames, which caused abysmal
> performance for uploads via the LVS server. Local performance to or
> from the box is still fine, because the stack doesn't care, only the
> forwarding part of LVS is running into the problem.
>
> Furthermore, disabling _everything_ reported by ethtool -k doesn't seem
> to change the result, even if I down/up the interface after, and even if
> I try on every single interface including the VLANned ones. This seems
> to be another bug. Reverting to 2.6.35 makes it all work again.
>
> Possibly related to commit 7fe876af921d1d2bc8353e0062c10ff35e902653
>
> So how should this be fixed? Should LVS be taught to fragment, or must
> we disable the merging in this case? It seems like it would work well if
> the sending side could do the same offload in reverse, but I'm not sure
> if that would be possible.
>
> Simon-
I believe Simon Horman has some patches for GRO and LVS.
Please send the results of "ethtool -k eth0" on all your nics / vlans ?
For TSO, I am not sure why and where it could matter...
next prev parent reply other threads:[~2010-12-03 11:29 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-12-03 10:34 TSO/GRO/LRO/somethingO breaks LVS on 2.6.36 Simon Kirby
2010-12-03 11:29 ` Eric Dumazet [this message]
2010-12-03 12:36 ` Simon Horman
2010-12-03 12:42 ` Simon Horman
2010-12-03 13:27 ` Ben Hutchings
2010-12-03 13:39 ` Simon Horman
2010-12-09 2:35 ` Simon Kirby
2010-12-09 2:51 ` Eric Dumazet
2010-12-22 23:39 ` Simon Kirby
2011-01-13 6:34 ` Simon Horman
2011-01-27 0:48 ` Simon Kirby
2011-01-27 1:36 ` Simon Horman
2011-01-27 7:42 ` Eric Dumazet
2011-01-27 14:42 ` Simon Horman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1291375743.2897.141.camel@edumazet-laptop \
--to=eric.dumazet@gmail.com \
--cc=horms@verge.net.au \
--cc=netdev@vger.kernel.org \
--cc=sim@hostway.ca \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.