From: Ben Greear <greearb@candelatech.com>
To: Stephen Hemminger <shemminger@vyatta.com>
Cc: netdev <netdev@vger.kernel.org>
Subject: Re: TCP funny-ness when over-driving a 1Gbps link.
Date: Thu, 19 May 2011 16:20:45 -0700 [thread overview]
Message-ID: <4DD5A5CD.7040303@candelatech.com> (raw)
In-Reply-To: <20110519161827.2ba4b40e@nehalam>
On 05/19/2011 04:18 PM, Stephen Hemminger wrote:
> On Thu, 19 May 2011 15:47:14 -0700
> Ben Greear<greearb@candelatech.com> wrote:
>
>> I noticed something that struck me as a bit weird today,
>> but perhaps it's normal.
>>
>> I was using our application to create 3 TCP streams from one port to
>> another (1Gbps, igb driver), running through a network emulator.
>> Traffic is flowing bi-directional in each connection.
>>
>> I am doing 24k byte writes per system call. I tried 100ms, 10ms, and 1ms
>> latency (one-way) in the emulator, but behaviour is similar in each case.
>> The rest of this info was gathered with 1ms delay in the emulator.
>>
>> If I ask all 3 connections to run 1Gbps, netstat shows 30+GB in the
>> sending queues and 1+ second latency (user-space to user-space). Aggregate
>> throughput is around 700Mbps in each direction.
>>
>> But, if I ask each of the connections to run at 300Mbps, latency averages
>> 2ms and each connection runs right at 300Mbps (950Mbps or so on the wire).
>>
>> It seems that when you over-drive the link, things back up and perform
>> quite badly over-all.
>>
>> This is a core-i7 3.2Ghz with 12GB RAM, Fedora 14, 2.6.38.6 kernel
>> (with some hacks), 64-bit OS and user-space app. Quick testing on 2.6.36.3
>> showed similar results, so I don't think it's a regression.
>>
>> I am curious if others see similar results?
>>
>> Thanks,
>> Ben
>>
>
> If you overdrive, TCP expects your network emulator to have
> a some but limited queueing (like a real router).
The emulator is fine, it's not being over-driven (and has limited queueing if it was
being over-driven). The queues that are backing up are in the tcp sockets on the
sending machine.
But, just to make sure, I'll re-run the test with a looped back cable...
Ben
>
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
next prev parent reply other threads:[~2011-05-19 23:20 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-05-19 22:47 TCP funny-ness when over-driving a 1Gbps link Ben Greear
2011-05-19 23:18 ` Stephen Hemminger
2011-05-19 23:20 ` Ben Greear [this message]
2011-05-19 23:42 ` Ben Greear
2011-05-20 0:05 ` Rick Jones
2011-05-20 0:12 ` Ben Greear
2011-05-20 0:24 ` Rick Jones
2011-05-20 0:37 ` Ben Greear
2011-05-20 0:46 ` Rick Jones
2011-05-20 3:39 ` Ben Greear
2011-05-20 21:33 ` TCP funny-ness when over-driving a 1Gbps link (and wifi) Ben Greear
2011-05-26 15:28 ` Chris Friesen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4DD5A5CD.7040303@candelatech.com \
--to=greearb@candelatech.com \
--cc=netdev@vger.kernel.org \
--cc=shemminger@vyatta.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.