TCP capture effect (was Re: Linux TCP impotency)

TCP capture effect (was Re: Linux TCP impotency)

[Samuel Meder]

Alan Cox wrote:

> > causes the earlier started one to survive and the later to
> > starve. Running bcp instead of the second (which uses UDP) at
> > 11000 bytes per second caused the utilization in both directions
> > to go up nearly to 100%.  
> > 
> > Is this a normal TCP stack behaviour? 
>
> Yes. TCP is not fair. Look up 'capture effect' if you want to know more. 


I'm seeing a similar effect myself. When I use all my available sdsl
bandwidth (say doing a bulk data transfer), DNS lookups will often
time out. This is with the default buffer settings/2.4.4. 
I'm curious about this effect so I've been trying to find information
on this and while I can find lots of information on the Ethernet
capture effect there doesn't seem to be anything on the TCP capture
effect. Could someone point me at an explanation of this effect?


Thanks

/Sam Meder

Re: TCP capture effect (was Re: Linux TCP impotency)

[Alan Cox]

> I'm curious about this effect so I've been trying to find information
> on this and while I can find lots of information on the Ethernet
> capture effect there doesn't seem to be anything on the TCP capture
> effect. Could someone point me at an explanation of this effect?

it is exactly the same thing but the backoffs are the TCP level backoffs and
the collisions are between TCP streams. There are statistical approaches to
fairness used in routers (a local capture effect annoyance to you
is rather bad news on backbones).

Alan

Re: TCP capture effect (was Re: Linux TCP impotency)

[Andi Kleen]

On Mon, May 14, 2001 at 04:15:12PM -0500, Samuel Meder wrote:
> I'm seeing a similar effect myself. When I use all my available sdsl
> bandwidth (say doing a bulk data transfer), DNS lookups will often
> time out. This is with the default buffer settings/2.4.4. 

The problem is that the DNS resolver in glibc has a too low setting
for retransmits before giving up (4-5 or so only, it's also a big problem
together with dynamic IP and dial-on-demand where the socket address rewriting
needs some time) Fix is to change glibc to retransmit more often and 
recompile it or alternatively use a local dns proxy that does more aggressive
retransmits (that won't fix the low timeout though)

> I'm curious about this effect so I've been trying to find information
> on this and while I can find lots of information on the Ethernet
> capture effect there doesn't seem to be anything on the TCP capture
> effect. Could someone point me at an explanation of this effect?

I also don't know that term, but the basic problem is that TCP only slows
down when its packets are dropped. Packets are dropped when a device queue
fills, and when one sender is much faster than the other the faster sender
often wins the race, while the packets of the slower one get dropped.
[this is a rough simplification; there have been many books and papers
written on queueing in the internet]

One problem is for example that Linux's default queue length is good
for ethernet, but not slower or higher latency links.
You can try to tune it using ifconfig device txqueuelen <number>. 
Try values between 5-30, default is 100.

DSL also has very asymetric bandwidth, if you're saturating e.g. the uplink
channel completely acks in the other direction will also have a hard
time to get through.

-Andi

Re: TCP capture effect :: estimate queue length ?

[God]

On Mon, 14 May 2001, Andi Kleen wrote:

[.....]

> Packets are dropped when a device queue
> fills, and when one sender is much faster than the other the faster sender
> often wins the race, while the packets of the slower one get dropped.

[.....]

Speaking of queues on routers/servers, does such a util exist that would
measure (even a rough estimate), what level of congestion (queueing) is
happening between point A and B ?  I'd be curious how badly congested some
things upstream from me are......   I know I can use ping or
traceroute ... but they don't report queueing or bursting.  Both measure
latency and packetloss ... short of stareing at a running ping that is
... <G>

Re: TCP capture effect :: estimate queue length ?

[Ralf Baechle]

On Mon, May 14, 2001 at 11:49:16PM -0400, God wrote:

> > Packets are dropped when a device queue
> > fills, and when one sender is much faster than the other the faster sender
> > often wins the race, while the packets of the slower one get dropped.
> 
> [.....]
> 
> Speaking of queues on routers/servers, does such a util exist that would
> measure (even a rough estimate), what level of congestion (queueing) is
> happening between point A and B ?  I'd be curious how badly congested some
> things upstream from me are......   I know I can use ping or
> traceroute ... but they don't report queueing or bursting.  Both measure
> latency and packetloss ... short of stareing at a running ping that is
> ... <G>

Pathchar, yet another Van Jacobsen toy does this.  Unfortunately the old
and rotten pre-version you can find in ftp.ee.lbl.gov:/pathchar/ is afaik
the last one.  In the past it served me well you find about how ISPs are
lying ...  100mbit backbone = fast ethernet in their computer room ...

  Ralf

Re: TCP capture effect :: estimate queue length ? :: pathchar

[God]

On Tue, 15 May 2001, Ralf Baechle wrote:

> On Mon, May 14, 2001 at 11:49:16PM -0400, God wrote:
> 
> > Speaking of queues on routers/servers, does such a util exist that would
> > measure (even a rough estimate), what level of congestion (queueing) is
> > happening between point A and B ?  I'd be curious how badly congested some
> > things upstream from me are......   I know I can use ping or
> > traceroute ... but they don't report queueing or bursting.  Both measure
> > latency and packetloss ... short of stareing at a running ping that is
> > ... <G>
> 
> Pathchar, yet another Van Jacobsen toy does this.  Unfortunately the old
> and rotten pre-version you can find in ftp.ee.lbl.gov:/pathchar/ is afaik
> the last one.  In the past it served me well you find about how ISPs are
> lying ...  100mbit backbone = fast ethernet in their computer room ...
> 


pathchar (last I used it), doesn't report queueing or bursting levels.  It
is purely a bandwidth estimator (and a dam fine one too!) ...  Could be
wrong though, I just don't recall seeing that as feature.

Re: TCP capture effect :: estimate queue length ?

[David S. Miller]

God writes:
 > Speaking of queues on routers/servers, does such a util exist that would
 > measure (even a rough estimate), what level of congestion (queueing) is
 > happening between point A and B ?

Not that I know of, but it is funny you mention this because this is
basically the kind of calculation the TCP Vegas congestion avoidance
scheme attempts to make.  At it's core, it is trying to estimate the
size of router queues from local machine to remote machine based upon
"congestion events" (packet drop, etc.).

Later,
David S. Miller
davem@redhat.com

Re: TCP capture effect :: estimate queue length ?

[God]

On Mon, 14 May 2001, David S. Miller wrote:

> God writes:
>  > Speaking of queues on routers/servers, does such a util exist that would
>  > measure (even a rough estimate), what level of congestion (queueing) is
>  > happening between point A and B ?
> 
> Not that I know of, but it is funny you mention this because this is
> basically the kind of calculation the TCP Vegas congestion avoidance
> scheme attempts to make.  At it's core, it is trying to estimate the
> size of router queues from local machine to remote machine based upon
> "congestion events" (packet drop, etc.).


Really? .. hmmm... might just have to go read up on it.  I have my own
ideas of how to go about doing it (pretty simple I think .. unless I'm
missing something other then coding ability .... heh). 

 My basic reason for wanting such a beast was to
convince one internet provider (who shall remain nameless), that from
at home *cough* to the gateway, was under extreame congestion.  After
finaly speaking with a tech who knew the difference between him and
a dip switch, he agreed there was a serious problem (queues
were exceeding 190/255 tx/rx, logs were showing the link upstream
from the gateway was loosing sync multiple times per hour .. etc etc).

The path to that however was long and very frustrating.  Trying to explain
that
although you can ping me fine and I can ping you fine, you still have a
serious problem; can be very confusing to some.


Thanks for the pointer :)