netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: David Newall <davidn@davidnewall.com>
To: Ingo Molnar <mingo@elte.hu>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	David Miller <davem@davemloft.net>,
	akpm@linux-foundation.org, netdev@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Stefan Richter <stefanr@s5r6.in-berlin.de>
Subject: Re: [TCP bug] stuck distcc connections in latest -git
Date: Tue, 22 Jul 2008 23:15:38 +0930	[thread overview]
Message-ID: <4885E482.5020502@davidnewall.com> (raw)
In-Reply-To: <20080722112133.GA6575@elte.hu>

Ingo Molnar wrote:
> hm, the distcc TCP hangs are back:
>   

The missing four client-side connections are more interesting than the
unsent data.

> I.e. the client side send-queue is stuck in established state, server 
> side thinks it's a proper established connection. Nobody makes any 
> progress.
>   

I might be missing something obvious, but I don't think there's anything
unusual in the three sessions displayed on the client.  They should be
"ESTABLISHED", and on the server, too, just as they are.

> Also note the final 4 connections on the server side - those are not 
> present on the client box.
>   

Now this is interesting.  I would be much more interested in how the
client's sides for these disappeared.

> The hung condition seemed permanent (i waited a couple of minutes).
>   

Not nearly long enough.  Retransmits can be sent as infrequently as per
180 seconds.  I think there's an argument to use one of the the various
patches that reduce your TCP_RTO_MAX, for example OBATA Noboru's
(http://marc.info/?l=linux-netdev&m=118422471428855): you don't have to
wait unreasonably long before seeing a retransmit.  Remember, three minutes!


> I retried the same build 10 times and it would not reproduce - so this 
> again is a hard to reproduce condition. (and there's no chance to get a 
> proper tcpdump either, at these traffic levels)

You really should start that capture, and on both client and server. 
You don't need to dump everything, only traffic to or from server:distcc.

  reply	other threads:[~2008-07-22 13:45 UTC|newest]

Thread overview: 204+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-07-20 17:44 [GIT]: Networking David Miller
2008-07-20 17:59 ` Arjan van de Ven
2008-07-20 23:52   ` David Miller
2008-07-21 20:32   ` David Miller
2008-07-21  0:54 ` Linus Torvalds
2008-07-21  1:03   ` David Miller
2008-07-21  1:09     ` Alexey Dobriyan
2008-07-21  1:14       ` David Miller
2008-07-21  1:22         ` Alexey Dobriyan
2008-07-21  2:40         ` Alexey Dobriyan
2008-07-21  2:48           ` David Miller
2008-07-21  5:11             ` David Miller
2008-07-21  9:48               ` Alexander Beregalov
2008-07-21 10:16                 ` Ben Hutchings
2008-07-21 15:35                   ` David Miller
2008-07-21 16:04                     ` Alexander Beregalov
2008-07-21 11:57               ` Alexey Dobriyan
2008-07-21 15:27                 ` David Miller
2008-07-21 16:49               ` Linus Torvalds
2008-07-21 16:53                 ` David Miller
2008-07-21  1:20     ` Patrick McHardy
2008-07-21 11:28       ` Stefan Richter
2008-07-21 11:45       ` James Morris
2008-07-21 12:05         ` Patrick McHardy
2008-07-21 17:28           ` David Miller
2008-07-21 17:40             ` Linus Torvalds
2008-07-21 20:33               ` Patrick McHardy
2008-07-23 23:42                 ` David Miller
2008-07-21  1:07   ` Linus Torvalds
2008-07-21  1:17     ` David Miller
2008-07-21  8:36 ` iwlwifi: fix build bug in "iwlwifi: fix LED stall" Ingo Molnar
2008-07-21 10:02   ` Winkler, Tomas
2008-07-21 10:53     ` Ingo Molnar
2008-07-21 12:12   ` [PATCH] iwlwifi: RS small compile warnings without CONFIG_IWLWIFI_DEBUG Tomas Winkler
2008-07-21 12:12     ` [PATCH] iwlwifi: " Tomas Winkler
2008-07-21 12:12       ` [PATCH] iwlwifi: compilation error when CONFIG_IWLWIFI_DEBUG is not set Tomas Winkler
2008-07-21 13:30 ` [crash, bisected] Kernel BUG at ffffffff8079afb1 (__netif_schedule()) Ingo Molnar
2008-07-21 13:45   ` [crash] BUG: unable to handle kernel NULL pointer dereference at 0000000000000370 Ingo Molnar
2008-07-21 14:30     ` Ingo Molnar
2008-07-21 15:04       ` Ingo Molnar
2008-07-21 15:24         ` David Miller
2008-07-21 18:18           ` Ian Schram
2008-07-21 19:06             ` Ingo Molnar
     [not found]               ` <20080721190646.GA19044-X9Un+BFzKDI@public.gmane.org>
2008-07-21 19:13                 ` Larry Finger
     [not found]                   ` <4884DFEE.4060003-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
2008-07-21 19:34                     ` Ingo Molnar
     [not found]                       ` <20080721193425.GA29287-X9Un+BFzKDI@public.gmane.org>
2008-07-21 19:43                         ` Larry Finger
     [not found]                           ` <4884E6E6.7010800-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
2008-07-21 19:47                             ` Linus Torvalds
2008-07-21 20:15                               ` David Miller
     [not found]                               ` <alpine.LFD.1.10.0807211246280.31863-5CScLwifNT1QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org>
2008-07-21 20:28                                 ` Larry Finger
2008-07-21 20:21                             ` David Miller
     [not found]                               ` <20080721.132100.128525904.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-21 20:38                                 ` Larry Finger
2008-07-21 20:46                                   ` David Miller
2008-07-21 20:51                                     ` Patrick McHardy
2008-07-21 21:01                                       ` David Miller
2008-07-21 21:06                                         ` Patrick McHardy
     [not found]                                           ` <4884FA3F.70109-dcUjhNyLwpNeoWH0uzbU5w@public.gmane.org>
2008-07-21 21:35                                             ` Patrick McHardy
     [not found]                                               ` <48850121.9090106-dcUjhNyLwpNeoWH0uzbU5w@public.gmane.org>
2008-07-21 21:42                                                 ` Patrick McHardy
2008-07-21 21:51                                                 ` Larry Finger
     [not found]                                                   ` <488504D9.5090100-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
2008-07-21 22:04                                                     ` Patrick McHardy
     [not found]                                                       ` <488507FB.9050101-dcUjhNyLwpNeoWH0uzbU5w@public.gmane.org>
2008-07-21 22:40                                                         ` Larry Finger
     [not found]                                                           ` <4885104A.2070201-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
2008-07-21 23:15                                                             ` David Miller
2008-07-22  6:34                                                               ` Larry Finger
     [not found]                                                                 ` <48857F74.2040406-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
2008-07-22 10:51                                                                   ` Jarek Poplawski
2008-07-22 11:32                                                                 ` David Miller
2008-07-22 12:52                                                                   ` Larry Finger
2008-07-22 20:43                                                                     ` David Miller
2008-07-22 13:02                                                                   ` Larry Finger
     [not found]                                                                     ` <4885DA49.50703-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
2008-07-22 14:53                                                                       ` Patrick McHardy
     [not found]                                                                         ` <4885F46A.30309-dcUjhNyLwpNeoWH0uzbU5w@public.gmane.org>
2008-07-22 21:17                                                                           ` David Miller
2008-07-22 16:39                                                                       ` Kernel WARNING: at net/core/dev.c:1330 __netif_schedule+0x2c/0x98() Larry Finger
     [not found]                                                                         ` <48860D4B.8070003-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org>
2008-07-22 17:20                                                                           ` Patrick McHardy
2008-07-22 18:39                                                                             ` Larry Finger
2008-07-22 18:44                                                                               ` Patrick McHardy
     [not found]                                                                                 ` <48862A76.8030307-dcUjhNyLwpNeoWH0uzbU5w@public.gmane.org>
2008-07-22 19:30                                                                                   ` Larry Finger
2008-07-22 23:04                                                                               ` David Miller
2008-07-23  6:20                                                                                 ` Jarek Poplawski
2008-07-23  7:59                                                                                   ` David Miller
     [not found]                                                                                     ` <20080723.005921.113868915.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-23  8:54                                                                                       ` Jarek Poplawski
     [not found]                                                                                         ` <20080723085452.GB4561-8HppEYmqbBCE+EvaaNYduQ@public.gmane.org>
2008-07-23  9:03                                                                                           ` Peter Zijlstra
2008-07-23  9:35                                                                                             ` Jarek Poplawski
     [not found]                                                                                               ` <20080723093459.GC4561-8HppEYmqbBCE+EvaaNYduQ@public.gmane.org>
2008-07-23  9:50                                                                                                 ` Peter Zijlstra
2008-07-23 10:13                                                                                                   ` Jarek Poplawski
2008-07-23 10:58                                                                                                     ` Peter Zijlstra
2008-07-23 11:35                                                                                                       ` Jarek Poplawski
     [not found]                                                                                                         ` <20080723113519.GE4561-8HppEYmqbBCE+EvaaNYduQ@public.gmane.org>
2008-07-23 11:49                                                                                                           ` Jarek Poplawski
2008-07-23 20:16                                                                                                             ` David Miller
2008-07-23 20:43                                                                                                               ` Jarek Poplawski
     [not found]                                                                                                                 ` <20080723204335.GA14788-dUp/P3zyUuaNj9Bq2fkWzw@public.gmane.org>
2008-07-23 20:55                                                                                                                   ` David Miller
     [not found]                                                                                                               ` <20080723.131607.79681752.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-24  9:10                                                                                                                 ` Peter Zijlstra
2008-07-24  9:20                                                                                                                   ` David Miller
     [not found]                                                                                                                     ` <20080724.022040.23129457.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-24  9:27                                                                                                                       ` Peter Zijlstra
2008-07-24  9:32                                                                                                                         ` David Miller
     [not found]                                                                                                                           ` <20080724.023210.229338550.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-24 10:08                                                                                                                             ` Peter Zijlstra
2008-07-24 10:38                                                                                                                               ` Nick Piggin
     [not found]                                                                                                                                 ` <200807242038.36693.nickpiggin-/E1597aS9LT0CCvOHzKKcA@public.gmane.org>
2008-07-24 10:55                                                                                                                                   ` Miklos Szeredi
     [not found]                                                                                                                                     ` <E1KLyUI-0006ZS-Cg-8f8m9JG5TPIdUIPVzhDTVZP2KDSNp7ea@public.gmane.org>
2008-07-24 11:06                                                                                                                                       ` Nick Piggin
     [not found]                                                                                                                                         ` <200807242106.52672.nickpiggin-/E1597aS9LT0CCvOHzKKcA@public.gmane.org>
2008-08-01 21:10                                                                                                                                           ` Paul E. McKenney
2008-07-24 10:59                                                                                                                                   ` Peter Zijlstra
2008-08-01 21:10                                                                                                                                 ` Paul E. McKenney
2008-07-23 20:14                                                                                                       ` David Miller
     [not found]                                                                                                         ` <20080723.131441.200166513.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-24  7:00                                                                                                           ` Peter Zijlstra
2008-07-25 17:04                                                                                                         ` Ingo Oeser
     [not found]                                                                                                           ` <200807251904.37302.netdev-BkyiQQGWkgE@public.gmane.org>
2008-07-25 18:36                                                                                                             ` Jarek Poplawski
2008-07-25 19:16                                                                                                               ` Johannes Berg
     [not found]                                                                                                                 ` <1217013384.4758.5.camel-YfaajirXv214zXjbi5bjpg@public.gmane.org>
2008-07-25 19:34                                                                                                                   ` Jarek Poplawski
2008-07-25 19:36                                                                                                                     ` Johannes Berg
2008-07-25 20:01                                                                                                                       ` Jarek Poplawski
     [not found]                                                                                                                         ` <20080725200137.GC3107-dUp/P3zyUuaNj9Bq2fkWzw@public.gmane.org>
2008-07-26  9:18                                                                                                                           ` David Miller
2008-07-26 10:53                                                                                                                             ` Jarek Poplawski
     [not found]                                                                                                                             ` <20080726.021846.236624483.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-26 13:18                                                                                                                               ` Jarek Poplawski
     [not found]                                                                                                                                 ` <20080726131838.GA2873-dUp/P3zyUuaNj9Bq2fkWzw@public.gmane.org>
2008-07-27  0:34                                                                                                                                   ` David Miller
     [not found]                                                                                                                                     ` <20080726.173434.48036095.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-27 20:37                                                                                                                                       ` Jarek Poplawski
     [not found]                                                                                                                                         ` <20080727203757.GA2527-dUp/P3zyUuaNj9Bq2fkWzw@public.gmane.org>
2008-07-31 12:29                                                                                                                                           ` David Miller
     [not found]                                                                                                                                             ` <20080731.052932.110299354.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-07-31 12:38                                                                                                                                               ` Nick Piggin
2008-07-31 12:44                                                                                                                                                 ` David Miller
2008-08-01  4:27                                                                                                                                             ` David Miller
     [not found]                                                                                                                                               ` <20080731.212729.138736823.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
2008-08-01  7:09                                                                                                                                                 ` Peter Zijlstra
2008-08-01  6:48                                                                                                                                             ` Jarek Poplawski
     [not found]                                                                                                                                               ` <20080801064810.GA4435-8HppEYmqbBCE+EvaaNYduQ@public.gmane.org>
2008-08-01  7:00                                                                                                                                                 ` David Miller
2008-08-01  7:01                                                                                                                                               ` Jarek Poplawski
     [not found]                                                                                                                                                 ` <20080801070150.GB4435-8HppEYmqbBCE+EvaaNYduQ@public.gmane.org>
2008-08-01  7:01                                                                                                                                                   ` David Miller
2008-08-01  7:41                                                                                                                                                     ` Jarek Poplawski
2008-07-25  6:20                                     ` [lockdep warning] AOE / networking: aoenet_xmit: noop_qdisc.q.lock, INFO: inconsistent lock state at 0000000000000370 Ingo Molnar
2008-07-25  6:25                                       ` David Miller
2008-07-25  7:26                                         ` Ingo Molnar
2008-07-25  8:23                                           ` David Miller
     [not found]       ` <20080721143023.GA32451-X9Un+BFzKDI@public.gmane.org>
2008-07-21 15:10         ` [crash] BUG: unable to handle kernel NULL pointer dereference " David Miller
2008-07-21 18:23     ` [crash] kernel BUG at net/core/dev.c:1328! Ingo Molnar
2008-07-21 18:35       ` Linus Torvalds
2008-07-21 18:46         ` Ingo Molnar
2008-07-21 19:30           ` Ingo Molnar
2008-07-22 11:21           ` [TCP bug] stuck distcc connections in latest -git Ingo Molnar
2008-07-22 13:45             ` David Newall [this message]
2008-07-22 13:57               ` Ingo Molnar
2008-07-22 14:54                 ` David Newall
2008-07-22 15:34                   ` Ingo Molnar
2008-07-22 21:12                     ` Willy Tarreau
2008-07-23  8:26                       ` Ingo Molnar
2008-07-24  6:04                         ` [TCP bug, regression] " Ingo Molnar
2008-07-24  6:32                           ` Ingo Molnar
2008-07-24  7:33                             ` Willy Tarreau
2008-07-24  8:35                               ` Ingo Molnar
2008-07-24  7:53                             ` Herbert Xu
2008-07-24  8:24                               ` Willy Tarreau
2008-07-24  8:27                               ` Ingo Molnar
2008-07-24  8:36                                 ` David Miller
2008-07-24  9:05                           ` Herbert Xu
2008-07-24  9:22                             ` David Miller
2008-07-24  9:34                               ` Ingo Molnar
2008-07-24 11:56                                 ` [regression] nf_iterate(), BUG: unable to handle kernel NULL pointer dereference Ingo Molnar
2008-07-24 11:59                                   ` Ingo Molnar
2008-07-24 12:03                                     ` Patrick McHardy
2008-07-24 12:22                                       ` Herbert Xu
2008-07-24 12:40                                         ` Pekka Enberg
2008-07-24 12:50                                           ` Herbert Xu
2008-07-24 12:56                                             ` Nick Piggin
2008-07-24 13:04                                               ` Herbert Xu
2008-07-24 13:13                                                 ` Nick Piggin
2008-07-24 13:32                                                   ` Pekka Enberg
2008-07-24 19:21                                                     ` Matt Mackall
2008-07-25  9:09                                                       ` Nick Piggin
2008-07-24 13:11                                             ` Matt Mackall
2008-07-24 14:37                                               ` Herbert Xu
2008-07-24 17:47                                                 ` Matt Mackall
2008-07-25  1:39                                                   ` Herbert Xu
2008-07-25  2:59                                                     ` Matt Mackall
2008-07-24 12:44                                       ` Pekka Enberg
2008-07-24 12:49                                         ` Patrick McHardy
2008-07-24 13:23                                           ` Pekka Enberg
2008-07-24 13:31                                             ` Patrick McHardy
2008-07-24 13:34                                               ` Pekka Enberg
2008-07-24 18:51                                                 ` Andrew Morton
2008-07-24 18:55                                                   ` Pekka Enberg
2008-07-24 20:58                                                     ` David Miller
2008-07-25  8:02                                                     ` Dieter Ries
2008-07-25 10:41                                                       ` Pekka Enberg
2008-07-24 19:35                                                   ` Ingo Molnar
2008-07-26 16:09                                                     ` Patrick McHardy
2008-07-26 17:34                                                       ` Ingo Molnar
2008-07-26 13:43                                                   ` Patrick McHardy
2008-07-24 21:13                                           ` Linus Torvalds
2008-07-24 22:09                                             ` David Miller
2008-07-26 13:47                                             ` Patrick McHardy
2008-08-01 21:10                                             ` Paul E. McKenney
2008-07-24 14:23                                     ` Ingo Molnar
2008-07-24 15:23                                       ` Patrick McHardy
2008-07-24 15:32                                         ` Ingo Molnar
2008-07-24 15:34                                           ` Patrick McHardy
2008-07-24 18:00                                           ` Krzysztof Oledzki
2008-07-24 13:01                               ` [TCP bug, regression] stuck distcc connections in latest -git Willy Tarreau
2008-07-24  9:25                             ` Ingo Molnar
2008-07-24  9:29                               ` David Miller
2008-07-24 11:12                               ` Herbert Xu
2008-07-24  9:36                             ` Ilpo Järvinen
2008-07-24 10:03                               ` Ilpo Järvinen
2008-07-21 19:00         ` [crash] kernel BUG at net/core/dev.c:1328! David Miller
2008-07-21 19:20           ` Stefan Richter
2008-07-21 20:11             ` David Miller
2008-07-21 21:26               ` Stefan Richter
2008-07-21 19:44           ` Ingo Molnar
2008-07-21 20:20             ` David Miller
2008-07-21 15:07   ` [crash, bisected] Kernel BUG at ffffffff8079afb1 (__netif_schedule()) David Miller
2008-07-21 13:50 ` [GIT]: Networking Ingo Molnar
2008-07-21 14:15   ` Stefan Richter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4885E482.5020502@davidnewall.com \
    --to=davidn@davidnewall.com \
    --cc=akpm@linux-foundation.org \
    --cc=davem@davemloft.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=netdev@vger.kernel.org \
    --cc=stefanr@s5r6.in-berlin.de \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).