* 8139cp TX stall, timeout, failed recovery
@ 2012-11-24 18:39 David Woodhouse
2012-11-24 21:33 ` Francois Romieu
2012-11-24 21:39 ` 8139cp TX stall, timeout, failed recovery David Woodhouse
0 siblings, 2 replies; 8+ messages in thread
From: David Woodhouse @ 2012-11-24 18:39 UTC (permalink / raw)
To: netdev; +Cc: romieu, jasowang, gilboad, jgarzik
[-- Attachment #1: Type: text/plain, Size: 33253 bytes --]
When I stress it hard, I see a Tx timeout that it doesn't recover from.
This is 3.6.7, with or without my recent init path patch (and with or
without my BQL patch).
To reproduce this, I send a *large* flood of packets (ping -l 1000) from
inside my network to the ADSL router. Its internal-facing interface is
an 8139cp. It seems to work best if the packets are destined for the
outside world over the ADSL lines, rather than the router itself. But
it's hard to tell. I need between 10 and 30 runs of 'ping -l 1000
$outside' before it happens. Or sometimes more...
Log shows it all running fine, followed by a flood of incoming packets
starting at around 28498.05. At 28498.060901 we queue a new tx, and
*immediately* afterwards (28498.060936) we get an interrupt with status
0x0080 (Tx Descriptor Unavailable, but no Tx Done). That's the last
tx-related interrupt we ever get.
This seems to be a consistent pattern — when we get that 0x0080
interrupt and it dies, it's *very* soon after queueing a new tx:
8139dead2.cap-[27538.841952] 8139cp 0000:00:0b.0: eth1: tx queued, slot 54, skblen 98
8139dead2.cap:[27538.841985] 8139cp 0000:00:0b.0: eth1: intr, status 0080 cmd 0c cpcmd 002b
--
8139dead3a.cap-[28498.060901] 8139cp 0000:00:0b.0: eth1: tx queued, slot 12, skblen 98
8139dead3a.cap:[28498.060936] 8139cp 0000:00:0b.0: eth1: intr, status 0080 cmd 0c cpcmd 002b
--
8139dead4.cap-[30230.248780] 8139cp 0000:00:0b.0: eth1: tx queued, slot 27, skblen 98
8139dead4.cap:[30230.248816] 8139cp 0000:00:0b.0: eth1: intr, status 0080 cmd 0c cpcmd 002b
--
8139dead.cap-[ 228.119159] 8139cp 0000:00:0b.0: eth1: tx queued, slot 63, skblen 98
8139dead.cap:[ 228.119195] 8139cp 0000:00:0b.0: eth1: intr, status 0080 cmd 0c cpcmd 002b
Here's a full log of one of those, showing the failure to recover too...
Looking at it now... do we wrap around the Rx ring buffer at 28498.055
and process some of the packets more than once? Or is the hardware
genuinely keeping up with us as we suck out, descriptors 51-63, 0-63,
and 0-39 again all off the same interrupt? Perhaps there lies the root
of the problem?
[28497.924957] 8139cp 0000:00:0b.0: eth1: rx slot 44 status 0x320141d8 len 468
[28498.032253] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.032300] 8139cp 0000:00:0b.0: eth1: rx slot 45 status 0x32022062 len 94
[28498.040770] 8139cp 0000:00:0b.0: eth1: tx queued, slot 7, skblen 110
[28498.040816] 8139cp 0000:00:0b.0: eth1: intr, status 0484 cmd 0c cpcmd 002b
[28498.040836] 8139cp 0000:00:0b.0: eth1: tx done, slot 6
[28498.041048] 8139cp 0000:00:0b.0: eth1: tx queued, slot 8, skblen 1514
[28498.041090] 8139cp 0000:00:0b.0: eth1: intr, status 0484 cmd 0c cpcmd 002b
[28498.041108] 8139cp 0000:00:0b.0: eth1: tx done, slot 7
[28498.041177] 8139cp 0000:00:0b.0: eth1: tx queued, slot 9, skblen 1514
[28498.041218] 8139cp 0000:00:0b.0: eth1: intr, status 0484 cmd 0c cpcmd 002b
[28498.041236] 8139cp 0000:00:0b.0: eth1: tx done, slot 8
[28498.041677] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.041709] 8139cp 0000:00:0b.0: eth1: rx slot 46 status 0x3200e04e len 74
[28498.041846] 8139cp 0000:00:0b.0: eth1: tx queued, slot 10, skblen 533
[28498.041882] 8139cp 0000:00:0b.0: eth1: intr, status 0484 cmd 0c cpcmd 002b
[28498.041901] 8139cp 0000:00:0b.0: eth1: tx done, slot 9
[28498.043028] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.043056] 8139cp 0000:00:0b.0: eth1: rx slot 47 status 0x3200e04e len 74
[28498.043264] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.043291] 8139cp 0000:00:0b.0: eth1: rx slot 48 status 0x3200e04e len 74
[28498.043456] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.043483] 8139cp 0000:00:0b.0: eth1: rx slot 49 status 0x3200e04e len 74
[28498.043905] 8139cp 0000:00:0b.0: eth1: tx queued, slot 11, skblen 245
[28498.043941] 8139cp 0000:00:0b.0: eth1: intr, status 0484 cmd 0c cpcmd 002b
[28498.043960] 8139cp 0000:00:0b.0: eth1: tx done, slot 10
[28498.051129] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.051165] 8139cp 0000:00:0b.0: eth1: rx slot 50 status 0x32036066 len 98
[28498.051333] 8139cp 0000:00:0b.0: eth1: rx slot 51 status 0x32036066 len 98
[28498.051394] 8139cp 0000:00:0b.0: eth1: rx slot 52 status 0x32036066 len 98
[28498.051451] 8139cp 0000:00:0b.0: eth1: rx slot 53 status 0x32036066 len 98
[28498.051523] 8139cp 0000:00:0b.0: eth1: rx slot 54 status 0x32036066 len 98
[28498.051575] 8139cp 0000:00:0b.0: eth1: rx slot 55 status 0x32036066 len 98
[28498.051623] 8139cp 0000:00:0b.0: eth1: rx slot 56 status 0x32036066 len 98
[28498.051672] 8139cp 0000:00:0b.0: eth1: rx slot 57 status 0x32036066 len 98
[28498.051720] 8139cp 0000:00:0b.0: eth1: rx slot 58 status 0x32036066 len 98
[28498.051771] 8139cp 0000:00:0b.0: eth1: rx slot 59 status 0x32036066 len 98
[28498.051821] 8139cp 0000:00:0b.0: eth1: rx slot 60 status 0x32036066 len 98
[28498.051871] 8139cp 0000:00:0b.0: eth1: rx slot 61 status 0x32036066 len 98
[28498.051962] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.051989] 8139cp 0000:00:0b.0: eth1: rx slot 62 status 0x32036066 len 98
[28498.052041] 8139cp 0000:00:0b.0: eth1: rx slot 63 status 0x72036066 len 98
[28498.052089] 8139cp 0000:00:0b.0: eth1: rx slot 0 status 0x32036066 len 98
[28498.052139] 8139cp 0000:00:0b.0: eth1: rx slot 1 status 0x32036066 len 98
[28498.052189] 8139cp 0000:00:0b.0: eth1: rx slot 2 status 0x32036066 len 98
[28498.052238] 8139cp 0000:00:0b.0: eth1: rx slot 3 status 0x32036066 len 98
[28498.052287] 8139cp 0000:00:0b.0: eth1: rx slot 4 status 0x32036066 len 98
[28498.052370] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.052396] 8139cp 0000:00:0b.0: eth1: rx slot 5 status 0x32036066 len 98
[28498.052446] 8139cp 0000:00:0b.0: eth1: rx slot 6 status 0x32036066 len 98
[28498.052496] 8139cp 0000:00:0b.0: eth1: rx slot 7 status 0x32036066 len 98
[28498.052562] 8139cp 0000:00:0b.0: eth1: rx slot 8 status 0x32036066 len 98
[28498.052612] 8139cp 0000:00:0b.0: eth1: rx slot 9 status 0x32036066 len 98
[28498.052661] 8139cp 0000:00:0b.0: eth1: rx slot 10 status 0x32036066 len 98
[28498.052711] 8139cp 0000:00:0b.0: eth1: rx slot 11 status 0x32036066 len 98
[28498.052760] 8139cp 0000:00:0b.0: eth1: rx slot 12 status 0x32036066 len 98
[28498.052810] 8139cp 0000:00:0b.0: eth1: rx slot 13 status 0x32036066 len 98
[28498.052860] 8139cp 0000:00:0b.0: eth1: rx slot 14 status 0x32036066 len 98
[28498.052975] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.053001] 8139cp 0000:00:0b.0: eth1: rx slot 15 status 0x32036066 len 98
[28498.053053] 8139cp 0000:00:0b.0: eth1: rx slot 16 status 0x32036066 len 98
[28498.053103] 8139cp 0000:00:0b.0: eth1: rx slot 17 status 0x32036066 len 98
[28498.053151] 8139cp 0000:00:0b.0: eth1: rx slot 18 status 0x32036066 len 98
[28498.053201] 8139cp 0000:00:0b.0: eth1: rx slot 19 status 0x32036066 len 98
[28498.053249] 8139cp 0000:00:0b.0: eth1: rx slot 20 status 0x32036066 len 98
[28498.053299] 8139cp 0000:00:0b.0: eth1: rx slot 21 status 0x32036066 len 98
[28498.053348] 8139cp 0000:00:0b.0: eth1: rx slot 22 status 0x32036066 len 98
[28498.053397] 8139cp 0000:00:0b.0: eth1: rx slot 23 status 0x32036066 len 98
[28498.053446] 8139cp 0000:00:0b.0: eth1: rx slot 24 status 0x32036066 len 98
[28498.053495] 8139cp 0000:00:0b.0: eth1: rx slot 25 status 0x32036066 len 98
[28498.053561] 8139cp 0000:00:0b.0: eth1: rx slot 26 status 0x32036066 len 98
[28498.053611] 8139cp 0000:00:0b.0: eth1: rx slot 27 status 0x32036066 len 98
[28498.053661] 8139cp 0000:00:0b.0: eth1: rx slot 28 status 0x32036066 len 98
[28498.053711] 8139cp 0000:00:0b.0: eth1: rx slot 29 status 0x32036066 len 98
[28498.053760] 8139cp 0000:00:0b.0: eth1: rx slot 30 status 0x32036066 len 98
[28498.053810] 8139cp 0000:00:0b.0: eth1: rx slot 31 status 0x32036066 len 98
[28498.053860] 8139cp 0000:00:0b.0: eth1: rx slot 32 status 0x32036066 len 98
[28498.053947] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.053973] 8139cp 0000:00:0b.0: eth1: rx slot 33 status 0x32036066 len 98
[28498.054025] 8139cp 0000:00:0b.0: eth1: rx slot 34 status 0x32036066 len 98
[28498.054074] 8139cp 0000:00:0b.0: eth1: rx slot 35 status 0x32036066 len 98
[28498.054123] 8139cp 0000:00:0b.0: eth1: rx slot 36 status 0x32036066 len 98
[28498.054173] 8139cp 0000:00:0b.0: eth1: rx slot 37 status 0x32036066 len 98
[28498.054222] 8139cp 0000:00:0b.0: eth1: rx slot 38 status 0x32036066 len 98
[28498.054272] 8139cp 0000:00:0b.0: eth1: rx slot 39 status 0x32036066 len 98
[28498.054320] 8139cp 0000:00:0b.0: eth1: rx slot 40 status 0x32036066 len 98
[28498.054405] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.054432] 8139cp 0000:00:0b.0: eth1: rx slot 41 status 0x32036066 len 98
[28498.054482] 8139cp 0000:00:0b.0: eth1: rx slot 42 status 0x32036066 len 98
[28498.054548] 8139cp 0000:00:0b.0: eth1: rx slot 43 status 0x32036066 len 98
[28498.054597] 8139cp 0000:00:0b.0: eth1: rx slot 44 status 0x32036066 len 98
[28498.054648] 8139cp 0000:00:0b.0: eth1: rx slot 45 status 0x32036066 len 98
[28498.054697] 8139cp 0000:00:0b.0: eth1: rx slot 46 status 0x32036066 len 98
[28498.054747] 8139cp 0000:00:0b.0: eth1: rx slot 47 status 0x32036066 len 98
[28498.054796] 8139cp 0000:00:0b.0: eth1: rx slot 48 status 0x32036066 len 98
[28498.054846] 8139cp 0000:00:0b.0: eth1: rx slot 49 status 0x32036066 len 98
[28498.054895] 8139cp 0000:00:0b.0: eth1: rx slot 50 status 0x32036066 len 98
[28498.055030] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.055057] 8139cp 0000:00:0b.0: eth1: rx slot 51 status 0x32036066 len 98
[28498.055109] 8139cp 0000:00:0b.0: eth1: rx slot 52 status 0x32036066 len 98
[28498.055158] 8139cp 0000:00:0b.0: eth1: rx slot 53 status 0x32036066 len 98
[28498.055206] 8139cp 0000:00:0b.0: eth1: rx slot 54 status 0x32036066 len 98
[28498.055256] 8139cp 0000:00:0b.0: eth1: rx slot 55 status 0x32036066 len 98
[28498.055305] 8139cp 0000:00:0b.0: eth1: rx slot 56 status 0x32036066 len 98
[28498.055355] 8139cp 0000:00:0b.0: eth1: rx slot 57 status 0x32036066 len 98
[28498.055403] 8139cp 0000:00:0b.0: eth1: rx slot 58 status 0x32036066 len 98
[28498.055453] 8139cp 0000:00:0b.0: eth1: rx slot 59 status 0x32036066 len 98
[28498.055519] 8139cp 0000:00:0b.0: eth1: rx slot 60 status 0x32036066 len 98
[28498.055568] 8139cp 0000:00:0b.0: eth1: rx slot 61 status 0x32036066 len 98
[28498.055617] 8139cp 0000:00:0b.0: eth1: rx slot 62 status 0x32036066 len 98
[28498.055666] 8139cp 0000:00:0b.0: eth1: rx slot 63 status 0x72036066 len 98
[28498.055715] 8139cp 0000:00:0b.0: eth1: rx slot 0 status 0x32036066 len 98
[28498.055763] 8139cp 0000:00:0b.0: eth1: rx slot 1 status 0x32036066 len 98
[28498.055812] 8139cp 0000:00:0b.0: eth1: rx slot 2 status 0x32036066 len 98
[28498.055861] 8139cp 0000:00:0b.0: eth1: rx slot 3 status 0x32036066 len 98
[28498.055910] 8139cp 0000:00:0b.0: eth1: rx slot 4 status 0x32036066 len 98
[28498.055958] 8139cp 0000:00:0b.0: eth1: rx slot 5 status 0x32036066 len 98
[28498.056007] 8139cp 0000:00:0b.0: eth1: rx slot 6 status 0x32036066 len 98
[28498.056056] 8139cp 0000:00:0b.0: eth1: rx slot 7 status 0x32036066 len 98
[28498.056105] 8139cp 0000:00:0b.0: eth1: rx slot 8 status 0x32036066 len 98
[28498.056154] 8139cp 0000:00:0b.0: eth1: rx slot 9 status 0x32036066 len 98
[28498.056202] 8139cp 0000:00:0b.0: eth1: rx slot 10 status 0x32036066 len 98
[28498.056251] 8139cp 0000:00:0b.0: eth1: rx slot 11 status 0x32036066 len 98
[28498.056300] 8139cp 0000:00:0b.0: eth1: rx slot 12 status 0x32036066 len 98
[28498.056349] 8139cp 0000:00:0b.0: eth1: rx slot 13 status 0x32036066 len 98
[28498.056398] 8139cp 0000:00:0b.0: eth1: rx slot 14 status 0x32036066 len 98
[28498.056447] 8139cp 0000:00:0b.0: eth1: rx slot 15 status 0x32036066 len 98
[28498.056495] 8139cp 0000:00:0b.0: eth1: rx slot 16 status 0x32036066 len 98
[28498.056544] 8139cp 0000:00:0b.0: eth1: rx slot 17 status 0x32036066 len 98
[28498.056593] 8139cp 0000:00:0b.0: eth1: rx slot 18 status 0x32036066 len 98
[28498.056678] 8139cp 0000:00:0b.0: eth1: rx slot 19 status 0x32036066 len 98
[28498.056728] 8139cp 0000:00:0b.0: eth1: rx slot 20 status 0x32036066 len 98
[28498.056777] 8139cp 0000:00:0b.0: eth1: rx slot 21 status 0x32036066 len 98
[28498.056826] 8139cp 0000:00:0b.0: eth1: rx slot 22 status 0x32036066 len 98
[28498.056875] 8139cp 0000:00:0b.0: eth1: rx slot 23 status 0x32036066 len 98
[28498.056923] 8139cp 0000:00:0b.0: eth1: rx slot 24 status 0x32036066 len 98
[28498.056972] 8139cp 0000:00:0b.0: eth1: rx slot 25 status 0x32036066 len 98
[28498.057020] 8139cp 0000:00:0b.0: eth1: rx slot 26 status 0x32036066 len 98
[28498.057069] 8139cp 0000:00:0b.0: eth1: rx slot 27 status 0x32036066 len 98
[28498.057117] 8139cp 0000:00:0b.0: eth1: rx slot 28 status 0x32036066 len 98
[28498.057166] 8139cp 0000:00:0b.0: eth1: rx slot 29 status 0x32036066 len 98
[28498.057214] 8139cp 0000:00:0b.0: eth1: rx slot 30 status 0x32036066 len 98
[28498.057263] 8139cp 0000:00:0b.0: eth1: rx slot 31 status 0x32036066 len 98
[28498.057312] 8139cp 0000:00:0b.0: eth1: rx slot 32 status 0x32036066 len 98
[28498.057362] 8139cp 0000:00:0b.0: eth1: rx slot 33 status 0x32036066 len 98
[28498.057410] 8139cp 0000:00:0b.0: eth1: rx slot 34 status 0x32036066 len 98
[28498.057459] 8139cp 0000:00:0b.0: eth1: rx slot 35 status 0x32036066 len 98
[28498.057508] 8139cp 0000:00:0b.0: eth1: rx slot 36 status 0x32036066 len 98
[28498.057557] 8139cp 0000:00:0b.0: eth1: rx slot 37 status 0x32036066 len 98
[28498.057606] 8139cp 0000:00:0b.0: eth1: rx slot 38 status 0x32036066 len 98
[28498.057654] 8139cp 0000:00:0b.0: eth1: rx slot 39 status 0x32036066 len 98
[28498.057703] 8139cp 0000:00:0b.0: eth1: rx slot 40 status 0x32036066 len 98
[28498.057752] 8139cp 0000:00:0b.0: eth1: rx slot 41 status 0x32036066 len 98
[28498.057800] 8139cp 0000:00:0b.0: eth1: rx slot 42 status 0x32036066 len 98
[28498.057848] 8139cp 0000:00:0b.0: eth1: rx slot 43 status 0x32036066 len 98
[28498.057897] 8139cp 0000:00:0b.0: eth1: rx slot 44 status 0x32036066 len 98
[28498.057946] 8139cp 0000:00:0b.0: eth1: rx slot 45 status 0x32036066 len 98
[28498.057994] 8139cp 0000:00:0b.0: eth1: rx slot 46 status 0x32036066 len 98
[28498.058043] 8139cp 0000:00:0b.0: eth1: rx slot 47 status 0x32036066 len 98
[28498.058091] 8139cp 0000:00:0b.0: eth1: rx slot 48 status 0x32036066 len 98
[28498.058141] 8139cp 0000:00:0b.0: eth1: rx slot 49 status 0x32036066 len 98
[28498.058190] 8139cp 0000:00:0b.0: eth1: rx slot 50 status 0x32036066 len 98
[28498.058238] 8139cp 0000:00:0b.0: eth1: rx slot 51 status 0x32036066 len 98
[28498.058287] 8139cp 0000:00:0b.0: eth1: rx slot 52 status 0x32036066 len 98
[28498.058337] 8139cp 0000:00:0b.0: eth1: rx slot 53 status 0x32036066 len 98
[28498.058385] 8139cp 0000:00:0b.0: eth1: rx slot 54 status 0x32036066 len 98
[28498.058434] 8139cp 0000:00:0b.0: eth1: rx slot 55 status 0x32036066 len 98
[28498.058482] 8139cp 0000:00:0b.0: eth1: rx slot 56 status 0x32036066 len 98
[28498.058531] 8139cp 0000:00:0b.0: eth1: rx slot 57 status 0x32036066 len 98
[28498.058579] 8139cp 0000:00:0b.0: eth1: rx slot 58 status 0x32036066 len 98
[28498.058629] 8139cp 0000:00:0b.0: eth1: rx slot 59 status 0x32036066 len 98
[28498.058678] 8139cp 0000:00:0b.0: eth1: rx slot 60 status 0x32036066 len 98
[28498.058727] 8139cp 0000:00:0b.0: eth1: rx slot 61 status 0x32036066 len 98
[28498.058775] 8139cp 0000:00:0b.0: eth1: rx slot 62 status 0x32036066 len 98
[28498.058824] 8139cp 0000:00:0b.0: eth1: rx slot 63 status 0x72036066 len 98
[28498.058872] 8139cp 0000:00:0b.0: eth1: rx slot 0 status 0x32036066 len 98
[28498.058921] 8139cp 0000:00:0b.0: eth1: rx slot 1 status 0x32036066 len 98
[28498.058969] 8139cp 0000:00:0b.0: eth1: rx slot 2 status 0x32036066 len 98
[28498.059018] 8139cp 0000:00:0b.0: eth1: rx slot 3 status 0x32036066 len 98
[28498.059066] 8139cp 0000:00:0b.0: eth1: rx slot 4 status 0x32036066 len 98
[28498.059115] 8139cp 0000:00:0b.0: eth1: rx slot 5 status 0x32036066 len 98
[28498.059164] 8139cp 0000:00:0b.0: eth1: rx slot 6 status 0x32036066 len 98
[28498.059212] 8139cp 0000:00:0b.0: eth1: rx slot 7 status 0x32036066 len 98
[28498.059261] 8139cp 0000:00:0b.0: eth1: rx slot 8 status 0x32036066 len 98
[28498.059309] 8139cp 0000:00:0b.0: eth1: rx slot 9 status 0x32036066 len 98
[28498.059358] 8139cp 0000:00:0b.0: eth1: rx slot 10 status 0x32036066 len 98
[28498.059407] 8139cp 0000:00:0b.0: eth1: rx slot 11 status 0x32036066 len 98
[28498.059455] 8139cp 0000:00:0b.0: eth1: rx slot 12 status 0x32036066 len 98
[28498.059504] 8139cp 0000:00:0b.0: eth1: rx slot 13 status 0x32036066 len 98
[28498.059553] 8139cp 0000:00:0b.0: eth1: rx slot 14 status 0x32036066 len 98
[28498.059602] 8139cp 0000:00:0b.0: eth1: rx slot 15 status 0x32036066 len 98
[28498.059651] 8139cp 0000:00:0b.0: eth1: rx slot 16 status 0x32036066 len 98
[28498.059700] 8139cp 0000:00:0b.0: eth1: rx slot 17 status 0x32036066 len 98
[28498.059748] 8139cp 0000:00:0b.0: eth1: rx slot 18 status 0x32036066 len 98
[28498.059814] 8139cp 0000:00:0b.0: eth1: rx slot 19 status 0x32036066 len 98
[28498.059863] 8139cp 0000:00:0b.0: eth1: rx slot 20 status 0x32036066 len 98
[28498.059913] 8139cp 0000:00:0b.0: eth1: rx slot 21 status 0x32036066 len 98
[28498.059961] 8139cp 0000:00:0b.0: eth1: rx slot 22 status 0x32036066 len 98
[28498.060011] 8139cp 0000:00:0b.0: eth1: rx slot 23 status 0x32036066 len 98
[28498.060059] 8139cp 0000:00:0b.0: eth1: rx slot 24 status 0x32036066 len 98
[28498.060108] 8139cp 0000:00:0b.0: eth1: rx slot 25 status 0x32036066 len 98
[28498.060156] 8139cp 0000:00:0b.0: eth1: rx slot 26 status 0x32036066 len 98
[28498.060205] 8139cp 0000:00:0b.0: eth1: rx slot 27 status 0x32036066 len 98
[28498.060253] 8139cp 0000:00:0b.0: eth1: rx slot 28 status 0x32036066 len 98
[28498.060302] 8139cp 0000:00:0b.0: eth1: rx slot 29 status 0x32036066 len 98
[28498.060350] 8139cp 0000:00:0b.0: eth1: rx slot 30 status 0x32036066 len 98
[28498.060399] 8139cp 0000:00:0b.0: eth1: rx slot 31 status 0x32036066 len 98
[28498.060447] 8139cp 0000:00:0b.0: eth1: rx slot 32 status 0x32036066 len 98
[28498.060496] 8139cp 0000:00:0b.0: eth1: rx slot 33 status 0x32036066 len 98
[28498.060544] 8139cp 0000:00:0b.0: eth1: rx slot 34 status 0x32036066 len 98
[28498.060594] 8139cp 0000:00:0b.0: eth1: rx slot 35 status 0x32036066 len 98
[28498.060642] 8139cp 0000:00:0b.0: eth1: rx slot 36 status 0x32036066 len 98
[28498.060691] 8139cp 0000:00:0b.0: eth1: rx slot 37 status 0x32036066 len 98
[28498.060740] 8139cp 0000:00:0b.0: eth1: rx slot 38 status 0x32036066 len 98
[28498.060789] 8139cp 0000:00:0b.0: eth1: rx slot 39 status 0x32036066 len 98
[28498.060901] 8139cp 0000:00:0b.0: eth1: tx queued, slot 12, skblen 98
[28498.060936] 8139cp 0000:00:0b.0: eth1: intr, status 0080 cmd 0c cpcmd 002b
[28498.061814] 8139cp 0000:00:0b.0: eth1: tx queued, slot 13, skblen 98
[28498.064007] 8139cp 0000:00:0b.0: eth1: tx queued, slot 14, skblen 98
[28498.065750] 8139cp 0000:00:0b.0: eth1: tx queued, slot 15, skblen 98
[28498.067472] 8139cp 0000:00:0b.0: eth1: tx queued, slot 16, skblen 98
[28498.068062] 8139cp 0000:00:0b.0: eth1: tx queued, slot 17, skblen 79
[28498.068221] 8139cp 0000:00:0b.0: eth1: tx queued, slot 18, skblen 74
[28498.070711] 8139cp 0000:00:0b.0: eth1: tx queued, slot 19, skblen 98
[28498.072137] 8139cp 0000:00:0b.0: eth1: tx queued, slot 20, skblen 98
[28498.074369] 8139cp 0000:00:0b.0: eth1: tx queued, slot 21, skblen 98
[28498.075829] 8139cp 0000:00:0b.0: eth1: tx queued, slot 22, skblen 98
[28498.078074] 8139cp 0000:00:0b.0: eth1: tx queued, slot 23, skblen 98
[28498.079772] 8139cp 0000:00:0b.0: eth1: tx queued, slot 24, skblen 98
[28498.082254] 8139cp 0000:00:0b.0: eth1: tx queued, slot 25, skblen 98
[28498.120915] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.120946] 8139cp 0000:00:0b.0: eth1: rx slot 40 status 0x3201404e len 74
[28498.140144] 8139cp 0000:00:0b.0: eth1: tx queued, slot 26, skblen 74
[28498.182530] 8139cp 0000:00:0b.0: eth1: tx queued, slot 27, skblen 66
[28498.276742] 8139cp 0000:00:0b.0: eth1: tx queued, slot 28, skblen 79
[28498.421426] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.421454] 8139cp 0000:00:0b.0: eth1: rx slot 41 status 0x320141d8 len 468
[28498.485291] 8139cp 0000:00:0b.0: eth1: tx queued, slot 29, skblen 74
[28498.578432] 8139cp 0000:00:0b.0: eth1: tx queued, slot 30, skblen 66
[28498.646429] 8139cp 0000:00:0b.0: eth1: tx queued, slot 31, skblen 118
[28498.649979] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.650008] 8139cp 0000:00:0b.0: eth1: rx slot 42 status 0x32036094 len 144
[28498.650098] 8139cp 0000:00:0b.0: eth1: rx slot 43 status 0x32036096 len 146
[28498.650165] 8139cp 0000:00:0b.0: eth1: rx slot 44 status 0x32036094 len 144
[28498.696744] 8139cp 0000:00:0b.0: eth1: tx queued, slot 32, skblen 79
[28498.804309] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28498.804338] 8139cp 0000:00:0b.0: eth1: rx slot 45 status 0x320140a0 len 156
[28498.885404] 8139cp 0000:00:0b.0: eth1: tx queued, slot 33, skblen 125
[28499.086435] 8139cp 0000:00:0b.0: eth1: tx queued, slot 34, skblen 74
[28499.098211] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28499.098239] 8139cp 0000:00:0b.0: eth1: rx slot 46 status 0x32014046 len 66
[28499.108839] 8139cp 0000:00:0b.0: eth1: tx queued, slot 35, skblen 66
[28499.121567] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28499.121596] 8139cp 0000:00:0b.0: eth1: rx slot 47 status 0x3201404e len 74
[28499.121992] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28499.122018] 8139cp 0000:00:0b.0: eth1: rx slot 48 status 0x3200e0a0 len 156
[28499.139903] 8139cp 0000:00:0b.0: eth1: tx queued, slot 36, skblen 74
[28499.401749] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28499.401777] 8139cp 0000:00:0b.0: eth1: rx slot 49 status 0x320141d8 len 468
[28499.558938] 8139cp 0000:00:0b.0: eth1: tx queued, slot 37, skblen 66
[28499.775920] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28499.775948] 8139cp 0000:00:0b.0: eth1: rx slot 50 status 0x3203605e len 90
[28499.888841] 8139cp 0000:00:0b.0: eth1: tx queued, slot 38, skblen 116
[28500.286470] 8139cp 0000:00:0b.0: eth1: tx queued, slot 39, skblen 74
[28500.353205] 8139cp 0000:00:0b.0: eth1: tx queued, slot 40, skblen 66
[28501.125519] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28501.125553] 8139cp 0000:00:0b.0: eth1: rx slot 51 status 0x3201404e len 74
[28501.143803] 8139cp 0000:00:0b.0: eth1: tx queued, slot 41, skblen 74
[28501.373644] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28501.373673] 8139cp 0000:00:0b.0: eth1: rx slot 52 status 0x320141d8 len 468
[28501.530366] 8139cp 0000:00:0b.0: eth1: tx queued, slot 42, skblen 66
[28501.648615] 8139cp 0000:00:0b.0: eth1: tx queued, slot 43, skblen 112
[28501.888561] 8139cp 0000:00:0b.0: eth1: tx queued, slot 44, skblen 121
[28502.094862] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28502.094891] 8139cp 0000:00:0b.0: eth1: rx slot 53 status 0x32014046 len 66
[28502.105856] 8139cp 0000:00:0b.0: eth1: tx queued, slot 45, skblen 66
[28502.113535] 8139cp 0000:00:0b.0: eth1: tx queued, slot 46, skblen 94
[28502.221163] 8139cp 0000:00:0b.0: eth1: tx queued, slot 47, skblen 58
[28502.507301] SysRq : Changing Loglevel
[28502.511000] Loglevel set to 9
[28502.690951] 8139cp 0000:00:0b.0: eth1: tx queued, slot 48, skblen 74
[28502.725709] 8139cp 0000:00:0b.0: eth1: tx queued, slot 49, skblen 62
[28502.891993] 8139cp 0000:00:0b.0: eth1: tx queued, slot 50, skblen 116
[28502.997597] 8139cp 0000:00:0b.0: eth1: tx queued, slot 51, skblen 102
[28504.652076] 8139cp 0000:00:0b.0: eth1: tx queued, slot 52, skblen 118
[28504.886623] 8139cp 0000:00:0b.0: eth1: tx queued, slot 53, skblen 147
[28504.894552] 8139cp 0000:00:0b.0: eth1: tx queued, slot 54, skblen 123
[28505.133020] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28505.140825] 8139cp 0000:00:0b.0: eth1: rx slot 54 status 0x3201404e len 74
[28505.149255] 8139cp 0000:00:0b.0: eth1: tx queued, slot 55, skblen 147
[28505.167408] 8139cp 0000:00:0b.0: eth1: tx queued, slot 56, skblen 74
[28505.317183] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28505.325489] 8139cp 0000:00:0b.0: eth1: rx slot 55 status 0x320141d8 len 468
[28508.094647] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28508.101681] 8139cp 0000:00:0b.0: eth1: rx slot 56 status 0x32014046 len 66
[28508.913790] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28508.921135] 8139cp 0000:00:0b.0: eth1: rx slot 57 status 0x32022040 len 60
[28510.009002] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28510.016533] 8139cp 0000:00:0b.0: eth1: rx slot 58 status 0x3201406b len 103
[28510.239735] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28510.246717] 8139cp 0000:00:0b.0: eth1: rx slot 59 status 0x3201406b len 103
[28510.702856] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28510.709886] 8139cp 0000:00:0b.0: eth1: rx slot 60 status 0x3201406b len 103
[28511.098502] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28511.105396] 8139cp 0000:00:0b.0: eth1: rx slot 61 status 0x3202205d len 89
[28511.629295] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28511.636190] 8139cp 0000:00:0b.0: eth1: rx slot 62 status 0x3201406b len 103
[28511.898925] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28511.905833] 8139cp 0000:00:0b.0: eth1: rx slot 63 status 0x7202205d len 89
[28512.011147] 8139cp 0000:00:0b.0: eth1: intr, status 0001 cmd 0c cpcmd 002b
[28512.018063] 8139cp 0000:00:0b.0: eth1: rx slot 0 status 0x3202206b len 103
[28513.017313] 8139cp 0000:00:0b.0: eth1: Transmit timeout, status c 2b 0 80ff
[28513.025121] 8139cp 0000:00:0b.0: eth1: tx queued, slot 1, skblen 66
[28513.031420] 8139cp 0000:00:0b.0: eth1: tx queued, slot 2, skblen 111
[28513.037801] 8139cp 0000:00:0b.0: eth1: tx queued, slot 3, skblen 116
[28513.044164] 8139cp 0000:00:0b.0: eth1: tx queued, slot 4, skblen 109
[28513.050549] 8139cp 0000:00:0b.0: eth1: tx queued, slot 5, skblen 118
[28513.056912] 8139cp 0000:00:0b.0: eth1: tx queued, slot 6, skblen 111
[28513.063300] 8139cp 0000:00:0b.0: eth1: tx queued, slot 7, skblen 114
[28513.069685] 8139cp 0000:00:0b.0: eth1: tx queued, slot 8, skblen 125
[28513.076049] 8139cp 0000:00:0b.0: eth1: tx queued, slot 9, skblen 147
[28513.082426] 8139cp 0000:00:0b.0: eth1: tx queued, slot 10, skblen 66
[28513.088804] 8139cp 0000:00:0b.0: eth1: tx queued, slot 11, skblen 147
[28513.095254] 8139cp 0000:00:0b.0: eth1: tx queued, slot 12, skblen 74
[28513.101631] 8139cp 0000:00:0b.0: eth1: tx queued, slot 13, skblen 66
[28513.108009] 8139cp 0000:00:0b.0: eth1: tx queued, slot 14, skblen 147
[28513.114461] 8139cp 0000:00:0b.0: eth1: tx queued, slot 15, skblen 66
[28513.120846] 8139cp 0000:00:0b.0: eth1: tx queued, slot 16, skblen 66
[28513.127210] 8139cp 0000:00:0b.0: eth1: tx queued, slot 17, skblen 94
[28513.133595] 8139cp 0000:00:0b.0: eth1: tx queued, slot 18, skblen 66
[28513.139973] 8139cp 0000:00:0b.0: eth1: tx queued, slot 19, skblen 108
[28513.146425] 8139cp 0000:00:0b.0: eth1: tx queued, slot 20, skblen 110
[28513.152897] 8139cp 0000:00:0b.0: eth1: tx queued, slot 21, skblen 62
[28513.159273] 8139cp 0000:00:0b.0: eth1: tx queued, slot 22, skblen 66
[28513.165639] 8139cp 0000:00:0b.0: eth1: tx queued, slot 23, skblen 110
[28513.172111] 8139cp 0000:00:0b.0: eth1: tx queued, slot 24, skblen 147
[28513.178575] 8139cp 0000:00:0b.0: eth1: tx queued, slot 25, skblen 135
[28513.185027] 8139cp 0000:00:0b.0: eth1: tx queued, slot 26, skblen 70
[28513.191411] 8139cp 0000:00:0b.0: eth1: tx queued, slot 27, skblen 70
[28513.661282] 8139cp 0000:00:0b.0: eth1: tx queued, slot 28, skblen 110
[28513.902182] 8139cp 0000:00:0b.0: eth1: tx queued, slot 29, skblen 114
[28514.152998] 8139cp 0000:00:0b.0: eth1: tx queued, slot 30, skblen 62
[28514.904881] 8139cp 0000:00:0b.0: eth1: tx queued, slot 31, skblen 109
[28515.996046] 8139cp 0000:00:0b.0: eth1: tx queued, slot 32, skblen 70
[28516.664680] 8139cp 0000:00:0b.0: eth1: tx queued, slot 33, skblen 125
[28516.904608] 8139cp 0000:00:0b.0: eth1: tx queued, slot 34, skblen 126
[28517.916878] 8139cp 0000:00:0b.0: eth1: tx queued, slot 35, skblen 115
[28518.352300] 8139cp 0000:00:0b.0: eth1: tx queued, slot 36, skblen 66
[28519.028318] 8139cp 0000:00:0b.0: eth1: tx queued, slot 37, skblen 102
[28519.339446] 8139cp 0000:00:0b.0: eth1: tx queued, slot 38, skblen 141
[28519.668103] 8139cp 0000:00:0b.0: eth1: tx queued, slot 39, skblen 112
[28519.756759] 8139cp 0000:00:0b.0: eth1: tx queued, slot 40, skblen 141
[28519.907770] 8139cp 0000:00:0b.0: eth1: tx queued, slot 41, skblen 109
[28520.188635] 8139cp 0000:00:0b.0: eth1: tx queued, slot 42, skblen 62
[28520.596861] 8139cp 0000:00:0b.0: eth1: tx queued, slot 43, skblen 141
[28520.635544] 8139cp 0000:00:0b.0: eth1: tx queued, slot 44, skblen 147
[28520.911469] 8139cp 0000:00:0b.0: eth1: tx queued, slot 45, skblen 125
[28531.018105] 8139cp 0000:00:0b.0: eth1: Transmit timeout, status c 2b 485 0
[28531.025915] 8139cp 0000:00:0b.0: eth1: tx queued, slot 1, skblen 141
[28531.032305] 8139cp 0000:00:0b.0: eth1: tx queued, slot 2, skblen 117
[28531.038685] 8139cp 0000:00:0b.0: eth1: tx queued, slot 3, skblen 115
[28531.045049] 8139cp 0000:00:0b.0: eth1: tx queued, slot 4, skblen 117
[28531.051433] 8139cp 0000:00:0b.0: eth1: tx queued, slot 5, skblen 126
[28531.057797] 8139cp 0000:00:0b.0: eth1: tx queued, slot 6, skblen 112
[28531.064184] 8139cp 0000:00:0b.0: eth1: tx queued, slot 7, skblen 121
[28531.070559] 8139cp 0000:00:0b.0: eth1: tx queued, slot 8, skblen 118
[28531.076925] 8139cp 0000:00:0b.0: eth1: tx queued, slot 9, skblen 118
[28531.083310] 8139cp 0000:00:0b.0: eth1: tx queued, slot 10, skblen 115
[28531.089779] 8139cp 0000:00:0b.0: eth1: tx queued, slot 11, skblen 79
[28531.096148] 8139cp 0000:00:0b.0: eth1: tx queued, slot 12, skblen 141
[28531.102613] 8139cp 0000:00:0b.0: eth1: tx queued, slot 13, skblen 94
[28531.108990] 8139cp 0000:00:0b.0: eth1: tx queued, slot 14, skblen 42
[28531.115354] 8139cp 0000:00:0b.0: eth1: tx queued, slot 15, skblen 42
[28531.121738] 8139cp 0000:00:0b.0: eth1: tx queued, slot 16, skblen 42
[28531.128116] 8139cp 0000:00:0b.0: eth1: tx queued, slot 17, skblen 42
[28531.858156] 8139cp 0000:00:0b.0: eth1: tx queued, slot 18, skblen 42
[28532.256232] 8139cp 0000:00:0b.0: eth1: tx queued, slot 19, skblen 141
[28532.858193] 8139cp 0000:00:0b.0: eth1: tx queued, slot 20, skblen 42
[28533.862354] 8139cp 0000:00:0b.0: eth1: tx queued, slot 21, skblen 42
[28534.858290] 8139cp 0000:00:0b.0: eth1: tx queued, slot 22, skblen 42
[28535.858336] 8139cp 0000:00:0b.0: eth1: tx queued, slot 23, skblen 42
[28536.634367] 8139cp 0000:00:0b.0: eth1: tx queued, slot 24, skblen 147
[28537.684826] 8139cp 0000:00:0b.0: eth1: tx queued, slot 25, skblen 42
[28538.678444] 8139cp 0000:00:0b.0: eth1: tx queued, slot 26, skblen 42
[28539.678487] 8139cp 0000:00:0b.0: eth1: tx queued, slot 27, skblen 42
[28540.688218] 8139cp 0000:00:0b.0: eth1: tx queued, slot 28, skblen 42
[28541.678581] 8139cp 0000:00:0b.0: eth1: tx queued, slot 29, skblen 42
[28542.678622] 8139cp 0000:00:0b.0: eth1: tx queued, slot 30, skblen 42
[28543.690661] 8139cp 0000:00:0b.0: eth1: tx queued, slot 31, skblen 42
[28544.688708] 8139cp 0000:00:0b.0: eth1: tx queued, slot 32, skblen 42
[28545.575351] 8139cp 0000:00:0b.0: eth1: tx queued, slot 33, skblen 141
[28545.688766] 8139cp 0000:00:0b.0: eth1: tx queued, slot 34, skblen 42
[28546.694068] 8139cp 0000:00:0b.0: eth1: tx queued, slot 35, skblen 42
[28547.688840] 8139cp 0000:00:0b.0: eth1: tx queued, slot 36, skblen 42
[28548.688882] 8139cp 0000:00:0b.0: eth1: tx queued, slot 37, skblen 42
[28549.697713] 8139cp 0000:00:0b.0: eth1: tx queued, slot 38, skblen 42
[28550.579007] 8139cp 0000:00:0b.0: eth1: tx queued, slot 39, skblen 86
[28550.688971] 8139cp 0000:00:0b.0: eth1: tx queued, slot 40, skblen 42
[28550.811035] SysRq : Changing Loglevel
[28550.814730] Loglevel set to 1
[28551.579045] 8139cp 0000:00:0b.0: eth1: tx queued, slot 41, skblen 86
[28551.689016] 8139cp 0000:00:0b.0: eth1: tx queued, slot 42, skblen 42
[28551.719077] 8139cp 0000:00:0b.0: eth1: tx queued, slot 43, skblen 79
[28552.579071] 8139cp 0000:00:0b.0: eth1: tx queued, slot 44, skblen 86
[28552.701599] 8139cp 0000:00:0b.0: eth1: tx queued, slot 45, skblen 42
[28555.116322] 8139cp 0000:00:0b.0: eth1: disabling interface
[28555.116663] br-lan: port 2(eth1) entered disabled state
[28555.120239] device eth1 left promiscuous mode
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 6171 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: 8139cp TX stall, timeout, failed recovery
2012-11-24 18:39 8139cp TX stall, timeout, failed recovery David Woodhouse
@ 2012-11-24 21:33 ` Francois Romieu
2012-11-24 22:11 ` [PATCH] 8139cp: re-enable interrupts after tx timeout David Woodhouse
2012-11-24 21:39 ` 8139cp TX stall, timeout, failed recovery David Woodhouse
1 sibling, 1 reply; 8+ messages in thread
From: Francois Romieu @ 2012-11-24 21:33 UTC (permalink / raw)
To: David Woodhouse; +Cc: netdev, jasowang, gilboad, jgarzik
David Woodhouse <dwmw2@infradead.org> :
[...]
> To reproduce this, I send a *large* flood of packets (ping -l 1000) from
> inside my network to the ADSL router. Its internal-facing interface is
> an 8139cp. It seems to work best if the packets are destined for the
> outside world over the ADSL lines, rather than the router itself. But
> it's hard to tell. I need between 10 and 30 runs of 'ping -l 1000
> $outside' before it happens. Or sometimes more...
Did you try pktgen as well ?
[...]
> Looking at it now... do we wrap around the Rx ring buffer at 28498.055
> and process some of the packets more than once? Or is the hardware
> genuinely keeping up with us as we suck out, descriptors 51-63, 0-63,
> and 0-39 again all off the same interrupt? Perhaps there lies the root
> of the problem?
Rx slots are always spaced by ~50 us, i.e. more than needed to get a 98 bytes
packets. I do not like the lack of barrier wrt opts2 and opts1 writes for rx
descriptors but it does nprobably not matter for your problem.
If the r8169 experience is any guide, cp_tx_timeout is too light. It should
imho match the init sequence more closely, especially wrt descriptor rings
addresses and CpCmd.
--
Ueimor
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: 8139cp TX stall, timeout, failed recovery
2012-11-24 18:39 8139cp TX stall, timeout, failed recovery David Woodhouse
2012-11-24 21:33 ` Francois Romieu
@ 2012-11-24 21:39 ` David Woodhouse
1 sibling, 0 replies; 8+ messages in thread
From: David Woodhouse @ 2012-11-24 21:39 UTC (permalink / raw)
To: netdev; +Cc: romieu, jasowang, gilboad, jgarzik, nathan
[-- Attachment #1: Type: text/plain, Size: 1672 bytes --]
On Sat, 2012-11-24 at 18:39 +0000, David Woodhouse wrote:
> This seems to be a consistent pattern — when we get that 0x0080
> interrupt and it dies, it's *very* soon after queueing a new tx:
Once I realise that the 'tx queued' message actually prints the number
of the *next* slot that'll be used, not the slot that was just filled,
that becomes obvious. The hardware takes only 30µs or so to consume the
descriptor that was just submitted. It isn't just coincidence that one
packet completes just as the *next* one is being submitted, as I
originally thought.
The hardware seems to asserts the 0x80 'Tx Descriptor Unavailable'
interrupt first, and the other bits (0x404) come later. I *often* get
into cp_tx() with only 0x80 in the IntrStatus bits, and the other bits
are often set before my heavily-debug-laden cp_tx() has even finished
running).
Register 0x82 indicates the low bits of the address of the most recently
consumed Tx descriptor, and always seems to agree with the driver's
processing of the ring. When we get a 0x80 interrupt, the most recently
consumed descriptor is always the one before tx_head, as you'd expect.
And tx_head always looks sane and (as long as you read it quickly) still
has the 'Own' bit set, as it should.
Eventually I get a 0x80 interrupt which *isn't* immediately followed by
the other 0x404 bits. And then the hardware has crapped itself and is no
longer eating descriptors. We submit more to the queue and we bang on
the TxPoll register *every* time, but that doesn't wake it.
Adding a cp_enable_irq() into the cp_tx_timeout() function at least
makes it recover *eventually*.
--
dwmw2
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 6171 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH] 8139cp: re-enable interrupts after tx timeout
2012-11-24 21:33 ` Francois Romieu
@ 2012-11-24 22:11 ` David Woodhouse
2012-11-24 22:58 ` Francois Romieu
0 siblings, 1 reply; 8+ messages in thread
From: David Woodhouse @ 2012-11-24 22:11 UTC (permalink / raw)
To: Francois Romieu; +Cc: netdev, jasowang, gilboad, jgarzik
[-- Attachment #1: Type: text/plain, Size: 735 bytes --]
Recovery doesn't work too well if we leave interrupts disabled...
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
---
For stable too, I suppose.
diff --git a/drivers/net/ethernet/realtek/8139cp.c b/drivers/net/ethernet/realtek/8139cp.c
index 1c81825..ad6afc3 100644
--- a/drivers/net/ethernet/realtek/8139cp.c
+++ b/drivers/net/ethernet/realtek/8139cp.c
@@ -1192,6 +1192,7 @@ static void cp_tx_timeout(struct net_device *dev)
cp_clean_rings(cp);
rc = cp_init_rings(cp);
cp_start_hw(cp);
+ cp_enable_irq(cp);
netif_wake_queue(dev);
--
David Woodhouse Open Source Technology Centre
David.Woodhouse@intel.com Intel Corporation
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 6171 bytes --]
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] 8139cp: re-enable interrupts after tx timeout
2012-11-24 22:11 ` [PATCH] 8139cp: re-enable interrupts after tx timeout David Woodhouse
@ 2012-11-24 22:58 ` Francois Romieu
2012-11-24 23:27 ` David Woodhouse
2012-11-25 20:57 ` David Miller
0 siblings, 2 replies; 8+ messages in thread
From: Francois Romieu @ 2012-11-24 22:58 UTC (permalink / raw)
To: David Woodhouse; +Cc: netdev, jasowang, gilboad, jgarzik
David Woodhouse <dwmw2@infradead.org> :
> Recovery doesn't work too well if we leave interrupts disabled...
>
> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Acked-by: Francois Romieu <romieu@fr.zoreil.com>
While you are at it, do you see what would prevent the watchdog
timer and the rx napi handler to race on two different CPUs
("using a Geode" could be an answer :o) ) ?
> ---
> For stable too, I suppose.
Yes. Watch for http://patchwork.ozlabs.org/bundle/davem/stable/
--
Ueimor
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] 8139cp: re-enable interrupts after tx timeout
2012-11-24 22:58 ` Francois Romieu
@ 2012-11-24 23:27 ` David Woodhouse
2012-11-25 22:43 ` Francois Romieu
2012-11-25 20:57 ` David Miller
1 sibling, 1 reply; 8+ messages in thread
From: David Woodhouse @ 2012-11-24 23:27 UTC (permalink / raw)
To: Francois Romieu; +Cc: netdev, jasowang, gilboad, jgarzik
[-- Attachment #1: Type: text/plain, Size: 379 bytes --]
On Sat, 2012-11-24 at 23:58 +0100, Francois Romieu wrote:
> While you are at it, do you see what would prevent the watchdog
> timer and the rx napi handler to race on two different CPUs
Hm, good point. Is it sufficient just to stick napi_disable() and
napi_enable() in there? And should we try to reset the TX state machine
without stomping on RX at all?
--
dwmw2
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 6171 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] 8139cp: re-enable interrupts after tx timeout
2012-11-24 22:58 ` Francois Romieu
2012-11-24 23:27 ` David Woodhouse
@ 2012-11-25 20:57 ` David Miller
1 sibling, 0 replies; 8+ messages in thread
From: David Miller @ 2012-11-25 20:57 UTC (permalink / raw)
To: romieu; +Cc: dwmw2, netdev, jasowang, gilboad, jgarzik
From: Francois Romieu <romieu@fr.zoreil.com>
Date: Sat, 24 Nov 2012 23:58:55 +0100
> David Woodhouse <dwmw2@infradead.org> :
>> Recovery doesn't work too well if we leave interrupts disabled...
>>
>> Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
>
> Acked-by: Francois Romieu <romieu@fr.zoreil.com>
Applied to net-next
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] 8139cp: re-enable interrupts after tx timeout
2012-11-24 23:27 ` David Woodhouse
@ 2012-11-25 22:43 ` Francois Romieu
0 siblings, 0 replies; 8+ messages in thread
From: Francois Romieu @ 2012-11-25 22:43 UTC (permalink / raw)
To: David Woodhouse; +Cc: netdev, jasowang, gilboad, jgarzik
David Woodhouse <dwmw2@infradead.org> :
> On Sat, 2012-11-24 at 23:58 +0100, Francois Romieu wrote:
> > While you are at it, do you see what would prevent the watchdog
> > timer and the rx napi handler to race on two different CPUs
>
> Hm, good point. Is it sufficient just to stick napi_disable() and
> napi_enable() in there?
No, napi_disable() may sleep.
> And should we try to reset the TX state machine without stomping on
> RX at all ?
Either that or the watchdog timer will have to queue some user context
work. Things may change later when you go lockless (hint, hint).
--
Ueimor
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-11-25 23:07 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-11-24 18:39 8139cp TX stall, timeout, failed recovery David Woodhouse
2012-11-24 21:33 ` Francois Romieu
2012-11-24 22:11 ` [PATCH] 8139cp: re-enable interrupts after tx timeout David Woodhouse
2012-11-24 22:58 ` Francois Romieu
2012-11-24 23:27 ` David Woodhouse
2012-11-25 22:43 ` Francois Romieu
2012-11-25 20:57 ` David Miller
2012-11-24 21:39 ` 8139cp TX stall, timeout, failed recovery David Woodhouse
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.