linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* 2.6.24-rc8-mm1 : net tcp_input.c warnings
@ 2008-01-21  7:55 Dave Young
  2008-01-21 21:14 ` Ilpo Järvinen
  0 siblings, 1 reply; 18+ messages in thread
From: Dave Young @ 2008-01-21  7:55 UTC (permalink / raw)
  To: linux-kernel; +Cc: davem, netdev, akpm

Please see the kernel messages following,(trigged while using some qemu session)
BTW, seems there's some e100 error message as well.

PCI: Setting latency timer of device 0000:00:1b.0 to 64
e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
e100: Copyright(c) 1999-2006 Intel Corporation
ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20
modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default
e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting.
ACPI: PCI interrupt for device 0000:03:08.0 disabled
e100: probe of 0000:03:08.0 failed with error -12
eth0:  setting full-duplex.
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
 [<c0132100>] ? printk+0x0/0x20
 [<c0131834>] warn_on_slowpath+0x54/0x80
 [<c03e8df8>] ? ip_finish_output+0x128/0x2e0
 [<c03e9527>] ? ip_output+0xe7/0x100
 [<c03e8a88>] ? ip_local_out+0x18/0x20
 [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470
 [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
 [<c0186be1>] ? check_pad_bytes+0x61/0x80
 [<c03f6031>] tcp_mark_head_lost+0x121/0x150
 [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
 [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
 [<c03f7d93>] tcp_ack+0x1b3/0x3a0
 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
 [<c0401967>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01c8>] ? sock_def_readable+0x48/0xa0
 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8d96>] netif_receive_skb+0x2b6/0x330
 [<c03c8c07>] ? netif_receive_skb+0x127/0x330
 [<c03c8e93>] ? process_backlog+0x83/0x100
 [<c03c8e9e>] process_backlog+0x8e/0x100
 [<c03c90ac>] net_rx_action+0x13c/0x230
 [<c03c8fc9>] ? net_rx_action+0x59/0x230
 [<c0136f2e>] ? __do_softirq+0x6e/0x120
 [<c0136f53>] __do_softirq+0x93/0x120
 [<c013705a>] do_softirq+0x7a/0x80
 [<c0137125>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04333bd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 97302d8bf57718dd ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2528 tcp_fastretrans_alert+0x675/0x6b0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
 [<c0132100>] ? printk+0x0/0x20
 [<c0131834>] warn_on_slowpath+0x54/0x80
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c03f6ff5>] tcp_fastretrans_alert+0x675/0x6b0
 [<c03f7d93>] tcp_ack+0x1b3/0x3a0
 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
 [<c0401967>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01c8>] ? sock_def_readable+0x48/0xa0
 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8d96>] netif_receive_skb+0x2b6/0x330
 [<c03c8c07>] ? netif_receive_skb+0x127/0x330
 [<c03c8e93>] ? process_backlog+0x83/0x100
 [<c03c8e9e>] process_backlog+0x8e/0x100
 [<c03c90ac>] net_rx_action+0x13c/0x230
 [<c03c8fc9>] ? net_rx_action+0x59/0x230
 [<c0136f2e>] ? __do_softirq+0x6e/0x120
 [<c0136f53>] __do_softirq+0x93/0x120
 [<c013705a>] do_softirq+0x7a/0x80
 [<c0137125>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04333bd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 97302d8bf57718dd ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
 [<c0132100>] ? printk+0x0/0x20
 [<c0131834>] warn_on_slowpath+0x54/0x80
 [<c0153241>] ? put_lock_stats+0x21/0x30
 [<c01532b0>] ? lock_release_holdtime+0x60/0x80
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c03f6031>] tcp_mark_head_lost+0x121/0x150
 [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
 [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
 [<c03f7d93>] tcp_ack+0x1b3/0x3a0
 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
 [<c0401967>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01c8>] ? sock_def_readable+0x48/0xa0
 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8d96>] netif_receive_skb+0x2b6/0x330
 [<c03c8c07>] ? netif_receive_skb+0x127/0x330
 [<c03c8e93>] ? process_backlog+0x83/0x100
 [<c03c8e9e>] process_backlog+0x8e/0x100
 [<c03c90ac>] net_rx_action+0x13c/0x230
 [<c03c8fc9>] ? net_rx_action+0x59/0x230
 [<c0136f2e>] ? __do_softirq+0x6e/0x120
 [<c0136f53>] __do_softirq+0x93/0x120
 [<c013705a>] do_softirq+0x7a/0x80
 [<c0137125>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04333bd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 97302d8bf57718dd ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2528 tcp_fastretrans_alert+0x675/0x6b0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
 [<c0132100>] ? printk+0x0/0x20
 [<c0131834>] warn_on_slowpath+0x54/0x80
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c0186ae4>] ? check_bytes_and_report+0x24/0xc0
 [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
 [<c0186be1>] ? check_pad_bytes+0x61/0x80
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c03f6ff5>] tcp_fastretrans_alert+0x675/0x6b0
 [<c03f7d93>] tcp_ack+0x1b3/0x3a0
 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
 [<c0401967>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01c8>] ? sock_def_readable+0x48/0xa0
 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8d96>] netif_receive_skb+0x2b6/0x330
 [<c03c8c07>] ? netif_receive_skb+0x127/0x330
 [<c03c8e93>] ? process_backlog+0x83/0x100
 [<c03c8e9e>] process_backlog+0x8e/0x100
 [<c03c90ac>] net_rx_action+0x13c/0x230
 [<c03c8fc9>] ? net_rx_action+0x59/0x230
 [<c0136f2e>] ? __do_softirq+0x6e/0x120
 [<c0136f53>] __do_softirq+0x93/0x120
 [<c013705a>] do_softirq+0x7a/0x80
 [<c0137125>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04333bd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 97302d8bf57718dd ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
 [<c0132100>] ? printk+0x0/0x20
 [<c0131834>] warn_on_slowpath+0x54/0x80
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c0153241>] ? put_lock_stats+0x21/0x30
 [<c01532b0>] ? lock_release_holdtime+0x60/0x80
 [<c0187918>] ? add_partial+0x48/0x90
 [<c03f6031>] tcp_mark_head_lost+0x121/0x150
 [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
 [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
 [<c03f7d93>] tcp_ack+0x1b3/0x3a0
 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
 [<c0401967>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01c8>] ? sock_def_readable+0x48/0xa0
 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8d96>] netif_receive_skb+0x2b6/0x330
 [<c03c8c07>] ? netif_receive_skb+0x127/0x330
 [<c03c8e93>] ? process_backlog+0x83/0x100
 [<c03c8e9e>] process_backlog+0x8e/0x100
 [<c03c90ac>] net_rx_action+0x13c/0x230
 [<c03c8fc9>] ? net_rx_action+0x59/0x230
 [<c0136f2e>] ? __do_softirq+0x6e/0x120
 [<c0136f53>] __do_softirq+0x93/0x120
 [<c013705a>] do_softirq+0x7a/0x80
 [<c0137125>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04333bd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 97302d8bf57718dd ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2528 tcp_fastretrans_alert+0x675/0x6b0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
 [<c0132100>] ? printk+0x0/0x20
 [<c0131834>] warn_on_slowpath+0x54/0x80
 [<c0186ae4>] ? check_bytes_and_report+0x24/0xc0
 [<c0186be1>] ? check_pad_bytes+0x61/0x80
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c03f6ff5>] tcp_fastretrans_alert+0x675/0x6b0
 [<c03f7d93>] tcp_ack+0x1b3/0x3a0
 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
 [<c0401967>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01c8>] ? sock_def_readable+0x48/0xa0
 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8d96>] netif_receive_skb+0x2b6/0x330
 [<c03c8c07>] ? netif_receive_skb+0x127/0x330
 [<c03c8e93>] ? process_backlog+0x83/0x100
 [<c03c8e9e>] process_backlog+0x8e/0x100
 [<c03c90ac>] net_rx_action+0x13c/0x230
 [<c03c8fc9>] ? net_rx_action+0x59/0x230
 [<c0136f2e>] ? __do_softirq+0x6e/0x120
 [<c0136f53>] __do_softirq+0x93/0x120
 [<c013705a>] do_softirq+0x7a/0x80
 [<c0137125>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04333bd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 97302d8bf57718dd ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
Pid: 2754, comm: X Not tainted 2.6.24-rc8-mm1 #4
 [<c0132100>] ? printk+0x0/0x20
 [<c0131834>] warn_on_slowpath+0x54/0x80
 [<c015322d>] ? put_lock_stats+0xd/0x30
 [<c01532b0>] ? lock_release_holdtime+0x60/0x80
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c04363f9>] ? _spin_unlock_irqrestore+0x39/0x70
 [<c013dbb4>] ? group_send_sig_info+0x74/0x80
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c03f6031>] tcp_mark_head_lost+0x121/0x150
 [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
 [<c03f5ed1>] ? tcp_time_to_recover+0xe1/0xf0
 [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
 [<c03f7d93>] tcp_ack+0x1b3/0x3a0
 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
 [<c0401967>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b87>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01c8>] ? sock_def_readable+0x48/0xa0
 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8d96>] netif_receive_skb+0x2b6/0x330
 [<c03c8c07>] ? netif_receive_skb+0x127/0x330
 [<c03c8e93>] ? process_backlog+0x83/0x100
 [<c03c8e9e>] process_backlog+0x8e/0x100
 [<c03c90ac>] net_rx_action+0x13c/0x230
 [<c03c8fc9>] ? net_rx_action+0x59/0x230
 [<c0136f2e>] ? __do_softirq+0x6e/0x120
 [<c0136f53>] __do_softirq+0x93/0x120
 [<c013705a>] do_softirq+0x7a/0x80
 [<c0137125>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155649>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 =======================
---[ end trace 97302d8bf57718dd ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:1767 tcp_enter_frto+0x27c/0x2a0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
 [<c0132100>] ? printk+0x0/0x20
 [<c0131834>] warn_on_slowpath+0x54/0x80
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0154972>] ? validate_chain+0x1d2/0x320
 [<c0156234>] ? __lock_acquire+0x1e4/0x730
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c03f58ac>] tcp_enter_frto+0x27c/0x2a0
 [<c03ff3c2>] tcp_retransmit_timer+0x102/0x400
 [<c043611b>] ? _spin_lock+0x4b/0x80
 [<c03ff6d4>] ? tcp_write_timer+0x14/0x110
 [<c03ff7b3>] tcp_write_timer+0xf3/0x110
 [<c013b94b>] run_timer_softirq+0xbb/0x190
 [<c03ff6c0>] ? tcp_write_timer+0x0/0x110
 [<c0136f53>] __do_softirq+0x93/0x120
 [<c013705a>] do_softirq+0x7a/0x80
 [<c0137125>] irq_exit+0x65/0x90
 [<c011a91d>] smp_apic_timer_interrupt+0x2d/0x40
 [<c0105aab>] apic_timer_interrupt+0x33/0x38
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04333bd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 97302d8bf57718dd ]---

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-21  7:55 2.6.24-rc8-mm1 : net tcp_input.c warnings Dave Young
@ 2008-01-21 21:14 ` Ilpo Järvinen
  2008-01-22  4:37   ` Dave Young
  0 siblings, 1 reply; 18+ messages in thread
From: Ilpo Järvinen @ 2008-01-21 21:14 UTC (permalink / raw)
  To: Dave Young; +Cc: LKML, David Miller, Netdev, Andrew Morton

On Mon, 21 Jan 2008, Dave Young wrote:

> Please see the kernel messages following,(trigged while using some qemu session)
> BTW, seems there's some e100 error message as well.
> 
> PCI: Setting latency timer of device 0000:00:1b.0 to 64
> e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
> e100: Copyright(c) 1999-2006 Intel Corporation
> ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20
> modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default
> e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting.
> ACPI: PCI interrupt for device 0000:03:08.0 disabled
> e100: probe of 0000:03:08.0 failed with error -12
> eth0:  setting full-duplex.
> ------------[ cut here ]------------
> WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
> Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
> Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
>  [<c0132100>] ? printk+0x0/0x20
>  [<c0131834>] warn_on_slowpath+0x54/0x80
>  [<c03e8df8>] ? ip_finish_output+0x128/0x2e0
>  [<c03e9527>] ? ip_output+0xe7/0x100
>  [<c03e8a88>] ? ip_local_out+0x18/0x20
>  [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470
>  [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
>  [<c0186be1>] ? check_pad_bytes+0x61/0x80
>  [<c03f6031>] tcp_mark_head_lost+0x121/0x150
>  [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
>  [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
>  [<c03f7d93>] tcp_ack+0x1b3/0x3a0
>  [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
>  [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
>  [<c0401bbb>] tcp_v4_rcv+0x5db/0x660

Doh, once more these S+L things..., the rest are symptom of the first 
problem.

What is strange is that it doesn't show up until now, the last TCP
changes that could have some significance are from early Dec/Nov. Is
there some reason why you haven't seen this before this (e.g., not
tested with similar cfg or so)? I'm a bit worried about its
reproducability if it takes this far to see it...


-- 
 i.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-21 21:14 ` Ilpo Järvinen
@ 2008-01-22  4:37   ` Dave Young
  2008-01-22  9:09     ` Dave Young
  0 siblings, 1 reply; 18+ messages in thread
From: Dave Young @ 2008-01-22  4:37 UTC (permalink / raw)
  To: Ilpo Järvinen; +Cc: LKML, David Miller, Netdev, Andrew Morton

On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
>
> On Mon, 21 Jan 2008, Dave Young wrote:
>
> > Please see the kernel messages following,(trigged while using some qemu session)
> > BTW, seems there's some e100 error message as well.
> >
> > PCI: Setting latency timer of device 0000:00:1b.0 to 64
> > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
> > e100: Copyright(c) 1999-2006 Intel Corporation
> > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20
> > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default
> > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting.
> > ACPI: PCI interrupt for device 0000:03:08.0 disabled
> > e100: probe of 0000:03:08.0 failed with error -12
> > eth0:  setting full-duplex.
> > ------------[ cut here ]------------
> > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
> > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
> > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
> >  [<c0132100>] ? printk+0x0/0x20
> >  [<c0131834>] warn_on_slowpath+0x54/0x80
> >  [<c03e8df8>] ? ip_finish_output+0x128/0x2e0
> >  [<c03e9527>] ? ip_output+0xe7/0x100
> >  [<c03e8a88>] ? ip_local_out+0x18/0x20
> >  [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470
> >  [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
> >  [<c0186be1>] ? check_pad_bytes+0x61/0x80
> >  [<c03f6031>] tcp_mark_head_lost+0x121/0x150
> >  [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
> >  [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
> >  [<c03f7d93>] tcp_ack+0x1b3/0x3a0
> >  [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
> >  [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
> >  [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
>
> Doh, once more these S+L things..., the rest are symptom of the first
> problem.

What is the S+L thing? Could you explain a bit?

>
> What is strange is that it doesn't show up until now, the last TCP
> changes that could have some significance are from early Dec/Nov. Is
> there some reason why you haven't seen this before this (e.g., not
> tested with similar cfg or so)?

Hmm, don't know how to answer ...

I'm a bit worried about its
> reproducability if it takes this far to see it...
>
>
> --
>  i.
>

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-22  4:37   ` Dave Young
@ 2008-01-22  9:09     ` Dave Young
  2008-01-22  9:18       ` Dave Young
  2008-01-22 10:47       ` Ilpo Järvinen
  0 siblings, 2 replies; 18+ messages in thread
From: Dave Young @ 2008-01-22  9:09 UTC (permalink / raw)
  To: Ilpo Järvinen; +Cc: LKML, David Miller, Netdev, Andrew Morton

On Jan 22, 2008 12:37 PM, Dave Young <hidave.darkstar@gmail.com> wrote:
>
> On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> >
> > On Mon, 21 Jan 2008, Dave Young wrote:
> >
> > > Please see the kernel messages following,(trigged while using some qemu session)
> > > BTW, seems there's some e100 error message as well.
> > >
> > > PCI: Setting latency timer of device 0000:00:1b.0 to 64
> > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
> > > e100: Copyright(c) 1999-2006 Intel Corporation
> > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20
> > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default
> > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting.
> > > ACPI: PCI interrupt for device 0000:03:08.0 disabled
> > > e100: probe of 0000:03:08.0 failed with error -12
> > > eth0:  setting full-duplex.
> > > ------------[ cut here ]------------
> > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
> > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
> > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
> > >  [<c0132100>] ? printk+0x0/0x20
> > >  [<c0131834>] warn_on_slowpath+0x54/0x80
> > >  [<c03e8df8>] ? ip_finish_output+0x128/0x2e0
> > >  [<c03e9527>] ? ip_output+0xe7/0x100
> > >  [<c03e8a88>] ? ip_local_out+0x18/0x20
> > >  [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470
> > >  [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
> > >  [<c0186be1>] ? check_pad_bytes+0x61/0x80
> > >  [<c03f6031>] tcp_mark_head_lost+0x121/0x150
> > >  [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
> > >  [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
> > >  [<c03f7d93>] tcp_ack+0x1b3/0x3a0
> > >  [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
> > >  [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
> > >  [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
> >
> > Doh, once more these S+L things..., the rest are symptom of the first
> > problem.
>
> What is the S+L thing? Could you explain a bit?
>
> >
> > What is strange is that it doesn't show up until now, the last TCP
> > changes that could have some significance are from early Dec/Nov. Is
> > there some reason why you haven't seen this before this (e.g., not
> > tested with similar cfg or so)?
>
> Hmm, don't know how to answer ...
>
>
> I'm a bit worried about its
> > reproducability if it takes this far to see it...
> >

It's trigged again in my pc, just while using firefox.

> >
> > --
> >  i.
> >
>

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-22  9:09     ` Dave Young
@ 2008-01-22  9:18       ` Dave Young
  2008-01-22 10:47       ` Ilpo Järvinen
  1 sibling, 0 replies; 18+ messages in thread
From: Dave Young @ 2008-01-22  9:18 UTC (permalink / raw)
  To: Ilpo Järvinen; +Cc: LKML, David Miller, Netdev, Andrew Morton

On Jan 22, 2008 5:09 PM, Dave Young <hidave.darkstar@gmail.com> wrote:
>
> On Jan 22, 2008 12:37 PM, Dave Young <hidave.darkstar@gmail.com> wrote:
> >
> > On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > >
> > > On Mon, 21 Jan 2008, Dave Young wrote:
> > >
> > > > Please see the kernel messages following,(trigged while using some qemu session)
> > > > BTW, seems there's some e100 error message as well.
> > > >
> > > > PCI: Setting latency timer of device 0000:00:1b.0 to 64
> > > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
> > > > e100: Copyright(c) 1999-2006 Intel Corporation
> > > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20
> > > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default
> > > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting.
> > > > ACPI: PCI interrupt for device 0000:03:08.0 disabled
> > > > e100: probe of 0000:03:08.0 failed with error -12
> > > > eth0:  setting full-duplex.
> > > > ------------[ cut here ]------------
> > > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
> > > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
> > > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
> > > >  [<c0132100>] ? printk+0x0/0x20
> > > >  [<c0131834>] warn_on_slowpath+0x54/0x80
> > > >  [<c03e8df8>] ? ip_finish_output+0x128/0x2e0
> > > >  [<c03e9527>] ? ip_output+0xe7/0x100
> > > >  [<c03e8a88>] ? ip_local_out+0x18/0x20
> > > >  [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470
> > > >  [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
> > > >  [<c0186be1>] ? check_pad_bytes+0x61/0x80
> > > >  [<c03f6031>] tcp_mark_head_lost+0x121/0x150
> > > >  [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
> > > >  [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
> > > >  [<c03f7d93>] tcp_ack+0x1b3/0x3a0
> > > >  [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
> > > >  [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
> > > >  [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
> > >
> > > Doh, once more these S+L things..., the rest are symptom of the first
> > > problem.
> >
> > What is the S+L thing? Could you explain a bit?
> >
> > >
> > > What is strange is that it doesn't show up until now, the last TCP
> > > changes that could have some significance are from early Dec/Nov. Is
> > > there some reason why you haven't seen this before this (e.g., not
> > > tested with similar cfg or so)?
> >
> > Hmm, don't know how to answer ...
> >
> >
> > I'm a bit worried about its
> > > reproducability if it takes this far to see it...
> > >
>
> It's trigged again in my pc, just while using firefox.

Maybe relate to the e100 error, I will apply jiri slaby's
e100-iomap-mem-accesses patch to test.
>
> > >
> > > --
> > >  i.
> > >
> >
>

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-22  9:09     ` Dave Young
  2008-01-22  9:18       ` Dave Young
@ 2008-01-22 10:47       ` Ilpo Järvinen
  2008-01-23  1:44         ` Dave Young
  1 sibling, 1 reply; 18+ messages in thread
From: Ilpo Järvinen @ 2008-01-22 10:47 UTC (permalink / raw)
  To: Dave Young; +Cc: LKML, David Miller, Netdev, Andrew Morton

[-- Attachment #1: Type: TEXT/PLAIN, Size: 11373 bytes --]

On Tue, 22 Jan 2008, Dave Young wrote:

> On Jan 22, 2008 12:37 PM, Dave Young <hidave.darkstar@gmail.com> wrote:
> >
> > On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > >
> > > On Mon, 21 Jan 2008, Dave Young wrote:
> > >
> > > > Please see the kernel messages following,(trigged while using some qemu session)
> > > > BTW, seems there's some e100 error message as well.
> > > >
> > > > PCI: Setting latency timer of device 0000:00:1b.0 to 64
> > > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
> > > > e100: Copyright(c) 1999-2006 Intel Corporation
> > > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20
> > > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default
> > > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting.
> > > > ACPI: PCI interrupt for device 0000:03:08.0 disabled
> > > > e100: probe of 0000:03:08.0 failed with error -12
> > > > eth0:  setting full-duplex.
> > > > ------------[ cut here ]------------
> > > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
> > > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
> > > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
> > > >  [<c0132100>] ? printk+0x0/0x20
> > > >  [<c0131834>] warn_on_slowpath+0x54/0x80
> > > >  [<c03e8df8>] ? ip_finish_output+0x128/0x2e0
> > > >  [<c03e9527>] ? ip_output+0xe7/0x100
> > > >  [<c03e8a88>] ? ip_local_out+0x18/0x20
> > > >  [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470
> > > >  [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
> > > >  [<c0186be1>] ? check_pad_bytes+0x61/0x80
> > > >  [<c03f6031>] tcp_mark_head_lost+0x121/0x150
> > > >  [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
> > > >  [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
> > > >  [<c03f7d93>] tcp_ack+0x1b3/0x3a0
> > > >  [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
> > > >  [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
> > > >  [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
> > >
> > > Doh, once more these S+L things..., the rest are symptom of the first
> > > problem.
> >
> > What is the S+L thing? Could you explain a bit?

It means that one of the skbs is both SACKed and marked as LOST at the
same time in the counters (might be due to miscount of lost/sacked_out
too, not necessarilily in the ->sacked bits). Such state is logically
invalid because it would mean that the sender thinks that the same packet 
both reached the receiver and is lost in the network.

Traditionally TCP has just silently "corrected" over-estimates
(sacked_out+lost_out > packets_out). I changed this couple of releases ago
because those over-estimates often are due to bugs that should be fixed  
(there have been couple of them but it has been very quite on this front  
long time, months or even half year already; but I might have broken
something with the early Dec changes).

These problem may originate from a bug that occurred a number of ACKs
earlier the WARN_ON triggered, therefore they are a bit tricky to track,
those WARN_ON serve just for alerting purposes and usually do not point
out where the bug actually occurred.

I usually just asked people to include exhaustive verifier which compares
->sacked bitmaps with sacked/lost_out counters and report immediately when
the problem shows up, rather than waiting for the cheaper S+L check we do
in the WARN_ON to trigger. I tried to collect tracking patch from the
previous efforts (hopefully got it right after modifications).

> > I'm a bit worried about its
> > > reproducability if it takes this far to see it...
> > >
> 
> It's trigged again in my pc, just while using firefox.

...Good, then there's some chance to catch it.

-- 
 i.

[PATCH] [TCP]: debug S+L

---
 include/net/tcp.h     |    8 +++-
 net/ipv4/tcp_input.c  |    6 +++
 net/ipv4/tcp_ipv4.c   |  101 +++++++++++++++++++++++++++++++++++++++++++++++++
 net/ipv4/tcp_output.c |   21 +++++++---
 4 files changed, 129 insertions(+), 7 deletions(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 7de4ea3..0685035 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -272,6 +272,8 @@ DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics);
 #define TCP_ADD_STATS_BH(field, val)	SNMP_ADD_STATS_BH(tcp_statistics, field, val)
 #define TCP_ADD_STATS_USER(field, val)	SNMP_ADD_STATS_USER(tcp_statistics, field, val)
 
+extern void			tcp_verify_wq(struct sock *sk);
+
 extern void			tcp_v4_err(struct sk_buff *skb, u32);
 
 extern void			tcp_shutdown (struct sock *sk, int how);
@@ -768,7 +770,11 @@ static inline __u32 tcp_current_ssthresh(const struct sock *sk)
 }
 
 /* Use define here intentionally to get WARN_ON location shown at the caller */
-#define tcp_verify_left_out(tp)	WARN_ON(tcp_left_out(tp) > tp->packets_out)
+#define tcp_verify_left_out(tp)	\
+	do { \
+		WARN_ON(tcp_left_out(tp) > tp->packets_out); \
+		tcp_verify_wq((struct sock *)tp); \
+	} while(0)
 
 extern void tcp_enter_cwr(struct sock *sk, const int set_ssthresh);
 extern __u32 tcp_init_cwnd(struct tcp_sock *tp, struct dst_entry *dst);
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index fa2c85c..0bda0e1 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -2645,6 +2645,10 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag)
 	if (do_lost || (tcp_is_fack(tp) && tcp_head_timedout(sk)))
 		tcp_update_scoreboard(sk, fast_rexmit);
 	tcp_cwnd_down(sk, flag);
+
+	WARN_ON(tcp_write_queue_head(sk) == NULL);
+	WARN_ON(!tp->packets_out);
+
 	tcp_xmit_retransmit_queue(sk);
 }
 
@@ -2848,6 +2852,8 @@ static int tcp_clean_rtx_queue(struct sock *sk, int prior_fackets)
 		tcp_clear_all_retrans_hints(tp);
 	}
 
+	tcp_verify_left_out(tp);
+
 	if (skb && (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED))
 		flag |= FLAG_SACK_RENEGING;
 
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 9aea88b..21f5888 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -108,6 +108,107 @@ struct inet_hashinfo __cacheline_aligned tcp_hashinfo = {
 	.lhash_wait  = __WAIT_QUEUE_HEAD_INITIALIZER(tcp_hashinfo.lhash_wait),
 };
 
+void tcp_print_queue(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct sk_buff *skb;
+	char s[50+1];
+	char h[50+1];
+	int idx = 0;
+	int i;
+
+	tcp_for_write_queue(skb, sk) {
+		if (skb == tcp_send_head(sk))
+			break;
+
+		for (i = 0; i < tcp_skb_pcount(skb); i++) {
+			if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) {
+				s[idx] = 'S';
+				if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+					s[idx] = 'B';
+
+			} else if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) {
+				s[idx] = 'L';
+			} else {
+				s[idx] = ' ';
+			}
+			if (s[idx] != ' ' && skb->len < tp->mss_cache)
+				s[idx] += 'a' - 'A';
+
+			if (i == 0) {
+				if (TCP_SKB_CB(skb)->seq == tcp_highest_sack_seq(tp))
+					h[idx] = 'h';
+				else
+					h[idx] = '+';
+			} else {
+				h[idx] = '-';
+			}
+
+			if (++idx >= 50) {
+				s[idx] = 0;
+				h[idx] = 0;
+				printk(KERN_ERR "TCP wq(s) %s\n", s);
+				printk(KERN_ERR "TCP wq(h) %s\n", h);
+				idx = 0;
+			}
+		}
+	}
+	if (idx) {
+		s[idx] = '<';
+		s[idx+1] = 0;
+		h[idx] = '<';
+		h[idx+1] = 0;
+		printk(KERN_ERR "TCP wq(s) %s\n", s);
+		printk(KERN_ERR "TCP wq(h) %s\n", h);
+	}
+	printk(KERN_ERR "l%u s%u f%u p%u seq: su%u hs%u sn%u\n",
+		tp->lost_out, tp->sacked_out, tp->fackets_out, tp->packets_out,
+		tp->snd_una, tcp_highest_sack_seq(tp), tp->snd_nxt);
+}
+
+void tcp_verify_wq(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+	u32 lost = 0;
+	u32 sacked = 0;
+	u32 packets = 0;
+	struct sk_buff *skb;
+
+	tcp_for_write_queue(skb, sk) {
+		if (skb == tcp_send_head(sk))
+			break;
+
+		if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) {
+			sacked += tcp_skb_pcount(skb);
+			if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+			printk(KERN_ERR "Sacked bitmap S+L: %u %u-%u/%u\n",
+				TCP_SKB_CB(skb)->sacked,
+				TCP_SKB_CB(skb)->end_seq - tp->snd_una,
+				TCP_SKB_CB(skb)->seq - tp->snd_una,
+				tp->snd_una);
+		}
+		if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+			lost += tcp_skb_pcount(skb);
+
+		packets += tcp_skb_pcount(skb);
+	}
+
+	WARN_ON(lost != tp->lost_out);
+	WARN_ON(sacked != tp->sacked_out);
+	WARN_ON(packets != tp->packets_out);
+	if ((lost != tp->lost_out) ||
+	    (sacked != tp->sacked_out) ||
+	    (packets != tp->packets_out)) {
+		printk(KERN_ERR "P: %u L: %u vs %u S: %u vs %u w: %u-%u (%u)\n",
+			tp->packets_out,
+			lost, tp->lost_out,
+			sacked, tp->sacked_out,
+			tp->snd_una, tp->snd_nxt,
+		       	tp->rx_opt.sack_ok);
+		tcp_print_queue(sk);
+	}
+}
+
 static int tcp_v4_get_port(struct sock *sk, unsigned short snum)
 {
 	return inet_csk_get_port(&tcp_hashinfo, sk, snum,
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 89f0188..648340f 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -779,10 +779,9 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len,
 			tp->lost_out -= diff;
 
 		/* Adjust Reno SACK estimate. */
-		if (tcp_is_reno(tp) && diff > 0) {
+		if (tcp_is_reno(tp) && diff > 0)
 			tcp_dec_pcount_approx_int(&tp->sacked_out, diff);
-			tcp_verify_left_out(tp);
-		}
+
 		tcp_adjust_fackets_out(sk, skb, diff);
 	}
 
@@ -790,6 +789,8 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len,
 	skb_header_release(buff);
 	tcp_insert_write_queue_after(skb, buff, sk);
 
+	tcp_verify_left_out(tp);
+
 	return 0;
 }
 
@@ -1463,6 +1464,7 @@ static int tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle)
 	} else if (result > 0) {
 		sent_pkts = 1;
 	}
+	tcp_verify_left_out(tp);
 
 	while ((skb = tcp_send_head(sk))) {
 		unsigned int limit;
@@ -1764,6 +1766,7 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *skb,
 	tcp_clear_retrans_hints_partial(tp);
 
 	sk_wmem_free_skb(sk, next_skb);
+	tcp_verify_left_out(tp);
 }
 
 /* Do a simple retransmit without using the backoff mechanisms in
@@ -1795,13 +1798,13 @@ void tcp_simple_retransmit(struct sock *sk)
 		}
 	}
 
+	tcp_verify_left_out(tp);
+
 	tcp_clear_all_retrans_hints(tp);
 
 	if (!lost)
 		return;
 
-	tcp_verify_left_out(tp);
-
 	/* Don't muck with the congestion window here.
 	 * Reason is that we do not increase amount of _data_
 	 * in network, but units changed and effective
@@ -1970,8 +1973,10 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 			 * packet to be MSS sized and all the
 			 * packet counting works out.
 			 */
-			if (tcp_packets_in_flight(tp) >= tp->snd_cwnd)
+			if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) {
+				tcp_verify_left_out(tp);
 				return;
+			}
 
 			if (sacked & TCPCB_LOST) {
 				if (!(sacked & (TCPCB_SACKED_ACKED|TCPCB_SACKED_RETRANS))) {
@@ -1997,6 +2002,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 		}
 	}
 
+	tcp_verify_left_out(tp);
+
 	/* OK, demanded retransmission is finished. */
 
 	/* Forward retransmissions are possible only during Recovery. */
@@ -2054,6 +2061,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 
 		NET_INC_STATS_BH(LINUX_MIB_TCPFORWARDRETRANS);
 	}
+
+	tcp_verify_left_out(tp);
 }
 
 /* Send a fin.  The caller locks the socket for us.  This cannot be
-- 
1.5.2.2

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-22 10:47       ` Ilpo Järvinen
@ 2008-01-23  1:44         ` Dave Young
  2008-01-23  3:01           ` David Miller
  0 siblings, 1 reply; 18+ messages in thread
From: Dave Young @ 2008-01-23  1:44 UTC (permalink / raw)
  To: Ilpo Järvinen; +Cc: LKML, David Miller, Netdev, Andrew Morton

On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
>
> On Tue, 22 Jan 2008, Dave Young wrote:
>
> > On Jan 22, 2008 12:37 PM, Dave Young <hidave.darkstar@gmail.com> wrote:
> > >
> > > On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > > >
> > > > On Mon, 21 Jan 2008, Dave Young wrote:
> > > >
> > > > > Please see the kernel messages following,(trigged while using some qemu session)
> > > > > BTW, seems there's some e100 error message as well.
> > > > >
> > > > > PCI: Setting latency timer of device 0000:00:1b.0 to 64
> > > > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
> > > > > e100: Copyright(c) 1999-2006 Intel Corporation
> > > > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20
> > > > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default
> > > > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting.
> > > > > ACPI: PCI interrupt for device 0000:03:08.0 disabled
> > > > > e100: probe of 0000:03:08.0 failed with error -12
> > > > > eth0:  setting full-duplex.
> > > > > ------------[ cut here ]------------
> > > > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150()
> > > > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc
> > > > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4
> > > > >  [<c0132100>] ? printk+0x0/0x20
> > > > >  [<c0131834>] warn_on_slowpath+0x54/0x80
> > > > >  [<c03e8df8>] ? ip_finish_output+0x128/0x2e0
> > > > >  [<c03e9527>] ? ip_output+0xe7/0x100
> > > > >  [<c03e8a88>] ? ip_local_out+0x18/0x20
> > > > >  [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470
> > > > >  [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70
> > > > >  [<c0186be1>] ? check_pad_bytes+0x61/0x80
> > > > >  [<c03f6031>] tcp_mark_head_lost+0x121/0x150
> > > > >  [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170
> > > > >  [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0
> > > > >  [<c03f7d93>] tcp_ack+0x1b3/0x3a0
> > > > >  [<c03fa14b>] tcp_rcv_established+0x3eb/0x710
> > > > >  [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100
> > > > >  [<c0401bbb>] tcp_v4_rcv+0x5db/0x660
> > > >
> > > > Doh, once more these S+L things..., the rest are symptom of the first
> > > > problem.
> > >
> > > What is the S+L thing? Could you explain a bit?
>
> It means that one of the skbs is both SACKed and marked as LOST at the
> same time in the counters (might be due to miscount of lost/sacked_out
> too, not necessarilily in the ->sacked bits). Such state is logically
> invalid because it would mean that the sender thinks that the same packet
> both reached the receiver and is lost in the network.
>
> Traditionally TCP has just silently "corrected" over-estimates
> (sacked_out+lost_out > packets_out). I changed this couple of releases ago
> because those over-estimates often are due to bugs that should be fixed
> (there have been couple of them but it has been very quite on this front
> long time, months or even half year already; but I might have broken
> something with the early Dec changes).
>
> These problem may originate from a bug that occurred a number of ACKs
> earlier the WARN_ON triggered, therefore they are a bit tricky to track,
> those WARN_ON serve just for alerting purposes and usually do not point
> out where the bug actually occurred.
>
> I usually just asked people to include exhaustive verifier which compares
> ->sacked bitmaps with sacked/lost_out counters and report immediately when
> the problem shows up, rather than waiting for the cheaper S+L check we do
> in the WARN_ON to trigger. I tried to collect tracking patch from the
> previous efforts (hopefully got it right after modifications).
>
> > > I'm a bit worried about its
> > > > reproducability if it takes this far to see it...
> > > >
> >
> > It's trigged again in my pc, just while using firefox.
>
> ...Good, then there's some chance to catch it.
>
> --
>  i.
>
> [PATCH] [TCP]: debug S+L

Thanks, If there's new findings I will let you know.

>
> ---
>  include/net/tcp.h     |    8 +++-
>  net/ipv4/tcp_input.c  |    6 +++
>  net/ipv4/tcp_ipv4.c   |  101 +++++++++++++++++++++++++++++++++++++++++++++++++
>  net/ipv4/tcp_output.c |   21 +++++++---
>  4 files changed, 129 insertions(+), 7 deletions(-)
>
> diff --git a/include/net/tcp.h b/include/net/tcp.h
> index 7de4ea3..0685035 100644
> --- a/include/net/tcp.h
> +++ b/include/net/tcp.h
> @@ -272,6 +272,8 @@ DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics);
>  #define TCP_ADD_STATS_BH(field, val)   SNMP_ADD_STATS_BH(tcp_statistics, field, val)
>  #define TCP_ADD_STATS_USER(field, val) SNMP_ADD_STATS_USER(tcp_statistics, field, val)
>
> +extern void                    tcp_verify_wq(struct sock *sk);
> +
>  extern void                    tcp_v4_err(struct sk_buff *skb, u32);
>
>  extern void                    tcp_shutdown (struct sock *sk, int how);
> @@ -768,7 +770,11 @@ static inline __u32 tcp_current_ssthresh(const struct sock *sk)
>  }
>
>  /* Use define here intentionally to get WARN_ON location shown at the caller */
> -#define tcp_verify_left_out(tp)        WARN_ON(tcp_left_out(tp) > tp->packets_out)
> +#define tcp_verify_left_out(tp)        \
> +       do { \
> +               WARN_ON(tcp_left_out(tp) > tp->packets_out); \
> +               tcp_verify_wq((struct sock *)tp); \
> +       } while(0)
>
>  extern void tcp_enter_cwr(struct sock *sk, const int set_ssthresh);
>  extern __u32 tcp_init_cwnd(struct tcp_sock *tp, struct dst_entry *dst);
> diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> index fa2c85c..0bda0e1 100644
> --- a/net/ipv4/tcp_input.c
> +++ b/net/ipv4/tcp_input.c
> @@ -2645,6 +2645,10 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag)
>         if (do_lost || (tcp_is_fack(tp) && tcp_head_timedout(sk)))
>                 tcp_update_scoreboard(sk, fast_rexmit);
>         tcp_cwnd_down(sk, flag);
> +
> +       WARN_ON(tcp_write_queue_head(sk) == NULL);
> +       WARN_ON(!tp->packets_out);
> +
>         tcp_xmit_retransmit_queue(sk);
>  }
>
> @@ -2848,6 +2852,8 @@ static int tcp_clean_rtx_queue(struct sock *sk, int prior_fackets)
>                 tcp_clear_all_retrans_hints(tp);
>         }
>
> +       tcp_verify_left_out(tp);
> +
>         if (skb && (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED))
>                 flag |= FLAG_SACK_RENEGING;
>
> diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
> index 9aea88b..21f5888 100644
> --- a/net/ipv4/tcp_ipv4.c
> +++ b/net/ipv4/tcp_ipv4.c
> @@ -108,6 +108,107 @@ struct inet_hashinfo __cacheline_aligned tcp_hashinfo = {
>         .lhash_wait  = __WAIT_QUEUE_HEAD_INITIALIZER(tcp_hashinfo.lhash_wait),
>  };
>
> +void tcp_print_queue(struct sock *sk)
> +{
> +       struct tcp_sock *tp = tcp_sk(sk);
> +       struct sk_buff *skb;
> +       char s[50+1];
> +       char h[50+1];
> +       int idx = 0;
> +       int i;
> +
> +       tcp_for_write_queue(skb, sk) {
> +               if (skb == tcp_send_head(sk))
> +                       break;
> +
> +               for (i = 0; i < tcp_skb_pcount(skb); i++) {
> +                       if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) {
> +                               s[idx] = 'S';
> +                               if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
> +                                       s[idx] = 'B';
> +
> +                       } else if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) {
> +                               s[idx] = 'L';
> +                       } else {
> +                               s[idx] = ' ';
> +                       }
> +                       if (s[idx] != ' ' && skb->len < tp->mss_cache)
> +                               s[idx] += 'a' - 'A';
> +
> +                       if (i == 0) {
> +                               if (TCP_SKB_CB(skb)->seq == tcp_highest_sack_seq(tp))
> +                                       h[idx] = 'h';
> +                               else
> +                                       h[idx] = '+';
> +                       } else {
> +                               h[idx] = '-';
> +                       }
> +
> +                       if (++idx >= 50) {
> +                               s[idx] = 0;
> +                               h[idx] = 0;
> +                               printk(KERN_ERR "TCP wq(s) %s\n", s);
> +                               printk(KERN_ERR "TCP wq(h) %s\n", h);
> +                               idx = 0;
> +                       }
> +               }
> +       }
> +       if (idx) {
> +               s[idx] = '<';
> +               s[idx+1] = 0;
> +               h[idx] = '<';
> +               h[idx+1] = 0;
> +               printk(KERN_ERR "TCP wq(s) %s\n", s);
> +               printk(KERN_ERR "TCP wq(h) %s\n", h);
> +       }
> +       printk(KERN_ERR "l%u s%u f%u p%u seq: su%u hs%u sn%u\n",
> +               tp->lost_out, tp->sacked_out, tp->fackets_out, tp->packets_out,
> +               tp->snd_una, tcp_highest_sack_seq(tp), tp->snd_nxt);
> +}
> +
> +void tcp_verify_wq(struct sock *sk)
> +{
> +       struct tcp_sock *tp = tcp_sk(sk);
> +       u32 lost = 0;
> +       u32 sacked = 0;
> +       u32 packets = 0;
> +       struct sk_buff *skb;
> +
> +       tcp_for_write_queue(skb, sk) {
> +               if (skb == tcp_send_head(sk))
> +                       break;
> +
> +               if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) {
> +                       sacked += tcp_skb_pcount(skb);
> +                       if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
> +                       printk(KERN_ERR "Sacked bitmap S+L: %u %u-%u/%u\n",
> +                               TCP_SKB_CB(skb)->sacked,
> +                               TCP_SKB_CB(skb)->end_seq - tp->snd_una,
> +                               TCP_SKB_CB(skb)->seq - tp->snd_una,
> +                               tp->snd_una);
> +               }
> +               if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
> +                       lost += tcp_skb_pcount(skb);
> +
> +               packets += tcp_skb_pcount(skb);
> +       }
> +
> +       WARN_ON(lost != tp->lost_out);
> +       WARN_ON(sacked != tp->sacked_out);
> +       WARN_ON(packets != tp->packets_out);
> +       if ((lost != tp->lost_out) ||
> +           (sacked != tp->sacked_out) ||
> +           (packets != tp->packets_out)) {
> +               printk(KERN_ERR "P: %u L: %u vs %u S: %u vs %u w: %u-%u (%u)\n",
> +                       tp->packets_out,
> +                       lost, tp->lost_out,
> +                       sacked, tp->sacked_out,
> +                       tp->snd_una, tp->snd_nxt,
> +                       tp->rx_opt.sack_ok);
> +               tcp_print_queue(sk);
> +       }
> +}
> +
>  static int tcp_v4_get_port(struct sock *sk, unsigned short snum)
>  {
>         return inet_csk_get_port(&tcp_hashinfo, sk, snum,
> diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
> index 89f0188..648340f 100644
> --- a/net/ipv4/tcp_output.c
> +++ b/net/ipv4/tcp_output.c
> @@ -779,10 +779,9 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len,
>                         tp->lost_out -= diff;
>
>                 /* Adjust Reno SACK estimate. */
> -               if (tcp_is_reno(tp) && diff > 0) {
> +               if (tcp_is_reno(tp) && diff > 0)
>                         tcp_dec_pcount_approx_int(&tp->sacked_out, diff);
> -                       tcp_verify_left_out(tp);
> -               }
> +
>                 tcp_adjust_fackets_out(sk, skb, diff);
>         }
>
> @@ -790,6 +789,8 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len,
>         skb_header_release(buff);
>         tcp_insert_write_queue_after(skb, buff, sk);
>
> +       tcp_verify_left_out(tp);
> +
>         return 0;
>  }
>
> @@ -1463,6 +1464,7 @@ static int tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle)
>         } else if (result > 0) {
>                 sent_pkts = 1;
>         }
> +       tcp_verify_left_out(tp);
>
>         while ((skb = tcp_send_head(sk))) {
>                 unsigned int limit;
> @@ -1764,6 +1766,7 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *skb,
>         tcp_clear_retrans_hints_partial(tp);
>
>         sk_wmem_free_skb(sk, next_skb);
> +       tcp_verify_left_out(tp);
>  }
>
>  /* Do a simple retransmit without using the backoff mechanisms in
> @@ -1795,13 +1798,13 @@ void tcp_simple_retransmit(struct sock *sk)
>                 }
>         }
>
> +       tcp_verify_left_out(tp);
> +
>         tcp_clear_all_retrans_hints(tp);
>
>         if (!lost)
>                 return;
>
> -       tcp_verify_left_out(tp);
> -
>         /* Don't muck with the congestion window here.
>          * Reason is that we do not increase amount of _data_
>          * in network, but units changed and effective
> @@ -1970,8 +1973,10 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
>                          * packet to be MSS sized and all the
>                          * packet counting works out.
>                          */
> -                       if (tcp_packets_in_flight(tp) >= tp->snd_cwnd)
> +                       if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) {
> +                               tcp_verify_left_out(tp);
>                                 return;
> +                       }
>
>                         if (sacked & TCPCB_LOST) {
>                                 if (!(sacked & (TCPCB_SACKED_ACKED|TCPCB_SACKED_RETRANS))) {
> @@ -1997,6 +2002,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
>                 }
>         }
>
> +       tcp_verify_left_out(tp);
> +
>         /* OK, demanded retransmission is finished. */
>
>         /* Forward retransmissions are possible only during Recovery. */
> @@ -2054,6 +2061,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
>
>                 NET_INC_STATS_BH(LINUX_MIB_TCPFORWARDRETRANS);
>         }
> +
> +       tcp_verify_left_out(tp);
>  }
>
>  /* Send a fin.  The caller locks the socket for us.  This cannot be
> --
> 1.5.2.2
>

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-23  1:44         ` Dave Young
@ 2008-01-23  3:01           ` David Miller
  2008-01-23  7:41             ` Ilpo Järvinen
  0 siblings, 1 reply; 18+ messages in thread
From: David Miller @ 2008-01-23  3:01 UTC (permalink / raw)
  To: hidave.darkstar; +Cc: ilpo.jarvinen, linux-kernel, netdev, akpm

From: "Dave Young" <hidave.darkstar@gmail.com>
Date: Wed, 23 Jan 2008 09:44:30 +0800

> On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > [PATCH] [TCP]: debug S+L
> 
> Thanks, If there's new findings I will let you know.

Thanks for helping with this bug Dave.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-23  3:01           ` David Miller
@ 2008-01-23  7:41             ` Ilpo Järvinen
  2008-01-23  7:44               ` Dave Young
  0 siblings, 1 reply; 18+ messages in thread
From: Ilpo Järvinen @ 2008-01-23  7:41 UTC (permalink / raw)
  To: hidave.darkstar; +Cc: LKML, Netdev, Andrew Morton

[-- Attachment #1: Type: TEXT/PLAIN, Size: 769 bytes --]

On Tue, 22 Jan 2008, David Miller wrote:

> From: "Dave Young" <hidave.darkstar@gmail.com>
> Date: Wed, 23 Jan 2008 09:44:30 +0800
> 
> > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > > [PATCH] [TCP]: debug S+L
> > 
> > Thanks, If there's new findings I will let you know.
> 
> Thanks for helping with this bug Dave.

I noticed btw that there thing might (is likely to) spuriously trigger at 
WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK 
is not enabled. If that does happen too often, I send a fixed patch for 
it, yet, the fact that I print print tp->rx_opt.sack_ok allows
identification of those cases already as it's zero when SACK is not 
enabled.

Just ask if you need the updated debug patch.

-- 
 i.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-23  7:41             ` Ilpo Järvinen
@ 2008-01-23  7:44               ` Dave Young
  2008-01-23  8:41                 ` Ilpo Järvinen
  0 siblings, 1 reply; 18+ messages in thread
From: Dave Young @ 2008-01-23  7:44 UTC (permalink / raw)
  To: Ilpo Järvinen; +Cc: LKML, Netdev, Andrew Morton

On Jan 23, 2008 3:41 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
>
> On Tue, 22 Jan 2008, David Miller wrote:
>
> > From: "Dave Young" <hidave.darkstar@gmail.com>
> > Date: Wed, 23 Jan 2008 09:44:30 +0800
> >
> > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > > > [PATCH] [TCP]: debug S+L
> > >
> > > Thanks, If there's new findings I will let you know.
> >
> > Thanks for helping with this bug Dave.
>
> I noticed btw that there thing might (is likely to) spuriously trigger at
> WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK
> is not enabled. If that does happen too often, I send a fixed patch for
> it, yet, the fact that I print print tp->rx_opt.sack_ok allows
> identification of those cases already as it's zero when SACK is not
> enabled.
>
> Just ask if you need the updated debug patch.

Thanks,  please send, I would like to get it.

>
> --
>  i.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-23  7:44               ` Dave Young
@ 2008-01-23  8:41                 ` Ilpo Järvinen
  2008-01-23 11:01                   ` Ilpo Järvinen
  0 siblings, 1 reply; 18+ messages in thread
From: Ilpo Järvinen @ 2008-01-23  8:41 UTC (permalink / raw)
  To: Dave Young; +Cc: LKML, Netdev, Andrew Morton

[-- Attachment #1: Type: TEXT/PLAIN, Size: 9210 bytes --]

On Wed, 23 Jan 2008, Dave Young wrote:

> On Jan 23, 2008 3:41 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> >
> > On Tue, 22 Jan 2008, David Miller wrote:
> >
> > > From: "Dave Young" <hidave.darkstar@gmail.com>
> > > Date: Wed, 23 Jan 2008 09:44:30 +0800
> > >
> > > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > > > > [PATCH] [TCP]: debug S+L
> > > >
> > > > Thanks, If there's new findings I will let you know.
> > >
> > > Thanks for helping with this bug Dave.
> >
> > I noticed btw that there thing might (is likely to) spuriously trigger at
> > WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK
> > is not enabled. If that does happen too often, I send a fixed patch for
> > it, yet, the fact that I print print tp->rx_opt.sack_ok allows
> > identification of those cases already as it's zero when SACK is not
> > enabled.
> >
> > Just ask if you need the updated debug patch.
> 
> Thanks,  please send, I would like to get it.

There you go. I fixed non-SACK case by adding tcp_is_sack checks there and 
also added two verifys to tcp_ack to see if there's corruption outside of 
TCP.

-- 
 i.

[PATCH] [TCP]: debug S+L

---
 include/net/tcp.h     |    8 +++-
 net/ipv4/tcp_input.c  |   10 +++++
 net/ipv4/tcp_ipv4.c   |  101 +++++++++++++++++++++++++++++++++++++++++++++++++
 net/ipv4/tcp_output.c |   21 +++++++---
 4 files changed, 133 insertions(+), 7 deletions(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 7de4ea3..0685035 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -272,6 +272,8 @@ DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics);
 #define TCP_ADD_STATS_BH(field, val)	SNMP_ADD_STATS_BH(tcp_statistics, field, val)
 #define TCP_ADD_STATS_USER(field, val)	SNMP_ADD_STATS_USER(tcp_statistics, field, val)
 
+extern void			tcp_verify_wq(struct sock *sk);
+
 extern void			tcp_v4_err(struct sk_buff *skb, u32);
 
 extern void			tcp_shutdown (struct sock *sk, int how);
@@ -768,7 +770,11 @@ static inline __u32 tcp_current_ssthresh(const struct sock *sk)
 }
 
 /* Use define here intentionally to get WARN_ON location shown at the caller */
-#define tcp_verify_left_out(tp)	WARN_ON(tcp_left_out(tp) > tp->packets_out)
+#define tcp_verify_left_out(tp)	\
+	do { \
+		WARN_ON(tcp_left_out(tp) > tp->packets_out); \
+		tcp_verify_wq((struct sock *)tp); \
+	} while(0)
 
 extern void tcp_enter_cwr(struct sock *sk, const int set_ssthresh);
 extern __u32 tcp_init_cwnd(struct tcp_sock *tp, struct dst_entry *dst);
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index fa2c85c..cdacf70 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -2645,6 +2645,10 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag)
 	if (do_lost || (tcp_is_fack(tp) && tcp_head_timedout(sk)))
 		tcp_update_scoreboard(sk, fast_rexmit);
 	tcp_cwnd_down(sk, flag);
+
+	WARN_ON(tcp_write_queue_head(sk) == NULL);
+	WARN_ON(!tp->packets_out);
+
 	tcp_xmit_retransmit_queue(sk);
 }
 
@@ -2848,6 +2852,8 @@ static int tcp_clean_rtx_queue(struct sock *sk, int prior_fackets)
 		tcp_clear_all_retrans_hints(tp);
 	}
 
+	tcp_verify_left_out(tp);
+
 	if (skb && (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED))
 		flag |= FLAG_SACK_RENEGING;
 
@@ -3175,6 +3181,8 @@ static int tcp_ack(struct sock *sk, struct sk_buff *skb, int flag)
 	prior_fackets = tp->fackets_out;
 	prior_in_flight = tcp_packets_in_flight(tp);
 
+	tcp_verify_left_out(tp);
+
 	if (!(flag & FLAG_SLOWPATH) && after(ack, prior_snd_una)) {
 		/* Window is constant, pure forward advance.
 		 * No more checks are required.
@@ -3237,6 +3245,8 @@ static int tcp_ack(struct sock *sk, struct sk_buff *skb, int flag)
 	if ((flag & FLAG_FORWARD_PROGRESS) || !(flag & FLAG_NOT_DUP))
 		dst_confirm(sk->sk_dst_cache);
 
+	tcp_verify_left_out(tp);
+
 	return 1;
 
 no_queue:
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 9aea88b..7e8ab40 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -108,6 +108,107 @@ struct inet_hashinfo __cacheline_aligned tcp_hashinfo = {
 	.lhash_wait  = __WAIT_QUEUE_HEAD_INITIALIZER(tcp_hashinfo.lhash_wait),
 };
 
+void tcp_print_queue(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct sk_buff *skb;
+	char s[50+1];
+	char h[50+1];
+	int idx = 0;
+	int i;
+
+	tcp_for_write_queue(skb, sk) {
+		if (skb == tcp_send_head(sk))
+			break;
+
+		for (i = 0; i < tcp_skb_pcount(skb); i++) {
+			if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) {
+				s[idx] = 'S';
+				if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+					s[idx] = 'B';
+
+			} else if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) {
+				s[idx] = 'L';
+			} else {
+				s[idx] = ' ';
+			}
+			if (s[idx] != ' ' && skb->len < tp->mss_cache)
+				s[idx] += 'a' - 'A';
+
+			if (i == 0) {
+				if (TCP_SKB_CB(skb)->seq == tcp_highest_sack_seq(tp))
+					h[idx] = 'h';
+				else
+					h[idx] = '+';
+			} else {
+				h[idx] = '-';
+			}
+
+			if (++idx >= 50) {
+				s[idx] = 0;
+				h[idx] = 0;
+				printk(KERN_ERR "TCP wq(s) %s\n", s);
+				printk(KERN_ERR "TCP wq(h) %s\n", h);
+				idx = 0;
+			}
+		}
+	}
+	if (idx) {
+		s[idx] = '<';
+		s[idx+1] = 0;
+		h[idx] = '<';
+		h[idx+1] = 0;
+		printk(KERN_ERR "TCP wq(s) %s\n", s);
+		printk(KERN_ERR "TCP wq(h) %s\n", h);
+	}
+	printk(KERN_ERR "l%u s%u f%u p%u seq: su%u hs%u sn%u\n",
+		tp->lost_out, tp->sacked_out, tp->fackets_out, tp->packets_out,
+		tp->snd_una, tcp_highest_sack_seq(tp), tp->snd_nxt);
+}
+
+void tcp_verify_wq(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+	u32 lost = 0;
+	u32 sacked = 0;
+	u32 packets = 0;
+	struct sk_buff *skb;
+
+	tcp_for_write_queue(skb, sk) {
+		if (skb == tcp_send_head(sk))
+			break;
+
+		if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) {
+			sacked += tcp_skb_pcount(skb);
+			if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+			printk(KERN_ERR "Sacked bitmap S+L: %u %u-%u/%u\n",
+				TCP_SKB_CB(skb)->sacked,
+				TCP_SKB_CB(skb)->end_seq - tp->snd_una,
+				TCP_SKB_CB(skb)->seq - tp->snd_una,
+				tp->snd_una);
+		}
+		if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+			lost += tcp_skb_pcount(skb);
+
+		packets += tcp_skb_pcount(skb);
+	}
+
+	WARN_ON(lost != tp->lost_out);
+	WARN_ON(tcp_is_sack(tp) && (sacked != tp->sacked_out));
+	WARN_ON(packets != tp->packets_out);
+	if ((lost != tp->lost_out) ||
+	    (tcp_is_sack(tp) && (sacked != tp->sacked_out)) ||
+	    (packets != tp->packets_out)) {
+		printk(KERN_ERR "P: %u L: %u vs %u S: %u vs %u w: %u-%u (%u)\n",
+			tp->packets_out,
+			lost, tp->lost_out,
+			sacked, tp->sacked_out,
+			tp->snd_una, tp->snd_nxt,
+		       	tp->rx_opt.sack_ok);
+		tcp_print_queue(sk);
+	}
+}
+
 static int tcp_v4_get_port(struct sock *sk, unsigned short snum)
 {
 	return inet_csk_get_port(&tcp_hashinfo, sk, snum,
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 89f0188..648340f 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -779,10 +779,9 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len,
 			tp->lost_out -= diff;
 
 		/* Adjust Reno SACK estimate. */
-		if (tcp_is_reno(tp) && diff > 0) {
+		if (tcp_is_reno(tp) && diff > 0)
 			tcp_dec_pcount_approx_int(&tp->sacked_out, diff);
-			tcp_verify_left_out(tp);
-		}
+
 		tcp_adjust_fackets_out(sk, skb, diff);
 	}
 
@@ -790,6 +789,8 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len,
 	skb_header_release(buff);
 	tcp_insert_write_queue_after(skb, buff, sk);
 
+	tcp_verify_left_out(tp);
+
 	return 0;
 }
 
@@ -1463,6 +1464,7 @@ static int tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle)
 	} else if (result > 0) {
 		sent_pkts = 1;
 	}
+	tcp_verify_left_out(tp);
 
 	while ((skb = tcp_send_head(sk))) {
 		unsigned int limit;
@@ -1764,6 +1766,7 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *skb,
 	tcp_clear_retrans_hints_partial(tp);
 
 	sk_wmem_free_skb(sk, next_skb);
+	tcp_verify_left_out(tp);
 }
 
 /* Do a simple retransmit without using the backoff mechanisms in
@@ -1795,13 +1798,13 @@ void tcp_simple_retransmit(struct sock *sk)
 		}
 	}
 
+	tcp_verify_left_out(tp);
+
 	tcp_clear_all_retrans_hints(tp);
 
 	if (!lost)
 		return;
 
-	tcp_verify_left_out(tp);
-
 	/* Don't muck with the congestion window here.
 	 * Reason is that we do not increase amount of _data_
 	 * in network, but units changed and effective
@@ -1970,8 +1973,10 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 			 * packet to be MSS sized and all the
 			 * packet counting works out.
 			 */
-			if (tcp_packets_in_flight(tp) >= tp->snd_cwnd)
+			if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) {
+				tcp_verify_left_out(tp);
 				return;
+			}
 
 			if (sacked & TCPCB_LOST) {
 				if (!(sacked & (TCPCB_SACKED_ACKED|TCPCB_SACKED_RETRANS))) {
@@ -1997,6 +2002,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 		}
 	}
 
+	tcp_verify_left_out(tp);
+
 	/* OK, demanded retransmission is finished. */
 
 	/* Forward retransmissions are possible only during Recovery. */
@@ -2054,6 +2061,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 
 		NET_INC_STATS_BH(LINUX_MIB_TCPFORWARDRETRANS);
 	}
+
+	tcp_verify_left_out(tp);
 }
 
 /* Send a fin.  The caller locks the socket for us.  This cannot be
-- 
1.5.2.2

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-23  8:41                 ` Ilpo Järvinen
@ 2008-01-23 11:01                   ` Ilpo Järvinen
  2008-01-24  2:42                     ` Dave Young
  0 siblings, 1 reply; 18+ messages in thread
From: Ilpo Järvinen @ 2008-01-23 11:01 UTC (permalink / raw)
  To: Dave Young; +Cc: LKML, Netdev, Andrew Morton

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1471 bytes --]

On Wed, 23 Jan 2008, Ilpo Järvinen wrote:

> On Wed, 23 Jan 2008, Dave Young wrote:
> 
> > On Jan 23, 2008 3:41 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > >
> > > On Tue, 22 Jan 2008, David Miller wrote:
> > >
> > > > From: "Dave Young" <hidave.darkstar@gmail.com>
> > > > Date: Wed, 23 Jan 2008 09:44:30 +0800
> > > >
> > > > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > > > > > [PATCH] [TCP]: debug S+L
> > > > >
> > > > > Thanks, If there's new findings I will let you know.
> > > >
> > > > Thanks for helping with this bug Dave.
> > >
> > > I noticed btw that there thing might (is likely to) spuriously trigger at
> > > WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK
> > > is not enabled. If that does happen too often, I send a fixed patch for
> > > it, yet, the fact that I print print tp->rx_opt.sack_ok allows
> > > identification of those cases already as it's zero when SACK is not
> > > enabled.
> > >
> > > Just ask if you need the updated debug patch.
> > 
> > Thanks,  please send, I would like to get it.
> 
> There you go. I fixed non-SACK case by adding tcp_is_sack checks there and 
> also added two verifys to tcp_ack to see if there's corruption outside of 
> TCP.

There's some discussion about a problem that is very likely the same as in 
here (sorry for not remembering to cc you in there due to rapid progress):

   http://marc.info/?t=120107174200003&r=1&w=2


-- 
 i.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-23 11:01                   ` Ilpo Järvinen
@ 2008-01-24  2:42                     ` Dave Young
  2008-01-24  9:54                       ` Ilpo Järvinen
  0 siblings, 1 reply; 18+ messages in thread
From: Dave Young @ 2008-01-24  2:42 UTC (permalink / raw)
  To: Ilpo Järvinen; +Cc: LKML, Netdev, Andrew Morton

On Jan 23, 2008 7:01 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> On Wed, 23 Jan 2008, Ilpo Järvinen wrote:
>
> > On Wed, 23 Jan 2008, Dave Young wrote:
> >
> > > On Jan 23, 2008 3:41 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > > >
> > > > On Tue, 22 Jan 2008, David Miller wrote:
> > > >
> > > > > From: "Dave Young" <hidave.darkstar@gmail.com>
> > > > > Date: Wed, 23 Jan 2008 09:44:30 +0800
> > > > >
> > > > > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> > > > > > > [PATCH] [TCP]: debug S+L
> > > > > >
> > > > > > Thanks, If there's new findings I will let you know.
> > > > >
> > > > > Thanks for helping with this bug Dave.
> > > >
> > > > I noticed btw that there thing might (is likely to) spuriously trigger at
> > > > WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK
> > > > is not enabled. If that does happen too often, I send a fixed patch for
> > > > it, yet, the fact that I print print tp->rx_opt.sack_ok allows
> > > > identification of those cases already as it's zero when SACK is not
> > > > enabled.
> > > >
> > > > Just ask if you need the updated debug patch.
> > >
> > > Thanks,  please send, I would like to get it.
> >
> > There you go. I fixed non-SACK case by adding tcp_is_sack checks there and
> > also added two verifys to tcp_ack to see if there's corruption outside of
> > TCP.
>
> There's some discussion about a problem that is very likely the same as in
> here (sorry for not remembering to cc you in there due to rapid progress):
>
>    http://marc.info/?t=120107174200003&r=1&w=2

Thanks.

New warning trigged with your debug patch:

ACPI: PCI Interrupt 0000:00:1b.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:1b.0 to 64
e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
e100: Copyright(c) 1999-2006 Intel Corporation
ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20
e100: eth1: e100_probe: addr 0xefaff000, irq 20, MAC addr 00:13:72:e7:4d:66
eth0:  setting full-duplex.
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0150030>] ? timer_list_show_tickdevices+0xf0/0x110
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c0153251>] ? put_lock_stats+0x21/0x30
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0
 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0
 [<c0186bf1>] ? check_pad_bytes+0x61/0x80
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03f7509>] ? tcp_clean_rtx_queue+0x2d9/0x5b0
 [<c03f5540>] tcp_add_reno_sack+0x30/0x50
 [<c03f6d72>] tcp_fastretrans_alert+0x3d2/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f3e>] ? __do_softirq+0x6e/0x120
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0250030>] ? generic_make_request+0x1c0/0x2e0
 [<c0132128>] ? printk+0x18/0x20
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c03f674e>] tcp_try_to_open+0x2e/0x160
 [<c03f6e9e>] tcp_fastretrans_alert+0x4fe/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f3e>] ? __do_softirq+0x6e/0x120
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0
 [<c0186bf1>] ? check_pad_bytes+0x61/0x80
 [<c0186dbd>] ? check_object+0xad/0x1e0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130
 [<c03c1578>] ? kfree_skbmem+0x98/0xc0
 [<c03fcf03>] tcp_write_xmit+0x63/0x290
 [<c03c1578>] ? kfree_skbmem+0x98/0xc0
 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80
 [<c03fa23e>] tcp_rcv_established+0x40e/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f3e>] ? __do_softirq+0x6e/0x120
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0132128>] ? printk+0x18/0x20
 [<c01317da>] ? print_oops_end_marker+0x2a/0x30
 [<c0131849>] ? warn_on_slowpath+0x59/0x80
 [<c025f9e4>] ? vsnprintf+0x2d4/0x4f0
 [<c0132128>] ? printk+0x18/0x20
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c0186bf1>] ? check_pad_bytes+0x61/0x80
 [<c03f7509>] tcp_clean_rtx_queue+0x2d9/0x5b0
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c03f7dd2>] tcp_ack+0x122/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f3e>] ? __do_softirq+0x6e/0x120
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0132438>] ? vprintk+0x308/0x320
 [<c025f9e4>] ? vsnprintf+0x2d4/0x4f0
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03f7509>] ? tcp_clean_rtx_queue+0x2d9/0x5b0
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c03f6a93>] tcp_fastretrans_alert+0xf3/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f3e>] ? __do_softirq+0x6e/0x120
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0250030>] ? generic_make_request+0x1c0/0x2e0
 [<c0132128>] ? printk+0x18/0x20
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c03f5540>] tcp_add_reno_sack+0x30/0x50
 [<c03f6d72>] tcp_fastretrans_alert+0x3d2/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f3e>] ? __do_softirq+0x6e/0x120
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0250030>] ? generic_make_request+0x1c0/0x2e0
 [<c0132128>] ? printk+0x18/0x20
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c03f674e>] tcp_try_to_open+0x2e/0x160
 [<c03f6e9e>] tcp_fastretrans_alert+0x4fe/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f3e>] ? __do_softirq+0x6e/0x120
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0186bf1>] ? check_pad_bytes+0x61/0x80
 [<c0186dbd>] ? check_object+0xad/0x1e0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130
 [<c03c1578>] ? kfree_skbmem+0x98/0xc0
 [<c03fcf03>] tcp_write_xmit+0x63/0x290
 [<c03c1578>] ? kfree_skbmem+0x98/0xc0
 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80
 [<c03fa23e>] tcp_rcv_established+0x40e/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f3e>] ? __do_softirq+0x6e/0x120
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0132128>] ? printk+0x18/0x20
 [<c01317da>] ? print_oops_end_marker+0x2a/0x30
 [<c0131849>] ? warn_on_slowpath+0x59/0x80
 [<c025f9e4>] ? vsnprintf+0x2d4/0x4f0
 [<c0132128>] ? printk+0x18/0x20
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c0186bf1>] ? check_pad_bytes+0x61/0x80
 [<c03f7509>] tcp_clean_rtx_queue+0x2d9/0x5b0
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c03f7dd2>] tcp_ack+0x122/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0132438>] ? vprintk+0x308/0x320
 [<c025f9e4>] ? vsnprintf+0x2d4/0x4f0
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03f7509>] ? tcp_clean_rtx_queue+0x2d9/0x5b0
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c03f6a93>] tcp_fastretrans_alert+0xf3/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0250030>] ? generic_make_request+0x1c0/0x2e0
 [<c0132128>] ? printk+0x18/0x20
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c03f5540>] tcp_add_reno_sack+0x30/0x50
 [<c03f6d72>] tcp_fastretrans_alert+0x3d2/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0)
TCP wq(s)      <
TCP wq(h) +++h+<
l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c03f6052>] tcp_mark_head_lost+0x122/0x150
 [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190
 [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c01317da>] ? print_oops_end_marker+0x2a/0x30
 [<c0131849>] ? warn_on_slowpath+0x59/0x80
 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0132438>] ? vprintk+0x308/0x320
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c03f5ffc>] tcp_mark_head_lost+0xcc/0x150
 [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190
 [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
TCP wq(s) LLLLl<
TCP wq(h) +++h+<
l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_output.c:1977 tcp_xmit_retransmit_queue+0x388/0x420()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0132438>] ? vprintk+0x308/0x320
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c03f5ffc>] ? tcp_mark_head_lost+0xcc/0x150
 [<c03fdf38>] tcp_xmit_retransmit_queue+0x388/0x420
 [<c03f6c27>] tcp_fastretrans_alert+0x287/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c01317da>] ? print_oops_end_marker+0x2a/0x30
 [<c0131849>] ? warn_on_slowpath+0x59/0x80
 [<c0132438>] ? vprintk+0x308/0x320
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03f5ffc>] ? tcp_mark_head_lost+0xcc/0x150
 [<c03fdea4>] tcp_xmit_retransmit_queue+0x2f4/0x420
 [<c03f6c27>] tcp_fastretrans_alert+0x287/0x700
 [<c03f7e63>] tcp_ack+0x1b3/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
TCP wq(s) LLLLl<
TCP wq(h) +++h+<
l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_output.c:1467 tcp_write_xmit+0x288/0x290()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c0186bf1>] ? check_pad_bytes+0x61/0x80
 [<c0186dbd>] ? check_object+0xad/0x1e0
 [<c0155539>] ? mark_held_locks+0x39/0x80
 [<c01880ac>] ? __slab_free+0xac/0x150
 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130
 [<c03c1578>] ? kfree_skbmem+0x98/0xc0
 [<c03fd128>] tcp_write_xmit+0x288/0x290
 [<c03c1578>] ? kfree_skbmem+0x98/0xc0
 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80
 [<c03fa23e>] tcp_rcv_established+0x40e/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c01317da>] ? print_oops_end_marker+0x2a/0x30
 [<c0131849>] ? warn_on_slowpath+0x59/0x80
 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0
 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
 [<c0186bf1>] ? check_pad_bytes+0x61/0x80
 [<c0186dbd>] ? check_object+0xad/0x1e0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130
 [<c03c1578>] ? kfree_skbmem+0x98/0xc0
 [<c03fcf03>] tcp_write_xmit+0x63/0x290
 [<c03c1578>] ? kfree_skbmem+0x98/0xc0
 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80
 [<c03fa23e>] tcp_rcv_established+0x40e/0x710
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c04021fb>] tcp_v4_rcv+0x5db/0x660
 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03e6147>] ip_local_deliver+0xb7/0xc0
 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
 [<c03c01d8>] ? sock_def_readable+0x48/0xa0
 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
 [<c03e669f>] ip_rcv+0x18f/0x290
 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
 [<c03c8da6>] netif_receive_skb+0x2b6/0x330
 [<c03c8c17>] ? netif_receive_skb+0x127/0x330
 [<c03c8ea3>] ? process_backlog+0x83/0x100
 [<c03c8eae>] process_backlog+0x8e/0x100
 [<c03c90bc>] net_rx_action+0x13c/0x230
 [<c03c8fd9>] ? net_rx_action+0x59/0x230
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c01078b1>] do_IRQ+0x41/0x80
 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
 [<c01059ca>] common_interrupt+0x2e/0x34
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
TCP wq(s) LLLLl<
TCP wq(h) +++h+<
l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:1767 tcp_enter_frto+0x28c/0x2a0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0156244>] ? __lock_acquire+0x1e4/0x730
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c03f58dc>] tcp_enter_frto+0x28c/0x2a0
 [<c03ff612>] tcp_retransmit_timer+0x102/0x400
 [<c043675b>] ? _spin_lock+0x4b/0x80
 [<c03ff924>] ? tcp_write_timer+0x14/0x110
 [<c03ffa03>] tcp_write_timer+0xf3/0x110
 [<c013b95b>] run_timer_softirq+0xbb/0x190
 [<c03ff910>] ? tcp_write_timer+0x0/0x110
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c011a91d>] smp_apic_timer_interrupt+0x2d/0x40
 [<c0105aab>] apic_timer_interrupt+0x33/0x38
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c01317da>] ? print_oops_end_marker+0x2a/0x30
 [<c0131849>] ? warn_on_slowpath+0x59/0x80
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c03f5748>] tcp_enter_frto+0xf8/0x2a0
 [<c03ff612>] tcp_retransmit_timer+0x102/0x400
 [<c043675b>] ? _spin_lock+0x4b/0x80
 [<c03ff924>] ? tcp_write_timer+0x14/0x110
 [<c03ffa03>] tcp_write_timer+0xf3/0x110
 [<c013b95b>] run_timer_softirq+0xbb/0x190
 [<c03ff910>] ? tcp_write_timer+0x0/0x110
 [<c0136f63>] __do_softirq+0x93/0x120
 [<c013706a>] do_softirq+0x7a/0x80
 [<c0137135>] irq_exit+0x65/0x90
 [<c011a91d>] smp_apic_timer_interrupt+0x2d/0x40
 [<c0105aab>] apic_timer_interrupt+0x33/0x38
 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
 [<c01033a0>] ? mwait_idle+0x0/0x20
 [<c01033b2>] mwait_idle+0x12/0x20
 [<c0103141>] cpu_idle+0x61/0x110
 [<c04339fd>] rest_init+0x5d/0x60
 [<c05a47fa>] start_kernel+0x1fa/0x260
 [<c05a4190>] ? unknown_bootoption+0x0/0x130
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
TCP wq(s) LLLLl<
TCP wq(h) +++h+<
l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_output.c:1467 tcp_write_xmit+0x288/0x290()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 2987, comm: ssmtp Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0156244>] ? __lock_acquire+0x1e4/0x730
 [<c0155567>] ? mark_held_locks+0x67/0x80
 [<c03fd128>] tcp_write_xmit+0x288/0x290
 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80
 [<c03f1130>] tcp_sendmsg+0x760/0xb40
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c03bb85c>] sock_sendmsg+0xdc/0x110
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0147420>] ? autoremove_wake_function+0x0/0x50
 [<c0156244>] ? __lock_acquire+0x1e4/0x730
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c03bb5ff>] ? sockfd_lookup_light+0x2f/0x70
 [<c03bcfe1>] sys_sendto+0xd1/0x120
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c015323d>] ? put_lock_stats+0xd/0x30
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c01413f1>] ? sys_times+0x81/0xe0
 [<c03bd062>] sys_send+0x32/0x40
 [<c03bd8e6>] sys_socketcall+0x136/0x230
 [<c010501b>] ? restore_nocheck+0x12/0x15
 [<c0104fba>] syscall_call+0x7/0xb
 =======================
---[ end trace 14b601818e6903ac ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 2987, comm: ssmtp Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c01317da>] ? print_oops_end_marker+0x2a/0x30
 [<c0131849>] ? warn_on_slowpath+0x59/0x80
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0156244>] ? __lock_acquire+0x1e4/0x730
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c03fcf03>] tcp_write_xmit+0x63/0x290
 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80
 [<c03f1130>] tcp_sendmsg+0x760/0xb40
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c03bb85c>] sock_sendmsg+0xdc/0x110
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0147420>] ? autoremove_wake_function+0x0/0x50
 [<c0156244>] ? __lock_acquire+0x1e4/0x730
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c03bb5ff>] ? sockfd_lookup_light+0x2f/0x70
 [<c03bcfe1>] sys_sendto+0xd1/0x120
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c015323d>] ? put_lock_stats+0xd/0x30
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c01413f1>] ? sys_times+0x81/0xe0
 [<c03bd062>] sys_send+0x32/0x40
 [<c03bd8e6>] sys_socketcall+0x136/0x230
 [<c010501b>] ? restore_nocheck+0x12/0x15
 [<c0104fba>] syscall_call+0x7/0xb
 =======================
---[ end trace 14b601818e6903ac ]---
P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
TCP wq(s) LLLLl<
TCP wq(h) +++h+<
l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_input.c:2855 tcp_clean_rtx_queue+0x552/0x5b0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 2987, comm: ssmtp Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c0186dbd>] ? check_object+0xad/0x1e0
 [<c0155539>] ? mark_held_locks+0x39/0x80
 [<c01880ac>] ? __slab_free+0xac/0x150
 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130
 [<c03c1534>] ? kfree_skbmem+0x54/0xc0
 [<c01881e7>] ? kmem_cache_free+0x97/0xe0
 [<c03c1534>] ? kfree_skbmem+0x54/0xc0
 [<c03c1534>] ? kfree_skbmem+0x54/0xc0
 [<c03c1534>] ? kfree_skbmem+0x54/0xc0
 [<c03f7782>] tcp_clean_rtx_queue+0x552/0x5b0
 [<c03f7dd2>] tcp_ack+0x122/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0147420>] ? autoremove_wake_function+0x0/0x50
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c03f17e3>] tcp_prequeue_process+0x73/0xa0
 [<c03f1dae>] tcp_recvmsg+0x42e/0x830
 [<c03c0873>] sock_common_recvmsg+0x43/0x60
 [<c03bbacf>] sock_recvmsg+0xef/0x120
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0147420>] ? autoremove_wake_function+0x0/0x50
 [<c03bb5ff>] ? sockfd_lookup_light+0x2f/0x70
 [<c03bd0f7>] sys_recvfrom+0x87/0xf0
 [<c015323d>] ? put_lock_stats+0xd/0x30
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c015323d>] ? put_lock_stats+0xd/0x30
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c013601f>] ? do_setitimer+0x1ef/0x2e0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03bd192>] sys_recv+0x32/0x40
 [<c03bd905>] sys_socketcall+0x155/0x230
 [<c010501b>] ? restore_nocheck+0x12/0x15
 [<c0104fba>] syscall_call+0x7/0xb
 =======================
---[ end trace 14b601818e6903ac ]---
------------[ cut here ]------------
WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
Pid: 2987, comm: ssmtp Not tainted 2.6.24-rc8-mm1 #8
 [<c0132100>] ? have_callable_console+0x20/0x30
 [<c0131844>] warn_on_slowpath+0x54/0x80
 [<c01317da>] ? print_oops_end_marker+0x2a/0x30
 [<c0131849>] ? warn_on_slowpath+0x59/0x80
 [<c0186dbd>] ? check_object+0xad/0x1e0
 [<c0155539>] ? mark_held_locks+0x39/0x80
 [<c01880ac>] ? __slab_free+0xac/0x150
 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130
 [<c03c1534>] ? kfree_skbmem+0x54/0xc0
 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
 [<c01881e7>] ? kmem_cache_free+0x97/0xe0
 [<c03c1534>] ? kfree_skbmem+0x54/0xc0
 [<c03c1534>] ? kfree_skbmem+0x54/0xc0
 [<c03f7509>] tcp_clean_rtx_queue+0x2d9/0x5b0
 [<c03f7dd2>] tcp_ack+0x122/0x3a0
 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
 [<c0147420>] ? autoremove_wake_function+0x0/0x50
 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
 [<c03f17e3>] tcp_prequeue_process+0x73/0xa0
 [<c03f1dae>] tcp_recvmsg+0x42e/0x830
 [<c03c0873>] sock_common_recvmsg+0x43/0x60
 [<c03bbacf>] sock_recvmsg+0xef/0x120
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0154982>] ? validate_chain+0x1d2/0x320
 [<c0147420>] ? autoremove_wake_function+0x0/0x50
 [<c03bb5ff>] ? sockfd_lookup_light+0x2f/0x70
 [<c03bd0f7>] sys_recvfrom+0x87/0xf0
 [<c015323d>] ? put_lock_stats+0xd/0x30
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c010af95>] ? native_sched_clock+0x85/0xe0
 [<c015323d>] ? put_lock_stats+0xd/0x30
 [<c01532c0>] ? lock_release_holdtime+0x60/0x80
 [<c013601f>] ? do_setitimer+0x1ef/0x2e0
 [<c0156b97>] ? __lock_release+0x47/0x70
 [<c03bd192>] sys_recv+0x32/0x40
 [<c03bd905>] sys_socketcall+0x155/0x230
 [<c010501b>] ? restore_nocheck+0x12/0x15
 [<c0104fba>] syscall_call+0x7/0xb
 =======================
---[ end trace 14b601818e6903ac ]---
P: 4 L: 4 vs 4 S: 0 vs 3 w: 2044792269-2044796616 (0)
TCP wq(s) LLLl<
TCP wq(h) ++h+<
l4 s3 f0 p4 seq: su2044792269 hs2044795029 sn2044796616

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-24  2:42                     ` Dave Young
@ 2008-01-24  9:54                       ` Ilpo Järvinen
  2008-01-24 10:24                         ` Ilpo Järvinen
                                           ` (3 more replies)
  0 siblings, 4 replies; 18+ messages in thread
From: Ilpo Järvinen @ 2008-01-24  9:54 UTC (permalink / raw)
  To: Dave Young, Kamalesh Babulal, Krishna Kumar2,
	Denys Fedoryshchenko, David Miller
  Cc: LKML, Netdev, Andrew Morton

[-- Attachment #1: Type: TEXT/PLAIN, Size: 8416 bytes --]

On Thu, 24 Jan 2008, Dave Young wrote:

Hi Dave (& others),

> Thanks.

Thanks a lot, I was first to ignore all these because they occurred 
with newreno, but looked again... :-/

> New warning trigged with your debug patch:

This was probably with the earlier one I sent to you because there's still 
this case remaining which itself is valid:

> P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)

...snip... this is still ok state (S+L <= P):

> P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0)
> TCP wq(s)      <
> TCP wq(h) +++h+<
> l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
> ------------[ cut here ]------------
> WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150()
> Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
> snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
> snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
> evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
> i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
> Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
>  [<c0132100>] ? have_callable_console+0x20/0x30
>  [<c0131844>] warn_on_slowpath+0x54/0x80
>  [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
>  [<c0132438>] ? vprintk+0x308/0x320
>  [<c0132438>] ? vprintk+0x308/0x320
>  [<c0132438>] ? vprintk+0x308/0x320
>  [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
>  [<c03f6052>] tcp_mark_head_lost+0x122/0x150
>  [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190
>  [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700
>  [<c03f7e63>] tcp_ack+0x1b3/0x3a0
>  [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
>  [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
>  [<c04021fb>] tcp_v4_rcv+0x5db/0x660
>  [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
>  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
>  [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
>  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
>  [<c0156b97>] ? __lock_release+0x47/0x70
>  [<c03e6147>] ip_local_deliver+0xb7/0xc0
>  [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
>  [<c03c01d8>] ? sock_def_readable+0x48/0xa0
>  [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
>  [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
>  [<c03e669f>] ip_rcv+0x18f/0x290
>  [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
>  [<c03c8da6>] netif_receive_skb+0x2b6/0x330
>  [<c03c8c17>] ? netif_receive_skb+0x127/0x330
>  [<c03c8ea3>] ? process_backlog+0x83/0x100
>  [<c03c8eae>] process_backlog+0x8e/0x100
>  [<c03c90bc>] net_rx_action+0x13c/0x230
>  [<c03c8fd9>] ? net_rx_action+0x59/0x230
>  [<c0136f63>] __do_softirq+0x93/0x120
>  [<c013706a>] do_softirq+0x7a/0x80
>  [<c0137135>] irq_exit+0x65/0x90
>  [<c01078b1>] do_IRQ+0x41/0x80
>  [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
>  [<c01059ca>] common_interrupt+0x2e/0x34
>  [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
>  [<c01033a0>] ? mwait_idle+0x0/0x20
>  [<c01033b2>] mwait_idle+0x12/0x20
>  [<c0103141>] cpu_idle+0x61/0x110
>  [<c04339fd>] rest_init+0x5d/0x60
>  [<c05a47fa>] start_kernel+0x1fa/0x260
>  [<c05a4190>] ? unknown_bootoption+0x0/0x130
>  =======================
> ---[ end trace 14b601818e6903ac ]---
> ------------[ cut here ]------------
> WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
> Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
> snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
> snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
> evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
> i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
> Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
>  [<c0132100>] ? have_callable_console+0x20/0x30
>  [<c0131844>] warn_on_slowpath+0x54/0x80
>  [<c01317da>] ? print_oops_end_marker+0x2a/0x30
>  [<c0131849>] ? warn_on_slowpath+0x59/0x80
>  [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
>  [<c0132438>] ? vprintk+0x308/0x320
>  [<c0132438>] ? vprintk+0x308/0x320
>  [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
>  [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
>  [<c03f5ffc>] tcp_mark_head_lost+0xcc/0x150
>  [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190
>  [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700
>  [<c03f7e63>] tcp_ack+0x1b3/0x3a0
>  [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
>  [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
>  [<c04021fb>] tcp_v4_rcv+0x5db/0x660
>  [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
>  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
>  [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
>  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
>  [<c0156b97>] ? __lock_release+0x47/0x70
>  [<c03e6147>] ip_local_deliver+0xb7/0xc0
>  [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
>  [<c03c01d8>] ? sock_def_readable+0x48/0xa0
>  [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
>  [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
>  [<c03e669f>] ip_rcv+0x18f/0x290
>  [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
>  [<c03c8da6>] netif_receive_skb+0x2b6/0x330
>  [<c03c8c17>] ? netif_receive_skb+0x127/0x330
>  [<c03c8ea3>] ? process_backlog+0x83/0x100
>  [<c03c8eae>] process_backlog+0x8e/0x100
>  [<c03c90bc>] net_rx_action+0x13c/0x230
>  [<c03c8fd9>] ? net_rx_action+0x59/0x230
>  [<c0136f63>] __do_softirq+0x93/0x120
>  [<c013706a>] do_softirq+0x7a/0x80
>  [<c0137135>] irq_exit+0x65/0x90
>  [<c01078b1>] do_IRQ+0x41/0x80
>  [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
>  [<c01059ca>] common_interrupt+0x2e/0x34
>  [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
>  [<c01033a0>] ? mwait_idle+0x0/0x20
>  [<c01033b2>] mwait_idle+0x12/0x20
>  [<c0103141>] cpu_idle+0x61/0x110
>  [<c04339fd>] rest_init+0x5d/0x60
>  [<c05a47fa>] start_kernel+0x1fa/0x260
>  [<c05a4190>] ? unknown_bootoption+0x0/0x130
>  =======================
> ---[ end trace 14b601818e6903ac ]---

...But this no longer is, and even more, L: 5 is not valid state at this 
point all (should only happen if we went to RTO but it would reset S to 
zero with newreno):

> P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
> TCP wq(s) LLLLl<
> TCP wq(h) +++h+<
> l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616

Surprisingly, it was the first time the WARN_ON for left_out returned 
correct location. This also explains why the patch I sent to Krishna 
didn't print anything (it didn't end up into printing because I forgot 
to add L+S>P check into to the state checking if).

...so please, could you (others than Denys) try this patch, it should 
solve the issue. And Denys, could you confirm (and if necessary double 
check) that the kernel you saw this similar problem with is the pure 
Linus' mainline, i.e., without any net-2.6.25 or mm bits please, if so, 
that problem persists. And anyway, there were some fackets_out related 
problems reported as well and this doesn't help for that but I think I've 
lost track of who was seeing it due to large number of reports :-), could 
somebody refresh my memory because I currently don't have time to dig it 
up from archives (at least on this week).


-- 
 i.

--
[PATCH] [TCP]: NewReno must count every skb while marking losses

NewReno should add cnt per skb (as with FACK) instead of depending
on SACKED_ACKED bits which won't be set with it at all.
Effectively, NewReno should always exists after the first
iteration anyway (or immediately if there's already head in
lost_out.

This was fixed earlier in net-2.6.25 but got reverted among other
stuff and I didn't notice that this is still necessary (actually
wasn't even considering this case while trying to figure out the
reports because I lived with different kind of code than it in
reality was).

This should solve the WARN_ONs in TCP code that as a result of
this triggered multiple times in every place we check for this
invariant.

Special thanks to Dave Young <hidave.darkstar@gmail.com> and
Krishna Kumar2 <krkumar2@in.ibm.com> for trying with my debug
patches.

Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi>
---
 net/ipv4/tcp_input.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 295490e..aa409a5 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -2156,7 +2156,7 @@ static void tcp_mark_head_lost(struct sock *sk, int packets, int fast_rexmit)
 		tp->lost_skb_hint = skb;
 		tp->lost_cnt_hint = cnt;
 
-		if (tcp_is_fack(tp) ||
+		if (tcp_is_fack(tp) || tcp_is_reno(tp) ||
 		    (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED))
 			cnt += tcp_skb_pcount(skb);
 
-- 
1.5.2.2

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-24  9:54                       ` Ilpo Järvinen
@ 2008-01-24 10:24                         ` Ilpo Järvinen
  2008-01-24 10:46                         ` Krishna Kumar2
                                           ` (2 subsequent siblings)
  3 siblings, 0 replies; 18+ messages in thread
From: Ilpo Järvinen @ 2008-01-24 10:24 UTC (permalink / raw)
  To: Netdev
  Cc: Dave Young, Kamalesh Babulal, Krishna Kumar2,
	Denys Fedoryshchenko, David Miller, LKML, Andrew Morton

[-- Attachment #1: Type: TEXT/PLAIN, Size: 10779 bytes --]

On Thu, 24 Jan 2008, Ilpo Järvinen wrote:

> And anyway, there were some fackets_out related 
> problems reported as well and this doesn't help for that but I think I've 
> lost track of who was seeing it due to large number of reports :-), could 
> somebody refresh my memory because I currently don't have time to dig it 
> up from archives (at least on this week).

Here's the updated debug patch for net-2.6.25/mm for tracking 
fackets_out inconsistencies (it won't work for trees which don't
include net-2.6.25, mm does of course :-)).

I hope I got it into good shape this time to avoid spurious stacktraces 
but still maintaining 100% accuracy, but it's not a simple oneliner so I 
might have missed something... :-)

I'd suggest that people trying with this first apply the newreno fix of 
the previous mail to avoid already-fixed case from triggering.

-- 
 i.

--
[PATCH] [TCP]: debug S+L (for net-2.5.26 / mm, incompatible with mainline)

---
 include/net/tcp.h     |    5 ++-
 net/ipv4/tcp_input.c  |   18 +++++++-
 net/ipv4/tcp_ipv4.c   |  127 +++++++++++++++++++++++++++++++++++++++++++++++++
 net/ipv4/tcp_output.c |   23 +++++++--
 4 files changed, 165 insertions(+), 8 deletions(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 7de4ea3..552aa71 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -272,6 +272,9 @@ DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics);
 #define TCP_ADD_STATS_BH(field, val)	SNMP_ADD_STATS_BH(tcp_statistics, field, val)
 #define TCP_ADD_STATS_USER(field, val)	SNMP_ADD_STATS_USER(tcp_statistics, field, val)
 
+extern void tcp_print_queue(struct sock *sk);
+extern void			tcp_verify_wq(struct sock *sk);
+
 extern void			tcp_v4_err(struct sk_buff *skb, u32);
 
 extern void			tcp_shutdown (struct sock *sk, int how);
@@ -768,7 +771,7 @@ static inline __u32 tcp_current_ssthresh(const struct sock *sk)
 }
 
 /* Use define here intentionally to get WARN_ON location shown at the caller */
-#define tcp_verify_left_out(tp)	WARN_ON(tcp_left_out(tp) > tp->packets_out)
+#define tcp_verify_left_out(tp)		tcp_verify_wq((struct sock *)tp)
 
 extern void tcp_enter_cwr(struct sock *sk, const int set_ssthresh);
 extern __u32 tcp_init_cwnd(struct tcp_sock *tp, struct dst_entry *dst);
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 19c449f..c897c93 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -1426,8 +1426,10 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb,
 	int first_sack_index;
 
 	if (!tp->sacked_out) {
-		if (WARN_ON(tp->fackets_out))
+		if (WARN_ON(tp->fackets_out)) {
+			tcp_verify_left_out(tp);
 			tp->fackets_out = 0;
+		}
 		tcp_highest_sack_reset(sk);
 	}
 
@@ -2136,6 +2138,8 @@ static void tcp_mark_head_lost(struct sock *sk, int packets, int fast_rexmit)
 	struct sk_buff *skb;
 	int cnt;
 
+	tcp_verify_left_out(tp);
+
 	BUG_TRAP(packets <= tp->packets_out);
 	if (tp->lost_skb_hint) {
 		skb = tp->lost_skb_hint;
@@ -2501,6 +2505,8 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag)
 				    (tcp_fackets_out(tp) > tp->reordering));
 	int fast_rexmit = 0;
 
+	tcp_verify_left_out(tp);
+
 	if (WARN_ON(!tp->packets_out && tp->sacked_out))
 		tp->sacked_out = 0;
 	if (WARN_ON(!tp->sacked_out && tp->fackets_out))
@@ -2645,6 +2651,10 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag)
 	if (do_lost || (tcp_is_fack(tp) && tcp_head_timedout(sk)))
 		tcp_update_scoreboard(sk, fast_rexmit);
 	tcp_cwnd_down(sk, flag);
+
+	WARN_ON(tcp_write_queue_head(sk) == NULL);
+	WARN_ON(!tp->packets_out);
+
 	tcp_xmit_retransmit_queue(sk);
 }
 
@@ -2848,6 +2858,8 @@ static int tcp_clean_rtx_queue(struct sock *sk, int prior_fackets)
 		tcp_clear_all_retrans_hints(tp);
 	}
 
+	tcp_verify_left_out(tp);
+
 	if (skb && (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED))
 		flag |= FLAG_SACK_RENEGING;
 
@@ -3175,6 +3187,8 @@ static int tcp_ack(struct sock *sk, struct sk_buff *skb, int flag)
 	prior_fackets = tp->fackets_out;
 	prior_in_flight = tcp_packets_in_flight(tp);
 
+	tcp_verify_left_out(tp);
+
 	if (!(flag & FLAG_SLOWPATH) && after(ack, prior_snd_una)) {
 		/* Window is constant, pure forward advance.
 		 * No more checks are required.
@@ -3237,6 +3251,8 @@ static int tcp_ack(struct sock *sk, struct sk_buff *skb, int flag)
 	if ((flag & FLAG_FORWARD_PROGRESS) || !(flag & FLAG_NOT_DUP))
 		dst_confirm(sk->sk_dst_cache);
 
+	tcp_verify_left_out(tp);
+
 	return 1;
 
 no_queue:
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 9aea88b..e6e3ad5 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -108,6 +108,133 @@ struct inet_hashinfo __cacheline_aligned tcp_hashinfo = {
 	.lhash_wait  = __WAIT_QUEUE_HEAD_INITIALIZER(tcp_hashinfo.lhash_wait),
 };
 
+void tcp_print_queue(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct sk_buff *skb;
+	char s[50+1];
+	char h[50+1];
+	int idx = 0;
+	int i;
+
+	i = 0;
+	tcp_for_write_queue(skb, sk) {
+		if (skb == tcp_send_head(sk))
+			printk(KERN_ERR "head %u %p\n", i, skb);		
+		else
+			printk(KERN_ERR "skb %u %p\n", i, skb);
+		i++;
+	}
+
+	tcp_for_write_queue(skb, sk) {
+		if (skb == tcp_send_head(sk))
+			break;
+
+		for (i = 0; i < tcp_skb_pcount(skb); i++) {
+			if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) {
+				s[idx] = 'S';
+				if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+					s[idx] = 'B';
+
+			} else if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) {
+				s[idx] = 'L';
+			} else {
+				s[idx] = ' ';
+			}
+			if (s[idx] != ' ' && skb->len < tp->mss_cache)
+				s[idx] += 'a' - 'A';
+
+			if (i == 0) {
+				if (skb == tcp_highest_sack(sk))
+					h[idx] = 'h';
+				else
+					h[idx] = '+';
+			} else {
+				h[idx] = '-';
+			}
+
+			if (++idx >= 50) {
+				s[idx] = 0;
+				h[idx] = 0;
+				printk(KERN_ERR "TCP wq(s) %s\n", s);
+				printk(KERN_ERR "TCP wq(h) %s\n", h);
+				idx = 0;
+			}
+		}
+	}
+	if (idx) {
+		s[idx] = '<';
+		s[idx+1] = 0;
+		h[idx] = '<';
+		h[idx+1] = 0;
+		printk(KERN_ERR "TCP wq(s) %s\n", s);
+		printk(KERN_ERR "TCP wq(h) %s\n", h);
+	}
+	printk(KERN_ERR "l%u s%u f%u p%u seq: su%u hs%u sn%u\n",
+		tp->lost_out, tp->sacked_out, tp->fackets_out, tp->packets_out,
+		tp->snd_una, tcp_highest_sack_seq(tp), tp->snd_nxt);
+}
+
+void tcp_verify_wq(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+	u32 lost = 0;
+	u32 sacked = 0;
+	u32 packets = 0;
+	u32 fackets = 0;
+	int hs_valid = 0;
+	struct sk_buff *skb;
+
+	tcp_for_write_queue(skb, sk) {
+		if (skb == tcp_send_head(sk))
+			break;
+
+		if ((fackets == packets) && (skb == tp->highest_sack))
+			hs_valid = 1;
+
+		packets += tcp_skb_pcount(skb);
+
+		if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) {
+			sacked += tcp_skb_pcount(skb);
+			if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+				printk(KERN_ERR "Sacked bitmap S+L: %u %u-%u/%u\n",
+					TCP_SKB_CB(skb)->sacked,
+					TCP_SKB_CB(skb)->end_seq - tp->snd_una,
+					TCP_SKB_CB(skb)->seq - tp->snd_una,
+					tp->snd_una);
+			fackets = packets;
+			hs_valid = 0;
+		}
+		if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST)
+			lost += tcp_skb_pcount(skb);
+	}
+
+	if ((fackets == packets) && (tp->highest_sack == tcp_send_head(sk)))
+		hs_valid = 1;
+
+	if ((lost != tp->lost_out) ||
+	    (tcp_is_sack(tp) && (sacked != tp->sacked_out)) ||
+	    ((sacked || (tcp_is_sack(tp) && tp->sacked_out)) && !hs_valid) ||
+	    (packets != tp->packets_out) ||
+	    (fackets != tp->fackets_out) ||
+	    tcp_left_out(tp) > tp->packets_out) {
+		printk(KERN_ERR "P: %u L: %u vs %u S: %u vs %u F: %u vs %u w: %u-%u (%u)\n",
+			tp->packets_out,
+			lost, tp->lost_out,
+			sacked, tp->sacked_out,
+			fackets, tp->fackets_out,
+			tp->snd_una, tp->snd_nxt,
+			tp->rx_opt.sack_ok);
+		tcp_print_queue(sk);
+	}
+
+	WARN_ON(lost != tp->lost_out);
+	WARN_ON(tcp_is_sack(tp) && (sacked != tp->sacked_out));
+	WARN_ON(packets != tp->packets_out);
+	WARN_ON(fackets != tp->fackets_out);
+	WARN_ON(tcp_left_out(tp) > tp->packets_out);
+}
+
 static int tcp_v4_get_port(struct sock *sk, unsigned short snum)
 {
 	return inet_csk_get_port(&tcp_hashinfo, sk, snum,
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 89f0188..8fb6628 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -779,10 +779,9 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len,
 			tp->lost_out -= diff;
 
 		/* Adjust Reno SACK estimate. */
-		if (tcp_is_reno(tp) && diff > 0) {
+		if (tcp_is_reno(tp) && diff > 0)
 			tcp_dec_pcount_approx_int(&tp->sacked_out, diff);
-			tcp_verify_left_out(tp);
-		}
+
 		tcp_adjust_fackets_out(sk, skb, diff);
 	}
 
@@ -790,6 +789,8 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len,
 	skb_header_release(buff);
 	tcp_insert_write_queue_after(skb, buff, sk);
 
+	tcp_verify_left_out(tp);
+
 	return 0;
 }
 
@@ -1463,6 +1464,7 @@ static int tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle)
 	} else if (result > 0) {
 		sent_pkts = 1;
 	}
+	tcp_verify_left_out(tp);
 
 	while ((skb = tcp_send_head(sk))) {
 		unsigned int limit;
@@ -1764,6 +1766,7 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *skb,
 	tcp_clear_retrans_hints_partial(tp);
 
 	sk_wmem_free_skb(sk, next_skb);
+	tcp_verify_left_out(tp);
 }
 
 /* Do a simple retransmit without using the backoff mechanisms in
@@ -1795,13 +1798,13 @@ void tcp_simple_retransmit(struct sock *sk)
 		}
 	}
 
+	tcp_verify_left_out(tp);
+
 	tcp_clear_all_retrans_hints(tp);
 
 	if (!lost)
 		return;
 
-	tcp_verify_left_out(tp);
-
 	/* Don't muck with the congestion window here.
 	 * Reason is that we do not increase amount of _data_
 	 * in network, but units changed and effective
@@ -1888,6 +1891,8 @@ int tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb)
 			tcp_init_nondata_skb(skb, TCP_SKB_CB(skb)->end_seq - 1,
 					     TCP_SKB_CB(skb)->flags);
 			skb->ip_summed = CHECKSUM_NONE;
+
+			tcp_verify_left_out(tp);
 		}
 	}
 
@@ -1970,8 +1975,10 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 			 * packet to be MSS sized and all the
 			 * packet counting works out.
 			 */
-			if (tcp_packets_in_flight(tp) >= tp->snd_cwnd)
+			if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) {
+				tcp_verify_left_out(tp);
 				return;
+			}
 
 			if (sacked & TCPCB_LOST) {
 				if (!(sacked & (TCPCB_SACKED_ACKED|TCPCB_SACKED_RETRANS))) {
@@ -1997,6 +2004,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 		}
 	}
 
+	tcp_verify_left_out(tp);
+
 	/* OK, demanded retransmission is finished. */
 
 	/* Forward retransmissions are possible only during Recovery. */
@@ -2054,6 +2063,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
 
 		NET_INC_STATS_BH(LINUX_MIB_TCPFORWARDRETRANS);
 	}
+
+	tcp_verify_left_out(tp);
 }
 
 /* Send a fin.  The caller locks the socket for us.  This cannot be
-- 
1.5.2.2

^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-24  9:54                       ` Ilpo Järvinen
  2008-01-24 10:24                         ` Ilpo Järvinen
@ 2008-01-24 10:46                         ` Krishna Kumar2
  2008-01-24 13:11                         ` Kamalesh Babulal
  2008-01-25  1:30                         ` Dave Young
  3 siblings, 0 replies; 18+ messages in thread
From: Krishna Kumar2 @ 2008-01-24 10:46 UTC (permalink / raw)
  To: Ilpo Järvinen
  Cc: Andrew Morton, David Miller, Denys Fedoryshchenko, Dave Young,
	Kamalesh Babulal, LKML, Netdev, netdev-owner

Hi Ilpo,

I have tried parallel iperfs with this patch and don't get any more
warnings.
I will run overnight to be sure.

thanks,

- KK

netdev-owner@vger.kernel.org wrote on 01/24/2008 03:24:18 PM:

> On Thu, 24 Jan 2008, Dave Young wrote:
>
> Hi Dave (& others),
>
> > Thanks.
>
> Thanks a lot, I was first to ignore all these because they occurred
> with newreno, but looked again... :-/
>
> > New warning trigged with your debug patch:
>
> This was probably with the earlier one I sent to you because there's
still
> this case remaining which itself is valid:
>
> > P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)
>
> ...snip... this is still ok state (S+L <= P):
>
> > P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0)
> > TCP wq(s)      <
> > TCP wq(h) +++h+<
> > l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
> > ------------[ cut here ]------------
> > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150()
> > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
> > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
> > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
> > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
> > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
> > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
> >  [<c0132100>] ? have_callable_console+0x20/0x30
> >  [<c0131844>] warn_on_slowpath+0x54/0x80
> >  [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
> >  [<c03f6052>] tcp_mark_head_lost+0x122/0x150
> >  [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190
> >  [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700
> >  [<c03f7e63>] tcp_ack+0x1b3/0x3a0
> >  [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
> >  [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
> >  [<c04021fb>] tcp_v4_rcv+0x5db/0x660
> >  [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
> >  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
> >  [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
> >  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
> >  [<c0156b97>] ? __lock_release+0x47/0x70
> >  [<c03e6147>] ip_local_deliver+0xb7/0xc0
> >  [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
> >  [<c03c01d8>] ? sock_def_readable+0x48/0xa0
> >  [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
> >  [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
> >  [<c03e669f>] ip_rcv+0x18f/0x290
> >  [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
> >  [<c03c8da6>] netif_receive_skb+0x2b6/0x330
> >  [<c03c8c17>] ? netif_receive_skb+0x127/0x330
> >  [<c03c8ea3>] ? process_backlog+0x83/0x100
> >  [<c03c8eae>] process_backlog+0x8e/0x100
> >  [<c03c90bc>] net_rx_action+0x13c/0x230
> >  [<c03c8fd9>] ? net_rx_action+0x59/0x230
> >  [<c0136f63>] __do_softirq+0x93/0x120
> >  [<c013706a>] do_softirq+0x7a/0x80
> >  [<c0137135>] irq_exit+0x65/0x90
> >  [<c01078b1>] do_IRQ+0x41/0x80
> >  [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
> >  [<c01059ca>] common_interrupt+0x2e/0x34
> >  [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
> >  [<c01033a0>] ? mwait_idle+0x0/0x20
> >  [<c01033b2>] mwait_idle+0x12/0x20
> >  [<c0103141>] cpu_idle+0x61/0x110
> >  [<c04339fd>] rest_init+0x5d/0x60
> >  [<c05a47fa>] start_kernel+0x1fa/0x260
> >  [<c05a4190>] ? unknown_bootoption+0x0/0x130
> >  =======================
> > ---[ end trace 14b601818e6903ac ]---
> > ------------[ cut here ]------------
> > WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
> > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
> > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
> > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
> > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
> > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
> > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
> >  [<c0132100>] ? have_callable_console+0x20/0x30
> >  [<c0131844>] warn_on_slowpath+0x54/0x80
> >  [<c01317da>] ? print_oops_end_marker+0x2a/0x30
> >  [<c0131849>] ? warn_on_slowpath+0x59/0x80
> >  [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
> >  [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
> >  [<c03f5ffc>] tcp_mark_head_lost+0xcc/0x150
> >  [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190
> >  [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700
> >  [<c03f7e63>] tcp_ack+0x1b3/0x3a0
> >  [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
> >  [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
> >  [<c04021fb>] tcp_v4_rcv+0x5db/0x660
> >  [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
> >  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
> >  [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
> >  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
> >  [<c0156b97>] ? __lock_release+0x47/0x70
> >  [<c03e6147>] ip_local_deliver+0xb7/0xc0
> >  [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
> >  [<c03c01d8>] ? sock_def_readable+0x48/0xa0
> >  [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
> >  [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
> >  [<c03e669f>] ip_rcv+0x18f/0x290
> >  [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
> >  [<c03c8da6>] netif_receive_skb+0x2b6/0x330
> >  [<c03c8c17>] ? netif_receive_skb+0x127/0x330
> >  [<c03c8ea3>] ? process_backlog+0x83/0x100
> >  [<c03c8eae>] process_backlog+0x8e/0x100
> >  [<c03c90bc>] net_rx_action+0x13c/0x230
> >  [<c03c8fd9>] ? net_rx_action+0x59/0x230
> >  [<c0136f63>] __do_softirq+0x93/0x120
> >  [<c013706a>] do_softirq+0x7a/0x80
> >  [<c0137135>] irq_exit+0x65/0x90
> >  [<c01078b1>] do_IRQ+0x41/0x80
> >  [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
> >  [<c01059ca>] common_interrupt+0x2e/0x34
> >  [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
> >  [<c01033a0>] ? mwait_idle+0x0/0x20
> >  [<c01033b2>] mwait_idle+0x12/0x20
> >  [<c0103141>] cpu_idle+0x61/0x110
> >  [<c04339fd>] rest_init+0x5d/0x60
> >  [<c05a47fa>] start_kernel+0x1fa/0x260
> >  [<c05a4190>] ? unknown_bootoption+0x0/0x130
> >  =======================
> > ---[ end trace 14b601818e6903ac ]---
>
> ...But this no longer is, and even more, L: 5 is not valid state at this
> point all (should only happen if we went to RTO but it would reset S to
> zero with newreno):
>
> > P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
> > TCP wq(s) LLLLl<
> > TCP wq(h) +++h+<
> > l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
>
> Surprisingly, it was the first time the WARN_ON for left_out returned
> correct location. This also explains why the patch I sent to Krishna
> didn't print anything (it didn't end up into printing because I forgot
> to add L+S>P check into to the state checking if).
>
> ...so please, could you (others than Denys) try this patch, it should
> solve the issue. And Denys, could you confirm (and if necessary double
> check) that the kernel you saw this similar problem with is the pure
> Linus' mainline, i.e., without any net-2.6.25 or mm bits please, if so,
> that problem persists. And anyway, there were some fackets_out related
> problems reported as well and this doesn't help for that but I think I've

> lost track of who was seeing it due to large number of reports :-), could

> somebody refresh my memory because I currently don't have time to dig it
> up from archives (at least on this week).
>
>
> --
>  i.
>
> --
> [PATCH] [TCP]: NewReno must count every skb while marking losses
>
> NewReno should add cnt per skb (as with FACK) instead of depending
> on SACKED_ACKED bits which won't be set with it at all.
> Effectively, NewReno should always exists after the first
> iteration anyway (or immediately if there's already head in
> lost_out.
>
> This was fixed earlier in net-2.6.25 but got reverted among other
> stuff and I didn't notice that this is still necessary (actually
> wasn't even considering this case while trying to figure out the
> reports because I lived with different kind of code than it in
> reality was).
>
> This should solve the WARN_ONs in TCP code that as a result of
> this triggered multiple times in every place we check for this
> invariant.
>
> Special thanks to Dave Young <hidave.darkstar@gmail.com> and
> Krishna Kumar2 <krkumar2@in.ibm.com> for trying with my debug
> patches.
>
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi>
> ---
>  net/ipv4/tcp_input.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> index 295490e..aa409a5 100644
> --- a/net/ipv4/tcp_input.c
> +++ b/net/ipv4/tcp_input.c
> @@ -2156,7 +2156,7 @@ static void tcp_mark_head_lost(struct sock *sk, int

> packets, int fast_rexmit)
>        tp->lost_skb_hint = skb;
>        tp->lost_cnt_hint = cnt;
>
> -      if (tcp_is_fack(tp) ||
> +      if (tcp_is_fack(tp) || tcp_is_reno(tp) ||
>            (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED))
>           cnt += tcp_skb_pcount(skb);
>
> --
> 1.5.2.2


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-24  9:54                       ` Ilpo Järvinen
  2008-01-24 10:24                         ` Ilpo Järvinen
  2008-01-24 10:46                         ` Krishna Kumar2
@ 2008-01-24 13:11                         ` Kamalesh Babulal
  2008-01-25  1:30                         ` Dave Young
  3 siblings, 0 replies; 18+ messages in thread
From: Kamalesh Babulal @ 2008-01-24 13:11 UTC (permalink / raw)
  To: Ilpo Järvinen
  Cc: Dave Young, Kamalesh Babulal, Krishna Kumar2,
	Denys Fedoryshchenko, David Miller, LKML, Netdev, Andrew Morton

On Thu, Jan 24, 2008 at 11:54:18AM +0200, Ilpo Järvinen wrote:
> On Thu, 24 Jan 2008, Dave Young wrote:
> 
> Hi Dave (& others),
> 
> > Thanks.
> 
> Thanks a lot, I was first to ignore all these because they occurred 
> with newreno, but looked again... :-/
> 
> > New warning trigged with your debug patch:
> 
> This was probably with the earlier one I sent to you because there's still 
> this case remaining which itself is valid:
> 
> > P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)
> 
> ...snip... this is still ok state (S+L <= P):
> 
> > P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0)
> > TCP wq(s)      <
> > TCP wq(h) +++h+<
> > l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
> > ------------[ cut here ]------------
> > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150()
> > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
> > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
> > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
> > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
> > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
> > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
> >  [<c0132100>] ? have_callable_console+0x20/0x30
> >  [<c0131844>] warn_on_slowpath+0x54/0x80
> >  [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
> >  [<c0132438>] ? vprintk+0x308/0x320
.
.
<snip>
.
.

> > ---[ end trace 14b601818e6903ac ]---
> 
> ...But this no longer is, and even more, L: 5 is not valid state at this 
> point all (should only happen if we went to RTO but it would reset S to 
> zero with newreno):
> 
> > P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
> > TCP wq(s) LLLLl<
> > TCP wq(h) +++h+<
> > l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
> 
> Surprisingly, it was the first time the WARN_ON for left_out returned 
> correct location. This also explains why the patch I sent to Krishna 
> didn't print anything (it didn't end up into printing because I forgot 
> to add L+S>P check into to the state checking if).
> 
> ...so please, could you (others than Denys) try this patch, it should 
> solve the issue. And Denys, could you confirm (and if necessary double 
> check) that the kernel you saw this similar problem with is the pure 
> Linus' mainline, i.e., without any net-2.6.25 or mm bits please, if so, 
> that problem persists. And anyway, there were some fackets_out related 
> problems reported as well and this doesn't help for that but I think I've 
> lost track of who was seeing it due to large number of reports :-), could 
> somebody refresh my memory because I currently don't have time to dig it 
> up from archives (at least on this week).
> 
> 
> -- 
>  i.
> 
> --
> [PATCH] [TCP]: NewReno must count every skb while marking losses
> 
> NewReno should add cnt per skb (as with FACK) instead of depending
> on SACKED_ACKED bits which won't be set with it at all.
> Effectively, NewReno should always exists after the first
> iteration anyway (or immediately if there's already head in
> lost_out.
> 
> This was fixed earlier in net-2.6.25 but got reverted among other
> stuff and I didn't notice that this is still necessary (actually
> wasn't even considering this case while trying to figure out the
> reports because I lived with different kind of code than it in
> reality was).
> 
> This should solve the WARN_ONs in TCP code that as a result of
> this triggered multiple times in every place we check for this
> invariant.
> 
> Special thanks to Dave Young <hidave.darkstar@gmail.com> and
> Krishna Kumar2 <krkumar2@in.ibm.com> for trying with my debug
> patches.

Hi,

Thanks, after applying the patch the warning is not seen.


  Tested-by: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> 
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi>
> ---
>  net/ipv4/tcp_input.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> index 295490e..aa409a5 100644
> --- a/net/ipv4/tcp_input.c
> +++ b/net/ipv4/tcp_input.c
> @@ -2156,7 +2156,7 @@ static void tcp_mark_head_lost(struct sock *sk, int packets, int fast_rexmit)
>  		tp->lost_skb_hint = skb;
>  		tp->lost_cnt_hint = cnt;
> 
> -		if (tcp_is_fack(tp) ||
> +		if (tcp_is_fack(tp) || tcp_is_reno(tp) ||
>  		    (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED))
>  			cnt += tcp_skb_pcount(skb);
> 
> -- 
> 1.5.2.2


-- 
Thanks & Regards,
Kamalesh Babulal,
Linux Technology Center,
IBM, ISTL.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings
  2008-01-24  9:54                       ` Ilpo Järvinen
                                           ` (2 preceding siblings ...)
  2008-01-24 13:11                         ` Kamalesh Babulal
@ 2008-01-25  1:30                         ` Dave Young
  3 siblings, 0 replies; 18+ messages in thread
From: Dave Young @ 2008-01-25  1:30 UTC (permalink / raw)
  To: Ilpo Järvinen
  Cc: Kamalesh Babulal, Krishna Kumar2, Denys Fedoryshchenko,
	David Miller, LKML, Netdev, Andrew Morton

On Jan 24, 2008 5:54 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote:
> On Thu, 24 Jan 2008, Dave Young wrote:
>
> Hi Dave (& others),
>
> > Thanks.
>
> Thanks a lot, I was first to ignore all these because they occurred
> with newreno, but looked again... :-/
>
> > New warning trigged with your debug patch:
>
> This was probably with the earlier one I sent to you because there's still
> this case remaining which itself is valid:
>
> > P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0)
>
> ...snip... this is still ok state (S+L <= P):
>
>
> > P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0)
> > TCP wq(s)      <
> > TCP wq(h) +++h+<
> > l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
> > ------------[ cut here ]------------
> > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150()
> > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
> > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
> > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
> > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
> > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
> > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
> >  [<c0132100>] ? have_callable_console+0x20/0x30
> >  [<c0131844>] warn_on_slowpath+0x54/0x80
> >  [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
> >  [<c03f6052>] tcp_mark_head_lost+0x122/0x150
> >  [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190
> >  [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700
> >  [<c03f7e63>] tcp_ack+0x1b3/0x3a0
> >  [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
> >  [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
> >  [<c04021fb>] tcp_v4_rcv+0x5db/0x660
> >  [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
> >  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
> >  [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
> >  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
> >  [<c0156b97>] ? __lock_release+0x47/0x70
> >  [<c03e6147>] ip_local_deliver+0xb7/0xc0
> >  [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
> >  [<c03c01d8>] ? sock_def_readable+0x48/0xa0
> >  [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
> >  [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
> >  [<c03e669f>] ip_rcv+0x18f/0x290
> >  [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
> >  [<c03c8da6>] netif_receive_skb+0x2b6/0x330
> >  [<c03c8c17>] ? netif_receive_skb+0x127/0x330
> >  [<c03c8ea3>] ? process_backlog+0x83/0x100
> >  [<c03c8eae>] process_backlog+0x8e/0x100
> >  [<c03c90bc>] net_rx_action+0x13c/0x230
> >  [<c03c8fd9>] ? net_rx_action+0x59/0x230
> >  [<c0136f63>] __do_softirq+0x93/0x120
> >  [<c013706a>] do_softirq+0x7a/0x80
> >  [<c0137135>] irq_exit+0x65/0x90
> >  [<c01078b1>] do_IRQ+0x41/0x80
> >  [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
> >  [<c01059ca>] common_interrupt+0x2e/0x34
> >  [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
> >  [<c01033a0>] ? mwait_idle+0x0/0x20
> >  [<c01033b2>] mwait_idle+0x12/0x20
> >  [<c0103141>] cpu_idle+0x61/0x110
> >  [<c04339fd>] rest_init+0x5d/0x60
> >  [<c05a47fa>] start_kernel+0x1fa/0x260
> >  [<c05a4190>] ? unknown_bootoption+0x0/0x130
> >  =======================
> > ---[ end trace 14b601818e6903ac ]---
> > ------------[ cut here ]------------
> > WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0()
> > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event
> > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse
> > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg
> > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib
> > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart
> > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8
> >  [<c0132100>] ? have_callable_console+0x20/0x30
> >  [<c0131844>] warn_on_slowpath+0x54/0x80
> >  [<c01317da>] ? print_oops_end_marker+0x2a/0x30
> >  [<c0131849>] ? warn_on_slowpath+0x59/0x80
> >  [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c0132438>] ? vprintk+0x308/0x320
> >  [<c0400096>] tcp_verify_wq+0x1b6/0x1c0
> >  [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0
> >  [<c03f5ffc>] tcp_mark_head_lost+0xcc/0x150
> >  [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190
> >  [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700
> >  [<c03f7e63>] tcp_ack+0x1b3/0x3a0
> >  [<c03fa21b>] tcp_rcv_established+0x3eb/0x710
> >  [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100
> >  [<c04021fb>] tcp_v4_rcv+0x5db/0x660
> >  [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660
> >  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
> >  [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0
> >  [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0
> >  [<c0156b97>] ? __lock_release+0x47/0x70
> >  [<c03e6147>] ip_local_deliver+0xb7/0xc0
> >  [<c03e6202>] ip_rcv_finish+0xb2/0x3c0
> >  [<c03c01d8>] ? sock_def_readable+0x48/0xa0
> >  [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0
> >  [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0
> >  [<c03e669f>] ip_rcv+0x18f/0x290
> >  [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130
> >  [<c03c8da6>] netif_receive_skb+0x2b6/0x330
> >  [<c03c8c17>] ? netif_receive_skb+0x127/0x330
> >  [<c03c8ea3>] ? process_backlog+0x83/0x100
> >  [<c03c8eae>] process_backlog+0x8e/0x100
> >  [<c03c90bc>] net_rx_action+0x13c/0x230
> >  [<c03c8fd9>] ? net_rx_action+0x59/0x230
> >  [<c0136f63>] __do_softirq+0x93/0x120
> >  [<c013706a>] do_softirq+0x7a/0x80
> >  [<c0137135>] irq_exit+0x65/0x90
> >  [<c01078b1>] do_IRQ+0x41/0x80
> >  [<c0155659>] ? trace_hardirqs_on+0xb9/0x130
> >  [<c01059ca>] common_interrupt+0x2e/0x34
> >  [<c0103390>] ? mwait_idle_with_hints+0x40/0x50
> >  [<c01033a0>] ? mwait_idle+0x0/0x20
> >  [<c01033b2>] mwait_idle+0x12/0x20
> >  [<c0103141>] cpu_idle+0x61/0x110
> >  [<c04339fd>] rest_init+0x5d/0x60
> >  [<c05a47fa>] start_kernel+0x1fa/0x260
> >  [<c05a4190>] ? unknown_bootoption+0x0/0x130
> >  =======================
> > ---[ end trace 14b601818e6903ac ]---
>
> ...But this no longer is, and even more, L: 5 is not valid state at this
> point all (should only happen if we went to RTO but it would reset S to
> zero with newreno):
>
> > P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0)
> > TCP wq(s) LLLLl<
> > TCP wq(h) +++h+<
> > l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616
>
> Surprisingly, it was the first time the WARN_ON for left_out returned
> correct location. This also explains why the patch I sent to Krishna
> didn't print anything (it didn't end up into printing because I forgot
> to add L+S>P check into to the state checking if).
>
> ...so please, could you (others than Denys) try this patch, it should
> solve the issue. And Denys, could you confirm (and if necessary double
> check) that the kernel you saw this similar problem with is the pure
> Linus' mainline, i.e., without any net-2.6.25 or mm bits please, if so,
> that problem persists. And anyway, there were some fackets_out related
> problems reported as well and this doesn't help for that but I think I've
> lost track of who was seeing it due to large number of reports :-), could
> somebody refresh my memory because I currently don't have time to dig it
> up from archives (at least on this week).

Thanks, applied on my test kernel, if there's no warnings I will just mute.

>
>
> --
>  i.
>
> --
> [PATCH] [TCP]: NewReno must count every skb while marking losses
>
> NewReno should add cnt per skb (as with FACK) instead of depending
> on SACKED_ACKED bits which won't be set with it at all.
> Effectively, NewReno should always exists after the first
> iteration anyway (or immediately if there's already head in
> lost_out.
>
> This was fixed earlier in net-2.6.25 but got reverted among other
> stuff and I didn't notice that this is still necessary (actually
> wasn't even considering this case while trying to figure out the
> reports because I lived with different kind of code than it in
> reality was).
>
> This should solve the WARN_ONs in TCP code that as a result of
> this triggered multiple times in every place we check for this
> invariant.
>
> Special thanks to Dave Young <hidave.darkstar@gmail.com> and
> Krishna Kumar2 <krkumar2@in.ibm.com> for trying with my debug
> patches.
>
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi>
> ---
>  net/ipv4/tcp_input.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> index 295490e..aa409a5 100644
> --- a/net/ipv4/tcp_input.c
> +++ b/net/ipv4/tcp_input.c
> @@ -2156,7 +2156,7 @@ static void tcp_mark_head_lost(struct sock *sk, int packets, int fast_rexmit)
>                 tp->lost_skb_hint = skb;
>                 tp->lost_cnt_hint = cnt;
>
> -               if (tcp_is_fack(tp) ||
> +               if (tcp_is_fack(tp) || tcp_is_reno(tp) ||
>                     (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED))
>                         cnt += tcp_skb_pcount(skb);
>
> --
> 1.5.2.2
>

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2008-01-25  1:30 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-01-21  7:55 2.6.24-rc8-mm1 : net tcp_input.c warnings Dave Young
2008-01-21 21:14 ` Ilpo Järvinen
2008-01-22  4:37   ` Dave Young
2008-01-22  9:09     ` Dave Young
2008-01-22  9:18       ` Dave Young
2008-01-22 10:47       ` Ilpo Järvinen
2008-01-23  1:44         ` Dave Young
2008-01-23  3:01           ` David Miller
2008-01-23  7:41             ` Ilpo Järvinen
2008-01-23  7:44               ` Dave Young
2008-01-23  8:41                 ` Ilpo Järvinen
2008-01-23 11:01                   ` Ilpo Järvinen
2008-01-24  2:42                     ` Dave Young
2008-01-24  9:54                       ` Ilpo Järvinen
2008-01-24 10:24                         ` Ilpo Järvinen
2008-01-24 10:46                         ` Krishna Kumar2
2008-01-24 13:11                         ` Kamalesh Babulal
2008-01-25  1:30                         ` Dave Young

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).