* 2.6.24-rc8-mm1 : net tcp_input.c warnings @ 2008-01-21 7:55 Dave Young 2008-01-21 21:14 ` Ilpo Järvinen 0 siblings, 1 reply; 18+ messages in thread From: Dave Young @ 2008-01-21 7:55 UTC (permalink / raw) To: linux-kernel; +Cc: davem, netdev, akpm Please see the kernel messages following,(trigged while using some qemu session) BTW, seems there's some e100 error message as well. PCI: Setting latency timer of device 0000:00:1b.0 to 64 e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI e100: Copyright(c) 1999-2006 Intel Corporation ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20 modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting. ACPI: PCI interrupt for device 0000:03:08.0 disabled e100: probe of 0000:03:08.0 failed with error -12 eth0: setting full-duplex. ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 [<c0132100>] ? printk+0x0/0x20 [<c0131834>] warn_on_slowpath+0x54/0x80 [<c03e8df8>] ? ip_finish_output+0x128/0x2e0 [<c03e9527>] ? ip_output+0xe7/0x100 [<c03e8a88>] ? ip_local_out+0x18/0x20 [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470 [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 [<c0186be1>] ? check_pad_bytes+0x61/0x80 [<c03f6031>] tcp_mark_head_lost+0x121/0x150 [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 [<c03f7d93>] tcp_ack+0x1b3/0x3a0 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 [<c0401967>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b87>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01c8>] ? sock_def_readable+0x48/0xa0 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8d96>] netif_receive_skb+0x2b6/0x330 [<c03c8c07>] ? netif_receive_skb+0x127/0x330 [<c03c8e93>] ? process_backlog+0x83/0x100 [<c03c8e9e>] process_backlog+0x8e/0x100 [<c03c90ac>] net_rx_action+0x13c/0x230 [<c03c8fc9>] ? net_rx_action+0x59/0x230 [<c0136f2e>] ? __do_softirq+0x6e/0x120 [<c0136f53>] __do_softirq+0x93/0x120 [<c013705a>] do_softirq+0x7a/0x80 [<c0137125>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04333bd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 97302d8bf57718dd ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2528 tcp_fastretrans_alert+0x675/0x6b0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 [<c0132100>] ? printk+0x0/0x20 [<c0131834>] warn_on_slowpath+0x54/0x80 [<c0156b87>] ? __lock_release+0x47/0x70 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c03f6ff5>] tcp_fastretrans_alert+0x675/0x6b0 [<c03f7d93>] tcp_ack+0x1b3/0x3a0 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 [<c0401967>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b87>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01c8>] ? sock_def_readable+0x48/0xa0 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8d96>] netif_receive_skb+0x2b6/0x330 [<c03c8c07>] ? netif_receive_skb+0x127/0x330 [<c03c8e93>] ? process_backlog+0x83/0x100 [<c03c8e9e>] process_backlog+0x8e/0x100 [<c03c90ac>] net_rx_action+0x13c/0x230 [<c03c8fc9>] ? net_rx_action+0x59/0x230 [<c0136f2e>] ? __do_softirq+0x6e/0x120 [<c0136f53>] __do_softirq+0x93/0x120 [<c013705a>] do_softirq+0x7a/0x80 [<c0137125>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04333bd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 97302d8bf57718dd ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 [<c0132100>] ? printk+0x0/0x20 [<c0131834>] warn_on_slowpath+0x54/0x80 [<c0153241>] ? put_lock_stats+0x21/0x30 [<c01532b0>] ? lock_release_holdtime+0x60/0x80 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c0156b87>] ? __lock_release+0x47/0x70 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c03f6031>] tcp_mark_head_lost+0x121/0x150 [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 [<c03f7d93>] tcp_ack+0x1b3/0x3a0 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 [<c0401967>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b87>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01c8>] ? sock_def_readable+0x48/0xa0 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8d96>] netif_receive_skb+0x2b6/0x330 [<c03c8c07>] ? netif_receive_skb+0x127/0x330 [<c03c8e93>] ? process_backlog+0x83/0x100 [<c03c8e9e>] process_backlog+0x8e/0x100 [<c03c90ac>] net_rx_action+0x13c/0x230 [<c03c8fc9>] ? net_rx_action+0x59/0x230 [<c0136f2e>] ? __do_softirq+0x6e/0x120 [<c0136f53>] __do_softirq+0x93/0x120 [<c013705a>] do_softirq+0x7a/0x80 [<c0137125>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04333bd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 97302d8bf57718dd ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2528 tcp_fastretrans_alert+0x675/0x6b0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 [<c0132100>] ? printk+0x0/0x20 [<c0131834>] warn_on_slowpath+0x54/0x80 [<c0156b87>] ? __lock_release+0x47/0x70 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c0186ae4>] ? check_bytes_and_report+0x24/0xc0 [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 [<c0186be1>] ? check_pad_bytes+0x61/0x80 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c03f6ff5>] tcp_fastretrans_alert+0x675/0x6b0 [<c03f7d93>] tcp_ack+0x1b3/0x3a0 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 [<c0401967>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b87>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01c8>] ? sock_def_readable+0x48/0xa0 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8d96>] netif_receive_skb+0x2b6/0x330 [<c03c8c07>] ? netif_receive_skb+0x127/0x330 [<c03c8e93>] ? process_backlog+0x83/0x100 [<c03c8e9e>] process_backlog+0x8e/0x100 [<c03c90ac>] net_rx_action+0x13c/0x230 [<c03c8fc9>] ? net_rx_action+0x59/0x230 [<c0136f2e>] ? __do_softirq+0x6e/0x120 [<c0136f53>] __do_softirq+0x93/0x120 [<c013705a>] do_softirq+0x7a/0x80 [<c0137125>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04333bd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 97302d8bf57718dd ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 [<c0132100>] ? printk+0x0/0x20 [<c0131834>] warn_on_slowpath+0x54/0x80 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c0153241>] ? put_lock_stats+0x21/0x30 [<c01532b0>] ? lock_release_holdtime+0x60/0x80 [<c0187918>] ? add_partial+0x48/0x90 [<c03f6031>] tcp_mark_head_lost+0x121/0x150 [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 [<c03f7d93>] tcp_ack+0x1b3/0x3a0 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 [<c0401967>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b87>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01c8>] ? sock_def_readable+0x48/0xa0 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8d96>] netif_receive_skb+0x2b6/0x330 [<c03c8c07>] ? netif_receive_skb+0x127/0x330 [<c03c8e93>] ? process_backlog+0x83/0x100 [<c03c8e9e>] process_backlog+0x8e/0x100 [<c03c90ac>] net_rx_action+0x13c/0x230 [<c03c8fc9>] ? net_rx_action+0x59/0x230 [<c0136f2e>] ? __do_softirq+0x6e/0x120 [<c0136f53>] __do_softirq+0x93/0x120 [<c013705a>] do_softirq+0x7a/0x80 [<c0137125>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04333bd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 97302d8bf57718dd ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2528 tcp_fastretrans_alert+0x675/0x6b0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 [<c0132100>] ? printk+0x0/0x20 [<c0131834>] warn_on_slowpath+0x54/0x80 [<c0186ae4>] ? check_bytes_and_report+0x24/0xc0 [<c0186be1>] ? check_pad_bytes+0x61/0x80 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c03f6ff5>] tcp_fastretrans_alert+0x675/0x6b0 [<c03f7d93>] tcp_ack+0x1b3/0x3a0 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 [<c0401967>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b87>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01c8>] ? sock_def_readable+0x48/0xa0 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8d96>] netif_receive_skb+0x2b6/0x330 [<c03c8c07>] ? netif_receive_skb+0x127/0x330 [<c03c8e93>] ? process_backlog+0x83/0x100 [<c03c8e9e>] process_backlog+0x8e/0x100 [<c03c90ac>] net_rx_action+0x13c/0x230 [<c03c8fc9>] ? net_rx_action+0x59/0x230 [<c0136f2e>] ? __do_softirq+0x6e/0x120 [<c0136f53>] __do_softirq+0x93/0x120 [<c013705a>] do_softirq+0x7a/0x80 [<c0137125>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0151d2c>] ? tick_nohz_stop_sched_tick+0x25c/0x350 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04333bd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 97302d8bf57718dd ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc Pid: 2754, comm: X Not tainted 2.6.24-rc8-mm1 #4 [<c0132100>] ? printk+0x0/0x20 [<c0131834>] warn_on_slowpath+0x54/0x80 [<c015322d>] ? put_lock_stats+0xd/0x30 [<c01532b0>] ? lock_release_holdtime+0x60/0x80 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c0156b87>] ? __lock_release+0x47/0x70 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c04363f9>] ? _spin_unlock_irqrestore+0x39/0x70 [<c013dbb4>] ? group_send_sig_info+0x74/0x80 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c03f6031>] tcp_mark_head_lost+0x121/0x150 [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 [<c03f5ed1>] ? tcp_time_to_recover+0xe1/0xf0 [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 [<c03f7d93>] tcp_ack+0x1b3/0x3a0 [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 [<c0401967>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b87>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01c8>] ? sock_def_readable+0x48/0xa0 [<c03be051>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be097>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042f4d0>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8d96>] netif_receive_skb+0x2b6/0x330 [<c03c8c07>] ? netif_receive_skb+0x127/0x330 [<c03c8e93>] ? process_backlog+0x83/0x100 [<c03c8e9e>] process_backlog+0x8e/0x100 [<c03c90ac>] net_rx_action+0x13c/0x230 [<c03c8fc9>] ? net_rx_action+0x59/0x230 [<c0136f2e>] ? __do_softirq+0x6e/0x120 [<c0136f53>] __do_softirq+0x93/0x120 [<c013705a>] do_softirq+0x7a/0x80 [<c0137125>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155649>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 ======================= ---[ end trace 97302d8bf57718dd ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:1767 tcp_enter_frto+0x27c/0x2a0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 [<c0132100>] ? printk+0x0/0x20 [<c0131834>] warn_on_slowpath+0x54/0x80 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0154972>] ? validate_chain+0x1d2/0x320 [<c0156234>] ? __lock_acquire+0x1e4/0x730 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c03f58ac>] tcp_enter_frto+0x27c/0x2a0 [<c03ff3c2>] tcp_retransmit_timer+0x102/0x400 [<c043611b>] ? _spin_lock+0x4b/0x80 [<c03ff6d4>] ? tcp_write_timer+0x14/0x110 [<c03ff7b3>] tcp_write_timer+0xf3/0x110 [<c013b94b>] run_timer_softirq+0xbb/0x190 [<c03ff6c0>] ? tcp_write_timer+0x0/0x110 [<c0136f53>] __do_softirq+0x93/0x120 [<c013705a>] do_softirq+0x7a/0x80 [<c0137125>] irq_exit+0x65/0x90 [<c011a91d>] smp_apic_timer_interrupt+0x2d/0x40 [<c0105aab>] apic_timer_interrupt+0x33/0x38 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04333bd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 97302d8bf57718dd ]--- ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-21 7:55 2.6.24-rc8-mm1 : net tcp_input.c warnings Dave Young @ 2008-01-21 21:14 ` Ilpo Järvinen 2008-01-22 4:37 ` Dave Young 0 siblings, 1 reply; 18+ messages in thread From: Ilpo Järvinen @ 2008-01-21 21:14 UTC (permalink / raw) To: Dave Young; +Cc: LKML, David Miller, Netdev, Andrew Morton On Mon, 21 Jan 2008, Dave Young wrote: > Please see the kernel messages following,(trigged while using some qemu session) > BTW, seems there's some e100 error message as well. > > PCI: Setting latency timer of device 0000:00:1b.0 to 64 > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI > e100: Copyright(c) 1999-2006 Intel Corporation > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20 > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting. > ACPI: PCI interrupt for device 0000:03:08.0 disabled > e100: probe of 0000:03:08.0 failed with error -12 > eth0: setting full-duplex. > ------------[ cut here ]------------ > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 > [<c0132100>] ? printk+0x0/0x20 > [<c0131834>] warn_on_slowpath+0x54/0x80 > [<c03e8df8>] ? ip_finish_output+0x128/0x2e0 > [<c03e9527>] ? ip_output+0xe7/0x100 > [<c03e8a88>] ? ip_local_out+0x18/0x20 > [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470 > [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 > [<c0186be1>] ? check_pad_bytes+0x61/0x80 > [<c03f6031>] tcp_mark_head_lost+0x121/0x150 > [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 > [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 > [<c03f7d93>] tcp_ack+0x1b3/0x3a0 > [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 > [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 > [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 Doh, once more these S+L things..., the rest are symptom of the first problem. What is strange is that it doesn't show up until now, the last TCP changes that could have some significance are from early Dec/Nov. Is there some reason why you haven't seen this before this (e.g., not tested with similar cfg or so)? I'm a bit worried about its reproducability if it takes this far to see it... -- i. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-21 21:14 ` Ilpo Järvinen @ 2008-01-22 4:37 ` Dave Young 2008-01-22 9:09 ` Dave Young 0 siblings, 1 reply; 18+ messages in thread From: Dave Young @ 2008-01-22 4:37 UTC (permalink / raw) To: Ilpo Järvinen; +Cc: LKML, David Miller, Netdev, Andrew Morton On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > On Mon, 21 Jan 2008, Dave Young wrote: > > > Please see the kernel messages following,(trigged while using some qemu session) > > BTW, seems there's some e100 error message as well. > > > > PCI: Setting latency timer of device 0000:00:1b.0 to 64 > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI > > e100: Copyright(c) 1999-2006 Intel Corporation > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20 > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting. > > ACPI: PCI interrupt for device 0000:03:08.0 disabled > > e100: probe of 0000:03:08.0 failed with error -12 > > eth0: setting full-duplex. > > ------------[ cut here ]------------ > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 > > [<c0132100>] ? printk+0x0/0x20 > > [<c0131834>] warn_on_slowpath+0x54/0x80 > > [<c03e8df8>] ? ip_finish_output+0x128/0x2e0 > > [<c03e9527>] ? ip_output+0xe7/0x100 > > [<c03e8a88>] ? ip_local_out+0x18/0x20 > > [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470 > > [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 > > [<c0186be1>] ? check_pad_bytes+0x61/0x80 > > [<c03f6031>] tcp_mark_head_lost+0x121/0x150 > > [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 > > [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 > > [<c03f7d93>] tcp_ack+0x1b3/0x3a0 > > [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 > > [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 > > [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 > > Doh, once more these S+L things..., the rest are symptom of the first > problem. What is the S+L thing? Could you explain a bit? > > What is strange is that it doesn't show up until now, the last TCP > changes that could have some significance are from early Dec/Nov. Is > there some reason why you haven't seen this before this (e.g., not > tested with similar cfg or so)? Hmm, don't know how to answer ... I'm a bit worried about its > reproducability if it takes this far to see it... > > > -- > i. > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-22 4:37 ` Dave Young @ 2008-01-22 9:09 ` Dave Young 2008-01-22 9:18 ` Dave Young 2008-01-22 10:47 ` Ilpo Järvinen 0 siblings, 2 replies; 18+ messages in thread From: Dave Young @ 2008-01-22 9:09 UTC (permalink / raw) To: Ilpo Järvinen; +Cc: LKML, David Miller, Netdev, Andrew Morton On Jan 22, 2008 12:37 PM, Dave Young <hidave.darkstar@gmail.com> wrote: > > On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > On Mon, 21 Jan 2008, Dave Young wrote: > > > > > Please see the kernel messages following,(trigged while using some qemu session) > > > BTW, seems there's some e100 error message as well. > > > > > > PCI: Setting latency timer of device 0000:00:1b.0 to 64 > > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI > > > e100: Copyright(c) 1999-2006 Intel Corporation > > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20 > > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default > > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting. > > > ACPI: PCI interrupt for device 0000:03:08.0 disabled > > > e100: probe of 0000:03:08.0 failed with error -12 > > > eth0: setting full-duplex. > > > ------------[ cut here ]------------ > > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() > > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc > > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 > > > [<c0132100>] ? printk+0x0/0x20 > > > [<c0131834>] warn_on_slowpath+0x54/0x80 > > > [<c03e8df8>] ? ip_finish_output+0x128/0x2e0 > > > [<c03e9527>] ? ip_output+0xe7/0x100 > > > [<c03e8a88>] ? ip_local_out+0x18/0x20 > > > [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470 > > > [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 > > > [<c0186be1>] ? check_pad_bytes+0x61/0x80 > > > [<c03f6031>] tcp_mark_head_lost+0x121/0x150 > > > [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 > > > [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 > > > [<c03f7d93>] tcp_ack+0x1b3/0x3a0 > > > [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 > > > [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 > > > [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 > > > > Doh, once more these S+L things..., the rest are symptom of the first > > problem. > > What is the S+L thing? Could you explain a bit? > > > > > What is strange is that it doesn't show up until now, the last TCP > > changes that could have some significance are from early Dec/Nov. Is > > there some reason why you haven't seen this before this (e.g., not > > tested with similar cfg or so)? > > Hmm, don't know how to answer ... > > > I'm a bit worried about its > > reproducability if it takes this far to see it... > > It's trigged again in my pc, just while using firefox. > > > > -- > > i. > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-22 9:09 ` Dave Young @ 2008-01-22 9:18 ` Dave Young 2008-01-22 10:47 ` Ilpo Järvinen 1 sibling, 0 replies; 18+ messages in thread From: Dave Young @ 2008-01-22 9:18 UTC (permalink / raw) To: Ilpo Järvinen; +Cc: LKML, David Miller, Netdev, Andrew Morton On Jan 22, 2008 5:09 PM, Dave Young <hidave.darkstar@gmail.com> wrote: > > On Jan 22, 2008 12:37 PM, Dave Young <hidave.darkstar@gmail.com> wrote: > > > > On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > > > On Mon, 21 Jan 2008, Dave Young wrote: > > > > > > > Please see the kernel messages following,(trigged while using some qemu session) > > > > BTW, seems there's some e100 error message as well. > > > > > > > > PCI: Setting latency timer of device 0000:00:1b.0 to 64 > > > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI > > > > e100: Copyright(c) 1999-2006 Intel Corporation > > > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20 > > > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default > > > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting. > > > > ACPI: PCI interrupt for device 0000:03:08.0 disabled > > > > e100: probe of 0000:03:08.0 failed with error -12 > > > > eth0: setting full-duplex. > > > > ------------[ cut here ]------------ > > > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() > > > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc > > > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 > > > > [<c0132100>] ? printk+0x0/0x20 > > > > [<c0131834>] warn_on_slowpath+0x54/0x80 > > > > [<c03e8df8>] ? ip_finish_output+0x128/0x2e0 > > > > [<c03e9527>] ? ip_output+0xe7/0x100 > > > > [<c03e8a88>] ? ip_local_out+0x18/0x20 > > > > [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470 > > > > [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 > > > > [<c0186be1>] ? check_pad_bytes+0x61/0x80 > > > > [<c03f6031>] tcp_mark_head_lost+0x121/0x150 > > > > [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 > > > > [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 > > > > [<c03f7d93>] tcp_ack+0x1b3/0x3a0 > > > > [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 > > > > [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 > > > > [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 > > > > > > Doh, once more these S+L things..., the rest are symptom of the first > > > problem. > > > > What is the S+L thing? Could you explain a bit? > > > > > > > > What is strange is that it doesn't show up until now, the last TCP > > > changes that could have some significance are from early Dec/Nov. Is > > > there some reason why you haven't seen this before this (e.g., not > > > tested with similar cfg or so)? > > > > Hmm, don't know how to answer ... > > > > > > I'm a bit worried about its > > > reproducability if it takes this far to see it... > > > > > It's trigged again in my pc, just while using firefox. Maybe relate to the e100 error, I will apply jiri slaby's e100-iomap-mem-accesses patch to test. > > > > > > > -- > > > i. > > > > > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-22 9:09 ` Dave Young 2008-01-22 9:18 ` Dave Young @ 2008-01-22 10:47 ` Ilpo Järvinen 2008-01-23 1:44 ` Dave Young 1 sibling, 1 reply; 18+ messages in thread From: Ilpo Järvinen @ 2008-01-22 10:47 UTC (permalink / raw) To: Dave Young; +Cc: LKML, David Miller, Netdev, Andrew Morton [-- Attachment #1: Type: TEXT/PLAIN, Size: 11373 bytes --] On Tue, 22 Jan 2008, Dave Young wrote: > On Jan 22, 2008 12:37 PM, Dave Young <hidave.darkstar@gmail.com> wrote: > > > > On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > > > On Mon, 21 Jan 2008, Dave Young wrote: > > > > > > > Please see the kernel messages following,(trigged while using some qemu session) > > > > BTW, seems there's some e100 error message as well. > > > > > > > > PCI: Setting latency timer of device 0000:00:1b.0 to 64 > > > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI > > > > e100: Copyright(c) 1999-2006 Intel Corporation > > > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20 > > > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default > > > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting. > > > > ACPI: PCI interrupt for device 0000:03:08.0 disabled > > > > e100: probe of 0000:03:08.0 failed with error -12 > > > > eth0: setting full-duplex. > > > > ------------[ cut here ]------------ > > > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() > > > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc > > > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 > > > > [<c0132100>] ? printk+0x0/0x20 > > > > [<c0131834>] warn_on_slowpath+0x54/0x80 > > > > [<c03e8df8>] ? ip_finish_output+0x128/0x2e0 > > > > [<c03e9527>] ? ip_output+0xe7/0x100 > > > > [<c03e8a88>] ? ip_local_out+0x18/0x20 > > > > [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470 > > > > [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 > > > > [<c0186be1>] ? check_pad_bytes+0x61/0x80 > > > > [<c03f6031>] tcp_mark_head_lost+0x121/0x150 > > > > [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 > > > > [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 > > > > [<c03f7d93>] tcp_ack+0x1b3/0x3a0 > > > > [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 > > > > [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 > > > > [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 > > > > > > Doh, once more these S+L things..., the rest are symptom of the first > > > problem. > > > > What is the S+L thing? Could you explain a bit? It means that one of the skbs is both SACKed and marked as LOST at the same time in the counters (might be due to miscount of lost/sacked_out too, not necessarilily in the ->sacked bits). Such state is logically invalid because it would mean that the sender thinks that the same packet both reached the receiver and is lost in the network. Traditionally TCP has just silently "corrected" over-estimates (sacked_out+lost_out > packets_out). I changed this couple of releases ago because those over-estimates often are due to bugs that should be fixed (there have been couple of them but it has been very quite on this front long time, months or even half year already; but I might have broken something with the early Dec changes). These problem may originate from a bug that occurred a number of ACKs earlier the WARN_ON triggered, therefore they are a bit tricky to track, those WARN_ON serve just for alerting purposes and usually do not point out where the bug actually occurred. I usually just asked people to include exhaustive verifier which compares ->sacked bitmaps with sacked/lost_out counters and report immediately when the problem shows up, rather than waiting for the cheaper S+L check we do in the WARN_ON to trigger. I tried to collect tracking patch from the previous efforts (hopefully got it right after modifications). > > I'm a bit worried about its > > > reproducability if it takes this far to see it... > > > > > It's trigged again in my pc, just while using firefox. ...Good, then there's some chance to catch it. -- i. [PATCH] [TCP]: debug S+L --- include/net/tcp.h | 8 +++- net/ipv4/tcp_input.c | 6 +++ net/ipv4/tcp_ipv4.c | 101 +++++++++++++++++++++++++++++++++++++++++++++++++ net/ipv4/tcp_output.c | 21 +++++++--- 4 files changed, 129 insertions(+), 7 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 7de4ea3..0685035 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -272,6 +272,8 @@ DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics); #define TCP_ADD_STATS_BH(field, val) SNMP_ADD_STATS_BH(tcp_statistics, field, val) #define TCP_ADD_STATS_USER(field, val) SNMP_ADD_STATS_USER(tcp_statistics, field, val) +extern void tcp_verify_wq(struct sock *sk); + extern void tcp_v4_err(struct sk_buff *skb, u32); extern void tcp_shutdown (struct sock *sk, int how); @@ -768,7 +770,11 @@ static inline __u32 tcp_current_ssthresh(const struct sock *sk) } /* Use define here intentionally to get WARN_ON location shown at the caller */ -#define tcp_verify_left_out(tp) WARN_ON(tcp_left_out(tp) > tp->packets_out) +#define tcp_verify_left_out(tp) \ + do { \ + WARN_ON(tcp_left_out(tp) > tp->packets_out); \ + tcp_verify_wq((struct sock *)tp); \ + } while(0) extern void tcp_enter_cwr(struct sock *sk, const int set_ssthresh); extern __u32 tcp_init_cwnd(struct tcp_sock *tp, struct dst_entry *dst); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index fa2c85c..0bda0e1 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -2645,6 +2645,10 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag) if (do_lost || (tcp_is_fack(tp) && tcp_head_timedout(sk))) tcp_update_scoreboard(sk, fast_rexmit); tcp_cwnd_down(sk, flag); + + WARN_ON(tcp_write_queue_head(sk) == NULL); + WARN_ON(!tp->packets_out); + tcp_xmit_retransmit_queue(sk); } @@ -2848,6 +2852,8 @@ static int tcp_clean_rtx_queue(struct sock *sk, int prior_fackets) tcp_clear_all_retrans_hints(tp); } + tcp_verify_left_out(tp); + if (skb && (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED)) flag |= FLAG_SACK_RENEGING; diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 9aea88b..21f5888 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -108,6 +108,107 @@ struct inet_hashinfo __cacheline_aligned tcp_hashinfo = { .lhash_wait = __WAIT_QUEUE_HEAD_INITIALIZER(tcp_hashinfo.lhash_wait), }; +void tcp_print_queue(struct sock *sk) +{ + struct tcp_sock *tp = tcp_sk(sk); + struct sk_buff *skb; + char s[50+1]; + char h[50+1]; + int idx = 0; + int i; + + tcp_for_write_queue(skb, sk) { + if (skb == tcp_send_head(sk)) + break; + + for (i = 0; i < tcp_skb_pcount(skb); i++) { + if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) { + s[idx] = 'S'; + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + s[idx] = 'B'; + + } else if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) { + s[idx] = 'L'; + } else { + s[idx] = ' '; + } + if (s[idx] != ' ' && skb->len < tp->mss_cache) + s[idx] += 'a' - 'A'; + + if (i == 0) { + if (TCP_SKB_CB(skb)->seq == tcp_highest_sack_seq(tp)) + h[idx] = 'h'; + else + h[idx] = '+'; + } else { + h[idx] = '-'; + } + + if (++idx >= 50) { + s[idx] = 0; + h[idx] = 0; + printk(KERN_ERR "TCP wq(s) %s\n", s); + printk(KERN_ERR "TCP wq(h) %s\n", h); + idx = 0; + } + } + } + if (idx) { + s[idx] = '<'; + s[idx+1] = 0; + h[idx] = '<'; + h[idx+1] = 0; + printk(KERN_ERR "TCP wq(s) %s\n", s); + printk(KERN_ERR "TCP wq(h) %s\n", h); + } + printk(KERN_ERR "l%u s%u f%u p%u seq: su%u hs%u sn%u\n", + tp->lost_out, tp->sacked_out, tp->fackets_out, tp->packets_out, + tp->snd_una, tcp_highest_sack_seq(tp), tp->snd_nxt); +} + +void tcp_verify_wq(struct sock *sk) +{ + struct tcp_sock *tp = tcp_sk(sk); + u32 lost = 0; + u32 sacked = 0; + u32 packets = 0; + struct sk_buff *skb; + + tcp_for_write_queue(skb, sk) { + if (skb == tcp_send_head(sk)) + break; + + if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) { + sacked += tcp_skb_pcount(skb); + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + printk(KERN_ERR "Sacked bitmap S+L: %u %u-%u/%u\n", + TCP_SKB_CB(skb)->sacked, + TCP_SKB_CB(skb)->end_seq - tp->snd_una, + TCP_SKB_CB(skb)->seq - tp->snd_una, + tp->snd_una); + } + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + lost += tcp_skb_pcount(skb); + + packets += tcp_skb_pcount(skb); + } + + WARN_ON(lost != tp->lost_out); + WARN_ON(sacked != tp->sacked_out); + WARN_ON(packets != tp->packets_out); + if ((lost != tp->lost_out) || + (sacked != tp->sacked_out) || + (packets != tp->packets_out)) { + printk(KERN_ERR "P: %u L: %u vs %u S: %u vs %u w: %u-%u (%u)\n", + tp->packets_out, + lost, tp->lost_out, + sacked, tp->sacked_out, + tp->snd_una, tp->snd_nxt, + tp->rx_opt.sack_ok); + tcp_print_queue(sk); + } +} + static int tcp_v4_get_port(struct sock *sk, unsigned short snum) { return inet_csk_get_port(&tcp_hashinfo, sk, snum, diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 89f0188..648340f 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -779,10 +779,9 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len, tp->lost_out -= diff; /* Adjust Reno SACK estimate. */ - if (tcp_is_reno(tp) && diff > 0) { + if (tcp_is_reno(tp) && diff > 0) tcp_dec_pcount_approx_int(&tp->sacked_out, diff); - tcp_verify_left_out(tp); - } + tcp_adjust_fackets_out(sk, skb, diff); } @@ -790,6 +789,8 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len, skb_header_release(buff); tcp_insert_write_queue_after(skb, buff, sk); + tcp_verify_left_out(tp); + return 0; } @@ -1463,6 +1464,7 @@ static int tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle) } else if (result > 0) { sent_pkts = 1; } + tcp_verify_left_out(tp); while ((skb = tcp_send_head(sk))) { unsigned int limit; @@ -1764,6 +1766,7 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *skb, tcp_clear_retrans_hints_partial(tp); sk_wmem_free_skb(sk, next_skb); + tcp_verify_left_out(tp); } /* Do a simple retransmit without using the backoff mechanisms in @@ -1795,13 +1798,13 @@ void tcp_simple_retransmit(struct sock *sk) } } + tcp_verify_left_out(tp); + tcp_clear_all_retrans_hints(tp); if (!lost) return; - tcp_verify_left_out(tp); - /* Don't muck with the congestion window here. * Reason is that we do not increase amount of _data_ * in network, but units changed and effective @@ -1970,8 +1973,10 @@ void tcp_xmit_retransmit_queue(struct sock *sk) * packet to be MSS sized and all the * packet counting works out. */ - if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) + if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) { + tcp_verify_left_out(tp); return; + } if (sacked & TCPCB_LOST) { if (!(sacked & (TCPCB_SACKED_ACKED|TCPCB_SACKED_RETRANS))) { @@ -1997,6 +2002,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk) } } + tcp_verify_left_out(tp); + /* OK, demanded retransmission is finished. */ /* Forward retransmissions are possible only during Recovery. */ @@ -2054,6 +2061,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk) NET_INC_STATS_BH(LINUX_MIB_TCPFORWARDRETRANS); } + + tcp_verify_left_out(tp); } /* Send a fin. The caller locks the socket for us. This cannot be -- 1.5.2.2 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-22 10:47 ` Ilpo Järvinen @ 2008-01-23 1:44 ` Dave Young 2008-01-23 3:01 ` David Miller 0 siblings, 1 reply; 18+ messages in thread From: Dave Young @ 2008-01-23 1:44 UTC (permalink / raw) To: Ilpo Järvinen; +Cc: LKML, David Miller, Netdev, Andrew Morton On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > On Tue, 22 Jan 2008, Dave Young wrote: > > > On Jan 22, 2008 12:37 PM, Dave Young <hidave.darkstar@gmail.com> wrote: > > > > > > On Jan 22, 2008 5:14 AM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > > > > > On Mon, 21 Jan 2008, Dave Young wrote: > > > > > > > > > Please see the kernel messages following,(trigged while using some qemu session) > > > > > BTW, seems there's some e100 error message as well. > > > > > > > > > > PCI: Setting latency timer of device 0000:00:1b.0 to 64 > > > > > e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI > > > > > e100: Copyright(c) 1999-2006 Intel Corporation > > > > > ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20 > > > > > modprobe:2331 conflicting cache attribute efaff000-efb00000 uncached<->default > > > > > e100: 0000:03:08.0: e100_probe: Cannot map device registers, aborting. > > > > > ACPI: PCI interrupt for device 0000:03:08.0 disabled > > > > > e100: probe of 0000:03:08.0 failed with error -12 > > > > > eth0: setting full-duplex. > > > > > ------------[ cut here ]------------ > > > > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x121/0x150() > > > > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb rtc_cmos thermal bluetooth rtc_core serio_raw intel_agp button processor sg snd rtc_lib i2c_i801 evdev agpgart soundcore dcdbas 3c59x pcspkr snd_page_alloc > > > > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #4 > > > > > [<c0132100>] ? printk+0x0/0x20 > > > > > [<c0131834>] warn_on_slowpath+0x54/0x80 > > > > > [<c03e8df8>] ? ip_finish_output+0x128/0x2e0 > > > > > [<c03e9527>] ? ip_output+0xe7/0x100 > > > > > [<c03e8a88>] ? ip_local_out+0x18/0x20 > > > > > [<c03e991c>] ? ip_queue_xmit+0x3dc/0x470 > > > > > [<c043641e>] ? _spin_unlock_irqrestore+0x5e/0x70 > > > > > [<c0186be1>] ? check_pad_bytes+0x61/0x80 > > > > > [<c03f6031>] tcp_mark_head_lost+0x121/0x150 > > > > > [<c03f60ac>] tcp_update_scoreboard+0x4c/0x170 > > > > > [<c03f6e0a>] tcp_fastretrans_alert+0x48a/0x6b0 > > > > > [<c03f7d93>] tcp_ack+0x1b3/0x3a0 > > > > > [<c03fa14b>] tcp_rcv_established+0x3eb/0x710 > > > > > [<c04015c5>] tcp_v4_do_rcv+0xe5/0x100 > > > > > [<c0401bbb>] tcp_v4_rcv+0x5db/0x660 > > > > > > > > Doh, once more these S+L things..., the rest are symptom of the first > > > > problem. > > > > > > What is the S+L thing? Could you explain a bit? > > It means that one of the skbs is both SACKed and marked as LOST at the > same time in the counters (might be due to miscount of lost/sacked_out > too, not necessarilily in the ->sacked bits). Such state is logically > invalid because it would mean that the sender thinks that the same packet > both reached the receiver and is lost in the network. > > Traditionally TCP has just silently "corrected" over-estimates > (sacked_out+lost_out > packets_out). I changed this couple of releases ago > because those over-estimates often are due to bugs that should be fixed > (there have been couple of them but it has been very quite on this front > long time, months or even half year already; but I might have broken > something with the early Dec changes). > > These problem may originate from a bug that occurred a number of ACKs > earlier the WARN_ON triggered, therefore they are a bit tricky to track, > those WARN_ON serve just for alerting purposes and usually do not point > out where the bug actually occurred. > > I usually just asked people to include exhaustive verifier which compares > ->sacked bitmaps with sacked/lost_out counters and report immediately when > the problem shows up, rather than waiting for the cheaper S+L check we do > in the WARN_ON to trigger. I tried to collect tracking patch from the > previous efforts (hopefully got it right after modifications). > > > > I'm a bit worried about its > > > > reproducability if it takes this far to see it... > > > > > > > > It's trigged again in my pc, just while using firefox. > > ...Good, then there's some chance to catch it. > > -- > i. > > [PATCH] [TCP]: debug S+L Thanks, If there's new findings I will let you know. > > --- > include/net/tcp.h | 8 +++- > net/ipv4/tcp_input.c | 6 +++ > net/ipv4/tcp_ipv4.c | 101 +++++++++++++++++++++++++++++++++++++++++++++++++ > net/ipv4/tcp_output.c | 21 +++++++--- > 4 files changed, 129 insertions(+), 7 deletions(-) > > diff --git a/include/net/tcp.h b/include/net/tcp.h > index 7de4ea3..0685035 100644 > --- a/include/net/tcp.h > +++ b/include/net/tcp.h > @@ -272,6 +272,8 @@ DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics); > #define TCP_ADD_STATS_BH(field, val) SNMP_ADD_STATS_BH(tcp_statistics, field, val) > #define TCP_ADD_STATS_USER(field, val) SNMP_ADD_STATS_USER(tcp_statistics, field, val) > > +extern void tcp_verify_wq(struct sock *sk); > + > extern void tcp_v4_err(struct sk_buff *skb, u32); > > extern void tcp_shutdown (struct sock *sk, int how); > @@ -768,7 +770,11 @@ static inline __u32 tcp_current_ssthresh(const struct sock *sk) > } > > /* Use define here intentionally to get WARN_ON location shown at the caller */ > -#define tcp_verify_left_out(tp) WARN_ON(tcp_left_out(tp) > tp->packets_out) > +#define tcp_verify_left_out(tp) \ > + do { \ > + WARN_ON(tcp_left_out(tp) > tp->packets_out); \ > + tcp_verify_wq((struct sock *)tp); \ > + } while(0) > > extern void tcp_enter_cwr(struct sock *sk, const int set_ssthresh); > extern __u32 tcp_init_cwnd(struct tcp_sock *tp, struct dst_entry *dst); > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c > index fa2c85c..0bda0e1 100644 > --- a/net/ipv4/tcp_input.c > +++ b/net/ipv4/tcp_input.c > @@ -2645,6 +2645,10 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag) > if (do_lost || (tcp_is_fack(tp) && tcp_head_timedout(sk))) > tcp_update_scoreboard(sk, fast_rexmit); > tcp_cwnd_down(sk, flag); > + > + WARN_ON(tcp_write_queue_head(sk) == NULL); > + WARN_ON(!tp->packets_out); > + > tcp_xmit_retransmit_queue(sk); > } > > @@ -2848,6 +2852,8 @@ static int tcp_clean_rtx_queue(struct sock *sk, int prior_fackets) > tcp_clear_all_retrans_hints(tp); > } > > + tcp_verify_left_out(tp); > + > if (skb && (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED)) > flag |= FLAG_SACK_RENEGING; > > diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c > index 9aea88b..21f5888 100644 > --- a/net/ipv4/tcp_ipv4.c > +++ b/net/ipv4/tcp_ipv4.c > @@ -108,6 +108,107 @@ struct inet_hashinfo __cacheline_aligned tcp_hashinfo = { > .lhash_wait = __WAIT_QUEUE_HEAD_INITIALIZER(tcp_hashinfo.lhash_wait), > }; > > +void tcp_print_queue(struct sock *sk) > +{ > + struct tcp_sock *tp = tcp_sk(sk); > + struct sk_buff *skb; > + char s[50+1]; > + char h[50+1]; > + int idx = 0; > + int i; > + > + tcp_for_write_queue(skb, sk) { > + if (skb == tcp_send_head(sk)) > + break; > + > + for (i = 0; i < tcp_skb_pcount(skb); i++) { > + if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) { > + s[idx] = 'S'; > + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) > + s[idx] = 'B'; > + > + } else if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) { > + s[idx] = 'L'; > + } else { > + s[idx] = ' '; > + } > + if (s[idx] != ' ' && skb->len < tp->mss_cache) > + s[idx] += 'a' - 'A'; > + > + if (i == 0) { > + if (TCP_SKB_CB(skb)->seq == tcp_highest_sack_seq(tp)) > + h[idx] = 'h'; > + else > + h[idx] = '+'; > + } else { > + h[idx] = '-'; > + } > + > + if (++idx >= 50) { > + s[idx] = 0; > + h[idx] = 0; > + printk(KERN_ERR "TCP wq(s) %s\n", s); > + printk(KERN_ERR "TCP wq(h) %s\n", h); > + idx = 0; > + } > + } > + } > + if (idx) { > + s[idx] = '<'; > + s[idx+1] = 0; > + h[idx] = '<'; > + h[idx+1] = 0; > + printk(KERN_ERR "TCP wq(s) %s\n", s); > + printk(KERN_ERR "TCP wq(h) %s\n", h); > + } > + printk(KERN_ERR "l%u s%u f%u p%u seq: su%u hs%u sn%u\n", > + tp->lost_out, tp->sacked_out, tp->fackets_out, tp->packets_out, > + tp->snd_una, tcp_highest_sack_seq(tp), tp->snd_nxt); > +} > + > +void tcp_verify_wq(struct sock *sk) > +{ > + struct tcp_sock *tp = tcp_sk(sk); > + u32 lost = 0; > + u32 sacked = 0; > + u32 packets = 0; > + struct sk_buff *skb; > + > + tcp_for_write_queue(skb, sk) { > + if (skb == tcp_send_head(sk)) > + break; > + > + if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) { > + sacked += tcp_skb_pcount(skb); > + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) > + printk(KERN_ERR "Sacked bitmap S+L: %u %u-%u/%u\n", > + TCP_SKB_CB(skb)->sacked, > + TCP_SKB_CB(skb)->end_seq - tp->snd_una, > + TCP_SKB_CB(skb)->seq - tp->snd_una, > + tp->snd_una); > + } > + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) > + lost += tcp_skb_pcount(skb); > + > + packets += tcp_skb_pcount(skb); > + } > + > + WARN_ON(lost != tp->lost_out); > + WARN_ON(sacked != tp->sacked_out); > + WARN_ON(packets != tp->packets_out); > + if ((lost != tp->lost_out) || > + (sacked != tp->sacked_out) || > + (packets != tp->packets_out)) { > + printk(KERN_ERR "P: %u L: %u vs %u S: %u vs %u w: %u-%u (%u)\n", > + tp->packets_out, > + lost, tp->lost_out, > + sacked, tp->sacked_out, > + tp->snd_una, tp->snd_nxt, > + tp->rx_opt.sack_ok); > + tcp_print_queue(sk); > + } > +} > + > static int tcp_v4_get_port(struct sock *sk, unsigned short snum) > { > return inet_csk_get_port(&tcp_hashinfo, sk, snum, > diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c > index 89f0188..648340f 100644 > --- a/net/ipv4/tcp_output.c > +++ b/net/ipv4/tcp_output.c > @@ -779,10 +779,9 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len, > tp->lost_out -= diff; > > /* Adjust Reno SACK estimate. */ > - if (tcp_is_reno(tp) && diff > 0) { > + if (tcp_is_reno(tp) && diff > 0) > tcp_dec_pcount_approx_int(&tp->sacked_out, diff); > - tcp_verify_left_out(tp); > - } > + > tcp_adjust_fackets_out(sk, skb, diff); > } > > @@ -790,6 +789,8 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len, > skb_header_release(buff); > tcp_insert_write_queue_after(skb, buff, sk); > > + tcp_verify_left_out(tp); > + > return 0; > } > > @@ -1463,6 +1464,7 @@ static int tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle) > } else if (result > 0) { > sent_pkts = 1; > } > + tcp_verify_left_out(tp); > > while ((skb = tcp_send_head(sk))) { > unsigned int limit; > @@ -1764,6 +1766,7 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *skb, > tcp_clear_retrans_hints_partial(tp); > > sk_wmem_free_skb(sk, next_skb); > + tcp_verify_left_out(tp); > } > > /* Do a simple retransmit without using the backoff mechanisms in > @@ -1795,13 +1798,13 @@ void tcp_simple_retransmit(struct sock *sk) > } > } > > + tcp_verify_left_out(tp); > + > tcp_clear_all_retrans_hints(tp); > > if (!lost) > return; > > - tcp_verify_left_out(tp); > - > /* Don't muck with the congestion window here. > * Reason is that we do not increase amount of _data_ > * in network, but units changed and effective > @@ -1970,8 +1973,10 @@ void tcp_xmit_retransmit_queue(struct sock *sk) > * packet to be MSS sized and all the > * packet counting works out. > */ > - if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) > + if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) { > + tcp_verify_left_out(tp); > return; > + } > > if (sacked & TCPCB_LOST) { > if (!(sacked & (TCPCB_SACKED_ACKED|TCPCB_SACKED_RETRANS))) { > @@ -1997,6 +2002,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk) > } > } > > + tcp_verify_left_out(tp); > + > /* OK, demanded retransmission is finished. */ > > /* Forward retransmissions are possible only during Recovery. */ > @@ -2054,6 +2061,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk) > > NET_INC_STATS_BH(LINUX_MIB_TCPFORWARDRETRANS); > } > + > + tcp_verify_left_out(tp); > } > > /* Send a fin. The caller locks the socket for us. This cannot be > -- > 1.5.2.2 > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-23 1:44 ` Dave Young @ 2008-01-23 3:01 ` David Miller 2008-01-23 7:41 ` Ilpo Järvinen 0 siblings, 1 reply; 18+ messages in thread From: David Miller @ 2008-01-23 3:01 UTC (permalink / raw) To: hidave.darkstar; +Cc: ilpo.jarvinen, linux-kernel, netdev, akpm From: "Dave Young" <hidave.darkstar@gmail.com> Date: Wed, 23 Jan 2008 09:44:30 +0800 > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > [PATCH] [TCP]: debug S+L > > Thanks, If there's new findings I will let you know. Thanks for helping with this bug Dave. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-23 3:01 ` David Miller @ 2008-01-23 7:41 ` Ilpo Järvinen 2008-01-23 7:44 ` Dave Young 0 siblings, 1 reply; 18+ messages in thread From: Ilpo Järvinen @ 2008-01-23 7:41 UTC (permalink / raw) To: hidave.darkstar; +Cc: LKML, Netdev, Andrew Morton [-- Attachment #1: Type: TEXT/PLAIN, Size: 769 bytes --] On Tue, 22 Jan 2008, David Miller wrote: > From: "Dave Young" <hidave.darkstar@gmail.com> > Date: Wed, 23 Jan 2008 09:44:30 +0800 > > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > [PATCH] [TCP]: debug S+L > > > > Thanks, If there's new findings I will let you know. > > Thanks for helping with this bug Dave. I noticed btw that there thing might (is likely to) spuriously trigger at WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK is not enabled. If that does happen too often, I send a fixed patch for it, yet, the fact that I print print tp->rx_opt.sack_ok allows identification of those cases already as it's zero when SACK is not enabled. Just ask if you need the updated debug patch. -- i. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-23 7:41 ` Ilpo Järvinen @ 2008-01-23 7:44 ` Dave Young 2008-01-23 8:41 ` Ilpo Järvinen 0 siblings, 1 reply; 18+ messages in thread From: Dave Young @ 2008-01-23 7:44 UTC (permalink / raw) To: Ilpo Järvinen; +Cc: LKML, Netdev, Andrew Morton On Jan 23, 2008 3:41 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > On Tue, 22 Jan 2008, David Miller wrote: > > > From: "Dave Young" <hidave.darkstar@gmail.com> > > Date: Wed, 23 Jan 2008 09:44:30 +0800 > > > > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > [PATCH] [TCP]: debug S+L > > > > > > Thanks, If there's new findings I will let you know. > > > > Thanks for helping with this bug Dave. > > I noticed btw that there thing might (is likely to) spuriously trigger at > WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK > is not enabled. If that does happen too often, I send a fixed patch for > it, yet, the fact that I print print tp->rx_opt.sack_ok allows > identification of those cases already as it's zero when SACK is not > enabled. > > Just ask if you need the updated debug patch. Thanks, please send, I would like to get it. > > -- > i. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-23 7:44 ` Dave Young @ 2008-01-23 8:41 ` Ilpo Järvinen 2008-01-23 11:01 ` Ilpo Järvinen 0 siblings, 1 reply; 18+ messages in thread From: Ilpo Järvinen @ 2008-01-23 8:41 UTC (permalink / raw) To: Dave Young; +Cc: LKML, Netdev, Andrew Morton [-- Attachment #1: Type: TEXT/PLAIN, Size: 9210 bytes --] On Wed, 23 Jan 2008, Dave Young wrote: > On Jan 23, 2008 3:41 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > On Tue, 22 Jan 2008, David Miller wrote: > > > > > From: "Dave Young" <hidave.darkstar@gmail.com> > > > Date: Wed, 23 Jan 2008 09:44:30 +0800 > > > > > > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > > [PATCH] [TCP]: debug S+L > > > > > > > > Thanks, If there's new findings I will let you know. > > > > > > Thanks for helping with this bug Dave. > > > > I noticed btw that there thing might (is likely to) spuriously trigger at > > WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK > > is not enabled. If that does happen too often, I send a fixed patch for > > it, yet, the fact that I print print tp->rx_opt.sack_ok allows > > identification of those cases already as it's zero when SACK is not > > enabled. > > > > Just ask if you need the updated debug patch. > > Thanks, please send, I would like to get it. There you go. I fixed non-SACK case by adding tcp_is_sack checks there and also added two verifys to tcp_ack to see if there's corruption outside of TCP. -- i. [PATCH] [TCP]: debug S+L --- include/net/tcp.h | 8 +++- net/ipv4/tcp_input.c | 10 +++++ net/ipv4/tcp_ipv4.c | 101 +++++++++++++++++++++++++++++++++++++++++++++++++ net/ipv4/tcp_output.c | 21 +++++++--- 4 files changed, 133 insertions(+), 7 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 7de4ea3..0685035 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -272,6 +272,8 @@ DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics); #define TCP_ADD_STATS_BH(field, val) SNMP_ADD_STATS_BH(tcp_statistics, field, val) #define TCP_ADD_STATS_USER(field, val) SNMP_ADD_STATS_USER(tcp_statistics, field, val) +extern void tcp_verify_wq(struct sock *sk); + extern void tcp_v4_err(struct sk_buff *skb, u32); extern void tcp_shutdown (struct sock *sk, int how); @@ -768,7 +770,11 @@ static inline __u32 tcp_current_ssthresh(const struct sock *sk) } /* Use define here intentionally to get WARN_ON location shown at the caller */ -#define tcp_verify_left_out(tp) WARN_ON(tcp_left_out(tp) > tp->packets_out) +#define tcp_verify_left_out(tp) \ + do { \ + WARN_ON(tcp_left_out(tp) > tp->packets_out); \ + tcp_verify_wq((struct sock *)tp); \ + } while(0) extern void tcp_enter_cwr(struct sock *sk, const int set_ssthresh); extern __u32 tcp_init_cwnd(struct tcp_sock *tp, struct dst_entry *dst); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index fa2c85c..cdacf70 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -2645,6 +2645,10 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag) if (do_lost || (tcp_is_fack(tp) && tcp_head_timedout(sk))) tcp_update_scoreboard(sk, fast_rexmit); tcp_cwnd_down(sk, flag); + + WARN_ON(tcp_write_queue_head(sk) == NULL); + WARN_ON(!tp->packets_out); + tcp_xmit_retransmit_queue(sk); } @@ -2848,6 +2852,8 @@ static int tcp_clean_rtx_queue(struct sock *sk, int prior_fackets) tcp_clear_all_retrans_hints(tp); } + tcp_verify_left_out(tp); + if (skb && (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED)) flag |= FLAG_SACK_RENEGING; @@ -3175,6 +3181,8 @@ static int tcp_ack(struct sock *sk, struct sk_buff *skb, int flag) prior_fackets = tp->fackets_out; prior_in_flight = tcp_packets_in_flight(tp); + tcp_verify_left_out(tp); + if (!(flag & FLAG_SLOWPATH) && after(ack, prior_snd_una)) { /* Window is constant, pure forward advance. * No more checks are required. @@ -3237,6 +3245,8 @@ static int tcp_ack(struct sock *sk, struct sk_buff *skb, int flag) if ((flag & FLAG_FORWARD_PROGRESS) || !(flag & FLAG_NOT_DUP)) dst_confirm(sk->sk_dst_cache); + tcp_verify_left_out(tp); + return 1; no_queue: diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 9aea88b..7e8ab40 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -108,6 +108,107 @@ struct inet_hashinfo __cacheline_aligned tcp_hashinfo = { .lhash_wait = __WAIT_QUEUE_HEAD_INITIALIZER(tcp_hashinfo.lhash_wait), }; +void tcp_print_queue(struct sock *sk) +{ + struct tcp_sock *tp = tcp_sk(sk); + struct sk_buff *skb; + char s[50+1]; + char h[50+1]; + int idx = 0; + int i; + + tcp_for_write_queue(skb, sk) { + if (skb == tcp_send_head(sk)) + break; + + for (i = 0; i < tcp_skb_pcount(skb); i++) { + if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) { + s[idx] = 'S'; + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + s[idx] = 'B'; + + } else if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) { + s[idx] = 'L'; + } else { + s[idx] = ' '; + } + if (s[idx] != ' ' && skb->len < tp->mss_cache) + s[idx] += 'a' - 'A'; + + if (i == 0) { + if (TCP_SKB_CB(skb)->seq == tcp_highest_sack_seq(tp)) + h[idx] = 'h'; + else + h[idx] = '+'; + } else { + h[idx] = '-'; + } + + if (++idx >= 50) { + s[idx] = 0; + h[idx] = 0; + printk(KERN_ERR "TCP wq(s) %s\n", s); + printk(KERN_ERR "TCP wq(h) %s\n", h); + idx = 0; + } + } + } + if (idx) { + s[idx] = '<'; + s[idx+1] = 0; + h[idx] = '<'; + h[idx+1] = 0; + printk(KERN_ERR "TCP wq(s) %s\n", s); + printk(KERN_ERR "TCP wq(h) %s\n", h); + } + printk(KERN_ERR "l%u s%u f%u p%u seq: su%u hs%u sn%u\n", + tp->lost_out, tp->sacked_out, tp->fackets_out, tp->packets_out, + tp->snd_una, tcp_highest_sack_seq(tp), tp->snd_nxt); +} + +void tcp_verify_wq(struct sock *sk) +{ + struct tcp_sock *tp = tcp_sk(sk); + u32 lost = 0; + u32 sacked = 0; + u32 packets = 0; + struct sk_buff *skb; + + tcp_for_write_queue(skb, sk) { + if (skb == tcp_send_head(sk)) + break; + + if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) { + sacked += tcp_skb_pcount(skb); + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + printk(KERN_ERR "Sacked bitmap S+L: %u %u-%u/%u\n", + TCP_SKB_CB(skb)->sacked, + TCP_SKB_CB(skb)->end_seq - tp->snd_una, + TCP_SKB_CB(skb)->seq - tp->snd_una, + tp->snd_una); + } + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + lost += tcp_skb_pcount(skb); + + packets += tcp_skb_pcount(skb); + } + + WARN_ON(lost != tp->lost_out); + WARN_ON(tcp_is_sack(tp) && (sacked != tp->sacked_out)); + WARN_ON(packets != tp->packets_out); + if ((lost != tp->lost_out) || + (tcp_is_sack(tp) && (sacked != tp->sacked_out)) || + (packets != tp->packets_out)) { + printk(KERN_ERR "P: %u L: %u vs %u S: %u vs %u w: %u-%u (%u)\n", + tp->packets_out, + lost, tp->lost_out, + sacked, tp->sacked_out, + tp->snd_una, tp->snd_nxt, + tp->rx_opt.sack_ok); + tcp_print_queue(sk); + } +} + static int tcp_v4_get_port(struct sock *sk, unsigned short snum) { return inet_csk_get_port(&tcp_hashinfo, sk, snum, diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 89f0188..648340f 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -779,10 +779,9 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len, tp->lost_out -= diff; /* Adjust Reno SACK estimate. */ - if (tcp_is_reno(tp) && diff > 0) { + if (tcp_is_reno(tp) && diff > 0) tcp_dec_pcount_approx_int(&tp->sacked_out, diff); - tcp_verify_left_out(tp); - } + tcp_adjust_fackets_out(sk, skb, diff); } @@ -790,6 +789,8 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len, skb_header_release(buff); tcp_insert_write_queue_after(skb, buff, sk); + tcp_verify_left_out(tp); + return 0; } @@ -1463,6 +1464,7 @@ static int tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle) } else if (result > 0) { sent_pkts = 1; } + tcp_verify_left_out(tp); while ((skb = tcp_send_head(sk))) { unsigned int limit; @@ -1764,6 +1766,7 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *skb, tcp_clear_retrans_hints_partial(tp); sk_wmem_free_skb(sk, next_skb); + tcp_verify_left_out(tp); } /* Do a simple retransmit without using the backoff mechanisms in @@ -1795,13 +1798,13 @@ void tcp_simple_retransmit(struct sock *sk) } } + tcp_verify_left_out(tp); + tcp_clear_all_retrans_hints(tp); if (!lost) return; - tcp_verify_left_out(tp); - /* Don't muck with the congestion window here. * Reason is that we do not increase amount of _data_ * in network, but units changed and effective @@ -1970,8 +1973,10 @@ void tcp_xmit_retransmit_queue(struct sock *sk) * packet to be MSS sized and all the * packet counting works out. */ - if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) + if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) { + tcp_verify_left_out(tp); return; + } if (sacked & TCPCB_LOST) { if (!(sacked & (TCPCB_SACKED_ACKED|TCPCB_SACKED_RETRANS))) { @@ -1997,6 +2002,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk) } } + tcp_verify_left_out(tp); + /* OK, demanded retransmission is finished. */ /* Forward retransmissions are possible only during Recovery. */ @@ -2054,6 +2061,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk) NET_INC_STATS_BH(LINUX_MIB_TCPFORWARDRETRANS); } + + tcp_verify_left_out(tp); } /* Send a fin. The caller locks the socket for us. This cannot be -- 1.5.2.2 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-23 8:41 ` Ilpo Järvinen @ 2008-01-23 11:01 ` Ilpo Järvinen 2008-01-24 2:42 ` Dave Young 0 siblings, 1 reply; 18+ messages in thread From: Ilpo Järvinen @ 2008-01-23 11:01 UTC (permalink / raw) To: Dave Young; +Cc: LKML, Netdev, Andrew Morton [-- Attachment #1: Type: TEXT/PLAIN, Size: 1471 bytes --] On Wed, 23 Jan 2008, Ilpo Järvinen wrote: > On Wed, 23 Jan 2008, Dave Young wrote: > > > On Jan 23, 2008 3:41 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > > > On Tue, 22 Jan 2008, David Miller wrote: > > > > > > > From: "Dave Young" <hidave.darkstar@gmail.com> > > > > Date: Wed, 23 Jan 2008 09:44:30 +0800 > > > > > > > > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > > > [PATCH] [TCP]: debug S+L > > > > > > > > > > Thanks, If there's new findings I will let you know. > > > > > > > > Thanks for helping with this bug Dave. > > > > > > I noticed btw that there thing might (is likely to) spuriously trigger at > > > WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK > > > is not enabled. If that does happen too often, I send a fixed patch for > > > it, yet, the fact that I print print tp->rx_opt.sack_ok allows > > > identification of those cases already as it's zero when SACK is not > > > enabled. > > > > > > Just ask if you need the updated debug patch. > > > > Thanks, please send, I would like to get it. > > There you go. I fixed non-SACK case by adding tcp_is_sack checks there and > also added two verifys to tcp_ack to see if there's corruption outside of > TCP. There's some discussion about a problem that is very likely the same as in here (sorry for not remembering to cc you in there due to rapid progress): http://marc.info/?t=120107174200003&r=1&w=2 -- i. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-23 11:01 ` Ilpo Järvinen @ 2008-01-24 2:42 ` Dave Young 2008-01-24 9:54 ` Ilpo Järvinen 0 siblings, 1 reply; 18+ messages in thread From: Dave Young @ 2008-01-24 2:42 UTC (permalink / raw) To: Ilpo Järvinen; +Cc: LKML, Netdev, Andrew Morton On Jan 23, 2008 7:01 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > On Wed, 23 Jan 2008, Ilpo Järvinen wrote: > > > On Wed, 23 Jan 2008, Dave Young wrote: > > > > > On Jan 23, 2008 3:41 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > > > > > On Tue, 22 Jan 2008, David Miller wrote: > > > > > > > > > From: "Dave Young" <hidave.darkstar@gmail.com> > > > > > Date: Wed, 23 Jan 2008 09:44:30 +0800 > > > > > > > > > > > On Jan 22, 2008 6:47 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > > > > > > > [PATCH] [TCP]: debug S+L > > > > > > > > > > > > Thanks, If there's new findings I will let you know. > > > > > > > > > > Thanks for helping with this bug Dave. > > > > > > > > I noticed btw that there thing might (is likely to) spuriously trigger at > > > > WARN_ON(sacked != tp->sacked_out); because those won't be equal when SACK > > > > is not enabled. If that does happen too often, I send a fixed patch for > > > > it, yet, the fact that I print print tp->rx_opt.sack_ok allows > > > > identification of those cases already as it's zero when SACK is not > > > > enabled. > > > > > > > > Just ask if you need the updated debug patch. > > > > > > Thanks, please send, I would like to get it. > > > > There you go. I fixed non-SACK case by adding tcp_is_sack checks there and > > also added two verifys to tcp_ack to see if there's corruption outside of > > TCP. > > There's some discussion about a problem that is very likely the same as in > here (sorry for not remembering to cc you in there due to rapid progress): > > http://marc.info/?t=120107174200003&r=1&w=2 Thanks. New warning trigged with your debug patch: ACPI: PCI Interrupt 0000:00:1b.0[A] -> GSI 16 (level, low) -> IRQ 16 PCI: Setting latency timer of device 0000:00:1b.0 to 64 e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI e100: Copyright(c) 1999-2006 Intel Corporation ACPI: PCI Interrupt 0000:03:08.0[A] -> GSI 20 (level, low) -> IRQ 20 e100: eth1: e100_probe: addr 0xefaff000, irq 20, MAC addr 00:13:72:e7:4d:66 eth0: setting full-duplex. ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0150030>] ? timer_list_show_tickdevices+0xf0/0x110 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c0153251>] ? put_lock_stats+0x21/0x30 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0 [<c0186bf1>] ? check_pad_bytes+0x61/0x80 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03f7509>] ? tcp_clean_rtx_queue+0x2d9/0x5b0 [<c03f5540>] tcp_add_reno_sack+0x30/0x50 [<c03f6d72>] tcp_fastretrans_alert+0x3d2/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f3e>] ? __do_softirq+0x6e/0x120 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0250030>] ? generic_make_request+0x1c0/0x2e0 [<c0132128>] ? printk+0x18/0x20 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c03f674e>] tcp_try_to_open+0x2e/0x160 [<c03f6e9e>] tcp_fastretrans_alert+0x4fe/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f3e>] ? __do_softirq+0x6e/0x120 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0 [<c0186bf1>] ? check_pad_bytes+0x61/0x80 [<c0186dbd>] ? check_object+0xad/0x1e0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130 [<c03c1578>] ? kfree_skbmem+0x98/0xc0 [<c03fcf03>] tcp_write_xmit+0x63/0x290 [<c03c1578>] ? kfree_skbmem+0x98/0xc0 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80 [<c03fa23e>] tcp_rcv_established+0x40e/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f3e>] ? __do_softirq+0x6e/0x120 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0132128>] ? printk+0x18/0x20 [<c01317da>] ? print_oops_end_marker+0x2a/0x30 [<c0131849>] ? warn_on_slowpath+0x59/0x80 [<c025f9e4>] ? vsnprintf+0x2d4/0x4f0 [<c0132128>] ? printk+0x18/0x20 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c0186bf1>] ? check_pad_bytes+0x61/0x80 [<c03f7509>] tcp_clean_rtx_queue+0x2d9/0x5b0 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c03f7dd2>] tcp_ack+0x122/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f3e>] ? __do_softirq+0x6e/0x120 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0132438>] ? vprintk+0x308/0x320 [<c025f9e4>] ? vsnprintf+0x2d4/0x4f0 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03f7509>] ? tcp_clean_rtx_queue+0x2d9/0x5b0 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c03f6a93>] tcp_fastretrans_alert+0xf3/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f3e>] ? __do_softirq+0x6e/0x120 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s1 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0250030>] ? generic_make_request+0x1c0/0x2e0 [<c0132128>] ? printk+0x18/0x20 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c03f5540>] tcp_add_reno_sack+0x30/0x50 [<c03f6d72>] tcp_fastretrans_alert+0x3d2/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f3e>] ? __do_softirq+0x6e/0x120 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0250030>] ? generic_make_request+0x1c0/0x2e0 [<c0132128>] ? printk+0x18/0x20 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c03f674e>] tcp_try_to_open+0x2e/0x160 [<c03f6e9e>] tcp_fastretrans_alert+0x4fe/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f3e>] ? __do_softirq+0x6e/0x120 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0 [<c0132438>] ? vprintk+0x308/0x320 [<c0186bf1>] ? check_pad_bytes+0x61/0x80 [<c0186dbd>] ? check_object+0xad/0x1e0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130 [<c03c1578>] ? kfree_skbmem+0x98/0xc0 [<c03fcf03>] tcp_write_xmit+0x63/0x290 [<c03c1578>] ? kfree_skbmem+0x98/0xc0 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80 [<c03fa23e>] tcp_rcv_established+0x40e/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f3e>] ? __do_softirq+0x6e/0x120 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0132128>] ? printk+0x18/0x20 [<c01317da>] ? print_oops_end_marker+0x2a/0x30 [<c0131849>] ? warn_on_slowpath+0x59/0x80 [<c025f9e4>] ? vsnprintf+0x2d4/0x4f0 [<c0132128>] ? printk+0x18/0x20 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c0186bf1>] ? check_pad_bytes+0x61/0x80 [<c03f7509>] tcp_clean_rtx_queue+0x2d9/0x5b0 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c03f7dd2>] tcp_ack+0x122/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0132438>] ? vprintk+0x308/0x320 [<c025f9e4>] ? vsnprintf+0x2d4/0x4f0 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03f7509>] ? tcp_clean_rtx_queue+0x2d9/0x5b0 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c03f6a93>] tcp_fastretrans_alert+0xf3/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 2 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s2 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0250030>] ? generic_make_request+0x1c0/0x2e0 [<c0132128>] ? printk+0x18/0x20 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c03f5540>] tcp_add_reno_sack+0x30/0x50 [<c03f6d72>] tcp_fastretrans_alert+0x3d2/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0) TCP wq(s) < TCP wq(h) +++h+< l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c03f6052>] tcp_mark_head_lost+0x122/0x150 [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190 [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c01317da>] ? print_oops_end_marker+0x2a/0x30 [<c0131849>] ? warn_on_slowpath+0x59/0x80 [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 [<c0132438>] ? vprintk+0x308/0x320 [<c0132438>] ? vprintk+0x308/0x320 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c03f5ffc>] tcp_mark_head_lost+0xcc/0x150 [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190 [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) TCP wq(s) LLLLl< TCP wq(h) +++h+< l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_output.c:1977 tcp_xmit_retransmit_queue+0x388/0x420() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0132438>] ? vprintk+0x308/0x320 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c03f5ffc>] ? tcp_mark_head_lost+0xcc/0x150 [<c03fdf38>] tcp_xmit_retransmit_queue+0x388/0x420 [<c03f6c27>] tcp_fastretrans_alert+0x287/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c01317da>] ? print_oops_end_marker+0x2a/0x30 [<c0131849>] ? warn_on_slowpath+0x59/0x80 [<c0132438>] ? vprintk+0x308/0x320 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03f5ffc>] ? tcp_mark_head_lost+0xcc/0x150 [<c03fdea4>] tcp_xmit_retransmit_queue+0x2f4/0x420 [<c03f6c27>] tcp_fastretrans_alert+0x287/0x700 [<c03f7e63>] tcp_ack+0x1b3/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) TCP wq(s) LLLLl< TCP wq(h) +++h+< l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_output.c:1467 tcp_write_xmit+0x288/0x290() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c0186bf1>] ? check_pad_bytes+0x61/0x80 [<c0186dbd>] ? check_object+0xad/0x1e0 [<c0155539>] ? mark_held_locks+0x39/0x80 [<c01880ac>] ? __slab_free+0xac/0x150 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130 [<c03c1578>] ? kfree_skbmem+0x98/0xc0 [<c03fd128>] tcp_write_xmit+0x288/0x290 [<c03c1578>] ? kfree_skbmem+0x98/0xc0 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80 [<c03fa23e>] tcp_rcv_established+0x40e/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c01317da>] ? print_oops_end_marker+0x2a/0x30 [<c0131849>] ? warn_on_slowpath+0x59/0x80 [<c0186af4>] ? check_bytes_and_report+0x24/0xc0 [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 [<c0186bf1>] ? check_pad_bytes+0x61/0x80 [<c0186dbd>] ? check_object+0xad/0x1e0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130 [<c03c1578>] ? kfree_skbmem+0x98/0xc0 [<c03fcf03>] tcp_write_xmit+0x63/0x290 [<c03c1578>] ? kfree_skbmem+0x98/0xc0 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80 [<c03fa23e>] tcp_rcv_established+0x40e/0x710 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c04021fb>] tcp_v4_rcv+0x5db/0x660 [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03e6147>] ip_local_deliver+0xb7/0xc0 [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 [<c03c01d8>] ? sock_def_readable+0x48/0xa0 [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 [<c03e669f>] ip_rcv+0x18f/0x290 [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 [<c03c8da6>] netif_receive_skb+0x2b6/0x330 [<c03c8c17>] ? netif_receive_skb+0x127/0x330 [<c03c8ea3>] ? process_backlog+0x83/0x100 [<c03c8eae>] process_backlog+0x8e/0x100 [<c03c90bc>] net_rx_action+0x13c/0x230 [<c03c8fd9>] ? net_rx_action+0x59/0x230 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c01078b1>] do_IRQ+0x41/0x80 [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 [<c01059ca>] common_interrupt+0x2e/0x34 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) TCP wq(s) LLLLl< TCP wq(h) +++h+< l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:1767 tcp_enter_frto+0x28c/0x2a0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0156244>] ? __lock_acquire+0x1e4/0x730 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c03f58dc>] tcp_enter_frto+0x28c/0x2a0 [<c03ff612>] tcp_retransmit_timer+0x102/0x400 [<c043675b>] ? _spin_lock+0x4b/0x80 [<c03ff924>] ? tcp_write_timer+0x14/0x110 [<c03ffa03>] tcp_write_timer+0xf3/0x110 [<c013b95b>] run_timer_softirq+0xbb/0x190 [<c03ff910>] ? tcp_write_timer+0x0/0x110 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c011a91d>] smp_apic_timer_interrupt+0x2d/0x40 [<c0105aab>] apic_timer_interrupt+0x33/0x38 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c01317da>] ? print_oops_end_marker+0x2a/0x30 [<c0131849>] ? warn_on_slowpath+0x59/0x80 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c03f5748>] tcp_enter_frto+0xf8/0x2a0 [<c03ff612>] tcp_retransmit_timer+0x102/0x400 [<c043675b>] ? _spin_lock+0x4b/0x80 [<c03ff924>] ? tcp_write_timer+0x14/0x110 [<c03ffa03>] tcp_write_timer+0xf3/0x110 [<c013b95b>] run_timer_softirq+0xbb/0x190 [<c03ff910>] ? tcp_write_timer+0x0/0x110 [<c0136f63>] __do_softirq+0x93/0x120 [<c013706a>] do_softirq+0x7a/0x80 [<c0137135>] irq_exit+0x65/0x90 [<c011a91d>] smp_apic_timer_interrupt+0x2d/0x40 [<c0105aab>] apic_timer_interrupt+0x33/0x38 [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 [<c01033a0>] ? mwait_idle+0x0/0x20 [<c01033b2>] mwait_idle+0x12/0x20 [<c0103141>] cpu_idle+0x61/0x110 [<c04339fd>] rest_init+0x5d/0x60 [<c05a47fa>] start_kernel+0x1fa/0x260 [<c05a4190>] ? unknown_bootoption+0x0/0x130 ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) TCP wq(s) LLLLl< TCP wq(h) +++h+< l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_output.c:1467 tcp_write_xmit+0x288/0x290() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 2987, comm: ssmtp Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0156244>] ? __lock_acquire+0x1e4/0x730 [<c0155567>] ? mark_held_locks+0x67/0x80 [<c03fd128>] tcp_write_xmit+0x288/0x290 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80 [<c03f1130>] tcp_sendmsg+0x760/0xb40 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c03bb85c>] sock_sendmsg+0xdc/0x110 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0147420>] ? autoremove_wake_function+0x0/0x50 [<c0156244>] ? __lock_acquire+0x1e4/0x730 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c03bb5ff>] ? sockfd_lookup_light+0x2f/0x70 [<c03bcfe1>] sys_sendto+0xd1/0x120 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c015323d>] ? put_lock_stats+0xd/0x30 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c01413f1>] ? sys_times+0x81/0xe0 [<c03bd062>] sys_send+0x32/0x40 [<c03bd8e6>] sys_socketcall+0x136/0x230 [<c010501b>] ? restore_nocheck+0x12/0x15 [<c0104fba>] syscall_call+0x7/0xb ======================= ---[ end trace 14b601818e6903ac ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 2987, comm: ssmtp Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c01317da>] ? print_oops_end_marker+0x2a/0x30 [<c0131849>] ? warn_on_slowpath+0x59/0x80 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0156244>] ? __lock_acquire+0x1e4/0x730 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c03fcf03>] tcp_write_xmit+0x63/0x290 [<c03fd147>] __tcp_push_pending_frames+0x17/0x80 [<c03f1130>] tcp_sendmsg+0x760/0xb40 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c03bb85c>] sock_sendmsg+0xdc/0x110 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0147420>] ? autoremove_wake_function+0x0/0x50 [<c0156244>] ? __lock_acquire+0x1e4/0x730 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c03bb5ff>] ? sockfd_lookup_light+0x2f/0x70 [<c03bcfe1>] sys_sendto+0xd1/0x120 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c015323d>] ? put_lock_stats+0xd/0x30 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c01413f1>] ? sys_times+0x81/0xe0 [<c03bd062>] sys_send+0x32/0x40 [<c03bd8e6>] sys_socketcall+0x136/0x230 [<c010501b>] ? restore_nocheck+0x12/0x15 [<c0104fba>] syscall_call+0x7/0xb ======================= ---[ end trace 14b601818e6903ac ]--- P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) TCP wq(s) LLLLl< TCP wq(h) +++h+< l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_input.c:2855 tcp_clean_rtx_queue+0x552/0x5b0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 2987, comm: ssmtp Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c0186dbd>] ? check_object+0xad/0x1e0 [<c0155539>] ? mark_held_locks+0x39/0x80 [<c01880ac>] ? __slab_free+0xac/0x150 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130 [<c03c1534>] ? kfree_skbmem+0x54/0xc0 [<c01881e7>] ? kmem_cache_free+0x97/0xe0 [<c03c1534>] ? kfree_skbmem+0x54/0xc0 [<c03c1534>] ? kfree_skbmem+0x54/0xc0 [<c03c1534>] ? kfree_skbmem+0x54/0xc0 [<c03f7782>] tcp_clean_rtx_queue+0x552/0x5b0 [<c03f7dd2>] tcp_ack+0x122/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0147420>] ? autoremove_wake_function+0x0/0x50 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c03f17e3>] tcp_prequeue_process+0x73/0xa0 [<c03f1dae>] tcp_recvmsg+0x42e/0x830 [<c03c0873>] sock_common_recvmsg+0x43/0x60 [<c03bbacf>] sock_recvmsg+0xef/0x120 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0147420>] ? autoremove_wake_function+0x0/0x50 [<c03bb5ff>] ? sockfd_lookup_light+0x2f/0x70 [<c03bd0f7>] sys_recvfrom+0x87/0xf0 [<c015323d>] ? put_lock_stats+0xd/0x30 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c015323d>] ? put_lock_stats+0xd/0x30 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c013601f>] ? do_setitimer+0x1ef/0x2e0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03bd192>] sys_recv+0x32/0x40 [<c03bd905>] sys_socketcall+0x155/0x230 [<c010501b>] ? restore_nocheck+0x12/0x15 [<c0104fba>] syscall_call+0x7/0xb ======================= ---[ end trace 14b601818e6903ac ]--- ------------[ cut here ]------------ WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib i2c_i801 processor button intel_agp dcdbas pcspkr agpgart Pid: 2987, comm: ssmtp Not tainted 2.6.24-rc8-mm1 #8 [<c0132100>] ? have_callable_console+0x20/0x30 [<c0131844>] warn_on_slowpath+0x54/0x80 [<c01317da>] ? print_oops_end_marker+0x2a/0x30 [<c0131849>] ? warn_on_slowpath+0x59/0x80 [<c0186dbd>] ? check_object+0xad/0x1e0 [<c0155539>] ? mark_held_locks+0x39/0x80 [<c01880ac>] ? __slab_free+0xac/0x150 [<c015561c>] ? trace_hardirqs_on+0x7c/0x130 [<c03c1534>] ? kfree_skbmem+0x54/0xc0 [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 [<c01881e7>] ? kmem_cache_free+0x97/0xe0 [<c03c1534>] ? kfree_skbmem+0x54/0xc0 [<c03c1534>] ? kfree_skbmem+0x54/0xc0 [<c03f7509>] tcp_clean_rtx_queue+0x2d9/0x5b0 [<c03f7dd2>] tcp_ack+0x122/0x3a0 [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 [<c0147420>] ? autoremove_wake_function+0x0/0x50 [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 [<c03f17e3>] tcp_prequeue_process+0x73/0xa0 [<c03f1dae>] tcp_recvmsg+0x42e/0x830 [<c03c0873>] sock_common_recvmsg+0x43/0x60 [<c03bbacf>] sock_recvmsg+0xef/0x120 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0154982>] ? validate_chain+0x1d2/0x320 [<c0147420>] ? autoremove_wake_function+0x0/0x50 [<c03bb5ff>] ? sockfd_lookup_light+0x2f/0x70 [<c03bd0f7>] sys_recvfrom+0x87/0xf0 [<c015323d>] ? put_lock_stats+0xd/0x30 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c010af95>] ? native_sched_clock+0x85/0xe0 [<c015323d>] ? put_lock_stats+0xd/0x30 [<c01532c0>] ? lock_release_holdtime+0x60/0x80 [<c013601f>] ? do_setitimer+0x1ef/0x2e0 [<c0156b97>] ? __lock_release+0x47/0x70 [<c03bd192>] sys_recv+0x32/0x40 [<c03bd905>] sys_socketcall+0x155/0x230 [<c010501b>] ? restore_nocheck+0x12/0x15 [<c0104fba>] syscall_call+0x7/0xb ======================= ---[ end trace 14b601818e6903ac ]--- P: 4 L: 4 vs 4 S: 0 vs 3 w: 2044792269-2044796616 (0) TCP wq(s) LLLl< TCP wq(h) ++h+< l4 s3 f0 p4 seq: su2044792269 hs2044795029 sn2044796616 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-24 2:42 ` Dave Young @ 2008-01-24 9:54 ` Ilpo Järvinen 2008-01-24 10:24 ` Ilpo Järvinen ` (3 more replies) 0 siblings, 4 replies; 18+ messages in thread From: Ilpo Järvinen @ 2008-01-24 9:54 UTC (permalink / raw) To: Dave Young, Kamalesh Babulal, Krishna Kumar2, Denys Fedoryshchenko, David Miller Cc: LKML, Netdev, Andrew Morton [-- Attachment #1: Type: TEXT/PLAIN, Size: 8416 bytes --] On Thu, 24 Jan 2008, Dave Young wrote: Hi Dave (& others), > Thanks. Thanks a lot, I was first to ignore all these because they occurred with newreno, but looked again... :-/ > New warning trigged with your debug patch: This was probably with the earlier one I sent to you because there's still this case remaining which itself is valid: > P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) ...snip... this is still ok state (S+L <= P): > P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0) > TCP wq(s) < > TCP wq(h) +++h+< > l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 > ------------[ cut here ]------------ > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150() > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 > [<c0132100>] ? have_callable_console+0x20/0x30 > [<c0131844>] warn_on_slowpath+0x54/0x80 > [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 > [<c0132438>] ? vprintk+0x308/0x320 > [<c0132438>] ? vprintk+0x308/0x320 > [<c0132438>] ? vprintk+0x308/0x320 > [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 > [<c03f6052>] tcp_mark_head_lost+0x122/0x150 > [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190 > [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700 > [<c03f7e63>] tcp_ack+0x1b3/0x3a0 > [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 > [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 > [<c04021fb>] tcp_v4_rcv+0x5db/0x660 > [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > [<c0156b97>] ? __lock_release+0x47/0x70 > [<c03e6147>] ip_local_deliver+0xb7/0xc0 > [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 > [<c03c01d8>] ? sock_def_readable+0x48/0xa0 > [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 > [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 > [<c03e669f>] ip_rcv+0x18f/0x290 > [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 > [<c03c8da6>] netif_receive_skb+0x2b6/0x330 > [<c03c8c17>] ? netif_receive_skb+0x127/0x330 > [<c03c8ea3>] ? process_backlog+0x83/0x100 > [<c03c8eae>] process_backlog+0x8e/0x100 > [<c03c90bc>] net_rx_action+0x13c/0x230 > [<c03c8fd9>] ? net_rx_action+0x59/0x230 > [<c0136f63>] __do_softirq+0x93/0x120 > [<c013706a>] do_softirq+0x7a/0x80 > [<c0137135>] irq_exit+0x65/0x90 > [<c01078b1>] do_IRQ+0x41/0x80 > [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 > [<c01059ca>] common_interrupt+0x2e/0x34 > [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 > [<c01033a0>] ? mwait_idle+0x0/0x20 > [<c01033b2>] mwait_idle+0x12/0x20 > [<c0103141>] cpu_idle+0x61/0x110 > [<c04339fd>] rest_init+0x5d/0x60 > [<c05a47fa>] start_kernel+0x1fa/0x260 > [<c05a4190>] ? unknown_bootoption+0x0/0x130 > ======================= > ---[ end trace 14b601818e6903ac ]--- > ------------[ cut here ]------------ > WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 > [<c0132100>] ? have_callable_console+0x20/0x30 > [<c0131844>] warn_on_slowpath+0x54/0x80 > [<c01317da>] ? print_oops_end_marker+0x2a/0x30 > [<c0131849>] ? warn_on_slowpath+0x59/0x80 > [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 > [<c0132438>] ? vprintk+0x308/0x320 > [<c0132438>] ? vprintk+0x308/0x320 > [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 > [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 > [<c03f5ffc>] tcp_mark_head_lost+0xcc/0x150 > [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190 > [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700 > [<c03f7e63>] tcp_ack+0x1b3/0x3a0 > [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 > [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 > [<c04021fb>] tcp_v4_rcv+0x5db/0x660 > [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > [<c0156b97>] ? __lock_release+0x47/0x70 > [<c03e6147>] ip_local_deliver+0xb7/0xc0 > [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 > [<c03c01d8>] ? sock_def_readable+0x48/0xa0 > [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 > [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 > [<c03e669f>] ip_rcv+0x18f/0x290 > [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 > [<c03c8da6>] netif_receive_skb+0x2b6/0x330 > [<c03c8c17>] ? netif_receive_skb+0x127/0x330 > [<c03c8ea3>] ? process_backlog+0x83/0x100 > [<c03c8eae>] process_backlog+0x8e/0x100 > [<c03c90bc>] net_rx_action+0x13c/0x230 > [<c03c8fd9>] ? net_rx_action+0x59/0x230 > [<c0136f63>] __do_softirq+0x93/0x120 > [<c013706a>] do_softirq+0x7a/0x80 > [<c0137135>] irq_exit+0x65/0x90 > [<c01078b1>] do_IRQ+0x41/0x80 > [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 > [<c01059ca>] common_interrupt+0x2e/0x34 > [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 > [<c01033a0>] ? mwait_idle+0x0/0x20 > [<c01033b2>] mwait_idle+0x12/0x20 > [<c0103141>] cpu_idle+0x61/0x110 > [<c04339fd>] rest_init+0x5d/0x60 > [<c05a47fa>] start_kernel+0x1fa/0x260 > [<c05a4190>] ? unknown_bootoption+0x0/0x130 > ======================= > ---[ end trace 14b601818e6903ac ]--- ...But this no longer is, and even more, L: 5 is not valid state at this point all (should only happen if we went to RTO but it would reset S to zero with newreno): > P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) > TCP wq(s) LLLLl< > TCP wq(h) +++h+< > l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 Surprisingly, it was the first time the WARN_ON for left_out returned correct location. This also explains why the patch I sent to Krishna didn't print anything (it didn't end up into printing because I forgot to add L+S>P check into to the state checking if). ...so please, could you (others than Denys) try this patch, it should solve the issue. And Denys, could you confirm (and if necessary double check) that the kernel you saw this similar problem with is the pure Linus' mainline, i.e., without any net-2.6.25 or mm bits please, if so, that problem persists. And anyway, there were some fackets_out related problems reported as well and this doesn't help for that but I think I've lost track of who was seeing it due to large number of reports :-), could somebody refresh my memory because I currently don't have time to dig it up from archives (at least on this week). -- i. -- [PATCH] [TCP]: NewReno must count every skb while marking losses NewReno should add cnt per skb (as with FACK) instead of depending on SACKED_ACKED bits which won't be set with it at all. Effectively, NewReno should always exists after the first iteration anyway (or immediately if there's already head in lost_out. This was fixed earlier in net-2.6.25 but got reverted among other stuff and I didn't notice that this is still necessary (actually wasn't even considering this case while trying to figure out the reports because I lived with different kind of code than it in reality was). This should solve the WARN_ONs in TCP code that as a result of this triggered multiple times in every place we check for this invariant. Special thanks to Dave Young <hidave.darkstar@gmail.com> and Krishna Kumar2 <krkumar2@in.ibm.com> for trying with my debug patches. Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> --- net/ipv4/tcp_input.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 295490e..aa409a5 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -2156,7 +2156,7 @@ static void tcp_mark_head_lost(struct sock *sk, int packets, int fast_rexmit) tp->lost_skb_hint = skb; tp->lost_cnt_hint = cnt; - if (tcp_is_fack(tp) || + if (tcp_is_fack(tp) || tcp_is_reno(tp) || (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED)) cnt += tcp_skb_pcount(skb); -- 1.5.2.2 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-24 9:54 ` Ilpo Järvinen @ 2008-01-24 10:24 ` Ilpo Järvinen 2008-01-24 10:46 ` Krishna Kumar2 ` (2 subsequent siblings) 3 siblings, 0 replies; 18+ messages in thread From: Ilpo Järvinen @ 2008-01-24 10:24 UTC (permalink / raw) To: Netdev Cc: Dave Young, Kamalesh Babulal, Krishna Kumar2, Denys Fedoryshchenko, David Miller, LKML, Andrew Morton [-- Attachment #1: Type: TEXT/PLAIN, Size: 10779 bytes --] On Thu, 24 Jan 2008, Ilpo Järvinen wrote: > And anyway, there were some fackets_out related > problems reported as well and this doesn't help for that but I think I've > lost track of who was seeing it due to large number of reports :-), could > somebody refresh my memory because I currently don't have time to dig it > up from archives (at least on this week). Here's the updated debug patch for net-2.6.25/mm for tracking fackets_out inconsistencies (it won't work for trees which don't include net-2.6.25, mm does of course :-)). I hope I got it into good shape this time to avoid spurious stacktraces but still maintaining 100% accuracy, but it's not a simple oneliner so I might have missed something... :-) I'd suggest that people trying with this first apply the newreno fix of the previous mail to avoid already-fixed case from triggering. -- i. -- [PATCH] [TCP]: debug S+L (for net-2.5.26 / mm, incompatible with mainline) --- include/net/tcp.h | 5 ++- net/ipv4/tcp_input.c | 18 +++++++- net/ipv4/tcp_ipv4.c | 127 +++++++++++++++++++++++++++++++++++++++++++++++++ net/ipv4/tcp_output.c | 23 +++++++-- 4 files changed, 165 insertions(+), 8 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 7de4ea3..552aa71 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -272,6 +272,9 @@ DECLARE_SNMP_STAT(struct tcp_mib, tcp_statistics); #define TCP_ADD_STATS_BH(field, val) SNMP_ADD_STATS_BH(tcp_statistics, field, val) #define TCP_ADD_STATS_USER(field, val) SNMP_ADD_STATS_USER(tcp_statistics, field, val) +extern void tcp_print_queue(struct sock *sk); +extern void tcp_verify_wq(struct sock *sk); + extern void tcp_v4_err(struct sk_buff *skb, u32); extern void tcp_shutdown (struct sock *sk, int how); @@ -768,7 +771,7 @@ static inline __u32 tcp_current_ssthresh(const struct sock *sk) } /* Use define here intentionally to get WARN_ON location shown at the caller */ -#define tcp_verify_left_out(tp) WARN_ON(tcp_left_out(tp) > tp->packets_out) +#define tcp_verify_left_out(tp) tcp_verify_wq((struct sock *)tp) extern void tcp_enter_cwr(struct sock *sk, const int set_ssthresh); extern __u32 tcp_init_cwnd(struct tcp_sock *tp, struct dst_entry *dst); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 19c449f..c897c93 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -1426,8 +1426,10 @@ tcp_sacktag_write_queue(struct sock *sk, struct sk_buff *ack_skb, int first_sack_index; if (!tp->sacked_out) { - if (WARN_ON(tp->fackets_out)) + if (WARN_ON(tp->fackets_out)) { + tcp_verify_left_out(tp); tp->fackets_out = 0; + } tcp_highest_sack_reset(sk); } @@ -2136,6 +2138,8 @@ static void tcp_mark_head_lost(struct sock *sk, int packets, int fast_rexmit) struct sk_buff *skb; int cnt; + tcp_verify_left_out(tp); + BUG_TRAP(packets <= tp->packets_out); if (tp->lost_skb_hint) { skb = tp->lost_skb_hint; @@ -2501,6 +2505,8 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag) (tcp_fackets_out(tp) > tp->reordering)); int fast_rexmit = 0; + tcp_verify_left_out(tp); + if (WARN_ON(!tp->packets_out && tp->sacked_out)) tp->sacked_out = 0; if (WARN_ON(!tp->sacked_out && tp->fackets_out)) @@ -2645,6 +2651,10 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked, int flag) if (do_lost || (tcp_is_fack(tp) && tcp_head_timedout(sk))) tcp_update_scoreboard(sk, fast_rexmit); tcp_cwnd_down(sk, flag); + + WARN_ON(tcp_write_queue_head(sk) == NULL); + WARN_ON(!tp->packets_out); + tcp_xmit_retransmit_queue(sk); } @@ -2848,6 +2858,8 @@ static int tcp_clean_rtx_queue(struct sock *sk, int prior_fackets) tcp_clear_all_retrans_hints(tp); } + tcp_verify_left_out(tp); + if (skb && (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED)) flag |= FLAG_SACK_RENEGING; @@ -3175,6 +3187,8 @@ static int tcp_ack(struct sock *sk, struct sk_buff *skb, int flag) prior_fackets = tp->fackets_out; prior_in_flight = tcp_packets_in_flight(tp); + tcp_verify_left_out(tp); + if (!(flag & FLAG_SLOWPATH) && after(ack, prior_snd_una)) { /* Window is constant, pure forward advance. * No more checks are required. @@ -3237,6 +3251,8 @@ static int tcp_ack(struct sock *sk, struct sk_buff *skb, int flag) if ((flag & FLAG_FORWARD_PROGRESS) || !(flag & FLAG_NOT_DUP)) dst_confirm(sk->sk_dst_cache); + tcp_verify_left_out(tp); + return 1; no_queue: diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 9aea88b..e6e3ad5 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -108,6 +108,133 @@ struct inet_hashinfo __cacheline_aligned tcp_hashinfo = { .lhash_wait = __WAIT_QUEUE_HEAD_INITIALIZER(tcp_hashinfo.lhash_wait), }; +void tcp_print_queue(struct sock *sk) +{ + struct tcp_sock *tp = tcp_sk(sk); + struct sk_buff *skb; + char s[50+1]; + char h[50+1]; + int idx = 0; + int i; + + i = 0; + tcp_for_write_queue(skb, sk) { + if (skb == tcp_send_head(sk)) + printk(KERN_ERR "head %u %p\n", i, skb); + else + printk(KERN_ERR "skb %u %p\n", i, skb); + i++; + } + + tcp_for_write_queue(skb, sk) { + if (skb == tcp_send_head(sk)) + break; + + for (i = 0; i < tcp_skb_pcount(skb); i++) { + if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) { + s[idx] = 'S'; + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + s[idx] = 'B'; + + } else if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) { + s[idx] = 'L'; + } else { + s[idx] = ' '; + } + if (s[idx] != ' ' && skb->len < tp->mss_cache) + s[idx] += 'a' - 'A'; + + if (i == 0) { + if (skb == tcp_highest_sack(sk)) + h[idx] = 'h'; + else + h[idx] = '+'; + } else { + h[idx] = '-'; + } + + if (++idx >= 50) { + s[idx] = 0; + h[idx] = 0; + printk(KERN_ERR "TCP wq(s) %s\n", s); + printk(KERN_ERR "TCP wq(h) %s\n", h); + idx = 0; + } + } + } + if (idx) { + s[idx] = '<'; + s[idx+1] = 0; + h[idx] = '<'; + h[idx+1] = 0; + printk(KERN_ERR "TCP wq(s) %s\n", s); + printk(KERN_ERR "TCP wq(h) %s\n", h); + } + printk(KERN_ERR "l%u s%u f%u p%u seq: su%u hs%u sn%u\n", + tp->lost_out, tp->sacked_out, tp->fackets_out, tp->packets_out, + tp->snd_una, tcp_highest_sack_seq(tp), tp->snd_nxt); +} + +void tcp_verify_wq(struct sock *sk) +{ + struct tcp_sock *tp = tcp_sk(sk); + u32 lost = 0; + u32 sacked = 0; + u32 packets = 0; + u32 fackets = 0; + int hs_valid = 0; + struct sk_buff *skb; + + tcp_for_write_queue(skb, sk) { + if (skb == tcp_send_head(sk)) + break; + + if ((fackets == packets) && (skb == tp->highest_sack)) + hs_valid = 1; + + packets += tcp_skb_pcount(skb); + + if (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED) { + sacked += tcp_skb_pcount(skb); + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + printk(KERN_ERR "Sacked bitmap S+L: %u %u-%u/%u\n", + TCP_SKB_CB(skb)->sacked, + TCP_SKB_CB(skb)->end_seq - tp->snd_una, + TCP_SKB_CB(skb)->seq - tp->snd_una, + tp->snd_una); + fackets = packets; + hs_valid = 0; + } + if (TCP_SKB_CB(skb)->sacked & TCPCB_LOST) + lost += tcp_skb_pcount(skb); + } + + if ((fackets == packets) && (tp->highest_sack == tcp_send_head(sk))) + hs_valid = 1; + + if ((lost != tp->lost_out) || + (tcp_is_sack(tp) && (sacked != tp->sacked_out)) || + ((sacked || (tcp_is_sack(tp) && tp->sacked_out)) && !hs_valid) || + (packets != tp->packets_out) || + (fackets != tp->fackets_out) || + tcp_left_out(tp) > tp->packets_out) { + printk(KERN_ERR "P: %u L: %u vs %u S: %u vs %u F: %u vs %u w: %u-%u (%u)\n", + tp->packets_out, + lost, tp->lost_out, + sacked, tp->sacked_out, + fackets, tp->fackets_out, + tp->snd_una, tp->snd_nxt, + tp->rx_opt.sack_ok); + tcp_print_queue(sk); + } + + WARN_ON(lost != tp->lost_out); + WARN_ON(tcp_is_sack(tp) && (sacked != tp->sacked_out)); + WARN_ON(packets != tp->packets_out); + WARN_ON(fackets != tp->fackets_out); + WARN_ON(tcp_left_out(tp) > tp->packets_out); +} + static int tcp_v4_get_port(struct sock *sk, unsigned short snum) { return inet_csk_get_port(&tcp_hashinfo, sk, snum, diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index 89f0188..8fb6628 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -779,10 +779,9 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len, tp->lost_out -= diff; /* Adjust Reno SACK estimate. */ - if (tcp_is_reno(tp) && diff > 0) { + if (tcp_is_reno(tp) && diff > 0) tcp_dec_pcount_approx_int(&tp->sacked_out, diff); - tcp_verify_left_out(tp); - } + tcp_adjust_fackets_out(sk, skb, diff); } @@ -790,6 +789,8 @@ int tcp_fragment(struct sock *sk, struct sk_buff *skb, u32 len, skb_header_release(buff); tcp_insert_write_queue_after(skb, buff, sk); + tcp_verify_left_out(tp); + return 0; } @@ -1463,6 +1464,7 @@ static int tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle) } else if (result > 0) { sent_pkts = 1; } + tcp_verify_left_out(tp); while ((skb = tcp_send_head(sk))) { unsigned int limit; @@ -1764,6 +1766,7 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *skb, tcp_clear_retrans_hints_partial(tp); sk_wmem_free_skb(sk, next_skb); + tcp_verify_left_out(tp); } /* Do a simple retransmit without using the backoff mechanisms in @@ -1795,13 +1798,13 @@ void tcp_simple_retransmit(struct sock *sk) } } + tcp_verify_left_out(tp); + tcp_clear_all_retrans_hints(tp); if (!lost) return; - tcp_verify_left_out(tp); - /* Don't muck with the congestion window here. * Reason is that we do not increase amount of _data_ * in network, but units changed and effective @@ -1888,6 +1891,8 @@ int tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb) tcp_init_nondata_skb(skb, TCP_SKB_CB(skb)->end_seq - 1, TCP_SKB_CB(skb)->flags); skb->ip_summed = CHECKSUM_NONE; + + tcp_verify_left_out(tp); } } @@ -1970,8 +1975,10 @@ void tcp_xmit_retransmit_queue(struct sock *sk) * packet to be MSS sized and all the * packet counting works out. */ - if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) + if (tcp_packets_in_flight(tp) >= tp->snd_cwnd) { + tcp_verify_left_out(tp); return; + } if (sacked & TCPCB_LOST) { if (!(sacked & (TCPCB_SACKED_ACKED|TCPCB_SACKED_RETRANS))) { @@ -1997,6 +2004,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk) } } + tcp_verify_left_out(tp); + /* OK, demanded retransmission is finished. */ /* Forward retransmissions are possible only during Recovery. */ @@ -2054,6 +2063,8 @@ void tcp_xmit_retransmit_queue(struct sock *sk) NET_INC_STATS_BH(LINUX_MIB_TCPFORWARDRETRANS); } + + tcp_verify_left_out(tp); } /* Send a fin. The caller locks the socket for us. This cannot be -- 1.5.2.2 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-24 9:54 ` Ilpo Järvinen 2008-01-24 10:24 ` Ilpo Järvinen @ 2008-01-24 10:46 ` Krishna Kumar2 2008-01-24 13:11 ` Kamalesh Babulal 2008-01-25 1:30 ` Dave Young 3 siblings, 0 replies; 18+ messages in thread From: Krishna Kumar2 @ 2008-01-24 10:46 UTC (permalink / raw) To: Ilpo Järvinen Cc: Andrew Morton, David Miller, Denys Fedoryshchenko, Dave Young, Kamalesh Babulal, LKML, Netdev, netdev-owner Hi Ilpo, I have tried parallel iperfs with this patch and don't get any more warnings. I will run overnight to be sure. thanks, - KK netdev-owner@vger.kernel.org wrote on 01/24/2008 03:24:18 PM: > On Thu, 24 Jan 2008, Dave Young wrote: > > Hi Dave (& others), > > > Thanks. > > Thanks a lot, I was first to ignore all these because they occurred > with newreno, but looked again... :-/ > > > New warning trigged with your debug patch: > > This was probably with the earlier one I sent to you because there's still > this case remaining which itself is valid: > > > P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) > > ...snip... this is still ok state (S+L <= P): > > > P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0) > > TCP wq(s) < > > TCP wq(h) +++h+< > > l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 > > ------------[ cut here ]------------ > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150() > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event > > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse > > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg > > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib > > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 > > [<c0132100>] ? have_callable_console+0x20/0x30 > > [<c0131844>] warn_on_slowpath+0x54/0x80 > > [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 > > [<c03f6052>] tcp_mark_head_lost+0x122/0x150 > > [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190 > > [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700 > > [<c03f7e63>] tcp_ack+0x1b3/0x3a0 > > [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 > > [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 > > [<c04021fb>] tcp_v4_rcv+0x5db/0x660 > > [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 > > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > > [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 > > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > > [<c0156b97>] ? __lock_release+0x47/0x70 > > [<c03e6147>] ip_local_deliver+0xb7/0xc0 > > [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 > > [<c03c01d8>] ? sock_def_readable+0x48/0xa0 > > [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 > > [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 > > [<c03e669f>] ip_rcv+0x18f/0x290 > > [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 > > [<c03c8da6>] netif_receive_skb+0x2b6/0x330 > > [<c03c8c17>] ? netif_receive_skb+0x127/0x330 > > [<c03c8ea3>] ? process_backlog+0x83/0x100 > > [<c03c8eae>] process_backlog+0x8e/0x100 > > [<c03c90bc>] net_rx_action+0x13c/0x230 > > [<c03c8fd9>] ? net_rx_action+0x59/0x230 > > [<c0136f63>] __do_softirq+0x93/0x120 > > [<c013706a>] do_softirq+0x7a/0x80 > > [<c0137135>] irq_exit+0x65/0x90 > > [<c01078b1>] do_IRQ+0x41/0x80 > > [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 > > [<c01059ca>] common_interrupt+0x2e/0x34 > > [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 > > [<c01033a0>] ? mwait_idle+0x0/0x20 > > [<c01033b2>] mwait_idle+0x12/0x20 > > [<c0103141>] cpu_idle+0x61/0x110 > > [<c04339fd>] rest_init+0x5d/0x60 > > [<c05a47fa>] start_kernel+0x1fa/0x260 > > [<c05a4190>] ? unknown_bootoption+0x0/0x130 > > ======================= > > ---[ end trace 14b601818e6903ac ]--- > > ------------[ cut here ]------------ > > WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event > > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse > > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg > > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib > > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 > > [<c0132100>] ? have_callable_console+0x20/0x30 > > [<c0131844>] warn_on_slowpath+0x54/0x80 > > [<c01317da>] ? print_oops_end_marker+0x2a/0x30 > > [<c0131849>] ? warn_on_slowpath+0x59/0x80 > > [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 > > [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 > > [<c03f5ffc>] tcp_mark_head_lost+0xcc/0x150 > > [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190 > > [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700 > > [<c03f7e63>] tcp_ack+0x1b3/0x3a0 > > [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 > > [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 > > [<c04021fb>] tcp_v4_rcv+0x5db/0x660 > > [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 > > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > > [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 > > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > > [<c0156b97>] ? __lock_release+0x47/0x70 > > [<c03e6147>] ip_local_deliver+0xb7/0xc0 > > [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 > > [<c03c01d8>] ? sock_def_readable+0x48/0xa0 > > [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 > > [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 > > [<c03e669f>] ip_rcv+0x18f/0x290 > > [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 > > [<c03c8da6>] netif_receive_skb+0x2b6/0x330 > > [<c03c8c17>] ? netif_receive_skb+0x127/0x330 > > [<c03c8ea3>] ? process_backlog+0x83/0x100 > > [<c03c8eae>] process_backlog+0x8e/0x100 > > [<c03c90bc>] net_rx_action+0x13c/0x230 > > [<c03c8fd9>] ? net_rx_action+0x59/0x230 > > [<c0136f63>] __do_softirq+0x93/0x120 > > [<c013706a>] do_softirq+0x7a/0x80 > > [<c0137135>] irq_exit+0x65/0x90 > > [<c01078b1>] do_IRQ+0x41/0x80 > > [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 > > [<c01059ca>] common_interrupt+0x2e/0x34 > > [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 > > [<c01033a0>] ? mwait_idle+0x0/0x20 > > [<c01033b2>] mwait_idle+0x12/0x20 > > [<c0103141>] cpu_idle+0x61/0x110 > > [<c04339fd>] rest_init+0x5d/0x60 > > [<c05a47fa>] start_kernel+0x1fa/0x260 > > [<c05a4190>] ? unknown_bootoption+0x0/0x130 > > ======================= > > ---[ end trace 14b601818e6903ac ]--- > > ...But this no longer is, and even more, L: 5 is not valid state at this > point all (should only happen if we went to RTO but it would reset S to > zero with newreno): > > > P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) > > TCP wq(s) LLLLl< > > TCP wq(h) +++h+< > > l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 > > Surprisingly, it was the first time the WARN_ON for left_out returned > correct location. This also explains why the patch I sent to Krishna > didn't print anything (it didn't end up into printing because I forgot > to add L+S>P check into to the state checking if). > > ...so please, could you (others than Denys) try this patch, it should > solve the issue. And Denys, could you confirm (and if necessary double > check) that the kernel you saw this similar problem with is the pure > Linus' mainline, i.e., without any net-2.6.25 or mm bits please, if so, > that problem persists. And anyway, there were some fackets_out related > problems reported as well and this doesn't help for that but I think I've > lost track of who was seeing it due to large number of reports :-), could > somebody refresh my memory because I currently don't have time to dig it > up from archives (at least on this week). > > > -- > i. > > -- > [PATCH] [TCP]: NewReno must count every skb while marking losses > > NewReno should add cnt per skb (as with FACK) instead of depending > on SACKED_ACKED bits which won't be set with it at all. > Effectively, NewReno should always exists after the first > iteration anyway (or immediately if there's already head in > lost_out. > > This was fixed earlier in net-2.6.25 but got reverted among other > stuff and I didn't notice that this is still necessary (actually > wasn't even considering this case while trying to figure out the > reports because I lived with different kind of code than it in > reality was). > > This should solve the WARN_ONs in TCP code that as a result of > this triggered multiple times in every place we check for this > invariant. > > Special thanks to Dave Young <hidave.darkstar@gmail.com> and > Krishna Kumar2 <krkumar2@in.ibm.com> for trying with my debug > patches. > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> > --- > net/ipv4/tcp_input.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c > index 295490e..aa409a5 100644 > --- a/net/ipv4/tcp_input.c > +++ b/net/ipv4/tcp_input.c > @@ -2156,7 +2156,7 @@ static void tcp_mark_head_lost(struct sock *sk, int > packets, int fast_rexmit) > tp->lost_skb_hint = skb; > tp->lost_cnt_hint = cnt; > > - if (tcp_is_fack(tp) || > + if (tcp_is_fack(tp) || tcp_is_reno(tp) || > (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED)) > cnt += tcp_skb_pcount(skb); > > -- > 1.5.2.2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-24 9:54 ` Ilpo Järvinen 2008-01-24 10:24 ` Ilpo Järvinen 2008-01-24 10:46 ` Krishna Kumar2 @ 2008-01-24 13:11 ` Kamalesh Babulal 2008-01-25 1:30 ` Dave Young 3 siblings, 0 replies; 18+ messages in thread From: Kamalesh Babulal @ 2008-01-24 13:11 UTC (permalink / raw) To: Ilpo Järvinen Cc: Dave Young, Kamalesh Babulal, Krishna Kumar2, Denys Fedoryshchenko, David Miller, LKML, Netdev, Andrew Morton On Thu, Jan 24, 2008 at 11:54:18AM +0200, Ilpo Järvinen wrote: > On Thu, 24 Jan 2008, Dave Young wrote: > > Hi Dave (& others), > > > Thanks. > > Thanks a lot, I was first to ignore all these because they occurred > with newreno, but looked again... :-/ > > > New warning trigged with your debug patch: > > This was probably with the earlier one I sent to you because there's still > this case remaining which itself is valid: > > > P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) > > ...snip... this is still ok state (S+L <= P): > > > P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0) > > TCP wq(s) < > > TCP wq(h) +++h+< > > l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 > > ------------[ cut here ]------------ > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150() > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event > > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse > > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg > > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib > > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 > > [<c0132100>] ? have_callable_console+0x20/0x30 > > [<c0131844>] warn_on_slowpath+0x54/0x80 > > [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 > > [<c0132438>] ? vprintk+0x308/0x320 . . <snip> . . > > ---[ end trace 14b601818e6903ac ]--- > > ...But this no longer is, and even more, L: 5 is not valid state at this > point all (should only happen if we went to RTO but it would reset S to > zero with newreno): > > > P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) > > TCP wq(s) LLLLl< > > TCP wq(h) +++h+< > > l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 > > Surprisingly, it was the first time the WARN_ON for left_out returned > correct location. This also explains why the patch I sent to Krishna > didn't print anything (it didn't end up into printing because I forgot > to add L+S>P check into to the state checking if). > > ...so please, could you (others than Denys) try this patch, it should > solve the issue. And Denys, could you confirm (and if necessary double > check) that the kernel you saw this similar problem with is the pure > Linus' mainline, i.e., without any net-2.6.25 or mm bits please, if so, > that problem persists. And anyway, there were some fackets_out related > problems reported as well and this doesn't help for that but I think I've > lost track of who was seeing it due to large number of reports :-), could > somebody refresh my memory because I currently don't have time to dig it > up from archives (at least on this week). > > > -- > i. > > -- > [PATCH] [TCP]: NewReno must count every skb while marking losses > > NewReno should add cnt per skb (as with FACK) instead of depending > on SACKED_ACKED bits which won't be set with it at all. > Effectively, NewReno should always exists after the first > iteration anyway (or immediately if there's already head in > lost_out. > > This was fixed earlier in net-2.6.25 but got reverted among other > stuff and I didn't notice that this is still necessary (actually > wasn't even considering this case while trying to figure out the > reports because I lived with different kind of code than it in > reality was). > > This should solve the WARN_ONs in TCP code that as a result of > this triggered multiple times in every place we check for this > invariant. > > Special thanks to Dave Young <hidave.darkstar@gmail.com> and > Krishna Kumar2 <krkumar2@in.ibm.com> for trying with my debug > patches. Hi, Thanks, after applying the patch the warning is not seen. Tested-by: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com> > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> > --- > net/ipv4/tcp_input.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c > index 295490e..aa409a5 100644 > --- a/net/ipv4/tcp_input.c > +++ b/net/ipv4/tcp_input.c > @@ -2156,7 +2156,7 @@ static void tcp_mark_head_lost(struct sock *sk, int packets, int fast_rexmit) > tp->lost_skb_hint = skb; > tp->lost_cnt_hint = cnt; > > - if (tcp_is_fack(tp) || > + if (tcp_is_fack(tp) || tcp_is_reno(tp) || > (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED)) > cnt += tcp_skb_pcount(skb); > > -- > 1.5.2.2 -- Thanks & Regards, Kamalesh Babulal, Linux Technology Center, IBM, ISTL. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: 2.6.24-rc8-mm1 : net tcp_input.c warnings 2008-01-24 9:54 ` Ilpo Järvinen ` (2 preceding siblings ...) 2008-01-24 13:11 ` Kamalesh Babulal @ 2008-01-25 1:30 ` Dave Young 3 siblings, 0 replies; 18+ messages in thread From: Dave Young @ 2008-01-25 1:30 UTC (permalink / raw) To: Ilpo Järvinen Cc: Kamalesh Babulal, Krishna Kumar2, Denys Fedoryshchenko, David Miller, LKML, Netdev, Andrew Morton On Jan 24, 2008 5:54 PM, Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> wrote: > On Thu, 24 Jan 2008, Dave Young wrote: > > Hi Dave (& others), > > > Thanks. > > Thanks a lot, I was first to ignore all these because they occurred > with newreno, but looked again... :-/ > > > New warning trigged with your debug patch: > > This was probably with the earlier one I sent to you because there's still > this case remaining which itself is valid: > > > P: 5 L: 0 vs 0 S: 0 vs 1 w: 2044790889-2044796616 (0) > > ...snip... this is still ok state (S+L <= P): > > > > P: 5 L: 0 vs 0 S: 0 vs 3 w: 2044790889-2044796616 (0) > > TCP wq(s) < > > TCP wq(h) +++h+< > > l0 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 > > ------------[ cut here ]------------ > > WARNING: at net/ipv4/tcp_input.c:2169 tcp_mark_head_lost+0x122/0x150() > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event > > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse > > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg > > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib > > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 > > [<c0132100>] ? have_callable_console+0x20/0x30 > > [<c0131844>] warn_on_slowpath+0x54/0x80 > > [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 > > [<c03f6052>] tcp_mark_head_lost+0x122/0x150 > > [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190 > > [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700 > > [<c03f7e63>] tcp_ack+0x1b3/0x3a0 > > [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 > > [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 > > [<c04021fb>] tcp_v4_rcv+0x5db/0x660 > > [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 > > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > > [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 > > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > > [<c0156b97>] ? __lock_release+0x47/0x70 > > [<c03e6147>] ip_local_deliver+0xb7/0xc0 > > [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 > > [<c03c01d8>] ? sock_def_readable+0x48/0xa0 > > [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 > > [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 > > [<c03e669f>] ip_rcv+0x18f/0x290 > > [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 > > [<c03c8da6>] netif_receive_skb+0x2b6/0x330 > > [<c03c8c17>] ? netif_receive_skb+0x127/0x330 > > [<c03c8ea3>] ? process_backlog+0x83/0x100 > > [<c03c8eae>] process_backlog+0x8e/0x100 > > [<c03c90bc>] net_rx_action+0x13c/0x230 > > [<c03c8fd9>] ? net_rx_action+0x59/0x230 > > [<c0136f63>] __do_softirq+0x93/0x120 > > [<c013706a>] do_softirq+0x7a/0x80 > > [<c0137135>] irq_exit+0x65/0x90 > > [<c01078b1>] do_IRQ+0x41/0x80 > > [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 > > [<c01059ca>] common_interrupt+0x2e/0x34 > > [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 > > [<c01033a0>] ? mwait_idle+0x0/0x20 > > [<c01033b2>] mwait_idle+0x12/0x20 > > [<c0103141>] cpu_idle+0x61/0x110 > > [<c04339fd>] rest_init+0x5d/0x60 > > [<c05a47fa>] start_kernel+0x1fa/0x260 > > [<c05a4190>] ? unknown_bootoption+0x0/0x130 > > ======================= > > ---[ end trace 14b601818e6903ac ]--- > > ------------[ cut here ]------------ > > WARNING: at net/ipv4/tcp_ipv4.c:197 tcp_verify_wq+0x1b6/0x1c0() > > Modules linked in: snd_seq_dummy snd_seq_oss snd_seq_midi_event > > snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss eeprom e100 psmouse > > snd_hda_intel snd_pcm snd_timer btusb bluetooth serio_raw snd 3c59x sg > > evdev thermal soundcore rtc_cmos snd_page_alloc rtc_core rtc_lib > > i2c_i801 processor button intel_agp dcdbas pcspkr agpgart > > Pid: 0, comm: swapper Not tainted 2.6.24-rc8-mm1 #8 > > [<c0132100>] ? have_callable_console+0x20/0x30 > > [<c0131844>] warn_on_slowpath+0x54/0x80 > > [<c01317da>] ? print_oops_end_marker+0x2a/0x30 > > [<c0131849>] ? warn_on_slowpath+0x59/0x80 > > [<c03ffe54>] ? tcp_print_queue+0x1a4/0x230 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c0132438>] ? vprintk+0x308/0x320 > > [<c0400096>] tcp_verify_wq+0x1b6/0x1c0 > > [<c03ffff6>] ? tcp_verify_wq+0x116/0x1c0 > > [<c03f5ffc>] tcp_mark_head_lost+0xcc/0x150 > > [<c03f60ca>] tcp_update_scoreboard+0x4a/0x190 > > [<c03f6e7a>] tcp_fastretrans_alert+0x4da/0x700 > > [<c03f7e63>] tcp_ack+0x1b3/0x3a0 > > [<c03fa21b>] tcp_rcv_established+0x3eb/0x710 > > [<c0401c05>] tcp_v4_do_rcv+0xe5/0x100 > > [<c04021fb>] tcp_v4_rcv+0x5db/0x660 > > [<c0401fa7>] ? tcp_v4_rcv+0x387/0x660 > > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > > [<c03e5f44>] ip_local_deliver_finish+0x84/0x1d0 > > [<c03e5eed>] ? ip_local_deliver_finish+0x2d/0x1d0 > > [<c0156b97>] ? __lock_release+0x47/0x70 > > [<c03e6147>] ip_local_deliver+0xb7/0xc0 > > [<c03e6202>] ip_rcv_finish+0xb2/0x3c0 > > [<c03c01d8>] ? sock_def_readable+0x48/0xa0 > > [<c03be061>] ? sock_queue_rcv_skb+0xb1/0x1a0 > > [<c03be0a7>] ? sock_queue_rcv_skb+0xf7/0x1a0 > > [<c03e669f>] ip_rcv+0x18f/0x290 > > [<c042fb10>] ? packet_rcv_spkt+0xd0/0x130 > > [<c03c8da6>] netif_receive_skb+0x2b6/0x330 > > [<c03c8c17>] ? netif_receive_skb+0x127/0x330 > > [<c03c8ea3>] ? process_backlog+0x83/0x100 > > [<c03c8eae>] process_backlog+0x8e/0x100 > > [<c03c90bc>] net_rx_action+0x13c/0x230 > > [<c03c8fd9>] ? net_rx_action+0x59/0x230 > > [<c0136f63>] __do_softirq+0x93/0x120 > > [<c013706a>] do_softirq+0x7a/0x80 > > [<c0137135>] irq_exit+0x65/0x90 > > [<c01078b1>] do_IRQ+0x41/0x80 > > [<c0155659>] ? trace_hardirqs_on+0xb9/0x130 > > [<c01059ca>] common_interrupt+0x2e/0x34 > > [<c0103390>] ? mwait_idle_with_hints+0x40/0x50 > > [<c01033a0>] ? mwait_idle+0x0/0x20 > > [<c01033b2>] mwait_idle+0x12/0x20 > > [<c0103141>] cpu_idle+0x61/0x110 > > [<c04339fd>] rest_init+0x5d/0x60 > > [<c05a47fa>] start_kernel+0x1fa/0x260 > > [<c05a4190>] ? unknown_bootoption+0x0/0x130 > > ======================= > > ---[ end trace 14b601818e6903ac ]--- > > ...But this no longer is, and even more, L: 5 is not valid state at this > point all (should only happen if we went to RTO but it would reset S to > zero with newreno): > > > P: 5 L: 5 vs 5 S: 0 vs 3 w: 2044790889-2044796616 (0) > > TCP wq(s) LLLLl< > > TCP wq(h) +++h+< > > l5 s3 f0 p5 seq: su2044790889 hs2044795029 sn2044796616 > > Surprisingly, it was the first time the WARN_ON for left_out returned > correct location. This also explains why the patch I sent to Krishna > didn't print anything (it didn't end up into printing because I forgot > to add L+S>P check into to the state checking if). > > ...so please, could you (others than Denys) try this patch, it should > solve the issue. And Denys, could you confirm (and if necessary double > check) that the kernel you saw this similar problem with is the pure > Linus' mainline, i.e., without any net-2.6.25 or mm bits please, if so, > that problem persists. And anyway, there were some fackets_out related > problems reported as well and this doesn't help for that but I think I've > lost track of who was seeing it due to large number of reports :-), could > somebody refresh my memory because I currently don't have time to dig it > up from archives (at least on this week). Thanks, applied on my test kernel, if there's no warnings I will just mute. > > > -- > i. > > -- > [PATCH] [TCP]: NewReno must count every skb while marking losses > > NewReno should add cnt per skb (as with FACK) instead of depending > on SACKED_ACKED bits which won't be set with it at all. > Effectively, NewReno should always exists after the first > iteration anyway (or immediately if there's already head in > lost_out. > > This was fixed earlier in net-2.6.25 but got reverted among other > stuff and I didn't notice that this is still necessary (actually > wasn't even considering this case while trying to figure out the > reports because I lived with different kind of code than it in > reality was). > > This should solve the WARN_ONs in TCP code that as a result of > this triggered multiple times in every place we check for this > invariant. > > Special thanks to Dave Young <hidave.darkstar@gmail.com> and > Krishna Kumar2 <krkumar2@in.ibm.com> for trying with my debug > patches. > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> > --- > net/ipv4/tcp_input.c | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c > index 295490e..aa409a5 100644 > --- a/net/ipv4/tcp_input.c > +++ b/net/ipv4/tcp_input.c > @@ -2156,7 +2156,7 @@ static void tcp_mark_head_lost(struct sock *sk, int packets, int fast_rexmit) > tp->lost_skb_hint = skb; > tp->lost_cnt_hint = cnt; > > - if (tcp_is_fack(tp) || > + if (tcp_is_fack(tp) || tcp_is_reno(tp) || > (TCP_SKB_CB(skb)->sacked & TCPCB_SACKED_ACKED)) > cnt += tcp_skb_pcount(skb); > > -- > 1.5.2.2 > ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2008-01-25 1:30 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2008-01-21 7:55 2.6.24-rc8-mm1 : net tcp_input.c warnings Dave Young 2008-01-21 21:14 ` Ilpo Järvinen 2008-01-22 4:37 ` Dave Young 2008-01-22 9:09 ` Dave Young 2008-01-22 9:18 ` Dave Young 2008-01-22 10:47 ` Ilpo Järvinen 2008-01-23 1:44 ` Dave Young 2008-01-23 3:01 ` David Miller 2008-01-23 7:41 ` Ilpo Järvinen 2008-01-23 7:44 ` Dave Young 2008-01-23 8:41 ` Ilpo Järvinen 2008-01-23 11:01 ` Ilpo Järvinen 2008-01-24 2:42 ` Dave Young 2008-01-24 9:54 ` Ilpo Järvinen 2008-01-24 10:24 ` Ilpo Järvinen 2008-01-24 10:46 ` Krishna Kumar2 2008-01-24 13:11 ` Kamalesh Babulal 2008-01-25 1:30 ` Dave Young
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).