netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Fwd: 3.10.0 network trace
       [not found] <CA+rTEUPfP8ouJHbaFrO=-zktg2D06kYgXOq-LnmyKbbEzj-TmA@mail.gmail.com>
@ 2013-07-12 20:44 ` Michael Sterrett
  2013-07-12 21:18   ` Yuchung Cheng
  0 siblings, 1 reply; 10+ messages in thread
From: Michael Sterrett @ 2013-07-12 20:44 UTC (permalink / raw)
  To: netdev

Upgraded to 3.10.0 and started getting these appearing in the log:

[21893.282786] ------------[ cut here ]------------
[21893.282802] WARNING: at net/ipv4/tcp_input.c:2776
tcp_fastretrans_alert+0xbac/0xbf0()
[21893.282805] Modules linked in:
[21893.282813] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0 #1
[21893.282816] Hardware name: ECS IC780M-A/IC780M-A, BIOS 080015  04/19/2010
[21893.282819]  ffffffff815186b2 0000000000000000 ffffffff810317ec
ffff88000a5ef500
[21893.282825]  0000000000000000 0000000000000007 0000000000004120
0000000000000001
[21893.282830]  ffffffff814647cc 0000000700000009 0000000000000000
2c81743c00000000
[21893.282835] Call Trace:
[21893.282838]  <IRQ>  [<ffffffff815186b2>] ? dump_stack+0xd/0x17
[21893.282854]  [<ffffffff810317ec>] ? warn_slowpath_common+0x6c/0xa0
[21893.282860]  [<ffffffff814647cc>] ? tcp_fastretrans_alert+0xbac/0xbf0
[21893.282865]  [<ffffffff81465257>] ? tcp_ack+0x997/0xd60
[21893.282872]  [<ffffffff81465aa1>] ? tcp_rcv_established+0xf1/0x640
[21893.282878]  [<ffffffff8146eb9e>] ? tcp_v4_do_rcv+0x1fe/0x4b0
[21893.282885]  [<ffffffff81434d9e>] ? nf_iterate+0x8e/0xc0
[21893.282890]  [<ffffffff81003cfc>] ? do_IRQ+0x5c/0xd0
[21893.282896]  [<ffffffff814711d7>] ? tcp_v4_rcv+0x757/0x780
[21893.282901]  [<ffffffff81434e47>] ? nf_hook_slow+0x77/0x150
[21893.282907]  [<ffffffff8144dfb0>] ? ip_rcv_finish+0x2a0/0x2a0
[21893.282914]  [<ffffffff8144e057>] ? ip_local_deliver_finish+0xa7/0x1e0
[21893.282921]  [<ffffffff8140c7cf>] ? __netif_receive_skb_core+0x49f/0x690
[21893.282926]  [<ffffffff8140cbb4>] ? netif_receive_skb+0x24/0x80
[21893.282931]  [<ffffffff8140e368>] ? napi_gro_receive+0xc8/0x100
[21893.282937]  [<ffffffff812ffecc>] ? rtl8169_poll+0x2cc/0x600
[21893.282942]  [<ffffffff8140ce29>] ? net_rx_action+0x79/0x130
[21893.282948]  [<ffffffff810382e8>] ? __do_softirq+0xc8/0x1a0
[21893.282953]  [<ffffffff81038516>] ? irq_exit+0x96/0xc0
[21893.282957]  [<ffffffff81003cfc>] ? do_IRQ+0x5c/0xd0
[21893.282963]  [<ffffffff8151caea>] ? common_interrupt+0x6a/0x6a
[21893.282965]  <EOI>  [<ffffffff8100a0a0>] ?
arch_remove_reservations+0x130/0x130
[21893.282978]  [<ffffffff8100a0a2>] ? default_idle+0x2/0x10
[21893.282983]  [<ffffffff810627f9>] ? cpu_startup_entry+0x79/0x160
[21893.282989]  [<ffffffff81732d54>] ? start_kernel+0x325/0x330
[21893.282993]  [<ffffffff81732848>] ? repair_env_string+0x58/0x58
[21893.282997] ---[ end trace d2be5719b9c30b65 ]---

If this is is already a known issue please let me know.  Otherwise,
I'm happy to provide any additional information that might be helpful
in tracking it down.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-07-12 20:44 ` Fwd: 3.10.0 network trace Michael Sterrett
@ 2013-07-12 21:18   ` Yuchung Cheng
  2013-07-12 21:33     ` Michael Sterrett
  0 siblings, 1 reply; 10+ messages in thread
From: Yuchung Cheng @ 2013-07-12 21:18 UTC (permalink / raw)
  To: Michael Sterrett; +Cc: netdev

On Fri, Jul 12, 2013 at 1:44 PM, Michael Sterrett <michael@sterretts.net> wrote:
> Upgraded to 3.10.0 and started getting these appearing in the log:
>
> [21893.282786] ------------[ cut here ]------------
> [21893.282802] WARNING: at net/ipv4/tcp_input.c:2776
> tcp_fastretrans_alert+0xbac/0xbf0()
> [21893.282805] Modules linked in:
> [21893.282813] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0 #1
> [21893.282816] Hardware name: ECS IC780M-A/IC780M-A, BIOS 080015  04/19/2010
> [21893.282819]  ffffffff815186b2 0000000000000000 ffffffff810317ec
> ffff88000a5ef500
> [21893.282825]  0000000000000000 0000000000000007 0000000000004120
> 0000000000000001
> [21893.282830]  ffffffff814647cc 0000000700000009 0000000000000000
> 2c81743c00000000
> [21893.282835] Call Trace:
> [21893.282838]  <IRQ>  [<ffffffff815186b2>] ? dump_stack+0xd/0x17
> [21893.282854]  [<ffffffff810317ec>] ? warn_slowpath_common+0x6c/0xa0
> [21893.282860]  [<ffffffff814647cc>] ? tcp_fastretrans_alert+0xbac/0xbf0
> [21893.282865]  [<ffffffff81465257>] ? tcp_ack+0x997/0xd60
> [21893.282872]  [<ffffffff81465aa1>] ? tcp_rcv_established+0xf1/0x640
> [21893.282878]  [<ffffffff8146eb9e>] ? tcp_v4_do_rcv+0x1fe/0x4b0
> [21893.282885]  [<ffffffff81434d9e>] ? nf_iterate+0x8e/0xc0
> [21893.282890]  [<ffffffff81003cfc>] ? do_IRQ+0x5c/0xd0
> [21893.282896]  [<ffffffff814711d7>] ? tcp_v4_rcv+0x757/0x780
> [21893.282901]  [<ffffffff81434e47>] ? nf_hook_slow+0x77/0x150
> [21893.282907]  [<ffffffff8144dfb0>] ? ip_rcv_finish+0x2a0/0x2a0
> [21893.282914]  [<ffffffff8144e057>] ? ip_local_deliver_finish+0xa7/0x1e0
> [21893.282921]  [<ffffffff8140c7cf>] ? __netif_receive_skb_core+0x49f/0x690
> [21893.282926]  [<ffffffff8140cbb4>] ? netif_receive_skb+0x24/0x80
> [21893.282931]  [<ffffffff8140e368>] ? napi_gro_receive+0xc8/0x100
> [21893.282937]  [<ffffffff812ffecc>] ? rtl8169_poll+0x2cc/0x600
> [21893.282942]  [<ffffffff8140ce29>] ? net_rx_action+0x79/0x130
> [21893.282948]  [<ffffffff810382e8>] ? __do_softirq+0xc8/0x1a0
> [21893.282953]  [<ffffffff81038516>] ? irq_exit+0x96/0xc0
> [21893.282957]  [<ffffffff81003cfc>] ? do_IRQ+0x5c/0xd0
> [21893.282963]  [<ffffffff8151caea>] ? common_interrupt+0x6a/0x6a
> [21893.282965]  <EOI>  [<ffffffff8100a0a0>] ?
> arch_remove_reservations+0x130/0x130
> [21893.282978]  [<ffffffff8100a0a2>] ? default_idle+0x2/0x10
> [21893.282983]  [<ffffffff810627f9>] ? cpu_startup_entry+0x79/0x160
> [21893.282989]  [<ffffffff81732d54>] ? start_kernel+0x325/0x330
> [21893.282993]  [<ffffffff81732848>] ? repair_env_string+0x58/0x58
> [21893.282997] ---[ end trace d2be5719b9c30b65 ]---
>
> If this is is already a known issue please let me know.  Otherwise,
> I'm happy to provide any additional information that might be helpful
> in tracking it down.
can you send the output of 'sysctl -a'.
one related fix is commit 93b174ad71b08e504c2cf6e8a58ecce778b77a40



> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-07-12 21:18   ` Yuchung Cheng
@ 2013-07-12 21:33     ` Michael Sterrett
  2013-07-12 21:39       ` Yuchung Cheng
  0 siblings, 1 reply; 10+ messages in thread
From: Michael Sterrett @ 2013-07-12 21:33 UTC (permalink / raw)
  To: Yuchung Cheng; +Cc: netdev

[-- Attachment #1: Type: text/plain, Size: 57 bytes --]

Thanks for the reply.  The sysctl -a output is attached.

[-- Attachment #2: sysctl-a --]
[-- Type: application/octet-stream, Size: 28172 bytes --]

abi.vsyscall32 = 1
debug.exception-trace = 1
dev.cdrom.autoclose = 1
dev.cdrom.autoeject = 0
dev.cdrom.check_media = 0
dev.cdrom.debug = 0
dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17
dev.cdrom.info = 
dev.cdrom.info = drive name:	
dev.cdrom.info = drive speed:	
dev.cdrom.info = drive # of slots:
dev.cdrom.info = Can close tray:	
dev.cdrom.info = Can open tray:	
dev.cdrom.info = Can lock tray:	
dev.cdrom.info = Can change speed:
dev.cdrom.info = Can select disk:
dev.cdrom.info = Can read multisession:
dev.cdrom.info = Can read MCN:	
dev.cdrom.info = Reports media changed:
dev.cdrom.info = Can play audio:	
dev.cdrom.info = Can write CD-R:	
dev.cdrom.info = Can write CD-RW:
dev.cdrom.info = Can read DVD:	
dev.cdrom.info = Can write DVD-R:
dev.cdrom.info = Can write DVD-RAM:
dev.cdrom.info = Can read MRW:	
dev.cdrom.info = Can write MRW:	
dev.cdrom.info = Can write RAM:	
dev.cdrom.info = 
dev.cdrom.info = 
dev.cdrom.lock = 1
dev.hpet.max-user-freq = 64
dev.raid.speed_limit_max = 200000
dev.raid.speed_limit_min = 1000
dev.scsi.logging_level = 0
fs.aio-max-nr = 65536
fs.aio-nr = 0
fs.dentry-state = 6950	4020	45	0	0	0
fs.dir-notify-enable = 1
fs.epoll.max_user_watches = 209080
fs.file-max = 102048
fs.file-nr = 1568	0	102048
fs.inode-nr = 10000	3641
fs.inode-state = 10000	3641	0	0	0	0	0
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 8192
fs.lease-break-time = 45
fs.leases-enable = 1
fs.mqueue.msg_default = 10
fs.mqueue.msg_max = 10
fs.mqueue.msgsize_default = 8192
fs.mqueue.msgsize_max = 8192
fs.mqueue.queues_max = 256
fs.nfs.nfs_congestion_kb = 32320
fs.nfs.nfs_mountpoint_timeout = 500
fs.nfs.nlm_grace_period = 0
fs.nfs.nlm_tcpport = 0
fs.nfs.nlm_timeout = 10
fs.nfs.nlm_udpport = 0
fs.nfs.nsm_local_state = 81
fs.nfs.nsm_use_hostnames = 0
fs.nr_open = 1048576
fs.overflowgid = 65534
fs.overflowuid = 65534
fs.pipe-max-size = 1048576
fs.protected_hardlinks = 0
fs.protected_symlinks = 0
fs.suid_dumpable = 0
kernel.acct = 4	2	30
kernel.auto_msgmni = 1
kernel.blk_iopoll = 1
kernel.bootloader_type = 113
kernel.bootloader_version = 1
kernel.cad_pid = 1
kernel.cap_last_cap = 36
kernel.compat-log = 1
kernel.core_pattern = core
kernel.core_pipe_limit = 0
kernel.core_uses_pid = 0
kernel.ctrl-alt-del = 0
kernel.dmesg_restrict = 0
kernel.domainname = yp.sterrettnet
kernel.hostname = eyes
kernel.hotplug = 
kernel.io_delay_type = 0
kernel.kptr_restrict = 0
kernel.kstack_depth_to_print = 12
kernel.max_lock_depth = 1024
kernel.modprobe = /sbin/modprobe
kernel.modules_disabled = 0
kernel.msgmax = 8192
kernel.msgmnb = 16384
kernel.msgmni = 1993
kernel.ngroups_max = 65536
kernel.osrelease = 3.10.0
kernel.ostype = Linux
kernel.overflowgid = 65534
kernel.overflowuid = 65534
kernel.panic = 0
kernel.panic_on_io_nmi = 0
kernel.panic_on_oops = 0
kernel.panic_on_unrecovered_nmi = 0
kernel.perf_event_max_sample_rate = 100000
kernel.perf_event_mlock_kb = 516
kernel.perf_event_paranoid = 1
kernel.pid_max = 32768
kernel.poweroff_cmd = /sbin/poweroff
kernel.print-fatal-signals = 0
kernel.printk = 1	4	1	7
kernel.printk_delay = 0
kernel.printk_ratelimit = 5
kernel.printk_ratelimit_burst = 10
kernel.pty.max = 4096
kernel.pty.nr = 1
kernel.pty.reserve = 1024
kernel.random.boot_id = 9cc64c2b-66ba-49a3-a0bf-e3080f97efdb
kernel.random.entropy_avail = 3968
kernel.random.poolsize = 4096
kernel.random.read_wakeup_threshold = 64
kernel.random.uuid = 993b1af4-4653-4dc4-b300-228834e09423
kernel.random.write_wakeup_threshold = 128
kernel.randomize_va_space = 2
kernel.real-root-dev = 0
kernel.sched_child_runs_first = 0
kernel.sched_rr_timeslice_ms = 100
kernel.sched_rt_period_us = 1000000
kernel.sched_rt_runtime_us = 950000
kernel.sem = 250	32000	32	128
kernel.shm_rmid_forced = 0
kernel.shmall = 2097152
kernel.shmmax = 33554432
kernel.shmmni = 4096
kernel.tainted = 512
kernel.threads-max = 15951
kernel.unknown_nmi_panic = 0
kernel.usermodehelper.bset = 4294967295	4294967295
kernel.usermodehelper.inheritable = 4294967295	4294967295
kernel.version = #1 SMP Wed Jul 10 00:47:26 EDT 2013
net.core.dev_weight = 64
net.core.message_burst = 10
net.core.message_cost = 5
net.core.netdev_budget = 300
net.core.netdev_max_backlog = 1000
net.core.netdev_tstamp_prequeue = 1
net.core.optmem_max = 20480
net.core.rmem_default = 212992
net.core.rmem_max = 212992
net.core.rps_sock_flow_entries = 0
net.core.somaxconn = 128
net.core.warnings = 1
net.core.wmem_default = 212992
net.core.wmem_max = 212992
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1
net.ipv4.conf.all.accept_local = 0
net.ipv4.conf.all.accept_redirects = 1
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.arp_accept = 0
net.ipv4.conf.all.arp_announce = 0
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_notify = 0
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.disable_policy = 0
net.ipv4.conf.all.disable_xfrm = 0
net.ipv4.conf.all.force_igmp_version = 0
net.ipv4.conf.all.forwarding = 0
net.ipv4.conf.all.log_martians = 0
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.all.medium_id = 0
net.ipv4.conf.all.promote_secondaries = 0
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.proxy_arp_pvlan = 0
net.ipv4.conf.all.route_localnet = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.secure_redirects = 1
net.ipv4.conf.all.send_redirects = 1
net.ipv4.conf.all.shared_media = 1
net.ipv4.conf.all.src_valid_mark = 0
net.ipv4.conf.all.tag = 0
net.ipv4.conf.default.accept_local = 0
net.ipv4.conf.default.accept_redirects = 1
net.ipv4.conf.default.accept_source_route = 1
net.ipv4.conf.default.arp_accept = 0
net.ipv4.conf.default.arp_announce = 0
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.arp_ignore = 0
net.ipv4.conf.default.arp_notify = 0
net.ipv4.conf.default.bootp_relay = 0
net.ipv4.conf.default.disable_policy = 0
net.ipv4.conf.default.disable_xfrm = 0
net.ipv4.conf.default.force_igmp_version = 0
net.ipv4.conf.default.forwarding = 0
net.ipv4.conf.default.log_martians = 0
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.default.medium_id = 0
net.ipv4.conf.default.promote_secondaries = 0
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.conf.default.proxy_arp_pvlan = 0
net.ipv4.conf.default.route_localnet = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.secure_redirects = 1
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.default.shared_media = 1
net.ipv4.conf.default.src_valid_mark = 0
net.ipv4.conf.default.tag = 0
net.ipv4.conf.dummy0.accept_local = 0
net.ipv4.conf.dummy0.accept_redirects = 1
net.ipv4.conf.dummy0.accept_source_route = 1
net.ipv4.conf.dummy0.arp_accept = 0
net.ipv4.conf.dummy0.arp_announce = 0
net.ipv4.conf.dummy0.arp_filter = 0
net.ipv4.conf.dummy0.arp_ignore = 0
net.ipv4.conf.dummy0.arp_notify = 0
net.ipv4.conf.dummy0.bootp_relay = 0
net.ipv4.conf.dummy0.disable_policy = 0
net.ipv4.conf.dummy0.disable_xfrm = 0
net.ipv4.conf.dummy0.force_igmp_version = 0
net.ipv4.conf.dummy0.forwarding = 0
net.ipv4.conf.dummy0.log_martians = 0
net.ipv4.conf.dummy0.mc_forwarding = 0
net.ipv4.conf.dummy0.medium_id = 0
net.ipv4.conf.dummy0.promote_secondaries = 0
net.ipv4.conf.dummy0.proxy_arp = 0
net.ipv4.conf.dummy0.proxy_arp_pvlan = 0
net.ipv4.conf.dummy0.route_localnet = 0
net.ipv4.conf.dummy0.rp_filter = 1
net.ipv4.conf.dummy0.secure_redirects = 1
net.ipv4.conf.dummy0.send_redirects = 1
net.ipv4.conf.dummy0.shared_media = 1
net.ipv4.conf.dummy0.src_valid_mark = 0
net.ipv4.conf.dummy0.tag = 0
net.ipv4.conf.eth0.accept_local = 0
net.ipv4.conf.eth0.accept_redirects = 1
net.ipv4.conf.eth0.accept_source_route = 1
net.ipv4.conf.eth0.arp_accept = 0
net.ipv4.conf.eth0.arp_announce = 0
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.arp_ignore = 0
net.ipv4.conf.eth0.arp_notify = 0
net.ipv4.conf.eth0.bootp_relay = 0
net.ipv4.conf.eth0.disable_policy = 0
net.ipv4.conf.eth0.disable_xfrm = 0
net.ipv4.conf.eth0.force_igmp_version = 0
net.ipv4.conf.eth0.forwarding = 0
net.ipv4.conf.eth0.log_martians = 0
net.ipv4.conf.eth0.mc_forwarding = 0
net.ipv4.conf.eth0.medium_id = 0
net.ipv4.conf.eth0.promote_secondaries = 0
net.ipv4.conf.eth0.proxy_arp = 0
net.ipv4.conf.eth0.proxy_arp_pvlan = 0
net.ipv4.conf.eth0.route_localnet = 0
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.eth0.secure_redirects = 1
net.ipv4.conf.eth0.send_redirects = 1
net.ipv4.conf.eth0.shared_media = 1
net.ipv4.conf.eth0.src_valid_mark = 0
net.ipv4.conf.eth0.tag = 0
net.ipv4.conf.lo.accept_local = 0
net.ipv4.conf.lo.accept_redirects = 1
net.ipv4.conf.lo.accept_source_route = 1
net.ipv4.conf.lo.arp_accept = 0
net.ipv4.conf.lo.arp_announce = 0
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.arp_ignore = 0
net.ipv4.conf.lo.arp_notify = 0
net.ipv4.conf.lo.bootp_relay = 0
net.ipv4.conf.lo.disable_policy = 1
net.ipv4.conf.lo.disable_xfrm = 1
net.ipv4.conf.lo.force_igmp_version = 0
net.ipv4.conf.lo.forwarding = 0
net.ipv4.conf.lo.log_martians = 0
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.lo.medium_id = 0
net.ipv4.conf.lo.promote_secondaries = 0
net.ipv4.conf.lo.proxy_arp = 0
net.ipv4.conf.lo.proxy_arp_pvlan = 0
net.ipv4.conf.lo.route_localnet = 0
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.lo.secure_redirects = 1
net.ipv4.conf.lo.send_redirects = 1
net.ipv4.conf.lo.shared_media = 1
net.ipv4.conf.lo.src_valid_mark = 0
net.ipv4.conf.lo.tag = 0
net.ipv4.conf.sit0.accept_local = 0
net.ipv4.conf.sit0.accept_redirects = 1
net.ipv4.conf.sit0.accept_source_route = 1
net.ipv4.conf.sit0.arp_accept = 0
net.ipv4.conf.sit0.arp_announce = 0
net.ipv4.conf.sit0.arp_filter = 0
net.ipv4.conf.sit0.arp_ignore = 0
net.ipv4.conf.sit0.arp_notify = 0
net.ipv4.conf.sit0.bootp_relay = 0
net.ipv4.conf.sit0.disable_policy = 0
net.ipv4.conf.sit0.disable_xfrm = 0
net.ipv4.conf.sit0.force_igmp_version = 0
net.ipv4.conf.sit0.forwarding = 0
net.ipv4.conf.sit0.log_martians = 0
net.ipv4.conf.sit0.mc_forwarding = 0
net.ipv4.conf.sit0.medium_id = 0
net.ipv4.conf.sit0.promote_secondaries = 0
net.ipv4.conf.sit0.proxy_arp = 0
net.ipv4.conf.sit0.proxy_arp_pvlan = 0
net.ipv4.conf.sit0.route_localnet = 0
net.ipv4.conf.sit0.rp_filter = 1
net.ipv4.conf.sit0.secure_redirects = 1
net.ipv4.conf.sit0.send_redirects = 1
net.ipv4.conf.sit0.shared_media = 1
net.ipv4.conf.sit0.src_valid_mark = 0
net.ipv4.conf.sit0.tag = 0
net.ipv4.icmp_echo_ignore_all = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_errors_use_inbound_ifaddr = 0
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.icmp_ratelimit = 1000
net.ipv4.icmp_ratemask = 6168
net.ipv4.igmp_max_memberships = 20
net.ipv4.igmp_max_msf = 10
net.ipv4.inet_peer_maxttl = 600
net.ipv4.inet_peer_minttl = 120
net.ipv4.inet_peer_threshold = 65664
net.ipv4.ip_default_ttl = 64
net.ipv4.ip_dynaddr = 0
net.ipv4.ip_early_demux = 1
net.ipv4.ip_forward = 0
net.ipv4.ip_local_port_range = 32768	61000
net.ipv4.ip_local_reserved_ports = 
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.ip_nonlocal_bind = 0
net.ipv4.ipfrag_high_thresh = 4194304
net.ipv4.ipfrag_low_thresh = 3145728
net.ipv4.ipfrag_max_dist = 64
net.ipv4.ipfrag_secret_interval = 600
net.ipv4.ipfrag_time = 30
net.ipv4.neigh.default.anycast_delay = 100
net.ipv4.neigh.default.app_solicit = 0
net.ipv4.neigh.default.base_reachable_time_ms = 30000
net.ipv4.neigh.default.delay_first_probe_time = 5
net.ipv4.neigh.default.gc_interval = 30
net.ipv4.neigh.default.gc_stale_time = 60
net.ipv4.neigh.default.gc_thresh1 = 128
net.ipv4.neigh.default.gc_thresh2 = 512
net.ipv4.neigh.default.gc_thresh3 = 1024
net.ipv4.neigh.default.locktime = 100
net.ipv4.neigh.default.mcast_solicit = 3
net.ipv4.neigh.default.proxy_delay = 80
net.ipv4.neigh.default.proxy_qlen = 64
net.ipv4.neigh.default.retrans_time_ms = 1000
net.ipv4.neigh.default.ucast_solicit = 3
net.ipv4.neigh.default.unres_qlen = 31
net.ipv4.neigh.default.unres_qlen_bytes = 65536
net.ipv4.neigh.dummy0.anycast_delay = 100
net.ipv4.neigh.dummy0.app_solicit = 0
net.ipv4.neigh.dummy0.base_reachable_time_ms = 30000
net.ipv4.neigh.dummy0.delay_first_probe_time = 5
net.ipv4.neigh.dummy0.gc_stale_time = 60
net.ipv4.neigh.dummy0.locktime = 100
net.ipv4.neigh.dummy0.mcast_solicit = 3
net.ipv4.neigh.dummy0.proxy_delay = 80
net.ipv4.neigh.dummy0.proxy_qlen = 64
net.ipv4.neigh.dummy0.retrans_time_ms = 1000
net.ipv4.neigh.dummy0.ucast_solicit = 3
net.ipv4.neigh.dummy0.unres_qlen = 31
net.ipv4.neigh.dummy0.unres_qlen_bytes = 65536
net.ipv4.neigh.eth0.anycast_delay = 100
net.ipv4.neigh.eth0.app_solicit = 0
net.ipv4.neigh.eth0.base_reachable_time_ms = 30000
net.ipv4.neigh.eth0.delay_first_probe_time = 5
net.ipv4.neigh.eth0.gc_stale_time = 60
net.ipv4.neigh.eth0.locktime = 100
net.ipv4.neigh.eth0.mcast_solicit = 3
net.ipv4.neigh.eth0.proxy_delay = 80
net.ipv4.neigh.eth0.proxy_qlen = 64
net.ipv4.neigh.eth0.retrans_time_ms = 1000
net.ipv4.neigh.eth0.ucast_solicit = 3
net.ipv4.neigh.eth0.unres_qlen = 31
net.ipv4.neigh.eth0.unres_qlen_bytes = 65536
net.ipv4.neigh.lo.anycast_delay = 100
net.ipv4.neigh.lo.app_solicit = 0
net.ipv4.neigh.lo.base_reachable_time_ms = 30000
net.ipv4.neigh.lo.delay_first_probe_time = 5
net.ipv4.neigh.lo.gc_stale_time = 60
net.ipv4.neigh.lo.locktime = 100
net.ipv4.neigh.lo.mcast_solicit = 3
net.ipv4.neigh.lo.proxy_delay = 80
net.ipv4.neigh.lo.proxy_qlen = 64
net.ipv4.neigh.lo.retrans_time_ms = 1000
net.ipv4.neigh.lo.ucast_solicit = 3
net.ipv4.neigh.lo.unres_qlen = 31
net.ipv4.neigh.lo.unres_qlen_bytes = 65536
net.ipv4.neigh.sit0.anycast_delay = 100
net.ipv4.neigh.sit0.app_solicit = 0
net.ipv4.neigh.sit0.base_reachable_time_ms = 30000
net.ipv4.neigh.sit0.delay_first_probe_time = 5
net.ipv4.neigh.sit0.gc_stale_time = 60
net.ipv4.neigh.sit0.locktime = 100
net.ipv4.neigh.sit0.mcast_solicit = 3
net.ipv4.neigh.sit0.proxy_delay = 80
net.ipv4.neigh.sit0.proxy_qlen = 64
net.ipv4.neigh.sit0.retrans_time_ms = 1000
net.ipv4.neigh.sit0.ucast_solicit = 3
net.ipv4.neigh.sit0.unres_qlen = 31
net.ipv4.neigh.sit0.unres_qlen_bytes = 65536
net.ipv4.netfilter.ip_conntrack_buckets = 8192
net.ipv4.netfilter.ip_conntrack_checksum = 1
net.ipv4.netfilter.ip_conntrack_count = 165
net.ipv4.netfilter.ip_conntrack_generic_timeout = 600
net.ipv4.netfilter.ip_conntrack_icmp_timeout = 30
net.ipv4.netfilter.ip_conntrack_log_invalid = 0
net.ipv4.netfilter.ip_conntrack_max = 31900
net.ipv4.netfilter.ip_conntrack_tcp_be_liberal = 0
net.ipv4.netfilter.ip_conntrack_tcp_loose = 1
net.ipv4.netfilter.ip_conntrack_tcp_max_retrans = 3
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 432000
net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_last_ack = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_max_retrans = 300
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_recv = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_sent = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_sent2 = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 120
net.ipv4.netfilter.ip_conntrack_udp_timeout = 30
net.ipv4.netfilter.ip_conntrack_udp_timeout_stream = 180
net.ipv4.ping_group_range = 1	0
net.ipv4.route.error_burst = 5000
net.ipv4.route.error_cost = 1000
net.ipv4.route.gc_elasticity = 8
net.ipv4.route.gc_interval = 60
net.ipv4.route.gc_min_interval = 0
net.ipv4.route.gc_min_interval_ms = 500
net.ipv4.route.gc_thresh = -1
net.ipv4.route.gc_timeout = 300
net.ipv4.route.max_size = 2147483647
net.ipv4.route.min_adv_mss = 256
net.ipv4.route.min_pmtu = 552
net.ipv4.route.mtu_expires = 600
net.ipv4.route.redirect_load = 20
net.ipv4.route.redirect_number = 9
net.ipv4.route.redirect_silence = 20480
net.ipv4.tcp_abort_on_overflow = 0
net.ipv4.tcp_adv_win_scale = 1
net.ipv4.tcp_allowed_congestion_control = cubic reno
net.ipv4.tcp_app_win = 31
net.ipv4.tcp_available_congestion_control = cubic reno
net.ipv4.tcp_base_mss = 512
net.ipv4.tcp_challenge_ack_limit = 100
net.ipv4.tcp_congestion_control = cubic
net.ipv4.tcp_dsack = 1
net.ipv4.tcp_early_retrans = 3
net.ipv4.tcp_ecn = 2
net.ipv4.tcp_fack = 1
net.ipv4.tcp_fastopen = 0
net.ipv4.tcp_fastopen_key = c1a9388e-f9e19b5a-27b15630-7b5cd491
net.ipv4.tcp_fin_timeout = 60
net.ipv4.tcp_frto = 2
net.ipv4.tcp_keepalive_intvl = 75
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_time = 7200
net.ipv4.tcp_limit_output_bytes = 131072
net.ipv4.tcp_low_latency = 0
net.ipv4.tcp_max_orphans = 4096
net.ipv4.tcp_max_ssthresh = 0
net.ipv4.tcp_max_syn_backlog = 128
net.ipv4.tcp_max_tw_buckets = 4096
net.ipv4.tcp_mem = 23784	31713	47568
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_mtu_probing = 0
net.ipv4.tcp_no_metrics_save = 0
net.ipv4.tcp_orphan_retries = 0
net.ipv4.tcp_reordering = 3
net.ipv4.tcp_retrans_collapse = 1
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_rfc1337 = 0
net.ipv4.tcp_rmem = 4096	87380	6291456
net.ipv4.tcp_sack = 1
net.ipv4.tcp_slow_start_after_idle = 1
net.ipv4.tcp_stdurg = 0
net.ipv4.tcp_syn_retries = 6
net.ipv4.tcp_synack_retries = 5
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_thin_dupack = 0
net.ipv4.tcp_thin_linear_timeouts = 0
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_tso_win_divisor = 3
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 0
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_wmem = 4096	16384	4194304
net.ipv4.tcp_workaround_signed_windows = 0
net.ipv4.udp_mem = 23925	31902	47850
net.ipv4.udp_rmem_min = 4096
net.ipv4.udp_wmem_min = 4096
net.ipv4.xfrm4_gc_thresh = 1024
net.ipv6.bindv6only = 0
net.ipv6.conf.all.accept_dad = 1
net.ipv6.conf.all.accept_ra = 1
net.ipv6.conf.all.accept_ra_defrtr = 1
net.ipv6.conf.all.accept_ra_pinfo = 1
net.ipv6.conf.all.accept_redirects = 1
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.all.autoconf = 1
net.ipv6.conf.all.dad_transmits = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.all.force_mld_version = 0
net.ipv6.conf.all.force_tllao = 0
net.ipv6.conf.all.forwarding = 0
net.ipv6.conf.all.hop_limit = 64
net.ipv6.conf.all.max_addresses = 16
net.ipv6.conf.all.mtu = 1280
net.ipv6.conf.all.ndisc_notify = 0
net.ipv6.conf.all.proxy_ndp = 0
net.ipv6.conf.all.router_solicitation_delay = 1
net.ipv6.conf.all.router_solicitation_interval = 4
net.ipv6.conf.all.router_solicitations = 3
net.ipv6.conf.default.accept_dad = 1
net.ipv6.conf.default.accept_ra = 1
net.ipv6.conf.default.accept_ra_defrtr = 1
net.ipv6.conf.default.accept_ra_pinfo = 1
net.ipv6.conf.default.accept_redirects = 1
net.ipv6.conf.default.accept_source_route = 0
net.ipv6.conf.default.autoconf = 1
net.ipv6.conf.default.dad_transmits = 1
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.default.force_mld_version = 0
net.ipv6.conf.default.force_tllao = 0
net.ipv6.conf.default.forwarding = 0
net.ipv6.conf.default.hop_limit = 64
net.ipv6.conf.default.max_addresses = 16
net.ipv6.conf.default.mtu = 1280
net.ipv6.conf.default.ndisc_notify = 0
net.ipv6.conf.default.proxy_ndp = 0
net.ipv6.conf.default.router_solicitation_delay = 1
net.ipv6.conf.default.router_solicitation_interval = 4
net.ipv6.conf.default.router_solicitations = 3
net.ipv6.conf.dummy0.accept_dad = -1
net.ipv6.conf.dummy0.accept_ra = 1
net.ipv6.conf.dummy0.accept_ra_defrtr = 1
net.ipv6.conf.dummy0.accept_ra_pinfo = 1
net.ipv6.conf.dummy0.accept_redirects = 1
net.ipv6.conf.dummy0.accept_source_route = 0
net.ipv6.conf.dummy0.autoconf = 1
net.ipv6.conf.dummy0.dad_transmits = 1
net.ipv6.conf.dummy0.disable_ipv6 = 0
net.ipv6.conf.dummy0.force_mld_version = 0
net.ipv6.conf.dummy0.force_tllao = 0
net.ipv6.conf.dummy0.forwarding = 0
net.ipv6.conf.dummy0.hop_limit = 64
net.ipv6.conf.dummy0.max_addresses = 16
net.ipv6.conf.dummy0.mtu = 1500
net.ipv6.conf.dummy0.ndisc_notify = 0
net.ipv6.conf.dummy0.proxy_ndp = 0
net.ipv6.conf.dummy0.router_solicitation_delay = 1
net.ipv6.conf.dummy0.router_solicitation_interval = 4
net.ipv6.conf.dummy0.router_solicitations = 3
net.ipv6.conf.eth0.accept_dad = 1
net.ipv6.conf.eth0.accept_ra = 1
net.ipv6.conf.eth0.accept_ra_defrtr = 1
net.ipv6.conf.eth0.accept_ra_pinfo = 1
net.ipv6.conf.eth0.accept_redirects = 1
net.ipv6.conf.eth0.accept_source_route = 0
net.ipv6.conf.eth0.autoconf = 1
net.ipv6.conf.eth0.dad_transmits = 1
net.ipv6.conf.eth0.disable_ipv6 = 0
net.ipv6.conf.eth0.force_mld_version = 0
net.ipv6.conf.eth0.force_tllao = 0
net.ipv6.conf.eth0.forwarding = 0
net.ipv6.conf.eth0.hop_limit = 64
net.ipv6.conf.eth0.max_addresses = 16
net.ipv6.conf.eth0.mtu = 1500
net.ipv6.conf.eth0.ndisc_notify = 0
net.ipv6.conf.eth0.proxy_ndp = 0
net.ipv6.conf.eth0.router_solicitation_delay = 1
net.ipv6.conf.eth0.router_solicitation_interval = 4
net.ipv6.conf.eth0.router_solicitations = 3
net.ipv6.conf.lo.accept_dad = -1
net.ipv6.conf.lo.accept_ra = 1
net.ipv6.conf.lo.accept_ra_defrtr = 1
net.ipv6.conf.lo.accept_ra_pinfo = 1
net.ipv6.conf.lo.accept_redirects = 1
net.ipv6.conf.lo.accept_source_route = 0
net.ipv6.conf.lo.autoconf = 1
net.ipv6.conf.lo.dad_transmits = 1
net.ipv6.conf.lo.disable_ipv6 = 0
net.ipv6.conf.lo.force_mld_version = 0
net.ipv6.conf.lo.force_tllao = 0
net.ipv6.conf.lo.forwarding = 0
net.ipv6.conf.lo.hop_limit = 64
net.ipv6.conf.lo.max_addresses = 16
net.ipv6.conf.lo.mtu = 65536
net.ipv6.conf.lo.ndisc_notify = 0
net.ipv6.conf.lo.proxy_ndp = 0
net.ipv6.conf.lo.router_solicitation_delay = 1
net.ipv6.conf.lo.router_solicitation_interval = 4
net.ipv6.conf.lo.router_solicitations = 3
net.ipv6.conf.sit0.accept_dad = -1
net.ipv6.conf.sit0.accept_ra = 1
net.ipv6.conf.sit0.accept_ra_defrtr = 1
net.ipv6.conf.sit0.accept_ra_pinfo = 1
net.ipv6.conf.sit0.accept_redirects = 1
net.ipv6.conf.sit0.accept_source_route = 0
net.ipv6.conf.sit0.autoconf = 1
net.ipv6.conf.sit0.dad_transmits = 1
net.ipv6.conf.sit0.disable_ipv6 = 0
net.ipv6.conf.sit0.force_mld_version = 0
net.ipv6.conf.sit0.force_tllao = 0
net.ipv6.conf.sit0.forwarding = 0
net.ipv6.conf.sit0.hop_limit = 64
net.ipv6.conf.sit0.max_addresses = 16
net.ipv6.conf.sit0.mtu = 1480
net.ipv6.conf.sit0.ndisc_notify = 0
net.ipv6.conf.sit0.proxy_ndp = 0
net.ipv6.conf.sit0.router_solicitation_delay = 1
net.ipv6.conf.sit0.router_solicitation_interval = 4
net.ipv6.conf.sit0.router_solicitations = 3
net.ipv6.icmp.ratelimit = 1000
net.ipv6.ip6frag_high_thresh = 4194304
net.ipv6.ip6frag_low_thresh = 3145728
net.ipv6.ip6frag_secret_interval = 600
net.ipv6.ip6frag_time = 60
net.ipv6.mld_max_msf = 64
net.ipv6.neigh.default.anycast_delay = 100
net.ipv6.neigh.default.app_solicit = 0
net.ipv6.neigh.default.base_reachable_time_ms = 30000
net.ipv6.neigh.default.delay_first_probe_time = 5
net.ipv6.neigh.default.gc_interval = 30
net.ipv6.neigh.default.gc_stale_time = 60
net.ipv6.neigh.default.gc_thresh1 = 128
net.ipv6.neigh.default.gc_thresh2 = 512
net.ipv6.neigh.default.gc_thresh3 = 1024
net.ipv6.neigh.default.locktime = 0
net.ipv6.neigh.default.mcast_solicit = 3
net.ipv6.neigh.default.proxy_delay = 80
net.ipv6.neigh.default.proxy_qlen = 64
net.ipv6.neigh.default.retrans_time_ms = 1000
net.ipv6.neigh.default.ucast_solicit = 3
net.ipv6.neigh.default.unres_qlen = 31
net.ipv6.neigh.default.unres_qlen_bytes = 65536
net.ipv6.neigh.dummy0.anycast_delay = 100
net.ipv6.neigh.dummy0.app_solicit = 0
net.ipv6.neigh.dummy0.base_reachable_time_ms = 30000
net.ipv6.neigh.dummy0.delay_first_probe_time = 5
net.ipv6.neigh.dummy0.gc_stale_time = 60
net.ipv6.neigh.dummy0.locktime = 0
net.ipv6.neigh.dummy0.mcast_solicit = 3
net.ipv6.neigh.dummy0.proxy_delay = 80
net.ipv6.neigh.dummy0.proxy_qlen = 64
net.ipv6.neigh.dummy0.retrans_time_ms = 1000
net.ipv6.neigh.dummy0.ucast_solicit = 3
net.ipv6.neigh.dummy0.unres_qlen = 31
net.ipv6.neigh.dummy0.unres_qlen_bytes = 65536
net.ipv6.neigh.eth0.anycast_delay = 100
net.ipv6.neigh.eth0.app_solicit = 0
net.ipv6.neigh.eth0.base_reachable_time_ms = 30000
net.ipv6.neigh.eth0.delay_first_probe_time = 5
net.ipv6.neigh.eth0.gc_stale_time = 60
net.ipv6.neigh.eth0.locktime = 0
net.ipv6.neigh.eth0.mcast_solicit = 3
net.ipv6.neigh.eth0.proxy_delay = 80
net.ipv6.neigh.eth0.proxy_qlen = 64
net.ipv6.neigh.eth0.retrans_time_ms = 1000
net.ipv6.neigh.eth0.ucast_solicit = 3
net.ipv6.neigh.eth0.unres_qlen = 31
net.ipv6.neigh.eth0.unres_qlen_bytes = 65536
net.ipv6.neigh.lo.anycast_delay = 100
net.ipv6.neigh.lo.app_solicit = 0
net.ipv6.neigh.lo.base_reachable_time_ms = 30000
net.ipv6.neigh.lo.delay_first_probe_time = 5
net.ipv6.neigh.lo.gc_stale_time = 60
net.ipv6.neigh.lo.locktime = 0
net.ipv6.neigh.lo.mcast_solicit = 3
net.ipv6.neigh.lo.proxy_delay = 80
net.ipv6.neigh.lo.proxy_qlen = 64
net.ipv6.neigh.lo.retrans_time_ms = 1000
net.ipv6.neigh.lo.ucast_solicit = 3
net.ipv6.neigh.lo.unres_qlen = 31
net.ipv6.neigh.lo.unres_qlen_bytes = 65536
net.ipv6.neigh.sit0.anycast_delay = 100
net.ipv6.neigh.sit0.app_solicit = 0
net.ipv6.neigh.sit0.base_reachable_time_ms = 30000
net.ipv6.neigh.sit0.delay_first_probe_time = 5
net.ipv6.neigh.sit0.gc_stale_time = 60
net.ipv6.neigh.sit0.locktime = 0
net.ipv6.neigh.sit0.mcast_solicit = 3
net.ipv6.neigh.sit0.proxy_delay = 80
net.ipv6.neigh.sit0.proxy_qlen = 64
net.ipv6.neigh.sit0.retrans_time_ms = 1000
net.ipv6.neigh.sit0.ucast_solicit = 3
net.ipv6.neigh.sit0.unres_qlen = 31
net.ipv6.neigh.sit0.unres_qlen_bytes = 65536
net.ipv6.route.gc_elasticity = 9
net.ipv6.route.gc_interval = 30
net.ipv6.route.gc_min_interval = 0
net.ipv6.route.gc_min_interval_ms = 500
net.ipv6.route.gc_thresh = 1024
net.ipv6.route.gc_timeout = 60
net.ipv6.route.max_size = 4096
net.ipv6.route.min_adv_mss = 1220
net.ipv6.route.mtu_expires = 600
net.ipv6.xfrm6_gc_thresh = 1024
net.netfilter.nf_conntrack_acct = 0
net.netfilter.nf_conntrack_buckets = 8192
net.netfilter.nf_conntrack_checksum = 1
net.netfilter.nf_conntrack_count = 165
net.netfilter.nf_conntrack_expect_max = 124
net.netfilter.nf_conntrack_frag6_high_thresh = 4194304
net.netfilter.nf_conntrack_frag6_low_thresh = 3145728
net.netfilter.nf_conntrack_frag6_timeout = 60
net.netfilter.nf_conntrack_generic_timeout = 600
net.netfilter.nf_conntrack_helper = 1
net.netfilter.nf_conntrack_icmp_timeout = 30
net.netfilter.nf_conntrack_icmpv6_timeout = 30
net.netfilter.nf_conntrack_log_invalid = 0
net.netfilter.nf_conntrack_max = 31900
net.netfilter.nf_conntrack_tcp_be_liberal = 0
net.netfilter.nf_conntrack_tcp_loose = 1
net.netfilter.nf_conntrack_tcp_max_retrans = 3
net.netfilter.nf_conntrack_tcp_timeout_close = 10
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_established = 432000
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_last_ack = 30
net.netfilter.nf_conntrack_tcp_timeout_max_retrans = 300
net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 60
net.netfilter.nf_conntrack_tcp_timeout_syn_sent = 120
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_unacknowledged = 300
net.netfilter.nf_conntrack_udp_timeout = 30
net.netfilter.nf_conntrack_udp_timeout_stream = 180
net.netfilter.nf_log.0 = NONE
net.netfilter.nf_log.1 = NONE
net.netfilter.nf_log.10 = NONE
net.netfilter.nf_log.11 = NONE
net.netfilter.nf_log.12 = NONE
net.netfilter.nf_log.2 = ipt_ULOG
net.netfilter.nf_log.3 = NONE
net.netfilter.nf_log.4 = NONE
net.netfilter.nf_log.5 = NONE
net.netfilter.nf_log.6 = NONE
net.netfilter.nf_log.7 = NONE
net.netfilter.nf_log.8 = NONE
net.netfilter.nf_log.9 = NONE
net.nf_conntrack_max = 31900
net.unix.max_dgram_qlen = 10
vm.admin_reserve_kbytes = 8192
vm.block_dump = 0
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 500
vm.drop_caches = 0
vm.hugepages_treat_as_movable = 0
vm.hugetlb_shm_group = 0
vm.laptop_mode = 0
vm.legacy_va_layout = 0
vm.lowmem_reserve_ratio = 256	256	32
vm.max_map_count = 65530
vm.memory_failure_early_kill = 0
vm.memory_failure_recovery = 1
vm.min_free_kbytes = 5120
vm.mmap_min_addr = 4096
vm.nr_hugepages = 0
vm.nr_overcommit_hugepages = 0
vm.nr_pdflush_threads = 0
vm.oom_dump_tasks = 1
vm.oom_kill_allocating_task = 0
vm.overcommit_memory = 0
vm.overcommit_ratio = 50
vm.page-cluster = 3
vm.panic_on_oom = 0
vm.percpu_pagelist_fraction = 0
vm.scan_unevictable_pages = 0
vm.stat_interval = 1
vm.swappiness = 60
vm.user_reserve_kbytes = 31797
vm.vfs_cache_pressure = 100

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-07-12 21:33     ` Michael Sterrett
@ 2013-07-12 21:39       ` Yuchung Cheng
  2013-07-13 21:10         ` Michael Sterrett
  0 siblings, 1 reply; 10+ messages in thread
From: Yuchung Cheng @ 2013-07-12 21:39 UTC (permalink / raw)
  To: Michael Sterrett; +Cc: netdev

On Fri, Jul 12, 2013 at 2:33 PM, Michael Sterrett <michael@sterretts.net> wrote:
> Thanks for the reply.  The sysctl -a output is attached.

Hi Michael: my guess is some of the recent changes like TLP or F-RTO
rewrite has bugs. It'd be best if you can provide tcp traces that
triggers this. If not and this WARN appears frequently enough, please
try i) sysctl tcp_early_retrans=2 ii) sysctl tcp_frto=0 and see if
that WARN goes away. Thanks.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-07-12 21:39       ` Yuchung Cheng
@ 2013-07-13 21:10         ` Michael Sterrett
  2013-07-21  6:04           ` Yuchung Cheng
  0 siblings, 1 reply; 10+ messages in thread
From: Michael Sterrett @ 2013-07-13 21:10 UTC (permalink / raw)
  To: Yuchung Cheng; +Cc: netdev

After changing those two sysctl settings the WARN hasn't reappeared.
I'm willing to mess around with this box though if you want to try to
narrow down the root cause with different settings and/or patches.

On Fri, Jul 12, 2013 at 5:39 PM, Yuchung Cheng <ycheng@google.com> wrote:
> On Fri, Jul 12, 2013 at 2:33 PM, Michael Sterrett <michael@sterretts.net> wrote:
>> Thanks for the reply.  The sysctl -a output is attached.
>
> Hi Michael: my guess is some of the recent changes like TLP or F-RTO
> rewrite has bugs. It'd be best if you can provide tcp traces that
> triggers this. If not and this WARN appears frequently enough, please
> try i) sysctl tcp_early_retrans=2 ii) sysctl tcp_frto=0 and see if
> that WARN goes away. Thanks.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-07-13 21:10         ` Michael Sterrett
@ 2013-07-21  6:04           ` Yuchung Cheng
  2013-07-22  1:36             ` Michael Sterrett
  0 siblings, 1 reply; 10+ messages in thread
From: Yuchung Cheng @ 2013-07-21  6:04 UTC (permalink / raw)
  To: Michael Sterrett; +Cc: netdev

On Sat, Jul 13, 2013 at 2:10 PM, Michael Sterrett <michael@sterretts.net> wrote:
> After changing those two sysctl settings the WARN hasn't reappeared.
> I'm willing to mess around with this box though if you want to try to
> narrow down the root cause with different settings and/or patches.
Could you try to disable F-RTO but enable loss probe by
sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=3
to help us identify if this is loss probe bug or f-rto. Thanks.

>
> On Fri, Jul 12, 2013 at 5:39 PM, Yuchung Cheng <ycheng@google.com> wrote:
>> On Fri, Jul 12, 2013 at 2:33 PM, Michael Sterrett <michael@sterretts.net> wrote:
>>> Thanks for the reply.  The sysctl -a output is attached.
>>
>> Hi Michael: my guess is some of the recent changes like TLP or F-RTO
>> rewrite has bugs. It'd be best if you can provide tcp traces that
>> triggers this. If not and this WARN appears frequently enough, please
>> try i) sysctl tcp_early_retrans=2 ii) sysctl tcp_frto=0 and see if
>> that WARN goes away. Thanks.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-07-21  6:04           ` Yuchung Cheng
@ 2013-07-22  1:36             ` Michael Sterrett
  2013-09-11 12:49               ` Josh Boyer
  0 siblings, 1 reply; 10+ messages in thread
From: Michael Sterrett @ 2013-07-22  1:36 UTC (permalink / raw)
  To: Yuchung Cheng; +Cc: netdev

Since upgraded to 3.10.1 which exhibits the same issue (probably not
surprising).

sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=3 seems to take
care of it.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-07-22  1:36             ` Michael Sterrett
@ 2013-09-11 12:49               ` Josh Boyer
  2013-09-11 16:32                 ` Yuchung Cheng
  0 siblings, 1 reply; 10+ messages in thread
From: Josh Boyer @ 2013-09-11 12:49 UTC (permalink / raw)
  To: Michael Sterrett; +Cc: Yuchung Cheng, netdev

On Sun, Jul 21, 2013 at 9:36 PM, Michael Sterrett <michael@sterretts.net> wrote:
> Since upgraded to 3.10.1 which exhibits the same issue (probably not
> surprising).
>
> sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=3 seems to take
> care of it.

We're still getting reports of this with 3.10.10 in Fedora [1].  At
least one of the reporters has said those sysctl settings didn't help.

Is there anything else that can be gathered to help track this down?

josh

[1] https://bugzilla.redhat.com/show_bug.cgi?id=989251

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-09-11 12:49               ` Josh Boyer
@ 2013-09-11 16:32                 ` Yuchung Cheng
  2013-09-12  1:51                   ` Josh Boyer
  0 siblings, 1 reply; 10+ messages in thread
From: Yuchung Cheng @ 2013-09-11 16:32 UTC (permalink / raw)
  To: Josh Boyer; +Cc: Michael Sterrett, netdev, Neal Cardwell, Nandita Dukkipati

[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]

On Wed, Sep 11, 2013 at 5:49 AM, Josh Boyer <jwboyer@fedoraproject.org> wrote:
> On Sun, Jul 21, 2013 at 9:36 PM, Michael Sterrett <michael@sterretts.net> wrote:
>> Since upgraded to 3.10.1 which exhibits the same issue (probably not
>> surprising).
>>
>> sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=3 seems to take
>> care of it.
>
> We're still getting reports of this with 3.10.10 in Fedora [1].  At
> least one of the reporters has said those sysctl settings didn't help.
>
> Is there anything else that can be gathered to help track this down?
since frto=0 didn't help, I suspect it's either a bug introduced in
the tail loss probe patch 9b717a8d (tcp: TLP loss detection) or other
changes related loss recovery.

could you try disable TLP by
sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=1

if it still does not work. disable any form of early retransmit by
sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=0

if you can test a custom build kernel, please try this debugging patch.

>
> josh
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=989251

[-- Attachment #2: 0001-tcp-debug-fastretrans-warning.patch --]
[-- Type: application/octet-stream, Size: 1525 bytes --]

From 62561c91d4655539781113e0fa04ee681b22a2a4 Mon Sep 17 00:00:00 2001
From: Yuchung Cheng <ycheng@google.com>
Date: Wed, 31 Jul 2013 09:22:35 -0700
Subject: [PATCH] tcp: debug fastretrans warning

printk a ton of states to debug fastretrans warning

Signed-off-by: Yuchung Cheng <ycheng@google.com>
---
 net/ipv4/tcp_input.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 9c62257..c18dab1 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -2773,7 +2773,20 @@ static void tcp_fastretrans_alert(struct sock *sk, int pkts_acked,
 	/* D. Check state exit conditions. State can be terminated
 	 *    when high_seq is ACKed. */
 	if (icsk->icsk_ca_state == TCP_CA_Open) {
-		WARN_ON(tp->retrans_out != 0);
+		if (WARN_ON(tp->retrans_out != 0)) {
+			printk(KERN_DEBUG "%pI4:%u F0x%x S%u s%d IF%u+%u-%u-%u"
+			       "f%u ur%u rr%u rt%u um%u hs%u nxt%u\n",
+			       &inet_sk(sk)->inet_daddr,
+			       ntohs(inet_sk(sk)->inet_dport),
+			       flag, sk->sk_state, tp->rx_opt.sack_ok,
+			       tp->packets_out, tp->retrans_out,
+			       tp->sacked_out, tp->lost_out,
+			       tp->frto, tp->undo_retrans,
+			       tp->reordering, icsk->icsk_retransmits,
+			       tp->undo_marker ? tp->undo_marker-tp->snd_una:0,
+			       tp->high_seq - tp->snd_una,
+			       tp->snd_nxt - tp->snd_una);
+		}
 		tp->retrans_stamp = 0;
 	} else if (!before(tp->snd_una, tp->high_seq)) {
 		switch (icsk->icsk_ca_state) {
-- 
1.8.3


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 3.10.0 network trace
  2013-09-11 16:32                 ` Yuchung Cheng
@ 2013-09-12  1:51                   ` Josh Boyer
  0 siblings, 0 replies; 10+ messages in thread
From: Josh Boyer @ 2013-09-12  1:51 UTC (permalink / raw)
  To: Yuchung Cheng; +Cc: Michael Sterrett, netdev, Neal Cardwell, Nandita Dukkipati

On Wed, Sep 11, 2013 at 12:32 PM, Yuchung Cheng <ycheng@google.com> wrote:
> On Wed, Sep 11, 2013 at 5:49 AM, Josh Boyer <jwboyer@fedoraproject.org> wrote:
>> On Sun, Jul 21, 2013 at 9:36 PM, Michael Sterrett <michael@sterretts.net> wrote:
>>> Since upgraded to 3.10.1 which exhibits the same issue (probably not
>>> surprising).
>>>
>>> sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=3 seems to take
>>> care of it.
>>
>> We're still getting reports of this with 3.10.10 in Fedora [1].  At
>> least one of the reporters has said those sysctl settings didn't help.
>>
>> Is there anything else that can be gathered to help track this down?
> since frto=0 didn't help, I suspect it's either a bug introduced in
> the tail loss probe patch 9b717a8d (tcp: TLP loss detection) or other
> changes related loss recovery.
>
> could you try disable TLP by
> sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=1
>
> if it still does not work. disable any form of early retransmit by
> sysctl net.ipv4.tcp_frto=0 net.ipv4.tcp_early_retrans=0

Thanks.  I've asked the people in the bug to try those settings.

> if you can test a custom build kernel, please try this debugging patch.

I'll get the patch included in a test build for the tomorrow as well.

josh

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2013-09-12  1:51 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <CA+rTEUPfP8ouJHbaFrO=-zktg2D06kYgXOq-LnmyKbbEzj-TmA@mail.gmail.com>
2013-07-12 20:44 ` Fwd: 3.10.0 network trace Michael Sterrett
2013-07-12 21:18   ` Yuchung Cheng
2013-07-12 21:33     ` Michael Sterrett
2013-07-12 21:39       ` Yuchung Cheng
2013-07-13 21:10         ` Michael Sterrett
2013-07-21  6:04           ` Yuchung Cheng
2013-07-22  1:36             ` Michael Sterrett
2013-09-11 12:49               ` Josh Boyer
2013-09-11 16:32                 ` Yuchung Cheng
2013-09-12  1:51                   ` Josh Boyer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).