wireguard.lists.zx2c4.com archive mirror
 help / color / mirror / Atom feed
* Wireguard causes system to hang
@ 2019-07-22  0:43 Nicolas Chan
  2019-08-25 15:50 ` Jason A. Donenfeld
  0 siblings, 1 reply; 4+ messages in thread
From: Nicolas Chan @ 2019-07-22  0:43 UTC (permalink / raw)
  To: wireguard


[-- Attachment #1.1: Type: text/plain, Size: 14721 bytes --]

Hi Wireguard team,

I've recently started using Wireguard on a new system I put together, but
after a few minutes of running Wireguard the system hangs. If I don't start
Wireguard, the system behaves normally. I am using the latest version of
Arch Linux. It could be an issue with Arch, and I have also posted on their
forums about this issue, but based on the dmesg log I think it may be a
problem with Wireguard.

I can consistently reproduce this issue on my machine. I am connected on
ethernet, and normal networking works fine, and I have tried both
NetworkManager and systemd-networkd. When I start up Wireguard, either
through systemd-networkd native support or with wireguard's
systemd-networkd service (`sudo systemctl start wg-quick@intranet`)
("intranet" is the name of my Wireguard network) (configured as per the
wiki's instructions), it works perfectly fine for a couple of minutes. I
can even ping other hosts on my Wireguard network. However, after a couple
minutes, the CPU load on a single CPU core spikes to 100% (I watch htop).
Shortly thereafter, all pings to the Wireguard network fail. Normal
internet keeps working for a bit, but then that also fails (all pings and
network-related commands hang). I then have to forcibly restart my computer
by holding the power button to be able to use anything network related
again.

Based on the output of dmesg (below), I'm thinking this may be a bug with
the Wireguard kernel module, but I'm not sure. I've looked online, and have
not found anything similar related to Wireguard. It could also be poor
support for my CPU since it is relatively new (Ryzen 9 3900X), which did
have an issue with the way systemd generated random numbers that has since
been patched (and the dmesg output does mention a get_random_u32 function)
[1]. If anyone knows how to resolve this issue, or next steps I should take
to troubleshoot, I'd really appreciate your advice.

Thank you,
Nicolas

[1]
https://www.phoronix.com/scan.php?page=news_item&px=Ryzen-3K-RdRand-Systemd-Maybe

Here is the dmesg output:

[  545.876435] wireguard: WireGuard 0.0.20190702 loaded. See
www.wireguard.com for information.
[  545.876436] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld
<Jason@zx2c4.com>. All Rights Reserved.
[  708.146753] watchdog: BUG: soft lockup - CPU#2 stuck for 22s!
[kworker/u64:5:808]
[  708.146755] Modules linked in: wireguard(OE) ip6_udp_tunnel
udp_tunnel fuse xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user
xfrm_algo iptable_nat xt_addrtype iptable_filter xt_conntrack nf_nat
nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge stp llc
edac_mce_amd snd_hda_codec_hdmi arc4 nvidia_drm(POE)
nvidia_modeset(POE) nls_iso8859_1 nls_cp437 vfat fat kvm iwlmvm
nvidia(POE) irqbypass mac80211 snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio btusb btrtl crct10dif_pclmul btbcm
snd_hda_intel btintel crc32_pclmul ghash_clmulni_intel hid_uclogic
joydev mousedev bluetooth wmi_bmof input_leds iwlwifi snd_hda_codec
drm_kms_helper snd_hda_core ecdh_generic ecc snd_hwdep crc16 drm
cfg80211 aesni_intel snd_pcm aes_x86_64 agpgart crypto_simd cryptd
ipmi_devintf igb glue_helper snd_timer ipmi_msghandler snd syscopyarea
sp5100_tco sysfillrect i2c_piix4 ccp sysimgblt pcspkr fb_sys_fops
evdev i2c_algo_bit soundcore rfkill rng_core dca mac_hid pinctrl_amd
wmi pcc_cpufreq
[  708.146779]  acpi_cpufreq sg crypto_user ip_tables x_tables btrfs
libcrc32c crc32c_generic xor raid6_pq sd_mod hid_generic usbhid hid
ahci libahci libata xhci_pci crc32c_intel scsi_mod xhci_hcd
[  708.146786] CPU: 2 PID: 808 Comm: kworker/u64:5 Tainted: P
 OE     5.2.1-arch1-1-ARCH #1
[  708.146787] Hardware name: Gigabyte Technology Co., Ltd. X570 I
AORUS PRO WIFI/X570 I AORUS PRO WIFI, BIOS F3 06/13/2019
[  708.146792] Workqueue: wg-kex-intranet
wg_packet_handshake_send_worker [wireguard]
[  708.146795] RIP: 0010:get_random_u32+0x21/0xa0
[  708.146796] Code: b8 01 00 00 00 eb cc 66 90 0f 1f 44 00 00 41 54
55 53 48 83 ec 08 0f 1f 44 00 00 ba 0a 00 00 00 eb 05 ba 0a 00 00 00
0f c7 f0 <72> 5f 83 ea 01 75 f6 48 8b 74 24 20 48 c7 c2 20 84 1c b5 48
c7 c7
[  708.146797] RSP: 0018:ffffa543c9123ca8 EFLAGS: 00000203 ORIG_RAX:
ffffffffffffff13
[  708.146797] RAX: 00000000ffffffff RBX: ffffa05d720b42c8 RCX: 0000000000000000
[  708.146798] RDX: 000000000000000a RSI: 0000000000000201 RDI: ffffffffc0ef6297
[  708.146798] RBP: ffffa05d3ac60000 R08: ffffffffc0f1eea0 R09: ffffa543c9123c58
[  708.146798] R10: 0000000000000000 R11: 073b28980328607f R12: ffffa05d3ac70000
[  708.146799] R13: ffffa05d720b42d0 R14: ffffa543c9123d30 R15: ffffa05d720b43f0
[  708.146799] FS:  0000000000000000(0000) GS:ffffa05dbe680000(0000)
knlGS:0000000000000000
[  708.146800] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  708.146800] CR2: 00007efbdec04000 CR3: 00000007ebd58000 CR4: 0000000000340ee0
[  708.146801] Call Trace:
[  708.146804]  ? wg_index_hashtable_insert+0x6f/0x110 [wireguard]
[  708.146807]  wg_index_hashtable_insert+0x5c/0x110 [wireguard]
[  708.146809]  wg_noise_handshake_create_initiation+0x2d6/0x300 [wireguard]
[  708.146812]  wg_packet_send_handshake_initiation+0xc9/0x160 [wireguard]
[  708.146814]  wg_packet_handshake_send_worker+0x18/0x30 [wireguard]
[  708.146816]  process_one_work+0x1d1/0x3e0
[  708.146817]  worker_thread+0x4a/0x3d0
[  708.146818]  kthread+0xfd/0x130
[  708.146819]  ? process_one_work+0x3e0/0x3e0
[  708.146820]  ? kthread_park+0x90/0x90
[  708.146822]  ret_from_fork+0x22/0x40
[  736.146757] watchdog: BUG: soft lockup - CPU#2 stuck for 22s!
[kworker/u64:5:808]
[  736.146759] Modules linked in: wireguard(OE) ip6_udp_tunnel
udp_tunnel fuse xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user
xfrm_algo iptable_nat xt_addrtype iptable_filter xt_conntrack nf_nat
nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge stp llc
edac_mce_amd snd_hda_codec_hdmi arc4 nvidia_drm(POE)
nvidia_modeset(POE) nls_iso8859_1 nls_cp437 vfat fat kvm iwlmvm
nvidia(POE) irqbypass mac80211 snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio btusb btrtl crct10dif_pclmul btbcm
snd_hda_intel btintel crc32_pclmul ghash_clmulni_intel hid_uclogic
joydev mousedev bluetooth wmi_bmof input_leds iwlwifi snd_hda_codec
drm_kms_helper snd_hda_core ecdh_generic ecc snd_hwdep crc16 drm
cfg80211 aesni_intel snd_pcm aes_x86_64 agpgart crypto_simd cryptd
ipmi_devintf igb glue_helper snd_timer ipmi_msghandler snd syscopyarea
sp5100_tco sysfillrect i2c_piix4 ccp sysimgblt pcspkr fb_sys_fops
evdev i2c_algo_bit soundcore rfkill rng_core dca mac_hid pinctrl_amd
wmi pcc_cpufreq
[  736.146783]  acpi_cpufreq sg crypto_user ip_tables x_tables btrfs
libcrc32c crc32c_generic xor raid6_pq sd_mod hid_generic usbhid hid
ahci libahci libata xhci_pci crc32c_intel scsi_mod xhci_hcd
[  736.146789] CPU: 2 PID: 808 Comm: kworker/u64:5 Tainted: P
 OEL    5.2.1-arch1-1-ARCH #1
[  736.146790] Hardware name: Gigabyte Technology Co., Ltd. X570 I
AORUS PRO WIFI/X570 I AORUS PRO WIFI, BIOS F3 06/13/2019
[  736.146796] Workqueue: wg-kex-intranet
wg_packet_handshake_send_worker [wireguard]
[  736.146799] RIP: 0010:get_random_u32+0x21/0xa0
[  736.146800] Code: b8 01 00 00 00 eb cc 66 90 0f 1f 44 00 00 41 54
55 53 48 83 ec 08 0f 1f 44 00 00 ba 0a 00 00 00 eb 05 ba 0a 00 00 00
0f c7 f0 <72> 5f 83 ea 01 75 f6 48 8b 74 24 20 48 c7 c2 20 84 1c b5 48
c7 c7
[  736.146800] RSP: 0018:ffffa543c9123ca8 EFLAGS: 00000203 ORIG_RAX:
ffffffffffffff13
[  736.146801] RAX: 00000000ffffffff RBX: ffffa05d720b42c8 RCX: 0000000000000000
[  736.146802] RDX: 000000000000000a RSI: 0000000000000201 RDI: ffffffffc0ef6297
[  736.146802] RBP: ffffa05d3ac60000 R08: ffffffffc0f1eea0 R09: ffffa543c9123c58
[  736.146802] R10: 0000000000000000 R11: 073b28980328607f R12: ffffa05d3ac70000
[  736.146803] R13: ffffa05d720b42d0 R14: ffffa543c9123d30 R15: ffffa05d720b43f0
[  736.146803] FS:  0000000000000000(0000) GS:ffffa05dbe680000(0000)
knlGS:0000000000000000
[  736.146804] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  736.146804] CR2: 00007efbdec04000 CR3: 00000007ebd58000 CR4: 0000000000340ee0
[  736.146805] Call Trace:
[  736.146809]  ? wg_index_hashtable_insert+0x7d/0x110 [wireguard]
[  736.146811]  wg_index_hashtable_insert+0x5c/0x110 [wireguard]
[  736.146813]  wg_noise_handshake_create_initiation+0x2d6/0x300 [wireguard]
[  736.146816]  wg_packet_send_handshake_initiation+0xc9/0x160 [wireguard]
[  736.146818]  wg_packet_handshake_send_worker+0x18/0x30 [wireguard]
[  736.146820]  process_one_work+0x1d1/0x3e0
[  736.146821]  worker_thread+0x4a/0x3d0
[  736.146822]  kthread+0xfd/0x130
[  736.146823]  ? process_one_work+0x3e0/0x3e0
[  736.146824]  ? kthread_park+0x90/0x90
[  736.146826]  ret_from_fork+0x22/0x40
[  740.753351] rcu: INFO: rcu_preempt self-detected stall on CPU
[  740.753355] rcu:     2-....: (8 GPs behind)
idle=bd2/1/0x4000000000000002 softirq=1024/1024 fqs=5999
last_accelerate: bdca/048c, Nonlazy posted: .LD
[  740.753358]  (t=18000 jiffies g=9985 q=125494)
[  740.753359] NMI backtrace for cpu 2
[  740.753361] CPU: 2 PID: 808 Comm: kworker/u64:5 Tainted: P
 OEL    5.2.1-arch1-1-ARCH #1
[  740.753361] Hardware name: Gigabyte Technology Co., Ltd. X570 I
AORUS PRO WIFI/X570 I AORUS PRO WIFI, BIOS F3 06/13/2019
[  740.753368] Workqueue: wg-kex-intranet
wg_packet_handshake_send_worker [wireguard]
[  740.753370] Call Trace:
[  740.753373]  <IRQ>
[  740.753377]  dump_stack+0x5c/0x78
[  740.753379]  nmi_cpu_backtrace.cold+0x13/0x51
[  740.753381]  ? lapic_can_unplug_cpu.cold+0x44/0x44
[  740.753383]  nmi_trigger_cpumask_backtrace+0x10e/0x11f
[  740.753385]  rcu_dump_cpu_stacks+0x92/0xc0
[  740.753387]  rcu_sched_clock_irq.cold+0x25c/0x4fb
[  740.753389]  update_process_times+0x29/0x60
[  740.753390]  tick_sched_handle.isra.0+0x22/0x60
[  740.753392]  tick_sched_timer+0x3b/0x80
[  740.753393]  ? tick_sched_handle.isra.0+0x60/0x60
[  740.753394]  __hrtimer_run_queues+0xfd/0x2c0
[  740.753396]  hrtimer_interrupt+0x118/0x240
[  740.753397]  ? sched_clock+0x5/0x10
[  740.753399]  smp_apic_timer_interrupt+0x84/0x180
[  740.753401]  apic_timer_interrupt+0xf/0x20
[  740.753402]  </IRQ>
[  740.753404] RIP: 0010:get_random_u32+0x21/0xa0
[  740.753405] Code: b8 01 00 00 00 eb cc 66 90 0f 1f 44 00 00 41 54
55 53 48 83 ec 08 0f 1f 44 00 00 ba 0a 00 00 00 eb 05 ba 0a 00 00 00
0f c7 f0 <72> 5f 83 ea 01 75 f6 48 8b 74 24 20 48 c7 c2 20 84 1c b5 48
c7 c7
[  740.753406] RSP: 0018:ffffa543c9123ca8 EFLAGS: 00000203 ORIG_RAX:
ffffffffffffff13
[  740.753407] RAX: 00000000ffffffff RBX: ffffa05d720b42c8 RCX: 0000000000000000
[  740.753407] RDX: 000000000000000a RSI: 0000000000000201 RDI: ffffffffc0ef6297
[  740.753408] RBP: ffffa05d3ac60000 R08: ffffffffc0f1eea0 R09: ffffa543c9123c58
[  740.753408] R10: 0000000000000000 R11: 073b28980328607f R12: ffffa05d3ac70000
[  740.753409] R13: ffffa05d720b42d0 R14: ffffa543c9123d30 R15: ffffa05d720b43f0
[  740.753413]  ? wg_index_hashtable_insert+0x57/0x110 [wireguard]
[  740.753416]  ? wg_index_hashtable_insert+0x7d/0x110 [wireguard]
[  740.753418]  wg_index_hashtable_insert+0x5c/0x110 [wireguard]
[  740.753421]  wg_noise_handshake_create_initiation+0x2d6/0x300 [wireguard]
[  740.753425]  wg_packet_send_handshake_initiation+0xc9/0x160 [wireguard]
[  740.753428]  wg_packet_handshake_send_worker+0x18/0x30 [wireguard]
[  740.753430]  process_one_work+0x1d1/0x3e0
[  740.753431]  worker_thread+0x4a/0x3d0
[  740.753433]  kthread+0xfd/0x130
[  740.753434]  ? process_one_work+0x3e0/0x3e0
[  740.753435]  ? kthread_park+0x90/0x90
[  740.753436]  ret_from_fork+0x22/0x40
[  768.146760] watchdog: BUG: soft lockup - CPU#2 stuck for 22s!
[kworker/u64:5:808]
[  768.146762] Modules linked in: wireguard(OE) ip6_udp_tunnel
udp_tunnel fuse xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user
xfrm_algo iptable_nat xt_addrtype iptable_filter xt_conntrack nf_nat
nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter bridge stp llc
edac_mce_amd snd_hda_codec_hdmi arc4 nvidia_drm(POE)
nvidia_modeset(POE) nls_iso8859_1 nls_cp437 vfat fat kvm iwlmvm
nvidia(POE) irqbypass mac80211 snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio btusb btrtl crct10dif_pclmul btbcm
snd_hda_intel btintel crc32_pclmul ghash_clmulni_intel hid_uclogic
joydev mousedev bluetooth wmi_bmof input_leds iwlwifi snd_hda_codec
drm_kms_helper snd_hda_core ecdh_generic ecc snd_hwdep crc16 drm
cfg80211 aesni_intel snd_pcm aes_x86_64 agpgart crypto_simd cryptd
ipmi_devintf igb glue_helper snd_timer ipmi_msghandler snd syscopyarea
sp5100_tco sysfillrect i2c_piix4 ccp sysimgblt pcspkr fb_sys_fops
evdev i2c_algo_bit soundcore rfkill rng_core dca mac_hid pinctrl_amd
wmi pcc_cpufreq
[  768.146785]  acpi_cpufreq sg crypto_user ip_tables x_tables btrfs
libcrc32c crc32c_generic xor raid6_pq sd_mod hid_generic usbhid hid
ahci libahci libata xhci_pci crc32c_intel scsi_mod xhci_hcd
[  768.146791] CPU: 2 PID: 808 Comm: kworker/u64:5 Tainted: P
 OEL    5.2.1-arch1-1-ARCH #1
[  768.146791] Hardware name: Gigabyte Technology Co., Ltd. X570 I
AORUS PRO WIFI/X570 I AORUS PRO WIFI, BIOS F3 06/13/2019
[  768.146797] Workqueue: wg-kex-intranet
wg_packet_handshake_send_worker [wireguard]
[  768.146800] RIP: 0010:get_random_u32+0x21/0xa0
[  768.146801] Code: b8 01 00 00 00 eb cc 66 90 0f 1f 44 00 00 41 54
55 53 48 83 ec 08 0f 1f 44 00 00 ba 0a 00 00 00 eb 05 ba 0a 00 00 00
0f c7 f0 <72> 5f 83 ea 01 75 f6 48 8b 74 24 20 48 c7 c2 20 84 1c b5 48
c7 c7
[  768.146801] RSP: 0018:ffffa543c9123ca8 EFLAGS: 00000203 ORIG_RAX:
ffffffffffffff13
[  768.146802] RAX: 00000000ffffffff RBX: ffffa05d720b42c8 RCX: 0000000000000000
[  768.146802] RDX: 000000000000000a RSI: 0000000000000201 RDI: ffffffffc0ef6297
[  768.146802] RBP: ffffa05d3ac60000 R08: ffffffffc0f1eea0 R09: ffffa543c9123c58
[  768.146803] R10: 0000000000000000 R11: 073b28980328607f R12: ffffa05d3ac70000
[  768.146803] R13: ffffa05d720b42d0 R14: ffffa543c9123d30 R15: ffffa05d720b43f0
[  768.146804] FS:  0000000000000000(0000) GS:ffffa05dbe680000(0000)
knlGS:0000000000000000
[  768.146804] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  768.146805] CR2: 00007efbdec04000 CR3: 00000007ebd58000 CR4: 0000000000340ee0
[  768.146805] Call Trace:
[  768.146808]  ? wg_index_hashtable_insert+0x6f/0x110 [wireguard]
[  768.146810]  wg_index_hashtable_insert+0x5c/0x110 [wireguard]
[  768.146813]  wg_noise_handshake_create_initiation+0x2d6/0x300 [wireguard]
[  768.146815]  wg_packet_send_handshake_initiation+0xc9/0x160 [wireguard]
[  768.146817]  wg_packet_handshake_send_worker+0x18/0x30 [wireguard]
[  768.146819]  process_one_work+0x1d1/0x3e0
[  768.146820]  worker_thread+0x4a/0x3d0
[  768.146821]  kthread+0xfd/0x130
[  768.146822]  ? process_one_work+0x3e0/0x3e0
[  768.146823]  ? kthread_park+0x90/0x90
[  768.146824]  ret_from_fork+0x22/0x40

[-- Attachment #1.2: Type: text/html, Size: 15422 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2019-08-25 19:42 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-22  0:43 Wireguard causes system to hang Nicolas Chan
2019-08-25 15:50 ` Jason A. Donenfeld
2019-08-25 19:17   ` Erik Jerde
2019-08-25 19:25   ` Nicolas Chan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).