* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
[not found] ` <CABXGCsNN+u2SqOvmw2JojTnESSLgxKgfJLQuB3Ne1fcNA47UZw@mail.gmail.com>
@ 2021-02-13 3:03 ` Hillf Danton
2021-02-28 13:22 ` Mikhail Gavrilov
0 siblings, 1 reply; 10+ messages in thread
From: Hillf Danton @ 2021-02-13 3:03 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Fri, 12 Feb 2021 18:28:12 +0500 Mikhail Gavrilov wrote:
> On Tue, 26 Jan 2021 at 13:28, Hillf Danton <hdanton@sina.com> wrote:
> >
> > BTW better run the reproducer again with KASAN enabled.
> >
>
> It happened today again with kernel 5.11 rc7 (e0756cfc7d7c)
Thanks again.
> Why not try your patch?
Simply because it was half baked - I was not convinced it was a fix
instead of papering over anything.
>
> list_del corruption, ffffdef70143e848->next is LIST_POISON1 (dead000000000100)
> ------------[ cut here ]------------
> kernel BUG at lib/list_debug.c:45!
> invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 13 PID: 263 Comm: kswapd0 Tainted: G W ---------
> --- 5.11.0-0.rc7.20210210gite0756cfc7d7c.150.fc35.x86_64 #1
> Hardware name: System manufacturer System Product Name/ROG STRIX
> X570-I GAMING, BIOS 3402 01/13/2021
> RIP: 0010:__list_del_entry_valid.cold+0xf/0x47
> Code: fe ff 0f 0b 48 89 d1 4c 89 c6 4c 89 ca 48 c7 c7 e0 26 64 9e e8
> 1b 12 fe ff 0f 0b 48 89 fe 48 c7 c7 70 27 64 9e e8 0a 12 fe ff <0f> 0b
> 48 c7 c7 20 28 64 9e e8 fc 11 fe ff 0f 0b 48 89 f2 48 89 fe
> RSP: 0018:ffff9f2180863908 EFLAGS: 00010286
> RAX: 000000000000004e RBX: ffff8f74d0fa1000 RCX: 0000000000000000
> RDX: ffff8f7c885e9f60 RSI: ffff8f7c885db2a0 RDI: ffff8f7c885db2a0
> RBP: ffff8f74d0fa1000 R08: 0000000000000000 R09: ffff9f2180863748
> R10: ffff9f2180863740 R11: 0000000000000000 R12: ffff8f758edd8e00
> R13: 0000000000012800 R14: ffffdef70143e840 R15: ffff8f758edd8e08
> FS: 0000000000000000(0000) GS:ffff8f7c88400000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000037203acb1000 CR3: 00000001d5c28000 CR4: 0000000000350ee0
> Call Trace:
> z3fold_zpool_malloc+0x3e3/0x780
There is a race producing the list corruption above.
> ? _raw_spin_unlock+0x1f/0x30
> zswap_frontswap_store+0x43e/0x890
> __frontswap_store+0xc8/0x170
> swap_writepage+0x39/0x70
> pageout+0x125/0x540
> shrink_page_list+0x1329/0x1bc0
> shrink_inactive_list+0x12a/0x440
> shrink_lruvec+0x4a9/0x6d0
> ? super_cache_count+0x79/0xf0
> shrink_node+0x2d1/0x700
> balance_pgdat+0x2f5/0x650
> kswapd+0x21d/0x4d0
> ? do_wait_intr_irq+0xd0/0xd0
> ? balance_pgdat+0x650/0x650
> kthread+0x13a/0x150
> ? __kthread_bind_mask+0x60/0x60
> ret_from_fork+0x22/0x30
> Modules linked in: tun snd_seq_dummy snd_hrtimer uinput rfcomm
> nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ip6table_nat
> ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat
> nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw
> iptable_security ip_set nf_tables nfnetlink ip6table_filter ip6_tables
> iptable_filter cmac bnep zstd sunrpc vfat fat hid_logitech_hidpp
> hid_logitech_dj snd_hda_codec_realtek snd_hda_codec_generic
> ledtrig_audio snd_hda_codec_hdmi snd_hda_intel mt76x2u
> snd_intel_dspcfg soundwire_intel mt76x2_common mt76x02_usb
> soundwire_generic_allocation mt76_usb intel_rapl_msr iwlmvm
> snd_soc_core snd_usb_audio intel_rapl_common mt76x02_lib mt76
> snd_compress snd_pcm_dmaengine snd_usbmidi_lib soundwire_cadence
> snd_rawmidi mac80211 snd_hda_codec joydev snd_hda_core uvcvideo
> ac97_bus snd_hwdep btusb snd_seq
> videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 snd_seq_device
> btrtl edac_mce_amd btbcm iwlwifi snd_pcm videobuf2_common btintel
> kvm_amd eeepc_wmi snd_timer bluetooth kvm videodev asus_wmi snd
> ecdh_generic sparse_keymap irqbypass xpad mc libarc4 sp5100_tco rapl
> ff_memless cfg80211 wmi_bmof ecc video pcspkr soundcore k10temp
> i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
> drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
> crc32_pclmul crc32c_intel cec drm ghash_clmulni_intel igb ccp nvme dca
> nvme_core i2c_algo_bit wmi pinctrl_amd fuse
> ---[ end trace a0c35e2a81af0791 ]---
> RIP: 0010:__list_del_entry_valid.cold+0xf/0x47
> Code: fe ff 0f 0b 48 89 d1 4c 89 c6 4c 89 ca 48 c7 c7 e0 26 64 9e e8
> 1b 12 fe ff 0f 0b 48 89 fe 48 c7 c7 70 27 64 9e e8 0a 12 fe ff <0f> 0b
> 48 c7 c7 20 28 64 9e e8 fc 11 fe ff 0f 0b 48 89 f2 48 89 fe
> RSP: 0018:ffff9f2180863908 EFLAGS: 00010286
> RAX: 000000000000004e RBX: ffff8f74d0fa1000 RCX: 0000000000000000
> RDX: ffff8f7c885e9f60 RSI: ffff8f7c885db2a0 RDI: ffff8f7c885db2a0
> RBP: ffff8f74d0fa1000 R08: 0000000000000000 R09: ffff9f2180863748
> R10: ffff9f2180863740 R11: 0000000000000000 R12: ffff8f758edd8e00
> R13: 0000000000012800 R14: ffffdef70143e840 R15: ffff8f758edd8e08
> FS: 0000000000000000(0000) GS:ffff8f7c88400000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000037203acb1000 CR3: 00000001d5c28000 CR4: 0000000000350ee0
> note: kswapd0[263] exited with preempt_count 2
>
>
> full kernel log: https://pastebin.com/FL1fZLJ0
>
> --
> Best Regards,
> Mike Gavrilov.
The comment below shows a race instance, though I failed to put things
together to see how within two hours. Cut it and see what will come up.
--- a/mm/z3fold.c
+++ b/mm/z3fold.c
@@ -1129,19 +1129,22 @@ retry:
page = NULL;
if (can_sleep) {
spin_lock(&pool->stale_lock);
+ spin_lock(&pool->lock);
zhdr = list_first_entry_or_null(&pool->stale,
struct z3fold_header, buddy);
/*
- * Before allocating a page, let's see if we can take one from
+ * Before allocating a page, lets see if we can take one from
* the stale pages list. cancel_work_sync() can sleep so we
* limit this case to the contexts where we can sleep
*/
if (zhdr) {
list_del(&zhdr->buddy);
+ spin_unlock(&pool->lock);
spin_unlock(&pool->stale_lock);
cancel_work_sync(&zhdr->work);
page = virt_to_page(zhdr);
} else {
+ spin_unlock(&pool->lock);
spin_unlock(&pool->stale_lock);
}
}
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-02-13 3:03 ` BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4) Hillf Danton
@ 2021-02-28 13:22 ` Mikhail Gavrilov
2021-03-01 3:11 ` Hillf Danton
0 siblings, 1 reply; 10+ messages in thread
From: Mikhail Gavrilov @ 2021-02-28 13:22 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Sat, 13 Feb 2021 at 08:03, Hillf Danton <hdanton@sina.com> wrote:
>
> The comment below shows a race instance, though I failed to put things
> together to see how within two hours. Cut it and see what will come up.
>
> --- a/mm/z3fold.c
> +++ b/mm/z3fold.c
> @@ -1129,19 +1129,22 @@ retry:
> page = NULL;
> if (can_sleep) {
> spin_lock(&pool->stale_lock);
> + spin_lock(&pool->lock);
> zhdr = list_first_entry_or_null(&pool->stale,
> struct z3fold_header, buddy);
> /*
> - * Before allocating a page, let's see if we can take one from
> + * Before allocating a page, lets see if we can take one from
> * the stale pages list. cancel_work_sync() can sleep so we
> * limit this case to the contexts where we can sleep
> */
> if (zhdr) {
> list_del(&zhdr->buddy);
> + spin_unlock(&pool->lock);
> spin_unlock(&pool->stale_lock);
> cancel_work_sync(&zhdr->work);
> page = virt_to_page(zhdr);
> } else {
> + spin_unlock(&pool->lock);
> spin_unlock(&pool->stale_lock);
> }
> }
Hi,
It happened again with the patch above.
Is anything cleared up now?
[32451.229358] list_add corruption. next->prev should be prev
(ffffd08fbc661cd0), but was ffffffffa7643650. (next=ffff9e4d2848f1c0).
[32451.229395] ------------[ cut here ]------------
[32451.229398] kernel BUG at lib/list_debug.c:23!
[32451.229408] invalid opcode: 0000 [#1] SMP NOPTI
[32451.229414] CPU: 4 PID: 80665 Comm: kworker/u64:0 Tainted: G
W --------- --- 5.11.0-155.fc35.x86_64+debug #1
[32451.229420] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32451.229424] Workqueue: zswap3 compact_page_work
[32451.229433] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[32451.229439] Code: 48 c7 c6 24 26 64 a8 48 89 ef 49 c7 c7 ea ff ff
ff e8 e8 71 01 00 e9 fa 10 9e ff 4c 89 c1 48 c7 c7 f0 26 64 a8 e8 50
12 fe ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 a0 27 64 a8 e8 39
12 fe
[32451.229444] RSP: 0018:ffffb08fd553fde0 EFLAGS: 00010286
[32451.229449] RAX: 0000000000000075 RBX: ffffe16d871550c0 RCX: 0000000000000000
[32451.229453] RDX: ffff9e53c73e9f60 RSI: ffff9e53c73db2a0 RDI: ffff9e53c73db2a0
[32451.229457] RBP: ffffd08fbc661cd0 R08: 0000000000000000 R09: ffffb08fd553fc20
[32451.229460] R10: ffffb08fd553fc18 R11: 0000000000000000 R12: ffff9e4ce4e29008
[32451.229464] R13: ffff9e4d85543010 R14: ffff9e4d2848f1c0 R15: ffff9e4d85543000
[32451.229468] FS: 0000000000000000(0000) GS:ffff9e53c7200000(0000)
knlGS:0000000000000000
[32451.229472] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32451.229476] CR2: 00000fb21455dfe8 CR3: 0000000142968000 CR4: 0000000000350ee0
[32451.229480] Call Trace:
[32451.229485] do_compact_page+0x28d/0xb60
[32451.229492] ? debug_object_deactivate+0x55/0x140
[32451.229499] ? lock_release+0x1e9/0x400
[32451.229505] ? lock_release+0x1e9/0x400
[32451.229511] process_one_work+0x2b0/0x5e0
[32451.229519] worker_thread+0x55/0x3c0
[32451.229524] ? process_one_work+0x5e0/0x5e0
[32451.229531] kthread+0x13a/0x150
[32451.229540] ? __kthread_bind_mask+0x60/0x60
[32451.229548] ret_from_fork+0x22/0x30
[32451.229558] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32451.229628] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32451.229696] ---[ end trace 80d86d6942435514 ]---
[32451.229701] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[32451.229706] Code: 48 c7 c6 24 26 64 a8 48 89 ef 49 c7 c7 ea ff ff
ff e8 e8 71 01 00 e9 fa 10 9e ff 4c 89 c1 48 c7 c7 f0 26 64 a8 e8 50
12 fe ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 a0 27 64 a8 e8 39
12 fe
[32451.229710] RSP: 0018:ffffb08fd553fde0 EFLAGS: 00010286
[32451.229715] RAX: 0000000000000075 RBX: ffffe16d871550c0 RCX: 0000000000000000
[32451.229721] RDX: ffff9e53c73e9f60 RSI: ffff9e53c73db2a0 RDI: ffff9e53c73db2a0
[32451.229725] RBP: ffffd08fbc661cd0 R08: 0000000000000000 R09: ffffb08fd553fc20
[32451.229729] R10: ffffb08fd553fc18 R11: 0000000000000000 R12: ffff9e4ce4e29008
[32451.229732] R13: ffff9e4d85543010 R14: ffff9e4d2848f1c0 R15: ffff9e4d85543000
[32451.229736] FS: 0000000000000000(0000) GS:ffff9e53c7200000(0000)
knlGS:0000000000000000
[32451.229740] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32451.229744] CR2: 00000fb21455dfe8 CR3: 0000000142968000 CR4: 0000000000350ee0
[32451.229748] note: kworker/u64:0[80665] exited with preempt_count 2
[32476.846645] watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
[vivaldi-bin:6991]
[32476.846658] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.846704] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.846874] irq event stamp: 0
[32476.846877] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.846883] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.846889] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.846892] softirqs last disabled at (0): [<0000000000000000>] 0x0
[32476.846896] CPU: 0 PID: 6991 Comm: vivaldi-bin Tainted: G D W
--------- --- 5.11.0-155.fc35.x86_64+debug #1
[32476.846900] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32476.846904] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[32476.846909] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 80 f1 1e 00 48 03 04 fd 00 39 6e a8 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[32476.846913] RSP: 0000:ffffb08fd2937c10 EFLAGS: 00000246
[32476.846917] RAX: 0000000000000000 RBX: ffffe16d9e844240 RCX: ffff9e53c6bef180
[32476.846920] RDX: ffff9e4cc11a3d28 RSI: 0000000000040000 RDI: 000000000000000d
[32476.846923] RBP: ffff9e4cc11a3d28 R08: 0000000000040000 R09: 0000000000000000
[32476.846926] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9e4cc11a3d40
[32476.846929] R13: ffff9e4cc11a3d28 R14: ffff9e4cc11a3d20 R15: 0000000000ddd8d8
[32476.846932] FS: 00007f4b852a0300(0000) GS:ffff9e53c6a00000(0000)
knlGS:0000000000000000
[32476.846935] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32476.846939] CR2: 00003d0509d83000 CR3: 000000017733a000 CR4: 0000000000350ef0
[32476.846942] Call Trace:
[32476.846946] do_raw_spin_lock+0x94/0xa0
[32476.846951] _raw_spin_lock+0x63/0x80
[32476.846955] zswap_frontswap_load+0x2f/0x2f0
[32476.846960] ? psi_group_change+0x27d/0x290
[32476.846965] __frontswap_load+0xc3/0x160
[32476.846969] swap_readpage+0x1ca/0x3a0
[32476.846974] swapin_readahead+0x2ee/0x4e0
[32476.846979] do_swap_page+0x4a4/0x900
[32476.846983] ? lock_release+0x1e9/0x400
[32476.846987] ? trace_hardirqs_on+0x1b/0xe0
[32476.846992] handle_mm_fault+0xe7d/0x19d0
[32476.846997] do_user_addr_fault+0x1c7/0x4c0
[32476.847003] exc_page_fault+0x67/0x2a0
[32476.847007] ? asm_exc_page_fault+0x8/0x30
[32476.847011] asm_exc_page_fault+0x1e/0x30
[32476.847015] RIP: 0033:0x55a5d9c33379
[32476.847018] Code: 00 00 4d 89 75 00 4c 89 f0 48 25 00 00 fc ff 48
8b 40 08 41 c7 46 03 03 00 00 00 49 8b 4d 00 44 89 61 07 49 8b 5d 00
4d 8b 37 <44> 89 73 0b a9 00 00 04 00 75 1a 83 e0 18 48 85 c0 74 12 49
8b 45
[32476.847022] RSP: 002b:00007fff34882340 EFLAGS: 00010206
[32476.847025] RAX: 0000000000000012 RBX: 00003d0509d82ff5 RCX: 00003d0509d82ff5
[32476.847028] RDX: 000055a5dbb578bb RSI: 0000000000000001 RDI: 0000000000000000
[32476.847031] RBP: 00007fff34882370 R08: 0000000000000000 R09: 0000000000000000
[32476.847034] R10: 00003d0500000000 R11: ffffffff00000000 R12: 0000000000000023
[32476.847037] R13: 0000376895df40a0 R14: 00003d0509d82f7d R15: 0000376895df4080
[32476.849645] watchdog: BUG: soft lockup - CPU#1 stuck for 22s!
[Chrome_ChildIOT:5472]
[32476.849652] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.849687] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.849713] irq event stamp: 0
[32476.849715] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.849719] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.849723] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.849726] softirqs last disabled at (0): [<0000000000000000>] 0x0
[32476.849728] CPU: 1 PID: 5472 Comm: Chrome_ChildIOT Tainted: G
D W L --------- --- 5.11.0-155.fc35.x86_64+debug #1
[32476.849732] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32476.849734] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[32476.849738] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 80 f1 1e 00 48 03 04 fd 00 39 6e a8 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[32476.849741] RSP: 0000:ffffb08fc6c4bc10 EFLAGS: 00000246
[32476.849744] RAX: 0000000000000000 RBX: ffffe16d96c11140 RCX: ffff9e53c6def180
[32476.849746] RDX: ffff9e4cc11a3d28 RSI: 0000000000080000 RDI: 0000000000000016
[32476.849749] RBP: ffff9e4cc11a3d28 R08: 0000000000080000 R09: 0000000000000000
[32476.849751] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9e4cc11a3d40
[32476.849753] R13: ffff9e4cc11a3d28 R14: ffff9e4cc11a3d20 R15: 0000000000f89940
[32476.849756] FS: 00007f9a02233640(0000) GS:ffff9e53c6c00000(0000)
knlGS:0000000000000000
[32476.849758] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32476.849761] CR2: 00002312fd44ecd8 CR3: 00000001618f6000 CR4: 0000000000350ee0
[32476.849763] Call Trace:
[32476.849766] do_raw_spin_lock+0x94/0xa0
[32476.849769] _raw_spin_lock+0x63/0x80
[32476.849772] zswap_frontswap_load+0x2f/0x2f0
[32476.849775] ? psi_group_change+0x27d/0x290
[32476.849779] __frontswap_load+0xc3/0x160
[32476.849782] swap_readpage+0x1ca/0x3a0
[32476.849786] swapin_readahead+0x450/0x4e0
[32476.849789] ? lock_release+0x1e9/0x400
[32476.849793] do_swap_page+0x4a4/0x900
[32476.849796] ? lock_release+0x1e9/0x400
[32476.849799] ? trace_hardirqs_on+0x1b/0xe0
[32476.849802] handle_mm_fault+0xe7d/0x19d0
[32476.849807] do_user_addr_fault+0x1c7/0x4c0
[32476.849810] exc_page_fault+0x67/0x2a0
[32476.849813] ? asm_exc_page_fault+0x8/0x30
[32476.849816] asm_exc_page_fault+0x1e/0x30
[32476.849819] RIP: 0033:0x555d3e644fe2
[32476.849822] Code: c3 cc cc cc cc cc cc cc 55 48 89 e5 41 57 41 56
53 48 83 ec 68 49 89 fe 4c 8b 3f 48 8b 05 76 4d bd 08 49 8b 1f 48 31
c3 74 67 <48> 33 43 08 49 39 c7 74 4e c7 45 b8 04 00 00 00 c7 45 c8 04
00 00
[32476.849824] RSP: 002b:00007f9a02231ab0 EFLAGS: 00010202
[32476.849827] RAX: fffffffd55160cdb RBX: 00002312fd44ecd0 RCX: 0000000000000005
[32476.850461] RDX: 0000000000000000 RSI: 0000000000000005 RDI: 00002312fcd3f2a0
[32476.850463] RBP: 00007f9a02231b30 R08: 00002312fcfa4003 R09: 00007ffd204a88d0
[32476.850465] R10: 0000000000000000 R11: 0000000000000246 R12: 0000555d3e492590
[32476.850467] R13: 0000555d3e4cd840 R14: 00002312fcd3f2a0 R15: 00002312fd2481e0
[32476.850644] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [brave:5451]
[32476.850652] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.850687] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.850714] irq event stamp: 0
[32476.850716] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.850719] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.850723] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.850726] softirqs last disabled at (0): [<0000000000000000>] 0x0
Full kernel log is here: https://pastebin.com/4SbhNp7V
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-02-28 13:22 ` Mikhail Gavrilov
@ 2021-03-01 3:11 ` Hillf Danton
2021-03-05 9:33 ` Mikhail Gavrilov
0 siblings, 1 reply; 10+ messages in thread
From: Hillf Danton @ 2021-03-01 3:11 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Sun, 28 Feb 2021 18:22:21 +0500 Mikhail Gavrilov wrote:
> On Sat, 13 Feb 2021 at 08:03, Hillf Danton <hdanton@sina.com> wrote:
> >
> > The comment below shows a race instance, though I failed to put things
> > together to see how within two hours. Cut it and see what will come up.
> >
> > --- a/mm/z3fold.c
> > +++ b/mm/z3fold.c
> > @@ -1129,19 +1129,22 @@ retry:
> > page = NULL;
> > if (can_sleep) {
> > spin_lock(&pool->stale_lock);
> > + spin_lock(&pool->lock);
> > zhdr = list_first_entry_or_null(&pool->stale,
> > struct z3fold_header, buddy);
> > /*
> > - * Before allocating a page, let's see if we can take one from
> > + * Before allocating a page, lets see if we can take one from
> > * the stale pages list. cancel_work_sync() can sleep so we
> > * limit this case to the contexts where we can sleep
> > */
> > if (zhdr) {
> > list_del(&zhdr->buddy);
> > + spin_unlock(&pool->lock);
> > spin_unlock(&pool->stale_lock);
> > cancel_work_sync(&zhdr->work);
> > page = virt_to_page(zhdr);
> > } else {
> > + spin_unlock(&pool->lock);
> > spin_unlock(&pool->stale_lock);
> > }
> > }
>
>
> Hi,
> It happened again with the patch above.
Thanks again.
> Is anything cleared up now?
See below.
>
> [32451.229358] list_add corruption. next->prev should be prev
> (ffffd08fbc661cd0), but was ffffffffa7643650. (next=ffff9e4d2848f1c0).
> [32451.229395] ------------[ cut here ]------------
> [32451.229398] kernel BUG at lib/list_debug.c:23!
> [32451.229408] invalid opcode: 0000 [#1] SMP NOPTI
> [32451.229414] CPU: 4 PID: 80665 Comm: kworker/u64:0 Tainted: G
> W --------- --- 5.11.0-155.fc35.x86_64+debug #1
> [32451.229420] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [32451.229424] Workqueue: zswap3 compact_page_work
> [32451.229433] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [32451.229439] Code: 48 c7 c6 24 26 64 a8 48 89 ef 49 c7 c7 ea ff ff
> ff e8 e8 71 01 00 e9 fa 10 9e ff 4c 89 c1 48 c7 c7 f0 26 64 a8 e8 50
> 12 fe ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 a0 27 64 a8 e8 39
> 12 fe
> [32451.229444] RSP: 0018:ffffb08fd553fde0 EFLAGS: 00010286
> [32451.229449] RAX: 0000000000000075 RBX: ffffe16d871550c0 RCX: 0000000000000000
> [32451.229453] RDX: ffff9e53c73e9f60 RSI: ffff9e53c73db2a0 RDI: ffff9e53c73db2a0
> [32451.229457] RBP: ffffd08fbc661cd0 R08: 0000000000000000 R09: ffffb08fd553fc20
> [32451.229460] R10: ffffb08fd553fc18 R11: 0000000000000000 R12: ffff9e4ce4e29008
> [32451.229464] R13: ffff9e4d85543010 R14: ffff9e4d2848f1c0 R15: ffff9e4d85543000
> [32451.229468] FS: 0000000000000000(0000) GS:ffff9e53c7200000(0000)
> knlGS:0000000000000000
> [32451.229472] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [32451.229476] CR2: 00000fb21455dfe8 CR3: 0000000142968000 CR4: 0000000000350ee0
> [32451.229480] Call Trace:
> [32451.229485] do_compact_page+0x28d/0xb60
> [32451.229492] ? debug_object_deactivate+0x55/0x140
> [32451.229499] ? lock_release+0x1e9/0x400
> [32451.229505] ? lock_release+0x1e9/0x400
> [32451.229511] process_one_work+0x2b0/0x5e0
> [32451.229519] worker_thread+0x55/0x3c0
> [32451.229524] ? process_one_work+0x5e0/0x5e0
> [32451.229531] kthread+0x13a/0x150
> [32451.229540] ? __kthread_bind_mask+0x60/0x60
> [32451.229548] ret_from_fork+0x22/0x30
> [32451.229558] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
> nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
> nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
> nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
> nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
> nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
> hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
> snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
> videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
> snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
> videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
> intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
> videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
> snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
> snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
> uas bluetooth ac97_bus
> [32451.229628] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
> snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
> eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
> wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
> ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
> crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
> ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
> pinctrl_amd fuse
> [32451.229696] ---[ end trace 80d86d6942435514 ]---
[...]
> Full kernel log is here: https://pastebin.com/4SbhNp7V
>
> --
> Best Regards,
> Mike Gavrilov.
What we learn from your reports is
1/ in z3fold_free(), kref_put() creates the ground zero for the race
cases reported,
2/ the stale_lock in combination with lock makes things more
complicated than thought.
Instead of dropping something in the zero spot, the fix below goes the
road mentioned before in this mail thread - add another list_head in
parallel to the buddy and s/buddy/stale_node/ under every case of
stale_lock.
--- x/mm/z3fold.c
+++ y/mm/z3fold.c
@@ -127,6 +127,7 @@ struct z3fold_header {
unsigned short first_num:2;
unsigned short mapped_count:2;
unsigned short foreign_handles:2;
+ struct list_head stale_node;
};
/**
@@ -429,6 +430,7 @@ static struct z3fold_header *init_z3fold
zhdr->slots = slots;
zhdr->pool = pool;
INIT_LIST_HEAD(&zhdr->buddy);
+ INIT_LIST_HEAD(&zhdr->stale_node);
INIT_WORK(&zhdr->work, compact_page_work);
return zhdr;
}
@@ -556,7 +558,7 @@ static void __release_z3fold_page(struct
z3fold_page_unlock(zhdr);
spin_lock(&pool->stale_lock);
- list_add(&zhdr->buddy, &pool->stale);
+ list_add(&zhdr->stale_node, &pool->stale);
queue_work(pool->release_wq, &pool->work);
spin_unlock(&pool->stale_lock);
}
@@ -598,10 +600,10 @@ static void free_pages_work(struct work_
spin_lock(&pool->stale_lock);
while (!list_empty(&pool->stale)) {
struct z3fold_header *zhdr = list_first_entry(&pool->stale,
- struct z3fold_header, buddy);
+ struct z3fold_header, stale_node);
struct page *page = virt_to_page(zhdr);
- list_del(&zhdr->buddy);
+ list_del(&zhdr->stale_node);
if (WARN_ON(!test_bit(PAGE_STALE, &page->private)))
continue;
spin_unlock(&pool->stale_lock);
@@ -1140,14 +1142,14 @@ retry:
if (can_sleep) {
spin_lock(&pool->stale_lock);
zhdr = list_first_entry_or_null(&pool->stale,
- struct z3fold_header, buddy);
+ struct z3fold_header, stale_node);
/*
* Before allocating a page, let's see if we can take one from
* the stale pages list. cancel_work_sync() can sleep so we
* limit this case to the contexts where we can sleep
*/
if (zhdr) {
- list_del(&zhdr->buddy);
+ list_del(&zhdr->stale_node);
spin_unlock(&pool->stale_lock);
cancel_work_sync(&zhdr->work);
page = virt_to_page(zhdr);
--
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-01 3:11 ` Hillf Danton
@ 2021-03-05 9:33 ` Mikhail Gavrilov
2021-03-05 14:22 ` Hillf Danton
0 siblings, 1 reply; 10+ messages in thread
From: Mikhail Gavrilov @ 2021-03-05 9:33 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Mon, 1 Mar 2021 at 08:11, Hillf Danton <hdanton@sina.com> wrote:
>
> What we learn from your reports is
>
> 1/ in z3fold_free(), kref_put() creates the ground zero for the race
> cases reported,
>
> 2/ the stale_lock in combination with lock makes things more
> complicated than thought.
>
> Instead of dropping something in the zero spot, the fix below goes the
> road mentioned before in this mail thread - add another list_head in
> parallel to the buddy and s/buddy/stale_node/ under every case of
> stale_lock.
>
> --- x/mm/z3fold.c
> +++ y/mm/z3fold.c
> @@ -127,6 +127,7 @@ struct z3fold_header {
> unsigned short first_num:2;
> unsigned short mapped_count:2;
> unsigned short foreign_handles:2;
> + struct list_head stale_node;
> };
>
> /**
> @@ -429,6 +430,7 @@ static struct z3fold_header *init_z3fold
> zhdr->slots = slots;
> zhdr->pool = pool;
> INIT_LIST_HEAD(&zhdr->buddy);
> + INIT_LIST_HEAD(&zhdr->stale_node);
> INIT_WORK(&zhdr->work, compact_page_work);
> return zhdr;
> }
> @@ -556,7 +558,7 @@ static void __release_z3fold_page(struct
> z3fold_page_unlock(zhdr);
>
> spin_lock(&pool->stale_lock);
> - list_add(&zhdr->buddy, &pool->stale);
> + list_add(&zhdr->stale_node, &pool->stale);
> queue_work(pool->release_wq, &pool->work);
> spin_unlock(&pool->stale_lock);
> }
> @@ -598,10 +600,10 @@ static void free_pages_work(struct work_
> spin_lock(&pool->stale_lock);
> while (!list_empty(&pool->stale)) {
> struct z3fold_header *zhdr = list_first_entry(&pool->stale,
> - struct z3fold_header, buddy);
> + struct z3fold_header, stale_node);
> struct page *page = virt_to_page(zhdr);
>
> - list_del(&zhdr->buddy);
> + list_del(&zhdr->stale_node);
> if (WARN_ON(!test_bit(PAGE_STALE, &page->private)))
> continue;
> spin_unlock(&pool->stale_lock);
> @@ -1140,14 +1142,14 @@ retry:
> if (can_sleep) {
> spin_lock(&pool->stale_lock);
> zhdr = list_first_entry_or_null(&pool->stale,
> - struct z3fold_header, buddy);
> + struct z3fold_header, stale_node);
> /*
> * Before allocating a page, let's see if we can take one from
> * the stale pages list. cancel_work_sync() can sleep so we
> * limit this case to the contexts where we can sleep
> */
> if (zhdr) {
> - list_del(&zhdr->buddy);
> + list_del(&zhdr->stale_node);
> spin_unlock(&pool->stale_lock);
> cancel_work_sync(&zhdr->work);
> page = virt_to_page(zhdr);
> --
The computer with patch above worked for a record time (3 days)
without freezing.
https://postimg.cc/VShF5cJN
But after 3 days hangs with follow trace:
[263314.718807] general protection fault, probably for non-canonical
address 0x72c1224000000000: 0000 [#1] SMP NOPTI
[263314.718828] CPU: 3 PID: 476750 Comm: Chrome_IOThread Tainted: G
W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263314.718831] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263314.718835] RIP: 0010:__list_add_valid+0x3/0x40
[263314.718841] Code: e9 5d ff ff ff b8 f4 ff ff ff e9 53 ff ff ff 48
c7 00 00 00 00 00 c7 40 08 00 00 00 00 e9 6b ff ff ff cc cc cc cc cc
49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 03 cf 62 00 4c 8b 0a 4d 39 c1 0f
85 1f
[263314.718845] RSP: 0018:ffffae4345b5fac0 EFLAGS: 00010282
[263314.718849] RAX: 00000000000003c0 RBX: ffffed09c02ade80 RCX:
0000000000000000
[263314.718851] RDX: 72c1224000000000 RSI: ffffce433c462004 RDI:
ffff9d414ab7a000
[263314.718853] RBP: ffffce433c462004 R08: 72c1224000000000 R09:
0000000000000000
[263314.718856] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053008
[263314.718858] R13: ffff9d414ab7a010 R14: 72c1224000000000 R15:
ffff9d414ab7a000
[263314.718860] FS: 00007f8ef636f640(0000) GS:ffff9d4947000000(0000)
knlGS:0000000000000000
[263314.718863] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263314.718865] CR2: 000055634cbef560 CR3: 00000002dbdbc000 CR4:
0000000000350ee0
[263314.718867] Call Trace:
[263314.718875] do_compact_page+0x28d/0xb60
[263314.718884] ? z3fold_zpool_free+0x3a8/0x590
[263314.718888] zswap_free_entry+0x43/0x70
[263314.718892] zswap_frontswap_invalidate_page+0x8c/0x90
[263314.718895] __frontswap_invalidate_page+0x5d/0x90
[263314.718898] swap_range_free+0xcd/0xf0
[263314.718901] swapcache_free_entries+0x128/0x1a0
[263314.718904] free_swap_slot+0xbb/0xd0
[263314.718907] __swap_entry_free+0x7a/0xa0
[263314.718910] free_swap_and_cache+0x35/0x80
[263314.718913] shmem_undo_range+0x188/0x7e0
[263314.718919] ? ldsem_down_read+0x1f/0x40
[263314.718925] shmem_evict_inode+0xe6/0x290
[263314.718928] ? lock_release+0x1ef/0x410
[263314.718932] ? var_wake_function+0x20/0x20
[263314.718936] evict+0xcf/0x1d0
[263314.718940] __dentry_kill+0xe8/0x190
[263314.718943] ? dput+0x20/0x480
[263314.718946] dput+0x2b8/0x480
[263314.718949] __fput+0x102/0x260
[263314.718952] task_work_run+0x5c/0xa0
[263314.718957] exit_to_user_mode_prepare+0x232/0x240
[263314.718960] syscall_exit_to_user_mode+0x27/0x70
[263314.718964] entry_SYSCALL_64_after_hwframe+0x44/0xae
[263314.718967] RIP: 0033:0x7f8f0b15d16b
[263314.718972] Code: 8b 15 09 7d 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff
ff ff eb 89 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 0b 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 7c 0c 00 f7 d8 64 89
01 48
[263314.718974] RSP: 002b:00007f8ef636d308 EFLAGS: 00000246 ORIG_RAX:
000000000000000b
[263314.718977] RAX: 0000000000000000 RBX: 00003e1813862928 RCX:
00007f8f0b15d16b
[263314.718979] RDX: 0000000000000000 RSI: 0000000000a4e000 RDI:
00007f8e5b43e000
[263314.718981] RBP: 00007f8ef636d320 R08: 0000000000000000 R09:
0000000000000000
[263314.718983] R10: 0000000000000000 R11: 0000000000000246 R12:
00007f8e5b43e000
[263314.718985] R13: 00003e18138628e0 R14: 00007f8ef636d330 R15:
00007f8ef636d330
[263314.718989] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263314.719032] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263314.719079] ---[ end trace ba885cda1af90fb7 ]---
[263314.719081] RIP: 0010:__list_add_valid+0x3/0x40
[263314.719084] Code: e9 5d ff ff ff b8 f4 ff ff ff e9 53 ff ff ff 48
c7 00 00 00 00 00 c7 40 08 00 00 00 00 e9 6b ff ff ff cc cc cc cc cc
49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 03 cf 62 00 4c 8b 0a 4d 39 c1 0f
85 1f
[263314.719086] RSP: 0018:ffffae4345b5fac0 EFLAGS: 00010282
[263314.719089] RAX: 00000000000003c0 RBX: ffffed09c02ade80 RCX:
0000000000000000
[263314.719091] RDX: 72c1224000000000 RSI: ffffce433c462004 RDI:
ffff9d414ab7a000
[263314.719093] RBP: ffffce433c462004 R08: 72c1224000000000 R09:
0000000000000000
[263314.719095] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053008
[263314.719097] R13: ffff9d414ab7a010 R14: 72c1224000000000 R15:
ffff9d414ab7a000
[263314.719099] FS: 00007f8ef636f640(0000) GS:ffff9d4947000000(0000)
knlGS:0000000000000000
[263314.719101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263314.719104] CR2: 000055634cbef560 CR3: 00000002dbdbc000 CR4:
0000000000350ee0
[263314.719106] note: Chrome_IOThread[476750] exited with preempt_count 5
[263341.868981] watchdog: BUG: soft lockup - CPU#0 stuck for 23s!
[ThreadPoolForeg:513140]
[263341.868991] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.869025] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.869052] irq event stamp: 0
[263341.869054] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[263341.869057] hardirqs last disabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.869061] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.869064] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.869067] CPU: 0 PID: 513140 Comm: ThreadPoolForeg Tainted: G
D W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.869070] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.869073] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[263341.869076] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[263341.869079] RSP: 0000:ffffae435505bbf0 EFLAGS: 00000246
[263341.869082] RAX: 0000000000000000 RBX: 0000000000e6c4f7 RCX:
ffff9d4946bef300
[263341.869084] RDX: ffff9d424197e4e8 RSI: 0000000000040000 RDI:
0000000000000014
[263341.869087] RBP: ffff9d424197e4e8 R08: 0000000000040000 R09:
0000000000000000
[263341.869089] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d424197e500
[263341.869091] R13: ffff9d424197e4e8 R14: ffffffffa9e87020 R15:
ffff9d424197e4e0
[263341.869094] FS: 00007ff2de694640(0000) GS:ffff9d4946a00000(0000)
knlGS:0000000000000000
[263341.869096] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.869099] CR2: 00000070bfa4d3c0 CR3: 00000002f2878000 CR4:
0000000000350ef0
[263341.869101] Call Trace:
[263341.869104] do_raw_spin_lock+0x94/0xa0
[263341.869107] _raw_spin_lock+0x63/0x80
[263341.869111] zswap_frontswap_load+0x30/0x2f0
[263341.869115] ? trace_hardirqs_on+0x1b/0xe0
[263341.869120] __frontswap_load+0xc3/0x160
[263341.869123] swap_readpage+0x25b/0x440
[263341.869127] swapin_readahead+0x450/0x4e0
[263341.869130] ? lock_release+0x1ef/0x410
[263341.869134] do_swap_page+0x4a4/0x900
[263341.869137] __handle_mm_fault+0xbd6/0x1610
[263341.869140] ? lock_acquire+0x177/0x3a0
[263341.869145] handle_mm_fault+0xa2/0x270
[263341.869148] do_user_addr_fault+0x1ea/0x6b0
[263341.869152] exc_page_fault+0x67/0x2a0
[263341.869155] ? asm_exc_page_fault+0x8/0x30
[263341.869158] asm_exc_page_fault+0x1e/0x30
[263341.869161] RIP: 0033:0x55e1b76f7713
[263341.869164] Code: 8b 9f c8 00 00 00 4d 8b b7 d0 00 00 00 4c 39 f3
74 5a 4c 8d 25 8e 18 3c 05 4c 8d 2d aa 59 e6 fb 0f 1f 80 00 00 00 00
48 8b 3b <48> 8b 07 48 89 c1 4c 29 e1 48 c1 c9 03 48 81 f9 9f 00 00 00
0f 87
[263341.869166] RSP: 002b:00007ff2de693430 EFLAGS: 00010287
[263341.869169] RAX: 0000000000000000 RBX: 00007ff2de693568 RCX:
00000070c00c7be0
[263341.869171] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
00000070bfa4d3c0
[263341.869174] RBP: 00007ff2de693480 R08: 0000000000000000 R09:
00000000000000ca
[263341.869176] R10: 00007fff78981080 R11: 00007fff78981090 R12:
000055e1bcab8f90
[263341.869178] R13: 000055e1b355d0b3 R14: 00007ff2de693578 R15:
00007ff2de693500
[263341.870981] watchdog: BUG: soft lockup - CPU#1 stuck for 23s!
[steamwebhelper:3496089]
[263341.870987] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.871021] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.871048] irq event stamp: 0
[263341.871050] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[263341.871054] hardirqs last disabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.871058] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.871061] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.871064] CPU: 1 PID: 3496089 Comm: steamwebhelper Tainted: G
D W L --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.871067] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.871069] RIP: 0010:native_queued_spin_lock_slowpath+0x137/0x200
[263341.871073] Code: e0 a9 1d 00 eb cb 41 83 c0 01 c1 e6 10 41 c1 e0
12 44 09 c6 89 f0 c1 e8 10 66 87 42 02 89 c7 c1 e7 10 75 73 31 ff eb
02 f3 90 <8b> 02 66 85 c0 75 f7 41 89 c0 66 45 31 c0 44 39 c6 0f 84 9b
00 00
[263341.871076] RSP: 0018:ffffae43549eb5f8 EFLAGS: 00000202
[263341.871078] RAX: 00000000000c0101 RBX: ffff9d414bc00000 RCX:
ffff9d4946def300
[263341.871081] RDX: ffff9d4253053008 RSI: 0000000000080000 RDI:
0000000000000000
[263341.871083] RBP: ffff9d4253053008 R08: 0000000000080000 R09:
0000000000000000
[263341.871085] R10: 0000000000000000 R11: 0000000000000001 R12:
ffff9d4253053020
[263341.871088] R13: ffff9d414bc00010 R14: 0000000000000000 R15:
ffffed09c02f0000
[263341.871090] FS: 00007f725d561d40(0000) GS:ffff9d4946c00000(0000)
knlGS:0000000000000000
[263341.871093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.871095] CR2: 0000366b51ea6fe8 CR3: 00000003865f2000 CR4:
0000000000350ee0
[263341.871098] Call Trace:
[263341.871101] do_raw_spin_lock+0x94/0xa0
[263341.871104] _raw_spin_lock+0x63/0x80
[263341.871107] z3fold_page_isolate+0xbd/0x1b0
[263341.871112] isolate_movable_page+0x94/0x180
[263341.871115] isolate_migratepages_block+0x5db/0x1120
[263341.871120] ? lock_release+0x1ef/0x410
[263341.871124] compact_zone+0x5a4/0xfd0
[263341.871129] compact_zone_order+0xaa/0xf0
[263341.871134] try_to_compact_pages+0x111/0x3b0
[263341.871138] __alloc_pages_direct_compact+0x79/0x210
[263341.871142] __alloc_pages_slowpath.constprop.0+0x1d0/0xf90
[263341.871147] ? __alloc_pages_nodemask+0x2e3/0x400
[263341.871151] ? lock_release+0x1ef/0x410
[263341.871154] __alloc_pages_nodemask+0x37d/0x400
[263341.871159] ttm_pool_alloc+0x2a3/0x630 [ttm]
[263341.871167] ttm_tt_populate+0x37/0xe0 [ttm]
[263341.871172] ttm_bo_handle_move_mem+0x13a/0x170 [ttm]
[263341.871179] ttm_bo_validate+0x15f/0x1b0 [ttm]
[263341.871184] ? lock_release+0x1ef/0x410
[263341.871189] ttm_bo_init_reserved+0x2f7/0x3e0 [ttm]
[263341.871195] amdgpu_bo_do_create+0x1a8/0x630 [amdgpu]
[263341.871312] ? amdgpu_bo_subtract_pin_size+0x50/0x50 [amdgpu]
[263341.871422] amdgpu_bo_create+0x30/0x2e0 [amdgpu]
[263341.871531] ? lock_acquire+0x177/0x3a0
[263341.871535] ? trace_hardirqs_on+0x1b/0xe0
[263341.871539] ? _raw_spin_unlock_irqrestore+0x37/0x40
[263341.871543] ? lock_release+0x1ef/0x410
[263341.871547] amdgpu_gem_create_ioctl+0x10e/0x370 [amdgpu]
[263341.871664] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
[263341.871774] drm_ioctl_kernel+0x89/0xe0 [drm]
[263341.871797] drm_ioctl+0x20f/0x3c0 [drm]
[263341.871816] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
[263341.871927] ? selinux_file_ioctl+0x147/0x200
[263341.871931] ? lock_acquired+0x200/0x390
[263341.871934] ? lock_release+0x1ef/0x410
[263341.871937] ? trace_hardirqs_on+0x1b/0xe0
[263341.871940] ? _raw_spin_unlock_irqrestore+0x37/0x40
[263341.871944] amdgpu_drm_ioctl+0x49/0x80 [amdgpu]
[263341.872053] __x64_sys_ioctl+0x82/0xb0
[263341.872058] do_syscall_64+0x33/0x40
[263341.872061] entry_SYSCALL_64_after_hwframe+0x44/0xae
[263341.872065] RIP: 0033:0x7f72610b22bb
[263341.872068] Code: ff ff ff 85 c0 79 9b 49 c7 c4 ff ff ff ff 5b 5d
4c 89 e0 41 5c c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 10 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 85 bb 0c 00 f7 d8 64 89
01 48
[263341.872071] RSP: 002b:00007ffcd94a01f8 EFLAGS: 00000246 ORIG_RAX:
0000000000000010
[263341.872074] RAX: ffffffffffffffda RBX: 00007ffcd94a0250 RCX:
00007f72610b22bb
[263341.872076] RDX: 00007ffcd94a0250 RSI: 00000000c0206440 RDI:
0000000000000016
[263341.872078] RBP: 00000000c0206440 R08: 0000000000000009 R09:
00000000000000b8
[263341.872081] R10: 00007ffcd9568080 R11: 0000000000000246 R12:
000008c86f1ae3c0
[263341.872083] R13: 0000000000000016 R14: 0000000000200000 R15:
00000000019c6000
[263341.872983] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kswapd0:288]
[263341.872991] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.873025] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.873052] irq event stamp: 36
[263341.873054] hardirqs last enabled at (35): [<ffffffffa8d61117>]
_raw_spin_unlock_irqrestore+0x37/0x40
[263341.873059] hardirqs last disabled at (36): [<ffffffffa8d5a8a9>]
__schedule+0x6e9/0xb20
[263341.873063] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.873066] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.873069] CPU: 2 PID: 288 Comm: kswapd0 Tainted: G D W L
--------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.873073] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.873075] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[263341.873079] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[263341.873082] RSP: 0018:ffffae4340943898 EFLAGS: 00000246
[263341.873085] RAX: 0000000000000000 RBX: ffff9d4253053000 RCX:
ffff9d4946fef300
[263341.873087] RDX: ffff9d4253053008 RSI: 00000000000c0000 RDI:
0000000000000001
[263341.873090] RBP: ffff9d4253053008 R08: 00000000000c0000 R09:
0000000000000000
[263341.873092] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053020
[263341.873094] R13: 0000000000000003 R14: 0000000000000003 R15:
ffff9d41760b2000
[263341.873097] FS: 0000000000000000(0000) GS:ffff9d4946e00000(0000)
knlGS:0000000000000000
[263341.873099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.873102] CR2: 0000021509b13000 CR3: 00000004130c8000 CR4:
0000000000350ee0
[263341.873104] Call Trace:
[263341.873107] do_raw_spin_lock+0x94/0xa0
[263341.873110] _raw_spin_lock+0x63/0x80
[263341.873114] __z3fold_alloc+0x78/0x3d0
[263341.873118] z3fold_zpool_malloc+0x4a5/0x7c0
[263341.873121] ? _raw_spin_unlock+0x1f/0x30
[263341.873125] zswap_frontswap_store+0x43e/0x890
[263341.873130] __frontswap_store+0xc8/0x170
[263341.873134] swap_writepage+0x39/0x70
[263341.873137] pageout+0x125/0x540
[263341.873142] shrink_page_list+0x131b/0x1bb0
[263341.873147] shrink_inactive_list+0x12a/0x440
[263341.873152] shrink_lruvec+0x4aa/0x6d0
[263341.873158] shrink_node+0x2d1/0x700
[263341.873163] balance_pgdat+0x2f5/0x650
[263341.873169] kswapd+0x21d/0x4d0
[263341.873172] ? do_wait_intr_irq+0xd0/0xd0
[263341.873176] ? balance_pgdat+0x650/0x650
[263341.873179] kthread+0x13a/0x150
[263341.873183] ? __kthread_bind_mask+0x60/0x60
[263341.873187] ret_from_fork+0x22/0x30
It related?
Full kernel log is here: https://pastebin.com/x0KbXN9L
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-05 9:33 ` Mikhail Gavrilov
@ 2021-03-05 14:22 ` Hillf Danton
2021-03-08 15:42 ` Mikhail Gavrilov
0 siblings, 1 reply; 10+ messages in thread
From: Hillf Danton @ 2021-03-05 14:22 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Fri, 5 Mar 2021 14:33:14 +0500 Mikhail Gavrilov wrote:
> On Mon, 1 Mar 2021 at 08:11, Hillf Danton <hdanton@sina.com> wrote:
> >
> > What we learn from your reports is
> >
> > 1/ in z3fold_free(), kref_put() creates the ground zero for the race
> > cases reported,
> >
> > 2/ the stale_lock in combination with lock makes things more
> > complicated than thought.
> >
> > Instead of dropping something in the zero spot, the fix below goes the
> > road mentioned before in this mail thread - add another list_head in
> > parallel to the buddy and s/buddy/stale_node/ under every case of
> > stale_lock.
> >
> > --- x/mm/z3fold.c
> > +++ y/mm/z3fold.c
> > @@ -127,6 +127,7 @@ struct z3fold_header {
> > unsigned short first_num:2;
> > unsigned short mapped_count:2;
> > unsigned short foreign_handles:2;
> > + struct list_head stale_node;
> > };
> >
> > /**
> > @@ -429,6 +430,7 @@ static struct z3fold_header *init_z3fold
> > zhdr->slots = slots;
> > zhdr->pool = pool;
> > INIT_LIST_HEAD(&zhdr->buddy);
> > + INIT_LIST_HEAD(&zhdr->stale_node);
> > INIT_WORK(&zhdr->work, compact_page_work);
> > return zhdr;
> > }
> > @@ -556,7 +558,7 @@ static void __release_z3fold_page(struct
> > z3fold_page_unlock(zhdr);
> >
> > spin_lock(&pool->stale_lock);
> > - list_add(&zhdr->buddy, &pool->stale);
> > + list_add(&zhdr->stale_node, &pool->stale);
> > queue_work(pool->release_wq, &pool->work);
> > spin_unlock(&pool->stale_lock);
> > }
> > @@ -598,10 +600,10 @@ static void free_pages_work(struct work_
> > spin_lock(&pool->stale_lock);
> > while (!list_empty(&pool->stale)) {
> > struct z3fold_header *zhdr = list_first_entry(&pool->stale,
> > - struct z3fold_header, buddy);
> > + struct z3fold_header, stale_node);
> > struct page *page = virt_to_page(zhdr);
> >
> > - list_del(&zhdr->buddy);
> > + list_del(&zhdr->stale_node);
> > if (WARN_ON(!test_bit(PAGE_STALE, &page->private)))
> > continue;
> > spin_unlock(&pool->stale_lock);
> > @@ -1140,14 +1142,14 @@ retry:
> > if (can_sleep) {
> > spin_lock(&pool->stale_lock);
> > zhdr = list_first_entry_or_null(&pool->stale,
> > - struct z3fold_header, buddy);
> > + struct z3fold_header, stale_node);
> > /*
> > * Before allocating a page, let's see if we can take one from
> > * the stale pages list. cancel_work_sync() can sleep so we
> > * limit this case to the contexts where we can sleep
> > */
> > if (zhdr) {
> > - list_del(&zhdr->buddy);
> > + list_del(&zhdr->stale_node);
> > spin_unlock(&pool->stale_lock);
> > cancel_work_sync(&zhdr->work);
> > page = virt_to_page(zhdr);
> > --
>
> The computer with patch above worked for a record time (3 days)
> without freezing.
> https://postimg.cc/VShF5cJN
>
>
> But after 3 days hangs with follow trace:
Thanks again for your report.
>
> [263314.718807] general protection fault, probably for non-canonical
> address 0x72c1224000000000: 0000 [#1] SMP NOPTI
> [263314.718828] CPU: 3 PID: 476750 Comm: Chrome_IOThread Tainted: G
> W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
> [263314.718831] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [263314.718835] RIP: 0010:__list_add_valid+0x3/0x40
> [263314.718841] Code: e9 5d ff ff ff b8 f4 ff ff ff e9 53 ff ff ff 48
> c7 00 00 00 00 00 c7 40 08 00 00 00 00 e9 6b ff ff ff cc cc cc cc cc
> 49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 03 cf 62 00 4c 8b 0a 4d 39 c1 0f
> 85 1f
> [263314.718845] RSP: 0018:ffffae4345b5fac0 EFLAGS: 00010282
> [263314.718849] RAX: 00000000000003c0 RBX: ffffed09c02ade80 RCX:
> 0000000000000000
> [263314.718851] RDX: 72c1224000000000 RSI: ffffce433c462004 RDI:
> ffff9d414ab7a000
> [263314.718853] RBP: ffffce433c462004 R08: 72c1224000000000 R09:
> 0000000000000000
> [263314.718856] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9d4253053008
> [263314.718858] R13: ffff9d414ab7a010 R14: 72c1224000000000 R15:
> ffff9d414ab7a000
> [263314.718860] FS: 00007f8ef636f640(0000) GS:ffff9d4947000000(0000)
> knlGS:0000000000000000
> [263314.718863] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [263314.718865] CR2: 000055634cbef560 CR3: 00000002dbdbc000 CR4:
> 0000000000350ee0
> [263314.718867] Call Trace:
> [263314.718875] do_compact_page+0x28d/0xb60
> [263314.718884] ? z3fold_zpool_free+0x3a8/0x590
One part of the race is the free path on CPU#3.
> [263314.718888] zswap_free_entry+0x43/0x70
> [263314.718892] zswap_frontswap_invalidate_page+0x8c/0x90
> [263314.718895] __frontswap_invalidate_page+0x5d/0x90
> [263314.718898] swap_range_free+0xcd/0xf0
> [263314.718901] swapcache_free_entries+0x128/0x1a0
> [263314.718904] free_swap_slot+0xbb/0xd0
> [263314.718907] __swap_entry_free+0x7a/0xa0
> [263314.718910] free_swap_and_cache+0x35/0x80
> [263314.718913] shmem_undo_range+0x188/0x7e0
> [263314.718919] ? ldsem_down_read+0x1f/0x40
> [263314.718925] shmem_evict_inode+0xe6/0x290
> [263314.718928] ? lock_release+0x1ef/0x410
> [263314.718932] ? var_wake_function+0x20/0x20
> [263314.718936] evict+0xcf/0x1d0
> [263314.718940] __dentry_kill+0xe8/0x190
> [263314.718943] ? dput+0x20/0x480
> [263314.718946] dput+0x2b8/0x480
> [263314.718949] __fput+0x102/0x260
> [263314.718952] task_work_run+0x5c/0xa0
> [263314.718957] exit_to_user_mode_prepare+0x232/0x240
> [263314.718960] syscall_exit_to_user_mode+0x27/0x70
> [263314.718964] entry_SYSCALL_64_after_hwframe+0x44/0xae
[...]
> [263341.868981] watchdog: BUG: soft lockup - CPU#0 stuck for 23s!
> [ThreadPoolForeg:513140]
[...]
> [263341.869052] irq event stamp: 0
> [263341.869054] hardirqs last enabled at (0): [<0000000000000000>] 0x0
> [263341.869057] hardirqs last disabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.869061] softirqs last enabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.869064] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [263341.869067] CPU: 0 PID: 513140 Comm: ThreadPoolForeg Tainted: G
> D W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
> [263341.869070] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [263341.869073] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
> [263341.869076] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
> 48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
> 09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
> 0d 0f
> [263341.869079] RSP: 0000:ffffae435505bbf0 EFLAGS: 00000246
> [263341.869082] RAX: 0000000000000000 RBX: 0000000000e6c4f7 RCX:
> ffff9d4946bef300
> [263341.869084] RDX: ffff9d424197e4e8 RSI: 0000000000040000 RDI:
> 0000000000000014
> [263341.869087] RBP: ffff9d424197e4e8 R08: 0000000000040000 R09:
> 0000000000000000
> [263341.869089] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9d424197e500
> [263341.869091] R13: ffff9d424197e4e8 R14: ffffffffa9e87020 R15:
> ffff9d424197e4e0
> [263341.869094] FS: 00007ff2de694640(0000) GS:ffff9d4946a00000(0000)
> knlGS:0000000000000000
> [263341.869096] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [263341.869099] CR2: 00000070bfa4d3c0 CR3: 00000002f2878000 CR4:
> 0000000000350ef0
> [263341.869101] Call Trace:
> [263341.869104] do_raw_spin_lock+0x94/0xa0
> [263341.869107] _raw_spin_lock+0x63/0x80
> [263341.869111] zswap_frontswap_load+0x30/0x2f0
A foot from z3fold on CPU#0.
> [263341.869115] ? trace_hardirqs_on+0x1b/0xe0
> [263341.869120] __frontswap_load+0xc3/0x160
> [263341.869123] swap_readpage+0x25b/0x440
> [263341.869127] swapin_readahead+0x450/0x4e0
> [263341.869130] ? lock_release+0x1ef/0x410
> [263341.869134] do_swap_page+0x4a4/0x900
> [263341.869137] __handle_mm_fault+0xbd6/0x1610
> [263341.869140] ? lock_acquire+0x177/0x3a0
> [263341.869145] handle_mm_fault+0xa2/0x270
> [263341.869148] do_user_addr_fault+0x1ea/0x6b0
> [263341.869152] exc_page_fault+0x67/0x2a0
> [263341.869155] ? asm_exc_page_fault+0x8/0x30
> [263341.869158] asm_exc_page_fault+0x1e/0x30
> [263341.869161] RIP: 0033:0x55e1b76f7713
> [263341.869164] Code: 8b 9f c8 00 00 00 4d 8b b7 d0 00 00 00 4c 39 f3
> 74 5a 4c 8d 25 8e 18 3c 05 4c 8d 2d aa 59 e6 fb 0f 1f 80 00 00 00 00
> 48 8b 3b <48> 8b 07 48 89 c1 4c 29 e1 48 c1 c9 03 48 81 f9 9f 00 00 00
> 0f 87
> [263341.869166] RSP: 002b:00007ff2de693430 EFLAGS: 00010287
> [263341.869169] RAX: 0000000000000000 RBX: 00007ff2de693568 RCX:
> 00000070c00c7be0
> [263341.869171] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> 00000070bfa4d3c0
> [263341.869174] RBP: 00007ff2de693480 R08: 0000000000000000 R09:
> 00000000000000ca
> [263341.869176] R10: 00007fff78981080 R11: 00007fff78981090 R12:
> 000055e1bcab8f90
> [263341.869178] R13: 000055e1b355d0b3 R14: 00007ff2de693578 R15:
> 00007ff2de693500
> [263341.870981] watchdog: BUG: soft lockup - CPU#1 stuck for 23s!
> [steamwebhelper:3496089]
> [263341.871048] irq event stamp: 0
> [263341.871050] hardirqs last enabled at (0): [<0000000000000000>] 0x0
> [263341.871054] hardirqs last disabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.871058] softirqs last enabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.871061] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [263341.871064] CPU: 1 PID: 3496089 Comm: steamwebhelper Tainted: G
> D W L --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
> [263341.871067] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [263341.871069] RIP: 0010:native_queued_spin_lock_slowpath+0x137/0x200
> [263341.871073] Code: e0 a9 1d 00 eb cb 41 83 c0 01 c1 e6 10 41 c1 e0
> 12 44 09 c6 89 f0 c1 e8 10 66 87 42 02 89 c7 c1 e7 10 75 73 31 ff eb
> 02 f3 90 <8b> 02 66 85 c0 75 f7 41 89 c0 66 45 31 c0 44 39 c6 0f 84 9b
> 00 00
> [263341.871076] RSP: 0018:ffffae43549eb5f8 EFLAGS: 00000202
> [263341.871078] RAX: 00000000000c0101 RBX: ffff9d414bc00000 RCX:
> ffff9d4946def300
> [263341.871081] RDX: ffff9d4253053008 RSI: 0000000000080000 RDI:
> 0000000000000000
> [263341.871083] RBP: ffff9d4253053008 R08: 0000000000080000 R09:
> 0000000000000000
> [263341.871085] R10: 0000000000000000 R11: 0000000000000001 R12:
> ffff9d4253053020
> [263341.871088] R13: ffff9d414bc00010 R14: 0000000000000000 R15:
> ffffed09c02f0000
> [263341.871090] FS: 00007f725d561d40(0000) GS:ffff9d4946c00000(0000)
> knlGS:0000000000000000
> [263341.871093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [263341.871095] CR2: 0000366b51ea6fe8 CR3: 00000003865f2000 CR4:
> 0000000000350ee0
> [263341.871098] Call Trace:
> [263341.871101] do_raw_spin_lock+0x94/0xa0
> [263341.871104] _raw_spin_lock+0x63/0x80
> [263341.871107] z3fold_page_isolate+0xbd/0x1b0
The isolate path on CPU#1.
> [263341.871112] isolate_movable_page+0x94/0x180
> [263341.871115] isolate_migratepages_block+0x5db/0x1120
> [263341.871120] ? lock_release+0x1ef/0x410
> [263341.871124] compact_zone+0x5a4/0xfd0
> [263341.871129] compact_zone_order+0xaa/0xf0
> [263341.871134] try_to_compact_pages+0x111/0x3b0
> [263341.871138] __alloc_pages_direct_compact+0x79/0x210
> [263341.871142] __alloc_pages_slowpath.constprop.0+0x1d0/0xf90
> [263341.871147] ? __alloc_pages_nodemask+0x2e3/0x400
> [263341.871151] ? lock_release+0x1ef/0x410
> [263341.871154] __alloc_pages_nodemask+0x37d/0x400
> [263341.871159] ttm_pool_alloc+0x2a3/0x630 [ttm]
> [263341.871167] ttm_tt_populate+0x37/0xe0 [ttm]
> [263341.871172] ttm_bo_handle_move_mem+0x13a/0x170 [ttm]
> [263341.871179] ttm_bo_validate+0x15f/0x1b0 [ttm]
> [263341.871184] ? lock_release+0x1ef/0x410
> [263341.871189] ttm_bo_init_reserved+0x2f7/0x3e0 [ttm]
> [263341.871195] amdgpu_bo_do_create+0x1a8/0x630 [amdgpu]
> [263341.871312] ? amdgpu_bo_subtract_pin_size+0x50/0x50 [amdgpu]
> [263341.871422] amdgpu_bo_create+0x30/0x2e0 [amdgpu]
> [263341.871531] ? lock_acquire+0x177/0x3a0
> [263341.871535] ? trace_hardirqs_on+0x1b/0xe0
> [263341.871539] ? _raw_spin_unlock_irqrestore+0x37/0x40
> [263341.871543] ? lock_release+0x1ef/0x410
> [263341.871547] amdgpu_gem_create_ioctl+0x10e/0x370 [amdgpu]
> [263341.871664] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
> [263341.871774] drm_ioctl_kernel+0x89/0xe0 [drm]
> [263341.871797] drm_ioctl+0x20f/0x3c0 [drm]
> [263341.871816] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
> [263341.871927] ? selinux_file_ioctl+0x147/0x200
> [263341.871931] ? lock_acquired+0x200/0x390
> [263341.871934] ? lock_release+0x1ef/0x410
> [263341.871937] ? trace_hardirqs_on+0x1b/0xe0
> [263341.871940] ? _raw_spin_unlock_irqrestore+0x37/0x40
> [263341.871944] amdgpu_drm_ioctl+0x49/0x80 [amdgpu]
> [263341.872053] __x64_sys_ioctl+0x82/0xb0
> [263341.872058] do_syscall_64+0x33/0x40
> [263341.872061] entry_SYSCALL_64_after_hwframe+0x44/0xae
> [263341.872065] RIP: 0033:0x7f72610b22bb
> [263341.872068] Code: ff ff ff 85 c0 79 9b 49 c7 c4 ff ff ff ff 5b 5d
> 4c 89 e0 41 5c c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 10 00 00
> 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 85 bb 0c 00 f7 d8 64 89
> 01 48
> [263341.872071] RSP: 002b:00007ffcd94a01f8 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000010
> [263341.872074] RAX: ffffffffffffffda RBX: 00007ffcd94a0250 RCX:
> 00007f72610b22bb
> [263341.872076] RDX: 00007ffcd94a0250 RSI: 00000000c0206440 RDI:
> 0000000000000016
> [263341.872078] RBP: 00000000c0206440 R08: 0000000000000009 R09:
> 00000000000000b8
> [263341.872081] R10: 00007ffcd9568080 R11: 0000000000000246 R12:
> 000008c86f1ae3c0
> [263341.872083] R13: 0000000000000016 R14: 0000000000200000 R15:
> 00000000019c6000
> [263341.872983] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kswapd0:288]
> [263341.873052] irq event stamp: 36
> [263341.873054] hardirqs last enabled at (35): [<ffffffffa8d61117>]
> _raw_spin_unlock_irqrestore+0x37/0x40
> [263341.873059] hardirqs last disabled at (36): [<ffffffffa8d5a8a9>]
> __schedule+0x6e9/0xb20
> [263341.873063] softirqs last enabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.873066] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [263341.873069] CPU: 2 PID: 288 Comm: kswapd0 Tainted: G D W L
> --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
> [263341.873073] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [263341.873075] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
> [263341.873079] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
> 48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
> 09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
> 0d 0f
> [263341.873082] RSP: 0018:ffffae4340943898 EFLAGS: 00000246
> [263341.873085] RAX: 0000000000000000 RBX: ffff9d4253053000 RCX:
> ffff9d4946fef300
> [263341.873087] RDX: ffff9d4253053008 RSI: 00000000000c0000 RDI:
> 0000000000000001
> [263341.873090] RBP: ffff9d4253053008 R08: 00000000000c0000 R09:
> 0000000000000000
> [263341.873092] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9d4253053020
> [263341.873094] R13: 0000000000000003 R14: 0000000000000003 R15:
> ffff9d41760b2000
> [263341.873097] FS: 0000000000000000(0000) GS:ffff9d4946e00000(0000)
> knlGS:0000000000000000
> [263341.873099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [263341.873102] CR2: 0000021509b13000 CR3: 00000004130c8000 CR4:
> 0000000000350ee0
> [263341.873104] Call Trace:
> [263341.873107] do_raw_spin_lock+0x94/0xa0
> [263341.873110] _raw_spin_lock+0x63/0x80
> [263341.873114] __z3fold_alloc+0x78/0x3d0
The alloc path on CPU#2.
> [263341.873118] z3fold_zpool_malloc+0x4a5/0x7c0
> [263341.873121] ? _raw_spin_unlock+0x1f/0x30
> [263341.873125] zswap_frontswap_store+0x43e/0x890
> [263341.873130] __frontswap_store+0xc8/0x170
> [263341.873134] swap_writepage+0x39/0x70
> [263341.873137] pageout+0x125/0x540
> [263341.873142] shrink_page_list+0x131b/0x1bb0
> [263341.873147] shrink_inactive_list+0x12a/0x440
> [263341.873152] shrink_lruvec+0x4aa/0x6d0
> [263341.873158] shrink_node+0x2d1/0x700
> [263341.873163] balance_pgdat+0x2f5/0x650
> [263341.873169] kswapd+0x21d/0x4d0
> [263341.873172] ? do_wait_intr_irq+0xd0/0xd0
> [263341.873176] ? balance_pgdat+0x650/0x650
> [263341.873179] kthread+0x13a/0x150
> [263341.873183] ? __kthread_bind_mask+0x60/0x60
> [263341.873187] ret_from_fork+0x22/0x30
>
>
> It related?
Yes, it is the same race as we saw before. But after cutting the race
between poo->stale_lock and pool->lock with the patch above, the race
between the free path and isolate/putback path came up.
>
>
> Full kernel log is here: https://pastebin.com/x0KbXN9L
>
>
> --
> Best Regards,
> Mike Gavrilov.
Try the diff below in combination with the patch above
--- x/mm/z3fold.c
+++ y/mm/z3fold.c
@@ -1676,8 +1676,10 @@ static void z3fold_page_putback(struct p
pool = zhdr_to_pool(zhdr);
z3fold_page_lock(zhdr);
+ spin_lock(&pool->lock);
if (!list_empty(&zhdr->buddy))
list_del_init(&zhdr->buddy);
+ spin_unlock(&pool->lock);
INIT_LIST_HEAD(&page->lru);
if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) {
atomic64_dec(&pool->pages_nr);
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-05 14:22 ` Hillf Danton
@ 2021-03-08 15:42 ` Mikhail Gavrilov
2021-03-09 2:31 ` Hillf Danton
0 siblings, 1 reply; 10+ messages in thread
From: Mikhail Gavrilov @ 2021-03-08 15:42 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Fri, 5 Mar 2021 at 19:22, Hillf Danton <hdanton@sina.com> wrote:
>
> Yes, it is the same race as we saw before. But after cutting the race
> between poo->stale_lock and pool->lock with the patch above, the race
> between the free path and isolate/putback path came up.
>
> Try the diff below in combination with the patch above
>
> --- x/mm/z3fold.c
> +++ y/mm/z3fold.c
> @@ -1676,8 +1676,10 @@ static void z3fold_page_putback(struct p
> pool = zhdr_to_pool(zhdr);
>
> z3fold_page_lock(zhdr);
> + spin_lock(&pool->lock);
> if (!list_empty(&zhdr->buddy))
> list_del_init(&zhdr->buddy);
> + spin_unlock(&pool->lock);
> INIT_LIST_HEAD(&page->lru);
> if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) {
> atomic64_dec(&pool->pages_nr);
Unfortunately even with combination of two latest patches computer
hanged again after two days uptime.
[185000.747401] list_add corruption. next->prev should be prev
(ffffe0c1bea61f40), but was 0000000000000000. (next=ffff9bb90b444000).
[185000.747438] ------------[ cut here ]------------
[185000.747441] kernel BUG at lib/list_debug.c:23!
[185000.747449] invalid opcode: 0000 [#1] SMP NOPTI
[185000.747454] CPU: 22 PID: 1588003 Comm: Web Content Tainted: G
W --------- ---
5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
[185000.747458] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[185000.747462] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[185000.747469] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
fd fd
[185000.747472] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
[185000.747476] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
0000000000000000
[185000.747479] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
ffff9bbc097daae0
[185000.747482] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
ffffc0c1c61cfa58
[185000.747485] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
ffff9bb537b4f008
[185000.747488] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
ffff9bba5ac29000
[185000.747491] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
knlGS:0000000000000000
[185000.747495] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185000.747498] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
0000000000350ee0
[185000.747501] Call Trace:
[185000.747504] do_compact_page+0x28d/0xb60
[185000.747509] ? _raw_spin_unlock+0x1f/0x30
[185000.747514] ? z3fold_zpool_free+0x3a8/0x590
[185000.747518] zswap_free_entry+0x43/0x70
[185000.747523] zswap_frontswap_invalidate_page+0x8c/0x90
[185000.747527] __frontswap_invalidate_page+0x5d/0x90
[185000.747531] swap_range_free+0xcd/0xf0
[185000.747535] swapcache_free_entries+0x128/0x1a0
[185000.747539] free_swap_slot+0xbb/0xd0
[185000.747543] __swap_entry_free+0x7a/0xa0
[185000.747547] do_swap_page+0x393/0x900
[185000.747551] __handle_mm_fault+0xbd6/0x1610
[185000.747557] handle_mm_fault+0xa2/0x270
[185000.747561] do_user_addr_fault+0x1ea/0x6b0
[185000.747566] exc_page_fault+0x67/0x2a0
[185000.747570] ? asm_exc_page_fault+0x8/0x30
[185000.747574] asm_exc_page_fault+0x1e/0x30
[185000.747578] RIP: 0033:0x7f198eb8be30
[185000.747582] Code: 9d 48 81 fa 80 00 00 00 77 19 c5 fe 7f 07 c5 fe
7f 47 20 c5 fe 7f 44 17 e0 c5 fe 7f 44 17 c0 c5 f8 77 c3 48 8d 8f 80
00 00 00 <c5> fe 7f 07 48 83 e1 80 c5 fe 7f 44 17 e0 c5 fe 7f 47 20 c5
fe 7f
[185000.747585] RSP: 002b:00007ffea7e406e8 EFLAGS: 00010202
[185000.747589] RAX: 00000fe3aa523500 RBX: 00007ffea7e40738 RCX:
00000fe3aa523580
[185000.747592] RDX: 0000000000004000 RSI: 00000000000000fa RDI:
00000fe3aa523500
[185000.747594] RBP: 0000600000000000 R08: 00007f195295a000 R09:
ffffffffffffffff
[185000.747597] R10: 0000556a267402c8 R11: 0000000000000206 R12:
00007f195295a800
[185000.747600] R13: fffffc0000000000 R14: 00007ffea7e40738 R15:
00007f195295a7f0
[185000.747605] Modules linked in: crypto_user tun snd_seq_dummy
snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
mc bluetooth rapl ff_memless snd_pcm
[185000.747647] iwlwifi asus_wmi sparse_keymap video snd_timer
ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[185000.747878] ---[ end trace df51d3d2498d767d ]---
[185000.747882] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[185000.747886] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
fd fd
[185000.747889] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
[185000.747893] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
0000000000000000
[185000.747895] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
ffff9bbc097daae0
[185000.747898] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
ffffc0c1c61cfa58
[185000.747901] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
ffff9bb537b4f008
[185000.747904] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
ffff9bba5ac29000
[185000.747907] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
knlGS:0000000000000000
[185000.747910] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185000.747913] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
0000000000350ee0
[185000.747916] note: Web Content[1588003] exited with preempt_count 6
[185026.580248] watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
[Chrome_ChildIOT:1951362]
[185026.580262] Modules linked in: crypto_user tun snd_seq_dummy
snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
mc bluetooth rapl ff_memless snd_pcm
[185026.580306] iwlwifi asus_wmi sparse_keymap video snd_timer
ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[185026.580334] irq event stamp: 0
[185026.580337] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[185026.580342] hardirqs last disabled at (0): [<ffffffff830dd962>]
copy_process+0x902/0x1df0
[185026.580349] softirqs last enabled at (0): [<ffffffff830dd962>]
copy_process+0x902/0x1df0
[185026.580353] softirqs last disabled at (0): [<0000000000000000>] 0x0
[185026.580357] CPU: 0 PID: 1951362 Comm: Chrome_ChildIOT Tainted: G
D W --------- ---
5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
[185026.580362] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[185026.580365] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[185026.580370] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e 84 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[185026.580374] RSP: 0000:ffffc0c1d596fbf0 EFLAGS: 00000246
[185026.580379] RAX: 0000000000000000 RBX: 0000000000f10bea RCX:
ffff9bbc06bef300
[185026.580382] RDX: ffff9bb500b9fb48 RSI: 0000000000040000 RDI:
0000000000000013
[185026.580385] RBP: ffff9bb500b9fb48 R08: 0000000000040000 R09:
0000000000000000
[185026.580388] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9bb500b9fb60
[185026.580391] R13: ffff9bb500b9fb48 R14: ffffffff84e87020 R15:
ffff9bb500b9fb40
[185026.580394] FS: 00007f2e2ffe6640(0000) GS:ffff9bbc06a00000(0000)
knlGS:0000000000000000
[185026.580398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185026.580401] CR2: 00002a8e6ef59188 CR3: 0000000306082000 CR4:
0000000000350ef0
[185026.580405] Call Trace:
[185026.580408] do_raw_spin_lock+0x94/0xa0
[185026.580665] _raw_spin_lock+0x63/0x80
[185026.580670] zswap_frontswap_load+0x30/0x2f0
[185026.580676] ? trace_hardirqs_on+0x1b/0xe0
[185026.580681] __frontswap_load+0xc3/0x160
[185026.580685] swap_readpage+0x257/0x430
[185026.580689] swapin_readahead+0x450/0x4e0
[185026.580693] ? lock_release+0x1ef/0x410
[185026.580698] do_swap_page+0x4a4/0x900
[185026.580703] __handle_mm_fault+0xbd6/0x1610
[185026.580795] handle_mm_fault+0xa2/0x270
[185026.580799] do_user_addr_fault+0x1ea/0x6b0
[185026.580804] exc_page_fault+0x67/0x2a0
[185026.580808] ? asm_exc_page_fault+0x8/0x30
[185026.580889] asm_exc_page_fault+0x1e/0x30
[185026.580893] RIP: 0033:0x55d9d6466038
[185026.580897] Code: cc cc 55 48 89 e5 48 89 7f 10 48 89 7f 18 5d c3
cc cc 55 48 89 e5 48 8b 47 10 48 8b 4f 18 48 89 41 10 48 8b 47 10 48
8b 4f 18 <48> 89 48 18 0f 57 c0 0f 11 47 10 5d c3 cc cc cc cc cc cc cc
cc cc
[185026.580900] RSP: 002b:00007f2e2ffe46b0 EFLAGS: 00010246
[185026.580968] RAX: 00002a8e6ef59170 RBX: 00002a8e6ef40420 RCX:
00002a8e6ef4e370
[185026.580972] RDX: 00002a8e6d6715e0 RSI: 00002a8e6dcb02e0 RDI:
00002a8e6ef40420
[185026.580974] RBP: 00007f2e2ffe46b0 R08: 0000000000000000 R09:
00007fff260af5d0
[185026.581039] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000020
[185026.581042] R13: 00002a8e6dcb02e0 R14: 000055d9deb9b1e0 R15:
00002a8e6dcb02e0
Full kernel log is here: https://pastebin.com/WmBLJ3MR
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-08 15:42 ` Mikhail Gavrilov
@ 2021-03-09 2:31 ` Hillf Danton
2021-03-15 19:18 ` Mikhail Gavrilov
2021-03-15 19:21 ` Mikhail Gavrilov
0 siblings, 2 replies; 10+ messages in thread
From: Hillf Danton @ 2021-03-09 2:31 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Mon, 8 Mar 2021 20:42:42 +0500 Mikhail Gavrilov wrote:
> On Fri, 5 Mar 2021 at 19:22, Hillf Danton <hdanton@sina.com> wrote:
> >
> > Yes, it is the same race as we saw before. But after cutting the race
> > between poo->stale_lock and pool->lock with the patch above, the race
> > between the free path and isolate/putback path came up.
> >
> > Try the diff below in combination with the patch above
> >
> > --- x/mm/z3fold.c
> > +++ y/mm/z3fold.c
> > @@ -1676,8 +1676,10 @@ static void z3fold_page_putback(struct p
> > pool = zhdr_to_pool(zhdr);
> >
> > z3fold_page_lock(zhdr);
> > + spin_lock(&pool->lock);
> > if (!list_empty(&zhdr->buddy))
> > list_del_init(&zhdr->buddy);
> > + spin_unlock(&pool->lock);
> > INIT_LIST_HEAD(&page->lru);
> > if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) {
> > atomic64_dec(&pool->pages_nr);
>
> Unfortunately even with combination of two latest patches computer
> hanged again after two days uptime.
Thanks again for your report.
>
> [185000.747401] list_add corruption. next->prev should be prev
> (ffffe0c1bea61f40), but was 0000000000000000. (next=ffff9bb90b444000).
At the first glance, the zero pointer goes out of the box of race because
1/ the Call Trace shows it is the free path (of the supposed race victim),
2/ on the race winner side however either list_del or list_del_init
would not leave a null pointer behind - the list_add captured in this
report is under pool->lock.
> [185000.747438] ------------[ cut here ]------------
> [185000.747441] kernel BUG at lib/list_debug.c:23!
> [185000.747449] invalid opcode: 0000 [#1] SMP NOPTI
> [185000.747454] CPU: 22 PID: 1588003 Comm: Web Content Tainted: G
> W --------- ---
> 5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
> [185000.747458] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [185000.747462] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [185000.747469] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
> ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
> fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
> fd fd
> [185000.747472] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
> [185000.747476] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
> 0000000000000000
> [185000.747479] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
> ffff9bbc097daae0
> [185000.747482] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
> ffffc0c1c61cfa58
> [185000.747485] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
> ffff9bb537b4f008
> [185000.747488] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
> ffff9bba5ac29000
> [185000.747491] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
> knlGS:0000000000000000
> [185000.747495] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [185000.747498] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
> 0000000000350ee0
> [185000.747501] Call Trace:
> [185000.747504] do_compact_page+0x28d/0xb60
> [185000.747509] ? _raw_spin_unlock+0x1f/0x30
> [185000.747514] ? z3fold_zpool_free+0x3a8/0x590
> [185000.747518] zswap_free_entry+0x43/0x70
> [185000.747523] zswap_frontswap_invalidate_page+0x8c/0x90
> [185000.747527] __frontswap_invalidate_page+0x5d/0x90
> [185000.747531] swap_range_free+0xcd/0xf0
> [185000.747535] swapcache_free_entries+0x128/0x1a0
> [185000.747539] free_swap_slot+0xbb/0xd0
> [185000.747543] __swap_entry_free+0x7a/0xa0
> [185000.747547] do_swap_page+0x393/0x900
> [185000.747551] __handle_mm_fault+0xbd6/0x1610
> [185000.747557] handle_mm_fault+0xa2/0x270
> [185000.747561] do_user_addr_fault+0x1ea/0x6b0
> [185000.747566] exc_page_fault+0x67/0x2a0
> [185000.747570] ? asm_exc_page_fault+0x8/0x30
> [185000.747574] asm_exc_page_fault+0x1e/0x30
> [185000.747578] RIP: 0033:0x7f198eb8be30
> [185000.747582] Code: 9d 48 81 fa 80 00 00 00 77 19 c5 fe 7f 07 c5 fe
> 7f 47 20 c5 fe 7f 44 17 e0 c5 fe 7f 44 17 c0 c5 f8 77 c3 48 8d 8f 80
> 00 00 00 <c5> fe 7f 07 48 83 e1 80 c5 fe 7f 44 17 e0 c5 fe 7f 47 20 c5
> fe 7f
> [185000.747585] RSP: 002b:00007ffea7e406e8 EFLAGS: 00010202
> [185000.747589] RAX: 00000fe3aa523500 RBX: 00007ffea7e40738 RCX:
> 00000fe3aa523580
> [185000.747592] RDX: 0000000000004000 RSI: 00000000000000fa RDI:
> 00000fe3aa523500
> [185000.747594] RBP: 0000600000000000 R08: 00007f195295a000 R09:
> ffffffffffffffff
> [185000.747597] R10: 0000556a267402c8 R11: 0000000000000206 R12:
> 00007f195295a800
> [185000.747600] R13: fffffc0000000000 R14: 00007ffea7e40738 R15:
> 00007f195295a7f0
> [185000.747605] Modules linked in: crypto_user tun snd_seq_dummy
> snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
> nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
> nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
> zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
> snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
> snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
> snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
> uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
> edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
> videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
> kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
> snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
> mc bluetooth rapl ff_memless snd_pcm
> [185000.747647] iwlwifi asus_wmi sparse_keymap video snd_timer
> ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
> k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
> usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
> crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
> ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
> pinctrl_amd fuse
> [185000.747878] ---[ end trace df51d3d2498d767d ]---
> [185000.747882] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [185000.747886] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
> ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
> fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
> fd fd
> [185000.747889] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
> [185000.747893] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
> 0000000000000000
> [185000.747895] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
> ffff9bbc097daae0
> [185000.747898] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
> ffffc0c1c61cfa58
> [185000.747901] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
> ffff9bb537b4f008
> [185000.747904] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
> ffff9bba5ac29000
> [185000.747907] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
> knlGS:0000000000000000
> [185000.747910] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [185000.747913] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
> 0000000000350ee0
> [185000.747916] note: Web Content[1588003] exited with preempt_count 6
> [185026.580248] watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
> [Chrome_ChildIOT:1951362]
> [185026.580262] Modules linked in: crypto_user tun snd_seq_dummy
> snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
> nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
> nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
> zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
> snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
> snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
> snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
> uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
> edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
> videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
> kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
> snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
> mc bluetooth rapl ff_memless snd_pcm
> [185026.580306] iwlwifi asus_wmi sparse_keymap video snd_timer
> ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
> k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
> usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
> crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
> ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
> pinctrl_amd fuse
> [185026.580334] irq event stamp: 0
> [185026.580337] hardirqs last enabled at (0): [<0000000000000000>] 0x0
> [185026.580342] hardirqs last disabled at (0): [<ffffffff830dd962>]
> copy_process+0x902/0x1df0
> [185026.580349] softirqs last enabled at (0): [<ffffffff830dd962>]
> copy_process+0x902/0x1df0
> [185026.580353] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [185026.580357] CPU: 0 PID: 1951362 Comm: Chrome_ChildIOT Tainted: G
> D W --------- ---
> 5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
> [185026.580362] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [185026.580365] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
> [185026.580370] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
> 48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e 84 48 89 08 8b 41 08 85 c0 75
> 09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
> 0d 0f
> [185026.580374] RSP: 0000:ffffc0c1d596fbf0 EFLAGS: 00000246
> [185026.580379] RAX: 0000000000000000 RBX: 0000000000f10bea RCX:
> ffff9bbc06bef300
> [185026.580382] RDX: ffff9bb500b9fb48 RSI: 0000000000040000 RDI:
> 0000000000000013
> [185026.580385] RBP: ffff9bb500b9fb48 R08: 0000000000040000 R09:
> 0000000000000000
> [185026.580388] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9bb500b9fb60
> [185026.580391] R13: ffff9bb500b9fb48 R14: ffffffff84e87020 R15:
> ffff9bb500b9fb40
> [185026.580394] FS: 00007f2e2ffe6640(0000) GS:ffff9bbc06a00000(0000)
> knlGS:0000000000000000
> [185026.580398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [185026.580401] CR2: 00002a8e6ef59188 CR3: 0000000306082000 CR4:
> 0000000000350ef0
> [185026.580405] Call Trace:
> [185026.580408] do_raw_spin_lock+0x94/0xa0
> [185026.580665] _raw_spin_lock+0x63/0x80
> [185026.580670] zswap_frontswap_load+0x30/0x2f0
> [185026.580676] ? trace_hardirqs_on+0x1b/0xe0
> [185026.580681] __frontswap_load+0xc3/0x160
> [185026.580685] swap_readpage+0x257/0x430
> [185026.580689] swapin_readahead+0x450/0x4e0
> [185026.580693] ? lock_release+0x1ef/0x410
> [185026.580698] do_swap_page+0x4a4/0x900
> [185026.580703] __handle_mm_fault+0xbd6/0x1610
> [185026.580795] handle_mm_fault+0xa2/0x270
> [185026.580799] do_user_addr_fault+0x1ea/0x6b0
> [185026.580804] exc_page_fault+0x67/0x2a0
> [185026.580808] ? asm_exc_page_fault+0x8/0x30
> [185026.580889] asm_exc_page_fault+0x1e/0x30
> [185026.580893] RIP: 0033:0x55d9d6466038
> [185026.580897] Code: cc cc 55 48 89 e5 48 89 7f 10 48 89 7f 18 5d c3
> cc cc 55 48 89 e5 48 8b 47 10 48 8b 4f 18 48 89 41 10 48 8b 47 10 48
> 8b 4f 18 <48> 89 48 18 0f 57 c0 0f 11 47 10 5d c3 cc cc cc cc cc cc cc
> cc cc
> [185026.580900] RSP: 002b:00007f2e2ffe46b0 EFLAGS: 00010246
> [185026.580968] RAX: 00002a8e6ef59170 RBX: 00002a8e6ef40420 RCX:
> 00002a8e6ef4e370
> [185026.580972] RDX: 00002a8e6d6715e0 RSI: 00002a8e6dcb02e0 RDI:
> 00002a8e6ef40420
> [185026.580974] RBP: 00007f2e2ffe46b0 R08: 0000000000000000 R09:
> 00007fff260af5d0
> [185026.581039] R10: 0000000000000000 R11: 0000000000000246 R12:
> 0000000000000020
> [185026.581042] R13: 00002a8e6dcb02e0 R14: 000055d9deb9b1e0 R15:
> 00002a8e6dcb02e0
>
> Full kernel log is here: https://pastebin.com/WmBLJ3MR
>
> --
> Best Regards,
> Mike Gavrilov.
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-09 2:31 ` Hillf Danton
@ 2021-03-15 19:18 ` Mikhail Gavrilov
2021-03-15 19:21 ` Mikhail Gavrilov
1 sibling, 0 replies; 10+ messages in thread
From: Mikhail Gavrilov @ 2021-03-15 19:18 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
[-- Attachment #1: Type: text/plain, Size: 18292 bytes --]
On Tue, 9 Mar 2021 at 07:31, Hillf Danton <hdanton@sina.com> wrote:
>
> At the first glance, the zero pointer goes out of the box of race because
>
> 1/ the Call Trace shows it is the free path (of the supposed race victim),
>
> 2/ on the race winner side however either list_del or list_del_init
> would not leave a null pointer behind - the list_add captured in this
> report is under pool->lock.
>
No more ideas how to fix it?
Kernel panics continue happens again and again with you patches and recent
commits.
[102491.134247] ------------[ cut here ]------------
[102491.134248] list_add corruption. next->prev should be prev
(ffffcc447ea60c78), but was ffffffff8a64ec20. (next=ffff8adc731d3f40).
[102491.134266] ODEBUG: free active (active state 0) object type:
work_struct hint: compact_page_work+0x0/0x10
[102491.134294] ------------[ cut here ]------------
[102491.134295] kernel BUG at lib/list_debug.c:23!
[102491.134299] invalid opcode: 0000 [#1] SMP NOPTI
[102491.134301] CPU: 22 PID: 863413 Comm: kworker/u64:0 Tainted: G
W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.134303] Hardware name: System manufacturer System Product Name/ROG
STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.134305] Workqueue: zswap3 compact_page_work
[102491.134309] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134312] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff ff e8
91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67 fd fd ff
<0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50 fd fd
[102491.134313] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134315] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134316] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.134316] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.134317] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.134318] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.134319] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.134320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.134320] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.134321] Call Trace:
[102491.134324] do_compact_page+0x28d/0xb60
[102491.134326] ? debug_object_deactivate+0x55/0x140
[102491.134329] ? lock_release+0x1ef/0x410
[102491.134331] ? lock_release+0x1ef/0x410
[102491.134333] process_one_work+0x2b0/0x5e0
[102491.134337] worker_thread+0x55/0x3c0
[102491.134339] ? process_one_work+0x5e0/0x5e0
[102491.134340] kthread+0x13a/0x150
[102491.134342] ? __kthread_bind_mask+0x60/0x60
[102491.134345] ret_from_fork+0x22/0x30
[102491.134349] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8 isofs
uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib
[102491.134375] WARNING: CPU: 18 PID: 182 at lib/debugobjects.c:505
debug_print_object+0x6e/0x90
[102491.134380] nft_reject_inet nf_reject_ipv4
[102491.134383] Modules linked in:
[102491.134385] nf_reject_ipv6 nft_reject
[102491.134388] snd_seq_dummy
[102491.134390] nft_ct
[102491.134393] snd_hrtimer
[102491.134395] nft_chain_nat nf_nat
[102491.134398] nls_utf8
[102491.134400] nf_conntrack nf_defrag_ipv6
[102491.134403] isofs
[102491.134405] nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd
sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi snd_intel_dspcfg mc
joydev snd_intel_sdw_acpi mac80211 kvm snd_hda_codec snd_hda_core iwlwifi
btusb snd_hwdep btrtl snd_seq btbcm btintel eeepc_wmi asus_wmi
snd_seq_device irqbypass xpad sparse_keymap bluetooth libarc4 rapl snd_pcm
video ff_memless wmi_bmof ecdh_generic ecc cfg80211 pcspkr snd_timer
k10temp snd sp5100_tco i2c_piix4 soundcore rfkill acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper
crct10dif_pclmul crc32_pclmul crc32c_intel cec igb drm nvme
ghash_clmulni_intel dca ccp
[102491.134476] nvme_core i2c_algo_bit wmi pinctrl_amd fuse
[102491.134484] ---[ end trace 562b0b01453e6613 ]---
[102491.134505] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134509] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff ff e8
91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67 fd fd ff
<0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50 fd fd
[102491.134511] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134514] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134992] uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat
[102491.135036] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.135039] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.135040] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.135041] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.135042] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.135047] nf_conntrack nf_defrag_ipv6
[102491.135051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135054] nf_defrag_ipv4
[102491.135056] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135059] ip_set
[102491.135061] note: kworker/u64:0[863413] exited with preempt_count 2
[102491.135064] nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi snd_intel_dspcfg mc
joydev snd_intel_sdw_acpi mac80211 kvm snd_hda_codec snd_hda_core iwlwifi
btusb snd_hwdep btrtl snd_seq btbcm btintel eeepc_wmi asus_wmi
snd_seq_device irqbypass xpad sparse_keymap bluetooth libarc4 rapl snd_pcm
video ff_memless wmi_bmof ecdh_generic ecc cfg80211 pcspkr snd_timer
k10temp snd sp5100_tco i2c_piix4 soundcore rfkill acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper
crct10dif_pclmul crc32_pclmul crc32c_intel cec igb drm nvme
ghash_clmulni_intel dca ccp nvme_core i2c_algo_bit wmi
[102491.135357] pinctrl_amd fuse
[102491.135366] CPU: 18 PID: 182 Comm: kcompactd0 Tainted: G D W
--------- --- 5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.135369] Hardware name: System manufacturer System Product Name/ROG
STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.135372] RIP: 0010:debug_print_object+0x6e/0x90
[102491.135403] Code: 49 89 c1 8b 43 10 83 c2 01 8b 4b 14 48 c7 c7 e8 ab 64
8b 89 15 a7 0b 37 03 4c 8b 45 00 48 8b 14 c5 a0 80 2a 8b e8 50 4d 60 00
<0f> 0b 83 05 25 0e 99 01 01 48 83 c4 08 5b 5d c3 83 05 17 0e 99 01
[102491.135406] RSP: 0018:ffffac448080bb78 EFLAGS: 00010296
[102491.135409] RAX: 000000000000005e RBX: ffff8add6de5c7a8 RCX:
0000000000000027
[102491.135412] RDX: ffff8ae348fdaae8 RSI: 0000000000000001 RDI:
ffff8ae348fdaae0
[102491.135415] RBP: ffffffff8b221320 R08: 0000000000000000 R09:
0000000000000000
[102491.135417] R10: 0000000000000000 R11: 0000000000000000 R12:
dead000000000122
[102491.135419] R13: dead000000000100 R14: ffffffff8b221320 R15:
0000000000000005
[102491.135421] FS: 0000000000000000(0000) GS:ffff8ae348e00000(0000)
knlGS:0000000000000000
[102491.135441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135443] CR2: 00007f41b9ee7000 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135446] Call Trace:
[102491.135451] debug_check_no_obj_freed+0x1db/0x220
[102491.135455] free_pcp_prepare+0x132/0x270
[102491.135459] free_unref_page+0x18/0xd0
[102491.135463] migrate_pages+0x8b9/0x1200
[102491.135467] ? isolate_freepages_block+0x4a0/0x4a0
[102491.135471] ? split_map_pages+0x160/0x160
[102491.135490] compact_zone+0x680/0xfd0
[102491.135493] ? __free_object+0x2b9/0x300
[102491.135496] ? lock_release+0x1ef/0x410
[102491.135500] proactive_compact_node+0x78/0xb0
[102491.135505] kcompactd+0x38a/0x440
[102491.135509] ? do_wait_intr_irq+0xd0/0xd0
[102491.135512] ? kcompactd_do_work+0x3a0/0x3a0
[102491.135515] kthread+0x13a/0x150
[102491.135520] ? __kthread_bind_mask+0x60/0x60
[102491.135533] ret_from_fork+0x22/0x30
[102491.135539] irq event stamp: 220
[102491.135541] hardirqs last enabled at (219): [<ffffffff8ad62217>]
_raw_spin_unlock_irqrestore+0x37/0x40
[102491.135545] hardirqs last disabled at (220): [<ffffffff8ad5b9a9>]
__schedule+0x6e9/0xb20
[102491.135548] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102491.135552] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102491.135555] ---[ end trace 562b0b01453e6614 ]---
[102494.954915] iwlwifi 0000:04:00.0: Error sending SCAN_REQ_UMAC: time out
after 2000ms.
[102494.954950] iwlwifi 0000:04:00.0: Current CMD queue read_ptr 93
write_ptr 94
[102494.956242] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956245] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 6
[102494.956248] iwlwifi 0000:04:00.0: Loaded firmware version:
59.601f3a66.0 cc-a0-59.ucode
[102494.956251] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
[102494.956255] iwlwifi 0000:04:00.0: 0x00A022F0 | trm_hw_status0
[102494.956257] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
[102494.956260] iwlwifi 0000:04:00.0: 0x004FAA36 | branchlink2
[102494.956262] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink1
[102494.956265] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink2
[102494.956268] iwlwifi 0000:04:00.0: 0x004F51B0 | data1
[102494.956270] iwlwifi 0000:04:00.0: 0x01000000 | data2
[102494.956272] iwlwifi 0000:04:00.0: 0x00000000 | data3
[102494.956275] iwlwifi 0000:04:00.0: 0x00000000 | beacon time
[102494.956277] iwlwifi 0000:04:00.0: 0xD78321C6 | tsf low
[102494.956279] iwlwifi 0000:04:00.0: 0x00000017 | tsf hi
[102494.956282] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
[102494.956284] iwlwifi 0000:04:00.0: 0xD783784B | time gp2
[102494.956286] iwlwifi 0000:04:00.0: 0x00000001 | uCode revision type
[102494.956289] iwlwifi 0000:04:00.0: 0x0000003B | uCode version major
[102494.956291] iwlwifi 0000:04:00.0: 0x601F3A66 | uCode version minor
[102494.956294] iwlwifi 0000:04:00.0: 0x00000340 | hw version
[102494.956296] iwlwifi 0000:04:00.0: 0x00C89000 | board version
[102494.956299] iwlwifi 0000:04:00.0: 0x807DFD04 | hcmd
[102494.956302] iwlwifi 0000:04:00.0: 0x00020000 | isr0
[102494.956304] iwlwifi 0000:04:00.0: 0x01000000 | isr1
[102494.956306] iwlwifi 0000:04:00.0: 0x08F04002 | isr2
[102494.956309] iwlwifi 0000:04:00.0: 0x04C3000C | isr3
[102494.956312] iwlwifi 0000:04:00.0: 0x00000000 | isr4
[102494.956315] iwlwifi 0000:04:00.0: 0x005C019C | last cmd Id
[102494.956318] iwlwifi 0000:04:00.0: 0x004F51B0 | wait_event
[102494.956320] iwlwifi 0000:04:00.0: 0x00004B99 | l2p_control
[102494.956322] iwlwifi 0000:04:00.0: 0x00000000 | l2p_duration
[102494.956325] iwlwifi 0000:04:00.0: 0x00000003 | l2p_mhvalid
[102494.956327] iwlwifi 0000:04:00.0: 0x00000000 | l2p_addr_match
[102494.956329] iwlwifi 0000:04:00.0: 0x0000000B | lmpm_pmg_sel
[102494.956332] iwlwifi 0000:04:00.0: 0x00000000 | timestamp
[102494.956334] iwlwifi 0000:04:00.0: 0x000080EC | flow_handler
[102494.956380] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956382] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 7
[102494.956385] iwlwifi 0000:04:00.0: 0x20000066 | NMI_INTERRUPT_HOST
[102494.956387] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
[102494.956390] iwlwifi 0000:04:00.0: 0x804568FC | umac branchlink2
[102494.956392] iwlwifi 0000:04:00.0: 0xC0084F3C | umac interruptlink1
[102494.956395] iwlwifi 0000:04:00.0: 0x80477750 | umac interruptlink2
[102494.956397] iwlwifi 0000:04:00.0: 0x01000000 | umac data1
[102494.956399] iwlwifi 0000:04:00.0: 0x80477750 | umac data2
[102494.956401] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
[102494.956404] iwlwifi 0000:04:00.0: 0x0000003B | umac major
[102494.956406] iwlwifi 0000:04:00.0: 0x601F3A66 | umac minor
[102494.956408] iwlwifi 0000:04:00.0: 0xD7837848 | frame pointer
[102494.956411] iwlwifi 0000:04:00.0: 0xC0885F30 | stack pointer
[102494.956413] iwlwifi 0000:04:00.0: 0x005D010D | last host cmd
[102494.956415] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
[102494.956430] iwlwifi 0000:04:00.0: IML/ROM dump:
[102494.956432] iwlwifi 0000:04:00.0: 0x00000003 | IML/ROM error/state
[102494.956446] iwlwifi 0000:04:00.0: 0x00005590 | IML/ROM data1
[102494.956460] iwlwifi 0000:04:00.0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
[102494.956470] iwlwifi 0000:04:00.0: Fseq Registers:
[102494.956475] iwlwifi 0000:04:00.0: 0x60000000 | FSEQ_ERROR_CODE
[102494.956480] iwlwifi 0000:04:00.0: 0x80290021 | FSEQ_TOP_INIT_VERSION
[102494.956486] iwlwifi 0000:04:00.0: 0x00050008 | FSEQ_CNVIO_INIT_VERSION
[102494.956491] iwlwifi 0000:04:00.0: 0x0000A503 | FSEQ_OTP_VERSION
[102494.956496] iwlwifi 0000:04:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
[102494.956502] iwlwifi 0000:04:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
[102494.956507] iwlwifi 0000:04:00.0: 0x00100530 | FSEQ_CNVI_ID
[102494.956512] iwlwifi 0000:04:00.0: 0x00000532 | FSEQ_CNVR_ID
[102494.956518] iwlwifi 0000:04:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
[102494.956525] iwlwifi 0000:04:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
[102494.956532] iwlwifi 0000:04:00.0: 0x05B0905B |
CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[102494.956540] iwlwifi 0000:04:00.0: 0x0000025B |
CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[102494.956762] iwlwifi 0000:04:00.0: WRT: Collecting data: ini trigger 4
fired.
[102494.956789] ieee80211 phy0: Hardware restart was requested
[102494.956816] iwlwifi 0000:04:00.0: Scan failed! ret -110
[102494.956925] ------------[ cut here ]------------
[102494.956928] WARNING: CPU: 30 PID: 930660 at net/mac80211/scan.c:411
__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
[102494.956962] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8 isofs
uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi snd_intel_dspcfg mc
joydev snd_intel_sdw_acpi mac80211 kvm snd_hda_codec snd_hda_core iwlwifi
btusb snd_hwdep btrtl snd_seq btbcm btintel eeepc_wmi asus_wmi
snd_seq_device irqbypass xpad sparse_keymap bluetooth libarc4
[102494.957007] rapl snd_pcm video ff_memless wmi_bmof ecdh_generic ecc
cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4 soundcore rfkill
acpi_cpufreq binfmt_misc ip_tables amdgpu drm_ttm_helper ttm iommu_v2
gpu_sched drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel cec igb
drm nvme ghash_clmulni_intel dca ccp nvme_core i2c_algo_bit wmi pinctrl_amd
fuse
[102494.957036] CPU: 30 PID: 930660 Comm: kworker/u64:2 Tainted: G D
W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102494.957039] Hardware name: System manufacturer System Product Name/ROG
STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102494.957042] Workqueue: phy0 ieee80211_scan_work [mac80211]
[102494.957073] RIP: 0010:__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
[102494.957826] Code: ca 0f 82 7d 01 00 00 48 89 ef e8 80 2f 00 00 e9 72 fe
ff ff 0f 0b 48 83 bd d8 1c 00 00 00 41 be 01 00 00 00 0f 85 9e fd ff ff
<0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 8b 85 a0 1c 00 00 e9 69
[102494.957830] RSP: 0018:ffffac4495033db0 EFLAGS: 00010246
[102494.957834] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000000
[102494.957836] RDX: 0000000000000000 RSI: 0000000000000001 RDI:
ffff8adc770f8e00
[102494.957839] RBP: ffff8adc770f8e00 R08: 0000000000000001 R09:
ffffffffc1395e40
[102494.957842] R10: ffffac4495033de8 R11: 0000000000000000 R12:
0000000000000001
[102494.957844] R13: 0000000000000000 R14: 0000000000000001 R15:
ffff8adc770f8e00
[102494.957847] FS: 0000000000000000(0000) GS:ffff8ae34a600000(0000)
knlGS:0000000000000000
[102494.957850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102494.957853] CR2: 00003de20ab47000 CR3: 00000001346e0000 CR4:
0000000000350ee0
[102494.957856] Call Trace:
[102494.957862] ieee80211_scan_work+0x15c/0x860 [mac80211]
[102494.957893] ? debug_object_deactivate+0x55/0x140
[102494.957899] ? lock_release+0x1ef/0x410
[102494.957913] ? lock_release+0x1ef/0x410
[102494.957917] process_one_work+0x2b0/0x5e0
[102494.957923] worker_thread+0x55/0x3c0
[102494.957926] ? process_one_work+0x5e0/0x5e0
[102494.957930] kthread+0x13a/0x150
[102494.957934] ? __kthread_bind_mask+0x60/0x60
[102494.957939] ret_from_fork+0x22/0x30
[102494.957945] irq event stamp: 0
[102494.957948] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[102494.957951] hardirqs last disabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957956] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957959] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102494.957962] ---[ end trace 562b0b01453e6615 ]---
Full kernel log is here: https://pastebin.com/A7dwr8ZV
--
Best Regards,
Mike Gavrilov.
[-- Attachment #2: Type: text/html, Size: 34632 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-09 2:31 ` Hillf Danton
2021-03-15 19:18 ` Mikhail Gavrilov
@ 2021-03-15 19:21 ` Mikhail Gavrilov
2021-03-16 6:13 ` Hillf Danton
1 sibling, 1 reply; 10+ messages in thread
From: Mikhail Gavrilov @ 2021-03-15 19:21 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Tue, 9 Mar 2021 at 07:31, Hillf Danton <hdanton@sina.com> wrote:
> At the first glance, the zero pointer goes out of the box of race because
>
> 1/ the Call Trace shows it is the free path (of the supposed race victim),
>
> 2/ on the race winner side however either list_del or list_del_init
> would not leave a null pointer behind - the list_add captured in this
> report is under pool->lock.
No more ideas how to fix it?
Kernel panics continue happens again and again with you patches and
recent commits.
[102491.134247] ------------[ cut here ]------------
[102491.134248] list_add corruption. next->prev should be prev
(ffffcc447ea60c78), but was ffffffff8a64ec20. (next=ffff8adc731d3f40).
[102491.134266] ODEBUG: free active (active state 0) object type:
work_struct hint: compact_page_work+0x0/0x10
[102491.134294] ------------[ cut here ]------------
[102491.134295] kernel BUG at lib/list_debug.c:23!
[102491.134299] invalid opcode: 0000 [#1] SMP NOPTI
[102491.134301] CPU: 22 PID: 863413 Comm: kworker/u64:0 Tainted: G
W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.134303] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.134305] Workqueue: zswap3 compact_page_work
[102491.134309] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134312] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
fd fd
[102491.134313] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134315] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134316] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.134316] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.134317] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.134318] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.134319] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.134320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.134320] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.134321] Call Trace:
[102491.134324] do_compact_page+0x28d/0xb60
[102491.134326] ? debug_object_deactivate+0x55/0x140
[102491.134329] ? lock_release+0x1ef/0x410
[102491.134331] ? lock_release+0x1ef/0x410
[102491.134333] process_one_work+0x2b0/0x5e0
[102491.134337] worker_thread+0x55/0x3c0
[102491.134339] ? process_one_work+0x5e0/0x5e0
[102491.134340] kthread+0x13a/0x150
[102491.134342] ? __kthread_bind_mask+0x60/0x60
[102491.134345] ret_from_fork+0x22/0x30
[102491.134349] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib
[102491.134375] WARNING: CPU: 18 PID: 182 at lib/debugobjects.c:505
debug_print_object+0x6e/0x90
[102491.134380] nft_reject_inet nf_reject_ipv4
[102491.134383] Modules linked in:
[102491.134385] nf_reject_ipv6 nft_reject
[102491.134388] snd_seq_dummy
[102491.134390] nft_ct
[102491.134393] snd_hrtimer
[102491.134395] nft_chain_nat nf_nat
[102491.134398] nls_utf8
[102491.134400] nf_conntrack nf_defrag_ipv6
[102491.134403] isofs
[102491.134405] nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
[102491.134476] nvme_core i2c_algo_bit wmi pinctrl_amd fuse
[102491.134484] ---[ end trace 562b0b01453e6613 ]---
[102491.134505] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134509] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
fd fd
[102491.134511] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134514] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134992] uas usb_storage tun uinput rfcomm netconsole
nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat
[102491.135036] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.135039] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.135040] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.135041] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.135042] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.135047] nf_conntrack nf_defrag_ipv6
[102491.135051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135054] nf_defrag_ipv4
[102491.135056] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135059] ip_set
[102491.135061] note: kworker/u64:0[863413] exited with preempt_count 2
[102491.135064] nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
nvme_core i2c_algo_bit wmi
[102491.135357] pinctrl_amd fuse
[102491.135366] CPU: 18 PID: 182 Comm: kcompactd0 Tainted: G D W
--------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.135369] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.135372] RIP: 0010:debug_print_object+0x6e/0x90
[102491.135403] Code: 49 89 c1 8b 43 10 83 c2 01 8b 4b 14 48 c7 c7 e8
ab 64 8b 89 15 a7 0b 37 03 4c 8b 45 00 48 8b 14 c5 a0 80 2a 8b e8 50
4d 60 00 <0f> 0b 83 05 25 0e 99 01 01 48 83 c4 08 5b 5d c3 83 05 17 0e
99 01
[102491.135406] RSP: 0018:ffffac448080bb78 EFLAGS: 00010296
[102491.135409] RAX: 000000000000005e RBX: ffff8add6de5c7a8 RCX:
0000000000000027
[102491.135412] RDX: ffff8ae348fdaae8 RSI: 0000000000000001 RDI:
ffff8ae348fdaae0
[102491.135415] RBP: ffffffff8b221320 R08: 0000000000000000 R09:
0000000000000000
[102491.135417] R10: 0000000000000000 R11: 0000000000000000 R12:
dead000000000122
[102491.135419] R13: dead000000000100 R14: ffffffff8b221320 R15:
0000000000000005
[102491.135421] FS: 0000000000000000(0000) GS:ffff8ae348e00000(0000)
knlGS:0000000000000000
[102491.135441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135443] CR2: 00007f41b9ee7000 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135446] Call Trace:
[102491.135451] debug_check_no_obj_freed+0x1db/0x220
[102491.135455] free_pcp_prepare+0x132/0x270
[102491.135459] free_unref_page+0x18/0xd0
[102491.135463] migrate_pages+0x8b9/0x1200
[102491.135467] ? isolate_freepages_block+0x4a0/0x4a0
[102491.135471] ? split_map_pages+0x160/0x160
[102491.135490] compact_zone+0x680/0xfd0
[102491.135493] ? __free_object+0x2b9/0x300
[102491.135496] ? lock_release+0x1ef/0x410
[102491.135500] proactive_compact_node+0x78/0xb0
[102491.135505] kcompactd+0x38a/0x440
[102491.135509] ? do_wait_intr_irq+0xd0/0xd0
[102491.135512] ? kcompactd_do_work+0x3a0/0x3a0
[102491.135515] kthread+0x13a/0x150
[102491.135520] ? __kthread_bind_mask+0x60/0x60
[102491.135533] ret_from_fork+0x22/0x30
[102491.135539] irq event stamp: 220
[102491.135541] hardirqs last enabled at (219): [<ffffffff8ad62217>]
_raw_spin_unlock_irqrestore+0x37/0x40
[102491.135545] hardirqs last disabled at (220): [<ffffffff8ad5b9a9>]
__schedule+0x6e9/0xb20
[102491.135548] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102491.135552] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102491.135555] ---[ end trace 562b0b01453e6614 ]---
[102494.954915] iwlwifi 0000:04:00.0: Error sending SCAN_REQ_UMAC:
time out after 2000ms.
[102494.954950] iwlwifi 0000:04:00.0: Current CMD queue read_ptr 93 write_ptr 94
[102494.956242] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956245] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 6
[102494.956248] iwlwifi 0000:04:00.0: Loaded firmware version:
59.601f3a66.0 cc-a0-59.ucode
[102494.956251] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
[102494.956255] iwlwifi 0000:04:00.0: 0x00A022F0 | trm_hw_status0
[102494.956257] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
[102494.956260] iwlwifi 0000:04:00.0: 0x004FAA36 | branchlink2
[102494.956262] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink1
[102494.956265] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink2
[102494.956268] iwlwifi 0000:04:00.0: 0x004F51B0 | data1
[102494.956270] iwlwifi 0000:04:00.0: 0x01000000 | data2
[102494.956272] iwlwifi 0000:04:00.0: 0x00000000 | data3
[102494.956275] iwlwifi 0000:04:00.0: 0x00000000 | beacon time
[102494.956277] iwlwifi 0000:04:00.0: 0xD78321C6 | tsf low
[102494.956279] iwlwifi 0000:04:00.0: 0x00000017 | tsf hi
[102494.956282] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
[102494.956284] iwlwifi 0000:04:00.0: 0xD783784B | time gp2
[102494.956286] iwlwifi 0000:04:00.0: 0x00000001 | uCode revision type
[102494.956289] iwlwifi 0000:04:00.0: 0x0000003B | uCode version major
[102494.956291] iwlwifi 0000:04:00.0: 0x601F3A66 | uCode version minor
[102494.956294] iwlwifi 0000:04:00.0: 0x00000340 | hw version
[102494.956296] iwlwifi 0000:04:00.0: 0x00C89000 | board version
[102494.956299] iwlwifi 0000:04:00.0: 0x807DFD04 | hcmd
[102494.956302] iwlwifi 0000:04:00.0: 0x00020000 | isr0
[102494.956304] iwlwifi 0000:04:00.0: 0x01000000 | isr1
[102494.956306] iwlwifi 0000:04:00.0: 0x08F04002 | isr2
[102494.956309] iwlwifi 0000:04:00.0: 0x04C3000C | isr3
[102494.956312] iwlwifi 0000:04:00.0: 0x00000000 | isr4
[102494.956315] iwlwifi 0000:04:00.0: 0x005C019C | last cmd Id
[102494.956318] iwlwifi 0000:04:00.0: 0x004F51B0 | wait_event
[102494.956320] iwlwifi 0000:04:00.0: 0x00004B99 | l2p_control
[102494.956322] iwlwifi 0000:04:00.0: 0x00000000 | l2p_duration
[102494.956325] iwlwifi 0000:04:00.0: 0x00000003 | l2p_mhvalid
[102494.956327] iwlwifi 0000:04:00.0: 0x00000000 | l2p_addr_match
[102494.956329] iwlwifi 0000:04:00.0: 0x0000000B | lmpm_pmg_sel
[102494.956332] iwlwifi 0000:04:00.0: 0x00000000 | timestamp
[102494.956334] iwlwifi 0000:04:00.0: 0x000080EC | flow_handler
[102494.956380] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956382] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 7
[102494.956385] iwlwifi 0000:04:00.0: 0x20000066 | NMI_INTERRUPT_HOST
[102494.956387] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
[102494.956390] iwlwifi 0000:04:00.0: 0x804568FC | umac branchlink2
[102494.956392] iwlwifi 0000:04:00.0: 0xC0084F3C | umac interruptlink1
[102494.956395] iwlwifi 0000:04:00.0: 0x80477750 | umac interruptlink2
[102494.956397] iwlwifi 0000:04:00.0: 0x01000000 | umac data1
[102494.956399] iwlwifi 0000:04:00.0: 0x80477750 | umac data2
[102494.956401] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
[102494.956404] iwlwifi 0000:04:00.0: 0x0000003B | umac major
[102494.956406] iwlwifi 0000:04:00.0: 0x601F3A66 | umac minor
[102494.956408] iwlwifi 0000:04:00.0: 0xD7837848 | frame pointer
[102494.956411] iwlwifi 0000:04:00.0: 0xC0885F30 | stack pointer
[102494.956413] iwlwifi 0000:04:00.0: 0x005D010D | last host cmd
[102494.956415] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
[102494.956430] iwlwifi 0000:04:00.0: IML/ROM dump:
[102494.956432] iwlwifi 0000:04:00.0: 0x00000003 | IML/ROM error/state
[102494.956446] iwlwifi 0000:04:00.0: 0x00005590 | IML/ROM data1
[102494.956460] iwlwifi 0000:04:00.0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
[102494.956470] iwlwifi 0000:04:00.0: Fseq Registers:
[102494.956475] iwlwifi 0000:04:00.0: 0x60000000 | FSEQ_ERROR_CODE
[102494.956480] iwlwifi 0000:04:00.0: 0x80290021 | FSEQ_TOP_INIT_VERSION
[102494.956486] iwlwifi 0000:04:00.0: 0x00050008 | FSEQ_CNVIO_INIT_VERSION
[102494.956491] iwlwifi 0000:04:00.0: 0x0000A503 | FSEQ_OTP_VERSION
[102494.956496] iwlwifi 0000:04:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
[102494.956502] iwlwifi 0000:04:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
[102494.956507] iwlwifi 0000:04:00.0: 0x00100530 | FSEQ_CNVI_ID
[102494.956512] iwlwifi 0000:04:00.0: 0x00000532 | FSEQ_CNVR_ID
[102494.956518] iwlwifi 0000:04:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
[102494.956525] iwlwifi 0000:04:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
[102494.956532] iwlwifi 0000:04:00.0: 0x05B0905B |
CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[102494.956540] iwlwifi 0000:04:00.0: 0x0000025B |
CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[102494.956762] iwlwifi 0000:04:00.0: WRT: Collecting data: ini trigger 4 fired.
[102494.956789] ieee80211 phy0: Hardware restart was requested
[102494.956816] iwlwifi 0000:04:00.0: Scan failed! ret -110
[102494.956925] ------------[ cut here ]------------
[102494.956928] WARNING: CPU: 30 PID: 930660 at
net/mac80211/scan.c:411 __ieee80211_scan_completed+0x2bb/0x520
[mac80211]
[102494.956962] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4
[102494.957007] rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
nvme_core i2c_algo_bit wmi pinctrl_amd fuse
[102494.957036] CPU: 30 PID: 930660 Comm: kworker/u64:2 Tainted: G
D W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102494.957039] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102494.957042] Workqueue: phy0 ieee80211_scan_work [mac80211]
[102494.957073] RIP: 0010:__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
[102494.957826] Code: ca 0f 82 7d 01 00 00 48 89 ef e8 80 2f 00 00 e9
72 fe ff ff 0f 0b 48 83 bd d8 1c 00 00 00 41 be 01 00 00 00 0f 85 9e
fd ff ff <0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 8b 85 a0 1c 00 00
e9 69
[102494.957830] RSP: 0018:ffffac4495033db0 EFLAGS: 00010246
[102494.957834] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000000
[102494.957836] RDX: 0000000000000000 RSI: 0000000000000001 RDI:
ffff8adc770f8e00
[102494.957839] RBP: ffff8adc770f8e00 R08: 0000000000000001 R09:
ffffffffc1395e40
[102494.957842] R10: ffffac4495033de8 R11: 0000000000000000 R12:
0000000000000001
[102494.957844] R13: 0000000000000000 R14: 0000000000000001 R15:
ffff8adc770f8e00
[102494.957847] FS: 0000000000000000(0000) GS:ffff8ae34a600000(0000)
knlGS:0000000000000000
[102494.957850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102494.957853] CR2: 00003de20ab47000 CR3: 00000001346e0000 CR4:
0000000000350ee0
[102494.957856] Call Trace:
[102494.957862] ieee80211_scan_work+0x15c/0x860 [mac80211]
[102494.957893] ? debug_object_deactivate+0x55/0x140
[102494.957899] ? lock_release+0x1ef/0x410
[102494.957913] ? lock_release+0x1ef/0x410
[102494.957917] process_one_work+0x2b0/0x5e0
[102494.957923] worker_thread+0x55/0x3c0
[102494.957926] ? process_one_work+0x5e0/0x5e0
[102494.957930] kthread+0x13a/0x150
[102494.957934] ? __kthread_bind_mask+0x60/0x60
[102494.957939] ret_from_fork+0x22/0x30
[102494.957945] irq event stamp: 0
[102494.957948] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[102494.957951] hardirqs last disabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957956] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957959] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102494.957962] ---[ end trace 562b0b01453e6615 ]---
Full kernel log is here: https://pastebin.com/A7dwr8ZV
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-15 19:21 ` Mikhail Gavrilov
@ 2021-03-16 6:13 ` Hillf Danton
0 siblings, 0 replies; 10+ messages in thread
From: Hillf Danton @ 2021-03-16 6:13 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Tue, 16 Mar 2021 00:21:05 +0500 Mikhail Gavrilov wrote:
> On Tue, 9 Mar 2021 at 07:31, Hillf Danton <hdanton@sina.com> wrote:
> > At the first glance, the zero pointer goes out of the box of race because
> >
> > 1/ the Call Trace shows it is the free path (of the supposed race victim),
> >
> > 2/ on the race winner side however either list_del or list_del_init
> > would not leave a null pointer behind - the list_add captured in this
> > report is under pool->lock.
>
>
> No more ideas how to fix it?
> Kernel panics continue happens again and again with you patches and
> recent commits.
Thanks for your report.
>
> [102491.134247] ------------[ cut here ]------------
> [102491.134248] list_add corruption. next->prev should be prev
> (ffffcc447ea60c78), but was ffffffff8a64ec20. (next=ffff8adc731d3f40).
The same race as we saw over a couple of weeks.
> [102491.134266] ODEBUG: free active (active state 0) object type:
> work_struct hint: compact_page_work+0x0/0x10
> [102491.134294] ------------[ cut here ]------------
This is a new one.
> [102491.134295] kernel BUG at lib/list_debug.c:23!
> [102491.134299] invalid opcode: 0000 [#1] SMP NOPTI
> [102491.134301] CPU: 22 PID: 863413 Comm: kworker/u64:0 Tainted: G
> W --------- ---
The victim was running on CPU22.
> 5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
> [102491.134303] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [102491.134305] Workqueue: zswap3 compact_page_work
> [102491.134309] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [102491.134312] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
> ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
> fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
> fd fd
> [102491.134313] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
> [102491.134315] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
> 0000000000000000
> [102491.134316] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
> ffff8ae3497daae0
> [102491.134316] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
> 0000000000000000
> [102491.134317] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff8adc4e317a08
> [102491.134318] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
> ffff8adceb216000
> [102491.134319] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
> knlGS:0000000000000000
> [102491.134320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [102491.134320] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
> 0000000000350ee0
> [102491.134321] Call Trace:
> [102491.134324] do_compact_page+0x28d/0xb60
> [102491.134326] ? debug_object_deactivate+0x55/0x140
> [102491.134329] ? lock_release+0x1ef/0x410
> [102491.134331] ? lock_release+0x1ef/0x410
> [102491.134333] process_one_work+0x2b0/0x5e0
> [102491.134337] worker_thread+0x55/0x3c0
> [102491.134339] ? process_one_work+0x5e0/0x5e0
> [102491.134340] kthread+0x13a/0x150
> [102491.134342] ? __kthread_bind_mask+0x60/0x60
> [102491.134345] ret_from_fork+0x22/0x30
> [102491.134349] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
> isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
> nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib
> [102491.134375] WARNING: CPU: 18 PID: 182 at lib/debugobjects.c:505
> debug_print_object+0x6e/0x90
> [102491.134380] nft_reject_inet nf_reject_ipv4
> [102491.134383] Modules linked in:
> [102491.134385] nf_reject_ipv6 nft_reject
> [102491.134388] snd_seq_dummy
> [102491.134390] nft_ct
> [102491.134393] snd_hrtimer
> [102491.134395] nft_chain_nat nf_nat
> [102491.134398] nls_utf8
> [102491.134400] nf_conntrack nf_defrag_ipv6
> [102491.134403] isofs
> [102491.134405] nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
> zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
> intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
> videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
> mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
> edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
> mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
> snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
> snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
> btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
> bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
> ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
> soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
> drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
> crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
> [102491.134476] nvme_core i2c_algo_bit wmi pinctrl_amd fuse
> [102491.134484] ---[ end trace 562b0b01453e6613 ]---
> [102491.134505] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [102491.134509] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
> ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
> fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
> fd fd
> [102491.134511] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
> [102491.134514] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
> 0000000000000000
> [102491.134992] uas usb_storage tun uinput rfcomm netconsole
> nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat
> [102491.135036] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
> ffff8ae3497daae0
> [102491.135039] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
> 0000000000000000
> [102491.135040] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff8adc4e317a08
> [102491.135041] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
> ffff8adceb216000
> [102491.135042] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
> knlGS:0000000000000000
> [102491.135047] nf_conntrack nf_defrag_ipv6
> [102491.135051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [102491.135054] nf_defrag_ipv4
> [102491.135056] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
> 0000000000350ee0
> [102491.135059] ip_set
> [102491.135061] note: kworker/u64:0[863413] exited with preempt_count 2
> [102491.135064] nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
> hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
> uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
> snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
> snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
> snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
> snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
> snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
> snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
> btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
> bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
> ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
> soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
> drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
> crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
> nvme_core i2c_algo_bit wmi
> [102491.135357] pinctrl_amd fuse
> [102491.135366] CPU: 18 PID: 182 Comm: kcompactd0 Tainted: G D W
> --------- ---
Was the culprit running on CPU 18?
> 5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
> [102491.135369] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [102491.135372] RIP: 0010:debug_print_object+0x6e/0x90
> [102491.135403] Code: 49 89 c1 8b 43 10 83 c2 01 8b 4b 14 48 c7 c7 e8
> ab 64 8b 89 15 a7 0b 37 03 4c 8b 45 00 48 8b 14 c5 a0 80 2a 8b e8 50
> 4d 60 00 <0f> 0b 83 05 25 0e 99 01 01 48 83 c4 08 5b 5d c3 83 05 17 0e
> 99 01
> [102491.135406] RSP: 0018:ffffac448080bb78 EFLAGS: 00010296
> [102491.135409] RAX: 000000000000005e RBX: ffff8add6de5c7a8 RCX:
> 0000000000000027
> [102491.135412] RDX: ffff8ae348fdaae8 RSI: 0000000000000001 RDI:
> ffff8ae348fdaae0
> [102491.135415] RBP: ffffffff8b221320 R08: 0000000000000000 R09:
> 0000000000000000
> [102491.135417] R10: 0000000000000000 R11: 0000000000000000 R12:
> dead000000000122
> [102491.135419] R13: dead000000000100 R14: ffffffff8b221320 R15:
> 0000000000000005
> [102491.135421] FS: 0000000000000000(0000) GS:ffff8ae348e00000(0000)
> knlGS:0000000000000000
> [102491.135441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [102491.135443] CR2: 00007f41b9ee7000 CR3: 00000003b1c28000 CR4:
> 0000000000350ee0
> [102491.135446] Call Trace:
> [102491.135451] debug_check_no_obj_freed+0x1db/0x220
> [102491.135455] free_pcp_prepare+0x132/0x270
> [102491.135459] free_unref_page+0x18/0xd0
> [102491.135463] migrate_pages+0x8b9/0x1200
> [102491.135467] ? isolate_freepages_block+0x4a0/0x4a0
> [102491.135471] ? split_map_pages+0x160/0x160
> [102491.135490] compact_zone+0x680/0xfd0
> [102491.135493] ? __free_object+0x2b9/0x300
> [102491.135496] ? lock_release+0x1ef/0x410
> [102491.135500] proactive_compact_node+0x78/0xb0
> [102491.135505] kcompactd+0x38a/0x440
> [102491.135509] ? do_wait_intr_irq+0xd0/0xd0
> [102491.135512] ? kcompactd_do_work+0x3a0/0x3a0
> [102491.135515] kthread+0x13a/0x150
> [102491.135520] ? __kthread_bind_mask+0x60/0x60
> [102491.135533] ret_from_fork+0x22/0x30
> [102491.135539] irq event stamp: 220
> [102491.135541] hardirqs last enabled at (219): [<ffffffff8ad62217>]
> _raw_spin_unlock_irqrestore+0x37/0x40
> [102491.135545] hardirqs last disabled at (220): [<ffffffff8ad5b9a9>]
> __schedule+0x6e9/0xb20
> [102491.135548] softirqs last enabled at (0): [<ffffffff8a0dd990>]
> copy_process+0x910/0x1e00
> [102491.135552] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [102491.135555] ---[ end trace 562b0b01453e6614 ]---
> [102494.954915] iwlwifi 0000:04:00.0: Error sending SCAN_REQ_UMAC:
> time out after 2000ms.
> [102494.954950] iwlwifi 0000:04:00.0: Current CMD queue read_ptr 93 write_ptr 94
> [102494.956242] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
> [102494.956245] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 6
> [102494.956248] iwlwifi 0000:04:00.0: Loaded firmware version:
> 59.601f3a66.0 cc-a0-59.ucode
> [102494.956251] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
> [102494.956255] iwlwifi 0000:04:00.0: 0x00A022F0 | trm_hw_status0
> [102494.956257] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
> [102494.956260] iwlwifi 0000:04:00.0: 0x004FAA36 | branchlink2
> [102494.956262] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink1
> [102494.956265] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink2
> [102494.956268] iwlwifi 0000:04:00.0: 0x004F51B0 | data1
> [102494.956270] iwlwifi 0000:04:00.0: 0x01000000 | data2
> [102494.956272] iwlwifi 0000:04:00.0: 0x00000000 | data3
> [102494.956275] iwlwifi 0000:04:00.0: 0x00000000 | beacon time
> [102494.956277] iwlwifi 0000:04:00.0: 0xD78321C6 | tsf low
> [102494.956279] iwlwifi 0000:04:00.0: 0x00000017 | tsf hi
> [102494.956282] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
> [102494.956284] iwlwifi 0000:04:00.0: 0xD783784B | time gp2
> [102494.956286] iwlwifi 0000:04:00.0: 0x00000001 | uCode revision type
> [102494.956289] iwlwifi 0000:04:00.0: 0x0000003B | uCode version major
> [102494.956291] iwlwifi 0000:04:00.0: 0x601F3A66 | uCode version minor
> [102494.956294] iwlwifi 0000:04:00.0: 0x00000340 | hw version
> [102494.956296] iwlwifi 0000:04:00.0: 0x00C89000 | board version
> [102494.956299] iwlwifi 0000:04:00.0: 0x807DFD04 | hcmd
> [102494.956302] iwlwifi 0000:04:00.0: 0x00020000 | isr0
> [102494.956304] iwlwifi 0000:04:00.0: 0x01000000 | isr1
> [102494.956306] iwlwifi 0000:04:00.0: 0x08F04002 | isr2
> [102494.956309] iwlwifi 0000:04:00.0: 0x04C3000C | isr3
> [102494.956312] iwlwifi 0000:04:00.0: 0x00000000 | isr4
> [102494.956315] iwlwifi 0000:04:00.0: 0x005C019C | last cmd Id
> [102494.956318] iwlwifi 0000:04:00.0: 0x004F51B0 | wait_event
> [102494.956320] iwlwifi 0000:04:00.0: 0x00004B99 | l2p_control
> [102494.956322] iwlwifi 0000:04:00.0: 0x00000000 | l2p_duration
> [102494.956325] iwlwifi 0000:04:00.0: 0x00000003 | l2p_mhvalid
> [102494.956327] iwlwifi 0000:04:00.0: 0x00000000 | l2p_addr_match
> [102494.956329] iwlwifi 0000:04:00.0: 0x0000000B | lmpm_pmg_sel
> [102494.956332] iwlwifi 0000:04:00.0: 0x00000000 | timestamp
> [102494.956334] iwlwifi 0000:04:00.0: 0x000080EC | flow_handler
> [102494.956380] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
> [102494.956382] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 7
> [102494.956385] iwlwifi 0000:04:00.0: 0x20000066 | NMI_INTERRUPT_HOST
> [102494.956387] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
> [102494.956390] iwlwifi 0000:04:00.0: 0x804568FC | umac branchlink2
> [102494.956392] iwlwifi 0000:04:00.0: 0xC0084F3C | umac interruptlink1
> [102494.956395] iwlwifi 0000:04:00.0: 0x80477750 | umac interruptlink2
> [102494.956397] iwlwifi 0000:04:00.0: 0x01000000 | umac data1
> [102494.956399] iwlwifi 0000:04:00.0: 0x80477750 | umac data2
> [102494.956401] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
> [102494.956404] iwlwifi 0000:04:00.0: 0x0000003B | umac major
> [102494.956406] iwlwifi 0000:04:00.0: 0x601F3A66 | umac minor
> [102494.956408] iwlwifi 0000:04:00.0: 0xD7837848 | frame pointer
> [102494.956411] iwlwifi 0000:04:00.0: 0xC0885F30 | stack pointer
> [102494.956413] iwlwifi 0000:04:00.0: 0x005D010D | last host cmd
> [102494.956415] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
> [102494.956430] iwlwifi 0000:04:00.0: IML/ROM dump:
> [102494.956432] iwlwifi 0000:04:00.0: 0x00000003 | IML/ROM error/state
> [102494.956446] iwlwifi 0000:04:00.0: 0x00005590 | IML/ROM data1
> [102494.956460] iwlwifi 0000:04:00.0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
> [102494.956470] iwlwifi 0000:04:00.0: Fseq Registers:
> [102494.956475] iwlwifi 0000:04:00.0: 0x60000000 | FSEQ_ERROR_CODE
> [102494.956480] iwlwifi 0000:04:00.0: 0x80290021 | FSEQ_TOP_INIT_VERSION
> [102494.956486] iwlwifi 0000:04:00.0: 0x00050008 | FSEQ_CNVIO_INIT_VERSION
> [102494.956491] iwlwifi 0000:04:00.0: 0x0000A503 | FSEQ_OTP_VERSION
> [102494.956496] iwlwifi 0000:04:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
> [102494.956502] iwlwifi 0000:04:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
> [102494.956507] iwlwifi 0000:04:00.0: 0x00100530 | FSEQ_CNVI_ID
> [102494.956512] iwlwifi 0000:04:00.0: 0x00000532 | FSEQ_CNVR_ID
> [102494.956518] iwlwifi 0000:04:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
> [102494.956525] iwlwifi 0000:04:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
> [102494.956532] iwlwifi 0000:04:00.0: 0x05B0905B |
> CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
> [102494.956540] iwlwifi 0000:04:00.0: 0x0000025B |
> CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
> [102494.956762] iwlwifi 0000:04:00.0: WRT: Collecting data: ini trigger 4 fired.
> [102494.956789] ieee80211 phy0: Hardware restart was requested
> [102494.956816] iwlwifi 0000:04:00.0: Scan failed! ret -110
> [102494.956925] ------------[ cut here ]------------
> [102494.956928] WARNING: CPU: 30 PID: 930660 at
> net/mac80211/scan.c:411 __ieee80211_scan_completed+0x2bb/0x520
> [mac80211]
> [102494.956962] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
> isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
> nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
> nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
> zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
> intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
> videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
> mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
> edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
> mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
> snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
> snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
> btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
> bluetooth libarc4
> [102494.957007] rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
> ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
> soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
> drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
> crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
> nvme_core i2c_algo_bit wmi pinctrl_amd fuse
> [102494.957036] CPU: 30 PID: 930660 Comm: kworker/u64:2 Tainted: G
> D W --------- ---
> 5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
> [102494.957039] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [102494.957042] Workqueue: phy0 ieee80211_scan_work [mac80211]
> [102494.957073] RIP: 0010:__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
> [102494.957826] Code: ca 0f 82 7d 01 00 00 48 89 ef e8 80 2f 00 00 e9
> 72 fe ff ff 0f 0b 48 83 bd d8 1c 00 00 00 41 be 01 00 00 00 0f 85 9e
> fd ff ff <0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 8b 85 a0 1c 00 00
> e9 69
> [102494.957830] RSP: 0018:ffffac4495033db0 EFLAGS: 00010246
> [102494.957834] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000000
> [102494.957836] RDX: 0000000000000000 RSI: 0000000000000001 RDI:
> ffff8adc770f8e00
> [102494.957839] RBP: ffff8adc770f8e00 R08: 0000000000000001 R09:
> ffffffffc1395e40
> [102494.957842] R10: ffffac4495033de8 R11: 0000000000000000 R12:
> 0000000000000001
> [102494.957844] R13: 0000000000000000 R14: 0000000000000001 R15:
> ffff8adc770f8e00
> [102494.957847] FS: 0000000000000000(0000) GS:ffff8ae34a600000(0000)
> knlGS:0000000000000000
> [102494.957850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [102494.957853] CR2: 00003de20ab47000 CR3: 00000001346e0000 CR4:
> 0000000000350ee0
> [102494.957856] Call Trace:
> [102494.957862] ieee80211_scan_work+0x15c/0x860 [mac80211]
> [102494.957893] ? debug_object_deactivate+0x55/0x140
> [102494.957899] ? lock_release+0x1ef/0x410
> [102494.957913] ? lock_release+0x1ef/0x410
> [102494.957917] process_one_work+0x2b0/0x5e0
> [102494.957923] worker_thread+0x55/0x3c0
> [102494.957926] ? process_one_work+0x5e0/0x5e0
> [102494.957930] kthread+0x13a/0x150
> [102494.957934] ? __kthread_bind_mask+0x60/0x60
> [102494.957939] ret_from_fork+0x22/0x30
> [102494.957945] irq event stamp: 0
> [102494.957948] hardirqs last enabled at (0): [<0000000000000000>] 0x0
> [102494.957951] hardirqs last disabled at (0): [<ffffffff8a0dd990>]
> copy_process+0x910/0x1e00
> [102494.957956] softirqs last enabled at (0): [<ffffffff8a0dd990>]
> copy_process+0x910/0x1e00
> [102494.957959] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [102494.957962] ---[ end trace 562b0b01453e6615 ]---
>
> Full kernel log is here: https://pastebin.com/A7dwr8ZV
>
>
> --
> Best Regards,
> Mike Gavrilov.
Lets see the race between the work and compact/migrate pathes.
work migrate
---- -------
VM_BUG_ON_PAGE(!test_bit(PAGE_CLAIMED,
&page->private), page);
zhdr = page_address(page);
if (!z3fold_page_trylock(zhdr))
return -EAGAIN;
if (work_pending(&zhdr->work)) {
z3fold_page_unlock(zhdr);
return -EAGAIN;
}
*&*--> page->private = 0;
z3fold_page_unlock(zhdr);
page_mapcount_reset(page);
clear_bit(PAGE_CLAIMED, &page->private);
put_page(page);
return 0;
z3fold_page_lock(zhdr);
if (test_bit(PAGE_STALE, &page->private) ||
test_and_set_bit(PAGE_CLAIMED, &page->private)) {
z3fold_page_unlock(zhdr);
return;
}
...
z3fold_page_unlock(zhdr);
====
1/ no chance for race at the first glance because 1) PAGE_CLAIMED is
checked on both sides and 2) both take z3fold_page_lock(zhdr).
2/ it is bad to reset page->private because it goes odd with the clearing
of PAGE_CLAIMED before put_page().
3/ the trigger of the second warning indicates it is necessary to wait
for the work to be done before put_page().
That said, the quick fix is to cancel work if the first hunk below wont
survive your test. Note it wont make sense without the previous diffs.
--- x/mm/z3fold.c
+++ y/mm/z3fold.c
@@ -1623,7 +1623,6 @@ static int z3fold_page_migrate(struct ad
new_zhdr = page_address(newpage);
memcpy(new_zhdr, zhdr, PAGE_SIZE);
newpage->private = page->private;
- page->private = 0;
z3fold_page_unlock(zhdr);
spin_lock_init(&new_zhdr->page_lock);
INIT_WORK(&new_zhdr->work, compact_page_work);
@@ -1654,6 +1653,7 @@ static int z3fold_page_migrate(struct ad
queue_work_on(new_zhdr->cpu, pool->compact_wq, &new_zhdr->work);
+ cancel_work_sync(&zhdr->work);
page_mapcount_reset(page);
clear_bit(PAGE_CLAIMED, &page->private);
put_page(page);
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2021-03-16 6:14 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20210126082834.2020-1-hdanton@sina.com>
[not found] ` <CABXGCsNN+u2SqOvmw2JojTnESSLgxKgfJLQuB3Ne1fcNA47UZw@mail.gmail.com>
2021-02-13 3:03 ` BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4) Hillf Danton
2021-02-28 13:22 ` Mikhail Gavrilov
2021-03-01 3:11 ` Hillf Danton
2021-03-05 9:33 ` Mikhail Gavrilov
2021-03-05 14:22 ` Hillf Danton
2021-03-08 15:42 ` Mikhail Gavrilov
2021-03-09 2:31 ` Hillf Danton
2021-03-15 19:18 ` Mikhail Gavrilov
2021-03-15 19:21 ` Mikhail Gavrilov
2021-03-16 6:13 ` Hillf Danton
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).