* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
[not found] <20210126082834.2020-1-hdanton@sina.com>
@ 2021-02-12 13:28 ` Mikhail Gavrilov
2021-02-13 3:03 ` Hillf Danton
0 siblings, 1 reply; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-02-12 13:28 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, Kees Cook, Paul E . McKenney
On Tue, 26 Jan 2021 at 13:28, Hillf Danton <hdanton@sina.com> wrote:
>
>
> BTW better run the reproducer again with KASAN enabled.
>
It happened today again with kernel 5.11 rc7 (e0756cfc7d7c)
Why not try your patch?
list_del corruption, ffffdef70143e848->next is LIST_POISON1 (dead000000000100)
------------[ cut here ]------------
kernel BUG at lib/list_debug.c:45!
invalid opcode: 0000 [#1] SMP NOPTI
CPU: 13 PID: 263 Comm: kswapd0 Tainted: G W ---------
--- 5.11.0-0.rc7.20210210gite0756cfc7d7c.150.fc35.x86_64 #1
Hardware name: System manufacturer System Product Name/ROG STRIX
X570-I GAMING, BIOS 3402 01/13/2021
RIP: 0010:__list_del_entry_valid.cold+0xf/0x47
Code: fe ff 0f 0b 48 89 d1 4c 89 c6 4c 89 ca 48 c7 c7 e0 26 64 9e e8
1b 12 fe ff 0f 0b 48 89 fe 48 c7 c7 70 27 64 9e e8 0a 12 fe ff <0f> 0b
48 c7 c7 20 28 64 9e e8 fc 11 fe ff 0f 0b 48 89 f2 48 89 fe
RSP: 0018:ffff9f2180863908 EFLAGS: 00010286
RAX: 000000000000004e RBX: ffff8f74d0fa1000 RCX: 0000000000000000
RDX: ffff8f7c885e9f60 RSI: ffff8f7c885db2a0 RDI: ffff8f7c885db2a0
RBP: ffff8f74d0fa1000 R08: 0000000000000000 R09: ffff9f2180863748
R10: ffff9f2180863740 R11: 0000000000000000 R12: ffff8f758edd8e00
R13: 0000000000012800 R14: ffffdef70143e840 R15: ffff8f758edd8e08
FS: 0000000000000000(0000) GS:ffff8f7c88400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000037203acb1000 CR3: 00000001d5c28000 CR4: 0000000000350ee0
Call Trace:
z3fold_zpool_malloc+0x3e3/0x780
? _raw_spin_unlock+0x1f/0x30
zswap_frontswap_store+0x43e/0x890
__frontswap_store+0xc8/0x170
swap_writepage+0x39/0x70
pageout+0x125/0x540
shrink_page_list+0x1329/0x1bc0
shrink_inactive_list+0x12a/0x440
shrink_lruvec+0x4a9/0x6d0
? super_cache_count+0x79/0xf0
shrink_node+0x2d1/0x700
balance_pgdat+0x2f5/0x650
kswapd+0x21d/0x4d0
? do_wait_intr_irq+0xd0/0xd0
? balance_pgdat+0x650/0x650
kthread+0x13a/0x150
? __kthread_bind_mask+0x60/0x60
ret_from_fork+0x22/0x30
Modules linked in: tun snd_seq_dummy snd_hrtimer uinput rfcomm
nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ip6table_nat
ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat
nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw
iptable_security ip_set nf_tables nfnetlink ip6table_filter ip6_tables
iptable_filter cmac bnep zstd sunrpc vfat fat hid_logitech_hidpp
hid_logitech_dj snd_hda_codec_realtek snd_hda_codec_generic
ledtrig_audio snd_hda_codec_hdmi snd_hda_intel mt76x2u
snd_intel_dspcfg soundwire_intel mt76x2_common mt76x02_usb
soundwire_generic_allocation mt76_usb intel_rapl_msr iwlmvm
snd_soc_core snd_usb_audio intel_rapl_common mt76x02_lib mt76
snd_compress snd_pcm_dmaengine snd_usbmidi_lib soundwire_cadence
snd_rawmidi mac80211 snd_hda_codec joydev snd_hda_core uvcvideo
ac97_bus snd_hwdep btusb snd_seq
videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 snd_seq_device
btrtl edac_mce_amd btbcm iwlwifi snd_pcm videobuf2_common btintel
kvm_amd eeepc_wmi snd_timer bluetooth kvm videodev asus_wmi snd
ecdh_generic sparse_keymap irqbypass xpad mc libarc4 sp5100_tco rapl
ff_memless cfg80211 wmi_bmof ecc video pcspkr soundcore k10temp
i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec drm ghash_clmulni_intel igb ccp nvme dca
nvme_core i2c_algo_bit wmi pinctrl_amd fuse
---[ end trace a0c35e2a81af0791 ]---
RIP: 0010:__list_del_entry_valid.cold+0xf/0x47
Code: fe ff 0f 0b 48 89 d1 4c 89 c6 4c 89 ca 48 c7 c7 e0 26 64 9e e8
1b 12 fe ff 0f 0b 48 89 fe 48 c7 c7 70 27 64 9e e8 0a 12 fe ff <0f> 0b
48 c7 c7 20 28 64 9e e8 fc 11 fe ff 0f 0b 48 89 f2 48 89 fe
RSP: 0018:ffff9f2180863908 EFLAGS: 00010286
RAX: 000000000000004e RBX: ffff8f74d0fa1000 RCX: 0000000000000000
RDX: ffff8f7c885e9f60 RSI: ffff8f7c885db2a0 RDI: ffff8f7c885db2a0
RBP: ffff8f74d0fa1000 R08: 0000000000000000 R09: ffff9f2180863748
R10: ffff9f2180863740 R11: 0000000000000000 R12: ffff8f758edd8e00
R13: 0000000000012800 R14: ffffdef70143e840 R15: ffff8f758edd8e08
FS: 0000000000000000(0000) GS:ffff8f7c88400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000037203acb1000 CR3: 00000001d5c28000 CR4: 0000000000350ee0
note: kswapd0[263] exited with preempt_count 2
full kernel log: https://pastebin.com/FL1fZLJ0
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-02-12 13:28 ` BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4) Mikhail Gavrilov
@ 2021-02-13 3:03 ` Hillf Danton
2021-02-28 13:22 ` Mikhail Gavrilov
0 siblings, 1 reply; 18+ messages in thread
From: Hillf Danton @ 2021-02-13 3:03 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Fri, 12 Feb 2021 18:28:12 +0500 Mikhail Gavrilov wrote:
> On Tue, 26 Jan 2021 at 13:28, Hillf Danton <hdanton@sina.com> wrote:
> >
> > BTW better run the reproducer again with KASAN enabled.
> >
>
> It happened today again with kernel 5.11 rc7 (e0756cfc7d7c)
Thanks again.
> Why not try your patch?
Simply because it was half baked - I was not convinced it was a fix
instead of papering over anything.
>
> list_del corruption, ffffdef70143e848->next is LIST_POISON1 (dead000000000100)
> ------------[ cut here ]------------
> kernel BUG at lib/list_debug.c:45!
> invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 13 PID: 263 Comm: kswapd0 Tainted: G W ---------
> --- 5.11.0-0.rc7.20210210gite0756cfc7d7c.150.fc35.x86_64 #1
> Hardware name: System manufacturer System Product Name/ROG STRIX
> X570-I GAMING, BIOS 3402 01/13/2021
> RIP: 0010:__list_del_entry_valid.cold+0xf/0x47
> Code: fe ff 0f 0b 48 89 d1 4c 89 c6 4c 89 ca 48 c7 c7 e0 26 64 9e e8
> 1b 12 fe ff 0f 0b 48 89 fe 48 c7 c7 70 27 64 9e e8 0a 12 fe ff <0f> 0b
> 48 c7 c7 20 28 64 9e e8 fc 11 fe ff 0f 0b 48 89 f2 48 89 fe
> RSP: 0018:ffff9f2180863908 EFLAGS: 00010286
> RAX: 000000000000004e RBX: ffff8f74d0fa1000 RCX: 0000000000000000
> RDX: ffff8f7c885e9f60 RSI: ffff8f7c885db2a0 RDI: ffff8f7c885db2a0
> RBP: ffff8f74d0fa1000 R08: 0000000000000000 R09: ffff9f2180863748
> R10: ffff9f2180863740 R11: 0000000000000000 R12: ffff8f758edd8e00
> R13: 0000000000012800 R14: ffffdef70143e840 R15: ffff8f758edd8e08
> FS: 0000000000000000(0000) GS:ffff8f7c88400000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000037203acb1000 CR3: 00000001d5c28000 CR4: 0000000000350ee0
> Call Trace:
> z3fold_zpool_malloc+0x3e3/0x780
There is a race producing the list corruption above.
> ? _raw_spin_unlock+0x1f/0x30
> zswap_frontswap_store+0x43e/0x890
> __frontswap_store+0xc8/0x170
> swap_writepage+0x39/0x70
> pageout+0x125/0x540
> shrink_page_list+0x1329/0x1bc0
> shrink_inactive_list+0x12a/0x440
> shrink_lruvec+0x4a9/0x6d0
> ? super_cache_count+0x79/0xf0
> shrink_node+0x2d1/0x700
> balance_pgdat+0x2f5/0x650
> kswapd+0x21d/0x4d0
> ? do_wait_intr_irq+0xd0/0xd0
> ? balance_pgdat+0x650/0x650
> kthread+0x13a/0x150
> ? __kthread_bind_mask+0x60/0x60
> ret_from_fork+0x22/0x30
> Modules linked in: tun snd_seq_dummy snd_hrtimer uinput rfcomm
> nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ip6table_nat
> ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat
> nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw
> iptable_security ip_set nf_tables nfnetlink ip6table_filter ip6_tables
> iptable_filter cmac bnep zstd sunrpc vfat fat hid_logitech_hidpp
> hid_logitech_dj snd_hda_codec_realtek snd_hda_codec_generic
> ledtrig_audio snd_hda_codec_hdmi snd_hda_intel mt76x2u
> snd_intel_dspcfg soundwire_intel mt76x2_common mt76x02_usb
> soundwire_generic_allocation mt76_usb intel_rapl_msr iwlmvm
> snd_soc_core snd_usb_audio intel_rapl_common mt76x02_lib mt76
> snd_compress snd_pcm_dmaengine snd_usbmidi_lib soundwire_cadence
> snd_rawmidi mac80211 snd_hda_codec joydev snd_hda_core uvcvideo
> ac97_bus snd_hwdep btusb snd_seq
> videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 snd_seq_device
> btrtl edac_mce_amd btbcm iwlwifi snd_pcm videobuf2_common btintel
> kvm_amd eeepc_wmi snd_timer bluetooth kvm videodev asus_wmi snd
> ecdh_generic sparse_keymap irqbypass xpad mc libarc4 sp5100_tco rapl
> ff_memless cfg80211 wmi_bmof ecc video pcspkr soundcore k10temp
> i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
> drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
> crc32_pclmul crc32c_intel cec drm ghash_clmulni_intel igb ccp nvme dca
> nvme_core i2c_algo_bit wmi pinctrl_amd fuse
> ---[ end trace a0c35e2a81af0791 ]---
> RIP: 0010:__list_del_entry_valid.cold+0xf/0x47
> Code: fe ff 0f 0b 48 89 d1 4c 89 c6 4c 89 ca 48 c7 c7 e0 26 64 9e e8
> 1b 12 fe ff 0f 0b 48 89 fe 48 c7 c7 70 27 64 9e e8 0a 12 fe ff <0f> 0b
> 48 c7 c7 20 28 64 9e e8 fc 11 fe ff 0f 0b 48 89 f2 48 89 fe
> RSP: 0018:ffff9f2180863908 EFLAGS: 00010286
> RAX: 000000000000004e RBX: ffff8f74d0fa1000 RCX: 0000000000000000
> RDX: ffff8f7c885e9f60 RSI: ffff8f7c885db2a0 RDI: ffff8f7c885db2a0
> RBP: ffff8f74d0fa1000 R08: 0000000000000000 R09: ffff9f2180863748
> R10: ffff9f2180863740 R11: 0000000000000000 R12: ffff8f758edd8e00
> R13: 0000000000012800 R14: ffffdef70143e840 R15: ffff8f758edd8e08
> FS: 0000000000000000(0000) GS:ffff8f7c88400000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000037203acb1000 CR3: 00000001d5c28000 CR4: 0000000000350ee0
> note: kswapd0[263] exited with preempt_count 2
>
>
> full kernel log: https://pastebin.com/FL1fZLJ0
>
> --
> Best Regards,
> Mike Gavrilov.
The comment below shows a race instance, though I failed to put things
together to see how within two hours. Cut it and see what will come up.
--- a/mm/z3fold.c
+++ b/mm/z3fold.c
@@ -1129,19 +1129,22 @@ retry:
page = NULL;
if (can_sleep) {
spin_lock(&pool->stale_lock);
+ spin_lock(&pool->lock);
zhdr = list_first_entry_or_null(&pool->stale,
struct z3fold_header, buddy);
/*
- * Before allocating a page, let's see if we can take one from
+ * Before allocating a page, lets see if we can take one from
* the stale pages list. cancel_work_sync() can sleep so we
* limit this case to the contexts where we can sleep
*/
if (zhdr) {
list_del(&zhdr->buddy);
+ spin_unlock(&pool->lock);
spin_unlock(&pool->stale_lock);
cancel_work_sync(&zhdr->work);
page = virt_to_page(zhdr);
} else {
+ spin_unlock(&pool->lock);
spin_unlock(&pool->stale_lock);
}
}
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-02-13 3:03 ` Hillf Danton
@ 2021-02-28 13:22 ` Mikhail Gavrilov
0 siblings, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-02-28 13:22 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Sat, 13 Feb 2021 at 08:03, Hillf Danton <hdanton@sina.com> wrote:
>
> The comment below shows a race instance, though I failed to put things
> together to see how within two hours. Cut it and see what will come up.
>
> --- a/mm/z3fold.c
> +++ b/mm/z3fold.c
> @@ -1129,19 +1129,22 @@ retry:
> page = NULL;
> if (can_sleep) {
> spin_lock(&pool->stale_lock);
> + spin_lock(&pool->lock);
> zhdr = list_first_entry_or_null(&pool->stale,
> struct z3fold_header, buddy);
> /*
> - * Before allocating a page, let's see if we can take one from
> + * Before allocating a page, lets see if we can take one from
> * the stale pages list. cancel_work_sync() can sleep so we
> * limit this case to the contexts where we can sleep
> */
> if (zhdr) {
> list_del(&zhdr->buddy);
> + spin_unlock(&pool->lock);
> spin_unlock(&pool->stale_lock);
> cancel_work_sync(&zhdr->work);
> page = virt_to_page(zhdr);
> } else {
> + spin_unlock(&pool->lock);
> spin_unlock(&pool->stale_lock);
> }
> }
Hi,
It happened again with the patch above.
Is anything cleared up now?
[32451.229358] list_add corruption. next->prev should be prev
(ffffd08fbc661cd0), but was ffffffffa7643650. (next=ffff9e4d2848f1c0).
[32451.229395] ------------[ cut here ]------------
[32451.229398] kernel BUG at lib/list_debug.c:23!
[32451.229408] invalid opcode: 0000 [#1] SMP NOPTI
[32451.229414] CPU: 4 PID: 80665 Comm: kworker/u64:0 Tainted: G
W --------- --- 5.11.0-155.fc35.x86_64+debug #1
[32451.229420] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32451.229424] Workqueue: zswap3 compact_page_work
[32451.229433] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[32451.229439] Code: 48 c7 c6 24 26 64 a8 48 89 ef 49 c7 c7 ea ff ff
ff e8 e8 71 01 00 e9 fa 10 9e ff 4c 89 c1 48 c7 c7 f0 26 64 a8 e8 50
12 fe ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 a0 27 64 a8 e8 39
12 fe
[32451.229444] RSP: 0018:ffffb08fd553fde0 EFLAGS: 00010286
[32451.229449] RAX: 0000000000000075 RBX: ffffe16d871550c0 RCX: 0000000000000000
[32451.229453] RDX: ffff9e53c73e9f60 RSI: ffff9e53c73db2a0 RDI: ffff9e53c73db2a0
[32451.229457] RBP: ffffd08fbc661cd0 R08: 0000000000000000 R09: ffffb08fd553fc20
[32451.229460] R10: ffffb08fd553fc18 R11: 0000000000000000 R12: ffff9e4ce4e29008
[32451.229464] R13: ffff9e4d85543010 R14: ffff9e4d2848f1c0 R15: ffff9e4d85543000
[32451.229468] FS: 0000000000000000(0000) GS:ffff9e53c7200000(0000)
knlGS:0000000000000000
[32451.229472] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32451.229476] CR2: 00000fb21455dfe8 CR3: 0000000142968000 CR4: 0000000000350ee0
[32451.229480] Call Trace:
[32451.229485] do_compact_page+0x28d/0xb60
[32451.229492] ? debug_object_deactivate+0x55/0x140
[32451.229499] ? lock_release+0x1e9/0x400
[32451.229505] ? lock_release+0x1e9/0x400
[32451.229511] process_one_work+0x2b0/0x5e0
[32451.229519] worker_thread+0x55/0x3c0
[32451.229524] ? process_one_work+0x5e0/0x5e0
[32451.229531] kthread+0x13a/0x150
[32451.229540] ? __kthread_bind_mask+0x60/0x60
[32451.229548] ret_from_fork+0x22/0x30
[32451.229558] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32451.229628] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32451.229696] ---[ end trace 80d86d6942435514 ]---
[32451.229701] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[32451.229706] Code: 48 c7 c6 24 26 64 a8 48 89 ef 49 c7 c7 ea ff ff
ff e8 e8 71 01 00 e9 fa 10 9e ff 4c 89 c1 48 c7 c7 f0 26 64 a8 e8 50
12 fe ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 a0 27 64 a8 e8 39
12 fe
[32451.229710] RSP: 0018:ffffb08fd553fde0 EFLAGS: 00010286
[32451.229715] RAX: 0000000000000075 RBX: ffffe16d871550c0 RCX: 0000000000000000
[32451.229721] RDX: ffff9e53c73e9f60 RSI: ffff9e53c73db2a0 RDI: ffff9e53c73db2a0
[32451.229725] RBP: ffffd08fbc661cd0 R08: 0000000000000000 R09: ffffb08fd553fc20
[32451.229729] R10: ffffb08fd553fc18 R11: 0000000000000000 R12: ffff9e4ce4e29008
[32451.229732] R13: ffff9e4d85543010 R14: ffff9e4d2848f1c0 R15: ffff9e4d85543000
[32451.229736] FS: 0000000000000000(0000) GS:ffff9e53c7200000(0000)
knlGS:0000000000000000
[32451.229740] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32451.229744] CR2: 00000fb21455dfe8 CR3: 0000000142968000 CR4: 0000000000350ee0
[32451.229748] note: kworker/u64:0[80665] exited with preempt_count 2
[32476.846645] watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
[vivaldi-bin:6991]
[32476.846658] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.846704] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.846874] irq event stamp: 0
[32476.846877] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.846883] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.846889] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.846892] softirqs last disabled at (0): [<0000000000000000>] 0x0
[32476.846896] CPU: 0 PID: 6991 Comm: vivaldi-bin Tainted: G D W
--------- --- 5.11.0-155.fc35.x86_64+debug #1
[32476.846900] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32476.846904] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[32476.846909] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 80 f1 1e 00 48 03 04 fd 00 39 6e a8 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[32476.846913] RSP: 0000:ffffb08fd2937c10 EFLAGS: 00000246
[32476.846917] RAX: 0000000000000000 RBX: ffffe16d9e844240 RCX: ffff9e53c6bef180
[32476.846920] RDX: ffff9e4cc11a3d28 RSI: 0000000000040000 RDI: 000000000000000d
[32476.846923] RBP: ffff9e4cc11a3d28 R08: 0000000000040000 R09: 0000000000000000
[32476.846926] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9e4cc11a3d40
[32476.846929] R13: ffff9e4cc11a3d28 R14: ffff9e4cc11a3d20 R15: 0000000000ddd8d8
[32476.846932] FS: 00007f4b852a0300(0000) GS:ffff9e53c6a00000(0000)
knlGS:0000000000000000
[32476.846935] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32476.846939] CR2: 00003d0509d83000 CR3: 000000017733a000 CR4: 0000000000350ef0
[32476.846942] Call Trace:
[32476.846946] do_raw_spin_lock+0x94/0xa0
[32476.846951] _raw_spin_lock+0x63/0x80
[32476.846955] zswap_frontswap_load+0x2f/0x2f0
[32476.846960] ? psi_group_change+0x27d/0x290
[32476.846965] __frontswap_load+0xc3/0x160
[32476.846969] swap_readpage+0x1ca/0x3a0
[32476.846974] swapin_readahead+0x2ee/0x4e0
[32476.846979] do_swap_page+0x4a4/0x900
[32476.846983] ? lock_release+0x1e9/0x400
[32476.846987] ? trace_hardirqs_on+0x1b/0xe0
[32476.846992] handle_mm_fault+0xe7d/0x19d0
[32476.846997] do_user_addr_fault+0x1c7/0x4c0
[32476.847003] exc_page_fault+0x67/0x2a0
[32476.847007] ? asm_exc_page_fault+0x8/0x30
[32476.847011] asm_exc_page_fault+0x1e/0x30
[32476.847015] RIP: 0033:0x55a5d9c33379
[32476.847018] Code: 00 00 4d 89 75 00 4c 89 f0 48 25 00 00 fc ff 48
8b 40 08 41 c7 46 03 03 00 00 00 49 8b 4d 00 44 89 61 07 49 8b 5d 00
4d 8b 37 <44> 89 73 0b a9 00 00 04 00 75 1a 83 e0 18 48 85 c0 74 12 49
8b 45
[32476.847022] RSP: 002b:00007fff34882340 EFLAGS: 00010206
[32476.847025] RAX: 0000000000000012 RBX: 00003d0509d82ff5 RCX: 00003d0509d82ff5
[32476.847028] RDX: 000055a5dbb578bb RSI: 0000000000000001 RDI: 0000000000000000
[32476.847031] RBP: 00007fff34882370 R08: 0000000000000000 R09: 0000000000000000
[32476.847034] R10: 00003d0500000000 R11: ffffffff00000000 R12: 0000000000000023
[32476.847037] R13: 0000376895df40a0 R14: 00003d0509d82f7d R15: 0000376895df4080
[32476.849645] watchdog: BUG: soft lockup - CPU#1 stuck for 22s!
[Chrome_ChildIOT:5472]
[32476.849652] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.849687] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.849713] irq event stamp: 0
[32476.849715] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.849719] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.849723] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.849726] softirqs last disabled at (0): [<0000000000000000>] 0x0
[32476.849728] CPU: 1 PID: 5472 Comm: Chrome_ChildIOT Tainted: G
D W L --------- --- 5.11.0-155.fc35.x86_64+debug #1
[32476.849732] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32476.849734] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[32476.849738] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 80 f1 1e 00 48 03 04 fd 00 39 6e a8 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[32476.849741] RSP: 0000:ffffb08fc6c4bc10 EFLAGS: 00000246
[32476.849744] RAX: 0000000000000000 RBX: ffffe16d96c11140 RCX: ffff9e53c6def180
[32476.849746] RDX: ffff9e4cc11a3d28 RSI: 0000000000080000 RDI: 0000000000000016
[32476.849749] RBP: ffff9e4cc11a3d28 R08: 0000000000080000 R09: 0000000000000000
[32476.849751] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9e4cc11a3d40
[32476.849753] R13: ffff9e4cc11a3d28 R14: ffff9e4cc11a3d20 R15: 0000000000f89940
[32476.849756] FS: 00007f9a02233640(0000) GS:ffff9e53c6c00000(0000)
knlGS:0000000000000000
[32476.849758] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32476.849761] CR2: 00002312fd44ecd8 CR3: 00000001618f6000 CR4: 0000000000350ee0
[32476.849763] Call Trace:
[32476.849766] do_raw_spin_lock+0x94/0xa0
[32476.849769] _raw_spin_lock+0x63/0x80
[32476.849772] zswap_frontswap_load+0x2f/0x2f0
[32476.849775] ? psi_group_change+0x27d/0x290
[32476.849779] __frontswap_load+0xc3/0x160
[32476.849782] swap_readpage+0x1ca/0x3a0
[32476.849786] swapin_readahead+0x450/0x4e0
[32476.849789] ? lock_release+0x1e9/0x400
[32476.849793] do_swap_page+0x4a4/0x900
[32476.849796] ? lock_release+0x1e9/0x400
[32476.849799] ? trace_hardirqs_on+0x1b/0xe0
[32476.849802] handle_mm_fault+0xe7d/0x19d0
[32476.849807] do_user_addr_fault+0x1c7/0x4c0
[32476.849810] exc_page_fault+0x67/0x2a0
[32476.849813] ? asm_exc_page_fault+0x8/0x30
[32476.849816] asm_exc_page_fault+0x1e/0x30
[32476.849819] RIP: 0033:0x555d3e644fe2
[32476.849822] Code: c3 cc cc cc cc cc cc cc 55 48 89 e5 41 57 41 56
53 48 83 ec 68 49 89 fe 4c 8b 3f 48 8b 05 76 4d bd 08 49 8b 1f 48 31
c3 74 67 <48> 33 43 08 49 39 c7 74 4e c7 45 b8 04 00 00 00 c7 45 c8 04
00 00
[32476.849824] RSP: 002b:00007f9a02231ab0 EFLAGS: 00010202
[32476.849827] RAX: fffffffd55160cdb RBX: 00002312fd44ecd0 RCX: 0000000000000005
[32476.850461] RDX: 0000000000000000 RSI: 0000000000000005 RDI: 00002312fcd3f2a0
[32476.850463] RBP: 00007f9a02231b30 R08: 00002312fcfa4003 R09: 00007ffd204a88d0
[32476.850465] R10: 0000000000000000 R11: 0000000000000246 R12: 0000555d3e492590
[32476.850467] R13: 0000555d3e4cd840 R14: 00002312fcd3f2a0 R15: 00002312fd2481e0
[32476.850644] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [brave:5451]
[32476.850652] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.850687] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.850714] irq event stamp: 0
[32476.850716] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.850719] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.850723] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.850726] softirqs last disabled at (0): [<0000000000000000>] 0x0
Full kernel log is here: https://pastebin.com/4SbhNp7V
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
@ 2021-02-28 13:22 ` Mikhail Gavrilov
0 siblings, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-02-28 13:22 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Sat, 13 Feb 2021 at 08:03, Hillf Danton <hdanton@sina.com> wrote:
>
> The comment below shows a race instance, though I failed to put things
> together to see how within two hours. Cut it and see what will come up.
>
> --- a/mm/z3fold.c
> +++ b/mm/z3fold.c
> @@ -1129,19 +1129,22 @@ retry:
> page = NULL;
> if (can_sleep) {
> spin_lock(&pool->stale_lock);
> + spin_lock(&pool->lock);
> zhdr = list_first_entry_or_null(&pool->stale,
> struct z3fold_header, buddy);
> /*
> - * Before allocating a page, let's see if we can take one from
> + * Before allocating a page, lets see if we can take one from
> * the stale pages list. cancel_work_sync() can sleep so we
> * limit this case to the contexts where we can sleep
> */
> if (zhdr) {
> list_del(&zhdr->buddy);
> + spin_unlock(&pool->lock);
> spin_unlock(&pool->stale_lock);
> cancel_work_sync(&zhdr->work);
> page = virt_to_page(zhdr);
> } else {
> + spin_unlock(&pool->lock);
> spin_unlock(&pool->stale_lock);
> }
> }
Hi,
It happened again with the patch above.
Is anything cleared up now?
[32451.229358] list_add corruption. next->prev should be prev
(ffffd08fbc661cd0), but was ffffffffa7643650. (next=ffff9e4d2848f1c0).
[32451.229395] ------------[ cut here ]------------
[32451.229398] kernel BUG at lib/list_debug.c:23!
[32451.229408] invalid opcode: 0000 [#1] SMP NOPTI
[32451.229414] CPU: 4 PID: 80665 Comm: kworker/u64:0 Tainted: G
W --------- --- 5.11.0-155.fc35.x86_64+debug #1
[32451.229420] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32451.229424] Workqueue: zswap3 compact_page_work
[32451.229433] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[32451.229439] Code: 48 c7 c6 24 26 64 a8 48 89 ef 49 c7 c7 ea ff ff
ff e8 e8 71 01 00 e9 fa 10 9e ff 4c 89 c1 48 c7 c7 f0 26 64 a8 e8 50
12 fe ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 a0 27 64 a8 e8 39
12 fe
[32451.229444] RSP: 0018:ffffb08fd553fde0 EFLAGS: 00010286
[32451.229449] RAX: 0000000000000075 RBX: ffffe16d871550c0 RCX: 0000000000000000
[32451.229453] RDX: ffff9e53c73e9f60 RSI: ffff9e53c73db2a0 RDI: ffff9e53c73db2a0
[32451.229457] RBP: ffffd08fbc661cd0 R08: 0000000000000000 R09: ffffb08fd553fc20
[32451.229460] R10: ffffb08fd553fc18 R11: 0000000000000000 R12: ffff9e4ce4e29008
[32451.229464] R13: ffff9e4d85543010 R14: ffff9e4d2848f1c0 R15: ffff9e4d85543000
[32451.229468] FS: 0000000000000000(0000) GS:ffff9e53c7200000(0000)
knlGS:0000000000000000
[32451.229472] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32451.229476] CR2: 00000fb21455dfe8 CR3: 0000000142968000 CR4: 0000000000350ee0
[32451.229480] Call Trace:
[32451.229485] do_compact_page+0x28d/0xb60
[32451.229492] ? debug_object_deactivate+0x55/0x140
[32451.229499] ? lock_release+0x1e9/0x400
[32451.229505] ? lock_release+0x1e9/0x400
[32451.229511] process_one_work+0x2b0/0x5e0
[32451.229519] worker_thread+0x55/0x3c0
[32451.229524] ? process_one_work+0x5e0/0x5e0
[32451.229531] kthread+0x13a/0x150
[32451.229540] ? __kthread_bind_mask+0x60/0x60
[32451.229548] ret_from_fork+0x22/0x30
[32451.229558] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32451.229628] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32451.229696] ---[ end trace 80d86d6942435514 ]---
[32451.229701] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[32451.229706] Code: 48 c7 c6 24 26 64 a8 48 89 ef 49 c7 c7 ea ff ff
ff e8 e8 71 01 00 e9 fa 10 9e ff 4c 89 c1 48 c7 c7 f0 26 64 a8 e8 50
12 fe ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 a0 27 64 a8 e8 39
12 fe
[32451.229710] RSP: 0018:ffffb08fd553fde0 EFLAGS: 00010286
[32451.229715] RAX: 0000000000000075 RBX: ffffe16d871550c0 RCX: 0000000000000000
[32451.229721] RDX: ffff9e53c73e9f60 RSI: ffff9e53c73db2a0 RDI: ffff9e53c73db2a0
[32451.229725] RBP: ffffd08fbc661cd0 R08: 0000000000000000 R09: ffffb08fd553fc20
[32451.229729] R10: ffffb08fd553fc18 R11: 0000000000000000 R12: ffff9e4ce4e29008
[32451.229732] R13: ffff9e4d85543010 R14: ffff9e4d2848f1c0 R15: ffff9e4d85543000
[32451.229736] FS: 0000000000000000(0000) GS:ffff9e53c7200000(0000)
knlGS:0000000000000000
[32451.229740] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32451.229744] CR2: 00000fb21455dfe8 CR3: 0000000142968000 CR4: 0000000000350ee0
[32451.229748] note: kworker/u64:0[80665] exited with preempt_count 2
[32476.846645] watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
[vivaldi-bin:6991]
[32476.846658] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.846704] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.846874] irq event stamp: 0
[32476.846877] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.846883] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.846889] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.846892] softirqs last disabled at (0): [<0000000000000000>] 0x0
[32476.846896] CPU: 0 PID: 6991 Comm: vivaldi-bin Tainted: G D W
--------- --- 5.11.0-155.fc35.x86_64+debug #1
[32476.846900] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32476.846904] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[32476.846909] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 80 f1 1e 00 48 03 04 fd 00 39 6e a8 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[32476.846913] RSP: 0000:ffffb08fd2937c10 EFLAGS: 00000246
[32476.846917] RAX: 0000000000000000 RBX: ffffe16d9e844240 RCX: ffff9e53c6bef180
[32476.846920] RDX: ffff9e4cc11a3d28 RSI: 0000000000040000 RDI: 000000000000000d
[32476.846923] RBP: ffff9e4cc11a3d28 R08: 0000000000040000 R09: 0000000000000000
[32476.846926] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9e4cc11a3d40
[32476.846929] R13: ffff9e4cc11a3d28 R14: ffff9e4cc11a3d20 R15: 0000000000ddd8d8
[32476.846932] FS: 00007f4b852a0300(0000) GS:ffff9e53c6a00000(0000)
knlGS:0000000000000000
[32476.846935] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32476.846939] CR2: 00003d0509d83000 CR3: 000000017733a000 CR4: 0000000000350ef0
[32476.846942] Call Trace:
[32476.846946] do_raw_spin_lock+0x94/0xa0
[32476.846951] _raw_spin_lock+0x63/0x80
[32476.846955] zswap_frontswap_load+0x2f/0x2f0
[32476.846960] ? psi_group_change+0x27d/0x290
[32476.846965] __frontswap_load+0xc3/0x160
[32476.846969] swap_readpage+0x1ca/0x3a0
[32476.846974] swapin_readahead+0x2ee/0x4e0
[32476.846979] do_swap_page+0x4a4/0x900
[32476.846983] ? lock_release+0x1e9/0x400
[32476.846987] ? trace_hardirqs_on+0x1b/0xe0
[32476.846992] handle_mm_fault+0xe7d/0x19d0
[32476.846997] do_user_addr_fault+0x1c7/0x4c0
[32476.847003] exc_page_fault+0x67/0x2a0
[32476.847007] ? asm_exc_page_fault+0x8/0x30
[32476.847011] asm_exc_page_fault+0x1e/0x30
[32476.847015] RIP: 0033:0x55a5d9c33379
[32476.847018] Code: 00 00 4d 89 75 00 4c 89 f0 48 25 00 00 fc ff 48
8b 40 08 41 c7 46 03 03 00 00 00 49 8b 4d 00 44 89 61 07 49 8b 5d 00
4d 8b 37 <44> 89 73 0b a9 00 00 04 00 75 1a 83 e0 18 48 85 c0 74 12 49
8b 45
[32476.847022] RSP: 002b:00007fff34882340 EFLAGS: 00010206
[32476.847025] RAX: 0000000000000012 RBX: 00003d0509d82ff5 RCX: 00003d0509d82ff5
[32476.847028] RDX: 000055a5dbb578bb RSI: 0000000000000001 RDI: 0000000000000000
[32476.847031] RBP: 00007fff34882370 R08: 0000000000000000 R09: 0000000000000000
[32476.847034] R10: 00003d0500000000 R11: ffffffff00000000 R12: 0000000000000023
[32476.847037] R13: 0000376895df40a0 R14: 00003d0509d82f7d R15: 0000376895df4080
[32476.849645] watchdog: BUG: soft lockup - CPU#1 stuck for 22s!
[Chrome_ChildIOT:5472]
[32476.849652] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.849687] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.849713] irq event stamp: 0
[32476.849715] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.849719] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.849723] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.849726] softirqs last disabled at (0): [<0000000000000000>] 0x0
[32476.849728] CPU: 1 PID: 5472 Comm: Chrome_ChildIOT Tainted: G
D W L --------- --- 5.11.0-155.fc35.x86_64+debug #1
[32476.849732] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[32476.849734] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[32476.849738] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 80 f1 1e 00 48 03 04 fd 00 39 6e a8 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[32476.849741] RSP: 0000:ffffb08fc6c4bc10 EFLAGS: 00000246
[32476.849744] RAX: 0000000000000000 RBX: ffffe16d96c11140 RCX: ffff9e53c6def180
[32476.849746] RDX: ffff9e4cc11a3d28 RSI: 0000000000080000 RDI: 0000000000000016
[32476.849749] RBP: ffff9e4cc11a3d28 R08: 0000000000080000 R09: 0000000000000000
[32476.849751] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9e4cc11a3d40
[32476.849753] R13: ffff9e4cc11a3d28 R14: ffff9e4cc11a3d20 R15: 0000000000f89940
[32476.849756] FS: 00007f9a02233640(0000) GS:ffff9e53c6c00000(0000)
knlGS:0000000000000000
[32476.849758] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[32476.849761] CR2: 00002312fd44ecd8 CR3: 00000001618f6000 CR4: 0000000000350ee0
[32476.849763] Call Trace:
[32476.849766] do_raw_spin_lock+0x94/0xa0
[32476.849769] _raw_spin_lock+0x63/0x80
[32476.849772] zswap_frontswap_load+0x2f/0x2f0
[32476.849775] ? psi_group_change+0x27d/0x290
[32476.849779] __frontswap_load+0xc3/0x160
[32476.849782] swap_readpage+0x1ca/0x3a0
[32476.849786] swapin_readahead+0x450/0x4e0
[32476.849789] ? lock_release+0x1e9/0x400
[32476.849793] do_swap_page+0x4a4/0x900
[32476.849796] ? lock_release+0x1e9/0x400
[32476.849799] ? trace_hardirqs_on+0x1b/0xe0
[32476.849802] handle_mm_fault+0xe7d/0x19d0
[32476.849807] do_user_addr_fault+0x1c7/0x4c0
[32476.849810] exc_page_fault+0x67/0x2a0
[32476.849813] ? asm_exc_page_fault+0x8/0x30
[32476.849816] asm_exc_page_fault+0x1e/0x30
[32476.849819] RIP: 0033:0x555d3e644fe2
[32476.849822] Code: c3 cc cc cc cc cc cc cc 55 48 89 e5 41 57 41 56
53 48 83 ec 68 49 89 fe 4c 8b 3f 48 8b 05 76 4d bd 08 49 8b 1f 48 31
c3 74 67 <48> 33 43 08 49 39 c7 74 4e c7 45 b8 04 00 00 00 c7 45 c8 04
00 00
[32476.849824] RSP: 002b:00007f9a02231ab0 EFLAGS: 00010202
[32476.849827] RAX: fffffffd55160cdb RBX: 00002312fd44ecd0 RCX: 0000000000000005
[32476.850461] RDX: 0000000000000000 RSI: 0000000000000005 RDI: 00002312fcd3f2a0
[32476.850463] RBP: 00007f9a02231b30 R08: 00002312fcfa4003 R09: 00007ffd204a88d0
[32476.850465] R10: 0000000000000000 R11: 0000000000000246 R12: 0000555d3e492590
[32476.850467] R13: 0000555d3e4cd840 R14: 00002312fcd3f2a0 R15: 00002312fd2481e0
[32476.850644] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [brave:5451]
[32476.850652] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
uas bluetooth ac97_bus
[32476.850687] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[32476.850714] irq event stamp: 0
[32476.850716] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[32476.850719] hardirqs last disabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.850723] softirqs last enabled at (0): [<ffffffffa70ddccb>]
copy_process+0x8fb/0x1de0
[32476.850726] softirqs last disabled at (0): [<0000000000000000>] 0x0
Full kernel log is here: https://pastebin.com/4SbhNp7V
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-02-28 13:22 ` Mikhail Gavrilov
(?)
@ 2021-03-01 3:11 ` Hillf Danton
2021-03-05 9:33 ` Mikhail Gavrilov
-1 siblings, 1 reply; 18+ messages in thread
From: Hillf Danton @ 2021-03-01 3:11 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Sun, 28 Feb 2021 18:22:21 +0500 Mikhail Gavrilov wrote:
> On Sat, 13 Feb 2021 at 08:03, Hillf Danton <hdanton@sina.com> wrote:
> >
> > The comment below shows a race instance, though I failed to put things
> > together to see how within two hours. Cut it and see what will come up.
> >
> > --- a/mm/z3fold.c
> > +++ b/mm/z3fold.c
> > @@ -1129,19 +1129,22 @@ retry:
> > page = NULL;
> > if (can_sleep) {
> > spin_lock(&pool->stale_lock);
> > + spin_lock(&pool->lock);
> > zhdr = list_first_entry_or_null(&pool->stale,
> > struct z3fold_header, buddy);
> > /*
> > - * Before allocating a page, let's see if we can take one from
> > + * Before allocating a page, lets see if we can take one from
> > * the stale pages list. cancel_work_sync() can sleep so we
> > * limit this case to the contexts where we can sleep
> > */
> > if (zhdr) {
> > list_del(&zhdr->buddy);
> > + spin_unlock(&pool->lock);
> > spin_unlock(&pool->stale_lock);
> > cancel_work_sync(&zhdr->work);
> > page = virt_to_page(zhdr);
> > } else {
> > + spin_unlock(&pool->lock);
> > spin_unlock(&pool->stale_lock);
> > }
> > }
>
>
> Hi,
> It happened again with the patch above.
Thanks again.
> Is anything cleared up now?
See below.
>
> [32451.229358] list_add corruption. next->prev should be prev
> (ffffd08fbc661cd0), but was ffffffffa7643650. (next=ffff9e4d2848f1c0).
> [32451.229395] ------------[ cut here ]------------
> [32451.229398] kernel BUG at lib/list_debug.c:23!
> [32451.229408] invalid opcode: 0000 [#1] SMP NOPTI
> [32451.229414] CPU: 4 PID: 80665 Comm: kworker/u64:0 Tainted: G
> W --------- --- 5.11.0-155.fc35.x86_64+debug #1
> [32451.229420] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [32451.229424] Workqueue: zswap3 compact_page_work
> [32451.229433] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [32451.229439] Code: 48 c7 c6 24 26 64 a8 48 89 ef 49 c7 c7 ea ff ff
> ff e8 e8 71 01 00 e9 fa 10 9e ff 4c 89 c1 48 c7 c7 f0 26 64 a8 e8 50
> 12 fe ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 a0 27 64 a8 e8 39
> 12 fe
> [32451.229444] RSP: 0018:ffffb08fd553fde0 EFLAGS: 00010286
> [32451.229449] RAX: 0000000000000075 RBX: ffffe16d871550c0 RCX: 0000000000000000
> [32451.229453] RDX: ffff9e53c73e9f60 RSI: ffff9e53c73db2a0 RDI: ffff9e53c73db2a0
> [32451.229457] RBP: ffffd08fbc661cd0 R08: 0000000000000000 R09: ffffb08fd553fc20
> [32451.229460] R10: ffffb08fd553fc18 R11: 0000000000000000 R12: ffff9e4ce4e29008
> [32451.229464] R13: ffff9e4d85543010 R14: ffff9e4d2848f1c0 R15: ffff9e4d85543000
> [32451.229468] FS: 0000000000000000(0000) GS:ffff9e53c7200000(0000)
> knlGS:0000000000000000
> [32451.229472] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [32451.229476] CR2: 00000fb21455dfe8 CR3: 0000000142968000 CR4: 0000000000350ee0
> [32451.229480] Call Trace:
> [32451.229485] do_compact_page+0x28d/0xb60
> [32451.229492] ? debug_object_deactivate+0x55/0x140
> [32451.229499] ? lock_release+0x1e9/0x400
> [32451.229505] ? lock_release+0x1e9/0x400
> [32451.229511] process_one_work+0x2b0/0x5e0
> [32451.229519] worker_thread+0x55/0x3c0
> [32451.229524] ? process_one_work+0x5e0/0x5e0
> [32451.229531] kthread+0x13a/0x150
> [32451.229540] ? __kthread_bind_mask+0x60/0x60
> [32451.229548] ret_from_fork+0x22/0x30
> [32451.229558] Modules linked in: snd_seq_dummy snd_hrtimer tun uinput
> nls_utf8 isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
> nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
> nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct
> nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set
> nf_tables nfnetlink cmac bnep zstd sunrpc xpad ff_memless vfat fat
> hid_logitech_hidpp hid_logitech_dj joydev snd_hda_codec_realtek
> snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi uvcvideo
> videobuf2_vmalloc snd_hda_intel videobuf2_memops videobuf2_v4l2
> snd_intel_dspcfg mt76x2u soundwire_intel mt76x2_common
> videobuf2_common mt76x02_usb soundwire_generic_allocation mt76_usb
> intel_rapl_msr intel_rapl_common iwlmvm snd_soc_core snd_usb_audio
> videodev snd_compress mt76x02_lib btusb snd_usbmidi_lib
> snd_pcm_dmaengine mt76 soundwire_cadence btrtl snd_rawmidi btbcm
> snd_hda_codec mc btintel edac_mce_amd mac80211 kvm_amd snd_hda_core
> uas bluetooth ac97_bus
> [32451.229628] iwlwifi snd_hwdep usb_storage kvm snd_seq libarc4
> snd_seq_device ecdh_generic cfg80211 ecc irqbypass snd_pcm rapl
> eeepc_wmi asus_wmi sparse_keymap snd_timer rfkill snd video sp5100_tco
> wmi_bmof i2c_piix4 k10temp soundcore acpi_cpufreq binfmt_misc
> ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
> crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel cec drm igb
> ghash_clmulni_intel nvme ccp dca nvme_core i2c_algo_bit wmi
> pinctrl_amd fuse
> [32451.229696] ---[ end trace 80d86d6942435514 ]---
[...]
> Full kernel log is here: https://pastebin.com/4SbhNp7V
>
> --
> Best Regards,
> Mike Gavrilov.
What we learn from your reports is
1/ in z3fold_free(), kref_put() creates the ground zero for the race
cases reported,
2/ the stale_lock in combination with lock makes things more
complicated than thought.
Instead of dropping something in the zero spot, the fix below goes the
road mentioned before in this mail thread - add another list_head in
parallel to the buddy and s/buddy/stale_node/ under every case of
stale_lock.
--- x/mm/z3fold.c
+++ y/mm/z3fold.c
@@ -127,6 +127,7 @@ struct z3fold_header {
unsigned short first_num:2;
unsigned short mapped_count:2;
unsigned short foreign_handles:2;
+ struct list_head stale_node;
};
/**
@@ -429,6 +430,7 @@ static struct z3fold_header *init_z3fold
zhdr->slots = slots;
zhdr->pool = pool;
INIT_LIST_HEAD(&zhdr->buddy);
+ INIT_LIST_HEAD(&zhdr->stale_node);
INIT_WORK(&zhdr->work, compact_page_work);
return zhdr;
}
@@ -556,7 +558,7 @@ static void __release_z3fold_page(struct
z3fold_page_unlock(zhdr);
spin_lock(&pool->stale_lock);
- list_add(&zhdr->buddy, &pool->stale);
+ list_add(&zhdr->stale_node, &pool->stale);
queue_work(pool->release_wq, &pool->work);
spin_unlock(&pool->stale_lock);
}
@@ -598,10 +600,10 @@ static void free_pages_work(struct work_
spin_lock(&pool->stale_lock);
while (!list_empty(&pool->stale)) {
struct z3fold_header *zhdr = list_first_entry(&pool->stale,
- struct z3fold_header, buddy);
+ struct z3fold_header, stale_node);
struct page *page = virt_to_page(zhdr);
- list_del(&zhdr->buddy);
+ list_del(&zhdr->stale_node);
if (WARN_ON(!test_bit(PAGE_STALE, &page->private)))
continue;
spin_unlock(&pool->stale_lock);
@@ -1140,14 +1142,14 @@ retry:
if (can_sleep) {
spin_lock(&pool->stale_lock);
zhdr = list_first_entry_or_null(&pool->stale,
- struct z3fold_header, buddy);
+ struct z3fold_header, stale_node);
/*
* Before allocating a page, let's see if we can take one from
* the stale pages list. cancel_work_sync() can sleep so we
* limit this case to the contexts where we can sleep
*/
if (zhdr) {
- list_del(&zhdr->buddy);
+ list_del(&zhdr->stale_node);
spin_unlock(&pool->stale_lock);
cancel_work_sync(&zhdr->work);
page = virt_to_page(zhdr);
--
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-01 3:11 ` Hillf Danton
@ 2021-03-05 9:33 ` Mikhail Gavrilov
0 siblings, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-03-05 9:33 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Mon, 1 Mar 2021 at 08:11, Hillf Danton <hdanton@sina.com> wrote:
>
> What we learn from your reports is
>
> 1/ in z3fold_free(), kref_put() creates the ground zero for the race
> cases reported,
>
> 2/ the stale_lock in combination with lock makes things more
> complicated than thought.
>
> Instead of dropping something in the zero spot, the fix below goes the
> road mentioned before in this mail thread - add another list_head in
> parallel to the buddy and s/buddy/stale_node/ under every case of
> stale_lock.
>
> --- x/mm/z3fold.c
> +++ y/mm/z3fold.c
> @@ -127,6 +127,7 @@ struct z3fold_header {
> unsigned short first_num:2;
> unsigned short mapped_count:2;
> unsigned short foreign_handles:2;
> + struct list_head stale_node;
> };
>
> /**
> @@ -429,6 +430,7 @@ static struct z3fold_header *init_z3fold
> zhdr->slots = slots;
> zhdr->pool = pool;
> INIT_LIST_HEAD(&zhdr->buddy);
> + INIT_LIST_HEAD(&zhdr->stale_node);
> INIT_WORK(&zhdr->work, compact_page_work);
> return zhdr;
> }
> @@ -556,7 +558,7 @@ static void __release_z3fold_page(struct
> z3fold_page_unlock(zhdr);
>
> spin_lock(&pool->stale_lock);
> - list_add(&zhdr->buddy, &pool->stale);
> + list_add(&zhdr->stale_node, &pool->stale);
> queue_work(pool->release_wq, &pool->work);
> spin_unlock(&pool->stale_lock);
> }
> @@ -598,10 +600,10 @@ static void free_pages_work(struct work_
> spin_lock(&pool->stale_lock);
> while (!list_empty(&pool->stale)) {
> struct z3fold_header *zhdr = list_first_entry(&pool->stale,
> - struct z3fold_header, buddy);
> + struct z3fold_header, stale_node);
> struct page *page = virt_to_page(zhdr);
>
> - list_del(&zhdr->buddy);
> + list_del(&zhdr->stale_node);
> if (WARN_ON(!test_bit(PAGE_STALE, &page->private)))
> continue;
> spin_unlock(&pool->stale_lock);
> @@ -1140,14 +1142,14 @@ retry:
> if (can_sleep) {
> spin_lock(&pool->stale_lock);
> zhdr = list_first_entry_or_null(&pool->stale,
> - struct z3fold_header, buddy);
> + struct z3fold_header, stale_node);
> /*
> * Before allocating a page, let's see if we can take one from
> * the stale pages list. cancel_work_sync() can sleep so we
> * limit this case to the contexts where we can sleep
> */
> if (zhdr) {
> - list_del(&zhdr->buddy);
> + list_del(&zhdr->stale_node);
> spin_unlock(&pool->stale_lock);
> cancel_work_sync(&zhdr->work);
> page = virt_to_page(zhdr);
> --
The computer with patch above worked for a record time (3 days)
without freezing.
https://postimg.cc/VShF5cJN
But after 3 days hangs with follow trace:
[263314.718807] general protection fault, probably for non-canonical
address 0x72c1224000000000: 0000 [#1] SMP NOPTI
[263314.718828] CPU: 3 PID: 476750 Comm: Chrome_IOThread Tainted: G
W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263314.718831] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263314.718835] RIP: 0010:__list_add_valid+0x3/0x40
[263314.718841] Code: e9 5d ff ff ff b8 f4 ff ff ff e9 53 ff ff ff 48
c7 00 00 00 00 00 c7 40 08 00 00 00 00 e9 6b ff ff ff cc cc cc cc cc
49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 03 cf 62 00 4c 8b 0a 4d 39 c1 0f
85 1f
[263314.718845] RSP: 0018:ffffae4345b5fac0 EFLAGS: 00010282
[263314.718849] RAX: 00000000000003c0 RBX: ffffed09c02ade80 RCX:
0000000000000000
[263314.718851] RDX: 72c1224000000000 RSI: ffffce433c462004 RDI:
ffff9d414ab7a000
[263314.718853] RBP: ffffce433c462004 R08: 72c1224000000000 R09:
0000000000000000
[263314.718856] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053008
[263314.718858] R13: ffff9d414ab7a010 R14: 72c1224000000000 R15:
ffff9d414ab7a000
[263314.718860] FS: 00007f8ef636f640(0000) GS:ffff9d4947000000(0000)
knlGS:0000000000000000
[263314.718863] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263314.718865] CR2: 000055634cbef560 CR3: 00000002dbdbc000 CR4:
0000000000350ee0
[263314.718867] Call Trace:
[263314.718875] do_compact_page+0x28d/0xb60
[263314.718884] ? z3fold_zpool_free+0x3a8/0x590
[263314.718888] zswap_free_entry+0x43/0x70
[263314.718892] zswap_frontswap_invalidate_page+0x8c/0x90
[263314.718895] __frontswap_invalidate_page+0x5d/0x90
[263314.718898] swap_range_free+0xcd/0xf0
[263314.718901] swapcache_free_entries+0x128/0x1a0
[263314.718904] free_swap_slot+0xbb/0xd0
[263314.718907] __swap_entry_free+0x7a/0xa0
[263314.718910] free_swap_and_cache+0x35/0x80
[263314.718913] shmem_undo_range+0x188/0x7e0
[263314.718919] ? ldsem_down_read+0x1f/0x40
[263314.718925] shmem_evict_inode+0xe6/0x290
[263314.718928] ? lock_release+0x1ef/0x410
[263314.718932] ? var_wake_function+0x20/0x20
[263314.718936] evict+0xcf/0x1d0
[263314.718940] __dentry_kill+0xe8/0x190
[263314.718943] ? dput+0x20/0x480
[263314.718946] dput+0x2b8/0x480
[263314.718949] __fput+0x102/0x260
[263314.718952] task_work_run+0x5c/0xa0
[263314.718957] exit_to_user_mode_prepare+0x232/0x240
[263314.718960] syscall_exit_to_user_mode+0x27/0x70
[263314.718964] entry_SYSCALL_64_after_hwframe+0x44/0xae
[263314.718967] RIP: 0033:0x7f8f0b15d16b
[263314.718972] Code: 8b 15 09 7d 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff
ff ff eb 89 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 0b 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 7c 0c 00 f7 d8 64 89
01 48
[263314.718974] RSP: 002b:00007f8ef636d308 EFLAGS: 00000246 ORIG_RAX:
000000000000000b
[263314.718977] RAX: 0000000000000000 RBX: 00003e1813862928 RCX:
00007f8f0b15d16b
[263314.718979] RDX: 0000000000000000 RSI: 0000000000a4e000 RDI:
00007f8e5b43e000
[263314.718981] RBP: 00007f8ef636d320 R08: 0000000000000000 R09:
0000000000000000
[263314.718983] R10: 0000000000000000 R11: 0000000000000246 R12:
00007f8e5b43e000
[263314.718985] R13: 00003e18138628e0 R14: 00007f8ef636d330 R15:
00007f8ef636d330
[263314.718989] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263314.719032] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263314.719079] ---[ end trace ba885cda1af90fb7 ]---
[263314.719081] RIP: 0010:__list_add_valid+0x3/0x40
[263314.719084] Code: e9 5d ff ff ff b8 f4 ff ff ff e9 53 ff ff ff 48
c7 00 00 00 00 00 c7 40 08 00 00 00 00 e9 6b ff ff ff cc cc cc cc cc
49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 03 cf 62 00 4c 8b 0a 4d 39 c1 0f
85 1f
[263314.719086] RSP: 0018:ffffae4345b5fac0 EFLAGS: 00010282
[263314.719089] RAX: 00000000000003c0 RBX: ffffed09c02ade80 RCX:
0000000000000000
[263314.719091] RDX: 72c1224000000000 RSI: ffffce433c462004 RDI:
ffff9d414ab7a000
[263314.719093] RBP: ffffce433c462004 R08: 72c1224000000000 R09:
0000000000000000
[263314.719095] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053008
[263314.719097] R13: ffff9d414ab7a010 R14: 72c1224000000000 R15:
ffff9d414ab7a000
[263314.719099] FS: 00007f8ef636f640(0000) GS:ffff9d4947000000(0000)
knlGS:0000000000000000
[263314.719101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263314.719104] CR2: 000055634cbef560 CR3: 00000002dbdbc000 CR4:
0000000000350ee0
[263314.719106] note: Chrome_IOThread[476750] exited with preempt_count 5
[263341.868981] watchdog: BUG: soft lockup - CPU#0 stuck for 23s!
[ThreadPoolForeg:513140]
[263341.868991] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.869025] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.869052] irq event stamp: 0
[263341.869054] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[263341.869057] hardirqs last disabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.869061] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.869064] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.869067] CPU: 0 PID: 513140 Comm: ThreadPoolForeg Tainted: G
D W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.869070] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.869073] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[263341.869076] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[263341.869079] RSP: 0000:ffffae435505bbf0 EFLAGS: 00000246
[263341.869082] RAX: 0000000000000000 RBX: 0000000000e6c4f7 RCX:
ffff9d4946bef300
[263341.869084] RDX: ffff9d424197e4e8 RSI: 0000000000040000 RDI:
0000000000000014
[263341.869087] RBP: ffff9d424197e4e8 R08: 0000000000040000 R09:
0000000000000000
[263341.869089] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d424197e500
[263341.869091] R13: ffff9d424197e4e8 R14: ffffffffa9e87020 R15:
ffff9d424197e4e0
[263341.869094] FS: 00007ff2de694640(0000) GS:ffff9d4946a00000(0000)
knlGS:0000000000000000
[263341.869096] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.869099] CR2: 00000070bfa4d3c0 CR3: 00000002f2878000 CR4:
0000000000350ef0
[263341.869101] Call Trace:
[263341.869104] do_raw_spin_lock+0x94/0xa0
[263341.869107] _raw_spin_lock+0x63/0x80
[263341.869111] zswap_frontswap_load+0x30/0x2f0
[263341.869115] ? trace_hardirqs_on+0x1b/0xe0
[263341.869120] __frontswap_load+0xc3/0x160
[263341.869123] swap_readpage+0x25b/0x440
[263341.869127] swapin_readahead+0x450/0x4e0
[263341.869130] ? lock_release+0x1ef/0x410
[263341.869134] do_swap_page+0x4a4/0x900
[263341.869137] __handle_mm_fault+0xbd6/0x1610
[263341.869140] ? lock_acquire+0x177/0x3a0
[263341.869145] handle_mm_fault+0xa2/0x270
[263341.869148] do_user_addr_fault+0x1ea/0x6b0
[263341.869152] exc_page_fault+0x67/0x2a0
[263341.869155] ? asm_exc_page_fault+0x8/0x30
[263341.869158] asm_exc_page_fault+0x1e/0x30
[263341.869161] RIP: 0033:0x55e1b76f7713
[263341.869164] Code: 8b 9f c8 00 00 00 4d 8b b7 d0 00 00 00 4c 39 f3
74 5a 4c 8d 25 8e 18 3c 05 4c 8d 2d aa 59 e6 fb 0f 1f 80 00 00 00 00
48 8b 3b <48> 8b 07 48 89 c1 4c 29 e1 48 c1 c9 03 48 81 f9 9f 00 00 00
0f 87
[263341.869166] RSP: 002b:00007ff2de693430 EFLAGS: 00010287
[263341.869169] RAX: 0000000000000000 RBX: 00007ff2de693568 RCX:
00000070c00c7be0
[263341.869171] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
00000070bfa4d3c0
[263341.869174] RBP: 00007ff2de693480 R08: 0000000000000000 R09:
00000000000000ca
[263341.869176] R10: 00007fff78981080 R11: 00007fff78981090 R12:
000055e1bcab8f90
[263341.869178] R13: 000055e1b355d0b3 R14: 00007ff2de693578 R15:
00007ff2de693500
[263341.870981] watchdog: BUG: soft lockup - CPU#1 stuck for 23s!
[steamwebhelper:3496089]
[263341.870987] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.871021] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.871048] irq event stamp: 0
[263341.871050] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[263341.871054] hardirqs last disabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.871058] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.871061] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.871064] CPU: 1 PID: 3496089 Comm: steamwebhelper Tainted: G
D W L --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.871067] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.871069] RIP: 0010:native_queued_spin_lock_slowpath+0x137/0x200
[263341.871073] Code: e0 a9 1d 00 eb cb 41 83 c0 01 c1 e6 10 41 c1 e0
12 44 09 c6 89 f0 c1 e8 10 66 87 42 02 89 c7 c1 e7 10 75 73 31 ff eb
02 f3 90 <8b> 02 66 85 c0 75 f7 41 89 c0 66 45 31 c0 44 39 c6 0f 84 9b
00 00
[263341.871076] RSP: 0018:ffffae43549eb5f8 EFLAGS: 00000202
[263341.871078] RAX: 00000000000c0101 RBX: ffff9d414bc00000 RCX:
ffff9d4946def300
[263341.871081] RDX: ffff9d4253053008 RSI: 0000000000080000 RDI:
0000000000000000
[263341.871083] RBP: ffff9d4253053008 R08: 0000000000080000 R09:
0000000000000000
[263341.871085] R10: 0000000000000000 R11: 0000000000000001 R12:
ffff9d4253053020
[263341.871088] R13: ffff9d414bc00010 R14: 0000000000000000 R15:
ffffed09c02f0000
[263341.871090] FS: 00007f725d561d40(0000) GS:ffff9d4946c00000(0000)
knlGS:0000000000000000
[263341.871093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.871095] CR2: 0000366b51ea6fe8 CR3: 00000003865f2000 CR4:
0000000000350ee0
[263341.871098] Call Trace:
[263341.871101] do_raw_spin_lock+0x94/0xa0
[263341.871104] _raw_spin_lock+0x63/0x80
[263341.871107] z3fold_page_isolate+0xbd/0x1b0
[263341.871112] isolate_movable_page+0x94/0x180
[263341.871115] isolate_migratepages_block+0x5db/0x1120
[263341.871120] ? lock_release+0x1ef/0x410
[263341.871124] compact_zone+0x5a4/0xfd0
[263341.871129] compact_zone_order+0xaa/0xf0
[263341.871134] try_to_compact_pages+0x111/0x3b0
[263341.871138] __alloc_pages_direct_compact+0x79/0x210
[263341.871142] __alloc_pages_slowpath.constprop.0+0x1d0/0xf90
[263341.871147] ? __alloc_pages_nodemask+0x2e3/0x400
[263341.871151] ? lock_release+0x1ef/0x410
[263341.871154] __alloc_pages_nodemask+0x37d/0x400
[263341.871159] ttm_pool_alloc+0x2a3/0x630 [ttm]
[263341.871167] ttm_tt_populate+0x37/0xe0 [ttm]
[263341.871172] ttm_bo_handle_move_mem+0x13a/0x170 [ttm]
[263341.871179] ttm_bo_validate+0x15f/0x1b0 [ttm]
[263341.871184] ? lock_release+0x1ef/0x410
[263341.871189] ttm_bo_init_reserved+0x2f7/0x3e0 [ttm]
[263341.871195] amdgpu_bo_do_create+0x1a8/0x630 [amdgpu]
[263341.871312] ? amdgpu_bo_subtract_pin_size+0x50/0x50 [amdgpu]
[263341.871422] amdgpu_bo_create+0x30/0x2e0 [amdgpu]
[263341.871531] ? lock_acquire+0x177/0x3a0
[263341.871535] ? trace_hardirqs_on+0x1b/0xe0
[263341.871539] ? _raw_spin_unlock_irqrestore+0x37/0x40
[263341.871543] ? lock_release+0x1ef/0x410
[263341.871547] amdgpu_gem_create_ioctl+0x10e/0x370 [amdgpu]
[263341.871664] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
[263341.871774] drm_ioctl_kernel+0x89/0xe0 [drm]
[263341.871797] drm_ioctl+0x20f/0x3c0 [drm]
[263341.871816] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
[263341.871927] ? selinux_file_ioctl+0x147/0x200
[263341.871931] ? lock_acquired+0x200/0x390
[263341.871934] ? lock_release+0x1ef/0x410
[263341.871937] ? trace_hardirqs_on+0x1b/0xe0
[263341.871940] ? _raw_spin_unlock_irqrestore+0x37/0x40
[263341.871944] amdgpu_drm_ioctl+0x49/0x80 [amdgpu]
[263341.872053] __x64_sys_ioctl+0x82/0xb0
[263341.872058] do_syscall_64+0x33/0x40
[263341.872061] entry_SYSCALL_64_after_hwframe+0x44/0xae
[263341.872065] RIP: 0033:0x7f72610b22bb
[263341.872068] Code: ff ff ff 85 c0 79 9b 49 c7 c4 ff ff ff ff 5b 5d
4c 89 e0 41 5c c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 10 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 85 bb 0c 00 f7 d8 64 89
01 48
[263341.872071] RSP: 002b:00007ffcd94a01f8 EFLAGS: 00000246 ORIG_RAX:
0000000000000010
[263341.872074] RAX: ffffffffffffffda RBX: 00007ffcd94a0250 RCX:
00007f72610b22bb
[263341.872076] RDX: 00007ffcd94a0250 RSI: 00000000c0206440 RDI:
0000000000000016
[263341.872078] RBP: 00000000c0206440 R08: 0000000000000009 R09:
00000000000000b8
[263341.872081] R10: 00007ffcd9568080 R11: 0000000000000246 R12:
000008c86f1ae3c0
[263341.872083] R13: 0000000000000016 R14: 0000000000200000 R15:
00000000019c6000
[263341.872983] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kswapd0:288]
[263341.872991] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.873025] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.873052] irq event stamp: 36
[263341.873054] hardirqs last enabled at (35): [<ffffffffa8d61117>]
_raw_spin_unlock_irqrestore+0x37/0x40
[263341.873059] hardirqs last disabled at (36): [<ffffffffa8d5a8a9>]
__schedule+0x6e9/0xb20
[263341.873063] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.873066] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.873069] CPU: 2 PID: 288 Comm: kswapd0 Tainted: G D W L
--------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.873073] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.873075] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[263341.873079] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[263341.873082] RSP: 0018:ffffae4340943898 EFLAGS: 00000246
[263341.873085] RAX: 0000000000000000 RBX: ffff9d4253053000 RCX:
ffff9d4946fef300
[263341.873087] RDX: ffff9d4253053008 RSI: 00000000000c0000 RDI:
0000000000000001
[263341.873090] RBP: ffff9d4253053008 R08: 00000000000c0000 R09:
0000000000000000
[263341.873092] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053020
[263341.873094] R13: 0000000000000003 R14: 0000000000000003 R15:
ffff9d41760b2000
[263341.873097] FS: 0000000000000000(0000) GS:ffff9d4946e00000(0000)
knlGS:0000000000000000
[263341.873099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.873102] CR2: 0000021509b13000 CR3: 00000004130c8000 CR4:
0000000000350ee0
[263341.873104] Call Trace:
[263341.873107] do_raw_spin_lock+0x94/0xa0
[263341.873110] _raw_spin_lock+0x63/0x80
[263341.873114] __z3fold_alloc+0x78/0x3d0
[263341.873118] z3fold_zpool_malloc+0x4a5/0x7c0
[263341.873121] ? _raw_spin_unlock+0x1f/0x30
[263341.873125] zswap_frontswap_store+0x43e/0x890
[263341.873130] __frontswap_store+0xc8/0x170
[263341.873134] swap_writepage+0x39/0x70
[263341.873137] pageout+0x125/0x540
[263341.873142] shrink_page_list+0x131b/0x1bb0
[263341.873147] shrink_inactive_list+0x12a/0x440
[263341.873152] shrink_lruvec+0x4aa/0x6d0
[263341.873158] shrink_node+0x2d1/0x700
[263341.873163] balance_pgdat+0x2f5/0x650
[263341.873169] kswapd+0x21d/0x4d0
[263341.873172] ? do_wait_intr_irq+0xd0/0xd0
[263341.873176] ? balance_pgdat+0x650/0x650
[263341.873179] kthread+0x13a/0x150
[263341.873183] ? __kthread_bind_mask+0x60/0x60
[263341.873187] ret_from_fork+0x22/0x30
It related?
Full kernel log is here: https://pastebin.com/x0KbXN9L
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
@ 2021-03-05 9:33 ` Mikhail Gavrilov
0 siblings, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-03-05 9:33 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Mon, 1 Mar 2021 at 08:11, Hillf Danton <hdanton@sina.com> wrote:
>
> What we learn from your reports is
>
> 1/ in z3fold_free(), kref_put() creates the ground zero for the race
> cases reported,
>
> 2/ the stale_lock in combination with lock makes things more
> complicated than thought.
>
> Instead of dropping something in the zero spot, the fix below goes the
> road mentioned before in this mail thread - add another list_head in
> parallel to the buddy and s/buddy/stale_node/ under every case of
> stale_lock.
>
> --- x/mm/z3fold.c
> +++ y/mm/z3fold.c
> @@ -127,6 +127,7 @@ struct z3fold_header {
> unsigned short first_num:2;
> unsigned short mapped_count:2;
> unsigned short foreign_handles:2;
> + struct list_head stale_node;
> };
>
> /**
> @@ -429,6 +430,7 @@ static struct z3fold_header *init_z3fold
> zhdr->slots = slots;
> zhdr->pool = pool;
> INIT_LIST_HEAD(&zhdr->buddy);
> + INIT_LIST_HEAD(&zhdr->stale_node);
> INIT_WORK(&zhdr->work, compact_page_work);
> return zhdr;
> }
> @@ -556,7 +558,7 @@ static void __release_z3fold_page(struct
> z3fold_page_unlock(zhdr);
>
> spin_lock(&pool->stale_lock);
> - list_add(&zhdr->buddy, &pool->stale);
> + list_add(&zhdr->stale_node, &pool->stale);
> queue_work(pool->release_wq, &pool->work);
> spin_unlock(&pool->stale_lock);
> }
> @@ -598,10 +600,10 @@ static void free_pages_work(struct work_
> spin_lock(&pool->stale_lock);
> while (!list_empty(&pool->stale)) {
> struct z3fold_header *zhdr = list_first_entry(&pool->stale,
> - struct z3fold_header, buddy);
> + struct z3fold_header, stale_node);
> struct page *page = virt_to_page(zhdr);
>
> - list_del(&zhdr->buddy);
> + list_del(&zhdr->stale_node);
> if (WARN_ON(!test_bit(PAGE_STALE, &page->private)))
> continue;
> spin_unlock(&pool->stale_lock);
> @@ -1140,14 +1142,14 @@ retry:
> if (can_sleep) {
> spin_lock(&pool->stale_lock);
> zhdr = list_first_entry_or_null(&pool->stale,
> - struct z3fold_header, buddy);
> + struct z3fold_header, stale_node);
> /*
> * Before allocating a page, let's see if we can take one from
> * the stale pages list. cancel_work_sync() can sleep so we
> * limit this case to the contexts where we can sleep
> */
> if (zhdr) {
> - list_del(&zhdr->buddy);
> + list_del(&zhdr->stale_node);
> spin_unlock(&pool->stale_lock);
> cancel_work_sync(&zhdr->work);
> page = virt_to_page(zhdr);
> --
The computer with patch above worked for a record time (3 days)
without freezing.
https://postimg.cc/VShF5cJN
But after 3 days hangs with follow trace:
[263314.718807] general protection fault, probably for non-canonical
address 0x72c1224000000000: 0000 [#1] SMP NOPTI
[263314.718828] CPU: 3 PID: 476750 Comm: Chrome_IOThread Tainted: G
W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263314.718831] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263314.718835] RIP: 0010:__list_add_valid+0x3/0x40
[263314.718841] Code: e9 5d ff ff ff b8 f4 ff ff ff e9 53 ff ff ff 48
c7 00 00 00 00 00 c7 40 08 00 00 00 00 e9 6b ff ff ff cc cc cc cc cc
49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 03 cf 62 00 4c 8b 0a 4d 39 c1 0f
85 1f
[263314.718845] RSP: 0018:ffffae4345b5fac0 EFLAGS: 00010282
[263314.718849] RAX: 00000000000003c0 RBX: ffffed09c02ade80 RCX:
0000000000000000
[263314.718851] RDX: 72c1224000000000 RSI: ffffce433c462004 RDI:
ffff9d414ab7a000
[263314.718853] RBP: ffffce433c462004 R08: 72c1224000000000 R09:
0000000000000000
[263314.718856] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053008
[263314.718858] R13: ffff9d414ab7a010 R14: 72c1224000000000 R15:
ffff9d414ab7a000
[263314.718860] FS: 00007f8ef636f640(0000) GS:ffff9d4947000000(0000)
knlGS:0000000000000000
[263314.718863] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263314.718865] CR2: 000055634cbef560 CR3: 00000002dbdbc000 CR4:
0000000000350ee0
[263314.718867] Call Trace:
[263314.718875] do_compact_page+0x28d/0xb60
[263314.718884] ? z3fold_zpool_free+0x3a8/0x590
[263314.718888] zswap_free_entry+0x43/0x70
[263314.718892] zswap_frontswap_invalidate_page+0x8c/0x90
[263314.718895] __frontswap_invalidate_page+0x5d/0x90
[263314.718898] swap_range_free+0xcd/0xf0
[263314.718901] swapcache_free_entries+0x128/0x1a0
[263314.718904] free_swap_slot+0xbb/0xd0
[263314.718907] __swap_entry_free+0x7a/0xa0
[263314.718910] free_swap_and_cache+0x35/0x80
[263314.718913] shmem_undo_range+0x188/0x7e0
[263314.718919] ? ldsem_down_read+0x1f/0x40
[263314.718925] shmem_evict_inode+0xe6/0x290
[263314.718928] ? lock_release+0x1ef/0x410
[263314.718932] ? var_wake_function+0x20/0x20
[263314.718936] evict+0xcf/0x1d0
[263314.718940] __dentry_kill+0xe8/0x190
[263314.718943] ? dput+0x20/0x480
[263314.718946] dput+0x2b8/0x480
[263314.718949] __fput+0x102/0x260
[263314.718952] task_work_run+0x5c/0xa0
[263314.718957] exit_to_user_mode_prepare+0x232/0x240
[263314.718960] syscall_exit_to_user_mode+0x27/0x70
[263314.718964] entry_SYSCALL_64_after_hwframe+0x44/0xae
[263314.718967] RIP: 0033:0x7f8f0b15d16b
[263314.718972] Code: 8b 15 09 7d 0c 00 f7 d8 64 89 02 48 c7 c0 ff ff
ff ff eb 89 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 0b 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 7c 0c 00 f7 d8 64 89
01 48
[263314.718974] RSP: 002b:00007f8ef636d308 EFLAGS: 00000246 ORIG_RAX:
000000000000000b
[263314.718977] RAX: 0000000000000000 RBX: 00003e1813862928 RCX:
00007f8f0b15d16b
[263314.718979] RDX: 0000000000000000 RSI: 0000000000a4e000 RDI:
00007f8e5b43e000
[263314.718981] RBP: 00007f8ef636d320 R08: 0000000000000000 R09:
0000000000000000
[263314.718983] R10: 0000000000000000 R11: 0000000000000246 R12:
00007f8e5b43e000
[263314.718985] R13: 00003e18138628e0 R14: 00007f8ef636d330 R15:
00007f8ef636d330
[263314.718989] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263314.719032] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263314.719079] ---[ end trace ba885cda1af90fb7 ]---
[263314.719081] RIP: 0010:__list_add_valid+0x3/0x40
[263314.719084] Code: e9 5d ff ff ff b8 f4 ff ff ff e9 53 ff ff ff 48
c7 00 00 00 00 00 c7 40 08 00 00 00 00 e9 6b ff ff ff cc cc cc cc cc
49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 03 cf 62 00 4c 8b 0a 4d 39 c1 0f
85 1f
[263314.719086] RSP: 0018:ffffae4345b5fac0 EFLAGS: 00010282
[263314.719089] RAX: 00000000000003c0 RBX: ffffed09c02ade80 RCX:
0000000000000000
[263314.719091] RDX: 72c1224000000000 RSI: ffffce433c462004 RDI:
ffff9d414ab7a000
[263314.719093] RBP: ffffce433c462004 R08: 72c1224000000000 R09:
0000000000000000
[263314.719095] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053008
[263314.719097] R13: ffff9d414ab7a010 R14: 72c1224000000000 R15:
ffff9d414ab7a000
[263314.719099] FS: 00007f8ef636f640(0000) GS:ffff9d4947000000(0000)
knlGS:0000000000000000
[263314.719101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263314.719104] CR2: 000055634cbef560 CR3: 00000002dbdbc000 CR4:
0000000000350ee0
[263314.719106] note: Chrome_IOThread[476750] exited with preempt_count 5
[263341.868981] watchdog: BUG: soft lockup - CPU#0 stuck for 23s!
[ThreadPoolForeg:513140]
[263341.868991] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.869025] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.869052] irq event stamp: 0
[263341.869054] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[263341.869057] hardirqs last disabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.869061] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.869064] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.869067] CPU: 0 PID: 513140 Comm: ThreadPoolForeg Tainted: G
D W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.869070] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.869073] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[263341.869076] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[263341.869079] RSP: 0000:ffffae435505bbf0 EFLAGS: 00000246
[263341.869082] RAX: 0000000000000000 RBX: 0000000000e6c4f7 RCX:
ffff9d4946bef300
[263341.869084] RDX: ffff9d424197e4e8 RSI: 0000000000040000 RDI:
0000000000000014
[263341.869087] RBP: ffff9d424197e4e8 R08: 0000000000040000 R09:
0000000000000000
[263341.869089] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d424197e500
[263341.869091] R13: ffff9d424197e4e8 R14: ffffffffa9e87020 R15:
ffff9d424197e4e0
[263341.869094] FS: 00007ff2de694640(0000) GS:ffff9d4946a00000(0000)
knlGS:0000000000000000
[263341.869096] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.869099] CR2: 00000070bfa4d3c0 CR3: 00000002f2878000 CR4:
0000000000350ef0
[263341.869101] Call Trace:
[263341.869104] do_raw_spin_lock+0x94/0xa0
[263341.869107] _raw_spin_lock+0x63/0x80
[263341.869111] zswap_frontswap_load+0x30/0x2f0
[263341.869115] ? trace_hardirqs_on+0x1b/0xe0
[263341.869120] __frontswap_load+0xc3/0x160
[263341.869123] swap_readpage+0x25b/0x440
[263341.869127] swapin_readahead+0x450/0x4e0
[263341.869130] ? lock_release+0x1ef/0x410
[263341.869134] do_swap_page+0x4a4/0x900
[263341.869137] __handle_mm_fault+0xbd6/0x1610
[263341.869140] ? lock_acquire+0x177/0x3a0
[263341.869145] handle_mm_fault+0xa2/0x270
[263341.869148] do_user_addr_fault+0x1ea/0x6b0
[263341.869152] exc_page_fault+0x67/0x2a0
[263341.869155] ? asm_exc_page_fault+0x8/0x30
[263341.869158] asm_exc_page_fault+0x1e/0x30
[263341.869161] RIP: 0033:0x55e1b76f7713
[263341.869164] Code: 8b 9f c8 00 00 00 4d 8b b7 d0 00 00 00 4c 39 f3
74 5a 4c 8d 25 8e 18 3c 05 4c 8d 2d aa 59 e6 fb 0f 1f 80 00 00 00 00
48 8b 3b <48> 8b 07 48 89 c1 4c 29 e1 48 c1 c9 03 48 81 f9 9f 00 00 00
0f 87
[263341.869166] RSP: 002b:00007ff2de693430 EFLAGS: 00010287
[263341.869169] RAX: 0000000000000000 RBX: 00007ff2de693568 RCX:
00000070c00c7be0
[263341.869171] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
00000070bfa4d3c0
[263341.869174] RBP: 00007ff2de693480 R08: 0000000000000000 R09:
00000000000000ca
[263341.869176] R10: 00007fff78981080 R11: 00007fff78981090 R12:
000055e1bcab8f90
[263341.869178] R13: 000055e1b355d0b3 R14: 00007ff2de693578 R15:
00007ff2de693500
[263341.870981] watchdog: BUG: soft lockup - CPU#1 stuck for 23s!
[steamwebhelper:3496089]
[263341.870987] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.871021] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.871048] irq event stamp: 0
[263341.871050] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[263341.871054] hardirqs last disabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.871058] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.871061] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.871064] CPU: 1 PID: 3496089 Comm: steamwebhelper Tainted: G
D W L --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.871067] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.871069] RIP: 0010:native_queued_spin_lock_slowpath+0x137/0x200
[263341.871073] Code: e0 a9 1d 00 eb cb 41 83 c0 01 c1 e6 10 41 c1 e0
12 44 09 c6 89 f0 c1 e8 10 66 87 42 02 89 c7 c1 e7 10 75 73 31 ff eb
02 f3 90 <8b> 02 66 85 c0 75 f7 41 89 c0 66 45 31 c0 44 39 c6 0f 84 9b
00 00
[263341.871076] RSP: 0018:ffffae43549eb5f8 EFLAGS: 00000202
[263341.871078] RAX: 00000000000c0101 RBX: ffff9d414bc00000 RCX:
ffff9d4946def300
[263341.871081] RDX: ffff9d4253053008 RSI: 0000000000080000 RDI:
0000000000000000
[263341.871083] RBP: ffff9d4253053008 R08: 0000000000080000 R09:
0000000000000000
[263341.871085] R10: 0000000000000000 R11: 0000000000000001 R12:
ffff9d4253053020
[263341.871088] R13: ffff9d414bc00010 R14: 0000000000000000 R15:
ffffed09c02f0000
[263341.871090] FS: 00007f725d561d40(0000) GS:ffff9d4946c00000(0000)
knlGS:0000000000000000
[263341.871093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.871095] CR2: 0000366b51ea6fe8 CR3: 00000003865f2000 CR4:
0000000000350ee0
[263341.871098] Call Trace:
[263341.871101] do_raw_spin_lock+0x94/0xa0
[263341.871104] _raw_spin_lock+0x63/0x80
[263341.871107] z3fold_page_isolate+0xbd/0x1b0
[263341.871112] isolate_movable_page+0x94/0x180
[263341.871115] isolate_migratepages_block+0x5db/0x1120
[263341.871120] ? lock_release+0x1ef/0x410
[263341.871124] compact_zone+0x5a4/0xfd0
[263341.871129] compact_zone_order+0xaa/0xf0
[263341.871134] try_to_compact_pages+0x111/0x3b0
[263341.871138] __alloc_pages_direct_compact+0x79/0x210
[263341.871142] __alloc_pages_slowpath.constprop.0+0x1d0/0xf90
[263341.871147] ? __alloc_pages_nodemask+0x2e3/0x400
[263341.871151] ? lock_release+0x1ef/0x410
[263341.871154] __alloc_pages_nodemask+0x37d/0x400
[263341.871159] ttm_pool_alloc+0x2a3/0x630 [ttm]
[263341.871167] ttm_tt_populate+0x37/0xe0 [ttm]
[263341.871172] ttm_bo_handle_move_mem+0x13a/0x170 [ttm]
[263341.871179] ttm_bo_validate+0x15f/0x1b0 [ttm]
[263341.871184] ? lock_release+0x1ef/0x410
[263341.871189] ttm_bo_init_reserved+0x2f7/0x3e0 [ttm]
[263341.871195] amdgpu_bo_do_create+0x1a8/0x630 [amdgpu]
[263341.871312] ? amdgpu_bo_subtract_pin_size+0x50/0x50 [amdgpu]
[263341.871422] amdgpu_bo_create+0x30/0x2e0 [amdgpu]
[263341.871531] ? lock_acquire+0x177/0x3a0
[263341.871535] ? trace_hardirqs_on+0x1b/0xe0
[263341.871539] ? _raw_spin_unlock_irqrestore+0x37/0x40
[263341.871543] ? lock_release+0x1ef/0x410
[263341.871547] amdgpu_gem_create_ioctl+0x10e/0x370 [amdgpu]
[263341.871664] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
[263341.871774] drm_ioctl_kernel+0x89/0xe0 [drm]
[263341.871797] drm_ioctl+0x20f/0x3c0 [drm]
[263341.871816] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
[263341.871927] ? selinux_file_ioctl+0x147/0x200
[263341.871931] ? lock_acquired+0x200/0x390
[263341.871934] ? lock_release+0x1ef/0x410
[263341.871937] ? trace_hardirqs_on+0x1b/0xe0
[263341.871940] ? _raw_spin_unlock_irqrestore+0x37/0x40
[263341.871944] amdgpu_drm_ioctl+0x49/0x80 [amdgpu]
[263341.872053] __x64_sys_ioctl+0x82/0xb0
[263341.872058] do_syscall_64+0x33/0x40
[263341.872061] entry_SYSCALL_64_after_hwframe+0x44/0xae
[263341.872065] RIP: 0033:0x7f72610b22bb
[263341.872068] Code: ff ff ff 85 c0 79 9b 49 c7 c4 ff ff ff ff 5b 5d
4c 89 e0 41 5c c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 10 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 85 bb 0c 00 f7 d8 64 89
01 48
[263341.872071] RSP: 002b:00007ffcd94a01f8 EFLAGS: 00000246 ORIG_RAX:
0000000000000010
[263341.872074] RAX: ffffffffffffffda RBX: 00007ffcd94a0250 RCX:
00007f72610b22bb
[263341.872076] RDX: 00007ffcd94a0250 RSI: 00000000c0206440 RDI:
0000000000000016
[263341.872078] RBP: 00000000c0206440 R08: 0000000000000009 R09:
00000000000000b8
[263341.872081] R10: 00007ffcd9568080 R11: 0000000000000246 R12:
000008c86f1ae3c0
[263341.872083] R13: 0000000000000016 R14: 0000000000200000 R15:
00000000019c6000
[263341.872983] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kswapd0:288]
[263341.872991] Modules linked in: nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib tun snd_seq_dummy snd_hrtimer uinput nls_utf8
isofs rfcomm netconsole nft_objref nf_conntrack_netbios_ns
nf_conntrack_broadcast nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat
fat hid_logitech_hidpp hid_logitech_dj snd_hda_codec_realtek
snd_hda_codec_generic mt76x2u mt76x2_common ledtrig_audio mt76x02_usb
mt76_usb snd_hda_codec_hdmi mt76x02_lib intel_rapl_msr
intel_rapl_common iwlmvm mt76 snd_hda_intel snd_intel_dspcfg
soundwire_intel soundwire_generic_allocation snd_soc_core edac_mce_amd
joydev mac80211 snd_compress snd_pcm_dmaengine snd_usb_audio
soundwire_cadence kvm_amd snd_hda_codec btusb btrtl btbcm btintel
snd_hda_core snd_usbmidi_lib uvcvideo ac97_bus snd_seq iwlwifi kvm
snd_rawmidi bluetooth snd_hwdep videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 videobuf2_common
[263341.873025] eeepc_wmi snd_seq_device asus_wmi snd_pcm videodev
sparse_keymap xpad ecdh_generic mc libarc4 irqbypass wmi_bmof
ff_memless cfg80211 video ecc snd_timer rapl sp5100_tco snd i2c_piix4
k10temp soundcore rfkill acpi_cpufreq binfmt_misc uas usb_storage
amdgpu drm_ttm_helper ttm iommu_v2 crct10dif_pclmul gpu_sched
crc32_pclmul crc32c_intel drm_kms_helper cec drm ghash_clmulni_intel
ccp igb nvme dca nvme_core i2c_algo_bit wmi pinctrl_amd fuse [last
unloaded: ip_tables]
[263341.873052] irq event stamp: 36
[263341.873054] hardirqs last enabled at (35): [<ffffffffa8d61117>]
_raw_spin_unlock_irqrestore+0x37/0x40
[263341.873059] hardirqs last disabled at (36): [<ffffffffa8d5a8a9>]
__schedule+0x6e9/0xb20
[263341.873063] softirqs last enabled at (0): [<ffffffffa80dd962>]
copy_process+0x902/0x1df0
[263341.873066] softirqs last disabled at (0): [<0000000000000000>] 0x0
[263341.873069] CPU: 2 PID: 288 Comm: kswapd0 Tainted: G D W L
--------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
[263341.873073] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[263341.873075] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[263341.873079] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[263341.873082] RSP: 0018:ffffae4340943898 EFLAGS: 00000246
[263341.873085] RAX: 0000000000000000 RBX: ffff9d4253053000 RCX:
ffff9d4946fef300
[263341.873087] RDX: ffff9d4253053008 RSI: 00000000000c0000 RDI:
0000000000000001
[263341.873090] RBP: ffff9d4253053008 R08: 00000000000c0000 R09:
0000000000000000
[263341.873092] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9d4253053020
[263341.873094] R13: 0000000000000003 R14: 0000000000000003 R15:
ffff9d41760b2000
[263341.873097] FS: 0000000000000000(0000) GS:ffff9d4946e00000(0000)
knlGS:0000000000000000
[263341.873099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[263341.873102] CR2: 0000021509b13000 CR3: 00000004130c8000 CR4:
0000000000350ee0
[263341.873104] Call Trace:
[263341.873107] do_raw_spin_lock+0x94/0xa0
[263341.873110] _raw_spin_lock+0x63/0x80
[263341.873114] __z3fold_alloc+0x78/0x3d0
[263341.873118] z3fold_zpool_malloc+0x4a5/0x7c0
[263341.873121] ? _raw_spin_unlock+0x1f/0x30
[263341.873125] zswap_frontswap_store+0x43e/0x890
[263341.873130] __frontswap_store+0xc8/0x170
[263341.873134] swap_writepage+0x39/0x70
[263341.873137] pageout+0x125/0x540
[263341.873142] shrink_page_list+0x131b/0x1bb0
[263341.873147] shrink_inactive_list+0x12a/0x440
[263341.873152] shrink_lruvec+0x4aa/0x6d0
[263341.873158] shrink_node+0x2d1/0x700
[263341.873163] balance_pgdat+0x2f5/0x650
[263341.873169] kswapd+0x21d/0x4d0
[263341.873172] ? do_wait_intr_irq+0xd0/0xd0
[263341.873176] ? balance_pgdat+0x650/0x650
[263341.873179] kthread+0x13a/0x150
[263341.873183] ? __kthread_bind_mask+0x60/0x60
[263341.873187] ret_from_fork+0x22/0x30
It related?
Full kernel log is here: https://pastebin.com/x0KbXN9L
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-05 9:33 ` Mikhail Gavrilov
(?)
@ 2021-03-05 14:22 ` Hillf Danton
2021-03-08 15:42 ` Mikhail Gavrilov
-1 siblings, 1 reply; 18+ messages in thread
From: Hillf Danton @ 2021-03-05 14:22 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Fri, 5 Mar 2021 14:33:14 +0500 Mikhail Gavrilov wrote:
> On Mon, 1 Mar 2021 at 08:11, Hillf Danton <hdanton@sina.com> wrote:
> >
> > What we learn from your reports is
> >
> > 1/ in z3fold_free(), kref_put() creates the ground zero for the race
> > cases reported,
> >
> > 2/ the stale_lock in combination with lock makes things more
> > complicated than thought.
> >
> > Instead of dropping something in the zero spot, the fix below goes the
> > road mentioned before in this mail thread - add another list_head in
> > parallel to the buddy and s/buddy/stale_node/ under every case of
> > stale_lock.
> >
> > --- x/mm/z3fold.c
> > +++ y/mm/z3fold.c
> > @@ -127,6 +127,7 @@ struct z3fold_header {
> > unsigned short first_num:2;
> > unsigned short mapped_count:2;
> > unsigned short foreign_handles:2;
> > + struct list_head stale_node;
> > };
> >
> > /**
> > @@ -429,6 +430,7 @@ static struct z3fold_header *init_z3fold
> > zhdr->slots = slots;
> > zhdr->pool = pool;
> > INIT_LIST_HEAD(&zhdr->buddy);
> > + INIT_LIST_HEAD(&zhdr->stale_node);
> > INIT_WORK(&zhdr->work, compact_page_work);
> > return zhdr;
> > }
> > @@ -556,7 +558,7 @@ static void __release_z3fold_page(struct
> > z3fold_page_unlock(zhdr);
> >
> > spin_lock(&pool->stale_lock);
> > - list_add(&zhdr->buddy, &pool->stale);
> > + list_add(&zhdr->stale_node, &pool->stale);
> > queue_work(pool->release_wq, &pool->work);
> > spin_unlock(&pool->stale_lock);
> > }
> > @@ -598,10 +600,10 @@ static void free_pages_work(struct work_
> > spin_lock(&pool->stale_lock);
> > while (!list_empty(&pool->stale)) {
> > struct z3fold_header *zhdr = list_first_entry(&pool->stale,
> > - struct z3fold_header, buddy);
> > + struct z3fold_header, stale_node);
> > struct page *page = virt_to_page(zhdr);
> >
> > - list_del(&zhdr->buddy);
> > + list_del(&zhdr->stale_node);
> > if (WARN_ON(!test_bit(PAGE_STALE, &page->private)))
> > continue;
> > spin_unlock(&pool->stale_lock);
> > @@ -1140,14 +1142,14 @@ retry:
> > if (can_sleep) {
> > spin_lock(&pool->stale_lock);
> > zhdr = list_first_entry_or_null(&pool->stale,
> > - struct z3fold_header, buddy);
> > + struct z3fold_header, stale_node);
> > /*
> > * Before allocating a page, let's see if we can take one from
> > * the stale pages list. cancel_work_sync() can sleep so we
> > * limit this case to the contexts where we can sleep
> > */
> > if (zhdr) {
> > - list_del(&zhdr->buddy);
> > + list_del(&zhdr->stale_node);
> > spin_unlock(&pool->stale_lock);
> > cancel_work_sync(&zhdr->work);
> > page = virt_to_page(zhdr);
> > --
>
> The computer with patch above worked for a record time (3 days)
> without freezing.
> https://postimg.cc/VShF5cJN
>
>
> But after 3 days hangs with follow trace:
Thanks again for your report.
>
> [263314.718807] general protection fault, probably for non-canonical
> address 0x72c1224000000000: 0000 [#1] SMP NOPTI
> [263314.718828] CPU: 3 PID: 476750 Comm: Chrome_IOThread Tainted: G
> W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
> [263314.718831] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [263314.718835] RIP: 0010:__list_add_valid+0x3/0x40
> [263314.718841] Code: e9 5d ff ff ff b8 f4 ff ff ff e9 53 ff ff ff 48
> c7 00 00 00 00 00 c7 40 08 00 00 00 00 e9 6b ff ff ff cc cc cc cc cc
> 49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 03 cf 62 00 4c 8b 0a 4d 39 c1 0f
> 85 1f
> [263314.718845] RSP: 0018:ffffae4345b5fac0 EFLAGS: 00010282
> [263314.718849] RAX: 00000000000003c0 RBX: ffffed09c02ade80 RCX:
> 0000000000000000
> [263314.718851] RDX: 72c1224000000000 RSI: ffffce433c462004 RDI:
> ffff9d414ab7a000
> [263314.718853] RBP: ffffce433c462004 R08: 72c1224000000000 R09:
> 0000000000000000
> [263314.718856] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9d4253053008
> [263314.718858] R13: ffff9d414ab7a010 R14: 72c1224000000000 R15:
> ffff9d414ab7a000
> [263314.718860] FS: 00007f8ef636f640(0000) GS:ffff9d4947000000(0000)
> knlGS:0000000000000000
> [263314.718863] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [263314.718865] CR2: 000055634cbef560 CR3: 00000002dbdbc000 CR4:
> 0000000000350ee0
> [263314.718867] Call Trace:
> [263314.718875] do_compact_page+0x28d/0xb60
> [263314.718884] ? z3fold_zpool_free+0x3a8/0x590
One part of the race is the free path on CPU#3.
> [263314.718888] zswap_free_entry+0x43/0x70
> [263314.718892] zswap_frontswap_invalidate_page+0x8c/0x90
> [263314.718895] __frontswap_invalidate_page+0x5d/0x90
> [263314.718898] swap_range_free+0xcd/0xf0
> [263314.718901] swapcache_free_entries+0x128/0x1a0
> [263314.718904] free_swap_slot+0xbb/0xd0
> [263314.718907] __swap_entry_free+0x7a/0xa0
> [263314.718910] free_swap_and_cache+0x35/0x80
> [263314.718913] shmem_undo_range+0x188/0x7e0
> [263314.718919] ? ldsem_down_read+0x1f/0x40
> [263314.718925] shmem_evict_inode+0xe6/0x290
> [263314.718928] ? lock_release+0x1ef/0x410
> [263314.718932] ? var_wake_function+0x20/0x20
> [263314.718936] evict+0xcf/0x1d0
> [263314.718940] __dentry_kill+0xe8/0x190
> [263314.718943] ? dput+0x20/0x480
> [263314.718946] dput+0x2b8/0x480
> [263314.718949] __fput+0x102/0x260
> [263314.718952] task_work_run+0x5c/0xa0
> [263314.718957] exit_to_user_mode_prepare+0x232/0x240
> [263314.718960] syscall_exit_to_user_mode+0x27/0x70
> [263314.718964] entry_SYSCALL_64_after_hwframe+0x44/0xae
[...]
> [263341.868981] watchdog: BUG: soft lockup - CPU#0 stuck for 23s!
> [ThreadPoolForeg:513140]
[...]
> [263341.869052] irq event stamp: 0
> [263341.869054] hardirqs last enabled at (0): [<0000000000000000>] 0x0
> [263341.869057] hardirqs last disabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.869061] softirqs last enabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.869064] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [263341.869067] CPU: 0 PID: 513140 Comm: ThreadPoolForeg Tainted: G
> D W --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
> [263341.869070] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [263341.869073] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
> [263341.869076] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
> 48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
> 09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
> 0d 0f
> [263341.869079] RSP: 0000:ffffae435505bbf0 EFLAGS: 00000246
> [263341.869082] RAX: 0000000000000000 RBX: 0000000000e6c4f7 RCX:
> ffff9d4946bef300
> [263341.869084] RDX: ffff9d424197e4e8 RSI: 0000000000040000 RDI:
> 0000000000000014
> [263341.869087] RBP: ffff9d424197e4e8 R08: 0000000000040000 R09:
> 0000000000000000
> [263341.869089] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9d424197e500
> [263341.869091] R13: ffff9d424197e4e8 R14: ffffffffa9e87020 R15:
> ffff9d424197e4e0
> [263341.869094] FS: 00007ff2de694640(0000) GS:ffff9d4946a00000(0000)
> knlGS:0000000000000000
> [263341.869096] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [263341.869099] CR2: 00000070bfa4d3c0 CR3: 00000002f2878000 CR4:
> 0000000000350ef0
> [263341.869101] Call Trace:
> [263341.869104] do_raw_spin_lock+0x94/0xa0
> [263341.869107] _raw_spin_lock+0x63/0x80
> [263341.869111] zswap_frontswap_load+0x30/0x2f0
A foot from z3fold on CPU#0.
> [263341.869115] ? trace_hardirqs_on+0x1b/0xe0
> [263341.869120] __frontswap_load+0xc3/0x160
> [263341.869123] swap_readpage+0x25b/0x440
> [263341.869127] swapin_readahead+0x450/0x4e0
> [263341.869130] ? lock_release+0x1ef/0x410
> [263341.869134] do_swap_page+0x4a4/0x900
> [263341.869137] __handle_mm_fault+0xbd6/0x1610
> [263341.869140] ? lock_acquire+0x177/0x3a0
> [263341.869145] handle_mm_fault+0xa2/0x270
> [263341.869148] do_user_addr_fault+0x1ea/0x6b0
> [263341.869152] exc_page_fault+0x67/0x2a0
> [263341.869155] ? asm_exc_page_fault+0x8/0x30
> [263341.869158] asm_exc_page_fault+0x1e/0x30
> [263341.869161] RIP: 0033:0x55e1b76f7713
> [263341.869164] Code: 8b 9f c8 00 00 00 4d 8b b7 d0 00 00 00 4c 39 f3
> 74 5a 4c 8d 25 8e 18 3c 05 4c 8d 2d aa 59 e6 fb 0f 1f 80 00 00 00 00
> 48 8b 3b <48> 8b 07 48 89 c1 4c 29 e1 48 c1 c9 03 48 81 f9 9f 00 00 00
> 0f 87
> [263341.869166] RSP: 002b:00007ff2de693430 EFLAGS: 00010287
> [263341.869169] RAX: 0000000000000000 RBX: 00007ff2de693568 RCX:
> 00000070c00c7be0
> [263341.869171] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> 00000070bfa4d3c0
> [263341.869174] RBP: 00007ff2de693480 R08: 0000000000000000 R09:
> 00000000000000ca
> [263341.869176] R10: 00007fff78981080 R11: 00007fff78981090 R12:
> 000055e1bcab8f90
> [263341.869178] R13: 000055e1b355d0b3 R14: 00007ff2de693578 R15:
> 00007ff2de693500
> [263341.870981] watchdog: BUG: soft lockup - CPU#1 stuck for 23s!
> [steamwebhelper:3496089]
> [263341.871048] irq event stamp: 0
> [263341.871050] hardirqs last enabled at (0): [<0000000000000000>] 0x0
> [263341.871054] hardirqs last disabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.871058] softirqs last enabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.871061] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [263341.871064] CPU: 1 PID: 3496089 Comm: steamwebhelper Tainted: G
> D W L --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
> [263341.871067] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [263341.871069] RIP: 0010:native_queued_spin_lock_slowpath+0x137/0x200
> [263341.871073] Code: e0 a9 1d 00 eb cb 41 83 c0 01 c1 e6 10 41 c1 e0
> 12 44 09 c6 89 f0 c1 e8 10 66 87 42 02 89 c7 c1 e7 10 75 73 31 ff eb
> 02 f3 90 <8b> 02 66 85 c0 75 f7 41 89 c0 66 45 31 c0 44 39 c6 0f 84 9b
> 00 00
> [263341.871076] RSP: 0018:ffffae43549eb5f8 EFLAGS: 00000202
> [263341.871078] RAX: 00000000000c0101 RBX: ffff9d414bc00000 RCX:
> ffff9d4946def300
> [263341.871081] RDX: ffff9d4253053008 RSI: 0000000000080000 RDI:
> 0000000000000000
> [263341.871083] RBP: ffff9d4253053008 R08: 0000000000080000 R09:
> 0000000000000000
> [263341.871085] R10: 0000000000000000 R11: 0000000000000001 R12:
> ffff9d4253053020
> [263341.871088] R13: ffff9d414bc00010 R14: 0000000000000000 R15:
> ffffed09c02f0000
> [263341.871090] FS: 00007f725d561d40(0000) GS:ffff9d4946c00000(0000)
> knlGS:0000000000000000
> [263341.871093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [263341.871095] CR2: 0000366b51ea6fe8 CR3: 00000003865f2000 CR4:
> 0000000000350ee0
> [263341.871098] Call Trace:
> [263341.871101] do_raw_spin_lock+0x94/0xa0
> [263341.871104] _raw_spin_lock+0x63/0x80
> [263341.871107] z3fold_page_isolate+0xbd/0x1b0
The isolate path on CPU#1.
> [263341.871112] isolate_movable_page+0x94/0x180
> [263341.871115] isolate_migratepages_block+0x5db/0x1120
> [263341.871120] ? lock_release+0x1ef/0x410
> [263341.871124] compact_zone+0x5a4/0xfd0
> [263341.871129] compact_zone_order+0xaa/0xf0
> [263341.871134] try_to_compact_pages+0x111/0x3b0
> [263341.871138] __alloc_pages_direct_compact+0x79/0x210
> [263341.871142] __alloc_pages_slowpath.constprop.0+0x1d0/0xf90
> [263341.871147] ? __alloc_pages_nodemask+0x2e3/0x400
> [263341.871151] ? lock_release+0x1ef/0x410
> [263341.871154] __alloc_pages_nodemask+0x37d/0x400
> [263341.871159] ttm_pool_alloc+0x2a3/0x630 [ttm]
> [263341.871167] ttm_tt_populate+0x37/0xe0 [ttm]
> [263341.871172] ttm_bo_handle_move_mem+0x13a/0x170 [ttm]
> [263341.871179] ttm_bo_validate+0x15f/0x1b0 [ttm]
> [263341.871184] ? lock_release+0x1ef/0x410
> [263341.871189] ttm_bo_init_reserved+0x2f7/0x3e0 [ttm]
> [263341.871195] amdgpu_bo_do_create+0x1a8/0x630 [amdgpu]
> [263341.871312] ? amdgpu_bo_subtract_pin_size+0x50/0x50 [amdgpu]
> [263341.871422] amdgpu_bo_create+0x30/0x2e0 [amdgpu]
> [263341.871531] ? lock_acquire+0x177/0x3a0
> [263341.871535] ? trace_hardirqs_on+0x1b/0xe0
> [263341.871539] ? _raw_spin_unlock_irqrestore+0x37/0x40
> [263341.871543] ? lock_release+0x1ef/0x410
> [263341.871547] amdgpu_gem_create_ioctl+0x10e/0x370 [amdgpu]
> [263341.871664] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
> [263341.871774] drm_ioctl_kernel+0x89/0xe0 [drm]
> [263341.871797] drm_ioctl+0x20f/0x3c0 [drm]
> [263341.871816] ? amdgpu_gem_force_release+0x130/0x130 [amdgpu]
> [263341.871927] ? selinux_file_ioctl+0x147/0x200
> [263341.871931] ? lock_acquired+0x200/0x390
> [263341.871934] ? lock_release+0x1ef/0x410
> [263341.871937] ? trace_hardirqs_on+0x1b/0xe0
> [263341.871940] ? _raw_spin_unlock_irqrestore+0x37/0x40
> [263341.871944] amdgpu_drm_ioctl+0x49/0x80 [amdgpu]
> [263341.872053] __x64_sys_ioctl+0x82/0xb0
> [263341.872058] do_syscall_64+0x33/0x40
> [263341.872061] entry_SYSCALL_64_after_hwframe+0x44/0xae
> [263341.872065] RIP: 0033:0x7f72610b22bb
> [263341.872068] Code: ff ff ff 85 c0 79 9b 49 c7 c4 ff ff ff ff 5b 5d
> 4c 89 e0 41 5c c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 10 00 00
> 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 85 bb 0c 00 f7 d8 64 89
> 01 48
> [263341.872071] RSP: 002b:00007ffcd94a01f8 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000010
> [263341.872074] RAX: ffffffffffffffda RBX: 00007ffcd94a0250 RCX:
> 00007f72610b22bb
> [263341.872076] RDX: 00007ffcd94a0250 RSI: 00000000c0206440 RDI:
> 0000000000000016
> [263341.872078] RBP: 00000000c0206440 R08: 0000000000000009 R09:
> 00000000000000b8
> [263341.872081] R10: 00007ffcd9568080 R11: 0000000000000246 R12:
> 000008c86f1ae3c0
> [263341.872083] R13: 0000000000000016 R14: 0000000000200000 R15:
> 00000000019c6000
> [263341.872983] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kswapd0:288]
> [263341.873052] irq event stamp: 36
> [263341.873054] hardirqs last enabled at (35): [<ffffffffa8d61117>]
> _raw_spin_unlock_irqrestore+0x37/0x40
> [263341.873059] hardirqs last disabled at (36): [<ffffffffa8d5a8a9>]
> __schedule+0x6e9/0xb20
> [263341.873063] softirqs last enabled at (0): [<ffffffffa80dd962>]
> copy_process+0x902/0x1df0
> [263341.873066] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [263341.873069] CPU: 2 PID: 288 Comm: kswapd0 Tainted: G D W L
> --------- --- 5.12.0-0.rc1.162.fc35.x86_64+debug #1
> [263341.873073] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [263341.873075] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
> [263341.873079] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
> 48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e a9 48 89 08 8b 41 08 85 c0 75
> 09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
> 0d 0f
> [263341.873082] RSP: 0018:ffffae4340943898 EFLAGS: 00000246
> [263341.873085] RAX: 0000000000000000 RBX: ffff9d4253053000 RCX:
> ffff9d4946fef300
> [263341.873087] RDX: ffff9d4253053008 RSI: 00000000000c0000 RDI:
> 0000000000000001
> [263341.873090] RBP: ffff9d4253053008 R08: 00000000000c0000 R09:
> 0000000000000000
> [263341.873092] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9d4253053020
> [263341.873094] R13: 0000000000000003 R14: 0000000000000003 R15:
> ffff9d41760b2000
> [263341.873097] FS: 0000000000000000(0000) GS:ffff9d4946e00000(0000)
> knlGS:0000000000000000
> [263341.873099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [263341.873102] CR2: 0000021509b13000 CR3: 00000004130c8000 CR4:
> 0000000000350ee0
> [263341.873104] Call Trace:
> [263341.873107] do_raw_spin_lock+0x94/0xa0
> [263341.873110] _raw_spin_lock+0x63/0x80
> [263341.873114] __z3fold_alloc+0x78/0x3d0
The alloc path on CPU#2.
> [263341.873118] z3fold_zpool_malloc+0x4a5/0x7c0
> [263341.873121] ? _raw_spin_unlock+0x1f/0x30
> [263341.873125] zswap_frontswap_store+0x43e/0x890
> [263341.873130] __frontswap_store+0xc8/0x170
> [263341.873134] swap_writepage+0x39/0x70
> [263341.873137] pageout+0x125/0x540
> [263341.873142] shrink_page_list+0x131b/0x1bb0
> [263341.873147] shrink_inactive_list+0x12a/0x440
> [263341.873152] shrink_lruvec+0x4aa/0x6d0
> [263341.873158] shrink_node+0x2d1/0x700
> [263341.873163] balance_pgdat+0x2f5/0x650
> [263341.873169] kswapd+0x21d/0x4d0
> [263341.873172] ? do_wait_intr_irq+0xd0/0xd0
> [263341.873176] ? balance_pgdat+0x650/0x650
> [263341.873179] kthread+0x13a/0x150
> [263341.873183] ? __kthread_bind_mask+0x60/0x60
> [263341.873187] ret_from_fork+0x22/0x30
>
>
> It related?
Yes, it is the same race as we saw before. But after cutting the race
between poo->stale_lock and pool->lock with the patch above, the race
between the free path and isolate/putback path came up.
>
>
> Full kernel log is here: https://pastebin.com/x0KbXN9L
>
>
> --
> Best Regards,
> Mike Gavrilov.
Try the diff below in combination with the patch above
--- x/mm/z3fold.c
+++ y/mm/z3fold.c
@@ -1676,8 +1676,10 @@ static void z3fold_page_putback(struct p
pool = zhdr_to_pool(zhdr);
z3fold_page_lock(zhdr);
+ spin_lock(&pool->lock);
if (!list_empty(&zhdr->buddy))
list_del_init(&zhdr->buddy);
+ spin_unlock(&pool->lock);
INIT_LIST_HEAD(&page->lru);
if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) {
atomic64_dec(&pool->pages_nr);
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-05 14:22 ` Hillf Danton
@ 2021-03-08 15:42 ` Mikhail Gavrilov
0 siblings, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-03-08 15:42 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Fri, 5 Mar 2021 at 19:22, Hillf Danton <hdanton@sina.com> wrote:
>
> Yes, it is the same race as we saw before. But after cutting the race
> between poo->stale_lock and pool->lock with the patch above, the race
> between the free path and isolate/putback path came up.
>
> Try the diff below in combination with the patch above
>
> --- x/mm/z3fold.c
> +++ y/mm/z3fold.c
> @@ -1676,8 +1676,10 @@ static void z3fold_page_putback(struct p
> pool = zhdr_to_pool(zhdr);
>
> z3fold_page_lock(zhdr);
> + spin_lock(&pool->lock);
> if (!list_empty(&zhdr->buddy))
> list_del_init(&zhdr->buddy);
> + spin_unlock(&pool->lock);
> INIT_LIST_HEAD(&page->lru);
> if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) {
> atomic64_dec(&pool->pages_nr);
Unfortunately even with combination of two latest patches computer
hanged again after two days uptime.
[185000.747401] list_add corruption. next->prev should be prev
(ffffe0c1bea61f40), but was 0000000000000000. (next=ffff9bb90b444000).
[185000.747438] ------------[ cut here ]------------
[185000.747441] kernel BUG at lib/list_debug.c:23!
[185000.747449] invalid opcode: 0000 [#1] SMP NOPTI
[185000.747454] CPU: 22 PID: 1588003 Comm: Web Content Tainted: G
W --------- ---
5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
[185000.747458] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[185000.747462] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[185000.747469] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
fd fd
[185000.747472] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
[185000.747476] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
0000000000000000
[185000.747479] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
ffff9bbc097daae0
[185000.747482] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
ffffc0c1c61cfa58
[185000.747485] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
ffff9bb537b4f008
[185000.747488] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
ffff9bba5ac29000
[185000.747491] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
knlGS:0000000000000000
[185000.747495] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185000.747498] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
0000000000350ee0
[185000.747501] Call Trace:
[185000.747504] do_compact_page+0x28d/0xb60
[185000.747509] ? _raw_spin_unlock+0x1f/0x30
[185000.747514] ? z3fold_zpool_free+0x3a8/0x590
[185000.747518] zswap_free_entry+0x43/0x70
[185000.747523] zswap_frontswap_invalidate_page+0x8c/0x90
[185000.747527] __frontswap_invalidate_page+0x5d/0x90
[185000.747531] swap_range_free+0xcd/0xf0
[185000.747535] swapcache_free_entries+0x128/0x1a0
[185000.747539] free_swap_slot+0xbb/0xd0
[185000.747543] __swap_entry_free+0x7a/0xa0
[185000.747547] do_swap_page+0x393/0x900
[185000.747551] __handle_mm_fault+0xbd6/0x1610
[185000.747557] handle_mm_fault+0xa2/0x270
[185000.747561] do_user_addr_fault+0x1ea/0x6b0
[185000.747566] exc_page_fault+0x67/0x2a0
[185000.747570] ? asm_exc_page_fault+0x8/0x30
[185000.747574] asm_exc_page_fault+0x1e/0x30
[185000.747578] RIP: 0033:0x7f198eb8be30
[185000.747582] Code: 9d 48 81 fa 80 00 00 00 77 19 c5 fe 7f 07 c5 fe
7f 47 20 c5 fe 7f 44 17 e0 c5 fe 7f 44 17 c0 c5 f8 77 c3 48 8d 8f 80
00 00 00 <c5> fe 7f 07 48 83 e1 80 c5 fe 7f 44 17 e0 c5 fe 7f 47 20 c5
fe 7f
[185000.747585] RSP: 002b:00007ffea7e406e8 EFLAGS: 00010202
[185000.747589] RAX: 00000fe3aa523500 RBX: 00007ffea7e40738 RCX:
00000fe3aa523580
[185000.747592] RDX: 0000000000004000 RSI: 00000000000000fa RDI:
00000fe3aa523500
[185000.747594] RBP: 0000600000000000 R08: 00007f195295a000 R09:
ffffffffffffffff
[185000.747597] R10: 0000556a267402c8 R11: 0000000000000206 R12:
00007f195295a800
[185000.747600] R13: fffffc0000000000 R14: 00007ffea7e40738 R15:
00007f195295a7f0
[185000.747605] Modules linked in: crypto_user tun snd_seq_dummy
snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
mc bluetooth rapl ff_memless snd_pcm
[185000.747647] iwlwifi asus_wmi sparse_keymap video snd_timer
ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[185000.747878] ---[ end trace df51d3d2498d767d ]---
[185000.747882] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[185000.747886] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
fd fd
[185000.747889] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
[185000.747893] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
0000000000000000
[185000.747895] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
ffff9bbc097daae0
[185000.747898] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
ffffc0c1c61cfa58
[185000.747901] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
ffff9bb537b4f008
[185000.747904] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
ffff9bba5ac29000
[185000.747907] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
knlGS:0000000000000000
[185000.747910] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185000.747913] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
0000000000350ee0
[185000.747916] note: Web Content[1588003] exited with preempt_count 6
[185026.580248] watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
[Chrome_ChildIOT:1951362]
[185026.580262] Modules linked in: crypto_user tun snd_seq_dummy
snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
mc bluetooth rapl ff_memless snd_pcm
[185026.580306] iwlwifi asus_wmi sparse_keymap video snd_timer
ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[185026.580334] irq event stamp: 0
[185026.580337] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[185026.580342] hardirqs last disabled at (0): [<ffffffff830dd962>]
copy_process+0x902/0x1df0
[185026.580349] softirqs last enabled at (0): [<ffffffff830dd962>]
copy_process+0x902/0x1df0
[185026.580353] softirqs last disabled at (0): [<0000000000000000>] 0x0
[185026.580357] CPU: 0 PID: 1951362 Comm: Chrome_ChildIOT Tainted: G
D W --------- ---
5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
[185026.580362] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[185026.580365] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[185026.580370] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e 84 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[185026.580374] RSP: 0000:ffffc0c1d596fbf0 EFLAGS: 00000246
[185026.580379] RAX: 0000000000000000 RBX: 0000000000f10bea RCX:
ffff9bbc06bef300
[185026.580382] RDX: ffff9bb500b9fb48 RSI: 0000000000040000 RDI:
0000000000000013
[185026.580385] RBP: ffff9bb500b9fb48 R08: 0000000000040000 R09:
0000000000000000
[185026.580388] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9bb500b9fb60
[185026.580391] R13: ffff9bb500b9fb48 R14: ffffffff84e87020 R15:
ffff9bb500b9fb40
[185026.580394] FS: 00007f2e2ffe6640(0000) GS:ffff9bbc06a00000(0000)
knlGS:0000000000000000
[185026.580398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185026.580401] CR2: 00002a8e6ef59188 CR3: 0000000306082000 CR4:
0000000000350ef0
[185026.580405] Call Trace:
[185026.580408] do_raw_spin_lock+0x94/0xa0
[185026.580665] _raw_spin_lock+0x63/0x80
[185026.580670] zswap_frontswap_load+0x30/0x2f0
[185026.580676] ? trace_hardirqs_on+0x1b/0xe0
[185026.580681] __frontswap_load+0xc3/0x160
[185026.580685] swap_readpage+0x257/0x430
[185026.580689] swapin_readahead+0x450/0x4e0
[185026.580693] ? lock_release+0x1ef/0x410
[185026.580698] do_swap_page+0x4a4/0x900
[185026.580703] __handle_mm_fault+0xbd6/0x1610
[185026.580795] handle_mm_fault+0xa2/0x270
[185026.580799] do_user_addr_fault+0x1ea/0x6b0
[185026.580804] exc_page_fault+0x67/0x2a0
[185026.580808] ? asm_exc_page_fault+0x8/0x30
[185026.580889] asm_exc_page_fault+0x1e/0x30
[185026.580893] RIP: 0033:0x55d9d6466038
[185026.580897] Code: cc cc 55 48 89 e5 48 89 7f 10 48 89 7f 18 5d c3
cc cc 55 48 89 e5 48 8b 47 10 48 8b 4f 18 48 89 41 10 48 8b 47 10 48
8b 4f 18 <48> 89 48 18 0f 57 c0 0f 11 47 10 5d c3 cc cc cc cc cc cc cc
cc cc
[185026.580900] RSP: 002b:00007f2e2ffe46b0 EFLAGS: 00010246
[185026.580968] RAX: 00002a8e6ef59170 RBX: 00002a8e6ef40420 RCX:
00002a8e6ef4e370
[185026.580972] RDX: 00002a8e6d6715e0 RSI: 00002a8e6dcb02e0 RDI:
00002a8e6ef40420
[185026.580974] RBP: 00007f2e2ffe46b0 R08: 0000000000000000 R09:
00007fff260af5d0
[185026.581039] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000020
[185026.581042] R13: 00002a8e6dcb02e0 R14: 000055d9deb9b1e0 R15:
00002a8e6dcb02e0
Full kernel log is here: https://pastebin.com/WmBLJ3MR
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
@ 2021-03-08 15:42 ` Mikhail Gavrilov
0 siblings, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-03-08 15:42 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Fri, 5 Mar 2021 at 19:22, Hillf Danton <hdanton@sina.com> wrote:
>
> Yes, it is the same race as we saw before. But after cutting the race
> between poo->stale_lock and pool->lock with the patch above, the race
> between the free path and isolate/putback path came up.
>
> Try the diff below in combination with the patch above
>
> --- x/mm/z3fold.c
> +++ y/mm/z3fold.c
> @@ -1676,8 +1676,10 @@ static void z3fold_page_putback(struct p
> pool = zhdr_to_pool(zhdr);
>
> z3fold_page_lock(zhdr);
> + spin_lock(&pool->lock);
> if (!list_empty(&zhdr->buddy))
> list_del_init(&zhdr->buddy);
> + spin_unlock(&pool->lock);
> INIT_LIST_HEAD(&page->lru);
> if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) {
> atomic64_dec(&pool->pages_nr);
Unfortunately even with combination of two latest patches computer
hanged again after two days uptime.
[185000.747401] list_add corruption. next->prev should be prev
(ffffe0c1bea61f40), but was 0000000000000000. (next=ffff9bb90b444000).
[185000.747438] ------------[ cut here ]------------
[185000.747441] kernel BUG at lib/list_debug.c:23!
[185000.747449] invalid opcode: 0000 [#1] SMP NOPTI
[185000.747454] CPU: 22 PID: 1588003 Comm: Web Content Tainted: G
W --------- ---
5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
[185000.747458] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[185000.747462] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[185000.747469] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
fd fd
[185000.747472] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
[185000.747476] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
0000000000000000
[185000.747479] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
ffff9bbc097daae0
[185000.747482] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
ffffc0c1c61cfa58
[185000.747485] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
ffff9bb537b4f008
[185000.747488] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
ffff9bba5ac29000
[185000.747491] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
knlGS:0000000000000000
[185000.747495] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185000.747498] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
0000000000350ee0
[185000.747501] Call Trace:
[185000.747504] do_compact_page+0x28d/0xb60
[185000.747509] ? _raw_spin_unlock+0x1f/0x30
[185000.747514] ? z3fold_zpool_free+0x3a8/0x590
[185000.747518] zswap_free_entry+0x43/0x70
[185000.747523] zswap_frontswap_invalidate_page+0x8c/0x90
[185000.747527] __frontswap_invalidate_page+0x5d/0x90
[185000.747531] swap_range_free+0xcd/0xf0
[185000.747535] swapcache_free_entries+0x128/0x1a0
[185000.747539] free_swap_slot+0xbb/0xd0
[185000.747543] __swap_entry_free+0x7a/0xa0
[185000.747547] do_swap_page+0x393/0x900
[185000.747551] __handle_mm_fault+0xbd6/0x1610
[185000.747557] handle_mm_fault+0xa2/0x270
[185000.747561] do_user_addr_fault+0x1ea/0x6b0
[185000.747566] exc_page_fault+0x67/0x2a0
[185000.747570] ? asm_exc_page_fault+0x8/0x30
[185000.747574] asm_exc_page_fault+0x1e/0x30
[185000.747578] RIP: 0033:0x7f198eb8be30
[185000.747582] Code: 9d 48 81 fa 80 00 00 00 77 19 c5 fe 7f 07 c5 fe
7f 47 20 c5 fe 7f 44 17 e0 c5 fe 7f 44 17 c0 c5 f8 77 c3 48 8d 8f 80
00 00 00 <c5> fe 7f 07 48 83 e1 80 c5 fe 7f 44 17 e0 c5 fe 7f 47 20 c5
fe 7f
[185000.747585] RSP: 002b:00007ffea7e406e8 EFLAGS: 00010202
[185000.747589] RAX: 00000fe3aa523500 RBX: 00007ffea7e40738 RCX:
00000fe3aa523580
[185000.747592] RDX: 0000000000004000 RSI: 00000000000000fa RDI:
00000fe3aa523500
[185000.747594] RBP: 0000600000000000 R08: 00007f195295a000 R09:
ffffffffffffffff
[185000.747597] R10: 0000556a267402c8 R11: 0000000000000206 R12:
00007f195295a800
[185000.747600] R13: fffffc0000000000 R14: 00007ffea7e40738 R15:
00007f195295a7f0
[185000.747605] Modules linked in: crypto_user tun snd_seq_dummy
snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
mc bluetooth rapl ff_memless snd_pcm
[185000.747647] iwlwifi asus_wmi sparse_keymap video snd_timer
ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[185000.747878] ---[ end trace df51d3d2498d767d ]---
[185000.747882] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[185000.747886] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
fd fd
[185000.747889] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
[185000.747893] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
0000000000000000
[185000.747895] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
ffff9bbc097daae0
[185000.747898] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
ffffc0c1c61cfa58
[185000.747901] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
ffff9bb537b4f008
[185000.747904] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
ffff9bba5ac29000
[185000.747907] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
knlGS:0000000000000000
[185000.747910] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185000.747913] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
0000000000350ee0
[185000.747916] note: Web Content[1588003] exited with preempt_count 6
[185026.580248] watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
[Chrome_ChildIOT:1951362]
[185026.580262] Modules linked in: crypto_user tun snd_seq_dummy
snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
mc bluetooth rapl ff_memless snd_pcm
[185026.580306] iwlwifi asus_wmi sparse_keymap video snd_timer
ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
pinctrl_amd fuse
[185026.580334] irq event stamp: 0
[185026.580337] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[185026.580342] hardirqs last disabled at (0): [<ffffffff830dd962>]
copy_process+0x902/0x1df0
[185026.580349] softirqs last enabled at (0): [<ffffffff830dd962>]
copy_process+0x902/0x1df0
[185026.580353] softirqs last disabled at (0): [<0000000000000000>] 0x0
[185026.580357] CPU: 0 PID: 1951362 Comm: Chrome_ChildIOT Tainted: G
D W --------- ---
5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
[185026.580362] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[185026.580365] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
[185026.580370] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e 84 48 89 08 8b 41 08 85 c0 75
09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
0d 0f
[185026.580374] RSP: 0000:ffffc0c1d596fbf0 EFLAGS: 00000246
[185026.580379] RAX: 0000000000000000 RBX: 0000000000f10bea RCX:
ffff9bbc06bef300
[185026.580382] RDX: ffff9bb500b9fb48 RSI: 0000000000040000 RDI:
0000000000000013
[185026.580385] RBP: ffff9bb500b9fb48 R08: 0000000000040000 R09:
0000000000000000
[185026.580388] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff9bb500b9fb60
[185026.580391] R13: ffff9bb500b9fb48 R14: ffffffff84e87020 R15:
ffff9bb500b9fb40
[185026.580394] FS: 00007f2e2ffe6640(0000) GS:ffff9bbc06a00000(0000)
knlGS:0000000000000000
[185026.580398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[185026.580401] CR2: 00002a8e6ef59188 CR3: 0000000306082000 CR4:
0000000000350ef0
[185026.580405] Call Trace:
[185026.580408] do_raw_spin_lock+0x94/0xa0
[185026.580665] _raw_spin_lock+0x63/0x80
[185026.580670] zswap_frontswap_load+0x30/0x2f0
[185026.580676] ? trace_hardirqs_on+0x1b/0xe0
[185026.580681] __frontswap_load+0xc3/0x160
[185026.580685] swap_readpage+0x257/0x430
[185026.580689] swapin_readahead+0x450/0x4e0
[185026.580693] ? lock_release+0x1ef/0x410
[185026.580698] do_swap_page+0x4a4/0x900
[185026.580703] __handle_mm_fault+0xbd6/0x1610
[185026.580795] handle_mm_fault+0xa2/0x270
[185026.580799] do_user_addr_fault+0x1ea/0x6b0
[185026.580804] exc_page_fault+0x67/0x2a0
[185026.580808] ? asm_exc_page_fault+0x8/0x30
[185026.580889] asm_exc_page_fault+0x1e/0x30
[185026.580893] RIP: 0033:0x55d9d6466038
[185026.580897] Code: cc cc 55 48 89 e5 48 89 7f 10 48 89 7f 18 5d c3
cc cc 55 48 89 e5 48 8b 47 10 48 8b 4f 18 48 89 41 10 48 8b 47 10 48
8b 4f 18 <48> 89 48 18 0f 57 c0 0f 11 47 10 5d c3 cc cc cc cc cc cc cc
cc cc
[185026.580900] RSP: 002b:00007f2e2ffe46b0 EFLAGS: 00010246
[185026.580968] RAX: 00002a8e6ef59170 RBX: 00002a8e6ef40420 RCX:
00002a8e6ef4e370
[185026.580972] RDX: 00002a8e6d6715e0 RSI: 00002a8e6dcb02e0 RDI:
00002a8e6ef40420
[185026.580974] RBP: 00007f2e2ffe46b0 R08: 0000000000000000 R09:
00007fff260af5d0
[185026.581039] R10: 0000000000000000 R11: 0000000000000246 R12:
0000000000000020
[185026.581042] R13: 00002a8e6dcb02e0 R14: 000055d9deb9b1e0 R15:
00002a8e6dcb02e0
Full kernel log is here: https://pastebin.com/WmBLJ3MR
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-08 15:42 ` Mikhail Gavrilov
(?)
@ 2021-03-09 2:31 ` Hillf Danton
2021-03-15 19:18 ` Mikhail Gavrilov
2021-03-15 19:21 ` Mikhail Gavrilov
-1 siblings, 2 replies; 18+ messages in thread
From: Hillf Danton @ 2021-03-09 2:31 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Mon, 8 Mar 2021 20:42:42 +0500 Mikhail Gavrilov wrote:
> On Fri, 5 Mar 2021 at 19:22, Hillf Danton <hdanton@sina.com> wrote:
> >
> > Yes, it is the same race as we saw before. But after cutting the race
> > between poo->stale_lock and pool->lock with the patch above, the race
> > between the free path and isolate/putback path came up.
> >
> > Try the diff below in combination with the patch above
> >
> > --- x/mm/z3fold.c
> > +++ y/mm/z3fold.c
> > @@ -1676,8 +1676,10 @@ static void z3fold_page_putback(struct p
> > pool = zhdr_to_pool(zhdr);
> >
> > z3fold_page_lock(zhdr);
> > + spin_lock(&pool->lock);
> > if (!list_empty(&zhdr->buddy))
> > list_del_init(&zhdr->buddy);
> > + spin_unlock(&pool->lock);
> > INIT_LIST_HEAD(&page->lru);
> > if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) {
> > atomic64_dec(&pool->pages_nr);
>
> Unfortunately even with combination of two latest patches computer
> hanged again after two days uptime.
Thanks again for your report.
>
> [185000.747401] list_add corruption. next->prev should be prev
> (ffffe0c1bea61f40), but was 0000000000000000. (next=ffff9bb90b444000).
At the first glance, the zero pointer goes out of the box of race because
1/ the Call Trace shows it is the free path (of the supposed race victim),
2/ on the race winner side however either list_del or list_del_init
would not leave a null pointer behind - the list_add captured in this
report is under pool->lock.
> [185000.747438] ------------[ cut here ]------------
> [185000.747441] kernel BUG at lib/list_debug.c:23!
> [185000.747449] invalid opcode: 0000 [#1] SMP NOPTI
> [185000.747454] CPU: 22 PID: 1588003 Comm: Web Content Tainted: G
> W --------- ---
> 5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
> [185000.747458] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [185000.747462] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [185000.747469] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
> ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
> fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
> fd fd
> [185000.747472] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
> [185000.747476] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
> 0000000000000000
> [185000.747479] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
> ffff9bbc097daae0
> [185000.747482] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
> ffffc0c1c61cfa58
> [185000.747485] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
> ffff9bb537b4f008
> [185000.747488] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
> ffff9bba5ac29000
> [185000.747491] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
> knlGS:0000000000000000
> [185000.747495] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [185000.747498] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
> 0000000000350ee0
> [185000.747501] Call Trace:
> [185000.747504] do_compact_page+0x28d/0xb60
> [185000.747509] ? _raw_spin_unlock+0x1f/0x30
> [185000.747514] ? z3fold_zpool_free+0x3a8/0x590
> [185000.747518] zswap_free_entry+0x43/0x70
> [185000.747523] zswap_frontswap_invalidate_page+0x8c/0x90
> [185000.747527] __frontswap_invalidate_page+0x5d/0x90
> [185000.747531] swap_range_free+0xcd/0xf0
> [185000.747535] swapcache_free_entries+0x128/0x1a0
> [185000.747539] free_swap_slot+0xbb/0xd0
> [185000.747543] __swap_entry_free+0x7a/0xa0
> [185000.747547] do_swap_page+0x393/0x900
> [185000.747551] __handle_mm_fault+0xbd6/0x1610
> [185000.747557] handle_mm_fault+0xa2/0x270
> [185000.747561] do_user_addr_fault+0x1ea/0x6b0
> [185000.747566] exc_page_fault+0x67/0x2a0
> [185000.747570] ? asm_exc_page_fault+0x8/0x30
> [185000.747574] asm_exc_page_fault+0x1e/0x30
> [185000.747578] RIP: 0033:0x7f198eb8be30
> [185000.747582] Code: 9d 48 81 fa 80 00 00 00 77 19 c5 fe 7f 07 c5 fe
> 7f 47 20 c5 fe 7f 44 17 e0 c5 fe 7f 44 17 c0 c5 f8 77 c3 48 8d 8f 80
> 00 00 00 <c5> fe 7f 07 48 83 e1 80 c5 fe 7f 44 17 e0 c5 fe 7f 47 20 c5
> fe 7f
> [185000.747585] RSP: 002b:00007ffea7e406e8 EFLAGS: 00010202
> [185000.747589] RAX: 00000fe3aa523500 RBX: 00007ffea7e40738 RCX:
> 00000fe3aa523580
> [185000.747592] RDX: 0000000000004000 RSI: 00000000000000fa RDI:
> 00000fe3aa523500
> [185000.747594] RBP: 0000600000000000 R08: 00007f195295a000 R09:
> ffffffffffffffff
> [185000.747597] R10: 0000556a267402c8 R11: 0000000000000206 R12:
> 00007f195295a800
> [185000.747600] R13: fffffc0000000000 R14: 00007ffea7e40738 R15:
> 00007f195295a7f0
> [185000.747605] Modules linked in: crypto_user tun snd_seq_dummy
> snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
> nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
> nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
> zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
> snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
> snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
> snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
> uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
> edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
> videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
> kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
> snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
> mc bluetooth rapl ff_memless snd_pcm
> [185000.747647] iwlwifi asus_wmi sparse_keymap video snd_timer
> ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
> k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
> usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
> crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
> ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
> pinctrl_amd fuse
> [185000.747878] ---[ end trace df51d3d2498d767d ]---
> [185000.747882] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [185000.747886] Code: 48 c7 c6 7c a9 64 84 48 89 ef 49 c7 c7 ea ff ff
> ff e8 9d 81 01 00 e9 5f ee 9c ff 4c 89 c1 48 c7 c7 48 aa 64 84 e8 74
> fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 f8 aa 64 84 e8 5d
> fd fd
> [185000.747889] RSP: 0000:ffffc0c1c61cfc10 EFLAGS: 00010286
> [185000.747893] RAX: 0000000000000075 RBX: fffffa0d596b0a40 RCX:
> 0000000000000000
> [185000.747895] RDX: ffff9bbc097e97a0 RSI: ffff9bbc097daae0 RDI:
> ffff9bbc097daae0
> [185000.747898] RBP: ffffe0c1bea61f40 R08: 0000000000000000 R09:
> ffffc0c1c61cfa58
> [185000.747901] R10: ffffc0c1c61cfa50 R11: 0000000000000000 R12:
> ffff9bb537b4f008
> [185000.747904] R13: ffff9bba5ac29010 R14: ffff9bb90b444000 R15:
> ffff9bba5ac29000
> [185000.747907] FS: 00007f198ea257c0(0000) GS:ffff9bbc09600000(0000)
> knlGS:0000000000000000
> [185000.747910] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [185000.747913] CR2: 00000fe3aa523500 CR3: 000000012f870000 CR4:
> 0000000000350ee0
> [185000.747916] note: Web Content[1588003] exited with preempt_count 6
> [185026.580248] watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
> [Chrome_ChildIOT:1951362]
> [185026.580262] Modules linked in: crypto_user tun snd_seq_dummy
> snd_hrtimer uinput nls_utf8 isofs rfcomm netconsole nft_objref
> nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
> nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
> zstd sunrpc vfat fat hid_logitech_hidpp usblp hid_logitech_dj
> snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio
> snd_hda_codec_hdmi intel_rapl_msr mt76x2u joydev intel_rapl_common
> snd_hda_intel mt76x2_common iwlmvm snd_intel_dspcfg mt76x02_usb
> uvcvideo snd_intel_sdw_acpi snd_usb_audio mt76_usb snd_hda_codec
> edac_mce_amd videobuf2_vmalloc mt76x02_lib videobuf2_memops
> videobuf2_v4l2 snd_hda_core mt76 videobuf2_common snd_usbmidi_lib
> kvm_amd btusb snd_hwdep snd_rawmidi videodev btrtl mac80211 kvm
> snd_seq btbcm btintel snd_seq_device irqbypass libarc4 eeepc_wmi xpad
> mc bluetooth rapl ff_memless snd_pcm
> [185026.580306] iwlwifi asus_wmi sparse_keymap video snd_timer
> ecdh_generic ecc wmi_bmof pcspkr snd cfg80211 soundcore sp5100_tco
> k10temp i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables uas
> usb_storage amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
> crct10dif_pclmul drm_kms_helper crc32_pclmul crc32c_intel cec drm
> ghash_clmulni_intel ccp igb nvme dca nvme_core i2c_algo_bit wmi
> pinctrl_amd fuse
> [185026.580334] irq event stamp: 0
> [185026.580337] hardirqs last enabled at (0): [<0000000000000000>] 0x0
> [185026.580342] hardirqs last disabled at (0): [<ffffffff830dd962>]
> copy_process+0x902/0x1df0
> [185026.580349] softirqs last enabled at (0): [<ffffffff830dd962>]
> copy_process+0x902/0x1df0
> [185026.580353] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [185026.580357] CPU: 0 PID: 1951362 Comm: Chrome_ChildIOT Tainted: G
> D W --------- ---
> 5.12.0-0.rc1.20210305git280d542f6ffa.164.fc35.x86_64 #1
> [185026.580362] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [185026.580365] RIP: 0010:native_queued_spin_lock_slowpath+0x1ce/0x200
> [185026.580370] Code: c1 ef 12 83 e0 03 83 ef 01 48 c1 e0 05 48 63 ff
> 48 05 00 f3 1e 00 48 03 04 fd 20 d9 6e 84 48 89 08 8b 41 08 85 c0 75
> 09 f3 90 <8b> 41 08 85 c0 74 f7 48 8b 39 48 85 ff 0f 84 56 ff ff ff 0f
> 0d 0f
> [185026.580374] RSP: 0000:ffffc0c1d596fbf0 EFLAGS: 00000246
> [185026.580379] RAX: 0000000000000000 RBX: 0000000000f10bea RCX:
> ffff9bbc06bef300
> [185026.580382] RDX: ffff9bb500b9fb48 RSI: 0000000000040000 RDI:
> 0000000000000013
> [185026.580385] RBP: ffff9bb500b9fb48 R08: 0000000000040000 R09:
> 0000000000000000
> [185026.580388] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff9bb500b9fb60
> [185026.580391] R13: ffff9bb500b9fb48 R14: ffffffff84e87020 R15:
> ffff9bb500b9fb40
> [185026.580394] FS: 00007f2e2ffe6640(0000) GS:ffff9bbc06a00000(0000)
> knlGS:0000000000000000
> [185026.580398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [185026.580401] CR2: 00002a8e6ef59188 CR3: 0000000306082000 CR4:
> 0000000000350ef0
> [185026.580405] Call Trace:
> [185026.580408] do_raw_spin_lock+0x94/0xa0
> [185026.580665] _raw_spin_lock+0x63/0x80
> [185026.580670] zswap_frontswap_load+0x30/0x2f0
> [185026.580676] ? trace_hardirqs_on+0x1b/0xe0
> [185026.580681] __frontswap_load+0xc3/0x160
> [185026.580685] swap_readpage+0x257/0x430
> [185026.580689] swapin_readahead+0x450/0x4e0
> [185026.580693] ? lock_release+0x1ef/0x410
> [185026.580698] do_swap_page+0x4a4/0x900
> [185026.580703] __handle_mm_fault+0xbd6/0x1610
> [185026.580795] handle_mm_fault+0xa2/0x270
> [185026.580799] do_user_addr_fault+0x1ea/0x6b0
> [185026.580804] exc_page_fault+0x67/0x2a0
> [185026.580808] ? asm_exc_page_fault+0x8/0x30
> [185026.580889] asm_exc_page_fault+0x1e/0x30
> [185026.580893] RIP: 0033:0x55d9d6466038
> [185026.580897] Code: cc cc 55 48 89 e5 48 89 7f 10 48 89 7f 18 5d c3
> cc cc 55 48 89 e5 48 8b 47 10 48 8b 4f 18 48 89 41 10 48 8b 47 10 48
> 8b 4f 18 <48> 89 48 18 0f 57 c0 0f 11 47 10 5d c3 cc cc cc cc cc cc cc
> cc cc
> [185026.580900] RSP: 002b:00007f2e2ffe46b0 EFLAGS: 00010246
> [185026.580968] RAX: 00002a8e6ef59170 RBX: 00002a8e6ef40420 RCX:
> 00002a8e6ef4e370
> [185026.580972] RDX: 00002a8e6d6715e0 RSI: 00002a8e6dcb02e0 RDI:
> 00002a8e6ef40420
> [185026.580974] RBP: 00007f2e2ffe46b0 R08: 0000000000000000 R09:
> 00007fff260af5d0
> [185026.581039] R10: 0000000000000000 R11: 0000000000000246 R12:
> 0000000000000020
> [185026.581042] R13: 00002a8e6dcb02e0 R14: 000055d9deb9b1e0 R15:
> 00002a8e6dcb02e0
>
> Full kernel log is here: https://pastebin.com/WmBLJ3MR
>
> --
> Best Regards,
> Mike Gavrilov.
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-09 2:31 ` Hillf Danton
@ 2021-03-15 19:18 ` Mikhail Gavrilov
2021-03-15 19:21 ` Mikhail Gavrilov
1 sibling, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-03-15 19:18 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
[-- Attachment #1: Type: text/plain, Size: 18292 bytes --]
On Tue, 9 Mar 2021 at 07:31, Hillf Danton <hdanton@sina.com> wrote:
>
> At the first glance, the zero pointer goes out of the box of race because
>
> 1/ the Call Trace shows it is the free path (of the supposed race victim),
>
> 2/ on the race winner side however either list_del or list_del_init
> would not leave a null pointer behind - the list_add captured in this
> report is under pool->lock.
>
No more ideas how to fix it?
Kernel panics continue happens again and again with you patches and recent
commits.
[102491.134247] ------------[ cut here ]------------
[102491.134248] list_add corruption. next->prev should be prev
(ffffcc447ea60c78), but was ffffffff8a64ec20. (next=ffff8adc731d3f40).
[102491.134266] ODEBUG: free active (active state 0) object type:
work_struct hint: compact_page_work+0x0/0x10
[102491.134294] ------------[ cut here ]------------
[102491.134295] kernel BUG at lib/list_debug.c:23!
[102491.134299] invalid opcode: 0000 [#1] SMP NOPTI
[102491.134301] CPU: 22 PID: 863413 Comm: kworker/u64:0 Tainted: G
W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.134303] Hardware name: System manufacturer System Product Name/ROG
STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.134305] Workqueue: zswap3 compact_page_work
[102491.134309] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134312] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff ff e8
91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67 fd fd ff
<0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50 fd fd
[102491.134313] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134315] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134316] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.134316] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.134317] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.134318] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.134319] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.134320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.134320] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.134321] Call Trace:
[102491.134324] do_compact_page+0x28d/0xb60
[102491.134326] ? debug_object_deactivate+0x55/0x140
[102491.134329] ? lock_release+0x1ef/0x410
[102491.134331] ? lock_release+0x1ef/0x410
[102491.134333] process_one_work+0x2b0/0x5e0
[102491.134337] worker_thread+0x55/0x3c0
[102491.134339] ? process_one_work+0x5e0/0x5e0
[102491.134340] kthread+0x13a/0x150
[102491.134342] ? __kthread_bind_mask+0x60/0x60
[102491.134345] ret_from_fork+0x22/0x30
[102491.134349] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8 isofs
uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib
[102491.134375] WARNING: CPU: 18 PID: 182 at lib/debugobjects.c:505
debug_print_object+0x6e/0x90
[102491.134380] nft_reject_inet nf_reject_ipv4
[102491.134383] Modules linked in:
[102491.134385] nf_reject_ipv6 nft_reject
[102491.134388] snd_seq_dummy
[102491.134390] nft_ct
[102491.134393] snd_hrtimer
[102491.134395] nft_chain_nat nf_nat
[102491.134398] nls_utf8
[102491.134400] nf_conntrack nf_defrag_ipv6
[102491.134403] isofs
[102491.134405] nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd
sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi snd_intel_dspcfg mc
joydev snd_intel_sdw_acpi mac80211 kvm snd_hda_codec snd_hda_core iwlwifi
btusb snd_hwdep btrtl snd_seq btbcm btintel eeepc_wmi asus_wmi
snd_seq_device irqbypass xpad sparse_keymap bluetooth libarc4 rapl snd_pcm
video ff_memless wmi_bmof ecdh_generic ecc cfg80211 pcspkr snd_timer
k10temp snd sp5100_tco i2c_piix4 soundcore rfkill acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper
crct10dif_pclmul crc32_pclmul crc32c_intel cec igb drm nvme
ghash_clmulni_intel dca ccp
[102491.134476] nvme_core i2c_algo_bit wmi pinctrl_amd fuse
[102491.134484] ---[ end trace 562b0b01453e6613 ]---
[102491.134505] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134509] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff ff e8
91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67 fd fd ff
<0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50 fd fd
[102491.134511] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134514] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134992] uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat
[102491.135036] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.135039] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.135040] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.135041] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.135042] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.135047] nf_conntrack nf_defrag_ipv6
[102491.135051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135054] nf_defrag_ipv4
[102491.135056] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135059] ip_set
[102491.135061] note: kworker/u64:0[863413] exited with preempt_count 2
[102491.135064] nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi snd_intel_dspcfg mc
joydev snd_intel_sdw_acpi mac80211 kvm snd_hda_codec snd_hda_core iwlwifi
btusb snd_hwdep btrtl snd_seq btbcm btintel eeepc_wmi asus_wmi
snd_seq_device irqbypass xpad sparse_keymap bluetooth libarc4 rapl snd_pcm
video ff_memless wmi_bmof ecdh_generic ecc cfg80211 pcspkr snd_timer
k10temp snd sp5100_tco i2c_piix4 soundcore rfkill acpi_cpufreq binfmt_misc
ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper
crct10dif_pclmul crc32_pclmul crc32c_intel cec igb drm nvme
ghash_clmulni_intel dca ccp nvme_core i2c_algo_bit wmi
[102491.135357] pinctrl_amd fuse
[102491.135366] CPU: 18 PID: 182 Comm: kcompactd0 Tainted: G D W
--------- --- 5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.135369] Hardware name: System manufacturer System Product Name/ROG
STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.135372] RIP: 0010:debug_print_object+0x6e/0x90
[102491.135403] Code: 49 89 c1 8b 43 10 83 c2 01 8b 4b 14 48 c7 c7 e8 ab 64
8b 89 15 a7 0b 37 03 4c 8b 45 00 48 8b 14 c5 a0 80 2a 8b e8 50 4d 60 00
<0f> 0b 83 05 25 0e 99 01 01 48 83 c4 08 5b 5d c3 83 05 17 0e 99 01
[102491.135406] RSP: 0018:ffffac448080bb78 EFLAGS: 00010296
[102491.135409] RAX: 000000000000005e RBX: ffff8add6de5c7a8 RCX:
0000000000000027
[102491.135412] RDX: ffff8ae348fdaae8 RSI: 0000000000000001 RDI:
ffff8ae348fdaae0
[102491.135415] RBP: ffffffff8b221320 R08: 0000000000000000 R09:
0000000000000000
[102491.135417] R10: 0000000000000000 R11: 0000000000000000 R12:
dead000000000122
[102491.135419] R13: dead000000000100 R14: ffffffff8b221320 R15:
0000000000000005
[102491.135421] FS: 0000000000000000(0000) GS:ffff8ae348e00000(0000)
knlGS:0000000000000000
[102491.135441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135443] CR2: 00007f41b9ee7000 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135446] Call Trace:
[102491.135451] debug_check_no_obj_freed+0x1db/0x220
[102491.135455] free_pcp_prepare+0x132/0x270
[102491.135459] free_unref_page+0x18/0xd0
[102491.135463] migrate_pages+0x8b9/0x1200
[102491.135467] ? isolate_freepages_block+0x4a0/0x4a0
[102491.135471] ? split_map_pages+0x160/0x160
[102491.135490] compact_zone+0x680/0xfd0
[102491.135493] ? __free_object+0x2b9/0x300
[102491.135496] ? lock_release+0x1ef/0x410
[102491.135500] proactive_compact_node+0x78/0xb0
[102491.135505] kcompactd+0x38a/0x440
[102491.135509] ? do_wait_intr_irq+0xd0/0xd0
[102491.135512] ? kcompactd_do_work+0x3a0/0x3a0
[102491.135515] kthread+0x13a/0x150
[102491.135520] ? __kthread_bind_mask+0x60/0x60
[102491.135533] ret_from_fork+0x22/0x30
[102491.135539] irq event stamp: 220
[102491.135541] hardirqs last enabled at (219): [<ffffffff8ad62217>]
_raw_spin_unlock_irqrestore+0x37/0x40
[102491.135545] hardirqs last disabled at (220): [<ffffffff8ad5b9a9>]
__schedule+0x6e9/0xb20
[102491.135548] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102491.135552] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102491.135555] ---[ end trace 562b0b01453e6614 ]---
[102494.954915] iwlwifi 0000:04:00.0: Error sending SCAN_REQ_UMAC: time out
after 2000ms.
[102494.954950] iwlwifi 0000:04:00.0: Current CMD queue read_ptr 93
write_ptr 94
[102494.956242] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956245] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 6
[102494.956248] iwlwifi 0000:04:00.0: Loaded firmware version:
59.601f3a66.0 cc-a0-59.ucode
[102494.956251] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
[102494.956255] iwlwifi 0000:04:00.0: 0x00A022F0 | trm_hw_status0
[102494.956257] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
[102494.956260] iwlwifi 0000:04:00.0: 0x004FAA36 | branchlink2
[102494.956262] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink1
[102494.956265] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink2
[102494.956268] iwlwifi 0000:04:00.0: 0x004F51B0 | data1
[102494.956270] iwlwifi 0000:04:00.0: 0x01000000 | data2
[102494.956272] iwlwifi 0000:04:00.0: 0x00000000 | data3
[102494.956275] iwlwifi 0000:04:00.0: 0x00000000 | beacon time
[102494.956277] iwlwifi 0000:04:00.0: 0xD78321C6 | tsf low
[102494.956279] iwlwifi 0000:04:00.0: 0x00000017 | tsf hi
[102494.956282] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
[102494.956284] iwlwifi 0000:04:00.0: 0xD783784B | time gp2
[102494.956286] iwlwifi 0000:04:00.0: 0x00000001 | uCode revision type
[102494.956289] iwlwifi 0000:04:00.0: 0x0000003B | uCode version major
[102494.956291] iwlwifi 0000:04:00.0: 0x601F3A66 | uCode version minor
[102494.956294] iwlwifi 0000:04:00.0: 0x00000340 | hw version
[102494.956296] iwlwifi 0000:04:00.0: 0x00C89000 | board version
[102494.956299] iwlwifi 0000:04:00.0: 0x807DFD04 | hcmd
[102494.956302] iwlwifi 0000:04:00.0: 0x00020000 | isr0
[102494.956304] iwlwifi 0000:04:00.0: 0x01000000 | isr1
[102494.956306] iwlwifi 0000:04:00.0: 0x08F04002 | isr2
[102494.956309] iwlwifi 0000:04:00.0: 0x04C3000C | isr3
[102494.956312] iwlwifi 0000:04:00.0: 0x00000000 | isr4
[102494.956315] iwlwifi 0000:04:00.0: 0x005C019C | last cmd Id
[102494.956318] iwlwifi 0000:04:00.0: 0x004F51B0 | wait_event
[102494.956320] iwlwifi 0000:04:00.0: 0x00004B99 | l2p_control
[102494.956322] iwlwifi 0000:04:00.0: 0x00000000 | l2p_duration
[102494.956325] iwlwifi 0000:04:00.0: 0x00000003 | l2p_mhvalid
[102494.956327] iwlwifi 0000:04:00.0: 0x00000000 | l2p_addr_match
[102494.956329] iwlwifi 0000:04:00.0: 0x0000000B | lmpm_pmg_sel
[102494.956332] iwlwifi 0000:04:00.0: 0x00000000 | timestamp
[102494.956334] iwlwifi 0000:04:00.0: 0x000080EC | flow_handler
[102494.956380] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956382] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 7
[102494.956385] iwlwifi 0000:04:00.0: 0x20000066 | NMI_INTERRUPT_HOST
[102494.956387] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
[102494.956390] iwlwifi 0000:04:00.0: 0x804568FC | umac branchlink2
[102494.956392] iwlwifi 0000:04:00.0: 0xC0084F3C | umac interruptlink1
[102494.956395] iwlwifi 0000:04:00.0: 0x80477750 | umac interruptlink2
[102494.956397] iwlwifi 0000:04:00.0: 0x01000000 | umac data1
[102494.956399] iwlwifi 0000:04:00.0: 0x80477750 | umac data2
[102494.956401] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
[102494.956404] iwlwifi 0000:04:00.0: 0x0000003B | umac major
[102494.956406] iwlwifi 0000:04:00.0: 0x601F3A66 | umac minor
[102494.956408] iwlwifi 0000:04:00.0: 0xD7837848 | frame pointer
[102494.956411] iwlwifi 0000:04:00.0: 0xC0885F30 | stack pointer
[102494.956413] iwlwifi 0000:04:00.0: 0x005D010D | last host cmd
[102494.956415] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
[102494.956430] iwlwifi 0000:04:00.0: IML/ROM dump:
[102494.956432] iwlwifi 0000:04:00.0: 0x00000003 | IML/ROM error/state
[102494.956446] iwlwifi 0000:04:00.0: 0x00005590 | IML/ROM data1
[102494.956460] iwlwifi 0000:04:00.0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
[102494.956470] iwlwifi 0000:04:00.0: Fseq Registers:
[102494.956475] iwlwifi 0000:04:00.0: 0x60000000 | FSEQ_ERROR_CODE
[102494.956480] iwlwifi 0000:04:00.0: 0x80290021 | FSEQ_TOP_INIT_VERSION
[102494.956486] iwlwifi 0000:04:00.0: 0x00050008 | FSEQ_CNVIO_INIT_VERSION
[102494.956491] iwlwifi 0000:04:00.0: 0x0000A503 | FSEQ_OTP_VERSION
[102494.956496] iwlwifi 0000:04:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
[102494.956502] iwlwifi 0000:04:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
[102494.956507] iwlwifi 0000:04:00.0: 0x00100530 | FSEQ_CNVI_ID
[102494.956512] iwlwifi 0000:04:00.0: 0x00000532 | FSEQ_CNVR_ID
[102494.956518] iwlwifi 0000:04:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
[102494.956525] iwlwifi 0000:04:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
[102494.956532] iwlwifi 0000:04:00.0: 0x05B0905B |
CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[102494.956540] iwlwifi 0000:04:00.0: 0x0000025B |
CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[102494.956762] iwlwifi 0000:04:00.0: WRT: Collecting data: ini trigger 4
fired.
[102494.956789] ieee80211 phy0: Hardware restart was requested
[102494.956816] iwlwifi 0000:04:00.0: Scan failed! ret -110
[102494.956925] ------------[ cut here ]------------
[102494.956928] WARNING: CPU: 30 PID: 930660 at net/mac80211/scan.c:411
__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
[102494.956962] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8 isofs
uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6
nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi snd_intel_dspcfg mc
joydev snd_intel_sdw_acpi mac80211 kvm snd_hda_codec snd_hda_core iwlwifi
btusb snd_hwdep btrtl snd_seq btbcm btintel eeepc_wmi asus_wmi
snd_seq_device irqbypass xpad sparse_keymap bluetooth libarc4
[102494.957007] rapl snd_pcm video ff_memless wmi_bmof ecdh_generic ecc
cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4 soundcore rfkill
acpi_cpufreq binfmt_misc ip_tables amdgpu drm_ttm_helper ttm iommu_v2
gpu_sched drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel cec igb
drm nvme ghash_clmulni_intel dca ccp nvme_core i2c_algo_bit wmi pinctrl_amd
fuse
[102494.957036] CPU: 30 PID: 930660 Comm: kworker/u64:2 Tainted: G D
W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102494.957039] Hardware name: System manufacturer System Product Name/ROG
STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102494.957042] Workqueue: phy0 ieee80211_scan_work [mac80211]
[102494.957073] RIP: 0010:__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
[102494.957826] Code: ca 0f 82 7d 01 00 00 48 89 ef e8 80 2f 00 00 e9 72 fe
ff ff 0f 0b 48 83 bd d8 1c 00 00 00 41 be 01 00 00 00 0f 85 9e fd ff ff
<0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 8b 85 a0 1c 00 00 e9 69
[102494.957830] RSP: 0018:ffffac4495033db0 EFLAGS: 00010246
[102494.957834] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000000
[102494.957836] RDX: 0000000000000000 RSI: 0000000000000001 RDI:
ffff8adc770f8e00
[102494.957839] RBP: ffff8adc770f8e00 R08: 0000000000000001 R09:
ffffffffc1395e40
[102494.957842] R10: ffffac4495033de8 R11: 0000000000000000 R12:
0000000000000001
[102494.957844] R13: 0000000000000000 R14: 0000000000000001 R15:
ffff8adc770f8e00
[102494.957847] FS: 0000000000000000(0000) GS:ffff8ae34a600000(0000)
knlGS:0000000000000000
[102494.957850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102494.957853] CR2: 00003de20ab47000 CR3: 00000001346e0000 CR4:
0000000000350ee0
[102494.957856] Call Trace:
[102494.957862] ieee80211_scan_work+0x15c/0x860 [mac80211]
[102494.957893] ? debug_object_deactivate+0x55/0x140
[102494.957899] ? lock_release+0x1ef/0x410
[102494.957913] ? lock_release+0x1ef/0x410
[102494.957917] process_one_work+0x2b0/0x5e0
[102494.957923] worker_thread+0x55/0x3c0
[102494.957926] ? process_one_work+0x5e0/0x5e0
[102494.957930] kthread+0x13a/0x150
[102494.957934] ? __kthread_bind_mask+0x60/0x60
[102494.957939] ret_from_fork+0x22/0x30
[102494.957945] irq event stamp: 0
[102494.957948] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[102494.957951] hardirqs last disabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957956] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957959] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102494.957962] ---[ end trace 562b0b01453e6615 ]---
Full kernel log is here: https://pastebin.com/A7dwr8ZV
--
Best Regards,
Mike Gavrilov.
[-- Attachment #2: Type: text/html, Size: 34632 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-09 2:31 ` Hillf Danton
@ 2021-03-15 19:21 ` Mikhail Gavrilov
2021-03-15 19:21 ` Mikhail Gavrilov
1 sibling, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-03-15 19:21 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Tue, 9 Mar 2021 at 07:31, Hillf Danton <hdanton@sina.com> wrote:
> At the first glance, the zero pointer goes out of the box of race because
>
> 1/ the Call Trace shows it is the free path (of the supposed race victim),
>
> 2/ on the race winner side however either list_del or list_del_init
> would not leave a null pointer behind - the list_add captured in this
> report is under pool->lock.
No more ideas how to fix it?
Kernel panics continue happens again and again with you patches and
recent commits.
[102491.134247] ------------[ cut here ]------------
[102491.134248] list_add corruption. next->prev should be prev
(ffffcc447ea60c78), but was ffffffff8a64ec20. (next=ffff8adc731d3f40).
[102491.134266] ODEBUG: free active (active state 0) object type:
work_struct hint: compact_page_work+0x0/0x10
[102491.134294] ------------[ cut here ]------------
[102491.134295] kernel BUG at lib/list_debug.c:23!
[102491.134299] invalid opcode: 0000 [#1] SMP NOPTI
[102491.134301] CPU: 22 PID: 863413 Comm: kworker/u64:0 Tainted: G
W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.134303] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.134305] Workqueue: zswap3 compact_page_work
[102491.134309] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134312] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
fd fd
[102491.134313] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134315] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134316] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.134316] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.134317] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.134318] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.134319] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.134320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.134320] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.134321] Call Trace:
[102491.134324] do_compact_page+0x28d/0xb60
[102491.134326] ? debug_object_deactivate+0x55/0x140
[102491.134329] ? lock_release+0x1ef/0x410
[102491.134331] ? lock_release+0x1ef/0x410
[102491.134333] process_one_work+0x2b0/0x5e0
[102491.134337] worker_thread+0x55/0x3c0
[102491.134339] ? process_one_work+0x5e0/0x5e0
[102491.134340] kthread+0x13a/0x150
[102491.134342] ? __kthread_bind_mask+0x60/0x60
[102491.134345] ret_from_fork+0x22/0x30
[102491.134349] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib
[102491.134375] WARNING: CPU: 18 PID: 182 at lib/debugobjects.c:505
debug_print_object+0x6e/0x90
[102491.134380] nft_reject_inet nf_reject_ipv4
[102491.134383] Modules linked in:
[102491.134385] nf_reject_ipv6 nft_reject
[102491.134388] snd_seq_dummy
[102491.134390] nft_ct
[102491.134393] snd_hrtimer
[102491.134395] nft_chain_nat nf_nat
[102491.134398] nls_utf8
[102491.134400] nf_conntrack nf_defrag_ipv6
[102491.134403] isofs
[102491.134405] nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
[102491.134476] nvme_core i2c_algo_bit wmi pinctrl_amd fuse
[102491.134484] ---[ end trace 562b0b01453e6613 ]---
[102491.134505] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134509] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
fd fd
[102491.134511] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134514] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134992] uas usb_storage tun uinput rfcomm netconsole
nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat
[102491.135036] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.135039] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.135040] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.135041] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.135042] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.135047] nf_conntrack nf_defrag_ipv6
[102491.135051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135054] nf_defrag_ipv4
[102491.135056] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135059] ip_set
[102491.135061] note: kworker/u64:0[863413] exited with preempt_count 2
[102491.135064] nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
nvme_core i2c_algo_bit wmi
[102491.135357] pinctrl_amd fuse
[102491.135366] CPU: 18 PID: 182 Comm: kcompactd0 Tainted: G D W
--------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.135369] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.135372] RIP: 0010:debug_print_object+0x6e/0x90
[102491.135403] Code: 49 89 c1 8b 43 10 83 c2 01 8b 4b 14 48 c7 c7 e8
ab 64 8b 89 15 a7 0b 37 03 4c 8b 45 00 48 8b 14 c5 a0 80 2a 8b e8 50
4d 60 00 <0f> 0b 83 05 25 0e 99 01 01 48 83 c4 08 5b 5d c3 83 05 17 0e
99 01
[102491.135406] RSP: 0018:ffffac448080bb78 EFLAGS: 00010296
[102491.135409] RAX: 000000000000005e RBX: ffff8add6de5c7a8 RCX:
0000000000000027
[102491.135412] RDX: ffff8ae348fdaae8 RSI: 0000000000000001 RDI:
ffff8ae348fdaae0
[102491.135415] RBP: ffffffff8b221320 R08: 0000000000000000 R09:
0000000000000000
[102491.135417] R10: 0000000000000000 R11: 0000000000000000 R12:
dead000000000122
[102491.135419] R13: dead000000000100 R14: ffffffff8b221320 R15:
0000000000000005
[102491.135421] FS: 0000000000000000(0000) GS:ffff8ae348e00000(0000)
knlGS:0000000000000000
[102491.135441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135443] CR2: 00007f41b9ee7000 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135446] Call Trace:
[102491.135451] debug_check_no_obj_freed+0x1db/0x220
[102491.135455] free_pcp_prepare+0x132/0x270
[102491.135459] free_unref_page+0x18/0xd0
[102491.135463] migrate_pages+0x8b9/0x1200
[102491.135467] ? isolate_freepages_block+0x4a0/0x4a0
[102491.135471] ? split_map_pages+0x160/0x160
[102491.135490] compact_zone+0x680/0xfd0
[102491.135493] ? __free_object+0x2b9/0x300
[102491.135496] ? lock_release+0x1ef/0x410
[102491.135500] proactive_compact_node+0x78/0xb0
[102491.135505] kcompactd+0x38a/0x440
[102491.135509] ? do_wait_intr_irq+0xd0/0xd0
[102491.135512] ? kcompactd_do_work+0x3a0/0x3a0
[102491.135515] kthread+0x13a/0x150
[102491.135520] ? __kthread_bind_mask+0x60/0x60
[102491.135533] ret_from_fork+0x22/0x30
[102491.135539] irq event stamp: 220
[102491.135541] hardirqs last enabled at (219): [<ffffffff8ad62217>]
_raw_spin_unlock_irqrestore+0x37/0x40
[102491.135545] hardirqs last disabled at (220): [<ffffffff8ad5b9a9>]
__schedule+0x6e9/0xb20
[102491.135548] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102491.135552] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102491.135555] ---[ end trace 562b0b01453e6614 ]---
[102494.954915] iwlwifi 0000:04:00.0: Error sending SCAN_REQ_UMAC:
time out after 2000ms.
[102494.954950] iwlwifi 0000:04:00.0: Current CMD queue read_ptr 93 write_ptr 94
[102494.956242] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956245] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 6
[102494.956248] iwlwifi 0000:04:00.0: Loaded firmware version:
59.601f3a66.0 cc-a0-59.ucode
[102494.956251] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
[102494.956255] iwlwifi 0000:04:00.0: 0x00A022F0 | trm_hw_status0
[102494.956257] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
[102494.956260] iwlwifi 0000:04:00.0: 0x004FAA36 | branchlink2
[102494.956262] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink1
[102494.956265] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink2
[102494.956268] iwlwifi 0000:04:00.0: 0x004F51B0 | data1
[102494.956270] iwlwifi 0000:04:00.0: 0x01000000 | data2
[102494.956272] iwlwifi 0000:04:00.0: 0x00000000 | data3
[102494.956275] iwlwifi 0000:04:00.0: 0x00000000 | beacon time
[102494.956277] iwlwifi 0000:04:00.0: 0xD78321C6 | tsf low
[102494.956279] iwlwifi 0000:04:00.0: 0x00000017 | tsf hi
[102494.956282] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
[102494.956284] iwlwifi 0000:04:00.0: 0xD783784B | time gp2
[102494.956286] iwlwifi 0000:04:00.0: 0x00000001 | uCode revision type
[102494.956289] iwlwifi 0000:04:00.0: 0x0000003B | uCode version major
[102494.956291] iwlwifi 0000:04:00.0: 0x601F3A66 | uCode version minor
[102494.956294] iwlwifi 0000:04:00.0: 0x00000340 | hw version
[102494.956296] iwlwifi 0000:04:00.0: 0x00C89000 | board version
[102494.956299] iwlwifi 0000:04:00.0: 0x807DFD04 | hcmd
[102494.956302] iwlwifi 0000:04:00.0: 0x00020000 | isr0
[102494.956304] iwlwifi 0000:04:00.0: 0x01000000 | isr1
[102494.956306] iwlwifi 0000:04:00.0: 0x08F04002 | isr2
[102494.956309] iwlwifi 0000:04:00.0: 0x04C3000C | isr3
[102494.956312] iwlwifi 0000:04:00.0: 0x00000000 | isr4
[102494.956315] iwlwifi 0000:04:00.0: 0x005C019C | last cmd Id
[102494.956318] iwlwifi 0000:04:00.0: 0x004F51B0 | wait_event
[102494.956320] iwlwifi 0000:04:00.0: 0x00004B99 | l2p_control
[102494.956322] iwlwifi 0000:04:00.0: 0x00000000 | l2p_duration
[102494.956325] iwlwifi 0000:04:00.0: 0x00000003 | l2p_mhvalid
[102494.956327] iwlwifi 0000:04:00.0: 0x00000000 | l2p_addr_match
[102494.956329] iwlwifi 0000:04:00.0: 0x0000000B | lmpm_pmg_sel
[102494.956332] iwlwifi 0000:04:00.0: 0x00000000 | timestamp
[102494.956334] iwlwifi 0000:04:00.0: 0x000080EC | flow_handler
[102494.956380] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956382] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 7
[102494.956385] iwlwifi 0000:04:00.0: 0x20000066 | NMI_INTERRUPT_HOST
[102494.956387] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
[102494.956390] iwlwifi 0000:04:00.0: 0x804568FC | umac branchlink2
[102494.956392] iwlwifi 0000:04:00.0: 0xC0084F3C | umac interruptlink1
[102494.956395] iwlwifi 0000:04:00.0: 0x80477750 | umac interruptlink2
[102494.956397] iwlwifi 0000:04:00.0: 0x01000000 | umac data1
[102494.956399] iwlwifi 0000:04:00.0: 0x80477750 | umac data2
[102494.956401] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
[102494.956404] iwlwifi 0000:04:00.0: 0x0000003B | umac major
[102494.956406] iwlwifi 0000:04:00.0: 0x601F3A66 | umac minor
[102494.956408] iwlwifi 0000:04:00.0: 0xD7837848 | frame pointer
[102494.956411] iwlwifi 0000:04:00.0: 0xC0885F30 | stack pointer
[102494.956413] iwlwifi 0000:04:00.0: 0x005D010D | last host cmd
[102494.956415] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
[102494.956430] iwlwifi 0000:04:00.0: IML/ROM dump:
[102494.956432] iwlwifi 0000:04:00.0: 0x00000003 | IML/ROM error/state
[102494.956446] iwlwifi 0000:04:00.0: 0x00005590 | IML/ROM data1
[102494.956460] iwlwifi 0000:04:00.0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
[102494.956470] iwlwifi 0000:04:00.0: Fseq Registers:
[102494.956475] iwlwifi 0000:04:00.0: 0x60000000 | FSEQ_ERROR_CODE
[102494.956480] iwlwifi 0000:04:00.0: 0x80290021 | FSEQ_TOP_INIT_VERSION
[102494.956486] iwlwifi 0000:04:00.0: 0x00050008 | FSEQ_CNVIO_INIT_VERSION
[102494.956491] iwlwifi 0000:04:00.0: 0x0000A503 | FSEQ_OTP_VERSION
[102494.956496] iwlwifi 0000:04:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
[102494.956502] iwlwifi 0000:04:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
[102494.956507] iwlwifi 0000:04:00.0: 0x00100530 | FSEQ_CNVI_ID
[102494.956512] iwlwifi 0000:04:00.0: 0x00000532 | FSEQ_CNVR_ID
[102494.956518] iwlwifi 0000:04:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
[102494.956525] iwlwifi 0000:04:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
[102494.956532] iwlwifi 0000:04:00.0: 0x05B0905B |
CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[102494.956540] iwlwifi 0000:04:00.0: 0x0000025B |
CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[102494.956762] iwlwifi 0000:04:00.0: WRT: Collecting data: ini trigger 4 fired.
[102494.956789] ieee80211 phy0: Hardware restart was requested
[102494.956816] iwlwifi 0000:04:00.0: Scan failed! ret -110
[102494.956925] ------------[ cut here ]------------
[102494.956928] WARNING: CPU: 30 PID: 930660 at
net/mac80211/scan.c:411 __ieee80211_scan_completed+0x2bb/0x520
[mac80211]
[102494.956962] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4
[102494.957007] rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
nvme_core i2c_algo_bit wmi pinctrl_amd fuse
[102494.957036] CPU: 30 PID: 930660 Comm: kworker/u64:2 Tainted: G
D W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102494.957039] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102494.957042] Workqueue: phy0 ieee80211_scan_work [mac80211]
[102494.957073] RIP: 0010:__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
[102494.957826] Code: ca 0f 82 7d 01 00 00 48 89 ef e8 80 2f 00 00 e9
72 fe ff ff 0f 0b 48 83 bd d8 1c 00 00 00 41 be 01 00 00 00 0f 85 9e
fd ff ff <0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 8b 85 a0 1c 00 00
e9 69
[102494.957830] RSP: 0018:ffffac4495033db0 EFLAGS: 00010246
[102494.957834] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000000
[102494.957836] RDX: 0000000000000000 RSI: 0000000000000001 RDI:
ffff8adc770f8e00
[102494.957839] RBP: ffff8adc770f8e00 R08: 0000000000000001 R09:
ffffffffc1395e40
[102494.957842] R10: ffffac4495033de8 R11: 0000000000000000 R12:
0000000000000001
[102494.957844] R13: 0000000000000000 R14: 0000000000000001 R15:
ffff8adc770f8e00
[102494.957847] FS: 0000000000000000(0000) GS:ffff8ae34a600000(0000)
knlGS:0000000000000000
[102494.957850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102494.957853] CR2: 00003de20ab47000 CR3: 00000001346e0000 CR4:
0000000000350ee0
[102494.957856] Call Trace:
[102494.957862] ieee80211_scan_work+0x15c/0x860 [mac80211]
[102494.957893] ? debug_object_deactivate+0x55/0x140
[102494.957899] ? lock_release+0x1ef/0x410
[102494.957913] ? lock_release+0x1ef/0x410
[102494.957917] process_one_work+0x2b0/0x5e0
[102494.957923] worker_thread+0x55/0x3c0
[102494.957926] ? process_one_work+0x5e0/0x5e0
[102494.957930] kthread+0x13a/0x150
[102494.957934] ? __kthread_bind_mask+0x60/0x60
[102494.957939] ret_from_fork+0x22/0x30
[102494.957945] irq event stamp: 0
[102494.957948] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[102494.957951] hardirqs last disabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957956] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957959] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102494.957962] ---[ end trace 562b0b01453e6615 ]---
Full kernel log is here: https://pastebin.com/A7dwr8ZV
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
@ 2021-03-15 19:21 ` Mikhail Gavrilov
0 siblings, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-03-15 19:21 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, MM, Kees Cook, Paul E . McKenney
On Tue, 9 Mar 2021 at 07:31, Hillf Danton <hdanton@sina.com> wrote:
> At the first glance, the zero pointer goes out of the box of race because
>
> 1/ the Call Trace shows it is the free path (of the supposed race victim),
>
> 2/ on the race winner side however either list_del or list_del_init
> would not leave a null pointer behind - the list_add captured in this
> report is under pool->lock.
No more ideas how to fix it?
Kernel panics continue happens again and again with you patches and
recent commits.
[102491.134247] ------------[ cut here ]------------
[102491.134248] list_add corruption. next->prev should be prev
(ffffcc447ea60c78), but was ffffffff8a64ec20. (next=ffff8adc731d3f40).
[102491.134266] ODEBUG: free active (active state 0) object type:
work_struct hint: compact_page_work+0x0/0x10
[102491.134294] ------------[ cut here ]------------
[102491.134295] kernel BUG at lib/list_debug.c:23!
[102491.134299] invalid opcode: 0000 [#1] SMP NOPTI
[102491.134301] CPU: 22 PID: 863413 Comm: kworker/u64:0 Tainted: G
W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.134303] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.134305] Workqueue: zswap3 compact_page_work
[102491.134309] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134312] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
fd fd
[102491.134313] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134315] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134316] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.134316] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.134317] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.134318] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.134319] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.134320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.134320] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.134321] Call Trace:
[102491.134324] do_compact_page+0x28d/0xb60
[102491.134326] ? debug_object_deactivate+0x55/0x140
[102491.134329] ? lock_release+0x1ef/0x410
[102491.134331] ? lock_release+0x1ef/0x410
[102491.134333] process_one_work+0x2b0/0x5e0
[102491.134337] worker_thread+0x55/0x3c0
[102491.134339] ? process_one_work+0x5e0/0x5e0
[102491.134340] kthread+0x13a/0x150
[102491.134342] ? __kthread_bind_mask+0x60/0x60
[102491.134345] ret_from_fork+0x22/0x30
[102491.134349] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib
[102491.134375] WARNING: CPU: 18 PID: 182 at lib/debugobjects.c:505
debug_print_object+0x6e/0x90
[102491.134380] nft_reject_inet nf_reject_ipv4
[102491.134383] Modules linked in:
[102491.134385] nf_reject_ipv6 nft_reject
[102491.134388] snd_seq_dummy
[102491.134390] nft_ct
[102491.134393] snd_hrtimer
[102491.134395] nft_chain_nat nf_nat
[102491.134398] nls_utf8
[102491.134400] nf_conntrack nf_defrag_ipv6
[102491.134403] isofs
[102491.134405] nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
[102491.134476] nvme_core i2c_algo_bit wmi pinctrl_amd fuse
[102491.134484] ---[ end trace 562b0b01453e6613 ]---
[102491.134505] RIP: 0010:__list_add_valid.cold+0xf/0x3f
[102491.134509] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
fd fd
[102491.134511] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
[102491.134514] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
0000000000000000
[102491.134992] uas usb_storage tun uinput rfcomm netconsole
nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat
[102491.135036] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
ffff8ae3497daae0
[102491.135039] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
0000000000000000
[102491.135040] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8adc4e317a08
[102491.135041] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
ffff8adceb216000
[102491.135042] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
knlGS:0000000000000000
[102491.135047] nf_conntrack nf_defrag_ipv6
[102491.135051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135054] nf_defrag_ipv4
[102491.135056] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135059] ip_set
[102491.135061] note: kworker/u64:0[863413] exited with preempt_count 2
[102491.135064] nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
nvme_core i2c_algo_bit wmi
[102491.135357] pinctrl_amd fuse
[102491.135366] CPU: 18 PID: 182 Comm: kcompactd0 Tainted: G D W
--------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102491.135369] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102491.135372] RIP: 0010:debug_print_object+0x6e/0x90
[102491.135403] Code: 49 89 c1 8b 43 10 83 c2 01 8b 4b 14 48 c7 c7 e8
ab 64 8b 89 15 a7 0b 37 03 4c 8b 45 00 48 8b 14 c5 a0 80 2a 8b e8 50
4d 60 00 <0f> 0b 83 05 25 0e 99 01 01 48 83 c4 08 5b 5d c3 83 05 17 0e
99 01
[102491.135406] RSP: 0018:ffffac448080bb78 EFLAGS: 00010296
[102491.135409] RAX: 000000000000005e RBX: ffff8add6de5c7a8 RCX:
0000000000000027
[102491.135412] RDX: ffff8ae348fdaae8 RSI: 0000000000000001 RDI:
ffff8ae348fdaae0
[102491.135415] RBP: ffffffff8b221320 R08: 0000000000000000 R09:
0000000000000000
[102491.135417] R10: 0000000000000000 R11: 0000000000000000 R12:
dead000000000122
[102491.135419] R13: dead000000000100 R14: ffffffff8b221320 R15:
0000000000000005
[102491.135421] FS: 0000000000000000(0000) GS:ffff8ae348e00000(0000)
knlGS:0000000000000000
[102491.135441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102491.135443] CR2: 00007f41b9ee7000 CR3: 00000003b1c28000 CR4:
0000000000350ee0
[102491.135446] Call Trace:
[102491.135451] debug_check_no_obj_freed+0x1db/0x220
[102491.135455] free_pcp_prepare+0x132/0x270
[102491.135459] free_unref_page+0x18/0xd0
[102491.135463] migrate_pages+0x8b9/0x1200
[102491.135467] ? isolate_freepages_block+0x4a0/0x4a0
[102491.135471] ? split_map_pages+0x160/0x160
[102491.135490] compact_zone+0x680/0xfd0
[102491.135493] ? __free_object+0x2b9/0x300
[102491.135496] ? lock_release+0x1ef/0x410
[102491.135500] proactive_compact_node+0x78/0xb0
[102491.135505] kcompactd+0x38a/0x440
[102491.135509] ? do_wait_intr_irq+0xd0/0xd0
[102491.135512] ? kcompactd_do_work+0x3a0/0x3a0
[102491.135515] kthread+0x13a/0x150
[102491.135520] ? __kthread_bind_mask+0x60/0x60
[102491.135533] ret_from_fork+0x22/0x30
[102491.135539] irq event stamp: 220
[102491.135541] hardirqs last enabled at (219): [<ffffffff8ad62217>]
_raw_spin_unlock_irqrestore+0x37/0x40
[102491.135545] hardirqs last disabled at (220): [<ffffffff8ad5b9a9>]
__schedule+0x6e9/0xb20
[102491.135548] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102491.135552] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102491.135555] ---[ end trace 562b0b01453e6614 ]---
[102494.954915] iwlwifi 0000:04:00.0: Error sending SCAN_REQ_UMAC:
time out after 2000ms.
[102494.954950] iwlwifi 0000:04:00.0: Current CMD queue read_ptr 93 write_ptr 94
[102494.956242] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956245] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 6
[102494.956248] iwlwifi 0000:04:00.0: Loaded firmware version:
59.601f3a66.0 cc-a0-59.ucode
[102494.956251] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
[102494.956255] iwlwifi 0000:04:00.0: 0x00A022F0 | trm_hw_status0
[102494.956257] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
[102494.956260] iwlwifi 0000:04:00.0: 0x004FAA36 | branchlink2
[102494.956262] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink1
[102494.956265] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink2
[102494.956268] iwlwifi 0000:04:00.0: 0x004F51B0 | data1
[102494.956270] iwlwifi 0000:04:00.0: 0x01000000 | data2
[102494.956272] iwlwifi 0000:04:00.0: 0x00000000 | data3
[102494.956275] iwlwifi 0000:04:00.0: 0x00000000 | beacon time
[102494.956277] iwlwifi 0000:04:00.0: 0xD78321C6 | tsf low
[102494.956279] iwlwifi 0000:04:00.0: 0x00000017 | tsf hi
[102494.956282] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
[102494.956284] iwlwifi 0000:04:00.0: 0xD783784B | time gp2
[102494.956286] iwlwifi 0000:04:00.0: 0x00000001 | uCode revision type
[102494.956289] iwlwifi 0000:04:00.0: 0x0000003B | uCode version major
[102494.956291] iwlwifi 0000:04:00.0: 0x601F3A66 | uCode version minor
[102494.956294] iwlwifi 0000:04:00.0: 0x00000340 | hw version
[102494.956296] iwlwifi 0000:04:00.0: 0x00C89000 | board version
[102494.956299] iwlwifi 0000:04:00.0: 0x807DFD04 | hcmd
[102494.956302] iwlwifi 0000:04:00.0: 0x00020000 | isr0
[102494.956304] iwlwifi 0000:04:00.0: 0x01000000 | isr1
[102494.956306] iwlwifi 0000:04:00.0: 0x08F04002 | isr2
[102494.956309] iwlwifi 0000:04:00.0: 0x04C3000C | isr3
[102494.956312] iwlwifi 0000:04:00.0: 0x00000000 | isr4
[102494.956315] iwlwifi 0000:04:00.0: 0x005C019C | last cmd Id
[102494.956318] iwlwifi 0000:04:00.0: 0x004F51B0 | wait_event
[102494.956320] iwlwifi 0000:04:00.0: 0x00004B99 | l2p_control
[102494.956322] iwlwifi 0000:04:00.0: 0x00000000 | l2p_duration
[102494.956325] iwlwifi 0000:04:00.0: 0x00000003 | l2p_mhvalid
[102494.956327] iwlwifi 0000:04:00.0: 0x00000000 | l2p_addr_match
[102494.956329] iwlwifi 0000:04:00.0: 0x0000000B | lmpm_pmg_sel
[102494.956332] iwlwifi 0000:04:00.0: 0x00000000 | timestamp
[102494.956334] iwlwifi 0000:04:00.0: 0x000080EC | flow_handler
[102494.956380] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
[102494.956382] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 7
[102494.956385] iwlwifi 0000:04:00.0: 0x20000066 | NMI_INTERRUPT_HOST
[102494.956387] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
[102494.956390] iwlwifi 0000:04:00.0: 0x804568FC | umac branchlink2
[102494.956392] iwlwifi 0000:04:00.0: 0xC0084F3C | umac interruptlink1
[102494.956395] iwlwifi 0000:04:00.0: 0x80477750 | umac interruptlink2
[102494.956397] iwlwifi 0000:04:00.0: 0x01000000 | umac data1
[102494.956399] iwlwifi 0000:04:00.0: 0x80477750 | umac data2
[102494.956401] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
[102494.956404] iwlwifi 0000:04:00.0: 0x0000003B | umac major
[102494.956406] iwlwifi 0000:04:00.0: 0x601F3A66 | umac minor
[102494.956408] iwlwifi 0000:04:00.0: 0xD7837848 | frame pointer
[102494.956411] iwlwifi 0000:04:00.0: 0xC0885F30 | stack pointer
[102494.956413] iwlwifi 0000:04:00.0: 0x005D010D | last host cmd
[102494.956415] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
[102494.956430] iwlwifi 0000:04:00.0: IML/ROM dump:
[102494.956432] iwlwifi 0000:04:00.0: 0x00000003 | IML/ROM error/state
[102494.956446] iwlwifi 0000:04:00.0: 0x00005590 | IML/ROM data1
[102494.956460] iwlwifi 0000:04:00.0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
[102494.956470] iwlwifi 0000:04:00.0: Fseq Registers:
[102494.956475] iwlwifi 0000:04:00.0: 0x60000000 | FSEQ_ERROR_CODE
[102494.956480] iwlwifi 0000:04:00.0: 0x80290021 | FSEQ_TOP_INIT_VERSION
[102494.956486] iwlwifi 0000:04:00.0: 0x00050008 | FSEQ_CNVIO_INIT_VERSION
[102494.956491] iwlwifi 0000:04:00.0: 0x0000A503 | FSEQ_OTP_VERSION
[102494.956496] iwlwifi 0000:04:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
[102494.956502] iwlwifi 0000:04:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
[102494.956507] iwlwifi 0000:04:00.0: 0x00100530 | FSEQ_CNVI_ID
[102494.956512] iwlwifi 0000:04:00.0: 0x00000532 | FSEQ_CNVR_ID
[102494.956518] iwlwifi 0000:04:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
[102494.956525] iwlwifi 0000:04:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
[102494.956532] iwlwifi 0000:04:00.0: 0x05B0905B |
CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
[102494.956540] iwlwifi 0000:04:00.0: 0x0000025B |
CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
[102494.956762] iwlwifi 0000:04:00.0: WRT: Collecting data: ini trigger 4 fired.
[102494.956789] ieee80211 phy0: Hardware restart was requested
[102494.956816] iwlwifi 0000:04:00.0: Scan failed! ret -110
[102494.956925] ------------[ cut here ]------------
[102494.956928] WARNING: CPU: 30 PID: 930660 at
net/mac80211/scan.c:411 __ieee80211_scan_completed+0x2bb/0x520
[mac80211]
[102494.956962] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
bluetooth libarc4
[102494.957007] rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
nvme_core i2c_algo_bit wmi pinctrl_amd fuse
[102494.957036] CPU: 30 PID: 930660 Comm: kworker/u64:2 Tainted: G
D W --------- ---
5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
[102494.957039] Hardware name: System manufacturer System Product
Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
[102494.957042] Workqueue: phy0 ieee80211_scan_work [mac80211]
[102494.957073] RIP: 0010:__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
[102494.957826] Code: ca 0f 82 7d 01 00 00 48 89 ef e8 80 2f 00 00 e9
72 fe ff ff 0f 0b 48 83 bd d8 1c 00 00 00 41 be 01 00 00 00 0f 85 9e
fd ff ff <0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 8b 85 a0 1c 00 00
e9 69
[102494.957830] RSP: 0018:ffffac4495033db0 EFLAGS: 00010246
[102494.957834] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
0000000000000000
[102494.957836] RDX: 0000000000000000 RSI: 0000000000000001 RDI:
ffff8adc770f8e00
[102494.957839] RBP: ffff8adc770f8e00 R08: 0000000000000001 R09:
ffffffffc1395e40
[102494.957842] R10: ffffac4495033de8 R11: 0000000000000000 R12:
0000000000000001
[102494.957844] R13: 0000000000000000 R14: 0000000000000001 R15:
ffff8adc770f8e00
[102494.957847] FS: 0000000000000000(0000) GS:ffff8ae34a600000(0000)
knlGS:0000000000000000
[102494.957850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[102494.957853] CR2: 00003de20ab47000 CR3: 00000001346e0000 CR4:
0000000000350ee0
[102494.957856] Call Trace:
[102494.957862] ieee80211_scan_work+0x15c/0x860 [mac80211]
[102494.957893] ? debug_object_deactivate+0x55/0x140
[102494.957899] ? lock_release+0x1ef/0x410
[102494.957913] ? lock_release+0x1ef/0x410
[102494.957917] process_one_work+0x2b0/0x5e0
[102494.957923] worker_thread+0x55/0x3c0
[102494.957926] ? process_one_work+0x5e0/0x5e0
[102494.957930] kthread+0x13a/0x150
[102494.957934] ? __kthread_bind_mask+0x60/0x60
[102494.957939] ret_from_fork+0x22/0x30
[102494.957945] irq event stamp: 0
[102494.957948] hardirqs last enabled at (0): [<0000000000000000>] 0x0
[102494.957951] hardirqs last disabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957956] softirqs last enabled at (0): [<ffffffff8a0dd990>]
copy_process+0x910/0x1e00
[102494.957959] softirqs last disabled at (0): [<0000000000000000>] 0x0
[102494.957962] ---[ end trace 562b0b01453e6615 ]---
Full kernel log is here: https://pastebin.com/A7dwr8ZV
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-03-15 19:21 ` Mikhail Gavrilov
(?)
@ 2021-03-16 6:13 ` Hillf Danton
-1 siblings, 0 replies; 18+ messages in thread
From: Hillf Danton @ 2021-03-16 6:13 UTC (permalink / raw)
To: Mikhail Gavrilov; +Cc: Hillf Danton, LKML, MM, Kees Cook, Paul E . McKenney
On Tue, 16 Mar 2021 00:21:05 +0500 Mikhail Gavrilov wrote:
> On Tue, 9 Mar 2021 at 07:31, Hillf Danton <hdanton@sina.com> wrote:
> > At the first glance, the zero pointer goes out of the box of race because
> >
> > 1/ the Call Trace shows it is the free path (of the supposed race victim),
> >
> > 2/ on the race winner side however either list_del or list_del_init
> > would not leave a null pointer behind - the list_add captured in this
> > report is under pool->lock.
>
>
> No more ideas how to fix it?
> Kernel panics continue happens again and again with you patches and
> recent commits.
Thanks for your report.
>
> [102491.134247] ------------[ cut here ]------------
> [102491.134248] list_add corruption. next->prev should be prev
> (ffffcc447ea60c78), but was ffffffff8a64ec20. (next=ffff8adc731d3f40).
The same race as we saw over a couple of weeks.
> [102491.134266] ODEBUG: free active (active state 0) object type:
> work_struct hint: compact_page_work+0x0/0x10
> [102491.134294] ------------[ cut here ]------------
This is a new one.
> [102491.134295] kernel BUG at lib/list_debug.c:23!
> [102491.134299] invalid opcode: 0000 [#1] SMP NOPTI
> [102491.134301] CPU: 22 PID: 863413 Comm: kworker/u64:0 Tainted: G
> W --------- ---
The victim was running on CPU22.
> 5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
> [102491.134303] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [102491.134305] Workqueue: zswap3 compact_page_work
> [102491.134309] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [102491.134312] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
> ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
> fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
> fd fd
> [102491.134313] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
> [102491.134315] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
> 0000000000000000
> [102491.134316] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
> ffff8ae3497daae0
> [102491.134316] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
> 0000000000000000
> [102491.134317] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff8adc4e317a08
> [102491.134318] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
> ffff8adceb216000
> [102491.134319] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
> knlGS:0000000000000000
> [102491.134320] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [102491.134320] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
> 0000000000350ee0
> [102491.134321] Call Trace:
> [102491.134324] do_compact_page+0x28d/0xb60
> [102491.134326] ? debug_object_deactivate+0x55/0x140
> [102491.134329] ? lock_release+0x1ef/0x410
> [102491.134331] ? lock_release+0x1ef/0x410
> [102491.134333] process_one_work+0x2b0/0x5e0
> [102491.134337] worker_thread+0x55/0x3c0
> [102491.134339] ? process_one_work+0x5e0/0x5e0
> [102491.134340] kthread+0x13a/0x150
> [102491.134342] ? __kthread_bind_mask+0x60/0x60
> [102491.134345] ret_from_fork+0x22/0x30
> [102491.134349] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
> isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
> nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib
> [102491.134375] WARNING: CPU: 18 PID: 182 at lib/debugobjects.c:505
> debug_print_object+0x6e/0x90
> [102491.134380] nft_reject_inet nf_reject_ipv4
> [102491.134383] Modules linked in:
> [102491.134385] nf_reject_ipv6 nft_reject
> [102491.134388] snd_seq_dummy
> [102491.134390] nft_ct
> [102491.134393] snd_hrtimer
> [102491.134395] nft_chain_nat nf_nat
> [102491.134398] nls_utf8
> [102491.134400] nf_conntrack nf_defrag_ipv6
> [102491.134403] isofs
> [102491.134405] nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
> zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
> intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
> videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
> mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
> edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
> mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
> snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
> snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
> btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
> bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
> ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
> soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
> drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
> crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
> [102491.134476] nvme_core i2c_algo_bit wmi pinctrl_amd fuse
> [102491.134484] ---[ end trace 562b0b01453e6613 ]---
> [102491.134505] RIP: 0010:__list_add_valid.cold+0xf/0x3f
> [102491.134509] Code: 48 c7 c6 34 a9 64 8b 48 89 ef 49 c7 c7 ea ff ff
> ff e8 91 81 01 00 e9 ac f5 9c ff 4c 89 c1 48 c7 c7 00 aa 64 8b e8 67
> fd fd ff <0f> 0b 48 89 f2 4c 89 c1 48 89 fe 48 c7 c7 b0 aa 64 8b e8 50
> fd fd
> [102491.134511] RSP: 0018:ffffac4494cabde0 EFLAGS: 00010282
> [102491.134514] RAX: 0000000000000075 RBX: ffffe6ed86ac8580 RCX:
> 0000000000000000
> [102491.134992] uas usb_storage tun uinput rfcomm netconsole
> nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat
> [102491.135036] RDX: ffff8ae3497e97a0 RSI: ffff8ae3497daae0 RDI:
> ffff8ae3497daae0
> [102491.135039] RBP: ffffcc447ea60c78 R08: 0000000000000000 R09:
> 0000000000000000
> [102491.135040] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff8adc4e317a08
> [102491.135041] R13: ffff8adceb216010 R14: ffff8adc731d3f40 R15:
> ffff8adceb216000
> [102491.135042] FS: 0000000000000000(0000) GS:ffff8ae349600000(0000)
> knlGS:0000000000000000
> [102491.135047] nf_conntrack nf_defrag_ipv6
> [102491.135051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [102491.135054] nf_defrag_ipv4
> [102491.135056] CR2: 0000376f1d60afe8 CR3: 00000003b1c28000 CR4:
> 0000000000350ee0
> [102491.135059] ip_set
> [102491.135061] note: kworker/u64:0[863413] exited with preempt_count 2
> [102491.135064] nf_tables nfnetlink cmac bnep zstd sunrpc vfat fat
> hid_logitech_hidpp hid_logitech_dj intel_rapl_msr intel_rapl_common
> uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2
> snd_hda_codec_realtek videobuf2_common mt76x2u mt76x2_common
> snd_hda_codec_generic mt76x02_usb iwlmvm videodev edac_mce_amd
> snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb mt76x02_lib
> snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
> snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
> snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
> btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
> bluetooth libarc4 rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
> ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
> soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
> drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
> crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
> nvme_core i2c_algo_bit wmi
> [102491.135357] pinctrl_amd fuse
> [102491.135366] CPU: 18 PID: 182 Comm: kcompactd0 Tainted: G D W
> --------- ---
Was the culprit running on CPU 18?
> 5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
> [102491.135369] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [102491.135372] RIP: 0010:debug_print_object+0x6e/0x90
> [102491.135403] Code: 49 89 c1 8b 43 10 83 c2 01 8b 4b 14 48 c7 c7 e8
> ab 64 8b 89 15 a7 0b 37 03 4c 8b 45 00 48 8b 14 c5 a0 80 2a 8b e8 50
> 4d 60 00 <0f> 0b 83 05 25 0e 99 01 01 48 83 c4 08 5b 5d c3 83 05 17 0e
> 99 01
> [102491.135406] RSP: 0018:ffffac448080bb78 EFLAGS: 00010296
> [102491.135409] RAX: 000000000000005e RBX: ffff8add6de5c7a8 RCX:
> 0000000000000027
> [102491.135412] RDX: ffff8ae348fdaae8 RSI: 0000000000000001 RDI:
> ffff8ae348fdaae0
> [102491.135415] RBP: ffffffff8b221320 R08: 0000000000000000 R09:
> 0000000000000000
> [102491.135417] R10: 0000000000000000 R11: 0000000000000000 R12:
> dead000000000122
> [102491.135419] R13: dead000000000100 R14: ffffffff8b221320 R15:
> 0000000000000005
> [102491.135421] FS: 0000000000000000(0000) GS:ffff8ae348e00000(0000)
> knlGS:0000000000000000
> [102491.135441] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [102491.135443] CR2: 00007f41b9ee7000 CR3: 00000003b1c28000 CR4:
> 0000000000350ee0
> [102491.135446] Call Trace:
> [102491.135451] debug_check_no_obj_freed+0x1db/0x220
> [102491.135455] free_pcp_prepare+0x132/0x270
> [102491.135459] free_unref_page+0x18/0xd0
> [102491.135463] migrate_pages+0x8b9/0x1200
> [102491.135467] ? isolate_freepages_block+0x4a0/0x4a0
> [102491.135471] ? split_map_pages+0x160/0x160
> [102491.135490] compact_zone+0x680/0xfd0
> [102491.135493] ? __free_object+0x2b9/0x300
> [102491.135496] ? lock_release+0x1ef/0x410
> [102491.135500] proactive_compact_node+0x78/0xb0
> [102491.135505] kcompactd+0x38a/0x440
> [102491.135509] ? do_wait_intr_irq+0xd0/0xd0
> [102491.135512] ? kcompactd_do_work+0x3a0/0x3a0
> [102491.135515] kthread+0x13a/0x150
> [102491.135520] ? __kthread_bind_mask+0x60/0x60
> [102491.135533] ret_from_fork+0x22/0x30
> [102491.135539] irq event stamp: 220
> [102491.135541] hardirqs last enabled at (219): [<ffffffff8ad62217>]
> _raw_spin_unlock_irqrestore+0x37/0x40
> [102491.135545] hardirqs last disabled at (220): [<ffffffff8ad5b9a9>]
> __schedule+0x6e9/0xb20
> [102491.135548] softirqs last enabled at (0): [<ffffffff8a0dd990>]
> copy_process+0x910/0x1e00
> [102491.135552] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [102491.135555] ---[ end trace 562b0b01453e6614 ]---
> [102494.954915] iwlwifi 0000:04:00.0: Error sending SCAN_REQ_UMAC:
> time out after 2000ms.
> [102494.954950] iwlwifi 0000:04:00.0: Current CMD queue read_ptr 93 write_ptr 94
> [102494.956242] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
> [102494.956245] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 6
> [102494.956248] iwlwifi 0000:04:00.0: Loaded firmware version:
> 59.601f3a66.0 cc-a0-59.ucode
> [102494.956251] iwlwifi 0000:04:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
> [102494.956255] iwlwifi 0000:04:00.0: 0x00A022F0 | trm_hw_status0
> [102494.956257] iwlwifi 0000:04:00.0: 0x00000000 | trm_hw_status1
> [102494.956260] iwlwifi 0000:04:00.0: 0x004FAA36 | branchlink2
> [102494.956262] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink1
> [102494.956265] iwlwifi 0000:04:00.0: 0x000145FA | interruptlink2
> [102494.956268] iwlwifi 0000:04:00.0: 0x004F51B0 | data1
> [102494.956270] iwlwifi 0000:04:00.0: 0x01000000 | data2
> [102494.956272] iwlwifi 0000:04:00.0: 0x00000000 | data3
> [102494.956275] iwlwifi 0000:04:00.0: 0x00000000 | beacon time
> [102494.956277] iwlwifi 0000:04:00.0: 0xD78321C6 | tsf low
> [102494.956279] iwlwifi 0000:04:00.0: 0x00000017 | tsf hi
> [102494.956282] iwlwifi 0000:04:00.0: 0x00000000 | time gp1
> [102494.956284] iwlwifi 0000:04:00.0: 0xD783784B | time gp2
> [102494.956286] iwlwifi 0000:04:00.0: 0x00000001 | uCode revision type
> [102494.956289] iwlwifi 0000:04:00.0: 0x0000003B | uCode version major
> [102494.956291] iwlwifi 0000:04:00.0: 0x601F3A66 | uCode version minor
> [102494.956294] iwlwifi 0000:04:00.0: 0x00000340 | hw version
> [102494.956296] iwlwifi 0000:04:00.0: 0x00C89000 | board version
> [102494.956299] iwlwifi 0000:04:00.0: 0x807DFD04 | hcmd
> [102494.956302] iwlwifi 0000:04:00.0: 0x00020000 | isr0
> [102494.956304] iwlwifi 0000:04:00.0: 0x01000000 | isr1
> [102494.956306] iwlwifi 0000:04:00.0: 0x08F04002 | isr2
> [102494.956309] iwlwifi 0000:04:00.0: 0x04C3000C | isr3
> [102494.956312] iwlwifi 0000:04:00.0: 0x00000000 | isr4
> [102494.956315] iwlwifi 0000:04:00.0: 0x005C019C | last cmd Id
> [102494.956318] iwlwifi 0000:04:00.0: 0x004F51B0 | wait_event
> [102494.956320] iwlwifi 0000:04:00.0: 0x00004B99 | l2p_control
> [102494.956322] iwlwifi 0000:04:00.0: 0x00000000 | l2p_duration
> [102494.956325] iwlwifi 0000:04:00.0: 0x00000003 | l2p_mhvalid
> [102494.956327] iwlwifi 0000:04:00.0: 0x00000000 | l2p_addr_match
> [102494.956329] iwlwifi 0000:04:00.0: 0x0000000B | lmpm_pmg_sel
> [102494.956332] iwlwifi 0000:04:00.0: 0x00000000 | timestamp
> [102494.956334] iwlwifi 0000:04:00.0: 0x000080EC | flow_handler
> [102494.956380] iwlwifi 0000:04:00.0: Start IWL Error Log Dump:
> [102494.956382] iwlwifi 0000:04:00.0: Status: 0x00000040, count: 7
> [102494.956385] iwlwifi 0000:04:00.0: 0x20000066 | NMI_INTERRUPT_HOST
> [102494.956387] iwlwifi 0000:04:00.0: 0x00000000 | umac branchlink1
> [102494.956390] iwlwifi 0000:04:00.0: 0x804568FC | umac branchlink2
> [102494.956392] iwlwifi 0000:04:00.0: 0xC0084F3C | umac interruptlink1
> [102494.956395] iwlwifi 0000:04:00.0: 0x80477750 | umac interruptlink2
> [102494.956397] iwlwifi 0000:04:00.0: 0x01000000 | umac data1
> [102494.956399] iwlwifi 0000:04:00.0: 0x80477750 | umac data2
> [102494.956401] iwlwifi 0000:04:00.0: 0x00000000 | umac data3
> [102494.956404] iwlwifi 0000:04:00.0: 0x0000003B | umac major
> [102494.956406] iwlwifi 0000:04:00.0: 0x601F3A66 | umac minor
> [102494.956408] iwlwifi 0000:04:00.0: 0xD7837848 | frame pointer
> [102494.956411] iwlwifi 0000:04:00.0: 0xC0885F30 | stack pointer
> [102494.956413] iwlwifi 0000:04:00.0: 0x005D010D | last host cmd
> [102494.956415] iwlwifi 0000:04:00.0: 0x00000000 | isr status reg
> [102494.956430] iwlwifi 0000:04:00.0: IML/ROM dump:
> [102494.956432] iwlwifi 0000:04:00.0: 0x00000003 | IML/ROM error/state
> [102494.956446] iwlwifi 0000:04:00.0: 0x00005590 | IML/ROM data1
> [102494.956460] iwlwifi 0000:04:00.0: 0x00000080 | IML/ROM WFPM_AUTH_KEY_0
> [102494.956470] iwlwifi 0000:04:00.0: Fseq Registers:
> [102494.956475] iwlwifi 0000:04:00.0: 0x60000000 | FSEQ_ERROR_CODE
> [102494.956480] iwlwifi 0000:04:00.0: 0x80290021 | FSEQ_TOP_INIT_VERSION
> [102494.956486] iwlwifi 0000:04:00.0: 0x00050008 | FSEQ_CNVIO_INIT_VERSION
> [102494.956491] iwlwifi 0000:04:00.0: 0x0000A503 | FSEQ_OTP_VERSION
> [102494.956496] iwlwifi 0000:04:00.0: 0x80000003 | FSEQ_TOP_CONTENT_VERSION
> [102494.956502] iwlwifi 0000:04:00.0: 0x4552414E | FSEQ_ALIVE_TOKEN
> [102494.956507] iwlwifi 0000:04:00.0: 0x00100530 | FSEQ_CNVI_ID
> [102494.956512] iwlwifi 0000:04:00.0: 0x00000532 | FSEQ_CNVR_ID
> [102494.956518] iwlwifi 0000:04:00.0: 0x00100530 | CNVI_AUX_MISC_CHIP
> [102494.956525] iwlwifi 0000:04:00.0: 0x00000532 | CNVR_AUX_MISC_CHIP
> [102494.956532] iwlwifi 0000:04:00.0: 0x05B0905B |
> CNVR_SCU_SD_REGS_SD_REG_DIG_DCDC_VTRIM
> [102494.956540] iwlwifi 0000:04:00.0: 0x0000025B |
> CNVR_SCU_SD_REGS_SD_REG_ACTIVE_VDIG_MIRROR
> [102494.956762] iwlwifi 0000:04:00.0: WRT: Collecting data: ini trigger 4 fired.
> [102494.956789] ieee80211 phy0: Hardware restart was requested
> [102494.956816] iwlwifi 0000:04:00.0: Scan failed! ret -110
> [102494.956925] ------------[ cut here ]------------
> [102494.956928] WARNING: CPU: 30 PID: 930660 at
> net/mac80211/scan.c:411 __ieee80211_scan_completed+0x2bb/0x520
> [mac80211]
> [102494.956962] Modules linked in: snd_seq_dummy snd_hrtimer nls_utf8
> isofs uas usb_storage tun uinput rfcomm netconsole nft_objref
> nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
> nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
> nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack
> nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink cmac bnep
> zstd sunrpc vfat fat hid_logitech_hidpp hid_logitech_dj intel_rapl_msr
> intel_rapl_common uvcvideo videobuf2_vmalloc videobuf2_memops
> videobuf2_v4l2 snd_hda_codec_realtek videobuf2_common mt76x2u
> mt76x2_common snd_hda_codec_generic mt76x02_usb iwlmvm videodev
> edac_mce_amd snd_usb_audio ledtrig_audio snd_hda_codec_hdmi mt76_usb
> mt76x02_lib snd_usbmidi_lib kvm_amd snd_hda_intel mt76 snd_rawmidi
> snd_intel_dspcfg mc joydev snd_intel_sdw_acpi mac80211 kvm
> snd_hda_codec snd_hda_core iwlwifi btusb snd_hwdep btrtl snd_seq btbcm
> btintel eeepc_wmi asus_wmi snd_seq_device irqbypass xpad sparse_keymap
> bluetooth libarc4
> [102494.957007] rapl snd_pcm video ff_memless wmi_bmof ecdh_generic
> ecc cfg80211 pcspkr snd_timer k10temp snd sp5100_tco i2c_piix4
> soundcore rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu
> drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul
> crc32_pclmul crc32c_intel cec igb drm nvme ghash_clmulni_intel dca ccp
> nvme_core i2c_algo_bit wmi pinctrl_amd fuse
> [102494.957036] CPU: 30 PID: 930660 Comm: kworker/u64:2 Tainted: G
> D W --------- ---
> 5.12.0-0.rc2.20210310git05a59d79793d.168.fc35.x86_64 #1
> [102494.957039] Hardware name: System manufacturer System Product
> Name/ROG STRIX X570-I GAMING, BIOS 3402 01/13/2021
> [102494.957042] Workqueue: phy0 ieee80211_scan_work [mac80211]
> [102494.957073] RIP: 0010:__ieee80211_scan_completed+0x2bb/0x520 [mac80211]
> [102494.957826] Code: ca 0f 82 7d 01 00 00 48 89 ef e8 80 2f 00 00 e9
> 72 fe ff ff 0f 0b 48 83 bd d8 1c 00 00 00 41 be 01 00 00 00 0f 85 9e
> fd ff ff <0f> 0b 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 8b 85 a0 1c 00 00
> e9 69
> [102494.957830] RSP: 0018:ffffac4495033db0 EFLAGS: 00010246
> [102494.957834] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000000
> [102494.957836] RDX: 0000000000000000 RSI: 0000000000000001 RDI:
> ffff8adc770f8e00
> [102494.957839] RBP: ffff8adc770f8e00 R08: 0000000000000001 R09:
> ffffffffc1395e40
> [102494.957842] R10: ffffac4495033de8 R11: 0000000000000000 R12:
> 0000000000000001
> [102494.957844] R13: 0000000000000000 R14: 0000000000000001 R15:
> ffff8adc770f8e00
> [102494.957847] FS: 0000000000000000(0000) GS:ffff8ae34a600000(0000)
> knlGS:0000000000000000
> [102494.957850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [102494.957853] CR2: 00003de20ab47000 CR3: 00000001346e0000 CR4:
> 0000000000350ee0
> [102494.957856] Call Trace:
> [102494.957862] ieee80211_scan_work+0x15c/0x860 [mac80211]
> [102494.957893] ? debug_object_deactivate+0x55/0x140
> [102494.957899] ? lock_release+0x1ef/0x410
> [102494.957913] ? lock_release+0x1ef/0x410
> [102494.957917] process_one_work+0x2b0/0x5e0
> [102494.957923] worker_thread+0x55/0x3c0
> [102494.957926] ? process_one_work+0x5e0/0x5e0
> [102494.957930] kthread+0x13a/0x150
> [102494.957934] ? __kthread_bind_mask+0x60/0x60
> [102494.957939] ret_from_fork+0x22/0x30
> [102494.957945] irq event stamp: 0
> [102494.957948] hardirqs last enabled at (0): [<0000000000000000>] 0x0
> [102494.957951] hardirqs last disabled at (0): [<ffffffff8a0dd990>]
> copy_process+0x910/0x1e00
> [102494.957956] softirqs last enabled at (0): [<ffffffff8a0dd990>]
> copy_process+0x910/0x1e00
> [102494.957959] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [102494.957962] ---[ end trace 562b0b01453e6615 ]---
>
> Full kernel log is here: https://pastebin.com/A7dwr8ZV
>
>
> --
> Best Regards,
> Mike Gavrilov.
Lets see the race between the work and compact/migrate pathes.
work migrate
---- -------
VM_BUG_ON_PAGE(!test_bit(PAGE_CLAIMED,
&page->private), page);
zhdr = page_address(page);
if (!z3fold_page_trylock(zhdr))
return -EAGAIN;
if (work_pending(&zhdr->work)) {
z3fold_page_unlock(zhdr);
return -EAGAIN;
}
*&*--> page->private = 0;
z3fold_page_unlock(zhdr);
page_mapcount_reset(page);
clear_bit(PAGE_CLAIMED, &page->private);
put_page(page);
return 0;
z3fold_page_lock(zhdr);
if (test_bit(PAGE_STALE, &page->private) ||
test_and_set_bit(PAGE_CLAIMED, &page->private)) {
z3fold_page_unlock(zhdr);
return;
}
...
z3fold_page_unlock(zhdr);
====
1/ no chance for race at the first glance because 1) PAGE_CLAIMED is
checked on both sides and 2) both take z3fold_page_lock(zhdr).
2/ it is bad to reset page->private because it goes odd with the clearing
of PAGE_CLAIMED before put_page().
3/ the trigger of the second warning indicates it is necessary to wait
for the work to be done before put_page().
That said, the quick fix is to cancel work if the first hunk below wont
survive your test. Note it wont make sense without the previous diffs.
--- x/mm/z3fold.c
+++ y/mm/z3fold.c
@@ -1623,7 +1623,6 @@ static int z3fold_page_migrate(struct ad
new_zhdr = page_address(newpage);
memcpy(new_zhdr, zhdr, PAGE_SIZE);
newpage->private = page->private;
- page->private = 0;
z3fold_page_unlock(zhdr);
spin_lock_init(&new_zhdr->page_lock);
INIT_WORK(&new_zhdr->work, compact_page_work);
@@ -1654,6 +1653,7 @@ static int z3fold_page_migrate(struct ad
queue_work_on(new_zhdr->cpu, pool->compact_wq, &new_zhdr->work);
+ cancel_work_sync(&zhdr->work);
page_mapcount_reset(page);
clear_bit(PAGE_CLAIMED, &page->private);
put_page(page);
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
2021-01-24 18:23 ` Mikhail Gavrilov
@ 2021-01-26 0:22 ` Mikhail Gavrilov
0 siblings, 0 replies; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-01-26 0:22 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, Kees Cook, paulmck
On Sun, 24 Jan 2021 at 23:23, Mikhail Gavrilov
<mikhail.v.gavrilov@gmail.com> wrote:
>
> Thanks for looking at the issue.
> Why the proposed patch not intended for testing?
> It is not the final (optimal) variant?
>
>
> --
> Best Regards,
> Mike Gavrilov.
With disabled kasan I got slightly different trace (which flooded the
kernel logs):
z3fold: No free chunks in unbuddied
------------[ cut here ]------------
WARNING: CPU: 16 PID: 270 at mm/z3fold.c:1120 z3fold_zpool_malloc+0xe4/0x780
Modules linked in: tun snd_seq_dummy snd_hrtimer uinput rfcomm
nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet
nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4
nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ip6table_nat
ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat
nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw
iptable_security ip_set nf_tables nfnetlink ip6table_filter ip6_tables
iptable_filter cmac bnep zstd sunrpc vfat fat uas usb_storage
hid_logitech_hidpp hid_logitech_dj mt76x2u mt76x2_common mt76x02_usb
mt76_usb mt76x02_lib mt76 gspca_zc3xx gspca_main snd_hda_codec_realtek
snd_hda_codec_generic intel_rapl_msr snd_hda_codec_hdmi ledtrig_audio
intel_rapl_common snd_hda_intel snd_intel_dspcfg iwlmvm
soundwire_intel soundwire_generic_allocation snd_soc_core mac80211
snd_compress snd_pcm_dmaengine soundwire_cadence snd_hda_codec joydev
edac_mce_amd uvcvideo snd_hda_core kvm_amd btusb
videobuf2_vmalloc btrtl videobuf2_memops ac97_bus videobuf2_v4l2
btbcm snd_usb_audio libarc4 btintel videobuf2_common snd_usbmidi_lib
kvm bluetooth snd_hwdep iwlwifi videodev snd_seq snd_rawmidi eeepc_wmi
asus_wmi snd_seq_device irqbypass mc sparse_keymap xpad ecdh_generic
snd_pcm rapl ff_memless wmi_bmof video ecc cfg80211 pcspkr snd_timer
k10temp snd sp5100_tco i2c_piix4 soundcore rfkill acpi_cpufreq
binfmt_misc ip_tables amdgpu drm_ttm_helper ttm iommu_v2 gpu_sched
drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel cec igb nvme
drm ghash_clmulni_intel ccp xhci_pci dca nvme_core xhci_pci_renesas
i2c_algo_bit wmi pinctrl_amd fuse
CPU: 16 PID: 270 Comm: kswapd0 Tainted: G W ---------
--- 5.11.0-0.rc4.20210120git45dfb8a5659a.133.fc34.x86_64 #1
Hardware name: System manufacturer System Product Name/ROG STRIX
X570-I GAMING, BIOS 3402 01/13/2021
RIP: 0010:z3fold_zpool_malloc+0xe4/0x780
Code: 0f c1 43 58 83 f8 01 0f 84 7c 06 00 00 85 c0 0f 8e 93 06 00 00
48 8d 7b 10 e8 a8 8c 9a 00 48 c7 c7 c8 b5 5f b2 e8 46 ce 93 00 <0f> 0b
eb 81 c7 04 24 00 00 00 00 8b 7c 24 18 85 ff 0f 84 a6 00 00
RSP: 0018:ffffb39dc086b910 EFLAGS: 00010282
RAX: 0000000000000023 RBX: ffff9c12bfc3f000 RCX: 0000000000000000
RDX: ffff9c1888be9f60 RSI: ffff9c1888bdb2a0 RDI: ffff9c1888bdb2a0
RBP: 0000000000000000 R08: 0000000000000000 R09: ffffb39dc086b750
R10: ffffb39dc086b748 R11: 0000000000000000 R12: ffff9c11b25cd400
R13: 0000000000012800 R14: 00000000000001a0 R15: 0000000000000007
FS: 0000000000000000(0000) GS:ffff9c1888a00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00001a925bb89fe8 CR3: 00000003862c4000 CR4: 0000000000350ee0
Call Trace:
? _raw_spin_unlock+0x1f/0x30
zswap_frontswap_store+0x43e/0x890
__frontswap_store+0xc8/0x170
swap_writepage+0x39/0x70
pageout+0x125/0x540
shrink_page_list+0x1329/0x1bc0
shrink_inactive_list+0x12a/0x440
shrink_lruvec+0x4a9/0x6d0
? super_cache_count+0x79/0xf0
shrink_node+0x2d1/0x700
balance_pgdat+0x2f5/0x650
kswapd+0x21d/0x4d0
? do_wait_intr_irq+0xd0/0xd0
? balance_pgdat+0x650/0x650
kthread+0x13a/0x150
? __kthread_bind_mask+0x60/0x60
ret_from_fork+0x22/0x30
irq event stamp: 46
hardirqs last enabled at (45): [<ffffffffb1d3fea1>]
_raw_spin_unlock_irqrestore+0x41/0x50
hardirqs last disabled at (46): [<ffffffffb1d39aaf>] __schedule+0x6ef/0xb20
softirqs last enabled at (0): [<ffffffffb10ddbbb>] copy_process+0x8fb/0x1de0
softirqs last disabled at (0): [<0000000000000000>] 0x0
---[ end trace d045ca861a4f792f ]---
z3fold: No free chunks in unbuddied
Full kernel log is here: https://pastebin.com/BTJ0Fz6d
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
[not found] ` <20210124111047.13404-1-hdanton@sina.com>
@ 2021-01-24 18:23 ` Mikhail Gavrilov
2021-01-26 0:22 ` Mikhail Gavrilov
0 siblings, 1 reply; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-01-24 18:23 UTC (permalink / raw)
To: Hillf Danton; +Cc: LKML, Kees Cook, paulmck
On Sun, 24 Jan 2021 at 16:11, Hillf Danton <hdanton@sina.com> wrote:
>
> If it is supposed due to the race between pool->stale_lock and
> pool->lock that are both protecting the buddy list_head then adding
> another one can be a cure. The diff below is not for any test.
Thanks for looking at the issue.
Why the proposed patch not intended for testing?
It is not the final (optimal) variant?
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
* BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4)
@ 2021-01-21 18:03 Mikhail Gavrilov
[not found] ` <20210124111047.13404-1-hdanton@sina.com>
0 siblings, 1 reply; 18+ messages in thread
From: Mikhail Gavrilov @ 2021-01-21 18:03 UTC (permalink / raw)
To: Linux List Kernel Mailing; +Cc: keescook, paulmck
Hi folks,
I am testing new kernels under high load and KASAN found some troubles:
BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0
Read of size 8 at addr ffff8881f2cda008 by task ThreadPoolForeg/110220
CPU: 22 PID: 110220 Comm: ThreadPoolForeg Tainted: G W
--------- --- 5.11.0-0.rc4.20210120git45dfb8a5659a.131.fc34.x86_64 #1
Hardware name: System manufacturer System Product Name/ROG STRIX
X570-I GAMING, BIOS 3402 01/13/2021
Call Trace:
dump_stack+0xae/0xe5
print_address_description.constprop.0+0x18/0x160
? __list_add_valid+0x81/0xa0
kasan_report.cold+0x7f/0x10e
? lock_contended+0xb10/0xbe0
? __list_add_valid+0x81/0xa0
__list_add_valid+0x81/0xa0
do_compact_page+0x8bf/0x2720
? z3fold_zpool_free+0x92d/0x1150
? lock_contended+0xbe0/0xbe0
zswap_free_entry+0xfa/0x1b0
zswap_frontswap_invalidate_page+0x14a/0x1a0
__frontswap_invalidate_page+0x104/0x1c0
swap_range_free+0x2ad/0x350
swapcache_free_entries+0x1e1/0x300
free_swap_slot+0x1d2/0x290
? enable_swap_slots_cache+0x90/0x90
__swap_entry_free+0x109/0x130
? __swap_entry_free_locked+0x1a0/0x1a0
free_swap_and_cache+0xb3/0x100
? get_swap_page_of_type+0x160/0x160
unmap_page_range+0xf3c/0x23e0
? lock_downgrade+0x6b0/0x6b0
? lru_add_drain_cpu+0x182/0x670
? vm_normal_page_pmd+0x350/0x350
zap_page_range+0x289/0x400
? unmap_vmas+0x250/0x250
? lock_downgrade+0x6b0/0x6b0
? lock_acquire+0x31d/0x7a0
? __init_rwsem+0x1a0/0x1a0
? find_vma_prev+0x21/0x1d0
do_madvise.part.0+0x10b6/0x2060
? do_wp_page+0x311/0xca0
? madvise_cold+0x1c0/0x1c0
? do_user_addr_fault+0x432/0x9b0
? __x64_sys_madvise+0xd8/0x140
__x64_sys_madvise+0xd8/0x140
do_syscall_64+0x33/0x40
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7f662cf620cb
Code: c3 66 0f 1f 44 00 00 48 8b 15 a1 7d 0c 00 f7 d8 64 89 02 b8 ff
ff ff ff eb bc 0f 1f 44 00 00 f3 0f 1e fa b8 1c 00 00 00 0f 05 <48> 3d
01 f0 ff ff 73 01 c3 48 8b 0d 75 7d 0c 00 f7 d8 64 89 01 48
RSP: 002b:00007f66038f4668 EFLAGS: 00000206 ORIG_RAX: 000000000000001c
RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f662cf620cb
RDX: 0000000000000004 RSI: 0000000000008000 RDI: 00003cb26d246000
RBP: 00007f66038f4690 R08: 0000000000000000 R09: aaaaaaaa00000000
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000008000
R13: 00000003cb26d246 R14: 00003cb26d24e000 R15: 00003cb26d246000
The buggy address belongs to the page:
page:00000000d921a94d refcount:0 mapcount:-128
mapping:0000000000000000 index:0x1 pfn:0x1f2cda
flags: 0x17ffffc0000000()
raw: 0017ffffc0000000 ffffea0007cb3a48 ffffea0007cb3588 0000000000000000
raw: 0000000000000001 0000000000000000 00000000ffffff7f 0000000000000000
page dumped because: kasan: bad access detected
Memory state around the buggy address:
ffff8881f2cd9f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ffff8881f2cd9f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>ffff8881f2cda000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
^
ffff8881f2cda080: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ffff8881f2cda100: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
==================================================================
list_add corruption. next->prev should be prev (ffffe8fffd662670), but
was 0000000000672100. (next=ffff8881f2cda000).
------------[ cut here ]------------
kernel BUG at lib/list_debug.c:23!
invalid opcode: 0000 [#1] SMP KASAN NOPTI
CPU: 22 PID: 107597 Comm: kworker/u64:2 Tainted: G B W
--------- --- 5.11.0-0.rc4.20210120git45dfb8a5659a.131.fc34.x86_64 #1
$ /usr/src/kernels/`uname -r`/scripts/faddr2line
/lib/debug/lib/modules/`uname -r`/vmlinux __list_add_valid+0x81
__list_add_valid+0x81/0xa0:
__list_add_valid at lib/list_debug.c:23
$ git checkout 45dfb8a5659a
Previous HEAD position was 19c329f68089 Linux 5.11-rc4
HEAD is now at 45dfb8a5659a Merge tag 'task_work-2021-01-19' of
git://git.kernel.dk/linux-block
$ git blame lib/list_debug.c -L14,33
Blaming lines: 32% (20/62), done.
199a9afc3dbe9 (Dave Jones 2006-09-29 01:59:00 -0700 14) /*
d7c816733d501 (Kees Cook 2016-08-17 14:42:08 -0700 15) * Check
that the data structures for the list manipulations are reasonably
d7c816733d501 (Kees Cook 2016-08-17 14:42:08 -0700 16) * valid.
Failures here indicate memory corruption (and possibly an exploit
d7c816733d501 (Kees Cook 2016-08-17 14:42:08 -0700 17) * attempt).
199a9afc3dbe9 (Dave Jones 2006-09-29 01:59:00 -0700 18) */
199a9afc3dbe9 (Dave Jones 2006-09-29 01:59:00 -0700 19)
d7c816733d501 (Kees Cook 2016-08-17 14:42:08 -0700 20) bool
__list_add_valid(struct list_head *new, struct list_head *prev,
d7c816733d501 (Kees Cook 2016-08-17 14:42:08 -0700 21)
struct list_head *next)
199a9afc3dbe9 (Dave Jones 2006-09-29 01:59:00 -0700 22) {
85caa95b9f19b (Kees Cook 2017-02-24 15:00:38 -0800 23) if
(CHECK_DATA_CORRUPTION(next->prev != prev,
68c1f08203f2b (Matthew Wilcox 2018-04-10 16:33:06 -0700 24)
"list_add corruption. next->prev should be prev (%px), but
was %px. (next=%px).\n",
85caa95b9f19b (Kees Cook 2017-02-24 15:00:38 -0800 25)
prev, next->prev, next) ||
85caa95b9f19b (Kees Cook 2017-02-24 15:00:38 -0800 26)
CHECK_DATA_CORRUPTION(prev->next != next,
68c1f08203f2b (Matthew Wilcox 2018-04-10 16:33:06 -0700 27)
"list_add corruption. prev->next should be next (%px), but
was %px. (prev=%px).\n",
85caa95b9f19b (Kees Cook 2017-02-24 15:00:38 -0800 28)
next, prev->next, prev) ||
85caa95b9f19b (Kees Cook 2017-02-24 15:00:38 -0800 29)
CHECK_DATA_CORRUPTION(new == prev || new == next,
68c1f08203f2b (Matthew Wilcox 2018-04-10 16:33:06 -0700 30)
"list_add double add: new=%px, prev=%px, next=%px.\n",
85caa95b9f19b (Kees Cook 2017-02-24 15:00:38 -0800 31)
new, prev, next))
85caa95b9f19b (Kees Cook 2017-02-24 15:00:38 -0800 32)
return false;
de54ebbe26bb3 (Kees Cook 2016-08-17 14:42:11 -0700 33)
Full kernel log here: https://pastebin.com/sycghWB5
I added to CC all who was involved in these lines of code.
I hope you help fix this issue.
--
Best Regards,
Mike Gavrilov.
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2021-03-16 6:14 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20210126082834.2020-1-hdanton@sina.com>
2021-02-12 13:28 ` BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4) Mikhail Gavrilov
2021-02-13 3:03 ` Hillf Danton
2021-02-28 13:22 ` Mikhail Gavrilov
2021-02-28 13:22 ` Mikhail Gavrilov
2021-03-01 3:11 ` Hillf Danton
2021-03-05 9:33 ` Mikhail Gavrilov
2021-03-05 9:33 ` Mikhail Gavrilov
2021-03-05 14:22 ` Hillf Danton
2021-03-08 15:42 ` Mikhail Gavrilov
2021-03-08 15:42 ` Mikhail Gavrilov
2021-03-09 2:31 ` Hillf Danton
2021-03-15 19:18 ` Mikhail Gavrilov
2021-03-15 19:21 ` Mikhail Gavrilov
2021-03-15 19:21 ` Mikhail Gavrilov
2021-03-16 6:13 ` Hillf Danton
2021-01-21 18:03 Mikhail Gavrilov
[not found] ` <20210124111047.13404-1-hdanton@sina.com>
2021-01-24 18:23 ` Mikhail Gavrilov
2021-01-26 0:22 ` Mikhail Gavrilov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.