From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.7 required=3.0 tests=BAYES_00, FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0EC1C433DB for ; Sat, 13 Feb 2021 03:03:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D4CA564E23 for ; Sat, 13 Feb 2021 03:03:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D4CA564E23 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sina.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 337938D00B1; Fri, 12 Feb 2021 22:03:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C0A88D0060; Fri, 12 Feb 2021 22:03:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B1C38D00B1; Fri, 12 Feb 2021 22:03:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id 01A028D0060 for ; Fri, 12 Feb 2021 22:03:43 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A89A3824805A for ; Sat, 13 Feb 2021 03:03:43 +0000 (UTC) X-FDA: 77811749526.13.FC8BF09 Received: from mail3-166.sinamail.sina.com.cn (mail3-166.sinamail.sina.com.cn [202.108.3.166]) by imf20.hostedemail.com (Postfix) with SMTP id A0810139 for ; Sat, 13 Feb 2021 03:03:39 +0000 (UTC) Received: from unknown (HELO localhost.localdomain)([114.254.172.92]) by sina.com (172.16.97.23) with ESMTP id 602741880001495D; Sat, 13 Feb 2021 11:03:37 +0800 (CST) X-Sender: hdanton@sina.com X-Auth-ID: hdanton@sina.com X-SMAIL-MID: 55467054919419 From: Hillf Danton To: Mikhail Gavrilov Cc: Hillf Danton , LKML , MM , Kees Cook , "Paul E . McKenney" Subject: Re: BUG: KASAN: use-after-free in __list_add_valid+0x81/0xa0 (5.11-rc4) Date: Sat, 13 Feb 2021 11:03:26 +0800 Message-Id: <20210213030327.4992-1-hdanton@sina.com> In-Reply-To: References: <20210126082834.2020-1-hdanton@sina.com> MIME-Version: 1.0 X-Stat-Signature: gpy5kfehp4ytbrzs647ysa93jfreoo9s X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: A0810139 Received-SPF: none (sina.com>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail3-166.sinamail.sina.com.cn; client-ip=202.108.3.166 X-HE-DKIM-Result: none/none X-HE-Tag: 1613185419-827313 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 12 Feb 2021 18:28:12 +0500 Mikhail Gavrilov wrote: > On Tue, 26 Jan 2021 at 13:28, Hillf Danton wrote: > > > > BTW better run the reproducer again with KASAN enabled. > > >=20 > It happened today again with kernel 5.11 rc7 (e0756cfc7d7c) Thanks again. > Why not try your patch? Simply because it was half baked - I was not convinced it was a fix instead of papering over anything. >=20 > list_del corruption, ffffdef70143e848->next is LIST_POISON1 (dead000000= 000100) > ------------[ cut here ]------------ > kernel BUG at lib/list_debug.c:45! > invalid opcode: 0000 [#1] SMP NOPTI > CPU: 13 PID: 263 Comm: kswapd0 Tainted: G W --------- > --- 5.11.0-0.rc7.20210210gite0756cfc7d7c.150.fc35.x86_64 #1 > Hardware name: System manufacturer System Product Name/ROG STRIX > X570-I GAMING, BIOS 3402 01/13/2021 > RIP: 0010:__list_del_entry_valid.cold+0xf/0x47 > Code: fe ff 0f 0b 48 89 d1 4c 89 c6 4c 89 ca 48 c7 c7 e0 26 64 9e e8 > 1b 12 fe ff 0f 0b 48 89 fe 48 c7 c7 70 27 64 9e e8 0a 12 fe ff <0f> 0b > 48 c7 c7 20 28 64 9e e8 fc 11 fe ff 0f 0b 48 89 f2 48 89 fe > RSP: 0018:ffff9f2180863908 EFLAGS: 00010286 > RAX: 000000000000004e RBX: ffff8f74d0fa1000 RCX: 0000000000000000 > RDX: ffff8f7c885e9f60 RSI: ffff8f7c885db2a0 RDI: ffff8f7c885db2a0 > RBP: ffff8f74d0fa1000 R08: 0000000000000000 R09: ffff9f2180863748 > R10: ffff9f2180863740 R11: 0000000000000000 R12: ffff8f758edd8e00 > R13: 0000000000012800 R14: ffffdef70143e840 R15: ffff8f758edd8e08 > FS: 0000000000000000(0000) GS:ffff8f7c88400000(0000) knlGS:00000000000= 00000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 000037203acb1000 CR3: 00000001d5c28000 CR4: 0000000000350ee0 > Call Trace: > z3fold_zpool_malloc+0x3e3/0x780 There is a race producing the list corruption above. > ? _raw_spin_unlock+0x1f/0x30 > zswap_frontswap_store+0x43e/0x890 > __frontswap_store+0xc8/0x170 > swap_writepage+0x39/0x70 > pageout+0x125/0x540 > shrink_page_list+0x1329/0x1bc0 > shrink_inactive_list+0x12a/0x440 > shrink_lruvec+0x4a9/0x6d0 > ? super_cache_count+0x79/0xf0 > shrink_node+0x2d1/0x700 > balance_pgdat+0x2f5/0x650 > kswapd+0x21d/0x4d0 > ? do_wait_intr_irq+0xd0/0xd0 > ? balance_pgdat+0x650/0x650 > kthread+0x13a/0x150 > ? __kthread_bind_mask+0x60/0x60 > ret_from_fork+0x22/0x30 > Modules linked in: tun snd_seq_dummy snd_hrtimer uinput rfcomm > nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet > nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 > nf_reject_ipv6 nft_reject nft_ct nft_chain_nat ip6table_nat > ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat > nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_raw > iptable_security ip_set nf_tables nfnetlink ip6table_filter ip6_tables > iptable_filter cmac bnep zstd sunrpc vfat fat hid_logitech_hidpp > hid_logitech_dj snd_hda_codec_realtek snd_hda_codec_generic > ledtrig_audio snd_hda_codec_hdmi snd_hda_intel mt76x2u > snd_intel_dspcfg soundwire_intel mt76x2_common mt76x02_usb > soundwire_generic_allocation mt76_usb intel_rapl_msr iwlmvm > snd_soc_core snd_usb_audio intel_rapl_common mt76x02_lib mt76 > snd_compress snd_pcm_dmaengine snd_usbmidi_lib soundwire_cadence > snd_rawmidi mac80211 snd_hda_codec joydev snd_hda_core uvcvideo > ac97_bus snd_hwdep btusb snd_seq > videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 snd_seq_device > btrtl edac_mce_amd btbcm iwlwifi snd_pcm videobuf2_common btintel > kvm_amd eeepc_wmi snd_timer bluetooth kvm videodev asus_wmi snd > ecdh_generic sparse_keymap irqbypass xpad mc libarc4 sp5100_tco rapl > ff_memless cfg80211 wmi_bmof ecc video pcspkr soundcore k10temp > i2c_piix4 rfkill acpi_cpufreq binfmt_misc ip_tables amdgpu > drm_ttm_helper ttm iommu_v2 gpu_sched drm_kms_helper crct10dif_pclmul > crc32_pclmul crc32c_intel cec drm ghash_clmulni_intel igb ccp nvme dca > nvme_core i2c_algo_bit wmi pinctrl_amd fuse > ---[ end trace a0c35e2a81af0791 ]--- > RIP: 0010:__list_del_entry_valid.cold+0xf/0x47 > Code: fe ff 0f 0b 48 89 d1 4c 89 c6 4c 89 ca 48 c7 c7 e0 26 64 9e e8 > 1b 12 fe ff 0f 0b 48 89 fe 48 c7 c7 70 27 64 9e e8 0a 12 fe ff <0f> 0b > 48 c7 c7 20 28 64 9e e8 fc 11 fe ff 0f 0b 48 89 f2 48 89 fe > RSP: 0018:ffff9f2180863908 EFLAGS: 00010286 > RAX: 000000000000004e RBX: ffff8f74d0fa1000 RCX: 0000000000000000 > RDX: ffff8f7c885e9f60 RSI: ffff8f7c885db2a0 RDI: ffff8f7c885db2a0 > RBP: ffff8f74d0fa1000 R08: 0000000000000000 R09: ffff9f2180863748 > R10: ffff9f2180863740 R11: 0000000000000000 R12: ffff8f758edd8e00 > R13: 0000000000012800 R14: ffffdef70143e840 R15: ffff8f758edd8e08 > FS: 0000000000000000(0000) GS:ffff8f7c88400000(0000) knlGS:00000000000= 00000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 000037203acb1000 CR3: 00000001d5c28000 CR4: 0000000000350ee0 > note: kswapd0[263] exited with preempt_count 2 >=20 >=20 > full kernel log: https://pastebin.com/FL1fZLJ0 >=20 > --=20 > Best Regards, > Mike Gavrilov. The comment below shows a race instance, though I failed to put things together to see how within two hours. Cut it and see what will come up. --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -1129,19 +1129,22 @@ retry: page =3D NULL; if (can_sleep) { spin_lock(&pool->stale_lock); + spin_lock(&pool->lock); zhdr =3D list_first_entry_or_null(&pool->stale, struct z3fold_header, buddy); /* - * Before allocating a page, let's see if we can take one from + * Before allocating a page, lets see if we can take one from * the stale pages list. cancel_work_sync() can sleep so we * limit this case to the contexts where we can sleep */ if (zhdr) { list_del(&zhdr->buddy); + spin_unlock(&pool->lock); spin_unlock(&pool->stale_lock); cancel_work_sync(&zhdr->work); page =3D virt_to_page(zhdr); } else { + spin_unlock(&pool->lock); spin_unlock(&pool->stale_lock); } }