From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD905ECAAA1 for ; Thu, 15 Sep 2022 20:39:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E8CC8D0002; Thu, 15 Sep 2022 16:39:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 198846B0073; Thu, 15 Sep 2022 16:39:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 060918D0002; Thu, 15 Sep 2022 16:39:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E70506B0072 for ; Thu, 15 Sep 2022 16:39:34 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BEBB51206FE for ; Thu, 15 Sep 2022 20:39:34 +0000 (UTC) X-FDA: 79915485468.22.6F1B0F3 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf04.hostedemail.com (Postfix) with ESMTP id 65D8F400A5 for ; Thu, 15 Sep 2022 20:39:34 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 687B5624EF; Thu, 15 Sep 2022 20:39:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A29C4C433D6; Thu, 15 Sep 2022 20:39:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1663274372; bh=vCa9ZEg+OCiw251DDGL9LH6MdofafNi/D4iZURD1y2A=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ffpVggv9WVbMvhReRmKgaG7utG7E5wDUKZ20YJbrQFyl+mfdgO5UqNZsRHfYOyQLT Ws7phQXzbbuS6NHuaD6zq4ZVfV1trPsoG+pn6k/v2XY7ABshKLPSg4KCJen6Sid0rB vbUVjXBzGae1GA+mKviy0kYCCEPGREmxV1PQMzGg= Date: Thu, 15 Sep 2022 13:39:31 -0700 From: Andrew Morton To: dev@der-flo.net, linux-mm@kvack.org, Uladzislau Rezki Cc: bugzilla-daemon@kernel.org Subject: Re: [Bug 216489] New: Machine freezes due to memory lock Message-Id: <20220915133931.ee0a6c8a86c59a144828eb60@linux-foundation.org> In-Reply-To: References: X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ffpVggv9; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663274374; a=rsa-sha256; cv=none; b=6JoYb/OVU+BmMKyjdAwFt0EwqMbNyieQFFmmaG+K5+zudkFanzttv7BkSrLs62h51sAbBH in9yAp/OeNmEcHR721QzLftSBOFvCriawkihXFOyWKr0eqPhGtpCRFM/IQw/EUqBugrVIY YR96KAsWZeW7DfxScmr13Sk0jBB30XQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663274374; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NuBFSl08/I4XwFWS84hoTnqCPXYBXjZJzT07+3q+zY0=; b=l8Z7P6XaDpPWLlS39GEWvkfVuVQ4JwbvWPj6Dwj6RJVwremsf0scsiRSp5Qpyt2Y3DWbn0 EaJUwgSU7mchb7xShUqkSG3rSbnTkD6FduikZ5NT0JpCMi1/IsAVV9qw8YJyeyNnnShJ8+ 0V2XeraI2pBBCxKQDCjmTJqNrXeu+y0= X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 65D8F400A5 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ffpVggv9; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: ij4xqirzxwtgmq4tg3c66dh3upg7u5hi X-HE-Tag: 1663274374-968990 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: (switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Wed, 14 Sep 2022 15:07:46 +0000 bugzilla-daemon@kernel.org wrote: > https://bugzilla.kernel.org/show_bug.cgi?id=216489 > > Bug ID: 216489 > Summary: Machine freezes due to memory lock > Product: Memory Management > Version: 2.5 > Kernel Version: 5.19.8 > Hardware: AMD > OS: Linux > Tree: Mainline > Status: NEW > Severity: high > Priority: P1 > Component: Other > Assignee: akpm@linux-foundation.org > Reporter: dev@der-flo.net > Regression: No > > Hi all, > With Kernel 5.19.x we noticed system freezes. This happens in virtual > environments as well as on real hardware. > On a real hardware machine we were able to catch the moment of freeze with > continuous profiling. Thanks. I forwarded this to Uladzislau and he offered to help. He said: : I can help with debugging. What i need is reproduce steps. Could you : please clarify if it is easy to hit and what kind of profiling triggers it? and : I do not think that Matthew Wilcox commits destroys it but... I see that : __vunmap() is invoked by the free_work() thus a caller is in atomic : context including IRQ context. > Specification of the machine where we captured the freeze: > Thinkpad T14 > CPU: AMD Ryzen 7 PRO 4750U > Kernel: 5.19.8-200.fc36.x86_64 > > Stacktrace of kworker/12:3 that is using all resources and causing the freeze: > > # Source Location Function Name Function Line > 0 arch/x86/include/asm/vdso/processor.h:13 rep_nop 11 > 1 arch/x86/include/asm/vdso/processor.h:18 cpu_relax 16 > 2 kernel/locking/qspinlock.c:514 native_queued_spin_lock_slowpath > 316 > 3 kernel/locking/qspinlock.c:316 native_queued_spin_lock_slowpath > N/A > 4 arch/x86/include/asm/paravirt.h:591 pv_queued_spin_lock_slowpath > 588 > 5 arch/x86/include/asm/qspinlock.h:51 queued_spin_lock_slowpath 49 > 6 include/asm-generic/qspinlock.h:114 queued_spin_lock 107 > 7 include/linux/spinlock.h:185 do_raw_spin_lock 182 > 8 include/linux/spinlock_api_smp.h:134 __raw_spin_lock 130 > 9 kernel/locking/spinlock.c:154 _raw_spin_lock 152 > 10 include/linux/spinlock.h:349 spin_lock 347 > 11 mm/vmalloc.c:1805 find_vmap_area 1801 > 12 mm/vmalloc.c:2525 find_vm_area 2521 > 13 mm/vmalloc.c:2639 __vunmap 2628 > 14 mm/vmalloc.c:97 free_work 91 > 15 kernel/workqueue.c:2289 process_one_work 2181 > 16 kernel/workqueue.c:2436 worker_thread 2378 > 17 kernel/kthread.c:376 kthread 330 > 18 N/A ret_from_fork N/A > > The functions in the above shown stacktrace hardly change. There is only one > commit 993d0b287e2ef7bee2e8b13b0ce4d2b5066f278e which introduces changes to > find_vmap_area() for 5.19. > > With this change in mind we looked for stacktraces which make also use of this > new commit. And in a different kernel thread we do notice the use of > check_heap_object(): > > # Source Location Function Name Function Line > 0 arch/x86/include/asm/paravirt.h:704 arch_local_irq_enable 702 > 1 arch/x86/include/asm/irqflags.h:138 arch_local_irq_restore 135 > 2 kernel/sched/sched.h:1330 raw_spin_rq_unlock_irqrestore 1327 > 3 kernel/sched/sched.h:1327 raw_spin_rq_unlock_irqrestore N/A > 4 kernel/sched/sched.h:1611 rq_unlock_irqrestore 1607 > 5 kernel/sched/fair.c:8288 update_blocked_averages 8272 > 6 kernel/sched/fair.c:11133 run_rebalance_domains 11115 > 7 kernel/softirq.c:571 __do_softirq 528 > 8 kernel/softirq.c:445 invoke_softirq 433 > 9 kernel/softirq.c:650 __irq_exit_rcu 640 > 10 arch/x86/kernel/apic/apic.c:1106 sysvec_apic_timer_interrupt N/A > 11 N/A asm_sysvec_apic_timer_interrupt N/A > 12 include/linux/mmzone.h:1403 __nr_to_section 1395 > 13 include/linux/mmzone.h:1488 __pfn_to_section 1486 > 14 include/linux/mmzone.h:1539 pfn_valid 1524 > 15 arch/x86/mm/physaddr.c:65 __virt_addr_valid 47 > 16 mm/usercopy.c:188 check_heap_object 161 > 17 mm/usercopy.c:250 __check_object_size 212 > 18 mm/usercopy.c:212 __check_object_size N/A > 19 include/linux/thread_info.h:199 check_object_size 195 > 20 lib/strncpy_from_user.c:137 strncpy_from_user 113 > 21 fs/namei.c:150 getname_flags 129 > 22 fs/namei.c:2896 user_path_at_empty 2893 > 23 include/linux/namei.h:57 user_path_at 54 > 24 fs/open.c:446 do_faccessat 420 > 25 arch/x86/entry/common.c:50 do_syscall_x64 40 > 26 arch/x86/entry/common.c:80 do_syscall_64 73 > 27 N/A entry_SYSCALL_64_after_hwframe N/A > > We are neither experts in the mm subsystem nor can provide a fix, but wanted to > let you know about our findings. > > Cheers, > Florian > > -- > You may reply to this email to add a comment. > > You are receiving this mail because: > You are the assignee for the bug.