From: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
To: Yu Zhao <yuzhao@google.com>,
Andrew Morton <akpm@linux-foundation.org>,
Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andi Kleen <ak@linux.intel.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
Hillf Danton <hdanton@sina.com>, Jens Axboe <axboe@kernel.dk>,
Jesse Barnes <jsbarnes@google.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Jonathan Corbet <corbet@lwn.net>,
Matthew Wilcox <willy@infradead.org>,
Mel Gorman <mgorman@suse.de>,
Michael Larabel <Michael@michaellarabel.com>,
Michal Hocko <mhocko@kernel.org>, Rik van Riel <riel@surriel.com>,
Vlastimil Babka <vbabka@suse.cz>, Will Deacon <will@kernel.org>,
Ying Huang <ying.huang@intel.com>,
linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
page-reclaim@google.com, x86@kernel.org,
Yu Zhao <yuzhao@google.com>,
Konstantin Kharlamov <Hi-Angel@yandex.ru>
Subject: Re: [PATCH v6 8/9] mm: multigenerational lru: user interface
Date: Thu, 13 Jan 2022 16:01:31 +0530 [thread overview]
Message-ID: <87a6g0nczg.fsf@linux.ibm.com> (raw)
In-Reply-To: <20220104202227.2903605-9-yuzhao@google.com>
Yu Zhao <yuzhao@google.com> writes:
> Add /sys/kernel/mm/lru_gen/enabled as a runtime kill switch.
Got the below lockdep warning while using the above kill/enable switch
[ 84.252952] ======================================================
[ 84.253012] WARNING: possible circular locking dependency detected
[ 84.253074] 5.16.0-rc8-16204-g1cdcf1120b31 #511 Not tainted
[ 84.253135] ------------------------------------------------------
[ 84.253194] bash/2862 is trying to acquire lock:
[ 84.253243] c0000000021ff740 (cgroup_mutex){+.+.}-{3:3}, at: store_enable+0x80/0x1510
[ 84.253340]
but task is already holding lock:
[ 84.253410] c000000002221348 (mem_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x30/0x50
[ 84.253503]
which lock already depends on the new lock.
[ 84.253608]
the existing dependency chain (in reverse order) is:
[ 84.253693]
-> #2 (mem_hotplug_lock){++++}-{0:0}:
[ 84.253768] lock_acquire+0x134/0x4a0
[ 84.253821] percpu_down_write+0x80/0x1c0
[ 84.253872] try_online_node+0x40/0x90
[ 84.253924] cpu_up+0x7c/0x160
[ 84.253976] bringup_nonboot_cpus+0xc4/0x120
[ 84.254027] smp_init+0x48/0xd4
[ 84.254079] kernel_init_freeable+0x274/0x45c
[ 84.254134] kernel_init+0x44/0x194
[ 84.254188] ret_from_kernel_thread+0x5c/0x64
[ 84.254241]
-> #1 (cpu_hotplug_lock){++++}-{0:0}:
[ 84.254321] lock_acquire+0x134/0x4a0
[ 84.254373] cpus_read_lock+0x6c/0x180
[ 84.254426] static_key_disable+0x24/0x50
[ 84.254477] rebind_subsystems+0x3b0/0x5a0
[ 84.254528] cgroup_setup_root+0x24c/0x530
[ 84.254581] cgroup1_get_tree+0x7d8/0xb80
[ 84.254638] vfs_get_tree+0x48/0x150
[ 84.254695] path_mount+0x8b8/0xd20
[ 84.254752] do_mount+0xb8/0xe0
[ 84.254808] sys_mount+0x250/0x390
[ 84.254863] system_call_exception+0x15c/0x2b0
[ 84.254932] system_call_common+0xec/0x250
[ 84.254989]
-> #0 (cgroup_mutex){+.+.}-{3:3}:
[ 84.255072] check_prev_add+0x180/0x1050
[ 84.255129] __lock_acquire+0x17b8/0x25c0
[ 84.255186] lock_acquire+0x134/0x4a0
[ 84.255243] __mutex_lock+0xdc/0xa90
[ 84.255300] store_enable+0x80/0x1510
[ 84.255356] kobj_attr_store+0x2c/0x50
[ 84.255413] sysfs_kf_write+0x6c/0xb0
[ 84.255471] kernfs_fop_write_iter+0x1bc/0x2b0
[ 84.255539] new_sync_write+0x130/0x1d0
[ 84.255594] vfs_write+0x2cc/0x4c0
[ 84.255645] ksys_write+0x84/0x140
[ 84.255699] system_call_exception+0x15c/0x2b0
[ 84.255771] system_call_common+0xec/0x250
[ 84.255829]
other info that might help us debug this:
[ 84.255933] Chain exists of:
cgroup_mutex --> cpu_hotplug_lock --> mem_hotplug_lock
[ 84.256070] Possible unsafe locking scenario:
[ 84.256149] CPU0 CPU1
[ 84.256201] ---- ----
[ 84.256255] lock(mem_hotplug_lock);
[ 84.256311] lock(cpu_hotplug_lock);
[ 84.256380] lock(mem_hotplug_lock);
[ 84.256448] lock(cgroup_mutex);
[ 84.256491]
*** DEADLOCK ***
[ 84.256571] 5 locks held by bash/2862:
[ 84.256626] #0: c00000002043d460 (sb_writers#5){.+.+}-{0:0}, at: ksys_write+0x84/0x140
[ 84.256728] #1: c00000004bafc888 (&of->mutex){+.+.}-{3:3}, at: kernfs_fop_write_iter+0x178/0x2b0
[ 84.256830] #2: c000000020b993b8 (kn->active#207){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x184/0x2b0
[ 84.256942] #3: c0000000020e5cd0 (cpu_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x20/0x50
[ 84.257045] #4: c000000002221348 (mem_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x30/0x50
[ 84.257152]
stack backtrace:
[ 84.257220] CPU: 107 PID: 2862 Comm: bash Not tainted 5.16.0-rc8-16204-g1cdcf1120b31 #511
[ 84.257309] Call Trace:
[ 84.257346] [c000000040d5b4a0] [c000000000a89f94] dump_stack_lvl+0x98/0xe0 (unreliable)
[ 84.257438] [c000000040d5b4e0] [c000000000267244] print_circular_bug.isra.0+0x3b4/0x3e0
[ 84.257528] [c000000040d5b580] [c0000000002673e0] check_noncircular+0x170/0x1a0
[ 84.257605] [c000000040d5b650] [c000000000268be0] check_prev_add+0x180/0x1050
[ 84.257683] [c000000040d5b710] [c00000000026ca48] __lock_acquire+0x17b8/0x25c0
[ 84.257760] [c000000040d5b840] [c00000000026e4c4] lock_acquire+0x134/0x4a0
[ 84.257837] [c000000040d5b940] [c00000000148a53c] __mutex_lock+0xdc/0xa90
[ 84.257914] [c000000040d5ba60] [c0000000004d5080] store_enable+0x80/0x1510
[ 84.257989] [c000000040d5bbc0] [c000000000a9286c] kobj_attr_store+0x2c/0x50
[ 84.258066] [c000000040d5bbe0] [c000000000752c4c] sysfs_kf_write+0x6c/0xb0
[ 84.258143] [c000000040d5bc20] [c000000000750fcc] kernfs_fop_write_iter+0x1bc/0x2b0
[ 84.258219] [c000000040d5bc70] [c000000000615df0] new_sync_write+0x130/0x1d0
[ 84.258295] [c000000040d5bd10] [c00000000061997c] vfs_write+0x2cc/0x4c0
[ 84.258373] [c000000040d5bd60] [c000000000619d54] ksys_write+0x84/0x140
[ 84.258450] [c000000040d5bdb0] [c00000000002c91c] system_call_exception+0x15c/0x2b0
[ 84.258528] [c000000040d5be10] [c00000000000c64c] system_call_common+0xec/0x250
[ 84.258604] --- interrupt: c00 at 0x79c551e76554
[ 84.258658] NIP: 000079c551e76554 LR: 000079c551de2674 CTR: 0000000000000000
[ 84.258732] REGS: c000000040d5be80 TRAP: 0c00 Not tainted (5.16.0-rc8-16204-g1cdcf1120b31)
[ 84.258817] MSR: 800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE> CR: 28422428 XER: 00000000
[ 84.258931] IRQMASK: 0
GPR00: 0000000000000004 00007fffc8e9a320 000079c551f77100 0000000000000001
GPR04: 0000017190973cc0 0000000000000002 0000000000000010 0000017190973cc0
GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR12: 0000000000000000 000079c5520ab1d0 0000017190943590 000001718749b738
GPR16: 00000171873b0ae0 0000000000000000 0000000020000000 0000017190973a60
GPR20: 0000000000000000 0000000000000001 0000017187443ca0 00007fffc8e9a514
GPR24: 00007fffc8e9a510 000001718749b0d0 000079c551f719d8 000079c551f72308
GPR28: 0000000000000002 000079c551f717e8 0000017190973cc0 0000000000000002
[ 84.259600] NIP [000079c551e76554] 0x79c551e76554
[ 84.259651] LR [000079c551de2674] 0x79c551de2674
[ 84.259701] --- interrupt: c00
next prev parent reply other threads:[~2022-01-13 10:33 UTC|newest]
Thread overview: 111+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-04 20:22 [PATCH v6 0/9] Multigenerational LRU Framework Yu Zhao
2022-01-04 20:22 ` [PATCH v6 1/9] mm: x86, arm64: add arch_has_hw_pte_young() Yu Zhao
2022-01-05 10:45 ` Will Deacon
2022-01-05 20:47 ` Yu Zhao
2022-01-06 10:30 ` Will Deacon
2022-01-07 7:25 ` Yu Zhao
2022-01-11 14:19 ` Will Deacon
2022-01-11 22:27 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 2/9] mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG Yu Zhao
2022-01-04 21:24 ` Linus Torvalds
2022-01-04 20:22 ` [PATCH v6 3/9] mm/vmscan.c: refactor shrink_node() Yu Zhao
2022-01-04 20:22 ` [PATCH v6 4/9] mm: multigenerational lru: groundwork Yu Zhao
2022-01-04 21:34 ` Linus Torvalds
2022-01-11 8:16 ` Aneesh Kumar K.V
2022-01-12 2:16 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 5/9] mm: multigenerational lru: mm_struct list Yu Zhao
2022-01-07 9:06 ` Michal Hocko
2022-01-08 0:19 ` Yu Zhao
2022-01-10 15:21 ` Michal Hocko
2022-01-12 8:08 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 6/9] mm: multigenerational lru: aging Yu Zhao
2022-01-06 16:06 ` Michal Hocko
2022-01-06 21:27 ` Yu Zhao
2022-01-07 8:43 ` Michal Hocko
2022-01-07 21:12 ` Yu Zhao
2022-01-06 16:12 ` Michal Hocko
2022-01-06 21:41 ` Yu Zhao
2022-01-07 8:55 ` Michal Hocko
2022-01-07 9:00 ` Michal Hocko
2022-01-10 3:58 ` Yu Zhao
2022-01-10 14:37 ` Michal Hocko
2022-01-13 9:43 ` Yu Zhao
2022-01-13 12:02 ` Michal Hocko
2022-01-19 6:31 ` Yu Zhao
2022-01-19 9:44 ` Michal Hocko
2022-01-10 15:01 ` Michal Hocko
2022-01-10 16:01 ` Vlastimil Babka
2022-01-10 16:25 ` Michal Hocko
2022-01-11 23:16 ` Yu Zhao
2022-01-12 10:28 ` Michal Hocko
2022-01-13 9:25 ` Yu Zhao
2022-01-07 13:11 ` Michal Hocko
2022-01-07 23:36 ` Yu Zhao
2022-01-10 15:35 ` Michal Hocko
2022-01-11 1:18 ` Yu Zhao
2022-01-11 9:00 ` Michal Hocko
[not found] ` <1641900108.61dd684cb0e59@mail.inbox.lv>
2022-01-11 12:15 ` Michal Hocko
2022-01-13 17:00 ` Alexey Avramov
2022-01-11 14:22 ` Alexey Avramov
2022-01-07 14:44 ` Michal Hocko
2022-01-10 4:47 ` Yu Zhao
2022-01-10 10:54 ` Michal Hocko
2022-01-19 7:04 ` Yu Zhao
2022-01-19 9:42 ` Michal Hocko
2022-01-23 21:28 ` Yu Zhao
2022-01-24 14:01 ` Michal Hocko
2022-01-10 16:57 ` Michal Hocko
2022-01-12 1:01 ` Yu Zhao
2022-01-12 10:17 ` Michal Hocko
2022-01-12 23:43 ` Yu Zhao
2022-01-13 11:57 ` Michal Hocko
2022-01-23 21:40 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 7/9] mm: multigenerational lru: eviction Yu Zhao
2022-01-11 10:37 ` Aneesh Kumar K.V
2022-01-12 8:05 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 8/9] mm: multigenerational lru: user interface Yu Zhao
2022-01-10 10:27 ` Mike Rapoport
2022-01-12 8:35 ` Yu Zhao
2022-01-12 10:31 ` Michal Hocko
2022-01-12 15:45 ` Mike Rapoport
2022-01-13 9:47 ` Yu Zhao
2022-01-13 10:31 ` Aneesh Kumar K.V [this message]
2022-01-13 23:02 ` Yu Zhao
2022-01-14 5:20 ` Aneesh Kumar K.V
2022-01-14 6:50 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 9/9] mm: multigenerational lru: Kconfig Yu Zhao
2022-01-04 21:39 ` Linus Torvalds
2022-01-04 20:22 ` [PATCH v6 0/9] Multigenerational LRU Framework Yu Zhao
2022-01-04 20:30 ` Yu Zhao
2022-01-04 21:43 ` Linus Torvalds
2022-01-05 21:12 ` Yu Zhao
2022-01-07 9:38 ` Michal Hocko
2022-01-07 18:45 ` Yu Zhao
2022-01-10 15:39 ` Michal Hocko
2022-01-10 22:04 ` Yu Zhao
2022-01-10 22:46 ` Jesse Barnes
2022-01-11 1:41 ` Linus Torvalds
2022-01-11 10:40 ` Michal Hocko
2022-01-11 8:41 ` Yu Zhao
2022-01-11 8:53 ` Holger Hoffstätte
2022-01-11 16:04 ` Shuang Zhai
2022-01-12 1:46 ` Suleiman Souhlal
2022-01-12 6:07 ` Sofia Trinh
2022-01-18 9:21 ` Yu Zhao
2022-01-18 9:36 ` Donald Carr
2022-01-19 20:19 ` Steven Barrett
2022-01-19 22:25 ` Brian Geffon
2022-01-05 2:44 ` Shuang Zhai
2022-01-05 8:55 ` SeongJae Park
2022-01-05 10:53 ` Yu Zhao
2022-01-05 11:12 ` Borislav Petkov
2022-01-05 11:25 ` SeongJae Park
2022-01-05 21:06 ` Yu Zhao
2022-01-10 14:49 ` Alexey Avramov
2022-01-11 10:24 ` Alexey Avramov
2022-01-12 20:56 ` Oleksandr Natalenko
2022-01-13 8:59 ` Yu Zhao
2022-01-23 5:43 ` Barry Song
2022-01-25 6:48 ` Yu Zhao
2022-01-28 8:54 ` Barry Song
2022-02-08 9:16 ` Yu Zhao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87a6g0nczg.fsf@linux.ibm.com \
--to=aneesh.kumar@linux.ibm.com \
--cc=Hi-Angel@yandex.ru \
--cc=Michael@michaellarabel.com \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=axboe@kernel.dk \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=hannes@cmpxchg.org \
--cc=hdanton@sina.com \
--cc=jsbarnes@google.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@kernel.org \
--cc=page-reclaim@google.com \
--cc=riel@surriel.com \
--cc=torvalds@linux-foundation.org \
--cc=vbabka@suse.cz \
--cc=will@kernel.org \
--cc=willy@infradead.org \
--cc=x86@kernel.org \
--cc=ying.huang@intel.com \
--cc=yuzhao@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).