From: Peter Zijlstra <peterz@infradead.org>
To: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Cc: Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
linux-block@vger.kernel.org,
Mike Galbraith <umgwanakikbuti@gmail.com>,
minchan@kernel.org, ngupta@vflare.org,
sergey.senozhatsky.work@gmail.com, bigeasy@linutronix.de
Subject: Re: swapon/913 is trying to acquire lock at zcomp_stream_get+0x5/0x90 [zram] but task is already holding lock at zs_map_object+0x7a/0x2e0
Date: Fri, 16 Oct 2020 14:40:09 +0200 [thread overview]
Message-ID: <20201016124009.GQ2611@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <CABXGCsOL0pW0Ghh-w5d12P75ve6FS9Rgmzm6DvsYbJY-jMTCdg@mail.gmail.com>
On Fri, Oct 16, 2020 at 11:21:47AM +0500, Mikhail Gavrilov wrote:
> Hi folks,
> today I joined to testing Kernel 5.10 and see that every boot happens
> this warning:
>
> [ 9.032096] ======================================================
> [ 9.032097] WARNING: possible circular locking dependency detected
> [ 9.032098] 5.10.0-0.rc0.20201014gitb5fc7a89e58b.41.fc34.x86_64 #1 Not tainted
> [ 9.032099] ------------------------------------------------------
> [ 9.032100] swapon/913 is trying to acquire lock:
> [ 9.032101] ffffc984fda4f948 (&zstrm->lock){+.+.}-{2:2}, at: zcomp_stream_get+0x5/0x90 [zram]
> [ 9.032106] but task is already holding lock:
> [ 9.032107] ffff993c54cdceb0 (&zspage->lock){.+.+}-{2:2}, at: zs_map_object+0x7a/0x2e0
> [ 9.032111] which lock already depends on the new lock.
> [ 9.032112] the existing dependency chain (in reverse order) is:
> [ 9.032112] -> #1 (&zspage->lock){.+.+}-{2:2}:
> [ 9.032116] _raw_read_lock+0x3d/0xa0
> [ 9.032118] zs_map_object+0x7a/0x2e0
> [ 9.032119] zram_bvec_rw.constprop.0.isra.0+0x287/0x730 [zram]
> [ 9.032121] zram_submit_bio+0x189/0x35d [zram]
> [ 9.032123] submit_bio_noacct+0xff/0x650
> [ 9.032124] submit_bh_wbc+0x17d/0x1a0
> [ 9.032126] __block_write_full_page+0x227/0x580
> [ 9.032128] __writepage+0x1a/0x70
> [ 9.032129] write_cache_pages+0x21c/0x540
> [ 9.032130] generic_writepages+0x41/0x60
> [ 9.032131] do_writepages+0x28/0xb0
> [ 9.032133] __filemap_fdatawrite_range+0xa7/0xe0
> [ 9.032134] file_write_and_wait_range+0x67/0xb0
> [ 9.032135] blkdev_fsync+0x17/0x40
> [ 9.032137] __x64_sys_fsync+0x34/0x60
> [ 9.032138] do_syscall_64+0x33/0x40
> [ 9.032140] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [ 9.032140]
> -> #0 (&zstrm->lock){+.+.}-{2:2}:
> [ 9.032169] 1 lock held by swapon/913:
> [ 9.032170] #0: ffff993c54cdceb0 (&zspage->lock){.+.+}-{2:2}, at: zs_map_object+0x7a/0x2e0
> [ 9.032176] Call Trace:
> [ 9.032179] dump_stack+0x8b/0xb0
> [ 9.032181] check_noncircular+0xd0/0xf0
> [ 9.032183] __lock_acquire+0x11e3/0x21f0
> [ 9.032185] lock_acquire+0xc8/0x400
> [ 9.032187] ? zcomp_stream_get+0x5/0x90 [zram]
> [ 9.032189] zcomp_stream_get+0x38/0x90 [zram]
> [ 9.032190] ? zcomp_stream_get+0x5/0x90 [zram]
> [ 9.032192] zram_bvec_rw.constprop.0.isra.0+0x4c1/0x730 [zram]
> [ 9.032194] ? __part_start_io_acct+0x4d/0xf0
> [ 9.032196] zram_rw_page+0xa9/0x130 [zram]
> [ 9.032197] bdev_read_page+0x71/0xa0
> [ 9.032199] do_mpage_readpage+0x5a8/0x800
> [ 9.032201] ? xa_load+0xbf/0x140
> [ 9.032203] mpage_readahead+0xfb/0x230
> [ 9.032205] ? bdev_evict_inode+0x1a0/0x1a0
> [ 9.032207] read_pages+0x60/0x1e0
> [ 9.032208] page_cache_readahead_unbounded+0x1da/0x270
> [ 9.032211] generic_file_buffered_read+0x69c/0xe00
> [ 9.032213] new_sync_read+0x108/0x180
> [ 9.032215] vfs_read+0x12e/0x1c0
> [ 9.032217] ksys_read+0x58/0xd0
> [ 9.032218] do_syscall_64+0x33/0x40
> [ 9.032219] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Joy... __zram_bvec_write() and __zram_bvec_read() take these locks in
opposite order.
Does something like the (_completely_) untested below cure things?
---
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 9100ac36670a..c1e2c2e1cde8 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -1216,10 +1216,11 @@ static void zram_free_page(struct zram *zram, size_t index)
static int __zram_bvec_read(struct zram *zram, struct page *page, u32 index,
struct bio *bio, bool partial_io)
{
- int ret;
+ struct zcomp_strm *zstrm;
unsigned long handle;
unsigned int size;
void *src, *dst;
+ int ret;
zram_slot_lock(zram, index);
if (zram_test_flag(zram, index, ZRAM_WB)) {
@@ -1250,6 +1251,9 @@ static int __zram_bvec_read(struct zram *zram, struct page *page, u32 index,
size = zram_get_obj_size(zram, index);
+ if (size != PAGE_SIZE)
+ zstrm = zcomp_stream_get(zram->comp);
+
src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO);
if (size == PAGE_SIZE) {
dst = kmap_atomic(page);
@@ -1257,8 +1261,6 @@ static int __zram_bvec_read(struct zram *zram, struct page *page, u32 index,
kunmap_atomic(dst);
ret = 0;
} else {
- struct zcomp_strm *zstrm = zcomp_stream_get(zram->comp);
-
dst = kmap_atomic(page);
ret = zcomp_decompress(zstrm, src, size, dst);
kunmap_atomic(dst);
next prev parent reply other threads:[~2020-10-16 12:40 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-16 6:21 swapon/913 is trying to acquire lock at zcomp_stream_get+0x5/0x90 [zram] but task is already holding lock at zs_map_object+0x7a/0x2e0 Mikhail Gavrilov
2020-10-16 12:40 ` Peter Zijlstra [this message]
2020-10-16 14:00 ` Mikhail Gavrilov
2020-10-16 15:33 ` Minchan Kim
2020-10-19 10:13 ` [PATCH] zram: Fix __zram_bvec_{read,write}() locking order Peter Zijlstra
2020-10-19 14:08 ` Minchan Kim
2020-10-19 15:29 ` Sebastian Andrzej Siewior
2020-10-19 15:32 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201016124009.GQ2611@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=bigeasy@linutronix.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mikhail.v.gavrilov@gmail.com \
--cc=minchan@kernel.org \
--cc=ngupta@vflare.org \
--cc=sergey.senozhatsky.work@gmail.com \
--cc=umgwanakikbuti@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).