From: "Theodore Y. Ts'o" <tytso@mit.edu>
To: Matthew Bobrowski <mbobrowski@mbobrowski.org>
Cc: jack@suse.cz, adilger.kernel@dilger.ca,
linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org,
hch@infradead.org, david@fromorbit.com, darrick.wong@oracle.com
Subject: Re: [PATCH v5 00/12] ext4: port direct I/O to iomap infrastructure
Date: Mon, 21 Oct 2019 09:31:12 -0400 [thread overview]
Message-ID: <20191021133111.GA4675@mit.edu> (raw)
In-Reply-To: <cover.1571647178.git.mbobrowski@mbobrowski.org>
Hi Matthew, thanks for your work on this patch series!
I applied it against 4c3, and ran a quick test run on it, and found
the following locking problem. To reproduce:
kvm-xfstests -c nojournal generic/113
generic/113 [09:27:19][ 5.841937] run fstests generic/113 at 2019-10-21 09:27:19
[ 7.959477]
[ 7.959798] ============================================
[ 7.960518] WARNING: possible recursive locking detected
[ 7.961225] 5.4.0-rc3-xfstests-00012-g7fe6ea084e48 #1238 Not tainted
[ 7.961991] --------------------------------------------
[ 7.962569] aio-stress/1516 is trying to acquire lock:
[ 7.963129] ffff9fd4791148c8 (&sb->s_type->i_mutex_key#12){++++}, at: __generic_file_fsync+0x3e/0xb0
[ 7.964109]
[ 7.964109] but task is already holding lock:
[ 7.964740] ffff9fd4791148c8 (&sb->s_type->i_mutex_key#12){++++}, at: ext4_dio_write_iter+0x15b/0x430
[ 7.965763]
[ 7.965763] other info that might help us debug this:
[ 7.966630] Possible unsafe locking scenario:
[ 7.966630]
[ 7.967424] CPU0
[ 7.967760] ----
[ 7.968097] lock(&sb->s_type->i_mutex_key#12);
[ 7.968827] lock(&sb->s_type->i_mutex_key#12);
[ 7.969558]
[ 7.969558] *** DEADLOCK ***
[ 7.969558]
[ 7.970518] May be due to missing lock nesting notation
[ 7.970518]
[ 7.971592] 1 lock held by aio-stress/1516:
[ 7.972267] #0: ffff9fd4791148c8 (&sb->s_type->i_mutex_key#12){++++}, at: ext4_dio_write_iter+0x15b/0x430
[ 7.973807]
[ 7.973807] stack backtrace:
[ 7.974510] CPU: 0 PID: 1516 Comm: aio-stress Not tainted 5.4.0-rc3-xfstests-00012-g7fe6ea084e48 #1238
[ 7.976053] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 7.977327] Call Trace:
[ 7.977700] dump_stack+0x67/0x90
[ 7.978198] __lock_acquire.cold+0x130/0x1f7
[ 7.978829] ? __switch_to_asm+0x40/0x70
[ 7.979659] lock_acquire+0x9a/0x160
[ 7.980320] ? __generic_file_fsync+0x3e/0xb0
[ 7.981014] down_write+0x40/0x110
[ 7.981717] ? __generic_file_fsync+0x3e/0xb0
[ 7.982676] __generic_file_fsync+0x3e/0xb0
[ 7.983454] ext4_sync_file+0x277/0x4e0
[ 7.984188] iomap_dio_complete+0x112/0x130
[ 7.984971] ? iomap_dio_rw+0x3a0/0x4b0
[ 7.985647] iomap_dio_rw+0x419/0x4b0
[ 7.986317] ? ext4_dio_write_iter+0x296/0x430
[ 7.987039] ext4_dio_write_iter+0x296/0x430
[ 7.987786] aio_write+0xef/0x1c0
[ 7.988284] ? kvm_sched_clock_read+0x14/0x30
[ 7.988822] ? sched_clock+0x5/0x10
[ 7.989234] ? sched_clock_cpu+0xc/0xc0
[ 7.989719] __io_submit_one.constprop.0+0x399/0x5f0
[ 7.990315] ? kvm_sched_clock_read+0x14/0x30
[ 7.990917] ? sched_clock+0x5/0x10
[ 7.991473] ? sched_clock_cpu+0xc/0xc0
[ 7.992097] ? io_submit_one+0x141/0x5a0
[ 7.992741] io_submit_one+0x141/0x5a0
[ 7.993354] __x64_sys_io_submit+0x9a/0x290
[ 7.993853] ? do_syscall_64+0x50/0x1b0
[ 7.994250] ? __ia32_sys_io_destroy+0x10/0x10
[ 7.994748] do_syscall_64+0x50/0x1b0
[ 7.995175] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 7.995761] RIP: 0033:0x55d1268c2d17
[ 7.996270] Code: 00 75 08 8b 47 0c 39 47 08 74 08 e9 b3 ff ff ff 0f 1f 00 31 c0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 b8 d1 00 00 00 0f 05 <c3> 0f 1f 84 00 00 00 00 00 48 63 ff b8 ce 00 00 00 0f 05 c3 0f 1f
[ 7.999131] RSP: 002b:00007f090fb0bd88 EFLAGS: 00000202 ORIG_RAX: 00000000000000d1
[ 7.999994] RAX: ffffffffffffffda RBX: 000055d128135010 RCX: 000055d1268c2d17
[ 8.000881] RDX: 000055d128135010 RSI: 0000000000000008 RDI: 00007f0907263000
[ 8.001765] RBP: 000055d128129560 R08: 00007fff421ae080 R09: 00007f090fb0bd68
[ 8.002824] R10: 00007f090fb0bd60 R11: 0000000000000202 R12: 0000000000000008
[ 8.004016] R13: 00007f090fb0bdb0 R14: 00007f090fb0bda0 R15: 000055d128129560
I'm other test configurations are still running, and I haven't had a
chance to do a detailed review on it, but I'll try to get to it this
week.
Thanks,
- Ted
next prev parent reply other threads:[~2019-10-21 13:31 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-21 9:17 [PATCH v5 00/12] ext4: port direct I/O to iomap infrastructure Matthew Bobrowski
2019-10-21 9:17 ` [PATCH v5 01/12] ext4: move set iomap routines into separate helper ext4_set_iomap() Matthew Bobrowski
2019-10-21 13:23 ` Jan Kara
2019-10-23 6:31 ` Ritesh Harjani
2019-10-23 10:14 ` Matthew Bobrowski
2019-10-21 9:17 ` [PATCH v5 02/12] ext4: iomap that extends beyond EOF should be marked dirty Matthew Bobrowski
2019-10-21 13:28 ` Jan Kara
2019-10-22 1:49 ` Matthew Bobrowski
2019-10-23 6:35 ` Ritesh Harjani
2019-10-23 10:20 ` Matthew Bobrowski
2019-10-21 9:18 ` [PATCH v5 03/12] ext4: split IOMAP_WRITE branch in ext4_iomap_begin() into helper Matthew Bobrowski
2019-10-21 13:31 ` Jan Kara
2019-10-23 6:37 ` Ritesh Harjani
2019-10-21 9:18 ` [PATCH v5 04/12] ext4: introduce new callback for IOMAP_REPORT Matthew Bobrowski
2019-10-21 13:37 ` Jan Kara
2019-10-22 1:55 ` Matthew Bobrowski
2019-10-23 6:39 ` Ritesh Harjani
2019-10-23 10:35 ` Matthew Bobrowski
2019-10-21 9:18 ` [PATCH v5 05/12] iomap: Allow forcing of waiting for running DIO in iomap_dio_rw() mbobrowski
2019-10-24 1:41 ` Christoph Hellwig
2019-10-24 11:17 ` Matthew Bobrowski
2019-10-21 9:18 ` [PATCH v5 06/12] xfs: Use iomap_dio_rw_wait() mbobrowski
2019-10-21 13:38 ` Jan Kara
2019-10-21 9:18 ` [PATCH v5 07/12] ext4: introduce direct I/O read using iomap infrastructure Matthew Bobrowski
2019-10-21 13:41 ` Jan Kara
2019-10-22 1:58 ` Matthew Bobrowski
2019-10-23 6:40 ` Ritesh Harjani
2019-10-21 9:18 ` [PATCH v5 08/12] ext4: update direct I/O read to do trylock in IOCB_NOWAIT cases Matthew Bobrowski
2019-10-21 13:48 ` Jan Kara
2019-10-22 2:04 ` Matthew Bobrowski
2019-10-22 7:50 ` Jan Kara
2019-10-23 6:51 ` Ritesh Harjani
2019-10-21 9:18 ` [PATCH v5 09/12] ext4: move inode extension/truncate code out from ->iomap_end() callback Matthew Bobrowski
2019-10-21 13:53 ` Jan Kara
2019-10-22 2:07 ` Matthew Bobrowski
2019-10-21 9:20 ` [PATCH v5 12/12] ext4: introduce direct I/O write using iomap infrastructure Matthew Bobrowski
2019-10-21 16:18 ` Jan Kara
2019-10-22 3:02 ` Matthew Bobrowski
2019-10-22 7:55 ` Jan Kara
2019-10-21 9:20 ` [PATCH v5 11/12] ext4: reorder map->m_flags checks in ext4_set_iomap() Matthew Bobrowski
2019-10-21 9:20 ` [PATCH v5 10/12] ext4: move inode extension check out from ext4_iomap_alloc() Matthew Bobrowski
2019-10-21 13:31 ` Theodore Y. Ts'o [this message]
2019-10-21 19:43 ` [PATCH v5 00/12] ext4: port direct I/O to iomap infrastructure Jan Kara
2019-10-21 22:38 ` Dave Chinner
2019-10-22 8:01 ` Jan Kara
2019-10-23 2:35 ` Matthew Bobrowski
2019-10-23 10:01 ` Jan Kara
2019-10-23 10:11 ` Matthew Bobrowski
2019-10-24 1:58 ` Christoph Hellwig
2019-10-24 11:09 ` Matthew Bobrowski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191021133111.GA4675@mit.edu \
--to=tytso@mit.edu \
--cc=adilger.kernel@dilger.ca \
--cc=darrick.wong@oracle.com \
--cc=david@fromorbit.com \
--cc=hch@infradead.org \
--cc=jack@suse.cz \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=mbobrowski@mbobrowski.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.