* xfs/319 vs 1k block size file systems
@ 2021-07-19 10:20 Christoph Hellwig
2021-07-19 18:00 ` Darrick J. Wong
0 siblings, 1 reply; 5+ messages in thread
From: Christoph Hellwig @ 2021-07-19 10:20 UTC (permalink / raw)
To: linux-xfs
Hi all,
xfs/319 keeps crashing for me when running it for a 1k block size
file systems on x86 with debugging enabled. The problem is that
xfs_bmapi_remap is called on a range that is not a hole.
The (slightly garbled) dmesg is below:
xfs/319 files ... [ 46.469049] run fstests xfs/319 at 2021-07-19 10:17:12
[ 52.470572] XFS (vdc): Mounting V5 Filesystem
[ 52.478587] XFS (vdc): Ending clean mount
[ 52.482375] xfs filesystem being mounted at /mnt/scratch supports timestamps until 2038 (0x7ff)
[ 52.511583] XFS (vdc): Unmounting Filesystem
[ 53.901990] XFS (vdc): Mounting V5 Filesystem
[ 53.912316] XFS (vdc): Ending clean mount
[ 53.915388] xfs filesystem being mounted at /mnt/scratch supports timestamps until 2038 (0x7ff)
[ 54.067688] XFS (vdc): Injecting error (false) at file fs/xfs/libxfs/xfs_bmap.c, line 6256, on"
[ 54.070661] XFS (vdc): Corruption of in-memory data (0x8) detected at xfs_defer_finish_noroll+m
[ 54.072637] XFS (vdc): Please unmount the filesystem and rectify the problem(s)
[ 54.094779] XFS (vdc): Unmounting Filesystem
[ 54.334642] XFS (vdc): Mounting V5 Filesystem
[ 54.344167] XFS (vdc): Starting recovery (logdev: internal)
[ 54.351700] XFS: Assertion failed: got.br_startoff > bno, file: fs/xfs/libxfs/xfs_bmap.c, line5
[ 54.353014] ------------[ cut here ]------------
[ 54.353876] kernel BUG at fs/xfs/xfs_message.c:110!
[ 54.354655] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 54.355335] CPU: 2 PID: 4644 Comm: mount Not tainted 5.14.0-rc2+ #19
[ 54.356198] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
[ 54.357311] RIP: 0010:assfail+0x1e/0x23
[ 54.357831] Code: b8 2b 07 83 e8 85 fc ff ff 0f 0b c3 41 89 c8 48 89 d1 48 89 f2 48 c7 c6 b8 24
[ 54.360251] RSP: 0018:ffffc9000168fb58 EFLAGS: 00010202
[ 54.360934] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[ 54.361973] RDX: 00000000ffffffc0 RSI: 0000000000000000 RDI: ffffffff82fdbcf9
[ 54.362888] RBP: ffffc9000168fbf0 R08: 0000000000000000 R09: 000000000000000a
[ 54.363817] R10: 000000000000000a R11: f000000000000000 R12: ffff888112ba8240
[ 54.364736] R13: ffff888112a4c000 R14: 0000000000000000 R15: ffff888114dc9948
[ 54.365655] FS: 00007f65e38a7100(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
[ 54.366693] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 54.367452] CR2: 00007ffc6c1d9ff8 CR3: 0000000116668000 CR4: 00000000000006e0
[ 54.368352] Call Trace:
[ 54.368658] xfs_bmapi_remap+0x3bf/0x3f0
[ 54.369143] xfs_bmap_finish_one+0x186/0x260
[ 54.369666] xfs_bui_item_recover+0x266/0x370
[ 54.370200] ? lock_release+0x13c/0x2e0
[ 54.370667] xlog_recover_process_intents+0xd0/0x400
[ 54.371276] xlog_recover_finish+0x14/0xc0
[ 54.371837] xfs_log_mount_finish+0x54/0x1b0
[ 54.372503] xfs_mountfs+0x580/0x9b0
[ 54.373008] xfs_fs_fill_super+0x3aa/0x7e0
[ 54.373535] ? xfs_fs_put_super+0x90/0x90
[ 54.374118] get_tree_bdev+0x17a/0x280
[ 54.374645] vfs_get_tree+0x23/0xc0
[ 54.375089] path_mount+0x2b1/0xb40
[ 54.375542] __x64_sys_mount+0xfe/0x140
[ 54.376028] do_syscall_64+0x3b/0x90
[ 54.376482] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 54.377118] RIP: 0033:0x7f65e3aa5fea
[ 54.377575] Code: 48 8b 0d a9 0e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 08
[ 54.379859] RSP: 002b:00007ffc6c1db238 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
[ 54.380777] RAX: ffffffffffffffda RBX: 000055a56b0df970 RCX: 00007f65e3aa5fea
[ 54.381642] RDX: 000055a56b0dfb80 RSI: 000055a56b0dfbc0 RDI: 000055a56b0dfba0
[ 54.382503] RBP: 00007f65e3df31c4 R08: 0000000000000000 R09: 000055a56b0e2890
[ 54.383378] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[ 54.384254] R13: 0000000000000000 R14: 000055a56b0dfba0 R15: 000055a56b0dfb80
[ 54.385068] Modules linked in:
[ 54.385495] ---[ end trace 81c36780b0c4c649 ]---
[ 54.386033] RIP: 0010:assfail+0x1e/0x23
[ 54.386500] Code: b8 2b 07 83 e8 85 fc ff ff 0f 0b c3 41 89 c8 48 89 d1 48
89 f2 48 c7 c6 b8 24
[ 54.388948] RSP: 0018:ffffc9000168fb58 EFLAGS: 00010202
[ 54.389674] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[ 54.390612] RDX: 00000000ffffffc0 RSI: 0000000000000000 RDI: ffffffff82fdbcf9
[ 54.391534] RBP: ffffc9000168fbf0 R08: 0000000000000000 R09: 000000000000000a
[ 54.392494] R10: 000000000000000a R11: f000000000000000 R12: ffff888112ba8240
[ 54.393400] R13: ffff888112a4c000 R14: 0000000000000000 R15: ffff888114dc9948
[ 54.394400] FS: 00007f65e38a7100(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
[ 54.395763] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 54.396425] CR2: 00007ffc6c1d9ff8 CR3: 0000000116668000 CR4: 00000000000006e0
[ 54.397302] Kernel panic - not syncing: Fatal exception
[ 54.398143] Kernel Offset: disabled
[ 54.398660] ---[ end Kernel panic - not syncing: Fatal exception ]---
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: xfs/319 vs 1k block size file systems
2021-07-19 10:20 xfs/319 vs 1k block size file systems Christoph Hellwig
@ 2021-07-19 18:00 ` Darrick J. Wong
2021-07-19 18:21 ` Christoph Hellwig
0 siblings, 1 reply; 5+ messages in thread
From: Darrick J. Wong @ 2021-07-19 18:00 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-xfs, Dave Chinner
On Mon, Jul 19, 2021 at 12:20:54PM +0200, Christoph Hellwig wrote:
> Hi all,
>
> xfs/319 keeps crashing for me when running it for a 1k block size
> file systems on x86 with debugging enabled. The problem is that
> xfs_bmapi_remap is called on a range that is not a hole.
Hmm. Dave sent me a warning late last night about some sort of log
recovery problem in -rc2 due to FUA changes or something. Did this
happen in -rc1?
--D
> The (slightly garbled) dmesg is below:
>
> xfs/319 files ... [ 46.469049] run fstests xfs/319 at 2021-07-19 10:17:12
> [ 52.470572] XFS (vdc): Mounting V5 Filesystem
> [ 52.478587] XFS (vdc): Ending clean mount
> [ 52.482375] xfs filesystem being mounted at /mnt/scratch supports timestamps until 2038 (0x7ff)
> [ 52.511583] XFS (vdc): Unmounting Filesystem
> [ 53.901990] XFS (vdc): Mounting V5 Filesystem
> [ 53.912316] XFS (vdc): Ending clean mount
> [ 53.915388] xfs filesystem being mounted at /mnt/scratch supports timestamps until 2038 (0x7ff)
> [ 54.067688] XFS (vdc): Injecting error (false) at file fs/xfs/libxfs/xfs_bmap.c, line 6256, on"
> [ 54.070661] XFS (vdc): Corruption of in-memory data (0x8) detected at xfs_defer_finish_noroll+m
> [ 54.072637] XFS (vdc): Please unmount the filesystem and rectify the problem(s)
> [ 54.094779] XFS (vdc): Unmounting Filesystem
> [ 54.334642] XFS (vdc): Mounting V5 Filesystem
> [ 54.344167] XFS (vdc): Starting recovery (logdev: internal)
> [ 54.351700] XFS: Assertion failed: got.br_startoff > bno, file: fs/xfs/libxfs/xfs_bmap.c, line5
> [ 54.353014] ------------[ cut here ]------------
> [ 54.353876] kernel BUG at fs/xfs/xfs_message.c:110!
> [ 54.354655] invalid opcode: 0000 [#1] PREEMPT SMP PTI
> [ 54.355335] CPU: 2 PID: 4644 Comm: mount Not tainted 5.14.0-rc2+ #19
> [ 54.356198] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
> [ 54.357311] RIP: 0010:assfail+0x1e/0x23
> [ 54.357831] Code: b8 2b 07 83 e8 85 fc ff ff 0f 0b c3 41 89 c8 48 89 d1 48 89 f2 48 c7 c6 b8 24
> [ 54.360251] RSP: 0018:ffffc9000168fb58 EFLAGS: 00010202
> [ 54.360934] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
> [ 54.361973] RDX: 00000000ffffffc0 RSI: 0000000000000000 RDI: ffffffff82fdbcf9
> [ 54.362888] RBP: ffffc9000168fbf0 R08: 0000000000000000 R09: 000000000000000a
> [ 54.363817] R10: 000000000000000a R11: f000000000000000 R12: ffff888112ba8240
> [ 54.364736] R13: ffff888112a4c000 R14: 0000000000000000 R15: ffff888114dc9948
> [ 54.365655] FS: 00007f65e38a7100(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
> [ 54.366693] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 54.367452] CR2: 00007ffc6c1d9ff8 CR3: 0000000116668000 CR4: 00000000000006e0
> [ 54.368352] Call Trace:
> [ 54.368658] xfs_bmapi_remap+0x3bf/0x3f0
> [ 54.369143] xfs_bmap_finish_one+0x186/0x260
> [ 54.369666] xfs_bui_item_recover+0x266/0x370
> [ 54.370200] ? lock_release+0x13c/0x2e0
> [ 54.370667] xlog_recover_process_intents+0xd0/0x400
> [ 54.371276] xlog_recover_finish+0x14/0xc0
> [ 54.371837] xfs_log_mount_finish+0x54/0x1b0
> [ 54.372503] xfs_mountfs+0x580/0x9b0
> [ 54.373008] xfs_fs_fill_super+0x3aa/0x7e0
> [ 54.373535] ? xfs_fs_put_super+0x90/0x90
> [ 54.374118] get_tree_bdev+0x17a/0x280
> [ 54.374645] vfs_get_tree+0x23/0xc0
> [ 54.375089] path_mount+0x2b1/0xb40
> [ 54.375542] __x64_sys_mount+0xfe/0x140
> [ 54.376028] do_syscall_64+0x3b/0x90
> [ 54.376482] entry_SYSCALL_64_after_hwframe+0x44/0xae
> [ 54.377118] RIP: 0033:0x7f65e3aa5fea
> [ 54.377575] Code: 48 8b 0d a9 0e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 08
> [ 54.379859] RSP: 002b:00007ffc6c1db238 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5
> [ 54.380777] RAX: ffffffffffffffda RBX: 000055a56b0df970 RCX: 00007f65e3aa5fea
> [ 54.381642] RDX: 000055a56b0dfb80 RSI: 000055a56b0dfbc0 RDI: 000055a56b0dfba0
> [ 54.382503] RBP: 00007f65e3df31c4 R08: 0000000000000000 R09: 000055a56b0e2890
> [ 54.383378] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
> [ 54.384254] R13: 0000000000000000 R14: 000055a56b0dfba0 R15: 000055a56b0dfb80
> [ 54.385068] Modules linked in:
> [ 54.385495] ---[ end trace 81c36780b0c4c649 ]---
> [ 54.386033] RIP: 0010:assfail+0x1e/0x23
> [ 54.386500] Code: b8 2b 07 83 e8 85 fc ff ff 0f 0b c3 41 89 c8 48 89 d1 48
> 89 f2 48 c7 c6 b8 24
> [ 54.388948] RSP: 0018:ffffc9000168fb58 EFLAGS: 00010202
> [ 54.389674] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
> [ 54.390612] RDX: 00000000ffffffc0 RSI: 0000000000000000 RDI: ffffffff82fdbcf9
> [ 54.391534] RBP: ffffc9000168fbf0 R08: 0000000000000000 R09: 000000000000000a
> [ 54.392494] R10: 000000000000000a R11: f000000000000000 R12: ffff888112ba8240
> [ 54.393400] R13: ffff888112a4c000 R14: 0000000000000000 R15: ffff888114dc9948
> [ 54.394400] FS: 00007f65e38a7100(0000) GS:ffff88813bd00000(0000) knlGS:0000000000000000
> [ 54.395763] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 54.396425] CR2: 00007ffc6c1d9ff8 CR3: 0000000116668000 CR4: 00000000000006e0
> [ 54.397302] Kernel panic - not syncing: Fatal exception
> [ 54.398143] Kernel Offset: disabled
> [ 54.398660] ---[ end Kernel panic - not syncing: Fatal exception ]---
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: xfs/319 vs 1k block size file systems
2021-07-19 18:00 ` Darrick J. Wong
@ 2021-07-19 18:21 ` Christoph Hellwig
2021-07-19 21:29 ` Dave Chinner
0 siblings, 1 reply; 5+ messages in thread
From: Christoph Hellwig @ 2021-07-19 18:21 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: Christoph Hellwig, linux-xfs, Dave Chinner
On Mon, Jul 19, 2021 at 11:00:15AM -0700, Darrick J. Wong wrote:
> On Mon, Jul 19, 2021 at 12:20:54PM +0200, Christoph Hellwig wrote:
> > Hi all,
> >
> > xfs/319 keeps crashing for me when running it for a 1k block size
> > file systems on x86 with debugging enabled. The problem is that
> > xfs_bmapi_remap is called on a range that is not a hole.
>
> Hmm. Dave sent me a warning late last night about some sort of log
> recovery problem in -rc2 due to FUA changes or something. Did this
> happen in -rc1?
I've reproduced it all the way back to 5.12 so far.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: xfs/319 vs 1k block size file systems
2021-07-19 18:21 ` Christoph Hellwig
@ 2021-07-19 21:29 ` Dave Chinner
2021-07-20 4:51 ` Christoph Hellwig
0 siblings, 1 reply; 5+ messages in thread
From: Dave Chinner @ 2021-07-19 21:29 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Darrick J. Wong, linux-xfs
On Mon, Jul 19, 2021 at 07:21:28PM +0100, Christoph Hellwig wrote:
> On Mon, Jul 19, 2021 at 11:00:15AM -0700, Darrick J. Wong wrote:
> > On Mon, Jul 19, 2021 at 12:20:54PM +0200, Christoph Hellwig wrote:
> > > Hi all,
> > >
> > > xfs/319 keeps crashing for me when running it for a 1k block size
> > > file systems on x86 with debugging enabled. The problem is that
> > > xfs_bmapi_remap is called on a range that is not a hole.
> >
> > Hmm. Dave sent me a warning late last night about some sort of log
> > recovery problem in -rc2 due to FUA changes or something. Did this
> > happen in -rc1?
>
> I've reproduced it all the way back to 5.12 so far.
Ok, so not the regression I introduced in 5.14-rc1 with the journal
flush/FUA changes (caused by flush vs tail_lsn updates racing). THat
smells like the problem both Brian and I have seen that has occurred
occasionally for some time (I see it maybe once a month here) but we
haven't been able to reproduce reliably enough debug it.
Sounds like it's reliable on your setup? What's your test machine
config?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: xfs/319 vs 1k block size file systems
2021-07-19 21:29 ` Dave Chinner
@ 2021-07-20 4:51 ` Christoph Hellwig
0 siblings, 0 replies; 5+ messages in thread
From: Christoph Hellwig @ 2021-07-20 4:51 UTC (permalink / raw)
To: Dave Chinner; +Cc: Christoph Hellwig, Darrick J. Wong, linux-xfs
[-- Attachment #1: Type: text/plain, Size: 197 bytes --]
On Tue, Jul 20, 2021 at 07:29:15AM +1000, Dave Chinner wrote:
> Sounds like it's reliable on your setup? What's your test machine
> config?
qemu command line and the debug-heavy .config attached.
[-- Attachment #2: kvm.sh --]
[-- Type: application/x-sh, Size: 501 bytes --]
[-- Attachment #3: config.gz --]
[-- Type: application/gzip, Size: 35884 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-07-20 4:52 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-19 10:20 xfs/319 vs 1k block size file systems Christoph Hellwig
2021-07-19 18:00 ` Darrick J. Wong
2021-07-19 18:21 ` Christoph Hellwig
2021-07-19 21:29 ` Dave Chinner
2021-07-20 4:51 ` Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).