* Transaction log reservation overrun when fallocating realtime file
@ 2019-11-26 20:27 Omar Sandoval
2019-11-27 0:34 ` Darrick J. Wong
0 siblings, 1 reply; 7+ messages in thread
From: Omar Sandoval @ 2019-11-26 20:27 UTC (permalink / raw)
To: linux-xfs
[-- Attachment #1: Type: text/plain, Size: 640 bytes --]
Hello,
The following reproducer results in a transaction log overrun warning
for me:
mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
mount -o rtdev=/dev/vdc /dev/vdb /mnt
fallocate -l 4G /mnt/foo
I've attached the full dmesg output. My guess at the problem is that the
tr_write reservation used by xfs_alloc_file_space is not taking the realtime
bitmap and realtime summary inodes into account (inode numbers 129 and 130 on
this filesystem, which I do see in some of the log items). However, I'm not
familiar enough with the XFS transaction guts to confidently fix this. Can
someone please help me out?
Thanks!
[-- Attachment #2: dmesg --]
[-- Type: text/plain, Size: 40640 bytes --]
[ 82.661688] ------------[ cut here ]------------
[ 82.661787] WARNING: CPU: 11 PID: 473 at fs/xfs/xfs_log_cil.c:450 xfs_log_commit_cil.cold+0x20/0x90 [xfs]
[ 82.661788] Modules linked in: xfs libcrc32c joydev mousedev psmouse input_leds intel_agp serio_raw evdev intel_gtt agpgart i6300esb atkbd i2c_piix4 mac_hid libps2 pcspkr qemu_fw_cfg ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 virtio_net net_failover virtio_rng rng_core failover virtio_blk ata_generic pata_acpi ata_piix libata virtio_pci i8042 scsi_mod floppy virtio_ring virtio serio
[ 82.661806] CPU: 11 PID: 473 Comm: fallocate Not tainted 5.3.13-arch1-1 #1
[ 82.661807] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-1.el7.centos 04/01/2014
[ 82.661833] RIP: 0010:xfs_log_commit_cil.cold+0x20/0x90 [xfs]
[ 82.661834] Code: ca 0f c8 0f 0b e9 ce 91 fc ff 48 c7 c7 d8 de 80 c0 44 89 5c 24 3c 44 89 44 24 38 44 89 54 24 30 4c 89 4c 24 28 e8 60 ca 0f c8 <0f> 0b 49 8b 7d 00 48 c7 c6 00 df 80 c0 e8 eb fa ff ff 44 8b 5c 24
[ 82.661835] RSP: 0018:ffffa2a740747cc0 EFLAGS: 00010246
[ 82.661836] RAX: 0000000000000024 RBX: ffffa082285189c0 RCX: 0000000000000000
[ 82.661837] RDX: 0000000000000000 RSI: ffffa0823fcd7708 RDI: 00000000ffffffff
[ 82.661838] RBP: ffffa08228518910 R08: 000000000000021f R09: 0000000000000004
[ 82.661838] R10: 0000000000000000 R11: 0000000000000001 R12: ffffa08231bc8600
[ 82.661839] R13: ffffa08231bc6000 R14: ffffa0823cadcb40 R15: 0000000000000430
[ 82.661844] FS: 00007feac9cc55c0(0000) GS:ffffa0823fcc0000(0000) knlGS:0000000000000000
[ 82.661845] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 82.661845] CR2: 00007feac9c907e0 CR3: 00000002284fa000 CR4: 00000000000006e0
[ 82.661875] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 82.661876] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 82.661877] Call Trace:
[ 82.661907] __xfs_trans_commit+0xa1/0x360 [xfs]
[ 82.661930] xfs_alloc_file_space+0x1c6/0x390 [xfs]
[ 82.661953] xfs_file_fallocate+0x1f5/0x3a0 [xfs]
[ 82.661962] vfs_fallocate+0x14a/0x290
[ 82.661976] ksys_fallocate+0x3a/0x70
[ 82.661977] __x64_sys_fallocate+0x1a/0x20
[ 82.661982] do_syscall_64+0x4e/0x110
[ 82.661988] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 82.661990] RIP: 0033:0x7feac9bf22da
[ 82.661992] Code: ff ff ff eb c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca 64 8b 04 25 18 00 00 00 85 c0 75 15 b8 1d 01 00 00 0f 05 <48> 3d 00 f0 ff ff 77 5e c3 0f 1f 44 00 00 48 83 ec 28 48 89 54 24
[ 82.661992] RSP: 002b:00007ffd1a423828 EFLAGS: 00000246 ORIG_RAX: 000000000000011d
[ 82.661993] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007feac9bf22da
[ 82.661994] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000003
[ 82.661995] RBP: 000055cb6e1aa5e0 R08: 0000000000000000 R09: 0000000000000000
[ 82.661995] R10: 0000000100000000 R11: 0000000000000246 R12: 00007ffd1a423a58
[ 82.661996] R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000000000
[ 82.661998] ---[ end trace 019d1402342f5767 ]---
[ 82.662000] XFS (vdb): Transaction log reservation overrun:
[ 82.662002] XFS (vdb): log items: 133384 bytes (iov hdrs: 864 bytes)
[ 82.662002] XFS (vdb): split region headers: 2620 bytes
[ 82.662003] XFS (vdb): ctx ticket: 0 bytes
[ 82.662004] XFS (vdb): transaction summary:
[ 82.662004] XFS (vdb): log res = 109048
[ 82.662005] XFS (vdb): log count = 2
[ 82.662005] XFS (vdb): flags = 0x7
[ 82.662006] XFS (vdb): ticket reservation summary:
[ 82.662007] XFS (vdb): unit res = 112720 bytes
[ 82.662007] XFS (vdb): current res = -23284 bytes
[ 82.662008] XFS (vdb): total reg = 0 bytes (o/flow = 0 bytes)
[ 82.662008] XFS (vdb): ophdrs = 0 (ophdr space = 0 bytes)
[ 82.662009] XFS (vdb): ophdr + reg = 0 bytes
[ 82.662009] XFS (vdb): num regions = 0
[ 82.662010] XFS (vdb): log item:
[ 82.662010] XFS (vdb): type = 0x123b
[ 82.662011] XFS (vdb): flags = 0x8
[ 82.662011] XFS (vdb): niovecs = 3
[ 82.662012] XFS (vdb): size = 696
[ 82.662012] XFS (vdb): bytes = 248
[ 82.662013] XFS (vdb): buf len = 248
[ 82.662013] XFS (vdb): iovec[0]
[ 82.662014] XFS (vdb): type = 0x5
[ 82.662014] XFS (vdb): len = 56
[ 82.662015] XFS (vdb): first 32 bytes of iovec[0]:
[ 82.662017] 00000000: 3b 12 03 00 05 00 00 00 00 00 10 00 00 00 00 00 ;...............
[ 82.669239] 00000010: 83 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.676262] XFS (vdb): iovec[1]
[ 82.676264] XFS (vdb): type = 0x6
[ 82.676264] XFS (vdb): len = 176
[ 82.676265] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.676266] 00000000: 4e 49 a4 81 03 02 00 00 00 00 00 00 00 00 00 00 NI..............
[ 82.683226] 00000010: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.690285] XFS (vdb): iovec[2]
[ 82.690286] XFS (vdb): type = 0x7
[ 82.690287] XFS (vdb): len = 16
[ 82.690288] XFS (vdb): first 16 bytes of iovec[2]:
[ 82.690289] 00000000: 80 00 00 00 00 00 00 00 00 00 00 00 00 10 00 00 ................
[ 82.697198] XFS (vdb): log item:
[ 82.697199] XFS (vdb): type = 0x123b
[ 82.697199] XFS (vdb): flags = 0x8
[ 82.697200] XFS (vdb): niovecs = 2
[ 82.697201] XFS (vdb): size = 336
[ 82.697201] XFS (vdb): bytes = 232
[ 82.697202] XFS (vdb): buf len = 232
[ 82.697202] XFS (vdb): iovec[0]
[ 82.697203] XFS (vdb): type = 0x5
[ 82.697203] XFS (vdb): len = 56
[ 82.697204] XFS (vdb): first 32 bytes of iovec[0]:
[ 82.697205] 00000000: 3b 12 02 00 01 00 00 00 00 00 00 00 00 00 00 00 ;...............
[ 82.704078] 00000010: 81 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.711072] XFS (vdb): iovec[1]
[ 82.711073] XFS (vdb): type = 0x6
[ 82.711074] XFS (vdb): len = 176
[ 82.711075] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.711076] 00000000: 4e 49 00 80 03 02 00 00 00 00 00 00 00 00 00 00 NI..............
[ 82.717987] 00000010: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.724973] XFS (vdb): log item:
[ 82.724975] XFS (vdb): type = 0x123b
[ 82.724976] XFS (vdb): flags = 0x0
[ 82.724976] XFS (vdb): log item:
[ 82.724977] XFS (vdb): type = 0x123c
[ 82.724977] XFS (vdb): flags = 0x8
[ 82.724978] XFS (vdb): niovecs = 3
[ 82.724979] XFS (vdb): size = 408
[ 82.724979] XFS (vdb): bytes = 280
[ 82.724980] XFS (vdb): buf len = 280
[ 82.724980] XFS (vdb): iovec[0]
[ 82.724981] XFS (vdb): type = 0x1
[ 82.724982] XFS (vdb): len = 24
[ 82.724982] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.724983] 00000000: 3c 12 03 00 00 a0 08 00 70 00 00 00 00 00 00 00 <.......p.......
[ 82.731906] 00000010: 01 00 00 00 00 00 02 01 ........
[ 82.738064] XFS (vdb): iovec[1]
[ 82.738065] XFS (vdb): type = 0x2
[ 82.738066] XFS (vdb): len = 128
[ 82.738066] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.738067] 00000000: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.745010] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.751943] XFS (vdb): iovec[2]
[ 82.751945] XFS (vdb): type = 0x2
[ 82.751945] XFS (vdb): len = 128
[ 82.751946] XFS (vdb): first 32 bytes of iovec[2]:
[ 82.751947] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.758845] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.765881] XFS (vdb): log item:
[ 82.765882] XFS (vdb): type = 0x123c
[ 82.765883] XFS (vdb): flags = 0x8
[ 82.765884] XFS (vdb): niovecs = 2
[ 82.765885] XFS (vdb): size = 4224
[ 82.765885] XFS (vdb): bytes = 4120
[ 82.765886] XFS (vdb): buf len = 4120
[ 82.765887] XFS (vdb): iovec[0]
[ 82.765887] XFS (vdb): type = 0x1
[ 82.765888] XFS (vdb): len = 24
[ 82.765888] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.765889] 00000000: 3c 12 02 00 00 98 08 00 c0 00 00 00 00 00 00 00 <...............
[ 82.772786] 00000010: 01 00 00 00 ff ff ff ff ........
[ 82.779369] XFS (vdb): iovec[1]
[ 82.779370] XFS (vdb): type = 0x2
[ 82.779371] XFS (vdb): len = 4096
[ 82.779371] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.779372] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.786529] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.793420] XFS (vdb): log item:
[ 82.793421] XFS (vdb): type = 0x123c
[ 82.793422] XFS (vdb): flags = 0x8
[ 82.793422] XFS (vdb): niovecs = 2
[ 82.793423] XFS (vdb): size = 4224
[ 82.793424] XFS (vdb): bytes = 4120
[ 82.793424] XFS (vdb): buf len = 4120
[ 82.793425] XFS (vdb): iovec[0]
[ 82.793426] XFS (vdb): type = 0x1
[ 82.793426] XFS (vdb): len = 24
[ 82.793427] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.793428] 00000000: 3c 12 02 00 00 98 08 00 c8 00 00 00 00 00 00 00 <...............
[ 82.801796] 00000010: 01 00 00 00 ff ff ff ff ........
[ 82.809042] XFS (vdb): iovec[1]
[ 82.809043] XFS (vdb): type = 0x2
[ 82.809043] XFS (vdb): len = 4096
[ 82.809044] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.809045] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.817197] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.824982] XFS (vdb): log item:
[ 82.824984] XFS (vdb): type = 0x123c
[ 82.824985] XFS (vdb): flags = 0x8
[ 82.824986] XFS (vdb): niovecs = 2
[ 82.824987] XFS (vdb): size = 4224
[ 82.824988] XFS (vdb): bytes = 4120
[ 82.824989] XFS (vdb): buf len = 4120
[ 82.824990] XFS (vdb): iovec[0]
[ 82.824991] XFS (vdb): type = 0x1
[ 82.824992] XFS (vdb): len = 24
[ 82.824993] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.824994] 00000000: 3c 12 02 00 00 98 08 00 d0 00 00 00 00 00 00 00 <...............
[ 82.833428] 00000010: 01 00 00 00 ff ff ff ff ........
[ 82.841352] XFS (vdb): iovec[1]
[ 82.841353] XFS (vdb): type = 0x2
[ 82.841354] XFS (vdb): len = 4096
[ 82.841355] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.841357] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.849908] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.857016] XFS (vdb): log item:
[ 82.857017] XFS (vdb): type = 0x123c
[ 82.857018] XFS (vdb): flags = 0x8
[ 82.857018] XFS (vdb): niovecs = 2
[ 82.857019] XFS (vdb): size = 4224
[ 82.857020] XFS (vdb): bytes = 4120
[ 82.857020] XFS (vdb): buf len = 4120
[ 82.857021] XFS (vdb): iovec[0]
[ 82.857021] XFS (vdb): type = 0x1
[ 82.857022] XFS (vdb): len = 24
[ 82.857023] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.857024] 00000000: 3c 12 02 00 00 98 08 00 d8 00 00 00 00 00 00 00 <...............
[ 82.865490] 00000010: 01 00 00 00 ff ff ff ff ........
[ 82.872273] XFS (vdb): iovec[1]
[ 82.872274] XFS (vdb): type = 0x2
[ 82.872274] XFS (vdb): len = 4096
[ 82.872275] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.872276] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.879792] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.886850] XFS (vdb): log item:
[ 82.886851] XFS (vdb): type = 0x123c
[ 82.886851] XFS (vdb): flags = 0x8
[ 82.886852] XFS (vdb): niovecs = 2
[ 82.886853] XFS (vdb): size = 4224
[ 82.886853] XFS (vdb): bytes = 4120
[ 82.886853] XFS (vdb): buf len = 4120
[ 82.886854] XFS (vdb): iovec[0]
[ 82.886855] XFS (vdb): type = 0x1
[ 82.886855] XFS (vdb): len = 24
[ 82.886879] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.886881] 00000000: 3c 12 02 00 00 98 08 00 e0 00 00 00 00 00 00 00 <...............
[ 82.893660] 00000010: 01 00 00 00 ff ff ff ff ........
[ 82.899881] XFS (vdb): iovec[1]
[ 82.899882] XFS (vdb): type = 0x2
[ 82.899882] XFS (vdb): len = 4096
[ 82.899883] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.899884] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.907257] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.914785] XFS (vdb): log item:
[ 82.914786] XFS (vdb): type = 0x123c
[ 82.914787] XFS (vdb): flags = 0x8
[ 82.914787] XFS (vdb): niovecs = 2
[ 82.914788] XFS (vdb): size = 4224
[ 82.914789] XFS (vdb): bytes = 4120
[ 82.914789] XFS (vdb): buf len = 4120
[ 82.914790] XFS (vdb): iovec[0]
[ 82.914790] XFS (vdb): type = 0x1
[ 82.914791] XFS (vdb): len = 24
[ 82.914792] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.914793] 00000000: 3c 12 02 00 00 98 08 00 e8 00 00 00 00 00 00 00 <...............
[ 82.921982] 00000010: 01 00 00 00 ff ff ff ff ........
[ 82.928199] XFS (vdb): iovec[1]
[ 82.928200] XFS (vdb): type = 0x2
[ 82.928201] XFS (vdb): len = 4096
[ 82.928202] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.928203] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.935001] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.941751] XFS (vdb): log item:
[ 82.941752] XFS (vdb): type = 0x123c
[ 82.941753] XFS (vdb): flags = 0x8
[ 82.941754] XFS (vdb): niovecs = 2
[ 82.941754] XFS (vdb): size = 4224
[ 82.941755] XFS (vdb): bytes = 4120
[ 82.941755] XFS (vdb): buf len = 4120
[ 82.941756] XFS (vdb): iovec[0]
[ 82.941757] XFS (vdb): type = 0x1
[ 82.941757] XFS (vdb): len = 24
[ 82.941758] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.941759] 00000000: 3c 12 02 00 00 98 08 00 f0 00 00 00 00 00 00 00 <...............
[ 82.948544] 00000010: 01 00 00 00 ff ff ff ff ........
[ 82.954697] XFS (vdb): iovec[1]
[ 82.954698] XFS (vdb): type = 0x2
[ 82.954698] XFS (vdb): len = 4096
[ 82.954699] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.954700] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.961543] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.969056] XFS (vdb): log item:
[ 82.969057] XFS (vdb): type = 0x123c
[ 82.969057] XFS (vdb): flags = 0x8
[ 82.969058] XFS (vdb): niovecs = 2
[ 82.969059] XFS (vdb): size = 4224
[ 82.969059] XFS (vdb): bytes = 4120
[ 82.969060] XFS (vdb): buf len = 4120
[ 82.969060] XFS (vdb): iovec[0]
[ 82.969061] XFS (vdb): type = 0x1
[ 82.969061] XFS (vdb): len = 24
[ 82.969062] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.969063] 00000000: 3c 12 02 00 00 98 08 00 f8 00 00 00 00 00 00 00 <...............
[ 82.975835] 00000010: 01 00 00 00 ff ff ff ff ........
[ 82.982103] XFS (vdb): iovec[1]
[ 82.982104] XFS (vdb): type = 0x2
[ 82.982105] XFS (vdb): len = 4096
[ 82.982105] XFS (vdb): first 32 bytes of iovec[1]:
[ 82.982106] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.988881] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 82.995712] XFS (vdb): log item:
[ 82.995713] XFS (vdb): type = 0x123c
[ 82.995714] XFS (vdb): flags = 0x8
[ 82.995715] XFS (vdb): niovecs = 2
[ 82.995715] XFS (vdb): size = 4224
[ 82.995716] XFS (vdb): bytes = 4120
[ 82.995716] XFS (vdb): buf len = 4120
[ 82.995717] XFS (vdb): iovec[0]
[ 82.995718] XFS (vdb): type = 0x1
[ 82.995718] XFS (vdb): len = 24
[ 82.995719] XFS (vdb): first 24 bytes of iovec[0]:
[ 82.995720] 00000000: 3c 12 02 00 00 98 08 00 00 01 00 00 00 00 00 00 <...............
[ 83.002469] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.008658] XFS (vdb): iovec[1]
[ 83.008659] XFS (vdb): type = 0x2
[ 83.008660] XFS (vdb): len = 4096
[ 83.008660] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.008661] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.015845] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.022456] XFS (vdb): log item:
[ 83.022458] XFS (vdb): type = 0x123c
[ 83.022458] XFS (vdb): flags = 0x8
[ 83.022459] XFS (vdb): niovecs = 2
[ 83.022460] XFS (vdb): size = 4224
[ 83.022460] XFS (vdb): bytes = 4120
[ 83.022461] XFS (vdb): buf len = 4120
[ 83.022461] XFS (vdb): iovec[0]
[ 83.022462] XFS (vdb): type = 0x1
[ 83.022462] XFS (vdb): len = 24
[ 83.022463] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.022464] 00000000: 3c 12 02 00 00 98 08 00 08 01 00 00 00 00 00 00 <...............
[ 83.029069] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.035121] XFS (vdb): iovec[1]
[ 83.035121] XFS (vdb): type = 0x2
[ 83.035122] XFS (vdb): len = 4096
[ 83.035123] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.035124] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.041816] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.048382] XFS (vdb): log item:
[ 83.048383] XFS (vdb): type = 0x123c
[ 83.048384] XFS (vdb): flags = 0x8
[ 83.048384] XFS (vdb): niovecs = 2
[ 83.048385] XFS (vdb): size = 4224
[ 83.048385] XFS (vdb): bytes = 4120
[ 83.048386] XFS (vdb): buf len = 4120
[ 83.048386] XFS (vdb): iovec[0]
[ 83.048387] XFS (vdb): type = 0x1
[ 83.048387] XFS (vdb): len = 24
[ 83.048388] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.048389] 00000000: 3c 12 02 00 00 98 08 00 10 01 00 00 00 00 00 00 <...............
[ 83.055004] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.061804] XFS (vdb): iovec[1]
[ 83.061805] XFS (vdb): type = 0x2
[ 83.061805] XFS (vdb): len = 4096
[ 83.061806] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.061807] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.068439] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.074969] XFS (vdb): log item:
[ 83.074970] XFS (vdb): type = 0x123c
[ 83.074971] XFS (vdb): flags = 0x8
[ 83.074971] XFS (vdb): niovecs = 2
[ 83.074972] XFS (vdb): size = 4224
[ 83.074972] XFS (vdb): bytes = 4120
[ 83.074973] XFS (vdb): buf len = 4120
[ 83.074973] XFS (vdb): iovec[0]
[ 83.074974] XFS (vdb): type = 0x1
[ 83.074974] XFS (vdb): len = 24
[ 83.074975] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.074976] 00000000: 3c 12 02 00 00 98 08 00 18 01 00 00 00 00 00 00 <...............
[ 83.081507] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.087516] XFS (vdb): iovec[1]
[ 83.087516] XFS (vdb): type = 0x2
[ 83.087517] XFS (vdb): len = 4096
[ 83.087518] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.087519] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.094142] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.100591] XFS (vdb): log item:
[ 83.100592] XFS (vdb): type = 0x123c
[ 83.100593] XFS (vdb): flags = 0x8
[ 83.100593] XFS (vdb): niovecs = 2
[ 83.100594] XFS (vdb): size = 4224
[ 83.100595] XFS (vdb): bytes = 4120
[ 83.100595] XFS (vdb): buf len = 4120
[ 83.100596] XFS (vdb): iovec[0]
[ 83.100596] XFS (vdb): type = 0x1
[ 83.100597] XFS (vdb): len = 24
[ 83.100597] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.100598] 00000000: 3c 12 02 00 00 98 08 00 20 01 00 00 00 00 00 00 <....... .......
[ 83.107841] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.113968] XFS (vdb): iovec[1]
[ 83.113969] XFS (vdb): type = 0x2
[ 83.113970] XFS (vdb): len = 4096
[ 83.113970] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.113971] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.120606] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.127198] XFS (vdb): log item:
[ 83.127199] XFS (vdb): type = 0x123c
[ 83.127200] XFS (vdb): flags = 0x8
[ 83.127200] XFS (vdb): niovecs = 2
[ 83.127201] XFS (vdb): size = 4224
[ 83.127202] XFS (vdb): bytes = 4120
[ 83.127202] XFS (vdb): buf len = 4120
[ 83.127203] XFS (vdb): iovec[0]
[ 83.127203] XFS (vdb): type = 0x1
[ 83.127204] XFS (vdb): len = 24
[ 83.127205] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.127206] 00000000: 3c 12 02 00 00 98 08 00 28 01 00 00 00 00 00 00 <.......(.......
[ 83.133803] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.139757] XFS (vdb): iovec[1]
[ 83.139758] XFS (vdb): type = 0x2
[ 83.139759] XFS (vdb): len = 4096
[ 83.139759] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.139760] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.146366] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.153396] XFS (vdb): log item:
[ 83.153398] XFS (vdb): type = 0x123c
[ 83.153398] XFS (vdb): flags = 0x8
[ 83.153399] XFS (vdb): niovecs = 2
[ 83.153399] XFS (vdb): size = 4224
[ 83.153400] XFS (vdb): bytes = 4120
[ 83.153400] XFS (vdb): buf len = 4120
[ 83.153401] XFS (vdb): iovec[0]
[ 83.153402] XFS (vdb): type = 0x1
[ 83.153402] XFS (vdb): len = 24
[ 83.153403] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.153404] 00000000: 3c 12 02 00 00 98 08 00 30 01 00 00 00 00 00 00 <.......0.......
[ 83.159950] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.165934] XFS (vdb): iovec[1]
[ 83.165935] XFS (vdb): type = 0x2
[ 83.165936] XFS (vdb): len = 4096
[ 83.165936] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.165937] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.172537] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.181475] XFS (vdb): log item:
[ 83.181476] XFS (vdb): type = 0x123c
[ 83.181477] XFS (vdb): flags = 0x8
[ 83.181478] XFS (vdb): niovecs = 2
[ 83.181479] XFS (vdb): size = 4224
[ 83.181480] XFS (vdb): bytes = 4120
[ 83.181481] XFS (vdb): buf len = 4120
[ 83.181482] XFS (vdb): iovec[0]
[ 83.181482] XFS (vdb): type = 0x1
[ 83.181483] XFS (vdb): len = 24
[ 83.181484] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.181486] 00000000: 3c 12 02 00 00 98 08 00 38 01 00 00 00 00 00 00 <.......8.......
[ 83.190113] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.196584] XFS (vdb): iovec[1]
[ 83.196585] XFS (vdb): type = 0x2
[ 83.196586] XFS (vdb): len = 4096
[ 83.196587] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.196588] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.203921] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.210494] XFS (vdb): log item:
[ 83.210495] XFS (vdb): type = 0x123c
[ 83.210496] XFS (vdb): flags = 0x8
[ 83.210496] XFS (vdb): niovecs = 2
[ 83.210497] XFS (vdb): size = 4224
[ 83.210498] XFS (vdb): bytes = 4120
[ 83.210498] XFS (vdb): buf len = 4120
[ 83.210499] XFS (vdb): iovec[0]
[ 83.210499] XFS (vdb): type = 0x1
[ 83.210500] XFS (vdb): len = 24
[ 83.210501] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.210502] 00000000: 3c 12 02 00 00 98 08 00 40 01 00 00 00 00 00 00 <.......@.......
[ 83.217178] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.223219] XFS (vdb): iovec[1]
[ 83.223220] XFS (vdb): type = 0x2
[ 83.223220] XFS (vdb): len = 4096
[ 83.223221] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.223222] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.229744] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.236426] XFS (vdb): log item:
[ 83.236427] XFS (vdb): type = 0x123c
[ 83.236427] XFS (vdb): flags = 0x8
[ 83.236428] XFS (vdb): niovecs = 2
[ 83.236428] XFS (vdb): size = 4224
[ 83.236429] XFS (vdb): bytes = 4120
[ 83.236430] XFS (vdb): buf len = 4120
[ 83.236430] XFS (vdb): iovec[0]
[ 83.236431] XFS (vdb): type = 0x1
[ 83.236431] XFS (vdb): len = 24
[ 83.236432] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.236433] 00000000: 3c 12 02 00 00 98 08 00 48 01 00 00 00 00 00 00 <.......H.......
[ 83.242938] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.248912] XFS (vdb): iovec[1]
[ 83.248912] XFS (vdb): type = 0x2
[ 83.248913] XFS (vdb): len = 4096
[ 83.248914] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.248915] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.256151] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.262936] XFS (vdb): log item:
[ 83.262938] XFS (vdb): type = 0x123c
[ 83.262938] XFS (vdb): flags = 0x8
[ 83.262939] XFS (vdb): niovecs = 2
[ 83.262939] XFS (vdb): size = 4224
[ 83.262940] XFS (vdb): bytes = 4120
[ 83.262941] XFS (vdb): buf len = 4120
[ 83.262941] XFS (vdb): iovec[0]
[ 83.262942] XFS (vdb): type = 0x1
[ 83.262942] XFS (vdb): len = 24
[ 83.262943] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.262944] 00000000: 3c 12 02 00 00 98 08 00 50 01 00 00 00 00 00 00 <.......P.......
[ 83.269615] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.275729] XFS (vdb): iovec[1]
[ 83.275730] XFS (vdb): type = 0x2
[ 83.275730] XFS (vdb): len = 4096
[ 83.275731] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.275732] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.282358] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.288899] XFS (vdb): log item:
[ 83.288900] XFS (vdb): type = 0x123c
[ 83.288900] XFS (vdb): flags = 0x8
[ 83.288901] XFS (vdb): niovecs = 2
[ 83.288901] XFS (vdb): size = 4224
[ 83.288902] XFS (vdb): bytes = 4120
[ 83.288902] XFS (vdb): buf len = 4120
[ 83.288903] XFS (vdb): iovec[0]
[ 83.288903] XFS (vdb): type = 0x1
[ 83.288904] XFS (vdb): len = 24
[ 83.288905] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.288906] 00000000: 3c 12 02 00 00 98 08 00 58 01 00 00 00 00 00 00 <.......X.......
[ 83.295485] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.301497] XFS (vdb): iovec[1]
[ 83.301498] XFS (vdb): type = 0x2
[ 83.301498] XFS (vdb): len = 4096
[ 83.301499] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.301500] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.308919] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.315543] XFS (vdb): log item:
[ 83.315544] XFS (vdb): type = 0x123c
[ 83.315544] XFS (vdb): flags = 0x8
[ 83.315545] XFS (vdb): niovecs = 2
[ 83.315546] XFS (vdb): size = 4224
[ 83.315546] XFS (vdb): bytes = 4120
[ 83.315547] XFS (vdb): buf len = 4120
[ 83.315547] XFS (vdb): iovec[0]
[ 83.315548] XFS (vdb): type = 0x1
[ 83.315548] XFS (vdb): len = 24
[ 83.315549] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.315550] 00000000: 3c 12 02 00 00 98 08 00 60 01 00 00 00 00 00 00 <.......`.......
[ 83.322409] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.328502] XFS (vdb): iovec[1]
[ 83.328503] XFS (vdb): type = 0x2
[ 83.328504] XFS (vdb): len = 4096
[ 83.328504] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.328505] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.335141] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.341703] XFS (vdb): log item:
[ 83.341704] XFS (vdb): type = 0x123c
[ 83.341704] XFS (vdb): flags = 0x8
[ 83.341705] XFS (vdb): niovecs = 2
[ 83.341706] XFS (vdb): size = 4224
[ 83.341706] XFS (vdb): bytes = 4120
[ 83.341707] XFS (vdb): buf len = 4120
[ 83.341707] XFS (vdb): iovec[0]
[ 83.341708] XFS (vdb): type = 0x1
[ 83.341708] XFS (vdb): len = 24
[ 83.341709] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.341710] 00000000: 3c 12 02 00 00 98 08 00 68 01 00 00 00 00 00 00 <.......h.......
[ 83.348424] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.355163] XFS (vdb): iovec[1]
[ 83.355164] XFS (vdb): type = 0x2
[ 83.355165] XFS (vdb): len = 4096
[ 83.355165] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.355166] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.361847] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.368416] XFS (vdb): log item:
[ 83.368418] XFS (vdb): type = 0x123c
[ 83.368418] XFS (vdb): flags = 0x8
[ 83.368419] XFS (vdb): niovecs = 2
[ 83.368420] XFS (vdb): size = 4224
[ 83.368420] XFS (vdb): bytes = 4120
[ 83.368421] XFS (vdb): buf len = 4120
[ 83.368421] XFS (vdb): iovec[0]
[ 83.368422] XFS (vdb): type = 0x1
[ 83.368422] XFS (vdb): len = 24
[ 83.368423] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.368424] 00000000: 3c 12 02 00 00 98 08 00 70 01 00 00 00 00 00 00 <.......p.......
[ 83.374980] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.381264] XFS (vdb): iovec[1]
[ 83.381265] XFS (vdb): type = 0x2
[ 83.381266] XFS (vdb): len = 4096
[ 83.381266] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.381267] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.387906] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.394588] XFS (vdb): log item:
[ 83.394589] XFS (vdb): type = 0x123c
[ 83.394589] XFS (vdb): flags = 0x8
[ 83.394590] XFS (vdb): niovecs = 2
[ 83.394590] XFS (vdb): size = 4224
[ 83.394591] XFS (vdb): bytes = 4120
[ 83.394591] XFS (vdb): buf len = 4120
[ 83.394592] XFS (vdb): iovec[0]
[ 83.394592] XFS (vdb): type = 0x1
[ 83.394593] XFS (vdb): len = 24
[ 83.394594] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.394595] 00000000: 3c 12 02 00 00 98 08 00 78 01 00 00 00 00 00 00 <.......x.......
[ 83.401891] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.407948] XFS (vdb): iovec[1]
[ 83.407949] XFS (vdb): type = 0x2
[ 83.407950] XFS (vdb): len = 4096
[ 83.407951] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.407951] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.414609] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.421180] XFS (vdb): log item:
[ 83.421181] XFS (vdb): type = 0x123c
[ 83.421182] XFS (vdb): flags = 0x8
[ 83.421182] XFS (vdb): niovecs = 2
[ 83.421183] XFS (vdb): size = 4224
[ 83.421184] XFS (vdb): bytes = 4120
[ 83.421184] XFS (vdb): buf len = 4120
[ 83.421185] XFS (vdb): iovec[0]
[ 83.421185] XFS (vdb): type = 0x1
[ 83.421186] XFS (vdb): len = 24
[ 83.421187] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.421188] 00000000: 3c 12 02 00 00 98 08 00 80 01 00 00 00 00 00 00 <...............
[ 83.427839] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.433969] XFS (vdb): iovec[1]
[ 83.433969] XFS (vdb): type = 0x2
[ 83.433970] XFS (vdb): len = 4096
[ 83.433971] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.433972] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.440509] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.447538] XFS (vdb): log item:
[ 83.447539] XFS (vdb): type = 0x123c
[ 83.447540] XFS (vdb): flags = 0x8
[ 83.447540] XFS (vdb): niovecs = 2
[ 83.447541] XFS (vdb): size = 4224
[ 83.447541] XFS (vdb): bytes = 4120
[ 83.447542] XFS (vdb): buf len = 4120
[ 83.447542] XFS (vdb): iovec[0]
[ 83.447543] XFS (vdb): type = 0x1
[ 83.447543] XFS (vdb): len = 24
[ 83.447544] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.447545] 00000000: 3c 12 02 00 00 98 08 00 88 01 00 00 00 00 00 00 <...............
[ 83.454288] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.460425] XFS (vdb): iovec[1]
[ 83.460426] XFS (vdb): type = 0x2
[ 83.460427] XFS (vdb): len = 4096
[ 83.460428] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.460430] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.467208] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.473782] XFS (vdb): log item:
[ 83.473783] XFS (vdb): type = 0x123c
[ 83.473784] XFS (vdb): flags = 0x8
[ 83.473785] XFS (vdb): niovecs = 2
[ 83.473786] XFS (vdb): size = 4224
[ 83.473786] XFS (vdb): bytes = 4120
[ 83.473787] XFS (vdb): buf len = 4120
[ 83.473788] XFS (vdb): iovec[0]
[ 83.473789] XFS (vdb): type = 0x1
[ 83.473790] XFS (vdb): len = 24
[ 83.473791] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.473792] 00000000: 3c 12 02 00 00 98 08 00 90 01 00 00 00 00 00 00 <...............
[ 83.480389] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.486512] XFS (vdb): iovec[1]
[ 83.486514] XFS (vdb): type = 0x2
[ 83.486515] XFS (vdb): len = 4096
[ 83.486516] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.486517] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.493638] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.500207] XFS (vdb): log item:
[ 83.500208] XFS (vdb): type = 0x123c
[ 83.500209] XFS (vdb): flags = 0x8
[ 83.500210] XFS (vdb): niovecs = 2
[ 83.500211] XFS (vdb): size = 4224
[ 83.500212] XFS (vdb): bytes = 4120
[ 83.500213] XFS (vdb): buf len = 4120
[ 83.500214] XFS (vdb): iovec[0]
[ 83.500214] XFS (vdb): type = 0x1
[ 83.500215] XFS (vdb): len = 24
[ 83.500216] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.500218] 00000000: 3c 12 02 00 00 98 08 00 98 01 00 00 00 00 00 00 <...............
[ 83.506777] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.512886] XFS (vdb): iovec[1]
[ 83.512887] XFS (vdb): type = 0x2
[ 83.512888] XFS (vdb): len = 4096
[ 83.512889] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.512890] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.519454] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.526155] XFS (vdb): log item:
[ 83.526157] XFS (vdb): type = 0x123c
[ 83.526158] XFS (vdb): flags = 0x8
[ 83.526159] XFS (vdb): niovecs = 2
[ 83.526160] XFS (vdb): size = 4224
[ 83.526160] XFS (vdb): bytes = 4120
[ 83.526161] XFS (vdb): buf len = 4120
[ 83.526162] XFS (vdb): iovec[0]
[ 83.526163] XFS (vdb): type = 0x1
[ 83.526164] XFS (vdb): len = 24
[ 83.526165] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.526166] 00000000: 3c 12 02 00 00 98 08 00 a0 01 00 00 00 00 00 00 <...............
[ 83.532795] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.539574] XFS (vdb): iovec[1]
[ 83.539575] XFS (vdb): type = 0x2
[ 83.539576] XFS (vdb): len = 4096
[ 83.539577] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.539579] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.546268] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.553048] XFS (vdb): log item:
[ 83.553050] XFS (vdb): type = 0x123c
[ 83.553051] XFS (vdb): flags = 0x8
[ 83.553052] XFS (vdb): niovecs = 2
[ 83.553053] XFS (vdb): size = 4224
[ 83.553053] XFS (vdb): bytes = 4120
[ 83.553054] XFS (vdb): buf len = 4120
[ 83.553055] XFS (vdb): iovec[0]
[ 83.553056] XFS (vdb): type = 0x1
[ 83.553057] XFS (vdb): len = 24
[ 83.553058] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.553059] 00000000: 3c 12 02 00 00 98 08 00 a8 01 00 00 00 00 00 00 <...............
[ 83.559902] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.566062] XFS (vdb): iovec[1]
[ 83.566063] XFS (vdb): type = 0x2
[ 83.566064] XFS (vdb): len = 4096
[ 83.566065] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.566067] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.572695] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.579308] XFS (vdb): log item:
[ 83.579310] XFS (vdb): type = 0x123c
[ 83.579311] XFS (vdb): flags = 0x8
[ 83.579311] XFS (vdb): niovecs = 2
[ 83.579312] XFS (vdb): size = 4224
[ 83.579313] XFS (vdb): bytes = 4120
[ 83.579314] XFS (vdb): buf len = 4120
[ 83.579315] XFS (vdb): iovec[0]
[ 83.579316] XFS (vdb): type = 0x1
[ 83.579316] XFS (vdb): len = 24
[ 83.579317] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.579319] 00000000: 3c 12 02 00 00 98 08 00 b0 01 00 00 00 00 00 00 <...............
[ 83.586602] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.592682] XFS (vdb): iovec[1]
[ 83.592684] XFS (vdb): type = 0x2
[ 83.592685] XFS (vdb): len = 4096
[ 83.592686] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.592687] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.599170] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.605842] XFS (vdb): log item:
[ 83.605844] XFS (vdb): type = 0x123c
[ 83.605845] XFS (vdb): flags = 0x8
[ 83.605846] XFS (vdb): niovecs = 2
[ 83.605847] XFS (vdb): size = 4224
[ 83.605848] XFS (vdb): bytes = 4120
[ 83.605848] XFS (vdb): buf len = 4120
[ 83.605849] XFS (vdb): iovec[0]
[ 83.605850] XFS (vdb): type = 0x1
[ 83.605851] XFS (vdb): len = 24
[ 83.605852] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.605853] 00000000: 3c 12 02 00 00 98 08 00 b8 01 00 00 00 00 00 00 <...............
[ 83.612352] 00000010: 01 00 00 00 ff ff ff ff ........
[ 83.618364] XFS (vdb): iovec[1]
[ 83.618366] XFS (vdb): type = 0x2
[ 83.618367] XFS (vdb): len = 4096
[ 83.618367] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.618369] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.624908] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
[ 83.631587] XFS (vdb): log item:
[ 83.631588] XFS (vdb): type = 0x123c
[ 83.631589] XFS (vdb): flags = 0x8
[ 83.631590] XFS (vdb): niovecs = 2
[ 83.631591] XFS (vdb): size = 256
[ 83.631592] XFS (vdb): bytes = 152
[ 83.631593] XFS (vdb): buf len = 152
[ 83.631594] XFS (vdb): iovec[0]
[ 83.631594] XFS (vdb): type = 0x1
[ 83.631595] XFS (vdb): len = 24
[ 83.631596] XFS (vdb): first 24 bytes of iovec[0]:
[ 83.631598] 00000000: 3c 12 02 00 00 90 01 00 00 00 00 00 00 00 00 00 <...............
[ 83.638780] 00000010: 01 00 00 00 02 00 00 00 ........
[ 83.644896] XFS (vdb): iovec[1]
[ 83.644898] XFS (vdb): type = 0x2
[ 83.644899] XFS (vdb): len = 128
[ 83.644900] XFS (vdb): first 32 bytes of iovec[1]:
[ 83.644901] 00000000: 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00 3d .......@.......=
[ 83.651821] 00000010: 00 00 00 00 00 7f ee de 00 00 00 00 00 70 00 00 .............p..
[ 83.659045] XFS (vdb): xfs_do_force_shutdown(0x2) called from line 484 of file fs/xfs/xfs_log_cil.c. Return address = 0000000090960c02
[ 83.659047] XFS (vdb): Log I/O Error Detected. Shutting down filesystem
[ 83.664758] XFS (vdb): Please unmount the filesystem and rectify the problem(s)
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transaction log reservation overrun when fallocating realtime file
2019-11-26 20:27 Transaction log reservation overrun when fallocating realtime file Omar Sandoval
@ 2019-11-27 0:34 ` Darrick J. Wong
2019-12-02 19:32 ` Omar Sandoval
2019-12-02 21:51 ` Dave Chinner
0 siblings, 2 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-11-27 0:34 UTC (permalink / raw)
To: Omar Sandoval; +Cc: linux-xfs
On Tue, Nov 26, 2019 at 12:27:14PM -0800, Omar Sandoval wrote:
> Hello,
>
> The following reproducer results in a transaction log overrun warning
> for me:
>
> mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
> mount -o rtdev=/dev/vdc /dev/vdb /mnt
> fallocate -l 4G /mnt/foo
>
> I've attached the full dmesg output. My guess at the problem is that the
> tr_write reservation used by xfs_alloc_file_space is not taking the realtime
> bitmap and realtime summary inodes into account (inode numbers 129 and 130 on
> this filesystem, which I do see in some of the log items). However, I'm not
> familiar enough with the XFS transaction guts to confidently fix this. Can
> someone please help me out?
Hmm...
/*
* In a write transaction we can allocate a maximum of 2
* extents. This gives:
* the inode getting the new extents: inode size
* the inode's bmap btree: max depth * block size
* the agfs of the ags from which the extents are allocated: 2 * sector
* the superblock free block counter: sector size
* the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
* And the bmap_finish transaction can free bmap blocks in a join:
* the agfs of the ags containing the blocks: 2 * sector size
* the agfls of the ags containing the blocks: 2 * sector size
* the super block free block counter: sector size
* the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
*/
STATIC uint
xfs_calc_write_reservation(...);
So this means that the rt allocator can burn through at most ...
1 ext * 2 trees * (2 * maxdepth - 1) * blocksize
... worth of log reservation as part of setting bits in the rtbitmap and
fiddling with the rtsummary information.
Instead, 4GB of 4k rt extents == 1 million rtexts to mark in use, which
is 131072 bytes of rtbitmap to log, and *kaboom* there goes the 109K log
reservation.
So I think you're right, and the fix is probably? to cap ralen further
in xfs_bmap_rtalloc(). Does the following patch fix it?
--D
From: Darrick J. Wong <darrick.wong@oracle.com>
xfs: cap realtime allocation length to something we can log
Omar Sandoval reported that a 4G fallocate on the realtime device causes
filesystem shutdowns due to a log reservation overflow that happens when
we log the rtbitmap updates.
The tr_write transaction reserves enough log reservation to handle a
full splits of both free space btrees, so cap the rt allocation at that
number of bits.
"The following reproducer results in a transaction log overrun warning
for me:
mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
mount -o rtdev=/dev/vdc /dev/vdb /mnt
fallocate -l 4G /mnt/foo
Reported-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
fs/xfs/xfs_bmap_util.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c
index 49d7b530c8f7..15c4e2790de3 100644
--- a/fs/xfs/xfs_bmap_util.c
+++ b/fs/xfs/xfs_bmap_util.c
@@ -69,6 +69,26 @@ xfs_zero_extent(
}
#ifdef CONFIG_XFS_RT
+/*
+ * tr_write allows for one full split in the bnobt and cntbt to record the
+ * allocation, and that's how many bits of rtbitmap we can log to the
+ * transaction. We leave one full block's worth of log space to handle the
+ * rtsummary update, though that's probably overkill.
+ */
+static inline uint64_t
+xfs_bmap_rtalloc_max(
+ struct xfs_mount *mp)
+{
+ uint64_t max_rtbitmap;
+
+ max_rtbitmap = xfs_allocfree_log_count(mp, 1) - 1;
+ max_rtbitmap *= XFS_FSB_TO_B(mp, 1);
+ max_rtbitmap *= NBBY;
+ max_rtbitmap *= mp->m_sb.sb_rextsize;
+
+ return max_rtbitmap;
+}
+
int
xfs_bmap_rtalloc(
struct xfs_bmalloca *ap) /* bmap alloc argument struct */
@@ -113,6 +133,9 @@ xfs_bmap_rtalloc(
if (ralen * mp->m_sb.sb_rextsize >= MAXEXTLEN)
ralen = MAXEXTLEN / mp->m_sb.sb_rextsize;
+ /* Don't allocate so much that we blow out the log reservation. */
+ ralen = min_t(uint64_t, ralen, xfs_bmap_rtalloc_max(mp));
+
/*
* Lock out modifications to both the RT bitmap and summary inodes
*/
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: Transaction log reservation overrun when fallocating realtime file
2019-11-27 0:34 ` Darrick J. Wong
@ 2019-12-02 19:32 ` Omar Sandoval
2019-12-02 21:51 ` Dave Chinner
1 sibling, 0 replies; 7+ messages in thread
From: Omar Sandoval @ 2019-12-02 19:32 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-xfs
On Tue, Nov 26, 2019 at 04:34:26PM -0800, Darrick J. Wong wrote:
> On Tue, Nov 26, 2019 at 12:27:14PM -0800, Omar Sandoval wrote:
> > Hello,
> >
> > The following reproducer results in a transaction log overrun warning
> > for me:
> >
> > mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
> > mount -o rtdev=/dev/vdc /dev/vdb /mnt
> > fallocate -l 4G /mnt/foo
> >
> > I've attached the full dmesg output. My guess at the problem is that the
> > tr_write reservation used by xfs_alloc_file_space is not taking the realtime
> > bitmap and realtime summary inodes into account (inode numbers 129 and 130 on
> > this filesystem, which I do see in some of the log items). However, I'm not
> > familiar enough with the XFS transaction guts to confidently fix this. Can
> > someone please help me out?
>
> Hmm...
>
> /*
> * In a write transaction we can allocate a maximum of 2
> * extents. This gives:
> * the inode getting the new extents: inode size
> * the inode's bmap btree: max depth * block size
> * the agfs of the ags from which the extents are allocated: 2 * sector
> * the superblock free block counter: sector size
> * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> * And the bmap_finish transaction can free bmap blocks in a join:
> * the agfs of the ags containing the blocks: 2 * sector size
> * the agfls of the ags containing the blocks: 2 * sector size
> * the super block free block counter: sector size
> * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> */
> STATIC uint
> xfs_calc_write_reservation(...);
>
> So this means that the rt allocator can burn through at most ...
> 1 ext * 2 trees * (2 * maxdepth - 1) * blocksize
> ... worth of log reservation as part of setting bits in the rtbitmap and
> fiddling with the rtsummary information.
>
> Instead, 4GB of 4k rt extents == 1 million rtexts to mark in use, which
> is 131072 bytes of rtbitmap to log, and *kaboom* there goes the 109K log
> reservation.
>
> So I think you're right, and the fix is probably? to cap ralen further
> in xfs_bmap_rtalloc(). Does the following patch fix it?
>
> --D
Yup, that seems to fix it.
Reported-and-tested-by: Omar Sandoval <osandov@fb.com>
Thanks, Darrick!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transaction log reservation overrun when fallocating realtime file
2019-11-27 0:34 ` Darrick J. Wong
2019-12-02 19:32 ` Omar Sandoval
@ 2019-12-02 21:51 ` Dave Chinner
2019-12-03 2:45 ` Darrick J. Wong
1 sibling, 1 reply; 7+ messages in thread
From: Dave Chinner @ 2019-12-02 21:51 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: Omar Sandoval, linux-xfs
On Tue, Nov 26, 2019 at 04:34:26PM -0800, Darrick J. Wong wrote:
> On Tue, Nov 26, 2019 at 12:27:14PM -0800, Omar Sandoval wrote:
> > Hello,
> >
> > The following reproducer results in a transaction log overrun warning
> > for me:
> >
> > mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
> > mount -o rtdev=/dev/vdc /dev/vdb /mnt
> > fallocate -l 4G /mnt/foo
> >
> > I've attached the full dmesg output. My guess at the problem is that the
> > tr_write reservation used by xfs_alloc_file_space is not taking the realtime
> > bitmap and realtime summary inodes into account (inode numbers 129 and 130 on
> > this filesystem, which I do see in some of the log items). However, I'm not
> > familiar enough with the XFS transaction guts to confidently fix this. Can
> > someone please help me out?
>
> Hmm...
>
> /*
> * In a write transaction we can allocate a maximum of 2
> * extents. This gives:
> * the inode getting the new extents: inode size
> * the inode's bmap btree: max depth * block size
> * the agfs of the ags from which the extents are allocated: 2 * sector
> * the superblock free block counter: sector size
> * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> * And the bmap_finish transaction can free bmap blocks in a join:
> * the agfs of the ags containing the blocks: 2 * sector size
> * the agfls of the ags containing the blocks: 2 * sector size
> * the super block free block counter: sector size
> * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> */
> STATIC uint
> xfs_calc_write_reservation(...);
>
> So this means that the rt allocator can burn through at most ...
> 1 ext * 2 trees * (2 * maxdepth - 1) * blocksize
> ... worth of log reservation as part of setting bits in the rtbitmap and
> fiddling with the rtsummary information.
>
> Instead, 4GB of 4k rt extents == 1 million rtexts to mark in use, which
> is 131072 bytes of rtbitmap to log, and *kaboom* there goes the 109K log
> reservation.
Ok, if that's the case, we still need to be able to allocate MAXEXTLEN in
a single transaction. That's 2^21 filesystem blocks, which at most
is 2^21 rtexts.
Hence I think we probably should have a separate rt-write
reservation that handles this case, and we use that for allocation
on rt devices rather than the bt-based allocation reservation.
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> xfs: cap realtime allocation length to something we can log
>
> Omar Sandoval reported that a 4G fallocate on the realtime device causes
> filesystem shutdowns due to a log reservation overflow that happens when
> we log the rtbitmap updates.
>
> The tr_write transaction reserves enough log reservation to handle a
> full splits of both free space btrees, so cap the rt allocation at that
> number of bits.
>
> "The following reproducer results in a transaction log overrun warning
> for me:
>
> mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
> mount -o rtdev=/dev/vdc /dev/vdb /mnt
> fallocate -l 4G /mnt/foo
>
> Reported-by: Omar Sandoval <osandov@osandov.com>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
> fs/xfs/xfs_bmap_util.c | 23 +++++++++++++++++++++++
> 1 file changed, 23 insertions(+)
>
> diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c
> index 49d7b530c8f7..15c4e2790de3 100644
> --- a/fs/xfs/xfs_bmap_util.c
> +++ b/fs/xfs/xfs_bmap_util.c
> @@ -69,6 +69,26 @@ xfs_zero_extent(
> }
>
> #ifdef CONFIG_XFS_RT
> +/*
> + * tr_write allows for one full split in the bnobt and cntbt to record the
> + * allocation, and that's how many bits of rtbitmap we can log to the
> + * transaction. We leave one full block's worth of log space to handle the
> + * rtsummary update, though that's probably overkill.
> + */
> +static inline uint64_t
> +xfs_bmap_rtalloc_max(
> + struct xfs_mount *mp)
> +{
> + uint64_t max_rtbitmap;
> +
> + max_rtbitmap = xfs_allocfree_log_count(mp, 1) - 1;
> + max_rtbitmap *= XFS_FSB_TO_B(mp, 1);
> + max_rtbitmap *= NBBY;
> + max_rtbitmap *= mp->m_sb.sb_rextsize;
I can see how this works, but it strikes me as a bit of a hack. We
calculate the worst case reservations up front to avoid having to
play games like this in the code. Hence I think the correct thing to
do is fix the reservation to ensure we can do MAXEXTLEN allocations
without overruns...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transaction log reservation overrun when fallocating realtime file
2019-12-02 21:51 ` Dave Chinner
@ 2019-12-03 2:45 ` Darrick J. Wong
2019-12-03 21:31 ` Dave Chinner
0 siblings, 1 reply; 7+ messages in thread
From: Darrick J. Wong @ 2019-12-03 2:45 UTC (permalink / raw)
To: Dave Chinner; +Cc: Omar Sandoval, linux-xfs
On Tue, Dec 03, 2019 at 08:51:13AM +1100, Dave Chinner wrote:
> On Tue, Nov 26, 2019 at 04:34:26PM -0800, Darrick J. Wong wrote:
> > On Tue, Nov 26, 2019 at 12:27:14PM -0800, Omar Sandoval wrote:
> > > Hello,
> > >
> > > The following reproducer results in a transaction log overrun warning
> > > for me:
> > >
> > > mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
> > > mount -o rtdev=/dev/vdc /dev/vdb /mnt
> > > fallocate -l 4G /mnt/foo
> > >
> > > I've attached the full dmesg output. My guess at the problem is that the
> > > tr_write reservation used by xfs_alloc_file_space is not taking the realtime
> > > bitmap and realtime summary inodes into account (inode numbers 129 and 130 on
> > > this filesystem, which I do see in some of the log items). However, I'm not
> > > familiar enough with the XFS transaction guts to confidently fix this. Can
> > > someone please help me out?
> >
> > Hmm...
> >
> > /*
> > * In a write transaction we can allocate a maximum of 2
> > * extents. This gives:
> > * the inode getting the new extents: inode size
> > * the inode's bmap btree: max depth * block size
> > * the agfs of the ags from which the extents are allocated: 2 * sector
> > * the superblock free block counter: sector size
> > * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> > * And the bmap_finish transaction can free bmap blocks in a join:
> > * the agfs of the ags containing the blocks: 2 * sector size
> > * the agfls of the ags containing the blocks: 2 * sector size
> > * the super block free block counter: sector size
> > * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> > */
> > STATIC uint
> > xfs_calc_write_reservation(...);
> >
> > So this means that the rt allocator can burn through at most ...
> > 1 ext * 2 trees * (2 * maxdepth - 1) * blocksize
> > ... worth of log reservation as part of setting bits in the rtbitmap and
> > fiddling with the rtsummary information.
> >
> > Instead, 4GB of 4k rt extents == 1 million rtexts to mark in use, which
> > is 131072 bytes of rtbitmap to log, and *kaboom* there goes the 109K log
> > reservation.
>
> Ok, if that's the case, we still need to be able to allocate MAXEXTLEN in
> a single transaction. That's 2^21 filesystem blocks, which at most
> is 2^21 rtexts.
>
> Hence I think we probably should have a separate rt-write
> reservation that handles this case, and we use that for allocation
> on rt devices rather than the bt-based allocation reservation.
2^21 rtexts is ... 2^18 bytes worth of rtbitmap block, which implies a
transaction reservation of around ... ~300K? I guess I'll have to go
play with xfs_db to see how small of a datadev you can make before that
causes us to fail the minimum log size checks.
As you said on IRC, it probably won't affect /most/ setups... but I
don't want to run around increasing support calls either. Even if most
distributors don't turn on rt support.
>
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> >
> > xfs: cap realtime allocation length to something we can log
> >
> > Omar Sandoval reported that a 4G fallocate on the realtime device causes
> > filesystem shutdowns due to a log reservation overflow that happens when
> > we log the rtbitmap updates.
> >
> > The tr_write transaction reserves enough log reservation to handle a
> > full splits of both free space btrees, so cap the rt allocation at that
> > number of bits.
> >
> > "The following reproducer results in a transaction log overrun warning
> > for me:
> >
> > mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
> > mount -o rtdev=/dev/vdc /dev/vdb /mnt
> > fallocate -l 4G /mnt/foo
> >
> > Reported-by: Omar Sandoval <osandov@osandov.com>
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> > fs/xfs/xfs_bmap_util.c | 23 +++++++++++++++++++++++
> > 1 file changed, 23 insertions(+)
> >
> > diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c
> > index 49d7b530c8f7..15c4e2790de3 100644
> > --- a/fs/xfs/xfs_bmap_util.c
> > +++ b/fs/xfs/xfs_bmap_util.c
> > @@ -69,6 +69,26 @@ xfs_zero_extent(
> > }
> >
> > #ifdef CONFIG_XFS_RT
> > +/*
> > + * tr_write allows for one full split in the bnobt and cntbt to record the
> > + * allocation, and that's how many bits of rtbitmap we can log to the
> > + * transaction. We leave one full block's worth of log space to handle the
> > + * rtsummary update, though that's probably overkill.
> > + */
> > +static inline uint64_t
> > +xfs_bmap_rtalloc_max(
> > + struct xfs_mount *mp)
> > +{
> > + uint64_t max_rtbitmap;
> > +
> > + max_rtbitmap = xfs_allocfree_log_count(mp, 1) - 1;
> > + max_rtbitmap *= XFS_FSB_TO_B(mp, 1);
> > + max_rtbitmap *= NBBY;
> > + max_rtbitmap *= mp->m_sb.sb_rextsize;
>
> I can see how this works, but it strikes me as a bit of a hack. We
> calculate the worst case reservations up front to avoid having to
> play games like this in the code. Hence I think the correct thing to
> do is fix the reservation to ensure we can do MAXEXTLEN allocations
> without overruns...
Yeah...
--D
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transaction log reservation overrun when fallocating realtime file
2019-12-03 2:45 ` Darrick J. Wong
@ 2019-12-03 21:31 ` Dave Chinner
2019-12-04 16:31 ` Darrick J. Wong
0 siblings, 1 reply; 7+ messages in thread
From: Dave Chinner @ 2019-12-03 21:31 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: Omar Sandoval, linux-xfs
On Mon, Dec 02, 2019 at 06:45:26PM -0800, Darrick J. Wong wrote:
> On Tue, Dec 03, 2019 at 08:51:13AM +1100, Dave Chinner wrote:
> > On Tue, Nov 26, 2019 at 04:34:26PM -0800, Darrick J. Wong wrote:
> > > On Tue, Nov 26, 2019 at 12:27:14PM -0800, Omar Sandoval wrote:
> > > > Hello,
> > > >
> > > > The following reproducer results in a transaction log overrun warning
> > > > for me:
> > > >
> > > > mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
> > > > mount -o rtdev=/dev/vdc /dev/vdb /mnt
> > > > fallocate -l 4G /mnt/foo
> > > >
> > > > I've attached the full dmesg output. My guess at the problem is that the
> > > > tr_write reservation used by xfs_alloc_file_space is not taking the realtime
> > > > bitmap and realtime summary inodes into account (inode numbers 129 and 130 on
> > > > this filesystem, which I do see in some of the log items). However, I'm not
> > > > familiar enough with the XFS transaction guts to confidently fix this. Can
> > > > someone please help me out?
> > >
> > > Hmm...
> > >
> > > /*
> > > * In a write transaction we can allocate a maximum of 2
> > > * extents. This gives:
> > > * the inode getting the new extents: inode size
> > > * the inode's bmap btree: max depth * block size
> > > * the agfs of the ags from which the extents are allocated: 2 * sector
> > > * the superblock free block counter: sector size
> > > * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> > > * And the bmap_finish transaction can free bmap blocks in a join:
> > > * the agfs of the ags containing the blocks: 2 * sector size
> > > * the agfls of the ags containing the blocks: 2 * sector size
> > > * the super block free block counter: sector size
> > > * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> > > */
> > > STATIC uint
> > > xfs_calc_write_reservation(...);
> > >
> > > So this means that the rt allocator can burn through at most ...
> > > 1 ext * 2 trees * (2 * maxdepth - 1) * blocksize
> > > ... worth of log reservation as part of setting bits in the rtbitmap and
> > > fiddling with the rtsummary information.
> > >
> > > Instead, 4GB of 4k rt extents == 1 million rtexts to mark in use, which
> > > is 131072 bytes of rtbitmap to log, and *kaboom* there goes the 109K log
> > > reservation.
> >
> > Ok, if that's the case, we still need to be able to allocate MAXEXTLEN in
> > a single transaction. That's 2^21 filesystem blocks, which at most
> > is 2^21 rtexts.
> >
> > Hence I think we probably should have a separate rt-write
> > reservation that handles this case, and we use that for allocation
> > on rt devices rather than the bt-based allocation reservation.
>
> 2^21 rtexts is ... 2^18 bytes worth of rtbitmap block, which implies a
> transaction reservation of around ... ~300K? I guess I'll have to go
> play with xfs_db to see how small of a datadev you can make before that
> causes us to fail the minimum log size checks.
Keep in mind that rtextsz is often larger than a single filesystem
block, so the bitmap size rapidly reduces as rtextsz goes up.
> As you said on IRC, it probably won't affect /most/ setups... but I
> don't want to run around increasing support calls either. Even if most
> distributors don't turn on rt support.
Sure, we can limit the size of the allocation based on the
transaction reservation limits, but I suspect this will only affect
filesystems with really, really small data devices that result in a
<10MB default log size. I don't think there is that many of these
around in production....
I'd prefer to fix the transaction size, and then if people start
reporting that the log size is too small, we can then
limit the extent size allocation and transaction reservation based
on the (tiny) log size we read out of the superblock...
Alternatively, we could implement log growing :)
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Transaction log reservation overrun when fallocating realtime file
2019-12-03 21:31 ` Dave Chinner
@ 2019-12-04 16:31 ` Darrick J. Wong
0 siblings, 0 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-12-04 16:31 UTC (permalink / raw)
To: Dave Chinner; +Cc: Omar Sandoval, linux-xfs
On Wed, Dec 04, 2019 at 08:31:17AM +1100, Dave Chinner wrote:
> On Mon, Dec 02, 2019 at 06:45:26PM -0800, Darrick J. Wong wrote:
> > On Tue, Dec 03, 2019 at 08:51:13AM +1100, Dave Chinner wrote:
> > > On Tue, Nov 26, 2019 at 04:34:26PM -0800, Darrick J. Wong wrote:
> > > > On Tue, Nov 26, 2019 at 12:27:14PM -0800, Omar Sandoval wrote:
> > > > > Hello,
> > > > >
> > > > > The following reproducer results in a transaction log overrun warning
> > > > > for me:
> > > > >
> > > > > mkfs.xfs -f -r rtdev=/dev/vdc -d rtinherit=1 -m reflink=0 /dev/vdb
> > > > > mount -o rtdev=/dev/vdc /dev/vdb /mnt
> > > > > fallocate -l 4G /mnt/foo
> > > > >
> > > > > I've attached the full dmesg output. My guess at the problem is that the
> > > > > tr_write reservation used by xfs_alloc_file_space is not taking the realtime
> > > > > bitmap and realtime summary inodes into account (inode numbers 129 and 130 on
> > > > > this filesystem, which I do see in some of the log items). However, I'm not
> > > > > familiar enough with the XFS transaction guts to confidently fix this. Can
> > > > > someone please help me out?
> > > >
> > > > Hmm...
> > > >
> > > > /*
> > > > * In a write transaction we can allocate a maximum of 2
> > > > * extents. This gives:
> > > > * the inode getting the new extents: inode size
> > > > * the inode's bmap btree: max depth * block size
> > > > * the agfs of the ags from which the extents are allocated: 2 * sector
> > > > * the superblock free block counter: sector size
> > > > * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> > > > * And the bmap_finish transaction can free bmap blocks in a join:
> > > > * the agfs of the ags containing the blocks: 2 * sector size
> > > > * the agfls of the ags containing the blocks: 2 * sector size
> > > > * the super block free block counter: sector size
> > > > * the allocation btrees: 2 exts * 2 trees * (2 * max depth - 1) * block size
> > > > */
> > > > STATIC uint
> > > > xfs_calc_write_reservation(...);
> > > >
> > > > So this means that the rt allocator can burn through at most ...
> > > > 1 ext * 2 trees * (2 * maxdepth - 1) * blocksize
> > > > ... worth of log reservation as part of setting bits in the rtbitmap and
> > > > fiddling with the rtsummary information.
> > > >
> > > > Instead, 4GB of 4k rt extents == 1 million rtexts to mark in use, which
> > > > is 131072 bytes of rtbitmap to log, and *kaboom* there goes the 109K log
> > > > reservation.
> > >
> > > Ok, if that's the case, we still need to be able to allocate MAXEXTLEN in
> > > a single transaction. That's 2^21 filesystem blocks, which at most
> > > is 2^21 rtexts.
> > >
> > > Hence I think we probably should have a separate rt-write
> > > reservation that handles this case, and we use that for allocation
> > > on rt devices rather than the bt-based allocation reservation.
> >
> > 2^21 rtexts is ... 2^18 bytes worth of rtbitmap block, which implies a
> > transaction reservation of around ... ~300K? I guess I'll have to go
> > play with xfs_db to see how small of a datadev you can make before that
> > causes us to fail the minimum log size checks.
>
> Keep in mind that rtextsz is often larger than a single filesystem
> block, so the bitmap size rapidly reduces as rtextsz goes up.
>
> > As you said on IRC, it probably won't affect /most/ setups... but I
> > don't want to run around increasing support calls either. Even if most
> > distributors don't turn on rt support.
>
> Sure, we can limit the size of the allocation based on the
> transaction reservation limits, but I suspect this will only affect
> filesystems with really, really small data devices that result in a
> <10MB default log size. I don't think there is that many of these
> around in production....
>
> I'd prefer to fix the transaction size, and then if people start
> reporting that the log size is too small, we can then
> limit the extent size allocation and transaction reservation based
> on the (tiny) log size we read out of the superblock...
Ok, I'll work on that.
> Alternatively, we could implement log growing :)
Heh. Wandering logs?
--D
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-12-04 16:33 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-26 20:27 Transaction log reservation overrun when fallocating realtime file Omar Sandoval
2019-11-27 0:34 ` Darrick J. Wong
2019-12-02 19:32 ` Omar Sandoval
2019-12-02 21:51 ` Dave Chinner
2019-12-03 2:45 ` Darrick J. Wong
2019-12-03 21:31 ` Dave Chinner
2019-12-04 16:31 ` Darrick J. Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).