[BUG] xfs_corruption_error after creating a swap file

[BUG] xfs_corruption_error after creating a swap file

[Bastien Traverse]

Hello everyone,

A couple of weeks back I got an xfs_corruption_error stack trace on my 
rootfs on Arch Linux, a few minutes after creating a swap file an 
enabling it. Here is the process I followed to do so:

    fallocate -l 4G /swapfile
    chmod 600 /swapfile
    mkswap /swapfile
    swapon /swapfile
    echo "/swapfile none swap defaults 0 0" >> /etc/fstab

And the trace appeared a few minutes later, without me doing much at 
that moment:

```
déc. 27 21:51:01 kernel: XFS (dm-0): Internal error !link at line 491 
of file fs/xfs/xfs_iops.c. Caller xfs_vn_get_link_inline+0x12/0x50 [xfs]
déc. 27 21:51:01 kernel: CPU: 4 PID: 1923 Comm: gmain Not tainted 
5.9.14-arch1-1 #1
déc. 27 21:51:01 kernel: Hardware name: ASUSTeK COMPUTER INC. ZenBook 
UX534FAC_UX534FA/UX534FAC, BIOS UX534FAC.307 04/20/2020
déc. 27 21:51:01 kernel: Call Trace:
déc. 27 21:51:01 kernel: dump_stack+0x6b/0x83
déc. 27 21:51:01 kernel: xfs_corruption_error+0x85/0x90 [xfs]
déc. 27 21:51:01 kernel: ? xfs_vn_get_link_inline+0x12/0x50 [xfs]
déc. 27 21:51:01 kernel: xfs_vn_get_link_inline+0x44/0x50 [xfs]
déc. 27 21:51:01 kernel: ? xfs_vn_get_link_inline+0x12/0x50 [xfs]
déc. 27 21:51:01 kernel: step_into+0x579/0x720
déc. 27 21:51:01 kernel: ? xfs_vn_get_link+0x90/0x90 [xfs]
déc. 27 21:51:01 kernel: walk_component+0x83/0x1b0
déc. 27 21:51:01 kernel: path_lookupat+0x5b/0x190
déc. 27 21:51:01 kernel: filename_lookup+0xbe/0x1d0
déc. 27 21:51:01 kernel: vfs_statx+0x8f/0x140
déc. 27 21:51:01 kernel: __do_sys_newstat+0x47/0x80
déc. 27 21:51:01 kernel: do_syscall_64+0x33/0x40
déc. 27 21:51:01 kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
déc. 27 21:51:01 kernel: RIP: 0033:0x7f5192dcc25a
déc. 27 21:51:01 kernel: Code: 00 00 75 05 48 83 c4 18 c3 e8 b2 f7 01 
00 66 90 f3 0f 1e fa 41 89 f8 48 89 f7 48 89 d6 41 83 f8 01 77 2d b8 04 
00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 06 c3 0f 1f 44 00 00 48 8b 15 e1 
1b 0d 00 f7
déc. 27 21:51:01 kernel: RSP: 002b:00007f518f82fa68 EFLAGS: 00000246 
ORIG_RAX: 0000000000000004
déc. 27 21:51:01 kernel: RAX: ffffffffffffffda RBX: 00007f5188001650 
RCX: 00007f5192dcc25a
déc. 27 21:51:01 kernel: RDX: 00007f518f82fa70 RSI: 00007f518f82fa70 
RDI: 000055909d4528a0
déc. 27 21:51:01 kernel: RBP: 000055909d4528a0 R08: 0000000000000001 
R09: 00007f5192e630c0
déc. 27 21:51:01 kernel: R10: 00007f5192e62fc0 R11: 0000000000000246 
R12: 0000000000000001
déc. 27 21:51:01 kernel: R13: 000055909d4229b0 R14: 000055909d4229b0 
R15: 000055909d457070
déc. 27 21:51:01 kernel: XFS (dm-0): Corruption detected. Unmount and 
run xfs_repair
```

Stack is a late-2019 laptop (Asus ZenBook UX534FA) with an Intel 660p 
512GB NVMe drive, running Arch Linux with one big LUKS2-encrypted 
partition (besides the ESP) directly formatted with XFS; kernel version 
at the time of the bug was :

    5.9.14-arch1-1 (linux@archlinux) (gcc (GCC) 10.2.0, GNU ld (GNU 
Binutils) 2.35.1) #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000

The filesystem was created with the November 2020 Arch Linux ISO 
(kernel 5.9.2) and command `mkfs.xfs -m rmapbt=1 -L system 
/dev/mapper/root`.  Here is the xfs_info output for said rootfs:

```
meta-data=/dev/mapper/root isize=512 agcount=4, agsize=31224405 blks
         = sectsz=512 attr=2, projid32bit=1
         = crc=1 finobt=1, sparse=1, rmapbt=1
         = reflink=1
data = bsize=4096 blocks=124897617, imaxpct=25
         = sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=60985, version=2
         = sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
```

dm-crypt is configured with `allow_discards` and periodic fstrim method 
is used; the fs is mounted only with `noatime` option besides the 
defaults:

    % findmnt /
TARGET SOURCE FSTYPE OPTIONS
/ /dev/mapper/root xfs 
rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota

And that's all I can think of to provide for context.

I rebooted on a Grml live ISO and ran xfs_repair, which found a few 
thing that I unfortunately didn't/couldn't copy (virtual console 
scrolling and the output disappeared...).
I haven't had any issues since, or none that I realized at least.

That's all for this report, if you have any hints please say so!

Thanks for reading,

PS: please Cc me when replying as I am not subscribed to the list!

Cheers,
Bastien

Re: [BUG] xfs_corruption_error after creating a swap file

[Dave Chinner]

On Tue, Jan 12, 2021 at 10:06:29PM +0100, Bastien Traverse wrote:
> Hello everyone,
> 
> A couple of weeks back I got an xfs_corruption_error stack trace on my
> rootfs on Arch Linux, a few minutes after creating a swap file an enabling
> it. Here is the process I followed to do so:
> 
>    fallocate -l 4G /swapfile
>    chmod 600 /swapfile
>    mkswap /swapfile
>    swapon /swapfile
>    echo "/swapfile none swap defaults 0 0" >> /etc/fstab
> 
> And the trace appeared a few minutes later, without me doing much at that
> moment:

Sounds like:

commit 41663430588c737dd735bad5a0d1ba325dcabd59
Author: Gao Xiang <hsiangkao@redhat.com>
Date:   Fri Sep 25 21:19:01 2020 -0700

    mm, THP, swap: fix allocating cluster for swapfile by mistake
    
    SWP_FS is used to make swap_{read,write}page() go through the
    filesystem, and it's only used for swap files over NFS.  So, !SWP_FS
    means non NFS for now, it could be either file backed or device backed.
    Something similar goes with legacy SWP_FILE.
    
    So in order to achieve the goal of the original patch, SWP_BLKDEV should
    be used instead.
    
    FS corruption can be observed with SSD device + XFS + fragmented
    swapfile due to CONFIG_THP_SWAP=y.
.....

But I thought that was fixed in 5.9-rc7 so should be in your kernel.
Can you confirm that your kernel has this fix?

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [BUG] xfs_corruption_error after creating a swap file

[Bastien Traverse]

Le mer. 13 janv. 2021 à 9:25, Dave Chinner <david@fromorbit.com> a 
écrit :

 > But I thought that was fixed in 5.9-rc7 so should be in your kernel.
 > Can you confirm that your kernel has this fix?

This commit does appear in Arch kernel repo, so I can only suppose it's 
in: 
https://git.archlinux.org/linux.git/commit/?id=41663430588c737dd735bad5a0d1ba325dcabd59

Arch tends to stay close to upstream and I am using the standard kernel.

Thanks,
Bastien

Re: [BUG] xfs_corruption_error after creating a swap file

[Dave Chinner]

On Tue, Jan 12, 2021 at 11:50:19PM +0100, Bastien Traverse wrote:
> Le mer. 13 janv. 2021 à 9:25, Dave Chinner <david@fromorbit.com> a
> écrit :
> 
> > But I thought that was fixed in 5.9-rc7 so should be in your
> > kernel.  Can you confirm that your kernel has this fix?
> 
> This commit does appear in Arch kernel repo, so I can only suppose
> it's in:
> https://git.archlinux.org/linux.git/commit/?id=41663430588c737dd735bad5a0d1ba325dcabd59
> 
> Arch tends to stay close to upstream and I am using the standard
> kernel.

Ok. Given that xfs_repair apparently found and corrected some
corruption and it hasn't happened since, I'm not sure that there is
much we can go on here. It may have been a pre-existing corruption,
and the swapfile thing is a red herring. If it happens again, let us
know and we can see if we can get more information out of your
system about it.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com