* Unmountable XFS file system after runnig stress-ng
@ 2019-07-25 17:21 Paul Menzel
2019-07-27 0:24 ` Luis Chamberlain
0 siblings, 1 reply; 3+ messages in thread
From: Paul Menzel @ 2019-07-25 17:21 UTC (permalink / raw)
To: linux-xfs
[-- Attachment #1: Type: text/plain, Size: 13315 bytes --]
Dear Linux folks,
With Linux 4.19.57 I ran `sudo ./stress-ng -a 10` [1], and it looks like the
file system got corrupted.
Rebooting and trying to mount it shows the errors below.
```
[94273.250116] hrtimer: interrupt took 2941668 ns
[448632.389303] scsi 0:2:1:0: Direct-Access ATA HUS722T2TALA600 MU03 PQ: 0 ANSI: 6
[448632.407830] sd 0:2:1:0: Attached scsi generic sg3 type 0
[448632.416356] sd 0:2:1:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[448632.431927] sd 0:2:1:0: [sdd] Write Protect is off
[448632.436823] sd 0:2:1:0: [sdd] Mode Sense: 9b 00 10 08
[448632.443394] sd 0:2:1:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA
[448632.528883] sdd: sdd1 sdd2 sdd3
[448632.646750] sd 0:2:1:0: [sdd] Attached SCSI disk
[448771.414262] XFS (sdd1): Mounting V5 Filesystem
[448771.551801] XFS (sdd1): Starting recovery (logdev: internal)
[448772.041683] XFS (sdd1): xfs_do_force_shutdown(0x8) called from line 368 of file fs/xfs/xfs_trans.c. Return address = 00000000e24e28d1
[448772.054016] XFS (sdd1): Corruption of in-memory data detected. Shutting down filesystem
[448772.062233] XFS (sdd1): Please umount the filesystem and rectify the problem(s)
[448772.069685] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.077119] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.085420] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.092846] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.101150] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.108576] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.116879] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.124308] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.132611] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.140037] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.148342] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.155767] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.164070] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.171502] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.179798] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.187233] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.195530] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.202964] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.211266] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.218699] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.227001] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.234437] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.242730] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.250160] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.258470] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.265898] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.274205] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.282576] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.291651] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.299795] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.308805] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.316949] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.325959] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.334112] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.343126] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.351273] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.360286] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.368432] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.377458] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.385610] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.395360] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.403540] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.412571] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.420735] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.429771] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.437912] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.446918] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.455058] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.464076] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.472217] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.481231] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.489367] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.498383] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.506529] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.515546] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.523688] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.532707] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.540847] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.549857] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.558003] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.567015] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.575154] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.584170] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.592311] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.601327] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.609462] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.618472] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.626615] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.635631] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.643773] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.652784] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.660925] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.669943] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.678076] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.687093] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.695228] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.704236] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.712379] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.721393] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.729530] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.738546] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.746683] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.755700] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.763846] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.772859] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.780994] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.790001] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.798136] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.807151] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.815299] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.824313] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.832456] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.841469] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.849607] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.858624] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.866763] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.875775] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.883909] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.892916] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.901053] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.910062] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.918202] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.927213] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.935355] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.944371] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.952517] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.961535] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.969678] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.978692] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448772.986837] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448772.995852] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.003987] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.013003] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.021138] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.030157] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.038299] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.047314] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.055458] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.064468] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.072612] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.081627] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.089772] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.098787] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.106930] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.115934] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.124071] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.133088] XFS (sdd1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5.
[448773.141234] XFS (sdd1): xlog_recover_clear_agi_bucket: failed to clear agi 2. Continuing.
[448773.150249] XFS (sdd1): Ending recovery (logdev: internal)
[448773.156875] XFS (sdd1): Error -5 reserving per-AG metadata reserve pool.
[448773.164404] XFS (sdd1): xfs_do_force_shutdown(0x8) called from line 548 of file fs/xfs/xfs_fsops.c. Return address = 00000000030066f7
```
Mounting did not work, but `xfs-report -L /dev/sdd1` fixed it.
I would put this issue to rest, and just wanted to document it
publicly. Please tell me if I should do anything else.
Kind regards,
Paul
[1]: https://kernel.ubuntu.com/~cking/stress-ng/
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5174 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Unmountable XFS file system after runnig stress-ng
2019-07-25 17:21 Unmountable XFS file system after runnig stress-ng Paul Menzel
@ 2019-07-27 0:24 ` Luis Chamberlain
[not found] ` <CA+VdTb8ZPLi0Eiq9SEhem4aEjWJ3rCn+NQ_vZrpc9mXo_WuDDQ@mail.gmail.com>
0 siblings, 1 reply; 3+ messages in thread
From: Luis Chamberlain @ 2019-07-27 0:24 UTC (permalink / raw)
To: Paul Menzel; +Cc: linux-xfs
On Thu, Jul 25, 2019 at 07:21:43PM +0200, Paul Menzel wrote:
> Dear Linux folks,
Hey Paul, thanks for reporting this!
> With Linux 4.19.57
v4.19.57 lacks a series of fixes I posted a few days ago.
They were merged on v4.19.61. You can try that. If that fails
another small series of fixes were posted recently which are
not merged yet ons table v4.19.y, but you can use this git
branch:
https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-stable.git/log/?h=20190726-linux-xfs-4.19.y-v1
> I ran `sudo ./stress-ng -a 10` [1], and it looks like the
> file system got corrupted.
Which version of stress-ng did you use?
Can you try also on the latest Linus' tree?
If brave enough, you can try the xfs for-next branch
as well:
https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git/log/?h=for-next
If the issue is still creeping up on the above stable kernels
can you create a bug entry on bugzilla.kernel.org to track this?
Luis
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Unmountable XFS file system after runnig stress-ng
[not found] ` <CA+VdTb8ZPLi0Eiq9SEhem4aEjWJ3rCn+NQ_vZrpc9mXo_WuDDQ@mail.gmail.com>
@ 2019-07-27 21:39 ` Paul Menzel
0 siblings, 0 replies; 3+ messages in thread
From: Paul Menzel @ 2019-07-27 21:39 UTC (permalink / raw)
To: Rob Townley, Luis Chamberlain; +Cc: linux-xfs
Dear Rob,
On 27.07.19 13:34, Rob Townley wrote:
> Is there ECC RAM in this PC?
>
> “Corruption of in-memory data detected. Shutting down filesystem“
Yes, there is ECC memory in that system.
Kind regards,
Paul
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-07-27 21:39 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-25 17:21 Unmountable XFS file system after runnig stress-ng Paul Menzel
2019-07-27 0:24 ` Luis Chamberlain
[not found] ` <CA+VdTb8ZPLi0Eiq9SEhem4aEjWJ3rCn+NQ_vZrpc9mXo_WuDDQ@mail.gmail.com>
2019-07-27 21:39 ` Paul Menzel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).