From: Peter Chant <pete@petezilla.co.uk>
To: Btrfs BTRFS <linux-btrfs@vger.kernel.org>
Subject: Chasing IO errors. BTRFS: error (device dm-2) in btrfs_run_delayed_refs:2907: errno=-5 IO failure
Date: Tue, 20 Aug 2019 21:36:03 +0100 [thread overview]
Message-ID: <fc2b166a-4466-4a5a-ee88-da5e57ee89b6@petezilla.co.uk> (raw)
Chasing IO errors. BTRFS: error (device dm-2) in
btrfs_run_delayed_refs:2907: errno=-5 IO failure
I've just had an odd one.
Over the last few days I've noticed a file system blocking, if that is
the correct term, and this morning go read only. This resulted in a lot
of checksum errors.
Having spotted the file system go read only in the logs and then noted
the error message in the subject shortly after booting I assumed a
hardware error and changed the SATA cable. That had no effect so I
isolated the disk and mounted the respective file system degraded.
Shortly after mounting the degraded file system I had the same error
again. So I unmounted the file system edited fstab and swapped the disk
which I though originally had the error with the one now showing an error.
The file system is btrfs, kernel 5.2.9, RAID 1 with three WD reds of 3,
3 and 4 TB. btrfs is on top of luks.
The original 'blocking' behaviour seemed to manifest itself as I
upgraded the kernel to 5.2.5 or 5.2.7 a day or two ago. So I tried
5.1.21 to see if that made a difference when the error was showing. It
did not. Yesterday I had a backup with rsync, started early in the
morning that should take minutes to complete still running 8h later with
two CPU cores maxed. Up until I had the file system go read only I had
not noticed anything amiss in the logs, but to be honest, I'd not looked
very hard.
smartctl did not show anything amiss with the drives.
Does this sound like a hardware error? I have ordered a replacement
drive, if it is not needed as a replacement I will put it into a
homebrew NAS.
I've hit the issue again. Hopefully the system is up long enough to
post this.
I'm a bit worried that trying to track this down disconnecting a disk at
a time I might hit the btrfs split brain issue.
Pete
next reply other threads:[~2019-08-20 21:10 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-20 20:36 Peter Chant [this message]
2019-08-20 21:59 ` Chasing IO errors. BTRFS: error (device dm-2) in btrfs_run_delayed_refs:2907: errno=-5 IO failure Chris Murphy
2019-08-20 23:47 ` Peter Chant
2019-08-21 8:05 ` Peter Chant
2019-08-21 7:29 ` Qu Wenruo
2019-08-21 21:38 ` Peter Chant
2019-08-21 23:32 ` Qu Wenruo
2019-09-08 7:14 ` Pete
2019-08-20 23:45 ` Qu Wenruo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fc2b166a-4466-4a5a-ee88-da5e57ee89b6@petezilla.co.uk \
--to=pete@petezilla.co.uk \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).