* Unmountable root partition
@ 2018-07-31 18:03 Cerem Cem ASLAN
2018-07-31 23:04 ` Hans van Kranenburg
2018-08-01 5:32 ` Chris Murphy
0 siblings, 2 replies; 6+ messages in thread
From: Cerem Cem ASLAN @ 2018-07-31 18:03 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 2001 bytes --]
Hi,
I'm having trouble with my server setup, which contains a BTRFS root
partition on top of LVM on top of LUKS partition.
Yesterday server was shut down unexpectedly. I booted the system with a
pendrive which contains Debian 4.9.18 and tried to mount the BTRFS root
partition manually.
1. cryptsetup luksOpen /dev/sda5
Seems to decrypt the partition (there are no errors)
2. /dev/mapper/foo--vg-root and /dev/mapper/foo--vg-swap_1 is created
automatically, so I suppose LVM works correctly.
3. mount -t btrfs /dev/mapper/foo--vg-root /mnt/foo
Gives the following error:
mount: wrong fs type, bad option, bad superblock on ...
4. dmesg | tail
Outputs the following:
[17755.840916] sd 3:0:0:0: [sda] tag#0 FAILED Result:
hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[17755.840919] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0 02 00
00 02 00
[17755.840921] blk_update_request: I/O error, dev sda, sector 507906
[17755.840941] EXT4-fs (dm-4): unable to read superblock
[18140.052300] sd 3:0:0:0: [sda] tag#0 FAILED Result:
hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[18140.052305] sd 3:0:0:0: [sda] tag#0 CDB: ATA command pass through(16) 85
06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00
[18142.991851] sd 3:0:0:0: [sda] tag#0 FAILED Result:
hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[18142.991856] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0 80 00
00 08 00
[18142.991860] blk_update_request: I/O error, dev sda, sector 508032
[18142.991869] Buffer I/O error on dev dm-4, logical block 16, async page
read
4.
# btrfs restore -i -D /dev/mapper/foo--vg-root /dev/null
No valid Btrfs found on /dev/mapper/foo--vg-root
Could not open root, trying backup super
No valid Btrfs found on /dev/mapper/foo--vg-root
Could not open root, trying backup super
No valid Btrfs found on /dev/mapper/foo--vg-root
Could not open root, trying backup super
We are pretty sure that no unexpected electric cuts has been happened.
At this point I don't know what information I should supply.
[-- Attachment #2: Type: text/html, Size: 2969 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unmountable root partition
2018-07-31 18:03 Unmountable root partition Cerem Cem ASLAN
@ 2018-07-31 23:04 ` Hans van Kranenburg
2018-08-01 4:42 ` Cerem Cem ASLAN
2018-08-01 5:32 ` Chris Murphy
1 sibling, 1 reply; 6+ messages in thread
From: Hans van Kranenburg @ 2018-07-31 23:04 UTC (permalink / raw)
To: Cerem Cem ASLAN, linux-btrfs
Hi,
On 07/31/2018 08:03 PM, Cerem Cem ASLAN wrote:
> Hi,
>
> I'm having trouble with my server setup, which contains a BTRFS root
> partition on top of LVM on top of LUKS partition.
>
> Yesterday server was shut down unexpectedly. I booted the system with a
> pendrive which contains Debian 4.9.18 and tried to mount the BTRFS root
> partition manually.
>
> 1. cryptsetup luksOpen /dev/sda5
>
> Seems to decrypt the partition (there are no errors)
>
> 2. /dev/mapper/foo--vg-root and /dev/mapper/foo--vg-swap_1 is created
> automatically, so I suppose LVM works correctly.
>
> 3. mount -t btrfs /dev/mapper/foo--vg-root /mnt/foo
> Gives the following error:
>
> mount: wrong fs type, bad option, bad superblock on ...
>
> 4. dmesg | tail
> Outputs the following:
>
>
> [17755.840916] sd 3:0:0:0: [sda] tag#0 FAILED Result:
> hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
> [17755.840919] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0
> 02 00 00 02 00
> [17755.840921] blk_update_request: I/O error, dev sda, sector 507906
> [17755.840941] EXT4-fs (dm-4): unable to read superblock
> [18140.052300] sd 3:0:0:0: [sda] tag#0 FAILED Result:
> hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
> [18140.052305] sd 3:0:0:0: [sda] tag#0 CDB: ATA command pass
> through(16) 85 06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00
> [18142.991851] sd 3:0:0:0: [sda] tag#0 FAILED Result:
> hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
> [18142.991856] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0
> 80 00 00 08 00
> [18142.991860] blk_update_request: I/O error, dev sda, sector 508032
> [18142.991869] Buffer I/O error on dev dm-4, logical block 16, async
> page read
This points at hardware level failure, regardless if the disk would hold
a btrfs or ext4 or whatever other kind of filesystem. The disk itself
cannot read data back when we ask for it.
> 4.
>
> # btrfs restore -i -D /dev/mapper/foo--vg-root /dev/null
> No valid Btrfs found on /dev/mapper/foo--vg-root
> Could not open root, trying backup super
> No valid Btrfs found on /dev/mapper/foo--vg-root
> Could not open root, trying backup super
> No valid Btrfs found on /dev/mapper/foo--vg-root
> Could not open root, trying backup super
>
> We are pretty sure that no unexpected electric cuts has been happened.
>
> At this point I don't know what information I should supply.
>
--
Hans van Kranenburg
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unmountable root partition
2018-07-31 23:04 ` Hans van Kranenburg
@ 2018-08-01 4:42 ` Cerem Cem ASLAN
0 siblings, 0 replies; 6+ messages in thread
From: Cerem Cem ASLAN @ 2018-08-01 4:42 UTC (permalink / raw)
To: Hans van Kranenburg; +Cc: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 2749 bytes --]
That might be the case. Can't we recover anything with some data loss?
2018-08-01 2:04 GMT+03:00 Hans van Kranenburg <
hans.van.kranenburg@mendix.com>:
> Hi,
>
> On 07/31/2018 08:03 PM, Cerem Cem ASLAN wrote:
> > Hi,
> >
> > I'm having trouble with my server setup, which contains a BTRFS root
> > partition on top of LVM on top of LUKS partition.
> >
> > Yesterday server was shut down unexpectedly. I booted the system with a
> > pendrive which contains Debian 4.9.18 and tried to mount the BTRFS root
> > partition manually.
> >
> > 1. cryptsetup luksOpen /dev/sda5
> >
> > Seems to decrypt the partition (there are no errors)
> >
> > 2. /dev/mapper/foo--vg-root and /dev/mapper/foo--vg-swap_1 is created
> > automatically, so I suppose LVM works correctly.
> >
> > 3. mount -t btrfs /dev/mapper/foo--vg-root /mnt/foo
> > Gives the following error:
> >
> > mount: wrong fs type, bad option, bad superblock on ...
> >
> > 4. dmesg | tail
> > Outputs the following:
> >
> >
> > [17755.840916] sd 3:0:0:0: [sda] tag#0 FAILED Result:
> > hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
> > [17755.840919] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0
> > 02 00 00 02 00
> > [17755.840921] blk_update_request: I/O error, dev sda, sector 507906
> > [17755.840941] EXT4-fs (dm-4): unable to read superblock
> > [18140.052300] sd 3:0:0:0: [sda] tag#0 FAILED Result:
> > hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
> > [18140.052305] sd 3:0:0:0: [sda] tag#0 CDB: ATA command pass
> > through(16) 85 06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00
> > [18142.991851] sd 3:0:0:0: [sda] tag#0 FAILED Result:
> > hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
> > [18142.991856] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0
> > 80 00 00 08 00
> > [18142.991860] blk_update_request: I/O error, dev sda, sector 508032
> > [18142.991869] Buffer I/O error on dev dm-4, logical block 16, async
> > page read
>
> This points at hardware level failure, regardless if the disk would hold
> a btrfs or ext4 or whatever other kind of filesystem. The disk itself
> cannot read data back when we ask for it.
>
> > 4.
> >
> > # btrfs restore -i -D /dev/mapper/foo--vg-root /dev/null
> > No valid Btrfs found on /dev/mapper/foo--vg-root
> > Could not open root, trying backup super
> > No valid Btrfs found on /dev/mapper/foo--vg-root
> > Could not open root, trying backup super
> > No valid Btrfs found on /dev/mapper/foo--vg-root
> > Could not open root, trying backup super
> >
> > We are pretty sure that no unexpected electric cuts has been happened.
> >
> > At this point I don't know what information I should supply.
> >
>
>
> --
> Hans van Kranenburg
>
[-- Attachment #2: Type: text/html, Size: 3694 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unmountable root partition
2018-07-31 18:03 Unmountable root partition Cerem Cem ASLAN
2018-07-31 23:04 ` Hans van Kranenburg
@ 2018-08-01 5:32 ` Chris Murphy
2018-08-01 12:07 ` Cerem Cem ASLAN
1 sibling, 1 reply; 6+ messages in thread
From: Chris Murphy @ 2018-08-01 5:32 UTC (permalink / raw)
To: Cerem Cem ASLAN; +Cc: Btrfs BTRFS
On Tue, Jul 31, 2018 at 12:03 PM, Cerem Cem ASLAN <ceremcem@ceremcem.net> wrote:
> 3. mount -t btrfs /dev/mapper/foo--vg-root /mnt/foo
> Gives the following error:
>
> mount: wrong fs type, bad option, bad superblock on ...
>
> 4. dmesg | tail
> Outputs the following:
>
>
> [17755.840916] sd 3:0:0:0: [sda] tag#0 FAILED Result:
> hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
> [17755.840919] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0 02 00 00
> 02 00
> [17755.840921] blk_update_request: I/O error, dev sda, sector 507906
> [17755.840941] EXT4-fs (dm-4): unable to read superblock
Are you sure this is the output for the command? Because you're
explicitly asking for type btrfs, which fails, and then the kernel
reports EXT4 superblock unreadable. What do you get if you omit -t
btrfs and just let it autodetect?
But yeah, this is an IO error from the device and there's nothing
Btrfs can do about that unless there is DUP or raid1+ metadata
available.
Is it possible this LV was accidentally reformatted ext4?
--
Chris Murphy
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unmountable root partition
2018-08-01 5:32 ` Chris Murphy
@ 2018-08-01 12:07 ` Cerem Cem ASLAN
2018-08-02 15:57 ` Cerem Cem ASLAN
0 siblings, 1 reply; 6+ messages in thread
From: Cerem Cem ASLAN @ 2018-08-01 12:07 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS
[-- Attachment #1: Type: text/plain, Size: 1628 bytes --]
Yes, command output is as is, because I just copied and pasted into the
mail. When I omit the `-t btrfs` part, result is the same.
I'm now trying to rescue what I can, so getting a image dump with
`ddrescue`. It's read about 25% without any errors but it will be expected
to finish in 6 hours. Then I'll try btrfscue to see what happens.
I know it's totally my fault because I must be ready for a total
disk/pc/building burn out. Lessons learned.
2018-08-01 8:32 GMT+03:00 Chris Murphy <lists@colorremedies.com>:
> On Tue, Jul 31, 2018 at 12:03 PM, Cerem Cem ASLAN <ceremcem@ceremcem.net>
> wrote:
>
> > 3. mount -t btrfs /dev/mapper/foo--vg-root /mnt/foo
> > Gives the following error:
> >
> > mount: wrong fs type, bad option, bad superblock on ...
> >
> > 4. dmesg | tail
> > Outputs the following:
> >
> >
> > [17755.840916] sd 3:0:0:0: [sda] tag#0 FAILED Result:
> > hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
> > [17755.840919] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0 02
> 00 00
> > 02 00
> > [17755.840921] blk_update_request: I/O error, dev sda, sector 507906
> > [17755.840941] EXT4-fs (dm-4): unable to read superblock
>
>
> Are you sure this is the output for the command? Because you're
> explicitly asking for type btrfs, which fails, and then the kernel
> reports EXT4 superblock unreadable. What do you get if you omit -t
> btrfs and just let it autodetect?
>
> But yeah, this is an IO error from the device and there's nothing
> Btrfs can do about that unless there is DUP or raid1+ metadata
> available.
>
> Is it possible this LV was accidentally reformatted ext4?
>
>
> --
> Chris Murphy
>
[-- Attachment #2: Type: text/html, Size: 2279 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unmountable root partition
2018-08-01 12:07 ` Cerem Cem ASLAN
@ 2018-08-02 15:57 ` Cerem Cem ASLAN
0 siblings, 0 replies; 6+ messages in thread
From: Cerem Cem ASLAN @ 2018-08-02 15:57 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS
[-- Attachment #1: Type: text/plain, Size: 1889 bytes --]
FYI: According to ddrescue, there were read errors in +45% of the
partition. There were no chances to save anything from the disk.
2018-08-01 15:07 GMT+03:00 Cerem Cem ASLAN <ceremcem@ceremcem.net>:
> Yes, command output is as is, because I just copied and pasted into the
> mail. When I omit the `-t btrfs` part, result is the same.
>
> I'm now trying to rescue what I can, so getting a image dump with
> `ddrescue`. It's read about 25% without any errors but it will be expected
> to finish in 6 hours. Then I'll try btrfscue to see what happens.
>
> I know it's totally my fault because I must be ready for a total
> disk/pc/building burn out. Lessons learned.
>
> 2018-08-01 8:32 GMT+03:00 Chris Murphy <lists@colorremedies.com>:
>
>> On Tue, Jul 31, 2018 at 12:03 PM, Cerem Cem ASLAN <ceremcem@ceremcem.net>
>> wrote:
>>
>> > 3. mount -t btrfs /dev/mapper/foo--vg-root /mnt/foo
>> > Gives the following error:
>> >
>> > mount: wrong fs type, bad option, bad superblock on ...
>> >
>> > 4. dmesg | tail
>> > Outputs the following:
>> >
>> >
>> > [17755.840916] sd 3:0:0:0: [sda] tag#0 FAILED Result:
>> > hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
>> > [17755.840919] sd 3:0:0:0: [sda] tag#0 CDB: Read(10) 28 00 00 07 c0 02
>> 00 00
>> > 02 00
>> > [17755.840921] blk_update_request: I/O error, dev sda, sector 507906
>> > [17755.840941] EXT4-fs (dm-4): unable to read superblock
>>
>>
>> Are you sure this is the output for the command? Because you're
>> explicitly asking for type btrfs, which fails, and then the kernel
>> reports EXT4 superblock unreadable. What do you get if you omit -t
>> btrfs and just let it autodetect?
>>
>> But yeah, this is an IO error from the device and there's nothing
>> Btrfs can do about that unless there is DUP or raid1+ metadata
>> available.
>>
>> Is it possible this LV was accidentally reformatted ext4?
>>
>>
>> --
>> Chris Murphy
>>
>
>
[-- Attachment #2: Type: text/html, Size: 2854 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-08-02 17:49 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-31 18:03 Unmountable root partition Cerem Cem ASLAN
2018-07-31 23:04 ` Hans van Kranenburg
2018-08-01 4:42 ` Cerem Cem ASLAN
2018-08-01 5:32 ` Chris Murphy
2018-08-01 12:07 ` Cerem Cem ASLAN
2018-08-02 15:57 ` Cerem Cem ASLAN
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.