* broken raid level 5 array caused by user error
@ 2015-11-09 11:27 Mathias Mueller
2015-11-09 11:56 ` Mikael Abrahamsson
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2015-11-09 11:27 UTC (permalink / raw)
To: linux-raid
Hi Folks,
I'm running a raid level 5 with 4 devices for some years and tried to
grow my array yesterday. I wanted to add two more devices and used the
following commands:
mdadm --add /dev/md0 /dev/sdf1 /dev/sdg1
mdadm --grow --raid-devices=6 /dev/md0
So far, so good. Everything seems to work, but after about 2 hours, the
reshape progress was still at 0.0% and now, my own stupidity kicked in.
I checked the logs via journalctl (I'm running Centos 7) and read
something about "main process died" or similar... then I decided to
reboot.
After reboot, assembling the array failed:
mdadm: Failed to restore critical section for reshape, sorry. Possibly
you needed to specify the --backup-file
But I did not have a backup file and so I panicked and made even worse
decisions.
First I tried to assemble the array using --invalid-backup but it did
not work. I should stop here and ask but I didn't. I read at some board,
that rebuilding the original array with 4 devices will fix my problem. I
did not validate this and entered the suggested command:
mdadm -CR /dev/md0 --metadata=1.2 -n4 -l5 -c512 /dev/sd[bcde]1
--assume-clean
But this did not work (the array assembled but I could not access the
ext4 filesystem), it seems that I assembled it in the wrong device
order, so I also tried different (i.e. all possible) orders, but nothing
helped ( I always used --assume-clean).
I guess this is the perfect guide for how _not_ to do it :(
I continued reading and found this:
http://serverfault.com/questions/347606/recover-raid-5-data-after-created-new-array-instead-of-re-using
This gave me some hope and now I wonder, if there is a way to get my
data back, maybe the offset is wrong?
Things I know about the array:
metadata: 1.2
Left-symetric
chunk-size: 512
When I run mdadm --detail /dev/md0 it still shows an array size of 6TB,
the UUID is also still the same
Version : 1.2
Creation Time : Mon Nov 9 00:00:40 2015
Raid Level : raid5
Array Size : 5860142592 (5588.67 GiB 6000.79 GB)
Used Dev Size : 1953380864 (1862.89 GiB 2000.26 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Mon Nov 9 00:00:45 2015
State : active
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : xxxx
UUID : 1d0fdb4e:6111bd7a:96cad2dd:b6a29039
Events : 1
Number Major Minor RaidDevice State
0 8 49 0 active sync /dev/sdd1
1 8 65 1 active sync /dev/sde1
2 8 33 2 active sync /dev/sdc1
3 8 17 3 active sync /dev/sdb1
an mdadm --examine gives this results:
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1d0fdb4e:6111bd7a:96cad2dd:b6a29039
Name : xxxx
Creation Time : Mon Nov 9 00:00:40 2015
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3906766961 (1862.89 GiB 2000.26 GB)
Array Size : 5860142592 (5588.67 GiB 6000.79 GB)
Used Dev Size : 3906761728 (1862.89 GiB 2000.26 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=5233 sectors
State : clean
Device UUID : e14f0e2d:a26a7b90:d7dbf780:e2218327
Internal Bitmap : 8 sectors from superblock
Update Time : Mon Nov 9 00:00:45 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 7a76b0d6 - correct
Events : 1
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1d0fdb4e:6111bd7a:96cad2dd:b6a29039
Name : xxxx
Creation Time : Mon Nov 9 00:00:40 2015
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3906764976 (1862.89 GiB 2000.26 GB)
Array Size : 5860142592 (5588.67 GiB 6000.79 GB)
Used Dev Size : 3906761728 (1862.89 GiB 2000.26 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=3248 sectors
State : clean
Device UUID : d408e617:37f3f0f5:feb5d77f:07e57668
Internal Bitmap : 8 sectors from superblock
Update Time : Mon Nov 9 00:00:45 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : a9787e9 - correct
Events : 1
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd:
MBR Magic : aa55
Partition[0] : 3907024002 sectors at 63 (type fd)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1d0fdb4e:6111bd7a:96cad2dd:b6a29039
Name : xxxx
Creation Time : Mon Nov 9 00:00:40 2015
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3906761858 (1862.89 GiB 2000.26 GB)
Array Size : 5860142592 (5588.67 GiB 6000.79 GB)
Used Dev Size : 3906761728 (1862.89 GiB 2000.26 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=130 sectors
State : clean
Device UUID : faf7ec39:e7c0cb77:770a439d:18dc65a0
Internal Bitmap : 8 sectors from superblock
Update Time : Mon Nov 9 00:00:45 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 3d38419 - correct
Events : 1
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1d0fdb4e:6111bd7a:96cad2dd:b6a29039
Name : xxx
Creation Time : Mon Nov 9 00:00:40 2015
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3906764976 (1862.89 GiB 2000.26 GB)
Array Size : 5860142592 (5588.67 GiB 6000.79 GB)
Used Dev Size : 3906761728 (1862.89 GiB 2000.26 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=3248 sectors
State : clean
Device UUID : fe31b351:3559f949:978035ae:616ae615
Internal Bitmap : 8 sectors from superblock
Update Time : Mon Nov 9 00:00:45 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 743a6702 - correct
Events : 1
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
I guess I know the old device order as well, I saved an old boot-log:
md: bind<sde1>
md: bind<sdb1>
md: bind<sdc1>
md: bind<sdd1>
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md/raid:md127: device sdd1 operational as raid disk 0
md/raid:md127: device sdc1 operational as raid disk 1
md/raid:md127: device sdb1 operational as raid disk 2
md/raid:md127: device sde1 operational as raid disk 3
md/raid:md127: allocated 4314kB
md/raid:md127: raid level 5 active with 4 out of 4 devices, algorithm 2
created bitmap (15 pages) for device md127
md127: bitmap initialized from disk: read 1 pages, set 0 of 29809 bits
md127: detected capacity change from 0 to 6001188667392
Please help me, I know I'm stupid and don't deserve it. I really hope,
there is a chance for reovering the array.
Thanks a lot in advance
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-09 11:27 broken raid level 5 array caused by user error Mathias Mueller
@ 2015-11-09 11:56 ` Mikael Abrahamsson
2015-11-09 13:50 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mikael Abrahamsson @ 2015-11-09 11:56 UTC (permalink / raw)
To: Mathias Mueller; +Cc: linux-raid
On Mon, 9 Nov 2015, Mathias Mueller wrote:
> Please help me, I know I'm stupid and don't deserve it. I really hope,
> there is a chance for reovering the array.
It's possible. There are numerous threads in the mailing list archives
with people who have been in the same situation as you are in now. Without
having a backup of the original superblocks, you're going to have to guess
offsets. It might make sense to investigate what the defaults were for the
mdadm you originally created the array with.
--
Mikael Abrahamsson email: swmike@swm.pp.se
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-09 11:56 ` Mikael Abrahamsson
@ 2015-11-09 13:50 ` Phil Turmel
[not found] ` <07de4cd96f39ecb6154794d072ca12e7@pingofdeath.de>
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2015-11-09 13:50 UTC (permalink / raw)
To: Mikael Abrahamsson, Mathias Mueller; +Cc: linux-raid
On 11/09/2015 06:56 AM, Mikael Abrahamsson wrote:
> On Mon, 9 Nov 2015, Mathias Mueller wrote:
>
>> Please help me, I know I'm stupid and don't deserve it. I really hope,
>> there is a chance for reovering the array.
>
> It's possible. There are numerous threads in the mailing list archives
> with people who have been in the same situation as you are in now.
> Without having a backup of the original superblocks, you're going to
> have to guess offsets. It might make sense to investigate what the
> defaults were for the mdadm you originally created the array with.
Chunks size defaults have varied, too.
I would search the known first drive for an ext4 filesystem signature
and work backwards from there. Show the output of this search:
dd if=/dev/sdd1 bs=1M count=1M |hexdump -C |grep ' 53 ef ' |head -n 1000
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
[not found] ` <5640B8AD.3030800@turmel.org>
@ 2015-11-09 15:41 ` Mathias Mueller
[not found] ` <d764bf541381927fa4183c9266fb3f5a@pingofdeath.de>
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2015-11-09 15:41 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid
> I trimmed the rest because a valid signature was at the very beginning
> of the device. So you had either v0.90 or v1.0 metadata. I would bet
> it's v0.90. I would also bet that the chunk size is 64k.
>
> I suggest you try that first:
>
> mdadm --create --assume-clean --level=5 --raid-devices=4 --chunk=64 \
> --metadata=0.90 /dev/md1 /dev/sd{d,c,b,e}1
>
> Do *not* mount it yet!
>
> Do a non-destructive fsck to see if my guess was accurate:
>
> fsck -n /dev/md1
>
> Generally, if this has many errors, you would stop the array, and try
> the other order combinations (but always with sdd1 first) in turn.
>
> However, you don't say, and we can't tell from the new metadata, how
> far into the reshape mdadm had gotten. If very far, there will be
> many errors no matter what order you try, and your array is toast.
>
> But a result with few fsck errors is almost certainly success,
> and you can mount your array.
>
> BTW, if you recall where you found the suggestion to re-create your
> array, please go back and report there how crazy that is, and point
> them at this mailing list.
>
> Phil
Hi Phil,
thanks for your reply and sorry for my mistakes, I'm new with the
mailinglist.
I tried your suggestion but it did not work, fsck could not find a
superblock. I'm very sure, that I used the metadata v1.2 (and also kind
of sure that I used a chunk size of 512k). Sadly I tried to mount the
array yesterday when I was trying the different device orders.
Is it helpful to post the hexdump of the other devices as well?
Thanks for your Help!
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
[not found] ` <a3a91665c4b7cdd70dacc7d8815cc365@pingofdeath.de>
@ 2015-11-09 21:13 ` Phil Turmel
2015-11-10 8:37 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2015-11-09 21:13 UTC (permalink / raw)
To: Mathias Mueller; +Cc: linux-raid
{Resend with list address corrected -- ignore the duplicate, Mathias }
Hi Matthias,
On 11/09/2015 02:36 PM, Mathias Mueller wrote:
>> for x in /dev/sd[b-e]1 ; do echo $x ; dd if=$x bs=4k count=2 |hexdump
>> -C ; done
>
> /dev/sdb1
> 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> *
> 00000400 00 20 47 07 16 10 1c 1d 67 9a 74 01 8b 5d a6 1c |. G.....g.t..]..|
> 00000410 f5 1f 47 07 00 00 00 00 02 00 00 00 02 00 00 00 |..G.............|
> 00000420 00 80 00 00 00 80 00 00 00 20 00 00 00 00 00 00 |......... ......|
> 00000430 d9 45 8b 4c 00 00 27 00 53 ef 01 00 01 00 00 00 |.E.L..'.S.......|
> 00000440 2e 44 8b 4c 00 4e ed 00 00 00 00 00 01 00 00 00 |.D.L.N..........|
> 00000450 00 00 00 00 0b 00 00 00 00 01 00 00 3c 00 00 00 |............<...|
False positive. (Not an ext4 signature).
> /dev/sdc1
> 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> *
> 00000400 00 20 47 07 16 10 1c 1d 67 9a 74 01 8b 5d a6 1c |. G.....g.t..]..|
> 00000410 f5 1f 47 07 00 00 00 00 02 00 00 00 02 00 00 00 |..G.............|
> 00000420 00 80 00 00 00 80 00 00 00 20 00 00 00 00 00 00 |......... ......|
> 00000430 d9 45 8b 4c 00 00 27 00 53 ef 01 00 01 00 00 00 |.E.L..'.S.......|
> 00000440 2e 44 8b 4c 00 4e ed 00 00 00 00 00 01 00 00 00 |.D.L.N..........|
Also a false positive.
> /dev/sdd1
> 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> *
> 00000400 00 20 47 07 90 0e 1c 1d 54 9a 74 01 75 aa c1 08 |. G.....T.t.u...|
> 00000410 35 47 33 07 00 00 00 00 02 00 00 00 02 00 00 00 |5G3.............|
> 00000420 00 80 00 00 00 80 00 00 00 20 00 00 0b b2 ca 4d |......... .....M|
> 00000430 7b f0 ca 4d 15 00 21 00 53 ef 01 00 01 00 00 00 |{..M..!.S.......|
> 00000440 9d 7f 3b 4d 00 4e ed 00 00 00 00 00 01 00 00 00 |..;M.N..........|
> 00000450 00 00 00 00 0b 00 00 00 00 01 00 00 3c 00 00 00 |............<...|
> 00000460 42 02 00 00 7b 00 00 00 4f a7 6a c3 f8 12 42 1c |B...{...O.j...B.|
> 00000470 8f d2 5b ec 1a e9 fb 4c 64 61 74 61 00 00 00 00 |..[....Ldata....|
> 00000480 00 00 00 00 00 00 00 00 2f 6d 65 64 69 61 2f 64 |......../media/d|
> 00000490 61 74 61 00 d8 dc 83 00 02 88 ff ff 38 da 59 4c |ata.........8.YL|
> 000004a0 00 00 00 00 00 c6 70 c3 01 88 ff ff 00 96 56 23 |......p.......V#|
> 000004b0 02 88 ff ff 40 7b 21 1d 02 88 ff ff dd 75 18 81 |....@{!......u..|
This really is an ext4 signature. Volume label 'data' last
mounted at '/media/data' on Wed, 11 May 2011 15:58:03 GMT.
Given that date and your certainty that your array was v1.2,
this must be garbage left over from a previous filesystem.
> /dev/sde1
> 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> *
> 00000400 00 20 47 07 16 10 1c 1d 67 9a 74 01 8b 5d a6 1c |. G.....g.t..]..|
> 00000410 f5 1f 47 07 00 00 00 00 02 00 00 00 02 00 00 00 |..G.............|
> 00000420 00 80 00 00 00 80 00 00 00 20 00 00 0a 88 62 51 |......... ....bQ|
> 00000430 d5 cb 62 51 05 00 24 00 53 ef 01 00 01 00 00 00 |..bQ..$.S.......|
> 00000440 54 d4 81 50 00 4e ed 00 00 00 00 00 01 00 00 00 |T..P.N..........|
> 00000450 00 00 00 00 0b 00 00 00 00 01 00 00 3c 00 00 00 |............<...|
Another false positive.
>> Also show the following more picky signature search for each drive:
>> dd if=/dev/sdd1 bs=1M count=1M |hexdump -C |grep -E '[048c]30 .+ 53 ef'
>
> I will provide this as soon as the processes are finished, seems to take
> some more hours.
We'll just have to wait for this. Your ext4 superblock should be in
there somewhere.
> there is also one more thing I remember, maybe it's helpful: when I was
> starting the reshape, there was this message:
>
> mdadm: Need to backup 8192K of critical section..
If this was an actual error message, it may not have proceeded. We just
don't know yet.
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-09 21:13 ` Phil Turmel
@ 2015-11-10 8:37 ` Mathias Mueller
2015-11-10 13:55 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2015-11-10 8:37 UTC (permalink / raw)
To: Phil Turmel; +Cc: linux-raid
Hi Phil,
sadly, my ssh-session died while hexdumping over night, but I post what
I got (restarted the processes anyway):
> dd if=/dev/sdd1 bs=1M count=1M |hexdump -C |grep -E '[048c]30 .+ 53
> ef'
sdb1:
http://pastebin.com/ERehrYYR
sdc1:
http://pastebin.com/WMpRtPs5
sdd1:
http://pastebin.com/5QxGuGGY
sde1:
http://pastebin.com/CAC8Pkt9
Best regards
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 8:37 ` Mathias Mueller
@ 2015-11-10 13:55 ` Phil Turmel
2015-11-10 14:55 ` Mathias Mueller
2015-11-10 15:20 ` Mathias Mueller
0 siblings, 2 replies; 31+ messages in thread
From: Phil Turmel @ 2015-11-10 13:55 UTC (permalink / raw)
To: Mathias Mueller; +Cc: linux-raid
Hi Mathias,
On 11/10/2015 03:37 AM, Mathias Mueller wrote:
> Hi Phil,
>
> sadly, my ssh-session died while hexdumping over night, but I post what
> I got (restarted the processes anyway):
'screen' is your friend for long-running server tasks. It's installed in many distros by default.
>
>> dd if=/dev/sdd1 bs=1M count=1M |hexdump -C |grep -E '[048c]30 .+ 53 ef'
I should have put "|head -n 50" on the end of that. :-)
> sdb1:
> http://pastebin.com/ERehrYYR
Pastebin expires. The list accepts up to ~100k. I'm pasting this
for the record:
> 00000430 d9 45 8b 4c 00 00 27 00 53 ef 01 00 01 00 00 00 |.E.L..'.S.......|
> 010d8030 9c a9 4c 27 20 97 fe 4c 53 ef 40 e8 dd 2c 1a d6 |..L' ..LS.@..,..|
> 04db5030 4b 5c ea 17 1d 0b b3 1c 53 ef 7c 3f 74 90 97 10 |K\......S.|?t...|
> 07ed8030 73 96 e8 8d 3a d5 9d c4 53 ef dc 1c d8 a5 1d 2b |s...:...S......+|
> 131af430 c1 77 bd d9 84 c3 25 8f 53 ef 0c e6 17 3e ea 06 |.w....%.S....>..|
> 1ea2b830 87 83 16 e7 b6 ac 24 61 53 ef 70 53 fa 10 dc 27 |......$aS.pS...'|
> 1f79a430 db b2 90 27 87 1c bb bf 53 ef 26 7c 6f 76 23 34 |...'....S.&|ov#4|
> 2a21e830 a7 63 f4 cc 85 1c 7a 8c 53 ef 69 b1 a8 5d ae b0 |.c....z.S.i..]..|
> 3562a830 9d f4 99 e3 62 ae 6b 3f 53 ef 65 04 12 0d 3c 77 |....b.k?S.e...<w|
> 3562c430 15 8e ab 0d 2c cf a2 77 53 ef 03 73 06 6f ab cd |....,..wS..s.o..|
> 3d254830 3a f2 3f 34 82 1f 8f 98 53 ef 9c 68 c3 81 dc f4 |:.?4....S..h....|
> 4040b030 01 01 10 06 5a 8b 26 43 53 ef ba e2 ce 33 ec 5c |....Z.&CS....3.\|
> 410dc430 f4 40 bf 4d 8a f1 43 06 53 ef d8 c2 a4 69 ab 0a |.@.M..C.S....i..|
> 4d8b2c30 32 ed b7 f6 15 bb ab d5 53 ef 1c 6e d6 4c 36 99 |2.......S..n.L6.|
> 4e100030 85 54 ac 57 46 8f a6 57 53 ef 9d 4c 50 60 cc f5 |.T.WF..WS..LP`..|
> 4f23fc30 a8 77 c6 f1 87 80 ca c1 53 ef c7 41 93 db 8b e6 |.w......S..A....|
> 52c41c30 d2 3e d6 ce 01 6d d4 f3 53 ef 3d 7a a2 db 95 5e |.>...m..S.=z...^|
> 597e1c30 7b 4a 38 58 ff 4c 38 f1 53 ef 77 b1 6c 6a 0a d4 |{J8X.L8.S.w.lj..|
> 5e634430 b7 f2 21 f2 f6 06 0c ff 53 ef 1b a7 bb 78 76 a0 |..!.....S....xv.|
> 6546d430 e7 63 6a 5b 93 69 9e 8b 53 ef 50 99 76 4f 1a 61 |.cj[.i..S.P.vO.a|
> 65d2c430 45 27 46 b4 56 8a 91 ab 53 ef af fa 08 26 a4 99 |E'F.V...S....&..|
> 6625ec30 cd 99 a9 ae 65 08 b3 97 53 ef e3 7e 83 9b 6d a2 |....e...S..~..m.|
> 6b3cc030 cd 6c f0 fd f2 b1 f7 b6 53 ef 8f 9d a3 2a 3f 76 |.l......S....*?v|
> 6d7f2430 07 9b 5a 4f 01 f0 d3 d8 53 ef ed ef b8 26 38 5f |..ZO....S....&8_|
> 70547030 74 8d 3b 05 2a 99 b4 4f 53 ef 69 20 2d ad 5f 2b |t.;.*..OS.i -._+|
> 75a74830 f5 c8 3f 03 a9 df 9f 59 53 ef 68 e1 d4 51 9d c5 |..?....YS.h..Q..|
> 83393430 39 54 23 d3 5e b3 7a ef 53 ef 31 90 7f fb ff f3 |9T#.^.z.S.1.....|
> 8442d830 d7 32 76 ca c3 f8 bd a1 53 ef f2 5a 81 ff 27 f2 |.2v.....S..Z..'.|
> 8b4d2c30 b8 97 0c 36 a7 1c a0 29 53 ef 1c dc 21 6d f2 21 |...6...)S...!m.!|
> 9d1f4830 9e 65 ff fe fc 2e 58 e8 53 ef ca 99 9c ad b1 19 |.e....X.S.......|
> a1806030 1d 8e 2b 24 e2 ae 30 54 53 ef 8b 96 16 0b d3 93 |..+$..0TS.......|
> a7c4c030 ef aa f0 35 2d 59 2b d1 53 ef 24 d4 fe e9 4f 35 |...5-Y+.S.$...O5|
> a9597430 75 3a 95 b9 7b 70 c7 9f 53 ef 5c f8 8b 69 20 f0 |u:..{p..S.\..i .|
> ad5ab830 bd 10 15 57 80 89 fd fd 53 ef 2f 04 63 e1 f1 95 |...W....S./.c...|
> b1863830 45 bc 30 8c 0b 9f 2f aa 53 ef bd d1 6e a8 a9 42 |E.0.../.S...n..B|
> b213ec30 a3 3d ba 3d 23 4f 6b fa 53 ef 38 b6 b3 96 37 aa |.=.=#Ok.S.8...7.|
> bb04a030 85 3b 32 5f 95 fb 42 f1 53 ef 63 7b f9 30 07 81 |.;2_..B.S.c{.0..|
> c7cf2830 66 51 61 82 c5 5a ec c4 53 ef 81 66 df f1 c6 00 |fQa..Z..S..f....|
> c8926c30 29 74 85 31 27 6d 49 1c 53 ef af cd 02 df 4c 2c |)t.1'mI.S.....L,|
> ca006030 1d 8e 2b 24 e2 ae 30 54 53 ef 8b 96 16 0b d3 93 |..+$..0TS.......|
> cadfbc30 2c c1 b5 c5 72 88 3d bb 53 ef bf de b8 ae b8 a5 |,...r.=.S.......|
> cee41430 a0 51 cf d2 f0 28 78 e2 53 ef 3b c4 f8 aa 95 6c |.Q...(x.S.;....l|
> d91a3c30 8e 28 30 d6 6f ed de 6e 53 ef 52 5e 27 67 cf f8 |.(0.o..nS.R^'g..|
> d9e76030 14 b0 31 80 64 52 f7 d4 53 ef 4a 48 e5 83 1e 55 |..1.dR..S.JH...U|
> da9ea830 d0 e5 18 b8 ed 03 90 be 53 ef d0 47 90 2f 01 e2 |........S..G./..|
> e6625c30 ca e9 df 4f 39 4d 6d 42 53 ef f1 3c 07 1b 58 de |...O9MmBS..<..X.|
> f7284830 e7 fa 23 37 40 bc e5 9e 53 ef 01 3e 7e 91 35 5a |..#7@...S..>~.5Z|
> f981ac30 20 17 eb ed 1e 8f e9 0e 53 ef ac f5 63 ee 9b 08 | .......S...c...|
> fa3cc030 b4 f6 f3 5f a7 16 6a 5c 53 ef 9c 6c 36 53 2c da |..._..j\S..l6S,.|
> fd574030 db 77 39 c7 8d fe bb 7e 53 ef 5a 85 ea 36 41 8e |.w9....~S.Z..6A.|
None of these have a write timestamp in the recent past. {Look for 0x55 or
0x56 in the 4th byte on each line.}
>
> sdc1:
> http://pastebin.com/WMpRtPs5
> 00000430 d9 45 8b 4c 00 00 27 00 53 ef 01 00 01 00 00 00 |.E.L..'.S.......|
> 00100430 28 ab 3f 56 06 00 19 00 53 ef 01 00 01 00 00 00 |(.?V....S.......|
This ^^ is encouraging: Sun, 08 Nov 2015 20:06:00 GMT
It's in the right spot for data offset == 2048 if this
is the first device.
> 02b80030 7c 6b 4b 55 12 00 19 00 53 ef 00 00 01 00 00 00 ||kKU....S.......|
> 08100030 7c 6b 4b 55 12 00 19 00 53 ef 00 00 01 00 00 00 ||kKU....S.......|
These ^^ look like backup ext4 superblocks, written Thu, 07 May 2015
13:41:16 GMT. If you had uptime from May all the way to November, this
is believable. And there's more of them scattered below.
> 0a7dc030 45 ae b5 ff 5b 76 73 5b 53 ef 28 ca 3c 11 0c 0f |E...[vs[S.(.<...|
> 0d600030 7c 6b 4b 55 12 00 19 00 53 ef 00 00 01 00 00 00 ||kKU....S.......|
> 0d7a9030 0b f3 69 7e c6 3f 43 15 53 ef 91 f2 36 8f 89 74 |..i~.?C.S...6..t|
> 11eb9430 73 e9 26 d8 d5 22 07 cd 53 ef 90 13 35 29 92 db |s.&.."..S...5)..|
> 12b80030 7c 6b 4b 55 12 00 19 00 53 ef 00 00 01 00 00 00 ||kKU....S.......|
> 15af5830 60 dc 27 cc 15 97 82 6d 53 ef ab b0 77 a3 84 15 |`.'....mS...w...|
> 18100030 7c 6b 4b 55 12 00 19 00 53 ef 00 00 01 00 00 00 ||kKU....S.......|
> 1e8e2030 c1 7a d1 05 cd a8 e0 c2 53 ef f6 7a 9c 33 50 1f |.z......S..z.3P.|
> 1f60ac30 48 3c 15 58 ad ef d9 15 53 ef 5b 18 c7 0c 81 1d |H<.X....S.[.....|
> 1fccf430 eb 93 ad 0e dc 75 56 3b 53 ef 6e d4 14 ab 2a 6d |.....uV;S.n...*m|
> 267ccc30 3c 24 c2 5b 10 a4 9e f5 53 ef bf e1 7f 24 c6 a0 |<$.[....S....$..|
> 27e15c30 03 07 98 da d5 86 d6 4a 53 ef 20 93 89 b0 3f de |.......JS. ...?.|
> 287bdc30 c1 6c 79 2f 8b e0 72 65 53 ef ed f8 92 5d 56 e6 |.ly/..reS....]V.|
> 2b843830 ab ad 5c 9a 3b e1 c5 22 53 ef ef 8d 1f a8 0f ff |..\.;.."S.......|
> 2cac8030 7b 6d 61 51 b9 45 58 87 53 ef 30 13 39 6e 2d 9b |{maQ.EX.S.0.9n-.|
> 35b55030 38 e4 b9 f2 9f 54 b2 2b 53 ef a0 b8 a6 4d f4 4b |8....T.+S....M.K|
> 362b7030 ec 54 b8 8d 9e b0 73 88 53 ef 45 22 58 6e 58 a5 |.T....s.S.E"XnX.|
> 383f0830 0b 36 cd 48 d3 a0 5c 5a 53 ef 28 8a e3 7e 28 e2 |.6.H..\ZS.(..~(.|
> 40db5830 99 22 6a bd 33 f2 39 99 53 ef 86 e4 70 e8 bd d6 |."j.3.9.S...p...|
> 42b80030 7c 6b 4b 55 12 00 19 00 53 ef 00 00 01 00 00 00 ||kKU....S.......|
> 4681d430 38 7b 67 9b a5 2e 96 44 53 ef 35 fd c9 e2 a9 71 |8{g....DS.5....q|
> 48100030 7c 6b 4b 55 12 00 19 00 53 ef 00 00 01 00 00 00 ||kKU....S.......|
> 4a403c30 b1 25 08 7b 88 0a 69 3f 53 ef fc 93 c8 46 93 f3 |.%.{..i?S....F..|
> 4bcad030 f4 bf 6f fc 48 34 d6 72 53 ef bb c5 69 d0 d7 27 |..o.H4.rS...i..'|
> 4eed7830 68 32 c6 73 f4 f0 57 68 53 ef d5 5c 3a 3b ca 49 |h2.s..WhS..\:;.I|
> 51807430 46 f8 72 b4 ac 58 46 b9 53 ef ad aa d3 de 8c 94 |F.r..XF.S.......|
> 51ad4430 28 16 ed f1 f2 00 98 b8 53 ef e3 04 2c ad e9 70 |(.......S...,..p|
> 55491430 d7 3f 87 00 15 70 40 bc 53 ef 7d 18 a1 6e 67 e8 |.?...p@.S.}..ng.|
> 576ed030 2a f0 e4 4c ad 4a 19 a4 53 ef d8 3c bd 00 c7 7f |*..L.J..S..<....|
> 62e40c30 6d 1e 1e 64 c6 29 fc 6c 53 ef 57 bd 2d 99 ca 87 |m..d.).lS.W.-...|
> 63d2d030 ab bc c3 76 a9 3e af 4c 53 ef 6e 72 11 6a cc 34 |...v.>.LS.nr.j.4|
> 6acc0c30 37 db ad 49 4c 9a 72 4a 53 ef 3b 6e 8f bb 18 6e |7..IL.rJS.;n...n|
> 75359830 7c d8 e0 62 51 e1 70 fa 53 ef 5f a2 ea c1 92 0f ||..bQ.p.S._.....|
> 799fb430 77 b2 62 cb 73 8c b5 48 53 ef 76 49 59 fc 97 ad |w.b.s..HS.vIY...|
> 7badc430 3d 3c 7c 72 b5 0f 6c f1 53 ef cb 6d 6e 69 d4 e5 |=<|r..l.S..mni..|
> 81413c30 a8 58 4a be a2 f6 36 3a 53 ef 0c 7b 31 3c bf 8a |.XJ...6:S..{1<..|
> 82b80030 7c 6b 4b 55 12 00 19 00 53 ef 00 00 01 00 00 00 ||kKU....S.......|
> 85182430 ef 21 64 5f 3b c4 46 7f 53 ef 9d fc 88 72 e8 d1 |.!d_;.F.S....r..|
> 86245030 86 ca b9 f4 cd ff a9 cd 53 ef ad 62 a8 e8 fb 0b |........S..b....|
> 864d1430 6c a0 5b 34 9d e6 4b 97 53 ef 87 9b ea 8e 4f ac |l.[4..K.S.....O.|
> 8cea9030 53 43 cc c0 93 a9 69 9b 53 ef 65 9f ee 82 75 ca |SC....i.S.e...u.|
> 940c0030 32 8d 87 1f 79 f2 f5 13 53 ef bf 0f 9b a6 e8 6a |2...y...S......j|
> 9bcd4430 e3 da 47 63 bc eb 10 54 53 ef 54 29 4d 82 07 d2 |..Gc...TS.T)M...|
> aa4e9430 b8 09 b8 5a ea 6e ab ad 53 ef 6f 0e 10 01 5f aa |...Z.n..S.o..._.|
> b5343830 37 3a 8b f3 c9 bc dd 77 53 ef 38 9a f1 08 13 0e |7:.....wS.8.....|
> b7c0b030 f6 cc 47 b4 b5 a7 cc 1b 53 ef 78 f8 9e 1e de 7d |..G.....S.x....}|
>
> sdd1:
> http://pastebin.com/5QxGuGGY
> 00000430 7b f0 ca 4d 15 00 21 00 53 ef 01 00 01 00 00 00 |{..M..!.S.......|
> 0171c430 b2 44 ab 17 a0 1e b0 43 53 ef c7 5c fb 83 95 47 |.D.....CS..\...G|
> 01734830 a7 81 e8 1d 83 04 88 1c 53 ef b4 e2 11 18 a9 90 |........S.......|
> 06715c30 16 cd a6 93 42 5a f1 21 53 ef 06 3c ea ac 96 2f |....BZ.!S..<.../|
> 0942cc30 4b 9d 1a a7 04 46 2d 55 53 ef 3f 8c 1e 25 26 94 |K....F-US.?..%&.|
> 0a46b430 c4 43 bc ab 49 30 46 93 53 ef 63 8e ff c9 0e c8 |.C..I0F.S.c.....|
> 0cd7d030 8c 26 bd 38 15 80 e9 d6 53 ef 2a 24 4f 4f a8 af |.&.8....S.*$OO..|
> 11faec30 7a 7d 11 5e bc b0 c7 4c 53 ef e7 81 e5 30 f2 7a |z}.^...LS....0.z|
> 187cac30 d4 ce cf bc a5 fc c5 fa 53 ef 8d a6 0e ef 87 7d |........S......}|
> 1df88030 96 4a 06 8f 63 89 90 9c 53 ef de 53 04 17 ea a6 |.J..c...S..S....|
> 1ec74830 95 08 9e 18 78 27 3c d2 53 ef 4f a8 d8 9b 5c 6a |....x'<.S.O...\j|
> 1f7ef830 96 37 d6 93 a2 cb ff 1f 53 ef 6d e8 fc 5d 7d 19 |.7......S.m..]}.|
> 22c9a830 6d 89 a4 e7 66 d6 0f 57 53 ef 2c 39 47 c8 a7 cf |m...f..WS.,9G...|
> 241d3c30 d6 67 39 79 e0 ef 6e a1 53 ef 28 00 fe 1d 38 89 |.g9y..n.S.(...8.|
> 243a8830 50 76 ae 32 e1 ae 73 c4 53 ef b8 30 2a ce cc 2e |Pv.2..s.S..0*...|
> 278ed430 4b a0 32 0f e8 06 41 ce 53 ef da c0 a4 6b 57 31 |K.2...A.S....kW1|
> 2c1fac30 84 52 60 2c 5e 93 52 45 53 ef 30 37 98 ec c2 56 |.R`,^.RES.07...V|
> 2e503030 22 b6 6b 9b c1 4b 20 7f 53 ef ed 5a 35 34 a0 a5 |".k..K .S..Z54..|
> 390cd830 66 08 74 9b b7 e9 24 16 53 ef 77 71 11 4e b3 28 |f.t...$.S.wq.N.(|
> 3a85ac30 4b 5c 7d fc c0 99 29 7c 53 ef 58 cf ff 1a cf ff |K\}...)|S.X.....|
> 66c15430 c1 29 7f aa 5e c2 9f ec 53 ef b5 32 ee 3d de ee |.)..^...S..2.=..|
> 76755830 bb 69 6b 92 cb ba bd 6d 53 ef d6 65 4b 31 ad 03 |.ik....mS..eK1..|
> 851a4830 5c 81 d7 1b d3 2a 5e 83 53 ef 98 d4 5b 2e 4b 2b |\....*^.S...[.K+|
> 8fda7030 4d f6 ca ba a9 34 f7 87 53 ef 7f 59 7b 70 cb e5 |M....4..S..Y{p..|
> 9cabe030 32 ee 5f 43 4d 11 0e 79 53 ef a5 60 1f c8 cb ac |2._CM..yS..`....|
> a1293c30 3d 51 81 9a 08 a2 e0 c7 53 ef b0 ec 68 cd 27 50 |=Q......S...h.'P|
> a595bc30 40 70 9e 35 40 fe b7 af 53 ef 52 10 a9 d7 ca 81 |@p.5@...S.R.....|
> a8d12c30 2c c1 b5 c5 72 88 3d bb 53 ef bf de b8 ae b8 a5 |,...r.=.S.......|
> aa15dc30 db b5 ed c3 ee a7 6b c2 53 ef f1 1d 7f 59 69 95 |......k.S....Yi.|
> b963d830 30 a6 25 18 e4 55 b7 3b 53 ef 67 5b 25 59 76 31 |0.%..U.;S.g[%Yv1|
> baefc830 07 88 0f 60 34 a7 8d f1 53 ef 0b 5e 68 21 7f 4d |...`4...S..^h!.M|
> bbb7b430 51 00 91 ad b6 c3 89 5b 53 ef cf 10 5c 54 ef ff |Q......[S...\T..|
> bc850c30 e1 d6 6f 4b 52 26 f3 5c 53 ef 1f fd e7 3e 0e f5 |..oKR&.\S....>..|
> c046cc30 4b ad 54 a6 92 54 91 f7 53 ef b5 1f ea 1d 99 68 |K.T..T..S......h|
> c17c0830 37 63 0e 04 5f 09 9a 9d 53 ef 13 74 94 c0 0f eb |7c.._...S..t....|
> c8080030 98 fd 13 ca db d8 86 59 53 ef 08 ce 0e 98 ec 76 |.......YS......v|
> cd36cc30 4b ad 54 a6 92 54 91 f7 53 ef b5 1f ea 1d 99 68 |K.T..T..S......h|
> d03e7830 f8 18 1d e9 70 61 5a e7 53 ef 3c d6 71 90 c2 23 |....paZ.S.<.q..#|
> d25fa430 a2 00 05 d1 6c e6 23 6d 53 ef 39 55 49 06 64 48 |....l.#mS.9UI.dH|
> d3e86430 b8 c8 8c 86 4b 03 1c 42 53 ef e1 16 79 c6 30 48 |....K..BS...y.0H|
> d7d0e030 64 dd c2 20 da e9 fe c1 53 ef e0 b7 04 71 be ed |d.. ....S....q..|
> d7d79430 e0 3f 7f f3 16 48 3d ac 53 ef 4a 63 25 69 5f 6f |.?...H=.S.Jc%i_o|
> d84e6c30 f2 de 6a 35 5d 41 94 d7 53 ef e9 39 2c f1 fb cd |..j5]A..S..9,...|
> ddd98c30 ca 84 3a 09 8b b0 fb ee 53 ef 45 0f 07 04 b6 04 |..:.....S.E.....|
> ea2e1030 50 be a2 62 3c f0 db 68 53 ef ff a6 85 32 77 e5 |P..b<..hS....2w.|
> ef0b9830 84 5e c3 0d 52 48 76 53 53 ef e8 24 96 73 12 ac |.^..RHvSS..$.s..|
> f8fe9830 f6 e8 24 f7 c2 9f 45 9a 53 ef 98 74 4b 5f 5d 80 |..$...E.S..tK_].|
> fac0e830 c2 56 65 0c e1 c0 cc ef 53 ef 12 18 b3 89 c4 2d |.Ve.....S......-|
> 1034b7c30 ea 4b 6a 1b 9a 63 fc 9d 53 ef aa 53 5b e7 43 5f |.Kj..c..S..S[.C_|
> 10a378430 ff d3 d8 4a 5f 7d 52 c0 53 ef 5b 50 66 f1 38 f5 |...J_}R.S.[Pf.8.|
Nothing here ^^.
>
> sde1:
> http://pastebin.com/CAC8Pkt9
> 00000430 d5 cb 62 51 05 00 24 00 53 ef 01 00 01 00 00 00 |..bQ..$.S.......|
> 00100430 28 ab 3f 56 06 00 19 00 53 ef 01 00 01 00 ef 1a |(.?V....S.......|
Another candidate for a first disk, with offset 2048. Also written
Sun, 08 Nov 2015 20:06:00 GMT. Either sdc or sde is your first disk.
The other is parity, which is the last disk on the first stripe, with
zeros in the matching offsets of the other two disks.
> 04975030 22 e0 84 41 68 0b 2a 3e 53 ef fa 4d ab cf 92 78 |"..Ah.*>S..M...x|
> 04d09830 09 96 1e 05 dd 53 70 02 53 ef c8 99 81 bb db 08 |.....Sp.S.......|
> 07554c30 ad fb d5 76 b9 04 8b ab 53 ef 6d 9e 7d 92 61 2b |...v....S.m.}.a+|
> 08100030 7c 6b 49 55 12 00 19 00 53 ef 00 00 01 20 76 ed ||kIU....S.... v.|
Another possible backup superblock, written Wed, 06 May 2015 01:16:44 GMT.
With more like it below.
> 0f3e5830 1b 2b 97 41 57 15 59 12 53 ef 5f f9 72 45 d5 67 |.+.AW.Y.S._.rE.g|
> 10e04430 32 95 f4 68 40 8b e0 b4 53 ef c8 fe 68 e4 05 3f |2..h@...S...h..?|
> 17fac430 cb de dc 31 a5 e4 56 b9 53 ef 76 e5 b9 6c cd 58 |...1..V.S.v..l.X|
> 18100030 7c 6b 49 55 12 00 19 00 53 ef 00 00 01 20 76 ed ||kIU....S.... v.|
> 1f9a1030 f3 95 67 eb f4 fb cb 82 53 ef a4 47 b5 aa 8f ab |..g.....S..G....|
> 21ecf830 af 55 bc 3f 03 10 fa c6 53 ef 27 50 7a a1 0b 7d |.U.?....S.'Pz..}|
> 25a48030 f0 f4 9e 58 8e a8 dc e8 53 ef 8f ee f7 5f f2 96 |...X....S...._..|
> 2693e830 14 01 94 72 71 13 73 fd 53 ef 59 75 aa 11 9d 0a |...rq.s.S.Yu....|
> 26daf430 a9 96 ae a0 5d 4a 71 ad 53 ef e2 64 ee d6 bc ec |....]Jq.S..d....|
> 2c3e2830 b5 77 fe 66 e0 9a 65 94 53 ef 2e 6e 4e dc 3d 4c |.w.f..e.S..nN.=L|
> 2d091030 a5 49 93 09 a1 21 16 11 53 ef a9 48 0a 30 43 9c |.I...!..S..H.0C.|
> 2dc5a030 b7 fd b1 20 3e 74 8b d6 53 ef 4f 78 06 30 88 69 |... >t..S.Ox.0.i|
> 2e604430 89 32 cb a1 9e 8e 87 37 53 ef a8 74 92 be 8f 57 |.2.....7S..t...W|
> 2ea74430 86 8d 2e 0e 11 8f 26 74 53 ef a1 8b 25 ff fb e2 |......&tS...%...|
> 35fdb430 ff 6d 5c 78 cd 3a 8c c3 53 ef 0a ac 87 9f a4 97 |.m\x.:..S.......|
> 399efc30 63 35 60 c7 52 2d 42 8d 53 ef 13 bd 8b 89 df bd |c5`.R-B.S.......|
> 43eaa030 5b 05 c1 f4 ae b1 b9 b3 53 ef 14 a5 8e 4a ce 1e |[.......S....J..|
> 48100030 7c 6b 49 55 12 00 19 00 53 ef 00 00 01 20 76 ed ||kIU....S.... v.|
> 4c538830 67 7e 80 bd 68 d0 d8 d4 53 ef 4d 53 20 e6 40 51 |g~..h...S.MS .@Q|
> 4f810c30 46 0f 08 65 ab e1 fe 6a 53 ef 9c 7b 26 64 bf ff |F..e...jS..{&d..|
> 59cf1430 df 83 79 60 ff 96 08 6d 53 ef 37 ef 98 a6 b6 29 |..y`...mS.7....)|
> 5f865030 59 fc ab 10 94 83 66 55 53 ef 75 da 43 d0 e8 c1 |Y.....fUS.u.C...|
> 6d536030 59 84 da 0c d1 7b 07 47 53 ef 10 cf 28 ea 73 6d |Y....{.GS...(.sm|
> 6df76430 d1 3e 2e 1f 15 0e b6 73 53 ef 44 16 7a e7 14 fa |.>.....sS.D.z...|
> 715f0430 bc bf 2c 95 a3 0d cb 9d 53 ef 4a 32 2d 40 ff f3 |..,.....S.J2-@..|
> 721f5c30 cb 64 cb ea 0a 89 90 a7 53 ef 5e 8f ff fb b0 6c |.d......S.^....l|
> 7a702c30 2b 84 11 5a 9a 9e 14 42 53 ef 0e ec d6 b5 02 c4 |+..Z...BS.......|
> 7fa5d430 93 0a 2d e1 20 b5 d6 b4 53 ef 07 46 95 9f cb 1c |..-. ...S..F....|
> 889fcc30 f0 d1 61 a4 73 79 ab b3 53 ef 06 90 38 a1 2a 25 |..a.sy..S...8.*%|
> 89611030 8e bd c1 20 bc 77 91 e9 53 ef b3 73 89 dc 6c 4a |... .w..S..s..lJ|
> 8e67cc30 43 11 37 a8 86 37 b2 66 53 ef 14 a4 a8 a3 91 15 |C.7..7.fS.......|
> a8029030 ae 74 9a f0 03 c7 66 80 53 ef 3f 30 4e 83 a4 c2 |.t....f.S.?0N...|
> a82ec030 ca 76 cc 64 79 6d 57 25 53 ef 46 65 e0 b7 08 7a |.v.dymW%S.Fe...z|
> addb3430 ff e7 ee 6b 67 9d 9e d7 53 ef e2 da b9 cf 5b f6 |...kg...S.....[.|
> b9411430 4a 01 00 3f c9 e2 22 15 53 ef dd 6f 45 ad fb f7 |J..?..".S..oE...|
> baf98030 ff fe e7 41 03 36 f7 11 53 ef ed fc 14 02 34 f0 |...A.6..S.....4.|
> bb6f5830 84 75 62 8d 11 48 20 f4 53 ef 04 36 62 71 20 d2 |.ub..H .S..6bq .|
> bcc59830 b5 e4 fd f7 b2 fe a3 30 53 ef 7a 0a b1 2e a2 9a |.......0S.z.....|
> c0e42030 6e d6 35 c8 53 64 55 a3 53 ef b1 39 d4 78 99 c2 |n.5.SdU.S..9.x..|
> c57a3c30 2f 51 ce a9 9a 5f c9 4d 53 ef 04 d4 eb 8f 52 e6 |/Q..._.MS.....R.|
> ccb2e030 12 df 4a ff a6 c6 37 96 53 ef a7 f6 1c d6 ad d1 |..J...7.S.......|
> cfef5030 59 5c 47 9d e3 5b c7 18 53 ef 6e 1a 61 99 95 fe |Y\G..[..S.n.a...|
> d63c1030 6c 4a f5 0f 8f 97 17 91 53 ef d1 cf 87 65 42 74 |lJ......S....eBt|
> d8100030 7c 6b 49 55 12 00 19 00 53 ef 00 00 01 20 76 ed ||kIU....S.... v.|
So, we now know your data offset == 2048, with pretty high confidence.
We know the possible order combinations:
sdc1 sdb1 sdd1 sde1
sdc1 sdd1 sdb1 sde1
sde1 sdb1 sdd1 sdc1
sde1 sdd1 sdb1 sdc1
We don't know the chunk size for sure. I don't remember when the default
changed from 64 to 512. I'd try 64 first.
It's possible you could map the standard locations of ext4 superblocks
versus the locations noted above, and work out the order and chunk
size. I'll leave that as an exercise for you, if you don't want to
just try the various combinations. :-)
Remember, "fsck -n" after each --create --assume-clean to see if it
works well.
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 13:55 ` Phil Turmel
@ 2015-11-10 14:55 ` Mathias Mueller
2015-11-10 15:20 ` Mathias Mueller
1 sibling, 0 replies; 31+ messages in thread
From: Mathias Mueller @ 2015-11-10 14:55 UTC (permalink / raw)
To: Phil Turmel; +Cc: linux-raid
Hi Phil,
> 'screen' is your friend for long-running server tasks. It's installed
> in many distros by default.
> I should have put "|head -n 50" on the end of that. :-)
I already restarted the process via screen, I just did not expect, that
it will take so much time :)
> This ^^ is encouraging: Sun, 08 Nov 2015 20:06:00 GMT
> It's in the right spot for data offset == 2048 if this
> is the first device.
yes, this is about the last time I mounted the array
> These ^^ look like backup ext4 superblocks, written Thu, 07 May 2015
> 13:41:16 GMT. If you had uptime from May all the way to November, this
> is believable. And there's more of them scattered below.
nope, I moved the array to another box last evening sunday
> Another candidate for a first disk, with offset 2048. Also written
> Sun, 08 Nov 2015 20:06:00 GMT. Either sdc or sde is your first disk.
> The other is parity, which is the last disk on the first stripe, with
> zeros in the matching offsets of the other two disks.
> So, we now know your data offset == 2048, with pretty high confidence.
> We know the possible order combinations:
> sdc1 sdb1 sdd1 sde1
> sdc1 sdd1 sdb1 sde1
> sde1 sdb1 sdd1 sdc1
> sde1 sdd1 sdb1 sdc1
>
> We don't know the chunk size for sure. I don't remember when the
> default
> changed from 64 to 512. I'd try 64 first.
>
> It's possible you could map the standard locations of ext4 superblocks
> versus the locations noted above, and work out the order and chunk
> size. I'll leave that as an exercise for you, if you don't want to
> just try the various combinations. :-)
>
> Remember, "fsck -n" after each --create --assume-clean to see if it
> works well.
You are great, I'll give it a try and try the combinations, thx a lot!
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 13:55 ` Phil Turmel
2015-11-10 14:55 ` Mathias Mueller
@ 2015-11-10 15:20 ` Mathias Mueller
2015-11-10 15:28 ` Phil Turmel
1 sibling, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2015-11-10 15:20 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid
Hi Phil,
Is it correct to use --data-offset=1024 when I want to set an offset of
2048? At least a mdadm --examine on one of the devices shows the offset
is set to 2048 when using --data-offset=1024.
With --data-offset=1024 and both chunk sizes (64 & 512) I have something
like a success when using this two orders:
sde sdd sdb sdc
sde sdb sdd sdc
I'm getting:
e2fsck 1.41.12 (17-May-2010)
/dev/md0 has unsupported feature(s): FEATURE_C16 FEATURE_C17 FEATURE_C18
FEATURE_C19 FEATURE_C21 FEATURE_C22 FEATURE_C23 FEATURE_C25 FEATURE_C27
FEATURE_C28 FEATURE_I29 FEATURE_R29
e2fsck: Get a newer version of e2fsck!
So I guess I should try some newer fsck version (I'm back on the old
machine with centos 6, the failed reshaping was done with the new
machine with centos 7).
Thanks
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 15:20 ` Mathias Mueller
@ 2015-11-10 15:28 ` Phil Turmel
2015-11-10 21:02 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2015-11-10 15:28 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid
On 11/10/2015 10:20 AM, Mathias Mueller wrote:
> Hi Phil,
>
> Is it correct to use --data-offset=1024 when I want to set an offset of
> 2048? At least a mdadm --examine on one of the devices shows the offset
> is set to 2048 when using --data-offset=1024.
This must be a bug in an old mdadm. Don't use it.
> With --data-offset=1024 and both chunk sizes (64 & 512) I have something
> like a success when using this two orders:
>
> sde sdd sdb sdc
> sde sdb sdd sdc
>
> I'm getting:
>
> e2fsck 1.41.12 (17-May-2010)
> /dev/md0 has unsupported feature(s): FEATURE_C16 FEATURE_C17 FEATURE_C18
> FEATURE_C19 FEATURE_C21 FEATURE_C22 FEATURE_C23 FEATURE_C25 FEATURE_C27
> FEATURE_C28 FEATURE_I29 FEATURE_R29
> e2fsck: Get a newer version of e2fsck!
>
> So I guess I should try some newer fsck version (I'm back on the old
> machine with centos 6, the failed reshaping was done with the new
> machine with centos 7).
Whoa! Don't do that. Upgrade to CentOS 7, or go back to the other
machine. You didn't mention switching machines, btw. That would
explain the different drive names vs. roles. You need to document the
device names vs. drive serial numbers on the current setup before moving
or upgrading. Then adjust the names worked out above based on the
serial numbers in the new environment.
In general, device names cannot be considered reliable -- that's why
labels and uuids are the preferred way to identify things in today's
systems. And why mdadm has superblocks to keep track of what's what.
And when you have your system fully running again, document the entire
layout with lsdrv [1].
Phil
[1] https://github.com/pturmel/lsdrv
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 15:28 ` Phil Turmel
@ 2015-11-10 21:02 ` Mathias Mueller
2015-11-10 21:11 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2015-11-10 21:02 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil
> Whoa! Don't do that. Upgrade to CentOS 7, or go back to the other
> machine. You didn't mention switching machines, btw. That would
> explain the different drive names vs. roles. You need to document the
> device names vs. drive serial numbers on the current setup before
> moving
> or upgrading. Then adjust the names worked out above based on the
> serial numbers in the new environment.
oh I'm sorry, I should have make more clear, that I went back to the old
system after the raid reshape failed. All outputs I posted were from the
same (old one) machine.
Now I booted the latest centos 7 rescue, but fsck still tells the same:
/dev/md0 has unsupported feature(s): FEATURE_C16 FEATURE_C17 FEATURE_C18
FEATURE_C19 FEATURE_C21 FEATURE_C22 FEATURE_C23 FEATURE_C25 FEATURE_C27
FEATURE_C28 FEATURE_I29 FEATURE_R29
e2fsck: Get a newer version of e2fsck!
I created the array with this command:
mdadm --create --assume-clean -l5 -n4 --chunk 64 --data-offset 1024
--metadata 1.2 /dev/md0 ......
This both combinations
The devices names are now different, but I adjusted the names by serial
numbers as you told.
Also mdadm --examine on on of the drives tells "Data offset: 2048
Sectors" when mdadm --create --data-offset 1024 is used. Is this normal?
It's confusing me (using --data-offset 2048 on creation gives "Data
offset: 2048 Sectors").
Thanks,
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 21:02 ` Mathias Mueller
@ 2015-11-10 21:11 ` Phil Turmel
0 siblings, 0 replies; 31+ messages in thread
From: Phil Turmel @ 2015-11-10 21:11 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
On 11/10/2015 04:02 PM, Mathias Mueller wrote:
> /dev/md0 has unsupported feature(s): FEATURE_C16 FEATURE_C17 FEATURE_C18
> FEATURE_C19 FEATURE_C21 FEATURE_C22 FEATURE_C23 FEATURE_C25 FEATURE_C27
> FEATURE_C28 FEATURE_I29 FEATURE_R29
> e2fsck: Get a newer version of e2fsck!
None of the feature bits are documented to exist. The choice of first
drive must be wrong.
> I created the array with this command:
>
> mdadm --create --assume-clean -l5 -n4 --chunk 64 --data-offset 1024
> --metadata 1.2 /dev/md0 ......
>
> This both combinations
Well, I gave you *four* combinations for order, and two suggestions for
chunk size. Eight combinations to try.
> The devices names are now different, but I adjusted the names by serial
> numbers as you told.
Ok.
> Also mdadm --examine on on of the drives tells "Data offset: 2048
> Sectors" when mdadm --create --data-offset 1024 is used. Is this normal?
> It's confusing me (using --data-offset 2048 on creation gives "Data
> offset: 2048 Sectors").
mdadm must be enforcing a minimum offset.
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-22 19:13 ` Phil Turmel
@ 2016-01-25 10:02 ` Mathias Mueller
0 siblings, 0 replies; 31+ messages in thread
From: Mathias Mueller @ 2016-01-25 10:02 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil,
thanks a lot once more, I would never managed it without your help. It's
very nice, that you spend so much time in supporting everyone and
writing very helpful tools. I learned a lot, technically and also to
have more patience and thinking twice _before_ doing things :)
> Sadly, it's hard to ship beer here in the US without government
> involvement.
That's a very sad fact! I send you something via paypal, so you might
get some german beer from some US store, if you like. Tell me, if you
need consulting ;)
Keep on going and best regards
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-22 17:39 ` Mathias Mueller
@ 2016-01-22 19:13 ` Phil Turmel
2016-01-25 10:02 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2016-01-22 19:13 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
On 01/22/2016 12:39 PM, Mathias Mueller wrote:
> Hi Phil,
>
> you are the man! The first one is working!
Ah, that's good.
> So please tell me, where does the beer delivery goes to? :)
Sadly, it's hard to ship beer here in the US without government involvement.
> Thank you so much, especially for so much patience with the dumbest user
> on earth.
You're welcome. But don't put yourself down -- it was your idea to
search for parts of a known file, after all.
> Mathias
>
> PS: I'm serious with the beer or whatever you prefer
A modest tip via paypal would be fine, if you insist. Or something from
my public wishlist:
http://smile.amazon.com/registry/wishlist/2EE91JWNIYSC3
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-22 17:16 ` Phil Turmel
@ 2016-01-22 17:39 ` Mathias Mueller
2016-01-22 19:13 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2016-01-22 17:39 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil,
you are the man! The first one is working!
So please tell me, where does the beer delivery goes to? :)
Thank you so much, especially for so much patience with the dumbest user
on earth.
Mathias
PS: I'm serious with the beer or whatever you prefer
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-22 9:30 ` Mathias Mueller
@ 2016-01-22 17:16 ` Phil Turmel
2016-01-22 17:39 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2016-01-22 17:16 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
Hi Mathias,
On 01/22/2016 04:30 AM, Mathias Mueller wrote:
> Hi Phil,
>
> it's done:
>
> 658 Unique hashes in picture.jpg
Looks good. Rearranging a bit:
> /dev/sdf1 @ 005400caa000:005400cfffff ~= 00000000:00055fff
> /dev/sde1 @ 005400b90000:005400c0ffff ~= 00056000:000d5fff
> /dev/sdd1 @ 005400c80000:005400cfffff ~= 000d6000:00155fff
> /dev/sdc1 @ 005400d00000:005400d7ffff ~= 00156000:001d5fff
> /dev/sdf1 @ 005400d00000:005400d7ffff ~= 001d6000:00255fff
> /dev/sde1 @ 005400c10000:005400c4bfff ~= 00256000:00291fff
> /dev/sdc1 @ 0087b4f29000:0087b4f7ffff ~= 00000000:00056fff
> /dev/sdf1 @ 0087b4f00000:0087b4f7ffff ~= 00057000:000d6fff
> /dev/sde1 @ 0087b4e10000:0087b4e8ffff ~= 000d7000:00156fff
> /dev/sdd1 @ 0087b4f80000:0087b4ffffff ~= 00157000:001d6fff
> /dev/sdc1 @ 0087b4f80000:0087b4ffffff ~= 001d7000:00256fff
> /dev/sdf1 @ 0087b4f80000:0087b4fbafff ~= 00257000:00291fff
> /dev/sdf1 @ 00cb2d956000:00cb2d97ffff ~= 00000000:00029fff
> /dev/sde1 @ 00cb2d810000:00cb2d88ffff ~= 0002a000:000a9fff
> /dev/sdd1 @ 00cb2d980000:00cb2d9fffff ~= 000aa000:00129fff
> /dev/sdc1 @ 00cb2d980000:00cb2d9fffff ~= 0012a000:001a9fff
> /dev/sdf1 @ 00cb2d980000:00cb2d9fffff ~= 001aa000:00229fff
> /dev/sde1 @ 00cb2d910000:00cb2d977fff ~= 0022a000:00291fff
Device order is definitely sdf1, sde1, sdd1, sdc1, sdf1 ....
> End of /dev/sdc1 at 01d1c1016000
> End of /dev/sdd1 at 01d1c1016000
> End of /dev/sde1 at 01d1c0e90000
> End of /dev/sdf1 at 01d1c110e000
sdc1 and sdd1 are the same size and have the same chunk alignment
offsets. sdf1 is bigger but has the same chunk offsets as sdc1 and
sdd1. sde1 is smaller than sd[cd]1 by 0x186000 and has chunk offsets
0xf0000 earlier than the others for a given stripe.
sde1 has a data offset that ends in 0x10000 or 0x90000. 0x10000 is 64k,
an offset used by default early in mdadm's history.
The others have data offsets that end in 0x00000 or 0x80000. 0x80000 is
512k. 0x100000 is 1M, another default offset in mdadm history, and
happily the correct increment (0xf0000) bigger than the 64k suspicion I
have for sde1.
Here are the four --create operations I would try:
(This first one is most likely it based on the signature analysis in
November.)
mdadm --create --assume-clean --data-offset=variable \
--raid-devices=4 --chunk=512 --level=5 /dev/mdX \
/dev/sdc1:1024 /dev/sdf1:1024 /dev/sde1:64 /dev/sdd1:1024
mdadm --create --assume-clean --data-offset=variable \
--raid-devices=4 --chunk=512 --level=5 /dev/mdX \
/dev/sdf1:1024 /dev/sde1:64 /dev/sdd1:1024 /dev/sdc1:1024
mdadm --create --assume-clean --data-offset=variable \
--raid-devices=4 --chunk=512 --level=5 /dev/mdX \
/dev/sde1:64 /dev/sdd1:1024 /dev/sdc1:1024 /dev/sdf1:1024
mdadm --create --assume-clean --data-offset=variable \
--raid-devices=4 --chunk=512 --level=5 /dev/mdX \
/dev/sdd1:1024 /dev/sdc1:1024 /dev/sdf1:1024 /dev/sde1:64
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-20 9:04 ` Mathias Mueller
@ 2016-01-22 9:30 ` Mathias Mueller
2016-01-22 17:16 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2016-01-22 9:30 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil,
it's done:
658 Unique hashes in picture.jpg
Searching for pieces of picture.jpg in /dev/sdc1:...
/dev/sdc1 @ 005400d00000:005400d7ffff ~= 00156000:001d5fff
/dev/sdc1 @ 0087b4f29000:0087b4f7ffff ~= 00000000:00056fff
/dev/sdc1 @ 0087b4f80000:0087b4ffffff ~= 001d7000:00256fff
/dev/sdc1 @ 00cb2d980000:00cb2d9fffff ~= 0012a000:001a9fff
End of /dev/sdc1 at 01d1c1016000
Searching for pieces of picture.jpg in /dev/sdd1:...
/dev/sdd1 @ 005400c80000:005400cfffff ~= 000d6000:00155fff
/dev/sdd1 @ 0087b4f80000:0087b4ffffff ~= 00157000:001d6fff
/dev/sdd1 @ 00cb2d980000:00cb2d9fffff ~= 000aa000:00129fff
End of /dev/sdd1 at 01d1c1016000
Searching for pieces of picture.jpg in /dev/sde1:...
/dev/sde1 @ 005400b90000:005400c0ffff ~= 00056000:000d5fff
/dev/sde1 @ 005400c10000:005400c4bfff ~= 00256000:00291fff
/dev/sde1 @ 0087b4e10000:0087b4e8ffff ~= 000d7000:00156fff
/dev/sde1 @ 00cb2d810000:00cb2d88ffff ~= 0002a000:000a9fff
/dev/sde1 @ 00cb2d910000:00cb2d977fff ~= 0022a000:00291fff
End of /dev/sde1 at 01d1c0e90000
Searching for pieces of picture.jpg in /dev/sdf1:...
/dev/sdf1 @ 005400caa000:005400cfffff ~= 00000000:00055fff
/dev/sdf1 @ 005400d00000:005400d7ffff ~= 001d6000:00255fff
/dev/sdf1 @ 0087b4f00000:0087b4f7ffff ~= 00057000:000d6fff
/dev/sdf1 @ 0087b4f80000:0087b4fbafff ~= 00257000:00291fff
/dev/sdf1 @ 00cb2d956000:00cb2d97ffff ~= 00000000:00029fff
/dev/sdf1 @ 00cb2d980000:00cb2d9fffff ~= 001aa000:00229fff
End of /dev/sdf1 at 01d1c110e000
Mathias
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-19 19:37 ` Phil Turmel
@ 2016-01-20 9:04 ` Mathias Mueller
2016-01-22 9:30 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2016-01-20 9:04 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil,
thanks a lot for your patience and sorry for driving you crazy. I will
post results as soon as your nice tool has finished :)
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-19 17:51 ` Phil Turmel
@ 2016-01-19 19:37 ` Phil Turmel
2016-01-20 9:04 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2016-01-19 19:37 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
On 01/19/2016 12:51 PM, Phil Turmel wrote:
> Sigh. I couldn't help myself. New utility attached. Curse you Mathias
> for an interesting problem! ;-)
I decided that this might be useful to others, particularly if someone
wants to tweak it for more parallelization, so I prettied it up for
github with a readme and a proper copyright statement.
https://github.com/pturmel/findHash
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-19 14:35 ` Mathias Mueller
@ 2016-01-19 17:51 ` Phil Turmel
2016-01-19 19:37 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2016-01-19 17:51 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
[-- Attachment #1: Type: text/plain, Size: 1389 bytes --]
Hi Mathias,
On 01/19/2016 09:35 AM, Mathias Mueller wrote:
> Hi Phil,
>
> I forgot to add some information: when I was creating the bytestrings
> from my jpg file, I did not start from 0k but from 100k of the jpg file
> (to skip the jpg header).
Ok. But I'm still not confident of chunk boundaries.
>> Very interesting. You could go one step further and compare the jpeg
>> file contents in the first 1M against the locations found to determine
>> where the chunks actually start and end on each device. The final
>> offset will be a chunk multiple before these boundaries. Or do md5 sums
>> of 4k blocks to reduce the amount to inspect.
>
> How exactly can I do this? Should I create more Bytestrings and do more
> brep with them on my physical devices? I have already results from
> searching bytestrings with an offset of 64k (starting from 100k to 612k
> of my jpeg file, so 9 bytestrings at all). Should I provide a table of
> the results?
Sigh. I couldn't help myself. New utility attached. Curse you Mathias
for an interesting problem! ;-)
Call it with your jpeg and the devices to search, like so:
findHash.py /path/to/picture.jpeg /dev/sd[bcde]
It'll make a map of hashes of each 4k block in the jpeg and then search
the listed devices for those hashes, building a map of the file
fragments. This will clearly show chunk boundaries.
Please show the output.
Phil
[-- Attachment #2: findHash.py --]
[-- Type: text/x-python, Size: 2243 bytes --]
#! /usr/bin/python2
#
# Locate 4k fragments of a subject file in one or more other files or
# devices. Only reports two or more consecutive matches.
#
# Usage:
# findHash.py /path/to/subject/file /dev/sdx|/path/to/image/file [/dev/sdy ...]
import hashlib, sys, datetime
# Read the known file 4k at a time, building a dictionary of
# md5 hashes vs. offset. Use a large buffer for speed.
# Drops any partial block at the end of the file.
d = {}
pos = long(0)
f = open(sys.argv[1], 'r', 1<<20)
b = f.read(4096)
while len(b)==4096:
md5 = hashlib.md5()
md5.update(b)
h = md5.digest()
hlist = d.get(h)
if not hlist:
hlist = []
d[h] = hlist
# print "New hash %s at %8.8x" % (h.encode('hex'), pos)
hlist.append(pos)
pos += 4096
b = f.read(4096)
f.close()
print "%d Unique hashes in %s" % (len(d), sys.argv[1])
def checkAndPrint(match):
if match[2]>4096:
print "%20s @ %12.12x:%12.12x ~= %8.8x:%8.8x" % (fname, match[1], match[1]+match[2]-1, match[0], match[0]+match[2]-1)
# Read the candidate files/devices, looking for possible matches. Match
# entries are vectors of known file offset, candidate file offset, and
# length.
for fname in sys.argv[2:]:
print "\nSearching for pieces of %s in %s:..." % (sys.argv[1], fname)
pos = long(0)
f = open(fname, 'r', 1<<24)
matches = []
b = f.read(4096)
lastts = None
while len(b)==4096:
if not (pos & 0x7ffffff):
ts = datetime.datetime.now()
if lastts:
print "@ %12.12x %.1fMB/s \r" % (pos, 128.0/((ts-lastts).total_seconds())),
else:
print "@ %12.12x...\r" % pos,
sys.stdout.flush()
lastts = ts
md5 = hashlib.md5()
md5.update(b)
h = md5.digest()
if h in d:
i = 0
while i<len(matches):
match = matches[i]
target = match[0]+match[2]
continuations = [x for x in d[h] if x==target]
if continuations:
match[2] += 4096
i += 1
else:
del matches[i]
checkAndPrint(match)
if not matches:
matches = [[x, pos, 4096] for x in d[h]]
else:
for match in matches:
checkAndPrint(match)
matches = []
pos += 4096
b = f.read(4096)
print "End of %s at %12.12x" % (fname, pos)
# show matches that continue to the end of the candidate file/device.
for match in matches:
checkAndPrint(match)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-18 19:09 ` Phil Turmel
@ 2016-01-19 14:35 ` Mathias Mueller
2016-01-19 17:51 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2016-01-19 14:35 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil,
I forgot to add some information: when I was creating the bytestrings
from my jpg file, I did not start from 0k but from 100k of the jpg file
(to skip the jpg header).
> Very interesting. You could go one step further and compare the jpeg
> file contents in the first 1M against the locations found to determine
> where the chunks actually start and end on each device. The final
> offset will be a chunk multiple before these boundaries. Or do md5
> sums
> of 4k blocks to reduce the amount to inspect.
How exactly can I do this? Should I create more Bytestrings and do more
brep with them on my physical devices? I have already results from
searching bytestrings with an offset of 64k (starting from 100k to 612k
of my jpeg file, so 9 bytestrings at all). Should I provide a table of
the results?
> This will help (and looks correct), but I don't remember what we did to
> try to find the beginning of your filesystem. Did we search for a
> filesystem signature? Please re-summarize that.
Yes, we were searching for an ext4 signature, there were some fragments
of old file systems, but you also found two promising signatures, both
with this timestamp: Sun, 08 Nov 2015 20:06:00 GMT (which is
approximately the last time I mounted the filesystem)
here is a link to the archived message:
http://www.spinics.net/lists/raid/msg50433.html
the device names have changed from the archived post to now:
sdb -> sdf Serial Number: JK1170YBHYV6MD
sdc -> sdc Serial Number: JK1100YAG64A1T
sdd -> sde Serial Number: JK1121YAG7YDLS
sde -> sdd Serial Number: ML0220F30PZUVD
Thanks and best regards
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2016-01-18 15:33 ` Mathias Mueller
@ 2016-01-18 19:09 ` Phil Turmel
2016-01-19 14:35 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2016-01-18 19:09 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
Hi Mathias,
On 01/18/2016 10:33 AM, Mathias Mueller wrote:
> Hi Phil,
>
> first thanks a lot for all your help and time you already spent on me
> and my problem. I gathered some more information and hope you can help
> me a bit more.
You're welcome.
> First I tried to figure out the chunk size. I used a jpg file that was
> stored on the raid for sure and read 8x32 Bytes with an offset of 64k
> from that file. Than I used bgrep to search for the bytestrings on my
> physical devices. I found five sequent byte strings on one physical
> device, so I think the chunk sice must be bigger than 256k.
Very interesting. You could go one step further and compare the jpeg
file contents in the first 1M against the locations found to determine
where the chunks actually start and end on each device. The final
offset will be a chunk multiple before these boundaries. Or do md5 sums
of 4k blocks to reduce the amount to inspect.
> If I am right, it would mean, that the order of my physical devices is
> sdf1 sdd1 sde1 sdc1 (without knowing which one is the first partition)
> and the offset between sdd1 and the other devices is f0000. Are my
> assumptions correct? How can I go on with this information?
This will help (and looks correct), but I don't remember what we did to
try to find the beginning of your filesystem. Did we search for a
filesystem signature? Please re-summarize that.
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-11 17:53 ` Mathias Mueller
@ 2016-01-18 15:33 ` Mathias Mueller
2016-01-18 19:09 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2016-01-18 15:33 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil,
first thanks a lot for all your help and time you already spent on me
and my problem. I gathered some more information and hope you can help
me a bit more.
I found this on serverfault.com and tried to figure out my raid layout
using some image file that is stored on my raid for sure:
http://serverfault.com/questions/347606/recover-raid-5-data-after-created-new-array-instead-of-re-using
this is my current device mapping:
JK1100YAG64A1T -> sdc
ML0220F30PZUVD -> sdd (this is the disk I added last)
JK1121YAG7YDLS -> sde
JK1170YBHYV6MD -> sdf
First I tried to figure out the chunk size. I used a jpg file that was
stored on the raid for sure and read 8x32 Bytes with an offset of 64k
from that file. Than I used bgrep to search for the bytestrings on my
physical devices. I found five sequent byte strings on one physical
device, so I think the chunk sice must be bigger than 256k.
Because I was pretty sure, that my chunk size is 512k, I was reading
5x32Byte with an offset of 512k from my jpg file, to determine the order
of the devices. After bgrep was done, I got the following results:
- Every bytestring was found 3x on my physical devices, I think it's
because the file is existing 3x in different folders on my raid device
(which is pretty certain, I know that the file is existing 2x at least)
- sdd1 seems to have a different offset as the other physical devices,
it's the disk I added last after switching from debian to centos some
years ago
- the following tables show at which offsets and on which physical
devices the five byte strings have been found:
offset sdc1 sdd1 sde1 sdf1
---------------------------------------------------
5400bd3000 2
5400cc3000 3 1
5400d43000 4 5
offset sdc1 sdd1 sde1 sdf1
---------------------------------------------------
87b4e52000 3
87b4f42000 1 2
87b4fc2000 5 4
offset sdc1 sdd1 sde1 sdf1
---------------------------------------------------
cb2d87f000 2
cb2d96f000 1
cb2d9ef000 4 3 5
So I think using the last table is the easiest one to figure out the
partition order and the data offset between sdd1 and the other devices?
offset sdc1 sdd1 sde1 sdf1
---------------------------------------------------
cb2d96f000 (2) P 1
cb2d9ef000 4 P 3 5
If I am right, it would mean, that the order of my physical devices is
sdf1 sdd1 sde1 sdc1 (without knowing which one is the first partition)
and the offset between sdd1 and the other devices is f0000. Are my
assumptions correct? How can I go on with this information?
Thanks a lot in advance
Best regards
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-11 1:00 ` Phil Turmel
@ 2015-11-11 17:53 ` Mathias Mueller
2016-01-18 15:33 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2015-11-11 17:53 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid
Hi Phil,
> So, I fully expected one of those first four combinations to work. I
> don't know why they didn't.
>
> The fsck version dates to 2013, so it might be something related to
> that. My next step would be to try all of this after booting into a
> modern environment, like the rescue cd from sysrescuecd.org. Recheck
> serial numbers and device names again, if you do that.
I tried the first four combinations with latest sysrescuecd, but the
results are the same.
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-11 1:03 ` Phil Turmel
@ 2015-11-11 1:29 ` Mathias Mueller
0 siblings, 0 replies; 31+ messages in thread
From: Mathias Mueller @ 2015-11-11 1:29 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil,
> I just noticed your response to this didn't make it to the list. It
> was
> 108k. If you do it again with count=8 -- that'll be enough for
> everyone
> to understand my other reply.
no problem, here you go:
/dev/sde1
00000000 00 00 c0 1f 10 00 c0 1f 20 00 c0 1f 00 00 00 20 |........
...... |
00000010 00 00 05 00 00 00 00 00 00 00 00 00 00 20 df 2d
|............. .-|
00000020 01 00 c0 1f 11 00 c0 1f 20 02 c0 1f 00 00 00 20 |........
...... |
00000030 00 00 05 00 00 00 00 00 00 00 00 00 00 20 04 8d
|............. ..|
00000040 02 00 c0 1f 12 00 c0 1f 20 04 c0 1f 00 00 00 20 |........
...... |
00000050 00 00 05 00 00 00 00 00 00 00 00 00 00 20 6a 2c
|............. j,|
00000060 03 00 c0 1f 13 00 c0 1f 20 06 c0 1f 00 00 00 20 |........
...... |
00000070 00 00 05 00 00 00 00 00 00 00 00 00 00 20 b1 8c
|............. ..|
00000080 04 00 c0 1f 14 00 c0 1f 20 08 c0 1f 00 00 00 20 |........
...... |
00000090 00 00 05 00 00 00 00 00 00 00 00 00 00 20 b5 2e
|............. ..|
000000a0 05 00 c0 1f 15 00 c0 1f 20 0a c0 1f 00 00 00 20 |........
...... |
000000b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 6e 8e
|............. n.|
000000c0 06 00 c0 1f 16 00 c0 1f 20 0c c0 1f 00 00 00 20 |........
...... |
000000d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 00 2f
|............. ./|
000000e0 07 00 c0 1f 17 00 c0 1f 20 0e c0 1f 00 00 00 20 |........
...... |
000000f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 db 8f
|............. ..|
00000100 08 00 c0 1f 18 00 c0 1f 20 10 c0 1f 00 00 00 20 |........
...... |
00000110 00 00 05 00 00 00 00 00 00 00 00 00 00 20 0b 2b
|............. .+|
00000120 09 00 c0 1f 19 00 c0 1f 20 12 c0 1f 00 00 00 20 |........
...... |
00000130 00 00 05 00 00 00 00 00 00 00 00 00 00 20 d0 8b
|............. ..|
00000140 0a 00 c0 1f 1a 00 c0 1f 20 14 c0 1f 00 00 00 20 |........
...... |
00000150 00 00 05 00 00 00 00 00 00 00 00 00 00 20 be 2a
|............. .*|
00000160 0b 00 c0 1f 1b 00 c0 1f 20 16 c0 1f 00 00 00 20 |........
...... |
00000170 00 00 05 00 00 00 00 00 00 00 00 00 00 20 65 8a
|............. e.|
00000180 0c 00 c0 1f 1c 00 c0 1f 20 18 c0 1f 00 00 00 20 |........
...... |
00000190 00 00 05 00 00 00 00 00 00 00 00 00 00 20 61 28
|............. a(|
000001a0 0d 00 c0 1f 1d 00 c0 1f 20 1a c0 1f 00 00 00 20 |........
...... |
000001b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 ba 88
|............. ..|
000001c0 0e 00 c0 1f 1e 00 c0 1f 20 1c c0 1f 00 00 00 20 |........
...... |
000001d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 d4 29
|............. .)|
000001e0 0f 00 c0 1f 1f 00 c0 1f 20 1e c0 1f 00 00 00 20 |........
...... |
000001f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 0f 89
|............. ..|
00000200 00 00 c8 1f 10 00 c8 1f 20 00 c8 1f 00 00 00 20 |........
...... |
00000210 00 00 05 00 00 00 00 00 00 00 00 00 00 20 c9 5e
|............. .^|
00000220 01 00 c8 1f 11 00 c8 1f 20 02 c8 1f 00 00 00 20 |........
...... |
00000230 00 00 05 00 00 00 00 00 00 00 00 00 00 20 12 fe
|............. ..|
00000240 02 00 c8 1f 12 00 c8 1f 20 04 c8 1f 00 00 00 20 |........
...... |
00000250 00 00 05 00 00 00 00 00 00 00 00 00 00 20 7c 5f
|............. |_|
00000260 03 00 c8 1f 13 00 c8 1f 20 06 c8 1f 00 00 00 20 |........
...... |
00000270 00 00 05 00 00 00 00 00 00 00 00 00 00 20 a7 ff
|............. ..|
00000280 04 00 c8 1f 14 00 c8 1f 20 08 c8 1f 00 00 00 20 |........
...... |
00000290 00 00 05 00 00 00 00 00 00 00 00 00 00 20 a3 5d
|............. .]|
000002a0 05 00 c8 1f 15 00 c8 1f 20 0a c8 1f 00 00 00 20 |........
...... |
000002b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 78 fd
|............. x.|
000002c0 06 00 c8 1f 16 00 c8 1f 20 0c c8 1f 00 00 00 20 |........
...... |
000002d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 16 5c
|............. .\|
000002e0 07 00 c8 1f 17 00 c8 1f 20 0e c8 1f 00 00 00 20 |........
...... |
000002f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 cd fc
|............. ..|
00000300 08 00 c8 1f 18 00 c8 1f 20 10 c8 1f 00 00 00 20 |........
...... |
00000310 00 00 05 00 00 00 00 00 00 00 00 00 00 20 1d 58
|............. .X|
00000320 09 00 c8 1f 19 00 c8 1f 20 12 c8 1f 00 00 00 20 |........
...... |
00000330 00 00 05 00 00 00 00 00 00 00 00 00 00 20 c6 f8
|............. ..|
00000340 0a 00 c8 1f 1a 00 c8 1f 20 14 c8 1f 00 00 00 20 |........
...... |
00000350 00 00 05 00 00 00 00 00 00 00 00 00 00 20 a8 59
|............. .Y|
00000360 0b 00 c8 1f 1b 00 c8 1f 20 16 c8 1f 00 00 00 20 |........
...... |
00000370 00 00 05 00 00 00 00 00 00 00 00 00 00 20 73 f9
|............. s.|
00000380 0c 00 c8 1f 1c 00 c8 1f 20 18 c8 1f 00 00 00 20 |........
...... |
00000390 00 00 05 00 00 00 00 00 00 00 00 00 00 20 77 5b
|............. w[|
000003a0 0d 00 c8 1f 1d 00 c8 1f 20 1a c8 1f 00 00 00 20 |........
...... |
000003b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 ac fb
|............. ..|
000003c0 0e 00 c8 1f 1e 00 c8 1f 20 1c c8 1f 00 00 00 20 |........
...... |
000003d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 c2 5a
|............. .Z|
000003e0 0f 00 c8 1f 1f 00 c8 1f 20 1e c8 1f 00 00 00 20 |........
...... |
000003f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 19 fa
|............. ..|
00000400 00 00 d0 1f 10 00 d0 1f 20 00 d0 1f 00 00 00 20 |........
...... |
00000410 00 00 05 00 00 00 00 00 00 00 00 00 00 20 f3 cb
|............. ..|
00000420 01 00 d0 1f 11 00 d0 1f 20 02 d0 1f 00 00 00 20 |........
...... |
00000430 00 00 05 00 00 00 00 00 00 00 00 00 00 20 28 6b
|............. (k|
00000440 02 00 d0 1f 12 00 d0 1f 20 04 d0 1f 00 00 00 20 |........
...... |
00000450 00 00 05 00 00 00 00 00 00 00 00 00 00 20 46 ca
|............. F.|
00000460 03 00 d0 1f 13 00 d0 1f 20 06 d0 1f 00 00 00 20 |........
...... |
00000470 00 00 05 00 00 00 00 00 00 00 00 00 00 20 9d 6a
|............. .j|
00000480 04 00 d0 1f 14 00 d0 1f 20 08 d0 1f 00 00 00 20 |........
...... |
00000490 00 00 05 00 00 00 00 00 00 00 00 00 00 20 99 c8
|............. ..|
000004a0 05 00 d0 1f 15 00 d0 1f 20 0a d0 1f 00 00 00 20 |........
...... |
000004b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 42 68
|............. Bh|
000004c0 06 00 d0 1f 16 00 d0 1f 20 0c d0 1f 00 00 00 20 |........
...... |
000004d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 2c c9
|............. ,.|
000004e0 07 00 d0 1f 17 00 d0 1f 20 0e d0 1f 00 00 00 20 |........
...... |
000004f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 f7 69
|............. .i|
00000500 08 00 d0 1f 18 00 d0 1f 20 10 d0 1f 00 00 00 20 |........
...... |
00000510 00 00 05 00 00 00 00 00 00 00 00 00 00 20 27 cd
|............. '.|
00000520 09 00 d0 1f 19 00 d0 1f 20 12 d0 1f 00 00 00 20 |........
...... |
00000530 00 00 05 00 00 00 00 00 00 00 00 00 00 20 fc 6d
|............. .m|
00000540 0a 00 d0 1f 1a 00 d0 1f 20 14 d0 1f 00 00 00 20 |........
...... |
00000550 00 00 05 00 00 00 00 00 00 00 00 00 00 20 92 cc
|............. ..|
00000560 0b 00 d0 1f 1b 00 d0 1f 20 16 d0 1f 00 00 00 20 |........
...... |
00000570 00 00 05 00 00 00 00 00 00 00 00 00 00 20 49 6c
|............. Il|
00000580 0c 00 d0 1f 1c 00 d0 1f 20 18 d0 1f 00 00 00 20 |........
...... |
00000590 00 00 05 00 00 00 00 00 00 00 00 00 00 20 4d ce
|............. M.|
000005a0 0d 00 d0 1f 1d 00 d0 1f 20 1a d0 1f 00 00 00 20 |........
...... |
000005b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 96 6e
|............. .n|
000005c0 0e 00 d0 1f 1e 00 d0 1f 20 1c d0 1f 00 00 00 20 |........
...... |
000005d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 f8 cf
|............. ..|
000005e0 0f 00 d0 1f 1f 00 d0 1f 20 1e d0 1f 00 00 00 20 |........
...... |
000005f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 23 6f
|............. #o|
00000600 00 00 d8 1f 10 00 d8 1f 20 00 d8 1f 00 00 00 20 |........
...... |
00000610 00 00 05 00 00 00 00 00 00 00 00 00 00 20 e5 b8
|............. ..|
00000620 01 00 d8 1f 11 00 d8 1f 20 02 d8 1f 00 00 00 20 |........
...... |
00000630 00 00 05 00 00 00 00 00 00 00 00 00 00 20 3e 18
|............. >.|
00000640 02 00 d8 1f 12 00 d8 1f 20 04 d8 1f 00 00 00 20 |........
...... |
00000650 00 00 05 00 00 00 00 00 00 00 00 00 00 20 50 b9
|............. P.|
00000660 03 00 d8 1f 13 00 d8 1f 20 06 d8 1f 00 00 00 20 |........
...... |
00000670 00 00 05 00 00 00 00 00 00 00 00 00 00 20 8b 19
|............. ..|
00000680 04 00 d8 1f 14 00 d8 1f 20 08 d8 1f 00 00 00 20 |........
...... |
00000690 00 00 05 00 00 00 00 00 00 00 00 00 00 20 8f bb
|............. ..|
000006a0 05 00 d8 1f 15 00 d8 1f 20 0a d8 1f 00 00 00 20 |........
...... |
000006b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 54 1b
|............. T.|
000006c0 06 00 d8 1f 16 00 d8 1f 20 0c d8 1f 00 00 00 20 |........
...... |
000006d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 3a ba
|............. :.|
000006e0 07 00 d8 1f 17 00 d8 1f 20 0e d8 1f 00 00 00 20 |........
...... |
000006f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 e1 1a
|............. ..|
00000700 08 00 d8 1f 18 00 d8 1f 20 10 d8 1f 00 00 00 20 |........
...... |
00000710 00 00 05 00 00 00 00 00 00 00 00 00 00 20 31 be
|............. 1.|
00000720 09 00 d8 1f 19 00 d8 1f 20 12 d8 1f 00 00 00 20 |........
...... |
00000730 00 00 05 00 00 00 00 00 00 00 00 00 00 20 ea 1e
|............. ..|
00000740 0a 00 d8 1f 1a 00 d8 1f 20 14 d8 1f 00 00 00 20 |........
...... |
00000750 00 00 05 00 00 00 00 00 00 00 00 00 00 20 84 bf
|............. ..|
00000760 0b 00 d8 1f 1b 00 d8 1f 20 16 d8 1f 00 00 00 20 |........
...... |
00000770 00 00 05 00 00 00 00 00 00 00 00 00 00 20 5f 1f
|............. _.|
00000780 0c 00 d8 1f 1c 00 d8 1f 20 18 d8 1f 00 00 00 20 |........
...... |
00000790 00 00 05 00 00 00 00 00 00 00 00 00 00 20 5b bd
|............. [.|
000007a0 0d 00 d8 1f 1d 00 d8 1f 20 1a d8 1f 00 00 00 20 |........
...... |
000007b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 80 1d
|............. ..|
000007c0 0e 00 d8 1f 1e 00 d8 1f 20 1c d8 1f 00 00 00 20 |........
...... |
000007d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 ee bc
|............. ..|
000007e0 0f 00 d8 1f 1f 00 d8 1f 20 1e d8 1f 00 00 00 20 |........
...... |
000007f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 35 1c
|............. 5.|
00000800 00 00 e0 1f 10 00 e0 1f 20 00 e0 1f 00 00 00 20 |........
...... |
00000810 00 00 05 00 00 00 00 00 00 00 00 00 00 20 84 a1
|............. ..|
00000820 01 00 e0 1f 11 00 e0 1f 20 02 e0 1f 00 00 00 20 |........
...... |
00000830 00 00 05 00 00 00 00 00 00 00 00 00 00 20 5f 01
|............. _.|
00000840 02 00 e0 1f 12 00 e0 1f 20 04 e0 1f 00 00 00 20 |........
...... |
00000850 00 00 05 00 00 00 00 00 00 00 00 00 00 20 31 a0
|............. 1.|
00000860 03 00 e0 1f 13 00 e0 1f 20 06 e0 1f 00 00 00 20 |........
...... |
00000870 00 00 05 00 00 00 00 00 00 00 00 00 00 20 ea 00
|............. ..|
00000880 04 00 e0 1f 14 00 e0 1f 20 08 e0 1f 00 00 00 20 |........
...... |
00000890 00 00 05 00 00 00 00 00 00 00 00 00 00 20 ee a2
|............. ..|
000008a0 05 00 e0 1f 15 00 e0 1f 20 0a e0 1f 00 00 00 20 |........
...... |
000008b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 35 02
|............. 5.|
000008c0 06 00 e0 1f 16 00 e0 1f 20 0c e0 1f 00 00 00 20 |........
...... |
000008d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 5b a3
|............. [.|
000008e0 07 00 e0 1f 17 00 e0 1f 20 0e e0 1f 00 00 00 20 |........
...... |
000008f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 80 03
|............. ..|
00000900 08 00 e0 1f 18 00 e0 1f 20 10 e0 1f 00 00 00 20 |........
...... |
00000910 00 00 05 00 00 00 00 00 00 00 00 00 00 20 50 a7
|............. P.|
00000920 09 00 e0 1f 19 00 e0 1f 20 12 e0 1f 00 00 00 20 |........
...... |
00000930 00 00 05 00 00 00 00 00 00 00 00 00 00 20 8b 07
|............. ..|
00000940 0a 00 e0 1f 1a 00 e0 1f 20 14 e0 1f 00 00 00 20 |........
...... |
00000950 00 00 05 00 00 00 00 00 00 00 00 00 00 20 e5 a6
|............. ..|
00000960 0b 00 e0 1f 1b 00 e0 1f 20 16 e0 1f 00 00 00 20 |........
...... |
00000970 00 00 05 00 00 00 00 00 00 00 00 00 00 20 3e 06
|............. >.|
00000980 0c 00 e0 1f 1c 00 e0 1f 20 18 e0 1f 00 00 00 20 |........
...... |
00000990 00 00 05 00 00 00 00 00 00 00 00 00 00 20 3a a4
|............. :.|
000009a0 0d 00 e0 1f 1d 00 e0 1f 20 1a e0 1f 00 00 00 20 |........
...... |
000009b0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 e1 04
|............. ..|
000009c0 0e 00 e0 1f 1e 00 e0 1f 20 1c e0 1f 00 00 00 20 |........
...... |
000009d0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 8f a5
|............. ..|
000009e0 0f 00 e0 1f 1f 00 e0 1f 20 1e e0 1f 00 00 00 20 |........
...... |
000009f0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 54 05
|............. T.|
00000a00 00 00 e8 1f 10 00 e8 1f 20 00 e8 1f 00 00 00 20 |........
...... |
00000a10 00 00 05 00 00 00 00 00 00 00 00 00 00 20 92 d2
|............. ..|
00000a20 01 00 e8 1f 11 00 e8 1f 20 02 e8 1f 00 00 00 20 |........
...... |
00000a30 00 00 05 00 00 00 00 00 00 00 00 00 00 20 49 72
|............. Ir|
00000a40 02 00 e8 1f 12 00 e8 1f 20 04 e8 1f 00 00 00 20 |........
...... |
00000a50 00 00 05 00 00 00 00 00 00 00 00 00 00 20 27 d3
|............. '.|
00000a60 03 00 e8 1f 13 00 e8 1f 20 06 e8 1f 00 00 00 20 |........
...... |
00000a70 00 00 05 00 00 00 00 00 00 00 00 00 00 20 fc 73
|............. .s|
00000a80 04 00 e8 1f 14 00 e8 1f 20 08 e8 1f 00 00 00 20 |........
...... |
00000a90 00 00 05 00 00 00 00 00 00 00 00 00 00 20 f8 d1
|............. ..|
00000aa0 05 00 e8 1f 15 00 e8 1f 20 0a e8 1f 00 00 00 20 |........
...... |
00000ab0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 23 71
|............. #q|
00000ac0 06 00 e8 1f 16 00 e8 1f 20 0c e8 1f 00 00 00 20 |........
...... |
00000ad0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 4d d0
|............. M.|
00000ae0 07 00 e8 1f 17 00 e8 1f 20 0e e8 1f 00 00 00 20 |........
...... |
00000af0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 96 70
|............. .p|
00000b00 08 00 e8 1f 18 00 e8 1f 20 10 e8 1f 00 00 00 20 |........
...... |
00000b10 00 00 05 00 00 00 00 00 00 00 00 00 00 20 46 d4
|............. F.|
00000b20 09 00 e8 1f 19 00 e8 1f 20 12 e8 1f 00 00 00 20 |........
...... |
00000b30 00 00 05 00 00 00 00 00 00 00 00 00 00 20 9d 74
|............. .t|
00000b40 0a 00 e8 1f 1a 00 e8 1f 20 14 e8 1f 00 00 00 20 |........
...... |
00000b50 00 00 05 00 00 00 00 00 00 00 00 00 00 20 f3 d5
|............. ..|
00000b60 0b 00 e8 1f 1b 00 e8 1f 20 16 e8 1f 00 00 00 20 |........
...... |
00000b70 00 00 05 00 00 00 00 00 00 00 00 00 00 20 28 75
|............. (u|
00000b80 0c 00 e8 1f 1c 00 e8 1f 20 18 e8 1f 00 00 00 20 |........
...... |
00000b90 00 00 05 00 00 00 00 00 00 00 00 00 00 20 2c d7
|............. ,.|
00000ba0 0d 00 e8 1f 1d 00 e8 1f 20 1a e8 1f 00 00 00 20 |........
...... |
00000bb0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 f7 77
|............. .w|
00000bc0 0e 00 e8 1f 1e 00 e8 1f 20 1c e8 1f 00 00 00 20 |........
...... |
00000bd0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 99 d6
|............. ..|
00000be0 0f 00 e8 1f 1f 00 e8 1f 20 1e e8 1f 00 00 00 20 |........
...... |
00000bf0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 42 76
|............. Bv|
00000c00 00 00 f0 1f 10 00 f0 1f 20 00 f0 1f 00 00 00 20 |........
...... |
00000c10 00 00 05 00 00 00 00 00 00 00 00 00 00 20 a8 47
|............. .G|
00000c20 01 00 f0 1f 11 00 f0 1f 20 02 f0 1f 00 00 00 20 |........
...... |
00000c30 00 00 05 00 00 00 00 00 00 00 00 00 00 20 73 e7
|............. s.|
00000c40 02 00 f0 1f 12 00 f0 1f 20 04 f0 1f 00 00 00 20 |........
...... |
00000c50 00 00 05 00 00 00 00 00 00 00 00 00 00 20 1d 46
|............. .F|
00000c60 03 00 f0 1f 13 00 f0 1f 20 06 f0 1f 00 00 00 20 |........
...... |
00000c70 00 00 05 00 00 00 00 00 00 00 00 00 00 20 c6 e6
|............. ..|
00000c80 04 00 f0 1f 14 00 f0 1f 20 08 f0 1f 00 00 00 20 |........
...... |
00000c90 00 00 05 00 00 00 00 00 00 00 00 00 00 20 c2 44
|............. .D|
00000ca0 05 00 f0 1f 15 00 f0 1f 20 0a f0 1f 00 00 00 20 |........
...... |
00000cb0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 19 e4
|............. ..|
00000cc0 06 00 f0 1f 16 00 f0 1f 20 0c f0 1f 00 00 00 20 |........
...... |
00000cd0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 77 45
|............. wE|
00000ce0 07 00 f0 1f 17 00 f0 1f 20 0e f0 1f 00 00 00 20 |........
...... |
00000cf0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 ac e5
|............. ..|
00000d00 08 00 f0 1f 18 00 f0 1f 20 10 f0 1f 00 00 00 20 |........
...... |
00000d10 00 00 05 00 00 00 00 00 00 00 00 00 00 20 7c 41
|............. |A|
00000d20 09 00 f0 1f 19 00 f0 1f 20 12 f0 1f 00 00 00 20 |........
...... |
00000d30 00 00 05 00 00 00 00 00 00 00 00 00 00 20 a7 e1
|............. ..|
00000d40 0a 00 f0 1f 1a 00 f0 1f 20 14 f0 1f 00 00 00 20 |........
...... |
00000d50 00 00 05 00 00 00 00 00 00 00 00 00 00 20 c9 40
|............. .@|
00000d60 0b 00 f0 1f 1b 00 f0 1f 20 16 f0 1f 00 00 00 20 |........
...... |
00000d70 00 00 05 00 00 00 00 00 00 00 00 00 00 20 12 e0
|............. ..|
00000d80 0c 00 f0 1f 1c 00 f0 1f 20 18 f0 1f 00 00 00 20 |........
...... |
00000d90 00 00 05 00 00 00 00 00 00 00 00 00 00 20 16 42
|............. .B|
00000da0 0d 00 f0 1f 1d 00 f0 1f 20 1a f0 1f 00 00 00 20 |........
...... |
00000db0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 cd e2
|............. ..|
00000dc0 0e 00 f0 1f 1e 00 f0 1f 20 1c f0 1f 00 00 00 20 |........
...... |
00000dd0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 a3 43
|............. .C|
00000de0 0f 00 f0 1f 1f 00 f0 1f 20 1e f0 1f 00 00 00 20 |........
...... |
00000df0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 78 e3
|............. x.|
00000e00 00 00 f8 1f 10 00 f8 1f 20 00 f8 1f 00 00 00 20 |........
...... |
00000e10 00 00 05 00 00 00 00 00 00 00 00 00 00 20 be 34
|............. .4|
00000e20 01 00 f8 1f 11 00 f8 1f 20 02 f8 1f 00 00 00 20 |........
...... |
00000e30 00 00 05 00 00 00 00 00 00 00 00 00 00 20 65 94
|............. e.|
00000e40 02 00 f8 1f 12 00 f8 1f 20 04 f8 1f 00 00 00 20 |........
...... |
00000e50 00 00 05 00 00 00 00 00 00 00 00 00 00 20 0b 35
|............. .5|
00000e60 03 00 f8 1f 13 00 f8 1f 20 06 f8 1f 00 00 00 20 |........
...... |
00000e70 00 00 05 00 00 00 00 00 00 00 00 00 00 20 d0 95
|............. ..|
00000e80 04 00 f8 1f 14 00 f8 1f 20 08 f8 1f 00 00 00 20 |........
...... |
00000e90 00 00 05 00 00 00 00 00 00 00 00 00 00 20 d4 37
|............. .7|
00000ea0 05 00 f8 1f 15 00 f8 1f 20 0a f8 1f 00 00 00 20 |........
...... |
00000eb0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 0f 97
|............. ..|
00000ec0 06 00 f8 1f 16 00 f8 1f 20 0c f8 1f 00 00 00 20 |........
...... |
00000ed0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 61 36
|............. a6|
00000ee0 07 00 f8 1f 17 00 f8 1f 20 0e f8 1f 00 00 00 20 |........
...... |
00000ef0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 ba 96
|............. ..|
00000f00 08 00 f8 1f 18 00 f8 1f 20 10 f8 1f 00 00 00 20 |........
...... |
00000f10 00 00 05 00 00 00 00 00 00 00 00 00 00 20 6a 32
|............. j2|
00000f20 09 00 f8 1f 19 00 f8 1f 20 12 f8 1f 00 00 00 20 |........
...... |
00000f30 00 00 05 00 00 00 00 00 00 00 00 00 00 20 b1 92
|............. ..|
00000f40 0a 00 f8 1f 1a 00 f8 1f 20 14 f8 1f 00 00 00 20 |........
...... |
00000f50 00 00 05 00 00 00 00 00 00 00 00 00 00 20 df 33
|............. .3|
00000f60 0b 00 f8 1f 1b 00 f8 1f 20 16 f8 1f 00 00 00 20 |........
...... |
00000f70 00 00 05 00 00 00 00 00 00 00 00 00 00 20 04 93
|............. ..|
00000f80 0c 00 f8 1f 1c 00 f8 1f 20 18 f8 1f 00 00 00 20 |........
...... |
00000f90 00 00 05 00 00 00 00 00 00 00 00 00 00 20 00 31
|............. .1|
00000fa0 0d 00 f8 1f 1d 00 f8 1f 20 1a f8 1f 00 00 00 20 |........
...... |
00000fb0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 db 91
|............. ..|
00000fc0 0e 00 f8 1f 1e 00 f8 1f 20 1c f8 1f 00 00 00 20 |........
...... |
00000fd0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 b5 30
|............. .0|
00000fe0 0f 00 f8 1f 1f 00 f8 1f 20 1e f8 1f 00 00 00 20 |........
...... |
00000ff0 00 00 05 00 00 00 00 00 00 00 00 00 00 20 6e 90
|............. n.|
00001000
/dev/sdf1
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00000400 00 20 d5 15 80 2b 54 57 00 00 00 00 c8 99 09 00 |.
...+TW........|
00000410 5f 91 bc 15 00 00 00 00 02 00 00 00 02 00 00 00
|_...............|
00000420 00 80 00 00 00 80 00 00 00 20 00 00 f8 aa 3f 56 |.........
....?V|
00000430 28 ab 3f 56 06 00 19 00 53 ef 01 00 01 00 00 00
|(.?V....S.......|
00000440 ac 31 3c 56 00 a7 76 00 00 00 00 00 01 00 00 00
|.1<V..v.........|
00000450 00 00 00 00 0b 00 00 00 00 01 00 00 3c 00 00 00
|............<...|
00000460 42 02 00 00 7b 00 00 00 73 6f 67 e3 a3 03 45 44
|B...{...sog...ED|
00000470 9c 0a 6a 32 29 00 5e 39 00 00 00 00 00 00 00 00
|..j2).^9........|
00000480 00 00 00 00 00 00 00 00 2f 73 72 76 00 18 fd e7
|......../srv....|
00000490 1e 01 88 ff ff 5b 29 7f 71 00 00 00 00 00 f9 fe
|.....[).q.......|
000004a0 3a 01 88 ff ff 00 f9 fe 3a 01 88 ff ff 80 65 4d
|:.......:.....eM|
000004b0 3d 01 88 ff ff 00 75 40 19 01 88 ff ff 10 ab 0b
|=.....u@........|
000004c0 a0 ff ff ff ff b8 44 5f 00 00 00 00 00 00 a2 02
|......D_........|
000004d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000004e0 08 00 00 00 00 00 00 00 00 00 00 00 72 a4 a9 9f
|............r...|
000004f0 f9 41 49 5d 88 a2 b1 6a c5 02 4d 5b 01 01 00 00
|.AI]...j..M[....|
00000500 00 00 00 00 00 00 00 00 d4 2c 8e 4c 0a f3 02 00
|.........,.L....|
00000510 04 00 00 00 00 00 00 00 00 00 00 00 ff 7f 00 00
|................|
00000520 00 80 88 0e ff 7f 00 00 01 00 00 00 ff ff 88 0e
|................|
00000530 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000540 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08
|................|
00000550 00 00 00 00 00 00 00 00 00 00 00 00 1c 00 1c 00
|................|
00000560 02 00 00 00 ef 7f 00 00 00 00 00 00 00 00 00 00
|................|
00000570 00 00 00 00 04 00 00 00 01 d3 b5 ba 01 00 00 00
|................|
00000580 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00001000
/dev/sdg1
00000000 70 81 00 00 70 81 01 00 70 81 02 00 70 81 03 00
|p...p...p...p...|
00000010 70 81 04 00 70 81 0c 00 70 81 0d 00 70 81 18 00
|p...p...p...p...|
00000020 70 81 28 00 70 81 3e 00 70 81 79 00 70 81 ab 00
|p.(.p.>.p.y.p...|
00000030 70 81 38 01 70 81 6c 01 70 81 45 04 70 81 b0 04
|p.8.p.l.p.E.p...|
00000040 70 81 1a 06 70 81 d0 0c 70 81 84 1e 70 81 d3 20
|p...p...p...p.. |
00000050 70 81 71 26 00 00 00 00 00 00 00 00 00 00 00 00
|p.q&............|
00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00001000
/dev/sdh1
00000000 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000020 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000040 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000060 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000080 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000000a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000000b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000000c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000000e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000100 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000120 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000140 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000160 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000180 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000001a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000001c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000001e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000200 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000210 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000220 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000240 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000260 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000270 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000280 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000290 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000002a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000002b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000002c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000002d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000002e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000002f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000300 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000310 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000320 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000330 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000340 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000350 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000360 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000370 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000380 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000390 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000003a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000003b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000003c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000003d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000003e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000003f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000400 00 20 d5 35 80 2b 54 77 00 00 00 20 c8 99 09 00 |. .5.+Tw...
....|
00000410 5f 91 bc 15 00 00 00 00 02 00 00 00 02 00 ef 1a
|_...............|
00000420 00 80 00 20 00 80 00 20 00 20 00 20 f8 aa 3f 56 |... ... . .
..?V|
00000430 28 ab 3f 56 06 00 19 00 53 ef 01 00 01 00 ef 1a
|(.?V....S.......|
00000440 ac 31 3c 76 00 a7 76 20 00 00 00 20 01 00 00 00 |.1<v..v ...
....|
00000450 00 00 00 00 0b 00 00 00 00 01 00 00 3c 00 ef 1a
|............<...|
00000460 42 02 00 20 7b 00 00 20 73 6f 67 c3 a3 03 45 44 |B.. {..
sog...ED|
00000470 9c 0a 6a 32 29 00 5e 39 00 00 00 00 00 00 ef 1a
|..j2).^9........|
00000480 00 00 00 20 00 00 00 20 2f 73 72 56 00 18 fd e7 |... ...
/srV....|
00000490 1e 01 88 ff ff 5b 29 7f 71 00 00 00 00 00 16 e4
|.....[).q.......|
000004a0 3a 01 88 df ff 00 f9 de 3a 01 88 df ff 80 65 4d
|:.......:.....eM|
000004b0 3d 01 88 ff ff 00 75 40 19 01 88 ff ff 10 44 11
|=.....u@......D.|
000004c0 a0 ff ff df ff b8 44 7f 00 00 00 20 00 00 a2 02 |......D....
....|
000004d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000004e0 08 00 00 20 00 00 00 20 00 00 00 20 72 a4 a9 9f |... ... ...
r...|
000004f0 f9 41 49 5d 88 a2 b1 6a c5 02 4d 5b 01 01 ef 1a
|.AI]...j..M[....|
00000500 00 00 00 20 00 00 00 20 d4 2c 8e 6c 0a f3 02 00 |... ...
.,.l....|
00000510 04 00 00 00 00 00 00 00 00 00 00 00 ff 7f ef 1a
|................|
00000520 00 80 88 2e ff 7f 00 20 01 00 00 20 ff ff 88 0e |....... ...
....|
00000530 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000540 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 08 |... ... ...
....|
00000550 00 00 00 00 00 00 00 00 00 00 00 00 1c 00 f3 1a
|................|
00000560 02 00 00 20 ef 7f 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000570 00 00 00 00 04 00 00 00 01 d3 b5 ba 01 00 ef 1a
|................|
00000580 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000590 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000005a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000005b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000005c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000005d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000005e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000005f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000600 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000610 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000620 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000630 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000640 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000650 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000660 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000670 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000680 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000690 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000006a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000006b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000006c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000006d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000006e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000006f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000700 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000710 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000720 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000730 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000740 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000750 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000760 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000770 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000780 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000790 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000007a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000007b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000007c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000007d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000007e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000007f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000800 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000810 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000820 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000830 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000840 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000850 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000860 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000870 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000880 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000890 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000008a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000008b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000008c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000008d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000008e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000008f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000900 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000910 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000920 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000930 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000940 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000950 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000960 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000970 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000980 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000990 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000009a0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000009b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000009c0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000009d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
000009e0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
000009f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000a00 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000a10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000a20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000a30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000a40 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000a50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000a60 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000a70 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000a80 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000a90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000aa0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000ab0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000ac0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000ad0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000ae0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000af0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000b00 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000b10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000b20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000b30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000b40 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000b50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000b60 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000b70 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000b80 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000b90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000ba0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000bb0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000bc0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000bd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000be0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000bf0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000c00 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000c10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000c20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000c30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000c40 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000c50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000c60 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000c70 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000c80 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000c90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000ca0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000cb0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000cc0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000cd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000ce0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000cf0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000d00 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000d10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000d20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000d30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000d40 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000d50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000d60 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000d70 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000d80 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000d90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000da0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000db0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000dc0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000dd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000de0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000df0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000e00 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000e10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000e20 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000e30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000e40 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000e50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000e60 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000e70 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000e80 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000e90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000ea0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000eb0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000ec0 00 00 00 20 00 00 00 20 00 00 00 20 eb 07 00 00 |... ... ...
....|
00000ed0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 18 fb
|................|
00000ee0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000ef0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000f00 00 00 00 20 00 00 00 20 00 00 00 20 ab 04 00 00 |... ... ...
....|
00000f10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fd df
|................|
00000f20 00 00 00 20 00 00 00 20 00 00 00 20 ff 07 00 00 |... ... ...
....|
00000f30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 5d 0f
|..............].|
00000f40 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000f50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000f60 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000f70 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000f80 00 00 00 20 00 00 00 20 00 00 00 20 f5 00 00 00 |... ... ...
....|
00000f90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 cc e4
|................|
00000fa0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000fb0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000fc0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000fd0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00000fe0 00 00 00 20 00 00 00 20 00 00 00 20 00 00 00 00 |... ... ...
....|
00000ff0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ef 1a
|................|
00001000
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 23:59 ` Phil Turmel
[not found] ` <b0cdddd4394bbc1356980bb61ac199c3@pingofdeath.de>
@ 2015-11-11 1:03 ` Phil Turmel
2015-11-11 1:29 ` Mathias Mueller
1 sibling, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2015-11-11 1:03 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
On 11/10/2015 06:59 PM, Phil Turmel wrote:
>>> For now, use data offset 2048 for all of them.
>>
>> output of all eight combinations:
>
> Hmmm. Very strange. Let's see the dump of the start of the data area of all four partitions:
>
> for x in /dev/sd[efgh]1 ; do echo -e "\n$x\n" ; dd if=$x skip=2048 count=16 2>/dev/null |hexdump -C ; done
I just noticed your response to this didn't make it to the list. It was
108k. If you do it again with count=8 -- that'll be enough for everyone
to understand my other reply.
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
[not found] ` <b0cdddd4394bbc1356980bb61ac199c3@pingofdeath.de>
@ 2015-11-11 1:00 ` Phil Turmel
2015-11-11 17:53 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2015-11-11 1:00 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
On 11/10/2015 07:06 PM, Mathias Mueller wrote:
> Hi Phil
>
>> Hmmm. Very strange. Let's see the dump of the start of the data area
>> of all four partitions:
>>
>> for x in /dev/sd[efgh]1 ; do echo -e "\n$x\n" ; dd if=$x skip=2048
>> count=16 2>/dev/null |hexdump -C ; done
Very good.
sdf1 is definitely first. sdh1 is definitely last. I don't see
anything obviously wrong with the superblock. The handful of fields
I've looked at are appropriate, including the magic number, write
timestamp, and last mount point. (/srv).
So, I fully expected one of those first four combinations to work. I
don't know why they didn't.
The fsck version dates to 2013, so it might be something related to
that. My next step would be to try all of this after booting into a
modern environment, like the rescue cd from sysrescuecd.org. Recheck
serial numbers and device names again, if you do that.
If that fails the same way, I'm at a loss. You might want to point the
ext4 mailing list at this thread's archive and ask for advice. There's
some possibility of trying various backup superblocks if they can be
located -- I'm not confident enough to advise you on that.
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 23:47 ` Mathias Mueller
@ 2015-11-10 23:59 ` Phil Turmel
[not found] ` <b0cdddd4394bbc1356980bb61ac199c3@pingofdeath.de>
2015-11-11 1:03 ` Phil Turmel
0 siblings, 2 replies; 31+ messages in thread
From: Phil Turmel @ 2015-11-10 23:59 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid, linux-raid-owner
On 11/10/2015 06:47 PM, Mathias Mueller wrote:
> Hi Phil
> sorry, here you go:
>
> sdb -> sde Serial Number: JK1170YBHYV6MD
> sdc -> sdf Serial Number: JK1100YAG64A1T
> sdd -> sdg Serial Number: JK1121YAG7YDLS
> sde -> sdh Serial Number: ML0220F30PZUVD
>
>
>> We know the possible order combinations:
>> sdc1 sdb1 sdd1 sde1
>> sdc1 sdd1 sdb1 sde1
>> sde1 sdb1 sdd1 sdc1
>> sde1 sdd1 sdb1 sdc1
>
> so I used this combinations:
>
> sdf1 sde1 sdg1 sdh1
> sdf1 sdg1 sde1 sdh1
> sdh1 sde1 sdg1 sdf1
> sdh1 sdg1 sde1 sdf1
>
>
>> For now, use data offset 2048 for all of them.
>
> output of all eight combinations:
Hmmm. Very strange. Let's see the dump of the start of the data area of all four partitions:
for x in /dev/sd[efgh]1 ; do echo -e "\n$x\n" ; dd if=$x skip=2048 count=16 2>/dev/null |hexdump -C ; done
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 21:41 ` Phil Turmel
@ 2015-11-10 23:47 ` Mathias Mueller
2015-11-10 23:59 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2015-11-10 23:47 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid, linux-raid-owner
Hi Phil
> You're not providing enough information. Show the new vs. old
> name/serial combinations, then show the --create and fsck
> command/output, one by one.
sorry, here you go:
sdb -> sde Serial Number: JK1170YBHYV6MD
sdc -> sdf Serial Number: JK1100YAG64A1T
sdd -> sdg Serial Number: JK1121YAG7YDLS
sde -> sdh Serial Number: ML0220F30PZUVD
> We know the possible order combinations:
> sdc1 sdb1 sdd1 sde1
> sdc1 sdd1 sdb1 sde1
> sde1 sdb1 sdd1 sdc1
> sde1 sdd1 sdb1 sdc1
so I used this combinations:
sdf1 sde1 sdg1 sdh1
sdf1 sdg1 sde1 sdh1
sdh1 sde1 sdg1 sdf1
sdh1 sdg1 sde1 sdf1
> For now, use data offset 2048 for all of them.
output of all eight combinations:
mdadm --create --assume-clean -l5 -n4 --chunk=64 --data-offset=2048
--metadata=1.2 /dev/md0 /dev/sdf1 /dev/sde1 /dev/sdg1 /dev/sdh1
mdadm: /dev/sdf1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Tue Nov 10 22:24:54 2015
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Tue Nov 10 22:24:54 2015
mdadm: /dev/sdg1 appears to contain an ext2fs file system
size=1953512000K mtime=Wed May 11 17:58:03 2011
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Tue Nov 10 22:24:54 2015
mdadm: /dev/sdh1 appears to contain an ext2fs file system
size=1953513560K mtime=Mon Apr 8 11:04:10 2013
mdadm: /dev/sdh1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Tue Nov 10 22:24:54 2015
Continue creating array? y
mdadm: array /dev/md0 started.
fsck -n /dev/md0
fsck from util-linux 2.23.2
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>
=======================
mdadm --create --assume-clean -l5 -n4 --chunk=512 --data-offset=2048
--metadata=1.2 /dev/md0 /dev/sdf1 /dev/sde1 /dev/sdg1 /dev/sdh1
mdadm: /dev/sdf1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:29:57 2015
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:29:57 2015
mdadm: /dev/sdg1 appears to contain an ext2fs file system
size=1953512000K mtime=Wed May 11 17:58:03 2011
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:29:57 2015
mdadm: /dev/sdh1 appears to contain an ext2fs file system
size=1953513560K mtime=Mon Apr 8 11:04:10 2013
mdadm: /dev/sdh1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:29:57 2015
Continue creating array? y
mdadm: array /dev/md0 started.
fsck -n /dev/md0
fsck from util-linux 2.23.2
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>
================================
================================
mdadm --create --assume-clean -l5 -n4 --chunk=64 --data-offset=2048
--metadata=1.2 /dev/md0 /dev/sdf1 /dev/sdg1 /dev/sde1 /dev/sdh1
mdadm: /dev/sdf1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:31:56 2015
mdadm: /dev/sdg1 appears to contain an ext2fs file system
size=1953512000K mtime=Wed May 11 17:58:03 2011
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:31:56 2015
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:31:56 2015
mdadm: /dev/sdh1 appears to contain an ext2fs file system
size=1953513560K mtime=Mon Apr 8 11:04:10 2013
mdadm: /dev/sdh1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:31:56 2015
Continue creating array? y
mdadm: array /dev/md0 started.
fsck -n /dev/md0
fsck from util-linux 2.23.2
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>
=====================
mdadm --create --assume-clean -l5 -n4 --chunk=512 --data-offset=2048
--metadata=1.2 /dev/md0 /dev/sdf1 /dev/sdg1 /dev/sde1 /dev/sdh1
mdadm: /dev/sdf1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:34:05 2015
mdadm: /dev/sdg1 appears to contain an ext2fs file system
size=1953512000K mtime=Wed May 11 17:58:03 2011
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:34:05 2015
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:34:05 2015
mdadm: /dev/sdh1 appears to contain an ext2fs file system
size=1953513560K mtime=Mon Apr 8 11:04:10 2013
mdadm: /dev/sdh1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:34:05 2015
Continue creating array? y
mdadm: array /dev/md0 started.
fsck -n /dev/md0
fsck from util-linux 2.23.2
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>
=====================
=====================
mdadm --create --assume-clean -l5 -n4 --chunk=64 --data-offset=2048
--metadata=1.2 /dev/md0 /dev/sdh1 /dev/sde1 /dev/sdg1 /dev/sdf1
mdadm: /dev/sdh1 appears to contain an ext2fs file system
size=1953513560K mtime=Mon Apr 8 11:04:10 2013
mdadm: /dev/sdh1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:37:01 2015
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:37:01 2015
mdadm: /dev/sdg1 appears to contain an ext2fs file system
size=1953512000K mtime=Wed May 11 17:58:03 2011
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:37:01 2015
mdadm: /dev/sdf1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:37:01 2015
Continue creating array? y
mdadm: array /dev/md0 started.
fsck -n /dev/md0
fsck from util-linux 2.23.2
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>
=====================
mdadm --create --assume-clean -l5 -n4 --chunk=512 --data-offset=2048
--metadata=1.2 /dev/md0 /dev/sdh1 /dev/sde1 /dev/sdg1 /dev/sdf1
mdadm: /dev/sdh1 appears to contain an ext2fs file system
size=1953513560K mtime=Mon Apr 8 11:04:10 2013
mdadm: /dev/sdh1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:34:59 2015
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:34:59 2015
mdadm: /dev/sdg1 appears to contain an ext2fs file system
size=1953512000K mtime=Wed May 11 17:58:03 2011
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:34:59 2015
mdadm: /dev/sdf1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:34:59 2015
Continue creating array? y
mdadm: array /dev/md0 started.
fsck -n /dev/md0
fsck from util-linux 2.23.2
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>
=====================
=====================
mdadm --create --assume-clean -l5 -n4 --chunk=64 --data-offset=2048
--metadata=1.2 /dev/md0 /dev/sdh1 /dev/sdg1 /dev/sde1 /dev/sdf1
mdadm: /dev/sdh1 appears to contain an ext2fs file system
size=1953513560K mtime=Mon Apr 8 11:04:10 2013
mdadm: /dev/sdh1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:38:17 2015
mdadm: /dev/sdg1 appears to contain an ext2fs file system
size=1953512000K mtime=Wed May 11 17:58:03 2011
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:38:17 2015
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:38:17 2015
mdadm: /dev/sdf1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:38:17 2015
Continue creating array? y
mdadm: array /dev/md0 started.
fsck -n /dev/md0
fsck from util-linux 2.23.2
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>
=====================
mdadm --create --assume-clean -l5 -n4 --chunk=512 --data-offset=2048
--metadata=1.2 /dev/md0 /dev/sdh1 /dev/sdg1 /dev/sde1 /dev/sdf1
mdadm: /dev/sdh1 appears to contain an ext2fs file system
size=1953513560K mtime=Mon Apr 8 11:04:10 2013
mdadm: /dev/sdh1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:39:16 2015
mdadm: /dev/sdg1 appears to contain an ext2fs file system
size=1953512000K mtime=Wed May 11 17:58:03 2011
mdadm: /dev/sdg1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:39:16 2015
mdadm: /dev/sde1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sde1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:39:16 2015
mdadm: /dev/sdf1 appears to contain an ext2fs file system
size=1953513560K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdf1 appears to be part of a raid array:
level=raid5 devices=4 ctime=Wed Nov 11 00:39:16 2015
Continue creating array? y
mdadm: array /dev/md0 started.
fsck -n /dev/md0
fsck from util-linux 2.23.2
e2fsck 1.42.9 (28-Dec-2013)
ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>
Thanks and best regards!
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
2015-11-10 21:33 Mathias Mueller
@ 2015-11-10 21:41 ` Phil Turmel
2015-11-10 23:47 ` Mathias Mueller
0 siblings, 1 reply; 31+ messages in thread
From: Phil Turmel @ 2015-11-10 21:41 UTC (permalink / raw)
To: Mathias Mueller; +Cc: Linux raid
On 11/10/2015 04:33 PM, Mathias Mueller wrote:
> Hi Phil,
>
>>> This both combinations
>>
>> Well, I gave you *four* combinations for order, and two suggestions for
>> chunk size. Eight combinations to try.
>
> Sorry, this two words were just the beginning of a sentence, which I
> didn't finish an forgot to remove :)
>
> I tried 16 combinations at all (eight with --data-offset=1024 and eight
> with --data-offset=2048), the following four combinations gave the
> "/dev/md0 has unsupported feature(s)" message occurs:
>
> --data-offset=1024 --chunk 64: sde sdd sdb sdc
> --data-offset=1024 --chunk 512: sde sdd sdb sdc
> --data-offset=1024 --chunk 64: sde sdb sdd sdc
> --data-offset=1024 --chunk 512: sde sdb sdd sdc
>
> the other four combinations with --data-offset=1024 and all eight
> combinations with --data-offset=2048 gave
>
> fsck.ext2: Superblock invalid, trying backup superblocks...
> fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
You're not providing enough information. Show the new vs. old
name/serial combinations, then show the --create and fsck
command/output, one by one.
For now, use data offset 2048 for all of them.
Phil
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: broken raid level 5 array caused by user error
@ 2015-11-10 21:33 Mathias Mueller
2015-11-10 21:41 ` Phil Turmel
0 siblings, 1 reply; 31+ messages in thread
From: Mathias Mueller @ 2015-11-10 21:33 UTC (permalink / raw)
To: Phil Turmel; +Cc: Linux raid
Hi Phil,
>> This both combinations
>
> Well, I gave you *four* combinations for order, and two suggestions for
> chunk size. Eight combinations to try.
Sorry, this two words were just the beginning of a sentence, which I
didn't finish an forgot to remove :)
I tried 16 combinations at all (eight with --data-offset=1024 and eight
with --data-offset=2048), the following four combinations gave the
"/dev/md0 has unsupported feature(s)" message occurs:
--data-offset=1024 --chunk 64: sde sdd sdb sdc
--data-offset=1024 --chunk 512: sde sdd sdb sdc
--data-offset=1024 --chunk 64: sde sdb sdd sdc
--data-offset=1024 --chunk 512: sde sdb sdd sdc
the other four combinations with --data-offset=1024 and all eight
combinations with --data-offset=2048 gave
fsck.ext2: Superblock invalid, trying backup superblocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/md0
>> /dev/md0 has unsupported feature(s): FEATURE_C16 FEATURE_C17
>> FEATURE_C18
>> FEATURE_C19 FEATURE_C21 FEATURE_C22 FEATURE_C23 FEATURE_C25
>> FEATURE_C27
>> FEATURE_C28 FEATURE_I29 FEATURE_R29
>> e2fsck: Get a newer version of e2fsck!
>
> None of the feature bits are documented to exist. The choice of first
> drive must be wrong.
oh that's good to know
>> Also mdadm --examine on on of the drives tells "Data offset: 2048
>> Sectors" when mdadm --create --data-offset 1024 is used. Is this
>> normal?
>> It's confusing me (using --data-offset 2048 on creation gives "Data
>> offset: 2048 Sectors").
>
> mdadm must be enforcing a minimum offset.
is it possible that the data-offset differs from device to device?
Thanks
Mathias
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2016-01-25 10:02 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-09 11:27 broken raid level 5 array caused by user error Mathias Mueller
2015-11-09 11:56 ` Mikael Abrahamsson
2015-11-09 13:50 ` Phil Turmel
[not found] ` <07de4cd96f39ecb6154794d072ca12e7@pingofdeath.de>
[not found] ` <5640B8AD.3030800@turmel.org>
2015-11-09 15:41 ` Mathias Mueller
[not found] ` <d764bf541381927fa4183c9266fb3f5a@pingofdeath.de>
[not found] ` <5640C38B.4060503@turmel.org>
[not found] ` <a3a91665c4b7cdd70dacc7d8815cc365@pingofdeath.de>
2015-11-09 21:13 ` Phil Turmel
2015-11-10 8:37 ` Mathias Mueller
2015-11-10 13:55 ` Phil Turmel
2015-11-10 14:55 ` Mathias Mueller
2015-11-10 15:20 ` Mathias Mueller
2015-11-10 15:28 ` Phil Turmel
2015-11-10 21:02 ` Mathias Mueller
2015-11-10 21:11 ` Phil Turmel
2015-11-10 21:33 Mathias Mueller
2015-11-10 21:41 ` Phil Turmel
2015-11-10 23:47 ` Mathias Mueller
2015-11-10 23:59 ` Phil Turmel
[not found] ` <b0cdddd4394bbc1356980bb61ac199c3@pingofdeath.de>
2015-11-11 1:00 ` Phil Turmel
2015-11-11 17:53 ` Mathias Mueller
2016-01-18 15:33 ` Mathias Mueller
2016-01-18 19:09 ` Phil Turmel
2016-01-19 14:35 ` Mathias Mueller
2016-01-19 17:51 ` Phil Turmel
2016-01-19 19:37 ` Phil Turmel
2016-01-20 9:04 ` Mathias Mueller
2016-01-22 9:30 ` Mathias Mueller
2016-01-22 17:16 ` Phil Turmel
2016-01-22 17:39 ` Mathias Mueller
2016-01-22 19:13 ` Phil Turmel
2016-01-25 10:02 ` Mathias Mueller
2015-11-11 1:03 ` Phil Turmel
2015-11-11 1:29 ` Mathias Mueller
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.