* Accidentally resized array to 9
@ 2017-09-29 4:23 Eli Ben-Shoshan
2017-09-29 12:38 ` John Stoffel
2017-09-29 12:55 ` Roman Mamedov
0 siblings, 2 replies; 13+ messages in thread
From: Eli Ben-Shoshan @ 2017-09-29 4:23 UTC (permalink / raw)
To: linux-raid
I need to add another disk to my array (/dev/md128) when I accidentally
did an array resize to 9 with the following command:
First I add the disk to the array with the following:
mdadm --manage /dev/md128 --add /dev/sdl
This was a RAID6 with 8 devices. Instead of using --grow with
--raid-devices set to 9, I did the following:
mdadm --grow /dev/md128 --size 9
This happily returned without any errors so I went to go look at
/proc/mdstat and did not see a resize operation going. So I shook my
head and read the output of --grow --help and did the right thing which is:
mdadm --grow /dev/md128 --raid-devices=9
Right after that everything hit the fan. dmesg reported a lot of
filesystem errors. I quickly stopped all processes that were using this
device and unmounted the filesystems. I then, stupidly, decided to
reboot before looking around.
I am now booted and can assemble this array but it seems like there is
no data there. Here is the output of --misc --examine:
ganon raid # cat md128
/dev/md128:
Version : 1.2
Creation Time : Sat Aug 30 22:01:09 2014
Raid Level : raid6
Used Dev Size : unknown
Raid Devices : 9
Total Devices : 9
Persistence : Superblock is persistent
Update Time : Thu Sep 28 19:44:39 2017
State : clean, Not Started
Active Devices : 9
Working Devices : 9
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : ganon:ganon - large raid6 (local to host ganon)
UUID : 2b3f41d5:ac904000:965be496:dd3ae4ae
Events : 84345
Number Major Minor RaidDevice State
0 8 32 0 active sync /dev/sdc
1 8 48 1 active sync /dev/sdd
6 8 128 2 active sync /dev/sdi
3 8 96 3 active sync /dev/sdg
4 8 80 4 active sync /dev/sdf
8 8 160 5 active sync /dev/sdk
7 8 64 6 active sync /dev/sde
9 8 112 7 active sync /dev/sdh
10 8 176 8 active sync /dev/sdl
You will note that the "Used Dev Size" is unknown. The output of --misc
--examine on each disk looks similar to this:
/dev/sdc:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 2b3f41d5:ac904000:965be496:dd3ae4ae
Name : ganon:ganon - large raid6 (local to host ganon)
Creation Time : Sat Aug 30 22:01:09 2014
Raid Level : raid6
Raid Devices : 9
Avail Dev Size : 3906767024 (1862.89 GiB 2000.26 GB)
Array Size : 0
Used Dev Size : 0
Data Offset : 239616 sectors
Super Offset : 8 sectors
Unused Space : before=239528 sectors, after=3906789552 sectors
State : clean
Device UUID : b1bd681a:36849191:b3fdad44:22567d99
Update Time : Thu Sep 28 19:44:39 2017
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : bca7b1d5 - correct
Events : 84345
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' ==
replacing)
I followed directions to create overlays and I tried to re-create the
array with the following:
mdadm --create /dev/md150 --assume-clean --metadata=1.2
--data-offset=117M --level=6 --layout=ls --chunk=512 --raid-devices=9
/dev/mapper/sdc /dev/mapper/sdd /dev/mapper/sdi /dev/mapper/sdg
/dev/mapper/sdf /dev/mapper/sdk /dev/mapper/sde /dev/mapper/sdh
/dev/mapper/sdl
while this creates a /dev/md150, it is basically empty. There should be
an LVM PV label on this disk but pvck returns:
Could not find LVM label on /dev/md150
The output of --misc --examine looks like this with the overlay:
/dev/md150:
Version : 1.2
Creation Time : Fri Sep 29 00:22:11 2017
Raid Level : raid6
Array Size : 13673762816 (13040.32 GiB 14001.93 GB)
Used Dev Size : 1953394688 (1862.90 GiB 2000.28 GB)
Raid Devices : 9
Total Devices : 9
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Fri Sep 29 00:22:11 2017
State : clean
Active Devices : 9
Working Devices : 9
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : ganon:150 (local to host ganon)
UUID : 84098bfe:74c1f70c:958a7d8a:ccb2ef74
Events : 0
Number Major Minor RaidDevice State
0 252 11 0 active sync /dev/dm-11
1 252 9 1 active sync /dev/dm-9
2 252 16 2 active sync /dev/dm-16
3 252 17 3 active sync /dev/dm-17
4 252 10 4 active sync /dev/dm-10
5 252 14 5 active sync /dev/dm-14
6 252 12 6 active sync /dev/dm-12
7 252 13 7 active sync /dev/dm-13
8 252 15 8 active sync /dev/dm-15
What do you think? Am I hosed here? Is there any way I can get my data back?
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 4:23 Accidentally resized array to 9 Eli Ben-Shoshan
@ 2017-09-29 12:38 ` John Stoffel
2017-09-29 14:47 ` Eli Ben-Shoshan
2017-09-29 12:55 ` Roman Mamedov
1 sibling, 1 reply; 13+ messages in thread
From: John Stoffel @ 2017-09-29 12:38 UTC (permalink / raw)
To: Eli Ben-Shoshan; +Cc: linux-raid
>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
Eli> I need to add another disk to my array (/dev/md128) when I accidentally
Eli> did an array resize to 9 with the following command:
Eli> First I add the disk to the array with the following:
Eli> mdadm --manage /dev/md128 --add /dev/sdl
Eli> This was a RAID6 with 8 devices. Instead of using --grow with
Eli> --raid-devices set to 9, I did the following:
Eli> mdadm --grow /dev/md128 --size 9
Eli> This happily returned without any errors so I went to go look at
Eli> /proc/mdstat and did not see a resize operation going. So I shook my
Eli> head and read the output of --grow --help and did the right thing which is:
Eli> mdadm --grow /dev/md128 --raid-devices=9
Eli> Right after that everything hit the fan. dmesg reported a lot of
Eli> filesystem errors. I quickly stopped all processes that were using this
Eli> device and unmounted the filesystems. I then, stupidly, decided to
Eli> reboot before looking around.
I think you *might* be able to fix this with just a simple:
mdadm --grow /dev/md128 --size max
And then try to scan for your LVM configuration, then fsck your volume
on there. I hope you had backups.
And maybe there should be a warning when re-sizing raid array elements
without a --force option if going smaller than the current size?
Eli> I am now booted and can assemble this array but it seems like there is
Eli> no data there. Here is the output of --misc --examine:
Eli> ganon raid # cat md128
Eli> /dev/md128:
Eli> Version : 1.2
Eli> Creation Time : Sat Aug 30 22:01:09 2014
Eli> Raid Level : raid6
Eli> Used Dev Size : unknown
Eli> Raid Devices : 9
Eli> Total Devices : 9
Eli> Persistence : Superblock is persistent
Eli> Update Time : Thu Sep 28 19:44:39 2017
Eli> State : clean, Not Started
Eli> Active Devices : 9
Eli> Working Devices : 9
Eli> Failed Devices : 0
Eli> Spare Devices : 0
Eli> Layout : left-symmetric
Eli> Chunk Size : 512K
Eli> Name : ganon:ganon - large raid6 (local to host ganon)
Eli> UUID : 2b3f41d5:ac904000:965be496:dd3ae4ae
Eli> Events : 84345
Eli> Number Major Minor RaidDevice State
Eli> 0 8 32 0 active sync /dev/sdc
Eli> 1 8 48 1 active sync /dev/sdd
Eli> 6 8 128 2 active sync /dev/sdi
Eli> 3 8 96 3 active sync /dev/sdg
Eli> 4 8 80 4 active sync /dev/sdf
Eli> 8 8 160 5 active sync /dev/sdk
Eli> 7 8 64 6 active sync /dev/sde
Eli> 9 8 112 7 active sync /dev/sdh
Eli> 10 8 176 8 active sync /dev/sdl
Eli> You will note that the "Used Dev Size" is unknown. The output of --misc
Eli> --examine on each disk looks similar to this:
Eli> /dev/sdc:
Eli> Magic : a92b4efc
Eli> Version : 1.2
Eli> Feature Map : 0x0
Eli> Array UUID : 2b3f41d5:ac904000:965be496:dd3ae4ae
Eli> Name : ganon:ganon - large raid6 (local to host ganon)
Eli> Creation Time : Sat Aug 30 22:01:09 2014
Eli> Raid Level : raid6
Eli> Raid Devices : 9
Eli> Avail Dev Size : 3906767024 (1862.89 GiB 2000.26 GB)
Eli> Array Size : 0
Eli> Used Dev Size : 0
Eli> Data Offset : 239616 sectors
Eli> Super Offset : 8 sectors
Eli> Unused Space : before=239528 sectors, after=3906789552 sectors
Eli> State : clean
Eli> Device UUID : b1bd681a:36849191:b3fdad44:22567d99
Eli> Update Time : Thu Sep 28 19:44:39 2017
Eli> Bad Block Log : 512 entries available at offset 72 sectors
Eli> Checksum : bca7b1d5 - correct
Eli> Events : 84345
Eli> Layout : left-symmetric
Eli> Chunk Size : 512K
Eli> Device Role : Active device 0
Eli> Array State : AAAAAAAAA ('A' == active, '.' == missing, 'R' ==
Eli> replacing)
Eli> I followed directions to create overlays and I tried to re-create the
Eli> array with the following:
Eli> mdadm --create /dev/md150 --assume-clean --metadata=1.2
Eli> --data-offset=117M --level=6 --layout=ls --chunk=512 --raid-devices=9
Eli> /dev/mapper/sdc /dev/mapper/sdd /dev/mapper/sdi /dev/mapper/sdg
Eli> /dev/mapper/sdf /dev/mapper/sdk /dev/mapper/sde /dev/mapper/sdh
Eli> /dev/mapper/sdl
Eli> while this creates a /dev/md150, it is basically empty. There should be
Eli> an LVM PV label on this disk but pvck returns:
Eli> Could not find LVM label on /dev/md150
Eli> The output of --misc --examine looks like this with the overlay:
Eli> /dev/md150:
Eli> Version : 1.2
Eli> Creation Time : Fri Sep 29 00:22:11 2017
Eli> Raid Level : raid6
Eli> Array Size : 13673762816 (13040.32 GiB 14001.93 GB)
Eli> Used Dev Size : 1953394688 (1862.90 GiB 2000.28 GB)
Eli> Raid Devices : 9
Eli> Total Devices : 9
Eli> Persistence : Superblock is persistent
Eli> Intent Bitmap : Internal
Eli> Update Time : Fri Sep 29 00:22:11 2017
Eli> State : clean
Eli> Active Devices : 9
Eli> Working Devices : 9
Eli> Failed Devices : 0
Eli> Spare Devices : 0
Eli> Layout : left-symmetric
Eli> Chunk Size : 512K
Eli> Name : ganon:150 (local to host ganon)
Eli> UUID : 84098bfe:74c1f70c:958a7d8a:ccb2ef74
Eli> Events : 0
Eli> Number Major Minor RaidDevice State
Eli> 0 252 11 0 active sync /dev/dm-11
Eli> 1 252 9 1 active sync /dev/dm-9
Eli> 2 252 16 2 active sync /dev/dm-16
Eli> 3 252 17 3 active sync /dev/dm-17
Eli> 4 252 10 4 active sync /dev/dm-10
Eli> 5 252 14 5 active sync /dev/dm-14
Eli> 6 252 12 6 active sync /dev/dm-12
Eli> 7 252 13 7 active sync /dev/dm-13
Eli> 8 252 15 8 active sync /dev/dm-15
Eli> What do you think? Am I hosed here? Is there any way I can get my data back?
Eli> --
Eli> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
Eli> the body of a message to majordomo@vger.kernel.org
Eli> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 4:23 Accidentally resized array to 9 Eli Ben-Shoshan
2017-09-29 12:38 ` John Stoffel
@ 2017-09-29 12:55 ` Roman Mamedov
2017-09-29 14:53 ` Eli Ben-Shoshan
1 sibling, 1 reply; 13+ messages in thread
From: Roman Mamedov @ 2017-09-29 12:55 UTC (permalink / raw)
To: Eli Ben-Shoshan; +Cc: linux-raid, John Stoffel
On Fri, 29 Sep 2017 00:23:28 -0400
Eli Ben-Shoshan <eli@benshoshan.com> wrote:
> This was a RAID6 with 8 devices. Instead of using --grow with
> --raid-devices set to 9, I did the following:
>
> mdadm --grow /dev/md128 --size 9
>
> This happily returned without any errors so I went to go look at
> /proc/mdstat and did not see a resize operation going. So I shook my
> head and read the output of --grow --help and did the right thing which is:
>
> mdadm --grow /dev/md128 --raid-devices=9
The output of the first command is:
# mdadm --grow /dev/md0 --size 9
mdadm: component size of /dev/md0 has been set to 9K
unfreeze
It didn't occur to you that you FIRST need to restore the "component size" back
to what it was previously?...
And yes as John says there should be a confirmation request on reducing the
array size. In fact I couldn't believe there isn't one already, that's why I
went checking. But nope, there are no warnings or confirmation requests
neither for reducing --size, nor --array-size. I might be remembering that
there were some, from LVM, not MD.
--
With respect,
Roman
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 12:38 ` John Stoffel
@ 2017-09-29 14:47 ` Eli Ben-Shoshan
2017-09-29 19:33 ` John Stoffel
0 siblings, 1 reply; 13+ messages in thread
From: Eli Ben-Shoshan @ 2017-09-29 14:47 UTC (permalink / raw)
To: John Stoffel; +Cc: linux-raid
On 09/29/2017 08:38 AM, John Stoffel wrote:
>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>
> Eli> I need to add another disk to my array (/dev/md128) when I accidentally
> Eli> did an array resize to 9 with the following command:
>
> Eli> First I add the disk to the array with the following:
>
> Eli> mdadm --manage /dev/md128 --add /dev/sdl
>
> Eli> This was a RAID6 with 8 devices. Instead of using --grow with
> Eli> --raid-devices set to 9, I did the following:
>
> Eli> mdadm --grow /dev/md128 --size 9
>
> Eli> This happily returned without any errors so I went to go look at
> Eli> /proc/mdstat and did not see a resize operation going. So I shook my
> Eli> head and read the output of --grow --help and did the right thing which is:
>
> Eli> mdadm --grow /dev/md128 --raid-devices=9
>
> Eli> Right after that everything hit the fan. dmesg reported a lot of
> Eli> filesystem errors. I quickly stopped all processes that were using this
> Eli> device and unmounted the filesystems. I then, stupidly, decided to
> Eli> reboot before looking around.
>
>
> I think you *might* be able to fix this with just a simple:
>
> mdadm --grow /dev/md128 --size max
>
> And then try to scan for your LVM configuration, then fsck your volume
> on there. I hope you had backups.
>
> And maybe there should be a warning when re-sizing raid array elements
> without a --force option if going smaller than the current size?
I just tried that and got the following error:
mdadm: Cannot set device size in this type of array
Trying to go further down this path, I also tried to set the size
explicitly with:
mdadm --grow /dev/md150 --size 1953383512
but got:
mdadm: Cannot set device size in this type of array
I am curious if my data is actually still there on disk.
What does the --size with --grow actually do?
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 12:55 ` Roman Mamedov
@ 2017-09-29 14:53 ` Eli Ben-Shoshan
2017-09-29 19:50 ` Roman Mamedov
0 siblings, 1 reply; 13+ messages in thread
From: Eli Ben-Shoshan @ 2017-09-29 14:53 UTC (permalink / raw)
To: Roman Mamedov; +Cc: linux-raid, John Stoffel
On 09/29/2017 08:55 AM, Roman Mamedov wrote:
> On Fri, 29 Sep 2017 00:23:28 -0400
> Eli Ben-Shoshan <eli@benshoshan.com> wrote:
>
>> This was a RAID6 with 8 devices. Instead of using --grow with
>> --raid-devices set to 9, I did the following:
>>
>> mdadm --grow /dev/md128 --size 9
>>
>> This happily returned without any errors so I went to go look at
>> /proc/mdstat and did not see a resize operation going. So I shook my
>> head and read the output of --grow --help and did the right thing which is:
>>
>> mdadm --grow /dev/md128 --raid-devices=9
>
> The output of the first command is:
>
> # mdadm --grow /dev/md0 --size 9
> mdadm: component size of /dev/md0 has been set to 9K
> unfreeze
>
> It didn't occur to you that you FIRST need to restore the "component size" back
> to what it was previously?...
I am not sure that I actually got any response at all from setting
--size. I am running version:
mdadm - v3.4 - 28th January 2016
If I did get output, I totally missed it. I get that this is my fault
for using the wrong commmand and any data loss is totally due to my
stupidity. I am just hoping that there might be a way that I can get the
data back.
>
> And yes as John says there should be a confirmation request on reducing the
> array size. In fact I couldn't believe there isn't one already, that's why I
> went checking. But nope, there are no warnings or confirmation requests
> neither for reducing --size, nor --array-size. I might be remembering that
> there were some, from LVM, not MD.
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 14:47 ` Eli Ben-Shoshan
@ 2017-09-29 19:33 ` John Stoffel
2017-09-29 21:04 ` Eli Ben-Shoshan
0 siblings, 1 reply; 13+ messages in thread
From: John Stoffel @ 2017-09-29 19:33 UTC (permalink / raw)
To: Eli Ben-Shoshan; +Cc: John Stoffel, linux-raid
>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
Eli> On 09/29/2017 08:38 AM, John Stoffel wrote:
>>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>>
Eli> I need to add another disk to my array (/dev/md128) when I accidentally
Eli> did an array resize to 9 with the following command:
>>
Eli> First I add the disk to the array with the following:
>>
Eli> mdadm --manage /dev/md128 --add /dev/sdl
>>
Eli> This was a RAID6 with 8 devices. Instead of using --grow with
Eli> --raid-devices set to 9, I did the following:
>>
Eli> mdadm --grow /dev/md128 --size 9
>>
Eli> This happily returned without any errors so I went to go look at
Eli> /proc/mdstat and did not see a resize operation going. So I shook my
Eli> head and read the output of --grow --help and did the right thing which is:
>>
Eli> mdadm --grow /dev/md128 --raid-devices=9
>>
Eli> Right after that everything hit the fan. dmesg reported a lot of
Eli> filesystem errors. I quickly stopped all processes that were using this
Eli> device and unmounted the filesystems. I then, stupidly, decided to
Eli> reboot before looking around.
>>
>>
>> I think you *might* be able to fix this with just a simple:
>>
>> mdadm --grow /dev/md128 --size max
>>
>> And then try to scan for your LVM configuration, then fsck your volume
>> on there. I hope you had backups.
>>
>> And maybe there should be a warning when re-sizing raid array elements
>> without a --force option if going smaller than the current size?
Eli> I just tried that and got the following error:
Eli> mdadm: Cannot set device size in this type of array
Eli> Trying to go further down this path, I also tried to set the size
Eli> explicitly with:
Eli> mdadm --grow /dev/md150 --size 1953383512
Eli> but got:
Eli> mdadm: Cannot set device size in this type of array
Eli> I am curious if my data is actually still there on disk.
Eli> What does the --size with --grow actually do?
It changes the size of each member of the array. The man page
explains it, though not ... obviously.
Are you still running with the overlays? That would explain why it
can't resize them bigger. But I'm also behind on email today...
John
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 14:53 ` Eli Ben-Shoshan
@ 2017-09-29 19:50 ` Roman Mamedov
2017-09-30 16:21 ` Phil Turmel
0 siblings, 1 reply; 13+ messages in thread
From: Roman Mamedov @ 2017-09-29 19:50 UTC (permalink / raw)
To: Eli Ben-Shoshan; +Cc: linux-raid, John Stoffel
On Fri, 29 Sep 2017 10:53:57 -0400
Eli Ben-Shoshan <eli@benshoshan.com> wrote:
> I am just hoping that there might be a way that I can get the
> data back.
In theory what you did was cut the array size to only use 9 KB of each device,
then reshaped THAT tiny array from 8 to 9 devices, with the rest left
completely untouched.
So you could try removing the "new" disk, then try --create --assume-clean
with old devices only and --raid-devices=8.
But I'm not sure how you would get the device order right.
Ideally what you can hope for, is you would get the bulk of array data intact,
only with the first 9 KB of each device *(8-2), so about the first 54 KB of
data on the md array, corrupted and unusable. It is likely the LVM and
filesystem tools will not recognize anything due to that, so you will need to
use some data recovery software to look for and save the data.
--
With respect,
Roman
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 19:33 ` John Stoffel
@ 2017-09-29 21:04 ` Eli Ben-Shoshan
2017-09-29 21:17 ` John Stoffel
0 siblings, 1 reply; 13+ messages in thread
From: Eli Ben-Shoshan @ 2017-09-29 21:04 UTC (permalink / raw)
To: John Stoffel; +Cc: linux-raid
On 09/29/2017 03:33 PM, John Stoffel wrote:
>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>
> Eli> On 09/29/2017 08:38 AM, John Stoffel wrote:
>>>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>>>
> Eli> I need to add another disk to my array (/dev/md128) when I accidentally
> Eli> did an array resize to 9 with the following command:
>>>
> Eli> First I add the disk to the array with the following:
>>>
> Eli> mdadm --manage /dev/md128 --add /dev/sdl
>>>
> Eli> This was a RAID6 with 8 devices. Instead of using --grow with
> Eli> --raid-devices set to 9, I did the following:
>>>
> Eli> mdadm --grow /dev/md128 --size 9
>>>
> Eli> This happily returned without any errors so I went to go look at
> Eli> /proc/mdstat and did not see a resize operation going. So I shook my
> Eli> head and read the output of --grow --help and did the right thing which is:
>>>
> Eli> mdadm --grow /dev/md128 --raid-devices=9
>>>
> Eli> Right after that everything hit the fan. dmesg reported a lot of
> Eli> filesystem errors. I quickly stopped all processes that were using this
> Eli> device and unmounted the filesystems. I then, stupidly, decided to
> Eli> reboot before looking around.
>>>
>>>
>>> I think you *might* be able to fix this with just a simple:
>>>
>>> mdadm --grow /dev/md128 --size max
>>>
>>> And then try to scan for your LVM configuration, then fsck your volume
>>> on there. I hope you had backups.
>>>
>>> And maybe there should be a warning when re-sizing raid array elements
>>> without a --force option if going smaller than the current size?
>
> Eli> I just tried that and got the following error:
>
> Eli> mdadm: Cannot set device size in this type of array
>
> Eli> Trying to go further down this path, I also tried to set the size
> Eli> explicitly with:
>
> Eli> mdadm --grow /dev/md150 --size 1953383512
>
> Eli> but got:
>
> Eli> mdadm: Cannot set device size in this type of array
>
> Eli> I am curious if my data is actually still there on disk.
>
> Eli> What does the --size with --grow actually do?
>
> It changes the size of each member of the array. The man page
> explains it, though not ... obviously.
>
> Are you still running with the overlays? That would explain why it
> can't resize them bigger. But I'm also behind on email today...
>
> John
>
I was still using the overlay. I just tried the grow without the overlay
and got the same error.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 21:04 ` Eli Ben-Shoshan
@ 2017-09-29 21:17 ` John Stoffel
2017-09-29 21:49 ` Eli Ben-Shoshan
0 siblings, 1 reply; 13+ messages in thread
From: John Stoffel @ 2017-09-29 21:17 UTC (permalink / raw)
To: Eli Ben-Shoshan; +Cc: John Stoffel, linux-raid
>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
Eli> On 09/29/2017 03:33 PM, John Stoffel wrote:
>>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>>
Eli> On 09/29/2017 08:38 AM, John Stoffel wrote:
>>>>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>>>>
Eli> I need to add another disk to my array (/dev/md128) when I accidentally
Eli> did an array resize to 9 with the following command:
>>>>
Eli> First I add the disk to the array with the following:
>>>>
Eli> mdadm --manage /dev/md128 --add /dev/sdl
>>>>
Eli> This was a RAID6 with 8 devices. Instead of using --grow with
Eli> --raid-devices set to 9, I did the following:
>>>>
Eli> mdadm --grow /dev/md128 --size 9
>>>>
Eli> This happily returned without any errors so I went to go look at
Eli> /proc/mdstat and did not see a resize operation going. So I shook my
Eli> head and read the output of --grow --help and did the right thing which is:
>>>>
Eli> mdadm --grow /dev/md128 --raid-devices=9
>>>>
Eli> Right after that everything hit the fan. dmesg reported a lot of
Eli> filesystem errors. I quickly stopped all processes that were using this
Eli> device and unmounted the filesystems. I then, stupidly, decided to
Eli> reboot before looking around.
>>>>
>>>>
>>>> I think you *might* be able to fix this with just a simple:
>>>>
>>>> mdadm --grow /dev/md128 --size max
>>>>
>>>> And then try to scan for your LVM configuration, then fsck your volume
>>>> on there. I hope you had backups.
>>>>
>>>> And maybe there should be a warning when re-sizing raid array elements
>>>> without a --force option if going smaller than the current size?
>>
Eli> I just tried that and got the following error:
>>
Eli> mdadm: Cannot set device size in this type of array
>>
Eli> Trying to go further down this path, I also tried to set the size
Eli> explicitly with:
>>
Eli> mdadm --grow /dev/md150 --size 1953383512
>>
Eli> but got:
>>
Eli> mdadm: Cannot set device size in this type of array
>>
Eli> I am curious if my data is actually still there on disk.
>>
Eli> What does the --size with --grow actually do?
>>
>> It changes the size of each member of the array. The man page
>> explains it, though not ... obviously.
>>
>> Are you still running with the overlays? That would explain why it
>> can't resize them bigger. But I'm also behind on email today...
Eli> I was still using the overlay. I just tried the grow without the overlay
Eli> and got the same error.
Hmm.. what do the partitions on the disk look like now? You might
need to do more digging. But I would say that using --grow and having
it *shrink* without any warnings is a bad idea for the mdadm tools.
It should scream loudly and only run when forced to like that.
Aw crap... you used the whole disk. I don't like doing this because
A) if I get a disk slightly *smaller* than what I currently have, it
will be painful, B) it's easy to use a small partition starting 4mb
from the start and a few hundred Mb (or even a Gb) from the end.
In your case, can you try to do the 'mdadm --grow /dev/md### --size
max' but with a version of mdadm compiled with debugging info, or at
least using the latest version of the code if at all possible.
Grab it from https://github.com/neilbrown/mdadm and when you
configure it, make sure you enable debugging. Or grab it from
https://www.kernel.org/pub/linux/utils/raid/mdadm/ and try the same
thing.
Can you show the output of: cat /proc/partitions as well? Maybe you
need to do:
mdadm --grow <dev> --size ########
which is the smallest of the max size of all your disks. Might
work...
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 21:17 ` John Stoffel
@ 2017-09-29 21:49 ` Eli Ben-Shoshan
0 siblings, 0 replies; 13+ messages in thread
From: Eli Ben-Shoshan @ 2017-09-29 21:49 UTC (permalink / raw)
To: John Stoffel; +Cc: linux-raid
On 09/29/2017 05:17 PM, John Stoffel wrote:
>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>
> Eli> On 09/29/2017 03:33 PM, John Stoffel wrote:
>>>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>>>
> Eli> On 09/29/2017 08:38 AM, John Stoffel wrote:
>>>>>>>>>> "Eli" == Eli Ben-Shoshan <eli@benshoshan.com> writes:
>>>>>
> Eli> I need to add another disk to my array (/dev/md128) when I accidentally
> Eli> did an array resize to 9 with the following command:
>>>>>
> Eli> First I add the disk to the array with the following:
>>>>>
> Eli> mdadm --manage /dev/md128 --add /dev/sdl
>>>>>
> Eli> This was a RAID6 with 8 devices. Instead of using --grow with
> Eli> --raid-devices set to 9, I did the following:
>>>>>
> Eli> mdadm --grow /dev/md128 --size 9
>>>>>
> Eli> This happily returned without any errors so I went to go look at
> Eli> /proc/mdstat and did not see a resize operation going. So I shook my
> Eli> head and read the output of --grow --help and did the right thing which is:
>>>>>
> Eli> mdadm --grow /dev/md128 --raid-devices=9
>>>>>
> Eli> Right after that everything hit the fan. dmesg reported a lot of
> Eli> filesystem errors. I quickly stopped all processes that were using this
> Eli> device and unmounted the filesystems. I then, stupidly, decided to
> Eli> reboot before looking around.
>>>>>
>>>>>
>>>>> I think you *might* be able to fix this with just a simple:
>>>>>
>>>>> mdadm --grow /dev/md128 --size max
>>>>>
>>>>> And then try to scan for your LVM configuration, then fsck your volume
>>>>> on there. I hope you had backups.
>>>>>
>>>>> And maybe there should be a warning when re-sizing raid array elements
>>>>> without a --force option if going smaller than the current size?
>>>
> Eli> I just tried that and got the following error:
>>>
> Eli> mdadm: Cannot set device size in this type of array
>>>
> Eli> Trying to go further down this path, I also tried to set the size
> Eli> explicitly with:
>>>
> Eli> mdadm --grow /dev/md150 --size 1953383512
>>>
> Eli> but got:
>>>
> Eli> mdadm: Cannot set device size in this type of array
>>>
> Eli> I am curious if my data is actually still there on disk.
>>>
> Eli> What does the --size with --grow actually do?
>>>
>>> It changes the size of each member of the array. The man page
>>> explains it, though not ... obviously.
>>>
>>> Are you still running with the overlays? That would explain why it
>>> can't resize them bigger. But I'm also behind on email today...
>
>
> Eli> I was still using the overlay. I just tried the grow without the overlay
> Eli> and got the same error.
>
> Hmm.. what do the partitions on the disk look like now? You might
> need to do more digging. But I would say that using --grow and having
> it *shrink* without any warnings is a bad idea for the mdadm tools.
> It should scream loudly and only run when forced to like that.
>
> Aw crap... you used the whole disk. I don't like doing this because
> A) if I get a disk slightly *smaller* than what I currently have, it
> will be painful, B) it's easy to use a small partition starting 4mb
> from the start and a few hundred Mb (or even a Gb) from the end.
>
> In your case, can you try to do the 'mdadm --grow /dev/md### --size
> max' but with a version of mdadm compiled with debugging info, or at
> least using the latest version of the code if at all possible.
>
> Grab it from https://github.com/neilbrown/mdadm and when you
> configure it, make sure you enable debugging. Or grab it from
> https://www.kernel.org/pub/linux/utils/raid/mdadm/ and try the same
> thing.
>
> Can you show the output of: cat /proc/partitions as well? Maybe you
> need to do:
>
> mdadm --grow <dev> --size ########
>
> which is the smallest of the max size of all your disks. Might
> work...
>
ganon mdadm-4.0 # cat /proc/partitions
major minor #blocks name
1 0 8192 ram0
1 1 8192 ram1
1 2 8192 ram2
1 3 8192 ram3
1 4 8192 ram4
1 5 8192 ram5
1 6 8192 ram6
1 7 8192 ram7
1 8 8192 ram8
1 9 8192 ram9
1 10 8192 ram10
1 11 8192 ram11
1 12 8192 ram12
1 13 8192 ram13
1 14 8192 ram14
1 15 8192 ram15
8 0 234431064 sda
8 1 262144 sda1
8 2 204472320 sda2
8 3 2097152 sda3
8 16 234431064 sdb
8 17 262144 sdb1
8 18 204472320 sdb2
8 19 2097152 sdb3
8 32 1953514584 sdc
8 48 1953514584 sdd
8 64 1953514584 sde
8 80 1953514584 sdf
8 96 1953514584 sdg
9 126 262080 md126
9 127 204341248 md127
252 0 1572864 dm-0
252 1 9437184 dm-1
252 2 4194304 dm-2
252 3 16777216 dm-3
252 4 25165824 dm-4
252 5 7340032 dm-5
252 6 6291456 dm-6
252 7 53477376 dm-7
252 8 1048576 dm-8
8 112 1953514584 sdh
8 128 1953514584 sdi
8 144 1953514584 sdj
8 145 4194304 sdj1
8 146 524288 sdj2
8 147 1948793912 sdj3
8 160 1953514584 sdk
8 176 1953514584 sdl
8 192 1953514584 sdm
8 193 4194304 sdm1
8 194 524288 sdm2
8 195 1948793912 sdm3
9 131 1948662656 md131
9 129 4192192 md129
9 130 523968 md130
I got mdadm-4.0 compile with debug flags. Here is the output starting
with --assemble --scan
ganon mdadm-4.0 # ./mdadm --assemble --scan
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:5/end_device-14:5/target14:0:5/14:0:5:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:4/end_device-14:4/target14:0:4/14:0:4:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:3/end_device-14:3/target14:0:3/14:0:3:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:2/end_device-14:2/target14:0:2/14:0:2:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:1/end_device-14:1/target14:0:1/14:0:1:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:0/end_device-14:0/target14:0:0/14:0:0:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:1c.5/0000:04:00.0/ata13/host12/target12:0:0/12:0:0:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata10/host9/target9:0:0/9:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata9/host8/target8:0:0/8:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata8/host7/target7:0:0/7:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata7/host6/target6:0:0/6:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata6/host5/target5:0:0/5:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata5/host4/target4:0:0/4:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: start_array: /dev/md128 has been started with 9 drives.
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:5/end_device-14:5/target14:0:5/14:0:5:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:4/end_device-14:4/target14:0:4/14:0:4:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:3/end_device-14:3/target14:0:3/14:0:3:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:2/end_device-14:2/target14:0:2/14:0:2:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:1/end_device-14:1/target14:0:1/14:0:1:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:03.0/0000:01:00.0/host14/port-14:0/end_device-14:0/target14:0:0/14:0:0:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk:
/sys/devices/pci0000:00/0000:00:1c.5/0000:04:00.0/ata13/host12/target12:0:0/12:0:0:0
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata10/host9/target9:0:0/9:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata9/host8/target8:0:0/8:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata8/host7/target7:0:0/7:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata7/host6/target6:0:0/6:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata6/host5/target5:0:0/5:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
mdadm: path_attached_to_hba: hba: /sys/devices/pci0000:00/0000:00:1f.2 -
disk: /sys/devices/pci0000:00/0000:00:1f.2/ata5/host4/target4:0:0/4:0:0:0
mdadm: scan: ptr->vendorID: 1103 __le16_to_cpu(ptr->deviceID): 2720
and now an attempt to --grow with --size max:
ganon mdadm-4.0 # ./mdadm --grow /dev/md128 --size max
mdadm: Grow_reshape: Cannot set device size in this type of array.
I am not using overlays with the above commands.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-29 19:50 ` Roman Mamedov
@ 2017-09-30 16:21 ` Phil Turmel
2017-09-30 16:29 ` Roman Mamedov
2017-09-30 23:30 ` John Stoffel
0 siblings, 2 replies; 13+ messages in thread
From: Phil Turmel @ 2017-09-30 16:21 UTC (permalink / raw)
To: Roman Mamedov, Eli Ben-Shoshan; +Cc: linux-raid, John Stoffel
On 09/29/2017 03:50 PM, Roman Mamedov wrote:
> On Fri, 29 Sep 2017 10:53:57 -0400
> Eli Ben-Shoshan <eli@benshoshan.com> wrote:
>
>> I am just hoping that there might be a way that I can get the
>> data back.
>
> In theory what you did was cut the array size to only use 9 KB of each device,
> then reshaped THAT tiny array from 8 to 9 devices, with the rest left
> completely untouched.
>
> So you could try removing the "new" disk, then try --create --assume-clean
> with old devices only and --raid-devices=8.
>
> But I'm not sure how you would get the device order right.
>
> Ideally what you can hope for, is you would get the bulk of array data intact,
> only with the first 9 KB of each device *(8-2), so about the first 54 KB of
> data on the md array, corrupted and unusable. It is likely the LVM and
> filesystem tools will not recognize anything due to that, so you will need to
> use some data recovery software to look for and save the data.
>
I agree with Roman. Most of your array should be still on the 8-disk
layout. But you were mounted and had writing processes immediately
after the broken grow, so there's probably other corruption due to
writes on the 9-disk pattern in the 8 disks.
Roman's suggestion is the best plan, but even after restoring LVM,
expect breakage all over. Use overlays.
Phil
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-30 16:21 ` Phil Turmel
@ 2017-09-30 16:29 ` Roman Mamedov
2017-09-30 23:30 ` John Stoffel
1 sibling, 0 replies; 13+ messages in thread
From: Roman Mamedov @ 2017-09-30 16:29 UTC (permalink / raw)
To: Phil Turmel; +Cc: Eli Ben-Shoshan, linux-raid, John Stoffel
On Sat, 30 Sep 2017 12:21:20 -0400
Phil Turmel <philip@turmel.org> wrote:
> > Ideally what you can hope for, is you would get the bulk of array data intact,
> > only with the first 9 KB of each device *(8-2), so about the first 54 KB of
> > data on the md array, corrupted and unusable. It is likely the LVM and
> > filesystem tools will not recognize anything due to that, so you will need to
> > use some data recovery software to look for and save the data.
> >
>
> I agree with Roman. Most of your array should be still on the 8-disk
> layout. But you were mounted and had writing processes immediately
> after the broken grow, so there's probably other corruption due to
> writes on the 9-disk pattern in the 8 disks.
Adding one afterthought that I had, you could probably salvage the 54 KB in
question by reading them in (and saving to a file) from the current
"9-device array of 9KB devices" that you got.
With respect,
Roman
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Accidentally resized array to 9
2017-09-30 16:21 ` Phil Turmel
2017-09-30 16:29 ` Roman Mamedov
@ 2017-09-30 23:30 ` John Stoffel
1 sibling, 0 replies; 13+ messages in thread
From: John Stoffel @ 2017-09-30 23:30 UTC (permalink / raw)
To: Phil Turmel; +Cc: Roman Mamedov, Eli Ben-Shoshan, linux-raid, John Stoffel
>>>>> "Phil" == Phil Turmel <philip@turmel.org> writes:
Phil> On 09/29/2017 03:50 PM, Roman Mamedov wrote:
>> On Fri, 29 Sep 2017 10:53:57 -0400
>> Eli Ben-Shoshan <eli@benshoshan.com> wrote:
>>
>>> I am just hoping that there might be a way that I can get the
>>> data back.
>>
>> In theory what you did was cut the array size to only use 9 KB of each device,
>> then reshaped THAT tiny array from 8 to 9 devices, with the rest left
>> completely untouched.
>>
>> So you could try removing the "new" disk, then try --create --assume-clean
>> with old devices only and --raid-devices=8.
>>
>> But I'm not sure how you would get the device order right.
>>
>> Ideally what you can hope for, is you would get the bulk of array data intact,
>> only with the first 9 KB of each device *(8-2), so about the first 54 KB of
>> data on the md array, corrupted and unusable. It is likely the LVM and
>> filesystem tools will not recognize anything due to that, so you will need to
>> use some data recovery software to look for and save the data.
>>
Phil> I agree with Roman. Most of your array should be still on the 8-disk
Phil> layout. But you were mounted and had writing processes immediately
Phil> after the broken grow, so there's probably other corruption due to
Phil> writes on the 9-disk pattern in the 8 disks.
Phil> Roman's suggestion is the best plan, but even after restoring LVM,
Phil> expect breakage all over. Use overlays.
Maybe the answer is to remove the added disk, setup an overlay on the
eight remaining disks, and then try to do mdadm --create ... on each
of the permutations. Then you would bring up the LVs on there and see
if you can fsck them and get some data back.
I think the grow isn't going to work, it's really quite hosed at this
point.
If I find some time, I think I'll try to spin up a patch to mdadm to
hopefully stop issues like this from happening, by stopping a --size
to a smaller size with an explicit confirmation being asked, or
overridden by a flag to force the shrink. Since it's so damn painful.
I don't have alot of hope here for you unfortunately. I think you're
now in the stage where a --create using the original eight disks is
the way to go.
You *might* be able to find RAID backups at some offset into the disks
to tell you what order each disk is in. So the steps, roughly, would
be:
1. stop /dev/md127
2. remove the new disk.
3. setup overlays again.
4. mdadm --create /dev/md127 --level 6 -n 8 /dev/sd{cdefghij}
5. vgchange -ay /dev/md127
6. lvs
7. fsck ....
8. if nothing, loop back to step four with a different order of
devices.
If you have any output from before of /proc/mdstat, that would be
helpful, as would a mapping of device name (/dev/sd*) to serial
number.
[ There's a neat script called 'lsdrv' which you can grab here
(https://github.com/pturmel/lsdrv) to grab and show all this data.
But it's busted for lvcache devices. Oops! Time for more hacking! ]
But I hate to say... I suspect you're toast. But don't listen to me.
John
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2017-09-30 23:30 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-29 4:23 Accidentally resized array to 9 Eli Ben-Shoshan
2017-09-29 12:38 ` John Stoffel
2017-09-29 14:47 ` Eli Ben-Shoshan
2017-09-29 19:33 ` John Stoffel
2017-09-29 21:04 ` Eli Ben-Shoshan
2017-09-29 21:17 ` John Stoffel
2017-09-29 21:49 ` Eli Ben-Shoshan
2017-09-29 12:55 ` Roman Mamedov
2017-09-29 14:53 ` Eli Ben-Shoshan
2017-09-29 19:50 ` Roman Mamedov
2017-09-30 16:21 ` Phil Turmel
2017-09-30 16:29 ` Roman Mamedov
2017-09-30 23:30 ` John Stoffel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.