Changing chunk size

Changing chunk size

[Bill Davidsen]

I have determined that a large array was created with an overly-large 
chunk size. Best way to resize?

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Changing chunk size

[Neil Brown]

On Thursday February 15, davidsen@tmr.com wrote:
> I have determined that a large array was created with an overly-large 
> chunk size. Best way to resize?

Dump and restore.

in-place reshapes (such as raid5 + 1 disk => raid6 or
change-chunk-size) are on my list of 'that might be interesting to
implement', but there are plenty of more interesting things.  And it
would be very slow.  It would need to copy some number of stripes to a
backup somewhere, then copy them back in the new layout, so every
block in the array would be written twice.

NeilBrown

Re: Changing chunk size

[Bill Davidsen]

Neil Brown wrote:
> On Thursday February 15, davidsen@tmr.com wrote:
>   
>> I have determined that a large array was created with an overly-large 
>> chunk size. Best way to resize?
>>     
>
> Dump and restore.
>
> in-place reshapes (such as raid5 + 1 disk => raid6 or
> change-chunk-size) are on my list of 'that might be interesting to
> implement', but there are plenty of more interesting things.  And it
> would be very slow.  It would need to copy some number of stripes to a
> backup somewhere, then copy them back in the new layout, so every
> block in the array would be written twice.
I'm sure "slow" is a relative term, compared to backing up TBs of data 
and trying to restore them. Not to mention the lack of inexpensive TB 
size backup media. That's totally unavailable at the moment, I'll live 
with what I have, thanks.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Changing chunk size

[Steve Cousins]

Bill Davidsen wrote:
> 
> I'm sure "slow" is a relative term, compared to backing up TBs of data 
> and trying to restore them. Not to mention the lack of inexpensive TB 
> size backup media. That's totally unavailable at the moment, I'll live 
> with what I have, thanks.

You don't backup your RAID arrays?  Yikes! For certain data this would 
be fine (data that you can recreate easily) but it sounds like this 
isn't the case for you otherwise you'd just wipe the array and recreate 
the data. There are other modes of failure than just the drives 
themselves (file system corruption for instance) so it is wise to do 
backups, even on "redundant" systems.

Good luck,

Steve
-- 
______________________________________________________________________
  Steve Cousins, Ocean Modeling Group    Email: cousins@umit.maine.edu
  Marine Sciences, 452 Aubert Hall       http://rocky.umeoce.maine.edu
  Univ. of Maine, Orono, ME 04469        Phone: (207) 581-4302

Re: Changing chunk size

[Justin Piszcz]

On Fri, 16 Feb 2007, Steve Cousins wrote:

>
>
> Bill Davidsen wrote:
>> 
>> I'm sure "slow" is a relative term, compared to backing up TBs of data and 
>> trying to restore them. Not to mention the lack of inexpensive TB size 
>> backup media. That's totally unavailable at the moment, I'll live with what 
>> I have, thanks.
>
> You don't backup your RAID arrays?  Yikes! For certain data this would be 
> fine (data that you can recreate easily) but it sounds like this isn't the 
> case for you otherwise you'd just wipe the array and recreate the data. There 
> are other modes of failure than just the drives themselves (file system 
> corruption for instance) so it is wise to do backups, even on "redundant" 
> systems.
>
> Good luck,
>
> Steve

I agree here, RAID is no substitute for backups.

Re: Changing chunk size

[Bill Davidsen]

Steve Cousins wrote:
>
>
> Bill Davidsen wrote:
>>
>> I'm sure "slow" is a relative term, compared to backing up TBs of 
>> data and trying to restore them. Not to mention the lack of 
>> inexpensive TB size backup media. That's totally unavailable at the 
>> moment, I'll live with what I have, thanks.
>
> You don't backup your RAID arrays?  Yikes! For certain data this would 
> be fine (data that you can recreate easily) but it sounds like this 
> isn't the case for you otherwise you'd just wipe the array and 
> recreate the data. There are other modes of failure than just the 
> drives themselves (file system corruption for instance) so it is wise 
> to do backups, even on "redundant" systems. 

My personal and business system are backed up. On "projects" those who 
write the budget set the policy and I just make sure I have a risk 
assessment with their acceptance in writing. University is not like real 
life in many cases.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Changing chunk size

[Bill Davidsen]

Justin Piszcz wrote:
>
>
> On Fri, 16 Feb 2007, Steve Cousins wrote:
>
>>
>>
>> Bill Davidsen wrote:
>>>
>>> I'm sure "slow" is a relative term, compared to backing up TBs of 
>>> data and trying to restore them. Not to mention the lack of 
>>> inexpensive TB size backup media. That's totally unavailable at the 
>>> moment, I'll live with what I have, thanks.
>>
>> You don't backup your RAID arrays?  Yikes! For certain data this 
>> would be fine (data that you can recreate easily) but it sounds like 
>> this isn't the case for you otherwise you'd just wipe the array and 
>> recreate the data. There are other modes of failure than just the 
>> drives themselves (file system corruption for instance) so it is wise 
>> to do backups, even on "redundant" systems.
>>
>> Good luck,
>>
>> Steve
>
> I agree here, RAID is no substitute for backups.
>
Neither is a 2nd copy on another machine, if it isn't something you can 
store off site, it's not a backup. One of the few things I liked about 
working for {a major telco} was that they had a "smoking hole" recovery 
policy, what do you do when the data center is a smoking hole. I don't 
get that kind of budget in other places.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Changing chunk size

[berk walker]

Bill Davidsen wrote:
> Neil Brown wrote:
>> On Thursday February 15, davidsen@tmr.com wrote:
>>  
>>> I have determined that a large array was created with an 
>>> overly-large chunk size. Best way to resize?
>>>     
>>
>> Dump and restore.
>>
>> in-place reshapes (such as raid5 + 1 disk => raid6 or
>> change-chunk-size) are on my list of 'that might be interesting to
>> implement', but there are plenty of more interesting things.  And it
>> would be very slow.  It would need to copy some number of stripes to a
>> backup somewhere, then copy them back in the new layout, so every
>> block in the array would be written twice.
> I'm sure "slow" is a relative term, compared to backing up TBs of data 
> and trying to restore them. Not to mention the lack of inexpensive TB 
> size backup media. That's totally unavailable at the moment, I'll live 
> with what I have, thanks.


> If you were to be a gambler, Bill - Get 2 disks big enough to store 
> your data, create a RAID5 mising one, and copy over the data.  Re do 
> the original array, and copy back.
Yeah, $0.02 doesn't buy much anymore.  Besides, you'll be needing bigger 
disks for the next machine you build,eh?

b-

Re: Changing chunk size

[Bill Davidsen]

berk walker wrote:
>
>
> Bill Davidsen wrote:
>> Neil Brown wrote:
>>> On Thursday February 15, davidsen@tmr.com wrote:
>>>  
>>>> I have determined that a large array was created with an 
>>>> overly-large chunk size. Best way to resize?
>>>>     
>>>
>>> Dump and restore.
>>>
>>> in-place reshapes (such as raid5 + 1 disk => raid6 or
>>> change-chunk-size) are on my list of 'that might be interesting to
>>> implement', but there are plenty of more interesting things.  And it
>>> would be very slow.  It would need to copy some number of stripes to a
>>> backup somewhere, then copy them back in the new layout, so every
>>> block in the array would be written twice.
>> I'm sure "slow" is a relative term, compared to backing up TBs of 
>> data and trying to restore them. Not to mention the lack of 
>> inexpensive TB size backup media. That's totally unavailable at the 
>> moment, I'll live with what I have, thanks.
>
>
>> If you were to be a gambler, Bill - Get 2 disks big enough to store 
>> your data, create a RAID5 mising one, and copy over the data.  Re do 
>> the original array, and copy back.
> Yeah, $0.02 doesn't buy much anymore.  Besides, you'll be needing 
> bigger disks for the next machine you build,eh? 

Not unless the project leader say so. He can send me a note on a capital 
expenses approval form ;-)

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979