Preparing for mdadm-3.3

Preparing for mdadm-3.3

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2251 bytes --]


Hi all,
 it has been way too long since an mdadm release.  So I'm hoping to pull one
 together over the next few weeks.  It will be mdadm-3.3
 My goal is to have it out by 25th July (i.e. one month).

 To this end I have tagged "mdadm-3.3-rc1" in my git tree and pushed it out
 in the hope that people might test it.
 You can "git pull" or "git clone" from
     git://neil.brown.name/mdadm/

 of if you really like tar files:

http://git.neil.brown.name/git?p=mdadm.git;a=snapshot;h=refs/tags/mdadm-3.3-rc1;sf=tgz

 Now would be a good time to test anything your are interested in, or remind
 me of something you wanted that hasn't been done yet, or send me your mdadm
 patches that you have been meaning to send me but never seem to get around
 to it.

 Some of the new things in mdadm-3.3 are:

- Reshape various arrays without needing backup file.  This is done by
  changing the 'data_offset' so we never need to write any data back over
  where it was before.
- "--assemble --update=metadata" can convert a 0.90 array to a 1.0 array.
- bad-block-logs are supported (but not heavily tested yet)
- "--assemble --update=revert-reshape" can be used to undo a reshape that has
  just been started but isn't really wanted.  This also needs lots of testing.
- improved locking between --incremental and --assemble
- uses systemd to run "mdmon" if systemd is configured to do that.
- kernel names of md devices can be non-numeric. e.g. "md_home" rather than
  "md0".  This will probably confuse lots of other tools, so you need to
      echo CREATE names=yes >> /etc/mdadm.conf
  or the feature will not be used.  (you also need a reasonably new kernel).
- "--stop" can be given a kernel name instead of a device name. i.e
    mdadm --stop md4
  will work even if /dev/md4 doesn't exist.
- "--detail --export" has some information about the devices in the array
- --dump and --retstore can be used to backup and restore the metadata on an
  array.

and lots of bugfixes and other little changes.

I somehow got unsubscribed from linux-raid (again) a few weeks back, so if
anything was sent to the list and not me that I should know about, now might
be a good time to resend that too.

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Preparing for mdadm-3.3

[Mark Knecht]

On Mon, Jun 24, 2013 at 12:18 AM, NeilBrown <neilb@suse.de> wrote:
<SNIP>
>
> I somehow got unsubscribed from linux-raid (again) a few weeks back, so if
> anything was sent to the list and not me that I should know about, now might
> be a good time to resend that too.
>
> Thanks,
> NeilBrown

Neil,
   Welcome back. :-)

   I currently run mdadm-3.1.4 which is Gentoo stable. It seems that I
cannot change chunk size if the new chunk size doesn't divide evenly
into the number of sectors on the partition, or at least that's the
way I understand the message. Is that anything you've either worked on
or considered for mdadm?

Thanks,
Mark

Re: Preparing for mdadm-3.3

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2042 bytes --]

On Mon, 24 Jun 2013 06:44:25 -0700 Mark Knecht <markknecht@gmail.com> wrote:

> On Mon, Jun 24, 2013 at 12:18 AM, NeilBrown <neilb@suse.de> wrote:
> <SNIP>
> >
> > I somehow got unsubscribed from linux-raid (again) a few weeks back, so if
> > anything was sent to the list and not me that I should know about, now might
> > be a good time to resend that too.
> >
> > Thanks,
> > NeilBrown
> 
> Neil,
>    Welcome back. :-)
> 
>    I currently run mdadm-3.1.4 which is Gentoo stable. It seems that I
> cannot change chunk size if the new chunk size doesn't divide evenly
> into the number of sectors on the partition, or at least that's the
> way I understand the message. Is that anything you've either worked on
> or considered for mdadm?
> 

In my mind, it should "obviously" work the way it does.  As you seem to think
differently, at least one of us must be missing something.

Each device (in a striped array) must contribute an integer number of chunks
to the array, otherwise you will get holes in the data.
So if you set the chunk size to something that doesn't divide into the size
of the device, then you will end up using less of the device.  And that means
that they array will become smaller.

Making the array smaller will mostly likely cause you to lose data.

If you really want to do that, you can.  Just set the size to something
appropriate and then change the chunk size.
  mdadm -G /dev/mdXX --size=smaller
  mdadm -G /dev/mdXX --chunk=different

Of course the "something appropriate" must be a multiple of both the old and
the new chunk sizes.  As chunk sizes are mostly powers of 2, that shouldn't be
a problem.

Maybe when mdadm reports that the chunk size doesn't divide the device size,
it could suggest a "--grow --size=" command to reduce the array size so that
it will.

Obviously you would need to resize any file system before resizing the array.

Does that address your issue?  Is there something that you are thinking that
I am missing?

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Preparing for mdadm-3.3

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 973 bytes --]

On Mon, 24 Jun 2013 18:10:23 +0200 Jes Sorensen <Jes.Sorensen@redhat.com>
wrote:

> NeilBrown <neilb@suse.de> writes:
> > Hi all,
> >  it has been way too long since an mdadm release.  So I'm hoping to pull one
> >  together over the next few weeks.  It will be mdadm-3.3
> >  My goal is to have it out by 25th July (i.e. one month).
> 
> Hi Neil,
> 
> I don't see any mention of hot replace support in your log. Did this get
> canned or did I just lose track of everything? Admittedly my marbles
> have gotten lost regularly the last couple of months.
> 

Hi Jes,
 yes, hot replace support is something I should have mentioned but didn't

   mdadm /dev/md0 --replace /dev/foo

 will try to replace /dev/foo with a spare.

   mdadm /dev/md0 --replace /dev/foo --with /dev/bar

 will try to replace it with the spare "/dev/bar".

 Testing of that would be good too.
 I'll be sure to mention that in the 3.3 announcement.  Thanks.

NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Preparing for mdadm-3.3

[Jes Sorensen]

NeilBrown <neilb@suse.de> writes:
> On Mon, 24 Jun 2013 18:10:23 +0200 Jes Sorensen <Jes.Sorensen@redhat.com>
> wrote:
>
>> NeilBrown <neilb@suse.de> writes:
>> > Hi all,
>> >  it has been way too long since an mdadm release.  So I'm hoping to pull one
>> >  together over the next few weeks.  It will be mdadm-3.3
>> >  My goal is to have it out by 25th July (i.e. one month).
>> 
>> Hi Neil,
>> 
>> I don't see any mention of hot replace support in your log. Did this get
>> canned or did I just lose track of everything? Admittedly my marbles
>> have gotten lost regularly the last couple of months.
>> 
>
> Hi Jes,
>  yes, hot replace support is something I should have mentioned but didn't
>
>    mdadm /dev/md0 --replace /dev/foo
>
>  will try to replace /dev/foo with a spare.
>
>    mdadm /dev/md0 --replace /dev/foo --with /dev/bar
>
>  will try to replace it with the spare "/dev/bar".
>
>  Testing of that would be good too.
>  I'll be sure to mention that in the 3.3 announcement.  Thanks.

Great, I'll try and package up mdadm-3.3-rc1 and see if I can get QA on
my end to test it.

Cheers,
Jes

Re: Preparing for mdadm-3.3

[Martin Wilck]

On 06/24/2013 09:18 AM, NeilBrown wrote:

>  it has been way too long since an mdadm release.  So I'm hoping to pull one
>  together over the next few weeks.  It will be mdadm-3.3
>  My goal is to have it out by 25th July (i.e. one month).

I am preparing a relatively big patch set to get full support for
DDF/RAID10. I guess that will need to go in after 3.3, then.

Regards
Martin

Re: Preparing for mdadm-3.3

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 837 bytes --]

On Thu, 27 Jun 2013 20:24:52 +0200 Martin Wilck <mwilck@arcor.de> wrote:

> On 06/24/2013 09:18 AM, NeilBrown wrote:
> 
> >  it has been way too long since an mdadm release.  So I'm hoping to pull one
> >  together over the next few weeks.  It will be mdadm-3.3
> >  My goal is to have it out by 25th July (i.e. one month).
> 
> I am preparing a relatively big patch set to get full support for
> DDF/RAID10. I guess that will need to go in after 3.3, then.

Depends on when it is ready.  I wouldn't in principle object to a large DDF
patch set shortly before the release.  Nor would I hold up the release beyond
31st July just to get it in.
And hopefully there will be a 3.3.1 before the year it out which can
presumably include your DDF work.

So just send what you have whenever you are ready.

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Preparing for mdadm-3.3

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 699 bytes --]

On Mon, 24 Jun 2013 17:18:18 +1000 NeilBrown <neilb@suse.de> wrote:

> 
> Hi all,
>  it has been way too long since an mdadm release.  So I'm hoping to pull one
>  together over the next few weeks.  It will be mdadm-3.3
>  My goal is to have it out by 25th July (i.e. one month).
> 

That would be .. today.  I didn't quite make it.
I've just tagged mdadm-3.3-rc2 and pushed that out for anyone who wants to
test.

I still need to resolve whether I'm going to make --add safer (haven't really
thought more about it yet) and I need some self-tests for the bad-block log

I'd like to release by July 31st, but I suspect it will be some time in the
first half of August.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]