ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[Jes Sorensen]

I am pleased to announce the availability of
   mdadm version 4.0

It is available at the usual places:
   http://www.kernel.org/pub/linux/utils/raid/mdadm/
and via git at
   git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
   http://git.kernel.org/cgit/utils/mdadm/

The update in major version number primarily indicates this is a
release by it's new maintainer. In addition it contains a large number
of fixes in particular for IMSM RAID and clustered RAID support.  In
addition this release includes support for IMSM 4k sector drives,
failfast and better documentation for journaled RAID.

This is my first release of mdadm. Please thank Neil Brown for his
previous work as maintainer and blame me for all the bugs I caused
since taking over.

Jes Sorensen, 2017-01-09

Re: ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[Bruce Dubbs]

Jes Sorensen wrote:
> I am pleased to announce the availability of
>     mdadm version 4.0
>
> It is available at the usual places:
>     http://www.kernel.org/pub/linux/utils/raid/mdadm/
> and via git at
>     git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
>     http://git.kernel.org/cgit/utils/mdadm/
>
> The update in major version number primarily indicates this is a
> release by it's new maintainer. In addition it contains a large number
> of fixes in particular for IMSM RAID and clustered RAID support.  In
> addition this release includes support for IMSM 4k sector drives,
> failfast and better documentation for journaled RAID.

Thank you for the new release.  Unfortunately I get 9 failures running the 
test suite:

tests/00raid1...          FAILED
tests/07autoassemble...   FAILED
tests/07changelevels...   FAILED
tests/07revert-grow...    FAILED
tests/07revert-inplace... FAILED
tests/07testreshape5...   FAILED
tests/10ddf-fail-twice... FAILED
tests/20raid5journal...   FAILED
tests/10ddf-incremental-wrong-order...  FAILED

The procedure I used was

make
sudo ./test --keep-going --logdir=test-logs --save-logs

I'll also note that there is an irritating message when a test fails:
cp: cannot stat '/var/tmp/log': No such file or directory

This can be fixed easily enough with:
sed -i 's# if.* == "1"#& -a -e $targetdir/log#' test

I don't know if this mailing list is the right place to report bugs or 
not.  I do not want to spam the list with the logs but they are available at:

http://anduin.linuxfromscratch.org/~bdubbs/mdadm-logs/

   -- Bruce Dubbs
      linuxfromscratch.org

Re: ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[Shaohua Li]

On Tue, Jan 10, 2017 at 11:49:04AM -0600, Bruce Dubbs wrote:
> Jes Sorensen wrote:
> > I am pleased to announce the availability of
> >     mdadm version 4.0
> > 
> > It is available at the usual places:
> >     http://www.kernel.org/pub/linux/utils/raid/mdadm/
> > and via git at
> >     git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
> >     http://git.kernel.org/cgit/utils/mdadm/
> > 
> > The update in major version number primarily indicates this is a
> > release by it's new maintainer. In addition it contains a large number
> > of fixes in particular for IMSM RAID and clustered RAID support.  In
> > addition this release includes support for IMSM 4k sector drives,
> > failfast and better documentation for journaled RAID.
> 
> Thank you for the new release.  Unfortunately I get 9 failures running the
> test suite:
> 
> tests/00raid1...          FAILED
> tests/07autoassemble...   FAILED
> tests/07changelevels...   FAILED
> tests/07revert-grow...    FAILED
> tests/07revert-inplace... FAILED
> tests/07testreshape5...   FAILED
> tests/10ddf-fail-twice... FAILED
> tests/20raid5journal...   FAILED
> tests/10ddf-incremental-wrong-order...  FAILED

Yep, several tests usually fail. It appears some checks aren't always good.  At
least the 'check' function for reshape/resync isn't reliable in my test, I saw
07changelevelintr fails frequently.

Thanks,
Shaohua

Re: ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[Jes Sorensen]

On 01/11/17 11:52, Shaohua Li wrote:
> On Tue, Jan 10, 2017 at 11:49:04AM -0600, Bruce Dubbs wrote:
>> Jes Sorensen wrote:
>>> I am pleased to announce the availability of
>>>     mdadm version 4.0
>>>
>>> It is available at the usual places:
>>>     http://www.kernel.org/pub/linux/utils/raid/mdadm/
>>> and via git at
>>>     git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
>>>     http://git.kernel.org/cgit/utils/mdadm/
>>>
>>> The update in major version number primarily indicates this is a
>>> release by it's new maintainer. In addition it contains a large number
>>> of fixes in particular for IMSM RAID and clustered RAID support.  In
>>> addition this release includes support for IMSM 4k sector drives,
>>> failfast and better documentation for journaled RAID.
>>
>> Thank you for the new release.  Unfortunately I get 9 failures running the
>> test suite:
>>
>> tests/00raid1...          FAILED
>> tests/07autoassemble...   FAILED
>> tests/07changelevels...   FAILED
>> tests/07revert-grow...    FAILED
>> tests/07revert-inplace... FAILED
>> tests/07testreshape5...   FAILED
>> tests/10ddf-fail-twice... FAILED
>> tests/20raid5journal...   FAILED
>> tests/10ddf-incremental-wrong-order...  FAILED
> 
> Yep, several tests usually fail. It appears some checks aren't always good.  At
> least the 'check' function for reshape/resync isn't reliable in my test, I saw
> 07changelevelintr fails frequently.

That is my experience as well - some of them are affected by the kernel
version too. We probably need to look into making them more reliable.

I am also not sure how reliable the DDF tests are on systems without DDF
support.

Cheers,
Jes

Re: ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[Guoqing Jiang]

On 01/12/2017 12:59 AM, Jes Sorensen wrote:
> On 01/11/17 11:52, Shaohua Li wrote:
>> On Tue, Jan 10, 2017 at 11:49:04AM -0600, Bruce Dubbs wrote:
>>> Jes Sorensen wrote:
>>>> I am pleased to announce the availability of
>>>>      mdadm version 4.0
>>>>
>>>> It is available at the usual places:
>>>>      http://www.kernel.org/pub/linux/utils/raid/mdadm/
>>>> and via git at
>>>>      git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
>>>>      http://git.kernel.org/cgit/utils/mdadm/
>>>>
>>>> The update in major version number primarily indicates this is a
>>>> release by it's new maintainer. In addition it contains a large number
>>>> of fixes in particular for IMSM RAID and clustered RAID support.  In
>>>> addition this release includes support for IMSM 4k sector drives,
>>>> failfast and better documentation for journaled RAID.
>>> Thank you for the new release.  Unfortunately I get 9 failures running the
>>> test suite:
>>>
>>> tests/00raid1...          FAILED
>>> tests/07autoassemble...   FAILED
>>> tests/07changelevels...   FAILED
>>> tests/07revert-grow...    FAILED
>>> tests/07revert-inplace... FAILED
>>> tests/07testreshape5...   FAILED
>>> tests/10ddf-fail-twice... FAILED
>>> tests/20raid5journal...   FAILED
>>> tests/10ddf-incremental-wrong-order...  FAILED
>> Yep, several tests usually fail. It appears some checks aren't always good.  At
>> least the 'check' function for reshape/resync isn't reliable in my test, I saw
>> 07changelevelintr fails frequently.
> That is my experience as well - some of them are affected by the kernel
> version too. We probably need to look into making them more reliable.

If possible, it could be a potential topic for lsf/mm raid discussion as 
Coly suggested
in previous mail.

Is current test can run the test for different raid level, say, "./test 
--raidtype=raid1" could
execute all the *r1* tests, does it make sense to do it if we don't 
support it now.

Thanks,
Guoqing

Re: ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[Jes Sorensen]

On 01/11/17 23:24, Guoqing Jiang wrote:
> 
> 
> On 01/12/2017 12:59 AM, Jes Sorensen wrote:
>> On 01/11/17 11:52, Shaohua Li wrote:
>>> On Tue, Jan 10, 2017 at 11:49:04AM -0600, Bruce Dubbs wrote:
>>>> Jes Sorensen wrote:
>>>>> I am pleased to announce the availability of
>>>>>      mdadm version 4.0
>>>>>
>>>>> It is available at the usual places:
>>>>>      http://www.kernel.org/pub/linux/utils/raid/mdadm/
>>>>> and via git at
>>>>>      git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
>>>>>      http://git.kernel.org/cgit/utils/mdadm/
>>>>>
>>>>> The update in major version number primarily indicates this is a
>>>>> release by it's new maintainer. In addition it contains a large number
>>>>> of fixes in particular for IMSM RAID and clustered RAID support.  In
>>>>> addition this release includes support for IMSM 4k sector drives,
>>>>> failfast and better documentation for journaled RAID.
>>>> Thank you for the new release.  Unfortunately I get 9 failures
>>>> running the
>>>> test suite:
>>>>
>>>> tests/00raid1...          FAILED
>>>> tests/07autoassemble...   FAILED
>>>> tests/07changelevels...   FAILED
>>>> tests/07revert-grow...    FAILED
>>>> tests/07revert-inplace... FAILED
>>>> tests/07testreshape5...   FAILED
>>>> tests/10ddf-fail-twice... FAILED
>>>> tests/20raid5journal...   FAILED
>>>> tests/10ddf-incremental-wrong-order...  FAILED
>>> Yep, several tests usually fail. It appears some checks aren't always
>>> good.  At
>>> least the 'check' function for reshape/resync isn't reliable in my
>>> test, I saw
>>> 07changelevelintr fails frequently.
>> That is my experience as well - some of them are affected by the kernel
>> version too. We probably need to look into making them more reliable.
> 
> If possible, it could be a potential topic for lsf/mm raid discussion as
> Coly suggested
> in previous mail.
> 
> Is current test can run the test for different raid level, say, "./test
> --raidtype=raid1" could
> execute all the *r1* tests, does it make sense to do it if we don't
> support it now.

We could have a discussion about this at LSF/MM, if someone is willing
to sponsor getting it accepted and we can get the right people there.

Note that the test suite also allows you to run all the 01 tests by
specifying ./test 01. I do like to see the test suite improved and made
more resilient.

Cheers,
Jes

Re: ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[zhilong]

Hi, Jes;


On 01/13/2017 12:41 AM, Jes Sorensen wrote:
> On 01/11/17 23:24, Guoqing Jiang wrote:
>>
>> On 01/12/2017 12:59 AM, Jes Sorensen wrote:
>>> On 01/11/17 11:52, Shaohua Li wrote:
>>>> On Tue, Jan 10, 2017 at 11:49:04AM -0600, Bruce Dubbs wrote:
>>>>> Jes Sorensen wrote:
>>>>>> I am pleased to announce the availability of
>>>>>>       mdadm version 4.0
>>>>>>
>>>>>> It is available at the usual places:
>>>>>>       http://www.kernel.org/pub/linux/utils/raid/mdadm/
>>>>>> and via git at
>>>>>>       git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
>>>>>>       http://git.kernel.org/cgit/utils/mdadm/
>>>>>>
>>>>>> The update in major version number primarily indicates this is a
>>>>>> release by it's new maintainer. In addition it contains a large number
>>>>>> of fixes in particular for IMSM RAID and clustered RAID support.  In
>>>>>> addition this release includes support for IMSM 4k sector drives,
>>>>>> failfast and better documentation for journaled RAID.
>>>>> Thank you for the new release.  Unfortunately I get 9 failures
>>>>> running the
>>>>> test suite:
>>>>>
>>>>> tests/00raid1...          FAILED
>>>>> tests/07autoassemble...   FAILED
>>>>> tests/07changelevels...   FAILED
>>>>> tests/07revert-grow...    FAILED
>>>>> tests/07revert-inplace... FAILED
>>>>> tests/07testreshape5...   FAILED
>>>>> tests/10ddf-fail-twice... FAILED
>>>>> tests/20raid5journal...   FAILED
>>>>> tests/10ddf-incremental-wrong-order...  FAILED
>>>> Yep, several tests usually fail. It appears some checks aren't always
>>>> good.  At
>>>> least the 'check' function for reshape/resync isn't reliable in my
>>>> test, I saw
>>>> 07changelevelintr fails frequently.
>>> That is my experience as well - some of them are affected by the kernel
>>> version too. We probably need to look into making them more reliable.
>> If possible, it could be a potential topic for lsf/mm raid discussion as
>> Coly suggested
>> in previous mail.
>>
>> Is current test can run the test for different raid level, say, "./test
>> --raidtype=raid1" could
>> execute all the *r1* tests, does it make sense to do it if we don't
>> support it now.
> We could have a discussion about this at LSF/MM, if someone is willing
> to sponsor getting it accepted and we can get the right people there.
>
> Note that the test suite also allows you to run all the 01 tests by
> specifying ./test 01. I do like to see the test suite improved and made
> more resilient.
I'm sorry for my late response, I'm just back to work today from 
vacation. In the past months, I learned and worked for cluster-md feature,
and I have draft one test suit for cluster-md feature. please refer to 
https://github.com/zhilongliu/clustermd-autotest
I'm very willing to do something for improving mdadm testing part, also 
wanna improve cluster-md test suit, welcome all comments for it.

> Cheers,
> Jes
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Thanks very much,
-Zhilong

Re: ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[zhilong]

On 02/13/2017 01:08 PM, zhilong wrote:
> Hi, Jes;
>
>
> On 01/13/2017 12:41 AM, Jes Sorensen wrote:
>> On 01/11/17 23:24, Guoqing Jiang wrote:
>>>
>>> On 01/12/2017 12:59 AM, Jes Sorensen wrote:
>>>> On 01/11/17 11:52, Shaohua Li wrote:
>>>>> On Tue, Jan 10, 2017 at 11:49:04AM -0600, Bruce Dubbs wrote:
>>>>>> Jes Sorensen wrote:
>>>>>>> I am pleased to announce the availability of
>>>>>>>       mdadm version 4.0
>>>>>>>
>>>>>>> It is available at the usual places:
>>>>>>> http://www.kernel.org/pub/linux/utils/raid/mdadm/
>>>>>>> and via git at
>>>>>>> git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
>>>>>>>       http://git.kernel.org/cgit/utils/mdadm/
>>>>>>>
>>>>>>> The update in major version number primarily indicates this is a
>>>>>>> release by it's new maintainer. In addition it contains a large 
>>>>>>> number
>>>>>>> of fixes in particular for IMSM RAID and clustered RAID 
>>>>>>> support.  In
>>>>>>> addition this release includes support for IMSM 4k sector drives,
>>>>>>> failfast and better documentation for journaled RAID.
>>>>>> Thank you for the new release.  Unfortunately I get 9 failures
>>>>>> running the
>>>>>> test suite:
>>>>>>
>>>>>> tests/00raid1...          FAILED
>>>>>> tests/07autoassemble...   FAILED
>>>>>> tests/07changelevels...   FAILED
>>>>>> tests/07revert-grow...    FAILED
>>>>>> tests/07revert-inplace... FAILED
>>>>>> tests/07testreshape5...   FAILED
>>>>>> tests/10ddf-fail-twice... FAILED
>>>>>> tests/20raid5journal...   FAILED
>>>>>> tests/10ddf-incremental-wrong-order...  FAILED
>>>>> Yep, several tests usually fail. It appears some checks aren't always
>>>>> good.  At
>>>>> least the 'check' function for reshape/resync isn't reliable in my
>>>>> test, I saw
>>>>> 07changelevelintr fails frequently.
>>>> That is my experience as well - some of them are affected by the 
>>>> kernel
>>>> version too. We probably need to look into making them more reliable.
>>> If possible, it could be a potential topic for lsf/mm raid 
>>> discussion as
>>> Coly suggested
>>> in previous mail.
>>>
>>> Is current test can run the test for different raid level, say, "./test
>>> --raidtype=raid1" could
>>> execute all the *r1* tests, does it make sense to do it if we don't
>>> support it now.
>> We could have a discussion about this at LSF/MM, if someone is willing
>> to sponsor getting it accepted and we can get the right people there.
>>
>> Note that the test suite also allows you to run all the 01 tests by
>> specifying ./test 01. I do like to see the test suite improved and made
>> more resilient.
> I'm sorry for my late response, I'm just back to work today from 
> vacation. In the past months, I learned and worked for cluster-md 
> feature,
> and I have draft one test suit for cluster-md feature. please refer to 
> https://github.com/zhilongliu/clustermd-autotest
> I'm very willing to do something for improving mdadm testing part, 
> also wanna improve cluster-md test suit, welcome all comments for it.
>
   I would keep making cluster-md test scripts more and more stable, and 
finally apply to integrate into mdadm test part. :-)

Best regards,
-Zhilong
>> Cheers,
>> Jes
>>
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> Thanks very much,
> -Zhilong
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: ANNOUNCE: mdadm 4.0 - A tool for managing md Soft RAID under Linux

[Jes Sorensen]

On 02/13/2017 12:54 AM, zhilong wrote:
> On 02/13/2017 01:08 PM, zhilong wrote:
>> Hi, Jes;
>> On 01/13/2017 12:41 AM, Jes Sorensen wrote:
>>> On 01/11/17 23:24, Guoqing Jiang wrote:
>>>>
>>>> On 01/12/2017 12:59 AM, Jes Sorensen wrote:
>>>>> On 01/11/17 11:52, Shaohua Li wrote:
>>>>>> On Tue, Jan 10, 2017 at 11:49:04AM -0600, Bruce Dubbs wrote:
>>>>>>> Jes Sorensen wrote:
>>>>>>>> I am pleased to announce the availability of
>>>>>>>>       mdadm version 4.0
>>>>>>>>
>>>>>>>> It is available at the usual places:
>>>>>>>> http://www.kernel.org/pub/linux/utils/raid/mdadm/
>>>>>>>> and via git at
>>>>>>>> git://git.kernel.org/pub/scm/utils/mdadm/mdadm.git
>>>>>>>>       http://git.kernel.org/cgit/utils/mdadm/
>>>>>>>>
>>>>>>>> The update in major version number primarily indicates this is a
>>>>>>>> release by it's new maintainer. In addition it contains a large
>>>>>>>> number
>>>>>>>> of fixes in particular for IMSM RAID and clustered RAID
>>>>>>>> support.  In
>>>>>>>> addition this release includes support for IMSM 4k sector drives,
>>>>>>>> failfast and better documentation for journaled RAID.
>>>>>>> Thank you for the new release.  Unfortunately I get 9 failures
>>>>>>> running the
>>>>>>> test suite:
>>>>>>>
>>>>>>> tests/00raid1...          FAILED
>>>>>>> tests/07autoassemble...   FAILED
>>>>>>> tests/07changelevels...   FAILED
>>>>>>> tests/07revert-grow...    FAILED
>>>>>>> tests/07revert-inplace... FAILED
>>>>>>> tests/07testreshape5...   FAILED
>>>>>>> tests/10ddf-fail-twice... FAILED
>>>>>>> tests/20raid5journal...   FAILED
>>>>>>> tests/10ddf-incremental-wrong-order...  FAILED
>>>>>> Yep, several tests usually fail. It appears some checks aren't always
>>>>>> good.  At
>>>>>> least the 'check' function for reshape/resync isn't reliable in my
>>>>>> test, I saw
>>>>>> 07changelevelintr fails frequently.
>>>>> That is my experience as well - some of them are affected by the
>>>>> kernel
>>>>> version too. We probably need to look into making them more reliable.
>>>> If possible, it could be a potential topic for lsf/mm raid
>>>> discussion as
>>>> Coly suggested
>>>> in previous mail.
>>>>
>>>> Is current test can run the test for different raid level, say, "./test
>>>> --raidtype=raid1" could
>>>> execute all the *r1* tests, does it make sense to do it if we don't
>>>> support it now.
>>> We could have a discussion about this at LSF/MM, if someone is willing
>>> to sponsor getting it accepted and we can get the right people there.
>>>
>>> Note that the test suite also allows you to run all the 01 tests by
>>> specifying ./test 01. I do like to see the test suite improved and made
>>> more resilient.
>> I'm sorry for my late response, I'm just back to work today from
>> vacation. In the past months, I learned and worked for cluster-md
>> feature,
>> and I have draft one test suit for cluster-md feature. please refer to
>> https://github.com/zhilongliu/clustermd-autotest
>> I'm very willing to do something for improving mdadm testing part,
>> also wanna improve cluster-md test suit, welcome all comments for it.
>>
>   I would keep making cluster-md test scripts more and more stable, and
> finally apply to integrate into mdadm test part. :-)

I'd very much like to see work to improve the test suite, so that is great.

Once you have the test suites ready, please post patches and I shall be 
happy to implement them.

Please make sure to test that they don't break if people haven't built 
cluster support into their kernels.

Cheers,
Jes