commit backport request

commit backport request

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]


Hi Mr Stable,

 please apply patch

commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
Author: Shaohua Li <shli@kernel.org>
Date:   Tue Jul 3 15:57:19 2012 +1000

    raid5: delayed stripe fix


to 3.4.y and 3.2.y.

It fixes a deadlock in RAID5 which can cause all IO to hang.
It has been demonstrated to fix a real-life problem (the race is not
theoretical).

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: commit backport request

[Igor M Podlesny]

On 9 July 2012 12:02, NeilBrown <neilb@suse.de> wrote:
>
> Hi Mr Stable,
>
>  please apply patch
>
> commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
> Author: Shaohua Li <shli@kernel.org>
> Date:   Tue Jul 3 15:57:19 2012 +1000
>
>     raid5: delayed stripe fix
>
>
> to 3.4.y and 3.2.y.
>
> It fixes a deadlock in RAID5 which can cause all IO to hang.

   What about RAID6 as well?

> It has been demonstrated to fix a real-life problem (the race is not
> theoretical).
>
> Thanks,
> NeilBrown

--

Re: commit backport request

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 573 bytes --]

On Mon, 9 Jul 2012 13:08:44 +0800 Igor M Podlesny <for.poige+lsr@gmail.com>
wrote:

> On 9 July 2012 12:02, NeilBrown <neilb@suse.de> wrote:
> >
> > Hi Mr Stable,
> >
> >  please apply patch
> >
> > commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
> > Author: Shaohua Li <shli@kernel.org>
> > Date:   Tue Jul 3 15:57:19 2012 +1000
> >
> >     raid5: delayed stripe fix
> >
> >
> > to 3.4.y and 3.2.y.
> >
> > It fixes a deadlock in RAID5 which can cause all IO to hang.
> 
>    What about RAID6 as well?
>

Yes, RAID4, RAID5, and RAID6.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: commit backport request

[Greg KH]

On Mon, Jul 09, 2012 at 02:02:30PM +1000, NeilBrown wrote:
> 
> Hi Mr Stable,
> 
>  please apply patch
> 
> commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
> Author: Shaohua Li <shli@kernel.org>
> Date:   Tue Jul 3 15:57:19 2012 +1000
> 
>     raid5: delayed stripe fix
> 
> 
> to 3.4.y and 3.2.y.
> 
> It fixes a deadlock in RAID5 which can cause all IO to hang.
> It has been demonstrated to fix a real-life problem (the race is not
> theoretical).

Is it also needed for 3.0.x?  It seems to apply there just fine.

thanks,

greg k-h

Re: commit backport request

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 671 bytes --]

On Mon, 2012-07-09 at 14:02 +1000, NeilBrown wrote:
> Hi Mr Stable,
> 
>  please apply patch
> 
> commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
> Author: Shaohua Li <shli@kernel.org>
> Date:   Tue Jul 3 15:57:19 2012 +1000
> 
>     raid5: delayed stripe fix
> 
> 
> to 3.4.y and 3.2.y.
> 
> It fixes a deadlock in RAID5 which can cause all IO to hang.
> It has been demonstrated to fix a real-life problem (the race is not
> theoretical).

Queued up for 3.2.y, thanks.

Ben.

-- 
Ben Hutchings
The generation of random numbers is too important to be left to chance.
                                                            - Robert Coveyou

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: commit backport request

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 761 bytes --]

On Wed, 11 Jul 2012 15:07:40 -0700 Greg KH <gregkh@linuxfoundation.org> wrote:

> On Mon, Jul 09, 2012 at 02:02:30PM +1000, NeilBrown wrote:
> > 
> > Hi Mr Stable,
> > 
> >  please apply patch
> > 
> > commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
> > Author: Shaohua Li <shli@kernel.org>
> > Date:   Tue Jul 3 15:57:19 2012 +1000
> > 
> >     raid5: delayed stripe fix
> > 
> > 
> > to 3.4.y and 3.2.y.
> > 
> > It fixes a deadlock in RAID5 which can cause all IO to hang.
> > It has been demonstrated to fix a real-life problem (the race is not
> > theoretical).
> 
> Is it also needed for 3.0.x?  It seems to apply there just fine.

Yes it is.  Don't know why I missed that.
I see you've already included it - thanks.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: commit backport request

[Alexander Lyakas]

Hi,
Is there any indication that can tell I am hitting this particular issue?
I think I hit something similar to that once, IOs were hang within a
raid5 and not returning. I issues w > sysrq-trigger and also have most
of the procfs/<pid> info, so is there something I can look at to
confirm?



On Thu, Jul 12, 2012 at 7:14 AM, NeilBrown <neilb@suse.de> wrote:
> On Wed, 11 Jul 2012 15:07:40 -0700 Greg KH <gregkh@linuxfoundation.org> wrote:
>
>> On Mon, Jul 09, 2012 at 02:02:30PM +1000, NeilBrown wrote:
>> >
>> > Hi Mr Stable,
>> >
>> >  please apply patch
>> >
>> > commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
>> > Author: Shaohua Li <shli@kernel.org>
>> > Date:   Tue Jul 3 15:57:19 2012 +1000
>> >
>> >     raid5: delayed stripe fix
>> >
>> >
>> > to 3.4.y and 3.2.y.
>> >
>> > It fixes a deadlock in RAID5 which can cause all IO to hang.
>> > It has been demonstrated to fix a real-life problem (the race is not
>> > theoretical).
>>
>> Is it also needed for 3.0.x?  It seems to apply there just fine.
>
> Yes it is.  Don't know why I missed that.
> I see you've already included it - thanks.
>
> NeilBrown

Re: commit backport request

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1488 bytes --]

On Fri, 13 Jul 2012 00:28:00 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
wrote:

> Hi,
> Is there any indication that can tell I am hitting this particular issue?
> I think I hit something similar to that once, IOs were hang within a
> raid5 and not returning. I issues w > sysrq-trigger and also have most
> of the procfs/<pid> info, so is there something I can look at to
> confirm?

IO stops, processes get blocked in get_active_stripe().  Increasing
stripe_cache_size allows a bit more IO through it eventually all fills up
again.

NeilBrown

> 
> 
> 
> On Thu, Jul 12, 2012 at 7:14 AM, NeilBrown <neilb@suse.de> wrote:
> > On Wed, 11 Jul 2012 15:07:40 -0700 Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> >> On Mon, Jul 09, 2012 at 02:02:30PM +1000, NeilBrown wrote:
> >> >
> >> > Hi Mr Stable,
> >> >
> >> >  please apply patch
> >> >
> >> > commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
> >> > Author: Shaohua Li <shli@kernel.org>
> >> > Date:   Tue Jul 3 15:57:19 2012 +1000
> >> >
> >> >     raid5: delayed stripe fix
> >> >
> >> >
> >> > to 3.4.y and 3.2.y.
> >> >
> >> > It fixes a deadlock in RAID5 which can cause all IO to hang.
> >> > It has been demonstrated to fix a real-life problem (the race is not
> >> > theoretical).
> >>
> >> Is it also needed for 3.0.x?  It seems to apply there just fine.
> >
> > Yes it is.  Don't know why I missed that.
> > I see you've already included it - thanks.
> >
> > NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: commit backport request

[Alexander Lyakas]

Thanks.
Not sure if this is the issue I saw. I did not see any thread blocked
on get_active_stripe(). In my case "stripe_cache_active" got stuck on
a high value and was never decreased. But some IOs (perhaps on other
stripes?) managed to go through. In addition, in one case, after
several hours(!), somehow the stuck IOs got freed and
stripe_cache_active dropped and everything got back to normal.

Alex.


On Mon, Jul 16, 2012 at 6:23 AM, NeilBrown <neilb@suse.de> wrote:
> On Fri, 13 Jul 2012 00:28:00 +0300 Alexander Lyakas <alex.bolshoy@gmail.com>
> wrote:
>
>> Hi,
>> Is there any indication that can tell I am hitting this particular issue?
>> I think I hit something similar to that once, IOs were hang within a
>> raid5 and not returning. I issues w > sysrq-trigger and also have most
>> of the procfs/<pid> info, so is there something I can look at to
>> confirm?
>
> IO stops, processes get blocked in get_active_stripe().  Increasing
> stripe_cache_size allows a bit more IO through it eventually all fills up
> again.
>
> NeilBrown
>
>>
>>
>>
>> On Thu, Jul 12, 2012 at 7:14 AM, NeilBrown <neilb@suse.de> wrote:
>> > On Wed, 11 Jul 2012 15:07:40 -0700 Greg KH <gregkh@linuxfoundation.org> wrote:
>> >
>> >> On Mon, Jul 09, 2012 at 02:02:30PM +1000, NeilBrown wrote:
>> >> >
>> >> > Hi Mr Stable,
>> >> >
>> >> >  please apply patch
>> >> >
>> >> > commit fab363b5ff502d1b39ddcfec04271f5858d9f26e
>> >> > Author: Shaohua Li <shli@kernel.org>
>> >> > Date:   Tue Jul 3 15:57:19 2012 +1000
>> >> >
>> >> >     raid5: delayed stripe fix
>> >> >
>> >> >
>> >> > to 3.4.y and 3.2.y.
>> >> >
>> >> > It fixes a deadlock in RAID5 which can cause all IO to hang.
>> >> > It has been demonstrated to fix a real-life problem (the race is not
>> >> > theoretical).
>> >>
>> >> Is it also needed for 3.0.x?  It seems to apply there just fine.
>> >
>> > Yes it is.  Don't know why I missed that.
>> > I see you've already included it - thanks.
>> >
>> > NeilBrown
>

Re: commit backport request

[Igor M Podlesny]

On 12 July 2012 11:52, Ben Hutchings <ben@decadent.org.uk> wrote:
> On Mon, 2012-07-09 at 14:02 +1000, NeilBrown wrote:
[...]
> Queued up for 3.2.y, thanks.
>
> Ben.

   Hm, it looks like 3.2.23 misses it — at least my during rebase the
cherry-picked fix applied again cleanly.

--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: commit backport request

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 694 bytes --]

On Tue, 17 Jul 2012 11:26:41 +0800 Igor M Podlesny <for.poige+lsr@gmail.com>
wrote:

> On 12 July 2012 11:52, Ben Hutchings <ben@decadent.org.uk> wrote:
> > On Mon, 2012-07-09 at 14:02 +1000, NeilBrown wrote:
> [...]
> > Queued up for 3.2.y, thanks.
> >
> > Ben.
> 
>    Hm, it looks like 3.2.23 misses it — at least my during rebase the
> cherry-picked fix applied again cleanly.
> 
> --

Given that the date on:

commit 8432d2280bd7558fde3a0385d1b35c947197d754
Author: Ben Hutchings <ben@decadent.org.uk>
Date:   Thu Jul 12 04:32:21 2012 +0100

    Linux 3.2.23


is before the date on Ben's email (by a few hours), I suspect it is queued for
3.2.24.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: commit backport request

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On Tue, 2012-07-17 at 11:26 +0800, Igor M Podlesny wrote:
> On 12 July 2012 11:52, Ben Hutchings <ben@decadent.org.uk> wrote:
> > On Mon, 2012-07-09 at 14:02 +1000, NeilBrown wrote:
> [...]
> > Queued up for 3.2.y, thanks.
> >
> > Ben.
> 
>    Hm, it looks like 3.2.23 misses it — at least my during rebase the
> cherry-picked fix applied again cleanly.

I queued it up some hours after tagging that release.  I *announced* the
release later, but that's because uploading tarballs etc. involves a
round trip via Greg.

Ben.

-- 
Ben Hutchings
Gates has joked that everything goes on and off unexepectedly in the house,
which is run by a high-end PC network built on Windows NT. - Seattle Times

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: commit backport request

[Igor M Podlesny]

On 17 July 2012 11:48, Ben Hutchings <ben@decadent.org.uk> wrote:
> On Tue, 2012-07-17 at 11:26 +0800, Igor M Podlesny wrote:
>> On 12 July 2012 11:52, Ben Hutchings <ben@decadent.org.uk> wrote:
>> > On Mon, 2012-07-09 at 14:02 +1000, NeilBrown wrote:
>> [...]
>> > Queued up for 3.2.y, thanks.
>> >
>> > Ben.
>>
>>    Hm, it looks like 3.2.23 misses it — at least my during rebase the
>> cherry-picked fix applied again cleanly.
>
> I queued it up some hours after tagging that release.  I *announced* the
> release later, but that's because uploading tarballs etc. involves a
> round trip via Greg.
>
> Ben.

   JFYI, 3.4.6 doesn't have it either...

commit 1c8f63c2758096c3b6425f4ecb274901151d6f17
Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date:   Thu Jul 19 12:11:49 2012 -0700

    Linux 3.4.6

--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: commit backport request

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1249 bytes --]

On Fri, 20 Jul 2012 10:38:50 +0800 Igor M Podlesny <for.poige+lsr@gmail.com>
wrote:

> On 17 July 2012 11:48, Ben Hutchings <ben@decadent.org.uk> wrote:
> > On Tue, 2012-07-17 at 11:26 +0800, Igor M Podlesny wrote:
> >> On 12 July 2012 11:52, Ben Hutchings <ben@decadent.org.uk> wrote:
> >> > On Mon, 2012-07-09 at 14:02 +1000, NeilBrown wrote:
> >> [...]
> >> > Queued up for 3.2.y, thanks.
> >> >
> >> > Ben.
> >>
> >>    Hm, it looks like 3.2.23 misses it — at least my during rebase the
> >> cherry-picked fix applied again cleanly.
> >
> > I queued it up some hours after tagging that release.  I *announced* the
> > release later, but that's because uploading tarballs etc. involves a
> > round trip via Greg.
> >
> > Ben.
> 
>    JFYI, 3.4.6 doesn't have it either...
> 
> commit 1c8f63c2758096c3b6425f4ecb274901151d6f17
> Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Date:   Thu Jul 19 12:11:49 2012 -0700
> 
>     Linux 3.4.6
> 
> --


Yes it does.

commit 5bbbd747918d241b41f3220ff26323d7ed943c52
Author: Shaohua Li <shli@kernel.org>
Date:   Tue Jul 3 15:57:19 2012 +1000

    raid5: delayed stripe fix
    
    commit fab363b5ff502d1b39ddcfec04271f5858d9f26e upstream.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: commit backport request

[Igor M Podlesny]

On 20 July 2012 10:58, NeilBrown <neilb@suse.de> wrote:
> On Fri, 20 Jul 2012 10:38:50 +0800 Igor M Podlesny <for.poige+lsr@gmail.com>
> wrote:
[…]
>>    JFYI, 3.4.6 doesn't have it either...
>
> Yes it does.

   Correct, my mistake -- I missed it on its way to 3.4.5, sorry.

--
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html