[GIT PULL] bcache changes for 3.17

[GIT PULL] bcache changes for 3.17

[Kent Overstreet]

Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
fixes in this one :)

The following changes since commit 4c834452aad01531db949414f94f817a86348d59:

  Linux 3.16-rc3 (2014-06-29 14:11:36 -0700)

are available in the git repository at:

  http://evilpiepirate.org/git/linux-bcache.git for-jens

for you to fetch changes up to 0781c8748cf1ea2b0dcd966571103909528c4efa:

  bcache: Drop unneeded blk_sync_queue() calls (2014-08-04 15:23:04 -0700)

----------------------------------------------------------------
Jianjian Huo (1):
      bcache: add mutex lock for bch_is_open

Kent Overstreet (7):
      bcache: Fix a bug when detaching
      bcache: Fix a journal replay bug
      bcache: Make sure to pass GFP_WAIT to mempool_alloc()
      bcache: Allocate bounce buffers with GFP_NOWAIT
      bcache: Fix an infinite loop in journal replay
      bcache: Fix more early shutdown bugs
      bcache: Drop unneeded blk_sync_queue() calls

Slava Pestov (12):
      bcache allocator: send discards with correct size
      bcache: fix lockdep warnings on shutdown
      bcache: fix crash on shutdown in passthrough mode
      bcache: wait for buckets when allocating new btree root
      bcache: fix uninterruptible sleep in writeback thread
      bcache: fix typo in bch_bkey_equal_header
      bcache: bcache_write tracepoint was crashing
      bcache: fix crash in bcache_btree_node_alloc_fail tracepoint
      bcache: fix use-after-free in btree_gc_coalesce()
      bcache: fix crash with incomplete cache set
      bcache: fix memory corruption in init error path
      bcache: try to set b->parent properly

Surbhi Palande (2):
      bcache: Fix to remove the rcu_sched stalls.
      bcache: Correct printing of btree_gc_max_duration_ms

 drivers/md/bcache/alloc.c     |  2 +-
 drivers/md/bcache/bcache.h    |  4 +++
 drivers/md/bcache/bset.c      |  2 +-
 drivers/md/bcache/bset.h      |  2 +-
 drivers/md/bcache/btree.c     | 50 ++++++++++++++++++++++---------------
 drivers/md/bcache/btree.h     |  5 ++--
 drivers/md/bcache/extents.c   | 13 +++++++---
 drivers/md/bcache/extents.h   |  1 +
 drivers/md/bcache/journal.c   | 24 +++++++++++-------
 drivers/md/bcache/request.c   |  3 ++-
 drivers/md/bcache/super.c     | 57 ++++++++++++++++++++++++++++---------------
 drivers/md/bcache/util.h      |  4 +--
 drivers/md/bcache/writeback.c | 14 ++++++++---
 drivers/md/bcache/writeback.h |  3 ++-
 include/trace/events/bcache.h | 21 +++++++++-------
 15 files changed, 131 insertions(+), 74 deletions(-)

Re: [GIT PULL] bcache changes for 3.17

[Jens Axboe]

On 08/04/2014 10:33 PM, Kent Overstreet wrote:
> Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
> fixes in this one :)

Normally I'd say no, but since it's basically just fixes, I guess we can
pull it in. But generally, it has to be in my hands a week before this,
so it can simmer a bit in for-next before going in...

-- 
Jens Axboe

Re: [GIT PULL] bcache changes for 3.17

[Peter Kieser]

[-- Attachment #1: Type: text/plain, Size: 501 bytes --]


On 2014-08-05 9:58 AM, Jens Axboe wrote:
> On 08/04/2014 10:33 PM, Kent Overstreet wrote:
>> Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
>> fixes in this one :)
> Normally I'd say no, but since it's basically just fixes, I guess we can
> pull it in. But generally, it has to be in my hands a week before this,
> so it can simmer a bit in for-next before going in...
>
Are these fixes going to be backported to 3.10 or other stable releases?

-Peter



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4504 bytes --]

Re: [GIT PULL] bcache changes for 3.17

[Francis Moreau]

On 08/10/2014 09:54 AM, Peter Kieser wrote:
> 
> On 2014-08-05 9:58 AM, Jens Axboe wrote:
>> On 08/04/2014 10:33 PM, Kent Overstreet wrote:
>>> Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
>>> fixes in this one :)
>> Normally I'd say no, but since it's basically just fixes, I guess we can
>> pull it in. But generally, it has to be in my hands a week before this,
>> so it can simmer a bit in for-next before going in...
>>
> Are these fixes going to be backported to 3.10 or other stable releases?
> 

Could you please answer this question ?

If you don't want to maintain bcache for stable kernels (I can
understand that), can you mark it at least as unstable/experimental
stuff since it really is ?

Thanks

Re: [GIT PULL] bcache changes for 3.17

[Jens Axboe]

On 09/05/2014 01:31 AM, Francis Moreau wrote:
> On 08/10/2014 09:54 AM, Peter Kieser wrote:
>>
>> On 2014-08-05 9:58 AM, Jens Axboe wrote:
>>> On 08/04/2014 10:33 PM, Kent Overstreet wrote:
>>>> Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
>>>> fixes in this one :)
>>> Normally I'd say no, but since it's basically just fixes, I guess we can
>>> pull it in. But generally, it has to be in my hands a week before this,
>>> so it can simmer a bit in for-next before going in...
>>>
>> Are these fixes going to be backported to 3.10 or other stable releases?
>>
> 
> Could you please answer this question ?
> 
> If you don't want to maintain bcache for stable kernels (I can
> understand that), can you mark it at least as unstable/experimental
> stuff since it really is ?

We need to do something about this. From this latest pull, looks like
all should go to stable:

5b1016e62f74c53e0330403025954c8d95384c03
9aa61a992acceeec0d1de2cd99938421498659d5
dbd810ab678d262d3772d29b65844d7b20dc47bc
8b326d3a2a76912dfed2f0ab937d59fae9512ca2
e5112201c1285841f8b565ece5d6ae7e0d7947a2
a664d0f05a2ec02c8f042db536d84d15d6e19e81
c5aa4a3157b55bdca18dd2a9d9f43314470b6d32
9e5c353510b26500bd6b8309823ac9ef2837b761
bcf090e0040e30f8409e6a535a01e6473afb096f
501d52a90cbe652b41336c206ff0e95799d5a9b5
8e0948080670f6330229718b15a6a1a011d441ce
60ae81eee86dd7a520db8c1e3d702b49fc0418b5
913dc33fb2720fb5f979011664294137ddd8b13b
6b708de64adb6dc8319e7aeac922b46904fbeeec
400ffaa2acd72274e2c7293a9724382383bebf3e
d83353b319d47ef8cce82467da6a25c2d558253f
bf0c55c986540483c34ca640f2eef4c3314388b1
c9a78332b42cbdcdd386a95192a716b67d1711a4
2452cc89063a2a6890368f185c4b6d7d8802175b
25abade29616d42d60f9bd5e6a5ad07f7314e39e
5b25abade29616d42d60f9bd5e6a5ad07f7314e3
789d21dbd9d8889e62c79ec19585fcc97e42ef07
0781c8748cf1ea2b0dcd966571103909528c4efa

(from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
all the way back to 3.10 would be a lot of work. If there's anyone that
cares about bcache on stable kernels (and actually use it), now would be
a good time to pipe up.

-- 
Jens Axboe

Re: [GIT PULL] bcache changes for 3.17

[Vasiliy Tolstov]

2014-09-05 18:17 GMT+04:00 Jens Axboe <axboe@kernel.dk>:
>
> We need to do something about this. From this latest pull, looks like
> all should go to stable:
>
> 5b1016e62f74c53e0330403025954c8d95384c03
> 9aa61a992acceeec0d1de2cd99938421498659d5
> dbd810ab678d262d3772d29b65844d7b20dc47bc
> 8b326d3a2a76912dfed2f0ab937d59fae9512ca2
> e5112201c1285841f8b565ece5d6ae7e0d7947a2
> a664d0f05a2ec02c8f042db536d84d15d6e19e81
> c5aa4a3157b55bdca18dd2a9d9f43314470b6d32
> 9e5c353510b26500bd6b8309823ac9ef2837b761
> bcf090e0040e30f8409e6a535a01e6473afb096f
> 501d52a90cbe652b41336c206ff0e95799d5a9b5
> 8e0948080670f6330229718b15a6a1a011d441ce
> 60ae81eee86dd7a520db8c1e3d702b49fc0418b5
> 913dc33fb2720fb5f979011664294137ddd8b13b
> 6b708de64adb6dc8319e7aeac922b46904fbeeec
> 400ffaa2acd72274e2c7293a9724382383bebf3e
> d83353b319d47ef8cce82467da6a25c2d558253f
> bf0c55c986540483c34ca640f2eef4c3314388b1
> c9a78332b42cbdcdd386a95192a716b67d1711a4
> 2452cc89063a2a6890368f185c4b6d7d8802175b
> 25abade29616d42d60f9bd5e6a5ad07f7314e39e
> 5b25abade29616d42d60f9bd5e6a5ad07f7314e3
> 789d21dbd9d8889e62c79ec19585fcc97e42ef07
> 0781c8748cf1ea2b0dcd966571103909528c4efa
>
> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
> all the way back to 3.10 would be a lot of work. If there's anyone that
> cares about bcache on stable kernels (and actually use it), now would be
> a good time to pipe up.


I'm interesting on backporting this to 3.14.y


-- 
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru
jabber: vase@selfip.ru

Re: [GIT PULL] bcache changes for 3.17

[Jens Axboe]

On 09/05/2014 08:28 AM, Vasiliy Tolstov wrote:
> 2014-09-05 18:17 GMT+04:00 Jens Axboe <axboe@kernel.dk>:
>>
>> We need to do something about this. From this latest pull, looks like
>> all should go to stable:
>>
>> 5b1016e62f74c53e0330403025954c8d95384c03
>> 9aa61a992acceeec0d1de2cd99938421498659d5
>> dbd810ab678d262d3772d29b65844d7b20dc47bc
>> 8b326d3a2a76912dfed2f0ab937d59fae9512ca2
>> e5112201c1285841f8b565ece5d6ae7e0d7947a2
>> a664d0f05a2ec02c8f042db536d84d15d6e19e81
>> c5aa4a3157b55bdca18dd2a9d9f43314470b6d32
>> 9e5c353510b26500bd6b8309823ac9ef2837b761
>> bcf090e0040e30f8409e6a535a01e6473afb096f
>> 501d52a90cbe652b41336c206ff0e95799d5a9b5
>> 8e0948080670f6330229718b15a6a1a011d441ce
>> 60ae81eee86dd7a520db8c1e3d702b49fc0418b5
>> 913dc33fb2720fb5f979011664294137ddd8b13b
>> 6b708de64adb6dc8319e7aeac922b46904fbeeec
>> 400ffaa2acd72274e2c7293a9724382383bebf3e
>> d83353b319d47ef8cce82467da6a25c2d558253f
>> bf0c55c986540483c34ca640f2eef4c3314388b1
>> c9a78332b42cbdcdd386a95192a716b67d1711a4
>> 2452cc89063a2a6890368f185c4b6d7d8802175b
>> 25abade29616d42d60f9bd5e6a5ad07f7314e39e
>> 5b25abade29616d42d60f9bd5e6a5ad07f7314e3
>> 789d21dbd9d8889e62c79ec19585fcc97e42ef07
>> 0781c8748cf1ea2b0dcd966571103909528c4efa
>>
>> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
>> all the way back to 3.10 would be a lot of work. If there's anyone that
>> cares about bcache on stable kernels (and actually use it), now would be
>> a good time to pipe up.
> 
> 
> I'm interesting on backporting this to 3.14.y

Before that can happen, you would probably need to comb the 3.15 and
3.16 bcache inclusions and dig out any potential stable candidates there
too.


-- 
Jens Axboe

Re: [GIT PULL] bcache changes for 3.17

[Francis Moreau]

On 09/05/2014 04:17 PM, Jens Axboe wrote:
> 
> We need to do something about this. From this latest pull, looks like
> all should go to stable:
> 
> 5b1016e62f74c53e0330403025954c8d95384c03
> 9aa61a992acceeec0d1de2cd99938421498659d5
> dbd810ab678d262d3772d29b65844d7b20dc47bc
> 8b326d3a2a76912dfed2f0ab937d59fae9512ca2
> e5112201c1285841f8b565ece5d6ae7e0d7947a2
> a664d0f05a2ec02c8f042db536d84d15d6e19e81
> c5aa4a3157b55bdca18dd2a9d9f43314470b6d32
> 9e5c353510b26500bd6b8309823ac9ef2837b761
> bcf090e0040e30f8409e6a535a01e6473afb096f
> 501d52a90cbe652b41336c206ff0e95799d5a9b5
> 8e0948080670f6330229718b15a6a1a011d441ce
> 60ae81eee86dd7a520db8c1e3d702b49fc0418b5
> 913dc33fb2720fb5f979011664294137ddd8b13b
> 6b708de64adb6dc8319e7aeac922b46904fbeeec
> 400ffaa2acd72274e2c7293a9724382383bebf3e
> d83353b319d47ef8cce82467da6a25c2d558253f
> bf0c55c986540483c34ca640f2eef4c3314388b1
> c9a78332b42cbdcdd386a95192a716b67d1711a4
> 2452cc89063a2a6890368f185c4b6d7d8802175b
> 25abade29616d42d60f9bd5e6a5ad07f7314e39e
> 5b25abade29616d42d60f9bd5e6a5ad07f7314e3
> 789d21dbd9d8889e62c79ec19585fcc97e42ef07
> 0781c8748cf1ea2b0dcd966571103909528c4efa
> 
> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
> all the way back to 3.10 would be a lot of work. If there's anyone that
> cares about bcache on stable kernels (and actually use it), now would be
> a good time to pipe up.
> 

Then if it's too much work, it just confirmed what was asked previously:
bcache is still experimental so mark it such for stable kernels.

I wouldn't have used bcache in that case.

Thanks

Re: [GIT PULL] bcache changes for 3.17

[Eddie Chapman]

On 05/09/14 15:17, Jens Axboe wrote:
> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
> all the way back to 3.10 would be a lot of work. If there's anyone that
> cares about bcache on stable kernels (and actually use it), now would be
> a good time to pipe up.

Just "piping up" as I care about bcache and actually use it in 
production on 3.10! Shame I don't have the knowledge to try and backport 
these though :-)

Eddie

Re: [GIT PULL] bcache changes for 3.17

[Peter Kieser]

[-- Attachment #1: Type: text/plain, Size: 597 bytes --]


On 2014-09-05 8:37 AM, Eddie Chapman wrote:
> On 05/09/14 15:17, Jens Axboe wrote:
>> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
>> all the way back to 3.10 would be a lot of work. If there's anyone that
>> cares about bcache on stable kernels (and actually use it), now would be
>> a good time to pipe up.
>
> Just "piping up" as I care about bcache and actually use it in 
> production on 3.10! Shame I don't have the knowledge to try and 
> backport these though :-)
>
> Eddie

I'm "piping up" as well, I use bcache on 3.10 in production.

-Peter


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4291 bytes --]

Re: [GIT PULL] bcache changes for 3.17

[Arne Wiebalck]

[-- Attachment #1: Type: text/plain, Size: 979 bytes --]


On Sep 5, 2014, at 6:41 PM, Peter Kieser <peter@kieser.ca>
 wrote:

> 
> On 2014-09-05 8:37 AM, Eddie Chapman wrote:
>> On 05/09/14 15:17, Jens Axboe wrote:
>>> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
>>> all the way back to 3.10 would be a lot of work. If there's anyone that
>>> cares about bcache on stable kernels (and actually use it), now would be
>>> a good time to pipe up.
>> 
>> Just "piping up" as I care about bcache and actually use it in production on 3.10! Shame I don't have the knowledge to try and backport these though :-)
>> 
>> Eddie
> 
> I'm "piping up" as well, I use bcache on 3.10 in production.
> 
> -Peter
> 


More "piping up": we currently use bcache on a few nodes in production, on 3.14 and 3.15, and plan to roll it out on a wider scale now.
If necessary we'll go with these kernels, but we'd certainly prefer our usual 3.10-based CentOS kernel.

Cheers,
 Arne

--
Arne Wiebalck
CERN IT

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 4311 bytes --]

Re: [GIT PULL] bcache changes for 3.17

[Jens Axboe]

On 09/05/2014 11:03 AM, Arne Wiebalck wrote:
> 
> On Sep 5, 2014, at 6:41 PM, Peter Kieser <peter@kieser.ca>
>  wrote:
> 
>>
>> On 2014-09-05 8:37 AM, Eddie Chapman wrote:
>>> On 05/09/14 15:17, Jens Axboe wrote:
>>>> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
>>>> all the way back to 3.10 would be a lot of work. If there's anyone that
>>>> cares about bcache on stable kernels (and actually use it), now would be
>>>> a good time to pipe up.
>>>
>>> Just "piping up" as I care about bcache and actually use it in production on 3.10! Shame I don't have the knowledge to try and backport these though :-)
>>>
>>> Eddie
>>
>> I'm "piping up" as well, I use bcache on 3.10 in production.
>>
>> -Peter
>>
> 
> 
> More "piping up": we currently use bcache on a few nodes in production, on 3.14 and 3.15, and plan to roll it out on a wider scale now.
> If necessary we'll go with these kernels, but we'd certainly prefer our usual 3.10-based CentOS kernel.

OK, so we definitely have people using it in production. My concern was
that whomever does the backport of the appropriate patches to 3.10/14/15
stable would have an audience for getting some amount of testing of such
a patch series.

Now we just need someone to line up to do the work...

-- 
Jens Axboe

Re: [GIT PULL] bcache changes for 3.17

[Kent Overstreet]

On Fri, Sep 05, 2014 at 11:10:13AM -0600, Jens Axboe wrote:
> On 09/05/2014 11:03 AM, Arne Wiebalck wrote:
> > 
> > On Sep 5, 2014, at 6:41 PM, Peter Kieser <peter@kieser.ca>
> >  wrote:
> > 
> >>
> >> On 2014-09-05 8:37 AM, Eddie Chapman wrote:
> >>> On 05/09/14 15:17, Jens Axboe wrote:
> >>>> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
> >>>> all the way back to 3.10 would be a lot of work. If there's anyone that
> >>>> cares about bcache on stable kernels (and actually use it), now would be
> >>>> a good time to pipe up.
> >>>
> >>> Just "piping up" as I care about bcache and actually use it in production on 3.10! Shame I don't have the knowledge to try and backport these though :-)
> >>>
> >>> Eddie
> >>
> >> I'm "piping up" as well, I use bcache on 3.10 in production.
> >>
> >> -Peter
> >>
> > 
> > 
> > More "piping up": we currently use bcache on a few nodes in production, on 3.14 and 3.15, and plan to roll it out on a wider scale now.
> > If necessary we'll go with these kernels, but we'd certainly prefer our usual 3.10-based CentOS kernel.
> 
> OK, so we definitely have people using it in production. My concern was
> that whomever does the backport of the appropriate patches to 3.10/14/15
> stable would have an audience for getting some amount of testing of such
> a patch series.
> 
> Now we just need someone to line up to do the work...

I can try and make some time for backporting; if we've got people lined up for
testing that will help a lot.

Backporting fixes to 3.10 will be harder, but if memory serves there hasn't been
as much churn since 3.14 so backporting fixes to then shouldn't be too bad. If
Stefan wants to post what he's got for 3.10 though I can try and backport some
more fixes on top of that, though

Re: [GIT PULL] bcache changes for 3.17

[Stefan Priebe]

Am 05.09.2014 20:33, schrieb Kent Overstreet:
> On Fri, Sep 05, 2014 at 11:10:13AM -0600, Jens Axboe wrote:
>> On 09/05/2014 11:03 AM, Arne Wiebalck wrote:
>>>
>>> On Sep 5, 2014, at 6:41 PM, Peter Kieser <peter@kieser.ca>
>>>   wrote:
>>>
>>>>
>>>> On 2014-09-05 8:37 AM, Eddie Chapman wrote:
>>>>> On 05/09/14 15:17, Jens Axboe wrote:
>>>>>> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
>>>>>> all the way back to 3.10 would be a lot of work. If there's anyone that
>>>>>> cares about bcache on stable kernels (and actually use it), now would be
>>>>>> a good time to pipe up.
>>>>>
>>>>> Just "piping up" as I care about bcache and actually use it in production on 3.10! Shame I don't have the knowledge to try and backport these though :-)
>>>>>
>>>>> Eddie
>>>>
>>>> I'm "piping up" as well, I use bcache on 3.10 in production.
>>>>
>>>> -Peter
>>>>
>>>
>>>
>>> More "piping up": we currently use bcache on a few nodes in production, on 3.14 and 3.15, and plan to roll it out on a wider scale now.
>>> If necessary we'll go with these kernels, but we'd certainly prefer our usual 3.10-based CentOS kernel.
>>
>> OK, so we definitely have people using it in production. My concern was
>> that whomever does the backport of the appropriate patches to 3.10/14/15
>> stable would have an audience for getting some amount of testing of such
>> a patch series.
>>
>> Now we just need someone to line up to do the work...
>
> I can try and make some time for backporting; if we've got people lined up for
> testing that will help a lot.
>
> Backporting fixes to 3.10 will be harder, but if memory serves there hasn't been
> as much churn since 3.14 so backporting fixes to then shouldn't be too bad. If
> Stefan wants to post what he's got for 3.10 though I can try and backport some
> more fixes on top of that, though

You'll find them here:
https://github.com/profihost/linux-stable/commits/bcache_latest_fixes

Most probably you don't want the last four.

Stefan

> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [GIT PULL] bcache changes for 3.17

[Greg KH]

On Fri, Sep 05, 2014 at 09:31:06AM +0200, Francis Moreau wrote:
> On 08/10/2014 09:54 AM, Peter Kieser wrote:
> > 
> > On 2014-08-05 9:58 AM, Jens Axboe wrote:
> >> On 08/04/2014 10:33 PM, Kent Overstreet wrote:
> >>> Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
> >>> fixes in this one :)
> >> Normally I'd say no, but since it's basically just fixes, I guess we can
> >> pull it in. But generally, it has to be in my hands a week before this,
> >> so it can simmer a bit in for-next before going in...
> >>
> > Are these fixes going to be backported to 3.10 or other stable releases?
> > 
> 
> Could you please answer this question ?
> 
> If you don't want to maintain bcache for stable kernels (I can
> understand that), can you mark it at least as unstable/experimental
> stuff since it really is ?

WTF?

Just because a maintainer/developer doesn't want to do anything for the
stable kernel releases does _NOT_ mean the code is
"unstable/expreimental" at all.

That's not how stable kernel releases work.  _IF_ a maintainer wants to
/ has the time to, they can mark patches for inclusion in stable kernel
releases.  Given the huge list of patches that Jens just posted, I doubt
that those are really something I would ever take for a stable kernel
release.

Please read Documentation/stable_kernel_rules.txt for more details
please.  And don't ask others to do backporting work for you, it's not
ok, and is something that I have always said is never required, and is
not going to be.

thanks,

greg k-h

Re: [GIT PULL] bcache changes for 3.17

[Greg KH]

On Fri, Sep 05, 2014 at 04:44:13PM +0200, Francis Moreau wrote:
> On 09/05/2014 04:17 PM, Jens Axboe wrote:
> > 
> > We need to do something about this. From this latest pull, looks like
> > all should go to stable:
> > 
> > 5b1016e62f74c53e0330403025954c8d95384c03
> > 9aa61a992acceeec0d1de2cd99938421498659d5
> > dbd810ab678d262d3772d29b65844d7b20dc47bc
> > 8b326d3a2a76912dfed2f0ab937d59fae9512ca2
> > e5112201c1285841f8b565ece5d6ae7e0d7947a2
> > a664d0f05a2ec02c8f042db536d84d15d6e19e81
> > c5aa4a3157b55bdca18dd2a9d9f43314470b6d32
> > 9e5c353510b26500bd6b8309823ac9ef2837b761
> > bcf090e0040e30f8409e6a535a01e6473afb096f
> > 501d52a90cbe652b41336c206ff0e95799d5a9b5
> > 8e0948080670f6330229718b15a6a1a011d441ce
> > 60ae81eee86dd7a520db8c1e3d702b49fc0418b5
> > 913dc33fb2720fb5f979011664294137ddd8b13b
> > 6b708de64adb6dc8319e7aeac922b46904fbeeec
> > 400ffaa2acd72274e2c7293a9724382383bebf3e
> > d83353b319d47ef8cce82467da6a25c2d558253f
> > bf0c55c986540483c34ca640f2eef4c3314388b1
> > c9a78332b42cbdcdd386a95192a716b67d1711a4
> > 2452cc89063a2a6890368f185c4b6d7d8802175b
> > 25abade29616d42d60f9bd5e6a5ad07f7314e39e
> > 5b25abade29616d42d60f9bd5e6a5ad07f7314e3
> > 789d21dbd9d8889e62c79ec19585fcc97e42ef07
> > 0781c8748cf1ea2b0dcd966571103909528c4efa
> > 
> > (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
> > all the way back to 3.10 would be a lot of work. If there's anyone that
> > cares about bcache on stable kernels (and actually use it), now would be
> > a good time to pipe up.
> > 
> 
> Then if it's too much work, it just confirmed what was asked previously:
> bcache is still experimental so mark it such for stable kernels.

You seem to have a misunderstanding of just exactly what the stable
kernels are, and what they are for.  See my other email for details.

greg k-h

Re: [GIT PULL] bcache changes for 3.17

[Jens Axboe]

On 09/05/2014 03:45 PM, Greg KH wrote:
> On Fri, Sep 05, 2014 at 09:31:06AM +0200, Francis Moreau wrote:
>> On 08/10/2014 09:54 AM, Peter Kieser wrote:
>>>
>>> On 2014-08-05 9:58 AM, Jens Axboe wrote:
>>>> On 08/04/2014 10:33 PM, Kent Overstreet wrote:
>>>>> Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
>>>>> fixes in this one :)
>>>> Normally I'd say no, but since it's basically just fixes, I guess we can
>>>> pull it in. But generally, it has to be in my hands a week before this,
>>>> so it can simmer a bit in for-next before going in...
>>>>
>>> Are these fixes going to be backported to 3.10 or other stable releases?
>>>
>>
>> Could you please answer this question ?
>>
>> If you don't want to maintain bcache for stable kernels (I can
>> understand that), can you mark it at least as unstable/experimental
>> stuff since it really is ?
> 
> WTF?
> 
> Just because a maintainer/developer doesn't want to do anything for the
> stable kernel releases does _NOT_ mean the code is
> "unstable/expreimental" at all.

That's not what he is saying at all. The code IS unstable in 3.10. And
the fact that nothing goes to stable for bcache, the situation wasn't
likely to change for 3.10. Nobody is saying "Oh nothing goes to stable,
lets mark it experimental".

> That's not how stable kernel releases work.  _IF_ a maintainer wants to
> / has the time to, they can mark patches for inclusion in stable kernel
> releases.  Given the huge list of patches that Jens just posted, I doubt
> that those are really something I would ever take for a stable kernel
> release.

Actually, all of those are pretty much stable material, since they fix
actual bugs that people hit. Which is the definition of what should go
to stable.

-- 
Jens Axboe

Re: [GIT PULL] bcache changes for 3.17

[Peter Kieser]

[-- Attachment #1: Type: text/plain, Size: 420 bytes --]


On 2014-09-05 2:45 PM, Greg KH wrote:
> Just because a maintainer/developer doesn't want to do anything for the
> stable kernel releases does_NOT_  mean the code is
> "unstable/expreimental" at all.

These are more bcache-ate-my-data unstable bugs. It's standard practice 
to backport fixes that cause instability/data corruption to a 'stable' 
release (otherwise, why would it be named 'stable')?

-Peter


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4291 bytes --]

Re: [GIT PULL] bcache changes for 3.17

[Chuck Ebbert]

On Fri, 5 Sep 2014 17:03:57 +0000
Arne Wiebalck <Arne.Wiebalck@cern.ch> wrote:

> 
> On Sep 5, 2014, at 6:41 PM, Peter Kieser <peter@kieser.ca>
>  wrote:
> 
> > 
> > On 2014-09-05 8:37 AM, Eddie Chapman wrote:
> >> On 05/09/14 15:17, Jens Axboe wrote:
> >>> (from oldest to newest). And that's just from 3.16 to 3.17-rc3,
> >>> going all the way back to 3.10 would be a lot of work. If there's
> >>> anyone that cares about bcache on stable kernels (and actually
> >>> use it), now would be a good time to pipe up.
> >> 
> >> Just "piping up" as I care about bcache and actually use it in
> >> production on 3.10! Shame I don't have the knowledge to try and
> >> backport these though :-)
> >> 
> >> Eddie
> > 
> > I'm "piping up" as well, I use bcache on 3.10 in production.
> > 
> > -Peter
> > 
> 
> 
> More "piping up": we currently use bcache on a few nodes in
> production, on 3.14 and 3.15, and plan to roll it out on a wider
> scale now. If necessary we'll go with these kernels, but we'd
> certainly prefer our usual 3.10-based CentOS kernel.
> 

CentOS does not ship bcache at all. In retrospect it's obvious why.

Re: [GIT PULL] bcache changes for 3.17

[Francis Moreau]

On 09/05/2014 11:45 PM, Greg KH wrote:
> On Fri, Sep 05, 2014 at 09:31:06AM +0200, Francis Moreau wrote:
>> On 08/10/2014 09:54 AM, Peter Kieser wrote:
>>>
>>> On 2014-08-05 9:58 AM, Jens Axboe wrote:
>>>> On 08/04/2014 10:33 PM, Kent Overstreet wrote:
>>>>> Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
>>>>> fixes in this one :)
>>>> Normally I'd say no, but since it's basically just fixes, I guess we can
>>>> pull it in. But generally, it has to be in my hands a week before this,
>>>> so it can simmer a bit in for-next before going in...
>>>>
>>> Are these fixes going to be backported to 3.10 or other stable releases?
>>>
>>
>> Could you please answer this question ?
>>
>> If you don't want to maintain bcache for stable kernels (I can
>> understand that), can you mark it at least as unstable/experimental
>> stuff since it really is ?
> 
> WTF?
> 
> Just because a maintainer/developer doesn't want to do anything for the
> stable kernel releases does _NOT_ mean the code is
> "unstable/expreimental" at all.
> 
> That's not how stable kernel releases work.  _IF_ a maintainer wants to
> / has the time to, they can mark patches for inclusion in stable kernel
> releases.  Given the huge list of patches that Jens just posted, I doubt
> that those are really something I would ever take for a stable kernel
> release.
> 
> Please read Documentation/stable_kernel_rules.txt for more details
> please.  And don't ask others to do backporting work for you, it's not
> ok, and is something that I have always said is never required, and is
> not going to be.
> 

wow, not sure why I deserve such anger...

Looks like you haven't understood me well and specially I *never* asked
others to do the backporting for me.

Please reread the thread, perhaps peaceful music can help too.

Re: [GIT PULL] bcache changes for 3.17

[Greg KH]

On Fri, Sep 05, 2014 at 04:21:48PM -0600, Jens Axboe wrote:
> On 09/05/2014 03:45 PM, Greg KH wrote:
> > On Fri, Sep 05, 2014 at 09:31:06AM +0200, Francis Moreau wrote:
> >> On 08/10/2014 09:54 AM, Peter Kieser wrote:
> >>>
> >>> On 2014-08-05 9:58 AM, Jens Axboe wrote:
> >>>> On 08/04/2014 10:33 PM, Kent Overstreet wrote:
> >>>>> Hey Jens, here's the pull request for 3.17 - typically late, but lots of tasty
> >>>>> fixes in this one :)
> >>>> Normally I'd say no, but since it's basically just fixes, I guess we can
> >>>> pull it in. But generally, it has to be in my hands a week before this,
> >>>> so it can simmer a bit in for-next before going in...
> >>>>
> >>> Are these fixes going to be backported to 3.10 or other stable releases?
> >>>
> >>
> >> Could you please answer this question ?
> >>
> >> If you don't want to maintain bcache for stable kernels (I can
> >> understand that), can you mark it at least as unstable/experimental
> >> stuff since it really is ?
> > 
> > WTF?
> > 
> > Just because a maintainer/developer doesn't want to do anything for the
> > stable kernel releases does _NOT_ mean the code is
> > "unstable/expreimental" at all.
> 
> That's not what he is saying at all. The code IS unstable in 3.10. And
> the fact that nothing goes to stable for bcache, the situation wasn't
> likely to change for 3.10. Nobody is saying "Oh nothing goes to stable,
> lets mark it experimental".

Sorry, but with only the context above which is what I was sent, you can
see how I can be confused here...

greg k-h

Re: [GIT PULL] bcache changes for 3.17

[Greg KH]

On Fri, Sep 05, 2014 at 04:17:55PM -0700, Peter Kieser wrote:
> 
> On 2014-09-05 2:45 PM, Greg KH wrote:
> >Just because a maintainer/developer doesn't want to do anything for the
> >stable kernel releases does_NOT_  mean the code is
> >"unstable/expreimental" at all.
> 
> These are more bcache-ate-my-data unstable bugs. It's standard practice to
> backport fixes that cause instability/data corruption to a 'stable' release
> (otherwise, why would it be named 'stable')?

That's fine, but it has nothing to do with what sounded like someone
wanting to go back and mark an older kernel feature as "unsupported".

greg k-h

Re: [GIT PULL] bcache changes for 3.17

[Francis Moreau]

On 09/05/2014 07:10 PM, Jens Axboe wrote:
> On 09/05/2014 11:03 AM, Arne Wiebalck wrote:
>>
>> On Sep 5, 2014, at 6:41 PM, Peter Kieser <peter@kieser.ca>
>>  wrote:
>>
>>>
>>> On 2014-09-05 8:37 AM, Eddie Chapman wrote:
>>>> On 05/09/14 15:17, Jens Axboe wrote:
>>>>> (from oldest to newest). And that's just from 3.16 to 3.17-rc3, going
>>>>> all the way back to 3.10 would be a lot of work. If there's anyone that
>>>>> cares about bcache on stable kernels (and actually use it), now would be
>>>>> a good time to pipe up.
>>>>
>>>> Just "piping up" as I care about bcache and actually use it in production on 3.10! Shame I don't have the knowledge to try and backport these though :-)
>>>>
>>>> Eddie
>>>
>>> I'm "piping up" as well, I use bcache on 3.10 in production.
>>>
>>> -Peter
>>>
>>
>>
>> More "piping up": we currently use bcache on a few nodes in production, on 3.14 and 3.15, and plan to roll it out on a wider scale now.
>> If necessary we'll go with these kernels, but we'd certainly prefer our usual 3.10-based CentOS kernel.
> 
> OK, so we definitely have people using it in production. My concern was
> that whomever does the backport of the appropriate patches to 3.10/14/15
> stable would have an audience for getting some amount of testing of such
> a patch series.
> 
> Now we just need someone to line up to do the work...
> 

Ok it's becoming insane: my system crashes every 2 days: any processes
that attempt a write to the disk get stuck, and cpu are at 100%.

So I can try to backport the fixes that address the following oops for
kernel 3.14 but someone has to point me the corresponding commits since
I don't know bcache.

Thanks.

BUG: soft lockup - CPU#0 stuck for 22s! [bcache_gc:152]
Modules linked in: tun xt_nat xt_tcpudp mmc_block btrfs raid6_pq xor ses
enclosure usb_storage veth xt_addrtype xt_conntrack ipt_MASQUERADE
iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat
nf_conntrack bridge stp llc dm_thin_pool dm_persistent_data
dm_bio_prison dm_bufio libcrc32c loop dm_mod iptable_filter ip_tables
x_tables hid_generic usbhid hid ctr ccm fuse joydev mousedev coretemp
hwmon arc4 iwldvm led_class nls_iso8859_1 nls_cp437 vfat mac80211 fat
intel_rapl x86_pkg_temp_thermal iTCO_wdt intel_powerclamp
iTCO_vendor_support kvm_intel snd_hda_codec_hdmi kvm snd_hda_codec_via
snd_hda_codec_generic crct10dif_pclmul iwlwifi crc32_pclmul crc32c_intel
btusb ghash_clmulni_intel bluetooth aesni_intel aes_x86_64 cfg80211 lrw
snd_hda_intel gf128mul glue_helper ablk_helper
 6lowpan_iphc cryptd r8169 snd_hda_codec psmouse rtsx_pci_ms i2c_i801
snd_hwdep serio_raw rfkill memstick mii snd_pcm wmi snd_timer snd evdev
tpm_infineon mei_me tpm_tis mei tpm soundcore shpchp mac_hid lpc_ich
battery ac processor thermal sch_fq_codel nfs lockd sunrpc fscache ext4
crc16 mbcache jbd2 bcache sd_mod sr_mod crc_t10dif cdrom
crct10dif_common rtsx_pci_sdmmc mmc_core atkbd libps2 ahci libahci
libata ehci_pci xhci_hcd ehci_hcd scsi_mod rtsx_pci usbcore usb_common
i8042 serio i915 video button intel_gtt i2c_algo_bit drm_kms_helper drm
i2c_core
CPU: 0 PID: 152 Comm: bcache_gc Not tainted 3.14.30-1-lts #1
Hardware name: CLEVO CO.                        W55xEU
        /W55xEU                          , BIOS 4.6.5 03/05/2013
task: ffff880406b1a780 ti: ffff88040461e000 task.ti: ffff88040461e000
RIP: 0010:[<ffffffffa0443af2>]  [<ffffffffa0443af2>]
bch_extent_bad+0x122/0x1d0 [bcache]
RSP: 0018:ffff88040461fa90  EFLAGS: 00000207
RAX: 9000000000800001 RBX: ffffffffa04439b9 RCX: ffffc90017452000
RDX: ffffc90017468f38 RSI: 000000007a6b5813 RDI: ffff88007ff20000
RBP: ffff88040461fac0 R08: 0000000000000013 R09: 0000000000000008
R10: 000007ffffffffff R11: ffff880405fe8000 R12: ffff8804055b08a0
R13: ffff8804055b08a0 R14: ffff880404844760 R15: 0000000000000018
FS:  0000000000000000(0000) GS:ffff88041e200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1b36926007 CR3: 000000000280c000 CR4: 00000000001427e0
Stack:
 ffff88040461faa0 ffff880404844760 ffff88040461fc48 ffffffffa043ba80
 ffff8804055b08a0 ffff880405e2dc60 ffff88040461fad0 ffffffffa043ba8a
 ffff88040461fb00 ffffffffa043b879 00000000000008e8 ffff8804055b08a0
Call Trace:
 [<ffffffffa043ba80>] ? bch_ptr_invalid+0x10/0x10 [bcache]
 [<ffffffffa043ba8a>] bch_ptr_bad+0xa/0x10 [bcache]
 [<ffffffffa043b879>] bch_btree_iter_next_filter+0x29/0x50 [bcache]
 [<ffffffffa04409f5>] btree_gc_recurse+0x175/0xc10 [bcache]
 [<ffffffffa043ba70>] ? bch_btree_keys_stats+0xf0/0xf0 [bcache]
 [<ffffffffa0444a85>] ? __bch_btree_ptr_invalid+0xa5/0xc0 [bcache]
 [<ffffffffa043ba70>] ? bch_btree_keys_stats+0xf0/0xf0 [bcache]
 [<ffffffffa043efc3>] ? btree_gc_mark_node+0x73/0x230 [bcache]
 [<ffffffffa0441bbf>] bch_btree_gc+0x50f/0x690 [bcache]
 [<ffffffff8109f59c>] ? try_to_wake_up+0x20c/0x2d0
 [<ffffffff810b23d0>] ? __wake_up_sync+0x20/0x20
 [<ffffffffa0441d88>] bch_gc_thread+0x48/0x130 [bcache]
 [<ffffffffa0441d40>] ? bch_btree_gc+0x690/0x690 [bcache]
 [<ffffffff8108e3aa>] kthread+0xea/0x100
 [<ffffffff8108e2c0>] ? kthread_create_on_node+0x1a0/0x1a0
 [<ffffffff8150e0bc>] ret_from_fork+0x7c/0xb0
 [<ffffffff8108e2c0>] ? kthread_create_on_node+0x1a0/0x1a0
Code: 00 00 4c 8b 84 d7 40 0c 00 00 48 89 f2 48 c1 ea 08 4c 21 fa 48 d3
ea 49 8b 88 00 0b 00 00 48 8d 14 52 48 8d 14 91 44 0f b6 42 06 <41> 29
f0 41 80 f8 80 77 75 41 80 f8 60 76 29 0f b6 8f 6e 0e 00
BUG: soft lockup - CPU#0 stuck for 23s! [bcache_gc:152]
Modules linked in: tun xt_nat xt_tcpudp mmc_block btrfs raid6_pq xor ses
enclosure usb_storage veth xt_addrtype xt_conntrack ipt_MASQUERADE
iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat
nf_conntrack bridge stp llc dm_thin_pool dm_persistent_data
dm_bio_prison dm_bufio libcrc32c loop dm_mod iptable_filter ip_tables
x_tables hid_generic usbhid hid ctr ccm fuse joydev mousedev coretemp
hwmon arc4 iwldvm led_class nls_iso8859_1 nls_cp437 vfat mac80211 fat
intel_rapl x86_pkg_temp_thermal iTCO_wdt intel_powerclamp
iTCO_vendor_support kvm_intel snd_hda_codec_hdmi kvm snd_hda_codec_via
snd_hda_codec_generic crct10dif_pclmul iwlwifi crc32_pclmul crc32c_intel
btusb ghash_clmulni_intel bluetooth aesni_intel aes_x86_64 cfg80211 lrw
snd_hda_intel gf128mul glue_helper ablk_helper
 6lowpan_iphc cryptd r8169 snd_hda_codec psmouse rtsx_pci_ms i2c_i801
snd_hwdep serio_raw rfkill memstick mii snd_pcm wmi snd_timer snd evdev
tpm_infineon mei_me tpm_tis mei tpm soundcore shpchp mac_hid lpc_ich
battery ac processor thermal sch_fq_codel nfs lockd sunrpc fscache ext4
crc16 mbcache jbd2 bcache sd_mod sr_mod crc_t10dif cdrom
crct10dif_common rtsx_pci_sdmmc mmc_core atkbd libps2 ahci libahci
libata ehci_pci xhci_hcd ehci_hcd scsi_mod rtsx_pci usbcore usb_common
i8042 serio i915 video button intel_gtt i2c_algo_bit drm_kms_helper drm
i2c_core
CPU: 0 PID: 152 Comm: bcache_gc Not tainted 3.14.30-1-lts #1
Hardware name: CLEVO CO.                        W55xEU
        /W55xEU                          , BIOS 4.6.5 03/05/2013
task: ffff880406b1a780 ti: ffff88040461e000 task.ti: ffff88040461e000
RIP: 0010:[<ffffffffa044394a>]  [<ffffffffa044394a>]
bch_extent_invalid+0x3a/0xc0 [bcache]
RSP: 0018:ffff88040461fa18  EFLAGS: 00000283
RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000010
RDX: 0000000000054b68 RSI: ffff8804048482e8 RDI: ffff8804055b08a0
RBP: ffff88040461fa80 R08: ffff88040461fc58 R09: ffff880404862820
R10: ffff880404848300 R11: ffff880405fe8000 R12: 000007ffffffffff
R13: ffff880405fe8000 R14: 0000000000000001 R15: ffff88040461fa08
FS:  0000000000000000(0000) GS:ffff88041e200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f1b36926007 CR3: 000000000280c000 CR4: 00000000001427e0
Stack:
 ffffffffa0444a85 000007ffffffffff ffff88040461fad0 0000000000000001
 ffff88040461fa58 ffffffffa0438e9f ffff880405b10004 0000000000000001
 ffff88040461fab8 ffffffffa043a681 00000000a5765a18 ffff8804048482e8
Call Trace:
 [<ffffffffa0444a85>] ? __bch_btree_ptr_invalid+0xa5/0xc0 [bcache]
 [<ffffffffa0438e9f>] ? tree_to_bkey+0x1f/0x50 [bcache]
 [<ffffffffa043a681>] ? __bch_bset_search+0x1e1/0x4c0 [bcache]
 [<ffffffffa0443a13>] bch_extent_bad+0x43/0x1d0 [bcache]
 [<ffffffffa043ba80>] ? bch_ptr_invalid+0x10/0x10 [bcache]
 [<ffffffffa043ba8a>] bch_ptr_bad+0xa/0x10 [bcache]
 [<ffffffffa043b879>] bch_btree_iter_next_filter+0x29/0x50 [bcache]
 [<ffffffffa04409f5>] btree_gc_recurse+0x175/0xc10 [bcache]
 [<ffffffffa043ba70>] ? bch_btree_keys_stats+0xf0/0xf0 [bcache]
 [<ffffffffa0444a85>] ? __bch_btree_ptr_invalid+0xa5/0xc0 [bcache]
 [<ffffffffa043ba70>] ? bch_btree_keys_stats+0xf0/0xf0 [bcache]
 [<ffffffffa043efc3>] ? btree_gc_mark_node+0x73/0x230 [bcache]
 [<ffffffffa0441bbf>] bch_btree_gc+0x50f/0x690 [bcache]
 [<ffffffff8109f59c>] ? try_to_wake_up+0x20c/0x2d0
 [<ffffffff810b23d0>] ? __wake_up_sync+0x20/0x20
 [<ffffffffa0441d88>] bch_gc_thread+0x48/0x130 [bcache]
 [<ffffffffa0441d40>] ? bch_btree_gc+0x690/0x690 [bcache]
 [<ffffffff8108e3aa>] kthread+0xea/0x100
 [<ffffffff8108e2c0>] ? kthread_create_on_node+0x1a0/0x1a0
 [<ffffffff8150e0bc>] ret_from_fork+0x7c/0xb0
 [<ffffffff8108e2c0>] ? kthread_create_on_node+0x1a0/0x1a0

...