3.18.1 + latest bcache-dev

3.18.1 + latest bcache-dev

[Stephen R. van den Berg]

Now I tried this:
Combine 3.18.1 and rebase all the patches from bcache-dev onto that.
But, in trying that, I get a consistent:

register_bcache() error opening /dev/sdb3: Invalid superblock: member info area missing

Anything I overlooked?   The bcache-tools are straight from git too.
-- 
Stephen.

Re: 3.18.1 + latest bcache-dev

[Slava Pestov]

Hi Stephen,

The bcache-dev branch has on-disk format changes that require a new
bcache-tools. I'm not sure where Kent posts the source for that.

On Sat, Dec 27, 2014 at 6:35 PM, Stephen R. van den Berg <srb@cuci.nl> wrote:
> Now I tried this:
> Combine 3.18.1 and rebase all the patches from bcache-dev onto that.
> But, in trying that, I get a consistent:
>
> register_bcache() error opening /dev/sdb3: Invalid superblock: member info area missing
>
> Anything I overlooked?   The bcache-tools are straight from git too.
> --
> Stephen.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 3.18.1 + latest bcache-dev

[Rolf Fokkens]

Hi Slava,

Sounds like the kernel (using bcache-dev) is not able to handle the 
current format. If so, bcache-tools may help in "reformatting" bcache 
devices, but that's no solution for existing systems.

So is my understanding correct that there's no easy kernel upgrade path 
(to bcache-dev) ?

Rolf

On 12/29/2014 05:05 AM, Slava Pestov wrote:
> Hi Stephen,
>
> The bcache-dev branch has on-disk format changes that require a new
> bcache-tools. I'm not sure where Kent posts the source for that.
>
> On Sat, Dec 27, 2014 at 6:35 PM, Stephen R. van den Berg <srb@cuci.nl> wrote:
>> Now I tried this:
>> Combine 3.18.1 and rebase all the patches from bcache-dev onto that.
>> But, in trying that, I get a consistent:
>>
>> register_bcache() error opening /dev/sdb3: Invalid superblock: member info area missing
>>
>> Anything I overlooked?   The bcache-tools are straight from git too.
>> --
>> Stephen.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 3.18.1 + latest bcache-dev

[Kent Overstreet]

On Sat, Jan 03, 2015 at 05:40:24PM +0100, Rolf Fokkens wrote:
> Hi Slava,
> 
> Sounds like the kernel (using bcache-dev) is not able to handle the current
> format. If so, bcache-tools may help in "reformatting" bcache devices, but
> that's no solution for existing systems.
> 
> So is my understanding correct that there's no easy kernel upgrade path (to
> bcache-dev) ?

Yes, correct. The on disk format changes in bcache-dev are just too deep to
feasibly write backwards compat code, unfortunately.

And the on disk format is still in flux - for just a bit longer hopefully
though, right now I'm working on a pretty major revamp that's getting close to
done (self describing and packed metadata). This revamp is making the on disk
format _much_ more flexible though, and cleaning up a lot of stuff at the same
time.

Re: 3.18.1 + latest bcache-dev

[Rolf Fokkens]

Please let us know when this will be pushed to the kernel, do you have 
any thoughts on the planning of this?

As a bcache-tools packager for Fedora I think I should add some package 
dependencies which prevent users from accidentally upgrading to that 
(breaking) version in the future.

It may also affect other tools like util-linux: 
https://github.com/karelzak/util-linux/blob/master/libblkid/src/superblocks/bcache.c

Furthermore I think I should postpone integration of bcache in the 
Fedora installer until further notice: 
https://bugzilla.redhat.com/show_bug.cgi?id=1003208

Any information in advance is appreciated.

Rolf

On 01/04/2015 01:46 AM, Kent Overstreet wrote:
> Yes, correct. The on disk format changes in bcache-dev are just too 
> deep to feasibly write backwards compat code, unfortunately. And the 
> on disk format is still in flux - for just a bit longer hopefully 
> though, right now I'm working on a pretty major revamp that's getting 
> close to done (self describing and packed metadata). This revamp is 
> making the on disk format _much_ more flexible though, and cleaning up 
> a lot of stuff at the same time. 

Re: 3.18.1 + latest bcache-dev

[Slava Pestov]

On Sun, Jan 4, 2015 at 6:11 AM, Rolf Fokkens <rolf@rolffokkens.nl> wrote:
> Please let us know when this will be pushed to the kernel, do you have any
> thoughts on the planning of this?
>
> As a bcache-tools packager for Fedora I think I should add some package
> dependencies which prevent users from accidentally upgrading to that
> (breaking) version in the future.

The plan is to incrementally backport bug fixes and optimizations from
bcache-dev to upstream for the foreseeable future.

The development branch is going through major changes to support
dynamically adding/removing cache devices and storing data in the
btree instead of using it as a cache, enabling usage without any
backing devices. We're actively working on this code and doing a lot
of stress testing of both the new stuff and backing device support.
However, it's its too early to tell what the new features will look
like by the time they're ready to go upstream, or what a transition
plan will look like for existing installs.

I wish we had more time for upstreaming stuff, but I can assure you
its not Kent's intent to just dump bcache-dev as one huge pull request
:-)

>
> It may also affect other tools like util-linux:
> https://github.com/karelzak/util-linux/blob/master/libblkid/src/superblocks/bcache.c
>
> Furthermore I think I should postpone integration of bcache in the Fedora
> installer until further notice:
> https://bugzilla.redhat.com/show_bug.cgi?id=1003208
>
> Any information in advance is appreciated.
>
> Rolf
>
>
> On 01/04/2015 01:46 AM, Kent Overstreet wrote:
>>
>> Yes, correct. The on disk format changes in bcache-dev are just too deep
>> to feasibly write backwards compat code, unfortunately. And the on disk
>> format is still in flux - for just a bit longer hopefully though, right now
>> I'm working on a pretty major revamp that's getting close to done (self
>> describing and packed metadata). This revamp is making the on disk format
>> _much_ more flexible though, and cleaning up a lot of stuff at the same
>> time.
>
>

Re: 3.18.1 + latest bcache-dev

[Stephen R. van den Berg]

Kent Overstreet wrote:
>Yes, correct. The on disk format changes in bcache-dev are just too deep to
>feasibly write backwards compat code, unfortunately.

>And the on disk format is still in flux - for just a bit longer hopefully
>though, right now I'm working on a pretty major revamp that's getting close to

Any chance we could get a git url of the bcache-tools-in-flux as well?
-- 
Stephen.

Re: 3.18.1 + latest bcache-dev

[Kent Overstreet]

On Mon, Jan 05, 2015 at 09:49:19AM +0100, Stephen R. van den Berg wrote:
> Kent Overstreet wrote:
> >Yes, correct. The on disk format changes in bcache-dev are just too deep to
> >feasibly write backwards compat code, unfortunately.
> 
> >And the on disk format is still in flux - for just a bit longer hopefully
> >though, right now I'm working on a pretty major revamp that's getting close to
> 
> Any chance we could get a git url of the bcache-tools-in-flux as well?

Yeah, I just pushed the current (very much WIP) stuff to the bcache-dev branch
under the main bcache-tools repo

Re: 3.18.1 + latest bcache-dev

[Stephen R. van den Berg]

Kent Overstreet wrote:
>On Mon, Jan 05, 2015 at 09:49:19AM +0100, Stephen R. van den Berg wrote:
>> Kent Overstreet wrote:
>> >Yes, correct. The on disk format changes in bcache-dev are just too deep to
>> >feasibly write backwards compat code, unfortunately.

>> >And the on disk format is still in flux - for just a bit longer hopefully
>> >though, right now I'm working on a pretty major revamp that's getting close to

>> Any chance we could get a git url of the bcache-tools-in-flux as well?

>Yeah, I just pushed the current (very much WIP) stuff to the bcache-dev branch
>under the main bcache-tools repo

Maybe I'm not looking in the right spot, but I do not see that branch.
What is the main bcache-tools repo?
-- 
Stephen.

Re: 3.18.1 + latest bcache-dev

[Kent Overstreet]

http://evilpiepirate.org/git/bcache-tools.git

On Tue, Jan 6, 2015 at 1:16 AM, Stephen R. van den Berg <srb@cuci.nl> wrote:
> Kent Overstreet wrote:
>>On Mon, Jan 05, 2015 at 09:49:19AM +0100, Stephen R. van den Berg wrote:
>>> Kent Overstreet wrote:
>>> >Yes, correct. The on disk format changes in bcache-dev are just too deep to
>>> >feasibly write backwards compat code, unfortunately.
>
>>> >And the on disk format is still in flux - for just a bit longer hopefully
>>> >though, right now I'm working on a pretty major revamp that's getting close to
>
>>> Any chance we could get a git url of the bcache-tools-in-flux as well?
>
>>Yeah, I just pushed the current (very much WIP) stuff to the bcache-dev branch
>>under the main bcache-tools repo
>
> Maybe I'm not looking in the right spot, but I do not see that branch.
> What is the main bcache-tools repo?
> --
> Stephen.

Re: 3.18.1 + latest bcache-dev

[Rolf Fokkens]

On 01/05/2015 08:47 AM, Slava Pestov wrote:
> The plan is to incrementally backport bug fixes and optimizations from 
> bcache-dev to upstream for the foreseeable future.
I assume (hope) this won't break bcache w.r.t. the current block device 
layout?
> The development branch is going through major changes to support 
> dynamically adding/removing cache devices
Sound nices, that creates great flexibility. Does this also introduce 
the option of storing mirrored writeback data, while storing single copy 
read (cached) data (when having multiple SSD's as a cache)?
> and storing data in the btree instead of using it as a cache, enabling 
> usage without any backing devices.
The needs some explanation. I assume you mean it operates as writeback 
bache that never flushes. If that's correct, I can only imagine it's 
useful when initially using it. And in that case seems a kind of a time 
bomb, because when it's "full" (and there's still no backing device)... 
what happens then?
> We're actively working on this code and doing a lot of stress testing 
> of both the new stuff and backing device support. However, it's its 
> too early to tell what the new features will look like by the time 
> they're ready to go upstream, or what a transition plan will look like 
> for existing installs. I wish we had more time for upstreaming stuff, 
> but I can assure you its not Kent's intent to just dump bcache-dev as 
> one huge pull request :-) 
As an ethousiastic bcache user myself I would be very happy if there's a 
transition plan! :-)

Out of curiosity, will the backing device layout in general change, or 
will specifically the superblock change? If it's 'only' the superblock 
(and not it's size) I can imagine a feasable migration.

Rolf

Re: 3.18.1 + latest bcache-dev

[Slava Pestov]

On Tue, Jan 6, 2015 at 2:18 PM, Rolf Fokkens <rolf@rolffokkens.nl> wrote:
> On 01/05/2015 08:47 AM, Slava Pestov wrote:
>>
>> The plan is to incrementally backport bug fixes and optimizations from
>> bcache-dev to upstream for the foreseeable future.
>
> I assume (hope) this won't break bcache w.r.t. the current block device
> layout?

Correct.

Slava

Re: 3.18.1 + latest bcache-dev

[Kent Overstreet]

On Tue, Jan 06, 2015 at 11:18:10PM +0100, Rolf Fokkens wrote:
> On 01/05/2015 08:47 AM, Slava Pestov wrote:
> >The plan is to incrementally backport bug fixes and optimizations from
> >bcache-dev to upstream for the foreseeable future.
> I assume (hope) this won't break bcache w.r.t. the current block device
> layout?
> >The development branch is going through major changes to support
> >dynamically adding/removing cache devices
> Sound nices, that creates great flexibility. Does this also introduce the
> option of storing mirrored writeback data, while storing single copy read
> (cached) data (when having multiple SSD's as a cache)?

Yup, and a lot more.

> >and storing data in the btree instead of using it as a cache, enabling
> >usage without any backing devices.
> The needs some explanation. I assume you mean it operates as writeback bache
> that never flushes. If that's correct, I can only imagine it's useful when
> initially using it. And in that case seems a kind of a time bomb, because
> when it's "full" (and there's still no backing device)... what happens then?

What Slava is alluding to is that bcache is becoming a full posix filesystem - I
was actually demoing it... nearly a year ago, I think. With eventual feature
parity with btrfs and much, much better performance.

None of the existing functionality is going away either, it'll just be faster
and more robust.

> >We're actively working on this code and doing a lot of stress testing of
> >both the new stuff and backing device support. However, it's its too early
> >to tell what the new features will look like by the time they're ready to
> >go upstream, or what a transition plan will look like for existing
> >installs. I wish we had more time for upstreaming stuff, but I can assure
> >you its not Kent's intent to just dump bcache-dev as one huge pull request
> >:-)
> As an ethousiastic bcache user myself I would be very happy if there's a
> transition plan! :-)

The exact transition plan is really up in the air, but for just caching backing
devices you'll always be able to detach the cache set, reformat the cache set
while using your backing device in passthrough mode, and then reattach.

Re: 3.18.1 + latest bcache-dev

[Jianjian Huo]

Hi Kent,

On Tue, Jan 6, 2015 at 5:48 PM, Kent Overstreet <kmo@daterainc.com> wrote:
>
> On Tue, Jan 06, 2015 at 11:18:10PM +0100, Rolf Fokkens wrote:
> > On 01/05/2015 08:47 AM, Slava Pestov wrote:
> > >The plan is to incrementally backport bug fixes and optimizations from
> > >bcache-dev to upstream for the foreseeable future.
> > I assume (hope) this won't break bcache w.r.t. the current block device
> > layout?
> > >The development branch is going through major changes to support
> > >dynamically adding/removing cache devices
> > Sound nices, that creates great flexibility. Does this also introduce the
> > option of storing mirrored writeback data, while storing single copy read
> > (cached) data (when having multiple SSD's as a cache)?
>
> Yup, and a lot more.
>
> > >and storing data in the btree instead of using it as a cache, enabling
> > >usage without any backing devices.
> > The needs some explanation. I assume you mean it operates as writeback bache
> > that never flushes. If that's correct, I can only imagine it's useful when
> > initially using it. And in that case seems a kind of a time bomb, because
> > when it's "full" (and there's still no backing device)... what happens then?
>
> What Slava is alluding to is that bcache is becoming a full posix filesystem - I
> was actually demoing it... nearly a year ago, I think. With eventual feature
> parity with btrfs and much, much better performance.
>

btrfs use COW b-tree shadowing to support snapshots and clones, what
approach will this new bcache FS use to support those features?

And will this new FS continue to use 2MB buckets without random writes
in them? if that's the case, when this new FS is used on SSDs only, I
guess FS GC has to be turned on for all buckets to reclaim free space.
Since a lot of SSDs use parallel writes, some use compression, it's
not possible to align one logical 2MB bucket at FS level to one single
physical NAND block, then GC will be performed twice for one dirty
bucket at different levels, so SSD life will be cut in half in the
worst case?

> None of the existing functionality is going away either, it'll just be faster
> and more robust.
>
> > >We're actively working on this code and doing a lot of stress testing of
> > >both the new stuff and backing device support. However, it's its too early
> > >to tell what the new features will look like by the time they're ready to
> > >go upstream, or what a transition plan will look like for existing
> > >installs. I wish we had more time for upstreaming stuff, but I can assure
> > >you its not Kent's intent to just dump bcache-dev as one huge pull request
> > >:-)
> > As an ethousiastic bcache user myself I would be very happy if there's a
> > transition plan! :-)
>
> The exact transition plan is really up in the air, but for just caching backing
> devices you'll always be able to detach the cache set, reformat the cache set
> while using your backing device in passthrough mode, and then reattach.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Jianjian

Re: 3.18.1 + latest bcache-dev

[Eric Wheeler]

On Tue, 6 Jan 2015, Rolf Fokkens wrote:

> On 01/05/2015 08:47 AM, Slava Pestov wrote:
> > The plan is to incrementally backport bug fixes and optimizations from
> > bcache-dev to upstream for the foreseeable future.
> As an ethousiastic bcache user myself I would be very happy if there's a
> transition plan! :-)

Would it make sense to call this bcache2 and merge both into the linux 
tree once bcache2 is stable enough for broader testing?

To the extent possible, you could reuse common code paths and simplify the 
backporting of bug fixes into the existing stable branch.

-Eric
 
> Out of curiosity, will the backing device layout in general change, or will
> specifically the superblock change? If it's 'only' the superblock (and not
> it's size) I can imagine a feasable migration.
> 
> Rolf
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: 3.18.1 + latest bcache-dev

[Kent Overstreet]

On Wed, Jan 07, 2015 at 09:41:19AM -0800, Jianjian Huo wrote:
> Hi Kent,
> 
> On Tue, Jan 6, 2015 at 5:48 PM, Kent Overstreet <kmo@daterainc.com> wrote:
> >
> > On Tue, Jan 06, 2015 at 11:18:10PM +0100, Rolf Fokkens wrote:
> > > On 01/05/2015 08:47 AM, Slava Pestov wrote:
> > > >The plan is to incrementally backport bug fixes and optimizations from
> > > >bcache-dev to upstream for the foreseeable future.
> > > I assume (hope) this won't break bcache w.r.t. the current block device
> > > layout?
> > > >The development branch is going through major changes to support
> > > >dynamically adding/removing cache devices
> > > Sound nices, that creates great flexibility. Does this also introduce the
> > > option of storing mirrored writeback data, while storing single copy read
> > > (cached) data (when having multiple SSD's as a cache)?
> >
> > Yup, and a lot more.
> >
> > > >and storing data in the btree instead of using it as a cache, enabling
> > > >usage without any backing devices.
> > > The needs some explanation. I assume you mean it operates as writeback bache
> > > that never flushes. If that's correct, I can only imagine it's useful when
> > > initially using it. And in that case seems a kind of a time bomb, because
> > > when it's "full" (and there's still no backing device)... what happens then?
> >
> > What Slava is alluding to is that bcache is becoming a full posix filesystem - I
> > was actually demoing it... nearly a year ago, I think. With eventual feature
> > parity with btrfs and much, much better performance.
> >
> 
> btrfs use COW b-tree shadowing to support snapshots and clones, what
> approach will this new bcache FS use to support those features?

bcache has always been COW for data, and it supports copy offload (i.e. btrfs
reflink) today via just multiple extents pointing to the same data.

I'll write up the plan for snapshots at some point, it's quite different from
the btrfs approach.

> And will this new FS continue to use 2MB buckets without random writes
> in them? if that's the case, when this new FS is used on SSDs only, I
> guess FS GC has to be turned on for all buckets to reclaim free space.
> Since a lot of SSDs use parallel writes, some use compression, it's
> not possible to align one logical 2MB bucket at FS level to one single
> physical NAND block, then GC will be performed twice for one dirty
> bucket at different levels, so SSD life will be cut in half in the
> worst case?

Yeah that is a potential issue, for SSDs what we really want is to get raw
access to flash and have bcache be the FTL. Even if that doesn't happen there
are things we can do to mitigate though, suffice it to say this issue has been
discussed a lot :)

Re: 3.18.1 + latest bcache-dev

[Kent Overstreet]

On Wed, Jan 07, 2015 at 12:17:19PM -0800, Eric Wheeler wrote:
> On Tue, 6 Jan 2015, Rolf Fokkens wrote:
> 
> > On 01/05/2015 08:47 AM, Slava Pestov wrote:
> > > The plan is to incrementally backport bug fixes and optimizations from
> > > bcache-dev to upstream for the foreseeable future.
> > As an ethousiastic bcache user myself I would be very happy if there's a
> > transition plan! :-)
> 
> Would it make sense to call this bcache2 and merge both into the linux 
> tree once bcache2 is stable enough for broader testing?

Yeah that might be what happens.

> To the extent possible, you could reuse common code paths and simplify the 
> backporting of bug fixes into the existing stable branch.

Code reuse is problematic because struct bkey itself changes, a lot of the code
that's factored out as a library now depends on the specific layout of struct
bkey :/

Gonna try and backport as much as possible that doesn't require changing the on
disk format when I have time, though.

Re: 3.18.1 + latest bcache-dev

[Rolf Fokkens]

On 01/08/2015 03:30 AM, Kent Overstreet wrote:
> On Wed, Jan 07, 2015 at 12:17:19PM -0800, Eric Wheeler wrote:
>> Would it make sense to call this bcache2 and merge both into the 
>> linux tree once bcache2 is stable enough for broader testing? 
> Yeah that might be what happens.
I would be very much in favour if this approach. The (temporary) 
availability of both bcache and bcache2 gives users not only time to 
test it, but to migrate as well.