How to use multiple backing devices

How to use multiple backing devices

[Nikolaus Rath]

Hello,

If I'm understanding Documentation/bcache.txt correctly, I should be
able to use one SSD to cache multiple spinning disks.

However, I'm at a loss of how to set this up in practice. I believe I
need to do

  make-bcache -B /dev/sda  # spinning rust
  make-bcache -B /dev/sdb  # spinning rust 
  make-bcache -C /dev/sdc  # ssd

and then something like

  echo <CSET-UUID> > /sys/block/bcacheX/bcache/attach


But what do I have to put for X, and what for CSET-UUID? I believe for
at least one of those values I will have two options (because there are
two backing devices).


Thanks,
-Nikolaus

(No Cc on replies please, I'm reading the list)
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: How to use multiple backing devices

[Kai Krakow]

Am Mon, 08 Feb 2016 20:47:10 -0800
schrieb Nikolaus Rath <Nikolaus@rath.org>:

> Hello,
> 
> If I'm understanding Documentation/bcache.txt correctly, I should be
> able to use one SSD to cache multiple spinning disks.
> 
> However, I'm at a loss of how to set this up in practice. I believe I
> need to do
> 
>   make-bcache -B /dev/sda  # spinning rust
>   make-bcache -B /dev/sdb  # spinning rust 
>   make-bcache -C /dev/sdc  # ssd
> 
> and then something like
> 
>   echo <CSET-UUID> > /sys/block/bcacheX/bcache/attach
> 
> 
> But what do I have to put for X, and what for CSET-UUID? I believe for
> at least one of those values I will have two options (because there
> are two backing devices).

Create it in one go:

make-bcache -C /dev/sdc -B /dev/sd{a,b}

It will take away the hassle. Everything should be setup now.

But if you want to do it manually: you have to run "echo" twice - for
each X (the backing devices). CSET-UUID comes from /sys/fs/bcache where
you will find a UUID per each bcache caching device, which is just one
for your case.

-- 
Regards,
Kai

Replies to list-only preferred.

Re: How to use multiple backing devices

[Nikolaus Rath]

On Feb 09 2016, Kai Krakow <hurikhan77@gmail.com> wrote:
> Am Mon, 08 Feb 2016 20:47:10 -0800
> schrieb Nikolaus Rath <Nikolaus@rath.org>:
>
>> Hello,
>> 
>> If I'm understanding Documentation/bcache.txt correctly, I should be
>> able to use one SSD to cache multiple spinning disks.
>> 
>> However, I'm at a loss of how to set this up in practice. I believe I
>> need to do
>> 
>>   make-bcache -B /dev/sda  # spinning rust
>>   make-bcache -B /dev/sdb  # spinning rust 
>>   make-bcache -C /dev/sdc  # ssd
>> 
>> and then something like
>> 
>>   echo <CSET-UUID> > /sys/block/bcacheX/bcache/attach
>> 
>> 
>> But what do I have to put for X, and what for CSET-UUID? I believe for
>> at least one of those values I will have two options (because there
>> are two backing devices).
>
> Create it in one go:
>
> make-bcache -C /dev/sdc -B /dev/sd{a,b}
>
> It will take away the hassle. Everything should be setup now.

Yeah, but unfortunately I don't have enough space for that. I need to
create one, move the data over from second, create the second, move data
over from the SSD, and then create the SSD is caching device.


> But if you want to do it manually: you have to run "echo" twice - for
> each X (the backing devices). CSET-UUID comes from /sys/fs/bcache where
> you will find a UUID per each bcache caching device, which is just one
> for your case.


Ah, ok. Intuitively attaching the same cache device to two different
backing devices sounds dangerous, but I'll  take your word that the code
is designed for that...

Thanks,
-Nikolaus

(No Cc on replies please, I'm reading the list)
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: How to use multiple backing devices

[Kai Krakow]

Am Tue, 09 Feb 2016 08:03:53 -0800
schrieb Nikolaus Rath <Nikolaus@rath.org>:

> On Feb 09 2016, Kai Krakow <hurikhan77@gmail.com> wrote:
> > Am Mon, 08 Feb 2016 20:47:10 -0800
> > schrieb Nikolaus Rath <Nikolaus@rath.org>:
> >
> >> Hello,
> >> 
> >> If I'm understanding Documentation/bcache.txt correctly, I should
> >> be able to use one SSD to cache multiple spinning disks.
> >> 
> >> However, I'm at a loss of how to set this up in practice. I
> >> believe I need to do
> >> 
> >>   make-bcache -B /dev/sda  # spinning rust
> >>   make-bcache -B /dev/sdb  # spinning rust 
> >>   make-bcache -C /dev/sdc  # ssd
> >> 
> >> and then something like
> >> 
> >>   echo <CSET-UUID> > /sys/block/bcacheX/bcache/attach
> >> 
> >> 
> >> But what do I have to put for X, and what for CSET-UUID? I believe
> >> for at least one of those values I will have two options (because
> >> there are two backing devices).
> >
> > Create it in one go:
> >
> > make-bcache -C /dev/sdc -B /dev/sd{a,b}
> >
> > It will take away the hassle. Everything should be setup now.
> 
> Yeah, but unfortunately I don't have enough space for that. I need to
> create one, move the data over from second, create the second, move
> data over from the SSD, and then create the SSD is caching device.

Then do it like me:

1. Backup!
2. Remove first device from btrfs pool, run wipefs over it then!!
3. Format backing storage on it
4. Add bcache device back to the pool.
5. Repeat with next device from step 2, prepare to wait long!
6. Rebalance
7. Format caching device, enable caching device
8. Migrate your fstab/initrd whatever...

You should not enable caching before step 6, or your SSD may wear out a
lot.

You may need to add some temporary spare disk if you don't have enough
space available.

> > But if you want to do it manually: you have to run "echo" twice -
> > for each X (the backing devices). CSET-UUID comes
> > from /sys/fs/bcache where you will find a UUID per each bcache
> > caching device, which is just one for your case.
> 
> 
> Ah, ok. Intuitively attaching the same cache device to two different
> backing devices sounds dangerous, but I'll  take your word that the
> code is designed for that...

It's actually documented somewhere on the web page: One caching device
can back multiple backing devices. No need to partition it.

There were (are?) plans for even allowing attaching one backing device
to multiple caching devices (even n:n attaching) to allow for
redundancy, performance pushes, and error resiliency. Tho, I don't know
if that is implemented yet.

Also, keep an eye on your SSD wear if you're using writeback mode (use
smartctl for it with mail notifications). You may want to switch to
write-around or write-through mode before it potentially wears out.
The latter two modes should be safe, tho. Go, replace the SSD then.
Maybe trim your SSD first, then only partition and ever use only 80% of
it to increase life time as the drive can do better wear-levelling then
(sometimes called over-provisioning by manufacturers, this is even one
performance tuning option in most manufacturers Windows tools).


-- 
Regards,
Kai

Replies to list-only preferred.