bcache and SSD over provisioning

bcache and SSD over provisioning

[Francis Moreau]

Hello,

I've recently learnt that SSDs (all ?) need to be over provisioned. My
SSD is a kingstom SSDNOW MS200 (controller is
SandForce-SF2241).

First I'd like to know if over-provisioning is still needed when
bcache is used ?

I'm assuming that I need over-provisioning.

Currently my whole SSD is used a cache device and my backing device is
a 900Mo partition with ext4 and no over-provisioning was foreseen.

What's the best practice to make my SSD over-provisioned ?

  - Should I create 2 partitions with one left emtpy and whose size is
roughly 7% of the device size ?
  - Should I create a "Host Protected Area" by using hdparm ?

And finally, can I create the over-provisioning without lost the
content of my backing device ?

Thanks for your help.
-- 
Francis

Re: bcache and SSD over provisioning

[Matthew Patton]

On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau  
<francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> First I'd like to know if over-provisioning is still needed when
> bcache is used ?

it's needed for any decent, sustained write load if you don't want  
performance to crash thru the floor. But it doesn't sound like this is  
your use case. Are you sustaining 5000+ write IOPs?

>   - Should I create 2 partitions with one left emtpy and whose size is
> roughly 7% of the device size ?

Your SSD already has 7% overprovisioning. For best sustained and  
consistent write performance you'll want to bump it to 25-30%. Yes you  
just need to partition/LVM down to the desired size.

>   - Should I create a "Host Protected Area" by using hdparm ?

http://www.thomas-krenn.com/en/wiki/SSD_Over-provisioning_using_hdparm

Either take the precautions outlined in the WIKI given the penchant of the  
kernel to ignore such settings, or just remove the chance for error and  
partition the device instead.

> And finally, can I create the over-provisioning without lost the
> content of my backing device ?

hell no.

-- 
NSA - subscribe to our Timeline and you'll know even before they've hit  
'ENTER'

"The budget should be balanced, the Treasury refilled, public debt  
reduced, the arrogance of officialdom tempered and controlled, and the  
assistance to foreign lands should be curtailed lest Rome become bankrupt.  
People must again learn to work, instead of living on public assistance."
  - Cicero (55 BC)

Re: bcache and SSD over provisioning

[Zachary Palmer]

On 11/13/2013 09:13 AM, Matthew Patton wrote:
> On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau 
> <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
>> And finally, can I create the over-provisioning without lost the
>> content of my backing device ?
>
> hell no.
>
I'm a bit confused: why not?  Francis seems to be asking if the entire 
bcache device needs to be recreated.  It definitely is necessary to 
rebuild the cache device, but the backing device shouldn't be affected.  
Let's call the backing device /dev/backdev and the cache device 
/dev/cachedev.  Then it should be possible to do the following:

0. If you are using writeback caching, "echo none > 
/sys/block/backdev/bcache/cache_mode" to tell bcache to flush the 
contents of the cache and then wait until "cat 
/sys/block/backdev/bcache/dirty_data" yields a zero
1. "echo 1 > /sys/block/backdev/bcache/detach" to remove the caching 
device from the bcache volume
2. "echo 1 >  /sys/fs/bcache/$UUID/unregister" to remove the caching 
device from the kernel
3. repartition the caching device so that /dev/cachedev is only e.g. 75% 
of the SSD
4. "make-bcache -C /dev/cachedev" to recreate the cache device
5. "echo /dev/cachedev > /sys/fs/bcache/register" to make the kernel 
aware of the new device
6. "ls /sys/fs/bcache" or just pay attention to make-bcache to find the 
cache volume UUID
7. "echo $UUID > /sys/block/backdev/bcache/attach" to reattach the 
caching device
8. "echo whatever-mode > /sys/block/backdev/bcache/cache_mode" to put it 
back into whichever mode you wish

Is there something wrong with my reasoning?  I may also have 
misunderstood the situation.  I'm just hoping I understand bcache as 
well as I think I do.  :)

Cheers,

Zach

Re: bcache and SSD over provisioning

[Matthew Patton]

On Wed, 13 Nov 2013 09:32:20 -0500, Zachary Palmer <zep_bcache-J5qI5MFTcs8@public.gmane.org>  
wrote:

> On 11/13/2013 09:13 AM, Matthew Patton wrote:
>> On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau  
>> <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>
>>> And finally, can I create the over-provisioning without lost the
>>> content of my backing device ?
>>
>> hell no.
>>
> I'm a bit confused: why not?

Gahhh! I totally read that as 'caching' device. Very sorry. I'll go shut  
up now...

Re: bcache and SSD over provisioning

[Zachary Palmer]

On 11/13/2013 09:42 AM, Matthew Patton wrote:
> On Wed, 13 Nov 2013 09:32:20 -0500, Zachary Palmer 
> <zep_bcache-J5qI5MFTcs8@public.gmane.org> wrote:
>
>> On 11/13/2013 09:13 AM, Matthew Patton wrote:
>>> On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau 
>>> <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>
>>>> And finally, can I create the over-provisioning without lost the
>>>> content of my backing device ?
>>>
>>> hell no.
>>>
>> I'm a bit confused: why not?
>
> Gahhh! I totally read that as 'caching' device. Very sorry. I'll go 
> shut up now...
>
Ah!  That makes a lot of sense; I should've inferred.  Well, the 
instructions I just posted might be helpful to someone someday.  I hope 
they're right.  :)  And thanks for the info on the overprovisioning, by 
the way; I wasn't sure if partitioning like that was sufficient and it's 
good to hear that it is.  (It means I've already done this for my laptop.)

Re: bcache and SSD over provisioning

[Adam Berkan]

On Wed, Nov 13, 2013 at 6:13 AM, Matthew Patton <pattonme-/E1597aS9LQAvxtiuMwx3w@public.gmane.org> wrote:
> On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> wrote:
>
>> First I'd like to know if over-provisioning is still needed when
>> bcache is used ?
>
>
> it's needed for any decent, sustained write load if you don't want
> performance to crash thru the floor. But it doesn't sound like this is your
> use case. Are you sustaining 5000+ write IOPs?
>
>
>>   - Should I create 2 partitions with one left emtpy and whose size is
>> roughly 7% of the device size ?
>
bcache has a pretty flash friendly write pattern, so you probably
don't need a huge lot of empty space.  The 7% might in fact be enough,
but bumping it up to 10%-12% wouldn't hurt.  I think the >25% numbers
are more relevant for non-flash friendly writes (like random writes).
In that case you definitely want lots of free space.
>
> Your SSD already has 7% overprovisioning. For best sustained and consistent
> write performance you'll want to bump it to 25-30%. Yes you just need to
> partition/LVM down to the desired size.
>
>
>>   - Should I create a "Host Protected Area" by using hdparm ?
>
>
> http://www.thomas-krenn.com/en/wiki/SSD_Over-provisioning_using_hdparm
>
> Either take the precautions outlined in the WIKI given the penchant of the
> kernel to ignore such settings, or just remove the chance for error and
> partition the device instead.
>
>
>> And finally, can I create the over-provisioning without lost the
>> content of my backing device ?
>
>
> hell no.
>
> --
> NSA - subscribe to our Timeline and you'll know even before they've hit
> 'ENTER'
>
> "The budget should be balanced, the Treasury refilled, public debt reduced,
> the arrogance of officialdom tempered and controlled, and the assistance to
> foreign lands should be curtailed lest Rome become bankrupt. People must
> again learn to work, instead of living on public assistance."
>  - Cicero (55 BC)
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: bcache and SSD over provisioning

[Francis Moreau]

Hello Matthew,

Le 13/11/2013 15:13, Matthew Patton a écrit :
> On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau  
> <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> 
>> First I'd like to know if over-provisioning is still needed when
>> bcache is used ?
> 
> it's needed for any decent, sustained write load if you don't want  
> performance to crash thru the floor. But it doesn't sound like this is  
> your use case. Are you sustaining 5000+ write IOPs?
> 

That's indeed not really required: my system is for a desktop usage.

>>   - Should I create 2 partitions with one left emtpy and whose size is
>> roughly 7% of the device size ?
> 
> Your SSD already has 7% overprovisioning. For best sustained and  
> consistent write performance you'll want to bump it to 25-30%. Yes you  
> just need to partition/LVM down to the desired size.

Hmm, I don't see what makes you think that my SSD has already 7%
overprovisioning ? Actually I don't think it's the case.

Thanks for your help.

Re: bcache and SSD over provisioning

[Francis Moreau]

Le 13/11/2013 19:23, Adam Berkan a écrit :
> On Wed, Nov 13, 2013 at 6:13 AM, Matthew Patton <pattonme-/E1597aS9LQAvxtiuMwx3w@public.gmane.org> wrote:
>> On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau <francis.moro@gmail.com>
>> wrote:
>>
>>> First I'd like to know if over-provisioning is still needed when
>>> bcache is used ?
>>
>>
>> it's needed for any decent, sustained write load if you don't want
>> performance to crash thru the floor. But it doesn't sound like this is your
>> use case. Are you sustaining 5000+ write IOPs?
>>
>>
>>>   - Should I create 2 partitions with one left emtpy and whose size is
>>> roughly 7% of the device size ?
>>
> bcache has a pretty flash friendly write pattern, so you probably
> don't need a huge lot of empty space.  The 7% might in fact be enough,
> but bumping it up to 10%-12% wouldn't hurt.  I think the >25% numbers
> are more relevant for non-flash friendly writes (like random writes).
> In that case you definitely want lots of free space.

Thanks for your enlightment.

I'll probably go for 10%.

Re: bcache and SSD over provisioning

[Francis Moreau]

Le 13/11/2013 16:17, Zachary Palmer a écrit :
> On 11/13/2013 09:42 AM, Matthew Patton wrote:
>> On Wed, 13 Nov 2013 09:32:20 -0500, Zachary Palmer 
>> <zep_bcache-J5qI5MFTcs8@public.gmane.org> wrote:
>>
>>> On 11/13/2013 09:13 AM, Matthew Patton wrote:
>>>> On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau 
>>>> <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>>
>>>>> And finally, can I create the over-provisioning without lost the
>>>>> content of my backing device ?
>>>>
>>>> hell no.
>>>>
>>> I'm a bit confused: why not?
>>
>> Gahhh! I totally read that as 'caching' device. Very sorry. I'll go 
>> shut up now...
>>
> Ah!  That makes a lot of sense; I should've inferred.  Well, the 
> instructions I just posted might be helpful to someone someday.  I hope 
> they're right.  :) 

That's definitively helpful, I'm going to follow your instructions soon.
In case something is going wrong, I'll report it.

Thanks !

RE: bcache and SSD over provisioning

[Paul B. Henson]

> From: Adam Berkan
> Sent: Wednesday, November 13, 2013 10:24 AM
>
> bcache has a pretty flash friendly write pattern, so you probably
> don't need a huge lot of empty space.

Depending on the performance impact on your particular SSD, you could also
try enabling TRIM in bcache which should reduce the need for over
provisioning by making sure the SSD knows which blocks it can erase in the
background. It would be nice if bcache supported a manual TRIM like fstrim
works for filesystems so if you have an SSD whose performance precludes real
time TRIM you could schedule a daily or weekly batch TRIM to tune things up.

Re: bcache and SSD over provisioning

[Alex Elsayed]

Francis Moreau wrote:

> Hello Matthew,
> 
> Le 13/11/2013 15:13, Matthew Patton a écrit :
>> On Wed, 13 Nov 2013 03:42:54 -0500, Francis Moreau
>> <francis.moro-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> 
>>> First I'd like to know if over-provisioning is still needed when
>>> bcache is used ?
>> 
>> it's needed for any decent, sustained write load if you don't want
>> performance to crash thru the floor. But it doesn't sound like this is
>> your use case. Are you sustaining 5000+ write IOPs?
>> 
> 
> That's indeed not really required: my system is for a desktop usage.
> 
>>>   - Should I create 2 partitions with one left emtpy and whose size is
>>> roughly 7% of the device size ?
>> 
>> Your SSD already has 7% overprovisioning. For best sustained and
>> consistent write performance you'll want to bump it to 25-30%. Yes you
>> just need to partition/LVM down to the desired size.
> 
> Hmm, I don't see what makes you think that my SSD has already 7%
> overprovisioning ? Actually I don't think it's the case.
> 
> Thanks for your help.

Because the raw flash chips are only available in power-of-two sizes. The 
available storage the controller allows access to is 120GB, and it's 
reserved 8GB for overprovisioning if I'm looking at the right model.

Re: bcache and SSD over provisioning

[Francis Moreau]

Le 13/11/2013 23:30, Alex Elsayed a écrit :

[...]

> 
> Because the raw flash chips are only available in power-of-two sizes. The 
> available storage the controller allows access to is 120GB, and it's 
> reserved 8GB for overprovisioning if I'm looking at the right model.
> 

I got it.

Thanks for your explanation.