[linux-lvm] bcache now to become io-manager?

[linux-lvm] bcache now to become io-manager?

[John Stoffel]

>>>>> "Joe" == Joe Thornber <thornber@redhat.com> writes:

[ I saw this and couldn't resist asking for more details ...]

Joe> Also there are big changes to bcache coming, that remove file
Joe> descriptors from the interface completely.  See the branch
Joe> 2019-09-05-add-io-manager for more info (bcache has been renamed
Joe> io_manager).

Can you post the full URL for this, and maybe give us a teaser on what
to expect?  I run lvache on my home system and I'd love to know how to
A) improve reporting of how well it works, and B) whether I'm using it
right, and of course C) if bcache would be a better replacement.

I'm using Linux v5.0.21 (self compiled) on Debian 9.11 with lvcache
setup.  It's a pair of mirrored 250gb SSDs in front of 4tb of mirrored
disk.

So anything where I can use my SSDs to cache my data accesses is
certainly interesting.

John

Re: [linux-lvm] bcache now to become io-manager?

[Marian Csontos]

On 10/30/19 1:25 AM, John Stoffel wrote:
>>>>>> "Joe" == Joe Thornber <thornber@redhat.com> writes:
> 
> [ I saw this and couldn't resist asking for more details ...]
> 
> Joe> Also there are big changes to bcache coming, that remove file
> Joe> descriptors from the interface completely.  See the branch
> Joe> 2019-09-05-add-io-manager for more info (bcache has been renamed
> Joe> io_manager).
> 
> Can you post the full URL for this, and maybe give us a teaser on what

For reference: 
https://sourceware.org/git/?p=lvm2.git;a=shortlog;h=refs/heads/2019-09-05-add-io-manager

> to expect?  I run lvache on my home system and I'd love to know how to
> A) improve reporting of how well it works, and B) whether I'm using it
> right, and of course C) if bcache would be a better replacement.
> 
> I'm using Linux v5.0.21 (self compiled) on Debian 9.11 with lvcache
> setup.  It's a pair of mirrored 250gb SSDs in front of 4tb of mirrored
> disk.
> 
> So anything where I can use my SSDs to cache my data accesses is
> certainly interesting.

"bcache" is a confusing name for internal block-cache used to read lvm 
metadata from disks, not that bcache one might think it is and does not 
aim to replace any of lvmcache, dm-cache, dm-writecache, or bcache. It 
should make lvm operations faster and is not supposed to affect data access.

-- Marian

> 
> John
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

Re: [linux-lvm] bcache now to become io-manager?

[Joe Thornber]

On Tue, Oct 29, 2019 at 08:25:23PM -0400, John Stoffel wrote:
> >>>>> "Joe" == Joe Thornber <thornber@redhat.com> writes:
> 
> [ I saw this and couldn't resist asking for more details ...]
> 
> Joe> Also there are big changes to bcache coming, that remove file
> Joe> descriptors from the interface completely.  See the branch
> Joe> 2019-09-05-add-io-manager for more info (bcache has been renamed
> Joe> io_manager).
> 
> Can you post the full URL for this, and maybe give us a teaser on what
> to expect?  I run lvache on my home system and I'd love to know how to
> A) improve reporting of how well it works, and B) whether I'm using it
> right, and of course C) if bcache would be a better replacement.
> 
> I'm using Linux v5.0.21 (self compiled) on Debian 9.11 with lvcache
> setup.  It's a pair of mirrored 250gb SSDs in front of 4tb of mirrored
> disk.
> 
> So anything where I can use my SSDs to cache my data accesses is
> certainly interesting.

Sorry, I think we're getting confused with similarly named things.  There
is a component in LVM source called 'block cache' that we use to scan
devices in parallel (under the hood it uses aio).  That component is being
renamed.  It's nothing to do with the disk caching technology called 'bcache'.

However, Dave Teigland has been adding support to LVM for a new caching
target that emphasises caching writes.  So keep an eye out for that.


- Joe

Re: [linux-lvm] bcache now to become io-manager?

[John Stoffel]

>>>>> "Joe" == Joe Thornber <thornber@redhat.com> writes:

Joe> On Tue, Oct 29, 2019 at 08:25:23PM -0400, John Stoffel wrote:
>> >>>>> "Joe" == Joe Thornber <thornber@redhat.com> writes:
>> 
>> [ I saw this and couldn't resist asking for more details ...]
>> 
Joe> Also there are big changes to bcache coming, that remove file
Joe> descriptors from the interface completely.  See the branch
Joe> 2019-09-05-add-io-manager for more info (bcache has been renamed
Joe> io_manager).
>> 
>> Can you post the full URL for this, and maybe give us a teaser on what
>> to expect?  I run lvache on my home system and I'd love to know how to
>> A) improve reporting of how well it works, and B) whether I'm using it
>> right, and of course C) if bcache would be a better replacement.
>> 
>> I'm using Linux v5.0.21 (self compiled) on Debian 9.11 with lvcache
>> setup.  It's a pair of mirrored 250gb SSDs in front of 4tb of mirrored
>> disk.
>> 
>> So anything where I can use my SSDs to cache my data accesses is
>> certainly interesting.

Joe> Sorry, I think we're getting confused with similarly named
Joe> things.  There is a component in LVM source called 'block cache'
Joe> that we use to scan devices in parallel (under the hood it uses
Joe> aio).  That component is being renamed.  It's nothing to do with
Joe> the disk caching technology called 'bcache'.

Ah... that's a good reason for this rename then. 

Joe> However, Dave Teigland has been adding support to LVM for a new
Joe> caching target that emphasises caching writes.  So keep an eye
Joe> out for that.

Will do. It's never really been clear if lvcache has really helped or
not, the testing was in-conclusive.

John

Re: [linux-lvm] bcache now to become io-manager?

[Zdenek Kabelac]

Dne 31. 10. 19 v 20:29 John Stoffel napsal(a):
>>>>>> "Joe" == Joe Thornber <thornber@redhat.com> writes:
> 
> Joe> On Tue, Oct 29, 2019 at 08:25:23PM -0400, John Stoffel wrote:
>>>>>>>> "Joe" == Joe Thornber <thornber@redhat.com> writes:
>>>
>>> [ I saw this and couldn't resist asking for more details ...]
>>>
> Joe> Also there are big changes to bcache coming, that remove file
> Joe> descriptors from the interface completely.  See the branch
> Joe> 2019-09-05-add-io-manager for more info (bcache has been renamed
> Joe> io_manager).
>>>
>>> Can you post the full URL for this, and maybe give us a teaser on what
>>> to expect?  I run lvache on my home system and I'd love to know how to
>>> A) improve reporting of how well it works, and B) whether I'm using it
>>> right, and of course C) if bcache would be a better replacement.
>>>
>>> I'm using Linux v5.0.21 (self compiled) on Debian 9.11 with lvcache
>>> setup.  It's a pair of mirrored 250gb SSDs in front of 4tb of mirrored
>>> disk.
>>>
>>> So anything where I can use my SSDs to cache my data accesses is
>>> certainly interesting.
> 
> Joe> Sorry, I think we're getting confused with similarly named
> Joe> things.  There is a component in LVM source called 'block cache'
> Joe> that we use to scan devices in parallel (under the hood it uses
> Joe> aio).  That component is being renamed.  It's nothing to do with
> Joe> the disk caching technology called 'bcache'.
> 
> Ah... that's a good reason for this rename then.

In this context bcache is purely an internal lvm2 coding naming -
it's never exposed to an lvm2 man page, never used as any argument or 
parameter in lvm2 command line - so it really is purely terminology for coders...

> 
> Joe> However, Dave Teigland has been adding support to LVM for a new
> Joe> caching target that emphasises caching writes.  So keep an eye
> Joe> out for that.
> 
> Will do. It's never really been clear if lvcache has really helped or
> not, the testing was in-conclusive.


dm-cache   (aka  'lvcreate --cache')  is  a hotspot cache - so
it does 'monitor' your device usage and moves to caching device hotspot areas 
(those which are frequently used).  And there need to be seen that often
there is also hidden 'page-cache' usage which avoids increasing
usage of real disk reads - so it simply takes times before dm-cache
gets well populated - this is something you could hardly 'benchmark' with like 
couple minutes long benchmark.

The recently introduced dm-writecache (aka 'lvconvert --type writecache) 
support on the other hand looks more or less like 'extended' page-caching - 
where all dirty pages from memory quickly lends in fast write caching device 
where they are copied to your slower physical storage device.


Regads

Zdenek