btrfs-tools in Debian squeeze-backports?

btrfs-tools in Debian squeeze-backports?

[Daniel Pocock]

These are the btrfs-tools versions on Debian:

squeeze:
kernel: 2.6.32
tools: 0.19+20100601-3

squeeze-backports:
kernel: 2.6.39
tools: nothing (so user ends up with 0.19+20100601-3)

wheezy/testing/sid:
kernel: 3.1.6-1
tools: 0.19+20111105-2

Using the 2.6.39 kernel from squeeze-backports, do I need a newer
btrfs-tools and is there a particular reason it is not in
squeeze-backports too?

Or should I not be trying to use the versions in squeeze at all - should
I be on testing/wheezy or unstable?

The Debian btrfs wiki and the regular btrfs wiki don't really suggest a
good starting point (other than suggesting the btrfs has been in Debian
since squeeze)
http://wiki.debian.org/Btrfs

If I try to use the version from testing or unstable, I get this error
on a squeeze setup:

 btrfs-tools depends on e2fslibs (>= 1.41.99); however:
  Version of e2fslibs on system is 1.41.12-4stable1.


http://packages.debian.org/search?keywords=btrfs-tools

Re: btrfs-tools in Debian squeeze-backports?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2362 bytes --]

On Mon, Jan 02, 2012 at 03:29:49PM +0100, Daniel Pocock wrote:
> These are the btrfs-tools versions on Debian:
> 
> squeeze:
> kernel: 2.6.32
> tools: 0.19+20100601-3
> 
> squeeze-backports:
> kernel: 2.6.39
> tools: nothing (so user ends up with 0.19+20100601-3)
> 
> wheezy/testing/sid:
> kernel: 3.1.6-1
> tools: 0.19+20111105-2
> 
> Using the 2.6.39 kernel from squeeze-backports,

   Don't do that. It's pretty old, and there's been a large number of
fixes in btrfs since then. I would recommend using the very latest
kernel you can -- preferably an -rc series kernel (after -rc3
or so, though), or the latest release kernel.

> do I need a newer btrfs-tools

   The only real reason to upgrade your btrfs tools is to get access
to the newer features of the FS.

> and is there a particular reason it is not in squeeze-backports too?

   You'll have to ask the backports people about that. The btrfs
developers don't have any control over what ends up in most
distributions.

> Or should I not be trying to use the versions in squeeze at all - should
> I be on testing/wheezy or unstable?

   You should definitely be on 3.1 or 3.2 kernel at the moment.

> The Debian btrfs wiki and the regular btrfs wiki don't really suggest a
> good starting point (other than suggesting the btrfs has been in Debian
> since squeeze)
> http://wiki.debian.org/Btrfs
> 
> If I try to use the version from testing or unstable, I get this error
> on a squeeze setup:
> 
>  btrfs-tools depends on e2fslibs (>= 1.41.99); however:
>   Version of e2fslibs on system is 1.41.12-4stable1.

   In that case, I'd suggest either building from the git source tree
for tools (see the wiki[1] for instructions), or, if you insist on
having .deb packages, doing your own backport:

$ apt-get build-dep btrfs-tools
$ apt-get source -t unstable btrfs-tools
$ cd btrfs-tools-0.19+20111105
$ fakeroot debian/rules
$ sudo dpkg -i ../btrfs-tools-0.19+20111105.deb   # This may be a different name

   Hugo.

[1] http://btrfs.ipv5.de/index.php?title=Btrfs_source_repositories#Official_repository

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- Anyone using a computer to generate random numbers is, of ---    
                       course,  in a state of sin.                       

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: btrfs-tools in Debian squeeze-backports?

[cwillu]

On Mon, Jan 2, 2012 at 8:29 AM, Daniel Pocock <daniel@pocock.com.au> wr=
ote:
>
>
> These are the btrfs-tools versions on Debian:
>
> squeeze:
> kernel: 2.6.32
> tools: 0.19+20100601-3
>
> squeeze-backports:
> kernel: 2.6.39
> tools: nothing (so user ends up with 0.19+20100601-3)
>
> wheezy/testing/sid:
> kernel: 3.1.6-1
> tools: 0.19+20111105-2
>
> Using the 2.6.39 kernel from squeeze-backports, do I need a newer
> btrfs-tools and is there a particular reason it is not in
> squeeze-backports too?
>
> Or should I not be trying to use the versions in squeeze at all - sho=
uld
> I be on testing/wheezy or unstable?
>
> The Debian btrfs wiki and the regular btrfs wiki don't really suggest=
 a
> good starting point (other than suggesting the btrfs has been in Debi=
an
> since squeeze)
> http://wiki.debian.org/Btrfs
>
> If I try to use the version from testing or unstable, I get this erro=
r
> on a squeeze setup:
>
> =C2=A0btrfs-tools depends on e2fslibs (>=3D 1.41.99); however:
> =C2=A0Version of e2fslibs on system is 1.41.12-4stable1.

0.19+20111105-2 should be sufficiently up to date for most day-to-day
purposes; the particular dependency you're running up against is
probably a quirk of the packaging.

Note that you really want to be running the latest kernel possible if
using btrfs;  since 2.6.39 there have been several major performance
fixes, stability fixes, crash-corruption fixes, which users did hit on
a somewhat regular basis.  Btrfs is not yet stable for the typical
user who just wants things to work, even when things don't.  I don't
know of any major distros that offer support services for btrfs
filesystems, for instance.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs-tools in Debian squeeze-backports?

[Daniel Pocock]

> 
> Note that you really want to be running the latest kernel possible if
> using btrfs;  since 2.6.39 there have been several major performance
> fixes, stability fixes, crash-corruption fixes, which users did hit on
> a somewhat regular basis.  Btrfs is not yet stable for the typical
> user who just wants things to work, even when things don't.  I don't
> know of any major distros that offer support services for btrfs
> filesystems, for instance.


I'm not planning to run my whole system on btrfs just yet - but I was
keen to start running one or two test filesystems on a server that I
currently have running Debian squeeze.  Everything else on the server
has to remain as-is if possible.

Thanks for the feedback - I will start looking into the newer kernels
and see if I can use one of them with squeeze, or maybe I will just set
up a VM for btrfs

One thing I've already noticed in 2.6.39 (and both versions of the
tools) is that df results are misleading.  E.g. if I run regular df (not
btrfs fi df), I am seeing the same amount of available space for all
filesystems.  Is there currently a way to see space used by each
subvolume and snapshot and which kernel and tools versions might be needed?

Re: btrfs-tools in Debian squeeze-backports?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2112 bytes --]

On Mon, Jan 02, 2012 at 04:01:48PM +0100, Daniel Pocock wrote:
> One thing I've already noticed in 2.6.39 (and both versions of the
> tools) is that df results are misleading.  E.g. if I run regular df (not
> btrfs fi df), I am seeing the same amount of available space for all
> filesystems.  Is there currently a way to see space used by each
> subvolume and snapshot and which kernel and tools versions might be needed?

   It's actually not possible in general. Since it's possible to have
different bits of the FS (data vs metadata) with different replication
levels, one byte written to the FS could take up either 1 or 2 bytes
of raw disk storage, and there's no way of predicting what the overall
usage will be, so it's not possible to give an accurate estimate of
free space.

   Similarly, if you have a 10G subvolume, and a snapshot of it, then
how much space does each one take up?

   If you said 10G, then you're wrong, because the total storage space
for the two together is only 10G.

   If, on the basis of that, you said 5G each, you're wrong, because
if you delete one of them, you get no free space back.

   If, on the basis of that, you said 0G each, you're wrong, because
if you delete both of them you get 10G of free space back.

   The best you can ask is "if I delete this subvolume, how much space
will I get back?", but even that's non-trivial: Arne Jansen is working
on that feature as a side-effect of his quota work, and it will be
coming at some point, but we can't do it right now.

   See the FAQ on filesystem usage[1] for how to interpret the btrfs
tools for examining space usage, and the mis-named "sysadmin's
guide"[2] for more horrible details of what's going on underneath.

   Hugo.

[1] http://btrfs.ipv5.de/index.php?title=FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F
[2] http://btrfs.ipv5.de/index.php?title=SysadminGuide

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
         --- I'm on a 30-day diet. So far I've lost 18 days. ---         

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: (renamed thread) btrfs metrics

[Daniel Pocock]

>    It's actually not possible in general. Since it's possible to have
> different bits of the FS (data vs metadata) with different replication
> levels, one byte written to the FS could take up either 1 or 2 bytes
> of raw disk storage, and there's no way of predicting what the overall
> usage will be, so it's not possible to give an accurate estimate of
> free space.
> 
>    Similarly, if you have a 10G subvolume, and a snapshot of it, then
> how much space does each one take up?

Yes, I agree it is not simple

However, the lvs command from LVM does provide a useful way to show how
full a snapshot volume is.  In the btrfs situation, the snapshot is
within the FS, so it is not quite the same metric

I am looking at what metrics are needed to monitor btrfs in production.
 I actually look after the ganglia-modules-linux package, which includes
some FS space metrics, but I figured that btrfs throws all that out the
window.

Can you suggest metrics that would be meaningful, do I look in /proc or
with syscalls, is there any code I should look at for an example of how
to extract them with C?  Ideally, Ganglia runs without root privileges
too, so please let me know if btrfs will allow me to access them

Re: (renamed thread) btrfs metrics

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2972 bytes --]

On Mon, Jan 02, 2012 at 05:25:41PM +0100, Daniel Pocock wrote:
> 
> >    It's actually not possible in general. Since it's possible to have
> > different bits of the FS (data vs metadata) with different replication
> > levels, one byte written to the FS could take up either 1 or 2 bytes
> > of raw disk storage, and there's no way of predicting what the overall
> > usage will be, so it's not possible to give an accurate estimate of
> > free space.
> > 
> >    Similarly, if you have a 10G subvolume, and a snapshot of it, then
> > how much space does each one take up?
> 
> Yes, I agree it is not simple
> 
> However, the lvs command from LVM does provide a useful way to show how
> full a snapshot volume is.  In the btrfs situation, the snapshot is
> within the FS, so it is not quite the same metric

   The way that LVM works is very different to the way that btrfs
works. If you try drawing parallels, you will probably go very wrong.
LVM snapshots are (IIRC) given a fixed-size pool of blocks on
creation, from which blocks are taken when modifications are made to
the original, so it's possible to know how much of the pool has been
taken. With btrfs, as you've spotted, the pool is the entirety of the
unused space in the filesystem, so there's no limit on snapshot size
or the divergence of a snapshot from its original subvolume.

> I am looking at what metrics are needed to monitor btrfs in production.
>  I actually look after the ganglia-modules-linux package, which includes
> some FS space metrics, but I figured that btrfs throws all that out the
> window.
> 
> Can you suggest metrics that would be meaningful, do I look in /proc or
> with syscalls, is there any code I should look at for an example of how
> to extract them with C?  Ideally, Ganglia runs without root privileges
> too, so please let me know if btrfs will allow me to access them

   It depends on what you want to know, really. If you want "how close
am I to a full filesystem?", then the output of df will give you a
measure, even if it could be up to a factor of 2 out -- you can use it
for predictive planning, though, as it'll be near zero when the FS
runs out of space.

   If you really want to, you can get your hands into the filesystem
structures on a read-only (and non-locked) basis using the
BTRFS_IOC_TREE_SEARCH ioctl, and the FS structures are documented at
[1]. However, that's generally going to be pretty ugly, and most
likely pretty slow for many operations at the subvolume level.

   If you want anything on a per-subvolume basis, then you'll have to
wait for Arne to finish the work on quotas.

   Hugo.

[1] http://btrfs.ipv5.de/index.php?title=Data_Structures

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- The Tao that is seen / Is not the true Tao, until / You ---     
                           bring fresh toner.                            

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: (renamed thread) btrfs metrics

[Daniel Pocock]

>> I am looking at what metrics are needed to monitor btrfs in production.
>>  I actually look after the ganglia-modules-linux package, which includes
>> some FS space metrics, but I figured that btrfs throws all that out the
>> window.
>>
>> Can you suggest metrics that would be meaningful, do I look in /proc or
>> with syscalls, is there any code I should look at for an example of how
>> to extract them with C?  Ideally, Ganglia runs without root privileges
>> too, so please let me know if btrfs will allow me to access them
> 
>    It depends on what you want to know, really. If you want "how close
> am I to a full filesystem?", then the output of df will give you a
> measure, even if it could be up to a factor of 2 out -- you can use it
> for predictive planning, though, as it'll be near zero when the FS
> runs out of space.


Maybe if you look at it from the point of the sysadmin and think about
what questions he might want to ask:

a) how much space would I reclaim if I deleted snapshot X?

b) how much space would I reclaim if I deleted all snapshots?

c) how much space would I need if I start making 4 snapshots a day and
keeping them for 48 hours?

(Ganglia also sums data across the enterprise, so if such metrics are
implemented, the admin can quickly see the sum of all snapshot usage on
his grid/cluster)

and also:

d) what metrics would be useful for someone developing or testing some
solution involving btrfs?

The Ganglia framework uses rrdtool to graph metric data and present it
alongside other system data (e.g. to see disk IO rates, CPU load, cache
activity all on the same graph) so it could provide some useful insights
into any performance testing of btrfs.  Even better, using rrdtool, you
can overlay some btrfs metric on the same graph as a system metric (e.g.
IO request sizes)


>    If you really want to, you can get your hands into the filesystem
> structures on a read-only (and non-locked) basis using the
> BTRFS_IOC_TREE_SEARCH ioctl, and the FS structures are documented at
> [1]. However, that's generally going to be pretty ugly, and most
> likely pretty slow for many operations at the subvolume level.
> 
>    If you want anything on a per-subvolume basis, then you'll have to
> wait for Arne to finish the work on quotas.
> 

Initially, I could just start with some simple metric (even just
retrieving the btrfs UUID) as a proof-of-concept, and then add more
stuff later, for example, when Arne has the quota work in a stable form.

Re: btrfs-tools in Debian squeeze-backports?

[Jan Schmidt]

On 02.01.2012 16:01, Daniel Pocock wrote:
> One thing I've already noticed in 2.6.39 (and both versions of the
> tools) is that df results are misleading.  E.g. if I run regular df (not
> btrfs fi df), I am seeing the same amount of available space for all
> filesystems.  Is there currently a way to see space used by each
> subvolume and snapshot and which kernel and tools versions might be needed?

If you're interested (and brave), you might give the subvolume quota
patches a try. Arne sent them last October, subject

   [PATCH v0 00/18] btfs: Subvolume Quota Groups

Be warned that this is a v0 patch, it's not been tested too much and
will very likely be reworked in the future. But that kind of
functionality will hopefully be added to btrfs, consequently.

-Jan

Re: (renamed thread) btrfs metrics

[Kok, Auke-jan H]

On Wed, Jan 4, 2012 at 3:48 AM, Daniel Pocock <daniel@pocock.com.au> wr=
ote:
>
>>> I am looking at what metrics are needed to monitor btrfs in product=
ion.
>>> =A0I actually look after the ganglia-modules-linux package, which i=
ncludes
>>> some FS space metrics, but I figured that btrfs throws all that out=
 the
>>> window.
>>>
>>> Can you suggest metrics that would be meaningful, do I look in /pro=
c or
>>> with syscalls, is there any code I should look at for an example of=
 how
>>> to extract them with C? =A0Ideally, Ganglia runs without root privi=
leges
>>> too, so please let me know if btrfs will allow me to access them
>>
>> =A0 =A0It depends on what you want to know, really. If you want "how=
 close
>> am I to a full filesystem?", then the output of df will give you a
>> measure, even if it could be up to a factor of 2 out -- you can use =
it
>> for predictive planning, though, as it'll be near zero when the FS
>> runs out of space.
>
>
> Maybe if you look at it from the point of the sysadmin and think abou=
t
> what questions he might want to ask:
>
> a) how much space would I reclaim if I deleted snapshot X?
>
> b) how much space would I reclaim if I deleted all snapshots?
>
> c) how much space would I need if I start making 4 snapshots a day an=
d
> keeping them for 48 hours?

chiming in on the discussion - what I'd like to personally see:

=46irst, probably easiest: Display per subvol the space used that is
"unique" (not used by other subvolumes), and shared (the opposite -
all blocks that appear in other subvolumes as well).

=46rom there on, one could potentially create a matrix: (proportional
font art, apologies):

          | subvol1  | subvol2  | subvol3  |
----------+----------+----------+----------+
 subvol1  |   200M   |     20M  |     50M  |
----------+----------+----------+----------+
 subvol2  |    20M   |    350M  |     22M  |
----------+----------+----------+----------+
 subvol3  |    50M   |     22M  |    634M  |
----------+----------+----------+----------+

The diagonal obviously shows the "unique" blocks, subvol2 and subvol1
share 20M data, etc. Missing from this plot would be "how much is
shared between subvol1, subvol2, and subvol3" together, but it's a
start and not something that hard to understand. One might add a
column for "total size" of each subvol, which may obviously not be an
addition of the rest of the columns in this diagram.

Anyway, something like this would be high on my list of `df` numbers
I'd like to see - since I think they are useful numbers.

Cheers,

Auke
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: (renamed thread) btrfs metrics

[Daniel Pocock]

> 
> From there on, one could potentially create a matrix: (proportional
> font art, apologies):
> 
>           | subvol1  | subvol2  | subvol3  |
> ----------+----------+----------+----------+
>  subvol1  |   200M   |     20M  |     50M  |
> ----------+----------+----------+----------+
>  subvol2  |    20M   |    350M  |     22M  |
> ----------+----------+----------+----------+
>  subvol3  |    50M   |     22M  |    634M  |
> ----------+----------+----------+----------+
> 
> The diagonal obviously shows the "unique" blocks, subvol2 and subvol1
> share 20M data, etc. Missing from this plot would be "how much is
> shared between subvol1, subvol2, and subvol3" together, but it's a
> start and not something that hard to understand. One might add a
> column for "total size" of each subvol, which may obviously not be an
> addition of the rest of the columns in this diagram.
> 
> Anyway, something like this would be high on my list of `df` numbers
> I'd like to see - since I think they are useful numbers.
> 

This is an interesting way to look at it

Ganglia typically records time series data, it is quite conceivable to
create a metric for every permutation in each and store that in rrdtool

The challenge would then be in reporting on the data: the rrdtool graphs
use time as an X-axis, and then it can display multiple Y values

However, now that I've started thinking about the type of data generated
from btrfs, I was wondering if some kind of rr3dtool is needed - a 3D
graphing solution - or potentially making graphs that do not include
time on any axis?

Has anyone seen anything similar for administering ZFS, for example?

Re: (renamed thread) btrfs metrics, free space reporting

[Daniel Pocock]

On 05/01/12 11:09, Daniel Pocock wrote:
> 
>>
>> From there on, one could potentially create a matrix: (proportional
>> font art, apologies):
>>
>>           | subvol1  | subvol2  | subvol3  |
>> ----------+----------+----------+----------+
>>  subvol1  |   200M   |     20M  |     50M  |
>> ----------+----------+----------+----------+
>>  subvol2  |    20M   |    350M  |     22M  |
>> ----------+----------+----------+----------+
>>  subvol3  |    50M   |     22M  |    634M  |
>> ----------+----------+----------+----------+
>>
>> The diagonal obviously shows the "unique" blocks, subvol2 and subvol1
>> share 20M data, etc. Missing from this plot would be "how much is
>> shared between subvol1, subvol2, and subvol3" together, but it's a
>> start and not something that hard to understand. One might add a
>> column for "total size" of each subvol, which may obviously not be an
>> addition of the rest of the columns in this diagram.
>>
>> Anyway, something like this would be high on my list of `df` numbers
>> I'd like to see - since I think they are useful numbers.
>>
> 
> This is an interesting way to look at it
> 
> Ganglia typically records time series data, it is quite conceivable to
> create a metric for every permutation in each and store that in rrdtool
> 
> The challenge would then be in reporting on the data: the rrdtool graphs
> use time as an X-axis, and then it can display multiple Y values
> 
> However, now that I've started thinking about the type of data generated
> from btrfs, I was wondering if some kind of rr3dtool is needed - a 3D
> graphing solution - or potentially making graphs that do not include
> time on any axis?
> 
> Has anyone seen anything similar for administering ZFS, for example?
> 


I just wanted to follow up on this and see if anybody had any more
comments or if the situation has changed?

One other thing that came to mind for me is the idea of letting the
local system administrator define views (similar to views in SQL) and
also nominate which of the views should be used to return values for the
standard df command.

This would allow existing monitoring tools and scripts to continue
getting some data that is considered sensible for a specific context.

Re: (renamed thread) btrfs metrics, free space reporting

[Qu Wenruo]

Daniel Pocock wrote on 2015/08/30 14:31 +0200:
>
>
> On 05/01/12 11:09, Daniel Pocock wrote:
>>
>>>
>>>  From there on, one could potentially create a matrix: (proportional
>>> font art, apologies):
>>>
>>>            | subvol1  | subvol2  | subvol3  |
>>> ----------+----------+----------+----------+
>>>   subvol1  |   200M   |     20M  |     50M  |
>>> ----------+----------+----------+----------+
>>>   subvol2  |    20M   |    350M  |     22M  |
>>> ----------+----------+----------+----------+
>>>   subvol3  |    50M   |     22M  |    634M  |
>>> ----------+----------+----------+----------+
>>>
>>> The diagonal obviously shows the "unique" blocks, subvol2 and subvol1
>>> share 20M data, etc. Missing from this plot would be "how much is
>>> shared between subvol1, subvol2, and subvol3" together, but it's a
>>> start and not something that hard to understand. One might add a
>>> column for "total size" of each subvol, which may obviously not be an
>>> addition of the rest of the columns in this diagram.
>>>
>>> Anyway, something like this would be high on my list of `df` numbers
>>> I'd like to see - since I think they are useful numbers.
>>>
>>
>> This is an interesting way to look at it
>>
>> Ganglia typically records time series data, it is quite conceivable to
>> create a metric for every permutation in each and store that in rrdtool
>>
>> The challenge would then be in reporting on the data: the rrdtool graphs
>> use time as an X-axis, and then it can display multiple Y values
>>
>> However, now that I've started thinking about the type of data generated
>> from btrfs, I was wondering if some kind of rr3dtool is needed - a 3D
>> graphing solution - or potentially making graphs that do not include
>> time on any axis?
>>
>> Has anyone seen anything similar for administering ZFS, for example?
>>
>
>
> I just wanted to follow up on this and see if anybody had any more
> comments or if the situation has changed?
>
> One other thing that came to mind for me is the idea of letting the
> local system administrator define views (similar to views in SQL) and
> also nominate which of the views should be used to return values for the
> standard df command.
>
> This would allow existing monitoring tools and scripts to continue
> getting some data that is considered sensible for a specific context.

The matrix seems intersting.

But have you guys tried qgroups?
That will do the thing for you and it should be more flex than you 
imagination.

For your use case, qgroup should be quite stable after v4.2-rc1, as you 
only want to see how much unique/shared extents between subvolumes.

Thanks,
Qu
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>