All of lore.kernel.org
 help / color / mirror / Atom feed
* btrfs filesystem usage - Wrong Unallocated indications - RAID10
@ 2016-05-23 13:34 Marco Lorenzo Crociani
  2016-05-23 18:34 ` Nicholas D Steeves
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Marco Lorenzo Crociani @ 2016-05-23 13:34 UTC (permalink / raw)
  To: linux-btrfs

Hi,
as I wrote today in IRCI experienced an issue with 'btrfs filesystem usage'.
I have a 4 partitions RAID10 btrfs filesystem almost full.
'btrfs filesystem usage' reports wrong "Unallocated" indications.

Linux 4.5.3
btrfs-progs v4.5.3


# btrfs fi usage /data/

Overall:
     Device size:          13.93TiB
     Device allocated:          13.77TiB
     Device unallocated:         167.54GiB
     Device missing:             0.00B
     Used:              13.44TiB
     Free (estimated):         244.39GiB    (min: 244.39GiB)
     Data ratio:                  2.00
     Metadata ratio:              2.00
     Global reserve:         512.00MiB    (used: 0.00B)

Data,single: Size:8.00MiB, Used:0.00B
    /dev/sda4       8.00MiB

Data,RAID10: Size:6.87TiB, Used:6.71TiB
    /dev/sda4       1.72TiB
    /dev/sdb3       1.72TiB
    /dev/sdc3       1.72TiB
    /dev/sdd3       1.72TiB

Metadata,single: Size:8.00MiB, Used:0.00B
    /dev/sda4       8.00MiB

Metadata,RAID10: Size:19.00GiB, Used:14.15GiB
    /dev/sda4       4.75GiB
    /dev/sdb3       4.75GiB
    /dev/sdc3       4.75GiB
    /dev/sdd3       4.75GiB

System,single: Size:4.00MiB, Used:0.00B
    /dev/sda4       4.00MiB

System,RAID10: Size:16.00MiB, Used:768.00KiB
    /dev/sda4       4.00MiB
    /dev/sdb3       4.00MiB
    /dev/sdc3       4.00MiB
    /dev/sdd3       4.00MiB

Unallocated:
    /dev/sda4       1.76TiB
    /dev/sdb3       1.76TiB
    /dev/sdc3       1.76TiB
    /dev/sdd3       1.76TiB

----------------------------------------------------------------------------------
# btrfs fi show /data/
Label: 'data'  uuid: df6639d5-3ef2-4ff6-a871-9ede440e2dae
     Total devices 4 FS bytes used 6.72TiB
     devid    1 size 3.48TiB used 3.44TiB path /dev/sda4
     devid    2 size 3.48TiB used 3.44TiB path /dev/sdb3
     devid    3 size 3.48TiB used 3.44TiB path /dev/sdc3
     devid    4 size 3.48TiB used 3.44TiB path /dev/sdd3

----------------------------------------------------------------------------------
# btrfs fi df /data/
Data, RAID10: total=6.87TiB, used=6.71TiB
Data, single: total=8.00MiB, used=0.00B
System, RAID10: total=16.00MiB, used=768.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, RAID10: total=19.00GiB, used=14.15GiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=512.00MiB, used=0.00B

----------------------------------------------------------------------------------
# df -h
/dev/sda4                     7,0T  6,8T    245G  97% /data

Regards,

-- 
Marco Crociani
Prisma Telecom Testing S.r.l.
via Petrocchi, 4  20127 MILANO  ITALY
Phone:  +39 02 26113507
Fax:  +39 02 26113597
e-mail:  marcoc@prismatelecomtesting.com
web:  http://www.prismatelecomtesting.com

Questa email (e I suoi allegati) costituisce informazione riservata e confidenziale e può essere soggetto a legal privilege. Può essere utilizzata esclusivamente dai suoi destinatari legittimi.  Se avete ricevuto questa email per errore, siete pregati di informarne immediatamente il mittente e quindi cancellarla.  A meno che non siate stati a ciò espressamente autorizzati, la diffusione o la riproduzione di questa email o del suo contenuto non sono consentiti.

  Salvo che questa email sia espressamente qualificata come offerta o accettazione contrattuale, il mittente non intende con questa email dare vita ad un vincolo giuridico e questa email non può essere interpretata quale offerta o accettazione che possa dare vita ad un contratto. Qualsiasi opinione manifestata in questa email è un'opinione personale del mittente, salvo che il mittente dichiari espressamente che si tratti di un'opinione di Prisma Engineering.

                                                             *******************************************************************************

  This e-mail (including any attachments) is private and confidential, and may be privileged.  It is for the exclusive use of the intended recipient(s).  If you have received this email in error, please inform the sender immediately and then delete this email.  Unless you have been given specific permission to do so, please do not distribute or copy this email or its contents.
  Unless the text of this email specifically states that it is a contractual offer or acceptance, the sender does not intend to create a legal relationship and this email shall not constitute an offer or acceptance which could give rise to a contract. Any views expressed in this communication are those of the individual sender, except where the sender specifically states them to be the views of Prisma Engineering.


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs filesystem usage - Wrong Unallocated indications - RAID10
  2016-05-23 13:34 btrfs filesystem usage - Wrong Unallocated indications - RAID10 Marco Lorenzo Crociani
@ 2016-05-23 18:34 ` Nicholas D Steeves
  2016-05-23 18:49 ` Zoiled
  2016-05-25 20:14 ` Henk Slager
  2 siblings, 0 replies; 5+ messages in thread
From: Nicholas D Steeves @ 2016-05-23 18:34 UTC (permalink / raw)
  To: Marco Lorenzo Crociani, Ivan Pilipenko; +Cc: Btrfs BTRFS

On 23 May 2016 at 09:34, Marco Lorenzo Crociani
<marcoc@prismatelecomtesting.com> wrote:
> Hi,
> as I wrote today in IRCI experienced an issue with 'btrfs filesystem usage'.
> I have a 4 partitions RAID10 btrfs filesystem almost full.
> 'btrfs filesystem usage' reports wrong "Unallocated" indications.
>
> Linux 4.5.3
> btrfs-progs v4.5.3
>
>
> # btrfs fi usage /data/
>
> Overall:
>     Device size:                       13.93TiB
>     Device allocated:               13.77TiB
>     Device unallocated:         167.54GiB

I wonder if this is related to whatever caused the free space cache
bug for Ivan Pilipenko and myself (linux 4.4.10, btrfs-progs 4.4.1)?

Cheers,
Nicholas

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs filesystem usage - Wrong Unallocated indications - RAID10
  2016-05-23 13:34 btrfs filesystem usage - Wrong Unallocated indications - RAID10 Marco Lorenzo Crociani
  2016-05-23 18:34 ` Nicholas D Steeves
@ 2016-05-23 18:49 ` Zoiled
  2016-05-23 19:19   ` Hugo Mills
  2016-05-25 20:14 ` Henk Slager
  2 siblings, 1 reply; 5+ messages in thread
From: Zoiled @ 2016-05-23 18:49 UTC (permalink / raw)
  To: Marco Lorenzo Crociani, linux-btrfs

Marco Lorenzo Crociani wrote:
> Hi,
> as I wrote today in IRCI experienced an issue with 'btrfs filesystem 
> usage'.
> I have a 4 partitions RAID10 btrfs filesystem almost full.
> 'btrfs filesystem usage' reports wrong "Unallocated" indications.
>
> Linux 4.5.3
> btrfs-progs v4.5.3
>
>
> # btrfs fi usage /data/
>
> Overall:
>     Device size:          13.93TiB
>     Device allocated:          13.77TiB
>     Device unallocated:         167.54GiB
>     Device missing:             0.00B
>     Used:              13.44TiB
>     Free (estimated):         244.39GiB    (min: 244.39GiB)
>     Data ratio:                  2.00
>     Metadata ratio:              2.00
>     Global reserve:         512.00MiB    (used: 0.00B)
>
> Data,single: Size:8.00MiB, Used:0.00B
>    /dev/sda4       8.00MiB
>
> Data,RAID10: Size:6.87TiB, Used:6.71TiB
>    /dev/sda4       1.72TiB
>    /dev/sdb3       1.72TiB
>    /dev/sdc3       1.72TiB
>    /dev/sdd3       1.72TiB
>
> Metadata,single: Size:8.00MiB, Used:0.00B
>    /dev/sda4       8.00MiB
>
> Metadata,RAID10: Size:19.00GiB, Used:14.15GiB
>    /dev/sda4       4.75GiB
>    /dev/sdb3       4.75GiB
>    /dev/sdc3       4.75GiB
>    /dev/sdd3       4.75GiB
>
> System,single: Size:4.00MiB, Used:0.00B
>    /dev/sda4       4.00MiB
>
> System,RAID10: Size:16.00MiB, Used:768.00KiB
>    /dev/sda4       4.00MiB
>    /dev/sdb3       4.00MiB
>    /dev/sdc3       4.00MiB
>    /dev/sdd3       4.00MiB
>
> Unallocated:
>    /dev/sda4       1.76TiB
>    /dev/sdb3       1.76TiB
>    /dev/sdc3       1.76TiB
>    /dev/sdd3       1.76TiB
>
> ---------------------------------------------------------------------------------- 
>
> # btrfs fi show /data/
> Label: 'data'  uuid: df6639d5-3ef2-4ff6-a871-9ede440e2dae
>     Total devices 4 FS bytes used 6.72TiB
>     devid    1 size 3.48TiB used 3.44TiB path /dev/sda4
>     devid    2 size 3.48TiB used 3.44TiB path /dev/sdb3
>     devid    3 size 3.48TiB used 3.44TiB path /dev/sdc3
>     devid    4 size 3.48TiB used 3.44TiB path /dev/sdd3
>
> ---------------------------------------------------------------------------------- 
>
> # btrfs fi df /data/
> Data, RAID10: total=6.87TiB, used=6.71TiB
> Data, single: total=8.00MiB, used=0.00B
> System, RAID10: total=16.00MiB, used=768.00KiB
> System, single: total=4.00MiB, used=0.00B
> Metadata, RAID10: total=19.00GiB, used=14.15GiB
> Metadata, single: total=8.00MiB, used=0.00B
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
> ---------------------------------------------------------------------------------- 
>
> # df -h
> /dev/sda4                     7,0T  6,8T    245G  97% /data
>
> Regards,
>
For what it's worth...  I have a 8 disk (7x 300GB + 1x 500GB) data 
raid10, metadata raid1 setup and I get the following output of btrfs...

Label: 'xxyyzz'  uuid: 12345678-9abc-def1-2345-6789abcdef01
         Total devices 8 FS bytes used 1.05TiB
         devid    1 size 268.05GiB used 265.94GiB path /dev/sda1
         devid    2 size 279.40GiB used 277.22GiB path /dev/sdb
         devid    3 size 279.40GiB used 277.32GiB path /dev/sdc
         devid    4 size 279.40GiB used 278.73GiB path /dev/sdd
         devid    5 size 279.40GiB used 277.72GiB path /dev/sde
         devid    6 size 279.40GiB used 278.61GiB path /dev/sdf
         devid    7 size 279.40GiB used 278.82GiB path /dev/sdg
         devid    8 size 465.76GiB used 230.99GiB path /dev/sdh

# btrfs filesystem usage -T /
Overall:
     Device size:                   2.35TiB
     Device allocated:              2.11TiB
     Device unallocated:          244.83GiB
     Device missing:                  0.00B
     Used:                          2.11TiB
     Free (estimated):            122.57GiB      (min: 122.57GiB)
     Data ratio:                       2.00
     Metadata ratio:                   2.00
     Global reserve:              512.00MiB      (used: 0.00B)

              Data       Data      Metadata System
Id Path      RAID1      RAID10    RAID1    RAID1 Unallocated
-- --------- ---------- --------- -------- --------- -----------
  1 /dev/sda1          - 132.97GiB        -         - 135.08GiB
  2 /dev/sdb           - 138.61GiB        -         - 140.79GiB
  3 /dev/sdc           - 138.66GiB        -         - 140.74GiB
  4 /dev/sdd           - 138.87GiB  1.00GiB         - 139.53GiB
  5 /dev/sde     1.00GiB 137.86GiB  1.00GiB         - 139.53GiB
  6 /dev/sdf           - 138.81GiB  1.00GiB         - 139.59GiB
  7 /dev/sdg     1.00GiB 138.38GiB  1.00GiB  64.00MiB 138.96GiB
  8 /dev/sdh           - 113.46GiB  4.00GiB  64.00MiB 348.24GiB
-- --------- ---------- --------- -------- --------- -----------
    Total        1.00GiB   1.05TiB  4.00GiB  64.00MiB 1.29TiB
Used      1007.30MiB   1.05TiB  1.66GiB 400.00KiB

What I don't get is... how can I have 244.8 GB unallocated when the 
table below clearly shows that there is as much as 1.29TiB 
unallocated.... does not appear to make sense to me at least...

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs filesystem usage - Wrong Unallocated indications - RAID10
  2016-05-23 18:49 ` Zoiled
@ 2016-05-23 19:19   ` Hugo Mills
  0 siblings, 0 replies; 5+ messages in thread
From: Hugo Mills @ 2016-05-23 19:19 UTC (permalink / raw)
  To: Zoiled; +Cc: Marco Lorenzo Crociani, linux-btrfs

[-- Attachment #1: Type: text/plain, Size: 2715 bytes --]

On Mon, May 23, 2016 at 08:49:04PM +0200, Zoiled wrote:
[snip]
> For what it's worth...  I have a 8 disk (7x 300GB + 1x 500GB) data
> raid10, metadata raid1 setup and I get the following output of
> btrfs...
> 
> Label: 'xxyyzz'  uuid: 12345678-9abc-def1-2345-6789abcdef01
>         Total devices 8 FS bytes used 1.05TiB
>         devid    1 size 268.05GiB used 265.94GiB path /dev/sda1
>         devid    2 size 279.40GiB used 277.22GiB path /dev/sdb
>         devid    3 size 279.40GiB used 277.32GiB path /dev/sdc
>         devid    4 size 279.40GiB used 278.73GiB path /dev/sdd
>         devid    5 size 279.40GiB used 277.72GiB path /dev/sde
>         devid    6 size 279.40GiB used 278.61GiB path /dev/sdf
>         devid    7 size 279.40GiB used 278.82GiB path /dev/sdg
>         devid    8 size 465.76GiB used 230.99GiB path /dev/sdh
> 
> # btrfs filesystem usage -T /
> Overall:
>     Device size:                   2.35TiB
>     Device allocated:              2.11TiB
>     Device unallocated:          244.83GiB
>     Device missing:                  0.00B
>     Used:                          2.11TiB
>     Free (estimated):            122.57GiB      (min: 122.57GiB)
>     Data ratio:                       2.00
>     Metadata ratio:                   2.00
>     Global reserve:              512.00MiB      (used: 0.00B)
> 
>              Data       Data      Metadata System
> Id Path      RAID1      RAID10    RAID1    RAID1 Unallocated
> -- --------- ---------- --------- -------- --------- -----------
>  1 /dev/sda1          - 132.97GiB        -         - 135.08GiB
>  2 /dev/sdb           - 138.61GiB        -         - 140.79GiB
>  3 /dev/sdc           - 138.66GiB        -         - 140.74GiB
>  4 /dev/sdd           - 138.87GiB  1.00GiB         - 139.53GiB
>  5 /dev/sde     1.00GiB 137.86GiB  1.00GiB         - 139.53GiB
>  6 /dev/sdf           - 138.81GiB  1.00GiB         - 139.59GiB
>  7 /dev/sdg     1.00GiB 138.38GiB  1.00GiB  64.00MiB 138.96GiB
>  8 /dev/sdh           - 113.46GiB  4.00GiB  64.00MiB 348.24GiB
> -- --------- ---------- --------- -------- --------- -----------
>    Total        1.00GiB   1.05TiB  4.00GiB  64.00MiB 1.29TiB
> Used      1007.30MiB   1.05TiB  1.66GiB 400.00KiB
> 
> What I don't get is... how can I have 244.8 GB unallocated when the
> table below clearly shows that there is as much as 1.29TiB
> unallocated.... does not appear to make sense to me at least...

   This is exactly the issue. The Unallocated value(s) from btrfs fi
usage on at least RAID-10 are simply wrong, any way you look at it.

   Hugo.

-- 
Hugo Mills             | Great films about cricket: The Third Man
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs filesystem usage - Wrong Unallocated indications - RAID10
  2016-05-23 13:34 btrfs filesystem usage - Wrong Unallocated indications - RAID10 Marco Lorenzo Crociani
  2016-05-23 18:34 ` Nicholas D Steeves
  2016-05-23 18:49 ` Zoiled
@ 2016-05-25 20:14 ` Henk Slager
  2 siblings, 0 replies; 5+ messages in thread
From: Henk Slager @ 2016-05-25 20:14 UTC (permalink / raw)
  To: linux-btrfs

There is a division by 2 missing in the code. With that added, the
RAID10 numbers make more sense. See also:
http://permalink.gmane.org/gmane.comp.file-systems.btrfs/53989

More detail in here:
https://www.spinics.net/lists/linux-btrfs/msg52882.html

And if you want to look at allocation in a different way, this might
be interesting:
https://github.com/knorrie/btrfs-heatmap/blob/master/show_usage.py

On Mon, May 23, 2016 at 3:34 PM, Marco Lorenzo Crociani
<marcoc@prismatelecomtesting.com> wrote:
> Hi,
> as I wrote today in IRCI experienced an issue with 'btrfs filesystem usage'.
> I have a 4 partitions RAID10 btrfs filesystem almost full.
> 'btrfs filesystem usage' reports wrong "Unallocated" indications.
>
> Linux 4.5.3
> btrfs-progs v4.5.3
>
>
> # btrfs fi usage /data/
>
> Overall:
>     Device size:          13.93TiB
>     Device allocated:          13.77TiB
>     Device unallocated:         167.54GiB
>     Device missing:             0.00B
>     Used:              13.44TiB
>     Free (estimated):         244.39GiB    (min: 244.39GiB)
>     Data ratio:                  2.00
>     Metadata ratio:              2.00
>     Global reserve:         512.00MiB    (used: 0.00B)
>
> Data,single: Size:8.00MiB, Used:0.00B
>    /dev/sda4       8.00MiB
>
> Data,RAID10: Size:6.87TiB, Used:6.71TiB
>    /dev/sda4       1.72TiB
>    /dev/sdb3       1.72TiB
>    /dev/sdc3       1.72TiB
>    /dev/sdd3       1.72TiB
>
> Metadata,single: Size:8.00MiB, Used:0.00B
>    /dev/sda4       8.00MiB
>
> Metadata,RAID10: Size:19.00GiB, Used:14.15GiB
>    /dev/sda4       4.75GiB
>    /dev/sdb3       4.75GiB
>    /dev/sdc3       4.75GiB
>    /dev/sdd3       4.75GiB
>
> System,single: Size:4.00MiB, Used:0.00B
>    /dev/sda4       4.00MiB
>
> System,RAID10: Size:16.00MiB, Used:768.00KiB
>    /dev/sda4       4.00MiB
>    /dev/sdb3       4.00MiB
>    /dev/sdc3       4.00MiB
>    /dev/sdd3       4.00MiB
>
> Unallocated:
>    /dev/sda4       1.76TiB
>    /dev/sdb3       1.76TiB
>    /dev/sdc3       1.76TiB
>    /dev/sdd3       1.76TiB
>
> ----------------------------------------------------------------------------------
> # btrfs fi show /data/
> Label: 'data'  uuid: df6639d5-3ef2-4ff6-a871-9ede440e2dae
>     Total devices 4 FS bytes used 6.72TiB
>     devid    1 size 3.48TiB used 3.44TiB path /dev/sda4
>     devid    2 size 3.48TiB used 3.44TiB path /dev/sdb3
>     devid    3 size 3.48TiB used 3.44TiB path /dev/sdc3
>     devid    4 size 3.48TiB used 3.44TiB path /dev/sdd3
>
> ----------------------------------------------------------------------------------
> # btrfs fi df /data/
> Data, RAID10: total=6.87TiB, used=6.71TiB
> Data, single: total=8.00MiB, used=0.00B
> System, RAID10: total=16.00MiB, used=768.00KiB
> System, single: total=4.00MiB, used=0.00B
> Metadata, RAID10: total=19.00GiB, used=14.15GiB
> Metadata, single: total=8.00MiB, used=0.00B
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
> ----------------------------------------------------------------------------------
> # df -h
> /dev/sda4                     7,0T  6,8T    245G  97% /data
>
> Regards,
>
> --
> Marco Crociani
> Prisma Telecom Testing S.r.l.
> via Petrocchi, 4  20127 MILANO  ITALY
> Phone:  +39 02 26113507
> Fax:  +39 02 26113597
> e-mail:  marcoc@prismatelecomtesting.com
> web:  http://www.prismatelecomtesting.com
>
> Questa email (e I suoi allegati) costituisce informazione riservata e
> confidenziale e può essere soggetto a legal privilege. Può essere utilizzata
> esclusivamente dai suoi destinatari legittimi.  Se avete ricevuto questa
> email per errore, siete pregati di informarne immediatamente il mittente e
> quindi cancellarla.  A meno che non siate stati a ciò espressamente
> autorizzati, la diffusione o la riproduzione di questa email o del suo
> contenuto non sono consentiti.
>
>  Salvo che questa email sia espressamente qualificata come offerta o
> accettazione contrattuale, il mittente non intende con questa email dare
> vita ad un vincolo giuridico e questa email non può essere interpretata
> quale offerta o accettazione che possa dare vita ad un contratto. Qualsiasi
> opinione manifestata in questa email è un'opinione personale del mittente,
> salvo che il mittente dichiari espressamente che si tratti di un'opinione di
> Prisma Engineering.
>
>
> *******************************************************************************
>
>  This e-mail (including any attachments) is private and confidential, and
> may be privileged.  It is for the exclusive use of the intended
> recipient(s).  If you have received this email in error, please inform the
> sender immediately and then delete this email.  Unless you have been given
> specific permission to do so, please do not distribute or copy this email or
> its contents.
>  Unless the text of this email specifically states that it is a contractual
> offer or acceptance, the sender does not intend to create a legal
> relationship and this email shall not constitute an offer or acceptance
> which could give rise to a contract. Any views expressed in this
> communication are those of the individual sender, except where the sender
> specifically states them to be the views of Prisma Engineering.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2016-05-25 20:14 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-05-23 13:34 btrfs filesystem usage - Wrong Unallocated indications - RAID10 Marco Lorenzo Crociani
2016-05-23 18:34 ` Nicholas D Steeves
2016-05-23 18:49 ` Zoiled
2016-05-23 19:19   ` Hugo Mills
2016-05-25 20:14 ` Henk Slager

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.