* Disk quota exceeded
@ 2015-12-19 10:38 Matthew Jurgens
2015-12-20 8:50 ` Duncan
2015-12-21 2:17 ` Qu Wenruo
0 siblings, 2 replies; 6+ messages in thread
From: Matthew Jurgens @ 2015-12-19 10:38 UTC (permalink / raw)
To: linux-btrfs
I've create a subvolume
CMD: btrfs quota enable /export/shared
CMD: btrfs subvol create /export/shared/TV/files
Create subvolume '/export/shared/TV/files'
CMD: btrfs quota rescan /export/shared/TV/files
I've assigned a small quota to it:
CMD: btrfs qgroup limit 100M /export/shared/TV/files
I've also disabled copy on write:
CMD: chattr -R -V +C /export/shared/TV/files
chattr 1.42.12 (29-Aug-2014)
Flags of /export/shared/TV/files set as ---------------C
CMD: sync; btrfs qgroup show -r /export/shared/TV/files
qgroupid rfer excl max_rfer
-------- ---- ---- --------
0/5 2.11TiB 2.11TiB none
0/1894 16.00KiB 16.00KiB 100.00MiB
I created one file until I exceeded the quota using:
CMD: dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
CMD: dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
dd: error writing ‘/export/shared/TV/files/junk1’: Disk quota exceeded
50+0 records in
49+0 records out
51380224 bytes (51 MB) copied, 0.0238218 s, 2.2 GB/s
then I removed the file
CMD: rm /export/shared/TV/files/junk1
I tried to create it again (and only got 900k written before the quota
was exceeded):
CMD: dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
dd: error writing ‘/export/shared/TV/files/junk1’: Disk quota exceeded
1+0 records in
0+0 records out
917504 bytes (918 kB) copied, 0.000568984 s, 1.6 GB/s
then I removed the file
CMD: rm /export/shared/TV/files/junk1
I tried to create it again (and this time got 900k again written before
the quota was exceeded):
dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
dd: error writing ‘/export/shared/TV/files/junk1’: Disk quota exceeded
1+0 records in
0+0 records out
917504 bytes (918 kB) copied, 0.000943652 s, 972 MB/s
The usage looks ok:
CMD: sync; btrfs qgroup show -r /export/shared/TV/files
qgroupid rfer excl max_rfer
-------- ---- ---- --------
0/5 3.10TiB 3.10TiB none
0/1894 912.00KiB 912.00KiB 100.00MiB
But I can not create any more files:
CMD: dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
dd: failed to open ‘/export/shared/TV/files/junk1’: Disk quota exceeded
CMD: lsattr /export/shared/TV/files
---------------C /export/shared/TV/files/junk1
During other tests I have managed to get it to the point where I can't
delete the file (since disk quota exceeded).
I've waited many minutes to ensure commits etc. The only way I can
seemingly consistently get the ability to write to the directory again
is to reboot. Immediately after a reboot I can create files as expected.
It seems like copy on write is still functioning for this directory? or
quota results have delays?
I've tried mounting the subvolume to another location, but same results.
Is this happening because quotas are not yet considered stable?
Is there something I am doing wrong?
Other Info:
The fstab entry:
LABEL=SHARED /export/shared btrfs
defaults,compress=lzo,relatime,nofail 0 1
uname -a
Linux gw.lambert.rd.to 4.2.7-200.fc22.x86_64 #1 SMP Thu Dec 10 03:28:47
UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
btrfs --version
btrfs-progs v4.2.2
btrfs fi show
Label: 'SHARED' uuid: 199672d4-69ef-4ecf-865a-851bed0167d5
Total devices 5 FS bytes used 3.10TiB
devid 1 size 2.73TiB used 1.27TiB path /dev/sdc
devid 2 size 2.73TiB used 1.27TiB path /dev/sdf
devid 3 size 2.73TiB used 1.27TiB path /dev/sdh
devid 4 size 2.73TiB used 1.27TiB path /dev/sde
devid 5 size 2.73TiB used 1.27TiB path /dev/sdd
btrfs fi df /export/shared
Data, RAID10: total=3.16TiB, used=3.09TiB
System, RAID10: total=64.00MiB, used=352.00KiB
Metadata, RAID10: total=9.00GiB, used=7.61GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
dmesg:
http://edcint.co.nz/tmp/dmesg_20151219.log
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Disk quota exceeded
2015-12-19 10:38 Disk quota exceeded Matthew Jurgens
@ 2015-12-20 8:50 ` Duncan
2015-12-21 2:17 ` Qu Wenruo
1 sibling, 0 replies; 6+ messages in thread
From: Duncan @ 2015-12-20 8:50 UTC (permalink / raw)
To: linux-btrfs
Matthew Jurgens posted on Sat, 19 Dec 2015 21:38:50 +1100 as excerpted:
> I've create a subvolume [and] assigned a small quota to it:
> CMD: btrfs qgroup limit 100M /export/shared/TV/files
> I've also disabled copy on write:
> CMD: chattr -R -V +C /export/shared/TV/files
[wrote only ~50 M of files, can't write more]
> It seems like copy on write is still functioning for this directory? or
> quota results have delays?
> Is this happening because quotas are not yet considered stable?
> Is there something I am doing wrong?
> uname [-r] 4.2.7-200.fc22.x86_64
> btrfs --version btrfs-progs v4.2.2
[Disclaimer: List regular and btrfs user. Not a dev. My own use-case
doesn't involve quotas or even subvolumes, so the following is based
purely on what I've seen on-list.]
Quotas are definitely not stable, thru 4.3 at least, and you're running
4.2.
IIRC They've on the product of the second major rewrite (so third try)
now and there have always been issues. While there's hard work going
into fixing the issues, full fixes have remained at least a kernel cycle
or two out for quite some time now. AFAIK, 4.4 is supposed to fix some
more issues, but I'm not sure if it fixes all known issues yet or if
there's more remaining, and even if it fixes all known issues, I'd
suggest waiting at least a complete kernel cycle and preferably two with
no known issues, before considering quotas actually reasonably
dependable, as given the history, there's a fair chance more issues will
popup.
Additionally, quotas thru at least 4.3 still have bad scaling issues that
slow down maintenance such as balance. (I'm not sure if check is
affected or not, but scrub shouldn't be.)
So I'd strongly suggest leaving quotas off thru the 4.4 cycle at least.
If 4.4 and the 4.5 development cycle ends up with no further issues, you
may wish to consider enabling them for 4.5, tho more conservative users
will wait another cycle and see if there's no further issues reported
thru the 4.6 development cycle as well, enabling them once they're on it.
(Since 4.4 will be an LTS kernel, if you're sticking to it, you could
consider enabling quotas on updates after 4.5 or 4.6 release, if there
have been no reports of quota issues on 4.4 by then.)
Meanwhile, a few more quota related factors to consider:
1) As I'm not a quota user I can't say for sure on this, but it seems
mighty coincidental that it failed at just over half the set quota... and
you're running btrfs raid10 data, so btrfs is keeping two copies and
doubling the usage. Maybe it's tracking the actual doubled due to the
raid10 usage instead of the single-copy value? (This actually sounds
like some of the sort of internal logic bugs they've been fixing tho I'm
unaware of one exactly like this, so you might try a current kernel 4.4-rc
and see if the results are different, and/or try it on a different btrfs
with replication set to single mode.)
2) Btrfs metadata is always copy-on-write. That can't be disabled.
Disabling COW only disables it for data.
I remember some discussion of whether quotas should track data only or
include metadata as well, or possibly make it optional to track both or
set quotas separately for each, but IDR the results of those discussions
and as I don't use them myself...
That would explain not being able to delete the file at times, because
doing so would change the metadata and thus require COWing it, and if
it's quota tracked and reporting over quota errors...
3) While this shouldn't apply to the scenario above because you followed
the recommendation and set nocow on the dir and /then/ created the file,
so the file should have inherited the nocow, and I suppose you know this
already, but for the benefit of others following alone, it should be
noted that nocow doesn't necessarily take effect on files that already
have content -- they need to be zero-sized for it to take effect.
The recommended procedure to avoid the problem is exactly what you did,
set nocow on the dir and take care to either create new files (as you
did) or copy/move them into place without reflinking, so they're actually
created before they have any content, so the nocow inherited from the dir
will actually take effect.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Disk quota exceeded
2015-12-19 10:38 Disk quota exceeded Matthew Jurgens
2015-12-20 8:50 ` Duncan
@ 2015-12-21 2:17 ` Qu Wenruo
1 sibling, 0 replies; 6+ messages in thread
From: Qu Wenruo @ 2015-12-21 2:17 UTC (permalink / raw)
To: Matthew Jurgens, linux-btrfs
Matthew Jurgens wrote on 2015/12/19 21:38 +1100:
> I've create a subvolume
> CMD: btrfs quota enable /export/shared
> CMD: btrfs subvol create /export/shared/TV/files
> Create subvolume '/export/shared/TV/files'
> CMD: btrfs quota rescan /export/shared/TV/files
>
> I've assigned a small quota to it:
> CMD: btrfs qgroup limit 100M /export/shared/TV/files
>
> I've also disabled copy on write:
> CMD: chattr -R -V +C /export/shared/TV/files
> chattr 1.42.12 (29-Aug-2014)
> Flags of /export/shared/TV/files set as ---------------C
>
> CMD: sync; btrfs qgroup show -r /export/shared/TV/files
> qgroupid rfer excl max_rfer
> -------- ---- ---- --------
> 0/5 2.11TiB 2.11TiB none
> 0/1894 16.00KiB 16.00KiB 100.00MiB
>
> I created one file until I exceeded the quota using:
> CMD: dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
> CMD: dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
> dd: error writing ‘/export/shared/TV/files/junk1’: Disk quota exceeded
> 50+0 records in
> 49+0 records out
> 51380224 bytes (51 MB) copied, 0.0238218 s, 2.2 GB/s
>
> then I removed the file
> CMD: rm /export/shared/TV/files/junk1
>
> I tried to create it again (and only got 900k written before the quota
> was exceeded):
> CMD: dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
> dd: error writing ‘/export/shared/TV/files/junk1’: Disk quota exceeded
> 1+0 records in
> 0+0 records out
> 917504 bytes (918 kB) copied, 0.000568984 s, 1.6 GB/s
>
> then I removed the file
> CMD: rm /export/shared/TV/files/junk1
>
> I tried to create it again (and this time got 900k again written before
> the quota was exceeded):
> dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
> dd: error writing ‘/export/shared/TV/files/junk1’: Disk quota exceeded
> 1+0 records in
> 0+0 records out
> 917504 bytes (918 kB) copied, 0.000943652 s, 972 MB/s
>
> The usage looks ok:
> CMD: sync; btrfs qgroup show -r /export/shared/TV/files
> qgroupid rfer excl max_rfer
> -------- ---- ---- --------
> 0/5 3.10TiB 3.10TiB none
> 0/1894 912.00KiB 912.00KiB 100.00MiB
The excl/rfer numbers will always be accurate, as that what 4.2 quota
rework does.
>
> But I can not create any more files:
> CMD: dd if=/dev/zero of=/export/shared/TV/files/junk1 count=50 bs=1M
> dd: failed to open ‘/export/shared/TV/files/junk1’: Disk quota exceeded
That's what exactly I fixed in 4.3-rc1.
And the operation to trigger the bug is almost the same with xfstest
test case: btrfs/099.
Btrfs doesn't handle quota reserved space well, and caused reserved
space leak. (fixed in 4.3-rc1, but more test would be better)
As long as you did overwrite, no matter whether you enabled or disabled
CoW, it will leak reserved space, and the reserved space won't be freed.
Finally eating up your quota space.
You can either try latest 4.3-rc, which does not only include the quota
fix, but also a lot of delayed ref fixes.
Or, there is a workaround:
Umount the fs and mount it again.
This will free the leaked space and make it work again for a short time,
until your next overwrite.
Thanks,
Qu
>
> CMD: lsattr /export/shared/TV/files
> ---------------C /export/shared/TV/files/junk1
>
> During other tests I have managed to get it to the point where I can't
> delete the file (since disk quota exceeded).
> I've waited many minutes to ensure commits etc. The only way I can
> seemingly consistently get the ability to write to the directory again
> is to reboot. Immediately after a reboot I can create files as expected.
>
> It seems like copy on write is still functioning for this directory? or
> quota results have delays?
> I've tried mounting the subvolume to another location, but same results.
>
> Is this happening because quotas are not yet considered stable?
> Is there something I am doing wrong?
>
>
>
>
> Other Info:
> The fstab entry:
> LABEL=SHARED /export/shared btrfs
> defaults,compress=lzo,relatime,nofail 0 1
>
> uname -a
> Linux gw.lambert.rd.to 4.2.7-200.fc22.x86_64 #1 SMP Thu Dec 10 03:28:47
> UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
>
> btrfs --version
> btrfs-progs v4.2.2
>
> btrfs fi show
> Label: 'SHARED' uuid: 199672d4-69ef-4ecf-865a-851bed0167d5
> Total devices 5 FS bytes used 3.10TiB
> devid 1 size 2.73TiB used 1.27TiB path /dev/sdc
> devid 2 size 2.73TiB used 1.27TiB path /dev/sdf
> devid 3 size 2.73TiB used 1.27TiB path /dev/sdh
> devid 4 size 2.73TiB used 1.27TiB path /dev/sde
> devid 5 size 2.73TiB used 1.27TiB path /dev/sdd
>
> btrfs fi df /export/shared
> Data, RAID10: total=3.16TiB, used=3.09TiB
> System, RAID10: total=64.00MiB, used=352.00KiB
> Metadata, RAID10: total=9.00GiB, used=7.61GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
> dmesg:
> http://edcint.co.nz/tmp/dmesg_20151219.log
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Disk quota exceeded
2016-06-29 17:19 ` Duncan
@ 2016-06-29 19:20 ` Benoit GEORGELIN - Association Web4all
0 siblings, 0 replies; 6+ messages in thread
From: Benoit GEORGELIN - Association Web4all @ 2016-06-29 19:20 UTC (permalink / raw)
To: Duncan; +Cc: linux-btrfs
Hi Duncan,
Thanks a lot for your answer. This give me a lot of information and a better idea of what I should do.
I just upgraded the kernel to 4.6.3-040603-generic and i will see if it's working better with this kernel.
If not, I will have to look to another FS with a more stable quotas :)
Cordialement,
Benoît
Afin de contribuer au respect de l'environnement, merci de n'imprimer ce mail qu'en cas de nécessité
----- Mail original -----
De: "Duncan" <1i5t5.duncan@cox.net>
À: "linux-btrfs" <linux-btrfs@vger.kernel.org>
Envoyé: Mercredi 29 Juin 2016 13:19:35
Objet: Re: Disk quota exceeded
Benoit GEORGELIN - Association Web4all posted on Wed, 29 Jun 2016 18:15:53
+0200 as excerpted:
> Hi dear users of the list.
>
> I'm new to the BTRFS file-system and I am having some problems with
> quotas I would like to share with you what I'm facing about "Disk quota
> exceeded" it's quite strange to me.
Three points.
1) Btrfs quotas have a long history of trouble, both in correctness and
in scaling ability (btrfs balances and checks can take far longer with
quotas enabled, often too long to be practical), and if they are now
stable, it's only a very recent development. As a result, my
recommendation has been and remains not to use btrfs quotas unless you
are specifically doing so to work with the devs to test and bugfix. If
you don't need them, simply turn them off and leave them off until they
are known to work. If you do actually need quota functionality, you're
far better off using a filesystem where quotas are actually stable and
work as intended.
> - Description of the environment I have a BTRFS volume "/var/lib/lxd"
> I have some subvolume into "/var/lib/lxd/containers"
>
> The volume have quota enable Every subvolume have their quota groupe
> created Limit of 10Go has been set to every single subvolumes
>
> -The problem Randomly I can't write to some of the subvolumes anymore
> without having datas added into the subvolume.
> I can't figure why and the subvolumes are randomly not usable with the
> error : Disk quota exceeded
2) I'm not sure of the current status here, but it's worth keeping in
mind that unlike other filesystems, btrfs separates data and metadata,
and it's possible for data to be under quota and still run out, if
metadata takes you over quota.
There was some talk a kernel cycle or two ago suggesting separating data
and metadata quotas and allowing setting them either separately or
overall capping them, but as I said, I'm not sure what the status is on
that.
> I did delete all the quota groups, disable quota on the volume, enable
> the quota group again on the volume, check if quota groups were back
> automatically as btrfs version 4 is supposed to do I added back the
> limit 10Go Rescan Everything work just fine but some time after, I'm
> back to over-quota on some subvolumes.
>
> Thanks for your help on this.
>
> Below, more details about the config and system point of view :
>
>
> uname -a Linux lxd-virt-01b 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26
> 00:58:07 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
3) Unfortunately, ubuntu chose what from upstream's viewpoint was a short-
term-stable kernel. 4.2 is no longer supported upstream, and while it's
possible that ubuntu is backporting as appropriate, on this list we track
upstream and don't know what individual distros have backported and what
they haven't.
And I know for sure that 4.2 still had serious quota issues that are
unlikely to have been fixed with backports because they never /did/ work
right by 4.2, so the quota issues weren't regressions and thus patches to
fix them would have been unlikely stable candidates.
You thus have a few choices:
4.1 and 4.4 are upstream LTS series, and continue to be supported (tho
I'd suggest 4.4 if you'll be adopting one, as btrfs is still developing
fast enough that we only tend to well support the last couple LTS, and
beyond that, while critical patches will be backported to the extent
possible, if you have a problem you're still likely to be asked to
upgrade and see if the problem is still there if you're out of that 2 LTS
series window, and 4.1 is getting a bit long in the tooth to be just
upgrading to now). So one choice is to move to the latest version of one
of them.
Another choice would be to continue with 4.2, but get your support from
your distro, who is better qualified to support it as they know what
they've backported and what they haven't.
And another choice is upgrade to current kernels, where again we
generally support the last two series, 4.6 and 4.5 ATM.
(FWIW, userspace version numbers are synced to kernelspace numbers.
Backward compatibility is supported, so a good rule of thumb there is to
run btrfs userspace at least as new as your kernel, which assuming you're
staying in line with the latest two kernels of either current or LTS
series, will mean your userspace doesn't get too far behind either. But
you're on 4.4 userspace already, so are fine there.)
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Disk quota exceeded
2016-06-29 16:15 ` Benoit GEORGELIN - Association Web4all
@ 2016-06-29 17:19 ` Duncan
2016-06-29 19:20 ` Benoit GEORGELIN - Association Web4all
0 siblings, 1 reply; 6+ messages in thread
From: Duncan @ 2016-06-29 17:19 UTC (permalink / raw)
To: linux-btrfs
Benoit GEORGELIN - Association Web4all posted on Wed, 29 Jun 2016 18:15:53
+0200 as excerpted:
> Hi dear users of the list.
>
> I'm new to the BTRFS file-system and I am having some problems with
> quotas I would like to share with you what I'm facing about "Disk quota
> exceeded" it's quite strange to me.
Three points.
1) Btrfs quotas have a long history of trouble, both in correctness and
in scaling ability (btrfs balances and checks can take far longer with
quotas enabled, often too long to be practical), and if they are now
stable, it's only a very recent development. As a result, my
recommendation has been and remains not to use btrfs quotas unless you
are specifically doing so to work with the devs to test and bugfix. If
you don't need them, simply turn them off and leave them off until they
are known to work. If you do actually need quota functionality, you're
far better off using a filesystem where quotas are actually stable and
work as intended.
> - Description of the environment I have a BTRFS volume "/var/lib/lxd"
> I have some subvolume into "/var/lib/lxd/containers"
>
> The volume have quota enable Every subvolume have their quota groupe
> created Limit of 10Go has been set to every single subvolumes
>
> -The problem Randomly I can't write to some of the subvolumes anymore
> without having datas added into the subvolume.
> I can't figure why and the subvolumes are randomly not usable with the
> error : Disk quota exceeded
2) I'm not sure of the current status here, but it's worth keeping in
mind that unlike other filesystems, btrfs separates data and metadata,
and it's possible for data to be under quota and still run out, if
metadata takes you over quota.
There was some talk a kernel cycle or two ago suggesting separating data
and metadata quotas and allowing setting them either separately or
overall capping them, but as I said, I'm not sure what the status is on
that.
> I did delete all the quota groups, disable quota on the volume, enable
> the quota group again on the volume, check if quota groups were back
> automatically as btrfs version 4 is supposed to do I added back the
> limit 10Go Rescan Everything work just fine but some time after, I'm
> back to over-quota on some subvolumes.
>
> Thanks for your help on this.
>
> Below, more details about the config and system point of view :
>
>
> uname -a Linux lxd-virt-01b 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26
> 00:58:07 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
3) Unfortunately, ubuntu chose what from upstream's viewpoint was a short-
term-stable kernel. 4.2 is no longer supported upstream, and while it's
possible that ubuntu is backporting as appropriate, on this list we track
upstream and don't know what individual distros have backported and what
they haven't.
And I know for sure that 4.2 still had serious quota issues that are
unlikely to have been fixed with backports because they never /did/ work
right by 4.2, so the quota issues weren't regressions and thus patches to
fix them would have been unlikely stable candidates.
You thus have a few choices:
4.1 and 4.4 are upstream LTS series, and continue to be supported (tho
I'd suggest 4.4 if you'll be adopting one, as btrfs is still developing
fast enough that we only tend to well support the last couple LTS, and
beyond that, while critical patches will be backported to the extent
possible, if you have a problem you're still likely to be asked to
upgrade and see if the problem is still there if you're out of that 2 LTS
series window, and 4.1 is getting a bit long in the tooth to be just
upgrading to now). So one choice is to move to the latest version of one
of them.
Another choice would be to continue with 4.2, but get your support from
your distro, who is better qualified to support it as they know what
they've backported and what they haven't.
And another choice is upgrade to current kernels, where again we
generally support the last two series, 4.6 and 4.5 ATM.
(FWIW, userspace version numbers are synced to kernelspace numbers.
Backward compatibility is supported, so a good rule of thumb there is to
run btrfs userspace at least as new as your kernel, which assuming you're
staying in line with the latest two kernels of either current or LTS
series, will mean your userspace doesn't get too far behind either. But
you're on 4.4 userspace already, so are fine there.)
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 6+ messages in thread
* Disk quota exceeded
[not found] <1872105242.302575.1467216517408.JavaMail.zimbra@web4all.fr>
@ 2016-06-29 16:15 ` Benoit GEORGELIN - Association Web4all
2016-06-29 17:19 ` Duncan
0 siblings, 1 reply; 6+ messages in thread
From: Benoit GEORGELIN - Association Web4all @ 2016-06-29 16:15 UTC (permalink / raw)
To: linux-btrfs
Hi dear users of the list.
I'm new to the BTRFS file-system and I am having some problems with quotas
I would like to share with you what I'm facing about "Disk quota exceeded" it's quite strange to me.
- Description of the environment
I have a BTRFS volume "/var/lib/lxd"
I have some subvolume into "/var/lib/lxd/containers"
The volume have quota enable
Every subvolume have their quota groupe created
Limit of 10Go has been set to every single subvolumes
-The problem
Randomly I can't write to some of the subvolumes anymore without having datas added into the subvolume.
I can't figure why and the subvolumes are randomly not usable with the error : Disk quota exceeded
I did delete all the quota groups, disable quota on the volume, enable the quota group again on the volume, check if quota groups were back automatically as btrfs version 4 is supposed to do
I added back the limit 10Go
Rescan
Everything work just fine but some time after, I'm back to over-quota on some subvolumes.
Thanks for your help on this.
Below, more details about the config and system point of view :
uname -a
Linux lxd-virt-01b 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
btrfs --version
btrfs-progs v4.4
btrfs fi show
Label: none uuid: b0b183ed-9970-4e54-b7d5-7b91f3bdea10
Total devices 2 FS bytes used 21.01GiB
devid 1 size 50.00GiB used 13.02GiB path /dev/sdb
devid 2 size 50.00GiB used 13.01GiB path /dev/sdc
btrfs fi df /var/lib/lxd
Data, RAID0: total=22.00GiB, used=20.36GiB
System, RAID1: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, RAID1: total=2.00GiB, used=660.97MiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=224.00MiB, used=0.00B
# Simple script that check the content of /var/lib/lxd/containers were subvolumes stands
# Try to touch a file into subvolume_name/rootfs/
./check_quotas_rw.sh
Fichier écrit sur /var/lib/lxd/containers/angristan/rootfs/tmp/test
touch: cannot touch ‘/var/lib/lxd/containers/ben/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
Fichier écrit sur /var/lib/lxd/containers/benltsp/rootfs/tmp/test
touch: cannot touch ‘/var/lib/lxd/containers/debsid/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
Fichier écrit sur /var/lib/lxd/containers/dworak-vps/rootfs/tmp/test
Fichier écrit sur /var/lib/lxd/containers/heimdaall/rootfs/tmp/test
Fichier écrit sur /var/lib/lxd/containers/louisjo/rootfs/tmp/test
touch: cannot touch ‘/var/lib/lxd/containers/onclben/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
touch: cannot touch ‘/var/lib/lxd/containers/testdisk/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
Fichier écrit sur /var/lib/lxd/containers/tyrower/rootfs/tmp/test
touch: cannot touch ‘/var/lib/lxd/containers/vega/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
# Script to check group id so you can correlate
/btrfsQuota.py --unit M /var/lib/lxd
subvol group total unshared
-------------------------------------------------------------------------------
(unknown) 0/5 649.89M 649.89M
containers/ben 0/261 1297.45M 1297.45M
images/0d07f11f3f2a0804f501967d278d72a8122b1ec49f01aae4483a41fd9fb546f3.btrfs 0/264 315.33M 315.33M
containers/vega 0/336 3211.92M 3211.92M
containers/onclben 0/391 384.45M 384.45M
images/61ba8f528725bda896eb6ca007f29f2d90a3f375379292e6a31eb851eb33e5ab.btrfs 0/423 563.02M 563.02M
containers/heimdaall 0/428 1936.61M 1653.59M
containers/tyrower 0/429 384.33M 101.31M
containers/angristan 0/435 1132.71M 846.59M
containers/dworak-vps 0/437 384.74M 98.62M
containers/louisjo 0/442 663.09M 377.49M
containers/benltsp 0/443 9209.45M 8933.05M
images/f452cda3bccb2903e56d53e402b9d35334b4276783d098a879be5d74b04e62e2.btrfs 0/448 631.32M 631.32M
containers/testdisk 0/449 383.99M 383.99M
containers/debsid 0/453 428.52M 428.52M
images/96a08bb9c674e092fe99dd2f96d61f8f0a2a1ea80eb01d384274c1ff8efefce1.btrfs 0/457 328.57M 328.57M
images/fca47fbd0bb34bd38fdbd67bcc90a00eb8f0420b29a048ff59125d5fc983016c.btrfs 0/458 315.15M 315.15M
images/c61ac213e75fb99e40d5ac49157314f002db257f38a692fc98c8d4be867f18a6.btrfs 0/463 367.45M 367.45M
btrfs qg show -r -e /var/lib/lxd -p -c
qgroupid rfer excl max_rfer max_excl parent child
-------- ---- ---- -------- -------- ------ -----
0/5 649.89MiB 649.89MiB none none --- ---
0/261 1.27GiB 1.27GiB 10.00GiB none --- ---
0/264 315.33MiB 315.33MiB none none --- ---
0/336 3.14GiB 3.14GiB 10.00GiB none --- ---
0/391 384.45MiB 384.45MiB 10.00GiB none --- ---
0/423 563.02MiB 563.02MiB none none --- ---
0/428 1.89GiB 1.61GiB 10.00GiB none --- ---
0/429 384.33MiB 101.31MiB 10.00GiB none --- ---
0/435 1.11GiB 846.59MiB 10.00GiB none --- ---
0/437 384.74MiB 98.62MiB 10.00GiB none --- ---
0/442 663.09MiB 377.49MiB 10.00GiB none --- ---
0/443 8.99GiB 8.72GiB 10.00GiB none --- ---
0/448 631.32MiB 631.32MiB none none --- ---
0/449 383.99MiB 383.99MiB 10.00GiB none --- ---
0/453 428.52MiB 428.52MiB 10.00GiB none --- ---
0/457 328.57MiB 328.57MiB none none --- ---
0/458 315.15MiB 315.15MiB none none --- ---
0/463 367.45MiB 367.45MiB none none --- ---
dmesg |grep -i btrfs
[ 2.712185] Btrfs loaded
[ 2.902305] BTRFS: device fsid b0b183ed-9970-4e54-b7d5-7b91f3bdea10 devid 2 transid 113894 /dev/sdc
[ 2.903883] BTRFS: device fsid b0b183ed-9970-4e54-b7d5-7b91f3bdea10 devid 1 transid 113894 /dev/sdb
[ 6.201318] BTRFS info (device sdb): disk space caching is enabled
[ 6.201321] BTRFS: has skinny extents
[ 39.732758] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 540.722625] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 1247.758789] BTRFS info (device sdb): qgroup_rescan_init failed with -115
[ 1265.879801] BTRFS info (device sdb): qgroup_rescan_init failed with -115
[ 1289.854705] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 1403.138613] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 1564.685350] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 1616.389847] BTRFS info (device sdb): The free space cache file (23651680256) is invalid. skip it
[ 1765.145922] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[36615.633743] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
Cordialement,
Benoît Georgelin - Expert Infrastructure
Web 4 all Hébergeur associatif
benoit.georgelin@web 4 all.fr
Afin de contribuer au respect de l'environnement, merci de n'imprimer ce mail qu'en cas de nécessité
----- Mail original -----
De: "Benoit GEORGELIN - Association Web4all" <benoit.georgelin@web4all.fr>
À: "linux-btrfs" <linux-btrfs@vger.kernel.org>
Envoyé: Mercredi 29 Juin 2016 12:08:37
Objet: Disk quota exceeded
I'm new to the BTRFS file-system and I am having some problems with quotas
I would like to share with you what I'm facing about "Disk quota exceeded" it's quite strange to me.
- Description of the environment
I have a BTRFS volume "/var/lib/lxd"
I have some subvolume into "/var/lib/lxd/containers"
The volume have quota enable
Every subvolume have their quota groupe created
Limit of 10Go has been set to every single subvolumes
-The problem
Randomly I can't write to some of the subvolumes anymore without having datas added into the subvolume.
I can't figure why and the subvolumes are randomly not usable with the error : Disk quota exceeded
I did delete all the quota groups, disable quota on the volume, enable the quota group again on the volume, check if quota groups were back automatically as btrfs version 4 is supposed to do
I added back the limit 10Go
Rescan
Everything work just fine but some time after, I'm back to over-quota on some subvolumes.
Thanks for your help on this.
Below, more details about the config and system point of view :
uname -a
Linux lxd-virt-01b 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
btrfs --version
btrfs-progs v4.4
btrfs fi show
Label: none uuid: b0b183ed-9970-4e54-b7d5-7b91f3bdea10
Total devices 2 FS bytes used 21.01GiB
devid 1 size 50.00GiB used 13.02GiB path /dev/sdb
devid 2 size 50.00GiB used 13.01GiB path /dev/sdc
btrfs fi df /var/lib/lxd
Data, RAID0: total=22.00GiB, used=20.36GiB
System, RAID1: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, RAID1: total=2.00GiB, used=660.97MiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=224.00MiB, used=0.00B
# Simple script that check the content of /var/lib/lxd/containers were subvolumes stands
# Try to touch a file into subvolume_name/rootfs/
./check_quotas_rw.sh
Fichier écrit sur /var/lib/lxd/containers/angristan/rootfs/tmp/test
touch: cannot touch ‘/var/lib/lxd/containers/ben/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
Fichier écrit sur /var/lib/lxd/containers/benltsp/rootfs/tmp/test
touch: cannot touch ‘/var/lib/lxd/containers/debsid/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
Fichier écrit sur /var/lib/lxd/containers/dworak-vps/rootfs/tmp/test
Fichier écrit sur /var/lib/lxd/containers/heimdaall/rootfs/tmp/test
Fichier écrit sur /var/lib/lxd/containers/louisjo/rootfs/tmp/test
touch: cannot touch ‘/var/lib/lxd/containers/onclben/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
touch: cannot touch ‘/var/lib/lxd/containers/testdisk/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
Fichier écrit sur /var/lib/lxd/containers/tyrower/rootfs/tmp/test
touch: cannot touch ‘/var/lib/lxd/containers/vega/rootfs/tmp/test’: Disk quota exceeded
Erreur de quota
# Script to check group id so you can correlate
/btrfsQuota.py --unit M /var/lib/lxd
subvol group total unshared
-------------------------------------------------------------------------------
(unknown) 0/5 649.89M 649.89M
containers/ben 0/261 1297.45M 1297.45M
images/0d07f11f3f2a0804f501967d278d72a8122b1ec49f01aae4483a41fd9fb546f3.btrfs 0/264 315.33M 315.33M
containers/vega 0/336 3211.92M 3211.92M
containers/onclben 0/391 384.45M 384.45M
images/61ba8f528725bda896eb6ca007f29f2d90a3f375379292e6a31eb851eb33e5ab.btrfs 0/423 563.02M 563.02M
containers/heimdaall 0/428 1936.61M 1653.59M
containers/tyrower 0/429 384.33M 101.31M
containers/angristan 0/435 1132.71M 846.59M
containers/dworak-vps 0/437 384.74M 98.62M
containers/louisjo 0/442 663.09M 377.49M
containers/benltsp 0/443 9209.45M 8933.05M
images/f452cda3bccb2903e56d53e402b9d35334b4276783d098a879be5d74b04e62e2.btrfs 0/448 631.32M 631.32M
containers/testdisk 0/449 383.99M 383.99M
containers/debsid 0/453 428.52M 428.52M
images/96a08bb9c674e092fe99dd2f96d61f8f0a2a1ea80eb01d384274c1ff8efefce1.btrfs 0/457 328.57M 328.57M
images/fca47fbd0bb34bd38fdbd67bcc90a00eb8f0420b29a048ff59125d5fc983016c.btrfs 0/458 315.15M 315.15M
images/c61ac213e75fb99e40d5ac49157314f002db257f38a692fc98c8d4be867f18a6.btrfs 0/463 367.45M 367.45M
btrfs qg show -r -e /var/lib/lxd -p -c
qgroupid rfer excl max_rfer max_excl parent child
-------- ---- ---- -------- -------- ------ -----
0/5 649.89MiB 649.89MiB none none --- ---
0/261 1.27GiB 1.27GiB 10.00GiB none --- ---
0/264 315.33MiB 315.33MiB none none --- ---
0/336 3.14GiB 3.14GiB 10.00GiB none --- ---
0/391 384.45MiB 384.45MiB 10.00GiB none --- ---
0/423 563.02MiB 563.02MiB none none --- ---
0/428 1.89GiB 1.61GiB 10.00GiB none --- ---
0/429 384.33MiB 101.31MiB 10.00GiB none --- ---
0/435 1.11GiB 846.59MiB 10.00GiB none --- ---
0/437 384.74MiB 98.62MiB 10.00GiB none --- ---
0/442 663.09MiB 377.49MiB 10.00GiB none --- ---
0/443 8.99GiB 8.72GiB 10.00GiB none --- ---
0/448 631.32MiB 631.32MiB none none --- ---
0/449 383.99MiB 383.99MiB 10.00GiB none --- ---
0/453 428.52MiB 428.52MiB 10.00GiB none --- ---
0/457 328.57MiB 328.57MiB none none --- ---
0/458 315.15MiB 315.15MiB none none --- ---
0/463 367.45MiB 367.45MiB none none --- ---
dmesg |grep -i btrfs
[ 2.712185] Btrfs loaded
[ 2.902305] BTRFS: device fsid b0b183ed-9970-4e54-b7d5-7b91f3bdea10 devid 2 transid 113894 /dev/sdc
[ 2.903883] BTRFS: device fsid b0b183ed-9970-4e54-b7d5-7b91f3bdea10 devid 1 transid 113894 /dev/sdb
[ 6.201318] BTRFS info (device sdb): disk space caching is enabled
[ 6.201321] BTRFS: has skinny extents
[ 39.732758] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 540.722625] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 1247.758789] BTRFS info (device sdb): qgroup_rescan_init failed with -115
[ 1265.879801] BTRFS info (device sdb): qgroup_rescan_init failed with -115
[ 1289.854705] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 1403.138613] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 1564.685350] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[ 1616.389847] BTRFS info (device sdb): The free space cache file (23651680256) is invalid. skip it
[ 1765.145922] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
[36615.633743] BTRFS info (device sdb): qgroup scan completed (inconsistency flag cleared)
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-06-29 19:20 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-12-19 10:38 Disk quota exceeded Matthew Jurgens
2015-12-20 8:50 ` Duncan
2015-12-21 2:17 ` Qu Wenruo
[not found] <1872105242.302575.1467216517408.JavaMail.zimbra@web4all.fr>
2016-06-29 16:15 ` Benoit GEORGELIN - Association Web4all
2016-06-29 17:19 ` Duncan
2016-06-29 19:20 ` Benoit GEORGELIN - Association Web4all
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.