* Qgroups are not applied when snapshotting a subvol?
@ 2017-03-25 22:03 Moritz Sichert
2017-03-26 5:45 ` Duncan
2017-03-27 0:39 ` Qu Wenruo
0 siblings, 2 replies; 24+ messages in thread
From: Moritz Sichert @ 2017-03-25 22:03 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 849 bytes --]
Hi,
I tried to configure qgroups on a btrfs filesystem but was really surprised that when you snapshot a subvolume, the snapshot will not be assigned to the qgroup the subvolume was in.
As an example consider the small terminal session in the attachment: I create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on that qgroup. Then I write a file into A and eventually get "disk quota exceeded". Then I create a snapshot of A and call it B. B will not be assigned to 1/1 and writing a file into B confirms that no limits at all are imposed for B.
I feel like I must be missing something here. Considering that creating a snapshot does not require root privileges this would mean that any user can just circumvent any quota and therefore make them useless.
Is there a way to enforce quotas even when a user creates snapshots?
Moritz
[-- Attachment #2: btrfs-qgroups.txt --]
[-- Type: text/plain, Size: 1826 bytes --]
$ btrfs quota enable .
$ btrfs subvolume create A
Create subvolume './A'
$ btrfs qgroup create 1/1 .
$ btrfs qgroup show -p --sync .
qgroupid rfer excl parent
-------- ---- ---- ------
0/5 16.00KiB 16.00KiB ---
0/257 16.00KiB 16.00KiB ---
1/1 0.00B 0.00B ---
$ btrfs qgroup assign 0/257 1/1 .
$ btrfs qgroup limit 5M 1/1 .
$ dd if=/dev/urandom of=A/foo bs=1M count=4
4+0 records in
4+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0529255 s, 79.2 MB/s
$ btrfs qgroup show -p --sync .
qgroupid rfer excl parent
-------- ---- ---- ------
0/5 16.00KiB 16.00KiB ---
0/257 4.02MiB 4.02MiB 1/1
1/1 4.02MiB 4.02MiB ---
$ dd if=/dev/urandom of=A/bar bs=1M count=4
dd: error writing 'A/bar': Disk quota exceeded
1+0 records in
0+0 records out
917504 bytes (918 kB, 896 KiB) copied, 0.00408068 s, 225 MB/s
$ btrfs subvolume snapshot A B
Create a snapshot of 'A' in './B'
$ btrfs qgroup show -p --sync .
qgroupid rfer excl parent
-------- ---- ---- ------
0/5 16.00KiB 16.00KiB ---
0/257 4.89MiB 16.00KiB 1/1
0/258 4.89MiB 16.00KiB ---
1/1 4.89MiB 4.89MiB ---
$ dd if=/dev/urandom of=B/baz bs=1M count=4
4+0 records in
4+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0174467 s, 240 MB/s
$ btrfs qgroup show -p --sync .
qgroupid rfer excl parent
-------- ---- ---- ------
0/5 16.00KiB 16.00KiB ---
0/257 4.89MiB 16.00KiB 1/1
0/258 8.89MiB 4.02MiB ---
1/1 4.89MiB 4.89MiB ---
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-25 22:03 Qgroups are not applied when snapshotting a subvol? Moritz Sichert
@ 2017-03-26 5:45 ` Duncan
2017-03-27 0:39 ` Qu Wenruo
1 sibling, 0 replies; 24+ messages in thread
From: Duncan @ 2017-03-26 5:45 UTC (permalink / raw)
To: linux-btrfs
Moritz Sichert posted on Sat, 25 Mar 2017 23:03:26 +0100 as excerpted:
> I tried to configure qgroups on a btrfs filesystem but was really
> surprised that when you snapshot a subvolume, the snapshot will not be
> assigned to the qgroup the subvolume was in.
> I feel like I must be missing something here. Considering that creating
> a snapshot does not require root privileges this would mean that any
> user can just circumvent any quota and therefore make them useless.
>
> Is there a way to enforce quotas even when a user creates snapshots?
This doesn't answer your question, but rather is general information you
should know about btrfs quotas, that I saw no hint in your post that
you're already aware of.
Btrfs' quota functionality has a long history of breakage, and if it's
actually working properly now, it's only very recently so. Don't rely on
the btrfs quota functionality working correctly or being stable, tho
certainly, developers are working very hard at making it eventually so.
Meanwhile, even if it is working correctly, btrfs quota functionality has
serious scaling issues that can seriously slow the progress of btrfs
maintenance commands such as balance and check to a crawl, making them
practically unworkable as they may take weeks to finish on today's
terabyte-scale filesystems -- declaring the filesystem dead, doing a
fresh mkfs, and restoring from backups can be far faster. These commands
are already slow and memory hungry, and having to deal with quotas makes
things far worse.
So basically you and your btrfs quota use-case fall into one of three
classes:
1) You know about the issues and are working with the devs to test and
fix them, helping to make btrfs qgroups and quotas to one day be fully
reliable and scalable. =:^)
If this matches your situation, THANKS! =:^)
Unfortunately, your post had more of the tone of btrfs quota newbie than
someone that knew about the situation, which means you're more likely to
fall into one of the two classes below.
2) Your use-case really needs and relies on quotas to function stably and
reliably.
Unfortunately, if you're in this class, btrfs is not at this time ready
for you -- you're better off on a more mature and fully stable filesystem
that has a reliable quota feature.
3) Your use-case doesn't rely on quotas and you simply believed they were
a nice "extra" feature you could play with.
In this case the best recommendation is to turn the feature off and leave
it off until such time as the quota feature is considered generally
stable and can be recommended, for me personally, several kernel cycles
after things on the quota front seem to have quieted down and it appears
to have been working without major problems.
Even then, quotas are likely to come at somewhat of a scalability and
performance cost, however, which may or may not be worth it, depending on
what the feature can do for your use-case.
(FWIW, the otherwise stable snapshots feature also has scalability
issues. That's why the strong recommendation is to keep them thinned
down to 1-300 per snapshotted subvolume, under 200 if reasonably
possible, and under 2000 per filesystem, under 1000 if possible. But at
least that feature is stable and the scalability cost arguably well
justified by the benefits if the numbers are managed well.
Unfortunately, the same thing can't presently be said about qgroups and
quotas.)
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-25 22:03 Qgroups are not applied when snapshotting a subvol? Moritz Sichert
2017-03-26 5:45 ` Duncan
@ 2017-03-27 0:39 ` Qu Wenruo
2017-03-27 3:26 ` Andrei Borzenkov
1 sibling, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-03-27 0:39 UTC (permalink / raw)
To: Moritz Sichert, linux-btrfs
At 03/26/2017 06:03 AM, Moritz Sichert wrote:
> Hi,
>
> I tried to configure qgroups on a btrfs filesystem but was really surprised that when you snapshot a subvolume, the snapshot will not be assigned to the qgroup the subvolume was in.
>
> As an example consider the small terminal session in the attachment: I create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on that qgroup. Then I write a file into A and eventually get "disk quota exceeded". Then I create a snapshot of A and call it B. B will not be assigned to 1/1 and writing a file into B confirms that no limits at all are imposed for B.
>
> I feel like I must be missing something here. Considering that creating a snapshot does not require root privileges this would mean that any user can just circumvent any quota and therefore make them useless.
>
> Is there a way to enforce quotas even when a user creates snapshots?
>
Yes, there is always method to attach the subvolume/snapshot to
specified higher level qgroup.
Just use "btrfs subvolume snapshot -i 1/1".
Thanks,
Qu
>
> Moritz
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 0:39 ` Qu Wenruo
@ 2017-03-27 3:26 ` Andrei Borzenkov
2017-03-27 3:46 ` Qu Wenruo
0 siblings, 1 reply; 24+ messages in thread
From: Andrei Borzenkov @ 2017-03-27 3:26 UTC (permalink / raw)
To: Qu Wenruo, Moritz Sichert, linux-btrfs
27.03.2017 03:39, Qu Wenruo пишет:
>
>
> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>> Hi,
>>
>> I tried to configure qgroups on a btrfs filesystem but was really
>> surprised that when you snapshot a subvolume, the snapshot will not be
>> assigned to the qgroup the subvolume was in.
>>
>> As an example consider the small terminal session in the attachment: I
>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>> that qgroup. Then I write a file into A and eventually get "disk quota
>> exceeded". Then I create a snapshot of A and call it B. B will not be
>> assigned to 1/1 and writing a file into B confirms that no limits at
>> all are imposed for B.
>>
>> I feel like I must be missing something here. Considering that
>> creating a snapshot does not require root privileges this would mean
>> that any user can just circumvent any quota and therefore make them
>> useless.
>>
>> Is there a way to enforce quotas even when a user creates snapshots?
>>
>
> Yes, there is always method to attach the subvolume/snapshot to
> specified higher level qgroup.
>
> Just use "btrfs subvolume snapshot -i 1/1".
>
This requires cooperation from whoever creates subvolume, while the
question was - is it possible to enforce it, without need for explicit
option/action when snapshot is created.
To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
enforcement.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 3:26 ` Andrei Borzenkov
@ 2017-03-27 3:46 ` Qu Wenruo
2017-03-27 11:02 ` Moritz Sichert
0 siblings, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-03-27 3:46 UTC (permalink / raw)
To: Andrei Borzenkov, Moritz Sichert, linux-btrfs
At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
> 27.03.2017 03:39, Qu Wenruo пишет:
>>
>>
>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>> Hi,
>>>
>>> I tried to configure qgroups on a btrfs filesystem but was really
>>> surprised that when you snapshot a subvolume, the snapshot will not be
>>> assigned to the qgroup the subvolume was in.
>>>
>>> As an example consider the small terminal session in the attachment: I
>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>> that qgroup. Then I write a file into A and eventually get "disk quota
>>> exceeded". Then I create a snapshot of A and call it B. B will not be
>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>> all are imposed for B.
>>>
>>> I feel like I must be missing something here. Considering that
>>> creating a snapshot does not require root privileges this would mean
>>> that any user can just circumvent any quota and therefore make them
>>> useless.
>>>
>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>
>>
>> Yes, there is always method to attach the subvolume/snapshot to
>> specified higher level qgroup.
>>
>> Just use "btrfs subvolume snapshot -i 1/1".
>>
>
> This requires cooperation from whoever creates subvolume, while the
> question was - is it possible to enforce it, without need for explicit
> option/action when snapshot is created.
>
> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
> enforcement.
What if user really want to create a subvolume assigned another group?
You're implying a *policy* that if source subvolume belongs to a higher
level qgroup, then snapshot created should also follow that higher level
qgroup.
However kernel should only provide *mechanisim*, not *policy*.
And btrfs does it, it provides method to do it, whether to do or not is
users responsibility.
If you want to implement that policy, please do it in a higher level,
something like SUSE snapper, not in kernel.
Thanks,
Qu
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 3:46 ` Qu Wenruo
@ 2017-03-27 11:02 ` Moritz Sichert
2017-03-27 12:01 ` Austin S. Hemmelgarn
0 siblings, 1 reply; 24+ messages in thread
From: Moritz Sichert @ 2017-03-27 11:02 UTC (permalink / raw)
To: Qu Wenruo, Andrei Borzenkov, linux-btrfs
Am 27.03.2017 um 05:46 schrieb Qu Wenruo:
>
>
> At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
>> 27.03.2017 03:39, Qu Wenruo пишет:
>>>
>>>
>>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>>> Hi,
>>>>
>>>> I tried to configure qgroups on a btrfs filesystem but was really
>>>> surprised that when you snapshot a subvolume, the snapshot will not be
>>>> assigned to the qgroup the subvolume was in.
>>>>
>>>> As an example consider the small terminal session in the attachment: I
>>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>>> that qgroup. Then I write a file into A and eventually get "disk quota
>>>> exceeded". Then I create a snapshot of A and call it B. B will not be
>>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>>> all are imposed for B.
>>>>
>>>> I feel like I must be missing something here. Considering that
>>>> creating a snapshot does not require root privileges this would mean
>>>> that any user can just circumvent any quota and therefore make them
>>>> useless.
>>>>
>>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>>
>>>
>>> Yes, there is always method to attach the subvolume/snapshot to
>>> specified higher level qgroup.
>>>
>>> Just use "btrfs subvolume snapshot -i 1/1".
>>>
>>
>> This requires cooperation from whoever creates subvolume, while the
>> question was - is it possible to enforce it, without need for explicit
>> option/action when snapshot is created.
>>
>> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
>> enforcement.
>
> What if user really want to create a subvolume assigned another group?
>
> You're implying a *policy* that if source subvolume belongs to a higher level qgroup, then snapshot created should also follow that higher level qgroup.
>
> However kernel should only provide *mechanisim*, not *policy*.
> And btrfs does it, it provides method to do it, whether to do or not is users responsibility.
>
> If you want to implement that policy, please do it in a higher level, something like SUSE snapper, not in kernel.
The problem is, I can't enforce the policy because *every user* can create snapshots. Even if I would restrict the btrfs executable so that only root can execute it, this doesn't help. As using the ioctl for btrfs is allowed for any user, they could just get the executable from somewhere else.
Moritz
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 11:02 ` Moritz Sichert
@ 2017-03-27 12:01 ` Austin S. Hemmelgarn
2017-03-27 19:32 ` Chris Murphy
2017-03-28 1:49 ` Qu Wenruo
0 siblings, 2 replies; 24+ messages in thread
From: Austin S. Hemmelgarn @ 2017-03-27 12:01 UTC (permalink / raw)
To: Moritz Sichert, Qu Wenruo, Andrei Borzenkov, linux-btrfs
On 2017-03-27 07:02, Moritz Sichert wrote:
> Am 27.03.2017 um 05:46 schrieb Qu Wenruo:
>>
>>
>> At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
>>> 27.03.2017 03:39, Qu Wenruo пишет:
>>>>
>>>>
>>>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>>>> Hi,
>>>>>
>>>>> I tried to configure qgroups on a btrfs filesystem but was really
>>>>> surprised that when you snapshot a subvolume, the snapshot will not be
>>>>> assigned to the qgroup the subvolume was in.
>>>>>
>>>>> As an example consider the small terminal session in the attachment: I
>>>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>>>> that qgroup. Then I write a file into A and eventually get "disk quota
>>>>> exceeded". Then I create a snapshot of A and call it B. B will not be
>>>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>>>> all are imposed for B.
>>>>>
>>>>> I feel like I must be missing something here. Considering that
>>>>> creating a snapshot does not require root privileges this would mean
>>>>> that any user can just circumvent any quota and therefore make them
>>>>> useless.
>>>>>
>>>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>>>
>>>>
>>>> Yes, there is always method to attach the subvolume/snapshot to
>>>> specified higher level qgroup.
>>>>
>>>> Just use "btrfs subvolume snapshot -i 1/1".
>>>>
>>>
>>> This requires cooperation from whoever creates subvolume, while the
>>> question was - is it possible to enforce it, without need for explicit
>>> option/action when snapshot is created.
>>>
>>> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
>>> enforcement.
>>
>> What if user really want to create a subvolume assigned another group?
>>
>> You're implying a *policy* that if source subvolume belongs to a higher level qgroup, then snapshot created should also follow that higher level qgroup.
>>
>> However kernel should only provide *mechanisim*, not *policy*.
>> And btrfs does it, it provides method to do it, whether to do or not is users responsibility.
>>
>> If you want to implement that policy, please do it in a higher level, something like SUSE snapper, not in kernel.
>
> The problem is, I can't enforce the policy because *every user* can create snapshots. Even if I would restrict the btrfs executable so that only root can execute it, this doesn't help. As using the ioctl for btrfs is allowed for any user, they could just get the executable from somewhere else.
To reiterate and reinforce this:
If it is not possible to enforce new subvolumes counting for their
parent quota, and there is no option to prevent non-root (or
non-CAP_SYS_ADMIN) users from creating new subvolumes, then BTRFS
qgroups are useless on any system with shell access because a user can
trivially escape their quota restrictions (or hide from accounting) by
creating a new subvolume which is outside of their qgroup and storing
data there.
Ideally, there should be an option to disable user subvolume creation
(it arguably should be the default, because of resource exhaustion
issues, but that's a separate argument), and there should be an option
in the kernel to force specific behavior. Both cases are policy, but
they are policy that can only be concretely enforced _by the kernel_.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 12:01 ` Austin S. Hemmelgarn
@ 2017-03-27 19:32 ` Chris Murphy
2017-03-27 19:53 ` Roman Mamedov
` (2 more replies)
2017-03-28 1:49 ` Qu Wenruo
1 sibling, 3 replies; 24+ messages in thread
From: Chris Murphy @ 2017-03-27 19:32 UTC (permalink / raw)
To: Austin S. Hemmelgarn
Cc: Moritz Sichert, Qu Wenruo, Andrei Borzenkov, Btrfs BTRFS
How about if qgroups are enabled, then non-root user is prevented from
creating new subvolumes?
Or is there a way for a new nested subvolume to be included in its
parent's quota, rather than the new subvolume having a whole new quota
limit?
Tricky problem.
Chris Murphy
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 19:32 ` Chris Murphy
@ 2017-03-27 19:53 ` Roman Mamedov
2017-03-27 20:06 ` Hans van Kranenburg
2017-03-28 3:56 ` Andrei Borzenkov
2017-03-28 11:24 ` Austin S. Hemmelgarn
2 siblings, 1 reply; 24+ messages in thread
From: Roman Mamedov @ 2017-03-27 19:53 UTC (permalink / raw)
To: Chris Murphy
Cc: Austin S. Hemmelgarn, Moritz Sichert, Qu Wenruo,
Andrei Borzenkov, Btrfs BTRFS
On Mon, 27 Mar 2017 13:32:47 -0600
Chris Murphy <lists@colorremedies.com> wrote:
> How about if qgroups are enabled, then non-root user is prevented from
> creating new subvolumes?
That sounds like, if you turn your headlights on in a car, then in-vehicle air
conditioner randomly stops working. :)
Two things only vaguely related from the end user's point of view.
> Or is there a way for a new nested subvolume to be included in its
> parent's quota, rather than the new subvolume having a whole new quota
> limit?
Either that, or a separate "allow non-root user subvolumes/snapshots creation"
mount option. There is already one for deletion, after all.
user_subvol_rm_allowed
Allow subvolumes to be deleted by a non-root user. Use with
caution.
--
With respect,
Roman
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 19:53 ` Roman Mamedov
@ 2017-03-27 20:06 ` Hans van Kranenburg
2017-03-27 21:11 ` Chris Murphy
0 siblings, 1 reply; 24+ messages in thread
From: Hans van Kranenburg @ 2017-03-27 20:06 UTC (permalink / raw)
To: Roman Mamedov, Chris Murphy
Cc: Austin S. Hemmelgarn, Moritz Sichert, Qu Wenruo,
Andrei Borzenkov, Btrfs BTRFS
On 03/27/2017 09:53 PM, Roman Mamedov wrote:
> On Mon, 27 Mar 2017 13:32:47 -0600
> Chris Murphy <lists@colorremedies.com> wrote:
>
>> How about if qgroups are enabled, then non-root user is prevented from
>> creating new subvolumes?
>
> That sounds like, if you turn your headlights on in a car, then in-vehicle air
> conditioner randomly stops working. :)
>
> Two things only vaguely related from the end user's point of view.
>
>> Or is there a way for a new nested subvolume to be included in its
>> parent's quota, rather than the new subvolume having a whole new quota
>> limit?
>
> Either that, or a separate "allow non-root user subvolumes/snapshots creation"
> mount option. There is already one for deletion, after all.
>
> user_subvol_rm_allowed
> Allow subvolumes to be deleted by a non-root user. Use with
> caution.
>
What are actual use cases for creating subvolumes by 'normal' users?
Does someone have an example?
Why is it possible at all, by default?
--
Hans van Kranenburg
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 20:06 ` Hans van Kranenburg
@ 2017-03-27 21:11 ` Chris Murphy
2017-03-28 2:41 ` Duncan
0 siblings, 1 reply; 24+ messages in thread
From: Chris Murphy @ 2017-03-27 21:11 UTC (permalink / raw)
To: Hans van Kranenburg
Cc: Roman Mamedov, Chris Murphy, Austin S. Hemmelgarn,
Moritz Sichert, Qu Wenruo, Andrei Borzenkov, Btrfs BTRFS
On Mon, Mar 27, 2017 at 2:06 PM, Hans van Kranenburg
<hans.van.kranenburg@mendix.com> wrote:
> On 03/27/2017 09:53 PM, Roman Mamedov wrote:
>> On Mon, 27 Mar 2017 13:32:47 -0600
>> Chris Murphy <lists@colorremedies.com> wrote:
>>
>>> How about if qgroups are enabled, then non-root user is prevented from
>>> creating new subvolumes?
>>
>> That sounds like, if you turn your headlights on in a car, then in-vehicle air
>> conditioner randomly stops working. :)
>>
>> Two things only vaguely related from the end user's point of view.
>>
>>> Or is there a way for a new nested subvolume to be included in its
>>> parent's quota, rather than the new subvolume having a whole new quota
>>> limit?
>>
>> Either that, or a separate "allow non-root user subvolumes/snapshots creation"
>> mount option. There is already one for deletion, after all.
>>
>> user_subvol_rm_allowed
>> Allow subvolumes to be deleted by a non-root user. Use with
>> caution.
>>
>
> What are actual use cases for creating subvolumes by 'normal' users?
>
> Does someone have an example?
>
> Why is it possible at all, by default?
I have a single git subvolume in my user directory, inside of which
are various git clones. And I periodically snapshot the git subvolume
as a regular user.
If I can't create subvolumes as a regular user then by extension it'd
mean I can't create snapshots of my own home directory, or any other
subvolumes I exclusively own.
--
Chris Murphy
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 12:01 ` Austin S. Hemmelgarn
2017-03-27 19:32 ` Chris Murphy
@ 2017-03-28 1:49 ` Qu Wenruo
2017-03-28 11:44 ` Austin S. Hemmelgarn
1 sibling, 1 reply; 24+ messages in thread
From: Qu Wenruo @ 2017-03-28 1:49 UTC (permalink / raw)
To: Austin S. Hemmelgarn, Moritz Sichert, Andrei Borzenkov, linux-btrfs
At 03/27/2017 08:01 PM, Austin S. Hemmelgarn wrote:
> On 2017-03-27 07:02, Moritz Sichert wrote:
>> Am 27.03.2017 um 05:46 schrieb Qu Wenruo:
>>>
>>>
>>> At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
>>>> 27.03.2017 03:39, Qu Wenruo пишет:
>>>>>
>>>>>
>>>>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I tried to configure qgroups on a btrfs filesystem but was really
>>>>>> surprised that when you snapshot a subvolume, the snapshot will
>>>>>> not be
>>>>>> assigned to the qgroup the subvolume was in.
>>>>>>
>>>>>> As an example consider the small terminal session in the
>>>>>> attachment: I
>>>>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>>>>> that qgroup. Then I write a file into A and eventually get "disk
>>>>>> quota
>>>>>> exceeded". Then I create a snapshot of A and call it B. B will not be
>>>>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>>>>> all are imposed for B.
>>>>>>
>>>>>> I feel like I must be missing something here. Considering that
>>>>>> creating a snapshot does not require root privileges this would mean
>>>>>> that any user can just circumvent any quota and therefore make them
>>>>>> useless.
>>>>>>
>>>>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>>>>
>>>>>
>>>>> Yes, there is always method to attach the subvolume/snapshot to
>>>>> specified higher level qgroup.
>>>>>
>>>>> Just use "btrfs subvolume snapshot -i 1/1".
>>>>>
>>>>
>>>> This requires cooperation from whoever creates subvolume, while the
>>>> question was - is it possible to enforce it, without need for explicit
>>>> option/action when snapshot is created.
>>>>
>>>> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
>>>> enforcement.
>>>
>>> What if user really want to create a subvolume assigned another group?
>>>
>>> You're implying a *policy* that if source subvolume belongs to a
>>> higher level qgroup, then snapshot created should also follow that
>>> higher level qgroup.
>>>
>>> However kernel should only provide *mechanisim*, not *policy*.
>>> And btrfs does it, it provides method to do it, whether to do or not
>>> is users responsibility.
>>>
>>> If you want to implement that policy, please do it in a higher level,
>>> something like SUSE snapper, not in kernel.
>>
>> The problem is, I can't enforce the policy because *every user* can
>> create snapshots. Even if I would restrict the btrfs executable so
>> that only root can execute it, this doesn't help. As using the ioctl
>> for btrfs is allowed for any user, they could just get the executable
>> from somewhere else.
> To reiterate and reinforce this:
> If it is not possible to enforce new subvolumes counting for their
> parent quota, and there is no option to prevent non-root (or
> non-CAP_SYS_ADMIN) users from creating new subvolumes, then BTRFS
> qgroups are useless on any system with shell access because a user can
> trivially escape their quota restrictions (or hide from accounting) by
> creating a new subvolume which is outside of their qgroup and storing
> data there.
>
> Ideally, there should be an option to disable user subvolume creation
> (it arguably should be the default, because of resource exhaustion
> issues, but that's a separate argument), and there should be an option
> in the kernel to force specific behavior. Both cases are policy, but
> they are policy that can only be concretely enforced _by the kernel_.
>
>
The problem is, how should we treat subvolume.
Btrfs subvolume sits in the middle of directory and (logical) volume
used in traditional stacked solution.
While we allow normal user to create/delete/modify dir as long as they
follow access control, we require privilege to create/delete/modify volumes.
Developers chose to treat btrfs subvolume as dir, makes it quite easy to
operate for normal use case, sacrificing qgroup limit which is not a
major function (or even did not exist) at that time.
IIRC at the beginning time of btrfs, we don't have a full idea of use
cases could be.
This is common, a lot of problems(even bad design) can only be found
after enough feedback from end users.
Personally speaking, I prefer to restrict subvolume creation/deletion to
privilege users only, and uses a daemon as a proxy to do such privilege
operation.
So we can do better accounting/access control without bothering the kernel.
But that makes a big behavior difference, I'm afraid this won't become true.
Thanks,
Qu
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 21:11 ` Chris Murphy
@ 2017-03-28 2:41 ` Duncan
2017-03-28 5:21 ` Duncan
0 siblings, 1 reply; 24+ messages in thread
From: Duncan @ 2017-03-28 2:41 UTC (permalink / raw)
To: linux-btrfs
Chris Murphy posted on Mon, 27 Mar 2017 15:11:34 -0600 as excerpted:
>> What are actual use cases for creating subvolumes by 'normal' users?
>>
>> Does someone have an example?
>>
>> Why is it possible at all, by default?
>
> I have a single git subvolume in my user directory, inside of which are
> various git clones. And I periodically snapshot the git subvolume as a
> regular user.
>
> If I can't create subvolumes as a regular user then by extension it'd
> mean I can't create snapshots of my own home directory, or any other
> subvolumes I exclusively own.
One rather big problem with what, with snapshots (which are a special
kind of subvolume), is that btrfs has known scaling issues when the
number of snapshots gets too high. Combine that with allowing users to
make but not delete snapshots, and you have a huge invitation to scaling
headaches due to the number of snapshots.
Really, the two permissions subvolume/snapshot creation, and deletion,
should be synchronized. Allowing subvolume deletion clearly has security
issues, but so does allowing creation without allowing deletion. They
both really have to go together, and be allowed only for "trusted" users,
with the option of whether that's root-only, or a subset of users (say
via group perms), or all users, being up to the local admin, basically, a
mount option.
Which in usual terms means making the perms root-only, with the binary
set to some controlled-access group and set-SUID-root (or appropriate
security attributes, I'm drawing a blank on the word I want ATM), and
then letting the admin control access via group membership.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 19:32 ` Chris Murphy
2017-03-27 19:53 ` Roman Mamedov
@ 2017-03-28 3:56 ` Andrei Borzenkov
2017-03-28 11:24 ` Austin S. Hemmelgarn
2 siblings, 0 replies; 24+ messages in thread
From: Andrei Borzenkov @ 2017-03-28 3:56 UTC (permalink / raw)
To: Chris Murphy, Austin S. Hemmelgarn; +Cc: Moritz Sichert, Qu Wenruo, Btrfs BTRFS
27.03.2017 22:32, Chris Murphy пишет:
> How about if qgroups are enabled, then non-root user is prevented from
> creating new subvolumes?
>
> Or is there a way for a new nested subvolume to be included in its
> parent's quota, rather than the new subvolume having a whole new quota
> limit?
>
They are not even nested in any sense.
btrfs sub create /some/parent/A /other/parent/B
There is no direct relation between A and B except that B is clone of A
(but I do not know whether this fact is recorded somewhere in subvolume
properties).
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-28 2:41 ` Duncan
@ 2017-03-28 5:21 ` Duncan
0 siblings, 0 replies; 24+ messages in thread
From: Duncan @ 2017-03-28 5:21 UTC (permalink / raw)
To: linux-btrfs
Duncan posted on Tue, 28 Mar 2017 02:41:52 +0000 as excerpted:
> Which in usual terms means making the perms root-only, with the binary
> set to some controlled-access group and set-SUID-root (or appropriate
> security attributes, I'm drawing a blank on the word I want ATM), and
> then letting the admin control access via group membership.
File capacities (what was eluding me).
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-27 19:32 ` Chris Murphy
2017-03-27 19:53 ` Roman Mamedov
2017-03-28 3:56 ` Andrei Borzenkov
@ 2017-03-28 11:24 ` Austin S. Hemmelgarn
2017-03-28 12:00 ` Marat Khalili
2 siblings, 1 reply; 24+ messages in thread
From: Austin S. Hemmelgarn @ 2017-03-28 11:24 UTC (permalink / raw)
To: Chris Murphy; +Cc: Moritz Sichert, Qu Wenruo, Andrei Borzenkov, Btrfs BTRFS
On 2017-03-27 15:32, Chris Murphy wrote:
> How about if qgroups are enabled, then non-root user is prevented from
> creating new subvolumes?
>
> Or is there a way for a new nested subvolume to be included in its
> parent's quota, rather than the new subvolume having a whole new quota
> limit?
>
> Tricky problem.
The default should be to inherit the qgroup of the parent subvolume.
The organization of subvolumes is hierarchical, and sane people expect
things to behave as they look. Taking another angle, on ZFS, 'nested'
(nested in quotes because ZFS' definition of 'nested' zvols is weird)
inherit their parent's quota and reservations (essentially reverse
quota), and they're not even inherently nested in the filesystem like
subvolumes are, so we're differing from the only other widely used
system that implements things in a similar manner.
As far as the subvolume thing, there should be an option to disable user
creation of subvolumes, and ideally it should be on by default because:
1. Users can't delete subvolumes by default. This means they can create
but not destroy a resource by default, which means that a user can
pretty easily accidentally cause issues for the system as a whole.
2. Correlating with 1, users being able to delete subvolumes by default
is not safe on multiple levels (easy accidental data loss, numerous
other issues), and thus user subvolume removal being off by default is
significantly safer.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-28 1:49 ` Qu Wenruo
@ 2017-03-28 11:44 ` Austin S. Hemmelgarn
2017-03-29 5:38 ` Duncan
0 siblings, 1 reply; 24+ messages in thread
From: Austin S. Hemmelgarn @ 2017-03-28 11:44 UTC (permalink / raw)
To: Qu Wenruo, Moritz Sichert, Andrei Borzenkov, linux-btrfs
On 2017-03-27 21:49, Qu Wenruo wrote:
>
>
> At 03/27/2017 08:01 PM, Austin S. Hemmelgarn wrote:
>> On 2017-03-27 07:02, Moritz Sichert wrote:
>>> Am 27.03.2017 um 05:46 schrieb Qu Wenruo:
>>>>
>>>>
>>>> At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
>>>>> 27.03.2017 03:39, Qu Wenruo пишет:
>>>>>>
>>>>>>
>>>>>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I tried to configure qgroups on a btrfs filesystem but was really
>>>>>>> surprised that when you snapshot a subvolume, the snapshot will
>>>>>>> not be
>>>>>>> assigned to the qgroup the subvolume was in.
>>>>>>>
>>>>>>> As an example consider the small terminal session in the
>>>>>>> attachment: I
>>>>>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>>>>>> that qgroup. Then I write a file into A and eventually get "disk
>>>>>>> quota
>>>>>>> exceeded". Then I create a snapshot of A and call it B. B will
>>>>>>> not be
>>>>>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>>>>>> all are imposed for B.
>>>>>>>
>>>>>>> I feel like I must be missing something here. Considering that
>>>>>>> creating a snapshot does not require root privileges this would mean
>>>>>>> that any user can just circumvent any quota and therefore make them
>>>>>>> useless.
>>>>>>>
>>>>>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>>>>>
>>>>>>
>>>>>> Yes, there is always method to attach the subvolume/snapshot to
>>>>>> specified higher level qgroup.
>>>>>>
>>>>>> Just use "btrfs subvolume snapshot -i 1/1".
>>>>>>
>>>>>
>>>>> This requires cooperation from whoever creates subvolume, while the
>>>>> question was - is it possible to enforce it, without need for explicit
>>>>> option/action when snapshot is created.
>>>>>
>>>>> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
>>>>> enforcement.
>>>>
>>>> What if user really want to create a subvolume assigned another group?
>>>>
>>>> You're implying a *policy* that if source subvolume belongs to a
>>>> higher level qgroup, then snapshot created should also follow that
>>>> higher level qgroup.
>>>>
>>>> However kernel should only provide *mechanisim*, not *policy*.
>>>> And btrfs does it, it provides method to do it, whether to do or not
>>>> is users responsibility.
>>>>
>>>> If you want to implement that policy, please do it in a higher level,
>>>> something like SUSE snapper, not in kernel.
>>>
>>> The problem is, I can't enforce the policy because *every user* can
>>> create snapshots. Even if I would restrict the btrfs executable so
>>> that only root can execute it, this doesn't help. As using the ioctl
>>> for btrfs is allowed for any user, they could just get the executable
>>> from somewhere else.
>> To reiterate and reinforce this:
>> If it is not possible to enforce new subvolumes counting for their
>> parent quota, and there is no option to prevent non-root (or
>> non-CAP_SYS_ADMIN) users from creating new subvolumes, then BTRFS
>> qgroups are useless on any system with shell access because a user can
>> trivially escape their quota restrictions (or hide from accounting) by
>> creating a new subvolume which is outside of their qgroup and storing
>> data there.
>>
>> Ideally, there should be an option to disable user subvolume creation
>> (it arguably should be the default, because of resource exhaustion
>> issues, but that's a separate argument), and there should be an option
>> in the kernel to force specific behavior. Both cases are policy, but
>> they are policy that can only be concretely enforced _by the kernel_.
>>
>>
> The problem is, how should we treat subvolume.
>
> Btrfs subvolume sits in the middle of directory and (logical) volume
> used in traditional stacked solution.
>
> While we allow normal user to create/delete/modify dir as long as they
> follow access control, we require privilege to create/delete/modify
> volumes.
No, we require privilege to do certain modifications or delete
subvolumes. Regular users can create subvolumes with no privileges
whatsoever, and most basic directory operations (rename, chown, chmod,
etc) work just fine within normal UNIX DAC permissions. Unless you're
running some specially patched kernel or some LSM (SELinux possibly)
that somehow restricts access to the ioctl, you can always create
subvolumes.
This is part of the reason that I'm personally hesitant to use BTRFS on
systems where end users have shell access, it's a DoS waiting to happen.
>
> Developers chose to treat btrfs subvolume as dir, makes it quite easy to
> operate for normal use case, sacrificing qgroup limit which is not a
> major function (or even did not exist) at that time.
>
> IIRC at the beginning time of btrfs, we don't have a full idea of use
> cases could be.
> This is common, a lot of problems(even bad design) can only be found
> after enough feedback from end users.
>
> Personally speaking, I prefer to restrict subvolume creation/deletion to
> privilege users only, and uses a daemon as a proxy to do such privilege
> operation.
> So we can do better accounting/access control without bothering the kernel.
I will agree that a daemon for this can be useful, but even if we add a
mount option to restrict operations on subvolumes by normal users, we
can still provide the option of a daemon. On something like a single
user system, there is not much advantage to having some complex access
control in place. I'm not saying we should put any kind of complex ACL
into the kernel, but that we should at least have some all-or-nothing
switch in the mount options that controls if unprivileged users can
perform subvolume operations. Ideally it would be just one option, but
unfortunately we provided somewhat nonsensical initial semantics that we
now have to continue to support.
Looking at the qgroup stuff specifically though, snapshots not
inheriting their parent's qgroup seems really odd to me. There are two
things that come to mind, and I'd love to see both personally:
1. Provide an option to have snapshots inherit their parent's qgroup.
This would eliminate the 'surprise' that initially sparked this
discussion, and deal with tools that aren't qgroup aware.
2. Provide an option to specify the default qgroup for new subvolumes,
instead of them being created with no qgroup. This would cover the rest
of things, and provide some further usefulness (for example, you could
leave this default qgroup empty most of the time and have monitoring
software alert you if it suddenly had data in it to detect stuff
creating subvolumes behind your back).
Both sound more to me like they should probably be specified somewhere
in the filesystem itself, not the mount options.
>
> But that makes a big behavior difference, I'm afraid this won't become
> true.
I'm definitely with you on having the ability to restrict subvolume
operations to privileged users, I just don't feel that the suggested
methodology is enough by itself.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-28 11:24 ` Austin S. Hemmelgarn
@ 2017-03-28 12:00 ` Marat Khalili
2017-03-28 12:20 ` Austin S. Hemmelgarn
0 siblings, 1 reply; 24+ messages in thread
From: Marat Khalili @ 2017-03-28 12:00 UTC (permalink / raw)
To: linux-btrfs
> The default should be to inherit the qgroup of the parent subvolume.
This behaviour is only good for this particular use-case. In general
case, qgroups of subvolume and snapshots should exist separately, and
both can be included in some higher level qgroup (after all, that's what
qgroup hierarchy is for).
In my system I found it convenient to include subvolume and its
snapshots in qgroup 1/N, where 0/N is qgroup of bare subvolume. I think
adopting this behaviour as default would be more sensible.
--
With Best Regards,
Marat Khalili
On 28/03/17 14:24, Austin S. Hemmelgarn wrote:
> On 2017-03-27 15:32, Chris Murphy wrote:
>> How about if qgroups are enabled, then non-root user is prevented from
>> creating new subvolumes?
>>
>> Or is there a way for a new nested subvolume to be included in its
>> parent's quota, rather than the new subvolume having a whole new quota
>> limit?
>>
>> Tricky problem.
> The default should be to inherit the qgroup of the parent subvolume.
> The organization of subvolumes is hierarchical, and sane people expect
> things to behave as they look. Taking another angle, on ZFS, 'nested'
> (nested in quotes because ZFS' definition of 'nested' zvols is weird)
> inherit their parent's quota and reservations (essentially reverse
> quota), and they're not even inherently nested in the filesystem like
> subvolumes are, so we're differing from the only other widely used
> system that implements things in a similar manner.
>
> As far as the subvolume thing, there should be an option to disable
> user creation of subvolumes, and ideally it should be on by default
> because:
> 1. Users can't delete subvolumes by default. This means they can
> create but not destroy a resource by default, which means that a user
> can pretty easily accidentally cause issues for the system as a whole.
> 2. Correlating with 1, users being able to delete subvolumes by
> default is not safe on multiple levels (easy accidental data loss,
> numerous other issues), and thus user subvolume removal being off by
> default is significantly safer.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-28 12:00 ` Marat Khalili
@ 2017-03-28 12:20 ` Austin S. Hemmelgarn
2017-03-28 13:53 ` Marat Khalili
0 siblings, 1 reply; 24+ messages in thread
From: Austin S. Hemmelgarn @ 2017-03-28 12:20 UTC (permalink / raw)
To: Marat Khalili, linux-btrfs
On 2017-03-28 08:00, Marat Khalili wrote:
>> The default should be to inherit the qgroup of the parent subvolume.
> This behaviour is only good for this particular use-case. In general
> case, qgroups of subvolume and snapshots should exist separately, and
> both can be included in some higher level qgroup (after all, that's what
> qgroup hierarchy is for).
>
> In my system I found it convenient to include subvolume and its
> snapshots in qgroup 1/N, where 0/N is qgroup of bare subvolume. I think
> adopting this behaviour as default would be more sensible.
There are a couple of reasons I'm advocating the specific behavior I
outlined:
1. It doesn't require any specific qgroup setup. By definition, you can
be 100% certain that the destination qgroup exists, and that you won't
need to create new qgroups behind the user's back (given your
suggestion, what happens when qgroup 1/N doesn't exist?).
2. Just because it's the default, doesn't mean that the subvolume can't
be reassigned to a different qgroup. This also would not remove the
ability to assign a specific qgroup through the snapshot creation
command. This is arguably a general point in favor of having any
default of course, but it's still worth pointing out.
3. Because BTRFS has COW semantics, the new snapshot should take up near
zero space in the qgroup of it's parent.
4. This correlates with the behavior most people expect based on ZFS and
LVM, which is that snapshots are tied to their parent.
At a minimum, it should belong to _some_ qgroup. This could also be
covered by having a designated 'default' qgroup that all new subvolumes
created without a specified qgroup get put in, but I feel that that is
somewhat orthogonal to the issue of how snapshots are handled.
> --
>
> With Best Regards,
> Marat Khalili
>
> On 28/03/17 14:24, Austin S. Hemmelgarn wrote:
>> On 2017-03-27 15:32, Chris Murphy wrote:
>>> How about if qgroups are enabled, then non-root user is prevented from
>>> creating new subvolumes?
>>>
>>> Or is there a way for a new nested subvolume to be included in its
>>> parent's quota, rather than the new subvolume having a whole new quota
>>> limit?
>>>
>>> Tricky problem.
>> The default should be to inherit the qgroup of the parent subvolume.
>> The organization of subvolumes is hierarchical, and sane people expect
>> things to behave as they look. Taking another angle, on ZFS, 'nested'
>> (nested in quotes because ZFS' definition of 'nested' zvols is weird)
>> inherit their parent's quota and reservations (essentially reverse
>> quota), and they're not even inherently nested in the filesystem like
>> subvolumes are, so we're differing from the only other widely used
>> system that implements things in a similar manner.
>>
>> As far as the subvolume thing, there should be an option to disable
>> user creation of subvolumes, and ideally it should be on by default
>> because:
>> 1. Users can't delete subvolumes by default. This means they can
>> create but not destroy a resource by default, which means that a user
>> can pretty easily accidentally cause issues for the system as a whole.
>> 2. Correlating with 1, users being able to delete subvolumes by
>> default is not safe on multiple levels (easy accidental data loss,
>> numerous other issues), and thus user subvolume removal being off by
>> default is significantly safer.
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-28 12:20 ` Austin S. Hemmelgarn
@ 2017-03-28 13:53 ` Marat Khalili
2017-03-28 15:24 ` Austin S. Hemmelgarn
0 siblings, 1 reply; 24+ messages in thread
From: Marat Khalili @ 2017-03-28 13:53 UTC (permalink / raw)
To: linux-btrfs
> There are a couple of reasons I'm advocating the specific behavior I
> outlined:
Some of your points are valid, but some break current behaviour and
expectations or create technical difficulties.
> 1. It doesn't require any specific qgroup setup. By definition, you
> can be 100% certain that the destination qgroup exists, and that you
> won't need to create new qgroups behind the user's back (given your
> suggestion, what happens when qgroup 1/N doesn't exist?).
This is a general problem with current qgroups: you have to reference
them by some random numbers, not by user-assigned names like files. It
would need to be fixed sooner or later with syntax like L:<path> in
place of L/N, or even some special syntax made specifically for path
snapshots.
BTW, what about reserving level 1 for qgroups describing subvolumes and
all their snapshots and forbidding manual management of qgroups at this
level?
> 2. Just because it's the default, doesn't mean that the subvolume
> can't be reassigned to a different qgroup. This also would not remove
> the ability to assign a specific qgroup through the snapshot creation
> command. This is arguably a general point in favor of having any
> default of course, but it's still worth pointing out.
Currently 0/N qgroups are special in that they are created automatically
and belong to the bottom of the hierarchy. It would be very nice to keep
it this way.
Changing qgroup assignments after snapshot creation is very inconvenient
because it requires quota rescan and thus blocks all other quota operations.
> 3. Because BTRFS has COW semantics, the new snapshot should take up
> near zero space in the qgroup of it's parent.
Indeed it works this way in my experiments if you assign snapshot to 1/N
qgroup at creation where 0/N also belongs. Moreover, it does not require
quota rescan, which is very nice.
> 4. This correlates with the behavior most people expect based on ZFS
> and LVM, which is that snapshots are tied to their parent.
I'm not against tying it to the parent. I'm against removing snapshot's
own qgroup.
> At a minimum, it should belong to _some_ qgroup. This could also be
> covered by having a designated 'default' qgroup that all new
> subvolumes created without a specified qgroup get put in, but I feel
> that that is somewhat orthogonal to the issue of how snapshots are
> handled.
It belongs to its own 0/N' qgroup, but this is not probably what you mean.
--
With Best Regards,
Marat Khalili
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-28 13:53 ` Marat Khalili
@ 2017-03-28 15:24 ` Austin S. Hemmelgarn
2017-03-29 5:53 ` Marat Khalili
0 siblings, 1 reply; 24+ messages in thread
From: Austin S. Hemmelgarn @ 2017-03-28 15:24 UTC (permalink / raw)
To: Marat Khalili, linux-btrfs
On 2017-03-28 09:53, Marat Khalili wrote:
>> There are a couple of reasons I'm advocating the specific behavior I
>> outlined:
> Some of your points are valid, but some break current behaviour and
> expectations or create technical difficulties.
>
>
>> 1. It doesn't require any specific qgroup setup. By definition, you
>> can be 100% certain that the destination qgroup exists, and that you
>> won't need to create new qgroups behind the user's back (given your
>> suggestion, what happens when qgroup 1/N doesn't exist?).
> This is a general problem with current qgroups: you have to reference
> them by some random numbers, not by user-assigned names like files. It
> would need to be fixed sooner or later with syntax like L:<path> in
> place of L/N, or even some special syntax made specifically for path
> snapshots.
>
> BTW, what about reserving level 1 for qgroups describing subvolumes and
> all their snapshots and forbidding manual management of qgroups at this
> level?
If we were going to reserve something, it should be a high number, not a
low one. Having 0 reserved makes some sense, but reserving other low
numbers seems kind of odd when they aren't already reserved.
>
>
>> 2. Just because it's the default, doesn't mean that the subvolume
>> can't be reassigned to a different qgroup. This also would not remove
>> the ability to assign a specific qgroup through the snapshot creation
>> command. This is arguably a general point in favor of having any
>> default of course, but it's still worth pointing out.
> Currently 0/N qgroups are special in that they are created automatically
> and belong to the bottom of the hierarchy. It would be very nice to keep
> it this way.
>
> Changing qgroup assignments after snapshot creation is very inconvenient
> because it requires quota rescan and thus blocks all other quota
> operations.
The subvolume create command lets you set a specific qgroup on subvolume
creation. When I said it won't prevent assigning a specific qgroup, I
meant using that, not some after-the-fact assignment, although I hadn't
realized that the snapshot command _does not_ have this argument, when
it absolutely should.
>
>
>> 3. Because BTRFS has COW semantics, the new snapshot should take up
>> near zero space in the qgroup of it's parent.
>
> Indeed it works this way in my experiments if you assign snapshot to 1/N
> qgroup at creation where 0/N also belongs. Moreover, it does not require
> quota rescan, which is very nice.
I mean if it's placed in that qgroup itself. If you create a snapshot
of an idle subvolume, you only effectively double the metadata (and
possibly not even that completely), so unless it's got a very large
number of small files, it should take up next to zero space if it gets
put in it's parent's qgroup.
>
>
>> 4. This correlates with the behavior most people expect based on ZFS
>> and LVM, which is that snapshots are tied to their parent.
>
> I'm not against tying it to the parent. I'm against removing snapshot's
> own qgroup.
Perhaps have an option (in the filesystem, not the mount options, this
is tied to qgroup policy, which should be independent of who mounts the
FS where) that specifies one of:
1. Current behavior.
2. Join parent's qgroup.
3. Join some other predefined qgroup.
>
>
>> At a minimum, it should belong to _some_ qgroup. This could also be
>> covered by having a designated 'default' qgroup that all new
>> subvolumes created without a specified qgroup get put in, but I feel
>> that that is somewhat orthogonal to the issue of how snapshots are
>> handled.
> It belongs to its own 0/N' qgroup, but this is not probably what you mean.
The 0/N groups are accounting only until some limit is set, correct? If
that is the case, then that really isn't what I mean. Quotas can be
used for accounting, but when used, they are often intended for
restricting resource usage. New subvolumes going into unrestricted
qgroups by default completely eliminates any usability for resource
restriction the moment anybody but the admin has any shell access to the
system, and that's my biggest complaint against the current behavior.
Putting a snapshot in it's parent's qgroup would completely close this
hole and provide semantics that most people are likely to understand
pretty easily.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-28 11:44 ` Austin S. Hemmelgarn
@ 2017-03-29 5:38 ` Duncan
2017-03-29 11:36 ` Austin S. Hemmelgarn
0 siblings, 1 reply; 24+ messages in thread
From: Duncan @ 2017-03-29 5:38 UTC (permalink / raw)
To: linux-btrfs
Austin S. Hemmelgarn posted on Tue, 28 Mar 2017 07:44:56 -0400 as
excerpted:
> On 2017-03-27 21:49, Qu Wenruo wrote:
>> The problem is, how should we treat subvolume.
>>
>> Btrfs subvolume sits in the middle of directory and (logical) volume
>> used in traditional stacked solution.
>>
>> While we allow normal user to create/delete/modify dir as long as they
>> follow access control, we require privilege to create/delete/modify
>> volumes.
> No, we require privilege to do certain modifications or delete
> subvolumes.
> Regular users can create subvolumes with no privileges whatsoever, and
> most basic directory operations (rename, chown, chmod, etc) work just
> fine within normal UNIX DAC permissions. Unless you're running some
> specially patched kernel or some LSM (SELinux possibly) that somehow
> restricts access to the ioctl, you can always create subvolumes.
(I believe) You misread what QW was trying to say. Note the word "volume"
as opposed to subvolume. Like most regulars here, Qu's too deep into
btrfs to make that sort of mistake, so he wasn't talking about subvolumes.
Rather, he's placing btrfs subvolumes in the middle between directories
and (generic/logical) volumes as normally used, and saying that for
(generic) directories normal users can create/modify/delete without
special privilege (beyond write in the parent), for (generic) volumes
special privileges are necessary to create/modify/delete, and btrfs
subvolumes fall in between, so there's a real decision to be made in
terms of privileges required, do they follow directories and require
nothing special or volumes and require special privs?
Meanwhile, my own position as I've argued is that regardless of the
theoretical debate, as long as we have the very real practical issue of
hard scaling issues for subvolumes/snapshots, there's an equally real
security issue in allowing unrestricted snapshot and to a lessor extent
normal subvolume creation.
So while we might /like/ to make subvolumes more like directories and
require no special privs, until the snapshot scaling and thus security
issue is fixed or at least greatly reduced, as a purely practical
security matter, we really need to restrict snapshot creation to
privileged users.
OTOH, it's worth pointing out that snapshots aren't unique in this
regard, reflinked files have exactly the /same/ scaling issues except
their one-by-one while snapshots tend to be a whole bunch of reflinked
files at once.
Which means the question must then be asked, if we choose the privileged
route for snapshots/subvolumes due to the reflinking security issues, why
aren't we making all reflinking ops privileged ops, or should we? If we
did, cp --reflink=always, among other things, would obviously fail as a
normal user.
So I guess it's a rather harder question than I considered at first.
Maybe we /should/ just treat subvolumes as directories in privileges
terms as well. In that case, however, we really need to have the same
default for subvolume deletion as creation.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-28 15:24 ` Austin S. Hemmelgarn
@ 2017-03-29 5:53 ` Marat Khalili
0 siblings, 0 replies; 24+ messages in thread
From: Marat Khalili @ 2017-03-29 5:53 UTC (permalink / raw)
To: linux-btrfs
> If we were going to reserve something, it should be a high number, not
> a low one. Having 0 reserved makes some sense, but reserving other
> low numbers seems kind of odd when they aren't already reserved.
I did some experiments.
Currently assigning higher-level qgroup to lower-level qgroup is not
possible. Consequently, assigning anything to 0-level qgroup is not
possible.
On the other hand, assigning qgroups while skipping levels (e.g. qgroup
2/P to 10/Q) is possible. So setting default snapshot level high is
technically possible, but you'll not be able to assign these high-level
qgroups anywhere low later.
> although I hadn't realized that the snapshot command _does not_ have
> this argument, when it absolutely should.
It does here in 4.4, it's just not documented :) I too found it by accident.
> Perhaps have an option
Options always suit everyone except developers who need to implement and
support them :)
Here I'd like to wrap up since I seriously doubt any real btrfs
developers are still reading our discussion :)
--
With Best Regards,
Marat Khalili
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Qgroups are not applied when snapshotting a subvol?
2017-03-29 5:38 ` Duncan
@ 2017-03-29 11:36 ` Austin S. Hemmelgarn
0 siblings, 0 replies; 24+ messages in thread
From: Austin S. Hemmelgarn @ 2017-03-29 11:36 UTC (permalink / raw)
To: linux-btrfs
On 2017-03-29 01:38, Duncan wrote:
> Austin S. Hemmelgarn posted on Tue, 28 Mar 2017 07:44:56 -0400 as
> excerpted:
>
>> On 2017-03-27 21:49, Qu Wenruo wrote:
>
>>> The problem is, how should we treat subvolume.
>>>
>>> Btrfs subvolume sits in the middle of directory and (logical) volume
>>> used in traditional stacked solution.
>>>
>>> While we allow normal user to create/delete/modify dir as long as they
>>> follow access control, we require privilege to create/delete/modify
>>> volumes.
>
>> No, we require privilege to do certain modifications or delete
>> subvolumes.
>
>> Regular users can create subvolumes with no privileges whatsoever, and
>> most basic directory operations (rename, chown, chmod, etc) work just
>> fine within normal UNIX DAC permissions. Unless you're running some
>> specially patched kernel or some LSM (SELinux possibly) that somehow
>> restricts access to the ioctl, you can always create subvolumes.
>
> (I believe) You misread what QW was trying to say. Note the word "volume"
> as opposed to subvolume. Like most regulars here, Qu's too deep into
> btrfs to make that sort of mistake, so he wasn't talking about subvolumes.
Ah, you're right, I need to quit reading so fast all the time :)
>
> Rather, he's placing btrfs subvolumes in the middle between directories
> and (generic/logical) volumes as normally used, and saying that for
> (generic) directories normal users can create/modify/delete without
> special privilege (beyond write in the parent), for (generic) volumes
> special privileges are necessary to create/modify/delete, and btrfs
> subvolumes fall in between, so there's a real decision to be made in
> terms of privileges required, do they follow directories and require
> nothing special or volumes and require special privs?
>
>
> Meanwhile, my own position as I've argued is that regardless of the
> theoretical debate, as long as we have the very real practical issue of
> hard scaling issues for subvolumes/snapshots, there's an equally real
> security issue in allowing unrestricted snapshot and to a lessor extent
> normal subvolume creation.
There is a bigger issue though in that subvolumes are a resource a
regular user can create but not destroy and there is no way to limit the
number a user can create. This is never a good situation to be in, but
it appears to have arisen because there are no proper privilege checks
on subvolume deletion, just the 'are you EUID 0?' thing. The ioctl
should require some proof that you could delete the subvolume if it were
a directory, then we could get rid of the whole 'user_subvol_rm_allowed'
thing and move on a bit more sensibly.
>
> So while we might /like/ to make subvolumes more like directories and
> require no special privs, until the snapshot scaling and thus security
> issue is fixed or at least greatly reduced, as a purely practical
> security matter, we really need to restrict snapshot creation to
> privileged users.
>
> OTOH, it's worth pointing out that snapshots aren't unique in this
> regard, reflinked files have exactly the /same/ scaling issues except
> their one-by-one while snapshots tend to be a whole bunch of reflinked
> files at once.
Snapshots and reflinks have scaling issues on every filesystem I've seen
that supports them internally that isn't inherently log structured. The
scaling issue is far worse on BTRFS than most others, but it's by no
means unique to BTRFS.
>
> Which means the question must then be asked, if we choose the privileged
> route for snapshots/subvolumes due to the reflinking security issues, why
> aren't we making all reflinking ops privileged ops, or should we? If we
> did, cp --reflink=always, among other things, would obviously fail as a
> normal user.
Normal users should be able to use reflinks. That is the expectation
set forth by BTRFS already, as well as other filesystems which support
reflinks.
>
> So I guess it's a rather harder question than I considered at first.
> Maybe we /should/ just treat subvolumes as directories in privileges
> terms as well. In that case, however, we really need to have the same
> default for subvolume deletion as creation.
>
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2017-03-29 11:36 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-25 22:03 Qgroups are not applied when snapshotting a subvol? Moritz Sichert
2017-03-26 5:45 ` Duncan
2017-03-27 0:39 ` Qu Wenruo
2017-03-27 3:26 ` Andrei Borzenkov
2017-03-27 3:46 ` Qu Wenruo
2017-03-27 11:02 ` Moritz Sichert
2017-03-27 12:01 ` Austin S. Hemmelgarn
2017-03-27 19:32 ` Chris Murphy
2017-03-27 19:53 ` Roman Mamedov
2017-03-27 20:06 ` Hans van Kranenburg
2017-03-27 21:11 ` Chris Murphy
2017-03-28 2:41 ` Duncan
2017-03-28 5:21 ` Duncan
2017-03-28 3:56 ` Andrei Borzenkov
2017-03-28 11:24 ` Austin S. Hemmelgarn
2017-03-28 12:00 ` Marat Khalili
2017-03-28 12:20 ` Austin S. Hemmelgarn
2017-03-28 13:53 ` Marat Khalili
2017-03-28 15:24 ` Austin S. Hemmelgarn
2017-03-29 5:53 ` Marat Khalili
2017-03-28 1:49 ` Qu Wenruo
2017-03-28 11:44 ` Austin S. Hemmelgarn
2017-03-29 5:38 ` Duncan
2017-03-29 11:36 ` Austin S. Hemmelgarn
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.