Qgroups are not applied when snapshotting a subvol?

Qgroups are not applied when snapshotting a subvol?

[Moritz Sichert]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

Hi,

I tried to configure qgroups on a btrfs filesystem but was really surprised that when you snapshot a subvolume, the snapshot will not be assigned to the qgroup the subvolume was in.

As an example consider the small terminal session in the attachment: I create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on that qgroup. Then I write a file into A and eventually get "disk quota exceeded". Then I create a snapshot of A and call it B. B will not be assigned to 1/1 and writing a file into B confirms that no limits at all are imposed for B.

I feel like I must be missing something here. Considering that creating a snapshot does not require root privileges this would mean that any user can just circumvent any quota and therefore make them useless.

Is there a way to enforce quotas even when a user creates snapshots?


Moritz

[-- Attachment #2: btrfs-qgroups.txt --]
[-- Type: text/plain, Size: 1826 bytes --]

$ btrfs quota enable .
$ btrfs subvolume create A
Create subvolume './A'
$ btrfs qgroup create 1/1 .
$ btrfs qgroup show -p --sync .
qgroupid         rfer         excl parent  
--------         ----         ---- ------  
0/5          16.00KiB     16.00KiB ---     
0/257        16.00KiB     16.00KiB ---     
1/1             0.00B        0.00B ---     
$ btrfs qgroup assign 0/257 1/1 .
$ btrfs qgroup limit 5M 1/1 .
$ dd if=/dev/urandom of=A/foo bs=1M count=4
4+0 records in
4+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0529255 s, 79.2 MB/s
$ btrfs qgroup show -p --sync .
qgroupid         rfer         excl parent  
--------         ----         ---- ------  
0/5          16.00KiB     16.00KiB ---     
0/257         4.02MiB      4.02MiB 1/1     
1/1           4.02MiB      4.02MiB ---     
$ dd if=/dev/urandom of=A/bar bs=1M count=4
dd: error writing 'A/bar': Disk quota exceeded
1+0 records in
0+0 records out
917504 bytes (918 kB, 896 KiB) copied, 0.00408068 s, 225 MB/s
$ btrfs subvolume snapshot A B
Create a snapshot of 'A' in './B'
$ btrfs qgroup show -p --sync .
qgroupid         rfer         excl parent  
--------         ----         ---- ------  
0/5          16.00KiB     16.00KiB ---     
0/257         4.89MiB     16.00KiB 1/1     
0/258         4.89MiB     16.00KiB ---     
1/1           4.89MiB      4.89MiB ---     
$ dd if=/dev/urandom of=B/baz bs=1M count=4
4+0 records in
4+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0174467 s, 240 MB/s
$ btrfs qgroup show -p --sync .
qgroupid         rfer         excl parent  
--------         ----         ---- ------  
0/5          16.00KiB     16.00KiB ---     
0/257         4.89MiB     16.00KiB 1/1     
0/258         8.89MiB      4.02MiB ---     
1/1           4.89MiB      4.89MiB ---

Re: Qgroups are not applied when snapshotting a subvol?

[Duncan]

Moritz Sichert posted on Sat, 25 Mar 2017 23:03:26 +0100 as excerpted:

> I tried to configure qgroups on a btrfs filesystem but was really
> surprised that when you snapshot a subvolume, the snapshot will not be
> assigned to the qgroup the subvolume was in.

> I feel like I must be missing something here. Considering that creating
> a snapshot does not require root privileges this would mean that any
> user can just circumvent any quota and therefore make them useless.
> 
> Is there a way to enforce quotas even when a user creates snapshots?

This doesn't answer your question, but rather is general information you 
should know about btrfs quotas, that I saw no hint in your post that 
you're already aware of.

Btrfs' quota functionality has a long history of breakage, and if it's 
actually working properly now, it's only very recently so.  Don't rely on 
the btrfs quota functionality working correctly or being stable, tho 
certainly, developers are working very hard at making it eventually so.

Meanwhile, even if it is working correctly, btrfs quota functionality has 
serious scaling issues that can seriously slow the progress of btrfs 
maintenance commands such as balance and check to a crawl, making them 
practically unworkable as they may take weeks to finish on today's 
terabyte-scale filesystems -- declaring the filesystem dead, doing a 
fresh mkfs, and restoring from backups can be far faster.  These commands 
are already slow and memory hungry, and having to deal with quotas makes 
things far worse.

So basically you and your btrfs quota use-case fall into one of three 
classes:

1) You know about the issues and are working with the devs to test and 
fix them, helping to make btrfs qgroups and quotas to one day be fully 
reliable and scalable. =:^)

If this matches your situation, THANKS! =:^)

Unfortunately, your post had more of the tone of btrfs quota newbie than 
someone that knew about the situation, which means you're more likely to 
fall into one of the two classes below.

2) Your use-case really needs and relies on quotas to function stably and 
reliably.

Unfortunately, if you're in this class, btrfs is not at this time ready 
for you -- you're better off on a more mature and fully stable filesystem 
that has a reliable quota feature.

3) Your use-case doesn't rely on quotas and you simply believed they were 
a nice "extra" feature you could play with.

In this case the best recommendation is to turn the feature off and leave 
it off until such time as the quota feature is considered generally 
stable and can be recommended, for me personally, several kernel cycles 
after things on the quota front seem to have quieted down and it appears 
to have been working without major problems.

Even then, quotas are likely to come at somewhat of a scalability and 
performance cost, however, which may or may not be worth it, depending on 
what the feature can do for your use-case.


(FWIW, the otherwise stable snapshots feature also has scalability 
issues.  That's why the strong recommendation is to keep them thinned 
down to 1-300 per snapshotted subvolume, under 200 if reasonably 
possible, and under 2000 per filesystem, under 1000 if possible.  But at 
least that feature is stable and the scalability cost arguably well 
justified by the benefits if the numbers are managed well.  
Unfortunately, the same thing can't presently be said about qgroups and 
quotas.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Qgroups are not applied when snapshotting a subvol?

[Qu Wenruo]

At 03/26/2017 06:03 AM, Moritz Sichert wrote:
> Hi,
>
> I tried to configure qgroups on a btrfs filesystem but was really surprised that when you snapshot a subvolume, the snapshot will not be assigned to the qgroup the subvolume was in.
>
> As an example consider the small terminal session in the attachment: I create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on that qgroup. Then I write a file into A and eventually get "disk quota exceeded". Then I create a snapshot of A and call it B. B will not be assigned to 1/1 and writing a file into B confirms that no limits at all are imposed for B.
>
> I feel like I must be missing something here. Considering that creating a snapshot does not require root privileges this would mean that any user can just circumvent any quota and therefore make them useless.
>
> Is there a way to enforce quotas even when a user creates snapshots?
>

Yes, there is always method to attach the subvolume/snapshot to 
specified higher level qgroup.

Just use "btrfs subvolume snapshot -i 1/1".

Thanks,
Qu

>
> Moritz
>
>

Re: Qgroups are not applied when snapshotting a subvol?

[Andrei Borzenkov]

27.03.2017 03:39, Qu Wenruo пишет:
> 
> 
> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>> Hi,
>>
>> I tried to configure qgroups on a btrfs filesystem but was really
>> surprised that when you snapshot a subvolume, the snapshot will not be
>> assigned to the qgroup the subvolume was in.
>>
>> As an example consider the small terminal session in the attachment: I
>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>> that qgroup. Then I write a file into A and eventually get "disk quota
>> exceeded". Then I create a snapshot of A and call it B. B will not be
>> assigned to 1/1 and writing a file into B confirms that no limits at
>> all are imposed for B.
>>
>> I feel like I must be missing something here. Considering that
>> creating a snapshot does not require root privileges this would mean
>> that any user can just circumvent any quota and therefore make them
>> useless.
>>
>> Is there a way to enforce quotas even when a user creates snapshots?
>>
> 
> Yes, there is always method to attach the subvolume/snapshot to
> specified higher level qgroup.
> 
> Just use "btrfs subvolume snapshot -i 1/1".
> 

This requires cooperation from whoever creates subvolume, while the
question was - is it possible to enforce it, without need for explicit
option/action when snapshot is created.

To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
enforcement.

Re: Qgroups are not applied when snapshotting a subvol?

[Qu Wenruo]

At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
> 27.03.2017 03:39, Qu Wenruo пишет:
>>
>>
>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>> Hi,
>>>
>>> I tried to configure qgroups on a btrfs filesystem but was really
>>> surprised that when you snapshot a subvolume, the snapshot will not be
>>> assigned to the qgroup the subvolume was in.
>>>
>>> As an example consider the small terminal session in the attachment: I
>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>> that qgroup. Then I write a file into A and eventually get "disk quota
>>> exceeded". Then I create a snapshot of A and call it B. B will not be
>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>> all are imposed for B.
>>>
>>> I feel like I must be missing something here. Considering that
>>> creating a snapshot does not require root privileges this would mean
>>> that any user can just circumvent any quota and therefore make them
>>> useless.
>>>
>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>
>>
>> Yes, there is always method to attach the subvolume/snapshot to
>> specified higher level qgroup.
>>
>> Just use "btrfs subvolume snapshot -i 1/1".
>>
>
> This requires cooperation from whoever creates subvolume, while the
> question was - is it possible to enforce it, without need for explicit
> option/action when snapshot is created.
>
> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
> enforcement.

What if user really want to create a subvolume assigned another group?

You're implying a *policy* that if source subvolume belongs to a higher 
level qgroup, then snapshot created should also follow that higher level 
qgroup.

However kernel should only provide *mechanisim*, not *policy*.
And btrfs does it, it provides method to do it, whether to do or not is 
users responsibility.

If you want to implement that policy, please do it in a higher level, 
something like SUSE snapper, not in kernel.

Thanks,

Qu

Re: Qgroups are not applied when snapshotting a subvol?

[Moritz Sichert]

Am 27.03.2017 um 05:46 schrieb Qu Wenruo:
> 
> 
> At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
>> 27.03.2017 03:39, Qu Wenruo пишет:
>>>
>>>
>>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>>> Hi,
>>>>
>>>> I tried to configure qgroups on a btrfs filesystem but was really
>>>> surprised that when you snapshot a subvolume, the snapshot will not be
>>>> assigned to the qgroup the subvolume was in.
>>>>
>>>> As an example consider the small terminal session in the attachment: I
>>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>>> that qgroup. Then I write a file into A and eventually get "disk quota
>>>> exceeded". Then I create a snapshot of A and call it B. B will not be
>>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>>> all are imposed for B.
>>>>
>>>> I feel like I must be missing something here. Considering that
>>>> creating a snapshot does not require root privileges this would mean
>>>> that any user can just circumvent any quota and therefore make them
>>>> useless.
>>>>
>>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>>
>>>
>>> Yes, there is always method to attach the subvolume/snapshot to
>>> specified higher level qgroup.
>>>
>>> Just use "btrfs subvolume snapshot -i 1/1".
>>>
>>
>> This requires cooperation from whoever creates subvolume, while the
>> question was - is it possible to enforce it, without need for explicit
>> option/action when snapshot is created.
>>
>> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
>> enforcement.
> 
> What if user really want to create a subvolume assigned another group?
> 
> You're implying a *policy* that if source subvolume belongs to a higher level qgroup, then snapshot created should also follow that higher level qgroup.
> 
> However kernel should only provide *mechanisim*, not *policy*.
> And btrfs does it, it provides method to do it, whether to do or not is users responsibility.
> 
> If you want to implement that policy, please do it in a higher level, something like SUSE snapper, not in kernel.

The problem is, I can't enforce the policy because *every user* can create snapshots. Even if I would restrict the btrfs executable so that only root can execute it, this doesn't help. As using the ioctl for btrfs is allowed for any user, they could just get the executable from somewhere else.


Moritz

Re: Qgroups are not applied when snapshotting a subvol?

[Austin S. Hemmelgarn]

On 2017-03-27 07:02, Moritz Sichert wrote:
> Am 27.03.2017 um 05:46 schrieb Qu Wenruo:
>>
>>
>> At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
>>> 27.03.2017 03:39, Qu Wenruo пишет:
>>>>
>>>>
>>>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>>>> Hi,
>>>>>
>>>>> I tried to configure qgroups on a btrfs filesystem but was really
>>>>> surprised that when you snapshot a subvolume, the snapshot will not be
>>>>> assigned to the qgroup the subvolume was in.
>>>>>
>>>>> As an example consider the small terminal session in the attachment: I
>>>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>>>> that qgroup. Then I write a file into A and eventually get "disk quota
>>>>> exceeded". Then I create a snapshot of A and call it B. B will not be
>>>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>>>> all are imposed for B.
>>>>>
>>>>> I feel like I must be missing something here. Considering that
>>>>> creating a snapshot does not require root privileges this would mean
>>>>> that any user can just circumvent any quota and therefore make them
>>>>> useless.
>>>>>
>>>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>>>
>>>>
>>>> Yes, there is always method to attach the subvolume/snapshot to
>>>> specified higher level qgroup.
>>>>
>>>> Just use "btrfs subvolume snapshot -i 1/1".
>>>>
>>>
>>> This requires cooperation from whoever creates subvolume, while the
>>> question was - is it possible to enforce it, without need for explicit
>>> option/action when snapshot is created.
>>>
>>> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
>>> enforcement.
>>
>> What if user really want to create a subvolume assigned another group?
>>
>> You're implying a *policy* that if source subvolume belongs to a higher level qgroup, then snapshot created should also follow that higher level qgroup.
>>
>> However kernel should only provide *mechanisim*, not *policy*.
>> And btrfs does it, it provides method to do it, whether to do or not is users responsibility.
>>
>> If you want to implement that policy, please do it in a higher level, something like SUSE snapper, not in kernel.
>
> The problem is, I can't enforce the policy because *every user* can create snapshots. Even if I would restrict the btrfs executable so that only root can execute it, this doesn't help. As using the ioctl for btrfs is allowed for any user, they could just get the executable from somewhere else.
To reiterate and reinforce this:
If it is not possible to enforce new subvolumes counting for their 
parent quota, and there is no option to prevent non-root (or 
non-CAP_SYS_ADMIN) users from creating new subvolumes, then BTRFS 
qgroups are useless on any system with shell access because a user can 
trivially escape their quota restrictions (or hide from accounting) by 
creating a new subvolume which is outside of their qgroup and storing 
data there.

Ideally, there should be an option to disable user subvolume creation 
(it arguably should be the default, because of resource exhaustion 
issues, but that's a separate argument), and there should be an option 
in the kernel to force specific behavior.  Both cases are policy, but 
they are policy that can only be concretely enforced _by the kernel_.

Re: Qgroups are not applied when snapshotting a subvol?

[Chris Murphy]

How about if qgroups are enabled, then non-root user is prevented from
creating new subvolumes?

Or is there a way for a new nested subvolume to be included in its
parent's quota, rather than the new subvolume having a whole new quota
limit?

Tricky problem.


Chris Murphy

Re: Qgroups are not applied when snapshotting a subvol?

[Roman Mamedov]

On Mon, 27 Mar 2017 13:32:47 -0600
Chris Murphy <lists@colorremedies.com> wrote:

> How about if qgroups are enabled, then non-root user is prevented from
> creating new subvolumes?

That sounds like, if you turn your headlights on in a car, then in-vehicle air
conditioner randomly stops working. :)

Two things only vaguely related from the end user's point of view.

> Or is there a way for a new nested subvolume to be included in its
> parent's quota, rather than the new subvolume having a whole new quota
> limit?

Either that, or a separate "allow non-root user subvolumes/snapshots creation"
mount option. There is already one for deletion, after all.

       user_subvol_rm_allowed
              Allow subvolumes to be deleted by a  non-root  user.   Use  with
              caution.

-- 
With respect,
Roman

Re: Qgroups are not applied when snapshotting a subvol?

[Hans van Kranenburg]

On 03/27/2017 09:53 PM, Roman Mamedov wrote:
> On Mon, 27 Mar 2017 13:32:47 -0600
> Chris Murphy <lists@colorremedies.com> wrote:
> 
>> How about if qgroups are enabled, then non-root user is prevented from
>> creating new subvolumes?
> 
> That sounds like, if you turn your headlights on in a car, then in-vehicle air
> conditioner randomly stops working. :)
> 
> Two things only vaguely related from the end user's point of view.
> 
>> Or is there a way for a new nested subvolume to be included in its
>> parent's quota, rather than the new subvolume having a whole new quota
>> limit?
> 
> Either that, or a separate "allow non-root user subvolumes/snapshots creation"
> mount option. There is already one for deletion, after all.
> 
>        user_subvol_rm_allowed
>               Allow subvolumes to be deleted by a  non-root  user.   Use  with
>               caution.
> 

What are actual use cases for creating subvolumes by 'normal' users?

Does someone have an example?

Why is it possible at all, by default?

-- 
Hans van Kranenburg

Re: Qgroups are not applied when snapshotting a subvol?

[Chris Murphy]

On Mon, Mar 27, 2017 at 2:06 PM, Hans van Kranenburg
<hans.van.kranenburg@mendix.com> wrote:
> On 03/27/2017 09:53 PM, Roman Mamedov wrote:
>> On Mon, 27 Mar 2017 13:32:47 -0600
>> Chris Murphy <lists@colorremedies.com> wrote:
>>
>>> How about if qgroups are enabled, then non-root user is prevented from
>>> creating new subvolumes?
>>
>> That sounds like, if you turn your headlights on in a car, then in-vehicle air
>> conditioner randomly stops working. :)
>>
>> Two things only vaguely related from the end user's point of view.
>>
>>> Or is there a way for a new nested subvolume to be included in its
>>> parent's quota, rather than the new subvolume having a whole new quota
>>> limit?
>>
>> Either that, or a separate "allow non-root user subvolumes/snapshots creation"
>> mount option. There is already one for deletion, after all.
>>
>>        user_subvol_rm_allowed
>>               Allow subvolumes to be deleted by a  non-root  user.   Use  with
>>               caution.
>>
>
> What are actual use cases for creating subvolumes by 'normal' users?
>
> Does someone have an example?
>
> Why is it possible at all, by default?

I have a single git subvolume in my user directory, inside of which
are various git clones. And I periodically snapshot the git subvolume
as a regular user.

If I can't create subvolumes as a regular user then by extension it'd
mean I can't create snapshots of my own home directory, or any other
subvolumes I exclusively own.

-- 
Chris Murphy

Re: Qgroups are not applied when snapshotting a subvol?

[Duncan]

Chris Murphy posted on Mon, 27 Mar 2017 15:11:34 -0600 as excerpted:

>> What are actual use cases for creating subvolumes by 'normal' users?
>>
>> Does someone have an example?
>>
>> Why is it possible at all, by default?
> 
> I have a single git subvolume in my user directory, inside of which are
> various git clones. And I periodically snapshot the git subvolume as a
> regular user.
> 
> If I can't create subvolumes as a regular user then by extension it'd
> mean I can't create snapshots of my own home directory, or any other
> subvolumes I exclusively own.

One rather big problem with what, with snapshots (which are a special 
kind of subvolume), is that btrfs has known scaling issues when the 
number of snapshots gets too high.  Combine that with allowing users to 
make but not delete snapshots, and you have a huge invitation to scaling 
headaches due to the number of snapshots.

Really, the two permissions subvolume/snapshot creation, and deletion, 
should be synchronized.  Allowing subvolume deletion clearly has security 
issues, but so does allowing creation without allowing deletion.  They 
both really have to go together, and be allowed only for "trusted" users, 
with the option of whether that's root-only, or a subset of users (say 
via group perms), or all users, being up to the local admin, basically, a 
mount option.

Which in usual terms means making the perms root-only, with the binary 
set to some controlled-access group and set-SUID-root (or appropriate 
security attributes, I'm drawing a blank on the word I want ATM), and 
then letting the admin control access via group membership.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Qgroups are not applied when snapshotting a subvol?

[Duncan]

Duncan posted on Tue, 28 Mar 2017 02:41:52 +0000 as excerpted:

> Which in usual terms means making the perms root-only, with the binary
> set to some controlled-access group and set-SUID-root (or appropriate
> security attributes, I'm drawing a blank on the word I want ATM), and
> then letting the admin control access via group membership.

File capacities (what was eluding me).

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Qgroups are not applied when snapshotting a subvol?

[Andrei Borzenkov]

27.03.2017 22:32, Chris Murphy пишет:
> How about if qgroups are enabled, then non-root user is prevented from
> creating new subvolumes?
> 
> Or is there a way for a new nested subvolume to be included in its
> parent's quota, rather than the new subvolume having a whole new quota
> limit?
> 

They are not even nested in any sense.

btrfs sub create /some/parent/A /other/parent/B

There is no direct relation between A and B except that B is clone of A
(but I do not know whether this fact is recorded somewhere in subvolume
properties).

Re: Qgroups are not applied when snapshotting a subvol?

[Austin S. Hemmelgarn]

On 2017-03-27 15:32, Chris Murphy wrote:
> How about if qgroups are enabled, then non-root user is prevented from
> creating new subvolumes?
>
> Or is there a way for a new nested subvolume to be included in its
> parent's quota, rather than the new subvolume having a whole new quota
> limit?
>
> Tricky problem.
The default should be to inherit the qgroup of the parent subvolume. 
The organization of subvolumes is hierarchical, and sane people expect 
things to behave as they look.  Taking another angle, on ZFS, 'nested' 
(nested in quotes because ZFS' definition of 'nested' zvols is weird) 
inherit their parent's quota and reservations (essentially reverse 
quota), and they're not even inherently nested in the filesystem like 
subvolumes are, so we're differing from the only other widely used 
system that implements things in a similar manner.

As far as the subvolume thing, there should be an option to disable user 
creation of subvolumes, and ideally it should be on by default because:
1. Users can't delete subvolumes by default.  This means they can create 
but not destroy a resource by default, which means that a user can 
pretty easily accidentally cause issues for the system as a whole.
2. Correlating with 1, users being able to delete subvolumes by default 
is not safe on multiple levels (easy accidental data loss, numerous 
other issues), and thus user subvolume removal being off by default is 
significantly safer.

Re: Qgroups are not applied when snapshotting a subvol?

[Marat Khalili]

> The default should be to inherit the qgroup of the parent subvolume.
This behaviour is only good for this particular use-case. In general 
case, qgroups of subvolume and snapshots should exist separately, and 
both can be included in some higher level qgroup (after all, that's what 
qgroup hierarchy is for).

In my system I found it convenient to include subvolume and its 
snapshots in qgroup 1/N, where 0/N is qgroup of bare subvolume. I think 
adopting this behaviour as default would be more sensible.

--

With Best Regards,
Marat Khalili

On 28/03/17 14:24, Austin S. Hemmelgarn wrote:
> On 2017-03-27 15:32, Chris Murphy wrote:
>> How about if qgroups are enabled, then non-root user is prevented from
>> creating new subvolumes?
>>
>> Or is there a way for a new nested subvolume to be included in its
>> parent's quota, rather than the new subvolume having a whole new quota
>> limit?
>>
>> Tricky problem.
> The default should be to inherit the qgroup of the parent subvolume. 
> The organization of subvolumes is hierarchical, and sane people expect 
> things to behave as they look.  Taking another angle, on ZFS, 'nested' 
> (nested in quotes because ZFS' definition of 'nested' zvols is weird) 
> inherit their parent's quota and reservations (essentially reverse 
> quota), and they're not even inherently nested in the filesystem like 
> subvolumes are, so we're differing from the only other widely used 
> system that implements things in a similar manner.
>
> As far as the subvolume thing, there should be an option to disable 
> user creation of subvolumes, and ideally it should be on by default 
> because:
> 1. Users can't delete subvolumes by default.  This means they can 
> create but not destroy a resource by default, which means that a user 
> can pretty easily accidentally cause issues for the system as a whole.
> 2. Correlating with 1, users being able to delete subvolumes by 
> default is not safe on multiple levels (easy accidental data loss, 
> numerous other issues), and thus user subvolume removal being off by 
> default is significantly safer.
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Qgroups are not applied when snapshotting a subvol?

[Austin S. Hemmelgarn]

On 2017-03-28 08:00, Marat Khalili wrote:
>> The default should be to inherit the qgroup of the parent subvolume.
> This behaviour is only good for this particular use-case. In general
> case, qgroups of subvolume and snapshots should exist separately, and
> both can be included in some higher level qgroup (after all, that's what
> qgroup hierarchy is for).
>
> In my system I found it convenient to include subvolume and its
> snapshots in qgroup 1/N, where 0/N is qgroup of bare subvolume. I think
> adopting this behaviour as default would be more sensible.
There are a couple of reasons I'm advocating the specific behavior I 
outlined:
1. It doesn't require any specific qgroup setup.  By definition, you can 
be 100% certain that the destination qgroup exists, and that you won't 
need to create new qgroups behind the user's back (given your 
suggestion, what happens when qgroup 1/N doesn't exist?).
2. Just because it's the default, doesn't mean that the subvolume can't 
be reassigned to a different qgroup.  This also would not remove the 
ability to assign a specific qgroup through the snapshot creation 
command.  This is arguably a general point in favor of having any 
default of course, but it's still worth pointing out.
3. Because BTRFS has COW semantics, the new snapshot should take up near 
zero space in the qgroup of it's parent.
4. This correlates with the behavior most people expect based on ZFS and 
LVM, which is that snapshots are tied to their parent.

At a minimum, it should belong to _some_ qgroup.  This could also be 
covered by having a designated 'default' qgroup that all new subvolumes 
created without a specified qgroup get put in, but I feel that that is 
somewhat orthogonal to the issue of how snapshots are handled.

> --
>
> With Best Regards,
> Marat Khalili
>
> On 28/03/17 14:24, Austin S. Hemmelgarn wrote:
>> On 2017-03-27 15:32, Chris Murphy wrote:
>>> How about if qgroups are enabled, then non-root user is prevented from
>>> creating new subvolumes?
>>>
>>> Or is there a way for a new nested subvolume to be included in its
>>> parent's quota, rather than the new subvolume having a whole new quota
>>> limit?
>>>
>>> Tricky problem.
>> The default should be to inherit the qgroup of the parent subvolume.
>> The organization of subvolumes is hierarchical, and sane people expect
>> things to behave as they look.  Taking another angle, on ZFS, 'nested'
>> (nested in quotes because ZFS' definition of 'nested' zvols is weird)
>> inherit their parent's quota and reservations (essentially reverse
>> quota), and they're not even inherently nested in the filesystem like
>> subvolumes are, so we're differing from the only other widely used
>> system that implements things in a similar manner.
>>
>> As far as the subvolume thing, there should be an option to disable
>> user creation of subvolumes, and ideally it should be on by default
>> because:
>> 1. Users can't delete subvolumes by default.  This means they can
>> create but not destroy a resource by default, which means that a user
>> can pretty easily accidentally cause issues for the system as a whole.
>> 2. Correlating with 1, users being able to delete subvolumes by
>> default is not safe on multiple levels (easy accidental data loss,
>> numerous other issues), and thus user subvolume removal being off by
>> default is significantly safer.
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Qgroups are not applied when snapshotting a subvol?

[Marat Khalili]

> There are a couple of reasons I'm advocating the specific behavior I 
> outlined:
Some of your points are valid, but some break current behaviour and 
expectations or create technical difficulties.


> 1. It doesn't require any specific qgroup setup.  By definition, you 
> can be 100% certain that the destination qgroup exists, and that you 
> won't need to create new qgroups behind the user's back (given your 
> suggestion, what happens when qgroup 1/N doesn't exist?).
This is a general problem with current qgroups: you have to reference 
them by some random numbers, not by user-assigned names like files. It 
would need to be fixed sooner or later with syntax like L:<path> in 
place of L/N, or even some special syntax made specifically for path 
snapshots.

BTW, what about reserving level 1 for qgroups describing subvolumes and 
all their snapshots and forbidding manual management of qgroups at this 
level?


> 2. Just because it's the default, doesn't mean that the subvolume 
> can't be reassigned to a different qgroup.  This also would not remove 
> the ability to assign a specific qgroup through the snapshot creation 
> command.  This is arguably a general point in favor of having any 
> default of course, but it's still worth pointing out.
Currently 0/N qgroups are special in that they are created automatically 
and belong to the bottom of the hierarchy. It would be very nice to keep 
it this way.

Changing qgroup assignments after snapshot creation is very inconvenient 
because it requires quota rescan and thus blocks all other quota operations.


> 3. Because BTRFS has COW semantics, the new snapshot should take up 
> near zero space in the qgroup of it's parent.

Indeed it works this way in my experiments if you assign snapshot to 1/N 
qgroup at creation where 0/N also belongs. Moreover, it does not require 
quota rescan, which is very nice.


> 4. This correlates with the behavior most people expect based on ZFS 
> and LVM, which is that snapshots are tied to their parent.

I'm not against tying it to the parent. I'm against removing snapshot's 
own qgroup.


> At a minimum, it should belong to _some_ qgroup.  This could also be 
> covered by having a designated 'default' qgroup that all new 
> subvolumes created without a specified qgroup get put in, but I feel 
> that that is somewhat orthogonal to the issue of how snapshots are 
> handled. 
It belongs to its own 0/N' qgroup, but this is not probably what you mean.


--

With Best Regards,
Marat Khalili

Re: Qgroups are not applied when snapshotting a subvol?

[Austin S. Hemmelgarn]

On 2017-03-28 09:53, Marat Khalili wrote:
>> There are a couple of reasons I'm advocating the specific behavior I
>> outlined:
> Some of your points are valid, but some break current behaviour and
> expectations or create technical difficulties.
>
>
>> 1. It doesn't require any specific qgroup setup.  By definition, you
>> can be 100% certain that the destination qgroup exists, and that you
>> won't need to create new qgroups behind the user's back (given your
>> suggestion, what happens when qgroup 1/N doesn't exist?).
> This is a general problem with current qgroups: you have to reference
> them by some random numbers, not by user-assigned names like files. It
> would need to be fixed sooner or later with syntax like L:<path> in
> place of L/N, or even some special syntax made specifically for path
> snapshots.
>
> BTW, what about reserving level 1 for qgroups describing subvolumes and
> all their snapshots and forbidding manual management of qgroups at this
> level?
If we were going to reserve something, it should be a high number, not a 
low one.  Having 0 reserved makes some sense, but reserving other low 
numbers seems kind of odd when they aren't already reserved.
>
>
>> 2. Just because it's the default, doesn't mean that the subvolume
>> can't be reassigned to a different qgroup.  This also would not remove
>> the ability to assign a specific qgroup through the snapshot creation
>> command.  This is arguably a general point in favor of having any
>> default of course, but it's still worth pointing out.
> Currently 0/N qgroups are special in that they are created automatically
> and belong to the bottom of the hierarchy. It would be very nice to keep
> it this way.
>
> Changing qgroup assignments after snapshot creation is very inconvenient
> because it requires quota rescan and thus blocks all other quota
> operations.
The subvolume create command lets you set a specific qgroup on subvolume 
creation.  When I said it won't prevent assigning a specific qgroup, I 
meant using that, not some after-the-fact assignment, although I hadn't 
realized that the snapshot command _does not_ have this argument, when 
it absolutely should.
>
>
>> 3. Because BTRFS has COW semantics, the new snapshot should take up
>> near zero space in the qgroup of it's parent.
>
> Indeed it works this way in my experiments if you assign snapshot to 1/N
> qgroup at creation where 0/N also belongs. Moreover, it does not require
> quota rescan, which is very nice.
I mean if it's placed in that qgroup itself.  If you create a snapshot 
of an idle subvolume, you only effectively double the metadata (and 
possibly not even that completely), so unless it's got a very large 
number of small files, it should take up next to zero space if it gets 
put in it's parent's qgroup.
>
>
>> 4. This correlates with the behavior most people expect based on ZFS
>> and LVM, which is that snapshots are tied to their parent.
>
> I'm not against tying it to the parent. I'm against removing snapshot's
> own qgroup.
Perhaps have an option (in the filesystem, not the mount options, this 
is tied to qgroup policy, which should be independent of who mounts the 
FS where) that specifies one of:
1. Current behavior.
2. Join parent's qgroup.
3. Join some other predefined qgroup.

>
>
>> At a minimum, it should belong to _some_ qgroup.  This could also be
>> covered by having a designated 'default' qgroup that all new
>> subvolumes created without a specified qgroup get put in, but I feel
>> that that is somewhat orthogonal to the issue of how snapshots are
>> handled.
> It belongs to its own 0/N' qgroup, but this is not probably what you mean.
The 0/N groups are accounting only until some limit is set, correct?  If 
that is the case, then that really isn't what I mean.  Quotas can be 
used for accounting, but when used, they are often intended for 
restricting resource usage.  New subvolumes going into unrestricted 
qgroups by default completely eliminates any usability for resource 
restriction the moment anybody but the admin has any shell access to the 
system, and that's my biggest complaint against the current behavior. 
Putting a snapshot in it's parent's qgroup would completely close this 
hole and provide semantics that most people are likely to understand 
pretty easily.

Re: Qgroups are not applied when snapshotting a subvol?

[Marat Khalili]

> If we were going to reserve something, it should be a high number, not 
> a low one.  Having 0 reserved makes some sense, but reserving other 
> low numbers seems kind of odd when they aren't already reserved. 

I did some experiments.

Currently assigning higher-level qgroup to lower-level qgroup is not 
possible. Consequently, assigning anything to 0-level qgroup is not 
possible.

On the other hand, assigning qgroups while skipping levels (e.g. qgroup 
2/P to 10/Q) is possible. So setting default snapshot level high is 
technically possible, but you'll not be able to assign these high-level 
qgroups anywhere low later.


> although I hadn't realized that the snapshot command _does not_ have 
> this argument, when it absolutely should. 
It does here in 4.4, it's just not documented :) I too found it by accident.


> Perhaps have an option
Options always suit everyone except developers who need to implement and 
support them :)

Here I'd like to wrap up since I seriously doubt any real btrfs 
developers are still reading our discussion :)

--

With Best Regards,
Marat Khalili

Re: Qgroups are not applied when snapshotting a subvol?

[Qu Wenruo]

At 03/27/2017 08:01 PM, Austin S. Hemmelgarn wrote:
> On 2017-03-27 07:02, Moritz Sichert wrote:
>> Am 27.03.2017 um 05:46 schrieb Qu Wenruo:
>>>
>>>
>>> At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
>>>> 27.03.2017 03:39, Qu Wenruo пишет:
>>>>>
>>>>>
>>>>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I tried to configure qgroups on a btrfs filesystem but was really
>>>>>> surprised that when you snapshot a subvolume, the snapshot will
>>>>>> not be
>>>>>> assigned to the qgroup the subvolume was in.
>>>>>>
>>>>>> As an example consider the small terminal session in the
>>>>>> attachment: I
>>>>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>>>>> that qgroup. Then I write a file into A and eventually get "disk
>>>>>> quota
>>>>>> exceeded". Then I create a snapshot of A and call it B. B will not be
>>>>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>>>>> all are imposed for B.
>>>>>>
>>>>>> I feel like I must be missing something here. Considering that
>>>>>> creating a snapshot does not require root privileges this would mean
>>>>>> that any user can just circumvent any quota and therefore make them
>>>>>> useless.
>>>>>>
>>>>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>>>>
>>>>>
>>>>> Yes, there is always method to attach the subvolume/snapshot to
>>>>> specified higher level qgroup.
>>>>>
>>>>> Just use "btrfs subvolume snapshot -i 1/1".
>>>>>
>>>>
>>>> This requires cooperation from whoever creates subvolume, while the
>>>> question was - is it possible to enforce it, without need for explicit
>>>> option/action when snapshot is created.
>>>>
>>>> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
>>>> enforcement.
>>>
>>> What if user really want to create a subvolume assigned another group?
>>>
>>> You're implying a *policy* that if source subvolume belongs to a
>>> higher level qgroup, then snapshot created should also follow that
>>> higher level qgroup.
>>>
>>> However kernel should only provide *mechanisim*, not *policy*.
>>> And btrfs does it, it provides method to do it, whether to do or not
>>> is users responsibility.
>>>
>>> If you want to implement that policy, please do it in a higher level,
>>> something like SUSE snapper, not in kernel.
>>
>> The problem is, I can't enforce the policy because *every user* can
>> create snapshots. Even if I would restrict the btrfs executable so
>> that only root can execute it, this doesn't help. As using the ioctl
>> for btrfs is allowed for any user, they could just get the executable
>> from somewhere else.
> To reiterate and reinforce this:
> If it is not possible to enforce new subvolumes counting for their
> parent quota, and there is no option to prevent non-root (or
> non-CAP_SYS_ADMIN) users from creating new subvolumes, then BTRFS
> qgroups are useless on any system with shell access because a user can
> trivially escape their quota restrictions (or hide from accounting) by
> creating a new subvolume which is outside of their qgroup and storing
> data there.
>
> Ideally, there should be an option to disable user subvolume creation
> (it arguably should be the default, because of resource exhaustion
> issues, but that's a separate argument), and there should be an option
> in the kernel to force specific behavior.  Both cases are policy, but
> they are policy that can only be concretely enforced _by the kernel_.
>
>
The problem is, how should we treat subvolume.

Btrfs subvolume sits in the middle of directory and (logical) volume 
used in traditional stacked solution.

While we allow normal user to create/delete/modify dir as long as they 
follow access control, we require privilege to create/delete/modify volumes.

Developers chose to treat btrfs subvolume as dir, makes it quite easy to 
operate for normal use case, sacrificing qgroup limit which is not a 
major function (or even did not exist) at that time.

IIRC at the beginning time of btrfs, we don't have a full idea of use 
cases could be.
This is common, a lot of problems(even bad design) can only be found 
after enough feedback from end users.

Personally speaking, I prefer to restrict subvolume creation/deletion to 
privilege users only, and uses a daemon as a proxy to do such privilege 
operation.
So we can do better accounting/access control without bothering the kernel.

But that makes a big behavior difference, I'm afraid this won't become true.

Thanks,
Qu

Re: Qgroups are not applied when snapshotting a subvol?

[Austin S. Hemmelgarn]

On 2017-03-27 21:49, Qu Wenruo wrote:
>
>
> At 03/27/2017 08:01 PM, Austin S. Hemmelgarn wrote:
>> On 2017-03-27 07:02, Moritz Sichert wrote:
>>> Am 27.03.2017 um 05:46 schrieb Qu Wenruo:
>>>>
>>>>
>>>> At 03/27/2017 11:26 AM, Andrei Borzenkov wrote:
>>>>> 27.03.2017 03:39, Qu Wenruo пишет:
>>>>>>
>>>>>>
>>>>>> At 03/26/2017 06:03 AM, Moritz Sichert wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I tried to configure qgroups on a btrfs filesystem but was really
>>>>>>> surprised that when you snapshot a subvolume, the snapshot will
>>>>>>> not be
>>>>>>> assigned to the qgroup the subvolume was in.
>>>>>>>
>>>>>>> As an example consider the small terminal session in the
>>>>>>> attachment: I
>>>>>>> create a subvol A, assign it to qgroup 1/1 and set a limit of 5M on
>>>>>>> that qgroup. Then I write a file into A and eventually get "disk
>>>>>>> quota
>>>>>>> exceeded". Then I create a snapshot of A and call it B. B will
>>>>>>> not be
>>>>>>> assigned to 1/1 and writing a file into B confirms that no limits at
>>>>>>> all are imposed for B.
>>>>>>>
>>>>>>> I feel like I must be missing something here. Considering that
>>>>>>> creating a snapshot does not require root privileges this would mean
>>>>>>> that any user can just circumvent any quota and therefore make them
>>>>>>> useless.
>>>>>>>
>>>>>>> Is there a way to enforce quotas even when a user creates snapshots?
>>>>>>>
>>>>>>
>>>>>> Yes, there is always method to attach the subvolume/snapshot to
>>>>>> specified higher level qgroup.
>>>>>>
>>>>>> Just use "btrfs subvolume snapshot -i 1/1".
>>>>>>
>>>>>
>>>>> This requires cooperation from whoever creates subvolume, while the
>>>>> question was - is it possible to enforce it, without need for explicit
>>>>> option/action when snapshot is created.
>>>>>
>>>>> To reiterate - if user omits "-i 1/1" (s)he "escapes" from quota
>>>>> enforcement.
>>>>
>>>> What if user really want to create a subvolume assigned another group?
>>>>
>>>> You're implying a *policy* that if source subvolume belongs to a
>>>> higher level qgroup, then snapshot created should also follow that
>>>> higher level qgroup.
>>>>
>>>> However kernel should only provide *mechanisim*, not *policy*.
>>>> And btrfs does it, it provides method to do it, whether to do or not
>>>> is users responsibility.
>>>>
>>>> If you want to implement that policy, please do it in a higher level,
>>>> something like SUSE snapper, not in kernel.
>>>
>>> The problem is, I can't enforce the policy because *every user* can
>>> create snapshots. Even if I would restrict the btrfs executable so
>>> that only root can execute it, this doesn't help. As using the ioctl
>>> for btrfs is allowed for any user, they could just get the executable
>>> from somewhere else.
>> To reiterate and reinforce this:
>> If it is not possible to enforce new subvolumes counting for their
>> parent quota, and there is no option to prevent non-root (or
>> non-CAP_SYS_ADMIN) users from creating new subvolumes, then BTRFS
>> qgroups are useless on any system with shell access because a user can
>> trivially escape their quota restrictions (or hide from accounting) by
>> creating a new subvolume which is outside of their qgroup and storing
>> data there.
>>
>> Ideally, there should be an option to disable user subvolume creation
>> (it arguably should be the default, because of resource exhaustion
>> issues, but that's a separate argument), and there should be an option
>> in the kernel to force specific behavior.  Both cases are policy, but
>> they are policy that can only be concretely enforced _by the kernel_.
>>
>>
> The problem is, how should we treat subvolume.
>
> Btrfs subvolume sits in the middle of directory and (logical) volume
> used in traditional stacked solution.
>
> While we allow normal user to create/delete/modify dir as long as they
> follow access control, we require privilege to create/delete/modify
> volumes.
No, we require privilege to do certain modifications or delete 
subvolumes.  Regular users can create subvolumes with no privileges 
whatsoever, and most basic directory operations (rename, chown, chmod, 
etc) work just fine within normal UNIX DAC permissions.  Unless you're 
running some specially patched kernel or some LSM (SELinux possibly) 
that somehow restricts access to the ioctl, you can always create 
subvolumes.

This is part of the reason that I'm personally hesitant to use BTRFS on 
systems where end users have shell access, it's a DoS waiting to happen.
>
> Developers chose to treat btrfs subvolume as dir, makes it quite easy to
> operate for normal use case, sacrificing qgroup limit which is not a
> major function (or even did not exist) at that time.
>
> IIRC at the beginning time of btrfs, we don't have a full idea of use
> cases could be.
> This is common, a lot of problems(even bad design) can only be found
> after enough feedback from end users.
>
> Personally speaking, I prefer to restrict subvolume creation/deletion to
> privilege users only, and uses a daemon as a proxy to do such privilege
> operation.
> So we can do better accounting/access control without bothering the kernel.
I will agree that a daemon for this can be useful, but even if we add a 
mount option to restrict operations on subvolumes by normal users, we 
can still provide the option of a daemon.  On something like a single 
user system, there is not much advantage to having some complex access 
control in place.  I'm not saying we should put any kind of complex ACL 
into the kernel, but that we should at least have some all-or-nothing 
switch in the mount options that controls if unprivileged users can 
perform subvolume operations.  Ideally it would be just one option, but 
unfortunately we provided somewhat nonsensical initial semantics that we 
now have to continue to support.

Looking at the qgroup stuff specifically though, snapshots not 
inheriting their parent's qgroup seems really odd to me.  There are two 
things that come to mind, and I'd love to see both personally:
1. Provide an option to have snapshots inherit their parent's qgroup. 
This would eliminate the 'surprise' that initially sparked this 
discussion, and deal with tools that aren't qgroup aware.
2. Provide an option to specify the default qgroup for new subvolumes, 
instead of them being created with no qgroup.  This would cover the rest 
of things, and provide some further usefulness (for example, you could 
leave this default qgroup empty most of the time and have monitoring 
software alert you if it suddenly had data in it to detect stuff 
creating subvolumes behind your back).
Both sound more to me like they should probably be specified somewhere 
in the filesystem itself, not the mount options.
>
> But that makes a big behavior difference, I'm afraid this won't become
> true.
I'm definitely with you on having the ability to restrict subvolume 
operations to privileged users, I just don't feel that the suggested 
methodology is enough by itself.

Re: Qgroups are not applied when snapshotting a subvol?

[Duncan]

Austin S. Hemmelgarn posted on Tue, 28 Mar 2017 07:44:56 -0400 as
excerpted:

> On 2017-03-27 21:49, Qu Wenruo wrote:

>> The problem is, how should we treat subvolume.
>>
>> Btrfs subvolume sits in the middle of directory and (logical) volume
>> used in traditional stacked solution.
>>
>> While we allow normal user to create/delete/modify dir as long as they
>> follow access control, we require privilege to create/delete/modify
>> volumes.

> No, we require privilege to do certain modifications or delete
> subvolumes.

>  Regular users can create subvolumes with no privileges whatsoever, and
> most basic directory operations (rename, chown, chmod, etc) work just
> fine within normal UNIX DAC permissions.  Unless you're running some
> specially patched kernel or some LSM (SELinux possibly) that somehow
> restricts access to the ioctl, you can always create subvolumes.

(I believe) You misread what QW was trying to say. Note the word "volume" 
as opposed to subvolume.  Like most regulars here, Qu's too deep into 
btrfs to make that sort of mistake, so he wasn't talking about subvolumes.

Rather, he's placing btrfs subvolumes in the middle between directories 
and (generic/logical) volumes as normally used, and saying that for 
(generic) directories normal users can create/modify/delete without 
special privilege (beyond write in the parent), for (generic) volumes 
special privileges are necessary to create/modify/delete, and btrfs 
subvolumes fall in between, so there's a real decision to be made in 
terms of privileges required, do they follow directories and require 
nothing special or volumes and require special privs?


Meanwhile, my own position as I've argued is that regardless of the 
theoretical debate, as long as we have the very real practical issue of 
hard scaling issues for subvolumes/snapshots, there's an equally real 
security issue in allowing unrestricted snapshot and to a lessor extent 
normal subvolume creation.

So while we might /like/ to make subvolumes more like directories and 
require no special privs, until the snapshot scaling and thus security 
issue is fixed or at least greatly reduced, as a purely practical 
security matter, we really need to restrict snapshot creation to 
privileged users.

OTOH, it's worth pointing out that snapshots aren't unique in this 
regard, reflinked files have exactly the /same/ scaling issues except 
their one-by-one while snapshots tend to be a whole bunch of reflinked 
files at once.

Which means the question must then be asked, if we choose the privileged 
route for snapshots/subvolumes due to the reflinking security issues, why 
aren't we making all reflinking ops privileged ops, or should we?  If we 
did, cp --reflink=always, among other things, would obviously fail as a 
normal user.

So I guess it's a rather harder question than I considered at first.  
Maybe we /should/ just treat subvolumes as directories in privileges 
terms as well.  In that case, however, we really need to have the same 
default for subvolume deletion as creation.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Qgroups are not applied when snapshotting a subvol?

[Austin S. Hemmelgarn]

On 2017-03-29 01:38, Duncan wrote:
> Austin S. Hemmelgarn posted on Tue, 28 Mar 2017 07:44:56 -0400 as
> excerpted:
>
>> On 2017-03-27 21:49, Qu Wenruo wrote:
>
>>> The problem is, how should we treat subvolume.
>>>
>>> Btrfs subvolume sits in the middle of directory and (logical) volume
>>> used in traditional stacked solution.
>>>
>>> While we allow normal user to create/delete/modify dir as long as they
>>> follow access control, we require privilege to create/delete/modify
>>> volumes.
>
>> No, we require privilege to do certain modifications or delete
>> subvolumes.
>
>>  Regular users can create subvolumes with no privileges whatsoever, and
>> most basic directory operations (rename, chown, chmod, etc) work just
>> fine within normal UNIX DAC permissions.  Unless you're running some
>> specially patched kernel or some LSM (SELinux possibly) that somehow
>> restricts access to the ioctl, you can always create subvolumes.
>
> (I believe) You misread what QW was trying to say. Note the word "volume"
> as opposed to subvolume.  Like most regulars here, Qu's too deep into
> btrfs to make that sort of mistake, so he wasn't talking about subvolumes.
Ah, you're right, I need to quit reading so fast all the time :)
>
> Rather, he's placing btrfs subvolumes in the middle between directories
> and (generic/logical) volumes as normally used, and saying that for
> (generic) directories normal users can create/modify/delete without
> special privilege (beyond write in the parent), for (generic) volumes
> special privileges are necessary to create/modify/delete, and btrfs
> subvolumes fall in between, so there's a real decision to be made in
> terms of privileges required, do they follow directories and require
> nothing special or volumes and require special privs?
>
>
> Meanwhile, my own position as I've argued is that regardless of the
> theoretical debate, as long as we have the very real practical issue of
> hard scaling issues for subvolumes/snapshots, there's an equally real
> security issue in allowing unrestricted snapshot and to a lessor extent
> normal subvolume creation.
There is a bigger issue though in that subvolumes are a resource a 
regular user can create but not destroy and there is no way to limit the 
number a user can create.  This is never a good situation to be in, but 
it appears to have arisen because there are no proper privilege checks 
on subvolume deletion, just the 'are you EUID 0?' thing.  The ioctl 
should require some proof that you could delete the subvolume if it were 
a directory, then we could get rid of the whole 'user_subvol_rm_allowed' 
thing and move on a bit more sensibly.
>
> So while we might /like/ to make subvolumes more like directories and
> require no special privs, until the snapshot scaling and thus security
> issue is fixed or at least greatly reduced, as a purely practical
> security matter, we really need to restrict snapshot creation to
> privileged users.
>
> OTOH, it's worth pointing out that snapshots aren't unique in this
> regard, reflinked files have exactly the /same/ scaling issues except
> their one-by-one while snapshots tend to be a whole bunch of reflinked
> files at once.
Snapshots and reflinks have scaling issues on every filesystem I've seen 
that supports them internally that isn't inherently log structured.  The 
scaling issue is far worse on BTRFS than most others, but it's by no 
means unique to BTRFS.
>
> Which means the question must then be asked, if we choose the privileged
> route for snapshots/subvolumes due to the reflinking security issues, why
> aren't we making all reflinking ops privileged ops, or should we?  If we
> did, cp --reflink=always, among other things, would obviously fail as a
> normal user.
Normal users should be able to use reflinks.  That is the expectation 
set forth by BTRFS already, as well as other filesystems which support 
reflinks.
>
> So I guess it's a rather harder question than I considered at first.
> Maybe we /should/ just treat subvolumes as directories in privileges
> terms as well.  In that case, however, we really need to have the same
> default for subvolume deletion as creation.
>