(Un)known issue with XFS group quotas on LVM?

(Un)known issue with XFS group quotas on LVM?

[John Desantis]

Hello list!

A big "thanks!" to all XFS developers for a great product.

Now, to the point!

We're using an XFS file system created on an LVM with a standard
kernel (2.6.32-431.3.1).  We're using quotas without an issue, with
the exception of the group quota.  It seems that group quotas are
ignored at times, at least from our configuration (we use remote user
and group look-ups and authentication).  I've only been able to get
root running sudo -g dd.. to create a group file which reflects
changes in the group quota limit.

We've mounted the file system in question with both user and group
quotas enabled;  I didn't see anywhere in the man page that said this
isn't possible, although I did see where group and project quotas
cannot be used simultaneously on the same file system.

Has anyone experienced this behavior before?  I feel that I did some
pretty good searching in the archives, Google, and the BugZilla page,
but there were no "real" results which seemed to describe the same
issue we're facing.  Strace didn't provide any clues either.  I don't
think it's a sync issue because I've allowed a few days to pass in
between "test" runs.

I've tested the quota by:

1.)  Using sudo -u user -g group dd.... on both the exported file
system and its mount path on the server itself.
2.)  Using sudo -u user -g group dd.. on the client and the file
system's mount path.
3.)  Using sudo -g group dd... on both the exported file system and
its mount path on the server itself;  this method produces expected
results.
4.)  Using sudo -g group dd.. on the client and the file system's
mount path;  this method also produces expected results.

Items #1 and #2 produce the error.

I just wanted to ping the list before creating a local testing group
to see if it's an issue with remote group look-ups.

Thank you,
John DeSantis

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: (Un)known issue with XFS group quotas on LVM?

[Dave Chinner]

On Wed, Jun 11, 2014 at 11:43:09AM -0400, John Desantis wrote:
> Hello list!
> 
> A big "thanks!" to all XFS developers for a great product.
> 
> Now, to the point!
> 
> We're using an XFS file system created on an LVM with a standard
> kernel (2.6.32-431.3.1). 

RHEL kernel? If so, you probably want to report it to your support
contact to see if it's a known issue with a fix/workaround.....

> We're using quotas without an issue, with
> the exception of the group quota.  It seems that group quotas are
> ignored at times, at least from our configuration (we use remote user
> and group look-ups and authentication).  I've only been able to get
> root running sudo -g dd.. to create a group file which reflects
> changes in the group quota limit.

That doesn't sound like a filesystem problem. Sounds more like the
remote user/group configuration is getting squashed. Do the systems
have the same user/group configurations, or are the clients sending
group IDs the server doesn't recognise?

As it is, describing the problem in text doesn't tell me anything
about the problem. You need to cut-n-paste the CLI and the output
showing the incorrect behaviour so we can attempt to reproduce the
same behaviour you are seeing.

> We've mounted the file system in question with both user and group
> quotas enabled;  I didn't see anywhere in the man page that said this
> isn't possible, although I did see where group and project quotas
> cannot be used simultaneously on the same file system.
> 
> Has anyone experienced this behavior before?  I feel that I did some
> pretty good searching in the archives, Google, and the BugZilla page,
> but there were no "real" results which seemed to describe the same
> issue we're facing.  Strace didn't provide any clues either.  I don't
> think it's a sync issue because I've allowed a few days to pass in
> between "test" runs.
> 
> I've tested the quota by:
> 
> 1.)  Using sudo -u user -g group dd.... on both the exported file
> system and its mount path on the server itself.
> 2.)  Using sudo -u user -g group dd.. on the client and the file
> system's mount path.
> 3.)  Using sudo -g group dd... on both the exported file system and
> its mount path on the server itself;  this method produces expected
> results.
> 4.)  Using sudo -g group dd.. on the client and the file system's
> mount path;  this method also produces expected results.
> 
> Items #1 and #2 produce the error.

I can't see how that is related to the filesystem - the filesystem
will either account for group quota usage or it won't. It doesn't
care what primary gid the process uses, it will simply account usage
to that...

> I just wanted to ping the list before creating a local testing group
> to see if it's an issue with remote group look-ups.

Quite possibly that's a problem when generating quota reports, but
again that's a userspace problem, not a filesystem quota accounting
issue...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: (Un)known issue with XFS group quotas on LVM?

[John Desantis]

Dave,

Thanks for your reply.  Let me address your questions and replies.

> That doesn't sound like a filesystem problem. Sounds more like the
> remote user/group configuration is getting squashed. Do the systems
> have the same user/group configurations, or are the clients sending
> group IDs the server doesn't recognise?

Stracing shows that the group and user ID's are not being squashed and
they are resolving each time to their respective UID/GID's.

> As it is, describing the problem in text doesn't tell me anything
> about the problem. You need to cut-n-paste the CLI and the output
> showing the incorrect behaviour so we can attempt to reproduce the
> same behaviour you are seeing.

Busted.  I figured before sending the usual gamut of console "action",
I'd "save" time by simply presenting the symptoms.

> I can't see how that is related to the filesystem - the filesystem
> will either account for group quota usage or it won't. It doesn't
> care what primary gid the process uses, it will simply account usage
> to that...

You're correct (but you already knew this!).  I can tell you that
running the quota report does show the group block count changing.

It looks like this "issue" is  my misunderstanding of how XFS is
checking the quota.  Perhaps you could shed some light?

When you're using active user and group quotas on the same file system
and the user quota is lower than the group quota, does XFS first check
against the user quota and then the group quota?  Is the order of
operations user, group, and then project or does it depend on the
mounting order of quotas?

In the case I presented yesterday, the user in question had a quota of
25 GB, but the group had an unlimited quota.  Each time that I used
sudo -u user -g group or logged into the user's account, the quota was
always exceeded.  Now that I have removed the user's quota, the group
quota is now being respected properly.

Again, thank you!

John DeSantis


2014-06-11 19:07 GMT-04:00 Dave Chinner <david@fromorbit.com>:
> On Wed, Jun 11, 2014 at 11:43:09AM -0400, John Desantis wrote:
>> Hello list!
>>
>> A big "thanks!" to all XFS developers for a great product.
>>
>> Now, to the point!
>>
>> We're using an XFS file system created on an LVM with a standard
>> kernel (2.6.32-431.3.1).
>
> RHEL kernel? If so, you probably want to report it to your support
> contact to see if it's a known issue with a fix/workaround.....
>
>> We're using quotas without an issue, with
>> the exception of the group quota.  It seems that group quotas are
>> ignored at times, at least from our configuration (we use remote user
>> and group look-ups and authentication).  I've only been able to get
>> root running sudo -g dd.. to create a group file which reflects
>> changes in the group quota limit.
>
> That doesn't sound like a filesystem problem. Sounds more like the
> remote user/group configuration is getting squashed. Do the systems
> have the same user/group configurations, or are the clients sending
> group IDs the server doesn't recognise?
>
> As it is, describing the problem in text doesn't tell me anything
> about the problem. You need to cut-n-paste the CLI and the output
> showing the incorrect behaviour so we can attempt to reproduce the
> same behaviour you are seeing.
>
>> We've mounted the file system in question with both user and group
>> quotas enabled;  I didn't see anywhere in the man page that said this
>> isn't possible, although I did see where group and project quotas
>> cannot be used simultaneously on the same file system.
>>
>> Has anyone experienced this behavior before?  I feel that I did some
>> pretty good searching in the archives, Google, and the BugZilla page,
>> but there were no "real" results which seemed to describe the same
>> issue we're facing.  Strace didn't provide any clues either.  I don't
>> think it's a sync issue because I've allowed a few days to pass in
>> between "test" runs.
>>
>> I've tested the quota by:
>>
>> 1.)  Using sudo -u user -g group dd.... on both the exported file
>> system and its mount path on the server itself.
>> 2.)  Using sudo -u user -g group dd.. on the client and the file
>> system's mount path.
>> 3.)  Using sudo -g group dd... on both the exported file system and
>> its mount path on the server itself;  this method produces expected
>> results.
>> 4.)  Using sudo -g group dd.. on the client and the file system's
>> mount path;  this method also produces expected results.
>>
>> Items #1 and #2 produce the error.
>
> I can't see how that is related to the filesystem - the filesystem
> will either account for group quota usage or it won't. It doesn't
> care what primary gid the process uses, it will simply account usage
> to that...
>
>> I just wanted to ping the list before creating a local testing group
>> to see if it's an issue with remote group look-ups.
>
> Quite possibly that's a problem when generating quota reports, but
> again that's a userspace problem, not a filesystem quota accounting
> issue...
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: (Un)known issue with XFS group quotas on LVM?

[Dave Chinner]

On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote:
> Dave,
> 
> Thanks for your reply.  Let me address your questions and replies.
> 
> > That doesn't sound like a filesystem problem. Sounds more like the
> > remote user/group configuration is getting squashed. Do the systems
> > have the same user/group configurations, or are the clients sending
> > group IDs the server doesn't recognise?
> 
> Stracing shows that the group and user ID's are not being squashed and
> they are resolving each time to their respective UID/GID's.
> 
> > As it is, describing the problem in text doesn't tell me anything
> > about the problem. You need to cut-n-paste the CLI and the output
> > showing the incorrect behaviour so we can attempt to reproduce the
> > same behaviour you are seeing.
> 
> Busted.  I figured before sending the usual gamut of console "action",
> I'd "save" time by simply presenting the symptoms.
> 
> > I can't see how that is related to the filesystem - the filesystem
> > will either account for group quota usage or it won't. It doesn't
> > care what primary gid the process uses, it will simply account usage
> > to that...
> 
> You're correct (but you already knew this!).  I can tell you that
> running the quota report does show the group block count changing.
> 
> It looks like this "issue" is  my misunderstanding of how XFS is
> checking the quota.  Perhaps you could shed some light?
> 
> When you're using active user and group quotas on the same file system
> and the user quota is lower than the group quota, does XFS first check
> against the user quota and then the group quota?  Is the order of
> operations user, group, and then project or does it depend on the
> mounting order of quotas?

Order doesn't matter because the filesystem is supposed to check
against all enforced quotas, not just the first one.

> In the case I presented yesterday, the user in question had a quota of
> 25 GB, but the group had an unlimited quota.  Each time that I used
> sudo -u user -g group or logged into the user's account, the quota was
> always exceeded.  Now that I have removed the user's quota, the group
> quota is now being respected properly.

Again, you need to post the commands and output that demonstrate the
problem you are describing. I'm having trouble understanding what
your problem is because I can't see the raw output that you are
seeing...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: (Un)known issue with XFS group quotas on LVM?

[John Desantis]

Dave,

> Order doesn't matter because the filesystem is supposed to check
> against all enforced quotas, not just the first one.

Thanks for confirming.

> Again, you need to post the commands and output that demonstrate the
> problem you are describing. I'm having trouble understanding what
> your problem is because I can't see the raw output that you are
> seeing...

Ok, here are the default user and group quotas of my test account:

xfs_quota> quota -uh desantis.admin
Disk quotas for User desantis.admin (663800172)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
               256K      0      0  00 [------] /export/shares

xfs_quota> quota -gh rcadm
Disk quotas for Group rcadm (1034)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
              43.5G      0      0  00 [------] /export/shares

Now, we'll limit them and then re-display the quotas:

xfs_quota> limit -g bsoft=50G bhard=50G rcadm
xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin

xfs_quota> quota -uh desantis.admin
Disk quotas for User desantis.admin (663800172)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
               256K    10G    10G  00 [------] /export/shares

xfs_quota> quota -gh rcadm
Disk quotas for Group rcadm (1034)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
              43.5G    50G    50G  00 [------] /export/shares

Here come the tests:

# sudo -u desantis.admin -g rcadm dd if=/dev/zero
of=/export/shares/rcadm/testfile_1 bs=1024M count=11
dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded
7+0 records in
6+0 records out
6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s

xfs_quota> quota -uh desantis.admin
Disk quotas for User desantis.admin (663800172)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
               6.5G    10G    10G  00 [------] /export/shares

xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin

xfs_quota> quota -uh desantis.admin
Disk quotas for User desantis.admin (663800172)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
               6.5G      0      0  00 [------] /export/shares

# sudo -u desantis.admin -g rcadm dd if=/dev/zero
of=/export/shares/rcadm/testfile_1 bs=1024M count=5
5+0 records in
5+0 records out
5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s

# sudo -u desantis.admin -g rcadm dd if=/dev/zero
of=/export/shares/rcadm/testfile_2 bs=1024M count=5
dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded
1+0 records in
0+0 records out
1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s

xfs_quota> quota -gh rcadm
Disk quotas for Group rcadm (1034)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
              50.0G    50G    50G  00 [------] /export/shares


As you can see with the output, my user quota of 10 GB was enforced
over the higher group quota of 50 GB.  My original thought process was
that since the file being created was specified for the group with the
larger quota, that XFS would enforce that limit over the user quota.
After reading over several pieces of quota documentation (RTFM!) from
different O.S. distributions, it looks like you can either have a user
or group quota getting enforced, but not both in the sense I was
originally thinking.

That being said, I really do appreciate the responses and extra set of
eyes and apologize if I've wasted too much time.

John DeSantis

2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>:
> On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote:
>> Dave,
>>
>> Thanks for your reply.  Let me address your questions and replies.
>>
>> > That doesn't sound like a filesystem problem. Sounds more like the
>> > remote user/group configuration is getting squashed. Do the systems
>> > have the same user/group configurations, or are the clients sending
>> > group IDs the server doesn't recognise?
>>
>> Stracing shows that the group and user ID's are not being squashed and
>> they are resolving each time to their respective UID/GID's.
>>
>> > As it is, describing the problem in text doesn't tell me anything
>> > about the problem. You need to cut-n-paste the CLI and the output
>> > showing the incorrect behaviour so we can attempt to reproduce the
>> > same behaviour you are seeing.
>>
>> Busted.  I figured before sending the usual gamut of console "action",
>> I'd "save" time by simply presenting the symptoms.
>>
>> > I can't see how that is related to the filesystem - the filesystem
>> > will either account for group quota usage or it won't. It doesn't
>> > care what primary gid the process uses, it will simply account usage
>> > to that...
>>
>> You're correct (but you already knew this!).  I can tell you that
>> running the quota report does show the group block count changing.
>>
>> It looks like this "issue" is  my misunderstanding of how XFS is
>> checking the quota.  Perhaps you could shed some light?
>>
>> When you're using active user and group quotas on the same file system
>> and the user quota is lower than the group quota, does XFS first check
>> against the user quota and then the group quota?  Is the order of
>> operations user, group, and then project or does it depend on the
>> mounting order of quotas?
>
> Order doesn't matter because the filesystem is supposed to check
> against all enforced quotas, not just the first one.
>
>> In the case I presented yesterday, the user in question had a quota of
>> 25 GB, but the group had an unlimited quota.  Each time that I used
>> sudo -u user -g group or logged into the user's account, the quota was
>> always exceeded.  Now that I have removed the user's quota, the group
>> quota is now being respected properly.
>
> Again, you need to post the commands and output that demonstrate the
> problem you are describing. I'm having trouble understanding what
> your problem is because I can't see the raw output that you are
> seeing...
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: (Un)known issue with XFS group quotas on LVM?

[Brian Foster]

On Fri, Jun 13, 2014 at 10:50:19AM -0400, John Desantis wrote:
> Dave,
> 
> > Order doesn't matter because the filesystem is supposed to check
> > against all enforced quotas, not just the first one.
> 
> Thanks for confirming.
> 
> > Again, you need to post the commands and output that demonstrate the
> > problem you are describing. I'm having trouble understanding what
> > your problem is because I can't see the raw output that you are
> > seeing...
> 
> Ok, here are the default user and group quotas of my test account:
> 
> xfs_quota> quota -uh desantis.admin
> Disk quotas for User desantis.admin (663800172)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>                256K      0      0  00 [------] /export/shares
> 
> xfs_quota> quota -gh rcadm
> Disk quotas for Group rcadm (1034)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>               43.5G      0      0  00 [------] /export/shares
> 
> Now, we'll limit them and then re-display the quotas:
> 
> xfs_quota> limit -g bsoft=50G bhard=50G rcadm
> xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin
> 
> xfs_quota> quota -uh desantis.admin
> Disk quotas for User desantis.admin (663800172)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>                256K    10G    10G  00 [------] /export/shares
> 
> xfs_quota> quota -gh rcadm
> Disk quotas for Group rcadm (1034)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>               43.5G    50G    50G  00 [------] /export/shares
> 

So you've got 10G quota with about as much available and a 50G quota
with ~6.5GB available.

> Here come the tests:
> 
> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
> of=/export/shares/rcadm/testfile_1 bs=1024M count=11
> dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded
> 7+0 records in
> 6+0 records out
> 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s
> 

It looks like you've hit the 50G quota.

> xfs_quota> quota -uh desantis.admin
> Disk quotas for User desantis.admin (663800172)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>                6.5G    10G    10G  00 [------] /export/shares
> 
> xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin
> 

Now the 10G quota is gone.

> xfs_quota> quota -uh desantis.admin
> Disk quotas for User desantis.admin (663800172)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>                6.5G      0      0  00 [------] /export/shares
> 
> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
> of=/export/shares/rcadm/testfile_1 bs=1024M count=5
> 5+0 records in
> 5+0 records out
> 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s
> 

You overwrite the previously written 6.5G file to 5.4G (dd will
truncate).

> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
> of=/export/shares/rcadm/testfile_2 bs=1024M count=5
> dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded
> 1+0 records in
> 0+0 records out
> 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s
> 

Then create a new file and run into the 50G limit again.

> xfs_quota> quota -gh rcadm
> Disk quotas for Group rcadm (1034)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>               50.0G    50G    50G  00 [------] /export/shares
> 
> 
> As you can see with the output, my user quota of 10 GB was enforced
> over the higher group quota of 50 GB.  My original thought process was
> that since the file being created was specified for the group with the
> larger quota, that XFS would enforce that limit over the user quota.
> After reading over several pieces of quota documentation (RTFM!) from
> different O.S. distributions, it looks like you can either have a user
> or group quota getting enforced, but not both in the sense I was
> originally thinking.
> 

This all seems like normal behavior to me. What gives you the impression
that you are hitting the 10G quota? The fact that the quota is reset and
writing continues? Note that dd will truncate a file before it starts
writing unless notrunc is specified.

Brian

> That being said, I really do appreciate the responses and extra set of
> eyes and apologize if I've wasted too much time.
> 
> John DeSantis
> 
> 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>:
> > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote:
> >> Dave,
> >>
> >> Thanks for your reply.  Let me address your questions and replies.
> >>
> >> > That doesn't sound like a filesystem problem. Sounds more like the
> >> > remote user/group configuration is getting squashed. Do the systems
> >> > have the same user/group configurations, or are the clients sending
> >> > group IDs the server doesn't recognise?
> >>
> >> Stracing shows that the group and user ID's are not being squashed and
> >> they are resolving each time to their respective UID/GID's.
> >>
> >> > As it is, describing the problem in text doesn't tell me anything
> >> > about the problem. You need to cut-n-paste the CLI and the output
> >> > showing the incorrect behaviour so we can attempt to reproduce the
> >> > same behaviour you are seeing.
> >>
> >> Busted.  I figured before sending the usual gamut of console "action",
> >> I'd "save" time by simply presenting the symptoms.
> >>
> >> > I can't see how that is related to the filesystem - the filesystem
> >> > will either account for group quota usage or it won't. It doesn't
> >> > care what primary gid the process uses, it will simply account usage
> >> > to that...
> >>
> >> You're correct (but you already knew this!).  I can tell you that
> >> running the quota report does show the group block count changing.
> >>
> >> It looks like this "issue" is  my misunderstanding of how XFS is
> >> checking the quota.  Perhaps you could shed some light?
> >>
> >> When you're using active user and group quotas on the same file system
> >> and the user quota is lower than the group quota, does XFS first check
> >> against the user quota and then the group quota?  Is the order of
> >> operations user, group, and then project or does it depend on the
> >> mounting order of quotas?
> >
> > Order doesn't matter because the filesystem is supposed to check
> > against all enforced quotas, not just the first one.
> >
> >> In the case I presented yesterday, the user in question had a quota of
> >> 25 GB, but the group had an unlimited quota.  Each time that I used
> >> sudo -u user -g group or logged into the user's account, the quota was
> >> always exceeded.  Now that I have removed the user's quota, the group
> >> quota is now being respected properly.
> >
> > Again, you need to post the commands and output that demonstrate the
> > problem you are describing. I'm having trouble understanding what
> > your problem is because I can't see the raw output that you are
> > seeing...
> >
> > Cheers,
> >
> > Dave.
> > --
> > Dave Chinner
> > david@fromorbit.com
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: (Un)known issue with XFS group quotas on LVM?

[John Desantis]

Brian,

> This all seems like normal behavior to me. What gives you the impression
> that you are hitting the 10G quota? The fact that the quota is reset and
> writing continues? Note that dd will truncate a file before it starts
> writing unless notrunc is specified.

Yes, I am aware of the truncating, but not that particular option - thanks!

I had experienced the "issue" when a user quota was set while the
group quota was unlimited.  See below.

xfs_quota> limit -u bsoft=5G bhard=5G desantis.admin

xfs_quota> quota -uh desantis.admin
Disk quotas for User desantis.admin (663800172)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
               256K     5G     5G  00 [------] /export/shares

xfs_quota> limit -g bsoft=unlimited bhard=unlimited rcadm

xfs_quota> quota -gh rcadm
Disk quotas for Group rcadm (1034)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
              43.5G    16E    16E  00 [------] /export/shares


# sudo -u desantis.admin -g rcadm dd if=/dev/zero
of=/export/shares/rcadm/testfile_morethan5G bs=1024M count=10
dd: writing `/export/shares/rcadm/testfile_morethan5G': Disk quota exceeded
5+0 records in
4+0 records out
5368442880 bytes (5.4 GB) copied, 10.5371 s, 509 MB/s

xfs_quota> quota -uh desantis.admin
Disk quotas for User desantis.admin (663800172)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
               5.0G     5G     5G  00 [------] /export/shares


xfs_quota> quota -gh rcadm
Disk quotas for Group rcadm (1034)
Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
/dev/mapper/vg_drbd-shares
              48.5G    16E    16E  00 [------] /export/shares

As you can see, although the group quota did augment 5 GB, the user
quota was enforced over the group quota.  This is what led to my
confusion.  I had thought that the group quota would be respected over
the user quota because I was specifying the group for the file being
created.

John DeSantis



2014-06-13 12:22 GMT-04:00 Brian Foster <bfoster@redhat.com>:
> On Fri, Jun 13, 2014 at 10:50:19AM -0400, John Desantis wrote:
>> Dave,
>>
>> > Order doesn't matter because the filesystem is supposed to check
>> > against all enforced quotas, not just the first one.
>>
>> Thanks for confirming.
>>
>> > Again, you need to post the commands and output that demonstrate the
>> > problem you are describing. I'm having trouble understanding what
>> > your problem is because I can't see the raw output that you are
>> > seeing...
>>
>> Ok, here are the default user and group quotas of my test account:
>>
>> xfs_quota> quota -uh desantis.admin
>> Disk quotas for User desantis.admin (663800172)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>                256K      0      0  00 [------] /export/shares
>>
>> xfs_quota> quota -gh rcadm
>> Disk quotas for Group rcadm (1034)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>               43.5G      0      0  00 [------] /export/shares
>>
>> Now, we'll limit them and then re-display the quotas:
>>
>> xfs_quota> limit -g bsoft=50G bhard=50G rcadm
>> xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin
>>
>> xfs_quota> quota -uh desantis.admin
>> Disk quotas for User desantis.admin (663800172)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>                256K    10G    10G  00 [------] /export/shares
>>
>> xfs_quota> quota -gh rcadm
>> Disk quotas for Group rcadm (1034)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>               43.5G    50G    50G  00 [------] /export/shares
>>
>
> So you've got 10G quota with about as much available and a 50G quota
> with ~6.5GB available.
>
>> Here come the tests:
>>
>> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
>> of=/export/shares/rcadm/testfile_1 bs=1024M count=11
>> dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded
>> 7+0 records in
>> 6+0 records out
>> 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s
>>
>
> It looks like you've hit the 50G quota.
>
>> xfs_quota> quota -uh desantis.admin
>> Disk quotas for User desantis.admin (663800172)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>                6.5G    10G    10G  00 [------] /export/shares
>>
>> xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin
>>
>
> Now the 10G quota is gone.
>
>> xfs_quota> quota -uh desantis.admin
>> Disk quotas for User desantis.admin (663800172)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>                6.5G      0      0  00 [------] /export/shares
>>
>> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
>> of=/export/shares/rcadm/testfile_1 bs=1024M count=5
>> 5+0 records in
>> 5+0 records out
>> 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s
>>
>
> You overwrite the previously written 6.5G file to 5.4G (dd will
> truncate).
>
>> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
>> of=/export/shares/rcadm/testfile_2 bs=1024M count=5
>> dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded
>> 1+0 records in
>> 0+0 records out
>> 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s
>>
>
> Then create a new file and run into the 50G limit again.
>
>> xfs_quota> quota -gh rcadm
>> Disk quotas for Group rcadm (1034)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>               50.0G    50G    50G  00 [------] /export/shares
>>
>>
>> As you can see with the output, my user quota of 10 GB was enforced
>> over the higher group quota of 50 GB.  My original thought process was
>> that since the file being created was specified for the group with the
>> larger quota, that XFS would enforce that limit over the user quota.
>> After reading over several pieces of quota documentation (RTFM!) from
>> different O.S. distributions, it looks like you can either have a user
>> or group quota getting enforced, but not both in the sense I was
>> originally thinking.
>>
>
> This all seems like normal behavior to me. What gives you the impression
> that you are hitting the 10G quota? The fact that the quota is reset and
> writing continues? Note that dd will truncate a file before it starts
> writing unless notrunc is specified.
>
> Brian
>
>> That being said, I really do appreciate the responses and extra set of
>> eyes and apologize if I've wasted too much time.
>>
>> John DeSantis
>>
>> 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>:
>> > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote:
>> >> Dave,
>> >>
>> >> Thanks for your reply.  Let me address your questions and replies.
>> >>
>> >> > That doesn't sound like a filesystem problem. Sounds more like the
>> >> > remote user/group configuration is getting squashed. Do the systems
>> >> > have the same user/group configurations, or are the clients sending
>> >> > group IDs the server doesn't recognise?
>> >>
>> >> Stracing shows that the group and user ID's are not being squashed and
>> >> they are resolving each time to their respective UID/GID's.
>> >>
>> >> > As it is, describing the problem in text doesn't tell me anything
>> >> > about the problem. You need to cut-n-paste the CLI and the output
>> >> > showing the incorrect behaviour so we can attempt to reproduce the
>> >> > same behaviour you are seeing.
>> >>
>> >> Busted.  I figured before sending the usual gamut of console "action",
>> >> I'd "save" time by simply presenting the symptoms.
>> >>
>> >> > I can't see how that is related to the filesystem - the filesystem
>> >> > will either account for group quota usage or it won't. It doesn't
>> >> > care what primary gid the process uses, it will simply account usage
>> >> > to that...
>> >>
>> >> You're correct (but you already knew this!).  I can tell you that
>> >> running the quota report does show the group block count changing.
>> >>
>> >> It looks like this "issue" is  my misunderstanding of how XFS is
>> >> checking the quota.  Perhaps you could shed some light?
>> >>
>> >> When you're using active user and group quotas on the same file system
>> >> and the user quota is lower than the group quota, does XFS first check
>> >> against the user quota and then the group quota?  Is the order of
>> >> operations user, group, and then project or does it depend on the
>> >> mounting order of quotas?
>> >
>> > Order doesn't matter because the filesystem is supposed to check
>> > against all enforced quotas, not just the first one.
>> >
>> >> In the case I presented yesterday, the user in question had a quota of
>> >> 25 GB, but the group had an unlimited quota.  Each time that I used
>> >> sudo -u user -g group or logged into the user's account, the quota was
>> >> always exceeded.  Now that I have removed the user's quota, the group
>> >> quota is now being respected properly.
>> >
>> > Again, you need to post the commands and output that demonstrate the
>> > problem you are describing. I'm having trouble understanding what
>> > your problem is because I can't see the raw output that you are
>> > seeing...
>> >
>> > Cheers,
>> >
>> > Dave.
>> > --
>> > Dave Chinner
>> > david@fromorbit.com
>>
>> _______________________________________________
>> xfs mailing list
>> xfs@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: (Un)known issue with XFS group quotas on LVM?

[Brian Foster]

On Fri, Jun 13, 2014 at 01:00:41PM -0400, John Desantis wrote:
> Brian,
> 
> > This all seems like normal behavior to me. What gives you the impression
> > that you are hitting the 10G quota? The fact that the quota is reset and
> > writing continues? Note that dd will truncate a file before it starts
> > writing unless notrunc is specified.
> 
> Yes, I am aware of the truncating, but not that particular option - thanks!
> 
> I had experienced the "issue" when a user quota was set while the
> group quota was unlimited.  See below.
> 
> xfs_quota> limit -u bsoft=5G bhard=5G desantis.admin
> 
> xfs_quota> quota -uh desantis.admin
> Disk quotas for User desantis.admin (663800172)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>                256K     5G     5G  00 [------] /export/shares
> 
> xfs_quota> limit -g bsoft=unlimited bhard=unlimited rcadm
> 
> xfs_quota> quota -gh rcadm
> Disk quotas for Group rcadm (1034)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>               43.5G    16E    16E  00 [------] /export/shares
> 
> 

Ok, so a 5G user quota and effectively unlimited group quota.

> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
> of=/export/shares/rcadm/testfile_morethan5G bs=1024M count=10
> dd: writing `/export/shares/rcadm/testfile_morethan5G': Disk quota exceeded
> 5+0 records in
> 4+0 records out
> 5368442880 bytes (5.4 GB) copied, 10.5371 s, 509 MB/s
> 

And you can write to the 5G quota.

> xfs_quota> quota -uh desantis.admin
> Disk quotas for User desantis.admin (663800172)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>                5.0G     5G     5G  00 [------] /export/shares
> 
> 
> xfs_quota> quota -gh rcadm
> Disk quotas for Group rcadm (1034)
> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> /dev/mapper/vg_drbd-shares
>               48.5G    16E    16E  00 [------] /export/shares
> 
> As you can see, although the group quota did augment 5 GB, the user
> quota was enforced over the group quota.  This is what led to my
> confusion.  I had thought that the group quota would be respected over
> the user quota because I was specifying the group for the file being
> created.
> 

Those are tracking how many blocks are accounted against each quota. As
the output shows, the 5G from this file is being accounted against both.
This means that for an allocation on this file to succeed, both quotas
must have enough room for the allocation.

It sounds like the misunderstanding is viewing a quota as an "allowance"
rather than a "limit?" In the scenario above, the group quota
effectively becomes a no-op. You'll never hit that limit, but that has
no effect on the enforcement of other quotas against which a particular
file might also be accounted.

For example, if you were to set a new project quota of 1GB on the root
directory of this mount, you won't have the ability to allocate any more
than 1GB regardless of the user/group of the underlying files.

Brian

> John DeSantis
> 
> 
> 
> 2014-06-13 12:22 GMT-04:00 Brian Foster <bfoster@redhat.com>:
> > On Fri, Jun 13, 2014 at 10:50:19AM -0400, John Desantis wrote:
> >> Dave,
> >>
> >> > Order doesn't matter because the filesystem is supposed to check
> >> > against all enforced quotas, not just the first one.
> >>
> >> Thanks for confirming.
> >>
> >> > Again, you need to post the commands and output that demonstrate the
> >> > problem you are describing. I'm having trouble understanding what
> >> > your problem is because I can't see the raw output that you are
> >> > seeing...
> >>
> >> Ok, here are the default user and group quotas of my test account:
> >>
> >> xfs_quota> quota -uh desantis.admin
> >> Disk quotas for User desantis.admin (663800172)
> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> >> /dev/mapper/vg_drbd-shares
> >>                256K      0      0  00 [------] /export/shares
> >>
> >> xfs_quota> quota -gh rcadm
> >> Disk quotas for Group rcadm (1034)
> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> >> /dev/mapper/vg_drbd-shares
> >>               43.5G      0      0  00 [------] /export/shares
> >>
> >> Now, we'll limit them and then re-display the quotas:
> >>
> >> xfs_quota> limit -g bsoft=50G bhard=50G rcadm
> >> xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin
> >>
> >> xfs_quota> quota -uh desantis.admin
> >> Disk quotas for User desantis.admin (663800172)
> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> >> /dev/mapper/vg_drbd-shares
> >>                256K    10G    10G  00 [------] /export/shares
> >>
> >> xfs_quota> quota -gh rcadm
> >> Disk quotas for Group rcadm (1034)
> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> >> /dev/mapper/vg_drbd-shares
> >>               43.5G    50G    50G  00 [------] /export/shares
> >>
> >
> > So you've got 10G quota with about as much available and a 50G quota
> > with ~6.5GB available.
> >
> >> Here come the tests:
> >>
> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
> >> of=/export/shares/rcadm/testfile_1 bs=1024M count=11
> >> dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded
> >> 7+0 records in
> >> 6+0 records out
> >> 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s
> >>
> >
> > It looks like you've hit the 50G quota.
> >
> >> xfs_quota> quota -uh desantis.admin
> >> Disk quotas for User desantis.admin (663800172)
> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> >> /dev/mapper/vg_drbd-shares
> >>                6.5G    10G    10G  00 [------] /export/shares
> >>
> >> xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin
> >>
> >
> > Now the 10G quota is gone.
> >
> >> xfs_quota> quota -uh desantis.admin
> >> Disk quotas for User desantis.admin (663800172)
> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> >> /dev/mapper/vg_drbd-shares
> >>                6.5G      0      0  00 [------] /export/shares
> >>
> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
> >> of=/export/shares/rcadm/testfile_1 bs=1024M count=5
> >> 5+0 records in
> >> 5+0 records out
> >> 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s
> >>
> >
> > You overwrite the previously written 6.5G file to 5.4G (dd will
> > truncate).
> >
> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
> >> of=/export/shares/rcadm/testfile_2 bs=1024M count=5
> >> dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded
> >> 1+0 records in
> >> 0+0 records out
> >> 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s
> >>
> >
> > Then create a new file and run into the 50G limit again.
> >
> >> xfs_quota> quota -gh rcadm
> >> Disk quotas for Group rcadm (1034)
> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
> >> /dev/mapper/vg_drbd-shares
> >>               50.0G    50G    50G  00 [------] /export/shares
> >>
> >>
> >> As you can see with the output, my user quota of 10 GB was enforced
> >> over the higher group quota of 50 GB.  My original thought process was
> >> that since the file being created was specified for the group with the
> >> larger quota, that XFS would enforce that limit over the user quota.
> >> After reading over several pieces of quota documentation (RTFM!) from
> >> different O.S. distributions, it looks like you can either have a user
> >> or group quota getting enforced, but not both in the sense I was
> >> originally thinking.
> >>
> >
> > This all seems like normal behavior to me. What gives you the impression
> > that you are hitting the 10G quota? The fact that the quota is reset and
> > writing continues? Note that dd will truncate a file before it starts
> > writing unless notrunc is specified.
> >
> > Brian
> >
> >> That being said, I really do appreciate the responses and extra set of
> >> eyes and apologize if I've wasted too much time.
> >>
> >> John DeSantis
> >>
> >> 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>:
> >> > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote:
> >> >> Dave,
> >> >>
> >> >> Thanks for your reply.  Let me address your questions and replies.
> >> >>
> >> >> > That doesn't sound like a filesystem problem. Sounds more like the
> >> >> > remote user/group configuration is getting squashed. Do the systems
> >> >> > have the same user/group configurations, or are the clients sending
> >> >> > group IDs the server doesn't recognise?
> >> >>
> >> >> Stracing shows that the group and user ID's are not being squashed and
> >> >> they are resolving each time to their respective UID/GID's.
> >> >>
> >> >> > As it is, describing the problem in text doesn't tell me anything
> >> >> > about the problem. You need to cut-n-paste the CLI and the output
> >> >> > showing the incorrect behaviour so we can attempt to reproduce the
> >> >> > same behaviour you are seeing.
> >> >>
> >> >> Busted.  I figured before sending the usual gamut of console "action",
> >> >> I'd "save" time by simply presenting the symptoms.
> >> >>
> >> >> > I can't see how that is related to the filesystem - the filesystem
> >> >> > will either account for group quota usage or it won't. It doesn't
> >> >> > care what primary gid the process uses, it will simply account usage
> >> >> > to that...
> >> >>
> >> >> You're correct (but you already knew this!).  I can tell you that
> >> >> running the quota report does show the group block count changing.
> >> >>
> >> >> It looks like this "issue" is  my misunderstanding of how XFS is
> >> >> checking the quota.  Perhaps you could shed some light?
> >> >>
> >> >> When you're using active user and group quotas on the same file system
> >> >> and the user quota is lower than the group quota, does XFS first check
> >> >> against the user quota and then the group quota?  Is the order of
> >> >> operations user, group, and then project or does it depend on the
> >> >> mounting order of quotas?
> >> >
> >> > Order doesn't matter because the filesystem is supposed to check
> >> > against all enforced quotas, not just the first one.
> >> >
> >> >> In the case I presented yesterday, the user in question had a quota of
> >> >> 25 GB, but the group had an unlimited quota.  Each time that I used
> >> >> sudo -u user -g group or logged into the user's account, the quota was
> >> >> always exceeded.  Now that I have removed the user's quota, the group
> >> >> quota is now being respected properly.
> >> >
> >> > Again, you need to post the commands and output that demonstrate the
> >> > problem you are describing. I'm having trouble understanding what
> >> > your problem is because I can't see the raw output that you are
> >> > seeing...
> >> >
> >> > Cheers,
> >> >
> >> > Dave.
> >> > --
> >> > Dave Chinner
> >> > david@fromorbit.com
> >>
> >> _______________________________________________
> >> xfs mailing list
> >> xfs@oss.sgi.com
> >> http://oss.sgi.com/mailman/listinfo/xfs
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: (Un)known issue with XFS group quotas on LVM?

[John Desantis]

Brian,

> Those are tracking how many blocks are accounted against each quota. As
> the output shows, the 5G from this file is being accounted against both.
> This means that for an allocation on this file to succeed, both quotas
> must have enough room for the allocation.

This is what I had did earlier to satisfy users while the confusion
lingered.  I basically augmented whichever the smaller quota was.

> It sounds like the misunderstanding is viewing a quota as an "allowance"
> rather than a "limit?" In the scenario above, the group quota
> effectively becomes a no-op. You'll never hit that limit, but that has
> no effect on the enforcement of other quotas against which a particular
> file might also be accounted.
>
> For example, if you were to set a new project quota of 1GB on the root
> directory of this mount, you won't have the ability to allocate any more
> than 1GB regardless of the user/group of the underlying files.

I do believe this is from where my confusion was stemming.  A big
thanks to you and Dave for clearing this up!

John DeSantis


2014-06-13 13:59 GMT-04:00 Brian Foster <bfoster@redhat.com>:
> On Fri, Jun 13, 2014 at 01:00:41PM -0400, John Desantis wrote:
>> Brian,
>>
>> > This all seems like normal behavior to me. What gives you the impression
>> > that you are hitting the 10G quota? The fact that the quota is reset and
>> > writing continues? Note that dd will truncate a file before it starts
>> > writing unless notrunc is specified.
>>
>> Yes, I am aware of the truncating, but not that particular option - thanks!
>>
>> I had experienced the "issue" when a user quota was set while the
>> group quota was unlimited.  See below.
>>
>> xfs_quota> limit -u bsoft=5G bhard=5G desantis.admin
>>
>> xfs_quota> quota -uh desantis.admin
>> Disk quotas for User desantis.admin (663800172)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>                256K     5G     5G  00 [------] /export/shares
>>
>> xfs_quota> limit -g bsoft=unlimited bhard=unlimited rcadm
>>
>> xfs_quota> quota -gh rcadm
>> Disk quotas for Group rcadm (1034)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>               43.5G    16E    16E  00 [------] /export/shares
>>
>>
>
> Ok, so a 5G user quota and effectively unlimited group quota.
>
>> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
>> of=/export/shares/rcadm/testfile_morethan5G bs=1024M count=10
>> dd: writing `/export/shares/rcadm/testfile_morethan5G': Disk quota exceeded
>> 5+0 records in
>> 4+0 records out
>> 5368442880 bytes (5.4 GB) copied, 10.5371 s, 509 MB/s
>>
>
> And you can write to the 5G quota.
>
>> xfs_quota> quota -uh desantis.admin
>> Disk quotas for User desantis.admin (663800172)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>                5.0G     5G     5G  00 [------] /export/shares
>>
>>
>> xfs_quota> quota -gh rcadm
>> Disk quotas for Group rcadm (1034)
>> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> /dev/mapper/vg_drbd-shares
>>               48.5G    16E    16E  00 [------] /export/shares
>>
>> As you can see, although the group quota did augment 5 GB, the user
>> quota was enforced over the group quota.  This is what led to my
>> confusion.  I had thought that the group quota would be respected over
>> the user quota because I was specifying the group for the file being
>> created.
>>
>
> Those are tracking how many blocks are accounted against each quota. As
> the output shows, the 5G from this file is being accounted against both.
> This means that for an allocation on this file to succeed, both quotas
> must have enough room for the allocation.
>
> It sounds like the misunderstanding is viewing a quota as an "allowance"
> rather than a "limit?" In the scenario above, the group quota
> effectively becomes a no-op. You'll never hit that limit, but that has
> no effect on the enforcement of other quotas against which a particular
> file might also be accounted.
>
> For example, if you were to set a new project quota of 1GB on the root
> directory of this mount, you won't have the ability to allocate any more
> than 1GB regardless of the user/group of the underlying files.
>
> Brian
>
>> John DeSantis
>>
>>
>>
>> 2014-06-13 12:22 GMT-04:00 Brian Foster <bfoster@redhat.com>:
>> > On Fri, Jun 13, 2014 at 10:50:19AM -0400, John Desantis wrote:
>> >> Dave,
>> >>
>> >> > Order doesn't matter because the filesystem is supposed to check
>> >> > against all enforced quotas, not just the first one.
>> >>
>> >> Thanks for confirming.
>> >>
>> >> > Again, you need to post the commands and output that demonstrate the
>> >> > problem you are describing. I'm having trouble understanding what
>> >> > your problem is because I can't see the raw output that you are
>> >> > seeing...
>> >>
>> >> Ok, here are the default user and group quotas of my test account:
>> >>
>> >> xfs_quota> quota -uh desantis.admin
>> >> Disk quotas for User desantis.admin (663800172)
>> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> >> /dev/mapper/vg_drbd-shares
>> >>                256K      0      0  00 [------] /export/shares
>> >>
>> >> xfs_quota> quota -gh rcadm
>> >> Disk quotas for Group rcadm (1034)
>> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> >> /dev/mapper/vg_drbd-shares
>> >>               43.5G      0      0  00 [------] /export/shares
>> >>
>> >> Now, we'll limit them and then re-display the quotas:
>> >>
>> >> xfs_quota> limit -g bsoft=50G bhard=50G rcadm
>> >> xfs_quota> limit -u bsoft=10G bhard=10G desantis.admin
>> >>
>> >> xfs_quota> quota -uh desantis.admin
>> >> Disk quotas for User desantis.admin (663800172)
>> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> >> /dev/mapper/vg_drbd-shares
>> >>                256K    10G    10G  00 [------] /export/shares
>> >>
>> >> xfs_quota> quota -gh rcadm
>> >> Disk quotas for Group rcadm (1034)
>> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> >> /dev/mapper/vg_drbd-shares
>> >>               43.5G    50G    50G  00 [------] /export/shares
>> >>
>> >
>> > So you've got 10G quota with about as much available and a 50G quota
>> > with ~6.5GB available.
>> >
>> >> Here come the tests:
>> >>
>> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
>> >> of=/export/shares/rcadm/testfile_1 bs=1024M count=11
>> >> dd: writing `/export/shares/rcadm/testfile_1': Disk quota exceeded
>> >> 7+0 records in
>> >> 6+0 records out
>> >> 6933651456 bytes (6.9 GB) copied, 19.7811 s, 351 MB/s
>> >>
>> >
>> > It looks like you've hit the 50G quota.
>> >
>> >> xfs_quota> quota -uh desantis.admin
>> >> Disk quotas for User desantis.admin (663800172)
>> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> >> /dev/mapper/vg_drbd-shares
>> >>                6.5G    10G    10G  00 [------] /export/shares
>> >>
>> >> xfs_quota> limit -u bsoft=0 bhard=0 desantis.admin
>> >>
>> >
>> > Now the 10G quota is gone.
>> >
>> >> xfs_quota> quota -uh desantis.admin
>> >> Disk quotas for User desantis.admin (663800172)
>> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> >> /dev/mapper/vg_drbd-shares
>> >>                6.5G      0      0  00 [------] /export/shares
>> >>
>> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
>> >> of=/export/shares/rcadm/testfile_1 bs=1024M count=5
>> >> 5+0 records in
>> >> 5+0 records out
>> >> 5368709120 bytes (5.4 GB) copied, 10.8275 s, 496 MB/s
>> >>
>> >
>> > You overwrite the previously written 6.5G file to 5.4G (dd will
>> > truncate).
>> >
>> >> # sudo -u desantis.admin -g rcadm dd if=/dev/zero
>> >> of=/export/shares/rcadm/testfile_2 bs=1024M count=5
>> >> dd: writing `/export/shares/rcadm/testfile_2': Disk quota exceeded
>> >> 1+0 records in
>> >> 0+0 records out
>> >> 1028136960 bytes (1.0 GB) copied, 4.60859 s, 223 MB/s
>> >>
>> >
>> > Then create a new file and run into the 50G limit again.
>> >
>> >> xfs_quota> quota -gh rcadm
>> >> Disk quotas for Group rcadm (1034)
>> >> Filesystem   Blocks  Quota  Limit Warn/Time    Mounted on
>> >> /dev/mapper/vg_drbd-shares
>> >>               50.0G    50G    50G  00 [------] /export/shares
>> >>
>> >>
>> >> As you can see with the output, my user quota of 10 GB was enforced
>> >> over the higher group quota of 50 GB.  My original thought process was
>> >> that since the file being created was specified for the group with the
>> >> larger quota, that XFS would enforce that limit over the user quota.
>> >> After reading over several pieces of quota documentation (RTFM!) from
>> >> different O.S. distributions, it looks like you can either have a user
>> >> or group quota getting enforced, but not both in the sense I was
>> >> originally thinking.
>> >>
>> >
>> > This all seems like normal behavior to me. What gives you the impression
>> > that you are hitting the 10G quota? The fact that the quota is reset and
>> > writing continues? Note that dd will truncate a file before it starts
>> > writing unless notrunc is specified.
>> >
>> > Brian
>> >
>> >> That being said, I really do appreciate the responses and extra set of
>> >> eyes and apologize if I've wasted too much time.
>> >>
>> >> John DeSantis
>> >>
>> >> 2014-06-12 20:01 GMT-04:00 Dave Chinner <david@fromorbit.com>:
>> >> > On Thu, Jun 12, 2014 at 10:04:29AM -0400, John Desantis wrote:
>> >> >> Dave,
>> >> >>
>> >> >> Thanks for your reply.  Let me address your questions and replies.
>> >> >>
>> >> >> > That doesn't sound like a filesystem problem. Sounds more like the
>> >> >> > remote user/group configuration is getting squashed. Do the systems
>> >> >> > have the same user/group configurations, or are the clients sending
>> >> >> > group IDs the server doesn't recognise?
>> >> >>
>> >> >> Stracing shows that the group and user ID's are not being squashed and
>> >> >> they are resolving each time to their respective UID/GID's.
>> >> >>
>> >> >> > As it is, describing the problem in text doesn't tell me anything
>> >> >> > about the problem. You need to cut-n-paste the CLI and the output
>> >> >> > showing the incorrect behaviour so we can attempt to reproduce the
>> >> >> > same behaviour you are seeing.
>> >> >>
>> >> >> Busted.  I figured before sending the usual gamut of console "action",
>> >> >> I'd "save" time by simply presenting the symptoms.
>> >> >>
>> >> >> > I can't see how that is related to the filesystem - the filesystem
>> >> >> > will either account for group quota usage or it won't. It doesn't
>> >> >> > care what primary gid the process uses, it will simply account usage
>> >> >> > to that...
>> >> >>
>> >> >> You're correct (but you already knew this!).  I can tell you that
>> >> >> running the quota report does show the group block count changing.
>> >> >>
>> >> >> It looks like this "issue" is  my misunderstanding of how XFS is
>> >> >> checking the quota.  Perhaps you could shed some light?
>> >> >>
>> >> >> When you're using active user and group quotas on the same file system
>> >> >> and the user quota is lower than the group quota, does XFS first check
>> >> >> against the user quota and then the group quota?  Is the order of
>> >> >> operations user, group, and then project or does it depend on the
>> >> >> mounting order of quotas?
>> >> >
>> >> > Order doesn't matter because the filesystem is supposed to check
>> >> > against all enforced quotas, not just the first one.
>> >> >
>> >> >> In the case I presented yesterday, the user in question had a quota of
>> >> >> 25 GB, but the group had an unlimited quota.  Each time that I used
>> >> >> sudo -u user -g group or logged into the user's account, the quota was
>> >> >> always exceeded.  Now that I have removed the user's quota, the group
>> >> >> quota is now being respected properly.
>> >> >
>> >> > Again, you need to post the commands and output that demonstrate the
>> >> > problem you are describing. I'm having trouble understanding what
>> >> > your problem is because I can't see the raw output that you are
>> >> > seeing...
>> >> >
>> >> > Cheers,
>> >> >
>> >> > Dave.
>> >> > --
>> >> > Dave Chinner
>> >> > david@fromorbit.com
>> >>
>> >> _______________________________________________
>> >> xfs mailing list
>> >> xfs@oss.sgi.com
>> >> http://oss.sgi.com/mailman/listinfo/xfs
>>
>> _______________________________________________
>> xfs mailing list
>> xfs@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs