Separate user- and project- quota ?

Separate user- and project- quota ?

[Diego Zuccato]

Hello all.

I'd need to have separate quota for users and projects on the same
filesystem.
I currently have /home/PERSONALE and /home/STUDENTI where homes gets
automatically created. I'd need to have /home/software/prjN directories
with independent quotas.

What I've seen is that if any user creates a file in prjN directory,
that file is counted twice: both in user's and project's quota.

Am I getting something wrong? Missing an option? Can it be done?

TIA.

-- 
Diego Zuccato
DIFA - Dip. di Fisica e Astronomia
Servizi Informatici
Alma Mater Studiorum - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786

Re: Separate user- and project- quota ?

[Eric Sandeen]

On 6/26/20 5:58 AM, Diego Zuccato wrote:
> Hello all.
> 
> I'd need to have separate quota for users and projects on the same
> filesystem.
> I currently have /home/PERSONALE and /home/STUDENTI where homes gets
> automatically created. I'd need to have /home/software/prjN directories
> with independent quotas.
> 
> What I've seen is that if any user creates a file in prjN directory,
> that file is counted twice: both in user's and project's quota.
> 
> Am I getting something wrong? Missing an option? Can it be done?
> 
> TIA.

I'm not certain, but I don't think there is any sort of hierarchy or
priority among the 3 quota types on a given filesystem.

In the same way that if you have user & group quotas active, a file
in a user:group pair would count against user and group limits, I'm
fairly certain that the same behavior applies to user & project quotas.

I think what you're asking is whether a file in a project quota directory
can avoid user quota accounting, and I'm pretty sure the answer is no.

I think you'll be limited by the first quota limit that gets reached.

The way around this would be to make /home/software/ a separate filesystem
with only project quotas active.

(Apologies for not digging into the code to be certain, but from what
you've seen and from what I think I know, the above is my best guess.)

-Eric

Re: Separate user- and project- quota ?

[Diego Zuccato]

Il 28/06/20 20:22, Eric Sandeen ha scritto:

> I think what you're asking is whether a file in a project quota directory
> can avoid user quota accounting, and I'm pretty sure the answer is no.
Maybe it could be a feature request :)

> I think you'll be limited by the first quota limit that gets reached.
Verified that. :(

Moreover, I've had troubles at reboot because the superblock did not
support both group and project quotas. Didn't see anywhere in the docs
you couldn't have both enabled at the same time. Maybe i'ts just because
it's an "old" filesystem (IIRC created about 10y ago). I'd suggest a
"graceful soft failure" in that case: just emit a warning and
disable/ignore last option requested. Recovery of a remote machine would
be way easier :)

-- 
Diego Zuccato
DIFA - Dip. di Fisica e Astronomia
Servizi Informatici
Alma Mater Studiorum - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786

Re: Separate user- and project- quota ?

[Eric Sandeen]

On 7/1/20 3:41 AM, Diego Zuccato wrote:
> Il 28/06/20 20:22, Eric Sandeen ha scritto:
> 
>> I think what you're asking is whether a file in a project quota directory
>> can avoid user quota accounting, and I'm pretty sure the answer is no.
> Maybe it could be a feature request :)
> 
>> I think you'll be limited by the first quota limit that gets reached.
> Verified that. :(
> 
> Moreover, I've had troubles at reboot because the superblock did not
> support both group and project quotas. Didn't see anywhere in the docs
> you couldn't have both enabled at the same time. Maybe i'ts just because
> it's an "old" filesystem (IIRC created about 10y ago). I'd suggest a
> "graceful soft failure" in that case: just emit a warning and
> disable/ignore last option requested. Recovery of a remote machine would
> be way easier :)

Hm, yes, worth a look.  All 3 have been supported together for quite some
time now, I didn't know it reacted badly on old filesystems.

What did the failure look like?

-Eric

Re: Separate user- and project- quota ?

[Diego Zuccato]

Il 01/07/20 20:46, Eric Sandeen ha scritto:

> Hm, yes, worth a look.  All 3 have been supported together for quite some
> time now, I didn't know it reacted badly on old filesystems.
> What did the failure look like?
Boot failure saying something about superblock not supporting both
group- and project- quotas at the same time.
I think it's related to
XFS (dm-0): Mounting V4 Filesystem
As I said, it' quite an old fs :)

xfs_info reports:
meta-data=/dev/sdb1      isize=256    agcount=41, agsize=268435455 blks
         =               sectsz=512   attr=2, projid32bit=0
         =               crc=0        finobt=0, sparse=0, rmapbt=0
         =               reflink=0
data     =               bsize=4096   blocks=10742852608, imaxpct=5
         =               sunit=0      swidth=0 blks
naming   =version 2      bsize=4096   ascii-ci=0, ftype=0
log      =internal log   bsize=4096   blocks=521728, version=2
         =               sectsz=512   sunit=0 blks, lazy-count=1
realtime =none           extsz=4096   blocks=0, rtextents=0

Too bad it's a production server (serving the home for the cluster) and
I can't down it now.

-- 
Diego Zuccato
DIFA - Dip. di Fisica e Astronomia
Servizi Informatici
Alma Mater Studiorum - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786

Re: Separate user- and project- quota ?

[Brian Foster]

On Thu, Jul 02, 2020 at 08:14:01AM +0200, Diego Zuccato wrote:
> Il 01/07/20 20:46, Eric Sandeen ha scritto:
> 
> > Hm, yes, worth a look.  All 3 have been supported together for quite some
> > time now, I didn't know it reacted badly on old filesystems.
> > What did the failure look like?
> Boot failure saying something about superblock not supporting both
> group- and project- quotas at the same time.
> I think it's related to
> XFS (dm-0): Mounting V4 Filesystem
> As I said, it' quite an old fs :)
> 
> xfs_info reports:
> meta-data=/dev/sdb1      isize=256    agcount=41, agsize=268435455 blks
>          =               sectsz=512   attr=2, projid32bit=0
>          =               crc=0        finobt=0, sparse=0, rmapbt=0
>          =               reflink=0
> data     =               bsize=4096   blocks=10742852608, imaxpct=5
>          =               sunit=0      swidth=0 blks
> naming   =version 2      bsize=4096   ascii-ci=0, ftype=0
> log      =internal log   bsize=4096   blocks=521728, version=2
>          =               sectsz=512   sunit=0 blks, lazy-count=1
> realtime =none           extsz=4096   blocks=0, rtextents=0
> 
> Too bad it's a production server (serving the home for the cluster) and
> I can't down it now.
> 

I missed how this went from a question around interaction between user
and project quotas to reporting of a problem associated with enablement
of group+project quotas and an old fs. The above shows a v4 superblock
(crc=0), which means project and group quotas share an inode and thus
are mutually exclusive. It sounds to me that the problem is simply that
you're specifying a set of incompatible mount options on a v4 fs, but
you haven't really stated the problem clearly. I.e.:

# mount /dev/test/scratch /mnt/ -o gquota,pquota
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/mapper/test-scratch, missing codepage or helper program, or other error.
# dmesg | tail
[  247.554345] XFS (dm-3): Super block does not support project and group quota together

We have to fail in this scenario (as opposed to randomly picking one)
because either one can work for any mount (presumably wiping out the old
quotas when changing from one mode to the other across a mount).

Brian

> -- 
> Diego Zuccato
> DIFA - Dip. di Fisica e Astronomia
> Servizi Informatici
> Alma Mater Studiorum - Università di Bologna
> V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
> tel.: +39 051 20 95786
> 

Re: Separate user- and project- quota ?

[Diego Zuccato]

Il 02/07/20 13:34, Brian Foster ha scritto:

> I missed how this went from a question around interaction between user
> and project quotas to reporting of a problem associated with enablement
> of group+project quotas and an old fs.
Detected the problem and reported it as an "aside", suggesting a
possible improvement.

> The above shows a v4 superblock
> (crc=0), which means project and group quotas share an inode and thus
> are mutually exclusive. It sounds to me that the problem is simply that
> you're specifying a set of incompatible mount options on a v4 fs, but
> you haven't really stated the problem clearly. I.e.:
> # mount /dev/test/scratch /mnt/ -o gquota,pquota
> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/mapper/test-scratch, missing codepage or helper program, or other error.
> # dmesg | tail
> [  247.554345] XFS (dm-3): Super block does not support project and group quota together
Seems you pinned it anyway.

> We have to fail in this scenario (as opposed to randomly picking one)
> because either one can work for any mount (presumably wiping out the old
> quotas when changing from one mode to the other across a mount).
So that's not possible because introducing such a change would create
problems in existingsystems. I understand, more or less: if they still
boot, they're using just one option and from down my ignorance it seemed
a good idea to just discard deterministically one of the options
allowing the system to boot anyway. The usual "it's easy if you don't
have to do it" :)

Tks a lot for the clear explanation. Today I learnt something new.

-- 
Diego Zuccato
DIFA - Dip. di Fisica e Astronomia
Servizi Informatici
Alma Mater Studiorum - Università di Bologna
V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
tel.: +39 051 20 95786

Re: Separate user- and project- quota ?

[Brian Foster]

On Thu, Jul 02, 2020 at 02:15:04PM +0200, Diego Zuccato wrote:
> Il 02/07/20 13:34, Brian Foster ha scritto:
> 
> > I missed how this went from a question around interaction between user
> > and project quotas to reporting of a problem associated with enablement
> > of group+project quotas and an old fs.
> Detected the problem and reported it as an "aside", suggesting a
> possible improvement.
> 
> > The above shows a v4 superblock
> > (crc=0), which means project and group quotas share an inode and thus
> > are mutually exclusive. It sounds to me that the problem is simply that
> > you're specifying a set of incompatible mount options on a v4 fs, but
> > you haven't really stated the problem clearly. I.e.:
> > # mount /dev/test/scratch /mnt/ -o gquota,pquota
> > mount: /mnt: wrong fs type, bad option, bad superblock on /dev/mapper/test-scratch, missing codepage or helper program, or other error.
> > # dmesg | tail
> > [  247.554345] XFS (dm-3): Super block does not support project and group quota together
> Seems you pinned it anyway.
> 
> > We have to fail in this scenario (as opposed to randomly picking one)
> > because either one can work for any mount (presumably wiping out the old
> > quotas when changing from one mode to the other across a mount).
> So that's not possible because introducing such a change would create
> problems in existingsystems. I understand, more or less: if they still
> boot, they're using just one option and from down my ignorance it seemed
> a good idea to just discard deterministically one of the options
> allowing the system to boot anyway. The usual "it's easy if you don't
> have to do it" :)
> 

Pretty much. ;) I think it's reasonable in theory to say something like
"pick one or the other for older fs," but then we have to get into
issues like being subtly affected by code changes that might reorder
mount options without any notion of that affecting behavior (i.e. very
brittle) and/or choosing one option of the other based on the current
status of the [pg]quota inode, which is more implementation and doesn't
rule out having to fail the mount in all cases anyways. Suffice it to
say I don't think it's worth going further down that path simply to
support passing a combination of mount options that has no runtime
effect and was never a supported combination for the associated version
of the fs in the first place.

Brian

> Tks a lot for the clear explanation. Today I learnt something new.
> 
> -- 
> Diego Zuccato
> DIFA - Dip. di Fisica e Astronomia
> Servizi Informatici
> Alma Mater Studiorum - Università di Bologna
> V.le Berti-Pichat 6/2 - 40127 Bologna - Italy
> tel.: +39 051 20 95786
> 

Re: Separate user- and project- quota ?

[Eric Sandeen]

On 7/2/20 7:50 AM, Brian Foster wrote:
> Pretty much. ;) I think it's reasonable in theory to say something like
> "pick one or the other for older fs," but then we have to get into
> issues like being subtly affected by code changes that might reorder
> mount options without any notion of that affecting behavior (i.e. very
> brittle) and/or choosing one option of the other based on the current
> status of the [pg]quota inode, which is more implementation and doesn't
> rule out having to fail the mount in all cases anyways. Suffice it to
> say I don't think it's worth going further down that path simply to
> support passing a combination of mount options that has no runtime
> effect and was never a supported combination for the associated version
> of the fs in the first place.

It might be reasonable to /disable/ both in the case of an invalid
specification, with a warning, but then that gets into "do something
behind the user's back" - they asked for something impossible, and we
gave them ... nothing, with a successful mount return?

The tradeoff between that and a failed boot & trip to the data center
can be a tough one, but we usually err on the side of "don't silently
carry on pretending it's all OK if we've been asked to do something
impossible."

Unfortunately the only test for "did you ask for an impossible quota
scenario in your mount flags?" is to mount it...

-Eric