RE: Maximum Number of ACL on NFSv4

RE: Maximum Number of ACL on NFSv4

[de Vandiere, Louis]

Hi,

I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.

On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
Thank you for your support.
Best,
Louis de Vandière

Re: Maximum Number of ACL on NFSv4

[Goetz, Patrick G]

I'm dying to know what the use case is for this, and why you can't just 
do this with group permissions (unless you're talking about hundreds of 
group ACLs).

On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> Hi,
> 
> I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> 
> On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> Thank you for your support.
> Best,
> Louis de Vandière
>>> This message is from an external sender. Learn more about why this <<
>>> matters at https://links.utexas.edu/rtyclf.                        <<
> 

RE: Maximum Number of ACL on NFSv4

[de Vandiere, Louis]

Yes, I assume it's not very frequent to have hundreds of NFSv4 ACLs. For compliance and organizational issue, we cannot use groups efficiently to manage access to the shares, so it's user-based and case by case.
 
My real goal is to be able to replicate some files to a new NFSv4 server while preserving the ACLs. By using "cp -R --preserve=all acl-folder/", I'm able to preserve the ACLs when their number does not exceed 200, above it, I see the "File too large" error while rsync does not work at all (even in version 3.1.3). That's why I'm digging into this and checking what possibly could go wrong.

Thank you.
Best,
Louis de Vandière


-----Original Message-----
From: Goetz, Patrick G <pgoetz@math.utexas.edu> 
Sent: Monday, August 26, 2019 8:44 AM
To: de Vandiere, Louis <louis.devandiere@atos.net>; linux-nfs@vger.kernel.org
Subject: Re: Maximum Number of ACL on NFSv4

I'm dying to know what the use case is for this, and why you can't just do this with group permissions (unless you're talking about hundreds of group ACLs).

On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> Hi,
> 
> I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> 
> On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> Thank you for your support.
> Best,
> Louis de Vandière
>>> This message is from an external sender. Learn more about why this <<
>>> matters at https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce6d6b4705cde46bf455a08d72a2b93df%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024238975648645&amp;sdata=EgTg3%2BPyIrnqG6axcHOybKZ1AldsGXj8CIC5z0F0Rac%3D&amp;reserved=0.                        <<
> 

Re: Maximum Number of ACL on NFSv4

[Niels de Vos]

On Mon, Aug 26, 2019 at 02:53:05PM +0000, de Vandiere, Louis wrote:
> Yes, I assume it's not very frequent to have hundreds of NFSv4 ACLs. For compliance and organizational issue, we cannot use groups efficiently to manage access to the shares, so it's user-based and case by case.
>  
> My real goal is to be able to replicate some files to a new NFSv4 server while preserving the ACLs. By using "cp -R --preserve=all acl-folder/", I'm able to preserve the ACLs when their number does not exceed 200, above it, I see the "File too large" error while rsync does not work at all (even in version 3.1.3). That's why I'm digging into this and checking what possibly could go wrong.

You might be hitting a limit in the filesystem on the NFS server. The
ACLs are stored in extended attributes. Depending on the filesystem, you
may be able to configure larger inode sizes (or other storage for
xattrs). With XFS this can be done with 'mkfs -t xfs -i size=.. ...',

HTH,
Niels


> 
> Thank you.
> Best,
> Louis de Vandière
> 
> 
> -----Original Message-----
> From: Goetz, Patrick G <pgoetz@math.utexas.edu> 
> Sent: Monday, August 26, 2019 8:44 AM
> To: de Vandiere, Louis <louis.devandiere@atos.net>; linux-nfs@vger.kernel.org
> Subject: Re: Maximum Number of ACL on NFSv4
> 
> I'm dying to know what the use case is for this, and why you can't just do this with group permissions (unless you're talking about hundreds of group ACLs).
> 
> On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> > Hi,
> > 
> > I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> > 
> > On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> > Thank you for your support.
> > Best,
> > Louis de Vandière
> >>> This message is from an external sender. Learn more about why this <<
> >>> matters at https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce6d6b4705cde46bf455a08d72a2b93df%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024238975648645&amp;sdata=EgTg3%2BPyIrnqG6axcHOybKZ1AldsGXj8CIC5z0F0Rac%3D&amp;reserved=0.                        <<
> > 

RE: Maximum Number of ACL on NFSv4

[de Vandiere, Louis]

Thanks Niels, I tried your suggestion. According to the documentation (https://linux.die.net/man/8/mkfs.xfs), the maximum size for the inode is 2048 byte. So I set it to this value, and faced the exact same limitation. On the other hand, when I used setfacl -m on the XFS mounted disk, I did not face any limitation and I was able to set thousands of ACLs on a single file.

When I do a strace, I see two different types of ACL used when the system calls setxattr: system.posix_acl_default and system.nfsv4_acl. I tried to look for hardcoded limits associated with system.nfsv4_acl but I don't have much experience with C and linux kernel.

Thank you for your help.
Best,
Louis de Vandière

-----Original Message-----
From: Niels de Vos <ndevos@redhat.com> 
Sent: Monday, August 26, 2019 11:46 AM
To: de Vandiere, Louis <louis.devandiere@atos.net>
Cc: linux-nfs@vger.kernel.org
Subject: Re: Maximum Number of ACL on NFSv4

On Mon, Aug 26, 2019 at 02:53:05PM +0000, de Vandiere, Louis wrote:
> Yes, I assume it's not very frequent to have hundreds of NFSv4 ACLs. For compliance and organizational issue, we cannot use groups efficiently to manage access to the shares, so it's user-based and case by case.
>  
> My real goal is to be able to replicate some files to a new NFSv4 server while preserving the ACLs. By using "cp -R --preserve=all acl-folder/", I'm able to preserve the ACLs when their number does not exceed 200, above it, I see the "File too large" error while rsync does not work at all (even in version 3.1.3). That's why I'm digging into this and checking what possibly could go wrong.

You might be hitting a limit in the filesystem on the NFS server. The ACLs are stored in extended attributes. Depending on the filesystem, you may be able to configure larger inode sizes (or other storage for xattrs). With XFS this can be done with 'mkfs -t xfs -i size=.. ...',

HTH,
Niels


> 
> Thank you.
> Best,
> Louis de Vandière
> 
> 
> -----Original Message-----
> From: Goetz, Patrick G <pgoetz@math.utexas.edu>
> Sent: Monday, August 26, 2019 8:44 AM
> To: de Vandiere, Louis <louis.devandiere@atos.net>; 
> linux-nfs@vger.kernel.org
> Subject: Re: Maximum Number of ACL on NFSv4
> 
> I'm dying to know what the use case is for this, and why you can't just do this with group permissions (unless you're talking about hundreds of group ACLs).
> 
> On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> > Hi,
> > 
> > I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> > 
> > On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> > Thank you for your support.
> > Best,
> > Louis de Vandière
> >>> This message is from an external sender. Learn more about why this <<
> >>> matters at https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce3e196698745444ba59208d72a44ed69%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024347858295832&amp;sdata=peZa9vHRp77QbOX2yTj204oWk8iCO%2FxNbSMzkylf38M%3D&amp;reserved=0.                        <<
> > 

Re: Maximum Number of ACL on NFSv4

[Olga Kornievskaia]

From fs/nfsd/acl.h
/*
 * Maximum ACL we'll accept from a client; chosen (somewhat
 * arbitrarily) so that kmalloc'ing the ACL shouldn't require a
 * high-order allocation.  This allows 204 ACEs on x86_64:
 */
#define NFS4_ACL_MAX ((PAGE_SIZE - sizeof(struct nfs4_acl)) \
                        / sizeof(struct nfs4_ace))

I don't know how Bruce feels about increasing that limit. Perhaps he'd
be opened to a patch that increases that.

On Mon, Aug 26, 2019 at 2:30 PM de Vandiere, Louis
<louis.devandiere@atos.net> wrote:
>
> Thanks Niels, I tried your suggestion. According to the documentation (https://linux.die.net/man/8/mkfs.xfs), the maximum size for the inode is 2048 byte. So I set it to this value, and faced the exact same limitation. On the other hand, when I used setfacl -m on the XFS mounted disk, I did not face any limitation and I was able to set thousands of ACLs on a single file.
>
> When I do a strace, I see two different types of ACL used when the system calls setxattr: system.posix_acl_default and system.nfsv4_acl. I tried to look for hardcoded limits associated with system.nfsv4_acl but I don't have much experience with C and linux kernel.
>
> Thank you for your help.
> Best,
> Louis de Vandière
>
> -----Original Message-----
> From: Niels de Vos <ndevos@redhat.com>
> Sent: Monday, August 26, 2019 11:46 AM
> To: de Vandiere, Louis <louis.devandiere@atos.net>
> Cc: linux-nfs@vger.kernel.org
> Subject: Re: Maximum Number of ACL on NFSv4
>
> On Mon, Aug 26, 2019 at 02:53:05PM +0000, de Vandiere, Louis wrote:
> > Yes, I assume it's not very frequent to have hundreds of NFSv4 ACLs. For compliance and organizational issue, we cannot use groups efficiently to manage access to the shares, so it's user-based and case by case.
> >
> > My real goal is to be able to replicate some files to a new NFSv4 server while preserving the ACLs. By using "cp -R --preserve=all acl-folder/", I'm able to preserve the ACLs when their number does not exceed 200, above it, I see the "File too large" error while rsync does not work at all (even in version 3.1.3). That's why I'm digging into this and checking what possibly could go wrong.
>
> You might be hitting a limit in the filesystem on the NFS server. The ACLs are stored in extended attributes. Depending on the filesystem, you may be able to configure larger inode sizes (or other storage for xattrs). With XFS this can be done with 'mkfs -t xfs -i size=.. ...',
>
> HTH,
> Niels
>
>
> >
> > Thank you.
> > Best,
> > Louis de Vandière
> >
> >
> > -----Original Message-----
> > From: Goetz, Patrick G <pgoetz@math.utexas.edu>
> > Sent: Monday, August 26, 2019 8:44 AM
> > To: de Vandiere, Louis <louis.devandiere@atos.net>;
> > linux-nfs@vger.kernel.org
> > Subject: Re: Maximum Number of ACL on NFSv4
> >
> > I'm dying to know what the use case is for this, and why you can't just do this with group permissions (unless you're talking about hundreds of group ACLs).
> >
> > On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> > > Hi,
> > >
> > > I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> > >
> > > On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> > > Thank you for your support.
> > > Best,
> > > Louis de Vandière
> > >>> This message is from an external sender. Learn more about why this <<
> > >>> matters at https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce3e196698745444ba59208d72a44ed69%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024347858295832&amp;sdata=peZa9vHRp77QbOX2yTj204oWk8iCO%2FxNbSMzkylf38M%3D&amp;reserved=0.                        <<
> > >

RE: Maximum Number of ACL on NFSv4

[de Vandiere, Louis]

Thank you Olga! Somehow, I failed to look into this file although I looked in fs/nfs/ without success and I understand why now.

I'd like to see it increased and be scalable like XFS is, but I understand it might impact multiple libraries. Should I open a bug/feature request somewhere?

Best,
Louis de Vandière

-----Original Message-----
From: Olga Kornievskaia <aglo@umich.edu> 
Sent: Monday, August 26, 2019 2:31 PM
To: de Vandiere, Louis <louis.devandiere@atos.net>
Cc: linux-nfs@vger.kernel.org
Subject: Re: Maximum Number of ACL on NFSv4

From fs/nfsd/acl.h
/*
 * Maximum ACL we'll accept from a client; chosen (somewhat
 * arbitrarily) so that kmalloc'ing the ACL shouldn't require a
 * high-order allocation.  This allows 204 ACEs on x86_64:
 */
#define NFS4_ACL_MAX ((PAGE_SIZE - sizeof(struct nfs4_acl)) \
                        / sizeof(struct nfs4_ace))

I don't know how Bruce feels about increasing that limit. Perhaps he'd be opened to a patch that increases that.

On Mon, Aug 26, 2019 at 2:30 PM de Vandiere, Louis <louis.devandiere@atos.net> wrote:
>
> Thanks Niels, I tried your suggestion. According to the documentation (https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinux.die.net%2Fman%2F8%2Fmkfs.xfs&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=HZbnVSzTKKCXpEv5JLgZKeEgQx5BPKeEs4SYZqRhhbk%3D&amp;reserved=0), the maximum size for the inode is 2048 byte. So I set it to this value, and faced the exact same limitation. On the other hand, when I used setfacl -m on the XFS mounted disk, I did not face any limitation and I was able to set thousands of ACLs on a single file.
>
> When I do a strace, I see two different types of ACL used when the system calls setxattr: system.posix_acl_default and system.nfsv4_acl. I tried to look for hardcoded limits associated with system.nfsv4_acl but I don't have much experience with C and linux kernel.
>
> Thank you for your help.
> Best,
> Louis de Vandière
>
> -----Original Message-----
> From: Niels de Vos <ndevos@redhat.com>
> Sent: Monday, August 26, 2019 11:46 AM
> To: de Vandiere, Louis <louis.devandiere@atos.net>
> Cc: linux-nfs@vger.kernel.org
> Subject: Re: Maximum Number of ACL on NFSv4
>
> On Mon, Aug 26, 2019 at 02:53:05PM +0000, de Vandiere, Louis wrote:
> > Yes, I assume it's not very frequent to have hundreds of NFSv4 ACLs. For compliance and organizational issue, we cannot use groups efficiently to manage access to the shares, so it's user-based and case by case.
> >
> > My real goal is to be able to replicate some files to a new NFSv4 server while preserving the ACLs. By using "cp -R --preserve=all acl-folder/", I'm able to preserve the ACLs when their number does not exceed 200, above it, I see the "File too large" error while rsync does not work at all (even in version 3.1.3). That's why I'm digging into this and checking what possibly could go wrong.
>
> You might be hitting a limit in the filesystem on the NFS server. The 
> ACLs are stored in extended attributes. Depending on the filesystem, 
> you may be able to configure larger inode sizes (or other storage for 
> xattrs). With XFS this can be done with 'mkfs -t xfs -i size=.. ...',
>
> HTH,
> Niels
>
>
> >
> > Thank you.
> > Best,
> > Louis de Vandière
> >
> >
> > -----Original Message-----
> > From: Goetz, Patrick G <pgoetz@math.utexas.edu>
> > Sent: Monday, August 26, 2019 8:44 AM
> > To: de Vandiere, Louis <louis.devandiere@atos.net>; 
> > linux-nfs@vger.kernel.org
> > Subject: Re: Maximum Number of ACL on NFSv4
> >
> > I'm dying to know what the use case is for this, and why you can't just do this with group permissions (unless you're talking about hundreds of group ACLs).
> >
> > On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> > > Hi,
> > >
> > > I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> > >
> > > On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> > > Thank you for your support.
> > > Best,
> > > Louis de Vandière
> > >>> This message is from an external sender. Learn more about why this <<
> > >>> matters at https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=r345rqWN4GKT0mBmQmMTnaC%2FFEyUTidjBlGeAMRdEpA%3D&amp;reserved=0.                        <<
> > >

Re: Maximum Number of ACL on NFSv4

[J. Bruce Fields]

On Mon, Aug 26, 2019 at 11:28:21PM +0000, de Vandiere, Louis wrote:
> Thank you Olga! Somehow, I failed to look into this file although I looked in fs/nfs/ without success and I understand why now.
> 
> I'd like to see it increased and be scalable like XFS is, but I understand it might impact multiple libraries. Should I open a bug/feature request somewhere?

I wonder if it'd be OK to remove the limit completely (and then leave
it to the filesystem to reject if if it wants).

It does mean we're passing an arbitrary client-supplied value to
kmalloc.  Is it OK to do that and just leave it to the allocator to
reject excessive requests, or do we risk pushing it into making heroic
efforts to satisfy a possibly malicious or broken client?

I wonder if there's also a risk in passing down posix ACLs larger than
could have been created with the setxattr system call.

Assuming it's still safest to have a limit....

XATTR_LIST_MAX is a global limit on the size of xattrs.  We could try to
estimate how big the converted posix ACL will be and work out a maximum
based on that.

--b.

> 
> Best,
> Louis de Vandière
> 
> -----Original Message-----
> From: Olga Kornievskaia <aglo@umich.edu> 
> Sent: Monday, August 26, 2019 2:31 PM
> To: de Vandiere, Louis <louis.devandiere@atos.net>
> Cc: linux-nfs@vger.kernel.org
> Subject: Re: Maximum Number of ACL on NFSv4
> 
> From fs/nfsd/acl.h
> /*
>  * Maximum ACL we'll accept from a client; chosen (somewhat
>  * arbitrarily) so that kmalloc'ing the ACL shouldn't require a
>  * high-order allocation.  This allows 204 ACEs on x86_64:
>  */
> #define NFS4_ACL_MAX ((PAGE_SIZE - sizeof(struct nfs4_acl)) \
>                         / sizeof(struct nfs4_ace))
> 
> I don't know how Bruce feels about increasing that limit. Perhaps he'd be opened to a patch that increases that.
> 
> On Mon, Aug 26, 2019 at 2:30 PM de Vandiere, Louis <louis.devandiere@atos.net> wrote:
> >
> > Thanks Niels, I tried your suggestion. According to the documentation (https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinux.die.net%2Fman%2F8%2Fmkfs.xfs&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=HZbnVSzTKKCXpEv5JLgZKeEgQx5BPKeEs4SYZqRhhbk%3D&amp;reserved=0), the maximum size for the inode is 2048 byte. So I set it to this value, and faced the exact same limitation. On the other hand, when I used setfacl -m on the XFS mounted disk, I did not face any limitation and I was able to set thousands of ACLs on a single file.
> >
> > When I do a strace, I see two different types of ACL used when the system calls setxattr: system.posix_acl_default and system.nfsv4_acl. I tried to look for hardcoded limits associated with system.nfsv4_acl but I don't have much experience with C and linux kernel.
> >
> > Thank you for your help.
> > Best,
> > Louis de Vandière
> >
> > -----Original Message-----
> > From: Niels de Vos <ndevos@redhat.com>
> > Sent: Monday, August 26, 2019 11:46 AM
> > To: de Vandiere, Louis <louis.devandiere@atos.net>
> > Cc: linux-nfs@vger.kernel.org
> > Subject: Re: Maximum Number of ACL on NFSv4
> >
> > On Mon, Aug 26, 2019 at 02:53:05PM +0000, de Vandiere, Louis wrote:
> > > Yes, I assume it's not very frequent to have hundreds of NFSv4 ACLs. For compliance and organizational issue, we cannot use groups efficiently to manage access to the shares, so it's user-based and case by case.
> > >
> > > My real goal is to be able to replicate some files to a new NFSv4 server while preserving the ACLs. By using "cp -R --preserve=all acl-folder/", I'm able to preserve the ACLs when their number does not exceed 200, above it, I see the "File too large" error while rsync does not work at all (even in version 3.1.3). That's why I'm digging into this and checking what possibly could go wrong.
> >
> > You might be hitting a limit in the filesystem on the NFS server. The 
> > ACLs are stored in extended attributes. Depending on the filesystem, 
> > you may be able to configure larger inode sizes (or other storage for 
> > xattrs). With XFS this can be done with 'mkfs -t xfs -i size=.. ...',
> >
> > HTH,
> > Niels
> >
> >
> > >
> > > Thank you.
> > > Best,
> > > Louis de Vandière
> > >
> > >
> > > -----Original Message-----
> > > From: Goetz, Patrick G <pgoetz@math.utexas.edu>
> > > Sent: Monday, August 26, 2019 8:44 AM
> > > To: de Vandiere, Louis <louis.devandiere@atos.net>; 
> > > linux-nfs@vger.kernel.org
> > > Subject: Re: Maximum Number of ACL on NFSv4
> > >
> > > I'm dying to know what the use case is for this, and why you can't just do this with group permissions (unless you're talking about hundreds of group ACLs).
> > >
> > > On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> > > > Hi,
> > > >
> > > > I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> > > >
> > > > On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> > > > Thank you for your support.
> > > > Best,
> > > > Louis de Vandière
> > > >>> This message is from an external sender. Learn more about why this <<
> > > >>> matters at https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=r345rqWN4GKT0mBmQmMTnaC%2FFEyUTidjBlGeAMRdEpA%3D&amp;reserved=0.                        <<
> > > >

Re: Maximum Number of ACL on NFSv4

[Olga Kornievskaia]

On Wed, Aug 28, 2019 at 2:06 PM J. Bruce Fields <bfields@fieldses.org> wrote:
>
> On Mon, Aug 26, 2019 at 11:28:21PM +0000, de Vandiere, Louis wrote:
> > Thank you Olga! Somehow, I failed to look into this file although I looked in fs/nfs/ without success and I understand why now.
> >
> > I'd like to see it increased and be scalable like XFS is, but I understand it might impact multiple libraries. Should I open a bug/feature request somewhere?
>
> I wonder if it'd be OK to remove the limit completely (and then leave
> it to the filesystem to reject if if it wants).
>
> It does mean we're passing an arbitrary client-supplied value to
> kmalloc.  Is it OK to do that and just leave it to the allocator to
> reject excessive requests, or do we risk pushing it into making heroic
> efforts to satisfy a possibly malicious or broken client?
>
> I wonder if there's also a risk in passing down posix ACLs larger than
> could have been created with the setxattr system call.
>
> Assuming it's still safest to have a limit....
>
> XATTR_LIST_MAX is a global limit on the size of xattrs.  We could try to
> estimate how big the converted posix ACL will be and work out a maximum
> based on that.

I agree there should be a limit and using XATTR_LIST_MAX sounds
reasonable to me.

>
> --b.
>
> >
> > Best,
> > Louis de Vandière
> >
> > -----Original Message-----
> > From: Olga Kornievskaia <aglo@umich.edu>
> > Sent: Monday, August 26, 2019 2:31 PM
> > To: de Vandiere, Louis <louis.devandiere@atos.net>
> > Cc: linux-nfs@vger.kernel.org
> > Subject: Re: Maximum Number of ACL on NFSv4
> >
> > From fs/nfsd/acl.h
> > /*
> >  * Maximum ACL we'll accept from a client; chosen (somewhat
> >  * arbitrarily) so that kmalloc'ing the ACL shouldn't require a
> >  * high-order allocation.  This allows 204 ACEs on x86_64:
> >  */
> > #define NFS4_ACL_MAX ((PAGE_SIZE - sizeof(struct nfs4_acl)) \
> >                         / sizeof(struct nfs4_ace))
> >
> > I don't know how Bruce feels about increasing that limit. Perhaps he'd be opened to a patch that increases that.
> >
> > On Mon, Aug 26, 2019 at 2:30 PM de Vandiere, Louis <louis.devandiere@atos.net> wrote:
> > >
> > > Thanks Niels, I tried your suggestion. According to the documentation (https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinux.die.net%2Fman%2F8%2Fmkfs.xfs&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=HZbnVSzTKKCXpEv5JLgZKeEgQx5BPKeEs4SYZqRhhbk%3D&amp;reserved=0), the maximum size for the inode is 2048 byte. So I set it to this value, and faced the exact same limitation. On the other hand, when I used setfacl -m on the XFS mounted disk, I did not face any limitation and I was able to set thousands of ACLs on a single file.
> > >
> > > When I do a strace, I see two different types of ACL used when the system calls setxattr: system.posix_acl_default and system.nfsv4_acl. I tried to look for hardcoded limits associated with system.nfsv4_acl but I don't have much experience with C and linux kernel.
> > >
> > > Thank you for your help.
> > > Best,
> > > Louis de Vandière
> > >
> > > -----Original Message-----
> > > From: Niels de Vos <ndevos@redhat.com>
> > > Sent: Monday, August 26, 2019 11:46 AM
> > > To: de Vandiere, Louis <louis.devandiere@atos.net>
> > > Cc: linux-nfs@vger.kernel.org
> > > Subject: Re: Maximum Number of ACL on NFSv4
> > >
> > > On Mon, Aug 26, 2019 at 02:53:05PM +0000, de Vandiere, Louis wrote:
> > > > Yes, I assume it's not very frequent to have hundreds of NFSv4 ACLs. For compliance and organizational issue, we cannot use groups efficiently to manage access to the shares, so it's user-based and case by case.
> > > >
> > > > My real goal is to be able to replicate some files to a new NFSv4 server while preserving the ACLs. By using "cp -R --preserve=all acl-folder/", I'm able to preserve the ACLs when their number does not exceed 200, above it, I see the "File too large" error while rsync does not work at all (even in version 3.1.3). That's why I'm digging into this and checking what possibly could go wrong.
> > >
> > > You might be hitting a limit in the filesystem on the NFS server. The
> > > ACLs are stored in extended attributes. Depending on the filesystem,
> > > you may be able to configure larger inode sizes (or other storage for
> > > xattrs). With XFS this can be done with 'mkfs -t xfs -i size=.. ...',
> > >
> > > HTH,
> > > Niels
> > >
> > >
> > > >
> > > > Thank you.
> > > > Best,
> > > > Louis de Vandière
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Goetz, Patrick G <pgoetz@math.utexas.edu>
> > > > Sent: Monday, August 26, 2019 8:44 AM
> > > > To: de Vandiere, Louis <louis.devandiere@atos.net>;
> > > > linux-nfs@vger.kernel.org
> > > > Subject: Re: Maximum Number of ACL on NFSv4
> > > >
> > > > I'm dying to know what the use case is for this, and why you can't just do this with group permissions (unless you're talking about hundreds of group ACLs).
> > > >
> > > > On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> > > > > Hi,
> > > > >
> > > > > I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> > > > >
> > > > > On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> > > > > Thank you for your support.
> > > > > Best,
> > > > > Louis de Vandière
> > > > >>> This message is from an external sender. Learn more about why this <<
> > > > >>> matters at https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=r345rqWN4GKT0mBmQmMTnaC%2FFEyUTidjBlGeAMRdEpA%3D&amp;reserved=0.                        <<
> > > > >

Re: Maximum Number of ACL on NFSv4

[J. Bruce Fields]

On Wed, Aug 28, 2019 at 03:09:03PM -0400, Olga Kornievskaia wrote:
> On Wed, Aug 28, 2019 at 2:06 PM J. Bruce Fields <bfields@fieldses.org> wrote:
> >
> > On Mon, Aug 26, 2019 at 11:28:21PM +0000, de Vandiere, Louis wrote:
> > > Thank you Olga! Somehow, I failed to look into this file although I looked in fs/nfs/ without success and I understand why now.
> > >
> > > I'd like to see it increased and be scalable like XFS is, but I understand it might impact multiple libraries. Should I open a bug/feature request somewhere?
> >
> > I wonder if it'd be OK to remove the limit completely (and then leave
> > it to the filesystem to reject if if it wants).
> >
> > It does mean we're passing an arbitrary client-supplied value to
> > kmalloc.  Is it OK to do that and just leave it to the allocator to
> > reject excessive requests, or do we risk pushing it into making heroic
> > efforts to satisfy a possibly malicious or broken client?
> >
> > I wonder if there's also a risk in passing down posix ACLs larger than
> > could have been created with the setxattr system call.
> >
> > Assuming it's still safest to have a limit....
> >
> > XATTR_LIST_MAX is a global limit on the size of xattrs.  We could try to
> > estimate how big the converted posix ACL will be and work out a maximum
> > based on that.
> 
> I agree there should be a limit and using XATTR_LIST_MAX sounds
> reasonable to me.

OK.  And, whoops, I meant to say XATTR_SIZE_MAX.  (Both are currently
64K, though.)

--b.

> 
> >
> > --b.
> >
> > >
> > > Best,
> > > Louis de Vandière
> > >
> > > -----Original Message-----
> > > From: Olga Kornievskaia <aglo@umich.edu>
> > > Sent: Monday, August 26, 2019 2:31 PM
> > > To: de Vandiere, Louis <louis.devandiere@atos.net>
> > > Cc: linux-nfs@vger.kernel.org
> > > Subject: Re: Maximum Number of ACL on NFSv4
> > >
> > > From fs/nfsd/acl.h
> > > /*
> > >  * Maximum ACL we'll accept from a client; chosen (somewhat
> > >  * arbitrarily) so that kmalloc'ing the ACL shouldn't require a
> > >  * high-order allocation.  This allows 204 ACEs on x86_64:
> > >  */
> > > #define NFS4_ACL_MAX ((PAGE_SIZE - sizeof(struct nfs4_acl)) \
> > >                         / sizeof(struct nfs4_ace))
> > >
> > > I don't know how Bruce feels about increasing that limit. Perhaps he'd be opened to a patch that increases that.
> > >
> > > On Mon, Aug 26, 2019 at 2:30 PM de Vandiere, Louis <louis.devandiere@atos.net> wrote:
> > > >
> > > > Thanks Niels, I tried your suggestion. According to the documentation (https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinux.die.net%2Fman%2F8%2Fmkfs.xfs&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=HZbnVSzTKKCXpEv5JLgZKeEgQx5BPKeEs4SYZqRhhbk%3D&amp;reserved=0), the maximum size for the inode is 2048 byte. So I set it to this value, and faced the exact same limitation. On the other hand, when I used setfacl -m on the XFS mounted disk, I did not face any limitation and I was able to set thousands of ACLs on a single file.
> > > >
> > > > When I do a strace, I see two different types of ACL used when the system calls setxattr: system.posix_acl_default and system.nfsv4_acl. I tried to look for hardcoded limits associated with system.nfsv4_acl but I don't have much experience with C and linux kernel.
> > > >
> > > > Thank you for your help.
> > > > Best,
> > > > Louis de Vandière
> > > >
> > > > -----Original Message-----
> > > > From: Niels de Vos <ndevos@redhat.com>
> > > > Sent: Monday, August 26, 2019 11:46 AM
> > > > To: de Vandiere, Louis <louis.devandiere@atos.net>
> > > > Cc: linux-nfs@vger.kernel.org
> > > > Subject: Re: Maximum Number of ACL on NFSv4
> > > >
> > > > On Mon, Aug 26, 2019 at 02:53:05PM +0000, de Vandiere, Louis wrote:
> > > > > Yes, I assume it's not very frequent to have hundreds of NFSv4 ACLs. For compliance and organizational issue, we cannot use groups efficiently to manage access to the shares, so it's user-based and case by case.
> > > > >
> > > > > My real goal is to be able to replicate some files to a new NFSv4 server while preserving the ACLs. By using "cp -R --preserve=all acl-folder/", I'm able to preserve the ACLs when their number does not exceed 200, above it, I see the "File too large" error while rsync does not work at all (even in version 3.1.3). That's why I'm digging into this and checking what possibly could go wrong.
> > > >
> > > > You might be hitting a limit in the filesystem on the NFS server. The
> > > > ACLs are stored in extended attributes. Depending on the filesystem,
> > > > you may be able to configure larger inode sizes (or other storage for
> > > > xattrs). With XFS this can be done with 'mkfs -t xfs -i size=.. ...',
> > > >
> > > > HTH,
> > > > Niels
> > > >
> > > >
> > > > >
> > > > > Thank you.
> > > > > Best,
> > > > > Louis de Vandière
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Goetz, Patrick G <pgoetz@math.utexas.edu>
> > > > > Sent: Monday, August 26, 2019 8:44 AM
> > > > > To: de Vandiere, Louis <louis.devandiere@atos.net>;
> > > > > linux-nfs@vger.kernel.org
> > > > > Subject: Re: Maximum Number of ACL on NFSv4
> > > > >
> > > > > I'm dying to know what the use case is for this, and why you can't just do this with group permissions (unless you're talking about hundreds of group ACLs).
> > > > >
> > > > > On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I'm currently trying to apply hundreds of ACLs on file hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that the limit I can apply is 207. After the limit is reached, the command "nfs4_setfacl -a" returned the error "Failed setxattr operation: File too large". The same problem happens if I use an ACL with more than 200 line in it. I did a little debugging session but I was not able to come up with an explanation on why I'm facing such an issue.
> > > > > >
> > > > > > On the other hand, I can apply hundreds of ACLs on XFS without issue. Do you know if it could be a bug with the nfs4-acl-tools package?
> > > > > > Thank you for your support.
> > > > > > Best,
> > > > > > Louis de Vandière
> > > > > >>> This message is from an external sender. Learn more about why this <<
> > > > > >>> matters at https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=r345rqWN4GKT0mBmQmMTnaC%2FFEyUTidjBlGeAMRdEpA%3D&amp;reserved=0.                        <<
> > > > > >

Re: Maximum Number of ACL on NFSv4

[Trond Myklebust]

On Wed, 2019-08-28 at 15:29 -0400, J. Bruce Fields wrote:
> On Wed, Aug 28, 2019 at 03:09:03PM -0400, Olga Kornievskaia wrote:
> > On Wed, Aug 28, 2019 at 2:06 PM J. Bruce Fields <
> > bfields@fieldses.org> wrote:
> > > On Mon, Aug 26, 2019 at 11:28:21PM +0000, de Vandiere, Louis
> > > wrote:
> > > > Thank you Olga! Somehow, I failed to look into this file
> > > > although I looked in fs/nfs/ without success and I understand
> > > > why now.
> > > > 
> > > > I'd like to see it increased and be scalable like XFS is, but I
> > > > understand it might impact multiple libraries. Should I open a
> > > > bug/feature request somewhere?
> > > 
> > > I wonder if it'd be OK to remove the limit completely (and then
> > > leave
> > > it to the filesystem to reject if if it wants).
> > > 
> > > It does mean we're passing an arbitrary client-supplied value to
> > > kmalloc.  Is it OK to do that and just leave it to the allocator
> > > to
> > > reject excessive requests, or do we risk pushing it into making
> > > heroic
> > > efforts to satisfy a possibly malicious or broken client?
> > > 
> > > I wonder if there's also a risk in passing down posix ACLs larger
> > > than
> > > could have been created with the setxattr system call.
> > > 
> > > Assuming it's still safest to have a limit....
> > > 
> > > XATTR_LIST_MAX is a global limit on the size of xattrs.  We could
> > > try to
> > > estimate how big the converted posix ACL will be and work out a
> > > maximum
> > > based on that.
> > 
> > I agree there should be a limit and using XATTR_LIST_MAX sounds
> > reasonable to me.
> 
> OK.  And, whoops, I meant to say XATTR_SIZE_MAX.  (Both are currently
> 64K, though.)
> 

Umm... Don't forget that NFSv4 ACL aces are typically much larger than
POSIX ACL aces because the user/group names are encoded as strings, not
binary uids and gids.

IOW: The size of the RPC message is likely to be a lot larger than the
resulting POSIX ACL...

> --b.
> 
> > > --b.
> > > 
> > > > Best,
> > > > Louis de Vandière
> > > > 
> > > > -----Original Message-----
> > > > From: Olga Kornievskaia <aglo@umich.edu>
> > > > Sent: Monday, August 26, 2019 2:31 PM
> > > > To: de Vandiere, Louis <louis.devandiere@atos.net>
> > > > Cc: linux-nfs@vger.kernel.org
> > > > Subject: Re: Maximum Number of ACL on NFSv4
> > > > 
> > > > From fs/nfsd/acl.h
> > > > /*
> > > >  * Maximum ACL we'll accept from a client; chosen (somewhat
> > > >  * arbitrarily) so that kmalloc'ing the ACL shouldn't require a
> > > >  * high-order allocation.  This allows 204 ACEs on x86_64:
> > > >  */
> > > > #define NFS4_ACL_MAX ((PAGE_SIZE - sizeof(struct nfs4_acl)) \
> > > >                         / sizeof(struct nfs4_ace))
> > > > 
> > > > I don't know how Bruce feels about increasing that limit.
> > > > Perhaps he'd be opened to a patch that increases that.
> > > > 
> > > > On Mon, Aug 26, 2019 at 2:30 PM de Vandiere, Louis <
> > > > louis.devandiere@atos.net> wrote:
> > > > > Thanks Niels, I tried your suggestion. According to the
> > > > > documentation (
> > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinux.die.net%2Fman%2F8%2Fmkfs.xfs&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=HZbnVSzTKKCXpEv5JLgZKeEgQx5BPKeEs4SYZqRhhbk%3D&amp;reserved=0
> > > > > ), the maximum size for the inode is 2048 byte. So I set it
> > > > > to this value, and faced the exact same limitation. On the
> > > > > other hand, when I used setfacl -m on the XFS mounted disk, I
> > > > > did not face any limitation and I was able to set thousands
> > > > > of ACLs on a single file.
> > > > > 
> > > > > When I do a strace, I see two different types of ACL used
> > > > > when the system calls setxattr: system.posix_acl_default and
> > > > > system.nfsv4_acl. I tried to look for hardcoded limits
> > > > > associated with system.nfsv4_acl but I don't have much
> > > > > experience with C and linux kernel.
> > > > > 
> > > > > Thank you for your help.
> > > > > Best,
> > > > > Louis de Vandière
> > > > > 
> > > > > -----Original Message-----
> > > > > From: Niels de Vos <ndevos@redhat.com>
> > > > > Sent: Monday, August 26, 2019 11:46 AM
> > > > > To: de Vandiere, Louis <louis.devandiere@atos.net>
> > > > > Cc: linux-nfs@vger.kernel.org
> > > > > Subject: Re: Maximum Number of ACL on NFSv4
> > > > > 
> > > > > On Mon, Aug 26, 2019 at 02:53:05PM +0000, de Vandiere, Louis
> > > > > wrote:
> > > > > > Yes, I assume it's not very frequent to have hundreds of
> > > > > > NFSv4 ACLs. For compliance and organizational issue, we
> > > > > > cannot use groups efficiently to manage access to the
> > > > > > shares, so it's user-based and case by case.
> > > > > > 
> > > > > > My real goal is to be able to replicate some files to a new
> > > > > > NFSv4 server while preserving the ACLs. By using "cp -R --
> > > > > > preserve=all acl-folder/", I'm able to preserve the ACLs
> > > > > > when their number does not exceed 200, above it, I see the
> > > > > > "File too large" error while rsync does not work at all
> > > > > > (even in version 3.1.3). That's why I'm digging into this
> > > > > > and checking what possibly could go wrong.
> > > > > 
> > > > > You might be hitting a limit in the filesystem on the NFS
> > > > > server. The
> > > > > ACLs are stored in extended attributes. Depending on the
> > > > > filesystem,
> > > > > you may be able to configure larger inode sizes (or other
> > > > > storage for
> > > > > xattrs). With XFS this can be done with 'mkfs -t xfs -i
> > > > > size=.. ...',
> > > > > 
> > > > > HTH,
> > > > > Niels
> > > > > 
> > > > > 
> > > > > > Thank you.
> > > > > > Best,
> > > > > > Louis de Vandière
> > > > > > 
> > > > > > 
> > > > > > -----Original Message-----
> > > > > > From: Goetz, Patrick G <pgoetz@math.utexas.edu>
> > > > > > Sent: Monday, August 26, 2019 8:44 AM
> > > > > > To: de Vandiere, Louis <louis.devandiere@atos.net>;
> > > > > > linux-nfs@vger.kernel.org
> > > > > > Subject: Re: Maximum Number of ACL on NFSv4
> > > > > > 
> > > > > > I'm dying to know what the use case is for this, and why
> > > > > > you can't just do this with group permissions (unless
> > > > > > you're talking about hundreds of group ACLs).
> > > > > > 
> > > > > > On 8/23/19 5:31 PM, de Vandiere, Louis wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > I'm currently trying to apply hundreds of ACLs on file
> > > > > > > hosted on a NFSv4 server (nfs-utils-1.3.0-0.61.el7.x86_64 
> > > > > > > and nfs4-acl-tools.0.3.3-19.el7.x86_64). It appears that
> > > > > > > the limit I can apply is 207. After the limit is reached,
> > > > > > > the command "nfs4_setfacl -a" returned the error "Failed
> > > > > > > setxattr operation: File too large". The same problem
> > > > > > > happens if I use an ACL with more than 200 line in it. I
> > > > > > > did a little debugging session but I was not able to come
> > > > > > > up with an explanation on why I'm facing such an issue.
> > > > > > > 
> > > > > > > On the other hand, I can apply hundreds of ACLs on XFS
> > > > > > > without issue. Do you know if it could be a bug with the
> > > > > > > nfs4-acl-tools package?
> > > > > > > Thank you for your support.
> > > > > > > Best,
> > > > > > > Louis de Vandière
> > > > > > > > > This message is from an external sender. Learn more
> > > > > > > > > about why this <<
> > > > > > > > > matters at 
> > > > > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinks.utexas.edu%2Frtyclf&amp;data=02%7C01%7Clouis.devandiere%40atos.net%7Ce185f99cb3ad476fd39308d72a5bf6d5%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637024446785324973&amp;sdata=r345rqWN4GKT0mBmQmMTnaC%2FFEyUTidjBlGeAMRdEpA%3D&amp;reserved=0
> > > > > > > > > .                        <<
-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: Maximum Number of ACL on NFSv4

[bfields]

On Wed, Aug 28, 2019 at 08:25:16PM +0000, Trond Myklebust wrote:
> Umm... Don't forget that NFSv4 ACL aces are typically much larger than
> POSIX ACL aces because the user/group names are encoded as strings, not
> binary uids and gids.
> 
> IOW: The size of the RPC message is likely to be a lot larger than the
> resulting POSIX ACL...

Actually this limit is post-idmapping, but, yes, before NFSv4->Posix
mapping (complicated in itself), which is why I talked about having to
estimate.

More interested to hear what you think about whether we need a limit at
all.  Do we have any ideas how big is too big a number to pass to
kmalloc?  Or is it OK to just let anything through and let kmalloc fail?

--b.

Re: Maximum Number of ACL on NFSv4

[Trond Myklebust]

On Wed, 2019-08-28 at 17:06 -0400, bfields@fieldses.org wrote:
> On Wed, Aug 28, 2019 at 08:25:16PM +0000, Trond Myklebust wrote:
> > Umm... Don't forget that NFSv4 ACL aces are typically much larger
> > than
> > POSIX ACL aces because the user/group names are encoded as strings,
> > not
> > binary uids and gids.
> > 
> > IOW: The size of the RPC message is likely to be a lot larger than
> > the
> > resulting POSIX ACL...
> 
> Actually this limit is post-idmapping, but, yes, before NFSv4->Posix
> mapping (complicated in itself), which is why I talked about having
> to
> estimate.
> 
> More interested to hear what you think about whether we need a limit
> at
> all.  Do we have any ideas how big is too big a number to pass to
> kmalloc?  Or is it OK to just let anything through and let kmalloc
> fail?
> 

A NFSv4.x client is always required to respect the max request size as
negotiated during CREATE_SESSION, so there is an upper limit right
there. On Linux, the client will never try to negotiate a limit greater
than 1MB.


-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: Maximum Number of ACL on NFSv4

[bfields]

On Wed, Aug 28, 2019 at 09:26:20PM +0000, Trond Myklebust wrote:
> On Wed, 2019-08-28 at 17:06 -0400, bfields@fieldses.org wrote:
> > On Wed, Aug 28, 2019 at 08:25:16PM +0000, Trond Myklebust wrote:
> > > Umm... Don't forget that NFSv4 ACL aces are typically much larger
> > > than
> > > POSIX ACL aces because the user/group names are encoded as strings,
> > > not
> > > binary uids and gids.
> > > 
> > > IOW: The size of the RPC message is likely to be a lot larger than
> > > the
> > > resulting POSIX ACL...
> > 
> > Actually this limit is post-idmapping, but, yes, before NFSv4->Posix
> > mapping (complicated in itself), which is why I talked about having
> > to
> > estimate.
> > 
> > More interested to hear what you think about whether we need a limit
> > at
> > all.  Do we have any ideas how big is too big a number to pass to
> > kmalloc?  Or is it OK to just let anything through and let kmalloc
> > fail?
> > 
> 
> A NFSv4.x client is always required to respect the max request size as
> negotiated during CREATE_SESSION, so there is an upper limit right
> there. On Linux, the client will never try to negotiate a limit greater
> than 1MB.

The limit's actually a sanity check on the number of ACEs.  Since that's
what we're about to use in the kmalloc; in the ACL xdr decoding:

	if (nace > NFS4_ACL_MAX)
		return nfserr_fbig;

	*acl = svcxdr_tmpalloc(argp, nfs4_acl_bytes(nace));

Maybe the simplest thing is only to reject an nace value that'd be
impossible given the size of the rpc call.

--b.

Re: Maximum Number of ACL on NFSv4

[bfields]

On Wed, Aug 28, 2019 at 05:44:31PM -0400, bfields@fieldses.org wrote:
> On Wed, Aug 28, 2019 at 09:26:20PM +0000, Trond Myklebust wrote:
> > On Wed, 2019-08-28 at 17:06 -0400, bfields@fieldses.org wrote:
> > > On Wed, Aug 28, 2019 at 08:25:16PM +0000, Trond Myklebust wrote:
> > > > Umm... Don't forget that NFSv4 ACL aces are typically much larger
> > > > than
> > > > POSIX ACL aces because the user/group names are encoded as strings,
> > > > not
> > > > binary uids and gids.
> > > > 
> > > > IOW: The size of the RPC message is likely to be a lot larger than
> > > > the
> > > > resulting POSIX ACL...
> > > 
> > > Actually this limit is post-idmapping, but, yes, before NFSv4->Posix
> > > mapping (complicated in itself), which is why I talked about having
> > > to
> > > estimate.
> > > 
> > > More interested to hear what you think about whether we need a limit
> > > at
> > > all.  Do we have any ideas how big is too big a number to pass to
> > > kmalloc?  Or is it OK to just let anything through and let kmalloc
> > > fail?
> > > 
> > 
> > A NFSv4.x client is always required to respect the max request size as
> > negotiated during CREATE_SESSION, so there is an upper limit right
> > there. On Linux, the client will never try to negotiate a limit greater
> > than 1MB.
> 
> The limit's actually a sanity check on the number of ACEs.  Since that's
> what we're about to use in the kmalloc; in the ACL xdr decoding:
> 
> 	if (nace > NFS4_ACL_MAX)
> 		return nfserr_fbig;
> 
> 	*acl = svcxdr_tmpalloc(argp, nfs4_acl_bytes(nace));
> 
> Maybe the simplest thing is only to reject an nace value that'd be
> impossible given the size of the rpc call.

Or I guess we could realloc as necessary as we actually read the
entries.

But the below seems simplest.  It still provides some check on the ace
count, but it'll ensure we're no longer imposing artificial limits not
already required by the session or the filesystem.

struct nfs4_ace is 20 bytes on my system so in practice this is limiting
that kmalloc to about a megabyte.

--b.

diff --git a/fs/nfsd/acl.h b/fs/nfsd/acl.h
index 4cd7c69a6cb9..ba14d2f4b64f 100644
--- a/fs/nfsd/acl.h
+++ b/fs/nfsd/acl.h
@@ -39,14 +39,6 @@ struct nfs4_acl;
 struct svc_fh;
 struct svc_rqst;
 
-/*
- * Maximum ACL we'll accept from a client; chosen (somewhat
- * arbitrarily) so that kmalloc'ing the ACL shouldn't require a
- * high-order allocation.  This allows 204 ACEs on x86_64:
- */
-#define NFS4_ACL_MAX ((PAGE_SIZE - sizeof(struct nfs4_acl)) \
-			/ sizeof(struct nfs4_ace))
-
 int nfs4_acl_bytes(int entries);
 int nfs4_acl_get_whotype(char *, u32);
 __be32 nfs4_acl_write_who(struct xdr_stream *xdr, int who);
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 565d2169902c..c1fc2641e3e7 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -204,6 +204,13 @@ static __be32 *read_buf(struct nfsd4_compoundargs *argp, u32 nbytes)
 	return p;
 }
 
+static unsigned int compoundargs_bytes_left(struct nfsd4_compoundargs *argp)
+{
+	unsigned int this = (char *)argp->end - (char *)argp->p;
+
+	return this + argp->pagelen;
+}
+
 static int zero_clientid(clientid_t *clid)
 {
 	return (clid->cl_boot == 0) && (clid->cl_id == 0);
@@ -348,7 +355,12 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval,
 		READ_BUF(4); len += 4;
 		nace = be32_to_cpup(p++);
 
-		if (nace > NFS4_ACL_MAX)
+		if (nace > compoundargs_bytes_left(argp)/20)
+			/*
+			 * Even with 4-byte names there wouldn't be
+			 * space for that many aces; something fishy is
+			 * going on:
+			 */
 			return nfserr_fbig;
 
 		*acl = svcxdr_tmpalloc(argp, nfs4_acl_bytes(nace));