NFSv4 server ignores local filesystem's POSIX ACL

NFSv4 server ignores local filesystem's POSIX ACL

[Jiri Horky]

Hi all,

we use NFSv4 with Kerberos and a custom idmap mapping plugin. The 
mapping is configured in a way that all principals that are not 
explicitly defined are mapped to nobody/nogroup on a server. Recently, 
the kerberos infrastructure within our organization expanded by 
crossrealming with other parties which should not be allowed to use our 
NFSv4 mounts.
It is my understanding that everybody who is able to authenticate 
against the used kerberos infrastructure can mount the filesystems but 
nonauthorized users will be mapped to user nobody/nogroup and according 
to server's filesystem rights can do other actions. Now, I would like to 
set deny ACL for user nobody to the server's /exports directory to 
restrict nobody user access. But it seems this ACL is ignored. In fact, 
local POSIX ACL's on any directory seems to ignored:

SERVER:
root@store4 /exports # mkdir local_tmp
root@store4 /exports # chmod 777 local_tmp/
root@store4 /exports # setfacl -m u:nobody:--- /exports/local_tmp/
root@store4 /exports # getfacl /exports/local_tmp/
getfacl: Removing leading '/' from absolute path names
# file: exports/local_tmp/
# owner: root
# group: root
user::rwx
user:nobody:---
group::rwx
mask::rwx
other::rwx
root@store4 /exports # su nobody -c "touch /exports/local_tmp/filelocal"
touch: cannot touch `/exports/local_tmp/filelocal': Permission denied

so far so good, now on a client:

CLIENT:
metex-1:~# mount -t nfs4 -o sec=krb5 store4.du1.cesnet.cz:/ /mnt
metex-1:~# touch /mnt/local_tmp/a
metex-1:~# ls -l /mnt/local_tmp/a
-rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /mnt/local_tmp/a

and on the SERVER again:
root@store4 /exports # ls -l /exports/local_tmp/a
-rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /exports/local_tmp/a

So the ACL is ignored when accessing through NFS. Is it the expected 
behavior and I am just doing something terribly wrong?

Some more info about server:

root@store4 /exports # cat /etc/exports
/exports 
*(sec=krb5:krb5i:krb5p,rw,fsid=0,sync,no_subtree_check,no_root_squash,insecure,crossmnt)
root@store4 /exports # uname -a
Linux fe4 2.6.32.59-0.7.1.du2-default #1 SMP 2012-07-13 15:50:56 +0200 
x86_64 x86_64 x86_64 GNU/Linux
OS is: SLES 11 SP1

root@store4 /exports # modinfo nfsd
filename: /lib/modules/2.6.32.59-0.7.1.du2-default/kernel/fs/nfsd/nfsd.ko
license:        GPL
author:         Olaf Kirch <okir@monad.swb.de>
srcversion:     74D3604622B7912E7C96E03
depends:        auth_rpcgss,sunrpc,lockd,exportfs,nfs_acl
supported:      yes
vermagic:       2.6.32.59-0.7.1.du2-default SMP mod_unload modversions

Our intention is  simply to force nobody users out of our NFS servers.

Regards
Jiri Horky

Re: NFSv4 server ignores local filesystem's POSIX ACL

[J. Bruce Fields]

On Mon, Feb 11, 2013 at 04:29:38PM +0100, Jiri Horky wrote:
> Hi all,
> 
> we use NFSv4 with Kerberos and a custom idmap mapping plugin. The
> mapping is configured in a way that all principals that are not
> explicitly defined are mapped to nobody/nogroup on a server.
> Recently, the kerberos infrastructure within our organization
> expanded by crossrealming with other parties which should not be
> allowed to use our NFSv4 mounts.
> It is my understanding that everybody who is able to authenticate
> against the used kerberos infrastructure can mount the filesystems
> but nonauthorized users will be mapped to user nobody/nogroup and
> according to server's filesystem rights can do other actions. Now, I
> would like to set deny ACL for user nobody to the server's /exports
> directory to restrict nobody user access. But it seems this ACL is
> ignored. In fact, local POSIX ACL's on any directory seems to
> ignored:
> 
> SERVER:
> root@store4 /exports # mkdir local_tmp
> root@store4 /exports # chmod 777 local_tmp/
> root@store4 /exports # setfacl -m u:nobody:--- /exports/local_tmp/
> root@store4 /exports # getfacl /exports/local_tmp/
> getfacl: Removing leading '/' from absolute path names
> # file: exports/local_tmp/
> # owner: root
> # group: root
> user::rwx
> user:nobody:---
> group::rwx
> mask::rwx
> other::rwx
> root@store4 /exports # su nobody -c "touch /exports/local_tmp/filelocal"
> touch: cannot touch `/exports/local_tmp/filelocal': Permission denied
> 
> so far so good, now on a client:
> 
> CLIENT:
> metex-1:~# mount -t nfs4 -o sec=krb5 store4.du1.cesnet.cz:/ /mnt
> metex-1:~# touch /mnt/local_tmp/a
> metex-1:~# ls -l /mnt/local_tmp/a
> -rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /mnt/local_tmp/a
> 
> and on the SERVER again:
> root@store4 /exports # ls -l /exports/local_tmp/a
> -rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /exports/local_tmp/a
> 
> So the ACL is ignored when accessing through NFS. Is it the expected
> behavior and I am just doing something terribly wrong?

That looks like a bug, but I'm at a loss to explain how it could have
happened.  A network trace (tcpdump -s0 -wtmp.pcap, then send me
tmp.pcap) taken during the file creation above might be interesting.

> Some more info about server:
> 
> root@store4 /exports # cat /etc/exports
> /exports *(sec=krb5:krb5i:krb5p,rw,fsid=0,sync,no_subtree_check,no_root_squash,insecure,crossmnt)
> root@store4 /exports # uname -a
> Linux fe4 2.6.32.59-0.7.1.du2-default #1 SMP 2012-07-13 15:50:56
> +0200 x86_64 x86_64 x86_64 GNU/Linux
> OS is: SLES 11 SP1
> 
> root@store4 /exports # modinfo nfsd
> filename: /lib/modules/2.6.32.59-0.7.1.du2-default/kernel/fs/nfsd/nfsd.ko
> license:        GPL
> author:         Olaf Kirch <okir@monad.swb.de>
> srcversion:     74D3604622B7912E7C96E03
> depends:        auth_rpcgss,sunrpc,lockd,exportfs,nfs_acl
> supported:      yes
> vermagic:       2.6.32.59-0.7.1.du2-default SMP mod_unload modversions
> 
> Our intention is  simply to force nobody users out of our NFS servers.

It might be better to teach rpc.svcgssd to fail authentications from any
users outside your local domain.  I'm not sure how to do that.

--b.

RE: NFSv4 server ignores local filesystem's POSIX ACL

[Myklebust, Trond]

> -----Original Message-----
> From: linux-nfs-owner@vger.kernel.org [mailto:linux-nfs-
> owner@vger.kernel.org] On Behalf Of J. Bruce Fields
> Sent: Monday, February 11, 2013 1:22 PM
> To: Jiri Horky
> Cc: linux-nfs@vger.kernel.org
> Subject: Re: NFSv4 server ignores local filesystem's POSIX ACL
> 
> On Mon, Feb 11, 2013 at 04:29:38PM +0100, Jiri Horky wrote:
> > Hi all,
> >
> > we use NFSv4 with Kerberos and a custom idmap mapping plugin. The
> > mapping is configured in a way that all principals that are not
> > explicitly defined are mapped to nobody/nogroup on a server.
> > Recently, the kerberos infrastructure within our organization expanded
> > by crossrealming with other parties which should not be allowed to use
> > our NFSv4 mounts.
> > It is my understanding that everybody who is able to authenticate
> > against the used kerberos infrastructure can mount the filesystems but
> > nonauthorized users will be mapped to user nobody/nogroup and
> > according to server's filesystem rights can do other actions. Now, I
> > would like to set deny ACL for user nobody to the server's /exports
> > directory to restrict nobody user access. But it seems this ACL is
> > ignored. In fact, local POSIX ACL's on any directory seems to
> > ignored:
> >
> > SERVER:
> > root@store4 /exports # mkdir local_tmp
> > root@store4 /exports # chmod 777 local_tmp/
> > root@store4 /exports # setfacl -m u:nobody:--- /exports/local_tmp/
> > root@store4 /exports # getfacl /exports/local_tmp/
> > getfacl: Removing leading '/' from absolute path names # file:
> > exports/local_tmp/ # owner: root # group: root user::rwx
> > user:nobody:---
> > group::rwx
> > mask::rwx
> > other::rwx
> > root@store4 /exports # su nobody -c "touch /exports/local_tmp/filelocal"
> > touch: cannot touch `/exports/local_tmp/filelocal': Permission denied
> >
> > so far so good, now on a client:
> >
> > CLIENT:
> > metex-1:~# mount -t nfs4 -o sec=krb5 store4.du1.cesnet.cz:/ /mnt
> > metex-1:~# touch /mnt/local_tmp/a metex-1:~# ls -l /mnt/local_tmp/a
> > -rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /mnt/local_tmp/a
> >
> > and on the SERVER again:
> > root@store4 /exports # ls -l /exports/local_tmp/a
> > -rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /exports/local_tmp/a
> >
> > So the ACL is ignored when accessing through NFS. Is it the expected
> > behavior and I am just doing something terribly wrong?
> 
> That looks like a bug, but I'm at a loss to explain how it could have happened.
> A network trace (tcpdump -s0 -wtmp.pcap, then send me
> tmp.pcap) taken during the file creation above might be interesting.

If everyone is being squashed to user, nobody and the 'nobody' user owns the file, isn't that just expected behaviour? NFSv3 servers are always supposed to allow reads and writes from the owner, since the protocol itself has no stateful equivalent of open("foo", O_RDWR|O_CREAT, 0);

> 
> > Some more info about server:
> >
> > root@store4 /exports # cat /etc/exports /exports
> >
> *(sec=krb5:krb5i:krb5p,rw,fsid=0,sync,no_subtree_check,no_root_squash,
> > insecure,crossmnt)
> > root@store4 /exports # uname -a
> > Linux fe4 2.6.32.59-0.7.1.du2-default #1 SMP 2012-07-13 15:50:56
> > +0200 x86_64 x86_64 x86_64 GNU/Linux
> > OS is: SLES 11 SP1
> >
> > root@store4 /exports # modinfo nfsd
> > filename: /lib/modules/2.6.32.59-0.7.1.du2-default/kernel/fs/nfsd/nfsd.ko
> > license:        GPL
> > author:         Olaf Kirch <okir@monad.swb.de>
> > srcversion:     74D3604622B7912E7C96E03
> > depends:        auth_rpcgss,sunrpc,lockd,exportfs,nfs_acl
> > supported:      yes
> > vermagic:       2.6.32.59-0.7.1.du2-default SMP mod_unload modversions
> >
> > Our intention is  simply to force nobody users out of our NFS servers.
> 
> It might be better to teach rpc.svcgssd to fail authentications from any users
> outside your local domain.  I'm not sure how to do that.
> 
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the
> body of a message to majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html

Re: NFSv4 server ignores local filesystem's POSIX ACL

[J. Bruce Fields]

On Mon, Feb 11, 2013 at 06:32:26PM +0000, Myklebust, Trond wrote:
> > -----Original Message-----
> > From: linux-nfs-owner@vger.kernel.org [mailto:linux-nfs-
> > owner@vger.kernel.org] On Behalf Of J. Bruce Fields
> > Sent: Monday, February 11, 2013 1:22 PM
> > To: Jiri Horky
> > Cc: linux-nfs@vger.kernel.org
> > Subject: Re: NFSv4 server ignores local filesystem's POSIX ACL
> > 
> > On Mon, Feb 11, 2013 at 04:29:38PM +0100, Jiri Horky wrote:
> > > Hi all,
> > >
> > > we use NFSv4 with Kerberos and a custom idmap mapping plugin. The
> > > mapping is configured in a way that all principals that are not
> > > explicitly defined are mapped to nobody/nogroup on a server.
> > > Recently, the kerberos infrastructure within our organization expanded
> > > by crossrealming with other parties which should not be allowed to use
> > > our NFSv4 mounts.
> > > It is my understanding that everybody who is able to authenticate
> > > against the used kerberos infrastructure can mount the filesystems but
> > > nonauthorized users will be mapped to user nobody/nogroup and
> > > according to server's filesystem rights can do other actions. Now, I
> > > would like to set deny ACL for user nobody to the server's /exports
> > > directory to restrict nobody user access. But it seems this ACL is
> > > ignored. In fact, local POSIX ACL's on any directory seems to
> > > ignored:
> > >
> > > SERVER:
> > > root@store4 /exports # mkdir local_tmp
> > > root@store4 /exports # chmod 777 local_tmp/
> > > root@store4 /exports # setfacl -m u:nobody:--- /exports/local_tmp/
> > > root@store4 /exports # getfacl /exports/local_tmp/
> > > getfacl: Removing leading '/' from absolute path names # file:
> > > exports/local_tmp/ # owner: root # group: root user::rwx
> > > user:nobody:---
> > > group::rwx
> > > mask::rwx
> > > other::rwx
> > > root@store4 /exports # su nobody -c "touch /exports/local_tmp/filelocal"
> > > touch: cannot touch `/exports/local_tmp/filelocal': Permission denied
> > >
> > > so far so good, now on a client:
> > >
> > > CLIENT:
> > > metex-1:~# mount -t nfs4 -o sec=krb5 store4.du1.cesnet.cz:/ /mnt
> > > metex-1:~# touch /mnt/local_tmp/a metex-1:~# ls -l /mnt/local_tmp/a
> > > -rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /mnt/local_tmp/a
> > >
> > > and on the SERVER again:
> > > root@store4 /exports # ls -l /exports/local_tmp/a
> > > -rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /exports/local_tmp/a
> > >
> > > So the ACL is ignored when accessing through NFS. Is it the expected
> > > behavior and I am just doing something terribly wrong?
> > 
> > That looks like a bug, but I'm at a loss to explain how it could have happened.
> > A network trace (tcpdump -s0 -wtmp.pcap, then send me
> > tmp.pcap) taken during the file creation above might be interesting.
> 
> If everyone is being squashed to user, nobody and the 'nobody' user owns the file, isn't that just expected behaviour? NFSv3 servers are always supposed to allow reads and writes from the owner, since the protocol itself has no stateful equivalent of open("foo", O_RDWR|O_CREAT, 0);

Oops, I was assuming the second touch above was doing a create--based on
the timestamps in the two ls's, you're right the second touch wasn't
creating.

So, agreed, this looks normal.

--b.

> 
> > 
> > > Some more info about server:
> > >
> > > root@store4 /exports # cat /etc/exports /exports
> > >
> > *(sec=krb5:krb5i:krb5p,rw,fsid=0,sync,no_subtree_check,no_root_squash,
> > > insecure,crossmnt)
> > > root@store4 /exports # uname -a
> > > Linux fe4 2.6.32.59-0.7.1.du2-default #1 SMP 2012-07-13 15:50:56
> > > +0200 x86_64 x86_64 x86_64 GNU/Linux
> > > OS is: SLES 11 SP1
> > >
> > > root@store4 /exports # modinfo nfsd
> > > filename: /lib/modules/2.6.32.59-0.7.1.du2-default/kernel/fs/nfsd/nfsd.ko
> > > license:        GPL
> > > author:         Olaf Kirch <okir@monad.swb.de>
> > > srcversion:     74D3604622B7912E7C96E03
> > > depends:        auth_rpcgss,sunrpc,lockd,exportfs,nfs_acl
> > > supported:      yes
> > > vermagic:       2.6.32.59-0.7.1.du2-default SMP mod_unload modversions
> > >
> > > Our intention is  simply to force nobody users out of our NFS servers.
> > 
> > It might be better to teach rpc.svcgssd to fail authentications from any users
> > outside your local domain.  I'm not sure how to do that.
> > 
> > --b.
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the
> > body of a message to majordomo@vger.kernel.org More majordomo info at
> > http://vger.kernel.org/majordomo-info.html

Re: NFSv4 server ignores local filesystem's POSIX ACL

[Jiri Horky]

Hi all,
On 02/11/2013 07:49 PM, J. Bruce Fields wrote:
> On Mon, Feb 11, 2013 at 06:32:26PM +0000, Myklebust, Trond wrote:
>>> -----Original Message-----
>>> From: linux-nfs-owner@vger.kernel.org [mailto:linux-nfs-
>>> owner@vger.kernel.org] On Behalf Of J. Bruce Fields
>>> Sent: Monday, February 11, 2013 1:22 PM
>>> To: Jiri Horky
>>> Cc: linux-nfs@vger.kernel.org
>>> Subject: Re: NFSv4 server ignores local filesystem's POSIX ACL
>>>
>>> On Mon, Feb 11, 2013 at 04:29:38PM +0100, Jiri Horky wrote:
>>>> Hi all,
>>>>
>>>> we use NFSv4 with Kerberos and a custom idmap mapping plugin. The
>>>> mapping is configured in a way that all principals that are not
>>>> explicitly defined are mapped to nobody/nogroup on a server.
>>>> Recently, the kerberos infrastructure within our organization expanded
>>>> by crossrealming with other parties which should not be allowed to use
>>>> our NFSv4 mounts.
>>>> It is my understanding that everybody who is able to authenticate
>>>> against the used kerberos infrastructure can mount the filesystems but
>>>> nonauthorized users will be mapped to user nobody/nogroup and
>>>> according to server's filesystem rights can do other actions. Now, I
>>>> would like to set deny ACL for user nobody to the server's /exports
>>>> directory to restrict nobody user access. But it seems this ACL is
>>>> ignored. In fact, local POSIX ACL's on any directory seems to
>>>> ignored:
>>>>
>>>> SERVER:
>>>> root@store4 /exports # mkdir local_tmp
>>>> root@store4 /exports # chmod 777 local_tmp/
>>>> root@store4 /exports # setfacl -m u:nobody:--- /exports/local_tmp/
>>>> root@store4 /exports # getfacl /exports/local_tmp/
>>>> getfacl: Removing leading '/' from absolute path names # file:
>>>> exports/local_tmp/ # owner: root # group: root user::rwx
>>>> user:nobody:---
>>>> group::rwx
>>>> mask::rwx
>>>> other::rwx
>>>> root@store4 /exports # su nobody -c "touch /exports/local_tmp/filelocal"
>>>> touch: cannot touch `/exports/local_tmp/filelocal': Permission denied
>>>>
>>>> so far so good, now on a client:
>>>>
>>>> CLIENT:
>>>> metex-1:~# mount -t nfs4 -o sec=krb5 store4.du1.cesnet.cz:/ /mnt
>>>> metex-1:~# touch /mnt/local_tmp/a metex-1:~# ls -l /mnt/local_tmp/a
>>>> -rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /mnt/local_tmp/a
>>>>
>>>> and on the SERVER again:
>>>> root@store4 /exports # ls -l /exports/local_tmp/a
>>>> -rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /exports/local_tmp/a
>>>>
>>>> So the ACL is ignored when accessing through NFS. Is it the expected
>>>> behavior and I am just doing something terribly wrong?
>>> That looks like a bug, but I'm at a loss to explain how it could have happened.
>>> A network trace (tcpdump -s0 -wtmp.pcap, then send me
>>> tmp.pcap) taken during the file creation above might be interesting.
>> If everyone is being squashed to user, nobody and the 'nobody' user owns the file, isn't that just expected behaviour? NFSv3 servers are always supposed to allow reads and writes from the owner, since the protocol itself has no stateful equivalent of open("foo", O_RDWR|O_CREAT, 0);
> Oops, I was assuming the second touch above was doing a create--based on
> the timestamps in the two ls's, you're right the second touch wasn't
> creating.
>
> So, agreed, this looks normal.
I am a bit lost here. The second  touch (the one from nfs client) 
actually was creating the file. What I wanted to show is that creating a 
file on the server itself failed with permission denied whereas through 
NFS, it worked just fine and created the file as user nobody on the 
server, which should not be possible as nobody user should not be able 
to enter the directory. Are you pointing to the fact that on the client, 
the "local_tmp" directory seems to be owned by user nobody?

I have sent you privately the requested tcpdump trace.

Thank you.
Jiri Horky

Re: NFSv4 server ignores local filesystem's POSIX ACL

[J. Bruce Fields]

On Mon, Feb 11, 2013 at 09:40:12PM +0100, Jiri Horky wrote:
> Hi all,
> On 02/11/2013 07:49 PM, J. Bruce Fields wrote:
> >On Mon, Feb 11, 2013 at 06:32:26PM +0000, Myklebust, Trond wrote:
> >>>-----Original Message-----
> >>>From: linux-nfs-owner@vger.kernel.org [mailto:linux-nfs-
> >>>owner@vger.kernel.org] On Behalf Of J. Bruce Fields
> >>>Sent: Monday, February 11, 2013 1:22 PM
> >>>To: Jiri Horky
> >>>Cc: linux-nfs@vger.kernel.org
> >>>Subject: Re: NFSv4 server ignores local filesystem's POSIX ACL
> >>>
> >>>On Mon, Feb 11, 2013 at 04:29:38PM +0100, Jiri Horky wrote:
> >>>>Hi all,
> >>>>
> >>>>we use NFSv4 with Kerberos and a custom idmap mapping plugin. The
> >>>>mapping is configured in a way that all principals that are not
> >>>>explicitly defined are mapped to nobody/nogroup on a server.
> >>>>Recently, the kerberos infrastructure within our organization expanded
> >>>>by crossrealming with other parties which should not be allowed to use
> >>>>our NFSv4 mounts.
> >>>>It is my understanding that everybody who is able to authenticate
> >>>>against the used kerberos infrastructure can mount the filesystems but
> >>>>nonauthorized users will be mapped to user nobody/nogroup and
> >>>>according to server's filesystem rights can do other actions. Now, I
> >>>>would like to set deny ACL for user nobody to the server's /exports
> >>>>directory to restrict nobody user access. But it seems this ACL is
> >>>>ignored. In fact, local POSIX ACL's on any directory seems to
> >>>>ignored:
> >>>>
> >>>>SERVER:
> >>>>root@store4 /exports # mkdir local_tmp
> >>>>root@store4 /exports # chmod 777 local_tmp/
> >>>>root@store4 /exports # setfacl -m u:nobody:--- /exports/local_tmp/
> >>>>root@store4 /exports # getfacl /exports/local_tmp/
> >>>>getfacl: Removing leading '/' from absolute path names # file:
> >>>>exports/local_tmp/ # owner: root # group: root user::rwx
> >>>>user:nobody:---
> >>>>group::rwx
> >>>>mask::rwx
> >>>>other::rwx
> >>>>root@store4 /exports # su nobody -c "touch /exports/local_tmp/filelocal"
> >>>>touch: cannot touch `/exports/local_tmp/filelocal': Permission denied
> >>>>
> >>>>so far so good, now on a client:
> >>>>
> >>>>CLIENT:
> >>>>metex-1:~# mount -t nfs4 -o sec=krb5 store4.du1.cesnet.cz:/ /mnt
> >>>>metex-1:~# touch /mnt/local_tmp/a metex-1:~# ls -l /mnt/local_tmp/a
> >>>>-rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /mnt/local_tmp/a
> >>>>
> >>>>and on the SERVER again:
> >>>>root@store4 /exports # ls -l /exports/local_tmp/a
> >>>>-rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /exports/local_tmp/a
> >>>>
> >>>>So the ACL is ignored when accessing through NFS. Is it the expected
> >>>>behavior and I am just doing something terribly wrong?
> >>>That looks like a bug, but I'm at a loss to explain how it could have happened.
> >>>A network trace (tcpdump -s0 -wtmp.pcap, then send me
> >>>tmp.pcap) taken during the file creation above might be interesting.
> >>If everyone is being squashed to user, nobody and the 'nobody' user owns the file, isn't that just expected behaviour? NFSv3 servers are always supposed to allow reads and writes from the owner, since the protocol itself has no stateful equivalent of open("foo", O_RDWR|O_CREAT, 0);
> >Oops, I was assuming the second touch above was doing a create--based on
> >the timestamps in the two ls's, you're right the second touch wasn't
> >creating.
> >
> >So, agreed, this looks normal.
> I am a bit lost here. The second  touch (the one from nfs client)
> actually was creating the file.

Oops, OK.

> What I wanted to show is that
> creating a file on the server itself failed with permission denied
> whereas through NFS, it worked just fine and created the file as
> user nobody on the server, which should not be possible as nobody
> user should not be able to enter the directory. Are you pointing to
> the fact that on the client, the "local_tmp" directory seems to be
> owned by user nobody?

That shouldn't matter (assuming it's not actually owned by "nobody" on
the server.)

--b.

> 
> I have sent you privately the requested tcpdump trace.
> 
> Thank you.
> Jiri Horky
> 
> 
> 

Re: NFSv4 server ignores local filesystem's POSIX ACL

[J. Bruce Fields]

On Mon, Feb 11, 2013 at 04:01:33PM -0500, J. Bruce Fields wrote:
> On Mon, Feb 11, 2013 at 09:40:12PM +0100, Jiri Horky wrote:
> > Hi all,
> > On 02/11/2013 07:49 PM, J. Bruce Fields wrote:
> > >On Mon, Feb 11, 2013 at 06:32:26PM +0000, Myklebust, Trond wrote:
> > >>>-----Original Message-----
> > >>>From: linux-nfs-owner@vger.kernel.org [mailto:linux-nfs-
> > >>>owner@vger.kernel.org] On Behalf Of J. Bruce Fields
> > >>>Sent: Monday, February 11, 2013 1:22 PM
> > >>>To: Jiri Horky
> > >>>Cc: linux-nfs@vger.kernel.org
> > >>>Subject: Re: NFSv4 server ignores local filesystem's POSIX ACL
> > >>>
> > >>>On Mon, Feb 11, 2013 at 04:29:38PM +0100, Jiri Horky wrote:
> > >>>>Hi all,
> > >>>>
> > >>>>we use NFSv4 with Kerberos and a custom idmap mapping plugin. The
> > >>>>mapping is configured in a way that all principals that are not
> > >>>>explicitly defined are mapped to nobody/nogroup on a server.
> > >>>>Recently, the kerberos infrastructure within our organization expanded
> > >>>>by crossrealming with other parties which should not be allowed to use
> > >>>>our NFSv4 mounts.
> > >>>>It is my understanding that everybody who is able to authenticate
> > >>>>against the used kerberos infrastructure can mount the filesystems but
> > >>>>nonauthorized users will be mapped to user nobody/nogroup and
> > >>>>according to server's filesystem rights can do other actions. Now, I
> > >>>>would like to set deny ACL for user nobody to the server's /exports
> > >>>>directory to restrict nobody user access. But it seems this ACL is
> > >>>>ignored. In fact, local POSIX ACL's on any directory seems to
> > >>>>ignored:
> > >>>>
> > >>>>SERVER:
> > >>>>root@store4 /exports # mkdir local_tmp
> > >>>>root@store4 /exports # chmod 777 local_tmp/
> > >>>>root@store4 /exports # setfacl -m u:nobody:--- /exports/local_tmp/
> > >>>>root@store4 /exports # getfacl /exports/local_tmp/
> > >>>>getfacl: Removing leading '/' from absolute path names # file:
> > >>>>exports/local_tmp/ # owner: root # group: root user::rwx
> > >>>>user:nobody:---
> > >>>>group::rwx
> > >>>>mask::rwx
> > >>>>other::rwx
> > >>>>root@store4 /exports # su nobody -c "touch /exports/local_tmp/filelocal"
> > >>>>touch: cannot touch `/exports/local_tmp/filelocal': Permission denied
> > >>>>
> > >>>>so far so good, now on a client:
> > >>>>
> > >>>>CLIENT:
> > >>>>metex-1:~# mount -t nfs4 -o sec=krb5 store4.du1.cesnet.cz:/ /mnt
> > >>>>metex-1:~# touch /mnt/local_tmp/a metex-1:~# ls -l /mnt/local_tmp/a
> > >>>>-rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /mnt/local_tmp/a
> > >>>>
> > >>>>and on the SERVER again:
> > >>>>root@store4 /exports # ls -l /exports/local_tmp/a
> > >>>>-rw-r--r-- 1 nobody nogroup 0 Feb 11 15:23 /exports/local_tmp/a
> > >>>>
> > >>>>So the ACL is ignored when accessing through NFS. Is it the expected
> > >>>>behavior and I am just doing something terribly wrong?
> > >>>That looks like a bug, but I'm at a loss to explain how it could have happened.
> > >>>A network trace (tcpdump -s0 -wtmp.pcap, then send me
> > >>>tmp.pcap) taken during the file creation above might be interesting.
> > >>If everyone is being squashed to user, nobody and the 'nobody' user owns the file, isn't that just expected behaviour? NFSv3 servers are always supposed to allow reads and writes from the owner, since the protocol itself has no stateful equivalent of open("foo", O_RDWR|O_CREAT, 0);
> > >Oops, I was assuming the second touch above was doing a create--based on
> > >the timestamps in the two ls's, you're right the second touch wasn't
> > >creating.
> > >
> > >So, agreed, this looks normal.
> > I am a bit lost here. The second  touch (the one from nfs client)
> > actually was creating the file.
> 
> Oops, OK.
> 
> > What I wanted to show is that
> > creating a file on the server itself failed with permission denied
> > whereas through NFS, it worked just fine and created the file as
> > user nobody on the server, which should not be possible as nobody
> > user should not be able to enter the directory. Are you pointing to
> > the fact that on the client, the "local_tmp" directory seems to be
> > owned by user nobody?
> 
> That shouldn't matter (assuming it's not actually owned by "nobody" on
> the server.)

Beats me.  The network trace does indeed seem to show a succesful create
(and a preceding access op that allows access that I would have expected
to fail).

I don't see how that could have happened--the server doesn't actually
enforce the ACL itself, it depends on common vfs code for that, so
there's probably something obvious we're overlooking.  You're positive
the file's being created in the same directory that you set that acl on?

--b.

Re: NFSv4 server ignores local filesystem's POSIX ACL

[Jiri Horky]

Hi,
On 02/11/2013 10:41 PM, J. Bruce Fields wrote:
> On Mon, Feb 11, 2013 at 04:01:33PM -0500, J. Bruce Fields wrote:
>> That shouldn't matter (assuming it's not actually owned by "nobody" on
>> the server.)
> Beats me.  The network trace does indeed seem to show a succesful create
> (and a preceding access op that allows access that I would have expected
> to fail).
>
> I don't see how that could have happened--the server doesn't actually
> enforce the ACL itself, it depends on common vfs code for that, so
> there's probably something obvious we're overlooking.  You're positive
> the file's being created in the same directory that you set that acl on?
>
> --b.
we tracked the cause down to the "enhanced xfs" module from SGI 
(sgi-enhancedxfs-kmp-default-2.5_2.6.32.12_0.7-sgi250rp13.sles11). With 
a standard xfs module 
(sgi-xfs-kmp-default-6.5.0.14_2.6.32.12_0.7-sgi12111921) or on a ext3 
file systems, the server respects ACL correctly. Seems like the 
enhancement went into wrong direction :) Lets file a bug against the vendor.

Cheers
Jiri Horky

Re: NFSv4 server ignores local filesystem's POSIX ACL

[J. Bruce Fields]

On Tue, Feb 12, 2013 at 03:02:56PM +0100, Jiri Horky wrote:
> Hi,
> On 02/11/2013 10:41 PM, J. Bruce Fields wrote:
> >On Mon, Feb 11, 2013 at 04:01:33PM -0500, J. Bruce Fields wrote:
> >>That shouldn't matter (assuming it's not actually owned by "nobody" on
> >>the server.)
> >Beats me.  The network trace does indeed seem to show a succesful create
> >(and a preceding access op that allows access that I would have expected
> >to fail).
> >
> >I don't see how that could have happened--the server doesn't actually
> >enforce the ACL itself, it depends on common vfs code for that, so
> >there's probably something obvious we're overlooking.  You're positive
> >the file's being created in the same directory that you set that acl on?
> >
> >--b.
> we tracked the cause down to the "enhanced xfs" module from SGI
> (sgi-enhancedxfs-kmp-default-2.5_2.6.32.12_0.7-sgi250rp13.sles11).
> With a standard xfs module
> (sgi-xfs-kmp-default-6.5.0.14_2.6.32.12_0.7-sgi12111921) or on a
> ext3 file systems, the server respects ACL correctly. Seems like the
> enhancement went into wrong direction :) Lets file a bug against the
> vendor.

OK, thanks for the followup.  That's extremely weird.

--b.