nfs4_acl restricts copy_up in overlayfs

nfs4_acl restricts copy_up in overlayfs

[Goldwyn Rodrigues]

While mounting overlayfs with NFS as a lower directory and a local
filesystem as an upper layer leads to copy_up failures because NFS4 has
an extra system.nfs4_acl which cannot be copied up. This has been
discussed before [1] and [2] with the suggestion that nfs4_acl is
derived from posix_acls or just inode->i_mode *most* of the times and
hence it can be mapped back.

The problem is NFS client knows nothing about nfs4_acl and it is decoded
in nfs4-acl-tools. Even if we make nfs client capable of understand
nfs4_acl xattr, can it be used to perform ACL's for the system. AFAIU,
it is uses user/group names as opposed uid/gid to perform id mapping.
Can the client map it back to user names and derive if it is just an
replica of inode's i_mode?

The idea is to suppress nfs4_acl if it is the same as inode's i_mode.
This means nfs4-acl-tools/nfs4_getacl would give no results when
requesting for ACLs. This would break existing applications if they
expect some output from nfs4_getfacl.

Is there a better way to identify if nfs4_acl is just a representation
of i_mode at the client end and can be safely ignored during an
overlayfs copy_up? Can we include a flag for this?


[1] https://www.spinics.net/lists/linux-nfs/msg61045.html
[2] https://www.spinics.net/lists/linux-unionfs/msg04736.html

-- 
Goldwyn

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

On Tue, 2018-05-29 at 15:32 -0500, Goldwyn Rodrigues wrote:
> While mounting overlayfs with NFS as a lower directory and a local
> filesystem as an upper layer leads to copy_up failures because NFS4
> has
> an extra system.nfs4_acl which cannot be copied up. This has been
> discussed before [1] and [2] with the suggestion that nfs4_acl is
> derived from posix_acls or just inode->i_mode *most* of the times and
> hence it can be mapped back.
> 
> The problem is NFS client knows nothing about nfs4_acl and it is
> decoded
> in nfs4-acl-tools. Even if we make nfs client capable of understand
> nfs4_acl xattr, can it be used to perform ACL's for the system.
> AFAIU,
> it is uses user/group names as opposed uid/gid to perform id mapping.
> Can the client map it back to user names and derive if it is just an
> replica of inode's i_mode?
> 
> The idea is to suppress nfs4_acl if it is the same as inode's i_mode.
> This means nfs4-acl-tools/nfs4_getacl would give no results when
> requesting for ACLs. This would break existing applications if they
> expect some output from nfs4_getfacl.
> 
> Is there a better way to identify if nfs4_acl is just a
> representation
> of i_mode at the client end and can be safely ignored during an
> overlayfs copy_up? Can we include a flag for this?
> 
> 
> [1] https://www.spinics.net/lists/linux-nfs/msg61045.html
> [2] https://www.spinics.net/lists/linux-unionfs/msg04736.html
> 

If the permissions are unchanged so that overlayfs is not trying to
override the way the server interprets the ACL, and credential held by
the user, then you are better off calling the Linux client for
open/close and permissions calls.

Once you've allowed the user to chmod or chown the file, you are on
your own, because your security model for the file will have forked
with the security model provided by NFS. At that point, the file had
better have been copied up, and you'd better be ready to manage it
entirely from overlayfs.

The above applies not only when the file is subject to NFSv4 acls, but
it is also true when you are using strong authentication (i.e. Kerberos
V).

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

T24gVHVlLCAyMDE4LTA1LTI5IGF0IDE1OjMyIC0wNTAwLCBHb2xkd3luIFJvZHJpZ3VlcyB3cm90
ZToNCj4gV2hpbGUgbW91bnRpbmcgb3ZlcmxheWZzIHdpdGggTkZTIGFzIGEgbG93ZXIgZGlyZWN0
b3J5IGFuZCBhIGxvY2FsDQo+IGZpbGVzeXN0ZW0gYXMgYW4gdXBwZXIgbGF5ZXIgbGVhZHMgdG8g
Y29weV91cCBmYWlsdXJlcyBiZWNhdXNlIE5GUzQNCj4gaGFzDQo+IGFuIGV4dHJhIHN5c3RlbS5u
ZnM0X2FjbCB3aGljaCBjYW5ub3QgYmUgY29waWVkIHVwLiBUaGlzIGhhcyBiZWVuDQo+IGRpc2N1
c3NlZCBiZWZvcmUgWzFdIGFuZCBbMl0gd2l0aCB0aGUgc3VnZ2VzdGlvbiB0aGF0IG5mczRfYWNs
IGlzDQo+IGRlcml2ZWQgZnJvbSBwb3NpeF9hY2xzIG9yIGp1c3QgaW5vZGUtPmlfbW9kZSAqbW9z
dCogb2YgdGhlIHRpbWVzIGFuZA0KPiBoZW5jZSBpdCBjYW4gYmUgbWFwcGVkIGJhY2suDQo+IA0K
PiBUaGUgcHJvYmxlbSBpcyBORlMgY2xpZW50IGtub3dzIG5vdGhpbmcgYWJvdXQgbmZzNF9hY2wg
YW5kIGl0IGlzDQo+IGRlY29kZWQNCj4gaW4gbmZzNC1hY2wtdG9vbHMuIEV2ZW4gaWYgd2UgbWFr
ZSBuZnMgY2xpZW50IGNhcGFibGUgb2YgdW5kZXJzdGFuZA0KPiBuZnM0X2FjbCB4YXR0ciwgY2Fu
IGl0IGJlIHVzZWQgdG8gcGVyZm9ybSBBQ0wncyBmb3IgdGhlIHN5c3RlbS4NCj4gQUZBSVUsDQo+
IGl0IGlzIHVzZXMgdXNlci9ncm91cCBuYW1lcyBhcyBvcHBvc2VkIHVpZC9naWQgdG8gcGVyZm9y
bSBpZCBtYXBwaW5nLg0KPiBDYW4gdGhlIGNsaWVudCBtYXAgaXQgYmFjayB0byB1c2VyIG5hbWVz
IGFuZCBkZXJpdmUgaWYgaXQgaXMganVzdCBhbg0KPiByZXBsaWNhIG9mIGlub2RlJ3MgaV9tb2Rl
Pw0KPiANCj4gVGhlIGlkZWEgaXMgdG8gc3VwcHJlc3MgbmZzNF9hY2wgaWYgaXQgaXMgdGhlIHNh
bWUgYXMgaW5vZGUncyBpX21vZGUuDQo+IFRoaXMgbWVhbnMgbmZzNC1hY2wtdG9vbHMvbmZzNF9n
ZXRhY2wgd291bGQgZ2l2ZSBubyByZXN1bHRzIHdoZW4NCj4gcmVxdWVzdGluZyBmb3IgQUNMcy4g
VGhpcyB3b3VsZCBicmVhayBleGlzdGluZyBhcHBsaWNhdGlvbnMgaWYgdGhleQ0KPiBleHBlY3Qg
c29tZSBvdXRwdXQgZnJvbSBuZnM0X2dldGZhY2wuDQo+IA0KPiBJcyB0aGVyZSBhIGJldHRlciB3
YXkgdG8gaWRlbnRpZnkgaWYgbmZzNF9hY2wgaXMganVzdCBhDQo+IHJlcHJlc2VudGF0aW9uDQo+
IG9mIGlfbW9kZSBhdCB0aGUgY2xpZW50IGVuZCBhbmQgY2FuIGJlIHNhZmVseSBpZ25vcmVkIGR1
cmluZyBhbg0KPiBvdmVybGF5ZnMgY29weV91cD8gQ2FuIHdlIGluY2x1ZGUgYSBmbGFnIGZvciB0
aGlzPw0KPiANCj4gDQo+IFsxXSBodHRwczovL3d3dy5zcGluaWNzLm5ldC9saXN0cy9saW51eC1u
ZnMvbXNnNjEwNDUuaHRtbA0KPiBbMl0gaHR0cHM6Ly93d3cuc3Bpbmljcy5uZXQvbGlzdHMvbGlu
dXgtdW5pb25mcy9tc2cwNDczNi5odG1sDQo+IA0KDQpJZiB0aGUgcGVybWlzc2lvbnMgYXJlIHVu
Y2hhbmdlZCBzbyB0aGF0IG92ZXJsYXlmcyBpcyBub3QgdHJ5aW5nIHRvDQpvdmVycmlkZSB0aGUg
d2F5IHRoZSBzZXJ2ZXIgaW50ZXJwcmV0cyB0aGUgQUNMLCBhbmQgY3JlZGVudGlhbCBoZWxkIGJ5
DQp0aGUgdXNlciwgdGhlbiB5b3UgYXJlIGJldHRlciBvZmYgY2FsbGluZyB0aGUgTGludXggY2xp
ZW50IGZvcg0Kb3Blbi9jbG9zZSBhbmQgcGVybWlzc2lvbnMgY2FsbHMuDQoNCk9uY2UgeW91J3Zl
IGFsbG93ZWQgdGhlIHVzZXIgdG8gY2htb2Qgb3IgY2hvd24gdGhlIGZpbGUsIHlvdSBhcmUgb24N
CnlvdXIgb3duLCBiZWNhdXNlIHlvdXIgc2VjdXJpdHkgbW9kZWwgZm9yIHRoZSBmaWxlIHdpbGwg
aGF2ZSBmb3JrZWQNCndpdGggdGhlIHNlY3VyaXR5IG1vZGVsIHByb3ZpZGVkIGJ5IE5GUy4gQXQg
dGhhdCBwb2ludCwgdGhlIGZpbGUgaGFkDQpiZXR0ZXIgaGF2ZSBiZWVuIGNvcGllZCB1cCwgYW5k
IHlvdSdkIGJldHRlciBiZSByZWFkeSB0byBtYW5hZ2UgaXQNCmVudGlyZWx5IGZyb20gb3Zlcmxh
eWZzLg0KDQpUaGUgYWJvdmUgYXBwbGllcyBub3Qgb25seSB3aGVuIHRoZSBmaWxlIGlzIHN1Ympl
Y3QgdG8gTkZTdjQgYWNscywgYnV0DQppdCBpcyBhbHNvIHRydWUgd2hlbiB5b3UgYXJlIHVzaW5n
IHN0cm9uZyBhdXRoZW50aWNhdGlvbiAoaS5lLiBLZXJiZXJvcw0KVikuDQoNCi0tIA0KVHJvbmQg
TXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXIsIEhhbW1lcnNwYWNlDQp0cm9u
ZC5teWtsZWJ1c3RAaGFtbWVyc3BhY2UuY29tDQoNCg==

Re: nfs4_acl restricts copy_up in overlayfs

[Goldwyn Rodrigues]

On 05/29/2018 04:37 PM, Trond Myklebust wrote:
> On Tue, 2018-05-29 at 15:32 -0500, Goldwyn Rodrigues wrote:
>> While mounting overlayfs with NFS as a lower directory and a local
>> filesystem as an upper layer leads to copy_up failures because NFS4
>> has
>> an extra system.nfs4_acl which cannot be copied up. This has been
>> discussed before [1] and [2] with the suggestion that nfs4_acl is
>> derived from posix_acls or just inode->i_mode *most* of the times and
>> hence it can be mapped back.
>>
>> The problem is NFS client knows nothing about nfs4_acl and it is
>> decoded
>> in nfs4-acl-tools. Even if we make nfs client capable of understand
>> nfs4_acl xattr, can it be used to perform ACL's for the system.
>> AFAIU,
>> it is uses user/group names as opposed uid/gid to perform id mapping.
>> Can the client map it back to user names and derive if it is just an
>> replica of inode's i_mode?
>>
>> The idea is to suppress nfs4_acl if it is the same as inode's i_mode.
>> This means nfs4-acl-tools/nfs4_getacl would give no results when
>> requesting for ACLs. This would break existing applications if they
>> expect some output from nfs4_getfacl.
>>
>> Is there a better way to identify if nfs4_acl is just a
>> representation
>> of i_mode at the client end and can be safely ignored during an
>> overlayfs copy_up? Can we include a flag for this?
>>
>>
>> [1] https://www.spinics.net/lists/linux-nfs/msg61045.html
>> [2] https://www.spinics.net/lists/linux-unionfs/msg04736.html
>>
> 
> If the permissions are unchanged so that overlayfs is not trying to
> override the way the server interprets the ACL, and credential held by
> the user, then you are better off calling the Linux client for
> open/close and permissions calls.
> 
> Once you've allowed the user to chmod or chown the file, you are on
> your own, because your security model for the file will have forked
> with the security model provided by NFS. At that point, the file had
> better have been copied up, and you'd better be ready to manage it
> entirely from overlayfs.
> 
> The above applies not only when the file is subject to NFSv4 acls, but
> it is also true when you are using strong authentication (i.e. Kerberos
> V).
> 

The files permissions are checked during copy ups because a data copy
also happens in case of a change. The problem however is not the checks
have to happen for this operation but if we should bypass NFS security
for consequent ones.

For example, If we copy_up the file just because the current user is
able to, it would not copy the nfs4_acl. If there is a NFS4 acl rule to
DENY another user, the earlier denied user will be able to access the
file after the copy_up because we did not copy the nfs4_acl.

Overlayfs tries to make sure that all xattr from an underlying
filesystem is compatible with the one being copied to. nfs4_acl is not
compatible with a local filesystem it rejects the copy_up as EOPNOTSUPP.

>From the nfsd code (nfsd_get_nfs4_acl), in *most* cases nfs4_acl is just
a representation of i_mode and does not require a special ACL. However,
just because there is a nfs4_acl, a copy_up does not happen in the client.

-- 
Goldwyn

Re: nfs4_acl restricts copy_up in overlayfs

[Goldwyn Rodrigues]

On 05/29/2018 04:37 PM, Trond Myklebust wrote:
> On Tue, 2018-05-29 at 15:32 -0500, Goldwyn Rodrigues wrote:
>> While mounting overlayfs with NFS as a lower directory and a local
>> filesystem as an upper layer leads to copy_up failures because NFS4
>> has
>> an extra system.nfs4_acl which cannot be copied up. This has been
>> discussed before [1] and [2] with the suggestion that nfs4_acl is
>> derived from posix_acls or just inode->i_mode *most* of the times and
>> hence it can be mapped back.
>>
>> The problem is NFS client knows nothing about nfs4_acl and it is
>> decoded
>> in nfs4-acl-tools. Even if we make nfs client capable of understand
>> nfs4_acl xattr, can it be used to perform ACL's for the system.
>> AFAIU,
>> it is uses user/group names as opposed uid/gid to perform id mapping.
>> Can the client map it back to user names and derive if it is just an
>> replica of inode's i_mode?
>>
>> The idea is to suppress nfs4_acl if it is the same as inode's i_mode.
>> This means nfs4-acl-tools/nfs4_getacl would give no results when
>> requesting for ACLs. This would break existing applications if they
>> expect some output from nfs4_getfacl.
>>
>> Is there a better way to identify if nfs4_acl is just a
>> representation
>> of i_mode at the client end and can be safely ignored during an
>> overlayfs copy_up? Can we include a flag for this?
>>
>>
>> [1] https://www.spinics.net/lists/linux-nfs/msg61045.html
>> [2] https://www.spinics.net/lists/linux-unionfs/msg04736.html
>>
> 
> If the permissions are unchanged so that overlayfs is not trying to
> override the way the server interprets the ACL, and credential held by
> the user, then you are better off calling the Linux client for
> open/close and permissions calls.
> 
> Once you've allowed the user to chmod or chown the file, you are on
> your own, because your security model for the file will have forked
> with the security model provided by NFS. At that point, the file had
> better have been copied up, and you'd better be ready to manage it
> entirely from overlayfs.
> 
> The above applies not only when the file is subject to NFSv4 acls, but
> it is also true when you are using strong authentication (i.e. Kerberos
> V).
> 

The files permissions are checked during copy ups because a data copy
also happens in case of a change. The problem however is not the checks
have to happen for this operation but if we should bypass NFS security
for consequent ones.

For example, If we copy_up the file just because the current user is
able to, it would not copy the nfs4_acl. If there is a NFS4 acl rule to
DENY another user, the earlier denied user will be able to access the
file after the copy_up because we did not copy the nfs4_acl.

Overlayfs tries to make sure that all xattr from an underlying
filesystem is compatible with the one being copied to. nfs4_acl is not
compatible with a local filesystem it rejects the copy_up as EOPNOTSUPP.

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

On Tue, 2018-05-29 at 20:08 -0500, Goldwyn Rodrigues wrote:
> 
> On 05/29/2018 04:37 PM, Trond Myklebust wrote:
> > On Tue, 2018-05-29 at 15:32 -0500, Goldwyn Rodrigues wrote:
> > > While mounting overlayfs with NFS as a lower directory and a
> > > local
> > > filesystem as an upper layer leads to copy_up failures because
> > > NFS4
> > > has
> > > an extra system.nfs4_acl which cannot be copied up. This has been
> > > discussed before [1] and [2] with the suggestion that nfs4_acl is
> > > derived from posix_acls or just inode->i_mode *most* of the times
> > > and
> > > hence it can be mapped back.
> > > 
> > > The problem is NFS client knows nothing about nfs4_acl and it is
> > > decoded
> > > in nfs4-acl-tools. Even if we make nfs client capable of
> > > understand
> > > nfs4_acl xattr, can it be used to perform ACL's for the system.
> > > AFAIU,
> > > it is uses user/group names as opposed uid/gid to perform id
> > > mapping.
> > > Can the client map it back to user names and derive if it is just
> > > an
> > > replica of inode's i_mode?
> > > 
> > > The idea is to suppress nfs4_acl if it is the same as inode's
> > > i_mode.
> > > This means nfs4-acl-tools/nfs4_getacl would give no results when
> > > requesting for ACLs. This would break existing applications if
> > > they
> > > expect some output from nfs4_getfacl.
> > > 
> > > Is there a better way to identify if nfs4_acl is just a
> > > representation
> > > of i_mode at the client end and can be safely ignored during an
> > > overlayfs copy_up? Can we include a flag for this?
> > > 
> > > 
> > > [1] https://www.spinics.net/lists/linux-nfs/msg61045.html
> > > [2] https://www.spinics.net/lists/linux-unionfs/msg04736.html
> > > 
> > 
> > If the permissions are unchanged so that overlayfs is not trying to
> > override the way the server interprets the ACL, and credential held
> > by
> > the user, then you are better off calling the Linux client for
> > open/close and permissions calls.
> > 
> > Once you've allowed the user to chmod or chown the file, you are on
> > your own, because your security model for the file will have forked
> > with the security model provided by NFS. At that point, the file
> > had
> > better have been copied up, and you'd better be ready to manage it
> > entirely from overlayfs.
> > 
> > The above applies not only when the file is subject to NFSv4 acls,
> > but
> > it is also true when you are using strong authentication (i.e.
> > Kerberos
> > V).
> > 
> 
> The files permissions are checked during copy ups because a data copy
> also happens in case of a change. The problem however is not the
> checks
> have to happen for this operation but if we should bypass NFS
> security
> for consequent ones.
> 
> For example, If we copy_up the file just because the current user is
> able to, it would not copy the nfs4_acl. If there is a NFS4 acl rule
> to
> DENY another user, the earlier denied user will be able to access the
> file after the copy_up because we did not copy the nfs4_acl.
> 

As I said, you are forking the security model. You are trying to make
the NFS client act as a proxy for the NFS server without giving it
enough information to do so.

The NFS client is careful to _always_ leave interpreting the ACL, mode,
and other security information to the server. If it must try to act as
a proxy for the server when serving up cached information, then it uses
the ACCESS rpc call to ensure that it doesn't give up information that
the server would normally deny to a particular user.

> Overlayfs tries to make sure that all xattr from an underlying
> filesystem is compatible with the one being copied to. nfs4_acl is
> not
> compatible with a local filesystem it rejects the copy_up as
> EOPNOTSUPP.
> 
> From the nfsd code (nfsd_get_nfs4_acl), in *most* cases nfs4_acl is
> just
> a representation of i_mode and does not require a special ACL. 

If your server is a knfsd based Linux server, then maybe. If it's
anything else, then the above statement is dead wrong. Either way,
you'd be opening a large can of security worms by making those
assumptions.

> However,
> just because there is a nfs4_acl, a copy_up does not happen in the
> client.

Which would appear to be the more secure behaviour if you are unable to
enforce the exact same security semantics as the server.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

T24gVHVlLCAyMDE4LTA1LTI5IGF0IDIwOjA4IC0wNTAwLCBHb2xkd3luIFJvZHJpZ3VlcyB3cm90
ZToNCj4gDQo+IE9uIDA1LzI5LzIwMTggMDQ6MzcgUE0sIFRyb25kIE15a2xlYnVzdCB3cm90ZToN
Cj4gPiBPbiBUdWUsIDIwMTgtMDUtMjkgYXQgMTU6MzIgLTA1MDAsIEdvbGR3eW4gUm9kcmlndWVz
IHdyb3RlOg0KPiA+ID4gV2hpbGUgbW91bnRpbmcgb3ZlcmxheWZzIHdpdGggTkZTIGFzIGEgbG93
ZXIgZGlyZWN0b3J5IGFuZCBhDQo+ID4gPiBsb2NhbA0KPiA+ID4gZmlsZXN5c3RlbSBhcyBhbiB1
cHBlciBsYXllciBsZWFkcyB0byBjb3B5X3VwIGZhaWx1cmVzIGJlY2F1c2UNCj4gPiA+IE5GUzQN
Cj4gPiA+IGhhcw0KPiA+ID4gYW4gZXh0cmEgc3lzdGVtLm5mczRfYWNsIHdoaWNoIGNhbm5vdCBi
ZSBjb3BpZWQgdXAuIFRoaXMgaGFzIGJlZW4NCj4gPiA+IGRpc2N1c3NlZCBiZWZvcmUgWzFdIGFu
ZCBbMl0gd2l0aCB0aGUgc3VnZ2VzdGlvbiB0aGF0IG5mczRfYWNsIGlzDQo+ID4gPiBkZXJpdmVk
IGZyb20gcG9zaXhfYWNscyBvciBqdXN0IGlub2RlLT5pX21vZGUgKm1vc3QqIG9mIHRoZSB0aW1l
cw0KPiA+ID4gYW5kDQo+ID4gPiBoZW5jZSBpdCBjYW4gYmUgbWFwcGVkIGJhY2suDQo+ID4gPiAN
Cj4gPiA+IFRoZSBwcm9ibGVtIGlzIE5GUyBjbGllbnQga25vd3Mgbm90aGluZyBhYm91dCBuZnM0
X2FjbCBhbmQgaXQgaXMNCj4gPiA+IGRlY29kZWQNCj4gPiA+IGluIG5mczQtYWNsLXRvb2xzLiBF
dmVuIGlmIHdlIG1ha2UgbmZzIGNsaWVudCBjYXBhYmxlIG9mDQo+ID4gPiB1bmRlcnN0YW5kDQo+
ID4gPiBuZnM0X2FjbCB4YXR0ciwgY2FuIGl0IGJlIHVzZWQgdG8gcGVyZm9ybSBBQ0wncyBmb3Ig
dGhlIHN5c3RlbS4NCj4gPiA+IEFGQUlVLA0KPiA+ID4gaXQgaXMgdXNlcyB1c2VyL2dyb3VwIG5h
bWVzIGFzIG9wcG9zZWQgdWlkL2dpZCB0byBwZXJmb3JtIGlkDQo+ID4gPiBtYXBwaW5nLg0KPiA+
ID4gQ2FuIHRoZSBjbGllbnQgbWFwIGl0IGJhY2sgdG8gdXNlciBuYW1lcyBhbmQgZGVyaXZlIGlm
IGl0IGlzIGp1c3QNCj4gPiA+IGFuDQo+ID4gPiByZXBsaWNhIG9mIGlub2RlJ3MgaV9tb2RlPw0K
PiA+ID4gDQo+ID4gPiBUaGUgaWRlYSBpcyB0byBzdXBwcmVzcyBuZnM0X2FjbCBpZiBpdCBpcyB0
aGUgc2FtZSBhcyBpbm9kZSdzDQo+ID4gPiBpX21vZGUuDQo+ID4gPiBUaGlzIG1lYW5zIG5mczQt
YWNsLXRvb2xzL25mczRfZ2V0YWNsIHdvdWxkIGdpdmUgbm8gcmVzdWx0cyB3aGVuDQo+ID4gPiBy
ZXF1ZXN0aW5nIGZvciBBQ0xzLiBUaGlzIHdvdWxkIGJyZWFrIGV4aXN0aW5nIGFwcGxpY2F0aW9u
cyBpZg0KPiA+ID4gdGhleQ0KPiA+ID4gZXhwZWN0IHNvbWUgb3V0cHV0IGZyb20gbmZzNF9nZXRm
YWNsLg0KPiA+ID4gDQo+ID4gPiBJcyB0aGVyZSBhIGJldHRlciB3YXkgdG8gaWRlbnRpZnkgaWYg
bmZzNF9hY2wgaXMganVzdCBhDQo+ID4gPiByZXByZXNlbnRhdGlvbg0KPiA+ID4gb2YgaV9tb2Rl
IGF0IHRoZSBjbGllbnQgZW5kIGFuZCBjYW4gYmUgc2FmZWx5IGlnbm9yZWQgZHVyaW5nIGFuDQo+
ID4gPiBvdmVybGF5ZnMgY29weV91cD8gQ2FuIHdlIGluY2x1ZGUgYSBmbGFnIGZvciB0aGlzPw0K
PiA+ID4gDQo+ID4gPiANCj4gPiA+IFsxXSBodHRwczovL3d3dy5zcGluaWNzLm5ldC9saXN0cy9s
aW51eC1uZnMvbXNnNjEwNDUuaHRtbA0KPiA+ID4gWzJdIGh0dHBzOi8vd3d3LnNwaW5pY3MubmV0
L2xpc3RzL2xpbnV4LXVuaW9uZnMvbXNnMDQ3MzYuaHRtbA0KPiA+ID4gDQo+ID4gDQo+ID4gSWYg
dGhlIHBlcm1pc3Npb25zIGFyZSB1bmNoYW5nZWQgc28gdGhhdCBvdmVybGF5ZnMgaXMgbm90IHRy
eWluZyB0bw0KPiA+IG92ZXJyaWRlIHRoZSB3YXkgdGhlIHNlcnZlciBpbnRlcnByZXRzIHRoZSBB
Q0wsIGFuZCBjcmVkZW50aWFsIGhlbGQNCj4gPiBieQ0KPiA+IHRoZSB1c2VyLCB0aGVuIHlvdSBh
cmUgYmV0dGVyIG9mZiBjYWxsaW5nIHRoZSBMaW51eCBjbGllbnQgZm9yDQo+ID4gb3Blbi9jbG9z
ZSBhbmQgcGVybWlzc2lvbnMgY2FsbHMuDQo+ID4gDQo+ID4gT25jZSB5b3UndmUgYWxsb3dlZCB0
aGUgdXNlciB0byBjaG1vZCBvciBjaG93biB0aGUgZmlsZSwgeW91IGFyZSBvbg0KPiA+IHlvdXIg
b3duLCBiZWNhdXNlIHlvdXIgc2VjdXJpdHkgbW9kZWwgZm9yIHRoZSBmaWxlIHdpbGwgaGF2ZSBm
b3JrZWQNCj4gPiB3aXRoIHRoZSBzZWN1cml0eSBtb2RlbCBwcm92aWRlZCBieSBORlMuIEF0IHRo
YXQgcG9pbnQsIHRoZSBmaWxlDQo+ID4gaGFkDQo+ID4gYmV0dGVyIGhhdmUgYmVlbiBjb3BpZWQg
dXAsIGFuZCB5b3UnZCBiZXR0ZXIgYmUgcmVhZHkgdG8gbWFuYWdlIGl0DQo+ID4gZW50aXJlbHkg
ZnJvbSBvdmVybGF5ZnMuDQo+ID4gDQo+ID4gVGhlIGFib3ZlIGFwcGxpZXMgbm90IG9ubHkgd2hl
biB0aGUgZmlsZSBpcyBzdWJqZWN0IHRvIE5GU3Y0IGFjbHMsDQo+ID4gYnV0DQo+ID4gaXQgaXMg
YWxzbyB0cnVlIHdoZW4geW91IGFyZSB1c2luZyBzdHJvbmcgYXV0aGVudGljYXRpb24gKGkuZS4N
Cj4gPiBLZXJiZXJvcw0KPiA+IFYpLg0KPiA+IA0KPiANCj4gVGhlIGZpbGVzIHBlcm1pc3Npb25z
IGFyZSBjaGVja2VkIGR1cmluZyBjb3B5IHVwcyBiZWNhdXNlIGEgZGF0YSBjb3B5DQo+IGFsc28g
aGFwcGVucyBpbiBjYXNlIG9mIGEgY2hhbmdlLiBUaGUgcHJvYmxlbSBob3dldmVyIGlzIG5vdCB0
aGUNCj4gY2hlY2tzDQo+IGhhdmUgdG8gaGFwcGVuIGZvciB0aGlzIG9wZXJhdGlvbiBidXQgaWYg
d2Ugc2hvdWxkIGJ5cGFzcyBORlMNCj4gc2VjdXJpdHkNCj4gZm9yIGNvbnNlcXVlbnQgb25lcy4N
Cj4gDQo+IEZvciBleGFtcGxlLCBJZiB3ZSBjb3B5X3VwIHRoZSBmaWxlIGp1c3QgYmVjYXVzZSB0
aGUgY3VycmVudCB1c2VyIGlzDQo+IGFibGUgdG8sIGl0IHdvdWxkIG5vdCBjb3B5IHRoZSBuZnM0
X2FjbC4gSWYgdGhlcmUgaXMgYSBORlM0IGFjbCBydWxlDQo+IHRvDQo+IERFTlkgYW5vdGhlciB1
c2VyLCB0aGUgZWFybGllciBkZW5pZWQgdXNlciB3aWxsIGJlIGFibGUgdG8gYWNjZXNzIHRoZQ0K
PiBmaWxlIGFmdGVyIHRoZSBjb3B5X3VwIGJlY2F1c2Ugd2UgZGlkIG5vdCBjb3B5IHRoZSBuZnM0
X2FjbC4NCj4gDQoNCkFzIEkgc2FpZCwgeW91IGFyZSBmb3JraW5nIHRoZSBzZWN1cml0eSBtb2Rl
bC4gWW91IGFyZSB0cnlpbmcgdG8gbWFrZQ0KdGhlIE5GUyBjbGllbnQgYWN0IGFzIGEgcHJveHkg
Zm9yIHRoZSBORlMgc2VydmVyIHdpdGhvdXQgZ2l2aW5nIGl0DQplbm91Z2ggaW5mb3JtYXRpb24g
dG8gZG8gc28uDQoNClRoZSBORlMgY2xpZW50IGlzIGNhcmVmdWwgdG8gX2Fsd2F5c18gbGVhdmUg
aW50ZXJwcmV0aW5nIHRoZSBBQ0wsIG1vZGUsDQphbmQgb3RoZXIgc2VjdXJpdHkgaW5mb3JtYXRp
b24gdG8gdGhlIHNlcnZlci4gSWYgaXQgbXVzdCB0cnkgdG8gYWN0IGFzDQphIHByb3h5IGZvciB0
aGUgc2VydmVyIHdoZW4gc2VydmluZyB1cCBjYWNoZWQgaW5mb3JtYXRpb24sIHRoZW4gaXQgdXNl
cw0KdGhlIEFDQ0VTUyBycGMgY2FsbCB0byBlbnN1cmUgdGhhdCBpdCBkb2Vzbid0IGdpdmUgdXAg
aW5mb3JtYXRpb24gdGhhdA0KdGhlIHNlcnZlciB3b3VsZCBub3JtYWxseSBkZW55IHRvIGEgcGFy
dGljdWxhciB1c2VyLg0KDQo+IE92ZXJsYXlmcyB0cmllcyB0byBtYWtlIHN1cmUgdGhhdCBhbGwg
eGF0dHIgZnJvbSBhbiB1bmRlcmx5aW5nDQo+IGZpbGVzeXN0ZW0gaXMgY29tcGF0aWJsZSB3aXRo
IHRoZSBvbmUgYmVpbmcgY29waWVkIHRvLiBuZnM0X2FjbCBpcw0KPiBub3QNCj4gY29tcGF0aWJs
ZSB3aXRoIGEgbG9jYWwgZmlsZXN5c3RlbSBpdCByZWplY3RzIHRoZSBjb3B5X3VwIGFzDQo+IEVP
UE5PVFNVUFAuDQo+IA0KPiBGcm9tIHRoZSBuZnNkIGNvZGUgKG5mc2RfZ2V0X25mczRfYWNsKSwg
aW4gKm1vc3QqIGNhc2VzIG5mczRfYWNsIGlzDQo+IGp1c3QNCj4gYSByZXByZXNlbnRhdGlvbiBv
ZiBpX21vZGUgYW5kIGRvZXMgbm90IHJlcXVpcmUgYSBzcGVjaWFsIEFDTC4gDQoNCklmIHlvdXIg
c2VydmVyIGlzIGEga25mc2QgYmFzZWQgTGludXggc2VydmVyLCB0aGVuIG1heWJlLiBJZiBpdCdz
DQphbnl0aGluZyBlbHNlLCB0aGVuIHRoZSBhYm92ZSBzdGF0ZW1lbnQgaXMgZGVhZCB3cm9uZy4g
RWl0aGVyIHdheSwNCnlvdSdkIGJlIG9wZW5pbmcgYSBsYXJnZSBjYW4gb2Ygc2VjdXJpdHkgd29y
bXMgYnkgbWFraW5nIHRob3NlDQphc3N1bXB0aW9ucy4NCg0KPiBIb3dldmVyLA0KPiBqdXN0IGJl
Y2F1c2UgdGhlcmUgaXMgYSBuZnM0X2FjbCwgYSBjb3B5X3VwIGRvZXMgbm90IGhhcHBlbiBpbiB0
aGUNCj4gY2xpZW50Lg0KDQpXaGljaCB3b3VsZCBhcHBlYXIgdG8gYmUgdGhlIG1vcmUgc2VjdXJl
IGJlaGF2aW91ciBpZiB5b3UgYXJlIHVuYWJsZSB0bw0KZW5mb3JjZSB0aGUgZXhhY3Qgc2FtZSBz
ZWN1cml0eSBzZW1hbnRpY3MgYXMgdGhlIHNlcnZlci4NCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QN
CkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lciwgSGFtbWVyc3BhY2UNCnRyb25kLm15a2xlYnVz
dEBoYW1tZXJzcGFjZS5jb20NCg0K

Re: nfs4_acl restricts copy_up in overlayfs

[Goldwyn Rodrigues]

On 05/29/2018 10:01 PM, Trond Myklebust wrote:
> On Tue, 2018-05-29 at 20:08 -0500, Goldwyn Rodrigues wrote:
>>
>> On 05/29/2018 04:37 PM, Trond Myklebust wrote:
>>> On Tue, 2018-05-29 at 15:32 -0500, Goldwyn Rodrigues wrote:
>>>> While mounting overlayfs with NFS as a lower directory and a
>>>> local
>>>> filesystem as an upper layer leads to copy_up failures because
>>>> NFS4
>>>> has
>>>> an extra system.nfs4_acl which cannot be copied up. This has been
>>>> discussed before [1] and [2] with the suggestion that nfs4_acl is
>>>> derived from posix_acls or just inode->i_mode *most* of the times
>>>> and
>>>> hence it can be mapped back.
>>>>
>>>> The problem is NFS client knows nothing about nfs4_acl and it is
>>>> decoded
>>>> in nfs4-acl-tools. Even if we make nfs client capable of
>>>> understand
>>>> nfs4_acl xattr, can it be used to perform ACL's for the system.
>>>> AFAIU,
>>>> it is uses user/group names as opposed uid/gid to perform id
>>>> mapping.
>>>> Can the client map it back to user names and derive if it is just
>>>> an
>>>> replica of inode's i_mode?
>>>>
>>>> The idea is to suppress nfs4_acl if it is the same as inode's
>>>> i_mode.
>>>> This means nfs4-acl-tools/nfs4_getacl would give no results when
>>>> requesting for ACLs. This would break existing applications if
>>>> they
>>>> expect some output from nfs4_getfacl.
>>>>
>>>> Is there a better way to identify if nfs4_acl is just a
>>>> representation
>>>> of i_mode at the client end and can be safely ignored during an
>>>> overlayfs copy_up? Can we include a flag for this?
>>>>
>>>>
>>>> [1] https://www.spinics.net/lists/linux-nfs/msg61045.html
>>>> [2] https://www.spinics.net/lists/linux-unionfs/msg04736.html
>>>>
>>>
>>> If the permissions are unchanged so that overlayfs is not trying to
>>> override the way the server interprets the ACL, and credential held
>>> by
>>> the user, then you are better off calling the Linux client for
>>> open/close and permissions calls.
>>>
>>> Once you've allowed the user to chmod or chown the file, you are on
>>> your own, because your security model for the file will have forked
>>> with the security model provided by NFS. At that point, the file
>>> had
>>> better have been copied up, and you'd better be ready to manage it
>>> entirely from overlayfs.
>>>
>>> The above applies not only when the file is subject to NFSv4 acls,
>>> but
>>> it is also true when you are using strong authentication (i.e.
>>> Kerberos
>>> V).
>>>
>>
>> The files permissions are checked during copy ups because a data copy
>> also happens in case of a change. The problem however is not the
>> checks
>> have to happen for this operation but if we should bypass NFS
>> security
>> for consequent ones.
>>
>> For example, If we copy_up the file just because the current user is
>> able to, it would not copy the nfs4_acl. If there is a NFS4 acl rule
>> to
>> DENY another user, the earlier denied user will be able to access the
>> file after the copy_up because we did not copy the nfs4_acl.
>>
> 
> As I said, you are forking the security model. You are trying to make
> the NFS client act as a proxy for the NFS server without giving it
> enough information to do so.
> 
> The NFS client is careful to _always_ leave interpreting the ACL, mode,
> and other security information to the server. If it must try to act as
> a proxy for the server when serving up cached information, then it uses
> the ACCESS rpc call to ensure that it doesn't give up information that
> the server would normally deny to a particular user.

I am in agreement with the security model and that is what I want to
preserve. My question is if it is possible to detect that the security
can be represented by existing in-inode security features alone.

If it was up to me, I would not create the nfs4_acl xattr at the server
if it is not required and the security can be represented by existing
parameters, but it seems to be the protocol. To give a comparison, in
local filsystems, if "system.posix_acl" is not required it will not be
created.

> 
>> Overlayfs tries to make sure that all xattr from an underlying
>> filesystem is compatible with the one being copied to. nfs4_acl is
>> not
>> compatible with a local filesystem it rejects the copy_up as
>> EOPNOTSUPP.
>>
>> From the nfsd code (nfsd_get_nfs4_acl), in *most* cases nfs4_acl is
>> just
>> a representation of i_mode and does not require a special ACL. 
> 
> If your server is a knfsd based Linux server, then maybe. If it's
> anything else, then the above statement is dead wrong. Either way,
> you'd be opening a large can of security worms by making those
> assumptions.

I agree. The man page says there are features (such as alarm and audit)
which are server dependent.

> 
>> However,
>> just because there is a nfs4_acl, a copy_up does not happen in the
>> client.
> 
> Which would appear to be the more secure behaviour if you are unable to
> enforce the exact same security semantics as the server.

I am not trying to override the security. I am trying to detect
duplication of security information. The common case of NFS
communication does not require the additional security parameters
(doesn't mean it is not required). So my question is: is it possible to
detect at the client that nfs4_acl is a duplicate of information which
can be and is represented by inode alone. If yes, can it be suppressed
by the client.

-- 
Goldwyn

Re: nfs4_acl restricts copy_up in overlayfs

[J. Bruce Fields]

On Wed, May 30, 2018 at 05:33:11AM -0500, Goldwyn Rodrigues wrote:
> I am not trying to override the security. I am trying to detect
> duplication of security information. The common case of NFS
> communication does not require the additional security parameters
> (doesn't mean it is not required). So my question is: is it possible to
> detect at the client that nfs4_acl is a duplicate of information which
> can be and is represented by inode alone. If yes, can it be suppressed
> by the client.

No, that's not possible.

The user's identity could be mapped in various ways.  You've got no way
to know whether root squashing is in effect, for example.  Or to know
what the user@EXAMPLE.COM krb5 identity that you're running as might map
to on the server.

So it's hard to even tell whether a given user matches the file's owner
or group.  So even the mode bits are kind of meaningless to the client.

--b.

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Thu, May 31, 2018 at 2:45 AM, J. Bruce Fields <bfields@fieldses.org> wrote:
> On Wed, May 30, 2018 at 05:33:11AM -0500, Goldwyn Rodrigues wrote:
>> I am not trying to override the security. I am trying to detect
>> duplication of security information. The common case of NFS
>> communication does not require the additional security parameters
>> (doesn't mean it is not required). So my question is: is it possible to
>> detect at the client that nfs4_acl is a duplicate of information which
>> can be and is represented by inode alone. If yes, can it be suppressed
>> by the client.
>
> No, that's not possible.
>
> The user's identity could be mapped in various ways.  You've got no way
> to know whether root squashing is in effect, for example.  Or to know
> what the user@EXAMPLE.COM krb5 identity that you're running as might map
> to on the server.
>
> So it's hard to even tell whether a given user matches the file's owner
> or group.  So even the mode bits are kind of meaningless to the client.

The basic security model for overlayfs is that underlying filesystems
are just storage.  Access to these filesystems is done with the
capabilities of the task that created the overlay instance with
mount(2).  That capability set is saved and used for any access to
underlying storage.

Access to overlayfs itself is controlled by metadata in the file
(mode, uid, gid, posix_acl, security xattr, etc...).

So if one of the layers is NFS, the permissions in the server are only
checked against the mounter's creds (usually superuser).  Access
checks are not performed by the server on behalf of the task accessing
the overlay.    This means, that overlayfs could give access to an NFS
file, where access on the NFS mount would be denied.  This needs to be
understood by the admin mounting the overlay.

So how to handle nfs4_acls with this model?

We could just ignore them and this can be achieved with mounting the
NFS filesystem with "noacl".  I'm not against specifically ignoring
nfs4_acl in overlayfs by default, as that seems to be the simplest
solution to this problem and fits the overlayfs security model.
Later, if we want to make use of this attribute to check access (on
the overlay, not in the NFS server), we can add an option to enable
this.  But AFAICS that one requires richacl's to make it upstream at
least.

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

On Thu, 2018-05-31 at 12:00 +0200, Miklos Szeredi wrote:
> On Thu, May 31, 2018 at 2:45 AM, J. Bruce Fields <bfields@fieldses.or
> g> wrote:
> > On Wed, May 30, 2018 at 05:33:11AM -0500, Goldwyn Rodrigues wrote:
> > > I am not trying to override the security. I am trying to detect
> > > duplication of security information. The common case of NFS
> > > communication does not require the additional security parameters
> > > (doesn't mean it is not required). So my question is: is it
> > > possible to
> > > detect at the client that nfs4_acl is a duplicate of information
> > > which
> > > can be and is represented by inode alone. If yes, can it be
> > > suppressed
> > > by the client.
> > 
> > No, that's not possible.
> > 
> > The user's identity could be mapped in various ways.  You've got no
> > way
> > to know whether root squashing is in effect, for example.  Or to
> > know
> > what the user@EXAMPLE.COM krb5 identity that you're running as
> > might map
> > to on the server.
> > 
> > So it's hard to even tell whether a given user matches the file's
> > owner
> > or group.  So even the mode bits are kind of meaningless to the
> > client.
> 
> The basic security model for overlayfs is that underlying filesystems
> are just storage.  Access to these filesystems is done with the
> capabilities of the task that created the overlay instance with
> mount(2).  That capability set is saved and used for any access to
> underlying storage.
> 
> Access to overlayfs itself is controlled by metadata in the file
> (mode, uid, gid, posix_acl, security xattr, etc...).
> 
> So if one of the layers is NFS, the permissions in the server are
> only
> checked against the mounter's creds (usually superuser).  Access
> checks are not performed by the server on behalf of the task
> accessing
> the overlay.    This means, that overlayfs could give access to an
> NFS
> file, where access on the NFS mount would be denied.  This needs to
> be
> understood by the admin mounting the overlay.
> 
> So how to handle nfs4_acls with this model?
> 
> We could just ignore them and this can be achieved with mounting the
> NFS filesystem with "noacl".  I'm not against specifically ignoring
> nfs4_acl in overlayfs by default, as that seems to be the simplest
> solution to this problem and fits the overlayfs security model.
> Later, if we want to make use of this attribute to check access (on
> the overlay, not in the NFS server), we can add an option to enable
> this.  But AFAICS that one requires richacl's to make it upstream at
> least.

'noacl' does not mean what you think it means. It doesn't mean that the
NFS security model is changed in any way. Security is still enforced by
the server.

And no, richacl won't help you get further either.

I'm still in strong disagreement with the model you are presenting
here. It is a client enforced model, which is not ever going to be
compatible with the NFS model.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

T24gVGh1LCAyMDE4LTA1LTMxIGF0IDEyOjAwICswMjAwLCBNaWtsb3MgU3plcmVkaSB3cm90ZToN
Cj4gT24gVGh1LCBNYXkgMzEsIDIwMTggYXQgMjo0NSBBTSwgSi4gQnJ1Y2UgRmllbGRzIDxiZmll
bGRzQGZpZWxkc2VzLm9yDQo+IGc+IHdyb3RlOg0KPiA+IE9uIFdlZCwgTWF5IDMwLCAyMDE4IGF0
IDA1OjMzOjExQU0gLTA1MDAsIEdvbGR3eW4gUm9kcmlndWVzIHdyb3RlOg0KPiA+ID4gSSBhbSBu
b3QgdHJ5aW5nIHRvIG92ZXJyaWRlIHRoZSBzZWN1cml0eS4gSSBhbSB0cnlpbmcgdG8gZGV0ZWN0
DQo+ID4gPiBkdXBsaWNhdGlvbiBvZiBzZWN1cml0eSBpbmZvcm1hdGlvbi4gVGhlIGNvbW1vbiBj
YXNlIG9mIE5GUw0KPiA+ID4gY29tbXVuaWNhdGlvbiBkb2VzIG5vdCByZXF1aXJlIHRoZSBhZGRp
dGlvbmFsIHNlY3VyaXR5IHBhcmFtZXRlcnMNCj4gPiA+IChkb2Vzbid0IG1lYW4gaXQgaXMgbm90
IHJlcXVpcmVkKS4gU28gbXkgcXVlc3Rpb24gaXM6IGlzIGl0DQo+ID4gPiBwb3NzaWJsZSB0bw0K
PiA+ID4gZGV0ZWN0IGF0IHRoZSBjbGllbnQgdGhhdCBuZnM0X2FjbCBpcyBhIGR1cGxpY2F0ZSBv
ZiBpbmZvcm1hdGlvbg0KPiA+ID4gd2hpY2gNCj4gPiA+IGNhbiBiZSBhbmQgaXMgcmVwcmVzZW50
ZWQgYnkgaW5vZGUgYWxvbmUuIElmIHllcywgY2FuIGl0IGJlDQo+ID4gPiBzdXBwcmVzc2VkDQo+
ID4gPiBieSB0aGUgY2xpZW50Lg0KPiA+IA0KPiA+IE5vLCB0aGF0J3Mgbm90IHBvc3NpYmxlLg0K
PiA+IA0KPiA+IFRoZSB1c2VyJ3MgaWRlbnRpdHkgY291bGQgYmUgbWFwcGVkIGluIHZhcmlvdXMg
d2F5cy4gIFlvdSd2ZSBnb3Qgbm8NCj4gPiB3YXkNCj4gPiB0byBrbm93IHdoZXRoZXIgcm9vdCBz
cXVhc2hpbmcgaXMgaW4gZWZmZWN0LCBmb3IgZXhhbXBsZS4gIE9yIHRvDQo+ID4ga25vdw0KPiA+
IHdoYXQgdGhlIHVzZXJARVhBTVBMRS5DT00ga3JiNSBpZGVudGl0eSB0aGF0IHlvdSdyZSBydW5u
aW5nIGFzDQo+ID4gbWlnaHQgbWFwDQo+ID4gdG8gb24gdGhlIHNlcnZlci4NCj4gPiANCj4gPiBT
byBpdCdzIGhhcmQgdG8gZXZlbiB0ZWxsIHdoZXRoZXIgYSBnaXZlbiB1c2VyIG1hdGNoZXMgdGhl
IGZpbGUncw0KPiA+IG93bmVyDQo+ID4gb3IgZ3JvdXAuICBTbyBldmVuIHRoZSBtb2RlIGJpdHMg
YXJlIGtpbmQgb2YgbWVhbmluZ2xlc3MgdG8gdGhlDQo+ID4gY2xpZW50Lg0KPiANCj4gVGhlIGJh
c2ljIHNlY3VyaXR5IG1vZGVsIGZvciBvdmVybGF5ZnMgaXMgdGhhdCB1bmRlcmx5aW5nIGZpbGVz
eXN0ZW1zDQo+IGFyZSBqdXN0IHN0b3JhZ2UuICBBY2Nlc3MgdG8gdGhlc2UgZmlsZXN5c3RlbXMg
aXMgZG9uZSB3aXRoIHRoZQ0KPiBjYXBhYmlsaXRpZXMgb2YgdGhlIHRhc2sgdGhhdCBjcmVhdGVk
IHRoZSBvdmVybGF5IGluc3RhbmNlIHdpdGgNCj4gbW91bnQoMikuICBUaGF0IGNhcGFiaWxpdHkg
c2V0IGlzIHNhdmVkIGFuZCB1c2VkIGZvciBhbnkgYWNjZXNzIHRvDQo+IHVuZGVybHlpbmcgc3Rv
cmFnZS4NCj4gDQo+IEFjY2VzcyB0byBvdmVybGF5ZnMgaXRzZWxmIGlzIGNvbnRyb2xsZWQgYnkg
bWV0YWRhdGEgaW4gdGhlIGZpbGUNCj4gKG1vZGUsIHVpZCwgZ2lkLCBwb3NpeF9hY2wsIHNlY3Vy
aXR5IHhhdHRyLCBldGMuLi4pLg0KPiANCj4gU28gaWYgb25lIG9mIHRoZSBsYXllcnMgaXMgTkZT
LCB0aGUgcGVybWlzc2lvbnMgaW4gdGhlIHNlcnZlciBhcmUNCj4gb25seQ0KPiBjaGVja2VkIGFn
YWluc3QgdGhlIG1vdW50ZXIncyBjcmVkcyAodXN1YWxseSBzdXBlcnVzZXIpLiAgQWNjZXNzDQo+
IGNoZWNrcyBhcmUgbm90IHBlcmZvcm1lZCBieSB0aGUgc2VydmVyIG9uIGJlaGFsZiBvZiB0aGUg
dGFzaw0KPiBhY2Nlc3NpbmcNCj4gdGhlIG92ZXJsYXkuICAgIFRoaXMgbWVhbnMsIHRoYXQgb3Zl
cmxheWZzIGNvdWxkIGdpdmUgYWNjZXNzIHRvIGFuDQo+IE5GUw0KPiBmaWxlLCB3aGVyZSBhY2Nl
c3Mgb24gdGhlIE5GUyBtb3VudCB3b3VsZCBiZSBkZW5pZWQuICBUaGlzIG5lZWRzIHRvDQo+IGJl
DQo+IHVuZGVyc3Rvb2QgYnkgdGhlIGFkbWluIG1vdW50aW5nIHRoZSBvdmVybGF5Lg0KPiANCj4g
U28gaG93IHRvIGhhbmRsZSBuZnM0X2FjbHMgd2l0aCB0aGlzIG1vZGVsPw0KPiANCj4gV2UgY291
bGQganVzdCBpZ25vcmUgdGhlbSBhbmQgdGhpcyBjYW4gYmUgYWNoaWV2ZWQgd2l0aCBtb3VudGlu
ZyB0aGUNCj4gTkZTIGZpbGVzeXN0ZW0gd2l0aCAibm9hY2wiLiAgSSdtIG5vdCBhZ2FpbnN0IHNw
ZWNpZmljYWxseSBpZ25vcmluZw0KPiBuZnM0X2FjbCBpbiBvdmVybGF5ZnMgYnkgZGVmYXVsdCwg
YXMgdGhhdCBzZWVtcyB0byBiZSB0aGUgc2ltcGxlc3QNCj4gc29sdXRpb24gdG8gdGhpcyBwcm9i
bGVtIGFuZCBmaXRzIHRoZSBvdmVybGF5ZnMgc2VjdXJpdHkgbW9kZWwuDQo+IExhdGVyLCBpZiB3
ZSB3YW50IHRvIG1ha2UgdXNlIG9mIHRoaXMgYXR0cmlidXRlIHRvIGNoZWNrIGFjY2VzcyAob24N
Cj4gdGhlIG92ZXJsYXksIG5vdCBpbiB0aGUgTkZTIHNlcnZlciksIHdlIGNhbiBhZGQgYW4gb3B0
aW9uIHRvIGVuYWJsZQ0KPiB0aGlzLiAgQnV0IEFGQUlDUyB0aGF0IG9uZSByZXF1aXJlcyByaWNo
YWNsJ3MgdG8gbWFrZSBpdCB1cHN0cmVhbSBhdA0KPiBsZWFzdC4NCg0KJ25vYWNsJyBkb2VzIG5v
dCBtZWFuIHdoYXQgeW91IHRoaW5rIGl0IG1lYW5zLiBJdCBkb2Vzbid0IG1lYW4gdGhhdCB0aGUN
Ck5GUyBzZWN1cml0eSBtb2RlbCBpcyBjaGFuZ2VkIGluIGFueSB3YXkuIFNlY3VyaXR5IGlzIHN0
aWxsIGVuZm9yY2VkIGJ5DQp0aGUgc2VydmVyLg0KDQpBbmQgbm8sIHJpY2hhY2wgd29uJ3QgaGVs
cCB5b3UgZ2V0IGZ1cnRoZXIgZWl0aGVyLg0KDQpJJ20gc3RpbGwgaW4gc3Ryb25nIGRpc2FncmVl
bWVudCB3aXRoIHRoZSBtb2RlbCB5b3UgYXJlIHByZXNlbnRpbmcNCmhlcmUuIEl0IGlzIGEgY2xp
ZW50IGVuZm9yY2VkIG1vZGVsLCB3aGljaCBpcyBub3QgZXZlciBnb2luZyB0byBiZQ0KY29tcGF0
aWJsZSB3aXRoIHRoZSBORlMgbW9kZWwuDQoNCi0tIA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBO
RlMgY2xpZW50IG1haW50YWluZXIsIEhhbW1lcnNwYWNlDQp0cm9uZC5teWtsZWJ1c3RAaGFtbWVy
c3BhY2UuY29tDQoNCg==

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Thu, May 31, 2018 at 2:47 PM, Trond Myklebust
<trondmy@hammerspace.com> wrote:
> On Thu, 2018-05-31 at 12:00 +0200, Miklos Szeredi wrote:
>> On Thu, May 31, 2018 at 2:45 AM, J. Bruce Fields <bfields@fieldses.or
>> g> wrote:
>> > On Wed, May 30, 2018 at 05:33:11AM -0500, Goldwyn Rodrigues wrote:
>> > > I am not trying to override the security. I am trying to detect
>> > > duplication of security information. The common case of NFS
>> > > communication does not require the additional security parameters
>> > > (doesn't mean it is not required). So my question is: is it
>> > > possible to
>> > > detect at the client that nfs4_acl is a duplicate of information
>> > > which
>> > > can be and is represented by inode alone. If yes, can it be
>> > > suppressed
>> > > by the client.
>> >
>> > No, that's not possible.
>> >
>> > The user's identity could be mapped in various ways.  You've got no
>> > way
>> > to know whether root squashing is in effect, for example.  Or to
>> > know
>> > what the user@EXAMPLE.COM krb5 identity that you're running as
>> > might map
>> > to on the server.
>> >
>> > So it's hard to even tell whether a given user matches the file's
>> > owner
>> > or group.  So even the mode bits are kind of meaningless to the
>> > client.
>>
>> The basic security model for overlayfs is that underlying filesystems
>> are just storage.  Access to these filesystems is done with the
>> capabilities of the task that created the overlay instance with
>> mount(2).  That capability set is saved and used for any access to
>> underlying storage.
>>
>> Access to overlayfs itself is controlled by metadata in the file
>> (mode, uid, gid, posix_acl, security xattr, etc...).
>>
>> So if one of the layers is NFS, the permissions in the server are
>> only
>> checked against the mounter's creds (usually superuser).  Access
>> checks are not performed by the server on behalf of the task
>> accessing
>> the overlay.    This means, that overlayfs could give access to an
>> NFS
>> file, where access on the NFS mount would be denied.  This needs to
>> be
>> understood by the admin mounting the overlay.
>>
>> So how to handle nfs4_acls with this model?
>>
>> We could just ignore them and this can be achieved with mounting the
>> NFS filesystem with "noacl".  I'm not against specifically ignoring
>> nfs4_acl in overlayfs by default, as that seems to be the simplest
>> solution to this problem and fits the overlayfs security model.
>> Later, if we want to make use of this attribute to check access (on
>> the overlay, not in the NFS server), we can add an option to enable
>> this.  But AFAICS that one requires richacl's to make it upstream at
>> least.
>
> 'noacl' does not mean what you think it means. It doesn't mean that the
> NFS security model is changed in any way. Security is still enforced by
> the server.

I understand.  Ignoring nfs4_acl in overlayfs will have the same
result as adding noacl to the underlying NFS mount.

> And no, richacl won't help you get further either.
>
> I'm still in strong disagreement with the model you are presenting
> here. It is a client enforced model, which is not ever going to be
> compatible with the NFS model.

It's the only sane model that overlayfs can do.

Think of it this way:  creating an image file on NFS, formating it to
ext4 and mounting it locally through the loop device is not going to
be compatible with the NFS security model either.  Should we care?

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

On Thu, 2018-05-31 at 14:55 +0200, Miklos Szeredi wrote:
> On Thu, May 31, 2018 at 2:47 PM, Trond Myklebust
> <trondmy@hammerspace.com> wrote:
> > On Thu, 2018-05-31 at 12:00 +0200, Miklos Szeredi wrote:
> > > On Thu, May 31, 2018 at 2:45 AM, J. Bruce Fields <bfields@fieldse
> > > s.or
> > > g> wrote:
> > > > On Wed, May 30, 2018 at 05:33:11AM -0500, Goldwyn Rodrigues
> > > > wrote:
> > > > > I am not trying to override the security. I am trying to
> > > > > detect
> > > > > duplication of security information. The common case of NFS
> > > > > communication does not require the additional security
> > > > > parameters
> > > > > (doesn't mean it is not required). So my question is: is it
> > > > > possible to
> > > > > detect at the client that nfs4_acl is a duplicate of
> > > > > information
> > > > > which
> > > > > can be and is represented by inode alone. If yes, can it be
> > > > > suppressed
> > > > > by the client.
> > > > 
> > > > No, that's not possible.
> > > > 
> > > > The user's identity could be mapped in various ways.  You've
> > > > got no
> > > > way
> > > > to know whether root squashing is in effect, for example.  Or
> > > > to
> > > > know
> > > > what the user@EXAMPLE.COM krb5 identity that you're running as
> > > > might map
> > > > to on the server.
> > > > 
> > > > So it's hard to even tell whether a given user matches the
> > > > file's
> > > > owner
> > > > or group.  So even the mode bits are kind of meaningless to the
> > > > client.
> > > 
> > > The basic security model for overlayfs is that underlying
> > > filesystems
> > > are just storage.  Access to these filesystems is done with the
> > > capabilities of the task that created the overlay instance with
> > > mount(2).  That capability set is saved and used for any access
> > > to
> > > underlying storage.
> > > 
> > > Access to overlayfs itself is controlled by metadata in the file
> > > (mode, uid, gid, posix_acl, security xattr, etc...).
> > > 
> > > So if one of the layers is NFS, the permissions in the server are
> > > only
> > > checked against the mounter's creds (usually superuser).  Access
> > > checks are not performed by the server on behalf of the task
> > > accessing
> > > the overlay.    This means, that overlayfs could give access to
> > > an
> > > NFS
> > > file, where access on the NFS mount would be denied.  This needs
> > > to
> > > be
> > > understood by the admin mounting the overlay.
> > > 
> > > So how to handle nfs4_acls with this model?
> > > 
> > > We could just ignore them and this can be achieved with mounting
> > > the
> > > NFS filesystem with "noacl".  I'm not against specifically
> > > ignoring
> > > nfs4_acl in overlayfs by default, as that seems to be the
> > > simplest
> > > solution to this problem and fits the overlayfs security model.
> > > Later, if we want to make use of this attribute to check access
> > > (on
> > > the overlay, not in the NFS server), we can add an option to
> > > enable
> > > this.  But AFAICS that one requires richacl's to make it upstream
> > > at
> > > least.
> > 
> > 'noacl' does not mean what you think it means. It doesn't mean that
> > the
> > NFS security model is changed in any way. Security is still
> > enforced by
> > the server.
> 
> I understand.  Ignoring nfs4_acl in overlayfs will have the same
> result as adding noacl to the underlying NFS mount.
> 
> > And no, richacl won't help you get further either.
> > 
> > I'm still in strong disagreement with the model you are presenting
> > here. It is a client enforced model, which is not ever going to be
> > compatible with the NFS model.
> 
> It's the only sane model that overlayfs can do.
> 
> Think of it this way:  creating an image file on NFS, formating it to
> ext4 and mounting it locally through the loop device is not going to
> be compatible with the NFS security model either.  Should we care?

Yes you should care because you are proposing that the simple act of
mounting through overlayfs will change who can access, read and modify
existing files from a NFS server.

The model for overlayfs and all unionfs should be that security is
enforced by the underlying filesystem _UNTIL_ the access mode is
modified on the top level filesystem.

IOW: if the user does a chmod, and that is authorised by the underlying
filesystem, then overlayfs is in charge of any further authorisation to
that file.
Adding richacls to that model means that you can attempt to copy the
ACL and allow the user to modify that instead of doing the chmod, but
the understanding should be that it's not the same ACL as was been
enforced by the server, so the copy up of the ACL should be treated as
a modification of the ACL (and should therefore first be subject to
authorisation by the server).

-- 
Trond Myklebust
CTO, Hammerspace Inc
4300 El Camino Real, Suite 105
Los Altos, CA 94022
www.hammer.space id="-x-evo-selection-end-marker">

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

T24gVGh1LCAyMDE4LTA1LTMxIGF0IDE0OjU1ICswMjAwLCBNaWtsb3MgU3plcmVkaSB3cm90ZToN
Cj4gT24gVGh1LCBNYXkgMzEsIDIwMTggYXQgMjo0NyBQTSwgVHJvbmQgTXlrbGVidXN0DQo+IDx0
cm9uZG15QGhhbW1lcnNwYWNlLmNvbT4gd3JvdGU6DQo+ID4gT24gVGh1LCAyMDE4LTA1LTMxIGF0
IDEyOjAwICswMjAwLCBNaWtsb3MgU3plcmVkaSB3cm90ZToNCj4gPiA+IE9uIFRodSwgTWF5IDMx
LCAyMDE4IGF0IDI6NDUgQU0sIEouIEJydWNlIEZpZWxkcyA8YmZpZWxkc0BmaWVsZHNlDQo+ID4g
PiBzLm9yDQo+ID4gPiBnPiB3cm90ZToNCj4gPiA+ID4gT24gV2VkLCBNYXkgMzAsIDIwMTggYXQg
MDU6MzM6MTFBTSAtMDUwMCwgR29sZHd5biBSb2RyaWd1ZXMNCj4gPiA+ID4gd3JvdGU6DQo+ID4g
PiA+ID4gSSBhbSBub3QgdHJ5aW5nIHRvIG92ZXJyaWRlIHRoZSBzZWN1cml0eS4gSSBhbSB0cnlp
bmcgdG8NCj4gPiA+ID4gPiBkZXRlY3QNCj4gPiA+ID4gPiBkdXBsaWNhdGlvbiBvZiBzZWN1cml0
eSBpbmZvcm1hdGlvbi4gVGhlIGNvbW1vbiBjYXNlIG9mIE5GUw0KPiA+ID4gPiA+IGNvbW11bmlj
YXRpb24gZG9lcyBub3QgcmVxdWlyZSB0aGUgYWRkaXRpb25hbCBzZWN1cml0eQ0KPiA+ID4gPiA+
IHBhcmFtZXRlcnMNCj4gPiA+ID4gPiAoZG9lc24ndCBtZWFuIGl0IGlzIG5vdCByZXF1aXJlZCku
IFNvIG15IHF1ZXN0aW9uIGlzOiBpcyBpdA0KPiA+ID4gPiA+IHBvc3NpYmxlIHRvDQo+ID4gPiA+
ID4gZGV0ZWN0IGF0IHRoZSBjbGllbnQgdGhhdCBuZnM0X2FjbCBpcyBhIGR1cGxpY2F0ZSBvZg0K
PiA+ID4gPiA+IGluZm9ybWF0aW9uDQo+ID4gPiA+ID4gd2hpY2gNCj4gPiA+ID4gPiBjYW4gYmUg
YW5kIGlzIHJlcHJlc2VudGVkIGJ5IGlub2RlIGFsb25lLiBJZiB5ZXMsIGNhbiBpdCBiZQ0KPiA+
ID4gPiA+IHN1cHByZXNzZWQNCj4gPiA+ID4gPiBieSB0aGUgY2xpZW50Lg0KPiA+ID4gPiANCj4g
PiA+ID4gTm8sIHRoYXQncyBub3QgcG9zc2libGUuDQo+ID4gPiA+IA0KPiA+ID4gPiBUaGUgdXNl
cidzIGlkZW50aXR5IGNvdWxkIGJlIG1hcHBlZCBpbiB2YXJpb3VzIHdheXMuICBZb3UndmUNCj4g
PiA+ID4gZ290IG5vDQo+ID4gPiA+IHdheQ0KPiA+ID4gPiB0byBrbm93IHdoZXRoZXIgcm9vdCBz
cXVhc2hpbmcgaXMgaW4gZWZmZWN0LCBmb3IgZXhhbXBsZS4gIE9yDQo+ID4gPiA+IHRvDQo+ID4g
PiA+IGtub3cNCj4gPiA+ID4gd2hhdCB0aGUgdXNlckBFWEFNUExFLkNPTSBrcmI1IGlkZW50aXR5
IHRoYXQgeW91J3JlIHJ1bm5pbmcgYXMNCj4gPiA+ID4gbWlnaHQgbWFwDQo+ID4gPiA+IHRvIG9u
IHRoZSBzZXJ2ZXIuDQo+ID4gPiA+IA0KPiA+ID4gPiBTbyBpdCdzIGhhcmQgdG8gZXZlbiB0ZWxs
IHdoZXRoZXIgYSBnaXZlbiB1c2VyIG1hdGNoZXMgdGhlDQo+ID4gPiA+IGZpbGUncw0KPiA+ID4g
PiBvd25lcg0KPiA+ID4gPiBvciBncm91cC4gIFNvIGV2ZW4gdGhlIG1vZGUgYml0cyBhcmUga2lu
ZCBvZiBtZWFuaW5nbGVzcyB0byB0aGUNCj4gPiA+ID4gY2xpZW50Lg0KPiA+ID4gDQo+ID4gPiBU
aGUgYmFzaWMgc2VjdXJpdHkgbW9kZWwgZm9yIG92ZXJsYXlmcyBpcyB0aGF0IHVuZGVybHlpbmcN
Cj4gPiA+IGZpbGVzeXN0ZW1zDQo+ID4gPiBhcmUganVzdCBzdG9yYWdlLiAgQWNjZXNzIHRvIHRo
ZXNlIGZpbGVzeXN0ZW1zIGlzIGRvbmUgd2l0aCB0aGUNCj4gPiA+IGNhcGFiaWxpdGllcyBvZiB0
aGUgdGFzayB0aGF0IGNyZWF0ZWQgdGhlIG92ZXJsYXkgaW5zdGFuY2Ugd2l0aA0KPiA+ID4gbW91
bnQoMikuICBUaGF0IGNhcGFiaWxpdHkgc2V0IGlzIHNhdmVkIGFuZCB1c2VkIGZvciBhbnkgYWNj
ZXNzDQo+ID4gPiB0bw0KPiA+ID4gdW5kZXJseWluZyBzdG9yYWdlLg0KPiA+ID4gDQo+ID4gPiBB
Y2Nlc3MgdG8gb3ZlcmxheWZzIGl0c2VsZiBpcyBjb250cm9sbGVkIGJ5IG1ldGFkYXRhIGluIHRo
ZSBmaWxlDQo+ID4gPiAobW9kZSwgdWlkLCBnaWQsIHBvc2l4X2FjbCwgc2VjdXJpdHkgeGF0dHIs
IGV0Yy4uLikuDQo+ID4gPiANCj4gPiA+IFNvIGlmIG9uZSBvZiB0aGUgbGF5ZXJzIGlzIE5GUywg
dGhlIHBlcm1pc3Npb25zIGluIHRoZSBzZXJ2ZXIgYXJlDQo+ID4gPiBvbmx5DQo+ID4gPiBjaGVj
a2VkIGFnYWluc3QgdGhlIG1vdW50ZXIncyBjcmVkcyAodXN1YWxseSBzdXBlcnVzZXIpLiAgQWNj
ZXNzDQo+ID4gPiBjaGVja3MgYXJlIG5vdCBwZXJmb3JtZWQgYnkgdGhlIHNlcnZlciBvbiBiZWhh
bGYgb2YgdGhlIHRhc2sNCj4gPiA+IGFjY2Vzc2luZw0KPiA+ID4gdGhlIG92ZXJsYXkuICAgIFRo
aXMgbWVhbnMsIHRoYXQgb3ZlcmxheWZzIGNvdWxkIGdpdmUgYWNjZXNzIHRvDQo+ID4gPiBhbg0K
PiA+ID4gTkZTDQo+ID4gPiBmaWxlLCB3aGVyZSBhY2Nlc3Mgb24gdGhlIE5GUyBtb3VudCB3b3Vs
ZCBiZSBkZW5pZWQuICBUaGlzIG5lZWRzDQo+ID4gPiB0bw0KPiA+ID4gYmUNCj4gPiA+IHVuZGVy
c3Rvb2QgYnkgdGhlIGFkbWluIG1vdW50aW5nIHRoZSBvdmVybGF5Lg0KPiA+ID4gDQo+ID4gPiBT
byBob3cgdG8gaGFuZGxlIG5mczRfYWNscyB3aXRoIHRoaXMgbW9kZWw/DQo+ID4gPiANCj4gPiA+
IFdlIGNvdWxkIGp1c3QgaWdub3JlIHRoZW0gYW5kIHRoaXMgY2FuIGJlIGFjaGlldmVkIHdpdGgg
bW91bnRpbmcNCj4gPiA+IHRoZQ0KPiA+ID4gTkZTIGZpbGVzeXN0ZW0gd2l0aCAibm9hY2wiLiAg
SSdtIG5vdCBhZ2FpbnN0IHNwZWNpZmljYWxseQ0KPiA+ID4gaWdub3JpbmcNCj4gPiA+IG5mczRf
YWNsIGluIG92ZXJsYXlmcyBieSBkZWZhdWx0LCBhcyB0aGF0IHNlZW1zIHRvIGJlIHRoZQ0KPiA+
ID4gc2ltcGxlc3QNCj4gPiA+IHNvbHV0aW9uIHRvIHRoaXMgcHJvYmxlbSBhbmQgZml0cyB0aGUg
b3ZlcmxheWZzIHNlY3VyaXR5IG1vZGVsLg0KPiA+ID4gTGF0ZXIsIGlmIHdlIHdhbnQgdG8gbWFr
ZSB1c2Ugb2YgdGhpcyBhdHRyaWJ1dGUgdG8gY2hlY2sgYWNjZXNzDQo+ID4gPiAob24NCj4gPiA+
IHRoZSBvdmVybGF5LCBub3QgaW4gdGhlIE5GUyBzZXJ2ZXIpLCB3ZSBjYW4gYWRkIGFuIG9wdGlv
biB0bw0KPiA+ID4gZW5hYmxlDQo+ID4gPiB0aGlzLiAgQnV0IEFGQUlDUyB0aGF0IG9uZSByZXF1
aXJlcyByaWNoYWNsJ3MgdG8gbWFrZSBpdCB1cHN0cmVhbQ0KPiA+ID4gYXQNCj4gPiA+IGxlYXN0
Lg0KPiA+IA0KPiA+ICdub2FjbCcgZG9lcyBub3QgbWVhbiB3aGF0IHlvdSB0aGluayBpdCBtZWFu
cy4gSXQgZG9lc24ndCBtZWFuIHRoYXQNCj4gPiB0aGUNCj4gPiBORlMgc2VjdXJpdHkgbW9kZWwg
aXMgY2hhbmdlZCBpbiBhbnkgd2F5LiBTZWN1cml0eSBpcyBzdGlsbA0KPiA+IGVuZm9yY2VkIGJ5
DQo+ID4gdGhlIHNlcnZlci4NCj4gDQo+IEkgdW5kZXJzdGFuZC4gIElnbm9yaW5nIG5mczRfYWNs
IGluIG92ZXJsYXlmcyB3aWxsIGhhdmUgdGhlIHNhbWUNCj4gcmVzdWx0IGFzIGFkZGluZyBub2Fj
bCB0byB0aGUgdW5kZXJseWluZyBORlMgbW91bnQuDQo+IA0KPiA+IEFuZCBubywgcmljaGFjbCB3
b24ndCBoZWxwIHlvdSBnZXQgZnVydGhlciBlaXRoZXIuDQo+ID4gDQo+ID4gSSdtIHN0aWxsIGlu
IHN0cm9uZyBkaXNhZ3JlZW1lbnQgd2l0aCB0aGUgbW9kZWwgeW91IGFyZSBwcmVzZW50aW5nDQo+
ID4gaGVyZS4gSXQgaXMgYSBjbGllbnQgZW5mb3JjZWQgbW9kZWwsIHdoaWNoIGlzIG5vdCBldmVy
IGdvaW5nIHRvIGJlDQo+ID4gY29tcGF0aWJsZSB3aXRoIHRoZSBORlMgbW9kZWwuDQo+IA0KPiBJ
dCdzIHRoZSBvbmx5IHNhbmUgbW9kZWwgdGhhdCBvdmVybGF5ZnMgY2FuIGRvLg0KPiANCj4gVGhp
bmsgb2YgaXQgdGhpcyB3YXk6ICBjcmVhdGluZyBhbiBpbWFnZSBmaWxlIG9uIE5GUywgZm9ybWF0
aW5nIGl0IHRvDQo+IGV4dDQgYW5kIG1vdW50aW5nIGl0IGxvY2FsbHkgdGhyb3VnaCB0aGUgbG9v
cCBkZXZpY2UgaXMgbm90IGdvaW5nIHRvDQo+IGJlIGNvbXBhdGlibGUgd2l0aCB0aGUgTkZTIHNl
Y3VyaXR5IG1vZGVsIGVpdGhlci4gIFNob3VsZCB3ZSBjYXJlPw0KDQpZZXMgeW91IHNob3VsZCBj
YXJlIGJlY2F1c2UgeW91IGFyZSBwcm9wb3NpbmcgdGhhdCB0aGUgc2ltcGxlIGFjdCBvZg0KbW91
bnRpbmcgdGhyb3VnaCBvdmVybGF5ZnMgd2lsbCBjaGFuZ2Ugd2hvIGNhbiBhY2Nlc3MsIHJlYWQg
YW5kIG1vZGlmeQ0KZXhpc3RpbmcgZmlsZXMgZnJvbSBhIE5GUyBzZXJ2ZXIuDQoNClRoZSBtb2Rl
bCBmb3Igb3ZlcmxheWZzIGFuZCBhbGwgdW5pb25mcyBzaG91bGQgYmUgdGhhdCBzZWN1cml0eSBp
cw0KZW5mb3JjZWQgYnkgdGhlIHVuZGVybHlpbmcgZmlsZXN5c3RlbSBfVU5USUxfIHRoZSBhY2Nl
c3MgbW9kZSBpcw0KbW9kaWZpZWQgb24gdGhlIHRvcCBsZXZlbCBmaWxlc3lzdGVtLg0KDQpJT1c6
IGlmIHRoZSB1c2VyIGRvZXMgYSBjaG1vZCwgYW5kIHRoYXQgaXMgYXV0aG9yaXNlZCBieSB0aGUg
dW5kZXJseWluZw0KZmlsZXN5c3RlbSwgdGhlbiBvdmVybGF5ZnMgaXMgaW4gY2hhcmdlIG9mIGFu
eSBmdXJ0aGVyIGF1dGhvcmlzYXRpb24gdG8NCnRoYXQgZmlsZS4NCkFkZGluZyByaWNoYWNscyB0
byB0aGF0IG1vZGVsIG1lYW5zIHRoYXQgeW91IGNhbiBhdHRlbXB0IHRvIGNvcHkgdGhlDQpBQ0wg
YW5kIGFsbG93IHRoZSB1c2VyIHRvIG1vZGlmeSB0aGF0IGluc3RlYWQgb2YgZG9pbmcgdGhlIGNo
bW9kLCBidXQNCnRoZSB1bmRlcnN0YW5kaW5nIHNob3VsZCBiZSB0aGF0IGl0J3Mgbm90IHRoZSBz
YW1lIEFDTCBhcyB3YXMgYmVlbg0KZW5mb3JjZWQgYnkgdGhlIHNlcnZlciwgc28gdGhlIGNvcHkg
dXAgb2YgdGhlIEFDTCBzaG91bGQgYmUgdHJlYXRlZCBhcw0KYSBtb2RpZmljYXRpb24gb2YgdGhl
IEFDTCAoYW5kIHNob3VsZCB0aGVyZWZvcmUgZmlyc3QgYmUgc3ViamVjdCB0bw0KYXV0aG9yaXNh
dGlvbiBieSB0aGUgc2VydmVyKS4NCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkNUTywgSGFtbWVy
c3BhY2UgSW5jDQo0MzAwIEVsIENhbWlubyBSZWFsLCBTdWl0ZSAxMDUNCkxvcyBBbHRvcywgQ0Eg
OTQwMjINCnd3dy5oYW1tZXIuc3BhY2UgaWQ9Ii14LWV2by1zZWxlY3Rpb24tZW5kLW1hcmtlciI+

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
<trondmy@hammerspace.com> wrote:
> On Thu, 2018-05-31 at 14:55 +0200, Miklos Szeredi wrote:
>> On Thu, May 31, 2018 at 2:47 PM, Trond Myklebust
>> <trondmy@hammerspace.com> wrote:
>> > On Thu, 2018-05-31 at 12:00 +0200, Miklos Szeredi wrote:
>> > > On Thu, May 31, 2018 at 2:45 AM, J. Bruce Fields <bfields@fieldse
>> > > s.or
>> > > g> wrote:
>> > > > On Wed, May 30, 2018 at 05:33:11AM -0500, Goldwyn Rodrigues
>> > > > wrote:
>> > > > > I am not trying to override the security. I am trying to
>> > > > > detect
>> > > > > duplication of security information. The common case of NFS
>> > > > > communication does not require the additional security
>> > > > > parameters
>> > > > > (doesn't mean it is not required). So my question is: is it
>> > > > > possible to
>> > > > > detect at the client that nfs4_acl is a duplicate of
>> > > > > information
>> > > > > which
>> > > > > can be and is represented by inode alone. If yes, can it be
>> > > > > suppressed
>> > > > > by the client.
>> > > >
>> > > > No, that's not possible.
>> > > >
>> > > > The user's identity could be mapped in various ways.  You've
>> > > > got no
>> > > > way
>> > > > to know whether root squashing is in effect, for example.  Or
>> > > > to
>> > > > know
>> > > > what the user@EXAMPLE.COM krb5 identity that you're running as
>> > > > might map
>> > > > to on the server.
>> > > >
>> > > > So it's hard to even tell whether a given user matches the
>> > > > file's
>> > > > owner
>> > > > or group.  So even the mode bits are kind of meaningless to the
>> > > > client.
>> > >
>> > > The basic security model for overlayfs is that underlying
>> > > filesystems
>> > > are just storage.  Access to these filesystems is done with the
>> > > capabilities of the task that created the overlay instance with
>> > > mount(2).  That capability set is saved and used for any access
>> > > to
>> > > underlying storage.
>> > >
>> > > Access to overlayfs itself is controlled by metadata in the file
>> > > (mode, uid, gid, posix_acl, security xattr, etc...).
>> > >
>> > > So if one of the layers is NFS, the permissions in the server are
>> > > only
>> > > checked against the mounter's creds (usually superuser).  Access
>> > > checks are not performed by the server on behalf of the task
>> > > accessing
>> > > the overlay.    This means, that overlayfs could give access to
>> > > an
>> > > NFS
>> > > file, where access on the NFS mount would be denied.  This needs
>> > > to
>> > > be
>> > > understood by the admin mounting the overlay.
>> > >
>> > > So how to handle nfs4_acls with this model?
>> > >
>> > > We could just ignore them and this can be achieved with mounting
>> > > the
>> > > NFS filesystem with "noacl".  I'm not against specifically
>> > > ignoring
>> > > nfs4_acl in overlayfs by default, as that seems to be the
>> > > simplest
>> > > solution to this problem and fits the overlayfs security model.
>> > > Later, if we want to make use of this attribute to check access
>> > > (on
>> > > the overlay, not in the NFS server), we can add an option to
>> > > enable
>> > > this.  But AFAICS that one requires richacl's to make it upstream
>> > > at
>> > > least.
>> >
>> > 'noacl' does not mean what you think it means. It doesn't mean that
>> > the
>> > NFS security model is changed in any way. Security is still
>> > enforced by
>> > the server.
>>
>> I understand.  Ignoring nfs4_acl in overlayfs will have the same
>> result as adding noacl to the underlying NFS mount.
>>
>> > And no, richacl won't help you get further either.
>> >
>> > I'm still in strong disagreement with the model you are presenting
>> > here. It is a client enforced model, which is not ever going to be
>> > compatible with the NFS model.
>>
>> It's the only sane model that overlayfs can do.
>>
>> Think of it this way:  creating an image file on NFS, formating it to
>> ext4 and mounting it locally through the loop device is not going to
>> be compatible with the NFS security model either.  Should we care?
>
> Yes you should care because you are proposing that the simple act of
> mounting through overlayfs will change who can access, read and modify
> existing files from a NFS server.

Only access/read: NFS can only be read-only layer.  So it's impossible
to actually modify a file on NFS through overlayfs.

>
> The model for overlayfs and all unionfs should be that security is
> enforced by the underlying filesystem _UNTIL_ the access mode is
> modified on the top level filesystem.

How?

We've been through this.  We can't ask an NFS server exported
read-only about what the permission to modify the filesystem would
have been if it were exported read-write.  Sure, the protocol could be
extended, etc, etc...  But it's just not a good fit.

>
> IOW: if the user does a chmod, and that is authorised by the underlying
> filesystem, then overlayfs is in charge of any further authorisation to
> that file.
> Adding richacls to that model means that you can attempt to copy the
> ACL and allow the user to modify that instead of doing the chmod, but
> the understanding should be that it's not the same ACL as was been
> enforced by the server, so the copy up of the ACL should be treated as
> a modification of the ACL (and should therefore first be subject to
> authorisation by the server).

If someone adds the interface for access checking in the NFS client
based on server sercurity model, but without actually having to do the
request, and it works for read-only exports (which make a LOT of sense
for the use cases where overlayfs may be used with NFS) then we can
use that from overlayfs.  Last time Bruce looked this issue, he ran
away screeming, IIRC.

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[bfields]

On Thu, May 31, 2018 at 03:30:04PM +0200, Miklos Szeredi wrote:
> On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
> <trondmy@hammerspace.com> wrote:
> > On Thu, 2018-05-31 at 14:55 +0200, Miklos Szeredi wrote:
> >> On Thu, May 31, 2018 at 2:47 PM, Trond Myklebust <trondmy@hammerspace.com> wrote:
> >> > I'm still in strong disagreement with the model you are presenting
> >> > here. It is a client enforced model, which is not ever going to be
> >> > compatible with the NFS model.
> >>
> >> It's the only sane model that overlayfs can do.
> >>
> >> Think of it this way:  creating an image file on NFS, formating it to
> >> ext4 and mounting it locally through the loop device is not going to
> >> be compatible with the NFS security model either.  Should we care?
> >
> > Yes you should care because you are proposing that the simple act of
> > mounting through overlayfs will change who can access, read and modify
> > existing files from a NFS server.
> 
> Only access/read: NFS can only be read-only layer.  So it's impossible
> to actually modify a file on NFS through overlayfs.

In addition to being read-only, I assume it's also unchanging?  I wonder
why you'd want to use NFS at all for this case--sharing a read-only
ext4-formatted block device would seem more straightforward.

Apologies if we've been through this before too, I've long ago paged out
any previous conversation....

> > The model for overlayfs and all unionfs should be that security is
> > enforced by the underlying filesystem _UNTIL_ the access mode is
> > modified on the top level filesystem.
> 
> How?
> 
> We've been through this.  We can't ask an NFS server exported
> read-only about what the permission to modify the filesystem would
> have been if it were exported read-write.  Sure, the protocol could be
> extended, etc, etc...  But it's just not a good fit.
> 
> >
> > IOW: if the user does a chmod, and that is authorised by the underlying
> > filesystem, then overlayfs is in charge of any further authorisation to
> > that file.
> > Adding richacls to that model means that you can attempt to copy the
> > ACL and allow the user to modify that instead of doing the chmod, but
> > the understanding should be that it's not the same ACL as was been
> > enforced by the server, so the copy up of the ACL should be treated as
> > a modification of the ACL (and should therefore first be subject to
> > authorisation by the server).
> 
> If someone adds the interface for access checking in the NFS client
> based on server sercurity model, but without actually having to do the
> request, and it works for read-only exports (which make a LOT of sense
> for the use cases where overlayfs may be used with NFS) then we can
> use that from overlayfs.  Last time Bruce looked this issue, he ran
> away screeming, IIRC.

In theory I suppose it's all possible, but I think the only practical
thing to do for now is just ignore NFSv4 ACLs.

--b.

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Thu, May 31, 2018 at 4:06 PM, bfields@fieldses.org
<bfields@fieldses.org> wrote:
> On Thu, May 31, 2018 at 03:30:04PM +0200, Miklos Szeredi wrote:
>> On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
>> <trondmy@hammerspace.com> wrote:
>> > On Thu, 2018-05-31 at 14:55 +0200, Miklos Szeredi wrote:
>> >> On Thu, May 31, 2018 at 2:47 PM, Trond Myklebust <trondmy@hammerspace.com> wrote:
>> >> > I'm still in strong disagreement with the model you are presenting
>> >> > here. It is a client enforced model, which is not ever going to be
>> >> > compatible with the NFS model.
>> >>
>> >> It's the only sane model that overlayfs can do.
>> >>
>> >> Think of it this way:  creating an image file on NFS, formating it to
>> >> ext4 and mounting it locally through the loop device is not going to
>> >> be compatible with the NFS security model either.  Should we care?
>> >
>> > Yes you should care because you are proposing that the simple act of
>> > mounting through overlayfs will change who can access, read and modify
>> > existing files from a NFS server.
>>
>> Only access/read: NFS can only be read-only layer.  So it's impossible
>> to actually modify a file on NFS through overlayfs.
>
> In addition to being read-only, I assume it's also unchanging?  I wonder
> why you'd want to use NFS at all for this case--sharing a read-only
> ext4-formatted block device would seem more straightforward.

Actually we could allow changes on the server side.   Overlayfs sort
of does this already (calls underlying fs's ->d_revalidate() from
ovl_dentry_revalidate()).  What's still needed is to detect rename by
rechecking parent and d_name to match overlay dentry's parent and
d_name.

But a shared block dev could be used in most cases, I guess.

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

On Thu, 2018-05-31 at 16:26 +0200, Miklos Szeredi wrote:
> On Thu, May 31, 2018 at 4:06 PM, bfields@fieldses.org
> <bfields@fieldses.org> wrote:
> > On Thu, May 31, 2018 at 03:30:04PM +0200, Miklos Szeredi wrote:
> > > On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
> > > <trondmy@hammerspace.com> wrote:
> > > > On Thu, 2018-05-31 at 14:55 +0200, Miklos Szeredi wrote:
> > > > > On Thu, May 31, 2018 at 2:47 PM, Trond Myklebust <trondmy@ham
> > > > > merspace.com> wrote:
> > > > > > I'm still in strong disagreement with the model you are
> > > > > > presenting
> > > > > > here. It is a client enforced model, which is not ever
> > > > > > going to be
> > > > > > compatible with the NFS model.
> > > > > 
> > > > > It's the only sane model that overlayfs can do.
> > > > > 
> > > > > Think of it this way:  creating an image file on NFS,
> > > > > formating it to
> > > > > ext4 and mounting it locally through the loop device is not
> > > > > going to
> > > > > be compatible with the NFS security model either.  Should we
> > > > > care?
> > > > 
> > > > Yes you should care because you are proposing that the simple
> > > > act of
> > > > mounting through overlayfs will change who can access, read and
> > > > modify
> > > > existing files from a NFS server.
> > > 
> > > Only access/read: NFS can only be read-only layer.  So it's
> > > impossible
> > > to actually modify a file on NFS through overlayfs.
> > 
> > In addition to being read-only, I assume it's also unchanging?  I
> > wonder
> > why you'd want to use NFS at all for this case--sharing a read-only
> > ext4-formatted block device would seem more straightforward.
> 
> Actually we could allow changes on the server side.   Overlayfs sort
> of does this already (calls underlying fs's ->d_revalidate() from
> ovl_dentry_revalidate()).  What's still needed is to detect rename by
> rechecking parent and d_name to match overlay dentry's parent and
> d_name.
> 
> But a shared block dev could be used in most cases, I guess.
> 

Why would you need to export the filesystem as read-only if it is going
to be overlayed anyway? Do the read-only protection on the client side.

-- 
Trond Myklebust
CTO, Hammerspace Inc
4300 El Camino Real, Suite 105
Los Altos, CA 94022
www.hammer.space id="-x-evo-selection-end-marker">

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

T24gVGh1LCAyMDE4LTA1LTMxIGF0IDE2OjI2ICswMjAwLCBNaWtsb3MgU3plcmVkaSB3cm90ZToN
Cj4gT24gVGh1LCBNYXkgMzEsIDIwMTggYXQgNDowNiBQTSwgYmZpZWxkc0BmaWVsZHNlcy5vcmcN
Cj4gPGJmaWVsZHNAZmllbGRzZXMub3JnPiB3cm90ZToNCj4gPiBPbiBUaHUsIE1heSAzMSwgMjAx
OCBhdCAwMzozMDowNFBNICswMjAwLCBNaWtsb3MgU3plcmVkaSB3cm90ZToNCj4gPiA+IE9uIFRo
dSwgTWF5IDMxLCAyMDE4IGF0IDM6MTAgUE0sIFRyb25kIE15a2xlYnVzdA0KPiA+ID4gPHRyb25k
bXlAaGFtbWVyc3BhY2UuY29tPiB3cm90ZToNCj4gPiA+ID4gT24gVGh1LCAyMDE4LTA1LTMxIGF0
IDE0OjU1ICswMjAwLCBNaWtsb3MgU3plcmVkaSB3cm90ZToNCj4gPiA+ID4gPiBPbiBUaHUsIE1h
eSAzMSwgMjAxOCBhdCAyOjQ3IFBNLCBUcm9uZCBNeWtsZWJ1c3QgPHRyb25kbXlAaGFtDQo+ID4g
PiA+ID4gbWVyc3BhY2UuY29tPiB3cm90ZToNCj4gPiA+ID4gPiA+IEknbSBzdGlsbCBpbiBzdHJv
bmcgZGlzYWdyZWVtZW50IHdpdGggdGhlIG1vZGVsIHlvdSBhcmUNCj4gPiA+ID4gPiA+IHByZXNl
bnRpbmcNCj4gPiA+ID4gPiA+IGhlcmUuIEl0IGlzIGEgY2xpZW50IGVuZm9yY2VkIG1vZGVsLCB3
aGljaCBpcyBub3QgZXZlcg0KPiA+ID4gPiA+ID4gZ29pbmcgdG8gYmUNCj4gPiA+ID4gPiA+IGNv
bXBhdGlibGUgd2l0aCB0aGUgTkZTIG1vZGVsLg0KPiA+ID4gPiA+IA0KPiA+ID4gPiA+IEl0J3Mg
dGhlIG9ubHkgc2FuZSBtb2RlbCB0aGF0IG92ZXJsYXlmcyBjYW4gZG8uDQo+ID4gPiA+ID4gDQo+
ID4gPiA+ID4gVGhpbmsgb2YgaXQgdGhpcyB3YXk6ICBjcmVhdGluZyBhbiBpbWFnZSBmaWxlIG9u
IE5GUywNCj4gPiA+ID4gPiBmb3JtYXRpbmcgaXQgdG8NCj4gPiA+ID4gPiBleHQ0IGFuZCBtb3Vu
dGluZyBpdCBsb2NhbGx5IHRocm91Z2ggdGhlIGxvb3AgZGV2aWNlIGlzIG5vdA0KPiA+ID4gPiA+
IGdvaW5nIHRvDQo+ID4gPiA+ID4gYmUgY29tcGF0aWJsZSB3aXRoIHRoZSBORlMgc2VjdXJpdHkg
bW9kZWwgZWl0aGVyLiAgU2hvdWxkIHdlDQo+ID4gPiA+ID4gY2FyZT8NCj4gPiA+ID4gDQo+ID4g
PiA+IFllcyB5b3Ugc2hvdWxkIGNhcmUgYmVjYXVzZSB5b3UgYXJlIHByb3Bvc2luZyB0aGF0IHRo
ZSBzaW1wbGUNCj4gPiA+ID4gYWN0IG9mDQo+ID4gPiA+IG1vdW50aW5nIHRocm91Z2ggb3Zlcmxh
eWZzIHdpbGwgY2hhbmdlIHdobyBjYW4gYWNjZXNzLCByZWFkIGFuZA0KPiA+ID4gPiBtb2RpZnkN
Cj4gPiA+ID4gZXhpc3RpbmcgZmlsZXMgZnJvbSBhIE5GUyBzZXJ2ZXIuDQo+ID4gPiANCj4gPiA+
IE9ubHkgYWNjZXNzL3JlYWQ6IE5GUyBjYW4gb25seSBiZSByZWFkLW9ubHkgbGF5ZXIuICBTbyBp
dCdzDQo+ID4gPiBpbXBvc3NpYmxlDQo+ID4gPiB0byBhY3R1YWxseSBtb2RpZnkgYSBmaWxlIG9u
IE5GUyB0aHJvdWdoIG92ZXJsYXlmcy4NCj4gPiANCj4gPiBJbiBhZGRpdGlvbiB0byBiZWluZyBy
ZWFkLW9ubHksIEkgYXNzdW1lIGl0J3MgYWxzbyB1bmNoYW5naW5nPyAgSQ0KPiA+IHdvbmRlcg0K
PiA+IHdoeSB5b3UnZCB3YW50IHRvIHVzZSBORlMgYXQgYWxsIGZvciB0aGlzIGNhc2UtLXNoYXJp
bmcgYSByZWFkLW9ubHkNCj4gPiBleHQ0LWZvcm1hdHRlZCBibG9jayBkZXZpY2Ugd291bGQgc2Vl
bSBtb3JlIHN0cmFpZ2h0Zm9yd2FyZC4NCj4gDQo+IEFjdHVhbGx5IHdlIGNvdWxkIGFsbG93IGNo
YW5nZXMgb24gdGhlIHNlcnZlciBzaWRlLiAgIE92ZXJsYXlmcyBzb3J0DQo+IG9mIGRvZXMgdGhp
cyBhbHJlYWR5IChjYWxscyB1bmRlcmx5aW5nIGZzJ3MgLT5kX3JldmFsaWRhdGUoKSBmcm9tDQo+
IG92bF9kZW50cnlfcmV2YWxpZGF0ZSgpKS4gIFdoYXQncyBzdGlsbCBuZWVkZWQgaXMgdG8gZGV0
ZWN0IHJlbmFtZSBieQ0KPiByZWNoZWNraW5nIHBhcmVudCBhbmQgZF9uYW1lIHRvIG1hdGNoIG92
ZXJsYXkgZGVudHJ5J3MgcGFyZW50IGFuZA0KPiBkX25hbWUuDQo+IA0KPiBCdXQgYSBzaGFyZWQg
YmxvY2sgZGV2IGNvdWxkIGJlIHVzZWQgaW4gbW9zdCBjYXNlcywgSSBndWVzcy4NCj4gDQoNCldo
eSB3b3VsZCB5b3UgbmVlZCB0byBleHBvcnQgdGhlIGZpbGVzeXN0ZW0gYXMgcmVhZC1vbmx5IGlm
IGl0IGlzIGdvaW5nDQp0byBiZSBvdmVybGF5ZWQgYW55d2F5PyBEbyB0aGUgcmVhZC1vbmx5IHBy
b3RlY3Rpb24gb24gdGhlIGNsaWVudCBzaWRlLg0KDQotLSANClRyb25kIE15a2xlYnVzdA0KQ1RP
LCBIYW1tZXJzcGFjZSBJbmMNCjQzMDAgRWwgQ2FtaW5vIFJlYWwsIFN1aXRlIDEwNQ0KTG9zIEFs
dG9zLCBDQSA5NDAyMg0Kd3d3LmhhbW1lci5zcGFjZSBpZD0iLXgtZXZvLXNlbGVjdGlvbi1lbmQt
bWFya2VyIj4=

Re: nfs4_acl restricts copy_up in overlayfs

[Goldwyn Rodrigues]

On 05/31/2018 09:06 AM, bfields@fieldses.org wrote:
> On Thu, May 31, 2018 at 03:30:04PM +0200, Miklos Szeredi wrote:
>> On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
>> <trondmy@hammerspace.com> wrote:
>>> On Thu, 2018-05-31 at 14:55 +0200, Miklos Szeredi wrote:
>>>> On Thu, May 31, 2018 at 2:47 PM, Trond Myklebust <trondmy@hammerspace.com> wrote:
>>>
>>> IOW: if the user does a chmod, and that is authorised by the underlying
>>> filesystem, then overlayfs is in charge of any further authorisation to
>>> that file.
>>> Adding richacls to that model means that you can attempt to copy the
>>> ACL and allow the user to modify that instead of doing the chmod, but
>>> the understanding should be that it's not the same ACL as was been
>>> enforced by the server, so the copy up of the ACL should be treated as
>>> a modification of the ACL (and should therefore first be subject to
>>> authorisation by the server).
>>
>> If someone adds the interface for access checking in the NFS client
>> based on server sercurity model, but without actually having to do the
>> request, and it works for read-only exports (which make a LOT of sense
>> for the use cases where overlayfs may be used with NFS) then we can
>> use that from overlayfs.  Last time Bruce looked this issue, he ran
>> away screeming, IIRC.
> 
> In theory I suppose it's all possible, but I think the only practical
> thing to do for now is just ignore NFSv4 ACLs.
> 

Ignoring nfs4_acl will override the NFS security model where a user
which is specifically denied read access in the nfs4_acl will get read
access if another user who is allowed to read/write edits the file.

I would agree ignoring NFS4 ACLs is the best option.


-- 
Goldwyn

Re: nfs4_acl restricts copy_up in overlayfs

[Goldwyn Rodrigues]

On 05/31/2018 08:30 AM, Miklos Szeredi wrote:
> On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
> <trondmy@hammerspace.com> wrote:

>>>
>>> I understand.  Ignoring nfs4_acl in overlayfs will have the same
>>> result as adding noacl to the underlying NFS mount.

Adding noacl in NFS client mount has no affect to nfs4_acl. Only if you
add noacl in the underlying filesystem of exported directory in the
server does the nfs4_acl go away.

-- 
Goldwyn

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

On Thu, 2018-05-31 at 16:53 -0500, Goldwyn Rodrigues wrote:
> 
> On 05/31/2018 08:30 AM, Miklos Szeredi wrote:
> > On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
> > <trondmy@hammerspace.com> wrote:
> > > > 
> > > > I understand.  Ignoring nfs4_acl in overlayfs will have the
> > > > same
> > > > result as adding noacl to the underlying NFS mount.
> 
> Adding noacl in NFS client mount has no affect to nfs4_acl. Only if
> you
> add noacl in the underlying filesystem of exported directory in the
> server does the nfs4_acl go away.

That would also be specific to Linux servers.

So if that is your final decision, then why not just state in the
overlayfs manpage that

"Our security model assumes that you are using a NFSv3-only Linux knfsd
based server that has mounted the filesystem with the 'noacl' option,
that exports the filesystem with sec=sys only, has disabled server side
identity squashing and has disabled the --manage-gids mountd option"? 

At least that would be honest...

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

T24gVGh1LCAyMDE4LTA1LTMxIGF0IDE2OjUzIC0wNTAwLCBHb2xkd3luIFJvZHJpZ3VlcyB3cm90
ZToNCj4gDQo+IE9uIDA1LzMxLzIwMTggMDg6MzAgQU0sIE1pa2xvcyBTemVyZWRpIHdyb3RlOg0K
PiA+IE9uIFRodSwgTWF5IDMxLCAyMDE4IGF0IDM6MTAgUE0sIFRyb25kIE15a2xlYnVzdA0KPiA+
IDx0cm9uZG15QGhhbW1lcnNwYWNlLmNvbT4gd3JvdGU6DQo+ID4gPiA+IA0KPiA+ID4gPiBJIHVu
ZGVyc3RhbmQuICBJZ25vcmluZyBuZnM0X2FjbCBpbiBvdmVybGF5ZnMgd2lsbCBoYXZlIHRoZQ0K
PiA+ID4gPiBzYW1lDQo+ID4gPiA+IHJlc3VsdCBhcyBhZGRpbmcgbm9hY2wgdG8gdGhlIHVuZGVy
bHlpbmcgTkZTIG1vdW50Lg0KPiANCj4gQWRkaW5nIG5vYWNsIGluIE5GUyBjbGllbnQgbW91bnQg
aGFzIG5vIGFmZmVjdCB0byBuZnM0X2FjbC4gT25seSBpZg0KPiB5b3UNCj4gYWRkIG5vYWNsIGlu
IHRoZSB1bmRlcmx5aW5nIGZpbGVzeXN0ZW0gb2YgZXhwb3J0ZWQgZGlyZWN0b3J5IGluIHRoZQ0K
PiBzZXJ2ZXIgZG9lcyB0aGUgbmZzNF9hY2wgZ28gYXdheS4NCg0KVGhhdCB3b3VsZCBhbHNvIGJl
IHNwZWNpZmljIHRvIExpbnV4IHNlcnZlcnMuDQoNClNvIGlmIHRoYXQgaXMgeW91ciBmaW5hbCBk
ZWNpc2lvbiwgdGhlbiB3aHkgbm90IGp1c3Qgc3RhdGUgaW4gdGhlDQpvdmVybGF5ZnMgbWFucGFn
ZSB0aGF0DQoNCiJPdXIgc2VjdXJpdHkgbW9kZWwgYXNzdW1lcyB0aGF0IHlvdSBhcmUgdXNpbmcg
YSBORlN2My1vbmx5IExpbnV4IGtuZnNkDQpiYXNlZCBzZXJ2ZXIgdGhhdCBoYXMgbW91bnRlZCB0
aGUgZmlsZXN5c3RlbSB3aXRoIHRoZSAnbm9hY2wnIG9wdGlvbiwNCnRoYXQgZXhwb3J0cyB0aGUg
ZmlsZXN5c3RlbSB3aXRoIHNlYz1zeXMgb25seSwgaGFzIGRpc2FibGVkIHNlcnZlciBzaWRlDQpp
ZGVudGl0eSBzcXVhc2hpbmcgYW5kIGhhcyBkaXNhYmxlZCB0aGUgLS1tYW5hZ2UtZ2lkcyBtb3Vu
dGQgb3B0aW9uIj8gDQoNCkF0IGxlYXN0IHRoYXQgd291bGQgYmUgaG9uZXN0Li4uDQoNCi0tIA0K
VHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXIsIEhhbW1lcnNwYWNl
DQp0cm9uZC5teWtsZWJ1c3RAaGFtbWVyc3BhY2UuY29tDQoNCg==

Re: nfs4_acl restricts copy_up in overlayfs

[Goldwyn Rodrigues]

On 05/31/2018 07:49 PM, Trond Myklebust wrote:
> On Thu, 2018-05-31 at 16:53 -0500, Goldwyn Rodrigues wrote:
>>
>> On 05/31/2018 08:30 AM, Miklos Szeredi wrote:
>>> On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
>>> <trondmy@hammerspace.com> wrote:
>>>>>
>>>>> I understand.  Ignoring nfs4_acl in overlayfs will have the
>>>>> same
>>>>> result as adding noacl to the underlying NFS mount.
>>
>> Adding noacl in NFS client mount has no affect to nfs4_acl. Only if
>> you
>> add noacl in the underlying filesystem of exported directory in the
>> server does the nfs4_acl go away.
> 
> That would also be specific to Linux servers.

Sorry, I don't have access to other NFS based servers. Does that mean
"noacl" option on NFS client mount has different interpretations for
different NFS servers? Or do you mean that nfs4_acl cannot be disabled
for other type of servers?

> 
> So if that is your final decision, then why not just state in the
> overlayfs manpage that

No, that is not my final decision. Neither is it for me to make. I am
merely trying to find a way to make writes on overlayfs possible with
NFSv4 in the lower layer.


-- 
Goldwyn

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

On Fri, 2018-06-01 at 06:40 -0500, Goldwyn Rodrigues wrote:
> 
> On 05/31/2018 07:49 PM, Trond Myklebust wrote:
> > On Thu, 2018-05-31 at 16:53 -0500, Goldwyn Rodrigues wrote:
> > > 
> > > On 05/31/2018 08:30 AM, Miklos Szeredi wrote:
> > > > On Thu, May 31, 2018 at 3:10 PM, Trond Myklebust
> > > > <trondmy@hammerspace.com> wrote:
> > > > > > 
> > > > > > I understand.  Ignoring nfs4_acl in overlayfs will have the
> > > > > > same
> > > > > > result as adding noacl to the underlying NFS mount.
> > > 
> > > Adding noacl in NFS client mount has no affect to nfs4_acl. Only
> > > if
> > > you
> > > add noacl in the underlying filesystem of exported directory in
> > > the
> > > server does the nfs4_acl go away.
> > 
> > That would also be specific to Linux servers.
> 
> Sorry, I don't have access to other NFS based servers. Does that mean
> "noacl" option on NFS client mount has different interpretations for
> different NFS servers? Or do you mean that nfs4_acl cannot be
> disabled
> for other type of servers?

I'm not sure it even makes sense to turn off filesystem acls if your
underlying filesystem is something like zfs or apfs (let alone NTFS).
Linux really is behind the curve here.

> > 
> > So if that is your final decision, then why not just state in the
> > overlayfs manpage that
> 
> No, that is not my final decision. Neither is it for me to make. I am
> merely trying to find a way to make writes on overlayfs possible with
> NFSv4 in the lower layer.
> 

...and all I'm doing is pointing out that as long as you insist on
client enforcement of file security, then you are heavily limiting the
list of servers and server configurations that you will be able to work
safely with. There is a reason why, in all the 30 years since the NFSv2
spec was released, nobody has built such a client.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com

Re: nfs4_acl restricts copy_up in overlayfs

[Trond Myklebust]

T24gRnJpLCAyMDE4LTA2LTAxIGF0IDA2OjQwIC0wNTAwLCBHb2xkd3luIFJvZHJpZ3VlcyB3cm90
ZToNCj4gDQo+IE9uIDA1LzMxLzIwMTggMDc6NDkgUE0sIFRyb25kIE15a2xlYnVzdCB3cm90ZToN
Cj4gPiBPbiBUaHUsIDIwMTgtMDUtMzEgYXQgMTY6NTMgLTA1MDAsIEdvbGR3eW4gUm9kcmlndWVz
IHdyb3RlOg0KPiA+ID4gDQo+ID4gPiBPbiAwNS8zMS8yMDE4IDA4OjMwIEFNLCBNaWtsb3MgU3pl
cmVkaSB3cm90ZToNCj4gPiA+ID4gT24gVGh1LCBNYXkgMzEsIDIwMTggYXQgMzoxMCBQTSwgVHJv
bmQgTXlrbGVidXN0DQo+ID4gPiA+IDx0cm9uZG15QGhhbW1lcnNwYWNlLmNvbT4gd3JvdGU6DQo+
ID4gPiA+ID4gPiANCj4gPiA+ID4gPiA+IEkgdW5kZXJzdGFuZC4gIElnbm9yaW5nIG5mczRfYWNs
IGluIG92ZXJsYXlmcyB3aWxsIGhhdmUgdGhlDQo+ID4gPiA+ID4gPiBzYW1lDQo+ID4gPiA+ID4g
PiByZXN1bHQgYXMgYWRkaW5nIG5vYWNsIHRvIHRoZSB1bmRlcmx5aW5nIE5GUyBtb3VudC4NCj4g
PiA+IA0KPiA+ID4gQWRkaW5nIG5vYWNsIGluIE5GUyBjbGllbnQgbW91bnQgaGFzIG5vIGFmZmVj
dCB0byBuZnM0X2FjbC4gT25seQ0KPiA+ID4gaWYNCj4gPiA+IHlvdQ0KPiA+ID4gYWRkIG5vYWNs
IGluIHRoZSB1bmRlcmx5aW5nIGZpbGVzeXN0ZW0gb2YgZXhwb3J0ZWQgZGlyZWN0b3J5IGluDQo+
ID4gPiB0aGUNCj4gPiA+IHNlcnZlciBkb2VzIHRoZSBuZnM0X2FjbCBnbyBhd2F5Lg0KPiA+IA0K
PiA+IFRoYXQgd291bGQgYWxzbyBiZSBzcGVjaWZpYyB0byBMaW51eCBzZXJ2ZXJzLg0KPiANCj4g
U29ycnksIEkgZG9uJ3QgaGF2ZSBhY2Nlc3MgdG8gb3RoZXIgTkZTIGJhc2VkIHNlcnZlcnMuIERv
ZXMgdGhhdCBtZWFuDQo+ICJub2FjbCIgb3B0aW9uIG9uIE5GUyBjbGllbnQgbW91bnQgaGFzIGRp
ZmZlcmVudCBpbnRlcnByZXRhdGlvbnMgZm9yDQo+IGRpZmZlcmVudCBORlMgc2VydmVycz8gT3Ig
ZG8geW91IG1lYW4gdGhhdCBuZnM0X2FjbCBjYW5ub3QgYmUNCj4gZGlzYWJsZWQNCj4gZm9yIG90
aGVyIHR5cGUgb2Ygc2VydmVycz8NCg0KSSdtIG5vdCBzdXJlIGl0IGV2ZW4gbWFrZXMgc2Vuc2Ug
dG8gdHVybiBvZmYgZmlsZXN5c3RlbSBhY2xzIGlmIHlvdXINCnVuZGVybHlpbmcgZmlsZXN5c3Rl
bSBpcyBzb21ldGhpbmcgbGlrZSB6ZnMgb3IgYXBmcyAobGV0IGFsb25lIE5URlMpLg0KTGludXgg
cmVhbGx5IGlzIGJlaGluZCB0aGUgY3VydmUgaGVyZS4NCg0KPiA+IA0KPiA+IFNvIGlmIHRoYXQg
aXMgeW91ciBmaW5hbCBkZWNpc2lvbiwgdGhlbiB3aHkgbm90IGp1c3Qgc3RhdGUgaW4gdGhlDQo+
ID4gb3ZlcmxheWZzIG1hbnBhZ2UgdGhhdA0KPiANCj4gTm8sIHRoYXQgaXMgbm90IG15IGZpbmFs
IGRlY2lzaW9uLiBOZWl0aGVyIGlzIGl0IGZvciBtZSB0byBtYWtlLiBJIGFtDQo+IG1lcmVseSB0
cnlpbmcgdG8gZmluZCBhIHdheSB0byBtYWtlIHdyaXRlcyBvbiBvdmVybGF5ZnMgcG9zc2libGUg
d2l0aA0KPiBORlN2NCBpbiB0aGUgbG93ZXIgbGF5ZXIuDQo+IA0KDQouLi5hbmQgYWxsIEknbSBk
b2luZyBpcyBwb2ludGluZyBvdXQgdGhhdCBhcyBsb25nIGFzIHlvdSBpbnNpc3Qgb24NCmNsaWVu
dCBlbmZvcmNlbWVudCBvZiBmaWxlIHNlY3VyaXR5LCB0aGVuIHlvdSBhcmUgaGVhdmlseSBsaW1p
dGluZyB0aGUNCmxpc3Qgb2Ygc2VydmVycyBhbmQgc2VydmVyIGNvbmZpZ3VyYXRpb25zIHRoYXQg
eW91IHdpbGwgYmUgYWJsZSB0byB3b3JrDQpzYWZlbHkgd2l0aC4gVGhlcmUgaXMgYSByZWFzb24g
d2h5LCBpbiBhbGwgdGhlIDMwIHllYXJzIHNpbmNlIHRoZSBORlN2Mg0Kc3BlYyB3YXMgcmVsZWFz
ZWQsIG5vYm9keSBoYXMgYnVpbHQgc3VjaCBhIGNsaWVudC4NCg0KLS0gDQpUcm9uZCBNeWtsZWJ1
c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lciwgSGFtbWVyc3BhY2UNCnRyb25kLm15a2xl
YnVzdEBoYW1tZXJzcGFjZS5jb20NCg0K

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Fri, Jun 1, 2018 at 3:16 PM, Trond Myklebust <trondmy@hammerspace.com> wrote:

> ...and all I'm doing is pointing out that as long as you insist on
> client enforcement of file security, then you are heavily limiting the
> list of servers and server configurations that you will be able to work
> safely with. There is a reason why, in all the 30 years since the NFSv2
> spec was released, nobody has built such a client.

How do you define "safely"?

Is it safe for root to do

  cp -a /nfs/remotedir /tmp/localdir

?

That's essentially what an overlayfs mount with an NFS layer does with
respect to access permissions:

 - remote files are not modifiable to anyone, unless server allows

 - remote files *readable to root* will provide access based on local DAC check.

Does that need to be made clear in the docs?  Surely.  But it does NOT
mean it's dangerous or that it's not useful with an arbitrary NFS
server (although my guess is that 99% will involve knfsd).

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[bfields]

On Fri, Jun 01, 2018 at 03:32:59PM +0200, Miklos Szeredi wrote:
> How do you define "safely"?
> 
> Is it safe for root to do
> 
>   cp -a /nfs/remotedir /tmp/localdir
> 
> ?
> 
> That's essentially what an overlayfs mount with an NFS layer does with
> respect to access permissions:
> 
>  - remote files are not modifiable to anyone, unless server allows
> 
>  - remote files *readable to root* will provide access based on local DAC check.
> 
> Does that need to be made clear in the docs?  Surely.  But it does NOT
> mean it's dangerous or that it's not useful with an arbitrary NFS
> server

We should definitely have clear documentation, but despite that, in
practice lots of people *will* be surprised when permissions are
enforced differently after copy-up, and those surprises may well have
unpleasant implications.

And the differences will depend on details of the server implementation,
which makes documenting the behavior and giving people the correct
expectations that much more complicated.

> (although my guess is that 99% will involve knfsd).

Why is that?  There are a *lot* of Linux clients out there talking to
non-knfsd servers.  I wouldn't have thought knfsd is even a
majority--but of course it depends very much on what kind of use exactly
we're measuring.  And I don't have any numbers.

--b.

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Fri, Jun 1, 2018 at 3:50 PM, bfields@fieldses.org
<bfields@fieldses.org> wrote:
> On Fri, Jun 01, 2018 at 03:32:59PM +0200, Miklos Szeredi wrote:
>> How do you define "safely"?
>>
>> Is it safe for root to do
>>
>>   cp -a /nfs/remotedir /tmp/localdir
>>
>> ?
>>
>> That's essentially what an overlayfs mount with an NFS layer does with
>> respect to access permissions:
>>
>>  - remote files are not modifiable to anyone, unless server allows
>>
>>  - remote files *readable to root* will provide access based on local DAC check.
>>
>> Does that need to be made clear in the docs?  Surely.  But it does NOT
>> mean it's dangerous or that it's not useful with an arbitrary NFS
>> server
>
> We should definitely have clear documentation, but despite that, in
> practice lots of people *will* be surprised when permissions are
> enforced differently after copy-up, and those surprises may well have
> unpleasant implications.

Permissions are enforced exactly the same before and after copy-up.
That's one of the good points in doing the permission checks locally.

That "cp -a /nfs/remotedir /tmp/localdir" example is almost exactly
equivalent to:

  mount -t overlay -olowerdir=/nfs/remotedir,upperdir=/tmp/upper,...
/tmp/localdir

except the copy is delayed until modification.

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[bfields]

On Fri, Jun 01, 2018 at 04:00:22PM +0200, Miklos Szeredi wrote:
> On Fri, Jun 1, 2018 at 3:50 PM, bfields@fieldses.org
> <bfields@fieldses.org> wrote:
> > On Fri, Jun 01, 2018 at 03:32:59PM +0200, Miklos Szeredi wrote:
> >> How do you define "safely"?
> >>
> >> Is it safe for root to do
> >>
> >>   cp -a /nfs/remotedir /tmp/localdir
> >>
> >> ?
> >>
> >> That's essentially what an overlayfs mount with an NFS layer does with
> >> respect to access permissions:
> >>
> >>  - remote files are not modifiable to anyone, unless server allows
> >>
> >>  - remote files *readable to root* will provide access based on local DAC check.
> >>
> >> Does that need to be made clear in the docs?  Surely.  But it does NOT
> >> mean it's dangerous or that it's not useful with an arbitrary NFS
> >> server
> >
> > We should definitely have clear documentation, but despite that, in
> > practice lots of people *will* be surprised when permissions are
> > enforced differently after copy-up, and those surprises may well have
> > unpleasant implications.
> 
> Permissions are enforced exactly the same before and after copy-up.
> That's one of the good points in doing the permission checks locally.

Whoops, sorry, I missed that.  So you always read owners and mode bits
out of the cached inode and used those to check permissions instead of
calling access?

That still sounds pretty confusing.  E.g. if the server's squashing root
to a user without permission to read a file, you'll pass local
permission checks, but the success a given read may actually depend on
whether the data's already cached?  I feel like I still must not
understand something.  Sorry if I'm making these threads repeat
themselves....

--b.

> 
> That "cp -a /nfs/remotedir /tmp/localdir" example is almost exactly
> equivalent to:
> 
>   mount -t overlay -olowerdir=/nfs/remotedir,upperdir=/tmp/upper,...
> /tmp/localdir
> 
> except the copy is delayed until modification.
> 
> Thanks,
> Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Fri, Jun 1, 2018 at 4:26 PM, bfields@fieldses.org
<bfields@fieldses.org> wrote:
> On Fri, Jun 01, 2018 at 04:00:22PM +0200, Miklos Szeredi wrote:
>> On Fri, Jun 1, 2018 at 3:50 PM, bfields@fieldses.org
>> <bfields@fieldses.org> wrote:
>> > On Fri, Jun 01, 2018 at 03:32:59PM +0200, Miklos Szeredi wrote:
>> >> How do you define "safely"?
>> >>
>> >> Is it safe for root to do
>> >>
>> >>   cp -a /nfs/remotedir /tmp/localdir
>> >>
>> >> ?
>> >>
>> >> That's essentially what an overlayfs mount with an NFS layer does with
>> >> respect to access permissions:
>> >>
>> >>  - remote files are not modifiable to anyone, unless server allows
>> >>
>> >>  - remote files *readable to root* will provide access based on local DAC check.
>> >>
>> >> Does that need to be made clear in the docs?  Surely.  But it does NOT
>> >> mean it's dangerous or that it's not useful with an arbitrary NFS
>> >> server
>> >
>> > We should definitely have clear documentation, but despite that, in
>> > practice lots of people *will* be surprised when permissions are
>> > enforced differently after copy-up, and those surprises may well have
>> > unpleasant implications.
>>
>> Permissions are enforced exactly the same before and after copy-up.
>> That's one of the good points in doing the permission checks locally.
>
> Whoops, sorry, I missed that.  So you always read owners and mode bits
> out of the cached inode and used those to check permissions instead of
> calling access?
>
> That still sounds pretty confusing.  E.g. if the server's squashing root
> to a user without permission to read a file, you'll pass local
> permission checks, but the success a given read may actually depend on
> whether the data's already cached?

You have a point there.  I think current code can be inconsistent like
that.  But that's only because it doesn't stack file operations.
Stacking f_ops is now queued up for 4.18, which means that *all* calls
into underlying layers should be with the same creds (those of the
mounting task), regardless of the creds of the task performing the
operation.

So if NFS server is denying read to mounter (because of root squashing
or for other reason), then that file will not be accessible from
overlayfs by anyone and will not be in the cache either.  If access to
mounter is allowed, then the access will be based on local DAC.

Look at ovl_permission(), I think it pretty clearly describes this model.

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[bfields]

On Fri, Jun 01, 2018 at 04:43:51PM +0200, Miklos Szeredi wrote:
> On Fri, Jun 1, 2018 at 4:26 PM, bfields@fieldses.org
> <bfields@fieldses.org> wrote:
> > On Fri, Jun 01, 2018 at 04:00:22PM +0200, Miklos Szeredi wrote:
> >> On Fri, Jun 1, 2018 at 3:50 PM, bfields@fieldses.org
> >> <bfields@fieldses.org> wrote:
> >> > On Fri, Jun 01, 2018 at 03:32:59PM +0200, Miklos Szeredi wrote:
> >> >> How do you define "safely"?
> >> >>
> >> >> Is it safe for root to do
> >> >>
> >> >>   cp -a /nfs/remotedir /tmp/localdir
> >> >>
> >> >> ?
> >> >>
> >> >> That's essentially what an overlayfs mount with an NFS layer does with
> >> >> respect to access permissions:
> >> >>
> >> >>  - remote files are not modifiable to anyone, unless server allows
> >> >>
> >> >>  - remote files *readable to root* will provide access based on local DAC check.
> >> >>
> >> >> Does that need to be made clear in the docs?  Surely.  But it does NOT
> >> >> mean it's dangerous or that it's not useful with an arbitrary NFS
> >> >> server
> >> >
> >> > We should definitely have clear documentation, but despite that, in
> >> > practice lots of people *will* be surprised when permissions are
> >> > enforced differently after copy-up, and those surprises may well have
> >> > unpleasant implications.
> >>
> >> Permissions are enforced exactly the same before and after copy-up.
> >> That's one of the good points in doing the permission checks locally.
> >
> > Whoops, sorry, I missed that.  So you always read owners and mode bits
> > out of the cached inode and used those to check permissions instead of
> > calling access?
> >
> > That still sounds pretty confusing.  E.g. if the server's squashing root
> > to a user without permission to read a file, you'll pass local
> > permission checks, but the success a given read may actually depend on
> > whether the data's already cached?
> 
> You have a point there.  I think current code can be inconsistent like
> that.  But that's only because it doesn't stack file operations.
> Stacking f_ops is now queued up for 4.18, which means that *all* calls
> into underlying layers should be with the same creds (those of the
> mounting task), regardless of the creds of the task performing the
> operation.
> 
> So if NFS server is denying read to mounter (because of root squashing
> or for other reason), then that file will not be accessible from
> overlayfs by anyone and will not be in the cache either.  If access to
> mounter is allowed, then the access will be based on local DAC.
> 
> Look at ovl_permission(), I think it pretty clearly describes this model.

Thanks!  Uh, so generic_permission is the thing that just does the usual
mode/acl checks on the in-core inode, and inode_permission is the one
that also calls into the filesystem?

But I'm still a little confused--if I'm reading right, "realinode" is
the lower inode before copyup, and the upper inode after, so can't
inode_permission(realinode, mask) return different results before and
after copyup?

--b.

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Fri, Jun 1, 2018 at 6:08 PM, bfields@fieldses.org
<bfields@fieldses.org> wrote:
> On Fri, Jun 01, 2018 at 04:43:51PM +0200, Miklos Szeredi wrote:
>> On Fri, Jun 1, 2018 at 4:26 PM, bfields@fieldses.org
>> <bfields@fieldses.org> wrote:
>> > On Fri, Jun 01, 2018 at 04:00:22PM +0200, Miklos Szeredi wrote:
>> >> On Fri, Jun 1, 2018 at 3:50 PM, bfields@fieldses.org
>> >> <bfields@fieldses.org> wrote:
>> >> > On Fri, Jun 01, 2018 at 03:32:59PM +0200, Miklos Szeredi wrote:
>> >> >> How do you define "safely"?
>> >> >>
>> >> >> Is it safe for root to do
>> >> >>
>> >> >>   cp -a /nfs/remotedir /tmp/localdir
>> >> >>
>> >> >> ?
>> >> >>
>> >> >> That's essentially what an overlayfs mount with an NFS layer does with
>> >> >> respect to access permissions:
>> >> >>
>> >> >>  - remote files are not modifiable to anyone, unless server allows
>> >> >>
>> >> >>  - remote files *readable to root* will provide access based on local DAC check.
>> >> >>
>> >> >> Does that need to be made clear in the docs?  Surely.  But it does NOT
>> >> >> mean it's dangerous or that it's not useful with an arbitrary NFS
>> >> >> server
>> >> >
>> >> > We should definitely have clear documentation, but despite that, in
>> >> > practice lots of people *will* be surprised when permissions are
>> >> > enforced differently after copy-up, and those surprises may well have
>> >> > unpleasant implications.
>> >>
>> >> Permissions are enforced exactly the same before and after copy-up.
>> >> That's one of the good points in doing the permission checks locally.
>> >
>> > Whoops, sorry, I missed that.  So you always read owners and mode bits
>> > out of the cached inode and used those to check permissions instead of
>> > calling access?
>> >
>> > That still sounds pretty confusing.  E.g. if the server's squashing root
>> > to a user without permission to read a file, you'll pass local
>> > permission checks, but the success a given read may actually depend on
>> > whether the data's already cached?
>>
>> You have a point there.  I think current code can be inconsistent like
>> that.  But that's only because it doesn't stack file operations.
>> Stacking f_ops is now queued up for 4.18, which means that *all* calls
>> into underlying layers should be with the same creds (those of the
>> mounting task), regardless of the creds of the task performing the
>> operation.
>>
>> So if NFS server is denying read to mounter (because of root squashing
>> or for other reason), then that file will not be accessible from
>> overlayfs by anyone and will not be in the cache either.  If access to
>> mounter is allowed, then the access will be based on local DAC.
>>
>> Look at ovl_permission(), I think it pretty clearly describes this model.
>
> Thanks!  Uh, so generic_permission is the thing that just does the usual
> mode/acl checks on the in-core inode, and inode_permission is the one
> that also calls into the filesystem?

Right.

> But I'm still a little confused--if I'm reading right, "realinode" is
> the lower inode before copyup, and the upper inode after, so can't
> inode_permission(realinode, mask) return different results before and
> after copyup?

Theoretically, yes.  Not in any sane setup, though.

The inode_permission() checks on realinode are for making sure the
mounter cannot gain undue privileges (will be especially important
with userns mounts).

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[bfields]

On Fri, Jun 01, 2018 at 07:02:20PM +0200, Miklos Szeredi wrote:
> On Fri, Jun 1, 2018 at 6:08 PM, bfields@fieldses.org
> <bfields@fieldses.org> wrote:
> > On Fri, Jun 01, 2018 at 04:43:51PM +0200, Miklos Szeredi wrote:
> >> Look at ovl_permission(), I think it pretty clearly describes this model.
> >
> > Thanks!  Uh, so generic_permission is the thing that just does the usual
> > mode/acl checks on the in-core inode, and inode_permission is the one
> > that also calls into the filesystem?
> 
> Right.
> 
> > But I'm still a little confused--if I'm reading right, "realinode" is
> > the lower inode before copyup, and the upper inode after, so can't
> > inode_permission(realinode, mask) return different results before and
> > after copyup?
> 
> Theoretically, yes.  Not in any sane setup, though.

If root squashing is enabled and you mount as root, then it will change.

That's not an unlikely case, it's pretty much the default.

--b.

> 
> The inode_permission() checks on realinode are for making sure the
> mounter cannot gain undue privileges (will be especially important
> with userns mounts).

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Fri, Jun 1, 2018 at 7:43 PM, bfields@fieldses.org
<bfields@fieldses.org> wrote:
> On Fri, Jun 01, 2018 at 07:02:20PM +0200, Miklos Szeredi wrote:
>> On Fri, Jun 1, 2018 at 6:08 PM, bfields@fieldses.org
>> <bfields@fieldses.org> wrote:
>> > On Fri, Jun 01, 2018 at 04:43:51PM +0200, Miklos Szeredi wrote:
>> >> Look at ovl_permission(), I think it pretty clearly describes this model.
>> >
>> > Thanks!  Uh, so generic_permission is the thing that just does the usual
>> > mode/acl checks on the in-core inode, and inode_permission is the one
>> > that also calls into the filesystem?
>>
>> Right.
>>
>> > But I'm still a little confused--if I'm reading right, "realinode" is
>> > the lower inode before copyup, and the upper inode after, so can't
>> > inode_permission(realinode, mask) return different results before and
>> > after copyup?
>>
>> Theoretically, yes.  Not in any sane setup, though.
>
> If root squashing is enabled and you mount as root, then it will change.
>
> That's not an unlikely case, it's pretty much the default.

Let's see: root squashing will change root creds to nobody's creds, right?

That results in a weird situation, where user foo is trying to access
a file owned by foo, but which doesn't have read permission for
"other" will not be able to perform a read on that file.  In fact, no
user will be able to read that file, including root.   Copying up the
file will also fail, since that requires read permission.

So for that case it's not true that permission will change, since
copy-up won't be possible.

That leaves the case where no read or write permissions are involved,
which is execute.  I think we should just mask that out from the
inode_permission() check.  Makes no sense to ask underlying filesystem
about execute permission.   With that we'll have consistent
permissions before and after copy-up.  Right?

Thanks,
Miklos

Re: nfs4_acl restricts copy_up in overlayfs

[bfields]

On Fri, Jun 01, 2018 at 09:14:38PM +0200, Miklos Szeredi wrote:
> On Fri, Jun 1, 2018 at 7:43 PM, bfields@fieldses.org
> <bfields@fieldses.org> wrote:
> > On Fri, Jun 01, 2018 at 07:02:20PM +0200, Miklos Szeredi wrote:
> >> On Fri, Jun 1, 2018 at 6:08 PM, bfields@fieldses.org
> >> <bfields@fieldses.org> wrote:
> >> > On Fri, Jun 01, 2018 at 04:43:51PM +0200, Miklos Szeredi wrote:
> >> >> Look at ovl_permission(), I think it pretty clearly describes this model.
> >> >
> >> > Thanks!  Uh, so generic_permission is the thing that just does the usual
> >> > mode/acl checks on the in-core inode, and inode_permission is the one
> >> > that also calls into the filesystem?
> >>
> >> Right.
> >>
> >> > But I'm still a little confused--if I'm reading right, "realinode" is
> >> > the lower inode before copyup, and the upper inode after, so can't
> >> > inode_permission(realinode, mask) return different results before and
> >> > after copyup?
> >>
> >> Theoretically, yes.  Not in any sane setup, though.
> >
> > If root squashing is enabled and you mount as root, then it will change.
> >
> > That's not an unlikely case, it's pretty much the default.
> 
> Let's see: root squashing will change root creds to nobody's creds, right?
> 
> That results in a weird situation, where user foo is trying to access
> a file owned by foo, but which doesn't have read permission for
> "other" will not be able to perform a read on that file.  In fact, no
> user will be able to read that file, including root.   Copying up the
> file will also fail, since that requires read permission.
> 
> So for that case it's not true that permission will change, since
> copy-up won't be possible.

Fair enough.  That's still pretty weird behavior.

Arguably if you're going to do something different here then maybe it's
good for the difference to manifest in a way that's really obvious in
common cases, so people don't just overlook it.

> That leaves the case where no read or write permissions are involved,
> which is execute.  I think we should just mask that out from the
> inode_permission() check.  Makes no sense to ask underlying filesystem
> about execute permission.

I suppose so.

> With that we'll have consistent permissions before and after copy-up.
> Right?

Hm.  If inode_permission(realinode, mask) changes across copy-up: 

	- if the difference is in execute, you're telling me we'll
	  ignore that.
	- if it changes from denying to allowing read, you don't care
	  since that will prevent the copy up in the first place.
	- if it changes from denying to allowing write, that has no
	  effect since you only care about read permission for copyup in
	  the case someone requests write permission.  (Hm, the special
	  file case should be weird.  I guess those just don't get
	  copied up?  Still I think people will be surprised by
	  permission failures here.)
	- if it changes from allowing read or write to denying--then
	  we're no longer talking about root, assuming the permission
	  check on the copied-up inode wouldn't fail for root.  We don't
	  currently allow non-root to mount NFS, but there are people
	  that would like to see that change.

--b.

Re: nfs4_acl restricts copy_up in overlayfs

[Miklos Szeredi]

On Sat, Jun 2, 2018 at 2:50 AM, bfields@fieldses.org
<bfields@fieldses.org> wrote:
> On Fri, Jun 01, 2018 at 09:14:38PM +0200, Miklos Szeredi wrote:
>> On Fri, Jun 1, 2018 at 7:43 PM, bfields@fieldses.org
>> <bfields@fieldses.org> wrote:
>> > On Fri, Jun 01, 2018 at 07:02:20PM +0200, Miklos Szeredi wrote:
>> >> On Fri, Jun 1, 2018 at 6:08 PM, bfields@fieldses.org
>> >> <bfields@fieldses.org> wrote:
>> >> > On Fri, Jun 01, 2018 at 04:43:51PM +0200, Miklos Szeredi wrote:
>> >> >> Look at ovl_permission(), I think it pretty clearly describes this model.
>> >> >
>> >> > Thanks!  Uh, so generic_permission is the thing that just does the usual
>> >> > mode/acl checks on the in-core inode, and inode_permission is the one
>> >> > that also calls into the filesystem?
>> >>
>> >> Right.
>> >>
>> >> > But I'm still a little confused--if I'm reading right, "realinode" is
>> >> > the lower inode before copyup, and the upper inode after, so can't
>> >> > inode_permission(realinode, mask) return different results before and
>> >> > after copyup?
>> >>
>> >> Theoretically, yes.  Not in any sane setup, though.
>> >
>> > If root squashing is enabled and you mount as root, then it will change.
>> >
>> > That's not an unlikely case, it's pretty much the default.
>>
>> Let's see: root squashing will change root creds to nobody's creds, right?
>>
>> That results in a weird situation, where user foo is trying to access
>> a file owned by foo, but which doesn't have read permission for
>> "other" will not be able to perform a read on that file.  In fact, no
>> user will be able to read that file, including root.   Copying up the
>> file will also fail, since that requires read permission.
>>
>> So for that case it's not true that permission will change, since
>> copy-up won't be possible.
>
> Fair enough.  That's still pretty weird behavior.
>
> Arguably if you're going to do something different here then maybe it's
> good for the difference to manifest in a way that's really obvious in
> common cases, so people don't just overlook it.
>
>> That leaves the case where no read or write permissions are involved,
>> which is execute.  I think we should just mask that out from the
>> inode_permission() check.  Makes no sense to ask underlying filesystem
>> about execute permission.
>
> I suppose so.
>
>> With that we'll have consistent permissions before and after copy-up.
>> Right?
>
> Hm.  If inode_permission(realinode, mask) changes across copy-up:
>
>         - if the difference is in execute, you're telling me we'll
>           ignore that.
>         - if it changes from denying to allowing read, you don't care
>           since that will prevent the copy up in the first place.
>         - if it changes from denying to allowing write, that has no
>           effect since you only care about read permission for copyup in
>           the case someone requests write permission.  (Hm, the special
>           file case should be weird.  I guess those just don't get
>           copied up?  Still I think people will be surprised by
>           permission failures here.)

Special file can get copied up on metadata change.  We shoudln't ask
server at all for permissions in this case.  That would be consistent
with the "cp -a" model.

>         - if it changes from allowing read or write to denying--then
>           we're no longer talking about root, assuming the permission
>           check on the copied-up inode wouldn't fail for root.  We don't
>           currently allow non-root to mount NFS, but there are people
>           that would like to see that change.

The "cp -a" model should work even for userns unprivileged mounts.  We
do assume a certain sanity with setups: I'm sure someone could come up
with a setup (e.g. with selinux) where weirdness happens after
copy-up.

Thanks,
Miklos



>
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nfs4_acl restricts copy_up in overlayfs

[J. R. Okajima]

Trond Myklebust:
> The model for overlayfs and all unionfs should be that security is
> enforced by the underlying filesystem _UNTIL_ the access mode is
> modified on the top level filesystem.

Agreed.
For your information, here is aufs's fundamental approach.
- don't change the task credential.
- support the writable NFS layer.
- respect the layer fs's security model.

In the case of "upper RW non-NFS + lower RO NFS layers" combination,
nfs4_acl can be a problem in the internal (copy,move)-(up,down). Yes,
this is common to overlayfs. In order to address this, aufs introduced a
rather generic solution called ICEX (Ignore Copy-Error on Xattr) and
specifying the mount option saves users from the internal error.
Still I don't think this is best and ideal solution, but the
"ignoring conditially" is a realistic solution.


(from aufs manual)
----------------------------------------
.TP
.B icexsec | icexsys | icextr | icexusr | icexoth | icex
Ignore the error on copying\-up/down XATTR.
When an internal copy\-up/down happens, aufs tries copying all XATTRs.
Here an error can happen because of the XATTR support on the dst
branch may different from the src branch. If you know how the branch
supports or unsupports XATTR, you can specify these attributes.
`icexsec' means to ignore an error on copying\-up/down XATTR categorized
as "security" (for LSM and capability). And `icexsys,' `icextr,' and
`icexusr,' are for "system" (for posix ACL), "trusted" and "user"
categories individually.
`icexoth' is for any other category. To be convenient, `icex` sets them
all.
See also linux/Documentation/filesystems/aufs/design/06xattr.txt.

(from linux/Documentation/filesystems/aufs/design/06xattr.txt in aufs source tree)
----------------------------------------
Generally the extended attributes of inode are categorized as these.
- "security" for LSM and capability.
- "system" for posix ACL, 'acl' mount option is required for the branch
  fs generally.
- "trusted" for userspace, CAP_SYS_ADMIN is required.
- "user" for userspace, 'user_xattr' mount option is required for the
  branch fs generally.

Moreover there are some other categories. Aufs handles these rather
unpopular categories as the ordinary ones, ie. there is no special
condition nor exception.

In copy-up, the support for XATTR on the dst branch may differ from the
src branch. In this case, the copy-up operation will get an error and
the original user operation which triggered the copy-up will fail. It
can happen that even all copy-up will fail.
When both of src and dst branches support XATTR and if an error occurs
during copying XATTR, then the copy-up should fail obviously. That is a
good reason and aufs should return an error to userspace. But when only
the src branch support that XATTR, aufs should not return an error.
For example, the src branch supports ACL but the dst branch doesn't
because the dst branch may natively un-support it or temporary
un-support it due to "noacl" mount option. Of course, the dst branch fs
may NOT return an error even if the XATTR is not supported. It is
totally up to the branch fs.

Anyway when the aufs internal copy-up gets an error from the dst branch
fs, then aufs tries removing the just copied entry and returns the error
to the userspace. The worst case of this situation will be all copy-up
will fail.

For the copy-up operation, there two basic approaches.
- copy the specified XATTR only (by category above), and return the
  error unconditionally if it happens.
- copy all XATTR, and ignore the error on the specified category only.

In order to support XATTR and to implement the correct behaviour, aufs
chooses the latter approach and introduces some new branch attributes,
"icexsec", "icexsys", "icextr", "icexusr", and "icexoth".
They correspond to the XATTR namespaces (see above). Additionally, to be
convenient, "icex" is also provided which means all "icex*" attributes
are set (here the word "icex" stands for "ignore copy-error on XATTR").

The meaning of these attributes is to ignore the error from setting
XATTR on that branch.
Note that aufs tries copying all XATTR unconditionally, and ignores the
error from the dst branch according to the specified attributes.

Some XATTR may have its default value. The default value may come from
the parent dir or the environment. If the default value is set at the
file creating-time, it will be overwritten by copy-up.
Some contradiction may happen I am afraid.
Do we need another attribute to stop copying XATTR? I am unsure. For
now, aufs implements the branch attributes to ignore the error.



J. R. Okajima