Interoperable junctions on Linux

Interoperable junctions on Linux

[Chuck Lever]

Hi-

I led a discussion on Friday at the Linux Storage and Filesystem Summit on how to store {DFS, FedFS} junctions in Linux filesystems.  I'd like to summarize the discussion, then ask a few follow-up questions.  I apologize in advance for the length.

FedFS is to NFS as Microsoft DFS is to SMB/CIFS.  FedFS uses NFS referrals to glue together a file namespace out of separate shares, starting with a root share that may contain nothing but referrals.  For more on FedFS, start with RFC 5716.

The physical object that stores referral location target information is called a junction.  It can contain an actual list of locations, or it can contain the DN of an LDAP record where the location list is maintained.

 +  Samba uses a symlink for this purpose.  The contents of the link represent the location information passed out to CIFS clients.

 +  FedFS uses an extended attribute on a directory for this purpose.  The xattr contains XML which encodes location information passed out to NFS clients.

The reasons for this difference are simply historical.

Linux is often used as a multi-protocol file service platform, where the same physical data is presented to clients via both NFS and CIFS.  For this reason, we think there would be value in using the same physical representation for both NFSD and Samba so that a single junction object on our exported filesystems can trigger a referral for both NFS and CIFS.

Samba has been around much longer, and DFS support is actually deployed.  FedFS is newer and still experimental.  Thus we decided to change FedFS to use a symlink instead of a directory.  Samba will still use the regular contents of the symlink, and FedFS will store its metadata in an extended attribute attached to the symlink.

There was a rough consensus to prefer JSON over XML in the FedFS xattr, though there are still some who dislike both.  I'm open to suggestion, since we're now essentially replacing the existing FedFS junction format and can change it to anything we like.

Today FedFS junctions can contain either a location list or an LDAP DN.  One option for FedFS is to support only the LDAP DN junction type, and have a (possibly local) LDAP service available to store the location information.  The FedFS junction xattr would then always contain an LDAP URL.  Storing complex data types (a list containing pathnames, hostnames, integers, and other values) would then be up to LDAP.

We will have to discuss a conjunction of administrative interfaces at some later point.  However, we should clarify how our junction management tools behave now that each junction can have metadata it did not have before.

FedFS:

  nfsref add - if no symlink exists, create it (what contents should it have?)
             - add an extended attribute

  nfsref remove - remove the extended attribute, leave the symlink
                - can we remove the symlink if its contents are meaningless?


Samba:

  add - if a symlink already exists, replace its contents, preserving xattrs

  remove - if a FedFS extended attribute exists, leave the symlink (what contents should it have?)


The FedFS extended attribute is called "trusted.junction.nfs".  Should we rename it?

Note that CAP_SYS_ADMIN capabilities are required to access the xattr.  Will that be a problem for Samba administration tools?

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: Interoperable junctions on Linux

[Simo Sorce]

On Mon, 2013-04-22 at 16:39 -0400, Chuck Lever wrote:
> Hi-
> 
> I led a discussion on Friday at the Linux Storage and Filesystem
> Summit on how to store {DFS, FedFS} junctions in Linux filesystems.
> I'd like to summarize the discussion, then ask a few follow-up
> questions.  I apologize in advance for the length.
> 
> FedFS is to NFS as Microsoft DFS is to SMB/CIFS.  FedFS uses NFS
> referrals to glue together a file namespace out of separate shares,
> starting with a root share that may contain nothing but referrals.
> For more on FedFS, start with RFC 5716.
> 
> The physical object that stores referral location target information
> is called a junction.  It can contain an actual list of locations, or
> it can contain the DN of an LDAP record where the location list is
> maintained.
> 
>  +  Samba uses a symlink for this purpose.  The contents of the link
> represent the location information passed out to CIFS clients.
> 
>  +  FedFS uses an extended attribute on a directory for this purpose.
> The xattr contains XML which encodes location information passed out
> to NFS clients.
> 
> The reasons for this difference are simply historical.
> 
> Linux is often used as a multi-protocol file service platform, where
> the same physical data is presented to clients via both NFS and CIFS.
> For this reason, we think there would be value in using the same
> physical representation for both NFSD and Samba so that a single
> junction object on our exported filesystems can trigger a referral for
> both NFS and CIFS.
> 
> Samba has been around much longer, and DFS support is actually
> deployed.  FedFS is newer and still experimental.  Thus we decided to
> change FedFS to use a symlink instead of a directory.  Samba will
> still use the regular contents of the symlink, and FedFS will store
> its metadata in an extended attribute attached to the symlink.
> 
> There was a rough consensus to prefer JSON over XML in the FedFS
> xattr, though there are still some who dislike both.  I'm open to
> suggestion, since we're now essentially replacing the existing FedFS
> junction format and can change it to anything we like.

Can you give an example or refernce of what is stored in this XML/JSON
blob ? Why do you need structured data there ?

> Today FedFS junctions can contain either a location list or an LDAP
> DN.  One option for FedFS is to support only the LDAP DN junction
> type, and have a (possibly local) LDAP service available to store the
> location information.  The FedFS junction xattr would then always
> contain an LDAP URL.  Storing complex data types (a list containing
> pathnames, hostnames, integers, and other values) would then be up to
> LDAP.

Having to install a whole LDAP server as a pre-requisite seem very heavy
handed.

> We will have to discuss a conjunction of administrative interfaces at
> some later point.  However, we should clarify how our junction
> management tools behave now that each junction can have metadata it
> did not have before.
> 
> FedFS:
> 
>   nfsref add - if no symlink exists, create it (what contents should
> it have?)
>              - add an extended attribute
> 
>   nfsref remove - remove the extended attribute, leave the symlink
>                 - can we remove the symlink if its contents are
> meaningless?
> 
Why should we leave a symlink ? Don't we expect to remove junctions for
all protocols ?
> 
> Samba:
> 
>   add - if a symlink already exists, replace its contents, preserving
> xattrs
> 
>   remove - if a FedFS extended attribute exists, leave the symlink
> (what contents should it have?)

Why should we leave a symlink ? Don't we expect to remove junctions for
all protocols ?


What I do not get is why are you trying to use the same mechanism (a
symlink) but then treat them as independent and separate entities ?
What is the aim ?
>From your premise I thought you wanted to allow parallel functionality,
ie a DFS created in samba would be seen as a junction for nfs and
vice-versa, but the latter points seem to not allow that ?


> The FedFS extended attribute is called "trusted.junction.nfs".  Should
> we rename it?

Shouldn't it be namespaced and use something like trusted.nfs.junction ?
Also why a xattr in the trusted namespace ? What are the security
considerations that warrants a trusted attribute rather than a normal
one ? (Links to RFCs or other docs are just fine)

> Note that CAP_SYS_ADMIN capabilities are required to access the xattr.
> Will that be a problem for Samba administration tools?
> 
It should not be a problem for Samba, as we can retain capabilities if
needed, and we already handle data in xattrs (not for DFS symlinks).

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

Re: Interoperable junctions on Linux

[Chuck Lever]

Hi Simo-

Thanks for taking the time to read through my post.

On Apr 23, 2013, at 10:51 AM, Simo Sorce <simo@redhat.com> wrote:

> On Mon, 2013-04-22 at 16:39 -0400, Chuck Lever wrote:
>> Hi-
>> 
>> I led a discussion on Friday at the Linux Storage and Filesystem
>> Summit on how to store {DFS, FedFS} junctions in Linux filesystems.
>> I'd like to summarize the discussion, then ask a few follow-up
>> questions.  I apologize in advance for the length.
>> 
>> FedFS is to NFS as Microsoft DFS is to SMB/CIFS.  FedFS uses NFS
>> referrals to glue together a file namespace out of separate shares,
>> starting with a root share that may contain nothing but referrals.
>> For more on FedFS, start with RFC 5716.
>> 
>> The physical object that stores referral location target information
>> is called a junction.  It can contain an actual list of locations, or
>> it can contain the DN of an LDAP record where the location list is
>> maintained.
>> 
>> +  Samba uses a symlink for this purpose.  The contents of the link
>> represent the location information passed out to CIFS clients.
>> 
>> +  FedFS uses an extended attribute on a directory for this purpose.
>> The xattr contains XML which encodes location information passed out
>> to NFS clients.
>> 
>> The reasons for this difference are simply historical.
>> 
>> Linux is often used as a multi-protocol file service platform, where
>> the same physical data is presented to clients via both NFS and CIFS.
>> For this reason, we think there would be value in using the same
>> physical representation for both NFSD and Samba so that a single
>> junction object on our exported filesystems can trigger a referral for
>> both NFS and CIFS.
>> 
>> Samba has been around much longer, and DFS support is actually
>> deployed.  FedFS is newer and still experimental.  Thus we decided to
>> change FedFS to use a symlink instead of a directory.  Samba will
>> still use the regular contents of the symlink, and FedFS will store
>> its metadata in an extended attribute attached to the symlink.
>> 
>> There was a rough consensus to prefer JSON over XML in the FedFS
>> xattr, though there are still some who dislike both.  I'm open to
>> suggestion, since we're now essentially replacing the existing FedFS
>> junction format and can change it to anything we like.
> 
> Can you give an example or refernce of what is stored in this XML/JSON
> blob ? Why do you need structured data there ?

An "NFS basic junction" stores the location list in place.  Each item in a location list contains a number of pieces of data, including: a server hostname, an export pathname (which is a list of path components), and a number of integer and boolean settings that help clients sort which replica of this data they should mount.

A full explanation of this data is in RFC 5661, section 11.10.  This data is returned to an NFS client when it encounters one of these objects.  The client can redirect its requests to one of the servers and exports listed in the returned data.

A "FedFS junction" stores a reference to a location list stored in LDAP.  The LDAP server's hostname and port number and the UUID of a FedFS Fileset Name record are stored in the junction.  The Fileset Name record has children, each of which constitute a location (see above).

An explanation of this data is in an IETF draft:

  http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/

See chapter 4 for an overview of the schema used for these lists.  An NFS fileserver converts the LDAP records into an fs_locations4 or fs_locations_info4 attribute for NFS clients.  Other protocols use a different representation for communicating this list to clients.


> 
>> Today FedFS junctions can contain either a location list or an LDAP
>> DN.  One option for FedFS is to support only the LDAP DN junction
>> type, and have a (possibly local) LDAP service available to store the
>> location information.  The FedFS junction xattr would then always
>> contain an LDAP URL.  Storing complex data types (a list containing
>> pathnames, hostnames, integers, and other values) would then be up to
>> LDAP.
> 
> Having to install a whole LDAP server as a pre-requisite seem very heavy
> handed.

True.  Today, the LDAP/NSDB pieces are optional if an admin wants to support only "NFS basic junctions," for just this reason.

However there are certain benefits to allowing location lists to be managed via LDAP, rather than being specified at junction creation.  Junctions can share the same location list, for example.  A filesystem migration can update a central location list once, rather than having to find every junction that references the migrated filesystem.

In addition, storing these lists in a publicly available LDAP service means that any fileserver, anywhere, can access the lists.

If we are really wily, maybe a small single-purpose daemon can be constructed from a minimal LDAP server implementation (or from scratch), and it can listen on its own port or only for loopback requests.

> 
>> We will have to discuss a conjunction of administrative interfaces at
>> some later point.  However, we should clarify how our junction
>> management tools behave now that each junction can have metadata it
>> did not have before.
>> 
>> FedFS:
>> 
>>  nfsref add - if no symlink exists, create it (what contents should
>> it have?)
>>             - add an extended attribute
>> 
>>  nfsref remove - remove the extended attribute, leave the symlink
>>                - can we remove the symlink if its contents are
>> meaningless?
>> 
> Why should we leave a symlink ? Don't we expect to remove junctions for
> all protocols ?
>> 
>> Samba:
>> 
>>  add - if a symlink already exists, replace its contents, preserving
>> xattrs
>> 
>>  remove - if a FedFS extended attribute exists, leave the symlink
>> (what contents should it have?)
> 
> Why should we leave a symlink ? Don't we expect to remove junctions for
> all protocols ?

The difficulty I have is how we are going to conjoin the administrative tools that manage junctions.  I imagine that for some time, the tool used for managing DFS junctions will be unaware of FedFS junction content, and vice versa.

> What I do not get is why are you trying to use the same mechanism (a
> symlink) but then treat them as independent and separate entities ?
> What is the aim ?
> From your premise I thought you wanted to allow parallel functionality,
> ie a DFS created in samba would be seen as a junction for nfs and
> vice-versa, but the latter points seem to not allow that ?

FedFS junctions can list both NFS and SMB (and other types).  The SMB parts are not defined by the IETF, since SMB is a proprietary protocol controlled by Microsoft.

One way to have DFS and FedFS information in the same filesystem object is to have one object that can contain both.  The tools then have to be designed not to step on each other.  Eventually we figure out how to make this seamless.

I think you are suggesting we ignore this problem for now, and just have the tools pretend the other protocol does not exist, while still allowing the possibility of storing both types of metadata in the same filesystem object.  That may be an easy way to get started.

> 
> 
>> The FedFS extended attribute is called "trusted.junction.nfs".  Should
>> we rename it?
> 
> Shouldn't it be namespaced and use something like trusted.nfs.junction ?

That is probably a matter of taste.  I'm not especially attached to the current form.  But originally we had the junction information split into several attributes in the "trusted.junction" namespace.

> Also why a xattr in the trusted namespace ? What are the security
> considerations that warrants a trusted attribute rather than a normal
> one ? (Links to RFCs or other docs are just fine)

This is another historical design decision.  If there is consensus that we don't need to protect junction metadata from unintended or malicious local changes, then we can put these in another namespace.  However, without strong security here, redirecting network clients to another server and export can be hijacked, sending remote users to who knows where.  Is it enough simply to insist that junctions be owned by root?

> 
>> Note that CAP_SYS_ADMIN capabilities are required to access the xattr.
>> Will that be a problem for Samba administration tools?
>> 
> It should not be a problem for Samba, as we can retain capabilities if
> needed, and we already handle data in xattrs (not for DFS symlinks).

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: Interoperable junctions on Linux

[Myklebust, Trond]

On Tue, 2013-04-23 at 11:42 -0400, Chuck Lever wrote:
> On Apr 23, 2013, at 10:51 AM, Simo Sorce <simo@redhat.com> wrote:

> > Also why a xattr in the trusted namespace ? What are the security
> > considerations that warrants a trusted attribute rather than a normal
> > one ? (Links to RFCs or other docs are just fine)
> 
> This is another historical design decision.  If there is consensus that we don't need to protect junction metadata from unintended or malicious local changes, then we can put these in another namespace.  However, without strong security here, redirecting network clients to another server and export can be hijacked, sending remote users to who knows where.  Is it enough simply to insist that junctions be owned by root?

Junctions resolve into mountpoints on clients. Allowing arbitrary users
to change the junction parameters basically means giving them the
ability to control the namespace on clients. They can for instance
redirect an application from a trusted server onto an untrusted one.

I therefore strongly recommend that we ensure the creation, deletion and
modification of a junction remains a privileged operation on the server.

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

Re: Interoperable junctions on Linux

[Simo Sorce]

On Tue, 2013-04-23 at 15:51 +0000, Myklebust, Trond wrote:
> On Tue, 2013-04-23 at 11:42 -0400, Chuck Lever wrote:
> > On Apr 23, 2013, at 10:51 AM, Simo Sorce <simo@redhat.com> wrote:
> 
> > > Also why a xattr in the trusted namespace ? What are the security
> > > considerations that warrants a trusted attribute rather than a normal
> > > one ? (Links to RFCs or other docs are just fine)
> > 
> > This is another historical design decision.  If there is consensus that we don't need to protect junction metadata from unintended or malicious local changes, then we can put these in another namespace.  However, without strong security here, redirecting network clients to another server and export can be hijacked, sending remote users to who knows where.  Is it enough simply to insist that junctions be owned by root?
> 
> Junctions resolve into mountpoints on clients. Allowing arbitrary users
> to change the junction parameters basically means giving them the
> ability to control the namespace on clients. They can for instance
> redirect an application from a trusted server onto an untrusted one.
> 
> I therefore strongly recommend that we ensure the creation, deletion and
> modification of a junction remains a privileged operation on the server.

Is it not sufficient to make sure the symlink is owned by root ?

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

RE: Interoperable junctions on Linux

[Myklebust, Trond]

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTaW1vIFNvcmNlIFttYWlsdG86
c2ltb0ByZWRoYXQuY29tXQ0KPiBTZW50OiBUdWVzZGF5LCBBcHJpbCAyMywgMjAxMyAxMjoyMCBQ
TQ0KPiBUbzogTXlrbGVidXN0LCBUcm9uZA0KPiBDYzogQ2h1Y2sgTGV2ZXI7IHNhbWJhLXRlY2hu
aWNhbEBsaXN0cy5zYW1iYS5vcmc7IGZlZGZzLXV0aWxzIERldmVsb3BlcnM7DQo+IExpbnV4IE5G
UyBNYWlsaW5nIExpc3QNCj4gU3ViamVjdDogUmU6IEludGVyb3BlcmFibGUganVuY3Rpb25zIG9u
IExpbnV4DQo+IA0KPiBPbiBUdWUsIDIwMTMtMDQtMjMgYXQgMTU6NTEgKzAwMDAsIE15a2xlYnVz
dCwgVHJvbmQgd3JvdGU6DQo+ID4gT24gVHVlLCAyMDEzLTA0LTIzIGF0IDExOjQyIC0wNDAwLCBD
aHVjayBMZXZlciB3cm90ZToNCj4gPiA+IE9uIEFwciAyMywgMjAxMywgYXQgMTA6NTEgQU0sIFNp
bW8gU29yY2UgPHNpbW9AcmVkaGF0LmNvbT4gd3JvdGU6DQo+ID4NCj4gPiA+ID4gQWxzbyB3aHkg
YSB4YXR0ciBpbiB0aGUgdHJ1c3RlZCBuYW1lc3BhY2UgPyBXaGF0IGFyZSB0aGUgc2VjdXJpdHkN
Cj4gPiA+ID4gY29uc2lkZXJhdGlvbnMgdGhhdCB3YXJyYW50cyBhIHRydXN0ZWQgYXR0cmlidXRl
IHJhdGhlciB0aGFuIGENCj4gPiA+ID4gbm9ybWFsIG9uZSA/IChMaW5rcyB0byBSRkNzIG9yIG90
aGVyIGRvY3MgYXJlIGp1c3QgZmluZSkNCj4gPiA+DQo+ID4gPiBUaGlzIGlzIGFub3RoZXIgaGlz
dG9yaWNhbCBkZXNpZ24gZGVjaXNpb24uICBJZiB0aGVyZSBpcyBjb25zZW5zdXMgdGhhdCB3ZQ0K
PiBkb24ndCBuZWVkIHRvIHByb3RlY3QganVuY3Rpb24gbWV0YWRhdGEgZnJvbSB1bmludGVuZGVk
IG9yIG1hbGljaW91cyBsb2NhbA0KPiBjaGFuZ2VzLCB0aGVuIHdlIGNhbiBwdXQgdGhlc2UgaW4g
YW5vdGhlciBuYW1lc3BhY2UuICBIb3dldmVyLCB3aXRob3V0DQo+IHN0cm9uZyBzZWN1cml0eSBo
ZXJlLCByZWRpcmVjdGluZyBuZXR3b3JrIGNsaWVudHMgdG8gYW5vdGhlciBzZXJ2ZXIgYW5kDQo+
IGV4cG9ydCBjYW4gYmUgaGlqYWNrZWQsIHNlbmRpbmcgcmVtb3RlIHVzZXJzIHRvIHdobyBrbm93
cyB3aGVyZS4gIElzIGl0DQo+IGVub3VnaCBzaW1wbHkgdG8gaW5zaXN0IHRoYXQganVuY3Rpb25z
IGJlIG93bmVkIGJ5IHJvb3Q/DQo+ID4NCj4gPiBKdW5jdGlvbnMgcmVzb2x2ZSBpbnRvIG1vdW50
cG9pbnRzIG9uIGNsaWVudHMuIEFsbG93aW5nIGFyYml0cmFyeQ0KPiA+IHVzZXJzIHRvIGNoYW5n
ZSB0aGUganVuY3Rpb24gcGFyYW1ldGVycyBiYXNpY2FsbHkgbWVhbnMgZ2l2aW5nIHRoZW0NCj4g
PiB0aGUgYWJpbGl0eSB0byBjb250cm9sIHRoZSBuYW1lc3BhY2Ugb24gY2xpZW50cy4gVGhleSBj
YW4gZm9yIGluc3RhbmNlDQo+ID4gcmVkaXJlY3QgYW4gYXBwbGljYXRpb24gZnJvbSBhIHRydXN0
ZWQgc2VydmVyIG9udG8gYW4gdW50cnVzdGVkIG9uZS4NCj4gPg0KPiA+IEkgdGhlcmVmb3JlIHN0
cm9uZ2x5IHJlY29tbWVuZCB0aGF0IHdlIGVuc3VyZSB0aGUgY3JlYXRpb24sIGRlbGV0aW9uDQo+
ID4gYW5kIG1vZGlmaWNhdGlvbiBvZiBhIGp1bmN0aW9uIHJlbWFpbnMgYSBwcml2aWxlZ2VkIG9w
ZXJhdGlvbiBvbiB0aGUgc2VydmVyLg0KPiANCj4gSXMgaXQgbm90IHN1ZmZpY2llbnQgdG8gbWFr
ZSBzdXJlIHRoZSBzeW1saW5rIGlzIG93bmVkIGJ5IHJvb3QgPw0KDQpIb3cgZG8geW91IGNoZWNr
IHRoYXQgYXRvbWljYWxseSB3aXRoIHRoZSBnZXR4YXR0cj8NCg0KVHJvbmQNCg==

RE: Interoperable junctions on Linux

[Simo Sorce]

On Tue, 2013-04-23 at 16:24 +0000, Myklebust, Trond wrote:
> > -----Original Message-----
> > From: Simo Sorce [mailto:simo@redhat.com]
> > Sent: Tuesday, April 23, 2013 12:20 PM
> > To: Myklebust, Trond
> > Cc: Chuck Lever; samba-technical@lists.samba.org; fedfs-utils Developers;
> > Linux NFS Mailing List
> > Subject: Re: Interoperable junctions on Linux
> > 
> > On Tue, 2013-04-23 at 15:51 +0000, Myklebust, Trond wrote:
> > > On Tue, 2013-04-23 at 11:42 -0400, Chuck Lever wrote:
> > > > On Apr 23, 2013, at 10:51 AM, Simo Sorce <simo@redhat.com> wrote:
> > >
> > > > > Also why a xattr in the trusted namespace ? What are the security
> > > > > considerations that warrants a trusted attribute rather than a
> > > > > normal one ? (Links to RFCs or other docs are just fine)
> > > >
> > > > This is another historical design decision.  If there is consensus that we
> > don't need to protect junction metadata from unintended or malicious local
> > changes, then we can put these in another namespace.  However, without
> > strong security here, redirecting network clients to another server and
> > export can be hijacked, sending remote users to who knows where.  Is it
> > enough simply to insist that junctions be owned by root?
> > >
> > > Junctions resolve into mountpoints on clients. Allowing arbitrary
> > > users to change the junction parameters basically means giving them
> > > the ability to control the namespace on clients. They can for instance
> > > redirect an application from a trusted server onto an untrusted one.
> > >
> > > I therefore strongly recommend that we ensure the creation, deletion
> > > and modification of a junction remains a privileged operation on the server.
> > 
> > Is it not sufficient to make sure the symlink is owned by root ?
> 
> How do you check that atomically with the getxattr?

Using fgetxattr() after an open and a fstat() ?

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

RE: Interoperable junctions on Linux

[Myklebust, Trond]

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBTaW1vIFNvcmNlIFttYWlsdG86
c2ltb0ByZWRoYXQuY29tXQ0KPiBTZW50OiBUdWVzZGF5LCBBcHJpbCAyMywgMjAxMyAxMjoyOSBQ
TQ0KPiBUbzogTXlrbGVidXN0LCBUcm9uZA0KPiBDYzogQ2h1Y2sgTGV2ZXI7IHNhbWJhLXRlY2hu
aWNhbEBsaXN0cy5zYW1iYS5vcmc7IGZlZGZzLXV0aWxzIERldmVsb3BlcnM7DQo+IExpbnV4IE5G
UyBNYWlsaW5nIExpc3QNCj4gU3ViamVjdDogUkU6IEludGVyb3BlcmFibGUganVuY3Rpb25zIG9u
IExpbnV4DQo+IA0KPiBPbiBUdWUsIDIwMTMtMDQtMjMgYXQgMTY6MjQgKzAwMDAsIE15a2xlYnVz
dCwgVHJvbmQgd3JvdGU6DQo+ID4gPiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiA+ID4g
RnJvbTogU2ltbyBTb3JjZSBbbWFpbHRvOnNpbW9AcmVkaGF0LmNvbV0NCj4gPiA+IFNlbnQ6IFR1
ZXNkYXksIEFwcmlsIDIzLCAyMDEzIDEyOjIwIFBNDQo+ID4gPiBUbzogTXlrbGVidXN0LCBUcm9u
ZA0KPiA+ID4gQ2M6IENodWNrIExldmVyOyBzYW1iYS10ZWNobmljYWxAbGlzdHMuc2FtYmEub3Jn
OyBmZWRmcy11dGlscw0KPiA+ID4gRGV2ZWxvcGVyczsgTGludXggTkZTIE1haWxpbmcgTGlzdA0K
PiA+ID4gU3ViamVjdDogUmU6IEludGVyb3BlcmFibGUganVuY3Rpb25zIG9uIExpbnV4DQo+ID4g
Pg0KPiA+ID4gT24gVHVlLCAyMDEzLTA0LTIzIGF0IDE1OjUxICswMDAwLCBNeWtsZWJ1c3QsIFRy
b25kIHdyb3RlOg0KPiA+ID4gPiBPbiBUdWUsIDIwMTMtMDQtMjMgYXQgMTE6NDIgLTA0MDAsIENo
dWNrIExldmVyIHdyb3RlOg0KPiA+ID4gPiA+IE9uIEFwciAyMywgMjAxMywgYXQgMTA6NTEgQU0s
IFNpbW8gU29yY2UgPHNpbW9AcmVkaGF0LmNvbT4gd3JvdGU6DQo+ID4gPiA+DQo+ID4gPiA+ID4g
PiBBbHNvIHdoeSBhIHhhdHRyIGluIHRoZSB0cnVzdGVkIG5hbWVzcGFjZSA/IFdoYXQgYXJlIHRo
ZQ0KPiA+ID4gPiA+ID4gc2VjdXJpdHkgY29uc2lkZXJhdGlvbnMgdGhhdCB3YXJyYW50cyBhIHRy
dXN0ZWQgYXR0cmlidXRlDQo+ID4gPiA+ID4gPiByYXRoZXIgdGhhbiBhIG5vcm1hbCBvbmUgPyAo
TGlua3MgdG8gUkZDcyBvciBvdGhlciBkb2NzIGFyZQ0KPiA+ID4gPiA+ID4ganVzdCBmaW5lKQ0K
PiA+ID4gPiA+DQo+ID4gPiA+ID4gVGhpcyBpcyBhbm90aGVyIGhpc3RvcmljYWwgZGVzaWduIGRl
Y2lzaW9uLiAgSWYgdGhlcmUgaXMNCj4gPiA+ID4gPiBjb25zZW5zdXMgdGhhdCB3ZQ0KPiA+ID4g
ZG9uJ3QgbmVlZCB0byBwcm90ZWN0IGp1bmN0aW9uIG1ldGFkYXRhIGZyb20gdW5pbnRlbmRlZCBv
ciBtYWxpY2lvdXMNCj4gPiA+IGxvY2FsIGNoYW5nZXMsIHRoZW4gd2UgY2FuIHB1dCB0aGVzZSBp
biBhbm90aGVyIG5hbWVzcGFjZS4gIEhvd2V2ZXIsDQo+ID4gPiB3aXRob3V0IHN0cm9uZyBzZWN1
cml0eSBoZXJlLCByZWRpcmVjdGluZyBuZXR3b3JrIGNsaWVudHMgdG8gYW5vdGhlcg0KPiA+ID4g
c2VydmVyIGFuZCBleHBvcnQgY2FuIGJlIGhpamFja2VkLCBzZW5kaW5nIHJlbW90ZSB1c2VycyB0
byB3aG8ga25vd3MNCj4gPiA+IHdoZXJlLiAgSXMgaXQgZW5vdWdoIHNpbXBseSB0byBpbnNpc3Qg
dGhhdCBqdW5jdGlvbnMgYmUgb3duZWQgYnkgcm9vdD8NCj4gPiA+ID4NCj4gPiA+ID4gSnVuY3Rp
b25zIHJlc29sdmUgaW50byBtb3VudHBvaW50cyBvbiBjbGllbnRzLiBBbGxvd2luZyBhcmJpdHJh
cnkNCj4gPiA+ID4gdXNlcnMgdG8gY2hhbmdlIHRoZSBqdW5jdGlvbiBwYXJhbWV0ZXJzIGJhc2lj
YWxseSBtZWFucyBnaXZpbmcNCj4gPiA+ID4gdGhlbSB0aGUgYWJpbGl0eSB0byBjb250cm9sIHRo
ZSBuYW1lc3BhY2Ugb24gY2xpZW50cy4gVGhleSBjYW4gZm9yDQo+ID4gPiA+IGluc3RhbmNlIHJl
ZGlyZWN0IGFuIGFwcGxpY2F0aW9uIGZyb20gYSB0cnVzdGVkIHNlcnZlciBvbnRvIGFuIHVudHJ1
c3RlZA0KPiBvbmUuDQo+ID4gPiA+DQo+ID4gPiA+IEkgdGhlcmVmb3JlIHN0cm9uZ2x5IHJlY29t
bWVuZCB0aGF0IHdlIGVuc3VyZSB0aGUgY3JlYXRpb24sDQo+ID4gPiA+IGRlbGV0aW9uIGFuZCBt
b2RpZmljYXRpb24gb2YgYSBqdW5jdGlvbiByZW1haW5zIGEgcHJpdmlsZWdlZCBvcGVyYXRpb24N
Cj4gb24gdGhlIHNlcnZlci4NCj4gPiA+DQo+ID4gPiBJcyBpdCBub3Qgc3VmZmljaWVudCB0byBt
YWtlIHN1cmUgdGhlIHN5bWxpbmsgaXMgb3duZWQgYnkgcm9vdCA/DQo+ID4NCj4gPiBIb3cgZG8g
eW91IGNoZWNrIHRoYXQgYXRvbWljYWxseSB3aXRoIHRoZSBnZXR4YXR0cj8NCj4gDQo+IFVzaW5n
IGZnZXR4YXR0cigpIGFmdGVyIGFuIG9wZW4gYW5kIGEgZnN0YXQoKSA/DQoNClNvLCAzIG9wZXJh
dGlvbnMgaW4gb3JkZXIgdG8gZG8gdGhlIHNhbWUgdGhpbmcgeW91IGNhbiBkbyB3aXRoIGp1c3Qg
MSBpZiB5b3UgY2hvb3NlIGEgdHJ1c3RlZCB4YXR0cj8NCg0KVHJvbmQNCg==

Re: Interoperable junctions on Linux

[Simo Sorce]

On Tue, 2013-04-23 at 11:42 -0400, Chuck Lever wrote:
> > Can you give an example or refernce of what is stored in this XML/JSON
> > blob ? Why do you need structured data there ?
> 
> An "NFS basic junction" stores the location list in place.  Each item
> in a location list contains a number of pieces of data, including: a
> server hostname, an export pathname (which is a list of path
> components), and a number of integer and boolean settings that help
> clients sort which replica of this data they should mount.
> 
> A full explanation of this data is in RFC 5661, section 11.10.  This
> data is returned to an NFS client when it encounters one of these
> objects.  The client can redirect its requests to one of the servers
> and exports listed in the returned data.
> 
> A "FedFS junction" stores a reference to a location list stored in
> LDAP.  The LDAP server's hostname and port number and the UUID of a
> FedFS Fileset Name record are stored in the junction.  The Fileset
> Name record has children, each of which constitute a location (see
> above).

I do not see mention of LDAP URIs, andyou seem to speak in the singular
when mention 'the LDAP server' does it mean you hve no way to specify a
pool of LDAP servers for HA ?

> An explanation of this data is in an IETF draft:
> 
> 
> http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/

> See chapter 4 for an overview of the schema used for these lists.  An
> NFS fileserver converts the LDAP records into an fs_locations4 or
> fs_locations_info4 attribute for NFS clients.  Other protocols use a
> different representation for communicating this list to clients.
> 
To be honest the constrains in this document on the LDAP DIT, seem to
indicate it will be possible to use this stuff primarily only with a
dedicated LDAP server, Dictating how the rootDSE/namingContext should
look like is a quite strong demand.

Why do you need a full namingContext for FedFS ?
Why a subtree is not sufficient ?


On the security side, you recommend RPCSEC_GSS for NFS, but then TLS for
LDAP, why not use SASL/GSSAPI for LDAP as well so you need a single set
of credentials ?

> 
> > 
> >> Today FedFS junctions can contain either a location list or an LDAP
> >> DN.  One option for FedFS is to support only the LDAP DN junction
> >> type, and have a (possibly local) LDAP service available to store the
> >> location information.  The FedFS junction xattr would then always
> >> contain an LDAP URL.  Storing complex data types (a list containing
> >> pathnames, hostnames, integers, and other values) would then be up to
> >> LDAP.
> > 
> > Having to install a whole LDAP server as a pre-requisite seem very heavy
> > handed.
> 
> True.  Today, the LDAP/NSDB pieces are optional if an admin wants to
> support only "NFS basic junctions," for just this reason.
> 
> However there are certain benefits to allowing location lists to be
> managed via LDAP, rather than being specified at junction creation.
> Junctions can share the same location list, for example.  A filesystem
> migration can update a central location list once, rather than having
> to find every junction that references the migrated filesystem.

Understood.

> In addition, storing these lists in a publicly available LDAP service
> means that any fileserver, anywhere, can access the lists.
> 
> If we are really wily, maybe a small single-purpose daemon can be
> constructed from a minimal LDAP server implementation (or from
> scratch), and it can listen on its own port or only for loopback
> requests.

I think in most cases this is what will actually happen, but you do not
need to use a special purpose built server, you can use an existing LDAP
server simply specially configured for your needs. It will cause
administrative overhead to handle this infrastructure though.

[..]

> > Why should we leave a symlink ? Don't we expect to remove junctions for
> > all protocols ?
> 
> The difficulty I have is how we are going to conjoin the
> administrative tools that manage junctions.  I imagine that for some
> time, the tool used for managing DFS junctions will be unaware of
> FedFS junction content, and vice versa.

Yes, for some time one may end up wiping out the other.

> > What I do not get is why are you trying to use the same mechanism (a
> > symlink) but then treat them as independent and separate entities ?
> > What is the aim ?
> > From your premise I thought you wanted to allow parallel functionality,
> > ie a DFS created in samba would be seen as a junction for nfs and
> > vice-versa, but the latter points seem to not allow that ?
> 
> FedFS junctions can list both NFS and SMB (and other types).  The SMB
> parts are not defined by the IETF, since SMB is a proprietary protocol
> controlled by Microsoft.
> 
> One way to have DFS and FedFS information in the same filesystem
> object is to have one object that can contain both.  The tools then
> have to be designed not to step on each other.  Eventually we figure
> out how to make this seamless.
> 
> I think you are suggesting we ignore this problem for now, and just
> have the tools pretend the other protocol does not exist, while still
> allowing the possibility of storing both types of metadata in the same
> filesystem object.  That may be an easy way to get started.

I thought this was what you were proposing actually. With my
'integrator' hat on I would rather quickly define common tools that can
handle both, and have the old tools return loud warnings (were possible)
if you try to use them.

In the Samba case the basic tool is 'ln', not sure we can do much about
it :-) But we can certainly patch the RPC code that allows handling via
SMB/RPC although I am not quite sure how to populate all the data the
SMB world has no concept of, perhaps using 'good defaults' ...

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

Re: Interoperable junctions on Linux

[Chuck Lever]

On Apr 23, 2013, at 12:54 PM, Simo Sorce <simo@redhat.com> wrote:

> On Tue, 2013-04-23 at 11:42 -0400, Chuck Lever wrote:
>>> Can you give an example or refernce of what is stored in this XML/JSON
>>> blob ? Why do you need structured data there ?
>> 
>> An "NFS basic junction" stores the location list in place.  Each item
>> in a location list contains a number of pieces of data, including: a
>> server hostname, an export pathname (which is a list of path
>> components), and a number of integer and boolean settings that help
>> clients sort which replica of this data they should mount.
>> 
>> A full explanation of this data is in RFC 5661, section 11.10.  This
>> data is returned to an NFS client when it encounters one of these
>> objects.  The client can redirect its requests to one of the servers
>> and exports listed in the returned data.
>> 
>> A "FedFS junction" stores a reference to a location list stored in
>> LDAP.  The LDAP server's hostname and port number and the UUID of a
>> FedFS Fileset Name record are stored in the junction.  The Fileset
>> Name record has children, each of which constitute a location (see
>> above).
> 
> I do not see mention of LDAP URIs, andyou seem to speak in the singular
> when mention 'the LDAP server' does it mean you hve no way to specify a
> pool of LDAP servers for HA ?

An "NSDB name" (used in a junction) is meant to represent a pool of replicated LDAP server resources, or just one LDAP server.  Note that specifying IP addresses in FedFS Fileset Names is specifically disallowed for this reason.

A FedFS Fileset Name can indeed be represented by an LDAP URI.  See the operational examples in the FedFS NSDB draft.

> 
>> An explanation of this data is in an IETF draft:
>> 
>> 
>> http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/
> 
>> See chapter 4 for an overview of the schema used for these lists.  An
>> NFS fileserver converts the LDAP records into an fs_locations4 or
>> fs_locations_info4 attribute for NFS clients.  Other protocols use a
>> different representation for communicating this list to clients.
>> 
> To be honest the constrains in this document on the LDAP DIT, seem to
> indicate it will be possible to use this stuff primarily only with a
> dedicated LDAP server, Dictating how the rootDSE/namingContext should
> look like is a quite strong demand.
> 
> Why do you need a full namingContext for FedFS ?
> Why a subtree is not sufficient ?

The document does recommend using a dedicated LDAP service for an NSDB, but does not require it.

One or more of the namingContext records on an NSDB has a new attribute (fedfsNceDN) that points to the FedFS DIT.  So it is entirely possible to use an existing LDAP server for storing FedFS records, simply by adding that attribute to the namingContext under which the FedFS subtree resides.

The goal is to allow automated discovery of the location of the FedFS DIT using a DN advertised in the LDAP server's rootDSE.  NSDB clients thus need know only the name of the NSDB service.

By default, NSDBs that my tools set up store the FedFS DIT under the LDAP server's domain controller suffix, rather than using the special "o=fedfs" DN.  By convention the FedFS domain name matches the domain controller suffix, but operation doesn't depend on this.

It is not quite as restrictive as you think, but it is very fiddly to set up on industrial strength LDAP servers like OpenLDAP or 389-ds.


> On the security side, you recommend RPCSEC_GSS for NFS, but then TLS for
> LDAP, why not use SASL/GSSAPI for LDAP as well so you need a single set
> of credentials ?

There are two classes of tasks involving NSDBs: administrative, where an administrator is available to provide credential material; and fileserver, which is an unattended access performed simply to retrieve location lists.

The purpose of TLS is to allow fileservers to authenticate NSDBs without requiring user intervention.  A certificate chain for each NSDB is provisioned on fileservers (using the FedFS ADMIN RPC protocol).  The fileservers use them with START_TLS each time a connection is established to that NSDB.

SASL/GSSAPI can of course be used when performing administrative operations on an NSDB, and that is the plan for the Linux FedFS implementation.  As you observed, an administrator can then use the same credentials for an RPCGSS-protected FedFS ADMIN service and her NSDB.


> 
>> 
>>> 
>>>> Today FedFS junctions can contain either a location list or an LDAP
>>>> DN.  One option for FedFS is to support only the LDAP DN junction
>>>> type, and have a (possibly local) LDAP service available to store the
>>>> location information.  The FedFS junction xattr would then always
>>>> contain an LDAP URL.  Storing complex data types (a list containing
>>>> pathnames, hostnames, integers, and other values) would then be up to
>>>> LDAP.
>>> 
>>> Having to install a whole LDAP server as a pre-requisite seem very heavy
>>> handed.
>> 
>> True.  Today, the LDAP/NSDB pieces are optional if an admin wants to
>> support only "NFS basic junctions," for just this reason.
>> 
>> However there are certain benefits to allowing location lists to be
>> managed via LDAP, rather than being specified at junction creation.
>> Junctions can share the same location list, for example.  A filesystem
>> migration can update a central location list once, rather than having
>> to find every junction that references the migrated filesystem.
> 
> Understood.
> 
>> In addition, storing these lists in a publicly available LDAP service
>> means that any fileserver, anywhere, can access the lists.
>> 
>> If we are really wily, maybe a small single-purpose daemon can be
>> constructed from a minimal LDAP server implementation (or from
>> scratch), and it can listen on its own port or only for loopback
>> requests.
> 
> I think in most cases this is what will actually happen, but you do not
> need to use a special purpose built server, you can use an existing LDAP
> server simply specially configured for your needs. It will cause
> administrative overhead to handle this infrastructure though.

Don't I know it.

To simplify initial configuration, I've implemented a script that installs OpenLDAP and performs enough configuration to provide a stand-alone NSDB, with or without TLS.  I already have a number of C tools that can set up the rootDSE and add Fileset Name records on either 389-ds or OpenLDAP servers.

The idea is to allow both types of NSDB (stand-alone, and as part of an existing LDAP service), but that kind of generality adds a lot of complexity to the set up process and documentation.  Going with just one or the other makes life much easier.


> 
> [..]
> 
>>> Why should we leave a symlink ? Don't we expect to remove junctions for
>>> all protocols ?
>> 
>> The difficulty I have is how we are going to conjoin the
>> administrative tools that manage junctions.  I imagine that for some
>> time, the tool used for managing DFS junctions will be unaware of
>> FedFS junction content, and vice versa.
> 
> Yes, for some time one may end up wiping out the other.
> 
>>> What I do not get is why are you trying to use the same mechanism (a
>>> symlink) but then treat them as independent and separate entities ?
>>> What is the aim ?
>>> From your premise I thought you wanted to allow parallel functionality,
>>> ie a DFS created in samba would be seen as a junction for nfs and
>>> vice-versa, but the latter points seem to not allow that ?
>> 
>> FedFS junctions can list both NFS and SMB (and other types).  The SMB
>> parts are not defined by the IETF, since SMB is a proprietary protocol
>> controlled by Microsoft.
>> 
>> One way to have DFS and FedFS information in the same filesystem
>> object is to have one object that can contain both.  The tools then
>> have to be designed not to step on each other.  Eventually we figure
>> out how to make this seamless.
>> 
>> I think you are suggesting we ignore this problem for now, and just
>> have the tools pretend the other protocol does not exist, while still
>> allowing the possibility of storing both types of metadata in the same
>> filesystem object.  That may be an easy way to get started.
> 
> I thought this was what you were proposing actually. With my
> 'integrator' hat on I would rather quickly define common tools that can
> handle both, and have the old tools return loud warnings (were possible)
> if you try to use them.

The "old" FedFS tools in this case are available because they are one of two independent reference implementations required for the IETF standardization process.  However, I think solving the problem of integrating FedFS ADMIN with an existing DFS implementation on a multi-protocol fileserver is going to be very valuable implementation experience that can be fed back into the IETF standards process.

I'm not confident we can fit rpc.fedfsd (Linux's implementation of the FedFS RPC ADMIN service) into this model, so we may end up abandoning its use for managing FedFS junctions, leaving it only to provisioning NSDB certificates.

> In the Samba case the basic tool is 'ln', not sure we can do much about
> it :-) But we can certainly patch the RPC code that allows handling via
> SMB/RPC although I am not quite sure how to populate all the data the
> SMB world has no concept of, perhaps using 'good defaults' ...

Exactly the problem.

My thought was to define a place-holder value for both protocols that can be used by our existing tools, for now.  No xattr for symlinks created on behalf of DFS, and maybe an empty symlink for junctions created by rpc.fedfsd or the nfsref command.

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: Interoperable junctions on Linux

[Simo Sorce]

On Tue, 2013-04-23 at 14:11 -0400, Chuck Lever wrote:
> On Apr 23, 2013, at 12:54 PM, Simo Sorce <simo@redhat.com> wrote:

> > I do not see mention of LDAP URIs, andyou seem to speak in the singular
> > when mention 'the LDAP server' does it mean you hve no way to specify a
> > pool of LDAP servers for HA ?
> 
> An "NSDB name" (used in a junction) is meant to represent a pool of
> replicated LDAP server resources, or just one LDAP server.  Note that
> specifying IP addresses in FedFS Fileset Names is specifically
> disallowed for this reason.
> 
> A FedFS Fileset Name can indeed be represented by an LDAP URI.  See
> the operational examples in the FedFS NSDB draft.

Sorry I found that after some more searching, the document is vast and I
missed it at first glance.

> > 
> >> An explanation of this data is in an IETF draft:
> >> 
> >> 
> >> http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/
> > 
> >> See chapter 4 for an overview of the schema used for these lists.  An
> >> NFS fileserver converts the LDAP records into an fs_locations4 or
> >> fs_locations_info4 attribute for NFS clients.  Other protocols use a
> >> different representation for communicating this list to clients.
> >> 
> > To be honest the constrains in this document on the LDAP DIT, seem to
> > indicate it will be possible to use this stuff primarily only with a
> > dedicated LDAP server, Dictating how the rootDSE/namingContext should
> > look like is a quite strong demand.
> > 
> > Why do you need a full namingContext for FedFS ?
> > Why a subtree is not sufficient ?
> 
> The document does recommend using a dedicated LDAP service for an
> NSDB, but does not require it.

Ah I didn't realize you recommend a dedicated LDAP server.

> One or more of the namingContext records on an NSDB has a new
> attribute (fedfsNceDN) that points to the FedFS DIT.  So it is
> entirely possible to use an existing LDAP server for storing FedFS
> records, simply by adding that attribute to the namingContext under
> which the FedFS subtree resides.

That 'simply' is not so simple, you may not be allowed to do that.

> The goal is to allow automated discovery of the location of the FedFS
> DIT using a DN advertised in the LDAP server's rootDSE.  NSDB clients
> thus need know only the name of the NSDB service.

You can do the same w/o forcing the attribute to be on the base suffix,
all you need to do is a subtree search with
'&(objctclass=fedfsNsdbContainerInfo)' as part of the filter.
Given this objectclass is used as the container entry main objectclass
it also means the fedfsNceDN would be redundant and could be removed
entirely.

> By default, NSDBs that my tools set up store the FedFS DIT under the
> LDAP server's domain controller suffix, rather than using the special
> "o=fedfs" DN.  By convention the FedFS domain name matches the domain
> controller suffix, but operation doesn't depend on this.
> 
> It is not quite as restrictive as you think, but it is very fiddly to
> set up on industrial strength LDAP servers like OpenLDAP or 389-ds.

Well I have some knowledge in this field, and I do not find it very
fiddly, but I may be biased as I have been working for the past 7 years
to make LDAP+Kerberos simple to manage within the FreeIPA project.
Your requirement to set an objectclass on the base suffix is something I
find particularly unappealing, and no other tool that I know of requires
this (because it is unnecessary).
> 
> > On the security side, you recommend RPCSEC_GSS for NFS, but then TLS
> for
> > LDAP, why not use SASL/GSSAPI for LDAP as well so you need a single
> set
> > of credentials ?
> 
> There are two classes of tasks involving NSDBs: administrative, where
> an administrator is available to provide credential material; and
> fileserver, which is an unattended access performed simply to retrieve
> location lists.
> 
> The purpose of TLS is to allow fileservers to authenticate NSDBs
> without requiring user intervention.  A certificate chain for each
> NSDB is provisioned on fileservers (using the FedFS ADMIN RPC
> protocol).  The fileservers use them with START_TLS each time a
> connection is established to that NSDB.
> 
> SASL/GSSAPI can of course be used when performing administrative
> operations on an NSDB, and that is the plan for the Linux FedFS
> implementation.  As you observed, an administrator can then use the
> same credentials for an RPCGSS-protected FedFS ADMIN service and her
> NSDB.

SASL/GSSAPI can also be used by the file server given the use of
RPCSEC_GSS requires you to have a keytab there. This means I have to
care only to drop a keytab and I get mutual authentication and
encryption of the channel to LDAP w/o having to care to get a x509
certificate and configure the server to trust it directly or drop a
suitable CA Certificate.

I find that properly configuring SSL is a lot more difficult than simply
dropping a keytab in place (especially given you alredy need to do it).

Basically in most setups I encounter for real SSL is effectively
"disabled" and by that I mean that peers are configured to ignore
untrusted certificates, also certificates expire and the whole lifecycle
of using SSL certificates *properly* is a lot more complex than using
GSSAPI with a simple keytab file.

So I strongly recommend you put language in your RFC allowing
SASL/GSSAPI as a mechanism for securing communication to the LDAP server
for both administrative and simple retrieval, you will do a favor to the
implementer allowing conformance w/o having to jump though hoops.


[Sorry for the OT, but this really is something you should get right]


> >> If we are really wily, maybe a small single-purpose daemon can be
> >> constructed from a minimal LDAP server implementation (or from
> >> scratch), and it can listen on its own port or only for loopback
> >> requests.
> > 
> > I think in most cases this is what will actually happen, but you do not
> > need to use a special purpose built server, you can use an existing LDAP
> > server simply specially configured for your needs. It will cause
> > administrative overhead to handle this infrastructure though.
> 
> Don't I know it.
> 
> To simplify initial configuration, I've implemented a script that
> installs OpenLDAP and performs enough configuration to provide a
> stand-alone NSDB, with or without TLS.  I already have a number of C
> tools that can set up the rootDSE and add Fileset Name records on
> either 389-ds or OpenLDAP servers.
> 
> The idea is to allow both types of NSDB (stand-alone, and as part of
> an existing LDAP service), but that kind of generality adds a lot of
> complexity to the set up process and documentation.  Going with just
> one or the other makes life much easier.

I think you should allow the broadest possibilities of course, which is
why I am picking on things like allowing SSAL/GSSAPI explicitly in the
RFC language. Whether people will integrate into existing LDAP server or
not remains to be seen, if we can avoid the need to add an objectlass on
the root suffix I see that we can easily add this a standard feature for
FreeIPA as well (we already provide automount data for example) and
provide management tools in our framework around it.

> >> I think you are suggesting we ignore this problem for now, and just
> >> have the tools pretend the other protocol does not exist, while still
> >> allowing the possibility of storing both types of metadata in the same
> >> filesystem object.  That may be an easy way to get started.
> > 
> > I thought this was what you were proposing actually. With my
> > 'integrator' hat on I would rather quickly define common tools that can
> > handle both, and have the old tools return loud warnings (were possible)
> > if you try to use them.
> 
> The "old" FedFS tools in this case are available because they are one
> of two independent reference implementations required for the IETF
> standardization process.  However, I think solving the problem of
> integrating FedFS ADMIN with an existing DFS implementation on a
> multi-protocol fileserver is going to be very valuable implementation
> experience that can be fed back into the IETF standards process.
> 
> I'm not confident we can fit rpc.fedfsd (Linux's implementation of the
> FedFS RPC ADMIN service) into this model, so we may end up abandoning
> its use for managing FedFS junctions, leaving it only to provisioning
> NSDB certificates.
> 
> > In the Samba case the basic tool is 'ln', not sure we can do much about
> > it :-) But we can certainly patch the RPC code that allows handling via
> > SMB/RPC although I am not quite sure how to populate all the data the
> > SMB world has no concept of, perhaps using 'good defaults' ...
> 
> Exactly the problem.
> 
> My thought was to define a place-holder value for both protocols that
> can be used by our existing tools, for now.  No xattr for symlinks
> created on behalf of DFS, and maybe an empty symlink for junctions
> created by rpc.fedfsd or the nfsref command.
> 
Sound like a good first start to me.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

Re: Interoperable junctions on Linux

[Matt W. Benjamin]

----- "Simo Sorce" <simo@redhat.com> wrote:

> 
> Well I have some knowledge in this field, and I do not find it very
> fiddly, but I may be biased as I have been working for the past 7
> years
> to make LDAP+Kerberos simple to manage within the FreeIPA project.
> Your requirement to set an objectclass on the base suffix is something
> I
> find particularly unappealing, and no other tool that I know of
> requires
> this (because it is unnecessary).

Yes.  Thank you.

> > 
> 
> I think you should allow the broadest possibilities of course, which
> is
> why I am picking on things like allowing SSAL/GSSAPI explicitly in
> the
> RFC language. Whether people will integrate into existing LDAP server
> or
> not remains to be seen, if we can avoid the need to add an objectlass
> on
> the root suffix I see that we can easily add this a standard feature
> for
> FreeIPA as well (we already provide automount data for example) and
> provide management tools in our framework around it.

I noted this point on a FedFS concall 2 years ago.  I dont understand why
not -permit- easy integration with existing LDAP infrastructure.


-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309 

Re: Interoperable junctions on Linux

[Chuck Lever]

On Apr 23, 2013, at 2:37 PM, Simo Sorce <simo@redhat.com> wrote:

> On Tue, 2013-04-23 at 14:11 -0400, Chuck Lever wrote:
>> On Apr 23, 2013, at 12:54 PM, Simo Sorce <simo@redhat.com> wrote:
> 
>>> I do not see mention of LDAP URIs, andyou seem to speak in the singular
>>> when mention 'the LDAP server' does it mean you hve no way to specify a
>>> pool of LDAP servers for HA ?
>> 
>> An "NSDB name" (used in a junction) is meant to represent a pool of
>> replicated LDAP server resources, or just one LDAP server.  Note that
>> specifying IP addresses in FedFS Fileset Names is specifically
>> disallowed for this reason.
>> 
>> A FedFS Fileset Name can indeed be represented by an LDAP URI.  See
>> the operational examples in the FedFS NSDB draft.
> 
> Sorry I found that after some more searching, the document is vast and I
> missed it at first glance.
> 
>>> 
>>>> An explanation of this data is in an IETF draft:
>>>> 
>>>> 
>>>> http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/
>>> 
>>>> See chapter 4 for an overview of the schema used for these lists.  An
>>>> NFS fileserver converts the LDAP records into an fs_locations4 or
>>>> fs_locations_info4 attribute for NFS clients.  Other protocols use a
>>>> different representation for communicating this list to clients.
>>>> 
>>> To be honest the constrains in this document on the LDAP DIT, seem to
>>> indicate it will be possible to use this stuff primarily only with a
>>> dedicated LDAP server, Dictating how the rootDSE/namingContext should
>>> look like is a quite strong demand.
>>> 
>>> Why do you need a full namingContext for FedFS ?
>>> Why a subtree is not sufficient ?
>> 
>> The document does recommend using a dedicated LDAP service for an
>> NSDB, but does not require it.
> 
> Ah I didn't realize you recommend a dedicated LDAP server.
> 
>> One or more of the namingContext records on an NSDB has a new
>> attribute (fedfsNceDN) that points to the FedFS DIT.  So it is
>> entirely possible to use an existing LDAP server for storing FedFS
>> records, simply by adding that attribute to the namingContext under
>> which the FedFS subtree resides.
> 
> That 'simply' is not so simple, you may not be allowed to do that.

In which case, you set up your own LDAP server, where you _are_ allowed to do it.

> 
>> The goal is to allow automated discovery of the location of the FedFS
>> DIT using a DN advertised in the LDAP server's rootDSE.  NSDB clients
>> thus need know only the name of the NSDB service.
> 
> You can do the same w/o forcing the attribute to be on the base suffix,
> all you need to do is a subtree search with
> '&(objctclass=fedfsNsdbContainerInfo)' as part of the filter.
> Given this objectclass is used as the container entry main objectclass
> it also means the fedfsNceDN would be redundant and could be removed
> entirely.

Would your search filter be able to find a fedfsNsdbContainerInfo objectclass in a namingContext _and_ under it?  The search base is determined first by this search, then the NSDB client can search for a particular Fileset Name under that base DN.

Although I have been working to simplify this mechanism as specified in the NSDB document, I don't remember the specific reason we require namingContext changes (I inherited this work from the original authors).  I agree that a subtree search would be simpler.

But we may be too far along in the standards process to address this concern in this iteration of the protocol. If we can't fix this protocol specification, it's certainly possible to construct our own implementation without fedfsNceDn that does not interoperate, and then document our implementation in a new protocol specification.

> 
>> By default, NSDBs that my tools set up store the FedFS DIT under the
>> LDAP server's domain controller suffix, rather than using the special
>> "o=fedfs" DN.  By convention the FedFS domain name matches the domain
>> controller suffix, but operation doesn't depend on this.
>> 
>> It is not quite as restrictive as you think, but it is very fiddly to
>> set up on industrial strength LDAP servers like OpenLDAP or 389-ds.
> 
> Well I have some knowledge in this field, and I do not find it very
> fiddly, but I may be biased as I have been working for the past 7 years
> to make LDAP+Kerberos simple to manage within the FreeIPA project.
> Your requirement to set an objectclass on the base suffix is something I
> find particularly unappealing, and no other tool that I know of requires
> this (because it is unnecessary).

I presented the NSDB schema to both OpenLDAP and 389-ds developers last year.  Neither group suggested that fiddling with an LDAP server's rootDSE was onerous (although they didn't like the way we were doing it, and that has since been changed).

If you can provide back up documentation for your root suffix issue, I can put this in front of the Working Group.  "It's particularly unappealing" is not going to fly ;-)  Can you help me formulate a proper problem statement (off list, of course)?


>> 
>>> On the security side, you recommend RPCSEC_GSS for NFS, but then TLS
>> for
>>> LDAP, why not use SASL/GSSAPI for LDAP as well so you need a single
>> set
>>> of credentials ?
>> 
>> There are two classes of tasks involving NSDBs: administrative, where
>> an administrator is available to provide credential material; and
>> fileserver, which is an unattended access performed simply to retrieve
>> location lists.
>> 
>> The purpose of TLS is to allow fileservers to authenticate NSDBs
>> without requiring user intervention.  A certificate chain for each
>> NSDB is provisioned on fileservers (using the FedFS ADMIN RPC
>> protocol).  The fileservers use them with START_TLS each time a
>> connection is established to that NSDB.
>> 
>> SASL/GSSAPI can of course be used when performing administrative
>> operations on an NSDB, and that is the plan for the Linux FedFS
>> implementation.  As you observed, an administrator can then use the
>> same credentials for an RPCGSS-protected FedFS ADMIN service and her
>> NSDB.
> 
> SASL/GSSAPI can also be used by the file server given the use of
> RPCSEC_GSS requires you to have a keytab there. This means I have to
> care only to drop a keytab and I get mutual authentication and
> encryption of the channel to LDAP w/o having to care to get a x509
> certificate and configure the server to trust it directly or drop a
> suitable CA Certificate.
> 
> I find that properly configuring SSL is a lot more difficult than simply
> dropping a keytab in place (especially given you alredy need to do it).
> 
> Basically in most setups I encounter for real SSL is effectively
> "disabled" and by that I mean that peers are configured to ignore
> untrusted certificates, also certificates expire and the whole lifecycle
> of using SSL certificates *properly* is a lot more complex than using
> GSSAPI with a simple keytab file.

Having just implemented TLS support in my NSDB client, I can attest to the challenges.

> So I strongly recommend you put language in your RFC allowing
> SASL/GSSAPI as a mechanism for securing communication to the LDAP server
> for both administrative and simple retrieval, you will do a favor to the
> implementer allowing conformance w/o having to jump though hoops.

We have discussed SASL/GSSAPI with LDAP server implementors, and it is already on the radar.  It should be possible to introduce with another FEDFS_SEC connection security type, but the plan was to introduce SASL/GSSAPI in the next version of the NSDB protocol.

Again, documentation references would help me formulate a problem statement to address this at this late date, and we can still take the approach of building something that doesn't interoperate for now if we think this is a strong requirement.

> [Sorry for the OT, but this really is something you should get right]
> 
> 
>>>> If we are really wily, maybe a small single-purpose daemon can be
>>>> constructed from a minimal LDAP server implementation (or from
>>>> scratch), and it can listen on its own port or only for loopback
>>>> requests.
>>> 
>>> I think in most cases this is what will actually happen, but you do not
>>> need to use a special purpose built server, you can use an existing LDAP
>>> server simply specially configured for your needs. It will cause
>>> administrative overhead to handle this infrastructure though.
>> 
>> Don't I know it.
>> 
>> To simplify initial configuration, I've implemented a script that
>> installs OpenLDAP and performs enough configuration to provide a
>> stand-alone NSDB, with or without TLS.  I already have a number of C
>> tools that can set up the rootDSE and add Fileset Name records on
>> either 389-ds or OpenLDAP servers.
>> 
>> The idea is to allow both types of NSDB (stand-alone, and as part of
>> an existing LDAP service), but that kind of generality adds a lot of
>> complexity to the set up process and documentation.  Going with just
>> one or the other makes life much easier.
> 
> I think you should allow the broadest possibilities of course, which is
> why I am picking on things like allowing SSAL/GSSAPI explicitly in the
> RFC language.

Because there are so many moving parts to FedFS, I have striven to make set up as simple as possible.  Just because the spec says you can do something, doesn't mean it is wise to do it, or that you should offer it as an option, nor is it appropriate for every implementation.

Practically speaking, implementation resources are limited.  I have to pick the features that offer the most bang for the buck.  I can't afford to implement everything and see which features are appealing.  (And besides, I thought the idea of open source was to let people scratch their own itches -- if someone wants a feature, they have the opportunity to add it themselves).

> Whether people will integrate into existing LDAP server or
> not remains to be seen, if we can avoid the need to add an objectlass on
> the root suffix I see that we can easily add this a standard feature for
> FreeIPA as well (we already provide automount data for example) and
> provide management tools in our framework around it.

Does FreeIPA manage DFS metadata as well?


> 
>>>> I think you are suggesting we ignore this problem for now, and just
>>>> have the tools pretend the other protocol does not exist, while still
>>>> allowing the possibility of storing both types of metadata in the same
>>>> filesystem object.  That may be an easy way to get started.
>>> 
>>> I thought this was what you were proposing actually. With my
>>> 'integrator' hat on I would rather quickly define common tools that can
>>> handle both, and have the old tools return loud warnings (were possible)
>>> if you try to use them.
>> 
>> The "old" FedFS tools in this case are available because they are one
>> of two independent reference implementations required for the IETF
>> standardization process.  However, I think solving the problem of
>> integrating FedFS ADMIN with an existing DFS implementation on a
>> multi-protocol fileserver is going to be very valuable implementation
>> experience that can be fed back into the IETF standards process.
>> 
>> I'm not confident we can fit rpc.fedfsd (Linux's implementation of the
>> FedFS RPC ADMIN service) into this model, so we may end up abandoning
>> its use for managing FedFS junctions, leaving it only to provisioning
>> NSDB certificates.
>> 
>>> In the Samba case the basic tool is 'ln', not sure we can do much about
>>> it :-) But we can certainly patch the RPC code that allows handling via
>>> SMB/RPC although I am not quite sure how to populate all the data the
>>> SMB world has no concept of, perhaps using 'good defaults' ...
>> 
>> Exactly the problem.
>> 
>> My thought was to define a place-holder value for both protocols that
>> can be used by our existing tools, for now.  No xattr for symlinks
>> created on behalf of DFS, and maybe an empty symlink for junctions
>> created by rpc.fedfsd or the nfsref command.
>> 
> Sound like a good first start to me.

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: Interoperable junctions on Linux

[Simo Sorce]

On Tue, 2013-04-23 at 15:52 -0400, Chuck Lever wrote:
> On Apr 23, 2013, at 2:37 PM, Simo Sorce <simo@redhat.com> wrote:
> 
> > On Tue, 2013-04-23 at 14:11 -0400, Chuck Lever wrote:

> >> One or more of the namingContext records on an NSDB has a new
> >> attribute (fedfsNceDN) that points to the FedFS DIT.  So it is
> >> entirely possible to use an existing LDAP server for storing FedFS
> >> records, simply by adding that attribute to the namingContext under
> >> which the FedFS subtree resides.
> > 
> > That 'simply' is not so simple, you may not be allowed to do that.
> 
> In which case, you set up your own LDAP server, where you _are_ allowed to do it.

Of course, if forced that is an option, but I would rather see to it
that we do not add barriers to adoption that force people to deploy
additional LDAP servers where possible.

> > 
> >> The goal is to allow automated discovery of the location of the FedFS
> >> DIT using a DN advertised in the LDAP server's rootDSE.  NSDB clients
> >> thus need know only the name of the NSDB service.
> > 
> > You can do the same w/o forcing the attribute to be on the base suffix,
> > all you need to do is a subtree search with
> > '&(objctclass=fedfsNsdbContainerInfo)' as part of the filter.
> > Given this objectclass is used as the container entry main objectclass
> > it also means the fedfsNceDN would be redundant and could be removed
> > entirely.
> 
> Would your search filter be able to find a fedfsNsdbContainerInfo
> objectclass in a namingContext _and_ under it?  The search base is
> determined first by this search, then the NSDB client can search for a
> particular Fileset Name under that base DN.

That's what subtree searches are for, if your search has scope subtree
you'll find *any* entry that has that objectclass in the specific
namingContext you are using for the search.

> Although I have been working to simplify this mechanism as specified
> in the NSDB document, I don't remember the specific reason we require
> namingContext changes (I inherited this work from the original
> authors).  I agree that a subtree search would be simpler.

It is, it would be nice if you'd allow that.

> But we may be too far along in the standards process to address this
> concern in this iteration of the protocol. If we can't fix this
> protocol specification, it's certainly possible to construct our own
> implementation without fedfsNceDn that does not interoperate, and then
> document our implementation in a new protocol specification.

Well the change is not huge, you could allow both with minor language
changes. Knowing there is an interoperability issue and not addressing
it when it is simple to do so is not really friendly to implementers.

> > 
> >> By default, NSDBs that my tools set up store the FedFS DIT under the
> >> LDAP server's domain controller suffix, rather than using the special
> >> "o=fedfs" DN.  By convention the FedFS domain name matches the domain
> >> controller suffix, but operation doesn't depend on this.
> >> 
> >> It is not quite as restrictive as you think, but it is very fiddly to
> >> set up on industrial strength LDAP servers like OpenLDAP or 389-ds.
> > 
> > Well I have some knowledge in this field, and I do not find it very
> > fiddly, but I may be biased as I have been working for the past 7 years
> > to make LDAP+Kerberos simple to manage within the FreeIPA project.
> > Your requirement to set an objectclass on the base suffix is something I
> > find particularly unappealing, and no other tool that I know of requires
> > this (because it is unnecessary).
> 
> I presented the NSDB schema to both OpenLDAP and 389-ds developers
> last year.  Neither group suggested that fiddling with an LDAP
> server's rootDSE was onerous (although they didn't like the way we
> were doing it, and that has since been changed).

I have the integrator perspective, which differ from the core developers
perspective at times.

> If you can provide back up documentation for your root suffix issue, I
> can put this in front of the Working Group.  "It's particularly
> unappealing" is not going to fly ;-)  Can you help me formulate a
> proper problem statement (off list, of course)?

I am not sure what documentation you require, what you ask *can* be done
to an LDAP server, it's not like your are doing something forbidden. It
is just not what all other extensions do. In FreeIPA I would be
uncomfortable adding that stuff in the root suffix, doesn't mean I will
Veto it, but I might see if I can get away not following the standard in
that regard if we come to integrate this as a default feature.

For Active Directory admins (should somoene want to use AD for this
information) changing the root suffix may also be very unappealing (if
possible at all), although they will find extending the schema
problematic already.

It is a matter of best practices and 'acceptability' more than anything.
It is more acceptable as a solution if you do not have this particular
constraint, which is technically unnecessary so I am not sure why the
RFC process should be dead set on mandating it. Surely you can relax
some MUST here and there ?

> > SASL/GSSAPI can also be used by the file server given the use of
> > RPCSEC_GSS requires you to have a keytab there. This means I have to
> > care only to drop a keytab and I get mutual authentication and
> > encryption of the channel to LDAP w/o having to care to get a x509
> > certificate and configure the server to trust it directly or drop a
> > suitable CA Certificate.
> > 
> > I find that properly configuring SSL is a lot more difficult than simply
> > dropping a keytab in place (especially given you alredy need to do it).
> > 
> > Basically in most setups I encounter for real SSL is effectively
> > "disabled" and by that I mean that peers are configured to ignore
> > untrusted certificates, also certificates expire and the whole lifecycle
> > of using SSL certificates *properly* is a lot more complex than using
> > GSSAPI with a simple keytab file.
> 
> Having just implemented TLS support in my NSDB client, I can attest to the challenges.
> 
> > So I strongly recommend you put language in your RFC allowing
> > SASL/GSSAPI as a mechanism for securing communication to the LDAP server
> > for both administrative and simple retrieval, you will do a favor to the
> > implementer allowing conformance w/o having to jump though hoops.
> 
> We have discussed SASL/GSSAPI with LDAP server implementors, and it is
> already on the radar.  It should be possible to introduce with another
> FEDFS_SEC connection security type, but the plan was to introduce
> SASL/GSSAPI in the next version of the NSDB protocol.

Ok, if it is on your radar I am fine. Not sure why you say NSDB
'protocol' I am assuming this is the LDAP protocol, or are you planning
on deviating from LDAP ??

> Again, documentation references would help me formulate a problem
> statement to address this at this late date, and we can still take the
> approach of building something that doesn't interoperate for now if we
> think this is a strong requirement.

What sort of documentation ?

SASL/GSSAPI is a standard SASL mechanism and SASL is a
standard ,echanism for LDAP binds, so all the references should be easy.

Using SASL/GSSAPI would make it easier for some directory services that
are tightly integrated with Kerberos.
In my case, the FreeIPA project uses SASl/GSSAPI with LDAP, although we
do configure and support also STRAT_TLS by default.
In the Active Directory case SSL is not enabled by default (and it is
hard and unconvenient to configure), while SASL/GSSAPI is.

So SASL/GSSAPI would be an easier route for adoption in some circles.


> > I think you should allow the broadest possibilities of course, which is
> > why I am picking on things like allowing SSAL/GSSAPI explicitly in the
> > RFC language.
> 
> Because there are so many moving parts to FedFS, I have striven to
> make set up as simple as possible.  Just because the spec says you can
> do something, doesn't mean it is wise to do it, or that you should
> offer it as an option, nor is it appropriate for every implementation.

I think we have al;ready determined SASL/GSSAPI is actually simpler than
TLS, so if you are striving for simplicity... :)

> Practically speaking, implementation resources are limited.  I have to
> pick the features that offer the most bang for the buck.  I can't
> afford to implement everything and see which features are appealing.
> (And besides, I thought the idea of open source was to let people
> scratch their own itches -- if someone wants a feature, they have the
> opportunity to add it themselves).

Not sure what there is to implement, if you are using a LDAP library in
your client, switching on SASL/GSSAPI is a very few lines of code if
any.

> > Whether people will integrate into existing LDAP server or
> > not remains to be seen, if we can avoid the need to add an objectlass on
> > the root suffix I see that we can easily add this a standard feature for
> > FreeIPA as well (we already provide automount data for example) and
> > provide management tools in our framework around it.
> 
> Does FreeIPA manage DFS metadata as well?

No, DFS data requires Active Directory in the CIFS worls, and freeIPA is
not an AD implementation. FreeIPA is focused on Linux clients though so
FedFS junctions may be appealing to support.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York