Configuring fs_locations on Linux upstream server pseudo fs for session trunking

Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[Adamson, Andy]

QW5uYSBTY2h1bWFrZXIgd2hvIHJldmlld2VkIG15IGNsaWVudCBzaWRlIHNlc3Npb24gdHJ1bmtp
bmcgcGF0Y2hzZXQsIHdhbnRzIGEgZnVsbCBmZWF0dXJlZCB2ZXJzaW9uIG9mIGJvdGggdGhlIGNs
aWVudCBhbmQgdGhlIHNlcnZlciBzZXNzaW9uIHRydW5raW5nIHBpZWNlcyBiZWZvcmUgYWNjZXB0
aW5nIHRoZSBzZXNzaW9uIHRydW5raW5nIGZlYXR1cmUgdXBzdHJlYW0uIFRvIHRoYXQgZW5kLCBJ
IHdhbnQgdG8gaW1wbGVtZW50IHRoZSBzZXJ2ZXIgbW91bnRkIFY0Uk9PVCBwcm9jZXNzaW5nIG9m
IGFuIGZzX2xvY2F0aW9ucyBjb25maWd1cmF0aW9uIHRvIHNhdGlzZnkgYW4gZnNfbG9jYXRpb25z
IHJlcXVlc3Qgb24gdGhlIHBzZXVkbyBmcy4NCg0KVGhlIGZvcndhcmRlZCBtZXNzYWdlIGlzIGZy
b20gYW4gZW1haWwgc3RyZWFtIGJldHdlZW4gQnJ1Y2UsIENodWNrIGFuZCBJIGNvbmNlcm5pbmcg
dGhlIHNlcnZlciBwc2V1Zm8gZnMgZnNfbG9jYXRpb25zIGNvbmZpZ3VyYXRpb24gdGhhdCBJ4oCZ
bSBub3cgc2hhcmluZyB3aXRoIHRoZSBsaXN0Lg0KDQpTb21lIGJhY2tncm91bmQ6DQoNClRoZSBy
ZWNlbnQgIk5GU1Y0LjEsMiBzZXNzaW9uIHRydW5raW5n4oCdIFZlcnNpb24tNSBwYXRjaCBzZXQg
c2VudCB0byB0aGUgbGlzdCBub3RlcyAoaW4gcGF0Y2ggMDAvMTApOg0KDQpUaGUgcHNldWRvLWZz
IEdFVEFUVFIoZnNfbG9jYXRpb25zKSBwcm9iZSBzZXNzaW9uIHRydW5raW5nDQp3YXMgdGVzdGVk
IGFnYWluc3QgYSBMaW51eCBzZXJ2ZXIgd2l0aCBhIHBzZXVkby1mcw0KZXhwb3J0IHN0YW56YSAo
ZS5nLiBhIHN0YW56YSB3aXRoIHRoZSBmc2lkPTAgb3IgZnNpZD1yb290DQpleHBvcnQgb3B0aW9u
KSBhbmQgYSByZXBsaWNhcz0gZXhwb3J0IG9wdGlvbg0KKHJlcGxpY2FzPTxwYXRoMT5APHNlcnZl
cjE+OjxwYXRoMj5APHNlcnZlcjI+Li4pDQpOb3RlIHRoYXQgdGhpcyBjb25maWd1cmF0aW9uIGlz
IGZvciB0ZXN0aW5nIG9ubHkuIEEgZnV0dXJlDQpwYXRjaHNldCB3aWxsIGFkZCB0aGUgcmVwbGlj
YXM9IGNvbmZpZ3VyYXRpb24gdG8gdGhlDQpORlNFWFBfVjRST09UIG5mc2QgYW5kIG1vdW50ZCBw
cm9jZXNzaW5nLg0KDQoNClRoZXJlIGFyZSBzZXZlcmFsIGlkZWFzIG9uIGhvdyB0byBhY2NvbXBs
aXNoIG1vdW50ZC9WNFJPT1QgZnNfbG9jYXRpb25zIGNvbmZpZ3VyYXRpb24gaW4gdGhlIGZvcndh
cmRlZCBtZXNzYWdlLiBTZWUgaW5saW5lLg0KDQoNCj4gQmVnaW4gZm9yd2FyZGVkIG1lc3NhZ2U6
DQo+IA0KPiBGcm9tOiBDaHVjayBMZXZlciA8Y2h1Y2subGV2ZXJAb3JhY2xlLmNvbT4NCj4gU3Vi
amVjdDogUmU6IENvbmZpZ3VyaW5nIGZzX2xvY2F0aW9ucyBvbiBMaW51eCB1cHN0cmVhbSBzZXJ2
ZXINCj4gRGF0ZTogTWF5IDYsIDIwMTYgYXQgNDozMTowMCBQTSBFRFQNCj4gVG86ICJKLiBCcnVj
ZSBGaWVsZHMiIDxiZmllbGRzQGZpZWxkc2VzLm9yZz4NCj4gQ2M6ICJBZGFtc29uLCBBbmR5IiA8
V2lsbGlhbS5BZGFtc29uQG5ldGFwcC5jb20+DQo+IA0KPiANCj4+IE9uIE1heSA2LCAyMDE2LCBh
dCA0OjE2IFBNLCBKLiBCcnVjZSBGaWVsZHMgPGJmaWVsZHNAZmllbGRzZXMub3JnPiB3cm90ZToN
Cj4+IA0KPj4gT24gRnJpLCBNYXkgMDYsIDIwMTYgYXQgMDI6MjA6MTJQTSAtMDQwMCwgQ2h1Y2sg
TGV2ZXIgd3JvdGU6DQo+Pj4gU2VlbXMgbGlrZSB3aGVuIGEgc2VydmVyIGRvZXMgbm90IHJldHVy
biBhIGxpc3QsIHRoYXQgaXMNCj4+PiBpbmZvcm1hdGlvbiB0aGUgY2xpZW50IGNhbiB1c2U6IGJh
c2ljYWxseSwgdGhlcmUgaXMgbm8NCj4+PiBhYmlsaXR5IHRvIGRvIGFueSBzZXNzaW9uIHRydW5r
aW5nLiBJdCBoYXMgdG8gYmUgc2V0IHVwDQo+Pj4gZXhwbGljaXRseTsgaXMgdGhhdCBhIGJhZCB0
aGluZywgb3BlcmF0aW9uYWxseT8NCj4+IA0KPj4gSSBsaWtlIHRoZSBpZGVhIG9mIGl0IGJlaW5n
IG9wdCBpbiBvbiB0aGUgc2VydmVyLg0KPj4gDQo+PiBTdXBwb3NlIHRoZSBzZXJ2ZXIgdHJhbnNw
YXJlbnRseSBzdGFydHMgYWR2ZXJ0aXNpbmcgYWxsIGF2YWlsYWJsZQ0KPj4gYWRkcmVzc2VzIGZv
ciBzZXNzaW9uIHRydW5raW5nLiAgSXQncyBub3QgaGFyZCB0byBpbWFnaW5lIGNhc2VzIHdoZXJl
DQo+PiB0aGF0IHdvdWxkIGdvIHdyb25nLiAgRS5nLiwgbWF5YmUgdGhlIHNlcnZlciBoYXMgdGhl
IG9kZCB3aXJlbGVzcyBvcg0KPj4gMTAwTWIgb3Igb3RoZXIgaW50ZXJmYWNlIHRoYXQgaGFwcGVu
cyB0byB3b3JrIGJ1dCB0aGF0J3Mgc2xvdy4gIFRoZW4NCj4+IHNvbWVib2R5IHVwZ3JhZGVzIHRo
ZWlyIHNlcnZlciBhbmQgcGVyZm9ybWFuY2UgZ29lcyBkb3duIGFuZCBpdCBtYXkgdGFrZQ0KPj4g
dGhlbSBhIHdoaWxlIHRvIGZpZ3VyZSBvdXQgd2h5LiAgV2hlcmVhcyBpZiB0aGV5J2QgaGFkIHRv
IG9wdCBpbiB0aGV5J2QNCj4+IHByb2JhYmx5IGhhdmUgYXZvaWRlZCBhZHZlcnRpc2luZyBhbiBp
bmFwcHJvcHJpYXRlIGludGVyZmFjZS4gIE9yIGF0DQo+PiBsZWFzdCB0aGV5J2QgaGF2ZSBhIGJl
dHRlciBjaGFuY2Ugb2YgZmlndXJpbmcgb3V0IHRoYXQgdHVybmluZyBvbg0KPj4gdHJ1bmtpbmcg
d2FzIHdoYXQgY2F1c2VkIHRoZSBwcm9ibGVtLg0KPj4gDQo+PiBJJ2QgcmF0aGVyIG5vdCBmb3Jj
ZSBwZW9wbGUgdG8gZXhwb3J0ICIvIiBleHBsaWNpdGx5LCB0aG91Z2guICBJdCdzIGZpbmUNCj4+
IGZvciB0ZXN0aW5nLCBidXQ6DQo+PiANCj4+IAktIEkgZG9uJ3QgdGhpbmsgd2UgZ2l2ZSBhIHdh
eSB0byBkbyBhbiBleHBsaWNpdCBWNFJPT1QgZXhwb3J0LA0KPj4gCSAgc28gdGhleSdkIGJlIGV4
cG9zaW5nIHRoZWlyIGVudGlyZSByb290IHBhcnRpdGlvbi4gIFdlIGNvdWxkDQo+PiAJICBmaXgg
dGhhdCwgYnV0DQo+PiAJLSB0aGUgcHNldWRvZnMganVzdCBzZWVtcyB0byBtZSBsaWtlIHNvbWV0
aGluZyBwZW9wbGUgc2hvdWxkbid0DQo+PiAJICBub3JtYWxseSBoYXZlIHRvIHRoaW5rIGFib3V0
LiAgSXQncyBhIHByb3RvY29sIGltcGxlbWVudGF0aW9uDQo+PiAJICBkZXRhaWwsIEknZCByYXRo
ZXIgaGlkZSBpdC4gIEl0J2QgYmUgdG8gZWFzeSB0byBjb25maWd1cmUgaXQgYQ0KPj4gCSAgbGl0
dGxlIHdyb25nLCBJIHRoaW5rLg0KPj4gDQo+PiBXZSBjYW4gc3RpbGwgZG8gdGhpcyBieSBhZGRp
bmcgYSByZXBsaWNhcz0gb3B0aW9uIHRvIHRoZSAvIGV4cG9ydCwgYnV0DQo+PiB3ZSBjYW4gbGV0
IHJwYy5tb3VudGQgZG8gdGhhdCBpbnRlcm5hbGx5IGluc3RlYWQgb2YgbWFraW5nIHRoZSBhZG1p
biBhZGQNCj4+IGl0IHRvIC9ldGMvZXhwb3J0cy4NCj4+IA0KPj4gQnV0IHRoZW4geW91IHN0aWxs
IG5lZWQgYSB3YXkgZm9yIHRoZSBhZG1pbiB0byB0ZWxsIHJwYy5tb3VudGQgdG8gY29vaw0KPj4g
dXAgdGhlIHJlcGxpY2FzPSBvcHRpb24uLi4uLiAgSSdtIG5vdCBzdXJlIHdoYXQgdGhhdCBzaG91
bGQgbG9vayBsaWtlLg0KDQpJZGVhIDE6IGV4dHJhIHN5bnRheCBpbiAvZXRjL2V4cG9ydHMNCg0K
DQo+PiBNYXliZSBzb21lIGV4dHJhIHN5bnRheCBpbiAvZXRjL2V4cG9ydHMsIGJ1dCB3aGF0IGRv
IHRoZXkgbmVlZCB0byBnaXZlDQo+PiB1cy0tanVzdCBvbmUgbGlzdCBvZiBJUCBhZGRyZXNzZXM/
ICBDaHVjaywgYW55IGlkZWFzPw0KDQpJZGVhIDI6IHhhdHRyIGF0dGFjaGVkIHRvIOKAnC8iDQoN
Cj4gDQo+IEhvdyBhYm91dCB1c2luZyB0aGUgc2FtZSBhcHByb2FjaCB1c2VkIGZvciBqdW5jdGlv
bnM6DQo+IHB1dCB0aGUgbGlzdCBpbiBhbiB4YXR0ciBhdHRhY2hlZCB0byAvID8gbW91bnRkIGNh
bg0KPiBleHRyYWN0IHRoYXQgd2hlbiB0aGUga2VybmVsIGFza3MgZm9yIGhlbHAgc2F0aXNmeWlu
Zw0KPiBhIEdFVEFUVFIoZnNfbG9jYXRpb25zKSBvbiBWNFJPT1QuDQoNCg0KSWRlYSAzOiBuZXcg
L2V0Yy8gY29uZmlnIGZpbGUNCj4gDQo+IE9yIGl0IGNvdWxkIGJlIHB1dCBpbiBhIHNlcGFyYXRl
IGNvbmZpZyBmaWxlIGluIC9ldGMuDQo+IFlvdSBtaWdodCB3YW50IHRvIHNwZWNpZnkgbW9yZSB0
aGFuIGp1c3QgdGhlIGkvZiBsaXN0DQo+IGhlcmU7IGZvciBpbnN0YW5jZSwgdGhlIHNlY3VyaXR5
IHBvbGljeSBmb3IgdGhlDQo+IHBzZXVkb2ZzLCBvciBhIGNvbnN0YW50IGZzaWQgVVVJRCwgYW1v
bmcgb3RoZXIgdGhpbmdzLg0KDQoNCkFQSSB0byB1cGRhdGUgdGhlIGkvZiBsaXN0LiAgVGhpcyBp
cyBub3QgYWJvdXQgd2hlcmUgdG8gaG9sZCBmc19sb2NhdGlvbnMgY29uZmlnIGluZm8sIGJ1dCBy
YXRoZXIgaG93IHRvIGluc2VydCB0aGUgKGNoYW5nZWQpIGluZm8gaW50byB0aGUgcnVubmluZyBz
eXN0ZW0uDQoNCj4gDQo+IEFsc28sIEkgc3VnZ2VzdGVkIHRvIEFuZHkgZWFybGllcjoNCj4gDQo+
PiBJIGZpbmQgbXlzZWxmIGxlYW5pbmcgdG93YXJkcyBtZWNoYW5pc21zIHRoYXQgYXJlIGVhc3kN
Cj4+IGJvdGggZm9yIGFkbWlucyBhbmQgZm9yIHByb2dyYW1zIChpZSwgYW4gQVBJKS4gUGVyaGFw
cw0KPj4gb25lIGRheSB5b3UgbWlnaHQgd2FudCB0byBhZGQgYSBjb21tYW5kIHRoYXQgdXBkYXRl
cyB0aGUNCj4+IGkvZiBsaXN0IGZyb20gdGhlIHNjcmlwdHMgaW4gL2V0Yy9zeXNjb25maWcvbmV0
d29yay1zY3JpcHRzLA0KPj4gZm9yIGluc3RhbmNlLg0KPj4gDQo+PiBBcyBwYXJ0IG9mIGFuIGlm
dXA6DQo+PiANCj4+IG5mc3BmcyBhZGQgPGFkZHI+DQo+PiANCj4+IGFuZCBpZmRvd246DQo+PiAN
Cj4+IG5mc3BmcyByZW1vdmUgPGFkZHI+DQo+PiANCj4+IEkgd3JvdGUgc29tZSBQeXRob24gY29k
ZSB0byBtYW5pcHVsYXRlIGVudHJpZXMgaW4NCj4+IC9ldGMvZXhwb3J0cywgbm93IGZvdW5kIGlu
IGZlZGZzLXV0aWxzLiBJdCdzIGlja3kuDQo+IA0KPiANCj4gSSB0aGluayB3ZSBzaG91bGQgbW92
ZSBhd2F5IGZyb20gImVkaXQgdGhpcyBmaWxlDQo+IGFuZCBzYXZlIGl0LCB0aGVuIHJlc3RhcnQg
cnBjLnh5enBkcSIuIEJ1aWxkIHNvbWUNCj4gY29tbWFuZCBsaW5lIGludGVyZmFjZXMgZm9yIHRo
aXMuDQo+IA0KPiBBbmQgYXMgeW91IGhhdmUgc3VnZ2VzdGVkIG1hbnkgdGltZXM6IHNlcGFyYXRl
DQo+IHBvbGljeSBmcm9tIG1lY2hhbmlzbS4gL2V0Yy9leHBvcnRzIGlzIHRoZQ0KPiBtZWNoYW5p
c20uDQo+IA0KPiAtLQ0KPiBDaHVjayBMZXZlcg0KDQpCcnVjZSAtIGRvIHlvdSBoYXZlIGEgcHJl
ZmVyZW5jZSBiZXR3ZWVuICMxIGFuZCAjMiBvciAjMyAob3IgYW5vdGhlciBpZGVhPykNCg0KVGhh
bmtzDQoNCuKAlD5BbmR5

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[bfields]

On Wed, May 25, 2016 at 05:29:35PM +0000, Adamson, Andy wrote:
> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs.
> 
> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list.
> 
> Some background:
> 
> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10):
> 
> The pseudo-fs GETATTR(fs_locations) probe session trunking
> was tested against a Linux server with a pseudo-fs
> export stanza (e.g. a stanza with the fsid=0 or fsid=root
> export option) and a replicas= export option
> (replicas=<path1>@<server1>:<path2>@<server2>..)
> Note that this configuration is for testing only. A future
> patchset will add the replicas= configuration to the
> NFSEXP_V4ROOT nfsd and mountd processing.
> 
> 
> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline.
> 
> 
> > Begin forwarded message:
> > 
> > From: Chuck Lever <chuck.lever@oracle.com>
> > Subject: Re: Configuring fs_locations on Linux upstream server
> > Date: May 6, 2016 at 4:31:00 PM EDT
> > To: "J. Bruce Fields" <bfields@fieldses.org>
> > Cc: "Adamson, Andy" <William.Adamson@netapp.com>
> > 
> > 
> >> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> >> 
> >> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote:
> >>> Seems like when a server does not return a list, that is
> >>> information the client can use: basically, there is no
> >>> ability to do any session trunking. It has to be set up
> >>> explicitly; is that a bad thing, operationally?
> >> 
> >> I like the idea of it being opt in on the server.
> >> 
> >> Suppose the server transparently starts advertising all available
> >> addresses for session trunking.  It's not hard to imagine cases where
> >> that would go wrong.  E.g., maybe the server has the odd wireless or
> >> 100Mb or other interface that happens to work but that's slow.  Then
> >> somebody upgrades their server and performance goes down and it may take
> >> them a while to figure out why.  Whereas if they'd had to opt in they'd
> >> probably have avoided advertising an inappropriate interface.  Or at
> >> least they'd have a better chance of figuring out that turning on
> >> trunking was what caused the problem.
> >> 
> >> I'd rather not force people to export "/" explicitly, though.  It's fine
> >> for testing, but:
> >> 
> >> 	- I don't think we give a way to do an explicit V4ROOT export,
> >> 	  so they'd be exposing their entire root partition.  We could
> >> 	  fix that, but
> >> 	- the pseudofs just seems to me like something people shouldn't
> >> 	  normally have to think about.  It's a protocol implementation
> >> 	  detail, I'd rather hide it.  It'd be to easy to configure it a
> >> 	  little wrong, I think.
> >> 
> >> We can still do this by adding a replicas= option to the / export, but
> >> we can let rpc.mountd do that internally instead of making the admin add
> >> it to /etc/exports.
> >> 
> >> But then you still need a way for the admin to tell rpc.mountd to cook
> >> up the replicas= option.....  I'm not sure what that should look like.
> 
> Idea 1: extra syntax in /etc/exports

It's not really export-specific information.  I wonder if it'd be better
to pass it on the rpc.nfsd commandline?

	rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2"

(and then that can be configured in /etc/sysconfig/nfs or whatever)?

> >> Maybe some extra syntax in /etc/exports, but what do they need to give
> >> us--just one list of IP addresses?  Chuck, any ideas?
> 
> Idea 2: xattr attached to “/"
> 
> > 
> > How about using the same approach used for junctions:
> > put the list in an xattr attached to / ? mountd can
> > extract that when the kernel asks for help satisfying
> > a GETATTR(fs_locations) on V4ROOT.

I don't think that works.  "/" isn't a good place to put configuration.
It could be read-only, among other things.

> Idea 3: new /etc/ config file
> > 
> > Or it could be put in a separate config file in /etc.
> > You might want to specify more than just the i/f list
> > here; for instance, the security policy for the
> > pseudofs, or a constant fsid UUID, among other things.
> 
> 
> API to update the i/f list.  This is not about where to hold fs_locations config info, but rather how to insert the (changed) info into the running system.
> 
> > 
> > Also, I suggested to Andy earlier:
> > 
> >> I find myself leaning towards mechanisms that are easy
> >> both for admins and for programs (ie, an API). Perhaps
> >> one day you might want to add a command that updates the
> >> i/f list from the scripts in /etc/sysconfig/network-scripts,
> >> for instance.
> >> 
> >> As part of an ifup:
> >> 
> >> nfspfs add <addr>
> >> 
> >> and ifdown:
> >> 
> >> nfspfs remove <addr>
> >> 
> >> I wrote some Python code to manipulate entries in
> >> /etc/exports, now found in fedfs-utils. It's icky.
> > 
> > I think we should move away from "edit this file
> > and save it, then restart rpc.xyzpdq". Build some
> > command line interfaces for this.

I'm OK with that.

(Note do have that for information in /etc/exports--we have exportfs.
Is there a reason that didn't work for fedfs-utils?)

--b.

> > 
> > And as you have suggested many times: separate
> > policy from mechanism. /etc/exports is the
> > mechanism.
> > 
> > --
> > Chuck Lever
> 
> Bruce - do you have a preference between #1 and #2 or #3 (or another idea?)
> 
> Thanks
> 
> —>Andy

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[Chuck Lever]

> On May 25, 2016, at 2:48 PM, bfields@fieldses.org wrote:
> 
> On Wed, May 25, 2016 at 05:29:35PM +0000, Adamson, Andy wrote:
>> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs.
>> 
>> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list.
>> 
>> Some background:
>> 
>> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10):
>> 
>> The pseudo-fs GETATTR(fs_locations) probe session trunking
>> was tested against a Linux server with a pseudo-fs
>> export stanza (e.g. a stanza with the fsid=0 or fsid=root
>> export option) and a replicas= export option
>> (replicas=<path1>@<server1>:<path2>@<server2>..)
>> Note that this configuration is for testing only. A future
>> patchset will add the replicas= configuration to the
>> NFSEXP_V4ROOT nfsd and mountd processing.
>> 
>> 
>> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline.
>> 
>> 
>>> Begin forwarded message:
>>> 
>>> From: Chuck Lever <chuck.lever@oracle.com>
>>> Subject: Re: Configuring fs_locations on Linux upstream server
>>> Date: May 6, 2016 at 4:31:00 PM EDT
>>> To: "J. Bruce Fields" <bfields@fieldses.org>
>>> Cc: "Adamson, Andy" <William.Adamson@netapp.com>
>>> 
>>> 
>>>> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>>> 
>>>> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote:
>>>>> Seems like when a server does not return a list, that is
>>>>> information the client can use: basically, there is no
>>>>> ability to do any session trunking. It has to be set up
>>>>> explicitly; is that a bad thing, operationally?
>>>> 
>>>> I like the idea of it being opt in on the server.
>>>> 
>>>> Suppose the server transparently starts advertising all available
>>>> addresses for session trunking.  It's not hard to imagine cases where
>>>> that would go wrong.  E.g., maybe the server has the odd wireless or
>>>> 100Mb or other interface that happens to work but that's slow.  Then
>>>> somebody upgrades their server and performance goes down and it may take
>>>> them a while to figure out why.  Whereas if they'd had to opt in they'd
>>>> probably have avoided advertising an inappropriate interface.  Or at
>>>> least they'd have a better chance of figuring out that turning on
>>>> trunking was what caused the problem.
>>>> 
>>>> I'd rather not force people to export "/" explicitly, though.  It's fine
>>>> for testing, but:
>>>> 
>>>> 	- I don't think we give a way to do an explicit V4ROOT export,
>>>> 	  so they'd be exposing their entire root partition.  We could
>>>> 	  fix that, but
>>>> 	- the pseudofs just seems to me like something people shouldn't
>>>> 	  normally have to think about.  It's a protocol implementation
>>>> 	  detail, I'd rather hide it.  It'd be to easy to configure it a
>>>> 	  little wrong, I think.
>>>> 
>>>> We can still do this by adding a replicas= option to the / export, but
>>>> we can let rpc.mountd do that internally instead of making the admin add
>>>> it to /etc/exports.
>>>> 
>>>> But then you still need a way for the admin to tell rpc.mountd to cook
>>>> up the replicas= option.....  I'm not sure what that should look like.
>> 
>> Idea 1: extra syntax in /etc/exports
> 
> It's not really export-specific information.  I wonder if it'd be better
> to pass it on the rpc.nfsd commandline?
> 
> 	rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2"
> 
> (and then that can be configured in /etc/sysconfig/nfs or whatever)?
> 
>>>> Maybe some extra syntax in /etc/exports, but what do they need to give
>>>> us--just one list of IP addresses?  Chuck, any ideas?
>> 
>> Idea 2: xattr attached to “/"
>> 
>>> 
>>> How about using the same approach used for junctions:
>>> put the list in an xattr attached to / ? mountd can
>>> extract that when the kernel asks for help satisfying
>>> a GETATTR(fs_locations) on V4ROOT.
> 
> I don't think that works.  "/" isn't a good place to put configuration.
> It could be read-only, among other things.
> 
>> Idea 3: new /etc/ config file
>>> 
>>> Or it could be put in a separate config file in /etc.
>>> You might want to specify more than just the i/f list
>>> here; for instance, the security policy for the
>>> pseudofs, or a constant fsid UUID, among other things.
>> 
>> 
>> API to update the i/f list.  This is not about where to hold fs_locations config info, but rather how to insert the (changed) info into the running system.
>> 
>>> 
>>> Also, I suggested to Andy earlier:
>>> 
>>>> I find myself leaning towards mechanisms that are easy
>>>> both for admins and for programs (ie, an API). Perhaps
>>>> one day you might want to add a command that updates the
>>>> i/f list from the scripts in /etc/sysconfig/network-scripts,
>>>> for instance.
>>>> 
>>>> As part of an ifup:
>>>> 
>>>> nfspfs add <addr>
>>>> 
>>>> and ifdown:
>>>> 
>>>> nfspfs remove <addr>
>>>> 
>>>> I wrote some Python code to manipulate entries in
>>>> /etc/exports, now found in fedfs-utils. It's icky.
>>> 
>>> I think we should move away from "edit this file
>>> and save it, then restart rpc.xyzpdq". Build some
>>> command line interfaces for this.
> 
> I'm OK with that.
> 
> (Note do have that for information in /etc/exports--we have exportfs.
> Is there a reason that didn't work for fedfs-utils?)

To make changes that can survive a server reboot,
you have to update /etc/exports.


> --b.
> 
>>> 
>>> And as you have suggested many times: separate
>>> policy from mechanism. /etc/exports is the
>>> mechanism.
>>> 
>>> --
>>> Chuck Lever
>> 
>> Bruce - do you have a preference between #1 and #2 or #3 (or another idea?)
>> 
>> Thanks
>> 
>> —>Andy
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Chuck Lever

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[Andy Adamson]

On Wed, May 25, 2016 at 2:55 PM, Chuck Lever <chuck.lever@oracle.com> wrote:
>
>> On May 25, 2016, at 2:48 PM, bfields@fieldses.org wrote:
>>
>> On Wed, May 25, 2016 at 05:29:35PM +0000, Adamson, Andy wrote:
>>> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs.
>>>
>>> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list.
>>>
>>> Some background:
>>>
>>> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10):
>>>
>>> The pseudo-fs GETATTR(fs_locations) probe session trunking
>>> was tested against a Linux server with a pseudo-fs
>>> export stanza (e.g. a stanza with the fsid=0 or fsid=root
>>> export option) and a replicas= export option
>>> (replicas=<path1>@<server1>:<path2>@<server2>..)
>>> Note that this configuration is for testing only. A future
>>> patchset will add the replicas= configuration to the
>>> NFSEXP_V4ROOT nfsd and mountd processing.
>>>
>>>
>>> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline.
>>>
>>>
>>>> Begin forwarded message:
>>>>
>>>> From: Chuck Lever <chuck.lever@oracle.com>
>>>> Subject: Re: Configuring fs_locations on Linux upstream server
>>>> Date: May 6, 2016 at 4:31:00 PM EDT
>>>> To: "J. Bruce Fields" <bfields@fieldses.org>
>>>> Cc: "Adamson, Andy" <William.Adamson@netapp.com>
>>>>
>>>>
>>>>> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>>>>
>>>>> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote:
>>>>>> Seems like when a server does not return a list, that is
>>>>>> information the client can use: basically, there is no
>>>>>> ability to do any session trunking. It has to be set up
>>>>>> explicitly; is that a bad thing, operationally?
>>>>>
>>>>> I like the idea of it being opt in on the server.
>>>>>
>>>>> Suppose the server transparently starts advertising all available
>>>>> addresses for session trunking.  It's not hard to imagine cases where
>>>>> that would go wrong.  E.g., maybe the server has the odd wireless or
>>>>> 100Mb or other interface that happens to work but that's slow.  Then
>>>>> somebody upgrades their server and performance goes down and it may take
>>>>> them a while to figure out why.  Whereas if they'd had to opt in they'd
>>>>> probably have avoided advertising an inappropriate interface.  Or at
>>>>> least they'd have a better chance of figuring out that turning on
>>>>> trunking was what caused the problem.
>>>>>
>>>>> I'd rather not force people to export "/" explicitly, though.  It's fine
>>>>> for testing, but:
>>>>>
>>>>>    - I don't think we give a way to do an explicit V4ROOT export,
>>>>>      so they'd be exposing their entire root partition.  We could
>>>>>      fix that, but
>>>>>    - the pseudofs just seems to me like something people shouldn't
>>>>>      normally have to think about.  It's a protocol implementation
>>>>>      detail, I'd rather hide it.  It'd be to easy to configure it a
>>>>>      little wrong, I think.
>>>>>
>>>>> We can still do this by adding a replicas= option to the / export, but
>>>>> we can let rpc.mountd do that internally instead of making the admin add
>>>>> it to /etc/exports.
>>>>>
>>>>> But then you still need a way for the admin to tell rpc.mountd to cook
>>>>> up the replicas= option.....  I'm not sure what that should look like.
>>>
>>> Idea 1: extra syntax in /etc/exports
>>
>> It's not really export-specific information.  I wonder if it'd be better
>> to pass it on the rpc.nfsd commandline?
>>
>>       rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2"
>>
>> (and then that can be configured in /etc/sysconfig/nfs or whatever)?

Is this (the rpc.nfsd command line and /etc/sysconfig/nfs entry) the
preferred way?
Is /etc/sysconfig/nfs read upon reboot?

-->Andy



>>
>>>>> Maybe some extra syntax in /etc/exports, but what do they need to give
>>>>> us--just one list of IP addresses?  Chuck, any ideas?
>>>
>>> Idea 2: xattr attached to “/"
>>>
>>>>
>>>> How about using the same approach used for junctions:
>>>> put the list in an xattr attached to / ? mountd can
>>>> extract that when the kernel asks for help satisfying
>>>> a GETATTR(fs_locations) on V4ROOT.
>>
>> I don't think that works.  "/" isn't a good place to put configuration.
>> It could be read-only, among other things.
>>
>>> Idea 3: new /etc/ config file
>>>>
>>>> Or it could be put in a separate config file in /etc.
>>>> You might want to specify more than just the i/f list
>>>> here; for instance, the security policy for the
>>>> pseudofs, or a constant fsid UUID, among other things.
>>>
>>>
>>> API to update the i/f list.  This is not about where to hold fs_locations config info, but rather how to insert the (changed) info into the running system.
>>>
>>>>
>>>> Also, I suggested to Andy earlier:
>>>>
>>>>> I find myself leaning towards mechanisms that are easy
>>>>> both for admins and for programs (ie, an API). Perhaps
>>>>> one day you might want to add a command that updates the
>>>>> i/f list from the scripts in /etc/sysconfig/network-scripts,
>>>>> for instance.
>>>>>
>>>>> As part of an ifup:
>>>>>
>>>>> nfspfs add <addr>
>>>>>
>>>>> and ifdown:
>>>>>
>>>>> nfspfs remove <addr>
>>>>>
>>>>> I wrote some Python code to manipulate entries in
>>>>> /etc/exports, now found in fedfs-utils. It's icky.
>>>>
>>>> I think we should move away from "edit this file
>>>> and save it, then restart rpc.xyzpdq". Build some
>>>> command line interfaces for this.
>>
>> I'm OK with that.
>>
>> (Note do have that for information in /etc/exports--we have exportfs.
>> Is there a reason that didn't work for fedfs-utils?)
>
> To make changes that can survive a server reboot,
> you have to update /etc/exports.
>
>
>> --b.
>>
>>>>
>>>> And as you have suggested many times: separate
>>>> policy from mechanism. /etc/exports is the
>>>> mechanism.
>>>>
>>>> --
>>>> Chuck Lever
>>>
>>> Bruce - do you have a preference between #1 and #2 or #3 (or another idea?)
>>>
>>> Thanks
>>>
>>> —>Andy
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> Chuck Lever
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[Chuck Lever]

> On May 26, 2016, at 9:54 AM, Andy Adamson <androsadamson@gmail.com> wrote:
> 
> On Wed, May 25, 2016 at 2:55 PM, Chuck Lever <chuck.lever@oracle.com> wrote:
>> 
>>> On May 25, 2016, at 2:48 PM, bfields@fieldses.org wrote:
>>> 
>>> On Wed, May 25, 2016 at 05:29:35PM +0000, Adamson, Andy wrote:
>>>> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs.
>>>> 
>>>> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list.
>>>> 
>>>> Some background:
>>>> 
>>>> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10):
>>>> 
>>>> The pseudo-fs GETATTR(fs_locations) probe session trunking
>>>> was tested against a Linux server with a pseudo-fs
>>>> export stanza (e.g. a stanza with the fsid=0 or fsid=root
>>>> export option) and a replicas= export option
>>>> (replicas=<path1>@<server1>:<path2>@<server2>..)
>>>> Note that this configuration is for testing only. A future
>>>> patchset will add the replicas= configuration to the
>>>> NFSEXP_V4ROOT nfsd and mountd processing.
>>>> 
>>>> 
>>>> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline.
>>>> 
>>>> 
>>>>> Begin forwarded message:
>>>>> 
>>>>> From: Chuck Lever <chuck.lever@oracle.com>
>>>>> Subject: Re: Configuring fs_locations on Linux upstream server
>>>>> Date: May 6, 2016 at 4:31:00 PM EDT
>>>>> To: "J. Bruce Fields" <bfields@fieldses.org>
>>>>> Cc: "Adamson, Andy" <William.Adamson@netapp.com>
>>>>> 
>>>>> 
>>>>>> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>>>>> 
>>>>>> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote:
>>>>>>> Seems like when a server does not return a list, that is
>>>>>>> information the client can use: basically, there is no
>>>>>>> ability to do any session trunking. It has to be set up
>>>>>>> explicitly; is that a bad thing, operationally?
>>>>>> 
>>>>>> I like the idea of it being opt in on the server.
>>>>>> 
>>>>>> Suppose the server transparently starts advertising all available
>>>>>> addresses for session trunking.  It's not hard to imagine cases where
>>>>>> that would go wrong.  E.g., maybe the server has the odd wireless or
>>>>>> 100Mb or other interface that happens to work but that's slow.  Then
>>>>>> somebody upgrades their server and performance goes down and it may take
>>>>>> them a while to figure out why.  Whereas if they'd had to opt in they'd
>>>>>> probably have avoided advertising an inappropriate interface.  Or at
>>>>>> least they'd have a better chance of figuring out that turning on
>>>>>> trunking was what caused the problem.
>>>>>> 
>>>>>> I'd rather not force people to export "/" explicitly, though.  It's fine
>>>>>> for testing, but:
>>>>>> 
>>>>>>   - I don't think we give a way to do an explicit V4ROOT export,
>>>>>>     so they'd be exposing their entire root partition.  We could
>>>>>>     fix that, but
>>>>>>   - the pseudofs just seems to me like something people shouldn't
>>>>>>     normally have to think about.  It's a protocol implementation
>>>>>>     detail, I'd rather hide it.  It'd be to easy to configure it a
>>>>>>     little wrong, I think.
>>>>>> 
>>>>>> We can still do this by adding a replicas= option to the / export, but
>>>>>> we can let rpc.mountd do that internally instead of making the admin add
>>>>>> it to /etc/exports.
>>>>>> 
>>>>>> But then you still need a way for the admin to tell rpc.mountd to cook
>>>>>> up the replicas= option.....  I'm not sure what that should look like.
>>>> 
>>>> Idea 1: extra syntax in /etc/exports
>>> 
>>> It's not really export-specific information.  I wonder if it'd be better
>>> to pass it on the rpc.nfsd commandline?
>>> 
>>>      rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2"
>>> 
>>> (and then that can be configured in /etc/sysconfig/nfs or whatever)?
> 
> Is this (the rpc.nfsd command line and /etc/sysconfig/nfs entry) the
> preferred way?

I don't prefer it.

See below: I think we want something that is more
convenient to update automatically.


> Is /etc/sysconfig/nfs read upon reboot?

It's read by all the start-up scripts related to NFS.


> -->Andy
> 
> 
> 
>>> 
>>>>>> Maybe some extra syntax in /etc/exports, but what do they need to give
>>>>>> us--just one list of IP addresses?  Chuck, any ideas?
>>>> 
>>>> Idea 2: xattr attached to “/"
>>>> 
>>>>> 
>>>>> How about using the same approach used for junctions:
>>>>> put the list in an xattr attached to / ? mountd can
>>>>> extract that when the kernel asks for help satisfying
>>>>> a GETATTR(fs_locations) on V4ROOT.
>>> 
>>> I don't think that works.  "/" isn't a good place to put configuration.
>>> It could be read-only, among other things.
>>> 
>>>> Idea 3: new /etc/ config file
>>>>> 
>>>>> Or it could be put in a separate config file in /etc.
>>>>> You might want to specify more than just the i/f list
>>>>> here; for instance, the security policy for the
>>>>> pseudofs, or a constant fsid UUID, among other things.
>>>> 
>>>> 
>>>> API to update the i/f list.  This is not about where to hold fs_locations config info, but rather how to insert the (changed) info into the running system.
>>>> 
>>>>> 
>>>>> Also, I suggested to Andy earlier:
>>>>> 
>>>>>> I find myself leaning towards mechanisms that are easy
>>>>>> both for admins and for programs (ie, an API). Perhaps
>>>>>> one day you might want to add a command that updates the
>>>>>> i/f list from the scripts in /etc/sysconfig/network-scripts,
>>>>>> for instance.
>>>>>> 
>>>>>> As part of an ifup:
>>>>>> 
>>>>>> nfspfs add <addr>
>>>>>> 
>>>>>> and ifdown:
>>>>>> 
>>>>>> nfspfs remove <addr>
>>>>>> 
>>>>>> I wrote some Python code to manipulate entries in
>>>>>> /etc/exports, now found in fedfs-utils. It's icky.
>>>>> 
>>>>> I think we should move away from "edit this file
>>>>> and save it, then restart rpc.xyzpdq". Build some
>>>>> command line interfaces for this.
>>> 
>>> I'm OK with that.
>>> 
>>> (Note do have that for information in /etc/exports--we have exportfs.
>>> Is there a reason that didn't work for fedfs-utils?)
>> 
>> To make changes that can survive a server reboot,
>> you have to update /etc/exports.
>> 
>> 
>>> --b.
>>> 
>>>>> 
>>>>> And as you have suggested many times: separate
>>>>> policy from mechanism. /etc/exports is the
>>>>> mechanism.
>>>>> 
>>>>> --
>>>>> Chuck Lever
>>>> 
>>>> Bruce - do you have a preference between #1 and #2 or #3 (or another idea?)
>>>> 
>>>> Thanks
>>>> 
>>>> —>Andy
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
>> --
>> Chuck Lever
>> 
>> 
>> 
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Chuck Lever

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[Adamson, Andy]

DQo+IE9uIE1heSAyNiwgMjAxNiwgYXQgMTA6MjUgQU0sIENodWNrIExldmVyIDxjaHVjay5sZXZl
ckBvcmFjbGUuY29tPiB3cm90ZToNCj4gDQo+IA0KPj4gT24gTWF5IDI2LCAyMDE2LCBhdCA5OjU0
IEFNLCBBbmR5IEFkYW1zb24gPGFuZHJvc2FkYW1zb25AZ21haWwuY29tPiB3cm90ZToNCj4+IA0K
Pj4gT24gV2VkLCBNYXkgMjUsIDIwMTYgYXQgMjo1NSBQTSwgQ2h1Y2sgTGV2ZXIgPGNodWNrLmxl
dmVyQG9yYWNsZS5jb20+IHdyb3RlOg0KPj4+IA0KPj4+PiBPbiBNYXkgMjUsIDIwMTYsIGF0IDI6
NDggUE0sIGJmaWVsZHNAZmllbGRzZXMub3JnIHdyb3RlOg0KPj4+PiANCj4+Pj4gT24gV2VkLCBN
YXkgMjUsIDIwMTYgYXQgMDU6Mjk6MzVQTSArMDAwMCwgQWRhbXNvbiwgQW5keSB3cm90ZToNCj4+
Pj4+IEFubmEgU2NodW1ha2VyIHdobyByZXZpZXdlZCBteSBjbGllbnQgc2lkZSBzZXNzaW9uIHRy
dW5raW5nIHBhdGNoc2V0LCB3YW50cyBhIGZ1bGwgZmVhdHVyZWQgdmVyc2lvbiBvZiBib3RoIHRo
ZSBjbGllbnQgYW5kIHRoZSBzZXJ2ZXIgc2Vzc2lvbiB0cnVua2luZyBwaWVjZXMgYmVmb3JlIGFj
Y2VwdGluZyB0aGUgc2Vzc2lvbiB0cnVua2luZyBmZWF0dXJlIHVwc3RyZWFtLiBUbyB0aGF0IGVu
ZCwgSSB3YW50IHRvIGltcGxlbWVudCB0aGUgc2VydmVyIG1vdW50ZCBWNFJPT1QgcHJvY2Vzc2lu
ZyBvZiBhbiBmc19sb2NhdGlvbnMgY29uZmlndXJhdGlvbiB0byBzYXRpc2Z5IGFuIGZzX2xvY2F0
aW9ucyByZXF1ZXN0IG9uIHRoZSBwc2V1ZG8gZnMuDQo+Pj4+PiANCj4+Pj4+IFRoZSBmb3J3YXJk
ZWQgbWVzc2FnZSBpcyBmcm9tIGFuIGVtYWlsIHN0cmVhbSBiZXR3ZWVuIEJydWNlLCBDaHVjayBh
bmQgSSBjb25jZXJuaW5nIHRoZSBzZXJ2ZXIgcHNldWZvIGZzIGZzX2xvY2F0aW9ucyBjb25maWd1
cmF0aW9uIHRoYXQgSeKAmW0gbm93IHNoYXJpbmcgd2l0aCB0aGUgbGlzdC4NCj4+Pj4+IA0KPj4+
Pj4gU29tZSBiYWNrZ3JvdW5kOg0KPj4+Pj4gDQo+Pj4+PiBUaGUgcmVjZW50ICJORlNWNC4xLDIg
c2Vzc2lvbiB0cnVua2luZ+KAnSBWZXJzaW9uLTUgcGF0Y2ggc2V0IHNlbnQgdG8gdGhlIGxpc3Qg
bm90ZXMgKGluIHBhdGNoIDAwLzEwKToNCj4+Pj4+IA0KPj4+Pj4gVGhlIHBzZXVkby1mcyBHRVRB
VFRSKGZzX2xvY2F0aW9ucykgcHJvYmUgc2Vzc2lvbiB0cnVua2luZw0KPj4+Pj4gd2FzIHRlc3Rl
ZCBhZ2FpbnN0IGEgTGludXggc2VydmVyIHdpdGggYSBwc2V1ZG8tZnMNCj4+Pj4+IGV4cG9ydCBz
dGFuemEgKGUuZy4gYSBzdGFuemEgd2l0aCB0aGUgZnNpZD0wIG9yIGZzaWQ9cm9vdA0KPj4+Pj4g
ZXhwb3J0IG9wdGlvbikgYW5kIGEgcmVwbGljYXM9IGV4cG9ydCBvcHRpb24NCj4+Pj4+IChyZXBs
aWNhcz08cGF0aDE+QDxzZXJ2ZXIxPjo8cGF0aDI+QDxzZXJ2ZXIyPi4uKQ0KPj4+Pj4gTm90ZSB0
aGF0IHRoaXMgY29uZmlndXJhdGlvbiBpcyBmb3IgdGVzdGluZyBvbmx5LiBBIGZ1dHVyZQ0KPj4+
Pj4gcGF0Y2hzZXQgd2lsbCBhZGQgdGhlIHJlcGxpY2FzPSBjb25maWd1cmF0aW9uIHRvIHRoZQ0K
Pj4+Pj4gTkZTRVhQX1Y0Uk9PVCBuZnNkIGFuZCBtb3VudGQgcHJvY2Vzc2luZy4NCj4+Pj4+IA0K
Pj4+Pj4gDQo+Pj4+PiBUaGVyZSBhcmUgc2V2ZXJhbCBpZGVhcyBvbiBob3cgdG8gYWNjb21wbGlz
aCBtb3VudGQvVjRST09UIGZzX2xvY2F0aW9ucyBjb25maWd1cmF0aW9uIGluIHRoZSBmb3J3YXJk
ZWQgbWVzc2FnZS4gU2VlIGlubGluZS4NCj4+Pj4+IA0KPj4+Pj4gDQo+Pj4+Pj4gQmVnaW4gZm9y
d2FyZGVkIG1lc3NhZ2U6DQo+Pj4+Pj4gDQo+Pj4+Pj4gRnJvbTogQ2h1Y2sgTGV2ZXIgPGNodWNr
LmxldmVyQG9yYWNsZS5jb20+DQo+Pj4+Pj4gU3ViamVjdDogUmU6IENvbmZpZ3VyaW5nIGZzX2xv
Y2F0aW9ucyBvbiBMaW51eCB1cHN0cmVhbSBzZXJ2ZXINCj4+Pj4+PiBEYXRlOiBNYXkgNiwgMjAx
NiBhdCA0OjMxOjAwIFBNIEVEVA0KPj4+Pj4+IFRvOiAiSi4gQnJ1Y2UgRmllbGRzIiA8YmZpZWxk
c0BmaWVsZHNlcy5vcmc+DQo+Pj4+Pj4gQ2M6ICJBZGFtc29uLCBBbmR5IiA8V2lsbGlhbS5BZGFt
c29uQG5ldGFwcC5jb20+DQo+Pj4+Pj4gDQo+Pj4+Pj4gDQo+Pj4+Pj4+IE9uIE1heSA2LCAyMDE2
LCBhdCA0OjE2IFBNLCBKLiBCcnVjZSBGaWVsZHMgPGJmaWVsZHNAZmllbGRzZXMub3JnPiB3cm90
ZToNCj4+Pj4+Pj4gDQo+Pj4+Pj4+IE9uIEZyaSwgTWF5IDA2LCAyMDE2IGF0IDAyOjIwOjEyUE0g
LTA0MDAsIENodWNrIExldmVyIHdyb3RlOg0KPj4+Pj4+Pj4gU2VlbXMgbGlrZSB3aGVuIGEgc2Vy
dmVyIGRvZXMgbm90IHJldHVybiBhIGxpc3QsIHRoYXQgaXMNCj4+Pj4+Pj4+IGluZm9ybWF0aW9u
IHRoZSBjbGllbnQgY2FuIHVzZTogYmFzaWNhbGx5LCB0aGVyZSBpcyBubw0KPj4+Pj4+Pj4gYWJp
bGl0eSB0byBkbyBhbnkgc2Vzc2lvbiB0cnVua2luZy4gSXQgaGFzIHRvIGJlIHNldCB1cA0KPj4+
Pj4+Pj4gZXhwbGljaXRseTsgaXMgdGhhdCBhIGJhZCB0aGluZywgb3BlcmF0aW9uYWxseT8NCj4+
Pj4+Pj4gDQo+Pj4+Pj4+IEkgbGlrZSB0aGUgaWRlYSBvZiBpdCBiZWluZyBvcHQgaW4gb24gdGhl
IHNlcnZlci4NCj4+Pj4+Pj4gDQo+Pj4+Pj4+IFN1cHBvc2UgdGhlIHNlcnZlciB0cmFuc3BhcmVu
dGx5IHN0YXJ0cyBhZHZlcnRpc2luZyBhbGwgYXZhaWxhYmxlDQo+Pj4+Pj4+IGFkZHJlc3NlcyBm
b3Igc2Vzc2lvbiB0cnVua2luZy4gIEl0J3Mgbm90IGhhcmQgdG8gaW1hZ2luZSBjYXNlcyB3aGVy
ZQ0KPj4+Pj4+PiB0aGF0IHdvdWxkIGdvIHdyb25nLiAgRS5nLiwgbWF5YmUgdGhlIHNlcnZlciBo
YXMgdGhlIG9kZCB3aXJlbGVzcyBvcg0KPj4+Pj4+PiAxMDBNYiBvciBvdGhlciBpbnRlcmZhY2Ug
dGhhdCBoYXBwZW5zIHRvIHdvcmsgYnV0IHRoYXQncyBzbG93LiAgVGhlbg0KPj4+Pj4+PiBzb21l
Ym9keSB1cGdyYWRlcyB0aGVpciBzZXJ2ZXIgYW5kIHBlcmZvcm1hbmNlIGdvZXMgZG93biBhbmQg
aXQgbWF5IHRha2UNCj4+Pj4+Pj4gdGhlbSBhIHdoaWxlIHRvIGZpZ3VyZSBvdXQgd2h5LiAgV2hl
cmVhcyBpZiB0aGV5J2QgaGFkIHRvIG9wdCBpbiB0aGV5J2QNCj4+Pj4+Pj4gcHJvYmFibHkgaGF2
ZSBhdm9pZGVkIGFkdmVydGlzaW5nIGFuIGluYXBwcm9wcmlhdGUgaW50ZXJmYWNlLiAgT3IgYXQN
Cj4+Pj4+Pj4gbGVhc3QgdGhleSdkIGhhdmUgYSBiZXR0ZXIgY2hhbmNlIG9mIGZpZ3VyaW5nIG91
dCB0aGF0IHR1cm5pbmcgb24NCj4+Pj4+Pj4gdHJ1bmtpbmcgd2FzIHdoYXQgY2F1c2VkIHRoZSBw
cm9ibGVtLg0KPj4+Pj4+PiANCj4+Pj4+Pj4gSSdkIHJhdGhlciBub3QgZm9yY2UgcGVvcGxlIHRv
IGV4cG9ydCAiLyIgZXhwbGljaXRseSwgdGhvdWdoLiAgSXQncyBmaW5lDQo+Pj4+Pj4+IGZvciB0
ZXN0aW5nLCBidXQ6DQo+Pj4+Pj4+IA0KPj4+Pj4+PiAgLSBJIGRvbid0IHRoaW5rIHdlIGdpdmUg
YSB3YXkgdG8gZG8gYW4gZXhwbGljaXQgVjRST09UIGV4cG9ydCwNCj4+Pj4+Pj4gICAgc28gdGhl
eSdkIGJlIGV4cG9zaW5nIHRoZWlyIGVudGlyZSByb290IHBhcnRpdGlvbi4gIFdlIGNvdWxkDQo+
Pj4+Pj4+ICAgIGZpeCB0aGF0LCBidXQNCj4+Pj4+Pj4gIC0gdGhlIHBzZXVkb2ZzIGp1c3Qgc2Vl
bXMgdG8gbWUgbGlrZSBzb21ldGhpbmcgcGVvcGxlIHNob3VsZG4ndA0KPj4+Pj4+PiAgICBub3Jt
YWxseSBoYXZlIHRvIHRoaW5rIGFib3V0LiAgSXQncyBhIHByb3RvY29sIGltcGxlbWVudGF0aW9u
DQo+Pj4+Pj4+ICAgIGRldGFpbCwgSSdkIHJhdGhlciBoaWRlIGl0LiAgSXQnZCBiZSB0byBlYXN5
IHRvIGNvbmZpZ3VyZSBpdCBhDQo+Pj4+Pj4+ICAgIGxpdHRsZSB3cm9uZywgSSB0aGluay4NCj4+
Pj4+Pj4gDQo+Pj4+Pj4+IFdlIGNhbiBzdGlsbCBkbyB0aGlzIGJ5IGFkZGluZyBhIHJlcGxpY2Fz
PSBvcHRpb24gdG8gdGhlIC8gZXhwb3J0LCBidXQNCj4+Pj4+Pj4gd2UgY2FuIGxldCBycGMubW91
bnRkIGRvIHRoYXQgaW50ZXJuYWxseSBpbnN0ZWFkIG9mIG1ha2luZyB0aGUgYWRtaW4gYWRkDQo+
Pj4+Pj4+IGl0IHRvIC9ldGMvZXhwb3J0cy4NCj4+Pj4+Pj4gDQo+Pj4+Pj4+IEJ1dCB0aGVuIHlv
dSBzdGlsbCBuZWVkIGEgd2F5IGZvciB0aGUgYWRtaW4gdG8gdGVsbCBycGMubW91bnRkIHRvIGNv
b2sNCj4+Pj4+Pj4gdXAgdGhlIHJlcGxpY2FzPSBvcHRpb24uLi4uLiAgSSdtIG5vdCBzdXJlIHdo
YXQgdGhhdCBzaG91bGQgbG9vayBsaWtlLg0KPj4+Pj4gDQo+Pj4+PiBJZGVhIDE6IGV4dHJhIHN5
bnRheCBpbiAvZXRjL2V4cG9ydHMNCj4+Pj4gDQo+Pj4+IEl0J3Mgbm90IHJlYWxseSBleHBvcnQt
c3BlY2lmaWMgaW5mb3JtYXRpb24uICBJIHdvbmRlciBpZiBpdCdkIGJlIGJldHRlcg0KPj4+PiB0
byBwYXNzIGl0IG9uIHRoZSBycGMubmZzZCBjb21tYW5kbGluZT8NCj4+Pj4gDQo+Pj4+ICAgICBy
cGMubmZzZCAtLW11bHRpcGF0aC1zZXQ9IjE5Mi4xNjguMC4xLDE5Mi4xNjguMC4yIg0KPj4+PiAN
Cj4+Pj4gKGFuZCB0aGVuIHRoYXQgY2FuIGJlIGNvbmZpZ3VyZWQgaW4gL2V0Yy9zeXNjb25maWcv
bmZzIG9yIHdoYXRldmVyKT8NCj4+IA0KPj4gSXMgdGhpcyAodGhlIHJwYy5uZnNkIGNvbW1hbmQg
bGluZSBhbmQgL2V0Yy9zeXNjb25maWcvbmZzIGVudHJ5KSB0aGUNCj4+IHByZWZlcnJlZCB3YXk/
DQo+IA0KPiBJIGRvbid0IHByZWZlciBpdC4NCj4gDQo+IFNlZSBiZWxvdzogSSB0aGluayB3ZSB3
YW50IHNvbWV0aGluZyB0aGF0IGlzIG1vcmUNCj4gY29udmVuaWVudCB0byB1cGRhdGUgYXV0b21h
dGljYWxseS4NCg0KRmluZSwgYnV0IEnigJltIGhhdmluZyBkaWZmaWN1bHR5IGluIHVuZGVyc3Rh
bmRpbmcgdGhlIGRlc2lnbiB5b3UgYXJlIHN1Z2dlc3RpbmcgdG8gZnVsZmlsbCB0aGUgdXBkYXRl
IGF1dG9tYXRpY2FsbHkgcmVxdWlyZW1lbnQuDQpTZWUgaW5saW5lIGJlbG93Lg0KDQo+IA0KPiAN
Cj4+IElzIC9ldGMvc3lzY29uZmlnL25mcyByZWFkIHVwb24gcmVib290Pw0KPiANCj4gSXQncyBy
ZWFkIGJ5IGFsbCB0aGUgc3RhcnQtdXAgc2NyaXB0cyByZWxhdGVkIHRvIE5GUy4NCj4gDQo+IA0K
Pj4gLS0+QW5keQ0KPj4gDQo+PiANCj4+IA0KPj4+PiANCj4+Pj4+Pj4gTWF5YmUgc29tZSBleHRy
YSBzeW50YXggaW4gL2V0Yy9leHBvcnRzLCBidXQgd2hhdCBkbyB0aGV5IG5lZWQgdG8gZ2l2ZQ0K
Pj4+Pj4+PiB1cy0tanVzdCBvbmUgbGlzdCBvZiBJUCBhZGRyZXNzZXM/ICBDaHVjaywgYW55IGlk
ZWFzPw0KPj4+Pj4gDQo+Pj4+PiBJZGVhIDI6IHhhdHRyIGF0dGFjaGVkIHRvIOKAnC8iDQo+Pj4+
PiANCj4+Pj4+PiANCj4+Pj4+PiBIb3cgYWJvdXQgdXNpbmcgdGhlIHNhbWUgYXBwcm9hY2ggdXNl
ZCBmb3IganVuY3Rpb25zOg0KPj4+Pj4+IHB1dCB0aGUgbGlzdCBpbiBhbiB4YXR0ciBhdHRhY2hl
ZCB0byAvID8gbW91bnRkIGNhbg0KPj4+Pj4+IGV4dHJhY3QgdGhhdCB3aGVuIHRoZSBrZXJuZWwg
YXNrcyBmb3IgaGVscCBzYXRpc2Z5aW5nDQo+Pj4+Pj4gYSBHRVRBVFRSKGZzX2xvY2F0aW9ucykg
b24gVjRST09ULg0KPj4+PiANCj4+Pj4gSSBkb24ndCB0aGluayB0aGF0IHdvcmtzLiAgIi8iIGlz
bid0IGEgZ29vZCBwbGFjZSB0byBwdXQgY29uZmlndXJhdGlvbi4NCj4+Pj4gSXQgY291bGQgYmUg
cmVhZC1vbmx5LCBhbW9uZyBvdGhlciB0aGluZ3MuDQo+Pj4+IA0KPj4+Pj4gSWRlYSAzOiBuZXcg
L2V0Yy8gY29uZmlnIGZpbGUNCj4+Pj4+PiANCj4+Pj4+PiBPciBpdCBjb3VsZCBiZSBwdXQgaW4g
YSBzZXBhcmF0ZSBjb25maWcgZmlsZSBpbiAvZXRjLg0KPj4+Pj4+IFlvdSBtaWdodCB3YW50IHRv
IHNwZWNpZnkgbW9yZSB0aGFuIGp1c3QgdGhlIGkvZiBsaXN0DQo+Pj4+Pj4gaGVyZTsgZm9yIGlu
c3RhbmNlLCB0aGUgc2VjdXJpdHkgcG9saWN5IGZvciB0aGUNCj4+Pj4+PiBwc2V1ZG9mcywgb3Ig
YSBjb25zdGFudCBmc2lkIFVVSUQsIGFtb25nIG90aGVyIHRoaW5ncy4NCj4+Pj4+IA0KPj4+Pj4g
DQo+Pj4+PiBBUEkgdG8gdXBkYXRlIHRoZSBpL2YgbGlzdC4gIFRoaXMgaXMgbm90IGFib3V0IHdo
ZXJlIHRvIGhvbGQgZnNfbG9jYXRpb25zIGNvbmZpZyBpbmZvLCBidXQgcmF0aGVyIGhvdyB0byBp
bnNlcnQgdGhlIChjaGFuZ2VkKSBpbmZvIGludG8gdGhlIHJ1bm5pbmcgc3lzdGVtLg0KPj4+Pj4g
DQo+Pj4+Pj4gDQo+Pj4+Pj4gQWxzbywgSSBzdWdnZXN0ZWQgdG8gQW5keSBlYXJsaWVyOg0KPj4+
Pj4+IA0KPj4+Pj4+PiBJIGZpbmQgbXlzZWxmIGxlYW5pbmcgdG93YXJkcyBtZWNoYW5pc21zIHRo
YXQgYXJlIGVhc3kNCj4+Pj4+Pj4gYm90aCBmb3IgYWRtaW5zIGFuZCBmb3IgcHJvZ3JhbXMgKGll
LCBhbiBBUEkpLiBQZXJoYXBzDQo+Pj4+Pj4+IG9uZSBkYXkgeW91IG1pZ2h0IHdhbnQgdG8gYWRk
IGEgY29tbWFuZCB0aGF0IHVwZGF0ZXMgdGhlDQo+Pj4+Pj4+IGkvZiBsaXN0IGZyb20gdGhlIHNj
cmlwdHMgaW4gL2V0Yy9zeXNjb25maWcvbmV0d29yay1zY3JpcHRzLA0KPj4+Pj4+PiBmb3IgaW5z
dGFuY2UuDQo+Pj4+Pj4+IA0KPj4+Pj4+PiBBcyBwYXJ0IG9mIGFuIGlmdXA6DQo+Pj4+Pj4+IA0K
Pj4+Pj4+PiBuZnNwZnMgYWRkIDxhZGRyPg0KPj4+Pj4+PiANCj4+Pj4+Pj4gYW5kIGlmZG93bjoN
Cj4+Pj4+Pj4gDQo+Pj4+Pj4+IG5mc3BmcyByZW1vdmUgPGFkZHI+DQo+Pj4+Pj4+IA0KPj4+Pj4+
PiBJIHdyb3RlIHNvbWUgUHl0aG9uIGNvZGUgdG8gbWFuaXB1bGF0ZSBlbnRyaWVzIGluDQo+Pj4+
Pj4+IC9ldGMvZXhwb3J0cywgbm93IGZvdW5kIGluIGZlZGZzLXV0aWxzLiBJdCdzIGlja3kuDQo+
Pj4+Pj4gDQo+Pj4+Pj4gSSB0aGluayB3ZSBzaG91bGQgbW92ZSBhd2F5IGZyb20gImVkaXQgdGhp
cyBmaWxlDQo+Pj4+Pj4gYW5kIHNhdmUgaXQsIHRoZW4gcmVzdGFydCBycGMueHl6cGRxIi4gQnVp
bGQgc29tZQ0KPj4+Pj4+IGNvbW1hbmQgbGluZSBpbnRlcmZhY2VzIGZvciB0aGlzLg0KPj4+PiAN
Cj4+Pj4gSSdtIE9LIHdpdGggdGhhdC4NCj4+Pj4gDQo+Pj4+IChOb3RlIGRvIGhhdmUgdGhhdCBm
b3IgaW5mb3JtYXRpb24gaW4gL2V0Yy9leHBvcnRzLS13ZSBoYXZlIGV4cG9ydGZzLg0KPj4+PiBJ
cyB0aGVyZSBhIHJlYXNvbiB0aGF0IGRpZG4ndCB3b3JrIGZvciBmZWRmcy11dGlscz8pDQo+Pj4g
DQo+Pj4gVG8gbWFrZSBjaGFuZ2VzIHRoYXQgY2FuIHN1cnZpdmUgYSBzZXJ2ZXIgcmVib290LA0K
Pj4+IHlvdSBoYXZlIHRvIHVwZGF0ZSAvZXRjL2V4cG9ydHMuDQoNCg0KDQpZb3VyIHN1Z2dlc3Rp
b24gdGhlbiBpcyB0byBidWlsZCBhIG5ldyBjb21tYW5kLWxpbmUgaW50ZXJmYWNlIHRvOg0KDQot
IHRlbGwgbW91bnRkIG9mIGEgVjRST09UIG11bHRpcGF0aCBsaXN0Pw0KLSBoYXZlIHNhaWQgbGlz
dCBzdXJ2aXZlIHJlYm9vdHMsIGUuZy4gc3RvcmVkIGluIGEgZmlsZT8NCg0KUGxlYXNlIHBvdmlk
ZSBtb3JlIGRldGFpbCBvbiB5b3VyIHRob3VnaHRzLg0KDQpUaGFua3MNCg0K4oCUPiBBbmR5DQoN
Cj4+PiANCj4+PiANCj4+PiANCj4+Pj4gLS1iLg0KPj4+PiANCj4+Pj4+PiANCj4+Pj4+PiBBbmQg
YXMgeW91IGhhdmUgc3VnZ2VzdGVkIG1hbnkgdGltZXM6IHNlcGFyYXRlDQo+Pj4+Pj4gcG9saWN5
IGZyb20gbWVjaGFuaXNtLiAvZXRjL2V4cG9ydHMgaXMgdGhlDQo+Pj4+Pj4gbWVjaGFuaXNtLg0K
Pj4+Pj4+IA0KPj4+Pj4+IC0tDQo+Pj4+Pj4gQ2h1Y2sgTGV2ZXINCj4+Pj4+IA0KPj4+Pj4gQnJ1
Y2UgLSBkbyB5b3UgaGF2ZSBhIHByZWZlcmVuY2UgYmV0d2VlbiAjMSBhbmQgIzIgb3IgIzMgKG9y
IGFub3RoZXIgaWRlYT8pDQo+Pj4+PiANCj4+Pj4+IFRoYW5rcw0KPj4+Pj4gDQo+Pj4+PiDigJQ+
QW5keQ0KPj4+PiAtLQ0KPj4+PiBUbyB1bnN1YnNjcmliZSBmcm9tIHRoaXMgbGlzdDogc2VuZCB0
aGUgbGluZSAidW5zdWJzY3JpYmUgbGludXgtbmZzIiBpbg0KPj4+PiB0aGUgYm9keSBvZiBhIG1l
c3NhZ2UgdG8gbWFqb3Jkb21vQHZnZXIua2VybmVsLm9yZw0KPj4+PiBNb3JlIG1ham9yZG9tbyBp
bmZvIGF0ICBodHRwOi8vdmdlci5rZXJuZWwub3JnL21ham9yZG9tby1pbmZvLmh0bWwNCj4+PiAN
Cj4+PiAtLQ0KPj4+IENodWNrIExldmVyDQo+Pj4gDQo+Pj4gDQo+Pj4gDQo+Pj4gLS0NCj4+PiBU
byB1bnN1YnNjcmliZSBmcm9tIHRoaXMgbGlzdDogc2VuZCB0aGUgbGluZSAidW5zdWJzY3JpYmUg
bGludXgtbmZzIiBpbg0KPj4+IHRoZSBib2R5IG9mIGEgbWVzc2FnZSB0byBtYWpvcmRvbW9Admdl
ci5rZXJuZWwub3JnDQo+Pj4gTW9yZSBtYWpvcmRvbW8gaW5mbyBhdCAgaHR0cDovL3ZnZXIua2Vy
bmVsLm9yZy9tYWpvcmRvbW8taW5mby5odG1sDQo+PiAtLQ0KPj4gVG8gdW5zdWJzY3JpYmUgZnJv
bSB0aGlzIGxpc3Q6IHNlbmQgdGhlIGxpbmUgInVuc3Vic2NyaWJlIGxpbnV4LW5mcyIgaW4NCj4+
IHRoZSBib2R5IG9mIGEgbWVzc2FnZSB0byBtYWpvcmRvbW9Admdlci5rZXJuZWwub3JnDQo+PiBN
b3JlIG1ham9yZG9tbyBpbmZvIGF0ICBodHRwOi8vdmdlci5rZXJuZWwub3JnL21ham9yZG9tby1p
bmZvLmh0bWwNCj4gDQo+IC0tDQo+IENodWNrIExldmVyDQo+IA0KPiANCj4gDQoNCg==

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[Chuck Lever]

> On May 26, 2016, at 10:44 AM, Adamson, Andy <William.Adamson@netapp.com> wrote:
> 
>> 
>> On May 26, 2016, at 10:25 AM, Chuck Lever <chuck.lever@oracle.com> wrote:
>> 
>> 
>>> On May 26, 2016, at 9:54 AM, Andy Adamson <androsadamson@gmail.com> wrote:
>>> 
>>> On Wed, May 25, 2016 at 2:55 PM, Chuck Lever <chuck.lever@oracle.com> wrote:
>>>> 
>>>>> On May 25, 2016, at 2:48 PM, bfields@fieldses.org wrote:
>>>>> 
>>>>> On Wed, May 25, 2016 at 05:29:35PM +0000, Adamson, Andy wrote:
>>>>>> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs.
>>>>>> 
>>>>>> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list.
>>>>>> 
>>>>>> Some background:
>>>>>> 
>>>>>> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10):
>>>>>> 
>>>>>> The pseudo-fs GETATTR(fs_locations) probe session trunking
>>>>>> was tested against a Linux server with a pseudo-fs
>>>>>> export stanza (e.g. a stanza with the fsid=0 or fsid=root
>>>>>> export option) and a replicas= export option
>>>>>> (replicas=<path1>@<server1>:<path2>@<server2>..)
>>>>>> Note that this configuration is for testing only. A future
>>>>>> patchset will add the replicas= configuration to the
>>>>>> NFSEXP_V4ROOT nfsd and mountd processing.
>>>>>> 
>>>>>> 
>>>>>> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline.
>>>>>> 
>>>>>> 
>>>>>>> Begin forwarded message:
>>>>>>> 
>>>>>>> From: Chuck Lever <chuck.lever@oracle.com>
>>>>>>> Subject: Re: Configuring fs_locations on Linux upstream server
>>>>>>> Date: May 6, 2016 at 4:31:00 PM EDT
>>>>>>> To: "J. Bruce Fields" <bfields@fieldses.org>
>>>>>>> Cc: "Adamson, Andy" <William.Adamson@netapp.com>
>>>>>>> 
>>>>>>> 
>>>>>>>> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>>>>>>> 
>>>>>>>> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote:
>>>>>>>>> Seems like when a server does not return a list, that is
>>>>>>>>> information the client can use: basically, there is no
>>>>>>>>> ability to do any session trunking. It has to be set up
>>>>>>>>> explicitly; is that a bad thing, operationally?
>>>>>>>> 
>>>>>>>> I like the idea of it being opt in on the server.
>>>>>>>> 
>>>>>>>> Suppose the server transparently starts advertising all available
>>>>>>>> addresses for session trunking.  It's not hard to imagine cases where
>>>>>>>> that would go wrong.  E.g., maybe the server has the odd wireless or
>>>>>>>> 100Mb or other interface that happens to work but that's slow.  Then
>>>>>>>> somebody upgrades their server and performance goes down and it may take
>>>>>>>> them a while to figure out why.  Whereas if they'd had to opt in they'd
>>>>>>>> probably have avoided advertising an inappropriate interface.  Or at
>>>>>>>> least they'd have a better chance of figuring out that turning on
>>>>>>>> trunking was what caused the problem.
>>>>>>>> 
>>>>>>>> I'd rather not force people to export "/" explicitly, though.  It's fine
>>>>>>>> for testing, but:
>>>>>>>> 
>>>>>>>> - I don't think we give a way to do an explicit V4ROOT export,
>>>>>>>>   so they'd be exposing their entire root partition.  We could
>>>>>>>>   fix that, but
>>>>>>>> - the pseudofs just seems to me like something people shouldn't
>>>>>>>>   normally have to think about.  It's a protocol implementation
>>>>>>>>   detail, I'd rather hide it.  It'd be to easy to configure it a
>>>>>>>>   little wrong, I think.
>>>>>>>> 
>>>>>>>> We can still do this by adding a replicas= option to the / export, but
>>>>>>>> we can let rpc.mountd do that internally instead of making the admin add
>>>>>>>> it to /etc/exports.
>>>>>>>> 
>>>>>>>> But then you still need a way for the admin to tell rpc.mountd to cook
>>>>>>>> up the replicas= option.....  I'm not sure what that should look like.
>>>>>> 
>>>>>> Idea 1: extra syntax in /etc/exports
>>>>> 
>>>>> It's not really export-specific information.  I wonder if it'd be better
>>>>> to pass it on the rpc.nfsd commandline?
>>>>> 
>>>>>    rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2"
>>>>> 
>>>>> (and then that can be configured in /etc/sysconfig/nfs or whatever)?
>>> 
>>> Is this (the rpc.nfsd command line and /etc/sysconfig/nfs entry) the
>>> preferred way?
>> 
>> I don't prefer it.
>> 
>> See below: I think we want something that is more
>> convenient to update automatically.
> 
> Fine, but I’m having difficulty in understanding the design you are suggesting to fulfill the update automatically requirement.
> See inline below.
> 
>> 
>> 
>>> Is /etc/sysconfig/nfs read upon reboot?
>> 
>> It's read by all the start-up scripts related to NFS.
>> 
>> 
>>> -->Andy
>>> 
>>> 
>>> 
>>>>> 
>>>>>>>> Maybe some extra syntax in /etc/exports, but what do they need to give
>>>>>>>> us--just one list of IP addresses?  Chuck, any ideas?
>>>>>> 
>>>>>> Idea 2: xattr attached to “/"
>>>>>> 
>>>>>>> 
>>>>>>> How about using the same approach used for junctions:
>>>>>>> put the list in an xattr attached to / ? mountd can
>>>>>>> extract that when the kernel asks for help satisfying
>>>>>>> a GETATTR(fs_locations) on V4ROOT.
>>>>> 
>>>>> I don't think that works.  "/" isn't a good place to put configuration.
>>>>> It could be read-only, among other things.
>>>>> 
>>>>>> Idea 3: new /etc/ config file
>>>>>>> 
>>>>>>> Or it could be put in a separate config file in /etc.
>>>>>>> You might want to specify more than just the i/f list
>>>>>>> here; for instance, the security policy for the
>>>>>>> pseudofs, or a constant fsid UUID, among other things.
>>>>>> 
>>>>>> 
>>>>>> API to update the i/f list.  This is not about where to hold fs_locations config info, but rather how to insert the (changed) info into the running system.
>>>>>> 
>>>>>>> 
>>>>>>> Also, I suggested to Andy earlier:
>>>>>>> 
>>>>>>>> I find myself leaning towards mechanisms that are easy
>>>>>>>> both for admins and for programs (ie, an API). Perhaps
>>>>>>>> one day you might want to add a command that updates the
>>>>>>>> i/f list from the scripts in /etc/sysconfig/network-scripts,
>>>>>>>> for instance.

>>>>>>>> As part of an ifup:
>>>>>>>> 
>>>>>>>> nfspfs add <addr>
>>>>>>>> 
>>>>>>>> and ifdown:
>>>>>>>> 
>>>>>>>> nfspfs remove <addr>
>>>>>>>> 
>>>>>>>> I wrote some Python code to manipulate entries in
>>>>>>>> /etc/exports, now found in fedfs-utils. It's icky.
>>>>>>> 
>>>>>>> I think we should move away from "edit this file
>>>>>>> and save it, then restart rpc.xyzpdq". Build some
>>>>>>> command line interfaces for this.
>>>>> 
>>>>> I'm OK with that.
>>>>> 
>>>>> (Note do have that for information in /etc/exports--we have exportfs.
>>>>> Is there a reason that didn't work for fedfs-utils?)
>>>> 
>>>> To make changes that can survive a server reboot,
>>>> you have to update /etc/exports.
> 
> 
> 
> Your suggestion then is to build a new command-line interface to:
> 
> - tell mountd of a V4ROOT multipath list?
> - have said list survive reboots, e.g. stored in a file?
> 
> Please povide more detail on your thoughts.

Persistence of multipath information doesn't
seem as necessary as it is for FedFS junctions.
I was just answering Bruce's question about why
exportfs is not adequate for FedFS junctions.

In fact, the multipath list itself will need
attention after any change of the server's
network configuration, including after a reboot
where an i/f can be potentially added or removed.
So the list is often not going to be persistent
at all.

However, the policy that determines how the
multipath list is rebuilt should probably be
stored in a configuration file. Eg, which i/f's
to ignore (like lo); classes of i/f's to
ignore (like IPv6 or RDMA); which i/f's should
always be considered for the list if they are
up; whether to advertise IP addresses or
hostnames; and so on.

So I agree, an xattr on / is right out, and
/etc/exports is probably not a good long term
solution either. But exportfs is a closer
analog to how to manage the multipath list.

If you can reuse exportfs that's OK too.


> Thanks
> 
> —> Andy
> 
>>>> 
>>>> 
>>>> 
>>>>> --b.
>>>>> 
>>>>>>> 
>>>>>>> And as you have suggested many times: separate
>>>>>>> policy from mechanism. /etc/exports is the
>>>>>>> mechanism.
>>>>>>> 
>>>>>>> --
>>>>>>> Chuck Lever
>>>>>> 
>>>>>> Bruce - do you have a preference between #1 and #2 or #3 (or another idea?)
>>>>>> 
>>>>>> Thanks
>>>>>> 
>>>>>> —>Andy
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> 
>>>> --
>>>> Chuck Lever
>>>> 
>>>> 
>>>> 
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
>> --
>> Chuck Lever

--
Chuck Lever

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[J. Bruce Fields]

On Thu, May 26, 2016 at 09:54:50AM -0400, Andy Adamson wrote:
> On Wed, May 25, 2016 at 2:55 PM, Chuck Lever <chuck.lever@oracle.com> wrote:
> >
> >> On May 25, 2016, at 2:48 PM, bfields@fieldses.org wrote:
> >>
> >> On Wed, May 25, 2016 at 05:29:35PM +0000, Adamson, Andy wrote:
> >>> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs.
> >>>
> >>> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list.
> >>>
> >>> Some background:
> >>>
> >>> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10):
> >>>
> >>> The pseudo-fs GETATTR(fs_locations) probe session trunking
> >>> was tested against a Linux server with a pseudo-fs
> >>> export stanza (e.g. a stanza with the fsid=0 or fsid=root
> >>> export option) and a replicas= export option
> >>> (replicas=<path1>@<server1>:<path2>@<server2>..)
> >>> Note that this configuration is for testing only. A future
> >>> patchset will add the replicas= configuration to the
> >>> NFSEXP_V4ROOT nfsd and mountd processing.
> >>>
> >>>
> >>> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline.
> >>>
> >>>
> >>>> Begin forwarded message:
> >>>>
> >>>> From: Chuck Lever <chuck.lever@oracle.com>
> >>>> Subject: Re: Configuring fs_locations on Linux upstream server
> >>>> Date: May 6, 2016 at 4:31:00 PM EDT
> >>>> To: "J. Bruce Fields" <bfields@fieldses.org>
> >>>> Cc: "Adamson, Andy" <William.Adamson@netapp.com>
> >>>>
> >>>>
> >>>>> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> >>>>>
> >>>>> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote:
> >>>>>> Seems like when a server does not return a list, that is
> >>>>>> information the client can use: basically, there is no
> >>>>>> ability to do any session trunking. It has to be set up
> >>>>>> explicitly; is that a bad thing, operationally?
> >>>>>
> >>>>> I like the idea of it being opt in on the server.
> >>>>>
> >>>>> Suppose the server transparently starts advertising all available
> >>>>> addresses for session trunking.  It's not hard to imagine cases where
> >>>>> that would go wrong.  E.g., maybe the server has the odd wireless or
> >>>>> 100Mb or other interface that happens to work but that's slow.  Then
> >>>>> somebody upgrades their server and performance goes down and it may take
> >>>>> them a while to figure out why.  Whereas if they'd had to opt in they'd
> >>>>> probably have avoided advertising an inappropriate interface.  Or at
> >>>>> least they'd have a better chance of figuring out that turning on
> >>>>> trunking was what caused the problem.
> >>>>>
> >>>>> I'd rather not force people to export "/" explicitly, though.  It's fine
> >>>>> for testing, but:
> >>>>>
> >>>>>    - I don't think we give a way to do an explicit V4ROOT export,
> >>>>>      so they'd be exposing their entire root partition.  We could
> >>>>>      fix that, but
> >>>>>    - the pseudofs just seems to me like something people shouldn't
> >>>>>      normally have to think about.  It's a protocol implementation
> >>>>>      detail, I'd rather hide it.  It'd be to easy to configure it a
> >>>>>      little wrong, I think.
> >>>>>
> >>>>> We can still do this by adding a replicas= option to the / export, but
> >>>>> we can let rpc.mountd do that internally instead of making the admin add
> >>>>> it to /etc/exports.
> >>>>>
> >>>>> But then you still need a way for the admin to tell rpc.mountd to cook
> >>>>> up the replicas= option.....  I'm not sure what that should look like.
> >>>
> >>> Idea 1: extra syntax in /etc/exports
> >>
> >> It's not really export-specific information.  I wonder if it'd be better
> >> to pass it on the rpc.nfsd commandline?
> >>
> >>       rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2"
> >>
> >> (and then that can be configured in /etc/sysconfig/nfs or whatever)?
> 
> Is this (the rpc.nfsd command line and /etc/sysconfig/nfs entry) the
> preferred way?
> Is /etc/sysconfig/nfs read upon reboot?

Yes.  (Well, the details are distribution-dependent, I think it's up to
the /usr/lib/systemd/scripts/nfs-utils_env.sh script referenced in
nfs-utils/systemd/nfs-config.service.)

The annoying thing about putting it on the rpc.nfsd commandline is that
it's mountd, not nfsd, that manages the NFSv4 pseudofs, and would be
responsible for cooking up the fs_locations info.

Let me think about it a little more....

--b.

Configuring flex file DS on Linux upstream

[Tom Haynes]

fwiw, I'm looking at being able to specify a remote DS for the next version
of the flex file server. Is it worth considering that in the discussion
being done in Andy's thread about "Configuring fs_locations on Linux upstream server pseudo fs for session trunking"?

For example:

/mds *(rw,pnfs,ff_ds=192.168.2.101.2049:/ds,insecure,no_root_squash,no_all_squash)

And in the far future, I'd want it to be a multipath list.

I don't see this as something that changes often and the exports
seem like the most natural fit.

Re: Configuring flex file DS on Linux upstream

[J. Bruce Fields]

On Thu, May 26, 2016 at 11:31:34AM -0700, Tom Haynes wrote:
> fwiw, I'm looking at being able to specify a remote DS for the next version
> of the flex file server. Is it worth considering that in the discussion
> being done in Andy's thread about "Configuring fs_locations on Linux upstream server pseudo fs for session trunking"?
> 
> For example:
> 
> /mds *(rw,pnfs,ff_ds=192.168.2.101.2049:/ds,insecure,no_root_squash,no_all_squash)
> 
> And in the far future, I'd want it to be a multipath list.
> 
> I don't see this as something that changes often and the exports
> seem like the most natural fit.

I don't understand the ":/ds" part.  You just look up stuff by
filehandle on the data server, so we shouldn't have to know anything
about paths there, should we?  Or is the problem that it might need to
mount something first in the v3 case?

Otherwise, I guess exports makes sense, as long as this really is
per-export information.

The thing that's a little different about the multipath information is
that that's really a global server property, not something that can vary
from one filesystem to another.

--b.

Re: Configuring flex file DS on Linux upstream

[Adamson, Andy]

DQo+IE9uIE1heSAyNiwgMjAxNiwgYXQgMjo0MSBQTSwgSi4gQnJ1Y2UgRmllbGRzIDxiZmllbGRz
QGZpZWxkc2VzLm9yZz4gd3JvdGU6DQo+IA0KPiBPbiBUaHUsIE1heSAyNiwgMjAxNiBhdCAxMToz
MTozNEFNIC0wNzAwLCBUb20gSGF5bmVzIHdyb3RlOg0KPj4gZndpdywgSSdtIGxvb2tpbmcgYXQg
YmVpbmcgYWJsZSB0byBzcGVjaWZ5IGEgcmVtb3RlIERTIGZvciB0aGUgbmV4dCB2ZXJzaW9uDQo+
PiBvZiB0aGUgZmxleCBmaWxlIHNlcnZlci4gSXMgaXQgd29ydGggY29uc2lkZXJpbmcgdGhhdCBp
biB0aGUgZGlzY3Vzc2lvbg0KPj4gYmVpbmcgZG9uZSBpbiBBbmR5J3MgdGhyZWFkIGFib3V0ICJD
b25maWd1cmluZyBmc19sb2NhdGlvbnMgb24gTGludXggdXBzdHJlYW0gc2VydmVyIHBzZXVkbyBm
cyBmb3Igc2Vzc2lvbiB0cnVua2luZyI/DQo+PiANCj4+IEZvciBleGFtcGxlOg0KPj4gDQo+PiAv
bWRzICoocncscG5mcyxmZl9kcz0xOTIuMTY4LjIuMTAxLjIwNDk6L2RzLGluc2VjdXJlLG5vX3Jv
b3Rfc3F1YXNoLG5vX2FsbF9zcXVhc2gpDQo+PiANCj4+IEFuZCBpbiB0aGUgZmFyIGZ1dHVyZSwg
SSdkIHdhbnQgaXQgdG8gYmUgYSBtdWx0aXBhdGggbGlzdC4NCj4+IA0KPj4gSSBkb24ndCBzZWUg
dGhpcyBhcyBzb21ldGhpbmcgdGhhdCBjaGFuZ2VzIG9mdGVuIGFuZCB0aGUgZXhwb3J0cw0KPj4g
c2VlbSBsaWtlIHRoZSBtb3N0IG5hdHVyYWwgZml0Lg0KPiANCj4gSSBkb24ndCB1bmRlcnN0YW5k
IHRoZSAiOi9kcyIgcGFydC4gIFlvdSBqdXN0IGxvb2sgdXAgc3R1ZmYgYnkNCj4gZmlsZWhhbmRs
ZSBvbiB0aGUgZGF0YSBzZXJ2ZXIsIHNvIHdlIHNob3VsZG4ndCBoYXZlIHRvIGtub3cgYW55dGhp
bmcNCj4gYWJvdXQgcGF0aHMgdGhlcmUsIHNob3VsZCB3ZT8gIE9yIGlzIHRoZSBwcm9ibGVtIHRo
YXQgaXQgbWlnaHQgbmVlZCB0bw0KPiBtb3VudCBzb21ldGhpbmcgZmlyc3QgaW4gdGhlIHYzIGNh
c2U/DQo+IA0KPiBPdGhlcndpc2UsIEkgZ3Vlc3MgZXhwb3J0cyBtYWtlcyBzZW5zZSwgYXMgbG9u
ZyBhcyB0aGlzIHJlYWxseSBpcw0KPiBwZXItZXhwb3J0IGluZm9ybWF0aW9uLg0KDQoNCklJVUMg
dGhlIGZmX2RzPSBsaW5lIHRlbGxzIHRoZSBtZHMgb2YgZGF0YSBzZXJ2ZXJzIGl0IGlzIG1hbmFn
aW5nLCBhbmQgaG93IHRvIG1vdW50IHRoZW0uIFNvIHRoaXMgaXMgbm90IG1kcyBleHBvcnQgaW5m
b3JtYXRpb24gLSBtZHMgaXMgbm90IGV4cG9ydGluZyB0aGUgZmZfZHM9ICwgcmF0aGVyIG1kcyBt
b3VudCBpbmZvcm1hdGlvbi4NCg0K4oCUPkFuZHkNCg0KPiANCj4gDQo+IFRoZSB0aGluZyB0aGF0
J3MgYSBsaXR0bGUgZGlmZmVyZW50IGFib3V0IHRoZSBtdWx0aXBhdGggaW5mb3JtYXRpb24gaXMN
Cj4gdGhhdCB0aGF0J3MgcmVhbGx5IGEgZ2xvYmFsIHNlcnZlciBwcm9wZXJ0eSwgbm90IHNvbWV0
aGluZyB0aGF0IGNhbiB2YXJ5DQo+IGZyb20gb25lIGZpbGVzeXN0ZW0gdG8gYW5vdGhlci4NCj4g
DQo+IC0tYi4NCg0K

Re: Configuring flex file DS on Linux upstream

[Tom Haynes]

On Thu, May 26, 2016 at 02:41:18PM -0400, J. Bruce Fields wrote:
> On Thu, May 26, 2016 at 11:31:34AM -0700, Tom Haynes wrote:
> > fwiw, I'm looking at being able to specify a remote DS for the next version
> > of the flex file server. Is it worth considering that in the discussion
> > being done in Andy's thread about "Configuring fs_locations on Linux upstream server pseudo fs for session trunking"?
> > 
> > For example:
> > 
> > /mds *(rw,pnfs,ff_ds=192.168.2.101.2049:/ds,insecure,no_root_squash,no_all_squash)
> > 
> > And in the far future, I'd want it to be a multipath list.
> > 
> > I don't see this as something that changes often and the exports
> > seem like the most natural fit.
> 
> I don't understand the ":/ds" part.  You just look up stuff by
> filehandle on the data server, so we shouldn't have to know anything
> about paths there, should we?  Or is the problem that it might need to
> mount something first in the v3 case?

You need at least the root fh in order to create/remove files:

 struct CREATE3args {
           diropargs3   where;
           createhow3   how;
 };

 struct REMOVE3args {
           diropargs3  object;
 };


> 
> Otherwise, I guess exports makes sense, as long as this really is
> per-export information.
> 
> The thing that's a little different about the multipath information is
> that that's really a global server property, not something that can vary
> from one filesystem to another.

Thanks for the explanation...

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[Steve Dickson]

Hello,

Sorry for coming the party late... 

On 05/25/2016 01:29 PM, Adamson, Andy wrote:
> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs.
> 
> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list.
> 
> Some background:
> 
> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10):
> 
> The pseudo-fs GETATTR(fs_locations) probe session trunking
> was tested against a Linux server with a pseudo-fs
> export stanza (e.g. a stanza with the fsid=0 or fsid=root
> export option) and a replicas= export option
> (replicas=<path1>@<server1>:<path2>@<server2>..)
> Note that this configuration is for testing only. A future
> patchset will add the replicas= configuration to the
> NFSEXP_V4ROOT nfsd and mountd processing.
> 
> 
> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline.
> 
> 
>> Begin forwarded message:
>>
>> From: Chuck Lever <chuck.lever@oracle.com>
>> Subject: Re: Configuring fs_locations on Linux upstream server
>> Date: May 6, 2016 at 4:31:00 PM EDT
>> To: "J. Bruce Fields" <bfields@fieldses.org>
>> Cc: "Adamson, Andy" <William.Adamson@netapp.com>
>>
>>
>>> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
>>>
>>> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote:
>>>> Seems like when a server does not return a list, that is
>>>> information the client can use: basically, there is no
>>>> ability to do any session trunking. It has to be set up
>>>> explicitly; is that a bad thing, operationally?
>>>
>>> I like the idea of it being opt in on the server.
>>>
>>> Suppose the server transparently starts advertising all available
>>> addresses for session trunking.  It's not hard to imagine cases where
>>> that would go wrong.  E.g., maybe the server has the odd wireless or
>>> 100Mb or other interface that happens to work but that's slow.  Then
>>> somebody upgrades their server and performance goes down and it may take
>>> them a while to figure out why.  Whereas if they'd had to opt in they'd
>>> probably have avoided advertising an inappropriate interface.  Or at
>>> least they'd have a better chance of figuring out that turning on
>>> trunking was what caused the problem.
>>>
>>> I'd rather not force people to export "/" explicitly, though.  It's fine
>>> for testing, but:
>>>
>>> 	- I don't think we give a way to do an explicit V4ROOT export,
>>> 	  so they'd be exposing their entire root partition.  We could
>>> 	  fix that, but
>>> 	- the pseudofs just seems to me like something people shouldn't
>>> 	  normally have to think about.  It's a protocol implementation
>>> 	  detail, I'd rather hide it.  It'd be to easy to configure it a
>>> 	  little wrong, I think.
>>>
>>> We can still do this by adding a replicas= option to the / export, but
>>> we can let rpc.mountd do that internally instead of making the admin add
>>> it to /etc/exports.
>>>
>>> But then you still need a way for the admin to tell rpc.mountd to cook
>>> up the replicas= option.....  I'm not sure what that should look like.
> 
> Idea 1: extra syntax in /etc/exports
As someone said during this thread we really should be moving
away from edit/write/rerun configuration... When you are
talking about 1000s of entries it becomes a bit painful.

> 
> 
>>> Maybe some extra syntax in /etc/exports, but what do they need to give
>>> us--just one list of IP addresses?  Chuck, any ideas?
> 
> Idea 2: xattr attached to “/"
I guess this is not possible... so its out.

> 
>>
>> How about using the same approach used for junctions:
>> put the list in an xattr attached to / ? mountd can
>> extract that when the kernel asks for help satisfying
>> a GETATTR(fs_locations) on V4ROOT.
> 
> 
> Idea 3: new /etc/ config file
>>
>> Or it could be put in a separate config file in /etc.
>> You might want to specify more than just the i/f list
>> here; for instance, the security policy for the
>> pseudofs, or a constant fsid UUID, among other things.
Add new configuration files is become very unpopular in
a particular distro I know of... ;-) and once again 
it comes down managing flat files... 
 
> 
> 
> API to update the i/f list.  This is not about where to hold fs_locations config info, but rather how to insert the (changed) info into the running system.
> 
>>
>> Also, I suggested to Andy earlier:
>>
>>> I find myself leaning towards mechanisms that are easy
>>> both for admins and for programs (ie, an API). Perhaps
>>> one day you might want to add a command that updates the
>>> i/f list from the scripts in /etc/sysconfig/network-scripts,
>>> for instance.
>>>
>>> As part of an ifup:
>>>
>>> nfspfs add <addr>
>>>
>>> and ifdown:
>>>
>>> nfspfs remove <addr>
What happens to this idea? This seems like the most straightforward...

I'm guess I'm leaning for new commands to new stuff instead
of globing on new stuff on old stuff... if that makes any sense. ;-)

Question, these list of ip address are they dynamic? Meaning
can the change on the fly or they are only set on startup
and restarts?

>>>
>>> I wrote some Python code to manipulate entries in
>>> /etc/exports, now found in fedfs-utils. It's icky.
I never like editing files that admins also edit. It
just seems like not a good thing to do.

>>
>>
>> I think we should move away from "edit this file
>> and save it, then restart rpc.xyzpdq". Build some
>> command line interfaces for this.
This what I was thinking about above... 

>>
>> And as you have suggested many times: separate
>> policy from mechanism. /etc/exports is the
>> mechanism.
Hmm.. I see this as adding more complexity to something
that is already complex... 

my two cents,

steved.

>>
>> --
>> Chuck Lever
> 
> Bruce - do you have a preference between #1 and #2 or #3 (or another idea?)
> 
> Thanks
> 
> —>AndyN�����r��y���b�X��ǧv�^�)޺{.n�+����{���"��^n�r���z�\x1a��h����&��\x1e�G���h�\x03(�階�ݢj"��\x1a�^[m�����z�ޖ���f���h���~�mml==
> 

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[Steve Dickson]

On 05/26/2016 11:22 AM, J. Bruce Fields wrote:
>>>>> Idea 1: extra syntax in /etc/exports
>>>> > >>
>>>> > >> It's not really export-specific information.  I wonder if it'd be better
>>>> > >> to pass it on the rpc.nfsd commandline?
>>>> > >>
>>>> > >>       rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2"
>>>> > >>
>>>> > >> (and then that can be configured in /etc/sysconfig/nfs or whatever)?
>> > 
>> > Is this (the rpc.nfsd command line and /etc/sysconfig/nfs entry) the
>> > preferred way?
>> > Is /etc/sysconfig/nfs read upon reboot?
Again, I think this is just globing new stuff on old stuff... 

I'm just thinking having new commands to manage new technology
better direction. 
  
> Yes.  (Well, the details are distribution-dependent, I think it's up to
> the /usr/lib/systemd/scripts/nfs-utils_env.sh script referenced in
> nfs-utils/systemd/nfs-config.service.)
These new command could have their very own system start up scripts

> 
> The annoying thing about putting it on the rpc.nfsd commandline is that
> it's mountd, not nfsd, that manages the NFSv4 pseudofs, and would be
> responsible for cooking up the fs_locations info.
I agree with this... structurally this just seems wrong.

steved. 

Re: Configuring fs_locations on Linux upstream server pseudo fs for session trunking

[J. Bruce Fields]

On Thu, May 26, 2016 at 11:22:50AM -0400, Chuck Lever wrote:
> 
> > On May 26, 2016, at 10:44 AM, Adamson, Andy <William.Adamson@netapp.com> wrote:
> > 
> >> 
> >> On May 26, 2016, at 10:25 AM, Chuck Lever <chuck.lever@oracle.com> wrote:
> >> 
> >> 
> >>> On May 26, 2016, at 9:54 AM, Andy Adamson <androsadamson@gmail.com> wrote:
> >>> 
> >>> On Wed, May 25, 2016 at 2:55 PM, Chuck Lever <chuck.lever@oracle.com> wrote:
> >>>> 
> >>>>> On May 25, 2016, at 2:48 PM, bfields@fieldses.org wrote:
> >>>>> 
> >>>>> On Wed, May 25, 2016 at 05:29:35PM +0000, Adamson, Andy wrote:
> >>>>>> Anna Schumaker who reviewed my client side session trunking patchset, wants a full featured version of both the client and the server session trunking pieces before accepting the session trunking feature upstream. To that end, I want to implement the server mountd V4ROOT processing of an fs_locations configuration to satisfy an fs_locations request on the pseudo fs.
> >>>>>> 
> >>>>>> The forwarded message is from an email stream between Bruce, Chuck and I concerning the server pseufo fs fs_locations configuration that I’m now sharing with the list.
> >>>>>> 
> >>>>>> Some background:
> >>>>>> 
> >>>>>> The recent "NFSV4.1,2 session trunking” Version-5 patch set sent to the list notes (in patch 00/10):
> >>>>>> 
> >>>>>> The pseudo-fs GETATTR(fs_locations) probe session trunking
> >>>>>> was tested against a Linux server with a pseudo-fs
> >>>>>> export stanza (e.g. a stanza with the fsid=0 or fsid=root
> >>>>>> export option) and a replicas= export option
> >>>>>> (replicas=<path1>@<server1>:<path2>@<server2>..)
> >>>>>> Note that this configuration is for testing only. A future
> >>>>>> patchset will add the replicas= configuration to the
> >>>>>> NFSEXP_V4ROOT nfsd and mountd processing.
> >>>>>> 
> >>>>>> 
> >>>>>> There are several ideas on how to accomplish mountd/V4ROOT fs_locations configuration in the forwarded message. See inline.
> >>>>>> 
> >>>>>> 
> >>>>>>> Begin forwarded message:
> >>>>>>> 
> >>>>>>> From: Chuck Lever <chuck.lever@oracle.com>
> >>>>>>> Subject: Re: Configuring fs_locations on Linux upstream server
> >>>>>>> Date: May 6, 2016 at 4:31:00 PM EDT
> >>>>>>> To: "J. Bruce Fields" <bfields@fieldses.org>
> >>>>>>> Cc: "Adamson, Andy" <William.Adamson@netapp.com>
> >>>>>>> 
> >>>>>>> 
> >>>>>>>> On May 6, 2016, at 4:16 PM, J. Bruce Fields <bfields@fieldses.org> wrote:
> >>>>>>>> 
> >>>>>>>> On Fri, May 06, 2016 at 02:20:12PM -0400, Chuck Lever wrote:
> >>>>>>>>> Seems like when a server does not return a list, that is
> >>>>>>>>> information the client can use: basically, there is no
> >>>>>>>>> ability to do any session trunking. It has to be set up
> >>>>>>>>> explicitly; is that a bad thing, operationally?
> >>>>>>>> 
> >>>>>>>> I like the idea of it being opt in on the server.
> >>>>>>>> 
> >>>>>>>> Suppose the server transparently starts advertising all available
> >>>>>>>> addresses for session trunking.  It's not hard to imagine cases where
> >>>>>>>> that would go wrong.  E.g., maybe the server has the odd wireless or
> >>>>>>>> 100Mb or other interface that happens to work but that's slow.  Then
> >>>>>>>> somebody upgrades their server and performance goes down and it may take
> >>>>>>>> them a while to figure out why.  Whereas if they'd had to opt in they'd
> >>>>>>>> probably have avoided advertising an inappropriate interface.  Or at
> >>>>>>>> least they'd have a better chance of figuring out that turning on
> >>>>>>>> trunking was what caused the problem.
> >>>>>>>> 
> >>>>>>>> I'd rather not force people to export "/" explicitly, though.  It's fine
> >>>>>>>> for testing, but:
> >>>>>>>> 
> >>>>>>>> - I don't think we give a way to do an explicit V4ROOT export,
> >>>>>>>>   so they'd be exposing their entire root partition.  We could
> >>>>>>>>   fix that, but
> >>>>>>>> - the pseudofs just seems to me like something people shouldn't
> >>>>>>>>   normally have to think about.  It's a protocol implementation
> >>>>>>>>   detail, I'd rather hide it.  It'd be to easy to configure it a
> >>>>>>>>   little wrong, I think.
> >>>>>>>> 
> >>>>>>>> We can still do this by adding a replicas= option to the / export, but
> >>>>>>>> we can let rpc.mountd do that internally instead of making the admin add
> >>>>>>>> it to /etc/exports.
> >>>>>>>> 
> >>>>>>>> But then you still need a way for the admin to tell rpc.mountd to cook
> >>>>>>>> up the replicas= option.....  I'm not sure what that should look like.
> >>>>>> 
> >>>>>> Idea 1: extra syntax in /etc/exports
> >>>>> 
> >>>>> It's not really export-specific information.  I wonder if it'd be better
> >>>>> to pass it on the rpc.nfsd commandline?
> >>>>> 
> >>>>>    rpc.nfsd --multipath-set="192.168.0.1,192.168.0.2"
> >>>>> 
> >>>>> (and then that can be configured in /etc/sysconfig/nfs or whatever)?
> >>> 
> >>> Is this (the rpc.nfsd command line and /etc/sysconfig/nfs entry) the
> >>> preferred way?
> >> 
> >> I don't prefer it.
> >> 
> >> See below: I think we want something that is more
> >> convenient to update automatically.
> > 
> > Fine, but I’m having difficulty in understanding the design you are suggesting to fulfill the update automatically requirement.
> > See inline below.
> > 
> >> 
> >> 
> >>> Is /etc/sysconfig/nfs read upon reboot?
> >> 
> >> It's read by all the start-up scripts related to NFS.
> >> 
> >> 
> >>> -->Andy
> >>> 
> >>> 
> >>> 
> >>>>> 
> >>>>>>>> Maybe some extra syntax in /etc/exports, but what do they need to give
> >>>>>>>> us--just one list of IP addresses?  Chuck, any ideas?
> >>>>>> 
> >>>>>> Idea 2: xattr attached to “/"
> >>>>>> 
> >>>>>>> 
> >>>>>>> How about using the same approach used for junctions:
> >>>>>>> put the list in an xattr attached to / ? mountd can
> >>>>>>> extract that when the kernel asks for help satisfying
> >>>>>>> a GETATTR(fs_locations) on V4ROOT.
> >>>>> 
> >>>>> I don't think that works.  "/" isn't a good place to put configuration.
> >>>>> It could be read-only, among other things.
> >>>>> 
> >>>>>> Idea 3: new /etc/ config file
> >>>>>>> 
> >>>>>>> Or it could be put in a separate config file in /etc.
> >>>>>>> You might want to specify more than just the i/f list
> >>>>>>> here; for instance, the security policy for the
> >>>>>>> pseudofs, or a constant fsid UUID, among other things.
> >>>>>> 
> >>>>>> 
> >>>>>> API to update the i/f list.  This is not about where to hold fs_locations config info, but rather how to insert the (changed) info into the running system.
> >>>>>> 
> >>>>>>> 
> >>>>>>> Also, I suggested to Andy earlier:
> >>>>>>> 
> >>>>>>>> I find myself leaning towards mechanisms that are easy
> >>>>>>>> both for admins and for programs (ie, an API). Perhaps
> >>>>>>>> one day you might want to add a command that updates the
> >>>>>>>> i/f list from the scripts in /etc/sysconfig/network-scripts,
> >>>>>>>> for instance.
> 
> >>>>>>>> As part of an ifup:
> >>>>>>>> 
> >>>>>>>> nfspfs add <addr>
> >>>>>>>> 
> >>>>>>>> and ifdown:
> >>>>>>>> 
> >>>>>>>> nfspfs remove <addr>
> >>>>>>>> 
> >>>>>>>> I wrote some Python code to manipulate entries in
> >>>>>>>> /etc/exports, now found in fedfs-utils. It's icky.
> >>>>>>> 
> >>>>>>> I think we should move away from "edit this file
> >>>>>>> and save it, then restart rpc.xyzpdq". Build some
> >>>>>>> command line interfaces for this.
> >>>>> 
> >>>>> I'm OK with that.
> >>>>> 
> >>>>> (Note do have that for information in /etc/exports--we have exportfs.
> >>>>> Is there a reason that didn't work for fedfs-utils?)
> >>>> 
> >>>> To make changes that can survive a server reboot,
> >>>> you have to update /etc/exports.
> > 
> > 
> > 
> > Your suggestion then is to build a new command-line interface to:
> > 
> > - tell mountd of a V4ROOT multipath list?
> > - have said list survive reboots, e.g. stored in a file?
> > 
> > Please povide more detail on your thoughts.
> 
> Persistence of multipath information doesn't
> seem as necessary as it is for FedFS junctions.
> I was just answering Bruce's question about why
> exportfs is not adequate for FedFS junctions.
> 
> In fact, the multipath list itself will need
> attention after any change of the server's
> network configuration, including after a reboot
> where an i/f can be potentially added or removed.
> So the list is often not going to be persistent
> at all.
> 
> However, the policy that determines how the
> multipath list is rebuilt should probably be
> stored in a configuration file. Eg, which i/f's
> to ignore (like lo); classes of i/f's to
> ignore (like IPv6 or RDMA); which i/f's should
> always be considered for the list if they are
> up; whether to advertise IP addresses or
> hostnames; and so on.
> 
> So I agree, an xattr on / is right out, and
> /etc/exports is probably not a good long term
> solution either. But exportfs is a closer
> analog to how to manage the multipath list.
> 
> If you can reuse exportfs that's OK too.

OK, so you could check whether we could piggyback on the same mechanism
exportfs and mountd use to communicate (which is basically just reading,
writing, and locking /var/lib/nfs/etab, if I remember right).  And then
add some syntax to the exportfs commandline.  As long as you're doing
that, I don't see why you couldn't also allow people to specify the same
information in /etc/exports, if they'd like.  But, whatever, maybe
that's not necessary.

A new rpc.mountd commandline option might work too?

Whatever we choose doesn't have to do everything we want--as long as we
can build whatever you want on top of it later.  So, ideally we choose
something simple that we can build on later, just to unblock Andy's
work.

--b.