SP4_MACH_CRED: v4 proc -> opcodes mapping

SP4_MACH_CRED: v4 proc -> opcodes mapping

[Adamson, Dros]

Hey,

I have a mostly functional client-side implementation of SP4_MACH_CRED! It still needs a lot of cleanup and testing.

I have one style issue that I want to run by the list before I post a patchset:

So, SP4_MACH_CRED negotiates two bitmaps in the EXCHANGE_ID (one "enforce", one "allow") for state protection. These bitmaps are indexed by the NFSv4 operation number.  The state protect check must happen in the nfs4proc.c layer (or before), right before we call rpc_call_sync or equivalent, so that it can select the right cred and rpc_client.

Here's the problem: we don't know what operations (opcodes) are actually in a compound until the XDR encode callback is called.  The rpc_procinfo array doesn't have this mapping - in fact, it only lives in the xdr encode layer.

One approach is to immediately translate the opcode bitmaps to "nfs4 procedure index" bitmaps, indexing into the rpc_procinfo array.  This would mean there is a second mapping of NFSv4 procedure -> opcodes that must be updated when an XDR encode callback is changed.

Another approach would be to add a callback to the XDR api so we could "ask" it if an NFSv4 procedure contains any of the opcodes in a bitmap. The nice thing about this approach is that the mapping of procedure to opcodes within will live right next to the XDR encode callback and make it obvious that both need to be changed together.

I suppose I'm leaning toward a combination of both of these approaches - keep the mapping in XDR-land and translate the bitmaps immediately on negation for fast lookups during normal operation.

Comments? Am I missing something?

-dros

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDE0OjQ2ICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiBIZXksDQo+IA0KPiBJIGhhdmUgYSBtb3N0bHkgZnVuY3Rpb25hbCBjbGllbnQtc2lkZSBpbXBs
ZW1lbnRhdGlvbiBvZiBTUDRfTUFDSF9DUkVEISBJdCBzdGlsbCBuZWVkcyBhIGxvdCBvZiBjbGVh
bnVwIGFuZCB0ZXN0aW5nLg0KPiANCj4gSSBoYXZlIG9uZSBzdHlsZSBpc3N1ZSB0aGF0IEkgd2Fu
dCB0byBydW4gYnkgdGhlIGxpc3QgYmVmb3JlIEkgcG9zdCBhIHBhdGNoc2V0Og0KPiANCj4gU28s
IFNQNF9NQUNIX0NSRUQgbmVnb3RpYXRlcyB0d28gYml0bWFwcyBpbiB0aGUgRVhDSEFOR0VfSUQg
KG9uZSAiZW5mb3JjZSIsIG9uZSAiYWxsb3ciKSBmb3Igc3RhdGUgcHJvdGVjdGlvbi4gVGhlc2Ug
Yml0bWFwcyBhcmUgaW5kZXhlZCBieSB0aGUgTkZTdjQgb3BlcmF0aW9uIG51bWJlci4gIFRoZSBz
dGF0ZSBwcm90ZWN0IGNoZWNrIG11c3QgaGFwcGVuIGluIHRoZSBuZnM0cHJvYy5jIGxheWVyIChv
ciBiZWZvcmUpLCByaWdodCBiZWZvcmUgd2UgY2FsbCBycGNfY2FsbF9zeW5jIG9yIGVxdWl2YWxl
bnQsIHNvIHRoYXQgaXQgY2FuIHNlbGVjdCB0aGUgcmlnaHQgY3JlZCBhbmQgcnBjX2NsaWVudC4N
Cj4gDQo+IEhlcmUncyB0aGUgcHJvYmxlbTogd2UgZG9uJ3Qga25vdyB3aGF0IG9wZXJhdGlvbnMg
KG9wY29kZXMpIGFyZSBhY3R1YWxseSBpbiBhIGNvbXBvdW5kIHVudGlsIHRoZSBYRFIgZW5jb2Rl
IGNhbGxiYWNrIGlzIGNhbGxlZC4gIFRoZSBycGNfcHJvY2luZm8gYXJyYXkgZG9lc24ndCBoYXZl
IHRoaXMgbWFwcGluZyAtIGluIGZhY3QsIGl0IG9ubHkgbGl2ZXMgaW4gdGhlIHhkciBlbmNvZGUg
bGF5ZXIuDQo+IA0KPiBPbmUgYXBwcm9hY2ggaXMgdG8gaW1tZWRpYXRlbHkgdHJhbnNsYXRlIHRo
ZSBvcGNvZGUgYml0bWFwcyB0byAibmZzNCBwcm9jZWR1cmUgaW5kZXgiIGJpdG1hcHMsIGluZGV4
aW5nIGludG8gdGhlIHJwY19wcm9jaW5mbyBhcnJheS4gIFRoaXMgd291bGQgbWVhbiB0aGVyZSBp
cyBhIHNlY29uZCBtYXBwaW5nIG9mIE5GU3Y0IHByb2NlZHVyZSAtPiBvcGNvZGVzIHRoYXQgbXVz
dCBiZSB1cGRhdGVkIHdoZW4gYW4gWERSIGVuY29kZSBjYWxsYmFjayBpcyBjaGFuZ2VkLg0KPiAN
Cj4gQW5vdGhlciBhcHByb2FjaCB3b3VsZCBiZSB0byBhZGQgYSBjYWxsYmFjayB0byB0aGUgWERS
IGFwaSBzbyB3ZSBjb3VsZCAiYXNrIiBpdCBpZiBhbiBORlN2NCBwcm9jZWR1cmUgY29udGFpbnMg
YW55IG9mIHRoZSBvcGNvZGVzIGluIGEgYml0bWFwLiBUaGUgbmljZSB0aGluZyBhYm91dCB0aGlz
IGFwcHJvYWNoIGlzIHRoYXQgdGhlIG1hcHBpbmcgb2YgcHJvY2VkdXJlIHRvIG9wY29kZXMgd2l0
aGluIHdpbGwgbGl2ZSByaWdodCBuZXh0IHRvIHRoZSBYRFIgZW5jb2RlIGNhbGxiYWNrIGFuZCBt
YWtlIGl0IG9idmlvdXMgdGhhdCBib3RoIG5lZWQgdG8gYmUgY2hhbmdlZCB0b2dldGhlci4NCj4g
DQo+IEkgc3VwcG9zZSBJJ20gbGVhbmluZyB0b3dhcmQgYSBjb21iaW5hdGlvbiBvZiBib3RoIG9m
IHRoZXNlIGFwcHJvYWNoZXMgLSBrZWVwIHRoZSBtYXBwaW5nIGluIFhEUi1sYW5kIGFuZCB0cmFu
c2xhdGUgdGhlIGJpdG1hcHMgaW1tZWRpYXRlbHkgb24gbmVnYXRpb24gZm9yIGZhc3QgbG9va3Vw
cyBkdXJpbmcgbm9ybWFsIG9wZXJhdGlvbi4NCj4gDQo+IENvbW1lbnRzPyBBbSBJIG1pc3Npbmcg
c29tZXRoaW5nPw0KDQpJJ20gbm90IHN1cmUgdGhhdCBJIHVuZGVyc3RhbmQuIFdlIGRvbid0IGRv
IGR5bmFtaWMgY3JlYXRpb24gb2YNCmNvbXBvdW5kczogd2UgcHJldHR5IG11Y2gga25vdyBpbiB0
aGUgbmZzNF9wcm9jXyogcm91dGluZSB3aGF0IHRoZSBtYWluDQpvcGVyYXRpb24gaXMgKHRoZSBv
bmUgZXhjZXB0aW9uIGJlaW5nIENMT1NFL09QRU5fRE9XTkdSQURFKS4gU28gd2h5DQpjYW4ndCB3
ZSB3b3JrIG91dCB0aGUgcHJvdGVjdGlvbiBzdGF0dXMgYXQgdGhhdCB0aW1lPyBJcyB0aGUgd29y
cnkgdGhhdA0KdGhlIHNlcnZlciBtaWdodCByZWJvb3QgYW5kIGNvbWUgYmFjayB3aXRoIGRpZmZl
cmVudCBNQUNIX0NSRUQNCnByb3RlY3Rpb25zPw0KDQotLSANClRyb25kIE15a2xlYnVzdA0KTGlu
dXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFw
cC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Adamson, Dros]

On Jul 18, 2013, at 10:57 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:

> On Thu, 2013-07-18 at 14:46 +0000, Adamson, Dros wrote:
>> Hey,
>> 
>> I have a mostly functional client-side implementation of SP4_MACH_CRED! It still needs a lot of cleanup and testing.
>> 
>> I have one style issue that I want to run by the list before I post a patchset:
>> 
>> So, SP4_MACH_CRED negotiates two bitmaps in the EXCHANGE_ID (one "enforce", one "allow") for state protection. These bitmaps are indexed by the NFSv4 operation number.  The state protect check must happen in the nfs4proc.c layer (or before), right before we call rpc_call_sync or equivalent, so that it can select the right cred and rpc_client.
>> 
>> Here's the problem: we don't know what operations (opcodes) are actually in a compound until the XDR encode callback is called.  The rpc_procinfo array doesn't have this mapping - in fact, it only lives in the xdr encode layer.
>> 
>> One approach is to immediately translate the opcode bitmaps to "nfs4 procedure index" bitmaps, indexing into the rpc_procinfo array.  This would mean there is a second mapping of NFSv4 procedure -> opcodes that must be updated when an XDR encode callback is changed.
>> 
>> Another approach would be to add a callback to the XDR api so we could "ask" it if an NFSv4 procedure contains any of the opcodes in a bitmap. The nice thing about this approach is that the mapping of procedure to opcodes within will live right next to the XDR encode callback and make it obvious that both need to be changed together.
>> 
>> I suppose I'm leaning toward a combination of both of these approaches - keep the mapping in XDR-land and translate the bitmaps immediately on negation for fast lookups during normal operation.
>> 
>> Comments? Am I missing something?
> 
> I'm not sure that I understand. We don't do dynamic creation of
> compounds: we pretty much know in the nfs4_proc_* routine what the main
> operation is (the one exception being CLOSE/OPEN_DOWNGRADE). So why
> can't we work out the protection status at that time? Is the worry that
> the server might reboot and come back with different MACH_CRED
> protections?

Sure, we know what the main operation is and we know what operations will end up in a compound - we can just look at the xdr encoder.  My question is simply one of style. Do we want to have each nfs4proc procedure to have a list of operations that must be updated if an xdr encoder is updated? I'm fine with doing it this way, it just seemed wrong to have the same mapping in two different places. This is why I'm asking ;)

Also, we can't just use the "main" operation, we must check every operation within the compound and if any are required to use SP4, then the whole compound does. I don't necessarily know why a server would do this, but if we follow the spec and a server informs the client that SETATTR must use state protection, then a WRITE with post-op SETATTR must use the state protection even though WRITE doesn't need it.

I'm not following the reboot question - that case should be handled just fine. Once the EXCHANGE_ID happens, the clp has two bitmaps (enforce and allow) that reflect the current mode of SP4_MACH_CRED.

-dros

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDE1OjEwICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiBPbiBKdWwgMTgsIDIwMTMsIGF0IDEwOjU3IEFNLCAiTXlrbGVidXN0LCBUcm9uZCIgPFRyb25k
Lk15a2xlYnVzdEBuZXRhcHAuY29tPiB3cm90ZToNCj4gDQo+ID4gT24gVGh1LCAyMDEzLTA3LTE4
IGF0IDE0OjQ2ICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0KPiA+PiBIZXksDQo+ID4+IA0K
PiA+PiBJIGhhdmUgYSBtb3N0bHkgZnVuY3Rpb25hbCBjbGllbnQtc2lkZSBpbXBsZW1lbnRhdGlv
biBvZiBTUDRfTUFDSF9DUkVEISBJdCBzdGlsbCBuZWVkcyBhIGxvdCBvZiBjbGVhbnVwIGFuZCB0
ZXN0aW5nLg0KPiA+PiANCj4gPj4gSSBoYXZlIG9uZSBzdHlsZSBpc3N1ZSB0aGF0IEkgd2FudCB0
byBydW4gYnkgdGhlIGxpc3QgYmVmb3JlIEkgcG9zdCBhIHBhdGNoc2V0Og0KPiA+PiANCj4gPj4g
U28sIFNQNF9NQUNIX0NSRUQgbmVnb3RpYXRlcyB0d28gYml0bWFwcyBpbiB0aGUgRVhDSEFOR0Vf
SUQgKG9uZSAiZW5mb3JjZSIsIG9uZSAiYWxsb3ciKSBmb3Igc3RhdGUgcHJvdGVjdGlvbi4gVGhl
c2UgYml0bWFwcyBhcmUgaW5kZXhlZCBieSB0aGUgTkZTdjQgb3BlcmF0aW9uIG51bWJlci4gIFRo
ZSBzdGF0ZSBwcm90ZWN0IGNoZWNrIG11c3QgaGFwcGVuIGluIHRoZSBuZnM0cHJvYy5jIGxheWVy
IChvciBiZWZvcmUpLCByaWdodCBiZWZvcmUgd2UgY2FsbCBycGNfY2FsbF9zeW5jIG9yIGVxdWl2
YWxlbnQsIHNvIHRoYXQgaXQgY2FuIHNlbGVjdCB0aGUgcmlnaHQgY3JlZCBhbmQgcnBjX2NsaWVu
dC4NCj4gPj4gDQo+ID4+IEhlcmUncyB0aGUgcHJvYmxlbTogd2UgZG9uJ3Qga25vdyB3aGF0IG9w
ZXJhdGlvbnMgKG9wY29kZXMpIGFyZSBhY3R1YWxseSBpbiBhIGNvbXBvdW5kIHVudGlsIHRoZSBY
RFIgZW5jb2RlIGNhbGxiYWNrIGlzIGNhbGxlZC4gIFRoZSBycGNfcHJvY2luZm8gYXJyYXkgZG9l
c24ndCBoYXZlIHRoaXMgbWFwcGluZyAtIGluIGZhY3QsIGl0IG9ubHkgbGl2ZXMgaW4gdGhlIHhk
ciBlbmNvZGUgbGF5ZXIuDQo+ID4+IA0KPiA+PiBPbmUgYXBwcm9hY2ggaXMgdG8gaW1tZWRpYXRl
bHkgdHJhbnNsYXRlIHRoZSBvcGNvZGUgYml0bWFwcyB0byAibmZzNCBwcm9jZWR1cmUgaW5kZXgi
IGJpdG1hcHMsIGluZGV4aW5nIGludG8gdGhlIHJwY19wcm9jaW5mbyBhcnJheS4gIFRoaXMgd291
bGQgbWVhbiB0aGVyZSBpcyBhIHNlY29uZCBtYXBwaW5nIG9mIE5GU3Y0IHByb2NlZHVyZSAtPiBv
cGNvZGVzIHRoYXQgbXVzdCBiZSB1cGRhdGVkIHdoZW4gYW4gWERSIGVuY29kZSBjYWxsYmFjayBp
cyBjaGFuZ2VkLg0KPiA+PiANCj4gPj4gQW5vdGhlciBhcHByb2FjaCB3b3VsZCBiZSB0byBhZGQg
YSBjYWxsYmFjayB0byB0aGUgWERSIGFwaSBzbyB3ZSBjb3VsZCAiYXNrIiBpdCBpZiBhbiBORlN2
NCBwcm9jZWR1cmUgY29udGFpbnMgYW55IG9mIHRoZSBvcGNvZGVzIGluIGEgYml0bWFwLiBUaGUg
bmljZSB0aGluZyBhYm91dCB0aGlzIGFwcHJvYWNoIGlzIHRoYXQgdGhlIG1hcHBpbmcgb2YgcHJv
Y2VkdXJlIHRvIG9wY29kZXMgd2l0aGluIHdpbGwgbGl2ZSByaWdodCBuZXh0IHRvIHRoZSBYRFIg
ZW5jb2RlIGNhbGxiYWNrIGFuZCBtYWtlIGl0IG9idmlvdXMgdGhhdCBib3RoIG5lZWQgdG8gYmUg
Y2hhbmdlZCB0b2dldGhlci4NCj4gPj4gDQo+ID4+IEkgc3VwcG9zZSBJJ20gbGVhbmluZyB0b3dh
cmQgYSBjb21iaW5hdGlvbiBvZiBib3RoIG9mIHRoZXNlIGFwcHJvYWNoZXMgLSBrZWVwIHRoZSBt
YXBwaW5nIGluIFhEUi1sYW5kIGFuZCB0cmFuc2xhdGUgdGhlIGJpdG1hcHMgaW1tZWRpYXRlbHkg
b24gbmVnYXRpb24gZm9yIGZhc3QgbG9va3VwcyBkdXJpbmcgbm9ybWFsIG9wZXJhdGlvbi4NCj4g
Pj4gDQo+ID4+IENvbW1lbnRzPyBBbSBJIG1pc3Npbmcgc29tZXRoaW5nPw0KPiA+IA0KPiA+IEkn
bSBub3Qgc3VyZSB0aGF0IEkgdW5kZXJzdGFuZC4gV2UgZG9uJ3QgZG8gZHluYW1pYyBjcmVhdGlv
biBvZg0KPiA+IGNvbXBvdW5kczogd2UgcHJldHR5IG11Y2gga25vdyBpbiB0aGUgbmZzNF9wcm9j
Xyogcm91dGluZSB3aGF0IHRoZSBtYWluDQo+ID4gb3BlcmF0aW9uIGlzICh0aGUgb25lIGV4Y2Vw
dGlvbiBiZWluZyBDTE9TRS9PUEVOX0RPV05HUkFERSkuIFNvIHdoeQ0KPiA+IGNhbid0IHdlIHdv
cmsgb3V0IHRoZSBwcm90ZWN0aW9uIHN0YXR1cyBhdCB0aGF0IHRpbWU/IElzIHRoZSB3b3JyeSB0
aGF0DQo+ID4gdGhlIHNlcnZlciBtaWdodCByZWJvb3QgYW5kIGNvbWUgYmFjayB3aXRoIGRpZmZl
cmVudCBNQUNIX0NSRUQNCj4gPiBwcm90ZWN0aW9ucz8NCj4gDQo+IFN1cmUsIHdlIGtub3cgd2hh
dCB0aGUgbWFpbiBvcGVyYXRpb24gaXMgYW5kIHdlIGtub3cgd2hhdCBvcGVyYXRpb25zIHdpbGwg
ZW5kIHVwIGluIGEgY29tcG91bmQgLSB3ZSBjYW4ganVzdCBsb29rIGF0IHRoZSB4ZHIgZW5jb2Rl
ci4gIE15IHF1ZXN0aW9uIGlzIHNpbXBseSBvbmUgb2Ygc3R5bGUuIERvIHdlIHdhbnQgdG8gaGF2
ZSBlYWNoIG5mczRwcm9jIHByb2NlZHVyZSB0byBoYXZlIGEgbGlzdCBvZiBvcGVyYXRpb25zIHRo
YXQgbXVzdCBiZSB1cGRhdGVkIGlmIGFuIHhkciBlbmNvZGVyIGlzIHVwZGF0ZWQ/IEknbSBmaW5l
IHdpdGggZG9pbmcgaXQgdGhpcyB3YXksIGl0IGp1c3Qgc2VlbWVkIHdyb25nIHRvIGhhdmUgdGhl
IHNhbWUgbWFwcGluZyBpbiB0d28gZGlmZmVyZW50IHBsYWNlcy4gVGhpcyBpcyB3aHkgSSdtIGFz
a2luZyA7KQ0KPiANCj4gQWxzbywgd2UgY2FuJ3QganVzdCB1c2UgdGhlICJtYWluIiBvcGVyYXRp
b24sIHdlIG11c3QgY2hlY2sgZXZlcnkgb3BlcmF0aW9uIHdpdGhpbiB0aGUgY29tcG91bmQgYW5k
IGlmIGFueSBhcmUgcmVxdWlyZWQgdG8gdXNlIFNQNCwgdGhlbiB0aGUgd2hvbGUgY29tcG91bmQg
ZG9lcy4gSSBkb24ndCBuZWNlc3NhcmlseSBrbm93IHdoeSBhIHNlcnZlciB3b3VsZCBkbyB0aGlz
LCBidXQgaWYgd2UgZm9sbG93IHRoZSBzcGVjIGFuZCBhIHNlcnZlciBpbmZvcm1zIHRoZSBjbGll
bnQgdGhhdCBTRVRBVFRSIG11c3QgdXNlIHN0YXRlIHByb3RlY3Rpb24sIHRoZW4gYSBXUklURSB3
aXRoIHBvc3Qtb3AgU0VUQVRUUiBtdXN0IHVzZSB0aGUgc3RhdGUgcHJvdGVjdGlvbiBldmVuIHRo
b3VnaCBXUklURSBkb2Vzbid0IG5lZWQgaXQuDQoNCldlIG9ubHkgbmVlZCB0byBjaGVjayBldmVy
eSBfc3RhdGVmdWxfIG9wZXJhdGlvbiwgcmlnaHQ/IEkgY2FuJ3QgdGhpbmsNCm9mIGFueSBjb21w
b3VuZHMgd2l0aCBtb3JlIHRoYW4gMSBzdGF0ZWZ1bCBvcGVyYXRpb24uIFdlJ3ZlIGRlbGliZXJh
dGVseQ0KYXZvaWRlZCB0aG9zZSBkdWUgdG8gdGhlIHByb2JsZW1zIHRoYXQgYXJpc2Ugd2hlbiB5
b3UgZ2V0IGENCk5GUzRFUlJfREVMQVkgb3Igc29tZXRoaW5nIGVxdWl2YWxlbnQgYXQgdGhlIHdy
b25nIG1vbWVudC4NCg0KPiBJJ20gbm90IGZvbGxvd2luZyB0aGUgcmVib290IHF1ZXN0aW9uIC0g
dGhhdCBjYXNlIHNob3VsZCBiZSBoYW5kbGVkIGp1c3QgZmluZS4gT25jZSB0aGUgRVhDSEFOR0Vf
SUQgaGFwcGVucywgdGhlIGNscCBoYXMgdHdvIGJpdG1hcHMgKGVuZm9yY2UgYW5kIGFsbG93KSB0
aGF0IHJlZmxlY3QgdGhlIGN1cnJlbnQgbW9kZSBvZiBTUDRfTUFDSF9DUkVELg0KDQpJJ20gdGhp
bmtpbmcgc29tZXRoaW5nIGFsb25nIHRoZSBmb2xsb3dpbmcgc2NlbmFyaW86DQoNCkEgcHJvY2Vz
cyBxdWV1ZXMgdXAgYW4gT1BFTi4gVGhlIHNlcnZlciByZXBsaWVzIE5GUzRFUlJfQkFEX1NFU1NJ
T04gYW5kDQpsYXRlciByZXBsaWVzIHdpdGggTkZTNEVSUl9TVEFMRV9DTElFTlRJRC4gU28gd2Ug
c3RhcnQgcmVib290IHJlY292ZXJ5LA0KYW5kIHRoZSBPUEVOIGdldHMgcXVldWVkIHdhaXRpbmcg
Zm9yIGEgc2xvdC4NCldlIHRoZW4gc2VuZCBhIG5ldyBFWENIQU5HRV9JRCwgYW5kIHRoZSBzZXJ2
ZXIgcmVwbGllcyB3aXRoIGENCl9kaWZmZXJlbnRfIE1BQ0hfQ1JFRCBwcm90ZWN0aW9uIGZvciBP
UEVOLg0KDQpUaGUgcHJvYmxlbSBhYm92ZSBpcyB0aGF0IHRoZSBPUEVOIGhhcyBhbHJlYWR5IGJl
ZW4gc3RhcnRlZCwgYW5kIHdlJ3ZlDQphbHJlYWR5IGFzc2lnbmVkIGl0IGEgY3JlZGVudGlhbC4g
SG93IGRvIHdlIGhhbmRsZSB0aGF0Pw0KRGl0dG8gcXVlc3Rpb24gZm9yIFJFQUQsIFdSSVRFLCBM
T0NLLCBMQVlPVVRHRVQsLi4uDQoNCkRvIHdlIGhhdmUgYSBwcm9ibGVtIGZvciBMT0NLVSwgQ0xP
U0UgYW5kIE9QRU5fRE9XTkdSQURFPyBJIHRoaW5rIHdlIGVuZA0KdXAganVzdCBza2lwcGluZyBy
ZWNvdmVyeSBpbiB0aG9zZSBjYXNlcy4uLg0KDQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXgg
TkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5j
b20NCnd3dy5uZXRhcHAuY29tDQo=

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Adamson, Dros]

On Jul 18, 2013, at 11:21 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
 wrote:

> On Thu, 2013-07-18 at 15:10 +0000, Adamson, Dros wrote:
>> On Jul 18, 2013, at 10:57 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>> 
>>> On Thu, 2013-07-18 at 14:46 +0000, Adamson, Dros wrote:
>>>> Hey,
>>>> 
>>>> I have a mostly functional client-side implementation of SP4_MACH_CRED! It still needs a lot of cleanup and testing.
>>>> 
>>>> I have one style issue that I want to run by the list before I post a patchset:
>>>> 
>>>> So, SP4_MACH_CRED negotiates two bitmaps in the EXCHANGE_ID (one "enforce", one "allow") for state protection. These bitmaps are indexed by the NFSv4 operation number.  The state protect check must happen in the nfs4proc.c layer (or before), right before we call rpc_call_sync or equivalent, so that it can select the right cred and rpc_client.
>>>> 
>>>> Here's the problem: we don't know what operations (opcodes) are actually in a compound until the XDR encode callback is called.  The rpc_procinfo array doesn't have this mapping - in fact, it only lives in the xdr encode layer.
>>>> 
>>>> One approach is to immediately translate the opcode bitmaps to "nfs4 procedure index" bitmaps, indexing into the rpc_procinfo array.  This would mean there is a second mapping of NFSv4 procedure -> opcodes that must be updated when an XDR encode callback is changed.
>>>> 
>>>> Another approach would be to add a callback to the XDR api so we could "ask" it if an NFSv4 procedure contains any of the opcodes in a bitmap. The nice thing about this approach is that the mapping of procedure to opcodes within will live right next to the XDR encode callback and make it obvious that both need to be changed together.
>>>> 
>>>> I suppose I'm leaning toward a combination of both of these approaches - keep the mapping in XDR-land and translate the bitmaps immediately on negation for fast lookups during normal operation.
>>>> 
>>>> Comments? Am I missing something?
>>> 
>>> I'm not sure that I understand. We don't do dynamic creation of
>>> compounds: we pretty much know in the nfs4_proc_* routine what the main
>>> operation is (the one exception being CLOSE/OPEN_DOWNGRADE). So why
>>> can't we work out the protection status at that time? Is the worry that
>>> the server might reboot and come back with different MACH_CRED
>>> protections?
>> 
>> Sure, we know what the main operation is and we know what operations will end up in a compound - we can just look at the xdr encoder.  My question is simply one of style. Do we want to have each nfs4proc procedure to have a list of operations that must be updated if an xdr encoder is updated? I'm fine with doing it this way, it just seemed wrong to have the same mapping in two different places. This is why I'm asking ;)
>> 
>> Also, we can't just use the "main" operation, we must check every operation within the compound and if any are required to use SP4, then the whole compound does. I don't necessarily know why a server would do this, but if we follow the spec and a server informs the client that SETATTR must use state protection, then a WRITE with post-op SETATTR must use the state protection even though WRITE doesn't need it.
> 
> We only need to check every _stateful_ operation, right? I can't think
> of any compounds with more than 1 stateful operation. We've deliberately
> avoided those due to the problems that arise when you get a
> NFS4ERR_DELAY or something equivalent at the wrong moment.

That may be the intention, but I see no such limitation in the spec. It's my understanding that the server could specify any operation as MUST or MAY use state protection.

> 
>> I'm not following the reboot question - that case should be handled just fine. Once the EXCHANGE_ID happens, the clp has two bitmaps (enforce and allow) that reflect the current mode of SP4_MACH_CRED.
> 
> I'm thinking something along the following scenario:
> 
> A process queues up an OPEN. The server replies NFS4ERR_BAD_SESSION and
> later replies with NFS4ERR_STALE_CLIENTID. So we start reboot recovery,
> and the OPEN gets queued waiting for a slot.
> We then send a new EXCHANGE_ID, and the server replies with a
> _different_ MACH_CRED protection for OPEN.
> 
> The problem above is that the OPEN has already been started, and we've
> already assigned it a credential. How do we handle that?
> Ditto question for READ, WRITE, LOCK, LAYOUTGET,...
> 
> Do we have a problem for LOCKU, CLOSE and OPEN_DOWNGRADE? I think we end
> up just skipping recovery in those cases…

These are good points. I'm not sure *why* the sp4_mach_cred config would change, but I suppose it could, and yes, in my current implementation there is no way to back out once a rpc task is invoked.  Any thoughts on this?  I can see two ways of handling this:
 - restarting all tasks (with rpc_clnt and rpc_cred appropriately changed) when a change in sp4_mach_cred config is detected.
 - somehow having the RPC layer call out to check SP4 status before sending each RPC.

-dros

> 
> -- 
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDE1OjUwICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiBPbiBKdWwgMTgsIDIwMTMsIGF0IDExOjIxIEFNLCAiTXlrbGVidXN0LCBUcm9uZCIgPFRyb25k
Lk15a2xlYnVzdEBuZXRhcHAuY29tPg0KPiAgd3JvdGU6DQo+IA0KPiA+IE9uIFRodSwgMjAxMy0w
Ny0xOCBhdCAxNToxMCArMDAwMCwgQWRhbXNvbiwgRHJvcyB3cm90ZToNCj4gPj4gT24gSnVsIDE4
LCAyMDEzLCBhdCAxMDo1NyBBTSwgIk15a2xlYnVzdCwgVHJvbmQiIDxUcm9uZC5NeWtsZWJ1c3RA
bmV0YXBwLmNvbT4gd3JvdGU6DQo+ID4+IA0KPiA+Pj4gT24gVGh1LCAyMDEzLTA3LTE4IGF0IDE0
OjQ2ICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0KPiA+Pj4+IEhleSwNCj4gPj4+PiANCj4g
Pj4+PiBJIGhhdmUgYSBtb3N0bHkgZnVuY3Rpb25hbCBjbGllbnQtc2lkZSBpbXBsZW1lbnRhdGlv
biBvZiBTUDRfTUFDSF9DUkVEISBJdCBzdGlsbCBuZWVkcyBhIGxvdCBvZiBjbGVhbnVwIGFuZCB0
ZXN0aW5nLg0KPiA+Pj4+IA0KPiA+Pj4+IEkgaGF2ZSBvbmUgc3R5bGUgaXNzdWUgdGhhdCBJIHdh
bnQgdG8gcnVuIGJ5IHRoZSBsaXN0IGJlZm9yZSBJIHBvc3QgYSBwYXRjaHNldDoNCj4gPj4+PiAN
Cj4gPj4+PiBTbywgU1A0X01BQ0hfQ1JFRCBuZWdvdGlhdGVzIHR3byBiaXRtYXBzIGluIHRoZSBF
WENIQU5HRV9JRCAob25lICJlbmZvcmNlIiwgb25lICJhbGxvdyIpIGZvciBzdGF0ZSBwcm90ZWN0
aW9uLiBUaGVzZSBiaXRtYXBzIGFyZSBpbmRleGVkIGJ5IHRoZSBORlN2NCBvcGVyYXRpb24gbnVt
YmVyLiAgVGhlIHN0YXRlIHByb3RlY3QgY2hlY2sgbXVzdCBoYXBwZW4gaW4gdGhlIG5mczRwcm9j
LmMgbGF5ZXIgKG9yIGJlZm9yZSksIHJpZ2h0IGJlZm9yZSB3ZSBjYWxsIHJwY19jYWxsX3N5bmMg
b3IgZXF1aXZhbGVudCwgc28gdGhhdCBpdCBjYW4gc2VsZWN0IHRoZSByaWdodCBjcmVkIGFuZCBy
cGNfY2xpZW50Lg0KPiA+Pj4+IA0KPiA+Pj4+IEhlcmUncyB0aGUgcHJvYmxlbTogd2UgZG9uJ3Qg
a25vdyB3aGF0IG9wZXJhdGlvbnMgKG9wY29kZXMpIGFyZSBhY3R1YWxseSBpbiBhIGNvbXBvdW5k
IHVudGlsIHRoZSBYRFIgZW5jb2RlIGNhbGxiYWNrIGlzIGNhbGxlZC4gIFRoZSBycGNfcHJvY2lu
Zm8gYXJyYXkgZG9lc24ndCBoYXZlIHRoaXMgbWFwcGluZyAtIGluIGZhY3QsIGl0IG9ubHkgbGl2
ZXMgaW4gdGhlIHhkciBlbmNvZGUgbGF5ZXIuDQo+ID4+Pj4gDQo+ID4+Pj4gT25lIGFwcHJvYWNo
IGlzIHRvIGltbWVkaWF0ZWx5IHRyYW5zbGF0ZSB0aGUgb3Bjb2RlIGJpdG1hcHMgdG8gIm5mczQg
cHJvY2VkdXJlIGluZGV4IiBiaXRtYXBzLCBpbmRleGluZyBpbnRvIHRoZSBycGNfcHJvY2luZm8g
YXJyYXkuICBUaGlzIHdvdWxkIG1lYW4gdGhlcmUgaXMgYSBzZWNvbmQgbWFwcGluZyBvZiBORlN2
NCBwcm9jZWR1cmUgLT4gb3Bjb2RlcyB0aGF0IG11c3QgYmUgdXBkYXRlZCB3aGVuIGFuIFhEUiBl
bmNvZGUgY2FsbGJhY2sgaXMgY2hhbmdlZC4NCj4gPj4+PiANCj4gPj4+PiBBbm90aGVyIGFwcHJv
YWNoIHdvdWxkIGJlIHRvIGFkZCBhIGNhbGxiYWNrIHRvIHRoZSBYRFIgYXBpIHNvIHdlIGNvdWxk
ICJhc2siIGl0IGlmIGFuIE5GU3Y0IHByb2NlZHVyZSBjb250YWlucyBhbnkgb2YgdGhlIG9wY29k
ZXMgaW4gYSBiaXRtYXAuIFRoZSBuaWNlIHRoaW5nIGFib3V0IHRoaXMgYXBwcm9hY2ggaXMgdGhh
dCB0aGUgbWFwcGluZyBvZiBwcm9jZWR1cmUgdG8gb3Bjb2RlcyB3aXRoaW4gd2lsbCBsaXZlIHJp
Z2h0IG5leHQgdG8gdGhlIFhEUiBlbmNvZGUgY2FsbGJhY2sgYW5kIG1ha2UgaXQgb2J2aW91cyB0
aGF0IGJvdGggbmVlZCB0byBiZSBjaGFuZ2VkIHRvZ2V0aGVyLg0KPiA+Pj4+IA0KPiA+Pj4+IEkg
c3VwcG9zZSBJJ20gbGVhbmluZyB0b3dhcmQgYSBjb21iaW5hdGlvbiBvZiBib3RoIG9mIHRoZXNl
IGFwcHJvYWNoZXMgLSBrZWVwIHRoZSBtYXBwaW5nIGluIFhEUi1sYW5kIGFuZCB0cmFuc2xhdGUg
dGhlIGJpdG1hcHMgaW1tZWRpYXRlbHkgb24gbmVnYXRpb24gZm9yIGZhc3QgbG9va3VwcyBkdXJp
bmcgbm9ybWFsIG9wZXJhdGlvbi4NCj4gPj4+PiANCj4gPj4+PiBDb21tZW50cz8gQW0gSSBtaXNz
aW5nIHNvbWV0aGluZz8NCj4gPj4+IA0KPiA+Pj4gSSdtIG5vdCBzdXJlIHRoYXQgSSB1bmRlcnN0
YW5kLiBXZSBkb24ndCBkbyBkeW5hbWljIGNyZWF0aW9uIG9mDQo+ID4+PiBjb21wb3VuZHM6IHdl
IHByZXR0eSBtdWNoIGtub3cgaW4gdGhlIG5mczRfcHJvY18qIHJvdXRpbmUgd2hhdCB0aGUgbWFp
bg0KPiA+Pj4gb3BlcmF0aW9uIGlzICh0aGUgb25lIGV4Y2VwdGlvbiBiZWluZyBDTE9TRS9PUEVO
X0RPV05HUkFERSkuIFNvIHdoeQ0KPiA+Pj4gY2FuJ3Qgd2Ugd29yayBvdXQgdGhlIHByb3RlY3Rp
b24gc3RhdHVzIGF0IHRoYXQgdGltZT8gSXMgdGhlIHdvcnJ5IHRoYXQNCj4gPj4+IHRoZSBzZXJ2
ZXIgbWlnaHQgcmVib290IGFuZCBjb21lIGJhY2sgd2l0aCBkaWZmZXJlbnQgTUFDSF9DUkVEDQo+
ID4+PiBwcm90ZWN0aW9ucz8NCj4gPj4gDQo+ID4+IFN1cmUsIHdlIGtub3cgd2hhdCB0aGUgbWFp
biBvcGVyYXRpb24gaXMgYW5kIHdlIGtub3cgd2hhdCBvcGVyYXRpb25zIHdpbGwgZW5kIHVwIGlu
IGEgY29tcG91bmQgLSB3ZSBjYW4ganVzdCBsb29rIGF0IHRoZSB4ZHIgZW5jb2Rlci4gIE15IHF1
ZXN0aW9uIGlzIHNpbXBseSBvbmUgb2Ygc3R5bGUuIERvIHdlIHdhbnQgdG8gaGF2ZSBlYWNoIG5m
czRwcm9jIHByb2NlZHVyZSB0byBoYXZlIGEgbGlzdCBvZiBvcGVyYXRpb25zIHRoYXQgbXVzdCBi
ZSB1cGRhdGVkIGlmIGFuIHhkciBlbmNvZGVyIGlzIHVwZGF0ZWQ/IEknbSBmaW5lIHdpdGggZG9p
bmcgaXQgdGhpcyB3YXksIGl0IGp1c3Qgc2VlbWVkIHdyb25nIHRvIGhhdmUgdGhlIHNhbWUgbWFw
cGluZyBpbiB0d28gZGlmZmVyZW50IHBsYWNlcy4gVGhpcyBpcyB3aHkgSSdtIGFza2luZyA7KQ0K
PiA+PiANCj4gPj4gQWxzbywgd2UgY2FuJ3QganVzdCB1c2UgdGhlICJtYWluIiBvcGVyYXRpb24s
IHdlIG11c3QgY2hlY2sgZXZlcnkgb3BlcmF0aW9uIHdpdGhpbiB0aGUgY29tcG91bmQgYW5kIGlm
IGFueSBhcmUgcmVxdWlyZWQgdG8gdXNlIFNQNCwgdGhlbiB0aGUgd2hvbGUgY29tcG91bmQgZG9l
cy4gSSBkb24ndCBuZWNlc3NhcmlseSBrbm93IHdoeSBhIHNlcnZlciB3b3VsZCBkbyB0aGlzLCBi
dXQgaWYgd2UgZm9sbG93IHRoZSBzcGVjIGFuZCBhIHNlcnZlciBpbmZvcm1zIHRoZSBjbGllbnQg
dGhhdCBTRVRBVFRSIG11c3QgdXNlIHN0YXRlIHByb3RlY3Rpb24sIHRoZW4gYSBXUklURSB3aXRo
IHBvc3Qtb3AgU0VUQVRUUiBtdXN0IHVzZSB0aGUgc3RhdGUgcHJvdGVjdGlvbiBldmVuIHRob3Vn
aCBXUklURSBkb2Vzbid0IG5lZWQgaXQuDQo+ID4gDQo+ID4gV2Ugb25seSBuZWVkIHRvIGNoZWNr
IGV2ZXJ5IF9zdGF0ZWZ1bF8gb3BlcmF0aW9uLCByaWdodD8gSSBjYW4ndCB0aGluaw0KPiA+IG9m
IGFueSBjb21wb3VuZHMgd2l0aCBtb3JlIHRoYW4gMSBzdGF0ZWZ1bCBvcGVyYXRpb24uIFdlJ3Zl
IGRlbGliZXJhdGVseQ0KPiA+IGF2b2lkZWQgdGhvc2UgZHVlIHRvIHRoZSBwcm9ibGVtcyB0aGF0
IGFyaXNlIHdoZW4geW91IGdldCBhDQo+ID4gTkZTNEVSUl9ERUxBWSBvciBzb21ldGhpbmcgZXF1
aXZhbGVudCBhdCB0aGUgd3JvbmcgbW9tZW50Lg0KPiANCj4gVGhhdCBtYXkgYmUgdGhlIGludGVu
dGlvbiwgYnV0IEkgc2VlIG5vIHN1Y2ggbGltaXRhdGlvbiBpbiB0aGUgc3BlYy4gSXQncyBteSB1
bmRlcnN0YW5kaW5nIHRoYXQgdGhlIHNlcnZlciBjb3VsZCBzcGVjaWZ5IGFueSBvcGVyYXRpb24g
YXMgTVVTVCBvciBNQVkgdXNlIHN0YXRlIHByb3RlY3Rpb24uDQoNClJpZ2h0LCBidXQgd2hhdCB3
b3VsZCBzdGF0ZSBwcm90ZWN0aW9uIG1lYW4gZm9yIGEgc3RhdGVsZXNzIG9wZXJhdGlvbg0Kc3Vj
aCBhcyBHRVRBVFRSPyBUaGF0IG9wZXJhdGlvbiBpc24ndCBldmVuIGFsbG93ZWQgdG8gcmV0dXJu
DQpORlM0RVJSX1dST05HX0NSRUQsIHNvIEkgZG9uJ3Qgc2VlIGhvdyBpdCB3b3VsZCBhcHBseS4N
Cg0KPiA+PiBJJ20gbm90IGZvbGxvd2luZyB0aGUgcmVib290IHF1ZXN0aW9uIC0gdGhhdCBjYXNl
IHNob3VsZCBiZSBoYW5kbGVkIGp1c3QgZmluZS4gT25jZSB0aGUgRVhDSEFOR0VfSUQgaGFwcGVu
cywgdGhlIGNscCBoYXMgdHdvIGJpdG1hcHMgKGVuZm9yY2UgYW5kIGFsbG93KSB0aGF0IHJlZmxl
Y3QgdGhlIGN1cnJlbnQgbW9kZSBvZiBTUDRfTUFDSF9DUkVELg0KPiA+IA0KPiA+IEknbSB0aGlu
a2luZyBzb21ldGhpbmcgYWxvbmcgdGhlIGZvbGxvd2luZyBzY2VuYXJpbzoNCj4gPiANCj4gPiBB
IHByb2Nlc3MgcXVldWVzIHVwIGFuIE9QRU4uIFRoZSBzZXJ2ZXIgcmVwbGllcyBORlM0RVJSX0JB
RF9TRVNTSU9OIGFuZA0KPiA+IGxhdGVyIHJlcGxpZXMgd2l0aCBORlM0RVJSX1NUQUxFX0NMSUVO
VElELiBTbyB3ZSBzdGFydCByZWJvb3QgcmVjb3ZlcnksDQo+ID4gYW5kIHRoZSBPUEVOIGdldHMg
cXVldWVkIHdhaXRpbmcgZm9yIGEgc2xvdC4NCj4gPiBXZSB0aGVuIHNlbmQgYSBuZXcgRVhDSEFO
R0VfSUQsIGFuZCB0aGUgc2VydmVyIHJlcGxpZXMgd2l0aCBhDQo+ID4gX2RpZmZlcmVudF8gTUFD
SF9DUkVEIHByb3RlY3Rpb24gZm9yIE9QRU4uDQo+ID4gDQo+ID4gVGhlIHByb2JsZW0gYWJvdmUg
aXMgdGhhdCB0aGUgT1BFTiBoYXMgYWxyZWFkeSBiZWVuIHN0YXJ0ZWQsIGFuZCB3ZSd2ZQ0KPiA+
IGFscmVhZHkgYXNzaWduZWQgaXQgYSBjcmVkZW50aWFsLiBIb3cgZG8gd2UgaGFuZGxlIHRoYXQ/
DQo+ID4gRGl0dG8gcXVlc3Rpb24gZm9yIFJFQUQsIFdSSVRFLCBMT0NLLCBMQVlPVVRHRVQsLi4u
DQo+ID4gDQo+ID4gRG8gd2UgaGF2ZSBhIHByb2JsZW0gZm9yIExPQ0tVLCBDTE9TRSBhbmQgT1BF
Tl9ET1dOR1JBREU/IEkgdGhpbmsgd2UgZW5kDQo+ID4gdXAganVzdCBza2lwcGluZyByZWNvdmVy
eSBpbiB0aG9zZSBjYXNlc+KApg0KPiANCj4gVGhlc2UgYXJlIGdvb2QgcG9pbnRzLiBJJ20gbm90
IHN1cmUgKndoeSogdGhlIHNwNF9tYWNoX2NyZWQgY29uZmlnIHdvdWxkIGNoYW5nZSwgYnV0IEkg
c3VwcG9zZSBpdCBjb3VsZCwgYW5kIHllcywgaW4gbXkgY3VycmVudCBpbXBsZW1lbnRhdGlvbiB0
aGVyZSBpcyBubyB3YXkgdG8gYmFjayBvdXQgb25jZSBhIHJwYyB0YXNrIGlzIGludm9rZWQuICBB
bnkgdGhvdWdodHMgb24gdGhpcz8gIEkgY2FuIHNlZSB0d28gd2F5cyBvZiBoYW5kbGluZyB0aGlz
Og0KPiAgLSByZXN0YXJ0aW5nIGFsbCB0YXNrcyAod2l0aCBycGNfY2xudCBhbmQgcnBjX2NyZWQg
YXBwcm9wcmlhdGVseSBjaGFuZ2VkKSB3aGVuIGEgY2hhbmdlIGluIHNwNF9tYWNoX2NyZWQgY29u
ZmlnIGlzIGRldGVjdGVkLg0KPiAgLSBzb21laG93IGhhdmluZyB0aGUgUlBDIGxheWVyIGNhbGwg
b3V0IHRvIGNoZWNrIFNQNCBzdGF0dXMgYmVmb3JlIHNlbmRpbmcgZWFjaCBSUEMuDQoNCkZvciBu
b3csIEkgdGhpbmsgd2Ugc2hvdWxkIG1ha2Ugc3VyZSB0aGF0IHdlIGhhbmRsZSBORlM0RVJSX1dS
T05HX0NSRUQNCmNvcnJlY3RseSBpbiB0aGVzZSBjYXNlcy4gSSBkb24ndCB0aGluayB3ZSBldmVy
IHdhbnQgdG8gaGFuZGxlIHRoYXQgaW4NCm5mczRfYXN5bmNfaGFuZGxlX2Vycm9yLCBidXQgd2Ug
c2hvdWxkIGRlZmluaXRlbHkgaGFuZGxlIGl0IGluDQpuZnM0X2hhbmRsZV9leGNlcHRpb24oKSwg
c2luY2UgdGhhdCBjYW4gZHJpdmUgYSBmdWxsIHJldHJ5IG9mIHRoZSBSUEMNCmNhbGwuDQoNCi0t
IA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCg0KTmV0QXBw
DQpUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5jb20NCg==

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Adamson, Dros]

Ok, I'll have the client only ask for state protection on operations that support NFS4ERR_WRONG_CRED - and I'll do the SP4_MACH_CRED checks only on these operations and only if they aren't already using the machine cred already. This simplifies my original problem - now I can just use the "main" opcode.

I'll also add error handlers for NFS4ERR_WRONG_CRED.

Thanks!

-dros

On Jul 18, 2013, at 1:08 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:

> On Thu, 2013-07-18 at 15:50 +0000, Adamson, Dros wrote:
>> On Jul 18, 2013, at 11:21 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
>> wrote:
>> 
>>> On Thu, 2013-07-18 at 15:10 +0000, Adamson, Dros wrote:
>>>> On Jul 18, 2013, at 10:57 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>>> 
>>>>> On Thu, 2013-07-18 at 14:46 +0000, Adamson, Dros wrote:
>>>>>> Hey,
>>>>>> 
>>>>>> I have a mostly functional client-side implementation of SP4_MACH_CRED! It still needs a lot of cleanup and testing.
>>>>>> 
>>>>>> I have one style issue that I want to run by the list before I post a patchset:
>>>>>> 
>>>>>> So, SP4_MACH_CRED negotiates two bitmaps in the EXCHANGE_ID (one "enforce", one "allow") for state protection. These bitmaps are indexed by the NFSv4 operation number.  The state protect check must happen in the nfs4proc.c layer (or before), right before we call rpc_call_sync or equivalent, so that it can select the right cred and rpc_client.
>>>>>> 
>>>>>> Here's the problem: we don't know what operations (opcodes) are actually in a compound until the XDR encode callback is called.  The rpc_procinfo array doesn't have this mapping - in fact, it only lives in the xdr encode layer.
>>>>>> 
>>>>>> One approach is to immediately translate the opcode bitmaps to "nfs4 procedure index" bitmaps, indexing into the rpc_procinfo array.  This would mean there is a second mapping of NFSv4 procedure -> opcodes that must be updated when an XDR encode callback is changed.
>>>>>> 
>>>>>> Another approach would be to add a callback to the XDR api so we could "ask" it if an NFSv4 procedure contains any of the opcodes in a bitmap. The nice thing about this approach is that the mapping of procedure to opcodes within will live right next to the XDR encode callback and make it obvious that both need to be changed together.
>>>>>> 
>>>>>> I suppose I'm leaning toward a combination of both of these approaches - keep the mapping in XDR-land and translate the bitmaps immediately on negation for fast lookups during normal operation.
>>>>>> 
>>>>>> Comments? Am I missing something?
>>>>> 
>>>>> I'm not sure that I understand. We don't do dynamic creation of
>>>>> compounds: we pretty much know in the nfs4_proc_* routine what the main
>>>>> operation is (the one exception being CLOSE/OPEN_DOWNGRADE). So why
>>>>> can't we work out the protection status at that time? Is the worry that
>>>>> the server might reboot and come back with different MACH_CRED
>>>>> protections?
>>>> 
>>>> Sure, we know what the main operation is and we know what operations will end up in a compound - we can just look at the xdr encoder.  My question is simply one of style. Do we want to have each nfs4proc procedure to have a list of operations that must be updated if an xdr encoder is updated? I'm fine with doing it this way, it just seemed wrong to have the same mapping in two different places. This is why I'm asking ;)
>>>> 
>>>> Also, we can't just use the "main" operation, we must check every operation within the compound and if any are required to use SP4, then the whole compound does. I don't necessarily know why a server would do this, but if we follow the spec and a server informs the client that SETATTR must use state protection, then a WRITE with post-op SETATTR must use the state protection even though WRITE doesn't need it.
>>> 
>>> We only need to check every _stateful_ operation, right? I can't think
>>> of any compounds with more than 1 stateful operation. We've deliberately
>>> avoided those due to the problems that arise when you get a
>>> NFS4ERR_DELAY or something equivalent at the wrong moment.
>> 
>> That may be the intention, but I see no such limitation in the spec. It's my understanding that the server could specify any operation as MUST or MAY use state protection.
> 
> Right, but what would state protection mean for a stateless operation
> such as GETATTR? That operation isn't even allowed to return
> NFS4ERR_WRONG_CRED, so I don't see how it would apply.
> 
>>>> I'm not following the reboot question - that case should be handled just fine. Once the EXCHANGE_ID happens, the clp has two bitmaps (enforce and allow) that reflect the current mode of SP4_MACH_CRED.
>>> 
>>> I'm thinking something along the following scenario:
>>> 
>>> A process queues up an OPEN. The server replies NFS4ERR_BAD_SESSION and
>>> later replies with NFS4ERR_STALE_CLIENTID. So we start reboot recovery,
>>> and the OPEN gets queued waiting for a slot.
>>> We then send a new EXCHANGE_ID, and the server replies with a
>>> _different_ MACH_CRED protection for OPEN.
>>> 
>>> The problem above is that the OPEN has already been started, and we've
>>> already assigned it a credential. How do we handle that?
>>> Ditto question for READ, WRITE, LOCK, LAYOUTGET,...
>>> 
>>> Do we have a problem for LOCKU, CLOSE and OPEN_DOWNGRADE? I think we end
>>> up just skipping recovery in those cases…
>> 
>> These are good points. I'm not sure *why* the sp4_mach_cred config would change, but I suppose it could, and yes, in my current implementation there is no way to back out once a rpc task is invoked.  Any thoughts on this?  I can see two ways of handling this:
>> - restarting all tasks (with rpc_clnt and rpc_cred appropriately changed) when a change in sp4_mach_cred config is detected.
>> - somehow having the RPC layer call out to check SP4 status before sending each RPC.
> 
> For now, I think we should make sure that we handle NFS4ERR_WRONG_CRED
> correctly in these cases. I don't think we ever want to handle that in
> nfs4_async_handle_error, but we should definitely handle it in
> nfs4_handle_exception(), since that can drive a full retry of the RPC
> call.
> 
> -- 
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Adamson, Dros]

Only supporting operations that have the error code NFS4ERR_WRONG_CRED seems to be wrong.  Operations like BIND_CONN_TO_SESSION don't support don't support this error code, but are explicitly mentioned in SP4_MACH_CRED sections of the spec.

I'll continue my work described below modulo limiting to NFS4ERR_WRONG_CRED enabled OPs and see how that looks...

-dros

On Jul 18, 2013, at 2:13 PM, "Adamson, Dros" <Weston.Adamson@netapp.com> wrote:

> Ok, I'll have the client only ask for state protection on operations that support NFS4ERR_WRONG_CRED - and I'll do the SP4_MACH_CRED checks only on these operations and only if they aren't already using the machine cred already. This simplifies my original problem - now I can just use the "main" opcode.
> 
> I'll also add error handlers for NFS4ERR_WRONG_CRED.
> 
> Thanks!
> 
> -dros
> 
> On Jul 18, 2013, at 1:08 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
> 
>> On Thu, 2013-07-18 at 15:50 +0000, Adamson, Dros wrote:
>>> On Jul 18, 2013, at 11:21 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
>>> wrote:
>>> 
>>>> On Thu, 2013-07-18 at 15:10 +0000, Adamson, Dros wrote:
>>>>> On Jul 18, 2013, at 10:57 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>>>> 
>>>>>> On Thu, 2013-07-18 at 14:46 +0000, Adamson, Dros wrote:
>>>>>>> Hey,
>>>>>>> 
>>>>>>> I have a mostly functional client-side implementation of SP4_MACH_CRED! It still needs a lot of cleanup and testing.
>>>>>>> 
>>>>>>> I have one style issue that I want to run by the list before I post a patchset:
>>>>>>> 
>>>>>>> So, SP4_MACH_CRED negotiates two bitmaps in the EXCHANGE_ID (one "enforce", one "allow") for state protection. These bitmaps are indexed by the NFSv4 operation number.  The state protect check must happen in the nfs4proc.c layer (or before), right before we call rpc_call_sync or equivalent, so that it can select the right cred and rpc_client.
>>>>>>> 
>>>>>>> Here's the problem: we don't know what operations (opcodes) are actually in a compound until the XDR encode callback is called.  The rpc_procinfo array doesn't have this mapping - in fact, it only lives in the xdr encode layer.
>>>>>>> 
>>>>>>> One approach is to immediately translate the opcode bitmaps to "nfs4 procedure index" bitmaps, indexing into the rpc_procinfo array.  This would mean there is a second mapping of NFSv4 procedure -> opcodes that must be updated when an XDR encode callback is changed.
>>>>>>> 
>>>>>>> Another approach would be to add a callback to the XDR api so we could "ask" it if an NFSv4 procedure contains any of the opcodes in a bitmap. The nice thing about this approach is that the mapping of procedure to opcodes within will live right next to the XDR encode callback and make it obvious that both need to be changed together.
>>>>>>> 
>>>>>>> I suppose I'm leaning toward a combination of both of these approaches - keep the mapping in XDR-land and translate the bitmaps immediately on negation for fast lookups during normal operation.
>>>>>>> 
>>>>>>> Comments? Am I missing something?
>>>>>> 
>>>>>> I'm not sure that I understand. We don't do dynamic creation of
>>>>>> compounds: we pretty much know in the nfs4_proc_* routine what the main
>>>>>> operation is (the one exception being CLOSE/OPEN_DOWNGRADE). So why
>>>>>> can't we work out the protection status at that time? Is the worry that
>>>>>> the server might reboot and come back with different MACH_CRED
>>>>>> protections?
>>>>> 
>>>>> Sure, we know what the main operation is and we know what operations will end up in a compound - we can just look at the xdr encoder.  My question is simply one of style. Do we want to have each nfs4proc procedure to have a list of operations that must be updated if an xdr encoder is updated? I'm fine with doing it this way, it just seemed wrong to have the same mapping in two different places. This is why I'm asking ;)
>>>>> 
>>>>> Also, we can't just use the "main" operation, we must check every operation within the compound and if any are required to use SP4, then the whole compound does. I don't necessarily know why a server would do this, but if we follow the spec and a server informs the client that SETATTR must use state protection, then a WRITE with post-op SETATTR must use the state protection even though WRITE doesn't need it.
>>>> 
>>>> We only need to check every _stateful_ operation, right? I can't think
>>>> of any compounds with more than 1 stateful operation. We've deliberately
>>>> avoided those due to the problems that arise when you get a
>>>> NFS4ERR_DELAY or something equivalent at the wrong moment.
>>> 
>>> That may be the intention, but I see no such limitation in the spec. It's my understanding that the server could specify any operation as MUST or MAY use state protection.
>> 
>> Right, but what would state protection mean for a stateless operation
>> such as GETATTR? That operation isn't even allowed to return
>> NFS4ERR_WRONG_CRED, so I don't see how it would apply.
>> 
>>>>> I'm not following the reboot question - that case should be handled just fine. Once the EXCHANGE_ID happens, the clp has two bitmaps (enforce and allow) that reflect the current mode of SP4_MACH_CRED.
>>>> 
>>>> I'm thinking something along the following scenario:
>>>> 
>>>> A process queues up an OPEN. The server replies NFS4ERR_BAD_SESSION and
>>>> later replies with NFS4ERR_STALE_CLIENTID. So we start reboot recovery,
>>>> and the OPEN gets queued waiting for a slot.
>>>> We then send a new EXCHANGE_ID, and the server replies with a
>>>> _different_ MACH_CRED protection for OPEN.
>>>> 
>>>> The problem above is that the OPEN has already been started, and we've
>>>> already assigned it a credential. How do we handle that?
>>>> Ditto question for READ, WRITE, LOCK, LAYOUTGET,...
>>>> 
>>>> Do we have a problem for LOCKU, CLOSE and OPEN_DOWNGRADE? I think we end
>>>> up just skipping recovery in those cases…
>>> 
>>> These are good points. I'm not sure *why* the sp4_mach_cred config would change, but I suppose it could, and yes, in my current implementation there is no way to back out once a rpc task is invoked.  Any thoughts on this?  I can see two ways of handling this:
>>> - restarting all tasks (with rpc_clnt and rpc_cred appropriately changed) when a change in sp4_mach_cred config is detected.
>>> - somehow having the RPC layer call out to check SP4 status before sending each RPC.
>> 
>> For now, I think we should make sure that we handle NFS4ERR_WRONG_CRED
>> correctly in these cases. I don't think we ever want to handle that in
>> nfs4_async_handle_error, but we should definitely handle it in
>> nfs4_handle_exception(), since that can drive a full retry of the RPC
>> call.
>> 
>> -- 
>> Trond Myklebust
>> Linux NFS client maintainer
>> 
>> NetApp
>> Trond.Myklebust@netapp.com
>> www.netapp.com
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDE5OjQ5ICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiBPbmx5IHN1cHBvcnRpbmcgb3BlcmF0aW9ucyB0aGF0IGhhdmUgdGhlIGVycm9yIGNvZGUgTkZT
NEVSUl9XUk9OR19DUkVEIHNlZW1zIHRvIGJlIHdyb25nLiAgT3BlcmF0aW9ucyBsaWtlIEJJTkRf
Q09OTl9UT19TRVNTSU9OIGRvbid0IHN1cHBvcnQgZG9uJ3Qgc3VwcG9ydCB0aGlzIGVycm9yIGNv
ZGUsIGJ1dCBhcmUgZXhwbGljaXRseSBtZW50aW9uZWQgaW4gU1A0X01BQ0hfQ1JFRCBzZWN0aW9u
cyBvZiB0aGUgc3BlYy4NCg0KTG9va2luZyBhdCB0aGUgYWxsb3dlZCBlcnJvciByZXR1cm4gdmFs
dWVzIGZvciBCSU5EX0NPTk5fVE9fU0VTU0lPTiwgSSdtDQphdCBhIGxvc3MgdG8gZmlndXJlIG91
dCBleGFjdGx5IHdoYXQgaXQgc2hvdWxkIHJldHVybiBpbiB0aGlzIGNhc2UuIEkNCnN1c3BlY3Qg
dGhhdCB0aGUgbGFjayBvZiBhbiBORlM0RVJSX1dST05HX0NSRUQgaXMgYWN0dWFsbHkgYSBwcm90
b2NvbA0KYnVnLg0KDQpUaW1lIHRvIGdvIGJhY2sgdG8gdGhlIGlldGYgbWFpbGluZyBsaXN0Li4u
DQoNCj4gSSdsbCBjb250aW51ZSBteSB3b3JrIGRlc2NyaWJlZCBiZWxvdyBtb2R1bG8gbGltaXRp
bmcgdG8gTkZTNEVSUl9XUk9OR19DUkVEIGVuYWJsZWQgT1BzIGFuZCBzZWUgaG93IHRoYXQgbG9v
a3MuLi4NCj4gDQo+IC1kcm9zDQo+IA0KPiBPbiBKdWwgMTgsIDIwMTMsIGF0IDI6MTMgUE0sICJB
ZGFtc29uLCBEcm9zIiA8V2VzdG9uLkFkYW1zb25AbmV0YXBwLmNvbT4gd3JvdGU6DQo+IA0KPiA+
IE9rLCBJJ2xsIGhhdmUgdGhlIGNsaWVudCBvbmx5IGFzayBmb3Igc3RhdGUgcHJvdGVjdGlvbiBv
biBvcGVyYXRpb25zIHRoYXQgc3VwcG9ydCBORlM0RVJSX1dST05HX0NSRUQgLSBhbmQgSSdsbCBk
byB0aGUgU1A0X01BQ0hfQ1JFRCBjaGVja3Mgb25seSBvbiB0aGVzZSBvcGVyYXRpb25zIGFuZCBv
bmx5IGlmIHRoZXkgYXJlbid0IGFscmVhZHkgdXNpbmcgdGhlIG1hY2hpbmUgY3JlZCBhbHJlYWR5
LiBUaGlzIHNpbXBsaWZpZXMgbXkgb3JpZ2luYWwgcHJvYmxlbSAtIG5vdyBJIGNhbiBqdXN0IHVz
ZSB0aGUgIm1haW4iIG9wY29kZS4NCj4gPiANCj4gPiBJJ2xsIGFsc28gYWRkIGVycm9yIGhhbmRs
ZXJzIGZvciBORlM0RVJSX1dST05HX0NSRUQuDQo+ID4gDQo+ID4gVGhhbmtzIQ0KPiA+IA0KPiA+
IC1kcm9zDQo+ID4gDQo+ID4gT24gSnVsIDE4LCAyMDEzLCBhdCAxOjA4IFBNLCAiTXlrbGVidXN0
LCBUcm9uZCIgPFRyb25kLk15a2xlYnVzdEBuZXRhcHAuY29tPiB3cm90ZToNCj4gPiANCj4gPj4g
T24gVGh1LCAyMDEzLTA3LTE4IGF0IDE1OjUwICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiA+Pj4gT24gSnVsIDE4LCAyMDEzLCBhdCAxMToyMSBBTSwgIk15a2xlYnVzdCwgVHJvbmQiIDxU
cm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbT4NCj4gPj4+IHdyb3RlOg0KPiA+Pj4gDQo+ID4+Pj4g
T24gVGh1LCAyMDEzLTA3LTE4IGF0IDE1OjEwICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiA+Pj4+PiBPbiBKdWwgMTgsIDIwMTMsIGF0IDEwOjU3IEFNLCAiTXlrbGVidXN0LCBUcm9uZCIg
PFRyb25kLk15a2xlYnVzdEBuZXRhcHAuY29tPiB3cm90ZToNCj4gPj4+Pj4gDQo+ID4+Pj4+PiBP
biBUaHUsIDIwMTMtMDctMTggYXQgMTQ6NDYgKzAwMDAsIEFkYW1zb24sIERyb3Mgd3JvdGU6DQo+
ID4+Pj4+Pj4gSGV5LA0KPiA+Pj4+Pj4+IA0KPiA+Pj4+Pj4+IEkgaGF2ZSBhIG1vc3RseSBmdW5j
dGlvbmFsIGNsaWVudC1zaWRlIGltcGxlbWVudGF0aW9uIG9mIFNQNF9NQUNIX0NSRUQhIEl0IHN0
aWxsIG5lZWRzIGEgbG90IG9mIGNsZWFudXAgYW5kIHRlc3RpbmcuDQo+ID4+Pj4+Pj4gDQo+ID4+
Pj4+Pj4gSSBoYXZlIG9uZSBzdHlsZSBpc3N1ZSB0aGF0IEkgd2FudCB0byBydW4gYnkgdGhlIGxp
c3QgYmVmb3JlIEkgcG9zdCBhIHBhdGNoc2V0Og0KPiA+Pj4+Pj4+IA0KPiA+Pj4+Pj4+IFNvLCBT
UDRfTUFDSF9DUkVEIG5lZ290aWF0ZXMgdHdvIGJpdG1hcHMgaW4gdGhlIEVYQ0hBTkdFX0lEIChv
bmUgImVuZm9yY2UiLCBvbmUgImFsbG93IikgZm9yIHN0YXRlIHByb3RlY3Rpb24uIFRoZXNlIGJp
dG1hcHMgYXJlIGluZGV4ZWQgYnkgdGhlIE5GU3Y0IG9wZXJhdGlvbiBudW1iZXIuICBUaGUgc3Rh
dGUgcHJvdGVjdCBjaGVjayBtdXN0IGhhcHBlbiBpbiB0aGUgbmZzNHByb2MuYyBsYXllciAob3Ig
YmVmb3JlKSwgcmlnaHQgYmVmb3JlIHdlIGNhbGwgcnBjX2NhbGxfc3luYyBvciBlcXVpdmFsZW50
LCBzbyB0aGF0IGl0IGNhbiBzZWxlY3QgdGhlIHJpZ2h0IGNyZWQgYW5kIHJwY19jbGllbnQuDQo+
ID4+Pj4+Pj4gDQo+ID4+Pj4+Pj4gSGVyZSdzIHRoZSBwcm9ibGVtOiB3ZSBkb24ndCBrbm93IHdo
YXQgb3BlcmF0aW9ucyAob3Bjb2RlcykgYXJlIGFjdHVhbGx5IGluIGEgY29tcG91bmQgdW50aWwg
dGhlIFhEUiBlbmNvZGUgY2FsbGJhY2sgaXMgY2FsbGVkLiAgVGhlIHJwY19wcm9jaW5mbyBhcnJh
eSBkb2Vzbid0IGhhdmUgdGhpcyBtYXBwaW5nIC0gaW4gZmFjdCwgaXQgb25seSBsaXZlcyBpbiB0
aGUgeGRyIGVuY29kZSBsYXllci4NCj4gPj4+Pj4+PiANCj4gPj4+Pj4+PiBPbmUgYXBwcm9hY2gg
aXMgdG8gaW1tZWRpYXRlbHkgdHJhbnNsYXRlIHRoZSBvcGNvZGUgYml0bWFwcyB0byAibmZzNCBw
cm9jZWR1cmUgaW5kZXgiIGJpdG1hcHMsIGluZGV4aW5nIGludG8gdGhlIHJwY19wcm9jaW5mbyBh
cnJheS4gIFRoaXMgd291bGQgbWVhbiB0aGVyZSBpcyBhIHNlY29uZCBtYXBwaW5nIG9mIE5GU3Y0
IHByb2NlZHVyZSAtPiBvcGNvZGVzIHRoYXQgbXVzdCBiZSB1cGRhdGVkIHdoZW4gYW4gWERSIGVu
Y29kZSBjYWxsYmFjayBpcyBjaGFuZ2VkLg0KPiA+Pj4+Pj4+IA0KPiA+Pj4+Pj4+IEFub3RoZXIg
YXBwcm9hY2ggd291bGQgYmUgdG8gYWRkIGEgY2FsbGJhY2sgdG8gdGhlIFhEUiBhcGkgc28gd2Ug
Y291bGQgImFzayIgaXQgaWYgYW4gTkZTdjQgcHJvY2VkdXJlIGNvbnRhaW5zIGFueSBvZiB0aGUg
b3Bjb2RlcyBpbiBhIGJpdG1hcC4gVGhlIG5pY2UgdGhpbmcgYWJvdXQgdGhpcyBhcHByb2FjaCBp
cyB0aGF0IHRoZSBtYXBwaW5nIG9mIHByb2NlZHVyZSB0byBvcGNvZGVzIHdpdGhpbiB3aWxsIGxp
dmUgcmlnaHQgbmV4dCB0byB0aGUgWERSIGVuY29kZSBjYWxsYmFjayBhbmQgbWFrZSBpdCBvYnZp
b3VzIHRoYXQgYm90aCBuZWVkIHRvIGJlIGNoYW5nZWQgdG9nZXRoZXIuDQo+ID4+Pj4+Pj4gDQo+
ID4+Pj4+Pj4gSSBzdXBwb3NlIEknbSBsZWFuaW5nIHRvd2FyZCBhIGNvbWJpbmF0aW9uIG9mIGJv
dGggb2YgdGhlc2UgYXBwcm9hY2hlcyAtIGtlZXAgdGhlIG1hcHBpbmcgaW4gWERSLWxhbmQgYW5k
IHRyYW5zbGF0ZSB0aGUgYml0bWFwcyBpbW1lZGlhdGVseSBvbiBuZWdhdGlvbiBmb3IgZmFzdCBs
b29rdXBzIGR1cmluZyBub3JtYWwgb3BlcmF0aW9uLg0KPiA+Pj4+Pj4+IA0KPiA+Pj4+Pj4+IENv
bW1lbnRzPyBBbSBJIG1pc3Npbmcgc29tZXRoaW5nPw0KPiA+Pj4+Pj4gDQo+ID4+Pj4+PiBJJ20g
bm90IHN1cmUgdGhhdCBJIHVuZGVyc3RhbmQuIFdlIGRvbid0IGRvIGR5bmFtaWMgY3JlYXRpb24g
b2YNCj4gPj4+Pj4+IGNvbXBvdW5kczogd2UgcHJldHR5IG11Y2gga25vdyBpbiB0aGUgbmZzNF9w
cm9jXyogcm91dGluZSB3aGF0IHRoZSBtYWluDQo+ID4+Pj4+PiBvcGVyYXRpb24gaXMgKHRoZSBv
bmUgZXhjZXB0aW9uIGJlaW5nIENMT1NFL09QRU5fRE9XTkdSQURFKS4gU28gd2h5DQo+ID4+Pj4+
PiBjYW4ndCB3ZSB3b3JrIG91dCB0aGUgcHJvdGVjdGlvbiBzdGF0dXMgYXQgdGhhdCB0aW1lPyBJ
cyB0aGUgd29ycnkgdGhhdA0KPiA+Pj4+Pj4gdGhlIHNlcnZlciBtaWdodCByZWJvb3QgYW5kIGNv
bWUgYmFjayB3aXRoIGRpZmZlcmVudCBNQUNIX0NSRUQNCj4gPj4+Pj4+IHByb3RlY3Rpb25zPw0K
PiA+Pj4+PiANCj4gPj4+Pj4gU3VyZSwgd2Uga25vdyB3aGF0IHRoZSBtYWluIG9wZXJhdGlvbiBp
cyBhbmQgd2Uga25vdyB3aGF0IG9wZXJhdGlvbnMgd2lsbCBlbmQgdXAgaW4gYSBjb21wb3VuZCAt
IHdlIGNhbiBqdXN0IGxvb2sgYXQgdGhlIHhkciBlbmNvZGVyLiAgTXkgcXVlc3Rpb24gaXMgc2lt
cGx5IG9uZSBvZiBzdHlsZS4gRG8gd2Ugd2FudCB0byBoYXZlIGVhY2ggbmZzNHByb2MgcHJvY2Vk
dXJlIHRvIGhhdmUgYSBsaXN0IG9mIG9wZXJhdGlvbnMgdGhhdCBtdXN0IGJlIHVwZGF0ZWQgaWYg
YW4geGRyIGVuY29kZXIgaXMgdXBkYXRlZD8gSSdtIGZpbmUgd2l0aCBkb2luZyBpdCB0aGlzIHdh
eSwgaXQganVzdCBzZWVtZWQgd3JvbmcgdG8gaGF2ZSB0aGUgc2FtZSBtYXBwaW5nIGluIHR3byBk
aWZmZXJlbnQgcGxhY2VzLiBUaGlzIGlzIHdoeSBJJ20gYXNraW5nIDspDQo+ID4+Pj4+IA0KPiA+
Pj4+PiBBbHNvLCB3ZSBjYW4ndCBqdXN0IHVzZSB0aGUgIm1haW4iIG9wZXJhdGlvbiwgd2UgbXVz
dCBjaGVjayBldmVyeSBvcGVyYXRpb24gd2l0aGluIHRoZSBjb21wb3VuZCBhbmQgaWYgYW55IGFy
ZSByZXF1aXJlZCB0byB1c2UgU1A0LCB0aGVuIHRoZSB3aG9sZSBjb21wb3VuZCBkb2VzLiBJIGRv
bid0IG5lY2Vzc2FyaWx5IGtub3cgd2h5IGEgc2VydmVyIHdvdWxkIGRvIHRoaXMsIGJ1dCBpZiB3
ZSBmb2xsb3cgdGhlIHNwZWMgYW5kIGEgc2VydmVyIGluZm9ybXMgdGhlIGNsaWVudCB0aGF0IFNF
VEFUVFIgbXVzdCB1c2Ugc3RhdGUgcHJvdGVjdGlvbiwgdGhlbiBhIFdSSVRFIHdpdGggcG9zdC1v
cCBTRVRBVFRSIG11c3QgdXNlIHRoZSBzdGF0ZSBwcm90ZWN0aW9uIGV2ZW4gdGhvdWdoIFdSSVRF
IGRvZXNuJ3QgbmVlZCBpdC4NCj4gPj4+PiANCj4gPj4+PiBXZSBvbmx5IG5lZWQgdG8gY2hlY2sg
ZXZlcnkgX3N0YXRlZnVsXyBvcGVyYXRpb24sIHJpZ2h0PyBJIGNhbid0IHRoaW5rDQo+ID4+Pj4g
b2YgYW55IGNvbXBvdW5kcyB3aXRoIG1vcmUgdGhhbiAxIHN0YXRlZnVsIG9wZXJhdGlvbi4gV2Un
dmUgZGVsaWJlcmF0ZWx5DQo+ID4+Pj4gYXZvaWRlZCB0aG9zZSBkdWUgdG8gdGhlIHByb2JsZW1z
IHRoYXQgYXJpc2Ugd2hlbiB5b3UgZ2V0IGENCj4gPj4+PiBORlM0RVJSX0RFTEFZIG9yIHNvbWV0
aGluZyBlcXVpdmFsZW50IGF0IHRoZSB3cm9uZyBtb21lbnQuDQo+ID4+PiANCj4gPj4+IFRoYXQg
bWF5IGJlIHRoZSBpbnRlbnRpb24sIGJ1dCBJIHNlZSBubyBzdWNoIGxpbWl0YXRpb24gaW4gdGhl
IHNwZWMuIEl0J3MgbXkgdW5kZXJzdGFuZGluZyB0aGF0IHRoZSBzZXJ2ZXIgY291bGQgc3BlY2lm
eSBhbnkgb3BlcmF0aW9uIGFzIE1VU1Qgb3IgTUFZIHVzZSBzdGF0ZSBwcm90ZWN0aW9uLg0KPiA+
PiANCj4gPj4gUmlnaHQsIGJ1dCB3aGF0IHdvdWxkIHN0YXRlIHByb3RlY3Rpb24gbWVhbiBmb3Ig
YSBzdGF0ZWxlc3Mgb3BlcmF0aW9uDQo+ID4+IHN1Y2ggYXMgR0VUQVRUUj8gVGhhdCBvcGVyYXRp
b24gaXNuJ3QgZXZlbiBhbGxvd2VkIHRvIHJldHVybg0KPiA+PiBORlM0RVJSX1dST05HX0NSRUQs
IHNvIEkgZG9uJ3Qgc2VlIGhvdyBpdCB3b3VsZCBhcHBseS4NCj4gPj4gDQo+ID4+Pj4+IEknbSBu
b3QgZm9sbG93aW5nIHRoZSByZWJvb3QgcXVlc3Rpb24gLSB0aGF0IGNhc2Ugc2hvdWxkIGJlIGhh
bmRsZWQganVzdCBmaW5lLiBPbmNlIHRoZSBFWENIQU5HRV9JRCBoYXBwZW5zLCB0aGUgY2xwIGhh
cyB0d28gYml0bWFwcyAoZW5mb3JjZSBhbmQgYWxsb3cpIHRoYXQgcmVmbGVjdCB0aGUgY3VycmVu
dCBtb2RlIG9mIFNQNF9NQUNIX0NSRUQuDQo+ID4+Pj4gDQo+ID4+Pj4gSSdtIHRoaW5raW5nIHNv
bWV0aGluZyBhbG9uZyB0aGUgZm9sbG93aW5nIHNjZW5hcmlvOg0KPiA+Pj4+IA0KPiA+Pj4+IEEg
cHJvY2VzcyBxdWV1ZXMgdXAgYW4gT1BFTi4gVGhlIHNlcnZlciByZXBsaWVzIE5GUzRFUlJfQkFE
X1NFU1NJT04gYW5kDQo+ID4+Pj4gbGF0ZXIgcmVwbGllcyB3aXRoIE5GUzRFUlJfU1RBTEVfQ0xJ
RU5USUQuIFNvIHdlIHN0YXJ0IHJlYm9vdCByZWNvdmVyeSwNCj4gPj4+PiBhbmQgdGhlIE9QRU4g
Z2V0cyBxdWV1ZWQgd2FpdGluZyBmb3IgYSBzbG90Lg0KPiA+Pj4+IFdlIHRoZW4gc2VuZCBhIG5l
dyBFWENIQU5HRV9JRCwgYW5kIHRoZSBzZXJ2ZXIgcmVwbGllcyB3aXRoIGENCj4gPj4+PiBfZGlm
ZmVyZW50XyBNQUNIX0NSRUQgcHJvdGVjdGlvbiBmb3IgT1BFTi4NCj4gPj4+PiANCj4gPj4+PiBU
aGUgcHJvYmxlbSBhYm92ZSBpcyB0aGF0IHRoZSBPUEVOIGhhcyBhbHJlYWR5IGJlZW4gc3RhcnRl
ZCwgYW5kIHdlJ3ZlDQo+ID4+Pj4gYWxyZWFkeSBhc3NpZ25lZCBpdCBhIGNyZWRlbnRpYWwuIEhv
dyBkbyB3ZSBoYW5kbGUgdGhhdD8NCj4gPj4+PiBEaXR0byBxdWVzdGlvbiBmb3IgUkVBRCwgV1JJ
VEUsIExPQ0ssIExBWU9VVEdFVCwuLi4NCj4gPj4+PiANCj4gPj4+PiBEbyB3ZSBoYXZlIGEgcHJv
YmxlbSBmb3IgTE9DS1UsIENMT1NFIGFuZCBPUEVOX0RPV05HUkFERT8gSSB0aGluayB3ZSBlbmQN
Cj4gPj4+PiB1cCBqdXN0IHNraXBwaW5nIHJlY292ZXJ5IGluIHRob3NlIGNhc2Vz4oCmDQo+ID4+
PiANCj4gPj4+IFRoZXNlIGFyZSBnb29kIHBvaW50cy4gSSdtIG5vdCBzdXJlICp3aHkqIHRoZSBz
cDRfbWFjaF9jcmVkIGNvbmZpZyB3b3VsZCBjaGFuZ2UsIGJ1dCBJIHN1cHBvc2UgaXQgY291bGQs
IGFuZCB5ZXMsIGluIG15IGN1cnJlbnQgaW1wbGVtZW50YXRpb24gdGhlcmUgaXMgbm8gd2F5IHRv
IGJhY2sgb3V0IG9uY2UgYSBycGMgdGFzayBpcyBpbnZva2VkLiAgQW55IHRob3VnaHRzIG9uIHRo
aXM/ICBJIGNhbiBzZWUgdHdvIHdheXMgb2YgaGFuZGxpbmcgdGhpczoNCj4gPj4+IC0gcmVzdGFy
dGluZyBhbGwgdGFza3MgKHdpdGggcnBjX2NsbnQgYW5kIHJwY19jcmVkIGFwcHJvcHJpYXRlbHkg
Y2hhbmdlZCkgd2hlbiBhIGNoYW5nZSBpbiBzcDRfbWFjaF9jcmVkIGNvbmZpZyBpcyBkZXRlY3Rl
ZC4NCj4gPj4+IC0gc29tZWhvdyBoYXZpbmcgdGhlIFJQQyBsYXllciBjYWxsIG91dCB0byBjaGVj
ayBTUDQgc3RhdHVzIGJlZm9yZSBzZW5kaW5nIGVhY2ggUlBDLg0KPiA+PiANCj4gPj4gRm9yIG5v
dywgSSB0aGluayB3ZSBzaG91bGQgbWFrZSBzdXJlIHRoYXQgd2UgaGFuZGxlIE5GUzRFUlJfV1JP
TkdfQ1JFRA0KPiA+PiBjb3JyZWN0bHkgaW4gdGhlc2UgY2FzZXMuIEkgZG9uJ3QgdGhpbmsgd2Ug
ZXZlciB3YW50IHRvIGhhbmRsZSB0aGF0IGluDQo+ID4+IG5mczRfYXN5bmNfaGFuZGxlX2Vycm9y
LCBidXQgd2Ugc2hvdWxkIGRlZmluaXRlbHkgaGFuZGxlIGl0IGluDQo+ID4+IG5mczRfaGFuZGxl
X2V4Y2VwdGlvbigpLCBzaW5jZSB0aGF0IGNhbiBkcml2ZSBhIGZ1bGwgcmV0cnkgb2YgdGhlIFJQ
Qw0KPiA+PiBjYWxsLg0KPiA+PiANCj4gPj4gLS0gDQo+ID4+IFRyb25kIE15a2xlYnVzdA0KPiA+
PiBMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCj4gPj4gDQo+ID4+IE5ldEFwcA0KPiA+PiBU
cm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0KPiA+PiB3d3cubmV0YXBwLmNvbQ0KPiA+IA0KPiA+
IC0tDQo+ID4gVG8gdW5zdWJzY3JpYmUgZnJvbSB0aGlzIGxpc3Q6IHNlbmQgdGhlIGxpbmUgInVu
c3Vic2NyaWJlIGxpbnV4LW5mcyIgaW4NCj4gPiB0aGUgYm9keSBvZiBhIG1lc3NhZ2UgdG8gbWFq
b3Jkb21vQHZnZXIua2VybmVsLm9yZw0KPiA+IE1vcmUgbWFqb3Jkb21vIGluZm8gYXQgIGh0dHA6
Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0KPiANCg0KLS0gDQpUcm9uZCBN
eWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25kLk15
a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0K

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Adamson, Dros]

Another issue: Is it out of spec to do what we plan and "just use" the nfs_client's rpc_client for all state manager operations without checking if the server supports it from the SP4_MACH_CRED EXCHANGE_ID negotiation?

RENEW for example is not explicitly mentioned in the SP4_MACH_CRED section and I'm looking at a server that doesn't claim state_protection support for RENEW.

-dros

On Jul 18, 2013, at 3:54 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:

> On Thu, 2013-07-18 at 19:49 +0000, Adamson, Dros wrote:
>> Only supporting operations that have the error code NFS4ERR_WRONG_CRED seems to be wrong.  Operations like BIND_CONN_TO_SESSION don't support don't support this error code, but are explicitly mentioned in SP4_MACH_CRED sections of the spec.
> 
> Looking at the allowed error return values for BIND_CONN_TO_SESSION, I'm
> at a loss to figure out exactly what it should return in this case. I
> suspect that the lack of an NFS4ERR_WRONG_CRED is actually a protocol
> bug.
> 
> Time to go back to the ietf mailing list...
> 
>> I'll continue my work described below modulo limiting to NFS4ERR_WRONG_CRED enabled OPs and see how that looks...
>> 
>> -dros
>> 
>> On Jul 18, 2013, at 2:13 PM, "Adamson, Dros" <Weston.Adamson@netapp.com> wrote:
>> 
>>> Ok, I'll have the client only ask for state protection on operations that support NFS4ERR_WRONG_CRED - and I'll do the SP4_MACH_CRED checks only on these operations and only if they aren't already using the machine cred already. This simplifies my original problem - now I can just use the "main" opcode.
>>> 
>>> I'll also add error handlers for NFS4ERR_WRONG_CRED.
>>> 
>>> Thanks!
>>> 
>>> -dros
>>> 
>>> On Jul 18, 2013, at 1:08 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>> 
>>>> On Thu, 2013-07-18 at 15:50 +0000, Adamson, Dros wrote:
>>>>> On Jul 18, 2013, at 11:21 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
>>>>> wrote:
>>>>> 
>>>>>> On Thu, 2013-07-18 at 15:10 +0000, Adamson, Dros wrote:
>>>>>>> On Jul 18, 2013, at 10:57 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>>>>>> 
>>>>>>>> On Thu, 2013-07-18 at 14:46 +0000, Adamson, Dros wrote:
>>>>>>>>> Hey,
>>>>>>>>> 
>>>>>>>>> I have a mostly functional client-side implementation of SP4_MACH_CRED! It still needs a lot of cleanup and testing.
>>>>>>>>> 
>>>>>>>>> I have one style issue that I want to run by the list before I post a patchset:
>>>>>>>>> 
>>>>>>>>> So, SP4_MACH_CRED negotiates two bitmaps in the EXCHANGE_ID (one "enforce", one "allow") for state protection. These bitmaps are indexed by the NFSv4 operation number.  The state protect check must happen in the nfs4proc.c layer (or before), right before we call rpc_call_sync or equivalent, so that it can select the right cred and rpc_client.
>>>>>>>>> 
>>>>>>>>> Here's the problem: we don't know what operations (opcodes) are actually in a compound until the XDR encode callback is called.  The rpc_procinfo array doesn't have this mapping - in fact, it only lives in the xdr encode layer.
>>>>>>>>> 
>>>>>>>>> One approach is to immediately translate the opcode bitmaps to "nfs4 procedure index" bitmaps, indexing into the rpc_procinfo array.  This would mean there is a second mapping of NFSv4 procedure -> opcodes that must be updated when an XDR encode callback is changed.
>>>>>>>>> 
>>>>>>>>> Another approach would be to add a callback to the XDR api so we could "ask" it if an NFSv4 procedure contains any of the opcodes in a bitmap. The nice thing about this approach is that the mapping of procedure to opcodes within will live right next to the XDR encode callback and make it obvious that both need to be changed together.
>>>>>>>>> 
>>>>>>>>> I suppose I'm leaning toward a combination of both of these approaches - keep the mapping in XDR-land and translate the bitmaps immediately on negation for fast lookups during normal operation.
>>>>>>>>> 
>>>>>>>>> Comments? Am I missing something?
>>>>>>>> 
>>>>>>>> I'm not sure that I understand. We don't do dynamic creation of
>>>>>>>> compounds: we pretty much know in the nfs4_proc_* routine what the main
>>>>>>>> operation is (the one exception being CLOSE/OPEN_DOWNGRADE). So why
>>>>>>>> can't we work out the protection status at that time? Is the worry that
>>>>>>>> the server might reboot and come back with different MACH_CRED
>>>>>>>> protections?
>>>>>>> 
>>>>>>> Sure, we know what the main operation is and we know what operations will end up in a compound - we can just look at the xdr encoder.  My question is simply one of style. Do we want to have each nfs4proc procedure to have a list of operations that must be updated if an xdr encoder is updated? I'm fine with doing it this way, it just seemed wrong to have the same mapping in two different places. This is why I'm asking ;)
>>>>>>> 
>>>>>>> Also, we can't just use the "main" operation, we must check every operation within the compound and if any are required to use SP4, then the whole compound does. I don't necessarily know why a server would do this, but if we follow the spec and a server informs the client that SETATTR must use state protection, then a WRITE with post-op SETATTR must use the state protection even though WRITE doesn't need it.
>>>>>> 
>>>>>> We only need to check every _stateful_ operation, right? I can't think
>>>>>> of any compounds with more than 1 stateful operation. We've deliberately
>>>>>> avoided those due to the problems that arise when you get a
>>>>>> NFS4ERR_DELAY or something equivalent at the wrong moment.
>>>>> 
>>>>> That may be the intention, but I see no such limitation in the spec. It's my understanding that the server could specify any operation as MUST or MAY use state protection.
>>>> 
>>>> Right, but what would state protection mean for a stateless operation
>>>> such as GETATTR? That operation isn't even allowed to return
>>>> NFS4ERR_WRONG_CRED, so I don't see how it would apply.
>>>> 
>>>>>>> I'm not following the reboot question - that case should be handled just fine. Once the EXCHANGE_ID happens, the clp has two bitmaps (enforce and allow) that reflect the current mode of SP4_MACH_CRED.
>>>>>> 
>>>>>> I'm thinking something along the following scenario:
>>>>>> 
>>>>>> A process queues up an OPEN. The server replies NFS4ERR_BAD_SESSION and
>>>>>> later replies with NFS4ERR_STALE_CLIENTID. So we start reboot recovery,
>>>>>> and the OPEN gets queued waiting for a slot.
>>>>>> We then send a new EXCHANGE_ID, and the server replies with a
>>>>>> _different_ MACH_CRED protection for OPEN.
>>>>>> 
>>>>>> The problem above is that the OPEN has already been started, and we've
>>>>>> already assigned it a credential. How do we handle that?
>>>>>> Ditto question for READ, WRITE, LOCK, LAYOUTGET,...
>>>>>> 
>>>>>> Do we have a problem for LOCKU, CLOSE and OPEN_DOWNGRADE? I think we end
>>>>>> up just skipping recovery in those cases…
>>>>> 
>>>>> These are good points. I'm not sure *why* the sp4_mach_cred config would change, but I suppose it could, and yes, in my current implementation there is no way to back out once a rpc task is invoked.  Any thoughts on this?  I can see two ways of handling this:
>>>>> - restarting all tasks (with rpc_clnt and rpc_cred appropriately changed) when a change in sp4_mach_cred config is detected.
>>>>> - somehow having the RPC layer call out to check SP4 status before sending each RPC.
>>>> 
>>>> For now, I think we should make sure that we handle NFS4ERR_WRONG_CRED
>>>> correctly in these cases. I don't think we ever want to handle that in
>>>> nfs4_async_handle_error, but we should definitely handle it in
>>>> nfs4_handle_exception(), since that can drive a full retry of the RPC
>>>> call.
>>>> 
>>>> -- 
>>>> Trond Myklebust
>>>> Linux NFS client maintainer
>>>> 
>>>> NetApp
>>>> Trond.Myklebust@netapp.com
>>>> www.netapp.com
>>> 
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
> 
> -- 
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com

Should BIND_CONN_TO_SESSION be allowed to return NFS4ERR_WRONG_CRED

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDE1OjU0IC0wNDAwLCBUcm9uZCBNeWtsZWJ1c3Qgd3JvdGU6
DQo+IE9uIFRodSwgMjAxMy0wNy0xOCBhdCAxOTo0OSArMDAwMCwgQWRhbXNvbiwgRHJvcyB3cm90
ZToNCj4gPiBPbmx5IHN1cHBvcnRpbmcgb3BlcmF0aW9ucyB0aGF0IGhhdmUgdGhlIGVycm9yIGNv
ZGUgTkZTNEVSUl9XUk9OR19DUkVEIHNlZW1zIHRvIGJlIHdyb25nLiAgT3BlcmF0aW9ucyBsaWtl
IEJJTkRfQ09OTl9UT19TRVNTSU9OIGRvbid0IHN1cHBvcnQgZG9uJ3Qgc3VwcG9ydCB0aGlzIGVy
cm9yIGNvZGUsIGJ1dCBhcmUgZXhwbGljaXRseSBtZW50aW9uZWQgaW4gU1A0X01BQ0hfQ1JFRCBz
ZWN0aW9ucyBvZiB0aGUgc3BlYy4NCj4gDQo+IExvb2tpbmcgYXQgdGhlIGFsbG93ZWQgZXJyb3Ig
cmV0dXJuIHZhbHVlcyBmb3IgQklORF9DT05OX1RPX1NFU1NJT04sIEknbQ0KPiBhdCBhIGxvc3Mg
dG8gZmlndXJlIG91dCBleGFjdGx5IHdoYXQgaXQgc2hvdWxkIHJldHVybiBpbiB0aGlzIGNhc2Uu
IEkNCj4gc3VzcGVjdCB0aGF0IHRoZSBsYWNrIG9mIGFuIE5GUzRFUlJfV1JPTkdfQ1JFRCBpcyBh
Y3R1YWxseSBhIHByb3RvY29sDQo+IGJ1Zy4NCj4gDQo+IFRpbWUgdG8gZ28gYmFjayB0byB0aGUg
aWV0ZiBtYWlsaW5nIGxpc3QuLi4NCg0KSGkgYWxsLA0KDQpXaGVuIGF0dGVtcHRpbmcgdG8gaW1w
bGVtZW50IHRoZSBTUDRfTUFDSF9DUkVEIHN0YXRlIHByb3RlY3Rpb24sIERyb3MNCnJhbiBpbnRv
IGFuIGlzc3VlLiBJZiB0aGUgQklORF9DT05OX1RPX1NFU1NJT04gb3BlcmF0aW9uIGlzIGxpc3Rl
ZCBpbg0KdGhlICJzcG9fbXVzdF9lbmZvcmNlIiBsaXN0IG9mIG9wZXJhdGlvbnMsIHdoYXQgc2hv
dWxkIGl0IG5vdCBiZSBhbGxvd2VkDQp0byByZXR1cm4gTkZTNEVSUl9XUk9OR19DUkVEIGlmIGNh
bGxlZCB3aXRoIGEgY3JlZGVudGlhbCB0aGF0IGlzIG5vdCB0aGUNCm1hY2hpbmUgb3IgU1NWIGNy
ZWRlbnRpYWw/DQoNCkNoZWVycw0KICBUcm9uZA0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4
IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25kLk15a2xlYnVzdEBuZXRhcHAu
Y29tDQp3d3cubmV0YXBwLmNvbQ0K

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDIwOjA0ICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiBBbm90aGVyIGlzc3VlOiBJcyBpdCBvdXQgb2Ygc3BlYyB0byBkbyB3aGF0IHdlIHBsYW4gYW5k
ICJqdXN0IHVzZSIgdGhlIG5mc19jbGllbnQncyBycGNfY2xpZW50IGZvciBhbGwgc3RhdGUgbWFu
YWdlciBvcGVyYXRpb25zIHdpdGhvdXQgY2hlY2tpbmcgaWYgdGhlIHNlcnZlciBzdXBwb3J0cyBp
dCBmcm9tIHRoZSBTUDRfTUFDSF9DUkVEIEVYQ0hBTkdFX0lEIG5lZ290aWF0aW9uPw0KPiANCj4g
UkVORVcgZm9yIGV4YW1wbGUgaXMgbm90IGV4cGxpY2l0bHkgbWVudGlvbmVkIGluIHRoZSBTUDRf
TUFDSF9DUkVEIHNlY3Rpb24gYW5kIEknbSBsb29raW5nIGF0IGEgc2VydmVyIHRoYXQgZG9lc24n
dCBjbGFpbSBzdGF0ZV9wcm90ZWN0aW9uIHN1cHBvcnQgZm9yIFJFTkVXLg0KDQpSRU5FVyBpcyBh
ICJtYW5kYXRvcnkgdG8gbm90IGltcGxlbWVudCIgb3BlcmF0aW9uIGluIE5GU3Y0LjEuDQoNCkNo
ZWVycw0KICBUcm9uZA0KDQo+IC1kcm9zDQo+IA0KPiBPbiBKdWwgMTgsIDIwMTMsIGF0IDM6NTQg
UE0sICJNeWtsZWJ1c3QsIFRyb25kIiA8VHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20+IHdyb3Rl
Og0KPiANCj4gPiBPbiBUaHUsIDIwMTMtMDctMTggYXQgMTk6NDkgKzAwMDAsIEFkYW1zb24sIERy
b3Mgd3JvdGU6DQo+ID4+IE9ubHkgc3VwcG9ydGluZyBvcGVyYXRpb25zIHRoYXQgaGF2ZSB0aGUg
ZXJyb3IgY29kZSBORlM0RVJSX1dST05HX0NSRUQgc2VlbXMgdG8gYmUgd3JvbmcuICBPcGVyYXRp
b25zIGxpa2UgQklORF9DT05OX1RPX1NFU1NJT04gZG9uJ3Qgc3VwcG9ydCBkb24ndCBzdXBwb3J0
IHRoaXMgZXJyb3IgY29kZSwgYnV0IGFyZSBleHBsaWNpdGx5IG1lbnRpb25lZCBpbiBTUDRfTUFD
SF9DUkVEIHNlY3Rpb25zIG9mIHRoZSBzcGVjLg0KPiA+IA0KPiA+IExvb2tpbmcgYXQgdGhlIGFs
bG93ZWQgZXJyb3IgcmV0dXJuIHZhbHVlcyBmb3IgQklORF9DT05OX1RPX1NFU1NJT04sIEknbQ0K
PiA+IGF0IGEgbG9zcyB0byBmaWd1cmUgb3V0IGV4YWN0bHkgd2hhdCBpdCBzaG91bGQgcmV0dXJu
IGluIHRoaXMgY2FzZS4gSQ0KPiA+IHN1c3BlY3QgdGhhdCB0aGUgbGFjayBvZiBhbiBORlM0RVJS
X1dST05HX0NSRUQgaXMgYWN0dWFsbHkgYSBwcm90b2NvbA0KPiA+IGJ1Zy4NCj4gPiANCj4gPiBU
aW1lIHRvIGdvIGJhY2sgdG8gdGhlIGlldGYgbWFpbGluZyBsaXN0Li4uDQo+ID4gDQo+ID4+IEkn
bGwgY29udGludWUgbXkgd29yayBkZXNjcmliZWQgYmVsb3cgbW9kdWxvIGxpbWl0aW5nIHRvIE5G
UzRFUlJfV1JPTkdfQ1JFRCBlbmFibGVkIE9QcyBhbmQgc2VlIGhvdyB0aGF0IGxvb2tzLi4uDQo+
ID4+IA0KPiA+PiAtZHJvcw0KPiA+PiANCj4gPj4gT24gSnVsIDE4LCAyMDEzLCBhdCAyOjEzIFBN
LCAiQWRhbXNvbiwgRHJvcyIgPFdlc3Rvbi5BZGFtc29uQG5ldGFwcC5jb20+IHdyb3RlOg0KPiA+
PiANCj4gPj4+IE9rLCBJJ2xsIGhhdmUgdGhlIGNsaWVudCBvbmx5IGFzayBmb3Igc3RhdGUgcHJv
dGVjdGlvbiBvbiBvcGVyYXRpb25zIHRoYXQgc3VwcG9ydCBORlM0RVJSX1dST05HX0NSRUQgLSBh
bmQgSSdsbCBkbyB0aGUgU1A0X01BQ0hfQ1JFRCBjaGVja3Mgb25seSBvbiB0aGVzZSBvcGVyYXRp
b25zIGFuZCBvbmx5IGlmIHRoZXkgYXJlbid0IGFscmVhZHkgdXNpbmcgdGhlIG1hY2hpbmUgY3Jl
ZCBhbHJlYWR5LiBUaGlzIHNpbXBsaWZpZXMgbXkgb3JpZ2luYWwgcHJvYmxlbSAtIG5vdyBJIGNh
biBqdXN0IHVzZSB0aGUgIm1haW4iIG9wY29kZS4NCj4gPj4+IA0KPiA+Pj4gSSdsbCBhbHNvIGFk
ZCBlcnJvciBoYW5kbGVycyBmb3IgTkZTNEVSUl9XUk9OR19DUkVELg0KPiA+Pj4gDQo+ID4+PiBU
aGFua3MhDQo+ID4+PiANCj4gPj4+IC1kcm9zDQo+ID4+PiANCj4gPj4+IE9uIEp1bCAxOCwgMjAx
MywgYXQgMTowOCBQTSwgIk15a2xlYnVzdCwgVHJvbmQiIDxUcm9uZC5NeWtsZWJ1c3RAbmV0YXBw
LmNvbT4gd3JvdGU6DQo+ID4+PiANCj4gPj4+PiBPbiBUaHUsIDIwMTMtMDctMTggYXQgMTU6NTAg
KzAwMDAsIEFkYW1zb24sIERyb3Mgd3JvdGU6DQo+ID4+Pj4+IE9uIEp1bCAxOCwgMjAxMywgYXQg
MTE6MjEgQU0sICJNeWtsZWJ1c3QsIFRyb25kIiA8VHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20+
DQo+ID4+Pj4+IHdyb3RlOg0KPiA+Pj4+PiANCj4gPj4+Pj4+IE9uIFRodSwgMjAxMy0wNy0xOCBh
dCAxNToxMCArMDAwMCwgQWRhbXNvbiwgRHJvcyB3cm90ZToNCj4gPj4+Pj4+PiBPbiBKdWwgMTgs
IDIwMTMsIGF0IDEwOjU3IEFNLCAiTXlrbGVidXN0LCBUcm9uZCIgPFRyb25kLk15a2xlYnVzdEBu
ZXRhcHAuY29tPiB3cm90ZToNCj4gPj4+Pj4+PiANCj4gPj4+Pj4+Pj4gT24gVGh1LCAyMDEzLTA3
LTE4IGF0IDE0OjQ2ICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0KPiA+Pj4+Pj4+Pj4gSGV5
LA0KPiA+Pj4+Pj4+Pj4gDQo+ID4+Pj4+Pj4+PiBJIGhhdmUgYSBtb3N0bHkgZnVuY3Rpb25hbCBj
bGllbnQtc2lkZSBpbXBsZW1lbnRhdGlvbiBvZiBTUDRfTUFDSF9DUkVEISBJdCBzdGlsbCBuZWVk
cyBhIGxvdCBvZiBjbGVhbnVwIGFuZCB0ZXN0aW5nLg0KPiA+Pj4+Pj4+Pj4gDQo+ID4+Pj4+Pj4+
PiBJIGhhdmUgb25lIHN0eWxlIGlzc3VlIHRoYXQgSSB3YW50IHRvIHJ1biBieSB0aGUgbGlzdCBi
ZWZvcmUgSSBwb3N0IGEgcGF0Y2hzZXQ6DQo+ID4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+IFNvLCBT
UDRfTUFDSF9DUkVEIG5lZ290aWF0ZXMgdHdvIGJpdG1hcHMgaW4gdGhlIEVYQ0hBTkdFX0lEIChv
bmUgImVuZm9yY2UiLCBvbmUgImFsbG93IikgZm9yIHN0YXRlIHByb3RlY3Rpb24uIFRoZXNlIGJp
dG1hcHMgYXJlIGluZGV4ZWQgYnkgdGhlIE5GU3Y0IG9wZXJhdGlvbiBudW1iZXIuICBUaGUgc3Rh
dGUgcHJvdGVjdCBjaGVjayBtdXN0IGhhcHBlbiBpbiB0aGUgbmZzNHByb2MuYyBsYXllciAob3Ig
YmVmb3JlKSwgcmlnaHQgYmVmb3JlIHdlIGNhbGwgcnBjX2NhbGxfc3luYyBvciBlcXVpdmFsZW50
LCBzbyB0aGF0IGl0IGNhbiBzZWxlY3QgdGhlIHJpZ2h0IGNyZWQgYW5kIHJwY19jbGllbnQuDQo+
ID4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+IEhlcmUncyB0aGUgcHJvYmxlbTogd2UgZG9uJ3Qga25v
dyB3aGF0IG9wZXJhdGlvbnMgKG9wY29kZXMpIGFyZSBhY3R1YWxseSBpbiBhIGNvbXBvdW5kIHVu
dGlsIHRoZSBYRFIgZW5jb2RlIGNhbGxiYWNrIGlzIGNhbGxlZC4gIFRoZSBycGNfcHJvY2luZm8g
YXJyYXkgZG9lc24ndCBoYXZlIHRoaXMgbWFwcGluZyAtIGluIGZhY3QsIGl0IG9ubHkgbGl2ZXMg
aW4gdGhlIHhkciBlbmNvZGUgbGF5ZXIuDQo+ID4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+IE9uZSBh
cHByb2FjaCBpcyB0byBpbW1lZGlhdGVseSB0cmFuc2xhdGUgdGhlIG9wY29kZSBiaXRtYXBzIHRv
ICJuZnM0IHByb2NlZHVyZSBpbmRleCIgYml0bWFwcywgaW5kZXhpbmcgaW50byB0aGUgcnBjX3By
b2NpbmZvIGFycmF5LiAgVGhpcyB3b3VsZCBtZWFuIHRoZXJlIGlzIGEgc2Vjb25kIG1hcHBpbmcg
b2YgTkZTdjQgcHJvY2VkdXJlIC0+IG9wY29kZXMgdGhhdCBtdXN0IGJlIHVwZGF0ZWQgd2hlbiBh
biBYRFIgZW5jb2RlIGNhbGxiYWNrIGlzIGNoYW5nZWQuDQo+ID4+Pj4+Pj4+PiANCj4gPj4+Pj4+
Pj4+IEFub3RoZXIgYXBwcm9hY2ggd291bGQgYmUgdG8gYWRkIGEgY2FsbGJhY2sgdG8gdGhlIFhE
UiBhcGkgc28gd2UgY291bGQgImFzayIgaXQgaWYgYW4gTkZTdjQgcHJvY2VkdXJlIGNvbnRhaW5z
IGFueSBvZiB0aGUgb3Bjb2RlcyBpbiBhIGJpdG1hcC4gVGhlIG5pY2UgdGhpbmcgYWJvdXQgdGhp
cyBhcHByb2FjaCBpcyB0aGF0IHRoZSBtYXBwaW5nIG9mIHByb2NlZHVyZSB0byBvcGNvZGVzIHdp
dGhpbiB3aWxsIGxpdmUgcmlnaHQgbmV4dCB0byB0aGUgWERSIGVuY29kZSBjYWxsYmFjayBhbmQg
bWFrZSBpdCBvYnZpb3VzIHRoYXQgYm90aCBuZWVkIHRvIGJlIGNoYW5nZWQgdG9nZXRoZXIuDQo+
ID4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+IEkgc3VwcG9zZSBJJ20gbGVhbmluZyB0b3dhcmQgYSBj
b21iaW5hdGlvbiBvZiBib3RoIG9mIHRoZXNlIGFwcHJvYWNoZXMgLSBrZWVwIHRoZSBtYXBwaW5n
IGluIFhEUi1sYW5kIGFuZCB0cmFuc2xhdGUgdGhlIGJpdG1hcHMgaW1tZWRpYXRlbHkgb24gbmVn
YXRpb24gZm9yIGZhc3QgbG9va3VwcyBkdXJpbmcgbm9ybWFsIG9wZXJhdGlvbi4NCj4gPj4+Pj4+
Pj4+IA0KPiA+Pj4+Pj4+Pj4gQ29tbWVudHM/IEFtIEkgbWlzc2luZyBzb21ldGhpbmc/DQo+ID4+
Pj4+Pj4+IA0KPiA+Pj4+Pj4+PiBJJ20gbm90IHN1cmUgdGhhdCBJIHVuZGVyc3RhbmQuIFdlIGRv
bid0IGRvIGR5bmFtaWMgY3JlYXRpb24gb2YNCj4gPj4+Pj4+Pj4gY29tcG91bmRzOiB3ZSBwcmV0
dHkgbXVjaCBrbm93IGluIHRoZSBuZnM0X3Byb2NfKiByb3V0aW5lIHdoYXQgdGhlIG1haW4NCj4g
Pj4+Pj4+Pj4gb3BlcmF0aW9uIGlzICh0aGUgb25lIGV4Y2VwdGlvbiBiZWluZyBDTE9TRS9PUEVO
X0RPV05HUkFERSkuIFNvIHdoeQ0KPiA+Pj4+Pj4+PiBjYW4ndCB3ZSB3b3JrIG91dCB0aGUgcHJv
dGVjdGlvbiBzdGF0dXMgYXQgdGhhdCB0aW1lPyBJcyB0aGUgd29ycnkgdGhhdA0KPiA+Pj4+Pj4+
PiB0aGUgc2VydmVyIG1pZ2h0IHJlYm9vdCBhbmQgY29tZSBiYWNrIHdpdGggZGlmZmVyZW50IE1B
Q0hfQ1JFRA0KPiA+Pj4+Pj4+PiBwcm90ZWN0aW9ucz8NCj4gPj4+Pj4+PiANCj4gPj4+Pj4+PiBT
dXJlLCB3ZSBrbm93IHdoYXQgdGhlIG1haW4gb3BlcmF0aW9uIGlzIGFuZCB3ZSBrbm93IHdoYXQg
b3BlcmF0aW9ucyB3aWxsIGVuZCB1cCBpbiBhIGNvbXBvdW5kIC0gd2UgY2FuIGp1c3QgbG9vayBh
dCB0aGUgeGRyIGVuY29kZXIuICBNeSBxdWVzdGlvbiBpcyBzaW1wbHkgb25lIG9mIHN0eWxlLiBE
byB3ZSB3YW50IHRvIGhhdmUgZWFjaCBuZnM0cHJvYyBwcm9jZWR1cmUgdG8gaGF2ZSBhIGxpc3Qg
b2Ygb3BlcmF0aW9ucyB0aGF0IG11c3QgYmUgdXBkYXRlZCBpZiBhbiB4ZHIgZW5jb2RlciBpcyB1
cGRhdGVkPyBJJ20gZmluZSB3aXRoIGRvaW5nIGl0IHRoaXMgd2F5LCBpdCBqdXN0IHNlZW1lZCB3
cm9uZyB0byBoYXZlIHRoZSBzYW1lIG1hcHBpbmcgaW4gdHdvIGRpZmZlcmVudCBwbGFjZXMuIFRo
aXMgaXMgd2h5IEknbSBhc2tpbmcgOykNCj4gPj4+Pj4+PiANCj4gPj4+Pj4+PiBBbHNvLCB3ZSBj
YW4ndCBqdXN0IHVzZSB0aGUgIm1haW4iIG9wZXJhdGlvbiwgd2UgbXVzdCBjaGVjayBldmVyeSBv
cGVyYXRpb24gd2l0aGluIHRoZSBjb21wb3VuZCBhbmQgaWYgYW55IGFyZSByZXF1aXJlZCB0byB1
c2UgU1A0LCB0aGVuIHRoZSB3aG9sZSBjb21wb3VuZCBkb2VzLiBJIGRvbid0IG5lY2Vzc2FyaWx5
IGtub3cgd2h5IGEgc2VydmVyIHdvdWxkIGRvIHRoaXMsIGJ1dCBpZiB3ZSBmb2xsb3cgdGhlIHNw
ZWMgYW5kIGEgc2VydmVyIGluZm9ybXMgdGhlIGNsaWVudCB0aGF0IFNFVEFUVFIgbXVzdCB1c2Ug
c3RhdGUgcHJvdGVjdGlvbiwgdGhlbiBhIFdSSVRFIHdpdGggcG9zdC1vcCBTRVRBVFRSIG11c3Qg
dXNlIHRoZSBzdGF0ZSBwcm90ZWN0aW9uIGV2ZW4gdGhvdWdoIFdSSVRFIGRvZXNuJ3QgbmVlZCBp
dC4NCj4gPj4+Pj4+IA0KPiA+Pj4+Pj4gV2Ugb25seSBuZWVkIHRvIGNoZWNrIGV2ZXJ5IF9zdGF0
ZWZ1bF8gb3BlcmF0aW9uLCByaWdodD8gSSBjYW4ndCB0aGluaw0KPiA+Pj4+Pj4gb2YgYW55IGNv
bXBvdW5kcyB3aXRoIG1vcmUgdGhhbiAxIHN0YXRlZnVsIG9wZXJhdGlvbi4gV2UndmUgZGVsaWJl
cmF0ZWx5DQo+ID4+Pj4+PiBhdm9pZGVkIHRob3NlIGR1ZSB0byB0aGUgcHJvYmxlbXMgdGhhdCBh
cmlzZSB3aGVuIHlvdSBnZXQgYQ0KPiA+Pj4+Pj4gTkZTNEVSUl9ERUxBWSBvciBzb21ldGhpbmcg
ZXF1aXZhbGVudCBhdCB0aGUgd3JvbmcgbW9tZW50Lg0KPiA+Pj4+PiANCj4gPj4+Pj4gVGhhdCBt
YXkgYmUgdGhlIGludGVudGlvbiwgYnV0IEkgc2VlIG5vIHN1Y2ggbGltaXRhdGlvbiBpbiB0aGUg
c3BlYy4gSXQncyBteSB1bmRlcnN0YW5kaW5nIHRoYXQgdGhlIHNlcnZlciBjb3VsZCBzcGVjaWZ5
IGFueSBvcGVyYXRpb24gYXMgTVVTVCBvciBNQVkgdXNlIHN0YXRlIHByb3RlY3Rpb24uDQo+ID4+
Pj4gDQo+ID4+Pj4gUmlnaHQsIGJ1dCB3aGF0IHdvdWxkIHN0YXRlIHByb3RlY3Rpb24gbWVhbiBm
b3IgYSBzdGF0ZWxlc3Mgb3BlcmF0aW9uDQo+ID4+Pj4gc3VjaCBhcyBHRVRBVFRSPyBUaGF0IG9w
ZXJhdGlvbiBpc24ndCBldmVuIGFsbG93ZWQgdG8gcmV0dXJuDQo+ID4+Pj4gTkZTNEVSUl9XUk9O
R19DUkVELCBzbyBJIGRvbid0IHNlZSBob3cgaXQgd291bGQgYXBwbHkuDQo+ID4+Pj4gDQo+ID4+
Pj4+Pj4gSSdtIG5vdCBmb2xsb3dpbmcgdGhlIHJlYm9vdCBxdWVzdGlvbiAtIHRoYXQgY2FzZSBz
aG91bGQgYmUgaGFuZGxlZCBqdXN0IGZpbmUuIE9uY2UgdGhlIEVYQ0hBTkdFX0lEIGhhcHBlbnMs
IHRoZSBjbHAgaGFzIHR3byBiaXRtYXBzIChlbmZvcmNlIGFuZCBhbGxvdykgdGhhdCByZWZsZWN0
IHRoZSBjdXJyZW50IG1vZGUgb2YgU1A0X01BQ0hfQ1JFRC4NCj4gPj4+Pj4+IA0KPiA+Pj4+Pj4g
SSdtIHRoaW5raW5nIHNvbWV0aGluZyBhbG9uZyB0aGUgZm9sbG93aW5nIHNjZW5hcmlvOg0KPiA+
Pj4+Pj4gDQo+ID4+Pj4+PiBBIHByb2Nlc3MgcXVldWVzIHVwIGFuIE9QRU4uIFRoZSBzZXJ2ZXIg
cmVwbGllcyBORlM0RVJSX0JBRF9TRVNTSU9OIGFuZA0KPiA+Pj4+Pj4gbGF0ZXIgcmVwbGllcyB3
aXRoIE5GUzRFUlJfU1RBTEVfQ0xJRU5USUQuIFNvIHdlIHN0YXJ0IHJlYm9vdCByZWNvdmVyeSwN
Cj4gPj4+Pj4+IGFuZCB0aGUgT1BFTiBnZXRzIHF1ZXVlZCB3YWl0aW5nIGZvciBhIHNsb3QuDQo+
ID4+Pj4+PiBXZSB0aGVuIHNlbmQgYSBuZXcgRVhDSEFOR0VfSUQsIGFuZCB0aGUgc2VydmVyIHJl
cGxpZXMgd2l0aCBhDQo+ID4+Pj4+PiBfZGlmZmVyZW50XyBNQUNIX0NSRUQgcHJvdGVjdGlvbiBm
b3IgT1BFTi4NCj4gPj4+Pj4+IA0KPiA+Pj4+Pj4gVGhlIHByb2JsZW0gYWJvdmUgaXMgdGhhdCB0
aGUgT1BFTiBoYXMgYWxyZWFkeSBiZWVuIHN0YXJ0ZWQsIGFuZCB3ZSd2ZQ0KPiA+Pj4+Pj4gYWxy
ZWFkeSBhc3NpZ25lZCBpdCBhIGNyZWRlbnRpYWwuIEhvdyBkbyB3ZSBoYW5kbGUgdGhhdD8NCj4g
Pj4+Pj4+IERpdHRvIHF1ZXN0aW9uIGZvciBSRUFELCBXUklURSwgTE9DSywgTEFZT1VUR0VULC4u
Lg0KPiA+Pj4+Pj4gDQo+ID4+Pj4+PiBEbyB3ZSBoYXZlIGEgcHJvYmxlbSBmb3IgTE9DS1UsIENM
T1NFIGFuZCBPUEVOX0RPV05HUkFERT8gSSB0aGluayB3ZSBlbmQNCj4gPj4+Pj4+IHVwIGp1c3Qg
c2tpcHBpbmcgcmVjb3ZlcnkgaW4gdGhvc2UgY2FzZXPigKYNCj4gPj4+Pj4gDQo+ID4+Pj4+IFRo
ZXNlIGFyZSBnb29kIHBvaW50cy4gSSdtIG5vdCBzdXJlICp3aHkqIHRoZSBzcDRfbWFjaF9jcmVk
IGNvbmZpZyB3b3VsZCBjaGFuZ2UsIGJ1dCBJIHN1cHBvc2UgaXQgY291bGQsIGFuZCB5ZXMsIGlu
IG15IGN1cnJlbnQgaW1wbGVtZW50YXRpb24gdGhlcmUgaXMgbm8gd2F5IHRvIGJhY2sgb3V0IG9u
Y2UgYSBycGMgdGFzayBpcyBpbnZva2VkLiAgQW55IHRob3VnaHRzIG9uIHRoaXM/ICBJIGNhbiBz
ZWUgdHdvIHdheXMgb2YgaGFuZGxpbmcgdGhpczoNCj4gPj4+Pj4gLSByZXN0YXJ0aW5nIGFsbCB0
YXNrcyAod2l0aCBycGNfY2xudCBhbmQgcnBjX2NyZWQgYXBwcm9wcmlhdGVseSBjaGFuZ2VkKSB3
aGVuIGEgY2hhbmdlIGluIHNwNF9tYWNoX2NyZWQgY29uZmlnIGlzIGRldGVjdGVkLg0KPiA+Pj4+
PiAtIHNvbWVob3cgaGF2aW5nIHRoZSBSUEMgbGF5ZXIgY2FsbCBvdXQgdG8gY2hlY2sgU1A0IHN0
YXR1cyBiZWZvcmUgc2VuZGluZyBlYWNoIFJQQy4NCj4gPj4+PiANCj4gPj4+PiBGb3Igbm93LCBJ
IHRoaW5rIHdlIHNob3VsZCBtYWtlIHN1cmUgdGhhdCB3ZSBoYW5kbGUgTkZTNEVSUl9XUk9OR19D
UkVEDQo+ID4+Pj4gY29ycmVjdGx5IGluIHRoZXNlIGNhc2VzLiBJIGRvbid0IHRoaW5rIHdlIGV2
ZXIgd2FudCB0byBoYW5kbGUgdGhhdCBpbg0KPiA+Pj4+IG5mczRfYXN5bmNfaGFuZGxlX2Vycm9y
LCBidXQgd2Ugc2hvdWxkIGRlZmluaXRlbHkgaGFuZGxlIGl0IGluDQo+ID4+Pj4gbmZzNF9oYW5k
bGVfZXhjZXB0aW9uKCksIHNpbmNlIHRoYXQgY2FuIGRyaXZlIGEgZnVsbCByZXRyeSBvZiB0aGUg
UlBDDQo+ID4+Pj4gY2FsbC4NCj4gPj4+PiANCj4gPj4+PiAtLSANCj4gPj4+PiBUcm9uZCBNeWts
ZWJ1c3QNCj4gPj4+PiBMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCj4gPj4+PiANCj4gPj4+
PiBOZXRBcHANCj4gPj4+PiBUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0KPiA+Pj4+IHd3dy5u
ZXRhcHAuY29tDQo+ID4+PiANCj4gPj4+IC0tDQo+ID4+PiBUbyB1bnN1YnNjcmliZSBmcm9tIHRo
aXMgbGlzdDogc2VuZCB0aGUgbGluZSAidW5zdWJzY3JpYmUgbGludXgtbmZzIiBpbg0KPiA+Pj4g
dGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtlcm5lbC5vcmcNCj4gPj4+
IE1vcmUgbWFqb3Jkb21vIGluZm8gYXQgIGh0dHA6Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21v
LWluZm8uaHRtbA0KPiA+PiANCj4gPiANCj4gPiAtLSANCj4gPiBUcm9uZCBNeWtsZWJ1c3QNCj4g
PiBMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCj4gPiANCj4gPiBOZXRBcHANCj4gPiBUcm9u
ZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0KPiA+IHd3dy5uZXRhcHAuY29tDQo+IA0KDQotLSANClRy
b25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJv
bmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Adamson, Dros]

Heh, good point ;)

-dros

On Jul 18, 2013, at 4:06 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
 wrote:

> On Thu, 2013-07-18 at 20:04 +0000, Adamson, Dros wrote:
>> Another issue: Is it out of spec to do what we plan and "just use" the nfs_client's rpc_client for all state manager operations without checking if the server supports it from the SP4_MACH_CRED EXCHANGE_ID negotiation?
>> 
>> RENEW for example is not explicitly mentioned in the SP4_MACH_CRED section and I'm looking at a server that doesn't claim state_protection support for RENEW.
> 
> RENEW is a "mandatory to not implement" operation in NFSv4.1.
> 
> Cheers
>  Trond
> 
>> -dros
>> 
>> On Jul 18, 2013, at 3:54 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>> 
>>> On Thu, 2013-07-18 at 19:49 +0000, Adamson, Dros wrote:
>>>> Only supporting operations that have the error code NFS4ERR_WRONG_CRED seems to be wrong.  Operations like BIND_CONN_TO_SESSION don't support don't support this error code, but are explicitly mentioned in SP4_MACH_CRED sections of the spec.
>>> 
>>> Looking at the allowed error return values for BIND_CONN_TO_SESSION, I'm
>>> at a loss to figure out exactly what it should return in this case. I
>>> suspect that the lack of an NFS4ERR_WRONG_CRED is actually a protocol
>>> bug.
>>> 
>>> Time to go back to the ietf mailing list...
>>> 
>>>> I'll continue my work described below modulo limiting to NFS4ERR_WRONG_CRED enabled OPs and see how that looks...
>>>> 
>>>> -dros
>>>> 
>>>> On Jul 18, 2013, at 2:13 PM, "Adamson, Dros" <Weston.Adamson@netapp.com> wrote:
>>>> 
>>>>> Ok, I'll have the client only ask for state protection on operations that support NFS4ERR_WRONG_CRED - and I'll do the SP4_MACH_CRED checks only on these operations and only if they aren't already using the machine cred already. This simplifies my original problem - now I can just use the "main" opcode.
>>>>> 
>>>>> I'll also add error handlers for NFS4ERR_WRONG_CRED.
>>>>> 
>>>>> Thanks!
>>>>> 
>>>>> -dros
>>>>> 
>>>>> On Jul 18, 2013, at 1:08 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>>>> 
>>>>>> On Thu, 2013-07-18 at 15:50 +0000, Adamson, Dros wrote:
>>>>>>> On Jul 18, 2013, at 11:21 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> On Thu, 2013-07-18 at 15:10 +0000, Adamson, Dros wrote:
>>>>>>>>> On Jul 18, 2013, at 10:57 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>>>>>>>> 
>>>>>>>>>> On Thu, 2013-07-18 at 14:46 +0000, Adamson, Dros wrote:
>>>>>>>>>>> Hey,
>>>>>>>>>>> 
>>>>>>>>>>> I have a mostly functional client-side implementation of SP4_MACH_CRED! It still needs a lot of cleanup and testing.
>>>>>>>>>>> 
>>>>>>>>>>> I have one style issue that I want to run by the list before I post a patchset:
>>>>>>>>>>> 
>>>>>>>>>>> So, SP4_MACH_CRED negotiates two bitmaps in the EXCHANGE_ID (one "enforce", one "allow") for state protection. These bitmaps are indexed by the NFSv4 operation number.  The state protect check must happen in the nfs4proc.c layer (or before), right before we call rpc_call_sync or equivalent, so that it can select the right cred and rpc_client.
>>>>>>>>>>> 
>>>>>>>>>>> Here's the problem: we don't know what operations (opcodes) are actually in a compound until the XDR encode callback is called.  The rpc_procinfo array doesn't have this mapping - in fact, it only lives in the xdr encode layer.
>>>>>>>>>>> 
>>>>>>>>>>> One approach is to immediately translate the opcode bitmaps to "nfs4 procedure index" bitmaps, indexing into the rpc_procinfo array.  This would mean there is a second mapping of NFSv4 procedure -> opcodes that must be updated when an XDR encode callback is changed.
>>>>>>>>>>> 
>>>>>>>>>>> Another approach would be to add a callback to the XDR api so we could "ask" it if an NFSv4 procedure contains any of the opcodes in a bitmap. The nice thing about this approach is that the mapping of procedure to opcodes within will live right next to the XDR encode callback and make it obvious that both need to be changed together.
>>>>>>>>>>> 
>>>>>>>>>>> I suppose I'm leaning toward a combination of both of these approaches - keep the mapping in XDR-land and translate the bitmaps immediately on negation for fast lookups during normal operation.
>>>>>>>>>>> 
>>>>>>>>>>> Comments? Am I missing something?
>>>>>>>>>> 
>>>>>>>>>> I'm not sure that I understand. We don't do dynamic creation of
>>>>>>>>>> compounds: we pretty much know in the nfs4_proc_* routine what the main
>>>>>>>>>> operation is (the one exception being CLOSE/OPEN_DOWNGRADE). So why
>>>>>>>>>> can't we work out the protection status at that time? Is the worry that
>>>>>>>>>> the server might reboot and come back with different MACH_CRED
>>>>>>>>>> protections?
>>>>>>>>> 
>>>>>>>>> Sure, we know what the main operation is and we know what operations will end up in a compound - we can just look at the xdr encoder.  My question is simply one of style. Do we want to have each nfs4proc procedure to have a list of operations that must be updated if an xdr encoder is updated? I'm fine with doing it this way, it just seemed wrong to have the same mapping in two different places. This is why I'm asking ;)
>>>>>>>>> 
>>>>>>>>> Also, we can't just use the "main" operation, we must check every operation within the compound and if any are required to use SP4, then the whole compound does. I don't necessarily know why a server would do this, but if we follow the spec and a server informs the client that SETATTR must use state protection, then a WRITE with post-op SETATTR must use the state protection even though WRITE doesn't need it.
>>>>>>>> 
>>>>>>>> We only need to check every _stateful_ operation, right? I can't think
>>>>>>>> of any compounds with more than 1 stateful operation. We've deliberately
>>>>>>>> avoided those due to the problems that arise when you get a
>>>>>>>> NFS4ERR_DELAY or something equivalent at the wrong moment.
>>>>>>> 
>>>>>>> That may be the intention, but I see no such limitation in the spec. It's my understanding that the server could specify any operation as MUST or MAY use state protection.
>>>>>> 
>>>>>> Right, but what would state protection mean for a stateless operation
>>>>>> such as GETATTR? That operation isn't even allowed to return
>>>>>> NFS4ERR_WRONG_CRED, so I don't see how it would apply.
>>>>>> 
>>>>>>>>> I'm not following the reboot question - that case should be handled just fine. Once the EXCHANGE_ID happens, the clp has two bitmaps (enforce and allow) that reflect the current mode of SP4_MACH_CRED.
>>>>>>>> 
>>>>>>>> I'm thinking something along the following scenario:
>>>>>>>> 
>>>>>>>> A process queues up an OPEN. The server replies NFS4ERR_BAD_SESSION and
>>>>>>>> later replies with NFS4ERR_STALE_CLIENTID. So we start reboot recovery,
>>>>>>>> and the OPEN gets queued waiting for a slot.
>>>>>>>> We then send a new EXCHANGE_ID, and the server replies with a
>>>>>>>> _different_ MACH_CRED protection for OPEN.
>>>>>>>> 
>>>>>>>> The problem above is that the OPEN has already been started, and we've
>>>>>>>> already assigned it a credential. How do we handle that?
>>>>>>>> Ditto question for READ, WRITE, LOCK, LAYOUTGET,...
>>>>>>>> 
>>>>>>>> Do we have a problem for LOCKU, CLOSE and OPEN_DOWNGRADE? I think we end
>>>>>>>> up just skipping recovery in those cases…
>>>>>>> 
>>>>>>> These are good points. I'm not sure *why* the sp4_mach_cred config would change, but I suppose it could, and yes, in my current implementation there is no way to back out once a rpc task is invoked.  Any thoughts on this?  I can see two ways of handling this:
>>>>>>> - restarting all tasks (with rpc_clnt and rpc_cred appropriately changed) when a change in sp4_mach_cred config is detected.
>>>>>>> - somehow having the RPC layer call out to check SP4 status before sending each RPC.
>>>>>> 
>>>>>> For now, I think we should make sure that we handle NFS4ERR_WRONG_CRED
>>>>>> correctly in these cases. I don't think we ever want to handle that in
>>>>>> nfs4_async_handle_error, but we should definitely handle it in
>>>>>> nfs4_handle_exception(), since that can drive a full retry of the RPC
>>>>>> call.
>>>>>> 
>>>>>> -- 
>>>>>> Trond Myklebust
>>>>>> Linux NFS client maintainer
>>>>>> 
>>>>>> NetApp
>>>>>> Trond.Myklebust@netapp.com
>>>>>> www.netapp.com
>>>>> 
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> 
>>> 
>>> -- 
>>> Trond Myklebust
>>> Linux NFS client maintainer
>>> 
>>> NetApp
>>> Trond.Myklebust@netapp.com
>>> www.netapp.com
>> 
> 
> -- 
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com

Re: Should BIND_CONN_TO_SESSION be allowed to return NFS4ERR_WRONG_CRED

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDE2OjA0IC0wNDAwLCBUcm9uZCBNeWtsZWJ1c3Qgd3JvdGU6
DQo+IE9uIFRodSwgMjAxMy0wNy0xOCBhdCAxNTo1NCAtMDQwMCwgVHJvbmQgTXlrbGVidXN0IHdy
b3RlOg0KPiA+IE9uIFRodSwgMjAxMy0wNy0xOCBhdCAxOTo0OSArMDAwMCwgQWRhbXNvbiwgRHJv
cyB3cm90ZToNCj4gPiA+IE9ubHkgc3VwcG9ydGluZyBvcGVyYXRpb25zIHRoYXQgaGF2ZSB0aGUg
ZXJyb3IgY29kZSBORlM0RVJSX1dST05HX0NSRUQgc2VlbXMgdG8gYmUgd3JvbmcuICBPcGVyYXRp
b25zIGxpa2UgQklORF9DT05OX1RPX1NFU1NJT04gZG9uJ3Qgc3VwcG9ydCBkb24ndCBzdXBwb3J0
IHRoaXMgZXJyb3IgY29kZSwgYnV0IGFyZSBleHBsaWNpdGx5IG1lbnRpb25lZCBpbiBTUDRfTUFD
SF9DUkVEIHNlY3Rpb25zIG9mIHRoZSBzcGVjLg0KPiA+IA0KPiA+IExvb2tpbmcgYXQgdGhlIGFs
bG93ZWQgZXJyb3IgcmV0dXJuIHZhbHVlcyBmb3IgQklORF9DT05OX1RPX1NFU1NJT04sIEknbQ0K
PiA+IGF0IGEgbG9zcyB0byBmaWd1cmUgb3V0IGV4YWN0bHkgd2hhdCBpdCBzaG91bGQgcmV0dXJu
IGluIHRoaXMgY2FzZS4gSQ0KPiA+IHN1c3BlY3QgdGhhdCB0aGUgbGFjayBvZiBhbiBORlM0RVJS
X1dST05HX0NSRUQgaXMgYWN0dWFsbHkgYSBwcm90b2NvbA0KPiA+IGJ1Zy4NCj4gPiANCj4gPiBU
aW1lIHRvIGdvIGJhY2sgdG8gdGhlIGlldGYgbWFpbGluZyBsaXN0Li4uDQo+IA0KPiBIaSBhbGws
DQo+IA0KPiBXaGVuIGF0dGVtcHRpbmcgdG8gaW1wbGVtZW50IHRoZSBTUDRfTUFDSF9DUkVEIHN0
YXRlIHByb3RlY3Rpb24sIERyb3MNCj4gcmFuIGludG8gYW4gaXNzdWUuIElmIHRoZSBCSU5EX0NP
Tk5fVE9fU0VTU0lPTiBvcGVyYXRpb24gaXMgbGlzdGVkIGluDQo+IHRoZSAic3BvX211c3RfZW5m
b3JjZSIgbGlzdCBvZiBvcGVyYXRpb25zLCB3aGF0IHNob3VsZCBpdCBub3QgYmUgYWxsb3dlZA0K
PiB0byByZXR1cm4gTkZTNEVSUl9XUk9OR19DUkVEIGlmIGNhbGxlZCB3aXRoIGEgY3JlZGVudGlh
bCB0aGF0IGlzIG5vdCB0aGUNCj4gbWFjaGluZSBvciBTU1YgY3JlZGVudGlhbD8NCg0KRGl0dG8g
cXVlc3Rpb24gZm9yIEJBQ0tDSEFOTkVMX0NUTCwgTEFZT1VUR0VULCBURVNUX1NUQVRFSUQsIFJF
QUQgYW5kDQpXUklURS4NCg0KRmluYWxseSwgc2hvdWxkIE9QRU4gYW5kIFdBTlRfREVMRUdBVElP
TiBiZSBhbGxvd2VkIHRvIHJldHVybg0KTkZTNEVSUl9XUk9OR19DUkVEIGluIHRoZSBjYXNlIHdo
ZW4gd2UncmUgZG9pbmcgcmVib290IHJlY292ZXJ5Pw0KDQotLSANClRyb25kIE15a2xlYnVzdA0K
TGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5l
dGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDIwOjA0ICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiBBbm90aGVyIGlzc3VlOiBJcyBpdCBvdXQgb2Ygc3BlYyB0byBkbyB3aGF0IHdlIHBsYW4gYW5k
ICJqdXN0IHVzZSIgdGhlIG5mc19jbGllbnQncyBycGNfY2xpZW50IGZvciBhbGwgc3RhdGUgbWFu
YWdlciBvcGVyYXRpb25zIHdpdGhvdXQgY2hlY2tpbmcgaWYgdGhlIHNlcnZlciBzdXBwb3J0cyBp
dCBmcm9tIHRoZSBTUDRfTUFDSF9DUkVEIEVYQ0hBTkdFX0lEIG5lZ290aWF0aW9uPw0KDQpJdCBz
aG91bGQgYWx3YXlzIGJlIE9LIHRvIHVzZSB0aGUgbWFjaGluZSBjcmVkZW50aWFsIChpLmUuIHRo
ZQ0KY3JlZGVudGlhbCB3ZSB1c2VkIGZvciBFWENIQU5HRV9JRCkgZm9yIG90aGVyIGdsb2JhbCBz
dGF0ZSBvcGVyYXRpb25zDQpzdWNoIGFzIERFU1RST1lfQ0xJRU5USUQsIENSRUFURV9TRVNTSU9O
LCBERVNUUk9ZX1NFU1NJT04sDQpCSU5EX0NPTk5fVE9fU0VTU0lPTiwgQkFDS0NIQU5ORUxfQ1RM
LCBTRVFVRU5DRSwgZXRjLg0KVGhlIHBvaW50IGlzIHRoYXQgaWYgd2UgZG9uJ3QgdXNlIHN0YXRl
IHByb3RlY3Rpb24sIHRoZW4gc29tZW9uZSBlbHNlIGlzDQphbGxvd2VkIHRvIGNvbWUgaW4gdXNp
bmcgYSBkaWZmZXJlbnQgY3JlZGVudGlhbCwgYW5kIGNhbGwgYW55IG9uZSBvZiB0aGUNCmFib3Zl
IG9wZXJhdGlvbnMuDQoNClNvIG91ciBjdXJyZW50IGltcGxlbWVudGF0aW9uIGlzIHF1aXRlIE9L
LiBJdCdzIGp1c3QgdGhhdCB3aXRob3V0IHN0YXRlDQpwcm90ZWN0aW9uLCB3ZSBjYW4ndCBzdG9w
IG90aGVycyBmcm9tIGRlc3Ryb3lpbmcgb3VyIHNlc3Npb24sIHNheS4NCg0KLS0gDQpUcm9uZCBN
eWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25kLk15
a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0K

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Adamson, Dros]

Ok, what about SEQUENCE? Same situation, except this time it's actually a v4.1 op.

-dros

On Jul 18, 2013, at 4:16 PM, "Adamson, Dros" <Weston.Adamson@netapp.com> wrote:

> Heh, good point ;)
> 
> -dros
> 
> On Jul 18, 2013, at 4:06 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
> wrote:
> 
>> On Thu, 2013-07-18 at 20:04 +0000, Adamson, Dros wrote:
>>> Another issue: Is it out of spec to do what we plan and "just use" the nfs_client's rpc_client for all state manager operations without checking if the server supports it from the SP4_MACH_CRED EXCHANGE_ID negotiation?
>>> 
>>> RENEW for example is not explicitly mentioned in the SP4_MACH_CRED section and I'm looking at a server that doesn't claim state_protection support for RENEW.
>> 
>> RENEW is a "mandatory to not implement" operation in NFSv4.1.
>> 
>> Cheers
>> Trond
>> 
>>> -dros
>>> 
>>> On Jul 18, 2013, at 3:54 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>> 
>>>> On Thu, 2013-07-18 at 19:49 +0000, Adamson, Dros wrote:
>>>>> Only supporting operations that have the error code NFS4ERR_WRONG_CRED seems to be wrong.  Operations like BIND_CONN_TO_SESSION don't support don't support this error code, but are explicitly mentioned in SP4_MACH_CRED sections of the spec.
>>>> 
>>>> Looking at the allowed error return values for BIND_CONN_TO_SESSION, I'm
>>>> at a loss to figure out exactly what it should return in this case. I
>>>> suspect that the lack of an NFS4ERR_WRONG_CRED is actually a protocol
>>>> bug.
>>>> 
>>>> Time to go back to the ietf mailing list...
>>>> 
>>>>> I'll continue my work described below modulo limiting to NFS4ERR_WRONG_CRED enabled OPs and see how that looks...
>>>>> 
>>>>> -dros
>>>>> 
>>>>> On Jul 18, 2013, at 2:13 PM, "Adamson, Dros" <Weston.Adamson@netapp.com> wrote:
>>>>> 
>>>>>> Ok, I'll have the client only ask for state protection on operations that support NFS4ERR_WRONG_CRED - and I'll do the SP4_MACH_CRED checks only on these operations and only if they aren't already using the machine cred already. This simplifies my original problem - now I can just use the "main" opcode.
>>>>>> 
>>>>>> I'll also add error handlers for NFS4ERR_WRONG_CRED.
>>>>>> 
>>>>>> Thanks!
>>>>>> 
>>>>>> -dros
>>>>>> 
>>>>>> On Jul 18, 2013, at 1:08 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>>>>> 
>>>>>>> On Thu, 2013-07-18 at 15:50 +0000, Adamson, Dros wrote:
>>>>>>>> On Jul 18, 2013, at 11:21 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> On Thu, 2013-07-18 at 15:10 +0000, Adamson, Dros wrote:
>>>>>>>>>> On Jul 18, 2013, at 10:57 AM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>>>>>>>>> 
>>>>>>>>>>> On Thu, 2013-07-18 at 14:46 +0000, Adamson, Dros wrote:
>>>>>>>>>>>> Hey,
>>>>>>>>>>>> 
>>>>>>>>>>>> I have a mostly functional client-side implementation of SP4_MACH_CRED! It still needs a lot of cleanup and testing.
>>>>>>>>>>>> 
>>>>>>>>>>>> I have one style issue that I want to run by the list before I post a patchset:
>>>>>>>>>>>> 
>>>>>>>>>>>> So, SP4_MACH_CRED negotiates two bitmaps in the EXCHANGE_ID (one "enforce", one "allow") for state protection. These bitmaps are indexed by the NFSv4 operation number.  The state protect check must happen in the nfs4proc.c layer (or before), right before we call rpc_call_sync or equivalent, so that it can select the right cred and rpc_client.
>>>>>>>>>>>> 
>>>>>>>>>>>> Here's the problem: we don't know what operations (opcodes) are actually in a compound until the XDR encode callback is called.  The rpc_procinfo array doesn't have this mapping - in fact, it only lives in the xdr encode layer.
>>>>>>>>>>>> 
>>>>>>>>>>>> One approach is to immediately translate the opcode bitmaps to "nfs4 procedure index" bitmaps, indexing into the rpc_procinfo array.  This would mean there is a second mapping of NFSv4 procedure -> opcodes that must be updated when an XDR encode callback is changed.
>>>>>>>>>>>> 
>>>>>>>>>>>> Another approach would be to add a callback to the XDR api so we could "ask" it if an NFSv4 procedure contains any of the opcodes in a bitmap. The nice thing about this approach is that the mapping of procedure to opcodes within will live right next to the XDR encode callback and make it obvious that both need to be changed together.
>>>>>>>>>>>> 
>>>>>>>>>>>> I suppose I'm leaning toward a combination of both of these approaches - keep the mapping in XDR-land and translate the bitmaps immediately on negation for fast lookups during normal operation.
>>>>>>>>>>>> 
>>>>>>>>>>>> Comments? Am I missing something?
>>>>>>>>>>> 
>>>>>>>>>>> I'm not sure that I understand. We don't do dynamic creation of
>>>>>>>>>>> compounds: we pretty much know in the nfs4_proc_* routine what the main
>>>>>>>>>>> operation is (the one exception being CLOSE/OPEN_DOWNGRADE). So why
>>>>>>>>>>> can't we work out the protection status at that time? Is the worry that
>>>>>>>>>>> the server might reboot and come back with different MACH_CRED
>>>>>>>>>>> protections?
>>>>>>>>>> 
>>>>>>>>>> Sure, we know what the main operation is and we know what operations will end up in a compound - we can just look at the xdr encoder.  My question is simply one of style. Do we want to have each nfs4proc procedure to have a list of operations that must be updated if an xdr encoder is updated? I'm fine with doing it this way, it just seemed wrong to have the same mapping in two different places. This is why I'm asking ;)
>>>>>>>>>> 
>>>>>>>>>> Also, we can't just use the "main" operation, we must check every operation within the compound and if any are required to use SP4, then the whole compound does. I don't necessarily know why a server would do this, but if we follow the spec and a server informs the client that SETATTR must use state protection, then a WRITE with post-op SETATTR must use the state protection even though WRITE doesn't need it.
>>>>>>>>> 
>>>>>>>>> We only need to check every _stateful_ operation, right? I can't think
>>>>>>>>> of any compounds with more than 1 stateful operation. We've deliberately
>>>>>>>>> avoided those due to the problems that arise when you get a
>>>>>>>>> NFS4ERR_DELAY or something equivalent at the wrong moment.
>>>>>>>> 
>>>>>>>> That may be the intention, but I see no such limitation in the spec. It's my understanding that the server could specify any operation as MUST or MAY use state protection.
>>>>>>> 
>>>>>>> Right, but what would state protection mean for a stateless operation
>>>>>>> such as GETATTR? That operation isn't even allowed to return
>>>>>>> NFS4ERR_WRONG_CRED, so I don't see how it would apply.
>>>>>>> 
>>>>>>>>>> I'm not following the reboot question - that case should be handled just fine. Once the EXCHANGE_ID happens, the clp has two bitmaps (enforce and allow) that reflect the current mode of SP4_MACH_CRED.
>>>>>>>>> 
>>>>>>>>> I'm thinking something along the following scenario:
>>>>>>>>> 
>>>>>>>>> A process queues up an OPEN. The server replies NFS4ERR_BAD_SESSION and
>>>>>>>>> later replies with NFS4ERR_STALE_CLIENTID. So we start reboot recovery,
>>>>>>>>> and the OPEN gets queued waiting for a slot.
>>>>>>>>> We then send a new EXCHANGE_ID, and the server replies with a
>>>>>>>>> _different_ MACH_CRED protection for OPEN.
>>>>>>>>> 
>>>>>>>>> The problem above is that the OPEN has already been started, and we've
>>>>>>>>> already assigned it a credential. How do we handle that?
>>>>>>>>> Ditto question for READ, WRITE, LOCK, LAYOUTGET,...
>>>>>>>>> 
>>>>>>>>> Do we have a problem for LOCKU, CLOSE and OPEN_DOWNGRADE? I think we end
>>>>>>>>> up just skipping recovery in those cases…
>>>>>>>> 
>>>>>>>> These are good points. I'm not sure *why* the sp4_mach_cred config would change, but I suppose it could, and yes, in my current implementation there is no way to back out once a rpc task is invoked.  Any thoughts on this?  I can see two ways of handling this:
>>>>>>>> - restarting all tasks (with rpc_clnt and rpc_cred appropriately changed) when a change in sp4_mach_cred config is detected.
>>>>>>>> - somehow having the RPC layer call out to check SP4 status before sending each RPC.
>>>>>>> 
>>>>>>> For now, I think we should make sure that we handle NFS4ERR_WRONG_CRED
>>>>>>> correctly in these cases. I don't think we ever want to handle that in
>>>>>>> nfs4_async_handle_error, but we should definitely handle it in
>>>>>>> nfs4_handle_exception(), since that can drive a full retry of the RPC
>>>>>>> call.
>>>>>>> 
>>>>>>> -- 
>>>>>>> Trond Myklebust
>>>>>>> Linux NFS client maintainer
>>>>>>> 
>>>>>>> NetApp
>>>>>>> Trond.Myklebust@netapp.com
>>>>>>> www.netapp.com
>>>>>> 
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>> 
>>>> 
>>>> -- 
>>>> Trond Myklebust
>>>> Linux NFS client maintainer
>>>> 
>>>> NetApp
>>>> Trond.Myklebust@netapp.com
>>>> www.netapp.com
>>> 
>> 
>> -- 
>> Trond Myklebust
>> Linux NFS client maintainer
>> 
>> NetApp
>> Trond.Myklebust@netapp.com
>> www.netapp.com
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDIwOjQxICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0K
PiBPaywgd2hhdCBhYm91dCBTRVFVRU5DRT8gU2FtZSBzaXR1YXRpb24sIGV4Y2VwdCB0aGlzIHRp
bWUgaXQncyBhY3R1YWxseSBhIHY0LjEgb3AuDQoNClNFUVVFTkNFIGlzIHNwZWNpYWwuIEkgY2Fu
J3Qgc2VlIGhvdyBpdCBpcyBwb3NzaWJsZSB0byBwdXQgaXQgaW4gdGhlDQpzcG9fbXVzdF9lbmZv
cmNlIGNhdGVnb3J5LCBzaW5jZSBpdCBuZWVkcyB0byBwcmVmaXggYWxsIG90aGVyDQpvcGVyYXRp
b25zLg0KSU9XOiB0aGF0IHdvdWxkIGF1dG9tYXRpY2FsbHkgaW1wbHkgdGhhdCB5b3UgY2FuIG9u
bHkgdXNlIHRoZSBtYWNoaW5lDQpjcmVkZW50aWFsIGZvciB0aGF0IE5GUyBzZXJ2ZXIuDQoNCj4g
LWRyb3MNCj4gDQo+IE9uIEp1bCAxOCwgMjAxMywgYXQgNDoxNiBQTSwgIkFkYW1zb24sIERyb3Mi
IDxXZXN0b24uQWRhbXNvbkBuZXRhcHAuY29tPiB3cm90ZToNCj4gDQo+ID4gSGVoLCBnb29kIHBv
aW50IDspDQo+ID4gDQo+ID4gLWRyb3MNCj4gPiANCj4gPiBPbiBKdWwgMTgsIDIwMTMsIGF0IDQ6
MDYgUE0sICJNeWtsZWJ1c3QsIFRyb25kIiA8VHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20+DQo+
ID4gd3JvdGU6DQo+ID4gDQo+ID4+IE9uIFRodSwgMjAxMy0wNy0xOCBhdCAyMDowNCArMDAwMCwg
QWRhbXNvbiwgRHJvcyB3cm90ZToNCj4gPj4+IEFub3RoZXIgaXNzdWU6IElzIGl0IG91dCBvZiBz
cGVjIHRvIGRvIHdoYXQgd2UgcGxhbiBhbmQgImp1c3QgdXNlIiB0aGUgbmZzX2NsaWVudCdzIHJw
Y19jbGllbnQgZm9yIGFsbCBzdGF0ZSBtYW5hZ2VyIG9wZXJhdGlvbnMgd2l0aG91dCBjaGVja2lu
ZyBpZiB0aGUgc2VydmVyIHN1cHBvcnRzIGl0IGZyb20gdGhlIFNQNF9NQUNIX0NSRUQgRVhDSEFO
R0VfSUQgbmVnb3RpYXRpb24/DQo+ID4+PiANCj4gPj4+IFJFTkVXIGZvciBleGFtcGxlIGlzIG5v
dCBleHBsaWNpdGx5IG1lbnRpb25lZCBpbiB0aGUgU1A0X01BQ0hfQ1JFRCBzZWN0aW9uIGFuZCBJ
J20gbG9va2luZyBhdCBhIHNlcnZlciB0aGF0IGRvZXNuJ3QgY2xhaW0gc3RhdGVfcHJvdGVjdGlv
biBzdXBwb3J0IGZvciBSRU5FVy4NCj4gPj4gDQo+ID4+IFJFTkVXIGlzIGEgIm1hbmRhdG9yeSB0
byBub3QgaW1wbGVtZW50IiBvcGVyYXRpb24gaW4gTkZTdjQuMS4NCj4gPj4gDQo+ID4+IENoZWVy
cw0KPiA+PiBUcm9uZA0KPiA+PiANCj4gPj4+IC1kcm9zDQo+ID4+PiANCj4gPj4+IE9uIEp1bCAx
OCwgMjAxMywgYXQgMzo1NCBQTSwgIk15a2xlYnVzdCwgVHJvbmQiIDxUcm9uZC5NeWtsZWJ1c3RA
bmV0YXBwLmNvbT4gd3JvdGU6DQo+ID4+PiANCj4gPj4+PiBPbiBUaHUsIDIwMTMtMDctMTggYXQg
MTk6NDkgKzAwMDAsIEFkYW1zb24sIERyb3Mgd3JvdGU6DQo+ID4+Pj4+IE9ubHkgc3VwcG9ydGlu
ZyBvcGVyYXRpb25zIHRoYXQgaGF2ZSB0aGUgZXJyb3IgY29kZSBORlM0RVJSX1dST05HX0NSRUQg
c2VlbXMgdG8gYmUgd3JvbmcuICBPcGVyYXRpb25zIGxpa2UgQklORF9DT05OX1RPX1NFU1NJT04g
ZG9uJ3Qgc3VwcG9ydCBkb24ndCBzdXBwb3J0IHRoaXMgZXJyb3IgY29kZSwgYnV0IGFyZSBleHBs
aWNpdGx5IG1lbnRpb25lZCBpbiBTUDRfTUFDSF9DUkVEIHNlY3Rpb25zIG9mIHRoZSBzcGVjLg0K
PiA+Pj4+IA0KPiA+Pj4+IExvb2tpbmcgYXQgdGhlIGFsbG93ZWQgZXJyb3IgcmV0dXJuIHZhbHVl
cyBmb3IgQklORF9DT05OX1RPX1NFU1NJT04sIEknbQ0KPiA+Pj4+IGF0IGEgbG9zcyB0byBmaWd1
cmUgb3V0IGV4YWN0bHkgd2hhdCBpdCBzaG91bGQgcmV0dXJuIGluIHRoaXMgY2FzZS4gSQ0KPiA+
Pj4+IHN1c3BlY3QgdGhhdCB0aGUgbGFjayBvZiBhbiBORlM0RVJSX1dST05HX0NSRUQgaXMgYWN0
dWFsbHkgYSBwcm90b2NvbA0KPiA+Pj4+IGJ1Zy4NCj4gPj4+PiANCj4gPj4+PiBUaW1lIHRvIGdv
IGJhY2sgdG8gdGhlIGlldGYgbWFpbGluZyBsaXN0Li4uDQo+ID4+Pj4gDQo+ID4+Pj4+IEknbGwg
Y29udGludWUgbXkgd29yayBkZXNjcmliZWQgYmVsb3cgbW9kdWxvIGxpbWl0aW5nIHRvIE5GUzRF
UlJfV1JPTkdfQ1JFRCBlbmFibGVkIE9QcyBhbmQgc2VlIGhvdyB0aGF0IGxvb2tzLi4uDQo+ID4+
Pj4+IA0KPiA+Pj4+PiAtZHJvcw0KPiA+Pj4+PiANCj4gPj4+Pj4gT24gSnVsIDE4LCAyMDEzLCBh
dCAyOjEzIFBNLCAiQWRhbXNvbiwgRHJvcyIgPFdlc3Rvbi5BZGFtc29uQG5ldGFwcC5jb20+IHdy
b3RlOg0KPiA+Pj4+PiANCj4gPj4+Pj4+IE9rLCBJJ2xsIGhhdmUgdGhlIGNsaWVudCBvbmx5IGFz
ayBmb3Igc3RhdGUgcHJvdGVjdGlvbiBvbiBvcGVyYXRpb25zIHRoYXQgc3VwcG9ydCBORlM0RVJS
X1dST05HX0NSRUQgLSBhbmQgSSdsbCBkbyB0aGUgU1A0X01BQ0hfQ1JFRCBjaGVja3Mgb25seSBv
biB0aGVzZSBvcGVyYXRpb25zIGFuZCBvbmx5IGlmIHRoZXkgYXJlbid0IGFscmVhZHkgdXNpbmcg
dGhlIG1hY2hpbmUgY3JlZCBhbHJlYWR5LiBUaGlzIHNpbXBsaWZpZXMgbXkgb3JpZ2luYWwgcHJv
YmxlbSAtIG5vdyBJIGNhbiBqdXN0IHVzZSB0aGUgIm1haW4iIG9wY29kZS4NCj4gPj4+Pj4+IA0K
PiA+Pj4+Pj4gSSdsbCBhbHNvIGFkZCBlcnJvciBoYW5kbGVycyBmb3IgTkZTNEVSUl9XUk9OR19D
UkVELg0KPiA+Pj4+Pj4gDQo+ID4+Pj4+PiBUaGFua3MhDQo+ID4+Pj4+PiANCj4gPj4+Pj4+IC1k
cm9zDQo+ID4+Pj4+PiANCj4gPj4+Pj4+IE9uIEp1bCAxOCwgMjAxMywgYXQgMTowOCBQTSwgIk15
a2xlYnVzdCwgVHJvbmQiIDxUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbT4gd3JvdGU6DQo+ID4+
Pj4+PiANCj4gPj4+Pj4+PiBPbiBUaHUsIDIwMTMtMDctMTggYXQgMTU6NTAgKzAwMDAsIEFkYW1z
b24sIERyb3Mgd3JvdGU6DQo+ID4+Pj4+Pj4+IE9uIEp1bCAxOCwgMjAxMywgYXQgMTE6MjEgQU0s
ICJNeWtsZWJ1c3QsIFRyb25kIiA8VHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20+DQo+ID4+Pj4+
Pj4+IHdyb3RlOg0KPiA+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+IE9uIFRodSwgMjAxMy0wNy0xOCBh
dCAxNToxMCArMDAwMCwgQWRhbXNvbiwgRHJvcyB3cm90ZToNCj4gPj4+Pj4+Pj4+PiBPbiBKdWwg
MTgsIDIwMTMsIGF0IDEwOjU3IEFNLCAiTXlrbGVidXN0LCBUcm9uZCIgPFRyb25kLk15a2xlYnVz
dEBuZXRhcHAuY29tPiB3cm90ZToNCj4gPj4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+Pj4gT24gVGh1
LCAyMDEzLTA3LTE4IGF0IDE0OjQ2ICswMDAwLCBBZGFtc29uLCBEcm9zIHdyb3RlOg0KPiA+Pj4+
Pj4+Pj4+Pj4gSGV5LA0KPiA+Pj4+Pj4+Pj4+Pj4gDQo+ID4+Pj4+Pj4+Pj4+PiBJIGhhdmUgYSBt
b3N0bHkgZnVuY3Rpb25hbCBjbGllbnQtc2lkZSBpbXBsZW1lbnRhdGlvbiBvZiBTUDRfTUFDSF9D
UkVEISBJdCBzdGlsbCBuZWVkcyBhIGxvdCBvZiBjbGVhbnVwIGFuZCB0ZXN0aW5nLg0KPiA+Pj4+
Pj4+Pj4+Pj4gDQo+ID4+Pj4+Pj4+Pj4+PiBJIGhhdmUgb25lIHN0eWxlIGlzc3VlIHRoYXQgSSB3
YW50IHRvIHJ1biBieSB0aGUgbGlzdCBiZWZvcmUgSSBwb3N0IGEgcGF0Y2hzZXQ6DQo+ID4+Pj4+
Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+Pj4+IFNvLCBTUDRfTUFDSF9DUkVEIG5lZ290aWF0ZXMgdHdv
IGJpdG1hcHMgaW4gdGhlIEVYQ0hBTkdFX0lEIChvbmUgImVuZm9yY2UiLCBvbmUgImFsbG93Iikg
Zm9yIHN0YXRlIHByb3RlY3Rpb24uIFRoZXNlIGJpdG1hcHMgYXJlIGluZGV4ZWQgYnkgdGhlIE5G
U3Y0IG9wZXJhdGlvbiBudW1iZXIuICBUaGUgc3RhdGUgcHJvdGVjdCBjaGVjayBtdXN0IGhhcHBl
biBpbiB0aGUgbmZzNHByb2MuYyBsYXllciAob3IgYmVmb3JlKSwgcmlnaHQgYmVmb3JlIHdlIGNh
bGwgcnBjX2NhbGxfc3luYyBvciBlcXVpdmFsZW50LCBzbyB0aGF0IGl0IGNhbiBzZWxlY3QgdGhl
IHJpZ2h0IGNyZWQgYW5kIHJwY19jbGllbnQuDQo+ID4+Pj4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+
Pj4+IEhlcmUncyB0aGUgcHJvYmxlbTogd2UgZG9uJ3Qga25vdyB3aGF0IG9wZXJhdGlvbnMgKG9w
Y29kZXMpIGFyZSBhY3R1YWxseSBpbiBhIGNvbXBvdW5kIHVudGlsIHRoZSBYRFIgZW5jb2RlIGNh
bGxiYWNrIGlzIGNhbGxlZC4gIFRoZSBycGNfcHJvY2luZm8gYXJyYXkgZG9lc24ndCBoYXZlIHRo
aXMgbWFwcGluZyAtIGluIGZhY3QsIGl0IG9ubHkgbGl2ZXMgaW4gdGhlIHhkciBlbmNvZGUgbGF5
ZXIuDQo+ID4+Pj4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+Pj4+IE9uZSBhcHByb2FjaCBpcyB0byBp
bW1lZGlhdGVseSB0cmFuc2xhdGUgdGhlIG9wY29kZSBiaXRtYXBzIHRvICJuZnM0IHByb2NlZHVy
ZSBpbmRleCIgYml0bWFwcywgaW5kZXhpbmcgaW50byB0aGUgcnBjX3Byb2NpbmZvIGFycmF5LiAg
VGhpcyB3b3VsZCBtZWFuIHRoZXJlIGlzIGEgc2Vjb25kIG1hcHBpbmcgb2YgTkZTdjQgcHJvY2Vk
dXJlIC0+IG9wY29kZXMgdGhhdCBtdXN0IGJlIHVwZGF0ZWQgd2hlbiBhbiBYRFIgZW5jb2RlIGNh
bGxiYWNrIGlzIGNoYW5nZWQuDQo+ID4+Pj4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+Pj4+IEFub3Ro
ZXIgYXBwcm9hY2ggd291bGQgYmUgdG8gYWRkIGEgY2FsbGJhY2sgdG8gdGhlIFhEUiBhcGkgc28g
d2UgY291bGQgImFzayIgaXQgaWYgYW4gTkZTdjQgcHJvY2VkdXJlIGNvbnRhaW5zIGFueSBvZiB0
aGUgb3Bjb2RlcyBpbiBhIGJpdG1hcC4gVGhlIG5pY2UgdGhpbmcgYWJvdXQgdGhpcyBhcHByb2Fj
aCBpcyB0aGF0IHRoZSBtYXBwaW5nIG9mIHByb2NlZHVyZSB0byBvcGNvZGVzIHdpdGhpbiB3aWxs
IGxpdmUgcmlnaHQgbmV4dCB0byB0aGUgWERSIGVuY29kZSBjYWxsYmFjayBhbmQgbWFrZSBpdCBv
YnZpb3VzIHRoYXQgYm90aCBuZWVkIHRvIGJlIGNoYW5nZWQgdG9nZXRoZXIuDQo+ID4+Pj4+Pj4+
Pj4+PiANCj4gPj4+Pj4+Pj4+Pj4+IEkgc3VwcG9zZSBJJ20gbGVhbmluZyB0b3dhcmQgYSBjb21i
aW5hdGlvbiBvZiBib3RoIG9mIHRoZXNlIGFwcHJvYWNoZXMgLSBrZWVwIHRoZSBtYXBwaW5nIGlu
IFhEUi1sYW5kIGFuZCB0cmFuc2xhdGUgdGhlIGJpdG1hcHMgaW1tZWRpYXRlbHkgb24gbmVnYXRp
b24gZm9yIGZhc3QgbG9va3VwcyBkdXJpbmcgbm9ybWFsIG9wZXJhdGlvbi4NCj4gPj4+Pj4+Pj4+
Pj4+IA0KPiA+Pj4+Pj4+Pj4+Pj4gQ29tbWVudHM/IEFtIEkgbWlzc2luZyBzb21ldGhpbmc/DQo+
ID4+Pj4+Pj4+Pj4+IA0KPiA+Pj4+Pj4+Pj4+PiBJJ20gbm90IHN1cmUgdGhhdCBJIHVuZGVyc3Rh
bmQuIFdlIGRvbid0IGRvIGR5bmFtaWMgY3JlYXRpb24gb2YNCj4gPj4+Pj4+Pj4+Pj4gY29tcG91
bmRzOiB3ZSBwcmV0dHkgbXVjaCBrbm93IGluIHRoZSBuZnM0X3Byb2NfKiByb3V0aW5lIHdoYXQg
dGhlIG1haW4NCj4gPj4+Pj4+Pj4+Pj4gb3BlcmF0aW9uIGlzICh0aGUgb25lIGV4Y2VwdGlvbiBi
ZWluZyBDTE9TRS9PUEVOX0RPV05HUkFERSkuIFNvIHdoeQ0KPiA+Pj4+Pj4+Pj4+PiBjYW4ndCB3
ZSB3b3JrIG91dCB0aGUgcHJvdGVjdGlvbiBzdGF0dXMgYXQgdGhhdCB0aW1lPyBJcyB0aGUgd29y
cnkgdGhhdA0KPiA+Pj4+Pj4+Pj4+PiB0aGUgc2VydmVyIG1pZ2h0IHJlYm9vdCBhbmQgY29tZSBi
YWNrIHdpdGggZGlmZmVyZW50IE1BQ0hfQ1JFRA0KPiA+Pj4+Pj4+Pj4+PiBwcm90ZWN0aW9ucz8N
Cj4gPj4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+PiBTdXJlLCB3ZSBrbm93IHdoYXQgdGhlIG1haW4g
b3BlcmF0aW9uIGlzIGFuZCB3ZSBrbm93IHdoYXQgb3BlcmF0aW9ucyB3aWxsIGVuZCB1cCBpbiBh
IGNvbXBvdW5kIC0gd2UgY2FuIGp1c3QgbG9vayBhdCB0aGUgeGRyIGVuY29kZXIuICBNeSBxdWVz
dGlvbiBpcyBzaW1wbHkgb25lIG9mIHN0eWxlLiBEbyB3ZSB3YW50IHRvIGhhdmUgZWFjaCBuZnM0
cHJvYyBwcm9jZWR1cmUgdG8gaGF2ZSBhIGxpc3Qgb2Ygb3BlcmF0aW9ucyB0aGF0IG11c3QgYmUg
dXBkYXRlZCBpZiBhbiB4ZHIgZW5jb2RlciBpcyB1cGRhdGVkPyBJJ20gZmluZSB3aXRoIGRvaW5n
IGl0IHRoaXMgd2F5LCBpdCBqdXN0IHNlZW1lZCB3cm9uZyB0byBoYXZlIHRoZSBzYW1lIG1hcHBp
bmcgaW4gdHdvIGRpZmZlcmVudCBwbGFjZXMuIFRoaXMgaXMgd2h5IEknbSBhc2tpbmcgOykNCj4g
Pj4+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4+PiBBbHNvLCB3ZSBjYW4ndCBqdXN0IHVzZSB0aGUgIm1h
aW4iIG9wZXJhdGlvbiwgd2UgbXVzdCBjaGVjayBldmVyeSBvcGVyYXRpb24gd2l0aGluIHRoZSBj
b21wb3VuZCBhbmQgaWYgYW55IGFyZSByZXF1aXJlZCB0byB1c2UgU1A0LCB0aGVuIHRoZSB3aG9s
ZSBjb21wb3VuZCBkb2VzLiBJIGRvbid0IG5lY2Vzc2FyaWx5IGtub3cgd2h5IGEgc2VydmVyIHdv
dWxkIGRvIHRoaXMsIGJ1dCBpZiB3ZSBmb2xsb3cgdGhlIHNwZWMgYW5kIGEgc2VydmVyIGluZm9y
bXMgdGhlIGNsaWVudCB0aGF0IFNFVEFUVFIgbXVzdCB1c2Ugc3RhdGUgcHJvdGVjdGlvbiwgdGhl
biBhIFdSSVRFIHdpdGggcG9zdC1vcCBTRVRBVFRSIG11c3QgdXNlIHRoZSBzdGF0ZSBwcm90ZWN0
aW9uIGV2ZW4gdGhvdWdoIFdSSVRFIGRvZXNuJ3QgbmVlZCBpdC4NCj4gPj4+Pj4+Pj4+IA0KPiA+
Pj4+Pj4+Pj4gV2Ugb25seSBuZWVkIHRvIGNoZWNrIGV2ZXJ5IF9zdGF0ZWZ1bF8gb3BlcmF0aW9u
LCByaWdodD8gSSBjYW4ndCB0aGluaw0KPiA+Pj4+Pj4+Pj4gb2YgYW55IGNvbXBvdW5kcyB3aXRo
IG1vcmUgdGhhbiAxIHN0YXRlZnVsIG9wZXJhdGlvbi4gV2UndmUgZGVsaWJlcmF0ZWx5DQo+ID4+
Pj4+Pj4+PiBhdm9pZGVkIHRob3NlIGR1ZSB0byB0aGUgcHJvYmxlbXMgdGhhdCBhcmlzZSB3aGVu
IHlvdSBnZXQgYQ0KPiA+Pj4+Pj4+Pj4gTkZTNEVSUl9ERUxBWSBvciBzb21ldGhpbmcgZXF1aXZh
bGVudCBhdCB0aGUgd3JvbmcgbW9tZW50Lg0KPiA+Pj4+Pj4+PiANCj4gPj4+Pj4+Pj4gVGhhdCBt
YXkgYmUgdGhlIGludGVudGlvbiwgYnV0IEkgc2VlIG5vIHN1Y2ggbGltaXRhdGlvbiBpbiB0aGUg
c3BlYy4gSXQncyBteSB1bmRlcnN0YW5kaW5nIHRoYXQgdGhlIHNlcnZlciBjb3VsZCBzcGVjaWZ5
IGFueSBvcGVyYXRpb24gYXMgTVVTVCBvciBNQVkgdXNlIHN0YXRlIHByb3RlY3Rpb24uDQo+ID4+
Pj4+Pj4gDQo+ID4+Pj4+Pj4gUmlnaHQsIGJ1dCB3aGF0IHdvdWxkIHN0YXRlIHByb3RlY3Rpb24g
bWVhbiBmb3IgYSBzdGF0ZWxlc3Mgb3BlcmF0aW9uDQo+ID4+Pj4+Pj4gc3VjaCBhcyBHRVRBVFRS
PyBUaGF0IG9wZXJhdGlvbiBpc24ndCBldmVuIGFsbG93ZWQgdG8gcmV0dXJuDQo+ID4+Pj4+Pj4g
TkZTNEVSUl9XUk9OR19DUkVELCBzbyBJIGRvbid0IHNlZSBob3cgaXQgd291bGQgYXBwbHkuDQo+
ID4+Pj4+Pj4gDQo+ID4+Pj4+Pj4+Pj4gSSdtIG5vdCBmb2xsb3dpbmcgdGhlIHJlYm9vdCBxdWVz
dGlvbiAtIHRoYXQgY2FzZSBzaG91bGQgYmUgaGFuZGxlZCBqdXN0IGZpbmUuIE9uY2UgdGhlIEVY
Q0hBTkdFX0lEIGhhcHBlbnMsIHRoZSBjbHAgaGFzIHR3byBiaXRtYXBzIChlbmZvcmNlIGFuZCBh
bGxvdykgdGhhdCByZWZsZWN0IHRoZSBjdXJyZW50IG1vZGUgb2YgU1A0X01BQ0hfQ1JFRC4NCj4g
Pj4+Pj4+Pj4+IA0KPiA+Pj4+Pj4+Pj4gSSdtIHRoaW5raW5nIHNvbWV0aGluZyBhbG9uZyB0aGUg
Zm9sbG93aW5nIHNjZW5hcmlvOg0KPiA+Pj4+Pj4+Pj4gDQo+ID4+Pj4+Pj4+PiBBIHByb2Nlc3Mg
cXVldWVzIHVwIGFuIE9QRU4uIFRoZSBzZXJ2ZXIgcmVwbGllcyBORlM0RVJSX0JBRF9TRVNTSU9O
IGFuZA0KPiA+Pj4+Pj4+Pj4gbGF0ZXIgcmVwbGllcyB3aXRoIE5GUzRFUlJfU1RBTEVfQ0xJRU5U
SUQuIFNvIHdlIHN0YXJ0IHJlYm9vdCByZWNvdmVyeSwNCj4gPj4+Pj4+Pj4+IGFuZCB0aGUgT1BF
TiBnZXRzIHF1ZXVlZCB3YWl0aW5nIGZvciBhIHNsb3QuDQo+ID4+Pj4+Pj4+PiBXZSB0aGVuIHNl
bmQgYSBuZXcgRVhDSEFOR0VfSUQsIGFuZCB0aGUgc2VydmVyIHJlcGxpZXMgd2l0aCBhDQo+ID4+
Pj4+Pj4+PiBfZGlmZmVyZW50XyBNQUNIX0NSRUQgcHJvdGVjdGlvbiBmb3IgT1BFTi4NCj4gPj4+
Pj4+Pj4+IA0KPiA+Pj4+Pj4+Pj4gVGhlIHByb2JsZW0gYWJvdmUgaXMgdGhhdCB0aGUgT1BFTiBo
YXMgYWxyZWFkeSBiZWVuIHN0YXJ0ZWQsIGFuZCB3ZSd2ZQ0KPiA+Pj4+Pj4+Pj4gYWxyZWFkeSBh
c3NpZ25lZCBpdCBhIGNyZWRlbnRpYWwuIEhvdyBkbyB3ZSBoYW5kbGUgdGhhdD8NCj4gPj4+Pj4+
Pj4+IERpdHRvIHF1ZXN0aW9uIGZvciBSRUFELCBXUklURSwgTE9DSywgTEFZT1VUR0VULC4uLg0K
PiA+Pj4+Pj4+Pj4gDQo+ID4+Pj4+Pj4+PiBEbyB3ZSBoYXZlIGEgcHJvYmxlbSBmb3IgTE9DS1Us
IENMT1NFIGFuZCBPUEVOX0RPV05HUkFERT8gSSB0aGluayB3ZSBlbmQNCj4gPj4+Pj4+Pj4+IHVw
IGp1c3Qgc2tpcHBpbmcgcmVjb3ZlcnkgaW4gdGhvc2UgY2FzZXPigKYNCj4gPj4+Pj4+Pj4gDQo+
ID4+Pj4+Pj4+IFRoZXNlIGFyZSBnb29kIHBvaW50cy4gSSdtIG5vdCBzdXJlICp3aHkqIHRoZSBz
cDRfbWFjaF9jcmVkIGNvbmZpZyB3b3VsZCBjaGFuZ2UsIGJ1dCBJIHN1cHBvc2UgaXQgY291bGQs
IGFuZCB5ZXMsIGluIG15IGN1cnJlbnQgaW1wbGVtZW50YXRpb24gdGhlcmUgaXMgbm8gd2F5IHRv
IGJhY2sgb3V0IG9uY2UgYSBycGMgdGFzayBpcyBpbnZva2VkLiAgQW55IHRob3VnaHRzIG9uIHRo
aXM/ICBJIGNhbiBzZWUgdHdvIHdheXMgb2YgaGFuZGxpbmcgdGhpczoNCj4gPj4+Pj4+Pj4gLSBy
ZXN0YXJ0aW5nIGFsbCB0YXNrcyAod2l0aCBycGNfY2xudCBhbmQgcnBjX2NyZWQgYXBwcm9wcmlh
dGVseSBjaGFuZ2VkKSB3aGVuIGEgY2hhbmdlIGluIHNwNF9tYWNoX2NyZWQgY29uZmlnIGlzIGRl
dGVjdGVkLg0KPiA+Pj4+Pj4+PiAtIHNvbWVob3cgaGF2aW5nIHRoZSBSUEMgbGF5ZXIgY2FsbCBv
dXQgdG8gY2hlY2sgU1A0IHN0YXR1cyBiZWZvcmUgc2VuZGluZyBlYWNoIFJQQy4NCj4gPj4+Pj4+
PiANCj4gPj4+Pj4+PiBGb3Igbm93LCBJIHRoaW5rIHdlIHNob3VsZCBtYWtlIHN1cmUgdGhhdCB3
ZSBoYW5kbGUgTkZTNEVSUl9XUk9OR19DUkVEDQo+ID4+Pj4+Pj4gY29ycmVjdGx5IGluIHRoZXNl
IGNhc2VzLiBJIGRvbid0IHRoaW5rIHdlIGV2ZXIgd2FudCB0byBoYW5kbGUgdGhhdCBpbg0KPiA+
Pj4+Pj4+IG5mczRfYXN5bmNfaGFuZGxlX2Vycm9yLCBidXQgd2Ugc2hvdWxkIGRlZmluaXRlbHkg
aGFuZGxlIGl0IGluDQo+ID4+Pj4+Pj4gbmZzNF9oYW5kbGVfZXhjZXB0aW9uKCksIHNpbmNlIHRo
YXQgY2FuIGRyaXZlIGEgZnVsbCByZXRyeSBvZiB0aGUgUlBDDQo+ID4+Pj4+Pj4gY2FsbC4NCj4g
Pj4+Pj4+PiANCj4gPj4+Pj4+PiAtLSANCj4gPj4+Pj4+PiBUcm9uZCBNeWtsZWJ1c3QNCj4gPj4+
Pj4+PiBMaW51eCBORlMgY2xpZW50IG1haW50YWluZXINCj4gPj4+Pj4+PiANCj4gPj4+Pj4+PiBO
ZXRBcHANCj4gPj4+Pj4+PiBUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0KPiA+Pj4+Pj4+IHd3
dy5uZXRhcHAuY29tDQo+ID4+Pj4+PiANCj4gPj4+Pj4+IC0tDQo+ID4+Pj4+PiBUbyB1bnN1YnNj
cmliZSBmcm9tIHRoaXMgbGlzdDogc2VuZCB0aGUgbGluZSAidW5zdWJzY3JpYmUgbGludXgtbmZz
IiBpbg0KPiA+Pj4+Pj4gdGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtl
cm5lbC5vcmcNCj4gPj4+Pj4+IE1vcmUgbWFqb3Jkb21vIGluZm8gYXQgIGh0dHA6Ly92Z2VyLmtl
cm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0KPiA+Pj4+PiANCj4gPj4+PiANCj4gPj4+PiAt
LSANCj4gPj4+PiBUcm9uZCBNeWtsZWJ1c3QNCj4gPj4+PiBMaW51eCBORlMgY2xpZW50IG1haW50
YWluZXINCj4gPj4+PiANCj4gPj4+PiBOZXRBcHANCj4gPj4+PiBUcm9uZC5NeWtsZWJ1c3RAbmV0
YXBwLmNvbQ0KPiA+Pj4+IHd3dy5uZXRhcHAuY29tDQo+ID4+PiANCj4gPj4gDQo+ID4+IC0tIA0K
PiA+PiBUcm9uZCBNeWtsZWJ1c3QNCj4gPj4gTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQo+
ID4+IA0KPiA+PiBOZXRBcHANCj4gPj4gVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCj4gPj4g
d3d3Lm5ldGFwcC5jb20NCj4gPiANCj4gPiAtLQ0KPiA+IFRvIHVuc3Vic2NyaWJlIGZyb20gdGhp
cyBsaXN0OiBzZW5kIHRoZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51eC1uZnMiIGluDQo+ID4gdGhl
IGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtlcm5lbC5vcmcNCj4gPiBNb3Jl
IG1ham9yZG9tbyBpbmZvIGF0ICBodHRwOi8vdmdlci5rZXJuZWwub3JnL21ham9yZG9tby1pbmZv
Lmh0bWwNCj4gDQo+IC0tDQo+IFRvIHVuc3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0OiBzZW5kIHRo
ZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51eC1uZnMiIGluDQo+IHRoZSBib2R5IG9mIGEgbWVzc2Fn
ZSB0byBtYWpvcmRvbW9Admdlci5rZXJuZWwub3JnDQo+IE1vcmUgbWFqb3Jkb21vIGluZm8gYXQg
IGh0dHA6Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0KDQotLSANClRyb25k
IE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQu
TXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

DQpPbiBUaHUsIDIwMTMtMDctMTggYXQgMjA6NDEgKzAwMDAsIEFkYW1zb24sIERyb3Mgd3JvdGU6
DQo+IE9rLCB3aGF0IGFib3V0IFNFUVVFTkNFPyBTYW1lIHNpdHVhdGlvbiwgZXhjZXB0IHRoaXMg
dGltZSBpdCdzIGFjdHVhbGx5IGEgdjQuMSBvcC4NCg0KU0VRVUVOQ0UgaXMgc3BlY2lhbC4gSSBj
YW4ndCBzZWUgaG93IGl0IGlzIHBvc3NpYmxlIHRvIHB1dCBpdCBpbiB0aGUNCnNwb19tdXN0X2Vu
Zm9yY2UgY2F0ZWdvcnksIHNpbmNlIGl0IG5lZWRzIHRvIHByZWZpeCBhbGwgb3RoZXINCm9wZXJh
dGlvbnMuDQpJT1c6IHRoYXQgd291bGQgYXV0b21hdGljYWxseSBpbXBseSB0aGF0IHlvdSBjYW4g
b25seSB1c2UgdGhlIG1hY2hpbmUNCmNyZWRlbnRpYWwgZm9yIHRoYXQgTkZTIHNlcnZlci4NCg0K
DQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5l
dEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

Re: SP4_MACH_CRED: v4 proc -> opcodes mapping

[Myklebust, Trond]

T24gVGh1LCAyMDEzLTA3LTE4IGF0IDIwOjQ5ICswMDAwLCBNeWtsZWJ1c3QsIFRyb25kIHdyb3Rl
Og0KPiBPbiBUaHUsIDIwMTMtMDctMTggYXQgMjA6NDEgKzAwMDAsIEFkYW1zb24sIERyb3Mgd3Jv
dGU6DQo+ID4gT2ssIHdoYXQgYWJvdXQgU0VRVUVOQ0U/IFNhbWUgc2l0dWF0aW9uLCBleGNlcHQg
dGhpcyB0aW1lIGl0J3MgYWN0dWFsbHkgYSB2NC4xIG9wLg0KPiANCj4gU0VRVUVOQ0UgaXMgc3Bl
Y2lhbC4gSSBjYW4ndCBzZWUgaG93IGl0IGlzIHBvc3NpYmxlIHRvIHB1dCBpdCBpbiB0aGUNCj4g
c3BvX211c3RfZW5mb3JjZSBjYXRlZ29yeSwgc2luY2UgaXQgbmVlZHMgdG8gcHJlZml4IGFsbCBv
dGhlcg0KPiBvcGVyYXRpb25zLg0KPiBJT1c6IHRoYXQgd291bGQgYXV0b21hdGljYWxseSBpbXBs
eSB0aGF0IHlvdSBjYW4gb25seSB1c2UgdGhlIG1hY2hpbmUNCj4gY3JlZGVudGlhbCBmb3IgdGhh
dCBORlMgc2VydmVyLg0KDQpBcG9sb2dpZXMgZm9yIHRoZSBkdXBsaWNhdGUsIGJ0dy4gRXZvbHV0
aW9uIGNyYXNoZWQgcmlnaHQgaW4gdGhlIG1pZGRsZQ0Kb2YgdGhlIHRyYW5zbWlzc2lvbiBvZiB0
aGUgZmlyc3QgZW1haWwuDQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBt
YWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRh
cHAuY29tDQo=

Re: [nfsv4] Should BIND_CONN_TO_SESSION be allowed to return NFS4ERR_WRONG_CRED

[J. Bruce Fields]

On Thu, Jul 18, 2013 at 08:04:58PM +0000, Myklebust, Trond wrote:
> On Thu, 2013-07-18 at 15:54 -0400, Trond Myklebust wrote:
> > On Thu, 2013-07-18 at 19:49 +0000, Adamson, Dros wrote:
> > > Only supporting operations that have the error code NFS4ERR_WRONG_CRED seems to be wrong.  Operations like BIND_CONN_TO_SESSION don't support don't support this error code, but are explicitly mentioned in SP4_MACH_CRED sections of the spec.
> > 
> > Looking at the allowed error return values for BIND_CONN_TO_SESSION, I'm
> > at a loss to figure out exactly what it should return in this case. I
> > suspect that the lack of an NFS4ERR_WRONG_CRED is actually a protocol
> > bug.
> > 
> > Time to go back to the ietf mailing list...
> 
> Hi all,
> 
> When attempting to implement the SP4_MACH_CRED state protection, Dros
> ran into an issue. If the BIND_CONN_TO_SESSION operation is listed in
> the "spo_must_enforce" list of operations, what should it not be allowed
> to return NFS4ERR_WRONG_CRED if called with a credential that is not the
> machine or SSV credential?

For what it's worth, the Linux server is returning WRONG_CRED, as you'd
expect, in this case.  Looks to me like a simple omission from the spec.

--b.