[PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[andros]

From: Andy Adamson <andros@netapp.com>

max_session_slots is a ushort. Bump NFS4_DEF_SLOT_TABLE_SIZE to the max ushort
value: e.g. ask for 256 slots and let the server negotiate down if needed.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4session.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/nfs/nfs4session.h b/fs/nfs/nfs4session.h
index 3a153d8..8b7899f 100644
--- a/fs/nfs/nfs4session.h
+++ b/fs/nfs/nfs4session.h
@@ -8,7 +8,7 @@
 #define __LINUX_FS_NFS_NFS4SESSION_H
 
 /* maximum number of slots to use */
-#define NFS4_DEF_SLOT_TABLE_SIZE (16U)
+#define NFS4_DEF_SLOT_TABLE_SIZE (256U)
 #define NFS4_MAX_SLOT_TABLE (1024U)
 #define NFS4_NO_SLOT ((u32)-1)
 
-- 
1.8.3.1

Re: [PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[Chuck Lever]

Hi Andy-

On Jul 19, 2013, at 5:09 PM, andros@netapp.com wrote:

> From: Andy Adamson <andros@netapp.com>
> 
> max_session_slots is a ushort. Bump NFS4_DEF_SLOT_TABLE_SIZE to the max ushort
> value: e.g. ask for 256 slots and let the server negotiate down if needed.

I don't have an objection to your patch, but the description is confusing.

In fs/nfs/super.c I see

   unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;

but /usr/include/limits.h has

   #define USHRT_MAX 65535

The maximum value you can store in an unsigned octet is 255 (UCHAR_MAX).

Why did you choose 256 and not 65535?


> Signed-off-by: Andy Adamson <andros@netapp.com>
> ---
> fs/nfs/nfs4session.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/nfs/nfs4session.h b/fs/nfs/nfs4session.h
> index 3a153d8..8b7899f 100644
> --- a/fs/nfs/nfs4session.h
> +++ b/fs/nfs/nfs4session.h
> @@ -8,7 +8,7 @@
> #define __LINUX_FS_NFS_NFS4SESSION_H
> 
> /* maximum number of slots to use */
> -#define NFS4_DEF_SLOT_TABLE_SIZE (16U)
> +#define NFS4_DEF_SLOT_TABLE_SIZE (256U)
> #define NFS4_MAX_SLOT_TABLE (1024U)
> #define NFS4_NO_SLOT ((u32)-1)
> 
> -- 
> 1.8.3.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

RE: [PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[Myklebust, Trond]

> -----Original Message-----
> From: linux-nfs-owner@vger.kernel.org [mailto:linux-nfs-
> owner@vger.kernel.org] On Behalf Of Chuck Lever
> Sent: Friday, July 19, 2013 7:57 PM
> To: Adamson, Andy
> Cc: Linux NFS Mailing List
> Subject: Re: [PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE
> 
> Hi Andy-
> 
> On Jul 19, 2013, at 5:09 PM, andros@netapp.com wrote:
> 
> > From: Andy Adamson <andros@netapp.com>
> >
> > max_session_slots is a ushort. Bump NFS4_DEF_SLOT_TABLE_SIZE to the
> > max ushort
> > value: e.g. ask for 256 slots and let the server negotiate down if needed.
> 
> I don't have an objection to your patch, but the description is confusing.
> 
> In fs/nfs/super.c I see
> 
>    unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;
> 
> but /usr/include/limits.h has
> 
>    #define USHRT_MAX 65535
> 
> The maximum value you can store in an unsigned octet is 255 (UCHAR_MAX).
> 
> Why did you choose 256 and not 65535?

This is the _minimum_ number of slots that the client and server will have to allocate, even when they both support dynamic slots, so we don't want to set it unreasonably high.
I realise that most servers will negotiate this value down, but we don't want to rely on that.

Cheers
  Trond

Re: [PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[Myklebust, Trond]

T24gRnJpLCAyMDEzLTA3LTE5IGF0IDE3OjA5IC0wNDAwLCBhbmRyb3NAbmV0YXBwLmNvbSB3cm90
ZToNCj4gRnJvbTogQW5keSBBZGFtc29uIDxhbmRyb3NAbmV0YXBwLmNvbT4NCj4gDQo+IG1heF9z
ZXNzaW9uX3Nsb3RzIGlzIGEgdXNob3J0LiBCdW1wIE5GUzRfREVGX1NMT1RfVEFCTEVfU0laRSB0
byB0aGUgbWF4IHVzaG9ydA0KPiB2YWx1ZTogZS5nLiBhc2sgZm9yIDI1NiBzbG90cyBhbmQgbGV0
IHRoZSBzZXJ2ZXIgbmVnb3RpYXRlIGRvd24gaWYgbmVlZGVkLg0KPiANCj4gU2lnbmVkLW9mZi1i
eTogQW5keSBBZGFtc29uIDxhbmRyb3NAbmV0YXBwLmNvbT4NCj4gLS0tDQo+ICBmcy9uZnMvbmZz
NHNlc3Npb24uaCB8IDIgKy0NCj4gIDEgZmlsZSBjaGFuZ2VkLCAxIGluc2VydGlvbigrKSwgMSBk
ZWxldGlvbigtKQ0KPiANCj4gZGlmZiAtLWdpdCBhL2ZzL25mcy9uZnM0c2Vzc2lvbi5oIGIvZnMv
bmZzL25mczRzZXNzaW9uLmgNCj4gaW5kZXggM2ExNTNkOC4uOGI3ODk5ZiAxMDA2NDQNCj4gLS0t
IGEvZnMvbmZzL25mczRzZXNzaW9uLmgNCj4gKysrIGIvZnMvbmZzL25mczRzZXNzaW9uLmgNCj4g
QEAgLTgsNyArOCw3IEBADQo+ICAjZGVmaW5lIF9fTElOVVhfRlNfTkZTX05GUzRTRVNTSU9OX0gN
Cj4gIA0KPiAgLyogbWF4aW11bSBudW1iZXIgb2Ygc2xvdHMgdG8gdXNlICovDQo+IC0jZGVmaW5l
IE5GUzRfREVGX1NMT1RfVEFCTEVfU0laRSAoMTZVKQ0KPiArI2RlZmluZSBORlM0X0RFRl9TTE9U
X1RBQkxFX1NJWkUgKDI1NlUpDQoNCkNvdWxkIHdlIHBsZWFzZSBtYWtlIHRoaXMgc21hbGxlcj8g
SSBhZ3JlZSB0aGF0IDE2IGlzIHRvbyBzbWFsbCAoYXMgbG9uZw0KYXMgc2VydmVycyBkb24ndCBk
byBkeW5hbWljIHNsb3QgYWxsb2NhdGlvbiksIGJ1dCAyNTYgaXMgdmVyeSBoaWdoIGZvciBhDQpk
ZWZhdWx0Lg0KDQpIb3cgYWJvdXQgNjQ/DQoNCj4gICNkZWZpbmUgTkZTNF9NQVhfU0xPVF9UQUJM
RSAoMTAyNFUpDQo+ICAjZGVmaW5lIE5GUzRfTk9fU0xPVCAoKHUzMiktMSkNCj4gIA0KDQotLSAN
ClRyb25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0K
VHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

Re: [PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[Adamson, Andy]

On Jul 22, 2013, at 2:36 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
 wrote:

> On Fri, 2013-07-19 at 17:09 -0400, andros@netapp.com wrote:
>> From: Andy Adamson <andros@netapp.com>
>> 
>> max_session_slots is a ushort. Bump NFS4_DEF_SLOT_TABLE_SIZE to the max ushort
>> value: e.g. ask for 256 slots and let the server negotiate down if needed.
>> 
>> Signed-off-by: Andy Adamson <andros@netapp.com>
>> ---
>> fs/nfs/nfs4session.h | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/fs/nfs/nfs4session.h b/fs/nfs/nfs4session.h
>> index 3a153d8..8b7899f 100644
>> --- a/fs/nfs/nfs4session.h
>> +++ b/fs/nfs/nfs4session.h
>> @@ -8,7 +8,7 @@
>> #define __LINUX_FS_NFS_NFS4SESSION_H
>> 
>> /* maximum number of slots to use */
>> -#define NFS4_DEF_SLOT_TABLE_SIZE (16U)
>> +#define NFS4_DEF_SLOT_TABLE_SIZE (256U)
> 
> Could we please make this smaller? I agree that 16 is too small (as long
> as servers don't do dynamic slot allocation), but 256 is very high for a
> default.
> 
> How about 64?

IIIRC the (non-dynamic) session implemenations that I've seen treat the ca_maxrequests coming from the client as the maximum number of session slots that the client can handle - IOW I don't see servers returning a higher ca_maxrequests than was sent by the client. This is why I chose a larger number for the default. It is, after all, a server resource and shouldn't the client grab as many session slots as it can?

-->Andy

> 
>> #define NFS4_MAX_SLOT_TABLE (1024U)
>> #define NFS4_NO_SLOT ((u32)-1)
>> 
> 
> -- 
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com

Re: [PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[Myklebust, Trond]

T24gTW9uLCAyMDEzLTA3LTIyIGF0IDE5OjAyICswMDAwLCBBZGFtc29uLCBBbmR5IHdyb3RlOg0K
PiBPbiBKdWwgMjIsIDIwMTMsIGF0IDI6MzYgUE0sICJNeWtsZWJ1c3QsIFRyb25kIiA8VHJvbmQu
TXlrbGVidXN0QG5ldGFwcC5jb20+DQo+ICB3cm90ZToNCj4gDQo+ID4gT24gRnJpLCAyMDEzLTA3
LTE5IGF0IDE3OjA5IC0wNDAwLCBhbmRyb3NAbmV0YXBwLmNvbSB3cm90ZToNCj4gPj4gRnJvbTog
QW5keSBBZGFtc29uIDxhbmRyb3NAbmV0YXBwLmNvbT4NCj4gPj4gDQo+ID4+IG1heF9zZXNzaW9u
X3Nsb3RzIGlzIGEgdXNob3J0LiBCdW1wIE5GUzRfREVGX1NMT1RfVEFCTEVfU0laRSB0byB0aGUg
bWF4IHVzaG9ydA0KPiA+PiB2YWx1ZTogZS5nLiBhc2sgZm9yIDI1NiBzbG90cyBhbmQgbGV0IHRo
ZSBzZXJ2ZXIgbmVnb3RpYXRlIGRvd24gaWYgbmVlZGVkLg0KPiA+PiANCj4gPj4gU2lnbmVkLW9m
Zi1ieTogQW5keSBBZGFtc29uIDxhbmRyb3NAbmV0YXBwLmNvbT4NCj4gPj4gLS0tDQo+ID4+IGZz
L25mcy9uZnM0c2Vzc2lvbi5oIHwgMiArLQ0KPiA+PiAxIGZpbGUgY2hhbmdlZCwgMSBpbnNlcnRp
b24oKyksIDEgZGVsZXRpb24oLSkNCj4gPj4gDQo+ID4+IGRpZmYgLS1naXQgYS9mcy9uZnMvbmZz
NHNlc3Npb24uaCBiL2ZzL25mcy9uZnM0c2Vzc2lvbi5oDQo+ID4+IGluZGV4IDNhMTUzZDguLjhi
Nzg5OWYgMTAwNjQ0DQo+ID4+IC0tLSBhL2ZzL25mcy9uZnM0c2Vzc2lvbi5oDQo+ID4+ICsrKyBi
L2ZzL25mcy9uZnM0c2Vzc2lvbi5oDQo+ID4+IEBAIC04LDcgKzgsNyBAQA0KPiA+PiAjZGVmaW5l
IF9fTElOVVhfRlNfTkZTX05GUzRTRVNTSU9OX0gNCj4gPj4gDQo+ID4+IC8qIG1heGltdW0gbnVt
YmVyIG9mIHNsb3RzIHRvIHVzZSAqLw0KPiA+PiAtI2RlZmluZSBORlM0X0RFRl9TTE9UX1RBQkxF
X1NJWkUgKDE2VSkNCj4gPj4gKyNkZWZpbmUgTkZTNF9ERUZfU0xPVF9UQUJMRV9TSVpFICgyNTZV
KQ0KPiA+IA0KPiA+IENvdWxkIHdlIHBsZWFzZSBtYWtlIHRoaXMgc21hbGxlcj8gSSBhZ3JlZSB0
aGF0IDE2IGlzIHRvbyBzbWFsbCAoYXMgbG9uZw0KPiA+IGFzIHNlcnZlcnMgZG9uJ3QgZG8gZHlu
YW1pYyBzbG90IGFsbG9jYXRpb24pLCBidXQgMjU2IGlzIHZlcnkgaGlnaCBmb3IgYQ0KPiA+IGRl
ZmF1bHQuDQo+ID4gDQo+ID4gSG93IGFib3V0IDY0Pw0KPiANCj4gSUlJUkMgdGhlIChub24tZHlu
YW1pYykgc2Vzc2lvbiBpbXBsZW1lbmF0aW9ucyB0aGF0IEkndmUgc2VlbiB0cmVhdCB0aGUgY2Ff
bWF4cmVxdWVzdHMgY29taW5nIGZyb20gdGhlIGNsaWVudCBhcyB0aGUgbWF4aW11bSBudW1iZXIg
b2Ygc2Vzc2lvbiBzbG90cyB0aGF0IHRoZSBjbGllbnQgY2FuIGhhbmRsZSAtIElPVyBJIGRvbid0
IHNlZSBzZXJ2ZXJzIHJldHVybmluZyBhIGhpZ2hlciBjYV9tYXhyZXF1ZXN0cyB0aGFuIHdhcyBz
ZW50IGJ5IHRoZSBjbGllbnQuIFRoaXMgaXMgd2h5IEkgY2hvc2UgYSBsYXJnZXIgbnVtYmVyIGZv
ciB0aGUgZGVmYXVsdC4gSXQgaXMsIGFmdGVyIGFsbCwgYSBzZXJ2ZXIgcmVzb3VyY2UgYW5kIHNo
b3VsZG4ndCB0aGUgY2xpZW50IGdyYWIgYXMgbWFueSBzZXNzaW9uIHNsb3RzIGFzIGl0IGNhbj8N
Cg0KVGhlIHByb2JsZW0gaXMgdGhhdCBvbmNlIHlvdSBpbXBsZW1lbnQgZHluYW1pYyBzbG90IHRh
YmxlcywgdGhlDQpuZWdvdGlhdGVkIHNlc3Npb24gc2xvdCB0YWJsZSBzaXplIGJlY29tZXMgdGhl
IF9taW5pbXVtXyBudW1iZXIgb2YNCnNsb3RzLiBUaGF0J3Mgd2h5IGEgY2xpZW50IHRoYXQgc3Vw
cG9ydHMgZHluYW1pYyBzbG90IHRhYmxlcyB3YW50cyB0bw0Ka2VlcCB0aGUgY2FfbWF4cmVxdWVz
dHMgX3NtYWxsXyBpbiB0aGUgaG9wZSB0aGF0IHRoZSBzZXJ2ZXIgY2FuIHRoZW4NCmR5bmFtaWNh
bGx5IGFkanVzdCB0aGUgdG90YWwgbnVtYmVyIG9mIHNsb3RzIHVwd2FyZHMgaW4gb3JkZXIgdG8g
bWVldA0KdGhlIGFjdHVhbCBsb2FkLg0KDQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXggTkZT
IGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20N
Cnd3dy5uZXRhcHAuY29tDQo=

Re: [PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[Adamson, Andy]

On Jul 22, 2013, at 3:17 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
 wrote:

> On Mon, 2013-07-22 at 19:02 +0000, Adamson, Andy wrote:
>> On Jul 22, 2013, at 2:36 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com>
>> wrote:
>> 
>>> On Fri, 2013-07-19 at 17:09 -0400, andros@netapp.com wrote:
>>>> From: Andy Adamson <andros@netapp.com>
>>>> 
>>>> max_session_slots is a ushort. Bump NFS4_DEF_SLOT_TABLE_SIZE to the max ushort
>>>> value: e.g. ask for 256 slots and let the server negotiate down if needed.
>>>> 
>>>> Signed-off-by: Andy Adamson <andros@netapp.com>
>>>> ---
>>>> fs/nfs/nfs4session.h | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>> 
>>>> diff --git a/fs/nfs/nfs4session.h b/fs/nfs/nfs4session.h
>>>> index 3a153d8..8b7899f 100644
>>>> --- a/fs/nfs/nfs4session.h
>>>> +++ b/fs/nfs/nfs4session.h
>>>> @@ -8,7 +8,7 @@
>>>> #define __LINUX_FS_NFS_NFS4SESSION_H
>>>> 
>>>> /* maximum number of slots to use */
>>>> -#define NFS4_DEF_SLOT_TABLE_SIZE (16U)
>>>> +#define NFS4_DEF_SLOT_TABLE_SIZE (256U)
>>> 
>>> Could we please make this smaller? I agree that 16 is too small (as long
>>> as servers don't do dynamic slot allocation), but 256 is very high for a
>>> default.
>>> 
>>> How about 64?
>> 
>> IIIRC the (non-dynamic) session implemenations that I've seen treat the ca_maxrequests coming from the client as the maximum number of session slots that the client can handle - IOW I don't see servers returning a higher ca_maxrequests than was sent by the client. This is why I chose a larger number for the default. It is, after all, a server resource and shouldn't the client grab as many session slots as it can?
> 

64 is OK with me - it's large enough to handle most workloads.  I think no matter what we choose we will have to note this as a performance parameter to adjust until the dynamic session implementations come on line.

> The problem is that once you implement dynamic slot tables, the
> negotiated session slot table size becomes the _minimum_ number of
> slots.

Could you explain this a bit more? I'd like to understand your reasoning.  I can see that the dynamic session goal is to end up with the highest_slotid value to be just high enough to meet the actual load, but not waste server slot resources. But I don't understand why the client value of ca_maxrequests makes a difference in the dynamic sessions case because whether the client starts high or low, the server will adjust the value to one that fits the load.

The initial (ca_maxrequests) value is a  guess for both the client and the server, until actual load occurs. So any initial value could be way off.  Even with 16 slots, if the load is mainly metadata, the highest_used_slotid could easily be 4. Or you could start off with 64 slots and need 150.

> That's why a client that supports dynamic slot tables wants to
> keep the ca_maxrequests _small_ in the hope that the server can then
> dynamically adjust the total number of slots upwards in order to meet
> the actual load.


I would think the client that supports dynamic sessions would want to keep the ca_maxrequests high so that if the actual load is high right out of the gate, the application doesn't have an initial slow start waiting for the server to dynamically bump up the slots. If the server wants to start the client at a low value, it returns a low ca_maxrequests vaule in CREATE_SESSION.  It is a server resource.



-->Andy

> 
> -- 
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com

Re: [PATCH 1/1] NFSv4.1 Increase NFS4_DEF_SLOT_TABLE_SIZE

[Myklebust, Trond]

T24gTW9uLCAyMDEzLTA3LTIyIGF0IDE5OjUyICswMDAwLCBBZGFtc29uLCBBbmR5IHdyb3RlOg0K
PiBPbiBKdWwgMjIsIDIwMTMsIGF0IDM6MTcgUE0sICJNeWtsZWJ1c3QsIFRyb25kIiA8VHJvbmQu
TXlrbGVidXN0QG5ldGFwcC5jb20+DQo+ICB3cm90ZToNCj4gPiBUaGUgcHJvYmxlbSBpcyB0aGF0
IG9uY2UgeW91IGltcGxlbWVudCBkeW5hbWljIHNsb3QgdGFibGVzLCB0aGUNCj4gPiBuZWdvdGlh
dGVkIHNlc3Npb24gc2xvdCB0YWJsZSBzaXplIGJlY29tZXMgdGhlIF9taW5pbXVtXyBudW1iZXIg
b2YNCj4gPiBzbG90cy4NCj4gDQo+IENvdWxkIHlvdSBleHBsYWluIHRoaXMgYSBiaXQgbW9yZT8g
SSdkIGxpa2UgdG8gdW5kZXJzdGFuZCB5b3VyIHJlYXNvbmluZy4gIEkgY2FuIHNlZSB0aGF0IHRo
ZSBkeW5hbWljIHNlc3Npb24gZ29hbCBpcyB0byBlbmQgdXAgd2l0aCB0aGUgaGlnaGVzdF9zbG90
aWQgdmFsdWUgdG8gYmUganVzdCBoaWdoIGVub3VnaCB0byBtZWV0IHRoZSBhY3R1YWwgbG9hZCwg
YnV0IG5vdCB3YXN0ZSBzZXJ2ZXIgc2xvdCByZXNvdXJjZXMuIEJ1dCBJIGRvbid0IHVuZGVyc3Rh
bmQgd2h5IHRoZSBjbGllbnQgdmFsdWUgb2YgY2FfbWF4cmVxdWVzdHMgbWFrZXMgYSBkaWZmZXJl
bmNlIGluIHRoZSBkeW5hbWljIHNlc3Npb25zIGNhc2UgYmVjYXVzZSB3aGV0aGVyIHRoZSBjbGll
bnQgc3RhcnRzIGhpZ2ggb3IgbG93LCB0aGUgc2VydmVyIHdpbGwgYWRqdXN0IHRoZSB2YWx1ZSB0
byBvbmUgdGhhdCBmaXRzIHRoZSBsb2FkLg0KDQpJZiBhbGwgY2xpZW50cyBzdXBwb3J0ZWQgZHlu
YW1pYyBzbG90IHRhYmxlcywgdGhlbiB0aGVyZSB3b3VsZCBiZSBubw0KcHJvYmxlbXMgd2l0aCB0
aGUgc2VydmVyIGxvd2VyaW5nIHRoZSBudW1iZXIgb2Ygc2xvdHMgYmV5b25kIHRoZSB2YWx1ZQ0K
bmVnb3RpYXRlZCBhdCBzZXNzaW9uIGNyZWF0aW9uIHRpbWUuDQpUaGUgcHJvYmxlbXMgb2NjdXIg
aWYgdGhlcmUgYXJlIGNsaWVudHMgb3V0IHRoZXJlIHRoYXQgZG9uJ3Qgc3VwcG9ydA0KZHluYW1p
YyBzbG90cywgYW5kIHRoYXQgZmFpbCB0byBpbnNwZWN0IHRoZSBzcl90YXJnZXRfaGlnaGVzdF9z
bG90aWQgYW5kDQpzcl9oaWdoZXN0X3Nsb3RpZCBmb3IgY2hhbmdlcyAodGhlIExpbnV4IGNsaWVu
dCB1c2VkIHRvIGJlaGF2ZSB0aGlzDQp3YXkpLiBJZiB0aGUgc2VydmVyIHRoZW4gbG93ZXJzIHNy
X2hpZ2hlc3Rfc2xvdGlkIHRvIGJlIGxlc3MgdGhhbiB0aGUNCmluaXRpYWwgdmFsdWUsIHRoZW4g
dGhlIGNsaWVudCB3b24ndCByZWFsaXNlIGl0IGhhcyBmZXdlciBzbG90cywgYW5kDQp3aWxsIGVu
ZCB1cCByZWNlaXZpbmcgTkZTNEVSUl9CQURTTE9UIGVycm9ycyB0aGF0IGl0IGRvZXNuJ3Qga25v
dyBob3cgdG8NCmhhbmRsZS4NCg0KSU9XOiB0aGUgb25seSBzYWZlIGJlaGF2aW91ciBmb3IgYSBz
ZXJ2ZXIgaW4gdGhhdCBjYXNlIGlzIHRvIHRyZWF0IHRoZQ0KaW5pdGlhbCB2YWx1ZSBhcyB0aGUg
bWluaW11bSB2YWx1ZSBmb3Igc3JfaGlnaGVzdF9zbG90aWQgYW5kDQpzcl90YXJnZXRfaGlnaGVz
dF9zbG90aWQuDQoNCi0tIA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50
YWluZXINCg0KTmV0QXBwDQpUcm9uZC5NeWtsZWJ1c3RAbmV0YXBwLmNvbQ0Kd3d3Lm5ldGFwcC5j
b20NCg==