From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============7070035023575163178==" MIME-Version: 1.0 From: Kariuki, John K Subject: Re: [SPDK] About per client QoS (IOPS rate limiting) Date: Thu, 12 Apr 2018 17:01:47 +0000 Message-ID: In-Reply-To: F009CE4E1CB4E047B169243B6A3189273155FD54@SHSMSX101.ccr.corp.intel.com List-ID: To: spdk@lists.01.org --===============7070035023575163178== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Gang/Paul Does this functionality belong in the passthru bdev or should it be impleme= nted in another vbdev called something like "qos"? My understanding of the = passthru bdev is that it's a template that anyone looking to build a vbdev = can use as a starting point. It has all the functions needed to create an S= PDK module stub out so that someone can pick it up, make a copy and a few m= odifications and then they are off to the races implementing their new virt= ual bdev From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Cao, Gang Sent: Tuesday, April 10, 2018 6:05 PM To: Storage Performance Development Kit Subject: Re: [SPDK] About per client QoS (IOPS rate limiting) Yes. It's a good extension to couple with the vbdev and bdev to meet the cl= ient's QoS goal. For a client, from my understanding, a "device" will alway= s be linked through its own specific "connection". The per client (or the p= er connection) QoS can be also achieved by the per "device". The other comment is that to move this QoS rate limiting from the client si= de to the target side. Client can has its own QoS for the connection before= submitting I/Os and with this work, it can also let the target side to do = the similar thing and may not need a QoS at client side. Thanks, Gang From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Luse, Paul E Sent: Tuesday, April 10, 2018 11:56 PM To: Storage Performance Development Kit > Subject: Re: [SPDK] About per client QoS (IOPS rate limiting) Hi Gang, This is pretty cool. So essentially the PT vbdev is just a per-client hook= for QoS? Thx Paul From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Cao, Gang Sent: Monday, April 9, 2018 9:03 PM To: Storage Performance Development Kit > Subject: [SPDK] About per client QoS (IOPS rate limiting) Hi all, There has been some work done to have SPDK support the QoS per bdev which m= eans that each bdev has its assigned IOPS rate limiting. This QoS functiona= lity is below the storage protocols like NVMe-oF, iSCSI and so on. At the time of last PRC SPDK summit, there are some questions on the per cl= ient QoS (i.e., the NVMe-oF iniator / host) besides the per bdev QoS. Each = client like NVMe-oF initiator can see its own IOPS rate limiting not matter= whether the connected device is exclusive or shared at the target side. Based on Paul's great stuff of the Passthru vbdev, I've updated through bel= ow three patches to achieve the per client QoS. Each client sees its own pa= ssthru vbdev and QoS assigned on this vbdev. These vbdev shares the same bd= ev. Currently it only supports shareable and read only vbdev on same bdev. https://review.gerrithub.io/#/c/406888/ vbdev: update and have a create= _passthru_disk function https://review.gerrithub.io/#/c/406891/ vbdev: add the construct_passth= ru_bdev RPC method https://review.gerrithub.io/#/c/406977/ vbdev: make the passthru vbdev = share the same underlying bdev The possible usage as following: [Take NVMe-oF target as example] 1. In the nvmf.conf file, # Configure the Passthru vbdev on same bdev like Malloc0 [Passthru] PT Malloc0 PT0 PT Malloc0 PT1 # The subsystem is configured with shareable bdev by Passthru vbdevs [Subsystem2] NQN nqn.2016-06.io.spdk:cnode2 Listen RDMA 192.168.2.21:4420 AllowAnyHost No Host nqn.2016-06.io.spdk:init SN SPDK00000000000002 Namespace PT0 >>>> This can be connected by one client Namespace PT1 >>>> This can be connected by another client # Assign different QoS IOPS limiting on Passthru vbdevs [QoS] Limit_IOPS PT0 20000 Limit_IOPS PT1 30000 2. Use RPC method to add the Passthru vbdev at runtime a. python ./scripts/rpc.py construct_passthru_bdev Malloc0 PT3 b. python ./scripts/rpc.py nvmf_subsystem_add_ns nqn.2016-06.io.spdk:cnode2= PT3 c. python ./scripts/rpc.py enable_bdev_qos PT3 20000 If there is any concern or comment, please feel free to let me know. Thanks, Gang --===============7070035023575163178== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+CjxoZWFkPgo8bWV0YSBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiIGNv bnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD11cy1hc2NpaSI+CjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPgo8c3R5bGU+ PCEtLQovKiBGb250IERlZmluaXRpb25zICovCkBmb250LWZhY2UKCXtmb250LWZhbWlseToiQ2Ft YnJpYSBNYXRoIjsKCXBhbm9zZS0xOjIgNCA1IDMgNSA0IDYgMyAyIDQ7fQpAZm9udC1mYWNlCgl7 Zm9udC1mYW1pbHk6Q2FsaWJyaTsKCXBhbm9zZS0xOjIgMTUgNSAyIDIgMiA0IDMgMiA0O30KLyog U3R5bGUgRGVmaW5pdGlvbnMgKi8KcC5Nc29Ob3JtYWwsIGxpLk1zb05vcm1hbCwgZGl2Lk1zb05v cm1hbAoJe21hcmdpbjowaW47CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7Cgl0ZXh0LWFsaWduOmp1 c3RpZnk7Cglmb250LXNpemU6MTAuNXB0OwoJZm9udC1mYW1pbHk6IkNhbGlicmkiLHNhbnMtc2Vy aWY7fQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rCgl7bXNvLXN0eWxlLXByaW9yaXR5Ojk5OwoJ Y29sb3I6IzA1NjNDMTsKCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQphOnZpc2l0ZWQsIHNw YW4uTXNvSHlwZXJsaW5rRm9sbG93ZWQKCXttc28tc3R5bGUtcHJpb3JpdHk6OTk7Cgljb2xvcjoj OTU0RjcyOwoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9CnNwYW4uRW1haWxTdHlsZTE3Cgl7 bXNvLXN0eWxlLXR5cGU6cGVyc29uYWw7Cglmb250LWZhbWlseToiQ2FsaWJyaSIsc2Fucy1zZXJp ZjsKCWNvbG9yOndpbmRvd3RleHQ7fQpzcGFuLkVtYWlsU3R5bGUxOAoJe21zby1zdHlsZS10eXBl OnBlcnNvbmFsOwoJZm9udC1mYW1pbHk6IkNhbGlicmkiLHNhbnMtc2VyaWY7Cgljb2xvcjojMUY0 OTdEO30Kc3Bhbi5FbWFpbFN0eWxlMTkKCXttc28tc3R5bGUtdHlwZTpwZXJzb25hbDsKCWZvbnQt ZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlmOwoJY29sb3I6IzFGNDk3RDt9CnNwYW4uRW1haWxT dHlsZTIwCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7Cglmb250LWZhbWlseToiQ2Fs aWJyaSIsc2Fucy1zZXJpZjsKCWNvbG9yOiMxRjQ5N0Q7fQouTXNvQ2hwRGVmYXVsdAoJe21zby1z dHlsZS10eXBlOmV4cG9ydC1vbmx5OwoJZm9udC1zaXplOjEwLjBwdDt9CkBwYWdlIFdvcmRTZWN0 aW9uMQoJe3NpemU6OC41aW4gMTEuMGluOwoJbWFyZ2luOjEuMGluIDEuMjVpbiAxLjBpbiAxLjI1 aW47fQpkaXYuV29yZFNlY3Rpb24xCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQotLT48L3N0eWxlPjwh LS1baWYgZ3RlIG1zbyA5XT48eG1sPgo8bzpzaGFwZWRlZmF1bHRzIHY6ZXh0PSJlZGl0IiBzcGlk bWF4PSIxMDI2IiAvPgo8L3htbD48IVtlbmRpZl0tLT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4K PG86c2hhcGVsYXlvdXQgdjpleHQ9ImVkaXQiPgo8bzppZG1hcCB2OmV4dD0iZWRpdCIgZGF0YT0i MSIgLz4KPC9vOnNoYXBlbGF5b3V0PjwveG1sPjwhW2VuZGlmXS0tPgo8L2hlYWQ+Cjxib2R5IGxh bmc9IkVOLVVTIiBsaW5rPSIjMDU2M0MxIiB2bGluaz0iIzk1NEY3MiIgc3R5bGU9InRleHQtanVz dGlmeS10cmltOnB1bmN0dWF0aW9uIj4KPGRpdiBjbGFzcz0iV29yZFNlY3Rpb24xIj4KPHAgY2xh c3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Y29sb3I6IzFGNDk3 RCI+R2FuZy9QYXVsPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48 c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtjb2xvcjojMUY0OTdEIj5Eb2VzIHRoaXMgZnVu Y3Rpb25hbGl0eSBiZWxvbmcgaW4gdGhlIHBhc3N0aHJ1IGJkZXYgb3Igc2hvdWxkIGl0IGJlIGlt cGxlbWVudGVkIGluIGFub3RoZXIgdmJkZXYgY2FsbGVkIHNvbWV0aGluZyBsaWtlICYjODIyMDtx b3MmIzgyMjE7PyBNeSB1bmRlcnN0YW5kaW5nIG9mIHRoZSBwYXNzdGhydSBiZGV2IGlzIHRoYXQg aXQmIzgyMTc7cyBhIHRlbXBsYXRlIHRoYXQKIGFueW9uZSBsb29raW5nIHRvIGJ1aWxkIGEgdmJk ZXYgY2FuIHVzZSBhcyBhIHN0YXJ0aW5nIHBvaW50LiBJdCBoYXMgYWxsIHRoZSBmdW5jdGlvbnMg bmVlZGVkIHRvIGNyZWF0ZSBhbiBTUERLIG1vZHVsZSBzdHViIG91dCBzbyB0aGF0IHNvbWVvbmUg Y2FuIHBpY2sgaXQgdXAsIG1ha2UgYSBjb3B5IGFuZCBhIGZldyBtb2RpZmljYXRpb25zIGFuZCB0 aGVuIHRoZXkgYXJlIG9mZiB0byB0aGUgcmFjZXMgaW1wbGVtZW50aW5nIHRoZWlyIG5ldyB2aXJ0 dWFsCiBiZGV2PG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48YSBu YW1lPSJfTWFpbEVuZENvbXBvc2UiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2NvbG9y OiMxRjQ5N0QiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvYT48L3A+CjxkaXY+CjxkaXYgc3R5 bGU9ImJvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQgI0UxRTFFMSAxLjBwdDtwYWRkaW5nOjMu MHB0IDBpbiAwaW4gMGluIj4KPHAgY2xhc3M9Ik1zb05vcm1hbCIgYWxpZ249ImxlZnQiIHN0eWxl PSJ0ZXh0LWFsaWduOmxlZnQiPjxhIG5hbWU9Il9fX19fcmVwbHlzZXBhcmF0b3IiPjwvYT48Yj48 c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdCI+RnJvbTo8L3NwYW4+PC9iPjxzcGFuIHN0eWxl PSJmb250LXNpemU6MTEuMHB0Ij4gU1BESyBbbWFpbHRvOnNwZGstYm91bmNlc0BsaXN0cy4wMS5v cmddCjxiPk9uIEJlaGFsZiBPZiA8L2I+Q2FvLCBHYW5nPGJyPgo8Yj5TZW50OjwvYj4gVHVlc2Rh eSwgQXByaWwgMTAsIDIwMTggNjowNSBQTTxicj4KPGI+VG86PC9iPiBTdG9yYWdlIFBlcmZvcm1h bmNlIERldmVsb3BtZW50IEtpdCAmbHQ7c3Bka0BsaXN0cy4wMS5vcmcmZ3Q7PGJyPgo8Yj5TdWJq ZWN0OjwvYj4gUmU6IFtTUERLXSBBYm91dCBwZXIgY2xpZW50IFFvUyAoSU9QUyByYXRlIGxpbWl0 aW5nKTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPC9kaXY+CjwvZGl2Pgo8cCBjbGFzcz0iTXNvTm9y bWFsIiBhbGlnbj0ibGVmdCIgc3R5bGU9InRleHQtYWxpZ246bGVmdCI+PG86cD4mbmJzcDs8L286 cD48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJjb2xvcjojMUY0OTdEO21z by1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj5ZZXMuIEl0JiM4MjE3O3MgYSBnb29kIGV4dGVuc2lv biB0byBjb3VwbGUgd2l0aCB0aGUgdmJkZXYgYW5kIGJkZXYgdG8gbWVldCB0aGUgY2xpZW50JiM4 MjE3O3MgUW9TIGdvYWwuIEZvciBhIGNsaWVudCwgZnJvbSBteSB1bmRlcnN0YW5kaW5nLCBhICYj ODIyMDtkZXZpY2UmIzgyMjE7IHdpbGwgYWx3YXlzIGJlIGxpbmtlZCB0aHJvdWdoIGl0cyBvd24g c3BlY2lmaWMKICYjODIyMDtjb25uZWN0aW9uJiM4MjIxOy4gVGhlIHBlciBjbGllbnQgKG9yIHRo ZSBwZXIgY29ubmVjdGlvbikgUW9TIGNhbiBiZSBhbHNvIGFjaGlldmVkIGJ5IHRoZSBwZXIgJiM4 MjIwO2RldmljZSYjODIyMTsuPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9y bWFsIj48c3BhbiBzdHlsZT0iY29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1D TiI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh biBzdHlsZT0iY29sb3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+VGhlIG90 aGVyIGNvbW1lbnQgaXMgdGhhdCB0byBtb3ZlIHRoaXMgUW9TIHJhdGUgbGltaXRpbmcgZnJvbSB0 aGUgY2xpZW50IHNpZGUgdG8gdGhlIHRhcmdldCBzaWRlLiBDbGllbnQgY2FuIGhhcyBpdHMgb3du IFFvUyBmb3IgdGhlIGNvbm5lY3Rpb24gYmVmb3JlIHN1Ym1pdHRpbmcgSS9PcyBhbmQgd2l0aCB0 aGlzCiB3b3JrLCBpdCBjYW4gYWxzbyBsZXQgdGhlIHRhcmdldCBzaWRlIHRvIGRvIHRoZSBzaW1p bGFyIHRoaW5nIGFuZCBtYXkgbm90IG5lZWQgYSBRb1MgYXQgY2xpZW50IHNpZGUuPG86cD48L286 cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iY29sb3I6IzFG NDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+ PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iY29sb3I6IzFGNDk3RDttc28t ZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+VGhhbmtzLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImNvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3Qt bGFuZ3VhZ2U6WkgtQ04iPkdhbmc8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29O b3JtYWwiPjxzcGFuIHN0eWxlPSJjb2xvcjojMUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOlpI LUNOIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+CjxkaXY+CjxkaXYgc3R5bGU9ImJvcmRl cjpub25lO2JvcmRlci10b3A6c29saWQgI0UxRTFFMSAxLjBwdDtwYWRkaW5nOjMuMHB0IDBpbiAw aW4gMGluIj4KPHAgY2xhc3M9Ik1zb05vcm1hbCIgYWxpZ249ImxlZnQiIHN0eWxlPSJ0ZXh0LWFs aWduOmxlZnQiPjxiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O21zby1mYXJlYXN0LWxh bmd1YWdlOlpILUNOIj5Gcm9tOjwvc3Bhbj48L2I+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4w cHQ7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPiBTUERLIFs8YSBocmVmPSJtYWlsdG86c3Bk ay1ib3VuY2VzQGxpc3RzLjAxLm9yZyI+bWFpbHRvOnNwZGstYm91bmNlc0BsaXN0cy4wMS5vcmc8 L2E+XQo8Yj5PbiBCZWhhbGYgT2YgPC9iPkx1c2UsIFBhdWwgRTxicj4KPGI+U2VudDo8L2I+IFR1 ZXNkYXksIEFwcmlsIDEwLCAyMDE4IDExOjU2IFBNPGJyPgo8Yj5Ubzo8L2I+IFN0b3JhZ2UgUGVy Zm9ybWFuY2UgRGV2ZWxvcG1lbnQgS2l0ICZsdDs8YSBocmVmPSJtYWlsdG86c3Bka0BsaXN0cy4w MS5vcmciPnNwZGtAbGlzdHMuMDEub3JnPC9hPiZndDs8YnI+CjxiPlN1YmplY3Q6PC9iPiBSZTog W1NQREtdIEFib3V0IHBlciBjbGllbnQgUW9TIChJT1BTIHJhdGUgbGltaXRpbmcpPG86cD48L286 cD48L3NwYW4+PC9wPgo8L2Rpdj4KPC9kaXY+CjxwIGNsYXNzPSJNc29Ob3JtYWwiIGFsaWduPSJs ZWZ0IiBzdHlsZT0idGV4dC1hbGlnbjpsZWZ0Ij48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFu Z3VhZ2U6WkgtQ04iPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Y29sb3I6IzFGNDk3RDttc28tZmFy ZWFzdC1sYW5ndWFnZTpaSC1DTiI+SGkgR2FuZyw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2NvbG9yOiMxRjQ5 N0Q7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwv cD4KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Y29s b3I6IzFGNDk3RDttc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+VGhpcyBpcyBwcmV0dHkgY29v bC4mbmJzcDsgU28gZXNzZW50aWFsbHkgdGhlIFBUIHZiZGV2IGlzIGp1c3QgYSBwZXItY2xpZW50 IGhvb2sgZm9yIFFvUz88bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwi PjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2NvbG9yOiMxRjQ5N0Q7bXNvLWZhcmVhc3Qt bGFuZ3VhZ2U6WkgtQ04iPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9Ik1z b05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Y29sb3I6IzFGNDk3RDttc28t ZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+VGh4PG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFz cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtjb2xvcjojMUY0OTdE O21zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj5QYXVsPG86cD48L286cD48L3NwYW4+PC9wPgo8 cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtjb2xvcjoj MUY0OTdEO21zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bh bj48L3A+CjxkaXY+CjxkaXYgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQgI0Ux RTFFMSAxLjBwdDtwYWRkaW5nOjMuMHB0IDBpbiAwaW4gMGluIj4KPHAgY2xhc3M9Ik1zb05vcm1h bCIgYWxpZ249ImxlZnQiIHN0eWxlPSJ0ZXh0LWFsaWduOmxlZnQiPjxiPjxzcGFuIHN0eWxlPSJm b250LXNpemU6MTEuMHB0O21zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj5Gcm9tOjwvc3Bhbj48 L2I+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6Wkgt Q04iPiBTUERLIFs8YSBocmVmPSJtYWlsdG86c3Bkay1ib3VuY2VzQGxpc3RzLjAxLm9yZyI+bWFp bHRvOnNwZGstYm91bmNlc0BsaXN0cy4wMS5vcmc8L2E+XQo8Yj5PbiBCZWhhbGYgT2YgPC9iPkNh bywgR2FuZzxicj4KPGI+U2VudDo8L2I+IE1vbmRheSwgQXByaWwgOSwgMjAxOCA5OjAzIFBNPGJy Pgo8Yj5Ubzo8L2I+IFN0b3JhZ2UgUGVyZm9ybWFuY2UgRGV2ZWxvcG1lbnQgS2l0ICZsdDs8YSBo cmVmPSJtYWlsdG86c3Bka0BsaXN0cy4wMS5vcmciPnNwZGtAbGlzdHMuMDEub3JnPC9hPiZndDs8 YnI+CjxiPlN1YmplY3Q6PC9iPiBbU1BES10gQWJvdXQgcGVyIGNsaWVudCBRb1MgKElPUFMgcmF0 ZSBsaW1pdGluZyk8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjwvZGl2Pgo8L2Rpdj4KPHAgY2xhc3M9 Ik1zb05vcm1hbCIgYWxpZ249ImxlZnQiIHN0eWxlPSJ0ZXh0LWFsaWduOmxlZnQiPjxzcGFuIHN0 eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+ PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3Vh Z2U6WkgtQ04iPkhpIGFsbCw8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+PG86cD4mbmJzcDs8 L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZh cmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPlRoZXJlIGhhcyBiZWVuIHNvbWUgd29yayBkb25lIHRvIGhh dmUgU1BESyBzdXBwb3J0IHRoZSBRb1MgcGVyIGJkZXYgd2hpY2ggbWVhbnMgdGhhdCBlYWNoIGJk ZXYgaGFzIGl0cyBhc3NpZ25lZCBJT1BTIHJhdGUgbGltaXRpbmcuIFRoaXMgUW9TIGZ1bmN0aW9u YWxpdHkgaXMgYmVsb3cgdGhlIHN0b3JhZ2UgcHJvdG9jb2xzIGxpa2UgTlZNZS1vRiwKIGlTQ1NJ IGFuZCBzbyBvbi48bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz cGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+PG86cD4mbmJzcDs8L286cD48 L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3Qt bGFuZ3VhZ2U6WkgtQ04iPkF0IHRoZSB0aW1lIG9mIGxhc3QgUFJDIFNQREsgc3VtbWl0LCB0aGVy ZSBhcmUgc29tZSBxdWVzdGlvbnMgb24gdGhlIHBlciBjbGllbnQgUW9TIChpLmUuLCB0aGUgTlZN ZS1vRiBpbmlhdG9yIC8gaG9zdCkgYmVzaWRlcyB0aGUgcGVyIGJkZXYgUW9TLiBFYWNoIGNsaWVu dCBsaWtlIE5WTWUtb0YgaW5pdGlhdG9yIGNhbiBzZWUgaXRzIG93bgogSU9QUyByYXRlIGxpbWl0 aW5nIG5vdCBtYXR0ZXIgd2hldGhlciB0aGUgY29ubmVjdGVkIGRldmljZSBpcyBleGNsdXNpdmUg b3Igc2hhcmVkIGF0IHRoZSB0YXJnZXQgc2lkZS48bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+ PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBz dHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPkJhc2VkIG9uIFBhdWwmIzgyMTc7cyBn cmVhdCBzdHVmZiBvZiB0aGUgUGFzc3RocnUgdmJkZXYsIEkmIzgyMTc7dmUgdXBkYXRlZCB0aHJv dWdoIGJlbG93IHRocmVlIHBhdGNoZXMgdG8gYWNoaWV2ZSB0aGUgcGVyIGNsaWVudCBRb1MuIEVh Y2ggY2xpZW50IHNlZXMgaXRzIG93biBwYXNzdGhydSB2YmRldiBhbmQgUW9TIGFzc2lnbmVkIG9u IHRoaXMgdmJkZXYuCiBUaGVzZSB2YmRldiBzaGFyZXMgdGhlIHNhbWUgYmRldi4gQ3VycmVudGx5 IGl0IG9ubHkgc3VwcG9ydHMgc2hhcmVhYmxlIGFuZCByZWFkIG9ubHkgdmJkZXYgb24gc2FtZSBi ZGV2LjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5 bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48 L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFn ZTpaSC1DTiI+PGEgaHJlZj0iaHR0cHM6Ly9yZXZpZXcuZ2Vycml0aHViLmlvLyMvYy80MDY4ODgv Ij5odHRwczovL3Jldmlldy5nZXJyaXRodWIuaW8vIy9jLzQwNjg4OC88L2E+Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7IHZiZGV2OiB1cGRhdGUgYW5kIGhhdmUgYSBjcmVhdGVfcGFzc3RocnVfZGlz ayBmdW5jdGlvbjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNw YW4gc3R5bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj48YSBocmVmPSJodHRwczovL3Jl dmlldy5nZXJyaXRodWIuaW8vIy9jLzQwNjg5MS8iPmh0dHBzOi8vcmV2aWV3LmdlcnJpdGh1Yi5p by8jL2MvNDA2ODkxLzwvYT4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsgdmJkZXY6IGFkZCB0aGUg Y29uc3RydWN0X3Bhc3N0aHJ1X2JkZXYgUlBDIG1ldGhvZDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4K PHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpI LUNOIj48YSBocmVmPSJodHRwczovL3Jldmlldy5nZXJyaXRodWIuaW8vIy9jLzQwNjk3Ny8iPmh0 dHBzOi8vcmV2aWV3LmdlcnJpdGh1Yi5pby8jL2MvNDA2OTc3LzwvYT4mbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsgdmJkZXY6IG1ha2UgdGhlIHBhc3N0aHJ1IHZiZGV2IHNoYXJlIHRoZSBzYW1lIHVu ZGVybHlpbmcgYmRldjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9Ik1zb05vcm1hbCI+ PHNwYW4gc3R5bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj48bzpwPiZuYnNwOzwvbzpw Pjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFz dC1sYW5ndWFnZTpaSC1DTiI+VGhlIHBvc3NpYmxlIHVzYWdlIGFzIGZvbGxvd2luZzo8bzpwPjwv bzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFy ZWFzdC1sYW5ndWFnZTpaSC1DTiI+W1Rha2UgTlZNZS1vRiB0YXJnZXQgYXMgZXhhbXBsZV08bzpw PjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28t ZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPgo8cCBj bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04i PjEuJm5ic3A7IEluIHRoZSBudm1mLmNvbmYgZmlsZSwKPG86cD48L286cD48L3NwYW4+PC9wPgo8 cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6Wkgt Q04iPiMgQ29uZmlndXJlIHRoZSBQYXNzdGhydSB2YmRldiBvbiBzYW1lIGJkZXYgbGlrZSBNYWxs b2MwPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls ZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPltQYXNzdGhydV08bzpwPjwvbzpwPjwvc3Bh bj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5n dWFnZTpaSC1DTiI+UFQgTWFsbG9jMCBQVDA8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNz PSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+UFQg TWFsbG9jMCBQVDE8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxz cGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+PG86cD4mbmJzcDs8L286cD48 L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3Qt bGFuZ3VhZ2U6WkgtQ04iPiMgVGhlIHN1YnN5c3RlbSBpcyBjb25maWd1cmVkIHdpdGggc2hhcmVh YmxlIGJkZXYgYnkgUGFzc3RocnUgdmJkZXZzPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFz cz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPltT dWJzeXN0ZW0yXTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNw YW4gc3R5bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj4mbmJzcDsgTlFOIG5xbi4yMDE2 LTA2LmlvLnNwZGs6Y25vZGUyPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9y bWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPiZuYnNwOyBMaXN0 ZW4gUkRNQSAxOTIuMTY4LjIuMjE6NDQyMDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9 Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj4mbmJz cDsgQWxsb3dBbnlIb3N0IE5vPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9y bWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPiZuYnNwOyBIb3N0 IG5xbi4yMDE2LTA2LmlvLnNwZGs6aW5pdDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9 Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj4mbmJz cDsgU04gU1BESzAwMDAwMDAwMDAwMDAyPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0i TXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPiZuYnNw OyBOYW1lc3BhY2UgUFQwJm5ic3A7Jm5ic3A7ICZndDsmZ3Q7Jmd0OyZndDsgVGhpcyBjYW4gYmUg Y29ubmVjdGVkIGJ5IG9uZSBjbGllbnQ8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJN c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+Jm5ic3A7 IE5hbWVzcGFjZSBQVDEmbmJzcDsmbmJzcDsgJmd0OyZndDsmZ3Q7Jmd0OyBUaGlzIGNhbiBiZSBj b25uZWN0ZWQgYnkgYW5vdGhlciBjbGllbnQ8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNz PSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+PG86 cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls ZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPiMgQXNzaWduIGRpZmZlcmVudCBRb1MgSU9Q UyBsaW1pdGluZyBvbiBQYXNzdGhydSB2YmRldnM8bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+ W1FvU108bzpwPjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0 eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+TGltaXRfSU9QUyBQVDAgMjAwMDA8bzpw PjwvbzpwPjwvc3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28t ZmFyZWFzdC1sYW5ndWFnZTpaSC1DTiI+TGltaXRfSU9QUyBQVDEgMzAwMDA8bzpwPjwvbzpwPjwv c3Bhbj48L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1s YW5ndWFnZTpaSC1DTiI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNv Tm9ybWFsIj48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPjIuIFVzZSBS UEMgbWV0aG9kIHRvIGFkZCB0aGUgUGFzc3RocnUgdmJkZXYgYXQgcnVudGltZTxvOnA+PC9vOnA+ PC9zcGFuPjwvcD4KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9Im1zby1mYXJlYXN0 LWxhbmd1YWdlOlpILUNOIj5hLiBweXRob24gLi9zY3JpcHRzL3JwYy5weSBjb25zdHJ1Y3RfcGFz c3RocnVfYmRldiBNYWxsb2MwIFBUMzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4KPHAgY2xhc3M9Ik1z b05vcm1hbCI+PHNwYW4gc3R5bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNOIj5iLiBweXRo b24gLi9zY3JpcHRzL3JwYy5weSBudm1mX3N1YnN5c3RlbV9hZGRfbnMgbnFuLjIwMTYtMDYuaW8u c3Bkazpjbm9kZTIgUFQzPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFs Ij48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPmMuIHB5dGhvbiAuL3Nj cmlwdHMvcnBjLnB5IGVuYWJsZV9iZGV2X3FvcyBQVDMgMjAwMDA8bzpwPjwvbzpwPjwvc3Bhbj48 L3A+CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJtc28tZmFyZWFzdC1sYW5ndWFn ZTpaSC1DTiI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFs Ij48c3BhbiBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPklmIHRoZXJlIGlzIGFu eSBjb25jZXJuIG9yIGNvbW1lbnQsIHBsZWFzZSBmZWVsIGZyZWUgdG8gbGV0IG1lIGtub3cuPG86 cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0ibXNv LWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9Im1zby1mYXJlYXN0LWxhbmd1YWdlOlpILUNO Ij5UaGFua3MsPG86cD48L286cD48L3NwYW4+PC9wPgo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh biBzdHlsZT0ibXNvLWZhcmVhc3QtbGFuZ3VhZ2U6WkgtQ04iPkdhbmc8bzpwPjwvbzpwPjwvc3Bh bj48L3A+CjwvZGl2Pgo8L2JvZHk+CjwvaHRtbD4K --===============7070035023575163178==--