From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============5473991519959481066==" MIME-Version: 1.0 From: Andrey Kuzmin Subject: Re: [SPDK] Handling of physical disk removals Date: Thu, 31 May 2018 10:54:04 +0300 Message-ID: In-Reply-To: CAKye4QahMFJpiejAnqtrSuZPPZUHT4AxLO-6EQzPimuC7WXKJQ@mail.gmail.com List-ID: To: spdk@lists.01.org --===============5473991519959481066== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Thu, May 31, 2018, 10:25 Baruch Even wrote: > > On Wed, May 30, 2018 at 8:49 PM Andrey Kuzmin > wrote: > >> On Wed, May 30, 2018 at 2:46 PM, Baruch Even wrote: >> >>> As for threading, the only thing I need to make things integrate better >>> into my green-threads is to use a function call to do a context switch = if >>> needed (spdk_yield() of sorts), that is empty/no-op for most users and = can >>> be a compile time option (weak symbol? define?) to use some user-provid= ed >>> function. This will immediately integrate into any green thread system = by >>> switching to another user-thread and returning for a poll when other >>> actions are taken. This way the posix-thread will not block and there i= s no >>> special async logic that needs to happen. >>> >>> As for the low level api, my application already has its own queue for >>> each device, and we split our requests as we need to and combine them as >>> possible to improve performance. It took us by surprise to find that SP= DK >>> internally breaks out the requests to smaller chunks, queues them >>> internally and completions may not really complete the full request. Th= ere >>> was even was scenario that we had a deadlock because we overflowed the = spdk >>> queues. Once we found these out we made sure to expose to our applicati= on >>> all the device constraints and break the IOs with the device limitation= s in >>> mind and now all the extra work and checks that SPDK does are a waste of >>> cpu cycles as far as we are concerned. >>> >>> There is also the issue that we track our IOs and SPDK because it does >>> the extra work has IO tracking of its own and it means there is an extra >>> memory allocation on the data path for no good reason from our point of >>> view. >>> >> >> A small but mighty step in this direction would be to make >> spdk_bdev_submit_io >> >> public. If the caller wishes to manage the setup and completion chaining >> (in the stacked case, it is trivial to do as the caller has per I/O cont= ext >> already allocated at this point), thus avoiding the overhead if the extra >> I/O allocation in spdk_bdev_{read,write}*, that would be the least >> resistance path. And the code would be much simpler, avoiding the >> > (often useless) io_type switch. >> > > I am not using bdev at all so such an approach might be useful to others > but not for me. > > > When I'm building my command in the queue I need to keep the list of >>> buffers in some data structure, currently I keep it in a vector and then >>> this gets translated to PRP/SGL when the command is submitted. I could = save >>> that step if I could just maintain the PRP/SGL myself and just pass the >>> ready-made list to the command. This will save me some memory and time. >>> >> >> If your use case is limited to NVMe hardware, you might want to consider >> NVMe passthrough which avoids the buffer setup overhead, and also gives = you >> complete control over completion status. >> > > I'm not sure what is the NVMe passthrough that you refer to, I can only > assume that you assume that I'm using bdev. I am in fact simply calling t= he > spdk_nvme_* functions directly and work with that. > Passthrough in storage typically refers to submitting a well-formed protocol command directly to the hardware. That sounds very much what you're looking for, judging by the discussion. Regards, Andrey > > Baruch > > -- > > > > *Baruch Even, Software Developer E baruch(a)weka.io > www.weka.io * > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk > --===============5473991519959481066== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGRpdiBkaXI9ImF1dG8iPjxkaXY+PGJyPjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+PGRp diBkaXI9Imx0ciI+T24gVGh1LCBNYXkgMzEsIDIwMTgsIDEwOjI1IEJhcnVjaCBFdmVuICZsdDs8 YSBocmVmPSJtYWlsdG86YmFydWNoQHdla2EuaW8iPmJhcnVjaEB3ZWthLmlvPC9hPiZndDsgd3Jv dGU6PGJyPjwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdp bjowIDAgMCAuOGV4O2JvcmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgi PjxkaXYgZGlyPSJsdHIiPjxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIj48ZGl2IGRpcj0ibHRyIj48 YnI+T24gV2VkLCBNYXkgMzAsIDIwMTggYXQgODo0OSBQTSBBbmRyZXkgS3V6bWluICZsdDs8YSBo cmVmPSJtYWlsdG86YW5kcmV5LnYua3V6bWluQGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiIHJl bD0ibm9yZWZlcnJlciI+YW5kcmV5LnYua3V6bWluQGdtYWlsLmNvbTwvYT4mZ3Q7IHdyb3RlOjxi cj48L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAw IDAgLjhleDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2 IGRpcj0ibHRyIj48ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+PGRpdiBjbGFzcz0iZ21haWxfcXVv dGUiPk9uIFdlZCwgTWF5IDMwLCAyMDE4IGF0IDI6NDYgUE0sIEJhcnVjaCBFdmVuIDxzcGFuIGRp cj0ibHRyIj4mbHQ7PGEgaHJlZj0ibWFpbHRvOmJhcnVjaEB3ZWthLmlvIiB0YXJnZXQ9Il9ibGFu ayIgcmVsPSJub3JlZmVycmVyIj5iYXJ1Y2hAd2VrYS5pbzwvYT4mZ3Q7PC9zcGFuPiB3cm90ZTo8 YnI+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAwIC44 ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiBkaXI9 Imx0ciI+PGRpdj5BcyBmb3IgdGhyZWFkaW5nLCB0aGUgb25seSB0aGluZyBJIG5lZWQgdG8gbWFr ZSB0aGluZ3MgaW50ZWdyYXRlIGJldHRlciBpbnRvIG15IGdyZWVuLXRocmVhZHMgaXMgdG8gdXNl IGEgZnVuY3Rpb24gY2FsbCB0byBkbyBhIGNvbnRleHQgc3dpdGNoIGlmIG5lZWRlZCAoc3Bka195 aWVsZCgpIG9mIHNvcnRzKSwgdGhhdCBpcyBlbXB0eS9uby1vcCBmb3IgbW9zdCB1c2VycyBhbmQg Y2FuIGJlIGEgY29tcGlsZSB0aW1lIG9wdGlvbiAod2VhayBzeW1ib2w/IGRlZmluZT8pIHRvIHVz ZSBzb21lIHVzZXItcHJvdmlkZWQgZnVuY3Rpb24uIFRoaXMgd2lsbCBpbW1lZGlhdGVseSBpbnRl Z3JhdGUgaW50byBhbnkgZ3JlZW4gdGhyZWFkIHN5c3RlbSBieSBzd2l0Y2hpbmcgdG8gYW5vdGhl ciB1c2VyLXRocmVhZCBhbmQgcmV0dXJuaW5nIGZvciBhIHBvbGwgd2hlbiBvdGhlciBhY3Rpb25z IGFyZSB0YWtlbi4gVGhpcyB3YXkgdGhlIHBvc2l4LXRocmVhZCB3aWxsIG5vdCBibG9jayBhbmQg dGhlcmUgaXMgbm8gc3BlY2lhbCBhc3luYyBsb2dpYyB0aGF0IG5lZWRzIHRvIGhhcHBlbi48YnI+ PGJyPjwvZGl2PjxkaXY+QXMgZm9yIHRoZSBsb3cgbGV2ZWwgYXBpLCBteSBhcHBsaWNhdGlvbiBh bHJlYWR5IGhhcyBpdHMgb3duIHF1ZXVlIGZvciBlYWNoIGRldmljZSwgYW5kIHdlIHNwbGl0IG91 ciByZXF1ZXN0cyBhcyB3ZSBuZWVkIHRvIGFuZCBjb21iaW5lIHRoZW0gYXMgcG9zc2libGUgdG8g aW1wcm92ZSBwZXJmb3JtYW5jZS4gSXQgdG9vayB1cyBieSBzdXJwcmlzZSB0byBmaW5kIHRoYXQg U1BESyBpbnRlcm5hbGx5IGJyZWFrcyBvdXQgdGhlIHJlcXVlc3RzIHRvIHNtYWxsZXIgY2h1bmtz LCBxdWV1ZXMgdGhlbSBpbnRlcm5hbGx5IGFuZCBjb21wbGV0aW9ucyBtYXkgbm90IHJlYWxseSBj b21wbGV0ZSB0aGUgZnVsbCByZXF1ZXN0LiBUaGVyZSB3YXMgZXZlbiB3YXMgc2NlbmFyaW8gdGhh dCB3ZSBoYWQgYSBkZWFkbG9jayBiZWNhdXNlIHdlIG92ZXJmbG93ZWQgdGhlIHNwZGsgcXVldWVz LiBPbmNlIHdlIGZvdW5kIHRoZXNlIG91dCB3ZSBtYWRlIHN1cmUgdG8gZXhwb3NlIHRvIG91ciBh cHBsaWNhdGlvbiBhbGwgdGhlIGRldmljZSBjb25zdHJhaW50cyBhbmQgYnJlYWsgdGhlIElPcyB3 aXRoIHRoZSBkZXZpY2UgbGltaXRhdGlvbnMgaW4gbWluZCBhbmQgbm93IGFsbCB0aGUgZXh0cmEg d29yayBhbmQgY2hlY2tzIHRoYXQgU1BESyBkb2VzIGFyZSBhIHdhc3RlIG9mIGNwdSBjeWNsZXMg YXMgZmFyIGFzIHdlIGFyZSBjb25jZXJuZWQuPGJyPjxicj48L2Rpdj48ZGl2PlRoZXJlIGlzIGFs c28gdGhlIGlzc3VlIHRoYXQgd2UgdHJhY2sgb3VyIElPcyBhbmQgU1BESyBiZWNhdXNlIGl0IGRv ZXMgdGhlIGV4dHJhIHdvcmsgaGFzIElPIHRyYWNraW5nIG9mIGl0cyBvd24gYW5kIGl0IG1lYW5z IHRoZXJlIGlzIGFuIGV4dHJhIG1lbW9yeSBhbGxvY2F0aW9uIG9uIHRoZSBkYXRhIHBhdGggZm9y IG5vIGdvb2QgcmVhc29uIGZyb20gb3VyIHBvaW50IG9mIHZpZXcuPGJyPjwvZGl2PjwvZGl2Pjwv YmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48ZGl2IGRpcj0ibHRy Ij48ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXY+ QSBzbWFsbCBidXQgbWlnaHR5IHN0ZXAgaW4gdGhpcyBkaXJlY3Rpb24gd291bGQgYmUgdG8gbWFr ZSA8YSBocmVmPSJodHRwczovL2dpdGh1Yi5jb20vc3Bkay9zcGRrL2Jsb2IvZDM0YmQwYTYwYmFj ZTFiN2U0MDIyZjEyZDA4OTJmMDI4YjdmMjM3Ny9saWIvYmRldi9iZGV2LmMjTDkxMSIgdGFyZ2V0 PSJfYmxhbmsiIHJlbD0ibm9yZWZlcnJlciI+c3Bka19iZGV2X3N1Ym1pdF9pbzwvYT4gcHVibGlj LiBJZiB0aGUgY2FsbGVyIHdpc2hlcyB0byBtYW5hZ2UgdGhlIHNldHVwIGFuZCBjb21wbGV0aW9u IGNoYWluaW5nIChpbiB0aGUgc3RhY2tlZCBjYXNlLCBpdCBpcyB0cml2aWFsIHRvIGRvIGFzIHRo ZSBjYWxsZXIgaGFzIHBlciBJL08gY29udGV4dCBhbHJlYWR5IGFsbG9jYXRlZCBhdCB0aGlzIHBv aW50KSwgdGh1cyBhdm9pZGluZyB0aGUgb3ZlcmhlYWQgaWYgdGhlIGV4dHJhIEkvTyBhbGxvY2F0 aW9uIGluIHNwZGtfYmRldl97cmVhZCx3cml0ZX0qLCB0aGF0IHdvdWxkIGJlIHRoZSBsZWFzdCBy ZXNpc3RhbmNlIHBhdGguIEFuZCB0aGUgY29kZSB3b3VsZCBiZSBtdWNoIHNpbXBsZXIsIGF2b2lk aW5nIHRoZSA8L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48L2Jsb2NrcXVvdGU+PGJsb2NrcXVvdGUg Y2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6 MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiBkaXI9Imx0ciI+PGRpdiBjbGFz cz0iZ21haWxfZXh0cmEiPjxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIj48ZGl2PihvZnRlbiB1c2Vs ZXNzKSBpb190eXBlIHN3aXRjaC48L2Rpdj48L2Rpdj48L2Rpdj48L2Rpdj48ZGl2IGRpcj0ibHRy Ij48ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXY+ PC9kaXY+PC9kaXY+PC9kaXY+PC9kaXY+PC9ibG9ja3F1b3RlPjxkaXY+wqA8YnI+PC9kaXY+PGRp dj5JIGFtIG5vdCB1c2luZyBiZGV2IGF0IGFsbCBzbyBzdWNoIGFuIGFwcHJvYWNoIG1pZ2h0IGJl IHVzZWZ1bCB0byBvdGhlcnMgYnV0IG5vdCBmb3IgbWUuPC9kaXY+PC9kaXY+PC9kaXY+PC9ibG9j a3F1b3RlPjwvZGl2PjwvZGl2PjxkaXYgZGlyPSJhdXRvIj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90 ZSI+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAwIC44 ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiBkaXI9 Imx0ciI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXY+PGJyPjxicj48L2Rpdj48YmxvY2tx dW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAgLjhleDtib3JkZXIt bGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2IGRpcj0ibHRyIj48ZGl2 IGNsYXNzPSJnbWFpbF9leHRyYSI+PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxibG9ja3F1b3Rl IGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAgMCAuOGV4O2JvcmRlci1sZWZ0 OjFweCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPjxkaXYgZGlyPSJsdHIiPjxkaXY+PGRp dj5XaGVuIEkmIzM5O20gYnVpbGRpbmcgbXkgY29tbWFuZCBpbiB0aGUgcXVldWUgSSBuZWVkIHRv IGtlZXAgdGhlIGxpc3Qgb2YgYnVmZmVycyBpbiBzb21lIGRhdGEgc3RydWN0dXJlLCBjdXJyZW50 bHkgSSBrZWVwIGl0IGluIGEgdmVjdG9yIGFuZCB0aGVuIHRoaXMgZ2V0cyB0cmFuc2xhdGVkIHRv IFBSUC9TR0wgd2hlbiB0aGUgY29tbWFuZCBpcyBzdWJtaXR0ZWQuIEkgY291bGQgc2F2ZSB0aGF0 IHN0ZXAgaWYgSSBjb3VsZCBqdXN0IG1haW50YWluIHRoZSBQUlAvU0dMIG15c2VsZiBhbmQganVz dCBwYXNzIHRoZSByZWFkeS1tYWRlIGxpc3QgdG8gdGhlIGNvbW1hbmQuIFRoaXMgd2lsbCBzYXZl IG1lIHNvbWUgbWVtb3J5IGFuZCB0aW1lLjxicj48L2Rpdj48L2Rpdj48L2Rpdj48L2Jsb2NrcXVv dGU+PGRpdj48YnI+PC9kaXY+PC9kaXY+PC9kaXY+PC9kaXY+PGRpdiBkaXI9Imx0ciI+PGRpdiBj bGFzcz0iZ21haWxfZXh0cmEiPjxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIj48ZGl2PklmIHlvdXIg dXNlIGNhc2UgaXMgbGltaXRlZCB0byBOVk1lIGhhcmR3YXJlLCB5b3UgbWlnaHQgd2FudCB0byBj b25zaWRlciBOVk1lIHBhc3N0aHJvdWdoIHdoaWNoIGF2b2lkcyB0aGUgYnVmZmVyIHNldHVwIG92 ZXJoZWFkLCBhbmQgYWxzbyBnaXZlcyB5b3UgY29tcGxldGUgY29udHJvbCBvdmVyIGNvbXBsZXRp b24gc3RhdHVzLjwvZGl2PjwvZGl2PjwvZGl2PjwvZGl2PjwvYmxvY2txdW90ZT48ZGl2Pjxicj48 L2Rpdj48ZGl2PkkmIzM5O20gbm90IHN1cmUgd2hhdCBpcyB0aGUgTlZNZSBwYXNzdGhyb3VnaCB0 aGF0IHlvdSByZWZlciB0bywgSSBjYW4gb25seSBhc3N1bWUgdGhhdCB5b3UgYXNzdW1lIHRoYXQg SSYjMzk7bSB1c2luZyBiZGV2LiBJIGFtIGluIGZhY3Qgc2ltcGx5IGNhbGxpbmcgdGhlIHNwZGtf bnZtZV8qIGZ1bmN0aW9ucyBkaXJlY3RseSBhbmQgd29yayB3aXRoIHRoYXQuPGJyPjwvZGl2Pjwv ZGl2PjwvZGl2PjwvYmxvY2txdW90ZT48L2Rpdj48L2Rpdj48ZGl2IGRpcj0iYXV0byI+PGJyPjwv ZGl2PjxkaXYgZGlyPSJhdXRvIj5QYXNzdGhyb3VnaCBpbiBzdG9yYWdlIHR5cGljYWxseSByZWZl cnMgdG8gc3VibWl0dGluZyBhIHdlbGwtZm9ybWVkIHByb3RvY29sIGNvbW1hbmQgZGlyZWN0bHkg dG8gdGhlIGhhcmR3YXJlLiBUaGF0IHNvdW5kcyB2ZXJ5IG11Y2ggd2hhdCB5b3UmIzM5O3JlIGxv b2tpbmcgZm9yLCBqdWRnaW5nIGJ5IHRoZSBkaXNjdXNzaW9uLjwvZGl2PjxkaXYgZGlyPSJhdXRv Ij48YnI+PC9kaXY+PGRpdiBkaXI9ImF1dG8iPlJlZ2FyZHMsPC9kaXY+PGRpdiBkaXI9ImF1dG8i PkFuZHJleTwvZGl2PjxkaXYgZGlyPSJhdXRvIj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+PGJs b2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAwIC44ZXg7Ym9y ZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+PGRpdiBkaXI9Imx0ciI+ PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXY+PGJyPjwvZGl2PjxkaXY+QmFydWNoPC9kaXY+ PGJyPjwvZGl2PjwvZGl2Pi0tIDxicj48ZGl2IGRpcj0ibHRyIiBjbGFzcz0ibV8tODk5ODcyMjYy MzAxMjA5MzA3OWdtYWlsX3NpZ25hdHVyZSIgZGF0YS1zbWFydG1haWw9ImdtYWlsX3NpZ25hdHVy ZSI+PGRpdiBkaXI9Imx0ciI+PHNwYW4+PGRpdiBjbGFzcz0ibV8tODk5ODcyMjYyMzAxMjA5MzA3 OWdtYWlsX3NpZ25hdHVyZSI+PGRpdiBkaXI9Imx0ciI+PHNwYW4+PGI+PGI+PHNwYW4gc3R5bGU9 ImxpbmUtaGVpZ2h0Om5vcm1hbCI+PHNwYW4gc3R5bGU9ImNvbG9yOnJnYigwLDAsMCk7Zm9udC1z dHlsZTpub3JtYWw7Zm9udC13ZWlnaHQ6bm9ybWFsO2xldHRlci1zcGFjaW5nOm5vcm1hbDt0ZXh0 LWFsaWduOnN0YXJ0O3RleHQtaW5kZW50OjBweDt0ZXh0LXRyYW5zZm9ybTpub25lO3doaXRlLXNw YWNlOm5vcm1hbDt3b3JkLXNwYWNpbmc6MHB4O2xpbmUtaGVpZ2h0Om5vcm1hbDtmb250LWZhbWls eTpoZWx2ZXRpY2E7Zm9udC1zaXplOjEzcHgiPjxiIHN0eWxlPSJsaW5lLWhlaWdodDoyMXB4Ij48 aW1nIHNyYz0iaHR0cHM6Ly9kb2NzLmdvb2dsZS5jb20vdWM/ZXhwb3J0PWRvd25sb2FkJmFtcDtp ZD0xdzZtbGhDSlpSbHZxVk9LUm91bEZ0YWxVM1RNWTQxVlkmYW1wO3JldmlkPTBCd182Y0plU1NN VkpSMkpNUXpkeFYzVnFWaTlJV0ZORE0xRlZjbkZvUmxjMk5rSnpQUSI+PGJyPkJhcnVjaCBFdmVu LCA8c3BhbiBzdHlsZT0iY29sb3I6cmdiKDE0OCwxNDgsMTQ4KSI+U29mdHdhcmUgRGV2ZWxvcGVy wqAgPC9zcGFuPjxicj48L2I+PC9zcGFuPjxkaXYgc3R5bGU9ImNvbG9yOnJnYigwLDAsMCk7Zm9u dC1zdHlsZTpub3JtYWw7Zm9udC13ZWlnaHQ6bm9ybWFsO2xldHRlci1zcGFjaW5nOm5vcm1hbDt0 ZXh0LWFsaWduOnN0YXJ0O3RleHQtaW5kZW50OjBweDt0ZXh0LXRyYW5zZm9ybTpub25lO3doaXRl LXNwYWNlOm5vcm1hbDt3b3JkLXNwYWNpbmc6MHB4O2xpbmUtaGVpZ2h0Om5vcm1hbDtmb250LWZh bWlseTpoZWx2ZXRpY2E7Zm9udC1zaXplOjEzcHgiPjxzcGFuIHN0eWxlPSJsaW5lLWhlaWdodDoy MXB4O2NvbG9yOnJnYigxNDgsMTQ4LDE0OCkiPjxiPkXCoDwvYj4gPC9zcGFuPjxhIGhyZWY9Im1h aWx0bzpsaXJhbkB3ZWthLmlvIiBzdHlsZT0iY29sb3I6YmxhY2s7dGV4dC1kZWNvcmF0aW9uOm5v bmU7bGluZS1oZWlnaHQ6MjFweCIgdGFyZ2V0PSJfYmxhbmsiIHJlbD0ibm9yZWZlcnJlciI+YmFy dWNoQHdla2EuaW88L2E+IDxiciBzdHlsZT0ibGluZS1oZWlnaHQ6MjFweCI+PHNwYW4gc3R5bGU9 ImxpbmUtaGVpZ2h0OjIxcHgiPjxhIGhyZWY9Imh0dHA6Ly93d3cud2VrYS5pbyIgdGFyZ2V0PSJf YmxhbmsiIHJlbD0ibm9yZWZlcnJlciI+d3d3Lndla2EuaW88L2E+PC9zcGFuPjwvZGl2Pjwvc3Bh bj48L2I+PC9iPjwvc3Bhbj48L2Rpdj48L2Rpdj48L3NwYW4+PC9kaXY+PC9kaXY+Cl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fPGJyPgpTUERLIG1haWxpbmcg bGlzdDxicj4KPGEgaHJlZj0ibWFpbHRvOlNQREtAbGlzdHMuMDEub3JnIiB0YXJnZXQ9Il9ibGFu ayIgcmVsPSJub3JlZmVycmVyIj5TUERLQGxpc3RzLjAxLm9yZzwvYT48YnI+CjxhIGhyZWY9Imh0 dHBzOi8vbGlzdHMuMDEub3JnL21haWxtYW4vbGlzdGluZm8vc3BkayIgcmVsPSJub3JlZmVycmVy IG5vcmVmZXJyZXIiIHRhcmdldD0iX2JsYW5rIj5odHRwczovL2xpc3RzLjAxLm9yZy9tYWlsbWFu L2xpc3RpbmZvL3NwZGs8L2E+PGJyPgo8L2Jsb2NrcXVvdGU+PC9kaXY+PC9kaXY+PC9kaXY+Cg== --===============5473991519959481066==--