From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============1472569104247668859==" MIME-Version: 1.0 From: Kariuki, John K Subject: Re: [SPDK] spdk peer2peer dma fio latency Date: Tue, 25 Apr 2017 20:59:28 +0000 Message-ID: In-Reply-To: CABwN-bEp4Ox82NzWks_BGTrOkpsb0HzADc1jELASEKz1TL0Jcw@mail.gmail.com List-ID: To: spdk@lists.01.org --===============1472569104247668859== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Hello Can you provide some additional information? 1) Have you pre-conditioned the NVMe SSDs? 2) Which Intel Data Center NVMe SSDs are you using? I would like to lo= ok at the device spec and see the expected QD 1 latencies? 3) Are you doing random/seq 4K reads from the device? Thanks. From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of PR PR Sent: Tuesday, April 18, 2017 7:05 PM To: spdk(a)lists.01.org Subject: [SPDK] spdk peer2peer dma fio latency Hi, I am trying some experiments to evaluate performance of peer2peer dma. = I am using spdk to control the nvme drives and fio-plugin compiled with spd= k. I am seeing a weird behavior where when I run 4K IOs with IO-Depth of 1 = peer2peer DMA from nvme drive to some pci device (which exposes memory via = Bar1) in a different numa node has a 50th percentile latency of 17 usecs. T= he same experiment but where nvme device and pcie device in same numa node= (node 0) has a latency of 38 usecs. In both cases fio was running in node = 0 cpu core and pci device (which exposes memory via Bar1) is connected to n= ode 1. DMA from nvme device to host memory also takes 38 usecs. To summarize the cases below 1. nvme (numa node 0) - pci device (numa node 1) --- 18 usecs 2. nvme (numa node 1) - pci device (numa node 1) --- 38 usecs 3. nvme (numa node 0) - host memory --- 38 usecs fio running in numa node 0 cpu core in all cases. For higher IO Depth values cross numa case (case 1 above), latency increase= s steeply and performs poorly than case 2 and case 3. Any pointers on why this could be happening? The nvme devices used are both identical intel datacenter ssd 400G. Thanks --===============1472569104247668859== Content-Type: text/html MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.html" PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy IDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8NCnAuTXNvTm9ybWFsLCBsaS5Nc29Ob3JtYWws IGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBpbjsNCgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJ Zm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWlseToiVGltZXMgTmV3IFJvbWFuIixzZXJpZjt9 DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCglj b2xvcjojMDU2M0MxOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0KYTp2aXNpdGVkLCBz cGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgljb2xv cjojOTU0RjcyOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0KcC5Nc29MaXN0UGFyYWdy YXBoLCBsaS5Nc29MaXN0UGFyYWdyYXBoLCBkaXYuTXNvTGlzdFBhcmFncmFwaA0KCXttc28tc3R5 bGUtcHJpb3JpdHk6MzQ7DQoJbWFyZ2luLXRvcDowaW47DQoJbWFyZ2luLXJpZ2h0OjBpbjsNCglt YXJnaW4tYm90dG9tOjBpbjsNCgltYXJnaW4tbGVmdDouNWluOw0KCW1hcmdpbi1ib3R0b206LjAw MDFwdDsNCglmb250LXNpemU6MTIuMHB0Ow0KCWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcgUm9tYW4i LHNlcmlmO30NCnNwYW4uRW1haWxTdHlsZTE3DQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLXJl cGx5Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixzYW5zLXNlcmlmOw0KCWNvbG9yOiMxRjQ5N0Q7 fQ0KLk1zb0NocERlZmF1bHQNCgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0LW9ubHk7DQoJZm9udC1m YW1pbHk6IkNhbGlicmkiLHNhbnMtc2VyaWY7fQ0KQHBhZ2UgV29yZFNlY3Rpb24xDQoJe3NpemU6 OC41aW4gMTEuMGluOw0KCW1hcmdpbjoxLjBpbiAxLjBpbiAxLjBpbiAxLjBpbjt9DQpkaXYuV29y ZFNlY3Rpb24xDQoJe3BhZ2U6V29yZFNlY3Rpb24xO30NCi8qIExpc3QgRGVmaW5pdGlvbnMgKi8N CkBsaXN0IGwwDQoJe21zby1saXN0LWlkOjE2NzYwMzExNTM7DQoJbXNvLWxpc3QtdHlwZTpoeWJy aWQ7DQoJbXNvLWxpc3QtdGVtcGxhdGUtaWRzOjE1MTY1MDU0MTIgNjc2OTg3MDUgNjc2OTg3MTMg Njc2OTg3MTUgNjc2OTg3MDMgNjc2OTg3MTMgNjc2OTg3MTUgNjc2OTg3MDMgNjc2OTg3MTMgNjc2 OTg3MTU7fQ0KQGxpc3QgbDA6bGV2ZWwxDQoJe21zby1sZXZlbC10ZXh0OiIlMVwpIjsNCgltc28t bGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJ dGV4dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGwwOmxldmVsMg0KCXttc28tbGV2ZWwtbnVtYmVy LWZvcm1hdDphbHBoYS1sb3dlcjsNCgltc28tbGV2ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2 ZWwtbnVtYmVyLXBvc2l0aW9uOmxlZnQ7DQoJdGV4dC1pbmRlbnQ6LS4yNWluO30NCkBsaXN0IGww OmxldmVsMw0KCXttc28tbGV2ZWwtbnVtYmVyLWZvcm1hdDpyb21hbi1sb3dlcjsNCgltc28tbGV2 ZWwtdGFiLXN0b3A6bm9uZTsNCgltc28tbGV2ZWwtbnVtYmVyLXBvc2l0aW9uOnJpZ2h0Ow0KCXRl eHQtaW5kZW50Oi05LjBwdDt9DQpAbGlzdCBsMDpsZXZlbDQNCgl7bXNvLWxldmVsLXRhYi1zdG9w Om5vbmU7DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpsZWZ0Ow0KCXRleHQtaW5kZW50Oi0u MjVpbjt9DQpAbGlzdCBsMDpsZXZlbDUNCgl7bXNvLWxldmVsLW51bWJlci1mb3JtYXQ6YWxwaGEt bG93ZXI7DQoJbXNvLWxldmVsLXRhYi1zdG9wOm5vbmU7DQoJbXNvLWxldmVsLW51bWJlci1wb3Np dGlvbjpsZWZ0Ow0KCXRleHQtaW5kZW50Oi0uMjVpbjt9DQpAbGlzdCBsMDpsZXZlbDYNCgl7bXNv LWxldmVsLW51bWJlci1mb3JtYXQ6cm9tYW4tbG93ZXI7DQoJbXNvLWxldmVsLXRhYi1zdG9wOm5v bmU7DQoJbXNvLWxldmVsLW51bWJlci1wb3NpdGlvbjpyaWdodDsNCgl0ZXh0LWluZGVudDotOS4w cHQ7fQ0KQGxpc3QgbDA6bGV2ZWw3DQoJe21zby1sZXZlbC10YWItc3RvcDpub25lOw0KCW1zby1s ZXZlbC1udW1iZXItcG9zaXRpb246bGVmdDsNCgl0ZXh0LWluZGVudDotLjI1aW47fQ0KQGxpc3Qg bDA6bGV2ZWw4DQoJe21zby1sZXZlbC1udW1iZXItZm9ybWF0OmFscGhhLWxvd2VyOw0KCW1zby1s ZXZlbC10YWItc3RvcDpub25lOw0KCW1zby1sZXZlbC1udW1iZXItcG9zaXRpb246bGVmdDsNCgl0 ZXh0LWluZGVudDotLjI1aW47fQ0KQGxpc3QgbDA6bGV2ZWw5DQoJe21zby1sZXZlbC1udW1iZXIt Zm9ybWF0OnJvbWFuLWxvd2VyOw0KCW1zby1sZXZlbC10YWItc3RvcDpub25lOw0KCW1zby1sZXZl bC1udW1iZXItcG9zaXRpb246cmlnaHQ7DQoJdGV4dC1pbmRlbnQ6LTkuMHB0O30NCm9sDQoJe21h cmdpbi1ib3R0b206MGluO30NCnVsDQoJe21hcmdpbi1ib3R0b206MGluO30NCi0tPjwvc3R5bGU+ PCEtLVtpZiBndGUgbXNvIDldPjx4bWw+DQo8bzpzaGFwZWRlZmF1bHRzIHY6ZXh0PSJlZGl0IiBz cGlkbWF4PSIxMDI2IiAvPg0KPC94bWw+PCFbZW5kaWZdLS0+PCEtLVtpZiBndGUgbXNvIDldPjx4 bWw+DQo8bzpzaGFwZWxheW91dCB2OmV4dD0iZWRpdCI+DQo8bzppZG1hcCB2OmV4dD0iZWRpdCIg ZGF0YT0iMSIgLz4NCjwvbzpzaGFwZWxheW91dD48L3htbD48IVtlbmRpZl0tLT4NCjwvaGVhZD4N Cjxib2R5IGxhbmc9IkVOLVVTIiBsaW5rPSIjMDU2M0MxIiB2bGluaz0iIzk1NEY3MiI+DQo8ZGl2 IGNsYXNzPSJXb3JkU2VjdGlvbjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9 ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNl cmlmO2NvbG9yOiMxRjQ5N0QiPkhlbGxvPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9 Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1 b3Q7Q2FsaWJyaSZxdW90OyxzYW5zLXNlcmlmO2NvbG9yOiMxRjQ5N0QiPkNhbiB5b3UgcHJvdmlk ZSBzb21lIGFkZGl0aW9uYWwgaW5mb3JtYXRpb24/PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAg Y2xhc3M9Ik1zb0xpc3RQYXJhZ3JhcGgiIHN0eWxlPSJ0ZXh0LWluZGVudDotLjI1aW47bXNvLWxp c3Q6bDAgbGV2ZWwxIGxmbzEiPjwhW2lmICFzdXBwb3J0TGlzdHNdPjxzcGFuIHN0eWxlPSJmb250 LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtj b2xvcjojMUY0OTdEIj48c3BhbiBzdHlsZT0ibXNvLWxpc3Q6SWdub3JlIj4xKTxzcGFuIHN0eWxl PSJmb250OjcuMHB0ICZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyI+Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7DQo8L3NwYW4+PC9zcGFuPjwvc3Bhbj48IVtlbmRpZl0+PHNwYW4gc3R5 bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7Q2FsaWJyaSZxdW90OyxzYW5z LXNlcmlmO2NvbG9yOiMxRjQ5N0QiPkhhdmUgeW91IHByZS1jb25kaXRpb25lZCB0aGUgTlZNZSBT U0RzPzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29MaXN0UGFyYWdyYXBoIiBz dHlsZT0idGV4dC1pbmRlbnQ6LS4yNWluO21zby1saXN0OmwwIGxldmVsMSBsZm8xIj48IVtpZiAh c3VwcG9ydExpc3RzXT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWlseTom cXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+PHNwYW4gc3R5bGU9 Im1zby1saXN0Oklnbm9yZSI+Mik8c3BhbiBzdHlsZT0iZm9udDo3LjBwdCAmcXVvdDtUaW1lcyBO ZXcgUm9tYW4mcXVvdDsiPiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOw0KPC9zcGFuPjwv c3Bhbj48L3NwYW4+PCFbZW5kaWZdPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQt ZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj5XaGlj aCBJbnRlbCBEYXRhIENlbnRlciBOVk1lIFNTRHMgYXJlIHlvdSB1c2luZz8gSSB3b3VsZCBsaWtl IHRvIGxvb2sgYXQgdGhlIGRldmljZSBzcGVjIGFuZCBzZWUgdGhlIGV4cGVjdGVkIFFEIDEgbGF0 ZW5jaWVzPzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29MaXN0UGFyYWdyYXBo IiBzdHlsZT0idGV4dC1pbmRlbnQ6LS4yNWluO21zby1saXN0OmwwIGxldmVsMSBsZm8xIj48IVtp ZiAhc3VwcG9ydExpc3RzXT48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWls eTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFGNDk3RCI+PHNwYW4gc3R5 bGU9Im1zby1saXN0Oklnbm9yZSI+Myk8c3BhbiBzdHlsZT0iZm9udDo3LjBwdCAmcXVvdDtUaW1l cyBOZXcgUm9tYW4mcXVvdDsiPiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOw0KPC9zcGFu Pjwvc3Bhbj48L3NwYW4+PCFbZW5kaWZdPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2Zv bnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xvcjojMUY0OTdEIj5B cmUgeW91IGRvaW5nIHJhbmRvbS9zZXEgNEsgcmVhZHMgZnJvbSB0aGUgZGV2aWNlPzxvOnA+PC9v OnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNp emU6MTEuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O0NhbGlicmkmcXVvdDssc2Fucy1zZXJpZjtjb2xv cjojMUY0OTdEIj5UaGFua3MuPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+PGEgbmFtZT0iX01haWxFbmRDb21wb3NlIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEx LjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWY7Y29sb3I6IzFG NDk3RCI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9hPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxhIG5hbWU9Il9fX19fcmVwbHlzZXBhcmF0b3IiPjwvYT48Yj48c3BhbiBzdHlsZT0iZm9u dC1zaXplOjExLjBwdDtmb250LWZhbWlseTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYi PkZyb206PC9zcGFuPjwvYj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtmb250LWZhbWls eTomcXVvdDtDYWxpYnJpJnF1b3Q7LHNhbnMtc2VyaWYiPiBTUERLIFttYWlsdG86c3Bkay1ib3Vu Y2VzQGxpc3RzLjAxLm9yZ10NCjxiPk9uIEJlaGFsZiBPZiA8L2I+UFIgUFI8YnI+DQo8Yj5TZW50 OjwvYj4gVHVlc2RheSwgQXByaWwgMTgsIDIwMTcgNzowNSBQTTxicj4NCjxiPlRvOjwvYj4gc3Bk a0BsaXN0cy4wMS5vcmc8YnI+DQo8Yj5TdWJqZWN0OjwvYj4gW1NQREtdIHNwZGsgcGVlcjJwZWVy IGRtYSBmaW8gbGF0ZW5jeTxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3Jt YWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkhp LCBJIGFtIHRyeWluZyBzb21lIGV4cGVyaW1lbnRzIHRvIGV2YWx1YXRlIHBlcmZvcm1hbmNlIG9m IHBlZXIycGVlciBkbWEuIEkgYW0gdXNpbmcgc3BkayB0byBjb250cm9sIHRoZSBudm1lIGRyaXZl cyBhbmQgZmlvLXBsdWdpbiBjb21waWxlZCB3aXRoIHNwZGsuIEkgYW0gc2VlaW5nIGEgd2VpcmQg YmVoYXZpb3Igd2hlcmUgd2hlbiBJIHJ1biA0SyBJT3Mgd2l0aCBJTy1EZXB0aCBvZiAxIHBlZXIy cGVlciBETUENCiBmcm9tIG52bWUgZHJpdmUgdG8gc29tZSBwY2kgZGV2aWNlICh3aGljaCBleHBv c2VzIG1lbW9yeSB2aWEgQmFyMSkmbmJzcDtpbiBhIGRpZmZlcmVudCBudW1hIG5vZGUgaGFzIGEg NTB0aCBwZXJjZW50aWxlIGxhdGVuY3kgb2YgMTcgdXNlY3MuIFRoZSBzYW1lICZuYnNwO2V4cGVy aW1lbnQgYnV0IHdoZXJlIG52bWUgZGV2aWNlIGFuZCBwY2llIGRldmljZSBpbiBzYW1lIG51bWEg bm9kZSAobm9kZSAwKSBoYXMgYSBsYXRlbmN5IG9mIDM4IHVzZWNzLiBJbiBib3RoDQogY2FzZXMg ZmlvIHdhcyBydW5uaW5nIGluIG5vZGUgMCBjcHUgY29yZSBhbmQgcGNpIGRldmljZSAod2hpY2gg ZXhwb3NlcyBtZW1vcnkgdmlhIEJhcjEpIGlzIGNvbm5lY3RlZCB0byBub2RlIDEuIERNQSBmcm9t IG52bWUgZGV2aWNlIHRvIGhvc3QgbWVtb3J5IGFsc28gdGFrZXMgMzggdXNlY3MuJm5ic3A7PG86 cD48L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286 cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5UbyBzdW1tYXJpemUg dGhlIGNhc2VzIGJlbG93PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0i TXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNz PSJNc29Ob3JtYWwiPjEuIG52bWUgKG51bWEgbm9kZSAwKSAtIHBjaSBkZXZpY2UgKG51bWEgbm9k ZSAxKSAmbmJzcDsgLS0tIDE4IHVzZWNzPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8 cCBjbGFzcz0iTXNvTm9ybWFsIj4yLiBudm1lIChudW1hIG5vZGUgMSkgLSBwY2kgZGV2aWNlIChu dW1hIG5vZGUgMSkgJm5ic3A7IC0tLSAzOCB1c2VjczxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8 ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+My4gbnZtZSAobnVtYSBub2RlIDApIC0gaG9zdCBt ZW1vcnkgJm5ic3A7IC0tLSAzOCB1c2VjczxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0K PHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+ DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5maW8gcnVubmluZyBpbiBudW1hIG5vZGUgMCBjcHUgY29y ZSBpbiBhbGwgY2FzZXMuJm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBj bGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPkZvciBoaWdoZXIgSU8gRGVwdGggdmFsdWVzIGNyb3NzIG51bWEg Y2FzZSAoY2FzZSAxIGFib3ZlKSwgbGF0ZW5jeSBpbmNyZWFzZXMgc3RlZXBseSBhbmQgcGVyZm9y bXMgcG9vcmx5IHRoYW4gY2FzZSAyIGFuZCBjYXNlIDMuJm5ic3A7PG86cD48L286cD48L3A+DQo8 L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4N CjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkFueSBwb2ludGVycyBvbiB3aHkg dGhpcyBjb3VsZCBiZSBoYXBwZW5pbmc/PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8 cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPlRoZSBudm1lIGRldmljZXMgdXNlZCBhcmUgYm90aCBpZGVu dGljYWwgaW50ZWwgZGF0YWNlbnRlciBzc2QgNDAwRy48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0K PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+ DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+VGhhbmtzPG86cD48L286cD48L3A+DQo8L2Rp dj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwv ZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4NCg== --===============1472569104247668859==--