From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kenneth Lee Subject: Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive Date: Fri, 3 Aug 2018 11:47:21 +0800 Message-ID: <20180803034721.GC91035@Turing-Arch-b> References: <20180801102221.5308-1-nek.in.cn@gmail.com> <20180801165644.GA3820@redhat.com> <20180802040557.GL160746@Turing-Arch-b> <20180802142243.GA3481@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: "Tian, Kevin" , Herbert Xu , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Jonathan Corbet , Greg Kroah-Hartman , Zaibo Xu , "linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "Kumar, Sanjay K" , Hao Fang , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linuxarm-hv44wF8Li93QT0dZR+AlfA@public.gmane.org" , Alex Williamson , "linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Philippe Ombredanne , Thomas Gleixner , Kenneth Lee , "David S . Miller" , "linux-accelerators-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org" Return-path: Content-Disposition: inline In-Reply-To: <20180802142243.GA3481-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org List-Id: linux-crypto.vger.kernel.org T24gVGh1LCBBdWcgMDIsIDIwMTggYXQgMTA6MjI6NDNBTSAtMDQwMCwgSmVyb21lIEdsaXNzZSB3 cm90ZToKPiBEYXRlOiBUaHUsIDIgQXVnIDIwMTggMTA6MjI6NDMgLTA0MDAKPiBGcm9tOiBKZXJv bWUgR2xpc3NlIDxqZ2xpc3NlQHJlZGhhdC5jb20+Cj4gVG86IEtlbm5ldGggTGVlIDxsaWd1b3po dUBoaXNpbGljb24uY29tPgo+IENDOiAiVGlhbiwgS2V2aW4iIDxrZXZpbi50aWFuQGludGVsLmNv bT4sIEhhbyBGYW5nIDxmYW5naGFvMTFAaHVhd2VpLmNvbT4sCj4gIEFsZXggV2lsbGlhbXNvbiA8 YWxleC53aWxsaWFtc29uQHJlZGhhdC5jb20+LCBIZXJiZXJ0IFh1Cj4gIDxoZXJiZXJ0QGdvbmRv ci5hcGFuYS5vcmcuYXU+LCAia3ZtQHZnZXIua2VybmVsLm9yZyIKPiAgPGt2bUB2Z2VyLmtlcm5l bC5vcmc+LCBKb25hdGhhbiBDb3JiZXQgPGNvcmJldEBsd24ubmV0PiwgR3JlZwo+ICBLcm9haC1I YXJ0bWFuIDxncmVna2hAbGludXhmb3VuZGF0aW9uLm9yZz4sIFphaWJvIFh1IDx4dXphaWJvQGh1 YXdlaS5jb20+LAo+ICAibGludXgtZG9jQHZnZXIua2VybmVsLm9yZyIgPGxpbnV4LWRvY0B2Z2Vy Lmtlcm5lbC5vcmc+LCAiS3VtYXIsIFNhbmpheSBLIgo+ICA8c2FuamF5Lmsua3VtYXJAaW50ZWwu Y29tPiwgS2VubmV0aCBMZWUgPG5lay5pbi5jbkBnbWFpbC5jb20+LAo+ICAiaW9tbXVAbGlzdHMu bGludXgtZm91bmRhdGlvbi5vcmciIDxpb21tdUBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZz4s Cj4gICJsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnIiA8bGludXgta2VybmVsQHZnZXIua2Vy bmVsLm9yZz4sCj4gICJsaW51eGFybUBodWF3ZWkuY29tIiA8bGludXhhcm1AaHVhd2VpLmNvbT4s Cj4gICJsaW51eC1jcnlwdG9Admdlci5rZXJuZWwub3JnIiA8bGludXgtY3J5cHRvQHZnZXIua2Vy bmVsLm9yZz4sIFBoaWxpcHBlCj4gIE9tYnJlZGFubmUgPHBvbWJyZWRhbm5lQG5leGIuY29tPiwg VGhvbWFzIEdsZWl4bmVyIDx0Z2x4QGxpbnV0cm9uaXguZGU+LAo+ICAiRGF2aWQgUyAuIE1pbGxl ciIgPGRhdmVtQGRhdmVtbG9mdC5uZXQ+LAo+ICAibGludXgtYWNjZWxlcmF0b3JzQGxpc3RzLm96 bGFicy5vcmciCj4gIDxsaW51eC1hY2NlbGVyYXRvcnNAbGlzdHMub3psYWJzLm9yZz4KPiBTdWJq ZWN0OiBSZTogW1JGQyBQQVRDSCAwLzddIEEgR2VuZXJhbCBBY2NlbGVyYXRvciBGcmFtZXdvcmss IFdhcnBEcml2ZQo+IFVzZXItQWdlbnQ6IE11dHQvMS4xMC4wICgyMDE4LTA1LTE3KQo+IE1lc3Nh Z2UtSUQ6IDwyMDE4MDgwMjE0MjI0My5HQTM0ODFAcmVkaGF0LmNvbT4KPiAKPiBPbiBUaHUsIEF1 ZyAwMiwgMjAxOCBhdCAxMjowNTo1N1BNICswODAwLCBLZW5uZXRoIExlZSB3cm90ZToKPiA+IE9u IFRodSwgQXVnIDAyLCAyMDE4IGF0IDAyOjMzOjEyQU0gKzAwMDAsIFRpYW4sIEtldmluIHdyb3Rl Ogo+ID4gPiBEYXRlOiBUaHUsIDIgQXVnIDIwMTggMDI6MzM6MTIgKzAwMDAKPiA+ID4gPiBGcm9t OiBKZXJvbWUgR2xpc3NlCj4gPiA+ID4gT24gV2VkLCBBdWcgMDEsIDIwMTggYXQgMDY6MjI6MTRQ TSArMDgwMCwgS2VubmV0aCBMZWUgd3JvdGU6Cj4gPiA+ID4gPiBGcm9tOiBLZW5uZXRoIExlZSA8 bGlndW96aHVAaGlzaWxpY29uLmNvbT4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBXYXJwRHJpdmUgaXMg YW4gYWNjZWxlcmF0b3IgZnJhbWV3b3JrIHRvIGV4cG9zZSB0aGUgaGFyZHdhcmUKPiA+ID4gPiBj YXBhYmlsaXRpZXMKPiA+ID4gPiA+IGRpcmVjdGx5IHRvIHRoZSB1c2VyIHNwYWNlLiBJdCBtYWtl cyB1c2Ugb2YgdGhlIGV4aXN0IHZmaW8gYW5kIHZmaW8tbWRldgo+ID4gPiA+ID4gZmFjaWxpdGll cy4gU28gdGhlIHVzZXIgYXBwbGljYXRpb24gY2FuIHNlbmQgcmVxdWVzdCBhbmQgRE1BIHRvIHRo ZQo+ID4gPiA+ID4gaGFyZHdhcmUgd2l0aG91dCBpbnRlcmFjdGlvbiB3aXRoIHRoZSBrZXJuZWwu IFRoaXMgcmVtb3ZlIHRoZSBsYXRlbmN5Cj4gPiA+ID4gPiBvZiBzeXNjYWxsIGFuZCBjb250ZXh0 IHN3aXRjaC4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBUaGUgcGF0Y2hzZXQgY29udGFpbnMgZG9jdW1l bnRzIGZvciB0aGUgZGV0YWlsLiBQbGVhc2UgcmVmZXIgdG8gaXQgZm9yCj4gPiA+ID4gbW9yZQo+ ID4gPiA+ID4gaW5mb3JtYXRpb24uCj4gPiA+ID4gPgo+ID4gPiA+ID4gVGhpcyBwYXRjaHNldCBp cyBpbnRlbmRlZCB0byBiZSB1c2VkIHdpdGggSmVhbiBQaGlsaXBwZSBCcnVja2VyJ3MgU1ZBCj4g PiA+ID4gPiBwYXRjaCBbMV0gKFdoaWNoIGlzIGFsc28gaW4gUkZDIHN0YWdlKS4gQnV0IGl0IGlz IG5vdCBtYW5kYXRvcnkuIFRoaXMKPiA+ID4gPiA+IHBhdGNoc2V0IGlzIHRlc3RlZCBpbiB0aGUg bGF0ZXN0IG1haW5saW5lIGtlcm5lbCB3aXRob3V0IHRoZSBTVkEgcGF0Y2hlcy4KPiA+ID4gPiA+ IFNvIGl0IHN1cHBvcnQgb25seSBvbmUgcHJvY2VzcyBmb3IgZWFjaCBhY2NlbGVyYXRvci4KPiA+ ID4gPiA+Cj4gPiA+ID4gPiBXaXRoIFNWQSBzdXBwb3J0LCBXYXJwRHJpdmUgY2FuIHN1cHBvcnQg bXVsdGktcHJvY2VzcyBpbiB0aGUgc2FtZQo+ID4gPiA+ID4gYWNjZWxlcmF0b3IgZGV2aWNlLiAg V2UgdGVzdGVkIGl0IGluIG91ciBTb0MgaW50ZWdyYXRlZCBBY2NlbGVyYXRvciAoYm9hcmQKPiA+ ID4gPiA+IElEOiBEMDYsIENoaXAgSUQ6IEhJUDA4KS4gQSByZWZlcmVuY2Ugd29yayB0cmVlIGNh biBiZSBmb3VuZCBoZXJlOiBbMl0uCj4gPiA+ID4gCj4gPiA+ID4gSSBoYXZlIG5vdCBmdWxseSBp bnNwZWN0ZWQgdGhpbmdzIG5vciBkbyBpIGtub3cgZW5vdWdoIGFib3V0Cj4gPiA+ID4gdGhpcyBI aXNpbGljb24gWklQIGFjY2VsZXJhdG9yIHRvIGFzY2VydGFpbiwgYnV0IGZyb20gZ2xpbXBzaW5n Cj4gPiA+ID4gYXQgdGhlIGNvZGUgaXQgc2VlbXMgdGhhdCBpdCBpcyB1bnNhZmUgdG8gdXNlIGV2 ZW4gd2l0aCBTVkEgZHVlCj4gPiA+ID4gdG8gdGhlIGRvb3JiZWxsLiBUaGVyZSBpcyBhIGNvbW1l bnQgdGFsa2luZyBhYm91dCBzYWZldHluZXNzCj4gPiA+ID4gaW4gcGF0Y2ggNy4KPiA+ID4gPiAK PiA+ID4gPiBFeHBvc2luZyB0aGluZyB0byB1c2Vyc3BhY2UgaXMgYWx3YXlzIGVudGljaW5nLCBi dXQgaWYgaXQgaXMKPiA+ID4gPiBhIHNlY3VyaXR5IHJpc2sgdGhlbiBpdCBzaG91bGQgY2xlYXJs eSBzYXkgc28gYW5kIG1heWJlIGEKPiA+ID4gPiBrZXJuZWwgYm9vdCBmbGFnIHNob3VsZCBiZSBu ZWNlc3NhcnkgdG8gYWxsb3cgc3VjaCBkZXZpY2UgdG8KPiA+ID4gPiBiZSB1c2UuCj4gPiA+ID4g Cj4gPiAKPiA+IEJ1dCBkb29yYmVsbCBpcyBqdXN0IGEgbm90aWZpY2F0aW9uLiBFeGNlcHQgZm9y IERPUyAodG8gbWFrZSBoYXJkd2FyZSBidXN5KSBpdAo+ID4gY2Fubm90IGFjdHVhbGx5IHRha2Ug b3IgY2hhbmdlIGFueXRoaW5nIGZyb20gdGhlIGtlcm5lbCBzcGFjZS4gQW5kIHRoZSBET1MKPiA+ IHByb2JsZW0gY2FuIGJlIGFsd2F5cyB0YWtlbiBhcyB0aGUgcHJvYmxlbSB0aGF0IGEgZ3JvdXAg b2YgcHJvY2Vzc2VzIHNoYXJlIHRoZQo+ID4gc2FtZSBrZXJuZWwgZW50aXR5Lgo+ID4gCj4gPiBJ biB0aGUgY29taW5nIEhJUDA5IGhhcmR3YXJlLCB0aGUgZG9vcmJlbGwgd2lsbCBjb21lIHdpdGgg YSByYW5kb20gbnVtYmVyIHNvCj4gPiBvbmx5IHRoZSBwcm9jZXNzIHdobyBhbGxvY2F0ZWQgdGhl IHF1ZXVlIGNhbiBrbm9jayBpdCBjb3JyZWN0bHkuCj4gCj4gV2hlbiBkb29yYmVsbCBpcyByaW5n IHRoZSBoYXJkd2FyZSBzdGFydCBmZXRjaGluZyBjb21tYW5kcyBmcm9tCj4gdGhlIHF1ZXVlIGFu ZCBleGVjdXRlIHRoZW0gPyBJZiBzbyB0aGFuIGEgcm9ndWUgcHJvY2VzcyBCIG1pZ2h0Cj4gcmlu ZyB0aGUgZG9vcmJlbGwgb2YgcHJvY2VzcyBBIHdoaWNoIHdvdWxkIHN0YXJ0cyBleGVjdXRpb24g b2YKPiByYW5kb20gY29tbWFuZHMgKGllIHdoYXRldmVyIHJhbmRvbSBtZW1vcnkgdmFsdWUgdGhl cmUgaXMgbGVmdAo+IGluc2lkZSB0aGUgY29tbWFuZCBidWZmZXIgbWVtb3J5LCBjb3VsZCBiZSBv bGQgY29tbWFuZHMgaSBndWVzcykuCj4gCj4gSWYgdGhpcyBpcyBub3QgaG93IHRoaXMgZG9vcmJl bGwgd29ya3MgdGhlbiwgeWVzIGl0IGNhbiBvbmx5IGRvCj4gYSBkZW5pYWwgb2Ygc2VydmljZSBp IGd1ZXNzLiBJc3N1ZSBpIGhhdmUgd2l0aCBkb29yYmVsbCBpcyB0aGF0Cj4gaSBoYXZlIHNlZW4g MTAgZGlmZmVyZW50cyBpbXBsZW1lbnRhdGlvbnMgaW4gMTAgZGlmZmVyZW50cyBodwo+IGFuZCBl YWNoIGFyZSBkaWZmZXJlbnQgYXMgdG8gd2hhdCByaW5naW5nIG9yIHZhbHVlIHdyaXR0ZW4gdG8g dGhlCj4gZG9vcmJlbGwgZG9lcy4gSXQgaXMgcGFpbmZ1bGwgdG8gdHJhY2sgd2hhdCBpcyB3aGF0 IGZvciBlYWNoIGh3Lgo+IAoKSW4gb3VyIGltcGxlbWVudGF0aW9uLCBkb29yYmVsbCBpcyBzaW1w bHkgYSBub3RpZmljYXRpb24sIGp1c3QgbGlrZSBhbiBpbnRlcnJ1cHQKdG8gdGhlIGFjY2VsZXJh dG9yLiBUaGUgY29tbWFuZCBpcyBhbGwgYWJvdXQgd2hhdCdzIGluIHRoZSBxdWV1ZS4KCkkgYWdy ZWUgdGhhdCB0aGVyZSBpcyBubyBzaW1wbGUgYW5kIHN0YW5kYXJkIHdheSB0byB0cmFjayB0aGUg c2hhcmVkIElPIHNwYWNlLgpCdXQgSSB0aGluayB3ZSBoYXZlIHRvIHRydXN0IHRoZSBkcml2ZXIg aW4gc29tZSB3YXkuIElmIHRoZSBkcml2ZXIgaXMgbWFsaWNpb3VzLApldmVuIGEgc2ltcGxlIGlv Y3RsIGNhbiBiZWNvbWUgYW4gYXR0YWNrLgoKPiAKPiA+ID4gPiBNeSBtb3JlIGdlbmVyYWwgcXVl c3Rpb24gaXMgZG8gd2Ugd2FudCB0byBncm93IFZGSU8gdG8gYmVjb21lCj4gPiA+ID4gYSBtb3Jl IGdlbmVyaWMgZGV2aWNlIGRyaXZlciBBUEkuIFRoaXMgcGF0Y2hzZXQgYWRkcyBhIGNvbW1hbmQK PiA+ID4gPiBxdWV1ZSBjb25jZXB0IHRvIGl0IChpIGRvbid0IHRoaW5rIGl0IGV4aXN0IHRvZGF5 IGJ1dCBpIGhhdmUKPiA+ID4gPiBub3QgZm9sbG93IFZGSU8gY2xvc2VseSkuCj4gPiA+ID4gCj4g PiAKPiA+IFRoZSB0aGluZyBpcywgVkZJTyBpcyB0aGUgb25seSBwbGFjZSB0byBzdXBwb3J0IERN QSBmcm9tIHVzZXIgbGFuZC4gSWYgd2UgZG9uJ3QKPiA+IHB1dCBpdCBoZXJlLCB3ZSBoYXZlIHRv IGNyZWF0ZSBhbm90aGVyIHNpbWlsYXIgZmFjaWxpdHkgdG8gc3VwcG9ydCB0aGUgc2FtZS4KPiAK PiBObyBpdCBpcyBub3QsIG5ldHdvcmsgZGV2aWNlLCBHUFUsIGJsb2NrIGRldmljZSwgLi4uIHRo ZXkgYWxsIGRvCj4gc3VwcG9ydCBETUEuIFRoZSBwb2ludCBpIGFtIHRyeWluZyB0byBtYWtlIGhl cmUgaXMgdGhhdCBldmVuIGluCgpTb3JyeSwgd2FpdCBhIG1pbnV0ZSwgYXJlIHdlIHRhbGtpbmcg dGhlIHNhbWUgdGhpbmc/IEkgbWVhbnQgIkRNQSBmcm9tIHVzZXIKbGFuZCIsIG5vdCAiRE1BIGZy b20ga2VybmVsIGRyaXZlciIuIFRvIGRvIHRoYXQgd2UgaGF2ZSB0byBtYW5pcHVsYXRlIHRoZQpJ T01NVShVbml0KS4gSSB0aGluayBpdCBjYW4gb25seSBiZSBkb25lIGJ5IGRlZmF1bHRfZG9tYWlu IG9yIHZmaW8gZG9tYWluLiBPcgp0aGUgdXNlciBzcGFjZSBoYXZlIHRvIGRpcmVjdGx5IGFjY2Vz cyB0aGUgSU9NTVUuCgo+IHlvdXIgbWVjaGFuaXNtcyB0aGUgdXNlcnNwYWNlIG11c3QgaGF2ZSBh IHNwZWNpZmljIHVzZXJzcGFjZQo+IGRyaXZlcnMgZm9yIGVhY2ggaGFyZHdhcmUgYW5kIHRodXMg dGhlcmUgYXJlIHZpcnR1YWxseSBubwo+IGRpZmZlcmVuY2VzIGJldHdlZW4gaGF2aW5nIHRoaXMg dXNlcnNwYWNlIGRyaXZlciBvcGVuIGEgZGV2aWNlCj4gZmlsZSBpbiB2ZmlvIG9yIHNvbWV3aGVy ZSBlbHNlIGluIHRoZSBkZXZpY2UgZmlsZXN5c3RlbS4gVGhpcyBpcwo+IGp1c3QgYSBkaWZmZXJl bnQgcGF0aC4KPiAKClRoZSBiYXNpYyBwcm9ibGVtIFdhcnBEcml2ZSB3YW50IHRvIHNvbHZlIGl0 IHRvIGF2b2lkIHN5c2NhbGwuIFRoaXMgaXMgaW1wb3J0YW50CnRvIGFjY2VsZXJhdG9ycy4gV2Ug aGF2ZSBzb21lIGRhdGEgaGVyZToKaHR0cHM6Ly93d3cuc2xpZGVzaGFyZS5uZXQvbGluYXJvb3Jn L3Byb2dyZXNzLWFuZC1kZW1vbnN0cmF0aW9uLW9mLXdyYXBkcml2ZS1hLWFjY2VsZXJhdG9yLWZy YW1ld29yay1zZm8xNzMxNwoKKHNlZSBwYWdlIDMpCgpUaGUgcGVyZm9ybWFuY2UgaXMgZGlmZmVy ZW50IG9uIHVzaW5nIGtlcm5lbCBhbmQgdXNlciBkcml2ZXJzLgoKQW5kIHdlIGFsc28gYmVsaWV2 ZSB0aGUgaGFyZHdhcmUgaW50ZXJmYWNlIGNhbiBiZWNvbWUgc3RhbmRhcmQgYWZ0ZXIgc29tZXRp bWUuClNvbWUgY29tcGFuaWVzIGhhdmUgc3RhcnRlZCB0byBkbyB0aGlzIChzdWNoIEFSTSdzIFJl dmVyZSkuIEJ1dCBiZWZvcmUgdGhhdCwgd2UKc2hvdWxkIGhhdmUgYSBzb2Z0d2FyZSBjaGFubmVs IGZvciBpdC4KCj4gU28gdGhpcyBpcyB3aHkgaSBkbyBub3Qgc2VlIGFueSBiZW5lZml0IHRvIGhh dmluZyBhbGwgZHJpdmVycyB3aXRoCj4gU1ZNIChjYW4gd2UgcGxlYXNlIHVzZSBTVk0gYW5kIG5v dCBTVkEgYXMgU1ZNIGlzIHdoYXQgaGF2ZSBiZWVuIHVzZQo+IGluIG1vcmUgcGxhY2VzIHNvIGZh cikuCj4gCgpQZXJzb25hbGx5LCB3ZSBkb24ndCBjYXJlIHdoYXQgbmFtZSB0byBiZSB1c2VkLiBJ IHVzZWQgU1ZNIHdoZW4gSSBzdGFydCB0aGlzCndvcmsuIEFuZCB0aGVuIEplYW4gc2FpZCBTVk0g aGFkIGJlZW4gdXNlZCBieSBBTUQgYXMgU2VjdXJlIFZpcnR1YWwgTWFjaGluZS4gU28KaGUgY2Fs bGVkIGl0IFNWQS4gQW5kIG5vdy4uLiB3aG8gc2hvdWxkIEkgZm9sbG93PyA6KQoKPiAKPiBDaGVl cnMsCj4gSsOpcsO0bWUKCi0tIAoJCQktS2VubmV0aChIaXNpbGljb24pCgo9PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PQrmnKzpgq7ku7blj4rlhbbpmYTku7blkKvmnInljY7kuLrlhazlj7jnmoTkv53l r4bkv6Hmga/vvIzku4XpmZDkuo7lj5HpgIHnu5nkuIrpnaLlnLDlnYDkuK3liJflh7rnmoTkuKrk urrmiJbnvqTnu4TjgILnpoEK5q2i5Lu75L2V5YW25LuW5Lq65Lul5Lu75L2V5b2i5byP5L2/55So 77yI5YyF5ous5L2G5LiN6ZmQ5LqO5YWo6YOo5oiW6YOo5YiG5Zyw5rOE6Zyy44CB5aSN5Yi244CB 5oiW5pWj5Y+R77yJ5pys6YKu5Lu25LitCueahOS/oeaBr+OAguWmguaenOaCqOmUmeaUtuS6huac rOmCruS7tu+8jOivt+aCqOeri+WNs+eUteivneaIlumCruS7tumAmuefpeWPkeS7tuS6uuW5tuWI oOmZpOacrOmCruS7tu+8gQpUaGlzIGUtbWFpbCBhbmQgaXRzIGF0dGFjaG1lbnRzIGNvbnRhaW4g Y29uZmlkZW50aWFsIGluZm9ybWF0aW9uIGZyb20gSFVBV0VJLAp3aGljaCBpcyBpbnRlbmRlZCBv bmx5IGZvciB0aGUgcGVyc29uIG9yIGVudGl0eSB3aG9zZSBhZGRyZXNzIGlzIGxpc3RlZCBhYm92 ZS4KQW55IHVzZSBvZiB0aGUgCmluZm9ybWF0aW9uIGNvbnRhaW5lZCBoZXJlaW4gaW4gYW55IHdh eSAoaW5jbHVkaW5nLCBidXQgbm90IGxpbWl0ZWQgdG8sIHRvdGFsIG9yCnBhcnRpYWwgZGlzY2xv c3VyZSwgcmVwcm9kdWN0aW9uLCBvciBkaXNzZW1pbmF0aW9uKSBieSBwZXJzb25zIG90aGVyIHRo YW4gdGhlCmludGVuZGVkIApyZWNpcGllbnQocykgaXMgcHJvaGliaXRlZC4gSWYgeW91IHJlY2Vp dmUgdGhpcyBlLW1haWwgaW4gZXJyb3IsIHBsZWFzZSBub3RpZnkKdGhlIHNlbmRlciBieSBwaG9u ZSBvciBlbWFpbCBpbW1lZGlhdGVseSBhbmQgZGVsZXRlIGl0IQoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11QGxp c3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9y Zy9tYWlsbWFuL2xpc3RpbmZvL2lvbW11 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22A2DC28CF6 for ; Fri, 3 Aug 2018 03:48:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C325321723 for ; Fri, 3 Aug 2018 03:48:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C325321723 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727246AbeHCFnK (ORCPT ); Fri, 3 Aug 2018 01:43:10 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:42190 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726012AbeHCFnK (ORCPT ); Fri, 3 Aug 2018 01:43:10 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 15A6A2B347DE4; Fri, 3 Aug 2018 11:48:47 +0800 (CST) Received: from localhost (10.67.212.75) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server (TLS) id 14.3.399.0; Fri, 3 Aug 2018 11:48:40 +0800 Date: Fri, 3 Aug 2018 11:47:21 +0800 From: Kenneth Lee To: Jerome Glisse CC: "Tian, Kevin" , Hao Fang , Alex Williamson , Herbert Xu , "kvm@vger.kernel.org" , Jonathan Corbet , Greg Kroah-Hartman , Zaibo Xu , "linux-doc@vger.kernel.org" , "Kumar, Sanjay K" , Kenneth Lee , "iommu@lists.linux-foundation.org" , "linux-kernel@vger.kernel.org" , "linuxarm@huawei.com" , "linux-crypto@vger.kernel.org" , Philippe Ombredanne , Thomas Gleixner , "David S . Miller" , "linux-accelerators@lists.ozlabs.org" Subject: Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive Message-ID: <20180803034721.GC91035@Turing-Arch-b> References: <20180801102221.5308-1-nek.in.cn@gmail.com> <20180801165644.GA3820@redhat.com> <20180802040557.GL160746@Turing-Arch-b> <20180802142243.GA3481@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180802142243.GA3481@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: [10.67.212.75] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Aug 02, 2018 at 10:22:43AM -0400, Jerome Glisse wrote: > Date: Thu, 2 Aug 2018 10:22:43 -0400 > From: Jerome Glisse > To: Kenneth Lee > CC: "Tian, Kevin" , Hao Fang , > Alex Williamson , Herbert Xu > , "kvm@vger.kernel.org" > , Jonathan Corbet , Greg > Kroah-Hartman , Zaibo Xu , > "linux-doc@vger.kernel.org" , "Kumar, Sanjay K" > , Kenneth Lee , > "iommu@lists.linux-foundation.org" , > "linux-kernel@vger.kernel.org" , > "linuxarm@huawei.com" , > "linux-crypto@vger.kernel.org" , Philippe > Ombredanne , Thomas Gleixner , > "David S . Miller" , > "linux-accelerators@lists.ozlabs.org" > > Subject: Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive > User-Agent: Mutt/1.10.0 (2018-05-17) > Message-ID: <20180802142243.GA3481@redhat.com> > > On Thu, Aug 02, 2018 at 12:05:57PM +0800, Kenneth Lee wrote: > > On Thu, Aug 02, 2018 at 02:33:12AM +0000, Tian, Kevin wrote: > > > Date: Thu, 2 Aug 2018 02:33:12 +0000 > > > > From: Jerome Glisse > > > > On Wed, Aug 01, 2018 at 06:22:14PM +0800, Kenneth Lee wrote: > > > > > From: Kenneth Lee > > > > > > > > > > WarpDrive is an accelerator framework to expose the hardware > > > > capabilities > > > > > directly to the user space. It makes use of the exist vfio and vfio-mdev > > > > > facilities. So the user application can send request and DMA to the > > > > > hardware without interaction with the kernel. This remove the latency > > > > > of syscall and context switch. > > > > > > > > > > The patchset contains documents for the detail. Please refer to it for > > > > more > > > > > information. > > > > > > > > > > This patchset is intended to be used with Jean Philippe Brucker's SVA > > > > > patch [1] (Which is also in RFC stage). But it is not mandatory. This > > > > > patchset is tested in the latest mainline kernel without the SVA patches. > > > > > So it support only one process for each accelerator. > > > > > > > > > > With SVA support, WarpDrive can support multi-process in the same > > > > > accelerator device. We tested it in our SoC integrated Accelerator (board > > > > > ID: D06, Chip ID: HIP08). A reference work tree can be found here: [2]. > > > > > > > > I have not fully inspected things nor do i know enough about > > > > this Hisilicon ZIP accelerator to ascertain, but from glimpsing > > > > at the code it seems that it is unsafe to use even with SVA due > > > > to the doorbell. There is a comment talking about safetyness > > > > in patch 7. > > > > > > > > Exposing thing to userspace is always enticing, but if it is > > > > a security risk then it should clearly say so and maybe a > > > > kernel boot flag should be necessary to allow such device to > > > > be use. > > > > > > > > But doorbell is just a notification. Except for DOS (to make hardware busy) it > > cannot actually take or change anything from the kernel space. And the DOS > > problem can be always taken as the problem that a group of processes share the > > same kernel entity. > > > > In the coming HIP09 hardware, the doorbell will come with a random number so > > only the process who allocated the queue can knock it correctly. > > When doorbell is ring the hardware start fetching commands from > the queue and execute them ? If so than a rogue process B might > ring the doorbell of process A which would starts execution of > random commands (ie whatever random memory value there is left > inside the command buffer memory, could be old commands i guess). > > If this is not how this doorbell works then, yes it can only do > a denial of service i guess. Issue i have with doorbell is that > i have seen 10 differents implementations in 10 differents hw > and each are different as to what ringing or value written to the > doorbell does. It is painfull to track what is what for each hw. > In our implementation, doorbell is simply a notification, just like an interrupt to the accelerator. The command is all about what's in the queue. I agree that there is no simple and standard way to track the shared IO space. But I think we have to trust the driver in some way. If the driver is malicious, even a simple ioctl can become an attack. > > > > > My more general question is do we want to grow VFIO to become > > > > a more generic device driver API. This patchset adds a command > > > > queue concept to it (i don't think it exist today but i have > > > > not follow VFIO closely). > > > > > > > > The thing is, VFIO is the only place to support DMA from user land. If we don't > > put it here, we have to create another similar facility to support the same. > > No it is not, network device, GPU, block device, ... they all do > support DMA. The point i am trying to make here is that even in Sorry, wait a minute, are we talking the same thing? I meant "DMA from user land", not "DMA from kernel driver". To do that we have to manipulate the IOMMU(Unit). I think it can only be done by default_domain or vfio domain. Or the user space have to directly access the IOMMU. > your mechanisms the userspace must have a specific userspace > drivers for each hardware and thus there are virtually no > differences between having this userspace driver open a device > file in vfio or somewhere else in the device filesystem. This is > just a different path. > The basic problem WarpDrive want to solve it to avoid syscall. This is important to accelerators. We have some data here: https://www.slideshare.net/linaroorg/progress-and-demonstration-of-wrapdrive-a-accelerator-framework-sfo17317 (see page 3) The performance is different on using kernel and user drivers. And we also believe the hardware interface can become standard after sometime. Some companies have started to do this (such ARM's Revere). But before that, we should have a software channel for it. > So this is why i do not see any benefit to having all drivers with > SVM (can we please use SVM and not SVA as SVM is what have been use > in more places so far). > Personally, we don't care what name to be used. I used SVM when I start this work. And then Jean said SVM had been used by AMD as Secure Virtual Machine. So he called it SVA. And now... who should I follow? :) > > Cheers, > Jérôme -- -Kenneth(Hisilicon) ================================================================================ 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on archive.lwn.net X-Spam-Level: X-Spam-Status: No, score=-6.0 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham autolearn_force=no version=3.4.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by archive.lwn.net (Postfix) with ESMTP id C47817E3B1 for ; Fri, 3 Aug 2018 03:48:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726353AbeHCFnK (ORCPT ); Fri, 3 Aug 2018 01:43:10 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:42190 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726012AbeHCFnK (ORCPT ); Fri, 3 Aug 2018 01:43:10 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 15A6A2B347DE4; Fri, 3 Aug 2018 11:48:47 +0800 (CST) Received: from localhost (10.67.212.75) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server (TLS) id 14.3.399.0; Fri, 3 Aug 2018 11:48:40 +0800 Date: Fri, 3 Aug 2018 11:47:21 +0800 From: Kenneth Lee To: Jerome Glisse CC: "Tian, Kevin" , Hao Fang , Alex Williamson , Herbert Xu , "kvm@vger.kernel.org" , Jonathan Corbet , Greg Kroah-Hartman , Zaibo Xu , "linux-doc@vger.kernel.org" , "Kumar, Sanjay K" , Kenneth Lee , "iommu@lists.linux-foundation.org" , "linux-kernel@vger.kernel.org" , "linuxarm@huawei.com" , "linux-crypto@vger.kernel.org" , Philippe Ombredanne , Thomas Gleixner , "David S . Miller" , "linux-accelerators@lists.ozlabs.org" Subject: Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive Message-ID: <20180803034721.GC91035@Turing-Arch-b> References: <20180801102221.5308-1-nek.in.cn@gmail.com> <20180801165644.GA3820@redhat.com> <20180802040557.GL160746@Turing-Arch-b> <20180802142243.GA3481@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180802142243.GA3481@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: [10.67.212.75] X-CFilter-Loop: Reflected Sender: linux-doc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On Thu, Aug 02, 2018 at 10:22:43AM -0400, Jerome Glisse wrote: > Date: Thu, 2 Aug 2018 10:22:43 -0400 > From: Jerome Glisse > To: Kenneth Lee > CC: "Tian, Kevin" , Hao Fang , > Alex Williamson , Herbert Xu > , "kvm@vger.kernel.org" > , Jonathan Corbet , Greg > Kroah-Hartman , Zaibo Xu , > "linux-doc@vger.kernel.org" , "Kumar, Sanjay K" > , Kenneth Lee , > "iommu@lists.linux-foundation.org" , > "linux-kernel@vger.kernel.org" , > "linuxarm@huawei.com" , > "linux-crypto@vger.kernel.org" , Philippe > Ombredanne , Thomas Gleixner , > "David S . Miller" , > "linux-accelerators@lists.ozlabs.org" > > Subject: Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive > User-Agent: Mutt/1.10.0 (2018-05-17) > Message-ID: <20180802142243.GA3481@redhat.com> > > On Thu, Aug 02, 2018 at 12:05:57PM +0800, Kenneth Lee wrote: > > On Thu, Aug 02, 2018 at 02:33:12AM +0000, Tian, Kevin wrote: > > > Date: Thu, 2 Aug 2018 02:33:12 +0000 > > > > From: Jerome Glisse > > > > On Wed, Aug 01, 2018 at 06:22:14PM +0800, Kenneth Lee wrote: > > > > > From: Kenneth Lee > > > > > > > > > > WarpDrive is an accelerator framework to expose the hardware > > > > capabilities > > > > > directly to the user space. It makes use of the exist vfio and vfio-mdev > > > > > facilities. So the user application can send request and DMA to the > > > > > hardware without interaction with the kernel. This remove the latency > > > > > of syscall and context switch. > > > > > > > > > > The patchset contains documents for the detail. Please refer to it for > > > > more > > > > > information. > > > > > > > > > > This patchset is intended to be used with Jean Philippe Brucker's SVA > > > > > patch [1] (Which is also in RFC stage). But it is not mandatory. This > > > > > patchset is tested in the latest mainline kernel without the SVA patches. > > > > > So it support only one process for each accelerator. > > > > > > > > > > With SVA support, WarpDrive can support multi-process in the same > > > > > accelerator device. We tested it in our SoC integrated Accelerator (board > > > > > ID: D06, Chip ID: HIP08). A reference work tree can be found here: [2]. > > > > > > > > I have not fully inspected things nor do i know enough about > > > > this Hisilicon ZIP accelerator to ascertain, but from glimpsing > > > > at the code it seems that it is unsafe to use even with SVA due > > > > to the doorbell. There is a comment talking about safetyness > > > > in patch 7. > > > > > > > > Exposing thing to userspace is always enticing, but if it is > > > > a security risk then it should clearly say so and maybe a > > > > kernel boot flag should be necessary to allow such device to > > > > be use. > > > > > > > > But doorbell is just a notification. Except for DOS (to make hardware busy) it > > cannot actually take or change anything from the kernel space. And the DOS > > problem can be always taken as the problem that a group of processes share the > > same kernel entity. > > > > In the coming HIP09 hardware, the doorbell will come with a random number so > > only the process who allocated the queue can knock it correctly. > > When doorbell is ring the hardware start fetching commands from > the queue and execute them ? If so than a rogue process B might > ring the doorbell of process A which would starts execution of > random commands (ie whatever random memory value there is left > inside the command buffer memory, could be old commands i guess). > > If this is not how this doorbell works then, yes it can only do > a denial of service i guess. Issue i have with doorbell is that > i have seen 10 differents implementations in 10 differents hw > and each are different as to what ringing or value written to the > doorbell does. It is painfull to track what is what for each hw. > In our implementation, doorbell is simply a notification, just like an interrupt to the accelerator. The command is all about what's in the queue. I agree that there is no simple and standard way to track the shared IO space. But I think we have to trust the driver in some way. If the driver is malicious, even a simple ioctl can become an attack. > > > > > My more general question is do we want to grow VFIO to become > > > > a more generic device driver API. This patchset adds a command > > > > queue concept to it (i don't think it exist today but i have > > > > not follow VFIO closely). > > > > > > > > The thing is, VFIO is the only place to support DMA from user land. If we don't > > put it here, we have to create another similar facility to support the same. > > No it is not, network device, GPU, block device, ... they all do > support DMA. The point i am trying to make here is that even in Sorry, wait a minute, are we talking the same thing? I meant "DMA from user land", not "DMA from kernel driver". To do that we have to manipulate the IOMMU(Unit). I think it can only be done by default_domain or vfio domain. Or the user space have to directly access the IOMMU. > your mechanisms the userspace must have a specific userspace > drivers for each hardware and thus there are virtually no > differences between having this userspace driver open a device > file in vfio or somewhere else in the device filesystem. This is > just a different path. > The basic problem WarpDrive want to solve it to avoid syscall. This is important to accelerators. We have some data here: https://www.slideshare.net/linaroorg/progress-and-demonstration-of-wrapdrive-a-accelerator-framework-sfo17317 (see page 3) The performance is different on using kernel and user drivers. And we also believe the hardware interface can become standard after sometime. Some companies have started to do this (such ARM's Revere). But before that, we should have a software channel for it. > So this is why i do not see any benefit to having all drivers with > SVM (can we please use SVM and not SVA as SVM is what have been use > in more places so far). > Personally, we don't care what name to be used. I used SVM when I start this work. And then Jean said SVM had been used by AMD as Secure Virtual Machine. So he called it SVA. And now... who should I follow? :) > > Cheers, > Jérôme -- -Kenneth(Hisilicon) ================================================================================ 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kenneth Lee Subject: Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive Date: Fri, 3 Aug 2018 11:47:21 +0800 Message-ID: <20180803034721.GC91035@Turing-Arch-b> References: <20180801102221.5308-1-nek.in.cn@gmail.com> <20180801165644.GA3820@redhat.com> <20180802040557.GL160746@Turing-Arch-b> <20180802142243.GA3481@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <20180802142243.GA3481-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Jerome Glisse Cc: "Tian, Kevin" , Herbert Xu , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Jonathan Corbet , Greg Kroah-Hartman , Zaibo Xu , "linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "Kumar, Sanjay K" , Hao Fang , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linuxarm-hv44wF8Li93QT0dZR+AlfA@public.gmane.org" , Alex Williamson , "linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Philippe Ombredanne , Thomas Gleixner , Kenneth Lee , "David S . Miller" , "linux-accelerators-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org" List-Id: iommu@lists.linux-foundation.org T24gVGh1LCBBdWcgMDIsIDIwMTggYXQgMTA6MjI6NDNBTSAtMDQwMCwgSmVyb21lIEdsaXNzZSB3 cm90ZToKPiBEYXRlOiBUaHUsIDIgQXVnIDIwMTggMTA6MjI6NDMgLTA0MDAKPiBGcm9tOiBKZXJv bWUgR2xpc3NlIDxqZ2xpc3NlQHJlZGhhdC5jb20+Cj4gVG86IEtlbm5ldGggTGVlIDxsaWd1b3po dUBoaXNpbGljb24uY29tPgo+IENDOiAiVGlhbiwgS2V2aW4iIDxrZXZpbi50aWFuQGludGVsLmNv bT4sIEhhbyBGYW5nIDxmYW5naGFvMTFAaHVhd2VpLmNvbT4sCj4gIEFsZXggV2lsbGlhbXNvbiA8 YWxleC53aWxsaWFtc29uQHJlZGhhdC5jb20+LCBIZXJiZXJ0IFh1Cj4gIDxoZXJiZXJ0QGdvbmRv ci5hcGFuYS5vcmcuYXU+LCAia3ZtQHZnZXIua2VybmVsLm9yZyIKPiAgPGt2bUB2Z2VyLmtlcm5l bC5vcmc+LCBKb25hdGhhbiBDb3JiZXQgPGNvcmJldEBsd24ubmV0PiwgR3JlZwo+ICBLcm9haC1I YXJ0bWFuIDxncmVna2hAbGludXhmb3VuZGF0aW9uLm9yZz4sIFphaWJvIFh1IDx4dXphaWJvQGh1 YXdlaS5jb20+LAo+ICAibGludXgtZG9jQHZnZXIua2VybmVsLm9yZyIgPGxpbnV4LWRvY0B2Z2Vy Lmtlcm5lbC5vcmc+LCAiS3VtYXIsIFNhbmpheSBLIgo+ICA8c2FuamF5Lmsua3VtYXJAaW50ZWwu Y29tPiwgS2VubmV0aCBMZWUgPG5lay5pbi5jbkBnbWFpbC5jb20+LAo+ICAiaW9tbXVAbGlzdHMu bGludXgtZm91bmRhdGlvbi5vcmciIDxpb21tdUBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZz4s Cj4gICJsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnIiA8bGludXgta2VybmVsQHZnZXIua2Vy bmVsLm9yZz4sCj4gICJsaW51eGFybUBodWF3ZWkuY29tIiA8bGludXhhcm1AaHVhd2VpLmNvbT4s Cj4gICJsaW51eC1jcnlwdG9Admdlci5rZXJuZWwub3JnIiA8bGludXgtY3J5cHRvQHZnZXIua2Vy bmVsLm9yZz4sIFBoaWxpcHBlCj4gIE9tYnJlZGFubmUgPHBvbWJyZWRhbm5lQG5leGIuY29tPiwg VGhvbWFzIEdsZWl4bmVyIDx0Z2x4QGxpbnV0cm9uaXguZGU+LAo+ICAiRGF2aWQgUyAuIE1pbGxl ciIgPGRhdmVtQGRhdmVtbG9mdC5uZXQ+LAo+ICAibGludXgtYWNjZWxlcmF0b3JzQGxpc3RzLm96 bGFicy5vcmciCj4gIDxsaW51eC1hY2NlbGVyYXRvcnNAbGlzdHMub3psYWJzLm9yZz4KPiBTdWJq ZWN0OiBSZTogW1JGQyBQQVRDSCAwLzddIEEgR2VuZXJhbCBBY2NlbGVyYXRvciBGcmFtZXdvcmss IFdhcnBEcml2ZQo+IFVzZXItQWdlbnQ6IE11dHQvMS4xMC4wICgyMDE4LTA1LTE3KQo+IE1lc3Nh Z2UtSUQ6IDwyMDE4MDgwMjE0MjI0My5HQTM0ODFAcmVkaGF0LmNvbT4KPiAKPiBPbiBUaHUsIEF1 ZyAwMiwgMjAxOCBhdCAxMjowNTo1N1BNICswODAwLCBLZW5uZXRoIExlZSB3cm90ZToKPiA+IE9u IFRodSwgQXVnIDAyLCAyMDE4IGF0IDAyOjMzOjEyQU0gKzAwMDAsIFRpYW4sIEtldmluIHdyb3Rl Ogo+ID4gPiBEYXRlOiBUaHUsIDIgQXVnIDIwMTggMDI6MzM6MTIgKzAwMDAKPiA+ID4gPiBGcm9t OiBKZXJvbWUgR2xpc3NlCj4gPiA+ID4gT24gV2VkLCBBdWcgMDEsIDIwMTggYXQgMDY6MjI6MTRQ TSArMDgwMCwgS2VubmV0aCBMZWUgd3JvdGU6Cj4gPiA+ID4gPiBGcm9tOiBLZW5uZXRoIExlZSA8 bGlndW96aHVAaGlzaWxpY29uLmNvbT4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBXYXJwRHJpdmUgaXMg YW4gYWNjZWxlcmF0b3IgZnJhbWV3b3JrIHRvIGV4cG9zZSB0aGUgaGFyZHdhcmUKPiA+ID4gPiBj YXBhYmlsaXRpZXMKPiA+ID4gPiA+IGRpcmVjdGx5IHRvIHRoZSB1c2VyIHNwYWNlLiBJdCBtYWtl cyB1c2Ugb2YgdGhlIGV4aXN0IHZmaW8gYW5kIHZmaW8tbWRldgo+ID4gPiA+ID4gZmFjaWxpdGll cy4gU28gdGhlIHVzZXIgYXBwbGljYXRpb24gY2FuIHNlbmQgcmVxdWVzdCBhbmQgRE1BIHRvIHRo ZQo+ID4gPiA+ID4gaGFyZHdhcmUgd2l0aG91dCBpbnRlcmFjdGlvbiB3aXRoIHRoZSBrZXJuZWwu IFRoaXMgcmVtb3ZlIHRoZSBsYXRlbmN5Cj4gPiA+ID4gPiBvZiBzeXNjYWxsIGFuZCBjb250ZXh0 IHN3aXRjaC4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBUaGUgcGF0Y2hzZXQgY29udGFpbnMgZG9jdW1l bnRzIGZvciB0aGUgZGV0YWlsLiBQbGVhc2UgcmVmZXIgdG8gaXQgZm9yCj4gPiA+ID4gbW9yZQo+ ID4gPiA+ID4gaW5mb3JtYXRpb24uCj4gPiA+ID4gPgo+ID4gPiA+ID4gVGhpcyBwYXRjaHNldCBp cyBpbnRlbmRlZCB0byBiZSB1c2VkIHdpdGggSmVhbiBQaGlsaXBwZSBCcnVja2VyJ3MgU1ZBCj4g PiA+ID4gPiBwYXRjaCBbMV0gKFdoaWNoIGlzIGFsc28gaW4gUkZDIHN0YWdlKS4gQnV0IGl0IGlz IG5vdCBtYW5kYXRvcnkuIFRoaXMKPiA+ID4gPiA+IHBhdGNoc2V0IGlzIHRlc3RlZCBpbiB0aGUg bGF0ZXN0IG1haW5saW5lIGtlcm5lbCB3aXRob3V0IHRoZSBTVkEgcGF0Y2hlcy4KPiA+ID4gPiA+ IFNvIGl0IHN1cHBvcnQgb25seSBvbmUgcHJvY2VzcyBmb3IgZWFjaCBhY2NlbGVyYXRvci4KPiA+ ID4gPiA+Cj4gPiA+ID4gPiBXaXRoIFNWQSBzdXBwb3J0LCBXYXJwRHJpdmUgY2FuIHN1cHBvcnQg bXVsdGktcHJvY2VzcyBpbiB0aGUgc2FtZQo+ID4gPiA+ID4gYWNjZWxlcmF0b3IgZGV2aWNlLiAg V2UgdGVzdGVkIGl0IGluIG91ciBTb0MgaW50ZWdyYXRlZCBBY2NlbGVyYXRvciAoYm9hcmQKPiA+ ID4gPiA+IElEOiBEMDYsIENoaXAgSUQ6IEhJUDA4KS4gQSByZWZlcmVuY2Ugd29yayB0cmVlIGNh biBiZSBmb3VuZCBoZXJlOiBbMl0uCj4gPiA+ID4gCj4gPiA+ID4gSSBoYXZlIG5vdCBmdWxseSBp bnNwZWN0ZWQgdGhpbmdzIG5vciBkbyBpIGtub3cgZW5vdWdoIGFib3V0Cj4gPiA+ID4gdGhpcyBI aXNpbGljb24gWklQIGFjY2VsZXJhdG9yIHRvIGFzY2VydGFpbiwgYnV0IGZyb20gZ2xpbXBzaW5n Cj4gPiA+ID4gYXQgdGhlIGNvZGUgaXQgc2VlbXMgdGhhdCBpdCBpcyB1bnNhZmUgdG8gdXNlIGV2 ZW4gd2l0aCBTVkEgZHVlCj4gPiA+ID4gdG8gdGhlIGRvb3JiZWxsLiBUaGVyZSBpcyBhIGNvbW1l bnQgdGFsa2luZyBhYm91dCBzYWZldHluZXNzCj4gPiA+ID4gaW4gcGF0Y2ggNy4KPiA+ID4gPiAK PiA+ID4gPiBFeHBvc2luZyB0aGluZyB0byB1c2Vyc3BhY2UgaXMgYWx3YXlzIGVudGljaW5nLCBi dXQgaWYgaXQgaXMKPiA+ID4gPiBhIHNlY3VyaXR5IHJpc2sgdGhlbiBpdCBzaG91bGQgY2xlYXJs eSBzYXkgc28gYW5kIG1heWJlIGEKPiA+ID4gPiBrZXJuZWwgYm9vdCBmbGFnIHNob3VsZCBiZSBu ZWNlc3NhcnkgdG8gYWxsb3cgc3VjaCBkZXZpY2UgdG8KPiA+ID4gPiBiZSB1c2UuCj4gPiA+ID4g Cj4gPiAKPiA+IEJ1dCBkb29yYmVsbCBpcyBqdXN0IGEgbm90aWZpY2F0aW9uLiBFeGNlcHQgZm9y IERPUyAodG8gbWFrZSBoYXJkd2FyZSBidXN5KSBpdAo+ID4gY2Fubm90IGFjdHVhbGx5IHRha2Ug b3IgY2hhbmdlIGFueXRoaW5nIGZyb20gdGhlIGtlcm5lbCBzcGFjZS4gQW5kIHRoZSBET1MKPiA+ IHByb2JsZW0gY2FuIGJlIGFsd2F5cyB0YWtlbiBhcyB0aGUgcHJvYmxlbSB0aGF0IGEgZ3JvdXAg b2YgcHJvY2Vzc2VzIHNoYXJlIHRoZQo+ID4gc2FtZSBrZXJuZWwgZW50aXR5Lgo+ID4gCj4gPiBJ biB0aGUgY29taW5nIEhJUDA5IGhhcmR3YXJlLCB0aGUgZG9vcmJlbGwgd2lsbCBjb21lIHdpdGgg YSByYW5kb20gbnVtYmVyIHNvCj4gPiBvbmx5IHRoZSBwcm9jZXNzIHdobyBhbGxvY2F0ZWQgdGhl IHF1ZXVlIGNhbiBrbm9jayBpdCBjb3JyZWN0bHkuCj4gCj4gV2hlbiBkb29yYmVsbCBpcyByaW5n IHRoZSBoYXJkd2FyZSBzdGFydCBmZXRjaGluZyBjb21tYW5kcyBmcm9tCj4gdGhlIHF1ZXVlIGFu ZCBleGVjdXRlIHRoZW0gPyBJZiBzbyB0aGFuIGEgcm9ndWUgcHJvY2VzcyBCIG1pZ2h0Cj4gcmlu ZyB0aGUgZG9vcmJlbGwgb2YgcHJvY2VzcyBBIHdoaWNoIHdvdWxkIHN0YXJ0cyBleGVjdXRpb24g b2YKPiByYW5kb20gY29tbWFuZHMgKGllIHdoYXRldmVyIHJhbmRvbSBtZW1vcnkgdmFsdWUgdGhl cmUgaXMgbGVmdAo+IGluc2lkZSB0aGUgY29tbWFuZCBidWZmZXIgbWVtb3J5LCBjb3VsZCBiZSBv bGQgY29tbWFuZHMgaSBndWVzcykuCj4gCj4gSWYgdGhpcyBpcyBub3QgaG93IHRoaXMgZG9vcmJl bGwgd29ya3MgdGhlbiwgeWVzIGl0IGNhbiBvbmx5IGRvCj4gYSBkZW5pYWwgb2Ygc2VydmljZSBp IGd1ZXNzLiBJc3N1ZSBpIGhhdmUgd2l0aCBkb29yYmVsbCBpcyB0aGF0Cj4gaSBoYXZlIHNlZW4g MTAgZGlmZmVyZW50cyBpbXBsZW1lbnRhdGlvbnMgaW4gMTAgZGlmZmVyZW50cyBodwo+IGFuZCBl YWNoIGFyZSBkaWZmZXJlbnQgYXMgdG8gd2hhdCByaW5naW5nIG9yIHZhbHVlIHdyaXR0ZW4gdG8g dGhlCj4gZG9vcmJlbGwgZG9lcy4gSXQgaXMgcGFpbmZ1bGwgdG8gdHJhY2sgd2hhdCBpcyB3aGF0 IGZvciBlYWNoIGh3Lgo+IAoKSW4gb3VyIGltcGxlbWVudGF0aW9uLCBkb29yYmVsbCBpcyBzaW1w bHkgYSBub3RpZmljYXRpb24sIGp1c3QgbGlrZSBhbiBpbnRlcnJ1cHQKdG8gdGhlIGFjY2VsZXJh dG9yLiBUaGUgY29tbWFuZCBpcyBhbGwgYWJvdXQgd2hhdCdzIGluIHRoZSBxdWV1ZS4KCkkgYWdy ZWUgdGhhdCB0aGVyZSBpcyBubyBzaW1wbGUgYW5kIHN0YW5kYXJkIHdheSB0byB0cmFjayB0aGUg c2hhcmVkIElPIHNwYWNlLgpCdXQgSSB0aGluayB3ZSBoYXZlIHRvIHRydXN0IHRoZSBkcml2ZXIg aW4gc29tZSB3YXkuIElmIHRoZSBkcml2ZXIgaXMgbWFsaWNpb3VzLApldmVuIGEgc2ltcGxlIGlv Y3RsIGNhbiBiZWNvbWUgYW4gYXR0YWNrLgoKPiAKPiA+ID4gPiBNeSBtb3JlIGdlbmVyYWwgcXVl c3Rpb24gaXMgZG8gd2Ugd2FudCB0byBncm93IFZGSU8gdG8gYmVjb21lCj4gPiA+ID4gYSBtb3Jl IGdlbmVyaWMgZGV2aWNlIGRyaXZlciBBUEkuIFRoaXMgcGF0Y2hzZXQgYWRkcyBhIGNvbW1hbmQK PiA+ID4gPiBxdWV1ZSBjb25jZXB0IHRvIGl0IChpIGRvbid0IHRoaW5rIGl0IGV4aXN0IHRvZGF5 IGJ1dCBpIGhhdmUKPiA+ID4gPiBub3QgZm9sbG93IFZGSU8gY2xvc2VseSkuCj4gPiA+ID4gCj4g PiAKPiA+IFRoZSB0aGluZyBpcywgVkZJTyBpcyB0aGUgb25seSBwbGFjZSB0byBzdXBwb3J0IERN QSBmcm9tIHVzZXIgbGFuZC4gSWYgd2UgZG9uJ3QKPiA+IHB1dCBpdCBoZXJlLCB3ZSBoYXZlIHRv IGNyZWF0ZSBhbm90aGVyIHNpbWlsYXIgZmFjaWxpdHkgdG8gc3VwcG9ydCB0aGUgc2FtZS4KPiAK PiBObyBpdCBpcyBub3QsIG5ldHdvcmsgZGV2aWNlLCBHUFUsIGJsb2NrIGRldmljZSwgLi4uIHRo ZXkgYWxsIGRvCj4gc3VwcG9ydCBETUEuIFRoZSBwb2ludCBpIGFtIHRyeWluZyB0byBtYWtlIGhl cmUgaXMgdGhhdCBldmVuIGluCgpTb3JyeSwgd2FpdCBhIG1pbnV0ZSwgYXJlIHdlIHRhbGtpbmcg dGhlIHNhbWUgdGhpbmc/IEkgbWVhbnQgIkRNQSBmcm9tIHVzZXIKbGFuZCIsIG5vdCAiRE1BIGZy b20ga2VybmVsIGRyaXZlciIuIFRvIGRvIHRoYXQgd2UgaGF2ZSB0byBtYW5pcHVsYXRlIHRoZQpJ T01NVShVbml0KS4gSSB0aGluayBpdCBjYW4gb25seSBiZSBkb25lIGJ5IGRlZmF1bHRfZG9tYWlu IG9yIHZmaW8gZG9tYWluLiBPcgp0aGUgdXNlciBzcGFjZSBoYXZlIHRvIGRpcmVjdGx5IGFjY2Vz cyB0aGUgSU9NTVUuCgo+IHlvdXIgbWVjaGFuaXNtcyB0aGUgdXNlcnNwYWNlIG11c3QgaGF2ZSBh IHNwZWNpZmljIHVzZXJzcGFjZQo+IGRyaXZlcnMgZm9yIGVhY2ggaGFyZHdhcmUgYW5kIHRodXMg dGhlcmUgYXJlIHZpcnR1YWxseSBubwo+IGRpZmZlcmVuY2VzIGJldHdlZW4gaGF2aW5nIHRoaXMg dXNlcnNwYWNlIGRyaXZlciBvcGVuIGEgZGV2aWNlCj4gZmlsZSBpbiB2ZmlvIG9yIHNvbWV3aGVy ZSBlbHNlIGluIHRoZSBkZXZpY2UgZmlsZXN5c3RlbS4gVGhpcyBpcwo+IGp1c3QgYSBkaWZmZXJl bnQgcGF0aC4KPiAKClRoZSBiYXNpYyBwcm9ibGVtIFdhcnBEcml2ZSB3YW50IHRvIHNvbHZlIGl0 IHRvIGF2b2lkIHN5c2NhbGwuIFRoaXMgaXMgaW1wb3J0YW50CnRvIGFjY2VsZXJhdG9ycy4gV2Ug aGF2ZSBzb21lIGRhdGEgaGVyZToKaHR0cHM6Ly93d3cuc2xpZGVzaGFyZS5uZXQvbGluYXJvb3Jn L3Byb2dyZXNzLWFuZC1kZW1vbnN0cmF0aW9uLW9mLXdyYXBkcml2ZS1hLWFjY2VsZXJhdG9yLWZy YW1ld29yay1zZm8xNzMxNwoKKHNlZSBwYWdlIDMpCgpUaGUgcGVyZm9ybWFuY2UgaXMgZGlmZmVy ZW50IG9uIHVzaW5nIGtlcm5lbCBhbmQgdXNlciBkcml2ZXJzLgoKQW5kIHdlIGFsc28gYmVsaWV2 ZSB0aGUgaGFyZHdhcmUgaW50ZXJmYWNlIGNhbiBiZWNvbWUgc3RhbmRhcmQgYWZ0ZXIgc29tZXRp bWUuClNvbWUgY29tcGFuaWVzIGhhdmUgc3RhcnRlZCB0byBkbyB0aGlzIChzdWNoIEFSTSdzIFJl dmVyZSkuIEJ1dCBiZWZvcmUgdGhhdCwgd2UKc2hvdWxkIGhhdmUgYSBzb2Z0d2FyZSBjaGFubmVs IGZvciBpdC4KCj4gU28gdGhpcyBpcyB3aHkgaSBkbyBub3Qgc2VlIGFueSBiZW5lZml0IHRvIGhh dmluZyBhbGwgZHJpdmVycyB3aXRoCj4gU1ZNIChjYW4gd2UgcGxlYXNlIHVzZSBTVk0gYW5kIG5v dCBTVkEgYXMgU1ZNIGlzIHdoYXQgaGF2ZSBiZWVuIHVzZQo+IGluIG1vcmUgcGxhY2VzIHNvIGZh cikuCj4gCgpQZXJzb25hbGx5LCB3ZSBkb24ndCBjYXJlIHdoYXQgbmFtZSB0byBiZSB1c2VkLiBJ IHVzZWQgU1ZNIHdoZW4gSSBzdGFydCB0aGlzCndvcmsuIEFuZCB0aGVuIEplYW4gc2FpZCBTVk0g aGFkIGJlZW4gdXNlZCBieSBBTUQgYXMgU2VjdXJlIFZpcnR1YWwgTWFjaGluZS4gU28KaGUgY2Fs bGVkIGl0IFNWQS4gQW5kIG5vdy4uLiB3aG8gc2hvdWxkIEkgZm9sbG93PyA6KQoKPiAKPiBDaGVl cnMsCj4gSsOpcsO0bWUKCi0tIAoJCQktS2VubmV0aChIaXNpbGljb24pCgo9PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PQrmnKzpgq7ku7blj4rlhbbpmYTku7blkKvmnInljY7kuLrlhazlj7jnmoTkv53l r4bkv6Hmga/vvIzku4XpmZDkuo7lj5HpgIHnu5nkuIrpnaLlnLDlnYDkuK3liJflh7rnmoTkuKrk urrmiJbnvqTnu4TjgILnpoEK5q2i5Lu75L2V5YW25LuW5Lq65Lul5Lu75L2V5b2i5byP5L2/55So 77yI5YyF5ous5L2G5LiN6ZmQ5LqO5YWo6YOo5oiW6YOo5YiG5Zyw5rOE6Zyy44CB5aSN5Yi244CB 5oiW5pWj5Y+R77yJ5pys6YKu5Lu25LitCueahOS/oeaBr+OAguWmguaenOaCqOmUmeaUtuS6huac rOmCruS7tu+8jOivt+aCqOeri+WNs+eUteivneaIlumCruS7tumAmuefpeWPkeS7tuS6uuW5tuWI oOmZpOacrOmCruS7tu+8gQpUaGlzIGUtbWFpbCBhbmQgaXRzIGF0dGFjaG1lbnRzIGNvbnRhaW4g Y29uZmlkZW50aWFsIGluZm9ybWF0aW9uIGZyb20gSFVBV0VJLAp3aGljaCBpcyBpbnRlbmRlZCBv bmx5IGZvciB0aGUgcGVyc29uIG9yIGVudGl0eSB3aG9zZSBhZGRyZXNzIGlzIGxpc3RlZCBhYm92 ZS4KQW55IHVzZSBvZiB0aGUgCmluZm9ybWF0aW9uIGNvbnRhaW5lZCBoZXJlaW4gaW4gYW55IHdh eSAoaW5jbHVkaW5nLCBidXQgbm90IGxpbWl0ZWQgdG8sIHRvdGFsIG9yCnBhcnRpYWwgZGlzY2xv c3VyZSwgcmVwcm9kdWN0aW9uLCBvciBkaXNzZW1pbmF0aW9uKSBieSBwZXJzb25zIG90aGVyIHRo YW4gdGhlCmludGVuZGVkIApyZWNpcGllbnQocykgaXMgcHJvaGliaXRlZC4gSWYgeW91IHJlY2Vp dmUgdGhpcyBlLW1haWwgaW4gZXJyb3IsIHBsZWFzZSBub3RpZnkKdGhlIHNlbmRlciBieSBwaG9u ZSBvciBlbWFpbCBpbW1lZGlhdGVseSBhbmQgZGVsZXRlIGl0IQoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11QGxp c3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9y Zy9tYWlsbWFuL2xpc3RpbmZvL2lvbW11