From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM05-BY2-obe.outbound.protection.outlook.com (mail-eopbgr710072.outbound.protection.outlook.com [40.107.71.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2C02C21106F31 for ; Fri, 31 Aug 2018 01:04:48 -0700 (PDT) Subject: Re: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> From: =?UTF-8?Q?Christian_K=c3=b6nig?= Message-ID: <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> Date: Fri, 31 Aug 2018 10:04:29 +0200 MIME-Version: 1.0 In-Reply-To: <20180830185352.3369-2-logang@deltatee.com> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Logan Gunthorpe , linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org Cc: Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-ID: QW0gMzAuMDguMjAxOCB1bSAyMDo1MyBzY2hyaWViIExvZ2FuIEd1bnRob3JwZToKPiBTb21lIFBD SSBkZXZpY2VzIG1heSBoYXZlIG1lbW9yeSBtYXBwZWQgaW4gYSBCQVIgc3BhY2UgdGhhdCdzCj4g aW50ZW5kZWQgZm9yIHVzZSBpbiBwZWVyLXRvLXBlZXIgdHJhbnNhY3Rpb25zLiBJbiBvcmRlciB0 byBlbmFibGUKPiBzdWNoIHRyYW5zYWN0aW9ucyB0aGUgbWVtb3J5IG11c3QgYmUgcmVnaXN0ZXJl ZCB3aXRoIFpPTkVfREVWSUNFIHBhZ2VzCj4gc28gaXQgY2FuIGJlIHVzZWQgYnkgRE1BIGludGVy ZmFjZXMgaW4gZXhpc3RpbmcgZHJpdmVycy4KCldlIHdhbnQgdG8gdXNlIHRoYXQgZmVhdHVyZSB3 aXRob3V0IFpPTkVfREVWSUNFIHBhZ2VzIGZvciBETUEtYnVmIGFzIHdlbGwuCgpIb3cgaGFyZCB3 b3VsZCBpdCBiZSB0byBzZXBhcmF0ZSBlbmFibGluZyBQMlAgZGV0ZWN0aW9uIChlLmcuIGRpc3Rh bmNlIApiZXR3ZWVuIHR3byBkZXZpY2VzKSBmcm9tIHRoaXM/CgpSZWdhcmRzLApDaHJpc3RpYW4u Cgo+Cj4gQWRkIGFuIGludGVyZmFjZSBmb3Igb3RoZXIgc3Vic3lzdGVtcyB0byBmaW5kIGFuZCBh bGxvY2F0ZSBjaHVua3Mgb2YgUDJQCj4gbWVtb3J5IGFzIG5lY2Vzc2FyeSB0byBmYWNpbGl0YXRl IHRyYW5zZmVycyBiZXR3ZWVuIHR3byBQQ0kgcGVlcnM6Cj4KPiBpbnQgcGNpX3AycGRtYV9hZGRf Y2xpZW50KCk7Cj4gc3RydWN0IHBjaV9kZXYgKnBjaV9wMnBtZW1fZmluZCgpOwo+IHZvaWQgKnBj aV9hbGxvY19wMnBtZW0oKTsKPgo+IFRoZSBuZXcgaW50ZXJmYWNlIHJlcXVpcmVzIGEgZHJpdmVy IHRvIGNvbGxlY3QgYSBsaXN0IG9mIGNsaWVudCBkZXZpY2VzCj4gaW52b2x2ZWQgaW4gdGhlIHRy YW5zYWN0aW9uIHdpdGggdGhlIHBjaV9wMnBtZW1fYWRkX2NsaWVudCooKSBmdW5jdGlvbnMKPiB0 aGVuIGNhbGwgcGNpX3AycG1lbV9maW5kKCkgdG8gb2J0YWluIGFueSBzdWl0YWJsZSBQMlAgbWVt b3J5LiBPbmNlCj4gdGhpcyBpcyBkb25lIHRoZSBsaXN0IGlzIGJvdW5kIHRvIHRoZSBtZW1vcnkg YW5kIHRoZSBjYWxsaW5nIGRyaXZlciBpcwo+IGZyZWUgdG8gYWRkIGFuZCByZW1vdmUgY2xpZW50 cyBhcyBuZWNlc3NhcnkgKGFkZGluZyBpbmNvbXBhdGlibGUgY2xpZW50cwo+IHdpbGwgZmFpbCku IFdpdGggYSBzdWl0YWJsZSBwMnBtZW0gZGV2aWNlLCBtZW1vcnkgY2FuIHRoZW4gYmUKPiBhbGxv Y2F0ZWQgd2l0aCBwY2lfYWxsb2NfcDJwbWVtKCkgZm9yIHVzZSBpbiBETUEgdHJhbnNhY3Rpb25z Lgo+Cj4gRGVwZW5kaW5nIG9uIGhhcmR3YXJlLCB1c2luZyBwZWVyLXRvLXBlZXIgbWVtb3J5IG1h eSByZWR1Y2UgdGhlIGJhbmR3aWR0aAo+IG9mIHRoZSB0cmFuc2ZlciBidXQgY2FuIHNpZ25pZmlj YW50bHkgcmVkdWNlIHByZXNzdXJlIG9uIHN5c3RlbSBtZW1vcnkuCj4gVGhpcyBtYXkgYmUgZGVz aXJhYmxlIGluIG1hbnkgY2FzZXM6IGZvciBleGFtcGxlIGEgc3lzdGVtIGNvdWxkIGJlIGRlc2ln bmVkCj4gd2l0aCBhIHNtYWxsIENQVSBjb25uZWN0ZWQgdG8gYSBQQ0llIHN3aXRjaCBieSBhIHNt YWxsIG51bWJlciBvZiBsYW5lcwo+IHdoaWNoIHdvdWxkIG1heGltaXplIHRoZSBudW1iZXIgb2Yg bGFuZXMgYXZhaWxhYmxlIHRvIGNvbm5lY3QgdG8gTlZNZQo+IGRldmljZXMuCj4KPiBUaGUgY29k ZSBpcyBkZXNpZ25lZCB0byBvbmx5IHV0aWxpemUgdGhlIHAycG1lbSBkZXZpY2UgaWYgYWxsIHRo ZSBkZXZpY2VzCj4gaW52b2x2ZWQgaW4gYSB0cmFuc2ZlciBhcmUgYmVoaW5kIHRoZSBzYW1lIFBD SSBicmlkZ2UuIFRoaXMgaXMgYmVjYXVzZSB3ZQo+IGhhdmUgbm8gd2F5IG9mIGtub3dpbmcgd2hl dGhlciBwZWVyLXRvLXBlZXIgcm91dGluZyBiZXR3ZWVuIFBDSWUgUm9vdCBQb3J0cwo+IGlzIHN1 cHBvcnRlZCAoUENJZSByNC4wLCBzZWMgMS4zLjEpLiBBZGRpdGlvbmFsbHksIHRoZSBiZW5lZml0 cyBvZiBQMlAKPiB0cmFuc2ZlcnMgdGhhdCBnbyB0aHJvdWdoIHRoZSBSQyBpcyBsaW1pdGVkIHRv IG9ubHkgcmVkdWNpbmcgRFJBTSB1c2FnZQo+IGFuZCwgaW4gc29tZSBjYXNlcywgY29kaW5nIGNv bnZlbmllbmNlLiBUaGUgUENJLVNJRyBtYXkgYmUgZXhwbG9yaW5nCj4gYWRkaW5nIGEgbmV3IGNh cGFiaWxpdHkgYml0IHRvIGFkdmVydGlzZSB3aGV0aGVyIHRoaXMgaXMgcG9zc2libGUgZm9yCj4g ZnV0dXJlIGhhcmR3YXJlLgo+Cj4gVGhpcyBjb21taXQgaW5jbHVkZXMgc2lnbmlmaWNhbnQgcmV3 b3JrIGFuZCBmZWVkYmFjayBmcm9tIENocmlzdG9waAo+IEhlbGx3aWcuCj4KPiBTaWduZWQtb2Zm LWJ5OiBDaHJpc3RvcGggSGVsbHdpZyA8aGNoQGxzdC5kZT4KPiBTaWduZWQtb2ZmLWJ5OiBMb2dh biBHdW50aG9ycGUgPGxvZ2FuZ0BkZWx0YXRlZS5jb20+Cj4gLS0tCj4gICBkcml2ZXJzL3BjaS9L Y29uZmlnICAgICAgICB8ICAxNyArCj4gICBkcml2ZXJzL3BjaS9NYWtlZmlsZSAgICAgICB8ICAg MSArCj4gICBkcml2ZXJzL3BjaS9wMnBkbWEuYyAgICAgICB8IDc2MSArKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysKPiAgIGluY2x1ZGUvbGludXgvbWVtcmVtYXAu aCAgIHwgICA1ICsKPiAgIGluY2x1ZGUvbGludXgvbW0uaCAgICAgICAgIHwgIDE4ICsrCj4gICBp bmNsdWRlL2xpbnV4L3BjaS1wMnBkbWEuaCB8IDEwMiArKysrKysKPiAgIGluY2x1ZGUvbGludXgv cGNpLmggICAgICAgIHwgICA0ICsKPiAgIDcgZmlsZXMgY2hhbmdlZCwgOTA4IGluc2VydGlvbnMo KykKPiAgIGNyZWF0ZSBtb2RlIDEwMDY0NCBkcml2ZXJzL3BjaS9wMnBkbWEuYwo+ICAgY3JlYXRl IG1vZGUgMTAwNjQ0IGluY2x1ZGUvbGludXgvcGNpLXAycGRtYS5oCj4KPiBkaWZmIC0tZ2l0IGEv ZHJpdmVycy9wY2kvS2NvbmZpZyBiL2RyaXZlcnMvcGNpL0tjb25maWcKPiBpbmRleCA1NmZmOGY2 ZDMxZmMuLmRlYjY4YmU0ZmRhYyAxMDA2NDQKPiAtLS0gYS9kcml2ZXJzL3BjaS9LY29uZmlnCj4g KysrIGIvZHJpdmVycy9wY2kvS2NvbmZpZwo+IEBAIC0xMzIsNiArMTMyLDIzIEBAIGNvbmZpZyBQ Q0lfUEFTSUQKPiAgIAo+ICAgCSAgSWYgdW5zdXJlLCBzYXkgTi4KPiAgIAo+ICtjb25maWcgUENJ X1AyUERNQQo+ICsJYm9vbCAiUENJIHBlZXItdG8tcGVlciB0cmFuc2ZlciBzdXBwb3J0Igo+ICsJ ZGVwZW5kcyBvbiBQQ0kgJiYgWk9ORV9ERVZJQ0UKPiArCXNlbGVjdCBHRU5FUklDX0FMTE9DQVRP Ugo+ICsJaGVscAo+ICsJICBFbmFibGXRlSBkcml2ZXJzIHRvIGRvIFBDSSBwZWVyLXRvLXBlZXIg dHJhbnNhY3Rpb25zIHRvIGFuZCBmcm9tCj4gKwkgIEJBUnMgdGhhdCBhcmUgZXhwb3NlZCBpbiBv dGhlciBkZXZpY2VzIHRoYXQgYXJlIHRoZSBwYXJ0IG9mCj4gKwkgIHRoZSBoaWVyYXJjaHkgd2hl cmUgcGVlci10by1wZWVyIERNQSBpcyBndWFyYW50ZWVkIGJ5IHRoZSBQQ0kKPiArCSAgc3BlY2lm aWNhdGlvbiB0byB3b3JrIChpZS4gYW55dGhpbmcgYmVsb3cgYSBzaW5nbGUgUENJIGJyaWRnZSku Cj4gKwo+ICsJICBNYW55IFBDSWUgcm9vdCBjb21wbGV4ZXMgZG8gbm90IHN1cHBvcnQgUDJQIHRy YW5zYWN0aW9ucyBhbmQKPiArCSAgaXQncyBoYXJkIHRvIHRlbGwgd2hpY2ggc3VwcG9ydCBpdCBh dCBhbGwsIHNvIGF0IHRoaXMgdGltZSwKPiArCSAgUDJQIERNQSB0cmFuc2F0aW9ucyBtdXN0IGJl IGJldHdlZW4gZGV2aWNlcyBiZWhpbmQgdGhlIHNhbWUgcm9vdAo+ICsJICBwb3J0Lgo+ICsKPiAr CSAgSWYgdW5zdXJlLCBzYXkgTi4KPiArCj4gICBjb25maWcgUENJX0xBQkVMCj4gICAJZGVmX2Jv b2wgeSBpZiAoRE1JIHx8IEFDUEkpCj4gICAJZGVwZW5kcyBvbiBQQ0kKPiBkaWZmIC0tZ2l0IGEv ZHJpdmVycy9wY2kvTWFrZWZpbGUgYi9kcml2ZXJzL3BjaS9NYWtlZmlsZQo+IGluZGV4IDFiMmNm ZTUxZThkNy4uODVmNGE3MDNiMmJlIDEwMDY0NAo+IC0tLSBhL2RyaXZlcnMvcGNpL01ha2VmaWxl Cj4gKysrIGIvZHJpdmVycy9wY2kvTWFrZWZpbGUKPiBAQCAtMjYsNiArMjYsNyBAQCBvYmotJChD T05GSUdfUENJX1NZU0NBTEwpCSs9IHN5c2NhbGwubwo+ICAgb2JqLSQoQ09ORklHX1BDSV9TVFVC KQkJKz0gcGNpLXN0dWIubwo+ICAgb2JqLSQoQ09ORklHX1BDSV9QRl9TVFVCKQkrPSBwY2ktcGYt c3R1Yi5vCj4gICBvYmotJChDT05GSUdfUENJX0VDQU0pCQkrPSBlY2FtLm8KPiArb2JqLSQoQ09O RklHX1BDSV9QMlBETUEpCSs9IHAycGRtYS5vCj4gICBvYmotJChDT05GSUdfWEVOX1BDSURFVl9G Uk9OVEVORCkgKz0geGVuLXBjaWZyb250Lm8KPiAgIAo+ICAgIyBFbmRwb2ludCBsaWJyYXJ5IG11 c3QgYmUgaW5pdGlhbGl6ZWQgYmVmb3JlIGl0cyB1c2Vycwo+IGRpZmYgLS1naXQgYS9kcml2ZXJz L3BjaS9wMnBkbWEuYyBiL2RyaXZlcnMvcGNpL3AycGRtYS5jCj4gbmV3IGZpbGUgbW9kZSAxMDA2 NDQKPiBpbmRleCAwMDAwMDAwMDAwMDAuLjg4YWFlYzUzNTFjZAo+IC0tLSAvZGV2L251bGwKPiAr KysgYi9kcml2ZXJzL3BjaS9wMnBkbWEuYwo+IEBAIC0wLDAgKzEsNzYxIEBACj4gKy8vIFNQRFgt TGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4wCj4gKy8qCj4gKyAqIFBDSSBQZWVyIDIgUGVlciBE TUEgc3VwcG9ydC4KPiArICoKPiArICogQ29weXJpZ2h0IChjKSAyMDE2LTIwMTgsIExvZ2FuIEd1 bnRob3JwZQo+ICsgKiBDb3B5cmlnaHQgKGMpIDIwMTYtMjAxNywgTWljcm9zZW1pIENvcnBvcmF0 aW9uCj4gKyAqIENvcHlyaWdodCAoYykgMjAxNywgQ2hyaXN0b3BoIEhlbGx3aWcKPiArICogQ29w eXJpZ2h0IChjKSAyMDE4LCBFaWRldGljb20gSW5jLgo+ICsgKi8KPiArCj4gKyNkZWZpbmUgcHJf Zm10KGZtdCkgInBjaS1wMnBkbWE6ICIgZm10Cj4gKyNpbmNsdWRlIDxsaW51eC9wY2ktcDJwZG1h Lmg+Cj4gKyNpbmNsdWRlIDxsaW51eC9tb2R1bGUuaD4KPiArI2luY2x1ZGUgPGxpbnV4L3NsYWIu aD4KPiArI2luY2x1ZGUgPGxpbnV4L2dlbmFsbG9jLmg+Cj4gKyNpbmNsdWRlIDxsaW51eC9tZW1y ZW1hcC5oPgo+ICsjaW5jbHVkZSA8bGludXgvcGVyY3B1LXJlZmNvdW50Lmg+Cj4gKyNpbmNsdWRl IDxsaW51eC9yYW5kb20uaD4KPiArI2luY2x1ZGUgPGxpbnV4L3NlcV9idWYuaD4KPiArCj4gK3N0 cnVjdCBwY2lfcDJwZG1hIHsKPiArCXN0cnVjdCBwZXJjcHVfcmVmIGRldm1hcF9yZWY7Cj4gKwlz dHJ1Y3QgY29tcGxldGlvbiBkZXZtYXBfcmVmX2RvbmU7Cj4gKwlzdHJ1Y3QgZ2VuX3Bvb2wgKnBv b2w7Cj4gKwlib29sIHAycG1lbV9wdWJsaXNoZWQ7Cj4gK307Cj4gKwo+ICtzdGF0aWMgdm9pZCBw Y2lfcDJwZG1hX3BlcmNwdV9yZWxlYXNlKHN0cnVjdCBwZXJjcHVfcmVmICpyZWYpCj4gK3sKPiAr CXN0cnVjdCBwY2lfcDJwZG1hICpwMnAgPQo+ICsJCWNvbnRhaW5lcl9vZihyZWYsIHN0cnVjdCBw Y2lfcDJwZG1hLCBkZXZtYXBfcmVmKTsKPiArCj4gKwljb21wbGV0ZV9hbGwoJnAycC0+ZGV2bWFw X3JlZl9kb25lKTsKPiArfQo+ICsKPiArc3RhdGljIHZvaWQgcGNpX3AycGRtYV9wZXJjcHVfa2ls bCh2b2lkICpkYXRhKQo+ICt7Cj4gKwlzdHJ1Y3QgcGVyY3B1X3JlZiAqcmVmID0gZGF0YTsKPiAr Cj4gKwlpZiAocGVyY3B1X3JlZl9pc19keWluZyhyZWYpKQo+ICsJCXJldHVybjsKPiArCj4gKwlw ZXJjcHVfcmVmX2tpbGwocmVmKTsKPiArfQo+ICsKPiArc3RhdGljIHZvaWQgcGNpX3AycGRtYV9y ZWxlYXNlKHZvaWQgKmRhdGEpCj4gK3sKPiArCXN0cnVjdCBwY2lfZGV2ICpwZGV2ID0gZGF0YTsK PiArCj4gKwlpZiAoIXBkZXYtPnAycGRtYSkKPiArCQlyZXR1cm47Cj4gKwo+ICsJd2FpdF9mb3Jf Y29tcGxldGlvbigmcGRldi0+cDJwZG1hLT5kZXZtYXBfcmVmX2RvbmUpOwo+ICsJcGVyY3B1X3Jl Zl9leGl0KCZwZGV2LT5wMnBkbWEtPmRldm1hcF9yZWYpOwo+ICsKPiArCWdlbl9wb29sX2Rlc3Ry b3kocGRldi0+cDJwZG1hLT5wb29sKTsKPiArCXBkZXYtPnAycGRtYSA9IE5VTEw7Cj4gK30KPiAr Cj4gK3N0YXRpYyBpbnQgcGNpX3AycGRtYV9zZXR1cChzdHJ1Y3QgcGNpX2RldiAqcGRldikKPiAr ewo+ICsJaW50IGVycm9yID0gLUVOT01FTTsKPiArCXN0cnVjdCBwY2lfcDJwZG1hICpwMnA7Cj4g Kwo+ICsJcDJwID0gZGV2bV9remFsbG9jKCZwZGV2LT5kZXYsIHNpemVvZigqcDJwKSwgR0ZQX0tF Uk5FTCk7Cj4gKwlpZiAoIXAycCkKPiArCQlyZXR1cm4gLUVOT01FTTsKPiArCj4gKwlwMnAtPnBv b2wgPSBnZW5fcG9vbF9jcmVhdGUoUEFHRV9TSElGVCwgZGV2X3RvX25vZGUoJnBkZXYtPmRldikp Owo+ICsJaWYgKCFwMnAtPnBvb2wpCj4gKwkJZ290byBvdXQ7Cj4gKwo+ICsJaW5pdF9jb21wbGV0 aW9uKCZwMnAtPmRldm1hcF9yZWZfZG9uZSk7Cj4gKwllcnJvciA9IHBlcmNwdV9yZWZfaW5pdCgm cDJwLT5kZXZtYXBfcmVmLAo+ICsJCQlwY2lfcDJwZG1hX3BlcmNwdV9yZWxlYXNlLCAwLCBHRlBf S0VSTkVMKTsKPiArCWlmIChlcnJvcikKPiArCQlnb3RvIG91dF9wb29sX2Rlc3Ryb3k7Cj4gKwo+ ICsJcGVyY3B1X3JlZl9zd2l0Y2hfdG9fYXRvbWljX3N5bmMoJnAycC0+ZGV2bWFwX3JlZik7Cj4g Kwo+ICsJZXJyb3IgPSBkZXZtX2FkZF9hY3Rpb25fb3JfcmVzZXQoJnBkZXYtPmRldiwgcGNpX3Ay cGRtYV9yZWxlYXNlLCBwZGV2KTsKPiArCWlmIChlcnJvcikKPiArCQlnb3RvIG91dF9wb29sX2Rl c3Ryb3k7Cj4gKwo+ICsJcGRldi0+cDJwZG1hID0gcDJwOwo+ICsKPiArCXJldHVybiAwOwo+ICsK PiArb3V0X3Bvb2xfZGVzdHJveToKPiArCWdlbl9wb29sX2Rlc3Ryb3kocDJwLT5wb29sKTsKPiAr b3V0Ogo+ICsJZGV2bV9rZnJlZSgmcGRldi0+ZGV2LCBwMnApOwo+ICsJcmV0dXJuIGVycm9yOwo+ ICt9Cj4gKwo+ICsvKioKPiArICogcGNpX3AycGRtYV9hZGRfcmVzb3VyY2UgLSBhZGQgbWVtb3J5 IGZvciB1c2UgYXMgcDJwIG1lbW9yeQo+ICsgKiBAcGRldjogdGhlIGRldmljZSB0byBhZGQgdGhl IG1lbW9yeSB0bwo+ICsgKiBAYmFyOiBQQ0kgQkFSIHRvIGFkZAo+ICsgKiBAc2l6ZTogc2l6ZSBv ZiB0aGUgbWVtb3J5IHRvIGFkZCwgbWF5IGJlIHplcm8gdG8gdXNlIHRoZSB3aG9sZSBCQVIKPiAr ICogQG9mZnNldDogb2Zmc2V0IGludG8gdGhlIFBDSSBCQVIKPiArICoKPiArICogVGhlIG1lbW9y eSB3aWxsIGJlIGdpdmVuIFpPTkVfREVWSUNFIHN0cnVjdCBwYWdlcyBzbyB0aGF0IGl0IG1heQo+ ICsgKiBiZSB1c2VkIHdpdGggYW55IERNQSByZXF1ZXN0Lgo+ICsgKi8KPiAraW50IHBjaV9wMnBk bWFfYWRkX3Jlc291cmNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBpbnQgYmFyLCBzaXplX3Qgc2l6 ZSwKPiArCQkJICAgIHU2NCBvZmZzZXQpCj4gK3sKPiArCXN0cnVjdCBkZXZfcGFnZW1hcCAqcGdt YXA7Cj4gKwl2b2lkICphZGRyOwo+ICsJaW50IGVycm9yOwo+ICsKPiArCWlmICghKHBjaV9yZXNv dXJjZV9mbGFncyhwZGV2LCBiYXIpICYgSU9SRVNPVVJDRV9NRU0pKQo+ICsJCXJldHVybiAtRUlO VkFMOwo+ICsKPiArCWlmIChvZmZzZXQgPj0gcGNpX3Jlc291cmNlX2xlbihwZGV2LCBiYXIpKQo+ ICsJCXJldHVybiAtRUlOVkFMOwo+ICsKPiArCWlmICghc2l6ZSkKPiArCQlzaXplID0gcGNpX3Jl c291cmNlX2xlbihwZGV2LCBiYXIpIC0gb2Zmc2V0Owo+ICsKPiArCWlmIChzaXplICsgb2Zmc2V0 ID4gcGNpX3Jlc291cmNlX2xlbihwZGV2LCBiYXIpKQo+ICsJCXJldHVybiAtRUlOVkFMOwo+ICsK PiArCWlmICghcGRldi0+cDJwZG1hKSB7Cj4gKwkJZXJyb3IgPSBwY2lfcDJwZG1hX3NldHVwKHBk ZXYpOwo+ICsJCWlmIChlcnJvcikKPiArCQkJcmV0dXJuIGVycm9yOwo+ICsJfQo+ICsKPiArCXBn bWFwID0gZGV2bV9remFsbG9jKCZwZGV2LT5kZXYsIHNpemVvZigqcGdtYXApLCBHRlBfS0VSTkVM KTsKPiArCWlmICghcGdtYXApCj4gKwkJcmV0dXJuIC1FTk9NRU07Cj4gKwo+ICsJcGdtYXAtPnJl cy5zdGFydCA9IHBjaV9yZXNvdXJjZV9zdGFydChwZGV2LCBiYXIpICsgb2Zmc2V0Owo+ICsJcGdt YXAtPnJlcy5lbmQgPSBwZ21hcC0+cmVzLnN0YXJ0ICsgc2l6ZSAtIDE7Cj4gKwlwZ21hcC0+cmVz LmZsYWdzID0gcGNpX3Jlc291cmNlX2ZsYWdzKHBkZXYsIGJhcik7Cj4gKwlwZ21hcC0+cmVmID0g JnBkZXYtPnAycGRtYS0+ZGV2bWFwX3JlZjsKPiArCXBnbWFwLT50eXBlID0gTUVNT1JZX0RFVklD RV9QQ0lfUDJQRE1BOwo+ICsKPiArCWFkZHIgPSBkZXZtX21lbXJlbWFwX3BhZ2VzKCZwZGV2LT5k ZXYsIHBnbWFwKTsKPiArCWlmIChJU19FUlIoYWRkcikpIHsKPiArCQllcnJvciA9IFBUUl9FUlIo YWRkcik7Cj4gKwkJZ290byBwZ21hcF9mcmVlOwo+ICsJfQo+ICsKPiArCWVycm9yID0gZ2VuX3Bv b2xfYWRkX3ZpcnQocGRldi0+cDJwZG1hLT5wb29sLCAodW5zaWduZWQgbG9uZylhZGRyLAo+ICsJ CQlwY2lfYnVzX2FkZHJlc3MocGRldiwgYmFyKSArIG9mZnNldCwKPiArCQkJcmVzb3VyY2Vfc2l6 ZSgmcGdtYXAtPnJlcyksIGRldl90b19ub2RlKCZwZGV2LT5kZXYpKTsKPiArCWlmIChlcnJvcikK PiArCQlnb3RvIHBnbWFwX2ZyZWU7Cj4gKwo+ICsJZXJyb3IgPSBkZXZtX2FkZF9hY3Rpb25fb3Jf cmVzZXQoJnBkZXYtPmRldiwgcGNpX3AycGRtYV9wZXJjcHVfa2lsbCwKPiArCQkJCQkgICZwZGV2 LT5wMnBkbWEtPmRldm1hcF9yZWYpOwo+ICsJaWYgKGVycm9yKQo+ICsJCWdvdG8gcGdtYXBfZnJl ZTsKPiArCj4gKwlwY2lfaW5mbyhwZGV2LCAiYWRkZWQgcGVlci10by1wZWVyIERNQSBtZW1vcnkg JXBSXG4iLAo+ICsJCSAmcGdtYXAtPnJlcyk7Cj4gKwo+ICsJcmV0dXJuIDA7Cj4gKwo+ICtwZ21h cF9mcmVlOgo+ICsJZGV2cmVzX2ZyZWUocGdtYXApOwo+ICsJcmV0dXJuIGVycm9yOwo+ICt9Cj4g K0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9wMnBkbWFfYWRkX3Jlc291cmNlKTsKPiArCj4gK3N0YXRp YyBzdHJ1Y3QgcGNpX2RldiAqZmluZF9wYXJlbnRfcGNpX2RldihzdHJ1Y3QgZGV2aWNlICpkZXYp Cj4gK3sKPiArCXN0cnVjdCBkZXZpY2UgKnBhcmVudDsKPiArCj4gKwlkZXYgPSBnZXRfZGV2aWNl KGRldik7Cj4gKwo+ICsJd2hpbGUgKGRldikgewo+ICsJCWlmIChkZXZfaXNfcGNpKGRldikpCj4g KwkJCXJldHVybiB0b19wY2lfZGV2KGRldik7Cj4gKwo+ICsJCXBhcmVudCA9IGdldF9kZXZpY2Uo ZGV2LT5wYXJlbnQpOwo+ICsJCXB1dF9kZXZpY2UoZGV2KTsKPiArCQlkZXYgPSBwYXJlbnQ7Cj4g Kwl9Cj4gKwo+ICsJcmV0dXJuIE5VTEw7Cj4gK30KPiArCj4gKy8qCj4gKyAqIENoZWNrIGlmIGEg UENJIGJyaWRnZSBoYXMgaXQncyBBQ1MgcmVkaXJlY3Rpb24gYml0cyBzZXQgdG8gcmVkaXJlY3Qg UDJQCj4gKyAqIFRMUHMgdXBzdHJlYW0gdmlhIEFDUy4gUmV0dXJucyAxIGlmIHRoZSBwYWNrZXRz IHdpbGwgYmUgcmVkaXJlY3RlZAo+ICsgKiB1cHN0cmVhbSwgMCBvdGhlcndpc2UuCj4gKyAqLwo+ ICtzdGF0aWMgaW50IHBjaV9icmlkZ2VfaGFzX2Fjc19yZWRpcihzdHJ1Y3QgcGNpX2RldiAqZGV2 KQo+ICt7Cj4gKwlpbnQgcG9zOwo+ICsJdTE2IGN0cmw7Cj4gKwo+ICsJcG9zID0gcGNpX2ZpbmRf ZXh0X2NhcGFiaWxpdHkoZGV2LCBQQ0lfRVhUX0NBUF9JRF9BQ1MpOwo+ICsJaWYgKCFwb3MpCj4g KwkJcmV0dXJuIDA7Cj4gKwo+ICsJcGNpX3JlYWRfY29uZmlnX3dvcmQoZGV2LCBwb3MgKyBQQ0lf QUNTX0NUUkwsICZjdHJsKTsKPiArCj4gKwlpZiAoY3RybCAmIChQQ0lfQUNTX1JSIHwgUENJX0FD U19DUiB8IFBDSV9BQ1NfRUMpKQo+ICsJCXJldHVybiAxOwo+ICsKPiArCXJldHVybiAwOwo+ICt9 Cj4gKwo+ICtzdGF0aWMgdm9pZCBzZXFfYnVmX3ByaW50X2J1c19kZXZmbihzdHJ1Y3Qgc2VxX2J1 ZiAqYnVmLCBzdHJ1Y3QgcGNpX2RldiAqZGV2KQo+ICt7Cj4gKwlpZiAoIWJ1ZikKPiArCQlyZXR1 cm47Cj4gKwo+ICsJc2VxX2J1Zl9wcmludGYoYnVmLCAiJTA0eDolMDJ4OiUwMnguJXg7IiwgcGNp X2RvbWFpbl9ucihkZXYtPmJ1cyksCj4gKwkJICAgICAgIGRldi0+YnVzLT5udW1iZXIsIFBDSV9T TE9UKGRldi0+ZGV2Zm4pLAo+ICsJCSAgICAgICBQQ0lfRlVOQyhkZXYtPmRldmZuKSk7Cj4gK30K PiArCj4gKy8qCj4gKyAqIEZpbmQgdGhlIGRpc3RhbmNlIHRocm91Z2ggdGhlIG5lYXJlc3QgY29t bW9uIHVwc3RyZWFtIGJyaWRnZSBiZXR3ZWVuCj4gKyAqIHR3byBQQ0kgZGV2aWNlcy4KPiArICoK PiArICogSWYgdGhlIHR3byBkZXZpY2VzIGFyZSB0aGUgc2FtZSBkZXZpY2UgdGhlbiAwIHdpbGwg YmUgcmV0dXJuZWQuCj4gKyAqCj4gKyAqIElmIHRoZXJlIGFyZSB0d28gdmlydHVhbCBmdW5jdGlv bnMgb2YgdGhlIHNhbWUgZGV2aWNlIGJlaGluZCB0aGUgc2FtZQo+ICsgKiBicmlkZ2UgcG9ydCB0 aGVuIDIgd2lsbCBiZSByZXR1cm5lZCAob25lIHN0ZXAgZG93biB0byB0aGUgUENJZSBzd2l0Y2gs Cj4gKyAqIHRoZW4gb25lIHN0ZXAgYmFjayB0byB0aGUgc2FtZSBkZXZpY2UpLgo+ICsgKgo+ICsg KiBJbiB0aGUgY2FzZSB3aGVyZSB0d28gZGV2aWNlcyBhcmUgY29ubmVjdGVkIHRvIHRoZSBzYW1l IFBDSWUgc3dpdGNoLCB0aGUKPiArICogdmFsdWUgNCB3aWxsIGJlIHJldHVybmVkLiBUaGlzIGNv cnJlc3BvbmRzIHRvIHRoZSBmb2xsb3dpbmcgUENJIHRyZWU6Cj4gKyAqCj4gKyAqICAgICAtKyAg Um9vdCBQb3J0Cj4gKyAqICAgICAgXCsgU3dpdGNoIFVwc3RyZWFtIFBvcnQKPiArICogICAgICAg Ky0rIFN3aXRjaCBEb3duc3RyZWFtIFBvcnQKPiArICogICAgICAgKyBcLSBEZXZpY2UgQQo+ICsg KiAgICAgICBcLSsgU3dpdGNoIERvd25zdHJlYW0gUG9ydAo+ICsgKiAgICAgICAgIFwtIERldmlj ZSBCCj4gKyAqCj4gKyAqIFRoZSBkaXN0YW5jZSBpcyA0IGJlY2F1c2Ugd2UgdHJhdmVyc2UgZnJv bSBEZXZpY2UgQSB0aHJvdWdoIHRoZSBkb3duc3RyZWFtCj4gKyAqIHBvcnQgb2YgdGhlIHN3aXRj aCwgdG8gdGhlIGNvbW1vbiB1cHN0cmVhbSBwb3J0LCBiYWNrIHVwIHRvIHRoZSBzZWNvbmQKPiAr ICogZG93bnN0cmVhbSBwb3J0IGFuZCB0aGVuIHRvIERldmljZSBCLgo+ICsgKgo+ICsgKiBBbnkg dHdvIGRldmljZXMgdGhhdCBkb24ndCBoYXZlIGEgY29tbW9uIHVwc3RyZWFtIGJyaWRnZSB3aWxs IHJldHVybiAtMS4KPiArICogSW4gdGhpcyB3YXkgZGV2aWNlcyBvbiBzZXBhcmF0ZSBQQ0llIHJv b3QgcG9ydHMgd2lsbCBiZSByZWplY3RlZCwgd2hpY2gKPiArICogaXMgd2hhdCB3ZSB3YW50IGZv ciBwZWVyLXRvLXBlZXIgc2VlaW5nIGVhY2ggUENJZSByb290IHBvcnQgZGVmaW5lcyBhCj4gKyAq IHNlcGFyYXRlIGhpZXJhcmNoeSBkb21haW4gYW5kIHRoZXJlJ3Mgbm8gd2F5IHRvIGRldGVybWlu ZSB3aGV0aGVyIHRoZSByb290Cj4gKyAqIGNvbXBsZXggc3VwcG9ydHMgZm9yd2FyZGluZyBiZXR3 ZWVuIHRoZW0uCj4gKyAqCj4gKyAqIEluIHRoZSBjYXNlIHdoZXJlIHR3byBkZXZpY2VzIGFyZSBj b25uZWN0ZWQgdG8gZGlmZmVyZW50IFBDSWUgc3dpdGNoZXMsCj4gKyAqIHRoaXMgZnVuY3Rpb24g d2lsbCBzdGlsbCByZXR1cm4gYSBwb3NpdGl2ZSBkaXN0YW5jZSBhcyBsb25nIGFzIGJvdGgKPiAr ICogc3dpdGNoZXMgZXZlbnV0YWxseSBoYXZlIGEgY29tbW9uIHVwc3RyZWFtIGJyaWRnZS4gTm90 ZSB0aGlzIGNvdmVycwo+ICsgKiB0aGUgY2FzZSBvZiB1c2luZyBtdWx0aXBsZSBQQ0llIHN3aXRj aGVzIHRvIGFjaGlldmUgYSBkZXNpcmVkIGxldmVsIG9mCj4gKyAqIGZhbi1vdXQgZnJvbSBhIHJv b3QgcG9ydC4gVGhlIGV4YWN0IGRpc3RhbmNlIHdpbGwgYmUgYSBmdW5jdGlvbiBvZiB0aGUKPiAr ICogbnVtYmVyIG9mIHN3aXRjaGVzIGJldHdlZW4gRGV2aWNlIEEgYW5kIERldmljZSBCLgo+ICsg Kgo+ICsgKiBJZiBhIGJyaWRnZSB3aGljaCBoYXMgYW55IEFDUyByZWRpcmVjdGlvbiBiaXRzIHNl dCBpcyBpbiB0aGUgcGF0aAo+ICsgKiB0aGVuIHRoaXMgZnVuY3Rpb25zIHdpbGwgcmV0dXJuIC0y LiBUaGlzIGlzIHNvIHdlIHJlamVjdCBhbnkKPiArICogY2FzZXMgd2hlcmUgdGhlIFRMUHMgYXJl IGZvcndhcmRlZCB1cCBpbnRvIHRoZSByb290IGNvbXBsZXguCj4gKyAqIEluIHRoaXMgY2FzZSwg YSBsaXN0IG9mIGFsbCBpbmZyaW5naW5nIGJyaWRnZSBhZGRyZXNzZXMgd2lsbCBiZQo+ICsgKiBw b3B1bGF0ZWQgaW4gYWNzX2xpc3QgKGFzc3VtaW5nIGl0J3Mgbm9uLW51bGwpIGZvciBwcmludGsg cHVycG9zZXMuCj4gKyAqLwo+ICtzdGF0aWMgaW50IHVwc3RyZWFtX2JyaWRnZV9kaXN0YW5jZShz dHJ1Y3QgcGNpX2RldiAqYSwKPiArCQkJCSAgICBzdHJ1Y3QgcGNpX2RldiAqYiwKPiArCQkJCSAg ICBzdHJ1Y3Qgc2VxX2J1ZiAqYWNzX2xpc3QpCj4gK3sKPiArCWludCBkaXN0X2EgPSAwOwo+ICsJ aW50IGRpc3RfYiA9IDA7Cj4gKwlzdHJ1Y3QgcGNpX2RldiAqYmIgPSBOVUxMOwo+ICsJaW50IGFj c19jbnQgPSAwOwo+ICsKPiArCS8qCj4gKwkgKiBOb3RlLCB3ZSBkb24ndCBuZWVkIHRvIHRha2Ug cmVmZXJlbmNlcyB0byBkZXZpY2VzIHJldHVybmVkIGJ5Cj4gKwkgKiBwY2lfdXBzdHJlYW1fYnJp ZGdlKCkgc2VlaW5nIHdlIGhvbGQgYSByZWZlcmVuY2UgdG8gYSBjaGlsZAo+ICsJICogZGV2aWNl IHdoaWNoIHdpbGwgYWxyZWFkeSBob2xkIGEgcmVmZXJlbmNlIHRvIHRoZSB1cHN0cmVhbSBicmlk Z2UuCj4gKwkgKi8KPiArCj4gKwl3aGlsZSAoYSkgewo+ICsJCWRpc3RfYiA9IDA7Cj4gKwo+ICsJ CWlmIChwY2lfYnJpZGdlX2hhc19hY3NfcmVkaXIoYSkpIHsKPiArCQkJc2VxX2J1Zl9wcmludF9i dXNfZGV2Zm4oYWNzX2xpc3QsIGEpOwo+ICsJCQlhY3NfY250Kys7Cj4gKwkJfQo+ICsKPiArCQli YiA9IGI7Cj4gKwo+ICsJCXdoaWxlIChiYikgewo+ICsJCQlpZiAoYSA9PSBiYikKPiArCQkJCWdv dG8gY2hlY2tfYl9wYXRoX2FjczsKPiArCj4gKwkJCWJiID0gcGNpX3Vwc3RyZWFtX2JyaWRnZShi Yik7Cj4gKwkJCWRpc3RfYisrOwo+ICsJCX0KPiArCj4gKwkJYSA9IHBjaV91cHN0cmVhbV9icmlk Z2UoYSk7Cj4gKwkJZGlzdF9hKys7Cj4gKwl9Cj4gKwo+ICsJcmV0dXJuIC0xOwo+ICsKPiArY2hl Y2tfYl9wYXRoX2FjczoKPiArCWJiID0gYjsKPiArCj4gKwl3aGlsZSAoYmIpIHsKPiArCQlpZiAo YSA9PSBiYikKPiArCQkJYnJlYWs7Cj4gKwo+ICsJCWlmIChwY2lfYnJpZGdlX2hhc19hY3NfcmVk aXIoYmIpKSB7Cj4gKwkJCXNlcV9idWZfcHJpbnRfYnVzX2RldmZuKGFjc19saXN0LCBiYik7Cj4g KwkJCWFjc19jbnQrKzsKPiArCQl9Cj4gKwo+ICsJCWJiID0gcGNpX3Vwc3RyZWFtX2JyaWRnZShi Yik7Cj4gKwl9Cj4gKwo+ICsJaWYgKGFjc19jbnQpCj4gKwkJcmV0dXJuIC0yOwo+ICsKPiArCXJl dHVybiBkaXN0X2EgKyBkaXN0X2I7Cj4gK30KPiArCj4gK3N0YXRpYyBpbnQgdXBzdHJlYW1fYnJp ZGdlX2Rpc3RhbmNlX3dhcm4oc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyLAo+ICsJCQkJCSBzdHJ1 Y3QgcGNpX2RldiAqY2xpZW50KQo+ICt7Cj4gKwlzdHJ1Y3Qgc2VxX2J1ZiBhY3NfbGlzdDsKPiAr CWludCByZXQ7Cj4gKwo+ICsJc2VxX2J1Zl9pbml0KCZhY3NfbGlzdCwga21hbGxvYyhQQUdFX1NJ WkUsIEdGUF9LRVJORUwpLCBQQUdFX1NJWkUpOwo+ICsKPiArCXJldCA9IHVwc3RyZWFtX2JyaWRn ZV9kaXN0YW5jZShwcm92aWRlciwgY2xpZW50LCAmYWNzX2xpc3QpOwo+ICsJaWYgKHJldCA9PSAt Mikgewo+ICsJCXBjaV93YXJuKGNsaWVudCwgImNhbm5vdCBiZSB1c2VkIGZvciBwZWVyLXRvLXBl ZXIgRE1BIGFzIEFDUyByZWRpcmVjdCBpcyBzZXQgYmV0d2VlbiB0aGUgY2xpZW50IGFuZCBwcm92 aWRlclxuIik7Cj4gKwkJLyogRHJvcCBmaW5hbCBzZW1pY29sb24gKi8KPiArCQlhY3NfbGlzdC5i dWZmZXJbYWNzX2xpc3QubGVuLTFdID0gMDsKPiArCQlwY2lfd2FybihjbGllbnQsICJ0byBkaXNh YmxlIEFDUyByZWRpcmVjdCBmb3IgdGhpcyBwYXRoLCBhZGQgdGhlIGtlcm5lbCBwYXJhbWV0ZXI6 IHBjaT1kaXNhYmxlX2Fjc19yZWRpcj0lc1xuIiwKPiArCQkJIGFjc19saXN0LmJ1ZmZlcik7Cj4g Kwo+ICsJfSBlbHNlIGlmIChyZXQgPCAwKSB7Cj4gKwkJcGNpX3dhcm4oY2xpZW50LCAiY2Fubm90 IGJlIHVzZWQgZm9yIHBlZXItdG8tcGVlciBETUEgYXMgdGhlIGNsaWVudCBhbmQgcHJvdmlkZXIg ZG8gbm90IHNoYXJlIGFuIHVwc3RyZWFtIGJyaWRnZVxuIik7Cj4gKwl9Cj4gKwo+ICsJa2ZyZWUo YWNzX2xpc3QuYnVmZmVyKTsKPiArCj4gKwlyZXR1cm4gcmV0Owo+ICt9Cj4gKwo+ICtzdHJ1Y3Qg cGNpX3AycGRtYV9jbGllbnQgewo+ICsJc3RydWN0IGxpc3RfaGVhZCBsaXN0Owo+ICsJc3RydWN0 IHBjaV9kZXYgKmNsaWVudDsKPiArCXN0cnVjdCBwY2lfZGV2ICpwcm92aWRlcjsKPiArfTsKPiAr Cj4gKy8qKgo+ICsgKiBwY2lfcDJwZG1hX2FkZF9jbGllbnQgLSBhbGxvY2F0ZSBhIG5ldyBlbGVt ZW50IGluIGEgY2xpZW50IGRldmljZSBsaXN0Cj4gKyAqIEBoZWFkOiBsaXN0IGhlYWQgb2YgcDJw ZG1hIGNsaWVudHMKPiArICogQGRldjogZGV2aWNlIHRvIGFkZCB0byB0aGUgbGlzdAo+ICsgKgo+ ICsgKiBUaGlzIGFkZHMgQGRldiB0byBhIGxpc3Qgb2YgY2xpZW50cyB1c2VkIGJ5IGEgcDJwZG1h IGRldmljZS4KPiArICogVGhpcyBsaXN0IHNob3VsZCBiZSBwYXNzZWQgdG8gcGNpX3AycG1lbV9m aW5kKCkuIE9uY2UgcGNpX3AycG1lbV9maW5kKCkgaGFzCj4gKyAqIGJlZW4gY2FsbGVkIHN1Y2Nl c3NmdWxseSwgdGhlIGxpc3Qgd2lsbCBiZSBib3VuZCB0byBhIHNwZWNpZmljIHAycGRtYQo+ICsg KiBkZXZpY2UgYW5kIG5ldyBjbGllbnRzIGNhbiBvbmx5IGJlIGFkZGVkIHRvIHRoZSBsaXN0IGlm IHRoZXkgYXJlCj4gKyAqIHN1cHBvcnRlZCBieSB0aGF0IHAycGRtYSBkZXZpY2UuCj4gKyAqCj4g KyAqIFRoZSBjYWxsZXIgaXMgZXhwZWN0ZWQgdG8gaGF2ZSBhIGxvY2sgd2hpY2ggcHJvdGVjdHMg QGhlYWQgYXMgbmVjZXNzYXJ5Cj4gKyAqIHNvIHRoYXQgbm9uZSBvZiB0aGUgcGNpX3AycCBmdW5j dGlvbnMgY2FuIGJlIGNhbGxlZCBjb25jdXJyZW50bHkKPiArICogb24gdGhhdCBsaXN0Lgo+ICsg Kgo+ICsgKiBSZXR1cm5zIDAgaWYgdGhlIGNsaWVudCB3YXMgc3VjY2Vzc2Z1bGx5IGFkZGVkLgo+ ICsgKi8KPiAraW50IHBjaV9wMnBkbWFfYWRkX2NsaWVudChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFk LCBzdHJ1Y3QgZGV2aWNlICpkZXYpCj4gK3sKPiArCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAq aXRlbSwgKm5ld19pdGVtOwo+ICsJc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyID0gTlVMTDsKPiAr CXN0cnVjdCBwY2lfZGV2ICpjbGllbnQ7Cj4gKwlpbnQgcmV0Owo+ICsKPiArCWlmIChJU19FTkFC TEVEKENPTkZJR19ETUFfVklSVF9PUFMpICYmIGRldi0+ZG1hX29wcyA9PSAmZG1hX3ZpcnRfb3Bz KSB7Cj4gKwkJZGV2X3dhcm4oZGV2LCAiY2Fubm90IGJlIHVzZWQgZm9yIHBlZXItdG8tcGVlciBE TUEgYmVjYXVzZSB0aGUgZHJpdmVyIG1ha2VzIHVzZSBvZiBkbWFfdmlydF9vcHNcbiIpOwo+ICsJ CXJldHVybiAtRU5PREVWOwo+ICsJfQo+ICsKPiArCWNsaWVudCA9IGZpbmRfcGFyZW50X3BjaV9k ZXYoZGV2KTsKPiArCWlmICghY2xpZW50KSB7Cj4gKwkJZGV2X3dhcm4oZGV2LCAiY2Fubm90IGJl IHVzZWQgZm9yIHBlZXItdG8tcGVlciBETUEgYXMgaXQgaXMgbm90IGEgUENJIGRldmljZVxuIik7 Cj4gKwkJcmV0dXJuIC1FTk9ERVY7Cj4gKwl9Cj4gKwo+ICsJaXRlbSA9IGxpc3RfZmlyc3RfZW50 cnlfb3JfbnVsbChoZWFkLCBzdHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQsIGxpc3QpOwo+ICsJaWYg KGl0ZW0gJiYgaXRlbS0+cHJvdmlkZXIpIHsKPiArCQlwcm92aWRlciA9IGl0ZW0tPnByb3ZpZGVy Owo+ICsKPiArCQlyZXQgPSB1cHN0cmVhbV9icmlkZ2VfZGlzdGFuY2Vfd2Fybihwcm92aWRlciwg Y2xpZW50KTsKPiArCQlpZiAocmV0IDwgMCkgewo+ICsJCQlyZXQgPSAtRVhERVY7Cj4gKwkJCWdv dG8gcHV0X2NsaWVudDsKPiArCQl9Cj4gKwl9Cj4gKwo+ICsJbmV3X2l0ZW0gPSBremFsbG9jKHNp emVvZigqbmV3X2l0ZW0pLCBHRlBfS0VSTkVMKTsKPiArCWlmICghbmV3X2l0ZW0pIHsKPiArCQly ZXQgPSAtRU5PTUVNOwo+ICsJCWdvdG8gcHV0X2NsaWVudDsKPiArCX0KPiArCj4gKwluZXdfaXRl bS0+Y2xpZW50ID0gY2xpZW50Owo+ICsJbmV3X2l0ZW0tPnByb3ZpZGVyID0gcGNpX2Rldl9nZXQo cHJvdmlkZXIpOwo+ICsKPiArCWxpc3RfYWRkX3RhaWwoJm5ld19pdGVtLT5saXN0LCBoZWFkKTsK PiArCj4gKwlyZXR1cm4gMDsKPiArCj4gK3B1dF9jbGllbnQ6Cj4gKwlwY2lfZGV2X3B1dChjbGll bnQpOwo+ICsJcmV0dXJuIHJldDsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lfcDJwZG1h X2FkZF9jbGllbnQpOwo+ICsKPiArc3RhdGljIHZvaWQgcGNpX3AycGRtYV9jbGllbnRfZnJlZShz dHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQgKml0ZW0pCj4gK3sKPiArCWxpc3RfZGVsKCZpdGVtLT5s aXN0KTsKPiArCXBjaV9kZXZfcHV0KGl0ZW0tPmNsaWVudCk7Cj4gKwlwY2lfZGV2X3B1dChpdGVt LT5wcm92aWRlcik7Cj4gKwlrZnJlZShpdGVtKTsKPiArfQo+ICsKPiArLyoqCj4gKyAqIHBjaV9w MnBkbWFfcmVtb3ZlX2NsaWVudCAtIHJlbW92ZSBhbmQgZnJlZSBhIHAycGRtYSBjbGllbnQKPiAr ICogQGhlYWQ6IGxpc3QgaGVhZCBvZiBwMnBkbWEgY2xpZW50cwo+ICsgKiBAZGV2OiBkZXZpY2Ug dG8gcmVtb3ZlIGZyb20gdGhlIGxpc3QKPiArICoKPiArICogVGhpcyByZW1vdmVzIEBkZXYgZnJv bSBhIGxpc3Qgb2YgY2xpZW50cyB1c2VkIGJ5IGEgcDJwZG1hIGRldmljZS4KPiArICogVGhlIGNh bGxlciBpcyBleHBlY3RlZCB0byBoYXZlIGEgbG9jayB3aGljaCBwcm90ZWN0cyBAaGVhZCBhcyBu ZWNlc3NhcnkKPiArICogc28gdGhhdCBub25lIG9mIHRoZSBwY2lfcDJwIGZ1bmN0aW9ucyBjYW4g YmUgY2FsbGVkIGNvbmN1cnJlbnRseQo+ICsgKiBvbiB0aGF0IGxpc3QuCj4gKyAqLwo+ICt2b2lk IHBjaV9wMnBkbWFfcmVtb3ZlX2NsaWVudChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkLCBzdHJ1Y3Qg ZGV2aWNlICpkZXYpCj4gK3sKPiArCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAqcG9zLCAqdG1w Owo+ICsJc3RydWN0IHBjaV9kZXYgKnBkZXY7Cj4gKwo+ICsJcGRldiA9IGZpbmRfcGFyZW50X3Bj aV9kZXYoZGV2KTsKPiArCWlmICghcGRldikKPiArCQlyZXR1cm47Cj4gKwo+ICsJbGlzdF9mb3Jf ZWFjaF9lbnRyeV9zYWZlKHBvcywgdG1wLCBoZWFkLCBsaXN0KSB7Cj4gKwkJaWYgKHBvcy0+Y2xp ZW50ICE9IHBkZXYpCj4gKwkJCWNvbnRpbnVlOwo+ICsKPiArCQlwY2lfcDJwZG1hX2NsaWVudF9m cmVlKHBvcyk7Cj4gKwl9Cj4gKwo+ICsJcGNpX2Rldl9wdXQocGRldik7Cj4gK30KPiArRVhQT1JU X1NZTUJPTF9HUEwocGNpX3AycGRtYV9yZW1vdmVfY2xpZW50KTsKPiArCj4gKy8qKgo+ICsgKiBw Y2lfcDJwZG1hX2NsaWVudF9saXN0X2ZyZWUgLSBmcmVlIGFuIGVudGlyZSBsaXN0IG9mIHAycGRt YSBjbGllbnRzCj4gKyAqIEBoZWFkOiBsaXN0IGhlYWQgb2YgcDJwZG1hIGNsaWVudHMKPiArICoK PiArICogVGhpcyByZW1vdmVzIGFsbCBkZXZpY2VzIGluIGEgbGlzdCBvZiBjbGllbnRzIHVzZWQg YnkgYSBwMnBkbWEgZGV2aWNlLgo+ICsgKiBUaGUgY2FsbGVyIGlzIGV4cGVjdGVkIHRvIGhhdmUg YSBsb2NrIHdoaWNoIHByb3RlY3RzIEBoZWFkIGFzIG5lY2Vzc2FyeQo+ICsgKiBzbyB0aGF0IG5v bmUgb2YgdGhlIHBjaV9wMnBkbWEgZnVuY3Rpb25zIGNhbiBiZSBjYWxsZWQgY29uY3VycmVudGx5 Cj4gKyAqIG9uIHRoYXQgbGlzdC4KPiArICovCj4gK3ZvaWQgcGNpX3AycGRtYV9jbGllbnRfbGlz dF9mcmVlKHN0cnVjdCBsaXN0X2hlYWQgKmhlYWQpCj4gK3sKPiArCXN0cnVjdCBwY2lfcDJwZG1h X2NsaWVudCAqcG9zLCAqdG1wOwo+ICsKPiArCWxpc3RfZm9yX2VhY2hfZW50cnlfc2FmZShwb3Ms IHRtcCwgaGVhZCwgbGlzdCkKPiArCQlwY2lfcDJwZG1hX2NsaWVudF9mcmVlKHBvcyk7Cj4gK30K PiArRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycGRtYV9jbGllbnRfbGlzdF9mcmVlKTsKPiArCj4g Ky8qKgo+ICsgKiBwY2lfcDJwZG1hX2Rpc3RhbmNlIC0gRGV0ZXJtaXZlIHRoZSBjdW11bGF0aXZl IGRpc3RhbmNlIGJldHdlZW4KPiArICoJYSBwMnBkbWEgcHJvdmlkZXIgYW5kIHRoZSBjbGllbnRz IGluIHVzZS4KPiArICogQHByb3ZpZGVyOiBwMnBkbWEgcHJvdmlkZXIgdG8gY2hlY2sgYWdhaW5z dCB0aGUgY2xpZW50IGxpc3QKPiArICogQGNsaWVudHM6IGxpc3Qgb2YgZGV2aWNlcyB0byBjaGVj ayAoTlVMTC10ZXJtaW5hdGVkKQo+ICsgKiBAdmVyYm9zZTogaWYgdHJ1ZSwgcHJpbnQgd2Fybmlu Z3MgZm9yIGRldmljZXMgd2hlbiB3ZSByZXR1cm4gLTEKPiArICoKPiArICogUmV0dXJucyAtMSBp ZiBhbnkgb2YgdGhlIGNsaWVudHMgYXJlIG5vdCBjb21wYXRpYmxlIChiZWhpbmQgdGhlIHNhbWUK PiArICogcm9vdCBwb3J0IGFzIHRoZSBwcm92aWRlciksIG90aGVyd2lzZSByZXR1cm5zIGEgcG9z aXRpdmUgbnVtYmVyIHdoZXJlCj4gKyAqIHRoZSBsb3dlciBudW1iZXIgaXMgdGhlIHByZWZlcnJh YmxlIGNob2ljZS4gKElmIHRoZXJlJ3Mgb25lIGNsaWVudAo+ICsgKiB0aGF0J3MgdGhlIHNhbWUg YXMgdGhlIHByb3ZpZGVyIGl0IHdpbGwgcmV0dXJuIDAsIHdoaWNoIGlzIGJlc3QgY2hvaWNlKS4K PiArICoKPiArICogRm9yIG5vdywgImNvbXBhdGlibGUiIG1lYW5zIHRoZSBwcm92aWRlciBhbmQg dGhlIGNsaWVudHMgYXJlIGFsbCBiZWhpbmQKPiArICogdGhlIHNhbWUgUENJIHJvb3QgcG9ydC4g VGhpcyBjdXRzIG91dCBjYXNlcyB0aGF0IG1heSB3b3JrIGJ1dCBpcyBzYWZlc3QKPiArICogZm9y IHRoZSB1c2VyLiBGdXR1cmUgd29yayBjYW4gZXhwYW5kIHRoaXMgdG8gd2hpdGUtbGlzdCByb290 IGNvbXBsZXhlcyB0aGF0Cj4gKyAqIGNhbiBzYWZlbHkgZm9yd2FyZCBiZXR3ZWVuIGVhY2ggcG9y dHMuCj4gKyAqLwo+ICtpbnQgcGNpX3AycGRtYV9kaXN0YW5jZShzdHJ1Y3QgcGNpX2RldiAqcHJv dmlkZXIsIHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMsCj4gKwkJCWJvb2wgdmVyYm9zZSkKPiAr ewo+ICsJc3RydWN0IHBjaV9wMnBkbWFfY2xpZW50ICpwb3M7Cj4gKwlpbnQgcmV0Owo+ICsJaW50 IGRpc3RhbmNlID0gMDsKPiArCWJvb2wgbm90X3N1cHBvcnRlZCA9IGZhbHNlOwo+ICsKPiArCWlm IChsaXN0X2VtcHR5KGNsaWVudHMpKQo+ICsJCXJldHVybiAtMTsKPiArCj4gKwlsaXN0X2Zvcl9l YWNoX2VudHJ5KHBvcywgY2xpZW50cywgbGlzdCkgewo+ICsJCWlmICh2ZXJib3NlKQo+ICsJCQly ZXQgPSB1cHN0cmVhbV9icmlkZ2VfZGlzdGFuY2Vfd2Fybihwcm92aWRlciwKPiArCQkJCQkJCSAg ICBwb3MtPmNsaWVudCk7Cj4gKwkJZWxzZQo+ICsJCQlyZXQgPSB1cHN0cmVhbV9icmlkZ2VfZGlz dGFuY2UocHJvdmlkZXIsIHBvcy0+Y2xpZW50LAo+ICsJCQkJCQkgICAgICAgTlVMTCk7Cj4gKwo+ ICsJCWlmIChyZXQgPCAwKQo+ICsJCQlub3Rfc3VwcG9ydGVkID0gdHJ1ZTsKPiArCj4gKwkJaWYg KG5vdF9zdXBwb3J0ZWQgJiYgIXZlcmJvc2UpCj4gKwkJCWJyZWFrOwo+ICsKPiArCQlkaXN0YW5j ZSArPSByZXQ7Cj4gKwl9Cj4gKwo+ICsJaWYgKG5vdF9zdXBwb3J0ZWQpCj4gKwkJcmV0dXJuIC0x Owo+ICsKPiArCXJldHVybiBkaXN0YW5jZTsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lf cDJwZG1hX2Rpc3RhbmNlKTsKPiArCj4gKy8qKgo+ICsgKiBwY2lfcDJwZG1hX2Fzc2lnbl9wcm92 aWRlciAtIENoZWNrIGNvbXBhdGliaWx5IChhcyBwZXIgcGNpX3AycGRtYV9kaXN0YW5jZSkKPiAr ICoJYW5kIGFzc2lnbiBhIHByb3ZpZGVyIHRvIGEgbGlzdCBvZiBjbGllbnRzCj4gKyAqIEBwcm92 aWRlcjogcDJwZG1hIHByb3ZpZGVyIHRvIGFzc2lnbiB0byB0aGUgY2xpZW50IGxpc3QKPiArICog QGNsaWVudHM6IGxpc3Qgb2YgZGV2aWNlcyB0byBjaGVjayAoTlVMTC10ZXJtaW5hdGVkKQo+ICsg Kgo+ICsgKiBSZXR1cm5zIGZhbHNlIGlmIGFueSBvZiB0aGUgY2xpZW50cyBhcmUgbm90IGNvbXBh dGlibGUsIHRydWUgaWYgdGhlCj4gKyAqIHByb3ZpZGVyIHdhcyBzdWNjZXNzZnVsbHkgYXNzaWdu ZWQgdG8gdGhlIGNsaWVudHMuCj4gKyAqLwo+ICtib29sIHBjaV9wMnBkbWFfYXNzaWduX3Byb3Zp ZGVyKHN0cnVjdCBwY2lfZGV2ICpwcm92aWRlciwKPiArCQkJCXN0cnVjdCBsaXN0X2hlYWQgKmNs aWVudHMpCj4gK3sKPiArCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAqcG9zOwo+ICsKPiArCWlm IChwY2lfcDJwZG1hX2Rpc3RhbmNlKHByb3ZpZGVyLCBjbGllbnRzLCB0cnVlKSA8IDApCj4gKwkJ cmV0dXJuIGZhbHNlOwo+ICsKPiArCWxpc3RfZm9yX2VhY2hfZW50cnkocG9zLCBjbGllbnRzLCBs aXN0KQo+ICsJCXBvcy0+cHJvdmlkZXIgPSBwcm92aWRlcjsKPiArCj4gKwlyZXR1cm4gdHJ1ZTsK PiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lfcDJwZG1hX2Fzc2lnbl9wcm92aWRlcik7Cj4g Kwo+ICsvKioKPiArICogcGNpX2hhc19wMnBtZW0gLSBjaGVjayBpZiBhIGdpdmVuIFBDSSBkZXZp Y2UgaGFzIHB1Ymxpc2hlZCBhbnkgcDJwbWVtCj4gKyAqIEBwZGV2OiBQQ0kgZGV2aWNlIHRvIGNo ZWNrCj4gKyAqLwo+ICtib29sIHBjaV9oYXNfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQo+ ICt7Cj4gKwlyZXR1cm4gcGRldi0+cDJwZG1hICYmIHBkZXYtPnAycGRtYS0+cDJwbWVtX3B1Ymxp c2hlZDsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lfaGFzX3AycG1lbSk7Cj4gKwo+ICsv KioKPiArICogcGNpX3AycG1lbV9maW5kIC0gZmluZCBhIHBlZXItdG8tcGVlciBETUEgbWVtb3J5 IGRldmljZSBjb21wYXRpYmxlIHdpdGgKPiArICoJdGhlIHNwZWNpZmllZCBsaXN0IG9mIGNsaWVu dHMgYW5kIHNob3J0ZXN0IGRpc3RhbmNlIChhcyBkZXRlcm1pbmVkCj4gKyAqCWJ5IHBjaV9wMnBt ZW1fZG1hKCkpCj4gKyAqIEBjbGllbnRzOiBsaXN0IG9mIGRldmljZXMgdG8gY2hlY2sgKE5VTEwt dGVybWluYXRlZCkKPiArICoKPiArICogSWYgbXVsdGlwbGUgZGV2aWNlcyBhcmUgYmVoaW5kIHRo ZSBzYW1lIHN3aXRjaCwgdGhlIG9uZSAiY2xvc2VzdCIgdG8gdGhlCj4gKyAqIGNsaWVudCBkZXZp Y2VzIGluIHVzZSB3aWxsIGJlIGNob3NlbiBmaXJzdC4gKFNvIGlmIG9uZSBvZiB0aGUgcHJvdmlk ZXJzIGFyZQo+ICsgKiB0aGUgc2FtZSBhcyBvbmUgb2YgdGhlIGNsaWVudHMsIHRoYXQgcHJvdmlk ZXIgd2lsbCBiZSB1c2VkIGFoZWFkIG9mIGFueQo+ICsgKiBvdGhlciBwcm92aWRlcnMgdGhhdCBh cmUgdW5yZWxhdGVkKS4gSWYgbXVsdGlwbGUgcHJvdmlkZXJzIGFyZSBhbiBlcXVhbAo+ICsgKiBk aXN0YW5jZSBhd2F5LCBvbmUgd2lsbCBiZSBjaG9zZW4gYXQgcmFuZG9tLgo+ICsgKgo+ICsgKiBS ZXR1cm5zIGEgcG9pbnRlciB0byB0aGUgUENJIGRldmljZSB3aXRoIGEgcmVmZXJlbmNlIHRha2Vu ICh1c2UgcGNpX2Rldl9wdXQKPiArICogdG8gcmV0dXJuIHRoZSByZWZlcmVuY2UpIG9yIE5VTEwg aWYgbm8gY29tcGF0aWJsZSBkZXZpY2UgaXMgZm91bmQuIFRoZQo+ICsgKiBmb3VuZCBwcm92aWRl ciB3aWxsIGFsc28gYmUgYXNzaWduZWQgdG8gdGhlIGNsaWVudCBsaXN0Lgo+ICsgKi8KPiArc3Ry dWN0IHBjaV9kZXYgKnBjaV9wMnBtZW1fZmluZChzdHJ1Y3QgbGlzdF9oZWFkICpjbGllbnRzKQo+ ICt7Cj4gKwlzdHJ1Y3QgcGNpX2RldiAqcGRldiA9IE5VTEw7Cj4gKwlzdHJ1Y3QgcGNpX3AycGRt YV9jbGllbnQgKnBvczsKPiArCWludCBkaXN0YW5jZTsKPiArCWludCBjbG9zZXN0X2Rpc3RhbmNl ID0gSU5UX01BWDsKPiArCXN0cnVjdCBwY2lfZGV2ICoqY2xvc2VzdF9wZGV2czsKPiArCWludCBk ZXZfY250ID0gMDsKPiArCWNvbnN0IGludCBtYXhfZGV2cyA9IFBBR0VfU0laRSAvIHNpemVvZigq Y2xvc2VzdF9wZGV2cyk7Cj4gKwlpbnQgaTsKPiArCj4gKwljbG9zZXN0X3BkZXZzID0ga21hbGxv YyhQQUdFX1NJWkUsIEdGUF9LRVJORUwpOwo+ICsKPiArCXdoaWxlICgocGRldiA9IHBjaV9nZXRf ZGV2aWNlKFBDSV9BTllfSUQsIFBDSV9BTllfSUQsIHBkZXYpKSkgewo+ICsJCWlmICghcGNpX2hh c19wMnBtZW0ocGRldikpCj4gKwkJCWNvbnRpbnVlOwo+ICsKPiArCQlkaXN0YW5jZSA9IHBjaV9w MnBkbWFfZGlzdGFuY2UocGRldiwgY2xpZW50cywgZmFsc2UpOwo+ICsJCWlmIChkaXN0YW5jZSA8 IDAgfHwgZGlzdGFuY2UgPiBjbG9zZXN0X2Rpc3RhbmNlKQo+ICsJCQljb250aW51ZTsKPiArCj4g KwkJaWYgKGRpc3RhbmNlID09IGNsb3Nlc3RfZGlzdGFuY2UgJiYgZGV2X2NudCA+PSBtYXhfZGV2 cykKPiArCQkJY29udGludWU7Cj4gKwo+ICsJCWlmIChkaXN0YW5jZSA8IGNsb3Nlc3RfZGlzdGFu Y2UpIHsKPiArCQkJZm9yIChpID0gMDsgaSA8IGRldl9jbnQ7IGkrKykKPiArCQkJCXBjaV9kZXZf cHV0KGNsb3Nlc3RfcGRldnNbaV0pOwo+ICsKPiArCQkJZGV2X2NudCA9IDA7Cj4gKwkJCWNsb3Nl c3RfZGlzdGFuY2UgPSBkaXN0YW5jZTsKPiArCQl9Cj4gKwo+ICsJCWNsb3Nlc3RfcGRldnNbZGV2 X2NudCsrXSA9IHBjaV9kZXZfZ2V0KHBkZXYpOwo+ICsJfQo+ICsKPiArCWlmIChkZXZfY250KQo+ ICsJCXBkZXYgPSBwY2lfZGV2X2dldChjbG9zZXN0X3BkZXZzW3ByYW5kb21fdTMyX21heChkZXZf Y250KV0pOwo+ICsKPiArCWZvciAoaSA9IDA7IGkgPCBkZXZfY250OyBpKyspCj4gKwkJcGNpX2Rl dl9wdXQoY2xvc2VzdF9wZGV2c1tpXSk7Cj4gKwo+ICsJaWYgKHBkZXYpCj4gKwkJbGlzdF9mb3Jf ZWFjaF9lbnRyeShwb3MsIGNsaWVudHMsIGxpc3QpCj4gKwkJCXBvcy0+cHJvdmlkZXIgPSBwZGV2 Owo+ICsKPiArCWtmcmVlKGNsb3Nlc3RfcGRldnMpOwo+ICsJcmV0dXJuIHBkZXY7Cj4gK30KPiAr RVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycG1lbV9maW5kKTsKPiArCj4gKy8qKgo+ICsgKiBwY2lf YWxsb2NfcDJwX21lbSAtIGFsbG9jYXRlIHBlZXItdG8tcGVlciBETUEgbWVtb3J5Cj4gKyAqIEBw ZGV2OiB0aGUgZGV2aWNlIHRvIGFsbG9jYXRlIG1lbW9yeSBmcm9tCj4gKyAqIEBzaXplOiBudW1i ZXIgb2YgYnl0ZXMgdG8gYWxsb2NhdGUKPiArICoKPiArICogUmV0dXJucyB0aGUgYWxsb2NhdGVk IG1lbW9yeSBvciBOVUxMIG9uIGVycm9yLgo+ICsgKi8KPiArdm9pZCAqcGNpX2FsbG9jX3AycG1l bShzdHJ1Y3QgcGNpX2RldiAqcGRldiwgc2l6ZV90IHNpemUpCj4gK3sKPiArCXZvaWQgKnJldDsK PiArCj4gKwlpZiAodW5saWtlbHkoIXBkZXYtPnAycGRtYSkpCj4gKwkJcmV0dXJuIE5VTEw7Cj4g Kwo+ICsJaWYgKHVubGlrZWx5KCFwZXJjcHVfcmVmX3RyeWdldF9saXZlKCZwZGV2LT5wMnBkbWEt PmRldm1hcF9yZWYpKSkKPiArCQlyZXR1cm4gTlVMTDsKPiArCj4gKwlyZXQgPSAodm9pZCAqKWdl bl9wb29sX2FsbG9jKHBkZXYtPnAycGRtYS0+cG9vbCwgc2l6ZSk7Cj4gKwo+ICsJaWYgKHVubGlr ZWx5KCFyZXQpKQo+ICsJCXBlcmNwdV9yZWZfcHV0KCZwZGV2LT5wMnBkbWEtPmRldm1hcF9yZWYp Owo+ICsKPiArCXJldHVybiByZXQ7Cj4gK30KPiArRVhQT1JUX1NZTUJPTF9HUEwocGNpX2FsbG9j X3AycG1lbSk7Cj4gKwo+ICsvKioKPiArICogcGNpX2ZyZWVfcDJwbWVtIC0gYWxsb2NhdGUgcGVl ci10by1wZWVyIERNQSBtZW1vcnkKPiArICogQHBkZXY6IHRoZSBkZXZpY2UgdGhlIG1lbW9yeSB3 YXMgYWxsb2NhdGVkIGZyb20KPiArICogQGFkZHI6IGFkZHJlc3Mgb2YgdGhlIG1lbW9yeSB0aGF0 IHdhcyBhbGxvY2F0ZWQKPiArICogQHNpemU6IG51bWJlciBvZiBieXRlcyB0aGF0IHdhcyBhbGxv Y2F0ZWQKPiArICovCj4gK3ZvaWQgcGNpX2ZyZWVfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpwZGV2 LCB2b2lkICphZGRyLCBzaXplX3Qgc2l6ZSkKPiArewo+ICsJZ2VuX3Bvb2xfZnJlZShwZGV2LT5w MnBkbWEtPnBvb2wsICh1aW50cHRyX3QpYWRkciwgc2l6ZSk7Cj4gKwlwZXJjcHVfcmVmX3B1dCgm cGRldi0+cDJwZG1hLT5kZXZtYXBfcmVmKTsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lf ZnJlZV9wMnBtZW0pOwo+ICsKPiArLyoqCj4gKyAqIHBjaV92aXJ0X3RvX2J1cyAtIHJldHVybiB0 aGUgUENJIGJ1cyBhZGRyZXNzIGZvciBhIGdpdmVuIHZpcnR1YWwKPiArICoJYWRkcmVzcyBvYnRh aW5lZCB3aXRoIHBjaV9hbGxvY19wMnBtZW0oKQo+ICsgKiBAcGRldjogdGhlIGRldmljZSB0aGUg bWVtb3J5IHdhcyBhbGxvY2F0ZWQgZnJvbQo+ICsgKiBAYWRkcjogYWRkcmVzcyBvZiB0aGUgbWVt b3J5IHRoYXQgd2FzIGFsbG9jYXRlZAo+ICsgKi8KPiArcGNpX2J1c19hZGRyX3QgcGNpX3AycG1l bV92aXJ0X3RvX2J1cyhzdHJ1Y3QgcGNpX2RldiAqcGRldiwgdm9pZCAqYWRkcikKPiArewo+ICsJ aWYgKCFhZGRyKQo+ICsJCXJldHVybiAwOwo+ICsJaWYgKCFwZGV2LT5wMnBkbWEpCj4gKwkJcmV0 dXJuIDA7Cj4gKwo+ICsJLyoKPiArCSAqIE5vdGU6IHdoZW4gd2UgYWRkZWQgdGhlIG1lbW9yeSB0 byB0aGUgcG9vbCB3ZSB1c2VkIHRoZSBQQ0kKPiArCSAqIGJ1cyBhZGRyZXNzIGFzIHRoZSBwaHlz aWNhbCBhZGRyZXNzLiBTbyBnZW5fcG9vbF92aXJ0X3RvX3BoeXMoKQo+ICsJICogYWN0dWFsbHkg cmV0dXJucyB0aGUgYnVzIGFkZHJlc3MgZGVzcGl0ZSB0aGUgbWlzbGVhZGluZyBuYW1lLgo+ICsJ ICovCj4gKwlyZXR1cm4gZ2VuX3Bvb2xfdmlydF90b19waHlzKHBkZXYtPnAycGRtYS0+cG9vbCwg KHVuc2lnbmVkIGxvbmcpYWRkcik7Cj4gK30KPiArRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycG1l bV92aXJ0X3RvX2J1cyk7Cj4gKwo+ICsvKioKPiArICogcGNpX3AycG1lbV9hbGxvY19zZ2wgLSBh bGxvY2F0ZSBwZWVyLXRvLXBlZXIgRE1BIG1lbW9yeSBpbiBhIHNjYXR0ZXJsaXN0Cj4gKyAqIEBw ZGV2OiB0aGUgZGV2aWNlIHRvIGFsbG9jYXRlIG1lbW9yeSBmcm9tCj4gKyAqIEBzZ2w6IHRoZSBh bGxvY2F0ZWQgc2NhdHRlcmxpc3QKPiArICogQG5lbnRzOiB0aGUgbnVtYmVyIG9mIFNHIGVudHJp ZXMgaW4gdGhlIGxpc3QKPiArICogQGxlbmd0aDogbnVtYmVyIG9mIGJ5dGVzIHRvIGFsbG9jYXRl Cj4gKyAqCj4gKyAqIFJldHVybnMgMCBvbiBzdWNjZXNzCj4gKyAqLwo+ICtzdHJ1Y3Qgc2NhdHRl cmxpc3QgKnBjaV9wMnBtZW1fYWxsb2Nfc2dsKHN0cnVjdCBwY2lfZGV2ICpwZGV2LAo+ICsJCQkJ CSB1bnNpZ25lZCBpbnQgKm5lbnRzLCB1MzIgbGVuZ3RoKQo+ICt7Cj4gKwlzdHJ1Y3Qgc2NhdHRl cmxpc3QgKnNnOwo+ICsJdm9pZCAqYWRkcjsKPiArCj4gKwlzZyA9IGt6YWxsb2Moc2l6ZW9mKCpz ZyksIEdGUF9LRVJORUwpOwo+ICsJaWYgKCFzZykKPiArCQlyZXR1cm4gTlVMTDsKPiArCj4gKwlz Z19pbml0X3RhYmxlKHNnLCAxKTsKPiArCj4gKwlhZGRyID0gcGNpX2FsbG9jX3AycG1lbShwZGV2 LCBsZW5ndGgpOwo+ICsJaWYgKCFhZGRyKQo+ICsJCWdvdG8gb3V0X2ZyZWVfc2c7Cj4gKwo+ICsJ c2dfc2V0X2J1ZihzZywgYWRkciwgbGVuZ3RoKTsKPiArCSpuZW50cyA9IDE7Cj4gKwlyZXR1cm4g c2c7Cj4gKwo+ICtvdXRfZnJlZV9zZzoKPiArCWtmcmVlKHNnKTsKPiArCXJldHVybiBOVUxMOwo+ ICt9Cj4gK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9wMnBtZW1fYWxsb2Nfc2dsKTsKPiArCj4gKy8q Kgo+ICsgKiBwY2lfcDJwbWVtX2ZyZWVfc2dsIC0gZnJlZSBhIHNjYXR0ZXJsaXN0IGFsbG9jYXRl ZCBieSBwY2lfcDJwbWVtX2FsbG9jX3NnbCgpCj4gKyAqIEBwZGV2OiB0aGUgZGV2aWNlIHRvIGFs bG9jYXRlIG1lbW9yeSBmcm9tCj4gKyAqIEBzZ2w6IHRoZSBhbGxvY2F0ZWQgc2NhdHRlcmxpc3QK PiArICogQG5lbnRzOiB0aGUgbnVtYmVyIG9mIFNHIGVudHJpZXMgaW4gdGhlIGxpc3QKPiArICov Cj4gK3ZvaWQgcGNpX3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwgc3RydWN0 IHNjYXR0ZXJsaXN0ICpzZ2wpCj4gK3sKPiArCXN0cnVjdCBzY2F0dGVybGlzdCAqc2c7Cj4gKwlp bnQgY291bnQ7Cj4gKwo+ICsJZm9yX2VhY2hfc2coc2dsLCBzZywgSU5UX01BWCwgY291bnQpIHsK PiArCQlpZiAoIXNnKQo+ICsJCQlicmVhazsKPiArCj4gKwkJcGNpX2ZyZWVfcDJwbWVtKHBkZXYs IHNnX3ZpcnQoc2cpLCBzZy0+bGVuZ3RoKTsKPiArCX0KPiArCWtmcmVlKHNnbCk7Cj4gK30KPiAr RVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycG1lbV9mcmVlX3NnbCk7Cj4gKwo+ICsvKioKPiArICog cGNpX3AycG1lbV9wdWJsaXNoIC0gcHVibGlzaCB0aGUgcGVlci10by1wZWVyIERNQSBtZW1vcnkg Zm9yIHVzZSBieQo+ICsgKglvdGhlciBkZXZpY2VzIHdpdGggcGNpX3AycG1lbV9maW5kKCkKPiAr ICogQHBkZXY6IHRoZSBkZXZpY2Ugd2l0aCBwZWVyLXRvLXBlZXIgRE1BIG1lbW9yeSB0byBwdWJs aXNoCj4gKyAqIEBwdWJsaXNoOiBzZXQgdG8gdHJ1ZSB0byBwdWJsaXNoIHRoZSBtZW1vcnksIGZh bHNlIHRvIHVucHVibGlzaCBpdAo+ICsgKgo+ICsgKiBQdWJsaXNoZWQgbWVtb3J5IGNhbiBiZSB1 c2VkIGJ5IG90aGVyIFBDSSBkZXZpY2UgZHJpdmVycyBmb3IKPiArICogcGVlci0yLXBlZXIgRE1B IG9wZXJhdGlvbnMuIE5vbi1wdWJsaXNoZWQgbWVtb3J5IGlzIHJlc2VydmVkIGZvcgo+ICsgKiBl eGx1c2l2ZSB1c2Ugb2YgdGhlIGRldmljZSBkcml2ZXIgdGhhdCByZWdpc3RlcnMgdGhlIHBlZXIt dG8tcGVlcgo+ICsgKiBtZW1vcnkuCj4gKyAqLwo+ICt2b2lkIHBjaV9wMnBtZW1fcHVibGlzaChz dHJ1Y3QgcGNpX2RldiAqcGRldiwgYm9vbCBwdWJsaXNoKQo+ICt7Cj4gKwlpZiAocHVibGlzaCAm JiAhcGRldi0+cDJwZG1hKQo+ICsJCXJldHVybjsKPiArCj4gKwlwZGV2LT5wMnBkbWEtPnAycG1l bV9wdWJsaXNoZWQgPSBwdWJsaXNoOwo+ICt9Cj4gK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9wMnBt ZW1fcHVibGlzaCk7Cj4gZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvbWVtcmVtYXAuaCBiL2lu Y2x1ZGUvbGludXgvbWVtcmVtYXAuaAo+IGluZGV4IGY5MWY5ZTc2MzU1Ny4uOTU1MzM3MGViZGFk IDEwMDY0NAo+IC0tLSBhL2luY2x1ZGUvbGludXgvbWVtcmVtYXAuaAo+ICsrKyBiL2luY2x1ZGUv bGludXgvbWVtcmVtYXAuaAo+IEBAIC01MywxMSArNTMsMTYgQEAgc3RydWN0IHZtZW1fYWx0bWFw IHsKPiAgICAqIHdha2V1cCBldmVudCB3aGVuZXZlciBhIHBhZ2UgaXMgdW5waW5uZWQgYW5kIGJl Y29tZXMgaWRsZS4gVGhpcwo+ICAgICogd2FrZXVwIGlzIHVzZWQgdG8gY29vcmRpbmF0ZSBwaHlz aWNhbCBhZGRyZXNzIHNwYWNlIG1hbmFnZW1lbnQgKGV4Ogo+ICAgICogZnMgdHJ1bmNhdGUvaG9s ZSBwdW5jaCkgdnMgcGlubmVkIHBhZ2VzIChleDogZGV2aWNlIGRtYSkuCj4gKyAqCj4gKyAqIE1F TU9SWV9ERVZJQ0VfUENJX1AyUERNQToKPiArICogRGV2aWNlIG1lbW9yeSByZXNpZGluZyBpbiBh IFBDSSBCQVIgaW50ZW5kZWQgZm9yIHVzZSB3aXRoIFBlZXItdG8tUGVlcgo+ICsgKiB0cmFuc2Fj dGlvbnMuCj4gICAgKi8KPiAgIGVudW0gbWVtb3J5X3R5cGUgewo+ICAgCU1FTU9SWV9ERVZJQ0Vf UFJJVkFURSA9IDEsCj4gICAJTUVNT1JZX0RFVklDRV9QVUJMSUMsCj4gICAJTUVNT1JZX0RFVklD RV9GU19EQVgsCj4gKwlNRU1PUllfREVWSUNFX1BDSV9QMlBETUEsCj4gICB9Owo+ICAgCj4gICAv Kgo+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L21tLmggYi9pbmNsdWRlL2xpbnV4L21tLmgK PiBpbmRleCBhNjFlYmU4YWQ0Y2EuLjIwNTVkZjQxMmE3NyAxMDA2NDQKPiAtLS0gYS9pbmNsdWRl L2xpbnV4L21tLmgKPiArKysgYi9pbmNsdWRlL2xpbnV4L21tLmgKPiBAQCAtODkwLDYgKzg5MCwx OSBAQCBzdGF0aWMgaW5saW5lIGJvb2wgaXNfZGV2aWNlX3B1YmxpY19wYWdlKGNvbnN0IHN0cnVj dCBwYWdlICpwYWdlKQo+ICAgCQlwYWdlLT5wZ21hcC0+dHlwZSA9PSBNRU1PUllfREVWSUNFX1BV QkxJQzsKPiAgIH0KPiAgIAo+ICsjaWZkZWYgQ09ORklHX1BDSV9QMlBETUEKPiArc3RhdGljIGlu bGluZSBib29sIGlzX3BjaV9wMnBkbWFfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiAr ewo+ICsJcmV0dXJuIGlzX3pvbmVfZGV2aWNlX3BhZ2UocGFnZSkgJiYKPiArCQlwYWdlLT5wZ21h cC0+dHlwZSA9PSBNRU1PUllfREVWSUNFX1BDSV9QMlBETUE7Cj4gK30KPiArI2Vsc2UgLyogQ09O RklHX1BDSV9QMlBETUEgKi8KPiArc3RhdGljIGlubGluZSBib29sIGlzX3BjaV9wMnBkbWFfcGFn ZShjb25zdCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiArewo+ICsJcmV0dXJuIGZhbHNlOwo+ICt9Cj4g KyNlbmRpZiAvKiBDT05GSUdfUENJX1AyUERNQSAqLwo+ICsKPiAgICNlbHNlIC8qIENPTkZJR19E RVZfUEFHRU1BUF9PUFMgKi8KPiAgIHN0YXRpYyBpbmxpbmUgdm9pZCBkZXZfcGFnZW1hcF9nZXRf b3BzKHZvaWQpCj4gICB7Cj4gQEAgLTkxMyw2ICs5MjYsMTEgQEAgc3RhdGljIGlubGluZSBib29s IGlzX2RldmljZV9wdWJsaWNfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiAgIHsKPiAg IAlyZXR1cm4gZmFsc2U7Cj4gICB9Cj4gKwo+ICtzdGF0aWMgaW5saW5lIGJvb2wgaXNfcGNpX3Ay cGRtYV9wYWdlKGNvbnN0IHN0cnVjdCBwYWdlICpwYWdlKQo+ICt7Cj4gKwlyZXR1cm4gZmFsc2U7 Cj4gK30KPiAgICNlbmRpZiAvKiBDT05GSUdfREVWX1BBR0VNQVBfT1BTICovCj4gICAKPiAgIHN0 YXRpYyBpbmxpbmUgdm9pZCBnZXRfcGFnZShzdHJ1Y3QgcGFnZSAqcGFnZSkKPiBkaWZmIC0tZ2l0 IGEvaW5jbHVkZS9saW51eC9wY2ktcDJwZG1hLmggYi9pbmNsdWRlL2xpbnV4L3BjaS1wMnBkbWEu aAo+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0Cj4gaW5kZXggMDAwMDAwMDAwMDAwLi43YjJiMGY1NDc1 MjgKPiAtLS0gL2Rldi9udWxsCj4gKysrIGIvaW5jbHVkZS9saW51eC9wY2ktcDJwZG1hLmgKPiBA QCAtMCwwICsxLDEwMiBAQAo+ICsvKiBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMCAq Lwo+ICsvKgo+ICsgKiBQQ0kgUGVlciAyIFBlZXIgRE1BIHN1cHBvcnQuCj4gKyAqCj4gKyAqIENv cHlyaWdodCAoYykgMjAxNi0yMDE4LCBMb2dhbiBHdW50aG9ycGUKPiArICogQ29weXJpZ2h0IChj KSAyMDE2LTIwMTcsIE1pY3Jvc2VtaSBDb3Jwb3JhdGlvbgo+ICsgKiBDb3B5cmlnaHQgKGMpIDIw MTcsIENocmlzdG9waCBIZWxsd2lnCj4gKyAqIENvcHlyaWdodCAoYykgMjAxOCwgRWlkZXRpY29t IEluYy4KPiArICoKPiArICovCj4gKwo+ICsjaWZuZGVmIF9MSU5VWF9QQ0lfUDJQRE1BX0gKPiAr I2RlZmluZSBfTElOVVhfUENJX1AyUERNQV9ICj4gKwo+ICsjaW5jbHVkZSA8bGludXgvcGNpLmg+ Cj4gKwo+ICtzdHJ1Y3QgYmxvY2tfZGV2aWNlOwo+ICtzdHJ1Y3Qgc2NhdHRlcmxpc3Q7Cj4gKwo+ ICsjaWZkZWYgQ09ORklHX1BDSV9QMlBETUEKPiAraW50IHBjaV9wMnBkbWFfYWRkX3Jlc291cmNl KHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBpbnQgYmFyLCBzaXplX3Qgc2l6ZSwKPiArCQl1NjQgb2Zm c2V0KTsKPiAraW50IHBjaV9wMnBkbWFfYWRkX2NsaWVudChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFk LCBzdHJ1Y3QgZGV2aWNlICpkZXYpOwo+ICt2b2lkIHBjaV9wMnBkbWFfcmVtb3ZlX2NsaWVudChz dHJ1Y3QgbGlzdF9oZWFkICpoZWFkLCBzdHJ1Y3QgZGV2aWNlICpkZXYpOwo+ICt2b2lkIHBjaV9w MnBkbWFfY2xpZW50X2xpc3RfZnJlZShzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkKTsKPiAraW50IHBj aV9wMnBkbWFfZGlzdGFuY2Uoc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyLCBzdHJ1Y3QgbGlzdF9o ZWFkICpjbGllbnRzLAo+ICsJCQlib29sIHZlcmJvc2UpOwo+ICtib29sIHBjaV9wMnBkbWFfYXNz aWduX3Byb3ZpZGVyKHN0cnVjdCBwY2lfZGV2ICpwcm92aWRlciwKPiArCQkJCXN0cnVjdCBsaXN0 X2hlYWQgKmNsaWVudHMpOwo+ICtib29sIHBjaV9oYXNfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpw ZGV2KTsKPiArc3RydWN0IHBjaV9kZXYgKnBjaV9wMnBtZW1fZmluZChzdHJ1Y3QgbGlzdF9oZWFk ICpjbGllbnRzKTsKPiArdm9pZCAqcGNpX2FsbG9jX3AycG1lbShzdHJ1Y3QgcGNpX2RldiAqcGRl diwgc2l6ZV90IHNpemUpOwo+ICt2b2lkIHBjaV9mcmVlX3AycG1lbShzdHJ1Y3QgcGNpX2RldiAq cGRldiwgdm9pZCAqYWRkciwgc2l6ZV90IHNpemUpOwo+ICtwY2lfYnVzX2FkZHJfdCBwY2lfcDJw bWVtX3ZpcnRfdG9fYnVzKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB2b2lkICphZGRyKTsKPiArc3Ry dWN0IHNjYXR0ZXJsaXN0ICpwY2lfcDJwbWVtX2FsbG9jX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRl diwKPiArCQkJCQkgdW5zaWduZWQgaW50ICpuZW50cywgdTMyIGxlbmd0aCk7Cj4gK3ZvaWQgcGNp X3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwgc3RydWN0IHNjYXR0ZXJsaXN0 ICpzZ2wpOwo+ICt2b2lkIHBjaV9wMnBtZW1fcHVibGlzaChzdHJ1Y3QgcGNpX2RldiAqcGRldiwg Ym9vbCBwdWJsaXNoKTsKPiArI2Vsc2UgLyogQ09ORklHX1BDSV9QMlBETUEgKi8KPiArc3RhdGlj IGlubGluZSBpbnQgcGNpX3AycGRtYV9hZGRfcmVzb3VyY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYs IGludCBiYXIsCj4gKwkJc2l6ZV90IHNpemUsIHU2NCBvZmZzZXQpCj4gK3sKPiArCXJldHVybiAt RU9QTk9UU1VQUDsKPiArfQo+ICtzdGF0aWMgaW5saW5lIGludCBwY2lfcDJwZG1hX2FkZF9jbGll bnQoc3RydWN0IGxpc3RfaGVhZCAqaGVhZCwKPiArCQlzdHJ1Y3QgZGV2aWNlICpkZXYpCj4gK3sK PiArCXJldHVybiAwOwo+ICt9Cj4gK3N0YXRpYyBpbmxpbmUgdm9pZCBwY2lfcDJwZG1hX3JlbW92 ZV9jbGllbnQoc3RydWN0IGxpc3RfaGVhZCAqaGVhZCwKPiArCQlzdHJ1Y3QgZGV2aWNlICpkZXYp Cj4gK3sKPiArfQo+ICtzdGF0aWMgaW5saW5lIHZvaWQgcGNpX3AycGRtYV9jbGllbnRfbGlzdF9m cmVlKHN0cnVjdCBsaXN0X2hlYWQgKmhlYWQpCj4gK3sKPiArfQo+ICtzdGF0aWMgaW5saW5lIGlu dCBwY2lfcDJwZG1hX2Rpc3RhbmNlKHN0cnVjdCBwY2lfZGV2ICpwcm92aWRlciwKPiArCQkJCSAg ICAgIHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMsCj4gKwkJCQkgICAgICBib29sIHZlcmJvc2Up Cj4gK3sKPiArCXJldHVybiAtMTsKPiArfQo+ICtzdGF0aWMgaW5saW5lIGJvb2wgcGNpX3AycGRt YV9hc3NpZ25fcHJvdmlkZXIoc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyLAo+ICsJCQkJCSAgICAg IHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMpCj4gK3sKPiArCXJldHVybiBmYWxzZTsKPiArfQo+ ICtzdGF0aWMgaW5saW5lIGJvb2wgcGNpX2hhc19wMnBtZW0oc3RydWN0IHBjaV9kZXYgKnBkZXYp Cj4gK3sKPiArCXJldHVybiBmYWxzZTsKPiArfQo+ICtzdGF0aWMgaW5saW5lIHN0cnVjdCBwY2lf ZGV2ICpwY2lfcDJwbWVtX2ZpbmQoc3RydWN0IGxpc3RfaGVhZCAqY2xpZW50cykKPiArewo+ICsJ cmV0dXJuIE5VTEw7Cj4gK30KPiArc3RhdGljIGlubGluZSB2b2lkICpwY2lfYWxsb2NfcDJwbWVt KHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBzaXplX3Qgc2l6ZSkKPiArewo+ICsJcmV0dXJuIE5VTEw7 Cj4gK30KPiArc3RhdGljIGlubGluZSB2b2lkIHBjaV9mcmVlX3AycG1lbShzdHJ1Y3QgcGNpX2Rl diAqcGRldiwgdm9pZCAqYWRkciwKPiArCQlzaXplX3Qgc2l6ZSkKPiArewo+ICt9Cj4gK3N0YXRp YyBpbmxpbmUgcGNpX2J1c19hZGRyX3QgcGNpX3AycG1lbV92aXJ0X3RvX2J1cyhzdHJ1Y3QgcGNp X2RldiAqcGRldiwKPiArCQkJCQkJICAgIHZvaWQgKmFkZHIpCj4gK3sKPiArCXJldHVybiAwOwo+ ICt9Cj4gK3N0YXRpYyBpbmxpbmUgc3RydWN0IHNjYXR0ZXJsaXN0ICpwY2lfcDJwbWVtX2FsbG9j X3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwKPiArCQl1bnNpZ25lZCBpbnQgKm5lbnRzLCB1MzIg bGVuZ3RoKQo+ICt7Cj4gKwlyZXR1cm4gTlVMTDsKPiArfQo+ICtzdGF0aWMgaW5saW5lIHZvaWQg cGNpX3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwKPiArCQlzdHJ1Y3Qgc2Nh dHRlcmxpc3QgKnNnbCkKPiArewo+ICt9Cj4gK3N0YXRpYyBpbmxpbmUgdm9pZCBwY2lfcDJwbWVt X3B1Ymxpc2goc3RydWN0IHBjaV9kZXYgKnBkZXYsIGJvb2wgcHVibGlzaCkKPiArewo+ICt9Cj4g KyNlbmRpZiAvKiBDT05GSUdfUENJX1AyUERNQSAqLwo+ICsjZW5kaWYgLyogX0xJTlVYX1BDSV9Q MlBfSCAqLwo+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L3BjaS5oIGIvaW5jbHVkZS9saW51 eC9wY2kuaAo+IGluZGV4IGU3MmNhOGRkNjI0MS4uNWQ5NWRiZjIxZjRhIDEwMDY0NAo+IC0tLSBh L2luY2x1ZGUvbGludXgvcGNpLmgKPiArKysgYi9pbmNsdWRlL2xpbnV4L3BjaS5oCj4gQEAgLTI4 MSw2ICsyODEsNyBAQCBzdHJ1Y3QgcGNpZV9saW5rX3N0YXRlOwo+ICAgc3RydWN0IHBjaV92cGQ7 Cj4gICBzdHJ1Y3QgcGNpX3NyaW92Owo+ICAgc3RydWN0IHBjaV9hdHM7Cj4gK3N0cnVjdCBwY2lf cDJwZG1hOwo+ICAgCj4gICAvKiBUaGUgcGNpX2RldiBzdHJ1Y3R1cmUgZGVzY3JpYmVzIFBDSSBk ZXZpY2VzICovCj4gICBzdHJ1Y3QgcGNpX2RldiB7Cj4gQEAgLTQzOSw2ICs0NDAsOSBAQCBzdHJ1 Y3QgcGNpX2RldiB7Cj4gICAjaWZkZWYgQ09ORklHX1BDSV9QQVNJRAo+ICAgCXUxNgkJcGFzaWRf ZmVhdHVyZXM7Cj4gICAjZW5kaWYKPiArI2lmZGVmIENPTkZJR19QQ0lfUDJQRE1BCj4gKwlzdHJ1 Y3QgcGNpX3AycGRtYSAqcDJwZG1hOwo+ICsjZW5kaWYKPiAgIAlwaHlzX2FkZHJfdAlyb207CQkv KiBQaHlzaWNhbCBhZGRyZXNzIGlmIG5vdCBmcm9tIEJBUiAqLwo+ICAgCXNpemVfdAkJcm9tbGVu OwkJLyogTGVuZ3RoIGlmIG5vdCBmcm9tIEJBUiAqLwo+ICAgCWNoYXIJCSpkcml2ZXJfb3ZlcnJp ZGU7IC8qIERyaXZlciBuYW1lIHRvIGZvcmNlIGEgbWF0Y2ggKi8KCl9fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkxpbnV4LW52ZGltbSBtYWlsaW5nIGxpc3QK TGludXgtbnZkaW1tQGxpc3RzLjAxLm9yZwpodHRwczovL2xpc3RzLjAxLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2xpbnV4LW52ZGltbQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-eopbgr710055.outbound.protection.outlook.com ([40.107.71.55]:27104 "EHLO NAM05-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727268AbeHaMLE (ORCPT ); Fri, 31 Aug 2018 08:11:04 -0400 Subject: Re: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory To: Logan Gunthorpe , linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org Cc: Stephen Bates , Christoph Hellwig , Keith Busch , Sagi Grimberg , Bjorn Helgaas , Jason Gunthorpe , Max Gurtovoy , Dan Williams , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Benjamin Herrenschmidt , Alex Williamson References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> From: =?UTF-8?Q?Christian_K=c3=b6nig?= Message-ID: <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> Date: Fri, 31 Aug 2018 10:04:29 +0200 MIME-Version: 1.0 In-Reply-To: <20180830185352.3369-2-logang@deltatee.com> Content-Type: text/plain; charset=utf-8; format=flowed Sender: linux-block-owner@vger.kernel.org List-Id: linux-block@vger.kernel.org Am 30.08.2018 um 20:53 schrieb Logan Gunthorpe: > Some PCI devices may have memory mapped in a BAR space that's > intended for use in peer-to-peer transactions. In order to enable > such transactions the memory must be registered with ZONE_DEVICE pages > so it can be used by DMA interfaces in existing drivers. We want to use that feature without ZONE_DEVICE pages for DMA-buf as well. How hard would it be to separate enabling P2P detection (e.g. distance between two devices) from this? Regards, Christian. > > Add an interface for other subsystems to find and allocate chunks of P2P > memory as necessary to facilitate transfers between two PCI peers: > > int pci_p2pdma_add_client(); > struct pci_dev *pci_p2pmem_find(); > void *pci_alloc_p2pmem(); > > The new interface requires a driver to collect a list of client devices > involved in the transaction with the pci_p2pmem_add_client*() functions > then call pci_p2pmem_find() to obtain any suitable P2P memory. Once > this is done the list is bound to the memory and the calling driver is > free to add and remove clients as necessary (adding incompatible clients > will fail). With a suitable p2pmem device, memory can then be > allocated with pci_alloc_p2pmem() for use in DMA transactions. > > Depending on hardware, using peer-to-peer memory may reduce the bandwidth > of the transfer but can significantly reduce pressure on system memory. > This may be desirable in many cases: for example a system could be designed > with a small CPU connected to a PCIe switch by a small number of lanes > which would maximize the number of lanes available to connect to NVMe > devices. > > The code is designed to only utilize the p2pmem device if all the devices > involved in a transfer are behind the same PCI bridge. This is because we > have no way of knowing whether peer-to-peer routing between PCIe Root Ports > is supported (PCIe r4.0, sec 1.3.1). Additionally, the benefits of P2P > transfers that go through the RC is limited to only reducing DRAM usage > and, in some cases, coding convenience. The PCI-SIG may be exploring > adding a new capability bit to advertise whether this is possible for > future hardware. > > This commit includes significant rework and feedback from Christoph > Hellwig. > > Signed-off-by: Christoph Hellwig > Signed-off-by: Logan Gunthorpe > --- > drivers/pci/Kconfig | 17 + > drivers/pci/Makefile | 1 + > drivers/pci/p2pdma.c | 761 +++++++++++++++++++++++++++++++++++++++++++++ > include/linux/memremap.h | 5 + > include/linux/mm.h | 18 ++ > include/linux/pci-p2pdma.h | 102 ++++++ > include/linux/pci.h | 4 + > 7 files changed, 908 insertions(+) > create mode 100644 drivers/pci/p2pdma.c > create mode 100644 include/linux/pci-p2pdma.h > > diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig > index 56ff8f6d31fc..deb68be4fdac 100644 > --- a/drivers/pci/Kconfig > +++ b/drivers/pci/Kconfig > @@ -132,6 +132,23 @@ config PCI_PASID > > If unsure, say N. > > +config PCI_P2PDMA > + bool "PCI peer-to-peer transfer support" > + depends on PCI && ZONE_DEVICE > + select GENERIC_ALLOCATOR > + help > + Enableѕ drivers to do PCI peer-to-peer transactions to and from > + BARs that are exposed in other devices that are the part of > + the hierarchy where peer-to-peer DMA is guaranteed by the PCI > + specification to work (ie. anything below a single PCI bridge). > + > + Many PCIe root complexes do not support P2P transactions and > + it's hard to tell which support it at all, so at this time, > + P2P DMA transations must be between devices behind the same root > + port. > + > + If unsure, say N. > + > config PCI_LABEL > def_bool y if (DMI || ACPI) > depends on PCI > diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile > index 1b2cfe51e8d7..85f4a703b2be 100644 > --- a/drivers/pci/Makefile > +++ b/drivers/pci/Makefile > @@ -26,6 +26,7 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o > obj-$(CONFIG_PCI_STUB) += pci-stub.o > obj-$(CONFIG_PCI_PF_STUB) += pci-pf-stub.o > obj-$(CONFIG_PCI_ECAM) += ecam.o > +obj-$(CONFIG_PCI_P2PDMA) += p2pdma.o > obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o > > # Endpoint library must be initialized before its users > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c > new file mode 100644 > index 000000000000..88aaec5351cd > --- /dev/null > +++ b/drivers/pci/p2pdma.c > @@ -0,0 +1,761 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * PCI Peer 2 Peer DMA support. > + * > + * Copyright (c) 2016-2018, Logan Gunthorpe > + * Copyright (c) 2016-2017, Microsemi Corporation > + * Copyright (c) 2017, Christoph Hellwig > + * Copyright (c) 2018, Eideticom Inc. > + */ > + > +#define pr_fmt(fmt) "pci-p2pdma: " fmt > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +struct pci_p2pdma { > + struct percpu_ref devmap_ref; > + struct completion devmap_ref_done; > + struct gen_pool *pool; > + bool p2pmem_published; > +}; > + > +static void pci_p2pdma_percpu_release(struct percpu_ref *ref) > +{ > + struct pci_p2pdma *p2p = > + container_of(ref, struct pci_p2pdma, devmap_ref); > + > + complete_all(&p2p->devmap_ref_done); > +} > + > +static void pci_p2pdma_percpu_kill(void *data) > +{ > + struct percpu_ref *ref = data; > + > + if (percpu_ref_is_dying(ref)) > + return; > + > + percpu_ref_kill(ref); > +} > + > +static void pci_p2pdma_release(void *data) > +{ > + struct pci_dev *pdev = data; > + > + if (!pdev->p2pdma) > + return; > + > + wait_for_completion(&pdev->p2pdma->devmap_ref_done); > + percpu_ref_exit(&pdev->p2pdma->devmap_ref); > + > + gen_pool_destroy(pdev->p2pdma->pool); > + pdev->p2pdma = NULL; > +} > + > +static int pci_p2pdma_setup(struct pci_dev *pdev) > +{ > + int error = -ENOMEM; > + struct pci_p2pdma *p2p; > + > + p2p = devm_kzalloc(&pdev->dev, sizeof(*p2p), GFP_KERNEL); > + if (!p2p) > + return -ENOMEM; > + > + p2p->pool = gen_pool_create(PAGE_SHIFT, dev_to_node(&pdev->dev)); > + if (!p2p->pool) > + goto out; > + > + init_completion(&p2p->devmap_ref_done); > + error = percpu_ref_init(&p2p->devmap_ref, > + pci_p2pdma_percpu_release, 0, GFP_KERNEL); > + if (error) > + goto out_pool_destroy; > + > + percpu_ref_switch_to_atomic_sync(&p2p->devmap_ref); > + > + error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev); > + if (error) > + goto out_pool_destroy; > + > + pdev->p2pdma = p2p; > + > + return 0; > + > +out_pool_destroy: > + gen_pool_destroy(p2p->pool); > +out: > + devm_kfree(&pdev->dev, p2p); > + return error; > +} > + > +/** > + * pci_p2pdma_add_resource - add memory for use as p2p memory > + * @pdev: the device to add the memory to > + * @bar: PCI BAR to add > + * @size: size of the memory to add, may be zero to use the whole BAR > + * @offset: offset into the PCI BAR > + * > + * The memory will be given ZONE_DEVICE struct pages so that it may > + * be used with any DMA request. > + */ > +int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, > + u64 offset) > +{ > + struct dev_pagemap *pgmap; > + void *addr; > + int error; > + > + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) > + return -EINVAL; > + > + if (offset >= pci_resource_len(pdev, bar)) > + return -EINVAL; > + > + if (!size) > + size = pci_resource_len(pdev, bar) - offset; > + > + if (size + offset > pci_resource_len(pdev, bar)) > + return -EINVAL; > + > + if (!pdev->p2pdma) { > + error = pci_p2pdma_setup(pdev); > + if (error) > + return error; > + } > + > + pgmap = devm_kzalloc(&pdev->dev, sizeof(*pgmap), GFP_KERNEL); > + if (!pgmap) > + return -ENOMEM; > + > + pgmap->res.start = pci_resource_start(pdev, bar) + offset; > + pgmap->res.end = pgmap->res.start + size - 1; > + pgmap->res.flags = pci_resource_flags(pdev, bar); > + pgmap->ref = &pdev->p2pdma->devmap_ref; > + pgmap->type = MEMORY_DEVICE_PCI_P2PDMA; > + > + addr = devm_memremap_pages(&pdev->dev, pgmap); > + if (IS_ERR(addr)) { > + error = PTR_ERR(addr); > + goto pgmap_free; > + } > + > + error = gen_pool_add_virt(pdev->p2pdma->pool, (unsigned long)addr, > + pci_bus_address(pdev, bar) + offset, > + resource_size(&pgmap->res), dev_to_node(&pdev->dev)); > + if (error) > + goto pgmap_free; > + > + error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_percpu_kill, > + &pdev->p2pdma->devmap_ref); > + if (error) > + goto pgmap_free; > + > + pci_info(pdev, "added peer-to-peer DMA memory %pR\n", > + &pgmap->res); > + > + return 0; > + > +pgmap_free: > + devres_free(pgmap); > + return error; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_add_resource); > + > +static struct pci_dev *find_parent_pci_dev(struct device *dev) > +{ > + struct device *parent; > + > + dev = get_device(dev); > + > + while (dev) { > + if (dev_is_pci(dev)) > + return to_pci_dev(dev); > + > + parent = get_device(dev->parent); > + put_device(dev); > + dev = parent; > + } > + > + return NULL; > +} > + > +/* > + * Check if a PCI bridge has it's ACS redirection bits set to redirect P2P > + * TLPs upstream via ACS. Returns 1 if the packets will be redirected > + * upstream, 0 otherwise. > + */ > +static int pci_bridge_has_acs_redir(struct pci_dev *dev) > +{ > + int pos; > + u16 ctrl; > + > + pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS); > + if (!pos) > + return 0; > + > + pci_read_config_word(dev, pos + PCI_ACS_CTRL, &ctrl); > + > + if (ctrl & (PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_EC)) > + return 1; > + > + return 0; > +} > + > +static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *dev) > +{ > + if (!buf) > + return; > + > + seq_buf_printf(buf, "%04x:%02x:%02x.%x;", pci_domain_nr(dev->bus), > + dev->bus->number, PCI_SLOT(dev->devfn), > + PCI_FUNC(dev->devfn)); > +} > + > +/* > + * Find the distance through the nearest common upstream bridge between > + * two PCI devices. > + * > + * If the two devices are the same device then 0 will be returned. > + * > + * If there are two virtual functions of the same device behind the same > + * bridge port then 2 will be returned (one step down to the PCIe switch, > + * then one step back to the same device). > + * > + * In the case where two devices are connected to the same PCIe switch, the > + * value 4 will be returned. This corresponds to the following PCI tree: > + * > + * -+ Root Port > + * \+ Switch Upstream Port > + * +-+ Switch Downstream Port > + * + \- Device A > + * \-+ Switch Downstream Port > + * \- Device B > + * > + * The distance is 4 because we traverse from Device A through the downstream > + * port of the switch, to the common upstream port, back up to the second > + * downstream port and then to Device B. > + * > + * Any two devices that don't have a common upstream bridge will return -1. > + * In this way devices on separate PCIe root ports will be rejected, which > + * is what we want for peer-to-peer seeing each PCIe root port defines a > + * separate hierarchy domain and there's no way to determine whether the root > + * complex supports forwarding between them. > + * > + * In the case where two devices are connected to different PCIe switches, > + * this function will still return a positive distance as long as both > + * switches evenutally have a common upstream bridge. Note this covers > + * the case of using multiple PCIe switches to achieve a desired level of > + * fan-out from a root port. The exact distance will be a function of the > + * number of switches between Device A and Device B. > + * > + * If a bridge which has any ACS redirection bits set is in the path > + * then this functions will return -2. This is so we reject any > + * cases where the TLPs are forwarded up into the root complex. > + * In this case, a list of all infringing bridge addresses will be > + * populated in acs_list (assuming it's non-null) for printk purposes. > + */ > +static int upstream_bridge_distance(struct pci_dev *a, > + struct pci_dev *b, > + struct seq_buf *acs_list) > +{ > + int dist_a = 0; > + int dist_b = 0; > + struct pci_dev *bb = NULL; > + int acs_cnt = 0; > + > + /* > + * Note, we don't need to take references to devices returned by > + * pci_upstream_bridge() seeing we hold a reference to a child > + * device which will already hold a reference to the upstream bridge. > + */ > + > + while (a) { > + dist_b = 0; > + > + if (pci_bridge_has_acs_redir(a)) { > + seq_buf_print_bus_devfn(acs_list, a); > + acs_cnt++; > + } > + > + bb = b; > + > + while (bb) { > + if (a == bb) > + goto check_b_path_acs; > + > + bb = pci_upstream_bridge(bb); > + dist_b++; > + } > + > + a = pci_upstream_bridge(a); > + dist_a++; > + } > + > + return -1; > + > +check_b_path_acs: > + bb = b; > + > + while (bb) { > + if (a == bb) > + break; > + > + if (pci_bridge_has_acs_redir(bb)) { > + seq_buf_print_bus_devfn(acs_list, bb); > + acs_cnt++; > + } > + > + bb = pci_upstream_bridge(bb); > + } > + > + if (acs_cnt) > + return -2; > + > + return dist_a + dist_b; > +} > + > +static int upstream_bridge_distance_warn(struct pci_dev *provider, > + struct pci_dev *client) > +{ > + struct seq_buf acs_list; > + int ret; > + > + seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); > + > + ret = upstream_bridge_distance(provider, client, &acs_list); > + if (ret == -2) { > + pci_warn(client, "cannot be used for peer-to-peer DMA as ACS redirect is set between the client and provider\n"); > + /* Drop final semicolon */ > + acs_list.buffer[acs_list.len-1] = 0; > + pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", > + acs_list.buffer); > + > + } else if (ret < 0) { > + pci_warn(client, "cannot be used for peer-to-peer DMA as the client and provider do not share an upstream bridge\n"); > + } > + > + kfree(acs_list.buffer); > + > + return ret; > +} > + > +struct pci_p2pdma_client { > + struct list_head list; > + struct pci_dev *client; > + struct pci_dev *provider; > +}; > + > +/** > + * pci_p2pdma_add_client - allocate a new element in a client device list > + * @head: list head of p2pdma clients > + * @dev: device to add to the list > + * > + * This adds @dev to a list of clients used by a p2pdma device. > + * This list should be passed to pci_p2pmem_find(). Once pci_p2pmem_find() has > + * been called successfully, the list will be bound to a specific p2pdma > + * device and new clients can only be added to the list if they are > + * supported by that p2pdma device. > + * > + * The caller is expected to have a lock which protects @head as necessary > + * so that none of the pci_p2p functions can be called concurrently > + * on that list. > + * > + * Returns 0 if the client was successfully added. > + */ > +int pci_p2pdma_add_client(struct list_head *head, struct device *dev) > +{ > + struct pci_p2pdma_client *item, *new_item; > + struct pci_dev *provider = NULL; > + struct pci_dev *client; > + int ret; > + > + if (IS_ENABLED(CONFIG_DMA_VIRT_OPS) && dev->dma_ops == &dma_virt_ops) { > + dev_warn(dev, "cannot be used for peer-to-peer DMA because the driver makes use of dma_virt_ops\n"); > + return -ENODEV; > + } > + > + client = find_parent_pci_dev(dev); > + if (!client) { > + dev_warn(dev, "cannot be used for peer-to-peer DMA as it is not a PCI device\n"); > + return -ENODEV; > + } > + > + item = list_first_entry_or_null(head, struct pci_p2pdma_client, list); > + if (item && item->provider) { > + provider = item->provider; > + > + ret = upstream_bridge_distance_warn(provider, client); > + if (ret < 0) { > + ret = -EXDEV; > + goto put_client; > + } > + } > + > + new_item = kzalloc(sizeof(*new_item), GFP_KERNEL); > + if (!new_item) { > + ret = -ENOMEM; > + goto put_client; > + } > + > + new_item->client = client; > + new_item->provider = pci_dev_get(provider); > + > + list_add_tail(&new_item->list, head); > + > + return 0; > + > +put_client: > + pci_dev_put(client); > + return ret; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_add_client); > + > +static void pci_p2pdma_client_free(struct pci_p2pdma_client *item) > +{ > + list_del(&item->list); > + pci_dev_put(item->client); > + pci_dev_put(item->provider); > + kfree(item); > +} > + > +/** > + * pci_p2pdma_remove_client - remove and free a p2pdma client > + * @head: list head of p2pdma clients > + * @dev: device to remove from the list > + * > + * This removes @dev from a list of clients used by a p2pdma device. > + * The caller is expected to have a lock which protects @head as necessary > + * so that none of the pci_p2p functions can be called concurrently > + * on that list. > + */ > +void pci_p2pdma_remove_client(struct list_head *head, struct device *dev) > +{ > + struct pci_p2pdma_client *pos, *tmp; > + struct pci_dev *pdev; > + > + pdev = find_parent_pci_dev(dev); > + if (!pdev) > + return; > + > + list_for_each_entry_safe(pos, tmp, head, list) { > + if (pos->client != pdev) > + continue; > + > + pci_p2pdma_client_free(pos); > + } > + > + pci_dev_put(pdev); > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_remove_client); > + > +/** > + * pci_p2pdma_client_list_free - free an entire list of p2pdma clients > + * @head: list head of p2pdma clients > + * > + * This removes all devices in a list of clients used by a p2pdma device. > + * The caller is expected to have a lock which protects @head as necessary > + * so that none of the pci_p2pdma functions can be called concurrently > + * on that list. > + */ > +void pci_p2pdma_client_list_free(struct list_head *head) > +{ > + struct pci_p2pdma_client *pos, *tmp; > + > + list_for_each_entry_safe(pos, tmp, head, list) > + pci_p2pdma_client_free(pos); > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_client_list_free); > + > +/** > + * pci_p2pdma_distance - Determive the cumulative distance between > + * a p2pdma provider and the clients in use. > + * @provider: p2pdma provider to check against the client list > + * @clients: list of devices to check (NULL-terminated) > + * @verbose: if true, print warnings for devices when we return -1 > + * > + * Returns -1 if any of the clients are not compatible (behind the same > + * root port as the provider), otherwise returns a positive number where > + * the lower number is the preferrable choice. (If there's one client > + * that's the same as the provider it will return 0, which is best choice). > + * > + * For now, "compatible" means the provider and the clients are all behind > + * the same PCI root port. This cuts out cases that may work but is safest > + * for the user. Future work can expand this to white-list root complexes that > + * can safely forward between each ports. > + */ > +int pci_p2pdma_distance(struct pci_dev *provider, struct list_head *clients, > + bool verbose) > +{ > + struct pci_p2pdma_client *pos; > + int ret; > + int distance = 0; > + bool not_supported = false; > + > + if (list_empty(clients)) > + return -1; > + > + list_for_each_entry(pos, clients, list) { > + if (verbose) > + ret = upstream_bridge_distance_warn(provider, > + pos->client); > + else > + ret = upstream_bridge_distance(provider, pos->client, > + NULL); > + > + if (ret < 0) > + not_supported = true; > + > + if (not_supported && !verbose) > + break; > + > + distance += ret; > + } > + > + if (not_supported) > + return -1; > + > + return distance; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_distance); > + > +/** > + * pci_p2pdma_assign_provider - Check compatibily (as per pci_p2pdma_distance) > + * and assign a provider to a list of clients > + * @provider: p2pdma provider to assign to the client list > + * @clients: list of devices to check (NULL-terminated) > + * > + * Returns false if any of the clients are not compatible, true if the > + * provider was successfully assigned to the clients. > + */ > +bool pci_p2pdma_assign_provider(struct pci_dev *provider, > + struct list_head *clients) > +{ > + struct pci_p2pdma_client *pos; > + > + if (pci_p2pdma_distance(provider, clients, true) < 0) > + return false; > + > + list_for_each_entry(pos, clients, list) > + pos->provider = provider; > + > + return true; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_assign_provider); > + > +/** > + * pci_has_p2pmem - check if a given PCI device has published any p2pmem > + * @pdev: PCI device to check > + */ > +bool pci_has_p2pmem(struct pci_dev *pdev) > +{ > + return pdev->p2pdma && pdev->p2pdma->p2pmem_published; > +} > +EXPORT_SYMBOL_GPL(pci_has_p2pmem); > + > +/** > + * pci_p2pmem_find - find a peer-to-peer DMA memory device compatible with > + * the specified list of clients and shortest distance (as determined > + * by pci_p2pmem_dma()) > + * @clients: list of devices to check (NULL-terminated) > + * > + * If multiple devices are behind the same switch, the one "closest" to the > + * client devices in use will be chosen first. (So if one of the providers are > + * the same as one of the clients, that provider will be used ahead of any > + * other providers that are unrelated). If multiple providers are an equal > + * distance away, one will be chosen at random. > + * > + * Returns a pointer to the PCI device with a reference taken (use pci_dev_put > + * to return the reference) or NULL if no compatible device is found. The > + * found provider will also be assigned to the client list. > + */ > +struct pci_dev *pci_p2pmem_find(struct list_head *clients) > +{ > + struct pci_dev *pdev = NULL; > + struct pci_p2pdma_client *pos; > + int distance; > + int closest_distance = INT_MAX; > + struct pci_dev **closest_pdevs; > + int dev_cnt = 0; > + const int max_devs = PAGE_SIZE / sizeof(*closest_pdevs); > + int i; > + > + closest_pdevs = kmalloc(PAGE_SIZE, GFP_KERNEL); > + > + while ((pdev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, pdev))) { > + if (!pci_has_p2pmem(pdev)) > + continue; > + > + distance = pci_p2pdma_distance(pdev, clients, false); > + if (distance < 0 || distance > closest_distance) > + continue; > + > + if (distance == closest_distance && dev_cnt >= max_devs) > + continue; > + > + if (distance < closest_distance) { > + for (i = 0; i < dev_cnt; i++) > + pci_dev_put(closest_pdevs[i]); > + > + dev_cnt = 0; > + closest_distance = distance; > + } > + > + closest_pdevs[dev_cnt++] = pci_dev_get(pdev); > + } > + > + if (dev_cnt) > + pdev = pci_dev_get(closest_pdevs[prandom_u32_max(dev_cnt)]); > + > + for (i = 0; i < dev_cnt; i++) > + pci_dev_put(closest_pdevs[i]); > + > + if (pdev) > + list_for_each_entry(pos, clients, list) > + pos->provider = pdev; > + > + kfree(closest_pdevs); > + return pdev; > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_find); > + > +/** > + * pci_alloc_p2p_mem - allocate peer-to-peer DMA memory > + * @pdev: the device to allocate memory from > + * @size: number of bytes to allocate > + * > + * Returns the allocated memory or NULL on error. > + */ > +void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) > +{ > + void *ret; > + > + if (unlikely(!pdev->p2pdma)) > + return NULL; > + > + if (unlikely(!percpu_ref_tryget_live(&pdev->p2pdma->devmap_ref))) > + return NULL; > + > + ret = (void *)gen_pool_alloc(pdev->p2pdma->pool, size); > + > + if (unlikely(!ret)) > + percpu_ref_put(&pdev->p2pdma->devmap_ref); > + > + return ret; > +} > +EXPORT_SYMBOL_GPL(pci_alloc_p2pmem); > + > +/** > + * pci_free_p2pmem - allocate peer-to-peer DMA memory > + * @pdev: the device the memory was allocated from > + * @addr: address of the memory that was allocated > + * @size: number of bytes that was allocated > + */ > +void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size) > +{ > + gen_pool_free(pdev->p2pdma->pool, (uintptr_t)addr, size); > + percpu_ref_put(&pdev->p2pdma->devmap_ref); > +} > +EXPORT_SYMBOL_GPL(pci_free_p2pmem); > + > +/** > + * pci_virt_to_bus - return the PCI bus address for a given virtual > + * address obtained with pci_alloc_p2pmem() > + * @pdev: the device the memory was allocated from > + * @addr: address of the memory that was allocated > + */ > +pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, void *addr) > +{ > + if (!addr) > + return 0; > + if (!pdev->p2pdma) > + return 0; > + > + /* > + * Note: when we added the memory to the pool we used the PCI > + * bus address as the physical address. So gen_pool_virt_to_phys() > + * actually returns the bus address despite the misleading name. > + */ > + return gen_pool_virt_to_phys(pdev->p2pdma->pool, (unsigned long)addr); > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_virt_to_bus); > + > +/** > + * pci_p2pmem_alloc_sgl - allocate peer-to-peer DMA memory in a scatterlist > + * @pdev: the device to allocate memory from > + * @sgl: the allocated scatterlist > + * @nents: the number of SG entries in the list > + * @length: number of bytes to allocate > + * > + * Returns 0 on success > + */ > +struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, > + unsigned int *nents, u32 length) > +{ > + struct scatterlist *sg; > + void *addr; > + > + sg = kzalloc(sizeof(*sg), GFP_KERNEL); > + if (!sg) > + return NULL; > + > + sg_init_table(sg, 1); > + > + addr = pci_alloc_p2pmem(pdev, length); > + if (!addr) > + goto out_free_sg; > + > + sg_set_buf(sg, addr, length); > + *nents = 1; > + return sg; > + > +out_free_sg: > + kfree(sg); > + return NULL; > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_alloc_sgl); > + > +/** > + * pci_p2pmem_free_sgl - free a scatterlist allocated by pci_p2pmem_alloc_sgl() > + * @pdev: the device to allocate memory from > + * @sgl: the allocated scatterlist > + * @nents: the number of SG entries in the list > + */ > +void pci_p2pmem_free_sgl(struct pci_dev *pdev, struct scatterlist *sgl) > +{ > + struct scatterlist *sg; > + int count; > + > + for_each_sg(sgl, sg, INT_MAX, count) { > + if (!sg) > + break; > + > + pci_free_p2pmem(pdev, sg_virt(sg), sg->length); > + } > + kfree(sgl); > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_free_sgl); > + > +/** > + * pci_p2pmem_publish - publish the peer-to-peer DMA memory for use by > + * other devices with pci_p2pmem_find() > + * @pdev: the device with peer-to-peer DMA memory to publish > + * @publish: set to true to publish the memory, false to unpublish it > + * > + * Published memory can be used by other PCI device drivers for > + * peer-2-peer DMA operations. Non-published memory is reserved for > + * exlusive use of the device driver that registers the peer-to-peer > + * memory. > + */ > +void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) > +{ > + if (publish && !pdev->p2pdma) > + return; > + > + pdev->p2pdma->p2pmem_published = publish; > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_publish); > diff --git a/include/linux/memremap.h b/include/linux/memremap.h > index f91f9e763557..9553370ebdad 100644 > --- a/include/linux/memremap.h > +++ b/include/linux/memremap.h > @@ -53,11 +53,16 @@ struct vmem_altmap { > * wakeup event whenever a page is unpinned and becomes idle. This > * wakeup is used to coordinate physical address space management (ex: > * fs truncate/hole punch) vs pinned pages (ex: device dma). > + * > + * MEMORY_DEVICE_PCI_P2PDMA: > + * Device memory residing in a PCI BAR intended for use with Peer-to-Peer > + * transactions. > */ > enum memory_type { > MEMORY_DEVICE_PRIVATE = 1, > MEMORY_DEVICE_PUBLIC, > MEMORY_DEVICE_FS_DAX, > + MEMORY_DEVICE_PCI_P2PDMA, > }; > > /* > diff --git a/include/linux/mm.h b/include/linux/mm.h > index a61ebe8ad4ca..2055df412a77 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -890,6 +890,19 @@ static inline bool is_device_public_page(const struct page *page) > page->pgmap->type == MEMORY_DEVICE_PUBLIC; > } > > +#ifdef CONFIG_PCI_P2PDMA > +static inline bool is_pci_p2pdma_page(const struct page *page) > +{ > + return is_zone_device_page(page) && > + page->pgmap->type == MEMORY_DEVICE_PCI_P2PDMA; > +} > +#else /* CONFIG_PCI_P2PDMA */ > +static inline bool is_pci_p2pdma_page(const struct page *page) > +{ > + return false; > +} > +#endif /* CONFIG_PCI_P2PDMA */ > + > #else /* CONFIG_DEV_PAGEMAP_OPS */ > static inline void dev_pagemap_get_ops(void) > { > @@ -913,6 +926,11 @@ static inline bool is_device_public_page(const struct page *page) > { > return false; > } > + > +static inline bool is_pci_p2pdma_page(const struct page *page) > +{ > + return false; > +} > #endif /* CONFIG_DEV_PAGEMAP_OPS */ > > static inline void get_page(struct page *page) > diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h > new file mode 100644 > index 000000000000..7b2b0f547528 > --- /dev/null > +++ b/include/linux/pci-p2pdma.h > @@ -0,0 +1,102 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * PCI Peer 2 Peer DMA support. > + * > + * Copyright (c) 2016-2018, Logan Gunthorpe > + * Copyright (c) 2016-2017, Microsemi Corporation > + * Copyright (c) 2017, Christoph Hellwig > + * Copyright (c) 2018, Eideticom Inc. > + * > + */ > + > +#ifndef _LINUX_PCI_P2PDMA_H > +#define _LINUX_PCI_P2PDMA_H > + > +#include > + > +struct block_device; > +struct scatterlist; > + > +#ifdef CONFIG_PCI_P2PDMA > +int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, > + u64 offset); > +int pci_p2pdma_add_client(struct list_head *head, struct device *dev); > +void pci_p2pdma_remove_client(struct list_head *head, struct device *dev); > +void pci_p2pdma_client_list_free(struct list_head *head); > +int pci_p2pdma_distance(struct pci_dev *provider, struct list_head *clients, > + bool verbose); > +bool pci_p2pdma_assign_provider(struct pci_dev *provider, > + struct list_head *clients); > +bool pci_has_p2pmem(struct pci_dev *pdev); > +struct pci_dev *pci_p2pmem_find(struct list_head *clients); > +void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size); > +void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size); > +pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, void *addr); > +struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, > + unsigned int *nents, u32 length); > +void pci_p2pmem_free_sgl(struct pci_dev *pdev, struct scatterlist *sgl); > +void pci_p2pmem_publish(struct pci_dev *pdev, bool publish); > +#else /* CONFIG_PCI_P2PDMA */ > +static inline int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, > + size_t size, u64 offset) > +{ > + return -EOPNOTSUPP; > +} > +static inline int pci_p2pdma_add_client(struct list_head *head, > + struct device *dev) > +{ > + return 0; > +} > +static inline void pci_p2pdma_remove_client(struct list_head *head, > + struct device *dev) > +{ > +} > +static inline void pci_p2pdma_client_list_free(struct list_head *head) > +{ > +} > +static inline int pci_p2pdma_distance(struct pci_dev *provider, > + struct list_head *clients, > + bool verbose) > +{ > + return -1; > +} > +static inline bool pci_p2pdma_assign_provider(struct pci_dev *provider, > + struct list_head *clients) > +{ > + return false; > +} > +static inline bool pci_has_p2pmem(struct pci_dev *pdev) > +{ > + return false; > +} > +static inline struct pci_dev *pci_p2pmem_find(struct list_head *clients) > +{ > + return NULL; > +} > +static inline void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) > +{ > + return NULL; > +} > +static inline void pci_free_p2pmem(struct pci_dev *pdev, void *addr, > + size_t size) > +{ > +} > +static inline pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, > + void *addr) > +{ > + return 0; > +} > +static inline struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, > + unsigned int *nents, u32 length) > +{ > + return NULL; > +} > +static inline void pci_p2pmem_free_sgl(struct pci_dev *pdev, > + struct scatterlist *sgl) > +{ > +} > +static inline void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) > +{ > +} > +#endif /* CONFIG_PCI_P2PDMA */ > +#endif /* _LINUX_PCI_P2P_H */ > diff --git a/include/linux/pci.h b/include/linux/pci.h > index e72ca8dd6241..5d95dbf21f4a 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -281,6 +281,7 @@ struct pcie_link_state; > struct pci_vpd; > struct pci_sriov; > struct pci_ats; > +struct pci_p2pdma; > > /* The pci_dev structure describes PCI devices */ > struct pci_dev { > @@ -439,6 +440,9 @@ struct pci_dev { > #ifdef CONFIG_PCI_PASID > u16 pasid_features; > #endif > +#ifdef CONFIG_PCI_P2PDMA > + struct pci_p2pdma *p2pdma; > +#endif > phys_addr_t rom; /* Physical address if not from BAR */ > size_t romlen; /* Length if not from BAR */ > char *driver_override; /* Driver name to force a match */ From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?Christian_K=c3=b6nig?= Subject: Re: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory Date: Fri, 31 Aug 2018 10:04:29 +0200 Message-ID: <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20180830185352.3369-2-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: Logan Gunthorpe , linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org, linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Cc: Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-Id: linux-rdma@vger.kernel.org QW0gMzAuMDguMjAxOCB1bSAyMDo1MyBzY2hyaWViIExvZ2FuIEd1bnRob3JwZToKPiBTb21lIFBD SSBkZXZpY2VzIG1heSBoYXZlIG1lbW9yeSBtYXBwZWQgaW4gYSBCQVIgc3BhY2UgdGhhdCdzCj4g aW50ZW5kZWQgZm9yIHVzZSBpbiBwZWVyLXRvLXBlZXIgdHJhbnNhY3Rpb25zLiBJbiBvcmRlciB0 byBlbmFibGUKPiBzdWNoIHRyYW5zYWN0aW9ucyB0aGUgbWVtb3J5IG11c3QgYmUgcmVnaXN0ZXJl ZCB3aXRoIFpPTkVfREVWSUNFIHBhZ2VzCj4gc28gaXQgY2FuIGJlIHVzZWQgYnkgRE1BIGludGVy ZmFjZXMgaW4gZXhpc3RpbmcgZHJpdmVycy4KCldlIHdhbnQgdG8gdXNlIHRoYXQgZmVhdHVyZSB3 aXRob3V0IFpPTkVfREVWSUNFIHBhZ2VzIGZvciBETUEtYnVmIGFzIHdlbGwuCgpIb3cgaGFyZCB3 b3VsZCBpdCBiZSB0byBzZXBhcmF0ZSBlbmFibGluZyBQMlAgZGV0ZWN0aW9uIChlLmcuIGRpc3Rh bmNlIApiZXR3ZWVuIHR3byBkZXZpY2VzKSBmcm9tIHRoaXM/CgpSZWdhcmRzLApDaHJpc3RpYW4u Cgo+Cj4gQWRkIGFuIGludGVyZmFjZSBmb3Igb3RoZXIgc3Vic3lzdGVtcyB0byBmaW5kIGFuZCBh bGxvY2F0ZSBjaHVua3Mgb2YgUDJQCj4gbWVtb3J5IGFzIG5lY2Vzc2FyeSB0byBmYWNpbGl0YXRl IHRyYW5zZmVycyBiZXR3ZWVuIHR3byBQQ0kgcGVlcnM6Cj4KPiBpbnQgcGNpX3AycGRtYV9hZGRf Y2xpZW50KCk7Cj4gc3RydWN0IHBjaV9kZXYgKnBjaV9wMnBtZW1fZmluZCgpOwo+IHZvaWQgKnBj aV9hbGxvY19wMnBtZW0oKTsKPgo+IFRoZSBuZXcgaW50ZXJmYWNlIHJlcXVpcmVzIGEgZHJpdmVy IHRvIGNvbGxlY3QgYSBsaXN0IG9mIGNsaWVudCBkZXZpY2VzCj4gaW52b2x2ZWQgaW4gdGhlIHRy YW5zYWN0aW9uIHdpdGggdGhlIHBjaV9wMnBtZW1fYWRkX2NsaWVudCooKSBmdW5jdGlvbnMKPiB0 aGVuIGNhbGwgcGNpX3AycG1lbV9maW5kKCkgdG8gb2J0YWluIGFueSBzdWl0YWJsZSBQMlAgbWVt b3J5LiBPbmNlCj4gdGhpcyBpcyBkb25lIHRoZSBsaXN0IGlzIGJvdW5kIHRvIHRoZSBtZW1vcnkg YW5kIHRoZSBjYWxsaW5nIGRyaXZlciBpcwo+IGZyZWUgdG8gYWRkIGFuZCByZW1vdmUgY2xpZW50 cyBhcyBuZWNlc3NhcnkgKGFkZGluZyBpbmNvbXBhdGlibGUgY2xpZW50cwo+IHdpbGwgZmFpbCku IFdpdGggYSBzdWl0YWJsZSBwMnBtZW0gZGV2aWNlLCBtZW1vcnkgY2FuIHRoZW4gYmUKPiBhbGxv Y2F0ZWQgd2l0aCBwY2lfYWxsb2NfcDJwbWVtKCkgZm9yIHVzZSBpbiBETUEgdHJhbnNhY3Rpb25z Lgo+Cj4gRGVwZW5kaW5nIG9uIGhhcmR3YXJlLCB1c2luZyBwZWVyLXRvLXBlZXIgbWVtb3J5IG1h eSByZWR1Y2UgdGhlIGJhbmR3aWR0aAo+IG9mIHRoZSB0cmFuc2ZlciBidXQgY2FuIHNpZ25pZmlj YW50bHkgcmVkdWNlIHByZXNzdXJlIG9uIHN5c3RlbSBtZW1vcnkuCj4gVGhpcyBtYXkgYmUgZGVz aXJhYmxlIGluIG1hbnkgY2FzZXM6IGZvciBleGFtcGxlIGEgc3lzdGVtIGNvdWxkIGJlIGRlc2ln bmVkCj4gd2l0aCBhIHNtYWxsIENQVSBjb25uZWN0ZWQgdG8gYSBQQ0llIHN3aXRjaCBieSBhIHNt YWxsIG51bWJlciBvZiBsYW5lcwo+IHdoaWNoIHdvdWxkIG1heGltaXplIHRoZSBudW1iZXIgb2Yg bGFuZXMgYXZhaWxhYmxlIHRvIGNvbm5lY3QgdG8gTlZNZQo+IGRldmljZXMuCj4KPiBUaGUgY29k ZSBpcyBkZXNpZ25lZCB0byBvbmx5IHV0aWxpemUgdGhlIHAycG1lbSBkZXZpY2UgaWYgYWxsIHRo ZSBkZXZpY2VzCj4gaW52b2x2ZWQgaW4gYSB0cmFuc2ZlciBhcmUgYmVoaW5kIHRoZSBzYW1lIFBD SSBicmlkZ2UuIFRoaXMgaXMgYmVjYXVzZSB3ZQo+IGhhdmUgbm8gd2F5IG9mIGtub3dpbmcgd2hl dGhlciBwZWVyLXRvLXBlZXIgcm91dGluZyBiZXR3ZWVuIFBDSWUgUm9vdCBQb3J0cwo+IGlzIHN1 cHBvcnRlZCAoUENJZSByNC4wLCBzZWMgMS4zLjEpLiBBZGRpdGlvbmFsbHksIHRoZSBiZW5lZml0 cyBvZiBQMlAKPiB0cmFuc2ZlcnMgdGhhdCBnbyB0aHJvdWdoIHRoZSBSQyBpcyBsaW1pdGVkIHRv IG9ubHkgcmVkdWNpbmcgRFJBTSB1c2FnZQo+IGFuZCwgaW4gc29tZSBjYXNlcywgY29kaW5nIGNv bnZlbmllbmNlLiBUaGUgUENJLVNJRyBtYXkgYmUgZXhwbG9yaW5nCj4gYWRkaW5nIGEgbmV3IGNh cGFiaWxpdHkgYml0IHRvIGFkdmVydGlzZSB3aGV0aGVyIHRoaXMgaXMgcG9zc2libGUgZm9yCj4g ZnV0dXJlIGhhcmR3YXJlLgo+Cj4gVGhpcyBjb21taXQgaW5jbHVkZXMgc2lnbmlmaWNhbnQgcmV3 b3JrIGFuZCBmZWVkYmFjayBmcm9tIENocmlzdG9waAo+IEhlbGx3aWcuCj4KPiBTaWduZWQtb2Zm LWJ5OiBDaHJpc3RvcGggSGVsbHdpZyA8aGNoQGxzdC5kZT4KPiBTaWduZWQtb2ZmLWJ5OiBMb2dh biBHdW50aG9ycGUgPGxvZ2FuZ0BkZWx0YXRlZS5jb20+Cj4gLS0tCj4gICBkcml2ZXJzL3BjaS9L Y29uZmlnICAgICAgICB8ICAxNyArCj4gICBkcml2ZXJzL3BjaS9NYWtlZmlsZSAgICAgICB8ICAg MSArCj4gICBkcml2ZXJzL3BjaS9wMnBkbWEuYyAgICAgICB8IDc2MSArKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysKPiAgIGluY2x1ZGUvbGludXgvbWVtcmVtYXAu aCAgIHwgICA1ICsKPiAgIGluY2x1ZGUvbGludXgvbW0uaCAgICAgICAgIHwgIDE4ICsrCj4gICBp bmNsdWRlL2xpbnV4L3BjaS1wMnBkbWEuaCB8IDEwMiArKysrKysKPiAgIGluY2x1ZGUvbGludXgv cGNpLmggICAgICAgIHwgICA0ICsKPiAgIDcgZmlsZXMgY2hhbmdlZCwgOTA4IGluc2VydGlvbnMo KykKPiAgIGNyZWF0ZSBtb2RlIDEwMDY0NCBkcml2ZXJzL3BjaS9wMnBkbWEuYwo+ICAgY3JlYXRl IG1vZGUgMTAwNjQ0IGluY2x1ZGUvbGludXgvcGNpLXAycGRtYS5oCj4KPiBkaWZmIC0tZ2l0IGEv ZHJpdmVycy9wY2kvS2NvbmZpZyBiL2RyaXZlcnMvcGNpL0tjb25maWcKPiBpbmRleCA1NmZmOGY2 ZDMxZmMuLmRlYjY4YmU0ZmRhYyAxMDA2NDQKPiAtLS0gYS9kcml2ZXJzL3BjaS9LY29uZmlnCj4g KysrIGIvZHJpdmVycy9wY2kvS2NvbmZpZwo+IEBAIC0xMzIsNiArMTMyLDIzIEBAIGNvbmZpZyBQ Q0lfUEFTSUQKPiAgIAo+ICAgCSAgSWYgdW5zdXJlLCBzYXkgTi4KPiAgIAo+ICtjb25maWcgUENJ X1AyUERNQQo+ICsJYm9vbCAiUENJIHBlZXItdG8tcGVlciB0cmFuc2ZlciBzdXBwb3J0Igo+ICsJ ZGVwZW5kcyBvbiBQQ0kgJiYgWk9ORV9ERVZJQ0UKPiArCXNlbGVjdCBHRU5FUklDX0FMTE9DQVRP Ugo+ICsJaGVscAo+ICsJICBFbmFibGXRlSBkcml2ZXJzIHRvIGRvIFBDSSBwZWVyLXRvLXBlZXIg dHJhbnNhY3Rpb25zIHRvIGFuZCBmcm9tCj4gKwkgIEJBUnMgdGhhdCBhcmUgZXhwb3NlZCBpbiBv dGhlciBkZXZpY2VzIHRoYXQgYXJlIHRoZSBwYXJ0IG9mCj4gKwkgIHRoZSBoaWVyYXJjaHkgd2hl cmUgcGVlci10by1wZWVyIERNQSBpcyBndWFyYW50ZWVkIGJ5IHRoZSBQQ0kKPiArCSAgc3BlY2lm aWNhdGlvbiB0byB3b3JrIChpZS4gYW55dGhpbmcgYmVsb3cgYSBzaW5nbGUgUENJIGJyaWRnZSku Cj4gKwo+ICsJICBNYW55IFBDSWUgcm9vdCBjb21wbGV4ZXMgZG8gbm90IHN1cHBvcnQgUDJQIHRy YW5zYWN0aW9ucyBhbmQKPiArCSAgaXQncyBoYXJkIHRvIHRlbGwgd2hpY2ggc3VwcG9ydCBpdCBh dCBhbGwsIHNvIGF0IHRoaXMgdGltZSwKPiArCSAgUDJQIERNQSB0cmFuc2F0aW9ucyBtdXN0IGJl IGJldHdlZW4gZGV2aWNlcyBiZWhpbmQgdGhlIHNhbWUgcm9vdAo+ICsJICBwb3J0Lgo+ICsKPiAr CSAgSWYgdW5zdXJlLCBzYXkgTi4KPiArCj4gICBjb25maWcgUENJX0xBQkVMCj4gICAJZGVmX2Jv b2wgeSBpZiAoRE1JIHx8IEFDUEkpCj4gICAJZGVwZW5kcyBvbiBQQ0kKPiBkaWZmIC0tZ2l0IGEv ZHJpdmVycy9wY2kvTWFrZWZpbGUgYi9kcml2ZXJzL3BjaS9NYWtlZmlsZQo+IGluZGV4IDFiMmNm ZTUxZThkNy4uODVmNGE3MDNiMmJlIDEwMDY0NAo+IC0tLSBhL2RyaXZlcnMvcGNpL01ha2VmaWxl Cj4gKysrIGIvZHJpdmVycy9wY2kvTWFrZWZpbGUKPiBAQCAtMjYsNiArMjYsNyBAQCBvYmotJChD T05GSUdfUENJX1NZU0NBTEwpCSs9IHN5c2NhbGwubwo+ICAgb2JqLSQoQ09ORklHX1BDSV9TVFVC KQkJKz0gcGNpLXN0dWIubwo+ICAgb2JqLSQoQ09ORklHX1BDSV9QRl9TVFVCKQkrPSBwY2ktcGYt c3R1Yi5vCj4gICBvYmotJChDT05GSUdfUENJX0VDQU0pCQkrPSBlY2FtLm8KPiArb2JqLSQoQ09O RklHX1BDSV9QMlBETUEpCSs9IHAycGRtYS5vCj4gICBvYmotJChDT05GSUdfWEVOX1BDSURFVl9G Uk9OVEVORCkgKz0geGVuLXBjaWZyb250Lm8KPiAgIAo+ICAgIyBFbmRwb2ludCBsaWJyYXJ5IG11 c3QgYmUgaW5pdGlhbGl6ZWQgYmVmb3JlIGl0cyB1c2Vycwo+IGRpZmYgLS1naXQgYS9kcml2ZXJz L3BjaS9wMnBkbWEuYyBiL2RyaXZlcnMvcGNpL3AycGRtYS5jCj4gbmV3IGZpbGUgbW9kZSAxMDA2 NDQKPiBpbmRleCAwMDAwMDAwMDAwMDAuLjg4YWFlYzUzNTFjZAo+IC0tLSAvZGV2L251bGwKPiAr KysgYi9kcml2ZXJzL3BjaS9wMnBkbWEuYwo+IEBAIC0wLDAgKzEsNzYxIEBACj4gKy8vIFNQRFgt TGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4wCj4gKy8qCj4gKyAqIFBDSSBQZWVyIDIgUGVlciBE TUEgc3VwcG9ydC4KPiArICoKPiArICogQ29weXJpZ2h0IChjKSAyMDE2LTIwMTgsIExvZ2FuIEd1 bnRob3JwZQo+ICsgKiBDb3B5cmlnaHQgKGMpIDIwMTYtMjAxNywgTWljcm9zZW1pIENvcnBvcmF0 aW9uCj4gKyAqIENvcHlyaWdodCAoYykgMjAxNywgQ2hyaXN0b3BoIEhlbGx3aWcKPiArICogQ29w eXJpZ2h0IChjKSAyMDE4LCBFaWRldGljb20gSW5jLgo+ICsgKi8KPiArCj4gKyNkZWZpbmUgcHJf Zm10KGZtdCkgInBjaS1wMnBkbWE6ICIgZm10Cj4gKyNpbmNsdWRlIDxsaW51eC9wY2ktcDJwZG1h Lmg+Cj4gKyNpbmNsdWRlIDxsaW51eC9tb2R1bGUuaD4KPiArI2luY2x1ZGUgPGxpbnV4L3NsYWIu aD4KPiArI2luY2x1ZGUgPGxpbnV4L2dlbmFsbG9jLmg+Cj4gKyNpbmNsdWRlIDxsaW51eC9tZW1y ZW1hcC5oPgo+ICsjaW5jbHVkZSA8bGludXgvcGVyY3B1LXJlZmNvdW50Lmg+Cj4gKyNpbmNsdWRl IDxsaW51eC9yYW5kb20uaD4KPiArI2luY2x1ZGUgPGxpbnV4L3NlcV9idWYuaD4KPiArCj4gK3N0 cnVjdCBwY2lfcDJwZG1hIHsKPiArCXN0cnVjdCBwZXJjcHVfcmVmIGRldm1hcF9yZWY7Cj4gKwlz dHJ1Y3QgY29tcGxldGlvbiBkZXZtYXBfcmVmX2RvbmU7Cj4gKwlzdHJ1Y3QgZ2VuX3Bvb2wgKnBv b2w7Cj4gKwlib29sIHAycG1lbV9wdWJsaXNoZWQ7Cj4gK307Cj4gKwo+ICtzdGF0aWMgdm9pZCBw Y2lfcDJwZG1hX3BlcmNwdV9yZWxlYXNlKHN0cnVjdCBwZXJjcHVfcmVmICpyZWYpCj4gK3sKPiAr CXN0cnVjdCBwY2lfcDJwZG1hICpwMnAgPQo+ICsJCWNvbnRhaW5lcl9vZihyZWYsIHN0cnVjdCBw Y2lfcDJwZG1hLCBkZXZtYXBfcmVmKTsKPiArCj4gKwljb21wbGV0ZV9hbGwoJnAycC0+ZGV2bWFw X3JlZl9kb25lKTsKPiArfQo+ICsKPiArc3RhdGljIHZvaWQgcGNpX3AycGRtYV9wZXJjcHVfa2ls bCh2b2lkICpkYXRhKQo+ICt7Cj4gKwlzdHJ1Y3QgcGVyY3B1X3JlZiAqcmVmID0gZGF0YTsKPiAr Cj4gKwlpZiAocGVyY3B1X3JlZl9pc19keWluZyhyZWYpKQo+ICsJCXJldHVybjsKPiArCj4gKwlw ZXJjcHVfcmVmX2tpbGwocmVmKTsKPiArfQo+ICsKPiArc3RhdGljIHZvaWQgcGNpX3AycGRtYV9y ZWxlYXNlKHZvaWQgKmRhdGEpCj4gK3sKPiArCXN0cnVjdCBwY2lfZGV2ICpwZGV2ID0gZGF0YTsK PiArCj4gKwlpZiAoIXBkZXYtPnAycGRtYSkKPiArCQlyZXR1cm47Cj4gKwo+ICsJd2FpdF9mb3Jf Y29tcGxldGlvbigmcGRldi0+cDJwZG1hLT5kZXZtYXBfcmVmX2RvbmUpOwo+ICsJcGVyY3B1X3Jl Zl9leGl0KCZwZGV2LT5wMnBkbWEtPmRldm1hcF9yZWYpOwo+ICsKPiArCWdlbl9wb29sX2Rlc3Ry b3kocGRldi0+cDJwZG1hLT5wb29sKTsKPiArCXBkZXYtPnAycGRtYSA9IE5VTEw7Cj4gK30KPiAr Cj4gK3N0YXRpYyBpbnQgcGNpX3AycGRtYV9zZXR1cChzdHJ1Y3QgcGNpX2RldiAqcGRldikKPiAr ewo+ICsJaW50IGVycm9yID0gLUVOT01FTTsKPiArCXN0cnVjdCBwY2lfcDJwZG1hICpwMnA7Cj4g Kwo+ICsJcDJwID0gZGV2bV9remFsbG9jKCZwZGV2LT5kZXYsIHNpemVvZigqcDJwKSwgR0ZQX0tF Uk5FTCk7Cj4gKwlpZiAoIXAycCkKPiArCQlyZXR1cm4gLUVOT01FTTsKPiArCj4gKwlwMnAtPnBv b2wgPSBnZW5fcG9vbF9jcmVhdGUoUEFHRV9TSElGVCwgZGV2X3RvX25vZGUoJnBkZXYtPmRldikp Owo+ICsJaWYgKCFwMnAtPnBvb2wpCj4gKwkJZ290byBvdXQ7Cj4gKwo+ICsJaW5pdF9jb21wbGV0 aW9uKCZwMnAtPmRldm1hcF9yZWZfZG9uZSk7Cj4gKwllcnJvciA9IHBlcmNwdV9yZWZfaW5pdCgm cDJwLT5kZXZtYXBfcmVmLAo+ICsJCQlwY2lfcDJwZG1hX3BlcmNwdV9yZWxlYXNlLCAwLCBHRlBf S0VSTkVMKTsKPiArCWlmIChlcnJvcikKPiArCQlnb3RvIG91dF9wb29sX2Rlc3Ryb3k7Cj4gKwo+ ICsJcGVyY3B1X3JlZl9zd2l0Y2hfdG9fYXRvbWljX3N5bmMoJnAycC0+ZGV2bWFwX3JlZik7Cj4g Kwo+ICsJZXJyb3IgPSBkZXZtX2FkZF9hY3Rpb25fb3JfcmVzZXQoJnBkZXYtPmRldiwgcGNpX3Ay cGRtYV9yZWxlYXNlLCBwZGV2KTsKPiArCWlmIChlcnJvcikKPiArCQlnb3RvIG91dF9wb29sX2Rl c3Ryb3k7Cj4gKwo+ICsJcGRldi0+cDJwZG1hID0gcDJwOwo+ICsKPiArCXJldHVybiAwOwo+ICsK PiArb3V0X3Bvb2xfZGVzdHJveToKPiArCWdlbl9wb29sX2Rlc3Ryb3kocDJwLT5wb29sKTsKPiAr b3V0Ogo+ICsJZGV2bV9rZnJlZSgmcGRldi0+ZGV2LCBwMnApOwo+ICsJcmV0dXJuIGVycm9yOwo+ ICt9Cj4gKwo+ICsvKioKPiArICogcGNpX3AycGRtYV9hZGRfcmVzb3VyY2UgLSBhZGQgbWVtb3J5 IGZvciB1c2UgYXMgcDJwIG1lbW9yeQo+ICsgKiBAcGRldjogdGhlIGRldmljZSB0byBhZGQgdGhl IG1lbW9yeSB0bwo+ICsgKiBAYmFyOiBQQ0kgQkFSIHRvIGFkZAo+ICsgKiBAc2l6ZTogc2l6ZSBv ZiB0aGUgbWVtb3J5IHRvIGFkZCwgbWF5IGJlIHplcm8gdG8gdXNlIHRoZSB3aG9sZSBCQVIKPiAr ICogQG9mZnNldDogb2Zmc2V0IGludG8gdGhlIFBDSSBCQVIKPiArICoKPiArICogVGhlIG1lbW9y eSB3aWxsIGJlIGdpdmVuIFpPTkVfREVWSUNFIHN0cnVjdCBwYWdlcyBzbyB0aGF0IGl0IG1heQo+ ICsgKiBiZSB1c2VkIHdpdGggYW55IERNQSByZXF1ZXN0Lgo+ICsgKi8KPiAraW50IHBjaV9wMnBk bWFfYWRkX3Jlc291cmNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBpbnQgYmFyLCBzaXplX3Qgc2l6 ZSwKPiArCQkJICAgIHU2NCBvZmZzZXQpCj4gK3sKPiArCXN0cnVjdCBkZXZfcGFnZW1hcCAqcGdt YXA7Cj4gKwl2b2lkICphZGRyOwo+ICsJaW50IGVycm9yOwo+ICsKPiArCWlmICghKHBjaV9yZXNv dXJjZV9mbGFncyhwZGV2LCBiYXIpICYgSU9SRVNPVVJDRV9NRU0pKQo+ICsJCXJldHVybiAtRUlO VkFMOwo+ICsKPiArCWlmIChvZmZzZXQgPj0gcGNpX3Jlc291cmNlX2xlbihwZGV2LCBiYXIpKQo+ ICsJCXJldHVybiAtRUlOVkFMOwo+ICsKPiArCWlmICghc2l6ZSkKPiArCQlzaXplID0gcGNpX3Jl c291cmNlX2xlbihwZGV2LCBiYXIpIC0gb2Zmc2V0Owo+ICsKPiArCWlmIChzaXplICsgb2Zmc2V0 ID4gcGNpX3Jlc291cmNlX2xlbihwZGV2LCBiYXIpKQo+ICsJCXJldHVybiAtRUlOVkFMOwo+ICsK PiArCWlmICghcGRldi0+cDJwZG1hKSB7Cj4gKwkJZXJyb3IgPSBwY2lfcDJwZG1hX3NldHVwKHBk ZXYpOwo+ICsJCWlmIChlcnJvcikKPiArCQkJcmV0dXJuIGVycm9yOwo+ICsJfQo+ICsKPiArCXBn bWFwID0gZGV2bV9remFsbG9jKCZwZGV2LT5kZXYsIHNpemVvZigqcGdtYXApLCBHRlBfS0VSTkVM KTsKPiArCWlmICghcGdtYXApCj4gKwkJcmV0dXJuIC1FTk9NRU07Cj4gKwo+ICsJcGdtYXAtPnJl cy5zdGFydCA9IHBjaV9yZXNvdXJjZV9zdGFydChwZGV2LCBiYXIpICsgb2Zmc2V0Owo+ICsJcGdt YXAtPnJlcy5lbmQgPSBwZ21hcC0+cmVzLnN0YXJ0ICsgc2l6ZSAtIDE7Cj4gKwlwZ21hcC0+cmVz LmZsYWdzID0gcGNpX3Jlc291cmNlX2ZsYWdzKHBkZXYsIGJhcik7Cj4gKwlwZ21hcC0+cmVmID0g JnBkZXYtPnAycGRtYS0+ZGV2bWFwX3JlZjsKPiArCXBnbWFwLT50eXBlID0gTUVNT1JZX0RFVklD RV9QQ0lfUDJQRE1BOwo+ICsKPiArCWFkZHIgPSBkZXZtX21lbXJlbWFwX3BhZ2VzKCZwZGV2LT5k ZXYsIHBnbWFwKTsKPiArCWlmIChJU19FUlIoYWRkcikpIHsKPiArCQllcnJvciA9IFBUUl9FUlIo YWRkcik7Cj4gKwkJZ290byBwZ21hcF9mcmVlOwo+ICsJfQo+ICsKPiArCWVycm9yID0gZ2VuX3Bv b2xfYWRkX3ZpcnQocGRldi0+cDJwZG1hLT5wb29sLCAodW5zaWduZWQgbG9uZylhZGRyLAo+ICsJ CQlwY2lfYnVzX2FkZHJlc3MocGRldiwgYmFyKSArIG9mZnNldCwKPiArCQkJcmVzb3VyY2Vfc2l6 ZSgmcGdtYXAtPnJlcyksIGRldl90b19ub2RlKCZwZGV2LT5kZXYpKTsKPiArCWlmIChlcnJvcikK PiArCQlnb3RvIHBnbWFwX2ZyZWU7Cj4gKwo+ICsJZXJyb3IgPSBkZXZtX2FkZF9hY3Rpb25fb3Jf cmVzZXQoJnBkZXYtPmRldiwgcGNpX3AycGRtYV9wZXJjcHVfa2lsbCwKPiArCQkJCQkgICZwZGV2 LT5wMnBkbWEtPmRldm1hcF9yZWYpOwo+ICsJaWYgKGVycm9yKQo+ICsJCWdvdG8gcGdtYXBfZnJl ZTsKPiArCj4gKwlwY2lfaW5mbyhwZGV2LCAiYWRkZWQgcGVlci10by1wZWVyIERNQSBtZW1vcnkg JXBSXG4iLAo+ICsJCSAmcGdtYXAtPnJlcyk7Cj4gKwo+ICsJcmV0dXJuIDA7Cj4gKwo+ICtwZ21h cF9mcmVlOgo+ICsJZGV2cmVzX2ZyZWUocGdtYXApOwo+ICsJcmV0dXJuIGVycm9yOwo+ICt9Cj4g K0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9wMnBkbWFfYWRkX3Jlc291cmNlKTsKPiArCj4gK3N0YXRp YyBzdHJ1Y3QgcGNpX2RldiAqZmluZF9wYXJlbnRfcGNpX2RldihzdHJ1Y3QgZGV2aWNlICpkZXYp Cj4gK3sKPiArCXN0cnVjdCBkZXZpY2UgKnBhcmVudDsKPiArCj4gKwlkZXYgPSBnZXRfZGV2aWNl KGRldik7Cj4gKwo+ICsJd2hpbGUgKGRldikgewo+ICsJCWlmIChkZXZfaXNfcGNpKGRldikpCj4g KwkJCXJldHVybiB0b19wY2lfZGV2KGRldik7Cj4gKwo+ICsJCXBhcmVudCA9IGdldF9kZXZpY2Uo ZGV2LT5wYXJlbnQpOwo+ICsJCXB1dF9kZXZpY2UoZGV2KTsKPiArCQlkZXYgPSBwYXJlbnQ7Cj4g Kwl9Cj4gKwo+ICsJcmV0dXJuIE5VTEw7Cj4gK30KPiArCj4gKy8qCj4gKyAqIENoZWNrIGlmIGEg UENJIGJyaWRnZSBoYXMgaXQncyBBQ1MgcmVkaXJlY3Rpb24gYml0cyBzZXQgdG8gcmVkaXJlY3Qg UDJQCj4gKyAqIFRMUHMgdXBzdHJlYW0gdmlhIEFDUy4gUmV0dXJucyAxIGlmIHRoZSBwYWNrZXRz IHdpbGwgYmUgcmVkaXJlY3RlZAo+ICsgKiB1cHN0cmVhbSwgMCBvdGhlcndpc2UuCj4gKyAqLwo+ ICtzdGF0aWMgaW50IHBjaV9icmlkZ2VfaGFzX2Fjc19yZWRpcihzdHJ1Y3QgcGNpX2RldiAqZGV2 KQo+ICt7Cj4gKwlpbnQgcG9zOwo+ICsJdTE2IGN0cmw7Cj4gKwo+ICsJcG9zID0gcGNpX2ZpbmRf ZXh0X2NhcGFiaWxpdHkoZGV2LCBQQ0lfRVhUX0NBUF9JRF9BQ1MpOwo+ICsJaWYgKCFwb3MpCj4g KwkJcmV0dXJuIDA7Cj4gKwo+ICsJcGNpX3JlYWRfY29uZmlnX3dvcmQoZGV2LCBwb3MgKyBQQ0lf QUNTX0NUUkwsICZjdHJsKTsKPiArCj4gKwlpZiAoY3RybCAmIChQQ0lfQUNTX1JSIHwgUENJX0FD U19DUiB8IFBDSV9BQ1NfRUMpKQo+ICsJCXJldHVybiAxOwo+ICsKPiArCXJldHVybiAwOwo+ICt9 Cj4gKwo+ICtzdGF0aWMgdm9pZCBzZXFfYnVmX3ByaW50X2J1c19kZXZmbihzdHJ1Y3Qgc2VxX2J1 ZiAqYnVmLCBzdHJ1Y3QgcGNpX2RldiAqZGV2KQo+ICt7Cj4gKwlpZiAoIWJ1ZikKPiArCQlyZXR1 cm47Cj4gKwo+ICsJc2VxX2J1Zl9wcmludGYoYnVmLCAiJTA0eDolMDJ4OiUwMnguJXg7IiwgcGNp X2RvbWFpbl9ucihkZXYtPmJ1cyksCj4gKwkJICAgICAgIGRldi0+YnVzLT5udW1iZXIsIFBDSV9T TE9UKGRldi0+ZGV2Zm4pLAo+ICsJCSAgICAgICBQQ0lfRlVOQyhkZXYtPmRldmZuKSk7Cj4gK30K PiArCj4gKy8qCj4gKyAqIEZpbmQgdGhlIGRpc3RhbmNlIHRocm91Z2ggdGhlIG5lYXJlc3QgY29t bW9uIHVwc3RyZWFtIGJyaWRnZSBiZXR3ZWVuCj4gKyAqIHR3byBQQ0kgZGV2aWNlcy4KPiArICoK PiArICogSWYgdGhlIHR3byBkZXZpY2VzIGFyZSB0aGUgc2FtZSBkZXZpY2UgdGhlbiAwIHdpbGwg YmUgcmV0dXJuZWQuCj4gKyAqCj4gKyAqIElmIHRoZXJlIGFyZSB0d28gdmlydHVhbCBmdW5jdGlv bnMgb2YgdGhlIHNhbWUgZGV2aWNlIGJlaGluZCB0aGUgc2FtZQo+ICsgKiBicmlkZ2UgcG9ydCB0 aGVuIDIgd2lsbCBiZSByZXR1cm5lZCAob25lIHN0ZXAgZG93biB0byB0aGUgUENJZSBzd2l0Y2gs Cj4gKyAqIHRoZW4gb25lIHN0ZXAgYmFjayB0byB0aGUgc2FtZSBkZXZpY2UpLgo+ICsgKgo+ICsg KiBJbiB0aGUgY2FzZSB3aGVyZSB0d28gZGV2aWNlcyBhcmUgY29ubmVjdGVkIHRvIHRoZSBzYW1l IFBDSWUgc3dpdGNoLCB0aGUKPiArICogdmFsdWUgNCB3aWxsIGJlIHJldHVybmVkLiBUaGlzIGNv cnJlc3BvbmRzIHRvIHRoZSBmb2xsb3dpbmcgUENJIHRyZWU6Cj4gKyAqCj4gKyAqICAgICAtKyAg Um9vdCBQb3J0Cj4gKyAqICAgICAgXCsgU3dpdGNoIFVwc3RyZWFtIFBvcnQKPiArICogICAgICAg Ky0rIFN3aXRjaCBEb3duc3RyZWFtIFBvcnQKPiArICogICAgICAgKyBcLSBEZXZpY2UgQQo+ICsg KiAgICAgICBcLSsgU3dpdGNoIERvd25zdHJlYW0gUG9ydAo+ICsgKiAgICAgICAgIFwtIERldmlj ZSBCCj4gKyAqCj4gKyAqIFRoZSBkaXN0YW5jZSBpcyA0IGJlY2F1c2Ugd2UgdHJhdmVyc2UgZnJv bSBEZXZpY2UgQSB0aHJvdWdoIHRoZSBkb3duc3RyZWFtCj4gKyAqIHBvcnQgb2YgdGhlIHN3aXRj aCwgdG8gdGhlIGNvbW1vbiB1cHN0cmVhbSBwb3J0LCBiYWNrIHVwIHRvIHRoZSBzZWNvbmQKPiAr ICogZG93bnN0cmVhbSBwb3J0IGFuZCB0aGVuIHRvIERldmljZSBCLgo+ICsgKgo+ICsgKiBBbnkg dHdvIGRldmljZXMgdGhhdCBkb24ndCBoYXZlIGEgY29tbW9uIHVwc3RyZWFtIGJyaWRnZSB3aWxs IHJldHVybiAtMS4KPiArICogSW4gdGhpcyB3YXkgZGV2aWNlcyBvbiBzZXBhcmF0ZSBQQ0llIHJv b3QgcG9ydHMgd2lsbCBiZSByZWplY3RlZCwgd2hpY2gKPiArICogaXMgd2hhdCB3ZSB3YW50IGZv ciBwZWVyLXRvLXBlZXIgc2VlaW5nIGVhY2ggUENJZSByb290IHBvcnQgZGVmaW5lcyBhCj4gKyAq IHNlcGFyYXRlIGhpZXJhcmNoeSBkb21haW4gYW5kIHRoZXJlJ3Mgbm8gd2F5IHRvIGRldGVybWlu ZSB3aGV0aGVyIHRoZSByb290Cj4gKyAqIGNvbXBsZXggc3VwcG9ydHMgZm9yd2FyZGluZyBiZXR3 ZWVuIHRoZW0uCj4gKyAqCj4gKyAqIEluIHRoZSBjYXNlIHdoZXJlIHR3byBkZXZpY2VzIGFyZSBj b25uZWN0ZWQgdG8gZGlmZmVyZW50IFBDSWUgc3dpdGNoZXMsCj4gKyAqIHRoaXMgZnVuY3Rpb24g d2lsbCBzdGlsbCByZXR1cm4gYSBwb3NpdGl2ZSBkaXN0YW5jZSBhcyBsb25nIGFzIGJvdGgKPiAr ICogc3dpdGNoZXMgZXZlbnV0YWxseSBoYXZlIGEgY29tbW9uIHVwc3RyZWFtIGJyaWRnZS4gTm90 ZSB0aGlzIGNvdmVycwo+ICsgKiB0aGUgY2FzZSBvZiB1c2luZyBtdWx0aXBsZSBQQ0llIHN3aXRj aGVzIHRvIGFjaGlldmUgYSBkZXNpcmVkIGxldmVsIG9mCj4gKyAqIGZhbi1vdXQgZnJvbSBhIHJv b3QgcG9ydC4gVGhlIGV4YWN0IGRpc3RhbmNlIHdpbGwgYmUgYSBmdW5jdGlvbiBvZiB0aGUKPiAr ICogbnVtYmVyIG9mIHN3aXRjaGVzIGJldHdlZW4gRGV2aWNlIEEgYW5kIERldmljZSBCLgo+ICsg Kgo+ICsgKiBJZiBhIGJyaWRnZSB3aGljaCBoYXMgYW55IEFDUyByZWRpcmVjdGlvbiBiaXRzIHNl dCBpcyBpbiB0aGUgcGF0aAo+ICsgKiB0aGVuIHRoaXMgZnVuY3Rpb25zIHdpbGwgcmV0dXJuIC0y LiBUaGlzIGlzIHNvIHdlIHJlamVjdCBhbnkKPiArICogY2FzZXMgd2hlcmUgdGhlIFRMUHMgYXJl IGZvcndhcmRlZCB1cCBpbnRvIHRoZSByb290IGNvbXBsZXguCj4gKyAqIEluIHRoaXMgY2FzZSwg YSBsaXN0IG9mIGFsbCBpbmZyaW5naW5nIGJyaWRnZSBhZGRyZXNzZXMgd2lsbCBiZQo+ICsgKiBw b3B1bGF0ZWQgaW4gYWNzX2xpc3QgKGFzc3VtaW5nIGl0J3Mgbm9uLW51bGwpIGZvciBwcmludGsg cHVycG9zZXMuCj4gKyAqLwo+ICtzdGF0aWMgaW50IHVwc3RyZWFtX2JyaWRnZV9kaXN0YW5jZShz dHJ1Y3QgcGNpX2RldiAqYSwKPiArCQkJCSAgICBzdHJ1Y3QgcGNpX2RldiAqYiwKPiArCQkJCSAg ICBzdHJ1Y3Qgc2VxX2J1ZiAqYWNzX2xpc3QpCj4gK3sKPiArCWludCBkaXN0X2EgPSAwOwo+ICsJ aW50IGRpc3RfYiA9IDA7Cj4gKwlzdHJ1Y3QgcGNpX2RldiAqYmIgPSBOVUxMOwo+ICsJaW50IGFj c19jbnQgPSAwOwo+ICsKPiArCS8qCj4gKwkgKiBOb3RlLCB3ZSBkb24ndCBuZWVkIHRvIHRha2Ug cmVmZXJlbmNlcyB0byBkZXZpY2VzIHJldHVybmVkIGJ5Cj4gKwkgKiBwY2lfdXBzdHJlYW1fYnJp ZGdlKCkgc2VlaW5nIHdlIGhvbGQgYSByZWZlcmVuY2UgdG8gYSBjaGlsZAo+ICsJICogZGV2aWNl IHdoaWNoIHdpbGwgYWxyZWFkeSBob2xkIGEgcmVmZXJlbmNlIHRvIHRoZSB1cHN0cmVhbSBicmlk Z2UuCj4gKwkgKi8KPiArCj4gKwl3aGlsZSAoYSkgewo+ICsJCWRpc3RfYiA9IDA7Cj4gKwo+ICsJ CWlmIChwY2lfYnJpZGdlX2hhc19hY3NfcmVkaXIoYSkpIHsKPiArCQkJc2VxX2J1Zl9wcmludF9i dXNfZGV2Zm4oYWNzX2xpc3QsIGEpOwo+ICsJCQlhY3NfY250Kys7Cj4gKwkJfQo+ICsKPiArCQli YiA9IGI7Cj4gKwo+ICsJCXdoaWxlIChiYikgewo+ICsJCQlpZiAoYSA9PSBiYikKPiArCQkJCWdv dG8gY2hlY2tfYl9wYXRoX2FjczsKPiArCj4gKwkJCWJiID0gcGNpX3Vwc3RyZWFtX2JyaWRnZShi Yik7Cj4gKwkJCWRpc3RfYisrOwo+ICsJCX0KPiArCj4gKwkJYSA9IHBjaV91cHN0cmVhbV9icmlk Z2UoYSk7Cj4gKwkJZGlzdF9hKys7Cj4gKwl9Cj4gKwo+ICsJcmV0dXJuIC0xOwo+ICsKPiArY2hl Y2tfYl9wYXRoX2FjczoKPiArCWJiID0gYjsKPiArCj4gKwl3aGlsZSAoYmIpIHsKPiArCQlpZiAo YSA9PSBiYikKPiArCQkJYnJlYWs7Cj4gKwo+ICsJCWlmIChwY2lfYnJpZGdlX2hhc19hY3NfcmVk aXIoYmIpKSB7Cj4gKwkJCXNlcV9idWZfcHJpbnRfYnVzX2RldmZuKGFjc19saXN0LCBiYik7Cj4g KwkJCWFjc19jbnQrKzsKPiArCQl9Cj4gKwo+ICsJCWJiID0gcGNpX3Vwc3RyZWFtX2JyaWRnZShi Yik7Cj4gKwl9Cj4gKwo+ICsJaWYgKGFjc19jbnQpCj4gKwkJcmV0dXJuIC0yOwo+ICsKPiArCXJl dHVybiBkaXN0X2EgKyBkaXN0X2I7Cj4gK30KPiArCj4gK3N0YXRpYyBpbnQgdXBzdHJlYW1fYnJp ZGdlX2Rpc3RhbmNlX3dhcm4oc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyLAo+ICsJCQkJCSBzdHJ1 Y3QgcGNpX2RldiAqY2xpZW50KQo+ICt7Cj4gKwlzdHJ1Y3Qgc2VxX2J1ZiBhY3NfbGlzdDsKPiAr CWludCByZXQ7Cj4gKwo+ICsJc2VxX2J1Zl9pbml0KCZhY3NfbGlzdCwga21hbGxvYyhQQUdFX1NJ WkUsIEdGUF9LRVJORUwpLCBQQUdFX1NJWkUpOwo+ICsKPiArCXJldCA9IHVwc3RyZWFtX2JyaWRn ZV9kaXN0YW5jZShwcm92aWRlciwgY2xpZW50LCAmYWNzX2xpc3QpOwo+ICsJaWYgKHJldCA9PSAt Mikgewo+ICsJCXBjaV93YXJuKGNsaWVudCwgImNhbm5vdCBiZSB1c2VkIGZvciBwZWVyLXRvLXBl ZXIgRE1BIGFzIEFDUyByZWRpcmVjdCBpcyBzZXQgYmV0d2VlbiB0aGUgY2xpZW50IGFuZCBwcm92 aWRlclxuIik7Cj4gKwkJLyogRHJvcCBmaW5hbCBzZW1pY29sb24gKi8KPiArCQlhY3NfbGlzdC5i dWZmZXJbYWNzX2xpc3QubGVuLTFdID0gMDsKPiArCQlwY2lfd2FybihjbGllbnQsICJ0byBkaXNh YmxlIEFDUyByZWRpcmVjdCBmb3IgdGhpcyBwYXRoLCBhZGQgdGhlIGtlcm5lbCBwYXJhbWV0ZXI6 IHBjaT1kaXNhYmxlX2Fjc19yZWRpcj0lc1xuIiwKPiArCQkJIGFjc19saXN0LmJ1ZmZlcik7Cj4g Kwo+ICsJfSBlbHNlIGlmIChyZXQgPCAwKSB7Cj4gKwkJcGNpX3dhcm4oY2xpZW50LCAiY2Fubm90 IGJlIHVzZWQgZm9yIHBlZXItdG8tcGVlciBETUEgYXMgdGhlIGNsaWVudCBhbmQgcHJvdmlkZXIg ZG8gbm90IHNoYXJlIGFuIHVwc3RyZWFtIGJyaWRnZVxuIik7Cj4gKwl9Cj4gKwo+ICsJa2ZyZWUo YWNzX2xpc3QuYnVmZmVyKTsKPiArCj4gKwlyZXR1cm4gcmV0Owo+ICt9Cj4gKwo+ICtzdHJ1Y3Qg cGNpX3AycGRtYV9jbGllbnQgewo+ICsJc3RydWN0IGxpc3RfaGVhZCBsaXN0Owo+ICsJc3RydWN0 IHBjaV9kZXYgKmNsaWVudDsKPiArCXN0cnVjdCBwY2lfZGV2ICpwcm92aWRlcjsKPiArfTsKPiAr Cj4gKy8qKgo+ICsgKiBwY2lfcDJwZG1hX2FkZF9jbGllbnQgLSBhbGxvY2F0ZSBhIG5ldyBlbGVt ZW50IGluIGEgY2xpZW50IGRldmljZSBsaXN0Cj4gKyAqIEBoZWFkOiBsaXN0IGhlYWQgb2YgcDJw ZG1hIGNsaWVudHMKPiArICogQGRldjogZGV2aWNlIHRvIGFkZCB0byB0aGUgbGlzdAo+ICsgKgo+ ICsgKiBUaGlzIGFkZHMgQGRldiB0byBhIGxpc3Qgb2YgY2xpZW50cyB1c2VkIGJ5IGEgcDJwZG1h IGRldmljZS4KPiArICogVGhpcyBsaXN0IHNob3VsZCBiZSBwYXNzZWQgdG8gcGNpX3AycG1lbV9m aW5kKCkuIE9uY2UgcGNpX3AycG1lbV9maW5kKCkgaGFzCj4gKyAqIGJlZW4gY2FsbGVkIHN1Y2Nl c3NmdWxseSwgdGhlIGxpc3Qgd2lsbCBiZSBib3VuZCB0byBhIHNwZWNpZmljIHAycGRtYQo+ICsg KiBkZXZpY2UgYW5kIG5ldyBjbGllbnRzIGNhbiBvbmx5IGJlIGFkZGVkIHRvIHRoZSBsaXN0IGlm IHRoZXkgYXJlCj4gKyAqIHN1cHBvcnRlZCBieSB0aGF0IHAycGRtYSBkZXZpY2UuCj4gKyAqCj4g KyAqIFRoZSBjYWxsZXIgaXMgZXhwZWN0ZWQgdG8gaGF2ZSBhIGxvY2sgd2hpY2ggcHJvdGVjdHMg QGhlYWQgYXMgbmVjZXNzYXJ5Cj4gKyAqIHNvIHRoYXQgbm9uZSBvZiB0aGUgcGNpX3AycCBmdW5j dGlvbnMgY2FuIGJlIGNhbGxlZCBjb25jdXJyZW50bHkKPiArICogb24gdGhhdCBsaXN0Lgo+ICsg Kgo+ICsgKiBSZXR1cm5zIDAgaWYgdGhlIGNsaWVudCB3YXMgc3VjY2Vzc2Z1bGx5IGFkZGVkLgo+ ICsgKi8KPiAraW50IHBjaV9wMnBkbWFfYWRkX2NsaWVudChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFk LCBzdHJ1Y3QgZGV2aWNlICpkZXYpCj4gK3sKPiArCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAq aXRlbSwgKm5ld19pdGVtOwo+ICsJc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyID0gTlVMTDsKPiAr CXN0cnVjdCBwY2lfZGV2ICpjbGllbnQ7Cj4gKwlpbnQgcmV0Owo+ICsKPiArCWlmIChJU19FTkFC TEVEKENPTkZJR19ETUFfVklSVF9PUFMpICYmIGRldi0+ZG1hX29wcyA9PSAmZG1hX3ZpcnRfb3Bz KSB7Cj4gKwkJZGV2X3dhcm4oZGV2LCAiY2Fubm90IGJlIHVzZWQgZm9yIHBlZXItdG8tcGVlciBE TUEgYmVjYXVzZSB0aGUgZHJpdmVyIG1ha2VzIHVzZSBvZiBkbWFfdmlydF9vcHNcbiIpOwo+ICsJ CXJldHVybiAtRU5PREVWOwo+ICsJfQo+ICsKPiArCWNsaWVudCA9IGZpbmRfcGFyZW50X3BjaV9k ZXYoZGV2KTsKPiArCWlmICghY2xpZW50KSB7Cj4gKwkJZGV2X3dhcm4oZGV2LCAiY2Fubm90IGJl IHVzZWQgZm9yIHBlZXItdG8tcGVlciBETUEgYXMgaXQgaXMgbm90IGEgUENJIGRldmljZVxuIik7 Cj4gKwkJcmV0dXJuIC1FTk9ERVY7Cj4gKwl9Cj4gKwo+ICsJaXRlbSA9IGxpc3RfZmlyc3RfZW50 cnlfb3JfbnVsbChoZWFkLCBzdHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQsIGxpc3QpOwo+ICsJaWYg KGl0ZW0gJiYgaXRlbS0+cHJvdmlkZXIpIHsKPiArCQlwcm92aWRlciA9IGl0ZW0tPnByb3ZpZGVy Owo+ICsKPiArCQlyZXQgPSB1cHN0cmVhbV9icmlkZ2VfZGlzdGFuY2Vfd2Fybihwcm92aWRlciwg Y2xpZW50KTsKPiArCQlpZiAocmV0IDwgMCkgewo+ICsJCQlyZXQgPSAtRVhERVY7Cj4gKwkJCWdv dG8gcHV0X2NsaWVudDsKPiArCQl9Cj4gKwl9Cj4gKwo+ICsJbmV3X2l0ZW0gPSBremFsbG9jKHNp emVvZigqbmV3X2l0ZW0pLCBHRlBfS0VSTkVMKTsKPiArCWlmICghbmV3X2l0ZW0pIHsKPiArCQly ZXQgPSAtRU5PTUVNOwo+ICsJCWdvdG8gcHV0X2NsaWVudDsKPiArCX0KPiArCj4gKwluZXdfaXRl bS0+Y2xpZW50ID0gY2xpZW50Owo+ICsJbmV3X2l0ZW0tPnByb3ZpZGVyID0gcGNpX2Rldl9nZXQo cHJvdmlkZXIpOwo+ICsKPiArCWxpc3RfYWRkX3RhaWwoJm5ld19pdGVtLT5saXN0LCBoZWFkKTsK PiArCj4gKwlyZXR1cm4gMDsKPiArCj4gK3B1dF9jbGllbnQ6Cj4gKwlwY2lfZGV2X3B1dChjbGll bnQpOwo+ICsJcmV0dXJuIHJldDsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lfcDJwZG1h X2FkZF9jbGllbnQpOwo+ICsKPiArc3RhdGljIHZvaWQgcGNpX3AycGRtYV9jbGllbnRfZnJlZShz dHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQgKml0ZW0pCj4gK3sKPiArCWxpc3RfZGVsKCZpdGVtLT5s aXN0KTsKPiArCXBjaV9kZXZfcHV0KGl0ZW0tPmNsaWVudCk7Cj4gKwlwY2lfZGV2X3B1dChpdGVt LT5wcm92aWRlcik7Cj4gKwlrZnJlZShpdGVtKTsKPiArfQo+ICsKPiArLyoqCj4gKyAqIHBjaV9w MnBkbWFfcmVtb3ZlX2NsaWVudCAtIHJlbW92ZSBhbmQgZnJlZSBhIHAycGRtYSBjbGllbnQKPiAr ICogQGhlYWQ6IGxpc3QgaGVhZCBvZiBwMnBkbWEgY2xpZW50cwo+ICsgKiBAZGV2OiBkZXZpY2Ug dG8gcmVtb3ZlIGZyb20gdGhlIGxpc3QKPiArICoKPiArICogVGhpcyByZW1vdmVzIEBkZXYgZnJv bSBhIGxpc3Qgb2YgY2xpZW50cyB1c2VkIGJ5IGEgcDJwZG1hIGRldmljZS4KPiArICogVGhlIGNh bGxlciBpcyBleHBlY3RlZCB0byBoYXZlIGEgbG9jayB3aGljaCBwcm90ZWN0cyBAaGVhZCBhcyBu ZWNlc3NhcnkKPiArICogc28gdGhhdCBub25lIG9mIHRoZSBwY2lfcDJwIGZ1bmN0aW9ucyBjYW4g YmUgY2FsbGVkIGNvbmN1cnJlbnRseQo+ICsgKiBvbiB0aGF0IGxpc3QuCj4gKyAqLwo+ICt2b2lk IHBjaV9wMnBkbWFfcmVtb3ZlX2NsaWVudChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkLCBzdHJ1Y3Qg ZGV2aWNlICpkZXYpCj4gK3sKPiArCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAqcG9zLCAqdG1w Owo+ICsJc3RydWN0IHBjaV9kZXYgKnBkZXY7Cj4gKwo+ICsJcGRldiA9IGZpbmRfcGFyZW50X3Bj aV9kZXYoZGV2KTsKPiArCWlmICghcGRldikKPiArCQlyZXR1cm47Cj4gKwo+ICsJbGlzdF9mb3Jf ZWFjaF9lbnRyeV9zYWZlKHBvcywgdG1wLCBoZWFkLCBsaXN0KSB7Cj4gKwkJaWYgKHBvcy0+Y2xp ZW50ICE9IHBkZXYpCj4gKwkJCWNvbnRpbnVlOwo+ICsKPiArCQlwY2lfcDJwZG1hX2NsaWVudF9m cmVlKHBvcyk7Cj4gKwl9Cj4gKwo+ICsJcGNpX2Rldl9wdXQocGRldik7Cj4gK30KPiArRVhQT1JU X1NZTUJPTF9HUEwocGNpX3AycGRtYV9yZW1vdmVfY2xpZW50KTsKPiArCj4gKy8qKgo+ICsgKiBw Y2lfcDJwZG1hX2NsaWVudF9saXN0X2ZyZWUgLSBmcmVlIGFuIGVudGlyZSBsaXN0IG9mIHAycGRt YSBjbGllbnRzCj4gKyAqIEBoZWFkOiBsaXN0IGhlYWQgb2YgcDJwZG1hIGNsaWVudHMKPiArICoK PiArICogVGhpcyByZW1vdmVzIGFsbCBkZXZpY2VzIGluIGEgbGlzdCBvZiBjbGllbnRzIHVzZWQg YnkgYSBwMnBkbWEgZGV2aWNlLgo+ICsgKiBUaGUgY2FsbGVyIGlzIGV4cGVjdGVkIHRvIGhhdmUg YSBsb2NrIHdoaWNoIHByb3RlY3RzIEBoZWFkIGFzIG5lY2Vzc2FyeQo+ICsgKiBzbyB0aGF0IG5v bmUgb2YgdGhlIHBjaV9wMnBkbWEgZnVuY3Rpb25zIGNhbiBiZSBjYWxsZWQgY29uY3VycmVudGx5 Cj4gKyAqIG9uIHRoYXQgbGlzdC4KPiArICovCj4gK3ZvaWQgcGNpX3AycGRtYV9jbGllbnRfbGlz dF9mcmVlKHN0cnVjdCBsaXN0X2hlYWQgKmhlYWQpCj4gK3sKPiArCXN0cnVjdCBwY2lfcDJwZG1h X2NsaWVudCAqcG9zLCAqdG1wOwo+ICsKPiArCWxpc3RfZm9yX2VhY2hfZW50cnlfc2FmZShwb3Ms IHRtcCwgaGVhZCwgbGlzdCkKPiArCQlwY2lfcDJwZG1hX2NsaWVudF9mcmVlKHBvcyk7Cj4gK30K PiArRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycGRtYV9jbGllbnRfbGlzdF9mcmVlKTsKPiArCj4g Ky8qKgo+ICsgKiBwY2lfcDJwZG1hX2Rpc3RhbmNlIC0gRGV0ZXJtaXZlIHRoZSBjdW11bGF0aXZl IGRpc3RhbmNlIGJldHdlZW4KPiArICoJYSBwMnBkbWEgcHJvdmlkZXIgYW5kIHRoZSBjbGllbnRz IGluIHVzZS4KPiArICogQHByb3ZpZGVyOiBwMnBkbWEgcHJvdmlkZXIgdG8gY2hlY2sgYWdhaW5z dCB0aGUgY2xpZW50IGxpc3QKPiArICogQGNsaWVudHM6IGxpc3Qgb2YgZGV2aWNlcyB0byBjaGVj ayAoTlVMTC10ZXJtaW5hdGVkKQo+ICsgKiBAdmVyYm9zZTogaWYgdHJ1ZSwgcHJpbnQgd2Fybmlu Z3MgZm9yIGRldmljZXMgd2hlbiB3ZSByZXR1cm4gLTEKPiArICoKPiArICogUmV0dXJucyAtMSBp ZiBhbnkgb2YgdGhlIGNsaWVudHMgYXJlIG5vdCBjb21wYXRpYmxlIChiZWhpbmQgdGhlIHNhbWUK PiArICogcm9vdCBwb3J0IGFzIHRoZSBwcm92aWRlciksIG90aGVyd2lzZSByZXR1cm5zIGEgcG9z aXRpdmUgbnVtYmVyIHdoZXJlCj4gKyAqIHRoZSBsb3dlciBudW1iZXIgaXMgdGhlIHByZWZlcnJh YmxlIGNob2ljZS4gKElmIHRoZXJlJ3Mgb25lIGNsaWVudAo+ICsgKiB0aGF0J3MgdGhlIHNhbWUg YXMgdGhlIHByb3ZpZGVyIGl0IHdpbGwgcmV0dXJuIDAsIHdoaWNoIGlzIGJlc3QgY2hvaWNlKS4K PiArICoKPiArICogRm9yIG5vdywgImNvbXBhdGlibGUiIG1lYW5zIHRoZSBwcm92aWRlciBhbmQg dGhlIGNsaWVudHMgYXJlIGFsbCBiZWhpbmQKPiArICogdGhlIHNhbWUgUENJIHJvb3QgcG9ydC4g VGhpcyBjdXRzIG91dCBjYXNlcyB0aGF0IG1heSB3b3JrIGJ1dCBpcyBzYWZlc3QKPiArICogZm9y IHRoZSB1c2VyLiBGdXR1cmUgd29yayBjYW4gZXhwYW5kIHRoaXMgdG8gd2hpdGUtbGlzdCByb290 IGNvbXBsZXhlcyB0aGF0Cj4gKyAqIGNhbiBzYWZlbHkgZm9yd2FyZCBiZXR3ZWVuIGVhY2ggcG9y dHMuCj4gKyAqLwo+ICtpbnQgcGNpX3AycGRtYV9kaXN0YW5jZShzdHJ1Y3QgcGNpX2RldiAqcHJv dmlkZXIsIHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMsCj4gKwkJCWJvb2wgdmVyYm9zZSkKPiAr ewo+ICsJc3RydWN0IHBjaV9wMnBkbWFfY2xpZW50ICpwb3M7Cj4gKwlpbnQgcmV0Owo+ICsJaW50 IGRpc3RhbmNlID0gMDsKPiArCWJvb2wgbm90X3N1cHBvcnRlZCA9IGZhbHNlOwo+ICsKPiArCWlm IChsaXN0X2VtcHR5KGNsaWVudHMpKQo+ICsJCXJldHVybiAtMTsKPiArCj4gKwlsaXN0X2Zvcl9l YWNoX2VudHJ5KHBvcywgY2xpZW50cywgbGlzdCkgewo+ICsJCWlmICh2ZXJib3NlKQo+ICsJCQly ZXQgPSB1cHN0cmVhbV9icmlkZ2VfZGlzdGFuY2Vfd2Fybihwcm92aWRlciwKPiArCQkJCQkJCSAg ICBwb3MtPmNsaWVudCk7Cj4gKwkJZWxzZQo+ICsJCQlyZXQgPSB1cHN0cmVhbV9icmlkZ2VfZGlz dGFuY2UocHJvdmlkZXIsIHBvcy0+Y2xpZW50LAo+ICsJCQkJCQkgICAgICAgTlVMTCk7Cj4gKwo+ ICsJCWlmIChyZXQgPCAwKQo+ICsJCQlub3Rfc3VwcG9ydGVkID0gdHJ1ZTsKPiArCj4gKwkJaWYg KG5vdF9zdXBwb3J0ZWQgJiYgIXZlcmJvc2UpCj4gKwkJCWJyZWFrOwo+ICsKPiArCQlkaXN0YW5j ZSArPSByZXQ7Cj4gKwl9Cj4gKwo+ICsJaWYgKG5vdF9zdXBwb3J0ZWQpCj4gKwkJcmV0dXJuIC0x Owo+ICsKPiArCXJldHVybiBkaXN0YW5jZTsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lf cDJwZG1hX2Rpc3RhbmNlKTsKPiArCj4gKy8qKgo+ICsgKiBwY2lfcDJwZG1hX2Fzc2lnbl9wcm92 aWRlciAtIENoZWNrIGNvbXBhdGliaWx5IChhcyBwZXIgcGNpX3AycGRtYV9kaXN0YW5jZSkKPiAr ICoJYW5kIGFzc2lnbiBhIHByb3ZpZGVyIHRvIGEgbGlzdCBvZiBjbGllbnRzCj4gKyAqIEBwcm92 aWRlcjogcDJwZG1hIHByb3ZpZGVyIHRvIGFzc2lnbiB0byB0aGUgY2xpZW50IGxpc3QKPiArICog QGNsaWVudHM6IGxpc3Qgb2YgZGV2aWNlcyB0byBjaGVjayAoTlVMTC10ZXJtaW5hdGVkKQo+ICsg Kgo+ICsgKiBSZXR1cm5zIGZhbHNlIGlmIGFueSBvZiB0aGUgY2xpZW50cyBhcmUgbm90IGNvbXBh dGlibGUsIHRydWUgaWYgdGhlCj4gKyAqIHByb3ZpZGVyIHdhcyBzdWNjZXNzZnVsbHkgYXNzaWdu ZWQgdG8gdGhlIGNsaWVudHMuCj4gKyAqLwo+ICtib29sIHBjaV9wMnBkbWFfYXNzaWduX3Byb3Zp ZGVyKHN0cnVjdCBwY2lfZGV2ICpwcm92aWRlciwKPiArCQkJCXN0cnVjdCBsaXN0X2hlYWQgKmNs aWVudHMpCj4gK3sKPiArCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAqcG9zOwo+ICsKPiArCWlm IChwY2lfcDJwZG1hX2Rpc3RhbmNlKHByb3ZpZGVyLCBjbGllbnRzLCB0cnVlKSA8IDApCj4gKwkJ cmV0dXJuIGZhbHNlOwo+ICsKPiArCWxpc3RfZm9yX2VhY2hfZW50cnkocG9zLCBjbGllbnRzLCBs aXN0KQo+ICsJCXBvcy0+cHJvdmlkZXIgPSBwcm92aWRlcjsKPiArCj4gKwlyZXR1cm4gdHJ1ZTsK PiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lfcDJwZG1hX2Fzc2lnbl9wcm92aWRlcik7Cj4g Kwo+ICsvKioKPiArICogcGNpX2hhc19wMnBtZW0gLSBjaGVjayBpZiBhIGdpdmVuIFBDSSBkZXZp Y2UgaGFzIHB1Ymxpc2hlZCBhbnkgcDJwbWVtCj4gKyAqIEBwZGV2OiBQQ0kgZGV2aWNlIHRvIGNo ZWNrCj4gKyAqLwo+ICtib29sIHBjaV9oYXNfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQo+ ICt7Cj4gKwlyZXR1cm4gcGRldi0+cDJwZG1hICYmIHBkZXYtPnAycGRtYS0+cDJwbWVtX3B1Ymxp c2hlZDsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lfaGFzX3AycG1lbSk7Cj4gKwo+ICsv KioKPiArICogcGNpX3AycG1lbV9maW5kIC0gZmluZCBhIHBlZXItdG8tcGVlciBETUEgbWVtb3J5 IGRldmljZSBjb21wYXRpYmxlIHdpdGgKPiArICoJdGhlIHNwZWNpZmllZCBsaXN0IG9mIGNsaWVu dHMgYW5kIHNob3J0ZXN0IGRpc3RhbmNlIChhcyBkZXRlcm1pbmVkCj4gKyAqCWJ5IHBjaV9wMnBt ZW1fZG1hKCkpCj4gKyAqIEBjbGllbnRzOiBsaXN0IG9mIGRldmljZXMgdG8gY2hlY2sgKE5VTEwt dGVybWluYXRlZCkKPiArICoKPiArICogSWYgbXVsdGlwbGUgZGV2aWNlcyBhcmUgYmVoaW5kIHRo ZSBzYW1lIHN3aXRjaCwgdGhlIG9uZSAiY2xvc2VzdCIgdG8gdGhlCj4gKyAqIGNsaWVudCBkZXZp Y2VzIGluIHVzZSB3aWxsIGJlIGNob3NlbiBmaXJzdC4gKFNvIGlmIG9uZSBvZiB0aGUgcHJvdmlk ZXJzIGFyZQo+ICsgKiB0aGUgc2FtZSBhcyBvbmUgb2YgdGhlIGNsaWVudHMsIHRoYXQgcHJvdmlk ZXIgd2lsbCBiZSB1c2VkIGFoZWFkIG9mIGFueQo+ICsgKiBvdGhlciBwcm92aWRlcnMgdGhhdCBh cmUgdW5yZWxhdGVkKS4gSWYgbXVsdGlwbGUgcHJvdmlkZXJzIGFyZSBhbiBlcXVhbAo+ICsgKiBk aXN0YW5jZSBhd2F5LCBvbmUgd2lsbCBiZSBjaG9zZW4gYXQgcmFuZG9tLgo+ICsgKgo+ICsgKiBS ZXR1cm5zIGEgcG9pbnRlciB0byB0aGUgUENJIGRldmljZSB3aXRoIGEgcmVmZXJlbmNlIHRha2Vu ICh1c2UgcGNpX2Rldl9wdXQKPiArICogdG8gcmV0dXJuIHRoZSByZWZlcmVuY2UpIG9yIE5VTEwg aWYgbm8gY29tcGF0aWJsZSBkZXZpY2UgaXMgZm91bmQuIFRoZQo+ICsgKiBmb3VuZCBwcm92aWRl ciB3aWxsIGFsc28gYmUgYXNzaWduZWQgdG8gdGhlIGNsaWVudCBsaXN0Lgo+ICsgKi8KPiArc3Ry dWN0IHBjaV9kZXYgKnBjaV9wMnBtZW1fZmluZChzdHJ1Y3QgbGlzdF9oZWFkICpjbGllbnRzKQo+ ICt7Cj4gKwlzdHJ1Y3QgcGNpX2RldiAqcGRldiA9IE5VTEw7Cj4gKwlzdHJ1Y3QgcGNpX3AycGRt YV9jbGllbnQgKnBvczsKPiArCWludCBkaXN0YW5jZTsKPiArCWludCBjbG9zZXN0X2Rpc3RhbmNl ID0gSU5UX01BWDsKPiArCXN0cnVjdCBwY2lfZGV2ICoqY2xvc2VzdF9wZGV2czsKPiArCWludCBk ZXZfY250ID0gMDsKPiArCWNvbnN0IGludCBtYXhfZGV2cyA9IFBBR0VfU0laRSAvIHNpemVvZigq Y2xvc2VzdF9wZGV2cyk7Cj4gKwlpbnQgaTsKPiArCj4gKwljbG9zZXN0X3BkZXZzID0ga21hbGxv YyhQQUdFX1NJWkUsIEdGUF9LRVJORUwpOwo+ICsKPiArCXdoaWxlICgocGRldiA9IHBjaV9nZXRf ZGV2aWNlKFBDSV9BTllfSUQsIFBDSV9BTllfSUQsIHBkZXYpKSkgewo+ICsJCWlmICghcGNpX2hh c19wMnBtZW0ocGRldikpCj4gKwkJCWNvbnRpbnVlOwo+ICsKPiArCQlkaXN0YW5jZSA9IHBjaV9w MnBkbWFfZGlzdGFuY2UocGRldiwgY2xpZW50cywgZmFsc2UpOwo+ICsJCWlmIChkaXN0YW5jZSA8 IDAgfHwgZGlzdGFuY2UgPiBjbG9zZXN0X2Rpc3RhbmNlKQo+ICsJCQljb250aW51ZTsKPiArCj4g KwkJaWYgKGRpc3RhbmNlID09IGNsb3Nlc3RfZGlzdGFuY2UgJiYgZGV2X2NudCA+PSBtYXhfZGV2 cykKPiArCQkJY29udGludWU7Cj4gKwo+ICsJCWlmIChkaXN0YW5jZSA8IGNsb3Nlc3RfZGlzdGFu Y2UpIHsKPiArCQkJZm9yIChpID0gMDsgaSA8IGRldl9jbnQ7IGkrKykKPiArCQkJCXBjaV9kZXZf cHV0KGNsb3Nlc3RfcGRldnNbaV0pOwo+ICsKPiArCQkJZGV2X2NudCA9IDA7Cj4gKwkJCWNsb3Nl c3RfZGlzdGFuY2UgPSBkaXN0YW5jZTsKPiArCQl9Cj4gKwo+ICsJCWNsb3Nlc3RfcGRldnNbZGV2 X2NudCsrXSA9IHBjaV9kZXZfZ2V0KHBkZXYpOwo+ICsJfQo+ICsKPiArCWlmIChkZXZfY250KQo+ ICsJCXBkZXYgPSBwY2lfZGV2X2dldChjbG9zZXN0X3BkZXZzW3ByYW5kb21fdTMyX21heChkZXZf Y250KV0pOwo+ICsKPiArCWZvciAoaSA9IDA7IGkgPCBkZXZfY250OyBpKyspCj4gKwkJcGNpX2Rl dl9wdXQoY2xvc2VzdF9wZGV2c1tpXSk7Cj4gKwo+ICsJaWYgKHBkZXYpCj4gKwkJbGlzdF9mb3Jf ZWFjaF9lbnRyeShwb3MsIGNsaWVudHMsIGxpc3QpCj4gKwkJCXBvcy0+cHJvdmlkZXIgPSBwZGV2 Owo+ICsKPiArCWtmcmVlKGNsb3Nlc3RfcGRldnMpOwo+ICsJcmV0dXJuIHBkZXY7Cj4gK30KPiAr RVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycG1lbV9maW5kKTsKPiArCj4gKy8qKgo+ICsgKiBwY2lf YWxsb2NfcDJwX21lbSAtIGFsbG9jYXRlIHBlZXItdG8tcGVlciBETUEgbWVtb3J5Cj4gKyAqIEBw ZGV2OiB0aGUgZGV2aWNlIHRvIGFsbG9jYXRlIG1lbW9yeSBmcm9tCj4gKyAqIEBzaXplOiBudW1i ZXIgb2YgYnl0ZXMgdG8gYWxsb2NhdGUKPiArICoKPiArICogUmV0dXJucyB0aGUgYWxsb2NhdGVk IG1lbW9yeSBvciBOVUxMIG9uIGVycm9yLgo+ICsgKi8KPiArdm9pZCAqcGNpX2FsbG9jX3AycG1l bShzdHJ1Y3QgcGNpX2RldiAqcGRldiwgc2l6ZV90IHNpemUpCj4gK3sKPiArCXZvaWQgKnJldDsK PiArCj4gKwlpZiAodW5saWtlbHkoIXBkZXYtPnAycGRtYSkpCj4gKwkJcmV0dXJuIE5VTEw7Cj4g Kwo+ICsJaWYgKHVubGlrZWx5KCFwZXJjcHVfcmVmX3RyeWdldF9saXZlKCZwZGV2LT5wMnBkbWEt PmRldm1hcF9yZWYpKSkKPiArCQlyZXR1cm4gTlVMTDsKPiArCj4gKwlyZXQgPSAodm9pZCAqKWdl bl9wb29sX2FsbG9jKHBkZXYtPnAycGRtYS0+cG9vbCwgc2l6ZSk7Cj4gKwo+ICsJaWYgKHVubGlr ZWx5KCFyZXQpKQo+ICsJCXBlcmNwdV9yZWZfcHV0KCZwZGV2LT5wMnBkbWEtPmRldm1hcF9yZWYp Owo+ICsKPiArCXJldHVybiByZXQ7Cj4gK30KPiArRVhQT1JUX1NZTUJPTF9HUEwocGNpX2FsbG9j X3AycG1lbSk7Cj4gKwo+ICsvKioKPiArICogcGNpX2ZyZWVfcDJwbWVtIC0gYWxsb2NhdGUgcGVl ci10by1wZWVyIERNQSBtZW1vcnkKPiArICogQHBkZXY6IHRoZSBkZXZpY2UgdGhlIG1lbW9yeSB3 YXMgYWxsb2NhdGVkIGZyb20KPiArICogQGFkZHI6IGFkZHJlc3Mgb2YgdGhlIG1lbW9yeSB0aGF0 IHdhcyBhbGxvY2F0ZWQKPiArICogQHNpemU6IG51bWJlciBvZiBieXRlcyB0aGF0IHdhcyBhbGxv Y2F0ZWQKPiArICovCj4gK3ZvaWQgcGNpX2ZyZWVfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpwZGV2 LCB2b2lkICphZGRyLCBzaXplX3Qgc2l6ZSkKPiArewo+ICsJZ2VuX3Bvb2xfZnJlZShwZGV2LT5w MnBkbWEtPnBvb2wsICh1aW50cHRyX3QpYWRkciwgc2l6ZSk7Cj4gKwlwZXJjcHVfcmVmX3B1dCgm cGRldi0+cDJwZG1hLT5kZXZtYXBfcmVmKTsKPiArfQo+ICtFWFBPUlRfU1lNQk9MX0dQTChwY2lf ZnJlZV9wMnBtZW0pOwo+ICsKPiArLyoqCj4gKyAqIHBjaV92aXJ0X3RvX2J1cyAtIHJldHVybiB0 aGUgUENJIGJ1cyBhZGRyZXNzIGZvciBhIGdpdmVuIHZpcnR1YWwKPiArICoJYWRkcmVzcyBvYnRh aW5lZCB3aXRoIHBjaV9hbGxvY19wMnBtZW0oKQo+ICsgKiBAcGRldjogdGhlIGRldmljZSB0aGUg bWVtb3J5IHdhcyBhbGxvY2F0ZWQgZnJvbQo+ICsgKiBAYWRkcjogYWRkcmVzcyBvZiB0aGUgbWVt b3J5IHRoYXQgd2FzIGFsbG9jYXRlZAo+ICsgKi8KPiArcGNpX2J1c19hZGRyX3QgcGNpX3AycG1l bV92aXJ0X3RvX2J1cyhzdHJ1Y3QgcGNpX2RldiAqcGRldiwgdm9pZCAqYWRkcikKPiArewo+ICsJ aWYgKCFhZGRyKQo+ICsJCXJldHVybiAwOwo+ICsJaWYgKCFwZGV2LT5wMnBkbWEpCj4gKwkJcmV0 dXJuIDA7Cj4gKwo+ICsJLyoKPiArCSAqIE5vdGU6IHdoZW4gd2UgYWRkZWQgdGhlIG1lbW9yeSB0 byB0aGUgcG9vbCB3ZSB1c2VkIHRoZSBQQ0kKPiArCSAqIGJ1cyBhZGRyZXNzIGFzIHRoZSBwaHlz aWNhbCBhZGRyZXNzLiBTbyBnZW5fcG9vbF92aXJ0X3RvX3BoeXMoKQo+ICsJICogYWN0dWFsbHkg cmV0dXJucyB0aGUgYnVzIGFkZHJlc3MgZGVzcGl0ZSB0aGUgbWlzbGVhZGluZyBuYW1lLgo+ICsJ ICovCj4gKwlyZXR1cm4gZ2VuX3Bvb2xfdmlydF90b19waHlzKHBkZXYtPnAycGRtYS0+cG9vbCwg KHVuc2lnbmVkIGxvbmcpYWRkcik7Cj4gK30KPiArRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycG1l bV92aXJ0X3RvX2J1cyk7Cj4gKwo+ICsvKioKPiArICogcGNpX3AycG1lbV9hbGxvY19zZ2wgLSBh bGxvY2F0ZSBwZWVyLXRvLXBlZXIgRE1BIG1lbW9yeSBpbiBhIHNjYXR0ZXJsaXN0Cj4gKyAqIEBw ZGV2OiB0aGUgZGV2aWNlIHRvIGFsbG9jYXRlIG1lbW9yeSBmcm9tCj4gKyAqIEBzZ2w6IHRoZSBh bGxvY2F0ZWQgc2NhdHRlcmxpc3QKPiArICogQG5lbnRzOiB0aGUgbnVtYmVyIG9mIFNHIGVudHJp ZXMgaW4gdGhlIGxpc3QKPiArICogQGxlbmd0aDogbnVtYmVyIG9mIGJ5dGVzIHRvIGFsbG9jYXRl Cj4gKyAqCj4gKyAqIFJldHVybnMgMCBvbiBzdWNjZXNzCj4gKyAqLwo+ICtzdHJ1Y3Qgc2NhdHRl cmxpc3QgKnBjaV9wMnBtZW1fYWxsb2Nfc2dsKHN0cnVjdCBwY2lfZGV2ICpwZGV2LAo+ICsJCQkJ CSB1bnNpZ25lZCBpbnQgKm5lbnRzLCB1MzIgbGVuZ3RoKQo+ICt7Cj4gKwlzdHJ1Y3Qgc2NhdHRl cmxpc3QgKnNnOwo+ICsJdm9pZCAqYWRkcjsKPiArCj4gKwlzZyA9IGt6YWxsb2Moc2l6ZW9mKCpz ZyksIEdGUF9LRVJORUwpOwo+ICsJaWYgKCFzZykKPiArCQlyZXR1cm4gTlVMTDsKPiArCj4gKwlz Z19pbml0X3RhYmxlKHNnLCAxKTsKPiArCj4gKwlhZGRyID0gcGNpX2FsbG9jX3AycG1lbShwZGV2 LCBsZW5ndGgpOwo+ICsJaWYgKCFhZGRyKQo+ICsJCWdvdG8gb3V0X2ZyZWVfc2c7Cj4gKwo+ICsJ c2dfc2V0X2J1ZihzZywgYWRkciwgbGVuZ3RoKTsKPiArCSpuZW50cyA9IDE7Cj4gKwlyZXR1cm4g c2c7Cj4gKwo+ICtvdXRfZnJlZV9zZzoKPiArCWtmcmVlKHNnKTsKPiArCXJldHVybiBOVUxMOwo+ ICt9Cj4gK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9wMnBtZW1fYWxsb2Nfc2dsKTsKPiArCj4gKy8q Kgo+ICsgKiBwY2lfcDJwbWVtX2ZyZWVfc2dsIC0gZnJlZSBhIHNjYXR0ZXJsaXN0IGFsbG9jYXRl ZCBieSBwY2lfcDJwbWVtX2FsbG9jX3NnbCgpCj4gKyAqIEBwZGV2OiB0aGUgZGV2aWNlIHRvIGFs bG9jYXRlIG1lbW9yeSBmcm9tCj4gKyAqIEBzZ2w6IHRoZSBhbGxvY2F0ZWQgc2NhdHRlcmxpc3QK PiArICogQG5lbnRzOiB0aGUgbnVtYmVyIG9mIFNHIGVudHJpZXMgaW4gdGhlIGxpc3QKPiArICov Cj4gK3ZvaWQgcGNpX3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwgc3RydWN0 IHNjYXR0ZXJsaXN0ICpzZ2wpCj4gK3sKPiArCXN0cnVjdCBzY2F0dGVybGlzdCAqc2c7Cj4gKwlp bnQgY291bnQ7Cj4gKwo+ICsJZm9yX2VhY2hfc2coc2dsLCBzZywgSU5UX01BWCwgY291bnQpIHsK PiArCQlpZiAoIXNnKQo+ICsJCQlicmVhazsKPiArCj4gKwkJcGNpX2ZyZWVfcDJwbWVtKHBkZXYs IHNnX3ZpcnQoc2cpLCBzZy0+bGVuZ3RoKTsKPiArCX0KPiArCWtmcmVlKHNnbCk7Cj4gK30KPiAr RVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycG1lbV9mcmVlX3NnbCk7Cj4gKwo+ICsvKioKPiArICog cGNpX3AycG1lbV9wdWJsaXNoIC0gcHVibGlzaCB0aGUgcGVlci10by1wZWVyIERNQSBtZW1vcnkg Zm9yIHVzZSBieQo+ICsgKglvdGhlciBkZXZpY2VzIHdpdGggcGNpX3AycG1lbV9maW5kKCkKPiAr ICogQHBkZXY6IHRoZSBkZXZpY2Ugd2l0aCBwZWVyLXRvLXBlZXIgRE1BIG1lbW9yeSB0byBwdWJs aXNoCj4gKyAqIEBwdWJsaXNoOiBzZXQgdG8gdHJ1ZSB0byBwdWJsaXNoIHRoZSBtZW1vcnksIGZh bHNlIHRvIHVucHVibGlzaCBpdAo+ICsgKgo+ICsgKiBQdWJsaXNoZWQgbWVtb3J5IGNhbiBiZSB1 c2VkIGJ5IG90aGVyIFBDSSBkZXZpY2UgZHJpdmVycyBmb3IKPiArICogcGVlci0yLXBlZXIgRE1B IG9wZXJhdGlvbnMuIE5vbi1wdWJsaXNoZWQgbWVtb3J5IGlzIHJlc2VydmVkIGZvcgo+ICsgKiBl eGx1c2l2ZSB1c2Ugb2YgdGhlIGRldmljZSBkcml2ZXIgdGhhdCByZWdpc3RlcnMgdGhlIHBlZXIt dG8tcGVlcgo+ICsgKiBtZW1vcnkuCj4gKyAqLwo+ICt2b2lkIHBjaV9wMnBtZW1fcHVibGlzaChz dHJ1Y3QgcGNpX2RldiAqcGRldiwgYm9vbCBwdWJsaXNoKQo+ICt7Cj4gKwlpZiAocHVibGlzaCAm JiAhcGRldi0+cDJwZG1hKQo+ICsJCXJldHVybjsKPiArCj4gKwlwZGV2LT5wMnBkbWEtPnAycG1l bV9wdWJsaXNoZWQgPSBwdWJsaXNoOwo+ICt9Cj4gK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9wMnBt ZW1fcHVibGlzaCk7Cj4gZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvbWVtcmVtYXAuaCBiL2lu Y2x1ZGUvbGludXgvbWVtcmVtYXAuaAo+IGluZGV4IGY5MWY5ZTc2MzU1Ny4uOTU1MzM3MGViZGFk IDEwMDY0NAo+IC0tLSBhL2luY2x1ZGUvbGludXgvbWVtcmVtYXAuaAo+ICsrKyBiL2luY2x1ZGUv bGludXgvbWVtcmVtYXAuaAo+IEBAIC01MywxMSArNTMsMTYgQEAgc3RydWN0IHZtZW1fYWx0bWFw IHsKPiAgICAqIHdha2V1cCBldmVudCB3aGVuZXZlciBhIHBhZ2UgaXMgdW5waW5uZWQgYW5kIGJl Y29tZXMgaWRsZS4gVGhpcwo+ICAgICogd2FrZXVwIGlzIHVzZWQgdG8gY29vcmRpbmF0ZSBwaHlz aWNhbCBhZGRyZXNzIHNwYWNlIG1hbmFnZW1lbnQgKGV4Ogo+ICAgICogZnMgdHJ1bmNhdGUvaG9s ZSBwdW5jaCkgdnMgcGlubmVkIHBhZ2VzIChleDogZGV2aWNlIGRtYSkuCj4gKyAqCj4gKyAqIE1F TU9SWV9ERVZJQ0VfUENJX1AyUERNQToKPiArICogRGV2aWNlIG1lbW9yeSByZXNpZGluZyBpbiBh IFBDSSBCQVIgaW50ZW5kZWQgZm9yIHVzZSB3aXRoIFBlZXItdG8tUGVlcgo+ICsgKiB0cmFuc2Fj dGlvbnMuCj4gICAgKi8KPiAgIGVudW0gbWVtb3J5X3R5cGUgewo+ICAgCU1FTU9SWV9ERVZJQ0Vf UFJJVkFURSA9IDEsCj4gICAJTUVNT1JZX0RFVklDRV9QVUJMSUMsCj4gICAJTUVNT1JZX0RFVklD RV9GU19EQVgsCj4gKwlNRU1PUllfREVWSUNFX1BDSV9QMlBETUEsCj4gICB9Owo+ICAgCj4gICAv Kgo+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L21tLmggYi9pbmNsdWRlL2xpbnV4L21tLmgK PiBpbmRleCBhNjFlYmU4YWQ0Y2EuLjIwNTVkZjQxMmE3NyAxMDA2NDQKPiAtLS0gYS9pbmNsdWRl L2xpbnV4L21tLmgKPiArKysgYi9pbmNsdWRlL2xpbnV4L21tLmgKPiBAQCAtODkwLDYgKzg5MCwx OSBAQCBzdGF0aWMgaW5saW5lIGJvb2wgaXNfZGV2aWNlX3B1YmxpY19wYWdlKGNvbnN0IHN0cnVj dCBwYWdlICpwYWdlKQo+ICAgCQlwYWdlLT5wZ21hcC0+dHlwZSA9PSBNRU1PUllfREVWSUNFX1BV QkxJQzsKPiAgIH0KPiAgIAo+ICsjaWZkZWYgQ09ORklHX1BDSV9QMlBETUEKPiArc3RhdGljIGlu bGluZSBib29sIGlzX3BjaV9wMnBkbWFfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiAr ewo+ICsJcmV0dXJuIGlzX3pvbmVfZGV2aWNlX3BhZ2UocGFnZSkgJiYKPiArCQlwYWdlLT5wZ21h cC0+dHlwZSA9PSBNRU1PUllfREVWSUNFX1BDSV9QMlBETUE7Cj4gK30KPiArI2Vsc2UgLyogQ09O RklHX1BDSV9QMlBETUEgKi8KPiArc3RhdGljIGlubGluZSBib29sIGlzX3BjaV9wMnBkbWFfcGFn ZShjb25zdCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiArewo+ICsJcmV0dXJuIGZhbHNlOwo+ICt9Cj4g KyNlbmRpZiAvKiBDT05GSUdfUENJX1AyUERNQSAqLwo+ICsKPiAgICNlbHNlIC8qIENPTkZJR19E RVZfUEFHRU1BUF9PUFMgKi8KPiAgIHN0YXRpYyBpbmxpbmUgdm9pZCBkZXZfcGFnZW1hcF9nZXRf b3BzKHZvaWQpCj4gICB7Cj4gQEAgLTkxMyw2ICs5MjYsMTEgQEAgc3RhdGljIGlubGluZSBib29s IGlzX2RldmljZV9wdWJsaWNfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiAgIHsKPiAg IAlyZXR1cm4gZmFsc2U7Cj4gICB9Cj4gKwo+ICtzdGF0aWMgaW5saW5lIGJvb2wgaXNfcGNpX3Ay cGRtYV9wYWdlKGNvbnN0IHN0cnVjdCBwYWdlICpwYWdlKQo+ICt7Cj4gKwlyZXR1cm4gZmFsc2U7 Cj4gK30KPiAgICNlbmRpZiAvKiBDT05GSUdfREVWX1BBR0VNQVBfT1BTICovCj4gICAKPiAgIHN0 YXRpYyBpbmxpbmUgdm9pZCBnZXRfcGFnZShzdHJ1Y3QgcGFnZSAqcGFnZSkKPiBkaWZmIC0tZ2l0 IGEvaW5jbHVkZS9saW51eC9wY2ktcDJwZG1hLmggYi9pbmNsdWRlL2xpbnV4L3BjaS1wMnBkbWEu aAo+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0Cj4gaW5kZXggMDAwMDAwMDAwMDAwLi43YjJiMGY1NDc1 MjgKPiAtLS0gL2Rldi9udWxsCj4gKysrIGIvaW5jbHVkZS9saW51eC9wY2ktcDJwZG1hLmgKPiBA QCAtMCwwICsxLDEwMiBAQAo+ICsvKiBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMCAq Lwo+ICsvKgo+ICsgKiBQQ0kgUGVlciAyIFBlZXIgRE1BIHN1cHBvcnQuCj4gKyAqCj4gKyAqIENv cHlyaWdodCAoYykgMjAxNi0yMDE4LCBMb2dhbiBHdW50aG9ycGUKPiArICogQ29weXJpZ2h0IChj KSAyMDE2LTIwMTcsIE1pY3Jvc2VtaSBDb3Jwb3JhdGlvbgo+ICsgKiBDb3B5cmlnaHQgKGMpIDIw MTcsIENocmlzdG9waCBIZWxsd2lnCj4gKyAqIENvcHlyaWdodCAoYykgMjAxOCwgRWlkZXRpY29t IEluYy4KPiArICoKPiArICovCj4gKwo+ICsjaWZuZGVmIF9MSU5VWF9QQ0lfUDJQRE1BX0gKPiAr I2RlZmluZSBfTElOVVhfUENJX1AyUERNQV9ICj4gKwo+ICsjaW5jbHVkZSA8bGludXgvcGNpLmg+ Cj4gKwo+ICtzdHJ1Y3QgYmxvY2tfZGV2aWNlOwo+ICtzdHJ1Y3Qgc2NhdHRlcmxpc3Q7Cj4gKwo+ ICsjaWZkZWYgQ09ORklHX1BDSV9QMlBETUEKPiAraW50IHBjaV9wMnBkbWFfYWRkX3Jlc291cmNl KHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBpbnQgYmFyLCBzaXplX3Qgc2l6ZSwKPiArCQl1NjQgb2Zm c2V0KTsKPiAraW50IHBjaV9wMnBkbWFfYWRkX2NsaWVudChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFk LCBzdHJ1Y3QgZGV2aWNlICpkZXYpOwo+ICt2b2lkIHBjaV9wMnBkbWFfcmVtb3ZlX2NsaWVudChz dHJ1Y3QgbGlzdF9oZWFkICpoZWFkLCBzdHJ1Y3QgZGV2aWNlICpkZXYpOwo+ICt2b2lkIHBjaV9w MnBkbWFfY2xpZW50X2xpc3RfZnJlZShzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkKTsKPiAraW50IHBj aV9wMnBkbWFfZGlzdGFuY2Uoc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyLCBzdHJ1Y3QgbGlzdF9o ZWFkICpjbGllbnRzLAo+ICsJCQlib29sIHZlcmJvc2UpOwo+ICtib29sIHBjaV9wMnBkbWFfYXNz aWduX3Byb3ZpZGVyKHN0cnVjdCBwY2lfZGV2ICpwcm92aWRlciwKPiArCQkJCXN0cnVjdCBsaXN0 X2hlYWQgKmNsaWVudHMpOwo+ICtib29sIHBjaV9oYXNfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpw ZGV2KTsKPiArc3RydWN0IHBjaV9kZXYgKnBjaV9wMnBtZW1fZmluZChzdHJ1Y3QgbGlzdF9oZWFk ICpjbGllbnRzKTsKPiArdm9pZCAqcGNpX2FsbG9jX3AycG1lbShzdHJ1Y3QgcGNpX2RldiAqcGRl diwgc2l6ZV90IHNpemUpOwo+ICt2b2lkIHBjaV9mcmVlX3AycG1lbShzdHJ1Y3QgcGNpX2RldiAq cGRldiwgdm9pZCAqYWRkciwgc2l6ZV90IHNpemUpOwo+ICtwY2lfYnVzX2FkZHJfdCBwY2lfcDJw bWVtX3ZpcnRfdG9fYnVzKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB2b2lkICphZGRyKTsKPiArc3Ry dWN0IHNjYXR0ZXJsaXN0ICpwY2lfcDJwbWVtX2FsbG9jX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRl diwKPiArCQkJCQkgdW5zaWduZWQgaW50ICpuZW50cywgdTMyIGxlbmd0aCk7Cj4gK3ZvaWQgcGNp X3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwgc3RydWN0IHNjYXR0ZXJsaXN0 ICpzZ2wpOwo+ICt2b2lkIHBjaV9wMnBtZW1fcHVibGlzaChzdHJ1Y3QgcGNpX2RldiAqcGRldiwg Ym9vbCBwdWJsaXNoKTsKPiArI2Vsc2UgLyogQ09ORklHX1BDSV9QMlBETUEgKi8KPiArc3RhdGlj IGlubGluZSBpbnQgcGNpX3AycGRtYV9hZGRfcmVzb3VyY2Uoc3RydWN0IHBjaV9kZXYgKnBkZXYs IGludCBiYXIsCj4gKwkJc2l6ZV90IHNpemUsIHU2NCBvZmZzZXQpCj4gK3sKPiArCXJldHVybiAt RU9QTk9UU1VQUDsKPiArfQo+ICtzdGF0aWMgaW5saW5lIGludCBwY2lfcDJwZG1hX2FkZF9jbGll bnQoc3RydWN0IGxpc3RfaGVhZCAqaGVhZCwKPiArCQlzdHJ1Y3QgZGV2aWNlICpkZXYpCj4gK3sK PiArCXJldHVybiAwOwo+ICt9Cj4gK3N0YXRpYyBpbmxpbmUgdm9pZCBwY2lfcDJwZG1hX3JlbW92 ZV9jbGllbnQoc3RydWN0IGxpc3RfaGVhZCAqaGVhZCwKPiArCQlzdHJ1Y3QgZGV2aWNlICpkZXYp Cj4gK3sKPiArfQo+ICtzdGF0aWMgaW5saW5lIHZvaWQgcGNpX3AycGRtYV9jbGllbnRfbGlzdF9m cmVlKHN0cnVjdCBsaXN0X2hlYWQgKmhlYWQpCj4gK3sKPiArfQo+ICtzdGF0aWMgaW5saW5lIGlu dCBwY2lfcDJwZG1hX2Rpc3RhbmNlKHN0cnVjdCBwY2lfZGV2ICpwcm92aWRlciwKPiArCQkJCSAg ICAgIHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMsCj4gKwkJCQkgICAgICBib29sIHZlcmJvc2Up Cj4gK3sKPiArCXJldHVybiAtMTsKPiArfQo+ICtzdGF0aWMgaW5saW5lIGJvb2wgcGNpX3AycGRt YV9hc3NpZ25fcHJvdmlkZXIoc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyLAo+ICsJCQkJCSAgICAg IHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMpCj4gK3sKPiArCXJldHVybiBmYWxzZTsKPiArfQo+ ICtzdGF0aWMgaW5saW5lIGJvb2wgcGNpX2hhc19wMnBtZW0oc3RydWN0IHBjaV9kZXYgKnBkZXYp Cj4gK3sKPiArCXJldHVybiBmYWxzZTsKPiArfQo+ICtzdGF0aWMgaW5saW5lIHN0cnVjdCBwY2lf ZGV2ICpwY2lfcDJwbWVtX2ZpbmQoc3RydWN0IGxpc3RfaGVhZCAqY2xpZW50cykKPiArewo+ICsJ cmV0dXJuIE5VTEw7Cj4gK30KPiArc3RhdGljIGlubGluZSB2b2lkICpwY2lfYWxsb2NfcDJwbWVt KHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBzaXplX3Qgc2l6ZSkKPiArewo+ICsJcmV0dXJuIE5VTEw7 Cj4gK30KPiArc3RhdGljIGlubGluZSB2b2lkIHBjaV9mcmVlX3AycG1lbShzdHJ1Y3QgcGNpX2Rl diAqcGRldiwgdm9pZCAqYWRkciwKPiArCQlzaXplX3Qgc2l6ZSkKPiArewo+ICt9Cj4gK3N0YXRp YyBpbmxpbmUgcGNpX2J1c19hZGRyX3QgcGNpX3AycG1lbV92aXJ0X3RvX2J1cyhzdHJ1Y3QgcGNp X2RldiAqcGRldiwKPiArCQkJCQkJICAgIHZvaWQgKmFkZHIpCj4gK3sKPiArCXJldHVybiAwOwo+ ICt9Cj4gK3N0YXRpYyBpbmxpbmUgc3RydWN0IHNjYXR0ZXJsaXN0ICpwY2lfcDJwbWVtX2FsbG9j X3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwKPiArCQl1bnNpZ25lZCBpbnQgKm5lbnRzLCB1MzIg bGVuZ3RoKQo+ICt7Cj4gKwlyZXR1cm4gTlVMTDsKPiArfQo+ICtzdGF0aWMgaW5saW5lIHZvaWQg cGNpX3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwKPiArCQlzdHJ1Y3Qgc2Nh dHRlcmxpc3QgKnNnbCkKPiArewo+ICt9Cj4gK3N0YXRpYyBpbmxpbmUgdm9pZCBwY2lfcDJwbWVt X3B1Ymxpc2goc3RydWN0IHBjaV9kZXYgKnBkZXYsIGJvb2wgcHVibGlzaCkKPiArewo+ICt9Cj4g KyNlbmRpZiAvKiBDT05GSUdfUENJX1AyUERNQSAqLwo+ICsjZW5kaWYgLyogX0xJTlVYX1BDSV9Q MlBfSCAqLwo+IGRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L3BjaS5oIGIvaW5jbHVkZS9saW51 eC9wY2kuaAo+IGluZGV4IGU3MmNhOGRkNjI0MS4uNWQ5NWRiZjIxZjRhIDEwMDY0NAo+IC0tLSBh L2luY2x1ZGUvbGludXgvcGNpLmgKPiArKysgYi9pbmNsdWRlL2xpbnV4L3BjaS5oCj4gQEAgLTI4 MSw2ICsyODEsNyBAQCBzdHJ1Y3QgcGNpZV9saW5rX3N0YXRlOwo+ICAgc3RydWN0IHBjaV92cGQ7 Cj4gICBzdHJ1Y3QgcGNpX3NyaW92Owo+ICAgc3RydWN0IHBjaV9hdHM7Cj4gK3N0cnVjdCBwY2lf cDJwZG1hOwo+ICAgCj4gICAvKiBUaGUgcGNpX2RldiBzdHJ1Y3R1cmUgZGVzY3JpYmVzIFBDSSBk ZXZpY2VzICovCj4gICBzdHJ1Y3QgcGNpX2RldiB7Cj4gQEAgLTQzOSw2ICs0NDAsOSBAQCBzdHJ1 Y3QgcGNpX2RldiB7Cj4gICAjaWZkZWYgQ09ORklHX1BDSV9QQVNJRAo+ICAgCXUxNgkJcGFzaWRf ZmVhdHVyZXM7Cj4gICAjZW5kaWYKPiArI2lmZGVmIENPTkZJR19QQ0lfUDJQRE1BCj4gKwlzdHJ1 Y3QgcGNpX3AycGRtYSAqcDJwZG1hOwo+ICsjZW5kaWYKPiAgIAlwaHlzX2FkZHJfdAlyb207CQkv KiBQaHlzaWNhbCBhZGRyZXNzIGlmIG5vdCBmcm9tIEJBUiAqLwo+ICAgCXNpemVfdAkJcm9tbGVu OwkJLyogTGVuZ3RoIGlmIG5vdCBmcm9tIEJBUiAqLwo+ICAgCWNoYXIJCSpkcml2ZXJfb3ZlcnJp ZGU7IC8qIERyaXZlciBuYW1lIHRvIGZvcmNlIGEgbWF0Y2ggKi8KCl9fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkxpbnV4LW52ZGltbSBtYWlsaW5nIGxpc3QK TGludXgtbnZkaW1tQGxpc3RzLjAxLm9yZwpodHRwczovL2xpc3RzLjAxLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2xpbnV4LW52ZGltbQo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: christian.koenig@amd.com (=?UTF-8?Q?Christian_K=c3=b6nig?=) Date: Fri, 31 Aug 2018 10:04:29 +0200 Subject: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory In-Reply-To: <20180830185352.3369-2-logang@deltatee.com> References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-2-logang@deltatee.com> Message-ID: <2711b3a7-f582-0dda-8ac2-530be7bda1bb@amd.com> Am 30.08.2018 um 20:53 schrieb Logan Gunthorpe: > Some PCI devices may have memory mapped in a BAR space that's > intended for use in peer-to-peer transactions. In order to enable > such transactions the memory must be registered with ZONE_DEVICE pages > so it can be used by DMA interfaces in existing drivers. We want to use that feature without ZONE_DEVICE pages for DMA-buf as well. How hard would it be to separate enabling P2P detection (e.g. distance between two devices) from this? Regards, Christian. > > Add an interface for other subsystems to find and allocate chunks of P2P > memory as necessary to facilitate transfers between two PCI peers: > > int pci_p2pdma_add_client(); > struct pci_dev *pci_p2pmem_find(); > void *pci_alloc_p2pmem(); > > The new interface requires a driver to collect a list of client devices > involved in the transaction with the pci_p2pmem_add_client*() functions > then call pci_p2pmem_find() to obtain any suitable P2P memory. Once > this is done the list is bound to the memory and the calling driver is > free to add and remove clients as necessary (adding incompatible clients > will fail). With a suitable p2pmem device, memory can then be > allocated with pci_alloc_p2pmem() for use in DMA transactions. > > Depending on hardware, using peer-to-peer memory may reduce the bandwidth > of the transfer but can significantly reduce pressure on system memory. > This may be desirable in many cases: for example a system could be designed > with a small CPU connected to a PCIe switch by a small number of lanes > which would maximize the number of lanes available to connect to NVMe > devices. > > The code is designed to only utilize the p2pmem device if all the devices > involved in a transfer are behind the same PCI bridge. This is because we > have no way of knowing whether peer-to-peer routing between PCIe Root Ports > is supported (PCIe r4.0, sec 1.3.1). Additionally, the benefits of P2P > transfers that go through the RC is limited to only reducing DRAM usage > and, in some cases, coding convenience. The PCI-SIG may be exploring > adding a new capability bit to advertise whether this is possible for > future hardware. > > This commit includes significant rework and feedback from Christoph > Hellwig. > > Signed-off-by: Christoph Hellwig > Signed-off-by: Logan Gunthorpe > --- > drivers/pci/Kconfig | 17 + > drivers/pci/Makefile | 1 + > drivers/pci/p2pdma.c | 761 +++++++++++++++++++++++++++++++++++++++++++++ > include/linux/memremap.h | 5 + > include/linux/mm.h | 18 ++ > include/linux/pci-p2pdma.h | 102 ++++++ > include/linux/pci.h | 4 + > 7 files changed, 908 insertions(+) > create mode 100644 drivers/pci/p2pdma.c > create mode 100644 include/linux/pci-p2pdma.h > > diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig > index 56ff8f6d31fc..deb68be4fdac 100644 > --- a/drivers/pci/Kconfig > +++ b/drivers/pci/Kconfig > @@ -132,6 +132,23 @@ config PCI_PASID > > If unsure, say N. > > +config PCI_P2PDMA > + bool "PCI peer-to-peer transfer support" > + depends on PCI && ZONE_DEVICE > + select GENERIC_ALLOCATOR > + help > + Enable? drivers to do PCI peer-to-peer transactions to and from > + BARs that are exposed in other devices that are the part of > + the hierarchy where peer-to-peer DMA is guaranteed by the PCI > + specification to work (ie. anything below a single PCI bridge). > + > + Many PCIe root complexes do not support P2P transactions and > + it's hard to tell which support it at all, so at this time, > + P2P DMA transations must be between devices behind the same root > + port. > + > + If unsure, say N. > + > config PCI_LABEL > def_bool y if (DMI || ACPI) > depends on PCI > diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile > index 1b2cfe51e8d7..85f4a703b2be 100644 > --- a/drivers/pci/Makefile > +++ b/drivers/pci/Makefile > @@ -26,6 +26,7 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o > obj-$(CONFIG_PCI_STUB) += pci-stub.o > obj-$(CONFIG_PCI_PF_STUB) += pci-pf-stub.o > obj-$(CONFIG_PCI_ECAM) += ecam.o > +obj-$(CONFIG_PCI_P2PDMA) += p2pdma.o > obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o > > # Endpoint library must be initialized before its users > diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c > new file mode 100644 > index 000000000000..88aaec5351cd > --- /dev/null > +++ b/drivers/pci/p2pdma.c > @@ -0,0 +1,761 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * PCI Peer 2 Peer DMA support. > + * > + * Copyright (c) 2016-2018, Logan Gunthorpe > + * Copyright (c) 2016-2017, Microsemi Corporation > + * Copyright (c) 2017, Christoph Hellwig > + * Copyright (c) 2018, Eideticom Inc. > + */ > + > +#define pr_fmt(fmt) "pci-p2pdma: " fmt > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +struct pci_p2pdma { > + struct percpu_ref devmap_ref; > + struct completion devmap_ref_done; > + struct gen_pool *pool; > + bool p2pmem_published; > +}; > + > +static void pci_p2pdma_percpu_release(struct percpu_ref *ref) > +{ > + struct pci_p2pdma *p2p = > + container_of(ref, struct pci_p2pdma, devmap_ref); > + > + complete_all(&p2p->devmap_ref_done); > +} > + > +static void pci_p2pdma_percpu_kill(void *data) > +{ > + struct percpu_ref *ref = data; > + > + if (percpu_ref_is_dying(ref)) > + return; > + > + percpu_ref_kill(ref); > +} > + > +static void pci_p2pdma_release(void *data) > +{ > + struct pci_dev *pdev = data; > + > + if (!pdev->p2pdma) > + return; > + > + wait_for_completion(&pdev->p2pdma->devmap_ref_done); > + percpu_ref_exit(&pdev->p2pdma->devmap_ref); > + > + gen_pool_destroy(pdev->p2pdma->pool); > + pdev->p2pdma = NULL; > +} > + > +static int pci_p2pdma_setup(struct pci_dev *pdev) > +{ > + int error = -ENOMEM; > + struct pci_p2pdma *p2p; > + > + p2p = devm_kzalloc(&pdev->dev, sizeof(*p2p), GFP_KERNEL); > + if (!p2p) > + return -ENOMEM; > + > + p2p->pool = gen_pool_create(PAGE_SHIFT, dev_to_node(&pdev->dev)); > + if (!p2p->pool) > + goto out; > + > + init_completion(&p2p->devmap_ref_done); > + error = percpu_ref_init(&p2p->devmap_ref, > + pci_p2pdma_percpu_release, 0, GFP_KERNEL); > + if (error) > + goto out_pool_destroy; > + > + percpu_ref_switch_to_atomic_sync(&p2p->devmap_ref); > + > + error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev); > + if (error) > + goto out_pool_destroy; > + > + pdev->p2pdma = p2p; > + > + return 0; > + > +out_pool_destroy: > + gen_pool_destroy(p2p->pool); > +out: > + devm_kfree(&pdev->dev, p2p); > + return error; > +} > + > +/** > + * pci_p2pdma_add_resource - add memory for use as p2p memory > + * @pdev: the device to add the memory to > + * @bar: PCI BAR to add > + * @size: size of the memory to add, may be zero to use the whole BAR > + * @offset: offset into the PCI BAR > + * > + * The memory will be given ZONE_DEVICE struct pages so that it may > + * be used with any DMA request. > + */ > +int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, > + u64 offset) > +{ > + struct dev_pagemap *pgmap; > + void *addr; > + int error; > + > + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) > + return -EINVAL; > + > + if (offset >= pci_resource_len(pdev, bar)) > + return -EINVAL; > + > + if (!size) > + size = pci_resource_len(pdev, bar) - offset; > + > + if (size + offset > pci_resource_len(pdev, bar)) > + return -EINVAL; > + > + if (!pdev->p2pdma) { > + error = pci_p2pdma_setup(pdev); > + if (error) > + return error; > + } > + > + pgmap = devm_kzalloc(&pdev->dev, sizeof(*pgmap), GFP_KERNEL); > + if (!pgmap) > + return -ENOMEM; > + > + pgmap->res.start = pci_resource_start(pdev, bar) + offset; > + pgmap->res.end = pgmap->res.start + size - 1; > + pgmap->res.flags = pci_resource_flags(pdev, bar); > + pgmap->ref = &pdev->p2pdma->devmap_ref; > + pgmap->type = MEMORY_DEVICE_PCI_P2PDMA; > + > + addr = devm_memremap_pages(&pdev->dev, pgmap); > + if (IS_ERR(addr)) { > + error = PTR_ERR(addr); > + goto pgmap_free; > + } > + > + error = gen_pool_add_virt(pdev->p2pdma->pool, (unsigned long)addr, > + pci_bus_address(pdev, bar) + offset, > + resource_size(&pgmap->res), dev_to_node(&pdev->dev)); > + if (error) > + goto pgmap_free; > + > + error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_percpu_kill, > + &pdev->p2pdma->devmap_ref); > + if (error) > + goto pgmap_free; > + > + pci_info(pdev, "added peer-to-peer DMA memory %pR\n", > + &pgmap->res); > + > + return 0; > + > +pgmap_free: > + devres_free(pgmap); > + return error; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_add_resource); > + > +static struct pci_dev *find_parent_pci_dev(struct device *dev) > +{ > + struct device *parent; > + > + dev = get_device(dev); > + > + while (dev) { > + if (dev_is_pci(dev)) > + return to_pci_dev(dev); > + > + parent = get_device(dev->parent); > + put_device(dev); > + dev = parent; > + } > + > + return NULL; > +} > + > +/* > + * Check if a PCI bridge has it's ACS redirection bits set to redirect P2P > + * TLPs upstream via ACS. Returns 1 if the packets will be redirected > + * upstream, 0 otherwise. > + */ > +static int pci_bridge_has_acs_redir(struct pci_dev *dev) > +{ > + int pos; > + u16 ctrl; > + > + pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS); > + if (!pos) > + return 0; > + > + pci_read_config_word(dev, pos + PCI_ACS_CTRL, &ctrl); > + > + if (ctrl & (PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_EC)) > + return 1; > + > + return 0; > +} > + > +static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *dev) > +{ > + if (!buf) > + return; > + > + seq_buf_printf(buf, "%04x:%02x:%02x.%x;", pci_domain_nr(dev->bus), > + dev->bus->number, PCI_SLOT(dev->devfn), > + PCI_FUNC(dev->devfn)); > +} > + > +/* > + * Find the distance through the nearest common upstream bridge between > + * two PCI devices. > + * > + * If the two devices are the same device then 0 will be returned. > + * > + * If there are two virtual functions of the same device behind the same > + * bridge port then 2 will be returned (one step down to the PCIe switch, > + * then one step back to the same device). > + * > + * In the case where two devices are connected to the same PCIe switch, the > + * value 4 will be returned. This corresponds to the following PCI tree: > + * > + * -+ Root Port > + * \+ Switch Upstream Port > + * +-+ Switch Downstream Port > + * + \- Device A > + * \-+ Switch Downstream Port > + * \- Device B > + * > + * The distance is 4 because we traverse from Device A through the downstream > + * port of the switch, to the common upstream port, back up to the second > + * downstream port and then to Device B. > + * > + * Any two devices that don't have a common upstream bridge will return -1. > + * In this way devices on separate PCIe root ports will be rejected, which > + * is what we want for peer-to-peer seeing each PCIe root port defines a > + * separate hierarchy domain and there's no way to determine whether the root > + * complex supports forwarding between them. > + * > + * In the case where two devices are connected to different PCIe switches, > + * this function will still return a positive distance as long as both > + * switches evenutally have a common upstream bridge. Note this covers > + * the case of using multiple PCIe switches to achieve a desired level of > + * fan-out from a root port. The exact distance will be a function of the > + * number of switches between Device A and Device B. > + * > + * If a bridge which has any ACS redirection bits set is in the path > + * then this functions will return -2. This is so we reject any > + * cases where the TLPs are forwarded up into the root complex. > + * In this case, a list of all infringing bridge addresses will be > + * populated in acs_list (assuming it's non-null) for printk purposes. > + */ > +static int upstream_bridge_distance(struct pci_dev *a, > + struct pci_dev *b, > + struct seq_buf *acs_list) > +{ > + int dist_a = 0; > + int dist_b = 0; > + struct pci_dev *bb = NULL; > + int acs_cnt = 0; > + > + /* > + * Note, we don't need to take references to devices returned by > + * pci_upstream_bridge() seeing we hold a reference to a child > + * device which will already hold a reference to the upstream bridge. > + */ > + > + while (a) { > + dist_b = 0; > + > + if (pci_bridge_has_acs_redir(a)) { > + seq_buf_print_bus_devfn(acs_list, a); > + acs_cnt++; > + } > + > + bb = b; > + > + while (bb) { > + if (a == bb) > + goto check_b_path_acs; > + > + bb = pci_upstream_bridge(bb); > + dist_b++; > + } > + > + a = pci_upstream_bridge(a); > + dist_a++; > + } > + > + return -1; > + > +check_b_path_acs: > + bb = b; > + > + while (bb) { > + if (a == bb) > + break; > + > + if (pci_bridge_has_acs_redir(bb)) { > + seq_buf_print_bus_devfn(acs_list, bb); > + acs_cnt++; > + } > + > + bb = pci_upstream_bridge(bb); > + } > + > + if (acs_cnt) > + return -2; > + > + return dist_a + dist_b; > +} > + > +static int upstream_bridge_distance_warn(struct pci_dev *provider, > + struct pci_dev *client) > +{ > + struct seq_buf acs_list; > + int ret; > + > + seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); > + > + ret = upstream_bridge_distance(provider, client, &acs_list); > + if (ret == -2) { > + pci_warn(client, "cannot be used for peer-to-peer DMA as ACS redirect is set between the client and provider\n"); > + /* Drop final semicolon */ > + acs_list.buffer[acs_list.len-1] = 0; > + pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", > + acs_list.buffer); > + > + } else if (ret < 0) { > + pci_warn(client, "cannot be used for peer-to-peer DMA as the client and provider do not share an upstream bridge\n"); > + } > + > + kfree(acs_list.buffer); > + > + return ret; > +} > + > +struct pci_p2pdma_client { > + struct list_head list; > + struct pci_dev *client; > + struct pci_dev *provider; > +}; > + > +/** > + * pci_p2pdma_add_client - allocate a new element in a client device list > + * @head: list head of p2pdma clients > + * @dev: device to add to the list > + * > + * This adds @dev to a list of clients used by a p2pdma device. > + * This list should be passed to pci_p2pmem_find(). Once pci_p2pmem_find() has > + * been called successfully, the list will be bound to a specific p2pdma > + * device and new clients can only be added to the list if they are > + * supported by that p2pdma device. > + * > + * The caller is expected to have a lock which protects @head as necessary > + * so that none of the pci_p2p functions can be called concurrently > + * on that list. > + * > + * Returns 0 if the client was successfully added. > + */ > +int pci_p2pdma_add_client(struct list_head *head, struct device *dev) > +{ > + struct pci_p2pdma_client *item, *new_item; > + struct pci_dev *provider = NULL; > + struct pci_dev *client; > + int ret; > + > + if (IS_ENABLED(CONFIG_DMA_VIRT_OPS) && dev->dma_ops == &dma_virt_ops) { > + dev_warn(dev, "cannot be used for peer-to-peer DMA because the driver makes use of dma_virt_ops\n"); > + return -ENODEV; > + } > + > + client = find_parent_pci_dev(dev); > + if (!client) { > + dev_warn(dev, "cannot be used for peer-to-peer DMA as it is not a PCI device\n"); > + return -ENODEV; > + } > + > + item = list_first_entry_or_null(head, struct pci_p2pdma_client, list); > + if (item && item->provider) { > + provider = item->provider; > + > + ret = upstream_bridge_distance_warn(provider, client); > + if (ret < 0) { > + ret = -EXDEV; > + goto put_client; > + } > + } > + > + new_item = kzalloc(sizeof(*new_item), GFP_KERNEL); > + if (!new_item) { > + ret = -ENOMEM; > + goto put_client; > + } > + > + new_item->client = client; > + new_item->provider = pci_dev_get(provider); > + > + list_add_tail(&new_item->list, head); > + > + return 0; > + > +put_client: > + pci_dev_put(client); > + return ret; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_add_client); > + > +static void pci_p2pdma_client_free(struct pci_p2pdma_client *item) > +{ > + list_del(&item->list); > + pci_dev_put(item->client); > + pci_dev_put(item->provider); > + kfree(item); > +} > + > +/** > + * pci_p2pdma_remove_client - remove and free a p2pdma client > + * @head: list head of p2pdma clients > + * @dev: device to remove from the list > + * > + * This removes @dev from a list of clients used by a p2pdma device. > + * The caller is expected to have a lock which protects @head as necessary > + * so that none of the pci_p2p functions can be called concurrently > + * on that list. > + */ > +void pci_p2pdma_remove_client(struct list_head *head, struct device *dev) > +{ > + struct pci_p2pdma_client *pos, *tmp; > + struct pci_dev *pdev; > + > + pdev = find_parent_pci_dev(dev); > + if (!pdev) > + return; > + > + list_for_each_entry_safe(pos, tmp, head, list) { > + if (pos->client != pdev) > + continue; > + > + pci_p2pdma_client_free(pos); > + } > + > + pci_dev_put(pdev); > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_remove_client); > + > +/** > + * pci_p2pdma_client_list_free - free an entire list of p2pdma clients > + * @head: list head of p2pdma clients > + * > + * This removes all devices in a list of clients used by a p2pdma device. > + * The caller is expected to have a lock which protects @head as necessary > + * so that none of the pci_p2pdma functions can be called concurrently > + * on that list. > + */ > +void pci_p2pdma_client_list_free(struct list_head *head) > +{ > + struct pci_p2pdma_client *pos, *tmp; > + > + list_for_each_entry_safe(pos, tmp, head, list) > + pci_p2pdma_client_free(pos); > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_client_list_free); > + > +/** > + * pci_p2pdma_distance - Determive the cumulative distance between > + * a p2pdma provider and the clients in use. > + * @provider: p2pdma provider to check against the client list > + * @clients: list of devices to check (NULL-terminated) > + * @verbose: if true, print warnings for devices when we return -1 > + * > + * Returns -1 if any of the clients are not compatible (behind the same > + * root port as the provider), otherwise returns a positive number where > + * the lower number is the preferrable choice. (If there's one client > + * that's the same as the provider it will return 0, which is best choice). > + * > + * For now, "compatible" means the provider and the clients are all behind > + * the same PCI root port. This cuts out cases that may work but is safest > + * for the user. Future work can expand this to white-list root complexes that > + * can safely forward between each ports. > + */ > +int pci_p2pdma_distance(struct pci_dev *provider, struct list_head *clients, > + bool verbose) > +{ > + struct pci_p2pdma_client *pos; > + int ret; > + int distance = 0; > + bool not_supported = false; > + > + if (list_empty(clients)) > + return -1; > + > + list_for_each_entry(pos, clients, list) { > + if (verbose) > + ret = upstream_bridge_distance_warn(provider, > + pos->client); > + else > + ret = upstream_bridge_distance(provider, pos->client, > + NULL); > + > + if (ret < 0) > + not_supported = true; > + > + if (not_supported && !verbose) > + break; > + > + distance += ret; > + } > + > + if (not_supported) > + return -1; > + > + return distance; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_distance); > + > +/** > + * pci_p2pdma_assign_provider - Check compatibily (as per pci_p2pdma_distance) > + * and assign a provider to a list of clients > + * @provider: p2pdma provider to assign to the client list > + * @clients: list of devices to check (NULL-terminated) > + * > + * Returns false if any of the clients are not compatible, true if the > + * provider was successfully assigned to the clients. > + */ > +bool pci_p2pdma_assign_provider(struct pci_dev *provider, > + struct list_head *clients) > +{ > + struct pci_p2pdma_client *pos; > + > + if (pci_p2pdma_distance(provider, clients, true) < 0) > + return false; > + > + list_for_each_entry(pos, clients, list) > + pos->provider = provider; > + > + return true; > +} > +EXPORT_SYMBOL_GPL(pci_p2pdma_assign_provider); > + > +/** > + * pci_has_p2pmem - check if a given PCI device has published any p2pmem > + * @pdev: PCI device to check > + */ > +bool pci_has_p2pmem(struct pci_dev *pdev) > +{ > + return pdev->p2pdma && pdev->p2pdma->p2pmem_published; > +} > +EXPORT_SYMBOL_GPL(pci_has_p2pmem); > + > +/** > + * pci_p2pmem_find - find a peer-to-peer DMA memory device compatible with > + * the specified list of clients and shortest distance (as determined > + * by pci_p2pmem_dma()) > + * @clients: list of devices to check (NULL-terminated) > + * > + * If multiple devices are behind the same switch, the one "closest" to the > + * client devices in use will be chosen first. (So if one of the providers are > + * the same as one of the clients, that provider will be used ahead of any > + * other providers that are unrelated). If multiple providers are an equal > + * distance away, one will be chosen at random. > + * > + * Returns a pointer to the PCI device with a reference taken (use pci_dev_put > + * to return the reference) or NULL if no compatible device is found. The > + * found provider will also be assigned to the client list. > + */ > +struct pci_dev *pci_p2pmem_find(struct list_head *clients) > +{ > + struct pci_dev *pdev = NULL; > + struct pci_p2pdma_client *pos; > + int distance; > + int closest_distance = INT_MAX; > + struct pci_dev **closest_pdevs; > + int dev_cnt = 0; > + const int max_devs = PAGE_SIZE / sizeof(*closest_pdevs); > + int i; > + > + closest_pdevs = kmalloc(PAGE_SIZE, GFP_KERNEL); > + > + while ((pdev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, pdev))) { > + if (!pci_has_p2pmem(pdev)) > + continue; > + > + distance = pci_p2pdma_distance(pdev, clients, false); > + if (distance < 0 || distance > closest_distance) > + continue; > + > + if (distance == closest_distance && dev_cnt >= max_devs) > + continue; > + > + if (distance < closest_distance) { > + for (i = 0; i < dev_cnt; i++) > + pci_dev_put(closest_pdevs[i]); > + > + dev_cnt = 0; > + closest_distance = distance; > + } > + > + closest_pdevs[dev_cnt++] = pci_dev_get(pdev); > + } > + > + if (dev_cnt) > + pdev = pci_dev_get(closest_pdevs[prandom_u32_max(dev_cnt)]); > + > + for (i = 0; i < dev_cnt; i++) > + pci_dev_put(closest_pdevs[i]); > + > + if (pdev) > + list_for_each_entry(pos, clients, list) > + pos->provider = pdev; > + > + kfree(closest_pdevs); > + return pdev; > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_find); > + > +/** > + * pci_alloc_p2p_mem - allocate peer-to-peer DMA memory > + * @pdev: the device to allocate memory from > + * @size: number of bytes to allocate > + * > + * Returns the allocated memory or NULL on error. > + */ > +void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) > +{ > + void *ret; > + > + if (unlikely(!pdev->p2pdma)) > + return NULL; > + > + if (unlikely(!percpu_ref_tryget_live(&pdev->p2pdma->devmap_ref))) > + return NULL; > + > + ret = (void *)gen_pool_alloc(pdev->p2pdma->pool, size); > + > + if (unlikely(!ret)) > + percpu_ref_put(&pdev->p2pdma->devmap_ref); > + > + return ret; > +} > +EXPORT_SYMBOL_GPL(pci_alloc_p2pmem); > + > +/** > + * pci_free_p2pmem - allocate peer-to-peer DMA memory > + * @pdev: the device the memory was allocated from > + * @addr: address of the memory that was allocated > + * @size: number of bytes that was allocated > + */ > +void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size) > +{ > + gen_pool_free(pdev->p2pdma->pool, (uintptr_t)addr, size); > + percpu_ref_put(&pdev->p2pdma->devmap_ref); > +} > +EXPORT_SYMBOL_GPL(pci_free_p2pmem); > + > +/** > + * pci_virt_to_bus - return the PCI bus address for a given virtual > + * address obtained with pci_alloc_p2pmem() > + * @pdev: the device the memory was allocated from > + * @addr: address of the memory that was allocated > + */ > +pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, void *addr) > +{ > + if (!addr) > + return 0; > + if (!pdev->p2pdma) > + return 0; > + > + /* > + * Note: when we added the memory to the pool we used the PCI > + * bus address as the physical address. So gen_pool_virt_to_phys() > + * actually returns the bus address despite the misleading name. > + */ > + return gen_pool_virt_to_phys(pdev->p2pdma->pool, (unsigned long)addr); > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_virt_to_bus); > + > +/** > + * pci_p2pmem_alloc_sgl - allocate peer-to-peer DMA memory in a scatterlist > + * @pdev: the device to allocate memory from > + * @sgl: the allocated scatterlist > + * @nents: the number of SG entries in the list > + * @length: number of bytes to allocate > + * > + * Returns 0 on success > + */ > +struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, > + unsigned int *nents, u32 length) > +{ > + struct scatterlist *sg; > + void *addr; > + > + sg = kzalloc(sizeof(*sg), GFP_KERNEL); > + if (!sg) > + return NULL; > + > + sg_init_table(sg, 1); > + > + addr = pci_alloc_p2pmem(pdev, length); > + if (!addr) > + goto out_free_sg; > + > + sg_set_buf(sg, addr, length); > + *nents = 1; > + return sg; > + > +out_free_sg: > + kfree(sg); > + return NULL; > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_alloc_sgl); > + > +/** > + * pci_p2pmem_free_sgl - free a scatterlist allocated by pci_p2pmem_alloc_sgl() > + * @pdev: the device to allocate memory from > + * @sgl: the allocated scatterlist > + * @nents: the number of SG entries in the list > + */ > +void pci_p2pmem_free_sgl(struct pci_dev *pdev, struct scatterlist *sgl) > +{ > + struct scatterlist *sg; > + int count; > + > + for_each_sg(sgl, sg, INT_MAX, count) { > + if (!sg) > + break; > + > + pci_free_p2pmem(pdev, sg_virt(sg), sg->length); > + } > + kfree(sgl); > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_free_sgl); > + > +/** > + * pci_p2pmem_publish - publish the peer-to-peer DMA memory for use by > + * other devices with pci_p2pmem_find() > + * @pdev: the device with peer-to-peer DMA memory to publish > + * @publish: set to true to publish the memory, false to unpublish it > + * > + * Published memory can be used by other PCI device drivers for > + * peer-2-peer DMA operations. Non-published memory is reserved for > + * exlusive use of the device driver that registers the peer-to-peer > + * memory. > + */ > +void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) > +{ > + if (publish && !pdev->p2pdma) > + return; > + > + pdev->p2pdma->p2pmem_published = publish; > +} > +EXPORT_SYMBOL_GPL(pci_p2pmem_publish); > diff --git a/include/linux/memremap.h b/include/linux/memremap.h > index f91f9e763557..9553370ebdad 100644 > --- a/include/linux/memremap.h > +++ b/include/linux/memremap.h > @@ -53,11 +53,16 @@ struct vmem_altmap { > * wakeup event whenever a page is unpinned and becomes idle. This > * wakeup is used to coordinate physical address space management (ex: > * fs truncate/hole punch) vs pinned pages (ex: device dma). > + * > + * MEMORY_DEVICE_PCI_P2PDMA: > + * Device memory residing in a PCI BAR intended for use with Peer-to-Peer > + * transactions. > */ > enum memory_type { > MEMORY_DEVICE_PRIVATE = 1, > MEMORY_DEVICE_PUBLIC, > MEMORY_DEVICE_FS_DAX, > + MEMORY_DEVICE_PCI_P2PDMA, > }; > > /* > diff --git a/include/linux/mm.h b/include/linux/mm.h > index a61ebe8ad4ca..2055df412a77 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -890,6 +890,19 @@ static inline bool is_device_public_page(const struct page *page) > page->pgmap->type == MEMORY_DEVICE_PUBLIC; > } > > +#ifdef CONFIG_PCI_P2PDMA > +static inline bool is_pci_p2pdma_page(const struct page *page) > +{ > + return is_zone_device_page(page) && > + page->pgmap->type == MEMORY_DEVICE_PCI_P2PDMA; > +} > +#else /* CONFIG_PCI_P2PDMA */ > +static inline bool is_pci_p2pdma_page(const struct page *page) > +{ > + return false; > +} > +#endif /* CONFIG_PCI_P2PDMA */ > + > #else /* CONFIG_DEV_PAGEMAP_OPS */ > static inline void dev_pagemap_get_ops(void) > { > @@ -913,6 +926,11 @@ static inline bool is_device_public_page(const struct page *page) > { > return false; > } > + > +static inline bool is_pci_p2pdma_page(const struct page *page) > +{ > + return false; > +} > #endif /* CONFIG_DEV_PAGEMAP_OPS */ > > static inline void get_page(struct page *page) > diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h > new file mode 100644 > index 000000000000..7b2b0f547528 > --- /dev/null > +++ b/include/linux/pci-p2pdma.h > @@ -0,0 +1,102 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * PCI Peer 2 Peer DMA support. > + * > + * Copyright (c) 2016-2018, Logan Gunthorpe > + * Copyright (c) 2016-2017, Microsemi Corporation > + * Copyright (c) 2017, Christoph Hellwig > + * Copyright (c) 2018, Eideticom Inc. > + * > + */ > + > +#ifndef _LINUX_PCI_P2PDMA_H > +#define _LINUX_PCI_P2PDMA_H > + > +#include > + > +struct block_device; > +struct scatterlist; > + > +#ifdef CONFIG_PCI_P2PDMA > +int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, > + u64 offset); > +int pci_p2pdma_add_client(struct list_head *head, struct device *dev); > +void pci_p2pdma_remove_client(struct list_head *head, struct device *dev); > +void pci_p2pdma_client_list_free(struct list_head *head); > +int pci_p2pdma_distance(struct pci_dev *provider, struct list_head *clients, > + bool verbose); > +bool pci_p2pdma_assign_provider(struct pci_dev *provider, > + struct list_head *clients); > +bool pci_has_p2pmem(struct pci_dev *pdev); > +struct pci_dev *pci_p2pmem_find(struct list_head *clients); > +void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size); > +void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size); > +pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, void *addr); > +struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, > + unsigned int *nents, u32 length); > +void pci_p2pmem_free_sgl(struct pci_dev *pdev, struct scatterlist *sgl); > +void pci_p2pmem_publish(struct pci_dev *pdev, bool publish); > +#else /* CONFIG_PCI_P2PDMA */ > +static inline int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, > + size_t size, u64 offset) > +{ > + return -EOPNOTSUPP; > +} > +static inline int pci_p2pdma_add_client(struct list_head *head, > + struct device *dev) > +{ > + return 0; > +} > +static inline void pci_p2pdma_remove_client(struct list_head *head, > + struct device *dev) > +{ > +} > +static inline void pci_p2pdma_client_list_free(struct list_head *head) > +{ > +} > +static inline int pci_p2pdma_distance(struct pci_dev *provider, > + struct list_head *clients, > + bool verbose) > +{ > + return -1; > +} > +static inline bool pci_p2pdma_assign_provider(struct pci_dev *provider, > + struct list_head *clients) > +{ > + return false; > +} > +static inline bool pci_has_p2pmem(struct pci_dev *pdev) > +{ > + return false; > +} > +static inline struct pci_dev *pci_p2pmem_find(struct list_head *clients) > +{ > + return NULL; > +} > +static inline void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) > +{ > + return NULL; > +} > +static inline void pci_free_p2pmem(struct pci_dev *pdev, void *addr, > + size_t size) > +{ > +} > +static inline pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, > + void *addr) > +{ > + return 0; > +} > +static inline struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, > + unsigned int *nents, u32 length) > +{ > + return NULL; > +} > +static inline void pci_p2pmem_free_sgl(struct pci_dev *pdev, > + struct scatterlist *sgl) > +{ > +} > +static inline void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) > +{ > +} > +#endif /* CONFIG_PCI_P2PDMA */ > +#endif /* _LINUX_PCI_P2P_H */ > diff --git a/include/linux/pci.h b/include/linux/pci.h > index e72ca8dd6241..5d95dbf21f4a 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -281,6 +281,7 @@ struct pcie_link_state; > struct pci_vpd; > struct pci_sriov; > struct pci_ats; > +struct pci_p2pdma; > > /* The pci_dev structure describes PCI devices */ > struct pci_dev { > @@ -439,6 +440,9 @@ struct pci_dev { > #ifdef CONFIG_PCI_PASID > u16 pasid_features; > #endif > +#ifdef CONFIG_PCI_P2PDMA > + struct pci_p2pdma *p2pdma; > +#endif > phys_addr_t rom; /* Physical address if not from BAR */ > size_t romlen; /* Length if not from BAR */ > char *driver_override; /* Driver name to force a match */