From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Subject: [RFC,v3,1/7] dmaengine: Add Synopsys eDMA IP core driver From: Gustavo Pimentel Message-Id: Date: Fri, 11 Jan 2019 19:33:37 +0100 To: linux-pci@vger.kernel.org, dmaengine@vger.kernel.org Cc: Gustavo Pimentel , Vinod Koul , Dan Williams , Eugeniy Paltsev , Andy Shevchenko , Russell King , Niklas Cassel , Joao Pinto , Jose Abreu , Luis Oliveira , Vitor Soares , Nelson Costa , Pedro Sousa List-ID: QWRkIFN5bm9wc3lzIGVETUEgSVAgY29yZSBkcml2ZXIgdG8ga2VybmVsLgoKVGhpcyBjb3JlIGRy aXZlciwgaW5pdGlhbGl6ZXMgYW5kIGNvbmZpZ3VyZXMgdGhlIGVETUEgSVAgdXNpbmcgdm1hLWhl bHBlcnMKZnVuY3Rpb25zIGFuZCBkbWEtZW5naW5lIHN1YnN5c3RlbS4KCkFsc28gY3JlYXRlcyBh biBhYnN0cmF0aW9uIGxheWVyIHRocm91Z2ggY2FsbGJhY2tzIGFsbG93aW5nIGRpZmZlcmVudApy ZWdpc3RlcnMgbWFwcGluZ3MgaW4gdGhlIGZ1dHVyZSwgb3JnYW5pemVkIGluIHRvIHZlcnNpb25z LgoKVGhpcyBkcml2ZXIgY2FuIGJlIGNvbXBpbGUgYXMgYnVpbHQtaW4gb3IgZXh0ZXJuYWwgbW9k dWxlIGluIGtlcm5lbC4KClRvIGVuYWJsZSB0aGlzIGRyaXZlciBqdXN0IHNlbGVjdCBEV19FRE1B IG9wdGlvbiBpbiBrZXJuZWwgY29uZmlndXJhdGlvbiwKaG93ZXZlciBpdCByZXF1aXJlcyBhbmQg c2VsZWN0cyBhdXRvbWF0aWNhbGx5IERNQV9FTkdJTkUgYW5kCkRNQV9WSVJUVUFMX0NIQU5ORUxT IG9wdGlvbiB0b28uCgpDaGFuZ2VzOgpSRkMgdjEtPlJGQyB2MjoKIC0gUmVwbGFjZSBjb21tZW50 cyAvLyAoQzk5IHN0eWxlKSBieSAvKiovCiAtIEZpeCB0aGUgaGVhZGVycyBvZiB0aGUgLmMgYW5k IC5oIGZpbGVzIGFjY29yZGluZyB0byB0aGUgbW9zdCByZWNlbnQKICAgY29udmVudGlvbgogLSBG aXggZXJyb3JzIGFuZCBjaGVja3MgcG9pbnRlZCBvdXQgYnkgY2hlY2twYXRjaCB3aXRoIC0tc3Ry aWN0IG9wdGlvbgogLSBSZXBsYWNlIHBhdGNoIHNtYWxsIGRlc2NyaXB0aW9uIHRhZyBmcm9tIGRt YSBieSBkbWFlbmdpbmUKIC0gQ2hhbmdlIHNvbWUgZGV2X2luZm8oKSBpbnRvIGRldl9kYmcoKQog LSBSZW1vdmUgdW5uZWNlc3NhcnkgemVybyBpbml0aWFsaXphdGlvbiBhZnRlciBremFsbG9jCiAt IFJlbW92ZSBkaXJlY3Rpb24gdmFsaWRhdGlvbiBvbiBjb25maWcoKSBBUEksIHNpbmNlIHRoZSBk aXJlY3Rpb24KICAgcGFyYW1ldGVyIGlzIGRlcHJlY2F0ZWQKIC0gUmVmYWN0b3IgY29kZSB0byBy ZXBsYWNlIGF0b21pY190IGJ5IHUzMiB2YXJpYWJsZSB0eXBlCiAtIFJlcGxhY2Ugc3RhcnRfdHJh bnNmZXIoKSBuYW1lIGJ5IGR3X2VkbWFfc3RhcnRfdHJhbnNmZXIoKQogLSBBZGQgc3BpbmxvY2sg dG8gZHdfZWRtYV9kZXZpY2VfcHJlcF9zbGF2ZV9zZygpCiAtIEFkZCBzcGlubG9jayB0byBkd19l ZG1hX2ZyZWVfY2h1bmsoKQogLSBTaW1wbGlmeSBzd2l0Y2ggY2FzZSBpbnRvIGlmIG9uIGR3X2Vk bWFfZGV2aWNlX3BhdXNlKCksCiAgIGR3X2VkbWFfZGV2aWNlX3Jlc3VtZSgpIGFuZCBkd19lZG1h X2RldmljZV90ZXJtaW5hdGVfYWxsKCkKUkZDIHYyLT5SRkMgdjM6CiAtIEFkZCBkcml2ZXIgcGFy YW1ldGVyIHRvIGRpc2FibGUgbXNpeCBmZWF0dXJlCiAtIEZpeCBwcmludGsgdmFyaWFibGUgb2Yg cGh5c19hZGRyX3QgdHlwZQogLSBGaXggcHJpbnRrIHZhcmlhYmxlIG9mIF9faW9tZW0gdHlwZQog LSBGaXggcHJpbnRrIHZhcmlhYmxlIG9mIHNpemVfdCB0eXBlCiAtIEFkZCBjb21tZW50cyBvciBp bXByb3ZlIGV4aXN0aW5nIG9uZXMKIC0gQWRkIHBvc3NpYmlsaXR5IHRvIHdvcmsgd2l0aCBtdWx0 aXBsZSBJUlFzIGZlYXR1cmUKIC0gRml4IHNvdXJjZSBhbmQgZGVzdGluYXRpb24gYWRkcmVzc2Vz CiAtIEFkZCBkZWZpbmUgdG8gbWFnaWMgbnVtYmVycwogLSBBZGQgRE1BIGN5Y2xpYyB0cmFuc2Zl ciBmZWF0dXJlCgpTaWduZWQtb2ZmLWJ5OiBHdXN0YXZvIFBpbWVudGVsIDxndXN0YXZvLnBpbWVu dGVsQHN5bm9wc3lzLmNvbT4KQ2M6IFZpbm9kIEtvdWwgPHZrb3VsQGtlcm5lbC5vcmc+CkNjOiBE YW4gV2lsbGlhbXMgPGRhbi5qLndpbGxpYW1zQGludGVsLmNvbT4KQ2M6IEV1Z2VuaXkgUGFsdHNl diA8cGFsdHNldkBzeW5vcHN5cy5jb20+CkNjOiBBbmR5IFNoZXZjaGVua28gPGFuZHJpeS5zaGV2 Y2hlbmtvQGxpbnV4LmludGVsLmNvbT4KQ2M6IFJ1c3NlbGwgS2luZyA8cm1rK2tlcm5lbEBhcm1s aW51eC5vcmcudWs+CkNjOiBOaWtsYXMgQ2Fzc2VsIDxuaWtsYXMuY2Fzc2VsQGxpbmFyby5vcmc+ CkNjOiBKb2FvIFBpbnRvIDxqcGludG9Ac3lub3BzeXMuY29tPgpDYzogSm9zZSBBYnJldSA8am9z ZS5hYnJldUBzeW5vcHN5cy5jb20+CkNjOiBMdWlzIE9saXZlaXJhIDxsb2xpdmVpQHN5bm9wc3lz LmNvbT4KQ2M6IFZpdG9yIFNvYXJlcyA8dml0b3Iuc29hcmVzQHN5bm9wc3lzLmNvbT4KQ2M6IE5l bHNvbiBDb3N0YSA8bmVsc29uLmNvc3RhQHN5bm9wc3lzLmNvbT4KQ2M6IFBlZHJvIFNvdXNhIDxw ZWRyb20uc291c2FAc3lub3BzeXMuY29tPgotLS0KIGRyaXZlcnMvZG1hL0tjb25maWcgICAgICAg ICAgICAgICAgfCAgICAyICsKIGRyaXZlcnMvZG1hL01ha2VmaWxlICAgICAgICAgICAgICAgfCAg ICAxICsKIGRyaXZlcnMvZG1hL2R3LWVkbWEvS2NvbmZpZyAgICAgICAgfCAgICA5ICsKIGRyaXZl cnMvZG1hL2R3LWVkbWEvTWFrZWZpbGUgICAgICAgfCAgICA0ICsKIGRyaXZlcnMvZG1hL2R3LWVk bWEvZHctZWRtYS1jb3JlLmMgfCAxMDU5ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrKwogZHJpdmVycy9kbWEvZHctZWRtYS9kdy1lZG1hLWNvcmUuaCB8ICAxNTEgKysrKysKIGlu Y2x1ZGUvbGludXgvZG1hL2VkbWEuaCAgICAgICAgICAgfCAgIDQzICsrCiA3IGZpbGVzIGNoYW5n ZWQsIDEyNjkgaW5zZXJ0aW9ucygrKQogY3JlYXRlIG1vZGUgMTAwNjQ0IGRyaXZlcnMvZG1hL2R3 LWVkbWEvS2NvbmZpZwogY3JlYXRlIG1vZGUgMTAwNjQ0IGRyaXZlcnMvZG1hL2R3LWVkbWEvTWFr ZWZpbGUKIGNyZWF0ZSBtb2RlIDEwMDY0NCBkcml2ZXJzL2RtYS9kdy1lZG1hL2R3LWVkbWEtY29y ZS5jCiBjcmVhdGUgbW9kZSAxMDA2NDQgZHJpdmVycy9kbWEvZHctZWRtYS9kdy1lZG1hLWNvcmUu aAogY3JlYXRlIG1vZGUgMTAwNjQ0IGluY2x1ZGUvbGludXgvZG1hL2VkbWEuaAoKZGlmZiAtLWdp dCBhL2RyaXZlcnMvZG1hL0tjb25maWcgYi9kcml2ZXJzL2RtYS9LY29uZmlnCmluZGV4IGQyMjg2 YzcuLjU4NzdlZTUgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZG1hL0tjb25maWcKKysrIGIvZHJpdmVy cy9kbWEvS2NvbmZpZwpAQCAtNjUxLDYgKzY1MSw4IEBAIHNvdXJjZSAiZHJpdmVycy9kbWEvcWNv bS9LY29uZmlnIgogCiBzb3VyY2UgImRyaXZlcnMvZG1hL2R3L0tjb25maWciCiAKK3NvdXJjZSAi ZHJpdmVycy9kbWEvZHctZWRtYS9LY29uZmlnIgorCiBzb3VyY2UgImRyaXZlcnMvZG1hL2hzdS9L Y29uZmlnIgogCiBzb3VyY2UgImRyaXZlcnMvZG1hL3NoL0tjb25maWciCmRpZmYgLS1naXQgYS9k cml2ZXJzL2RtYS9NYWtlZmlsZSBiL2RyaXZlcnMvZG1hL01ha2VmaWxlCmluZGV4IDA5NTcxYTgu LmU0MmU3ZDYgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZG1hL01ha2VmaWxlCisrKyBiL2RyaXZlcnMv ZG1hL01ha2VmaWxlCkBAIC0yOSw2ICsyOSw3IEBAIG9iai0kKENPTkZJR19ETUFfU1VONEkpICs9 IHN1bjRpLWRtYS5vCiBvYmotJChDT05GSUdfRE1BX1NVTjZJKSArPSBzdW42aS1kbWEubwogb2Jq LSQoQ09ORklHX0RXX0FYSV9ETUFDKSArPSBkdy1heGktZG1hYy8KIG9iai0kKENPTkZJR19EV19E TUFDX0NPUkUpICs9IGR3Lworb2JqLSQoQ09ORklHX0RXX0VETUEpICs9IGR3LWVkbWEvCiBvYmot JChDT05GSUdfRVA5M1hYX0RNQSkgKz0gZXA5M3h4X2RtYS5vCiBvYmotJChDT05GSUdfRlNMX0RN QSkgKz0gZnNsZG1hLm8KIG9iai0kKENPTkZJR19GU0xfRURNQSkgKz0gZnNsLWVkbWEubyBmc2wt ZWRtYS1jb21tb24ubwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9kbWEvZHctZWRtYS9LY29uZmlnIGIv ZHJpdmVycy9kbWEvZHctZWRtYS9LY29uZmlnCm5ldyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAw MDAwMDAuLjMwMTZiZWQKLS0tIC9kZXYvbnVsbAorKysgYi9kcml2ZXJzL2RtYS9kdy1lZG1hL0tj b25maWcKQEAgLTAsMCArMSw5IEBACisjIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4w CisKK2NvbmZpZyBEV19FRE1BCisJdHJpc3RhdGUgIlN5bm9wc3lzIERlc2lnbldhcmUgZURNQSBj b250cm9sbGVyIGRyaXZlciIKKwlzZWxlY3QgRE1BX0VOR0lORQorCXNlbGVjdCBETUFfVklSVFVB TF9DSEFOTkVMUworCWhlbHAKKwkgIFN1cHBvcnQgdGhlIFN5bm9wc3lzIERlc2lnbldhcmUgZURN QSBjb250cm9sbGVyLCBub3JtYWxseQorCSAgaW1wbGVtZW50ZWQgb24gZW5kcG9pbnRzIFNvQ3Mu CmRpZmYgLS1naXQgYS9kcml2ZXJzL2RtYS9kdy1lZG1hL01ha2VmaWxlIGIvZHJpdmVycy9kbWEv ZHctZWRtYS9NYWtlZmlsZQpuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwLi4zMjI0 MDEwCi0tLSAvZGV2L251bGwKKysrIGIvZHJpdmVycy9kbWEvZHctZWRtYS9NYWtlZmlsZQpAQCAt MCwwICsxLDQgQEAKKyMgU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IEdQTC0yLjAKKworb2JqLSQo Q09ORklHX0RXX0VETUEpCQkrPSBkdy1lZG1hLm8KK2R3LWVkbWEtb2JqcwkJCTo9IGR3LWVkbWEt Y29yZS5vCmRpZmYgLS1naXQgYS9kcml2ZXJzL2RtYS9kdy1lZG1hL2R3LWVkbWEtY29yZS5jIGIv ZHJpdmVycy9kbWEvZHctZWRtYS9kdy1lZG1hLWNvcmUuYwpuZXcgZmlsZSBtb2RlIDEwMDY0NApp bmRleCAwMDAwMDAwLi4yYjZiNzBmCi0tLSAvZGV2L251bGwKKysrIGIvZHJpdmVycy9kbWEvZHct ZWRtYS9kdy1lZG1hLWNvcmUuYwpAQCAtMCwwICsxLDEwNTkgQEAKKy8vIFNQRFgtTGljZW5zZS1J ZGVudGlmaWVyOiBHUEwtMi4wCisvKgorICogQ29weXJpZ2h0IChjKSAyMDE4IFN5bm9wc3lzLCBJ bmMuIGFuZC9vciBpdHMgYWZmaWxpYXRlcy4KKyAqIFN5bm9wc3lzIERlc2lnbldhcmUgZURNQSBj b3JlIGRyaXZlcgorICovCisKKyNpbmNsdWRlIDxsaW51eC9tb2R1bGUuaD4KKyNpbmNsdWRlIDxs aW51eC9kZXZpY2UuaD4KKyNpbmNsdWRlIDxsaW51eC9rZXJuZWwuaD4KKyNpbmNsdWRlIDxsaW51 eC9wbV9ydW50aW1lLmg+CisjaW5jbHVkZSA8bGludXgvZG1hZW5naW5lLmg+CisjaW5jbHVkZSA8 bGludXgvZXJyLmg+CisjaW5jbHVkZSA8bGludXgvaW50ZXJydXB0Lmg+CisjaW5jbHVkZSA8bGlu dXgvZG1hL2VkbWEuaD4KKyNpbmNsdWRlIDxsaW51eC9wY2kuaD4KKworI2luY2x1ZGUgImR3LWVk bWEtY29yZS5oIgorI2luY2x1ZGUgIi4uL2RtYWVuZ2luZS5oIgorI2luY2x1ZGUgIi4uL3ZpcnQt ZG1hLmgiCisKKyNkZWZpbmUgU0VUKHJlZywgbmFtZSwgdmFsKQkJCVwKKwlyZWcubmFtZSA9IHZh bAorCisjZGVmaW5lIFNFVF9CT1RIX0NIKG5hbWUsIHZhbHVlKQkJXAorCWRvIHsJCQkJCVwKKwkJ U0VUKGR3LT53cl9lZG1hLCBuYW1lLCB2YWx1ZSk7CVwKKwkJU0VUKGR3LT5yZF9lZG1hLCBuYW1l LCB2YWx1ZSk7CVwKKwl9IHdoaWxlICgwKQorCitzdGF0aWMgaW5saW5lCitzdHJ1Y3QgZGV2aWNl ICpkY2hhbjJkZXYoc3RydWN0IGRtYV9jaGFuICpkY2hhbikKK3sKKwlyZXR1cm4gJmRjaGFuLT5k ZXYtPmRldmljZTsKK30KKworc3RhdGljIGlubGluZQorc3RydWN0IGRldmljZSAqY2hhbjJkZXYo c3RydWN0IGR3X2VkbWFfY2hhbiAqY2hhbikKK3sKKwlyZXR1cm4gJmNoYW4tPnZjLmNoYW4uZGV2 LT5kZXZpY2U7Cit9CisKK3N0YXRpYyBpbmxpbmUKK2NvbnN0IHN0cnVjdCBkd19lZG1hX2NvcmVf b3BzICpjaGFuMm9wcyhzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuKQoreworCXJldHVybiBjaGFu LT5jaGlwLT5kdy0+b3BzOworfQorCitzdGF0aWMgaW5saW5lCitzdHJ1Y3QgZHdfZWRtYV9kZXNj ICp2ZDJkd19lZG1hX2Rlc2Moc3RydWN0IHZpcnRfZG1hX2Rlc2MgKnZkKQoreworCXJldHVybiBj b250YWluZXJfb2YodmQsIHN0cnVjdCBkd19lZG1hX2Rlc2MsIHZkKTsKK30KKworc3RhdGljIHN0 cnVjdCBkd19lZG1hX2J1cnN0ICpkd19lZG1hX2FsbG9jX2J1cnN0KHN0cnVjdCBkd19lZG1hX2No dW5rICpjaHVuaykKK3sKKwlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuID0gY2h1bmstPmNoYW47 CisJc3RydWN0IGR3X2VkbWFfYnVyc3QgKmJ1cnN0OworCisJYnVyc3QgPSBrdnphbGxvYyhzaXpl b2YoKmJ1cnN0KSwgR0ZQX05PV0FJVCk7CisJaWYgKHVubGlrZWx5KCFidXJzdCkpCisJCXJldHVy biBOVUxMOworCisJSU5JVF9MSVNUX0hFQUQoJmJ1cnN0LT5saXN0KTsKKworCWlmIChjaHVuay0+ YnVyc3QpIHsKKwkJLyogQ3JlYXRlIGFuZCBhZGQgbmV3IGVsZW1lbnQgaW50byB0aGUgbGlua2Vk IGxpc3QgKi8KKwkJY2h1bmstPmJ1cnN0c19hbGxvYysrOworCQlkZXZfZGJnKGNoYW4yZGV2KGNo YW4pLCAiYWxsb2MgbmV3IGJ1cnN0IGVsZW1lbnQgKCVkKVxuIiwKKwkJCWNodW5rLT5idXJzdHNf YWxsb2MpOworCQlsaXN0X2FkZF90YWlsKCZidXJzdC0+bGlzdCwgJmNodW5rLT5idXJzdC0+bGlz dCk7CisJfSBlbHNlIHsKKwkJLyogTGlzdCBoZWFkICovCisJCWNodW5rLT5idXJzdHNfYWxsb2Mg PSAwOworCQljaHVuay0+YnVyc3QgPSBidXJzdDsKKwkJZGV2X2RiZyhjaGFuMmRldihjaGFuKSwg ImFsbG9jIG5ldyBidXJzdCBoZWFkXG4iKTsKKwl9CisKKwlyZXR1cm4gYnVyc3Q7Cit9CisKK3N0 YXRpYyBzdHJ1Y3QgZHdfZWRtYV9jaHVuayAqZHdfZWRtYV9hbGxvY19jaHVuayhzdHJ1Y3QgZHdf ZWRtYV9kZXNjICpkZXNjKQoreworCXN0cnVjdCBkd19lZG1hX2NoYW4gKmNoYW4gPSBkZXNjLT5j aGFuOworCXN0cnVjdCBkd19lZG1hICpkdyA9IGNoYW4tPmNoaXAtPmR3OworCXN0cnVjdCBkd19l ZG1hX2NodW5rICpjaHVuazsKKworCWNodW5rID0ga3Z6YWxsb2Moc2l6ZW9mKCpjaHVuayksIEdG UF9OT1dBSVQpOworCWlmICh1bmxpa2VseSghY2h1bmspKQorCQlyZXR1cm4gTlVMTDsKKworCUlO SVRfTElTVF9IRUFEKCZjaHVuay0+bGlzdCk7CisJY2h1bmstPmNoYW4gPSBjaGFuOworCWNodW5r LT5jYiA9ICEoZGVzYy0+Y2h1bmtzX2FsbG9jICUgMik7CisJY2h1bmstPmxsX3JlZ2lvbi5wYWRk ciA9IGR3LT5sbF9yZWdpb24ucGFkZHIgKyBjaGFuLT5sbF9vZmY7CisJY2h1bmstPmxsX3JlZ2lv bi52YWRkciA9IGR3LT5sbF9yZWdpb24udmFkZHIgKyBjaGFuLT5sbF9vZmY7CisKKwlpZiAoZGVz Yy0+Y2h1bmspIHsKKwkJLyogQ3JlYXRlIGFuZCBhZGQgbmV3IGVsZW1lbnQgaW50byB0aGUgbGlu a2VkIGxpc3QgKi8KKwkJZGVzYy0+Y2h1bmtzX2FsbG9jKys7CisJCWRldl9kYmcoY2hhbjJkZXYo Y2hhbiksICJhbGxvYyBuZXcgY2h1bmsgZWxlbWVudCAoJWQpXG4iLAorCQkJZGVzYy0+Y2h1bmtz X2FsbG9jKTsKKwkJbGlzdF9hZGRfdGFpbCgmY2h1bmstPmxpc3QsICZkZXNjLT5jaHVuay0+bGlz dCk7CisJCWR3X2VkbWFfYWxsb2NfYnVyc3QoY2h1bmspOworCX0gZWxzZSB7CisJCS8qIExpc3Qg aGVhZCAqLworCQljaHVuay0+YnVyc3QgPSBOVUxMOworCQlkZXNjLT5jaHVua3NfYWxsb2MgPSAw OworCQlkZXNjLT5jaHVuayA9IGNodW5rOworCQlkZXZfZGJnKGNoYW4yZGV2KGNoYW4pLCAiYWxs b2MgbmV3IGNodW5rIGhlYWRcbiIpOworCX0KKworCXJldHVybiBjaHVuazsKK30KKworc3RhdGlj IHN0cnVjdCBkd19lZG1hX2Rlc2MgKmR3X2VkbWFfYWxsb2NfZGVzYyhzdHJ1Y3QgZHdfZWRtYV9j aGFuICpjaGFuKQoreworCXN0cnVjdCBkd19lZG1hX2Rlc2MgKmRlc2M7CisKKwlkZXZfZGJnKGNo YW4yZGV2KGNoYW4pLCAiYWxsb2MgbmV3IGRlc2NyaXB0b3JcbiIpOworCisJZGVzYyA9IGt2emFs bG9jKHNpemVvZigqZGVzYyksIEdGUF9OT1dBSVQpOworCWlmICh1bmxpa2VseSghZGVzYykpCisJ CXJldHVybiBOVUxMOworCisJZGVzYy0+Y2hhbiA9IGNoYW47CisJZHdfZWRtYV9hbGxvY19jaHVu ayhkZXNjKTsKKworCXJldHVybiBkZXNjOworfQorCitzdGF0aWMgdm9pZCBkd19lZG1hX2ZyZWVf YnVyc3Qoc3RydWN0IGR3X2VkbWFfY2h1bmsgKmNodW5rKQoreworCXN0cnVjdCBkd19lZG1hX2J1 cnN0ICpjaGlsZCwgKl9uZXh0OworCisJaWYgKCFjaHVuay0+YnVyc3QpCisJCXJldHVybjsKKwor CS8qIFJlbW92ZSBhbGwgdGhlIGxpc3QgZWxlbWVudHMgKi8KKwlsaXN0X2Zvcl9lYWNoX2VudHJ5 X3NhZmUoY2hpbGQsIF9uZXh0LCAmY2h1bmstPmJ1cnN0LT5saXN0LCBsaXN0KSB7CisJCWxpc3Rf ZGVsKCZjaGlsZC0+bGlzdCk7CisJCWt2ZnJlZShjaGlsZCk7CisJCWNodW5rLT5idXJzdHNfYWxs b2MtLTsKKwl9CisKKwkvKiBSZW1vdmUgdGhlIGxpc3QgaGVhZCAqLworCWt2ZnJlZShjaGlsZCk7 CisJY2h1bmstPmJ1cnN0ID0gTlVMTDsKK30KKworc3RhdGljIHZvaWQgZHdfZWRtYV9mcmVlX2No dW5rKHN0cnVjdCBkd19lZG1hX2Rlc2MgKmRlc2MpCit7CisJc3RydWN0IGR3X2VkbWFfY2hhbiAq Y2hhbiA9IGRlc2MtPmNoYW47CisJc3RydWN0IGR3X2VkbWFfY2h1bmsgKmNoaWxkLCAqX25leHQ7 CisKKwlpZiAoIWRlc2MtPmNodW5rKQorCQlyZXR1cm47CisKKwkvKiBSZW1vdmUgYWxsIHRoZSBs aXN0IGVsZW1lbnRzICovCisJbGlzdF9mb3JfZWFjaF9lbnRyeV9zYWZlKGNoaWxkLCBfbmV4dCwg JmRlc2MtPmNodW5rLT5saXN0LCBsaXN0KSB7CisJCWR3X2VkbWFfZnJlZV9idXJzdChjaGlsZCk7 CisJCWlmIChjaGlsZC0+YnVyc3RzX2FsbG9jKQorCQkJZGV2X2RiZyhjaGFuMmRldihjaGFuKSwJ IiV1IGJ1cnN0cyBzdGlsbCBhbGxvY2F0ZWRcbiIsCisJCQkJY2hpbGQtPmJ1cnN0c19hbGxvYyk7 CisJCWxpc3RfZGVsKCZjaGlsZC0+bGlzdCk7CisJCWt2ZnJlZShjaGlsZCk7CisJCWRlc2MtPmNo dW5rc19hbGxvYy0tOworCX0KKworCS8qIFJlbW92ZSB0aGUgbGlzdCBoZWFkICovCisJa3ZmcmVl KGNoaWxkKTsKKwlkZXNjLT5jaHVuayA9IE5VTEw7Cit9CisKK3N0YXRpYyB2b2lkIGR3X2VkbWFf ZnJlZV9kZXNjKHN0cnVjdCBkd19lZG1hX2Rlc2MgKmRlc2MpCit7CisJc3RydWN0IGR3X2VkbWFf Y2hhbiAqY2hhbiA9IGRlc2MtPmNoYW47CisJdW5zaWduZWQgbG9uZyBmbGFnczsKKworCXNwaW5f bG9ja19pcnFzYXZlKCZjaGFuLT52Yy5sb2NrLCBmbGFncyk7CisKKwlkd19lZG1hX2ZyZWVfY2h1 bmsoZGVzYyk7CisJaWYgKGRlc2MtPmNodW5rc19hbGxvYykKKwkJZGV2X2RiZyhjaGFuMmRldihj aGFuKSwgIiV1IGNodW5rcyBzdGlsbCBhbGxvY2F0ZWRcbiIsCisJCQlkZXNjLT5jaHVua3NfYWxs b2MpOworCisJc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmY2hhbi0+dmMubG9jaywgZmxhZ3MpOwor fQorCitzdGF0aWMgdm9pZCB2Y2hhbl9mcmVlX2Rlc2Moc3RydWN0IHZpcnRfZG1hX2Rlc2MgKnZk ZXNjKQoreworCWR3X2VkbWFfZnJlZV9kZXNjKHZkMmR3X2VkbWFfZGVzYyh2ZGVzYykpOworfQor CitzdGF0aWMgdm9pZCBkd19lZG1hX3N0YXJ0X3RyYW5zZmVyKHN0cnVjdCBkd19lZG1hX2NoYW4g KmNoYW4pCit7CisJc3RydWN0IHZpcnRfZG1hX2Rlc2MgKnZkOworCXN0cnVjdCBkd19lZG1hX2Rl c2MgKmRlc2M7CisJc3RydWN0IGR3X2VkbWFfY2h1bmsgKmNoaWxkOworCWNvbnN0IHN0cnVjdCBk d19lZG1hX2NvcmVfb3BzICpvcHMgPSBjaGFuMm9wcyhjaGFuKTsKKworCXZkID0gdmNoYW5fbmV4 dF9kZXNjKCZjaGFuLT52Yyk7CisJaWYgKCF2ZCkKKwkJcmV0dXJuOworCisJZGVzYyA9IHZkMmR3 X2VkbWFfZGVzYyh2ZCk7CisJaWYgKCFkZXNjKQorCQlyZXR1cm47CisKKwljaGlsZCA9IGxpc3Rf Zmlyc3RfZW50cnlfb3JfbnVsbCgmZGVzYy0+Y2h1bmstPmxpc3QsCisJCQkJCSBzdHJ1Y3QgZHdf ZWRtYV9jaHVuaywgbGlzdCk7CisJaWYgKCFjaGlsZCkKKwkJcmV0dXJuOworCisJb3BzLT5zdGFy dChjaGlsZCwgIWRlc2MtPnhmZXJfc3opOworCWRlc2MtPnhmZXJfc3ogKz0gY2hpbGQtPmxsX3Jl Z2lvbi5zejsKKwlkZXZfZGJnKGNoYW4yZGV2KGNoYW4pLCAidHJhbnNmZXIgb2YgJXUgYnl0ZXMg c3RhcnRlZFxuIiwKKwkJY2hpbGQtPmxsX3JlZ2lvbi5zeik7CisKKwlkd19lZG1hX2ZyZWVfYnVy c3QoY2hpbGQpOworCWlmIChjaGlsZC0+YnVyc3RzX2FsbG9jKQorCQlkZXZfZGJnKGNoYW4yZGV2 KGNoYW4pLAkiJXUgYnVyc3RzIHN0aWxsIGFsbG9jYXRlZFxuIiwKKwkJCWNoaWxkLT5idXJzdHNf YWxsb2MpOworCWxpc3RfZGVsKCZjaGlsZC0+bGlzdCk7CisJa3ZmcmVlKGNoaWxkKTsKKwlkZXNj LT5jaHVua3NfYWxsb2MtLTsKK30KKworc3RhdGljIGludCBkd19lZG1hX2RldmljZV9jb25maWco c3RydWN0IGRtYV9jaGFuICpkY2hhbiwKKwkJCQkgc3RydWN0IGRtYV9zbGF2ZV9jb25maWcgKmNv bmZpZykKK3sKKwlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuID0gZGNoYW4yZHdfZWRtYV9jaGFu KGRjaGFuKTsKKwljb25zdCBzdHJ1Y3QgZHdfZWRtYV9jb3JlX29wcyAqb3BzID0gY2hhbjJvcHMo Y2hhbik7CisJdW5zaWduZWQgbG9uZyBmbGFnczsKKwlpbnQgZXJyID0gMDsKKworCXNwaW5fbG9j a19pcnFzYXZlKCZjaGFuLT52Yy5sb2NrLCBmbGFncyk7CisKKwlpZiAoIWNvbmZpZykgeworCQll cnIgPSAtRUlOVkFMOworCQlnb3RvIGVycl9jb25maWc7CisJfQorCisJaWYgKGNoYW4tPnN0YXR1 cyAhPSBFRE1BX1NUX0lETEUpIHsKKwkJZGV2X2VycihjaGFuMmRldihjaGFuKSwgImNoYW5uZWwg aXMgYnVzeSBvciBwYXVzZWRcbiIpOworCQllcnIgPSAtRVBFUk07CisJCWdvdG8gZXJyX2NvbmZp ZzsKKwl9CisKKwlkZXZfZGJnKGNoYW4yZGV2KGNoYW4pLCAiYWRkcihwaHlzaWNhbCkgc3JjPSVw YSwgZHN0PSVwYVxuIiwKKwkJJmNvbmZpZy0+c3JjX2FkZHIsICZjb25maWctPmRzdF9hZGRyKTsK KworCWNoYW4tPnNyY19hZGRyID0gY29uZmlnLT5zcmNfYWRkcjsKKwljaGFuLT5kc3RfYWRkciA9 IGNvbmZpZy0+ZHN0X2FkZHI7CisKKwllcnIgPSBvcHMtPmRldmljZV9jb25maWcoZGNoYW4pOwor CWlmICghZXJyKSB7CisJCWNoYW4tPmNvbmZpZ3VyZWQgPSB0cnVlOworCQlkZXZfZGJnKGNoYW4y ZGV2KGNoYW4pLAkiY2hhbm5lbCBjb25maWd1cmVkXG4iKTsKKwl9CisKK2Vycl9jb25maWc6CisJ c3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmY2hhbi0+dmMubG9jaywgZmxhZ3MpOworCXJldHVybiBl cnI7Cit9CisKK3N0YXRpYyBpbnQgZHdfZWRtYV9kZXZpY2VfcGF1c2Uoc3RydWN0IGRtYV9jaGFu ICpkY2hhbikKK3sKKwlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuID0gZGNoYW4yZHdfZWRtYV9j aGFuKGRjaGFuKTsKKwl1bnNpZ25lZCBsb25nIGZsYWdzOworCWludCBlcnIgPSAwOworCisJc3Bp bl9sb2NrX2lycXNhdmUoJmNoYW4tPnZjLmxvY2ssIGZsYWdzKTsKKworCWlmICghY2hhbi0+Y29u ZmlndXJlZCkgeworCQlkZXZfZXJyKGNoYW4yZGV2KGNoYW4pLCAiKHBhdXNlKSBjaGFubmVsIG5v dCBjb25maWd1cmVkXG4iKTsKKwkJZXJyID0gLUVQRVJNOworCQlnb3RvIGVycl9wYXVzZTsKKwl9 CisKKwlpZiAoY2hhbi0+c3RhdHVzICE9IEVETUFfU1RfQlVTWSkgeworCQllcnIgPSAtRVBFUk07 CisJCWdvdG8gZXJyX3BhdXNlOworCX0KKworCWlmIChjaGFuLT5yZXF1ZXN0ICE9IEVETUFfUkVR X05PTkUpIHsKKwkJZXJyID0gLUVQRVJNOworCQlnb3RvIGVycl9wYXVzZTsKKwl9CisKKwljaGFu LT5yZXF1ZXN0ID0gRURNQV9SRVFfUEFVU0U7CisJZGV2X2RiZyhjaGFuMmRldihjaGFuKSwgInBh dXNlIHJlcXVlc3RlZFxuIik7CisKK2Vycl9wYXVzZToKKwlzcGluX3VubG9ja19pcnFyZXN0b3Jl KCZjaGFuLT52Yy5sb2NrLCBmbGFncyk7CisJcmV0dXJuIGVycjsKK30KKworc3RhdGljIGludCBk d19lZG1hX2RldmljZV9yZXN1bWUoc3RydWN0IGRtYV9jaGFuICpkY2hhbikKK3sKKwlzdHJ1Y3Qg ZHdfZWRtYV9jaGFuICpjaGFuID0gZGNoYW4yZHdfZWRtYV9jaGFuKGRjaGFuKTsKKwl1bnNpZ25l ZCBsb25nIGZsYWdzOworCWludCBlcnIgPSAwOworCisJc3Bpbl9sb2NrX2lycXNhdmUoJmNoYW4t PnZjLmxvY2ssIGZsYWdzKTsKKworCWlmICghY2hhbi0+Y29uZmlndXJlZCkgeworCQlkZXZfZXJy KGNoYW4yZGV2KGNoYW4pLCAiKHJlc3VtZSkgY2hhbm5lbCBub3QgY29uZmlndXJlZFxuIik7CisJ CWVyciA9IC1FUEVSTTsKKwkJZ290byBlcnJfcmVzdW1lOworCX0KKworCWlmIChjaGFuLT5zdGF0 dXMgIT0gRURNQV9TVF9QQVVTRSkgeworCQllcnIgPSAtRVBFUk07CisJCWdvdG8gZXJyX3Jlc3Vt ZTsKKwl9CisKKwlpZiAoY2hhbi0+cmVxdWVzdCAhPSBFRE1BX1JFUV9OT05FKSB7CisJCWVyciA9 IC1FUEVSTTsKKwkJZ290byBlcnJfcmVzdW1lOworCX0KKworCWNoYW4tPnN0YXR1cyA9IEVETUFf U1RfQlVTWTsKKwlkZXZfZGJnKGRjaGFuMmRldihkY2hhbiksICJ0cmFuc2ZlciByZXN1bWVkXG4i KTsKKwlkd19lZG1hX3N0YXJ0X3RyYW5zZmVyKGNoYW4pOworCitlcnJfcmVzdW1lOgorCXNwaW5f dW5sb2NrX2lycXJlc3RvcmUoJmNoYW4tPnZjLmxvY2ssIGZsYWdzKTsKKwlyZXR1cm4gZXJyOwor fQorCitzdGF0aWMgaW50IGR3X2VkbWFfZGV2aWNlX3Rlcm1pbmF0ZV9hbGwoc3RydWN0IGRtYV9j aGFuICpkY2hhbikKK3sKKwlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuID0gZGNoYW4yZHdfZWRt YV9jaGFuKGRjaGFuKTsKKwljb25zdCBzdHJ1Y3QgZHdfZWRtYV9jb3JlX29wcyAqb3BzID0gY2hh bjJvcHMoY2hhbik7CisJdW5zaWduZWQgbG9uZyBmbGFnczsKKwlpbnQgZXJyID0gMDsKKwlMSVNU X0hFQUQoaGVhZCk7CisKKwlzcGluX2xvY2tfaXJxc2F2ZSgmY2hhbi0+dmMubG9jaywgZmxhZ3Mp OworCisJaWYgKCFjaGFuLT5jb25maWd1cmVkKQorCQlnb3RvIGVycl90ZXJtaW5hdGU7CisKKwlp ZiAoY2hhbi0+c3RhdHVzID09IEVETUFfU1RfUEFVU0UpIHsKKwkJZGV2X2RiZyhkY2hhbjJkZXYo ZGNoYW4pLCAiY2hhbm5lbCBpcyBwYXVzZWQsIHN0b3BwaW5nIGltbWVkaWF0ZWx5XG4iKTsKKwkJ Y2hhbi0+c3RhdHVzID0gRURNQV9TVF9JRExFOworCQljaGFuLT5jb25maWd1cmVkID0gZmFsc2U7 CisJCWdvdG8gZXJyX3Rlcm1pbmF0ZTsKKwl9IGVsc2UgaWYgKGNoYW4tPnN0YXR1cyA9PSBFRE1B X1NUX0lETEUpIHsKKwkJY2hhbi0+Y29uZmlndXJlZCA9IGZhbHNlOworCQlnb3RvIGVycl90ZXJt aW5hdGU7CisJfSBlbHNlIGlmIChvcHMtPmNoX3N0YXR1cyhjaGFuKSA9PSBETUFfQ09NUExFVEUp IHsKKwkJLyoKKwkJICogVGhlIGNoYW5uZWwgaXMgaW4gYSBmYWxzZSBCVVNZIHN0YXRlLCBwcm9i YWJseSBkaWRuJ3QKKwkJICogcmVjZWl2ZSBvciBsb3N0IGFuIGludGVycnVwdAorCQkgKi8KKwkJ Y2hhbi0+c3RhdHVzID0gRURNQV9TVF9JRExFOworCQljaGFuLT5jb25maWd1cmVkID0gZmFsc2U7 CisJCWdvdG8gZXJyX3Rlcm1pbmF0ZTsKKwl9CisKKwlpZiAoY2hhbi0+cmVxdWVzdCA+IEVETUFf UkVRX1BBVVNFKSB7CisJCWVyciA9IC1FUEVSTTsKKwkJZ290byBlcnJfdGVybWluYXRlOworCX0K KworCWNoYW4tPnJlcXVlc3QgPSBFRE1BX1JFUV9TVE9QOworCWRldl9kYmcoZGNoYW4yZGV2KGRj aGFuKSwgInRlcm1pbmF0aW9uIHJlcXVlc3RlZFxuIik7CisKK2Vycl90ZXJtaW5hdGU6CisJc3Bp bl91bmxvY2tfaXJxcmVzdG9yZSgmY2hhbi0+dmMubG9jaywgZmxhZ3MpOworCXJldHVybiBlcnI7 Cit9CisKK3N0YXRpYyB2b2lkIGR3X2VkbWFfZGV2aWNlX2lzc3VlX3BlbmRpbmcoc3RydWN0IGRt YV9jaGFuICpkY2hhbikKK3sKKwlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuID0gZGNoYW4yZHdf ZWRtYV9jaGFuKGRjaGFuKTsKKwl1bnNpZ25lZCBsb25nIGZsYWdzOworCisJc3Bpbl9sb2NrX2ly cXNhdmUoJmNoYW4tPnZjLmxvY2ssIGZsYWdzKTsKKworCWlmIChjaGFuLT5jb25maWd1cmVkICYm IGNoYW4tPnJlcXVlc3QgPT0gRURNQV9SRVFfTk9ORSAmJgorCSAgICBjaGFuLT5zdGF0dXMgPT0g RURNQV9TVF9JRExFICYmIHZjaGFuX2lzc3VlX3BlbmRpbmcoJmNoYW4tPnZjKSkgeworCQlkZXZf ZGJnKGRjaGFuMmRldihkY2hhbiksICJ0cmFuc2ZlciBpc3N1ZWRcbiIpOworCQljaGFuLT5zdGF0 dXMgPSBFRE1BX1NUX0JVU1k7CisJCWR3X2VkbWFfc3RhcnRfdHJhbnNmZXIoY2hhbik7CisJfQor CisJc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmY2hhbi0+dmMubG9jaywgZmxhZ3MpOworfQorCitz dGF0aWMgZW51bSBkbWFfc3RhdHVzCitkd19lZG1hX2RldmljZV90eF9zdGF0dXMoc3RydWN0IGRt YV9jaGFuICpkY2hhbiwgZG1hX2Nvb2tpZV90IGNvb2tpZSwKKwkJCSBzdHJ1Y3QgZG1hX3R4X3N0 YXRlICp0eHN0YXRlKQoreworCXN0cnVjdCBkd19lZG1hX2NoYW4gKmNoYW4gPSBkY2hhbjJkd19l ZG1hX2NoYW4oZGNoYW4pOworCWNvbnN0IHN0cnVjdCBkd19lZG1hX2NvcmVfb3BzICpvcHMgPSBj aGFuMm9wcyhjaGFuKTsKKwl1bnNpZ25lZCBsb25nIGZsYWdzOworCWVudW0gZG1hX3N0YXR1cyBy ZXQ7CisKKwlzcGluX2xvY2tfaXJxc2F2ZSgmY2hhbi0+dmMubG9jaywgZmxhZ3MpOworCisJcmV0 ID0gb3BzLT5jaF9zdGF0dXMoY2hhbik7CisJaWYgKHJldCA9PSBETUFfRVJST1IpIHsKKwkJZ290 byByZXRfc3RhdHVzOworCX0gZWxzZSBpZiAocmV0ID09IERNQV9JTl9QUk9HUkVTUykgeworCQlj aGFuLT5zdGF0dXMgPSBFRE1BX1NUX0JVU1k7CisJCWdvdG8gcmV0X3N0YXR1czsKKwl9IGVsc2Ug eworCQkvKiBETUFfQ09NUExFVEUgKi8KKwkJaWYgKGNoYW4tPnN0YXR1cyA9PSBFRE1BX1NUX1BB VVNFKQorCQkJcmV0ID0gRE1BX1BBVVNFRDsKKwkJZWxzZSBpZiAoY2hhbi0+c3RhdHVzID09IEVE TUFfU1RfQlVTWSkKKwkJCXJldCA9IERNQV9JTl9QUk9HUkVTUzsKKwkJZWxzZQorCQkJcmV0ID0g RE1BX0NPTVBMRVRFOworCX0KKworcmV0X3N0YXR1czoKKwlzcGluX3VubG9ja19pcnFyZXN0b3Jl KCZjaGFuLT52Yy5sb2NrLCBmbGFncyk7CisJZG1hX3NldF9yZXNpZHVlKHR4c3RhdGUsIDApOwor CisJcmV0dXJuIHJldDsKK30KKworc3RhdGljIHN0cnVjdCBkbWFfYXN5bmNfdHhfZGVzY3JpcHRv ciAqCitkd19lZG1hX2RldmljZV9wcmVwX3NsYXZlX3NnKHN0cnVjdCBkbWFfY2hhbiAqZGNoYW4s IHN0cnVjdCBzY2F0dGVybGlzdCAqc2dsLAorCQkJICAgICB1bnNpZ25lZCBpbnQgc2dfbGVuLAor CQkJICAgICBlbnVtIGRtYV90cmFuc2Zlcl9kaXJlY3Rpb24gZGlyZWN0aW9uLAorCQkJICAgICB1 bnNpZ25lZCBsb25nIGZsYWdzLCB2b2lkICpjb250ZXh0KQoreworCXN0cnVjdCBkd19lZG1hX2No YW4gKmNoYW4gPSBkY2hhbjJkd19lZG1hX2NoYW4oZGNoYW4pOworCXN0cnVjdCBkd19lZG1hX2Rl c2MgKmRlc2M7CisJc3RydWN0IGR3X2VkbWFfY2h1bmsgKmNodW5rOworCXN0cnVjdCBkd19lZG1h X2J1cnN0ICpidXJzdDsKKwlzdHJ1Y3Qgc2NhdHRlcmxpc3QgKnNnOworCXVuc2lnbmVkIGxvbmcg c2ZsYWdzOworCXBoeXNfYWRkcl90IHNyY19hZGRyOworCXBoeXNfYWRkcl90IGRzdF9hZGRyOwor CWludCBpOworCisJaWYgKHNnX2xlbiA8IDEpIHsKKwkJZGV2X2VycihjaGFuMmRldihjaGFuKSwg ImludmFsaWQgc2cgbGVuZ3RoICV1XG4iLCBzZ19sZW4pOworCQlyZXR1cm4gTlVMTDsKKwl9CisK KwlpZiAoZGlyZWN0aW9uID09IERNQV9ERVZfVE9fTUVNICYmIGNoYW4tPmRpciA9PSBFRE1BX0RJ Ul9XUklURSkgeworCQlkZXZfZGJnKGNoYW4yZGV2KGNoYW4pLAkicHJlcGFyZSBvcGVyYXRpb24g KFdSSVRFKVxuIik7CisJfSBlbHNlIGlmIChkaXJlY3Rpb24gPT0gRE1BX01FTV9UT19ERVYgJiYg Y2hhbi0+ZGlyID09IEVETUFfRElSX1JFQUQpIHsKKwkJZGV2X2RiZyhjaGFuMmRldihjaGFuKSwJ InByZXBhcmUgb3BlcmF0aW9uIChSRUFEKVxuIik7CisJfSBlbHNlIHsKKwkJZGV2X2VycihjaGFu MmRldihjaGFuKSwgImludmFsaWQgZGlyZWN0aW9uXG4iKTsKKwkJcmV0dXJuIE5VTEw7CisJfQor CisJaWYgKCFjaGFuLT5jb25maWd1cmVkKSB7CisJCWRldl9lcnIoY2hhbjJkZXYoY2hhbiksICIo cHJlcF9zbGF2ZV9zZykgY2hhbm5lbCBub3QgY29uZmlndXJlZFxuIik7CisJCXJldHVybiBOVUxM OworCX0KKworCWlmIChjaGFuLT5zdGF0dXMgIT0gRURNQV9TVF9JRExFKSB7CisJCWRldl9lcnIo Y2hhbjJkZXYoY2hhbiksICJjaGFubmVsIGlzIGJ1c3kgb3IgcGF1c2VkXG4iKTsKKwkJcmV0dXJu IE5VTEw7CisJfQorCisJc3Bpbl9sb2NrX2lycXNhdmUoJmNoYW4tPnZjLmxvY2ssIHNmbGFncyk7 CisKKwlkZXNjID0gZHdfZWRtYV9hbGxvY19kZXNjKGNoYW4pOworCWlmICh1bmxpa2VseSghZGVz YykpCisJCWdvdG8gZXJyX2FsbG9jOworCisJY2h1bmsgPSBkd19lZG1hX2FsbG9jX2NodW5rKGRl c2MpOworCWlmICh1bmxpa2VseSghY2h1bmspKQorCQlnb3RvIGVycl9hbGxvYzsKKworCXNyY19h ZGRyID0gY2hhbi0+c3JjX2FkZHI7CisJZHN0X2FkZHIgPSBjaGFuLT5kc3RfYWRkcjsKKworCWZv cl9lYWNoX3NnKHNnbCwgc2csIHNnX2xlbiwgaSkgeworCQlpZiAoY2h1bmstPmJ1cnN0c19hbGxv YyA9PSBjaGFuLT5sbF9tYXgpIHsKKwkJCWNodW5rID0gZHdfZWRtYV9hbGxvY19jaHVuayhkZXNj KTsKKwkJCWlmICh1bmxpa2VseSghY2h1bmspKQorCQkJCWdvdG8gZXJyX2FsbG9jOworCQl9CisK KwkJYnVyc3QgPSBkd19lZG1hX2FsbG9jX2J1cnN0KGNodW5rKTsKKworCQlpZiAodW5saWtlbHko IWJ1cnN0KSkKKwkJCWdvdG8gZXJyX2FsbG9jOworCisJCWJ1cnN0LT5zeiA9IHNnX2RtYV9sZW4o c2cpOworCQljaHVuay0+bGxfcmVnaW9uLnN6ICs9IGJ1cnN0LT5zejsKKwkJZGVzYy0+YWxsb2Nf c3ogKz0gYnVyc3QtPnN6OworCisJCWlmIChkaXJlY3Rpb24gPT0gRE1BX01FTV9UT19ERVYpIHsK KwkJCWJ1cnN0LT5zYXIgPSBzZ19kbWFfYWRkcmVzcyhzZyk7CisJCQlidXJzdC0+ZGFyID0gZHN0 X2FkZHI7CisJCQlkc3RfYWRkciArPSBzZ19kbWFfbGVuKHNnKTsKKwkJfSBlbHNlIHsKKwkJCWJ1 cnN0LT5zYXIgPSBzcmNfYWRkcjsKKwkJCWJ1cnN0LT5kYXIgPSBzZ19kbWFfYWRkcmVzcyhzZyk7 CisJCQlzcmNfYWRkciArPSBzZ19kbWFfbGVuKHNnKTsKKwkJfQorCisJCWRldl9kYmcoY2hhbjJk ZXYoY2hhbiksICJsbGkgJXUvJXUsIHNhcj0weCUuOGxseCwgZGFyPTB4JS44bGx4LCBzaXplPSV1 IGJ5dGVzXG4iLAorCQkJaSArIDEsIHNnX2xlbiwgYnVyc3QtPnNhciwgYnVyc3QtPmRhciwgYnVy c3QtPnN6KTsKKwl9CisKKwlzcGluX3VubG9ja19pcnFyZXN0b3JlKCZjaGFuLT52Yy5sb2NrLCBz ZmxhZ3MpOworCXJldHVybiB2Y2hhbl90eF9wcmVwKCZjaGFuLT52YywgJmRlc2MtPnZkLCBmbGFn cyk7CisKK2Vycl9hbGxvYzoKKwlpZiAoZGVzYykKKwkJZHdfZWRtYV9mcmVlX2Rlc2MoZGVzYyk7 CisJc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmY2hhbi0+dmMubG9jaywgc2ZsYWdzKTsKKwlyZXR1 cm4gTlVMTDsKK30KKworc3RhdGljIHN0cnVjdCBkbWFfYXN5bmNfdHhfZGVzY3JpcHRvciAqCitk d19lZG1hX2RldmljZV9wcmVwX2RtYV9jeWNsaWMoc3RydWN0IGRtYV9jaGFuICpkY2hhbiwgZG1h X2FkZHJfdCBidWZfYWRkciwKKwkJCSAgICAgICBzaXplX3QgbGVuLCBzaXplX3QgY3ljbGljX2Nu dCwKKwkJCSAgICAgICBlbnVtIGRtYV90cmFuc2Zlcl9kaXJlY3Rpb24gZGlyZWN0aW9uLAorCQkJ ICAgICAgIHVuc2lnbmVkIGxvbmcgZmxhZ3MpCit7CisJc3RydWN0IGR3X2VkbWFfY2hhbiAqY2hh biA9IGRjaGFuMmR3X2VkbWFfY2hhbihkY2hhbik7CisJc3RydWN0IGR3X2VkbWFfZGVzYyAqZGVz YzsKKwlzdHJ1Y3QgZHdfZWRtYV9jaHVuayAqY2h1bms7CisJc3RydWN0IGR3X2VkbWFfYnVyc3Qg KmJ1cnN0OworCXVuc2lnbmVkIGxvbmcgc2ZsYWdzOworCXBoeXNfYWRkcl90IHNyY19hZGRyOwor CXBoeXNfYWRkcl90IGRzdF9hZGRyOworCXUzMiBpOworCisJaWYgKCFsZW4gfHwgIWN5Y2xpY19j bnQpIHsKKwkJZGV2X2VycihjaGFuMmRldihjaGFuKSwgImludmFsaWQgbGVuIG9yIGN5Y2xpYyBj b3VudFxuIik7CisJCXJldHVybiBOVUxMOworCX0KKworCWlmIChkaXJlY3Rpb24gPT0gRE1BX0RF Vl9UT19NRU0gJiYgY2hhbi0+ZGlyID09IEVETUFfRElSX1dSSVRFKSB7CisJCWRldl9kYmcoY2hh bjJkZXYoY2hhbiksCSJwcmVwYXJlIG9wZXJhdGlvbiAoV1JJVEUpXG4iKTsKKwl9IGVsc2UgaWYg KGRpcmVjdGlvbiA9PSBETUFfTUVNX1RPX0RFViAmJiBjaGFuLT5kaXIgPT0gRURNQV9ESVJfUkVB RCkgeworCQlkZXZfZGJnKGNoYW4yZGV2KGNoYW4pLAkicHJlcGFyZSBvcGVyYXRpb24gKFJFQUQp XG4iKTsKKwl9IGVsc2UgeworCQlkZXZfZXJyKGNoYW4yZGV2KGNoYW4pLCAiaW52YWxpZCBkaXJl Y3Rpb25cbiIpOworCQlyZXR1cm4gTlVMTDsKKwl9CisKKwlpZiAoIWNoYW4tPmNvbmZpZ3VyZWQp IHsKKwkJZGV2X2VycihjaGFuMmRldihjaGFuKSwgIihwcmVwX2RtYV9jeWNsaWMpIGNoYW5uZWwg bm90IGNvbmZpZ3VyZWRcbiIpOworCQlyZXR1cm4gTlVMTDsKKwl9CisKKwlpZiAoY2hhbi0+c3Rh dHVzICE9IEVETUFfU1RfSURMRSkgeworCQlkZXZfZXJyKGNoYW4yZGV2KGNoYW4pLCAiY2hhbm5l bCBpcyBidXN5IG9yIHBhdXNlZFxuIik7CisJCXJldHVybiBOVUxMOworCX0KKworCXNwaW5fbG9j a19pcnFzYXZlKCZjaGFuLT52Yy5sb2NrLCBzZmxhZ3MpOworCisJZGVzYyA9IGR3X2VkbWFfYWxs b2NfZGVzYyhjaGFuKTsKKwlpZiAodW5saWtlbHkoIWRlc2MpKQorCQlnb3RvIGVycl9hbGxvYzsK KworCWNodW5rID0gZHdfZWRtYV9hbGxvY19jaHVuayhkZXNjKTsKKwlpZiAodW5saWtlbHkoIWNo dW5rKSkKKwkJZ290byBlcnJfYWxsb2M7CisKKwlzcmNfYWRkciA9IGNoYW4tPnNyY19hZGRyOwor CWRzdF9hZGRyID0gY2hhbi0+ZHN0X2FkZHI7CisKKwlmb3IgKGkgPSAwOyBpIDwgY3ljbGljX2Nu dDsgaSsrKSB7CisJCWlmIChjaHVuay0+YnVyc3RzX2FsbG9jID09IGNoYW4tPmxsX21heCkgewor CQkJY2h1bmsgPSBkd19lZG1hX2FsbG9jX2NodW5rKGRlc2MpOworCQkJaWYgKHVubGlrZWx5KCFj aHVuaykpCisJCQkJZ290byBlcnJfYWxsb2M7CisJCX0KKworCQlidXJzdCA9IGR3X2VkbWFfYWxs b2NfYnVyc3QoY2h1bmspOworCisJCWlmICh1bmxpa2VseSghYnVyc3QpKQorCQkJZ290byBlcnJf YWxsb2M7CisKKwkJYnVyc3QtPnN6ID0gbGVuOworCQljaHVuay0+bGxfcmVnaW9uLnN6ICs9IGJ1 cnN0LT5zejsKKwkJZGVzYy0+YWxsb2Nfc3ogKz0gYnVyc3QtPnN6OworCisJCWJ1cnN0LT5zYXIg PSBzcmNfYWRkcjsKKwkJYnVyc3QtPmRhciA9IGRzdF9hZGRyOworCisJCWRldl9kYmcoY2hhbjJk ZXYoY2hhbiksICJsbGkgJXUvJXUsIHNhcj0weCUuOGxseCwgZGFyPTB4JS44bGx4LCBzaXplPSV1 IGJ5dGVzXG4iLAorCQkJaSArIDEsIGN5Y2xpY19jbnQsIGJ1cnN0LT5zYXIsIGJ1cnN0LT5kYXIs IGJ1cnN0LT5zeik7CisJfQorCisJc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmY2hhbi0+dmMubG9j aywgc2ZsYWdzKTsKKwlyZXR1cm4gdmNoYW5fdHhfcHJlcCgmY2hhbi0+dmMsICZkZXNjLT52ZCwg ZmxhZ3MpOworCitlcnJfYWxsb2M6CisJaWYgKGRlc2MpCisJCWR3X2VkbWFfZnJlZV9kZXNjKGRl c2MpOworCXNwaW5fdW5sb2NrX2lycXJlc3RvcmUoJmNoYW4tPnZjLmxvY2ssIHNmbGFncyk7CisJ cmV0dXJuIE5VTEw7Cit9CisKK3N0YXRpYyB2b2lkIGR3X2VkbWFfZG9uZV9pbnRlcnJ1cHQoc3Ry dWN0IGR3X2VkbWFfY2hhbiAqY2hhbikKK3sKKwlzdHJ1Y3QgZHdfZWRtYSAqZHcgPSBjaGFuLT5j aGlwLT5kdzsKKwljb25zdCBzdHJ1Y3QgZHdfZWRtYV9jb3JlX29wcyAqb3BzID0gZHctPm9wczsK KwlzdHJ1Y3QgdmlydF9kbWFfZGVzYyAqdmQ7CisJc3RydWN0IGR3X2VkbWFfZGVzYyAqZGVzYzsK Kwl1bnNpZ25lZCBsb25nIGZsYWdzOworCisJb3BzLT5jbGVhcl9kb25lX2ludChjaGFuKTsKKwlk ZXZfZGJnKGNoYW4yZGV2KGNoYW4pLCAiY2xlYXIgZG9uZSBpbnRlcnJ1cHRcbiIpOworCisJc3Bp bl9sb2NrX2lycXNhdmUoJmNoYW4tPnZjLmxvY2ssIGZsYWdzKTsKKwl2ZCA9IHZjaGFuX25leHRf ZGVzYygmY2hhbi0+dmMpOworCXN3aXRjaCAoY2hhbi0+cmVxdWVzdCkgeworCWNhc2UgRURNQV9S RVFfTk9ORToKKwkJaWYgKCF2ZCkKKwkJCWJyZWFrOworCisJCWRlc2MgPSB2ZDJkd19lZG1hX2Rl c2ModmQpOworCQlpZiAoZGVzYy0+Y2h1bmtzX2FsbG9jKSB7CisJCQlkZXZfZGJnKGNoYW4yZGV2 KGNoYW4pLCAic3ViLXRyYW5zZmVyIGNvbXBsZXRlXG4iKTsKKwkJCWNoYW4tPnN0YXR1cyA9IEVE TUFfU1RfQlVTWTsKKwkJCWRldl9kYmcoY2hhbjJkZXYoY2hhbiksICJ0cmFuc2ZlcnJlZCAldSBi eXRlc1xuIiwKKwkJCQlkZXNjLT54ZmVyX3N6KTsKKwkJCWR3X2VkbWFfc3RhcnRfdHJhbnNmZXIo Y2hhbik7CisJCX0gZWxzZSB7CisJCQlsaXN0X2RlbCgmdmQtPm5vZGUpOworCQkJdmNoYW5fY29v a2llX2NvbXBsZXRlKHZkKTsKKwkJCWNoYW4tPnN0YXR1cyA9IEVETUFfU1RfSURMRTsKKwkJCWRl dl9kYmcoY2hhbjJkZXYoY2hhbiksICJ0cmFuc2ZlciBjb21wbGV0ZVxuIik7CisJCX0KKwkJYnJl YWs7CisJY2FzZSBFRE1BX1JFUV9TVE9QOgorCQlpZiAoIXZkKQorCQkJYnJlYWs7CisKKwkJbGlz dF9kZWwoJnZkLT5ub2RlKTsKKwkJdmNoYW5fY29va2llX2NvbXBsZXRlKHZkKTsKKwkJY2hhbi0+ cmVxdWVzdCA9IEVETUFfUkVRX05PTkU7CisJCWNoYW4tPnN0YXR1cyA9IEVETUFfU1RfSURMRTsK KwkJY2hhbi0+Y29uZmlndXJlZCA9IGZhbHNlOworCQlkZXZfZGJnKGNoYW4yZGV2KGNoYW4pLCAi dHJhbnNmZXIgc3RvcFxuIik7CisJCWJyZWFrOworCWNhc2UgRURNQV9SRVFfUEFVU0U6CisJCWNo YW4tPnJlcXVlc3QgPSBFRE1BX1JFUV9OT05FOworCQljaGFuLT5zdGF0dXMgPSBFRE1BX1NUX1BB VVNFOworCQlicmVhazsKKwlkZWZhdWx0OgorCQlkZXZfZXJyKGNoYW4yZGV2KGNoYW4pLCAiaW52 YWxpZCBzdGF0dXMgc3RhdGVcbiIpOworCQlicmVhazsKKwl9CisJc3Bpbl91bmxvY2tfaXJxcmVz dG9yZSgmY2hhbi0+dmMubG9jaywgZmxhZ3MpOworfQorCitzdGF0aWMgdm9pZCBkd19lZG1hX2Fi b3J0X2ludGVycnVwdChzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuKQoreworCXN0cnVjdCBkd19l ZG1hICpkdyA9IGNoYW4tPmNoaXAtPmR3OworCWNvbnN0IHN0cnVjdCBkd19lZG1hX2NvcmVfb3Bz ICpvcHMgPSBkdy0+b3BzOworCXN0cnVjdCB2aXJ0X2RtYV9kZXNjICp2ZDsKKwl1bnNpZ25lZCBs b25nIGZsYWdzOworCisJb3BzLT5jbGVhcl9hYm9ydF9pbnQoY2hhbik7CisJZGV2X2RiZyhjaGFu MmRldihjaGFuKSwgImNsZWFyIGFib3J0IGludGVycnVwdFxuIik7CisKKwlzcGluX2xvY2tfaXJx c2F2ZSgmY2hhbi0+dmMubG9jaywgZmxhZ3MpOworCXZkID0gdmNoYW5fbmV4dF9kZXNjKCZjaGFu LT52Yyk7CisJaWYgKHZkKSB7CisJCWxpc3RfZGVsKCZ2ZC0+bm9kZSk7CisJCXZjaGFuX2Nvb2tp ZV9jb21wbGV0ZSh2ZCk7CisJfQorCWNoYW4tPnJlcXVlc3QgPSBFRE1BX1JFUV9OT05FOworCWNo YW4tPnN0YXR1cyA9IEVETUFfU1RfSURMRTsKKworCXNwaW5fdW5sb2NrX2lycXJlc3RvcmUoJmNo YW4tPnZjLmxvY2ssIGZsYWdzKTsKK30KKworc3RhdGljIGlycXJldHVybl90IGR3X2VkbWFfaW50 ZXJydXB0X3dyaXRlKGludCBpcnEsIHZvaWQgKmRhdGEpCit7CisJc3RydWN0IGR3X2VkbWFfY2hp cCAqY2hpcCA9IGRhdGE7CisJc3RydWN0IGR3X2VkbWEgKmR3ID0gY2hpcC0+ZHc7CisJY29uc3Qg c3RydWN0IGR3X2VkbWFfY29yZV9vcHMgKm9wcyA9IGR3LT5vcHM7CisJdW5zaWduZWQgbG9uZyB0 b3QgPSBkdy0+d3JfY2hfY250OworCXVuc2lnbmVkIGxvbmcgcG9zID0gMDsKKwl1bnNpZ25lZCBs b25nIHZhbDsKKworCXBvcyA9IDA7CisJdmFsID0gb3BzLT5zdGF0dXNfZG9uZV9pbnQoZHcsIEVE TUFfRElSX1dSSVRFKTsKKwl3aGlsZSAoKHBvcyA9IGZpbmRfbmV4dF9iaXQoJnZhbCwgdG90LCBw b3MpKSAhPSB0b3QpIHsKKwkJc3RydWN0IGR3X2VkbWFfY2hhbiAqY2hhbiA9ICZkdy0+Y2hhbltw b3NdOworCisJCWR3X2VkbWFfZG9uZV9pbnRlcnJ1cHQoY2hhbik7CisJCXBvcysrOworCX0KKwor CXBvcyA9IDA7CisJdmFsID0gb3BzLT5zdGF0dXNfYWJvcnRfaW50KGR3LCBFRE1BX0RJUl9XUklU RSk7CisJd2hpbGUgKChwb3MgPSBmaW5kX25leHRfYml0KCZ2YWwsIHRvdCwgcG9zKSkgIT0gdG90 KSB7CisJCXN0cnVjdCBkd19lZG1hX2NoYW4gKmNoYW4gPSAmZHctPmNoYW5bcG9zXTsKKworCQlk d19lZG1hX2Fib3J0X2ludGVycnVwdChjaGFuKTsKKwkJcG9zKys7CisJfQorCisJcmV0dXJuIElS UV9IQU5ETEVEOworfQorCitzdGF0aWMgaXJxcmV0dXJuX3QgZHdfZWRtYV9pbnRlcnJ1cHRfcmVh ZChpbnQgaXJxLCB2b2lkICpkYXRhKQoreworCXN0cnVjdCBkd19lZG1hX2NoaXAgKmNoaXAgPSBk YXRhOworCXN0cnVjdCBkd19lZG1hICpkdyA9IGNoaXAtPmR3OworCWNvbnN0IHN0cnVjdCBkd19l ZG1hX2NvcmVfb3BzICpvcHMgPSBkdy0+b3BzOworCXVuc2lnbmVkIGxvbmcgdG90ID0gZHctPnJk X2NoX2NudDsKKwl1bnNpZ25lZCBsb25nIG9mZiA9IGR3LT53cl9jaF9jbnQ7CisJdW5zaWduZWQg bG9uZyBwb3MsIHZhbDsKKworCXBvcyA9IDA7CisJdmFsID0gb3BzLT5zdGF0dXNfZG9uZV9pbnQo ZHcsIEVETUFfRElSX1JFQUQpOworCXdoaWxlICgocG9zID0gZmluZF9uZXh0X2JpdCgmdmFsLCB0 b3QsIHBvcykpICE9IHRvdCkgeworCQlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuID0gJmR3LT5j aGFuW3BvcyArIG9mZl07CisKKwkJZHdfZWRtYV9kb25lX2ludGVycnVwdChjaGFuKTsKKwkJcG9z Kys7CisJfQorCisJcG9zID0gMDsKKwl2YWwgPSBvcHMtPnN0YXR1c19hYm9ydF9pbnQoZHcsIEVE TUFfRElSX1JFQUQpOworCXdoaWxlICgocG9zID0gZmluZF9uZXh0X2JpdCgmdmFsLCB0b3QsIHBv cykpICE9IHRvdCkgeworCQlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuID0gJmR3LT5jaGFuW3Bv cyArIG9mZl07CisKKwkJZHdfZWRtYV9hYm9ydF9pbnRlcnJ1cHQoY2hhbik7CisJCXBvcysrOwor CX0KKworCXJldHVybiBJUlFfSEFORExFRDsKK30KKworc3RhdGljIGlycXJldHVybl90IGR3X2Vk bWFfaW50ZXJydXB0X2FsbChpbnQgaXJxLCB2b2lkICpkYXRhKQoreworCWR3X2VkbWFfaW50ZXJy dXB0X3dyaXRlKGlycSwgZGF0YSk7CisJZHdfZWRtYV9pbnRlcnJ1cHRfcmVhZChpcnEsIGRhdGEp OworCisJcmV0dXJuIElSUV9IQU5ETEVEOworfQorCitzdGF0aWMgaW50IGR3X2VkbWFfYWxsb2Nf Y2hhbl9yZXNvdXJjZXMoc3RydWN0IGRtYV9jaGFuICpkY2hhbikKK3sKKwlzdHJ1Y3QgZHdfZWRt YV9jaGFuICpjaGFuID0gZGNoYW4yZHdfZWRtYV9jaGFuKGRjaGFuKTsKKworCWlmIChjaGFuLT5z dGF0dXMgIT0gRURNQV9TVF9JRExFKSB7CisJCWRldl9lcnIoY2hhbjJkZXYoY2hhbiksICJjaGFu bmVsIGlzIGJ1c3lcbiIpOworCQlyZXR1cm4gLUVCVVNZOworCX0KKworCWRldl9kYmcoZGNoYW4y ZGV2KGRjaGFuKSwgImFsbG9jYXRlZFxuIik7CisKKwlwbV9ydW50aW1lX2dldChjaGFuLT5jaGlw LT5kZXYpOworCisJcmV0dXJuIDA7Cit9CisKK3N0YXRpYyB2b2lkIGR3X2VkbWFfZnJlZV9jaGFu X3Jlc291cmNlcyhzdHJ1Y3QgZG1hX2NoYW4gKmRjaGFuKQoreworCXN0cnVjdCBkd19lZG1hX2No YW4gKmNoYW4gPSBkY2hhbjJkd19lZG1hX2NoYW4oZGNoYW4pOworCXVuc2lnbmVkIGxvbmcgdGlt ZW91dCA9IGppZmZpZXMgKyBtc2Vjc190b19qaWZmaWVzKDUwMDApOworCWludCByZXQ7CisKKwlp ZiAoY2hhbi0+c3RhdHVzICE9IEVETUFfU1RfSURMRSkKKwkJZGV2X2VycihjaGFuMmRldihjaGFu KSwgImNoYW5uZWwgaXMgYnVzeVxuIik7CisKKwlkbyB7CisJCXJldCA9IGR3X2VkbWFfZGV2aWNl X3Rlcm1pbmF0ZV9hbGwoZGNoYW4pOworCQlpZiAoIXJldCkKKwkJCWJyZWFrOworCisJCWlmICh0 aW1lX2FmdGVyX2VxKGppZmZpZXMsIHRpbWVvdXQpKSB7CisJCQlkZXZfZXJyKGNoYW4yZGV2KGNo YW4pLCAiZnJlZSB0aW1lb3V0XG4iKTsKKwkJCXJldHVybjsKKwkJfQorCisJCWNwdV9yZWxheCgp OworCX0gd2hpbGUgKDEpOworCisJZGV2X2RiZyhkY2hhbjJkZXYoZGNoYW4pLCAiY2hhbm5lbCBm cmVlZFxuIik7CisKKwlwbV9ydW50aW1lX3B1dChjaGFuLT5jaGlwLT5kZXYpOworfQorCitpbnQg ZHdfZWRtYV9wcm9iZShzdHJ1Y3QgZHdfZWRtYV9jaGlwICpjaGlwKQoreworCXN0cnVjdCBkd19l ZG1hICpkdyA9IGNoaXAtPmR3OworCXN0cnVjdCBkZXZpY2UgKmRldiA9IGNoaXAtPmRldjsKKwlj b25zdCBzdHJ1Y3QgZHdfZWRtYV9jb3JlX29wcyAqb3BzOworCXNpemVfdCBsbF9jaHVuayA9IGR3 LT5sbF9yZWdpb24uc3o7CisJc2l6ZV90IGR0X2NodW5rID0gZHctPmR0X3JlZ2lvbi5zejsKKwl1 MzIgY2hfdG90OworCWludCBpLCBqLCBlcnI7CisKKwlyYXdfc3Bpbl9sb2NrX2luaXQoJmR3LT5s b2NrKTsKKworCS8qIENhbGxiYWNrIG9wZXJhdGlvbiBzZWxlY3Rpb24gYWNjb3JkaW5nbHkgdG8g ZURNQSB2ZXJzaW9uICovCisJc3dpdGNoIChkdy0+dmVyc2lvbikgeworCWRlZmF1bHQ6CisJCWRl dl9lcnIoZGV2LCAidW5zdXBwb3J0ZWQgdmVyc2lvblxuIik7CisJCXJldHVybiAtRVBFUk07CisJ fQorCisJcG1fcnVudGltZV9nZXRfc3luYyhkZXYpOworCisJLyogRmluZCBvdXQgaG93IG1hbnkg d3JpdGUgY2hhbm5lbHMgYXJlIHN1cHBvcnRlZCBieSBoYXJkd2FyZSAqLworCWR3LT53cl9jaF9j bnQgPSBvcHMtPmNoX2NvdW50KGR3LCBFRE1BX0RJUl9XUklURSk7CisJaWYgKCFkdy0+d3JfY2hf Y250KSB7CisJCWRldl9lcnIoZGV2LCAiaW52YWxpZCBudW1iZXIgb2Ygd3JpdGUgY2hhbm5lbHMo MClcbiIpOworCQlyZXR1cm4gLUVJTlZBTDsKKwl9CisKKwkvKiBGaW5kIG91dCBob3cgbWFueSBy ZWFkIGNoYW5uZWxzIGFyZSBzdXBwb3J0ZWQgYnkgaGFyZHdhcmUgKi8KKwlkdy0+cmRfY2hfY250 ID0gb3BzLT5jaF9jb3VudChkdywgRURNQV9ESVJfUkVBRCk7CisJaWYgKCFkdy0+cmRfY2hfY250 KSB7CisJCWRldl9lcnIoZGV2LCAiaW52YWxpZCBudW1iZXIgb2YgcmVhZCBjaGFubmVscygwKVxu Iik7CisJCXJldHVybiAtRUlOVkFMOworCX0KKworCWRldl9kYmcoZGV2LCAiQ2hhbm5lbHM6XHR3 cml0ZT0lZCwgcmVhZD0lZFxuIiwKKwkJZHctPndyX2NoX2NudCwgZHctPnJkX2NoX2NudCk7CisK KwljaF90b3QgPSBkdy0+d3JfY2hfY250ICsgZHctPnJkX2NoX2NudDsKKworCS8qIEFsbG9jYXRl IGNoYW5uZWxzICovCisJZHctPmNoYW4gPSBkZXZtX2tjYWxsb2MoZGV2LCBjaF90b3QsIHNpemVv ZigqZHctPmNoYW4pLCBHRlBfS0VSTkVMKTsKKwlpZiAoIWR3LT5jaGFuKQorCQlyZXR1cm4gLUVO T01FTTsKKworCS8qIENhbGN1bGF0ZSB0aGUgbGlua2VkIGxpc3QgY2h1bmsgZm9yIGVhY2ggY2hh bm5lbCAqLworCWxsX2NodW5rIC89IHJvdW5kdXBfcG93X29mX3R3byhjaF90b3QpOworCisJLyog Q2FsY3VsYXRlIHRoZSBsaW5rZWQgbGlzdCBjaHVuayBmb3IgZWFjaCBjaGFubmVsICovCisJZHRf Y2h1bmsgLz0gcm91bmR1cF9wb3dfb2ZfdHdvKGNoX3RvdCk7CisKKwkvKiBEaXNhYmxlIGVETUEs IG9ubHkgdG8gZXN0YWJsaXNoIHRoZSBpZGVhbCBpbml0aWFsIGNvbmRpdGlvbnMgKi8KKwlvcHMt Pm9mZihkdyk7CisKKwlzbnByaW50Zihkdy0+bmFtZSwgc2l6ZW9mKGR3LT5uYW1lKSwgImR3LWVk bWEtY29yZTolZCIsIGNoaXAtPmlkKTsKKworCS8qIFJlcXVlc3QgSVJRcyAqLworCWlmIChkdy0+ bnJfaXJxcyAhPSAxKSB7CisJCWRldl9lcnIoZGV2LCAiaW52YWxpZCBudW1iZXIgb2YgaXJxcyAo JXUpXG4iLCBkdy0+bnJfaXJxcyk7CisJCXJldHVybiAtRUlOVkFMOworCX0KKworCWZvciAoaSA9 IDA7IGkgPCBkdy0+bnJfaXJxczsgaSsrKSB7CisJCWVyciA9IGRldm1fcmVxdWVzdF9pcnEoZGV2 LCBwY2lfaXJxX3ZlY3Rvcih0b19wY2lfZGV2KGRldiksIGkpLAorCQkJCSAgICAgICBkd19lZG1h X2ludGVycnVwdF9hbGwsCisJCQkJICAgICAgIElSUUZfU0hBUkVELCBkdy0+bmFtZSwgY2hpcCk7 CisJCWlmIChlcnIpCisJCQlyZXR1cm4gZXJyOworCX0KKworCS8qIENyZWF0ZSB3cml0ZSBjaGFu bmVscyAqLworCUlOSVRfTElTVF9IRUFEKCZkdy0+d3JfZWRtYS5jaGFubmVscyk7CisJZm9yIChp ID0gMDsgaSA8IGR3LT53cl9jaF9jbnQ7IGkrKykgeworCQlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpj aGFuID0gJmR3LT5jaGFuW2ldOworCQlzdHJ1Y3QgZHdfZWRtYV9yZWdpb24gKmR0X3JlZ2lvbjsK KworCQlkdF9yZWdpb24gPSBkZXZtX2t6YWxsb2MoZGV2LCBzaXplb2YoKmR0X3JlZ2lvbiksIEdG UF9LRVJORUwpOworCQlpZiAoIWR0X3JlZ2lvbikKKwkJCXJldHVybiAtRU5PTUVNOworCisJCWNo YW4tPnZjLmNoYW4ucHJpdmF0ZSA9IGR0X3JlZ2lvbjsKKworCQljaGFuLT5jaGlwID0gY2hpcDsK KwkJY2hhbi0+aWQgPSBpOworCQljaGFuLT5kaXIgPSBFRE1BX0RJUl9XUklURTsKKwkJY2hhbi0+ Y29uZmlndXJlZCA9IGZhbHNlOworCQljaGFuLT5yZXF1ZXN0ID0gRURNQV9SRVFfTk9ORTsKKwkJ Y2hhbi0+c3RhdHVzID0gRURNQV9TVF9JRExFOworCisJCWNoYW4tPmxsX29mZiA9IChsbF9jaHVu ayAqIGkpOworCQljaGFuLT5sbF9tYXggPSAobGxfY2h1bmsgLyBFRE1BX0xMX1NaKSAtIDE7CisK KwkJY2hhbi0+ZHRfb2ZmID0gKGR0X2NodW5rICogaSk7CisKKwkJZGV2X2RiZyhkZXYsICJMLiBM aXN0Olx0Q2hhbm5lbCB3cml0ZVsldV0gb2ZmPTB4JS44bHgsIG1heF9jbnQ9JXVcbiIsCisJCQlp LCBjaGFuLT5sbF9vZmYsIGNoYW4tPmxsX21heCk7CisKKwkJbWVtY3B5KCZjaGFuLT5tc2ksICZk dy0+bXNpWzBdLCBzaXplb2YoY2hhbi0+bXNpKSk7CisKKwkJZGV2X2RiZyhkZXYsICJNU0k6XHRc dENoYW5uZWwgd3JpdGVbJXVdIGFkZHI9MHglLjh4JS44eCwgZGF0YT0weCUuOHhcbiIsCisJCQlp LCBjaGFuLT5tc2kuYWRkcmVzc19oaSwgY2hhbi0+bXNpLmFkZHJlc3NfbG8sCisJCQljaGFuLT5t c2kuZGF0YSk7CisKKwkJY2hhbi0+dmMuZGVzY19mcmVlID0gdmNoYW5fZnJlZV9kZXNjOworCQl2 Y2hhbl9pbml0KCZjaGFuLT52YywgJmR3LT53cl9lZG1hKTsKKworCQlkdF9yZWdpb24tPnBhZGRy ID0gZHctPmR0X3JlZ2lvbi5wYWRkciArIGNoYW4tPmR0X29mZjsKKwkJZHRfcmVnaW9uLT52YWRk ciA9IGR3LT5kdF9yZWdpb24udmFkZHIgKyBjaGFuLT5kdF9vZmY7CisJCWR0X3JlZ2lvbi0+c3og PSBkdF9jaHVuazsKKworCQlkZXZfZGJnKGRldiwgIkRhdGE6XHRDaGFubmVsIHdyaXRlWyV1XSBv ZmY9MHglLjhseFxuIiwKKwkJCWksIGNoYW4tPmR0X29mZik7CisJfQorCWRtYV9jYXBfemVybyhk dy0+d3JfZWRtYS5jYXBfbWFzayk7CisJZG1hX2NhcF9zZXQoRE1BX1NMQVZFLCBkdy0+d3JfZWRt YS5jYXBfbWFzayk7CisJZG1hX2NhcF9zZXQoRE1BX0NZQ0xJQywgZHctPndyX2VkbWEuY2FwX21h c2spOworCWR3LT53cl9lZG1hLmRpcmVjdGlvbnMgPSBCSVQoRE1BX0RFVl9UT19NRU0pOworCWR3 LT53cl9lZG1hLmNoYW5jbnQgPSBkdy0+d3JfY2hfY250OworCisJLyogQ3JlYXRlIHJlYWQgY2hh bm5lbHMgKi8KKwlJTklUX0xJU1RfSEVBRCgmZHctPnJkX2VkbWEuY2hhbm5lbHMpOworCWZvciAo aiA9IDA7IGogPCBkdy0+cmRfY2hfY250OyBqKyssIGkrKykgeworCQlzdHJ1Y3QgZHdfZWRtYV9j aGFuICpjaGFuID0gJmR3LT5jaGFuW2ldOworCQlzdHJ1Y3QgZHdfZWRtYV9yZWdpb24gKmR0X3Jl Z2lvbjsKKworCQlkdF9yZWdpb24gPSBkZXZtX2t6YWxsb2MoZGV2LCBzaXplb2YoKmR0X3JlZ2lv biksIEdGUF9LRVJORUwpOworCQlpZiAoIWR0X3JlZ2lvbikKKwkJCXJldHVybiAtRU5PTUVNOwor CisJCWNoYW4tPnZjLmNoYW4ucHJpdmF0ZSA9IGR0X3JlZ2lvbjsKKworCQljaGFuLT5jaGlwID0g Y2hpcDsKKwkJY2hhbi0+aWQgPSBqOworCQljaGFuLT5kaXIgPSBFRE1BX0RJUl9SRUFEOworCQlj aGFuLT5jb25maWd1cmVkID0gZmFsc2U7CisJCWNoYW4tPnJlcXVlc3QgPSBFRE1BX1JFUV9OT05F OworCQljaGFuLT5zdGF0dXMgPSBFRE1BX1NUX0lETEU7CisKKwkJY2hhbi0+bGxfb2ZmID0gKGxs X2NodW5rICogaSk7CisJCWNoYW4tPmxsX21heCA9IChsbF9jaHVuayAvIEVETUFfTExfU1opIC0g MTsKKworCQljaGFuLT5kdF9vZmYgPSAoZHRfY2h1bmsgKiBpKTsKKworCQlkZXZfZGJnKGRldiwg IkwuIExpc3Q6XHRDaGFubmVsIHJlYWRbJXVdIG9mZj0weCUuOGx4LCBtYXhfY250PSV1XG4iLAor CQkJaiwgY2hhbi0+bGxfb2ZmLCBjaGFuLT5sbF9tYXgpOworCisJCW1lbWNweSgmY2hhbi0+bXNp LCAmZHctPm1zaVswXSwgc2l6ZW9mKGNoYW4tPm1zaSkpOworCisJCWRldl9kYmcoZGV2LCAiTVNJ Olx0XHRDaGFubmVsIHJlYWRbJXVdIGFkZHI9MHglLjh4JS44eCwgZGF0YT0weCUuOHhcbiIsCisJ CQlqLCBjaGFuLT5tc2kuYWRkcmVzc19oaSwgY2hhbi0+bXNpLmFkZHJlc3NfbG8sCisJCQljaGFu LT5tc2kuZGF0YSk7CisKKwkJY2hhbi0+dmMuZGVzY19mcmVlID0gdmNoYW5fZnJlZV9kZXNjOwor CQl2Y2hhbl9pbml0KCZjaGFuLT52YywgJmR3LT5yZF9lZG1hKTsKKworCQlkdF9yZWdpb24tPnBh ZGRyID0gZHctPmR0X3JlZ2lvbi5wYWRkciArIGNoYW4tPmR0X29mZjsKKwkJZHRfcmVnaW9uLT52 YWRkciA9IGR3LT5kdF9yZWdpb24udmFkZHIgKyBjaGFuLT5kdF9vZmY7CisJCWR0X3JlZ2lvbi0+ c3ogPSBkdF9jaHVuazsKKworCQlkZXZfZGJnKGRldiwgIkRhdGE6XHRDaGFubmVsIHJlYWRbJXVd IG9mZj0weCUuOGx4XG4iLAorCQkJaSwgY2hhbi0+ZHRfb2ZmKTsKKwl9CisJZG1hX2NhcF96ZXJv KGR3LT5yZF9lZG1hLmNhcF9tYXNrKTsKKwlkbWFfY2FwX3NldChETUFfU0xBVkUsIGR3LT5yZF9l ZG1hLmNhcF9tYXNrKTsKKwlkbWFfY2FwX3NldChETUFfQ1lDTElDLCBkdy0+cmRfZWRtYS5jYXBf bWFzayk7CisJZHctPnJkX2VkbWEuZGlyZWN0aW9ucyA9IEJJVChETUFfTUVNX1RPX0RFVik7CisJ ZHctPnJkX2VkbWEuY2hhbmNudCA9IGR3LT5yZF9jaF9jbnQ7CisKKwkvKiBTZXQgRE1BIGNoYW5u ZWxzICBjYXBhYmlsaXRpZXMgKi8KKwlTRVRfQk9USF9DSChzcmNfYWRkcl93aWR0aHMsIEJJVChE TUFfU0xBVkVfQlVTV0lEVEhfNF9CWVRFUykpOworCVNFVF9CT1RIX0NIKGRzdF9hZGRyX3dpZHRo cywgQklUKERNQV9TTEFWRV9CVVNXSURUSF80X0JZVEVTKSk7CisJU0VUX0JPVEhfQ0gocmVzaWR1 ZV9ncmFudWxhcml0eSwgRE1BX1JFU0lEVUVfR1JBTlVMQVJJVFlfREVTQ1JJUFRPUik7CisKKwlT RVRfQk9USF9DSChkZXYsIGRldik7CisKKwlTRVRfQk9USF9DSChkZXZpY2VfYWxsb2NfY2hhbl9y ZXNvdXJjZXMsIGR3X2VkbWFfYWxsb2NfY2hhbl9yZXNvdXJjZXMpOworCVNFVF9CT1RIX0NIKGRl dmljZV9mcmVlX2NoYW5fcmVzb3VyY2VzLCBkd19lZG1hX2ZyZWVfY2hhbl9yZXNvdXJjZXMpOwor CisJU0VUX0JPVEhfQ0goZGV2aWNlX2NvbmZpZywgZHdfZWRtYV9kZXZpY2VfY29uZmlnKTsKKwlT RVRfQk9USF9DSChkZXZpY2VfcGF1c2UsIGR3X2VkbWFfZGV2aWNlX3BhdXNlKTsKKwlTRVRfQk9U SF9DSChkZXZpY2VfcmVzdW1lLCBkd19lZG1hX2RldmljZV9yZXN1bWUpOworCVNFVF9CT1RIX0NI KGRldmljZV90ZXJtaW5hdGVfYWxsLCBkd19lZG1hX2RldmljZV90ZXJtaW5hdGVfYWxsKTsKKwlT RVRfQk9USF9DSChkZXZpY2VfaXNzdWVfcGVuZGluZywgZHdfZWRtYV9kZXZpY2VfaXNzdWVfcGVu ZGluZyk7CisJU0VUX0JPVEhfQ0goZGV2aWNlX3R4X3N0YXR1cywgZHdfZWRtYV9kZXZpY2VfdHhf c3RhdHVzKTsKKwlTRVRfQk9USF9DSChkZXZpY2VfcHJlcF9zbGF2ZV9zZywgZHdfZWRtYV9kZXZp Y2VfcHJlcF9zbGF2ZV9zZyk7CisJU0VUX0JPVEhfQ0goZGV2aWNlX3ByZXBfZG1hX2N5Y2xpYywg ZHdfZWRtYV9kZXZpY2VfcHJlcF9kbWFfY3ljbGljKTsKKworCS8qIFBvd2VyIG1hbmFnZW1lbnQg Ki8KKwlwbV9ydW50aW1lX2VuYWJsZShkZXYpOworCisJLyogUmVnaXN0ZXIgRE1BIGRldmljZSAq LworCWVyciA9IGRtYV9hc3luY19kZXZpY2VfcmVnaXN0ZXIoJmR3LT53cl9lZG1hKTsKKwlpZiAo ZXJyKQorCQlnb3RvIGVycl9wbV9kaXNhYmxlOworCisJZXJyID0gZG1hX2FzeW5jX2RldmljZV9y ZWdpc3RlcigmZHctPnJkX2VkbWEpOworCWlmIChlcnIpCisJCWdvdG8gZXJyX3BtX2Rpc2FibGU7 CisKKwkvKiBUdXJuIGRlYnVnZnMgb24gKi8KKwllcnIgPSBvcHMtPmRlYnVnZnNfb24oY2hpcCk7 CisJaWYgKGVycikgeworCQlkZXZfZXJyKGRldiwgInVuYWJsZSB0byBjcmVhdGUgZGVidWdmcyBz dHJ1Y3R1cmVcbiIpOworCQlnb3RvIGVycl9wbV9kaXNhYmxlOworCX0KKworCWRldl9pbmZvKGRl diwgIkRlc2lnbldhcmUgZURNQSBjb250cm9sbGVyIGRyaXZlciBsb2FkZWQgY29tcGxldGVseVxu Iik7CisKKwlyZXR1cm4gMDsKKworZXJyX3BtX2Rpc2FibGU6CisJcG1fcnVudGltZV9kaXNhYmxl KGRldik7CisKKwlyZXR1cm4gZXJyOworfQorRVhQT1JUX1NZTUJPTF9HUEwoZHdfZWRtYV9wcm9i ZSk7CisKK2ludCBkd19lZG1hX3JlbW92ZShzdHJ1Y3QgZHdfZWRtYV9jaGlwICpjaGlwKQorewor CXN0cnVjdCBkd19lZG1hICpkdyA9IGNoaXAtPmR3OworCXN0cnVjdCBkZXZpY2UgKmRldiA9IGNo aXAtPmRldjsKKwljb25zdCBzdHJ1Y3QgZHdfZWRtYV9jb3JlX29wcyAqb3BzID0gZHctPm9wczsK KwlzdHJ1Y3QgZHdfZWRtYV9jaGFuICpjaGFuLCAqX2NoYW47CisJaW50IGk7CisKKwkvKiBEaXNh YmxlIGVETUEgKi8KKwlpZiAob3BzKQorCQlvcHMtPm9mZihkdyk7CisKKwkvKiBGcmVlIGlycXMg Ki8KKwlmb3IgKGkgPSAwOyBpIDwgZHctPm5yX2lycXM7IGkrKykgeworCQlpZiAocGNpX2lycV92 ZWN0b3IodG9fcGNpX2RldihkZXYpLCBpKSA8IDApCisJCQljb250aW51ZTsKKworCQlkZXZtX2Zy ZWVfaXJxKGRldiwgcGNpX2lycV92ZWN0b3IodG9fcGNpX2RldihkZXYpLCBpKSwgY2hpcCk7CisJ fQorCisJLyogUG93ZXIgbWFuYWdlbWVudCAqLworCXBtX3J1bnRpbWVfZGlzYWJsZShkZXYpOwor CisJbGlzdF9mb3JfZWFjaF9lbnRyeV9zYWZlKGNoYW4sIF9jaGFuLCAmZHctPndyX2VkbWEuY2hh bm5lbHMsCisJCQkJIHZjLmNoYW4uZGV2aWNlX25vZGUpIHsKKwkJbGlzdF9kZWwoJmNoYW4tPnZj LmNoYW4uZGV2aWNlX25vZGUpOworCQl0YXNrbGV0X2tpbGwoJmNoYW4tPnZjLnRhc2spOworCX0K KworCWxpc3RfZm9yX2VhY2hfZW50cnlfc2FmZShjaGFuLCBfY2hhbiwgJmR3LT5yZF9lZG1hLmNo YW5uZWxzLAorCQkJCSB2Yy5jaGFuLmRldmljZV9ub2RlKSB7CisJCWxpc3RfZGVsKCZjaGFuLT52 Yy5jaGFuLmRldmljZV9ub2RlKTsKKwkJdGFza2xldF9raWxsKCZjaGFuLT52Yy50YXNrKTsKKwl9 CisKKwkvKiBEZXJlZ2lzdGVyIGVETUEgZGV2aWNlICovCisJZG1hX2FzeW5jX2RldmljZV91bnJl Z2lzdGVyKCZkdy0+d3JfZWRtYSk7CisJZG1hX2FzeW5jX2RldmljZV91bnJlZ2lzdGVyKCZkdy0+ cmRfZWRtYSk7CisKKwkvKiBUdXJuIGRlYnVnZnMgb2ZmICovCisJaWYgKG9wcykKKwkJb3BzLT5k ZWJ1Z2ZzX29mZigpOworCisJZGV2X2luZm8oZGV2LCAiRGVzaWduV2FyZSBlRE1BIGNvbnRyb2xs ZXIgZHJpdmVyIHVubG9hZGVkIGNvbXBsZXRlXG4iKTsKKworCXJldHVybiAwOworfQorRVhQT1JU X1NZTUJPTF9HUEwoZHdfZWRtYV9yZW1vdmUpOworCitNT0RVTEVfTElDRU5TRSgiR1BMIHYyIik7 CitNT0RVTEVfREVTQ1JJUFRJT04oIlN5bm9wc3lzIERlc2lnbldhcmUgZURNQSBjb250cm9sbGVy IGNvcmUgZHJpdmVyIik7CitNT0RVTEVfQVVUSE9SKCJHdXN0YXZvIFBpbWVudGVsIDxndXN0YXZv LnBpbWVudGVsQHN5bm9wc3lzLmNvbT4iKTsKZGlmZiAtLWdpdCBhL2RyaXZlcnMvZG1hL2R3LWVk bWEvZHctZWRtYS1jb3JlLmggYi9kcml2ZXJzL2RtYS9kdy1lZG1hL2R3LWVkbWEtY29yZS5oCm5l dyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAuLjJkOThhMTAKLS0tIC9kZXYvbnVsbAor KysgYi9kcml2ZXJzL2RtYS9kdy1lZG1hL2R3LWVkbWEtY29yZS5oCkBAIC0wLDAgKzEsMTUxIEBA CisvKiBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMCAqLworLyoKKyAqIENvcHlyaWdo dCAoYykgMjAxOCBTeW5vcHN5cywgSW5jLiBhbmQvb3IgaXRzIGFmZmlsaWF0ZXMuCisgKiBTeW5v cHN5cyBEZXNpZ25XYXJlIGVETUEgY29yZSBkcml2ZXIKKyAqLworCisjaWZuZGVmIF9EV19FRE1B X0NPUkVfSAorI2RlZmluZSBfRFdfRURNQV9DT1JFX0gKKworI2luY2x1ZGUgPGxpbnV4L21zaS5o PgorI2luY2x1ZGUgPGxpbnV4L2RtYS9lZG1hLmg+CisKKyNpbmNsdWRlICIuLi92aXJ0LWRtYS5o IgorCisjZGVmaW5lIEVETUFfTExfU1oJCQkJCTI0CisKK2VudW0gZHdfZWRtYV9kaXIgeworCUVE TUFfRElSX1dSSVRFID0gMCwKKwlFRE1BX0RJUl9SRUFECit9OworCitlbnVtIGR3X2VkbWFfbW9k ZSB7CisJRURNQV9NT0RFX0xFR0FDWSA9IDAsCisJRURNQV9NT0RFX1VOUk9MTAorfTsKKworZW51 bSBkd19lZG1hX3JlcXVlc3QgeworCUVETUFfUkVRX05PTkUgPSAwLAorCUVETUFfUkVRX1NUT1As CisJRURNQV9SRVFfUEFVU0UKK307CisKK2VudW0gZHdfZWRtYV9zdGF0dXMgeworCUVETUFfU1Rf SURMRSA9IDAsCisJRURNQV9TVF9QQVVTRSwKKwlFRE1BX1NUX0JVU1kKK307CisKK3N0cnVjdCBk d19lZG1hX2NoYW47CitzdHJ1Y3QgZHdfZWRtYV9jaHVuazsKKworc3RydWN0IGR3X2VkbWFfY29y ZV9vcHMgeworCS8qIGVETUEgbWFuYWdlbWVudCBjYWxsYmFja3MgKi8KKwl2b2lkICgqb2ZmKShz dHJ1Y3QgZHdfZWRtYSAqZHcpOworCXUxNiAoKmNoX2NvdW50KShzdHJ1Y3QgZHdfZWRtYSAqZHcs IGVudW0gZHdfZWRtYV9kaXIgZGlyKTsKKwllbnVtIGRtYV9zdGF0dXMgKCpjaF9zdGF0dXMpKHN0 cnVjdCBkd19lZG1hX2NoYW4gKmNoYW4pOworCXZvaWQgKCpjbGVhcl9kb25lX2ludCkoc3RydWN0 IGR3X2VkbWFfY2hhbiAqY2hhbik7CisJdm9pZCAoKmNsZWFyX2Fib3J0X2ludCkoc3RydWN0IGR3 X2VkbWFfY2hhbiAqY2hhbik7CisJdTMyICgqc3RhdHVzX2RvbmVfaW50KShzdHJ1Y3QgZHdfZWRt YSAqY2hhbiwgZW51bSBkd19lZG1hX2RpciBkaXIpOworCXUzMiAoKnN0YXR1c19hYm9ydF9pbnQp KHN0cnVjdCBkd19lZG1hICpjaGFuLCBlbnVtIGR3X2VkbWFfZGlyIGRpcik7CisJdm9pZCAoKnN0 YXJ0KShzdHJ1Y3QgZHdfZWRtYV9jaHVuayAqY2h1bmssIGJvb2wgZmlyc3QpOworCWludCAoKmRl dmljZV9jb25maWcpKHN0cnVjdCBkbWFfY2hhbiAqZGNoYW4pOworCS8qIGVETUEgZGVidWcgZnMg Y2FsbGJhY2tzICovCisJaW50ICgqZGVidWdmc19vbikoc3RydWN0IGR3X2VkbWFfY2hpcCAqY2hp cCk7CisJdm9pZCAoKmRlYnVnZnNfb2ZmKSh2b2lkKTsKK307CisKK3N0cnVjdCBkd19lZG1hX2J1 cnN0IHsKKwlzdHJ1Y3QgbGlzdF9oZWFkCQlsaXN0OworCXU2NAkJCQlzYXI7CisJdTY0CQkJCWRh cjsKKwl1MzIJCQkJc3o7Cit9OworCitzdHJ1Y3QgZHdfZWRtYV9yZWdpb24geworCXBoeXNfYWRk cl90CQkJcGFkZHI7CisJZG1hX2FkZHJfdAkJCXZhZGRyOworCXNpemVfdAkJCQlzejsKK307CisK K3N0cnVjdCBkd19lZG1hX2NodW5rIHsKKwlzdHJ1Y3QgbGlzdF9oZWFkCQlsaXN0OworCXN0cnVj dCBkd19lZG1hX2NoYW4JCSpjaGFuOworCXN0cnVjdCBkd19lZG1hX2J1cnN0CQkqYnVyc3Q7CisK Kwl1MzIJCQkJYnVyc3RzX2FsbG9jOworCisJdTgJCQkJY2I7CisJc3RydWN0IGR3X2VkbWFfcmVn aW9uCQlsbF9yZWdpb247CS8qIExpbmtlZCBsaXN0ICovCit9OworCitzdHJ1Y3QgZHdfZWRtYV9k ZXNjIHsKKwlzdHJ1Y3QgdmlydF9kbWFfZGVzYwkJdmQ7CisJc3RydWN0IGR3X2VkbWFfY2hhbgkJ KmNoYW47CisJc3RydWN0IGR3X2VkbWFfY2h1bmsJCSpjaHVuazsKKworCXUzMgkJCQljaHVua3Nf YWxsb2M7CisKKwl1MzIJCQkJYWxsb2Nfc3o7CisJdTMyCQkJCXhmZXJfc3o7Cit9OworCitzdHJ1 Y3QgZHdfZWRtYV9jaGFuIHsKKwlzdHJ1Y3QgdmlydF9kbWFfY2hhbgkJdmM7CisJc3RydWN0IGR3 X2VkbWFfY2hpcAkJKmNoaXA7CisJaW50CQkJCWlkOworCWVudW0gZHdfZWRtYV9kaXIJCWRpcjsK KworCW9mZl90CQkJCWxsX29mZjsKKwl1MzIJCQkJbGxfbWF4OworCisJb2ZmX3QJCQkJZHRfb2Zm OworCisJc3RydWN0IG1zaV9tc2cJCQltc2k7CisKKwllbnVtIGR3X2VkbWFfcmVxdWVzdAkJcmVx dWVzdDsKKwllbnVtIGR3X2VkbWFfc3RhdHVzCQlzdGF0dXM7CisJdTgJCQkJY29uZmlndXJlZDsK KworCXBoeXNfYWRkcl90CQkJc3JjX2FkZHI7CisJcGh5c19hZGRyX3QJCQlkc3RfYWRkcjsKK307 CisKK3N0cnVjdCBkd19lZG1hIHsKKwljaGFyCQkJCW5hbWVbMjBdOworCisJc3RydWN0IGRtYV9k ZXZpY2UJCXdyX2VkbWE7CisJdTE2CQkJCXdyX2NoX2NudDsKKworCXN0cnVjdCBkbWFfZGV2aWNl CQlyZF9lZG1hOworCXUxNgkJCQlyZF9jaF9jbnQ7CisKKwlzdHJ1Y3QgZHdfZWRtYV9yZWdpb24J CXJnX3JlZ2lvbjsJLyogUmVnaXN0ZXJzICovCisJc3RydWN0IGR3X2VkbWFfcmVnaW9uCQlsbF9y ZWdpb247CS8qIExpbmtlZCBsaXN0ICovCisJc3RydWN0IGR3X2VkbWFfcmVnaW9uCQlkdF9yZWdp b247CS8qIERhdGEgKi8KKworCXN0cnVjdCBtc2lfbXNnCQkJKm1zaTsKKwlpbnQJCQkJbnJfaXJx czsKKworCXUzMgkJCQl2ZXJzaW9uOworCWVudW0gZHdfZWRtYV9tb2RlCQltb2RlOworCisJc3Ry dWN0IGR3X2VkbWFfY2hhbgkJKmNoYW47CisJY29uc3Qgc3RydWN0IGR3X2VkbWFfY29yZV9vcHMJ Km9wczsKKworCXJhd19zcGlubG9ja190CQkJbG9jazsJCS8qIE9ubHkgZm9yIGxlZ2FjeSAqLwor fTsKKworc3RhdGljIGlubGluZQorc3RydWN0IGR3X2VkbWFfY2hhbiAqdmMyZHdfZWRtYV9jaGFu KHN0cnVjdCB2aXJ0X2RtYV9jaGFuICp2YykKK3sKKwlyZXR1cm4gY29udGFpbmVyX29mKHZjLCBz dHJ1Y3QgZHdfZWRtYV9jaGFuLCB2Yyk7Cit9CisKK3N0YXRpYyBpbmxpbmUKK3N0cnVjdCBkd19l ZG1hX2NoYW4gKmRjaGFuMmR3X2VkbWFfY2hhbihzdHJ1Y3QgZG1hX2NoYW4gKmRjaGFuKQorewor CXJldHVybiB2YzJkd19lZG1hX2NoYW4odG9fdmlydF9jaGFuKGRjaGFuKSk7Cit9CisKKyNlbmRp ZiAvKiBfRFdfRURNQV9DT1JFX0ggKi8KZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvZG1hL2Vk bWEuaCBiL2luY2x1ZGUvbGludXgvZG1hL2VkbWEuaApuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRl eCAwMDAwMDAwLi4zNDllNTQyCi0tLSAvZGV2L251bGwKKysrIGIvaW5jbHVkZS9saW51eC9kbWEv ZWRtYS5oCkBAIC0wLDAgKzEsNDMgQEAKKy8qIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBHUEwt Mi4wICovCisvLyBDb3B5cmlnaHQgKGMpIDIwMTggU3lub3BzeXMsIEluYy4gYW5kL29yIGl0cyBh ZmZpbGlhdGVzLgorLy8gU3lub3BzeXMgRGVzaWduV2FyZSBlRE1BIGNvcmUgZHJpdmVyCisKKyNp Zm5kZWYgX0RXX0VETUFfSAorI2RlZmluZSBfRFdfRURNQV9ICisKKyNpbmNsdWRlIDxsaW51eC9k ZXZpY2UuaD4KKyNpbmNsdWRlIDxsaW51eC9kbWFlbmdpbmUuaD4KKworc3RydWN0IGR3X2VkbWE7 CisKKy8qKgorICogc3RydWN0IGR3X2VkbWFfY2hpcCAtIHJlcHJlc2VudGF0aW9uIG9mIERlc2ln bldhcmUgZURNQSBjb250cm9sbGVyIGhhcmR3YXJlCisgKiBAZGV2OgkJIHN0cnVjdCBkZXZpY2Ug b2YgdGhlIGVETUEgY29udHJvbGxlcgorICogQGlkOgkJCSBpbnN0YW5jZSBJRAorICogQGlycToJ CSBpcnEgbGluZQorICogQGR3OgkJCSBzdHJ1Y3QgZHdfZWRtYSB0aGF0IGlzIGZpbGVkIGJ5IGR3 X2VkbWFfcHJvYmUoKQorICovCitzdHJ1Y3QgZHdfZWRtYV9jaGlwIHsKKwlzdHJ1Y3QgZGV2aWNl CQkqZGV2OworCWludAkJCWlkOworCWludAkJCWlycTsKKwlzdHJ1Y3QgZHdfZWRtYQkJKmR3Owor fTsKKworLyogRXhwb3J0IHRvIHRoZSBwbGF0Zm9ybSBkcml2ZXJzICovCisjaWYgSVNfRU5BQkxF RChDT05GSUdfRFdfRURNQSkKK2ludCBkd19lZG1hX3Byb2JlKHN0cnVjdCBkd19lZG1hX2NoaXAg KmNoaXApOworaW50IGR3X2VkbWFfcmVtb3ZlKHN0cnVjdCBkd19lZG1hX2NoaXAgKmNoaXApOwor I2Vsc2UKK3N0YXRpYyBpbmxpbmUgaW50IGR3X2VkbWFfcHJvYmUoc3RydWN0IGR3X2VkbWFfY2hp cCAqY2hpcCkKK3sKKwlyZXR1cm4gLUVOT0RFVjsKK30KKworc3RhdGljIGlubGluZSBpbnQgZHdf ZWRtYV9yZW1vdmUoc3RydWN0IGR3X2VkbWFfY2hpcCAqY2hpcCkKK3sKKwlyZXR1cm4gMDsKK30K KyNlbmRpZiAvKiBDT05GSUdfRFdfRURNQSAqLworCisjZW5kaWYgLyogX0RXX0VETUFfSCAqLwo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E504C43444 for ; Fri, 11 Jan 2019 18:33:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F0E9920872 for ; Fri, 11 Jan 2019 18:33:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=synopsys.com header.i=@synopsys.com header.b="JMGrui2s" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729700AbfAKSdy (ORCPT ); Fri, 11 Jan 2019 13:33:54 -0500 Received: from smtprelay.synopsys.com ([198.182.47.9]:34026 "EHLO smtprelay.synopsys.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733145AbfAKSdx (ORCPT ); Fri, 11 Jan 2019 13:33:53 -0500 Received: from mailhost.synopsys.com (mailhost3.synopsys.com [10.12.238.238]) by smtprelay.synopsys.com (Postfix) with ESMTP id E96AE24E0F62; Fri, 11 Jan 2019 10:33:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=synopsys.com; s=mail; t=1547231633; bh=9uqqe4K8PCynJOS3y1gZ6/57XAdYEunv6wkgAoXqrdo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:In-Reply-To: References:From; b=JMGrui2syPkFfRPCPn3Fi+4vy60O/JEUEhTXKIJ6Nr8L2UstxCwKeykk0maCXkXb4 X0GmrWpII41hjcbc8s4NkeYlF80E928jafY1S1A9Prt7hgJ/7D7Cvfpbx9JDZVDehx qbt+Bf8ttMnW+FIEMOIDsyO2VrQj35M2kKI1j+ad5XDG+f7dOed6bJgmkZyP96ym2c toSrocaH+Ye7Ca6i2s5h6GNplUfVeYy8PIHRh0qjdMnz7DrZcXO7pUM38dm5ooAiyY DwSp3wnJIKQuNhLWJGBEjkm5oMCm1xOY/NYWiScAUqTt4qngSyQmP4hOJYFcn8gh4A Z/yl8yD8YYlEw== Received: from de02.synopsys.com (de02.internal.synopsys.com [10.225.17.21]) by mailhost.synopsys.com (Postfix) with ESMTP id 529B33EF1; Fri, 11 Jan 2019 10:33:51 -0800 (PST) Received: from de02dwia024.internal.synopsys.com (de02dwia024.internal.synopsys.com [10.225.19.81]) by de02.synopsys.com (Postfix) with ESMTP id A9D313ED7F; Fri, 11 Jan 2019 19:33:50 +0100 (CET) From: Gustavo Pimentel To: linux-pci@vger.kernel.org, dmaengine@vger.kernel.org Cc: Gustavo Pimentel , Vinod Koul , Dan Williams , Eugeniy Paltsev , Andy Shevchenko , Russell King , Niklas Cassel , Joao Pinto , Jose Abreu , Luis Oliveira , Vitor Soares , Nelson Costa , Pedro Sousa Subject: [RFC v3 1/7] dmaengine: Add Synopsys eDMA IP core driver Date: Fri, 11 Jan 2019 19:33:37 +0100 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add Synopsys eDMA IP core driver to kernel. This core driver, initializes and configures the eDMA IP using vma-helpers functions and dma-engine subsystem. Also creates an abstration layer through callbacks allowing different registers mappings in the future, organized in to versions. This driver can be compile as built-in or external module in kernel. To enable this driver just select DW_EDMA option in kernel configuration, however it requires and selects automatically DMA_ENGINE and DMA_VIRTUAL_CHANNELS option too. Changes: RFC v1->RFC v2: - Replace comments // (C99 style) by /**/ - Fix the headers of the .c and .h files according to the most recent convention - Fix errors and checks pointed out by checkpatch with --strict option - Replace patch small description tag from dma by dmaengine - Change some dev_info() into dev_dbg() - Remove unnecessary zero initialization after kzalloc - Remove direction validation on config() API, since the direction parameter is deprecated - Refactor code to replace atomic_t by u32 variable type - Replace start_transfer() name by dw_edma_start_transfer() - Add spinlock to dw_edma_device_prep_slave_sg() - Add spinlock to dw_edma_free_chunk() - Simplify switch case into if on dw_edma_device_pause(), dw_edma_device_resume() and dw_edma_device_terminate_all() RFC v2->RFC v3: - Add driver parameter to disable msix feature - Fix printk variable of phys_addr_t type - Fix printk variable of __iomem type - Fix printk variable of size_t type - Add comments or improve existing ones - Add possibility to work with multiple IRQs feature - Fix source and destination addresses - Add define to magic numbers - Add DMA cyclic transfer feature Signed-off-by: Gustavo Pimentel Cc: Vinod Koul Cc: Dan Williams Cc: Eugeniy Paltsev Cc: Andy Shevchenko Cc: Russell King Cc: Niklas Cassel Cc: Joao Pinto Cc: Jose Abreu Cc: Luis Oliveira Cc: Vitor Soares Cc: Nelson Costa Cc: Pedro Sousa --- drivers/dma/Kconfig | 2 + drivers/dma/Makefile | 1 + drivers/dma/dw-edma/Kconfig | 9 + drivers/dma/dw-edma/Makefile | 4 + drivers/dma/dw-edma/dw-edma-core.c | 1059 ++++++++++++++++++++++++++++++++++++ drivers/dma/dw-edma/dw-edma-core.h | 151 +++++ include/linux/dma/edma.h | 43 ++ 7 files changed, 1269 insertions(+) create mode 100644 drivers/dma/dw-edma/Kconfig create mode 100644 drivers/dma/dw-edma/Makefile create mode 100644 drivers/dma/dw-edma/dw-edma-core.c create mode 100644 drivers/dma/dw-edma/dw-edma-core.h create mode 100644 include/linux/dma/edma.h diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig index d2286c7..5877ee5 100644 --- a/drivers/dma/Kconfig +++ b/drivers/dma/Kconfig @@ -651,6 +651,8 @@ source "drivers/dma/qcom/Kconfig" source "drivers/dma/dw/Kconfig" +source "drivers/dma/dw-edma/Kconfig" + source "drivers/dma/hsu/Kconfig" source "drivers/dma/sh/Kconfig" diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile index 09571a8..e42e7d6 100644 --- a/drivers/dma/Makefile +++ b/drivers/dma/Makefile @@ -29,6 +29,7 @@ obj-$(CONFIG_DMA_SUN4I) += sun4i-dma.o obj-$(CONFIG_DMA_SUN6I) += sun6i-dma.o obj-$(CONFIG_DW_AXI_DMAC) += dw-axi-dmac/ obj-$(CONFIG_DW_DMAC_CORE) += dw/ +obj-$(CONFIG_DW_EDMA) += dw-edma/ obj-$(CONFIG_EP93XX_DMA) += ep93xx_dma.o obj-$(CONFIG_FSL_DMA) += fsldma.o obj-$(CONFIG_FSL_EDMA) += fsl-edma.o fsl-edma-common.o diff --git a/drivers/dma/dw-edma/Kconfig b/drivers/dma/dw-edma/Kconfig new file mode 100644 index 0000000..3016bed --- /dev/null +++ b/drivers/dma/dw-edma/Kconfig @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0 + +config DW_EDMA + tristate "Synopsys DesignWare eDMA controller driver" + select DMA_ENGINE + select DMA_VIRTUAL_CHANNELS + help + Support the Synopsys DesignWare eDMA controller, normally + implemented on endpoints SoCs. diff --git a/drivers/dma/dw-edma/Makefile b/drivers/dma/dw-edma/Makefile new file mode 100644 index 0000000..3224010 --- /dev/null +++ b/drivers/dma/dw-edma/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_DW_EDMA) += dw-edma.o +dw-edma-objs := dw-edma-core.o diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-edma-core.c new file mode 100644 index 0000000..2b6b70f --- /dev/null +++ b/drivers/dma/dw-edma/dw-edma-core.c @@ -0,0 +1,1059 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2018 Synopsys, Inc. and/or its affiliates. + * Synopsys DesignWare eDMA core driver + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "dw-edma-core.h" +#include "../dmaengine.h" +#include "../virt-dma.h" + +#define SET(reg, name, val) \ + reg.name = val + +#define SET_BOTH_CH(name, value) \ + do { \ + SET(dw->wr_edma, name, value); \ + SET(dw->rd_edma, name, value); \ + } while (0) + +static inline +struct device *dchan2dev(struct dma_chan *dchan) +{ + return &dchan->dev->device; +} + +static inline +struct device *chan2dev(struct dw_edma_chan *chan) +{ + return &chan->vc.chan.dev->device; +} + +static inline +const struct dw_edma_core_ops *chan2ops(struct dw_edma_chan *chan) +{ + return chan->chip->dw->ops; +} + +static inline +struct dw_edma_desc *vd2dw_edma_desc(struct virt_dma_desc *vd) +{ + return container_of(vd, struct dw_edma_desc, vd); +} + +static struct dw_edma_burst *dw_edma_alloc_burst(struct dw_edma_chunk *chunk) +{ + struct dw_edma_chan *chan = chunk->chan; + struct dw_edma_burst *burst; + + burst = kvzalloc(sizeof(*burst), GFP_NOWAIT); + if (unlikely(!burst)) + return NULL; + + INIT_LIST_HEAD(&burst->list); + + if (chunk->burst) { + /* Create and add new element into the linked list */ + chunk->bursts_alloc++; + dev_dbg(chan2dev(chan), "alloc new burst element (%d)\n", + chunk->bursts_alloc); + list_add_tail(&burst->list, &chunk->burst->list); + } else { + /* List head */ + chunk->bursts_alloc = 0; + chunk->burst = burst; + dev_dbg(chan2dev(chan), "alloc new burst head\n"); + } + + return burst; +} + +static struct dw_edma_chunk *dw_edma_alloc_chunk(struct dw_edma_desc *desc) +{ + struct dw_edma_chan *chan = desc->chan; + struct dw_edma *dw = chan->chip->dw; + struct dw_edma_chunk *chunk; + + chunk = kvzalloc(sizeof(*chunk), GFP_NOWAIT); + if (unlikely(!chunk)) + return NULL; + + INIT_LIST_HEAD(&chunk->list); + chunk->chan = chan; + chunk->cb = !(desc->chunks_alloc % 2); + chunk->ll_region.paddr = dw->ll_region.paddr + chan->ll_off; + chunk->ll_region.vaddr = dw->ll_region.vaddr + chan->ll_off; + + if (desc->chunk) { + /* Create and add new element into the linked list */ + desc->chunks_alloc++; + dev_dbg(chan2dev(chan), "alloc new chunk element (%d)\n", + desc->chunks_alloc); + list_add_tail(&chunk->list, &desc->chunk->list); + dw_edma_alloc_burst(chunk); + } else { + /* List head */ + chunk->burst = NULL; + desc->chunks_alloc = 0; + desc->chunk = chunk; + dev_dbg(chan2dev(chan), "alloc new chunk head\n"); + } + + return chunk; +} + +static struct dw_edma_desc *dw_edma_alloc_desc(struct dw_edma_chan *chan) +{ + struct dw_edma_desc *desc; + + dev_dbg(chan2dev(chan), "alloc new descriptor\n"); + + desc = kvzalloc(sizeof(*desc), GFP_NOWAIT); + if (unlikely(!desc)) + return NULL; + + desc->chan = chan; + dw_edma_alloc_chunk(desc); + + return desc; +} + +static void dw_edma_free_burst(struct dw_edma_chunk *chunk) +{ + struct dw_edma_burst *child, *_next; + + if (!chunk->burst) + return; + + /* Remove all the list elements */ + list_for_each_entry_safe(child, _next, &chunk->burst->list, list) { + list_del(&child->list); + kvfree(child); + chunk->bursts_alloc--; + } + + /* Remove the list head */ + kvfree(child); + chunk->burst = NULL; +} + +static void dw_edma_free_chunk(struct dw_edma_desc *desc) +{ + struct dw_edma_chan *chan = desc->chan; + struct dw_edma_chunk *child, *_next; + + if (!desc->chunk) + return; + + /* Remove all the list elements */ + list_for_each_entry_safe(child, _next, &desc->chunk->list, list) { + dw_edma_free_burst(child); + if (child->bursts_alloc) + dev_dbg(chan2dev(chan), "%u bursts still allocated\n", + child->bursts_alloc); + list_del(&child->list); + kvfree(child); + desc->chunks_alloc--; + } + + /* Remove the list head */ + kvfree(child); + desc->chunk = NULL; +} + +static void dw_edma_free_desc(struct dw_edma_desc *desc) +{ + struct dw_edma_chan *chan = desc->chan; + unsigned long flags; + + spin_lock_irqsave(&chan->vc.lock, flags); + + dw_edma_free_chunk(desc); + if (desc->chunks_alloc) + dev_dbg(chan2dev(chan), "%u chunks still allocated\n", + desc->chunks_alloc); + + spin_unlock_irqrestore(&chan->vc.lock, flags); +} + +static void vchan_free_desc(struct virt_dma_desc *vdesc) +{ + dw_edma_free_desc(vd2dw_edma_desc(vdesc)); +} + +static void dw_edma_start_transfer(struct dw_edma_chan *chan) +{ + struct virt_dma_desc *vd; + struct dw_edma_desc *desc; + struct dw_edma_chunk *child; + const struct dw_edma_core_ops *ops = chan2ops(chan); + + vd = vchan_next_desc(&chan->vc); + if (!vd) + return; + + desc = vd2dw_edma_desc(vd); + if (!desc) + return; + + child = list_first_entry_or_null(&desc->chunk->list, + struct dw_edma_chunk, list); + if (!child) + return; + + ops->start(child, !desc->xfer_sz); + desc->xfer_sz += child->ll_region.sz; + dev_dbg(chan2dev(chan), "transfer of %u bytes started\n", + child->ll_region.sz); + + dw_edma_free_burst(child); + if (child->bursts_alloc) + dev_dbg(chan2dev(chan), "%u bursts still allocated\n", + child->bursts_alloc); + list_del(&child->list); + kvfree(child); + desc->chunks_alloc--; +} + +static int dw_edma_device_config(struct dma_chan *dchan, + struct dma_slave_config *config) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + const struct dw_edma_core_ops *ops = chan2ops(chan); + unsigned long flags; + int err = 0; + + spin_lock_irqsave(&chan->vc.lock, flags); + + if (!config) { + err = -EINVAL; + goto err_config; + } + + if (chan->status != EDMA_ST_IDLE) { + dev_err(chan2dev(chan), "channel is busy or paused\n"); + err = -EPERM; + goto err_config; + } + + dev_dbg(chan2dev(chan), "addr(physical) src=%pa, dst=%pa\n", + &config->src_addr, &config->dst_addr); + + chan->src_addr = config->src_addr; + chan->dst_addr = config->dst_addr; + + err = ops->device_config(dchan); + if (!err) { + chan->configured = true; + dev_dbg(chan2dev(chan), "channel configured\n"); + } + +err_config: + spin_unlock_irqrestore(&chan->vc.lock, flags); + return err; +} + +static int dw_edma_device_pause(struct dma_chan *dchan) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + unsigned long flags; + int err = 0; + + spin_lock_irqsave(&chan->vc.lock, flags); + + if (!chan->configured) { + dev_err(chan2dev(chan), "(pause) channel not configured\n"); + err = -EPERM; + goto err_pause; + } + + if (chan->status != EDMA_ST_BUSY) { + err = -EPERM; + goto err_pause; + } + + if (chan->request != EDMA_REQ_NONE) { + err = -EPERM; + goto err_pause; + } + + chan->request = EDMA_REQ_PAUSE; + dev_dbg(chan2dev(chan), "pause requested\n"); + +err_pause: + spin_unlock_irqrestore(&chan->vc.lock, flags); + return err; +} + +static int dw_edma_device_resume(struct dma_chan *dchan) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + unsigned long flags; + int err = 0; + + spin_lock_irqsave(&chan->vc.lock, flags); + + if (!chan->configured) { + dev_err(chan2dev(chan), "(resume) channel not configured\n"); + err = -EPERM; + goto err_resume; + } + + if (chan->status != EDMA_ST_PAUSE) { + err = -EPERM; + goto err_resume; + } + + if (chan->request != EDMA_REQ_NONE) { + err = -EPERM; + goto err_resume; + } + + chan->status = EDMA_ST_BUSY; + dev_dbg(dchan2dev(dchan), "transfer resumed\n"); + dw_edma_start_transfer(chan); + +err_resume: + spin_unlock_irqrestore(&chan->vc.lock, flags); + return err; +} + +static int dw_edma_device_terminate_all(struct dma_chan *dchan) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + const struct dw_edma_core_ops *ops = chan2ops(chan); + unsigned long flags; + int err = 0; + LIST_HEAD(head); + + spin_lock_irqsave(&chan->vc.lock, flags); + + if (!chan->configured) + goto err_terminate; + + if (chan->status == EDMA_ST_PAUSE) { + dev_dbg(dchan2dev(dchan), "channel is paused, stopping immediately\n"); + chan->status = EDMA_ST_IDLE; + chan->configured = false; + goto err_terminate; + } else if (chan->status == EDMA_ST_IDLE) { + chan->configured = false; + goto err_terminate; + } else if (ops->ch_status(chan) == DMA_COMPLETE) { + /* + * The channel is in a false BUSY state, probably didn't + * receive or lost an interrupt + */ + chan->status = EDMA_ST_IDLE; + chan->configured = false; + goto err_terminate; + } + + if (chan->request > EDMA_REQ_PAUSE) { + err = -EPERM; + goto err_terminate; + } + + chan->request = EDMA_REQ_STOP; + dev_dbg(dchan2dev(dchan), "termination requested\n"); + +err_terminate: + spin_unlock_irqrestore(&chan->vc.lock, flags); + return err; +} + +static void dw_edma_device_issue_pending(struct dma_chan *dchan) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + unsigned long flags; + + spin_lock_irqsave(&chan->vc.lock, flags); + + if (chan->configured && chan->request == EDMA_REQ_NONE && + chan->status == EDMA_ST_IDLE && vchan_issue_pending(&chan->vc)) { + dev_dbg(dchan2dev(dchan), "transfer issued\n"); + chan->status = EDMA_ST_BUSY; + dw_edma_start_transfer(chan); + } + + spin_unlock_irqrestore(&chan->vc.lock, flags); +} + +static enum dma_status +dw_edma_device_tx_status(struct dma_chan *dchan, dma_cookie_t cookie, + struct dma_tx_state *txstate) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + const struct dw_edma_core_ops *ops = chan2ops(chan); + unsigned long flags; + enum dma_status ret; + + spin_lock_irqsave(&chan->vc.lock, flags); + + ret = ops->ch_status(chan); + if (ret == DMA_ERROR) { + goto ret_status; + } else if (ret == DMA_IN_PROGRESS) { + chan->status = EDMA_ST_BUSY; + goto ret_status; + } else { + /* DMA_COMPLETE */ + if (chan->status == EDMA_ST_PAUSE) + ret = DMA_PAUSED; + else if (chan->status == EDMA_ST_BUSY) + ret = DMA_IN_PROGRESS; + else + ret = DMA_COMPLETE; + } + +ret_status: + spin_unlock_irqrestore(&chan->vc.lock, flags); + dma_set_residue(txstate, 0); + + return ret; +} + +static struct dma_async_tx_descriptor * +dw_edma_device_prep_slave_sg(struct dma_chan *dchan, struct scatterlist *sgl, + unsigned int sg_len, + enum dma_transfer_direction direction, + unsigned long flags, void *context) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + struct dw_edma_desc *desc; + struct dw_edma_chunk *chunk; + struct dw_edma_burst *burst; + struct scatterlist *sg; + unsigned long sflags; + phys_addr_t src_addr; + phys_addr_t dst_addr; + int i; + + if (sg_len < 1) { + dev_err(chan2dev(chan), "invalid sg length %u\n", sg_len); + return NULL; + } + + if (direction == DMA_DEV_TO_MEM && chan->dir == EDMA_DIR_WRITE) { + dev_dbg(chan2dev(chan), "prepare operation (WRITE)\n"); + } else if (direction == DMA_MEM_TO_DEV && chan->dir == EDMA_DIR_READ) { + dev_dbg(chan2dev(chan), "prepare operation (READ)\n"); + } else { + dev_err(chan2dev(chan), "invalid direction\n"); + return NULL; + } + + if (!chan->configured) { + dev_err(chan2dev(chan), "(prep_slave_sg) channel not configured\n"); + return NULL; + } + + if (chan->status != EDMA_ST_IDLE) { + dev_err(chan2dev(chan), "channel is busy or paused\n"); + return NULL; + } + + spin_lock_irqsave(&chan->vc.lock, sflags); + + desc = dw_edma_alloc_desc(chan); + if (unlikely(!desc)) + goto err_alloc; + + chunk = dw_edma_alloc_chunk(desc); + if (unlikely(!chunk)) + goto err_alloc; + + src_addr = chan->src_addr; + dst_addr = chan->dst_addr; + + for_each_sg(sgl, sg, sg_len, i) { + if (chunk->bursts_alloc == chan->ll_max) { + chunk = dw_edma_alloc_chunk(desc); + if (unlikely(!chunk)) + goto err_alloc; + } + + burst = dw_edma_alloc_burst(chunk); + + if (unlikely(!burst)) + goto err_alloc; + + burst->sz = sg_dma_len(sg); + chunk->ll_region.sz += burst->sz; + desc->alloc_sz += burst->sz; + + if (direction == DMA_MEM_TO_DEV) { + burst->sar = sg_dma_address(sg); + burst->dar = dst_addr; + dst_addr += sg_dma_len(sg); + } else { + burst->sar = src_addr; + burst->dar = sg_dma_address(sg); + src_addr += sg_dma_len(sg); + } + + dev_dbg(chan2dev(chan), "lli %u/%u, sar=0x%.8llx, dar=0x%.8llx, size=%u bytes\n", + i + 1, sg_len, burst->sar, burst->dar, burst->sz); + } + + spin_unlock_irqrestore(&chan->vc.lock, sflags); + return vchan_tx_prep(&chan->vc, &desc->vd, flags); + +err_alloc: + if (desc) + dw_edma_free_desc(desc); + spin_unlock_irqrestore(&chan->vc.lock, sflags); + return NULL; +} + +static struct dma_async_tx_descriptor * +dw_edma_device_prep_dma_cyclic(struct dma_chan *dchan, dma_addr_t buf_addr, + size_t len, size_t cyclic_cnt, + enum dma_transfer_direction direction, + unsigned long flags) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + struct dw_edma_desc *desc; + struct dw_edma_chunk *chunk; + struct dw_edma_burst *burst; + unsigned long sflags; + phys_addr_t src_addr; + phys_addr_t dst_addr; + u32 i; + + if (!len || !cyclic_cnt) { + dev_err(chan2dev(chan), "invalid len or cyclic count\n"); + return NULL; + } + + if (direction == DMA_DEV_TO_MEM && chan->dir == EDMA_DIR_WRITE) { + dev_dbg(chan2dev(chan), "prepare operation (WRITE)\n"); + } else if (direction == DMA_MEM_TO_DEV && chan->dir == EDMA_DIR_READ) { + dev_dbg(chan2dev(chan), "prepare operation (READ)\n"); + } else { + dev_err(chan2dev(chan), "invalid direction\n"); + return NULL; + } + + if (!chan->configured) { + dev_err(chan2dev(chan), "(prep_dma_cyclic) channel not configured\n"); + return NULL; + } + + if (chan->status != EDMA_ST_IDLE) { + dev_err(chan2dev(chan), "channel is busy or paused\n"); + return NULL; + } + + spin_lock_irqsave(&chan->vc.lock, sflags); + + desc = dw_edma_alloc_desc(chan); + if (unlikely(!desc)) + goto err_alloc; + + chunk = dw_edma_alloc_chunk(desc); + if (unlikely(!chunk)) + goto err_alloc; + + src_addr = chan->src_addr; + dst_addr = chan->dst_addr; + + for (i = 0; i < cyclic_cnt; i++) { + if (chunk->bursts_alloc == chan->ll_max) { + chunk = dw_edma_alloc_chunk(desc); + if (unlikely(!chunk)) + goto err_alloc; + } + + burst = dw_edma_alloc_burst(chunk); + + if (unlikely(!burst)) + goto err_alloc; + + burst->sz = len; + chunk->ll_region.sz += burst->sz; + desc->alloc_sz += burst->sz; + + burst->sar = src_addr; + burst->dar = dst_addr; + + dev_dbg(chan2dev(chan), "lli %u/%u, sar=0x%.8llx, dar=0x%.8llx, size=%u bytes\n", + i + 1, cyclic_cnt, burst->sar, burst->dar, burst->sz); + } + + spin_unlock_irqrestore(&chan->vc.lock, sflags); + return vchan_tx_prep(&chan->vc, &desc->vd, flags); + +err_alloc: + if (desc) + dw_edma_free_desc(desc); + spin_unlock_irqrestore(&chan->vc.lock, sflags); + return NULL; +} + +static void dw_edma_done_interrupt(struct dw_edma_chan *chan) +{ + struct dw_edma *dw = chan->chip->dw; + const struct dw_edma_core_ops *ops = dw->ops; + struct virt_dma_desc *vd; + struct dw_edma_desc *desc; + unsigned long flags; + + ops->clear_done_int(chan); + dev_dbg(chan2dev(chan), "clear done interrupt\n"); + + spin_lock_irqsave(&chan->vc.lock, flags); + vd = vchan_next_desc(&chan->vc); + switch (chan->request) { + case EDMA_REQ_NONE: + if (!vd) + break; + + desc = vd2dw_edma_desc(vd); + if (desc->chunks_alloc) { + dev_dbg(chan2dev(chan), "sub-transfer complete\n"); + chan->status = EDMA_ST_BUSY; + dev_dbg(chan2dev(chan), "transferred %u bytes\n", + desc->xfer_sz); + dw_edma_start_transfer(chan); + } else { + list_del(&vd->node); + vchan_cookie_complete(vd); + chan->status = EDMA_ST_IDLE; + dev_dbg(chan2dev(chan), "transfer complete\n"); + } + break; + case EDMA_REQ_STOP: + if (!vd) + break; + + list_del(&vd->node); + vchan_cookie_complete(vd); + chan->request = EDMA_REQ_NONE; + chan->status = EDMA_ST_IDLE; + chan->configured = false; + dev_dbg(chan2dev(chan), "transfer stop\n"); + break; + case EDMA_REQ_PAUSE: + chan->request = EDMA_REQ_NONE; + chan->status = EDMA_ST_PAUSE; + break; + default: + dev_err(chan2dev(chan), "invalid status state\n"); + break; + } + spin_unlock_irqrestore(&chan->vc.lock, flags); +} + +static void dw_edma_abort_interrupt(struct dw_edma_chan *chan) +{ + struct dw_edma *dw = chan->chip->dw; + const struct dw_edma_core_ops *ops = dw->ops; + struct virt_dma_desc *vd; + unsigned long flags; + + ops->clear_abort_int(chan); + dev_dbg(chan2dev(chan), "clear abort interrupt\n"); + + spin_lock_irqsave(&chan->vc.lock, flags); + vd = vchan_next_desc(&chan->vc); + if (vd) { + list_del(&vd->node); + vchan_cookie_complete(vd); + } + chan->request = EDMA_REQ_NONE; + chan->status = EDMA_ST_IDLE; + + spin_unlock_irqrestore(&chan->vc.lock, flags); +} + +static irqreturn_t dw_edma_interrupt_write(int irq, void *data) +{ + struct dw_edma_chip *chip = data; + struct dw_edma *dw = chip->dw; + const struct dw_edma_core_ops *ops = dw->ops; + unsigned long tot = dw->wr_ch_cnt; + unsigned long pos = 0; + unsigned long val; + + pos = 0; + val = ops->status_done_int(dw, EDMA_DIR_WRITE); + while ((pos = find_next_bit(&val, tot, pos)) != tot) { + struct dw_edma_chan *chan = &dw->chan[pos]; + + dw_edma_done_interrupt(chan); + pos++; + } + + pos = 0; + val = ops->status_abort_int(dw, EDMA_DIR_WRITE); + while ((pos = find_next_bit(&val, tot, pos)) != tot) { + struct dw_edma_chan *chan = &dw->chan[pos]; + + dw_edma_abort_interrupt(chan); + pos++; + } + + return IRQ_HANDLED; +} + +static irqreturn_t dw_edma_interrupt_read(int irq, void *data) +{ + struct dw_edma_chip *chip = data; + struct dw_edma *dw = chip->dw; + const struct dw_edma_core_ops *ops = dw->ops; + unsigned long tot = dw->rd_ch_cnt; + unsigned long off = dw->wr_ch_cnt; + unsigned long pos, val; + + pos = 0; + val = ops->status_done_int(dw, EDMA_DIR_READ); + while ((pos = find_next_bit(&val, tot, pos)) != tot) { + struct dw_edma_chan *chan = &dw->chan[pos + off]; + + dw_edma_done_interrupt(chan); + pos++; + } + + pos = 0; + val = ops->status_abort_int(dw, EDMA_DIR_READ); + while ((pos = find_next_bit(&val, tot, pos)) != tot) { + struct dw_edma_chan *chan = &dw->chan[pos + off]; + + dw_edma_abort_interrupt(chan); + pos++; + } + + return IRQ_HANDLED; +} + +static irqreturn_t dw_edma_interrupt_all(int irq, void *data) +{ + dw_edma_interrupt_write(irq, data); + dw_edma_interrupt_read(irq, data); + + return IRQ_HANDLED; +} + +static int dw_edma_alloc_chan_resources(struct dma_chan *dchan) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + + if (chan->status != EDMA_ST_IDLE) { + dev_err(chan2dev(chan), "channel is busy\n"); + return -EBUSY; + } + + dev_dbg(dchan2dev(dchan), "allocated\n"); + + pm_runtime_get(chan->chip->dev); + + return 0; +} + +static void dw_edma_free_chan_resources(struct dma_chan *dchan) +{ + struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan); + unsigned long timeout = jiffies + msecs_to_jiffies(5000); + int ret; + + if (chan->status != EDMA_ST_IDLE) + dev_err(chan2dev(chan), "channel is busy\n"); + + do { + ret = dw_edma_device_terminate_all(dchan); + if (!ret) + break; + + if (time_after_eq(jiffies, timeout)) { + dev_err(chan2dev(chan), "free timeout\n"); + return; + } + + cpu_relax(); + } while (1); + + dev_dbg(dchan2dev(dchan), "channel freed\n"); + + pm_runtime_put(chan->chip->dev); +} + +int dw_edma_probe(struct dw_edma_chip *chip) +{ + struct dw_edma *dw = chip->dw; + struct device *dev = chip->dev; + const struct dw_edma_core_ops *ops; + size_t ll_chunk = dw->ll_region.sz; + size_t dt_chunk = dw->dt_region.sz; + u32 ch_tot; + int i, j, err; + + raw_spin_lock_init(&dw->lock); + + /* Callback operation selection accordingly to eDMA version */ + switch (dw->version) { + default: + dev_err(dev, "unsupported version\n"); + return -EPERM; + } + + pm_runtime_get_sync(dev); + + /* Find out how many write channels are supported by hardware */ + dw->wr_ch_cnt = ops->ch_count(dw, EDMA_DIR_WRITE); + if (!dw->wr_ch_cnt) { + dev_err(dev, "invalid number of write channels(0)\n"); + return -EINVAL; + } + + /* Find out how many read channels are supported by hardware */ + dw->rd_ch_cnt = ops->ch_count(dw, EDMA_DIR_READ); + if (!dw->rd_ch_cnt) { + dev_err(dev, "invalid number of read channels(0)\n"); + return -EINVAL; + } + + dev_dbg(dev, "Channels:\twrite=%d, read=%d\n", + dw->wr_ch_cnt, dw->rd_ch_cnt); + + ch_tot = dw->wr_ch_cnt + dw->rd_ch_cnt; + + /* Allocate channels */ + dw->chan = devm_kcalloc(dev, ch_tot, sizeof(*dw->chan), GFP_KERNEL); + if (!dw->chan) + return -ENOMEM; + + /* Calculate the linked list chunk for each channel */ + ll_chunk /= roundup_pow_of_two(ch_tot); + + /* Calculate the linked list chunk for each channel */ + dt_chunk /= roundup_pow_of_two(ch_tot); + + /* Disable eDMA, only to establish the ideal initial conditions */ + ops->off(dw); + + snprintf(dw->name, sizeof(dw->name), "dw-edma-core:%d", chip->id); + + /* Request IRQs */ + if (dw->nr_irqs != 1) { + dev_err(dev, "invalid number of irqs (%u)\n", dw->nr_irqs); + return -EINVAL; + } + + for (i = 0; i < dw->nr_irqs; i++) { + err = devm_request_irq(dev, pci_irq_vector(to_pci_dev(dev), i), + dw_edma_interrupt_all, + IRQF_SHARED, dw->name, chip); + if (err) + return err; + } + + /* Create write channels */ + INIT_LIST_HEAD(&dw->wr_edma.channels); + for (i = 0; i < dw->wr_ch_cnt; i++) { + struct dw_edma_chan *chan = &dw->chan[i]; + struct dw_edma_region *dt_region; + + dt_region = devm_kzalloc(dev, sizeof(*dt_region), GFP_KERNEL); + if (!dt_region) + return -ENOMEM; + + chan->vc.chan.private = dt_region; + + chan->chip = chip; + chan->id = i; + chan->dir = EDMA_DIR_WRITE; + chan->configured = false; + chan->request = EDMA_REQ_NONE; + chan->status = EDMA_ST_IDLE; + + chan->ll_off = (ll_chunk * i); + chan->ll_max = (ll_chunk / EDMA_LL_SZ) - 1; + + chan->dt_off = (dt_chunk * i); + + dev_dbg(dev, "L. List:\tChannel write[%u] off=0x%.8lx, max_cnt=%u\n", + i, chan->ll_off, chan->ll_max); + + memcpy(&chan->msi, &dw->msi[0], sizeof(chan->msi)); + + dev_dbg(dev, "MSI:\t\tChannel write[%u] addr=0x%.8x%.8x, data=0x%.8x\n", + i, chan->msi.address_hi, chan->msi.address_lo, + chan->msi.data); + + chan->vc.desc_free = vchan_free_desc; + vchan_init(&chan->vc, &dw->wr_edma); + + dt_region->paddr = dw->dt_region.paddr + chan->dt_off; + dt_region->vaddr = dw->dt_region.vaddr + chan->dt_off; + dt_region->sz = dt_chunk; + + dev_dbg(dev, "Data:\tChannel write[%u] off=0x%.8lx\n", + i, chan->dt_off); + } + dma_cap_zero(dw->wr_edma.cap_mask); + dma_cap_set(DMA_SLAVE, dw->wr_edma.cap_mask); + dma_cap_set(DMA_CYCLIC, dw->wr_edma.cap_mask); + dw->wr_edma.directions = BIT(DMA_DEV_TO_MEM); + dw->wr_edma.chancnt = dw->wr_ch_cnt; + + /* Create read channels */ + INIT_LIST_HEAD(&dw->rd_edma.channels); + for (j = 0; j < dw->rd_ch_cnt; j++, i++) { + struct dw_edma_chan *chan = &dw->chan[i]; + struct dw_edma_region *dt_region; + + dt_region = devm_kzalloc(dev, sizeof(*dt_region), GFP_KERNEL); + if (!dt_region) + return -ENOMEM; + + chan->vc.chan.private = dt_region; + + chan->chip = chip; + chan->id = j; + chan->dir = EDMA_DIR_READ; + chan->configured = false; + chan->request = EDMA_REQ_NONE; + chan->status = EDMA_ST_IDLE; + + chan->ll_off = (ll_chunk * i); + chan->ll_max = (ll_chunk / EDMA_LL_SZ) - 1; + + chan->dt_off = (dt_chunk * i); + + dev_dbg(dev, "L. List:\tChannel read[%u] off=0x%.8lx, max_cnt=%u\n", + j, chan->ll_off, chan->ll_max); + + memcpy(&chan->msi, &dw->msi[0], sizeof(chan->msi)); + + dev_dbg(dev, "MSI:\t\tChannel read[%u] addr=0x%.8x%.8x, data=0x%.8x\n", + j, chan->msi.address_hi, chan->msi.address_lo, + chan->msi.data); + + chan->vc.desc_free = vchan_free_desc; + vchan_init(&chan->vc, &dw->rd_edma); + + dt_region->paddr = dw->dt_region.paddr + chan->dt_off; + dt_region->vaddr = dw->dt_region.vaddr + chan->dt_off; + dt_region->sz = dt_chunk; + + dev_dbg(dev, "Data:\tChannel read[%u] off=0x%.8lx\n", + i, chan->dt_off); + } + dma_cap_zero(dw->rd_edma.cap_mask); + dma_cap_set(DMA_SLAVE, dw->rd_edma.cap_mask); + dma_cap_set(DMA_CYCLIC, dw->rd_edma.cap_mask); + dw->rd_edma.directions = BIT(DMA_MEM_TO_DEV); + dw->rd_edma.chancnt = dw->rd_ch_cnt; + + /* Set DMA channels capabilities */ + SET_BOTH_CH(src_addr_widths, BIT(DMA_SLAVE_BUSWIDTH_4_BYTES)); + SET_BOTH_CH(dst_addr_widths, BIT(DMA_SLAVE_BUSWIDTH_4_BYTES)); + SET_BOTH_CH(residue_granularity, DMA_RESIDUE_GRANULARITY_DESCRIPTOR); + + SET_BOTH_CH(dev, dev); + + SET_BOTH_CH(device_alloc_chan_resources, dw_edma_alloc_chan_resources); + SET_BOTH_CH(device_free_chan_resources, dw_edma_free_chan_resources); + + SET_BOTH_CH(device_config, dw_edma_device_config); + SET_BOTH_CH(device_pause, dw_edma_device_pause); + SET_BOTH_CH(device_resume, dw_edma_device_resume); + SET_BOTH_CH(device_terminate_all, dw_edma_device_terminate_all); + SET_BOTH_CH(device_issue_pending, dw_edma_device_issue_pending); + SET_BOTH_CH(device_tx_status, dw_edma_device_tx_status); + SET_BOTH_CH(device_prep_slave_sg, dw_edma_device_prep_slave_sg); + SET_BOTH_CH(device_prep_dma_cyclic, dw_edma_device_prep_dma_cyclic); + + /* Power management */ + pm_runtime_enable(dev); + + /* Register DMA device */ + err = dma_async_device_register(&dw->wr_edma); + if (err) + goto err_pm_disable; + + err = dma_async_device_register(&dw->rd_edma); + if (err) + goto err_pm_disable; + + /* Turn debugfs on */ + err = ops->debugfs_on(chip); + if (err) { + dev_err(dev, "unable to create debugfs structure\n"); + goto err_pm_disable; + } + + dev_info(dev, "DesignWare eDMA controller driver loaded completely\n"); + + return 0; + +err_pm_disable: + pm_runtime_disable(dev); + + return err; +} +EXPORT_SYMBOL_GPL(dw_edma_probe); + +int dw_edma_remove(struct dw_edma_chip *chip) +{ + struct dw_edma *dw = chip->dw; + struct device *dev = chip->dev; + const struct dw_edma_core_ops *ops = dw->ops; + struct dw_edma_chan *chan, *_chan; + int i; + + /* Disable eDMA */ + if (ops) + ops->off(dw); + + /* Free irqs */ + for (i = 0; i < dw->nr_irqs; i++) { + if (pci_irq_vector(to_pci_dev(dev), i) < 0) + continue; + + devm_free_irq(dev, pci_irq_vector(to_pci_dev(dev), i), chip); + } + + /* Power management */ + pm_runtime_disable(dev); + + list_for_each_entry_safe(chan, _chan, &dw->wr_edma.channels, + vc.chan.device_node) { + list_del(&chan->vc.chan.device_node); + tasklet_kill(&chan->vc.task); + } + + list_for_each_entry_safe(chan, _chan, &dw->rd_edma.channels, + vc.chan.device_node) { + list_del(&chan->vc.chan.device_node); + tasklet_kill(&chan->vc.task); + } + + /* Deregister eDMA device */ + dma_async_device_unregister(&dw->wr_edma); + dma_async_device_unregister(&dw->rd_edma); + + /* Turn debugfs off */ + if (ops) + ops->debugfs_off(); + + dev_info(dev, "DesignWare eDMA controller driver unloaded complete\n"); + + return 0; +} +EXPORT_SYMBOL_GPL(dw_edma_remove); + +MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("Synopsys DesignWare eDMA controller core driver"); +MODULE_AUTHOR("Gustavo Pimentel "); diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-edma-core.h new file mode 100644 index 0000000..2d98a10 --- /dev/null +++ b/drivers/dma/dw-edma/dw-edma-core.h @@ -0,0 +1,151 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2018 Synopsys, Inc. and/or its affiliates. + * Synopsys DesignWare eDMA core driver + */ + +#ifndef _DW_EDMA_CORE_H +#define _DW_EDMA_CORE_H + +#include +#include + +#include "../virt-dma.h" + +#define EDMA_LL_SZ 24 + +enum dw_edma_dir { + EDMA_DIR_WRITE = 0, + EDMA_DIR_READ +}; + +enum dw_edma_mode { + EDMA_MODE_LEGACY = 0, + EDMA_MODE_UNROLL +}; + +enum dw_edma_request { + EDMA_REQ_NONE = 0, + EDMA_REQ_STOP, + EDMA_REQ_PAUSE +}; + +enum dw_edma_status { + EDMA_ST_IDLE = 0, + EDMA_ST_PAUSE, + EDMA_ST_BUSY +}; + +struct dw_edma_chan; +struct dw_edma_chunk; + +struct dw_edma_core_ops { + /* eDMA management callbacks */ + void (*off)(struct dw_edma *dw); + u16 (*ch_count)(struct dw_edma *dw, enum dw_edma_dir dir); + enum dma_status (*ch_status)(struct dw_edma_chan *chan); + void (*clear_done_int)(struct dw_edma_chan *chan); + void (*clear_abort_int)(struct dw_edma_chan *chan); + u32 (*status_done_int)(struct dw_edma *chan, enum dw_edma_dir dir); + u32 (*status_abort_int)(struct dw_edma *chan, enum dw_edma_dir dir); + void (*start)(struct dw_edma_chunk *chunk, bool first); + int (*device_config)(struct dma_chan *dchan); + /* eDMA debug fs callbacks */ + int (*debugfs_on)(struct dw_edma_chip *chip); + void (*debugfs_off)(void); +}; + +struct dw_edma_burst { + struct list_head list; + u64 sar; + u64 dar; + u32 sz; +}; + +struct dw_edma_region { + phys_addr_t paddr; + dma_addr_t vaddr; + size_t sz; +}; + +struct dw_edma_chunk { + struct list_head list; + struct dw_edma_chan *chan; + struct dw_edma_burst *burst; + + u32 bursts_alloc; + + u8 cb; + struct dw_edma_region ll_region; /* Linked list */ +}; + +struct dw_edma_desc { + struct virt_dma_desc vd; + struct dw_edma_chan *chan; + struct dw_edma_chunk *chunk; + + u32 chunks_alloc; + + u32 alloc_sz; + u32 xfer_sz; +}; + +struct dw_edma_chan { + struct virt_dma_chan vc; + struct dw_edma_chip *chip; + int id; + enum dw_edma_dir dir; + + off_t ll_off; + u32 ll_max; + + off_t dt_off; + + struct msi_msg msi; + + enum dw_edma_request request; + enum dw_edma_status status; + u8 configured; + + phys_addr_t src_addr; + phys_addr_t dst_addr; +}; + +struct dw_edma { + char name[20]; + + struct dma_device wr_edma; + u16 wr_ch_cnt; + + struct dma_device rd_edma; + u16 rd_ch_cnt; + + struct dw_edma_region rg_region; /* Registers */ + struct dw_edma_region ll_region; /* Linked list */ + struct dw_edma_region dt_region; /* Data */ + + struct msi_msg *msi; + int nr_irqs; + + u32 version; + enum dw_edma_mode mode; + + struct dw_edma_chan *chan; + const struct dw_edma_core_ops *ops; + + raw_spinlock_t lock; /* Only for legacy */ +}; + +static inline +struct dw_edma_chan *vc2dw_edma_chan(struct virt_dma_chan *vc) +{ + return container_of(vc, struct dw_edma_chan, vc); +} + +static inline +struct dw_edma_chan *dchan2dw_edma_chan(struct dma_chan *dchan) +{ + return vc2dw_edma_chan(to_virt_chan(dchan)); +} + +#endif /* _DW_EDMA_CORE_H */ diff --git a/include/linux/dma/edma.h b/include/linux/dma/edma.h new file mode 100644 index 0000000..349e542 --- /dev/null +++ b/include/linux/dma/edma.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +// Copyright (c) 2018 Synopsys, Inc. and/or its affiliates. +// Synopsys DesignWare eDMA core driver + +#ifndef _DW_EDMA_H +#define _DW_EDMA_H + +#include +#include + +struct dw_edma; + +/** + * struct dw_edma_chip - representation of DesignWare eDMA controller hardware + * @dev: struct device of the eDMA controller + * @id: instance ID + * @irq: irq line + * @dw: struct dw_edma that is filed by dw_edma_probe() + */ +struct dw_edma_chip { + struct device *dev; + int id; + int irq; + struct dw_edma *dw; +}; + +/* Export to the platform drivers */ +#if IS_ENABLED(CONFIG_DW_EDMA) +int dw_edma_probe(struct dw_edma_chip *chip); +int dw_edma_remove(struct dw_edma_chip *chip); +#else +static inline int dw_edma_probe(struct dw_edma_chip *chip) +{ + return -ENODEV; +} + +static inline int dw_edma_remove(struct dw_edma_chip *chip) +{ + return 0; +} +#endif /* CONFIG_DW_EDMA */ + +#endif /* _DW_EDMA_H */ -- 2.7.4