From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.4 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AC02C433E0 for ; Wed, 10 Mar 2021 06:50:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C84B164F59 for ; Wed, 10 Mar 2021 06:50:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229927AbhCJGtm (ORCPT ); Wed, 10 Mar 2021 01:49:42 -0500 Received: from Mailgw01.mediatek.com ([1.203.163.78]:8722 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S229525AbhCJGtR (ORCPT ); Wed, 10 Mar 2021 01:49:17 -0500 X-UUID: 002c64e8b546470ea124308c3d39ca54-20210310 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=D09TyNylBKbStaKtchQcUElsm48BUYZhG4qREnS+nW8=; b=iNSLqXwKCGEvjDb4TpnaEt0ZEOldsuTr1Up6CViLIgoKTf26+FVek+pbSiXUkyubvK1b3dnpWztPXmYus7GOnGmdL6eqmkXMTqyQugdx6ndutXj9hTaBRWZw98E7Jnqy99uqfgkOJ2zKYQgjFUt0wYesSRf94esS7i69NXnS3k8=; X-UUID: 002c64e8b546470ea124308c3d39ca54-20210310 Received: from mtkcas32.mediatek.inc [(172.27.4.253)] by mailgw01.mediatek.com (envelope-from ) (mailgw01.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 670937930; Wed, 10 Mar 2021 14:48:59 +0800 Received: from MTKCAS36.mediatek.inc (172.27.4.186) by MTKMBS31N1.mediatek.inc (172.27.4.69) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 10 Mar 2021 14:48:49 +0800 Received: from [10.17.3.153] (10.17.3.153) by MTKCAS36.mediatek.inc (172.27.4.170) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 10 Mar 2021 14:48:49 +0800 Message-ID: <1615358929.25662.47.camel@mhfsdcap03> Subject: Re: [v8,5/7] PCI: mediatek-gen3: Add MSI support From: Jianjun Wang To: Marc Zyngier CC: Bjorn Helgaas , Rob Herring , Lorenzo Pieralisi , Ryder Lee , Philipp Zabel , "Matthias Brugger" , , , , , , "Sj Huang" , , , , , , , Date: Wed, 10 Mar 2021 14:48:49 +0800 In-Reply-To: <87pn08y3ja.wl-maz@kernel.org> References: <20210224061132.26526-1-jianjun.wang@mediatek.com> <20210224061132.26526-6-jianjun.wang@mediatek.com> <87pn08y3ja.wl-maz@kernel.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-TM-SNTS-SMTP: 9BAB50CC64708D9103176E08435B1B5ED04930DF72E0FB2407164EEC1CDFC2732000:8 X-MTK: N Content-Transfer-Encoding: base64 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org SGkgTWFyYywNCg0KVGhhbmtzIGZvciB5b3VyIHJldmlldy4NCg0KT24gVHVlLCAyMDIxLTAzLTA5 IGF0IDExOjIzICswMDAwLCBNYXJjIFp5bmdpZXIgd3JvdGU6DQo+IE9uIFdlZCwgMjQgRmViIDIw MjEgMDY6MTE6MzAgKzAwMDAsDQo+IEppYW5qdW4gV2FuZyA8amlhbmp1bi53YW5nQG1lZGlhdGVr LmNvbT4gd3JvdGU6DQo+ID4gDQo+ID4gQWRkIE1TSSBzdXBwb3J0IGZvciBNZWRpYVRlayBHZW4z IFBDSWUgY29udHJvbGxlci4NCj4gPiANCj4gPiBUaGlzIFBDSWUgY29udHJvbGxlciBzdXBwb3J0 cyB1cCB0byAyNTYgTVNJIHZlY3RvcnMsIHRoZSBNU0kgaGFyZHdhcmUNCj4gPiBibG9jayBkaWFn cmFtIGlzIGFzIGZvbGxvd3M6DQo+ID4gDQo+ID4gICAgICAgICAgICAgICAgICAgKy0tLS0tKw0K PiA+ICAgICAgICAgICAgICAgICAgIHwgR0lDIHwNCj4gPiAgICAgICAgICAgICAgICAgICArLS0t LS0rDQo+ID4gICAgICAgICAgICAgICAgICAgICAgXg0KPiA+ICAgICAgICAgICAgICAgICAgICAg IHwNCj4gPiAgICAgICAgICAgICAgICAgIHBvcnQtPmlycQ0KPiA+ICAgICAgICAgICAgICAgICAg ICAgIHwNCj4gPiAgICAgICAgICAgICAgKy0rLSstKy0rLSstKy0rLSsNCj4gPiAgICAgICAgICAg ICAgfDB8MXwyfDN8NHw1fDZ8N3wgKFBDSWUgaW50YykNCj4gPiAgICAgICAgICAgICAgKy0rLSst Ky0rLSstKy0rLSsNCj4gPiAgICAgICAgICAgICAgIF4gXiAgICAgICAgICAgXg0KPiA+ICAgICAg ICAgICAgICAgfCB8ICAgIC4uLiAgICB8DQo+ID4gICAgICAgKy0tLS0tLS0rICstLS0tLS0rICAg ICstLS0tLS0tLS0tLSsNCj4gPiAgICAgICB8ICAgICAgICAgICAgICAgIHwgICAgICAgICAgICAg ICAgfA0KPiA+ICstKy0rLS0tKy0tKy0tKyAgKy0rLSstLS0rLS0rLS0rICArLSstKy0tLSstLSst LSsNCj4gPiB8MHwxfC4uLnwzMHwzMXwgIHwwfDF8Li4ufDMwfDMxfCAgfDB8MXwuLi58MzB8MzF8 IChNU0kgc2V0cykNCj4gPiArLSstKy0tLSstLSstLSsgICstKy0rLS0tKy0tKy0tKyAgKy0rLSst LS0rLS0rLS0rDQo+ID4gIF4gXiAgICAgIF4gIF4gICAgXiBeICAgICAgXiAgXiAgICBeIF4gICAg ICBeICBeDQo+ID4gIHwgfCAgICAgIHwgIHwgICAgfCB8ICAgICAgfCAgfCAgICB8IHwgICAgICB8 ICB8ICAoTVNJIHZlY3RvcnMpDQo+ID4gIHwgfCAgICAgIHwgIHwgICAgfCB8ICAgICAgfCAgfCAg ICB8IHwgICAgICB8ICB8DQo+ID4gDQo+ID4gICAoTVNJIFNFVDApICAgICAgIChNU0kgU0VUMSkg IC4uLiAgIChNU0kgU0VUNykNCj4gPiANCj4gPiBXaXRoIDI1NiBNU0kgdmVjdG9ycyBzdXBwb3J0 ZWQsIHRoZSBNU0kgdmVjdG9ycyBhcmUgY29tcG9zZWQgb2YgOCBzZXRzLA0KPiA+IGVhY2ggc2V0 IGhhcyBpdHMgb3duIGFkZHJlc3MgZm9yIE1TSSBtZXNzYWdlLCBhbmQgc3VwcG9ydHMgMzIgTVNJ IHZlY3RvcnMNCj4gPiB0byBnZW5lcmF0ZSBpbnRlcnJ1cHQuDQo+ID4gDQo+ID4gU2lnbmVkLW9m Zi1ieTogSmlhbmp1biBXYW5nIDxqaWFuanVuLndhbmdAbWVkaWF0ZWsuY29tPg0KPiA+IEFja2Vk LWJ5OiBSeWRlciBMZWUgPHJ5ZGVyLmxlZUBtZWRpYXRlay5jb20+DQo+ID4gLS0tDQo+ID4gIGRy aXZlcnMvcGNpL2NvbnRyb2xsZXIvcGNpZS1tZWRpYXRlay1nZW4zLmMgfCAyNzcgKysrKysrKysr KysrKysrKysrKysNCj4gPiAgMSBmaWxlIGNoYW5nZWQsIDI3NyBpbnNlcnRpb25zKCspDQo+ID4g DQo+ID4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvcGNpL2NvbnRyb2xsZXIvcGNpZS1tZWRpYXRlay1n ZW4zLmMgYi9kcml2ZXJzL3BjaS9jb250cm9sbGVyL3BjaWUtbWVkaWF0ZWstZ2VuMy5jDQo+ID4g aW5kZXggOGIzYjVmODM4YjY5Li4zY2JlYzIyZWNlMGMgMTAwNjQ0DQo+ID4gLS0tIGEvZHJpdmVy cy9wY2kvY29udHJvbGxlci9wY2llLW1lZGlhdGVrLWdlbjMuYw0KPiA+ICsrKyBiL2RyaXZlcnMv cGNpL2NvbnRyb2xsZXIvcGNpZS1tZWRpYXRlay1nZW4zLmMNCj4gPiBAQCAtMTQsNiArMTQsNyBA QA0KPiA+ICAjaW5jbHVkZSA8bGludXgvaXJxZG9tYWluLmg+DQo+ID4gICNpbmNsdWRlIDxsaW51 eC9rZXJuZWwuaD4NCj4gPiAgI2luY2x1ZGUgPGxpbnV4L21vZHVsZS5oPg0KPiA+ICsjaW5jbHVk ZSA8bGludXgvbXNpLmg+DQo+ID4gICNpbmNsdWRlIDxsaW51eC9wY2kuaD4NCj4gPiAgI2luY2x1 ZGUgPGxpbnV4L3BoeS9waHkuaD4NCj4gPiAgI2luY2x1ZGUgPGxpbnV4L3BsYXRmb3JtX2Rldmlj ZS5oPg0KPiA+IEBAIC00OCwxMiArNDksMjkgQEANCj4gPiAgI2RlZmluZSBQQ0lFX0xJTktfU1RB VFVTX1JFRwkJMHgxNTQNCj4gPiAgI2RlZmluZSBQQ0lFX1BPUlRfTElOS1VQCQlCSVQoOCkNCj4g PiAgDQo+ID4gKyNkZWZpbmUgUENJRV9NU0lfU0VUX05VTQkJOA0KPiA+ICsjZGVmaW5lIFBDSUVf TVNJX0lSUVNfUEVSX1NFVAkJMzINCj4gPiArI2RlZmluZSBQQ0lFX01TSV9JUlFTX05VTSBcDQo+ ID4gKwkoUENJRV9NU0lfSVJRU19QRVJfU0VUICogUENJRV9NU0lfU0VUX05VTSkNCj4gPiArDQo+ ID4gICNkZWZpbmUgUENJRV9JTlRfRU5BQkxFX1JFRwkJMHgxODANCj4gPiArI2RlZmluZSBQQ0lF X01TSV9FTkFCTEUJCQlHRU5NQVNLKFBDSUVfTVNJX1NFVF9OVU0gKyA4IC0gMSwgOCkNCj4gPiAr I2RlZmluZSBQQ0lFX01TSV9TSElGVAkJCTgNCj4gPiAgI2RlZmluZSBQQ0lFX0lOVFhfU0hJRlQJ CQkyNA0KPiA+ICAjZGVmaW5lIFBDSUVfSU5UWF9FTkFCTEUgXA0KPiA+ICAJR0VOTUFTSyhQQ0lF X0lOVFhfU0hJRlQgKyBQQ0lfTlVNX0lOVFggLSAxLCBQQ0lFX0lOVFhfU0hJRlQpDQo+ID4gIA0K PiA+ICAjZGVmaW5lIFBDSUVfSU5UX1NUQVRVU19SRUcJCTB4MTg0DQo+ID4gKyNkZWZpbmUgUENJ RV9NU0lfU0VUX0VOQUJMRV9SRUcJCTB4MTkwDQo+ID4gKyNkZWZpbmUgUENJRV9NU0lfU0VUX0VO QUJMRQkJR0VOTUFTSyhQQ0lFX01TSV9TRVRfTlVNIC0gMSwgMCkNCj4gPiArDQo+ID4gKyNkZWZp bmUgUENJRV9NU0lfU0VUX0JBU0VfUkVHCQkweGMwMA0KPiA+ICsjZGVmaW5lIFBDSUVfTVNJX1NF VF9PRkZTRVQJCTB4MTANCj4gPiArI2RlZmluZSBQQ0lFX01TSV9TRVRfU1RBVFVTX09GRlNFVAkw eDA0DQo+ID4gKyNkZWZpbmUgUENJRV9NU0lfU0VUX0VOQUJMRV9PRkZTRVQJMHgwOA0KPiA+ICsN Cj4gPiArI2RlZmluZSBQQ0lFX01TSV9TRVRfQUREUl9ISV9CQVNFCTB4YzgwDQo+ID4gKyNkZWZp bmUgUENJRV9NU0lfU0VUX0FERFJfSElfT0ZGU0VUCTB4MDQNCj4gPiAgDQo+ID4gICNkZWZpbmUg UENJRV9UUkFOU19UQUJMRV9CQVNFX1JFRwkweDgwMA0KPiA+ICAjZGVmaW5lIFBDSUVfQVRSX1NS Q19BRERSX01TQl9PRkZTRVQJMHg0DQo+ID4gQEAgLTczLDYgKzkxLDE2IEBADQo+ID4gICNkZWZp bmUgUENJRV9BVFJfVExQX1RZUEVfTUVNCQlQQ0lFX0FUUl9UTFBfVFlQRSgwKQ0KPiA+ICAjZGVm aW5lIFBDSUVfQVRSX1RMUF9UWVBFX0lPCQlQQ0lFX0FUUl9UTFBfVFlQRSgyKQ0KPiA+ICANCj4g PiArLyoqDQo+ID4gKyAqIHN0cnVjdCBtdGtfcGNpZV9tc2kgLSBNU0kgaW5mb3JtYXRpb24gZm9y IGVhY2ggc2V0DQo+ID4gKyAqIEBiYXNlOiBJTyBtYXBwZWQgcmVnaXN0ZXIgYmFzZQ0KPiA+ICsg KiBAbXNnX2FkZHI6IE1TSSBtZXNzYWdlIGFkZHJlc3MNCj4gPiArICovDQo+ID4gK3N0cnVjdCBt dGtfbXNpX3NldCB7DQo+ID4gKwl2b2lkIF9faW9tZW0gKmJhc2U7DQo+ID4gKwlwaHlzX2FkZHJf dCBtc2dfYWRkcjsNCj4gPiArfTsNCj4gPiArDQo+ID4gIC8qKg0KPiA+ICAgKiBzdHJ1Y3QgbXRr X3BjaWVfcG9ydCAtIFBDSWUgcG9ydCBpbmZvcm1hdGlvbg0KPiA+ICAgKiBAZGV2OiBwb2ludGVy IHRvIFBDSWUgZGV2aWNlDQo+ID4gQEAgLTg2LDYgKzExNCwxMSBAQA0KPiA+ICAgKiBAaXJxOiBQ Q0llIGNvbnRyb2xsZXIgaW50ZXJydXB0IG51bWJlcg0KPiA+ICAgKiBAaXJxX2xvY2s6IGxvY2sg cHJvdGVjdGluZyBJUlEgcmVnaXN0ZXIgYWNjZXNzDQo+ID4gICAqIEBpbnR4X2RvbWFpbjogbGVn YWN5IElOVHggSVJRIGRvbWFpbg0KPiA+ICsgKiBAbXNpX2RvbWFpbjogTVNJIElSUSBkb21haW4N Cj4gPiArICogQG1zaV9ib3R0b21fZG9tYWluOiBNU0kgSVJRIGJvdHRvbSBkb21haW4NCj4gPiAr ICogQG1zaV9zZXRzOiBNU0kgc2V0cyBpbmZvcm1hdGlvbg0KPiA+ICsgKiBAbG9jazogbG9jayBw cm90ZWN0aW5nIElSUSBiaXQgbWFwDQo+ID4gKyAqIEBtc2lfaXJxX2luX3VzZTogYml0IG1hcCBm b3IgYXNzaWduZWQgTVNJIElSUQ0KPiA+ICAgKi8NCj4gPiAgc3RydWN0IG10a19wY2llX3BvcnQg ew0KPiA+ICAJc3RydWN0IGRldmljZSAqZGV2Ow0KPiA+IEBAIC0xMDAsNiArMTMzLDExIEBAIHN0 cnVjdCBtdGtfcGNpZV9wb3J0IHsNCj4gPiAgCWludCBpcnE7DQo+ID4gIAlyYXdfc3BpbmxvY2tf dCBpcnFfbG9jazsNCj4gPiAgCXN0cnVjdCBpcnFfZG9tYWluICppbnR4X2RvbWFpbjsNCj4gPiAr CXN0cnVjdCBpcnFfZG9tYWluICptc2lfZG9tYWluOw0KPiA+ICsJc3RydWN0IGlycV9kb21haW4g Km1zaV9ib3R0b21fZG9tYWluOw0KPiA+ICsJc3RydWN0IG10a19tc2lfc2V0IG1zaV9zZXRzW1BD SUVfTVNJX1NFVF9OVU1dOw0KPiA+ICsJc3RydWN0IG11dGV4IGxvY2s7DQo+ID4gKwlERUNMQVJF X0JJVE1BUChtc2lfaXJxX2luX3VzZSwgUENJRV9NU0lfSVJRU19OVU0pOw0KPiA+ICB9Ow0KPiA+ ICANCj4gPiAgLyoqDQo+ID4gQEAgLTE5Nyw2ICsyMzUsMzUgQEAgc3RhdGljIGludCBtdGtfcGNp ZV9zZXRfdHJhbnNfdGFibGUoc3RydWN0IG10a19wY2llX3BvcnQgKnBvcnQsDQo+ID4gIAlyZXR1 cm4gMDsNCj4gPiAgfQ0KPiA+ICANCj4gPiArc3RhdGljIHZvaWQgbXRrX3BjaWVfZW5hYmxlX21z aShzdHJ1Y3QgbXRrX3BjaWVfcG9ydCAqcG9ydCkNCj4gPiArew0KPiA+ICsJaW50IGk7DQo+ID4g Kwl1MzIgdmFsOw0KPiA+ICsNCj4gPiArCXZhbCA9IHJlYWRsX3JlbGF4ZWQocG9ydC0+YmFzZSAr IFBDSUVfTVNJX1NFVF9FTkFCTEVfUkVHKTsNCj4gPiArCXZhbCB8PSBQQ0lFX01TSV9TRVRfRU5B QkxFOw0KPiA+ICsJd3JpdGVsX3JlbGF4ZWQodmFsLCBwb3J0LT5iYXNlICsgUENJRV9NU0lfU0VU X0VOQUJMRV9SRUcpOw0KPiA+ICsNCj4gPiArCXZhbCA9IHJlYWRsX3JlbGF4ZWQocG9ydC0+YmFz ZSArIFBDSUVfSU5UX0VOQUJMRV9SRUcpOw0KPiA+ICsJdmFsIHw9IFBDSUVfTVNJX0VOQUJMRTsN Cj4gPiArCXdyaXRlbF9yZWxheGVkKHZhbCwgcG9ydC0+YmFzZSArIFBDSUVfSU5UX0VOQUJMRV9S RUcpOw0KPiANCj4gU2hvdWxkbid0IHlvdSBjb25maWd1cmUgdGhlIGNhcHR1cmUgYWRkcmVzc2Vz ICpiZWZvcmUqIGVuYWJsaW5nDQo+IHRoaW5ncz8gSXMgdGhlcmUgYW55IG5lZWQgZm9yIGxvY2tp bmcgaGVyZSwgZ2l2ZW4gdGhhdCB5b3UgYXJlDQo+IG1vZGlmeWluZyBnbG9iYWwgcmVnaXN0ZXJz Pw0KDQpZZXMsIEkgd2lsbCBtb3ZlIHRoZXNlIGNvZGVzIHRvIHRoZSBiYWNrIG9mIHRoZSBjb25m aWd1cmUgY2FwdHVyZQ0KYWRkcmVzcyBpbiB0aGUgbmV4dCB2ZXJzaW9uLg0KDQpJIHRoaW5rIHRo ZSBsb2NrIG1heSBub3QgYmUgbmVlZGVkIGJlY2F1c2UgdGhpcyBmdW5jdGlvbiBpcyBvbmx5DQpl eGVjdXRlZCBvbmNlIHdoZW4gZHJpdmVyIHByb2JlLg0KDQo+IA0KPiA+ICsNCj4gPiArCWZvciAo aSA9IDA7IGkgPCBQQ0lFX01TSV9TRVRfTlVNOyBpKyspIHsNCj4gPiArCQlzdHJ1Y3QgbXRrX21z aV9zZXQgKm1zaV9zZXQgPSAmcG9ydC0+bXNpX3NldHNbaV07DQo+ID4gKw0KPiA+ICsJCW1zaV9z ZXQtPmJhc2UgPSBwb3J0LT5iYXNlICsgUENJRV9NU0lfU0VUX0JBU0VfUkVHICsNCj4gPiArCQkJ CWkgKiBQQ0lFX01TSV9TRVRfT0ZGU0VUOw0KPiA+ICsJCW1zaV9zZXQtPm1zZ19hZGRyID0gcG9y dC0+cmVnX2Jhc2UgKyBQQ0lFX01TSV9TRVRfQkFTRV9SRUcgKw0KPiA+ICsJCQkJICAgIGkgKiBQ Q0lFX01TSV9TRVRfT0ZGU0VUOw0KPiA+ICsNCj4gPiArCQkvKiBDb25maWd1cmUgdGhlIE1TSSBj YXB0dXJlIGFkZHJlc3MgKi8NCj4gPiArCQl3cml0ZWxfcmVsYXhlZChsb3dlcl8zMl9iaXRzKG1z aV9zZXQtPm1zZ19hZGRyKSwgbXNpX3NldC0+YmFzZSk7DQo+ID4gKwkJd3JpdGVsX3JlbGF4ZWQo dXBwZXJfMzJfYml0cyhtc2lfc2V0LT5tc2dfYWRkciksDQo+ID4gKwkJCSAgICAgICBwb3J0LT5i YXNlICsgUENJRV9NU0lfU0VUX0FERFJfSElfQkFTRSArDQo+ID4gKwkJCSAgICAgICBpICogUENJ RV9NU0lfU0VUX0FERFJfSElfT0ZGU0VUKTsNCj4gPiArCX0NCj4gPiArfQ0KPiA+ICsNCj4gPiAg c3RhdGljIGludCBtdGtfcGNpZV9zdGFydHVwX3BvcnQoc3RydWN0IG10a19wY2llX3BvcnQgKnBv cnQpDQo+ID4gIHsNCj4gPiAgCXN0cnVjdCByZXNvdXJjZV9lbnRyeSAqZW50cnk7DQo+ID4gQEAg LTI0Nyw2ICszMTQsOCBAQCBzdGF0aWMgaW50IG10a19wY2llX3N0YXJ0dXBfcG9ydChzdHJ1Y3Qg bXRrX3BjaWVfcG9ydCAqcG9ydCkNCj4gPiAgCQlyZXR1cm4gZXJyOw0KPiA+ICAJfQ0KPiA+ICAN Cj4gPiArCW10a19wY2llX2VuYWJsZV9tc2kocG9ydCk7DQo+ID4gKw0KPiA+ICAJLyogU2V0IFBD SWUgdHJhbnNsYXRpb24gd2luZG93cyAqLw0KPiA+ICAJcmVzb3VyY2VfbGlzdF9mb3JfZWFjaF9l bnRyeShlbnRyeSwgJmhvc3QtPndpbmRvd3MpIHsNCj4gPiAgCQlzdHJ1Y3QgcmVzb3VyY2UgKnJl cyA9IGVudHJ5LT5yZXM7DQo+ID4gQEAgLTI5MCw2ICszNTksMTQ4IEBAIHN0YXRpYyBpbnQgbXRr X3BjaWVfc2V0X2FmZmluaXR5KHN0cnVjdCBpcnFfZGF0YSAqZGF0YSwNCj4gPiAgCXJldHVybiAt RUlOVkFMOw0KPiA+ICB9DQo+ID4gIA0KPiA+ICtzdGF0aWMgdm9pZCBtdGtfcGNpZV9pcnFfbWFz ayhzdHJ1Y3QgaXJxX2RhdGEgKmRhdGEpDQo+IA0KPiBJdCdkIGJlIGdvb2QgaWYgeW91IHVzZWQg X21zaV8gaW4gZnVuY3Rpb24gbmFtZXMgdGhhdCBkZWFsIHdpdGggTVNJcy4NCg0KT0ssIEkgd2ls bCBmaXggaXQgaW4gdGhlIG5leHQgdmVyc2lvbi4NCg0KPiANCj4gPiArew0KPiA+ICsJcGNpX21z aV9tYXNrX2lycShkYXRhKTsNCj4gPiArCWlycV9jaGlwX21hc2tfcGFyZW50KGRhdGEpOw0KPiA+ ICt9DQo+ID4gKw0KPiA+ICtzdGF0aWMgdm9pZCBtdGtfcGNpZV9pcnFfdW5tYXNrKHN0cnVjdCBp cnFfZGF0YSAqZGF0YSkNCj4gPiArew0KPiA+ICsJcGNpX21zaV91bm1hc2tfaXJxKGRhdGEpOw0K PiA+ICsJaXJxX2NoaXBfdW5tYXNrX3BhcmVudChkYXRhKTsNCj4gPiArfQ0KPiA+ICsNCj4gPiAr c3RhdGljIHN0cnVjdCBpcnFfY2hpcCBtdGtfbXNpX2lycV9jaGlwID0gew0KPiA+ICsJLm5hbWUg PSAiTVNJIiwNCj4gPiArCS5pcnFfZW5hYmxlID0gbXRrX3BjaWVfaXJxX3VubWFzaywNCj4gPiAr CS5pcnFfZGlzYWJsZSA9IG10a19wY2llX2lycV9tYXNrLA0KPiANCj4gU2FtZSBjb21tZW50IGFz IGZvciB0aGUgcHJldmlvdXMgcGF0Y2g6IGVuYWJsZS9kaXNhYmxlIHNlcnZlIG5vDQo+IHB1cnBv c2UgaGVyZS4NCg0KUmVwbGllZCBpbiB0aGUgcHJldmlvdXMgcGF0Y2gsIHRoZSBlbmFibGUvZGlz YWJsZSBjYWxsYmFjayBpcyB1c2VkIHdoZW4NCnRoZSBzeXN0ZW0gc3VzcGVuZC9yZXN1bWUuDQoN Cj4gDQo+ID4gKwkuaXJxX2FjayA9IGlycV9jaGlwX2Fja19wYXJlbnQsDQo+ID4gKwkuaXJxX21h c2sgPSBtdGtfcGNpZV9pcnFfbWFzaywNCj4gPiArCS5pcnFfdW5tYXNrID0gbXRrX3BjaWVfaXJx X3VubWFzaywNCj4gPiArfTsNCj4gPiArDQo+ID4gK3N0YXRpYyBzdHJ1Y3QgbXNpX2RvbWFpbl9p bmZvIG10a19tc2lfZG9tYWluX2luZm8gPSB7DQo+ID4gKwkuZmxhZ3MJPSAoTVNJX0ZMQUdfVVNF X0RFRl9ET01fT1BTIHwgTVNJX0ZMQUdfUENJX01TSVggfA0KPiA+ICsJCSAgIE1TSV9GTEFHX1VT RV9ERUZfQ0hJUF9PUFMgfCBNU0lfRkxBR19NVUxUSV9QQ0lfTVNJKSwNCj4gDQo+IG1pbm9yIG5p dDoga2VlcCB0aGUgKl9PUFMgZmxhZyBvbiBvbmUgbGluZSwgYW5kIHRoZSAqX1BDSV8qIGZsYWdz IG9uDQo+IGFub3RoZXIuDQoNClN1cmUsIEkgd2lsbCBmaXggaXQgaW4gdGhlIG5leHQgdmVyc2lv bi4NCg0KPiANCj4gPiArCS5jaGlwCT0gJm10a19tc2lfaXJxX2NoaXAsDQo+ID4gK307DQo+ID4g Kw0KPiA+ICtzdGF0aWMgdm9pZCBtdGtfY29tcG9zZV9tc2lfbXNnKHN0cnVjdCBpcnFfZGF0YSAq ZGF0YSwgc3RydWN0IG1zaV9tc2cgKm1zZykNCj4gPiArew0KPiA+ICsJc3RydWN0IG10a19tc2lf c2V0ICptc2lfc2V0ID0gaXJxX2RhdGFfZ2V0X2lycV9jaGlwX2RhdGEoZGF0YSk7DQo+ID4gKwlz dHJ1Y3QgbXRrX3BjaWVfcG9ydCAqcG9ydCA9IGRhdGEtPmRvbWFpbi0+aG9zdF9kYXRhOw0KPiA+ ICsJdW5zaWduZWQgbG9uZyBod2lycTsNCj4gPiArDQo+ID4gKwlod2lycSA9CWRhdGEtPmh3aXJx ICUgUENJRV9NU0lfSVJRU19QRVJfU0VUOw0KPiA+ICsNCj4gPiArCW1zZy0+YWRkcmVzc19oaSA9 IHVwcGVyXzMyX2JpdHMobXNpX3NldC0+bXNnX2FkZHIpOw0KPiA+ICsJbXNnLT5hZGRyZXNzX2xv ID0gbG93ZXJfMzJfYml0cyhtc2lfc2V0LT5tc2dfYWRkcik7DQo+ID4gKwltc2ctPmRhdGEgPSBo d2lycTsNCj4gPiArCWRldl9kYmcocG9ydC0+ZGV2LCAibXNpIyUjbHggYWRkcmVzc19oaSAlI3gg YWRkcmVzc19sbyAlI3ggZGF0YSAlZFxuIiwNCj4gPiArCQlod2lycSwgbXNnLT5hZGRyZXNzX2hp LCBtc2ctPmFkZHJlc3NfbG8sIG1zZy0+ZGF0YSk7DQo+ID4gK30NCj4gPiArDQo+ID4gK3N0YXRp YyB2b2lkIG10a19tc2lfYm90dG9tX2lycV9hY2soc3RydWN0IGlycV9kYXRhICpkYXRhKQ0KPiA+ ICt7DQo+ID4gKwlzdHJ1Y3QgbXRrX21zaV9zZXQgKm1zaV9zZXQgPSBpcnFfZGF0YV9nZXRfaXJx X2NoaXBfZGF0YShkYXRhKTsNCj4gPiArCXVuc2lnbmVkIGxvbmcgaHdpcnE7DQo+ID4gKw0KPiA+ ICsJaHdpcnEgPQlkYXRhLT5od2lycSAlIFBDSUVfTVNJX0lSUVNfUEVSX1NFVDsNCj4gPiArDQo+ ID4gKwl3cml0ZWxfcmVsYXhlZChCSVQoaHdpcnEpLCBtc2lfc2V0LT5iYXNlICsgUENJRV9NU0lf U0VUX1NUQVRVU19PRkZTRVQpOw0KPiA+ICt9DQo+ID4gKw0KPiA+ICtzdGF0aWMgdm9pZCBtdGtf bXNpX2JvdHRvbV9pcnFfbWFzayhzdHJ1Y3QgaXJxX2RhdGEgKmRhdGEpDQo+ID4gK3sNCj4gPiAr CXN0cnVjdCBtdGtfbXNpX3NldCAqbXNpX3NldCA9IGlycV9kYXRhX2dldF9pcnFfY2hpcF9kYXRh KGRhdGEpOw0KPiA+ICsJc3RydWN0IG10a19wY2llX3BvcnQgKnBvcnQgPSBkYXRhLT5kb21haW4t Pmhvc3RfZGF0YTsNCj4gPiArCXVuc2lnbmVkIGxvbmcgaHdpcnEsIGZsYWdzOw0KPiA+ICsJdTMy IHZhbDsNCj4gPiArDQo+ID4gKwlod2lycSA9CWRhdGEtPmh3aXJxICUgUENJRV9NU0lfSVJRU19Q RVJfU0VUOw0KPiA+ICsNCj4gPiArCXJhd19zcGluX2xvY2tfaXJxc2F2ZSgmcG9ydC0+aXJxX2xv Y2ssIGZsYWdzKTsNCj4gPiArCXZhbCA9IHJlYWRsX3JlbGF4ZWQobXNpX3NldC0+YmFzZSArIFBD SUVfTVNJX1NFVF9FTkFCTEVfT0ZGU0VUKTsNCj4gPiArCXZhbCAmPSB+QklUKGh3aXJxKTsNCj4g PiArCXdyaXRlbF9yZWxheGVkKHZhbCwgbXNpX3NldC0+YmFzZSArIFBDSUVfTVNJX1NFVF9FTkFC TEVfT0ZGU0VUKTsNCj4gPiArCXJhd19zcGluX3VubG9ja19pcnFyZXN0b3JlKCZwb3J0LT5pcnFf bG9jaywgZmxhZ3MpOw0KPiA+ICt9DQo+ID4gKw0KPiA+ICtzdGF0aWMgdm9pZCBtdGtfbXNpX2Jv dHRvbV9pcnFfdW5tYXNrKHN0cnVjdCBpcnFfZGF0YSAqZGF0YSkNCj4gPiArew0KPiA+ICsJc3Ry dWN0IG10a19tc2lfc2V0ICptc2lfc2V0ID0gaXJxX2RhdGFfZ2V0X2lycV9jaGlwX2RhdGEoZGF0 YSk7DQo+ID4gKwlzdHJ1Y3QgbXRrX3BjaWVfcG9ydCAqcG9ydCA9IGRhdGEtPmRvbWFpbi0+aG9z dF9kYXRhOw0KPiA+ICsJdW5zaWduZWQgbG9uZyBod2lycSwgZmxhZ3M7DQo+ID4gKwl1MzIgdmFs Ow0KPiA+ICsNCj4gPiArCWh3aXJxID0JZGF0YS0+aHdpcnEgJSBQQ0lFX01TSV9JUlFTX1BFUl9T RVQ7DQo+ID4gKw0KPiA+ICsJcmF3X3NwaW5fbG9ja19pcnFzYXZlKCZwb3J0LT5pcnFfbG9jaywg ZmxhZ3MpOw0KPiA+ICsJdmFsID0gcmVhZGxfcmVsYXhlZChtc2lfc2V0LT5iYXNlICsgUENJRV9N U0lfU0VUX0VOQUJMRV9PRkZTRVQpOw0KPiA+ICsJdmFsIHw9IEJJVChod2lycSk7DQo+ID4gKwl3 cml0ZWxfcmVsYXhlZCh2YWwsIG1zaV9zZXQtPmJhc2UgKyBQQ0lFX01TSV9TRVRfRU5BQkxFX09G RlNFVCk7DQo+ID4gKwlyYXdfc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmcG9ydC0+aXJxX2xvY2ss IGZsYWdzKTsNCj4gPiArfQ0KPiA+ICsNCj4gPiArc3RhdGljIHN0cnVjdCBpcnFfY2hpcCBtdGtf bXNpX2JvdHRvbV9pcnFfY2hpcCA9IHsNCj4gPiArCS5pcnFfYWNrCQk9IG10a19tc2lfYm90dG9t X2lycV9hY2ssDQo+ID4gKwkuaXJxX21hc2sJCT0gbXRrX21zaV9ib3R0b21faXJxX21hc2ssDQo+ ID4gKwkuaXJxX3VubWFzawkJPSBtdGtfbXNpX2JvdHRvbV9pcnFfdW5tYXNrLA0KPiA+ICsJLmly cV9jb21wb3NlX21zaV9tc2cJPSBtdGtfY29tcG9zZV9tc2lfbXNnLA0KPiA+ICsJLmlycV9zZXRf YWZmaW5pdHkJPSBtdGtfcGNpZV9zZXRfYWZmaW5pdHksDQo+ID4gKwkubmFtZQkJCT0gIk1TSSIs DQo+ID4gK307DQo+ID4gKw0KPiA+ICtzdGF0aWMgaW50IG10a19tc2lfYm90dG9tX2RvbWFpbl9h bGxvYyhzdHJ1Y3QgaXJxX2RvbWFpbiAqZG9tYWluLA0KPiA+ICsJCQkJICAgICAgIHVuc2lnbmVk IGludCB2aXJxLCB1bnNpZ25lZCBpbnQgbnJfaXJxcywNCj4gPiArCQkJCSAgICAgICB2b2lkICph cmcpDQo+ID4gK3sNCj4gPiArCXN0cnVjdCBtdGtfcGNpZV9wb3J0ICpwb3J0ID0gZG9tYWluLT5o b3N0X2RhdGE7DQo+ID4gKwlzdHJ1Y3QgbXRrX21zaV9zZXQgKm1zaV9zZXQ7DQo+ID4gKwlpbnQg aSwgaHdpcnEsIHNldF9pZHg7DQo+ID4gKw0KPiA+ICsJbXV0ZXhfbG9jaygmcG9ydC0+bG9jayk7 DQo+ID4gKw0KPiA+ICsJaHdpcnEgPSBiaXRtYXBfZmluZF9mcmVlX3JlZ2lvbihwb3J0LT5tc2lf aXJxX2luX3VzZSwgUENJRV9NU0lfSVJRU19OVU0sDQo+ID4gKwkJCQkJb3JkZXJfYmFzZV8yKG5y X2lycXMpKTsNCj4gPiArDQo+ID4gKwltdXRleF91bmxvY2soJnBvcnQtPmxvY2spOw0KPiA+ICsN Cj4gPiArCWlmIChod2lycSA8IDApDQo+ID4gKwkJcmV0dXJuIC1FTk9TUEM7DQo+ID4gKw0KPiA+ ICsJc2V0X2lkeCA9IGh3aXJxIC8gUENJRV9NU0lfSVJRU19QRVJfU0VUOw0KPiA+ICsJbXNpX3Nl dCA9ICZwb3J0LT5tc2lfc2V0c1tzZXRfaWR4XTsNCj4gPiArDQo+ID4gKwlmb3IgKGkgPSAwOyBp IDwgbnJfaXJxczsgaSsrKQ0KPiA+ICsJCWlycV9kb21haW5fc2V0X2luZm8oZG9tYWluLCB2aXJx ICsgaSwgaHdpcnEgKyBpLA0KPiA+ICsJCQkJICAgICZtdGtfbXNpX2JvdHRvbV9pcnFfY2hpcCwg bXNpX3NldCwNCj4gPiArCQkJCSAgICBoYW5kbGVfZWRnZV9pcnEsIE5VTEwsIE5VTEwpOw0KPiA+ ICsNCj4gPiArCXJldHVybiAwOw0KPiA+ICt9DQo+ID4gKw0KPiA+ICtzdGF0aWMgdm9pZCBtdGtf bXNpX2JvdHRvbV9kb21haW5fZnJlZShzdHJ1Y3QgaXJxX2RvbWFpbiAqZG9tYWluLA0KPiA+ICsJ CQkJICAgICAgIHVuc2lnbmVkIGludCB2aXJxLCB1bnNpZ25lZCBpbnQgbnJfaXJxcykNCj4gPiAr ew0KPiA+ICsJc3RydWN0IG10a19wY2llX3BvcnQgKnBvcnQgPSBkb21haW4tPmhvc3RfZGF0YTsN Cj4gPiArCXN0cnVjdCBpcnFfZGF0YSAqZGF0YSA9IGlycV9kb21haW5fZ2V0X2lycV9kYXRhKGRv bWFpbiwgdmlycSk7DQo+ID4gKw0KPiA+ICsJbXV0ZXhfbG9jaygmcG9ydC0+bG9jayk7DQo+ID4g Kw0KPiA+ICsJYml0bWFwX2NsZWFyKHBvcnQtPm1zaV9pcnFfaW5fdXNlLCBkYXRhLT5od2lycSwg bnJfaXJxcyk7DQo+ID4gKw0KPiA+ICsJbXV0ZXhfdW5sb2NrKCZwb3J0LT5sb2NrKTsNCj4gPiAr DQo+ID4gKwlpcnFfZG9tYWluX2ZyZWVfaXJxc19jb21tb24oZG9tYWluLCB2aXJxLCBucl9pcnFz KTsNCj4gPiArfQ0KPiA+ICsNCj4gPiArc3RhdGljIGNvbnN0IHN0cnVjdCBpcnFfZG9tYWluX29w cyBtdGtfbXNpX2JvdHRvbV9kb21haW5fb3BzID0gew0KPiA+ICsJLmFsbG9jID0gbXRrX21zaV9i b3R0b21fZG9tYWluX2FsbG9jLA0KPiA+ICsJLmZyZWUgPSBtdGtfbXNpX2JvdHRvbV9kb21haW5f ZnJlZSwNCj4gPiArfTsNCj4gPiArDQo+ID4gIHN0YXRpYyB2b2lkIG10a19pbnR4X21hc2soc3Ry dWN0IGlycV9kYXRhICpkYXRhKQ0KPiA+ICB7DQo+ID4gIAlzdHJ1Y3QgbXRrX3BjaWVfcG9ydCAq cG9ydCA9IGlycV9kYXRhX2dldF9pcnFfY2hpcF9kYXRhKGRhdGEpOw0KPiA+IEBAIC0zNjAsNiAr NTcxLDcgQEAgc3RhdGljIGludCBtdGtfcGNpZV9pbml0X2lycV9kb21haW5zKHN0cnVjdCBtdGtf cGNpZV9wb3J0ICpwb3J0KQ0KPiA+ICB7DQo+ID4gIAlzdHJ1Y3QgZGV2aWNlICpkZXYgPSBwb3J0 LT5kZXY7DQo+ID4gIAlzdHJ1Y3QgZGV2aWNlX25vZGUgKmludGNfbm9kZSwgKm5vZGUgPSBkZXYt Pm9mX25vZGU7DQo+ID4gKwlpbnQgcmV0Ow0KPiA+ICANCj4gPiAgCXJhd19zcGluX2xvY2tfaW5p dCgmcG9ydC0+aXJxX2xvY2spOw0KPiA+ICANCj4gPiBAQCAtMzc3LDcgKzU4OSwzNCBAQCBzdGF0 aWMgaW50IG10a19wY2llX2luaXRfaXJxX2RvbWFpbnMoc3RydWN0IG10a19wY2llX3BvcnQgKnBv cnQpDQo+ID4gIAkJcmV0dXJuIC1FTk9ERVY7DQo+ID4gIAl9DQo+ID4gIA0KPiA+ICsJLyogU2V0 dXAgTVNJICovDQo+ID4gKwltdXRleF9pbml0KCZwb3J0LT5sb2NrKTsNCj4gPiArDQo+ID4gKwlw b3J0LT5tc2lfYm90dG9tX2RvbWFpbiA9IGlycV9kb21haW5fYWRkX2xpbmVhcihub2RlLCBQQ0lF X01TSV9JUlFTX05VTSwNCj4gPiArCQkJCSAgJm10a19tc2lfYm90dG9tX2RvbWFpbl9vcHMsIHBv cnQpOw0KPiA+ICsJaWYgKCFwb3J0LT5tc2lfYm90dG9tX2RvbWFpbikgew0KPiA+ICsJCWRldl9p bmZvKGRldiwgImZhaWxlZCB0byBjcmVhdGUgTVNJIGJvdHRvbSBkb21haW5cbiIpOw0KPiA+ICsJ CXJldCA9IC1FTk9ERVY7DQo+ID4gKwkJZ290byBlcnJfbXNpX2JvdHRvbV9kb21haW47DQo+ID4g Kwl9DQo+ID4gKw0KPiA+ICsJcG9ydC0+bXNpX2RvbWFpbiA9IHBjaV9tc2lfY3JlYXRlX2lycV9k b21haW4oZGV2LT5md25vZGUsDQo+ID4gKwkJCQkJCSAgICAgJm10a19tc2lfZG9tYWluX2luZm8s DQo+ID4gKwkJCQkJCSAgICAgcG9ydC0+bXNpX2JvdHRvbV9kb21haW4pOw0KPiA+ICsJaWYgKCFw b3J0LT5tc2lfZG9tYWluKSB7DQo+ID4gKwkJZGV2X2luZm8oZGV2LCAiZmFpbGVkIHRvIGNyZWF0 ZSBNU0kgZG9tYWluXG4iKTsNCj4gPiArCQlyZXQgPSAtRU5PREVWOw0KPiA+ICsJCWdvdG8gZXJy X21zaV9kb21haW47DQo+ID4gKwl9DQo+ID4gKw0KPiA+ICAJcmV0dXJuIDA7DQo+ID4gKw0KPiA+ ICtlcnJfbXNpX2RvbWFpbjoNCj4gPiArCWlycV9kb21haW5fcmVtb3ZlKHBvcnQtPm1zaV9ib3R0 b21fZG9tYWluKTsNCj4gPiArZXJyX21zaV9ib3R0b21fZG9tYWluOg0KPiA+ICsJaXJxX2RvbWFp bl9yZW1vdmUocG9ydC0+aW50eF9kb21haW4pOw0KPiA+ICsNCj4gPiArCXJldHVybiByZXQ7DQo+ ID4gIH0NCj4gPiAgDQo+ID4gIHN0YXRpYyB2b2lkIG10a19wY2llX2lycV90ZWFyZG93bihzdHJ1 Y3QgbXRrX3BjaWVfcG9ydCAqcG9ydCkNCj4gPiBAQCAtMzg3LDkgKzYyNiwzOSBAQCBzdGF0aWMg dm9pZCBtdGtfcGNpZV9pcnFfdGVhcmRvd24oc3RydWN0IG10a19wY2llX3BvcnQgKnBvcnQpDQo+ ID4gIAlpZiAocG9ydC0+aW50eF9kb21haW4pDQo+ID4gIAkJaXJxX2RvbWFpbl9yZW1vdmUocG9y dC0+aW50eF9kb21haW4pOw0KPiA+ICANCj4gPiArCWlmIChwb3J0LT5tc2lfZG9tYWluKQ0KPiA+ ICsJCWlycV9kb21haW5fcmVtb3ZlKHBvcnQtPm1zaV9kb21haW4pOw0KPiA+ICsNCj4gPiArCWlm IChwb3J0LT5tc2lfYm90dG9tX2RvbWFpbikNCj4gPiArCQlpcnFfZG9tYWluX3JlbW92ZShwb3J0 LT5tc2lfYm90dG9tX2RvbWFpbik7DQo+ID4gKw0KPiA+ICAJaXJxX2Rpc3Bvc2VfbWFwcGluZyhw b3J0LT5pcnEpOw0KPiA+ICB9DQo+ID4gIA0KPiA+ICtzdGF0aWMgdm9pZCBtdGtfcGNpZV9tc2lf aGFuZGxlcihzdHJ1Y3QgbXRrX3BjaWVfcG9ydCAqcG9ydCwgaW50IHNldF9pZHgpDQo+ID4gK3sN Cj4gPiArCXN0cnVjdCBtdGtfbXNpX3NldCAqbXNpX3NldCA9ICZwb3J0LT5tc2lfc2V0c1tzZXRf aWR4XTsNCj4gPiArCXVuc2lnbmVkIGxvbmcgbXNpX2VuYWJsZSwgbXNpX3N0YXR1czsNCj4gPiAr CXVuc2lnbmVkIGludCB2aXJxOw0KPiA+ICsJaXJxX2h3X251bWJlcl90IGJpdCwgaHdpcnE7DQo+ ID4gKw0KPiA+ICsJbXNpX2VuYWJsZSA9IHJlYWRsX3JlbGF4ZWQobXNpX3NldC0+YmFzZSArIFBD SUVfTVNJX1NFVF9FTkFCTEVfT0ZGU0VUKTsNCj4gPiArDQo+ID4gKwlkbyB7DQo+ID4gKwkJbXNp X3N0YXR1cyA9IHJlYWRsX3JlbGF4ZWQobXNpX3NldC0+YmFzZSArDQo+ID4gKwkJCQkJICAgUENJ RV9NU0lfU0VUX1NUQVRVU19PRkZTRVQpOw0KPiA+ICsJCW1zaV9zdGF0dXMgJj0gbXNpX2VuYWJs ZTsNCj4gPiArCQlpZiAoIW1zaV9zdGF0dXMpDQo+ID4gKwkJCWJyZWFrOw0KPiA+ICsNCj4gPiAr CQlmb3JfZWFjaF9zZXRfYml0KGJpdCwgJm1zaV9zdGF0dXMsIFBDSUVfTVNJX0lSUVNfUEVSX1NF VCkgew0KPiA+ICsJCQlod2lycSA9IGJpdCArIHNldF9pZHggKiBQQ0lFX01TSV9JUlFTX1BFUl9T RVQ7DQo+ID4gKwkJCXZpcnEgPSBpcnFfZmluZF9tYXBwaW5nKHBvcnQtPm1zaV9ib3R0b21fZG9t YWluLCBod2lycSk7DQo+ID4gKwkJCWdlbmVyaWNfaGFuZGxlX2lycSh2aXJxKTsNCj4gPiArCQl9 DQo+ID4gKwl9IHdoaWxlICh0cnVlKTsNCj4gPiArfQ0KPiA+ICsNCj4gPiAgc3RhdGljIHZvaWQg bXRrX3BjaWVfaXJxX2hhbmRsZXIoc3RydWN0IGlycV9kZXNjICpkZXNjKQ0KPiA+ICB7DQo+ID4g IAlzdHJ1Y3QgbXRrX3BjaWVfcG9ydCAqcG9ydCA9IGlycV9kZXNjX2dldF9oYW5kbGVyX2RhdGEo ZGVzYyk7DQo+ID4gQEAgLTQwOCw2ICs2NzcsMTQgQEAgc3RhdGljIHZvaWQgbXRrX3BjaWVfaXJx X2hhbmRsZXIoc3RydWN0IGlycV9kZXNjICpkZXNjKQ0KPiA+ICAJCWdlbmVyaWNfaGFuZGxlX2ly cSh2aXJxKTsNCj4gPiAgCX0NCj4gPiAgDQo+ID4gKwlpcnFfYml0ID0gUENJRV9NU0lfU0hJRlQ7 DQo+ID4gKwlmb3JfZWFjaF9zZXRfYml0X2Zyb20oaXJxX2JpdCwgJnN0YXR1cywgUENJRV9NU0lf U0VUX05VTSArDQo+ID4gKwkJCSAgICAgIFBDSUVfTVNJX1NISUZUKSB7DQo+ID4gKwkJbXRrX3Bj aWVfbXNpX2hhbmRsZXIocG9ydCwgaXJxX2JpdCAtIFBDSUVfTVNJX1NISUZUKTsNCj4gPiArDQo+ ID4gKwkJd3JpdGVsX3JlbGF4ZWQoQklUKGlycV9iaXQpLCBwb3J0LT5iYXNlICsgUENJRV9JTlRf U1RBVFVTX1JFRyk7DQo+IA0KPiBJc24ndCB0aGlzIHdyaXRlIHRoZSBzYW1lIHRoaW5nIHlvdSBo YXZlIGZvciBFT0kgaW4gdGhlIElOVHggY2FzZT8NCj4gV2hpbGUgSSBjb3VsZCB1bmRlcnN0YW5k IHlvdXIgZGVzY3JpcHRpb24gaW4gdGhhdCBjYXNlICh0aGlzIGlzIGENCj4gcmVzYW1wbGluZyBv cGVyYXRpb24pLCBJIGRvbid0IGdldCB3aGF0IHRoaXMgZG9lcyBoZXJlLiBFaXRoZXIgdGhpcyBp cw0KPiBhbHNvIGFuIEVPSSwgYnV0IHlvdXIgaW5pdGlhbCBkZXNjcmlwdGlvbiBkb2Vzbid0IG1h a2Ugc2Vuc2UsIG9yIGl0IGlzDQo+IGFuIEFjaywgYW5kIGl0IHNob3VsZCBiZSBtb3ZlZCB0byB0 aGUgcmlnaHQgcGxhY2UuDQo+IA0KPiBXaGljaCBvbmUgaXMgaXQ/DQoNCkkgdGhpbmsgaXQgc2hv dWxkIGJlIGFuIEVPSSB3aGljaCB1c2VkIHRvIGNsZWFyIHRoZSBpbnRlcnJ1cHQgc3RhdHVzIG9m DQphIHNpbmdsZSBzZXQgaW4gdGhlIFBDSWUgaW50YyBmaWVsZCwgbWF5YmUgSSBzaG91bGQgbW92 ZSBpdCB0byB0aGUgZW5kDQpvZiB0aGUgbXRrX3BjaWVfbXNpX2hhbmRsZXIoKSBmdW5jdGlvbi4N Cg0KICAgICAgICAgICAgICAgICAgKy0tLS0tKw0KICAgICAgICAgICAgICAgICAgfCBHSUMgfA0K ICAgICAgICAgICAgICAgICAgKy0tLS0tKw0KICAgICAgICAgICAgICAgICAgICAgXg0KICAgICAg ICAgICAgICAgICAgICAgfA0KICAgICAgICAgICAgICAgICBwb3J0LT5pcnENCiAgICAgICAgICAg ICAgICAgICAgIHwNCiAgICAgICAgICAgICArLSstKy0rLSstKy0rLSstKw0KICAgICAgICAgICAg IHwwfDF8MnwzfDR8NXw2fDd8IChQQ0llIGludGMpDQogICAgICAgICAgICAgKy0rLSstKy0rLSst Ky0rLSsNCiAgICAgICAgICAgICAgXiBeICAgICAgICAgICBeDQogICAgICAgICAgICAgIHwgfCAg ICAuLi4gICAgfA0KICAgICAgKy0tLS0tLS0rICstLS0tLS0rICAgICstLS0tLS0tLS0tLSsNCiAg ICAgIHwgICAgICAgICAgICAgICAgfCAgICAgICAgICAgICAgICB8DQorLSstKy0tLSstLSstLSsg ICstKy0rLS0tKy0tKy0tKyAgKy0rLSstLS0rLS0rLS0rDQp8MHwxfC4uLnwzMHwzMXwgIHwwfDF8 Li4ufDMwfDMxfCAgfDB8MXwuLi58MzB8MzF8IChNU0kgc2V0cykNCistKy0rLS0tKy0tKy0tKyAg Ky0rLSstLS0rLS0rLS0rICArLSstKy0tLSstLSstLSsNCiBeIF4gICAgICBeICBeICAgIF4gXiAg ICAgIF4gIF4gICAgXiBeICAgICAgXiAgXg0KIHwgfCAgICAgIHwgIHwgICAgfCB8ICAgICAgfCAg fCAgICB8IHwgICAgICB8ICB8ICAoTVNJIHZlY3RvcnMpDQogfCB8ICAgICAgfCAgfCAgICB8IHwg ICAgICB8ICB8ICAgIHwgfCAgICAgIHwgIHwNCg0KICAoTVNJIFNFVDApICAgICAgIChNU0kgU0VU MSkgIC4uLiAgIChNU0kgU0VUNykNCg0KSSB3b3VsZCBsaWtlIHRvIGFzayBhbm90aGVyIHF1ZXN0 aW9uLiBJbiB0aGlzIGludGVycnVwdCBhcmNoaXRlY3R1cmUsIHdlDQpjYW5ub3QgaW1wbGVtZW50 IGFuIGFmZmluaXR5IGZvciBQQ0llIGludGVycnVwdHMsIHNvIHdlIHJldHVybiBhDQpuZWdhdGl2 ZSB2YWx1ZSBpbiB0aGUgbXRrX3BjaWVfc2V0X2FmZmluaXR5IGNhbGxiYWNrIGFzIGZvbGxvd3M6 IA0KDQorc3RhdGljIGludCBtdGtfcGNpZV9zZXRfYWZmaW5pdHkoc3RydWN0IGlycV9kYXRhICpk YXRhLA0KKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uc3Qgc3RydWN0IGNwdW1h c2sgKm1hc2ssIGJvb2wgZm9yY2UpDQorew0KKyAgICAgICByZXR1cm4gLUVJTlZBTDsNCit9DQoN CkJ1dCB0aGVyZSB3aWxsIGFsd2F5cyBiZSBlcnJvciBsb2dzIHdoZW4gaG90cGx1ZyBhIENQVToN Cg0KfiAjIGVjaG8gMCA+IC9zeXMvZGV2aWNlcy9zeXN0ZW0vY3B1L2NwdTEvb25saW5lDQpbICAg OTMuNjMzMDU5XSBJUlEyNTU6IHNldCBhZmZpbml0eSBmYWlsZWQoLTIyKS4NClsgICA5My42MzM2 MjRdIElSUTI1Njogc2V0IGFmZmluaXR5IGZhaWxlZCgtMjIpLg0KWyAgIDkzLjYzNDIyMl0gQ1BV MTogc2h1dGRvd24NClsgICA5My42MzQ1ODZdIHBzY2k6IENQVTEga2lsbGVkIChwb2xsZWQgMCBt cykNCg0KT3Igd2hlbiB0aGUgc3lzdGVtIHN1c3BlbmRzOg0KDQp+ICMgZWNobyBtZW0gPiAvc3lz L3Bvd2VyL3N0YXRlDQpbICAgOTMuNjM1MTQ1XSBjcHVocDogY3B1X29mZiBjbHVzdGVyPTAsIGNw dT0xDQpbICAxNjkuODM1NjUzXSBQTTogc3VzcGVuZCBlbnRyeSAoZGVlcCkNClsgIDE2OS44MzY3 MTddIEZpbGVzeXN0ZW1zIHN5bmM6IDAuMDAwIHNlY29uZHMNClsgIDE2OS44Mzc5MjRdIEZyZWV6 aW5nIHVzZXIgc3BhY2UgcHJvY2Vzc2VzIC4uLiAoZWxhcHNlZCAwLjAwMSBzZWNvbmRzKQ0KZG9u ZS4NClsgIDE2OS44Mzk5MjJdIE9PTSBraWxsZXIgZGlzYWJsZWQuDQpbICAxNjkuODQwMzM2XSBG cmVlemluZyByZW1haW5pbmcgZnJlZXphYmxlIHRhc2tzIC4uLiAoZWxhcHNlZCAwLjAwMQ0Kc2Vj b25kcykgZG9uZS4NClsgIDE2OS44NDQ3MTVdIERpc2FibGluZyBub24tYm9vdCBDUFVzIC4uLg0K WyAgMTY5Ljg0NjQ0M10gSVJRMjU1OiBzZXQgYWZmaW5pdHkgZmFpbGVkKC0yMikuDQpbICAxNjku ODQ3MDAyXSBJUlEyNTY6IHNldCBhZmZpbml0eSBmYWlsZWQoLTIyKS4NClsgIDE2OS44NDc1ODZd IENQVTI6IHNodXRkb3duDQpbICAxNjkuODQ3OTQzXSBwc2NpOiBDUFUyIGtpbGxlZCAocG9sbGVk IDAgbXMpDQpbICAxNjkuODQ4NDg5XSBjcHVocDogY3B1X29mZiBjbHVzdGVyPTAsIGNwdT0yDQpb ICAxNjkuODUwMjg1XSBJUlEyNTU6IHNldCBhZmZpbml0eSBmYWlsZWQoLTIyKS4NClsgIDE2OS44 NTEzNjldIElSUTI1Njogc2V0IGFmZmluaXR5IGZhaWxlZCgtMjIpLg0KLi4uDQoNClNvbWV0aW1l cyB0aGlzIGNhbiBjYXVzZSBtaXN1bmRlcnN0YW5kaW5ncyB0byB1c2VycywgZG8gd2UgaGF2ZSBh IGNoYW5jZQ0KdG8gcHJldmVudCB0aGlzIGVycm9yIGxvZz8NCg0KPiANCj4gVGhhbmtzLA0KPiAN Cj4gCU0uDQo+IA0KDQpUaGFua3MuDQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63EFEC433E0 for ; Wed, 10 Mar 2021 06:59:41 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C11E164FF6 for ; Wed, 10 Mar 2021 06:59:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C11E164FF6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Date:CC:To:From: Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=u53YYGyYSVuJ0zLsXCUphADJt0H2CNGuX7CNHXU83oo=; b=Rq+LBsR3WfIyj1Jj9pUZTKBgY tff6X4b2clNxdPG4dnBeUqRqo3DFdruJe9DUPLVKccBrCxOt8Pkw2ZeR3ULTLsdpIO8DlXSYlhjDH NR7c/gvFDqCd2QpEtNi6oWfrUBmascAtEZ1TA6Ql1BG2G49vW2Cr91neNTPl7QWt0zP2J/cLBb1Td /deOrBjgFMVu/BDD9bIx66MLsKCyiqlUYXzNysUAPRjlH6FPCUB8T261G/QNLIFuHYoASb/KZ6SY9 qMmfEBw4I1UXW0b7i/Qz0a+6K6kMQCe5Qk7tlUMdlv2d6MsGcAJQiqiTfsYq3bWv8jq++KTjW+e5G v1GXc8Yng==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lJsoX-006AQy-C3; Wed, 10 Mar 2021 06:59:25 +0000 Received: from mailgw01.mediatek.com ([216.200.240.184]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lJsoF-006AP5-VO; Wed, 10 Mar 2021 06:59:19 +0000 X-UUID: 52d9b93c075f42c4a029153a52a8687f-20210309 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=D09TyNylBKbStaKtchQcUElsm48BUYZhG4qREnS+nW8=; b=iNSLqXwKCGEvjDb4TpnaEt0ZEOldsuTr1Up6CViLIgoKTf26+FVek+pbSiXUkyubvK1b3dnpWztPXmYus7GOnGmdL6eqmkXMTqyQugdx6ndutXj9hTaBRWZw98E7Jnqy99uqfgkOJ2zKYQgjFUt0wYesSRf94esS7i69NXnS3k8=; X-UUID: 52d9b93c075f42c4a029153a52a8687f-20210309 Received: from mtkcas66.mediatek.inc [(172.29.193.44)] by mailgw01.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 678790565; Tue, 09 Mar 2021 22:59:02 -0800 Received: from MTKMBS31N1.mediatek.inc (172.27.4.69) by MTKMBS62N1.mediatek.inc (172.29.193.41) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 22:49:00 -0800 Received: from MTKCAS36.mediatek.inc (172.27.4.186) by MTKMBS31N1.mediatek.inc (172.27.4.69) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 10 Mar 2021 14:48:49 +0800 Received: from [10.17.3.153] (10.17.3.153) by MTKCAS36.mediatek.inc (172.27.4.170) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 10 Mar 2021 14:48:49 +0800 Message-ID: <1615358929.25662.47.camel@mhfsdcap03> Subject: Re: [v8,5/7] PCI: mediatek-gen3: Add MSI support From: Jianjun Wang To: Marc Zyngier CC: Bjorn Helgaas , Rob Herring , Lorenzo Pieralisi , Ryder Lee , Philipp Zabel , "Matthias Brugger" , , , , , , "Sj Huang" , , , , , , , Date: Wed, 10 Mar 2021 14:48:49 +0800 In-Reply-To: <87pn08y3ja.wl-maz@kernel.org> References: <20210224061132.26526-1-jianjun.wang@mediatek.com> <20210224061132.26526-6-jianjun.wang@mediatek.com> <87pn08y3ja.wl-maz@kernel.org> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-TM-SNTS-SMTP: 9BAB50CC64708D9103176E08435B1B5ED04930DF72E0FB2407164EEC1CDFC2732000:8 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210310_065916_285125_0F89FD2B X-CRM114-Status: GOOD ( 36.89 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org Hi Marc, Thanks for your review. On Tue, 2021-03-09 at 11:23 +0000, Marc Zyngier wrote: > On Wed, 24 Feb 2021 06:11:30 +0000, > Jianjun Wang wrote: > > > > Add MSI support for MediaTek Gen3 PCIe controller. > > > > This PCIe controller supports up to 256 MSI vectors, the MSI hardware > > block diagram is as follows: > > > > +-----+ > > | GIC | > > +-----+ > > ^ > > | > > port->irq > > | > > +-+-+-+-+-+-+-+-+ > > |0|1|2|3|4|5|6|7| (PCIe intc) > > +-+-+-+-+-+-+-+-+ > > ^ ^ ^ > > | | ... | > > +-------+ +------+ +-----------+ > > | | | > > +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ > > |0|1|...|30|31| |0|1|...|30|31| |0|1|...|30|31| (MSI sets) > > +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ > > ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ > > | | | | | | | | | | | | (MSI vectors) > > | | | | | | | | | | | | > > > > (MSI SET0) (MSI SET1) ... (MSI SET7) > > > > With 256 MSI vectors supported, the MSI vectors are composed of 8 sets, > > each set has its own address for MSI message, and supports 32 MSI vectors > > to generate interrupt. > > > > Signed-off-by: Jianjun Wang > > Acked-by: Ryder Lee > > --- > > drivers/pci/controller/pcie-mediatek-gen3.c | 277 ++++++++++++++++++++ > > 1 file changed, 277 insertions(+) > > > > diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c > > index 8b3b5f838b69..3cbec22ece0c 100644 > > --- a/drivers/pci/controller/pcie-mediatek-gen3.c > > +++ b/drivers/pci/controller/pcie-mediatek-gen3.c > > @@ -14,6 +14,7 @@ > > #include > > #include > > #include > > +#include > > #include > > #include > > #include > > @@ -48,12 +49,29 @@ > > #define PCIE_LINK_STATUS_REG 0x154 > > #define PCIE_PORT_LINKUP BIT(8) > > > > +#define PCIE_MSI_SET_NUM 8 > > +#define PCIE_MSI_IRQS_PER_SET 32 > > +#define PCIE_MSI_IRQS_NUM \ > > + (PCIE_MSI_IRQS_PER_SET * PCIE_MSI_SET_NUM) > > + > > #define PCIE_INT_ENABLE_REG 0x180 > > +#define PCIE_MSI_ENABLE GENMASK(PCIE_MSI_SET_NUM + 8 - 1, 8) > > +#define PCIE_MSI_SHIFT 8 > > #define PCIE_INTX_SHIFT 24 > > #define PCIE_INTX_ENABLE \ > > GENMASK(PCIE_INTX_SHIFT + PCI_NUM_INTX - 1, PCIE_INTX_SHIFT) > > > > #define PCIE_INT_STATUS_REG 0x184 > > +#define PCIE_MSI_SET_ENABLE_REG 0x190 > > +#define PCIE_MSI_SET_ENABLE GENMASK(PCIE_MSI_SET_NUM - 1, 0) > > + > > +#define PCIE_MSI_SET_BASE_REG 0xc00 > > +#define PCIE_MSI_SET_OFFSET 0x10 > > +#define PCIE_MSI_SET_STATUS_OFFSET 0x04 > > +#define PCIE_MSI_SET_ENABLE_OFFSET 0x08 > > + > > +#define PCIE_MSI_SET_ADDR_HI_BASE 0xc80 > > +#define PCIE_MSI_SET_ADDR_HI_OFFSET 0x04 > > > > #define PCIE_TRANS_TABLE_BASE_REG 0x800 > > #define PCIE_ATR_SRC_ADDR_MSB_OFFSET 0x4 > > @@ -73,6 +91,16 @@ > > #define PCIE_ATR_TLP_TYPE_MEM PCIE_ATR_TLP_TYPE(0) > > #define PCIE_ATR_TLP_TYPE_IO PCIE_ATR_TLP_TYPE(2) > > > > +/** > > + * struct mtk_pcie_msi - MSI information for each set > > + * @base: IO mapped register base > > + * @msg_addr: MSI message address > > + */ > > +struct mtk_msi_set { > > + void __iomem *base; > > + phys_addr_t msg_addr; > > +}; > > + > > /** > > * struct mtk_pcie_port - PCIe port information > > * @dev: pointer to PCIe device > > @@ -86,6 +114,11 @@ > > * @irq: PCIe controller interrupt number > > * @irq_lock: lock protecting IRQ register access > > * @intx_domain: legacy INTx IRQ domain > > + * @msi_domain: MSI IRQ domain > > + * @msi_bottom_domain: MSI IRQ bottom domain > > + * @msi_sets: MSI sets information > > + * @lock: lock protecting IRQ bit map > > + * @msi_irq_in_use: bit map for assigned MSI IRQ > > */ > > struct mtk_pcie_port { > > struct device *dev; > > @@ -100,6 +133,11 @@ struct mtk_pcie_port { > > int irq; > > raw_spinlock_t irq_lock; > > struct irq_domain *intx_domain; > > + struct irq_domain *msi_domain; > > + struct irq_domain *msi_bottom_domain; > > + struct mtk_msi_set msi_sets[PCIE_MSI_SET_NUM]; > > + struct mutex lock; > > + DECLARE_BITMAP(msi_irq_in_use, PCIE_MSI_IRQS_NUM); > > }; > > > > /** > > @@ -197,6 +235,35 @@ static int mtk_pcie_set_trans_table(struct mtk_pcie_port *port, > > return 0; > > } > > > > +static void mtk_pcie_enable_msi(struct mtk_pcie_port *port) > > +{ > > + int i; > > + u32 val; > > + > > + val = readl_relaxed(port->base + PCIE_MSI_SET_ENABLE_REG); > > + val |= PCIE_MSI_SET_ENABLE; > > + writel_relaxed(val, port->base + PCIE_MSI_SET_ENABLE_REG); > > + > > + val = readl_relaxed(port->base + PCIE_INT_ENABLE_REG); > > + val |= PCIE_MSI_ENABLE; > > + writel_relaxed(val, port->base + PCIE_INT_ENABLE_REG); > > Shouldn't you configure the capture addresses *before* enabling > things? Is there any need for locking here, given that you are > modifying global registers? Yes, I will move these codes to the back of the configure capture address in the next version. I think the lock may not be needed because this function is only executed once when driver probe. > > > + > > + for (i = 0; i < PCIE_MSI_SET_NUM; i++) { > > + struct mtk_msi_set *msi_set = &port->msi_sets[i]; > > + > > + msi_set->base = port->base + PCIE_MSI_SET_BASE_REG + > > + i * PCIE_MSI_SET_OFFSET; > > + msi_set->msg_addr = port->reg_base + PCIE_MSI_SET_BASE_REG + > > + i * PCIE_MSI_SET_OFFSET; > > + > > + /* Configure the MSI capture address */ > > + writel_relaxed(lower_32_bits(msi_set->msg_addr), msi_set->base); > > + writel_relaxed(upper_32_bits(msi_set->msg_addr), > > + port->base + PCIE_MSI_SET_ADDR_HI_BASE + > > + i * PCIE_MSI_SET_ADDR_HI_OFFSET); > > + } > > +} > > + > > static int mtk_pcie_startup_port(struct mtk_pcie_port *port) > > { > > struct resource_entry *entry; > > @@ -247,6 +314,8 @@ static int mtk_pcie_startup_port(struct mtk_pcie_port *port) > > return err; > > } > > > > + mtk_pcie_enable_msi(port); > > + > > /* Set PCIe translation windows */ > > resource_list_for_each_entry(entry, &host->windows) { > > struct resource *res = entry->res; > > @@ -290,6 +359,148 @@ static int mtk_pcie_set_affinity(struct irq_data *data, > > return -EINVAL; > > } > > > > +static void mtk_pcie_irq_mask(struct irq_data *data) > > It'd be good if you used _msi_ in function names that deal with MSIs. OK, I will fix it in the next version. > > > +{ > > + pci_msi_mask_irq(data); > > + irq_chip_mask_parent(data); > > +} > > + > > +static void mtk_pcie_irq_unmask(struct irq_data *data) > > +{ > > + pci_msi_unmask_irq(data); > > + irq_chip_unmask_parent(data); > > +} > > + > > +static struct irq_chip mtk_msi_irq_chip = { > > + .name = "MSI", > > + .irq_enable = mtk_pcie_irq_unmask, > > + .irq_disable = mtk_pcie_irq_mask, > > Same comment as for the previous patch: enable/disable serve no > purpose here. Replied in the previous patch, the enable/disable callback is used when the system suspend/resume. > > > + .irq_ack = irq_chip_ack_parent, > > + .irq_mask = mtk_pcie_irq_mask, > > + .irq_unmask = mtk_pcie_irq_unmask, > > +}; > > + > > +static struct msi_domain_info mtk_msi_domain_info = { > > + .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_PCI_MSIX | > > + MSI_FLAG_USE_DEF_CHIP_OPS | MSI_FLAG_MULTI_PCI_MSI), > > minor nit: keep the *_OPS flag on one line, and the *_PCI_* flags on > another. Sure, I will fix it in the next version. > > > + .chip = &mtk_msi_irq_chip, > > +}; > > + > > +static void mtk_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) > > +{ > > + struct mtk_msi_set *msi_set = irq_data_get_irq_chip_data(data); > > + struct mtk_pcie_port *port = data->domain->host_data; > > + unsigned long hwirq; > > + > > + hwirq = data->hwirq % PCIE_MSI_IRQS_PER_SET; > > + > > + msg->address_hi = upper_32_bits(msi_set->msg_addr); > > + msg->address_lo = lower_32_bits(msi_set->msg_addr); > > + msg->data = hwirq; > > + dev_dbg(port->dev, "msi#%#lx address_hi %#x address_lo %#x data %d\n", > > + hwirq, msg->address_hi, msg->address_lo, msg->data); > > +} > > + > > +static void mtk_msi_bottom_irq_ack(struct irq_data *data) > > +{ > > + struct mtk_msi_set *msi_set = irq_data_get_irq_chip_data(data); > > + unsigned long hwirq; > > + > > + hwirq = data->hwirq % PCIE_MSI_IRQS_PER_SET; > > + > > + writel_relaxed(BIT(hwirq), msi_set->base + PCIE_MSI_SET_STATUS_OFFSET); > > +} > > + > > +static void mtk_msi_bottom_irq_mask(struct irq_data *data) > > +{ > > + struct mtk_msi_set *msi_set = irq_data_get_irq_chip_data(data); > > + struct mtk_pcie_port *port = data->domain->host_data; > > + unsigned long hwirq, flags; > > + u32 val; > > + > > + hwirq = data->hwirq % PCIE_MSI_IRQS_PER_SET; > > + > > + raw_spin_lock_irqsave(&port->irq_lock, flags); > > + val = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + val &= ~BIT(hwirq); > > + writel_relaxed(val, msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + raw_spin_unlock_irqrestore(&port->irq_lock, flags); > > +} > > + > > +static void mtk_msi_bottom_irq_unmask(struct irq_data *data) > > +{ > > + struct mtk_msi_set *msi_set = irq_data_get_irq_chip_data(data); > > + struct mtk_pcie_port *port = data->domain->host_data; > > + unsigned long hwirq, flags; > > + u32 val; > > + > > + hwirq = data->hwirq % PCIE_MSI_IRQS_PER_SET; > > + > > + raw_spin_lock_irqsave(&port->irq_lock, flags); > > + val = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + val |= BIT(hwirq); > > + writel_relaxed(val, msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + raw_spin_unlock_irqrestore(&port->irq_lock, flags); > > +} > > + > > +static struct irq_chip mtk_msi_bottom_irq_chip = { > > + .irq_ack = mtk_msi_bottom_irq_ack, > > + .irq_mask = mtk_msi_bottom_irq_mask, > > + .irq_unmask = mtk_msi_bottom_irq_unmask, > > + .irq_compose_msi_msg = mtk_compose_msi_msg, > > + .irq_set_affinity = mtk_pcie_set_affinity, > > + .name = "MSI", > > +}; > > + > > +static int mtk_msi_bottom_domain_alloc(struct irq_domain *domain, > > + unsigned int virq, unsigned int nr_irqs, > > + void *arg) > > +{ > > + struct mtk_pcie_port *port = domain->host_data; > > + struct mtk_msi_set *msi_set; > > + int i, hwirq, set_idx; > > + > > + mutex_lock(&port->lock); > > + > > + hwirq = bitmap_find_free_region(port->msi_irq_in_use, PCIE_MSI_IRQS_NUM, > > + order_base_2(nr_irqs)); > > + > > + mutex_unlock(&port->lock); > > + > > + if (hwirq < 0) > > + return -ENOSPC; > > + > > + set_idx = hwirq / PCIE_MSI_IRQS_PER_SET; > > + msi_set = &port->msi_sets[set_idx]; > > + > > + for (i = 0; i < nr_irqs; i++) > > + irq_domain_set_info(domain, virq + i, hwirq + i, > > + &mtk_msi_bottom_irq_chip, msi_set, > > + handle_edge_irq, NULL, NULL); > > + > > + return 0; > > +} > > + > > +static void mtk_msi_bottom_domain_free(struct irq_domain *domain, > > + unsigned int virq, unsigned int nr_irqs) > > +{ > > + struct mtk_pcie_port *port = domain->host_data; > > + struct irq_data *data = irq_domain_get_irq_data(domain, virq); > > + > > + mutex_lock(&port->lock); > > + > > + bitmap_clear(port->msi_irq_in_use, data->hwirq, nr_irqs); > > + > > + mutex_unlock(&port->lock); > > + > > + irq_domain_free_irqs_common(domain, virq, nr_irqs); > > +} > > + > > +static const struct irq_domain_ops mtk_msi_bottom_domain_ops = { > > + .alloc = mtk_msi_bottom_domain_alloc, > > + .free = mtk_msi_bottom_domain_free, > > +}; > > + > > static void mtk_intx_mask(struct irq_data *data) > > { > > struct mtk_pcie_port *port = irq_data_get_irq_chip_data(data); > > @@ -360,6 +571,7 @@ static int mtk_pcie_init_irq_domains(struct mtk_pcie_port *port) > > { > > struct device *dev = port->dev; > > struct device_node *intc_node, *node = dev->of_node; > > + int ret; > > > > raw_spin_lock_init(&port->irq_lock); > > > > @@ -377,7 +589,34 @@ static int mtk_pcie_init_irq_domains(struct mtk_pcie_port *port) > > return -ENODEV; > > } > > > > + /* Setup MSI */ > > + mutex_init(&port->lock); > > + > > + port->msi_bottom_domain = irq_domain_add_linear(node, PCIE_MSI_IRQS_NUM, > > + &mtk_msi_bottom_domain_ops, port); > > + if (!port->msi_bottom_domain) { > > + dev_info(dev, "failed to create MSI bottom domain\n"); > > + ret = -ENODEV; > > + goto err_msi_bottom_domain; > > + } > > + > > + port->msi_domain = pci_msi_create_irq_domain(dev->fwnode, > > + &mtk_msi_domain_info, > > + port->msi_bottom_domain); > > + if (!port->msi_domain) { > > + dev_info(dev, "failed to create MSI domain\n"); > > + ret = -ENODEV; > > + goto err_msi_domain; > > + } > > + > > return 0; > > + > > +err_msi_domain: > > + irq_domain_remove(port->msi_bottom_domain); > > +err_msi_bottom_domain: > > + irq_domain_remove(port->intx_domain); > > + > > + return ret; > > } > > > > static void mtk_pcie_irq_teardown(struct mtk_pcie_port *port) > > @@ -387,9 +626,39 @@ static void mtk_pcie_irq_teardown(struct mtk_pcie_port *port) > > if (port->intx_domain) > > irq_domain_remove(port->intx_domain); > > > > + if (port->msi_domain) > > + irq_domain_remove(port->msi_domain); > > + > > + if (port->msi_bottom_domain) > > + irq_domain_remove(port->msi_bottom_domain); > > + > > irq_dispose_mapping(port->irq); > > } > > > > +static void mtk_pcie_msi_handler(struct mtk_pcie_port *port, int set_idx) > > +{ > > + struct mtk_msi_set *msi_set = &port->msi_sets[set_idx]; > > + unsigned long msi_enable, msi_status; > > + unsigned int virq; > > + irq_hw_number_t bit, hwirq; > > + > > + msi_enable = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + > > + do { > > + msi_status = readl_relaxed(msi_set->base + > > + PCIE_MSI_SET_STATUS_OFFSET); > > + msi_status &= msi_enable; > > + if (!msi_status) > > + break; > > + > > + for_each_set_bit(bit, &msi_status, PCIE_MSI_IRQS_PER_SET) { > > + hwirq = bit + set_idx * PCIE_MSI_IRQS_PER_SET; > > + virq = irq_find_mapping(port->msi_bottom_domain, hwirq); > > + generic_handle_irq(virq); > > + } > > + } while (true); > > +} > > + > > static void mtk_pcie_irq_handler(struct irq_desc *desc) > > { > > struct mtk_pcie_port *port = irq_desc_get_handler_data(desc); > > @@ -408,6 +677,14 @@ static void mtk_pcie_irq_handler(struct irq_desc *desc) > > generic_handle_irq(virq); > > } > > > > + irq_bit = PCIE_MSI_SHIFT; > > + for_each_set_bit_from(irq_bit, &status, PCIE_MSI_SET_NUM + > > + PCIE_MSI_SHIFT) { > > + mtk_pcie_msi_handler(port, irq_bit - PCIE_MSI_SHIFT); > > + > > + writel_relaxed(BIT(irq_bit), port->base + PCIE_INT_STATUS_REG); > > Isn't this write the same thing you have for EOI in the INTx case? > While I could understand your description in that case (this is a > resampling operation), I don't get what this does here. Either this is > also an EOI, but your initial description doesn't make sense, or it is > an Ack, and it should be moved to the right place. > > Which one is it? I think it should be an EOI which used to clear the interrupt status of a single set in the PCIe intc field, maybe I should move it to the end of the mtk_pcie_msi_handler() function. +-----+ | GIC | +-----+ ^ | port->irq | +-+-+-+-+-+-+-+-+ |0|1|2|3|4|5|6|7| (PCIe intc) +-+-+-+-+-+-+-+-+ ^ ^ ^ | | ... | +-------+ +------+ +-----------+ | | | +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ |0|1|...|30|31| |0|1|...|30|31| |0|1|...|30|31| (MSI sets) +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ | | | | | | | | | | | | (MSI vectors) | | | | | | | | | | | | (MSI SET0) (MSI SET1) ... (MSI SET7) I would like to ask another question. In this interrupt architecture, we cannot implement an affinity for PCIe interrupts, so we return a negative value in the mtk_pcie_set_affinity callback as follows: +static int mtk_pcie_set_affinity(struct irq_data *data, + const struct cpumask *mask, bool force) +{ + return -EINVAL; +} But there will always be error logs when hotplug a CPU: ~ # echo 0 > /sys/devices/system/cpu/cpu1/online [ 93.633059] IRQ255: set affinity failed(-22). [ 93.633624] IRQ256: set affinity failed(-22). [ 93.634222] CPU1: shutdown [ 93.634586] psci: CPU1 killed (polled 0 ms) Or when the system suspends: ~ # echo mem > /sys/power/state [ 93.635145] cpuhp: cpu_off cluster=0, cpu=1 [ 169.835653] PM: suspend entry (deep) [ 169.836717] Filesystems sync: 0.000 seconds [ 169.837924] Freezing user space processes ... (elapsed 0.001 seconds) done. [ 169.839922] OOM killer disabled. [ 169.840336] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. [ 169.844715] Disabling non-boot CPUs ... [ 169.846443] IRQ255: set affinity failed(-22). [ 169.847002] IRQ256: set affinity failed(-22). [ 169.847586] CPU2: shutdown [ 169.847943] psci: CPU2 killed (polled 0 ms) [ 169.848489] cpuhp: cpu_off cluster=0, cpu=2 [ 169.850285] IRQ255: set affinity failed(-22). [ 169.851369] IRQ256: set affinity failed(-22). ... Sometimes this can cause misunderstandings to users, do we have a chance to prevent this error log? > > Thanks, > > M. > Thanks. _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CB17C433E0 for ; Wed, 10 Mar 2021 07:01:17 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BD30964FE8 for ; Wed, 10 Mar 2021 07:01:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD30964FE8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Date:CC:To:From: Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=FoU7ud4FUd2Sfz/HhGf3lY2ZO6zU8Qjp2KgyaoZG0Fk=; b=hzJB9lOlFUYIB3vv0KoXX12QK jOsCe7r4Nymxo5bQfD+m7DQZLlzo5zTmbT496+So+F9ZrTFLtau/aL3IOfbi4EyIA6mdcYvE2jPUD Ctjh5Jx04a+tgLrmK86Gu/ZQTL9fIuFKbH8+5oWPS+XXBh3gHRc7e30EGlVThdwNOFjLtp6lj37gk 52SM0gXHR+DlE/9JmNVqpL600Q0jbTHUOl4MO+p7vgEmbM3GCwn0zQWrcRqf7iCS05Dby6K5seKH2 2kvrKZMiH7Mc/oOuRXenUSTXM7TfA5qZ5x+CgRd972GD+zplPvwBLxr057pRzIP8guHzXfvWBk3qd qo5xknJQw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lJsoZ-006ARV-PM; Wed, 10 Mar 2021 06:59:27 +0000 Received: from mailgw01.mediatek.com ([216.200.240.184]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lJsoF-006AP5-VO; Wed, 10 Mar 2021 06:59:19 +0000 X-UUID: 52d9b93c075f42c4a029153a52a8687f-20210309 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=D09TyNylBKbStaKtchQcUElsm48BUYZhG4qREnS+nW8=; b=iNSLqXwKCGEvjDb4TpnaEt0ZEOldsuTr1Up6CViLIgoKTf26+FVek+pbSiXUkyubvK1b3dnpWztPXmYus7GOnGmdL6eqmkXMTqyQugdx6ndutXj9hTaBRWZw98E7Jnqy99uqfgkOJ2zKYQgjFUt0wYesSRf94esS7i69NXnS3k8=; X-UUID: 52d9b93c075f42c4a029153a52a8687f-20210309 Received: from mtkcas66.mediatek.inc [(172.29.193.44)] by mailgw01.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 678790565; Tue, 09 Mar 2021 22:59:02 -0800 Received: from MTKMBS31N1.mediatek.inc (172.27.4.69) by MTKMBS62N1.mediatek.inc (172.29.193.41) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 22:49:00 -0800 Received: from MTKCAS36.mediatek.inc (172.27.4.186) by MTKMBS31N1.mediatek.inc (172.27.4.69) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 10 Mar 2021 14:48:49 +0800 Received: from [10.17.3.153] (10.17.3.153) by MTKCAS36.mediatek.inc (172.27.4.170) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 10 Mar 2021 14:48:49 +0800 Message-ID: <1615358929.25662.47.camel@mhfsdcap03> Subject: Re: [v8,5/7] PCI: mediatek-gen3: Add MSI support From: Jianjun Wang To: Marc Zyngier CC: Bjorn Helgaas , Rob Herring , Lorenzo Pieralisi , Ryder Lee , Philipp Zabel , "Matthias Brugger" , , , , , , "Sj Huang" , , , , , , , Date: Wed, 10 Mar 2021 14:48:49 +0800 In-Reply-To: <87pn08y3ja.wl-maz@kernel.org> References: <20210224061132.26526-1-jianjun.wang@mediatek.com> <20210224061132.26526-6-jianjun.wang@mediatek.com> <87pn08y3ja.wl-maz@kernel.org> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-TM-SNTS-SMTP: 9BAB50CC64708D9103176E08435B1B5ED04930DF72E0FB2407164EEC1CDFC2732000:8 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210310_065916_285125_0F89FD2B X-CRM114-Status: GOOD ( 36.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Marc, Thanks for your review. On Tue, 2021-03-09 at 11:23 +0000, Marc Zyngier wrote: > On Wed, 24 Feb 2021 06:11:30 +0000, > Jianjun Wang wrote: > > > > Add MSI support for MediaTek Gen3 PCIe controller. > > > > This PCIe controller supports up to 256 MSI vectors, the MSI hardware > > block diagram is as follows: > > > > +-----+ > > | GIC | > > +-----+ > > ^ > > | > > port->irq > > | > > +-+-+-+-+-+-+-+-+ > > |0|1|2|3|4|5|6|7| (PCIe intc) > > +-+-+-+-+-+-+-+-+ > > ^ ^ ^ > > | | ... | > > +-------+ +------+ +-----------+ > > | | | > > +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ > > |0|1|...|30|31| |0|1|...|30|31| |0|1|...|30|31| (MSI sets) > > +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ > > ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ > > | | | | | | | | | | | | (MSI vectors) > > | | | | | | | | | | | | > > > > (MSI SET0) (MSI SET1) ... (MSI SET7) > > > > With 256 MSI vectors supported, the MSI vectors are composed of 8 sets, > > each set has its own address for MSI message, and supports 32 MSI vectors > > to generate interrupt. > > > > Signed-off-by: Jianjun Wang > > Acked-by: Ryder Lee > > --- > > drivers/pci/controller/pcie-mediatek-gen3.c | 277 ++++++++++++++++++++ > > 1 file changed, 277 insertions(+) > > > > diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c > > index 8b3b5f838b69..3cbec22ece0c 100644 > > --- a/drivers/pci/controller/pcie-mediatek-gen3.c > > +++ b/drivers/pci/controller/pcie-mediatek-gen3.c > > @@ -14,6 +14,7 @@ > > #include > > #include > > #include > > +#include > > #include > > #include > > #include > > @@ -48,12 +49,29 @@ > > #define PCIE_LINK_STATUS_REG 0x154 > > #define PCIE_PORT_LINKUP BIT(8) > > > > +#define PCIE_MSI_SET_NUM 8 > > +#define PCIE_MSI_IRQS_PER_SET 32 > > +#define PCIE_MSI_IRQS_NUM \ > > + (PCIE_MSI_IRQS_PER_SET * PCIE_MSI_SET_NUM) > > + > > #define PCIE_INT_ENABLE_REG 0x180 > > +#define PCIE_MSI_ENABLE GENMASK(PCIE_MSI_SET_NUM + 8 - 1, 8) > > +#define PCIE_MSI_SHIFT 8 > > #define PCIE_INTX_SHIFT 24 > > #define PCIE_INTX_ENABLE \ > > GENMASK(PCIE_INTX_SHIFT + PCI_NUM_INTX - 1, PCIE_INTX_SHIFT) > > > > #define PCIE_INT_STATUS_REG 0x184 > > +#define PCIE_MSI_SET_ENABLE_REG 0x190 > > +#define PCIE_MSI_SET_ENABLE GENMASK(PCIE_MSI_SET_NUM - 1, 0) > > + > > +#define PCIE_MSI_SET_BASE_REG 0xc00 > > +#define PCIE_MSI_SET_OFFSET 0x10 > > +#define PCIE_MSI_SET_STATUS_OFFSET 0x04 > > +#define PCIE_MSI_SET_ENABLE_OFFSET 0x08 > > + > > +#define PCIE_MSI_SET_ADDR_HI_BASE 0xc80 > > +#define PCIE_MSI_SET_ADDR_HI_OFFSET 0x04 > > > > #define PCIE_TRANS_TABLE_BASE_REG 0x800 > > #define PCIE_ATR_SRC_ADDR_MSB_OFFSET 0x4 > > @@ -73,6 +91,16 @@ > > #define PCIE_ATR_TLP_TYPE_MEM PCIE_ATR_TLP_TYPE(0) > > #define PCIE_ATR_TLP_TYPE_IO PCIE_ATR_TLP_TYPE(2) > > > > +/** > > + * struct mtk_pcie_msi - MSI information for each set > > + * @base: IO mapped register base > > + * @msg_addr: MSI message address > > + */ > > +struct mtk_msi_set { > > + void __iomem *base; > > + phys_addr_t msg_addr; > > +}; > > + > > /** > > * struct mtk_pcie_port - PCIe port information > > * @dev: pointer to PCIe device > > @@ -86,6 +114,11 @@ > > * @irq: PCIe controller interrupt number > > * @irq_lock: lock protecting IRQ register access > > * @intx_domain: legacy INTx IRQ domain > > + * @msi_domain: MSI IRQ domain > > + * @msi_bottom_domain: MSI IRQ bottom domain > > + * @msi_sets: MSI sets information > > + * @lock: lock protecting IRQ bit map > > + * @msi_irq_in_use: bit map for assigned MSI IRQ > > */ > > struct mtk_pcie_port { > > struct device *dev; > > @@ -100,6 +133,11 @@ struct mtk_pcie_port { > > int irq; > > raw_spinlock_t irq_lock; > > struct irq_domain *intx_domain; > > + struct irq_domain *msi_domain; > > + struct irq_domain *msi_bottom_domain; > > + struct mtk_msi_set msi_sets[PCIE_MSI_SET_NUM]; > > + struct mutex lock; > > + DECLARE_BITMAP(msi_irq_in_use, PCIE_MSI_IRQS_NUM); > > }; > > > > /** > > @@ -197,6 +235,35 @@ static int mtk_pcie_set_trans_table(struct mtk_pcie_port *port, > > return 0; > > } > > > > +static void mtk_pcie_enable_msi(struct mtk_pcie_port *port) > > +{ > > + int i; > > + u32 val; > > + > > + val = readl_relaxed(port->base + PCIE_MSI_SET_ENABLE_REG); > > + val |= PCIE_MSI_SET_ENABLE; > > + writel_relaxed(val, port->base + PCIE_MSI_SET_ENABLE_REG); > > + > > + val = readl_relaxed(port->base + PCIE_INT_ENABLE_REG); > > + val |= PCIE_MSI_ENABLE; > > + writel_relaxed(val, port->base + PCIE_INT_ENABLE_REG); > > Shouldn't you configure the capture addresses *before* enabling > things? Is there any need for locking here, given that you are > modifying global registers? Yes, I will move these codes to the back of the configure capture address in the next version. I think the lock may not be needed because this function is only executed once when driver probe. > > > + > > + for (i = 0; i < PCIE_MSI_SET_NUM; i++) { > > + struct mtk_msi_set *msi_set = &port->msi_sets[i]; > > + > > + msi_set->base = port->base + PCIE_MSI_SET_BASE_REG + > > + i * PCIE_MSI_SET_OFFSET; > > + msi_set->msg_addr = port->reg_base + PCIE_MSI_SET_BASE_REG + > > + i * PCIE_MSI_SET_OFFSET; > > + > > + /* Configure the MSI capture address */ > > + writel_relaxed(lower_32_bits(msi_set->msg_addr), msi_set->base); > > + writel_relaxed(upper_32_bits(msi_set->msg_addr), > > + port->base + PCIE_MSI_SET_ADDR_HI_BASE + > > + i * PCIE_MSI_SET_ADDR_HI_OFFSET); > > + } > > +} > > + > > static int mtk_pcie_startup_port(struct mtk_pcie_port *port) > > { > > struct resource_entry *entry; > > @@ -247,6 +314,8 @@ static int mtk_pcie_startup_port(struct mtk_pcie_port *port) > > return err; > > } > > > > + mtk_pcie_enable_msi(port); > > + > > /* Set PCIe translation windows */ > > resource_list_for_each_entry(entry, &host->windows) { > > struct resource *res = entry->res; > > @@ -290,6 +359,148 @@ static int mtk_pcie_set_affinity(struct irq_data *data, > > return -EINVAL; > > } > > > > +static void mtk_pcie_irq_mask(struct irq_data *data) > > It'd be good if you used _msi_ in function names that deal with MSIs. OK, I will fix it in the next version. > > > +{ > > + pci_msi_mask_irq(data); > > + irq_chip_mask_parent(data); > > +} > > + > > +static void mtk_pcie_irq_unmask(struct irq_data *data) > > +{ > > + pci_msi_unmask_irq(data); > > + irq_chip_unmask_parent(data); > > +} > > + > > +static struct irq_chip mtk_msi_irq_chip = { > > + .name = "MSI", > > + .irq_enable = mtk_pcie_irq_unmask, > > + .irq_disable = mtk_pcie_irq_mask, > > Same comment as for the previous patch: enable/disable serve no > purpose here. Replied in the previous patch, the enable/disable callback is used when the system suspend/resume. > > > + .irq_ack = irq_chip_ack_parent, > > + .irq_mask = mtk_pcie_irq_mask, > > + .irq_unmask = mtk_pcie_irq_unmask, > > +}; > > + > > +static struct msi_domain_info mtk_msi_domain_info = { > > + .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_PCI_MSIX | > > + MSI_FLAG_USE_DEF_CHIP_OPS | MSI_FLAG_MULTI_PCI_MSI), > > minor nit: keep the *_OPS flag on one line, and the *_PCI_* flags on > another. Sure, I will fix it in the next version. > > > + .chip = &mtk_msi_irq_chip, > > +}; > > + > > +static void mtk_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) > > +{ > > + struct mtk_msi_set *msi_set = irq_data_get_irq_chip_data(data); > > + struct mtk_pcie_port *port = data->domain->host_data; > > + unsigned long hwirq; > > + > > + hwirq = data->hwirq % PCIE_MSI_IRQS_PER_SET; > > + > > + msg->address_hi = upper_32_bits(msi_set->msg_addr); > > + msg->address_lo = lower_32_bits(msi_set->msg_addr); > > + msg->data = hwirq; > > + dev_dbg(port->dev, "msi#%#lx address_hi %#x address_lo %#x data %d\n", > > + hwirq, msg->address_hi, msg->address_lo, msg->data); > > +} > > + > > +static void mtk_msi_bottom_irq_ack(struct irq_data *data) > > +{ > > + struct mtk_msi_set *msi_set = irq_data_get_irq_chip_data(data); > > + unsigned long hwirq; > > + > > + hwirq = data->hwirq % PCIE_MSI_IRQS_PER_SET; > > + > > + writel_relaxed(BIT(hwirq), msi_set->base + PCIE_MSI_SET_STATUS_OFFSET); > > +} > > + > > +static void mtk_msi_bottom_irq_mask(struct irq_data *data) > > +{ > > + struct mtk_msi_set *msi_set = irq_data_get_irq_chip_data(data); > > + struct mtk_pcie_port *port = data->domain->host_data; > > + unsigned long hwirq, flags; > > + u32 val; > > + > > + hwirq = data->hwirq % PCIE_MSI_IRQS_PER_SET; > > + > > + raw_spin_lock_irqsave(&port->irq_lock, flags); > > + val = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + val &= ~BIT(hwirq); > > + writel_relaxed(val, msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + raw_spin_unlock_irqrestore(&port->irq_lock, flags); > > +} > > + > > +static void mtk_msi_bottom_irq_unmask(struct irq_data *data) > > +{ > > + struct mtk_msi_set *msi_set = irq_data_get_irq_chip_data(data); > > + struct mtk_pcie_port *port = data->domain->host_data; > > + unsigned long hwirq, flags; > > + u32 val; > > + > > + hwirq = data->hwirq % PCIE_MSI_IRQS_PER_SET; > > + > > + raw_spin_lock_irqsave(&port->irq_lock, flags); > > + val = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + val |= BIT(hwirq); > > + writel_relaxed(val, msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + raw_spin_unlock_irqrestore(&port->irq_lock, flags); > > +} > > + > > +static struct irq_chip mtk_msi_bottom_irq_chip = { > > + .irq_ack = mtk_msi_bottom_irq_ack, > > + .irq_mask = mtk_msi_bottom_irq_mask, > > + .irq_unmask = mtk_msi_bottom_irq_unmask, > > + .irq_compose_msi_msg = mtk_compose_msi_msg, > > + .irq_set_affinity = mtk_pcie_set_affinity, > > + .name = "MSI", > > +}; > > + > > +static int mtk_msi_bottom_domain_alloc(struct irq_domain *domain, > > + unsigned int virq, unsigned int nr_irqs, > > + void *arg) > > +{ > > + struct mtk_pcie_port *port = domain->host_data; > > + struct mtk_msi_set *msi_set; > > + int i, hwirq, set_idx; > > + > > + mutex_lock(&port->lock); > > + > > + hwirq = bitmap_find_free_region(port->msi_irq_in_use, PCIE_MSI_IRQS_NUM, > > + order_base_2(nr_irqs)); > > + > > + mutex_unlock(&port->lock); > > + > > + if (hwirq < 0) > > + return -ENOSPC; > > + > > + set_idx = hwirq / PCIE_MSI_IRQS_PER_SET; > > + msi_set = &port->msi_sets[set_idx]; > > + > > + for (i = 0; i < nr_irqs; i++) > > + irq_domain_set_info(domain, virq + i, hwirq + i, > > + &mtk_msi_bottom_irq_chip, msi_set, > > + handle_edge_irq, NULL, NULL); > > + > > + return 0; > > +} > > + > > +static void mtk_msi_bottom_domain_free(struct irq_domain *domain, > > + unsigned int virq, unsigned int nr_irqs) > > +{ > > + struct mtk_pcie_port *port = domain->host_data; > > + struct irq_data *data = irq_domain_get_irq_data(domain, virq); > > + > > + mutex_lock(&port->lock); > > + > > + bitmap_clear(port->msi_irq_in_use, data->hwirq, nr_irqs); > > + > > + mutex_unlock(&port->lock); > > + > > + irq_domain_free_irqs_common(domain, virq, nr_irqs); > > +} > > + > > +static const struct irq_domain_ops mtk_msi_bottom_domain_ops = { > > + .alloc = mtk_msi_bottom_domain_alloc, > > + .free = mtk_msi_bottom_domain_free, > > +}; > > + > > static void mtk_intx_mask(struct irq_data *data) > > { > > struct mtk_pcie_port *port = irq_data_get_irq_chip_data(data); > > @@ -360,6 +571,7 @@ static int mtk_pcie_init_irq_domains(struct mtk_pcie_port *port) > > { > > struct device *dev = port->dev; > > struct device_node *intc_node, *node = dev->of_node; > > + int ret; > > > > raw_spin_lock_init(&port->irq_lock); > > > > @@ -377,7 +589,34 @@ static int mtk_pcie_init_irq_domains(struct mtk_pcie_port *port) > > return -ENODEV; > > } > > > > + /* Setup MSI */ > > + mutex_init(&port->lock); > > + > > + port->msi_bottom_domain = irq_domain_add_linear(node, PCIE_MSI_IRQS_NUM, > > + &mtk_msi_bottom_domain_ops, port); > > + if (!port->msi_bottom_domain) { > > + dev_info(dev, "failed to create MSI bottom domain\n"); > > + ret = -ENODEV; > > + goto err_msi_bottom_domain; > > + } > > + > > + port->msi_domain = pci_msi_create_irq_domain(dev->fwnode, > > + &mtk_msi_domain_info, > > + port->msi_bottom_domain); > > + if (!port->msi_domain) { > > + dev_info(dev, "failed to create MSI domain\n"); > > + ret = -ENODEV; > > + goto err_msi_domain; > > + } > > + > > return 0; > > + > > +err_msi_domain: > > + irq_domain_remove(port->msi_bottom_domain); > > +err_msi_bottom_domain: > > + irq_domain_remove(port->intx_domain); > > + > > + return ret; > > } > > > > static void mtk_pcie_irq_teardown(struct mtk_pcie_port *port) > > @@ -387,9 +626,39 @@ static void mtk_pcie_irq_teardown(struct mtk_pcie_port *port) > > if (port->intx_domain) > > irq_domain_remove(port->intx_domain); > > > > + if (port->msi_domain) > > + irq_domain_remove(port->msi_domain); > > + > > + if (port->msi_bottom_domain) > > + irq_domain_remove(port->msi_bottom_domain); > > + > > irq_dispose_mapping(port->irq); > > } > > > > +static void mtk_pcie_msi_handler(struct mtk_pcie_port *port, int set_idx) > > +{ > > + struct mtk_msi_set *msi_set = &port->msi_sets[set_idx]; > > + unsigned long msi_enable, msi_status; > > + unsigned int virq; > > + irq_hw_number_t bit, hwirq; > > + > > + msi_enable = readl_relaxed(msi_set->base + PCIE_MSI_SET_ENABLE_OFFSET); > > + > > + do { > > + msi_status = readl_relaxed(msi_set->base + > > + PCIE_MSI_SET_STATUS_OFFSET); > > + msi_status &= msi_enable; > > + if (!msi_status) > > + break; > > + > > + for_each_set_bit(bit, &msi_status, PCIE_MSI_IRQS_PER_SET) { > > + hwirq = bit + set_idx * PCIE_MSI_IRQS_PER_SET; > > + virq = irq_find_mapping(port->msi_bottom_domain, hwirq); > > + generic_handle_irq(virq); > > + } > > + } while (true); > > +} > > + > > static void mtk_pcie_irq_handler(struct irq_desc *desc) > > { > > struct mtk_pcie_port *port = irq_desc_get_handler_data(desc); > > @@ -408,6 +677,14 @@ static void mtk_pcie_irq_handler(struct irq_desc *desc) > > generic_handle_irq(virq); > > } > > > > + irq_bit = PCIE_MSI_SHIFT; > > + for_each_set_bit_from(irq_bit, &status, PCIE_MSI_SET_NUM + > > + PCIE_MSI_SHIFT) { > > + mtk_pcie_msi_handler(port, irq_bit - PCIE_MSI_SHIFT); > > + > > + writel_relaxed(BIT(irq_bit), port->base + PCIE_INT_STATUS_REG); > > Isn't this write the same thing you have for EOI in the INTx case? > While I could understand your description in that case (this is a > resampling operation), I don't get what this does here. Either this is > also an EOI, but your initial description doesn't make sense, or it is > an Ack, and it should be moved to the right place. > > Which one is it? I think it should be an EOI which used to clear the interrupt status of a single set in the PCIe intc field, maybe I should move it to the end of the mtk_pcie_msi_handler() function. +-----+ | GIC | +-----+ ^ | port->irq | +-+-+-+-+-+-+-+-+ |0|1|2|3|4|5|6|7| (PCIe intc) +-+-+-+-+-+-+-+-+ ^ ^ ^ | | ... | +-------+ +------+ +-----------+ | | | +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ |0|1|...|30|31| |0|1|...|30|31| |0|1|...|30|31| (MSI sets) +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ | | | | | | | | | | | | (MSI vectors) | | | | | | | | | | | | (MSI SET0) (MSI SET1) ... (MSI SET7) I would like to ask another question. In this interrupt architecture, we cannot implement an affinity for PCIe interrupts, so we return a negative value in the mtk_pcie_set_affinity callback as follows: +static int mtk_pcie_set_affinity(struct irq_data *data, + const struct cpumask *mask, bool force) +{ + return -EINVAL; +} But there will always be error logs when hotplug a CPU: ~ # echo 0 > /sys/devices/system/cpu/cpu1/online [ 93.633059] IRQ255: set affinity failed(-22). [ 93.633624] IRQ256: set affinity failed(-22). [ 93.634222] CPU1: shutdown [ 93.634586] psci: CPU1 killed (polled 0 ms) Or when the system suspends: ~ # echo mem > /sys/power/state [ 93.635145] cpuhp: cpu_off cluster=0, cpu=1 [ 169.835653] PM: suspend entry (deep) [ 169.836717] Filesystems sync: 0.000 seconds [ 169.837924] Freezing user space processes ... (elapsed 0.001 seconds) done. [ 169.839922] OOM killer disabled. [ 169.840336] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. [ 169.844715] Disabling non-boot CPUs ... [ 169.846443] IRQ255: set affinity failed(-22). [ 169.847002] IRQ256: set affinity failed(-22). [ 169.847586] CPU2: shutdown [ 169.847943] psci: CPU2 killed (polled 0 ms) [ 169.848489] cpuhp: cpu_off cluster=0, cpu=2 [ 169.850285] IRQ255: set affinity failed(-22). [ 169.851369] IRQ256: set affinity failed(-22). ... Sometimes this can cause misunderstandings to users, do we have a chance to prevent this error log? > > Thanks, > > M. > Thanks. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel