From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61A7EC43381 for ; Thu, 11 Mar 2021 09:48:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0F16A61481 for ; Thu, 11 Mar 2021 09:48:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232106AbhCKJsI (ORCPT ); Thu, 11 Mar 2021 04:48:08 -0500 Received: from mailgw02.mediatek.com ([1.203.163.81]:22066 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S232078AbhCKJr5 (ORCPT ); Thu, 11 Mar 2021 04:47:57 -0500 X-UUID: e815e2896fb546e58ba5ab0c1064d138-20210311 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=RNnASkbqhv8ODVj5ReFqDIzkkJKHV9MWUz02H5UDhV0=; b=QjgGCaYJOe1peZaZHiPDwcpAnJBubRkweCWEdGJ9NgJ8lRVB5LT7LycrhEQAAPUGyBT2mbKBfj5TEhjK+4bB6UxGkqSLu+DwzbNevXw0qoyf++g2qtBGhqRjNXxHGtJAfLNjfwPT74zjobpBqb8UulW40I4gkuL/MTgvxehrr4Q=; X-UUID: e815e2896fb546e58ba5ab0c1064d138-20210311 Received: from mtkcas34.mediatek.inc [(172.27.4.253)] by mailgw02.mediatek.com (envelope-from ) (mailgw01.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 757726739; Thu, 11 Mar 2021 17:47:48 +0800 Received: from MTKCAS32.mediatek.inc (172.27.4.184) by MTKMBS31N2.mediatek.inc (172.27.4.87) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 11 Mar 2021 17:47:46 +0800 Received: from [10.17.3.153] (10.17.3.153) by MTKCAS32.mediatek.inc (172.27.4.170) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 11 Mar 2021 17:47:45 +0800 Message-ID: <1615456065.25662.60.camel@mhfsdcap03> Subject: Re: [v8,5/7] PCI: mediatek-gen3: Add MSI support From: Jianjun Wang To: Marc Zyngier CC: Bjorn Helgaas , Rob Herring , Lorenzo Pieralisi , Ryder Lee , Philipp Zabel , "Matthias Brugger" , , , , , , "Sj Huang" , , , , , , , Date: Thu, 11 Mar 2021 17:47:45 +0800 In-Reply-To: <87a6rbxs4w.wl-maz@kernel.org> References: <20210224061132.26526-1-jianjun.wang@mediatek.com> <20210224061132.26526-6-jianjun.wang@mediatek.com> <87pn08y3ja.wl-maz@kernel.org> <1615358929.25662.47.camel@mhfsdcap03> <87a6rbxs4w.wl-maz@kernel.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-TM-SNTS-SMTP: 75906D7FE3E9CB7C6B4B63E7D7430F2D0076DFB9B07E89AAFCC25501E7DC0BD72000:8 X-MTK: N Content-Transfer-Encoding: base64 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org T24gV2VkLCAyMDIxLTAzLTEwIGF0IDA5OjQxICswMDAwLCBNYXJjIFp5bmdpZXIgd3JvdGU6DQo+ IE9uIFdlZCwgMTAgTWFyIDIwMjEgMDY6NDg6NDkgKzAwMDAsDQo+IEppYW5qdW4gV2FuZyA8amlh bmp1bi53YW5nQG1lZGlhdGVrLmNvbT4gd3JvdGU6DQo+ID4gPiA+ICtzdGF0aWMgc3RydWN0IGly cV9jaGlwIG10a19tc2lfaXJxX2NoaXAgPSB7DQo+ID4gPiA+ICsJLm5hbWUgPSAiTVNJIiwNCj4g PiA+ID4gKwkuaXJxX2VuYWJsZSA9IG10a19wY2llX2lycV91bm1hc2ssDQo+ID4gPiA+ICsJLmly cV9kaXNhYmxlID0gbXRrX3BjaWVfaXJxX21hc2ssDQo+ID4gPiANCj4gPiA+IFNhbWUgY29tbWVu dCBhcyBmb3IgdGhlIHByZXZpb3VzIHBhdGNoOiBlbmFibGUvZGlzYWJsZSBzZXJ2ZSBubw0KPiA+ ID4gcHVycG9zZSBoZXJlLg0KPiA+IA0KPiA+IFJlcGxpZWQgaW4gdGhlIHByZXZpb3VzIHBhdGNo LCB0aGUgZW5hYmxlL2Rpc2FibGUgY2FsbGJhY2sgaXMgdXNlZCB3aGVuDQo+ID4gdGhlIHN5c3Rl bSBzdXNwZW5kL3Jlc3VtZS4NCj4gDQo+IEFzIEkgc2FpZCwgeW91ciBzdXNwZW5kL3Jlc3VtZSBz aG91bGQgYmUgc2VsZiBjb250YWluZWQsIGFuZCBub3QgcmVseQ0KPiBvbiB0aGUgaXJxIHN1YnN5 c3RlbSB0byByZXN0b3JlIGEgdmlhYmxlIHN0YXRlLg0KDQpPSywgSSB3aWxsIHRyeSB0byBmaW5k IGFub3RoZXIgd2F5IHRvIHNhdmUgYW5kIHJlc3RvcmUgdGhlIGVuYWJsZWQgc3RhdGUNCm9mIGlu dGVycnVwdHMgd2hlbiB0aGUgc3lzdGVtIHN1c3BlbmQvcmVzdW1lLg0KDQo+IA0KPiBbLi4uXQ0K PiANCj4gPiA+ID4gQEAgLTQwOCw2ICs2NzcsMTQgQEAgc3RhdGljIHZvaWQgbXRrX3BjaWVfaXJx X2hhbmRsZXIoc3RydWN0IGlycV9kZXNjICpkZXNjKQ0KPiA+ID4gPiAgCQlnZW5lcmljX2hhbmRs ZV9pcnEodmlycSk7DQo+ID4gPiA+ICAJfQ0KPiA+ID4gPiAgDQo+ID4gPiA+ICsJaXJxX2JpdCA9 IFBDSUVfTVNJX1NISUZUOw0KPiA+ID4gPiArCWZvcl9lYWNoX3NldF9iaXRfZnJvbShpcnFfYml0 LCAmc3RhdHVzLCBQQ0lFX01TSV9TRVRfTlVNICsNCj4gPiA+ID4gKwkJCSAgICAgIFBDSUVfTVNJ X1NISUZUKSB7DQo+ID4gPiA+ICsJCW10a19wY2llX21zaV9oYW5kbGVyKHBvcnQsIGlycV9iaXQg LSBQQ0lFX01TSV9TSElGVCk7DQo+ID4gPiA+ICsNCj4gPiA+ID4gKwkJd3JpdGVsX3JlbGF4ZWQo QklUKGlycV9iaXQpLCBwb3J0LT5iYXNlICsgUENJRV9JTlRfU1RBVFVTX1JFRyk7DQo+ID4gPiAN Cj4gPiA+IElzbid0IHRoaXMgd3JpdGUgdGhlIHNhbWUgdGhpbmcgeW91IGhhdmUgZm9yIEVPSSBp biB0aGUgSU5UeCBjYXNlPw0KPiA+ID4gV2hpbGUgSSBjb3VsZCB1bmRlcnN0YW5kIHlvdXIgZGVz Y3JpcHRpb24gaW4gdGhhdCBjYXNlICh0aGlzIGlzIGENCj4gPiA+IHJlc2FtcGxpbmcgb3BlcmF0 aW9uKSwgSSBkb24ndCBnZXQgd2hhdCB0aGlzIGRvZXMgaGVyZS4gRWl0aGVyIHRoaXMgaXMNCj4g PiA+IGFsc28gYW4gRU9JLCBidXQgeW91ciBpbml0aWFsIGRlc2NyaXB0aW9uIGRvZXNuJ3QgbWFr ZSBzZW5zZSwgb3IgaXQgaXMNCj4gPiA+IGFuIEFjaywgYW5kIGl0IHNob3VsZCBiZSBtb3ZlZCB0 byB0aGUgcmlnaHQgcGxhY2UuDQo+ID4gPiANCj4gPiA+IFdoaWNoIG9uZSBpcyBpdD8NCj4gPiAN Cj4gPiBJIHRoaW5rIGl0IHNob3VsZCBiZSBhbiBFT0kgd2hpY2ggdXNlZCB0byBjbGVhciB0aGUg aW50ZXJydXB0IHN0YXR1cyBvZg0KPiA+IGEgc2luZ2xlIHNldCBpbiB0aGUgUENJZSBpbnRjIGZp ZWxkLCBtYXliZSBJIHNob3VsZCBtb3ZlIGl0IHRvIHRoZSBlbmQNCj4gPiBvZiB0aGUgbXRrX3Bj aWVfbXNpX2hhbmRsZXIoKSBmdW5jdGlvbi4NCj4gDQo+IEkgZG91YnQgdGhpcyBpcyBhbiBFT0ku IElmLCBhcyBJIHN1c3BlY3QsIGl0IGluc3RydWN0cyB0aGUgSFcgdG8gY2xlYXINCj4gdGhlIGJp dCBzbyB0aGF0IG5ldyBwZW5kaW5nIGJpdHMgY2FuIGJlIHJlY29yZGVkLCBpdCBtdXN0IHRha2Ug cGxhY2UNCj4gKmJlZm9yZSogdGhlIGludGVycnVwdCBpcyBoYW5kbGVkLCBvciB5b3UgbWF5IGxv c2UgTVNJcyBpbiB0aGUNCj4gaW50ZXJ2YWwgYmV0d2VlbiB0aGUgaGFuZGxpbmcgb2YgdGhlIGlu dGVycnVwdCBhbmQgdGhlIGNsZWFyaW5nIG9mIHRoZQ0KPiBwZW5kaW5nIGJpdC4gVG8gc2F0aXNm eSB0aGlzIHJlcXVpcmVtZW50LCB0aGlzIHNob3VsZCBiZSBhbiBBQ0ssIHdoaWNoDQo+IGlzIGNv bnNpc3RlbnQgd2l0aCB0aGUgd2F5IG1vc3QgTVNJIGNvbnRyb2xsZXJzIHN1Y2ggYXMgdGhpcyBv bmUgd29yay4NCg0KVGhlc2UgYml0cyBhcmUgc2ltaWxhciB3aXRoIHRoZSBpbnRlcnJ1cHQgc3Rh dHVzIG9mIElOVHgsIGFuZCB0aGUNCmludGVycnVwdCBzdGF0dXMgd2lsbCByZW1haW4gdW50aWwg YWxsIHRoZSBzdGF0dXMgb2YgdGhlIGNvcnJlc3BvbmRpbmcNCnNldCBhcmUgY2xlYXJlZC4gVGhl cmUgaXMgYSB3aGlsZSBsb29wIGluIG10a19wY2llX21zaV9oYW5kbGVyKCkgd2hpY2gNCmlzIHVz ZWQgdG8gY29udGludW91c2x5IHBvbGxpbmcgYW5kIEFDSyB0aGUgc3RhdHVzIG9mIHRoZSBNU0kg c2V0LCBJDQp0aGluayB0aGUgTVNJIG1heSBub3QgYmUgbG9zZSBpbiB0aGlzIGNhc2UuDQogDQo+ IA0KPiA+IA0KPiA+ICAgICAgICAgICAgICAgICAgICstLS0tLSsNCj4gPiAgICAgICAgICAgICAg ICAgICB8IEdJQyB8DQo+ID4gICAgICAgICAgICAgICAgICAgKy0tLS0tKw0KPiA+ICAgICAgICAg ICAgICAgICAgICAgIF4NCj4gPiAgICAgICAgICAgICAgICAgICAgICB8DQo+ID4gICAgICAgICAg ICAgICAgICBwb3J0LT5pcnENCj4gPiAgICAgICAgICAgICAgICAgICAgICB8DQo+ID4gICAgICAg ICAgICAgICstKy0rLSstKy0rLSstKy0rDQo+ID4gICAgICAgICAgICAgIHwwfDF8MnwzfDR8NXw2 fDd8IChQQ0llIGludGMpDQo+ID4gICAgICAgICAgICAgICstKy0rLSstKy0rLSstKy0rDQo+ID4g ICAgICAgICAgICAgICBeIF4gICAgICAgICAgIF4NCj4gPiAgICAgICAgICAgICAgIHwgfCAgICAu Li4gICAgfA0KPiA+ICAgICAgICstLS0tLS0tKyArLS0tLS0tKyAgICArLS0tLS0tLS0tLS0rDQo+ ID4gICAgICAgfCAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgIHwNCj4gPiArLSstKy0t LSstLSstLSsgICstKy0rLS0tKy0tKy0tKyAgKy0rLSstLS0rLS0rLS0rDQo+ID4gfDB8MXwuLi58 MzB8MzF8ICB8MHwxfC4uLnwzMHwzMXwgIHwwfDF8Li4ufDMwfDMxfCAoTVNJIHNldHMpDQo+ID4g Ky0rLSstLS0rLS0rLS0rICArLSstKy0tLSstLSstLSsgICstKy0rLS0tKy0tKy0tKw0KPiA+ICBe IF4gICAgICBeICBeICAgIF4gXiAgICAgIF4gIF4gICAgXiBeICAgICAgXiAgXg0KPiA+ICB8IHwg ICAgICB8ICB8ICAgIHwgfCAgICAgIHwgIHwgICAgfCB8ICAgICAgfCAgfCAgKE1TSSB2ZWN0b3Jz KQ0KPiA+ICB8IHwgICAgICB8ICB8ICAgIHwgfCAgICAgIHwgIHwgICAgfCB8ICAgICAgfCAgfA0K PiA+IA0KPiA+ICAgKE1TSSBTRVQwKSAgICAgICAoTVNJIFNFVDEpICAuLi4gICAoTVNJIFNFVDcp DQo+ID4gDQo+ID4gSSB3b3VsZCBsaWtlIHRvIGFzayBhbm90aGVyIHF1ZXN0aW9uLiBJbiB0aGlz IGludGVycnVwdCBhcmNoaXRlY3R1cmUsIHdlDQo+ID4gY2Fubm90IGltcGxlbWVudCBhbiBhZmZp bml0eSBmb3IgUENJZSBpbnRlcnJ1cHRzLCBzbyB3ZSByZXR1cm4gYQ0KPiA+IG5lZ2F0aXZlIHZh bHVlIGluIHRoZSBtdGtfcGNpZV9zZXRfYWZmaW5pdHkgY2FsbGJhY2sgYXMgZm9sbG93czogDQo+ ID4gDQo+ID4gK3N0YXRpYyBpbnQgbXRrX3BjaWVfc2V0X2FmZmluaXR5KHN0cnVjdCBpcnFfZGF0 YSAqZGF0YSwNCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb25zdCBzdHJ1 Y3QgY3B1bWFzayAqbWFzaywgYm9vbCBmb3JjZSkNCj4gPiArew0KPiA+ICsgICAgICAgcmV0dXJu IC1FSU5WQUw7DQo+ID4gK30NCj4gPiANCj4gPiBCdXQgdGhlcmUgd2lsbCBhbHdheXMgYmUgZXJy b3IgbG9ncyB3aGVuIGhvdHBsdWcgYSBDUFU6DQo+ID4gDQo+ID4gfiAjIGVjaG8gMCA+IC9zeXMv ZGV2aWNlcy9zeXN0ZW0vY3B1L2NwdTEvb25saW5lDQo+ID4gWyAgIDkzLjYzMzA1OV0gSVJRMjU1 OiBzZXQgYWZmaW5pdHkgZmFpbGVkKC0yMikuDQo+ID4gWyAgIDkzLjYzMzYyNF0gSVJRMjU2OiBz ZXQgYWZmaW5pdHkgZmFpbGVkKC0yMikuDQo+ID4gWyAgIDkzLjYzNDIyMl0gQ1BVMTogc2h1dGRv d24NCj4gPiBbICAgOTMuNjM0NTg2XSBwc2NpOiBDUFUxIGtpbGxlZCAocG9sbGVkIDAgbXMpDQo+ ID4gDQo+ID4gT3Igd2hlbiB0aGUgc3lzdGVtIHN1c3BlbmRzOg0KPiA+IA0KPiA+IH4gIyBlY2hv IG1lbSA+IC9zeXMvcG93ZXIvc3RhdGUNCj4gPiBbICAgOTMuNjM1MTQ1XSBjcHVocDogY3B1X29m ZiBjbHVzdGVyPTAsIGNwdT0xDQo+ID4gWyAgMTY5LjgzNTY1M10gUE06IHN1c3BlbmQgZW50cnkg KGRlZXApDQo+ID4gWyAgMTY5LjgzNjcxN10gRmlsZXN5c3RlbXMgc3luYzogMC4wMDAgc2Vjb25k cw0KPiA+IFsgIDE2OS44Mzc5MjRdIEZyZWV6aW5nIHVzZXIgc3BhY2UgcHJvY2Vzc2VzIC4uLiAo ZWxhcHNlZCAwLjAwMSBzZWNvbmRzKQ0KPiA+IGRvbmUuDQo+ID4gWyAgMTY5LjgzOTkyMl0gT09N IGtpbGxlciBkaXNhYmxlZC4NCj4gPiBbICAxNjkuODQwMzM2XSBGcmVlemluZyByZW1haW5pbmcg ZnJlZXphYmxlIHRhc2tzIC4uLiAoZWxhcHNlZCAwLjAwMQ0KPiA+IHNlY29uZHMpIGRvbmUuDQo+ ID4gWyAgMTY5Ljg0NDcxNV0gRGlzYWJsaW5nIG5vbi1ib290IENQVXMgLi4uDQo+ID4gWyAgMTY5 Ljg0NjQ0M10gSVJRMjU1OiBzZXQgYWZmaW5pdHkgZmFpbGVkKC0yMikuDQo+ID4gWyAgMTY5Ljg0 NzAwMl0gSVJRMjU2OiBzZXQgYWZmaW5pdHkgZmFpbGVkKC0yMikuDQo+ID4gWyAgMTY5Ljg0NzU4 Nl0gQ1BVMjogc2h1dGRvd24NCj4gPiBbICAxNjkuODQ3OTQzXSBwc2NpOiBDUFUyIGtpbGxlZCAo cG9sbGVkIDAgbXMpDQo+ID4gWyAgMTY5Ljg0ODQ4OV0gY3B1aHA6IGNwdV9vZmYgY2x1c3Rlcj0w LCBjcHU9Mg0KPiA+IFsgIDE2OS44NTAyODVdIElSUTI1NTogc2V0IGFmZmluaXR5IGZhaWxlZCgt MjIpLg0KPiA+IFsgIDE2OS44NTEzNjldIElSUTI1Njogc2V0IGFmZmluaXR5IGZhaWxlZCgtMjIp Lg0KPiA+IC4uLg0KPiA+IA0KPiA+IFNvbWV0aW1lcyB0aGlzIGNhbiBjYXVzZSBtaXN1bmRlcnN0 YW5kaW5ncyB0byB1c2VycywgZG8gd2UgaGF2ZSBhIGNoYW5jZQ0KPiA+IHRvIHByZXZlbnQgdGhp cyBlcnJvciBsb2c/DQo+IA0KPiBOby4gVGhpcyBIVyBkb2Vzbid0IGFsbG93IE1TSXMgdG8gYmUg aW5kaXZpZHVhbGx5IHJldGFyZ2V0ZWQsIGFuZCB0aGUNCj4ga2VybmVsIGlzbid0IGhhcHB5IGFi b3V0IHRoYXQuIFRoYXQncyBvbmUgb2YgdGhlIG1hbnkgcmVhc29ucyB3aHkNCj4gaGlkaW5nIE1T SXMgYmVoaW5kIGEgbXV4IChvciB0d28gaW4geW91ciBjYXNlKSBpcyBhICp2ZXJ5IGJhZCBpZGVh Ki4NCj4gDQo+IFRoYW5rcywNCj4gDQo+IAlNLg0KPiANCg0K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2663DC433E0 for ; Thu, 11 Mar 2021 09:48:14 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8AD5164F95 for ; Thu, 11 Mar 2021 09:48:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8AD5164F95 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Date:CC:To:From: Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=X+6f0mw1VCRmy2z02oeEagqY6vqXgEASDHdGJCsRw5I=; b=lkJlL38EiujF+B3c/rGzDAIsQ HNVVMim6ePn2XMPjSAvPipd9Z6sRasSZOH2Q1RReAaxACMv6/r4r3GrhTZ0FiqtrxqdVXVgldXinX uAYzcSWbMuNkw+xIUR8vBLAURTX6fKinawoAArPkfSnX2S4YhC1Xx6o0tx8WG9UnvllSFHKTP+afH GDlMTZsxScIMdKS1fGVEFGqnBGzwVb8Q00rU7ccvlJu/yyVhVtpqGVSDw68wW9KP6R6cPzUIfnk4W IGIZ3RaZoGbSEUSk1gkM0sP0NMg9XJUN4ZcHGp/ZO7lCbmGvHvKe7IToUOJP7VOvb1Ghm5JP0Yttd MWncjihGg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKHvJ-008mGb-BD; Thu, 11 Mar 2021 09:48:05 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lKHvB-008mFJ-W1; Thu, 11 Mar 2021 09:48:02 +0000 X-UUID: 6b81fd031a314f30929c85be43b0076d-20210311 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=RNnASkbqhv8ODVj5ReFqDIzkkJKHV9MWUz02H5UDhV0=; b=QjgGCaYJOe1peZaZHiPDwcpAnJBubRkweCWEdGJ9NgJ8lRVB5LT7LycrhEQAAPUGyBT2mbKBfj5TEhjK+4bB6UxGkqSLu+DwzbNevXw0qoyf++g2qtBGhqRjNXxHGtJAfLNjfwPT74zjobpBqb8UulW40I4gkuL/MTgvxehrr4Q=; X-UUID: 6b81fd031a314f30929c85be43b0076d-20210311 Received: from mtkcas68.mediatek.inc [(172.29.94.19)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 61747741; Thu, 11 Mar 2021 01:47:50 -0800 Received: from MTKMBS31N2.mediatek.inc (172.27.4.87) by MTKMBS62DR.mediatek.inc (172.29.94.18) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 11 Mar 2021 01:47:48 -0800 Received: from MTKCAS32.mediatek.inc (172.27.4.184) by MTKMBS31N2.mediatek.inc (172.27.4.87) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 11 Mar 2021 17:47:46 +0800 Received: from [10.17.3.153] (10.17.3.153) by MTKCAS32.mediatek.inc (172.27.4.170) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 11 Mar 2021 17:47:45 +0800 Message-ID: <1615456065.25662.60.camel@mhfsdcap03> Subject: Re: [v8,5/7] PCI: mediatek-gen3: Add MSI support From: Jianjun Wang To: Marc Zyngier CC: Bjorn Helgaas , Rob Herring , Lorenzo Pieralisi , Ryder Lee , Philipp Zabel , "Matthias Brugger" , , , , , , "Sj Huang" , , , , , , , Date: Thu, 11 Mar 2021 17:47:45 +0800 In-Reply-To: <87a6rbxs4w.wl-maz@kernel.org> References: <20210224061132.26526-1-jianjun.wang@mediatek.com> <20210224061132.26526-6-jianjun.wang@mediatek.com> <87pn08y3ja.wl-maz@kernel.org> <1615358929.25662.47.camel@mhfsdcap03> <87a6rbxs4w.wl-maz@kernel.org> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-TM-SNTS-SMTP: 75906D7FE3E9CB7C6B4B63E7D7430F2D0076DFB9B07E89AAFCC25501E7DC0BD72000:8 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210311_094758_598070_F8C7B4D0 X-CRM114-Status: GOOD ( 38.85 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org On Wed, 2021-03-10 at 09:41 +0000, Marc Zyngier wrote: > On Wed, 10 Mar 2021 06:48:49 +0000, > Jianjun Wang wrote: > > > > +static struct irq_chip mtk_msi_irq_chip = { > > > > + .name = "MSI", > > > > + .irq_enable = mtk_pcie_irq_unmask, > > > > + .irq_disable = mtk_pcie_irq_mask, > > > > > > Same comment as for the previous patch: enable/disable serve no > > > purpose here. > > > > Replied in the previous patch, the enable/disable callback is used when > > the system suspend/resume. > > As I said, your suspend/resume should be self contained, and not rely > on the irq subsystem to restore a viable state. OK, I will try to find another way to save and restore the enabled state of interrupts when the system suspend/resume. > > [...] > > > > > @@ -408,6 +677,14 @@ static void mtk_pcie_irq_handler(struct irq_desc *desc) > > > > generic_handle_irq(virq); > > > > } > > > > > > > > + irq_bit = PCIE_MSI_SHIFT; > > > > + for_each_set_bit_from(irq_bit, &status, PCIE_MSI_SET_NUM + > > > > + PCIE_MSI_SHIFT) { > > > > + mtk_pcie_msi_handler(port, irq_bit - PCIE_MSI_SHIFT); > > > > + > > > > + writel_relaxed(BIT(irq_bit), port->base + PCIE_INT_STATUS_REG); > > > > > > Isn't this write the same thing you have for EOI in the INTx case? > > > While I could understand your description in that case (this is a > > > resampling operation), I don't get what this does here. Either this is > > > also an EOI, but your initial description doesn't make sense, or it is > > > an Ack, and it should be moved to the right place. > > > > > > Which one is it? > > > > I think it should be an EOI which used to clear the interrupt status of > > a single set in the PCIe intc field, maybe I should move it to the end > > of the mtk_pcie_msi_handler() function. > > I doubt this is an EOI. If, as I suspect, it instructs the HW to clear > the bit so that new pending bits can be recorded, it must take place > *before* the interrupt is handled, or you may lose MSIs in the > interval between the handling of the interrupt and the clearing of the > pending bit. To satisfy this requirement, this should be an ACK, which > is consistent with the way most MSI controllers such as this one work. These bits are similar with the interrupt status of INTx, and the interrupt status will remain until all the status of the corresponding set are cleared. There is a while loop in mtk_pcie_msi_handler() which is used to continuously polling and ACK the status of the MSI set, I think the MSI may not be lose in this case. > > > > > +-----+ > > | GIC | > > +-----+ > > ^ > > | > > port->irq > > | > > +-+-+-+-+-+-+-+-+ > > |0|1|2|3|4|5|6|7| (PCIe intc) > > +-+-+-+-+-+-+-+-+ > > ^ ^ ^ > > | | ... | > > +-------+ +------+ +-----------+ > > | | | > > +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ > > |0|1|...|30|31| |0|1|...|30|31| |0|1|...|30|31| (MSI sets) > > +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ > > ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ > > | | | | | | | | | | | | (MSI vectors) > > | | | | | | | | | | | | > > > > (MSI SET0) (MSI SET1) ... (MSI SET7) > > > > I would like to ask another question. In this interrupt architecture, we > > cannot implement an affinity for PCIe interrupts, so we return a > > negative value in the mtk_pcie_set_affinity callback as follows: > > > > +static int mtk_pcie_set_affinity(struct irq_data *data, > > + const struct cpumask *mask, bool force) > > +{ > > + return -EINVAL; > > +} > > > > But there will always be error logs when hotplug a CPU: > > > > ~ # echo 0 > /sys/devices/system/cpu/cpu1/online > > [ 93.633059] IRQ255: set affinity failed(-22). > > [ 93.633624] IRQ256: set affinity failed(-22). > > [ 93.634222] CPU1: shutdown > > [ 93.634586] psci: CPU1 killed (polled 0 ms) > > > > Or when the system suspends: > > > > ~ # echo mem > /sys/power/state > > [ 93.635145] cpuhp: cpu_off cluster=0, cpu=1 > > [ 169.835653] PM: suspend entry (deep) > > [ 169.836717] Filesystems sync: 0.000 seconds > > [ 169.837924] Freezing user space processes ... (elapsed 0.001 seconds) > > done. > > [ 169.839922] OOM killer disabled. > > [ 169.840336] Freezing remaining freezable tasks ... (elapsed 0.001 > > seconds) done. > > [ 169.844715] Disabling non-boot CPUs ... > > [ 169.846443] IRQ255: set affinity failed(-22). > > [ 169.847002] IRQ256: set affinity failed(-22). > > [ 169.847586] CPU2: shutdown > > [ 169.847943] psci: CPU2 killed (polled 0 ms) > > [ 169.848489] cpuhp: cpu_off cluster=0, cpu=2 > > [ 169.850285] IRQ255: set affinity failed(-22). > > [ 169.851369] IRQ256: set affinity failed(-22). > > ... > > > > Sometimes this can cause misunderstandings to users, do we have a chance > > to prevent this error log? > > No. This HW doesn't allow MSIs to be individually retargeted, and the > kernel isn't happy about that. That's one of the many reasons why > hiding MSIs behind a mux (or two in your case) is a *very bad idea*. > > Thanks, > > M. > _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,URIBL_BLOCKED, USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA576C433E0 for ; Thu, 11 Mar 2021 09:49:42 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0845360235 for ; Thu, 11 Mar 2021 09:49:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0845360235 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=mediatek.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Date:CC:To:From: Subject:Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KSVWLVSy9+RAEOSNfJtjViuQWd7RKKadEYRAYFm+iNk=; b=od2YjY7V75JHVxhTt+2cDc85p HL43y/oTTUXcXapAmbEMsZsLHF6eeUgAAZolO4ji23WrCDLRMpyRhCkKF9hT8X7rnYUQkKWMqo4pD /anhu1ikmjjEUnfkYoRrwrLkjuWFue9EDoxyrjen1rP6tdedMhJ0VYzE0brL8kIuZqV0cS7fMqGUy PMBtR6BqZeh9hJxI9DJIPvAxRkkzTFzS6ZelLf6hq+L5lr4HjCMH2o+1zaXn1Ck5RENgB8gKd4MRn HZkTWlwZeSbzX8L4PqotebWleB8geYOR+JcIZafhIKYynqkHjbJ42c/5OBiSB8QaPwEwqoBQccZIm ggi1N5V2w==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKHvK-008mGq-Vc; Thu, 11 Mar 2021 09:48:07 +0000 Received: from mailgw02.mediatek.com ([216.200.240.185]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lKHvB-008mFJ-W1; Thu, 11 Mar 2021 09:48:02 +0000 X-UUID: 6b81fd031a314f30929c85be43b0076d-20210311 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References:In-Reply-To:Date:CC:To:From:Subject:Message-ID; bh=RNnASkbqhv8ODVj5ReFqDIzkkJKHV9MWUz02H5UDhV0=; b=QjgGCaYJOe1peZaZHiPDwcpAnJBubRkweCWEdGJ9NgJ8lRVB5LT7LycrhEQAAPUGyBT2mbKBfj5TEhjK+4bB6UxGkqSLu+DwzbNevXw0qoyf++g2qtBGhqRjNXxHGtJAfLNjfwPT74zjobpBqb8UulW40I4gkuL/MTgvxehrr4Q=; X-UUID: 6b81fd031a314f30929c85be43b0076d-20210311 Received: from mtkcas68.mediatek.inc [(172.29.94.19)] by mailgw02.mediatek.com (envelope-from ) (musrelay.mediatek.com ESMTP with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 61747741; Thu, 11 Mar 2021 01:47:50 -0800 Received: from MTKMBS31N2.mediatek.inc (172.27.4.87) by MTKMBS62DR.mediatek.inc (172.29.94.18) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 11 Mar 2021 01:47:48 -0800 Received: from MTKCAS32.mediatek.inc (172.27.4.184) by MTKMBS31N2.mediatek.inc (172.27.4.87) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 11 Mar 2021 17:47:46 +0800 Received: from [10.17.3.153] (10.17.3.153) by MTKCAS32.mediatek.inc (172.27.4.170) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 11 Mar 2021 17:47:45 +0800 Message-ID: <1615456065.25662.60.camel@mhfsdcap03> Subject: Re: [v8,5/7] PCI: mediatek-gen3: Add MSI support From: Jianjun Wang To: Marc Zyngier CC: Bjorn Helgaas , Rob Herring , Lorenzo Pieralisi , Ryder Lee , Philipp Zabel , "Matthias Brugger" , , , , , , "Sj Huang" , , , , , , , Date: Thu, 11 Mar 2021 17:47:45 +0800 In-Reply-To: <87a6rbxs4w.wl-maz@kernel.org> References: <20210224061132.26526-1-jianjun.wang@mediatek.com> <20210224061132.26526-6-jianjun.wang@mediatek.com> <87pn08y3ja.wl-maz@kernel.org> <1615358929.25662.47.camel@mhfsdcap03> <87a6rbxs4w.wl-maz@kernel.org> X-Mailer: Evolution 3.10.4-0ubuntu2 MIME-Version: 1.0 X-TM-SNTS-SMTP: 75906D7FE3E9CB7C6B4B63E7D7430F2D0076DFB9B07E89AAFCC25501E7DC0BD72000:8 X-MTK: N X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210311_094758_598070_F8C7B4D0 X-CRM114-Status: GOOD ( 38.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 2021-03-10 at 09:41 +0000, Marc Zyngier wrote: > On Wed, 10 Mar 2021 06:48:49 +0000, > Jianjun Wang wrote: > > > > +static struct irq_chip mtk_msi_irq_chip = { > > > > + .name = "MSI", > > > > + .irq_enable = mtk_pcie_irq_unmask, > > > > + .irq_disable = mtk_pcie_irq_mask, > > > > > > Same comment as for the previous patch: enable/disable serve no > > > purpose here. > > > > Replied in the previous patch, the enable/disable callback is used when > > the system suspend/resume. > > As I said, your suspend/resume should be self contained, and not rely > on the irq subsystem to restore a viable state. OK, I will try to find another way to save and restore the enabled state of interrupts when the system suspend/resume. > > [...] > > > > > @@ -408,6 +677,14 @@ static void mtk_pcie_irq_handler(struct irq_desc *desc) > > > > generic_handle_irq(virq); > > > > } > > > > > > > > + irq_bit = PCIE_MSI_SHIFT; > > > > + for_each_set_bit_from(irq_bit, &status, PCIE_MSI_SET_NUM + > > > > + PCIE_MSI_SHIFT) { > > > > + mtk_pcie_msi_handler(port, irq_bit - PCIE_MSI_SHIFT); > > > > + > > > > + writel_relaxed(BIT(irq_bit), port->base + PCIE_INT_STATUS_REG); > > > > > > Isn't this write the same thing you have for EOI in the INTx case? > > > While I could understand your description in that case (this is a > > > resampling operation), I don't get what this does here. Either this is > > > also an EOI, but your initial description doesn't make sense, or it is > > > an Ack, and it should be moved to the right place. > > > > > > Which one is it? > > > > I think it should be an EOI which used to clear the interrupt status of > > a single set in the PCIe intc field, maybe I should move it to the end > > of the mtk_pcie_msi_handler() function. > > I doubt this is an EOI. If, as I suspect, it instructs the HW to clear > the bit so that new pending bits can be recorded, it must take place > *before* the interrupt is handled, or you may lose MSIs in the > interval between the handling of the interrupt and the clearing of the > pending bit. To satisfy this requirement, this should be an ACK, which > is consistent with the way most MSI controllers such as this one work. These bits are similar with the interrupt status of INTx, and the interrupt status will remain until all the status of the corresponding set are cleared. There is a while loop in mtk_pcie_msi_handler() which is used to continuously polling and ACK the status of the MSI set, I think the MSI may not be lose in this case. > > > > > +-----+ > > | GIC | > > +-----+ > > ^ > > | > > port->irq > > | > > +-+-+-+-+-+-+-+-+ > > |0|1|2|3|4|5|6|7| (PCIe intc) > > +-+-+-+-+-+-+-+-+ > > ^ ^ ^ > > | | ... | > > +-------+ +------+ +-----------+ > > | | | > > +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ > > |0|1|...|30|31| |0|1|...|30|31| |0|1|...|30|31| (MSI sets) > > +-+-+---+--+--+ +-+-+---+--+--+ +-+-+---+--+--+ > > ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ > > | | | | | | | | | | | | (MSI vectors) > > | | | | | | | | | | | | > > > > (MSI SET0) (MSI SET1) ... (MSI SET7) > > > > I would like to ask another question. In this interrupt architecture, we > > cannot implement an affinity for PCIe interrupts, so we return a > > negative value in the mtk_pcie_set_affinity callback as follows: > > > > +static int mtk_pcie_set_affinity(struct irq_data *data, > > + const struct cpumask *mask, bool force) > > +{ > > + return -EINVAL; > > +} > > > > But there will always be error logs when hotplug a CPU: > > > > ~ # echo 0 > /sys/devices/system/cpu/cpu1/online > > [ 93.633059] IRQ255: set affinity failed(-22). > > [ 93.633624] IRQ256: set affinity failed(-22). > > [ 93.634222] CPU1: shutdown > > [ 93.634586] psci: CPU1 killed (polled 0 ms) > > > > Or when the system suspends: > > > > ~ # echo mem > /sys/power/state > > [ 93.635145] cpuhp: cpu_off cluster=0, cpu=1 > > [ 169.835653] PM: suspend entry (deep) > > [ 169.836717] Filesystems sync: 0.000 seconds > > [ 169.837924] Freezing user space processes ... (elapsed 0.001 seconds) > > done. > > [ 169.839922] OOM killer disabled. > > [ 169.840336] Freezing remaining freezable tasks ... (elapsed 0.001 > > seconds) done. > > [ 169.844715] Disabling non-boot CPUs ... > > [ 169.846443] IRQ255: set affinity failed(-22). > > [ 169.847002] IRQ256: set affinity failed(-22). > > [ 169.847586] CPU2: shutdown > > [ 169.847943] psci: CPU2 killed (polled 0 ms) > > [ 169.848489] cpuhp: cpu_off cluster=0, cpu=2 > > [ 169.850285] IRQ255: set affinity failed(-22). > > [ 169.851369] IRQ256: set affinity failed(-22). > > ... > > > > Sometimes this can cause misunderstandings to users, do we have a chance > > to prevent this error log? > > No. This HW doesn't allow MSIs to be individually retargeted, and the > kernel isn't happy about that. That's one of the many reasons why > hiding MSIs behind a mux (or two in your case) is a *very bad idea*. > > Thanks, > > M. > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel