From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757207AbcHaJ4P (ORCPT ); Wed, 31 Aug 2016 05:56:15 -0400 Received: from mail-dm3nam03on0076.outbound.protection.outlook.com ([104.47.41.76]:42273 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750914AbcHaJ4L (ORCPT ); Wed, 31 Aug 2016 05:56:11 -0400 X-Greylist: delayed 82841 seconds by postgrey-1.27 at vger.kernel.org; Wed, 31 Aug 2016 05:56:11 EDT Authentication-Results: spf=pass (sender IP is 149.199.60.83) smtp.mailfrom=xilinx.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=bestguesspass action=none header.from=xilinx.com; From: Bharat Kumar Gogada To: Marc Zyngier , "robh@kernel.org" , "bhelgaas@google.com" , "colin.king@canonical.com" , Soren Brinkmann , Michal Simek , "arnd@arndb.de" CC: "linux-arm-kernel@lists.infradead.org" , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" , Ravikiran Gummaluri Subject: RE: [PATCH 3/3] PCI: Xilinx NWL PCIe: Fix Error for multi function device for legacy interrupts. Thread-Topic: [PATCH 3/3] PCI: Xilinx NWL PCIe: Fix Error for multi function device for legacy interrupts. Thread-Index: AQHSAqrdY8Zj+O9d0E2xRiMSs6E3q6Bg5aOAgACfgzD//45TAIABrX2A Date: Wed, 31 Aug 2016 09:56:02 +0000 Message-ID: <8520D5D51A55D047800579B094147198258D28AF@XAP-PVEXMBX01.xlnx.xilinx.com> References: <1472553558-27215-1-git-send-email-bharatku@xilinx.com> <1472553558-27215-3-git-send-email-bharatku@xilinx.com> <57C57975.7040306@arm.com> <8520D5D51A55D047800579B094147198258D239D@XAP-PVEXMBX01.xlnx.xilinx.com> <57C59FE8.30307@arm.com> In-Reply-To: <57C59FE8.30307@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.23.94.45] Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 X-RCIS-Action: ALLOW X-TM-AS-Product-Ver: IMSS-7.1.0.1224-8.0.0.1202-22546.005 X-TM-AS-User-Approved-Sender: Yes;Yes X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:149.199.60.83;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(7916002)(2980300002)(438002)(189002)(199003)(24454002)(5660300001)(92566002)(54356999)(626004)(189998001)(2906002)(106116001)(4326007)(86362001)(50466002)(5250100002)(76176999)(81166006)(97756001)(87936001)(19580395003)(93886004)(81156014)(46406003)(4001430100002)(2201001)(5001770100001)(2920100001)(2950100001)(55846006)(8936002)(11100500001)(107886002)(50986999)(7846002)(8676002)(47776003)(2900100001)(7696003)(23726003)(2501003)(63266004)(3846002)(586003)(6116002)(7736002)(106466001)(356003)(33656002)(305945005)(102836003)(8746002)(107986001);DIR:OUT;SFP:1101;SCL:1;SRVR:BY2PR0201MB1493;H:xsj-pvapsmtpgw01;FPR:;SPF:Pass;PTR:unknown-60-83.xilinx.com;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;CY1NAM02FT037;1:4KqtFBgI6mOHtYMg0Sd4X7Fo7cleHjgEyYLnPaa/mZCMqJeYE2LZLMkoX/T7DEWsYuEJDutFo20KCayYdpSQhtm7A6bcbtajLAF7G69saEoImArLyEvJUKmcVXigS8mqIfCqY22Zx70NF7C/V3rg5tDlYfOta7GPF20tTXkn3eyuR/wITJ3lPuGrVu+tZW2e1hHbjn1YI9OSpnKGSwkoSbO5DBog0/GVQ4+WLD4p64k6autQkLBL4nka5PzpTaYVkzrpzquZHdk9/VFqnRT9vXM/IXVhwngSsXhPhmneSKN8d6ihrGzNbj2MLoYJfcjg+tLbtkxBPYYvJx/TnK+fksXSs7BnAUtjfrwXTQtqt1ktPiF955qnmwPUbsGu/iQUbP3MDjpBp9uUjzAn+gD266kDg6/utzfYfPtQkK2kO6krQU/OHqICrKN9N+WKVz4mkXsyNSeryDVpYVKbf0PIw54e+vKQpVvz624a/6TqnmEfDaPLUQzqoxJ1k1Xw4NIphXxy9jtixmwWHn5Ohdg+aYUR0SvAflXh7NpFcRV547Oph1bOyGpAoDG32XgYlX6VqbWh5R8QH9jK9Lt+TpwKHExwRFhp6Kl/gec7y7yJ56hHfWXGBppPwkN9uL058vZE X-MS-Office365-Filtering-Correlation-Id: 112d8de5-b2ae-4345-a65e-08d3d18507da X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1493;2:XR3Q97t/jMPoxSf7gcp+rLAaz3A8COFyGozdU6nlDJIc1TeN9t5nMcVnnKrZ+z6R1D77EaKxUu8C7ojqyQttzN8RnQDW+JLil2aVu7TjeZ/mdYMYtLLRIG8v+3KQizcPcg8MCsvDvv/0qZj8Ts0clEDCKl0oCwurw3uSqSCC8743q0qiuncuNhMeSMlwFGzn;3:CQhPFlCfQR+t8crofUJVNs1MdGPriOhHMhBVnOAFXLk29iXzwU02FDRsaCfN6coTkH34UBVpHJP5WoGMlRgKvzRQaAC2q+wxMGP5V8rg2Qyri6Ao0Cs/p5NcOqzlmPAR/WNEONNsHAYsXhff8BdTsyjpQFfHHJpVNv3b7i0XSMIW4u59JijY0gHeetWm4CCLGTGVnOCGWRAJPcXZhWzurFUP0p8DCd0Ed/i+GtRLwhs3vycPy3evuMklmDYkdBonfYUxl76+freJePI6+sXpvw== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(8251501002);SRVR:BY2PR0201MB1493; X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1493;25:HnHAjCIDLncEhuGUAQ3Os/gga4cDY61bADWnSTQC7SwuSsNW8dWXGDYNnCv//KGa5fML0MeD4+y+2p9T/O8dr5wPjKjW/3zCV/cF5tMDPbHeC9bRf2+k5ly6AHelPpYPAu3/huk/esbP7sxcWzbLVYtmodQmJ3ahC1+cjxSgkikQesgnRrczhazbvE8z/Qg4Y/KudUwiLx/rXQBSttGflZwl3iUA4NzbYm6sEthY54FOA6Qc/wbyp4hXrgaSGgYkaoegHd8BgFF1ky79164YRQtkakQJp3OfDSPsO0d9XLpVsQQsLIorgXP0awwdKcTFLN1ih2jD8NlOwg561XbvKwJZcIaHULbD+z7EB+oLfbhfwKmFtFYmRQ820ZKKFOFHzRPUCw7Mp0+8x2nFFqB2lVjDjUxVtVDV5st/4aSj0Bq04FSg6SoR5PgslsJtu/z4LdqdHD3hYib2otqLKOue3snD1P792CG10DEjgsrtjQt3JBUnZU65GuHgDhkPT3U2fcx9tiBenWw7nbWQvI5vc47dJF35mM0mW0JelsRa8xUohS6/PL5hld8uuwvTbAC0buTlzdytQnLNgFzaf+4pFyWJtQMMqUup/uJsZV8PInXRqYkpXtr+xW+zxALjT1OlMlqgtfKcqJ9pL2O2EfXfH1xvsN5WchV08cP+pe/h1DMZ+ZXryVxPpNxyrVTq1C9BK1RSVW6TmClxJqeC/ekWnpZBQ6XOp4BQsCV53od6QlRETrvd85TrslYiSRllcuIxRiw53xqnbKYXAxt336bJfl7zjqAoGmehwoPUzROHuYk= X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1493;31:C72+d+paNtp9tXKdzipmhN2gj9LooE+ktFKTw54qm0PzT7kvudEbLGeyhawNFj/U3m0vFqaK16f58L9Zr5DPrdYNkTa6aqUDea2DcR3F1Y/yigSK3+fvUbTRqDInRvDaYSG2JV8Eq5EnBgUUv+u3/xLbiKOW/SxccwML2l7+7gbSjJtKSmhz3rxdg+a4WfGex3u0ohOXVr0Z0T7G2NnkLGMysDKHA1AA000LOPPUTXA=;20:t7DJ01JyEpwbBIXxZDAAsrLkmcFrfvmLzITqAwzyVAntNXkh6ScebhkcmDBrEpzVWMIfhoggq42QCC7nECXC6zCbNYauly9hlPzuRE8ziTPHUm20CMj2VVTTqgrWaw0QgATNuBKk9LDbgYdCBC9bRn4wKdplIZNRQaKi9DXyKsOdxXnfVSxwml1XGaEmX2HlcGy69NmbWQ8QQKawKyNoJjNVflsBJLUOw1Gs7OUmXLtHCUrI7inv+dG4iWh8pMYqYr6liIJwQvyFuF0qD+NOeQl8HZz4oYCncHpKUWE5RzwJ8R/EsZmYZAehzr3YZ6ZZJB8m6cn8Kp0/lT8flhel4XdR4BgC28YivPgtJF++pT+hhpcvaDiG4VU6j79d1Un5MiZARzWUKnf/3jxIzhapgWg3uqVE8AvSwpK3xd+xmVhe/bCkFtq3Hir+ujYa2iZhkGxjb276Muv56r2JFDI5YQ1kdFm8OdCdm3/olUPsw1h4C6Ebcizsr3211kaNEHvn X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(13015025)(13017025)(13023025)(13024025)(13018025)(8121501046)(5005006)(10201501046)(3002001)(6055026);SRVR:BY2PR0201MB1493;BCL:0;PCL:0;RULEID:;SRVR:BY2PR0201MB1493; X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1493;4:JPbZzThpsPZWMPki9KL1RKYHtgfFRlJZy2Q9/Ggn59/KIrXFJq83CpKSLuzrvie/9ur9JzI6B8d5KrIF5uFxrtsRUwhWKPIjxLfqJ1h1QM6AFG+/ccFmiWkrbzIQuLWbfPXl7ojxC6baXBLdksF2Sa7Ma/2zAmohoRGUMu2Sqsg3fWi82tpKoRtrdv2OV9gk3FHifrlzhx8JE95ewLS3xbwLE2tXHV9LfhKcXpbkTat6g3yuePT6t4gx7pPpqgZRRqupQdmLz6N2e8bHhSbuuxXMV0TRtu7BvbSL1cSW8XA3TqyjzVSYo67q1rC/eZeRBESvb0Nv59duCwtpDmC+HvdXkYvp62rmtAvVYGTtqR9VJrUbozE1XQhgCg8vK6Mw+SZ2ZsPcmd9TVqPfeIB7x9by8hGT1ighexCsi/2BPU7se+V1Z5+d5TOhHmNaV9YWhR6e0R4OOSm8GeNBzXEQFE/y/qLOAk8IfvAanfqivCm44EP38uYqpNzeFTdfmmau X-Forefront-PRVS: 00514A2FE6 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BY2PR0201MB1493;23:Q38cBdeRXKJNfj5NOvYJV/8TCJzGHgxJQ8qBgby?= =?us-ascii?Q?38EqTOq8NlzauNHps4+GvVh/DcPZ+Ub75U3bRoC21hHRWjXPnvAIT4CFWk7f?= =?us-ascii?Q?C56Pl1uA+xXRSh5acDHZJSHjHEhLCybr7bfRX2i/20paDSHTGwZiTtSCR/Fw?= =?us-ascii?Q?ZShTPnVDrERnsPoGR+PIt7UGUaAEscGX9zjTN9zO8IJlxD8JdUCcji6IrwOX?= =?us-ascii?Q?Qzizyc83N9EgMKaLU3BUBUdjZnrAgdpDXokvjzx+8UJqZr3H4KzAa+hlhsDW?= =?us-ascii?Q?UX3Fa43+34uBrdqXkxaFoyJr6etBZDyam7Jw5PTCS5q0DzFo+fQMOAPAtWKV?= =?us-ascii?Q?TSKLk3HjhYoqUamKNMdJ9a6yg8aaCL8CH5JgPXM7WHaVFo/jdNe2kFXU2o3u?= =?us-ascii?Q?dGB4TLWsI9AL0F/7ft8PDWTgdUSfw9NGwo9CJkhWxuJglQ6eiJQymHyCvnus?= =?us-ascii?Q?zn/epsjCSgT/RY1j8pChVfWlARxnAVhN4oQrBkvToCMLmvwc0CS4D/kL0EhN?= =?us-ascii?Q?eY7aF/CcTlezZlGm9x+KWyiV/92HQcFNbCH3gQ/z0JEA8ksipQG6pvVpx6ml?= =?us-ascii?Q?D0s2G8RE9DLJyIv5Upsm6ztwbOtcUMASBCl1PIKo8cBjRxOpsXx1HB6oX5B2?= =?us-ascii?Q?qeISK8nL809zjyj8O4MqaQ8lYF0Ybax09Xur0SNLaKuYehsvefgP4RlSj4CO?= =?us-ascii?Q?TE56D2rw6yRyMahhbcaUKJoTbY0KHhvopXinx8mbITYsXCGLKzFpTYgRcKEB?= =?us-ascii?Q?JFUZ2SEXaQuGrc4Y04xZUV4Y14VTLsqBzEx1ApcH6W6MTwpfzebb/ZRvcGbD?= =?us-ascii?Q?U13h9LTH6Cqx/IE8rAAXWRzDwYiuULm0T+pWDlMvneq+l8JT3NTYFTILlcXa?= =?us-ascii?Q?ORKGmPLtP7Q7O5gjwqanjd6tcEnXquF4MybHYBJVwfsl9XLNJe4kqjU/WNUS?= =?us-ascii?Q?xvjWgaS5w/ct3CrmkGeqTFdCR7g034pCVD9KOdnfBrTkvz1Fgs8uZiRVyYZo?= =?us-ascii?Q?dkO45ck+2CXNxbNN4ZEGi10hiOqr9Ed8OUtdea5bPKYaC0FZIR0+DxMcUBez?= =?us-ascii?Q?RgIjAPsWT5O3bAdxBdK7wmQkPn3JvhDg9NmqG3tJMg4Wl9XnbywSCgM90eyk?= =?us-ascii?Q?tD6VSX5FHyr/PoVcynBqyGUB4Ewzvd41nE9BWjDdOE6MnQfM/yykxONkocj7?= =?us-ascii?Q?LrPWEMuAgbB6dou30t21uAwbWAfHJVaB39Ixk5qKXFo7e2ryFLfA/ucDW+Ax?= =?us-ascii?Q?ruinhAWHS5qm2XNX/JgXdkM3Tssc1lwUyRKxBxoeu4RhMJJHbn6q2mF4JakW?= =?us-ascii?Q?MN1nc4XXl9gcT5pRd0UM8zuw=3D?= X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1493;6:emDmtRNaU6haxAB+wzFoq79QZ5Byf5nn7l1G0m9ObNirvjmbidxeVGqmmgtAAmWoJ2zoItK+G/Ma3rkeLxcVZC8JVYgE+RAKEq998iWjYC+xV2f3ndOjzRW41unrPjejLXefW+cH9hq4n93JA1SYVjCtQkgdMAfZ+LGAi6XpZZoyu0qYCeRwQ52qxrh409EJdy80txQ4RvYyjlda1bXGO+S1RUyAXCxjgrhUeTk3FAxO5h9TS/k3PQz3MKt2XXAkaXwj2112QAAerNhrsMCcGt+6EvD13itlhnhLPAWcNU+yiUNmNcSm+rYPalCo4MkgaQJS4otlXcdzegzWHrx6qw==;5:OumE5EwB8cbcblWpPbcBXlyuxFjtFTeW9z8P0EVgs8mDuFq2hZDoS2QKrVhhiUP3PTO16KcEEGqfVV1sIxGeDqUxoMdKejCwN8/O92BNAXK9/EVmM0ssfZ8ZUtX6Q8a1Qc/s5PHbndnQG0HmFq7qDg==;24:YJPabRZ0AUAjGOd7RDeN8VD8xkmzhUw2xNXarGFlxnlfJ3a3HH3lwQmuTTi/EceOflxMHb2NmvcQ4NWmPxxDAEIIDUggixhKbI3q/bwkHJk=;7:p3sikBUnP2PNhmdHYvA0IsTq17cXPYz8xBKLQ2alF4c3pENK/Xi7F4aImoQyzzuq/1vp0FLIFu8qmfR4arbSFADsgPgFt1doEjexbeDEYPgkTZTNu3euU9AHDQDkpEJQkS/NIvdDoAyIYIYvF4ER+mh4GbHCWMzf+d1079Dxwm0vDAYwv6o4Zc1NyNgU4ItspbetbcvWGxaRt6PJ3imuuQlTdHNcQEZNnA2sehIbpAUOkixC2qh6xDATOTKsHRgn SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2016 09:56:07.6684 (UTC) X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c;Ip=[149.199.60.83];Helo=[xsj-pvapsmtpgw01] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR0201MB1493 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by mail.home.local id u7V9uJX3003201 > On 30/08/16 15:13, Bharat Kumar Gogada wrote: > >> Hi Bharat, > >>> @@ -561,7 +561,7 @@ static int nwl_pcie_init_irq_domain(struct > >>> nwl_pcie > >> *pcie) > >>> } > >>> > >>> pcie->legacy_irq_domain = irq_domain_add_linear(legacy_intc_node, > >>> - INTX_NUM, > >>> + INTX_NUM + 1, > >>> &legacy_domain_ops, > >>> pcie); > >> > >> This feels like the wrong thing to do. You have INTX_NUM irqs, so the > >> domain allocation should reflect this. On the other hand, the way the > >> driver currently deals with mappings is quite broken (consistently adding 1 to > the HW interrupt). > >> > > Hi Marc, > > > > Without above change I get following crash in kernel while booting. > > > > [ 2.441684] error: hwirq 0x4 is too large for dummy > > > > [ 2.441694] ------------[ cut here ]------------ > > > > [ 2.441698] WARNING: at kernel/irq/irqdomain.c:344 > > > > [ 2.441702] Modules linked in: > > > > [ 2.441706] > > > > [ 2.441714] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.4.0 #8 > > > > [ 2.441718] Hardware name: xlnx,zynqmp (DT) > > > > [ 2.441723] task: ffffffc071886b80 ti: ffffffc071888000 task.ti: > ffffffc071888000 > > > > [ 2.441732] PC is at irq_domain_associate+0x138/0x1c0 > > > > [ 2.441738] LR is at irq_domain_associate+0x138/0x1c0 > > > > In kernel/irq/irqdomain.c function irq_domain_associate > > > > if (WARN(hwirq >= domain->hwirq_max, > > "error: hwirq 0x%x is too large for %s\n", (int)hwirq, domain->name)) > > return -EINVAL; > > > > Here the hwirq and hwirq_max are equal to 4 without the above condition > (INTX_NUM + 1) due to which crash is coming. > > This is happening as the legacy interrupts are starting from 1 (INTA). > > I understood that. I'm still persisting in saying that you have the wrong fix. > > Your domain should always allocate many interrupts as you have interrupt > sources. These interrupts (hwirq) should be numbered from 0 to (n-1). Agreed, but here comes the problem the hwirq for legacy interrupts will start at 0x1 to 0x4 (INTA to INTD) and these values are as per PCIe specification for legacy interrupts. So these cannot be numbered from 0. So when 0x4 (INTD) for a multi-function device comes the crash occurs. > > > And I'm consistently adding 1 to the HW interrupt as in > > nwl_pcie_leg_handler I get 0th bit set from MSGF_LEG_STATUS if INTA > > interrupt is raised but my hwirq number being mapped for INTA is 0x1 > > so that's I'm adding 1 to obtain correct virtual irq. Same case in > > nwl_pcie_free_irq_domain since hwirq starts from one I'm adding 1 to > > obtain virtual irq and free it. > > I can see that. Nonetheless, this is wrong. Can you please test the patch I > provided in my reply and report what happens? Can you be more specific on what is the wrong, I'm adding one since the hwirq starts from 0x1 as mentioned above. I did try your suggestion with Ethernet card, but kernel hangs (it does not show any crash also, just hangs) when I do interface up (without bit + 1, using only bit position in handler). This is not working because in the legacy domain virq mapping starts with hwirq 0x1, there is no mapping for 0x0 in the domain, so EP interrupt is not serviced since virq being returned is zero. Thanks & Regards, Bharat