From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753916AbbKLI5Q (ORCPT ); Thu, 12 Nov 2015 03:57:16 -0500 Received: from relmlor2.renesas.com ([210.160.252.172]:47715 "EHLO relmlie1.idc.renesas.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753790AbbKLI5K convert rfc822-to-8bit (ORCPT ); Thu, 12 Nov 2015 03:57:10 -0500 X-IronPort-AV: E=Sophos;i="5.20,280,1444662000"; d="scan'208";a="198374603" From: Phil Edworthy To: Marc Zyngier , Thierry Reding CC: Bjorn Helgaas , Wolfram Sang , Geert Uytterhoeven , Simon Horman , "linux-pci@vger.kernel.org" , "linux-sh@vger.kernel.org" , "linux-kernel@vger.kernel.org" , Ley Foon Tan , Jingoo Han Subject: RE: [PATCH] PCI: pcie-rcar: Fix OF node passed to MSI irq domain Thread-Topic: [PATCH] PCI: pcie-rcar: Fix OF node passed to MSI irq domain Thread-Index: AQHRFhpazS7O4def1E+1x+YbvdM7WJ6T04kggAASxwCAABC7IIAADPxggAFvZICAAZ8KAIABBYmA Date: Thu, 12 Nov 2015 08:57:03 +0000 Message-ID: References: <1446542899-25137-1-git-send-email-phil.edworthy@renesas.com> <20151109161115.GA13870@ulmo.nvidia.com> <20151110155232.GA25368@ulmo.nvidia.com> <20151111163802.3a96080c@arm.com> In-Reply-To: <20151111163802.3a96080c@arm.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=phil.edworthy@renesas.com; x-originating-ip: [193.141.220.21] x-microsoft-exchange-diagnostics: 1;PS1PR06MB1178;5:qlXWSQf2feEkM4aqzz3cyQv37xL3OgWJCfhtqJVh+Tkjj3Q6rJeSzAo8ccWocLtFD985MU8qXBigGoxB+oMQl+De7JqemMZqkMsuKImKL1I1NmfteZUNX1ClCge5Vw5mDvDYQj/eyBVJyMKn1Fqivw==;24:sSOd/7/WFTC0175VA6D1Q6lkrLkKHVL9/OoQ1BMCNZZ6WJhzytj2V3bZl25YFwBqUWd1JNZXF0Arg2RWWCcdu8HHjiQqKElG+0iKCWqJD1c=;20:Mx4NWTOjLwZynLVe/C7J4/UlQNcwePJDyQwYNHOe+/comwpLUsH0llEiqkXus6ofhHC6uPXyOWYjhPdJ8N47YE+tz8p2LaCi6bKuWFe8Sg7I8yrtm5XvGztuauCN6hC0Yg8rtb+Tx70ukA/LOVdXfE58tw8kKI0xH8QDMqN7Wj8= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:PS1PR06MB1178; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(85106069007906); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(601004)(2401047)(520078)(8121501046)(5005006)(3002001)(10201501046);SRVR:PS1PR06MB1178;BCL:0;PCL:0;RULEID:;SRVR:PS1PR06MB1178; x-forefront-prvs: 07584EDBCD x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(24454002)(189002)(199003)(52604005)(87936001)(189998001)(5007970100001)(5001770100001)(5001920100001)(66066001)(105586002)(19580405001)(76576001)(5004730100002)(5001960100002)(2900100001)(19580395003)(106356001)(106116001)(97736004)(2950100001)(5890100001)(86362001)(81156007)(575784001)(5003600100002)(102836002)(33656002)(5002640100001)(101416001)(92566002)(50986999)(11100500001)(54356999)(74316001)(122556002)(10400500002)(93886004)(76176999)(40100003)(5008740100001)(77096005)(7059030);DIR:OUT;SFP:1102;SCL:1;SRVR:PS1PR06MB1178;H:PS1PR06MB1180.apcprd06.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8BIT MIME-Version: 1.0 X-OriginatorOrg: renesas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Nov 2015 08:57:03.6214 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 53d82571-da19-47e4-9cb4-625a166a4a2a X-MS-Exchange-Transport-CrossTenantHeadersStamped: PS1PR06MB1178 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Marc, On 11 November 2015 16:38, Marc Zyngier wrote: > On Tue, 10 Nov 2015 16:52:33 +0100 > Thierry Reding wrote: > > > On Mon, Nov 09, 2015 at 06:01:49PM +0000, Phil Edworthy wrote: > > > Hi Thierry, > > > > > > On 09 November 2015 17:24, Phil wrote: > > > > On 09 November 2015 16:11, Thierry wrote: > > > > > On Mon, Nov 09, 2015 at 03:20:24PM +0000, Phil Edworthy wrote: > > > > > > cc'ing others (Tegra, Altera, Designware) who may have the same bug > > > > > > > > > > > > On 03 November 2015 09:28, Phil Edworthy wrote: > > > > > > > The OF node passed to irq_domain_add_linear() should be a > > > > > > > pointer to interrupt controller's device tree node, or NULL, > > > > > > > but not the PCI controller's node. > > > > > > > > > > > > > > This fixes an oops in msi_domain_alloc_irqs() when it tries > > > > > > > to call msi_check(). > > > > > > > > > > > > > > Signed-off-by: Phil Edworthy > > > > > > > --- > > > > > > > drivers/pci/host/pcie-rcar.c | 2 +- > > > > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > > > > > > > diff --git a/drivers/pci/host/pcie-rcar.c b/drivers/pci/host/pcie-rcar.c > > > > > > > index 2377bf0..c6fa562 100644 > > > > > > > --- a/drivers/pci/host/pcie-rcar.c > > > > > > > +++ b/drivers/pci/host/pcie-rcar.c > > > > > > > @@ -709,7 +709,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie > > > > *pcie) > > > > > > > msi->chip.setup_irq = rcar_msi_setup_irq; > > > > > > > msi->chip.teardown_irq = rcar_msi_teardown_irq; > > > > > > > > > > > > > > - msi->domain = irq_domain_add_linear(pcie->dev->of_node, > > > > > > > INT_PCI_MSI_NR, > > > > > > > + msi->domain = irq_domain_add_linear(NULL, INT_PCI_MSI_NR, > > > > > > > &msi_domain_ops, &msi- > >chip); > > > > > > > if (!msi->domain) { > > > > > > > dev_err(&pdev->dev, "failed to create IRQ domain\n"); > > > > > > > > > > On Tegra the PCI controller is in fact the interrupt controller for > > > > > MSIs. And looking at the code here it seems like the same would apply to > > > > > RCAR. > > > > Yes you are correct here. > > > > > > > > > I'm also slightly confused as to why this would cause ->msi_check() to > > > > > fail. The default implementation (msi_domain_ops_check()) doesn't do > > > > > anything. > > > > > > > > > > Also, how is passing in NULL instead of a valid struct device_node * > > > > > going to prevent an oops? Perhaps this is one of those reference count > > > > > imbalance bugs that have recently been showing up? > > > > On arm64 (previously I didn't realise this just affects arm64, not arm), > > > > the changes in commit f075915ac0b11 ("PCI/MSI: Drop domain field from > > > > msi_controller") and d8a1cb757550 ("PCI/MSI: Let pci_msi_get_domain use > > > > struct device::msi_domain") return an uninitialized msi domain that leads > > > > to the oops. It appears that these changes assume that msi interrupt > > > > controller is separate from the PCI controller. > > > More accurately, when CONFIG_GENERIC_MSI_IRQ_DOMAIN is enabled, > > > pci_msi_get_domain() calls dev_get_msi_domain() and at this point > > > dev->msi_domain is uninitialized. > > > > Marc, any idea what's going on here? > > Thanks for putting me in the loop. > > No precise idea yet, but the proposed fix definitely looks like the > wrong one. Actually, not passing a node identifier to any domain > constructor is pretty much always a mistake when using DT. > > Can someone post a stack trace for this issue so that I can have a > look? I'm currently traveling, so expect a slightly delayed reply... Unfortunately, not all the code for this arm64 board is upstream yet, this code base is off 4.3-rc7. systemd-udevd[1315]: undefined instruction: pc=ffffffc03106d41c Code: ffffffc0 311f9740 ffffffc0 3106d138 (ffffffc0) Internal error: Oops - undefined instruction: 0 [#1] PREEMPT SMP Modules linked in: e1000e(+) CPU: 0 PID: 1315 Comm: systemd-udevd Not tainted 4.3.0-rc7+ #4 Hardware name: Renesas Salvator-X board based on r8a7795 (DT) task: ffffffc0307af080 ti: ffffffc030ecc000 task.ti: ffffffc030ecc000 PC is at 0xffffffc03106d41c LR is at msi_domain_alloc_irqs+0x3c/0x1e0 pc : [] lr : [] pstate: 00000145 sp : ffffffc030ecf930 x29: ffffffc030ecf930 x28: 0000000000000124 x27: ffffffc0312628dc x26: ffffffc03193c890 x25: ffffffc0311fd000 x24: 00000000000001b4 x23: ffffffc0311fd160 x22: ffffffc03193a810 x21: ffffffc03193c800 x20: ffffffc0312628c0 x19: 0000000000000003 x18: 0000000000770000 x17: 0000007f96948570 x16: ffffffc0001dfa70 x15: 003b9aca00000000 x14: 63696d616e796420 x13: 6f74207465732029 x12: 0000000000000038 x11: ffffff8000668fff x10: 0140000000000000 x9 : 0000000000000000 x8 : ffffffc030d7f500 x7 : 0000000000000000 x6 : 000000000000003f x5 : 0000000000000040 x4 : 0000000000000000 x3 : ffffffc03106d418 x2 : ffffffc03193c890 x1 : ffffffc0311fd160 x0 : ffffffc0311fd000 Process systemd-udevd (pid: 1315, stack limit = 0xffffffc030ecc020) Stack: (0xffffffc030ecf930 to 0xffffffc030ed0000) f920: ffffffc030ecf9a0 ffffffc00036537c f940: ffffffc03193c800 ffffffc0312628c0 ffffffc03193c800 ffffff8000668000 f960: ffffffc03193c890 0000000000000001 0000000000000003 ffffffc03193ca70 f980: ffffffc03193c890 0000000000000001 0000000000000002 ffffffc03193ca70 f9a0: ffffffc030ecf9d0 ffffffc000365990 0000000000000003 ffffffc000365960 f9c0: 0000000000000003 ffffffc0003658dc ffffffc030ecfa40 ffffffc000365a30 f9e0: 0000000000000003 ffffffc0312628c0 ffffffc03193c800 0000000000000003 fa00: ffffffc0304107c0 ffffffbffc020250 ffffffc030410000 0000000000020000 fa20: 0000000000000001 ffffffc03193c800 ffffffc03193c890 000000000004e5e0 fa40: ffffffc030ecfa70 ffffffbffc0192ec ffffffc0304107c0 ffffffc03193c800 fa60: ffffffc03193c890 ffffffbffc01e5e0 ffffffc030ecfa90 ffffffbffc01c9b8 fa80: ffffffc0304127c0 ffffffbffc01e5e0 ffffffc030ecfb10 ffffffc00035aaf0 faa0: ffffffc03193c890 ffffffbffc025f98 ffffffc03193c800 ffffffbffc025f30 fac0: ffffffbffc0206e8 ffffffbffc026090 ffffffc030ff3d00 ffffff800029d000 fae0: ffffffc0001173bc ffffffc00035aadc ffffffc03193c890 0000ffbffc025f98 fb00: ffffffc03193c800 ffffffbffc025f30 ffffffc030ecfb50 ffffffc0003d6030 fb20: ffffffc03193c890 ffffffc0009ca000 0000000000000000 ffffffbffc025f98 fb40: 0000000000000002 ffffffc0009ca000 ffffffc030ecfb90 ffffffc0003d61c0 fb60: ffffffc03193c890 ffffffbffc025f98 ffffffc03193c8f0 ffffffc00095bf08 fb80: ffffffc000968000 ffffffc0003d614c ffffffc030ecfbc0 ffffffc0003d4240 fba0: 0000000000000000 ffffffbffc025f98 ffffffc0003d6124 ffffffc030ecfc10 fbc0: ffffffc030ecfc00 ffffffc0003d5a9c ffffffbffc025f98 ffffffc030502cc0 fbe0: 0000000000000000 ffffffc000608b28 ffffffc0318e04a8 ffffffc0319d47e8 fc00: ffffffc030ecfc10 ffffffc0003d56d8 ffffffc030ecfc50 ffffffc0003d6a38 fc20: ffffffbffc025f98 ffffffc000934b20 ffffffc031262d00 ffffffbffc031000 fc40: 0000000000000000 0000000000000008 ffffffc030ecfc70 ffffffc00035995c fc60: ffffffc000934b20 ffffffc000934b20 ffffffc030ecfc80 ffffffbffc031034 fc80: ffffffc030ecfc90 ffffffc0000828ec ffffffc030ecfd10 ffffffc00013ecc4 fca0: ffffffbffc026040 ffffffc000941000 ffffffbffc026040 ffffffc031262dc0 fcc0: 0000000000000001 ffffffc000941000 ffffffbffc026040 0000000000000001 fce0: 0000000000000001 ffffffbffc026090 ffffffc030ff3d00 ffffff800029d000 fd00: ffffffc0001173bc ffffffc030ff3d10 ffffffc030ecfd40 ffffffc00011a700 fd20: ffffffc030ecfe70 ffffffc030ff3d10 ffffffbffc026040 0000000000000001 fd40: ffffffc030ecfe40 ffffffc00011ac3c 0000000000000000 0000000000000009 fd60: 0000007f96809f40 0000007f96944c14 0000000080000000 0000000000000015 fd80: 000000000000011c 0000000000000111 ffffffc00061b000 ffffffc030ecc000 fda0: ffffffc000000076 0000006400000077 ffffffbf00000072 ffffff800000006e fdc0: ffffffbf0000003f ffffffbffc031070 ffffffc0306125c0 ffffffc0306125c0 fde0: 000081a40000000f 0000000000000001 0000000000000000 00000000003c10e3 fe00: 00000000564449f9 0000000000000000 0000000000000000 0000000000000000 fe20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 fe40: 0000007fef721500 ffffffc000085cb0 0000000000000000 0000007f96809f40 fe60: ffffffffffffffff ffffffc030ecfed0 ffffff800029d000 00000000003c10e3 fe80: ffffff8000497b90 ffffff80004979d5 ffffff8000659d80 0000000000026370 fea0: 000000000002c418 0000000000000000 0000000000000000 0000002d0000002c fec0: 0000000000000017 0000000000000012 0000000000000009 0000007f96809f40 fee0: 0000000000000000 0000000000000009 0000000000000000 60ceffffffffffff ff00: ffffffffffffffff ffffffffffffffff 0000000000000111 fefefefefefeff42 ff20: 0101010101010101 0000000000000018 0000000000000000 ffffffffffff0000 ff40: 0000007f9688366c 0000000000000020 0000007f96944bf0 0000007f9681a768 ff60: 0000000000520000 000000559d66f190 0000007f96809f40 0000000000020000 ff80: 0000000000000000 0000007fef721ad8 0000000000000000 000000559d671360 ffa0: 0000000000000000 0000007fef721ac8 0000000000020000 0000007fef721500 ffc0: 0000007f96803c44 0000007fef721500 0000007f96944c14 0000000080000000 ffe0: 0000000000000009 0000000000000111 ffffffffffffffff ffffffffffffffff Call trace: [] 0xffffffc03106d41c [] pci_msi_setup_msi_irqs+0x24/0x64 [] pci_enable_msix+0x3cc/0x438 [] pci_enable_msix_range+0x34/0x80 [] e1000e_set_interrupt_capability+0xd0/0x110 [e1000e] [] e1000_probe+0x444/0xcb8 [e1000e] [] pci_device_probe+0x94/0x10c [] driver_probe_device+0x1e8/0x2e0 [] __driver_attach+0x98/0xa0 [] bus_for_each_dev+0x54/0x98 [] driver_attach+0x1c/0x28 [] bus_add_driver+0x1c0/0x228 [] driver_register+0x5c/0xf4 [] __pci_register_driver+0x40/0x4c [] e1000_init_module+0x30/0x70 [e1000e] [] do_one_initcall+0x88/0x198 [] do_init_module+0x58/0x1c4 [] load_module+0x1954/0x1ce0 [] SyS_finit_module+0x78/0x88 [] el0_svc_naked+0x20/0x28 Code: ffffffc0 311f9740 ffffffc0 3106d138 (ffffffc0) ---[ end trace 3249daca187c36b0 ]--- Thanks Phil