From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 204A4C433EF for ; Tue, 22 Mar 2022 15:41:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238682AbiCVPmz (ORCPT ); Tue, 22 Mar 2022 11:42:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238753AbiCVPmw (ORCPT ); Tue, 22 Mar 2022 11:42:52 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C67C8BF70; Tue, 22 Mar 2022 08:41:22 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 8FFEEB81BC2; Tue, 22 Mar 2022 15:41:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC0A0C340F0; Tue, 22 Mar 2022 15:41:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647963680; bh=9lulS55rYge/f8W4ZmQhmOklPUPvVcTI/5fpSeo3QfU=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=fp+8gdct1DJvbNMsy39hAXeyD9lXCvxA3iEOwKO572M19fRn3QP/St5RrWWK3KP3Y CQ1DaQ5x0zUaMxrDl1AWLIwhkeEVGQFIFxyziNBLYOPJgh9RvO6lKQokoKekCuRGvz QF5d8J18mW7aXCw8HNLiwIk0eZVWM9nJicBrEp5nA5q0BuB+X1karYG8oOnzmB8gMe Yg/C7+6Z0cuIahWHjk13kYBhPWeJx3fYqs9zaMzQoXTVDx2QDgxpUeweJJwbgl4dQu O5T+mHXlvgy6EyVxp2LIiXQenuYedgXeiKgVHH6oNgJEpTxIVblHKXxNy3YgdUMpNz FTfk7k8NLj7Sg== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nWgdH-00GJ9a-PK; Tue, 22 Mar 2022 15:41:16 +0000 Date: Tue, 22 Mar 2022 15:41:15 +0000 Message-ID: <87bkxyxaec.wl-maz@kernel.org> From: Marc Zyngier To: Rob Herring Cc: Robin Murphy , dann frazier , "linux-kernel@vger.kernel.org" , linux-arm-kernel , PCI , Toan Le , Lorenzo Pieralisi , Krzysztof =?UTF-8?B?V2lsY3p5xYRza2k=?= , Bjorn Helgaas , =?UTF-8?B?U3TDqXBo?= =?UTF-8?B?YW5l?= Graber , Android Kernel Team Subject: Re: [PATCH v2 0/2] PCI: xgene: Restore working PCIe functionnality In-Reply-To: References: <20220321104843.949645-1-maz@kernel.org> <87h77rxnyl.wl-maz@kernel.org> <87fsnbxgau.wl-maz@kernel.org> <61809b8f-acaa-bae2-ac5e-aa47c55eea23@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: robh@kernel.org, robin.murphy@arm.com, dann.frazier@canonical.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, toan@os.amperecomputing.com, lorenzo.pieralisi@arm.com, kw@linux.com, bhelgaas@google.com, stgraber@ubuntu.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 22 Mar 2022 14:39:43 +0000, Rob Herring wrote: >=20 > On Tue, Mar 22, 2022 at 01:16:35PM +0000, Robin Murphy wrote: > > On 2022-03-21 20:06, Robin Murphy wrote: > > > On 2022-03-21 19:21, Marc Zyngier wrote: > > > > On Mon, 21 Mar 2022 18:03:27 +0000, > > > > Rob Herring wrote: > > > > >=20 > > > > > On Mon, Mar 21, 2022 at 11:36 AM Marc Zyngier wr= ote: > > > > > >=20 > > > > > > On Mon, 21 Mar 2022 15:17:34 +0000, > > > > > > Rob Herring wrote: > > > > > > >=20 > > > > > > > On Mon, Mar 21, 2022 at 5:49 AM Marc Zyngier = wrote: > > > > > > > >=20 > > > > > > > For XGene-1, I'd still like to understand what the issue is. = Reverting > > > > > > > the first fix and fixing 'dma-ranges' should have fixed it. I= need a > > > > > > > dump of how the IB registers are initialized in both cases. I= 'm not > > > > > > > saying changing 'dma-ranges' in the firmware is going to be r= equired > > > > > > > here. There's a couple of other ways we could fix that withou= t a > > > > > > > firmware change, but first I need to understand why it broke. > > > > > >=20 > > > > > > Reverting 6dce5aa59e0b was enough for me, without changing anyt= hing > > > > > > else. > > > > >=20 > > > > > Meaning c7a75d07827a didn't matter for you. I'm not sure that it = would. > > > > >=20 > > > > > Can you tell me what 'dma-ranges' contains on your system? > > > >=20 > > > > Each pcie node (all 5 of them) has: > > > >=20 > > > > dma-ranges =3D <0x42000000 0x80 0x00 0x80 0x00 0x00 0x80000000 > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 0x42000000 0x00 0x00 0x00 0x00 0x80 0x00>; >=20 > This is the same as what St=C3=A9phane has for Merlin. So c7a75d07827a ("= PCI:=20 > xgene: Fix IB window setup") should have fixed Mustang. Should, but didn't. The DT also carries additional properties: ib-ranges =3D <0x42000000 0x40 0x00 0x40 0x00 0x40 0x00 0x00 0x00 0x79000000 0x00 0x79000000 0x00 0x800000>; which the driver ignores, but that could be relevant. FWIW, I've stashed the DT at [1]. M. [1] http://www.loen.fr/tmp/mustang.dts --=20 Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 12299C433F5 for ; Tue, 22 Mar 2022 15:42:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rhXTKwLD0SD/HfMr8JQKF4xdmC5bhwHR/EATi8NrsS4=; b=vXj3k6/9fgDPTm Wmi/Kx3FTPAFkr5aF7ilNDV/TkqMqcsuVgDUKYbwTRr2BbsvKapoOEaZreAoetOhLzlWkme3MX+QU ApBLQhnb7Nk+n7X5ikhODFiRvwBYyxLQPmjQEokkbxVtZsC8Cd9U5fnFnuKvE9fp7VaTMUzmyCpPx 1UUU6StZO4uqaNMUIfF2N/V4yMWpCp+vXOKEaOCOA4CRsTma68K+Hb0kJwPmmaee0ku5yDOK1ZKWC fxFKMRA+ugSRq7lMtDmVg1d3ZW+Vsw2fF07M/bYONQD5qZmFc72DvLh6qsQ0hDxeTp0MEwI+Viy+Z pa8pg1kthbcWrEog0+ug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nWgdR-00BYlt-8j; Tue, 22 Mar 2022 15:41:25 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nWgdN-00BYl9-Ox for linux-arm-kernel@lists.infradead.org; Tue, 22 Mar 2022 15:41:23 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8B528611FD; Tue, 22 Mar 2022 15:41:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC0A0C340F0; Tue, 22 Mar 2022 15:41:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647963680; bh=9lulS55rYge/f8W4ZmQhmOklPUPvVcTI/5fpSeo3QfU=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=fp+8gdct1DJvbNMsy39hAXeyD9lXCvxA3iEOwKO572M19fRn3QP/St5RrWWK3KP3Y CQ1DaQ5x0zUaMxrDl1AWLIwhkeEVGQFIFxyziNBLYOPJgh9RvO6lKQokoKekCuRGvz QF5d8J18mW7aXCw8HNLiwIk0eZVWM9nJicBrEp5nA5q0BuB+X1karYG8oOnzmB8gMe Yg/C7+6Z0cuIahWHjk13kYBhPWeJx3fYqs9zaMzQoXTVDx2QDgxpUeweJJwbgl4dQu O5T+mHXlvgy6EyVxp2LIiXQenuYedgXeiKgVHH6oNgJEpTxIVblHKXxNy3YgdUMpNz FTfk7k8NLj7Sg== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nWgdH-00GJ9a-PK; Tue, 22 Mar 2022 15:41:16 +0000 Date: Tue, 22 Mar 2022 15:41:15 +0000 Message-ID: <87bkxyxaec.wl-maz@kernel.org> From: Marc Zyngier To: Rob Herring Cc: Robin Murphy , dann frazier , "linux-kernel@vger.kernel.org" , linux-arm-kernel , PCI , Toan Le , Lorenzo Pieralisi , Krzysztof =?UTF-8?B?V2lsY3p5xYRza2k=?= , Bjorn Helgaas , =?UTF-8?B?U3TDqXBo?= =?UTF-8?B?YW5l?= Graber , Android Kernel Team Subject: Re: [PATCH v2 0/2] PCI: xgene: Restore working PCIe functionnality In-Reply-To: References: <20220321104843.949645-1-maz@kernel.org> <87h77rxnyl.wl-maz@kernel.org> <87fsnbxgau.wl-maz@kernel.org> <61809b8f-acaa-bae2-ac5e-aa47c55eea23@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: robh@kernel.org, robin.murphy@arm.com, dann.frazier@canonical.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, toan@os.amperecomputing.com, lorenzo.pieralisi@arm.com, kw@linux.com, bhelgaas@google.com, stgraber@ubuntu.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220322_084121_932010_77743377 X-CRM114-Status: GOOD ( 26.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVHVlLCAyMiBNYXIgMjAyMiAxNDozOTo0MyArMDAwMCwKUm9iIEhlcnJpbmcgPHJvYmhAa2Vy bmVsLm9yZz4gd3JvdGU6Cj4gCj4gT24gVHVlLCBNYXIgMjIsIDIwMjIgYXQgMDE6MTY6MzVQTSAr MDAwMCwgUm9iaW4gTXVycGh5IHdyb3RlOgo+ID4gT24gMjAyMi0wMy0yMSAyMDowNiwgUm9iaW4g TXVycGh5IHdyb3RlOgo+ID4gPiBPbiAyMDIyLTAzLTIxIDE5OjIxLCBNYXJjIFp5bmdpZXIgd3Jv dGU6Cj4gPiA+ID4gT24gTW9uLCAyMSBNYXIgMjAyMiAxODowMzoyNyArMDAwMCwKPiA+ID4gPiBS b2IgSGVycmluZyA8cm9iaEBrZXJuZWwub3JnPiB3cm90ZToKPiA+ID4gPiA+IAo+ID4gPiA+ID4g T24gTW9uLCBNYXIgMjEsIDIwMjIgYXQgMTE6MzYgQU0gTWFyYyBaeW5naWVyIDxtYXpAa2VybmVs Lm9yZz4gd3JvdGU6Cj4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiBPbiBNb24sIDIxIE1hciAyMDIy IDE1OjE3OjM0ICswMDAwLAo+ID4gPiA+ID4gPiBSb2IgSGVycmluZyA8cm9iaEBrZXJuZWwub3Jn PiB3cm90ZToKPiA+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gPiBPbiBNb24sIE1hciAyMSwgMjAy MiBhdCA1OjQ5IEFNIE1hcmMgWnluZ2llciA8bWF6QGtlcm5lbC5vcmc+IHdyb3RlOgo+ID4gPiA+ ID4gPiA+ID4gCj4gPiA+ID4gPiA+ID4gRm9yIFhHZW5lLTEsIEknZCBzdGlsbCBsaWtlIHRvIHVu ZGVyc3RhbmQgd2hhdCB0aGUgaXNzdWUgaXMuIFJldmVydGluZwo+ID4gPiA+ID4gPiA+IHRoZSBm aXJzdCBmaXggYW5kIGZpeGluZyAnZG1hLXJhbmdlcycgc2hvdWxkIGhhdmUgZml4ZWQgaXQuIEkg bmVlZCBhCj4gPiA+ID4gPiA+ID4gZHVtcCBvZiBob3cgdGhlIElCIHJlZ2lzdGVycyBhcmUgaW5p dGlhbGl6ZWQgaW4gYm90aCBjYXNlcy4gSSdtIG5vdAo+ID4gPiA+ID4gPiA+IHNheWluZyBjaGFu Z2luZyAnZG1hLXJhbmdlcycgaW4gdGhlIGZpcm13YXJlIGlzIGdvaW5nIHRvIGJlIHJlcXVpcmVk Cj4gPiA+ID4gPiA+ID4gaGVyZS4gVGhlcmUncyBhIGNvdXBsZSBvZiBvdGhlciB3YXlzIHdlIGNv dWxkIGZpeCB0aGF0IHdpdGhvdXQgYQo+ID4gPiA+ID4gPiA+IGZpcm13YXJlIGNoYW5nZSwgYnV0 IGZpcnN0IEkgbmVlZCB0byB1bmRlcnN0YW5kIHdoeSBpdCBicm9rZS4KPiA+ID4gPiA+ID4gCj4g PiA+ID4gPiA+IFJldmVydGluZyA2ZGNlNWFhNTllMGIgd2FzIGVub3VnaCBmb3IgbWUsIHdpdGhv dXQgY2hhbmdpbmcgYW55dGhpbmcKPiA+ID4gPiA+ID4gZWxzZS4KPiA+ID4gPiA+IAo+ID4gPiA+ ID4gTWVhbmluZyBjN2E3NWQwNzgyN2EgZGlkbid0IG1hdHRlciBmb3IgeW91LiBJJ20gbm90IHN1 cmUgdGhhdCBpdCB3b3VsZC4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gQ2FuIHlvdSB0ZWxsIG1lIHdo YXQgJ2RtYS1yYW5nZXMnIGNvbnRhaW5zIG9uIHlvdXIgc3lzdGVtPwo+ID4gPiA+IAo+ID4gPiA+ IEVhY2ggcGNpZSBub2RlIChhbGwgNSBvZiB0aGVtKSBoYXM6Cj4gPiA+ID4gCj4gPiA+ID4gZG1h LXJhbmdlcyA9IDwweDQyMDAwMDAwIDB4ODAgMHgwMCAweDgwIDB4MDAgMHgwMCAweDgwMDAwMDAw Cj4gPiA+ID4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAweDQyMDAwMDAwIDB4MDAgMHgw MCAweDAwIDB4MDAgMHg4MCAweDAwPjsKPiAKPiBUaGlzIGlzIHRoZSBzYW1lIGFzIHdoYXQgU3TD qXBoYW5lIGhhcyBmb3IgTWVybGluLiBTbyBjN2E3NWQwNzgyN2EgKCJQQ0k6IAo+IHhnZW5lOiBG aXggSUIgd2luZG93IHNldHVwIikgc2hvdWxkIGhhdmUgZml4ZWQgTXVzdGFuZy4KClNob3VsZCwg YnV0IGRpZG4ndC4gVGhlIERUIGFsc28gY2FycmllcyBhZGRpdGlvbmFsIHByb3BlcnRpZXM6Cgpp Yi1yYW5nZXMgPSA8MHg0MjAwMDAwMCAweDQwIDB4MDAgMHg0MCAweDAwIDB4NDAgMHgwMAogICAg ICAgICAgICAgMHgwMCAweDAwIDB4NzkwMDAwMDAgMHgwMCAweDc5MDAwMDAwIDB4MDAgMHg4MDAw MDA+OwoKd2hpY2ggdGhlIGRyaXZlciBpZ25vcmVzLCBidXQgdGhhdCBjb3VsZCBiZSByZWxldmFu dC4gRldJVywgSSd2ZQpzdGFzaGVkIHRoZSBEVCBhdCBbMV0uCgoJTS4KClsxXSBodHRwOi8vd3d3 LmxvZW4uZnIvdG1wL211c3RhbmcuZHRzCgotLSAKV2l0aG91dCBkZXZpYXRpb24gZnJvbSB0aGUg bm9ybSwgcHJvZ3Jlc3MgaXMgbm90IHBvc3NpYmxlLgoKX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGlu dXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQu b3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=