From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CD69C433F5 for ; Sat, 5 Feb 2022 21:13:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381447AbiBEVNN (ORCPT ); Sat, 5 Feb 2022 16:13:13 -0500 Received: from smtp-relay-internal-0.canonical.com ([185.125.188.122]:59758 "EHLO smtp-relay-internal-0.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233073AbiBEVNM (ORCPT ); Sat, 5 Feb 2022 16:13:12 -0500 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id E54C23FE4B for ; Sat, 5 Feb 2022 21:13:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1644095590; bh=nz2Z4bFotqetThuLUskNVZCWGhwv1YrdlfnjDQe5gQs=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=K0puRBppIpE4rnEKVNnKGfOPwH0dhCZq5XLEdIgCOBGZByOikF33flBu2vbchkgEJ 74GKJBsu7xZoKu7/wzGJ7dxbzIsjbturV6JFTk8SlCLw3jZbSNmHwGEJp/SiMxTxH1 E/tl2PinE41s5nJH+FFUhmoL8/K0fTenK8d8irWz7Kc9pa26KlN3EciJwQlxWzm818 yoUFnz/zOyBUFv33vWOtK/vTMKaClWwyHRUFL+blv47CDPU1YgEsUOoloQcJ0xQVQ2 TbmmnU7W5BAPyiSysO4LM3DW38fqbe8E62+hzmuejUMp3IJyMBsLLQ/IiOpvVU644O K9pfcTyAi2YxA== Received: by mail-wm1-f69.google.com with SMTP id s187-20020a1ca9c4000000b0037bb8547d64so584896wme.0 for ; Sat, 05 Feb 2022 13:13:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=nz2Z4bFotqetThuLUskNVZCWGhwv1YrdlfnjDQe5gQs=; b=prwdYqjsSYv43bADxLLLBbOVtHvI2i9FAR2y251V5Q/jxkYWq88EELOP8QPAhqzngd uEBwqute1FKXHbj3D9KrP5UknBUARVVyquy4T1JE8F0y/o6UKSp7YI0xnaCkQksfldBS SCvqQJyZHulwJ4x3zDv1to8Sl8/op1q4iIo6zUZ1mPW/o8A+OjlRF8VYjVNXVDVuJ2/V zBQOmm8/Ii1gt02K/tLE+8zM4n8oBAKUsDhIvmVOqCHdzgVhvKVo+qgtZIpFmjpWTGvA 2Ot03i++zsLp7bNB8UvnGMhtFhV6TjXVcMr00kQq8vfajlydYvAyAFqVDOAHLTg+8em8 h6Yg== X-Gm-Message-State: AOAM533Ba4+yMNHKwBmD81UF6DwZ+H6MWPg9kMoYD31+ncY9LnHnZ7xt TG+6nw6HFxsjuFf2Xdp1OCQoYhC6JdCOLFZ3DA1DW9ScqevAHxxXJhZB5dEq991bIIQQ6IQopV5 FnxeS/eduYXlC90zRMiw9l6vh8kXOr3P76C9TloE6GXKghCq1897C3ydIEw== X-Received: by 2002:a05:6000:1707:: with SMTP id n7mr4286322wrc.234.1644095590469; Sat, 05 Feb 2022 13:13:10 -0800 (PST) X-Google-Smtp-Source: ABdhPJx0O5lMWQqmnSzmHWQqbnIIH10oDMuf24v+GRzu1vHjjMNiBTdjAaypokB3rE4ZxL9Iq0Fr05Wxc5BZQWT7mig= X-Received: by 2002:a05:6000:1707:: with SMTP id n7mr4286310wrc.234.1644095590242; Sat, 05 Feb 2022 13:13:10 -0800 (PST) MIME-Version: 1.0 References: <20211129173637.303201-1-robh@kernel.org> In-Reply-To: From: dann frazier Date: Sat, 5 Feb 2022 14:12:57 -0700 Message-ID: Subject: Re: [PATCH] PCI: xgene: Fix IB window setup To: Rob Herring Cc: Toan Le , Lorenzo Pieralisi , =?UTF-8?Q?Krzysztof_Wilczy=C5=84ski?= , Bjorn Helgaas , Andrew Murray , =?UTF-8?Q?St=C3=A9phane_Graber?= , stable , PCI , linux-arm-kernel , "linux-kernel@vger.kernel.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, Feb 5, 2022 at 9:05 AM Rob Herring wrote: > > On Fri, Feb 4, 2022 at 5:01 PM dann frazier = wrote: > > > > On Mon, Nov 29, 2021 at 11:36:37AM -0600, Rob Herring wrote: > > > Commit 6dce5aa59e0b ("PCI: xgene: Use inbound resources for setup") > > > broke PCI support on XGene. The cause is the IB resources are now sor= ted > > > in address order instead of being in DT dma-ranges order. The result = is > > > which inbound registers are used for each region are swapped. I don't > > > know the details about this h/w, but it appears that IB region 0 > > > registers can't handle a size greater than 4GB. In any case, limiting > > > the size for region 0 is enough to get back to the original assignmen= t > > > of dma-ranges to regions. > > > > hey Rob! > > > > I've been seeing a panic on HP Moonshoot m400 cartridges (X-Gene1) - > > only during network installs - that I also bisected down to commit > > 6dce5aa59e0b ("PCI: xgene: Use inbound resources for setup"). I was > > hoping that this patch that fixed the issue on St=C3=A9phane's X-Gene2 > > system would also fix my issue, but no luck. In fact, it seems to just > > makes it fail differently. Reverting both patches is required to get a > > v5.17-rc kernel to boot. > > > > I've collected the following logs - let me know if anything else would > > be useful. > > > > 1) v5.17-rc2+ (unmodified): > > http://dannf.org/bugs/m400-no-reverts.log > > Note that the mlx4 driver fails initialization. > > > > 2) v5.17-rc2+, w/o the commit that fixed St=C3=A9phane's system: > > http://dannf.org/bugs/m400-xgene2-fix-reverted.log > > Note the mlx4 MSI-X timeout, and later panic. > > > > 3) v5.17-rc2+, w/ both commits reverted (works) > > http://dannf.org/bugs/m400-both-reverted.log > > The ranges and dma-ranges addresses don't appear to match up with any > upstream dts files. Can you send me the DT? Sure: http://dannf.org/bugs/fdt -dann > Otherwise, we're going to need some debugging added to > xgene_pcie_setup_ib_reg() to see if the register setup changed. I can > come up with something next week. > > Rob From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B8A0C433EF for ; Sat, 5 Feb 2022 21:15:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SGOTBUItmHAZwDTeNVIwTvuieFKziTVfa/LR8L4Ioqo=; b=KuSo1o4fTdfQDK y5hRCk3390FScGkM6ONRE8kSWEpYMnSElu8tF8vbW/ncUVdT7peWvQYOpCGhSH3kYfoWYeqEMWZ4p ROdiulDqd6LOTlDCFTk80VanVAZGn8PmOUa/dhPQn+LA0Nr29Iht6PiOzYXtpwLDbZfyNGIY1Aw0q 7d4moaabJw7l6H8+FppoRs2Vx0YpcOaTv7DwYraBNXMbjG0yEx4E8KimE1DRyCHNEX0WXYTB0u/+f 2+kxzpGJZsrop4OsC9BqHXtBLXb63/5UJ31k9A8vOOAQtNcV7lrdDWezwaZg8tBx/S9nCRgxEWWe+ zjLsVJ5nXz2cAjKiD9zg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nGSN2-007Umd-Tt; Sat, 05 Feb 2022 21:13:25 +0000 Received: from smtp-relay-internal-1.canonical.com ([185.125.188.123]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nGSMy-007Um3-Sv for linux-arm-kernel@lists.infradead.org; Sat, 05 Feb 2022 21:13:22 +0000 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 0A50940038 for ; Sat, 5 Feb 2022 21:13:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1644095591; bh=nz2Z4bFotqetThuLUskNVZCWGhwv1YrdlfnjDQe5gQs=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=j9KYmL+2qTNozmxeiIe/qty03ZLOSzZdU4+u1hlOqmIUl29eq4eEYDYhJvVfj+Yl0 NhvbbmWMY+0qNN51Bm7aQ/H+3u9aDUFrNpcollZnZfVWuK3WnHnScmTjaTD7bIHbMI BsgaHjASVKOiilkQwCDddB/9kuPmubkjNXr/XV6+NLSKiq2Dd2NFYBCn941odAT1Hp fDxr8jfG1AdOQnbB5jBOra0y5XAxUVKqK754wXEze6b0IWi0LjKOhjk63lcGH9aTLl Ol7MHiFkHE9EOpg+absoZAma6lA9cbF5WS2xSfI7hzXX5TgJbzUmvduvIJbdk6ZDl8 1gHaf27hlfZJQ== Received: by mail-wm1-f69.google.com with SMTP id r16-20020a05600c2c5000b0037bb20c50b8so366882wmg.3 for ; Sat, 05 Feb 2022 13:13:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=nz2Z4bFotqetThuLUskNVZCWGhwv1YrdlfnjDQe5gQs=; b=m3+rs2abtJPrAmyZL3HEGuyIscDaWfqrCPkqLtbuxmtBMjW636SCI91y4TFG2JAahj jEJh+V1Q8BWFuyOzmjRqmtMB+FlaEKb/MbAMBXmwGitHtfyPMTw+WGYMcF8tWCMAEO79 d0QknNmxudryjnKZpPIAykaDyTfEr8xpNDyKNzjOt0KvBXdJWtUe5td20HlafHwVzlCb cKq9p2mLFC2NzAQRLtGpspsBgWRx6IC7tBN1yBkMGlc65Dvqd9gQ/PdlszOpeXOGVQjK OJ6JtawP1Tbch+WKgr2Pp7eaz66aSvz+cBnQBhfEsr6LpUiXATdf4g2iPZONddRSSd/I A4gw== X-Gm-Message-State: AOAM5309u3vv/q9rxajXIQJJt3ICsoLoIrAKfqEdaIfPM+FMMLJ3ou7T Vk2U2xCy20DXkc6PxLmHHyTCHS5LIBsIyjjviFO7FJ6IzScbMmM5T5XQeacdWM0HyI7T5s7SQFc ShmVUUsAz/7MwfjTy8aRG9GS6l7UYkMCzFrPQSZrrTTmSIiJ7wYYM1wwhq19t6Cwq6Jo/ X-Received: by 2002:a05:6000:1707:: with SMTP id n7mr4286324wrc.234.1644095590470; Sat, 05 Feb 2022 13:13:10 -0800 (PST) X-Google-Smtp-Source: ABdhPJx0O5lMWQqmnSzmHWQqbnIIH10oDMuf24v+GRzu1vHjjMNiBTdjAaypokB3rE4ZxL9Iq0Fr05Wxc5BZQWT7mig= X-Received: by 2002:a05:6000:1707:: with SMTP id n7mr4286310wrc.234.1644095590242; Sat, 05 Feb 2022 13:13:10 -0800 (PST) MIME-Version: 1.0 References: <20211129173637.303201-1-robh@kernel.org> In-Reply-To: From: dann frazier Date: Sat, 5 Feb 2022 14:12:57 -0700 Message-ID: Subject: Re: [PATCH] PCI: xgene: Fix IB window setup To: Rob Herring Cc: Toan Le , Lorenzo Pieralisi , =?UTF-8?Q?Krzysztof_Wilczy=C5=84ski?= , Bjorn Helgaas , Andrew Murray , =?UTF-8?Q?St=C3=A9phane_Graber?= , stable , PCI , linux-arm-kernel , "linux-kernel@vger.kernel.org" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220205_131321_076118_616563AE X-CRM114-Status: GOOD ( 29.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gU2F0LCBGZWIgNSwgMjAyMiBhdCA5OjA1IEFNIFJvYiBIZXJyaW5nIDxyb2JoQGtlcm5lbC5v cmc+IHdyb3RlOgo+Cj4gT24gRnJpLCBGZWIgNCwgMjAyMiBhdCA1OjAxIFBNIGRhbm4gZnJhemll ciA8ZGFubi5mcmF6aWVyQGNhbm9uaWNhbC5jb20+IHdyb3RlOgo+ID4KPiA+IE9uIE1vbiwgTm92 IDI5LCAyMDIxIGF0IDExOjM2OjM3QU0gLTA2MDAsIFJvYiBIZXJyaW5nIHdyb3RlOgo+ID4gPiBD b21taXQgNmRjZTVhYTU5ZTBiICgiUENJOiB4Z2VuZTogVXNlIGluYm91bmQgcmVzb3VyY2VzIGZv ciBzZXR1cCIpCj4gPiA+IGJyb2tlIFBDSSBzdXBwb3J0IG9uIFhHZW5lLiBUaGUgY2F1c2UgaXMg dGhlIElCIHJlc291cmNlcyBhcmUgbm93IHNvcnRlZAo+ID4gPiBpbiBhZGRyZXNzIG9yZGVyIGlu c3RlYWQgb2YgYmVpbmcgaW4gRFQgZG1hLXJhbmdlcyBvcmRlci4gVGhlIHJlc3VsdCBpcwo+ID4g PiB3aGljaCBpbmJvdW5kIHJlZ2lzdGVycyBhcmUgdXNlZCBmb3IgZWFjaCByZWdpb24gYXJlIHN3 YXBwZWQuIEkgZG9uJ3QKPiA+ID4ga25vdyB0aGUgZGV0YWlscyBhYm91dCB0aGlzIGgvdywgYnV0 IGl0IGFwcGVhcnMgdGhhdCBJQiByZWdpb24gMAo+ID4gPiByZWdpc3RlcnMgY2FuJ3QgaGFuZGxl IGEgc2l6ZSBncmVhdGVyIHRoYW4gNEdCLiBJbiBhbnkgY2FzZSwgbGltaXRpbmcKPiA+ID4gdGhl IHNpemUgZm9yIHJlZ2lvbiAwIGlzIGVub3VnaCB0byBnZXQgYmFjayB0byB0aGUgb3JpZ2luYWwg YXNzaWdubWVudAo+ID4gPiBvZiBkbWEtcmFuZ2VzIHRvIHJlZ2lvbnMuCj4gPgo+ID4gaGV5IFJv YiEKPiA+Cj4gPiBJJ3ZlIGJlZW4gc2VlaW5nIGEgcGFuaWMgb24gSFAgTW9vbnNob290IG00MDAg Y2FydHJpZGdlcyAoWC1HZW5lMSkgLQo+ID4gb25seSBkdXJpbmcgbmV0d29yayBpbnN0YWxscyAt IHRoYXQgSSBhbHNvIGJpc2VjdGVkIGRvd24gdG8gY29tbWl0Cj4gPiA2ZGNlNWFhNTllMGIgKCJQ Q0k6IHhnZW5lOiBVc2UgaW5ib3VuZCByZXNvdXJjZXMgZm9yIHNldHVwIikuIEkgd2FzCj4gPiBo b3BpbmcgdGhhdCB0aGlzIHBhdGNoIHRoYXQgZml4ZWQgdGhlIGlzc3VlIG9uIFN0w6lwaGFuZSdz IFgtR2VuZTIKPiA+IHN5c3RlbSB3b3VsZCBhbHNvIGZpeCBteSBpc3N1ZSwgYnV0IG5vIGx1Y2su IEluIGZhY3QsIGl0IHNlZW1zIHRvIGp1c3QKPiA+IG1ha2VzIGl0IGZhaWwgZGlmZmVyZW50bHku IFJldmVydGluZyBib3RoIHBhdGNoZXMgaXMgcmVxdWlyZWQgdG8gZ2V0IGEKPiA+IHY1LjE3LXJj IGtlcm5lbCB0byBib290Lgo+ID4KPiA+IEkndmUgY29sbGVjdGVkIHRoZSBmb2xsb3dpbmcgbG9n cyAtIGxldCBtZSBrbm93IGlmIGFueXRoaW5nIGVsc2Ugd291bGQKPiA+IGJlIHVzZWZ1bC4KPiA+ Cj4gPiAxKSB2NS4xNy1yYzIrICh1bm1vZGlmaWVkKToKPiA+ICAgIGh0dHA6Ly9kYW5uZi5vcmcv YnVncy9tNDAwLW5vLXJldmVydHMubG9nCj4gPiAgICBOb3RlIHRoYXQgdGhlIG1seDQgZHJpdmVy IGZhaWxzIGluaXRpYWxpemF0aW9uLgo+ID4KPiA+IDIpIHY1LjE3LXJjMissIHcvbyB0aGUgY29t bWl0IHRoYXQgZml4ZWQgU3TDqXBoYW5lJ3Mgc3lzdGVtOgo+ID4gICAgaHR0cDovL2Rhbm5mLm9y Zy9idWdzL200MDAteGdlbmUyLWZpeC1yZXZlcnRlZC5sb2cKPiA+ICAgIE5vdGUgdGhlIG1seDQg TVNJLVggdGltZW91dCwgYW5kIGxhdGVyIHBhbmljLgo+ID4KPiA+IDMpIHY1LjE3LXJjMissIHcv IGJvdGggY29tbWl0cyByZXZlcnRlZCAod29ya3MpCj4gPiAgICBodHRwOi8vZGFubmYub3JnL2J1 Z3MvbTQwMC1ib3RoLXJldmVydGVkLmxvZwo+Cj4gVGhlIHJhbmdlcyBhbmQgZG1hLXJhbmdlcyBh ZGRyZXNzZXMgZG9uJ3QgYXBwZWFyIHRvIG1hdGNoIHVwIHdpdGggYW55Cj4gdXBzdHJlYW0gZHRz IGZpbGVzLiBDYW4geW91IHNlbmQgbWUgdGhlIERUPwoKU3VyZTogaHR0cDovL2Rhbm5mLm9yZy9i dWdzL2ZkdAoKIC1kYW5uCgo+IE90aGVyd2lzZSwgd2UncmUgZ29pbmcgdG8gbmVlZCBzb21lIGRl YnVnZ2luZyBhZGRlZCB0bwo+IHhnZW5lX3BjaWVfc2V0dXBfaWJfcmVnKCkgdG8gc2VlIGlmIHRo ZSByZWdpc3RlciBzZXR1cCBjaGFuZ2VkLiBJIGNhbgo+IGNvbWUgdXAgd2l0aCBzb21ldGhpbmcg bmV4dCB3ZWVrLgo+Cj4gUm9iCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVs QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9s aXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==