Re: [PATCH V13 4/6] PCI: loongson: Improve the MRRS quirk for LS7A

From: Huacai Chen <chenhuacai@gmail.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: "Jiaxun Yang" <jiaxun.yang@flygoat.com>,
	"Huacai Chen" <chenhuacai@loongson.cn>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Lorenzo Pieralisi" <lorenzo.pieralisi@arm.com>,
	"Rob Herring" <robh@kernel.org>,
	"Krzysztof Wilczyński" <kw@linux.com>,
	linux-pci <linux-pci@vger.kernel.org>,
	"Xuefeng Li" <lixuefeng@loongson.cn>,
	"Jingoo Han" <jingoohan1@gmail.com>,
	"Gustavo Pimentel" <gustavo.pimentel@synopsys.com>,
	"Kishon Vijay Abraham I" <kishon@ti.com>,
	"Krzysztof Hałasa" <khalasa@piap.pl>
Subject: Re: [PATCH V13 4/6] PCI: loongson: Improve the MRRS quirk for LS7A
Date: Wed, 8 Jun 2022 16:29:37 +0800	[thread overview]
Message-ID: <CAAhV-H6TYqB8SE=RR9UdDUdw5z6m3buZT=0aGFj+fFoQmyvNqg@mail.gmail.com> (raw)
In-Reply-To: <20220604000712.GA118018@bhelgaas>

Hi, Bjorn, Jiaxun,

On Sat, Jun 4, 2022 at 8:07 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> On Fri, Jun 03, 2022 at 11:57:47PM +0100, Jiaxun Yang wrote:
> > 在2022年6月2日六月 下午5:20，Bjorn Helgaas写道：.
> > >
> > > I'd really like to have a single implementation of whatever quirk
> > > works around this.  I don't think we should have multiple copies
> > > just because we assume some firmware takes care of part of this
> > > for us.
> > >
> > Yeah that was my idea when I was writing the present version of
> > workaround.  However in later LS7A revisions Loongson somehow raised
> > MRRS for several PCIe controllers on chip to 1024 and other ports
> > remains to be 256. Kernel have no way to aware of this change and we
> > can only rely on firmware to set proper value.
>
> That's fine; we need a controller-specific way to find the limit
> (whether it's fixed for all versions or discovered from firmware
> settings or whatever).
>
> My hope is that given that controller-specific value, we can have a
> single quirk that works on keystone, loongson, etc. to enforce the
> limit on all relevant devices.  Some platform firmware might do that
> configuration already, but it's OK if a generic quirk re-does it.
>
> I don't think it's worth having two quirks, one that does the
> configuration, and another that relies on firmware having done it.
I think it is better to let keystone and loongson to both use the
no_inc_mrrs quirk.

Huacai

>
> > I have no idea how Loongson achieved this in hardware. All those
> > PCIe controllers are attached under the same AXI bus should share
> > the same AXI to HyperTransport bridge as AXI slave behind a bus
> > matrix. Perhaps instead of fixing error handling of their AXI
> > protocol implementation they just increased the buffer size in AXI
> > bridge so it can accomplish larger requests at one time.
>
> > >> In keystone’s case it’s likely that their firmware won’t do such thing, so
> > >> their workaround shouldn’t be removed.
> > >> And  no_inc_mrrs should be set for them to prevent device drivers modifying
> > >> MRRS afterwards.
> > >
> > > I have the vague impression that this issue is related to an arm64 AXI
> > > bus property [2] or maybe a DesignWare controller property [3], so
> > > this might affect several PCIe controller drivers.
> >
> > In my understanding it’s likely to be a AXI implementation issue.
>
> I know almost nothing about AXI, but this concerns me because it
> sounds like other drivers could be affected.
>
> Bjorn