Re: [PATCH] arm64: PCI: Enable SMC conduit

From: Jason Gunthorpe <jgg@nvidia.com>
To: Jon Masters <jcm@redhat.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Will Deacon <will@kernel.org>, Vikram Sethi <vsethi@nvidia.com>,
	Vidya Sagar <vidyas@nvidia.com>,
	Thierry Reding <treding@nvidia.com>,
	Jon Masters <jcm@jonmasters.org>,
	Jeremy Linton <jeremy.linton@arm.com>,
	Mark Rutland <mark.rutland@arm.com>,
	linux-pci@vger.kernel.org, Sudeep Holla <sudeep.holla@arm.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	linux-arm-kernel@lists.infradead.org,
	Eric Brower <ebrower@nvidia.com>,
	Grant.Likely@arm.com
Subject: Re: [PATCH] arm64: PCI: Enable SMC conduit
Date: Fri, 18 Jun 2021 11:05:54 -0300	[thread overview]
Message-ID: <20210618140554.GD1002214@nvidia.com> (raw)
In-Reply-To: <CA+kK7ZijdNERQSauEvAffR7JLbfZ512na2-9cJrU0vFbNnDGwQ@mail.gmail.com>

On Fri, Jun 18, 2021 at 09:21:54AM -0400, Jon Masters wrote:
>    Hi Jason,
>    On Wed, Jun 16, 2021 at 1:38 PM Jason Gunthorpe <[1]jgg@nvidia.com>
>    wrote:
> 
>      On Thu, Mar 25, 2021 at 01:12:31PM +0000, Lorenzo Pieralisi wrote:
>      However, in modern server type systems the PCI config space is often
>      a
>      software fiction being created by firmware throughout the PCI
>      space. This has become necessary as the config space has exploded in
>      size and complexity and PCI devices themselves have become very,
>      very
>      complicated. Not just the config space of single devices, but even
>      bridges and topology are SW created in some cases.
>      HW that is doing this is already trapping the config cycles somehow,
>      presumably with some very ugly way like x86's SMM. Allowing a
>      designed
>      in way to inject software into the config space cycles does sound a
>      lot cleaner and better to me.
> 
>    This is not required. SMM is terrible, indeed. But we don't have to
>    relive it in Arm just because that's [EL3] the easy place to shove
>    things :)

"This is not required"? What does that mean?

>      For instance it may solve other pain points if ARM systems had a
>      cheap
>      way to emulate up a "PCI device" to wrapper around some IP blob on
>      chip. The x86 world has really driven this approach where everything
>      on SOC is PCI discoverable, and it does seem to work well.
>      IMHO SW emulation of config space is an important ingredient to do
>      this.
> 
>    There are certainly ways to build PCI configuration space in a
>    programmable way that does not require software trapping into
>    MM. 

Can you elaborate on what you'd like to see here? Where do you want to
put the software then?

>    I strongly agree with the value of an industry standard approach
>    to this in hardware, particularly if the PCIe vendors would offer
>    this as IP.  In a perfect world, ECAM would simply be an
>    abstraction and never directly map to fixed hardware, thus one
>    could correct defects in behavior in the field. I believe on the
>    x86 side of the house, there is some interesting trapping support
>    in the LPC/IOH already and this is absolutely what Arm should be
>    doing.

AFAIK x86 has HW that traps the read/writes to the ECAM and can
trigger a FW flow to emulate them, maybe in SMM, I don't know the
details. It ceratinly used to be like this when SMM could trap the
config space io read/write registers.

Is that what you want to see for ARM? Is that better than a SMC?

That is alot of special magic hardware to avoid a SMC call...

Jason