Re: [PATCH v9 00/26] PCI: Allow BAR movement during boot and hotplug

From: Mika Westerberg <mika.westerberg@linux.intel.com>
To: Lukas Wunner <lukas@wunner.de>
Cc: "Bjorn Helgaas" <helgaas@kernel.org>,
	"Sergei Miroshnichenko" <s.miroshnichenko@yadro.com>,
	linux-pci@vger.kernel.org, "Stefan Roese" <sr@denx.de>,
	"Andy Lavr" <andy.lavr@gmail.com>,
	"Christian König" <christian.koenig@amd.com>,
	"David Laight" <David.Laight@aculab.com>,
	"Rajat Jain" <rajatja@google.com>,
	linux@yadro.com, "Yehezkel Bernat" <YehezkelShB@gmail.com>,
	"Mario Limonciello" <mario.limonciello@dell.com>,
	"Christian Kellner" <christian@kellner.me>
Subject: Re: [PATCH v9 00/26] PCI: Allow BAR movement during boot and hotplug
Date: Mon, 1 Feb 2021 14:55:23 +0200	[thread overview]
Message-ID: <20210201125523.GN2542@lahna.fi.intel.com> (raw)
In-Reply-To: <20210128203929.GB6613@wunner.de>

Hi,

On Thu, Jan 28, 2021 at 09:39:29PM +0100, Lukas Wunner wrote:
> On Thu, Jan 28, 2021 at 08:53:16AM -0600, Bjorn Helgaas wrote:
> > On Fri, Dec 18, 2020 at 08:39:45PM +0300, Sergei Miroshnichenko wrote:
> > > Currently PCI hotplug works on top of resources which are usually reserved:
> > > by BIOS, bootloader, firmware, or by the kernel (pci=hpmemsize=XM). These
> > > resources are gaps in the address space where BARs of new devices may fit,
> > > and extra bus number per port, so bridges can be hot-added. This series aim
> > > the BARs problem: it shows the kernel how to redistribute them on the run,
> > > so the hotplug becomes predictable and cross-platform. A follow-up patchset
> > > will propose a solution for bus numbers. And another one -- for the powerpc
> > > arch-specific problems.
> > 
> > I can certainly see scenarios where this functionality will be useful,
> > but the series currently doesn't mention bug reports that it fixes.  I
> > suspect there *are* some related bug reports, e.g., for Thunderbolt
> > hotplug.  We should dig them up, include pointers to them, and get the
> > reporters to test the series and provide feedback.
> 
> In case it helps, an earlier version of the series was referenced
> in this LWN article more than 2 years ago (scroll down to the
> "Moving BARs" section at the end of the article):
> 
> https://lwn.net/Articles/767885/
> 
> The article provides some context:  Specifically, macOS is capable
> of resizing and moving BARs, so this series sort of helps us catch
> up with the competition.
> 
> With Thunderbolt, this series is particularly useful if
> (a) PCIe cards are hot-added with large BARs (such as GPUs) and/or
> (b) the Thunderbolt daisy-chain is very long.
> 
> Thunderbolt is essentially a cascade of nested hotplug ports,
> so if more and more devices are added, it's easy to see that
> the top-level hotplug port's BAR window may run out of space.
> 
> My understanding is that Sergei's use case doesn't involve
> Thunderbolt at all but rather hotplugging of GPUs and network
> cards in PowerPC servers in a datacenter, which may have the
> same kinds of issues.
> 
> I intended to review and test this iteration of the series more
> closely, but haven't been able to carve out the required time.
> I'm adding some Thunderbolt folks to cc in the hope that they
> can at least test the series on their development branch.
> Getting this upstreamed should really be in the best interest
> of Intel and other promulgators of Thunderbolt.

Sure. It seems that this series was submitted in December so probably
not applicable to the pci.git/next anymore. Anyways, I can give it a try
on a TBT capable system if someone tells me what exactly to test ;-)
Probably at least that the existing functionality still works but
something else maybe too?