Re: Are back to back PCIe BAR allocations supported by Linux?

["Christian König" <christian.koenig@amd.com>]

Ping?

Am 01.04.21 um 14:03 schrieb Christian König:
> Hello everyone,
>
> we recently had a bug report of a system which works fine when a PCIe 
> hotplug device is connected on boot, but fails to initialize if those 
> device are disconnected and then reconnected again.
>
> During investigation I've found that Linux isn't able to assign the 
> BARs of the device correctly while reconnecting. The problem seems to 
> be that the Linux PCI code doesn't seem to use back to back BAR 
> allocations.
>
> Now what's back to back BAR allocation? Let's assume you have two 
> devices with a 256MiB BAR and a 2MiB BAR each behind a common upstream 
> bridge.
>
> The configuration Linux seems to use is the following:
> Device A - 256MiB BAR
> Device A -     2MiB BAR
> Padding     254MiB
> Device B - 256MiB BAR
> Device B -     2MIB BAR
>
> With padding this results in at least 770MiB address space requirement 
> for the common upstream bridge, with alignment this is probably more 
> like 1GiB.
>
> The BIOS on the other hand seems to be capable of configuring the BARs 
> like this:
>
> Device A - 256MiB BAR
> Device A -     2MiB BAR
> Padding     252MiB
> Device B -     2MIB BAR
> Device B - 256MiB BAR
>
> The result is that you only need 768MiB address space for the upstream 
> bridge which then perfectly fits into what is assigned for hotplug here.
>
> Is that already supported by the Linux PCIe code? If yes then how? 
> I've tried to read a bit into the BAR allocation code, but it is kind 
> of hard to understand.
>
> Regards,
> Christian.