* MSI number limit for PCI hotplug under PCI bridge on ARM platform @ 2019-04-01 13:55 Heyi Guo 2019-04-02 5:00 ` Marc Zyngier 0 siblings, 1 reply; 4+ messages in thread From: Heyi Guo @ 2019-04-01 13:55 UTC (permalink / raw) To: linux-kernel Cc: Marc Zyngier, Thomas Gleixner, Jason Cooper, wanghaibin 00208455 Hi folks, In current kernel implementation for ARM platform, all devices under one PCI bridge share a same device ID and the total number of MSI interrupts is fixed at the first time any child device is allocating MSI. However, this may cause failure of allocating MSI if the system supports device hot-plug under the PCI bridge, which is possible for ARM virtual machine with generic pcie-to-pci-bridge and kernel config HOTPLUG_PCI_SHPC enabled. Does it make sense to add support for the above scenario? If it does, any suggestion for how to do that? Thanks, Heyi ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: MSI number limit for PCI hotplug under PCI bridge on ARM platform 2019-04-01 13:55 MSI number limit for PCI hotplug under PCI bridge on ARM platform Heyi Guo @ 2019-04-02 5:00 ` Marc Zyngier 2019-04-02 9:30 ` Heyi Guo 0 siblings, 1 reply; 4+ messages in thread From: Marc Zyngier @ 2019-04-02 5:00 UTC (permalink / raw) To: Heyi Guo; +Cc: linux-kernel, Thomas Gleixner, Jason Cooper, wanghaibin 00208455 On Mon, 01 Apr 2019 14:55:52 +0100, Heyi Guo <guoheyi@huawei.com> wrote: > > Hi folks, > > In current kernel implementation for ARM platform, all devices under > one PCI bridge share a same device ID and the total number of MSI > interrupts is fixed at the first time any child device is allocating > MSI. However, this may cause failure of allocating MSI if the system > supports device hot-plug under the PCI bridge, which is possible for > ARM virtual machine with generic pcie-to-pci-bridge and kernel > config HOTPLUG_PCI_SHPC enabled. > > Does it make sense to add support for the above scenario? If it > does, any suggestion for how to do that? I don't think it makes much sense. You have the flexibility not to add such a broken setup to your guests, and instead have enough pcie ports so that you can always have an exact allocation and no DevID aliasing. The alternative is to dynamically grow the ITT for a given DevID, which cannot be done without unmapping it first. This in turn will result in interrupts being lost while the DevID was unmapped, and they'd need to be pessimistically reinjected. This also involves a substantial amount of data structure repainting, as you're pretty much guaranteed not to be able to reuse the same LPI range. Given that this is arbitrarily self-inflicted, I'm not overly keen on even trying to support this. Thanks, M. -- Jazz is not dead, it just smell funny. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: MSI number limit for PCI hotplug under PCI bridge on ARM platform 2019-04-02 5:00 ` Marc Zyngier @ 2019-04-02 9:30 ` Heyi Guo 2019-04-02 10:07 ` Marc Zyngier 0 siblings, 1 reply; 4+ messages in thread From: Heyi Guo @ 2019-04-02 9:30 UTC (permalink / raw) To: Marc Zyngier Cc: linux-kernel, Thomas Gleixner, Jason Cooper, wanghaibin 00208455 On 2019/4/2 13:00, Marc Zyngier wrote: > On Mon, 01 Apr 2019 14:55:52 +0100, > Heyi Guo <guoheyi@huawei.com> wrote: >> Hi folks, >> >> In current kernel implementation for ARM platform, all devices under >> one PCI bridge share a same device ID and the total number of MSI >> interrupts is fixed at the first time any child device is allocating >> MSI. However, this may cause failure of allocating MSI if the system >> supports device hot-plug under the PCI bridge, which is possible for >> ARM virtual machine with generic pcie-to-pci-bridge and kernel >> config HOTPLUG_PCI_SHPC enabled. >> >> Does it make sense to add support for the above scenario? If it >> does, any suggestion for how to do that? > I don't think it makes much sense. You have the flexibility not to add > such a broken setup to your guests, and instead have enough pcie ports > so that you can always have an exact allocation and no DevID aliasing. > > The alternative is to dynamically grow the ITT for a given DevID, > which cannot be done without unmapping it first. This in turn will > result in interrupts being lost while the DevID was unmapped, and > they'd need to be pessimistically reinjected. This also involves a > substantial amount of data structure repainting, as you're pretty much > guaranteed not to be able to reuse the same LPI range. > > Given that this is arbitrarily self-inflicted, I'm not overly keen on > even trying to support this. SHPC hot plug under PCI-bridge for virtual machine is attracting us for its larger capacity, i.e. one bridge can have up to hot-plugable 31 devices, while each PCIe root port or downstream port can only have one. Anyway, the reason for not supporting this also makes sense to me. Thanks for your advice, Heyi > > Thanks, > > M. > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: MSI number limit for PCI hotplug under PCI bridge on ARM platform 2019-04-02 9:30 ` Heyi Guo @ 2019-04-02 10:07 ` Marc Zyngier 0 siblings, 0 replies; 4+ messages in thread From: Marc Zyngier @ 2019-04-02 10:07 UTC (permalink / raw) To: Heyi Guo; +Cc: linux-kernel, Thomas Gleixner, Jason Cooper, wanghaibin 00208455 On Tue, 02 Apr 2019 10:30:35 +0100, Heyi Guo <guoheyi@huawei.com> wrote: > > > > On 2019/4/2 13:00, Marc Zyngier wrote: > > On Mon, 01 Apr 2019 14:55:52 +0100, > > Heyi Guo <guoheyi@huawei.com> wrote: > >> Hi folks, > >> > >> In current kernel implementation for ARM platform, all devices under > >> one PCI bridge share a same device ID and the total number of MSI > >> interrupts is fixed at the first time any child device is allocating > >> MSI. However, this may cause failure of allocating MSI if the system > >> supports device hot-plug under the PCI bridge, which is possible for > >> ARM virtual machine with generic pcie-to-pci-bridge and kernel > >> config HOTPLUG_PCI_SHPC enabled. > >> > >> Does it make sense to add support for the above scenario? If it > >> does, any suggestion for how to do that? > > I don't think it makes much sense. You have the flexibility not to add > > such a broken setup to your guests, and instead have enough pcie ports > > so that you can always have an exact allocation and no DevID aliasing. > > > > The alternative is to dynamically grow the ITT for a given DevID, > > which cannot be done without unmapping it first. This in turn will > > result in interrupts being lost while the DevID was unmapped, and > > they'd need to be pessimistically reinjected. This also involves a > > substantial amount of data structure repainting, as you're pretty much > > guaranteed not to be able to reuse the same LPI range. > > > > Given that this is arbitrarily self-inflicted, I'm not overly keen on > > even trying to support this. > SHPC hot plug under PCI-bridge for virtual machine is attracting us > for its larger capacity, i.e. one bridge can have up to hot-plugable > 31 devices, while each PCIe root port or downstream port can only > have one. I understand that. But the real reason why people use the PCI bridge model instead of PCIe is because some VM models have a shortage of ECAM config space. This is now solved in QEMU (see [1]), and you get an extra 256MB of ECAM space. It should be enough for everybody. > Anyway, the reason for not supporting this also makes sense to me. Yeah, the ITS was never designed for aliasing bridges. If you can use another method to achieve the same thing, I'd rather see that. Thanks, M. [1] https://events.linuxfoundation.org/wp-content/uploads/2017/12/ARM-virt-3.0-and-Beyond-Towards-a-Better-Scalability-Eric-Auger-Red-Hat.pdf -- Jazz is not dead, it just smell funny. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-04-02 10:07 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-04-01 13:55 MSI number limit for PCI hotplug under PCI bridge on ARM platform Heyi Guo 2019-04-02 5:00 ` Marc Zyngier 2019-04-02 9:30 ` Heyi Guo 2019-04-02 10:07 ` Marc Zyngier
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).