Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses

["Daniel P. Berrangé" <berrange@redhat.com>]

On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> On Wed, 20 May 2020 05:47:53 -0400
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> 
> > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > > On Fri, 15 May 2020 12:13:53 +0000
> > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > >   
> > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > > > > wrote:   
> > > > >> 
> > > > >>     
> > > > >>> Will following hack work for you?
> > > > >>> possible permutations
> > > > >>> 1) ACPI hotplug everywhere
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1 
> > > > >>> 
> > > > >>> 2) No hotplug at all
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > >>> 
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter  -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1      
> > > > >> 
> > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > >> following in the order of preference:
> > > > >> 
> > > > >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> > > > >> that Windows does not even show this HW in the “safely remove
> > > > >> HW” option. If we can do this then from OS perspective the GUI
> > > > >> options will be same as what is available with PCIE/q35 - none
> > > > >> of the devices will be hot ejectable if the hot plug option is
> > > > >> turned off from the PCIE slots where devices are plugged into.
> > > > >> I looked at the code. It seems to manipulate ACPI tables of
> > > > >> the empty slots of the root bus where no devices are attached
> > > > >> (see comment "/* add hotplug slots for non present devices */
> > > > >> “). For cold plugged bridges, it recurses down to scan the
> > > > >> slots of the bridge. Is it possible to disable hot plug for
> > > > >> the slot to which the bridge is attached?    
> > > > > 
> > > > > I don't think it's possible to have per slot hotplug on
> > > > > conventional PCI hardware. it's per bridge property.    
> > > > 
> > > > We add the AMLs per empty slot though. When the pic bridge is
> > > > attached, we do nothing, just recurse into the bridge slots. That
> > > > is what I was asking, if it was possible to just disable the AMLs
> > > > or use some tricks to say that this particular slot is not
> > > > hotpluggable. I am not sure why Windows is trying to eject the
> > > > PCI bridge and failing. Maybe something related to this comment?
> > > > 
> > > > 
> > > > /* When hotplug for bridges is enabled, bridges are
> > > >                               
> > > >          * described in ACPI separately (see build_pci_bus_end).
> > > >                                       
> > > >          * In this case they aren't themselves hot-pluggable.
> > > >                                       
> > > >          * Hotplugged bridges *are* hot-pluggable.
> > > > */  
> > > 
> > > thinking some more on this topic, it seems that with ACPI hotplug we
> > > already have implicit non-hotpluggble slot (slot with bridge) while
> > > the rest are staying hotpluggable.
> > > 
> > > So my question is: if it's acceptable to add
> > > 'PCIDevice::hotpluggable" property to all PCI devices so that user
> > > / libvirt could set it to false in case they do not want
> > > coldplugged device be considered as hotpluggable? (this way other
> > > devices could be treated the same way as bridges)
> > > 
> > > [...]  
> > 
> > 
> > I think Julia already posted a patch adding this to downstream pcie
> > bridges. Adding this to pci slots sounds like a reasonable thing.
> Question was more about external interface, were we do not have ports
> as separate devices with conventional PCI. The only knob we have is a
> a PCI device, where we have a property to turn on/off hotplug. ex:
>   -device e1000,hotpluggable=off
> and if libvirt would be able to use it

Libvirt can certainly use a property that is settable per-device,
instead of against the pcie-root-port.  In fact the application
that requested hotplug control (KubeVirt) would strongly prefer
if this was doable per-device, because they really dislike the
idea of having to deal with pcie-root-port devices.

So if you added it per-device, libvirt would support both the
per-device option, and the pcie-root-port option.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|