* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 10:16 [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses Ani Sinha
@ 2020-04-28 10:20 ` Ani Sinha
2020-04-28 16:05 ` Michael S. Tsirkin
2020-05-11 18:53 ` Igor Mammedov
2 siblings, 0 replies; 52+ messages in thread
From: Ani Sinha @ 2020-04-28 10:20 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Michael S. Tsirkin, Julia Suvorova,
Aleksandar Markovic, Laine Stump, Paolo Bonzini, ani,
Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
+julie
+ laine
Rebased patch to latest Qemu master. Only minimal changes.
> On Apr 28, 2020, at 3:46 PM, Ani Sinha <ani.sinha@nutanix.com> wrote:
>
> A new option "use_acpi_unplug" is introduced for PIIX which will
> selectively only disable hot unplugging of both hot plugged and
> cold plugged PCI devices on non-root PCI buses. This will prevent
> hot unplugging of devices from Windows based guests from system
> tray but will not prevent devices from being hot plugged into the
> guest.
>
> It has been tested on Windows guests.
>
> Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
> ---
> hw/acpi/piix4.c | 3 +++
> hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> 2 files changed, 29 insertions(+), 14 deletions(-)
>
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 964d6f5..59fa707 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
>
> AcpiPciHpState acpi_pci_hotplug;
> bool use_acpi_pci_hotplug;
> + bool use_acpi_unplug;
>
> uint8_t disable_s3;
> uint8_t disable_s4;
> @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> use_acpi_pci_hotplug, true),
> + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> + use_acpi_unplug, true),
> DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> acpi_memory_hotplug.is_enabled, true),
> DEFINE_PROP_END_OF_LIST(),
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 23c77ee..71b3ac3 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> bool s3_disabled;
> bool s4_disabled;
> bool pcihp_bridge_en;
> + bool pcihup_bridge_en;
> uint8_t s4_val;
> AcpiFadtData fadt;
> uint16_t cpu_hp_io_base;
> @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
> pm->pcihp_bridge_en =
> object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> NULL);
> + pm->pcihup_bridge_en =
> + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> + NULL);
> }
>
> static void acpi_get_misc_info(AcpiMiscInfo *info)
> @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml *method, int slot)
> }
>
> static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> - bool pcihp_bridge_en)
> + bool pcihp_bridge_en,
> + bool pcihup_bridge_en)
> {
> Aml *dev, *notify_method = NULL, *method;
> QObject *bsel;
> @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> - aml_append(method,
> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> - );
> - aml_append(dev, method);
> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> + aml_append(method,
> + aml_call2("PCEJ", aml_name("BSEL"),
> + aml_name("_SUN"))
> + );
> + aml_append(dev, method);
> + }
> aml_append(parent_scope, dev);
>
> build_append_pcihp_notify_entry(notify_method, slot);
> @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> /* add _SUN/_EJ0 to make slot hotpluggable */
> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
>
> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> - aml_append(method,
> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> - );
> - aml_append(dev, method);
> -
> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> + aml_append(method,
> + aml_call2("PCEJ", aml_name("BSEL"),
> + aml_name("_SUN"))
> + );
> + aml_append(dev, method);
> + }
> if (bsel) {
> build_append_pcihp_notify_entry(notify_method, slot);
> }
> @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> */
> PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
>
> - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> + pcihup_bridge_en);
> }
> /* slot descriptor has been composed, add it into parent context */
> aml_append(parent_scope, dev);
> @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> if (bus) {
> Aml *scope = aml_scope("PCI0");
> /* Scan all PCI buses. Generate tables to support hotplug. */
> - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> + pm->pcihup_bridge_en);
>
> if (TPM_IS_TIS_ISA(tpm)) {
> if (misc->tpm_version == TPM_VERSION_2_0) {
> --
> 1.9.4
>
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 10:16 [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses Ani Sinha
2020-04-28 10:20 ` Ani Sinha
@ 2020-04-28 16:05 ` Michael S. Tsirkin
2020-04-28 16:09 ` Ani Sinha
2020-04-28 16:28 ` Daniel P. Berrangé
2020-05-11 18:53 ` Igor Mammedov
2 siblings, 2 replies; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-28 16:05 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
ani, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> A new option "use_acpi_unplug" is introduced for PIIX which will
> selectively only disable hot unplugging of both hot plugged and
> cold plugged PCI devices on non-root PCI buses. This will prevent
> hot unplugging of devices from Windows based guests from system
> tray but will not prevent devices from being hot plugged into the
> guest.
>
> It has been tested on Windows guests.
>
> Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
It's still a non starter until we find something similar for PCIE and
SHPC. Do guests check command status? Can some unplug commands fail?
> ---
> hw/acpi/piix4.c | 3 +++
> hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> 2 files changed, 29 insertions(+), 14 deletions(-)
>
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 964d6f5..59fa707 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
>
> AcpiPciHpState acpi_pci_hotplug;
> bool use_acpi_pci_hotplug;
> + bool use_acpi_unplug;
>
> uint8_t disable_s3;
> uint8_t disable_s4;
> @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> use_acpi_pci_hotplug, true),
> + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> + use_acpi_unplug, true),
> DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> acpi_memory_hotplug.is_enabled, true),
> DEFINE_PROP_END_OF_LIST(),
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 23c77ee..71b3ac3 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> bool s3_disabled;
> bool s4_disabled;
> bool pcihp_bridge_en;
> + bool pcihup_bridge_en;
> uint8_t s4_val;
> AcpiFadtData fadt;
> uint16_t cpu_hp_io_base;
> @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
> pm->pcihp_bridge_en =
> object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> NULL);
> + pm->pcihup_bridge_en =
> + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> + NULL);
> }
>
> static void acpi_get_misc_info(AcpiMiscInfo *info)
> @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml *method, int slot)
> }
>
> static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> - bool pcihp_bridge_en)
> + bool pcihp_bridge_en,
> + bool pcihup_bridge_en)
> {
> Aml *dev, *notify_method = NULL, *method;
> QObject *bsel;
> @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> - aml_append(method,
> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> - );
> - aml_append(dev, method);
> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> + aml_append(method,
> + aml_call2("PCEJ", aml_name("BSEL"),
> + aml_name("_SUN"))
> + );
> + aml_append(dev, method);
> + }
> aml_append(parent_scope, dev);
>
> build_append_pcihp_notify_entry(notify_method, slot);
> @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> /* add _SUN/_EJ0 to make slot hotpluggable */
> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
>
> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> - aml_append(method,
> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> - );
> - aml_append(dev, method);
> -
> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> + aml_append(method,
> + aml_call2("PCEJ", aml_name("BSEL"),
> + aml_name("_SUN"))
> + );
> + aml_append(dev, method);
> + }
> if (bsel) {
> build_append_pcihp_notify_entry(notify_method, slot);
> }
> @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> */
> PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
>
> - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> + pcihup_bridge_en);
> }
> /* slot descriptor has been composed, add it into parent context */
> aml_append(parent_scope, dev);
> @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> if (bus) {
> Aml *scope = aml_scope("PCI0");
> /* Scan all PCI buses. Generate tables to support hotplug. */
> - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> + pm->pcihup_bridge_en);
>
> if (TPM_IS_TIS_ISA(tpm)) {
> if (misc->tpm_version == TPM_VERSION_2_0) {
> --
> 1.9.4
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 16:05 ` Michael S. Tsirkin
@ 2020-04-28 16:09 ` Ani Sinha
2020-04-28 16:21 ` Michael S. Tsirkin
2020-04-28 16:28 ` Daniel P. Berrangé
1 sibling, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-04-28 16:09 UTC (permalink / raw)
To: Ani Sinha, Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
[-- Attachment #1: Type: text/plain, Size: 5478 bytes --]
Ani
On Apr 28, 2020, 21:35 +0530, Michael S. Tsirkin <mst@redhat.com>, wrote:
> On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> > A new option "use_acpi_unplug" is introduced for PIIX which will
> > selectively only disable hot unplugging of both hot plugged and
> > cold plugged PCI devices on non-root PCI buses. This will prevent
> > hot unplugging of devices from Windows based guests from system
> > tray but will not prevent devices from being hot plugged into the
> > guest.
> >
> > It has been tested on Windows guests.
> >
> > Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
>
> It's still a non starter until we find something similar for PCIE and
> SHPC. Do guests check command status? Can some unplug commands fail?
Ok I give up! I thought we debated this on the other thread.
>
>
> > ---
> > hw/acpi/piix4.c | 3 +++
> > hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> > 2 files changed, 29 insertions(+), 14 deletions(-)
> >
> > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > index 964d6f5..59fa707 100644
> > --- a/hw/acpi/piix4.c
> > +++ b/hw/acpi/piix4.c
> > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> >
> > AcpiPciHpState acpi_pci_hotplug;
> > bool use_acpi_pci_hotplug;
> > + bool use_acpi_unplug;
> >
> > uint8_t disable_s3;
> > uint8_t disable_s4;
> > @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> > use_acpi_pci_hotplug, true),
> > + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> > + use_acpi_unplug, true),
> > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > acpi_memory_hotplug.is_enabled, true),
> > DEFINE_PROP_END_OF_LIST(),
> > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > index 23c77ee..71b3ac3 100644
> > --- a/hw/i386/acpi-build.c
> > +++ b/hw/i386/acpi-build.c
> > @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> > bool s3_disabled;
> > bool s4_disabled;
> > bool pcihp_bridge_en;
> > + bool pcihup_bridge_en;
> > uint8_t s4_val;
> > AcpiFadtData fadt;
> > uint16_t cpu_hp_io_base;
> > @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
> > pm->pcihp_bridge_en =
> > object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> > NULL);
> > + pm->pcihup_bridge_en =
> > + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> > + NULL);
> > }
> >
> > static void acpi_get_misc_info(AcpiMiscInfo *info)
> > @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml *method, int slot)
> > }
> >
> > static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > - bool pcihp_bridge_en)
> > + bool pcihp_bridge_en,
> > + bool pcihup_bridge_en)
> > {
> > Aml *dev, *notify_method = NULL, *method;
> > QObject *bsel;
> > @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > - aml_append(method,
> > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > - );
> > - aml_append(dev, method);
> > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > + aml_append(method,
> > + aml_call2("PCEJ", aml_name("BSEL"),
> > + aml_name("_SUN"))
> > + );
> > + aml_append(dev, method);
> > + }
> > aml_append(parent_scope, dev);
> >
> > build_append_pcihp_notify_entry(notify_method, slot);
> > @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > /* add _SUN/_EJ0 to make slot hotpluggable */
> > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> >
> > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > - aml_append(method,
> > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > - );
> > - aml_append(dev, method);
> > -
> > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > + aml_append(method,
> > + aml_call2("PCEJ", aml_name("BSEL"),
> > + aml_name("_SUN"))
> > + );
> > + aml_append(dev, method);
> > + }
> > if (bsel) {
> > build_append_pcihp_notify_entry(notify_method, slot);
> > }
> > @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > */
> > PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> >
> > - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> > + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> > + pcihup_bridge_en);
> > }
> > /* slot descriptor has been composed, add it into parent context */
> > aml_append(parent_scope, dev);
> > @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> > if (bus) {
> > Aml *scope = aml_scope("PCI0");
> > /* Scan all PCI buses. Generate tables to support hotplug. */
> > - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> > + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> > + pm->pcihup_bridge_en);
> >
> > if (TPM_IS_TIS_ISA(tpm)) {
> > if (misc->tpm_version == TPM_VERSION_2_0) {
> > --
> > 1.9.4
>
[-- Attachment #2: Type: text/html, Size: 6659 bytes --]
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 16:09 ` Ani Sinha
@ 2020-04-28 16:21 ` Michael S. Tsirkin
2020-04-28 16:40 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-28 16:21 UTC (permalink / raw)
To: Ani Sinha
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Paolo Bonzini, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, Apr 28, 2020 at 09:39:16PM +0530, Ani Sinha wrote:
>
> Ani
> On Apr 28, 2020, 21:35 +0530, Michael S. Tsirkin <mst@redhat.com>, wrote:
>
> On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
>
> A new option "use_acpi_unplug" is introduced for PIIX which will
> selectively only disable hot unplugging of both hot plugged and
> cold plugged PCI devices on non-root PCI buses. This will prevent
> hot unplugging of devices from Windows based guests from system
> tray but will not prevent devices from being hot plugged into the
> guest.
>
> It has been tested on Windows guests.
>
> Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
>
>
> It's still a non starter until we find something similar for PCIE and
> SHPC. Do guests check command status? Can some unplug commands fail?
>
>
> Ok I give up! I thought we debated this on the other thread.
Sorry to hear that.
I'd rather you didn't, and worked on a solution that works for everyone.
Pushing back on merging code is unfortunately the only mechanism
maintainers have to make sure features are complete and
orthogonal to each other, so I'm not sure I can help otherwise.
>
>
>
>
> ---
> hw/acpi/piix4.c | 3 +++
> hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> 2 files changed, 29 insertions(+), 14 deletions(-)
>
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 964d6f5..59fa707 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
>
> AcpiPciHpState acpi_pci_hotplug;
> bool use_acpi_pci_hotplug;
> + bool use_acpi_unplug;
>
> uint8_t disable_s3;
> uint8_t disable_s4;
> @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> use_acpi_pci_hotplug, true),
> + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> + use_acpi_unplug, true),
> DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> acpi_memory_hotplug.is_enabled, true),
> DEFINE_PROP_END_OF_LIST(),
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 23c77ee..71b3ac3 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> bool s3_disabled;
> bool s4_disabled;
> bool pcihp_bridge_en;
> + bool pcihup_bridge_en;
> uint8_t s4_val;
> AcpiFadtData fadt;
> uint16_t cpu_hp_io_base;
> @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine,
> AcpiPmInfo *pm)
> pm->pcihp_bridge_en =
> object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> NULL);
> + pm->pcihup_bridge_en =
> + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> + NULL);
> }
>
> static void acpi_get_misc_info(AcpiMiscInfo *info)
> @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml
> *method, int slot)
> }
>
> static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus
> *bus,
> - bool pcihp_bridge_en)
> + bool pcihp_bridge_en,
> + bool pcihup_bridge_en)
> {
> Aml *dev, *notify_method = NULL, *method;
> QObject *bsel;
> @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml
> *parent_scope, PCIBus *bus,
> dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> - aml_append(method,
> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> - );
> - aml_append(dev, method);
> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> + aml_append(method,
> + aml_call2("PCEJ", aml_name("BSEL"),
> + aml_name("_SUN"))
> + );
> + aml_append(dev, method);
> + }
> aml_append(parent_scope, dev);
>
> build_append_pcihp_notify_entry(notify_method, slot);
> @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml
> *parent_scope, PCIBus *bus,
> /* add _SUN/_EJ0 to make slot hotpluggable */
> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
>
> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> - aml_append(method,
> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> - );
> - aml_append(dev, method);
> -
> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> + aml_append(method,
> + aml_call2("PCEJ", aml_name("BSEL"),
> + aml_name("_SUN"))
> + );
> + aml_append(dev, method);
> + }
> if (bsel) {
> build_append_pcihp_notify_entry(notify_method, slot);
> }
> @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml
> *parent_scope, PCIBus *bus,
> */
> PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
>
> - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> + pcihup_bridge_en);
> }
> /* slot descriptor has been composed, add it into parent context */
> aml_append(parent_scope, dev);
> @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker
> *linker,
> if (bus) {
> Aml *scope = aml_scope("PCI0");
> /* Scan all PCI buses. Generate tables to support hotplug. */
> - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> + pm->pcihup_bridge_en);
>
> if (TPM_IS_TIS_ISA(tpm)) {
> if (misc->tpm_version == TPM_VERSION_2_0) {
> --
> 1.9.4
>
>
>
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 16:21 ` Michael S. Tsirkin
@ 2020-04-28 16:40 ` Ani Sinha
2020-04-28 20:45 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-04-28 16:40 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Paolo Bonzini, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
[-- Attachment #1: Type: text/plain, Size: 7134 bytes --]
On Tue, Apr 28, 2020 at 9:51 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> On Tue, Apr 28, 2020 at 09:39:16PM +0530, Ani Sinha wrote:
> >
> > Ani
> > On Apr 28, 2020, 21:35 +0530, Michael S. Tsirkin <mst@redhat.com>,
> wrote:
> >
> > On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> >
> > A new option "use_acpi_unplug" is introduced for PIIX which will
> > selectively only disable hot unplugging of both hot plugged and
> > cold plugged PCI devices on non-root PCI buses. This will prevent
> > hot unplugging of devices from Windows based guests from system
> > tray but will not prevent devices from being hot plugged into the
> > guest.
> >
> > It has been tested on Windows guests.
> >
> > Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
> >
> >
> > It's still a non starter until we find something similar for PCIE and
> > SHPC. Do guests check command status? Can some unplug commands fail?
> >
> >
> > Ok I give up! I thought we debated this on the other thread.
>
> Sorry to hear that.
> I'd rather you didn't, and worked on a solution that works for everyone.
That is extremely hard for one person to do, without inputs and ideas from
the community. Satisfying the entire world requires lot of time and energy
investment, not to mention a broad expertise in multiple technologies.
> Pushing back on merging code is unfortunately the only mechanism
> maintainers have to make sure features are complete and
> orthogonal to each other, so I'm not sure I can help otherwise.
>
> >
> >
> >
> >
> > ---
> > hw/acpi/piix4.c | 3 +++
> > hw/i386/acpi-build.c | 40
> ++++++++++++++++++++++++++--------------
> > 2 files changed, 29 insertions(+), 14 deletions(-)
> >
> > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > index 964d6f5..59fa707 100644
> > --- a/hw/acpi/piix4.c
> > +++ b/hw/acpi/piix4.c
> > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> >
> > AcpiPciHpState acpi_pci_hotplug;
> > bool use_acpi_pci_hotplug;
> > + bool use_acpi_unplug;
> >
> > uint8_t disable_s3;
> > uint8_t disable_s4;
> > @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support",
> PIIX4PMState,
> > use_acpi_pci_hotplug, true),
> > + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge",
> PIIX4PMState,
> > + use_acpi_unplug, true),
> > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > acpi_memory_hotplug.is_enabled, true),
> > DEFINE_PROP_END_OF_LIST(),
> > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > index 23c77ee..71b3ac3 100644
> > --- a/hw/i386/acpi-build.c
> > +++ b/hw/i386/acpi-build.c
> > @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> > bool s3_disabled;
> > bool s4_disabled;
> > bool pcihp_bridge_en;
> > + bool pcihup_bridge_en;
> > uint8_t s4_val;
> > AcpiFadtData fadt;
> > uint16_t cpu_hp_io_base;
> > @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState
> *machine,
> > AcpiPmInfo *pm)
> > pm->pcihp_bridge_en =
> > object_property_get_bool(obj,
> "acpi-pci-hotplug-with-bridge-support",
> > NULL);
> > + pm->pcihup_bridge_en =
> > + object_property_get_bool(obj,
> "acpi-pci-hotunplug-enable-bridge",
> > + NULL);
> > }
> >
> > static void acpi_get_misc_info(AcpiMiscInfo *info)
> > @@ -451,7 +455,8 @@ static void
> build_append_pcihp_notify_entry(Aml
> > *method, int slot)
> > }
> >
> > static void build_append_pci_bus_devices(Aml *parent_scope,
> PCIBus
> > *bus,
> > - bool pcihp_bridge_en)
> > + bool pcihp_bridge_en,
> > + bool pcihup_bridge_en)
> > {
> > Aml *dev, *notify_method = NULL, *method;
> > QObject *bsel;
> > @@ -479,11 +484,14 @@ static void
> build_append_pci_bus_devices(Aml
> > *parent_scope, PCIBus *bus,
> > dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > - aml_append(method,
> > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > - );
> > - aml_append(dev, method);
> > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > + aml_append(method,
> > + aml_call2("PCEJ", aml_name("BSEL"),
> > + aml_name("_SUN"))
> > + );
> > + aml_append(dev, method);
> > + }
> > aml_append(parent_scope, dev);
> >
> > build_append_pcihp_notify_entry(notify_method, slot);
> > @@ -537,12 +545,14 @@ static void
> build_append_pci_bus_devices(Aml
> > *parent_scope, PCIBus *bus,
> > /* add _SUN/_EJ0 to make slot hotpluggable */
> > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> >
> > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > - aml_append(method,
> > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > - );
> > - aml_append(dev, method);
> > -
> > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > + aml_append(method,
> > + aml_call2("PCEJ", aml_name("BSEL"),
> > + aml_name("_SUN"))
> > + );
> > + aml_append(dev, method);
> > + }
> > if (bsel) {
> > build_append_pcihp_notify_entry(notify_method, slot);
> > }
> > @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml
> > *parent_scope, PCIBus *bus,
> > */
> > PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> >
> > - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> > + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> > + pcihup_bridge_en);
> > }
> > /* slot descriptor has been composed, add it into parent context
> */
> > aml_append(parent_scope, dev);
> > @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker
> > *linker,
> > if (bus) {
> > Aml *scope = aml_scope("PCI0");
> > /* Scan all PCI buses. Generate tables to support hotplug. */
> > - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> > + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> > + pm->pcihup_bridge_en);
> >
> > if (TPM_IS_TIS_ISA(tpm)) {
> > if (misc->tpm_version == TPM_VERSION_2_0) {
> > --
> > 1.9.4
> >
> >
> >
>
>
[-- Attachment #2: Type: text/html, Size: 9638 bytes --]
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 16:40 ` Ani Sinha
@ 2020-04-28 20:45 ` Michael S. Tsirkin
2020-04-29 0:58 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-28 20:45 UTC (permalink / raw)
To: Ani Sinha
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Paolo Bonzini, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, Apr 28, 2020 at 10:10:18PM +0530, Ani Sinha wrote:
>
>
> On Tue, Apr 28, 2020 at 9:51 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Tue, Apr 28, 2020 at 09:39:16PM +0530, Ani Sinha wrote:
> >
> > Ani
> > On Apr 28, 2020, 21:35 +0530, Michael S. Tsirkin <mst@redhat.com>, wrote:
> >
> > On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> >
> > A new option "use_acpi_unplug" is introduced for PIIX which will
> > selectively only disable hot unplugging of both hot plugged and
> > cold plugged PCI devices on non-root PCI buses. This will prevent
> > hot unplugging of devices from Windows based guests from system
> > tray but will not prevent devices from being hot plugged into the
> > guest.
> >
> > It has been tested on Windows guests.
> >
> > Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
> >
> >
> > It's still a non starter until we find something similar for PCIE and
> > SHPC. Do guests check command status? Can some unplug commands fail?
> >
> >
> > Ok I give up! I thought we debated this on the other thread.
>
> Sorry to hear that.
> I'd rather you didn't, and worked on a solution that works for everyone.
>
>
> That is extremely hard for one person to do, without inputs and ideas from the
> community.
What kind of input are you looking for?
> Satisfying the entire world requires lot of time and energy
> investment, not to mention a broad expertise in multiple technologies.
>
>
>
> Pushing back on merging code is unfortunately the only mechanism
> maintainers have to make sure features are complete and
> orthogonal to each other, so I'm not sure I can help otherwise.
>
> >
> >
> >
> >
> > ---
> > hw/acpi/piix4.c | 3 +++
> > hw/i386/acpi-build.c | 40
> ++++++++++++++++++++++++++--------------
> > 2 files changed, 29 insertions(+), 14 deletions(-)
> >
> > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > index 964d6f5..59fa707 100644
> > --- a/hw/acpi/piix4.c
> > +++ b/hw/acpi/piix4.c
> > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> >
> > AcpiPciHpState acpi_pci_hotplug;
> > bool use_acpi_pci_hotplug;
> > + bool use_acpi_unplug;
> >
> > uint8_t disable_s3;
> > uint8_t disable_s4;
> > @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support",
> PIIX4PMState,
> > use_acpi_pci_hotplug, true),
> > + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge",
> PIIX4PMState,
> > + use_acpi_unplug, true),
> > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > acpi_memory_hotplug.is_enabled, true),
> > DEFINE_PROP_END_OF_LIST(),
> > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > index 23c77ee..71b3ac3 100644
> > --- a/hw/i386/acpi-build.c
> > +++ b/hw/i386/acpi-build.c
> > @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> > bool s3_disabled;
> > bool s4_disabled;
> > bool pcihp_bridge_en;
> > + bool pcihup_bridge_en;
> > uint8_t s4_val;
> > AcpiFadtData fadt;
> > uint16_t cpu_hp_io_base;
> > @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState
> *machine,
> > AcpiPmInfo *pm)
> > pm->pcihp_bridge_en =
> > object_property_get_bool(obj,
> "acpi-pci-hotplug-with-bridge-support",
> > NULL);
> > + pm->pcihup_bridge_en =
> > + object_property_get_bool(obj,
> "acpi-pci-hotunplug-enable-bridge",
> > + NULL);
> > }
> >
> > static void acpi_get_misc_info(AcpiMiscInfo *info)
> > @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry
> (Aml
> > *method, int slot)
> > }
> >
> > static void build_append_pci_bus_devices(Aml *parent_scope,
> PCIBus
> > *bus,
> > - bool pcihp_bridge_en)
> > + bool pcihp_bridge_en,
> > + bool pcihup_bridge_en)
> > {
> > Aml *dev, *notify_method = NULL, *method;
> > QObject *bsel;
> > @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices
> (Aml
> > *parent_scope, PCIBus *bus,
> > dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > - aml_append(method,
> > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > - );
> > - aml_append(dev, method);
> > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > + aml_append(method,
> > + aml_call2("PCEJ", aml_name("BSEL"),
> > + aml_name("_SUN"))
> > + );
> > + aml_append(dev, method);
> > + }
> > aml_append(parent_scope, dev);
> >
> > build_append_pcihp_notify_entry(notify_method, slot);
> > @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices
> (Aml
> > *parent_scope, PCIBus *bus,
> > /* add _SUN/_EJ0 to make slot hotpluggable */
> > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> >
> > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > - aml_append(method,
> > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > - );
> > - aml_append(dev, method);
> > -
> > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > + aml_append(method,
> > + aml_call2("PCEJ", aml_name("BSEL"),
> > + aml_name("_SUN"))
> > + );
> > + aml_append(dev, method);
> > + }
> > if (bsel) {
> > build_append_pcihp_notify_entry(notify_method, slot);
> > }
> > @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml
> > *parent_scope, PCIBus *bus,
> > */
> > PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> >
> > - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> > + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> > + pcihup_bridge_en);
> > }
> > /* slot descriptor has been composed, add it into parent context
> */
> > aml_append(parent_scope, dev);
> > @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker
> > *linker,
> > if (bus) {
> > Aml *scope = aml_scope("PCI0");
> > /* Scan all PCI buses. Generate tables to support hotplug. */
> > - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> > + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> > + pm->pcihup_bridge_en);
> >
> > if (TPM_IS_TIS_ISA(tpm)) {
> > if (misc->tpm_version == TPM_VERSION_2_0) {
> > --
> > 1.9.4
> >
> >
> >
>
>
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 20:45 ` Michael S. Tsirkin
@ 2020-04-29 0:58 ` Ani Sinha
2020-04-29 5:28 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 0:58 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Paolo Bonzini, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
[-- Attachment #1: Type: text/plain, Size: 9219 bytes --]
On Wed, Apr 29, 2020 at 2:15 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> On Tue, Apr 28, 2020 at 10:10:18PM +0530, Ani Sinha wrote:
> >
> >
> > On Tue, Apr 28, 2020 at 9:51 PM Michael S. Tsirkin <mst@redhat.com>
> wrote:
> >
> > On Tue, Apr 28, 2020 at 09:39:16PM +0530, Ani Sinha wrote:
> > >
> > > Ani
> > > On Apr 28, 2020, 21:35 +0530, Michael S. Tsirkin <mst@redhat.com>,
> wrote:
> > >
> > > On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> > >
> > > A new option "use_acpi_unplug" is introduced for PIIX
> which will
> > > selectively only disable hot unplugging of both hot
> plugged and
> > > cold plugged PCI devices on non-root PCI buses. This will
> prevent
> > > hot unplugging of devices from Windows based guests from
> system
> > > tray but will not prevent devices from being hot plugged
> into the
> > > guest.
> > >
> > > It has been tested on Windows guests.
> > >
> > > Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
> > >
> > >
> > > It's still a non starter until we find something similar for
> PCIE and
> > > SHPC. Do guests check command status? Can some unplug commands
> fail?
> > >
> > >
> > > Ok I give up! I thought we debated this on the other thread.
> >
> > Sorry to hear that.
> > I'd rather you didn't, and worked on a solution that works for
> everyone.
> >
> >
> > That is extremely hard for one person to do, without inputs and ideas
> from the
> > community.
>
> What kind of input are you looking for?
Well there were several discussions in the other thread around how PCIE
behaves and how we can't change the slot features without a HW reset. Those
were useful inputs.
The approach you are taking as a maintainer is very discouraging. All I
have gotten from you is negativity and push back. There has been no other
engagement from you. If you expect one person to fix every use case, that
is an unrealistic expectation IMHO. Even if I could come up with a solution
for every case, testing every use case is a huge investment in time and
effort. No one outside the big distros will be motivated to do that. So
involvement from outside the distro community will be limited to minor
changes, bug fixes and easy code reworks.
My 2 cents.
>
> > Satisfying the entire world requires lot of time and energy
> > investment, not to mention a broad expertise in multiple technologies.
> >
> >
> >
> > Pushing back on merging code is unfortunately the only mechanism
> > maintainers have to make sure features are complete and
> > orthogonal to each other, so I'm not sure I can help otherwise.
> >
> > >
> > >
> > >
> > >
> > > ---
> > > hw/acpi/piix4.c | 3 +++
> > > hw/i386/acpi-build.c | 40
> > ++++++++++++++++++++++++++--------------
> > > 2 files changed, 29 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > > index 964d6f5..59fa707 100644
> > > --- a/hw/acpi/piix4.c
> > > +++ b/hw/acpi/piix4.c
> > > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> > >
> > > AcpiPciHpState acpi_pci_hotplug;
> > > bool use_acpi_pci_hotplug;
> > > + bool use_acpi_unplug;
> > >
> > > uint8_t disable_s3;
> > > uint8_t disable_s4;
> > > @@ -633,6 +634,8 @@ static Property piix4_pm_properties[]
> = {
> > > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState,
> s4_val, 2),
> > > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support",
> > PIIX4PMState,
> > > use_acpi_pci_hotplug, true),
> > > + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge",
> > PIIX4PMState,
> > > + use_acpi_unplug, true),
> > > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > > acpi_memory_hotplug.is_enabled, true),
> > > DEFINE_PROP_END_OF_LIST(),
> > > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > > index 23c77ee..71b3ac3 100644
> > > --- a/hw/i386/acpi-build.c
> > > +++ b/hw/i386/acpi-build.c
> > > @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> > > bool s3_disabled;
> > > bool s4_disabled;
> > > bool pcihp_bridge_en;
> > > + bool pcihup_bridge_en;
> > > uint8_t s4_val;
> > > AcpiFadtData fadt;
> > > uint16_t cpu_hp_io_base;
> > > @@ -240,6 +241,9 @@ static void
> acpi_get_pm_info(MachineState
> > *machine,
> > > AcpiPmInfo *pm)
> > > pm->pcihp_bridge_en =
> > > object_property_get_bool(obj,
> > "acpi-pci-hotplug-with-bridge-support",
> > > NULL);
> > > + pm->pcihup_bridge_en =
> > > + object_property_get_bool(obj,
> > "acpi-pci-hotunplug-enable-bridge",
> > > + NULL);
> > > }
> > >
> > > static void acpi_get_misc_info(AcpiMiscInfo *info)
> > > @@ -451,7 +455,8 @@ static void
> build_append_pcihp_notify_entry
> > (Aml
> > > *method, int slot)
> > > }
> > >
> > > static void build_append_pci_bus_devices(Aml *parent_scope,
> > PCIBus
> > > *bus,
> > > - bool pcihp_bridge_en)
> > > + bool pcihp_bridge_en,
> > > + bool pcihup_bridge_en)
> > > {
> > > Aml *dev, *notify_method = NULL, *method;
> > > QObject *bsel;
> > > @@ -479,11 +484,14 @@ static void
> build_append_pci_bus_devices
> > (Aml
> > > *parent_scope, PCIBus *bus,
> > > dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> > > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > > aml_append(dev, aml_name_decl("_ADR", aml_int(slot <<
> 16)));
> > > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > - aml_append(method,
> > > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > > - );
> > > - aml_append(dev, method);
> > > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > + aml_append(method,
> > > + aml_call2("PCEJ", aml_name("BSEL"),
> > > + aml_name("_SUN"))
> > > + );
> > > + aml_append(dev, method);
> > > + }
> > > aml_append(parent_scope, dev);
> > >
> > > build_append_pcihp_notify_entry(notify_method, slot);
> > > @@ -537,12 +545,14 @@ static void
> build_append_pci_bus_devices
> > (Aml
> > > *parent_scope, PCIBus *bus,
> > > /* add _SUN/_EJ0 to make slot hotpluggable */
> > > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > >
> > > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > - aml_append(method,
> > > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > > - );
> > > - aml_append(dev, method);
> > > -
> > > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > + aml_append(method,
> > > + aml_call2("PCEJ", aml_name("BSEL"),
> > > + aml_name("_SUN"))
> > > + );
> > > + aml_append(dev, method);
> > > + }
> > > if (bsel) {
> > > build_append_pcihp_notify_entry(notify_method, slot);
> > > }
> > > @@ -553,7 +563,8 @@ static void
> build_append_pci_bus_devices(Aml
> > > *parent_scope, PCIBus *bus,
> > > */
> > > PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> > >
> > > - build_append_pci_bus_devices(dev, sec_bus,
> pcihp_bridge_en);
> > > + build_append_pci_bus_devices(dev, sec_bus,
> pcihp_bridge_en,
> > > + pcihup_bridge_en);
> > > }
> > > /* slot descriptor has been composed, add it into parent
> context
> > */
> > > aml_append(parent_scope, dev);
> > > @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data,
> BIOSLinker
> > > *linker,
> > > if (bus) {
> > > Aml *scope = aml_scope("PCI0");
> > > /* Scan all PCI buses. Generate tables to support hotplug.
> */
> > > - build_append_pci_bus_devices(scope, bus,
> pm->pcihp_bridge_en);
> > > + build_append_pci_bus_devices(scope, bus,
> pm->pcihp_bridge_en,
> > > + pm->pcihup_bridge_en);
> > >
> > > if (TPM_IS_TIS_ISA(tpm)) {
> > > if (misc->tpm_version == TPM_VERSION_2_0) {
> > > --
> > > 1.9.4
> > >
> > >
> > >
> >
> >
>
>
[-- Attachment #2: Type: text/html, Size: 13027 bytes --]
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 0:58 ` Ani Sinha
@ 2020-04-29 5:28 ` Michael S. Tsirkin
2020-04-29 5:59 ` Ani Sinha
2020-04-29 6:11 ` Ani Sinha
0 siblings, 2 replies; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-29 5:28 UTC (permalink / raw)
To: Ani Sinha
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Paolo Bonzini, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, Apr 29, 2020 at 06:28:46AM +0530, Ani Sinha wrote:
> Well there were several discussions in the other thread around how PCIE behaves
> and how we can't change the slot features without a HW reset. Those were useful
> inputs.
OK so I'd expect these to be addressed in some way. If we commit to
support a feature which has no chance to work on real hardware, we paint
ourselves into a tight corner. This kind of thing tends to create
maintainance problems down the road. Disabling both hotplug and unplug
sounds like a reasonable thing to do, so if there's a need to disable
just one of these, commit log needs to do a better job documenting the
usecase.
Alternatively, we need to be more creative with achieving what you are
trying to do in ways that can work on real hardware.
For example, how about hot-plugging a bridge which doesn't
support hotplug itself? Would that happen to make windows
do what you want, for both PCI and PCIE? We don't support
hotplugging bridges with devices behind them ATM, but
that sounds like a useful option.
Also, pcie root ports recently gained ability to disable hotplug, see
commit 530a0963184e57e71a5b538e9161f115df533e96
Author: Julia Suvorova <jusual@redhat.com>
Date: Wed Feb 26 18:46:07 2020 +0100
pcie_root_port: Add hotplug disabling option
adding this to pci and pcie bridges sounds very reasonable.
--
MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 5:28 ` Michael S. Tsirkin
@ 2020-04-29 5:59 ` Ani Sinha
2020-04-29 6:11 ` Ani Sinha
1 sibling, 0 replies; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 5:59 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> For example, how about hot-plugging a bridge which doesn't
> support hotplug itself?
So now for every device we want to hot plug, we will add a new bridge? Sounds not a scalable and elegant idea.
> Would that happen to make windows
> do what you want, for both PCI and PCIE? We don't support
> hotplugging bridges with devices behind them ATM, but
> that sounds like a useful option.
>
>
> Also, pcie root ports recently gained ability to disable hotplug, see
> commit 530a0963184e57e71a5b538e9161f115df533e96
> Author: Julia Suvorova <jusual@redhat.com>
> Date: Wed Feb 26 18:46:07 2020 +0100
>
> pcie_root_port: Add hotplug disabling option
>
> adding this to pci and pcie bridges sounds very reasonable.
This patch uses PCIE native hot plug capability to turn on or turn off per slot hot plug option. This does not enable hot unplugging while allowing devices to be hot plugged on a PCIE port. Besides, PCI-PCI bridges already have an option to disable hot plug capability for all the slots of that bridge. My patch follows that approach.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 5:28 ` Michael S. Tsirkin
2020-04-29 5:59 ` Ani Sinha
@ 2020-04-29 6:11 ` Ani Sinha
2020-04-29 6:52 ` Michael S. Tsirkin
1 sibling, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 6:11 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> o if there's a need to disable
> just one of these, commit log needs to do a better job documenting the
> usecase.
The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
(a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
(b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
(c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
(d) I am told that the driver level solution does not persist across a reboot.
Ani
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 6:11 ` Ani Sinha
@ 2020-04-29 6:52 ` Michael S. Tsirkin
2020-04-29 6:54 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-29 6:52 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
>
>
> > On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > o if there's a need to disable
> > just one of these, commit log needs to do a better job documenting the
> > usecase.
>
> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
> (d) I am told that the driver level solution does not persist across a reboot.
>
> Ani
Looks like disabling both plug and unplug would also address these needs.
--
MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 6:52 ` Michael S. Tsirkin
@ 2020-04-29 6:54 ` Ani Sinha
2020-04-29 6:57 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 6:54 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 12:22 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
>>
>>
>>> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>
>>> o if there's a need to disable
>>> just one of these, commit log needs to do a better job documenting the
>>> usecase.
>>
>> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
>> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
>> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
>> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
>> (d) I am told that the driver level solution does not persist across a reboot.
>>
>> Ani
>
> Looks like disabling both plug and unplug would also address these needs.
No the driver level solution does not prevent hot plugging of devices but blocks just hot unplugging. The solution I am proposing tries to do the same but from the bus/hypervisor level.
> --
> MST
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 6:54 ` Ani Sinha
@ 2020-04-29 6:57 ` Michael S. Tsirkin
2020-04-29 7:02 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-29 6:57 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, Apr 29, 2020 at 06:54:52AM +0000, Ani Sinha wrote:
>
>
> > On Apr 29, 2020, at 12:22 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
> >>
> >>
> >>> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>
> >>> o if there's a need to disable
> >>> just one of these, commit log needs to do a better job documenting the
> >>> usecase.
> >>
> >> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
> >> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
> >> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
> >> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
> >> (d) I am told that the driver level solution does not persist across a reboot.
> >>
> >> Ani
> >
> > Looks like disabling both plug and unplug would also address these needs.
>
> No the driver level solution does not prevent hot plugging of devices but blocks just hot unplugging. The solution I am proposing tries to do the same but from the bus/hypervisor level.
Why does the driver level solution need to prevent just hot unplugging?
>
> > --
> > MST
> >
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 6:57 ` Michael S. Tsirkin
@ 2020-04-29 7:02 ` Ani Sinha
2020-04-29 7:38 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 7:02 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 12:27 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Apr 29, 2020 at 06:54:52AM +0000, Ani Sinha wrote:
>>
>>
>>> On Apr 29, 2020, at 12:22 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>
>>> On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
>>>>
>>>>
>>>>> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>>>
>>>>> o if there's a need to disable
>>>>> just one of these, commit log needs to do a better job documenting the
>>>>> usecase.
>>>>
>>>> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
>>>> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
>>>> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
>>>> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
>>>> (d) I am told that the driver level solution does not persist across a reboot.
>>>>
>>>> Ani
>>>
>>> Looks like disabling both plug and unplug would also address these needs.
>>
>> No the driver level solution does not prevent hot plugging of devices but blocks just hot unplugging. The solution I am proposing tries to do the same but from the bus/hypervisor level.
>
> Why does the driver level solution need to prevent just hot unplugging?
Because it not fair to prevent end users from hot plugging new devices when it is the (accidental?) hot unplugging of existing devices which causes issues.
>
>
>>
>>> --
>>> MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 7:02 ` Ani Sinha
@ 2020-04-29 7:38 ` Michael S. Tsirkin
2020-04-29 7:43 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-29 7:38 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, Apr 29, 2020 at 07:02:56AM +0000, Ani Sinha wrote:
>
>
> > On Apr 29, 2020, at 12:27 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Wed, Apr 29, 2020 at 06:54:52AM +0000, Ani Sinha wrote:
> >>
> >>
> >>> On Apr 29, 2020, at 12:22 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>
> >>> On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
> >>>>
> >>>>
> >>>>> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>>>
> >>>>> o if there's a need to disable
> >>>>> just one of these, commit log needs to do a better job documenting the
> >>>>> usecase.
> >>>>
> >>>> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
> >>>> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
> >>>> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
> >>>> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
> >>>> (d) I am told that the driver level solution does not persist across a reboot.
> >>>>
> >>>> Ani
> >>>
> >>> Looks like disabling both plug and unplug would also address these needs.
> >>
> >> No the driver level solution does not prevent hot plugging of devices but blocks just hot unplugging. The solution I am proposing tries to do the same but from the bus/hypervisor level.
> >
> > Why does the driver level solution need to prevent just hot unplugging?
>
> Because it not fair to prevent end users from hot plugging new devices when it is the (accidental?) hot unplugging of existing devices which causes issues.
Accidental? So maybe what you need is actually something else then -
avoid *removing* the device when it's powered down.
> >
> >
> >>
> >>> --
> >>> MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 7:38 ` Michael S. Tsirkin
@ 2020-04-29 7:43 ` Ani Sinha
2020-04-29 8:09 ` Michael S. Tsirkin
2020-04-30 17:12 ` Ani Sinha
0 siblings, 2 replies; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 7:43 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 1:08 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Apr 29, 2020 at 07:02:56AM +0000, Ani Sinha wrote:
>>
>>
>>> On Apr 29, 2020, at 12:27 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>
>>> On Wed, Apr 29, 2020 at 06:54:52AM +0000, Ani Sinha wrote:
>>>>
>>>>
>>>>> On Apr 29, 2020, at 12:22 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>>>
>>>>> On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
>>>>>>
>>>>>>
>>>>>>> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>>>>>
>>>>>>> o if there's a need to disable
>>>>>>> just one of these, commit log needs to do a better job documenting the
>>>>>>> usecase.
>>>>>>
>>>>>> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
>>>>>> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
>>>>>> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
>>>>>> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
>>>>>> (d) I am told that the driver level solution does not persist across a reboot.
>>>>>>
>>>>>> Ani
>>>>>
>>>>> Looks like disabling both plug and unplug would also address these needs.
>>>>
>>>> No the driver level solution does not prevent hot plugging of devices but blocks just hot unplugging. The solution I am proposing tries to do the same but from the bus/hypervisor level.
>>>
>>> Why does the driver level solution need to prevent just hot unplugging?
>>
>> Because it not fair to prevent end users from hot plugging new devices when it is the (accidental?) hot unplugging of existing devices which causes issues.
>
> Accidental? So maybe what you need is actually something else then -
> avoid *removing* the device when it's powered down.
You don’t get it. It is not hypervisor admins who are unplugging it. It is the end users. Even RedHat customers want this feature. See following resources:
https://www.redhat.com/archives/libvir-list/2020-February/msg00110.html
https://bugzilla.redhat.com/show_bug.cgi?id=1802592
https://bugzilla.redhat.com/show_bug.cgi?id=1790899
My approach is much more fine grained than just disable everything approach that we have for q35. For i440fx we can do better than that.
>
>>>
>>>
>>>>
>>>>> --
>>>>> MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 7:43 ` Ani Sinha
@ 2020-04-29 8:09 ` Michael S. Tsirkin
2020-04-29 8:14 ` Ani Sinha
2020-04-30 17:12 ` Ani Sinha
1 sibling, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-29 8:09 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, Apr 29, 2020 at 07:43:04AM +0000, Ani Sinha wrote:
>
>
> > On Apr 29, 2020, at 1:08 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Wed, Apr 29, 2020 at 07:02:56AM +0000, Ani Sinha wrote:
> >>
> >>
> >>> On Apr 29, 2020, at 12:27 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>
> >>> On Wed, Apr 29, 2020 at 06:54:52AM +0000, Ani Sinha wrote:
> >>>>
> >>>>
> >>>>> On Apr 29, 2020, at 12:22 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>>>
> >>>>> On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
> >>>>>>
> >>>>>>
> >>>>>>> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>>>>>
> >>>>>>> o if there's a need to disable
> >>>>>>> just one of these, commit log needs to do a better job documenting the
> >>>>>>> usecase.
> >>>>>>
> >>>>>> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
> >>>>>> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
> >>>>>> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
> >>>>>> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
> >>>>>> (d) I am told that the driver level solution does not persist across a reboot.
> >>>>>>
> >>>>>> Ani
> >>>>>
> >>>>> Looks like disabling both plug and unplug would also address these needs.
> >>>>
> >>>> No the driver level solution does not prevent hot plugging of devices but blocks just hot unplugging. The solution I am proposing tries to do the same but from the bus/hypervisor level.
> >>>
> >>> Why does the driver level solution need to prevent just hot unplugging?
> >>
> >> Because it not fair to prevent end users from hot plugging new devices when it is the (accidental?) hot unplugging of existing devices which causes issues.
> >
> > Accidental? So maybe what you need is actually something else then -
> > avoid *removing* the device when it's powered down.
>
> You don’t get it. It is not hypervisor admins who are unplugging it. It is the end users. Even RedHat customers want this feature. See following resources:
> https://www.redhat.com/archives/libvir-list/2020-February/msg00110.html
> https://bugzilla.redhat.com/show_bug.cgi?id=1802592
> https://bugzilla.redhat.com/show_bug.cgi?id=1790899
That doesn't seem to require that hotplug keeps working.
> My approach is much more fine grained than just disable everything approach that we have for q35. For i440fx we can do better than that.
>
>
> >
> >>>
> >>>
> >>>>
> >>>>> --
> >>>>> MST
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 8:09 ` Michael S. Tsirkin
@ 2020-04-29 8:14 ` Ani Sinha
2020-04-29 8:56 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 8:14 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 1:39 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Apr 29, 2020 at 07:43:04AM +0000, Ani Sinha wrote:
>>
>>
>>> On Apr 29, 2020, at 1:08 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>
>>> On Wed, Apr 29, 2020 at 07:02:56AM +0000, Ani Sinha wrote:
>>>>
>>>>
>>>>> On Apr 29, 2020, at 12:27 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>>>
>>>>> On Wed, Apr 29, 2020 at 06:54:52AM +0000, Ani Sinha wrote:
>>>>>>
>>>>>>
>>>>>>> On Apr 29, 2020, at 12:22 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>>>>>
>>>>>>> On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>>>>>>>
>>>>>>>>> o if there's a need to disable
>>>>>>>>> just one of these, commit log needs to do a better job documenting the
>>>>>>>>> usecase.
>>>>>>>>
>>>>>>>> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
>>>>>>>> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
>>>>>>>> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
>>>>>>>> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
>>>>>>>> (d) I am told that the driver level solution does not persist across a reboot.
>>>>>>>>
>>>>>>>> Ani
>>>>>>>
>>>>>>> Looks like disabling both plug and unplug would also address these needs.
>>>>>>
>>>>>> No the driver level solution does not prevent hot plugging of devices but blocks just hot unplugging. The solution I am proposing tries to do the same but from the bus/hypervisor level.
>>>>>
>>>>> Why does the driver level solution need to prevent just hot unplugging?
>>>>
>>>> Because it not fair to prevent end users from hot plugging new devices when it is the (accidental?) hot unplugging of existing devices which causes issues.
>>>
>>> Accidental? So maybe what you need is actually something else then -
>>> avoid *removing* the device when it's powered down.
>>
>> You don’t get it. It is not hypervisor admins who are unplugging it. It is the end users. Even RedHat customers want this feature. See following resources:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_archives_libvir-2Dlist_2020-2DFebruary_msg00110.html&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=Mf70_yU9LUbRFZOy4rYM5N43B_MDbO7SxEMSSJKVaJY&s=KgR1-KlzL-bGr51uY1vupOIgTpTjNAecbuOUIpcuMUs&e=
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1802592&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=Mf70_yU9LUbRFZOy4rYM5N43B_MDbO7SxEMSSJKVaJY&s=KVis9gzVeA7nnGauZpXWm_sEnl_UpsIzSlggwb60Fg8&e=
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1790899&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=Mf70_yU9LUbRFZOy4rYM5N43B_MDbO7SxEMSSJKVaJY&s=Y0K8tiyqpmKeXU245pnhTTAr2e3YSuxxw4BkiDxGwB8&e=
>
> That doesn't seem to require that hotplug keeps working.
Like I said, that is because PCIE limits this at this moment. We can do better than this solution on i440fx. We can be narrower in limiting just hot unplug leaving hot plugging as is. Why should we always go with the least common denominator for all the features? Why do we have to be limited when we can do better?
>
>> My approach is much more fine grained than just disable everything approach that we have for q35. For i440fx we can do better than that.
>>
>>
>>>
>>>>>
>>>>>
>>>>>>
>>>>>>> --
>>>>>>> MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 8:14 ` Ani Sinha
@ 2020-04-29 8:56 ` Michael S. Tsirkin
2020-04-29 9:14 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-29 8:56 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, Apr 29, 2020 at 08:14:01AM +0000, Ani Sinha wrote:
>
>
> > On Apr 29, 2020, at 1:39 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Wed, Apr 29, 2020 at 07:43:04AM +0000, Ani Sinha wrote:
> >>
> >>
> >>> On Apr 29, 2020, at 1:08 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>
> >>> On Wed, Apr 29, 2020 at 07:02:56AM +0000, Ani Sinha wrote:
> >>>>
> >>>>
> >>>>> On Apr 29, 2020, at 12:27 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>>>
> >>>>> On Wed, Apr 29, 2020 at 06:54:52AM +0000, Ani Sinha wrote:
> >>>>>>
> >>>>>>
> >>>>>>> On Apr 29, 2020, at 12:22 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>>>>>
> >>>>>>> On Wed, Apr 29, 2020 at 06:11:20AM +0000, Ani Sinha wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Apr 29, 2020, at 10:58 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>>>>>>>
> >>>>>>>>> o if there's a need to disable
> >>>>>>>>> just one of these, commit log needs to do a better job documenting the
> >>>>>>>>> usecase.
> >>>>>>>>
> >>>>>>>> The use case is simple. With this feature admins will be able to do what they were forced to do from Windows driver level but now at the bus level. Hence,
> >>>>>>>> (a) They need not have access to the guest VM to change or update windows driver registry settings. They can enable the same setting from admin management console without any access to VM.
> >>>>>>>> (b) It is more robust. No need to mess with driver settings. Incorrect settings can brick guest OS. Also no guest specific knowhow required.
> >>>>>>>> (c) It is more scalable since a single cluster wide setting can be used for all VM power ons and the management plane can take care of the rest automatically. No need to access individual VMs to enforce this.
> >>>>>>>> (d) I am told that the driver level solution does not persist across a reboot.
> >>>>>>>>
> >>>>>>>> Ani
> >>>>>>>
> >>>>>>> Looks like disabling both plug and unplug would also address these needs.
> >>>>>>
> >>>>>> No the driver level solution does not prevent hot plugging of devices but blocks just hot unplugging. The solution I am proposing tries to do the same but from the bus/hypervisor level.
> >>>>>
> >>>>> Why does the driver level solution need to prevent just hot unplugging?
> >>>>
> >>>> Because it not fair to prevent end users from hot plugging new devices when it is the (accidental?) hot unplugging of existing devices which causes issues.
> >>>
> >>> Accidental? So maybe what you need is actually something else then -
> >>> avoid *removing* the device when it's powered down.
> >>
> >> You don’t get it. It is not hypervisor admins who are unplugging it. It is the end users. Even RedHat customers want this feature. See following resources:
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_archives_libvir-2Dlist_2020-2DFebruary_msg00110.html&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=Mf70_yU9LUbRFZOy4rYM5N43B_MDbO7SxEMSSJKVaJY&s=KgR1-KlzL-bGr51uY1vupOIgTpTjNAecbuOUIpcuMUs&e=
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1802592&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=Mf70_yU9LUbRFZOy4rYM5N43B_MDbO7SxEMSSJKVaJY&s=KVis9gzVeA7nnGauZpXWm_sEnl_UpsIzSlggwb60Fg8&e=
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1790899&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=Mf70_yU9LUbRFZOy4rYM5N43B_MDbO7SxEMSSJKVaJY&s=Y0K8tiyqpmKeXU245pnhTTAr2e3YSuxxw4BkiDxGwB8&e=
> >
> > That doesn't seem to require that hotplug keeps working.
>
> Like I said, that is because PCIE limits this at this moment. We can
> do better than this solution on i440fx. We can be narrower in limiting
> just hot unplug leaving hot plugging as is. Why should we always go
> with the least common denominator for all the features? Why do we have
> to be limited when we can do better?
Even if it seems to work for guests now, if we don't stick to emulating
capabilities that hardware interfaces provide we can never be sure it
will keep working.
>
> >
> >> My approach is much more fine grained than just disable everything approach that we have for q35. For i440fx we can do better than that.
> >>
> >>
> >>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>> --
> >>>>>>> MST
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 8:56 ` Michael S. Tsirkin
@ 2020-04-29 9:14 ` Ani Sinha
2020-04-29 9:18 ` Ani Sinha
2020-04-29 10:15 ` Michael S. Tsirkin
0 siblings, 2 replies; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 9:14 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 2:26 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> Even if it seems to work for guests now, if we don't stick to emulating
> capabilities that hardware interfaces provide we can never be sure it
> will keep working.
OS es use ACPI for PCI bridges: https://github.com/qemu/qemu/blob/master/docs/pcie.txt
They use “_EJ0” to detect jot unplug capability: https://uefi.org/sites/default/files/resources/ACPI_3.pdf
6.3.3 _EJx (Eject) These control methods are optional and are supplied for devices that support a software-controlled VCRstyle ejection mechanism or that require an action be performed such as isolation of power/data lines before the device can be removed from the system. To support warm (system is in a sleep state) and hot (system is in S0) removal, an _EJx control method is listed for each sleep state from which the device supports removal, where x is the sleeping state supported. For example, _EJ0 indicates the device supports hot removal; _EJ1–EJ4 indicate the device supports warm removal.
Note that “these control methods are optional” line. If the OS adheres to the spec, it should not expect them to exist all the time.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 9:14 ` Ani Sinha
@ 2020-04-29 9:18 ` Ani Sinha
2020-04-29 10:15 ` Michael S. Tsirkin
1 sibling, 0 replies; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 9:18 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 2:44 PM, Anirban Sinha <ani.sinha@nutanix.com> wrote:
>
>
> Note that “these control methods are optional” line. If the OS adheres to the spec, it should not expect them to exist all the time
Linux for example, detects it here: https://elixir.bootlin.com/linux/latest/source/drivers/acpi/utils.c#L571
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 9:14 ` Ani Sinha
2020-04-29 9:18 ` Ani Sinha
@ 2020-04-29 10:15 ` Michael S. Tsirkin
2020-04-29 10:20 ` Ani Sinha
1 sibling, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-29 10:15 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, Apr 29, 2020 at 09:14:26AM +0000, Ani Sinha wrote:
>
>
> > On Apr 29, 2020, at 2:26 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > Even if it seems to work for guests now, if we don't stick to emulating
> > capabilities that hardware interfaces provide we can never be sure it
> > will keep working.
>
> OS es use ACPI for PCI bridges: https://github.com/qemu/qemu/blob/master/docs/pcie.txt
> They use “_EJ0” to detect jot unplug capability: https://uefi.org/sites/default/files/resources/ACPI_3.pdf
>
> 6.3.3 _EJx (Eject) These control methods are optional and are supplied for devices that support a software-controlled VCRstyle ejection mechanism or that require an action be performed such as isolation of power/data lines before the device can be removed from the system. To support warm (system is in a sleep state) and hot (system is in S0) removal, an _EJx control method is listed for each sleep state from which the device supports removal, where x is the sleeping state supported. For example, _EJ0 indicates the device supports hot removal; _EJ1–EJ4 indicate the device supports warm removal.
Yes. So if there's no _EJx then it's reasonable to assume you can't
isolate the slot, and so no hot-plug will happen either.
>
> Note that “these control methods are optional” line. If the OS adheres to the spec, it should not expect them to exist all the time.
>
--
MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 10:15 ` Michael S. Tsirkin
@ 2020-04-29 10:20 ` Ani Sinha
2020-04-29 10:30 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 10:20 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On Apr 29, 2020, at 3:45 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Apr 29, 2020 at 09:14:26AM +0000, Ani Sinha wrote:
>>
>>
>>> On Apr 29, 2020, at 2:26 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>>
>>> Even if it seems to work for guests now, if we don't stick to emulating
>>> capabilities that hardware interfaces provide we can never be sure it
>>> will keep working.
>>
>> OS es use ACPI for PCI bridges: https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_qemu_qemu_blob_master_docs_pcie.txt&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=9AnuR62iRsUnmV9ggkS_lqen67etHeCjATFLwCJWxQs&s=As__N9BXf0DTSZkD4cxaQsTeg0exh5GSPqkFSh7fxRk&e=
>> They use “_EJ0” to detect jot unplug capability: https://urldefense.proofpoint.com/v2/url?u=https-3A__uefi.org_sites_default_files_resources_ACPI-5F3.pdf&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=9AnuR62iRsUnmV9ggkS_lqen67etHeCjATFLwCJWxQs&s=SDgOhKGpnlzfbvAOFl2m9a-tdPZEUcCwKWsgkVDafyM&e=
>>
>> 6.3.3 _EJx (Eject) These control methods are optional and are supplied for devices that support a software-controlled VCRstyle ejection mechanism or that require an action be performed such as isolation of power/data lines before the device can be removed from the system. To support warm (system is in a sleep state) and hot (system is in S0) removal, an _EJx control method is listed for each sleep state from which the device supports removal, where x is the sleeping state supported. For example, _EJ0 indicates the device supports hot removal; _EJ1–EJ4 indicate the device supports warm removal.
>
> Yes. So if there's no _EJx then it's reasonable to assume you can't
> isolate the slot, and so no hot-plug will happen either.
Where are you getting that? This isn’t true. “_SUN” is used to detect the slot unique number. Windows does allow hot plugging on the bridge on which EJ0 has been turned off.
>
>>
>> Note that “these control methods are optional” line. If the OS adheres to the spec, it should not expect them to exist all the time.
>>
>
>
>
> --
> MST
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 10:20 ` Ani Sinha
@ 2020-04-29 10:30 ` Michael S. Tsirkin
2020-04-29 10:37 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-29 10:30 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, Apr 29, 2020 at 10:20:31AM +0000, Ani Sinha wrote:
>
>
> > On Apr 29, 2020, at 3:45 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Wed, Apr 29, 2020 at 09:14:26AM +0000, Ani Sinha wrote:
> >>
> >>
> >>> On Apr 29, 2020, at 2:26 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> >>>
> >>> Even if it seems to work for guests now, if we don't stick to emulating
> >>> capabilities that hardware interfaces provide we can never be sure it
> >>> will keep working.
> >>
> >> OS es use ACPI for PCI bridges: https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_qemu_qemu_blob_master_docs_pcie.txt&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=9AnuR62iRsUnmV9ggkS_lqen67etHeCjATFLwCJWxQs&s=As__N9BXf0DTSZkD4cxaQsTeg0exh5GSPqkFSh7fxRk&e=
> >> They use “_EJ0” to detect jot unplug capability: https://urldefense.proofpoint.com/v2/url?u=https-3A__uefi.org_sites_default_files_resources_ACPI-5F3.pdf&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=IIUxIyRwG4RGy57y2nvMNYcDkqW-NHozZ2R38VYcg5U&m=9AnuR62iRsUnmV9ggkS_lqen67etHeCjATFLwCJWxQs&s=SDgOhKGpnlzfbvAOFl2m9a-tdPZEUcCwKWsgkVDafyM&e=
> >>
> >> 6.3.3 _EJx (Eject) These control methods are optional and are supplied for devices that support a software-controlled VCRstyle ejection mechanism or that require an action be performed such as isolation of power/data lines before the device can be removed from the system. To support warm (system is in a sleep state) and hot (system is in S0) removal, an _EJx control method is listed for each sleep state from which the device supports removal, where x is the sleeping state supported. For example, _EJ0 indicates the device supports hot removal; _EJ1–EJ4 indicate the device supports warm removal.
> >
> > Yes. So if there's no _EJx then it's reasonable to assume you can't
> > isolate the slot, and so no hot-plug will happen either.
>
> Where are you getting that?
It's well known. For example, the pci hot-plug specification, version
1.1, states:
1.6 Asssumptions
...
1.6.2 Orderly Removal and Insertion
...
Furthermore, PCI add-in cards are not generally designed to be connected to a slot that
has power applied. Therefore, the operating-system vendor and Platform vendor define a
sequence of user actions and system behavior that guarantees that power is always
removed from a slot before a card is inserted.
Inserting or removing an add-in card without following the proper sequence leads to
unpredictable results, including data corruption, abnormal termination of the operating
system, or damage to card or Platform hardware. Unless otherwise stated, it is assumed
throughout the remainder of this specification that the user always follows the proper
removal and insertion sequence.
...
2.1 System Components
...
The physical
insertion or removal must not occur until the system software has:
• Quiesced any operating system services or drivers using the add-in card
• Isolated and powered down the slot
• Indicated to the user that it is ready
If an add-in card is inserted or removed without following the proper sequence, this is
considered an improper operation and error conditions and other unexpected events are
possible, including data corruption and hardware damage.
and so on.
> This isn’t true. “_SUN” is used to detect the slot unique number.
That's just enumeration.
> Windows does allow hot plugging on the bridge on which EJ0 has been turned off.
Given no real or virtual hardware that does that, there's no guarantee
that I can see that it will keep supporting that in future versions.
> >
> >>
> >> Note that “these control methods are optional” line. If the OS adheres to the spec, it should not expect them to exist all the time.
> >>
> >
> >
> >
> > --
> > MST
> >
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 10:30 ` Michael S. Tsirkin
@ 2020-04-29 10:37 ` Ani Sinha
0 siblings, 0 replies; 52+ messages in thread
From: Ani Sinha @ 2020-04-29 10:37 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eduardo Habkost, qemu-devel, Aleksandar Markovic, Paolo Bonzini,
Ani Sinha, Igor Mammedov, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
>>>
>>> Yes. So if there's no _EJx then it's reasonable to assume you can't
>>> isolate the slot, and so no hot-plug will happen either.
>>
>> Where are you getting that?
>
> It's well known. For example, the pci hot-plug specification, version
> 1.1, states:
>
> 1.6 Asssumptions
>
> ...
>
> 1.6.2 Orderly Removal and Insertion
>
> ...
>
> Furthermore, PCI add-in cards are not generally designed to be connected to a slot that
> has power applied. Therefore, the operating-system vendor and Platform vendor define a
> sequence of user actions and system behavior that guarantees that power is always
> removed from a slot before a card is inserted.
>
> Inserting or removing an add-in card without following the proper sequence leads to
> unpredictable results, including data corruption, abnormal termination of the operating
> system, or damage to card or Platform hardware. Unless otherwise stated, it is assumed
> throughout the remainder of this specification that the user always follows the proper
> removal and insertion sequence.
>
> ...
>
>
> 2.1 System Components
>
> ...
>
> The physical
> insertion or removal must not occur until the system software has:
> • Quiesced any operating system services or drivers using the add-in card
> • Isolated and powered down the slot
> • Indicated to the user that it is ready
> If an add-in card is inserted or removed without following the proper sequence, this is
> considered an improper operation and error conditions and other unexpected events are
> possible, including data corruption and hardware damage.
>
> and so on.
What does all of this got to so with ACPI hot plug spec which I showed you where it clearly says that availability of EJ0 is optional? How are we violating the spec? I also gave evidence of Linux detecting it at runtime and Windows also seems to allow it.
>
>
>> This isn’t true. “_SUN” is used to detect the slot unique number.
>
> That's just enumeration.
>
>> Windows does allow hot plugging on the bridge on which EJ0 has been turned off.
>
> Given no real or virtual hardware that does that, there's no guarantee
> that I can see that it will keep supporting that in future versions.
You do not know that. I gave evidence that we are not violating the spec.
If RedHat had not moved to q35 and they had come up with this patch for i440fx, would you have given this much biased pushback? I seriously doubt it.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-29 7:43 ` Ani Sinha
2020-04-29 8:09 ` Michael S. Tsirkin
@ 2020-04-30 17:12 ` Ani Sinha
1 sibling, 0 replies; 52+ messages in thread
From: Ani Sinha @ 2020-04-30 17:12 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, Michael S. Tsirkin, qemu-devel,
Aleksandar Markovic, Paolo Bonzini, Igor Mammedov,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]
.
Accidental? So maybe what you need is actually something else then -
avoid *removing* the device when it's powered down.
You don’t get it. It is not hypervisor admins who are unplugging it. It is
the end users. Even RedHat customers want this feature. See following
resources:
https://www.redhat.com/archives/libvir-list/2020-February/msg00110.html
https://bugzilla.redhat.com/show_bug.cgi?id=1802592
https://bugzilla.redhat.com/show_bug.cgi?id=1790899
My approach is much more fine grained than just disable everything approach
that we have for q35. For i440fx we can do better than that
By the way, here's another glaring feature disparity between i440fx and q35
which we perhaps did not debate with as much fervour and push back as we
debated here. When we implemented per slot hotplug disable for PCIE, we
ignored to implement the same per slot capability for conventional PCI. Why
was feature disparity across two machine types wasn't so much of an issue
then?
--
MST
[-- Attachment #2: Type: text/html, Size: 2778 bytes --]
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 16:05 ` Michael S. Tsirkin
2020-04-28 16:09 ` Ani Sinha
@ 2020-04-28 16:28 ` Daniel P. Berrangé
2020-04-28 16:30 ` Michael S. Tsirkin
1 sibling, 1 reply; 52+ messages in thread
From: Daniel P. Berrangé @ 2020-04-28 16:28 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Igor Mammedov, ani, Paolo Bonzini, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, Apr 28, 2020 at 12:05:47PM -0400, Michael S. Tsirkin wrote:
> On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> > A new option "use_acpi_unplug" is introduced for PIIX which will
> > selectively only disable hot unplugging of both hot plugged and
> > cold plugged PCI devices on non-root PCI buses. This will prevent
> > hot unplugging of devices from Windows based guests from system
> > tray but will not prevent devices from being hot plugged into the
> > guest.
> >
> > It has been tested on Windows guests.
> >
> > Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
>
> It's still a non starter until we find something similar for PCIE and
> SHPC. Do guests check command status? Can some unplug commands fail?
Why does PCIE need anything ? For that we already have ability to
control hotplugging per-slot in pcie-root-port. If SHPC doesn't
support this that's fine too, it isn't a reason to block its merge
and use with x86 i440fx machine.
>
>
> > ---
> > hw/acpi/piix4.c | 3 +++
> > hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> > 2 files changed, 29 insertions(+), 14 deletions(-)
> >
> > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > index 964d6f5..59fa707 100644
> > --- a/hw/acpi/piix4.c
> > +++ b/hw/acpi/piix4.c
> > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> >
> > AcpiPciHpState acpi_pci_hotplug;
> > bool use_acpi_pci_hotplug;
> > + bool use_acpi_unplug;
> >
> > uint8_t disable_s3;
> > uint8_t disable_s4;
> > @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> > use_acpi_pci_hotplug, true),
> > + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> > + use_acpi_unplug, true),
> > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > acpi_memory_hotplug.is_enabled, true),
> > DEFINE_PROP_END_OF_LIST(),
> > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > index 23c77ee..71b3ac3 100644
> > --- a/hw/i386/acpi-build.c
> > +++ b/hw/i386/acpi-build.c
> > @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> > bool s3_disabled;
> > bool s4_disabled;
> > bool pcihp_bridge_en;
> > + bool pcihup_bridge_en;
> > uint8_t s4_val;
> > AcpiFadtData fadt;
> > uint16_t cpu_hp_io_base;
> > @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
> > pm->pcihp_bridge_en =
> > object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> > NULL);
> > + pm->pcihup_bridge_en =
> > + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> > + NULL);
> > }
> >
> > static void acpi_get_misc_info(AcpiMiscInfo *info)
> > @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml *method, int slot)
> > }
> >
> > static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > - bool pcihp_bridge_en)
> > + bool pcihp_bridge_en,
> > + bool pcihup_bridge_en)
> > {
> > Aml *dev, *notify_method = NULL, *method;
> > QObject *bsel;
> > @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > - aml_append(method,
> > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > - );
> > - aml_append(dev, method);
> > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > + aml_append(method,
> > + aml_call2("PCEJ", aml_name("BSEL"),
> > + aml_name("_SUN"))
> > + );
> > + aml_append(dev, method);
> > + }
> > aml_append(parent_scope, dev);
> >
> > build_append_pcihp_notify_entry(notify_method, slot);
> > @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > /* add _SUN/_EJ0 to make slot hotpluggable */
> > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> >
> > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > - aml_append(method,
> > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > - );
> > - aml_append(dev, method);
> > -
> > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > + aml_append(method,
> > + aml_call2("PCEJ", aml_name("BSEL"),
> > + aml_name("_SUN"))
> > + );
> > + aml_append(dev, method);
> > + }
> > if (bsel) {
> > build_append_pcihp_notify_entry(notify_method, slot);
> > }
> > @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > */
> > PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> >
> > - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> > + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> > + pcihup_bridge_en);
> > }
> > /* slot descriptor has been composed, add it into parent context */
> > aml_append(parent_scope, dev);
> > @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> > if (bus) {
> > Aml *scope = aml_scope("PCI0");
> > /* Scan all PCI buses. Generate tables to support hotplug. */
> > - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> > + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> > + pm->pcihup_bridge_en);
> >
> > if (TPM_IS_TIS_ISA(tpm)) {
> > if (misc->tpm_version == TPM_VERSION_2_0) {
> > --
> > 1.9.4
>
>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 16:28 ` Daniel P. Berrangé
@ 2020-04-28 16:30 ` Michael S. Tsirkin
2020-04-28 16:33 ` Daniel P. Berrangé
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-28 16:30 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Igor Mammedov, ani, Paolo Bonzini, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, Apr 28, 2020 at 05:28:36PM +0100, Daniel P. Berrangé wrote:
> On Tue, Apr 28, 2020 at 12:05:47PM -0400, Michael S. Tsirkin wrote:
> > On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> > > A new option "use_acpi_unplug" is introduced for PIIX which will
> > > selectively only disable hot unplugging of both hot plugged and
> > > cold plugged PCI devices on non-root PCI buses. This will prevent
> > > hot unplugging of devices from Windows based guests from system
> > > tray but will not prevent devices from being hot plugged into the
> > > guest.
> > >
> > > It has been tested on Windows guests.
> > >
> > > Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
> >
> > It's still a non starter until we find something similar for PCIE and
> > SHPC. Do guests check command status? Can some unplug commands fail?
>
> Why does PCIE need anything ? For that we already have ability to
> control hotplugging per-slot in pcie-root-port.
At the moment that does not support unplug of hotplugged devices.
> If SHPC doesn't
> support this that's fine too, it isn't a reason to block its merge
> and use with x86 i440fx machine.
>
> >
> >
> > > ---
> > > hw/acpi/piix4.c | 3 +++
> > > hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> > > 2 files changed, 29 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > > index 964d6f5..59fa707 100644
> > > --- a/hw/acpi/piix4.c
> > > +++ b/hw/acpi/piix4.c
> > > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> > >
> > > AcpiPciHpState acpi_pci_hotplug;
> > > bool use_acpi_pci_hotplug;
> > > + bool use_acpi_unplug;
> > >
> > > uint8_t disable_s3;
> > > uint8_t disable_s4;
> > > @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> > > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> > > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> > > use_acpi_pci_hotplug, true),
> > > + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> > > + use_acpi_unplug, true),
> > > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > > acpi_memory_hotplug.is_enabled, true),
> > > DEFINE_PROP_END_OF_LIST(),
> > > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > > index 23c77ee..71b3ac3 100644
> > > --- a/hw/i386/acpi-build.c
> > > +++ b/hw/i386/acpi-build.c
> > > @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> > > bool s3_disabled;
> > > bool s4_disabled;
> > > bool pcihp_bridge_en;
> > > + bool pcihup_bridge_en;
> > > uint8_t s4_val;
> > > AcpiFadtData fadt;
> > > uint16_t cpu_hp_io_base;
> > > @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
> > > pm->pcihp_bridge_en =
> > > object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> > > NULL);
> > > + pm->pcihup_bridge_en =
> > > + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> > > + NULL);
> > > }
> > >
> > > static void acpi_get_misc_info(AcpiMiscInfo *info)
> > > @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml *method, int slot)
> > > }
> > >
> > > static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > - bool pcihp_bridge_en)
> > > + bool pcihp_bridge_en,
> > > + bool pcihup_bridge_en)
> > > {
> > > Aml *dev, *notify_method = NULL, *method;
> > > QObject *bsel;
> > > @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> > > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > > aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> > > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > - aml_append(method,
> > > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > > - );
> > > - aml_append(dev, method);
> > > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > + aml_append(method,
> > > + aml_call2("PCEJ", aml_name("BSEL"),
> > > + aml_name("_SUN"))
> > > + );
> > > + aml_append(dev, method);
> > > + }
> > > aml_append(parent_scope, dev);
> > >
> > > build_append_pcihp_notify_entry(notify_method, slot);
> > > @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > /* add _SUN/_EJ0 to make slot hotpluggable */
> > > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > >
> > > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > - aml_append(method,
> > > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > > - );
> > > - aml_append(dev, method);
> > > -
> > > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > + aml_append(method,
> > > + aml_call2("PCEJ", aml_name("BSEL"),
> > > + aml_name("_SUN"))
> > > + );
> > > + aml_append(dev, method);
> > > + }
> > > if (bsel) {
> > > build_append_pcihp_notify_entry(notify_method, slot);
> > > }
> > > @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > */
> > > PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> > >
> > > - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> > > + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> > > + pcihup_bridge_en);
> > > }
> > > /* slot descriptor has been composed, add it into parent context */
> > > aml_append(parent_scope, dev);
> > > @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> > > if (bus) {
> > > Aml *scope = aml_scope("PCI0");
> > > /* Scan all PCI buses. Generate tables to support hotplug. */
> > > - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> > > + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> > > + pm->pcihup_bridge_en);
> > >
> > > if (TPM_IS_TIS_ISA(tpm)) {
> > > if (misc->tpm_version == TPM_VERSION_2_0) {
> > > --
> > > 1.9.4
> >
> >
>
> Regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 16:30 ` Michael S. Tsirkin
@ 2020-04-28 16:33 ` Daniel P. Berrangé
2020-04-28 20:44 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Daniel P. Berrangé @ 2020-04-28 16:33 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Igor Mammedov, ani, Paolo Bonzini, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, Apr 28, 2020 at 12:30:53PM -0400, Michael S. Tsirkin wrote:
> On Tue, Apr 28, 2020 at 05:28:36PM +0100, Daniel P. Berrangé wrote:
> > On Tue, Apr 28, 2020 at 12:05:47PM -0400, Michael S. Tsirkin wrote:
> > > On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> > > > A new option "use_acpi_unplug" is introduced for PIIX which will
> > > > selectively only disable hot unplugging of both hot plugged and
> > > > cold plugged PCI devices on non-root PCI buses. This will prevent
> > > > hot unplugging of devices from Windows based guests from system
> > > > tray but will not prevent devices from being hot plugged into the
> > > > guest.
> > > >
> > > > It has been tested on Windows guests.
> > > >
> > > > Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
> > >
> > > It's still a non starter until we find something similar for PCIE and
> > > SHPC. Do guests check command status? Can some unplug commands fail?
> >
> > Why does PCIE need anything ? For that we already have ability to
> > control hotplugging per-slot in pcie-root-port.
>
> At the moment that does not support unplug of hotplugged devices.
I don't see why this patch has to deal with that limitation though,
it is a independant problem from this patch which is PCI focused,
not PCIe.
>
>
> > If SHPC doesn't
> > support this that's fine too, it isn't a reason to block its merge
> > and use with x86 i440fx machine.
> >
> > >
> > >
> > > > ---
> > > > hw/acpi/piix4.c | 3 +++
> > > > hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> > > > 2 files changed, 29 insertions(+), 14 deletions(-)
> > > >
> > > > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > > > index 964d6f5..59fa707 100644
> > > > --- a/hw/acpi/piix4.c
> > > > +++ b/hw/acpi/piix4.c
> > > > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> > > >
> > > > AcpiPciHpState acpi_pci_hotplug;
> > > > bool use_acpi_pci_hotplug;
> > > > + bool use_acpi_unplug;
> > > >
> > > > uint8_t disable_s3;
> > > > uint8_t disable_s4;
> > > > @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> > > > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> > > > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> > > > use_acpi_pci_hotplug, true),
> > > > + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> > > > + use_acpi_unplug, true),
> > > > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > > > acpi_memory_hotplug.is_enabled, true),
> > > > DEFINE_PROP_END_OF_LIST(),
> > > > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > > > index 23c77ee..71b3ac3 100644
> > > > --- a/hw/i386/acpi-build.c
> > > > +++ b/hw/i386/acpi-build.c
> > > > @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> > > > bool s3_disabled;
> > > > bool s4_disabled;
> > > > bool pcihp_bridge_en;
> > > > + bool pcihup_bridge_en;
> > > > uint8_t s4_val;
> > > > AcpiFadtData fadt;
> > > > uint16_t cpu_hp_io_base;
> > > > @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
> > > > pm->pcihp_bridge_en =
> > > > object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> > > > NULL);
> > > > + pm->pcihup_bridge_en =
> > > > + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> > > > + NULL);
> > > > }
> > > >
> > > > static void acpi_get_misc_info(AcpiMiscInfo *info)
> > > > @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml *method, int slot)
> > > > }
> > > >
> > > > static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > > - bool pcihp_bridge_en)
> > > > + bool pcihp_bridge_en,
> > > > + bool pcihup_bridge_en)
> > > > {
> > > > Aml *dev, *notify_method = NULL, *method;
> > > > QObject *bsel;
> > > > @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > > dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> > > > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > > > aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> > > > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > > - aml_append(method,
> > > > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > > > - );
> > > > - aml_append(dev, method);
> > > > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > > > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > > + aml_append(method,
> > > > + aml_call2("PCEJ", aml_name("BSEL"),
> > > > + aml_name("_SUN"))
> > > > + );
> > > > + aml_append(dev, method);
> > > > + }
> > > > aml_append(parent_scope, dev);
> > > >
> > > > build_append_pcihp_notify_entry(notify_method, slot);
> > > > @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > > /* add _SUN/_EJ0 to make slot hotpluggable */
> > > > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > > >
> > > > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > > - aml_append(method,
> > > > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > > > - );
> > > > - aml_append(dev, method);
> > > > -
> > > > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > > > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > > + aml_append(method,
> > > > + aml_call2("PCEJ", aml_name("BSEL"),
> > > > + aml_name("_SUN"))
> > > > + );
> > > > + aml_append(dev, method);
> > > > + }
> > > > if (bsel) {
> > > > build_append_pcihp_notify_entry(notify_method, slot);
> > > > }
> > > > @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > > */
> > > > PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> > > >
> > > > - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> > > > + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> > > > + pcihup_bridge_en);
> > > > }
> > > > /* slot descriptor has been composed, add it into parent context */
> > > > aml_append(parent_scope, dev);
> > > > @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> > > > if (bus) {
> > > > Aml *scope = aml_scope("PCI0");
> > > > /* Scan all PCI buses. Generate tables to support hotplug. */
> > > > - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> > > > + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> > > > + pm->pcihup_bridge_en);
> > > >
> > > > if (TPM_IS_TIS_ISA(tpm)) {
> > > > if (misc->tpm_version == TPM_VERSION_2_0) {
> > > > --
> > > > 1.9.4
> > >
> > >
> >
> > Regards,
> > Daniel
> > --
> > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> > |: https://libvirt.org -o- https://fstop138.berrange.com :|
> > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 16:33 ` Daniel P. Berrangé
@ 2020-04-28 20:44 ` Michael S. Tsirkin
0 siblings, 0 replies; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-04-28 20:44 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Ani Sinha, Eduardo Habkost, qemu-devel, Aleksandar Markovic,
Igor Mammedov, ani, Paolo Bonzini, Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, Apr 28, 2020 at 05:33:08PM +0100, Daniel P. Berrangé wrote:
> On Tue, Apr 28, 2020 at 12:30:53PM -0400, Michael S. Tsirkin wrote:
> > On Tue, Apr 28, 2020 at 05:28:36PM +0100, Daniel P. Berrangé wrote:
> > > On Tue, Apr 28, 2020 at 12:05:47PM -0400, Michael S. Tsirkin wrote:
> > > > On Tue, Apr 28, 2020 at 10:16:52AM +0000, Ani Sinha wrote:
> > > > > A new option "use_acpi_unplug" is introduced for PIIX which will
> > > > > selectively only disable hot unplugging of both hot plugged and
> > > > > cold plugged PCI devices on non-root PCI buses. This will prevent
> > > > > hot unplugging of devices from Windows based guests from system
> > > > > tray but will not prevent devices from being hot plugged into the
> > > > > guest.
> > > > >
> > > > > It has been tested on Windows guests.
> > > > >
> > > > > Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
> > > >
> > > > It's still a non starter until we find something similar for PCIE and
> > > > SHPC. Do guests check command status? Can some unplug commands fail?
> > >
> > > Why does PCIE need anything ? For that we already have ability to
> > > control hotplugging per-slot in pcie-root-port.
> >
> > At the moment that does not support unplug of hotplugged devices.
>
> I don't see why this patch has to deal with that limitation though,
> it is a independant problem from this patch which is PCI focused,
> not PCIe.
And that's par for the course, each contributor wants to care only about
his own corner. The only tool I as a maintainer have for keeping things
consistent is by deferring merge until they are.
> >
> >
> > > If SHPC doesn't
> > > support this that's fine too, it isn't a reason to block its merge
> > > and use with x86 i440fx machine.
> > >
> > > >
> > > >
> > > > > ---
> > > > > hw/acpi/piix4.c | 3 +++
> > > > > hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> > > > > 2 files changed, 29 insertions(+), 14 deletions(-)
> > > > >
> > > > > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > > > > index 964d6f5..59fa707 100644
> > > > > --- a/hw/acpi/piix4.c
> > > > > +++ b/hw/acpi/piix4.c
> > > > > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> > > > >
> > > > > AcpiPciHpState acpi_pci_hotplug;
> > > > > bool use_acpi_pci_hotplug;
> > > > > + bool use_acpi_unplug;
> > > > >
> > > > > uint8_t disable_s3;
> > > > > uint8_t disable_s4;
> > > > > @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> > > > > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> > > > > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> > > > > use_acpi_pci_hotplug, true),
> > > > > + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> > > > > + use_acpi_unplug, true),
> > > > > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > > > > acpi_memory_hotplug.is_enabled, true),
> > > > > DEFINE_PROP_END_OF_LIST(),
> > > > > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> > > > > index 23c77ee..71b3ac3 100644
> > > > > --- a/hw/i386/acpi-build.c
> > > > > +++ b/hw/i386/acpi-build.c
> > > > > @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> > > > > bool s3_disabled;
> > > > > bool s4_disabled;
> > > > > bool pcihp_bridge_en;
> > > > > + bool pcihup_bridge_en;
> > > > > uint8_t s4_val;
> > > > > AcpiFadtData fadt;
> > > > > uint16_t cpu_hp_io_base;
> > > > > @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
> > > > > pm->pcihp_bridge_en =
> > > > > object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> > > > > NULL);
> > > > > + pm->pcihup_bridge_en =
> > > > > + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> > > > > + NULL);
> > > > > }
> > > > >
> > > > > static void acpi_get_misc_info(AcpiMiscInfo *info)
> > > > > @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml *method, int slot)
> > > > > }
> > > > >
> > > > > static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > > > - bool pcihp_bridge_en)
> > > > > + bool pcihp_bridge_en,
> > > > > + bool pcihup_bridge_en)
> > > > > {
> > > > > Aml *dev, *notify_method = NULL, *method;
> > > > > QObject *bsel;
> > > > > @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > > > dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> > > > > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > > > > aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> > > > > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > > > - aml_append(method,
> > > > > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > > > > - );
> > > > > - aml_append(dev, method);
> > > > > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > > > > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > > > + aml_append(method,
> > > > > + aml_call2("PCEJ", aml_name("BSEL"),
> > > > > + aml_name("_SUN"))
> > > > > + );
> > > > > + aml_append(dev, method);
> > > > > + }
> > > > > aml_append(parent_scope, dev);
> > > > >
> > > > > build_append_pcihp_notify_entry(notify_method, slot);
> > > > > @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > > > /* add _SUN/_EJ0 to make slot hotpluggable */
> > > > > aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> > > > >
> > > > > - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > > > - aml_append(method,
> > > > > - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> > > > > - );
> > > > > - aml_append(dev, method);
> > > > > -
> > > > > + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> > > > > + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> > > > > + aml_append(method,
> > > > > + aml_call2("PCEJ", aml_name("BSEL"),
> > > > > + aml_name("_SUN"))
> > > > > + );
> > > > > + aml_append(dev, method);
> > > > > + }
> > > > > if (bsel) {
> > > > > build_append_pcihp_notify_entry(notify_method, slot);
> > > > > }
> > > > > @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> > > > > */
> > > > > PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> > > > >
> > > > > - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> > > > > + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> > > > > + pcihup_bridge_en);
> > > > > }
> > > > > /* slot descriptor has been composed, add it into parent context */
> > > > > aml_append(parent_scope, dev);
> > > > > @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> > > > > if (bus) {
> > > > > Aml *scope = aml_scope("PCI0");
> > > > > /* Scan all PCI buses. Generate tables to support hotplug. */
> > > > > - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> > > > > + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> > > > > + pm->pcihup_bridge_en);
> > > > >
> > > > > if (TPM_IS_TIS_ISA(tpm)) {
> > > > > if (misc->tpm_version == TPM_VERSION_2_0) {
> > > > > --
> > > > > 1.9.4
> > > >
> > > >
> > >
> > > Regards,
> > > Daniel
> > > --
> > > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> > > |: https://libvirt.org -o- https://fstop138.berrange.com :|
> > > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
> >
>
> Regards,
> Daniel
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-04-28 10:16 [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses Ani Sinha
2020-04-28 10:20 ` Ani Sinha
2020-04-28 16:05 ` Michael S. Tsirkin
@ 2020-05-11 18:53 ` Igor Mammedov
2020-05-12 5:26 ` Ani Sinha
2 siblings, 1 reply; 52+ messages in thread
From: Igor Mammedov @ 2020-05-11 18:53 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, Michael S. Tsirkin, qemu-devel,
Aleksandar Markovic, ani, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, 28 Apr 2020 10:16:52 +0000
Ani Sinha <ani.sinha@nutanix.com> wrote:
> A new option "use_acpi_unplug" is introduced for PIIX which will
> selectively only disable hot unplugging of both hot plugged and
> cold plugged PCI devices on non-root PCI buses. This will prevent
> hot unplugging of devices from Windows based guests from system
> tray but will not prevent devices from being hot plugged into the
> guest.
>
> It has been tested on Windows guests.
>
> Signed-off-by: Ani Sinha <ani.sinha@nutanix.com>
> ---
> hw/acpi/piix4.c | 3 +++
> hw/i386/acpi-build.c | 40 ++++++++++++++++++++++++++--------------
> 2 files changed, 29 insertions(+), 14 deletions(-)
>
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 964d6f5..59fa707 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
>
> AcpiPciHpState acpi_pci_hotplug;
> bool use_acpi_pci_hotplug;
> + bool use_acpi_unplug;
>
> uint8_t disable_s3;
> uint8_t disable_s4;
> @@ -633,6 +634,8 @@ static Property piix4_pm_properties[] = {
> DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> use_acpi_pci_hotplug, true),
> + DEFINE_PROP_BOOL("acpi-pci-hotunplug-enable-bridge", PIIX4PMState,
> + use_acpi_unplug, true),
> DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> acpi_memory_hotplug.is_enabled, true),
> DEFINE_PROP_END_OF_LIST(),
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 23c77ee..71b3ac3 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -96,6 +96,7 @@ typedef struct AcpiPmInfo {
> bool s3_disabled;
> bool s4_disabled;
> bool pcihp_bridge_en;
> + bool pcihup_bridge_en;
> uint8_t s4_val;
> AcpiFadtData fadt;
> uint16_t cpu_hp_io_base;
> @@ -240,6 +241,9 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
> pm->pcihp_bridge_en =
> object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support",
> NULL);
> + pm->pcihup_bridge_en =
> + object_property_get_bool(obj, "acpi-pci-hotunplug-enable-bridge",
> + NULL);
> }
>
> static void acpi_get_misc_info(AcpiMiscInfo *info)
> @@ -451,7 +455,8 @@ static void build_append_pcihp_notify_entry(Aml *method, int slot)
> }
>
> static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> - bool pcihp_bridge_en)
> + bool pcihp_bridge_en,
> + bool pcihup_bridge_en)
> {
> Aml *dev, *notify_method = NULL, *method;
> QObject *bsel;
> @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> - aml_append(method,
> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> - );
> - aml_append(dev, method);
> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
so you are keeping unplug anyway in case of host bridge, so user will see
eject icon if device is in root bus?
Other thing about this patch is that it only partially disable hotplug,
I'd rather do it the way hardware does i.e. full hotplug or no hotplug at all.
(like the other hypervisors have done it, to workaround this Windows 'feature')
which is possible is one puts device on pci bridge without hotplug, i.e.
-global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
that of cause leaves apci hotplug on and as you noticed earlier
Windows will offer to eject any device on root bus including directly
attached bridges. And currently there is no way to disable that.
Will following hack work for you?
possible permutations
1) ACPI hotplug everywhere
-global PIIX4_PM.acpi-pci-hotplug=on -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=doesnt_matter -device e1000,bus=pci.1,addr=01,id=netdev1
2) No hotplug at all
-global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=off -device e1000,bus=pci.1,addr=01,id=netdev1
-global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device pci-bridge,chassis_nr=1,shpc=doesnt_matter -device e1000,bus=pci.1,addr=01,id=netdev1
3) looks like SHPC kicks in, but it still needs to some bridge description in ACPI that
acpi-pci-hotplug-with-bridge-support provides, probably with this you can individually flip hotplug on
colplugged bridges using 'shpc' property (requires Vista or newer, tested win10).
This needs some investigation so we could remove unsed AML and IO ports, but I'm not really interested
in PCI stuff. So if 1+2 works for you, I'll post formal patches. If #3 is required feel free
to use this patch as a starting base to make it complete.
-global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=on -device e1000,bus=pci.1,addr=01,id=netdev1
---
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 964d6f5990..5f05b2cb87 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
AcpiPciHpState acpi_pci_hotplug;
bool use_acpi_pci_hotplug;
+ bool use_acpi_pci_hotplug_on_bridges;
uint8_t disable_s3;
uint8_t disable_s4;
@@ -207,13 +208,13 @@ static const VMStateDescription vmstate_pci_status = {
static bool vmstate_test_use_acpi_pci_hotplug(void *opaque, int version_id)
{
PIIX4PMState *s = opaque;
- return s->use_acpi_pci_hotplug;
+ return s->use_acpi_pci_hotplug_on_bridges;
}
static bool vmstate_test_no_use_acpi_pci_hotplug(void *opaque, int version_id)
{
PIIX4PMState *s = opaque;
- return !s->use_acpi_pci_hotplug;
+ return !s->use_acpi_pci_hotplug_on_bridges;
}
static bool vmstate_test_use_memhp(void *opaque)
@@ -505,7 +506,6 @@ static void piix4_pm_realize(PCIDevice *dev, Error **errp)
piix4_acpi_system_hot_add_init(pci_address_space_io(dev),
pci_get_bus(dev), s);
- qbus_set_hotplug_handler(BUS(pci_get_bus(dev)), OBJECT(s), &error_abort);
piix4_pm_add_propeties(s);
}
@@ -528,7 +528,7 @@ I2CBus *piix4_pm_init(PCIBus *bus, int devfn, uint32_t smb_io_base,
s->smi_irq = smi_irq;
s->smm_enabled = smm_enabled;
if (xen_enabled()) {
- s->use_acpi_pci_hotplug = false;
+ s->use_acpi_pci_hotplug_on_bridges = false;
}
qdev_init_nofail(dev);
@@ -593,7 +593,10 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
memory_region_add_subregion(parent, GPE_BASE, &s->io_gpe);
acpi_pcihp_init(OBJECT(s), &s->acpi_pci_hotplug, bus, parent,
- s->use_acpi_pci_hotplug);
+ s->use_acpi_pci_hotplug_on_bridges);
+ if (s->use_acpi_pci_hotplug) {
+ qbus_set_hotplug_handler(BUS(bus), OBJECT(s), &error_abort);
+ }
s->cpu_hotplug_legacy = true;
object_property_add_bool(OBJECT(s), "cpu-hotplug-legacy",
@@ -632,6 +635,8 @@ static Property piix4_pm_properties[] = {
DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_DISABLED, PIIX4PMState, disable_s4, 0),
DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
+ use_acpi_pci_hotplug_on_bridges, true),
+ DEFINE_PROP_BOOL("acpi-pci-hotplug", PIIX4PMState,
use_acpi_pci_hotplug, true),
DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
acpi_memory_hotplug.is_enabled, true),
---
> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> + aml_append(method,
> + aml_call2("PCEJ", aml_name("BSEL"),
> + aml_name("_SUN"))
> + );
> + aml_append(dev, method);
> + }
> aml_append(parent_scope, dev);
>
> build_append_pcihp_notify_entry(notify_method, slot);
> @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> /* add _SUN/_EJ0 to make slot hotpluggable */
> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
>
> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> - aml_append(method,
> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> - );
> - aml_append(dev, method);
> -
> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> + aml_append(method,
> + aml_call2("PCEJ", aml_name("BSEL"),
> + aml_name("_SUN"))
> + );
> + aml_append(dev, method);
> + }
> if (bsel) {
> build_append_pcihp_notify_entry(notify_method, slot);
> }
> @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> */
> PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
>
> - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> + pcihup_bridge_en);
> }
> /* slot descriptor has been composed, add it into parent context */
> aml_append(parent_scope, dev);
> @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> if (bus) {
> Aml *scope = aml_scope("PCI0");
> /* Scan all PCI buses. Generate tables to support hotplug. */
> - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> + pm->pcihup_bridge_en);
>
> if (TPM_IS_TIS_ISA(tpm)) {
> if (misc->tpm_version == TPM_VERSION_2_0) {
^ permalink raw reply related [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-11 18:53 ` Igor Mammedov
@ 2020-05-12 5:26 ` Ani Sinha
2020-05-13 19:43 ` Igor Mammedov
0 siblings, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-05-12 5:26 UTC (permalink / raw)
To: Igor Mammedov
Cc: Eduardo Habkost, Michael S. Tsirkin, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On May 12, 2020, at 12:23 AM, Igor Mammedov <imammedo@redhat.com> wrote:
>
>>
>> static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>> - bool pcihp_bridge_en)
>> + bool pcihp_bridge_en,
>> + bool pcihup_bridge_en)
>> {
>> Aml *dev, *notify_method = NULL, *method;
>> QObject *bsel;
>> @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>> dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
>> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
>> aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
>> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
>> - aml_append(method,
>> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
>> - );
>> - aml_append(dev, method);
>> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
>
> so you are keeping unplug anyway in case of host bridge, so user will see
> eject icon if device is in root bus?
Yes, the user will see the eject option from system tray for devices plugged into the root bus. The idea is that whereas we disallow some devices from hot-unplugging, other devices which are plugged into the root bus can be hot plugged and unplugged. This leaves some room for flexibility across devices and VMs.
>
> Other thing about this patch is that it only partially disable hotplug,
> I'd rather do it the way hardware does i.e. full hotplug or no hotplug at all.
> (like the other hypervisors have done it, to workaround this Windows 'feature’)
So the main objection against this patch is that with this option enabled, we are violating what real HW does and since we want emulated HW to mimic real HW behavior as close as possible, we are breaking this assumption. Am I correct?
>
> which is possible is one puts device on pci bridge without hotplug, i.e.
>
> -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
right.
>
> that of cause leaves apci hotplug on and as you noticed earlier
> Windows will offer to eject any device on root bus including directly
> attached bridges. And currently there is no way to disable that.
Right. However, I have tested that even though the PCI bridge shows up as a device in the “safely remove HW” option in the system tray, trying to eject a PCI bridge with devices attached will result in failure with the error message “this device is currently in use”.
>
> Will following hack work for you?
> possible permutations
> 1) ACPI hotplug everywhere
> -global PIIX4_PM.acpi-pci-hotplug=on -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=doesnt_matter -device e1000,bus=pci.1,addr=01,id=netdev1
>
> 2) No hotplug at all
> -global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=off -device e1000,bus=pci.1,addr=01,id=netdev1
>
> -global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device pci-bridge,chassis_nr=1,shpc=doesnt_matter -device e1000,bus=pci.1,addr=01,id=netdev1
Given that my patch is not acceptable, I’d prefer the following in the order of preference:
(a) Have an option to disable hot ejection of PCI-PCI bridge so that Windows does not even show this HW in the “safely remove HW” option. If we can do this then from OS perspective the GUI options will be same as what is available with PCIE/q35 - none of the devices will be hot ejectable if the hot plug option is turned off from the PCIE slots where devices are plugged into.
I looked at the code. It seems to manipulate ACPI tables of the empty slots of the root bus where no devices are attached (see comment "/* add hotplug slots for non present devices */ “). For cold plugged bridges, it recurses down to scan the slots of the bridge. Is it possible to disable hot plug for the slot to which the bridge is attached?
(b) Failing above, having a global option to disable all hot plug, including the 32 slots of the root bus would be good. However, this does not give us the flexibility we have with PCIE (that is, to hot plug a device, we can always plug it to a slot with hot plug enabled).
Thanks for looking into my requirement more seriously,
ani
>
> 3) looks like SHPC kicks in, but it still needs to some bridge description in ACPI that
> acpi-pci-hotplug-with-bridge-support provides, probably with this you can individually flip hotplug on
> colplugged bridges using 'shpc' property (requires Vista or newer, tested win10).
>
> This needs some investigation so we could remove unsed AML and IO ports, but I'm not really interested
> in PCI stuff. So if 1+2 works for you, I'll post formal patches. If #3 is required feel free
> to use this patch as a starting base to make it complete.
>
> -global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=on -device e1000,bus=pci.1,addr=01,id=netdev1
>
> ---
>
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 964d6f5990..5f05b2cb87 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
>
> AcpiPciHpState acpi_pci_hotplug;
> bool use_acpi_pci_hotplug;
> + bool use_acpi_pci_hotplug_on_bridges;
>
> uint8_t disable_s3;
> uint8_t disable_s4;
> @@ -207,13 +208,13 @@ static const VMStateDescription vmstate_pci_status = {
> static bool vmstate_test_use_acpi_pci_hotplug(void *opaque, int version_id)
> {
> PIIX4PMState *s = opaque;
> - return s->use_acpi_pci_hotplug;
> + return s->use_acpi_pci_hotplug_on_bridges;
> }
>
> static bool vmstate_test_no_use_acpi_pci_hotplug(void *opaque, int version_id)
> {
> PIIX4PMState *s = opaque;
> - return !s->use_acpi_pci_hotplug;
> + return !s->use_acpi_pci_hotplug_on_bridges;
> }
>
> static bool vmstate_test_use_memhp(void *opaque)
> @@ -505,7 +506,6 @@ static void piix4_pm_realize(PCIDevice *dev, Error **errp)
>
> piix4_acpi_system_hot_add_init(pci_address_space_io(dev),
> pci_get_bus(dev), s);
> - qbus_set_hotplug_handler(BUS(pci_get_bus(dev)), OBJECT(s), &error_abort);
>
> piix4_pm_add_propeties(s);
> }
> @@ -528,7 +528,7 @@ I2CBus *piix4_pm_init(PCIBus *bus, int devfn, uint32_t smb_io_base,
> s->smi_irq = smi_irq;
> s->smm_enabled = smm_enabled;
> if (xen_enabled()) {
> - s->use_acpi_pci_hotplug = false;
> + s->use_acpi_pci_hotplug_on_bridges = false;
> }
>
> qdev_init_nofail(dev);
> @@ -593,7 +593,10 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> memory_region_add_subregion(parent, GPE_BASE, &s->io_gpe);
>
> acpi_pcihp_init(OBJECT(s), &s->acpi_pci_hotplug, bus, parent,
> - s->use_acpi_pci_hotplug);
> + s->use_acpi_pci_hotplug_on_bridges);
> + if (s->use_acpi_pci_hotplug) {
> + qbus_set_hotplug_handler(BUS(bus), OBJECT(s), &error_abort);
> + }
>
> s->cpu_hotplug_legacy = true;
> object_property_add_bool(OBJECT(s), "cpu-hotplug-legacy",
> @@ -632,6 +635,8 @@ static Property piix4_pm_properties[] = {
> DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_DISABLED, PIIX4PMState, disable_s4, 0),
> DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> + use_acpi_pci_hotplug_on_bridges, true),
> + DEFINE_PROP_BOOL("acpi-pci-hotplug", PIIX4PMState,
> use_acpi_pci_hotplug, true),
> DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> acpi_memory_hotplug.is_enabled, true),
>
> ---
>
>
>> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
>> + aml_append(method,
>> + aml_call2("PCEJ", aml_name("BSEL"),
>> + aml_name("_SUN"))
>> + );
>> + aml_append(dev, method);
>> + }
>> aml_append(parent_scope, dev);
>>
>> build_append_pcihp_notify_entry(notify_method, slot);
>> @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>> /* add _SUN/_EJ0 to make slot hotpluggable */
>> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
>>
>> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
>> - aml_append(method,
>> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
>> - );
>> - aml_append(dev, method);
>> -
>> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
>> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
>> + aml_append(method,
>> + aml_call2("PCEJ", aml_name("BSEL"),
>> + aml_name("_SUN"))
>> + );
>> + aml_append(dev, method);
>> + }
>> if (bsel) {
>> build_append_pcihp_notify_entry(notify_method, slot);
>> }
>> @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>> */
>> PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
>>
>> - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
>> + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
>> + pcihup_bridge_en);
>> }
>> /* slot descriptor has been composed, add it into parent context */
>> aml_append(parent_scope, dev);
>> @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
>> if (bus) {
>> Aml *scope = aml_scope("PCI0");
>> /* Scan all PCI buses. Generate tables to support hotplug. */
>> - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
>> + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
>> + pm->pcihup_bridge_en);
>>
>> if (TPM_IS_TIS_ISA(tpm)) {
>> if (misc->tpm_version == TPM_VERSION_2_0) {
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-12 5:26 ` Ani Sinha
@ 2020-05-13 19:43 ` Igor Mammedov
2020-05-15 12:13 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Igor Mammedov @ 2020-05-13 19:43 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, Michael S. Tsirkin, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Tue, 12 May 2020 05:26:47 +0000
Ani Sinha <ani.sinha@nutanix.com> wrote:
> > On May 12, 2020, at 12:23 AM, Igor Mammedov <imammedo@redhat.com> wrote:
> >
> >>
> >> static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> >> - bool pcihp_bridge_en)
> >> + bool pcihp_bridge_en,
> >> + bool pcihup_bridge_en)
> >> {
> >> Aml *dev, *notify_method = NULL, *method;
> >> QObject *bsel;
> >> @@ -479,11 +484,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> >> dev = aml_device("S%.02X", PCI_DEVFN(slot, 0));
> >> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> >> aml_append(dev, aml_name_decl("_ADR", aml_int(slot << 16)));
> >> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> >> - aml_append(method,
> >> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> >> - );
> >> - aml_append(dev, method);
> >> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> >
> > so you are keeping unplug anyway in case of host bridge, so user will see
> > eject icon if device is in root bus?
>
> Yes, the user will see the eject option from system tray for devices plugged into the root bus. The idea is that whereas we disallow some devices from hot-unplugging, other devices which are plugged into the root bus can be hot plugged and unplugged. This leaves some room for flexibility across devices and VMs.
>
> >
> > Other thing about this patch is that it only partially disable hotplug,
> > I'd rather do it the way hardware does i.e. full hotplug or no hotplug at all.
> > (like the other hypervisors have done it, to workaround this Windows 'feature’)
>
> So the main objection against this patch is that with this option enabled, we are violating what real HW does and since we want emulated HW to mimic real HW behavior as close as possible, we are breaking this assumption. Am I correct?
Yep, from the very begining I was suggesting to use global hotplug/no hotplug switch.
> > which is possible is one puts device on pci bridge without hotplug, i.e.
> >
> > -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
>
> right.
>
> >
> > that of cause leaves apci hotplug on and as you noticed earlier
> > Windows will offer to eject any device on root bus including directly
> > attached bridges. And currently there is no way to disable that.
>
> Right. However, I have tested that even though the PCI bridge shows up as a device in the “safely remove HW” option in the system tray, trying to eject a PCI bridge with devices attached will result in failure with the error message “this device is currently in use”.
>
> > Will following hack work for you?
> > possible permutations
> > 1) ACPI hotplug everywhere
> > -global PIIX4_PM.acpi-pci-hotplug=on -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=doesnt_matter -device e1000,bus=pci.1,addr=01,id=netdev1
> >
> > 2) No hotplug at all
> > -global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=off -device e1000,bus=pci.1,addr=01,id=netdev1
> >
> > -global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device pci-bridge,chassis_nr=1,shpc=doesnt_matter -device e1000,bus=pci.1,addr=01,id=netdev1
>
> Given that my patch is not acceptable, I’d prefer the following in the order of preference:
>
> (a) Have an option to disable hot ejection of PCI-PCI bridge so that Windows does not even show this HW in the “safely remove HW” option. If we can do this then from OS perspective the GUI options will be same as what is available with PCIE/q35 - none of the devices will be hot ejectable if the hot plug option is turned off from the PCIE slots where devices are plugged into.
> I looked at the code. It seems to manipulate ACPI tables of the empty slots of the root bus where no devices are attached (see comment "/* add hotplug slots for non present devices */ “). For cold plugged bridges, it recurses down to scan the slots of the bridge. Is it possible to disable hot plug for the slot to which the bridge is attached?
I don't think it's possible to have per slot hotplug on conventional PCI hardware.
it's per bridge property.
> (b) Failing above, having a global option to disable all hot plug, including the 32 slots of the root bus would be good. However, this does not give us the flexibility we have with PCIE (that is, to hot plug a device, we can always plug it to a slot with hot plug enabled).
sounds fine to me, at least it will address problem.
Can you post a patch to that effect please?
/It should disable all AML related to hotplug and related hadware code/
As for a way to make it more felixible, theoretically it should be possible to use SHPC on bridges.
So one could have a bridge with SHPC enabled to allow hotplug devices during runtime.
It's rummored that SHPC should be supported since Vista, but it doesn't work
(at least on QEMU with Windows 10 guest and I don't have a real machine to verify that).
That's something to explore (perhaps QEMU doesn't implement it completely so Windows doesn't see it).
>
> Thanks for looking into my requirement more seriously,
> ani
>
>
> >
> > 3) looks like SHPC kicks in, but it still needs to some bridge description in ACPI that
> > acpi-pci-hotplug-with-bridge-support provides, probably with this you can individually flip hotplug on
> > colplugged bridges using 'shpc' property (requires Vista or newer, tested win10).
> >
> > This needs some investigation so we could remove unsed AML and IO ports, but I'm not really interested
> > in PCI stuff. So if 1+2 works for you, I'll post formal patches. If #3 is required feel free
> > to use this patch as a starting base to make it complete.
> >
> > -global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=on -device e1000,bus=pci.1,addr=01,id=netdev1
above is a bug in patch below, look like ,shpc=on tricks code to belive that ACPI hotplug is present
and add AML for it.
> >
> > ---
> >
> > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > index 964d6f5990..5f05b2cb87 100644
> > --- a/hw/acpi/piix4.c
> > +++ b/hw/acpi/piix4.c
> > @@ -78,6 +78,7 @@ typedef struct PIIX4PMState {
> >
> > AcpiPciHpState acpi_pci_hotplug;
> > bool use_acpi_pci_hotplug;
> > + bool use_acpi_pci_hotplug_on_bridges;
> >
> > uint8_t disable_s3;
> > uint8_t disable_s4;
> > @@ -207,13 +208,13 @@ static const VMStateDescription vmstate_pci_status = {
> > static bool vmstate_test_use_acpi_pci_hotplug(void *opaque, int version_id)
> > {
> > PIIX4PMState *s = opaque;
> > - return s->use_acpi_pci_hotplug;
> > + return s->use_acpi_pci_hotplug_on_bridges;
> > }
> >
> > static bool vmstate_test_no_use_acpi_pci_hotplug(void *opaque, int version_id)
> > {
> > PIIX4PMState *s = opaque;
> > - return !s->use_acpi_pci_hotplug;
> > + return !s->use_acpi_pci_hotplug_on_bridges;
> > }
> >
> > static bool vmstate_test_use_memhp(void *opaque)
> > @@ -505,7 +506,6 @@ static void piix4_pm_realize(PCIDevice *dev, Error **errp)
> >
> > piix4_acpi_system_hot_add_init(pci_address_space_io(dev),
> > pci_get_bus(dev), s);
> > - qbus_set_hotplug_handler(BUS(pci_get_bus(dev)), OBJECT(s), &error_abort);
> >
> > piix4_pm_add_propeties(s);
> > }
> > @@ -528,7 +528,7 @@ I2CBus *piix4_pm_init(PCIBus *bus, int devfn, uint32_t smb_io_base,
> > s->smi_irq = smi_irq;
> > s->smm_enabled = smm_enabled;
> > if (xen_enabled()) {
> > - s->use_acpi_pci_hotplug = false;
> > + s->use_acpi_pci_hotplug_on_bridges = false;
> > }
> >
> > qdev_init_nofail(dev);
> > @@ -593,7 +593,10 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> > memory_region_add_subregion(parent, GPE_BASE, &s->io_gpe);
> >
> > acpi_pcihp_init(OBJECT(s), &s->acpi_pci_hotplug, bus, parent,
> > - s->use_acpi_pci_hotplug);
> > + s->use_acpi_pci_hotplug_on_bridges);
> > + if (s->use_acpi_pci_hotplug) {
> > + qbus_set_hotplug_handler(BUS(bus), OBJECT(s), &error_abort);
> > + }
> >
> > s->cpu_hotplug_legacy = true;
> > object_property_add_bool(OBJECT(s), "cpu-hotplug-legacy",
> > @@ -632,6 +635,8 @@ static Property piix4_pm_properties[] = {
> > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_DISABLED, PIIX4PMState, disable_s4, 0),
> > DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2),
> > DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState,
> > + use_acpi_pci_hotplug_on_bridges, true),
> > + DEFINE_PROP_BOOL("acpi-pci-hotplug", PIIX4PMState,
> > use_acpi_pci_hotplug, true),
> > DEFINE_PROP_BOOL("memory-hotplug-support", PIIX4PMState,
> > acpi_memory_hotplug.is_enabled, true),
> >
> > ---
> >
> >
> >> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> >> + aml_append(method,
> >> + aml_call2("PCEJ", aml_name("BSEL"),
> >> + aml_name("_SUN"))
> >> + );
> >> + aml_append(dev, method);
> >> + }
> >> aml_append(parent_scope, dev);
> >>
> >> build_append_pcihp_notify_entry(notify_method, slot);
> >> @@ -537,12 +545,14 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> >> /* add _SUN/_EJ0 to make slot hotpluggable */
> >> aml_append(dev, aml_name_decl("_SUN", aml_int(slot)));
> >>
> >> - method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> >> - aml_append(method,
> >> - aml_call2("PCEJ", aml_name("BSEL"), aml_name("_SUN"))
> >> - );
> >> - aml_append(dev, method);
> >> -
> >> + if (pcihup_bridge_en || pci_bus_is_root(bus)) {
> >> + method = aml_method("_EJ0", 1, AML_NOTSERIALIZED);
> >> + aml_append(method,
> >> + aml_call2("PCEJ", aml_name("BSEL"),
> >> + aml_name("_SUN"))
> >> + );
> >> + aml_append(dev, method);
> >> + }
> >> if (bsel) {
> >> build_append_pcihp_notify_entry(notify_method, slot);
> >> }
> >> @@ -553,7 +563,8 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
> >> */
> >> PCIBus *sec_bus = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
> >>
> >> - build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en);
> >> + build_append_pci_bus_devices(dev, sec_bus, pcihp_bridge_en,
> >> + pcihup_bridge_en);
> >> }
> >> /* slot descriptor has been composed, add it into parent context */
> >> aml_append(parent_scope, dev);
> >> @@ -2196,7 +2207,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> >> if (bus) {
> >> Aml *scope = aml_scope("PCI0");
> >> /* Scan all PCI buses. Generate tables to support hotplug. */
> >> - build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en);
> >> + build_append_pci_bus_devices(scope, bus, pm->pcihp_bridge_en,
> >> + pm->pcihup_bridge_en);
> >>
> >> if (TPM_IS_TIS_ISA(tpm)) {
> >> if (misc->tpm_version == TPM_VERSION_2_0) {
>
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-13 19:43 ` Igor Mammedov
@ 2020-05-15 12:13 ` Ani Sinha
2020-05-20 9:43 ` Igor Mammedow
0 siblings, 1 reply; 52+ messages in thread
From: Ani Sinha @ 2020-05-15 12:13 UTC (permalink / raw)
To: Igor Mammedov
Cc: Eduardo Habkost, Michael S. Tsirkin, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com> wrote:
>
>>
>>
>>> Will following hack work for you?
>>> possible permutations
>>> 1) ACPI hotplug everywhere
>>> -global PIIX4_PM.acpi-pci-hotplug=on -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=doesnt_matter -device e1000,bus=pci.1,addr=01,id=netdev1
>>>
>>> 2) No hotplug at all
>>> -global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device pci-bridge,chassis_nr=1,shpc=off -device e1000,bus=pci.1,addr=01,id=netdev1
>>>
>>> -global PIIX4_PM.acpi-pci-hotplug=off -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device pci-bridge,chassis_nr=1,shpc=doesnt_matter -device e1000,bus=pci.1,addr=01,id=netdev1
>>
>> Given that my patch is not acceptable, I’d prefer the following in the order of preference:
>>
>> (a) Have an option to disable hot ejection of PCI-PCI bridge so that Windows does not even show this HW in the “safely remove HW” option. If we can do this then from OS perspective the GUI options will be same as what is available with PCIE/q35 - none of the devices will be hot ejectable if the hot plug option is turned off from the PCIE slots where devices are plugged into.
>> I looked at the code. It seems to manipulate ACPI tables of the empty slots of the root bus where no devices are attached (see comment "/* add hotplug slots for non present devices */ “). For cold plugged bridges, it recurses down to scan the slots of the bridge. Is it possible to disable hot plug for the slot to which the bridge is attached?
>
> I don't think it's possible to have per slot hotplug on conventional PCI hardware.
> it's per bridge property.
We add the AMLs per empty slot though. When the pic bridge is attached, we do nothing, just recurse into the bridge slots. That is what I was asking, if it was possible to just disable the AMLs or use some tricks to say that this particular slot is not hotpluggable. I am not sure why Windows is trying to eject the PCI bridge and failing. Maybe something related to this comment?
/* When hotplug for bridges is enabled, bridges are
* described in ACPI separately (see build_pci_bus_end).
* In this case they aren't themselves hot-pluggable.
* Hotplugged bridges *are* hot-pluggable.
*/
>
>
>> (b) Failing above, having a global option to disable all hot plug, including the 32 slots of the root bus would be good. However, this does not give us the flexibility we have with PCIE (that is, to hot plug a device, we can always plug it to a slot with hot plug enabled).
>
> sounds fine to me, at least it will address problem.
> Can you post a patch to that effect please?
> /It should disable all AML related to hotplug and related hadware code/
I have tried a patch and sent it.
^ permalink raw reply [flat|nested] 52+ messages in thread* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-15 12:13 ` Ani Sinha
@ 2020-05-20 9:43 ` Igor Mammedow
2020-05-20 9:47 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Igor Mammedow @ 2020-05-20 9:43 UTC (permalink / raw)
To: Ani Sinha
Cc: Eduardo Habkost, Michael S. Tsirkin, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Fri, 15 May 2020 12:13:53 +0000
Ani Sinha <ani.sinha@nutanix.com> wrote:
> > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > wrote:
> >>
> >>
> >>> Will following hack work for you?
> >>> possible permutations
> >>> 1) ACPI hotplug everywhere
> >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> >>> e1000,bus=pci.1,addr=01,id=netdev1
> >>>
> >>> 2) No hotplug at all
> >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> >>> pci-bridge,chassis_nr=1,shpc=off -device
> >>> e1000,bus=pci.1,addr=01,id=netdev1
> >>>
> >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> >>> e1000,bus=pci.1,addr=01,id=netdev1
> >>
> >> Given that my patch is not acceptable, I’d prefer the following in
> >> the order of preference:
> >>
> >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> >> that Windows does not even show this HW in the “safely remove HW”
> >> option. If we can do this then from OS perspective the GUI options
> >> will be same as what is available with PCIE/q35 - none of the
> >> devices will be hot ejectable if the hot plug option is turned off
> >> from the PCIE slots where devices are plugged into. I looked at
> >> the code. It seems to manipulate ACPI tables of the empty slots of
> >> the root bus where no devices are attached (see comment "/* add
> >> hotplug slots for non present devices */ “). For cold plugged
> >> bridges, it recurses down to scan the slots of the bridge. Is it
> >> possible to disable hot plug for the slot to which the bridge is
> >> attached?
> >
> > I don't think it's possible to have per slot hotplug on
> > conventional PCI hardware. it's per bridge property.
>
> We add the AMLs per empty slot though. When the pic bridge is
> attached, we do nothing, just recurse into the bridge slots. That is
> what I was asking, if it was possible to just disable the AMLs or use
> some tricks to say that this particular slot is not hotpluggable. I
> am not sure why Windows is trying to eject the PCI bridge and
> failing. Maybe something related to this comment?
>
>
> /* When hotplug for bridges is enabled, bridges are
>
> * described in ACPI separately (see build_pci_bus_end).
>
> * In this case they aren't themselves hot-pluggable.
>
> * Hotplugged bridges *are* hot-pluggable.
> */
thinking some more on this topic, it seems that with ACPI hotplug we
already have implicit non-hotpluggble slot (slot with bridge) while
the rest are staying hotpluggable.
So my question is: if it's acceptable to add 'PCIDevice::hotpluggable"
property to all PCI devices so that user / libvirt could set it to
false in case they do not want coldplugged device be considered as
hotpluggable? (this way other devices could be treated the same way as
bridges)
[...]
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 9:43 ` Igor Mammedow
@ 2020-05-20 9:47 ` Michael S. Tsirkin
2020-05-20 9:56 ` Igor Mammedow
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-05-20 9:47 UTC (permalink / raw)
To: Igor Mammedow
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> On Fri, 15 May 2020 12:13:53 +0000
> Ani Sinha <ani.sinha@nutanix.com> wrote:
>
> > > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > > wrote:
> > >>
> > >>
> > >>> Will following hack work for you?
> > >>> possible permutations
> > >>> 1) ACPI hotplug everywhere
> > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > >>>
> > >>> 2) No hotplug at all
> > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > >>>
> > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > >>
> > >> Given that my patch is not acceptable, I’d prefer the following in
> > >> the order of preference:
> > >>
> > >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> > >> that Windows does not even show this HW in the “safely remove HW”
> > >> option. If we can do this then from OS perspective the GUI options
> > >> will be same as what is available with PCIE/q35 - none of the
> > >> devices will be hot ejectable if the hot plug option is turned off
> > >> from the PCIE slots where devices are plugged into. I looked at
> > >> the code. It seems to manipulate ACPI tables of the empty slots of
> > >> the root bus where no devices are attached (see comment "/* add
> > >> hotplug slots for non present devices */ “). For cold plugged
> > >> bridges, it recurses down to scan the slots of the bridge. Is it
> > >> possible to disable hot plug for the slot to which the bridge is
> > >> attached?
> > >
> > > I don't think it's possible to have per slot hotplug on
> > > conventional PCI hardware. it's per bridge property.
> >
> > We add the AMLs per empty slot though. When the pic bridge is
> > attached, we do nothing, just recurse into the bridge slots. That is
> > what I was asking, if it was possible to just disable the AMLs or use
> > some tricks to say that this particular slot is not hotpluggable. I
> > am not sure why Windows is trying to eject the PCI bridge and
> > failing. Maybe something related to this comment?
> >
> >
> > /* When hotplug for bridges is enabled, bridges are
> >
> > * described in ACPI separately (see build_pci_bus_end).
> >
> > * In this case they aren't themselves hot-pluggable.
> >
> > * Hotplugged bridges *are* hot-pluggable.
> > */
>
> thinking some more on this topic, it seems that with ACPI hotplug we
> already have implicit non-hotpluggble slot (slot with bridge) while
> the rest are staying hotpluggable.
>
> So my question is: if it's acceptable to add 'PCIDevice::hotpluggable"
> property to all PCI devices so that user / libvirt could set it to
> false in case they do not want coldplugged device be considered as
> hotpluggable? (this way other devices could be treated the same way as
> bridges)
>
> [...]
I think Julia already posted a patch adding this to downstream pcie
bridges. Adding this to pci slots sounds like a reasonable thing.
--
MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 9:47 ` Michael S. Tsirkin
@ 2020-05-20 9:56 ` Igor Mammedow
2020-05-20 10:06 ` Daniel P. Berrangé
2020-05-20 10:28 ` Michael S. Tsirkin
0 siblings, 2 replies; 52+ messages in thread
From: Igor Mammedow @ 2020-05-20 9:56 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, 20 May 2020 05:47:53 -0400
"Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > On Fri, 15 May 2020 12:13:53 +0000
> > Ani Sinha <ani.sinha@nutanix.com> wrote:
> >
> > > > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > > > wrote:
> > > >>
> > > >>
> > > >>> Will following hack work for you?
> > > >>> possible permutations
> > > >>> 1) ACPI hotplug everywhere
> > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > >>>
> > > >>> 2) No hotplug at all
> > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > >>>
> > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > >>
> > > >> Given that my patch is not acceptable, I’d prefer the
> > > >> following in the order of preference:
> > > >>
> > > >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> > > >> that Windows does not even show this HW in the “safely remove
> > > >> HW” option. If we can do this then from OS perspective the GUI
> > > >> options will be same as what is available with PCIE/q35 - none
> > > >> of the devices will be hot ejectable if the hot plug option is
> > > >> turned off from the PCIE slots where devices are plugged into.
> > > >> I looked at the code. It seems to manipulate ACPI tables of
> > > >> the empty slots of the root bus where no devices are attached
> > > >> (see comment "/* add hotplug slots for non present devices */
> > > >> “). For cold plugged bridges, it recurses down to scan the
> > > >> slots of the bridge. Is it possible to disable hot plug for
> > > >> the slot to which the bridge is attached?
> > > >
> > > > I don't think it's possible to have per slot hotplug on
> > > > conventional PCI hardware. it's per bridge property.
> > >
> > > We add the AMLs per empty slot though. When the pic bridge is
> > > attached, we do nothing, just recurse into the bridge slots. That
> > > is what I was asking, if it was possible to just disable the AMLs
> > > or use some tricks to say that this particular slot is not
> > > hotpluggable. I am not sure why Windows is trying to eject the
> > > PCI bridge and failing. Maybe something related to this comment?
> > >
> > >
> > > /* When hotplug for bridges is enabled, bridges are
> > >
> > > * described in ACPI separately (see build_pci_bus_end).
> > >
> > > * In this case they aren't themselves hot-pluggable.
> > >
> > > * Hotplugged bridges *are* hot-pluggable.
> > > */
> >
> > thinking some more on this topic, it seems that with ACPI hotplug we
> > already have implicit non-hotpluggble slot (slot with bridge) while
> > the rest are staying hotpluggable.
> >
> > So my question is: if it's acceptable to add
> > 'PCIDevice::hotpluggable" property to all PCI devices so that user
> > / libvirt could set it to false in case they do not want
> > coldplugged device be considered as hotpluggable? (this way other
> > devices could be treated the same way as bridges)
> >
> > [...]
>
>
> I think Julia already posted a patch adding this to downstream pcie
> bridges. Adding this to pci slots sounds like a reasonable thing.
Question was more about external interface, were we do not have ports
as separate devices with conventional PCI. The only knob we have is a
a PCI device, where we have a property to turn on/off hotplug. ex:
-device e1000,hotpluggable=off
and if libvirt would be able to use it
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 9:56 ` Igor Mammedow
@ 2020-05-20 10:06 ` Daniel P. Berrangé
2020-05-20 11:29 ` Michael S. Tsirkin
2020-05-20 10:28 ` Michael S. Tsirkin
1 sibling, 1 reply; 52+ messages in thread
From: Daniel P. Berrangé @ 2020-05-20 10:06 UTC (permalink / raw)
To: Igor Mammedow
Cc: Ani Sinha, Eduardo Habkost, Michael S. Tsirkin, Julia Suvorova,
qemu-devel, Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> On Wed, 20 May 2020 05:47:53 -0400
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > > On Fri, 15 May 2020 12:13:53 +0000
> > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > >
> > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > > > > wrote:
> > > > >>
> > > > >>
> > > > >>> Will following hack work for you?
> > > > >>> possible permutations
> > > > >>> 1) ACPI hotplug everywhere
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > >>>
> > > > >>> 2) No hotplug at all
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > >>>
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > >>
> > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > >> following in the order of preference:
> > > > >>
> > > > >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> > > > >> that Windows does not even show this HW in the “safely remove
> > > > >> HW” option. If we can do this then from OS perspective the GUI
> > > > >> options will be same as what is available with PCIE/q35 - none
> > > > >> of the devices will be hot ejectable if the hot plug option is
> > > > >> turned off from the PCIE slots where devices are plugged into.
> > > > >> I looked at the code. It seems to manipulate ACPI tables of
> > > > >> the empty slots of the root bus where no devices are attached
> > > > >> (see comment "/* add hotplug slots for non present devices */
> > > > >> “). For cold plugged bridges, it recurses down to scan the
> > > > >> slots of the bridge. Is it possible to disable hot plug for
> > > > >> the slot to which the bridge is attached?
> > > > >
> > > > > I don't think it's possible to have per slot hotplug on
> > > > > conventional PCI hardware. it's per bridge property.
> > > >
> > > > We add the AMLs per empty slot though. When the pic bridge is
> > > > attached, we do nothing, just recurse into the bridge slots. That
> > > > is what I was asking, if it was possible to just disable the AMLs
> > > > or use some tricks to say that this particular slot is not
> > > > hotpluggable. I am not sure why Windows is trying to eject the
> > > > PCI bridge and failing. Maybe something related to this comment?
> > > >
> > > >
> > > > /* When hotplug for bridges is enabled, bridges are
> > > >
> > > > * described in ACPI separately (see build_pci_bus_end).
> > > >
> > > > * In this case they aren't themselves hot-pluggable.
> > > >
> > > > * Hotplugged bridges *are* hot-pluggable.
> > > > */
> > >
> > > thinking some more on this topic, it seems that with ACPI hotplug we
> > > already have implicit non-hotpluggble slot (slot with bridge) while
> > > the rest are staying hotpluggable.
> > >
> > > So my question is: if it's acceptable to add
> > > 'PCIDevice::hotpluggable" property to all PCI devices so that user
> > > / libvirt could set it to false in case they do not want
> > > coldplugged device be considered as hotpluggable? (this way other
> > > devices could be treated the same way as bridges)
> > >
> > > [...]
> >
> >
> > I think Julia already posted a patch adding this to downstream pcie
> > bridges. Adding this to pci slots sounds like a reasonable thing.
> Question was more about external interface, were we do not have ports
> as separate devices with conventional PCI. The only knob we have is a
> a PCI device, where we have a property to turn on/off hotplug. ex:
> -device e1000,hotpluggable=off
> and if libvirt would be able to use it
Libvirt can certainly use a property that is settable per-device,
instead of against the pcie-root-port. In fact the application
that requested hotplug control (KubeVirt) would strongly prefer
if this was doable per-device, because they really dislike the
idea of having to deal with pcie-root-port devices.
So if you added it per-device, libvirt would support both the
per-device option, and the pcie-root-port option.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 10:06 ` Daniel P. Berrangé
@ 2020-05-20 11:29 ` Michael S. Tsirkin
2020-05-20 11:42 ` Daniel P. Berrangé
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-05-20 11:29 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Paolo Bonzini, Ani Sinha, Igor Mammedow,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, May 20, 2020 at 11:06:28AM +0100, Daniel P. Berrangé wrote:
> On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> > On Wed, 20 May 2020 05:47:53 -0400
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >
> > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > >
> > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > > > > > wrote:
> > > > > >>
> > > > > >>
> > > > > >>> Will following hack work for you?
> > > > > >>> possible permutations
> > > > > >>> 1) ACPI hotplug everywhere
> > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > >>>
> > > > > >>> 2) No hotplug at all
> > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > >>>
> > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > >>
> > > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > > >> following in the order of preference:
> > > > > >>
> > > > > >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> > > > > >> that Windows does not even show this HW in the “safely remove
> > > > > >> HW” option. If we can do this then from OS perspective the GUI
> > > > > >> options will be same as what is available with PCIE/q35 - none
> > > > > >> of the devices will be hot ejectable if the hot plug option is
> > > > > >> turned off from the PCIE slots where devices are plugged into.
> > > > > >> I looked at the code. It seems to manipulate ACPI tables of
> > > > > >> the empty slots of the root bus where no devices are attached
> > > > > >> (see comment "/* add hotplug slots for non present devices */
> > > > > >> “). For cold plugged bridges, it recurses down to scan the
> > > > > >> slots of the bridge. Is it possible to disable hot plug for
> > > > > >> the slot to which the bridge is attached?
> > > > > >
> > > > > > I don't think it's possible to have per slot hotplug on
> > > > > > conventional PCI hardware. it's per bridge property.
> > > > >
> > > > > We add the AMLs per empty slot though. When the pic bridge is
> > > > > attached, we do nothing, just recurse into the bridge slots. That
> > > > > is what I was asking, if it was possible to just disable the AMLs
> > > > > or use some tricks to say that this particular slot is not
> > > > > hotpluggable. I am not sure why Windows is trying to eject the
> > > > > PCI bridge and failing. Maybe something related to this comment?
> > > > >
> > > > >
> > > > > /* When hotplug for bridges is enabled, bridges are
> > > > >
> > > > > * described in ACPI separately (see build_pci_bus_end).
> > > > >
> > > > > * In this case they aren't themselves hot-pluggable.
> > > > >
> > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > */
> > > >
> > > > thinking some more on this topic, it seems that with ACPI hotplug we
> > > > already have implicit non-hotpluggble slot (slot with bridge) while
> > > > the rest are staying hotpluggable.
> > > >
> > > > So my question is: if it's acceptable to add
> > > > 'PCIDevice::hotpluggable" property to all PCI devices so that user
> > > > / libvirt could set it to false in case they do not want
> > > > coldplugged device be considered as hotpluggable? (this way other
> > > > devices could be treated the same way as bridges)
> > > >
> > > > [...]
> > >
> > >
> > > I think Julia already posted a patch adding this to downstream pcie
> > > bridges. Adding this to pci slots sounds like a reasonable thing.
> > Question was more about external interface, were we do not have ports
> > as separate devices with conventional PCI. The only knob we have is a
> > a PCI device, where we have a property to turn on/off hotplug. ex:
> > -device e1000,hotpluggable=off
> > and if libvirt would be able to use it
>
> Libvirt can certainly use a property that is settable per-device,
> instead of against the pcie-root-port. In fact the application
> that requested hotplug control (KubeVirt) would strongly prefer
> if this was doable per-device, because they really dislike the
> idea of having to deal with pcie-root-port devices.
>
> So if you added it per-device, libvirt would support both the
> per-device option, and the pcie-root-port option.
>
> Regards,
> Daniel
The issue is that there then is no a way to check whether the option
is supported on a given slot, without simply trying.
This is why we added it on the port, there, presence of the option
is a signal that it will work.
Is this a concern for libvirt?
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 11:29 ` Michael S. Tsirkin
@ 2020-05-20 11:42 ` Daniel P. Berrangé
2020-05-20 11:44 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Daniel P. Berrangé @ 2020-05-20 11:42 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Paolo Bonzini, Ani Sinha, Igor Mammedow,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, May 20, 2020 at 07:29:25AM -0400, Michael S. Tsirkin wrote:
> On Wed, May 20, 2020 at 11:06:28AM +0100, Daniel P. Berrangé wrote:
> > On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> > > On Wed, 20 May 2020 05:47:53 -0400
> > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > >
> > > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > > >
> > > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > > > > > > wrote:
> > > > > > >>
> > > > > > >>
> > > > > > >>> Will following hack work for you?
> > > > > > >>> possible permutations
> > > > > > >>> 1) ACPI hotplug everywhere
> > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > >>>
> > > > > > >>> 2) No hotplug at all
> > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > >>>
> > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > >>
> > > > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > > > >> following in the order of preference:
> > > > > > >>
> > > > > > >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> > > > > > >> that Windows does not even show this HW in the “safely remove
> > > > > > >> HW” option. If we can do this then from OS perspective the GUI
> > > > > > >> options will be same as what is available with PCIE/q35 - none
> > > > > > >> of the devices will be hot ejectable if the hot plug option is
> > > > > > >> turned off from the PCIE slots where devices are plugged into.
> > > > > > >> I looked at the code. It seems to manipulate ACPI tables of
> > > > > > >> the empty slots of the root bus where no devices are attached
> > > > > > >> (see comment "/* add hotplug slots for non present devices */
> > > > > > >> “). For cold plugged bridges, it recurses down to scan the
> > > > > > >> slots of the bridge. Is it possible to disable hot plug for
> > > > > > >> the slot to which the bridge is attached?
> > > > > > >
> > > > > > > I don't think it's possible to have per slot hotplug on
> > > > > > > conventional PCI hardware. it's per bridge property.
> > > > > >
> > > > > > We add the AMLs per empty slot though. When the pic bridge is
> > > > > > attached, we do nothing, just recurse into the bridge slots. That
> > > > > > is what I was asking, if it was possible to just disable the AMLs
> > > > > > or use some tricks to say that this particular slot is not
> > > > > > hotpluggable. I am not sure why Windows is trying to eject the
> > > > > > PCI bridge and failing. Maybe something related to this comment?
> > > > > >
> > > > > >
> > > > > > /* When hotplug for bridges is enabled, bridges are
> > > > > >
> > > > > > * described in ACPI separately (see build_pci_bus_end).
> > > > > >
> > > > > > * In this case they aren't themselves hot-pluggable.
> > > > > >
> > > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > > */
> > > > >
> > > > > thinking some more on this topic, it seems that with ACPI hotplug we
> > > > > already have implicit non-hotpluggble slot (slot with bridge) while
> > > > > the rest are staying hotpluggable.
> > > > >
> > > > > So my question is: if it's acceptable to add
> > > > > 'PCIDevice::hotpluggable" property to all PCI devices so that user
> > > > > / libvirt could set it to false in case they do not want
> > > > > coldplugged device be considered as hotpluggable? (this way other
> > > > > devices could be treated the same way as bridges)
> > > > >
> > > > > [...]
> > > >
> > > >
> > > > I think Julia already posted a patch adding this to downstream pcie
> > > > bridges. Adding this to pci slots sounds like a reasonable thing.
> > > Question was more about external interface, were we do not have ports
> > > as separate devices with conventional PCI. The only knob we have is a
> > > a PCI device, where we have a property to turn on/off hotplug. ex:
> > > -device e1000,hotpluggable=off
> > > and if libvirt would be able to use it
> >
> > Libvirt can certainly use a property that is settable per-device,
> > instead of against the pcie-root-port. In fact the application
> > that requested hotplug control (KubeVirt) would strongly prefer
> > if this was doable per-device, because they really dislike the
> > idea of having to deal with pcie-root-port devices.
> >
> > So if you added it per-device, libvirt would support both the
> > per-device option, and the pcie-root-port option.
>
> The issue is that there then is no a way to check whether the option
> is supported on a given slot, without simply trying.
> This is why we added it on the port, there, presence of the option
> is a signal that it will work.
> Is this a concern for libvirt?
I'm not sure. What are the scenarios in which it is supported vs not
supported, as I'm unclear if they affect libvirt or not ?
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 11:42 ` Daniel P. Berrangé
@ 2020-05-20 11:44 ` Michael S. Tsirkin
0 siblings, 0 replies; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-05-20 11:44 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Paolo Bonzini, Ani Sinha, Igor Mammedow,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, May 20, 2020 at 12:42:57PM +0100, Daniel P. Berrangé wrote:
> On Wed, May 20, 2020 at 07:29:25AM -0400, Michael S. Tsirkin wrote:
> > On Wed, May 20, 2020 at 11:06:28AM +0100, Daniel P. Berrangé wrote:
> > > On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> > > > On Wed, 20 May 2020 05:47:53 -0400
> > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > >
> > > > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > > > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > > > >
> > > > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>> Will following hack work for you?
> > > > > > > >>> possible permutations
> > > > > > > >>> 1) ACPI hotplug everywhere
> > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > >>>
> > > > > > > >>> 2) No hotplug at all
> > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > >>>
> > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > >>
> > > > > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > > > > >> following in the order of preference:
> > > > > > > >>
> > > > > > > >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> > > > > > > >> that Windows does not even show this HW in the “safely remove
> > > > > > > >> HW” option. If we can do this then from OS perspective the GUI
> > > > > > > >> options will be same as what is available with PCIE/q35 - none
> > > > > > > >> of the devices will be hot ejectable if the hot plug option is
> > > > > > > >> turned off from the PCIE slots where devices are plugged into.
> > > > > > > >> I looked at the code. It seems to manipulate ACPI tables of
> > > > > > > >> the empty slots of the root bus where no devices are attached
> > > > > > > >> (see comment "/* add hotplug slots for non present devices */
> > > > > > > >> “). For cold plugged bridges, it recurses down to scan the
> > > > > > > >> slots of the bridge. Is it possible to disable hot plug for
> > > > > > > >> the slot to which the bridge is attached?
> > > > > > > >
> > > > > > > > I don't think it's possible to have per slot hotplug on
> > > > > > > > conventional PCI hardware. it's per bridge property.
> > > > > > >
> > > > > > > We add the AMLs per empty slot though. When the pic bridge is
> > > > > > > attached, we do nothing, just recurse into the bridge slots. That
> > > > > > > is what I was asking, if it was possible to just disable the AMLs
> > > > > > > or use some tricks to say that this particular slot is not
> > > > > > > hotpluggable. I am not sure why Windows is trying to eject the
> > > > > > > PCI bridge and failing. Maybe something related to this comment?
> > > > > > >
> > > > > > >
> > > > > > > /* When hotplug for bridges is enabled, bridges are
> > > > > > >
> > > > > > > * described in ACPI separately (see build_pci_bus_end).
> > > > > > >
> > > > > > > * In this case they aren't themselves hot-pluggable.
> > > > > > >
> > > > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > > > */
> > > > > >
> > > > > > thinking some more on this topic, it seems that with ACPI hotplug we
> > > > > > already have implicit non-hotpluggble slot (slot with bridge) while
> > > > > > the rest are staying hotpluggable.
> > > > > >
> > > > > > So my question is: if it's acceptable to add
> > > > > > 'PCIDevice::hotpluggable" property to all PCI devices so that user
> > > > > > / libvirt could set it to false in case they do not want
> > > > > > coldplugged device be considered as hotpluggable? (this way other
> > > > > > devices could be treated the same way as bridges)
> > > > > >
> > > > > > [...]
> > > > >
> > > > >
> > > > > I think Julia already posted a patch adding this to downstream pcie
> > > > > bridges. Adding this to pci slots sounds like a reasonable thing.
> > > > Question was more about external interface, were we do not have ports
> > > > as separate devices with conventional PCI. The only knob we have is a
> > > > a PCI device, where we have a property to turn on/off hotplug. ex:
> > > > -device e1000,hotpluggable=off
> > > > and if libvirt would be able to use it
> > >
> > > Libvirt can certainly use a property that is settable per-device,
> > > instead of against the pcie-root-port. In fact the application
> > > that requested hotplug control (KubeVirt) would strongly prefer
> > > if this was doable per-device, because they really dislike the
> > > idea of having to deal with pcie-root-port devices.
> > >
> > > So if you added it per-device, libvirt would support both the
> > > per-device option, and the pcie-root-port option.
> >
> > The issue is that there then is no a way to check whether the option
> > is supported on a given slot, without simply trying.
> > This is why we added it on the port, there, presence of the option
> > is a signal that it will work.
> > Is this a concern for libvirt?
>
> I'm not sure. What are the scenarios in which it is supported vs not
> supported, as I'm unclear if they affect libvirt or not ?
>
>
> Regards,
> Daniel
It would depend on things like the type of the host and pci to pci bridge used.
> --
> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o- https://fstop138.berrange.com :|
> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 9:56 ` Igor Mammedow
2020-05-20 10:06 ` Daniel P. Berrangé
@ 2020-05-20 10:28 ` Michael S. Tsirkin
2020-05-20 11:05 ` Igor Mammedow
1 sibling, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-05-20 10:28 UTC (permalink / raw)
To: Igor Mammedow
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> On Wed, 20 May 2020 05:47:53 -0400
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > > On Fri, 15 May 2020 12:13:53 +0000
> > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > >
> > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov <imammedo@redhat.com>
> > > > > wrote:
> > > > >>
> > > > >>
> > > > >>> Will following hack work for you?
> > > > >>> possible permutations
> > > > >>> 1) ACPI hotplug everywhere
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > >>>
> > > > >>> 2) No hotplug at all
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > >>>
> > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > >>
> > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > >> following in the order of preference:
> > > > >>
> > > > >> (a) Have an option to disable hot ejection of PCI-PCI bridge so
> > > > >> that Windows does not even show this HW in the “safely remove
> > > > >> HW” option. If we can do this then from OS perspective the GUI
> > > > >> options will be same as what is available with PCIE/q35 - none
> > > > >> of the devices will be hot ejectable if the hot plug option is
> > > > >> turned off from the PCIE slots where devices are plugged into.
> > > > >> I looked at the code. It seems to manipulate ACPI tables of
> > > > >> the empty slots of the root bus where no devices are attached
> > > > >> (see comment "/* add hotplug slots for non present devices */
> > > > >> “). For cold plugged bridges, it recurses down to scan the
> > > > >> slots of the bridge. Is it possible to disable hot plug for
> > > > >> the slot to which the bridge is attached?
> > > > >
> > > > > I don't think it's possible to have per slot hotplug on
> > > > > conventional PCI hardware. it's per bridge property.
> > > >
> > > > We add the AMLs per empty slot though. When the pic bridge is
> > > > attached, we do nothing, just recurse into the bridge slots. That
> > > > is what I was asking, if it was possible to just disable the AMLs
> > > > or use some tricks to say that this particular slot is not
> > > > hotpluggable. I am not sure why Windows is trying to eject the
> > > > PCI bridge and failing. Maybe something related to this comment?
> > > >
> > > >
> > > > /* When hotplug for bridges is enabled, bridges are
> > > >
> > > > * described in ACPI separately (see build_pci_bus_end).
> > > >
> > > > * In this case they aren't themselves hot-pluggable.
> > > >
> > > > * Hotplugged bridges *are* hot-pluggable.
> > > > */
> > >
> > > thinking some more on this topic, it seems that with ACPI hotplug we
> > > already have implicit non-hotpluggble slot (slot with bridge) while
> > > the rest are staying hotpluggable.
> > >
> > > So my question is: if it's acceptable to add
> > > 'PCIDevice::hotpluggable" property to all PCI devices so that user
> > > / libvirt could set it to false in case they do not want
> > > coldplugged device be considered as hotpluggable? (this way other
> > > devices could be treated the same way as bridges)
> > >
> > > [...]
> >
> >
> > I think Julia already posted a patch adding this to downstream pcie
> > bridges. Adding this to pci slots sounds like a reasonable thing.
> Question was more about external interface, were we do not have ports
> as separate devices with conventional PCI. The only knob we have is a
> a PCI device, where we have a property to turn on/off hotplug. ex:
> -device e1000,hotpluggable=off
> and if libvirt would be able to use it
It would make sense but is it practical to add the capability is added
in a generic way to all bridges and hosts?
If not how do users probe for presence of the capability?
--
MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 10:28 ` Michael S. Tsirkin
@ 2020-05-20 11:05 ` Igor Mammedow
2020-05-20 11:23 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Igor Mammedow @ 2020-05-20 11:05 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, 20 May 2020 06:28:37 -0400
"Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> > On Wed, 20 May 2020 05:47:53 -0400
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >
> > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > >
> > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov
> > > > > > <imammedo@redhat.com> wrote:
> > > > > >>
> > > > > >>
> > > > > >>> Will following hack work for you?
> > > > > >>> possible permutations
> > > > > >>> 1) ACPI hotplug everywhere
> > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > >>>
> > > > > >>> 2) No hotplug at all
> > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > >>>
> > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > >>
> > > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > > >> following in the order of preference:
> > > > > >>
> > > > > >> (a) Have an option to disable hot ejection of PCI-PCI
> > > > > >> bridge so that Windows does not even show this HW in the
> > > > > >> “safely remove HW” option. If we can do this then from OS
> > > > > >> perspective the GUI options will be same as what is
> > > > > >> available with PCIE/q35 - none of the devices will be hot
> > > > > >> ejectable if the hot plug option is turned off from the
> > > > > >> PCIE slots where devices are plugged into. I looked at the
> > > > > >> code. It seems to manipulate ACPI tables of the empty
> > > > > >> slots of the root bus where no devices are attached (see
> > > > > >> comment "/* add hotplug slots for non present devices */
> > > > > >> “). For cold plugged bridges, it recurses down to scan the
> > > > > >> slots of the bridge. Is it possible to disable hot plug
> > > > > >> for the slot to which the bridge is attached?
> > > > > >
> > > > > > I don't think it's possible to have per slot hotplug on
> > > > > > conventional PCI hardware. it's per bridge property.
> > > > >
> > > > > We add the AMLs per empty slot though. When the pic bridge is
> > > > > attached, we do nothing, just recurse into the bridge slots.
> > > > > That is what I was asking, if it was possible to just disable
> > > > > the AMLs or use some tricks to say that this particular slot
> > > > > is not hotpluggable. I am not sure why Windows is trying to
> > > > > eject the PCI bridge and failing. Maybe something related to
> > > > > this comment?
> > > > >
> > > > >
> > > > > /* When hotplug for bridges is enabled, bridges are
> > > > >
> > > > > * described in ACPI separately (see
> > > > > build_pci_bus_end).
> > > > > * In this case they aren't themselves hot-pluggable.
> > > > >
> > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > */
> > > >
> > > > thinking some more on this topic, it seems that with ACPI
> > > > hotplug we already have implicit non-hotpluggble slot (slot
> > > > with bridge) while the rest are staying hotpluggable.
> > > >
> > > > So my question is: if it's acceptable to add
> > > > 'PCIDevice::hotpluggable" property to all PCI devices so that
> > > > user / libvirt could set it to false in case they do not want
> > > > coldplugged device be considered as hotpluggable? (this way
> > > > other devices could be treated the same way as bridges)
> > > >
> > > > [...]
> > >
> > >
> > > I think Julia already posted a patch adding this to downstream
> > > pcie bridges. Adding this to pci slots sounds like a reasonable
> > > thing.
> > Question was more about external interface, were we do not have
> > ports as separate devices with conventional PCI. The only knob we
> > have is a a PCI device, where we have a property to turn on/off
> > hotplug. ex: -device e1000,hotpluggable=off
> > and if libvirt would be able to use it
>
> It would make sense but is it practical to add the capability is added
> in a generic way to all bridges and hosts?
> If not how do users probe for presence of the capability?
it probably won't work with native SHPC hotplug (which looks to be
incomplete in QEMU anyway), but it should work with ACPI and per port
PCIE hotplugs.
In case of SHPC, we probably should be able to cleanly error out with
'unsupported' reason if "hotpluggable" conflicts with bridge policy.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 11:05 ` Igor Mammedow
@ 2020-05-20 11:23 ` Michael S. Tsirkin
2020-05-20 12:20 ` Igor Mammedow
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-05-20 11:23 UTC (permalink / raw)
To: Igor Mammedow
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, May 20, 2020 at 01:05:47PM +0200, Igor Mammedow wrote:
> On Wed, 20 May 2020 06:28:37 -0400
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> > On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> > > On Wed, 20 May 2020 05:47:53 -0400
> > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > >
> > > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow wrote:
> > > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > > >
> > > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov
> > > > > > > <imammedo@redhat.com> wrote:
> > > > > > >>
> > > > > > >>
> > > > > > >>> Will following hack work for you?
> > > > > > >>> possible permutations
> > > > > > >>> 1) ACPI hotplug everywhere
> > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > >>>
> > > > > > >>> 2) No hotplug at all
> > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on -device
> > > > > > >>> pci-bridge,chassis_nr=1,shpc=off -device
> > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > >>>
> > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off -device
> > > > > > >>> pci-bridge,chassis_nr=1,shpc=doesnt_matter -device
> > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > >>
> > > > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > > > >> following in the order of preference:
> > > > > > >>
> > > > > > >> (a) Have an option to disable hot ejection of PCI-PCI
> > > > > > >> bridge so that Windows does not even show this HW in the
> > > > > > >> “safely remove HW” option. If we can do this then from OS
> > > > > > >> perspective the GUI options will be same as what is
> > > > > > >> available with PCIE/q35 - none of the devices will be hot
> > > > > > >> ejectable if the hot plug option is turned off from the
> > > > > > >> PCIE slots where devices are plugged into. I looked at the
> > > > > > >> code. It seems to manipulate ACPI tables of the empty
> > > > > > >> slots of the root bus where no devices are attached (see
> > > > > > >> comment "/* add hotplug slots for non present devices */
> > > > > > >> “). For cold plugged bridges, it recurses down to scan the
> > > > > > >> slots of the bridge. Is it possible to disable hot plug
> > > > > > >> for the slot to which the bridge is attached?
> > > > > > >
> > > > > > > I don't think it's possible to have per slot hotplug on
> > > > > > > conventional PCI hardware. it's per bridge property.
> > > > > >
> > > > > > We add the AMLs per empty slot though. When the pic bridge is
> > > > > > attached, we do nothing, just recurse into the bridge slots.
> > > > > > That is what I was asking, if it was possible to just disable
> > > > > > the AMLs or use some tricks to say that this particular slot
> > > > > > is not hotpluggable. I am not sure why Windows is trying to
> > > > > > eject the PCI bridge and failing. Maybe something related to
> > > > > > this comment?
> > > > > >
> > > > > >
> > > > > > /* When hotplug for bridges is enabled, bridges are
> > > > > >
> > > > > > * described in ACPI separately (see
> > > > > > build_pci_bus_end).
> > > > > > * In this case they aren't themselves hot-pluggable.
> > > > > >
> > > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > > */
> > > > >
> > > > > thinking some more on this topic, it seems that with ACPI
> > > > > hotplug we already have implicit non-hotpluggble slot (slot
> > > > > with bridge) while the rest are staying hotpluggable.
> > > > >
> > > > > So my question is: if it's acceptable to add
> > > > > 'PCIDevice::hotpluggable" property to all PCI devices so that
> > > > > user / libvirt could set it to false in case they do not want
> > > > > coldplugged device be considered as hotpluggable? (this way
> > > > > other devices could be treated the same way as bridges)
> > > > >
> > > > > [...]
> > > >
> > > >
> > > > I think Julia already posted a patch adding this to downstream
> > > > pcie bridges. Adding this to pci slots sounds like a reasonable
> > > > thing.
> > > Question was more about external interface, were we do not have
> > > ports as separate devices with conventional PCI. The only knob we
> > > have is a a PCI device, where we have a property to turn on/off
> > > hotplug. ex: -device e1000,hotpluggable=off
> > > and if libvirt would be able to use it
> >
> > It would make sense but is it practical to add the capability is added
> > in a generic way to all bridges and hosts?
> > If not how do users probe for presence of the capability?
> it probably won't work with native SHPC hotplug (which looks to be
> incomplete in QEMU anyway), but it should work with ACPI and per port
> PCIE hotplugs.
> In case of SHPC, we probably should be able to cleanly error out with
> 'unsupported' reason if "hotpluggable" conflicts with bridge policy.
"Try it and see if it works" is somewhat problematic from management POV
since there's a never ending stream of new things they would have to
try. If this approach is taken, we'd have to try to loop in some people
from libvirt and see what's their take.
--
MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 11:23 ` Michael S. Tsirkin
@ 2020-05-20 12:20 ` Igor Mammedow
2020-05-20 16:13 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Igor Mammedow @ 2020-05-20 12:20 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, 20 May 2020 07:23:21 -0400
"Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, May 20, 2020 at 01:05:47PM +0200, Igor Mammedow wrote:
> > On Wed, 20 May 2020 06:28:37 -0400
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >
> > > On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> > > > On Wed, 20 May 2020 05:47:53 -0400
> > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > >
> > > > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow
> > > > > wrote:
> > > > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > > > >
> > > > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov
> > > > > > > > <imammedo@redhat.com> wrote:
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>> Will following hack work for you?
> > > > > > > >>> possible permutations
> > > > > > > >>> 1) ACPI hotplug everywhere
> > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > >>>
> > > > > > > >>> 2) No hotplug at all
> > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=off -device
> > > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > >>>
> > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
> > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > >>
> > > > > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > > > > >> following in the order of preference:
> > > > > > > >>
> > > > > > > >> (a) Have an option to disable hot ejection of PCI-PCI
> > > > > > > >> bridge so that Windows does not even show this HW in
> > > > > > > >> the “safely remove HW” option. If we can do this then
> > > > > > > >> from OS perspective the GUI options will be same as
> > > > > > > >> what is available with PCIE/q35 - none of the devices
> > > > > > > >> will be hot ejectable if the hot plug option is turned
> > > > > > > >> off from the PCIE slots where devices are plugged
> > > > > > > >> into. I looked at the code. It seems to manipulate
> > > > > > > >> ACPI tables of the empty slots of the root bus where
> > > > > > > >> no devices are attached (see comment "/* add hotplug
> > > > > > > >> slots for non present devices */ “). For cold plugged
> > > > > > > >> bridges, it recurses down to scan the slots of the
> > > > > > > >> bridge. Is it possible to disable hot plug for the
> > > > > > > >> slot to which the bridge is attached?
> > > > > > > >
> > > > > > > > I don't think it's possible to have per slot hotplug on
> > > > > > > > conventional PCI hardware. it's per bridge property.
> > > > > > > >
> > > > > > >
> > > > > > > We add the AMLs per empty slot though. When the pic
> > > > > > > bridge is attached, we do nothing, just recurse into the
> > > > > > > bridge slots. That is what I was asking, if it was
> > > > > > > possible to just disable the AMLs or use some tricks to
> > > > > > > say that this particular slot is not hotpluggable. I am
> > > > > > > not sure why Windows is trying to eject the PCI bridge
> > > > > > > and failing. Maybe something related to this comment?
> > > > > > >
> > > > > > >
> > > > > > > /* When hotplug for bridges is enabled, bridges are
> > > > > > >
> > > > > > > * described in ACPI separately (see
> > > > > > > build_pci_bus_end).
> > > > > > > * In this case they aren't themselves
> > > > > > > hot-pluggable.
> > > > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > > > */
> > > > > >
> > > > > > thinking some more on this topic, it seems that with ACPI
> > > > > > hotplug we already have implicit non-hotpluggble slot (slot
> > > > > > with bridge) while the rest are staying hotpluggable.
> > > > > >
> > > > > > So my question is: if it's acceptable to add
> > > > > > 'PCIDevice::hotpluggable" property to all PCI devices so
> > > > > > that user / libvirt could set it to false in case they do
> > > > > > not want coldplugged device be considered as hotpluggable?
> > > > > > (this way other devices could be treated the same way as
> > > > > > bridges)
> > > > > >
> > > > > > [...]
> > > > >
> > > > >
> > > > > I think Julia already posted a patch adding this to downstream
> > > > > pcie bridges. Adding this to pci slots sounds like a
> > > > > reasonable thing.
> > > > Question was more about external interface, were we do not have
> > > > ports as separate devices with conventional PCI. The only knob
> > > > we have is a a PCI device, where we have a property to turn
> > > > on/off hotplug. ex: -device e1000,hotpluggable=off
> > > > and if libvirt would be able to use it
> > >
> > > It would make sense but is it practical to add the capability is
> > > added in a generic way to all bridges and hosts?
> > > If not how do users probe for presence of the capability?
> > it probably won't work with native SHPC hotplug (which looks to be
> > incomplete in QEMU anyway), but it should work with ACPI and per
> > port PCIE hotplugs.
> > In case of SHPC, we probably should be able to cleanly error out
> > with 'unsupported' reason if "hotpluggable" conflicts with bridge
> > policy.
>
> "Try it and see if it works" is somewhat problematic from management
> POV since there's a never ending stream of new things they would have
> to try. If this approach is taken, we'd have to try to loop in some
> people from libvirt and see what's their take.
to clarify, we are talking here about bridges to conventional
PCI with native SHPC hotplug semantics wrt mgmt and
potential pcidevice.hotpluggable property.
(the later should work fine in ACPI and PCIE hoptlug cases).
currently by default pci bridges have property shpc=off, so mgmt
should know that deals with PCI bridge and has to enable SHPC
on bridge explicitly, in which case it could probably be taught that
using conflicting hotpluggable for device attached to bridge and shpc
values is wrong thing.
If that's not it, then I'm not sure what kind of discovery you are
talking about.
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 12:20 ` Igor Mammedow
@ 2020-05-20 16:13 ` Michael S. Tsirkin
2020-05-21 7:32 ` Igor Mammedow
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-05-20 16:13 UTC (permalink / raw)
To: Igor Mammedow
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, May 20, 2020 at 02:20:12PM +0200, Igor Mammedow wrote:
> On Wed, 20 May 2020 07:23:21 -0400
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> > On Wed, May 20, 2020 at 01:05:47PM +0200, Igor Mammedow wrote:
> > > On Wed, 20 May 2020 06:28:37 -0400
> > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > >
> > > > On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow wrote:
> > > > > On Wed, 20 May 2020 05:47:53 -0400
> > > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > >
> > > > > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow
> > > > > > wrote:
> > > > > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > > > > >
> > > > > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov
> > > > > > > > > <imammedo@redhat.com> wrote:
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >>> Will following hack work for you?
> > > > > > > > >>> possible permutations
> > > > > > > > >>> 1) ACPI hotplug everywhere
> > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > >>>
> > > > > > > > >>> 2) No hotplug at all
> > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=off -device
> > > > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > >>>
> > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
> > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > >>
> > > > > > > > >> Given that my patch is not acceptable, I’d prefer the
> > > > > > > > >> following in the order of preference:
> > > > > > > > >>
> > > > > > > > >> (a) Have an option to disable hot ejection of PCI-PCI
> > > > > > > > >> bridge so that Windows does not even show this HW in
> > > > > > > > >> the “safely remove HW” option. If we can do this then
> > > > > > > > >> from OS perspective the GUI options will be same as
> > > > > > > > >> what is available with PCIE/q35 - none of the devices
> > > > > > > > >> will be hot ejectable if the hot plug option is turned
> > > > > > > > >> off from the PCIE slots where devices are plugged
> > > > > > > > >> into. I looked at the code. It seems to manipulate
> > > > > > > > >> ACPI tables of the empty slots of the root bus where
> > > > > > > > >> no devices are attached (see comment "/* add hotplug
> > > > > > > > >> slots for non present devices */ “). For cold plugged
> > > > > > > > >> bridges, it recurses down to scan the slots of the
> > > > > > > > >> bridge. Is it possible to disable hot plug for the
> > > > > > > > >> slot to which the bridge is attached?
> > > > > > > > >
> > > > > > > > > I don't think it's possible to have per slot hotplug on
> > > > > > > > > conventional PCI hardware. it's per bridge property.
> > > > > > > > >
> > > > > > > >
> > > > > > > > We add the AMLs per empty slot though. When the pic
> > > > > > > > bridge is attached, we do nothing, just recurse into the
> > > > > > > > bridge slots. That is what I was asking, if it was
> > > > > > > > possible to just disable the AMLs or use some tricks to
> > > > > > > > say that this particular slot is not hotpluggable. I am
> > > > > > > > not sure why Windows is trying to eject the PCI bridge
> > > > > > > > and failing. Maybe something related to this comment?
> > > > > > > >
> > > > > > > >
> > > > > > > > /* When hotplug for bridges is enabled, bridges are
> > > > > > > >
> > > > > > > > * described in ACPI separately (see
> > > > > > > > build_pci_bus_end).
> > > > > > > > * In this case they aren't themselves
> > > > > > > > hot-pluggable.
> > > > > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > > > > */
> > > > > > >
> > > > > > > thinking some more on this topic, it seems that with ACPI
> > > > > > > hotplug we already have implicit non-hotpluggble slot (slot
> > > > > > > with bridge) while the rest are staying hotpluggable.
> > > > > > >
> > > > > > > So my question is: if it's acceptable to add
> > > > > > > 'PCIDevice::hotpluggable" property to all PCI devices so
> > > > > > > that user / libvirt could set it to false in case they do
> > > > > > > not want coldplugged device be considered as hotpluggable?
> > > > > > > (this way other devices could be treated the same way as
> > > > > > > bridges)
> > > > > > >
> > > > > > > [...]
> > > > > >
> > > > > >
> > > > > > I think Julia already posted a patch adding this to downstream
> > > > > > pcie bridges. Adding this to pci slots sounds like a
> > > > > > reasonable thing.
> > > > > Question was more about external interface, were we do not have
> > > > > ports as separate devices with conventional PCI. The only knob
> > > > > we have is a a PCI device, where we have a property to turn
> > > > > on/off hotplug. ex: -device e1000,hotpluggable=off
> > > > > and if libvirt would be able to use it
> > > >
> > > > It would make sense but is it practical to add the capability is
> > > > added in a generic way to all bridges and hosts?
> > > > If not how do users probe for presence of the capability?
> > > it probably won't work with native SHPC hotplug (which looks to be
> > > incomplete in QEMU anyway), but it should work with ACPI and per
> > > port PCIE hotplugs.
> > > In case of SHPC, we probably should be able to cleanly error out
> > > with 'unsupported' reason if "hotpluggable" conflicts with bridge
> > > policy.
> >
> > "Try it and see if it works" is somewhat problematic from management
> > POV since there's a never ending stream of new things they would have
> > to try. If this approach is taken, we'd have to try to loop in some
> > people from libvirt and see what's their take.
> to clarify, we are talking here about bridges to conventional
> PCI with native SHPC hotplug semantics wrt mgmt and
> potential pcidevice.hotpluggable property.
> (the later should work fine in ACPI and PCIE hoptlug cases).
>
> currently by default pci bridges have property shpc=off, so mgmt
> should know that deals with PCI bridge and has to enable SHPC
> on bridge explicitly,
Wait a second does that actually affect hotplug with ACPI too?
> in which case it could probably be taught that
> using conflicting hotpluggable for device attached to bridge and shpc
> values is wrong thing.
> If that's not it, then I'm not sure what kind of discovery you are
> talking about.
--
MST
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-20 16:13 ` Michael S. Tsirkin
@ 2020-05-21 7:32 ` Igor Mammedow
2020-05-21 10:07 ` Michael S. Tsirkin
0 siblings, 1 reply; 52+ messages in thread
From: Igor Mammedow @ 2020-05-21 7:32 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Wed, 20 May 2020 12:13:35 -0400
"Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Wed, May 20, 2020 at 02:20:12PM +0200, Igor Mammedow wrote:
> > On Wed, 20 May 2020 07:23:21 -0400
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >
> > > On Wed, May 20, 2020 at 01:05:47PM +0200, Igor Mammedow wrote:
> > > > On Wed, 20 May 2020 06:28:37 -0400
> > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > >
> > > > > On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow
> > > > > wrote:
> > > > > > On Wed, 20 May 2020 05:47:53 -0400
> > > > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > >
> > > > > > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow
> > > > > > > wrote:
> > > > > > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > > > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > > > > > >
> > > > > > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov
> > > > > > > > > > <imammedo@redhat.com> wrote:
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >>> Will following hack work for you?
> > > > > > > > > >>> possible permutations
> > > > > > > > > >>> 1) ACPI hotplug everywhere
> > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > >>>
> > > > > > > > > >>> 2) No hotplug at all
> > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=off -device
> > > > > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > >>>
> > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
> > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > >>>
> > > > > > > > > >>
> > > > > > > > > >> Given that my patch is not acceptable, I’d prefer
> > > > > > > > > >> the following in the order of preference:
> > > > > > > > > >>
> > > > > > > > > >> (a) Have an option to disable hot ejection of
> > > > > > > > > >> PCI-PCI bridge so that Windows does not even show
> > > > > > > > > >> this HW in the “safely remove HW” option. If we
> > > > > > > > > >> can do this then from OS perspective the GUI
> > > > > > > > > >> options will be same as what is available with
> > > > > > > > > >> PCIE/q35 - none of the devices will be hot
> > > > > > > > > >> ejectable if the hot plug option is turned off
> > > > > > > > > >> from the PCIE slots where devices are plugged
> > > > > > > > > >> into. I looked at the code. It seems to manipulate
> > > > > > > > > >> ACPI tables of the empty slots of the root bus
> > > > > > > > > >> where no devices are attached (see comment "/* add
> > > > > > > > > >> hotplug slots for non present devices */ “). For
> > > > > > > > > >> cold plugged bridges, it recurses down to scan the
> > > > > > > > > >> slots of the bridge. Is it possible to disable hot
> > > > > > > > > >> plug for the slot to which the bridge is attached?
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > > I don't think it's possible to have per slot
> > > > > > > > > > hotplug on conventional PCI hardware. it's per
> > > > > > > > > > bridge property.
> > > > > > > > >
> > > > > > > > > We add the AMLs per empty slot though. When the pic
> > > > > > > > > bridge is attached, we do nothing, just recurse into
> > > > > > > > > the bridge slots. That is what I was asking, if it was
> > > > > > > > > possible to just disable the AMLs or use some tricks
> > > > > > > > > to say that this particular slot is not hotpluggable.
> > > > > > > > > I am not sure why Windows is trying to eject the PCI
> > > > > > > > > bridge and failing. Maybe something related to this
> > > > > > > > > comment?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > /* When hotplug for bridges is enabled, bridges are
> > > > > > > > >
> > > > > > > > > * described in ACPI separately (see
> > > > > > > > > build_pci_bus_end).
> > > > > > > > > * In this case they aren't themselves
> > > > > > > > > hot-pluggable.
> > > > > > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > > > > > */
> > > > > > > >
> > > > > > > > thinking some more on this topic, it seems that with
> > > > > > > > ACPI hotplug we already have implicit non-hotpluggble
> > > > > > > > slot (slot with bridge) while the rest are staying
> > > > > > > > hotpluggable.
> > > > > > > >
> > > > > > > > So my question is: if it's acceptable to add
> > > > > > > > 'PCIDevice::hotpluggable" property to all PCI devices so
> > > > > > > > that user / libvirt could set it to false in case they
> > > > > > > > do not want coldplugged device be considered as
> > > > > > > > hotpluggable? (this way other devices could be treated
> > > > > > > > the same way as bridges)
> > > > > > > >
> > > > > > > > [...]
> > > > > > >
> > > > > > >
> > > > > > > I think Julia already posted a patch adding this to
> > > > > > > downstream pcie bridges. Adding this to pci slots sounds
> > > > > > > like a reasonable thing.
> > > > > > Question was more about external interface, were we do not
> > > > > > have ports as separate devices with conventional PCI. The
> > > > > > only knob we have is a a PCI device, where we have a
> > > > > > property to turn on/off hotplug. ex: -device
> > > > > > e1000,hotpluggable=off and if libvirt would be able to use
> > > > > > it
> > > > >
> > > > > It would make sense but is it practical to add the capability
> > > > > is added in a generic way to all bridges and hosts?
> > > > > If not how do users probe for presence of the capability?
> > > > it probably won't work with native SHPC hotplug (which looks to
> > > > be incomplete in QEMU anyway), but it should work with ACPI and
> > > > per port PCIE hotplugs.
> > > > In case of SHPC, we probably should be able to cleanly error out
> > > > with 'unsupported' reason if "hotpluggable" conflicts with
> > > > bridge policy.
> > >
> > > "Try it and see if it works" is somewhat problematic from
> > > management POV since there's a never ending stream of new things
> > > they would have to try. If this approach is taken, we'd have to
> > > try to loop in some people from libvirt and see what's their
> > > take.
> > to clarify, we are talking here about bridges to conventional
> > PCI with native SHPC hotplug semantics wrt mgmt and
> > potential pcidevice.hotpluggable property.
> > (the later should work fine in ACPI and PCIE hoptlug cases).
> >
> > currently by default pci bridges have property shpc=off, so mgmt
> > should know that deals with PCI bridge and has to enable SHPC
> > on bridge explicitly,
>
> Wait a second does that actually affect hotplug with ACPI too?
What do you mean exactly?
> > in which case it could probably be taught that
> > using conflicting hotpluggable for device attached to bridge and
> > shpc values is wrong thing.
> > If that's not it, then I'm not sure what kind of discovery you are
> > talking about.
>
>
>
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-21 7:32 ` Igor Mammedow
@ 2020-05-21 10:07 ` Michael S. Tsirkin
2020-05-21 13:23 ` Igor Mammedov
0 siblings, 1 reply; 52+ messages in thread
From: Michael S. Tsirkin @ 2020-05-21 10:07 UTC (permalink / raw)
To: Igor Mammedow
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Thu, May 21, 2020 at 09:32:17AM +0200, Igor Mammedow wrote:
> On Wed, 20 May 2020 12:13:35 -0400
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> > On Wed, May 20, 2020 at 02:20:12PM +0200, Igor Mammedow wrote:
> > > On Wed, 20 May 2020 07:23:21 -0400
> > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > >
> > > > On Wed, May 20, 2020 at 01:05:47PM +0200, Igor Mammedow wrote:
> > > > > On Wed, 20 May 2020 06:28:37 -0400
> > > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > >
> > > > > > On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow
> > > > > > wrote:
> > > > > > > On Wed, 20 May 2020 05:47:53 -0400
> > > > > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > > >
> > > > > > > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow
> > > > > > > > wrote:
> > > > > > > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > > > > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > > > > > > >
> > > > > > > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov
> > > > > > > > > > > <imammedo@redhat.com> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >>> Will following hack work for you?
> > > > > > > > > > >>> possible permutations
> > > > > > > > > > >>> 1) ACPI hotplug everywhere
> > > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > > >>>
> > > > > > > > > > >>> 2) No hotplug at all
> > > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=off -device
> > > > > > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > > >>>
> > > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
> > > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > > >>>
> > > > > > > > > > >>
> > > > > > > > > > >> Given that my patch is not acceptable, I’d prefer
> > > > > > > > > > >> the following in the order of preference:
> > > > > > > > > > >>
> > > > > > > > > > >> (a) Have an option to disable hot ejection of
> > > > > > > > > > >> PCI-PCI bridge so that Windows does not even show
> > > > > > > > > > >> this HW in the “safely remove HW” option. If we
> > > > > > > > > > >> can do this then from OS perspective the GUI
> > > > > > > > > > >> options will be same as what is available with
> > > > > > > > > > >> PCIE/q35 - none of the devices will be hot
> > > > > > > > > > >> ejectable if the hot plug option is turned off
> > > > > > > > > > >> from the PCIE slots where devices are plugged
> > > > > > > > > > >> into. I looked at the code. It seems to manipulate
> > > > > > > > > > >> ACPI tables of the empty slots of the root bus
> > > > > > > > > > >> where no devices are attached (see comment "/* add
> > > > > > > > > > >> hotplug slots for non present devices */ “). For
> > > > > > > > > > >> cold plugged bridges, it recurses down to scan the
> > > > > > > > > > >> slots of the bridge. Is it possible to disable hot
> > > > > > > > > > >> plug for the slot to which the bridge is attached?
> > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > > > I don't think it's possible to have per slot
> > > > > > > > > > > hotplug on conventional PCI hardware. it's per
> > > > > > > > > > > bridge property.
> > > > > > > > > >
> > > > > > > > > > We add the AMLs per empty slot though. When the pic
> > > > > > > > > > bridge is attached, we do nothing, just recurse into
> > > > > > > > > > the bridge slots. That is what I was asking, if it was
> > > > > > > > > > possible to just disable the AMLs or use some tricks
> > > > > > > > > > to say that this particular slot is not hotpluggable.
> > > > > > > > > > I am not sure why Windows is trying to eject the PCI
> > > > > > > > > > bridge and failing. Maybe something related to this
> > > > > > > > > > comment?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > /* When hotplug for bridges is enabled, bridges are
> > > > > > > > > >
> > > > > > > > > > * described in ACPI separately (see
> > > > > > > > > > build_pci_bus_end).
> > > > > > > > > > * In this case they aren't themselves
> > > > > > > > > > hot-pluggable.
> > > > > > > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > > > > > > */
> > > > > > > > >
> > > > > > > > > thinking some more on this topic, it seems that with
> > > > > > > > > ACPI hotplug we already have implicit non-hotpluggble
> > > > > > > > > slot (slot with bridge) while the rest are staying
> > > > > > > > > hotpluggable.
> > > > > > > > >
> > > > > > > > > So my question is: if it's acceptable to add
> > > > > > > > > 'PCIDevice::hotpluggable" property to all PCI devices so
> > > > > > > > > that user / libvirt could set it to false in case they
> > > > > > > > > do not want coldplugged device be considered as
> > > > > > > > > hotpluggable? (this way other devices could be treated
> > > > > > > > > the same way as bridges)
> > > > > > > > >
> > > > > > > > > [...]
> > > > > > > >
> > > > > > > >
> > > > > > > > I think Julia already posted a patch adding this to
> > > > > > > > downstream pcie bridges. Adding this to pci slots sounds
> > > > > > > > like a reasonable thing.
> > > > > > > Question was more about external interface, were we do not
> > > > > > > have ports as separate devices with conventional PCI. The
> > > > > > > only knob we have is a a PCI device, where we have a
> > > > > > > property to turn on/off hotplug. ex: -device
> > > > > > > e1000,hotpluggable=off and if libvirt would be able to use
> > > > > > > it
> > > > > >
> > > > > > It would make sense but is it practical to add the capability
> > > > > > is added in a generic way to all bridges and hosts?
> > > > > > If not how do users probe for presence of the capability?
> > > > > it probably won't work with native SHPC hotplug (which looks to
> > > > > be incomplete in QEMU anyway), but it should work with ACPI and
> > > > > per port PCIE hotplugs.
> > > > > In case of SHPC, we probably should be able to cleanly error out
> > > > > with 'unsupported' reason if "hotpluggable" conflicts with
> > > > > bridge policy.
> > > >
> > > > "Try it and see if it works" is somewhat problematic from
> > > > management POV since there's a never ending stream of new things
> > > > they would have to try. If this approach is taken, we'd have to
> > > > try to loop in some people from libvirt and see what's their
> > > > take.
> > > to clarify, we are talking here about bridges to conventional
> > > PCI with native SHPC hotplug semantics wrt mgmt and
> > > potential pcidevice.hotpluggable property.
> > > (the later should work fine in ACPI and PCIE hoptlug cases).
> > >
> > > currently by default pci bridges have property shpc=off, so mgmt
> > > should know that deals with PCI bridge and has to enable SHPC
> > > on bridge explicitly,
> >
> > Wait a second does that actually affect hotplug with ACPI too?
> What do you mean exactly?
I just tried to figure out why do you say
"mgmt should know that deals with PCI bridge and has to enable SHPC
on bridge explicitly,"
since on x86 bridges support hotplug through ACPI bypassing SHPC.
>
> > > in which case it could probably be taught that
> > > using conflicting hotpluggable for device attached to bridge and
> > > shpc values is wrong thing.
> > > If that's not it, then I'm not sure what kind of discovery you are
> > > talking about.
> >
> >
> >
> >
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-21 10:07 ` Michael S. Tsirkin
@ 2020-05-21 13:23 ` Igor Mammedov
2020-05-21 15:40 ` Laine Stump
0 siblings, 1 reply; 52+ messages in thread
From: Igor Mammedov @ 2020-05-21 13:23 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Laine Stump, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On Thu, 21 May 2020 06:07:25 -0400
"Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Thu, May 21, 2020 at 09:32:17AM +0200, Igor Mammedow wrote:
> > On Wed, 20 May 2020 12:13:35 -0400
> > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >
> > > On Wed, May 20, 2020 at 02:20:12PM +0200, Igor Mammedow wrote:
> > > > On Wed, 20 May 2020 07:23:21 -0400
> > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > >
> > > > > On Wed, May 20, 2020 at 01:05:47PM +0200, Igor Mammedow wrote:
> > > > > > On Wed, 20 May 2020 06:28:37 -0400
> > > > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > >
> > > > > > > On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow
> > > > > > > wrote:
> > > > > > > > On Wed, 20 May 2020 05:47:53 -0400
> > > > > > > > "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > > > >
> > > > > > > > > On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow
> > > > > > > > > wrote:
> > > > > > > > > > On Fri, 15 May 2020 12:13:53 +0000
> > > > > > > > > > Ani Sinha <ani.sinha@nutanix.com> wrote:
> > > > > > > > > >
> > > > > > > > > > > > On May 14, 2020, at 1:13 AM, Igor Mammedov
> > > > > > > > > > > > <imammedo@redhat.com> wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >>> Will following hack work for you?
> > > > > > > > > > > >>> possible permutations
> > > > > > > > > > > >>> 1) ACPI hotplug everywhere
> > > > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=on -global
> > > > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> 2) No hotplug at all
> > > > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
> > > > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=off -device
> > > > > > > > > > > >>> e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> -global PIIX4_PM.acpi-pci-hotplug=off -global
> > > > > > > > > > > >>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
> > > > > > > > > > > >>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
> > > > > > > > > > > >>> -device e1000,bus=pci.1,addr=01,id=netdev1
> > > > > > > > > > > >>>
> > > > > > > > > > > >>
> > > > > > > > > > > >> Given that my patch is not acceptable, I’d prefer
> > > > > > > > > > > >> the following in the order of preference:
> > > > > > > > > > > >>
> > > > > > > > > > > >> (a) Have an option to disable hot ejection of
> > > > > > > > > > > >> PCI-PCI bridge so that Windows does not even show
> > > > > > > > > > > >> this HW in the “safely remove HW” option. If we
> > > > > > > > > > > >> can do this then from OS perspective the GUI
> > > > > > > > > > > >> options will be same as what is available with
> > > > > > > > > > > >> PCIE/q35 - none of the devices will be hot
> > > > > > > > > > > >> ejectable if the hot plug option is turned off
> > > > > > > > > > > >> from the PCIE slots where devices are plugged
> > > > > > > > > > > >> into. I looked at the code. It seems to manipulate
> > > > > > > > > > > >> ACPI tables of the empty slots of the root bus
> > > > > > > > > > > >> where no devices are attached (see comment "/* add
> > > > > > > > > > > >> hotplug slots for non present devices */ “). For
> > > > > > > > > > > >> cold plugged bridges, it recurses down to scan the
> > > > > > > > > > > >> slots of the bridge. Is it possible to disable hot
> > > > > > > > > > > >> plug for the slot to which the bridge is attached?
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > > > I don't think it's possible to have per slot
> > > > > > > > > > > > hotplug on conventional PCI hardware. it's per
> > > > > > > > > > > > bridge property.
> > > > > > > > > > >
> > > > > > > > > > > We add the AMLs per empty slot though. When the pic
> > > > > > > > > > > bridge is attached, we do nothing, just recurse into
> > > > > > > > > > > the bridge slots. That is what I was asking, if it was
> > > > > > > > > > > possible to just disable the AMLs or use some tricks
> > > > > > > > > > > to say that this particular slot is not hotpluggable.
> > > > > > > > > > > I am not sure why Windows is trying to eject the PCI
> > > > > > > > > > > bridge and failing. Maybe something related to this
> > > > > > > > > > > comment?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > /* When hotplug for bridges is enabled, bridges are
> > > > > > > > > > >
> > > > > > > > > > > * described in ACPI separately (see
> > > > > > > > > > > build_pci_bus_end).
> > > > > > > > > > > * In this case they aren't themselves
> > > > > > > > > > > hot-pluggable.
> > > > > > > > > > > * Hotplugged bridges *are* hot-pluggable.
> > > > > > > > > > > */
> > > > > > > > > >
> > > > > > > > > > thinking some more on this topic, it seems that with
> > > > > > > > > > ACPI hotplug we already have implicit non-hotpluggble
> > > > > > > > > > slot (slot with bridge) while the rest are staying
> > > > > > > > > > hotpluggable.
> > > > > > > > > >
> > > > > > > > > > So my question is: if it's acceptable to add
> > > > > > > > > > 'PCIDevice::hotpluggable" property to all PCI devices so
> > > > > > > > > > that user / libvirt could set it to false in case they
> > > > > > > > > > do not want coldplugged device be considered as
> > > > > > > > > > hotpluggable? (this way other devices could be treated
> > > > > > > > > > the same way as bridges)
> > > > > > > > > >
> > > > > > > > > > [...]
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I think Julia already posted a patch adding this to
> > > > > > > > > downstream pcie bridges. Adding this to pci slots sounds
> > > > > > > > > like a reasonable thing.
> > > > > > > > Question was more about external interface, were we do not
> > > > > > > > have ports as separate devices with conventional PCI. The
> > > > > > > > only knob we have is a a PCI device, where we have a
> > > > > > > > property to turn on/off hotplug. ex: -device
> > > > > > > > e1000,hotpluggable=off and if libvirt would be able to use
> > > > > > > > it
> > > > > > >
> > > > > > > It would make sense but is it practical to add the capability
> > > > > > > is added in a generic way to all bridges and hosts?
> > > > > > > If not how do users probe for presence of the capability?
> > > > > > it probably won't work with native SHPC hotplug (which looks to
> > > > > > be incomplete in QEMU anyway), but it should work with ACPI and
> > > > > > per port PCIE hotplugs.
> > > > > > In case of SHPC, we probably should be able to cleanly error out
> > > > > > with 'unsupported' reason if "hotpluggable" conflicts with
> > > > > > bridge policy.
> > > > >
> > > > > "Try it and see if it works" is somewhat problematic from
> > > > > management POV since there's a never ending stream of new things
> > > > > they would have to try. If this approach is taken, we'd have to
> > > > > try to loop in some people from libvirt and see what's their
> > > > > take.
> > > > to clarify, we are talking here about bridges to conventional
> > > > PCI with native SHPC hotplug semantics wrt mgmt and
> > > > potential pcidevice.hotpluggable property.
> > > > (the later should work fine in ACPI and PCIE hoptlug cases).
> > > >
> > > > currently by default pci bridges have property shpc=off, so mgmt
> > > > should know that deals with PCI bridge and has to enable SHPC
> > > > on bridge explicitly,
> > >
> > > Wait a second does that actually affect hotplug with ACPI too?
> > What do you mean exactly?
>
> I just tried to figure out why do you say
> "mgmt should know that deals with PCI bridge and has to enable SHPC
> on bridge explicitly,"
> since on x86 bridges support hotplug through ACPI bypassing SHPC.
Current state in QEMU is that for ACPI hotplug, it doesn't have to do anything
"acpi-pci-hotplug-with-bridge-support=on" by default and it's global setting for
coldplugged bridges, also it (sort of) overrides SHPC (even if was enabled with
'shpc=on' on a concrete coldplugged bridge).
(I wouldn't be surprised if mixing modes actually isn't functional)
Then there is hotplugged bridges (added at -S time or at runtime), for those
mgmt has to set 'shpc=on' explicitly to be able to hotplug devices into it.
(so mgmt has to know that it's dealing wants SHPC and I suppose it can be taught
that SHPC doesn't support mixed hotplug policy on slots/devices attached to it).
Whether mgmt actually does something with shpc or not, I have no slightest idea.
(CCing Laine, perhaps he can clarify if we should care about SHPC here)
> > > > in which case it could probably be taught that
> > > > using conflicting hotpluggable for device attached to bridge and
> > > > shpc values is wrong thing.
> > > > If that's not it, then I'm not sure what kind of discovery you are
> > > > talking about.
> > >
> > >
> > >
> > >
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-21 13:23 ` Igor Mammedov
@ 2020-05-21 15:40 ` Laine Stump
2020-05-26 5:32 ` Ani Sinha
0 siblings, 1 reply; 52+ messages in thread
From: Laine Stump @ 2020-05-21 15:40 UTC (permalink / raw)
To: Igor Mammedov, Michael S. Tsirkin
Cc: Ani Sinha, Eduardo Habkost, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Ani Sinha, Paolo Bonzini,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
On 5/21/20 9:23 AM, Igor Mammedov wrote:
> On Thu, 21 May 2020 06:07:25 -0400
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
>> On Thu, May 21, 2020 at 09:32:17AM +0200, Igor Mammedow wrote:
>>> On Wed, 20 May 2020 12:13:35 -0400
>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>>
>>>> On Wed, May 20, 2020 at 02:20:12PM +0200, Igor Mammedow wrote:
>>>>> On Wed, 20 May 2020 07:23:21 -0400
>>>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>>>>
>>>>>> On Wed, May 20, 2020 at 01:05:47PM +0200, Igor Mammedow wrote:
>>>>>>> On Wed, 20 May 2020 06:28:37 -0400
>>>>>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>>>>>>
>>>>>>>> On Wed, May 20, 2020 at 11:56:26AM +0200, Igor Mammedow
>>>>>>>> wrote:
>>>>>>>>> On Wed, 20 May 2020 05:47:53 -0400
>>>>>>>>> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> On Wed, May 20, 2020 at 11:43:54AM +0200, Igor Mammedow
>>>>>>>>>> wrote:
>>>>>>>>>>> On Fri, 15 May 2020 12:13:53 +0000
>>>>>>>>>>> Ani Sinha <ani.sinha@nutanix.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>> On May 14, 2020, at 1:13 AM, Igor Mammedov
>>>>>>>>>>>>> <imammedo@redhat.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Will following hack work for you?
>>>>>>>>>>>>>>> possible permutations
>>>>>>>>>>>>>>> 1) ACPI hotplug everywhere
>>>>>>>>>>>>>>> -global PIIX4_PM.acpi-pci-hotplug=on -global
>>>>>>>>>>>>>>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
>>>>>>>>>>>>>>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
>>>>>>>>>>>>>>> -device e1000,bus=pci.1,addr=01,id=netdev1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) No hotplug at all
>>>>>>>>>>>>>>> -global PIIX4_PM.acpi-pci-hotplug=off -global
>>>>>>>>>>>>>>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on
>>>>>>>>>>>>>>> -device pci-bridge,chassis_nr=1,shpc=off -device
>>>>>>>>>>>>>>> e1000,bus=pci.1,addr=01,id=netdev1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -global PIIX4_PM.acpi-pci-hotplug=off -global
>>>>>>>>>>>>>>> PIIX4_PM.acpi-pci-hotplug-with-bridge-support=off
>>>>>>>>>>>>>>> -device pci-bridge,chassis_nr=1,shpc=doesnt_matter
>>>>>>>>>>>>>>> -device e1000,bus=pci.1,addr=01,id=netdev1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Given that my patch is not acceptable, I’d prefer
>>>>>>>>>>>>>> the following in the order of preference:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (a) Have an option to disable hot ejection of
>>>>>>>>>>>>>> PCI-PCI bridge so that Windows does not even show
>>>>>>>>>>>>>> this HW in the “safely remove HW” option. If we
>>>>>>>>>>>>>> can do this then from OS perspective the GUI
>>>>>>>>>>>>>> options will be same as what is available with
>>>>>>>>>>>>>> PCIE/q35 - none of the devices will be hot
>>>>>>>>>>>>>> ejectable if the hot plug option is turned off
>>>>>>>>>>>>>> from the PCIE slots where devices are plugged
>>>>>>>>>>>>>> into. I looked at the code. It seems to manipulate
>>>>>>>>>>>>>> ACPI tables of the empty slots of the root bus
>>>>>>>>>>>>>> where no devices are attached (see comment "/* add
>>>>>>>>>>>>>> hotplug slots for non present devices */ “). For
>>>>>>>>>>>>>> cold plugged bridges, it recurses down to scan the
>>>>>>>>>>>>>> slots of the bridge. Is it possible to disable hot
>>>>>>>>>>>>>> plug for the slot to which the bridge is attached?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't think it's possible to have per slot
>>>>>>>>>>>>> hotplug on conventional PCI hardware. it's per
>>>>>>>>>>>>> bridge property.
>>>>>>>>>>>>
>>>>>>>>>>>> We add the AMLs per empty slot though. When the pic
>>>>>>>>>>>> bridge is attached, we do nothing, just recurse into
>>>>>>>>>>>> the bridge slots. That is what I was asking, if it was
>>>>>>>>>>>> possible to just disable the AMLs or use some tricks
>>>>>>>>>>>> to say that this particular slot is not hotpluggable.
>>>>>>>>>>>> I am not sure why Windows is trying to eject the PCI
>>>>>>>>>>>> bridge and failing. Maybe something related to this
>>>>>>>>>>>> comment?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> /* When hotplug for bridges is enabled, bridges are
>>>>>>>>>>>>
>>>>>>>>>>>> * described in ACPI separately (see
>>>>>>>>>>>> build_pci_bus_end).
>>>>>>>>>>>> * In this case they aren't themselves
>>>>>>>>>>>> hot-pluggable.
>>>>>>>>>>>> * Hotplugged bridges *are* hot-pluggable.
>>>>>>>>>>>> */
>>>>>>>>>>>
>>>>>>>>>>> thinking some more on this topic, it seems that with
>>>>>>>>>>> ACPI hotplug we already have implicit non-hotpluggble
>>>>>>>>>>> slot (slot with bridge) while the rest are staying
>>>>>>>>>>> hotpluggable.
>>>>>>>>>>>
>>>>>>>>>>> So my question is: if it's acceptable to add
>>>>>>>>>>> 'PCIDevice::hotpluggable" property to all PCI devices so
>>>>>>>>>>> that user / libvirt could set it to false in case they
>>>>>>>>>>> do not want coldplugged device be considered as
>>>>>>>>>>> hotpluggable? (this way other devices could be treated
>>>>>>>>>>> the same way as bridges)
>>>>>>>>>>>
>>>>>>>>>>> [...]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I think Julia already posted a patch adding this to
>>>>>>>>>> downstream pcie bridges. Adding this to pci slots sounds
>>>>>>>>>> like a reasonable thing.
>>>>>>>>> Question was more about external interface, were we do not
>>>>>>>>> have ports as separate devices with conventional PCI. The
>>>>>>>>> only knob we have is a a PCI device, where we have a
>>>>>>>>> property to turn on/off hotplug. ex: -device
>>>>>>>>> e1000,hotpluggable=off and if libvirt would be able to use
>>>>>>>>> it
>>>>>>>>
>>>>>>>> It would make sense but is it practical to add the capability
>>>>>>>> is added in a generic way to all bridges and hosts?
>>>>>>>> If not how do users probe for presence of the capability?
>>>>>>> it probably won't work with native SHPC hotplug (which looks to
>>>>>>> be incomplete in QEMU anyway), but it should work with ACPI and
>>>>>>> per port PCIE hotplugs.
>>>>>>> In case of SHPC, we probably should be able to cleanly error out
>>>>>>> with 'unsupported' reason if "hotpluggable" conflicts with
>>>>>>> bridge policy.
>>>>>>
>>>>>> "Try it and see if it works" is somewhat problematic from
>>>>>> management POV since there's a never ending stream of new things
>>>>>> they would have to try. If this approach is taken, we'd have to
>>>>>> try to loop in some people from libvirt and see what's their
>>>>>> take.
>>>>> to clarify, we are talking here about bridges to conventional
>>>>> PCI with native SHPC hotplug semantics wrt mgmt and
>>>>> potential pcidevice.hotpluggable property.
>>>>> (the later should work fine in ACPI and PCIE hoptlug cases).
>>>>>
>>>>> currently by default pci bridges have property shpc=off, so mgmt
>>>>> should know that deals with PCI bridge and has to enable SHPC
>>>>> on bridge explicitly,
>>>>
>>>> Wait a second does that actually affect hotplug with ACPI too?
>>> What do you mean exactly?
>>
>> I just tried to figure out why do you say
>> "mgmt should know that deals with PCI bridge and has to enable SHPC
>> on bridge explicitly,"
>> since on x86 bridges support hotplug through ACPI bypassing SHPC.
>
> Current state in QEMU is that for ACPI hotplug, it doesn't have to do anything
> "acpi-pci-hotplug-with-bridge-support=on" by default and it's global setting for
> coldplugged bridges, also it (sort of) overrides SHPC (even if was enabled with
> 'shpc=on' on a concrete coldplugged bridge).
> (I wouldn't be surprised if mixing modes actually isn't functional)
>
> Then there is hotplugged bridges (added at -S time or at runtime),
Aren't hotplugged bridges ignored by essentially every guest OS in
existence? (since it requires the entire topology to be rescanned, or
something like that - forgive my lack of knowledge about the details, I
just remember being told "that doesn't work")
> for those
> mgmt has to set 'shpc=on' explicitly to be able to hotplug devices into it.
> (so mgmt has to know that it's dealing wants SHPC and I suppose it can be taught
> that SHPC doesn't support mixed hotplug policy on slots/devices attached to it).
> Whether mgmt actually does something with shpc or not, I have no slightest idea.
> (CCing Laine, perhaps he can clarify if we should care about SHPC here)
libvirt does nothing with shpc. It uses the bridges with whatever is the
default setting of shpc in qemu. I recall at one time someone (Marcel
maybe?) proposed changing the default of that setting, but don't
remember why.
>
>
>>>>> in which case it could probably be taught that
>>>>> using conflicting hotpluggable for device attached to bridge and
>>>>> shpc values is wrong thing.
>>>>> If that's not it, then I'm not sure what kind of discovery you are
>>>>> talking about.
>>>>
>>>>
>>>>
>>>>
>>
>
^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [PATCH V2] Add a new PIIX option to control PCI hot unplugging of devices on non-root buses
2020-05-21 15:40 ` Laine Stump
@ 2020-05-26 5:32 ` Ani Sinha
0 siblings, 0 replies; 52+ messages in thread
From: Ani Sinha @ 2020-05-26 5:32 UTC (permalink / raw)
To: Laine Stump
Cc: Eduardo Habkost, Michael S. Tsirkin, Julia Suvorova, qemu-devel,
Aleksandar Markovic, Paolo Bonzini, Ani Sinha, Igor Mammedov,
Philippe Mathieu-Daudé,
Aurelien Jarno, Richard Henderson
> On May 21, 2020, at 9:10 PM, Laine Stump <laine@redhat.com> wrote:
>
> I recall at one time someone (Marcel maybe?) proposed changing the default of that setting, but don't remember why.
https://patchwork.kernel.org/patch/9409349/
^ permalink raw reply [flat|nested] 52+ messages in thread