[PATCH 7/7] [V3] kvm: qemu: fix hot remove assigned device with iommu

[PATCH 7/7] [V3] kvm: qemu: fix hot remove assigned device with iommu

[Han, Weidong]

[-- Attachment #1: Type: text/plain, Size: 3542 bytes --]

when hot remove the assigned device with iommu, it should
deassign it from guest and free it from qemu.

assign_dev_update_irqs may not be invoked when hot add a device,
so need to assign irq after device assignment in
init_assigned_device.

Acked-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Weidong Han <weidong.han@intel.com>
---
 qemu/hw/device-assignment.c |   20 ++++++++++++++++++++
 qemu/hw/device-assignment.h |    1 +
 qemu/hw/device-hotplug.c    |   17 +++++++++++++++++
 3 files changed, 38 insertions(+), 0 deletions(-)

diff --git a/qemu/hw/device-assignment.c b/qemu/hw/device-assignment.c
index fc89c6f..d6acc67 100644
--- a/qemu/hw/device-assignment.c
+++ b/qemu/hw/device-assignment.c
@@ -571,6 +571,21 @@ void remove_assigned_device(AssignedDevInfo *adev)
     free_assigned_device(adev);
 }
 
+AssignedDevInfo *get_assigned_device(int pcibus, int slot)
+{
+    AssignedDevice *assigned_dev = NULL;
+    AssignedDevInfo *adev = NULL;
+
+    LIST_FOREACH(adev, &adev_head, next) {
+        assigned_dev = adev->assigned_dev;
+        if (pci_bus_num(assigned_dev->dev.bus) == pcibus &&
+            PCI_SLOT(assigned_dev->dev.devfn) == slot)
+            return adev;
+    }
+
+    return NULL;
+}
+
 /* The pci config space got updated. Check if irq numbers have changed
  * for our devices
  */
@@ -639,6 +654,11 @@ struct PCIDevice *init_assigned_device(AssignedDevInfo *adev, PCIBus *bus)
     if (r < 0)
         goto assigned_out;
 
+    /* assign irq for the device */
+    r = assign_irq(adev);
+    if (r < 0)
+        goto assigned_out;
+ 
     return &dev->dev;
 
 assigned_out:
diff --git a/qemu/hw/device-assignment.h b/qemu/hw/device-assignment.h
index 84f3f32..da775d7 100644
--- a/qemu/hw/device-assignment.h
+++ b/qemu/hw/device-assignment.h
@@ -98,6 +98,7 @@ PCIDevice *init_assigned_device(AssignedDevInfo *adev, PCIBus *bus);
 AssignedDevInfo *add_assigned_device(const char *arg);
 void add_assigned_devices(PCIBus *bus, const char **devices, int n_devices);
 void remove_assigned_device(AssignedDevInfo *adev);
+AssignedDevInfo *get_assigned_device(int pcibus, int slot);
 ram_addr_t assigned_dev_load_option_roms(ram_addr_t rom_base_offset);
 void assigned_dev_update_irqs(void);
 
diff --git a/qemu/hw/device-hotplug.c b/qemu/hw/device-hotplug.c
index 671acb2..03987ab 100644
--- a/qemu/hw/device-hotplug.c
+++ b/qemu/hw/device-hotplug.c
@@ -63,6 +63,14 @@ static PCIDevice *qemu_system_hot_assign_device(const char *opts, int bus_nr)
     return ret;
 }
 
+static void qemu_system_hot_deassign_device(AssignedDevInfo *adev)
+{
+    remove_assigned_device(adev);
+
+    term_printf("Unregister host PCI device %02x:%02x.%1x "
+		"(\"%s\") from guest\n", 
+		adev->bus, adev->dev, adev->func, adev->name);
+}
 #endif /* USE_KVM_DEVICE_ASSIGNMENT */
 
 static int add_init_drive(const char *opts)
@@ -240,12 +248,21 @@ void device_hot_remove_success(int pcibus, int slot)
 {
     PCIDevice *d = pci_find_device(pcibus, slot);
     int class_code;
+    AssignedDevInfo *adev;
 
     if (!d) {
         term_printf("invalid slot %d\n", slot);
         return;
     }
 
+#ifdef USE_KVM_DEVICE_ASSIGNMENT
+    adev = get_assigned_device(pcibus, slot);
+    if (adev) {
+        qemu_system_hot_deassign_device(adev);
+        return;
+    }
+#endif /* USE_KVM_DEVICE_ASSIGNMENT */
+
     class_code = d->config_read(d, PCI_CLASS_DEVICE+1, 1);
 
     pci_unregister_device(d);
-- 
1.6.0.4

[-- Attachment #2: 0007-kvm-qemu-fix-hot-remove-assigned-device-with-iommu.patch --]
[-- Type: application/octet-stream, Size: 3657 bytes --]

From 9fa576fda95805f6887219862cb02f23f71ff7d6 Mon Sep 17 00:00:00 2001
From: Weidong Han <weidong.han@intel.com>
Date: Fri, 13 Feb 2009 17:38:02 +0800
Subject: [PATCH] kvm: qemu: fix hot remove assigned device with iommu

when hot remove the assigned device with iommu, it should
deassign it from guest and free it from qemu.

assign_dev_update_irqs may not be invoked when hot add a device,
so need to assign irq after device assignment in
init_assigned_device.

Acked-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Weidong Han <weidong.han@intel.com>
---
 qemu/hw/device-assignment.c |   20 ++++++++++++++++++++
 qemu/hw/device-assignment.h |    1 +
 qemu/hw/device-hotplug.c    |   17 +++++++++++++++++
 3 files changed, 38 insertions(+), 0 deletions(-)

diff --git a/qemu/hw/device-assignment.c b/qemu/hw/device-assignment.c
index fc89c6f..d6acc67 100644
--- a/qemu/hw/device-assignment.c
+++ b/qemu/hw/device-assignment.c
@@ -571,6 +571,21 @@ void remove_assigned_device(AssignedDevInfo *adev)
     free_assigned_device(adev);
 }
 
+AssignedDevInfo *get_assigned_device(int pcibus, int slot)
+{
+    AssignedDevice *assigned_dev = NULL;
+    AssignedDevInfo *adev = NULL;
+
+    LIST_FOREACH(adev, &adev_head, next) {
+        assigned_dev = adev->assigned_dev;
+        if (pci_bus_num(assigned_dev->dev.bus) == pcibus &&
+            PCI_SLOT(assigned_dev->dev.devfn) == slot)
+            return adev;
+    }
+
+    return NULL;
+}
+
 /* The pci config space got updated. Check if irq numbers have changed
  * for our devices
  */
@@ -639,6 +654,11 @@ struct PCIDevice *init_assigned_device(AssignedDevInfo *adev, PCIBus *bus)
     if (r < 0)
         goto assigned_out;
 
+    /* assign irq for the device */
+    r = assign_irq(adev);
+    if (r < 0)
+        goto assigned_out;
+ 
     return &dev->dev;
 
 assigned_out:
diff --git a/qemu/hw/device-assignment.h b/qemu/hw/device-assignment.h
index 84f3f32..da775d7 100644
--- a/qemu/hw/device-assignment.h
+++ b/qemu/hw/device-assignment.h
@@ -98,6 +98,7 @@ PCIDevice *init_assigned_device(AssignedDevInfo *adev, PCIBus *bus);
 AssignedDevInfo *add_assigned_device(const char *arg);
 void add_assigned_devices(PCIBus *bus, const char **devices, int n_devices);
 void remove_assigned_device(AssignedDevInfo *adev);
+AssignedDevInfo *get_assigned_device(int pcibus, int slot);
 ram_addr_t assigned_dev_load_option_roms(ram_addr_t rom_base_offset);
 void assigned_dev_update_irqs(void);
 
diff --git a/qemu/hw/device-hotplug.c b/qemu/hw/device-hotplug.c
index 671acb2..03987ab 100644
--- a/qemu/hw/device-hotplug.c
+++ b/qemu/hw/device-hotplug.c
@@ -63,6 +63,14 @@ static PCIDevice *qemu_system_hot_assign_device(const char *opts, int bus_nr)
     return ret;
 }
 
+static void qemu_system_hot_deassign_device(AssignedDevInfo *adev)
+{
+    remove_assigned_device(adev);
+
+    term_printf("Unregister host PCI device %02x:%02x.%1x "
+		"(\"%s\") from guest\n", 
+		adev->bus, adev->dev, adev->func, adev->name);
+}
 #endif /* USE_KVM_DEVICE_ASSIGNMENT */
 
 static int add_init_drive(const char *opts)
@@ -240,12 +248,21 @@ void device_hot_remove_success(int pcibus, int slot)
 {
     PCIDevice *d = pci_find_device(pcibus, slot);
     int class_code;
+    AssignedDevInfo *adev;
 
     if (!d) {
         term_printf("invalid slot %d\n", slot);
         return;
     }
 
+#ifdef USE_KVM_DEVICE_ASSIGNMENT
+    adev = get_assigned_device(pcibus, slot);
+    if (adev) {
+        qemu_system_hot_deassign_device(adev);
+        return;
+    }
+#endif /* USE_KVM_DEVICE_ASSIGNMENT */
+
     class_code = d->config_read(d, PCI_CLASS_DEVICE+1, 1);
 
     pci_unregister_device(d);
-- 
1.6.0.4

Re: [PATCH 7/7] [V3] kvm: qemu: fix hot remove assigned device with iommu

[Marcelo Tosatti]

On Fri, Feb 13, 2009 at 05:49:49PM +0800, Han, Weidong wrote:
> when hot remove the assigned device with iommu, it should
> deassign it from guest and free it from qemu.
> 
> assign_dev_update_irqs may not be invoked when hot add a device,
> so need to assign irq after device assignment in
> init_assigned_device.

Weidong,

Can you please test device assignment hotplug with the current tree?

There was a qemu merge yesterday.

Thanks

RE: [PATCH 7/7] [V3] kvm: qemu: fix hot remove assigned device with iommu

[Han, Weidong]

Marcelo Tosatti wrote:
> On Fri, Feb 13, 2009 at 05:49:49PM +0800, Han, Weidong wrote:
>> when hot remove the assigned device with iommu, it should
>> deassign it from guest and free it from qemu.
>> 
>> assign_dev_update_irqs may not be invoked when hot add a device,
>> so need to assign irq after device assignment in
>> init_assigned_device.
> 
> Weidong,
> 
> Can you please test device assignment hotplug with the current tree?
> 
> There was a qemu merge yesterday.
> 
> Thanks

Ok. 

Regards,
Weidong

RE: [PATCH 7/7] [V3] kvm: qemu: fix hot remove assigned device with iommu

[Han, Weidong]

Han, Weidong wrote:
> Marcelo Tosatti wrote:
>> On Fri, Feb 13, 2009 at 05:49:49PM +0800, Han, Weidong wrote:
>>> when hot remove the assigned device with iommu, it should
>>> deassign it from guest and free it from qemu.
>>> 
>>> assign_dev_update_irqs may not be invoked when hot add a device,
>>> so need to assign irq after device assignment in
>>> init_assigned_device.
>> 
>> Weidong,
>> 
>> Can you please test device assignment hotplug with the current tree?
>> 

device assignment hotplug doesn't work on current tree. I had a quick glance, found device assignment didn't be considered, qemu_system_hot_assign_device is not used at all. I will fix it.

Regards,
Weidong

>> There was a qemu merge yesterday.
>> 
>> Thanks
> 
> Ok.
> 
> Regards,
> Weidong

Re: [PATCH 7/7] [V3] kvm: qemu: fix hot remove assigned device with iommu

[Avi Kivity]

Han, Weidong wrote:
> device assignment hotplug doesn't work on current tree. I had a quick glance, found device assignment didn't be considered, qemu_system_hot_assign_device is not used at all. I will fix it.
>   

I noticed that during the merge, but wasn't familiar in the code to fix 
it myself.  Thanks for taking care of it.

We should work to merge the code in upstream qemu so that this doesn't 
happen again.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.