* XEN 4.1 pci passthru performance regression?
[not found] <b9bc7c5f-5617-4273-a293-7e5b98fb61d9@orsmsx602.amr.corp.intel.com>
@ 2011-05-30 9:08 ` Cao, Buddy
2011-05-31 12:14 ` Ian Campbell
2011-05-30 9:12 ` Any suggesions for "Gfx passthru" on GTS 450? Cao, Buddy
1 sibling, 1 reply; 7+ messages in thread
From: Cao, Buddy @ 2011-05-30 9:08 UTC (permalink / raw)
To: xen-devel
I migrate Xen4.0 to XEN4.1, and found XEN4.1 performance is not as good as 4.0.2. xm shutdown takes longer time, control thru libvirt take longer time, and since I'm testing PCI/VGA passthru, the "createdevice: pci" also take around 5~6 seconds comparing to 1- second in XEN4.0.2. below is my configuration, do you have any suggestions to make the performance better on XEN4.1?
xend-config.sxp:
(xend-unix-server yes)
(xend-http-server yes)
(xend-relocation-server yes)
(xend-port 8000)
(xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
(network-script network-bridge)
(vif-script vif-bridge)
(dom0-min-mem 768)
(enable-dom0-ballooning yes)
(total_available_memory 0)
(dom0-cpus 0)
(vnc-listen '0.0.0.0')
(vncpasswd '')
vm1.cfg:
import os, re
arch = os.uname()[4]
name = "b"
maxmem = 2048
memory = 2048
vcpus = 4
builder = "hvm"
kernel = "/usr/lib/xen/boot/hvmloader"
boot = "c"
pae = 0
acpi = 1
apic = 1
timer_mode =1
shadow_memory = 32
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
device_model = "/usr/lib/xen/bin/qemu-dm"
vnc = 1
disk = [ "file:/opt/EVM/domain/a_disk.img,hda,w" ]
vif = [ "type=ioemu" ]
gfx_passthru=1
pci= ['00:02.0','00:1a.0','00:1d.0','00:1b.0' , '00:1f.0']
xend.log
[2011-05-24 16:01:59 2335] DEBUG (SrvDomain:76) Starting domain EVM_BackUp False
[2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2498) XendDomainInfo.constructDomain
[2011-05-24 16:01:59 2335] DEBUG (balloon:187) Balloon: 2150848 KiB free; need 16384; done.
[2011-05-24 16:01:59 2335] DEBUG (XendDomain:476) Adding Domain: 3
[2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2836) XendDomainInfo.initDomain: 3 256
[2011-05-24 16:01:59 2335] DEBUG (image:339) No VNC passwd configured for vfb access
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: boot, val: c
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: fda, val: None
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: fdb, val: None
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: soundhw, val: None
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: localtime, val: 0
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: serial, val: None
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: std-vga, val: 0
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: isa, val: 0
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: acpi, val: 1
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: usb, val: 0
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: usbdevice, val: None
[2011-05-24 16:01:59 2335] DEBUG (image:891) args: gfx_passthru, val: 1
[2011-05-24 16:01:59 2335] INFO (image:822) Need to create platform device.[domid:3]
[2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2863) _initDomain:shadow_memory=0x20, memory_static_max=0x80000000, memory_static_min=0x0.
[2011-05-24 16:01:59 2335] INFO (image:182) buildDomain os=hvm dom=3 vcpus=4
[2011-05-24 16:01:59 2335] DEBUG (image:949) domid = 3
[2011-05-24 16:01:59 2335] DEBUG (image:950) image = /usr/lib/xen/boot/hvmloader
[2011-05-24 16:01:59 2335] DEBUG (image:951) store_evtchn = 5
[2011-05-24 16:01:59 2335] DEBUG (image:952) memsize = 2048
[2011-05-24 16:01:59 2335] DEBUG (image:953) target = 2048
[2011-05-24 16:01:59 2335] DEBUG (image:954) vcpus = 4
[2011-05-24 16:01:59 2335] DEBUG (image:955) vcpu_avail = 15
[2011-05-24 16:01:59 2335] DEBUG (image:956) acpi = 1
[2011-05-24 16:01:59 2335] DEBUG (image:957) apic = 1
[2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: console : {'devid': 0, 'protocol': 'vt100', 'location': '6', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50'}
[2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/console/3/0'} to /local/domain/3/device/console/0.
[2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/console/0', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50', 'frontend-id': '3', 'state': '1', 'location': '6', 'online': '1', 'protocol': 'vt100'} to /local/domain/0/backend/console/3/0.
[2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vfb : {'vncunused': '1', 'other_config': {'vncunused': '1', 'vnc': '1'}, 'vnc': '1', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'devid': 0, 'location': '0.0.0.0:5900'}
[2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vfb/3/0'} to /local/domain/3/device/vfb/0.
[2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'vncunused': '1', 'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vfb/0', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'frontend-id': '3', 'state': '1', 'location': '0.0.0.0:5900', 'online': '1', 'vnc': '1'} to /local/domain/0/backend/vfb/3/0.
[2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vbd : {'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': 1, 'devid': 768, 'driver': 'paravirtualised', 'dev': 'hda:disk', 'uname': 'file:/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'VDI': '', 'backend': '0'}
[2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'backend-id': '0', 'virtual-device': '768', 'device-type': 'disk', 'state': '1', 'backend': '/local/domain/0/backend/vbd/3/768'} to /local/domain/3/device/vbd/768.
[2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vbd/768', 'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': '1', 'dev': 'hda', 'state': '1', 'params': '/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'online': '1', 'frontend-id': '3', 'type': 'file'} to /local/domain/0/backend/vbd/3/768.
[2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vif : {'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'devid': 0, 'mac': '00:16:3e:01:a9:c0', 'type': 'ioemu', 'backend': '0'}
[2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vif/3/0'} to /local/domain/3/device/vif/0.
[2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vif/0', 'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'mac': '00:16:3e:01:a9:c0', 'frontend-id': '3', 'state': '1', 'online': '1', 'handle': '0', 'type': 'ioemu'} to /local/domain/0/backend/vif/3/0.
[2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: pci : {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}, {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}, {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}, {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}, {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803'}
[2011-05-24 16:02:04 2335] INFO (image:418) spawning device models: /usr/lib/xen/bin/qemu-dm ['/usr/lib/xen/bin/qemu-dm', '-d', '3', '-domain-name', 'EVM_BackUp', '-videoram', '4', '-vnc', '0.0.0.0:0', '-vncunused', '-vcpus', '4', '-vcpu_avail', '0xfL', '-boot', 'c', '-acpi', '-gfx_passthru', '1', '-net', 'nic,vlan=1,macaddr=00:16:3e:01:a9:c0,model=rtl8139', '-net', 'tap,vlan=1,ifname=tap3.0,bridge=xenbr0', '-M', 'xenfv']
[2011-05-24 16:02:04 2335] INFO (image:467) device model pid: 4092
[2011-05-24 16:02:04 2335] INFO (image:590) waiting for sentinel_fifo
[2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x02'], ['domain', '0x0000'], ['key', '00:02.0'], ['bus', '0x00'], ['vdevfn', '0x10'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '16d91793-5fe2-e409-9a8c-1bae56a4ff48']], ['state', 'Initialising'], ['sub_state', 'Booting']]
[2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}], 'states': ['Initialising']}
[2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}
[2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:02.0@10,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:02.0@10,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:04 2335] DEBUG (image:508) signalDeviceModel: orig_state is None, retrying
[2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
[2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
[2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
[2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
[2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
[2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
[2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
[2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
[2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 23
[2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
[2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 22
[2011-05-24 16:02:05 2335] DEBUG (pciif:456) pci: register aer watch /local/domain/0/backend/pci/3/0/aerState
[2011-05-24 16:02:05 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/pci/3/0'} to /local/domain/3/device/pci/0.
[2011-05-24 16:02:05 2335] DEBUG (DevController:97) DevController: writing {'key-3': '00:1b.0', 'key-2': '00:1d.0', 'key-1': '00:1a.0', 'key-0': '00:02.0', 'frontend': '/local/domain/3/device/pci/0', 'key-4': '00:1f.0', 'domain': 'EVM_BackUp', 'uuid-4': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93', 'uuid-0': '16d91793-5fe2-e409-9a8c-1bae56a4ff48', 'uuid-1': '20820900-e0aa-0fa8-00a5-07a5c80ff58d', 'uuid-2': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea', 'uuid-3': 'e693ed56-4bb1-244a-32c2-2b1467da9788', 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803', 'state': '1', 'online': '1', 'opts-4': 'msitranslate=1,power_mgmt=0', 'opts-3': 'msitranslate=1,power_mgmt=0', 'opts-2': 'msitranslate=1,power_mgmt=0', 'opts-1': 'msitranslate=1,power_mgmt=0', 'opts-0': 'msitranslate=1,power_mgmt=0', 'dev-3': '0000:00:1b.0', 'dev-2': '0000:00:1d.0', 'dev-1': '0000:00:1a.0', 'dev-0': '0000:00:02.0', 'dev-4': '0000:00:1f.0', 'vdevfn-4': '40', 'vdevfn-0': '10', 'vdevfn-1': '28', 'vdevfn-2': '30', 'vdevfn-3': '38', 'frontend-id': '3', 'num_devs': '5'} to /local/domain/0/backend/pci/3/0.
[2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:02.0.
[2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
[2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
[2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
[2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
[2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1a'], ['domain', '0x0000'], ['key', '00:1a.0'], ['bus', '0x00'], ['vdevfn', '0x28'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '20820900-e0aa-0fa8-00a5-07a5c80ff58d']], ['state', 'Initialising'], ['sub_state', 'Booting']]
[2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}], 'states': ['Initialising']}
[2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}
[2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
[2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1a.0.
[2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
[2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
[2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1d'], ['domain', '0x0000'], ['key', '00:1d.0'], ['bus', '0x00'], ['vdevfn', '0x30'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea']], ['state', 'Initialising'], ['sub_state', 'Booting']]
[2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}], 'states': ['Initialising']}
[2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}
[2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:06 2335] INFO (image:538) signalDeviceModel:restore dm state to running
[2011-05-24 16:02:06 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1d.0.
[2011-05-24 16:02:06 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
[2011-05-24 16:02:06 2335] DEBUG (pciif:351) pci: enabling irq 23
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1b'], ['domain', '0x0000'], ['key', '00:1b.0'], ['bus', '0x00'], ['vdevfn', '0x38'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'e693ed56-4bb1-244a-32c2-2b1467da9788']], ['state', 'Initialising'], ['sub_state', 'Booting']]
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}], 'states': ['Initialising']}
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:07 2335] INFO (image:538) signalDeviceModel:restore dm state to running
[2011-05-24 16:02:07 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1b.0.
[2011-05-24 16:02:07 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
[2011-05-24 16:02:07 2335] DEBUG (pciif:351) pci: enabling irq 22
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1f'], ['domain', '0x0000'], ['key', '00:1f.0'], ['bus', '0x00'], ['vdevfn', '0x40'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93']], ['state', 'Initialising'], ['sub_state', 'Booting']]
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'states': ['Initialising']}
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
[2011-05-24 16:02:08 2335] INFO (image:538) signalDeviceModel:restore dm state to running
[2011-05-24 16:02:08 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1f.0.
[2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:3420) Storing VM details: {'on_xend_stop': 'ignore', 'pool_name': 'Pool-0', 'shadow_memory': '32', 'uuid': 'a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'on_reboot': 'restart', 'start_time': '1306281728.59', 'on_poweroff': 'destroy', 'bootloader_args': '', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '4', 'vcpu_avail': '15', 'bootloader': '', 'image': "(hvm (kernel '') (superpages 0) (videoram 4) (hpet 0) (stdvga 0) (loader /usr/lib/xen/boot/hvmloader) (xen_platform_pci 1) (rtc_timeoffset 0) (pci ((0x0000 0x00 0x02 0x0 0x10 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1a 0x0 0x28 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1d 0x0 0x30 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1b 0x0 0x38 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1f 0x0 0x40 ((msitranslate 1) (power_mgmt 0))))) (hap 1) (localtime 0) (timer_mode 1) (pci_msitranslate 1) (oos 1) (apic 1) (vpt_align 1) (vncunused 1) (gfx_passthru 1) (boot c) (pae 0) (viridian 0) (acpi 1) (vnc 1) (nographic 0) (nomigrate 0) (usb 0) (tsc_mode 0) (guest_os_type default) (device_model /usr/lib/xen/bin/qemu-dm) (pci_power_mgmt 0) (xauthority /root/.Xauthority) (isa 0) (notes (SUSPEND_CANCEL 1)))", 'name': 'EVM_BackUp'}
[2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1794) Storing domain details: {'console/port': '6', 'cpu/3/availability': 'online', 'description': '', 'console/limit': '1048576', 'store/port': '5', 'cpu/2/availability': 'online', 'vm': '/vm/a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'domid': '3', 'image/suspend-cancel': '1', 'cpu/0/availability': 'online', 'memory/target': '2097152', 'control/platform-feature-multiprocessor-suspend': '1', 'store/ring-ref': '1044476', 'cpu/1/availability': 'online', 'console/type': 'ioemu', 'name': 'EVM_BackUp'}
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap2.
[2011-05-24 16:02:08 2335] DEBUG (pciif:460) XendDomainInfo.handleAerStateWatch
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif.
[2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
[2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1881) XendDomainInfo.handleShutdownWatch
[2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vif/3/0/hotplug-status.
[2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vkbd.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices ioports.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif2.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices console.
[2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vscsi.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vbd.
[2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 768.
[2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vbd/3/768/hotplug-status.
[2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices irq.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vfb.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices pci.
[2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vusb.
[2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vtpm.
[2011-05-24 16:09:13 2335] DEBUG (XendDomainInfo:3071) XendDomainInfo.destroy: domid=3
Thanks,
Wei Cao | ZZH2 2W081 | 8 821 5739 | 021 61165739 | http://flexservices.intel.com/
^ permalink raw reply [flat|nested] 7+ messages in thread
* Any suggesions for "Gfx passthru" on GTS 450?
[not found] <b9bc7c5f-5617-4273-a293-7e5b98fb61d9@orsmsx602.amr.corp.intel.com>
2011-05-30 9:08 ` XEN 4.1 pci passthru performance regression? Cao, Buddy
@ 2011-05-30 9:12 ` Cao, Buddy
2011-05-30 15:37 ` Pasi Kärkkäinen
1 sibling, 1 reply; 7+ messages in thread
From: Cao, Buddy @ 2011-05-30 9:12 UTC (permalink / raw)
To: xen-devel
Hi, I want to have a try for vga passthru on nvidia GTS 450. Two questions regarding "Gfx passthru" at http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00441.html
1. does XEN4.1 production release includes the patches in the link?
2. If not, what suggestions from you, can I still utilize the patches in the link?
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Any suggesions for "Gfx passthru" on GTS 450?
2011-05-30 9:12 ` Any suggesions for "Gfx passthru" on GTS 450? Cao, Buddy
@ 2011-05-30 15:37 ` Pasi Kärkkäinen
0 siblings, 0 replies; 7+ messages in thread
From: Pasi Kärkkäinen @ 2011-05-30 15:37 UTC (permalink / raw)
To: Cao, Buddy; +Cc: xen-devel
On Mon, May 30, 2011 at 05:12:40PM +0800, Cao, Buddy wrote:
> Hi, I want to have a try for vga passthru on nvidia GTS 450. Two questions regarding "Gfx passthru" at http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00441.html
> 1. does XEN4.1 production release includes the patches in the link?
> 2. If not, what suggestions from you, can I still utilize the patches in the link?
>
I don't think those patches are merged in..
so you need to apply the patches, and probably
do some modifications to match your hardware/setup.
Are you interested in cleaning up the patches and getting them
merged in to Xen source trees?
-- Pasi
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: XEN 4.1 pci passthru performance regression?
2011-05-30 9:08 ` XEN 4.1 pci passthru performance regression? Cao, Buddy
@ 2011-05-31 12:14 ` Ian Campbell
2011-06-01 1:41 ` Cao, Buddy
0 siblings, 1 reply; 7+ messages in thread
From: Ian Campbell @ 2011-05-31 12:14 UTC (permalink / raw)
To: Cao, Buddy; +Cc: xen-devel
On Mon, 2011-05-30 at 10:08 +0100, Cao, Buddy wrote:
> I migrate Xen4.0 to XEN4.1, and found XEN4.1 performance is not as
> good as 4.0.2. xm shutdown takes longer time, control thru libvirt
> take longer time, and since I'm testing PCI/VGA passthru, the
> "createdevice: pci" also take around 5~6 seconds comparing to 1-
> second in XEN4.0.2.
Are you referring only to control operation performance or also to
actual guest performance?
Do you find that guests without any PCI passthrough have also slowed
down or is the issue purely for guests with a passed-through device?
> below is my configuration, do you have any suggestions to make the
> performance better on XEN4.1?
I'm afraid the only suggestion I can think of is to dig in and begin
instrumenting the toolstack in order to figure out where the time is
going.
I suppose it might be worth trying xl on 4.1 to see if it seems slow.
I'm not sure if xl in 4.0 will be functional enough to also provide a
comparison, although I guess you could try.
Ian.
>
> xend-config.sxp:
> (xend-unix-server yes)
> (xend-http-server yes)
> (xend-relocation-server yes)
> (xend-port 8000)
> (xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
> (network-script network-bridge)
> (vif-script vif-bridge)
> (dom0-min-mem 768)
> (enable-dom0-ballooning yes)
> (total_available_memory 0)
> (dom0-cpus 0)
> (vnc-listen '0.0.0.0')
> (vncpasswd '')
>
>
>
> vm1.cfg:
> import os, re
> arch = os.uname()[4]
> name = "b"
> maxmem = 2048
> memory = 2048
> vcpus = 4
> builder = "hvm"
> kernel = "/usr/lib/xen/boot/hvmloader"
> boot = "c"
> pae = 0
> acpi = 1
> apic = 1
> timer_mode =1
> shadow_memory = 32
> on_poweroff = "destroy"
> on_reboot = "restart"
> on_crash = "restart"
> device_model = "/usr/lib/xen/bin/qemu-dm"
> vnc = 1
> disk = [ "file:/opt/EVM/domain/a_disk.img,hda,w" ]
> vif = [ "type=ioemu" ]
> gfx_passthru=1
> pci= ['00:02.0','00:1a.0','00:1d.0','00:1b.0' , '00:1f.0']
>
>
>
> xend.log
> [2011-05-24 16:01:59 2335] DEBUG (SrvDomain:76) Starting domain EVM_BackUp False
> [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2498) XendDomainInfo.constructDomain
> [2011-05-24 16:01:59 2335] DEBUG (balloon:187) Balloon: 2150848 KiB free; need 16384; done.
> [2011-05-24 16:01:59 2335] DEBUG (XendDomain:476) Adding Domain: 3
> [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2836) XendDomainInfo.initDomain: 3 256
> [2011-05-24 16:01:59 2335] DEBUG (image:339) No VNC passwd configured for vfb access
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: boot, val: c
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: fda, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: fdb, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: soundhw, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: localtime, val: 0
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: serial, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: std-vga, val: 0
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: isa, val: 0
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: acpi, val: 1
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: usb, val: 0
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: usbdevice, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: gfx_passthru, val: 1
> [2011-05-24 16:01:59 2335] INFO (image:822) Need to create platform device.[domid:3]
> [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2863) _initDomain:shadow_memory=0x20, memory_static_max=0x80000000, memory_static_min=0x0.
> [2011-05-24 16:01:59 2335] INFO (image:182) buildDomain os=hvm dom=3 vcpus=4
> [2011-05-24 16:01:59 2335] DEBUG (image:949) domid = 3
> [2011-05-24 16:01:59 2335] DEBUG (image:950) image = /usr/lib/xen/boot/hvmloader
> [2011-05-24 16:01:59 2335] DEBUG (image:951) store_evtchn = 5
> [2011-05-24 16:01:59 2335] DEBUG (image:952) memsize = 2048
> [2011-05-24 16:01:59 2335] DEBUG (image:953) target = 2048
> [2011-05-24 16:01:59 2335] DEBUG (image:954) vcpus = 4
> [2011-05-24 16:01:59 2335] DEBUG (image:955) vcpu_avail = 15
> [2011-05-24 16:01:59 2335] DEBUG (image:956) acpi = 1
> [2011-05-24 16:01:59 2335] DEBUG (image:957) apic = 1
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: console : {'devid': 0, 'protocol': 'vt100', 'location': '6', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50'}
> [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/console/3/0'} to /local/domain/3/device/console/0.
> [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/console/0', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50', 'frontend-id': '3', 'state': '1', 'location': '6', 'online': '1', 'protocol': 'vt100'} to /local/domain/0/backend/console/3/0.
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vfb : {'vncunused': '1', 'other_config': {'vncunused': '1', 'vnc': '1'}, 'vnc': '1', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'devid': 0, 'location': '0.0.0.0:5900'}
> [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vfb/3/0'} to /local/domain/3/device/vfb/0.
> [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'vncunused': '1', 'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vfb/0', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'frontend-id': '3', 'state': '1', 'location': '0.0.0.0:5900', 'online': '1', 'vnc': '1'} to /local/domain/0/backend/vfb/3/0.
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vbd : {'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': 1, 'devid': 768, 'driver': 'paravirtualised', 'dev': 'hda:disk', 'uname': 'file:/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'VDI': '', 'backend': '0'}
> [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'backend-id': '0', 'virtual-device': '768', 'device-type': 'disk', 'state': '1', 'backend': '/local/domain/0/backend/vbd/3/768'} to /local/domain/3/device/vbd/768.
> [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vbd/768', 'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': '1', 'dev': 'hda', 'state': '1', 'params': '/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'online': '1', 'frontend-id': '3', 'type': 'file'} to /local/domain/0/backend/vbd/3/768.
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vif : {'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'devid': 0, 'mac': '00:16:3e:01:a9:c0', 'type': 'ioemu', 'backend': '0'}
> [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vif/3/0'} to /local/domain/3/device/vif/0.
> [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vif/0', 'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'mac': '00:16:3e:01:a9:c0', 'frontend-id': '3', 'state': '1', 'online': '1', 'handle': '0', 'type': 'ioemu'} to /local/domain/0/backend/vif/3/0.
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: pci : {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}, {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}, {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}, {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}, {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803'}
> [2011-05-24 16:02:04 2335] INFO (image:418) spawning device models: /usr/lib/xen/bin/qemu-dm ['/usr/lib/xen/bin/qemu-dm', '-d', '3', '-domain-name', 'EVM_BackUp', '-videoram', '4', '-vnc', '0.0.0.0:0', '-vncunused', '-vcpus', '4', '-vcpu_avail', '0xfL', '-boot', 'c', '-acpi', '-gfx_passthru', '1', '-net', 'nic,vlan=1,macaddr=00:16:3e:01:a9:c0,model=rtl8139', '-net', 'tap,vlan=1,ifname=tap3.0,bridge=xenbr0', '-M', 'xenfv']
> [2011-05-24 16:02:04 2335] INFO (image:467) device model pid: 4092
> [2011-05-24 16:02:04 2335] INFO (image:590) waiting for sentinel_fifo
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x02'], ['domain', '0x0000'], ['key', '00:02.0'], ['bus', '0x00'], ['vdevfn', '0x10'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '16d91793-5fe2-e409-9a8c-1bae56a4ff48']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}], 'states': ['Initialising']}
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:02.0@10,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:02.0@10,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:04 2335] DEBUG (image:508) signalDeviceModel: orig_state is None, retrying
> [2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 23
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 22
> [2011-05-24 16:02:05 2335] DEBUG (pciif:456) pci: register aer watch /local/domain/0/backend/pci/3/0/aerState
> [2011-05-24 16:02:05 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/pci/3/0'} to /local/domain/3/device/pci/0.
> [2011-05-24 16:02:05 2335] DEBUG (DevController:97) DevController: writing {'key-3': '00:1b.0', 'key-2': '00:1d.0', 'key-1': '00:1a.0', 'key-0': '00:02.0', 'frontend': '/local/domain/3/device/pci/0', 'key-4': '00:1f.0', 'domain': 'EVM_BackUp', 'uuid-4': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93', 'uuid-0': '16d91793-5fe2-e409-9a8c-1bae56a4ff48', 'uuid-1': '20820900-e0aa-0fa8-00a5-07a5c80ff58d', 'uuid-2': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea', 'uuid-3': 'e693ed56-4bb1-244a-32c2-2b1467da9788', 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803', 'state': '1', 'online': '1', 'opts-4': 'msitranslate=1,power_mgmt=0', 'opts-3': 'msitranslate=1,power_mgmt=0', 'opts-2': 'msitranslate=1,power_mgmt=0', 'opts-1': 'msitranslate=1,power_mgmt=0', 'opts-0': 'msitranslate=1,power_mgmt=0', 'dev-3': '0000:00:1b.0', 'dev-2': '0000:00:1d.0', 'dev-1': '0000:00:1a.0', 'dev-0': '0000:00:02.0', 'dev-4': '0000:00:1f.0', 'vdevfn-4': '40', 'vdevfn-0': '10', 'vdevfn-1': '28', 'vdevfn-2': '30', 'vdevfn-3': '38', 'frontend-id': '3', 'num_devs': '5'} to /local/domain/0/backend/pci/3/0.
> [2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:02.0.
> [2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1a'], ['domain', '0x0000'], ['key', '00:1a.0'], ['bus', '0x00'], ['vdevfn', '0x28'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '20820900-e0aa-0fa8-00a5-07a5c80ff58d']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}], 'states': ['Initialising']}
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1a.0.
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1d'], ['domain', '0x0000'], ['key', '00:1d.0'], ['bus', '0x00'], ['vdevfn', '0x30'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}], 'states': ['Initialising']}
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:06 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:06 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1d.0.
> [2011-05-24 16:02:06 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
> [2011-05-24 16:02:06 2335] DEBUG (pciif:351) pci: enabling irq 23
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1b'], ['domain', '0x0000'], ['key', '00:1b.0'], ['bus', '0x00'], ['vdevfn', '0x38'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'e693ed56-4bb1-244a-32c2-2b1467da9788']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}], 'states': ['Initialising']}
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:07 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:07 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1b.0.
> [2011-05-24 16:02:07 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
> [2011-05-24 16:02:07 2335] DEBUG (pciif:351) pci: enabling irq 22
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1f'], ['domain', '0x0000'], ['key', '00:1f.0'], ['bus', '0x00'], ['vdevfn', '0x40'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'states': ['Initialising']}
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:08 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:08 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1f.0.
> [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:3420) Storing VM details: {'on_xend_stop': 'ignore', 'pool_name': 'Pool-0', 'shadow_memory': '32', 'uuid': 'a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'on_reboot': 'restart', 'start_time': '1306281728.59', 'on_poweroff': 'destroy', 'bootloader_args': '', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '4', 'vcpu_avail': '15', 'bootloader': '', 'image': "(hvm (kernel '') (superpages 0) (videoram 4) (hpet 0) (stdvga 0) (loader /usr/lib/xen/boot/hvmloader) (xen_platform_pci 1) (rtc_timeoffset 0) (pci ((0x0000 0x00 0x02 0x0 0x10 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1a 0x0 0x28 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1d 0x0 0x30 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1b 0x0 0x38 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1f 0x0 0x40 ((msitranslate 1) (power_mgmt 0))))) (hap 1) (localtime 0) (timer_mode 1) (pci_msitranslate 1) (oos 1) (apic 1) (vpt_align 1) (vncunused 1) (gfx_passthru 1) (boot c) (pae 0) (viridian 0) (acpi 1) (vnc 1) (nographic 0) (nomigrate 0) (usb 0) (tsc_mode 0) (guest_os_type default) (device_model /usr/lib/xen/bin/qemu-dm) (pci_power_mgmt 0) (xauthority /root/.Xauthority) (isa 0) (notes (SUSPEND_CANCEL 1)))", 'name': 'EVM_BackUp'}
> [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1794) Storing domain details: {'console/port': '6', 'cpu/3/availability': 'online', 'description': '', 'console/limit': '1048576', 'store/port': '5', 'cpu/2/availability': 'online', 'vm': '/vm/a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'domid': '3', 'image/suspend-cancel': '1', 'cpu/0/availability': 'online', 'memory/target': '2097152', 'control/platform-feature-multiprocessor-suspend': '1', 'store/ring-ref': '1044476', 'cpu/1/availability': 'online', 'console/type': 'ioemu', 'name': 'EVM_BackUp'}
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap2.
> [2011-05-24 16:02:08 2335] DEBUG (pciif:460) XendDomainInfo.handleAerStateWatch
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1881) XendDomainInfo.handleShutdownWatch
> [2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vif/3/0/hotplug-status.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vkbd.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices ioports.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif2.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices console.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vscsi.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vbd.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 768.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vbd/3/768/hotplug-status.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices irq.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vfb.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices pci.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vusb.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vtpm.
> [2011-05-24 16:09:13 2335] DEBUG (XendDomainInfo:3071) XendDomainInfo.destroy: domid=3
>
>
> Thanks,
> Wei Cao | ZZH2 2W081 | 8 821 5739 | 021 61165739 | http://flexservices.intel.com/
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: XEN 4.1 pci passthru performance regression?
2011-05-31 12:14 ` Ian Campbell
@ 2011-06-01 1:41 ` Cao, Buddy
2011-06-01 7:55 ` Ian Campbell
0 siblings, 1 reply; 7+ messages in thread
From: Cao, Buddy @ 2011-06-01 1:41 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
[-- Attachment #1: Type: text/plain, Size: 25014 bytes --]
Thanks Ian for you reply.
>Are you referring only to control operation performance or also to
>actual guest performance?
It does not impact actual guest performance, only make vm start and shutdown slow.
>Do you find that guests without any PCI passthrough have also slowed
>down or is the issue purely for guests with a passed-through device?
The guests without any PCI passthrough works as well as xen4.0.2.
I tired xl, but looks it always returns "libxl....." relevant error, and none of the command can run properly
Thanks,
Wei Cao | ZZH2 2W081 | 8 821 5739 | 021 61165739 | http://flexservices.intel.com/
-----Original Message-----
From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
Sent: Tuesday, May 31, 2011 8:15 PM
To: Cao, Buddy
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] XEN 4.1 pci passthru performance regression?
On Mon, 2011-05-30 at 10:08 +0100, Cao, Buddy wrote:
> I migrate Xen4.0 to XEN4.1, and found XEN4.1 performance is not as
> good as 4.0.2. xm shutdown takes longer time, control thru libvirt
> take longer time, and since I'm testing PCI/VGA passthru, the
> "createdevice: pci" also take around 5~6 seconds comparing to 1-
> second in XEN4.0.2.
Are you referring only to control operation performance or also to
actual guest performance?
Do you find that guests without any PCI passthrough have also slowed
down or is the issue purely for guests with a passed-through device?
> below is my configuration, do you have any suggestions to make the
> performance better on XEN4.1?
I'm afraid the only suggestion I can think of is to dig in and begin
instrumenting the toolstack in order to figure out where the time is
going.
I suppose it might be worth trying xl on 4.1 to see if it seems slow.
I'm not sure if xl in 4.0 will be functional enough to also provide a
comparison, although I guess you could try.
Ian.
>
> xend-config.sxp:
> (xend-unix-server yes)
> (xend-http-server yes)
> (xend-relocation-server yes)
> (xend-port 8000)
> (xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
> (network-script network-bridge)
> (vif-script vif-bridge)
> (dom0-min-mem 768)
> (enable-dom0-ballooning yes)
> (total_available_memory 0)
> (dom0-cpus 0)
> (vnc-listen '0.0.0.0')
> (vncpasswd '')
>
>
>
> vm1.cfg:
> import os, re
> arch = os.uname()[4]
> name = "b"
> maxmem = 2048
> memory = 2048
> vcpus = 4
> builder = "hvm"
> kernel = "/usr/lib/xen/boot/hvmloader"
> boot = "c"
> pae = 0
> acpi = 1
> apic = 1
> timer_mode =1
> shadow_memory = 32
> on_poweroff = "destroy"
> on_reboot = "restart"
> on_crash = "restart"
> device_model = "/usr/lib/xen/bin/qemu-dm"
> vnc = 1
> disk = [ "file:/opt/EVM/domain/a_disk.img,hda,w" ]
> vif = [ "type=ioemu" ]
> gfx_passthru=1
> pci= ['00:02.0','00:1a.0','00:1d.0','00:1b.0' , '00:1f.0']
>
>
>
> xend.log
> [2011-05-24 16:01:59 2335] DEBUG (SrvDomain:76) Starting domain EVM_BackUp False
> [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2498) XendDomainInfo.constructDomain
> [2011-05-24 16:01:59 2335] DEBUG (balloon:187) Balloon: 2150848 KiB free; need 16384; done.
> [2011-05-24 16:01:59 2335] DEBUG (XendDomain:476) Adding Domain: 3
> [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2836) XendDomainInfo.initDomain: 3 256
> [2011-05-24 16:01:59 2335] DEBUG (image:339) No VNC passwd configured for vfb access
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: boot, val: c
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: fda, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: fdb, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: soundhw, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: localtime, val: 0
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: serial, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: std-vga, val: 0
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: isa, val: 0
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: acpi, val: 1
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: usb, val: 0
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: usbdevice, val: None
> [2011-05-24 16:01:59 2335] DEBUG (image:891) args: gfx_passthru, val: 1
> [2011-05-24 16:01:59 2335] INFO (image:822) Need to create platform device.[domid:3]
> [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2863) _initDomain:shadow_memory=0x20, memory_static_max=0x80000000, memory_static_min=0x0.
> [2011-05-24 16:01:59 2335] INFO (image:182) buildDomain os=hvm dom=3 vcpus=4
> [2011-05-24 16:01:59 2335] DEBUG (image:949) domid = 3
> [2011-05-24 16:01:59 2335] DEBUG (image:950) image = /usr/lib/xen/boot/hvmloader
> [2011-05-24 16:01:59 2335] DEBUG (image:951) store_evtchn = 5
> [2011-05-24 16:01:59 2335] DEBUG (image:952) memsize = 2048
> [2011-05-24 16:01:59 2335] DEBUG (image:953) target = 2048
> [2011-05-24 16:01:59 2335] DEBUG (image:954) vcpus = 4
> [2011-05-24 16:01:59 2335] DEBUG (image:955) vcpu_avail = 15
> [2011-05-24 16:01:59 2335] DEBUG (image:956) acpi = 1
> [2011-05-24 16:01:59 2335] DEBUG (image:957) apic = 1
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: console : {'devid': 0, 'protocol': 'vt100', 'location': '6', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50'}
> [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/console/3/0'} to /local/domain/3/device/console/0.
> [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/console/0', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50', 'frontend-id': '3', 'state': '1', 'location': '6', 'online': '1', 'protocol': 'vt100'} to /local/domain/0/backend/console/3/0.
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vfb : {'vncunused': '1', 'other_config': {'vncunused': '1', 'vnc': '1'}, 'vnc': '1', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'devid': 0, 'location': '0.0.0.0:5900'}
> [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vfb/3/0'} to /local/domain/3/device/vfb/0.
> [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'vncunused': '1', 'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vfb/0', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'frontend-id': '3', 'state': '1', 'location': '0.0.0.0:5900', 'online': '1', 'vnc': '1'} to /local/domain/0/backend/vfb/3/0.
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vbd : {'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': 1, 'devid': 768, 'driver': 'paravirtualised', 'dev': 'hda:disk', 'uname': 'file:/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'VDI': '', 'backend': '0'}
> [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'backend-id': '0', 'virtual-device': '768', 'device-type': 'disk', 'state': '1', 'backend': '/local/domain/0/backend/vbd/3/768'} to /local/domain/3/device/vbd/768.
> [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vbd/768', 'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': '1', 'dev': 'hda', 'state': '1', 'params': '/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'online': '1', 'frontend-id': '3', 'type': 'file'} to /local/domain/0/backend/vbd/3/768.
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vif : {'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'devid': 0, 'mac': '00:16:3e:01:a9:c0', 'type': 'ioemu', 'backend': '0'}
> [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vif/3/0'} to /local/domain/3/device/vif/0.
> [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vif/0', 'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'mac': '00:16:3e:01:a9:c0', 'frontend-id': '3', 'state': '1', 'online': '1', 'handle': '0', 'type': 'ioemu'} to /local/domain/0/backend/vif/3/0.
> [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: pci : {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}, {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}, {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}, {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}, {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803'}
> [2011-05-24 16:02:04 2335] INFO (image:418) spawning device models: /usr/lib/xen/bin/qemu-dm ['/usr/lib/xen/bin/qemu-dm', '-d', '3', '-domain-name', 'EVM_BackUp', '-videoram', '4', '-vnc', '0.0.0.0:0', '-vncunused', '-vcpus', '4', '-vcpu_avail', '0xfL', '-boot', 'c', '-acpi', '-gfx_passthru', '1', '-net', 'nic,vlan=1,macaddr=00:16:3e:01:a9:c0,model=rtl8139', '-net', 'tap,vlan=1,ifname=tap3.0,bridge=xenbr0', '-M', 'xenfv']
> [2011-05-24 16:02:04 2335] INFO (image:467) device model pid: 4092
> [2011-05-24 16:02:04 2335] INFO (image:590) waiting for sentinel_fifo
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x02'], ['domain', '0x0000'], ['key', '00:02.0'], ['bus', '0x00'], ['vdevfn', '0x10'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '16d91793-5fe2-e409-9a8c-1bae56a4ff48']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}], 'states': ['Initialising']}
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:02.0@10,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:02.0@10,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:04 2335] DEBUG (image:508) signalDeviceModel: orig_state is None, retrying
> [2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 23
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 22
> [2011-05-24 16:02:05 2335] DEBUG (pciif:456) pci: register aer watch /local/domain/0/backend/pci/3/0/aerState
> [2011-05-24 16:02:05 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/pci/3/0'} to /local/domain/3/device/pci/0.
> [2011-05-24 16:02:05 2335] DEBUG (DevController:97) DevController: writing {'key-3': '00:1b.0', 'key-2': '00:1d.0', 'key-1': '00:1a.0', 'key-0': '00:02.0', 'frontend': '/local/domain/3/device/pci/0', 'key-4': '00:1f.0', 'domain': 'EVM_BackUp', 'uuid-4': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93', 'uuid-0': '16d91793-5fe2-e409-9a8c-1bae56a4ff48', 'uuid-1': '20820900-e0aa-0fa8-00a5-07a5c80ff58d', 'uuid-2': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea', 'uuid-3': 'e693ed56-4bb1-244a-32c2-2b1467da9788', 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803', 'state': '1', 'online': '1', 'opts-4': 'msitranslate=1,power_mgmt=0', 'opts-3': 'msitranslate=1,power_mgmt=0', 'opts-2': 'msitranslate=1,power_mgmt=0', 'opts-1': 'msitranslate=1,power_mgmt=0', 'opts-0': 'msitranslate=1,power_mgmt=0', 'dev-3': '0000:00:1b.0', 'dev-2': '0000:00:1d.0', 'dev-1': '0000:00:1a.0', 'dev-0': '0000:00:02.0', 'dev-4': '0000:00:1f.0', 'vdevfn-4': '40', 'vdevfn-0': '10', 'vdevfn-1': '28', 'vdevfn-2': '30', 'vdevfn-3': '38', 'frontend-id': '3', 'num_devs': '5'} to /local/domain/0/backend/pci/3/0.
> [2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:02.0.
> [2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1a'], ['domain', '0x0000'], ['key', '00:1a.0'], ['bus', '0x00'], ['vdevfn', '0x28'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '20820900-e0aa-0fa8-00a5-07a5c80ff58d']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}], 'states': ['Initialising']}
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1a.0.
> [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
> [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1d'], ['domain', '0x0000'], ['key', '00:1d.0'], ['bus', '0x00'], ['vdevfn', '0x30'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}], 'states': ['Initialising']}
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:06 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:06 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1d.0.
> [2011-05-24 16:02:06 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
> [2011-05-24 16:02:06 2335] DEBUG (pciif:351) pci: enabling irq 23
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1b'], ['domain', '0x0000'], ['key', '00:1b.0'], ['bus', '0x00'], ['vdevfn', '0x38'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'e693ed56-4bb1-244a-32c2-2b1467da9788']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}], 'states': ['Initialising']}
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:07 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:07 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1b.0.
> [2011-05-24 16:02:07 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
> [2011-05-24 16:02:07 2335] DEBUG (pciif:351) pci: enabling irq 22
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1f'], ['domain', '0x0000'], ['key', '00:1f.0'], ['bus', '0x00'], ['vdevfn', '0x40'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'states': ['Initialising']}
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
> [2011-05-24 16:02:08 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> [2011-05-24 16:02:08 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1f.0.
> [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:3420) Storing VM details: {'on_xend_stop': 'ignore', 'pool_name': 'Pool-0', 'shadow_memory': '32', 'uuid': 'a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'on_reboot': 'restart', 'start_time': '1306281728.59', 'on_poweroff': 'destroy', 'bootloader_args': '', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '4', 'vcpu_avail': '15', 'bootloader': '', 'image': "(hvm (kernel '') (superpages 0) (videoram 4) (hpet 0) (stdvga 0) (loader /usr/lib/xen/boot/hvmloader) (xen_platform_pci 1) (rtc_timeoffset 0) (pci ((0x0000 0x00 0x02 0x0 0x10 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1a 0x0 0x28 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1d 0x0 0x30 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1b 0x0 0x38 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1f 0x0 0x40 ((msitranslate 1) (power_mgmt 0))))) (hap 1) (localtime 0) (timer_mode 1) (pci_msitranslate 1) (oos 1) (apic 1) (vpt_align 1) (vncunused 1) (gfx_passthru 1) (boot c) (pae 0) (viridian 0) (acpi 1) (vnc 1) (nographic 0) (nomigrate 0) (usb 0) (tsc_mode 0) (guest_os_type default) (device_model /usr/lib/xen/bin/qemu-dm) (pci_power_mgmt 0) (xauthority /root/.Xauthority) (isa 0) (notes (SUSPEND_CANCEL 1)))", 'name': 'EVM_BackUp'}
> [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1794) Storing domain details: {'console/port': '6', 'cpu/3/availability': 'online', 'description': '', 'console/limit': '1048576', 'store/port': '5', 'cpu/2/availability': 'online', 'vm': '/vm/a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'domid': '3', 'image/suspend-cancel': '1', 'cpu/0/availability': 'online', 'memory/target': '2097152', 'control/platform-feature-multiprocessor-suspend': '1', 'store/ring-ref': '1044476', 'cpu/1/availability': 'online', 'console/type': 'ioemu', 'name': 'EVM_BackUp'}
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap2.
> [2011-05-24 16:02:08 2335] DEBUG (pciif:460) XendDomainInfo.handleAerStateWatch
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1881) XendDomainInfo.handleShutdownWatch
> [2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vif/3/0/hotplug-status.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vkbd.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices ioports.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif2.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices console.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vscsi.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vbd.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 768.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vbd/3/768/hotplug-status.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices irq.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vfb.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices pci.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vusb.
> [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vtpm.
> [2011-05-24 16:09:13 2335] DEBUG (XendDomainInfo:3071) XendDomainInfo.destroy: domid=3
>
>
> Thanks,
> Wei Cao | ZZH2 2W081 | 8 821 5739 | 021 61165739 | http://flexservices.intel.com/
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: XEN 4.1 pci passthru performance regression?
2011-06-01 1:41 ` Cao, Buddy
@ 2011-06-01 7:55 ` Ian Campbell
2011-06-01 12:44 ` Ian Campbell
0 siblings, 1 reply; 7+ messages in thread
From: Ian Campbell @ 2011-06-01 7:55 UTC (permalink / raw)
To: Cao, Buddy; +Cc: xen-devel
On Wed, 2011-06-01 at 02:41 +0100, Cao, Buddy wrote:
> Thanks Ian for you reply.
>
> >Are you referring only to control operation performance or also to
> >actual guest performance?
>
> It does not impact actual guest performance, only make vm start and shutdown slow.
Oh good!
> >Do you find that guests without any PCI passthrough have also slowed
> >down or is the issue purely for guests with a passed-through device?
>
> The guests without any PCI passthrough works as well as xen4.0.2.
Also good.
> I tired xl, but looks it always returns "libxl....." relevant error,
> and none of the command can run properly
Well, we should also try and resolve this -- xl is expected to work in
4.1. Did you see: http://wiki.xen.org/xenwiki/MigrationGuideToXen4.1+ ?
Please tell us the specifics of the xl failure.
I tried starting a PV guest with a passed through device with both xm
and xl and there was no obvious delay on startup or shutdown.
Unfortunately I don't have a test box which can do passthrough to HVM
guests at the moment (I'll see if I can scare one up). In the meantime
perhaps you could add some extra logging to xend to see where the time
is spent and/or report the xl failues as requested above.
Also, please can you try xen-unstable.hg to see if the problem still
exists there.
Thanks,
Ian.
>
> Thanks,
> Wei Cao | ZZH2 2W081 | 8 821 5739 | 021 61165739 | http://flexservices.intel.com/
>
>
> -----Original Message-----
> From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
> Sent: Tuesday, May 31, 2011 8:15 PM
> To: Cao, Buddy
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] XEN 4.1 pci passthru performance regression?
>
> On Mon, 2011-05-30 at 10:08 +0100, Cao, Buddy wrote:
> > I migrate Xen4.0 to XEN4.1, and found XEN4.1 performance is not as
> > good as 4.0.2. xm shutdown takes longer time, control thru libvirt
> > take longer time, and since I'm testing PCI/VGA passthru, the
> > "createdevice: pci" also take around 5~6 seconds comparing to 1-
> > second in XEN4.0.2.
>
> Are you referring only to control operation performance or also to
> actual guest performance?
>
> Do you find that guests without any PCI passthrough have also slowed
> down or is the issue purely for guests with a passed-through device?
>
> > below is my configuration, do you have any suggestions to make the
> > performance better on XEN4.1?
>
> I'm afraid the only suggestion I can think of is to dig in and begin
> instrumenting the toolstack in order to figure out where the time is
> going.
>
> I suppose it might be worth trying xl on 4.1 to see if it seems slow.
> I'm not sure if xl in 4.0 will be functional enough to also provide a
> comparison, although I guess you could try.
>
> Ian.
>
> >
> > xend-config.sxp:
> > (xend-unix-server yes)
> > (xend-http-server yes)
> > (xend-relocation-server yes)
> > (xend-port 8000)
> > (xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
> > (network-script network-bridge)
> > (vif-script vif-bridge)
> > (dom0-min-mem 768)
> > (enable-dom0-ballooning yes)
> > (total_available_memory 0)
> > (dom0-cpus 0)
> > (vnc-listen '0.0.0.0')
> > (vncpasswd '')
> >
> >
> >
> > vm1.cfg:
> > import os, re
> > arch = os.uname()[4]
> > name = "b"
> > maxmem = 2048
> > memory = 2048
> > vcpus = 4
> > builder = "hvm"
> > kernel = "/usr/lib/xen/boot/hvmloader"
> > boot = "c"
> > pae = 0
> > acpi = 1
> > apic = 1
> > timer_mode =1
> > shadow_memory = 32
> > on_poweroff = "destroy"
> > on_reboot = "restart"
> > on_crash = "restart"
> > device_model = "/usr/lib/xen/bin/qemu-dm"
> > vnc = 1
> > disk = [ "file:/opt/EVM/domain/a_disk.img,hda,w" ]
> > vif = [ "type=ioemu" ]
> > gfx_passthru=1
> > pci= ['00:02.0','00:1a.0','00:1d.0','00:1b.0' , '00:1f.0']
> >
> >
> >
> > xend.log
> > [2011-05-24 16:01:59 2335] DEBUG (SrvDomain:76) Starting domain EVM_BackUp False
> > [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2498) XendDomainInfo.constructDomain
> > [2011-05-24 16:01:59 2335] DEBUG (balloon:187) Balloon: 2150848 KiB free; need 16384; done.
> > [2011-05-24 16:01:59 2335] DEBUG (XendDomain:476) Adding Domain: 3
> > [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2836) XendDomainInfo.initDomain: 3 256
> > [2011-05-24 16:01:59 2335] DEBUG (image:339) No VNC passwd configured for vfb access
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: boot, val: c
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: fda, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: fdb, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: soundhw, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: localtime, val: 0
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: serial, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: std-vga, val: 0
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: isa, val: 0
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: acpi, val: 1
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: usb, val: 0
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: usbdevice, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: gfx_passthru, val: 1
> > [2011-05-24 16:01:59 2335] INFO (image:822) Need to create platform device.[domid:3]
> > [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2863) _initDomain:shadow_memory=0x20, memory_static_max=0x80000000, memory_static_min=0x0.
> > [2011-05-24 16:01:59 2335] INFO (image:182) buildDomain os=hvm dom=3 vcpus=4
> > [2011-05-24 16:01:59 2335] DEBUG (image:949) domid = 3
> > [2011-05-24 16:01:59 2335] DEBUG (image:950) image = /usr/lib/xen/boot/hvmloader
> > [2011-05-24 16:01:59 2335] DEBUG (image:951) store_evtchn = 5
> > [2011-05-24 16:01:59 2335] DEBUG (image:952) memsize = 2048
> > [2011-05-24 16:01:59 2335] DEBUG (image:953) target = 2048
> > [2011-05-24 16:01:59 2335] DEBUG (image:954) vcpus = 4
> > [2011-05-24 16:01:59 2335] DEBUG (image:955) vcpu_avail = 15
> > [2011-05-24 16:01:59 2335] DEBUG (image:956) acpi = 1
> > [2011-05-24 16:01:59 2335] DEBUG (image:957) apic = 1
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: console : {'devid': 0, 'protocol': 'vt100', 'location': '6', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50'}
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/console/3/0'} to /local/domain/3/device/console/0.
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/console/0', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50', 'frontend-id': '3', 'state': '1', 'location': '6', 'online': '1', 'protocol': 'vt100'} to /local/domain/0/backend/console/3/0.
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vfb : {'vncunused': '1', 'other_config': {'vncunused': '1', 'vnc': '1'}, 'vnc': '1', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'devid': 0, 'location': '0.0.0.0:5900'}
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vfb/3/0'} to /local/domain/3/device/vfb/0.
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'vncunused': '1', 'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vfb/0', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'frontend-id': '3', 'state': '1', 'location': '0.0.0.0:5900', 'online': '1', 'vnc': '1'} to /local/domain/0/backend/vfb/3/0.
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vbd : {'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': 1, 'devid': 768, 'driver': 'paravirtualised', 'dev': 'hda:disk', 'uname': 'file:/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'VDI': '', 'backend': '0'}
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'backend-id': '0', 'virtual-device': '768', 'device-type': 'disk', 'state': '1', 'backend': '/local/domain/0/backend/vbd/3/768'} to /local/domain/3/device/vbd/768.
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vbd/768', 'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': '1', 'dev': 'hda', 'state': '1', 'params': '/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'online': '1', 'frontend-id': '3', 'type': 'file'} to /local/domain/0/backend/vbd/3/768.
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vif : {'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'devid': 0, 'mac': '00:16:3e:01:a9:c0', 'type': 'ioemu', 'backend': '0'}
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vif/3/0'} to /local/domain/3/device/vif/0.
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vif/0', 'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'mac': '00:16:3e:01:a9:c0', 'frontend-id': '3', 'state': '1', 'online': '1', 'handle': '0', 'type': 'ioemu'} to /local/domain/0/backend/vif/3/0.
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: pci : {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}, {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}, {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}, {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}, {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803'}
> > [2011-05-24 16:02:04 2335] INFO (image:418) spawning device models: /usr/lib/xen/bin/qemu-dm ['/usr/lib/xen/bin/qemu-dm', '-d', '3', '-domain-name', 'EVM_BackUp', '-videoram', '4', '-vnc', '0.0.0.0:0', '-vncunused', '-vcpus', '4', '-vcpu_avail', '0xfL', '-boot', 'c', '-acpi', '-gfx_passthru', '1', '-net', 'nic,vlan=1,macaddr=00:16:3e:01:a9:c0,model=rtl8139', '-net', 'tap,vlan=1,ifname=tap3.0,bridge=xenbr0', '-M', 'xenfv']
> > [2011-05-24 16:02:04 2335] INFO (image:467) device model pid: 4092
> > [2011-05-24 16:02:04 2335] INFO (image:590) waiting for sentinel_fifo
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x02'], ['domain', '0x0000'], ['key', '00:02.0'], ['bus', '0x00'], ['vdevfn', '0x10'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '16d91793-5fe2-e409-9a8c-1bae56a4ff48']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:02.0@10,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:02.0@10,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:04 2335] DEBUG (image:508) signalDeviceModel: orig_state is None, retrying
> > [2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 23
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 22
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:456) pci: register aer watch /local/domain/0/backend/pci/3/0/aerState
> > [2011-05-24 16:02:05 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/pci/3/0'} to /local/domain/3/device/pci/0.
> > [2011-05-24 16:02:05 2335] DEBUG (DevController:97) DevController: writing {'key-3': '00:1b.0', 'key-2': '00:1d.0', 'key-1': '00:1a.0', 'key-0': '00:02.0', 'frontend': '/local/domain/3/device/pci/0', 'key-4': '00:1f.0', 'domain': 'EVM_BackUp', 'uuid-4': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93', 'uuid-0': '16d91793-5fe2-e409-9a8c-1bae56a4ff48', 'uuid-1': '20820900-e0aa-0fa8-00a5-07a5c80ff58d', 'uuid-2': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea', 'uuid-3': 'e693ed56-4bb1-244a-32c2-2b1467da9788', 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803', 'state': '1', 'online': '1', 'opts-4': 'msitranslate=1,power_mgmt=0', 'opts-3': 'msitranslate=1,power_mgmt=0', 'opts-2': 'msitranslate=1,power_mgmt=0', 'opts-1': 'msitranslate=1,power_mgmt=0', 'opts-0': 'msitranslate=1,power_mgmt=0', 'dev-3': '0000:00:1b.0', 'dev-2': '0000:00:1d.0', 'dev-1': '0000:00:1a.0', 'dev-0': '0000:00:02.0', 'dev-4': '0000:00:1f.0', 'vdevfn-4': '40', 'vdevfn-0': '10', 'vdevfn-1': '28', 'vdevfn-2': '30', 'vdevfn-3': '38', 'frontend-id': '3', 'num_devs': '5'} to /local/domain/0/backend/pci/3/0.
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:02.0.
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1a'], ['domain', '0x0000'], ['key', '00:1a.0'], ['bus', '0x00'], ['vdevfn', '0x28'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '20820900-e0aa-0fa8-00a5-07a5c80ff58d']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1a.0.
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1d'], ['domain', '0x0000'], ['key', '00:1d.0'], ['bus', '0x00'], ['vdevfn', '0x30'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:06 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:06 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1d.0.
> > [2011-05-24 16:02:06 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
> > [2011-05-24 16:02:06 2335] DEBUG (pciif:351) pci: enabling irq 23
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1b'], ['domain', '0x0000'], ['key', '00:1b.0'], ['bus', '0x00'], ['vdevfn', '0x38'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'e693ed56-4bb1-244a-32c2-2b1467da9788']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:07 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:07 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1b.0.
> > [2011-05-24 16:02:07 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
> > [2011-05-24 16:02:07 2335] DEBUG (pciif:351) pci: enabling irq 22
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1f'], ['domain', '0x0000'], ['key', '00:1f.0'], ['bus', '0x00'], ['vdevfn', '0x40'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:08 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:08 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1f.0.
> > [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:3420) Storing VM details: {'on_xend_stop': 'ignore', 'pool_name': 'Pool-0', 'shadow_memory': '32', 'uuid': 'a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'on_reboot': 'restart', 'start_time': '1306281728.59', 'on_poweroff': 'destroy', 'bootloader_args': '', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '4', 'vcpu_avail': '15', 'bootloader': '', 'image': "(hvm (kernel '') (superpages 0) (videoram 4) (hpet 0) (stdvga 0) (loader /usr/lib/xen/boot/hvmloader) (xen_platform_pci 1) (rtc_timeoffset 0) (pci ((0x0000 0x00 0x02 0x0 0x10 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1a 0x0 0x28 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1d 0x0 0x30 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1b 0x0 0x38 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1f 0x0 0x40 ((msitranslate 1) (power_mgmt 0))))) (hap 1) (localtime 0) (timer_mode 1) (pci_msitranslate 1) (oos 1) (apic 1) (vpt_align 1) (vncunused 1) (gfx_passthru 1) (boot c) (pae 0) (viridian 0) (acpi 1) (vnc 1) (nographic 0) (nomigrate 0) (usb 0) (tsc_mode 0) (guest_os_type default) (device_model /usr/lib/xen/bin/qemu-dm) (pci_power_mgmt 0) (xauthority /root/.Xauthority) (isa 0) (notes (SUSPEND_CANCEL 1)))", 'name': 'EVM_BackUp'}
> > [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1794) Storing domain details: {'console/port': '6', 'cpu/3/availability': 'online', 'description': '', 'console/limit': '1048576', 'store/port': '5', 'cpu/2/availability': 'online', 'vm': '/vm/a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'domid': '3', 'image/suspend-cancel': '1', 'cpu/0/availability': 'online', 'memory/target': '2097152', 'control/platform-feature-multiprocessor-suspend': '1', 'store/ring-ref': '1044476', 'cpu/1/availability': 'online', 'console/type': 'ioemu', 'name': 'EVM_BackUp'}
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap2.
> > [2011-05-24 16:02:08 2335] DEBUG (pciif:460) XendDomainInfo.handleAerStateWatch
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> > [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1881) XendDomainInfo.handleShutdownWatch
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vif/3/0/hotplug-status.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vkbd.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices ioports.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif2.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices console.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vscsi.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vbd.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 768.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vbd/3/768/hotplug-status.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices irq.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vfb.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices pci.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vusb.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vtpm.
> > [2011-05-24 16:09:13 2335] DEBUG (XendDomainInfo:3071) XendDomainInfo.destroy: domid=3
> >
> >
> > Thanks,
> > Wei Cao | ZZH2 2W081 | 8 821 5739 | 021 61165739 | http://flexservices.intel.com/
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: XEN 4.1 pci passthru performance regression?
2011-06-01 7:55 ` Ian Campbell
@ 2011-06-01 12:44 ` Ian Campbell
0 siblings, 0 replies; 7+ messages in thread
From: Ian Campbell @ 2011-06-01 12:44 UTC (permalink / raw)
To: Cao, Buddy; +Cc: xen-devel
On Wed, 2011-06-01 at 08:55 +0100, Ian Campbell wrote:
> On Wed, 2011-06-01 at 02:41 +0100, Cao, Buddy wrote:
> > Thanks Ian for you reply.
> >
> > >Are you referring only to control operation performance or also to
> > >actual guest performance?
> >
> > It does not impact actual guest performance, only make vm start and shutdown slow.
>
> Oh good!
>
> > >Do you find that guests without any PCI passthrough have also slowed
> > >down or is the issue purely for guests with a passed-through device?
> >
> > The guests without any PCI passthrough works as well as xen4.0.2.
>
> Also good.
>
> > I tired xl, but looks it always returns "libxl....." relevant error,
> > and none of the command can run properly
>
> Well, we should also try and resolve this -- xl is expected to work in
> 4.1. Did you see: http://wiki.xen.org/xenwiki/MigrationGuideToXen4.1+ ?
>
> Please tell us the specifics of the xl failure.
>
> I tried starting a PV guest with a passed through device with both xm
> and xl and there was no obvious delay on startup or shutdown.
> Unfortunately I don't have a test box which can do passthrough to HVM
> guests at the moment (I'll see if I can scare one up). In the meantime
> perhaps you could add some extra logging to xend to see where the time
> is spent and/or report the xl failues as requested above.
>
> Also, please can you try xen-unstable.hg to see if the problem still
> exists there.
I found a suitable box and tested xen-unstable.hg.
xm does seem to suffer from a delay of some sort on startup, but xl
doesn't seem to.
I'm afraid I don't have time to dig into xend any further myself.
Ian.
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2011-06-01 12:44 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <b9bc7c5f-5617-4273-a293-7e5b98fb61d9@orsmsx602.amr.corp.intel.com>
2011-05-30 9:08 ` XEN 4.1 pci passthru performance regression? Cao, Buddy
2011-05-31 12:14 ` Ian Campbell
2011-06-01 1:41 ` Cao, Buddy
2011-06-01 7:55 ` Ian Campbell
2011-06-01 12:44 ` Ian Campbell
2011-05-30 9:12 ` Any suggesions for "Gfx passthru" on GTS 450? Cao, Buddy
2011-05-30 15:37 ` Pasi Kärkkäinen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.