RE: XEN 4.1 pci passthru performance regression?

[Ian Campbell <Ian.Campbell@eu.citrix.com>]

On Wed, 2011-06-01 at 02:41 +0100, Cao, Buddy wrote:
> Thanks Ian for you reply.
> 
> >Are you referring only to control operation performance or also to
> >actual guest performance?
> 
> It does not impact actual guest performance, only make vm start and shutdown slow.

Oh good!

> >Do you find that guests without any PCI passthrough have also slowed
> >down or is the issue purely for guests with a passed-through device?
> 
> The guests without any PCI passthrough works as well as xen4.0.2.

Also good.

> I tired xl, but looks it always returns "libxl....." relevant error,
> and none of the command can run properly

Well, we should also try and resolve this -- xl is expected to work in
4.1. Did you see: http://wiki.xen.org/xenwiki/MigrationGuideToXen4.1+ ?

Please tell us the specifics of the xl failure.

I tried starting a PV guest with a passed through device with both xm
and xl and there was no obvious delay on startup or shutdown.
Unfortunately I don't have a test box which can do passthrough to HVM
guests at the moment (I'll see if I can scare one up). In the meantime
perhaps you could add some extra logging to xend to see where the time
is spent and/or report the xl failues as requested above.

Also, please can you try xen-unstable.hg to see if the problem still
exists there.

Thanks,
Ian.

> 
> Thanks,
> Wei Cao | ZZH2 2W081 | 8 821 5739 | 021 61165739 | http://flexservices.intel.com/
> 
> 
> -----Original Message-----
> From: Ian Campbell [mailto:Ian.Campbell@citrix.com]
> Sent: Tuesday, May 31, 2011 8:15 PM
> To: Cao, Buddy
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] XEN 4.1 pci passthru performance regression?
> 
> On Mon, 2011-05-30 at 10:08 +0100, Cao, Buddy wrote:
> > I migrate Xen4.0 to XEN4.1, and found XEN4.1 performance is not as
> > good as 4.0.2. xm shutdown takes longer time, control thru libvirt
> > take longer time, and since I'm testing PCI/VGA passthru, the
> > "createdevice: pci" also take around 5~6 seconds comparing to 1-
> > second in XEN4.0.2.
> 
> Are you referring only to control operation performance or also to
> actual guest performance?
> 
> Do you find that guests without any PCI passthrough have also slowed
> down or is the issue purely for guests with a passed-through device?
> 
> >  below is my configuration, do you have any suggestions to make the
> > performance better on XEN4.1?
> 
> I'm afraid the only suggestion I can think of is to dig in and begin
> instrumenting the toolstack in order to figure out where the time is
> going.
> 
> I suppose it might be worth trying xl on 4.1 to see if it seems slow.
> I'm not sure if xl in 4.0 will be functional enough to also provide a
> comparison, although I guess you could try.
> 
> Ian.
> 
> >
> > xend-config.sxp:
> >  (xend-unix-server yes)
> > (xend-http-server yes)
> > (xend-relocation-server yes)
> > (xend-port 8000)
> > (xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
> > (network-script network-bridge)
> > (vif-script vif-bridge)
> > (dom0-min-mem 768)
> > (enable-dom0-ballooning yes)
> > (total_available_memory 0)
> > (dom0-cpus 0)
> > (vnc-listen '0.0.0.0')
> > (vncpasswd '')
> >
> >
> >
> > vm1.cfg:
> >  import os, re
> > arch = os.uname()[4]
> > name = "b"
> > maxmem = 2048
> > memory = 2048
> > vcpus = 4
> > builder = "hvm"
> > kernel = "/usr/lib/xen/boot/hvmloader"
> > boot = "c"
> > pae = 0
> > acpi = 1
> > apic = 1
> > timer_mode =1
> > shadow_memory = 32
> > on_poweroff = "destroy"
> > on_reboot = "restart"
> > on_crash = "restart"
> > device_model = "/usr/lib/xen/bin/qemu-dm"
> > vnc = 1
> > disk = [ "file:/opt/EVM/domain/a_disk.img,hda,w" ]
> > vif = [ "type=ioemu" ]
> > gfx_passthru=1
> > pci= ['00:02.0','00:1a.0','00:1d.0','00:1b.0' , '00:1f.0']
> >
> >
> >
> > xend.log
> > [2011-05-24 16:01:59 2335] DEBUG (SrvDomain:76) Starting domain EVM_BackUp False
> > [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2498) XendDomainInfo.constructDomain
> > [2011-05-24 16:01:59 2335] DEBUG (balloon:187) Balloon: 2150848 KiB free; need 16384; done.
> > [2011-05-24 16:01:59 2335] DEBUG (XendDomain:476) Adding Domain: 3
> > [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2836) XendDomainInfo.initDomain: 3 256
> > [2011-05-24 16:01:59 2335] DEBUG (image:339) No VNC passwd configured for vfb access
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: boot, val: c
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: fda, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: fdb, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: soundhw, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: localtime, val: 0
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: serial, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: std-vga, val: 0
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: isa, val: 0
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: acpi, val: 1
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: usb, val: 0
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: usbdevice, val: None
> > [2011-05-24 16:01:59 2335] DEBUG (image:891) args: gfx_passthru, val: 1
> > [2011-05-24 16:01:59 2335] INFO (image:822) Need to create platform device.[domid:3]
> > [2011-05-24 16:01:59 2335] DEBUG (XendDomainInfo:2863) _initDomain:shadow_memory=0x20, memory_static_max=0x80000000, memory_static_min=0x0.
> > [2011-05-24 16:01:59 2335] INFO (image:182) buildDomain os=hvm dom=3 vcpus=4
> > [2011-05-24 16:01:59 2335] DEBUG (image:949) domid          = 3
> > [2011-05-24 16:01:59 2335] DEBUG (image:950) image          = /usr/lib/xen/boot/hvmloader
> > [2011-05-24 16:01:59 2335] DEBUG (image:951) store_evtchn   = 5
> > [2011-05-24 16:01:59 2335] DEBUG (image:952) memsize        = 2048
> > [2011-05-24 16:01:59 2335] DEBUG (image:953) target         = 2048
> > [2011-05-24 16:01:59 2335] DEBUG (image:954) vcpus          = 4
> > [2011-05-24 16:01:59 2335] DEBUG (image:955) vcpu_avail     = 15
> > [2011-05-24 16:01:59 2335] DEBUG (image:956) acpi           = 1
> > [2011-05-24 16:01:59 2335] DEBUG (image:957) apic           = 1
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: console : {'devid': 0, 'protocol': 'vt100', 'location': '6', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50'}
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/console/3/0'} to /local/domain/3/device/console/0.
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/console/0', 'uuid': 'c8266944-6af7-3eda-d4d7-c3ada2e64e50', 'frontend-id': '3', 'state': '1', 'location': '6', 'online': '1', 'protocol': 'vt100'} to /local/domain/0/backend/console/3/0.
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vfb : {'vncunused': '1', 'other_config': {'vncunused': '1', 'vnc': '1'}, 'vnc': '1', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'devid': 0, 'location': '0.0.0.0:5900'}
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vfb/3/0'} to /local/domain/3/device/vfb/0.
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'vncunused': '1', 'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vfb/0', 'uuid': '352c58cb-0c8c-22b7-83fb-ff61d7ff469f', 'frontend-id': '3', 'state': '1', 'location': '0.0.0.0:5900', 'online': '1', 'vnc': '1'} to /local/domain/0/backend/vfb/3/0.
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vbd : {'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': 1, 'devid': 768, 'driver': 'paravirtualised', 'dev': 'hda:disk', 'uname': 'file:/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'VDI': '', 'backend': '0'}
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'backend-id': '0', 'virtual-device': '768', 'device-type': 'disk', 'state': '1', 'backend': '/local/domain/0/backend/vbd/3/768'} to /local/domain/3/device/vbd/768.
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vbd/768', 'uuid': 'd360eafb-1349-a3e4-0270-6c529fbf57fc', 'bootable': '1', 'dev': 'hda', 'state': '1', 'params': '/opt/EVM/domain/EVM_BackUp_disk.img', 'mode': 'w', 'online': '1', 'frontend-id': '3', 'type': 'file'} to /local/domain/0/backend/vbd/3/768.
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: vif : {'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'devid': 0, 'mac': '00:16:3e:01:a9:c0', 'type': 'ioemu', 'backend': '0'}
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/vif/3/0'} to /local/domain/3/device/vif/0.
> > [2011-05-24 16:01:59 2335] DEBUG (DevController:97) DevController: writing {'domain': 'EVM_BackUp', 'frontend': '/local/domain/3/device/vif/0', 'uuid': '3c9ae273-99e5-051a-5113-5bc559750b97', 'script': '/etc/xen/scripts/vif-bridge', 'mac': '00:16:3e:01:a9:c0', 'frontend-id': '3', 'state': '1', 'online': '1', 'handle': '0', 'type': 'ioemu'} to /local/domain/0/backend/vif/3/0.
> > [2011-05-24 16:01:59 2335] INFO (XendDomainInfo:2357) createDevice: pci : {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}, {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}, {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}, {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}, {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803'}
> > [2011-05-24 16:02:04 2335] INFO (image:418) spawning device models: /usr/lib/xen/bin/qemu-dm ['/usr/lib/xen/bin/qemu-dm', '-d', '3', '-domain-name', 'EVM_BackUp', '-videoram', '4', '-vnc', '0.0.0.0:0', '-vncunused', '-vcpus', '4', '-vcpu_avail', '0xfL', '-boot', 'c', '-acpi', '-gfx_passthru', '1', '-net', 'nic,vlan=1,macaddr=00:16:3e:01:a9:c0,model=rtl8139', '-net', 'tap,vlan=1,ifname=tap3.0,bridge=xenbr0', '-M', 'xenfv']
> > [2011-05-24 16:02:04 2335] INFO (image:467) device model pid: 4092
> > [2011-05-24 16:02:04 2335] INFO (image:590) waiting for sentinel_fifo
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x02'], ['domain', '0x0000'], ['key', '00:02.0'], ['bus', '0x00'], ['vdevfn', '0x10'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '16d91793-5fe2-e409-9a8c-1bae56a4ff48']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x02', 'domain': '0x0000', 'key': '00:02.0', 'bus': '0x00', 'vdevfn': '0x10', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '16d91793-5fe2-e409-9a8c-1bae56a4ff48'}
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:02.0@10,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:04 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:02.0@10,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:04 2335] DEBUG (image:508) signalDeviceModel: orig_state is None, retrying
> > [2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 23
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 22
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:456) pci: register aer watch /local/domain/0/backend/pci/3/0/aerState
> > [2011-05-24 16:02:05 2335] DEBUG (DevController:95) DevController: writing {'state': '1', 'backend-id': '0', 'backend': '/local/domain/0/backend/pci/3/0'} to /local/domain/3/device/pci/0.
> > [2011-05-24 16:02:05 2335] DEBUG (DevController:97) DevController: writing {'key-3': '00:1b.0', 'key-2': '00:1d.0', 'key-1': '00:1a.0', 'key-0': '00:02.0', 'frontend': '/local/domain/3/device/pci/0', 'key-4': '00:1f.0', 'domain': 'EVM_BackUp', 'uuid-4': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93', 'uuid-0': '16d91793-5fe2-e409-9a8c-1bae56a4ff48', 'uuid-1': '20820900-e0aa-0fa8-00a5-07a5c80ff58d', 'uuid-2': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea', 'uuid-3': 'e693ed56-4bb1-244a-32c2-2b1467da9788', 'uuid': '3dacc5c2-f64f-98fa-67eb-8cf98e226803', 'state': '1', 'online': '1', 'opts-4': 'msitranslate=1,power_mgmt=0', 'opts-3': 'msitranslate=1,power_mgmt=0', 'opts-2': 'msitranslate=1,power_mgmt=0', 'opts-1': 'msitranslate=1,power_mgmt=0', 'opts-0': 'msitranslate=1,power_mgmt=0', 'dev-3': '0000:00:1b.0', 'dev-2': '0000:00:1d.0', 'dev-1': '0000:00:1a.0', 'dev-0': '0000:00:02.0', 'dev-4': '0000:00:1f.0', 'vdevfn-4': '40', 'vdevfn-0': '10', 'vdevfn-1': '28', 'vdevfn-2': '30', 'vdevfn-3': '38', 'frontend-id': '3', 'num_devs': '5'} to /local/domain/0/backend/pci/3/0.
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:02.0.
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:320) pci: enabling ioport 0xf160/0x8
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe000000/0x400000 pfn 0xfe000/0x400
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xd0000000/0x10000000 pfn 0xd0000/0x10000
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1a'], ['domain', '0x0000'], ['key', '00:1a.0'], ['bus', '0x00'], ['vdevfn', '0x28'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', '20820900-e0aa-0fa8-00a5-07a5c80ff58d']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1a', 'domain': '0x0000', 'key': '00:1a.0', 'bus': '0x00', 'vdevfn': '0x28', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': '20820900-e0aa-0fa8-00a5-07a5c80ff58d'}
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:05 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1a.0@28,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:05 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1a.0.
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe527000/0x400 pfn 0xfe527/0x1
> > [2011-05-24 16:02:05 2335] DEBUG (pciif:351) pci: enabling irq 16
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1d'], ['domain', '0x0000'], ['key', '00:1d.0'], ['bus', '0x00'], ['vdevfn', '0x30'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1d', 'domain': '0x0000', 'key': '00:1d.0', 'bus': '0x00', 'vdevfn': '0x30', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'f3a4fdef-26d6-8e5a-3ea4-58512ad32cea'}
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:06 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1d.0@30,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:06 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:06 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1d.0.
> > [2011-05-24 16:02:06 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe526000/0x400 pfn 0xfe526/0x1
> > [2011-05-24 16:02:06 2335] DEBUG (pciif:351) pci: enabling irq 23
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1b'], ['domain', '0x0000'], ['key', '00:1b.0'], ['bus', '0x00'], ['vdevfn', '0x38'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'e693ed56-4bb1-244a-32c2-2b1467da9788']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1b', 'domain': '0x0000', 'key': '00:1b.0', 'bus': '0x00', 'vdevfn': '0x38', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'e693ed56-4bb1-244a-32c2-2b1467da9788'}
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1b.0@38,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:07 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:07 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1b.0.
> > [2011-05-24 16:02:07 2335] DEBUG (pciif:334) pci: enabling iomem 0xfe520000/0x4000 pfn 0xfe520/0x4
> > [2011-05-24 16:02:07 2335] DEBUG (pciif:351) pci: enabling irq 22
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:893) XendDomainInfo.pci_device_configure: ['pci', ['dev', ['slot', '0x1f'], ['domain', '0x0000'], ['key', '00:1f.0'], ['bus', '0x00'], ['vdevfn', '0x40'], ['func', '0x0'], ['opts', ['msitranslate', '1']], ['opts', ['power_mgmt', '0']], ['uuid', 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93']], ['state', 'Initialising'], ['sub_state', 'Booting']]
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:779) XendDomainInfo.hvm_pci_device_insert: {'devs': [{'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}], 'states': ['Initialising']}
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:790) XendDomainInfo.hvm_pci_device_insert_dev: {'slot': '0x1f', 'domain': '0x0000', 'key': '00:1f.0', 'bus': '0x00', 'vdevfn': '0x40', 'func': '0x0', 'opts': [['msitranslate', '1'], ['power_mgmt', '0']], 'uuid': 'ee1d3ed1-c7b0-0c7d-2391-6fbd7e72ef93'}
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:811) XendDomainInfo.hvm_pci_device_insert_dev: 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:07 2335] DEBUG (XendDomainInfo:815) pci: assign device 0000:00:1f.0@40,msitranslate=1,power_mgmt=0
> > [2011-05-24 16:02:08 2335] INFO (image:538) signalDeviceModel:restore dm state to running
> > [2011-05-24 16:02:08 2335] DEBUG (pciif:169) Reconfiguring PCI device 0000:00:1f.0.
> > [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:3420) Storing VM details: {'on_xend_stop': 'ignore', 'pool_name': 'Pool-0', 'shadow_memory': '32', 'uuid': 'a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'on_reboot': 'restart', 'start_time': '1306281728.59', 'on_poweroff': 'destroy', 'bootloader_args': '', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count': '0', 'vcpus': '4', 'vcpu_avail': '15', 'bootloader': '', 'image': "(hvm (kernel '') (superpages 0) (videoram 4) (hpet 0) (stdvga 0) (loader /usr/lib/xen/boot/hvmloader) (xen_platform_pci 1) (rtc_timeoffset 0) (pci ((0x0000 0x00 0x02 0x0 0x10 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1a 0x0 0x28 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1d 0x0 0x30 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1b 0x0 0x38 ((msitranslate 1) (power_mgmt 0))) (0x0000 0x00 0x1f 0x0 0x40 ((msitranslate 1) (power_mgmt 0))))) (hap 1) (localtime 0) (timer_mode 1) (pci_msitranslate 1) (oos 1) (apic 1) (vpt_align 1) (vncunused 1) (gfx_passthru 1) (boot c) (pae 0) (viridian 0) (acpi 1) (vnc 1) (nographic 0) (nomigrate 0) (usb 0) (tsc_mode 0) (guest_os_type default) (device_model /usr/lib/xen/bin/qemu-dm) (pci_power_mgmt 0) (xauthority /root/.Xauthority) (isa 0) (notes (SUSPEND_CANCEL 1)))", 'name': 'EVM_BackUp'}
> > [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1794) Storing domain details: {'console/port': '6', 'cpu/3/availability': 'online', 'description': '', 'console/limit': '1048576', 'store/port': '5', 'cpu/2/availability': 'online', 'vm': '/vm/a6b7d2aa-574e-10b0-3b0a-23f2b3cd4d27', 'domid': '3', 'image/suspend-cancel': '1', 'cpu/0/availability': 'online', 'memory/target': '2097152', 'control/platform-feature-multiprocessor-suspend': '1', 'store/ring-ref': '1044476', 'cpu/1/availability': 'online', 'console/type': 'ioemu', 'name': 'EVM_BackUp'}
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap2.
> > [2011-05-24 16:02:08 2335] DEBUG (pciif:460) XendDomainInfo.handleAerStateWatch
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> > [2011-05-24 16:02:08 2335] DEBUG (XendDomainInfo:1881) XendDomainInfo.handleShutdownWatch
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vif/3/0/hotplug-status.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vkbd.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices ioports.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices tap.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vif2.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices console.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vscsi.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vbd.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 768.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:628) hotplugStatusCallback /local/domain/0/backend/vbd/3/768/hotplug-status.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:642) hotplugStatusCallback 1.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices irq.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vfb.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices pci.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:144) Waiting for 0.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vusb.
> > [2011-05-24 16:02:08 2335] DEBUG (DevController:139) Waiting for devices vtpm.
> > [2011-05-24 16:09:13 2335] DEBUG (XendDomainInfo:3071) XendDomainInfo.destroy: domid=3
> >
> >
> > Thanks,
> > Wei Cao | ZZH2 2W081 | 8 821 5739 | 021 61165739 | http://flexservices.intel.com/
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
>