could not reboot domU in latest xen-unstable

could not reboot domU in latest xen-unstable

[Wensheng Wang]

With lastest unstable, I can no longer reboot domU( either within the
domU itself or using "xm shutdown -R").
xm create and destroy is working fine.

After I reboot a domU and do "xm list" a few times, it seems hang, but
after more than 1 minute the prompt came back but with only domain-0,
domU didn't get rebooted.
The xend.log show following errors:

----------------------------------------------------------------------
[2005-11-06 02:06:21 xend.XendDomainInfo] ERROR (XendDomainInfo:158)
Domain construction failed
Traceback (most recent call last):
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 152, in create
   vm.initDomain()
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1109, in initDomain
   self.createDevices()
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1222, in createDevices
   self.createDevice(n, c)
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 893, in createDevice
   return self.getDeviceController(deviceClass).createDevice(devconfig)
 File "/usr/lib/python/xen/xend/server/DevController.py", line 66, in
createDevice
   raise VmError( ("Device %s (%s) could not be connected. "
VmError: Device 2049 (vbd) could not be connected. Hotplug scripts not working
[2005-11-06 02:06:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:1156)
XendDomainInfo.destroy: domid=8
[2005-11-06 02:06:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:1164)
XendDomainInfo.destroyDomain(8)
[2005-11-06 02:06:22 xend.XendDomainInfo] ERROR (XendDomainInfo:1306)
Failed to restart domain 7.
Traceback (most recent call last):
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1299, in restart
   new_dom = xd.domain_create(config)
 File "/usr/lib/python/xen/xend/XendDomain.py", line 215, in domain_create
   dominfo = XendDomainInfo.create(config)
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 152, in create
   vm.initDomain()
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1109, in initDomain
   self.createDevices()
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1222, in createDevices
   self.createDevice(n, c)
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 893, in createDevice
   return self.getDeviceController(deviceClass).createDevice(devconfig)
 File "/usr/lib/python/xen/xend/server/DevController.py", line 66, in
createDevice
   raise VmError( ("Device %s (%s) could not be connected. "
VmError: Device 2049 (vbd) could not be connected. Hotplug scripts not working
[2005-11-06 02:06:22 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
 File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
   watch.fn(*watch.args, **watch.kwargs)
 File "/usr/lib/python/xen/xend/XendDomain.py", line 112, in onReleaseDomain
   self.refresh()
 File "/usr/lib/python/xen/xend/XendDomain.py", line 176, in refresh
   d.update(info)
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 849, in update
   self.refreshShutdown(info)
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 781, in refreshShutdown
   self.maybeRestart(restart_reason)
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 801, in maybeRestart
   {"destroy"        : self.destroy,
 File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1312, in restart
   new_dom.removeVm('xend/restart_in_progress')
UnboundLocalError: local variable 'new_dom' referenced before assignment
[2005-11-06 02:06:22 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
 File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
   watch.fn(*watch.args, **watch.kwargs)
 File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in
hotplugStatus
   status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
 File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in
readBackend
   return xstransact.Read(backpath, *args)
 File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
   t = cls(path)
 File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
   self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
----------------------------------------------------------------------------------

Did I do anything wrong or indeed hotplug script is broken?

Thanks

Re: could not reboot domU in latest xen-unstable

[Ewan Mellor]

On Sun, Nov 06, 2005 at 01:30:20AM -0600, Wensheng Wang wrote:

> With lastest unstable, I can no longer reboot domU( either within the
> domU itself or using "xm shutdown -R").
> xm create and destroy is working fine.
> 
> After I reboot a domU and do "xm list" a few times, it seems hang, but
> after more than 1 minute the prompt came back but with only domain-0,
> domU didn't get rebooted.

The primary problem is that your hotplug scripts seem not to be working.  Do
you see any evidence that they have stalled or bailed out early?  There might
be something in your /var/log/debug, /var/log/messages, or /var/log/syslog, or
you might see processes hanging around.

You are the only person experiencing this that I know of, so you may have to
do a little hunting around to figure out what the real problem is.

The seconday problem you have is that Xend is behaving badly when this first
error occurs, and I shall look into that.

Thanks for your bug report,

Ewan.

Re: could not reboot domU in latest xen-unstable

[Ewan Mellor]

On Sun, Nov 06, 2005 at 09:22:18AM +0000, Ewan Mellor wrote:

> On Sun, Nov 06, 2005 at 01:30:20AM -0600, Wensheng Wang wrote:
> 
> The seconday problem you have is that Xend is behaving badly when this first
> error occurs, and I shall look into that.

I've assigned this secondary problem #384.  I'll let you investigate the
primary problem a little further before filing it.

Cheers,

Ewan.

Re: could not reboot domU in latest xen-unstable

[Wensheng Wang]

[-- Attachment #1: Type: text/plain, Size: 1765 bytes --]

Hi, Ewan,
Thank for for the help.

I looked at my /var/log/message
The last line when the xend unable to restart domain is:
Nov  7 00:08:07 m8 logger: /etc/xen/scripts/block: xenstore-read
backend/vbd/2/2049/type failed.

I attached part of xend.log file,  the timeline in that log file is:
23:57:47 machine booted
00:05:21 xm create fc4
00:06:07 xm shutdown -R fc4
1minute after, I did a "xm list", it hangs, another minute later, the
prompt came back showing only domain-0


I also find if I interrupt that "xm list" after domU reboot, then do a
"xend restart", the domainU will shown paused. When I unpause it, it
will be in "r" state forever, even though in reality it doesn't exist
anymore.

My base system is fedora core 4.  gcc version is 4.01(20050727).

A side note is I copied /usr/lib/python/xen to
/usr/lib/python2.4/site-packages/ (the default python package location
on my system)  to get rid of "unable to read memory/target" warning( I
think it just warning, it doesn't seems to affecting anything).  
Though I don't think this has anything to do with the problem I have.

I hope I can provide more detail,  but I don't know whereelse to look,
 If you give me some pointer I can get more information.

thanks

Wensheng Wang



On 11/6/05, Ewan Mellor <ewan@xensource.com> wrote:
> On Sun, Nov 06, 2005 at 09:22:18AM +0000, Ewan Mellor wrote:
>
> > On Sun, Nov 06, 2005 at 01:30:20AM -0600, Wensheng Wang wrote:
> >
> > The seconday problem you have is that Xend is behaving badly when this first
> > error occurs, and I shall look into that.
>
> I've assigned this secondary problem #384.  I'll let you investigate the
> primary problem a little further before filing it.
>
> Cheers,
>
> Ewan.
>

[-- Attachment #2: xend.log --]
[-- Type: application/octet-stream, Size: 20022 bytes --]

[2005-11-06 23:57:47 xend] INFO (SrvDaemon:268) Xend Daemon started
[2005-11-06 23:57:47 xend] INFO (SrvDaemon:272) Xend changeset: Sat Nov  5 10:30:01 2005 +0100 7622:0cae0c6436f5.
[2005-11-06 23:57:47 xend.XendDomainInfo] DEBUG (XendDomainInfo:168) XendDomainInfo.recreate({'paused': 0, 'cpu_time': 14324101330L, 'ssidref': 0, 'handle': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'shutdown_reason': 0, 'dying': 0, 'dom': 0, 'mem_kb': 262144, 'maxmem_kb': -4, 'max_vcpu_id': 1, 'crashed': 0, 'running': 1, 'shutdown': 0, 'online_vcpus': 1, 'blocked': 0})
[2005-11-06 23:57:47 xend.XendDomainInfo] INFO (XendDomainInfo:180) Recreating domain 0, UUID 00000000-0000-0000-0000-000000000000.
[2005-11-06 23:57:47 xend] DEBUG (XendDomain:140) number of vcpus to use is 0
[2005-11-06 23:57:47 xend] INFO (SrvServer:112) unix path=/var/lib/xend/xend-socket
[2005-11-07 00:05:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:147) XendDomainInfo.create(['vm', ['name', 'fc4'], ['memory', '128'], ['restart', 'onreboot'], ['image', ['linux', ['kernel', '/boot/vmlinuz-2.6.12.6-xenU'], ['ip', '192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off'], ['root', '/dev/sda1 ro'], ['vcpus', '2'], ['vcpus', '2'], ['boot', 'c']]], ['device', ['vbd', ['uname', 'file:/root/fc4img'], ['dev', 'sda1'], ['mode', 'w']]], ['device', ['vif', ['mac', 'aa:00:00:0f:9b:11']]]])
[2005-11-07 00:05:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:250) parseConfig: config is ['vm', ['name', 'fc4'], ['memory', '128'], ['restart', 'onreboot'], ['image', ['linux', ['kernel', '/boot/vmlinuz-2.6.12.6-xenU'], ['ip', '192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off'], ['root', '/dev/sda1 ro'], ['vcpus', '2'], ['vcpus', '2'], ['boot', 'c']]], ['device', ['vbd', ['uname', 'file:/root/fc4img'], ['dev', 'sda1'], ['mode', 'w']]], ['device', ['vif', ['mac', 'aa:00:00:0f:9b:11']]]]
[2005-11-07 00:05:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:308) parseConfig: result is {'ssidref': None, 'name': 'fc4', 'on_reboot': 'restart', 'maxmem': None, 'on_poweroff': 'destroy', 'mem_kb': None, 'maxmem_kb': None, 'on_crash': 'destroy', 'backend': [], 'vcpus': 2, 'cpu_weight': None, 'vcpu_avail': None, 'memory': 128, 'device': [('vbd', ['vbd', ['uname', 'file:/root/fc4img'], ['dev', 'sda1'], ['mode', 'w']]), ('vif', ['vif', ['mac', 'aa:00:00:0f:9b:11']])], 'bootloader': None, 'image': ['linux', ['kernel', '/boot/vmlinuz-2.6.12.6-xenU'], ['ip', '192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off'], ['root', '/dev/sda1 ro'], ['vcpus', '2'], ['vcpus', '2'], ['boot', 'c']], 'cpu': None, 'uuid': None}
[2005-11-07 00:05:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:1047) XendDomainInfo.construct: None 0
[2005-11-07 00:05:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:1077) XendDomainInfo.initDomain: 1 131072 1.0
[2005-11-07 00:05:21 xend] INFO (image:131) buildDomain os=linux dom=1 vcpus=2
[2005-11-07 00:05:21 xend] DEBUG (image:169) dom            = 1
[2005-11-07 00:05:21 xend] DEBUG (image:170) image          = /boot/vmlinuz-2.6.12.6-xenU
[2005-11-07 00:05:21 xend] DEBUG (image:171) store_evtchn   = 1
[2005-11-07 00:05:21 xend] DEBUG (image:172) console_evtchn = 2
[2005-11-07 00:05:21 xend] DEBUG (image:173) cmdline        =  ip=192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off root=/dev/sda1 ro
[2005-11-07 00:05:21 xend] DEBUG (image:174) ramdisk        = 
[2005-11-07 00:05:21 xend] DEBUG (image:175) vcpus          = 2
[2005-11-07 00:05:21 xend] DEBUG (DevController:256) DevController: writing {'virtual-device': '2049', 'backend-id': '0', 'backend': '/local/domain/0/backend/vbd/1/2049'} to /local/domain/1/device/vbd/2049.
[2005-11-07 00:05:21 xend] DEBUG (DevController:258) DevController: writing {'domain': 'fc4', 'params': '/root/fc4img', 'frontend-id': '1', 'type': 'file', 'dev': 'sda1', 'frontend': '/local/domain/1/device/vbd/2049'} to /local/domain/0/backend/vbd/1/2049.
[2005-11-07 00:05:21 xend] DEBUG (DevController:256) DevController: writing {'backend-id': '0', 'mac': 'aa:00:00:0f:9b:11', 'handle': '0', 'backend': '/local/domain/0/backend/vif/1/0'} to /local/domain/1/device/vif/0.
[2005-11-07 00:05:21 xend] DEBUG (DevController:258) DevController: writing {'mac': 'aa:00:00:0f:9b:11', 'handle': '0', 'script': '/etc/xen/scripts/vif-bridge', 'frontend-id': '1', 'domain': 'fc4', 'frontend': '/local/domain/1/device/vif/0'} to /local/domain/0/backend/vif/1/0.
[2005-11-07 00:05:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:589) Storing VM details: {'ssidref': '0', 'uuid': '4bf4ead7-03fd-3f21-2faa-d0711c8533d0', 'on_reboot': 'restart', 'image': "(linux (kernel /boot/vmlinuz-2.6.12.6-xenU) (ip 192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off) (root '/dev/sda1 ro') (vcpus 2) (vcpus 2) (boot c))", 'on_poweroff': 'destroy', 'vcpus': '2', 'on_crash': 'destroy', 'memory/target': '131072', 'vcpu_avail': '3', 'name': 'fc4'}
[2005-11-07 00:05:21 xend.XendDomainInfo] DEBUG (XendDomainInfo:614) Storing domain details: {'console/ring-ref': '799102', 'console/port': '2', 'name': 'fc4', 'console/limit': '1048576', 'vm': '/vm/4bf4ead7-03fd-3f21-2faa-d0711c8533d0', 'domid': '1', 'cpu/0/availability': 'online', 'memory/target': '131072', 'store/ring-ref': '799103', 'cpu/1/availability': 'online', 'store/port': '1'}
[2005-11-07 00:05:21 xend] INFO (XendDomain:349) Domain fc4 (1) unpaused.
[2005-11-07 00:06:07 xend.XendDomainInfo] INFO (XendDomainInfo:741) Domain has shutdown: name=fc4 id=1 reason=reboot.
[2005-11-07 00:06:07 xend.XendDomainInfo] DEBUG (XendDomainInfo:1164) XendDomainInfo.destroyDomain(1)
[2005-11-07 00:06:07 xend.XendDomainInfo] DEBUG (XendDomainInfo:147) XendDomainInfo.create(['domain', ['domid', 1], ['memory', 127], ['name', 'fc4'], ['uuid', '4bf4ead7-03fd-3f21-2faa-d0711c8533d0'], ['ssidref', 0], ['vcpus', 2], ['vcpu_avail', 3], ['cpu_weight', 1.0], ['on_poweroff', 'destroy'], ['on_reboot', 'restart'], ['on_crash', 'destroy'], ['maxmem', 127], ['image', ['linux', ['kernel', '/boot/vmlinuz-2.6.12.6-xenU'], ['ip', '192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off'], ['root', '/dev/sda1 ro'], ['vcpus', '2'], ['vcpus', '2'], ['boot', 'c']]], ['device', ['vbd', ['uname', 'file:/root/fc4img'], ['dev', 'sda1'], ['mode', 'w']]], ['device', ['vif', ['mac', 'aa:00:00:0f:9b:11']]], ['state', '---s--'], ['shutdown_reason', 'reboot'], ['cpu_time', 11.624615014], ['vcpus', 2], ['up_time', '45.1864478588'], ['start_time', '1131343521.92'], ['store_mfn', 799103], ['console_m
 fn', 799102]])
[2005-11-07 00:06:07 xend.XendDomainInfo] DEBUG (XendDomainInfo:250) parseConfig: config is ['domain', ['domid', 1], ['memory', 127], ['name', 'fc4'], ['uuid', '4bf4ead7-03fd-3f21-2faa-d0711c8533d0'], ['ssidref', 0], ['vcpus', 2], ['vcpu_avail', 3], ['cpu_weight', 1.0], ['on_poweroff', 'destroy'], ['on_reboot', 'restart'], ['on_crash', 'destroy'], ['maxmem', 127], ['image', ['linux', ['kernel', '/boot/vmlinuz-2.6.12.6-xenU'], ['ip', '192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off'], ['root', '/dev/sda1 ro'], ['vcpus', '2'], ['vcpus', '2'], ['boot', 'c']]], ['device', ['vbd', ['uname', 'file:/root/fc4img'], ['dev', 'sda1'], ['mode', 'w']]], ['device', ['vif', ['mac', 'aa:00:00:0f:9b:11']]], ['state', '---s--'], ['shutdown_reason', 'reboot'], ['cpu_time', 11.624615014], ['vcpus', 2], ['up_time', '45.1864478588'], ['start_time', '1131343521.92'], ['store_mfn', 799103], ['console_
 mfn', 799102]]
[2005-11-07 00:06:07 xend.XendDomainInfo] DEBUG (XendDomainInfo:308) parseConfig: result is {'ssidref': 0, 'name': 'fc4', 'on_reboot': 'restart', 'maxmem': 127, 'on_poweroff': 'destroy', 'mem_kb': None, 'maxmem_kb': None, 'on_crash': 'destroy', 'backend': [], 'vcpus': 2, 'cpu_weight': 1.0, 'vcpu_avail': 3, 'memory': 127, 'device': [('vbd', ['vbd', ['uname', 'file:/root/fc4img'], ['dev', 'sda1'], ['mode', 'w']]), ('vif', ['vif', ['mac', 'aa:00:00:0f:9b:11']])], 'bootloader': None, 'image': ['linux', ['kernel', '/boot/vmlinuz-2.6.12.6-xenU'], ['ip', '192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off'], ['root', '/dev/sda1 ro'], ['vcpus', '2'], ['vcpus', '2'], ['boot', 'c']], 'cpu': None, 'uuid': '4bf4ead7-03fd-3f21-2faa-d0711c8533d0'}
[2005-11-07 00:06:07 xend.XendDomainInfo] DEBUG (XendDomainInfo:1047) XendDomainInfo.construct: None 0
[2005-11-07 00:06:07 xend.XendDomainInfo] DEBUG (XendDomainInfo:1077) XendDomainInfo.initDomain: 2 130048 1.0
[2005-11-07 00:06:07 xend] INFO (image:131) buildDomain os=linux dom=2 vcpus=2
[2005-11-07 00:06:07 xend] DEBUG (image:169) dom            = 2
[2005-11-07 00:06:07 xend] DEBUG (image:170) image          = /boot/vmlinuz-2.6.12.6-xenU
[2005-11-07 00:06:07 xend] DEBUG (image:171) store_evtchn   = 1
[2005-11-07 00:06:07 xend] DEBUG (image:172) console_evtchn = 2
[2005-11-07 00:06:07 xend] DEBUG (image:173) cmdline        =  ip=192.168.1.221:1.2.3.4:192.168.1.1:255.255.255.0::eth0:off root=/dev/sda1 ro
[2005-11-07 00:06:07 xend] DEBUG (image:174) ramdisk        = 
[2005-11-07 00:06:07 xend] DEBUG (image:175) vcpus          = 2
[2005-11-07 00:06:07 xend] DEBUG (DevController:256) DevController: writing {'virtual-device': '2049', 'backend-id': '0', 'backend': '/local/domain/0/backend/vbd/2/2049'} to /local/domain/2/device/vbd/2049.
[2005-11-07 00:06:07 xend] DEBUG (DevController:258) DevController: writing {'domain': 'fc4', 'params': '/root/fc4img', 'frontend-id': '2', 'type': 'file', 'dev': 'sda1', 'frontend': '/local/domain/2/device/vbd/2049'} to /local/domain/0/backend/vbd/2/2049.
[2005-11-07 00:08:07 xend.XendDomainInfo] ERROR (XendDomainInfo:158) Domain construction failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 152, in create
    vm.initDomain()
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1109, in initDomain
    self.createDevices()
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1222, in createDevices
    self.createDevice(n, c)
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 893, in createDevice
    return self.getDeviceController(deviceClass).createDevice(devconfig)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 66, in createDevice
    raise VmError( ("Device %s (%s) could not be connected. "
VmError: Device 2049 (vbd) could not be connected. Hotplug scripts not working
[2005-11-07 00:08:07 xend.XendDomainInfo] DEBUG (XendDomainInfo:1156) XendDomainInfo.destroy: domid=2
[2005-11-07 00:08:07 xend.XendDomainInfo] DEBUG (XendDomainInfo:1164) XendDomainInfo.destroyDomain(2)
[2005-11-07 00:08:07 xend.XendDomainInfo] ERROR (XendDomainInfo:1306) Failed to restart domain 1.
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1299, in restart
    new_dom = xd.domain_create(config)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 215, in domain_create
    dominfo = XendDomainInfo.create(config)
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 152, in create
    vm.initDomain()
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1109, in initDomain
    self.createDevices()
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1222, in createDevices
    self.createDevice(n, c)
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 893, in createDevice
    return self.getDeviceController(deviceClass).createDevice(devconfig)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 66, in createDevice
    raise VmError( ("Device %s (%s) could not be connected. "
VmError: Device 2049 (vbd) could not be connected. Hotplug scripts not working
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 112, in onReleaseDomain
    self.refresh()
  File "/usr/lib/python/xen/xend/XendDomain.py", line 176, in refresh
    d.update(info)
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 849, in update
    self.refreshShutdown(info)
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 781, in refreshShutdown
    self.maybeRestart(restart_reason)
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 801, in maybeRestart
    {"destroy"        : self.destroy,
  File "/usr/lib/python/xen/xend/XendDomainInfo.py", line 1312, in restart
    new_dom.removeVm('xend/restart_in_progress')
UnboundLocalError: local variable 'new_dom' referenced before assignment
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'
[2005-11-07 00:08:07 xend] ERROR (xswatch:50) read_watch failed
Traceback (most recent call last):
  File "/usr/lib/python/xen/xend/xenstore/xswatch.py", line 48, in watchMain
    watch.fn(*watch.args, **watch.kwargs)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 267, in hotplugStatus
    status = self.readBackend(devid, HOTPLUG_STATUS_NODE)
  File "/usr/lib/python/xen/xend/server/DevController.py", line 201, in readBackend
    return xstransact.Read(backpath, *args)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 225, in Read
    t = cls(path)
  File "/usr/lib/python/xen/xend/xenstore/xstransact.py", line 17, in __init__
    self.path = path.rstrip("/")
AttributeError: 'NoneType' object has no attribute 'rstrip'















[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: could not reboot domU in latest xen-unstable

[Toens Bueker]

Ewan Mellor <ewan@xensource.com> wrote:

>> With lastest unstable, I can no longer reboot domU( either within the
>> domU itself or using "xm shutdown -R").
>> xm create and destroy is working fine.
>> 
>> After I reboot a domU and do "xm list" a few times, it seems hang, but
>> after more than 1 minute the prompt came back but with only domain-0,
>> domU didn't get rebooted.

[...]

> You are the only person experiencing this that I know of, so you may have to
> do a little hunting around to figure out what the real problem is.

<aol>
Me too.
</aol>

system             : Linux
host               : gretel
release            : 2.6.12.6-xen0
version            : #3 SMP Fri Nov 4 21:17:01 CET 2005
machine            : i686
nr_cpus            : 2
nr_nodes           : 1
sockets_per_node   : 1
cores_per_socket   : 1
threads_per_core   : 2
cpu_mhz            : 2665
hw_caps            : bfebfbff:00000000:00000000:00000080:00004400
memory             : 2047
free_memory        : 1763
xen_major          : 3
xen_minor          : 0
xen_extra          : -devel
xen_caps           : xen_3.0_x86_32
xen_params         : virt_start=0xfc000000
xen_changeset      : Thu Nov  3 09:58:45 2005 +0100 7611:289b73a5ace7
cc_compiler        : gcc-Version 3.3.5 (Debian 1:3.3.5-13)
cc_compile_by      : toens
cc_compile_domain  : domain.de
cc_compile_date    : Do Nov  3 18:49:02 CET 2005


Package: hotplug
Status: install ok installed
Priority: standard
Section: admin
Installed-Size: 380
Maintainer: Fumitoshi UKAI <ukai@debian.or.jp>
Architecture: all
Version: 0.0.20040329-25

by
Töns
-- 
There is no safe distance.

Re: could not reboot domU in latest xen-unstable

[Ewan Mellor]

On Mon, Nov 07, 2005 at 12:34:20AM -0600, Wensheng Wang wrote:

> Hi, Ewan,
> Thank for for the help.
> 
> I looked at my /var/log/message
> The last line when the xend unable to restart domain is:
> Nov  7 00:08:07 m8 logger: /etc/xen/scripts/block: xenstore-read
> backend/vbd/2/2049/type failed.
> 
> I attached part of xend.log file,  the timeline in that log file is:
> 23:57:47 machine booted
> 00:05:21 xm create fc4
> 00:06:07 xm shutdown -R fc4
> 1minute after, I did a "xm list", it hangs, another minute later, the
> prompt came back showing only domain-0

This was assigned bug #387, and is now fixed, as of changeset
7651:7e3faea076ec.

Cheers,

Ewan.

Re: could not reboot domU in latest xen-unstable

[Wensheng Wang]

That works, thank you for fixing it.

Today I noticed another case of this when I tried to create a domU:
Device 0 (vif) could not be connected. Hotplug scripts not working

I got it when I don't specify vif in my domU config file or when I
specify but incorrectly had a xen-br0(should be xenbr0) in vif line.
If I correctly specify vif,  that errer goes away.


Wensheng Wang


On 11/8/05, Ewan Mellor <ewan@xensource.com> wrote:
> On Mon, Nov 07, 2005 at 12:34:20AM -0600, Wensheng Wang wrote:
>
> > Hi, Ewan,
> > Thank for for the help.
> >
> > I looked at my /var/log/message
> > The last line when the xend unable to restart domain is:
> > Nov  7 00:08:07 m8 logger: /etc/xen/scripts/block: xenstore-read
> > backend/vbd/2/2049/type failed.
> >
> > I attached part of xend.log file,  the timeline in that log file is:
> > 23:57:47 machine booted
> > 00:05:21 xm create fc4
> > 00:06:07 xm shutdown -R fc4
> > 1minute after, I did a "xm list", it hangs, another minute later, the
> > prompt came back showing only domain-0
>
> This was assigned bug #387, and is now fixed, as of changeset
> 7651:7e3faea076ec.
>
> Cheers,
>
> Ewan.
>