Problem starting xend in latest 3.4 release candidate

Problem starting xend in latest 3.4 release candidate

[Tim Post]

Hi,

Several days ago, I upgraded to the latest 3.4 release candidate.
Everything went very well, no problems whatsoever. I re-booted without
issue.

Several days later (today), I can not get Xend to start. There have been
no changes in Python, no configuration changes, no OS updates ..
nothing.

This is the output in xend.log is below. Can someone please shed some
light on what the cause might be?

I'm using Debian's 2.6.26 kernel on x86_32, Python 2.5.4. Xenstored is
started, I can do a xenstore-ls and see dom-0's event channel, but
nothing else since Xend has not set it up.

I'm really, really bewildered. Any help is appreciated. I can't tell if
this is due to upgrading, or if some other strange problem has appeared
on my system. My Python is not the best :)

Last night, it worked. Today, it doesn't .. and I can't think of
anything that happened overnight , especially since the system was shut
down.

Thanks in advance.
--Tim

[2009-05-05 11:32:16 5541] INFO (SrvDaemon:332) Xend Daemon started
[2009-05-05 11:32:16 5541] INFO (SrvDaemon:336) Xend changeset: Mon May
04 22:52:18 2009 +0100 19591:7a73e3aeb224.
[2009-05-05 11:32:16 5541] ERROR (SrvDaemon:349) Exception starting xend
(syntax error: line 1, column 0)
Traceback (most recent call last):
  File "usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py",
line 341, in run
    servers = SrvServer.create()
  File "usr/lib/python2.5/site-packages/xen/xend/server/SrvServer.py",
line 251, in create
    root.putChild('xend', SrvRoot())
  File "usr/lib/python2.5/site-packages/xen/xend/server/SrvRoot.py",
line 40, in __init__
    self.get(name)
  File "usr/lib/python2.5/site-packages/xen/web/SrvDir.py", line 84, in
get
    val = val.getobj()
  File "usr/lib/python2.5/site-packages/xen/web/SrvDir.py", line 52, in
getobj
    self.obj = klassobj()
  File "usr/lib/python2.5/site-packages/xen/xend/server/SrvNode.py",
line 30, in __init__
    self.xn = XendNode.instance()
  File "usr/lib/python2.5/site-packages/xen/xend/XendNode.py", line 948,
in instance
    inst = XendNode()
  File "usr/lib/python2.5/site-packages/xen/xend/XendNode.py", line 68,
in __init__
    saved_host = self.state_store.load_state('host')
  File "usr/lib/python2.5/site-packages/xen/xend/XendStateStore.py",
line 104, in load_state
    dom = minidom.parse(xml_path)
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/minidom.py", line
1915, in parse
    return expatbuilder.parse(file)
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/expatbuilder.py",
line 926, in parse
    result = builder.parseFile(fp)
  File "/usr/lib/python2.5/site-packages/_xmlplus/dom/expatbuilder.py",
line 207, in parseFile
    parser.Parse(buffer, 0)
ExpatError: syntax error: line 1, column 0
[2009-05-05 11:32:16 5540] INFO (SrvDaemon:220) Xend exited with status
1.

Re: Problem starting xend in latest 3.4 release candidate

[Keir Fraser]

On 05/05/2009 04:40, "Tim Post" <echo@echoreply.us> wrote:

> I'm really, really bewildered. Any help is appreciated. I can't tell if
> this is due to upgrading, or if some other strange problem has appeared
> on my system. My Python is not the best :)
> 
> Last night, it worked. Today, it doesn't .. and I can't think of
> anything that happened overnight , especially since the system was shut
> down.

Looks like xend has probably tried to persist something and got itself
confused in the process. Try deleting /var/lib/xen/* before starting xend.

 -- Keir

Re: Problem starting xend in latest 3.4 release candidate

[Tim Post]

Hi Keir,

On Tue, 2009-05-05 at 08:28 +0100, Keir Fraser wrote:
> On 05/05/2009 04:40, "Tim Post" <echo@echoreply.us> wrote:
> 
> > I'm really, really bewildered. Any help is appreciated. I can't tell if
> > this is due to upgrading, or if some other strange problem has appeared
> > on my system. My Python is not the best :)
> > 
> > Last night, it worked. Today, it doesn't .. and I can't think of
> > anything that happened overnight , especially since the system was shut
> > down.
> 
> Looks like xend has probably tried to persist something and got itself
> confused in the process. Try deleting /var/lib/xen/* before starting xend.
> 
>  -- Keir

I ended up just completely un-installing Xen (make uninstall) and
re-installing the 3.4 rc, everything is fine now. I'll see if I can
reproduce this and let you know if removing that helps.

I've been doing a lot of xenstore smashing, plus working on a split
driver .. so its very possible that my efforts are what confused Xend.

Thanks for the fast reply.

Cheers,
--Tim

Re: Problem starting xend in latest 3.4 release candidate

[Simon Horman]

On Tue, May 05, 2009 at 08:28:45AM +0100, Keir Fraser wrote:
> On 05/05/2009 04:40, "Tim Post" <echo@echoreply.us> wrote:
> 
> > I'm really, really bewildered. Any help is appreciated. I can't tell if
> > this is due to upgrading, or if some other strange problem has appeared
> > on my system. My Python is not the best :)
> > 
> > Last night, it worked. Today, it doesn't .. and I can't think of
> > anything that happened overnight , especially since the system was shut
> > down.
> 
> Looks like xend has probably tried to persist something and got itself
> confused in the process. Try deleting /var/lib/xen/* before starting xend.

I saw a very similar problem the other day, which was
caused by an empty file in /var/lib/xen/
I never got to the bottom of how the file became empty.

-- 
Simon Horman
  VA Linux Systems Japan K.K. Satellite Lab in Sydney, Australia
  H: www.vergenet.net/~horms/            W: www.valinux.co.jp/en