* XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? @ 2011-02-08 17:02 Yuvraj Agarwal 2011-02-09 8:22 ` Vincent Hanquez 2011-02-09 18:26 ` Yuvraj Agarwal 0 siblings, 2 replies; 9+ messages in thread From: Yuvraj Agarwal @ 2011-02-08 17:02 UTC (permalink / raw) To: xen-devel Hi All, I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am having some trouble starting XEN. I am using the latest mercurial checkout (4.0.2-rc2) and the only change I made was to Config.mk to enable building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML version of xenstored since it is supposed to be much better/faster when starting up a larger number of domUs. The system starts up, and I can see xend started up, but when I do "xm list" or "xm info" I get an error message that xend has not started up? root@EM:/etc# xm list Error: Unable to connect to xend: No such file or directory. Is xend running? root@EM:/etc# root@EM:/etc# ps -efw | grep xen root 88 2 0 Feb07 ? 00:00:00 [xenwatch] root 89 2 0 Feb07 ? 00:00:00 [xenbus] root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python /usr/sbin/xend start root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python /usr/sbin/xend start root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen root@EM:/etc# root@EM:/etc# uname -a Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux root@EM:/etc# I did check and found two difference with my earlier installation of Xen 4.0.0 with the same kernel (although with the standard xenstored). 1. The xen 4.0 installation does show a "xenstored" process also running in addition to these ones. 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that doesn't seem to exist on my system? Any pointers to fix this issue? Thanks! Yuvraj ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? 2011-02-08 17:02 XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? Yuvraj Agarwal @ 2011-02-09 8:22 ` Vincent Hanquez 2011-02-10 8:42 ` Ian Campbell 2011-02-09 18:26 ` Yuvraj Agarwal 1 sibling, 1 reply; 9+ messages in thread From: Vincent Hanquez @ 2011-02-09 8:22 UTC (permalink / raw) To: Yuvraj Agarwal; +Cc: xen-devel On 08/02/11 17:02, Yuvraj Agarwal wrote: > Hi All, > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > having some trouble starting XEN. I am using the latest mercurial checkout > (4.0.2-rc2) and the only change I made was to Config.mk to enable building > the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using > the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen > 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML > version of xenstored since it is supposed to be much better/faster when > starting up a larger number of domUs. > > The system starts up, and I can see xend started up, but when I do "xm > list" or "xm info" I get an error message that xend has not started up? > > root@EM:/etc# xm list > Error: Unable to connect to xend: No such file or directory. Is xend > running? > root@EM:/etc# > > root@EM:/etc# ps -efw | grep xen > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > /usr/sbin/xend start > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > /usr/sbin/xend start > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > root@EM:/etc# > > root@EM:/etc# uname -a > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux > root@EM:/etc# > > I did check and found two difference with my earlier installation of Xen > 4.0.0 with the same kernel (although with the standard xenstored). > 1. The xen 4.0 installation does show a "xenstored" process also running > in addition to these ones. > 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that > doesn't seem to exist on my system? > > Any pointers to fix this issue? The reason xend died is that xenstored hasn't started (or something happened to it), you need to figure out why. You can start xenstored directly on the cmdline with the no daemonize flag, to see if the program stays running, and if not you'll see the reason why it dies. -- Vincent ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? 2011-02-09 8:22 ` Vincent Hanquez @ 2011-02-10 8:42 ` Ian Campbell 2011-02-10 17:53 ` Yuvraj Agarwal 0 siblings, 1 reply; 9+ messages in thread From: Ian Campbell @ 2011-02-10 8:42 UTC (permalink / raw) To: Vincent Hanquez; +Cc: xen-devel, Yuvraj Agarwal On Wed, 2011-02-09 at 08:22 +0000, Vincent Hanquez wrote: > On 08/02/11 17:02, Yuvraj Agarwal wrote: > > Hi All, > > > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > > having some trouble starting XEN. I am using the latest mercurial checkout > > (4.0.2-rc2) and the only change I made was to Config.mk to enable building > > the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using > > the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen > > 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML > > version of xenstored since it is supposed to be much better/faster when > > starting up a larger number of domUs. > > > > The system starts up, and I can see xend started up, but when I do "xm > > list" or "xm info" I get an error message that xend has not started up? > > > > root@EM:/etc# xm list > > Error: Unable to connect to xend: No such file or directory. Is xend > > running? > > root@EM:/etc# > > > > root@EM:/etc# ps -efw | grep xen > > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > > /usr/sbin/xend start > > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > > /usr/sbin/xend start > > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > > root@EM:/etc# > > > > root@EM:/etc# uname -a > > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux > > root@EM:/etc# > > > > I did check and found two difference with my earlier installation of Xen > > 4.0.0 with the same kernel (although with the standard xenstored). > > 1. The xen 4.0 installation does show a "xenstored" process also running > > in addition to these ones. > > 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that > > doesn't seem to exist on my system? > > > > Any pointers to fix this issue? > > The reason xend died is that xenstored hasn't started (or something > happened to it), you need to figure out why. > > You can start xenstored directly on the cmdline with the no daemonize > flag, to see if the program stays running, and if not you'll see > the reason why it dies. Xen 4.0 appears to use oxenstored from http://xenbits.xensource.com/ext/xen-ocaml-tools.hg (it only moved in-tree in the 4.1 dev cycle). IIRC the ocaml evtchn bindings used to blow away the /dev/evtchn node and replace it with their own, with hardcoded major:minor or 10:61. However that "61" is not really a static number and can vary depending on kernel version / configuration. Judging from Yuvraj's debugging his evtchn node has ended up at minor 60. Looks like http://xenbits.xensource.com/xen-unstable.hg?rev/e7b55cc5533a needs porting to xen-ocaml-tools.hg. Ian. ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? 2011-02-10 8:42 ` Ian Campbell @ 2011-02-10 17:53 ` Yuvraj Agarwal 2011-02-10 17:56 ` John McCullough 2011-02-10 18:08 ` Ian Campbell 0 siblings, 2 replies; 9+ messages in thread From: Yuvraj Agarwal @ 2011-02-10 17:53 UTC (permalink / raw) To: 'Ian Campbell', 'Vincent Hanquez'; +Cc: xen-devel Hi Ian and Vincent, Thank you for your responses - There are several possible issues it seems: ** I actually have both /dev/xen/eventchn and /dev/xen/evtchn but no /dev/xen/gntdev currently? root@EM:/home/yuvraj# ll /dev/xen/ total 0 drwxr-xr-x 2 root root 80 2011-02-10 09:18 ./ drwxr-xr-x 17 root root 6120 2011-02-10 09:18 ../ crw-r----- 1 root root 10, 63 2011-02-10 09:18 eventchn crw-rw---- 1 root root 10, 60 2011-02-10 09:18 evtchn root@EM:/home/yuvraj# ** It also looks like the minor numbers are mixed up, for example /proc/misc shows minor number 63 also for vga_arbitrer? Not sure if that matters. root@EM:/home/yuvraj# cat /proc/misc < snip > 59 gntdev 60 evtchn < snip > 63 vga_arbiter root@EM:/home/yuvraj# ** I also tried to start xenstored without the daemonize flag as Vincent mentioned, and posted the results in a separate email, (re-posting it here). root@EM:/home/yuvraj# strace -f /usr/bin/xenstored --no-fork <snip> open("/dev/xen/eventchn", O_RDWR) = 6 open("/proc/xen/xsd_port", O_RDONLY) = 7 read(7, "145", 20) = 3 close(7) = 0 open("/proc/xen/xsd_kva", O_RDWR) = 7 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0) = 0x7f5bbd2ef000 close(7) = 0 ioctl(6, EVIOCGVERSION, 0x7fffd0cd3e30) = -1 ENOTTY (Inappropriate ioctl for device) write(2, "Fatal error: exception Failure(\""..., 64Fatal error: exception Failure("ioctl bind_interdomain failed") ) = 64 exit_group(2) = ? Is there a way I can fix (e.g. a patch I can apply) this on my end? I can try it out and post the results back so that this makes it into the tree, and helps anyone else using the OCAML version of xenstored. Thanks! Yuvraj -----Original Message----- From: Ian Campbell [mailto:Ian.Campbell@citrix.com] Sent: Thursday, February 10, 2011 12:43 AM To: Vincent Hanquez Cc: Yuvraj Agarwal; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? On Wed, 2011-02-09 at 08:22 +0000, Vincent Hanquez wrote: > On 08/02/11 17:02, Yuvraj Agarwal wrote: > > Hi All, > > > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > > having some trouble starting XEN. I am using the latest mercurial > > checkout > > (4.0.2-rc2) and the only change I made was to Config.mk to enable > > building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= > > y". I am using the same kernel I built earlier for a similar Ubuntu > > 10.04-64bit / Xen > > 4.0.0 installation (version 2.6.32.12). I would like to use the > > OCAML version of xenstored since it is supposed to be much > > better/faster when starting up a larger number of domUs. > > > > The system starts up, and I can see xend started up, but when I do > > "xm list" or "xm info" I get an error message that xend has not started > > up? > > > > root@EM:/etc# xm list > > Error: Unable to connect to xend: No such file or directory. Is xend > > running? > > root@EM:/etc# > > > > root@EM:/etc# ps -efw | grep xen > > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > > /usr/sbin/xend start > > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > > /usr/sbin/xend start > > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > > root@EM:/etc# > > > > root@EM:/etc# uname -a > > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 > > GNU/Linux root@EM:/etc# > > > > I did check and found two difference with my earlier installation of > > Xen > > 4.0.0 with the same kernel (although with the standard xenstored). > > 1. The xen 4.0 installation does show a "xenstored" process also > > running in addition to these ones. > > 2. "xend" seems to be looking for a directory, " > > /var/lib/xenstored" that doesn't seem to exist on my system? > > > > Any pointers to fix this issue? > > The reason xend died is that xenstored hasn't started (or something > happened to it), you need to figure out why. > > You can start xenstored directly on the cmdline with the no daemonize > flag, to see if the program stays running, and if not you'll see the > reason why it dies. Xen 4.0 appears to use oxenstored from http://xenbits.xensource.com/ext/xen-ocaml-tools.hg (it only moved in-tree in the 4.1 dev cycle). IIRC the ocaml evtchn bindings used to blow away the /dev/evtchn node and replace it with their own, with hardcoded major:minor or 10:61. However that "61" is not really a static number and can vary depending on kernel version / configuration. Judging from Yuvraj's debugging his evtchn node has ended up at minor 60. Looks like http://xenbits.xensource.com/xen-unstable.hg?rev/e7b55cc5533a needs porting to xen-ocaml-tools.hg. Ian. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? 2011-02-10 17:53 ` Yuvraj Agarwal @ 2011-02-10 17:56 ` John McCullough 2011-02-10 18:08 ` Ian Campbell 1 sibling, 0 replies; 9+ messages in thread From: John McCullough @ 2011-02-10 17:56 UTC (permalink / raw) To: Yuvraj Agarwal; +Cc: xen-devel, Vincent Hanquez, Ian Campbell [-- Attachment #1.1: Type: text/plain, Size: 5423 bytes --] ocaml-xenstored made 10 63 eventchn. On Thu, Feb 10, 2011 at 9:53 AM, Yuvraj Agarwal <yuvraj@cs.ucsd.edu> wrote: > Hi Ian and Vincent, > > Thank you for your responses - There are several possible issues it seems: > > ** I actually have both /dev/xen/eventchn and /dev/xen/evtchn but no > /dev/xen/gntdev currently? > > root@EM:/home/yuvraj# ll /dev/xen/ > total 0 > drwxr-xr-x 2 root root 80 2011-02-10 09:18 ./ > drwxr-xr-x 17 root root 6120 2011-02-10 09:18 ../ > crw-r----- 1 root root 10, 63 2011-02-10 09:18 eventchn > crw-rw---- 1 root root 10, 60 2011-02-10 09:18 evtchn > root@EM:/home/yuvraj# > > ** It also looks like the minor numbers are mixed up, for example > /proc/misc > shows minor number 63 also for vga_arbitrer? Not sure if that matters. > > root@EM:/home/yuvraj# cat /proc/misc > < snip > > 59 gntdev > 60 evtchn > < snip > > 63 vga_arbiter > root@EM:/home/yuvraj# > > ** I also tried to start xenstored without the daemonize flag as Vincent > mentioned, and posted the results in a separate email, (re-posting it > here). > > root@EM:/home/yuvraj# strace -f /usr/bin/xenstored --no-fork > > <snip> > open("/dev/xen/eventchn", O_RDWR) = 6 > open("/proc/xen/xsd_port", O_RDONLY) = 7 > read(7, "145", 20) = 3 > close(7) = 0 > open("/proc/xen/xsd_kva", O_RDWR) = 7 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0) = 0x7f5bbd2ef000 > close(7) = 0 > ioctl(6, EVIOCGVERSION, 0x7fffd0cd3e30) = -1 ENOTTY (Inappropriate ioctl > for > device) > write(2, "Fatal error: exception Failure(\""..., 64Fatal error: exception > Failure("ioctl bind_interdomain failed") > ) = 64 > exit_group(2) = ? > > Is there a way I can fix (e.g. a patch I can apply) this on my end? I can > try it out and post the results back so that this makes it into the tree, > and helps anyone else using the OCAML version of xenstored. > > Thanks! > Yuvraj > > > -----Original Message----- > From: Ian Campbell [mailto:Ian.Campbell@citrix.com] > Sent: Thursday, February 10, 2011 12:43 AM > To: Vincent Hanquez > Cc: Yuvraj Agarwal; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not > starting? > > On Wed, 2011-02-09 at 08:22 +0000, Vincent Hanquez wrote: > > On 08/02/11 17:02, Yuvraj Agarwal wrote: > > > Hi All, > > > > > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > > > having some trouble starting XEN. I am using the latest mercurial > > > checkout > > > (4.0.2-rc2) and the only change I made was to Config.mk to enable > > > building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= > > > y". I am using the same kernel I built earlier for a similar Ubuntu > > > 10.04-64bit / Xen > > > 4.0.0 installation (version 2.6.32.12). I would like to use the > > > OCAML version of xenstored since it is supposed to be much > > > better/faster when starting up a larger number of domUs. > > > > > > The system starts up, and I can see xend started up, but when I do > > > "xm list" or "xm info" I get an error message that xend has not started > > > up? > > > > > > root@EM:/etc# xm list > > > Error: Unable to connect to xend: No such file or directory. Is xend > > > running? > > > root@EM:/etc# > > > > > > root@EM:/etc# ps -efw | grep xen > > > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > > > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > > > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > > > /usr/sbin/xend start > > > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > > > /usr/sbin/xend start > > > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > > > root@EM:/etc# > > > > > > root@EM:/etc# uname -a > > > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 > > > GNU/Linux root@EM:/etc# > > > > > > I did check and found two difference with my earlier installation of > > > Xen > > > 4.0.0 with the same kernel (although with the standard xenstored). > > > 1. The xen 4.0 installation does show a "xenstored" process also > > > running in addition to these ones. > > > 2. "xend" seems to be looking for a directory, " > > > /var/lib/xenstored" that doesn't seem to exist on my system? > > > > > > Any pointers to fix this issue? > > > > The reason xend died is that xenstored hasn't started (or something > > happened to it), you need to figure out why. > > > > You can start xenstored directly on the cmdline with the no daemonize > > flag, to see if the program stays running, and if not you'll see the > > reason why it dies. > > Xen 4.0 appears to use oxenstored from > http://xenbits.xensource.com/ext/xen-ocaml-tools.hg (it only moved in-tree > in the 4.1 dev cycle). > > IIRC the ocaml evtchn bindings used to blow away the /dev/evtchn node and > replace it with their own, with hardcoded major:minor or 10:61. > However that "61" is not really a static number and can vary depending on > kernel version / configuration. > > Judging from Yuvraj's debugging his evtchn node has ended up at minor 60. > > Looks like http://xenbits.xensource.com/xen-unstable.hg?rev/e7b55cc5533a > needs porting to xen-ocaml-tools.hg. > > Ian. > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > [-- Attachment #1.2: Type: text/html, Size: 7067 bytes --] [-- Attachment #2: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? 2011-02-10 17:53 ` Yuvraj Agarwal 2011-02-10 17:56 ` John McCullough @ 2011-02-10 18:08 ` Ian Campbell 1 sibling, 0 replies; 9+ messages in thread From: Ian Campbell @ 2011-02-10 18:08 UTC (permalink / raw) To: Yuvraj Agarwal; +Cc: xen-devel, Vincent Hanquez On Thu, 2011-02-10 at 17:53 +0000, Yuvraj Agarwal wrote: > Hi Ian and Vincent, > > Thank you for your responses - There are several possible issues it seems: No, just one -- the event channel one. > ** I actually have both /dev/xen/eventchn and /dev/xen/evtchn > but no /dev/xen/gntdev currently? AFAIK oxenstored does not use gntdev. > root@EM:/home/yuvraj# > > ** It also looks like the minor numbers are mixed up, for example /proc/misc > shows minor number 63 also for vga_arbitrer? Not sure if that matters. The misc device minor numbers are completely dynamic so becoming "mixed up" is normal. The bug is that this version of oxenstored hardcodes a specific minor number, which is wrong. It also uses the device name /dev/xen/eventchn which is different to the /dev/xen/evtchn that the system creates, but that's a minor aside. > root@EM:/home/yuvraj# cat /proc/misc > < snip > > 59 gntdev > 60 evtchn > < snip > > 63 vga_arbiter > root@EM:/home/yuvraj# [...] > root@EM:/home/yuvraj# strace -f /usr/bin/xenstored --no-fork > > <snip> > open("/dev/xen/eventchn", O_RDWR) = 6 So here you open /dev/xen/eventchn, which oxenstored created as 10:63. But, as shown in /proc/misc above, the event channel driver actually ends up with minor 60 on this kernel and so the device you have just opened is the vga_arbiter, whatever that is. Hence... > ioctl(6, EVIOCGVERSION, 0x7fffd0cd3e30) = -1 ENOTTY (Inappropriate ioctl for > device) ... this fails, since that ioctl is not valid for the vga_arbiter. > Is there a way I can fix (e.g. a patch I can apply) this on my end? Yes. Apply the patch I linked to in my previous mail (e.g. http://xenbits.xensource.com/xen-unstable.hg?rev/e7b55cc5533a) to the tree under tools/ocaml-xenstored. You may need to adjust the paths in the (hopefully) obvious way. This patch causes oxenstored to use the exiting evtchn device node which is correctly created by udev with the correct minor number. Ian. ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? 2011-02-08 17:02 XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? Yuvraj Agarwal 2011-02-09 8:22 ` Vincent Hanquez @ 2011-02-09 18:26 ` Yuvraj Agarwal 2011-02-10 4:39 ` Yuvraj Agarwal 1 sibling, 1 reply; 9+ messages in thread From: Yuvraj Agarwal @ 2011-02-09 18:26 UTC (permalink / raw) To: xen-devel; +Cc: Asim [-- Attachment #1.1: Type: text/plain, Size: 3356 bytes --] I did some more debugging and based on some of the instructions on http://wiki.xen.org/xenwiki/XenParavirtOps I found out that I may have been missing the "gntdev" module. I rebuilt the kernel and added that in (part of the kernel rather than as a separate module), but still have the same issues. Adding some more debugging information: root@nor:/home/yuvraj# cat /proc/misc 54 network_throughput 55 network_latency 56 cpu_dma_latency 57 device-mapper 221 mpt2ctl 144 nvram 228 hpet 58 blktap-control 59 gntdev 60 evtchn 61 autofs 231 snapshot 184 microcode 227 mcelog 62 rfkill 63 vga_arbiter root@nor:/home/yuvraj# ll /dev/xen/ total 0 drwxr-xr-x 2 root root 80 2011-02-08 22:07 ./ drwxr-xr-x 17 root root 6120 2011-02-08 22:07 ../ crw-r----- 1 root root 10, 63 2011-02-08 22:07 eventchn crw-rw---- 1 root root 10, 60 2011-02-08 22:07 evtchn root@nor:/home/yuvraj# Seems like I am missing the /dev/xen/gntdev entries? Shouldn't these be created automatically? Thanks Yuvraj -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal Sent: Tuesday, February 08, 2011 9:02 AM To: xen-devel@lists.xensource.com Subject: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? Hi All, I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am having some trouble starting XEN. I am using the latest mercurial checkout (4.0.2-rc2) and the only change I made was to Config.mk to enable building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML version of xenstored since it is supposed to be much better/faster when starting up a larger number of domUs. The system starts up, and I can see xend started up, but when I do "xm list" or "xm info" I get an error message that xend has not started up? root@EM:/etc# xm list Error: Unable to connect to xend: No such file or directory. Is xend running? root@EM:/etc# root@EM:/etc# ps -efw | grep xen root 88 2 0 Feb07 ? 00:00:00 [xenwatch] root 89 2 0 Feb07 ? 00:00:00 [xenbus] root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python /usr/sbin/xend start root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python /usr/sbin/xend start root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen root@EM:/etc# root@EM:/etc# uname -a Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux root@EM:/etc# I did check and found two difference with my earlier installation of Xen 4.0.0 with the same kernel (although with the standard xenstored). 1. The xen 4.0 installation does show a "xenstored" process also running in addition to these ones. 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that doesn't seem to exist on my system? Any pointers to fix this issue? Thanks! Yuvraj _______________________________________________ Xen-devel mailing list <mailto:Xen-devel@lists.xensource.com> Xen-devel@lists.xensource.com <http://lists.xensource.com/xen-devel> http://lists.xensource.com/xen-devel [-- Attachment #1.2: Type: text/html, Size: 9645 bytes --] [-- Attachment #2: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? 2011-02-09 18:26 ` Yuvraj Agarwal @ 2011-02-10 4:39 ` Yuvraj Agarwal 2011-02-10 7:11 ` Pasi Kärkkäinen 0 siblings, 1 reply; 9+ messages in thread From: Yuvraj Agarwal @ 2011-02-10 4:39 UTC (permalink / raw) To: xen-devel; +Cc: John McCullough, 'Asim' [-- Attachment #1.1: Type: text/plain, Size: 4914 bytes --] Hi All, I did some further debugging and am attaching some more information here. Any help is appreciated of course. Out of curiosity, are we amongst the few people who are trying to use the OCAML version of xenstored? If I change the Config.mk to disable building the OCAML version of xenstored xend starts up and I can do "xm info" and everything works fine. If I build the OCAML version of xenstored instead, I get the "xend is not started error" even though I see two xend processes. Furthermore, if I run the OCAML xentored with an strace, I get the following error: "strace -f /usr/bin/xenstored --no-fork" open("/dev/xen/eventchn", O_RDWR) = 6 open("/proc/xen/xsd_port", O_RDONLY) = 7 read(7, "145", 20) = 3 close(7) = 0 open("/proc/xen/xsd_kva", O_RDWR) = 7 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0) = 0x7f2a4f682000 close(7) = 0 ioctl(6, EVIOCGVERSION, 0x7fff86ad7320) = -1 ENOTTY (Inappropriate ioctl for device) write(2, "Fatal error: exception Failure(\""..., 64Fatal error: exception Failure("ioctl bind_interdomain failed") ) = 64 exit_group(2) = ? thanks Yuvraj From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal Sent: Wednesday, February 09, 2011 10:26 AM To: xen-devel@lists.xensource.com Cc: Asim Subject: RE: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? I did some more debugging and based on some of the instructions on http://wiki.xen.org/xenwiki/XenParavirtOps I found out that I may have been missing the "gntdev" module. I rebuilt the kernel and added that in (part of the kernel rather than as a separate module), but still have the same issues. Adding some more debugging information: root@nor:/home/yuvraj# cat /proc/misc 54 network_throughput 55 network_latency 56 cpu_dma_latency 57 device-mapper 221 mpt2ctl 144 nvram 228 hpet 58 blktap-control 59 gntdev 60 evtchn 61 autofs 231 snapshot 184 microcode 227 mcelog 62 rfkill 63 vga_arbiter root@nor:/home/yuvraj# ll /dev/xen/ total 0 drwxr-xr-x 2 root root 80 2011-02-08 22:07 ./ drwxr-xr-x 17 root root 6120 2011-02-08 22:07 ../ crw-r----- 1 root root 10, 63 2011-02-08 22:07 eventchn crw-rw---- 1 root root 10, 60 2011-02-08 22:07 evtchn root@nor:/home/yuvraj# Seems like I am missing the /dev/xen/gntdev entries? Shouldn't these be created automatically? Thanks Yuvraj -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal Sent: Tuesday, February 08, 2011 9:02 AM To: xen-devel@lists.xensource.com Subject: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? Hi All, I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am having some trouble starting XEN. I am using the latest mercurial checkout (4.0.2-rc2) and the only change I made was to Config.mk to enable building the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML version of xenstored since it is supposed to be much better/faster when starting up a larger number of domUs. The system starts up, and I can see xend started up, but when I do "xm list" or "xm info" I get an error message that xend has not started up? root@EM:/etc# xm list Error: Unable to connect to xend: No such file or directory. Is xend running? root@EM:/etc# root@EM:/etc# ps -efw | grep xen root 88 2 0 Feb07 ? 00:00:00 [xenwatch] root 89 2 0 Feb07 ? 00:00:00 [xenbus] root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python /usr/sbin/xend start root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python /usr/sbin/xend start root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen root@EM:/etc# root@EM:/etc# uname -a Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux root@EM:/etc# I did check and found two difference with my earlier installation of Xen 4.0.0 with the same kernel (although with the standard xenstored). 1. The xen 4.0 installation does show a "xenstored" process also running in addition to these ones. 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that doesn't seem to exist on my system? Any pointers to fix this issue? Thanks! Yuvraj _______________________________________________ Xen-devel mailing list <mailto:Xen-devel@lists.xensource.com> Xen-devel@lists.xensource.com <http://lists.xensource.com/xen-devel> http://lists.xensource.com/xen-devel [-- Attachment #1.2: Type: text/html, Size: 14740 bytes --] [-- Attachment #2: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? 2011-02-10 4:39 ` Yuvraj Agarwal @ 2011-02-10 7:11 ` Pasi Kärkkäinen 0 siblings, 0 replies; 9+ messages in thread From: Pasi Kärkkäinen @ 2011-02-10 7:11 UTC (permalink / raw) To: Yuvraj Agarwal; +Cc: xen-devel, John McCullough, 'Asim' On Wed, Feb 09, 2011 at 08:39:37PM -0800, Yuvraj Agarwal wrote: > Hi All, > > > > I did some further debugging and am attaching some more information here. > Any help is appreciated of course. Out of curiosity, are we amongst the > few people who are trying to use the OCAML version of xenstored? > At least Citrix XenServer and XCP do use oxenstored.. -- Pasi > If I change the Config.mk to disable building the OCAML version of > xenstored xend starts up and I can do "xm info" and everything works fine. > > > > If I build the OCAML version of xenstored instead, I get the "xend is not > started error" even though I see two xend processes. > > Furthermore, if I run the OCAML xentored with an strace, I get the > following error: > > > > "strace -f /usr/bin/xenstored --no-fork" > > > > open("/dev/xen/eventchn", O_RDWR) = 6 > > open("/proc/xen/xsd_port", O_RDONLY) = 7 > > read(7, "145", 20) = 3 > > close(7) = 0 > > open("/proc/xen/xsd_kva", O_RDWR) = 7 > > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 7, 0) = 0x7f2a4f682000 > > close(7) = 0 > > ioctl(6, EVIOCGVERSION, 0x7fff86ad7320) = -1 ENOTTY (Inappropriate ioctl > for device) > > write(2, "Fatal error: exception Failure(\""..., 64Fatal error: exception > Failure("ioctl bind_interdomain failed") > > ) = 64 > > exit_group(2) = ? > > > > thanks > > Yuvraj > > > > > > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal > Sent: Wednesday, February 09, 2011 10:26 AM > To: xen-devel@lists.xensource.com > Cc: Asim > Subject: RE: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not > starting? > > > > I did some more debugging and based on some of the instructions on > [1]http://wiki.xen.org/xenwiki/XenParavirtOps I found out that I may have > been missing the "gntdev" module. > > I rebuilt the kernel and added that in (part of the kernel rather than as > a separate module), but still have the same issues. Adding some more > debugging information: > > > > root@nor:/home/yuvraj# cat /proc/misc > > 54 network_throughput > > 55 network_latency > > 56 cpu_dma_latency > > 57 device-mapper > > 221 mpt2ctl > > 144 nvram > > 228 hpet > > 58 blktap-control > > 59 gntdev > > 60 evtchn > > 61 autofs > > 231 snapshot > > 184 microcode > > 227 mcelog > > 62 rfkill > > 63 vga_arbiter > > root@nor:/home/yuvraj# ll /dev/xen/ > > total 0 > > drwxr-xr-x 2 root root 80 2011-02-08 22:07 ./ > > drwxr-xr-x 17 root root 6120 2011-02-08 22:07 ../ > > crw-r----- 1 root root 10, 63 2011-02-08 22:07 eventchn > > crw-rw---- 1 root root 10, 60 2011-02-08 22:07 evtchn > > root@nor:/home/yuvraj# > > > > Seems like I am missing the /dev/xen/gntdev entries? Shouldn't these be > created automatically? > > Thanks > > Yuvraj > > > > -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Yuvraj Agarwal > Sent: Tuesday, February 08, 2011 9:02 AM > To: xen-devel@lists.xensource.com > Subject: [Xen-devel] XEN 4.0.2-rc2 - using OCAML xenstored -- xend not > starting? > > > > Hi All, > > > > I just tried a new installation of Ubuntu 10.04 (64bit) + Xen and am > having some trouble starting XEN. I am using the latest mercurial checkout > > (4.0.2-rc2) and the only change I made was to Config.mk to enable building > the OCAML version of xenstored. " CONFIG_OCAML_XENSTORED ?= y". I am using > the same kernel I built earlier for a similar Ubuntu 10.04-64bit / Xen > > 4.0.0 installation (version 2.6.32.12). I would like to use the OCAML > version of xenstored since it is supposed to be much better/faster when > starting up a larger number of domUs. > > > > The system starts up, and I can see xend started up, but when I do "xm > list" or "xm info" I get an error message that xend has not started up? > > > > root@EM:/etc# xm list > > Error: Unable to connect to xend: No such file or directory. Is xend > running? > > root@EM:/etc# > > > > root@EM:/etc# ps -efw | grep xen > > root 88 2 0 Feb07 ? 00:00:00 [xenwatch] > > root 89 2 0 Feb07 ? 00:00:00 [xenbus] > > root 6254 1 0 Feb07 ? 00:00:00 /usr/bin/python > > /usr/sbin/xend start > > root 6255 6254 0 Feb07 ? 00:00:17 /usr/bin/python > > /usr/sbin/xend start > > root 7043 5919 0 08:55 pts/2 00:00:00 grep --color=auto xen > > root@EM:/etc# > > > > root@EM:/etc# uname -a > > Linux EM 2.6.32.12 #11 SMP Fri Jun 18 20:52:17 PDT 2010 x86_64 GNU/Linux > root@EM:/etc# > > > > I did check and found two difference with my earlier installation of Xen > > 4.0.0 with the same kernel (although with the standard xenstored). > > 1. The xen 4.0 installation does show a "xenstored" process also running > in addition to these ones. > > 2. "xend" seems to be looking for a directory, " /var/lib/xenstored" that > doesn't seem to exist on my system? > > > > Any pointers to fix this issue? > > > > Thanks! > > Yuvraj > > > > _______________________________________________ > > Xen-devel mailing list > > [2]Xen-devel@lists.xensource.com > > [3]http://lists.xensource.com/xen-devel > > References > > Visible links > 1. http://wiki.xen.org/xenwiki/XenParavirtOps > 2. mailto:Xen-devel@lists.xensource.com > 3. http://lists.xensource.com/xen-devel > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2011-02-10 18:08 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-02-08 17:02 XEN 4.0.2-rc2 - using OCAML xenstored -- xend not starting? Yuvraj Agarwal 2011-02-09 8:22 ` Vincent Hanquez 2011-02-10 8:42 ` Ian Campbell 2011-02-10 17:53 ` Yuvraj Agarwal 2011-02-10 17:56 ` John McCullough 2011-02-10 18:08 ` Ian Campbell 2011-02-09 18:26 ` Yuvraj Agarwal 2011-02-10 4:39 ` Yuvraj Agarwal 2011-02-10 7:11 ` Pasi Kärkkäinen
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.