* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS [not found] <948996368.3219599.1417477301121.JavaMail.yahoo@jws10637.mail.bf1.yahoo.com> @ 2014-12-02 9:29 ` Ian Campbell 2014-12-02 15:17 ` Olaf Hering 0 siblings, 1 reply; 11+ messages in thread From: Ian Campbell @ 2014-12-02 9:29 UTC (permalink / raw) To: Mark Pryor Cc: Olaf Hering, Wei Liu, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Mon, 2014-12-01 at 23:41 +0000, Mark Pryor wrote: > list, Thanks. If you've identified a buggy changeset then it is fine to post to the devel lists. I've added a CC. I've also CCd everyone listed in the commit which you've fingered. Olaf, does this suggested change look correct? If so then can you turn it into a patch please. > > > this commit to 4.5: > > git show 4542ae340d75bd6319e3fcd94e6c9336e210aeef > > > > ^^ causes my C7 install to hang at shutdown. > xenstored.service will shutdown in parallel with xendomains before it > should > and its broken > > its by Olaf > I took the time to narrow the commit down exactly.. first time I ever > did that > Its a regression. The xen shutdown process in systemd was OK before > that commit > > > > xenconsoled.service needs > After=xenstored.service > > > not, > After=xenstored.socket > > > regards, > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-02 9:29 ` [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS Ian Campbell @ 2014-12-02 15:17 ` Olaf Hering 2014-12-02 15:39 ` Ian Campbell 2014-12-05 17:11 ` Olaf Hering 0 siblings, 2 replies; 11+ messages in thread From: Olaf Hering @ 2014-12-02 15:17 UTC (permalink / raw) To: Ian Campbell Cc: Mark Pryor, Wei Liu, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Tue, Dec 02, Ian Campbell wrote: > On Mon, 2014-12-01 at 23:41 +0000, Mark Pryor wrote: > > list, > > Thanks. If you've identified a buggy changeset then it is fine to post > to the devel lists. I've added a CC. I've also CCd everyone listed in > the commit which you've fingered. > > Olaf, does this suggested change look correct? If so then can you turn > it into a patch please. Yes, something like this (sed -i 's@socket@service@g' *.in): diff --git a/tools/hotplug/Linux/systemd/xen-init-dom0.service.in b/tools/hotplug/Linux/systemd/xen-init-dom0.service.in index 4d4cb23..3befadc 100644 --- a/tools/hotplug/Linux/systemd/xen-init-dom0.service.in +++ b/tools/hotplug/Linux/systemd/xen-init-dom0.service.in @@ -1,7 +1,7 @@ [Unit] Description=xen-init-dom0, initialise Dom0 configuration (xenstore nodes, JSON configuration stub) -Requires=xenstored.socket proc-xen.mount -After=xenstored.socket proc-xen.mount +Requires=xenstored.service proc-xen.mount +After=xenstored.service proc-xen.mount ConditionPathExists=/proc/xen/capabilities [Service] diff --git a/tools/hotplug/Linux/systemd/xen-qemu-dom0-disk-backend.service.in b/tools/hotplug/Linux/systemd/xen-qemu-dom0-disk-backend.service.in index 6b9c96e..0a5807a 100644 --- a/tools/hotplug/Linux/systemd/xen-qemu-dom0-disk-backend.service.in +++ b/tools/hotplug/Linux/systemd/xen-qemu-dom0-disk-backend.service.in @@ -1,7 +1,7 @@ [Unit] Description=qemu for xen dom0 disk backend -Requires=proc-xen.mount xenstored.socket -After=proc-xen.mount xenstored.socket xenconsoled.service +Requires=proc-xen.mount xenstored.service +After=proc-xen.mount xenstored.service xenconsoled.service Before=xendomains.service libvirtd.service libvirt-guests.service RefuseManualStop=true ConditionPathExists=/proc/xen/capabilities diff --git a/tools/hotplug/Linux/systemd/xenconsoled.service.in b/tools/hotplug/Linux/systemd/xenconsoled.service.in index 2c5d99f..cb44cd6 100644 --- a/tools/hotplug/Linux/systemd/xenconsoled.service.in +++ b/tools/hotplug/Linux/systemd/xenconsoled.service.in @@ -1,7 +1,7 @@ [Unit] Description=Xenconsoled - handles logging from guest consoles and hypervisor -Requires=proc-xen.mount xenstored.socket -After=proc-xen.mount xenstored.socket +Requires=proc-xen.mount xenstored.service +After=proc-xen.mount xenstored.service ConditionPathExists=/proc/xen/capabilities [Service] diff --git a/tools/hotplug/Linux/systemd/xendomains.service.in b/tools/hotplug/Linux/systemd/xendomains.service.in index 757278f..9962671 100644 --- a/tools/hotplug/Linux/systemd/xendomains.service.in +++ b/tools/hotplug/Linux/systemd/xendomains.service.in @@ -1,7 +1,7 @@ [Unit] Description=Xendomains - start and stop guests on boot and shutdown -Requires=proc-xen.mount xenstored.socket -After=proc-xen.mount xenstored.socket xenconsoled.service xen-init-dom0.service +Requires=proc-xen.mount xenstored.service +After=proc-xen.mount xenstored.service xenconsoled.service xen-init-dom0.service ConditionPathExists=/proc/xen/capabilities [Service] Olaf ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-02 15:17 ` Olaf Hering @ 2014-12-02 15:39 ` Ian Campbell 2014-12-02 16:06 ` Olaf Hering 2014-12-05 17:11 ` Olaf Hering 1 sibling, 1 reply; 11+ messages in thread From: Ian Campbell @ 2014-12-02 15:39 UTC (permalink / raw) To: Olaf Hering Cc: Mark Pryor, Wei Liu, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Tue, 2014-12-02 at 16:17 +0100, Olaf Hering wrote: > On Tue, Dec 02, Ian Campbell wrote: > > > On Mon, 2014-12-01 at 23:41 +0000, Mark Pryor wrote: > > > list, > > > > Thanks. If you've identified a buggy changeset then it is fine to post > > to the devel lists. I've added a CC. I've also CCd everyone listed in > > the commit which you've fingered. > > > > Olaf, does this suggested change look correct? If so then can you turn > > it into a patch please. > > Yes, something like this (sed -i 's@socket@service@g' *.in): Can you submit to -devel with a commit log and S-o-b etc please. > > > diff --git a/tools/hotplug/Linux/systemd/xen-init-dom0.service.in b/tools/hotplug/Linux/systemd/xen-init-dom0.service.in > index 4d4cb23..3befadc 100644 > --- a/tools/hotplug/Linux/systemd/xen-init-dom0.service.in > +++ b/tools/hotplug/Linux/systemd/xen-init-dom0.service.in > @@ -1,7 +1,7 @@ > [Unit] > Description=xen-init-dom0, initialise Dom0 configuration (xenstore nodes, JSON configuration stub) > -Requires=xenstored.socket proc-xen.mount > -After=xenstored.socket proc-xen.mount > +Requires=xenstored.service proc-xen.mount > +After=xenstored.service proc-xen.mount > ConditionPathExists=/proc/xen/capabilities > > [Service] > diff --git a/tools/hotplug/Linux/systemd/xen-qemu-dom0-disk-backend.service.in b/tools/hotplug/Linux/systemd/xen-qemu-dom0-disk-backend.service.in > index 6b9c96e..0a5807a 100644 > --- a/tools/hotplug/Linux/systemd/xen-qemu-dom0-disk-backend.service.in > +++ b/tools/hotplug/Linux/systemd/xen-qemu-dom0-disk-backend.service.in > @@ -1,7 +1,7 @@ > [Unit] > Description=qemu for xen dom0 disk backend > -Requires=proc-xen.mount xenstored.socket > -After=proc-xen.mount xenstored.socket xenconsoled.service > +Requires=proc-xen.mount xenstored.service > +After=proc-xen.mount xenstored.service xenconsoled.service > Before=xendomains.service libvirtd.service libvirt-guests.service > RefuseManualStop=true > ConditionPathExists=/proc/xen/capabilities > diff --git a/tools/hotplug/Linux/systemd/xenconsoled.service.in b/tools/hotplug/Linux/systemd/xenconsoled.service.in > index 2c5d99f..cb44cd6 100644 > --- a/tools/hotplug/Linux/systemd/xenconsoled.service.in > +++ b/tools/hotplug/Linux/systemd/xenconsoled.service.in > @@ -1,7 +1,7 @@ > [Unit] > Description=Xenconsoled - handles logging from guest consoles and hypervisor > -Requires=proc-xen.mount xenstored.socket > -After=proc-xen.mount xenstored.socket > +Requires=proc-xen.mount xenstored.service > +After=proc-xen.mount xenstored.service > ConditionPathExists=/proc/xen/capabilities > > [Service] > diff --git a/tools/hotplug/Linux/systemd/xendomains.service.in b/tools/hotplug/Linux/systemd/xendomains.service.in > index 757278f..9962671 100644 > --- a/tools/hotplug/Linux/systemd/xendomains.service.in > +++ b/tools/hotplug/Linux/systemd/xendomains.service.in > @@ -1,7 +1,7 @@ > [Unit] > Description=Xendomains - start and stop guests on boot and shutdown > -Requires=proc-xen.mount xenstored.socket > -After=proc-xen.mount xenstored.socket xenconsoled.service xen-init-dom0.service > +Requires=proc-xen.mount xenstored.service > +After=proc-xen.mount xenstored.service xenconsoled.service xen-init-dom0.service > ConditionPathExists=/proc/xen/capabilities > > [Service] > > Olaf > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-02 15:39 ` Ian Campbell @ 2014-12-02 16:06 ` Olaf Hering 0 siblings, 0 replies; 11+ messages in thread From: Olaf Hering @ 2014-12-02 16:06 UTC (permalink / raw) To: Ian Campbell Cc: Mark Pryor, Wei Liu, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Tue, Dec 02, Ian Campbell wrote: > On Tue, 2014-12-02 at 16:17 +0100, Olaf Hering wrote: > > On Tue, Dec 02, Ian Campbell wrote: > > > > > On Mon, 2014-12-01 at 23:41 +0000, Mark Pryor wrote: > > > > list, > > > > > > Thanks. If you've identified a buggy changeset then it is fine to post > > > to the devel lists. I've added a CC. I've also CCd everyone listed in > > > the commit which you've fingered. > > > > > > Olaf, does this suggested change look correct? If so then can you turn > > > it into a patch please. > > > > Yes, something like this (sed -i 's@socket@service@g' *.in): > > Can you submit to -devel with a commit log and S-o-b etc please. Yes, I did already. Looks like git send-email does not process a Reported-by tag so Mark did not actually got a copy of that submission. But the actual patch did not change so what I sent him earlier can be used as is. Olaf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-02 15:17 ` Olaf Hering 2014-12-02 15:39 ` Ian Campbell @ 2014-12-05 17:11 ` Olaf Hering 2014-12-08 17:19 ` Olaf Hering 1 sibling, 1 reply; 11+ messages in thread From: Olaf Hering @ 2014-12-05 17:11 UTC (permalink / raw) To: Ian Campbell Cc: Mark Pryor, Wei Liu, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Tue, Dec 02, Olaf Hering wrote: > On Tue, Dec 02, Ian Campbell wrote: > > > On Mon, 2014-12-01 at 23:41 +0000, Mark Pryor wrote: > > > list, > > > > Thanks. If you've identified a buggy changeset then it is fine to post > > to the devel lists. I've added a CC. I've also CCd everyone listed in > > the commit which you've fingered. > > > > Olaf, does this suggested change look correct? If so then can you turn > > it into a patch please. > > Yes, something like this (sed -i 's@socket@service@g' *.in): But even with that change xendomains is hanging if it cant talk to xenstored for whatever reason. The result is that the sytem hangs forever at shutdown. I will try to fix that for 4.5. Olaf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-05 17:11 ` Olaf Hering @ 2014-12-08 17:19 ` Olaf Hering 2014-12-08 17:23 ` Wei Liu 0 siblings, 1 reply; 11+ messages in thread From: Olaf Hering @ 2014-12-08 17:19 UTC (permalink / raw) To: Ian Campbell Cc: Mark Pryor, Wei Liu, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Fri, Dec 05, Olaf Hering wrote: > But even with that change xendomains is hanging if it cant talk to > xenstored for whatever reason. The result is that the sytem hangs > forever at shutdown. This looks like a bug in the tools. xl or xenstore-ls hang forever if xenstored disappears. Looks like xs_open() is the culprit. The socket functions fail, then it attempts to use xs_domain_dev() which happens to succeed. Only xenstore-ls has the '-s' option to force operating on a socket. I wonder why libxl does not pass XS_OPEN_SOCKETONLY? It just uses the wrappers for xs_open. Can xenstored run in another domain? Olaf 18:02:43.522767 stat("/var/run/xenstored/socket", {st_mode=S_IFSOCK|0600, st_size=0, ...}) = 0 18:02:43.522864 socket(PF_LOCAL, SOCK_STREAM, 0) = 3 18:02:43.522964 fcntl(3, F_GETFD) = 0 18:02:43.523035 fcntl(3, F_SETFD, FD_CLOEXEC) = 0 18:02:43.523117 connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/xenstored/socket"}, 110) = -1 ECONNREFUSED (Connection refused) 18:02:43.523224 close(3) = 0 18:02:43.523305 stat("/proc/xen/xenbus", {st_mode=S_IFREG|0400, st_size=0, ...}) = 0 18:02:43.523393 open("/proc/xen/xenbus", O_RDWR) = 3 18:02:43.523562 brk(0) = 0x250c000 18:02:43.523638 brk(0x252d000) = 0x252d000 18:02:43.523748 rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER, 0x7fc9518e3200}, {SIG_DFL, [], 0}, 8) = 0 18:02:43.523834 write(3, "\1\0\0\0\0\0\0\0\0\0\0\0\2\0\0\0", 16) = 16 18:02:43.523938 write(3, "/\0", 2 ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-08 17:19 ` Olaf Hering @ 2014-12-08 17:23 ` Wei Liu 2014-12-08 17:37 ` Olaf Hering 0 siblings, 1 reply; 11+ messages in thread From: Wei Liu @ 2014-12-08 17:23 UTC (permalink / raw) To: Olaf Hering Cc: Mark Pryor, Wei Liu, Ian Campbell, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Mon, Dec 08, 2014 at 06:19:49PM +0100, Olaf Hering wrote: > On Fri, Dec 05, Olaf Hering wrote: > > > But even with that change xendomains is hanging if it cant talk to > > xenstored for whatever reason. The result is that the sytem hangs > > forever at shutdown. > > This looks like a bug in the tools. xl or xenstore-ls hang forever if xenstored > disappears. Looks like xs_open() is the culprit. The socket functions fail, > then it attempts to use xs_domain_dev() which happens to succeed. Only > xenstore-ls has the '-s' option to force operating on a socket. I wonder why > libxl does not pass XS_OPEN_SOCKETONLY? It just uses the wrappers for xs_open. > > Can xenstored run in another domain? > I think we have xenstored stubdom in tree, don't we? > Olaf > > 18:02:43.522767 stat("/var/run/xenstored/socket", {st_mode=S_IFSOCK|0600, st_size=0, ...}) = 0 > 18:02:43.522864 socket(PF_LOCAL, SOCK_STREAM, 0) = 3 > 18:02:43.522964 fcntl(3, F_GETFD) = 0 > 18:02:43.523035 fcntl(3, F_SETFD, FD_CLOEXEC) = 0 > 18:02:43.523117 connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/xenstored/socket"}, 110) = -1 ECONNREFUSED (Connection refused) > 18:02:43.523224 close(3) = 0 > 18:02:43.523305 stat("/proc/xen/xenbus", {st_mode=S_IFREG|0400, st_size=0, ...}) = 0 > 18:02:43.523393 open("/proc/xen/xenbus", O_RDWR) = 3 > 18:02:43.523562 brk(0) = 0x250c000 > 18:02:43.523638 brk(0x252d000) = 0x252d000 > 18:02:43.523748 rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER, 0x7fc9518e3200}, {SIG_DFL, [], 0}, 8) = 0 > 18:02:43.523834 write(3, "\1\0\0\0\0\0\0\0\0\0\0\0\2\0\0\0", 16) = 16 > 18:02:43.523938 write(3, "/\0", 2 ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-08 17:23 ` Wei Liu @ 2014-12-08 17:37 ` Olaf Hering 2014-12-08 17:48 ` Wei Liu 0 siblings, 1 reply; 11+ messages in thread From: Olaf Hering @ 2014-12-08 17:37 UTC (permalink / raw) To: Wei Liu Cc: Mark Pryor, Ian Campbell, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Mon, Dec 08, Wei Liu wrote: > I think we have xenstored stubdom in tree, don't we? If we do, isnt that a "special" setup which does not appear all by its own? If so, shouldnt there be some config file knob to let xs_open use socket only unless told otherwise by the to-be-created knob? Olaf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-08 17:37 ` Olaf Hering @ 2014-12-08 17:48 ` Wei Liu 2014-12-09 8:33 ` Olaf Hering 0 siblings, 1 reply; 11+ messages in thread From: Wei Liu @ 2014-12-08 17:48 UTC (permalink / raw) To: Olaf Hering Cc: Mark Pryor, Wei Liu, Ian Campbell, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Mon, Dec 08, 2014 at 06:37:08PM +0100, Olaf Hering wrote: > On Mon, Dec 08, Wei Liu wrote: > > > I think we have xenstored stubdom in tree, don't we? > > If we do, isnt that a "special" setup which does not appear all by > its own? If so, shouldnt there be some config file knob to let xs_open > use socket only unless told otherwise by the to-be-created knob? > At the very least it needs to run init-xenstore-domain to set it up. I have no idea how this supposes to work though. Maybe other people have better idea. However I think the problem you're seeing is due to xenstored exits too early, which should be fixable by rearranging the shutdown order? I.e. xenstored shuts down after xendomains. Wei. > Olaf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-08 17:48 ` Wei Liu @ 2014-12-09 8:33 ` Olaf Hering 2014-12-09 16:10 ` Ian Jackson 0 siblings, 1 reply; 11+ messages in thread From: Olaf Hering @ 2014-12-09 8:33 UTC (permalink / raw) To: Wei Liu Cc: Mark Pryor, Ian Campbell, Stefano Stabellini, Ian Jackson, xen-devel, Xen-users On Mon, Dec 08, Wei Liu wrote: > However I think the problem you're seeing is due to xenstored exits too > early, which should be fixable by rearranging the shutdown order? I.e. > xenstored shuts down after xendomains. True, and this is addressed by 268b8f1. Perhaps a crashed or otherwise unavailable xenstored isnt such a problem because its like that since a very long time. Olaf ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS 2014-12-09 8:33 ` Olaf Hering @ 2014-12-09 16:10 ` Ian Jackson 0 siblings, 0 replies; 11+ messages in thread From: Ian Jackson @ 2014-12-09 16:10 UTC (permalink / raw) To: Olaf Hering Cc: Mark Pryor, Wei Liu, Ian Campbell, Stefano Stabellini, xen-devel, Xen-users Olaf Hering writes ("Re: [Xen-devel] [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS"): > Perhaps a crashed or otherwise unavailable xenstored isnt such a problem > because its like that since a very long time. Without xenstored, the system is basically hosed. And it's not really restartable. Ian. ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2014-12-09 16:10 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <948996368.3219599.1417477301121.JavaMail.yahoo@jws10637.mail.bf1.yahoo.com> 2014-12-02 9:29 ` [Xen-users] 4.5 git: regression in xen systemd shutdown hangs the OS Ian Campbell 2014-12-02 15:17 ` Olaf Hering 2014-12-02 15:39 ` Ian Campbell 2014-12-02 16:06 ` Olaf Hering 2014-12-05 17:11 ` Olaf Hering 2014-12-08 17:19 ` Olaf Hering 2014-12-08 17:23 ` Wei Liu 2014-12-08 17:37 ` Olaf Hering 2014-12-08 17:48 ` Wei Liu 2014-12-09 8:33 ` Olaf Hering 2014-12-09 16:10 ` Ian Jackson
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.