* [xen-unstable test] 35257: regressions - FAIL
@ 2015-02-26 20:14 xen.org
2015-02-27 9:42 ` Ian Campbell
2015-03-03 12:44 ` Ian Campbell
0 siblings, 2 replies; 21+ messages in thread
From: xen.org @ 2015-02-26 20:14 UTC (permalink / raw)
To: xen-devel; +Cc: ian.jackson
flight 35257 xen-unstable real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-armhf-armhf-libvirt 12 guest-start.2 fail REGR. vs. 34629
Regressions which are regarded as allowable (not blocking):
test-amd64-i386-pair 17 guest-migrate/src_host/dst_host fail like 34629
Tests which did not succeed, but are not blocking:
test-armhf-armhf-xl-sedf-pin 10 migrate-support-check fail never pass
test-armhf-armhf-xl 10 migrate-support-check fail never pass
test-armhf-armhf-xl-sedf 10 migrate-support-check fail never pass
test-amd64-amd64-rumpuserxen-amd64 13 rumpuserxen-demo-xenstorels/xenstorels.repeat fail never pass
test-armhf-armhf-xl-credit2 10 migrate-support-check fail never pass
test-armhf-armhf-xl-midway 10 migrate-support-check fail never pass
test-amd64-amd64-xl-pvh-amd 9 guest-start fail never pass
test-armhf-armhf-libvirt 10 migrate-support-check fail never pass
test-amd64-amd64-xl-pvh-intel 9 guest-start fail never pass
test-amd64-i386-libvirt 10 migrate-support-check fail never pass
test-armhf-armhf-xl-multivcpu 10 migrate-support-check fail never pass
test-amd64-amd64-xl-pcipt-intel 9 guest-start fail never pass
test-amd64-amd64-libvirt 10 migrate-support-check fail never pass
test-amd64-i386-xl-qemut-winxpsp3 14 guest-stop fail never pass
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop fail never pass
test-amd64-i386-xl-qemuu-winxpsp3 14 guest-stop fail never pass
test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop fail never pass
test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop fail never pass
test-amd64-amd64-xl-winxpsp3 14 guest-stop fail never pass
test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop fail never pass
test-amd64-amd64-xl-win7-amd64 14 guest-stop fail never pass
test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop fail never pass
test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop fail never pass
test-amd64-i386-xl-win7-amd64 14 guest-stop fail never pass
test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop fail never pass
test-amd64-i386-xl-winxpsp3 14 guest-stop fail never pass
test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop fail never pass
test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop fail never pass
version targeted for testing:
xen 24b2b8dea180098a3acc809a91cde6c0cc4c8607
baseline version:
xen cb34a7c8d741aa447d79e1b01d71168a4088a4d7
------------------------------------------------------------
People who touched revisions under test:
Andrew Cooper <andrew.cooper3@citrix.com>
Dario Faggioli <dario.faggioli@citrix.com>
David Scott <dave.scott@citrix.com>
Don Slutz <dslutz@verizon.com>
Elena Ufimsteva <ufimtseva@gmail.com>
George Dunlap <george.dunlap@eu.citrix.com>
Ian Campbell <ian.campbell@citrix.com>
Ian Jackson <Ian.Jackson@eu.citrix.com>
Jan Beulich <jbeulich@suse.com>
Jintack Lim <jintack@cs.columbia.edu>
Julien Grall <julien.grall@linaro.org>
Michael Young <m.a.young@durham.ac.uk>
Olaf Hering <olaf@aepfle.de>
Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Wei Liu <wei.liu2@citrix.com>
------------------------------------------------------------
jobs:
build-amd64 pass
build-armhf pass
build-i386 pass
build-amd64-libvirt pass
build-armhf-libvirt pass
build-i386-libvirt pass
build-amd64-oldkern pass
build-i386-oldkern pass
build-amd64-pvops pass
build-armhf-pvops pass
build-i386-pvops pass
build-amd64-rumpuserxen pass
build-i386-rumpuserxen pass
test-amd64-amd64-xl pass
test-armhf-armhf-xl pass
test-amd64-i386-xl pass
test-amd64-amd64-xl-pvh-amd fail
test-amd64-i386-rhel6hvm-amd pass
test-amd64-i386-qemut-rhel6hvm-amd pass
test-amd64-i386-qemuu-rhel6hvm-amd pass
test-amd64-amd64-xl-qemut-debianhvm-amd64 pass
test-amd64-i386-xl-qemut-debianhvm-amd64 pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass
test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
test-amd64-i386-freebsd10-amd64 pass
test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
test-amd64-i386-xl-qemuu-ovmf-amd64 pass
test-amd64-amd64-rumpuserxen-amd64 fail
test-amd64-amd64-xl-qemut-win7-amd64 fail
test-amd64-i386-xl-qemut-win7-amd64 fail
test-amd64-amd64-xl-qemuu-win7-amd64 fail
test-amd64-i386-xl-qemuu-win7-amd64 fail
test-amd64-amd64-xl-win7-amd64 fail
test-amd64-i386-xl-win7-amd64 fail
test-amd64-amd64-xl-credit2 pass
test-armhf-armhf-xl-credit2 pass
test-amd64-i386-freebsd10-i386 pass
test-amd64-i386-rumpuserxen-i386 pass
test-amd64-amd64-xl-pcipt-intel fail
test-amd64-amd64-xl-pvh-intel fail
test-amd64-i386-rhel6hvm-intel pass
test-amd64-i386-qemut-rhel6hvm-intel pass
test-amd64-i386-qemuu-rhel6hvm-intel pass
test-amd64-amd64-libvirt pass
test-armhf-armhf-libvirt fail
test-amd64-i386-libvirt pass
test-armhf-armhf-xl-midway pass
test-amd64-amd64-xl-multivcpu pass
test-armhf-armhf-xl-multivcpu pass
test-amd64-amd64-pair pass
test-amd64-i386-pair fail
test-amd64-amd64-xl-sedf-pin pass
test-armhf-armhf-xl-sedf-pin pass
test-amd64-amd64-xl-sedf pass
test-armhf-armhf-xl-sedf pass
test-amd64-i386-xl-qemut-winxpsp3-vcpus1 fail
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 fail
test-amd64-i386-xl-winxpsp3-vcpus1 fail
test-amd64-amd64-xl-qemut-winxpsp3 fail
test-amd64-i386-xl-qemut-winxpsp3 fail
test-amd64-amd64-xl-qemuu-winxpsp3 fail
test-amd64-i386-xl-qemuu-winxpsp3 fail
test-amd64-amd64-xl-winxpsp3 fail
test-amd64-i386-xl-winxpsp3 fail
------------------------------------------------------------
sg-report-flight on osstest.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images
Logs, config files, etc. are available at
http://www.chiark.greenend.org.uk/~xensrcts/logs
Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
Not pushing.
(No revision log; it would be 884 lines long.)
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-02-26 20:14 [xen-unstable test] 35257: regressions - FAIL xen.org
@ 2015-02-27 9:42 ` Ian Campbell
2015-02-27 10:48 ` Wei Liu
2015-02-27 18:51 ` Jim Fehlig
2015-03-03 12:44 ` Ian Campbell
1 sibling, 2 replies; 21+ messages in thread
From: Ian Campbell @ 2015-02-27 9:42 UTC (permalink / raw)
To: xen.org, Jim Fehlig; +Cc: xen-devel
On Thu, 2015-02-26 at 20:14 +0000, xen.org wrote:
> flight 35257 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
> test-armhf-armhf-libvirt 12 guest-start.2 fail REGR. vs. 34629
logs:
http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/info.html
http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/12.ts-guest-start.log
2015-02-23 20:21:48 Z executing ssh ... root@10.80.229.106 virsh domxml-from-native xen-xl /etc/xen/debian.guest.osstest.cfg > /etc/xen/debian.guest.osstest.cfg.xml
error: failed to connect to the hypervisor
error: no valid connection
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4-output-ps_wwwaxf_-eo_pid%2Ctty%2Cstat%2Ctime%2Cnice%2Cpsr%2Cpcpu%2Cpmem%2Cnwchan%2Cwchan%2325%2Cargs
appears to show no libvirtd process.
http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4---var-log-libvirt-libvirtd.log
says:
2015-02-23 20:13:15.556+0000: 2133: info : libvirt version: 1.2.13
2015-02-23 20:13:15.556+0000: 2133: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory
2015-02-23 20:13:15.845+0000: 2133: error : virFirewallValidateBackend:193 : direct firewall backend requested, but /sbin/ebtables is not available: No such file or directory
I think these are just spurious.
2015-02-23 20:13:15.845+0000: 2133: error : virFirewallApply:936 : out of memory
2015-02-23 20:13:16.092+0000: 2133: error : virExec:491 : Cannot find 'pm-is-supported' in path: No such file or directory
2015-02-23 20:13:16.092+0000: 2133: warning : virQEMUCapsInit:999 : Failed to get host power management capabilities
As are these two.
2015-02-23 20:13:16.400+0000: 2133: error : virFirewallApply:936 : out of memory
Has these OOM messages resulted in libvirtd exiting? I don't see any
evidence of a crash elsewhere in the logs (i.e. no "process segfaulted"
in dmesg, no OOM killing going on etc).
We don't seem to collect dom0 freemem info, but that most likely
wouldn't help given the libvirtd process has exited.
Any ideas where to look next?
Ian.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-02-27 9:42 ` Ian Campbell
@ 2015-02-27 10:48 ` Wei Liu
2015-02-27 13:13 ` Ian Campbell
2015-02-27 18:51 ` Jim Fehlig
1 sibling, 1 reply; 21+ messages in thread
From: Wei Liu @ 2015-02-27 10:48 UTC (permalink / raw)
To: Ian Campbell; +Cc: wei.liu2, Jim Fehlig, xen-devel, xen.org
On Fri, Feb 27, 2015 at 09:42:29AM +0000, Ian Campbell wrote:
> On Thu, 2015-02-26 at 20:14 +0000, xen.org wrote:
> > flight 35257 xen-unstable real [real]
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/
> >
> > Regressions :-(
> >
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> > test-armhf-armhf-libvirt 12 guest-start.2 fail REGR. vs. 34629
>
> logs:
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/info.html
>
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/12.ts-guest-start.log
> 2015-02-23 20:21:48 Z executing ssh ... root@10.80.229.106 virsh domxml-from-native xen-xl /etc/xen/debian.guest.osstest.cfg > /etc/xen/debian.guest.osstest.cfg.xml
> error: failed to connect to the hypervisor
> error: no valid connection
> error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
>
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4-output-ps_wwwaxf_-eo_pid%2Ctty%2Cstat%2Ctime%2Cnice%2Cpsr%2Cpcpu%2Cpmem%2Cnwchan%2Cwchan%2325%2Cargs
> appears to show no libvirtd process.
>
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4---var-log-libvirt-libvirtd.log
> says:
> 2015-02-23 20:13:15.556+0000: 2133: info : libvirt version: 1.2.13
> 2015-02-23 20:13:15.556+0000: 2133: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory
> 2015-02-23 20:13:15.845+0000: 2133: error : virFirewallValidateBackend:193 : direct firewall backend requested, but /sbin/ebtables is not available: No such file or directory
>
> I think these are just spurious.
>
> 2015-02-23 20:13:15.845+0000: 2133: error : virFirewallApply:936 : out of memory
>
>
> 2015-02-23 20:13:16.092+0000: 2133: error : virExec:491 : Cannot find 'pm-is-supported' in path: No such file or directory
> 2015-02-23 20:13:16.092+0000: 2133: warning : virQEMUCapsInit:999 : Failed to get host power management capabilities
>
> As are these two.
>
> 2015-02-23 20:13:16.400+0000: 2133: error : virFirewallApply:936 : out of memory
>
Last time Ian and I debugged a libvirt crashing bug, "out of memory"
didn't cause libvirtd to exit. It turned out it's some bug in libxl
event machinery that caused libvirt to exit, but the assertion message
was not shown anywhere.
I think we might need to login to that host and run libvirtd in
foreground to determine what goes wrong.
Wei.
> Has these OOM messages resulted in libvirtd exiting? I don't see any
> evidence of a crash elsewhere in the logs (i.e. no "process segfaulted"
> in dmesg, no OOM killing going on etc).
>
> We don't seem to collect dom0 freemem info, but that most likely
> wouldn't help given the libvirtd process has exited.
>
> Any ideas where to look next?
>
> Ian.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-02-27 10:48 ` Wei Liu
@ 2015-02-27 13:13 ` Ian Campbell
2015-02-27 20:47 ` Jim Fehlig
0 siblings, 1 reply; 21+ messages in thread
From: Ian Campbell @ 2015-02-27 13:13 UTC (permalink / raw)
To: Wei Liu; +Cc: Jim Fehlig, xen-devel, xen.org
On Fri, 2015-02-27 at 10:48 +0000, Wei Liu wrote:
> On Fri, Feb 27, 2015 at 09:42:29AM +0000, Ian Campbell wrote:
> > On Thu, 2015-02-26 at 20:14 +0000, xen.org wrote:
> > > flight 35257 xen-unstable real [real]
> > > http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/
> > >
> > > Regressions :-(
> > >
> > > Tests which did not succeed and are blocking,
> > > including tests which could not be run:
> > > test-armhf-armhf-libvirt 12 guest-start.2 fail REGR. vs. 34629
> >
> > logs:
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/info.html
> >
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/12.ts-guest-start.log
> > 2015-02-23 20:21:48 Z executing ssh ... root@10.80.229.106 virsh domxml-from-native xen-xl /etc/xen/debian.guest.osstest.cfg > /etc/xen/debian.guest.osstest.cfg.xml
> > error: failed to connect to the hypervisor
> > error: no valid connection
> > error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
> >
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4-output-ps_wwwaxf_-eo_pid%2Ctty%2Cstat%2Ctime%2Cnice%2Cpsr%2Cpcpu%2Cpmem%2Cnwchan%2Cwchan%2325%2Cargs
> > appears to show no libvirtd process.
> >
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4---var-log-libvirt-libvirtd.log
> > says:
> > 2015-02-23 20:13:15.556+0000: 2133: info : libvirt version: 1.2.13
> > 2015-02-23 20:13:15.556+0000: 2133: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory
> > 2015-02-23 20:13:15.845+0000: 2133: error : virFirewallValidateBackend:193 : direct firewall backend requested, but /sbin/ebtables is not available: No such file or directory
> >
> > I think these are just spurious.
> >
> > 2015-02-23 20:13:15.845+0000: 2133: error : virFirewallApply:936 : out of memory
> >
> >
> > 2015-02-23 20:13:16.092+0000: 2133: error : virExec:491 : Cannot find 'pm-is-supported' in path: No such file or directory
> > 2015-02-23 20:13:16.092+0000: 2133: warning : virQEMUCapsInit:999 : Failed to get host power management capabilities
> >
> > As are these two.
> >
> > 2015-02-23 20:13:16.400+0000: 2133: error : virFirewallApply:936 : out of memory
> >
>
> Last time Ian and I debugged a libvirt crashing bug, "out of memory"
> didn't cause libvirtd to exit. It turned out it's some bug in libxl
> event machinery that caused libvirt to exit, but the assertion message
> was not shown anywhere.
>
> I think we might need to login to that host and run libvirtd in
> foreground to determine what goes wrong.
That's possible I suppose, but it would be nice to arrange not to have
to in the future.
Perhaps we should be forcing higher log levels on libvirtd when
installing, patching /usr/local/etc/libvirt/libvirtd.conf to set
log_level=2 (or even 1) perhaps? (Default is 3 == warnings+error, 2 is
info, 1 is debug)
Jim, what debug level would you recommend for automated test? Unless it
is super verbose I suppose 1=debug is the way to go?
Adding -v to libvirtd command line would be an easier patch, but only
gives the effect of log_level=2 AFAICT. Perhaps that is considered
sufficient?
Ian.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-02-27 9:42 ` Ian Campbell
2015-02-27 10:48 ` Wei Liu
@ 2015-02-27 18:51 ` Jim Fehlig
2015-02-27 20:20 ` [libvirt] [Xen-devel] " Ian Campbell
1 sibling, 1 reply; 21+ messages in thread
From: Jim Fehlig @ 2015-02-27 18:51 UTC (permalink / raw)
To: Ian Campbell; +Cc: LibVir, xen-devel, xen.org
Ian Campbell wrote:
> On Thu, 2015-02-26 at 20:14 +0000, xen.org wrote:
>
>> flight 35257 xen-unstable real [real]
>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>> test-armhf-armhf-libvirt 12 guest-start.2 fail REGR. vs. 34629
>>
>
> logs:
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/info.html
>
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/12.ts-guest-start.log
> 2015-02-23 20:21:48 Z executing ssh ... root@10.80.229.106 virsh domxml-from-native xen-xl /etc/xen/debian.guest.osstest.cfg > /etc/xen/debian.guest.osstest.cfg.xml
> error: failed to connect to the hypervisor
> error: no valid connection
> error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
>
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4-output-ps_wwwaxf_-eo_pid%2Ctty%2Cstat%2Ctime%2Cnice%2Cpsr%2Cpcpu%2Cpmem%2Cnwchan%2Cwchan%2325%2Cargs
> appears to show no libvirtd process.
>
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4---var-log-libvirt-libvirtd.log
> says:
> 2015-02-23 20:13:15.556+0000: 2133: info : libvirt version: 1.2.13
> 2015-02-23 20:13:15.556+0000: 2133: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory
>
> 2015-02-23 20:13:15.845+0000: 2133: error :
> virFirewallValidateBackend:193 : direct firewall backend requested,
> but /sbin/ebtables is not available: No such file or directory
Odd, since ebtables was found when building
checking for ebtables... /sbin/ebtables
But AFAICT, that wont prevent libvirtd from starting.
>
> I think these are just spurious.
>
> 2015-02-23 20:13:15.845+0000: 2133: error : virFirewallApply:936 : out of memory
>
>
> 2015-02-23 20:13:16.092+0000: 2133: error : virExec:491 : Cannot find 'pm-is-supported' in path: No such file or directory
> 2015-02-23 20:13:16.092+0000: 2133: warning : virQEMUCapsInit:999 : Failed to get host power management capabilities
>
> As are these two.
>
> 2015-02-23 20:13:16.400+0000: 2133: error : virFirewallApply:936 : out of memory
>
> Has these OOM messages resulted in libvirtd exiting?
No, I don't think so. The related code is
int
virFirewallApply(virFirewallPtr firewall)
{
size_t i, j;
int ret = -1;
virMutexLock(&ruleLock);
if (!firewall || firewall->err == ENOMEM) {
virReportOOMError();
goto cleanup;
...
}
I suspect 'firewall' is null, so OOM error is reported and the function
returns -1. But I also don't see this preventing libvirtd from
starting. I've cc'd the libvirt list for verification that these errors
won't prevent libvirtd from starting.
> I don't see any
> evidence of a crash elsewhere in the logs (i.e. no "process segfaulted"
> in dmesg, no OOM killing going on etc).
>
> We don't seem to collect dom0 freemem info, but that most likely
> wouldn't help given the libvirtd process has exited.
>
>
> Any ideas where to look next?
Can you access the test environment and try starting libvirtd in the
foreground? Or enable debug log level in /etc/libvirt/libvirtd.conf?
Regards,
Jim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [libvirt] [Xen-devel] [xen-unstable test] 35257: regressions - FAIL
2015-02-27 18:51 ` Jim Fehlig
@ 2015-02-27 20:20 ` Ian Campbell
0 siblings, 0 replies; 21+ messages in thread
From: Ian Campbell @ 2015-02-27 20:20 UTC (permalink / raw)
To: Jim Fehlig; +Cc: LibVir, xen-devel, xen.org
On Fri, 2015-02-27 at 11:51 -0700, Jim Fehlig wrote:
> > 2015-02-23 20:13:15.845+0000: 2133: error :
> > virFirewallValidateBackend:193 : direct firewall backend requested,
> > but /sbin/ebtables is not available: No such file or directory
>
> Odd, since ebtables was found when building
>
> checking for ebtables... /sbin/ebtables
>
> But AFAICT, that wont prevent libvirtd from starting.
The build host and the runtime host will likely be different (or at
least reinstalled).
The base set of packages should be the same, but the build one will
install a bunch of libfoo-dev while the runtime host will only get
libfoo. Perhaps some libfoo-dev is pulling in ebtables somehow while
just libfoo is not. I'll have a look next week. I think its probably
non-critical to the error here.
> > I think these are just spurious.
> >
> > 2015-02-23 20:13:15.845+0000: 2133: error : virFirewallApply:936 : out of memory
> >
> >
> > 2015-02-23 20:13:16.092+0000: 2133: error : virExec:491 : Cannot find 'pm-is-supported' in path: No such file or directory
> > 2015-02-23 20:13:16.092+0000: 2133: warning : virQEMUCapsInit:999 : Failed to get host power management capabilities
> >
> > As are these two.
> >
> > 2015-02-23 20:13:16.400+0000: 2133: error : virFirewallApply:936 : out of memory
> >
> > Has these OOM messages resulted in libvirtd exiting?
>
> No, I don't think so. The related code is
>
> int
> virFirewallApply(virFirewallPtr firewall)
> {
> size_t i, j;
> int ret = -1;
>
> virMutexLock(&ruleLock);
>
> if (!firewall || firewall->err == ENOMEM) {
> virReportOOMError();
> goto cleanup;
> ...
> }
>
> I suspect 'firewall' is null, so OOM error is reported and the function
> returns -1. But I also don't see this preventing libvirtd from
> starting. I've cc'd the libvirt list for verification that these errors
> won't prevent libvirtd from starting.
I'm pretty sure libvirtd did successfully start, since we have
successfully done a guest start and stop.
The failing step is a second guest start, so it seems like libvirtd has
either crashed or exited.
I suppose these messages are from start of day and therefore
red-herrings wrt the reason libvirtd went away.
> > I don't see any
> > evidence of a crash elsewhere in the logs (i.e. no "process segfaulted"
> > in dmesg, no OOM killing going on etc).
> >
> > We don't seem to collect dom0 freemem info, but that most likely
> > wouldn't help given the libvirtd process has exited.
> >
> >
> > Any ideas where to look next?
>
> Can you access the test environment and try starting libvirtd in the
> foreground? Or enable debug log level in /etc/libvirt/libvirtd.conf?
The test env will have been recycled, I could try and replicate it
manually, but I think to start with I should arrange for the test env to
have more logging enabled, in the hopes that if it happens again we get
more information. I had some question around this in my reply Wei in
this thread at <1425042785.14641.188.camel@citrix.com>.
Cheers,
Ian.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-02-27 13:13 ` Ian Campbell
@ 2015-02-27 20:47 ` Jim Fehlig
2015-03-02 10:04 ` [PATCH] ts-xen-install: Enable debug level logging in libvirt Ian Campbell
` (2 more replies)
0 siblings, 3 replies; 21+ messages in thread
From: Jim Fehlig @ 2015-02-27 20:47 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen.org, xen-devel, Wei Liu
Ian Campbell wrote:
> On Fri, 2015-02-27 at 10:48 +0000, Wei Liu wrote:
>
>> On Fri, Feb 27, 2015 at 09:42:29AM +0000, Ian Campbell wrote:
>>
>>> On Thu, 2015-02-26 at 20:14 +0000, xen.org wrote:
>>>
>>>> flight 35257 xen-unstable real [real]
>>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/
>>>>
>>>> Regressions :-(
>>>>
>>>> Tests which did not succeed and are blocking,
>>>> including tests which could not be run:
>>>> test-armhf-armhf-libvirt 12 guest-start.2 fail REGR. vs. 34629
>>>>
>>> logs:
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/info.html
>>>
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/12.ts-guest-start.log
>>> 2015-02-23 20:21:48 Z executing ssh ... root@10.80.229.106 virsh domxml-from-native xen-xl /etc/xen/debian.guest.osstest.cfg > /etc/xen/debian.guest.osstest.cfg.xml
>>> error: failed to connect to the hypervisor
>>> error: no valid connection
>>> error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
>>>
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4-output-ps_wwwaxf_-eo_pid%2Ctty%2Cstat%2Ctime%2Cnice%2Cpsr%2Cpcpu%2Cpmem%2Cnwchan%2Cwchan%2325%2Cargs
>>> appears to show no libvirtd process.
>>>
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4---var-log-libvirt-libvirtd.log
>>> says:
>>> 2015-02-23 20:13:15.556+0000: 2133: info : libvirt version: 1.2.13
>>> 2015-02-23 20:13:15.556+0000: 2133: error : dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No such file or directory
>>> 2015-02-23 20:13:15.845+0000: 2133: error : virFirewallValidateBackend:193 : direct firewall backend requested, but /sbin/ebtables is not available: No such file or directory
>>>
>>> I think these are just spurious.
>>>
>>> 2015-02-23 20:13:15.845+0000: 2133: error : virFirewallApply:936 : out of memory
>>>
>>>
>>> 2015-02-23 20:13:16.092+0000: 2133: error : virExec:491 : Cannot find 'pm-is-supported' in path: No such file or directory
>>> 2015-02-23 20:13:16.092+0000: 2133: warning : virQEMUCapsInit:999 : Failed to get host power management capabilities
>>>
>>> As are these two.
>>>
>>> 2015-02-23 20:13:16.400+0000: 2133: error : virFirewallApply:936 : out of memory
>>>
>>>
>> Last time Ian and I debugged a libvirt crashing bug, "out of memory"
>> didn't cause libvirtd to exit. It turned out it's some bug in libxl
>> event machinery that caused libvirt to exit, but the assertion message
>> was not shown anywhere.
>>
>> I think we might need to login to that host and run libvirtd in
>> foreground to determine what goes wrong.
>>
>
> That's possible I suppose, but it would be nice to arrange not to have
> to in the future.
>
> Perhaps we should be forcing higher log levels on libvirtd when
> installing, patching /usr/local/etc/libvirt/libvirtd.conf to set
> log_level=2 (or even 1) perhaps? (Default is 3 == warnings+error, 2 is
> info, 1 is debug)
>
> Jim, what debug level would you recommend for automated test? Unless it
>
> is super verbose I suppose 1=debug is the way to go?
I think we need DEBUG log level, although it is rather verbose. If that
becomes a problem, we could experiment with a minimally useful
log_filters setting, e.g.
log_filters="1:daemon 1:libxl"
> Adding -v to libvirtd command line would be an easier patch, but only
> gives the effect of log_level=2 AFAICT. Perhaps that is considered
> sufficient?
>
In my experience, if ERROR is insufficient, INFO and WARNING don't
help. DEBUG is needed.
Regards,
Jim
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH] ts-xen-install: Enable debug level logging in libvirt
2015-02-27 20:47 ` Jim Fehlig
@ 2015-03-02 10:04 ` Ian Campbell
2015-03-02 11:06 ` Ian Jackson
2015-03-02 10:05 ` [xen-unstable test] 35257: regressions - FAIL Ian Campbell
2015-03-02 11:14 ` Ian Jackson
2 siblings, 1 reply; 21+ messages in thread
From: Ian Campbell @ 2015-03-02 10:04 UTC (permalink / raw)
To: ian.jackson; +Cc: Jim Fehlig, Ian Campbell, xen-devel
This should make diagnosing libvirt issues a bit easier.
Refactor the existing loop to edit the xencommons kvp style config
file editing code so it can be reused for libvirtd.conf.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
---
ts-xen-install | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/ts-xen-install b/ts-xen-install
index 7cfe344..a102b0d 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -114,19 +114,26 @@ sub adjustconfig () {
}
die unless defined $trace_config_file;
- target_editfile_root($ho, $trace_config_file, sub {
+ my $kvp_replace = sub($$) {
+ my ($key,$value) = @_;
my $prnow;
$prnow= sub {
- print EO "XENCONSOLED_TRACE=guest\n" or die $!;
+ print EO "$key=$value\n" or die $!;
$prnow= sub { };
};
while (<EI>) {
- print EO or die $! unless m/^XENCONSOLED_TRACE/;
- $prnow->() if m/^#XENCONSOLED_TRACE/;
+ print EO or die $! unless m/^$key/;
+ $prnow->() if m/^#$key/;
}
print EO "\n" or die $!;
$prnow->();
- });
+ };
+
+ target_editfile_root($ho, $trace_config_file,
+ sub { $kvp_replace->("XENCONSOLED_TRACE", "guest") });
+
+ target_editfile_root($ho, '/etc/libvirt/libvirtd.conf',
+ sub { $kvp_replace->("log_level", "1") });
target_cmd_root($ho, 'mkdir -p /var/log/xen/console');
--
2.1.4
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-02-27 20:47 ` Jim Fehlig
2015-03-02 10:04 ` [PATCH] ts-xen-install: Enable debug level logging in libvirt Ian Campbell
@ 2015-03-02 10:05 ` Ian Campbell
2015-03-02 11:14 ` Ian Jackson
2 siblings, 0 replies; 21+ messages in thread
From: Ian Campbell @ 2015-03-02 10:05 UTC (permalink / raw)
To: Jim Fehlig; +Cc: xen.org, xen-devel, Wei Liu
On Fri, 2015-02-27 at 13:47 -0700, Jim Fehlig wrote:
> I think we need DEBUG log level, although it is rather verbose.[...]
> In my experience, if ERROR is insufficient, INFO and WARNING don't
> help. DEBUG is needed.
Thanks, I've just sent out an osstest patch to enable log_level=1.
Ian, I think we should apply that to pretest ASAP to help diagnose the
libvirt issue we are seeing at the moment.
Ian.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH] ts-xen-install: Enable debug level logging in libvirt
2015-03-02 10:04 ` [PATCH] ts-xen-install: Enable debug level logging in libvirt Ian Campbell
@ 2015-03-02 11:06 ` Ian Jackson
2015-03-02 11:33 ` [PATCH v2] " Ian Campbell
2015-03-03 10:22 ` [PATCH v3] " Ian Campbell
0 siblings, 2 replies; 21+ messages in thread
From: Ian Jackson @ 2015-03-02 11:06 UTC (permalink / raw)
To: Ian Campbell; +Cc: Jim Fehlig, xen-devel
Ian Campbell writes ("[PATCH] ts-xen-install: Enable debug level logging in libvirt"):
> This should make diagnosing libvirt issues a bit easier.
If you're generalising this function,
> + print EO or die $! unless m/^$key/;
^ \b ?
?
Ian.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-02-27 20:47 ` Jim Fehlig
2015-03-02 10:04 ` [PATCH] ts-xen-install: Enable debug level logging in libvirt Ian Campbell
2015-03-02 10:05 ` [xen-unstable test] 35257: regressions - FAIL Ian Campbell
@ 2015-03-02 11:14 ` Ian Jackson
2015-03-02 11:38 ` Ian Campbell
2 siblings, 1 reply; 21+ messages in thread
From: Ian Jackson @ 2015-03-02 11:14 UTC (permalink / raw)
To: Jim Fehlig; +Cc: xen.org, xen-devel, Wei Liu, Ian Campbell
Jim Fehlig writes ("Re: [Xen-devel] [xen-unstable test] 35257: regressions - FAIL"):
...
> In my experience, if ERROR is insufficient, INFO and WARNING don't
> help. DEBUG is needed.
Where does virtd's stderr go ? That is where any assertion failure
message would be written.
If we were more clever, osstest would collect corefiles.
Ian.
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2] ts-xen-install: Enable debug level logging in libvirt
2015-03-02 11:06 ` Ian Jackson
@ 2015-03-02 11:33 ` Ian Campbell
2015-03-02 11:37 ` Ian Jackson
2015-03-03 10:22 ` [PATCH v3] " Ian Campbell
1 sibling, 1 reply; 21+ messages in thread
From: Ian Campbell @ 2015-03-02 11:33 UTC (permalink / raw)
To: ian.jackson; +Cc: Jim Fehlig, Ian Campbell, xen-devel
This should make diagnosing libvirt issues a bit easier.
Refactor the existing loop to edit the xencommons kvp style config
file editing code so it can be reused for libvirtd.conf.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
---
v2: Match key\b
---
ts-xen-install | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/ts-xen-install b/ts-xen-install
index 7cfe344..044bcc8 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -114,19 +114,26 @@ sub adjustconfig () {
}
die unless defined $trace_config_file;
- target_editfile_root($ho, $trace_config_file, sub {
+ my $kvp_replace = sub($$) {
+ my ($key,$value) = @_;
my $prnow;
$prnow= sub {
- print EO "XENCONSOLED_TRACE=guest\n" or die $!;
+ print EO "$key=$value\n" or die $!;
$prnow= sub { };
};
while (<EI>) {
- print EO or die $! unless m/^XENCONSOLED_TRACE/;
- $prnow->() if m/^#XENCONSOLED_TRACE/;
+ print EO or die $! unless m/^$key\b/;
+ $prnow->() if m/^#$key/;
}
print EO "\n" or die $!;
$prnow->();
- });
+ };
+
+ target_editfile_root($ho, $trace_config_file,
+ sub { $kvp_replace->("XENCONSOLED_TRACE", "guest") });
+
+ target_editfile_root($ho, '/etc/libvirt/libvirtd.conf',
+ sub { $kvp_replace->("log_level", "1") });
target_cmd_root($ho, 'mkdir -p /var/log/xen/console');
--
2.1.4
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH v2] ts-xen-install: Enable debug level logging in libvirt
2015-03-02 11:33 ` [PATCH v2] " Ian Campbell
@ 2015-03-02 11:37 ` Ian Jackson
2015-03-02 11:45 ` Ian Campbell
0 siblings, 1 reply; 21+ messages in thread
From: Ian Jackson @ 2015-03-02 11:37 UTC (permalink / raw)
To: Ian Campbell; +Cc: Jim Fehlig, xen-devel
Ian Campbell writes ("[PATCH v2] ts-xen-install: Enable debug level logging in libvirt"):
> This should make diagnosing libvirt issues a bit easier.
>
> Refactor the existing loop to edit the xencommons kvp style config
> file editing code so it can be reused for libvirtd.conf.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-03-02 11:14 ` Ian Jackson
@ 2015-03-02 11:38 ` Ian Campbell
2015-03-02 17:59 ` Jim Fehlig
0 siblings, 1 reply; 21+ messages in thread
From: Ian Campbell @ 2015-03-02 11:38 UTC (permalink / raw)
To: Ian Jackson; +Cc: Jim Fehlig, xen-devel, Wei Liu
On Mon, 2015-03-02 at 11:14 +0000, Ian Jackson wrote:
> Jim Fehlig writes ("Re: [Xen-devel] [xen-unstable test] 35257: regressions - FAIL"):
> ...
> > In my experience, if ERROR is insufficient, INFO and WARNING don't
> > help. DEBUG is needed.
>
> Where does virtd's stderr go ? That is where any assertion failure
> message would be written.
/dev/null from the looks of the code:
http://libvirt.org/git/?p=libvirt.git;a=blob;f=daemon/libvirtd.c;h=2366d63cd3c70f75df5af82d2c241df163293eab;hb=HEAD#l174
Ian
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2] ts-xen-install: Enable debug level logging in libvirt
2015-03-02 11:37 ` Ian Jackson
@ 2015-03-02 11:45 ` Ian Campbell
0 siblings, 0 replies; 21+ messages in thread
From: Ian Campbell @ 2015-03-02 11:45 UTC (permalink / raw)
To: Ian Jackson; +Cc: Jim Fehlig, xen-devel
On Mon, 2015-03-02 at 11:37 +0000, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH v2] ts-xen-install: Enable debug level logging in libvirt"):
> > This should make diagnosing libvirt issues a bit easier.
> >
> > Refactor the existing loop to edit the xencommons kvp style config
> > file editing code so it can be reused for libvirtd.conf.
>
> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Thanks. Pushed to pretest.
Ian.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-03-02 11:38 ` Ian Campbell
@ 2015-03-02 17:59 ` Jim Fehlig
2015-03-03 10:37 ` Ian Campbell
0 siblings, 1 reply; 21+ messages in thread
From: Jim Fehlig @ 2015-03-02 17:59 UTC (permalink / raw)
To: Ian Campbell; +Cc: Wei Liu, xen-devel, Ian Jackson
Ian Campbell wrote:
> On Mon, 2015-03-02 at 11:14 +0000, Ian Jackson wrote:
>
>> Jim Fehlig writes ("Re: [Xen-devel] [xen-unstable test] 35257: regressions - FAIL"):
>> ...
>>
>>> In my experience, if ERROR is insufficient, INFO and WARNING don't
>>> help. DEBUG is needed.
>>>
>> Where does virtd's stderr go ? That is where any assertion failure
>> message would be written.
>>
>
> /dev/null from the looks of the code:
> http://libvirt.org/git/?p=libvirt.git;a=blob;f=daemon/libvirtd.c;h=2366d63cd3c70f75df5af82d2c241df163293eab;hb=HEAD#l174
>
When daemonized, yes :-/. You would need to run libvirtd without the
'-d' option to see the assert. Or collect a core as Ian suggested.
Regards,
Jim
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v3] ts-xen-install: Enable debug level logging in libvirt
2015-03-02 11:06 ` Ian Jackson
2015-03-02 11:33 ` [PATCH v2] " Ian Campbell
@ 2015-03-03 10:22 ` Ian Campbell
2015-03-03 11:25 ` Ian Campbell
1 sibling, 1 reply; 21+ messages in thread
From: Ian Campbell @ 2015-03-03 10:22 UTC (permalink / raw)
To: ian.jackson; +Cc: Jim Fehlig, Ian Campbell, xen-devel
This should make diagnosing libvirt issues a bit easier.
Refactor the existing loop to edit the xencommons kvp style config
file editing code so it can be reused for libvirtd.conf.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v3: Only apply if toolstack==libvirt, else we break non-libvirt
tests. Now tested with test-amd64-amd64-xl as well as -libvirt.
v2: Match key\b
---
ts-xen-install | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/ts-xen-install b/ts-xen-install
index 7cfe344..5282f0a 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -114,19 +114,27 @@ sub adjustconfig () {
}
die unless defined $trace_config_file;
- target_editfile_root($ho, $trace_config_file, sub {
+ my $kvp_replace = sub($$) {
+ my ($key,$value) = @_;
my $prnow;
$prnow= sub {
- print EO "XENCONSOLED_TRACE=guest\n" or die $!;
+ print EO "$key=$value\n" or die $!;
$prnow= sub { };
};
while (<EI>) {
- print EO or die $! unless m/^XENCONSOLED_TRACE/;
- $prnow->() if m/^#XENCONSOLED_TRACE/;
+ print EO or die $! unless m/^$key\b/;
+ $prnow->() if m/^#$key/;
}
print EO "\n" or die $!;
$prnow->();
- });
+ };
+
+ target_editfile_root($ho, $trace_config_file,
+ sub { $kvp_replace->("XENCONSOLED_TRACE", "guest") });
+
+ target_editfile_root($ho, '/etc/libvirt/libvirtd.conf',
+ sub { $kvp_replace->("log_level", "1") })
+ if toolstack($ho)->{Name} eq "libvirt";
target_cmd_root($ho, 'mkdir -p /var/log/xen/console');
--
2.1.4
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-03-02 17:59 ` Jim Fehlig
@ 2015-03-03 10:37 ` Ian Campbell
0 siblings, 0 replies; 21+ messages in thread
From: Ian Campbell @ 2015-03-03 10:37 UTC (permalink / raw)
To: Jim Fehlig; +Cc: Wei Liu, xen-devel, Ian Jackson
On Mon, 2015-03-02 at 10:59 -0700, Jim Fehlig wrote:
> Ian Campbell wrote:
> > On Mon, 2015-03-02 at 11:14 +0000, Ian Jackson wrote:
> >
> >> Jim Fehlig writes ("Re: [Xen-devel] [xen-unstable test] 35257: regressions - FAIL"):
> >> ...
> >>
> >>> In my experience, if ERROR is insufficient, INFO and WARNING don't
> >>> help. DEBUG is needed.
> >>>
> >> Where does virtd's stderr go ? That is where any assertion failure
> >> message would be written.
> >>
> >
> > /dev/null from the looks of the code:
> > http://libvirt.org/git/?p=libvirt.git;a=blob;f=daemon/libvirtd.c;h=2366d63cd3c70f75df5af82d2c241df163293eab;hb=HEAD#l174
> >
>
> When daemonized, yes :-/. You would need to run libvirtd without the
> '-d' option to see the assert.
Which would run in the foreground, I think? So we would need to handle
daemonisation some other way? Probably best to avoid that I think.
> Or collect a core as Ian suggested.
This would seem to be a generically useful thing to do, as well as
helping here.
I think it could be achieved by ts-host-install editing /etc/sysctl.conf
to set sys.kernel.core_pattern to something (perhaps a new dir
under /var/). http://man7.org/linux/man-pages/man5/core.5.html describes
the substitutions and says it is there since 2.4.21 and 2.6.
Some sort of ulimit frobbing might also be needed I suppose.
Then ts-logs-capture would pick up anything which appeared in the
directory which is in core_pattern. Or perhaps a new test step
ts-core-check could be added which fails the test if anything is found?
Ian, if you think this makes sense I could add to my TODO.
Ian.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v3] ts-xen-install: Enable debug level logging in libvirt
2015-03-03 10:22 ` [PATCH v3] " Ian Campbell
@ 2015-03-03 11:25 ` Ian Campbell
0 siblings, 0 replies; 21+ messages in thread
From: Ian Campbell @ 2015-03-03 11:25 UTC (permalink / raw)
To: ian.jackson; +Cc: Jim Fehlig, xen-devel
On Tue, 2015-03-03 at 10:22 +0000, Ian Campbell wrote:
> This should make diagnosing libvirt issues a bit easier.
>
> Refactor the existing loop to edit the xencommons kvp style config
> file editing code so it can be reused for libvirtd.conf.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: Jim Fehlig <jfehlig@suse.com>
> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Pushed to pretest as requested on IRC.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-02-26 20:14 [xen-unstable test] 35257: regressions - FAIL xen.org
2015-02-27 9:42 ` Ian Campbell
@ 2015-03-03 12:44 ` Ian Campbell
2015-03-03 12:46 ` Ian Campbell
1 sibling, 1 reply; 21+ messages in thread
From: Ian Campbell @ 2015-03-03 12:44 UTC (permalink / raw)
To: xen.org; +Cc: xen-devel
On Thu, 2015-02-26 at 20:14 +0000, xen.org wrote:
> flight 35257 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
> test-armhf-armhf-libvirt 12 guest-start.2 fail REGR. vs. 34629
libvirt is unreliable right now, Ian already added to allow.all:
@@-libvirt@@ guest-start
Ian is going to push a change to the allow file to make at least libvirt
guest-state.* and guest-destroy allowable fails but in the meantime:
[...]
> version targeted for testing:
> xen 24b2b8dea180098a3acc809a91cde6c0cc4c8607
I have force pushed this revision.
Ian.
osstest@osstest:~/branches/for-xen-unstable.git$ OSSTEST_CONFIG=production-config ./ap-push 24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ branch=24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ revision=
+ . cri-lock-repos
++ . cri-common
+++ . cri-getconfig
+++ umask 002
+++ getconfig Repos
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
++ repos=/export/home/osstest/repos
++ repos_lock=/export/home/osstest/repos/lock
++ '[' x '!=' x/export/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/export/home/osstest/repos/lock
++ exec with-lock-ex -w /export/home/osstest/repos/lock ./ap-push 24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ branch=24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ revision=
+ . cri-lock-repos
++ . cri-common
+++ . cri-getconfig
+++ umask 002
+++ getconfig Repos
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
++ repos=/export/home/osstest/repos
++ repos_lock=/export/home/osstest/repos/lock
++ '[' x/export/home/osstest/repos/lock '!=' x/export/home/osstest/repos/lock ']'
+ . cri-common
++ . cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ '[' x = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-
+ : tested/2.6.39.x
+ . ap-common
++ : osstest@xenbits.xensource.com
++ : git://xenbits.xen.org/xen.git
++ : osstest@xenbits.xensource.com:/home/xen/git/xen.git
++ : git://xenbits.xen.org/staging/qemu-.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://libvirt.org/libvirt.git
++ : osstest@xenbits.xensource.com:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : https://github.com/rumpkernel/rumprun-xen
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osstest@xenbits.xensource.com:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src '[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
++++ getconfig GitCacheProxy
++++ perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://drall.uk.xensource.com:9419/
+++ '[' xgit://drall.uk.xensource.com:9419/ '!=' x ']'
+++ echo 'git://drall.uk.xensource.com:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=try]'
++ : 'git://drall.uk.xensource.com:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=try]'
++ : git
++ : git://git.seabios.org/seabios.git
++ : osstest@xenbits.xensource.com:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osstest@xenbits.xensource.com:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osstest@xenbits.xensource.com:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : osstest@xenbits.xensource.com:/home/xen/git/linux-pvops.git
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ : tested/linux-arm-xen
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-arm-xen
++ : git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
++ : tested/2.6.39.x
++ : daily-cron.24b2b8dea180098a3acc809a91cde6c0cc4c8607
++ : daily-cron.24b2b8dea180098a3acc809a91cde6c0cc4c8607
++ : daily-cron.24b2b8dea180098a3acc809a91cde6c0cc4c8607
++ : daily-cron.24b2b8dea180098a3acc809a91cde6c0cc4c8607
++ : daily-cron.24b2b8dea180098a3acc809a91cde6c0cc4c8607
++ : daily-cron.24b2b8dea180098a3acc809a91cde6c0cc4c8607
++ : http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27
++ : git://xenbits.xen.org/staging/qemu-upstream-.git
++ : daily-cron.24b2b8dea180098a3acc809a91cde6c0cc4c8607
++ : git://git.qemu.org/qemu.git
++ : git://xenbits.xen.org/osstest/qemu.git
++ : osstest@xenbits.xensource.com:/home/xen/git/osstest/qemu.git
++ : daily-cron.24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ TREE_LINUX=osstest@xenbits.xensource.com:/home/xen/git/linux-pvops.git
+ TREE_QEMU_MAINLINE=osstest@xenbits.xensource.com:/home/xen/git/osstest/qemu.git
+ TREE_QEMU_UPSTREAM=osstest@xenbits.xensource.com:/home/xen/git/qemu-upstream-.git
+ TREE_XEN=osstest@xenbits.xensource.com:/home/xen/git/xen.git
+ TREE_LIBVIRT=osstest@xenbits.xensource.com:/home/xen/git/libvirt.git
+ TREE_RUMPUSERXEN=osstest@xenbits.xensource.com:/home/xen/git/rumpuser-xen.git
+ TREE_SEABIOS=osstest@xenbits.xensource.com:/home/xen/git/osstest/seabios.git
+ TREE_OVMF=osstest@xenbits.xensource.com:/home/xen/git/osstest/ovmf.git
+ info_linux_tree 24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ case $1 in
+ return 1
+ case "$branch" in
+ echo 'branch 24b2b8dea180098a3acc809a91cde6c0cc4c8607 ?'
branch 24b2b8dea180098a3acc809a91cde6c0cc4c8607 ?
+ exit 1
osstest@osstest:~/branches/for-xen-unstable.git$
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [xen-unstable test] 35257: regressions - FAIL
2015-03-03 12:44 ` Ian Campbell
@ 2015-03-03 12:46 ` Ian Campbell
0 siblings, 0 replies; 21+ messages in thread
From: Ian Campbell @ 2015-03-03 12:46 UTC (permalink / raw)
To: xen.org; +Cc: xen-devel
On Tue, 2015-03-03 at 12:44 +0000, Ian Campbell wrote:
> On Thu, 2015-02-26 at 20:14 +0000, xen.org wrote:
> > version targeted for testing:
> > xen 24b2b8dea180098a3acc809a91cde6c0cc4c8607
>
> I have force pushed this revision.
[...]
> + echo 'branch 24b2b8dea180098a3acc809a91cde6c0cc4c8607 ?'
> branch 24b2b8dea180098a3acc809a91cde6c0cc4c8607 ?
> + exit 1
Ahem. Lets try that again.
osstest@osstest:~/branches/for-xen-unstable.git$ OSSTEST_CONFIG=production-config ./ap-push xen-unstable 24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ branch=xen-unstable
+ revision=24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ . cri-lock-repos
++ . cri-common
+++ . cri-getconfig
+++ umask 002
+++ getconfig Repos
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
++ repos=/export/home/osstest/repos
++ repos_lock=/export/home/osstest/repos/lock
++ '[' x '!=' x/export/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/export/home/osstest/repos/lock
++ exec with-lock-ex -w /export/home/osstest/repos/lock ./ap-push xen-unstable 24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ branch=xen-unstable
+ revision=24b2b8dea180098a3acc809a91cde6c0cc4c8607
+ . cri-lock-repos
++ . cri-common
+++ . cri-getconfig
+++ umask 002
+++ getconfig Repos
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
++ repos=/export/home/osstest/repos
++ repos_lock=/export/home/osstest/repos/lock
++ '[' x/export/home/osstest/repos/lock '!=' x/export/home/osstest/repos/lock ']'
+ . cri-common
++ . cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ : tested/2.6.39.x
+ . ap-common
++ : osstest@xenbits.xensource.com
++ : git://xenbits.xen.org/xen.git
++ : osstest@xenbits.xensource.com:/home/xen/git/xen.git
++ : git://xenbits.xen.org/staging/qemu-xen-unstable.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://libvirt.org/libvirt.git
++ : osstest@xenbits.xensource.com:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : https://github.com/rumpkernel/rumprun-xen
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osstest@xenbits.xensource.com:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src '[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
++++ getconfig GitCacheProxy
++++ perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://drall.uk.xensource.com:9419/
+++ '[' xgit://drall.uk.xensource.com:9419/ '!=' x ']'
+++ echo 'git://drall.uk.xensource.com:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=try]'
++ : 'git://drall.uk.xensource.com:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=try]'
++ : git
++ : git://git.seabios.org/seabios.git
++ : osstest@xenbits.xensource.com:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : osstest@xenbits.xensource.com:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : osstest@xenbits.xensource.com:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : osstest@xenbits.xensource.com:/home/xen/git/linux-pvops.git
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ : tested/linux-arm-xen
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-arm-xen
++ : git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
++ : tested/2.6.39.x
++ : daily-cron.xen-unstable
++ : daily-cron.xen-unstable
++ : daily-cron.xen-unstable
++ : daily-cron.xen-unstable
++ : daily-cron.xen-unstable
++ : daily-cron.xen-unstable
++ : http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27
++ : git://xenbits.xen.org/staging/qemu-upstream-unstable.git
++ : daily-cron.xen-unstable
++ : git://git.qemu.org/qemu.git
++ : git://xenbits.xen.org/osstest/qemu.git
++ : osstest@xenbits.xensource.com:/home/xen/git/osstest/qemu.git
++ : daily-cron.xen-unstable
+ TREE_LINUX=osstest@xenbits.xensource.com:/home/xen/git/linux-pvops.git
+ TREE_QEMU_MAINLINE=osstest@xenbits.xensource.com:/home/xen/git/osstest/qemu.git
+ TREE_QEMU_UPSTREAM=osstest@xenbits.xensource.com:/home/xen/git/qemu-upstream-unstable.git
+ TREE_XEN=osstest@xenbits.xensource.com:/home/xen/git/xen.git
+ TREE_LIBVIRT=osstest@xenbits.xensource.com:/home/xen/git/libvirt.git
+ TREE_RUMPUSERXEN=osstest@xenbits.xensource.com:/home/xen/git/rumpuser-xen.git
+ TREE_SEABIOS=osstest@xenbits.xensource.com:/home/xen/git/osstest/seabios.git
+ TREE_OVMF=osstest@xenbits.xensource.com:/home/xen/git/osstest/ovmf.git
+ info_linux_tree xen-unstable
+ case $1 in
+ return 1
+ case "$branch" in
+ cd /export/home/osstest/repos/xen
+ git push osstest@xenbits.xensource.com:/home/xen/git/xen.git 24b2b8dea180098a3acc809a91cde6c0cc4c8607:master
Total 0 (delta 0), reused 0 (delta 0)
To osstest@xenbits.xensource.com:/home/xen/git/xen.git
cb34a7c..24b2b8d 24b2b8dea180098a3acc809a91cde6c0cc4c8607 -> master
osstest@osstest:~/branches/for-xen-unstable.git$
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2015-03-03 12:46 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-02-26 20:14 [xen-unstable test] 35257: regressions - FAIL xen.org
2015-02-27 9:42 ` Ian Campbell
2015-02-27 10:48 ` Wei Liu
2015-02-27 13:13 ` Ian Campbell
2015-02-27 20:47 ` Jim Fehlig
2015-03-02 10:04 ` [PATCH] ts-xen-install: Enable debug level logging in libvirt Ian Campbell
2015-03-02 11:06 ` Ian Jackson
2015-03-02 11:33 ` [PATCH v2] " Ian Campbell
2015-03-02 11:37 ` Ian Jackson
2015-03-02 11:45 ` Ian Campbell
2015-03-03 10:22 ` [PATCH v3] " Ian Campbell
2015-03-03 11:25 ` Ian Campbell
2015-03-02 10:05 ` [xen-unstable test] 35257: regressions - FAIL Ian Campbell
2015-03-02 11:14 ` Ian Jackson
2015-03-02 11:38 ` Ian Campbell
2015-03-02 17:59 ` Jim Fehlig
2015-03-03 10:37 ` Ian Campbell
2015-02-27 18:51 ` Jim Fehlig
2015-02-27 20:20 ` [libvirt] [Xen-devel] " Ian Campbell
2015-03-03 12:44 ` Ian Campbell
2015-03-03 12:46 ` Ian Campbell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.