* [xen-unstable test] 108068: regressions - FAIL
@ 2017-05-01 18:49 osstest service owner
2017-05-02 9:10 ` Jan Beulich
0 siblings, 1 reply; 25+ messages in thread
From: osstest service owner @ 2017-05-01 18:49 UTC (permalink / raw)
To: xen-devel, osstest-admin
flight 108068 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/108068/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-i386-xl-qemut-winxpsp3-vcpus1 17 guest-start/win.repeat fail REGR. vs. 107900
Tests which are failing intermittently (not blocking):
test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 16 guest-stop fail in 108038 pass in 108068
test-amd64-amd64-i386-pvgrub 9 debian-di-install fail in 108038 pass in 108068
test-amd64-i386-rumprun-i386 16 rumprun-demo-xenstorels/xenstorels.repeat fail pass in 108038
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 17 guest-start/win.repeat fail pass in 108038
test-amd64-amd64-xl-qemut-winxpsp3 17 guest-start/win.repeat fail pass in 108038
Tests which did not succeed, but are not blocking:
test-amd64-i386-xl-qemuu-win7-amd64 17 guest-start/win.repeat fail in 108038 blocked in 107900
test-amd64-i386-xl-qemut-win7-amd64 15 guest-localmigrate/x10 fail in 108038 like 107900
test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 107791
test-amd64-amd64-xl-qemuu-win7-amd64 15 guest-localmigrate/x10 fail like 107840
test-amd64-amd64-xl-qemut-win7-amd64 15 guest-localmigrate/x10 fail like 107900
test-armhf-armhf-libvirt 13 saverestore-support-check fail like 107900
test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 107900
test-armhf-armhf-libvirt-xsm 13 saverestore-support-check fail like 107900
test-amd64-amd64-xl-rtds 9 debian-install fail like 107900
test-armhf-armhf-libvirt-raw 12 saverestore-support-check fail like 107900
test-amd64-i386-libvirt-xsm 12 migrate-support-check fail never pass
test-amd64-i386-libvirt 12 migrate-support-check fail never pass
test-amd64-amd64-libvirt 12 migrate-support-check fail never pass
test-amd64-amd64-libvirt-xsm 12 migrate-support-check fail never pass
test-arm64-arm64-xl-xsm 12 migrate-support-check fail never pass
test-arm64-arm64-xl-xsm 13 saverestore-support-check fail never pass
test-arm64-arm64-xl-multivcpu 12 migrate-support-check fail never pass
test-arm64-arm64-xl-multivcpu 13 saverestore-support-check fail never pass
test-arm64-arm64-xl-credit2 12 migrate-support-check fail never pass
test-arm64-arm64-xl-credit2 13 saverestore-support-check fail never pass
test-arm64-arm64-libvirt 12 migrate-support-check fail never pass
test-arm64-arm64-libvirt-xsm 12 migrate-support-check fail never pass
test-arm64-arm64-libvirt 13 saverestore-support-check fail never pass
test-arm64-arm64-libvirt-xsm 13 saverestore-support-check fail never pass
test-arm64-arm64-xl 12 migrate-support-check fail never pass
test-arm64-arm64-xl 13 saverestore-support-check fail never pass
test-arm64-arm64-xl-rtds 12 migrate-support-check fail never pass
test-arm64-arm64-xl-rtds 13 saverestore-support-check fail never pass
test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass
test-arm64-arm64-libvirt-qcow2 11 migrate-support-check fail never pass
test-arm64-arm64-libvirt-qcow2 12 saverestore-support-check fail never pass
test-amd64-amd64-libvirt-vhd 11 migrate-support-check fail never pass
test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2 fail never pass
test-armhf-armhf-xl-credit2 12 migrate-support-check fail never pass
test-armhf-armhf-xl-credit2 13 saverestore-support-check fail never pass
test-armhf-armhf-xl-cubietruck 12 migrate-support-check fail never pass
test-armhf-armhf-xl-multivcpu 12 migrate-support-check fail never pass
test-armhf-armhf-xl-multivcpu 13 saverestore-support-check fail never pass
test-armhf-armhf-xl-cubietruck 13 saverestore-support-check fail never pass
test-armhf-armhf-xl 12 migrate-support-check fail never pass
test-armhf-armhf-xl 13 saverestore-support-check fail never pass
test-armhf-armhf-xl-xsm 12 migrate-support-check fail never pass
test-armhf-armhf-xl-xsm 13 saverestore-support-check fail never pass
test-armhf-armhf-xl-arndale 12 migrate-support-check fail never pass
test-armhf-armhf-xl-arndale 13 saverestore-support-check fail never pass
test-armhf-armhf-libvirt 12 migrate-support-check fail never pass
test-armhf-armhf-libvirt-xsm 12 migrate-support-check fail never pass
test-armhf-armhf-xl-rtds 12 migrate-support-check fail never pass
test-armhf-armhf-xl-rtds 13 saverestore-support-check fail never pass
test-armhf-armhf-libvirt-raw 11 migrate-support-check fail never pass
test-armhf-armhf-xl-vhd 11 migrate-support-check fail never pass
test-armhf-armhf-xl-vhd 12 saverestore-support-check fail never pass
version targeted for testing:
xen ba39e9b2108319d2b7b842781106386b8ed62fab
baseline version:
xen 0a5370ee1f9808fbb16bb03d7f349921cf73a2d4
Last test of basis 107900 2017-04-28 14:06:22 Z 3 days
Testing same since 107940 2017-04-29 06:51:56 Z 2 days 4 attempts
------------------------------------------------------------
People who touched revisions under test:
Andrew Cooper <andrew.cooper3@citrix.com>
Boris Ostrovsky <boris.ostrovsky@oracle.com>
Jan Beulich <jbeulich@suse.com>
jobs:
build-amd64-xsm pass
build-arm64-xsm pass
build-armhf-xsm pass
build-i386-xsm pass
build-amd64-xtf pass
build-amd64 pass
build-arm64 pass
build-armhf pass
build-i386 pass
build-amd64-libvirt pass
build-arm64-libvirt pass
build-armhf-libvirt pass
build-i386-libvirt pass
build-amd64-oldkern pass
build-i386-oldkern pass
build-amd64-prev pass
build-i386-prev pass
build-amd64-pvops pass
build-arm64-pvops pass
build-armhf-pvops pass
build-i386-pvops pass
build-amd64-rumprun pass
build-i386-rumprun pass
test-xtf-amd64-amd64-1 pass
test-xtf-amd64-amd64-2 pass
test-xtf-amd64-amd64-3 pass
test-xtf-amd64-amd64-4 pass
test-xtf-amd64-amd64-5 pass
test-amd64-amd64-xl pass
test-arm64-arm64-xl pass
test-armhf-armhf-xl pass
test-amd64-i386-xl pass
test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm pass
test-amd64-i386-xl-qemut-debianhvm-amd64-xsm pass
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm pass
test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm pass
test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm pass
test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm pass
test-amd64-amd64-libvirt-xsm pass
test-arm64-arm64-libvirt-xsm pass
test-armhf-armhf-libvirt-xsm pass
test-amd64-i386-libvirt-xsm pass
test-amd64-amd64-xl-xsm pass
test-arm64-arm64-xl-xsm pass
test-armhf-armhf-xl-xsm pass
test-amd64-i386-xl-xsm pass
test-amd64-amd64-qemuu-nested-amd fail
test-amd64-amd64-xl-pvh-amd pass
test-amd64-i386-qemut-rhel6hvm-amd pass
test-amd64-i386-qemuu-rhel6hvm-amd pass
test-amd64-amd64-xl-qemut-debianhvm-amd64 pass
test-amd64-i386-xl-qemut-debianhvm-amd64 pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass
test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
test-amd64-i386-freebsd10-amd64 pass
test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
test-amd64-i386-xl-qemuu-ovmf-amd64 pass
test-amd64-amd64-rumprun-amd64 pass
test-amd64-amd64-xl-qemut-win7-amd64 fail
test-amd64-i386-xl-qemut-win7-amd64 fail
test-amd64-amd64-xl-qemuu-win7-amd64 fail
test-amd64-i386-xl-qemuu-win7-amd64 fail
test-armhf-armhf-xl-arndale pass
test-amd64-amd64-xl-credit2 pass
test-arm64-arm64-xl-credit2 pass
test-armhf-armhf-xl-credit2 pass
test-armhf-armhf-xl-cubietruck pass
test-amd64-i386-freebsd10-i386 pass
test-amd64-i386-rumprun-i386 fail
test-amd64-amd64-qemuu-nested-intel pass
test-amd64-amd64-xl-pvh-intel pass
test-amd64-i386-qemut-rhel6hvm-intel pass
test-amd64-i386-qemuu-rhel6hvm-intel pass
test-amd64-amd64-libvirt pass
test-arm64-arm64-libvirt pass
test-armhf-armhf-libvirt pass
test-amd64-i386-libvirt pass
test-amd64-amd64-migrupgrade pass
test-amd64-i386-migrupgrade pass
test-amd64-amd64-xl-multivcpu pass
test-arm64-arm64-xl-multivcpu pass
test-armhf-armhf-xl-multivcpu pass
test-amd64-amd64-pair pass
test-amd64-i386-pair pass
test-amd64-amd64-libvirt-pair pass
test-amd64-i386-libvirt-pair pass
test-amd64-amd64-amd64-pvgrub pass
test-amd64-amd64-i386-pvgrub pass
test-amd64-amd64-pygrub pass
test-arm64-arm64-libvirt-qcow2 pass
test-amd64-amd64-xl-qcow2 pass
test-armhf-armhf-libvirt-raw pass
test-amd64-i386-xl-raw pass
test-amd64-amd64-xl-rtds fail
test-arm64-arm64-xl-rtds pass
test-armhf-armhf-xl-rtds pass
test-amd64-i386-xl-qemut-winxpsp3-vcpus1 fail
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 fail
test-amd64-amd64-libvirt-vhd pass
test-armhf-armhf-xl-vhd pass
test-amd64-amd64-xl-qemut-winxpsp3 fail
test-amd64-i386-xl-qemut-winxpsp3 pass
test-amd64-amd64-xl-qemuu-winxpsp3 pass
test-amd64-i386-xl-qemuu-winxpsp3 pass
------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images
Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs
Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
Not pushing.
------------------------------------------------------------
commit ba39e9b2108319d2b7b842781106386b8ed62fab
Author: Jan Beulich <jbeulich@suse.com>
Date: Fri Apr 28 16:03:40 2017 +0200
x86emul: correct stub invocation constraints again
While the hypervisor side of commit cd91ab08ea ("x86emul: correct stub
invocation constraints") was fine, the tools side triggered a bogus
error with old gcc (4.3 and 4.4 at least). Use a slightly less
appropriate variant instead, proven to be good enough to not
re-introduce the original problem: Which of the addresses is actually
used doesn't matter much as long as the compiler can't prove that the
two pointers don't alias one another.
Reported-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Release-acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
(qemu changes not included)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-01 18:49 [xen-unstable test] 108068: regressions - FAIL osstest service owner
@ 2017-05-02 9:10 ` Jan Beulich
2017-05-02 12:45 ` Ian Jackson
0 siblings, 1 reply; 25+ messages in thread
From: Jan Beulich @ 2017-05-02 9:10 UTC (permalink / raw)
To: osstest-admin; +Cc: xen-devel
>>> On 01.05.17 at 20:49, <osstest-admin@xenproject.org> wrote:
> flight 108068 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/108068/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
> test-amd64-i386-xl-qemut-winxpsp3-vcpus1 17 guest-start/win.repeat fail REGR. vs. 107900
This has been recurring for the last few flights, but I wonder whether
2017-05-01 13:18:52 Z executing ssh ... root@172.16.144.40 readlink /dev/italia0-vg/win.guest.osstest-disk
2017-05-01 13:18:52 Z executing ssh ... root@172.16.144.40 lvdisplay --colon /dev/italia0-vg/win.guest.osstest-disk
2017-05-01 13:18:53 Z lvdisplay output says device is still open: /dev/italia0-vg/win.guest.osstest-disk:italia0-vg:3:1:-1:2:20480000:2500:-1:0:-1:253:2
2017-05-01 13:18:53 Z executing ssh ... root@172.16.144.40 umount /dev/italia0-vg/win.guest.osstest-disk
umount: /dev/italia0-vg/win.guest.osstest-disk: not mounted
2017-05-01 13:18:53 Z command nonzero waitstatus 8192: timeout 60 ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_108068.test-amd64-i386-xl-qemut-winxpsp3-vcpus1 root@172.16.144.40 umount /dev/italia0-vg/win.guest.osstest-disk
status 8192 at Osstest/TestSupport.pm line 442.
indicates an environmental problem rather than a
software-under-test one (the more that the single commit
being tested can't possibly influence host or guest behavior).
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 9:10 ` Jan Beulich
@ 2017-05-02 12:45 ` Ian Jackson
2017-05-02 13:27 ` Andrew Cooper
` (2 more replies)
0 siblings, 3 replies; 25+ messages in thread
From: Ian Jackson @ 2017-05-02 12:45 UTC (permalink / raw)
To: Jan Beulich; +Cc: xen-devel, Wei Liu
Jan Beulich writes ("Re: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> On 01.05.17 at 20:49, <osstest-admin@xenproject.org> wrote:
> This has been recurring for the last few flights, but I wonder whether
>
> 2017-05-01 13:18:52 Z executing ssh ... root@172.16.144.40 readlink /dev/italia0-vg/win.guest.osstest-disk
> 2017-05-01 13:18:52 Z executing ssh ... root@172.16.144.40 lvdisplay --colon /dev/italia0-vg/win.guest.osstest-disk
> 2017-05-01 13:18:53 Z lvdisplay output says device is still open: /dev/italia0-vg/win.guest.osstest-disk:italia0-vg:3:1:-1:2:20480000:2500:-1:0:-1:253:2
> 2017-05-01 13:18:53 Z executing ssh ... root@172.16.144.40 umount /dev/italia0-vg/win.guest.osstest-disk
> umount: /dev/italia0-vg/win.guest.osstest-disk: not mounted
> 2017-05-01 13:18:53 Z command nonzero waitstatus 8192: timeout 60 ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_108068.test-amd64-i386-xl-qemut-winxpsp3-vcpus1 root@172.16.144.40 umount /dev/italia0-vg/win.guest.osstest-disk
> status 8192 at Osstest/TestSupport.pm line 442.
>
> indicates an environmental problem rather than a
> software-under-test one (the more that the single commit
> being tested can't possibly influence host or guest behavior).
This is almost certainly not an environmental problem. What seems to
be happening is that the guest shutdown/teardown is going wrong
somehow.
http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/16.ts-guest-stop.log
shows this:
2017-05-01 13:18:27 Z executing ssh ... root@172.16.144.40 xl shutdown -wF win.guest.osstest
Shutting down domain 17
PV control interface not available: sending ACPI power button event.
Waiting for 1 domains
Domain 17 has been shut down, reason code 1
2017-05-01 13:18:36 Z executing ssh ... root@172.16.144.40 xl list
2017-05-01 13:18:36 Z guest win.guest.osstest state is psr
So the guest has been shut down in the sense that xl shutdown -w
has exited (-w means to wait for the shutdown), but not in the sense
that the domain has been destroyed.
osstest spends 14 seconds checking that the guest doesn't respond to
ping (this is probably a bit pointless, TBH):
2017-05-01 13:18:50 Z ping 172.16.146.243 down
Then the next step tries to start the guest. But it finds that the
backing block device is in use. The command that fails is there so
that this test script can be re-run in certain ad-hoc by-hand tests:
it is trying to unmount the block device, on the theory that if it is
shown as open in LVM, that is probably because it's mounted. The
unmount fails.
The underlying problem is that the block backend still has the guest
block device open. Indeed, during the logs capture we see
http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0-output-xl_list
the guest is still there:
Name ID Mem VCPUs State Time(s)
Domain-0 0 511 4 r----- 913.9
win.guest.osstest 18 1536 1 r----- 16.0
(that's at 2017-05-01 13:18:56)
I think the guest that was shut down was domid 17 and this new one is
domid 18. This logfile
http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0---var-log-xen-xl-win.guest.osstest--incoming.log
shows domid 17 shutting down and then this message
Done. Rebooting now
and then it seems to start the domain again.
Is it possible that something has changed which means that Windows
(sometimes?) doesn't respond to an ACPI power button event by shutting
down, but by rebooting ?
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 12:45 ` Ian Jackson
@ 2017-05-02 13:27 ` Andrew Cooper
2017-05-02 13:42 ` Ian Jackson
2017-05-02 13:30 ` Wei Liu
2017-05-02 13:32 ` Jan Beulich
2 siblings, 1 reply; 25+ messages in thread
From: Andrew Cooper @ 2017-05-02 13:27 UTC (permalink / raw)
To: Ian Jackson, Jan Beulich; +Cc: xen-devel, Wei Liu
On 02/05/17 13:45, Ian Jackson wrote:
> Jan Beulich writes ("Re: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
>> On 01.05.17 at 20:49, <osstest-admin@xenproject.org> wrote:
>> This has been recurring for the last few flights, but I wonder whether
>>
>> 2017-05-01 13:18:52 Z executing ssh ... root@172.16.144.40 readlink /dev/italia0-vg/win.guest.osstest-disk
>> 2017-05-01 13:18:52 Z executing ssh ... root@172.16.144.40 lvdisplay --colon /dev/italia0-vg/win.guest.osstest-disk
>> 2017-05-01 13:18:53 Z lvdisplay output says device is still open: /dev/italia0-vg/win.guest.osstest-disk:italia0-vg:3:1:-1:2:20480000:2500:-1:0:-1:253:2
>> 2017-05-01 13:18:53 Z executing ssh ... root@172.16.144.40 umount /dev/italia0-vg/win.guest.osstest-disk
>> umount: /dev/italia0-vg/win.guest.osstest-disk: not mounted
>> 2017-05-01 13:18:53 Z command nonzero waitstatus 8192: timeout 60 ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_108068.test-amd64-i386-xl-qemut-winxpsp3-vcpus1 root@172.16.144.40 umount /dev/italia0-vg/win.guest.osstest-disk
>> status 8192 at Osstest/TestSupport.pm line 442.
>>
>> indicates an environmental problem rather than a
>> software-under-test one (the more that the single commit
>> being tested can't possibly influence host or guest behavior).
> This is almost certainly not an environmental problem. What seems to
> be happening is that the guest shutdown/teardown is going wrong
> somehow.
>
> http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/16.ts-guest-stop.log
>
> shows this:
>
> 2017-05-01 13:18:27 Z executing ssh ... root@172.16.144.40 xl shutdown -wF win.guest.osstest
> Shutting down domain 17
> PV control interface not available: sending ACPI power button event.
> Waiting for 1 domains
> Domain 17 has been shut down, reason code 1
> 2017-05-01 13:18:36 Z executing ssh ... root@172.16.144.40 xl list
> 2017-05-01 13:18:36 Z guest win.guest.osstest state is psr
>
> So the guest has been shut down in the sense that xl shutdown -w
> has exited (-w means to wait for the shutdown), but not in the sense
> that the domain has been destroyed.
>
> osstest spends 14 seconds checking that the guest doesn't respond to
> ping (this is probably a bit pointless, TBH):
>
> 2017-05-01 13:18:50 Z ping 172.16.146.243 down
>
> Then the next step tries to start the guest. But it finds that the
> backing block device is in use. The command that fails is there so
> that this test script can be re-run in certain ad-hoc by-hand tests:
> it is trying to unmount the block device, on the theory that if it is
> shown as open in LVM, that is probably because it's mounted. The
> unmount fails.
>
> The underlying problem is that the block backend still has the guest
> block device open. Indeed, during the logs capture we see
>
> http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0-output-xl_list
>
> the guest is still there:
>
> Name ID Mem VCPUs State Time(s)
> Domain-0 0 511 4 r----- 913.9
> win.guest.osstest 18 1536 1 r----- 16.0
>
> (that's at 2017-05-01 13:18:56)
>
> I think the guest that was shut down was domid 17 and this new one is
> domid 18. This logfile
>
> http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0---var-log-xen-xl-win.guest.osstest--incoming.log
>
> shows domid 17 shutting down and then this message
>
> Done. Rebooting now
>
> and then it seems to start the domain again.
>
> Is it possible that something has changed which means that Windows
> (sometimes?) doesn't respond to an ACPI power button event by shutting
> down, but by rebooting ?
This is known, and has definitely been discussed before on xen-devel
before (although it involved IanC last time he looked at these tests, so
a while ago now).
In an APCI view of the world, the two pieces of information you can
convey is "The user pressed the power button", and "The user pressed the
sleep button".
Windows typically defaults these to suspend and sleep, not shutdown.
Neither suspend nor sleep are typically available to VMs (unless you
alter the apci_* defaults in the xl.cfg file).
You must explicitly change the defaults to always treat the power button
as poweroff, or install PV drivers which will intercept the PV protocol
and DTRT before the toolstack falls back to ACPI event.
Another issue which gets in the way is windows deciding to install
updates, which can result in reboots at any point when other options
have been selected. I presume the COLO firewall should prevent all
behaviour like that?
~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 12:45 ` Ian Jackson
2017-05-02 13:27 ` Andrew Cooper
@ 2017-05-02 13:30 ` Wei Liu
2017-05-02 13:32 ` Jan Beulich
2 siblings, 0 replies; 25+ messages in thread
From: Wei Liu @ 2017-05-02 13:30 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel, Wei Liu, Jan Beulich
On Tue, May 02, 2017 at 01:45:28PM +0100, Ian Jackson wrote:
> Jan Beulich writes ("Re: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> > On 01.05.17 at 20:49, <osstest-admin@xenproject.org> wrote:
> > This has been recurring for the last few flights, but I wonder whether
> >
> > 2017-05-01 13:18:52 Z executing ssh ... root@172.16.144.40 readlink /dev/italia0-vg/win.guest.osstest-disk
> > 2017-05-01 13:18:52 Z executing ssh ... root@172.16.144.40 lvdisplay --colon /dev/italia0-vg/win.guest.osstest-disk
> > 2017-05-01 13:18:53 Z lvdisplay output says device is still open: /dev/italia0-vg/win.guest.osstest-disk:italia0-vg:3:1:-1:2:20480000:2500:-1:0:-1:253:2
> > 2017-05-01 13:18:53 Z executing ssh ... root@172.16.144.40 umount /dev/italia0-vg/win.guest.osstest-disk
> > umount: /dev/italia0-vg/win.guest.osstest-disk: not mounted
> > 2017-05-01 13:18:53 Z command nonzero waitstatus 8192: timeout 60 ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_108068.test-amd64-i386-xl-qemut-winxpsp3-vcpus1 root@172.16.144.40 umount /dev/italia0-vg/win.guest.osstest-disk
> > status 8192 at Osstest/TestSupport.pm line 442.
> >
> > indicates an environmental problem rather than a
> > software-under-test one (the more that the single commit
> > being tested can't possibly influence host or guest behavior).
>
> This is almost certainly not an environmental problem. What seems to
> be happening is that the guest shutdown/teardown is going wrong
> somehow.
>
> http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/16.ts-guest-stop.log
>
> shows this:
>
> 2017-05-01 13:18:27 Z executing ssh ... root@172.16.144.40 xl shutdown -wF win.guest.osstest
> Shutting down domain 17
> PV control interface not available: sending ACPI power button event.
> Waiting for 1 domains
> Domain 17 has been shut down, reason code 1
> 2017-05-01 13:18:36 Z executing ssh ... root@172.16.144.40 xl list
> 2017-05-01 13:18:36 Z guest win.guest.osstest state is psr
>
> So the guest has been shut down in the sense that xl shutdown -w
> has exited (-w means to wait for the shutdown), but not in the sense
> that the domain has been destroyed.
>
> osstest spends 14 seconds checking that the guest doesn't respond to
> ping (this is probably a bit pointless, TBH):
>
> 2017-05-01 13:18:50 Z ping 172.16.146.243 down
>
> Then the next step tries to start the guest. But it finds that the
> backing block device is in use. The command that fails is there so
> that this test script can be re-run in certain ad-hoc by-hand tests:
> it is trying to unmount the block device, on the theory that if it is
> shown as open in LVM, that is probably because it's mounted. The
> unmount fails.
>
> The underlying problem is that the block backend still has the guest
> block device open. Indeed, during the logs capture we see
>
> http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0-output-xl_list
>
> the guest is still there:
>
> Name ID Mem VCPUs State Time(s)
> Domain-0 0 511 4 r----- 913.9
> win.guest.osstest 18 1536 1 r----- 16.0
>
> (that's at 2017-05-01 13:18:56)
>
> I think the guest that was shut down was domid 17 and this new one is
> domid 18. This logfile
>
> http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0---var-log-xen-xl-win.guest.osstest--incoming.log
>
> shows domid 17 shutting down and then this message
>
> Done. Rebooting now
>
> and then it seems to start the domain again.
>
> Is it possible that something has changed which means that Windows
> (sometimes?) doesn't respond to an ACPI power button event by shutting
> down, but by rebooting ?
>
We applied a patch to add ACPI device for Windows laptop/slate mode. I
don't see how that can cause problem though. It has passed the pushgate
long ago and the option is off by default.
Wei.
> Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 12:45 ` Ian Jackson
2017-05-02 13:27 ` Andrew Cooper
2017-05-02 13:30 ` Wei Liu
@ 2017-05-02 13:32 ` Jan Beulich
2017-05-02 13:40 ` Paul Durrant
2 siblings, 1 reply; 25+ messages in thread
From: Jan Beulich @ 2017-05-02 13:32 UTC (permalink / raw)
To: Paul Durrant, Ian Jackson; +Cc: xen-devel, Wei Liu
>>> On 02.05.17 at 14:45, <ian.jackson@eu.citrix.com> wrote:
> Is it possible that something has changed which means that Windows
> (sometimes?) doesn't respond to an ACPI power button event by shutting
> down, but by rebooting ?
I can't exclude it, and I vaguely recall the same question having been
asked a while ago already (with no clear answer). Let's see if Paul has
better memory than me ...
Nevertheless the recurrence of this problem doesn't suggest this is
something that happens just "sometimes", unless there's once again
something hardware specific here, and the stickiness of osstest now
doesn't allow this to pass. But that doesn't seem very likely to me,
or else we'd have the same issue at least every couple of weeks...
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 13:32 ` Jan Beulich
@ 2017-05-02 13:40 ` Paul Durrant
2017-05-02 13:44 ` Ian Jackson
0 siblings, 1 reply; 25+ messages in thread
From: Paul Durrant @ 2017-05-02 13:40 UTC (permalink / raw)
To: 'Jan Beulich', Ian Jackson; +Cc: xen-devel, Wei Liu
> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: 02 May 2017 14:33
> To: Paul Durrant <Paul.Durrant@citrix.com>; Ian Jackson
> <Ian.Jackson@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>; xen-devel <xen-
> devel@lists.xenproject.org>
> Subject: Re: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL
>
> >>> On 02.05.17 at 14:45, <ian.jackson@eu.citrix.com> wrote:
> > Is it possible that something has changed which means that Windows
> > (sometimes?) doesn't respond to an ACPI power button event by shutting
> > down, but by rebooting ?
>
> I can't exclude it, and I vaguely recall the same question having been
> asked a while ago already (with no clear answer). Let's see if Paul has
> better memory than me ...
In my experience Windows is usually consistent in its behaviour on power or sleep events so it's unlikely that it would reboot sometime and not others.
BTW, is this test really testing XP SP3? It's been out of support for a number of years now.
Paul
>
> Nevertheless the recurrence of this problem doesn't suggest this is
> something that happens just "sometimes", unless there's once again
> something hardware specific here, and the stickiness of osstest now
> doesn't allow this to pass. But that doesn't seem very likely to me,
> or else we'd have the same issue at least every couple of weeks...
>
> Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 13:27 ` Andrew Cooper
@ 2017-05-02 13:42 ` Ian Jackson
2017-05-02 13:46 ` Andrew Cooper
0 siblings, 1 reply; 25+ messages in thread
From: Ian Jackson @ 2017-05-02 13:42 UTC (permalink / raw)
To: Andrew Cooper; +Cc: xen-devel, Wei Liu, Jan Beulich
Andrew Cooper writes ("Re: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> This is known, and has definitely been discussed before on xen-devel
> before (although it involved IanC last time he looked at these tests, so
> a while ago now).
>
> In an APCI view of the world, the two pieces of information you can
> convey is "The user pressed the power button", and "The user pressed the
> sleep button".
>
> Windows typically defaults these to suspend and sleep, not shutdown.
> Neither suspend nor sleep are typically available to VMs (unless you
> alter the apci_* defaults in the xl.cfg file).
>
> You must explicitly change the defaults to always treat the power button
> as poweroff, or install PV drivers which will intercept the PV protocol
> and DTRT before the toolstack falls back to ACPI event.
The VM images we are using are the ones from Citrix's XenRT. They are
supposed to have the PV drivers, I think. If this theory were the
whole of the explanation, the problem would be repeatable and
universal.
> Another issue which gets in the way is windows deciding to install
> updates, which can result in reboots at any point when other options
> have been selected. I presume the COLO firewall should prevent all
> behaviour like that?
In theory the guest could guess at the location of the squid proxy and
get out that way. But I have just checked the squid logs and there is
nothing from the guest's IP address there. So I think I can rule that
out.
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 13:40 ` Paul Durrant
@ 2017-05-02 13:44 ` Ian Jackson
2017-05-02 13:58 ` Paul Durrant
0 siblings, 1 reply; 25+ messages in thread
From: Ian Jackson @ 2017-05-02 13:44 UTC (permalink / raw)
To: Paul Durrant; +Cc: xen-devel, Wei Liu, 'Jan Beulich'
Paul Durrant writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> In my experience Windows is usually consistent in its behaviour on power or sleep events so it's unlikely that it would reboot sometime and not others.
> BTW, is this test really testing XP SP3? It's been out of support for a number of years now.
2017-05-01 13:18:27 Z setting win_image=winxpsp3.iso
So, yes.
One option would be to simply drop the test, of course, but do we
really think this is not a bug in something to do with us ?
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 13:42 ` Ian Jackson
@ 2017-05-02 13:46 ` Andrew Cooper
0 siblings, 0 replies; 25+ messages in thread
From: Andrew Cooper @ 2017-05-02 13:46 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel, Wei Liu, Jan Beulich
On 02/05/17 14:42, Ian Jackson wrote:
> Andrew Cooper writes ("Re: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
>> This is known, and has definitely been discussed before on xen-devel
>> before (although it involved IanC last time he looked at these tests, so
>> a while ago now).
>>
>> In an APCI view of the world, the two pieces of information you can
>> convey is "The user pressed the power button", and "The user pressed the
>> sleep button".
>>
>> Windows typically defaults these to suspend and sleep, not shutdown.
>> Neither suspend nor sleep are typically available to VMs (unless you
>> alter the apci_* defaults in the xl.cfg file).
>>
>> You must explicitly change the defaults to always treat the power button
>> as poweroff, or install PV drivers which will intercept the PV protocol
>> and DTRT before the toolstack falls back to ACPI event.
> The VM images we are using are the ones from Citrix's XenRT. They are
> supposed to have the PV drivers, I think. If this theory were the
> whole of the explanation, the problem would be repeatable and
> universal.
The XenRT images very specifically don't have PV drivers installed,
because large parts of our testing is installation of the drivers.
Also, it is not representative of what end users would do, who would
first install windows natively from an ISO, then find some PV drivers to
install.
I see no evidence in the qemu logs of PV drivers in any boot of that guest.
>
>> Another issue which gets in the way is windows deciding to install
>> updates, which can result in reboots at any point when other options
>> have been selected. I presume the COLO firewall should prevent all
>> behaviour like that?
> In theory the guest could guess at the location of the squid proxy and
> get out that way. But I have just checked the squid logs and there is
> nothing from the guest's IP address there. So I think I can rule that
> out.
Ok. That is good to hear, as it takes one variable off the table.
~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 13:44 ` Ian Jackson
@ 2017-05-02 13:58 ` Paul Durrant
2017-05-02 14:04 ` Ian Jackson
2017-05-02 14:11 ` Jan Beulich
0 siblings, 2 replies; 25+ messages in thread
From: Paul Durrant @ 2017-05-02 13:58 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel, Wei Liu, 'Jan Beulich'
> -----Original Message-----
> From: Ian Jackson [mailto:ian.jackson@eu.citrix.com]
> Sent: 02 May 2017 14:44
> To: Paul Durrant <Paul.Durrant@citrix.com>
> Cc: 'Jan Beulich' <JBeulich@suse.com>; Wei Liu <wei.liu2@citrix.com>; xen-
> devel <xen-devel@lists.xenproject.org>
> Subject: RE: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL
>
> Paul Durrant writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions
> - FAIL"):
> > In my experience Windows is usually consistent in its behaviour on power
> or sleep events so it's unlikely that it would reboot sometime and not others.
> > BTW, is this test really testing XP SP3? It's been out of support for a number
> of years now.
>
> 2017-05-01 13:18:27 Z setting win_image=winxpsp3.iso
>
> So, yes.
>
> One option would be to simply drop the test, of course, but do we
> really think this is not a bug in something to do with us ?
It's very hard to say. There's nothing particularly conclusive in the logs. QEMU has definitely requested a reset as can be seen in
http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0---var-log-xen-qemu-dm-win.guest.osstest--incoming.log
but really no clue why. The most obvious reason would be a BSOD in the OS, but without catching it in the act there's not much we can tell.
IMO testing OS that are so massively out of date is probably not worthwhile. If there is any real problem then I'd expect it to manifest across multiple versions of Windows and so other tests should catch such an issue.
Paul
>
> Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 13:58 ` Paul Durrant
@ 2017-05-02 14:04 ` Ian Jackson
2017-05-02 14:05 ` Wei Liu
2017-05-02 14:06 ` Paul Durrant
2017-05-02 14:11 ` Jan Beulich
1 sibling, 2 replies; 25+ messages in thread
From: Ian Jackson @ 2017-05-02 14:04 UTC (permalink / raw)
To: Paul Durrant; +Cc: xen-devel, Wei Liu, 'Jan Beulich'
Paul Durrant writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> It's very hard to say. There's nothing particularly conclusive in the logs. QEMU has definitely requested a reset as can be seen in
>
> http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0---var-log-xen-qemu-dm-win.guest.osstest--incoming.log
>
> but really no clue why. The most obvious reason would be a BSOD in the OS, but without catching it in the act there's not much we can tell.
Does a Windows BSOD not crash or hang the VM ? Rebooting as a result
is pretty unhelpful in our context (and many others I guess).
> IMO testing OS that are so massively out of date is probably not worthwhile. If there is any real problem then I'd expect it to manifest across multiple versions of Windows and so other tests should catch such an issue.
I don't have an opinion about this. I can drop this test.
Does anyone object ?
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 14:04 ` Ian Jackson
@ 2017-05-02 14:05 ` Wei Liu
2017-05-02 14:06 ` Paul Durrant
1 sibling, 0 replies; 25+ messages in thread
From: Wei Liu @ 2017-05-02 14:05 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel, Paul Durrant, Wei Liu, 'Jan Beulich'
On Tue, May 02, 2017 at 03:04:27PM +0100, Ian Jackson wrote:
> Paul Durrant writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> > It's very hard to say. There's nothing particularly conclusive in the logs. QEMU has definitely requested a reset as can be seen in
> >
> > http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-xl-qemut-winxpsp3-vcpus1/italia0---var-log-xen-qemu-dm-win.guest.osstest--incoming.log
> >
> > but really no clue why. The most obvious reason would be a BSOD in the OS, but without catching it in the act there's not much we can tell.
>
> Does a Windows BSOD not crash or hang the VM ? Rebooting as a result
> is pretty unhelpful in our context (and many others I guess).
>
> > IMO testing OS that are so massively out of date is probably not worthwhile. If there is any real problem then I'd expect it to manifest across multiple versions of Windows and so other tests should catch such an issue.
>
> I don't have an opinion about this. I can drop this test.
>
> Does anyone object ?
>
No objection. Actually +1 from me. :-)
> Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 14:04 ` Ian Jackson
2017-05-02 14:05 ` Wei Liu
@ 2017-05-02 14:06 ` Paul Durrant
1 sibling, 0 replies; 25+ messages in thread
From: Paul Durrant @ 2017-05-02 14:06 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel, Wei Liu, 'Jan Beulich'
> -----Original Message-----
> From: Ian Jackson [mailto:ian.jackson@eu.citrix.com]
> Sent: 02 May 2017 15:04
> To: Paul Durrant <Paul.Durrant@citrix.com>
> Cc: 'Jan Beulich' <JBeulich@suse.com>; Wei Liu <wei.liu2@citrix.com>; xen-
> devel <xen-devel@lists.xenproject.org>
> Subject: RE: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL
>
> Paul Durrant writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions
> - FAIL"):
> > It's very hard to say. There's nothing particularly conclusive in the logs.
> QEMU has definitely requested a reset as can be seen in
> >
> > http://logs.test-lab.xenproject.org/osstest/logs/108068/test-amd64-i386-
> xl-qemut-winxpsp3-vcpus1/italia0---var-log-xen-qemu-dm-
> win.guest.osstest--incoming.log
> >
> > but really no clue why. The most obvious reason would be a BSOD in the
> OS, but without catching it in the act there's not much we can tell.
>
> Does a Windows BSOD not crash or hang the VM ? Rebooting as a result
> is pretty unhelpful in our context (and many others I guess).
Unless you modify the config of the OS (as XenRT does) then the default action is to reboot. I assume this is to attempt to maintain availability on the normal case.
Paul
>
> > IMO testing OS that are so massively out of date is probably not
> worthwhile. If there is any real problem then I'd expect it to manifest across
> multiple versions of Windows and so other tests should catch such an issue.
>
> I don't have an opinion about this. I can drop this test.
>
> Does anyone object ?
>
> Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 13:58 ` Paul Durrant
2017-05-02 14:04 ` Ian Jackson
@ 2017-05-02 14:11 ` Jan Beulich
2017-05-02 14:14 ` Paul Durrant
1 sibling, 1 reply; 25+ messages in thread
From: Jan Beulich @ 2017-05-02 14:11 UTC (permalink / raw)
To: Paul Durrant; +Cc: Ian Jackson, Wei Liu, xen-devel
>>> On 02.05.17 at 15:58, <Paul.Durrant@citrix.com> wrote:
> IMO testing OS that are so massively out of date is probably not worthwhile.
> If there is any real problem then I'd expect it to manifest across multiple
> versions of Windows and so other tests should catch such an issue.
While I agree with the first sentence, I'm afraid with XP tests dropped
there'll be only one Windows version left that we test (Win7).
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 14:11 ` Jan Beulich
@ 2017-05-02 14:14 ` Paul Durrant
2017-05-02 14:29 ` Ian Jackson
0 siblings, 1 reply; 25+ messages in thread
From: Paul Durrant @ 2017-05-02 14:14 UTC (permalink / raw)
To: 'Jan Beulich'; +Cc: Ian Jackson, Wei Liu, xen-devel
> -----Original Message-----
> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Jan
> Beulich
> Sent: 02 May 2017 15:11
> To: Paul Durrant <Paul.Durrant@citrix.com>
> Cc: Ian Jackson <Ian.Jackson@citrix.com>; Wei Liu <wei.liu2@citrix.com>;
> xen-devel <xen-devel@lists.xenproject.org>
> Subject: Re: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL
>
> >>> On 02.05.17 at 15:58, <Paul.Durrant@citrix.com> wrote:
> > IMO testing OS that are so massively out of date is probably not
> worthwhile.
> > If there is any real problem then I'd expect it to manifest across multiple
> > versions of Windows and so other tests should catch such an issue.
>
> While I agree with the first sentence, I'm afraid with XP tests dropped
> there'll be only one Windows version left that we test (Win7).
>
Ok, there really should be tests for Windows 10 and/or Server 2016 in there. A lot of the newer viridian features only apply to those OS.
Paul
> Jan
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [xen-unstable test] 108068: regressions - FAIL
2017-05-02 14:14 ` Paul Durrant
@ 2017-05-02 14:29 ` Ian Jackson
2017-05-03 16:05 ` Newer Windows in Xen Project osstest test lab Ian Jackson
0 siblings, 1 reply; 25+ messages in thread
From: Ian Jackson @ 2017-05-02 14:29 UTC (permalink / raw)
To: Paul Durrant; +Cc: xen-devel, Wei Liu, 'Jan Beulich'
Paul Durrant writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> Ok, there really should be tests for Windows 10 and/or Server 2016 in there. A lot of the newer viridian features only apply to those OS.
I'll look into this.
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Newer Windows in Xen Project osstest test lab
2017-05-02 14:29 ` Ian Jackson
@ 2017-05-03 16:05 ` Ian Jackson
2017-05-03 16:19 ` Paul Durrant
0 siblings, 1 reply; 25+ messages in thread
From: Ian Jackson @ 2017-05-03 16:05 UTC (permalink / raw)
To: Paul Durrant, 'Jan Beulich', Wei Liu, xen-devel
Ian Jackson writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> Paul Durrant writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions - FAIL"):
> > Ok, there really should be tests for Windows 10 and/or Server 2016 in there. A lot of the newer viridian features only apply to those OS.
>
> I'll look into this.
Citrix's XenRT have a large number of autoinstall images which we can
use. Looking at the list available I propose to test
win10v1703-x86.iso
ws16-x64.iso
(There are also various other win10v*, -x86 versions, _uefi versions
available, but we have limited test bandwidth so I would like to
keep it to about two new images to replace winxpsp3.)
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Newer Windows in Xen Project osstest test lab
2017-05-03 16:05 ` Newer Windows in Xen Project osstest test lab Ian Jackson
@ 2017-05-03 16:19 ` Paul Durrant
2017-05-04 12:20 ` [OSSTEST PATCH 1/3] make-flight: Factor out do_hvm_win_test_one Ian Jackson
0 siblings, 1 reply; 25+ messages in thread
From: Paul Durrant @ 2017-05-03 16:19 UTC (permalink / raw)
To: Ian Jackson, 'Jan Beulich', Wei Liu, xen-devel
> -----Original Message-----
> From: Ian Jackson [mailto:ian.jackson@eu.citrix.com]
> Sent: 03 May 2017 17:06
> To: Paul Durrant <Paul.Durrant@citrix.com>; 'Jan Beulich'
> <JBeulich@suse.com>; Wei Liu <wei.liu2@citrix.com>; xen-devel <xen-
> devel@lists.xenproject.org>
> Subject: Newer Windows in Xen Project osstest test lab
>
> Ian Jackson writes ("RE: [Xen-devel] [xen-unstable test] 108068: regressions -
> FAIL"):
> > Paul Durrant writes ("RE: [Xen-devel] [xen-unstable test] 108068:
> regressions - FAIL"):
> > > Ok, there really should be tests for Windows 10 and/or Server 2016 in
> there. A lot of the newer viridian features only apply to those OS.
> >
> > I'll look into this.
>
> Citrix's XenRT have a large number of autoinstall images which we can
> use. Looking at the list available I propose to test
> win10v1703-x86.iso
> ws16-x64.iso
They look like the right ones to me.
Cheers,
Paul
> (There are also various other win10v*, -x86 versions, _uefi versions
> available, but we have limited test bandwidth so I would like to
> keep it to about two new images to replace winxpsp3.)
>
> Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* [OSSTEST PATCH 1/3] make-flight: Factor out do_hvm_win_test_one
2017-05-03 16:19 ` Paul Durrant
@ 2017-05-04 12:20 ` Ian Jackson
2017-05-04 12:20 ` [OSSTEST PATCH 2/3] make-flight: Add tests for some more recent Windows versions Ian Jackson
2017-05-04 12:20 ` [OSSTEST PATCH 3/3] make-flight: Drop Windows XP tests Ian Jackson
0 siblings, 2 replies; 25+ messages in thread
From: Ian Jackson @ 2017-05-04 12:20 UTC (permalink / raw)
To: xen-devel; +Cc: Ian Jackson
No functional change. (Verified with
standalone-generate-dump-flight-runvars.)
Don't bother messing with do_hvm_winxp_tests as 1. that uses testids
without the guest arch, so isn't compatible with this unless we make
it more general 2. we intend to abolish that for most branches
shortly.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
make-flight | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/make-flight b/make-flight
index f513b80..e9016fc 100755
--- a/make-flight
+++ b/make-flight
@@ -292,18 +292,35 @@ do_hvm_winxp_tests () {
done
}
-do_hvm_win7_x64_tests () {
+do_hvm_win_test_one () {
+ local testidpart=$1
+ local isobase=$2
+ local guestarch=$3
+
if [ $xenarch != amd64 ]; then
return
fi
- job_create_test test-$xenarch$kern-$dom0arch-xl$qemuu_suffix-win7-amd64 \
+ case "$guestarch" in
+ amd64) win_arch=x64 ;;
+ i386) win_arch=x86 ;;
+ *) win_arch=$guestarch ;; # probably wrong
+ esac
+
+ local iso=$isobase-$win_arch.iso
+
+ job_create_test \
+ test-$xenarch$kern-$dom0arch-xl$qemuu_suffix-$testidpart-$guestarch \
test-win xl $xenarch $dom0arch $qemuu_runvar \
- win_image=win7-x64.iso \
+ win_image=$iso \
win_acpi_shutdown=true \
all_hostflags=$most_hostflags,hvm
}
+do_hvm_win7_x64_tests () {
+ do_hvm_win_test_one win7 win7 amd64
+}
+
do_hvm_debian_nested_tests () {
bios=$1
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [OSSTEST PATCH 2/3] make-flight: Add tests for some more recent Windows versions
2017-05-04 12:20 ` [OSSTEST PATCH 1/3] make-flight: Factor out do_hvm_win_test_one Ian Jackson
@ 2017-05-04 12:20 ` Ian Jackson
2017-05-04 12:20 ` [OSSTEST PATCH 3/3] make-flight: Drop Windows XP tests Ian Jackson
1 sibling, 0 replies; 25+ messages in thread
From: Ian Jackson @ 2017-05-04 12:20 UTC (permalink / raw)
To: xen-devel; +Cc: Lars Kurth, Paul Durrant, Ian Jackson, Jan Beulich
We call this function `do_hvm_win_2017_tests' because 2017 is the year
in which we are adding these tests and there isn't otherwise a good
common name. This is in the expectation we might want to retire them
at a similar point.
Do these new tests for all branches. If they fail on some old
branches, they fail; but even very old branches should not regress
even for these newer guests.
I have run standalone-generate-dump-flight-runvars and eyeballed the
diff and the diff appears appropriate. The new jobs are
test-amd64-amd64-xl-qemut-win10-i386
test-amd64-amd64-xl-qemut-ws16-amd64
test-amd64-amd64-xl-qemuu-win10-i386
test-amd64-amd64-xl-qemuu-ws16-amd64
test-amd64-i386-xl-qemut-win10-i386
test-amd64-i386-xl-qemut-ws16-amd64
test-amd64-i386-xl-qemuu-win10-i386
test-amd64-i386-xl-qemuu-ws16-amd64
The images are from Citrix XenRT's autoinstall image directory,
retrieved today (2017-05-04). As with the existing images, we are
using them by virtue of the MSDN subscriptions of all the developers
who will work with them. (Pending possible future changes to the
licensing arrangements for our Windows tests.)
CC: Paul Durrant <Paul.Durrant@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Lars Kurth <lars.kurth@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
make-flight | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/make-flight b/make-flight
index e9016fc..2324b74 100755
--- a/make-flight
+++ b/make-flight
@@ -321,6 +321,11 @@ do_hvm_win7_x64_tests () {
do_hvm_win_test_one win7 win7 amd64
}
+do_hvm_win_2017_tests () {
+ do_hvm_win_test_one ws16 ws16 amd64
+ do_hvm_win_test_one win10 win10v1703 i386
+}
+
do_hvm_debian_nested_tests () {
bios=$1
@@ -716,6 +721,7 @@ test_matrix_do_one () {
do_hvm_winxp_tests
do_hvm_win7_x64_tests
+ do_hvm_win_2017_tests
do_hvm_rhel6_tests
do_hvm_debian_tests
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [OSSTEST PATCH 3/3] make-flight: Drop Windows XP tests
2017-05-04 12:20 ` [OSSTEST PATCH 1/3] make-flight: Factor out do_hvm_win_test_one Ian Jackson
2017-05-04 12:20 ` [OSSTEST PATCH 2/3] make-flight: Add tests for some more recent Windows versions Ian Jackson
@ 2017-05-04 12:20 ` Ian Jackson
2017-05-04 13:19 ` Jan Beulich
2017-05-04 16:00 ` Ian Jackson
1 sibling, 2 replies; 25+ messages in thread
From: Ian Jackson @ 2017-05-04 12:20 UTC (permalink / raw)
To: xen-devel; +Cc: Paul Durrant, Ian Jackson, Jan Beulich
XP is really quite obsolete now.
From diffing standalone-generate-dump-flight-runvars output,
the tests that are dropped are:
test-amd64-amd64-xl-qemut-winxpsp3
test-amd64-amd64-xl-qemuu-winxpsp3
test-amd64-i386-xl-qemut-winxpsp3
test-amd64-i386-xl-qemut-winxpsp3-vcpus1
test-amd64-i386-xl-qemuu-winxpsp3
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1
This means we are introducing 8 tests and dropping 7.
We drop the tests only on the branches:
linux-3.0
linux-3.10
linux-3.14
linux-3.16
linux-3.18
linux-3.4
linux-4.1
linux-4.9
linux-linus
linux-next
osstest
qemu-mainline
qemu-upstream-4.7-testing
qemu-upstream-4.8-testing
qemu-upstream-unstable
seabios
xen-4.7-testing
xen-4.8-testing
xen-unstable
The other branches are mostly out-of-support Xen branches. These are
either old ones we are still doing security support for (and would
like to know about regressions on, even for old guests), or very old
ones which we don't expect to change ever.
CC: Paul Durrant <Paul.Durrant@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
make-flight | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/make-flight b/make-flight
index 2324b74..d078b33 100755
--- a/make-flight
+++ b/make-flight
@@ -255,6 +255,18 @@ do_freebsd_tests () {
}
do_hvm_winxp_tests () {
+ case $xenbranch in
+ xen-3.*-testing) ;;
+ xen-4.0-testing) ;;
+ xen-4.1-testing) ;;
+ xen-4.2-testing) ;;
+ xen-4.3-testing) ;;
+ xen-4.4-testing) ;;
+ xen-4.5-testing) ;;
+ xen-4.6-testing) ;;
+ *) return;;
+ esac
+
for vcpus in '' 1; do
case "$vcpus" in
'') vcpus_runvars=''; vcpus_suffix='' ;;
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [OSSTEST PATCH 3/3] make-flight: Drop Windows XP tests
2017-05-04 12:20 ` [OSSTEST PATCH 3/3] make-flight: Drop Windows XP tests Ian Jackson
@ 2017-05-04 13:19 ` Jan Beulich
2017-05-11 14:37 ` Proposal to drop Windows XP tests from Xen Project CI Ian Jackson
2017-05-04 16:00 ` Ian Jackson
1 sibling, 1 reply; 25+ messages in thread
From: Jan Beulich @ 2017-05-04 13:19 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel, Paul Durrant
>>> On 04.05.17 at 14:20, <ian.jackson@eu.citrix.com> wrote:
> XP is really quite obsolete now.
>
> From diffing standalone-generate-dump-flight-runvars output,
> the tests that are dropped are:
> test-amd64-amd64-xl-qemut-winxpsp3
> test-amd64-amd64-xl-qemuu-winxpsp3
> test-amd64-i386-xl-qemut-winxpsp3
> test-amd64-i386-xl-qemut-winxpsp3-vcpus1
> test-amd64-i386-xl-qemuu-winxpsp3
> test-amd64-i386-xl-qemuu-winxpsp3-vcpus1
>
> This means we are introducing 8 tests and dropping 7.
>
> We drop the tests only on the branches:
> linux-3.0
> linux-3.10
> linux-3.14
> linux-3.16
> linux-3.18
> linux-3.4
> linux-4.1
> linux-4.9
> linux-linus
> linux-next
> osstest
> qemu-mainline
> qemu-upstream-4.7-testing
> qemu-upstream-4.8-testing
> qemu-upstream-unstable
> seabios
> xen-4.7-testing
> xen-4.8-testing
> xen-unstable
>
> The other branches are mostly out-of-support Xen branches. These are
> either old ones we are still doing security support for (and would
> like to know about regressions on, even for old guests), or very old
> ones which we don't expect to change ever.
4.6, while having passed the 1.5 year general support limit, had
its most recent stable release go out before that time span was
over, so I think we owe the community another stable release
there. Hence I'm wondering whether it should be grouped with
4.7 and 4.8 above.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Proposal to drop Windows XP tests from Xen Project CI
2017-05-04 12:20 ` [OSSTEST PATCH 3/3] make-flight: Drop Windows XP tests Ian Jackson
2017-05-04 13:19 ` Jan Beulich
@ 2017-05-04 16:00 ` Ian Jackson
1 sibling, 0 replies; 25+ messages in thread
From: Ian Jackson @ 2017-05-04 16:00 UTC (permalink / raw)
To: xen-devel, xen-users; +Cc: Paul Durrant, Jan Beulich
Recently, the tests of Windows XP SP3 that are run by osstest, as part
of the Xen Project's test suite, have started failing a lot more.
It is not clear what has caused this. The failures are causing
blockages: several of our `staging' branches are not gettting pushed
to the corresponding `stable' or `master' branches. Older Xen
releases, as well as current master (4.9-rc) are affected. This is a
problem for Xen development.
In my capacity as osstest administrator, I have tried to find someone
to help debug these. (I don't have the knowledge to do so myself.)
I haven't had a good response. My colleagues at Citrix tell me that
their internal XenRT system, used for XenServer, will be dropping its
own tests of Windows XP. It's been suggested to me to simply drop the
Xen Project tests of Windows XP.
If you thinks that XP should continue to work well, and therefore to
be tested, I'm afraid I need your help. Please contact me at the
address above, and I can provide more details, help, etc.
I myself investigated one failure and found that the VM seemed to have
responded to an ACPI power button event by rebooting, rather than by
shutting down as expected. Wei looked at another failure and found
that the VM had failed to shut down because the XenRT guest agent
(which is baked into the autoinstall images we are using) was blocking
the shutdown. There are also obviously other failure modes.
For now, here are some example logs:
http://logs.test-lab.xenproject.org/osstest/logs/108185/test-amd64-amd64-xl-qemut-winxpsp3/info.html
http://logs.test-lab.xenproject.org/osstest/logs/108185/test-amd64-amd64-xl-qemuu-winxpsp3/info.html
Thanks,
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Proposal to drop Windows XP tests from Xen Project CI
2017-05-04 13:19 ` Jan Beulich
@ 2017-05-11 14:37 ` Ian Jackson
0 siblings, 0 replies; 25+ messages in thread
From: Ian Jackson @ 2017-05-11 14:37 UTC (permalink / raw)
To: Jan Beulich; +Cc: xen-devel, Paul Durrant, xen-users
Ian Jackson writes ("Proposal to drop Windows XP tests from Xen Project CI"):
> Recently, the tests of Windows XP SP3 that are run by osstest, as part
> of the Xen Project's test suite, have started failing a lot more.
>
> It is not clear what has caused this. The failures are causing
> blockages: several of our `staging' branches are not gettting pushed
> to the corresponding `stable' or `master' branches. Older Xen
> releases, as well as current master (4.9-rc) are affected. This is a
> problem for Xen development.
>
> In my capacity as osstest administrator, I have tried to find someone
> to help debug these. (I don't have the knowledge to do so myself.)
>
> I haven't had a good response. My colleagues at Citrix tell me that
> their internal XenRT system, used for XenServer, will be dropping its
> own tests of Windows XP. It's been suggested to me to simply drop the
> Xen Project tests of Windows XP.
>
> If you thinks that XP should continue to work well, and therefore to
> be tested, I'm afraid I need your help. Please contact me at the
> address above, and I can provide more details, help, etc.
I have just pushed to osstest pretest the change to drop testing of
Windows XP. We are replacing it with tests of Windows Server 2016 and
Windows 10 (but of course as those are new tests, they will not count
for blocking regressions until they have passed).
Jan Beulich writes ("Re: [OSSTEST PATCH 3/3] make-flight: Drop Windows XP tests"):
> On 04.05.17 at 14:20, <ian.jackson@eu.citrix.com> wrote:
> > We drop the tests only on the branches:
...
> > xen-4.7-testing
> > xen-4.8-testing
> > xen-unstable
> >
> > The other branches are mostly out-of-support Xen branches. These are
> > either old ones we are still doing security support for (and would
> > like to know about regressions on, even for old guests), or very old
> > ones which we don't expect to change ever.
>
> 4.6, while having passed the 1.5 year general support limit, had
> its most recent stable release go out before that time span was
> over, so I think we owe the community another stable release
> there. Hence I'm wondering whether it should be grouped with
> 4.7 and 4.8 above.
Done. (Ie, I am dropping the tests on Xen 4.6 too.)
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2017-05-11 14:37 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-01 18:49 [xen-unstable test] 108068: regressions - FAIL osstest service owner
2017-05-02 9:10 ` Jan Beulich
2017-05-02 12:45 ` Ian Jackson
2017-05-02 13:27 ` Andrew Cooper
2017-05-02 13:42 ` Ian Jackson
2017-05-02 13:46 ` Andrew Cooper
2017-05-02 13:30 ` Wei Liu
2017-05-02 13:32 ` Jan Beulich
2017-05-02 13:40 ` Paul Durrant
2017-05-02 13:44 ` Ian Jackson
2017-05-02 13:58 ` Paul Durrant
2017-05-02 14:04 ` Ian Jackson
2017-05-02 14:05 ` Wei Liu
2017-05-02 14:06 ` Paul Durrant
2017-05-02 14:11 ` Jan Beulich
2017-05-02 14:14 ` Paul Durrant
2017-05-02 14:29 ` Ian Jackson
2017-05-03 16:05 ` Newer Windows in Xen Project osstest test lab Ian Jackson
2017-05-03 16:19 ` Paul Durrant
2017-05-04 12:20 ` [OSSTEST PATCH 1/3] make-flight: Factor out do_hvm_win_test_one Ian Jackson
2017-05-04 12:20 ` [OSSTEST PATCH 2/3] make-flight: Add tests for some more recent Windows versions Ian Jackson
2017-05-04 12:20 ` [OSSTEST PATCH 3/3] make-flight: Drop Windows XP tests Ian Jackson
2017-05-04 13:19 ` Jan Beulich
2017-05-11 14:37 ` Proposal to drop Windows XP tests from Xen Project CI Ian Jackson
2017-05-04 16:00 ` Ian Jackson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.