* [xen-unstable test] 106698: regressions - FAIL
@ 2017-03-16 9:03 osstest service owner
2017-03-16 11:15 ` Jan Beulich
0 siblings, 1 reply; 4+ messages in thread
From: osstest service owner @ 2017-03-16 9:03 UTC (permalink / raw)
To: xen-devel, osstest-admin
[-- Attachment #1: Type: text/plain, Size: 13555 bytes --]
flight 106698 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/106698/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail REGR. vs. 106652
Regressions which are regarded as allowable (not blocking):
test-amd64-amd64-rumprun-amd64 16 rumprun-demo-xenstorels/xenstorels.repeat fail REGR. vs. 106652
test-armhf-armhf-libvirt-raw 12 saverestore-support-check fail like 106642
test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 106652
test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 106652
test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop fail like 106652
test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 106652
test-armhf-armhf-libvirt 13 saverestore-support-check fail like 106652
test-armhf-armhf-libvirt-xsm 13 saverestore-support-check fail like 106652
test-amd64-amd64-xl-rtds 9 debian-install fail like 106652
Tests which did not succeed, but are not blocking:
test-arm64-arm64-libvirt-xsm 1 build-check(1) blocked n/a
test-arm64-arm64-xl 1 build-check(1) blocked n/a
build-arm64-libvirt 1 build-check(1) blocked n/a
test-arm64-arm64-libvirt-qcow2 1 build-check(1) blocked n/a
test-arm64-arm64-libvirt 1 build-check(1) blocked n/a
test-arm64-arm64-xl-credit2 1 build-check(1) blocked n/a
test-arm64-arm64-xl-rtds 1 build-check(1) blocked n/a
test-arm64-arm64-xl-multivcpu 1 build-check(1) blocked n/a
test-arm64-arm64-xl-xsm 1 build-check(1) blocked n/a
test-amd64-amd64-xl-pvh-amd 11 guest-start fail never pass
test-amd64-amd64-xl-pvh-intel 11 guest-start fail never pass
test-amd64-i386-libvirt 12 migrate-support-check fail never pass
test-amd64-i386-libvirt-xsm 12 migrate-support-check fail never pass
test-amd64-amd64-libvirt 12 migrate-support-check fail never pass
build-arm64-xsm 5 xen-build fail never pass
build-arm64 5 xen-build fail never pass
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass
test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass
build-arm64-pvops 5 kernel-build fail never pass
test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2 fail never pass
test-amd64-amd64-libvirt-vhd 11 migrate-support-check fail never pass
test-armhf-armhf-xl-cubietruck 12 migrate-support-check fail never pass
test-armhf-armhf-xl-cubietruck 13 saverestore-support-check fail never pass
test-armhf-armhf-xl 12 migrate-support-check fail never pass
test-armhf-armhf-xl 13 saverestore-support-check fail never pass
test-armhf-armhf-xl-credit2 12 migrate-support-check fail never pass
test-armhf-armhf-xl-credit2 13 saverestore-support-check fail never pass
test-armhf-armhf-xl-multivcpu 12 migrate-support-check fail never pass
test-armhf-armhf-xl-multivcpu 13 saverestore-support-check fail never pass
test-amd64-amd64-libvirt-xsm 12 migrate-support-check fail never pass
test-armhf-armhf-xl-vhd 11 migrate-support-check fail never pass
test-armhf-armhf-xl-vhd 12 saverestore-support-check fail never pass
test-armhf-armhf-xl-arndale 12 migrate-support-check fail never pass
test-armhf-armhf-xl-arndale 13 saverestore-support-check fail never pass
test-armhf-armhf-xl-xsm 12 migrate-support-check fail never pass
test-armhf-armhf-xl-xsm 13 saverestore-support-check fail never pass
test-armhf-armhf-libvirt 12 migrate-support-check fail never pass
test-armhf-armhf-libvirt-xsm 12 migrate-support-check fail never pass
test-armhf-armhf-xl-rtds 12 migrate-support-check fail never pass
test-armhf-armhf-xl-rtds 13 saverestore-support-check fail never pass
test-armhf-armhf-libvirt-raw 11 migrate-support-check fail never pass
version targeted for testing:
xen bfd9a2095f1882e8c074b2d911bcb07d12cf6cf5
baseline version:
xen bd8ad2a52aba4911ada897c72f8795172a09a193
Last test of basis 106652 2017-03-14 08:24:37 Z 2 days
Failing since 106671 2017-03-14 20:44:11 Z 1 days 2 attempts
Testing same since 106698 2017-03-15 17:48:34 Z 0 days 1 attempts
------------------------------------------------------------
People who touched revisions under test:
Andrew Cooper <andrew.cooper3@citrix.com>
Daniel De Graaf <dgdegra@tycho.nsa.gov>
Jan Beulich <jbeulich@suse.com>
Juergen Gross <jgross@suse.com>
Julien Grall <julien.grall@arm.com>
Kevin Tian <kevin.tian@intel.com>
Olaf Hering <olaf@aepfle.de>
Razvan Cojocaru <rcojocaru@bitdefender.com>
Roger Pau Monné <roger.pau@citrix.com>
Sergey Dyasli <sergey.dyasli@citrix.com>
Tim Deegan <tim@xen.org>
Wei Liu <wei.liu2@citrix.com>
Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
jobs:
build-amd64-xsm pass
build-arm64-xsm fail
build-armhf-xsm pass
build-i386-xsm pass
build-amd64-xtf pass
build-amd64 pass
build-arm64 fail
build-armhf pass
build-i386 pass
build-amd64-libvirt pass
build-arm64-libvirt blocked
build-armhf-libvirt pass
build-i386-libvirt pass
build-amd64-oldkern pass
build-i386-oldkern pass
build-amd64-prev pass
build-i386-prev pass
build-amd64-pvops pass
build-arm64-pvops fail
build-armhf-pvops pass
build-i386-pvops pass
build-amd64-rumprun pass
build-i386-rumprun pass
test-xtf-amd64-amd64-1 pass
test-xtf-amd64-amd64-2 pass
test-xtf-amd64-amd64-3 pass
test-xtf-amd64-amd64-4 pass
test-xtf-amd64-amd64-5 pass
test-amd64-amd64-xl pass
test-arm64-arm64-xl blocked
test-armhf-armhf-xl pass
test-amd64-i386-xl pass
test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm pass
test-amd64-i386-xl-qemut-debianhvm-amd64-xsm pass
test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm pass
test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm pass
test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass
test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm pass
test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm pass
test-amd64-amd64-libvirt-xsm pass
test-arm64-arm64-libvirt-xsm blocked
test-armhf-armhf-libvirt-xsm pass
test-amd64-i386-libvirt-xsm pass
test-amd64-amd64-xl-xsm pass
test-arm64-arm64-xl-xsm blocked
test-armhf-armhf-xl-xsm pass
test-amd64-i386-xl-xsm pass
test-amd64-amd64-qemuu-nested-amd fail
test-amd64-amd64-xl-pvh-amd fail
test-amd64-i386-qemut-rhel6hvm-amd pass
test-amd64-i386-qemuu-rhel6hvm-amd pass
test-amd64-amd64-xl-qemut-debianhvm-amd64 pass
test-amd64-i386-xl-qemut-debianhvm-amd64 pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass
test-amd64-i386-xl-qemuu-debianhvm-amd64 pass
test-amd64-i386-freebsd10-amd64 pass
test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
test-amd64-i386-xl-qemuu-ovmf-amd64 pass
test-amd64-amd64-rumprun-amd64 fail
test-amd64-amd64-xl-qemut-win7-amd64 fail
test-amd64-i386-xl-qemut-win7-amd64 fail
test-amd64-amd64-xl-qemuu-win7-amd64 fail
test-amd64-i386-xl-qemuu-win7-amd64 fail
test-armhf-armhf-xl-arndale pass
test-amd64-amd64-xl-credit2 pass
test-arm64-arm64-xl-credit2 blocked
test-armhf-armhf-xl-credit2 pass
test-armhf-armhf-xl-cubietruck pass
test-amd64-i386-freebsd10-i386 pass
test-amd64-i386-rumprun-i386 pass
test-amd64-amd64-qemuu-nested-intel fail
test-amd64-amd64-xl-pvh-intel fail
test-amd64-i386-qemut-rhel6hvm-intel pass
test-amd64-i386-qemuu-rhel6hvm-intel pass
test-amd64-amd64-libvirt pass
test-arm64-arm64-libvirt blocked
test-armhf-armhf-libvirt pass
test-amd64-i386-libvirt pass
test-amd64-amd64-migrupgrade pass
test-amd64-i386-migrupgrade pass
test-amd64-amd64-xl-multivcpu pass
test-arm64-arm64-xl-multivcpu blocked
test-armhf-armhf-xl-multivcpu pass
test-amd64-amd64-pair pass
test-amd64-i386-pair pass
test-amd64-amd64-libvirt-pair pass
test-amd64-i386-libvirt-pair pass
test-amd64-amd64-amd64-pvgrub pass
test-amd64-amd64-i386-pvgrub pass
test-amd64-amd64-pygrub pass
test-arm64-arm64-libvirt-qcow2 blocked
test-amd64-amd64-xl-qcow2 pass
test-armhf-armhf-libvirt-raw pass
test-amd64-i386-xl-raw pass
test-amd64-amd64-xl-rtds fail
test-arm64-arm64-xl-rtds blocked
test-armhf-armhf-xl-rtds pass
test-amd64-i386-xl-qemut-winxpsp3-vcpus1 pass
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass
test-amd64-amd64-libvirt-vhd pass
test-armhf-armhf-xl-vhd pass
test-amd64-amd64-xl-qemut-winxpsp3 pass
test-amd64-i386-xl-qemut-winxpsp3 pass
test-amd64-amd64-xl-qemuu-winxpsp3 pass
test-amd64-i386-xl-qemuu-winxpsp3 pass
------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images
Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs
Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
Not pushing.
(No revision log; it would be 601 lines long.)
[-- Attachment #2: Type: text/plain, Size: 127 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [xen-unstable test] 106698: regressions - FAIL
2017-03-16 9:03 [xen-unstable test] 106698: regressions - FAIL osstest service owner
@ 2017-03-16 11:15 ` Jan Beulich
2017-03-16 14:26 ` Sergey Dyasli
0 siblings, 1 reply; 4+ messages in thread
From: Jan Beulich @ 2017-03-16 11:15 UTC (permalink / raw)
To: Sergey Dyasli; +Cc: xen-devel, osstest-admin
>>> On 16.03.17 at 10:03, <osstest-admin@xenproject.org> wrote:
> flight 106698 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/106698/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
> test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail REGR. vs. 106652
While there's quite a bit of stuff under test, your recent vVMX series
would seem to be the most likely candidate for a regression here. I
am, however, puzzled by
(XEN) d1v0 VMLAUNCH error: 0
(XEN) domain_crash_sync called from vmcs.c:1712
in the L1 log - error 0 is supposed to be "no error", and I can't see
how VM_INSTRUCTION_ERROR would ever be written to zero.
Which leaves there being a path (which I can't spot) where it's not
being written, or a problem handling the respective vmread by the
guest.
Could you take a look, please?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [xen-unstable test] 106698: regressions - FAIL
2017-03-16 11:15 ` Jan Beulich
@ 2017-03-16 14:26 ` Sergey Dyasli
2017-03-16 15:01 ` Andrew Cooper
0 siblings, 1 reply; 4+ messages in thread
From: Sergey Dyasli @ 2017-03-16 14:26 UTC (permalink / raw)
To: JBeulich; +Cc: Sergey Dyasli, Kevin Tian, osstest-admin, xen-devel
On Thu, 2017-03-16 at 05:15 -0600, Jan Beulich wrote:
> > > > On 16.03.17 at 10:03, <osstest-admin@xenproject.org> wrote:
> >
> > flight 106698 xen-unstable real [real]
> > http://logs.test-lab.xenproject.org/osstest/logs/106698/
> >
> > Regressions :-(
> >
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> > test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail REGR. vs. 106652
>
> While there's quite a bit of stuff under test, your recent vVMX series
> would seem to be the most likely candidate for a regression here. I
> am, however, puzzled by
>
> (XEN) d1v0 VMLAUNCH error: 0
> (XEN) domain_crash_sync called from vmcs.c:1712
>
> in the L1 log - error 0 is supposed to be "no error", and I can't see
> how VM_INSTRUCTION_ERROR would ever be written to zero.
> Which leaves there being a path (which I can't spot) where it's not
> being written, or a problem handling the respective vmread by the
> guest.
>
> Could you take a look, please?
L1:vmlaunch failed and vmx_vmentry_failure() was called. However it
doesn't check if the fail was Valid or Invalid. In the latter case
VM_INSTRUCTION_ERROR would be meaningless.
There are only 2 cases for vmfail_invalid() inside nvmx_handle_vmlaunch():
1. if ( vcpu_nestedhvm(v).nv_vvmcxaddr == INVALID_PADDR )
That would imply that L0:nvmx_handle_vmptrld() returned VMfail
and L1:__vmptrld() hit BUG() which is not the case.
2. if ( nvmx->shadow_vmcs )
I have identified one possible issue with that. H/W looks like Haswell
and L0 has:
(XEN) - VMCS shadowing
However L1 is missing "VMCS shadowing" in "VMX advanced features".
I didn't expect that fact since L1 sees VMX_MISC_VMWRITE_ALL
in MSR_IA32_VMX_MISC. It must be something else that prevents L1 from
enabling vmcs shadowing.
Above makes the follwing check inside nvmx_handle_vmptrld() incorrect:
(!cpu_has_vmx_vmcs_shadowing && nvmx->shadow_vmcs)
Since cpu_has_vmx_vmcs_shadowing tests L0's capability and not L1's.
Shadow bit will be set by L0:nvmx_set_vmcs_pointer() which might
suggest that there are other cases with nvmx_handle_vmptrld() re-entrancy
that I have missed. If the following scenario is possible:
nvmx_handle_vmptrld()
nvcpu->nv_vvmcxaddr == INVALID_PADDR
nvmx->shadow_vmcs = false
vvmcs->vmcs_revision_id |= VMCS_RID_TYPE_MASK;
// no nvmx_clear_vmcs_pointer() in between
nvmx_handle_vmptrld()
nvcpu->nv_vvmcxaddr == INVALID_PADDR
nvmx->shadow_vmcs = true
(!cpu_has_vmx_vmcs_shadowing && nvmx->shadow_vmcs) == false
nvmx_handle_vmlaunch()
nvmx->shadow_vmcs == true
vmfail_invalid(regs);
Then it would explain the regression.
--
Thanks,
Sergey
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [xen-unstable test] 106698: regressions - FAIL
2017-03-16 14:26 ` Sergey Dyasli
@ 2017-03-16 15:01 ` Andrew Cooper
0 siblings, 0 replies; 4+ messages in thread
From: Andrew Cooper @ 2017-03-16 15:01 UTC (permalink / raw)
To: Sergey Dyasli, JBeulich; +Cc: xen-devel, Kevin Tian, osstest-admin
On 16/03/17 14:26, Sergey Dyasli wrote:
> On Thu, 2017-03-16 at 05:15 -0600, Jan Beulich wrote:
>>>>> On 16.03.17 at 10:03, <osstest-admin@xenproject.org> wrote:
>>> flight 106698 xen-unstable real [real]
>>> http://logs.test-lab.xenproject.org/osstest/logs/106698/
>>>
>>> Regressions :-(
>>>
>>> Tests which did not succeed and are blocking,
>>> including tests which could not be run:
>>> test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail REGR. vs. 106652
>> While there's quite a bit of stuff under test, your recent vVMX series
>> would seem to be the most likely candidate for a regression here. I
>> am, however, puzzled by
>>
>> (XEN) d1v0 VMLAUNCH error: 0
>> (XEN) domain_crash_sync called from vmcs.c:1712
>>
>> in the L1 log - error 0 is supposed to be "no error", and I can't see
>> how VM_INSTRUCTION_ERROR would ever be written to zero.
>> Which leaves there being a path (which I can't spot) where it's not
>> being written, or a problem handling the respective vmread by the
>> guest.
>>
>> Could you take a look, please?
> L1:vmlaunch failed and vmx_vmentry_failure() was called. However it
> doesn't check if the fail was Valid or Invalid. In the latter case
> VM_INSTRUCTION_ERROR would be meaningless.
>
> There are only 2 cases for vmfail_invalid() inside nvmx_handle_vmlaunch():
>
> 1. if ( vcpu_nestedhvm(v).nv_vvmcxaddr == INVALID_PADDR )
>
> That would imply that L0:nvmx_handle_vmptrld() returned VMfail
> and L1:__vmptrld() hit BUG() which is not the case.
>
> 2. if ( nvmx->shadow_vmcs )
>
> I have identified one possible issue with that. H/W looks like Haswell
> and L0 has:
>
> (XEN) - VMCS shadowing
>
> However L1 is missing "VMCS shadowing" in "VMX advanced features".
> I didn't expect that fact since L1 sees VMX_MISC_VMWRITE_ALL
> in MSR_IA32_VMX_MISC. It must be something else that prevents L1 from
> enabling vmcs shadowing.
>
> Above makes the follwing check inside nvmx_handle_vmptrld() incorrect:
>
> (!cpu_has_vmx_vmcs_shadowing && nvmx->shadow_vmcs)
>
> Since cpu_has_vmx_vmcs_shadowing tests L0's capability and not L1's.
>
> Shadow bit will be set by L0:nvmx_set_vmcs_pointer() which might
> suggest that there are other cases with nvmx_handle_vmptrld() re-entrancy
> that I have missed. If the following scenario is possible:
>
> nvmx_handle_vmptrld()
> nvcpu->nv_vvmcxaddr == INVALID_PADDR
> nvmx->shadow_vmcs = false
> vvmcs->vmcs_revision_id |= VMCS_RID_TYPE_MASK;
>
> // no nvmx_clear_vmcs_pointer() in between
>
> nvmx_handle_vmptrld()
> nvcpu->nv_vvmcxaddr == INVALID_PADDR
> nvmx->shadow_vmcs = true
> (!cpu_has_vmx_vmcs_shadowing && nvmx->shadow_vmcs) == false
>
> nvmx_handle_vmlaunch()
> nvmx->shadow_vmcs == true
> vmfail_invalid(regs);
>
> Then it would explain the regression.
Ok - we should revert dc05c0ceeb8609b6d60f6a117a0192e9160946b8 and
b22ee98c4ecc4e7c827451dee01181529df4d26c to unblock master.
I will get to this shortly, unless there are sudden objections.
~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-03-16 15:02 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-16 9:03 [xen-unstable test] 106698: regressions - FAIL osstest service owner
2017-03-16 11:15 ` Jan Beulich
2017-03-16 14:26 ` Sergey Dyasli
2017-03-16 15:01 ` Andrew Cooper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.