* [xen-unstable test] 65141: regressions - FAIL @ 2015-11-27 15:45 osstest service owner 2015-11-27 15:54 ` Ian Jackson 0 siblings, 1 reply; 23+ messages in thread From: osstest service owner @ 2015-11-27 15:45 UTC (permalink / raw) To: xen-devel, osstest-admin [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 17275 bytes --] flight 65141 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/65141/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail REGR. vs. 65114 Regressions which are regarded as allowable (not blocking): test-armhf-armhf-xl-rtds 15 guest-start.2 fail REGR. vs. 65114 test-armhf-armhf-libvirt 7 host-ping-check-xen fail blocked in 65114 test-amd64-i386-rumpuserxen-i386 10 guest-start fail like 65114 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail like 65114 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 65114 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 9 debian-hvm-install fail like 65114 test-amd64-amd64-libvirt-vhd 9 debian-di-install fail like 65114 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate fail like 65114 Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-pvh-intel 11 guest-start fail never pass test-armhf-armhf-libvirt-raw 9 debian-di-install fail never pass test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2 fail never pass test-amd64-amd64-libvirt-xsm 12 migrate-support-check fail never pass test-amd64-i386-libvirt 12 migrate-support-check fail never pass test-armhf-armhf-xl-multivcpu 13 saverestore-support-check fail never pass test-armhf-armhf-xl-multivcpu 12 migrate-support-check fail never pass test-armhf-armhf-xl-rtds 13 saverestore-support-check fail never pass test-armhf-armhf-xl-rtds 12 migrate-support-check fail never pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass test-amd64-i386-libvirt-xsm 12 migrate-support-check fail never pass test-armhf-armhf-xl-arndale 12 migrate-support-check fail never pass test-armhf-armhf-xl-arndale 13 saverestore-support-check fail never pass test-armhf-armhf-xl-cubietruck 12 migrate-support-check fail never pass test-armhf-armhf-xl-cubietruck 13 saverestore-support-check fail never pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check fail never pass test-armhf-armhf-libvirt-qcow2 9 debian-di-install fail never pass test-armhf-armhf-libvirt-xsm 12 migrate-support-check fail never pass test-armhf-armhf-libvirt-xsm 14 guest-saverestore fail never pass test-amd64-amd64-xl-pvh-amd 11 guest-start fail never pass test-armhf-armhf-xl-vhd 9 debian-di-install fail never pass test-armhf-armhf-xl-xsm 13 saverestore-support-check fail never pass test-armhf-armhf-xl-xsm 12 migrate-support-check fail never pass test-armhf-armhf-xl 12 migrate-support-check fail never pass test-armhf-armhf-xl 13 saverestore-support-check fail never pass test-amd64-amd64-libvirt 12 migrate-support-check fail never pass test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail never pass test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass test-armhf-armhf-xl-credit2 13 saverestore-support-check fail never pass test-armhf-armhf-xl-credit2 12 migrate-support-check fail never pass version targeted for testing: xen b1d398b67781140d1c6efd05778d0ad4103b2a32 baseline version: xen 713b7e4ef2aa4ec3ae697cde9c81d5a57548f9b1 Last test of basis 65114 2015-11-25 19:42:37 Z 1 days Testing same since 65141 2015-11-26 20:45:33 Z 0 days 1 attempts ------------------------------------------------------------ People who touched revisions under test: Ian Campbell <ian.campbell@citrix.com> Jan Beulich <jbeulich@suse.com> Kevin Tian <kevin.tian@intel.com> Paul Durrant <paul.durrant@citrix.com> Roger Pau Monné <roger.pau@citrix.com> Wei Liu <wei.liu2@citrix.com> Yang Zhang <yang.z.zhang@intel.com> jobs: build-amd64-xsm pass build-armhf-xsm pass build-i386-xsm pass build-amd64 pass build-armhf pass build-i386 pass build-amd64-libvirt pass build-armhf-libvirt pass build-i386-libvirt pass build-amd64-oldkern pass build-i386-oldkern pass build-amd64-prev pass build-i386-prev pass build-amd64-pvops pass build-armhf-pvops pass build-i386-pvops pass build-amd64-rumpuserxen pass build-i386-rumpuserxen pass test-amd64-amd64-xl pass test-armhf-armhf-xl pass test-amd64-i386-xl pass test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm pass test-amd64-i386-xl-qemut-debianhvm-amd64-xsm pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm pass test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm pass test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm pass test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm fail test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm fail test-amd64-amd64-libvirt-xsm pass test-armhf-armhf-libvirt-xsm fail test-amd64-i386-libvirt-xsm pass test-amd64-amd64-xl-xsm pass test-armhf-armhf-xl-xsm pass test-amd64-i386-xl-xsm pass test-amd64-amd64-qemuu-nested-amd fail test-amd64-amd64-xl-pvh-amd fail test-amd64-i386-qemut-rhel6hvm-amd pass test-amd64-i386-qemuu-rhel6hvm-amd pass test-amd64-amd64-xl-qemut-debianhvm-amd64 pass test-amd64-i386-xl-qemut-debianhvm-amd64 pass test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass test-amd64-i386-xl-qemuu-debianhvm-amd64 pass test-amd64-i386-freebsd10-amd64 pass test-amd64-amd64-xl-qemuu-ovmf-amd64 pass test-amd64-i386-xl-qemuu-ovmf-amd64 pass test-amd64-amd64-rumpuserxen-amd64 pass test-amd64-amd64-xl-qemut-win7-amd64 fail test-amd64-i386-xl-qemut-win7-amd64 fail test-amd64-amd64-xl-qemuu-win7-amd64 fail test-amd64-i386-xl-qemuu-win7-amd64 fail test-armhf-armhf-xl-arndale pass test-amd64-amd64-xl-credit2 pass test-armhf-armhf-xl-credit2 pass test-armhf-armhf-xl-cubietruck pass test-amd64-i386-freebsd10-i386 pass test-amd64-i386-rumpuserxen-i386 fail test-amd64-amd64-qemuu-nested-intel fail test-amd64-amd64-xl-pvh-intel fail test-amd64-i386-qemut-rhel6hvm-intel pass test-amd64-i386-qemuu-rhel6hvm-intel pass test-amd64-amd64-libvirt pass test-armhf-armhf-libvirt fail test-amd64-i386-libvirt pass test-amd64-amd64-migrupgrade pass test-amd64-i386-migrupgrade pass test-amd64-amd64-xl-multivcpu pass test-armhf-armhf-xl-multivcpu pass test-amd64-amd64-pair pass test-amd64-i386-pair pass test-amd64-amd64-libvirt-pair pass test-amd64-i386-libvirt-pair pass test-amd64-amd64-amd64-pvgrub pass test-amd64-amd64-i386-pvgrub pass test-amd64-amd64-pygrub pass test-armhf-armhf-libvirt-qcow2 fail test-amd64-amd64-xl-qcow2 pass test-armhf-armhf-libvirt-raw fail test-amd64-i386-xl-raw pass test-amd64-amd64-xl-rtds pass test-armhf-armhf-xl-rtds fail test-amd64-i386-xl-qemut-winxpsp3-vcpus1 pass test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 pass test-amd64-amd64-libvirt-vhd fail test-armhf-armhf-xl-vhd fail test-amd64-amd64-xl-qemut-winxpsp3 pass test-amd64-i386-xl-qemut-winxpsp3 pass test-amd64-amd64-xl-qemuu-winxpsp3 pass test-amd64-i386-xl-qemuu-winxpsp3 pass ------------------------------------------------------------ sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary Not pushing. ------------------------------------------------------------ commit b1d398b67781140d1c6efd05778d0ad4103b2a32 Author: Roger Pau Monné <roger.pau@citrix.com> Date: Thu Nov 26 16:01:27 2015 +0100 x86: allow disabling the emulated local apic Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Jan Beulich <jbeulich@suse.com> Acked-by: Kevin Tian <kevin.tian@intel.com> commit 302a8519fb8537f619569873d605f65eb18f4bdc Author: Roger Pau Monné <roger.pau@citrix.com> Date: Thu Nov 26 16:00:56 2015 +0100 x86/vlapic: fixes for HVM code when running without a vlapic The HVM related code (SVM, VMX) generally assumed that a local apic is always present. With the introduction of a HVM mode were the local apic can be removed, some of this broken code paths arised. The SVM exit/resume paths unconditionally checked the state of the lapic, which is wrong if it's been disabled by hardware, fix this by adding the necessary checks. On the VMX side, make sure we don't add mappings for a local apic if it's disabled. In the generic vlapic code, add checks to prevent setting the TSC deadline timer if the lapic is disabled, and also prevent trying to inject interrupts from the PIC is the lapic is also disabled. Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> Acked-by: Kevin Tian <kevin.tian@intel.com> commit 0ce647ad6f70c5ec0aeee66ce74429982b81911a Author: Jan Beulich <jbeulich@suse.com> Date: Thu Nov 26 15:51:49 2015 +0100 x86: suppress bogus log message The way we populate mpc_cpufeature is not compatible with modern CPUs, and hence the message printed using that information is useless/bogus. It's of interest only anyway when not using ACPI, so move it into MPS parsing code. This at once significantly reduces boot time logging on huge systems. Signed-off-by: Jan Beulich <jbeulich@suse.com> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> commit 4243baf61acf24d972eb456aea8353481af31100 Author: Roger Pau Monné <roger.pau@citrix.com> Date: Thu Nov 26 15:51:00 2015 +0100 HVM/save: allow the usage of zeroextend and a fixup function With the current compat implementation in the save/restore context handling, only one compat structure is allowed, and using _zeroextend prevents the fixup function from being called. In order to allow for the compat handling layer to be able to handle different compat versions allow calling the fixup function with hvm_load_entry_zeroextend. Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> commit fafa16e3d006d6d5f62b71374991360697db2467 Author: Roger Pau Monné <roger.pau@citrix.com> Date: Thu Nov 26 15:50:36 2015 +0100 HVM/save: pass a size parameter to the HVM compat functions In order to cope with types having multiple compat versions pass a size parameter to the fixup function so we can identify which compat version Xen is dealing with. Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> commit 08f3fad0f30a7de35d60c2615c2ee56d3f7b77c8 Author: Jan Beulich <jbeulich@suse.com> Date: Thu Nov 26 15:50:07 2015 +0100 build: fix dependencies for files compiled from their parent directory The use of $(basename ...) here was wrong (yet I'm sure I tested it). Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Ian Campbell <ian.campbell@citrix.com> commit bdead3f25b75772e954d9c93e3ba36c7a926a030 Author: Yang Zhang <yang.z.zhang@intel.com> Date: Thu Nov 26 15:49:29 2015 +0100 MAINTAINERS: change the vt-d maintainer add Feng as the new maintainer of VT-d stuff Signed-off-by: Yang Zhang <yang.z.zhang@intel.com> Acked-by: Kevin Tian <kevin.tian@intel.com> commit b38d426ad09b3e142f474c1dadfd6d6903664a48 Author: Paul Durrant <paul.durrant@citrix.com> Date: Thu Nov 26 15:48:41 2015 +0100 x86/viridian: flush remote tlbs by hypercall The Microsoft Hypervisor Top Level Functional Spec. (section 3.4) defines two bits in CPUID leaf 0x40000004:EAX for the hypervisor to recommend whether or not to issue a hypercall for local or remote TLB flush. Whilst it's doubtful whether using a hypercall for local TLB flush would be any more efficient than a specific INVLPG VMEXIT, a remote TLB flush may well be more efficiently done. This is because the alternative mechanism is to IPI all the vCPUs in question which (in the absence of APIC virtualisation) will require emulation and scheduling of the vCPUs only to have them immediately VMEXIT for local TLB flush. This patch therefore adds a viridian option which, if selected, enables the hypercall for remote TLB flush and implements it using ASID invalidation for targetted vCPUs followed by an IPI only to the set of CPUs that happened to be running a targetted vCPU (which may be the empty set). The flush may be more severe than requested since the hypercall can request flush only for a specific address space (CR3) but Xen neither keeps a mapping of ASID to guest CR3 nor allows invalidation of a specific ASID, but on a host with contended CPUs performance is still likely to be better than a more specific flush using IPIs. The implementation of the patch introduces per-vCPU viridian_init() and viridian_deinit() functions to allow a scratch cpumask to be allocated. This avoids needing to put this potentially large data structure on stack during hypercall processing. It also modifies the hypercall input and output bit-fields to allow a check for the 'fast' calling convention, and a white-space fix in the definition of HVMPV_feature_mask (to remove hard tabs). Signed-off-by: Paul Durrant <paul.durrant@citrix.com> Reviewed-by: Jan Beulich <jbeulich@suse.com> Acked-by: Wei Liu <wei.liu2@citrix.com> (qemu changes not included) [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-11-27 15:45 [xen-unstable test] 65141: regressions - FAIL osstest service owner @ 2015-11-27 15:54 ` Ian Jackson 2015-11-30 5:35 ` Hu, Robert 0 siblings, 1 reply; 23+ messages in thread From: Ian Jackson @ 2015-11-27 15:54 UTC (permalink / raw) To: Hu, Robert; +Cc: xen-devel, osstest service owner osstest service owner writes ("[xen-unstable test] 65141: regressions - FAIL"): > flight 65141 xen-unstable real [real] > http://logs.test-lab.xenproject.org/osstest/logs/65141/ > > Regressions :-( > > Tests which did not succeed and are blocking, > including tests which could not be run: > test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail REGR. vs. 65114 Hi, Robert. I hope you don't mind me asking you about these Nested HVM tests in osstest production; if there's someone else we should contact please let us know. Anyway: it seems this test failed just now. osstest thinks it is a regression, but I think it is more likely that this test either exhibits a heisenbug, or that there is some problem which is specific to the particular host. We'd appreciate it if you and your colleagues could take a look at this and analyse the failure. In the meantime the osstest bisector will try to start work on it and I will report what it discovers. Ian. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-11-27 15:54 ` Ian Jackson @ 2015-11-30 5:35 ` Hu, Robert 2015-12-02 10:34 ` Ian Campbell 0 siblings, 1 reply; 23+ messages in thread From: Hu, Robert @ 2015-11-30 5:35 UTC (permalink / raw) To: Ian Jackson; +Cc: xen-devel, osstest service owner > -----Original Message----- > From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com] > Sent: Friday, November 27, 2015 11:54 PM > To: Hu, Robert <robert.hu@intel.com> > Cc: xen-devel@lists.xensource.com; osstest service owner > <osstest-admin@xenproject.org> > Subject: Re: [xen-unstable test] 65141: regressions - FAIL > > osstest service owner writes ("[xen-unstable test] 65141: regressions - > FAIL"): > > flight 65141 xen-unstable real [real] > > http://logs.test-lab.xenproject.org/osstest/logs/65141/ > > > > Regressions :-( > > > > Tests which did not succeed and are blocking, > > including tests which could not be run: > > test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail > REGR. vs. 65114 > > Hi, Robert. I hope you don't mind me asking you about these Nested > HVM tests in osstest production; if there's someone else we should > contact please let us know. [Hu, Robert] I'm trying to look for... But can I get the bad commit first? > > Anyway: it seems this test failed just now. > > osstest thinks it is a regression, but I think it is more likely that > this test either exhibits a heisenbug, or that there is some problem > which is specific to the particular host. > > We'd appreciate it if you and your colleagues could take a look at > this and analyse the failure. > > In the meantime the osstest bisector will try to start work on it and > I will report what it discovers. > > Ian. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-11-30 5:35 ` Hu, Robert @ 2015-12-02 10:34 ` Ian Campbell 2015-12-02 10:52 ` Jan Beulich 2015-12-02 13:51 ` Ian Campbell 0 siblings, 2 replies; 23+ messages in thread From: Ian Campbell @ 2015-12-02 10:34 UTC (permalink / raw) To: Hu, Robert, Ian Jackson, Jun Nakajima, Kevin Tian Cc: Andrew Cooper, xen-devel, osstest service owner, Jan Beulich On Mon, 2015-11-30 at 05:35 +0000, Hu, Robert wrote: Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm not sure if there is a dedicated nested-vmx maintainer. This failure has been blocking the xen-unstable push gate for a week now so it really does need looking into. Also CC the arch X86 maintainers, for-their-info. > > -----Original Message----- > > From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com] > > Sent: Friday, November 27, 2015 11:54 PM > > To: Hu, Robert <robert.hu@intel.com> > > Cc: xen-devel@lists.xensource.com; osstest service owner > > <osstest-admin@xenproject.org> > > Subject: Re: [xen-unstable test] 65141: regressions - FAIL > > > > osstest service owner writes ("[xen-unstable test] 65141: regressions - > > FAIL"): > > > flight 65141 xen-unstable real [real] > > > http://logs.test-lab.xenproject.org/osstest/logs/65141/ > > > > > > Regressions :-( > > > > > > Tests which did not succeed and are blocking, > > > including tests which could not be run: > > > test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail > > REGR. vs. 65114 > > > > Hi, Robert. I hope you don't mind me asking you about these Nested > > HVM tests in osstest production; if there's someone else we should > > contact please let us know. > [Hu, Robert] > > I'm trying to look for... > But can I get the bad commit first? It is in the original mail report, which is in the ML archives: http://lists.xenproject.org/archives/html/xen-devel/2015-11/msg03222.html version targeted for testing: xen b1d398b67781140d1c6efd05778d0ad4103b2a32 baseline version: xen 713b7e4ef2aa4ec3ae697cde9c81d5a57548f9b1 You can also find it via the logs at http://logs.test-lab.xenproject.org/osstest/logs/65141 by clicking the header of one of the build-$arch jobs e.g. http://logs.test-lab.xenproject.org/osstest/logs/65141/build-amd64/info.html and looking at the various tree_* and revision_* in the "Test control variables" table, e.g.: revision_xen b1d398b67781140d1c6efd05778d0ad4103b2a32 definition > > Anyway: it seems this test failed just now. > > > > osstest thinks it is a regression, but I think it is more likely that > > this test either exhibits a heisenbug, or that there is some problem > > which is specific to the particular host. FWIW looking at the different branches in http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested-intel/ While xen-unstable has started failing on godello1 xen-4.6-testing does seem to be passing there. The history for the previous job name (before splitting into -intel and -amd) also shows a pass on godello1: http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested/xen-unstable.html Of course this is a new test so there isn't very much historical data to draw conclusions from. > > We'd appreciate it if you and your colleagues could take a look at > > this and analyse the failure. > > > > In the meantime the osstest bisector will try to start work on it and > > I will report what it discovers. According to http://osstest.test-lab.xenproject.org/~osstest/pub/results/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.html it was unable to reproduce a baseline, probably because it didn't have enough historical data. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-02 10:34 ` Ian Campbell @ 2015-12-02 10:52 ` Jan Beulich 2015-12-02 10:57 ` Andrew Cooper 2015-12-02 11:07 ` Ian Campbell 2015-12-02 13:51 ` Ian Campbell 1 sibling, 2 replies; 23+ messages in thread From: Jan Beulich @ 2015-12-02 10:52 UTC (permalink / raw) To: Ian Campbell, Ian Jackson Cc: KevinTian, xen-devel, Andrew Cooper, osstestservice owner, Jun Nakajima, Robert Hu >>> On 02.12.15 at 11:34, <ian.campbell@citrix.com> wrote: > On Mon, 2015-11-30 at 05:35 +0000, Hu, Robert wrote: > > Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm not > sure if there is a dedicated nested-vmx maintainer. This failure has been > blocking the xen-unstable push gate for a week now so it really does need > looking into. > > Also CC the arch X86 maintainers, for-their-info. I was actually waiting for the bisector to point at something. Also looking at the history this might be machine specific (the sole initial pass was on italia0, all failures are on godello1). Should new tests perhaps be capable of causing regressions only once they passed on every host they may (usefully) run on? Jan ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-02 10:52 ` Jan Beulich @ 2015-12-02 10:57 ` Andrew Cooper 2015-12-02 11:07 ` Ian Campbell 1 sibling, 0 replies; 23+ messages in thread From: Andrew Cooper @ 2015-12-02 10:57 UTC (permalink / raw) To: Jan Beulich, Ian Campbell, Ian Jackson Cc: Robert Hu, KevinTian, xen-devel, osstestservice owner, Jun Nakajima On 02/12/15 10:52, Jan Beulich wrote: >>>> On 02.12.15 at 11:34, <ian.campbell@citrix.com> wrote: >> On Mon, 2015-11-30 at 05:35 +0000, Hu, Robert wrote: >> >> Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm not >> sure if there is a dedicated nested-vmx maintainer. This failure has been >> blocking the xen-unstable push gate for a week now so it really does need >> looking into. >> >> Also CC the arch X86 maintainers, for-their-info. > I was actually waiting for the bisector to point at something. Also > looking at the history this might be machine specific (the sole > initial pass was on italia0, all failures are on godello1). Should new > tests perhaps be capable of causing regressions only once they > passed on every host they may (usefully) run on? As a general improvement, OSSTest should not make new tests blocking by default. They should need to be proved stable (5 consecutive passes?) before they become blocking, to prevent a new test spuriously passing and subsequently blocking pushes. During this time, the author of the new test has the onus to ensure test stability; either modifications to the test, or bugfixes to master. ~Andrew ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-02 10:52 ` Jan Beulich 2015-12-02 10:57 ` Andrew Cooper @ 2015-12-02 11:07 ` Ian Campbell 1 sibling, 0 replies; 23+ messages in thread From: Ian Campbell @ 2015-12-02 11:07 UTC (permalink / raw) To: Jan Beulich, Ian Jackson Cc: KevinTian, xen-devel, Andrew Cooper, osstestservice owner, Jun Nakajima, Robert Hu On Wed, 2015-12-02 at 03:52 -0700, Jan Beulich wrote: > > > > On 02.12.15 at 11:34, <ian.campbell@citrix.com> wrote: > > On Mon, 2015-11-30 at 05:35 +0000, Hu, Robert wrote: > > > > Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm > > not > > sure if there is a dedicated nested-vmx maintainer. This failure has > > been > > blocking the xen-unstable push gate for a week now so it really does > > need > > looking into. > > > > Also CC the arch X86 maintainers, for-their-info. > > I was actually waiting for the bisector to point at something. Also > looking at the history this might be machine specific (the sole > initial pass was on italia0, all failures are on godello1). Should new > tests perhaps be capable of causing regressions only once they > passed on every host they may (usefully) run on? We renamed the test (to add an -intel/-amd suffix and run on an appropriate host) so http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested/xen-unstable.html is also relevant, rimava is an amd machine (which lead to the rename/split. That older named test also shows a pass on godello1. That initial pass on godello1 might have been a fluke, but the current run of seven failures seems unlikely to be a fluke in the opposite direction. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-02 10:34 ` Ian Campbell 2015-12-02 10:52 ` Jan Beulich @ 2015-12-02 13:51 ` Ian Campbell 2015-12-03 5:58 ` Tian, Kevin 2015-12-05 8:09 ` Ian Campbell 1 sibling, 2 replies; 23+ messages in thread From: Ian Campbell @ 2015-12-02 13:51 UTC (permalink / raw) To: Hu, Robert, Ian Jackson, Jun Nakajima, Kevin Tian Cc: Andrew Cooper, xen-devel, osstest service owner, Jan Beulich On Wed, 2015-12-02 at 10:34 +0000, Ian Campbell wrote: > [...] > FWIW looking at the different branches in > http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested-intel/ > > While xen-unstable has started failing on godello1 xen-4.6-testing does > seem to be passing there. > > The history for the previous job name (before splitting into -intel and > -amd) also shows a pass on godello1: > > http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested/xen-unstable.html > > Of course this is a new test so there isn't very much historical data to > draw conclusions from. > > > > We'd appreciate it if you and your colleagues could take a look at > > > this and analyse the failure. > > > > > > In the meantime the osstest bisector will try to start work on it and > > > I will report what it discovers. > > According to > http://osstest.test-lab.xenproject.org/~osstest/pub/results/bisect/xen-un > stable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1 > --l2.html > it was unable to reproduce a baseline, probably because it didn't have > enough historical data. So I have run an adhoc test of the version of Xen tested by flight 64494, i.e. the one which passed on godello under the old job name test-amd64- amd64-qemuu-nested but using the new name and current test harness and it was successful: http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ I think that ought to give a baseline for the bisector to work with. I'll prod it to do so. Given the number of commits I expect this is going to take a little while to produce results, given that the regression is already a week old it would be good if the VT-d maintainers could investigate in parallel. The changes to xen.git in the region are below in case that flags anything to anyone. Note that other trees have changed as well though. Ian. $ git log --oneline d07f63fa6e70..b1d398b67781 b1d398b x86: allow disabling the emulated local apic 302a851 x86/vlapic: fixes for HVM code when running without a vlapic 0ce647a x86: suppress bogus log message 4243baf HVM/save: allow the usage of zeroextend and a fixup function fafa16e HVM/save: pass a size parameter to the HVM compat functions 08f3fad build: fix dependencies for files compiled from their parent directory bdead3f MAINTAINERS: change the vt-d maintainer b38d426 x86/viridian: flush remote tlbs by hypercall 713b7e4 public/event_channel.h: correct comment d380b35 x86/boot: check for not allowed sections before linking c708181 libxc: expose xsaves/xgetbv1/xsavec to hvm guest 460b9a4 x86/xsaves: enable xsaves/xrstors for hvm guest da62246 x86/xsaves: enable xsaves/xrstors/xsavec in xen 3ebe992 x86/xsaves: using named operand instead numbered operand in xrstor d9d610d build: remove .d files from xen/ on a clean 1e8193b console: make printk() line continuation tracking per-CPU 2a91f05 xen/arm: vgic-v3: Make clear that GICD_*SPI_* registers are reserved c38f9e4 xen/arm: vgic-v3: Don't implement write-only register read as zero cdae22a xen/arm: vgic-v3: Remove spurious return in GICR_INVALLR 833a693 xen/arm: vgic-v3: Emulate read to GICD_ICACTIVER<n> 5482d13 xen/arm: vgic: Re-order the register emulations to match the memory map 49b6d4c xen/arm: vgic-v3: Remove GICR_MOVALLR and GICR_MOVLPIR a2b83f9 xen/arm: vgic: Properly emulate the full register 84ce5f4 xen/arm: vgic-v3: Only emulate identification registers required by the spec cf1a142 xen/arm: vgic-v3: Use the correct offset GICR_IGRPMODR0 675b68f xen/arm: vgic-v3: Don't try to emulate IROUTER which do not exist in the spec afbbf2c xen/arm: vgic-v2: Implement correctly ICFGR{0, 1} read-only c4d6bbd xen/arm: vgic-v3: Support 32-bit access for 64-bit registers 423e9ec xen/arm: vgic: Introduce helpers to extract/update/clear/set vGIC register ... 5d495f4 xen/arm: vgic: Optimize the way to store the target vCPU in the rank e99e162 xen/arm: vgic-v2: Don't ignore a write in ITARGETSR if one field is 0 9f5e16e xen/arm: vgic-v2: Handle correctly byte write in ITARGETSR bc50de8 xen/arm: vgic-v2: Implement correctly ITARGETSR0 - ITARGETSR7 read-only ca55006 xen/arm: move ticks conversions function declarations to the header file 6c31176 arm: export platform_op XENPF_settime64 f1e5b11 xen: move wallclock functions from x86 to common 7d596f5 x86/VPMU: return correct fixed PMC count 3acb0d6 public/io/netif.h: tidy up and remove duplicate comments 8267253 public/io/netif.h: add definition of gso_prefix flag 19167b1 public/io/netif.h: document the reality of netif_rx_request/reponse 0c3f246 x86/VPMU: Initialize VPMU's lvtpc vector c03480c x86/vPMU: document as unsupported 9b43668 Merge branch 'staging' of ssh://xenbits.xen.org/home/xen/git/xen into staging 9cf5f0a x86/kexec: hide more kexec infrastructure behind CONFIG_KEXEC 0aa684b x86: drop MAX_APICID c87303c libxl: fix line wrapping issues introduced by automatic replacement 91cee73 libxl: convert libxl__sprintf(gc) to GCSPRINTF 0a62798 tools/hotplug: quote all variables in vif-bridge 26b4f4d docs: Introduce xenstore paths for guest network address information e8f98cb docs: Introduce xenstore paths for hotplug features 71e64e1 docs: Introduce xenstore paths for PV driver information 85e3b08 docs: Introduce xenstore paths for PV control features 31aa811 get_maintainer: fix perl 5.22/5.24 deprecated/incompatible "\C" use 2dd7eaf tools/libxl: Drop dead code following calls to libxl__exec() 0188415 xen/arm: use masking operation instead of test_bit for MCSF bits bb4b673 MAINTAINERS: mini-os patches should be copied to minios-devel 969eb34 MINIOS_UPSTREAM_REVISION Update 032dbba Config.mk: Update SEABIOS_UPSTREAM_TAG to rel-1.9.0 cb6be0d sched: get rid of the per domain vCPU list in Credit2 e604225 sched: get rid of the per domain vCPU list in RTDS 6b53bb4 sched: better handle (not) inserting idle vCPUs in runqueues a8c6c62 sched: clarify use cases of schedule_cpu_switch() ae2f41e sched: fix locking for insert_vcpu() in credit1 and RTDS e4fd700 x86/HVM: type adjustments 81a28f1 VMX: fix/adjust trap injection 2bbe500 ACPI 6.0: Add changes for FADT table 975efe1 acpi/NUMA: build NUMA for x86 only dee1aee VT-d: dump the posted format IRTE 853f8af vt-d: extend struct iremap_entry to support VT-d Posted-Interrupts 0ba0faa VT-d: remove pointless casts 2944689 vmx: initialize VT-d Posted-Interrupts Descriptor 03523ab vmx: add some helper functions for Posted-Interrupts 30e3f32 vmx: extend struct pi_desc to support VT-d Posted-Interrupts 1d028f7 VT-d Posted-Interrupts feature detection 9670e7d iommu: add iommu_intpost to control VT-d Posted-Interrupts feature d02e84b vVMX: use latched VMCS machine address 3b47431 VMX: allocate VMCS pages from domain heap 827db7b MINIOS_UPSTREAM_REVISION Update aec3811 tools/libxc: Correct XC_DOM_PAGE_SIZE() to return a long long 225166e libxl: correct bug in domain builder regarding page tables for pvh c35eefd x86/P2M: consolidate handling of types not requiring a valid MFN 513c203 x86/PoD: tighten conditions for checking super page 65288cf x86/IO-APIC: fix setting of destinations 5ed662e x86: fixes to LAPIC probing b9730aa ns16550: limit mapped MMIO size 7efcc58 ns16550: reset bar_64 on each iteration f2d2de5 x86: move some APIC related macros to apicdef.h e0116ac x86: add cmpxchg16b support a2b3502 blkif.h: document blkif multi-queue/ring extension _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-02 13:51 ` Ian Campbell @ 2015-12-03 5:58 ` Tian, Kevin 2015-12-03 9:25 ` Ian Campbell 2015-12-05 8:09 ` Ian Campbell 1 sibling, 1 reply; 23+ messages in thread From: Tian, Kevin @ 2015-12-03 5:58 UTC (permalink / raw) To: Ian Campbell, Hu, Robert, Ian Jackson, Nakajima, Jun Cc: Andrew Cooper, xen-devel, osstest service owner, Jan Beulich > From: Ian Campbell [mailto:ian.campbell@citrix.com] > Sent: Wednesday, December 02, 2015 9:51 PM > > > > According to > > http://osstest.test-lab.xenproject.org/~osstest/pub/results/bisect/xen-un > > stable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1 > > --l2.html > > it was unable to reproduce a baseline, probably because it didn't have > > enough historical data. > > So I have run an adhoc test of the version of Xen tested by flight 64494, > i.e. the one which passed on godello under the old job name test-amd64- > amd64-qemuu-nested but using the new name and current test harness and it > was successful: > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > I think that ought to give a baseline for the bisector to work with. I'll > prod it to do so. > > Given the number of commits I expect this is going to take a little while > to produce results, given that the regression is already a week old it > would be good if the VT-d maintainers could investigate in parallel. Is the test case VT-d related? From earlier thread looks it's related to nested virtualization? I can't access above link. If you can help describe the test steps in plain text, I can check whether we can reproduce locally. Thanks Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-03 5:58 ` Tian, Kevin @ 2015-12-03 9:25 ` Ian Campbell 0 siblings, 0 replies; 23+ messages in thread From: Ian Campbell @ 2015-12-03 9:25 UTC (permalink / raw) To: Tian, Kevin, Hu, Robert, Ian Jackson, Nakajima, Jun Cc: Andrew Cooper, xen-devel, osstest service owner, Jan Beulich On Thu, 2015-12-03 at 05:58 +0000, Tian, Kevin wrote: > > From: Ian Campbell [mailto:ian.campbell@citrix.com] > > Sent: Wednesday, December 02, 2015 9:51 PM > > > > > > According to > > > http://osstest.test-lab.xenproject.org/~osstest/pub/results/bisect/xe > > > n-un > > > stable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1 > > > --l2.html > > > it was unable to reproduce a baseline, probably because it didn't > > > have > > > enough historical data. > > > > So I have run an adhoc test of the version of Xen tested by flight > > 64494, > > i.e. the one which passed on godello under the old job name test-amd64- > > amd64-qemuu-nested but using the new name and current test harness and > > it > > was successful: > > > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > > > I think that ought to give a baseline for the bisector to work with. > > I'll > > prod it to do so. > > > > Given the number of commits I expect this is going to take a little > > while > > to produce results, given that the regression is already a week old it > > would be good if the VT-d maintainers could investigate in parallel. > > Is the test case VT-d related? From earlier thread looks it's related to > nested virtualization? It's the nested-vmx/VT-x test case which Intel contributed to osstest over the last few months. AFAIK VT-d is not involved in any way (no passthrough etc). > I can't access above link. If you can help describe the test steps in > plain text, I can check whether we can reproduce locally. Sorry, I hadn't spotted that the adhoc link was internal to the colo when I c&p'd it. This test case has failed in the last half-dozen real xen-unstable flights, including the one which started this thread and the latest one which is at http://logs.test-lab.xenproject.org/osstest/logs/65287/test-amd64-amd64-qemuu-nested-intel/info.html The test case is installing Xen in an HVM guest to produce an L1 host and then trying to boot an L2 HVM guest within that L1, which fails. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-02 13:51 ` Ian Campbell 2015-12-03 5:58 ` Tian, Kevin @ 2015-12-05 8:09 ` Ian Campbell 2015-12-07 16:18 ` Jan Beulich 2015-12-08 8:06 ` Hu, Robert 1 sibling, 2 replies; 23+ messages in thread From: Ian Campbell @ 2015-12-05 8:09 UTC (permalink / raw) To: Hu, Robert, Ian Jackson, Jun Nakajima, Kevin Tian Cc: Andrew Cooper, xen-devel, osstest service owner, Jan Beulich On Wed, 2015-12-02 at 13:51 +0000, Ian Campbell wrote: > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > I think that ought to give a baseline for the bisector to work with. I'll > prod it to do so. Results are below. TL;DR: d02e84b9d9d "vVMX: use latched VMCS machine address" is somehow at fault. It appears to be somewhat machine specific, the one this has been failing on is godello* which says "CPU0: Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz stepping 03" in its serial log. Andy suggested this might be related to cpu_has_vmx_vmcs_shadowing so Haswell and newer vs IvyBridge and older. Ian. branch xen-unstable xenbranch xen-unstable job test-amd64-amd64-qemuu-nested-intel testid debian-hvm-install/l1/l2 Tree: linux git://xenbits.xen.org/linux-pvops.git Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git Tree: qemuu git://xenbits.xen.org/qemu-xen.git Tree: xen git://xenbits.xen.org/xen.git *** Found and reproduced problem changeset *** Bug is in tree: xen git://xenbits.xen.org/xen.git Bug introduced: d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 Bug not present: 3b47431691409004c7218f6a6ba5c9c0bcf483ea Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/65388/ commit d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 Author: Jan Beulich <jbeulich@suse.com> Date: Tue Nov 24 12:07:27 2015 +0100 vVMX: use latched VMCS machine address Instead of calling domain_page_map_to_mfn() over and over, latch the guest VMCS machine address unconditionally (i.e. independent of whether VMCS shadowing is supported by the hardware). Since this requires altering the parameters of __[gs]et_vmcs{,_real}() (and hence all their callers) anyway, take the opportunity to also drop the bogus double underscores from their names (and from __[gs]et_vmcs_virtual() as well). Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Kevin Tian <kevin.tian@intel.com> For bisection revision-tuple graph see: http://logs.test-lab.xenproject.org/osstest/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.html Revision IDs in each graph node refer, respectively, to the Trees above. ---------------------------------------- Running cs-bisection-step --graph-out=/home/logs/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2 --summary-out=tmp/65388.bisection-summary --blessings=real,real-bisect,adhoc-bisect --basis-template=65301 --basis-flight=65301 xen-unstable test-amd64-amd64-qemuu-nested-intel debian-hvm-install/l1/l2 Searching for failure / basis pass: 65314 fail [host=godello1] / template as basis? using template as basis. Failure / basis pass flights: 65314 / 65301 (tree with no url: ovmf) (tree with no url: seabios) Tree: linux git://xenbits.xen.org/linux-pvops.git Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git Tree: qemuu git://xenbits.xen.org/qemu-xen.git Tree: xen git://xenbits.xen.org/xen.git Latest 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 2c4f313a7e62c7e559a469d4af4c3d03c49afa43 Basis pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 816609b2841297925a223ec377c336360e044ee5 d07f63fa6e70350b23e7acbde06129247c4e655d Generating revisions with ./adhoc-revtuple-generator git://xenbits.xen.org/linux-pvops.git#1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9-769b79eb206ad5b0249a08665fefb913c3d1998e git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 git://xenbits.xen.org/qemu-xen-traditional.git#bc00cad75d8bcc3ba696992bec219c21db8406aa-bc00cad75d8bcc3ba696992bec219c21db8406aa git://xenbits.xen.org/qemu-xen.git#816609b2841297925a223ec377c336360e044ee5-3fb401edbd8e9741c611bfddf6a2032ca91f55ed git://xenbits.xen.org/xen.git#d07f63fa6e70350b23e7acbde06129247c4e655d-2c4f313a7e62c7e559a469d4af4c3d03c49afa43 Loaded 17133 nodes in revision graph Searching for test results: 65114 [host=italia0] 65141 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed b1d398b67781140d1c6efd05778d0ad4103b2a32 65162 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 713b7e4ef2aa4ec3ae697cde9c81d5a57548f9b1 65164 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed b1d398b67781140d1c6efd05778d0ad4103b2a32 65186 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed b1d398b67781140d1c6efd05778d0ad4103b2a32 65217 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed b1d398b67781140d1c6efd05778d0ad4103b2a32 65233 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed b1d398b67781140d1c6efd05778d0ad4103b2a32 65287 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed c8eb0ec277ae387e78d685523e0fee633e46f046 65333 pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 816609b2841297925a223ec377c336360e044ee5 d07f63fa6e70350b23e7acbde06129247c4e655d 65314 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 2c4f313a7e62c7e559a469d4af4c3d03c49afa43 65354 pass 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 816609b2841297925a223ec377c336360e044ee5 c35eefded2992fc9b979f99190422527650872fd 65267 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 4c6cd64519f9bc270a7278128c94e4b66e3d2077 65301 pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 816609b2841297925a223ec377c336360e044ee5 d07f63fa6e70350b23e7acbde06129247c4e655d 65351 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 0ba0faa7f02fb310a330579adeef3534e6eca5a8 65343 pass 997badf1e03dfd39854094f7767e1a4cf5ed310b c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 816609b2841297925a223ec377c336360e044ee5 d07f63fa6e70350b23e7acbde06129247c4e655d 65325 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed c8eb0ec277ae387e78d685523e0fee633e46f046 65345 pass e7a10d9297c1abfd27138d86a13f7b6435634a46 c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 816609b2841297925a223ec377c336360e044ee5 d07f63fa6e70350b23e7acbde06129247c4e655d 65341 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 2c4f313a7e62c7e559a469d4af4c3d03c49afa43 65342 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed c87303c04738b0e837da6e891eb561de0bf1b64e 65362 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 65367 pass 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 816609b2841297925a223ec377c336360e044ee5 827db7b26384ce083df7154d77f13379b2cf4121 65370 pass 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 827db7b26384ce083df7154d77f13379b2cf4121 65374 pass 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 3b47431691409004c7218f6a6ba5c9c0bcf483ea 65379 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 65380 pass 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 3b47431691409004c7218f6a6ba5c9c0bcf483ea 65383 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 65384 pass 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 3b47431691409004c7218f6a6ba5c9c0bcf483ea 65388 fail 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 Searching for interesting versions Result found: flight 65301 (pass), for basis pass Result found: flight 65314 (fail), for basis failure Repro found: flight 65333 (pass), for basis pass Repro found: flight 65341 (fail), for basis failure 0 revisions at 769b79eb206ad5b0249a08665fefb913c3d1998e c530a75c1e6a472b0eb9558310b518f0dfcd8860 bc00cad75d8bcc3ba696992bec219c21db8406aa 3fb401edbd8e9741c611bfddf6a2032ca91f55ed 3b47431691409004c7218f6a6ba5c9c0bcf483ea No revisions left to test, checking graph state. Result found: flight 65374 (pass), for last pass Result found: flight 65379 (fail), for first failure Repro found: flight 65380 (pass), for last pass Repro found: flight 65383 (fail), for first failure Repro found: flight 65384 (pass), for last pass Repro found: flight 65388 (fail), for first failure *** Found and reproduced problem changeset *** Bug is in tree: xen git://xenbits.xen.org/xen.git Bug introduced: d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 Bug not present: 3b47431691409004c7218f6a6ba5c9c0bcf483ea Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/65388/ commit d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 Author: Jan Beulich <jbeulich@suse.com> Date: Tue Nov 24 12:07:27 2015 +0100 vVMX: use latched VMCS machine address Instead of calling domain_page_map_to_mfn() over and over, latch the guest VMCS machine address unconditionally (i.e. independent of whether VMCS shadowing is supported by the hardware). Since this requires altering the parameters of __[gs]et_vmcs{,_real}() (and hence all their callers) anyway, take the opportunity to also drop the bogus double underscores from their names (and from __[gs]et_vmcs_virtual() as well). Signed-off-by: Jan Beulich <jbeulich@suse.com> Acked-by: Kevin Tian <kevin.tian@intel.com> Revision graph left in /home/logs/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.{dot,ps,png,html,svg}. ---------------------------------------- 65388: tolerable ALL FAIL flight 65388 xen-unstable adhoc-bisect [real] http://logs.test-lab.xenproject.org/osstest/logs/65388/ Failures :-/ but no regressions. Tests which did not succeed, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail baseline untested jobs: test-amd64-amd64-qemuu-nested-intel fail ------------------------------------------------------------ sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-05 8:09 ` Ian Campbell @ 2015-12-07 16:18 ` Jan Beulich 2015-12-07 16:28 ` Ian Campbell 2015-12-08 2:46 ` Tian, Kevin 2015-12-08 8:06 ` Hu, Robert 1 sibling, 2 replies; 23+ messages in thread From: Jan Beulich @ 2015-12-07 16:18 UTC (permalink / raw) To: Ian Campbell, Ian Jackson, Jun Nakajima, KevinTian, Robert Hu Cc: Andrew Cooper, xen-devel, osstestservice owner >>> On 05.12.15 at 09:09, <ian.campbell@citrix.com> wrote: > On Wed, 2015-12-02 at 13:51 +0000, Ian Campbell wrote: > >> http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ >> >> I think that ought to give a baseline for the bisector to work with. I'll >> prod it to do so. > > Results are below. TL;DR: d02e84b9d9d "vVMX: use latched VMCS machine > address" is somehow at fault. > > It appears to be somewhat machine specific, the one this has been > failing on is godello* which says "CPU0: Intel(R) Xeon(R) CPU E3-1220 > v3 @ 3.10GHz stepping 03" in its serial log. > > Andy suggested this might be related to cpu_has_vmx_vmcs_shadowing > so Haswell and newer vs IvyBridge and older. Yeah, but on irc it was also made clear that the regression is on a system without that capability. At this point we certainly need to seriously consider reverting the whole change. The reason I continue to be hesitant is that I'm afraid this may result in no-one trying to find out what the problem here is. While I could certainly try to, I'm sure I won't find time to do so within the foreseeable future. And since we didn't get any real feedback from Intel so far, I thought I'd ping them to at least share some status before we decide. That pinging has happened a few minutes ago. I'd therefore like to give it, say, another day, and if by then we don't have an estimate for when a fix might become available, I'd do the revert. Unless of course somebody feels strongly about doing the revert immediately. Jan ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-07 16:18 ` Jan Beulich @ 2015-12-07 16:28 ` Ian Campbell 2015-12-07 16:48 ` Jan Beulich 2015-12-08 2:46 ` Tian, Kevin 1 sibling, 1 reply; 23+ messages in thread From: Ian Campbell @ 2015-12-07 16:28 UTC (permalink / raw) To: Jan Beulich, Ian Jackson, Jun Nakajima, KevinTian, Robert Hu Cc: Andrew Cooper, xen-devel, osstestservice owner On Mon, 2015-12-07 at 09:18 -0700, Jan Beulich wrote: > > > > On 05.12.15 at 09:09, <ian.campbell@citrix.com> wrote: > > On Wed, 2015-12-02 at 13:51 +0000, Ian Campbell wrote: > > > > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > > > > > I think that ought to give a baseline for the bisector to work with. > > > I'll > > > prod it to do so. > > > > Results are below. TL;DR: d02e84b9d9d "vVMX: use latched VMCS machine > > address" is somehow at fault. > > > > It appears to be somewhat machine specific, the one this has been > > failing on is godello* which says "CPU0: Intel(R) Xeon(R) CPU E3-1220 > > v3 @ 3.10GHz stepping 03" in its serial log. > > > > Andy suggested this might be related to cpu_has_vmx_vmcs_shadowing > > so Haswell and newer vs IvyBridge and older. > > Yeah, but on irc it was also made clear that the regression is on a > system without that capability. What I was trying to say he said was that the difference between working and broken hosts might be spread along the lines of >=Haswell vs <=IvyBridge. How that maps onto E3-1220, which is what is exhibiting the issue, I leave to you guys. > At this point we certainly need to seriously consider reverting the > whole change. The reason I continue to be hesitant is that I'm > afraid this may result in no-one trying to find out what the problem > here is. While I could certainly try to, I'm sure I won't find time to > do so within the foreseeable future. And since we didn't get any > real feedback from Intel so far, I thought I'd ping them to at least > share some status before we decide. That pinging has happened > a few minutes ago. I'd therefore like to give it, say, another day, > and if by then we don't have an estimate for when a fix might > become available, I'd do the revert. Unless of course somebody > feels strongly about doing the revert immediately. I don't mind waiting. One approach to fixing might be to disentangle the various things which this patch did, such that the actual culprit is a smaller thing to analyse. Ian. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-07 16:28 ` Ian Campbell @ 2015-12-07 16:48 ` Jan Beulich 0 siblings, 0 replies; 23+ messages in thread From: Jan Beulich @ 2015-12-07 16:48 UTC (permalink / raw) To: Ian Campbell Cc: KevinTian, xen-devel, Andrew Cooper, Ian Jackson, osstestservice owner, Jun Nakajima, Robert Hu >>> On 07.12.15 at 17:28, <ian.campbell@citrix.com> wrote: > One approach to fixing might be to disentangle the various things which > this patch did, such that the actual culprit is a smaller thing to analyse. Yes, if we're going to revert, I'll try to do something in that direction. I don't expect there to be too much room for splitting though. Jan ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-07 16:18 ` Jan Beulich 2015-12-07 16:28 ` Ian Campbell @ 2015-12-08 2:46 ` Tian, Kevin 2015-12-08 7:28 ` Jan Beulich 1 sibling, 1 reply; 23+ messages in thread From: Tian, Kevin @ 2015-12-08 2:46 UTC (permalink / raw) To: Jan Beulich, Ian Campbell, Ian Jackson, Nakajima, Jun, Hu, Robert Cc: Andrew Cooper, xen-devel, osstestservice owner > From: Jan Beulich [mailto:JBeulich@suse.com] > Sent: Tuesday, December 08, 2015 12:18 AM > > >>> On 05.12.15 at 09:09, <ian.campbell@citrix.com> wrote: > > On Wed, 2015-12-02 at 13:51 +0000, Ian Campbell wrote: > > > >> http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > >> > >> I think that ought to give a baseline for the bisector to work with. I'll > >> prod it to do so. > > > > Results are below. TL;DR: d02e84b9d9d "vVMX: use latched VMCS machine > > address" is somehow at fault. > > > > It appears to be somewhat machine specific, the one this has been > > failing on is godello* which says "CPU0: Intel(R) Xeon(R) CPU E3-1220 > > v3 @ 3.10GHz stepping 03" in its serial log. > > > > Andy suggested this might be related to cpu_has_vmx_vmcs_shadowing > > so Haswell and newer vs IvyBridge and older. > > Yeah, but on irc it was also made clear that the regression is on a > system without that capability. > > At this point we certainly need to seriously consider reverting the > whole change. The reason I continue to be hesitant is that I'm > afraid this may result in no-one trying to find out what the problem > here is. While I could certainly try to, I'm sure I won't find time to > do so within the foreseeable future. And since we didn't get any > real feedback from Intel so far, I thought I'd ping them to at least > share some status before we decide. That pinging has happened > a few minutes ago. I'd therefore like to give it, say, another day, > and if by then we don't have an estimate for when a fix might > become available, I'd do the revert. Unless of course somebody > feels strongly about doing the revert immediately. > I didn't see an obvious error from the commit, so some debug would be required to identify the problematic code. However this issue was not reproduced immediately in our internal environment, and the guy familiar with this area (Yang) just left Intel. It takes some time to identify a new developer and get him ramped up to fix issues in this area. Given that fact, I'd suggest to revert related code now (as you discussed not the whole commit). In parallel we'll find someone to look at original commit as a rampup task. Thanks Kevin ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-08 2:46 ` Tian, Kevin @ 2015-12-08 7:28 ` Jan Beulich 0 siblings, 0 replies; 23+ messages in thread From: Jan Beulich @ 2015-12-08 7:28 UTC (permalink / raw) To: Kevin Tian Cc: xen-devel, Ian Campbell, Andrew Cooper, Ian Jackson, osstestservice owner, Jun Nakajima, Robert Hu >>> On 08.12.15 at 03:46, <kevin.tian@intel.com> wrote: > I didn't see an obvious error from the commit, so some debug > would be required to identify the problematic code. However this > issue was not reproduced immediately in our internal environment, > and the guy familiar with this area (Yang) just left Intel. It takes > some time to identify a new developer and get him ramped up to > fix issues in this area. Given that fact, I'd suggest to revert related > code now (as you discussed not the whole commit). In parallel > we'll find someone to look at original commit as a rampup task. Thanks Kevin! Jan ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-05 8:09 ` Ian Campbell 2015-12-07 16:18 ` Jan Beulich @ 2015-12-08 8:06 ` Hu, Robert 2015-12-08 10:29 ` Ian Campbell 1 sibling, 1 reply; 23+ messages in thread From: Hu, Robert @ 2015-12-08 8:06 UTC (permalink / raw) To: Ian Campbell, Ian Jackson, Nakajima, Jun, Tian, Kevin Cc: xen-devel, Andrew Cooper, osstest service owner, Wang, Yong Y, Jan Beulich, Jin, Gordon > -----Original Message----- > From: Ian Campbell [mailto:ian.campbell@citrix.com] > Sent: Saturday, December 5, 2015 4:10 PM > To: Hu, Robert <robert.hu@intel.com>; Ian Jackson > <Ian.Jackson@eu.citrix.com>; Nakajima, Jun <jun.nakajima@intel.com>; > Tian, Kevin <kevin.tian@intel.com> > Cc: xen-devel@lists.xensource.com; osstest service owner > <osstest-admin@xenproject.org>; Jan Beulich <jbeulich@suse.com>; > Andrew Cooper <andrew.cooper3@citrix.com> > Subject: Re: [xen-unstable test] 65141: regressions - FAIL > > On Wed, 2015-12-02 at 13:51 +0000, Ian Campbell wrote: > > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > > > I think that ought to give a baseline for the bisector to work with. I'll > > prod it to do so. > > Results are below. TL;DR: d02e84b9d9d "vVMX: use latched VMCS machine > address" is somehow at fault. > > It appears to be somewhat machine specific, the one this has been > failing on is godello* which says "CPU0: Intel(R) Xeon(R) CPU E3-1220 > v3 @ 3.10GHz stepping 03" in its serial log. > > Andy suggested this might be related to cpu_has_vmx_vmcs_shadowing > so Haswell and newer vs IvyBridge and older. > > Ian. > > branch xen-unstable > xenbranch xen-unstable > job test-amd64-amd64-qemuu-nested-intel > testid debian-hvm-install/l1/l2 > > Tree: linux git://xenbits.xen.org/linux-pvops.git > Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git > Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git > Tree: qemuu git://xenbits.xen.org/qemu-xen.git > Tree: xen git://xenbits.xen.org/xen.git > > *** Found and reproduced problem changeset *** > > Bug is in tree: xen git://xenbits.xen.org/xen.git > Bug introduced: d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 > Bug not present: 3b47431691409004c7218f6a6ba5c9c0bcf483ea > Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/65388/ > > > commit d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 > Author: Jan Beulich <jbeulich@suse.com> > Date: Tue Nov 24 12:07:27 2015 +0100 > > vVMX: use latched VMCS machine address > > Instead of calling domain_page_map_to_mfn() over and over, latch > the > guest VMCS machine address unconditionally (i.e. independent of > whether > VMCS shadowing is supported by the hardware). > > Since this requires altering the parameters of __[gs]et_vmcs{,_real}() > (and hence all their callers) anyway, take the opportunity to also drop > the bogus double underscores from their names (and from > __[gs]et_vmcs_virtual() as well). > > Signed-off-by: Jan Beulich <jbeulich@suse.com> > Acked-by: Kevin Tian <kevin.tian@intel.com> > > > For bisection revision-tuple graph see: > > http://logs.test-lab.xenproject.org/osstest/results-adhoc/bisect/xen-unstabl > e/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.html > Revision IDs in each graph node refer, respectively, to the Trees above. > > ---------------------------------------- > Running cs-bisection-step > --graph-out=/home/logs/results-adhoc/bisect/xen-unstable/test-amd64-am > d64-qemuu-nested-intel.debian-hvm-install--l1--l2 > --summary-out=tmp/65388.bisection-summary > --blessings=real,real-bisect,adhoc-bisect --basis-template=65301 > --basis-flight=65301 xen-unstable test-amd64-amd64-qemuu-nested-intel > debian-hvm-install/l1/l2 > Searching for failure / basis pass: > 65314 fail [host=godello1] / template as basis? using template as basis. > Failure / basis pass flights: 65314 / 65301 > (tree with no url: ovmf) > (tree with no url: seabios) > Tree: linux git://xenbits.xen.org/linux-pvops.git > Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git > Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git > Tree: qemuu git://xenbits.xen.org/qemu-xen.git > Tree: xen git://xenbits.xen.org/xen.git > Latest 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 2c4f313a7e62c7e559a469d4af4c3d03c49afa43 > Basis pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 816609b2841297925a223ec377c336360e044ee5 > d07f63fa6e70350b23e7acbde06129247c4e655d > Generating revisions with ./adhoc-revtuple-generator > git://xenbits.xen.org/linux-pvops.git#1230ae0e99e05ced8a945a1a2c5762ce > 5c6c97c9-769b79eb206ad5b0249a08665fefb913c3d1998e > git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb955 > 8310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 > git://xenbits.xen.org/qemu-xen-traditional.git#bc00cad75d8bcc3ba696992b > ec219c21db8406aa-bc00cad75d8bcc3ba696992bec219c21db8406aa > git://xenbits.xen.org/qemu-xen.git#816609b2841297925a223ec377c336360 > e044ee5-3fb401edbd8e9741c611bfddf6a2032ca91f55ed > git://xenbits.xen.org/xen.git#d07f63fa6e70350b23e7acbde06129247c4e655 > d-2c4f313a7e62c7e559a469d4af4c3d03c49afa43 > Loaded 17133 nodes in revision graph > Searching for test results: > 65114 [host=italia0] > 65141 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > b1d398b67781140d1c6efd05778d0ad4103b2a32 > 65162 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 713b7e4ef2aa4ec3ae697cde9c81d5a57548f9b1 > 65164 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > b1d398b67781140d1c6efd05778d0ad4103b2a32 > 65186 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > b1d398b67781140d1c6efd05778d0ad4103b2a32 > 65217 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > b1d398b67781140d1c6efd05778d0ad4103b2a32 > 65233 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > b1d398b67781140d1c6efd05778d0ad4103b2a32 > 65287 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > c8eb0ec277ae387e78d685523e0fee633e46f046 > 65333 pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 816609b2841297925a223ec377c336360e044ee5 > d07f63fa6e70350b23e7acbde06129247c4e655d > 65314 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 2c4f313a7e62c7e559a469d4af4c3d03c49afa43 > 65354 pass 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 816609b2841297925a223ec377c336360e044ee5 > c35eefded2992fc9b979f99190422527650872fd > 65267 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 4c6cd64519f9bc270a7278128c94e4b66e3d2077 > 65301 pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 816609b2841297925a223ec377c336360e044ee5 > d07f63fa6e70350b23e7acbde06129247c4e655d > 65351 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 0ba0faa7f02fb310a330579adeef3534e6eca5a8 > 65343 pass 997badf1e03dfd39854094f7767e1a4cf5ed310b > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 816609b2841297925a223ec377c336360e044ee5 > d07f63fa6e70350b23e7acbde06129247c4e655d > 65325 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > c8eb0ec277ae387e78d685523e0fee633e46f046 > 65345 pass e7a10d9297c1abfd27138d86a13f7b6435634a46 > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 816609b2841297925a223ec377c336360e044ee5 > d07f63fa6e70350b23e7acbde06129247c4e655d > 65341 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 2c4f313a7e62c7e559a469d4af4c3d03c49afa43 > 65342 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > c87303c04738b0e837da6e891eb561de0bf1b64e > 65362 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 > 65367 pass 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 816609b2841297925a223ec377c336360e044ee5 > 827db7b26384ce083df7154d77f13379b2cf4121 > 65370 pass 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 827db7b26384ce083df7154d77f13379b2cf4121 > 65374 pass 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 3b47431691409004c7218f6a6ba5c9c0bcf483ea > 65379 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 > 65380 pass 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 3b47431691409004c7218f6a6ba5c9c0bcf483ea > 65383 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 > 65384 pass 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 3b47431691409004c7218f6a6ba5c9c0bcf483ea > 65388 fail 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 > Searching for interesting versions > Result found: flight 65301 (pass), for basis pass > Result found: flight 65314 (fail), for basis failure > Repro found: flight 65333 (pass), for basis pass > Repro found: flight 65341 (fail), for basis failure > 0 revisions at 769b79eb206ad5b0249a08665fefb913c3d1998e > c530a75c1e6a472b0eb9558310b518f0dfcd8860 > bc00cad75d8bcc3ba696992bec219c21db8406aa > 3fb401edbd8e9741c611bfddf6a2032ca91f55ed > 3b47431691409004c7218f6a6ba5c9c0bcf483ea > No revisions left to test, checking graph state. > Result found: flight 65374 (pass), for last pass > Result found: flight 65379 (fail), for first failure > Repro found: flight 65380 (pass), for last pass > Repro found: flight 65383 (fail), for first failure > Repro found: flight 65384 (pass), for last pass > Repro found: flight 65388 (fail), for first failure > > *** Found and reproduced problem changeset *** > > Bug is in tree: xen git://xenbits.xen.org/xen.git > Bug introduced: d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 > Bug not present: 3b47431691409004c7218f6a6ba5c9c0bcf483ea > Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/65388/ > > > commit d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3 > Author: Jan Beulich <jbeulich@suse.com> > Date: Tue Nov 24 12:07:27 2015 +0100 > > vVMX: use latched VMCS machine address > > Instead of calling domain_page_map_to_mfn() over and over, latch > the > guest VMCS machine address unconditionally (i.e. independent of > whether > VMCS shadowing is supported by the hardware). > > Since this requires altering the parameters of __[gs]et_vmcs{,_real}() > (and hence all their callers) anyway, take the opportunity to also drop > the bogus double underscores from their names (and from > __[gs]et_vmcs_virtual() as well). > > Signed-off-by: Jan Beulich <jbeulich@suse.com> > Acked-by: Kevin Tian <kevin.tian@intel.com> > > Revision graph left in > /home/logs/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-n > ested-intel.debian-hvm-install--l1--l2.{dot,ps,png,html,svg}. > ---------------------------------------- > 65388: tolerable ALL FAIL > > flight 65388 xen-unstable adhoc-bisect [real] > http://logs.test-lab.xenproject.org/osstest/logs/65388/ > > Failures :-/ but no regressions. > > Tests which did not succeed, > including tests which could not be run: > test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail > baseline untested > > > jobs: > test-amd64-amd64-qemuu-nested-intel fail [Hu, Robert] I tried to reproduce the failure in my environment, with your Qemu/Xen/Dom0 /Seabios/OVMF designation, but failed at even earlier stage -- normal L1 guest installation. xen be: vkbd-0: initialise() failed xen be: vkbd-0: initialise() failed xen be: vkbd-0: initialise() failed Not sure if this related to linux-pvops tree, for I previously using linux-stable tree. For your failure, as Kevin mentioned in other mail, we will find someone to look into. Would you find out the detailed log of ' debian-hvm-install--l1--l2' step? so that he can start analyze, as I cannot reproduce it right now. (I didn't managed to find out the log in your web links) > > > ------------------------------------------------------------ > sg-report-flight on osstest.test-lab.xenproject.org > logs: /home/logs/logs > images: /home/logs/images > > Logs, config files, etc. are available at > http://logs.test-lab.xenproject.org/osstest/logs > > Explanation of these reports, and of osstest in general, is at > > http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=m > aster > > http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master > > Test harness code can be found at > http://xenbits.xen.org/gitweb?p=osstest.git;a=summary ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-08 8:06 ` Hu, Robert @ 2015-12-08 10:29 ` Ian Campbell 2015-12-09 6:27 ` Robert Hu 2015-12-11 4:04 ` Robert Hu 0 siblings, 2 replies; 23+ messages in thread From: Ian Campbell @ 2015-12-08 10:29 UTC (permalink / raw) To: Hu, Robert, Ian Jackson, Nakajima, Jun, Tian, Kevin Cc: xen-devel, Andrew Cooper, osstest service owner, Wang, Yong Y, Jan Beulich, Jin, Gordon On Tue, 2015-12-08 at 08:06 +0000, Hu, Robert wrote: > > > [...] Please trim your quotes. > For your failure, as Kevin mentioned in other mail, we will find someone > to look into. > Would you find out the detailed log of ' debian-hvm-install--l1--l2' > step? so that he > can start analyze, as I cannot reproduce it right now. > (I didn't managed to find out the log in your web links) From http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ (in this thread, or any other relevant logs/NNNNN/ link), you can click the "test -amd64 -amd64 -qemuu -nested -intel" header to go to http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/test-amd64-amd64-qemuu-nested-intel/info.html Then click the result of the "debian-hvm-install/l1/l2" step from that table to go to the step log: http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/test-amd64-amd64-qemuu-nested-intel/16.ts-debian-hvm-install.log Which I think is what you were after? All of the other related logs are at the end of the same http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/test-amd64-amd64-qemuu-nested-intel/info.html page. See also README in osstest.git which explains the structure of the log pages/grids/etc. Ian. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-08 10:29 ` Ian Campbell @ 2015-12-09 6:27 ` Robert Hu 2015-12-09 8:35 ` Jin, Gordon 2015-12-11 4:04 ` Robert Hu 1 sibling, 1 reply; 23+ messages in thread From: Robert Hu @ 2015-12-09 6:27 UTC (permalink / raw) To: Ian Campbell Cc: Tian, Kevin, xen-devel, Jan Beulich, guangrong.xiao, Andrew Cooper, Ian Jackson, osstest service owner, Wang, Yong Y, Nakajima, Jun, Jin, Gordon, Hu, Robert On Tue, 2015-12-08 at 10:29 +0000, Ian Campbell wrote: > On Tue, 2015-12-08 at 08:06 +0000, Hu, Robert wrote: > > > > > [...] > > Please trim your quotes. > > > For your failure, as Kevin mentioned in other mail, we will find someone > > to look into. > > Would you find out the detailed log of ' debian-hvm-install--l1--l2' > > step? so that he > > can start analyze, as I cannot reproduce it right now. > > (I didn't managed to find out the log in your web links) > > From > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > [...] OK, thanks Ian. I'd like to confirm if this can be reproduced on Ivybridge? I'm now trying to set up environment for our developer to dig into. > > Ian. > > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-09 6:27 ` Robert Hu @ 2015-12-09 8:35 ` Jin, Gordon 2015-12-09 10:21 ` Ian Campbell 0 siblings, 1 reply; 23+ messages in thread From: Jin, Gordon @ 2015-12-09 8:35 UTC (permalink / raw) To: Hu, Robert, Ian Campbell Cc: Tian, Kevin, xen-devel, Nakajima, Jun, Xiao, Guangrong, Andrew Cooper, Ian Jackson, osstest service owner, Wang, Yong Y, Jan Beulich > -----Original Message----- > From: Robert Hu [mailto:robert.hu@vmm.sh.intel.com] > Sent: Wednesday, December 09, 2015 2:27 PM > To: Ian Campbell > Cc: Hu, Robert; Ian Jackson; Nakajima, Jun; Tian, Kevin; > xen-devel@lists.xensource.com; osstest service owner; Jan Beulich; Andrew > Cooper; Wang, Yong Y; Jin, Gordon; Xiao, Guangrong > Subject: Re: [xen-unstable test] 65141: regressions - FAIL > > On Tue, 2015-12-08 at 10:29 +0000, Ian Campbell wrote: > > On Tue, 2015-12-08 at 08:06 +0000, Hu, Robert wrote: > > > > > > > [...] > > > > Please trim your quotes. > > > > > For your failure, as Kevin mentioned in other mail, we will find > > > someone to look into. > > > Would you find out the detailed log of ' debian-hvm-install--l1--l2' > > > step? so that he > > > can start analyze, as I cannot reproduce it right now. > > > (I didn't managed to find out the log in your web links) > > > > From > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > > > [...] > > OK, thanks Ian. > > I'd like to confirm if this can be reproduced on Ivybridge? I'm now trying to set up > environment for our developer to dig into. I checked the name convention, and found Xeon E3-1220 v3 is Haswell. Thanks Gordon ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-09 8:35 ` Jin, Gordon @ 2015-12-09 10:21 ` Ian Campbell 0 siblings, 0 replies; 23+ messages in thread From: Ian Campbell @ 2015-12-09 10:21 UTC (permalink / raw) To: Jin, Gordon, Hu, Robert Cc: Tian, Kevin, xen-devel, Nakajima, Jun, Xiao, Guangrong, Andrew Cooper, Ian Jackson, osstest service owner, Wang, Yong Y, Jan Beulich On Wed, 2015-12-09 at 08:35 +0000, Jin, Gordon wrote: > > -----Original Message----- > > From: Robert Hu [mailto:robert.hu@vmm.sh.intel.com] > > Sent: Wednesday, December 09, 2015 2:27 PM > > To: Ian Campbell > > Cc: Hu, Robert; Ian Jackson; Nakajima, Jun; Tian, Kevin; > > xen-devel@lists.xensource.com; osstest service owner; Jan Beulich; > > Andrew > > Cooper; Wang, Yong Y; Jin, Gordon; Xiao, Guangrong > > Subject: Re: [xen-unstable test] 65141: regressions - FAIL > > > > On Tue, 2015-12-08 at 10:29 +0000, Ian Campbell wrote: > > > On Tue, 2015-12-08 at 08:06 +0000, Hu, Robert wrote: > > > > > > > > > [...] > > > > > > Please trim your quotes. > > > > > > > For your failure, as Kevin mentioned in other mail, we will find > > > > someone to look into. > > > > Would you find out the detailed log of ' debian-hvm-install--l1 > > > > --l2' > > > > step? so that he > > > > can start analyze, as I cannot reproduce it right now. > > > > (I didn't managed to find out the log in your web links) > > > > > > From > > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > > > > > [...] > > > > OK, thanks Ian. > > > > I'd like to confirm if this can be reproduced on Ivybridge? I'm now trying to set up > > environment for our developer to dig into. http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested-intel/xen-unstable.html godello1, which is exhibiting the problem is (from the serial logs)a "CPU0: Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz stepping 03". Version 713b7e4ef2aa from xen.git also ran and passed on italia0 and 713b7e4ef2aa was after the bad commit (d02e84b9d9d). The serial logs say that italia0 is "CPU0: Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz stepping 09". > I checked the name convention, and found Xeon E3-1220 v3 is Haswell. So it _does_ happen on Haswell then. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-08 10:29 ` Ian Campbell 2015-12-09 6:27 ` Robert Hu @ 2015-12-11 4:04 ` Robert Hu 2015-12-11 11:49 ` Ian Campbell 1 sibling, 1 reply; 23+ messages in thread From: Robert Hu @ 2015-12-11 4:04 UTC (permalink / raw) To: Ian Campbell Cc: Tian, Kevin, xen-devel, Jan Beulich, guangrong.xiao, Andrew Cooper, Ian Jackson, osstest service owner, Wang, Yong Y, Nakajima, Jun, Jin, Gordon, Hu, Robert On Tue, 2015-12-08 at 10:29 +0000, Ian Campbell wrote: > On Tue, 2015-12-08 at 08:06 +0000, Hu, Robert wrote: > > > > > [...] > > Please trim your quotes. > > > For your failure, as Kevin mentioned in other mail, we will find someone > > to look into. > > Would you find out the detailed log of ' debian-hvm-install--l1--l2' > > step? so that he > > can start analyze, as I cannot reproduce it right now. > > (I didn't managed to find out the log in your web links) > > From > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > (in this thread, or any other relevant logs/NNNNN/ link), you can click the > "test -amd64 -amd64 -qemuu -nested -intel" header to go to > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/test-amd64-amd64-qemuu-nested-intel/info.html > > Then click the result of the "debian-hvm-install/l1/l2" step from that > table to go to the step log: > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/test-amd64-amd64-qemuu-nested-intel/16.ts-debian-hvm-install.log > > Which I think is what you were after? Yes, I had tracked to here. However, from logs like (the above cannot open now, let me take the latest for example) http://logs.test-lab.xenproject.org/osstest/logs/65633/test-amd64-amd64-qemuu-nested-intel/16.ts-debian-hvm-install.log 2015-12-10 20:45:53 Z executing ssh ... root@172.16.146.205 xl -vvv create /etc/xen/l2.guest.osstest.cfg Parsing config from /etc/xen/l2.guest.osstest.cfg libxl: debug: libxl_create.c:1561:do_domain_create: ao 0xe8abd0: create: how=(nil) callback=(nil) poller=0xe8b840 libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk vdev=hda spec.backend=unknown libxl: debug: libxl_device.c:298:libxl__device_disk_set_backend: Disk vdev=hda, using backend phy libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk vdev=hdc spec.backend=unknown libxl: debug: libxl_device.c:298:libxl__device_disk_set_backend: Disk vdev=hdc, using backend phy libxl: debug: libxl_create.c:942:initiate_domain_create: running bootloader libxl: debug: libxl_bootloader.c:324:libxl__bootloader_run: not a PV domain, skipping bootloader libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0xe8c1f8: deregister unregistered 2015-12-10 20:47:33 Z command timed out [100]: timeout 130 ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_65633.test-amd64-amd64-qemuu-nested-intel root@172.16.146.205 xl -vvv create /etc/xen/l2.guest.osstest.cfg status (timed out) at Osstest/TestSupport.pm line 414. + rc=4 + date -u +%Y-%m-%d %H:%M:%S Z exit status 4 2015-12-10 20:47:33 Z exit status 4 + exit 4 I can only see L1 try to create L2 timeout. We'd like to what's happening inside L1 hypervisor. Where can I find it? or cannot? > > All of the other related logs are at the end of the same > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/test-amd64-amd64-qemuu-nested-intel/info.html > page. > > See also README in osstest.git which explains the structure of the log > pages/grids/etc. > > Ian. > > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [xen-unstable test] 65141: regressions - FAIL 2015-12-11 4:04 ` Robert Hu @ 2015-12-11 11:49 ` Ian Campbell 0 siblings, 0 replies; 23+ messages in thread From: Ian Campbell @ 2015-12-11 11:49 UTC (permalink / raw) To: robert.hu Cc: Tian, Kevin, xen-devel, Nakajima, Jun, guangrong.xiao, Andrew Cooper, Ian Jackson, osstest service owner, Wang, Yong Y, Jan Beulich, Jin, Gordon On Fri, 2015-12-11 at 12:04 +0800, Robert Hu wrote: > On Tue, 2015-12-08 at 10:29 +0000, Ian Campbell wrote: > > On Tue, 2015-12-08 at 08:06 +0000, Hu, Robert wrote: > > > > > > > [...] > > > > Please trim your quotes. > > > > > For your failure, as Kevin mentioned in other mail, we will find > > > someone > > > to look into. > > > Would you find out the detailed log of ' debian-hvm-install--l1--l2' > > > step? so that he > > > can start analyze, as I cannot reproduce it right now. > > > (I didn't managed to find out the log in your web links) > > > > From > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/ > > > > (in this thread, or any other relevant logs/NNNNN/ link), you can click > > the > > "test -amd64 -amd64 -qemuu -nested -intel" header to go to > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/test-amd > > 64-amd64-qemuu-nested-intel/info.html > > > > Then click the result of the "debian-hvm-install/l1/l2" step from that > > table to go to the step log: > > http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/test-amd > > 64-amd64-qemuu-nested-intel/16.ts-debian-hvm-install.log > > > > Which I think is what you were after? > > Yes, I had tracked to here. However, from logs like (the above cannot > open now, let me take the latest for example) > http://logs.test-lab.xenproject.org/osstest/logs/65633/test-amd64-amd64-q > emuu-nested-intel/16.ts-debian-hvm-install.log > [...] > I can only see L1 try to create L2 timeout. We'd like to what's > happening inside L1 hypervisor. Where can I find it? or cannot? Please take a look through the big list of logfiles in the test case report: http://logs.test-lab.xenproject.org/osstest/logs/65633/test-amd64-amd64-qemuu-nested-intel/info.html There is lots of stuff there, including plenty of stuff from the L1 host (which generally has names beginning <host>_l1.guest.osstest..., i.e. godello0_l1.guest.osstest) This should include everything which is normally collected from a host, it doesn't matter if it is L0 or L1. If something is missing then we should add it to ts-logs-capture. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2015-12-11 11:49 UTC | newest] Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-11-27 15:45 [xen-unstable test] 65141: regressions - FAIL osstest service owner 2015-11-27 15:54 ` Ian Jackson 2015-11-30 5:35 ` Hu, Robert 2015-12-02 10:34 ` Ian Campbell 2015-12-02 10:52 ` Jan Beulich 2015-12-02 10:57 ` Andrew Cooper 2015-12-02 11:07 ` Ian Campbell 2015-12-02 13:51 ` Ian Campbell 2015-12-03 5:58 ` Tian, Kevin 2015-12-03 9:25 ` Ian Campbell 2015-12-05 8:09 ` Ian Campbell 2015-12-07 16:18 ` Jan Beulich 2015-12-07 16:28 ` Ian Campbell 2015-12-07 16:48 ` Jan Beulich 2015-12-08 2:46 ` Tian, Kevin 2015-12-08 7:28 ` Jan Beulich 2015-12-08 8:06 ` Hu, Robert 2015-12-08 10:29 ` Ian Campbell 2015-12-09 6:27 ` Robert Hu 2015-12-09 8:35 ` Jin, Gordon 2015-12-09 10:21 ` Ian Campbell 2015-12-11 4:04 ` Robert Hu 2015-12-11 11:49 ` Ian Campbell
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.