* [Xen-devel] [linux-linus bisection] complete test-amd64-amd64-qemuu-nested-intel
@ 2020-03-12 23:56 osstest service owner
2020-03-13 16:03 ` Andrew Cooper
0 siblings, 1 reply; 4+ messages in thread
From: osstest service owner @ 2020-03-12 23:56 UTC (permalink / raw)
To: xen-devel, osstest-admin
[-- Attachment #1: Type: text/plain, Size: 15786 bytes --]
branch xen-unstable
xenbranch xen-unstable
job test-amd64-amd64-qemuu-nested-intel
testid debian-hvm-install/l1/l2
Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
*** Found and reproduced problem changeset ***
Bug is in tree: xen git://xenbits.xen.org/xen.git
Bug introduced: f96e1469ad06b61796c60193daaeb9f8a96d7458
Bug not present: 0729830cc425a8ff27a3137e87b93768ae3c853c
Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/148496/
commit f96e1469ad06b61796c60193daaeb9f8a96d7458
Author: Roger Pau Monné <roger.pau@citrix.com>
Date: Wed Feb 5 13:49:09 2020 +0100
x86/vvmx: fix virtual interrupt injection when Ack on exit control is used
When doing a virtual vmexit (ie: a vmexit handled by the L1 VMM)
interrupts shouldn't be injected using the virtual interrupt delivery
mechanism unless the Ack on exit vmexit control bit isn't set in the
nested vmcs.
Gate the call to nvmx_update_apicv helper on whether the nested vmcs
has the Ack on exit bit set in the vmexit control field.
Note that this fixes the usage of x2APIC by the L1 VMM, at least when
the L1 VMM is Xen.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
For bisection revision-tuple graph see:
http://logs.test-lab.xenproject.org/osstest/results/bisect/linux-linus/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.html
Revision IDs in each graph node refer, respectively, to the Trees above.
----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/linux-linus/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2 --summary-out=tmp/148496.bisection-summary --basis-template=133580 --blessings=real,real-bisect linux-linus test-amd64-amd64-qemuu-nested-intel debian-hvm-install/l1/l2
Searching for failure / basis pass:
148417 fail [host=chardonnay0] / 147082 [host=albana0] 147029 [host=godello0] 146972 [host=godello1] 146904 [host=italia0] 143848 [host=godello1] 143581 [host=godello0] 143450 [host=huxelrebe0] 143363 [host=albana0] 143277 [host=fiano0] 143242 [host=huxelrebe1] 143169 [host=elbling1] 143087 [host=godello1] 143060 [host=chardonnay1] 143006 ok.
Failure / basis pass flights: 148417 / 143006
(tree with no url: minios)
Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: seabios git://xenbits.xen.org/osstest/seabios.git
Tree: xen git://xenbits.xen.org/xen.git
Latest f35111a946548e3b34a55abbad3e9bacce6cb10f c530a75c1e6a472b0eb9558310b518f0dfcd8860 a3e25cc8a1dd3d1ea24ed02f90c44221e015e965 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 24d3938ca96a6420ec1a5f1f8479f90f2e9fdd56 0d99c909d7e1cbe69329a00f7772946f10a7865b
Basis pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 2bbbdeeea21113185912a6a3ec8cdcaf862d8568 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef fc92d092ea4f704bc4d283c3911ee9894733f4ce 518c935fac4d30b3ec35d4b6add82b17b7d7aca3
Generating revisions with ./adhoc-revtuple-generator git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git#7d194c2100ad2a6dded545887d02754948ca5241-f35111a946548e3b34a55abbad3e9bacce6cb10f git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 git://xenbits.xen.org/osstest/ovmf.git#2bbbdeeea21113185912a6a3ec8cdcaf862d8568-a3e25cc8a1dd3d1ea24ed02f90c44221e015e965 git://xenbits.xen.org/qemu-xen-traditional.\
git#d0d8ad39ecb51cd7497cd524484fe09f50876798-d0d8ad39ecb51cd7497cd524484fe09f50876798 git://xenbits.xen.org/qemu-xen.git#933ebad2470a169504799a1d95b8e410bd9847ef-933ebad2470a169504799a1d95b8e410bd9847ef git://xenbits.xen.org/osstest/seabios.git#fc92d092ea4f704bc4d283c3911ee9894733f4ce-24d3938ca96a6420ec1a5f1f8479f90f2e9fdd56 git://xenbits.xen.org/xen.git#518c935fac4d30b3ec35d4b6add82b17b7d7aca3-0d99c909d7e1cbe69329a00f7772946f10a7865b
From git://cache:9419/git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
3cc6e2c599cd..807f030b44cc master -> origin/master
adhoc-revtuple-generator: tree discontiguous: linux-2.6
Use of uninitialized value $parents in array dereference at ./adhoc-revtuple-generator line 465.
Use of uninitialized value in concatenation (.) or string at ./adhoc-revtuple-generator line 465.
Loaded 8274 nodes in revision graph
Searching for test results:
143006 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 2bbbdeeea21113185912a6a3ec8cdcaf862d8568 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef fc92d092ea4f704bc4d283c3911ee9894733f4ce 518c935fac4d30b3ec35d4b6add82b17b7d7aca3
143087 [host=godello1]
143060 [host=chardonnay1]
143169 [host=elbling1]
143202 [host=huxelrebe1]
143242 [host=huxelrebe1]
143277 [host=fiano0]
143363 [host=albana0]
143450 [host=huxelrebe0]
143581 [host=godello0]
143848 [host=godello1]
146850 []
146904 [host=italia0]
146972 [host=godello1]
147029 [host=godello0]
147082 [host=albana0]
147236 fail irrelevant
147157 fail irrelevant
147320 fail irrelevant
147410 fail irrelevant
147541 fail irrelevant
147480 fail irrelevant
147640 fail irrelevant
147706 fail irrelevant
147749 fail irrelevant
147820 fail irrelevant
147912 fail irrelevant
148038 fail irrelevant
148180 fail irrelevant
148119 fail irrelevant
148258 fail irrelevant
148335 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 2bbbdeeea21113185912a6a3ec8cdcaf862d8568 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef fc92d092ea4f704bc4d283c3911ee9894733f4ce 518c935fac4d30b3ec35d4b6add82b17b7d7aca3
148417 fail f35111a946548e3b34a55abbad3e9bacce6cb10f c530a75c1e6a472b0eb9558310b518f0dfcd8860 a3e25cc8a1dd3d1ea24ed02f90c44221e015e965 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 24d3938ca96a6420ec1a5f1f8479f90f2e9fdd56 0d99c909d7e1cbe69329a00f7772946f10a7865b
148422 fail 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 93b8ed68ddad01e072d617b763aa613832d7416c d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d 3dd724dff085e13ad520f8e35aea717db2ff07d0
148418 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 1f0d8096291651e6c20dbbc57d108321c1443563 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef f21b5a4aeb020f2a5e2c6503f906a9349dd2f069 fae249d23413b2bf7d98a97d8f649cf7d102c1ae
148405 fail irrelevant
148410 blocked 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 665afccc52e1a02ee329147e02f04b8e9cf1d571 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef f21b5a4aeb020f2a5e2c6503f906a9349dd2f069 0cd791c499bdc698d14a24050ec56d60b45732e0
148435 fail 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 75839f977d377eb24ab55f5245977412a82844d4 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d ac661ada11723182957dfe11e17b3ede4914d21e
148427 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 c8b8157e126ae2fb6f65842677251d300ceff104 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d 0b70b2ff8f5a61196d090cc70040a20178327347
148440 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 eafd990f2606431d45cf0bbdbfee6d5959628de7 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d 957bd8e7e302f1a0d31c95fc9910429f10a04d54
148452 fail 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d f96e1469ad06b61796c60193daaeb9f8a96d7458
148457 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 eafd990f2606431d45cf0bbdbfee6d5959628de7 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d be8c70373bd157c7ec74eca511aa33b427075c39
148482 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d 0729830cc425a8ff27a3137e87b93768ae3c853c
148460 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 3feea54eae33a6689bedf1e023edeb219faa76d6 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d f7fb9a0aa9fbb81292e1626ce40340d3f6d8d505
148463 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d f7fb9a0aa9fbb81292e1626ce40340d3f6d8d505
148464 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d d00377890a3c62e0e925d171b6b839406b6d134d
148467 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 2bbbdeeea21113185912a6a3ec8cdcaf862d8568 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef fc92d092ea4f704bc4d283c3911ee9894733f4ce 518c935fac4d30b3ec35d4b6add82b17b7d7aca3
148486 fail 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d f96e1469ad06b61796c60193daaeb9f8a96d7458
148470 fail f35111a946548e3b34a55abbad3e9bacce6cb10f c530a75c1e6a472b0eb9558310b518f0dfcd8860 a3e25cc8a1dd3d1ea24ed02f90c44221e015e965 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 24d3938ca96a6420ec1a5f1f8479f90f2e9fdd56 0d99c909d7e1cbe69329a00f7772946f10a7865b
148474 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d 0729830cc425a8ff27a3137e87b93768ae3c853c
148492 pass 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d 0729830cc425a8ff27a3137e87b93768ae3c853c
148476 fail 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d f96e1469ad06b61796c60193daaeb9f8a96d7458
148496 fail 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d f96e1469ad06b61796c60193daaeb9f8a96d7458
Searching for interesting versions
Result found: flight 143006 (pass), for basis pass
Result found: flight 148417 (fail), for basis failure
Repro found: flight 148467 (pass), for basis pass
Repro found: flight 148470 (fail), for basis failure
0 revisions at 7d194c2100ad2a6dded545887d02754948ca5241 c530a75c1e6a472b0eb9558310b518f0dfcd8860 422da35375c6d95dae9d5c56530d255b672b1f59 d0d8ad39ecb51cd7497cd524484fe09f50876798 933ebad2470a169504799a1d95b8e410bd9847ef 76551856b28d227cb0386a1ab0e774329b941f7d 0729830cc425a8ff27a3137e87b93768ae3c853c
No revisions left to test, checking graph state.
Result found: flight 148474 (pass), for last pass
Result found: flight 148476 (fail), for first failure
Repro found: flight 148482 (pass), for last pass
Repro found: flight 148486 (fail), for first failure
Repro found: flight 148492 (pass), for last pass
Repro found: flight 148496 (fail), for first failure
*** Found and reproduced problem changeset ***
Bug is in tree: xen git://xenbits.xen.org/xen.git
Bug introduced: f96e1469ad06b61796c60193daaeb9f8a96d7458
Bug not present: 0729830cc425a8ff27a3137e87b93768ae3c853c
Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/148496/
commit f96e1469ad06b61796c60193daaeb9f8a96d7458
Author: Roger Pau Monné <roger.pau@citrix.com>
Date: Wed Feb 5 13:49:09 2020 +0100
x86/vvmx: fix virtual interrupt injection when Ack on exit control is used
When doing a virtual vmexit (ie: a vmexit handled by the L1 VMM)
interrupts shouldn't be injected using the virtual interrupt delivery
mechanism unless the Ack on exit vmexit control bit isn't set in the
nested vmcs.
Gate the call to nvmx_update_apicv helper on whether the nested vmcs
has the Ack on exit bit set in the vmexit control field.
Note that this fixes the usage of x2APIC by the L1 VMM, at least when
the L1 VMM is Xen.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
dot: graph is too large for cairo-renderer bitmaps. Scaling by 0.41573 to fit
pnmtopng: 65 colors found
Revision graph left in /home/logs/results/bisect/linux-linus/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.{dot,ps,png,html,svg}.
----------------------------------------
148496: tolerable ALL FAIL
flight 148496 linux-linus real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/148496/
Failures :-/ but no regressions.
Tests which did not succeed,
including tests which could not be run:
test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail baseline untested
jobs:
test-amd64-amd64-qemuu-nested-intel fail
------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images
Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs
Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Xen-devel] [linux-linus bisection] complete test-amd64-amd64-qemuu-nested-intel
2020-03-12 23:56 [Xen-devel] [linux-linus bisection] complete test-amd64-amd64-qemuu-nested-intel osstest service owner
@ 2020-03-13 16:03 ` Andrew Cooper
2020-03-17 6:22 ` Tian, Kevin
0 siblings, 1 reply; 4+ messages in thread
From: Andrew Cooper @ 2020-03-13 16:03 UTC (permalink / raw)
To: osstest service owner, xen-devel; +Cc: Kevin Tian, Jan Beulich, Roger Pau Monne
On 12/03/2020 23:56, osstest service owner wrote:
> branch xen-unstable
> xenbranch xen-unstable
> job test-amd64-amd64-qemuu-nested-intel
> testid debian-hvm-install/l1/l2
>
> Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
> Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
> Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> Tree: seabios git://xenbits.xen.org/osstest/seabios.git
> Tree: xen git://xenbits.xen.org/xen.git
>
> *** Found and reproduced problem changeset ***
>
> Bug is in tree: xen git://xenbits.xen.org/xen.git
> Bug introduced: f96e1469ad06b61796c60193daaeb9f8a96d7458
> Bug not present: 0729830cc425a8ff27a3137e87b93768ae3c853c
> Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/148496/
>
>
> commit f96e1469ad06b61796c60193daaeb9f8a96d7458
> Author: Roger Pau Monné <roger.pau@citrix.com>
> Date: Wed Feb 5 13:49:09 2020 +0100
>
> x86/vvmx: fix virtual interrupt injection when Ack on exit control is used
>
> When doing a virtual vmexit (ie: a vmexit handled by the L1 VMM)
> interrupts shouldn't be injected using the virtual interrupt delivery
> mechanism unless the Ack on exit vmexit control bit isn't set in the
> nested vmcs.
>
> Gate the call to nvmx_update_apicv helper on whether the nested vmcs
> has the Ack on exit bit set in the vmexit control field.
>
> Note that this fixes the usage of x2APIC by the L1 VMM, at least when
> the L1 VMM is Xen.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Looks like there are further problems here. I'm afraid I haven't
investigated further, but this also might be the source of the
intermittent problems in staging.
~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Xen-devel] [linux-linus bisection] complete test-amd64-amd64-qemuu-nested-intel
2020-03-13 16:03 ` Andrew Cooper
@ 2020-03-17 6:22 ` Tian, Kevin
2020-03-17 8:54 ` Roger Pau Monné
0 siblings, 1 reply; 4+ messages in thread
From: Tian, Kevin @ 2020-03-17 6:22 UTC (permalink / raw)
To: Andrew Cooper, osstest service owner, xen-devel
Cc: Jan Beulich, Roger Pau Monne
> From: Andrew Cooper <andrew.cooper3@citrix.com>
> Sent: Saturday, March 14, 2020 12:03 AM
>
> On 12/03/2020 23:56, osstest service owner wrote:
> > branch xen-unstable
> > xenbranch xen-unstable
> > job test-amd64-amd64-qemuu-nested-intel
> > testid debian-hvm-install/l1/l2
> >
> > Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-
> 2.6.git
> > Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> > Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
> > Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
> > Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> > Tree: seabios git://xenbits.xen.org/osstest/seabios.git
> > Tree: xen git://xenbits.xen.org/xen.git
> >
> > *** Found and reproduced problem changeset ***
> >
> > Bug is in tree: xen git://xenbits.xen.org/xen.git
> > Bug introduced: f96e1469ad06b61796c60193daaeb9f8a96d7458
> > Bug not present: 0729830cc425a8ff27a3137e87b93768ae3c853c
> > Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/148496/
> >
> >
> > commit f96e1469ad06b61796c60193daaeb9f8a96d7458
> > Author: Roger Pau Monné <roger.pau@citrix.com>
> > Date: Wed Feb 5 13:49:09 2020 +0100
> >
> > x86/vvmx: fix virtual interrupt injection when Ack on exit control is used
> >
> > When doing a virtual vmexit (ie: a vmexit handled by the L1 VMM)
> > interrupts shouldn't be injected using the virtual interrupt delivery
> > mechanism unless the Ack on exit vmexit control bit isn't set in the
> > nested vmcs.
> >
> > Gate the call to nvmx_update_apicv helper on whether the nested vmcs
> > has the Ack on exit bit set in the vmexit control field.
> >
> > Note that this fixes the usage of x2APIC by the L1 VMM, at least when
> > the L1 VMM is Xen.
> >
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > Reviewed-by: Kevin Tian <kevin.tian@intel.com>
>
> Looks like there are further problems here. I'm afraid I haven't
> investigated further, but this also might be the source of the
> intermittent problems in staging.
>
any error message or stack dump for this failure?
Thanks
Kevin
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Xen-devel] [linux-linus bisection] complete test-amd64-amd64-qemuu-nested-intel
2020-03-17 6:22 ` Tian, Kevin
@ 2020-03-17 8:54 ` Roger Pau Monné
0 siblings, 0 replies; 4+ messages in thread
From: Roger Pau Monné @ 2020-03-17 8:54 UTC (permalink / raw)
To: Tian, Kevin; +Cc: Andrew Cooper, osstest service owner, Jan Beulich, xen-devel
On Tue, Mar 17, 2020 at 06:22:55AM +0000, Tian, Kevin wrote:
> > From: Andrew Cooper <andrew.cooper3@citrix.com>
> > Sent: Saturday, March 14, 2020 12:03 AM
> >
> > On 12/03/2020 23:56, osstest service owner wrote:
> > > branch xen-unstable
> > > xenbranch xen-unstable
> > > job test-amd64-amd64-qemuu-nested-intel
> > > testid debian-hvm-install/l1/l2
> > >
> > > Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-
> > 2.6.git
> > > Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> > > Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git
> > > Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
> > > Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> > > Tree: seabios git://xenbits.xen.org/osstest/seabios.git
> > > Tree: xen git://xenbits.xen.org/xen.git
> > >
> > > *** Found and reproduced problem changeset ***
> > >
> > > Bug is in tree: xen git://xenbits.xen.org/xen.git
> > > Bug introduced: f96e1469ad06b61796c60193daaeb9f8a96d7458
> > > Bug not present: 0729830cc425a8ff27a3137e87b93768ae3c853c
> > > Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/148496/
> > >
> > >
> > > commit f96e1469ad06b61796c60193daaeb9f8a96d7458
> > > Author: Roger Pau Monné <roger.pau@citrix.com>
> > > Date: Wed Feb 5 13:49:09 2020 +0100
> > >
> > > x86/vvmx: fix virtual interrupt injection when Ack on exit control is used
> > >
> > > When doing a virtual vmexit (ie: a vmexit handled by the L1 VMM)
> > > interrupts shouldn't be injected using the virtual interrupt delivery
> > > mechanism unless the Ack on exit vmexit control bit isn't set in the
> > > nested vmcs.
> > >
> > > Gate the call to nvmx_update_apicv helper on whether the nested vmcs
> > > has the Ack on exit bit set in the vmexit control field.
> > >
> > > Note that this fixes the usage of x2APIC by the L1 VMM, at least when
> > > the L1 VMM is Xen.
> > >
> > > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > > Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> >
> > Looks like there are further problems here. I'm afraid I haven't
> > investigated further, but this also might be the source of the
> > intermittent problems in staging.
> >
>
> any error message or stack dump for this failure?
Sorry, forgot to reply yesterday. I'm already looking into it, the
crash seems to be from L1 getting wrong interrupt injection:
(XEN) Assertion '!sp || (peoi[sp - 1].vector < vector)' failed at irq.c:1862
(XEN) ----[ Xen-4.14-unstable x86_64 debug=y Not tainted ]----
(XEN) CPU: 1
(XEN) RIP: e008:[<ffff82d080289dfe>] do_IRQ+0x4e3/0x6f8
(XEN) RFLAGS: 0000000000010046 CONTEXT: hypervisor (d1v0)
(XEN) rax: 000000000000006e rbx: 000000000000006e rcx: ffff8300b93212a0
(XEN) rdx: 0000000000000001 rsi: 0000000000000001 rdi: ffff8300b9321280
(XEN) rbp: ffff8300bf5efdb8 rsp: ffff8300bf5efd48 r8: 0000000000000001
(XEN) r9: ffff8300bf5b6068 r10: 0000166c3548742a r11: 000000550f69eeb0
(XEN) r12: ffff8300bf291d40 r13: 000000000000006e r14: ffff8300bf5c2000
(XEN) r15: ffff8300b93214a0 cr0: 0000000080050033 cr4: 00000000003526e0
(XEN) cr3: 00000000bf28c000 cr2: ffff8a4b06f3a000
(XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen code around <ffff82d080289dfe> (do_IRQ+0x4e3/0x6f8):
(XEN) 39 c3 0f 87 6c ff ff ff <0f> 0b 0f 0b 0f 0b b8 00 00 00 00 eb 49 0f ae e8
(XEN) Xen stack trace from rsp=ffff8300bf5efd48:
(XEN) ffff82d08038e851 0000003038d61000 ffff82d000000000 ffff8300bf5c2024
(XEN) 0000000000000000 000000208038e845 ffff82d08038e851 ffff82d08038e845
(XEN) ffff82d08038e851 0000000000000000 0000000000000000 0000000000000000
(XEN) ffff8300bf5effff 0000000000000000 00007cff40a10217 ffff82d08038e8ba
(XEN) ffff8300bf290000 0000000000000000 0000000000000001 ffff8300b92bc000
(XEN) ffff8300bf5efee8 ffff8300bf5efef8 0000000000000019 000000000001aa55
(XEN) 000000550caca034 ffff8300bf291d50 ffff82d0805c07e8 0000000000000000
(XEN) 0000003038d61000 0000000000000000 ffff8300bf290000 0000006e00000000
(XEN) ffff82d080335c04 000000000000e008 0000000000000286 ffff8300bf5efe78
(XEN) 0000000000000000 ffff82d080335b29 ffff82d08033ccd1 ffff82d08033ccc5
(XEN) 0000000000000000 ffff82d08033ccc5 ffff82d08033ccd1 ffff82d08033ccc5
(XEN) ffff82d08033ccd1 ffff82d08033ccc5 ffff82d08033ccd1 ffff8300b92bc000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 00007cff40a100e7 ffff82d08033cd1a 000000000000008e ffff8a4b2d18cedc
(XEN) ffffffffa2482f86 00000000000003c8 ffffffffa2b1a120 ffff8a4b2d18cebe
(XEN) ffff8a4b2d19cfa0 ffff8a4b000b8000 ffff8a4b2d19c000 0000000000000720
(XEN) 000000000000002a 0000000000000190 00000000000003c9 ffffffffa2482f80
(XEN) ffff8a4b2d18cc00 0000006e0000beef ffffffffa1f9dad7 000000bf0000beef
(XEN) 0000000000000246 ffffffffa2803ec8 000000000000beef e486997731cdbeef
(XEN) 085780c33cb2beef 5d42c695b7d9beef 4fa29714ad64beef 0000e01000000001
(XEN) Xen call trace:
(XEN) [<ffff82d080289dfe>] R do_IRQ+0x4e3/0x6f8
(XEN) [<ffff82d08038e851>] S common_interrupt+0xa1/0x120
(XEN) [<ffff82d08038e8ba>] F common_interrupt+0x10a/0x120
(XEN) [<ffff82d080335c04>] F vmx_vmexit_handler+0x32b/0x1e60
(XEN) [<ffff82d08033cd1a>] F vmx_asm_vmexit_handler+0xfa/0x270
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 1:
(XEN) Assertion '!sp || (peoi[sp - 1].vector < vector)' failed at irq.c:1862
(XEN) ****************************************
(XEN)
(XEN) Manual reset required ('noreboot' specified)
This is however not reproducible on all boxes, as it needs a special
set of VMX features available AFAIK in order to trigger it. I've
managed to get such a box and I'm debugging it.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-03-17 9:02 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-12 23:56 [Xen-devel] [linux-linus bisection] complete test-amd64-amd64-qemuu-nested-intel osstest service owner
2020-03-13 16:03 ` Andrew Cooper
2020-03-17 6:22 ` Tian, Kevin
2020-03-17 8:54 ` Roger Pau Monné
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.