* [xen-unstable bisection] complete test-armhf-armhf-xl-multivcpu
@ 2021-04-11 2:12 osstest service owner
2021-04-12 8:53 ` Jan Beulich
0 siblings, 1 reply; 3+ messages in thread
From: osstest service owner @ 2021-04-11 2:12 UTC (permalink / raw)
To: xen-devel, osstest-admin
branch xen-unstable
xenbranch xen-unstable
job test-armhf-armhf-xl-multivcpu
testid guest-start/debian.repeat
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
*** Found and reproduced problem changeset ***
Bug is in tree: xen git://xenbits.xen.org/xen.git
Bug introduced: 9617d5f9c19d1d157629e1e436791509526e0ce5
Bug not present: 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7
Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/160931/
commit 9617d5f9c19d1d157629e1e436791509526e0ce5
Author: Julien Grall <jgrall@amazon.com>
Date: Sat Feb 20 17:54:13 2021 +0000
xen/arm: mm: flush_page_to_ram() only need to clean to PoC
At the moment, flush_page_to_ram() is both cleaning and invalidate to
PoC the page.
The goal of flush_page_to_ram() is to prevent corruption when the guest
has disabled the cache (the cache line may be dirty) and the guest to
read previous content.
Per this definition, the invalidating the line is not necessary. So
invalidating the cache is unnecessary. In fact, it may be counter-
productive as the line may be (speculatively) accessed a bit after.
So this will incurr an expensive access to the memory.
More generally, we should avoid interferring too much with cache.
Therefore, flush_page_to_ram() is updated to only clean to PoC the page.
The performance impact of this change will depend on your
workload/processor.
Signed-off-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
For bisection revision-tuple graph see:
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-armhf-armhf-xl-multivcpu.guest-start--debian.repeat.html
Revision IDs in each graph node refer, respectively, to the Trees above.
----------------------------------------
Running cs-bisection-step --graph-out=/home/logs/results/bisect/xen-unstable/test-armhf-armhf-xl-multivcpu.guest-start--debian.repeat --summary-out=tmp/160931.bisection-summary --basis-template=160646 --blessings=real,real-bisect,real-retry xen-unstable test-armhf-armhf-xl-multivcpu guest-start/debian.repeat
Searching for failure / basis pass:
160878 fail [host=arndale-bluewater] / 160646 [host=cubietruck-gleizes] 160630 [host=cubietruck-braque] 160581 [host=cubietruck-picasso] 160559 [host=arndale-metrocentre] 160537 [host=arndale-lakeside] 160517 ok.
Failure / basis pass flights: 160878 / 160517
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 935d501ccbf5b8c4db1f6d0730a4a4c998e9e76a
Basis pass a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 e680cc48b7184d3489873d6776f84ba1fc238ced
Generating revisions with ./adhoc-revtuple-generator git://xenbits.xen.org/linux-pvops.git#a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9-a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 git://xenbits.xen.org/qemu-xen.git#7ea428895af2840d85c524f0bd11a38aac308308-7ea428895af2840d85c524f0bd11a38aac308308 git://xenbits.xen.org/xen.git#e680cc48b7184d3489873d6776f84ba1fc238ced-935d501\
ccbf5b8c4db1f6d0730a4a4c998e9e76a
Loaded 5001 nodes in revision graph
Searching for test results:
160471 [host=cubietruck-braque]
160492 [host=arndale-westfield]
160517 pass a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 e680cc48b7184d3489873d6776f84ba1fc238ced
160537 [host=arndale-lakeside]
160559 [host=arndale-metrocentre]
160581 [host=cubietruck-picasso]
160630 [host=cubietruck-braque]
160646 [host=cubietruck-gleizes]
160733 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 0435784cc75dcfef3b5f59c29deb1dbb84265ddb
160745 blocked a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 0435784cc75dcfef3b5f59c29deb1dbb84265ddb
160758 blocked a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 0435784cc75dcfef3b5f59c29deb1dbb84265ddb
160776 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 0435784cc75dcfef3b5f59c29deb1dbb84265ddb
160796 blocked a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 625faf9f002bd6ff4b6457a016b8ff338223b659
160820 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 025eacc13f6147ffa99da5ecee4ed96e7fe8e887
160850 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 025eacc13f6147ffa99da5ecee4ed96e7fe8e887
160899 pass a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 e680cc48b7184d3489873d6776f84ba1fc238ced
160901 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 025eacc13f6147ffa99da5ecee4ed96e7fe8e887
160903 pass a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 c58fbc38d208f2046e077833fe2b071ff479d7e6
160878 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 935d501ccbf5b8c4db1f6d0730a4a4c998e9e76a
160906 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 00948dcd6a5412695b42c6d5045b0d3075b14114
160909 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 935d501ccbf5b8c4db1f6d0730a4a4c998e9e76a
160913 pass a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 0dc28066e9f0339ad8f4aea233cc5912139c5f79
160915 pass a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7
160918 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 9617d5f9c19d1d157629e1e436791509526e0ce5
160919 pass a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7
160921 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 9617d5f9c19d1d157629e1e436791509526e0ce5
160925 pass a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7
160931 fail a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 9617d5f9c19d1d157629e1e436791509526e0ce5
Searching for interesting versions
Result found: flight 160517 (pass), for basis pass
For basis failure, parent search stopping at a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 625faf9f002bd6ff4b6457a016b8ff338223b659, results HASH(0x564fb28b8210) For basis failure, parent search stopping at a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 0435784cc75dcfef3b5f59c29deb1dbb84265ddb, results HASH(0x564fb28af148) HASH(0x564fb28b0b50) \
For basis failure, parent search stopping at a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7, results HASH(0x564fb2935120) HASH(0x564fb28a2df0) HASH(0x564fb28aa510) For basis failure, parent search stopping at a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 0dc28066e9f0339ad8f4aea233cc5912139c5f79, results H\
ASH(0x564fb2941b08) For basis failure, parent search stopping at a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 c58fbc38d208f2046e077833fe2b071ff479d7e6, results HASH(0x564fb2937188) For basis failure, parent search stopping at a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 e680cc48b7184d3489873d6776f84ba1fc238ced, results HASH(0x564fb28a6800) HA\
SH(0x564fb28a2af0) Result found: flight 160733 (fail), for basis failure (at ancestor ~234)
Repro found: flight 160899 (pass), for basis pass
Repro found: flight 160909 (fail), for basis failure
0 revisions at a6c5dd1dbaffe4cc398d8454546ba9246b9a95c9 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7ea428895af2840d85c524f0bd11a38aac308308 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7
No revisions left to test, checking graph state.
Result found: flight 160915 (pass), for last pass
Result found: flight 160918 (fail), for first failure
Repro found: flight 160919 (pass), for last pass
Repro found: flight 160921 (fail), for first failure
Repro found: flight 160925 (pass), for last pass
Repro found: flight 160931 (fail), for first failure
*** Found and reproduced problem changeset ***
Bug is in tree: xen git://xenbits.xen.org/xen.git
Bug introduced: 9617d5f9c19d1d157629e1e436791509526e0ce5
Bug not present: 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7
Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/160931/
commit 9617d5f9c19d1d157629e1e436791509526e0ce5
Author: Julien Grall <jgrall@amazon.com>
Date: Sat Feb 20 17:54:13 2021 +0000
xen/arm: mm: flush_page_to_ram() only need to clean to PoC
At the moment, flush_page_to_ram() is both cleaning and invalidate to
PoC the page.
The goal of flush_page_to_ram() is to prevent corruption when the guest
has disabled the cache (the cache line may be dirty) and the guest to
read previous content.
Per this definition, the invalidating the line is not necessary. So
invalidating the cache is unnecessary. In fact, it may be counter-
productive as the line may be (speculatively) accessed a bit after.
So this will incurr an expensive access to the memory.
More generally, we should avoid interferring too much with cache.
Therefore, flush_page_to_ram() is updated to only clean to PoC the page.
The performance impact of this change will depend on your
workload/processor.
Signed-off-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Revision graph left in /home/logs/results/bisect/xen-unstable/test-armhf-armhf-xl-multivcpu.guest-start--debian.repeat.{dot,ps,png,html,svg}.
----------------------------------------
160931: tolerable ALL FAIL
flight 160931 xen-unstable real-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/160931/
Failures :-/ but no regressions.
Tests which did not succeed,
including tests which could not be run:
test-armhf-armhf-xl-multivcpu 18 guest-start/debian.repeat fail baseline untested
test-armhf-armhf-xl-multivcpu 15 migrate-support-check fail never pass
test-armhf-armhf-xl-multivcpu 16 saverestore-support-check fail never pass
jobs:
test-armhf-armhf-xl-multivcpu fail
------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images
Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs
Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [xen-unstable bisection] complete test-armhf-armhf-xl-multivcpu
2021-04-11 2:12 [xen-unstable bisection] complete test-armhf-armhf-xl-multivcpu osstest service owner
@ 2021-04-12 8:53 ` Jan Beulich
2021-04-13 16:17 ` Julien Grall
0 siblings, 1 reply; 3+ messages in thread
From: Jan Beulich @ 2021-04-12 8:53 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini, Bertrand Marquis
Cc: xen-devel, osstest service owner
On 11.04.2021 04:12, osstest service owner wrote:
> branch xen-unstable
> xenbranch xen-unstable
> job test-armhf-armhf-xl-multivcpu
> testid guest-start/debian.repeat
>
> Tree: linux git://xenbits.xen.org/linux-pvops.git
> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> Tree: qemuu git://xenbits.xen.org/qemu-xen.git
> Tree: xen git://xenbits.xen.org/xen.git
>
> *** Found and reproduced problem changeset ***
>
> Bug is in tree: xen git://xenbits.xen.org/xen.git
> Bug introduced: 9617d5f9c19d1d157629e1e436791509526e0ce5
> Bug not present: 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7
> Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/160931/
>
>
> commit 9617d5f9c19d1d157629e1e436791509526e0ce5
> Author: Julien Grall <jgrall@amazon.com>
> Date: Sat Feb 20 17:54:13 2021 +0000
>
> xen/arm: mm: flush_page_to_ram() only need to clean to PoC
>
> At the moment, flush_page_to_ram() is both cleaning and invalidate to
> PoC the page.
>
> The goal of flush_page_to_ram() is to prevent corruption when the guest
> has disabled the cache (the cache line may be dirty) and the guest to
> read previous content.
>
> Per this definition, the invalidating the line is not necessary. So
> invalidating the cache is unnecessary. In fact, it may be counter-
> productive as the line may be (speculatively) accessed a bit after.
> So this will incurr an expensive access to the memory.
>
> More generally, we should avoid interferring too much with cache.
> Therefore, flush_page_to_ram() is updated to only clean to PoC the page.
>
> The performance impact of this change will depend on your
> workload/processor.
>
> Signed-off-by: Julien Grall <jgrall@amazon.com>
> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
> Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Is it possible that other code (guest one in particular considering the
failure pattern, but possibly not limited to that) has developed a
dependency on the prior behavior?
Jan
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [xen-unstable bisection] complete test-armhf-armhf-xl-multivcpu
2021-04-12 8:53 ` Jan Beulich
@ 2021-04-13 16:17 ` Julien Grall
0 siblings, 0 replies; 3+ messages in thread
From: Julien Grall @ 2021-04-13 16:17 UTC (permalink / raw)
To: Jan Beulich, Stefano Stabellini, Bertrand Marquis
Cc: xen-devel, osstest service owner
Hi Jan,
On 12/04/2021 09:53, Jan Beulich wrote:
> On 11.04.2021 04:12, osstest service owner wrote:
>> branch xen-unstable
>> xenbranch xen-unstable
>> job test-armhf-armhf-xl-multivcpu
>> testid guest-start/debian.repeat
>>
>> Tree: linux git://xenbits.xen.org/linux-pvops.git
>> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
>> Tree: qemuu git://xenbits.xen.org/qemu-xen.git
>> Tree: xen git://xenbits.xen.org/xen.git
>>
>> *** Found and reproduced problem changeset ***
>>
>> Bug is in tree: xen git://xenbits.xen.org/xen.git
>> Bug introduced: 9617d5f9c19d1d157629e1e436791509526e0ce5
>> Bug not present: 5c3c410bd2ea8d2cc520e8e8f83ad143c9c5cff7
>> Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/160931/
>>
>>
>> commit 9617d5f9c19d1d157629e1e436791509526e0ce5
>> Author: Julien Grall <jgrall@amazon.com>
>> Date: Sat Feb 20 17:54:13 2021 +0000
>>
>> xen/arm: mm: flush_page_to_ram() only need to clean to PoC
>>
>> At the moment, flush_page_to_ram() is both cleaning and invalidate to
>> PoC the page.
>>
>> The goal of flush_page_to_ram() is to prevent corruption when the guest
>> has disabled the cache (the cache line may be dirty) and the guest to
>> read previous content.
>>
>> Per this definition, the invalidating the line is not necessary. So
>> invalidating the cache is unnecessary. In fact, it may be counter-
>> productive as the line may be (speculatively) accessed a bit after.
>> So this will incurr an expensive access to the memory.
>>
>> More generally, we should avoid interferring too much with cache.
>> Therefore, flush_page_to_ram() is updated to only clean to PoC the page.
>>
>> The performance impact of this change will depend on your
>> workload/processor.
>>
>> Signed-off-by: Julien Grall <jgrall@amazon.com>
>> Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
>> Acked-by: Stefano Stabellini <sstabellini@kernel.org>
>
> Is it possible that other code (guest one in particular considering the
> failure pattern, but possibly not limited to that) has developed a
> dependency on the prior behavior?
This is not a problem in the guest. I overlooked that
flush_page_to_ram() is also used when emulating the instruction to
invalidate by set/way the data cache.
We would need to tell flush_page_to_ram() which type of operation we
want to do. I will not have time to work on it right now, so I have
reverted the patch to unblock OssTest.
Sorry for the breakage.
Cheers,
--
Julien Grall
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-04-13 16:17 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-11 2:12 [xen-unstable bisection] complete test-armhf-armhf-xl-multivcpu osstest service owner
2021-04-12 8:53 ` Jan Beulich
2021-04-13 16:17 ` Julien Grall
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).