* [xen-unstable-smoke test] 162597: regressions - FAIL
@ 2021-06-10 10:50 osstest service owner
2021-06-10 11:32 ` Jan Beulich
0 siblings, 1 reply; 6+ messages in thread
From: osstest service owner @ 2021-06-10 10:50 UTC (permalink / raw)
To: xen-devel, osstest-admin
flight 162597 xen-unstable-smoke real [real]
flight 162602 xen-unstable-smoke real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/162597/
http://logs.test-lab.xenproject.org/osstest/logs/162602/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-armhf-armhf-xl 18 guest-start/debian.repeat fail REGR. vs. 162574
Tests which did not succeed, but are not blocking:
test-amd64-amd64-libvirt 15 migrate-support-check fail never pass
test-arm64-arm64-xl-xsm 15 migrate-support-check fail never pass
test-arm64-arm64-xl-xsm 16 saverestore-support-check fail never pass
test-armhf-armhf-xl 15 migrate-support-check fail never pass
test-armhf-armhf-xl 16 saverestore-support-check fail never pass
version targeted for testing:
xen dfcffb128be46a3e413eaa941744536fe53c94b6
baseline version:
xen 3e09045991cde360432bc7437103f8f8a6699359
Last test of basis 162574 2021-06-09 14:00:34 Z 0 days
Testing same since 162584 2021-06-10 00:00:27 Z 0 days 3 attempts
------------------------------------------------------------
People who touched revisions under test:
Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Stefano Stabellini <sstabellini@kernel.org>
Stefano Stabellini <stefano.stabellini@xilinx.com>
jobs:
build-arm64-xsm pass
build-amd64 pass
build-armhf pass
build-amd64-libvirt pass
test-armhf-armhf-xl fail
test-arm64-arm64-xl-xsm pass
test-amd64-amd64-xl-qemuu-debianhvm-amd64 pass
test-amd64-amd64-libvirt pass
------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images
Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs
Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
Not pushing.
------------------------------------------------------------
commit dfcffb128be46a3e413eaa941744536fe53c94b6
Author: Stefano Stabellini <sstabellini@kernel.org>
Date: Wed Jun 9 10:37:59 2021 -0700
xen/arm32: SPSR_hyp/SPSR
SPSR_hyp is not meant to be accessed from Hyp mode (EL2); accesses
trigger UNPREDICTABLE behaviour. Xen should read/write SPSR instead.
See: ARM DDI 0487D.b page G8-5993.
This fixes booting Xen/arm32 on QEMU.
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
(qemu changes not included)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [xen-unstable-smoke test] 162597: regressions - FAIL
2021-06-10 10:50 [xen-unstable-smoke test] 162597: regressions - FAIL osstest service owner
@ 2021-06-10 11:32 ` Jan Beulich
2021-06-10 16:19 ` Bertrand Marquis
0 siblings, 1 reply; 6+ messages in thread
From: Jan Beulich @ 2021-06-10 11:32 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall; +Cc: osstest service owner, xen-devel
On 10.06.2021 12:50, osstest service owner wrote:
> flight 162597 xen-unstable-smoke real [real]
> flight 162602 xen-unstable-smoke real-retest [real]
> http://logs.test-lab.xenproject.org/osstest/logs/162597/
> http://logs.test-lab.xenproject.org/osstest/logs/162602/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
> test-armhf-armhf-xl 18 guest-start/debian.repeat fail REGR. vs. 162574
This now being the 3rd failure in a row, I guess there's a fair chance
of there actually being something wrong with ...
> commit dfcffb128be46a3e413eaa941744536fe53c94b6
> Author: Stefano Stabellini <sstabellini@kernel.org>
> Date: Wed Jun 9 10:37:59 2021 -0700
>
> xen/arm32: SPSR_hyp/SPSR
>
> SPSR_hyp is not meant to be accessed from Hyp mode (EL2); accesses
> trigger UNPREDICTABLE behaviour. Xen should read/write SPSR instead.
> See: ARM DDI 0487D.b page G8-5993.
>
> This fixes booting Xen/arm32 on QEMU.
>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> Reviewed-by: Julien Grall <jgrall@amazon.com>
> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
> Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
... this. My Arm-untrained eye couldn't spot anything in the logs.
Jan
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [xen-unstable-smoke test] 162597: regressions - FAIL
2021-06-10 11:32 ` Jan Beulich
@ 2021-06-10 16:19 ` Bertrand Marquis
2021-06-11 1:49 ` Stefano Stabellini
0 siblings, 1 reply; 6+ messages in thread
From: Bertrand Marquis @ 2021-06-10 16:19 UTC (permalink / raw)
To: Jan Beulich
Cc: Stefano Stabellini, Julien Grall, osstest service owner, xen-devel
Hi Jan,
> On 10 Jun 2021, at 12:32, Jan Beulich <jbeulich@suse.com> wrote:
>
> On 10.06.2021 12:50, osstest service owner wrote:
>> flight 162597 xen-unstable-smoke real [real]
>> flight 162602 xen-unstable-smoke real-retest [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/162597/
>> http://logs.test-lab.xenproject.org/osstest/logs/162602/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>> test-armhf-armhf-xl 18 guest-start/debian.repeat fail REGR. vs. 162574
>
> This now being the 3rd failure in a row, I guess there's a fair chance
> of there actually being something wrong with ...
>
>> commit dfcffb128be46a3e413eaa941744536fe53c94b6
>> Author: Stefano Stabellini <sstabellini@kernel.org>
>> Date: Wed Jun 9 10:37:59 2021 -0700
>>
>> xen/arm32: SPSR_hyp/SPSR
>>
>> SPSR_hyp is not meant to be accessed from Hyp mode (EL2); accesses
>> trigger UNPREDICTABLE behaviour. Xen should read/write SPSR instead.
>> See: ARM DDI 0487D.b page G8-5993.
>>
>> This fixes booting Xen/arm32 on QEMU.
>>
>> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
>> Reviewed-by: Julien Grall <jgrall@amazon.com>
>> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
>> Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
>
> ... this. My Arm-untrained eye couldn't spot anything in the logs.
I am not sure to read the log correctly so do I see it right that dom0 started and it failed then to start a guest ?
Regards
Bertrand
>
> Jan
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [xen-unstable-smoke test] 162597: regressions - FAIL
2021-06-10 16:19 ` Bertrand Marquis
@ 2021-06-11 1:49 ` Stefano Stabellini
2021-06-11 6:58 ` Jan Beulich
2021-06-11 7:38 ` Jan Beulich
0 siblings, 2 replies; 6+ messages in thread
From: Stefano Stabellini @ 2021-06-11 1:49 UTC (permalink / raw)
To: Bertrand Marquis
Cc: Jan Beulich, Stefano Stabellini, Julien Grall,
osstest service owner, xen-devel
On Thu, 10 Jun 2021, Bertrand Marquis wrote:
> Hi Jan,
>
> > On 10 Jun 2021, at 12:32, Jan Beulich <jbeulich@suse.com> wrote:
> >
> > On 10.06.2021 12:50, osstest service owner wrote:
> >> flight 162597 xen-unstable-smoke real [real]
> >> flight 162602 xen-unstable-smoke real-retest [real]
> >> http://logs.test-lab.xenproject.org/osstest/logs/162597/
> >> http://logs.test-lab.xenproject.org/osstest/logs/162602/
> >>
> >> Regressions :-(
> >>
> >> Tests which did not succeed and are blocking,
> >> including tests which could not be run:
> >> test-armhf-armhf-xl 18 guest-start/debian.repeat fail REGR. vs. 162574
> >
> > This now being the 3rd failure in a row, I guess there's a fair chance
> > of there actually being something wrong with ...
> >
> >> commit dfcffb128be46a3e413eaa941744536fe53c94b6
> >> Author: Stefano Stabellini <sstabellini@kernel.org>
> >> Date: Wed Jun 9 10:37:59 2021 -0700
> >>
> >> xen/arm32: SPSR_hyp/SPSR
> >>
> >> SPSR_hyp is not meant to be accessed from Hyp mode (EL2); accesses
> >> trigger UNPREDICTABLE behaviour. Xen should read/write SPSR instead.
> >> See: ARM DDI 0487D.b page G8-5993.
> >>
> >> This fixes booting Xen/arm32 on QEMU.
> >>
> >> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> >> Reviewed-by: Julien Grall <jgrall@amazon.com>
> >> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
> >> Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
> >
> > ... this. My Arm-untrained eye couldn't spot anything in the logs.
>
> I am not sure to read the log correctly so do I see it right that dom0 started and it failed then to start a guest ?
Thanks Jan for bringing it to my attention.
I am not an expert in reading OSSTest logs. From the following:
http://logs.test-lab.xenproject.org/osstest/logs/162597/test-armhf-armhf-xl/info.html
I understand that Xen booted and a DomU was started. However,
"migrate-support-check" and "saverestore-support-check" failed. Is that
correct?
If so, it would be really strange for SPSR_hyp/SPSR to cause the problem
because I would expect Xen to hang at boot before Dom0 is started
instead.
I don't have any ARMv7 hardware to try to repro this issue, and ARMv7 is
most certainly required (ARMv8/aarch32 won't repro.)
Could someone more at ease with OSSTest than me arrange for a run with
this commit reverted to verify that it is the issue?
In any case, I tried to figure it out. I guessed it could be a compiler
error. I followed the white rabbit down the ARM ARM hole. I disassebled
the Xen binary [1] from the failed job. "msr SPSR, r11" is 0x0026a38c.
The encoding should be at B9.3.12 of the ARMv7-A DDI 0406C and F5.1.121
of ARMv8 DDI 0487D.b. Unfortunately it doesn't seem to match either one
of them and I don't understand why.
The "mrs r11, SPSR" is generated as 0x00262ecc. That should be described
at F5.1.117 for ARMv8 and B9.3.9 for ARMv7. Also doesn't seem to match.
I guess I am looking at the wrong encoding but I am not exactly sure why.
[1] http://logs.test-lab.xenproject.org/osstest/logs/162597/build-armhf/build/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [xen-unstable-smoke test] 162597: regressions - FAIL
2021-06-11 1:49 ` Stefano Stabellini
@ 2021-06-11 6:58 ` Jan Beulich
2021-06-11 7:38 ` Jan Beulich
1 sibling, 0 replies; 6+ messages in thread
From: Jan Beulich @ 2021-06-11 6:58 UTC (permalink / raw)
To: Stefano Stabellini, Bertrand Marquis
Cc: Julien Grall, osstest service owner, xen-devel
On 11.06.2021 03:49, Stefano Stabellini wrote:
> On Thu, 10 Jun 2021, Bertrand Marquis wrote:
>>> On 10 Jun 2021, at 12:32, Jan Beulich <jbeulich@suse.com> wrote:
>>> On 10.06.2021 12:50, osstest service owner wrote:
>>>> flight 162597 xen-unstable-smoke real [real]
>>>> flight 162602 xen-unstable-smoke real-retest [real]
>>>> http://logs.test-lab.xenproject.org/osstest/logs/162597/
>>>> http://logs.test-lab.xenproject.org/osstest/logs/162602/
>>>>
>>>> Regressions :-(
>>>>
>>>> Tests which did not succeed and are blocking,
>>>> including tests which could not be run:
>>>> test-armhf-armhf-xl 18 guest-start/debian.repeat fail REGR. vs. 162574
>>>
>>> This now being the 3rd failure in a row, I guess there's a fair chance
>>> of there actually being something wrong with ...
>>>
>>>> commit dfcffb128be46a3e413eaa941744536fe53c94b6
>>>> Author: Stefano Stabellini <sstabellini@kernel.org>
>>>> Date: Wed Jun 9 10:37:59 2021 -0700
>>>>
>>>> xen/arm32: SPSR_hyp/SPSR
>>>>
>>>> SPSR_hyp is not meant to be accessed from Hyp mode (EL2); accesses
>>>> trigger UNPREDICTABLE behaviour. Xen should read/write SPSR instead.
>>>> See: ARM DDI 0487D.b page G8-5993.
>>>>
>>>> This fixes booting Xen/arm32 on QEMU.
>>>>
>>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
>>>> Reviewed-by: Julien Grall <jgrall@amazon.com>
>>>> Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
>>>> Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
>>>
>>> ... this. My Arm-untrained eye couldn't spot anything in the logs.
>>
>> I am not sure to read the log correctly so do I see it right that dom0 started and it failed then to start a guest ?
Well, in this particular flight it succeeded to create Dom1 (for
guest-start) and then it managed to also create Dom2, but failed to
get the expected "sign of life". It varies at which of the repeated
attempts the failure occurs (in one of the flights it also occurred
right at guest-start), but failure chances are high enough such that
so far in all of the flights things didn't complete successfully.
And with this high a failure rate, it accidentally succeeding and
thus leading to a push would probably do us more bad than good.
> Thanks Jan for bringing it to my attention.
>
> I am not an expert in reading OSSTest logs. From the following:
>
> http://logs.test-lab.xenproject.org/osstest/logs/162597/test-armhf-armhf-xl/info.html
>
> I understand that Xen booted and a DomU was started. However,
> "migrate-support-check" and "saverestore-support-check" failed. Is that
> correct?
Yes, but these two steps aren't the problem - afaict they always fail,
and hence wouldn't prevent a push.
It's guest-start/debian.repeat which is the problem in this flight.
> If so, it would be really strange for SPSR_hyp/SPSR to cause the problem
> because I would expect Xen to hang at boot before Dom0 is started
> instead.
>
>
> I don't have any ARMv7 hardware to try to repro this issue, and ARMv7 is
> most certainly required (ARMv8/aarch32 won't repro.)
>
> Could someone more at ease with OSSTest than me arrange for a run with
> this commit reverted to verify that it is the issue?
>
>
>
> In any case, I tried to figure it out. I guessed it could be a compiler
> error. I followed the white rabbit down the ARM ARM hole. I disassebled
> the Xen binary [1] from the failed job. "msr SPSR, r11" is 0x0026a38c.
>
> The encoding should be at B9.3.12 of the ARMv7-A DDI 0406C and F5.1.121
> of ARMv8 DDI 0487D.b. Unfortunately it doesn't seem to match either one
> of them and I don't understand why.
>
>
> The "mrs r11, SPSR" is generated as 0x00262ecc. That should be described
> at F5.1.117 for ARMv8 and B9.3.9 for ARMv7. Also doesn't seem to match.
Indeed I was wondering whether perhaps the tool chain has an issue here.
Otoh I'd expect a tool chain issue to yield consistent failures rather
than ones with just a fair probability. Unless, of course, unspecified
behavior is hit, and the hardware indeed behaves randomly in this case.
Jan
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [xen-unstable-smoke test] 162597: regressions - FAIL
2021-06-11 1:49 ` Stefano Stabellini
2021-06-11 6:58 ` Jan Beulich
@ 2021-06-11 7:38 ` Jan Beulich
1 sibling, 0 replies; 6+ messages in thread
From: Jan Beulich @ 2021-06-11 7:38 UTC (permalink / raw)
To: Stefano Stabellini
Cc: Julien Grall, osstest service owner, xen-devel, Bertrand Marquis
On 11.06.2021 03:49, Stefano Stabellini wrote:
> In any case, I tried to figure it out. I guessed it could be a compiler
> error. I followed the white rabbit down the ARM ARM hole. I disassebled
> the Xen binary [1] from the failed job. "msr SPSR, r11" is 0x0026a38c.
>
> The encoding should be at B9.3.12 of the ARMv7-A DDI 0406C and F5.1.121
> of ARMv8 DDI 0487D.b. Unfortunately it doesn't seem to match either one
> of them and I don't understand why.
>
>
> The "mrs r11, SPSR" is generated as 0x00262ecc. That should be described
> at F5.1.117 for ARMv8 and B9.3.9 for ARMv7. Also doesn't seem to match.
According to my looking at the disassembly, the two numbers you've
quoted are the addresses, not insn encodings. Using my own disassembler
(i.e. there's room for that one being wrong), I do get
E169F00B msr spsr_cf, r11
E14FB000 mrs r11, spsr
the former of which doesn't look like an exact equivalent of the input
instruction. I guess it really is "msr spsr_cxsf, r11" which is meant?
In gas sources I find this:
/* Unadorned APSR is equivalent to APSR_nzcvq/CPSR_f (for writes). This
is deprecated, but allow it anyway. */
if (is_apsr && lhs)
{
psr_field |= PSR_f;
as_tsktsk (_("writing to APSR without specifying a bitmask is "
"deprecated"));
}
else if (!m_profile)
/* These bits are never right for M-profile devices: don't set them
(only code paths which read/write APSR reach here). */
psr_field |= (PSR_c | PSR_f);
There's clearly a comment missing to talk about the "unadorned" SPSR
case, but the effect is exactly what is observed: Rather than
defaulting to the setting of all 4 bits, only two of them get set
when plain "SPSR" is used. I've not been able to spot a place where
the Arm ARM specifies this, but given its size I'm not surprised at
all. I'd like to note though that the MSR description doesn't even
allow for plain "SPSR" (unlike MRS); only SPSR_<...> is described
there.
Based on this analysis I guess I can make a patch despite not being
able to test it, as I'm pretty certain you really want to restore
all of PSR; not just the low half ...
Jan
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-06-11 7:39 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-10 10:50 [xen-unstable-smoke test] 162597: regressions - FAIL osstest service owner
2021-06-10 11:32 ` Jan Beulich
2021-06-10 16:19 ` Bertrand Marquis
2021-06-11 1:49 ` Stefano Stabellini
2021-06-11 6:58 ` Jan Beulich
2021-06-11 7:38 ` Jan Beulich
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.