xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Andrew Cooper <andrew.cooper3@citrix.com>
To: "osstest service owner" <osstest-admin@xenproject.org>,
	xen-devel@lists.xenproject.org,
	"Ian Jackson" <iwj@xenproject.org>,
	"Jan Beulich" <jbeulich@suse.com>,
	"Roger Pau Monné" <roger.pau@citrix.com>
Subject: Regressed XSA-286, was [xen-unstable test] 161917: regressions - FAIL
Date: Thu, 13 May 2021 21:15:10 +0100	[thread overview]
Message-ID: <7cfa28ae-2fbe-0945-8a6c-a965ec52155f@citrix.com> (raw)
In-Reply-To: <osstest-161917-mainreport@xen.org>

On 13/05/2021 04:56, osstest service owner wrote:
> flight 161917 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/161917/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 161898
>  test-arm64-arm64-xl-thunderx  8 xen-boot                 fail REGR. vs. 161898
>  test-arm64-arm64-xl-credit1   8 xen-boot                 fail REGR. vs. 161898
>  test-arm64-arm64-xl-credit2   8 xen-boot                 fail REGR. vs. 161898
>  test-arm64-arm64-xl           8 xen-boot                 fail REGR. vs. 161898

I reported these on IRC, and Julien/Stefano have already committed a fix.

> Tests which are failing intermittently (not blocking):
>  test-xtf-amd64-amd64-3 92 xtf/test-pv32pae-xsa-286 fail in 161909 pass in 161917

While noticing the ARM issue above, I also spotted this one by chance. 
There are two issues.

First, I have reverted bed7e6cad30 and edcfce55917.  The XTF test is
correct, and they really do reintroduce XSA-286.  It is a miracle of
timing that we don't need an XSA/CVE against Xen 4.15.

Given that I was unhappy with the changes in the first place, I don't
particularly want to see an attempt to resurrect them.  I did not find
the claim that they were a perf improvement in the first place very
convincing, and the XTF test demonstrates that the reasoning about their
safety was incorrect.


Second, the unexplained OSSTest behaviour.

When I repro'd this on pinot1, test-pv32pae-xsa-286 failing was totally
deterministic and repeatable (I tried 100 times because the test is a
fraction of a second).

From the log trawling which Ian already did, the first recorded failure
was flight 160912 on April 11th.  All failures (12, but this number is a
few flights old now) were on pinot*.

What would be interesting to see is whether there have been any passes
on pinot since 160912.

I can't see any reason why the test would be reliable for me, but
unreliable for OSSTest, so I'm wondering whether it is actually
reliable, and something is wrong with the stickiness heuristic.

~Andrew



  reply	other threads:[~2021-05-13 20:15 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-13  3:56 osstest service owner
2021-05-13 20:15 ` Andrew Cooper [this message]
2021-05-17  8:43   ` Regressed XSA-286, was " Jan Beulich
2021-05-17 10:59     ` Jan Beulich
2021-06-16  8:48   ` Jan Beulich
2021-06-16 15:43     ` Andrew Cooper
2021-06-17 11:56       ` Jan Beulich
2021-06-17 13:05         ` Ian Jackson
2021-06-17 14:40           ` Jan Beulich
2021-06-17 14:49             ` Ian Jackson
2021-06-17 14:55               ` Jan Beulich
2021-06-28 12:35           ` Ping: " Jan Beulich
2021-06-17 21:26         ` Stefano Stabellini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7cfa28ae-2fbe-0945-8a6c-a965ec52155f@citrix.com \
    --to=andrew.cooper3@citrix.com \
    --cc=iwj@xenproject.org \
    --cc=jbeulich@suse.com \
    --cc=osstest-admin@xenproject.org \
    --cc=roger.pau@citrix.com \
    --cc=xen-devel@lists.xenproject.org \
    --subject='Re: Regressed XSA-286, was [xen-unstable test] 161917: regressions - FAIL' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).