On Thu, Mar 25, 2021 at 01:17:50PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> 25.03.2021 12:56, Stefan Hajnoczi wrote:
> > On Wed, Mar 24, 2021 at 08:42:27PM +0000, Peter Maydell wrote:
> > > On Wed, 24 Mar 2021 at 20:18, Vladimir Sementsov-Ogievskiy
> > > <vsementsov@virtuozzo.com> wrote:
> > > > 
> > > > 24.03.2021 21:05, Peter Maydell wrote:
> > > > > On Wed, 24 Mar 2021 at 14:52, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > > > > > 
> > > > > > Vladimir Sementsov-Ogievskiy (2):
> > > > > >     migration/block-dirty-bitmap: make incoming disabled bitmaps busy
> > > > > >     migrate-bitmaps-postcopy-test: check that we can't remove in-flight
> > > > > >       bitmaps
> > > > > 
> > > > > This failed the 'qsd-jobs' iotest on s390x:
> > > 
> > > > I can't believe it related. My commit modifies bitmap status during bitmaps migration on target vm. There is no kind of migration in qsd-jobs test.
> > > 
> > > It's possible it's an intermittent, but it's not one I've seen
> > > before. We still have lots of time this release cycle to figure
> > > out the issue and get this fix in.
> > 
> > Vladimir: I'll get hold of an s390 machine and try to reproduce the
> > failure. I should have some news by Monday.
> 
> Thanks! My path modifies migration/block-dirty-bitmap.c. qsd-jobs runs block-commit and block-stream jobs and don't start any kind of migration or snapshot or savevm, so it seems impossible that qsd-jobs runs the code touched by my patch..

Confirmed. The failure is not related to this pull request.

I reproduced the same s390x host failure on commit
9e2e9fe3df9f539f8b6941ceb96d25355fdae47e (HEAD -> master, tag:
v6.0.0-rc0, origin/master, origin/HEAD):

qsd-jobs                        fail       [11:04:58] [11:04:58]   0.1s                 output mismatch (see qsd-jobs.out.bad)
--- /root/qemu/tests/qemu-iotests/tests/qsd-jobs.out
+++ qsd-jobs.out.bad
@@ -9,11 +9,11 @@
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
 {"return": {}}
+{"return": {}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "job0"}}
+{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}}
-{"return": {}}
-{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "job0"}}
-{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}}

Peter: Please merge this pull request since the failure is pre-existing.

Thanks,
Stefan