[Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"

From: "Dr. David Alan Gilbert" <1761798@bugs.launchpad.net>
To: qemu-devel@nongnu.org
Subject: [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Date: Wed, 19 May 2021 15:50:40 -0000	[thread overview]
Message-ID: <162143944120.1787.9082604949025768706.malone@soybean.canonical.com> (raw)
In-Reply-To: 152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com

Be careful, it might not be the same bug.

Yes, it *shouldn't* be a problem, but if the virtio code in qemu is
broken then it will keep accepting incoming packets even when the guest
is stopped in the final part of the migration and you get the contents
of the RAM taken before the reception ofthe packet, but hte virtio state
that's in the migration stream after the reception of the packet, and
it's inconsistent.

But the case the other reporter mentioned is on a virtio-blk device; the
same thing can happen if the storage device stalls/is slow during the
migration code - i.e. a block read takes ages to complete and happens to
complete after the point it should have stopped for migration.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798

Title:
  live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
  index 0x12c inconsistent with Host index 0x134: delta 0xfff8"

Status in OpenStack Compute (nova):
  Confirmed
Status in QEMU:
  Incomplete

Bug description:
  Seen here:

  http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
  multinode-live-
  migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz

  2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
  warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
  2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
  2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
  2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
  2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
  2018-04-05 21:48:43.198+0000: shutting down, reason=crashed

  And in the n-cpu logs on the other host:

  http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
  multinode-live-
  migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541

  There is a related Red Hat bug:

  https://bugzilla.redhat.com/show_bug.cgi?id=1450524

  The CI job failures are at present using the Pike UCA:

  ii  libvirt-bin                         3.6.0-1ubuntu6.2~cloud0

  ii  qemu-system-x86                     1:2.10+dfsg-0ubuntu3.5~cloud0

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions