* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
@ 2020-11-23 15:17 ` Balazs Gibizer
2020-11-23 15:30 ` sean mooney
` (11 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: Balazs Gibizer @ 2020-11-23 15:17 UTC (permalink / raw)
To: qemu-devel
Source node
https://zuul.opendev.org/t/openstack/build/d50877ae15db4022b82f4bb1d1d52cea/log/logs/subnode-2/libvirt/qemu/instance-0000001a.txt
2020-11-20 14:25:24.887+0000: starting up libvirt version: 6.0.0,
package: 0ubuntu8.4~cloud0 (Openstack Ubuntu Testing Bot <openstack-
testing-bot@ubuntu.com> Tue, 15 Sep 2020 20:36:28 +0000), qemu version:
4.2.1Debian 1:4.2-3ubuntu6.7~cloud0, kernel: 4.15.0-124-generic,
hostname: ubuntu-bionic-ovh-bhs1-0021872195
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-19-instance-0000001a \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-19-instance-
0000001a/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-19-instance-0000001a/.cache
\
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-19-instance-
0000001a/.config \
QEMU_AUDIO_DRV=none \
/usr/bin/qemu-system-x86_64 \
-name guest=instance-0000001a,debug-threads=on \
-S \
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-19
-instance-0000001a/master-key.aes \
-machine pc-i440fx-4.2,accel=tcg,usb=off,dump-guest-core=off \
-cpu qemu64,hypervisor=on,lahf-lm=on \
-m 128 \
-overcommit mem-lock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid 2c468d92-4b19-426a-8c25-16b4624c21a4 \
-smbios 'type=1,manufacturer=OpenStack Foundation,product=OpenStack
Nova,version=22.1.0,serial=2c468d92-4b19-426a-
8c25-16b4624c21a4,uuid=2c468d92-4b19-426a-
8c25-16b4624c21a4,family=Virtual Machine' \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=35,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc \
-no-shutdown \
-boot strict=on \
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
-blockdev
'{"driver":"file","filename":"/opt/stack/data/nova/instances/_base/61bd5e531ab4c82456aa5300ede7266b3610be79
","node-name":"libvirt-2-storage","cache":{"direct":true,"no-
flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-
only":true,"cache":{"direct":true,"no-
flush":false},"driver":"raw","file":"libvirt-2-storage"}' \
-blockdev
'{"driver":"file","filename":"/opt/stack/data/nova/instances/2c468d92-4b19
-426a-8c25-16b4624c21a4/disk","node-
name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false
},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-
only":false,"cache":{"direct":true,"no-
flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}'
\
-device virtio-blk-
pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,id=virtio-
disk0,bootindex=1,write-cache=on \
-netdev tap,fd=37,id=hostnet0 \
-device virtio-net-
pci,host_mtu=1400,netdev=hostnet0,id=net0,mac=fa:16:3e:43:11:f4,bus=pci.0,addr=0x3
\
-add-fd set=2,fd=39 \
-chardev pty,id=charserial0,logfile=/dev/fdset/2,logappend=on \
-device isa-serial,chardev=charserial0,id=serial0 \
-vnc 127.0.0.1:0 \
-device cirrus-vga,id=video0,bus=pci.0,addr=0x2 \
-incoming defer \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 \
-object rng-random,id=objrng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x6 \
-sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
\
-msg timestamp=on
char device redirected to /dev/pts/2 (label charserial0)
virtio: bogus descriptor or out of resources
2020-11-20 14:25:39.911+0000: initiating migration
2020-11-20T14:26:21.409517Z qemu-system-x86_64: terminating on signal 15
from pid 17395 (/usr/sbin/libvirtd)
2020-11-20 14:26:21.610+0000: shutting down, reason=destroyed
Target node
https://zuul.opendev.org/t/openstack/build/d50877ae15db4022b82f4bb1d1d52cea/log/logs/libvir
t/qemu/instance-0000001a.txt
2020-11-20 14:25:11.589+0000: starting up libvirt version: 6.0.0,
package: 0ubuntu8.4~cloud0 (Openstack Ubuntu Testing Bot <openstack-
testing-bot@ubuntu.com> Tue, 15 Sep 2020 20:36:28 +0000), qemu version:
4.2.1Debian 1:4.2-3ubuntu6.7~cloud0, kernel: 4.15.0-124-generic,
hostname: ubuntu-bionic-ovh-bhs1-0021872194
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-10-instance-0000001a \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-10-instance-
0000001a/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-10-instance-0000001a/.cache
\
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-10-instance-
0000001a/.config \
QEMU_AUDIO_DRV=none \
/usr/bin/qemu-system-x86_64 \
-name guest=instance-0000001a,debug-threads=on \
-S \
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10
-instance-0000001a/master-key.aes \
-machine pc-i440fx-4.2,accel=tcg,usb=off,dump-guest-core=off \
-cpu qemu64 \
-m 128 \
-overcommit mem-lock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid 2c468d92-4b19-426a-8c25-16b4624c21a4 \
-smbios 'type=1,manufacturer=OpenStack Foundation,product=OpenStack
Nova,version=22.1.0,serial=2c468d92-4b19-426a-
8c25-16b4624c21a4,uuid=2c468d92-4b19-426a-
8c25-16b4624c21a4,family=Virtual Machine' \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=32,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc \
-no-shutdown \
-boot strict=on \
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
-blockdev
'{"driver":"file","filename":"/opt/stack/data/nova/instances/_base/61bd5e531ab4c82456aa5300ede7266b3610be79
","node-name":"libvirt-2-storage","cache":{"direct":true,"no-
flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-
only":true,"cache":{"direct":true,"no-
flush":false},"driver":"raw","file":"libvirt-2-storage"}' \
-blockdev
'{"driver":"file","filename":"/opt/stack/data/nova/instances/2c468d92-4b19
-426a-8c25-16b4624c21a4/disk","node-
name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false
},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-
only":false,"cache":{"direct":true,"no-
flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}'
\
-device virtio-blk-
pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,id=virtio-
disk0,bootindex=1,write-cache=on \
-netdev tap,fd=34,id=hostnet0 \
-device virtio-net-
pci,host_mtu=1400,netdev=hostnet0,id=net0,mac=fa:16:3e:43:11:f4,bus=pci.0,addr=0x3
\
-add-fd set=2,fd=36 \
-chardev pty,id=charserial0,logfile=/dev/fdset/2,logappend=on \
-device isa-serial,chardev=charserial0,id=serial0 \
-vnc 0.0.0.0:0 \
-device cirrus-vga,id=video0,bus=pci.0,addr=0x2 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 \
-object rng-random,id=objrng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x6 \
-sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
\
-msg timestamp=on
char device redirected to /dev/pts/0 (label charserial0)
2020-11-20 14:25:25.637+0000: initiating migration
2020-11-20 14:25:26.776+0000: shutting down, reason=migrated
2020-11-20T14:25:26.777394Z qemu-system-x86_64: terminating on signal 15
from pid 31113 (/usr/sbin/libvirtd)
2020-11-20 14:25:38.909+0000: starting up libvirt version: 6.0.0,
package: 0ubuntu8.4~cloud0 (Openstack Ubuntu Testing Bot <openstack-
testing-bot@ubuntu.com> Tue, 15 Sep 2020 20:36:28 +0000), qemu version:
4.2.1Debian 1:4.2-3ubuntu6.7~cloud0, kernel: 4.15.0-124-generic,
hostname: ubuntu-bionic-ovh-bhs1-0021872194
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-13-instance-0000001a \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-13-instance-
0000001a/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-13-instance-0000001a/.cache
\
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-13-instance-
0000001a/.config \
QEMU_AUDIO_DRV=none \
/usr/bin/qemu-system-x86_64 \
-name guest=instance-0000001a,debug-threads=on \
-S \
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-13
-instance-0000001a/master-key.aes \
-machine pc-i440fx-4.2,accel=tcg,usb=off,dump-guest-core=off \
-cpu qemu64,hypervisor=on,lahf-lm=on \
-m 128 \
-overcommit mem-lock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid 2c468d92-4b19-426a-8c25-16b4624c21a4 \
-smbios 'type=1,manufacturer=OpenStack Foundation,product=OpenStack
Nova,version=22.1.0,serial=2c468d92-4b19-426a-
8c25-16b4624c21a4,uuid=2c468d92-4b19-426a-
8c25-16b4624c21a4,family=Virtual Machine' \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=34,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc \
-no-shutdown \
-boot strict=on \
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
-blockdev
'{"driver":"file","filename":"/opt/stack/data/nova/instances/_base/61bd5e531ab4c82456aa5300ede7266b3610be79
","node-name":"libvirt-2-storage","cache":{"direct":true,"no-
flush":false},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-2-format","read-
only":true,"cache":{"direct":true,"no-
flush":false},"driver":"raw","file":"libvirt-2-storage"}' \
-blockdev
'{"driver":"file","filename":"/opt/stack/data/nova/instances/2c468d92-4b19
-426a-8c25-16b4624c21a4/disk","node-
name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false
},"auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-
only":false,"cache":{"direct":true,"no-
flush":false},"driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-2-format"}'
\
-device virtio-blk-
pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,id=virtio-
disk0,bootindex=1,write-cache=on \
-netdev tap,fd=36,id=hostnet0 \
-device virtio-net-
pci,host_mtu=1400,netdev=hostnet0,id=net0,mac=fa:16:3e:43:11:f4,bus=pci.0,addr=0x3
\
-add-fd set=2,fd=38 \
-chardev pty,id=charserial0,logfile=/dev/fdset/2,logappend=on \
-device isa-serial,chardev=charserial0,id=serial0 \
-vnc 0.0.0.0:0 \
-device cirrus-vga,id=video0,bus=pci.0,addr=0x2 \
-incoming defer \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 \
-object rng-random,id=objrng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x6 \
-sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
\
-msg timestamp=on
char device redirected to /dev/pts/1 (label charserial0)
2020-11-20T14:25:40.720757Z qemu-system-x86_64: VQ 0 size 0x80 Guest
index 0xb8 inconsistent with Host index 0xe0: delta 0xffd8
2020-11-20T14:25:40.720785Z qemu-system-x86_64: Failed to load virtio-
blk:virtio
2020-11-20T14:25:40.720790Z qemu-system-x86_64: error while loading
state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2020-11-20T14:25:40.720824Z qemu-system-x86_64: load of migration
failed: Operation not permitted
2020-11-20 14:25:40.778+0000: shutting down, reason=failed
** Also affects: qemu
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
New
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
2020-11-23 15:17 ` [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8" Balazs Gibizer
@ 2020-11-23 15:30 ` sean mooney
2020-11-23 16:56 ` Kashyap Chamarthy
` (10 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: sean mooney @ 2020-11-23 15:30 UTC (permalink / raw)
To: qemu-devel
in the new case we have the same qemu/libvirt version in both nodes
4.2.1Debian 1:4.2-3ubuntu6.7~cloud0
and in this cae its failng for the virtio-blk device not the memroy
ballon
before the migration starts we see
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
char device redirected to /dev/pts/2 (label charserial0)
virtio: bogus descriptor or out of resources
2020-11-20 14:25:39.911+0000: initiating migration
on the destination
https://zuul.opendev.org/t/openstack/build/d50877ae15db4022b82f4bb1d1d52cea/log/logs/subnode-2/libvirt/qemu/instance-0000001a.txt
then on the source we see
char device redirected to /dev/pts/1 (label charserial0)
2020-11-20T14:25:40.720757Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0xb8 inconsistent with Host index 0xe0: delta 0xffd8
2020-11-20T14:25:40.720785Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2020-11-20T14:25:40.720790Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2020-11-20T14:25:40.720824Z qemu-system-x86_64: load of migration failed: Operation not permitted
when it fails
https://zuul.opendev.org/t/openstack/build/d50877ae15db4022b82f4bb1d1d52cea/log/logs/libvirt/qemu/instance-0000001a.txt
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
New
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
2020-11-23 15:17 ` [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8" Balazs Gibizer
2020-11-23 15:30 ` sean mooney
@ 2020-11-23 16:56 ` Kashyap Chamarthy
2020-11-24 12:17 ` Dr. David Alan Gilbert
` (9 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: Kashyap Chamarthy @ 2020-11-23 16:56 UTC (permalink / raw)
To: qemu-devel
[Copy/pasting my comment from here:
https://bugs.launchpad.net/nova/+bug/1737625/comments/4]
I just talked to Dave Gilbert from upstream QEMU. Overall, as I implied
in comment#2, this gnarly issue requires specialized debugging, digging
deep into the bowels of QEMU, 'virtio-blk' and 'virtio.
That said, Dave notes that we get this "guest index inconsistent" error
when the migrated RAM is inconsistent with the migrated 'virtio' device
state. And a common case is where a 'virtio' device does an operation
after the vCPU is stopped and after RAM has been transmitted.
Dave makes some guesswork of a potential scenario where this can occur:
- Guest is running
- ... live migration starts
- ... a "block read" request gets submitteed
- ... live migration stops the vCPUs, finishes transmitting RAM
- ... the "block read" completes, 'virtio-blk' updates pointers
- ... live migration "serializes" the 'virito-blk' state
So the "guest index inconsistent" state would only happen if you got
unlucky with the timing of that read.
Another possibility, Dave points out, is that the guest has screwed up
the device state somehow; the migration code in 'virtio' checks the
state a lot. We have ruled this possibility out becausethe guest is just
a garden-variety CirrOS instance idling; nothing special about it.
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
New
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (2 preceding siblings ...)
2020-11-23 16:56 ` Kashyap Chamarthy
@ 2020-11-24 12:17 ` Dr. David Alan Gilbert
2020-11-25 13:44 ` Balazs Gibizer
` (8 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: Dr. David Alan Gilbert @ 2020-11-24 12:17 UTC (permalink / raw)
To: qemu-devel
hang on, I've just noticed the :
char device redirected to /dev/pts/2 (label charserial0)
virtio: bogus descriptor or out of resources
2020-11-20 14:25:39.911+0000: initiating migration
I've never seen that 'bogus descriptor or out of resources' one before; the fact that's happening before the migration starts is suspicious that something has already gone wrong before migration started.
Is that warning present in all these failures?
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
New
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (3 preceding siblings ...)
2020-11-24 12:17 ` Dr. David Alan Gilbert
@ 2020-11-25 13:44 ` Balazs Gibizer
2020-11-25 14:03 ` Dr. David Alan Gilbert
` (7 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: Balazs Gibizer @ 2020-11-25 13:44 UTC (permalink / raw)
To: qemu-devel
I see two recent hits we have still logs for.
The one described in comment #10
And another but there the error message is a bit different:
on the migration source host:
: https://6f0be18d925d64906a23-689ad0b9b6f06bc0c51bfb99bf86ea04.ssl.cf5.rackcdn.com/698706/4/check/nova-grenade-multinode/ee2dbea/logs/libvirt/qemu/instance-0000001b.txt
char device redirected to /dev/pts/0 (label charserial0)
virtio: zero sized buffers are not allowed
2020-11-23 22:20:54.297+0000: initiating migration
on the migration destination
https://6f0be18d925d64906a23-689ad0b9b6f06bc0c51bfb99bf86ea04.ssl.cf5.rackcdn.com/698706/4/check/nova-grenade-multinode/ee2dbea/logs/subnode-2/libvirt/qemu/instance-0000001b.txt
char device redirected to /dev/pts/0 (label charserial0)
2020-11-23T22:20:55.129189Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x62 inconsistent with Host index 0xa1: delta 0xffc1
2020-11-23T22:20:55.129230Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2020-11-23T22:20:55.129241Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-blk'
2020-11-23T22:20:55.129259Z qemu-system-x86_64: load of migration failed: Operation not permitted
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
New
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (4 preceding siblings ...)
2020-11-25 13:44 ` Balazs Gibizer
@ 2020-11-25 14:03 ` Dr. David Alan Gilbert
2021-05-10 4:32 ` Thomas Huth
` (6 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: Dr. David Alan Gilbert @ 2020-11-25 14:03 UTC (permalink / raw)
To: qemu-devel
OK, but that still says in both cases here we've got a virtio error
telling us that the queues are broken before migration even starts; so
we should try and figure out why that's happening first.
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
New
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (5 preceding siblings ...)
2020-11-25 14:03 ` Dr. David Alan Gilbert
@ 2021-05-10 4:32 ` Thomas Huth
2021-05-19 9:53 ` ignazio
` (5 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: Thomas Huth @ 2021-05-10 4:32 UTC (permalink / raw)
To: qemu-devel
Is this still an issue with the latest release of QEMU (v6.0)?
** Changed in: qemu
Status: New => Incomplete
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
Incomplete
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (6 preceding siblings ...)
2021-05-10 4:32 ` Thomas Huth
@ 2021-05-19 9:53 ` ignazio
2021-05-19 15:18 ` ignazio
` (4 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: ignazio @ 2021-05-19 9:53 UTC (permalink / raw)
To: qemu-devel
I got same issue on centos 7 stein
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
Incomplete
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (7 preceding siblings ...)
2021-05-19 9:53 ` ignazio
@ 2021-05-19 15:18 ` ignazio
2021-05-19 15:50 ` Dr. David Alan Gilbert
` (3 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: ignazio @ 2021-05-19 15:18 UTC (permalink / raw)
To: qemu-devel
Hello, some news ....I wonder if they can help:
I am testing with some virtual machine again.
If I follows this steps it works (but I lost network connection):
1) Detach network interface from instance
2) Attach network interface to instance
3) Migrate instance
4) Loggin into instance using console and restart networking
while if I restart networking before live migration it does not work.
So, when someone mentioned
########################
we get this "guest index inconsistent" error when the migrated RAM is inconsistent with the migrated 'virtio' device state. And a common case is where a 'virtio' device does an operation after the vCPU is stopped and after RAM has been transmitted.
#############################à
the network traffic could be the problem ?
Ignazio
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
Incomplete
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (8 preceding siblings ...)
2021-05-19 15:18 ` ignazio
@ 2021-05-19 15:50 ` Dr. David Alan Gilbert
2021-08-25 7:28 ` Thomas Huth
` (2 subsequent siblings)
12 siblings, 0 replies; 13+ messages in thread
From: Dr. David Alan Gilbert @ 2021-05-19 15:50 UTC (permalink / raw)
To: qemu-devel
Be careful, it might not be the same bug.
Yes, it *shouldn't* be a problem, but if the virtio code in qemu is
broken then it will keep accepting incoming packets even when the guest
is stopped in the final part of the migration and you get the contents
of the RAM taken before the reception ofthe packet, but hte virtio state
that's in the migration stream after the reception of the packet, and
it's inconsistent.
But the case the other reporter mentioned is on a virtio-blk device; the
same thing can happen if the storage device stalls/is slow during the
migration code - i.e. a block read takes ages to complete and happens to
complete after the point it should have stopped for migration.
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Confirmed
Status in QEMU:
Incomplete
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/screen-n-cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (9 preceding siblings ...)
2021-05-19 15:50 ` Dr. David Alan Gilbert
@ 2021-08-25 7:28 ` Thomas Huth
2021-10-25 4:17 ` Launchpad Bug Tracker
2021-10-25 4:17 ` Launchpad Bug Tracker
12 siblings, 0 replies; 13+ messages in thread
From: Thomas Huth @ 2021-08-25 7:28 UTC (permalink / raw)
To: qemu-devel
Is this still happening with the latest release?
** Changed in: nova
Status: Confirmed => Incomplete
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Incomplete
Status in QEMU:
Incomplete
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-migration/8de6e74/logs/screen-n-
cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (10 preceding siblings ...)
2021-08-25 7:28 ` Thomas Huth
@ 2021-10-25 4:17 ` Launchpad Bug Tracker
2021-10-25 4:17 ` Launchpad Bug Tracker
12 siblings, 0 replies; 13+ messages in thread
From: Launchpad Bug Tracker @ 2021-10-25 4:17 UTC (permalink / raw)
To: qemu-devel
[Expired for OpenStack Compute (nova) because there has been no activity
for 60 days.]
** Changed in: nova
Status: Incomplete => Expired
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Expired
Status in QEMU:
Expired
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-migration/8de6e74/logs/screen-n-
cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
` (11 preceding siblings ...)
2021-10-25 4:17 ` Launchpad Bug Tracker
@ 2021-10-25 4:17 ` Launchpad Bug Tracker
12 siblings, 0 replies; 13+ messages in thread
From: Launchpad Bug Tracker @ 2021-10-25 4:17 UTC (permalink / raw)
To: qemu-devel
[Expired for QEMU because there has been no activity for 60 days.]
** Changed in: qemu
Status: Incomplete => Expired
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1761798
Title:
live migration intermittently fails in CI with "VQ 0 size 0x80 Guest
index 0x12c inconsistent with Host index 0x134: delta 0xfff8"
Status in OpenStack Compute (nova):
Expired
Status in QEMU:
Expired
Bug description:
Seen here:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-
migration/8de6e74/logs/subnode-2/libvirt/qemu/instance-00000002.txt.gz
2018-04-05T21:48:38.205752Z qemu-system-x86_64: -chardev pty,id=charserial0,logfile=/dev/fdset/1,logappend=on: char device redirected to /dev/pts/0 (label charserial0)
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
2018-04-05T21:48:43.153268Z qemu-system-x86_64: VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8
2018-04-05T21:48:43.153288Z qemu-system-x86_64: Failed to load virtio-blk:virtio
2018-04-05T21:48:43.153292Z qemu-system-x86_64: error while loading state for instance 0x0 of device '0000:00:04.0/virtio-blk'
2018-04-05T21:48:43.153347Z qemu-system-x86_64: load of migration failed: Operation not permitted
2018-04-05 21:48:43.198+0000: shutting down, reason=crashed
And in the n-cpu logs on the other host:
http://logs.openstack.org/37/522537/20/check/legacy-tempest-dsvm-
multinode-live-migration/8de6e74/logs/screen-n-
cpu.txt.gz#_Apr_05_21_48_43_257541
There is a related Red Hat bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1450524
The CI job failures are at present using the Pike UCA:
ii libvirt-bin 3.6.0-1ubuntu6.2~cloud0
ii qemu-system-x86 1:2.10+dfsg-0ubuntu3.5~cloud0
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1761798/+subscriptions
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2021-10-25 4:32 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <152303245198.3233.1033096993665779324.malonedeb@wampee.canonical.com>
2020-11-23 15:17 ` [Bug 1761798] Re: live migration intermittently fails in CI with "VQ 0 size 0x80 Guest index 0x12c inconsistent with Host index 0x134: delta 0xfff8" Balazs Gibizer
2020-11-23 15:30 ` sean mooney
2020-11-23 16:56 ` Kashyap Chamarthy
2020-11-24 12:17 ` Dr. David Alan Gilbert
2020-11-25 13:44 ` Balazs Gibizer
2020-11-25 14:03 ` Dr. David Alan Gilbert
2021-05-10 4:32 ` Thomas Huth
2021-05-19 9:53 ` ignazio
2021-05-19 15:18 ` ignazio
2021-05-19 15:50 ` Dr. David Alan Gilbert
2021-08-25 7:28 ` Thomas Huth
2021-10-25 4:17 ` Launchpad Bug Tracker
2021-10-25 4:17 ` Launchpad Bug Tracker
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.