[Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Fernando Casas Schössow]

Hi there,


Last night while doing a backup of a guest using the live snapshot mechanism the qemu process for the guest seem to had crashed.

The snapshot succeeded then the backup of the VM disk had place and also succeeded but the commit to the original disk after the backup seem to have failed.

The command I use in the script to take the snapshot is:


virsh snapshot-create-as --domain $VM backup-job.qcow2 --disk-only --atomic --quiesce --no-metadata


And then to commit back is:


virsh blockcommit $VM $TARGETDISK --base $DISKFILE --top $SNAPFILE --active --pivot


In the qemu log for the guest I found the following while the commit back was having place:


Assertion failed: QLIST_EMPTY(&bs->tracked_requests) (/home/buildozer/aports/main/qemu/src/qemu-2.10.1/block/mirror.c: mirror_run: 884)

I'm running qemu 2.10.1 with libvirt 3.9.0 and kernel 4.9.65 on Alpine Linux 3.7.

This is the complete guest info from the logs:


LC_ALL=C PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin HOME=/root USER=root QEMU_AUDIO_DRV=spice /usr/bin/qemu-system-x86_64 -name guest=DOCKER01,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-6-DOCKER01/master-key.aes -machine pc-i440fx-2.8,accel=kvm,usb=off,dump-guest-core=off -cpu IvyBridge,ss=on,vmx=on,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,xsaveopt=on -drive file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/DOCKER01_VARS.fd,if=pflash,format=raw,unit=1 -m 2048 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 4705b146-3b14-4c20-923c-42105d47e7fc -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-6-DOCKER01/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x4.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x4.0x2 -device ahci,id=sata0,bus=pci.0,addr=0x9 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/storage/storage-ssd-vms/virtual_machines_ssd/docker01.qcow2,format=qcow2,if=none,id=drive-sata0-0-0,cache=none,aio=threads -device ide-hd,bus=sata0.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -netdev tap,fd=33,id=hostnet0,vhost=on,vhostfd=35 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:1c:af:ce,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-6-DOCKER01/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -spice port=5905,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -object rng-random,id=objrng0,filename=/dev/random -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x8 -msg timestamp=on



I was running on qemu 2.8.1 for months and didn't have any problems with the backups but yesterday I updated to qemu 2.10.1 and I hit this problem last night.


Is this a bug? Any ideas will be appreciated.


Thanks.

Kind regards,


Fer

Re: [Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 4328 bytes --]

On Thu, Dec 07, 2017 at 10:18:52AM +0000, Fernando Casas Schössow wrote:
> Hi there,
> 
> 
> Last night while doing a backup of a guest using the live snapshot mechanism the qemu process for the guest seem to had crashed.
> 
> The snapshot succeeded then the backup of the VM disk had place and also succeeded but the commit to the original disk after the backup seem to have failed.
> 
> The command I use in the script to take the snapshot is:
> 
> 
> virsh snapshot-create-as --domain $VM backup-job.qcow2 --disk-only --atomic --quiesce --no-metadata
> 
> 
> And then to commit back is:
> 
> 
> virsh blockcommit $VM $TARGETDISK --base $DISKFILE --top $SNAPFILE --active --pivot
> 
> 
> In the qemu log for the guest I found the following while the commit back was having place:
> 
> 
> Assertion failed: QLIST_EMPTY(&bs->tracked_requests) (/home/buildozer/aports/main/qemu/src/qemu-2.10.1/block/mirror.c: mirror_run: 884)
> 
> I'm running qemu 2.10.1 with libvirt 3.9.0 and kernel 4.9.65 on Alpine Linux 3.7.
> 
> This is the complete guest info from the logs:
> 
> 
> LC_ALL=C PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin HOME=/root USER=root QEMU_AUDIO_DRV=spice /usr/bin/qemu-system-x86_64 -name guest=DOCKER01,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-6-DOCKER01/master-key.aes -machine pc-i440fx-2.8,accel=kvm,usb=off,dump-guest-core=off -cpu IvyBridge,ss=on,vmx=on,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,xsaveopt=on -drive file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/DOCKER01_VARS.fd,if=pflash,format=raw,unit=1 -m 2048 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 4705b146-3b14-4c20-923c-42105d47e7fc -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-6-DOCKER01/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x4.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x4.0x2 -device ahci,id=sata0,bus=pci.0,addr=0x9 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/storage/storage-ssd-vms/virtual_machines_ssd/docker01.qcow2,format=qcow2,if=none,id=drive-sata0-0-0,cache=none,aio=threads -device ide-hd,bus=sata0.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -netdev tap,fd=33,id=hostnet0,vhost=on,vhostfd=35 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:1c:af:ce,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-6-DOCKER01/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -spice port=5905,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -object rng-random,id=objrng0,filename=/dev/random -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x8 -msg timestamp=on
> 
> 
> 
> I was running on qemu 2.8.1 for months and didn't have any problems with the backups but yesterday I updated to qemu 2.10.1 and I hit this problem last night.
> 
> 
> Is this a bug? Any ideas will be appreciated.

Thanks for reporting this bug.  Can you reproduce it reliably?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Fernando Casas Schössow]

Hello Stefan,

Thanks for your reply.
Fortunately I didn’t have the problem again and it’s not clear how it can be consistently reproduced. Daily backups are running as usual at the moment.

If there is anything I can do from my side or if you have any ideas to try to reproduce it let me know.

Thanks.

Fer

Sent from my iPhone

On 11 Dec 2017, at 12:56, Stefan Hajnoczi <stefanha@gmail.com<mailto:stefanha@gmail.com>> wrote:

On Thu, Dec 07, 2017 at 10:18:52AM +0000, Fernando Casas Schössow wrote:
Hi there,


Last night while doing a backup of a guest using the live snapshot mechanism the qemu process for the guest seem to had crashed.

The snapshot succeeded then the backup of the VM disk had place and also succeeded but the commit to the original disk after the backup seem to have failed.

The command I use in the script to take the snapshot is:


virsh snapshot-create-as --domain $VM backup-job.qcow2 --disk-only --atomic --quiesce --no-metadata


And then to commit back is:


virsh blockcommit $VM $TARGETDISK --base $DISKFILE --top $SNAPFILE --active --pivot


In the qemu log for the guest I found the following while the commit back was having place:


Assertion failed: QLIST_EMPTY(&bs->tracked_requests) (/home/buildozer/aports/main/qemu/src/qemu-2.10.1/block/mirror.c: mirror_run: 884)

I'm running qemu 2.10.1 with libvirt 3.9.0 and kernel 4.9.65 on Alpine Linux 3.7.

This is the complete guest info from the logs:


LC_ALL=C PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin HOME=/root USER=root QEMU_AUDIO_DRV=spice /usr/bin/qemu-system-x86_64 -name guest=DOCKER01,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-6-DOCKER01/master-key.aes -machine pc-i440fx-2.8,accel=kvm,usb=off,dump-guest-core=off -cpu IvyBridge,ss=on,vmx=on,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,xsaveopt=on -drive file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/DOCKER01_VARS.fd,if=pflash,format=raw,unit=1 -m 2048 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 4705b146-3b14-4c20-923c-42105d47e7fc -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-6-DOCKER01/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x4.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x4.0x2 -device ahci,id=sata0,bus=pci.0,addr=0x9 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/storage/storage-ssd-vms/virtual_machines_ssd/docker01.qcow2,format=qcow2,if=none,id=drive-sata0-0-0,cache=none,aio=threads -device ide-hd,bus=sata0.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -netdev tap,fd=33,id=hostnet0,vhost=on,vhostfd=35 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:1c:af:ce,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-6-DOCKER01/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -spice port=5905,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -object rng-random,id=objrng0,filename=/dev/random -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x8 -msg timestamp=on



I was running on qemu 2.8.1 for months and didn't have any problems with the backups but yesterday I updated to qemu 2.10.1 and I hit this problem last night.


Is this a bug? Any ideas will be appreciated.

Thanks for reporting this bug.  Can you reproduce it reliably?

Stefan

Re: [Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2052 bytes --]

On Mon, Dec 11, 2017 at 01:42:38PM +0000, Fernando Casas Schössow wrote:
> Hello Stefan,
> 
> Thanks for your reply.
> Fortunately I didn’t have the problem again and it’s not clear how it can be consistently reproduced. Daily backups are running as usual at the moment.
> 
> If there is anything I can do from my side or if you have any ideas to try to reproduce it let me know.

Okay, thanks.

In case it happens again, here is some information about the assertion
failure.  It will probably be necessary to use GDB to inspect the failed
process (core dump).  The starting point for debugging is mirror_run()
in block/mirror.c:

  if (cnt == 0 && should_complete) {
      /* The dirty bitmap is not updated while operations are pending.
       * If we're about to exit, wait for pending operations before
       * calling bdrv_get_dirty_count(bs), or we may exit while the
       * source has dirty data to copy!
       *
       * Note that I/O can be submitted by the guest while
       * mirror_populate runs, so pause it now.  Before deciding
       * whether to switch to target check one last time if I/O has
       * come in the meanwhile, and if not flush the data to disk.
       */
      trace_mirror_before_drain(s, cnt);

      bdrv_drained_begin(bs);
      cnt = bdrv_get_dirty_count(s->dirty_bitmap);
      if (cnt > 0 || mirror_flush(s) < 0) {
          bdrv_drained_end(bs);
          continue;
      }

      /* The two disks are in sync.  Exit and report successful
       * completion.
       */
      assert(QLIST_EMPTY(&bs->tracked_requests));

The reason why this assertion makes sense is because
bdrv_drained_begin(bs) starts a region where I/O is quiesced (all
requests should have been completed).

Either there is a bug with blockjobs vs bdrv_drained_debug(), or maybe
mirror_flush() submitted new I/O requests.

It would be interesting to print the bs->tracked_requests linked list
in the GDB debugger.  Those requests might give a clue about the root
cause.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Fernando Casas Schössow]

Thanks for the explanation Stefan.

Maybe I’m missing something here but, if I recall correctly, the qemu process for the guest is terminated when this problem happens. So how a debugger will be attached to a process that is gone?

Also I don’t think the Alpine packages for qemu include the debug info. :(
But I’m not 100% sure at the moment.

Sent from my iPhone

On 13 Dec 2017, at 12:23, Stefan Hajnoczi <stefanha@gmail.com<mailto:stefanha@gmail.com>> wrote:

On Mon, Dec 11, 2017 at 01:42:38PM +0000, Fernando Casas Schössow wrote:
Hello Stefan,

Thanks for your reply.
Fortunately I didn’t have the problem again and it’s not clear how it can be consistently reproduced. Daily backups are running as usual at the moment.

If there is anything I can do from my side or if you have any ideas to try to reproduce it let me know.

Okay, thanks.

In case it happens again, here is some information about the assertion
failure.  It will probably be necessary to use GDB to inspect the failed
process (core dump).  The starting point for debugging is mirror_run()
in block/mirror.c:

 if (cnt == 0 && should_complete) {
     /* The dirty bitmap is not updated while operations are pending.
      * If we're about to exit, wait for pending operations before
      * calling bdrv_get_dirty_count(bs), or we may exit while the
      * source has dirty data to copy!
      *
      * Note that I/O can be submitted by the guest while
      * mirror_populate runs, so pause it now.  Before deciding
      * whether to switch to target check one last time if I/O has
      * come in the meanwhile, and if not flush the data to disk.
      */
     trace_mirror_before_drain(s, cnt);

     bdrv_drained_begin(bs);
     cnt = bdrv_get_dirty_count(s->dirty_bitmap);
     if (cnt > 0 || mirror_flush(s) < 0) {
         bdrv_drained_end(bs);
         continue;
     }

     /* The two disks are in sync.  Exit and report successful
      * completion.
      */
     assert(QLIST_EMPTY(&bs->tracked_requests));

The reason why this assertion makes sense is because
bdrv_drained_begin(bs) starts a region where I/O is quiesced (all
requests should have been completed).

Either there is a bug with blockjobs vs bdrv_drained_debug(), or maybe
mirror_flush() submitted new I/O requests.

It would be interesting to print the bs->tracked_requests linked list
in the GDB debugger.  Those requests might give a clue about the root
cause.

Stefan

Re: [Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

On Wed, Dec 13, 2017 at 03:33:01PM +0000, Fernando Casas Schössow wrote:
> Maybe I’m missing something here but, if I recall correctly, the qemu process for the guest is terminated when this problem happens. So how a debugger will be attached to a process that is gone?

Sorry, this got lost in my inbox.

assert(false) sends SIGABRT to the process.  The default behavior is to
dump a core file that can be analyzed later with GDB.

Your system must have core dumps enabled (i.e. ulimit -c unlimited).
Also, various pieces of software like systemd's coredumpctl can
influence where and how core dumps are stored.

But in short, an assertion failure produces a core dump.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Fernando Casas Schössow]

Hi Stefan,

I hope this email finds you well and I apologize in advance for resurrecting this thread.
I'm currently running on qemu 2.12.1 and I'm still having this issue every few days but now I managed to get a core dump generated (without including the guest memory).
Would you take a look at the dump? Please let me know how do you prefer me to share it. The file is around 290MB as is but I can try to compress it.

Looking forward your reply.
Thanks
Kind regards,

Fernando
________________________________
From: Stefan Hajnoczi <stefanha@gmail.com>
Sent: Monday, January 29, 2018 5:01 PM
To: Fernando Casas Schössow
Cc: qemu-devel; qemu-block@nongnu.org; Jeff Cody
Subject: Re: [Qemu-devel] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

On Wed, Dec 13, 2017 at 03:33:01PM +0000, Fernando Casas Schössow wrote:
> Maybe I’m missing something here but, if I recall correctly, the qemu process for the guest is terminated when this problem happens. So how a debugger will be attached to a process that is gone?

Sorry, this got lost in my inbox.

assert(false) sends SIGABRT to the process.  The default behavior is to
dump a core file that can be analyzed later with GDB.

Your system must have core dumps enabled (i.e. ulimit -c unlimited).
Also, various pieces of software like systemd's coredumpctl can
influence where and how core dumps are stored.

But in short, an assertion failure produces a core dump.

Stefan

Re: [Qemu-devel] [Qemu-block] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Kevin Wolf]

Hi Fernando,

Am 18.10.2018 um 14:25 hat Fernando Casas Schössow geschrieben:
> I hope this email finds you well and I apologize in advance for resurrecting this thread.
> I'm currently running on qemu 2.12.1 and I'm still having this issue every few days but now I managed to get a core dump generated (without including the guest memory).
> Would you take a look at the dump? Please let me know how do you prefer me to share it. The file is around 290MB as is but I can try to compress it.

would it be possible for you to test whether the bug still exists in
QEMU 3.0? There were a few fixes related to request draining in that
release, so maybe it's already solved now.

Kevin

Re: [Qemu-devel] [Qemu-block] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Fernando Casas Schössow]

Hi Kevin,

Not at the moment. This is a production system and pretty much up to date but can't upgrade to 3.0 yet.
If the dump can be of any use, I can upload it somewhere for analysis.

BR,

Fernando

On jue, oct 18, 2018 at 2:38 PM, Kevin Wolf <kwolf@redhat.com> wrote:
Hi Fernando, Am 18.10.2018 um 14:25 hat Fernando Casas Schössow geschrieben:
I hope this email finds you well and I apologize in advance for resurrecting this thread. I'm currently running on qemu 2.12.1 and I'm still having this issue every few days but now I managed to get a core dump generated (without including the guest memory). Would you take a look at the dump? Please let me know how do you prefer me to share it. The file is around 290MB as is but I can try to compress it.
would it be possible for you to test whether the bug still exists in QEMU 3.0? There were a few fixes related to request draining in that release, so maybe it's already solved now. Kevin

Re: [Qemu-devel] [Qemu-block] qemu process crash: Assertion failed: QLIST_EMPTY(&bs->tracked_requests)

[Fernando Casas Schössow]

Hi again,

I hit another occurrence of this issue and collected the qemy dump as well. So at the moment I have two qemu dumps for this issue happening on two different guests.
I wish I know how to analyze them myself but is not the case so if anyone in the list is willing to take a look I'm more than willing to share them.

Thanks in advance.

Fernando

On jue, oct 18, 2018 at 3:40 PM, Fernando Casas Schössow <casasfernando@hotmail.com> wrote:
Hi Kevin,

Not at the moment. This is a production system and pretty much up to date but can't upgrade to 3.0 yet.
If the dump can be of any use, I can upload it somewhere for analysis.

BR,

Fernando

On jue, oct 18, 2018 at 2:38 PM, Kevin Wolf <kwolf@redhat.com> wrote:
Hi Fernando, Am 18.10.2018 um 14:25 hat Fernando Casas Schössow geschrieben:
I hope this email finds you well and I apologize in advance for resurrecting this thread. I'm currently running on qemu 2.12.1 and I'm still having this issue every few days but now I managed to get a core dump generated (without including the guest memory). Would you take a look at the dump? Please let me know how do you prefer me to share it. The file is around 290MB as is but I can try to compress it.
would it be possible for you to test whether the bug still exists in QEMU 3.0? There were a few fixes related to request draining in that release, so maybe it's already solved now. Kevin