* qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
@ 2019-10-18 21:41 Fernando Casas Schössow
2019-10-23 15:34 ` John Snow
0 siblings, 1 reply; 11+ messages in thread
From: Fernando Casas Schössow @ 2019-10-18 21:41 UTC (permalink / raw)
To: QEMU Developers; +Cc: qemu-block
Hi,
Today while working with two different Windows Server 2012 R2 guests I
found that when I try to attach an ISO file to a SCSI CD-ROM device
through libvirt (virsh or virt-manager) while the guest is running,
qemu crashes and the following message is logged:
Assertion failed: blk_get_aio_context(d->conf.blk) == s->ctx
(/home/buildozer/aports/main/qemu/src/qemu-4.0.0/hw/scsi/virtio-scsi.c:
virtio_scsi_ctx_check: 246)
I can repro this at will. All I have to do is to try to attach an ISO
file to the SCSI CDROM while the guest is running.
The SCSI controller model is virtio-scsi with iothread enabled.
Please find below all the details about my setup that I considered
relevant but I missed something please don't hesitate to let me know:
Host arch: x86_64
Distro: Alpine Linux 3.10.2
qemu version: 4.0
Linux kernel version: 4.19.67
libvirt: 5.5.0
Emulated SCSI controller: virtio-scsi (with iothread enabled)
Guest firmware: OVMF-EFI
Guest OS: Window Server 2012 R2
Guest virtio drivers version: 171 (current stable)
qemu command line:
/usr/bin/qemu-system-x86_64 -name guest=DCHOMENET01,debug-threads=on -S
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-78-DCHOMENET01/master-key.aes
-machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=on -cpu
IvyBridge,ss=on,vmx=off,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,xsaveopt=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff
-drive
file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on
-drive
file=/var/lib/libvirt/qemu/nvram/DCHOMENET01_VARS.fd,if=pflash,format=raw,unit=1
-m 1536 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1
-object iothread,id=iothread1 -uuid
f06978ad-2734-44ab-a518-5dfcf71d625e -no-user-config -nodefaults
-chardev socket,id=charmonitor,fd=33,server,nowait -mon
chardev=charmonitor,id=monitor,mode=control -rtc
base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay
-no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global
PIIX4_PM.disable_s4=1 -boot strict=on -device
qemu-xhci,id=usb,bus=pci.0,addr=0x4 -device
virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=1,bus=pci.0,addr=0x5
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive
file=/storage/storage-hdd-vms/virtual_machines_hdd/dchomenet01.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none,discard=unmap,aio=threads
-device
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on
-drive if=none,id=drive-scsi0-0-0-1,readonly=on -device
scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1
-netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=43 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:99:b5:62,bus=pci.0,addr=0x3
-chardev
socket,id=charserial0,host=127.0.0.1,port=4900,telnet,server,nowait
-device isa-serial,chardev=charserial0,id=serial0 -chardev
spicevmc,id=charchannel0,name=vdagent -device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
-chardev socket,id=charchannel1,fd=45,server,nowait -device
virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
-chardev spiceport,id=charchannel2,name=org.spice-space.webdav.0
-device
virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=org.spice-space.webdav.0
-device virtio-tablet-pci,id=input2,bus=pci.0,addr=0x7 -spice
port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on
-device
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2
-chardev spicevmc,id=charredir0,name=usbredir -device
usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev
spicevmc,id=charredir1,name=usbredir -device
usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
-msg timestamp=on
I can provide a core dump of the process if needed for debugging and
the guest XML as well.
Thanks.
Fernando
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
2019-10-18 21:41 qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt Fernando Casas Schössow
@ 2019-10-23 15:34 ` John Snow
2019-10-23 17:28 ` Fernando Casas Schössow
0 siblings, 1 reply; 11+ messages in thread
From: John Snow @ 2019-10-23 15:34 UTC (permalink / raw)
To: Fernando Casas Schössow, QEMU Developers; +Cc: Peter Krempa, qemu-block
On 10/18/19 5:41 PM, Fernando Casas Schössow wrote:
> Hi,
>
Hi! Thanks for the report.
> Today while working with two different Windows Server 2012 R2 guests I
> found that when I try to attach an ISO file to a SCSI CD-ROM device
> through libvirt (virsh or virt-manager) while the guest is running,
> qemu crashes and the following message is logged:
>
> Assertion failed: blk_get_aio_context(d->conf.blk) == s->ctx
> (/home/buildozer/aports/main/qemu/src/qemu-4.0.0/hw/scsi/virtio-scsi.c:
> virtio_scsi_ctx_check: 246)
>
> I can repro this at will. All I have to do is to try to attach an ISO
> file to the SCSI CDROM while the guest is running.
> The SCSI controller model is virtio-scsi with iothread enabled.
> Please find below all the details about my setup that I considered
> relevant but I missed something please don't hesitate to let me know:
>
Looks like we got aio_context management wrong with iothread for the
media change events somewhere. Should be easy enough to fix if we figure
out where the bad assumption is.
> Host arch: x86_64
> Distro: Alpine Linux 3.10.2
> qemu version: 4.0
Do you have the ability to try 4.1, or the latest development head with
debugging symbols enabled?
> Linux kernel version: 4.19.67
> libvirt: 5.5.0
> Emulated SCSI controller: virtio-scsi (with iothread enabled)
> Guest firmware: OVMF-EFI
> Guest OS: Window Server 2012 R2
> Guest virtio drivers version: 171 (current stable)
>
> qemu command line:
>
> /usr/bin/qemu-system-x86_64 -name guest=DCHOMENET01,debug-threads=on -S
> -object
> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-78-DCHOMENET01/master-key.aes
> -machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=on -cpu
> IvyBridge,ss=on,vmx=off,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,xsaveopt=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff
> -drive
> file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on
> -drive
> file=/var/lib/libvirt/qemu/nvram/DCHOMENET01_VARS.fd,if=pflash,format=raw,unit=1
> -m 1536 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1
> -object iothread,id=iothread1 -uuid
> f06978ad-2734-44ab-a518-5dfcf71d625e -no-user-config -nodefaults
> -chardev socket,id=charmonitor,fd=33,server,nowait -mon
> chardev=charmonitor,id=monitor,mode=control -rtc
> base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay
> -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global
> PIIX4_PM.disable_s4=1 -boot strict=on -device
> qemu-xhci,id=usb,bus=pci.0,addr=0x4 -device
> virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=1,bus=pci.0,addr=0x5
> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive
> file=/storage/storage-hdd-vms/virtual_machines_hdd/dchomenet01.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none,discard=unmap,aio=threads
> -device
> scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on
> -drive if=none,id=drive-scsi0-0-0-1,readonly=on -device
> scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1
> -netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=43 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:99:b5:62,bus=pci.0,addr=0x3
> -chardev
> socket,id=charserial0,host=127.0.0.1,port=4900,telnet,server,nowait
> -device isa-serial,chardev=charserial0,id=serial0 -chardev
> spicevmc,id=charchannel0,name=vdagent -device
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
> -chardev socket,id=charchannel1,fd=45,server,nowait -device
> virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
> -chardev spiceport,id=charchannel2,name=org.spice-space.webdav.0
> -device
> virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=org.spice-space.webdav.0
> -device virtio-tablet-pci,id=input2,bus=pci.0,addr=0x7 -spice
> port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on
> -device
> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2
> -chardev spicevmc,id=charredir0,name=usbredir -device
> usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev
> spicevmc,id=charredir1,name=usbredir -device
> usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox
> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
> -msg timestamp=on
>
> I can provide a core dump of the process if needed for debugging and
> the guest XML as well.
>
A backtrace is probably a great starting point (from gdb: `thread apply
all bt`.) I don't know exactly what codepath is being exercised when you
"attach an ISO file" through libvirt's interface.
If you don't mind the hassle, trying on the 4.1 (or a development build
would be even more luxurious) and giving a stacktrace would be nice.
> Thanks.
>
> Fernando
>
>
Thanks!
--js
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
2019-10-23 15:34 ` John Snow
@ 2019-10-23 17:28 ` Fernando Casas Schössow
2019-10-23 17:33 ` John Snow
2019-10-25 10:07 ` Kevin Wolf
0 siblings, 2 replies; 11+ messages in thread
From: Fernando Casas Schössow @ 2019-10-23 17:28 UTC (permalink / raw)
To: John Snow; +Cc: Peter Krempa, QEMU Developers, qemu-block
[-- Attachment #1: Type: text/plain, Size: 5438 bytes --]
Hi John,
Thanks for looking into this.
I can quickly repro the problem with qemu 4.0 binary with debugging symbols enabled as I have it available already.
Doing the same with qemu 4.1 or development head may be too much hassle but if it's really the only way I can give it try.
Would it worth it to try with 4.0 first and get the strack trace or it will not help and the only way to go is with 4.1 (or dev)?
Thanks,
Fernando
On mié, oct 23, 2019 at 5:34 PM, John Snow <jsnow@redhat.com> wrote:
On 10/18/19 5:41 PM, Fernando Casas Schössow wrote:
Hi,
Hi! Thanks for the report.
Today while working with two different Windows Server 2012 R2 guests I found that when I try to attach an ISO file to a SCSI CD-ROM device through libvirt (virsh or virt-manager) while the guest is running, qemu crashes and the following message is logged: Assertion failed: blk_get_aio_context(d->conf.blk) == s->ctx (/home/buildozer/aports/main/qemu/src/qemu-4.0.0/hw/scsi/virtio-scsi.c: virtio_scsi_ctx_check: 246) I can repro this at will. All I have to do is to try to attach an ISO file to the SCSI CDROM while the guest is running. The SCSI controller model is virtio-scsi with iothread enabled. Please find below all the details about my setup that I considered relevant but I missed something please don't hesitate to let me know:
Looks like we got aio_context management wrong with iothread for the media change events somewhere. Should be easy enough to fix if we figure out where the bad assumption is.
Host arch: x86_64 Distro: Alpine Linux 3.10.2 qemu version: 4.0
Do you have the ability to try 4.1, or the latest development head with debugging symbols enabled?
Linux kernel version: 4.19.67 libvirt: 5.5.0 Emulated SCSI controller: virtio-scsi (with iothread enabled) Guest firmware: OVMF-EFI Guest OS: Window Server 2012 R2 Guest virtio drivers version: 171 (current stable) qemu command line: /usr/bin/qemu-system-x86_64 -name guest=DCHOMENET01,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-78-DCHOMENET01/master-key.aes -machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=on -cpu IvyBridge,ss=on,vmx=off,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,xsaveopt=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff -drive file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/DCHOMENET01_VARS.fd,if=pflash,format=raw,unit=1 -m 1536 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1 -uuid f06978ad-2734-44ab-a518-5dfcf71d625e -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=33,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x4 -device virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=1,bus=pci.0,addr=0x5 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/storage/storage-hdd-vms/virtual_machines_hdd/dchomenet01.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none,discard=unmap,aio=threads -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on -drive if=none,id=drive-scsi0-0-0-1,readonly=on -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1 -netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=43 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:99:b5:62,bus=pci.0,addr=0x3 -chardev socket,id=charserial0,host=127.0.0.1,port=4900,telnet,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -chardev socket,id=charchannel1,fd=45,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spiceport,id=charchannel2,name=org.spice-space.webdav.0 -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=org.spice-space.webdav.0 -device virtio-tablet-pci,id=input2,bus=pci.0,addr=0x7 -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on I can provide a core dump of the process if needed for debugging and the guest XML as well.
A backtrace is probably a great starting point (from gdb: `thread apply all bt`.) I don't know exactly what codepath is being exercised when you "attach an ISO file" through libvirt's interface. If you don't mind the hassle, trying on the 4.1 (or a development build would be even more luxurious) and giving a stacktrace would be nice.
Thanks. Fernando
Thanks! --js
[-- Attachment #2: Type: text/html, Size: 6023 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
2019-10-23 17:28 ` Fernando Casas Schössow
@ 2019-10-23 17:33 ` John Snow
2019-10-23 17:57 ` Fernando Casas Schössow
2019-10-25 10:07 ` Kevin Wolf
1 sibling, 1 reply; 11+ messages in thread
From: John Snow @ 2019-10-23 17:33 UTC (permalink / raw)
To: Fernando Casas Schössow; +Cc: Peter Krempa, QEMU Developers, qemu-block
On 10/23/19 1:28 PM, Fernando Casas Schössow wrote:
> Hi John,
>
> Thanks for looking into this.
> I can quickly repro the problem with qemu 4.0 binary with debugging
> symbols enabled as I have it available already.
> Doing the same with qemu 4.1 or development head may be too much hassle
> but if it's really the only way I can give it try.
>
> Would it worth it to try with 4.0 first and get the strack trace or it
> will not help and the only way to go is with 4.1 (or dev)?
>
> Thanks,
>
> Fernando
>
If 4.0 is what you have access to, having the stack trace for that is
better than not, but confirming it happens on the latest release would
be nice.
Can you share your workflow for virsh that reproduces the failure?
--js
> On mié, oct 23, 2019 at 5:34 PM, John Snow <jsnow@redhat.com> wrote:
>> On 10/18/19 5:41 PM, Fernando Casas Schössow wrote:
>>
>> Hi,
>>
>> Hi! Thanks for the report.
>>
>> Today while working with two different Windows Server 2012 R2
>> guests I found that when I try to attach an ISO file to a SCSI
>> CD-ROM device through libvirt (virsh or virt-manager) while the
>> guest is running, qemu crashes and the following message is
>> logged: Assertion failed: blk_get_aio_context(d->conf.blk) ==
>> s->ctx
>> (/home/buildozer/aports/main/qemu/src/qemu-4.0.0/hw/scsi/virtio-scsi.c:
>> virtio_scsi_ctx_check: 246) I can repro this at will. All I have
>> to do is to try to attach an ISO file to the SCSI CDROM while the
>> guest is running. The SCSI controller model is virtio-scsi with
>> iothread enabled. Please find below all the details about my setup
>> that I considered relevant but I missed something please don't
>> hesitate to let me know:
>>
>> Looks like we got aio_context management wrong with iothread for the
>> media change events somewhere. Should be easy enough to fix if we
>> figure out where the bad assumption is.
>>
>> Host arch: x86_64 Distro: Alpine Linux 3.10.2 qemu version: 4.0
>>
>> Do you have the ability to try 4.1, or the latest development head
>> with debugging symbols enabled?
>>
>> Linux kernel version: 4.19.67 libvirt: 5.5.0 Emulated SCSI
>> controller: virtio-scsi (with iothread enabled) Guest firmware:
>> OVMF-EFI Guest OS: Window Server 2012 R2 Guest virtio drivers
>> version: 171 (current stable) qemu command line:
>> /usr/bin/qemu-system-x86_64 -name
>> guest=DCHOMENET01,debug-threads=on -S -object
>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-78-DCHOMENET01/master-key.aes
>> -machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=on -cpu
>> IvyBridge,ss=on,vmx=off,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,xsaveopt=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff
>> -drive
>> file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on
>> -drive
>> file=/var/lib/libvirt/qemu/nvram/DCHOMENET01_VARS.fd,if=pflash,format=raw,unit=1
>> -m 1536 -overcommit mem-lock=off -smp
>> 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1 -uuid
>> f06978ad-2734-44ab-a518-5dfcf71d625e -no-user-config -nodefaults
>> -chardev socket,id=charmonitor,fd=33,server,nowait -mon
>> chardev=charmonitor,id=monitor,mode=control -rtc
>> base=localtime,driftfix=slew -global
>> kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global
>> PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot
>> strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x4 -device
>> virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=1,bus=pci.0,addr=0x5
>> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6
>> -drive
>> file=/storage/storage-hdd-vms/virtual_machines_hdd/dchomenet01.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none,discard=unmap,aio=threads
>> -device
>> scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on
>> -drive if=none,id=drive-scsi0-0-0-1,readonly=on -device
>> scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1
>> -netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=43 -device
>> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:99:b5:62,bus=pci.0,addr=0x3
>> -chardev
>> socket,id=charserial0,host=127.0.0.1,port=4900,telnet,server,nowait -device
>> isa-serial,chardev=charserial0,id=serial0 -chardev
>> spicevmc,id=charchannel0,name=vdagent -device
>> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
>> -chardev socket,id=charchannel1,fd=45,server,nowait -device
>> virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
>> -chardev spiceport,id=charchannel2,name=org.spice-space.webdav.0
>> -device
>> virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=org.spice-space.webdav.0
>> -device virtio-tablet-pci,id=input2,bus=pci.0,addr=0x7 -spice
>> port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on
>> -device
>> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2
>> -chardev spicevmc,id=charredir0,name=usbredir -device
>> usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev
>> spicevmc,id=charredir1,name=usbredir -device
>> usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox
>> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
>> -msg timestamp=on I can provide a core dump of the process if
>> needed for debugging and the guest XML as well.
>>
>> A backtrace is probably a great starting point (from gdb: `thread
>> apply all bt`.) I don't know exactly what codepath is being exercised
>> when you "attach an ISO file" through libvirt's interface. If you
>> don't mind the hassle, trying on the 4.1 (or a development build would
>> be even more luxurious) and giving a stacktrace would be nice.
>>
>> Thanks. Fernando
>>
>> Thanks! --js
>
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
2019-10-23 17:33 ` John Snow
@ 2019-10-23 17:57 ` Fernando Casas Schössow
[not found] ` <VI1PR03MB48142C8AFFD94EDD5339F6D6A46B0@VI1PR03MB4814.eurprd03.prod.outlook.c om>
[not found] ` <VI1PR03MB48142C8AFFD94EDD5339F6D6A46B0@VI1PR03MB4814.eurprd03.prod.outlook.c>
0 siblings, 2 replies; 11+ messages in thread
From: Fernando Casas Schössow @ 2019-10-23 17:57 UTC (permalink / raw)
To: John Snow; +Cc: Peter Krempa, QEMU Developers, qemu-block
[-- Attachment #1: Type: text/plain, Size: 6394 bytes --]
In virsh I would do this while the guest is running:
virsh attach-disk <vmname> <path_to_ISO_file> <guest_device> --type cdrom --mode readonly
Following the example for guest from my first email:
virsh attach-disk DCHOMENET01 /resources/virtio-win-0.1.171-stable.iso sdb --type cdrom --mode readonly
Right after executing this, qemu crashes and log the assertion.
I can repro this also from virt-manager by selecting the cdrom device -> Connect -> selecting the ISO file -> Choose volume -> Ok (basically the same but in the gui).
I may be able to try 4.1. Will look into it and report back.
On mié, oct 23, 2019 at 7:33 PM, John Snow <jsnow@redhat.com> wrote:
On 10/23/19 1:28 PM, Fernando Casas Schössow wrote:
Hi John, Thanks for looking into this. I can quickly repro the problem with qemu 4.0 binary with debugging symbols enabled as I have it available already. Doing the same with qemu 4.1 or development head may be too much hassle but if it's really the only way I can give it try. Would it worth it to try with 4.0 first and get the strack trace or it will not help and the only way to go is with 4.1 (or dev)? Thanks, Fernando
If 4.0 is what you have access to, having the stack trace for that is better than not, but confirming it happens on the latest release would be nice. Can you share your workflow for virsh that reproduces the failure? --js
On mié, oct 23, 2019 at 5:34 PM, John Snow <jsnow@redhat.com<mailto:jsnow@redhat.com>> wrote:
On 10/18/19 5:41 PM, Fernando Casas Schössow wrote: Hi, Hi! Thanks for the report. Today while working with two different Windows Server 2012 R2 guests I found that when I try to attach an ISO file to a SCSI CD-ROM device through libvirt (virsh or virt-manager) while the guest is running, qemu crashes and the following message is logged: Assertion failed: blk_get_aio_context(d->conf.blk) == s->ctx (/home/buildozer/aports/main/qemu/src/qemu-4.0.0/hw/scsi/virtio-scsi.c: virtio_scsi_ctx_check: 246) I can repro this at will. All I have to do is to try to attach an ISO file to the SCSI CDROM while the guest is running. The SCSI controller model is virtio-scsi with iothread enabled. Please find below all the details about my setup that I considered relevant but I missed something please don't hesitate to let me know: Looks like we got aio_context management wrong with iothread for the media change events somewhere. Should be easy enough to fix if we figure out where the bad assumption is. Host arch: x86_64 Distro: Alpine Linux 3.10.2 qemu version: 4.0 Do you have the ability to try 4.1, or the latest development head with debugging symbols enabled? Linux kernel version: 4.19.67 libvirt: 5.5.0 Emulated SCSI controller: virtio-scsi (with iothread enabled) Guest firmware: OVMF-EFI Guest OS: Window Server 2012 R2 Guest virtio drivers version: 171 (current stable) qemu command line: /usr/bin/qemu-system-x86_64 -name guest=DCHOMENET01,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-78-DCHOMENET01/master-key.aes -machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=on -cpu IvyBridge,ss=on,vmx=off,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,xsaveopt=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff -drive file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/DCHOMENET01_VARS.fd,if=pflash,format=raw,unit=1 -m 1536 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1 -uuid f06978ad-2734-44ab-a518-5dfcf71d625e -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=33,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x4 -device virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=1,bus=pci.0,addr=0x5 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/storage/storage-hdd-vms/virtual_machines_hdd/dchomenet01.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none,discard=unmap,aio=threads -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on -drive if=none,id=drive-scsi0-0-0-1,readonly=on -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1 -netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=43 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:99:b5:62,bus=pci.0,addr=0x3 -chardev socket,id=charserial0,host=127.0.0.1,port=4900,telnet,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -chardev socket,id=charchannel1,fd=45,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spiceport,id=charchannel2,name=org.spice-space.webdav.0 -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=org.spice-space.webdav.0 -device virtio-tablet-pci,id=input2,bus=pci.0,addr=0x7 -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on I can provide a core dump of the process if needed for debugging and the guest XML as well. A backtrace is probably a great starting point (from gdb: `thread apply all bt`.) I don't know exactly what codepath is being exercised when you "attach an ISO file" through libvirt's interface. If you don't mind the hassle, trying on the 4.1 (or a development build would be even more luxurious) and giving a stacktrace would be nice. Thanks. Fernando Thanks! --js
[-- Attachment #2: Type: text/html, Size: 6988 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
[not found] ` <VI1PR03MB48142C8AFFD94EDD5339F6D6A46B0@VI1PR03MB4814.eurprd03.prod.outlook.c om>
@ 2019-10-24 21:07 ` Fernando Casas Schössow
0 siblings, 0 replies; 11+ messages in thread
From: Fernando Casas Schössow @ 2019-10-24 21:07 UTC (permalink / raw)
To: John Snow; +Cc: Peter Krempa, QEMU Developers, qemu-block
Today I updated to qemu 4.0.1 since this was the latest version
available for Alpine and I can confirm that I can repro the issue with
this version as well.
Not sure if relevant but I can also confirm that the problem happens
with Windows Server 2012 R2 but also with Linux guests (it doesn't
matter if the guest use uefi or bios firmware). I made this tests just
to discard things.
Also as discussed I compiled qemu with debug symbols, repro the
problem, collected a core dump and run both through gdb. This is the
result:
(gdb) thread apply all bt
Thread 42 (LWP 33704):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fee02380b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 41 (LWP 33837):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1ad5b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 40 (LWP 33719):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fee02266b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 39 (LWP 33696):
#0 0x00007fee04233171 in syscall () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee02be8b64 in ?? ()
#2 0x0000000000000030 in ?? ()
#3 0x00007fee02be2540 in ?? ()
#4 0x00007fee02be2500 in ?? ()
#5 0x00007fee02be2548 in ?? ()
#6 0x000055d7e4987f28 in rcu_gp_event ()
#7 0x0000000000000000 in ?? ()
Thread 38 (LWP 33839):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1a83b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 37 (LWP 33841):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1737b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 36 (LWP 33863):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8c83b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 35 (LWP 33842):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc170eb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 34 (LWP 33862):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8cacb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 33 (LWP 33843):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc16e5b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 32 (LWP 33861):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8cd5b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 31 (LWP 33844):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc16bcb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 30 (LWP 33858):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8e83b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 29 (LWP 33845):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1693b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 28 (LWP 33857):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8eacb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 27 (LWP 33846):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc166ab64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 26 (LWP 33856):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8ed5b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 25 (LWP 33847):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc142ab64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 24 (LWP 33855):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8efeb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 23 (LWP 33848):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedbd0feb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 22 (LWP 33854):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedbd031b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 21 (LWP 33849):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedbd0d5b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 20 (LWP 33852):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedbd05ab64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 19 (LWP 33850):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedbd0acb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 18 (LWP 33851):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedbd083b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 17 (LWP 33836):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1afeb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 16 (LWP 33835):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1c5ab64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 15 (LWP 33834):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1c83b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 14 (LWP 33833):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1cacb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 13 (LWP 33677):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x000055d7e516656c in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 12 (LWP 33832):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1cd5b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 11 (LWP 33831):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1cfeb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 10 (LWP 33829):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1e67b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 9 (LWP 33828):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1e90b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 8 (LWP 33827):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fee02a95b64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 7 (LWP 33732):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fee0223db64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 6 (LWP 33706):
#0 0x00007fee0423263d in ioctl () from /lib/ld-musl-x86_64.so.1
#1 0x0000000000000001 in ?? ()
#2 0x00007fee00000010 in ?? ()
#3 0x00007fee02351440 in ?? ()
#4 0x00007fee02351400 in ?? ()
#5 0x0000000000000000 in ?? ()
Thread 5 (LWP 33838):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1aacb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 4 (LWP 33860):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8cfeb64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 3 (LWP 33859):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedb8e5ab64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 2 (LWP 33840):
#0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
#2 0x00007fedc1a5ab64 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 1 (LWP 33701):
#0 0x00007fee0421b7a1 in abort () from /lib/ld-musl-x86_64.so.1
#1 0x000055d7e6012b70 in ?? ()
#2 0x0000000000000020 in ?? ()
#3 0x0000000000000000 in ?? ()
(gdb)
I'm not a developer nor skilled with gdb but if you provide me the
debugging commands I can execute them and reply back with the results.
I can also provide the binary and the core dump for analysis if needed.
While waiting for replies I will check if I can upgrade to qemu 4.1.0,
try to repro and provide the results.
Thanks.
Fernando
On mié, oct 23, 2019 at 7:57 PM, Fernando Casas Schössow
<casasfernando@outlook.com> wrote:
> In virsh I would do this while the guest is running:
>
> virsh attach-disk <vmname> <path_to_ISO_file> <guest_device> --type
> cdrom --mode readonly
>
> Following the example for guest from my first email:
>
> virsh attach-disk DCHOMENET01
> /resources/virtio-win-0.1.171-stable.iso sdb --type cdrom --mode
> readonly
>
> Right after executing this, qemu crashes and log the assertion.
> I can repro this also from virt-manager by selecting the cdrom device
> -> Connect -> selecting the ISO file -> Choose volume -> Ok
> (basically the same but in the gui).
>
> I may be able to try 4.1. Will look into it and report back.
>
> On mié, oct 23, 2019 at 7:33 PM, John Snow <jsnow@redhat.com> wrote:
>>
>>
>> On 10/23/19 1:28 PM, Fernando Casas Schössow wrote:
>>> Hi John,
>>>
>>> Thanks for looking into this.
>>> I can quickly repro the problem with qemu 4.0 binary with debugging
>>> symbols enabled as I have it available already.
>>> Doing the same with qemu 4.1 or development head may be too much
>>> hassle
>>> but if it's really the only way I can give it try.
>>>
>>> Would it worth it to try with 4.0 first and get the strack trace
>>> or it
>>> will not help and the only way to go is with 4.1 (or dev)?
>>>
>>> Thanks,
>>>
>>> Fernando
>>>
>>
>> If 4.0 is what you have access to, having the stack trace for that is
>> better than not, but confirming it happens on the latest release
>> would
>> be nice.
>>
>> Can you share your workflow for virsh that reproduces the failure?
>>
>> --js
>>
>>> On mié, oct 23, 2019 at 5:34 PM, John Snow <jsnow@redhat.com>
>>> wrote:
>>>> On 10/18/19 5:41 PM, Fernando Casas Schössow wrote:
>>>>
>>>> Hi,
>>>>
>>>> Hi! Thanks for the report.
>>>>
>>>> Today while working with two different Windows Server 2012 R2
>>>> guests I found that when I try to attach an ISO file to a SCSI
>>>> CD-ROM device through libvirt (virsh or virt-manager) while
>>>> the
>>>> guest is running, qemu crashes and the following message is
>>>> logged: Assertion failed: blk_get_aio_context(d->conf.blk) ==
>>>> s->ctx
>>>>
>>>> (/home/buildozer/aports/main/qemu/src/qemu-4.0.0/hw/scsi/virtio-scsi.c:
>>>> virtio_scsi_ctx_check: 246) I can repro this at will. All I
>>>> have
>>>> to do is to try to attach an ISO file to the SCSI CDROM while
>>>> the
>>>> guest is running. The SCSI controller model is virtio-scsi
>>>> with
>>>> iothread enabled. Please find below all the details about my
>>>> setup
>>>> that I considered relevant but I missed something please don't
>>>> hesitate to let me know:
>>>>
>>>> Looks like we got aio_context management wrong with iothread for
>>>> the
>>>> media change events somewhere. Should be easy enough to fix if we
>>>> figure out where the bad assumption is.
>>>>
>>>> Host arch: x86_64 Distro: Alpine Linux 3.10.2 qemu version:
>>>> 4.0
>>>>
>>>> Do you have the ability to try 4.1, or the latest development head
>>>> with debugging symbols enabled?
>>>>
>>>> Linux kernel version: 4.19.67 libvirt: 5.5.0 Emulated SCSI
>>>> controller: virtio-scsi (with iothread enabled) Guest
>>>> firmware:
>>>> OVMF-EFI Guest OS: Window Server 2012 R2 Guest virtio drivers
>>>> version: 171 (current stable) qemu command line:
>>>> /usr/bin/qemu-system-x86_64 -name
>>>> guest=DCHOMENET01,debug-threads=on -S -object
>>>>
>>>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-78-DCHOMENET01/master-key.aes
>>>> -machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=on
>>>> -cpu
>>>>
>>>> IvyBridge,ss=on,vmx=off,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,xsaveopt=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff
>>>> -drive
>>>>
>>>> file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on
>>>> -drive
>>>>
>>>> file=/var/lib/libvirt/qemu/nvram/DCHOMENET01_VARS.fd,if=pflash,format=raw,unit=1
>>>> -m 1536 -overcommit mem-lock=off -smp
>>>> 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1
>>>> -uuid
>>>> f06978ad-2734-44ab-a518-5dfcf71d625e -no-user-config
>>>> -nodefaults
>>>> -chardev socket,id=charmonitor,fd=33,server,nowait -mon
>>>> chardev=charmonitor,id=monitor,mode=control -rtc
>>>> base=localtime,driftfix=slew -global
>>>> kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global
>>>> PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot
>>>> strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x4 -device
>>>>
>>>> virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=1,bus=pci.0,addr=0x5
>>>> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6
>>>> -drive
>>>>
>>>> file=/storage/storage-hdd-vms/virtual_machines_hdd/dchomenet01.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none,discard=unmap,aio=threads
>>>> -device
>>>>
>>>> scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on
>>>> -drive if=none,id=drive-scsi0-0-0-1,readonly=on -device
>>>>
>>>> scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1
>>>> -netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=43 -device
>>>>
>>>> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:99:b5:62,bus=pci.0,addr=0x3
>>>> -chardev
>>>>
>>>> socket,id=charserial0,host=127.0.0.1,port=4900,telnet,server,nowait
>>>> -device
>>>> isa-serial,chardev=charserial0,id=serial0 -chardev
>>>> spicevmc,id=charchannel0,name=vdagent -device
>>>>
>>>> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
>>>> -chardev socket,id=charchannel1,fd=45,server,nowait -device
>>>>
>>>> virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
>>>> -chardev
>>>> spiceport,id=charchannel2,name=org.spice-space.webdav.0
>>>> -device
>>>>
>>>> virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=org.spice-space.webdav.0
>>>> -device virtio-tablet-pci,id=input2,bus=pci.0,addr=0x7 -spice
>>>>
>>>> port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on
>>>> -device
>>>>
>>>> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2
>>>> -chardev spicevmc,id=charredir0,name=usbredir -device
>>>> usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2
>>>> -chardev
>>>> spicevmc,id=charredir1,name=usbredir -device
>>>> usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3
>>>> -device
>>>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox
>>>>
>>>> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
>>>> -msg timestamp=on I can provide a core dump of the process if
>>>> needed for debugging and the guest XML as well.
>>>>
>>>> A backtrace is probably a great starting point (from gdb: `thread
>>>> apply all bt`.) I don't know exactly what codepath is being
>>>> exercised
>>>> when you "attach an ISO file" through libvirt's interface. If you
>>>> don't mind the hassle, trying on the 4.1 (or a development build
>>>> would
>>>> be even more luxurious) and giving a stacktrace would be nice.
>>>>
>>>> Thanks. Fernando
>>>>
>>>> Thanks! --js
>>>
>>>
>>
>
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
[not found] ` <VI1PR03MB48149BEC6D7D02A08EECDE9BA46A0@VI1PR03MB4814.eurprd03.prod.outlook.c om>
@ 2019-10-24 21:25 ` Fernando Casas Schössow
0 siblings, 0 replies; 11+ messages in thread
From: Fernando Casas Schössow @ 2019-10-24 21:25 UTC (permalink / raw)
To: John Snow; +Cc: Peter Krempa, QEMU Developers, qemu-block
BTW just to be clear, qemu is crashing in this scenario *only* if
iothread is enabled for the guest.
Without iothread enabled the operation is completed without any
problems.
On jue, oct 24, 2019 at 11:07 PM, Fernando Casas Schössow
<casasfernando@outlook.com> wrote:
> Today I updated to qemu 4.0.1 since this was the latest version
> available for Alpine and I can confirm that I can repro the issue
> with this version as well.
> Not sure if relevant but I can also confirm that the problem happens
> with Windows Server 2012 R2 but also with Linux guests (it doesn't
> matter if the guest use uefi or bios firmware). I made this tests
> just to discard things.
>
> Also as discussed I compiled qemu with debug symbols, repro the
> problem, collected a core dump and run both through gdb. This is the
> result:
>
> (gdb) thread apply all bt
>
> Thread 42 (LWP 33704):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fee02380b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 41 (LWP 33837):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1ad5b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 40 (LWP 33719):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fee02266b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 39 (LWP 33696):
> #0 0x00007fee04233171 in syscall () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee02be8b64 in ?? ()
> #2 0x0000000000000030 in ?? ()
> #3 0x00007fee02be2540 in ?? ()
> #4 0x00007fee02be2500 in ?? ()
> #5 0x00007fee02be2548 in ?? ()
> #6 0x000055d7e4987f28 in rcu_gp_event ()
> #7 0x0000000000000000 in ?? ()
>
> Thread 38 (LWP 33839):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1a83b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 37 (LWP 33841):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1737b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 36 (LWP 33863):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8c83b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 35 (LWP 33842):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc170eb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 34 (LWP 33862):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8cacb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 33 (LWP 33843):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc16e5b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 32 (LWP 33861):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8cd5b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 31 (LWP 33844):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc16bcb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 30 (LWP 33858):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8e83b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 29 (LWP 33845):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1693b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 28 (LWP 33857):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8eacb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 27 (LWP 33846):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc166ab64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 26 (LWP 33856):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8ed5b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 25 (LWP 33847):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc142ab64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 24 (LWP 33855):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8efeb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 23 (LWP 33848):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedbd0feb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 22 (LWP 33854):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedbd031b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 21 (LWP 33849):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedbd0d5b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 20 (LWP 33852):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedbd05ab64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 19 (LWP 33850):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedbd0acb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 18 (LWP 33851):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedbd083b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 17 (LWP 33836):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1afeb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 16 (LWP 33835):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1c5ab64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 15 (LWP 33834):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1c83b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 14 (LWP 33833):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1cacb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 13 (LWP 33677):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x000055d7e516656c in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 12 (LWP 33832):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1cd5b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 11 (LWP 33831):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1cfeb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 10 (LWP 33829):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1e67b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 9 (LWP 33828):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1e90b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 8 (LWP 33827):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fee02a95b64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 7 (LWP 33732):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fee0223db64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 6 (LWP 33706):
> #0 0x00007fee0423263d in ioctl () from /lib/ld-musl-x86_64.so.1
> #1 0x0000000000000001 in ?? ()
> #2 0x00007fee00000010 in ?? ()
> #3 0x00007fee02351440 in ?? ()
> #4 0x00007fee02351400 in ?? ()
> #5 0x0000000000000000 in ?? ()
>
> Thread 5 (LWP 33838):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1aacb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 4 (LWP 33860):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8cfeb64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 3 (LWP 33859):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedb8e5ab64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 2 (LWP 33840):
> #0 0x00007fee04252080 in ?? () from /lib/ld-musl-x86_64.so.1
> #1 0x00007fee0424f4b6 in ?? () from /lib/ld-musl-x86_64.so.1
> #2 0x00007fedc1a5ab64 in ?? ()
> #3 0x0000000000000000 in ?? ()
>
> Thread 1 (LWP 33701):
> #0 0x00007fee0421b7a1 in abort () from /lib/ld-musl-x86_64.so.1
> #1 0x000055d7e6012b70 in ?? ()
> #2 0x0000000000000020 in ?? ()
> #3 0x0000000000000000 in ?? ()
> (gdb)
>
>
> I'm not a developer nor skilled with gdb but if you provide me the
> debugging commands I can execute them and reply back with the results.
> I can also provide the binary and the core dump for analysis if
> needed.
>
> While waiting for replies I will check if I can upgrade to qemu
> 4.1.0, try to repro and provide the results.
>
> Thanks.
>
> Fernando
>
> On mié, oct 23, 2019 at 7:57 PM, Fernando Casas Schössow
> <casasfernando@outlook.com> wrote:
>> In virsh I would do this while the guest is running:
>>
>> virsh attach-disk <vmname> <path_to_ISO_file> <guest_device> --type
>> cdrom --mode readonly
>>
>> Following the example for guest from my first email:
>>
>> virsh attach-disk DCHOMENET01
>> /resources/virtio-win-0.1.171-stable.iso sdb --type cdrom --mode
>> readonly
>>
>> Right after executing this, qemu crashes and log the assertion.
>> I can repro this also from virt-manager by selecting the cdrom
>> device -> Connect -> selecting the ISO file -> Choose volume -> Ok
>> (basically the same but in the gui).
>>
>> I may be able to try 4.1. Will look into it and report back.
>>
>> On mié, oct 23, 2019 at 7:33 PM, John Snow <jsnow@redhat.com> wrote:
>>>
>>>
>>> On 10/23/19 1:28 PM, Fernando Casas Schössow wrote:
>>>> Hi John,
>>>>
>>>> Thanks for looking into this.
>>>> I can quickly repro the problem with qemu 4.0 binary with
>>>> debugging
>>>> symbols enabled as I have it available already.
>>>> Doing the same with qemu 4.1 or development head may be too much
>>>> hassle
>>>> but if it's really the only way I can give it try.
>>>>
>>>> Would it worth it to try with 4.0 first and get the strack trace
>>>> or it
>>>> will not help and the only way to go is with 4.1 (or dev)?
>>>>
>>>> Thanks,
>>>>
>>>> Fernando
>>>>
>>>
>>> If 4.0 is what you have access to, having the stack trace for that
>>> is
>>> better than not, but confirming it happens on the latest release
>>> would
>>> be nice.
>>>
>>> Can you share your workflow for virsh that reproduces the failure?
>>>
>>> --js
>>>
>>>> On mié, oct 23, 2019 at 5:34 PM, John Snow <jsnow@redhat.com>
>>>> wrote:
>>>>> On 10/18/19 5:41 PM, Fernando Casas Schössow wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Hi! Thanks for the report.
>>>>>
>>>>> Today while working with two different Windows Server 2012 R2
>>>>> guests I found that when I try to attach an ISO file to a
>>>>> SCSI
>>>>> CD-ROM device through libvirt (virsh or virt-manager) while
>>>>> the
>>>>> guest is running, qemu crashes and the following message is
>>>>> logged: Assertion failed: blk_get_aio_context(d->conf.blk) ==
>>>>> s->ctx
>>>>>
>>>>> (/home/buildozer/aports/main/qemu/src/qemu-4.0.0/hw/scsi/virtio-scsi.c:
>>>>> virtio_scsi_ctx_check: 246) I can repro this at will. All I
>>>>> have
>>>>> to do is to try to attach an ISO file to the SCSI CDROM
>>>>> while the
>>>>> guest is running. The SCSI controller model is virtio-scsi
>>>>> with
>>>>> iothread enabled. Please find below all the details about my
>>>>> setup
>>>>> that I considered relevant but I missed something please
>>>>> don't
>>>>> hesitate to let me know:
>>>>>
>>>>> Looks like we got aio_context management wrong with iothread for
>>>>> the
>>>>> media change events somewhere. Should be easy enough to fix if we
>>>>> figure out where the bad assumption is.
>>>>>
>>>>> Host arch: x86_64 Distro: Alpine Linux 3.10.2 qemu version:
>>>>> 4.0
>>>>>
>>>>> Do you have the ability to try 4.1, or the latest development
>>>>> head
>>>>> with debugging symbols enabled?
>>>>>
>>>>> Linux kernel version: 4.19.67 libvirt: 5.5.0 Emulated SCSI
>>>>> controller: virtio-scsi (with iothread enabled) Guest
>>>>> firmware:
>>>>> OVMF-EFI Guest OS: Window Server 2012 R2 Guest virtio drivers
>>>>> version: 171 (current stable) qemu command line:
>>>>> /usr/bin/qemu-system-x86_64 -name
>>>>> guest=DCHOMENET01,debug-threads=on -S -object
>>>>>
>>>>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-78-DCHOMENET01/master-key.aes
>>>>> -machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=on
>>>>> -cpu
>>>>>
>>>>> IvyBridge,ss=on,vmx=off,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,xsaveopt=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff
>>>>> -drive
>>>>>
>>>>> file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on
>>>>> -drive
>>>>>
>>>>> file=/var/lib/libvirt/qemu/nvram/DCHOMENET01_VARS.fd,if=pflash,format=raw,unit=1
>>>>> -m 1536 -overcommit mem-lock=off -smp
>>>>> 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1
>>>>> -uuid
>>>>> f06978ad-2734-44ab-a518-5dfcf71d625e -no-user-config
>>>>> -nodefaults
>>>>> -chardev socket,id=charmonitor,fd=33,server,nowait -mon
>>>>> chardev=charmonitor,id=monitor,mode=control -rtc
>>>>> base=localtime,driftfix=slew -global
>>>>> kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global
>>>>> PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot
>>>>> strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x4 -device
>>>>>
>>>>> virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=1,bus=pci.0,addr=0x5
>>>>> -device
>>>>> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6
>>>>> -drive
>>>>>
>>>>> file=/storage/storage-hdd-vms/virtual_machines_hdd/dchomenet01.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none,discard=unmap,aio=threads
>>>>> -device
>>>>>
>>>>> scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on
>>>>> -drive if=none,id=drive-scsi0-0-0-1,readonly=on -device
>>>>>
>>>>> scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1
>>>>> -netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=43 -device
>>>>>
>>>>> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:99:b5:62,bus=pci.0,addr=0x3
>>>>> -chardev
>>>>>
>>>>> socket,id=charserial0,host=127.0.0.1,port=4900,telnet,server,nowait
>>>>> -device
>>>>> isa-serial,chardev=charserial0,id=serial0 -chardev
>>>>> spicevmc,id=charchannel0,name=vdagent -device
>>>>>
>>>>> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
>>>>> -chardev socket,id=charchannel1,fd=45,server,nowait -device
>>>>>
>>>>> virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
>>>>> -chardev
>>>>> spiceport,id=charchannel2,name=org.spice-space.webdav.0
>>>>> -device
>>>>>
>>>>> virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=org.spice-space.webdav.0
>>>>> -device virtio-tablet-pci,id=input2,bus=pci.0,addr=0x7 -spice
>>>>>
>>>>> port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on
>>>>> -device
>>>>>
>>>>> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2
>>>>> -chardev spicevmc,id=charredir0,name=usbredir -device
>>>>> usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2
>>>>> -chardev
>>>>> spicevmc,id=charredir1,name=usbredir -device
>>>>> usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3
>>>>> -device
>>>>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox
>>>>>
>>>>> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
>>>>> -msg timestamp=on I can provide a core dump of the process if
>>>>> needed for debugging and the guest XML as well.
>>>>>
>>>>> A backtrace is probably a great starting point (from gdb: `thread
>>>>> apply all bt`.) I don't know exactly what codepath is being
>>>>> exercised
>>>>> when you "attach an ISO file" through libvirt's interface. If you
>>>>> don't mind the hassle, trying on the 4.1 (or a development build
>>>>> would
>>>>> be even more luxurious) and giving a stacktrace would be nice.
>>>>>
>>>>> Thanks. Fernando
>>>>>
>>>>> Thanks! --js
>>>>
>>>>
>>>
>>
>>
>
>
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
2019-10-23 17:28 ` Fernando Casas Schössow
2019-10-23 17:33 ` John Snow
@ 2019-10-25 10:07 ` Kevin Wolf
2019-10-25 10:28 ` Fernando Casas Schössow
1 sibling, 1 reply; 11+ messages in thread
From: Kevin Wolf @ 2019-10-25 10:07 UTC (permalink / raw)
To: Fernando Casas Schössow; +Cc: John Snow, QEMU Developers, qemu-block
Am 23.10.2019 um 19:28 hat Fernando Casas Schössow geschrieben:
> Hi John,
>
> Thanks for looking into this. I can quickly repro the problem with
> qemu 4.0 binary with debugging symbols enabled as I have it available
> already. Doing the same with qemu 4.1 or development head may be too
> much hassle but if it's really the only way I can give it try.
We had a lot of iothread related fixes in 4.1, so this would really be
the only way to tell if it's a bug that still exists. I suspect that
it's already fixed (and to be more precise, I assume that commit
d0ee0204f fixed it).
Kevin
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
2019-10-25 10:07 ` Kevin Wolf
@ 2019-10-25 10:28 ` Fernando Casas Schössow
[not found] ` <VI1PR03MB48140DB2BA084EAEBD980302A4650@VI1PR03MB4814.eurprd03.prod.outlook.c om>
0 siblings, 1 reply; 11+ messages in thread
From: Fernando Casas Schössow @ 2019-10-25 10:28 UTC (permalink / raw)
To: Kevin Wolf; +Cc: John Snow, QEMU Developers, qemu-block
[-- Attachment #1: Type: text/plain, Size: 822 bytes --]
Thanks for the reply Kevin.
I will do my best to upgrade to 4.1, test again and report back if this is fixed or not in that version.
Hopefully it is.
Fernando
On vie, oct 25, 2019 at 12:07 PM, Kevin Wolf <kwolf@redhat.com> wrote:
Am 23.10.2019 um 19:28 hat Fernando Casas Schössow geschrieben:
Hi John, Thanks for looking into this. I can quickly repro the problem with qemu 4.0 binary with debugging symbols enabled as I have it available already. Doing the same with qemu 4.1 or development head may be too much hassle but if it's really the only way I can give it try.
We had a lot of iothread related fixes in 4.1, so this would really be the only way to tell if it's a bug that still exists. I suspect that it's already fixed (and to be more precise, I assume that commit d0ee0204f fixed it). Kevin
[-- Attachment #2: Type: text/html, Size: 1208 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
[not found] ` <VI1PR03MB48140DB2BA084EAEBD980302A4650@VI1PR03MB4814.eurprd03.prod.outlook.c om>
@ 2019-10-25 10:45 ` Fernando Casas Schössow
0 siblings, 0 replies; 11+ messages in thread
From: Fernando Casas Schössow @ 2019-10-25 10:45 UTC (permalink / raw)
To: Kevin Wolf; +Cc: John Snow, QEMU Developers, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1154 bytes --]
I managed to upgrade to qemu 4.1 on a test KVM host and I can confirm I can't repro the issue in this version.
Great news that is fixed in 4.1.
Thanks everyone for your inputs and the fast replies.
Kind regards,
Fernando
On vie, oct 25, 2019 at 12:28 PM, Fernando Casas Schössow <casasfernando@outlook.com> wrote:
Thanks for the reply Kevin.
I will do my best to upgrade to 4.1, test again and report back if this is fixed or not in that version.
Hopefully it is.
Fernando
On vie, oct 25, 2019 at 12:07 PM, Kevin Wolf <kwolf@redhat.com> wrote:
Am 23.10.2019 um 19:28 hat Fernando Casas Schössow geschrieben:
Hi John, Thanks for looking into this. I can quickly repro the problem with qemu 4.0 binary with debugging symbols enabled as I have it available already. Doing the same with qemu 4.1 or development head may be too much hassle but if it's really the only way I can give it try.
We had a lot of iothread related fixes in 4.1, so this would really be the only way to tell if it's a bug that still exists. I suspect that it's already fixed (and to be more precise, I assume that commit d0ee0204f fixed it). Kevin
[-- Attachment #2: Type: text/html, Size: 1715 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt
@ 2019-10-18 21:26 Fernando Casas Schössow
0 siblings, 0 replies; 11+ messages in thread
From: Fernando Casas Schössow @ 2019-10-18 21:26 UTC (permalink / raw)
To: QEMU Developers, qemu-block
[-- Attachment #1: Type: text/plain, Size: 4251 bytes --]
Hi,
Today while working with two different Windows Server 2012 R2 guests I found that when I try to attach an ISO file to a SCSI CD-ROM device through libvirt (virsh or virt-manager) while the guest is running, qemu crashes and the following message is logged:
Assertion failed: blk_get_aio_context(d->conf.blk) == s->ctx (/home/buildozer/aports/main/qemu/src/qemu-4.0.0/hw/scsi/virtio-scsi.c: virtio_scsi_ctx_check: 246)
I can repro this at will. All I have to do is to try to attach an ISO file to the SCSI CDROM while the guest is running.
The SCSI controller model is virtio-scsi with iothread enabled.
Please find below all the details about my setup that I considered relevant but I missed something please don't hesitate to let me know:
Host arch: x86_64
Distro: Alpine Linux 3.10.2
qemu version: 4.0
Linux kernel version: 4.19.67
libvirt: 5.5.0
Emulated SCSI controller: virtio-scsi (with iothread enabled)
Guest firmware: OVMF-EFI
Guest OS: Window Server 2012 R2
Guest virtio drivers version: 171 (current stable)
qemu command line:
/usr/bin/qemu-system-x86_64 -name guest=DCHOMENET01,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-78-DCHOMENET01/master-key.aes -machine pc-i440fx-4.0,accel=kvm,usb=off,dump-guest-core=on -cpu IvyBridge,ss=on,vmx=off,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,xsaveopt=on,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff -drive file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/DCHOMENET01_VARS.fd,if=pflash,format=raw,unit=1 -m 1536 -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -object iothread,id=iothread1 -uuid f06978ad-2734-44ab-a518-5dfcf71d625e -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=33,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device qemu-xhci,id=usb,bus=pci.0,addr=0x4 -device virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=1,bus=pci.0,addr=0x5 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/storage/storage-hdd-vms/virtual_machines_hdd/dchomenet01.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0,cache=none,discard=unmap,aio=threads -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,device_id=drive-scsi0-0-0-0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1,write-cache=on -drive if=none,id=drive-scsi0-0-0-1,readonly=on -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,device_id=drive-scsi0-0-0-1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1 -netdev tap,fd=41,id=hostnet0,vhost=on,vhostfd=43 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:99:b5:62,bus=pci.0,addr=0x3 -chardev socket,id=charserial0,host=127.0.0.1,port=4900,telnet,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -chardev socket,id=charchannel1,fd=45,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spiceport,id=charchannel2,name=org.spice-space.webdav.0 -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=org.spice-space.webdav.0 -device virtio-tablet-pci,id=input2,bus=pci.0,addr=0x7 -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pci.0,addr=0x2 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
I can provide a core dump of the process if needed for debugging and the guest XML as well.
Thanks.
Fernando
[-- Attachment #2: Type: text/html, Size: 4804 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2019-10-25 11:04 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-18 21:41 qemu crashing when attaching an ISO file to a virtio-scsi CD-ROM device through libvirt Fernando Casas Schössow
2019-10-23 15:34 ` John Snow
2019-10-23 17:28 ` Fernando Casas Schössow
2019-10-23 17:33 ` John Snow
2019-10-23 17:57 ` Fernando Casas Schössow
[not found] ` <VI1PR03MB48142C8AFFD94EDD5339F6D6A46B0@VI1PR03MB4814.eurprd03.prod.outlook.c om>
2019-10-24 21:07 ` Fernando Casas Schössow
[not found] ` <VI1PR03MB48142C8AFFD94EDD5339F6D6A46B0@VI1PR03MB4814.eurprd03.prod.outlook.c>
[not found] ` <VI1PR03MB48149BEC6D7D02A08EECDE9BA46A0@VI1PR03MB4814.eurprd03.prod.outlook.c om>
2019-10-24 21:25 ` Fernando Casas Schössow
2019-10-25 10:07 ` Kevin Wolf
2019-10-25 10:28 ` Fernando Casas Schössow
[not found] ` <VI1PR03MB48140DB2BA084EAEBD980302A4650@VI1PR03MB4814.eurprd03.prod.outlook.c om>
2019-10-25 10:45 ` Fernando Casas Schössow
-- strict thread matches above, loose matches on Subject: below --
2019-10-18 21:26 Fernando Casas Schössow
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.