[Bug 1890290] Re: PowerPC L2(nested virt) kvm guest fails to boot with ic-mode=dual, kernel-irqchip=on - `KVM is too old to support ic-mode=dual, kernel-irqchip=on`

[Greg Kurz <1890290@bugs.launchpad.net>]

This is currently expected because the L2 KVM guest uses the historical
KVM XICS device (not the XICS-on-XIVE one) and this can be only created
once during the VM lifetime for the moment.

This is a limitation in KVM, that can be addressed in several ways:
1) change the historical KVM XICS device to implement the release() method instead of destroy(), so that userspace can close() and re-create the device multiple times during the VM lifetime, as we have already done in KVM XIVE and KVM XICS-on-XIVE for powernv
2) have the KVM XICS-on-XIVE device to work under pseries

I already have a tentative patch for 1) and I guess 2) would be part of
a more global work to supporting nested KVM XIVE.

But it is definitely not an issue in QEMU.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1890290

Title:
  PowerPC L2(nested virt) kvm guest fails to boot with ic-mode=dual
  ,kernel-irqchip=on - `KVM is too old to support ic-mode=dual,kernel-
  irqchip=on`

Status in QEMU:
  New

Bug description:
  Env:
  HW: Power 9 DD2.3
  Host L0: 5.8.0-rc5-g8ba4ffcd8
  Qemu: 5.0.50 (v5.0.0-533-gdebe78ce14)
  Libvirt: 6.4.0
  L1: 5.8.0-rc5-ge9919e11e
  qemu_version': '5.0.50 (v5.1.0-rc2-dirty)
  libvirt_version': '6.4.0'
  L2: 5.8.0-rc7-g6ba1b005f

  
  1. boot a L2 KVM guest with `ic-mode=dual,kernel-irqchip=on`

  /usr/bin/virt-install --connect=qemu:///system --hvm --accelerate --name 'vm1' --machine pseries --memory=8192 --vcpu=8,maxvcpus=8,sockets=1,cores=2,t
  hreads=4 --import --nographics --serial pty --memballoon model=virtio --disk path=/home/tests/data/avocado-vt/images/f31-ppc64le.qcow2,bus=virtio,size=10,format=qcow2 --network
  =bridge=virbr0,model=virtio,mac=52:54:00:e6:fe:f6 --mac=52:54:00:e6:fe:f6 --boot emulator=/usr/share/avocado-plugins-vt/bin/qemu,kernel=/tmp/linux/vmlinux,kernel_args="root=/de
  v/vda2 rw console=tty0 console=ttyS0,115200 init=/sbin/init initcall_debug selinux=0" --noautoconsole --qemu-commandline=" -M pseries,ic-mode=dual,kernel-irqchip=on"

  
  ERROR    internal error: process exited while connecting to monitor: 2020-08-04T11:12:53.304482Z qemu: KVM is too old to support ic-mode=dual,kernel-irqchip=on


  
  Qemu Log:
  ```
  /usr/share/avocado-plugins-vt/bin/qemu \
  -name guest=vm1,debug-threads=on \
  -S \
  -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-5-vm1/master-key.aes \
  -machine pseries-5.1,accel=kvm,usb=off,dump-guest-core=off \
  -cpu POWER9 \
  -m 8192 \
  -overcommit mem-lock=off \
  -smp 8,sockets=1,dies=1,cores=2,threads=4 \
  -uuid 20a3351b-2776-4e75-9059-c070fe3dd44b \
  -display none \
  -no-user-config \
  -nodefaults \
  -chardev socket,id=charmonitor,fd=34,server,nowait \
  -mon chardev=charmonitor,id=monitor,mode=control \
  -rtc base=utc \
  -no-shutdown \
  -boot strict=on \
  -kernel /tmp/linux/vmlinux \
  -append 'root=/dev/vda2 rw console=tty0 console=ttyS0,115200 init=/sbin/init initcall_debug selinux=0' \
  -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.0,addr=0x2 \
  -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
  -blockdev '{"driver":"file","filename":"/home/tests/data/avocado-vt/images/f31-ppc64le.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
  -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}' \
  -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=libvirt-1-format,id=virtio-disk0,bootindex=1 \
  -netdev tap,fd=37,id=hostnet0,vhost=on,vhostfd=38 \
  -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:e6:fe:f6,bus=pci.0,addr=0x1 \
  -chardev pty,id=charserial0 \
  -device spapr-vty,chardev=charserial0,id=serial0,reg=0x30000000 \
  -chardev socket,id=charchannel0,fd=39,server,nowait \
  -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 \
  -M pseries,ic-mode=dual,kernel-irqchip=on \
  -msg timestamp=on
  2020-08-04 11:12:53.169+0000: Domain id=5 is tainted: custom-argv
  2020-08-04 11:12:53.179+0000: 11120: info : libvirt version: 6.4.0, package: 1.fc31 (Unknown, 2020-06-02-05:09:40, ltc-wspoon4.aus.stglabs.ibm.com)
  2020-08-04 11:12:53.179+0000: 11120: info : hostname: atest-guest
  2020-08-04 11:12:53.179+0000: 11120: info : virObjectUnref:347 : OBJECT_UNREF: obj=0x7fff0c117c40
  char device redirected to /dev/pts/0 (label charserial0)
  2020-08-04T11:12:53.304482Z qemu: KVM is too old to support ic-mode=dual,kernel-irqchip=on
  2020-08-04 11:12:53.694+0000: shutting down, reason=failed
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1890290/+subscriptions