* [PATCH] virtio-pci: fix up config interrupt handling @ 2022-01-09 17:49 Michael S. Tsirkin 2022-01-10 7:20 ` Cédric Le Goater 2022-01-10 20:37 ` Marc Zyngier 0 siblings, 2 replies; 4+ messages in thread From: Michael S. Tsirkin @ 2022-01-09 17:49 UTC (permalink / raw) To: qemu-devel; +Cc: Volker Rümelin, Cédric Le Goater, Cindy Lu Fixes a couple of issues with irqfd use by config interrupt: - Rearrange initialization so cleanup happens in the reverse order - Don't use irqfd for config when not in use for data path I am not sure this is a complete fix though: I think we are better off limiting the effect to vdpa devices with config interrupt support. Or even bypass irqfd for config completely and inject into KVM using ioctl? The advantage would be less FDs used. This would mean mostly reverting the patchset though. Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt") Cc: "Cindy Lu" <lulu@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> --- hw/virtio/virtio-pci.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 98fb5493ae..b77cd69f97 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -1130,15 +1130,15 @@ static int virtio_pci_set_guest_notifiers(DeviceState *d, int nvqs, bool assign) proxy->vector_irqfd = g_malloc0(sizeof(*proxy->vector_irqfd) * msix_nr_vectors_allocated(&proxy->pci_dev)); + r = kvm_virtio_pci_vector_config_use(proxy); + if (r < 0) { + goto config_error; + } r = kvm_virtio_pci_vector_use(proxy, nvqs); if (r < 0) { goto config_assign_error; } } - r = kvm_virtio_pci_vector_config_use(proxy); - if (r < 0) { - goto config_error; - } r = msix_set_vector_notifiers(&proxy->pci_dev, virtio_pci_vector_unmask, virtio_pci_vector_mask, virtio_pci_vector_poll); @@ -1155,7 +1155,9 @@ notifiers_error: kvm_virtio_pci_vector_release(proxy, nvqs); } config_error: - kvm_virtio_pci_vector_config_release(proxy); + if (with_irqfd) { + kvm_virtio_pci_vector_config_release(proxy); + } config_assign_error: virtio_pci_set_guest_notifier(d, VIRTIO_CONFIG_IRQ_IDX, !assign, with_irqfd); -- MST ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] virtio-pci: fix up config interrupt handling 2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin @ 2022-01-10 7:20 ` Cédric Le Goater 2022-01-10 20:37 ` Marc Zyngier 1 sibling, 0 replies; 4+ messages in thread From: Cédric Le Goater @ 2022-01-10 7:20 UTC (permalink / raw) To: Michael S. Tsirkin, qemu-devel; +Cc: Volker Rümelin, Cindy Lu On 1/9/22 18:49, Michael S. Tsirkin wrote: > Fixes a couple of issues with irqfd use by config interrupt: > - Rearrange initialization so cleanup happens in the reverse order > - Don't use irqfd for config when not in use for data path > I am not sure this is a complete fix though: I think we > are better off limiting the effect to vdpa devices > with config interrupt support. Or even bypass irqfd > for config completely and inject into KVM using ioctl? > The advantage would be less FDs used. > This would mean mostly reverting the patchset though. > > Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt") > Cc: "Cindy Lu" <lulu@redhat.com> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> On a KVM guest with vhost, I am still seeing at reboot an issue :/ ../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier' C. > --- > hw/virtio/virtio-pci.c | 12 +++++++----- > 1 file changed, 7 insertions(+), 5 deletions(-) > > diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c > index 98fb5493ae..b77cd69f97 100644 > --- a/hw/virtio/virtio-pci.c > +++ b/hw/virtio/virtio-pci.c > @@ -1130,15 +1130,15 @@ static int virtio_pci_set_guest_notifiers(DeviceState *d, int nvqs, bool assign) > proxy->vector_irqfd = > g_malloc0(sizeof(*proxy->vector_irqfd) * > msix_nr_vectors_allocated(&proxy->pci_dev)); > + r = kvm_virtio_pci_vector_config_use(proxy); > + if (r < 0) { > + goto config_error; > + } > r = kvm_virtio_pci_vector_use(proxy, nvqs); > if (r < 0) { > goto config_assign_error; > } > } > - r = kvm_virtio_pci_vector_config_use(proxy); > - if (r < 0) { > - goto config_error; > - } > r = msix_set_vector_notifiers(&proxy->pci_dev, virtio_pci_vector_unmask, > virtio_pci_vector_mask, > virtio_pci_vector_poll); > @@ -1155,7 +1155,9 @@ notifiers_error: > kvm_virtio_pci_vector_release(proxy, nvqs); > } > config_error: > - kvm_virtio_pci_vector_config_release(proxy); > + if (with_irqfd) { > + kvm_virtio_pci_vector_config_release(proxy); > + } > config_assign_error: > virtio_pci_set_guest_notifier(d, VIRTIO_CONFIG_IRQ_IDX, !assign, > with_irqfd); > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] virtio-pci: fix up config interrupt handling 2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin 2022-01-10 7:20 ` Cédric Le Goater @ 2022-01-10 20:37 ` Marc Zyngier 2022-01-10 21:09 ` Michael S. Tsirkin 1 sibling, 1 reply; 4+ messages in thread From: Marc Zyngier @ 2022-01-10 20:37 UTC (permalink / raw) To: Michael S. Tsirkin Cc: Volker Rümelin, qemu-devel, Cindy Lu, Cédric Le Goater Hi Michael, On Sun, 09 Jan 2022 17:49:19 +0000, "Michael S. Tsirkin" <mst@redhat.com> wrote: > > Fixes a couple of issues with irqfd use by config interrupt: > - Rearrange initialization so cleanup happens in the reverse order > - Don't use irqfd for config when not in use for data path > I am not sure this is a complete fix though: I think we > are better off limiting the effect to vdpa devices > with config interrupt support. Or even bypass irqfd > for config completely and inject into KVM using ioctl? > The advantage would be less FDs used. > This would mean mostly reverting the patchset though. > > Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt") > Cc: "Cindy Lu" <lulu@redhat.com> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> This doesn't seem to fix the problems I'm seeing here with a KVM/arm64 guest (the issue exists with and without this patch) On initial boot: <quote> Loading Linux 5.7.0-1-arm64 ... Loading initial ramdisk ... qemu-system-aarch64: virtio-blk failed to set guest notifier (-16), ensure -accel kvm is set. qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower). qemu-system-aarch64: virtio-scsi: Failed to set guest notifiers (-16), ensure -accel kvm is set. qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower). </quote> The guest is functional though. However, on reboot: <quote> Loading Linux 5.7.0-1-arm64 ... Loading initial ramdisk ... qemu-system-aarch64: ../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier' failed. </quote> Reverting d5d24d859c fixes the issue. For the record, my qemu command line: /home/maz/qemu/build/qemu-system-aarch64 -m 1G -smp 8 -cpu host,aarch64=on -machine virt,accel=kvm,gic-version=host -nographic -drive if=pflash,format=raw,readonly=on,file=/usr/share/AAVMF/AAVMF_CODE.fd -drive if=pflash,format=raw,file=bullseye/bBwcgtDY2UwXklV6.fd -netdev user,id=hostnet0 -device virtio-net-pci,netdev=hostnet0 -drive if=none,format=raw,cache=none,aio=native,file=bullseye/bBwcgtDY2UwXklV6.img,id=disk0 -device virtio-blk-pci,drive=disk0 -drive file=debian-testing-arm64-netinst-preseed.iso,id=cdrom,if=none,media=cdrom -device virtio-scsi-pci -device scsi-cd,drive=cdrom Thanks, M. -- Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] virtio-pci: fix up config interrupt handling 2022-01-10 20:37 ` Marc Zyngier @ 2022-01-10 21:09 ` Michael S. Tsirkin 0 siblings, 0 replies; 4+ messages in thread From: Michael S. Tsirkin @ 2022-01-10 21:09 UTC (permalink / raw) To: Marc Zyngier Cc: Volker Rümelin, qemu-devel, Cindy Lu, Cédric Le Goater On Mon, Jan 10, 2022 at 08:37:43PM +0000, Marc Zyngier wrote: > Hi Michael, > > On Sun, 09 Jan 2022 17:49:19 +0000, > "Michael S. Tsirkin" <mst@redhat.com> wrote: > > > > Fixes a couple of issues with irqfd use by config interrupt: > > - Rearrange initialization so cleanup happens in the reverse order > > - Don't use irqfd for config when not in use for data path > > I am not sure this is a complete fix though: I think we > > are better off limiting the effect to vdpa devices > > with config interrupt support. Or even bypass irqfd > > for config completely and inject into KVM using ioctl? > > The advantage would be less FDs used. > > This would mean mostly reverting the patchset though. > > > > Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt") > > Cc: "Cindy Lu" <lulu@redhat.com> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com> > > This doesn't seem to fix the problems I'm seeing here with a KVM/arm64 > guest (the issue exists with and without this patch) > > On initial boot: > > <quote> > Loading Linux 5.7.0-1-arm64 ... > Loading initial ramdisk ... > qemu-system-aarch64: virtio-blk failed to set guest notifier (-16), ensure -accel kvm is set. > qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower). > qemu-system-aarch64: virtio-scsi: Failed to set guest notifiers (-16), ensure -accel kvm is set. > qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower). > </quote> > > The guest is functional though. However, on reboot: > > <quote> > Loading Linux 5.7.0-1-arm64 ... > Loading initial ramdisk ... > qemu-system-aarch64: ../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier' failed. > </quote> > > Reverting d5d24d859c fixes the issue. For the record, my qemu command > line: > > /home/maz/qemu/build/qemu-system-aarch64 -m 1G -smp 8 -cpu host,aarch64=on -machine virt,accel=kvm,gic-version=host -nographic -drive if=pflash,format=raw,readonly=on,file=/usr/share/AAVMF/AAVMF_CODE.fd -drive if=pflash,format=raw,file=bullseye/bBwcgtDY2UwXklV6.fd -netdev user,id=hostnet0 -device virtio-net-pci,netdev=hostnet0 -drive if=none,format=raw,cache=none,aio=native,file=bullseye/bBwcgtDY2UwXklV6.img,id=disk0 -device virtio-blk-pci,drive=disk0 -drive file=debian-testing-arm64-netinst-preseed.iso,id=cdrom,if=none,media=cdrom -device virtio-scsi-pci -device scsi-cd,drive=cdrom > > Thanks, > > M. Thanks, I'm reverting these changes for now. Cindy, once you have your patches ready pinging people who encountered problems with the previous versions is a very good idea. > -- > Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-01-10 21:19 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin 2022-01-10 7:20 ` Cédric Le Goater 2022-01-10 20:37 ` Marc Zyngier 2022-01-10 21:09 ` Michael S. Tsirkin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.