* [PATCH] virtio-pci: fix up config interrupt handling
@ 2022-01-09 17:49 Michael S. Tsirkin
2022-01-10 7:20 ` Cédric Le Goater
2022-01-10 20:37 ` Marc Zyngier
0 siblings, 2 replies; 4+ messages in thread
From: Michael S. Tsirkin @ 2022-01-09 17:49 UTC (permalink / raw)
To: qemu-devel; +Cc: Volker Rümelin, Cédric Le Goater, Cindy Lu
Fixes a couple of issues with irqfd use by config interrupt:
- Rearrange initialization so cleanup happens in the reverse order
- Don't use irqfd for config when not in use for data path
I am not sure this is a complete fix though: I think we
are better off limiting the effect to vdpa devices
with config interrupt support. Or even bypass irqfd
for config completely and inject into KVM using ioctl?
The advantage would be less FDs used.
This would mean mostly reverting the patchset though.
Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt")
Cc: "Cindy Lu" <lulu@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
hw/virtio/virtio-pci.c | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 98fb5493ae..b77cd69f97 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1130,15 +1130,15 @@ static int virtio_pci_set_guest_notifiers(DeviceState *d, int nvqs, bool assign)
proxy->vector_irqfd =
g_malloc0(sizeof(*proxy->vector_irqfd) *
msix_nr_vectors_allocated(&proxy->pci_dev));
+ r = kvm_virtio_pci_vector_config_use(proxy);
+ if (r < 0) {
+ goto config_error;
+ }
r = kvm_virtio_pci_vector_use(proxy, nvqs);
if (r < 0) {
goto config_assign_error;
}
}
- r = kvm_virtio_pci_vector_config_use(proxy);
- if (r < 0) {
- goto config_error;
- }
r = msix_set_vector_notifiers(&proxy->pci_dev, virtio_pci_vector_unmask,
virtio_pci_vector_mask,
virtio_pci_vector_poll);
@@ -1155,7 +1155,9 @@ notifiers_error:
kvm_virtio_pci_vector_release(proxy, nvqs);
}
config_error:
- kvm_virtio_pci_vector_config_release(proxy);
+ if (with_irqfd) {
+ kvm_virtio_pci_vector_config_release(proxy);
+ }
config_assign_error:
virtio_pci_set_guest_notifier(d, VIRTIO_CONFIG_IRQ_IDX, !assign,
with_irqfd);
--
MST
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] virtio-pci: fix up config interrupt handling
2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin
@ 2022-01-10 7:20 ` Cédric Le Goater
2022-01-10 20:37 ` Marc Zyngier
1 sibling, 0 replies; 4+ messages in thread
From: Cédric Le Goater @ 2022-01-10 7:20 UTC (permalink / raw)
To: Michael S. Tsirkin, qemu-devel; +Cc: Volker Rümelin, Cindy Lu
On 1/9/22 18:49, Michael S. Tsirkin wrote:
> Fixes a couple of issues with irqfd use by config interrupt:
> - Rearrange initialization so cleanup happens in the reverse order
> - Don't use irqfd for config when not in use for data path
> I am not sure this is a complete fix though: I think we
> are better off limiting the effect to vdpa devices
> with config interrupt support. Or even bypass irqfd
> for config completely and inject into KVM using ioctl?
> The advantage would be less FDs used.
> This would mean mostly reverting the patchset though.
>
> Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt")
> Cc: "Cindy Lu" <lulu@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
On a KVM guest with vhost, I am still seeing at reboot an issue :/
../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier'
C.
> ---
> hw/virtio/virtio-pci.c | 12 +++++++-----
> 1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> index 98fb5493ae..b77cd69f97 100644
> --- a/hw/virtio/virtio-pci.c
> +++ b/hw/virtio/virtio-pci.c
> @@ -1130,15 +1130,15 @@ static int virtio_pci_set_guest_notifiers(DeviceState *d, int nvqs, bool assign)
> proxy->vector_irqfd =
> g_malloc0(sizeof(*proxy->vector_irqfd) *
> msix_nr_vectors_allocated(&proxy->pci_dev));
> + r = kvm_virtio_pci_vector_config_use(proxy);
> + if (r < 0) {
> + goto config_error;
> + }
> r = kvm_virtio_pci_vector_use(proxy, nvqs);
> if (r < 0) {
> goto config_assign_error;
> }
> }
> - r = kvm_virtio_pci_vector_config_use(proxy);
> - if (r < 0) {
> - goto config_error;
> - }
> r = msix_set_vector_notifiers(&proxy->pci_dev, virtio_pci_vector_unmask,
> virtio_pci_vector_mask,
> virtio_pci_vector_poll);
> @@ -1155,7 +1155,9 @@ notifiers_error:
> kvm_virtio_pci_vector_release(proxy, nvqs);
> }
> config_error:
> - kvm_virtio_pci_vector_config_release(proxy);
> + if (with_irqfd) {
> + kvm_virtio_pci_vector_config_release(proxy);
> + }
> config_assign_error:
> virtio_pci_set_guest_notifier(d, VIRTIO_CONFIG_IRQ_IDX, !assign,
> with_irqfd);
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] virtio-pci: fix up config interrupt handling
2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin
2022-01-10 7:20 ` Cédric Le Goater
@ 2022-01-10 20:37 ` Marc Zyngier
2022-01-10 21:09 ` Michael S. Tsirkin
1 sibling, 1 reply; 4+ messages in thread
From: Marc Zyngier @ 2022-01-10 20:37 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Volker Rümelin, qemu-devel, Cindy Lu, Cédric Le Goater
Hi Michael,
On Sun, 09 Jan 2022 17:49:19 +0000,
"Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> Fixes a couple of issues with irqfd use by config interrupt:
> - Rearrange initialization so cleanup happens in the reverse order
> - Don't use irqfd for config when not in use for data path
> I am not sure this is a complete fix though: I think we
> are better off limiting the effect to vdpa devices
> with config interrupt support. Or even bypass irqfd
> for config completely and inject into KVM using ioctl?
> The advantage would be less FDs used.
> This would mean mostly reverting the patchset though.
>
> Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt")
> Cc: "Cindy Lu" <lulu@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
This doesn't seem to fix the problems I'm seeing here with a KVM/arm64
guest (the issue exists with and without this patch)
On initial boot:
<quote>
Loading Linux 5.7.0-1-arm64 ...
Loading initial ramdisk ...
qemu-system-aarch64: virtio-blk failed to set guest notifier (-16), ensure -accel kvm is set.
qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
qemu-system-aarch64: virtio-scsi: Failed to set guest notifiers (-16), ensure -accel kvm is set.
qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
</quote>
The guest is functional though. However, on reboot:
<quote>
Loading Linux 5.7.0-1-arm64 ...
Loading initial ramdisk ...
qemu-system-aarch64: ../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier' failed.
</quote>
Reverting d5d24d859c fixes the issue. For the record, my qemu command
line:
/home/maz/qemu/build/qemu-system-aarch64 -m 1G -smp 8 -cpu host,aarch64=on -machine virt,accel=kvm,gic-version=host -nographic -drive if=pflash,format=raw,readonly=on,file=/usr/share/AAVMF/AAVMF_CODE.fd -drive if=pflash,format=raw,file=bullseye/bBwcgtDY2UwXklV6.fd -netdev user,id=hostnet0 -device virtio-net-pci,netdev=hostnet0 -drive if=none,format=raw,cache=none,aio=native,file=bullseye/bBwcgtDY2UwXklV6.img,id=disk0 -device virtio-blk-pci,drive=disk0 -drive file=debian-testing-arm64-netinst-preseed.iso,id=cdrom,if=none,media=cdrom -device virtio-scsi-pci -device scsi-cd,drive=cdrom
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] virtio-pci: fix up config interrupt handling
2022-01-10 20:37 ` Marc Zyngier
@ 2022-01-10 21:09 ` Michael S. Tsirkin
0 siblings, 0 replies; 4+ messages in thread
From: Michael S. Tsirkin @ 2022-01-10 21:09 UTC (permalink / raw)
To: Marc Zyngier
Cc: Volker Rümelin, qemu-devel, Cindy Lu, Cédric Le Goater
On Mon, Jan 10, 2022 at 08:37:43PM +0000, Marc Zyngier wrote:
> Hi Michael,
>
> On Sun, 09 Jan 2022 17:49:19 +0000,
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> >
> > Fixes a couple of issues with irqfd use by config interrupt:
> > - Rearrange initialization so cleanup happens in the reverse order
> > - Don't use irqfd for config when not in use for data path
> > I am not sure this is a complete fix though: I think we
> > are better off limiting the effect to vdpa devices
> > with config interrupt support. Or even bypass irqfd
> > for config completely and inject into KVM using ioctl?
> > The advantage would be less FDs used.
> > This would mean mostly reverting the patchset though.
> >
> > Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt")
> > Cc: "Cindy Lu" <lulu@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> This doesn't seem to fix the problems I'm seeing here with a KVM/arm64
> guest (the issue exists with and without this patch)
>
> On initial boot:
>
> <quote>
> Loading Linux 5.7.0-1-arm64 ...
> Loading initial ramdisk ...
> qemu-system-aarch64: virtio-blk failed to set guest notifier (-16), ensure -accel kvm is set.
> qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
> qemu-system-aarch64: virtio-scsi: Failed to set guest notifiers (-16), ensure -accel kvm is set.
> qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
> </quote>
>
> The guest is functional though. However, on reboot:
>
> <quote>
> Loading Linux 5.7.0-1-arm64 ...
> Loading initial ramdisk ...
> qemu-system-aarch64: ../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier' failed.
> </quote>
>
> Reverting d5d24d859c fixes the issue. For the record, my qemu command
> line:
>
> /home/maz/qemu/build/qemu-system-aarch64 -m 1G -smp 8 -cpu host,aarch64=on -machine virt,accel=kvm,gic-version=host -nographic -drive if=pflash,format=raw,readonly=on,file=/usr/share/AAVMF/AAVMF_CODE.fd -drive if=pflash,format=raw,file=bullseye/bBwcgtDY2UwXklV6.fd -netdev user,id=hostnet0 -device virtio-net-pci,netdev=hostnet0 -drive if=none,format=raw,cache=none,aio=native,file=bullseye/bBwcgtDY2UwXklV6.img,id=disk0 -device virtio-blk-pci,drive=disk0 -drive file=debian-testing-arm64-netinst-preseed.iso,id=cdrom,if=none,media=cdrom -device virtio-scsi-pci -device scsi-cd,drive=cdrom
>
> Thanks,
>
> M.
Thanks, I'm reverting these changes for now.
Cindy, once you have your patches ready pinging people who encountered
problems with the previous versions is a very good idea.
> --
> Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-01-10 21:19 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin
2022-01-10 7:20 ` Cédric Le Goater
2022-01-10 20:37 ` Marc Zyngier
2022-01-10 21:09 ` Michael S. Tsirkin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.