All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] virtio-pci: fix up config interrupt handling
@ 2022-01-09 17:49 Michael S. Tsirkin
  2022-01-10  7:20 ` Cédric Le Goater
  2022-01-10 20:37 ` Marc Zyngier
  0 siblings, 2 replies; 4+ messages in thread
From: Michael S. Tsirkin @ 2022-01-09 17:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Volker Rümelin, Cédric Le Goater, Cindy Lu

Fixes a couple of issues with irqfd use by config interrupt:
- Rearrange initialization so cleanup happens in the reverse order
- Don't use irqfd for config when not in use for data path
I am not sure this is a complete fix though: I think we
are better off limiting the effect to vdpa devices
with config interrupt support. Or even bypass irqfd
for config completely and inject into KVM using ioctl?
The advantage would be less FDs used.
This would mean mostly reverting the patchset though.

Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt")
Cc: "Cindy Lu" <lulu@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 hw/virtio/virtio-pci.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 98fb5493ae..b77cd69f97 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1130,15 +1130,15 @@ static int virtio_pci_set_guest_notifiers(DeviceState *d, int nvqs, bool assign)
             proxy->vector_irqfd =
                 g_malloc0(sizeof(*proxy->vector_irqfd) *
                           msix_nr_vectors_allocated(&proxy->pci_dev));
+            r = kvm_virtio_pci_vector_config_use(proxy);
+            if (r < 0) {
+                goto config_error;
+            }
             r = kvm_virtio_pci_vector_use(proxy, nvqs);
             if (r < 0) {
                 goto config_assign_error;
             }
         }
-        r = kvm_virtio_pci_vector_config_use(proxy);
-        if (r < 0) {
-            goto config_error;
-        }
         r = msix_set_vector_notifiers(&proxy->pci_dev, virtio_pci_vector_unmask,
                                       virtio_pci_vector_mask,
                                       virtio_pci_vector_poll);
@@ -1155,7 +1155,9 @@ notifiers_error:
         kvm_virtio_pci_vector_release(proxy, nvqs);
     }
 config_error:
-    kvm_virtio_pci_vector_config_release(proxy);
+    if (with_irqfd) {
+        kvm_virtio_pci_vector_config_release(proxy);
+    }
 config_assign_error:
     virtio_pci_set_guest_notifier(d, VIRTIO_CONFIG_IRQ_IDX, !assign,
                                   with_irqfd);
-- 
MST



^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [PATCH] virtio-pci: fix up config interrupt handling
  2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin
@ 2022-01-10  7:20 ` Cédric Le Goater
  2022-01-10 20:37 ` Marc Zyngier
  1 sibling, 0 replies; 4+ messages in thread
From: Cédric Le Goater @ 2022-01-10  7:20 UTC (permalink / raw)
  To: Michael S. Tsirkin, qemu-devel; +Cc: Volker Rümelin, Cindy Lu

On 1/9/22 18:49, Michael S. Tsirkin wrote:
> Fixes a couple of issues with irqfd use by config interrupt:
> - Rearrange initialization so cleanup happens in the reverse order
> - Don't use irqfd for config when not in use for data path
> I am not sure this is a complete fix though: I think we
> are better off limiting the effect to vdpa devices
> with config interrupt support. Or even bypass irqfd
> for config completely and inject into KVM using ioctl?
> The advantage would be less FDs used.
> This would mean mostly reverting the patchset though.
> 
> Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt")
> Cc: "Cindy Lu" <lulu@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

On a KVM guest with vhost, I am still seeing at reboot an issue :/

../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier'


C.

> ---
>   hw/virtio/virtio-pci.c | 12 +++++++-----
>   1 file changed, 7 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> index 98fb5493ae..b77cd69f97 100644
> --- a/hw/virtio/virtio-pci.c
> +++ b/hw/virtio/virtio-pci.c
> @@ -1130,15 +1130,15 @@ static int virtio_pci_set_guest_notifiers(DeviceState *d, int nvqs, bool assign)
>               proxy->vector_irqfd =
>                   g_malloc0(sizeof(*proxy->vector_irqfd) *
>                             msix_nr_vectors_allocated(&proxy->pci_dev));
> +            r = kvm_virtio_pci_vector_config_use(proxy);
> +            if (r < 0) {
> +                goto config_error;
> +            }
>               r = kvm_virtio_pci_vector_use(proxy, nvqs);
>               if (r < 0) {
>                   goto config_assign_error;
>               }
>           }
> -        r = kvm_virtio_pci_vector_config_use(proxy);
> -        if (r < 0) {
> -            goto config_error;
> -        }
>           r = msix_set_vector_notifiers(&proxy->pci_dev, virtio_pci_vector_unmask,
>                                         virtio_pci_vector_mask,
>                                         virtio_pci_vector_poll);
> @@ -1155,7 +1155,9 @@ notifiers_error:
>           kvm_virtio_pci_vector_release(proxy, nvqs);
>       }
>   config_error:
> -    kvm_virtio_pci_vector_config_release(proxy);
> +    if (with_irqfd) {
> +        kvm_virtio_pci_vector_config_release(proxy);
> +    }
>   config_assign_error:
>       virtio_pci_set_guest_notifier(d, VIRTIO_CONFIG_IRQ_IDX, !assign,
>                                     with_irqfd);
> 



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] virtio-pci: fix up config interrupt handling
  2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin
  2022-01-10  7:20 ` Cédric Le Goater
@ 2022-01-10 20:37 ` Marc Zyngier
  2022-01-10 21:09   ` Michael S. Tsirkin
  1 sibling, 1 reply; 4+ messages in thread
From: Marc Zyngier @ 2022-01-10 20:37 UTC (permalink / raw)
  To: Michael S. Tsirkin
  Cc: Volker Rümelin, qemu-devel, Cindy Lu, Cédric Le Goater

Hi Michael,

On Sun, 09 Jan 2022 17:49:19 +0000,
"Michael S. Tsirkin" <mst@redhat.com> wrote:
> 
> Fixes a couple of issues with irqfd use by config interrupt:
> - Rearrange initialization so cleanup happens in the reverse order
> - Don't use irqfd for config when not in use for data path
> I am not sure this is a complete fix though: I think we
> are better off limiting the effect to vdpa devices
> with config interrupt support. Or even bypass irqfd
> for config completely and inject into KVM using ioctl?
> The advantage would be less FDs used.
> This would mean mostly reverting the patchset though.
> 
> Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt")
> Cc: "Cindy Lu" <lulu@redhat.com>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

This doesn't seem to fix the problems I'm seeing here with a KVM/arm64
guest (the issue exists with and without this patch)

On initial boot:

<quote>
Loading Linux 5.7.0-1-arm64 ...
Loading initial ramdisk ...
qemu-system-aarch64: virtio-blk failed to set guest notifier (-16), ensure -accel kvm is set.
qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
qemu-system-aarch64: virtio-scsi: Failed to set guest notifiers (-16), ensure -accel kvm is set.
qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
</quote>

The guest is functional though. However, on reboot:

<quote>
Loading Linux 5.7.0-1-arm64 ...
Loading initial ramdisk ...
qemu-system-aarch64: ../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier' failed.
</quote>

Reverting d5d24d859c fixes the issue. For the record, my qemu command
line:

/home/maz/qemu/build/qemu-system-aarch64 -m 1G -smp 8 -cpu host,aarch64=on -machine virt,accel=kvm,gic-version=host -nographic -drive if=pflash,format=raw,readonly=on,file=/usr/share/AAVMF/AAVMF_CODE.fd -drive if=pflash,format=raw,file=bullseye/bBwcgtDY2UwXklV6.fd -netdev user,id=hostnet0 -device virtio-net-pci,netdev=hostnet0 -drive if=none,format=raw,cache=none,aio=native,file=bullseye/bBwcgtDY2UwXklV6.img,id=disk0 -device virtio-blk-pci,drive=disk0 -drive file=debian-testing-arm64-netinst-preseed.iso,id=cdrom,if=none,media=cdrom -device virtio-scsi-pci -device scsi-cd,drive=cdrom

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] virtio-pci: fix up config interrupt handling
  2022-01-10 20:37 ` Marc Zyngier
@ 2022-01-10 21:09   ` Michael S. Tsirkin
  0 siblings, 0 replies; 4+ messages in thread
From: Michael S. Tsirkin @ 2022-01-10 21:09 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Volker Rümelin, qemu-devel, Cindy Lu, Cédric Le Goater

On Mon, Jan 10, 2022 at 08:37:43PM +0000, Marc Zyngier wrote:
> Hi Michael,
> 
> On Sun, 09 Jan 2022 17:49:19 +0000,
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > 
> > Fixes a couple of issues with irqfd use by config interrupt:
> > - Rearrange initialization so cleanup happens in the reverse order
> > - Don't use irqfd for config when not in use for data path
> > I am not sure this is a complete fix though: I think we
> > are better off limiting the effect to vdpa devices
> > with config interrupt support. Or even bypass irqfd
> > for config completely and inject into KVM using ioctl?
> > The advantage would be less FDs used.
> > This would mean mostly reverting the patchset though.
> > 
> > Fixes: d5d24d859c ("virtio-pci: add support for configure interrupt")
> > Cc: "Cindy Lu" <lulu@redhat.com>
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> This doesn't seem to fix the problems I'm seeing here with a KVM/arm64
> guest (the issue exists with and without this patch)
> 
> On initial boot:
> 
> <quote>
> Loading Linux 5.7.0-1-arm64 ...
> Loading initial ramdisk ...
> qemu-system-aarch64: virtio-blk failed to set guest notifier (-16), ensure -accel kvm is set.
> qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
> qemu-system-aarch64: virtio-scsi: Failed to set guest notifiers (-16), ensure -accel kvm is set.
> qemu-system-aarch64: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
> </quote>
> 
> The guest is functional though. However, on reboot:
> 
> <quote>
> Loading Linux 5.7.0-1-arm64 ...
> Loading initial ramdisk ...
> qemu-system-aarch64: ../hw/pci/msix.c:622: msix_unset_vector_notifiers: Assertion `dev->msix_vector_use_notifier && dev->msix_vector_release_notifier' failed.
> </quote>
> 
> Reverting d5d24d859c fixes the issue. For the record, my qemu command
> line:
> 
> /home/maz/qemu/build/qemu-system-aarch64 -m 1G -smp 8 -cpu host,aarch64=on -machine virt,accel=kvm,gic-version=host -nographic -drive if=pflash,format=raw,readonly=on,file=/usr/share/AAVMF/AAVMF_CODE.fd -drive if=pflash,format=raw,file=bullseye/bBwcgtDY2UwXklV6.fd -netdev user,id=hostnet0 -device virtio-net-pci,netdev=hostnet0 -drive if=none,format=raw,cache=none,aio=native,file=bullseye/bBwcgtDY2UwXklV6.img,id=disk0 -device virtio-blk-pci,drive=disk0 -drive file=debian-testing-arm64-netinst-preseed.iso,id=cdrom,if=none,media=cdrom -device virtio-scsi-pci -device scsi-cd,drive=cdrom
> 
> Thanks,
> 
> 	M.


Thanks, I'm reverting these changes for now.
Cindy, once you have your patches ready pinging people who encountered
problems with the previous versions is a very good idea.

> -- 
> Without deviation from the norm, progress is not possible.



^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-01-10 21:19 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-09 17:49 [PATCH] virtio-pci: fix up config interrupt handling Michael S. Tsirkin
2022-01-10  7:20 ` Cédric Le Goater
2022-01-10 20:37 ` Marc Zyngier
2022-01-10 21:09   ` Michael S. Tsirkin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.