From: Stefan Hajnoczi <stefanha@redhat.com> To: Sebastien Boeuf <sebastien.boeuf@intel.com> Cc: virtio-fs@redhat.com, qemu-devel@nongnu.org Subject: vhost-user reconnection and crash recovery Date: Tue, 11 May 2021 13:45:13 +0100 [thread overview] Message-ID: <YJp8WbMaf3Y8/F7l@stefanha-x1.localdomain> (raw) [-- Attachment #1: Type: text/plain, Size: 2243 bytes --] Hi Sebastien, On #virtio-fs IRC you asked: I have a vhost-user question regarding disconnection/reconnection. How should this be handled? Let's say the vhost-user backend disconnects, and reconnects later on, does QEMU reset the virtio device by notifying the guest? Or does it simply reconnects to the backend without letting the guest know about what happened? The vhost-user protocol does not have a generic reconnection solution. Reconnection is handled on a case-by-case basis because device-specific and implementation-specific state is involved. The vhost-user-fs-pci device in QEMU has not been tested with reconnection as far as I know. The ideal reconnection behavior is to resume the device from its previous state without disrupting the guest. Device state must survive reconnection in order for this to work. Neither QEMU virtiofsd nor virtiofsd-rs implement this today. virtiofs has a lot of state, making it particularly difficult to support either DEVICE_NEEDS_RESET or transparent vhost-user reconnection. We have discussed virtiofs crash recovery on the bi-weekly virtiofs call (https://etherpad.opendev.org/p/virtiofs-external-meeting). If you want to work on this then joining the call would be a good starting point to coordinate with others. One approach for transparent crash recovery is for virtiofsd to keep its state in tmpfs (e.g. inode/fd mappings) and open fds shared with a clone(2) process via CLONE_FILES. This way the virtiofsd process can terminate but its state persists in memory thanks to its clone process. The clone can then be used to launch the new virtiofsd process from the old state. This would allow the device to resume transparently with QEMU only reconnecting the vhost-user UNIX domain socket. This is an idea that we discussed in the bi-weekly virtiofs call. You mentioned device reset. VIRTIO 1.1 has the Device Status Field DEVICE_NEEDS_RESET flat that the device can use to tell the driver that a reset is necessary. This feature is present in the specification but not implemented in the Linux guest drivers. Again the reason is that handling it requires driver-specific logic for restoring state after reset...otherwise the device reset would be visible to userspace. Stefan [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --]
WARNING: multiple messages have this Message-ID (diff)
From: Stefan Hajnoczi <stefanha@redhat.com> To: Sebastien Boeuf <sebastien.boeuf@intel.com> Cc: virtio-fs@redhat.com, qemu-devel@nongnu.org Subject: [Virtio-fs] vhost-user reconnection and crash recovery Date: Tue, 11 May 2021 13:45:13 +0100 [thread overview] Message-ID: <YJp8WbMaf3Y8/F7l@stefanha-x1.localdomain> (raw) [-- Attachment #1: Type: text/plain, Size: 2243 bytes --] Hi Sebastien, On #virtio-fs IRC you asked: I have a vhost-user question regarding disconnection/reconnection. How should this be handled? Let's say the vhost-user backend disconnects, and reconnects later on, does QEMU reset the virtio device by notifying the guest? Or does it simply reconnects to the backend without letting the guest know about what happened? The vhost-user protocol does not have a generic reconnection solution. Reconnection is handled on a case-by-case basis because device-specific and implementation-specific state is involved. The vhost-user-fs-pci device in QEMU has not been tested with reconnection as far as I know. The ideal reconnection behavior is to resume the device from its previous state without disrupting the guest. Device state must survive reconnection in order for this to work. Neither QEMU virtiofsd nor virtiofsd-rs implement this today. virtiofs has a lot of state, making it particularly difficult to support either DEVICE_NEEDS_RESET or transparent vhost-user reconnection. We have discussed virtiofs crash recovery on the bi-weekly virtiofs call (https://etherpad.opendev.org/p/virtiofs-external-meeting). If you want to work on this then joining the call would be a good starting point to coordinate with others. One approach for transparent crash recovery is for virtiofsd to keep its state in tmpfs (e.g. inode/fd mappings) and open fds shared with a clone(2) process via CLONE_FILES. This way the virtiofsd process can terminate but its state persists in memory thanks to its clone process. The clone can then be used to launch the new virtiofsd process from the old state. This would allow the device to resume transparently with QEMU only reconnecting the vhost-user UNIX domain socket. This is an idea that we discussed in the bi-weekly virtiofs call. You mentioned device reset. VIRTIO 1.1 has the Device Status Field DEVICE_NEEDS_RESET flat that the device can use to tell the driver that a reset is necessary. This feature is present in the specification but not implemented in the Linux guest drivers. Again the reason is that handling it requires driver-specific logic for restoring state after reset...otherwise the device reset would be visible to userspace. Stefan [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --]
next reply other threads:[~2021-05-11 12:47 UTC|newest] Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-05-11 12:45 Stefan Hajnoczi [this message] 2021-05-11 12:45 ` [Virtio-fs] vhost-user reconnection and crash recovery Stefan Hajnoczi 2021-05-11 15:00 Boeuf, Sebastien 2021-05-11 15:37 ` Stefan Hajnoczi
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=YJp8WbMaf3Y8/F7l@stefanha-x1.localdomain \ --to=stefanha@redhat.com \ --cc=qemu-devel@nongnu.org \ --cc=sebastien.boeuf@intel.com \ --cc=virtio-fs@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.