On Tue, Jul 06, 2021 at 12:33:32PM +0800, Jason Wang wrote:
> Hi All:
> 
> This is an updated version to implement virtqueue state
> synchronization which is a must for the migration support.
> 
> The first patch introduces virtqueue states as a new basic facility of
> the virtio device. This is used by the driver to save and restore
> virtqueue state. The states were split into available state and used
> state to ease the transport specific implementation. It is also
> allowed for the device to have its own device specific way to save and
> resotre extra virtqueue states like in flight request.
> 
> The second patch introduce a new status bit STOP. This bit is used for
> the driver to stop the device. The major difference from reset is that
> STOP must preserve all the virtqueue state plus the device state.
> 
> A driver can then:
> 
> - Get the virtqueue state if STOP status bit is set
> - Set the virtqueue state after FEATURE_OK but before DRIVER_OK
> 
> Device specific state synchronization could be built on top.

Will you send a proof-of-concept implementation to demonstrate how it
works in practice?

You mentioned being able to migrate virtio-net devices using this
interface, but what about state like VIRTIO_NET_S_LINK_UP that is either
per-device or associated with a non-rx/tx virtqueue?

Basically I'm not sure if the scope of this is just to migrate state
associated with offloaded virtqueues (vDPA, VFIO/mdev, etc) or if it's
really supposed to migrate the entire device?

Do you have an approach in mind for saving/loading device-specific
state? Here are devices and their state:
- virtio-blk: a list of requests that the destination device can
  re-submit
- virtio-scsi: a list of requests that the destination device can
  re-submit
- virtio-serial: active ports, including the current buffer being
  transferred
- virtio-net: MAC address, status, etc

Stefan