Re: VFIO Migration

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3138 bytes --]

On Wed, Nov 04, 2020 at 02:50:58AM -0500, Michael S. Tsirkin wrote:
> On Mon, Nov 02, 2020 at 11:11:53AM +0000, Stefan Hajnoczi wrote:
> > Device States
> > -------------
> > The details of the device state representation are not covered in this document
> > but the general requirements are discussed here.
> > 
> > The device state consists of data accessible through the device's hardware
> > interface and internal state that is needed to restore device operation.
> > State in the hardware interface includes the values of hardware registers.
> > An example of internal state is an index value needed to avoid processing
> > queued requests more than once.
> > 
> > Changes can be made to the device state representation as follows. Each change
> > to device state must have a corresponding device configuration parameter that
> > allows the change to toggled:
> > 
> > * When the parameter is disabled the hardware interface and device state
> >   representation are unchanged. This allows old device states to be loaded.
> > 
> > * When the parameter is enabled the change comes into effect.
> > 
> > * The parameter's default value disables the change. Therefore old versions do
> >   not have to explicitly specify the parameter.
> > 
> > The following example illustrates migration from an old device
> > implementation to a new one. A version=1 network card is migrated to a
> > new device implementation that is also capable of version=2 and adds the
> > rx-filter-size=32 parameter. The new device is instantiated with
> > version=1, which disables rx-filter-size and is capable of loading the
> > version=1 device state. The migration completes successfully but note
> > the device is still operating at version=1 level in the new device.
> > 
> > The following example illustrates migration from a new device
> > implementation back to an older one. The new device implementation
> > supports version=1 and version=2. The old device implementation supports
> > version=1 only. Therefore the device can only be migrated when
> > instantiated with version=1 or the equivalent full configuration
> > parameters.
> 
> So all this is pretty complex and easy for vendors to get wrong.  How
> about we introduce a directory under docs/interop/ where each supported
> device can list the format of its state and parameters and what is tied
> to what?

Yes, that would be great for standardizing the device state
representations and migration parameters. I'm not aware of any devices
that need standardization yet but let's do it for vfio-user VIRTIO
devices.

> I am a bit unsure about the usefulness of the version shortcut.
> It would be handy if all this was used directly by users
> but these are unlikely to want to orchestrate cross version
> migrations, and tools do not need shortcuts like these ...

Me too. It's much easier for humans to compare version 1 and 2 than to
compare a potentially long list of parameters, but if it's always done
by the tooling then it doesn't matter. The device version can be dropped
for now and we can bring it back if we need it.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]