Re: device compatibility interface for live migration with assigned devices

[Yan Zhao <yan.y.zhao@intel.com>]

hi All,
Per our previous discussion, there are two main concerns to the previous
proposal:
(1) it's currently hard for openstack to match mdev types.
(2) complicated.

so, we further propose below changes:
(1) requiring two compatible mdevs to have the same mdev type for now.
    (though kernel still exposes compatible_type attributes for future use)  
(2) requiring 1:1 match for other attributes under sysfs type node for now
    (those attributes are specified via compatible_<attribute name> but
    with only 1 value in it.)
(3) do not match attributes under device instance node.
    rather, they are regarded as part of resource claiming process.
    so src and dest values are ensured to be 1:1.
    A dynamic_resources attribute under sysfs <type-id> node is added to
    list the attributes under device instance that mgt tools need to
    ensure 1:1 from src and dest.
    the "aggregator" attribute under device instance node is such one that
    needs to be listed.
    Those listed attributes can actually be treated as device state set by
    vendor driver during live migration. but we still want to ask for them to
    be set by mgt tools before live migration starts, in oder to reduce the
    chance of live migration failure.

do you like those changes?

after the changes, the sysfs interface would look like blow:

  |- [parent physical device]
  |--- Vendor-specific-attributes [optional]
  |--- [mdev_supported_types]
  |     |--- [<type-id>]
  |     |   |--- create
  |     |   |--- name
  |     |   |--- available_instances
  |     |   |--- device_api
  |     |   |--- software_version
  |     |   |--- compatible_type
  |     |   |--- compatible_<device_api_specific_field>
  |     |   |--- compatible_<type_specific_field>
  |     |   |--- dynamic_resources
  |     |   |--- description
  |     |   |--- [devices]

- device_api : exact match between src and dest is required.
               its value can be one of 
               "vfio-pci", "vfio-platform", "vfio-amba", "vfio-ccw", "vfio-ap"
- software_version: version of vendor driver.
                    in major.minor.bugfix scheme. 
                    dest major should be equal to src major,
	            dest minor should be no less than src minor.
		    once migration stream related code changed, vendor
		    drivers need to bump the version.
- compatible_type: not used by mgt tools currently.
                   vendor drivers can provide this attribute, but need to
		   know that mgt apps would ignore it.
		   when in future mgt tools support this attribute, it
		   would allow migration across different mdev types,
		   so that devices of older generation may be able to
		   migrate to newer generations.

- compatible_<device_api_specific_field>: for device api specific attributes,
                  e.g. compatible_subchannel_type,
                  dest values should be superset of arc values.
		  vendor drivers can specify only one value in this attribute,
		  in order to do exact match between src and dest.
		  It's ok for mgt tools to only read one value in the
		  attribute so that src:dest values are 1:1.

- compatible_<type_specific_field>: for mdev type specific attributes,
                  e.g. compatible_pci_ids, compatible_chpid_type
                  dest values should be superset of arc values.
		  vendor drivers can specify only one value in the attribute
		  in order to do exact match between src and dest.
		  It's ok for mgt tools to only read one value in the
		  attribute so that src:dest values are 1:1.

- dynamic_resources: though defined statically under <type-id>,
                  this attribute lists attributes under device instance that
		  need to be set as part of claiming dest resources.
		  e.g. $cat dynamic_resources: aggregator, fps,...
		  then after dest device is created, values of its device
		  attributes need to be set to that of src device attributes.
		  Failure in syncing src device values to dest device
		  values is treated the same as failing to claiming
		  dest resources.
		  attributes under device instance that are not listed
		  in this attribute would not be part of resource checking in
		  mgt tools.



Thanks
Yan