Re: [virtio-comment] [PATCH V2 2/2] virtio: introduce STOP status bit

[Jason Wang <jasowang@redhat.com>]

在 2021/8/3 下午8:22, Dr. David Alan Gilbert 写道:
> * Jason Wang (jasowang@redhat.com) wrote:
>> 在 2021/8/3 下午6:37, Stefan Hajnoczi 写道:
>>> On Tue, Aug 03, 2021 at 02:33:20PM +0800, Jason Wang wrote:
>>>> 在 2021/7/26 下午11:07, Stefan Hajnoczi 写道:
>>>>> I guess this is just a summary of what we've already discussed and not
>>>>> new information. I think an implementation today would use DBus VMState
>>>>> to transfer implementation-specific device state (an opaque blob).
>>>> Instead of trying to migrate those opaque stuffs which is kind of tricky, I
>>>> wonder if we can avoid them by recording the mapping in the shared
>>>> filesystem itself.
>>> The problem is that virtiofsd has no way of reopening the exact same
>>> files without Linux file handles.
>>
>> I believe if we want to support live migration of the passthrough
>> filesystem. The filesystem itself must be shared? (like NFS)
>>
>> Assuming this is true. Can we store those mapping (e.g fuse inode -> host
>> inode) in a known path/file in the passthrough filesystem itself and hide
>> that file from the guest?
> That's pretty dangerous; it assumes that the filesystem is only used
> together with virtiofs; as a *shared* filesystem it's possible that it's
> being used directly by normal NFS clients as well.
> It's also very racy; trying to make sure those mappings reflect the
> *current* meaning of inodes even while they're changing under your feet
> is non-trivial.


Right, it's just a thought to avoid migrating implementation specific 
stuffs.


>
>> The destination can simply open this unkown file and do the lookup the
>> mapping and reopen the file if necessary.
>>
>> Then we don't need the Linux file handle.
>>
>>
>>>    So they need to be transferred to the
>>> destination (or stored on a shared file system as you suggested),
>>> regardless of whether they are part of the VIRTIO spec's device state or
>>> not.
>>>
>>> Implementation-specific state can be considered outside the scope of the
>>> VIRTIO spec. In other words, we could exclude it from the VIRTIO-level
>>> device state that save/load operate on. This does not solve the problem,
>>> it just shifts the responsibility to the virtualization stack to migrate
>>> this state.
>>>
>>> The Linux file handles or other virtiofsd implementation-specific state
>>> would be migrated separately (e.g. using DBus VMstate) so that by the
>>> time the destination device does a VIRTIO load operation, it has the
>>> necessary implementation-specific state ready.
>>
>> That may work but I want to get rid of the implementation specific stuffs
>> like linux handles completely.
> I'm not sure how much implementation specific you can get rid of; but
> you should be able to comparmentalise it, and you should be able to make
> it so that common things can be shared;


Yes, that's is the way we need to go.


>   i.e. if I have two
> implementations of virtiofs, both running on Linux, then it might be
> good if we can live migrate between them, and standardise the format.


As replied in the previous version, I'm not sure how hard it is consider 
the file_handle mentioned by Stefan is not a part of uABI and it depends 
on specific kernel config to work.


>
> So, I'd expect the core virtiofs data to be standardised globally,


Yes, maybe start at the FUSE level.


>   then
> I'd expect how Linux implementations work to be standardised.


Does it mean we need:

1) port virtiofsd to multiple platforms
2) only support live migration among virtiofds

?

Thanks


>
> Dave
>
>>> I prefer to support in-band migration of implementation-specific state
>>> because it's less complex to have a single device state instead of
>>> splitting it.
>>
>> I wonder how to deal with migration compatibility in this case.
>>
>>
>>> Is this the direction you were thinking in?
>>
>> Somehow.
>>
>> Thanks
>>
>>
>>> Stefan