Re: bitmaps -- copying allocation status into dirty bitmaps

[Eric Blake <eblake@redhat.com>]

On 11/1/19 4:49 PM, Denis V. Lunev wrote:
> On 11/1/19 4:42 PM, John Snow wrote:
>> Hi, in one of my infamously unreadable and long status emails, I
>> mentioned possibly wanting to copy allocation data into bitmaps as a way
>> to enable users to create (external) snapshots from outside of the
>> libvirt/qemu context.
>>
>> (That is: to repair checkpoints in libvirt after a user extended the
>> backing chain themselves, you want to restore bitmap information for
>> that node. Conveniently, this information IS the allocation map, so we
>> can do this.)
>>
>> It came up at KVM Forum that we probably do want this, because oVirt
>> likes the idea of being able to manipulate these chains from outside of
>> libvirt/qemu.
>>
>> Denis suggested that instead of a new command, we can create a special
>> name -- maybe "#ALLOCATED" or something similar that can never be
>> allocated as a user-defined bitmap name -- as a special source for the
>> merge command.
>>
>> You'd issue a merge from "#ALLOCATED" to "myBitmap0" to copy the current
>> allocation data into "myBitmap0", for instance.
> original problem was a little bit incorrect. After some thoughts I found
> that this is NOT enough. We should also add zeroed clusters to the
> bitmap to merge! They do cover some data clusters from the original
> image.
> 
> Thus we should either provide "ALLOCATED" bitmap for other purposes,
> and we should supply "CHANGED" which contains allocated AND
> explicitly zeroed clusters.

I'm also wondering if 'nbd-server-add' should add support to expose a 
'qemu:allocation:XXX' meta context, in addition to the existing 
'qemu:dirty-bitmap:XXX' contexts (would it just be a 0/1 bit for what is 
allocated in block layer XXX, or would it be an integer depth 0,1,2,... 
based on how deep in the chain a cluster is allocated, or?)

It may also be interesting to have a way to have 'nbd-server-add' force 
EIO errors to reads from an area not covered by a bitmap (whether that 
be the dirty bitmap or the #ALLOCATED bitmap), rather than falling back 
to a read from the backing chain, to ensure that an NBD client using 
such a backup image cannot read any more data than what the bitmap says 
is available.

> 
> Den
>> Some thoughts:
>>
>> - The only commands where this pseudo-bitmap makes sense is merge.
>> enable/disable/remove/clear/add don't make sense here.
>>
>> - This pseudo bitmap might make sense for backup, but it's not needed;
>> you can just merge into an empty/enabled bitmap and then use that.
>>
>> - Creating an allocation bitmap on-the-fly is probably not possible
>> directly in the merge command, because the disk status calls might take
>> too long...
>>
>> Hm, actually, I'm not sure how to solve that one. Merge would need to
>> become a job (or an async QMP command?) or we'd need to keep an
>> allocation bitmap object around and in-sync. I don't really want to do
>> either, so maybe I'm missing an obvious/better solution.
>>
>> Also, with regards to introspection, if we do create a special reserved
>> name like #ALLOCATED, we need to make sure that this is available and
>> obvious via the QAPI schema.

If nothing else, the recent addition of introspectible QMP features 
should make this possible.


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org