Re: KVM call agenda for June 28

[Stefan Hajnoczi <stefanha@gmail.com>]

On Wed, Jun 29, 2011 at 4:41 PM, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> On Wed, Jun 29, 2011 at 11:08:23AM +0100, Stefan Hajnoczi wrote:
>>  This can be used to merge data from an intermediate image without
>> merging the base image.  When streaming completes the backing file
>> will be set to the base image.  The backing file relationship would
>> typically look like this:
>>
>> 1. Before block_stream -a -b base.img ide0-hd completion:
>>
>> base.img <- sn1 <- ... <- ide0-hd.qed
>>
>> 2. After streaming completes:
>>
>> base.img <- ide0-hd.qed
>>
>> This describes the image streaming use cases that I, Adam, and Anthony
>> propose to support.  In the course of the discussion we've sometimes
>> been distracted with the internals of what a unified live block
>> copy/image streaming implementation should do.  I wanted to post this
>> summary of image streaming to refocus us on the use case and the APIs
>> that users will see.
>>
>> Stefan
>
> OK, with an external COW file for formats that do not support it the
> interface can be similar. Also there is no need to mirror writes,
> no switch operation, always use destination image.

Marcelo, does this mean you are happy with how management deals with
power failure/crash during streaming?

Are we settled on the approach where the destination file always has
the source file as its backing file?

Here are the components that I can identify:

1. blkmirror - used by live block copy to keep source and destination
in sync.  Already implemented as a block driver by Marcelo.

2. External COW overlay - can be used to add backing file (COW)
support on top of any image, including raw.  Currently unimplemented,
needs to be a block driver.  Kevin, do you want to write this?

3. Unified background copy - image format-independent mechanism for
copy contents of a backing file chain into the image file (with
exception of backing files chained below base).  Needs to play nice
with blkmirror.  Stefan can write this.

4. Live block copy API and high-level control - the main code that
adds the live block copy feature.  Existing patches by Marcelo, can be
restructured to use common core by Marcelo.

5. Image streaming API and high-level control - the main code that
adds the image streaming feature.  Existing patches by Stefan, Adam,
Anthony, can be restructured to use common core by Stefan.

I previously posted a proposed API for the unified background copy
mechanism.  I'm thinking that background copy is not the best name
since it is limited to copying the backing file into the image file.

/**
 * Start a background copy operation
 *
 * Unallocated clusters in the image will be populated with data
 * from its backing file.  This operation runs in the background and a
 * completion function is invoked when it is finished.
 */
BackgroundCopy *background_copy_start(
   BlockDriverState *bs,

   /**
    * Note: Kevin suggests we migrate this into BlockDriverState
    *       in order to enable copy-on-read.
    *
    * Base image that both source and destination have as a
    * backing file ancestor.  Data will not be copied from base
    * since both source and destination will have access to base
    * image.  This may be NULL to copy all data.
    */
   BlockDriverState *base,

   BlockDriverCompletionFunc *cb, void *opaque);

/**
 * Cancel a background copy operation
 *
 * This function marks the background copy operation for cancellation and the
 * completion function is invoked once the operation has been cancelled.
 */
void background_copy_cancel(BackgroundCopy *bgc,
                            BlockDriverCompletionFunc *cb, void *opaque);

/**
 * Get progress of a running background copy operation
 */
void background_copy_get_status(BackgroundCopy *bgc,
                                BackgroundCopyStatus *status);

Stefan