Re: [Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap

[Kevin Wolf <kwolf@redhat.com>]

Am 26.01.2016 um 11:38 hat Fam Zheng geschrieben:
> This series introduces a simple format to enable support of persistence of
> block dirty bitmaps. Block dirty bitmap is the tool to achieve incremental
> backup, and persistence of block dirty bitmap makes incrememtal backup possible
> across VM shutdowns, where existing in-memory dirty bitmaps cannot survive.
> 
> When user creates a "persisted" dirty bitmap, the QBM driver will create a
> binary file and synchronize it with the existing in-memory block dirty bitmap
> (BdrvDirtyBitmap). When the VM is powered down, the binary file has all the
> bits saved on disk, which will be loaded and used to initialize the in-memory
> block dirty bitmap next time the guest is started.
> 
> The idea of the format is to reuse as much existing infrastructure as possible
> and avoid introducing complex data structures - it works with any image format,
> by gluing it together plain bitmap files with a json descriptor file. The
> advantage of this approach over extending existing formats, such as qcow2, is
> that the new feature is implemented by an orthogonal driver, in a format
> agnostic way. This way, even raw images can have their persistent dirty
> bitmaps.  (And you will notice in this series, with a little forging to the
> spec, raw images can also have backing files through a QBM overlay!)
> 
> Rather than superseding it, this intends to be coexistent in parallel with the
> qcow2 bitmap extension that Vladimir is working on.  The block driver interface
> changes in this series also try to be generic and compatible for both drivers.

So as I already told Fam last week, before we discuss any technical
details here, we first need to discuss whether this is even the right
thing to do. Currently I'm doubtful, as this is another attempt to
introduce a new native image format in qemu.

Let's recap the image formats and what we tell users about them today:

* qcow2: This is the default choice for disk images. It gives you access
  to all of the features in qemu at a good performance. If it doesn't
  perform well in your case, we'll fix it.

* raw: Use this when you need absolute performance and don't need any
  features from an image format, so you want to get any complexity just
  out of the way and pass requests as directly as possible from the
  guest device to the host kernel.

* Anything else: Only use them to convert into raw or qcow2.

Now using bitmaps is clearly on the "features" side, which suggests that
qcow2 is the format of choice for this. If you want to introduce a new
format, you need to justify it with evidence that...

1. there is a relevant use case that qcow2 doesn't cover
2. qcow2 can't be fixed/enhanced to cover the use case

The one thing that people have claimed in the past that qcow2 can't
provide is enough performance. This is where QED tried to come in and
promised a compromise between performance (then a bit faster than qcow2)
and features (almost none, but supports backing files). We all know that
it was a failure because you had to sacrifice features and still the
idea that qcow2 couldn't be fixed was wrong, so today we have a QED
driver that is much slower than qcow2 despite having less features.

Now for QBM. First, let's have a look at the image format that it can be
used with. qcow2 doesn't need it if we continue with Vladimir's
extension. Other non-raw formats are only supposed to be used for
conversion. The only thing that's really left is raw. Now adding a
feature only for raw, as a compromise between features and performance,
looks an awful lot like what QED tried. We don't want to go there.

Even if we wanted to support persistent dirty bitmaps with raw images
(which has to be discussed based on use cases), it's still questionable
whether we need a new image format with JSON descriptor files instead of
just raw bitmaps that can be added with a QMP command.


tl;dr: Where is the justification for a new image format? You need a
good one.

Kevin