From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50380) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aY92o-0003l3-NX for qemu-devel@nongnu.org; Tue, 23 Feb 2016 04:14:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aY92n-0006zW-IW for qemu-devel@nongnu.org; Tue, 23 Feb 2016 04:14:10 -0500 From: Markus Armbruster References: <1453804705-7205-1-git-send-email-famz@redhat.com> <20160222142415.GG5387@noname.str.redhat.com> Date: Tue, 23 Feb 2016 10:14:00 +0100 In-Reply-To: <20160222142415.GG5387@noname.str.redhat.com> (Kevin Wolf's message of "Mon, 22 Feb 2016 15:24:15 +0100") Message-ID: <87k2lv94br.fsf@blackfin.pond.sub.org> MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Qemu-devel] [RFC PATCH 00/16] Qemu Bit Map (QBM) - an overlay format for persistent dirty bitmap List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Kevin Wolf Cc: Fam Zheng , qemu-block@nongnu.org, qemu-devel@nongnu.org, mreitz@redhat.com, vsementsov@parallels.com, Stefan Hajnoczi , jsnow@redhat.com Kevin Wolf writes: > Am 26.01.2016 um 11:38 hat Fam Zheng geschrieben: >> This series introduces a simple format to enable support of persistence of >> block dirty bitmaps. Block dirty bitmap is the tool to achieve incremental >> backup, and persistence of block dirty bitmap makes incrememtal backup possible >> across VM shutdowns, where existing in-memory dirty bitmaps cannot survive. >> >> When user creates a "persisted" dirty bitmap, the QBM driver will create a >> binary file and synchronize it with the existing in-memory block dirty bitmap >> (BdrvDirtyBitmap). When the VM is powered down, the binary file has all the >> bits saved on disk, which will be loaded and used to initialize the in-memory >> block dirty bitmap next time the guest is started. >> >> The idea of the format is to reuse as much existing infrastructure as possible >> and avoid introducing complex data structures - it works with any image format, >> by gluing it together plain bitmap files with a json descriptor file. The >> advantage of this approach over extending existing formats, such as qcow2, is >> that the new feature is implemented by an orthogonal driver, in a format >> agnostic way. This way, even raw images can have their persistent dirty >> bitmaps. (And you will notice in this series, with a little forging to the >> spec, raw images can also have backing files through a QBM overlay!) >> >> Rather than superseding it, this intends to be coexistent in parallel with the >> qcow2 bitmap extension that Vladimir is working on. The block driver interface >> changes in this series also try to be generic and compatible for both drivers. > > So as I already told Fam last week, before we discuss any technical > details here, we first need to discuss whether this is even the right > thing to do. Yes, this must come first. > Currently I'm doubtful, as this is another attempt to > introduce a new native image format in qemu. > > Let's recap the image formats and what we tell users about them today: > > * qcow2: This is the default choice for disk images. It gives you access > to all of the features in qemu at a good performance. If it doesn't > perform well in your case, we'll fix it. Rather: we'll fix it if we can. > * raw: Use this when you need absolute performance and don't need any > features from an image format, so you want to get any complexity just > out of the way and pass requests as directly as possible from the > guest device to the host kernel. > > * Anything else: Only use them to convert into raw or qcow2. > > Now using bitmaps is clearly on the "features" side, which suggests that > qcow2 is the format of choice for this. I'd agree with a general "extra feature suggests QCOW2" maxim, with stress on "suggests". However, the "extraness" of bitmaps is perhaps less clear than for other features. Bitmap-like things occur not just in formats: sparse files, thinly provisioned SCSI devices, ... > If you want to introduce a new > format, you need to justify it with evidence that... > > 1. there is a relevant use case that qcow2 doesn't cover > 2. qcow2 can't be fixed/enhanced to cover the use case > > The one thing that people have claimed in the past that qcow2 can't > provide is enough performance. This is where QED tried to come in and > promised a compromise between performance (then a bit faster than qcow2) > and features (almost none, but supports backing files). We all know that > it was a failure because you had to sacrifice features and still the > idea that qcow2 couldn't be fixed was wrong, so today we have a QED > driver that is much slower than qcow2 despite having less features. Yes. We thought QCOW2 could not be made to perform[*], until you did. New storage hardware will bring back performance pressure with a vengeance, though. > Now for QBM. First, let's have a look at the image format that it can be > used with. qcow2 doesn't need it if we continue with Vladimir's > extension. Other non-raw formats are only supposed to be used for > conversion. The only thing that's really left is raw. Now adding a > feature only for raw, as a compromise between features and performance, > looks an awful lot like what QED tried. We don't want to go there. A possible difference: complexity. Adding another QEMU-native format in QCOW2's complexity class would be highly problematic. We tried with QED, because we thought we'd need it to support different tradeoffs, but it turned out to be a dead end. Doesn't mean there's absolutely no space for a *simple* format to support different tradeoffs. Is QBM simple enough? Will it stay simple enough? > Even if we wanted to support persistent dirty bitmaps with raw images > (which has to be discussed based on use cases), it's still questionable > whether we need a new image format with JSON descriptor files instead of > just raw bitmaps that can be added with a QMP command. > > > tl;dr: Where is the justification for a new image format? You need a > good one. Yes. > Kevin [*] Mostly because we thought QCOW2 could not be hacked.