From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50957) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aJq4N-0001PL-4t for qemu-devel@nongnu.org; Thu, 14 Jan 2016 17:08:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aJq4J-0003T7-Nj for qemu-devel@nongnu.org; Thu, 14 Jan 2016 17:08:38 -0500 Received: from mx1.redhat.com ([209.132.183.28]:48950) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aJq4J-0003Sr-DP for qemu-devel@nongnu.org; Thu, 14 Jan 2016 17:08:35 -0500 References: <1452517517-3953-1-git-send-email-vsementsov@virtuozzo.com> From: Eric Blake Message-ID: <56981C60.9020005@redhat.com> Date: Thu, 14 Jan 2016 15:08:32 -0700 MIME-Version: 1.0 In-Reply-To: <1452517517-3953-1-git-send-email-vsementsov@virtuozzo.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="QuPJScAbsLJ3Wh8v7u9EN11SNOB5x1La3" Subject: Re: [Qemu-devel] [PATCH v7] spec: add qcow2 bitmaps extension specification List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladimir Sementsov-Ogievskiy , qemu-devel@nongnu.org Cc: kwolf@redhat.com, famz@redhat.com, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org, jsnow@redhat.com This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --QuPJScAbsLJ3Wh8v7u9EN11SNOB5x1La3 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 01/11/2016 06:05 AM, Vladimir Sementsov-Ogievskiy wrote: > The new feature for qcow2: storing bitmaps. >=20 > This patch adds new header extension to qcow2 - Bitmaps Extension. It > provides an ability to store virtual disk related bitmaps in a qcow2 > image. For now there is only one type of such bitmaps: Dirty Tracking > Bitmap, which just tracks virtual disk changes from some moment. >=20 > Note: Only bitmaps, relative to the virtual disk, stored in qcow2 file,= > should be stored in this qcow2 file. The size of each bitmap > (considering its granularity) is equal to virtual disk size. >=20 > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- >=20 > @@ -166,6 +178,34 @@ the header extension data. Each entry look like th= is: > terminated if it has full length) > =20 > =20 > +=3D=3D Bitmaps extension =3D=3D > + 0 - 3: nb_bitmaps > + The number of bitmaps contained in the image. Must = be > + greater than or equal to 1. > + > + Note: Qemu currently only supports up to 65535 bitm= aps per > + image. > + > + 4 - 7: bitmap_directory_size > + Size of the bitmap directory in bytes. It is the cu= mulative > + size of all (nb_bitmaps) bitmap headers. Only 4 bytes - if we ever raise our 64k entry restriction (nb_bitmaps), could we run into an image that has so many directory entries as to make the directory itself spill past 4G? But I don't think it is likely, so I can live with your choice. > + > +=3D=3D Bitmaps =3D=3D > + > +As mentioned above, the bitmaps extension provides the ability to stor= e bitmaps > +related a virtual disk. This section describes how these bitmaps are s= tored. > + > +Note: all bitmaps are related to the virtual disk stored in this image= =2E > + > +=3D=3D=3D Bitmap directory =3D=3D=3D > + > +Each bitmap saved in the image is described in a bitmap directory entr= y. The > +bitmap directory is a contiguous area in the image file, whose startin= g offset > +and length are given by the header extension fields bitmap_directory_o= ffset and > +bitmap_directory_size. The entries of the bitmap directory have variab= le > +length, depending on the length of the bitmap name and extra data. The= se > +entries are also called bitmap headers. > + > +Structure of a bitmap directory entry: > + > + Byte 0 - 7: bitmap_table_offset > + Offset into the image file at which the bitmap tab= le > + (described below) for the bitmap starts. Must be a= ligned to > + a cluster boundary. > + > + 8 - 11: bitmap_table_size > + Number of entries in the bitmap table of the bitma= p. Should this be the size in bytes, instead of the number of entries? But at least the entries are fixed width of 8 bytes each, so this lets you get a bitmap table up to 32G bytes rather than just 4G in size. (Let's see here - if we have 32G bytes in the bitmap table, that means 4G clusters occupied by the bitmap itself; in the worst case of 512-byte clusters and granularity 0, that is a maximum bitmap size of 2T describing 16T of virtual guest image; with larger cluster size and/or larger granularity, we cover a lot more virtual guest space with less bitmap size; so I guess we aren't too worried about running out of space?= ). > + 20 - 23: extra_data_size > + Size of type-specific extra data. > + > + For now, as no extra data is defined, extra_data_s= ize is > + reserved and must be zero. > + > + variable: Type-specific extra data for the bitmap. I'd write this as: variable: extra_data Type-specific extra data for the bitmap, occupying extra_data_size bytes. > + > + variable: The name of the bitmap (not null terminated). Must= be > + unique among all bitmap names within the bitmaps e= xtension. > + > + variable: Padding to round up the bitmap directory entry siz= e to the > + next multiple of 8. Should we require the padding to be all NUL bytes? (We aren't consistent on whether we require that for other locations of padding in the spec, so that could be a followup patch). > + > +=3D=3D=3D Bitmap table =3D=3D=3D > + > +Bitmaps are stored using a one-level structure (as opposed to two-leve= l > +structure like for refcounts and guest clusters mapping) for the mappi= ng of > +bitmap data to host clusters. This structure is called the bitmap tabl= e. Possible wording tweak: Bitmaps are stored using a one-level structure (as opposed to the two-level structures for refcounts and guest cluster mapping), and are used for the mapping of bitmap data to host clusters > + > +Each bitmap table has a variable size (stored in the bitmap directory = Entry) Does 'Entry' still need to be capitalized? > +and may use multiple clusters, however, it must be contiguous in the i= mage > +file. > + > +Structure of a bitmap table entry: > + > + Bit 0: Reserved and must be zero if bits 9 - 55 are non-z= ero. > + If bits 9 - 55 are zero: > + 0: Cluster should be read as all zeros. > + 1: Cluster should be read as all ones. > + > + 1 - 8: Reserved and must be zero. > + > + 9 - 55: Bits 9 - 55 of the host cluster offset. Must be al= igned to > + a cluster boundary. If the offset is 0, the cluste= r is > + unallocated; in that case, bit 0 determines how th= is > + cluster should be treated when read from. Possible wording tweak: s/when read from/during reads/. > + > + 56 - 63: Reserved and must be zero. > + > +=3D=3D=3D Bitmap data =3D=3D=3D > + > +As noted above, bitmap data is stored in separate clusters, described = by the > +bitmap table. Given an offset (in bytes) into the bitmap data, the off= set into > +the image file can be obtained as follows: > + > + image_offset =3D > + bitmap_table[bitmap_data_offset / cluster_size] + > + (bitmap_data_offset % cluster_size) > + > +This offset is not defined if bits 9 - 55 of bitmap table entry are ze= ro (see > +above). > + > +Given an offset byte_nr into the virtual disk and the bitmap's granula= rity, the > +bit offset into the bitmap can be calculated like this: > + > + bit_offset =3D > + image_offset(byte_nr / granularity / 8) * 8 + > + (byte_nr / granularity) % 8 > + > +If the size of the bitmap data is not a multiply of cluster size then = the last s/multiply of cluster size/multiple of the cluster size,/ > +cluster of the bitmap data contains some unused tail bits. These bits = must be > +zero. >=20 --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --QuPJScAbsLJ3Wh8v7u9EN11SNOB5x1La3 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJWmBxgAAoJEKeha0olJ0Nqe/8H/i/Ad3CfQTzpBsJuLFx1tevQ cUmUYsrLIi2yGjwnVO+0xLUHx7wHVUSIXI6OBIGBnAjlBF9M/D6NPpibxCdFuTc+ 5a/jqPc5RMcC9XpEUIqeDJm4FHgpBycntdqHjdpNgAOFMkRG53XvZ10bOZYuG/WI XOM6ilumh+HUqdFJ8IUwLhWAkfDLYu8/1nfI/u2vvUyluPus0neD1/pR8RelF6PF GPmW1vkBXGhP964Z4wuJVG/+DqX5QttY09/cKi3CCENRM54MW4n8p8m6xapf7H8Z n+9yJADFFPY801XgFcdzPiLONqLoya2Y6StP3eGzOc6kHeDIcnampTy+QIjCagc= =1GkP -----END PGP SIGNATURE----- --QuPJScAbsLJ3Wh8v7u9EN11SNOB5x1La3--