* [Qemu-devel] [PATCH 01/12] hbitmap: serialization
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-08-20 14:53 ` Stefan Hajnoczi
2015-08-07 9:32 ` [Qemu-devel] [PATCH 02/12] block: BdrvDirtyBitmap serialization interface Vladimir Sementsov-Ogievskiy
` (12 subsequent siblings)
13 siblings, 1 reply; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Functions to serialize / deserialize(restore) HBitmap. HBitmap should be
saved to linear sequence of bits independently of endianness and bitmap
array element (unsigned long) size. Therefore Little Endian is chosen.
These functions are appropriate for dirty bitmap migration, restoring
the bitmap in several steps is available. To save performance, every
step writes only the last level of the bitmap. All other levels are
restored by hbitmap_deserialize_finish() as a last step of restoring.
So, HBitmap is inconsistent while restoring.
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/qemu/hbitmap.h | 59 ++++++++++++++++++++++++++++++
util/hbitmap.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 157 insertions(+)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index bb94a00..1da5b35 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -149,6 +149,65 @@ void hbitmap_reset_all(HBitmap *hb);
bool hbitmap_get(const HBitmap *hb, uint64_t item);
/**
+ * hbitmap_data_size:
+ * @hb: HBitmap to operate on.
+ * @count: Number of bits
+ *
+ * Return amount of bytes hbitmap_serialize_part needs
+ */
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count);
+
+/**
+ * hbitmap_serialize_part
+ * @hb: HBitmap to oprate on.
+ * @buf: Buffer to store serialized bitmap.
+ * @start: First bit to store.
+ * @count: Number of bits to store.
+ *
+ * Stores HBitmap data corresponding to given region. The format of saved data
+ * is linear sequence of bits, so it can be used by hbitmap_deserialize_part
+ * independently of endianness and size of HBitmap level array elements
+ */
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_part
+ * @hb: HBitmap to operate on.
+ * @buf: Buffer to restore bitmap data from.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Retores HBitmap data corresponding to given region. The format is the same
+ * as for hbitmap_serialize_part.
+ *
+ * ! The bitmap becomes inconsistent after this operation.
+ * hbitmap_serialize_finish should be called before using the bitmap after
+ * data restoring.
+ */
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_zeroes
+ * @hb: HBitmap to operate on.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Same as hbitmap_serialize_part, but fills the bitmap with zeroes.
+ */
+void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_finish
+ * @hb: HBitmap to operate on.
+ *
+ * Repair HBitmap after calling hbitmap_deserialize_data. Actually, all HBitmap
+ * layers are restored here.
+ */
+void hbitmap_deserialize_finish(HBitmap *hb);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 50b888f..c7c21fe 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -378,6 +378,104 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != 0;
}
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count)
+{
+ uint64_t size, gran;
+
+ if (count == 0) {
+ return 0;
+ }
+
+ gran = 1ll << hb->granularity;
+ size = (((gran + count - 2) >> hb->granularity) >> BITS_PER_LEVEL) + 1;
+
+ return size * sizeof(unsigned long);
+}
+
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+ uint64_t i;
+ uint64_t last = start + count - 1;
+ unsigned long *out = (unsigned long *)buf;
+
+ if (count == 0) {
+ return;
+ }
+
+ start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+ last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+ count = last - start + 1;
+
+ for (i = start; i <= last; ++i) {
+ unsigned long el = hb->levels[HBITMAP_LEVELS - 1][i];
+ out[i - start] =
+ (BITS_PER_LONG == 32 ? cpu_to_le32(el) : cpu_to_le64(el));
+ }
+}
+
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+ uint64_t i;
+ uint64_t last = start + count - 1;
+ unsigned long *in = (unsigned long *)buf;
+
+ if (count == 0) {
+ return;
+ }
+
+ start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+ last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+ count = last - start + 1;
+
+ for (i = start; i <= last; ++i) {
+ hb->levels[HBITMAP_LEVELS - 1][i] =
+ (BITS_PER_LONG == 32 ? le32_to_cpu(in[i - start]) :
+ le64_to_cpu(in[i - start]));
+ }
+}
+
+void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count)
+{
+ uint64_t last = start + count - 1;
+
+ if (count == 0) {
+ return;
+ }
+
+ start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+ last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+ count = last - start + 1;
+
+ memset(hb->levels[HBITMAP_LEVELS - 1] + start, 0,
+ count * sizeof(unsigned long));
+}
+
+void hbitmap_deserialize_finish(HBitmap *bitmap)
+{
+ int64_t i, size, prev_size;
+ int lev;
+
+ /* restore levels starting from penultimate to zero level, assuming
+ * that the last level is ok */
+ size = MAX((bitmap->size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+ for (lev = HBITMAP_LEVELS - 1; lev-- > 0; ) {
+ prev_size = size;
+ size = MAX((size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+ memset(bitmap->levels[lev], 0, size * sizeof(unsigned long));
+
+ for (i = 0; i < prev_size; ++i) {
+ if (bitmap->levels[lev + 1][i]) {
+ bitmap->levels[lev][i >> BITS_PER_LEVEL] |=
+ 1 << (i & (BITS_PER_LONG - 1));
+ }
+ }
+ }
+
+ bitmap->levels[0][0] |= 1UL << (BITS_PER_LONG - 1);
+}
+
void hbitmap_free(HBitmap *hb)
{
unsigned i;
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 01/12] hbitmap: serialization
2015-08-07 9:32 ` [Qemu-devel] [PATCH 01/12] hbitmap: serialization Vladimir Sementsov-Ogievskiy
@ 2015-08-20 14:53 ` Stefan Hajnoczi
0 siblings, 0 replies; 22+ messages in thread
From: Stefan Hajnoczi @ 2015-08-20 14:53 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: kwolf, peter.maydell, quintela, qemu-devel, dgilbert, vsementsov,
den, amit.shah, pbonzini, jsnow
[-- Attachment #1: Type: text/plain, Size: 4378 bytes --]
On Fri, Aug 07, 2015 at 12:32:33PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> +/**
> + * hbitmap_serialize_part
> + * @hb: HBitmap to oprate on.
s/oprate/operate/
> + * @buf: Buffer to store serialized bitmap.
> + * @start: First bit to store.
> + * @count: Number of bits to store.
> + *
> + * Stores HBitmap data corresponding to given region. The format of saved data
> + * is linear sequence of bits, so it can be used by hbitmap_deserialize_part
> + * independently of endianness and size of HBitmap level array elements
These functions *are* dependent of HBitmap level array element size.
They always assign full array elements (unsigned long). If count <
BITS_PER_LONG at some point before the end, the bitmap will be corrupt
because leading bits will be zeroed when the next
hbitmap_deserialize_part() call is made!
> + */
> +void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
> + uint64_t start, uint64_t count);
> +
> +/**
> + * hbitmap_deserialize_part
> + * @hb: HBitmap to operate on.
> + * @buf: Buffer to restore bitmap data from.
> + * @start: First bit to restore.
> + * @count: Number of bits to restore.
> + *
> + * Retores HBitmap data corresponding to given region. The format is the same
s/Retores/Restores/
> + * as for hbitmap_serialize_part.
> + *
> + * ! The bitmap becomes inconsistent after this operation.
> + * hbitmap_serialize_finish should be called before using the bitmap after
> + * data restoring.
> + */
> +void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
> + uint64_t start, uint64_t count);
> +
> +/**
> + * hbitmap_deserialize_zeroes
> + * @hb: HBitmap to operate on.
> + * @start: First bit to restore.
> + * @count: Number of bits to restore.
> + *
> + * Same as hbitmap_serialize_part, but fills the bitmap with zeroes.
> + */
> +void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count);
> +
> +/**
> + * hbitmap_deserialize_finish
> + * @hb: HBitmap to operate on.
> + *
> + * Repair HBitmap after calling hbitmap_deserialize_data. Actually, all HBitmap
> + * layers are restored here.
> + */
> +void hbitmap_deserialize_finish(HBitmap *hb);
> +
> +/**
> * hbitmap_free:
> * @hb: HBitmap to operate on.
> *
> diff --git a/util/hbitmap.c b/util/hbitmap.c
> index 50b888f..c7c21fe 100644
> --- a/util/hbitmap.c
> +++ b/util/hbitmap.c
> @@ -378,6 +378,104 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
> return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != 0;
> }
>
> +uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count)
> +{
> + uint64_t size, gran;
> +
> + if (count == 0) {
> + return 0;
> + }
> +
> + gran = 1ll << hb->granularity;
> + size = (((gran + count - 2) >> hb->granularity) >> BITS_PER_LEVEL) + 1;
> +
> + return size * sizeof(unsigned long);
> +}
> +
> +void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
> + uint64_t start, uint64_t count)
> +{
> + uint64_t i;
> + uint64_t last = start + count - 1;
> + unsigned long *out = (unsigned long *)buf;
I'm not sure if we care but this can lead to unaligned stores if buf
isn't aligned to sizeof(unsigned long). Unaligned stores are best
avoided:
https://www.linux-mips.org/wiki/Alignment
If you replace out[i - start] = ... with a memcpy() call then the
alignment problem is solved. If you are worried that all these memcpy()
calls are slow, check the compiler output since gcc probably optimizes
away the memcpy().
> +
> + if (count == 0) {
> + return;
> + }
> +
> + start = (start >> hb->granularity) >> BITS_PER_LEVEL;
> + last = (last >> hb->granularity) >> BITS_PER_LEVEL;
> + count = last - start + 1;
> +
> + for (i = start; i <= last; ++i) {
> + unsigned long el = hb->levels[HBITMAP_LEVELS - 1][i];
> + out[i - start] =
> + (BITS_PER_LONG == 32 ? cpu_to_le32(el) : cpu_to_le64(el));
> + }
> +}
> +
> +void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
> + uint64_t start, uint64_t count)
> +{
> + uint64_t i;
> + uint64_t last = start + count - 1;
> + unsigned long *in = (unsigned long *)buf;
Same here.
[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 02/12] block: BdrvDirtyBitmap serialization interface
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 01/12] hbitmap: serialization Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 03/12] block: tiny refactoring: minimize hbitmap_(set/reset) usage Vladimir Sementsov-Ogievskiy
` (11 subsequent siblings)
13 siblings, 0 replies; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Several functions to provide necessary access to BdrvDirtyBitmap for
block-migration.c
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block.c | 36 ++++++++++++++++++++++++++++++++++++
include/block/block.h | 13 +++++++++++++
2 files changed, 49 insertions(+)
diff --git a/block.c b/block.c
index d088ee0..8937296 100644
--- a/block.c
+++ b/block.c
@@ -3572,6 +3572,42 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
hbitmap_reset_all(bitmap->bitmap);
}
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
+{
+ return bitmap->name;
+}
+
+uint64_t bdrv_dirty_bitmap_data_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t count)
+{
+ return hbitmap_data_size(bitmap->bitmap, count);
+}
+
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count)
+{
+ hbitmap_serialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count)
+{
+ hbitmap_deserialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count)
+{
+ hbitmap_deserialize_zeroes(bitmap->bitmap, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap)
+{
+ hbitmap_deserialize_finish(bitmap->bitmap);
+}
+
void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
int nr_sectors)
{
diff --git a/include/block/block.h b/include/block/block.h
index 37916f7..b82f957 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -508,6 +508,19 @@ void bdrv_dirty_iter_init(BdrvDirtyBitmap *bitmap, struct HBitmapIter *hbi);
void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap);
+uint64_t bdrv_dirty_bitmap_data_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t count);
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count);
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count);
+void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count);
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 03/12] block: tiny refactoring: minimize hbitmap_(set/reset) usage
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 01/12] hbitmap: serialization Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 02/12] block: BdrvDirtyBitmap serialization interface Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 04/12] block: add meta bitmaps Vladimir Sementsov-Ogievskiy
` (10 subsequent siblings)
13 siblings, 0 replies; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/block.c b/block.c
index 8937296..7a7307c 100644
--- a/block.c
+++ b/block.c
@@ -3616,7 +3616,7 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
if (!bdrv_dirty_bitmap_enabled(bitmap)) {
continue;
}
- hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+ bdrv_set_dirty_bitmap(bitmap, cur_sector, nr_sectors);
}
}
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 04/12] block: add meta bitmaps
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (2 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 03/12] block: tiny refactoring: minimize hbitmap_(set/reset) usage Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 05/12] block: add bdrv_next_dirty_bitmap() Vladimir Sementsov-Ogievskiy
` (9 subsequent siblings)
13 siblings, 0 replies; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks
changes (set/unset) of this BdrvDirtyBitmap. It is needed for live
migration of block dirty bitmaps.
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
include/block/block.h | 7 +++++++
2 files changed, 51 insertions(+)
diff --git a/block.c b/block.c
index 7a7307c..c954fc1 100644
--- a/block.c
+++ b/block.c
@@ -61,9 +61,15 @@
* (3) successor is set: frozen mode.
* A frozen bitmap cannot be renamed, deleted, anonymized, cleared, set,
* or enabled. A frozen bitmap can only abdicate() or reclaim().
+ *
+ * Meta bitmap:
+ * Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks changes
+ * (set/unset) of this BdrvDirtyBitmap. It is needed for live migration of
+ * block dirty bitmaps.
*/
struct BdrvDirtyBitmap {
HBitmap *bitmap; /* Dirty sector bitmap implementation */
+ HBitmap *meta_bitmap; /* Meta bitmap */
BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap (Number of sectors) */
@@ -3312,6 +3318,35 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
bitmap->name = NULL;
}
+HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
+ uint64_t chunk_size)
+{
+ uint64_t sector_granularity;
+
+ assert((chunk_size & (chunk_size - 1)) == 0);
+
+ /* one chunk is corresponding to one bit of the meta bitmap, and each bit
+ * of the chunk is corresponding to 'bdrv_dirty_bitmap_granularity(bitmap)'
+ * bytes of the node */
+ sector_granularity =
+ (chunk_size * 8 * bdrv_dirty_bitmap_granularity(bitmap))
+ >> BDRV_SECTOR_BITS;
+ assert(sector_granularity);
+
+ bitmap->meta_bitmap =
+ hbitmap_alloc(bitmap->size, ffsll(sector_granularity) - 1);
+
+ return bitmap->meta_bitmap;
+}
+
+void bdrv_release_meta_bitmap(BdrvDirtyBitmap *bitmap)
+{
+ if (bitmap->meta_bitmap) {
+ hbitmap_free(bitmap->meta_bitmap);
+ bitmap->meta_bitmap = NULL;
+ }
+}
+
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
uint32_t granularity,
const char *name,
@@ -3472,6 +3507,9 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
assert(!bdrv_dirty_bitmap_frozen(bm));
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
+ if (bitmap->meta_bitmap) {
+ hbitmap_free(bitmap->meta_bitmap);
+ }
g_free(bitmap->name);
g_free(bitmap);
return;
@@ -3557,6 +3595,9 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
{
assert(bdrv_dirty_bitmap_enabled(bitmap));
hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+ if (bitmap->meta_bitmap) {
+ hbitmap_set(bitmap->meta_bitmap, cur_sector, nr_sectors);
+ }
}
void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
@@ -3564,6 +3605,9 @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
{
assert(bdrv_dirty_bitmap_enabled(bitmap));
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+ if (bitmap->meta_bitmap) {
+ hbitmap_set(bitmap->meta_bitmap, cur_sector, nr_sectors);
+ }
}
void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
diff --git a/include/block/block.h b/include/block/block.h
index b82f957..54930d5 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -4,6 +4,7 @@
#include "block/aio.h"
#include "qemu-common.h"
#include "qemu/option.h"
+#include "qemu/hbitmap.h"
#include "block/coroutine.h"
#include "block/accounting.h"
#include "qapi/qmp/qobject.h"
@@ -521,6 +522,12 @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
uint64_t start, uint64_t count);
void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+/* chunk size here is number of bytes of the @bitmap data per one bit of the
+ * meta bitmap being created */
+HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
+ uint64_t granularity);
+void bdrv_release_meta_bitmap(BdrvDirtyBitmap *bitmap);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 05/12] block: add bdrv_next_dirty_bitmap()
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (3 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 04/12] block: add meta bitmaps Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 06/12] qapi: add dirty-bitmaps migration capability Vladimir Sementsov-Ogievskiy
` (8 subsequent siblings)
13 siblings, 0 replies; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Like bdrv_next() - bdrv_next_dirty_bitmap() is a function to provide
access to private dirty bitmaps list.
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block.c | 10 ++++++++++
include/block/block.h | 2 ++
2 files changed, 12 insertions(+)
diff --git a/block.c b/block.c
index c954fc1..d9da767 100644
--- a/block.c
+++ b/block.c
@@ -3551,6 +3551,16 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs)
return list;
}
+BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs,
+ BdrvDirtyBitmap *bitmap)
+{
+ if (bitmap == NULL) {
+ return QLIST_FIRST(&bs->dirty_bitmaps);
+ }
+
+ return QLIST_NEXT(bitmap, list);
+}
+
int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, int64_t sector)
{
if (bitmap) {
diff --git a/include/block/block.h b/include/block/block.h
index 54930d5..edc1510 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -521,6 +521,8 @@ void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
uint64_t start, uint64_t count);
void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs,
+ BdrvDirtyBitmap *bitmap);
/* chunk size here is number of bytes of the @bitmap data per one bit of the
* meta bitmap being created */
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 06/12] qapi: add dirty-bitmaps migration capability
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (4 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 05/12] block: add bdrv_next_dirty_bitmap() Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-09-15 16:27 ` Eric Blake
2015-08-07 9:32 ` [Qemu-devel] [PATCH 07/12] migration/qemu-file: add qemu_put_counted_string() Vladimir Sementsov-Ogievskiy
` (7 subsequent siblings)
13 siblings, 1 reply; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/migration/migration.h | 1 +
migration/migration.c | 9 +++++++++
qapi-schema.json | 4 +++-
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/include/migration/migration.h b/include/migration/migration.h
index 8334621..1dc4e25 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -161,6 +161,7 @@ void migrate_add_blocker(Error *reason);
void migrate_del_blocker(Error *reason);
bool migrate_zero_blocks(void);
+bool migrate_dirty_bitmaps(void);
bool migrate_auto_converge(void);
diff --git a/migration/migration.c b/migration/migration.c
index 662e77e..0016ee1 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -886,6 +886,15 @@ int migrate_decompress_threads(void)
return s->parameters[MIGRATION_PARAMETER_DECOMPRESS_THREADS];
}
+bool migrate_dirty_bitmaps(void)
+{
+ MigrationState *s;
+
+ s = migrate_get_current();
+
+ return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTY_BITMAPS];
+}
+
bool migrate_use_events(void)
{
MigrationState *s;
diff --git a/qapi-schema.json b/qapi-schema.json
index 4342a08..80d577f 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -529,11 +529,13 @@
# @auto-converge: If enabled, QEMU will automatically throttle down the guest
# to speed up convergence of RAM migration. (since 1.6)
#
+# @dirty-bitmaps: If enabled, QEMU will migrate named dirty bitmaps. (since 2.4)
+#
# Since: 1.2
##
{ 'enum': 'MigrationCapability',
'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks',
- 'compress', 'events'] }
+ 'compress', 'events', 'dirty-bitmaps'] }
##
# @MigrationCapabilityStatus
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 06/12] qapi: add dirty-bitmaps migration capability
2015-08-07 9:32 ` [Qemu-devel] [PATCH 06/12] qapi: add dirty-bitmaps migration capability Vladimir Sementsov-Ogievskiy
@ 2015-09-15 16:27 ` Eric Blake
0 siblings, 0 replies; 22+ messages in thread
From: Eric Blake @ 2015-09-15 16:27 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: kwolf, peter.maydell, quintela, dgilbert, vsementsov, stefanha,
pbonzini, amit.shah, den, jsnow
[-- Attachment #1: Type: text/plain, Size: 926 bytes --]
On 08/07/2015 03:32 AM, Vladimir Sementsov-Ogievskiy wrote:
> From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
>
> Reviewed-by: John Snow <jsnow@redhat.com>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> include/migration/migration.h | 1 +
> migration/migration.c | 9 +++++++++
> qapi-schema.json | 4 +++-
> 3 files changed, 13 insertions(+), 1 deletion(-)
>
> +++ b/qapi-schema.json
> @@ -529,11 +529,13 @@
> # @auto-converge: If enabled, QEMU will automatically throttle down the guest
> # to speed up convergence of RAM migration. (since 1.6)
> #
> +# @dirty-bitmaps: If enabled, QEMU will migrate named dirty bitmaps. (since 2.4)
2.5 now
With that fixed,
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 07/12] migration/qemu-file: add qemu_put_counted_string()
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (5 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 06/12] qapi: add dirty-bitmaps migration capability Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-09-11 0:02 ` John Snow
2015-08-07 9:32 ` [Qemu-devel] [PATCH 08/12] migration: add migration/block-dirty-bitmap.c Vladimir Sementsov-Ogievskiy
` (6 subsequent siblings)
13 siblings, 1 reply; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
Add function opposite to qemu_get_counted_string.
qemu_put_counted_string puts one-byte length of the string (string
should not be longer than 255 characters), and then it puts the string,
without last zero byte.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/migration/qemu-file.h | 2 ++
migration/qemu-file.c | 9 +++++++++
2 files changed, 11 insertions(+)
diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index ea49f33..cfbbdcb 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -319,4 +319,6 @@ static inline void qemu_get_sbe64s(QEMUFile *f, int64_t *pv)
size_t qemu_get_counted_string(QEMUFile *f, char buf[256]);
+void qemu_put_counted_string(QEMUFile *f, const char *name);
+
#endif
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 6bb3dc1..206cb54 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -611,3 +611,12 @@ size_t qemu_get_counted_string(QEMUFile *f, char buf[256])
return res == len ? res : 0;
}
+
+void qemu_put_counted_string(QEMUFile *f, const char *name)
+{
+ int len = strlen(name);
+
+ assert(len < 256);
+ qemu_put_byte(f, len);
+ qemu_put_buffer(f, (const uint8_t *)name, len);
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 07/12] migration/qemu-file: add qemu_put_counted_string()
2015-08-07 9:32 ` [Qemu-devel] [PATCH 07/12] migration/qemu-file: add qemu_put_counted_string() Vladimir Sementsov-Ogievskiy
@ 2015-09-11 0:02 ` John Snow
0 siblings, 0 replies; 22+ messages in thread
From: John Snow @ 2015-09-11 0:02 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: kwolf, peter.maydell, quintela, dgilbert, vsementsov, stefanha,
pbonzini, amit.shah, den
On 08/07/2015 05:32 AM, Vladimir Sementsov-Ogievskiy wrote:
> Add function opposite to qemu_get_counted_string.
> qemu_put_counted_string puts one-byte length of the string (string
> should not be longer than 255 characters), and then it puts the string,
> without last zero byte.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> include/migration/qemu-file.h | 2 ++
> migration/qemu-file.c | 9 +++++++++
> 2 files changed, 11 insertions(+)
>
> diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
> index ea49f33..cfbbdcb 100644
> --- a/include/migration/qemu-file.h
> +++ b/include/migration/qemu-file.h
> @@ -319,4 +319,6 @@ static inline void qemu_get_sbe64s(QEMUFile *f, int64_t *pv)
>
> size_t qemu_get_counted_string(QEMUFile *f, char buf[256]);
>
> +void qemu_put_counted_string(QEMUFile *f, const char *name);
> +
> #endif
> diff --git a/migration/qemu-file.c b/migration/qemu-file.c
> index 6bb3dc1..206cb54 100644
> --- a/migration/qemu-file.c
> +++ b/migration/qemu-file.c
> @@ -611,3 +611,12 @@ size_t qemu_get_counted_string(QEMUFile *f, char buf[256])
>
> return res == len ? res : 0;
> }
> +
> +void qemu_put_counted_string(QEMUFile *f, const char *name)
> +{
> + int len = strlen(name);
> +
> + assert(len < 256);
> + qemu_put_byte(f, len);
> + qemu_put_buffer(f, (const uint8_t *)name, len);
> +}
>
Reviewed-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 08/12] migration: add migration/block-dirty-bitmap.c
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (6 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 07/12] migration/qemu-file: add qemu_put_counted_string() Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-09-11 0:10 ` John Snow
2015-08-07 9:32 ` [Qemu-devel] [PATCH 09/12] iotests: maintain several vms in test Vladimir Sementsov-Ogievskiy
` (5 subsequent siblings)
13 siblings, 1 reply; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Live migration of dirty bitmaps. Only named dirty bitmaps, associated with
root nodes and non-root named nodes are migrated.
If destination qemu is already containing a dirty bitmap with the same name
as a migrated bitmap (for the same node), than, if their granularities are
the same the migration will be done, otherwise the error will be generated.
If destination qemu doesn't contain such bitmap it will be created.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/migration/block.h | 1 +
migration/Makefile.objs | 2 +-
migration/block-dirty-bitmap.c | 699 +++++++++++++++++++++++++++++++++++++++++
vl.c | 1 +
4 files changed, 702 insertions(+), 1 deletion(-)
create mode 100644 migration/block-dirty-bitmap.c
diff --git a/include/migration/block.h b/include/migration/block.h
index ffa8ac0..566bb9f 100644
--- a/include/migration/block.h
+++ b/include/migration/block.h
@@ -14,6 +14,7 @@
#ifndef BLOCK_MIGRATION_H
#define BLOCK_MIGRATION_H
+void dirty_bitmap_mig_init(void);
void blk_mig_init(void);
int blk_mig_active(void);
uint64_t blk_mig_bytes_transferred(void);
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index d929e96..128612d 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -6,5 +6,5 @@ common-obj-y += xbzrle.o
common-obj-$(CONFIG_RDMA) += rdma.o
common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o
-common-obj-y += block.o
+common-obj-y += block.o block-dirty-bitmap.o
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
new file mode 100644
index 0000000..4095772
--- /dev/null
+++ b/migration/block-dirty-bitmap.c
@@ -0,0 +1,699 @@
+/*
+ * QEMU dirty bitmap migration
+ *
+ * Live migration of dirty bitmaps. Only named dirty bitmaps, associated with
+ * root nodes and non-root named nodes are migrated. Live iteration is disabled
+ * for small data amount (see MIN_LIVE_SIZE).
+ *
+ * If destination qemu is already containing a dirty bitmap with the same name
+ * as a migrated bitmap (for the same node), than, if their granularities are
+ * the same the migration will be done, otherwise the error will be generated.
+ *
+ * If destination qemu doesn't contain such bitmap it will be created.
+ *
+ * format of migration:
+ *
+ * # Header (shared for different chunk types)
+ * 1, 2 or 4 bytes: flags (see qemu_{put,put}_flags)
+ * [ 1 byte: node name size ] \ flags & DEVICE_NAME
+ * [ n bytes: node name ] /
+ * [ 1 byte: bitmap name size ] \ flags & BITMAP_NAME
+ * [ n bytes: bitmap name ] /
+ *
+ * # Start of bitmap migration (flags & START)
+ * header
+ * be64: granularity
+ *
+ * # Complete of bitmap migration (flags & COMPLETE)
+ * header
+ * 1 byte: bitmap enabled flag
+ *
+ * # Data chunk of bitmap migration
+ * header
+ * be64: start sector
+ * be32: number of sectors
+ * [ be64: buffer size ] \ ! (flags & ZEROES)
+ * [ n bytes: buffer ] /
+ *
+ * The last chunk in stream should contain flags & EOS. The chunk may skip
+ * device and/or bitmap names, assuming them to be the same with the previous
+ * chunk.
+ *
+ *
+ * This file is derived from migration/block.c
+ *
+ * Author:
+ * Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
+ *
+ * original copyright message:
+ * =====================================================================
+ * Copyright IBM, Corp. 2009
+ *
+ * Authors:
+ * Liran Schour <lirans@il.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ *
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
+ * =====================================================================
+ */
+
+#include "block/block.h"
+#include "block/block_int.h"
+#include "sysemu/block-backend.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "migration/block.h"
+#include "migration/migration.h"
+#include "qemu/hbitmap.h"
+#include <assert.h>
+
+#define CHUNK_SIZE (1 << 10)
+#define MIN_LIVE_SIZE (1 << 20)
+
+/* Flags occupy from one to four bytes. In all but one the 7-th (EXTRA_FLAGS)
+ * bit should be set. */
+#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01
+#define DIRTY_BITMAP_MIG_FLAG_ZEROES 0x02
+#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x04
+#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x08
+#define DIRTY_BITMAP_MIG_FLAG_START 0x10
+#define DIRTY_BITMAP_MIG_FLAG_COMPLETE 0x20
+#define DIRTY_BITMAP_MIG_FLAG_BITS 0x40
+
+#define DIRTY_BITMAP_MIG_EXTRA_FLAGS 0x80
+#define DIRTY_BITMAP_MIG_FLAGS_SIZE_16 0x8000
+#define DIRTY_BITMAP_MIG_FLAGS_SIZE_32 0x8080
+
+#define DEBUG_DIRTY_BITMAP_MIGRATION 0
+
+#define DPRINTF(fmt, args...) \
+ do { \
+ if (DEBUG_DIRTY_BITMAP_MIGRATION) { \
+ printf("DMIG %s:%d", __func__, __LINE__); \
+ printf(fmt, ##args); \
+ } \
+ } while (0)
+
+typedef struct DirtyBitmapMigBitmapState {
+ /* Written during setup phase. */
+ BlockDriverState *bs;
+ const char *node_name;
+ BdrvDirtyBitmap *bitmap;
+ HBitmap *meta_bitmap;
+ uint64_t total_sectors;
+ uint64_t sectors_per_chunk;
+ QSIMPLEQ_ENTRY(DirtyBitmapMigBitmapState) entry;
+
+ /* For bulk phase. */
+ bool bulk_completed;
+ uint64_t cur_sector;
+
+ /* For dirty phase. */
+ HBitmapIter iter_dirty;
+} DirtyBitmapMigBitmapState;
+
+typedef struct DirtyBitmapMigState {
+ QSIMPLEQ_HEAD(dbms_list, DirtyBitmapMigBitmapState) dbms_list;
+
+ bool bulk_completed;
+ bool is_live_iterative;
+
+ /* for send_bitmap_bits() */
+ BlockDriverState *prev_bs;
+ BdrvDirtyBitmap *prev_bitmap;
+} DirtyBitmapMigState;
+
+typedef struct DirtyBitmapLoadState {
+ uint32_t flags;
+ char node_name[256];
+ char bitmap_name[256];
+ BlockDriverState *bs;
+ BdrvDirtyBitmap *bitmap;
+} DirtyBitmapLoadState;
+
+static DirtyBitmapMigState dirty_bitmap_mig_state;
+
+static uint32_t qemu_get_flags(QEMUFile *f)
+{
+ uint8_t flags = qemu_get_byte(f);
+ if (flags & DIRTY_BITMAP_MIG_EXTRA_FLAGS) {
+ flags = flags << 8 | qemu_get_byte(f);
+ if (flags & DIRTY_BITMAP_MIG_EXTRA_FLAGS) {
+ flags = flags << 16 | qemu_get_be16(f);
+ }
+ }
+
+ return flags;
+}
+
+static void qemu_put_flags(QEMUFile *f, uint32_t flags)
+{
+ if (!(flags & 0xffffff00)) {
+ qemu_put_byte(f, flags);
+ return;
+ }
+
+ if (!(flags & 0xffff0000)) {
+ qemu_put_be16(f, flags | DIRTY_BITMAP_MIG_FLAGS_SIZE_16);
+ return;
+ }
+
+ qemu_put_be32(f, flags | DIRTY_BITMAP_MIG_FLAGS_SIZE_32);
+}
+
+static void send_bitmap_header(QEMUFile *f, DirtyBitmapMigBitmapState *dbms,
+ uint32_t additional_flags)
+{
+ BlockDriverState *bs = dbms->bs;
+ BdrvDirtyBitmap *bitmap = dbms->bitmap;
+ uint32_t flags = additional_flags;
+
+ if (bs != dirty_bitmap_mig_state.prev_bs) {
+ dirty_bitmap_mig_state.prev_bs = bs;
+ flags |= DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME;
+ }
+
+ if (bitmap != dirty_bitmap_mig_state.prev_bitmap) {
+ dirty_bitmap_mig_state.prev_bitmap = bitmap;
+ flags |= DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME;
+ }
+
+ qemu_put_flags(f, flags);
+
+ if (flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) {
+ qemu_put_counted_string(f, dbms->node_name);
+ }
+
+ if (flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) {
+ qemu_put_counted_string(f, bdrv_dirty_bitmap_name(bitmap));
+ }
+}
+
+static void send_bitmap_start(QEMUFile *f, DirtyBitmapMigBitmapState *dbms)
+{
+ send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_START);
+ qemu_put_be32(f, bdrv_dirty_bitmap_granularity(dbms->bitmap));
+}
+
+static void send_bitmap_complete(QEMUFile *f, DirtyBitmapMigBitmapState *dbms)
+{
+ send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_COMPLETE);
+ qemu_put_byte(f, bdrv_dirty_bitmap_enabled(dbms->bitmap));
+}
+
+static void send_bitmap_bits(QEMUFile *f, DirtyBitmapMigBitmapState *dbms,
+ uint64_t start_sector, uint32_t nr_sectors)
+{
+ /* align for buffer_is_zero() */
+ uint64_t align = 4 * sizeof(long);
+ uint64_t buf_size =
+ (bdrv_dirty_bitmap_data_size(dbms->bitmap, nr_sectors) + align - 1) &
+ ~(align - 1);
+ uint8_t *buf = g_malloc0(buf_size);
+ uint32_t flags = DIRTY_BITMAP_MIG_FLAG_BITS;
+
+ bdrv_dirty_bitmap_serialize_part(dbms->bitmap, buf,
+ start_sector, nr_sectors);
+
+ if (buffer_is_zero(buf, buf_size)) {
+ g_free(buf);
+ buf = NULL;
+ flags |= DIRTY_BITMAP_MIG_FLAG_ZEROES;
+ }
+
+ DPRINTF("parameters:"
+ "\n flags: %x"
+ "\n start_sector: %" PRIu64
+ "\n nr_sectors: %" PRIu32
+ "\n data_size: %" PRIu64 "\n",
+ flags, start_sector, nr_sectors, buf_size);
+
+ send_bitmap_header(f, dbms, flags);
+
+ qemu_put_be64(f, start_sector);
+ qemu_put_be32(f, nr_sectors);
+
+ /* if a block is zero we need to flush here since the network
+ * bandwidth is now a lot higher than the storage device bandwidth.
+ * thus if we queue zero blocks we slow down the migration.
+ * also, skip writing block when migrate only dirty bitmaps. */
+ if (flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) {
+ qemu_fflush(f);
+ return;
+ }
+
+ qemu_put_be64(f, buf_size);
+ qemu_put_buffer(f, buf, buf_size);
+ g_free(buf);
+}
+
+
+/* Called with iothread lock taken. */
+
+static void set_dirty_tracking(void)
+{
+ DirtyBitmapMigBitmapState *dbms;
+
+ QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+ dbms->meta_bitmap =
+ bdrv_create_meta_bitmap(dbms->bitmap, CHUNK_SIZE);
+ dbms->sectors_per_chunk =
+ UINT64_C(1) << hbitmap_granularity(dbms->meta_bitmap);
+ }
+}
+
+static void unset_dirty_tracking(void)
+{
+ DirtyBitmapMigBitmapState *dbms;
+
+ QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+ bdrv_release_meta_bitmap(dbms->bitmap);
+ }
+}
+
+static void init_dirty_bitmap_migration(void)
+{
+ BlockDriverState *bs;
+ BdrvDirtyBitmap *bitmap;
+ DirtyBitmapMigBitmapState *dbms;
+ uint64_t total_bytes = 0;
+
+ dirty_bitmap_mig_state.bulk_completed = false;
+ dirty_bitmap_mig_state.prev_bs = NULL;
+ dirty_bitmap_mig_state.prev_bitmap = NULL;
+
+ for (bs = bdrv_next(NULL); bs; bs = bdrv_next(bs)) {
+ for (bitmap = bdrv_next_dirty_bitmap(bs, NULL); bitmap;
+ bitmap = bdrv_next_dirty_bitmap(bs, bitmap)) {
+ if (!bdrv_dirty_bitmap_name(bitmap)) {
+ continue;
+ }
+
+ if (!bdrv_get_node_name(bs) && blk_bs(bs->blk) != bs) {
+ /* not named non-root node */
+ continue;
+ }
+
+ dbms = g_new0(DirtyBitmapMigBitmapState, 1);
+ dbms->bs = bs;
+ dbms->node_name = bdrv_get_node_name(bs);
+ if (!dbms->node_name || dbms->node_name[0] == '\0') {
+ dbms->node_name = bdrv_get_device_name(bs);
+ }
+ dbms->bitmap = bitmap;
+ dbms->total_sectors = bdrv_nb_sectors(bs);
+ total_bytes +=
+ bdrv_dirty_bitmap_data_size(bitmap, dbms->total_sectors);
+
+ QSIMPLEQ_INSERT_TAIL(&dirty_bitmap_mig_state.dbms_list,
+ dbms, entry);
+ }
+ }
+
+ dirty_bitmap_mig_state.is_live_iterative = total_bytes > MIN_LIVE_SIZE;
+}
+
+/* Called with no lock taken. */
+static void bulk_phase_send_chunk(QEMUFile *f, DirtyBitmapMigBitmapState *dbms)
+{
+ uint32_t nr_sectors = MIN(dbms->total_sectors - dbms->cur_sector,
+ dbms->sectors_per_chunk);
+
+ send_bitmap_bits(f, dbms, dbms->cur_sector, nr_sectors);
+
+ dbms->cur_sector += nr_sectors;
+ if (dbms->cur_sector >= dbms->total_sectors) {
+ dbms->bulk_completed = true;
+ }
+}
+
+/* Called with no lock taken. */
+static void bulk_phase(QEMUFile *f, bool limit)
+{
+ DirtyBitmapMigBitmapState *dbms;
+
+ QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+ while (!dbms->bulk_completed) {
+ bulk_phase_send_chunk(f, dbms);
+ if (limit && qemu_file_rate_limit(f)) {
+ return;
+ }
+ }
+ }
+
+ dirty_bitmap_mig_state.bulk_completed = true;
+}
+
+static void blk_mig_reset_dirty_cursor(void)
+{
+ DirtyBitmapMigBitmapState *dbms;
+
+ QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+ hbitmap_iter_init(&dbms->iter_dirty, dbms->meta_bitmap, 0);
+ }
+}
+
+/* Called with iothread lock taken. */
+static bool dirty_phase_send_chunk(QEMUFile *f, DirtyBitmapMigBitmapState *dbms)
+{
+ uint32_t nr_sectors;
+ size_t old_pos = dbms->iter_dirty.pos;
+ int64_t cur = hbitmap_iter_next(&dbms->iter_dirty);
+
+ /* restart search from the beginning */
+ if (old_pos && cur == -1) {
+ hbitmap_iter_init(&dbms->iter_dirty, dbms->meta_bitmap, 0);
+ cur = hbitmap_iter_next(&dbms->iter_dirty);
+ }
+
+ if (cur == -1) {
+ hbitmap_iter_init(&dbms->iter_dirty, dbms->meta_bitmap, 0);
+ return false;
+ }
+
+ nr_sectors = MIN(dbms->total_sectors - cur, dbms->sectors_per_chunk);
+ send_bitmap_bits(f, dbms, cur, nr_sectors);
+ hbitmap_reset(dbms->meta_bitmap, cur, dbms->sectors_per_chunk);
+
+ return true;
+}
+
+/* Called with iothread lock taken. */
+static void dirty_phase(QEMUFile *f, bool limit)
+{
+ DirtyBitmapMigBitmapState *dbms;
+
+ QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+ while (dirty_phase_send_chunk(f, dbms)) {
+ if (limit && qemu_file_rate_limit(f)) {
+ return;
+ }
+ }
+ }
+}
+
+
+/* Called with iothread lock taken. */
+static void dirty_bitmap_mig_cleanup(void)
+{
+ DirtyBitmapMigBitmapState *dbms;
+
+ unset_dirty_tracking();
+
+ while ((dbms = QSIMPLEQ_FIRST(&dirty_bitmap_mig_state.dbms_list)) != NULL) {
+ QSIMPLEQ_REMOVE_HEAD(&dirty_bitmap_mig_state.dbms_list, entry);
+ g_free(dbms);
+ }
+}
+
+static void dirty_bitmap_migration_cancel(void *opaque)
+{
+ dirty_bitmap_mig_cleanup();
+}
+
+static int dirty_bitmap_save_iterate(QEMUFile *f, void *opaque)
+{
+ DPRINTF("enter\n");
+
+ if (dirty_bitmap_mig_state.bulk_completed) {
+ qemu_mutex_lock_iothread();
+ dirty_phase(f, true);
+ qemu_mutex_unlock_iothread();
+ } else {
+ bulk_phase(f, true);
+ }
+
+ qemu_put_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS);
+
+ return dirty_bitmap_mig_state.bulk_completed;
+}
+
+/* Called with iothread lock taken. */
+
+static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque)
+{
+ DirtyBitmapMigBitmapState *dbms;
+ DPRINTF("enter\n");
+
+ if (!dirty_bitmap_mig_state.bulk_completed) {
+ bulk_phase(f, false);
+ }
+
+ blk_mig_reset_dirty_cursor();
+ dirty_phase(f, false);
+
+ QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+ send_bitmap_complete(f, dbms);
+ }
+
+ qemu_put_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS);
+
+ DPRINTF("Dirty bitmaps migration completed\n");
+
+ dirty_bitmap_mig_cleanup();
+ return 0;
+}
+
+static uint64_t dirty_bitmap_save_pending(QEMUFile *f, void *opaque,
+ uint64_t max_size)
+{
+ DirtyBitmapMigBitmapState *dbms;
+ uint64_t pending = 0;
+
+ qemu_mutex_lock_iothread();
+
+ QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+ uint64_t sectors = hbitmap_count(dbms->meta_bitmap);
+ if (!dbms->bulk_completed) {
+ sectors += dbms->total_sectors - dbms->cur_sector;
+ }
+ pending += bdrv_dirty_bitmap_data_size(dbms->bitmap, sectors);
+ }
+
+ qemu_mutex_unlock_iothread();
+
+ DPRINTF("pending %" PRIu64 ", max: %" PRIu64 "\n",
+ pending, max_size);
+ return pending;
+}
+
+/* First occurrence of this bitmap. It should be created if doesn't exist */
+static int dirty_bitmap_load_start(QEMUFile *f, DirtyBitmapLoadState *s)
+{
+ uint32_t granularity = qemu_get_be32(f);
+ if (!s->bitmap) {
+ Error *local_err = NULL;
+ s->bitmap = bdrv_create_dirty_bitmap(s->bs, granularity,
+ s->bitmap_name, &local_err);
+ if (!s->bitmap) {
+ error_report("%s", error_get_pretty(local_err));
+ error_free(local_err);
+ return -EINVAL;
+ }
+ } else {
+ uint32_t dest_granularity =
+ bdrv_dirty_bitmap_granularity(s->bitmap);
+ if (dest_granularity != granularity) {
+ fprintf(stderr,
+ "Error: "
+ "Migrated bitmap granularity (%" PRIu32 ") "
+ "doesn't match the destination bitmap '%s' "
+ "granularity (%" PRIu32 ")\n",
+ granularity,
+ bdrv_dirty_bitmap_name(s->bitmap),
+ dest_granularity);
+ return -EINVAL;
+ }
+ }
+
+ bdrv_disable_dirty_bitmap(s->bitmap);
+
+ return 0;
+}
+
+static void dirty_bitmap_load_complete(QEMUFile *f, DirtyBitmapLoadState *s)
+{
+ bool enabled;
+
+ bdrv_dirty_bitmap_deserialize_finish(s->bitmap);
+
+ enabled = qemu_get_byte(f);
+ if (enabled) {
+ bdrv_enable_dirty_bitmap(s->bitmap);
+ }
+}
+
+static int dirty_bitmap_load_bits(QEMUFile *f, DirtyBitmapLoadState *s)
+{
+ uint64_t first_sector = qemu_get_be64(f);
+ uint32_t nr_sectors = qemu_get_be32(f);
+ DPRINTF("chunk: %" PRIu64 " %" PRIu32 "\n", first_sector, nr_sectors);
+
+
+ if (s->flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) {
+ DPRINTF(" - zeroes\n");
+ bdrv_dirty_bitmap_deserialize_zeroes(s->bitmap, first_sector,
+ nr_sectors);
+ } else {
+ uint8_t *buf;
+ uint64_t buf_size = qemu_get_be64(f);
+ uint64_t needed_size =
+ bdrv_dirty_bitmap_data_size(s->bitmap, nr_sectors);
+
+ if (needed_size > buf_size) {
+ fprintf(stderr,
+ "Error: Migrated bitmap granularity doesn't "
+ "match the destination bitmap '%s' granularity\n",
+ bdrv_dirty_bitmap_name(s->bitmap));
+ return -EINVAL;
+ }
+
+ buf = g_malloc(buf_size);
+ qemu_get_buffer(f, buf, buf_size);
+ bdrv_dirty_bitmap_deserialize_part(s->bitmap, buf,
+ first_sector,
+ nr_sectors);
+ g_free(buf);
+ }
+
+ return 0;
+}
+
+static int dirty_bitmap_load_header(QEMUFile *f, DirtyBitmapLoadState *s)
+{
+ Error *local_err = NULL;
+ s->flags = qemu_get_flags(f);
+ DPRINTF("flags: %x\n", s->flags);
+
+ if (s->flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) {
+ if (!qemu_get_counted_string(f, s->node_name)) {
+ fprintf(stderr, "Unable to read node name string\n");
+ return -EINVAL;
+ }
+ s->bs = bdrv_lookup_bs(s->node_name, s->node_name, &local_err);
+ if (!s->bs) {
+ error_report("%s", error_get_pretty(local_err));
+ error_free(local_err);
+ return -EINVAL;
+ }
+ } else if (!s->bs) {
+ fprintf(stderr, "Error: block device name is not set\n");
+ return -EINVAL;
+ }
+
+ if (s->flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) {
+ if (!qemu_get_counted_string(f, s->bitmap_name)) {
+ fprintf(stderr, "Unable to read node name string\n");
+ return -EINVAL;
+ }
+ s->bitmap = bdrv_find_dirty_bitmap(s->bs, s->bitmap_name);
+
+ /* bitmap may be NULL here, it wouldn't be an error if it is the
+ * first occurrence of the bitmap */
+ if (!s->bitmap && !(s->flags & DIRTY_BITMAP_MIG_FLAG_START)) {
+ fprintf(stderr, "Error: unknown dirty bitmap "
+ "'%s' for block device '%s'\n",
+ s->bitmap_name, s->node_name);
+ return -EINVAL;
+ }
+ } else if (!s->bitmap) {
+ fprintf(stderr, "Error: block device name is not set\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id)
+{
+ static DirtyBitmapLoadState s;
+
+ int ret = 0;
+
+ DPRINTF("load start\n");
+
+ do {
+ dirty_bitmap_load_header(f, &s);
+
+ if (s.flags & DIRTY_BITMAP_MIG_FLAG_START) {
+ ret = dirty_bitmap_load_start(f, &s);
+ } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_COMPLETE) {
+ dirty_bitmap_load_complete(f, &s);
+ } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_BITS) {
+ ret = dirty_bitmap_load_bits(f, &s);
+ }
+
+ DPRINTF("ret: %d\n", ret);
+ if (!ret) {
+ ret = qemu_file_get_error(f);
+ }
+
+ DPRINTF("ret: %d\n", ret);
+ if (ret) {
+ return ret;
+ }
+ } while (!(s.flags & DIRTY_BITMAP_MIG_FLAG_EOS));
+
+ DPRINTF("load finish\n");
+ return 0;
+}
+
+static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque)
+{
+ DirtyBitmapMigBitmapState *dbms = NULL;
+ init_dirty_bitmap_migration();
+
+ qemu_mutex_lock_iothread();
+ /* start track dirtiness of dirty bitmaps */
+ set_dirty_tracking();
+ qemu_mutex_unlock_iothread();
+
+ blk_mig_reset_dirty_cursor();
+
+ QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+ send_bitmap_start(f, dbms);
+ }
+ qemu_put_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS);
+
+ return 0;
+}
+
+static bool dirty_bitmap_is_active(void *opaque)
+{
+ return migrate_dirty_bitmaps();
+}
+
+static bool dirty_bitmap_live_iterate_is_active(void *opaque)
+{
+ return dirty_bitmap_mig_state.is_live_iterative && migrate_dirty_bitmaps();
+}
+
+static SaveVMHandlers savevm_dirty_bitmap_handlers = {
+ .save_live_setup = dirty_bitmap_save_setup,
+ .save_live_complete = dirty_bitmap_save_complete,
+ .save_live_pending = dirty_bitmap_save_pending,
+ .load_state = dirty_bitmap_load,
+ .cancel = dirty_bitmap_migration_cancel,
+ .is_active = dirty_bitmap_is_active,
+};
+
+static SaveVMHandlers savevm_dirty_bitmap_live_iterate_handlers = {
+ .save_live_iterate = dirty_bitmap_save_iterate,
+ .load_state = dirty_bitmap_load,
+ .is_active = dirty_bitmap_live_iterate_is_active,
+};
+
+void dirty_bitmap_mig_init(void)
+{
+ QSIMPLEQ_INIT(&dirty_bitmap_mig_state.dbms_list);
+
+ register_savevm_live(NULL, "dirty-bitmap", 0, 1,
+ &savevm_dirty_bitmap_handlers,
+ &dirty_bitmap_mig_state);
+ register_savevm_live(NULL, "dirty-bitmap-live-iterate", 0, 1,
+ &savevm_dirty_bitmap_live_iterate_handlers,
+ &dirty_bitmap_mig_state);
+}
diff --git a/vl.c b/vl.c
index 0adbbd6..7d745b5 100644
--- a/vl.c
+++ b/vl.c
@@ -4433,6 +4433,7 @@ int main(int argc, char **argv, char **envp)
blk_mig_init();
ram_mig_init();
+ dirty_bitmap_mig_init();
/* If the currently selected machine wishes to override the units-per-bus
* property of its default HBA interface type, do so now. */
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 08/12] migration: add migration/block-dirty-bitmap.c
2015-08-07 9:32 ` [Qemu-devel] [PATCH 08/12] migration: add migration/block-dirty-bitmap.c Vladimir Sementsov-Ogievskiy
@ 2015-09-11 0:10 ` John Snow
0 siblings, 0 replies; 22+ messages in thread
From: John Snow @ 2015-09-11 0:10 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: kwolf, peter.maydell, quintela, dgilbert, vsementsov, stefanha,
pbonzini, amit.shah, den
On 08/07/2015 05:32 AM, Vladimir Sementsov-Ogievskiy wrote:
> From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
>
> Live migration of dirty bitmaps. Only named dirty bitmaps, associated with
> root nodes and non-root named nodes are migrated.
>
> If destination qemu is already containing a dirty bitmap with the same name
> as a migrated bitmap (for the same node), than, if their granularities are
> the same the migration will be done, otherwise the error will be generated.
>
> If destination qemu doesn't contain such bitmap it will be created.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> include/migration/block.h | 1 +
> migration/Makefile.objs | 2 +-
> migration/block-dirty-bitmap.c | 699 +++++++++++++++++++++++++++++++++++++++++
> vl.c | 1 +
> 4 files changed, 702 insertions(+), 1 deletion(-)
> create mode 100644 migration/block-dirty-bitmap.c
>
> diff --git a/include/migration/block.h b/include/migration/block.h
> index ffa8ac0..566bb9f 100644
> --- a/include/migration/block.h
> +++ b/include/migration/block.h
> @@ -14,6 +14,7 @@
> #ifndef BLOCK_MIGRATION_H
> #define BLOCK_MIGRATION_H
>
> +void dirty_bitmap_mig_init(void);
> void blk_mig_init(void);
> int blk_mig_active(void);
> uint64_t blk_mig_bytes_transferred(void);
> diff --git a/migration/Makefile.objs b/migration/Makefile.objs
> index d929e96..128612d 100644
> --- a/migration/Makefile.objs
> +++ b/migration/Makefile.objs
> @@ -6,5 +6,5 @@ common-obj-y += xbzrle.o
> common-obj-$(CONFIG_RDMA) += rdma.o
> common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o
>
> -common-obj-y += block.o
> +common-obj-y += block.o block-dirty-bitmap.o
>
> diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
> new file mode 100644
> index 0000000..4095772
> --- /dev/null
> +++ b/migration/block-dirty-bitmap.c
> @@ -0,0 +1,699 @@
> +/*
> + * QEMU dirty bitmap migration
> + *
> + * Live migration of dirty bitmaps. Only named dirty bitmaps, associated with
> + * root nodes and non-root named nodes are migrated. Live iteration is disabled
> + * for small data amount (see MIN_LIVE_SIZE).
> + *
> + * If destination qemu is already containing a dirty bitmap with the same name
> + * as a migrated bitmap (for the same node), than, if their granularities are
> + * the same the migration will be done, otherwise the error will be generated.
> + *
> + * If destination qemu doesn't contain such bitmap it will be created.
> + *
> + * format of migration:
> + *
> + * # Header (shared for different chunk types)
> + * 1, 2 or 4 bytes: flags (see qemu_{put,put}_flags)
> + * [ 1 byte: node name size ] \ flags & DEVICE_NAME
> + * [ n bytes: node name ] /
> + * [ 1 byte: bitmap name size ] \ flags & BITMAP_NAME
> + * [ n bytes: bitmap name ] /
> + *
> + * # Start of bitmap migration (flags & START)
> + * header
> + * be64: granularity
> + *
> + * # Complete of bitmap migration (flags & COMPLETE)
> + * header
> + * 1 byte: bitmap enabled flag
> + *
> + * # Data chunk of bitmap migration
> + * header
> + * be64: start sector
> + * be32: number of sectors
> + * [ be64: buffer size ] \ ! (flags & ZEROES)
> + * [ n bytes: buffer ] /
> + *
> + * The last chunk in stream should contain flags & EOS. The chunk may skip
> + * device and/or bitmap names, assuming them to be the same with the previous
> + * chunk.
> + *
> + *
> + * This file is derived from migration/block.c
> + *
> + * Author:
> + * Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
> + *
> + * original copyright message:
> + * =====================================================================
> + * Copyright IBM, Corp. 2009
> + *
> + * Authors:
> + * Liran Schour <lirans@il.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2. See
> + * the COPYING file in the top-level directory.
> + *
> + * Contributions after 2012-01-13 are licensed under the terms of the
> + * GNU GPL, version 2 or (at your option) any later version.
> + * =====================================================================
> + */
> +
> +#include "block/block.h"
> +#include "block/block_int.h"
> +#include "sysemu/block-backend.h"
> +#include "qemu/main-loop.h"
> +#include "qemu/error-report.h"
> +#include "migration/block.h"
> +#include "migration/migration.h"
> +#include "qemu/hbitmap.h"
> +#include <assert.h>
> +
> +#define CHUNK_SIZE (1 << 10)
> +#define MIN_LIVE_SIZE (1 << 20)
> +
> +/* Flags occupy from one to four bytes. In all but one the 7-th (EXTRA_FLAGS)
> + * bit should be set. */
> +#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01
> +#define DIRTY_BITMAP_MIG_FLAG_ZEROES 0x02
> +#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x04
> +#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x08
> +#define DIRTY_BITMAP_MIG_FLAG_START 0x10
> +#define DIRTY_BITMAP_MIG_FLAG_COMPLETE 0x20
> +#define DIRTY_BITMAP_MIG_FLAG_BITS 0x40
> +
> +#define DIRTY_BITMAP_MIG_EXTRA_FLAGS 0x80
> +#define DIRTY_BITMAP_MIG_FLAGS_SIZE_16 0x8000
> +#define DIRTY_BITMAP_MIG_FLAGS_SIZE_32 0x8080
> +
> +#define DEBUG_DIRTY_BITMAP_MIGRATION 0
> +
> +#define DPRINTF(fmt, args...) \
> + do { \
> + if (DEBUG_DIRTY_BITMAP_MIGRATION) { \
> + printf("DMIG %s:%d", __func__, __LINE__); \
> + printf(fmt, ##args); \
> + } \
> + } while (0)
> +
> +typedef struct DirtyBitmapMigBitmapState {
> + /* Written during setup phase. */
> + BlockDriverState *bs;
> + const char *node_name;
> + BdrvDirtyBitmap *bitmap;
> + HBitmap *meta_bitmap;
> + uint64_t total_sectors;
> + uint64_t sectors_per_chunk;
> + QSIMPLEQ_ENTRY(DirtyBitmapMigBitmapState) entry;
> +
> + /* For bulk phase. */
> + bool bulk_completed;
> + uint64_t cur_sector;
> +
> + /* For dirty phase. */
> + HBitmapIter iter_dirty;
> +} DirtyBitmapMigBitmapState;
> +
> +typedef struct DirtyBitmapMigState {
> + QSIMPLEQ_HEAD(dbms_list, DirtyBitmapMigBitmapState) dbms_list;
> +
> + bool bulk_completed;
> + bool is_live_iterative;
> +
> + /* for send_bitmap_bits() */
> + BlockDriverState *prev_bs;
> + BdrvDirtyBitmap *prev_bitmap;
> +} DirtyBitmapMigState;
> +
> +typedef struct DirtyBitmapLoadState {
> + uint32_t flags;
> + char node_name[256];
> + char bitmap_name[256];
> + BlockDriverState *bs;
> + BdrvDirtyBitmap *bitmap;
> +} DirtyBitmapLoadState;
> +
> +static DirtyBitmapMigState dirty_bitmap_mig_state;
> +
> +static uint32_t qemu_get_flags(QEMUFile *f)
> +{
> + uint8_t flags = qemu_get_byte(f);
> + if (flags & DIRTY_BITMAP_MIG_EXTRA_FLAGS) {
> + flags = flags << 8 | qemu_get_byte(f);
> + if (flags & DIRTY_BITMAP_MIG_EXTRA_FLAGS) {
> + flags = flags << 16 | qemu_get_be16(f);
> + }
> + }
> +
> + return flags;
> +}
> +
> +static void qemu_put_flags(QEMUFile *f, uint32_t flags)
> +{
> + if (!(flags & 0xffffff00)) {
> + qemu_put_byte(f, flags);
> + return;
> + }
> +
> + if (!(flags & 0xffff0000)) {
> + qemu_put_be16(f, flags | DIRTY_BITMAP_MIG_FLAGS_SIZE_16);
> + return;
> + }
> +
> + qemu_put_be32(f, flags | DIRTY_BITMAP_MIG_FLAGS_SIZE_32);
> +}
> +
> +static void send_bitmap_header(QEMUFile *f, DirtyBitmapMigBitmapState *dbms,
> + uint32_t additional_flags)
> +{
> + BlockDriverState *bs = dbms->bs;
> + BdrvDirtyBitmap *bitmap = dbms->bitmap;
> + uint32_t flags = additional_flags;
> +
> + if (bs != dirty_bitmap_mig_state.prev_bs) {
> + dirty_bitmap_mig_state.prev_bs = bs;
> + flags |= DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME;
> + }
> +
> + if (bitmap != dirty_bitmap_mig_state.prev_bitmap) {
> + dirty_bitmap_mig_state.prev_bitmap = bitmap;
> + flags |= DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME;
> + }
> +
> + qemu_put_flags(f, flags);
> +
> + if (flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) {
> + qemu_put_counted_string(f, dbms->node_name);
> + }
> +
> + if (flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) {
> + qemu_put_counted_string(f, bdrv_dirty_bitmap_name(bitmap));
> + }
> +}
> +
> +static void send_bitmap_start(QEMUFile *f, DirtyBitmapMigBitmapState *dbms)
> +{
> + send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_START);
> + qemu_put_be32(f, bdrv_dirty_bitmap_granularity(dbms->bitmap));
> +}
> +
> +static void send_bitmap_complete(QEMUFile *f, DirtyBitmapMigBitmapState *dbms)
> +{
> + send_bitmap_header(f, dbms, DIRTY_BITMAP_MIG_FLAG_COMPLETE);
> + qemu_put_byte(f, bdrv_dirty_bitmap_enabled(dbms->bitmap));
> +}
> +
> +static void send_bitmap_bits(QEMUFile *f, DirtyBitmapMigBitmapState *dbms,
> + uint64_t start_sector, uint32_t nr_sectors)
> +{
> + /* align for buffer_is_zero() */
> + uint64_t align = 4 * sizeof(long);
> + uint64_t buf_size =
> + (bdrv_dirty_bitmap_data_size(dbms->bitmap, nr_sectors) + align - 1) &
> + ~(align - 1);
> + uint8_t *buf = g_malloc0(buf_size);
> + uint32_t flags = DIRTY_BITMAP_MIG_FLAG_BITS;
> +
> + bdrv_dirty_bitmap_serialize_part(dbms->bitmap, buf,
> + start_sector, nr_sectors);
> +
> + if (buffer_is_zero(buf, buf_size)) {
> + g_free(buf);
> + buf = NULL;
> + flags |= DIRTY_BITMAP_MIG_FLAG_ZEROES;
> + }
> +
> + DPRINTF("parameters:"
> + "\n flags: %x"
> + "\n start_sector: %" PRIu64
> + "\n nr_sectors: %" PRIu32
> + "\n data_size: %" PRIu64 "\n",
> + flags, start_sector, nr_sectors, buf_size);
> +
> + send_bitmap_header(f, dbms, flags);
> +
> + qemu_put_be64(f, start_sector);
> + qemu_put_be32(f, nr_sectors);
> +
> + /* if a block is zero we need to flush here since the network
> + * bandwidth is now a lot higher than the storage device bandwidth.
> + * thus if we queue zero blocks we slow down the migration.
> + * also, skip writing block when migrate only dirty bitmaps. */
> + if (flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) {
> + qemu_fflush(f);
> + return;
> + }
> +
> + qemu_put_be64(f, buf_size);
> + qemu_put_buffer(f, buf, buf_size);
> + g_free(buf);
> +}
> +
> +
> +/* Called with iothread lock taken. */
> +
> +static void set_dirty_tracking(void)
> +{
> + DirtyBitmapMigBitmapState *dbms;
> +
> + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
> + dbms->meta_bitmap =
> + bdrv_create_meta_bitmap(dbms->bitmap, CHUNK_SIZE);
> + dbms->sectors_per_chunk =
> + UINT64_C(1) << hbitmap_granularity(dbms->meta_bitmap);
> + }
> +}
> +
> +static void unset_dirty_tracking(void)
> +{
> + DirtyBitmapMigBitmapState *dbms;
> +
> + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
> + bdrv_release_meta_bitmap(dbms->bitmap);
> + }
> +}
> +
> +static void init_dirty_bitmap_migration(void)
> +{
> + BlockDriverState *bs;
> + BdrvDirtyBitmap *bitmap;
> + DirtyBitmapMigBitmapState *dbms;
> + uint64_t total_bytes = 0;
> +
> + dirty_bitmap_mig_state.bulk_completed = false;
> + dirty_bitmap_mig_state.prev_bs = NULL;
> + dirty_bitmap_mig_state.prev_bitmap = NULL;
> +
> + for (bs = bdrv_next(NULL); bs; bs = bdrv_next(bs)) {
> + for (bitmap = bdrv_next_dirty_bitmap(bs, NULL); bitmap;
> + bitmap = bdrv_next_dirty_bitmap(bs, bitmap)) {
> + if (!bdrv_dirty_bitmap_name(bitmap)) {
> + continue;
> + }
> +
> + if (!bdrv_get_node_name(bs) && blk_bs(bs->blk) != bs) {
> + /* not named non-root node */
> + continue;
> + }
> +
> + dbms = g_new0(DirtyBitmapMigBitmapState, 1);
> + dbms->bs = bs;
> + dbms->node_name = bdrv_get_node_name(bs);
> + if (!dbms->node_name || dbms->node_name[0] == '\0') {
> + dbms->node_name = bdrv_get_device_name(bs);
> + }
> + dbms->bitmap = bitmap;
> + dbms->total_sectors = bdrv_nb_sectors(bs);
> + total_bytes +=
> + bdrv_dirty_bitmap_data_size(bitmap, dbms->total_sectors);
> +
> + QSIMPLEQ_INSERT_TAIL(&dirty_bitmap_mig_state.dbms_list,
> + dbms, entry);
> + }
> + }
> +
> + dirty_bitmap_mig_state.is_live_iterative = total_bytes > MIN_LIVE_SIZE;
> +}
> +
> +/* Called with no lock taken. */
> +static void bulk_phase_send_chunk(QEMUFile *f, DirtyBitmapMigBitmapState *dbms)
> +{
> + uint32_t nr_sectors = MIN(dbms->total_sectors - dbms->cur_sector,
> + dbms->sectors_per_chunk);
> +
> + send_bitmap_bits(f, dbms, dbms->cur_sector, nr_sectors);
> +
> + dbms->cur_sector += nr_sectors;
> + if (dbms->cur_sector >= dbms->total_sectors) {
> + dbms->bulk_completed = true;
> + }
> +}
> +
> +/* Called with no lock taken. */
> +static void bulk_phase(QEMUFile *f, bool limit)
> +{
> + DirtyBitmapMigBitmapState *dbms;
> +
> + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
> + while (!dbms->bulk_completed) {
> + bulk_phase_send_chunk(f, dbms);
> + if (limit && qemu_file_rate_limit(f)) {
> + return;
> + }
> + }
> + }
> +
> + dirty_bitmap_mig_state.bulk_completed = true;
> +}
> +
> +static void blk_mig_reset_dirty_cursor(void)
> +{
> + DirtyBitmapMigBitmapState *dbms;
> +
> + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
> + hbitmap_iter_init(&dbms->iter_dirty, dbms->meta_bitmap, 0);
> + }
> +}
> +
> +/* Called with iothread lock taken. */
> +static bool dirty_phase_send_chunk(QEMUFile *f, DirtyBitmapMigBitmapState *dbms)
> +{
> + uint32_t nr_sectors;
> + size_t old_pos = dbms->iter_dirty.pos;
> + int64_t cur = hbitmap_iter_next(&dbms->iter_dirty);
> +
> + /* restart search from the beginning */
> + if (old_pos && cur == -1) {
> + hbitmap_iter_init(&dbms->iter_dirty, dbms->meta_bitmap, 0);
> + cur = hbitmap_iter_next(&dbms->iter_dirty);
> + }
> +
> + if (cur == -1) {
> + hbitmap_iter_init(&dbms->iter_dirty, dbms->meta_bitmap, 0);
> + return false;
> + }
> +
> + nr_sectors = MIN(dbms->total_sectors - cur, dbms->sectors_per_chunk);
> + send_bitmap_bits(f, dbms, cur, nr_sectors);
> + hbitmap_reset(dbms->meta_bitmap, cur, dbms->sectors_per_chunk);
> +
> + return true;
> +}
> +
> +/* Called with iothread lock taken. */
> +static void dirty_phase(QEMUFile *f, bool limit)
> +{
> + DirtyBitmapMigBitmapState *dbms;
> +
> + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
> + while (dirty_phase_send_chunk(f, dbms)) {
> + if (limit && qemu_file_rate_limit(f)) {
> + return;
> + }
> + }
> + }
> +}
> +
> +
> +/* Called with iothread lock taken. */
> +static void dirty_bitmap_mig_cleanup(void)
> +{
> + DirtyBitmapMigBitmapState *dbms;
> +
> + unset_dirty_tracking();
> +
> + while ((dbms = QSIMPLEQ_FIRST(&dirty_bitmap_mig_state.dbms_list)) != NULL) {
> + QSIMPLEQ_REMOVE_HEAD(&dirty_bitmap_mig_state.dbms_list, entry);
> + g_free(dbms);
> + }
> +}
> +
> +static void dirty_bitmap_migration_cancel(void *opaque)
> +{
> + dirty_bitmap_mig_cleanup();
> +}
> +
> +static int dirty_bitmap_save_iterate(QEMUFile *f, void *opaque)
> +{
> + DPRINTF("enter\n");
> +
> + if (dirty_bitmap_mig_state.bulk_completed) {
> + qemu_mutex_lock_iothread();
> + dirty_phase(f, true);
> + qemu_mutex_unlock_iothread();
> + } else {
> + bulk_phase(f, true);
> + }
> +
> + qemu_put_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS);
> +
> + return dirty_bitmap_mig_state.bulk_completed;
> +}
> +
> +/* Called with iothread lock taken. */
> +
> +static int dirty_bitmap_save_complete(QEMUFile *f, void *opaque)
> +{
> + DirtyBitmapMigBitmapState *dbms;
> + DPRINTF("enter\n");
> +
> + if (!dirty_bitmap_mig_state.bulk_completed) {
> + bulk_phase(f, false);
> + }
> +
> + blk_mig_reset_dirty_cursor();
> + dirty_phase(f, false);
> +
> + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
> + send_bitmap_complete(f, dbms);
> + }
> +
> + qemu_put_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS);
> +
> + DPRINTF("Dirty bitmaps migration completed\n");
> +
> + dirty_bitmap_mig_cleanup();
> + return 0;
> +}
> +
> +static uint64_t dirty_bitmap_save_pending(QEMUFile *f, void *opaque,
> + uint64_t max_size)
> +{
> + DirtyBitmapMigBitmapState *dbms;
> + uint64_t pending = 0;
> +
> + qemu_mutex_lock_iothread();
> +
> + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
> + uint64_t sectors = hbitmap_count(dbms->meta_bitmap);
> + if (!dbms->bulk_completed) {
> + sectors += dbms->total_sectors - dbms->cur_sector;
> + }
> + pending += bdrv_dirty_bitmap_data_size(dbms->bitmap, sectors);
> + }
> +
> + qemu_mutex_unlock_iothread();
> +
> + DPRINTF("pending %" PRIu64 ", max: %" PRIu64 "\n",
> + pending, max_size);
> + return pending;
> +}
> +
> +/* First occurrence of this bitmap. It should be created if doesn't exist */
> +static int dirty_bitmap_load_start(QEMUFile *f, DirtyBitmapLoadState *s)
> +{
> + uint32_t granularity = qemu_get_be32(f);
> + if (!s->bitmap) {
> + Error *local_err = NULL;
> + s->bitmap = bdrv_create_dirty_bitmap(s->bs, granularity,
> + s->bitmap_name, &local_err);
> + if (!s->bitmap) {
> + error_report("%s", error_get_pretty(local_err));
> + error_free(local_err);
> + return -EINVAL;
> + }
> + } else {
> + uint32_t dest_granularity =
> + bdrv_dirty_bitmap_granularity(s->bitmap);
> + if (dest_granularity != granularity) {
> + fprintf(stderr,
> + "Error: "
> + "Migrated bitmap granularity (%" PRIu32 ") "
> + "doesn't match the destination bitmap '%s' "
> + "granularity (%" PRIu32 ")\n",
> + granularity,
> + bdrv_dirty_bitmap_name(s->bitmap),
> + dest_granularity);
> + return -EINVAL;
> + }
> + }
> +
> + bdrv_disable_dirty_bitmap(s->bitmap);
> +
> + return 0;
> +}
> +
> +static void dirty_bitmap_load_complete(QEMUFile *f, DirtyBitmapLoadState *s)
> +{
> + bool enabled;
> +
> + bdrv_dirty_bitmap_deserialize_finish(s->bitmap);
> +
> + enabled = qemu_get_byte(f);
> + if (enabled) {
> + bdrv_enable_dirty_bitmap(s->bitmap);
> + }
> +}
> +
> +static int dirty_bitmap_load_bits(QEMUFile *f, DirtyBitmapLoadState *s)
> +{
> + uint64_t first_sector = qemu_get_be64(f);
> + uint32_t nr_sectors = qemu_get_be32(f);
> + DPRINTF("chunk: %" PRIu64 " %" PRIu32 "\n", first_sector, nr_sectors);
> +
> +
> + if (s->flags & DIRTY_BITMAP_MIG_FLAG_ZEROES) {
> + DPRINTF(" - zeroes\n");
> + bdrv_dirty_bitmap_deserialize_zeroes(s->bitmap, first_sector,
> + nr_sectors);
> + } else {
> + uint8_t *buf;
> + uint64_t buf_size = qemu_get_be64(f);
> + uint64_t needed_size =
> + bdrv_dirty_bitmap_data_size(s->bitmap, nr_sectors);
> +
> + if (needed_size > buf_size) {
> + fprintf(stderr,
> + "Error: Migrated bitmap granularity doesn't "
> + "match the destination bitmap '%s' granularity\n",
> + bdrv_dirty_bitmap_name(s->bitmap));
> + return -EINVAL;
> + }
> +
> + buf = g_malloc(buf_size);
> + qemu_get_buffer(f, buf, buf_size);
> + bdrv_dirty_bitmap_deserialize_part(s->bitmap, buf,
> + first_sector,
> + nr_sectors);
> + g_free(buf);
> + }
> +
> + return 0;
> +}
> +
> +static int dirty_bitmap_load_header(QEMUFile *f, DirtyBitmapLoadState *s)
> +{
> + Error *local_err = NULL;
> + s->flags = qemu_get_flags(f);
> + DPRINTF("flags: %x\n", s->flags);
> +
> + if (s->flags & DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME) {
> + if (!qemu_get_counted_string(f, s->node_name)) {
> + fprintf(stderr, "Unable to read node name string\n");
> + return -EINVAL;
> + }
> + s->bs = bdrv_lookup_bs(s->node_name, s->node_name, &local_err);
> + if (!s->bs) {
> + error_report("%s", error_get_pretty(local_err));
> + error_free(local_err);
> + return -EINVAL;
> + }
> + } else if (!s->bs) {
> + fprintf(stderr, "Error: block device name is not set\n");
> + return -EINVAL;
> + }
> +
> + if (s->flags & DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME) {
> + if (!qemu_get_counted_string(f, s->bitmap_name)) {
> + fprintf(stderr, "Unable to read node name string\n");
> + return -EINVAL;
> + }
> + s->bitmap = bdrv_find_dirty_bitmap(s->bs, s->bitmap_name);
> +
> + /* bitmap may be NULL here, it wouldn't be an error if it is the
> + * first occurrence of the bitmap */
> + if (!s->bitmap && !(s->flags & DIRTY_BITMAP_MIG_FLAG_START)) {
> + fprintf(stderr, "Error: unknown dirty bitmap "
> + "'%s' for block device '%s'\n",
> + s->bitmap_name, s->node_name);
> + return -EINVAL;
> + }
> + } else if (!s->bitmap) {
> + fprintf(stderr, "Error: block device name is not set\n");
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id)
> +{
> + static DirtyBitmapLoadState s;
> +
> + int ret = 0;
> +
> + DPRINTF("load start\n");
> +
> + do {
> + dirty_bitmap_load_header(f, &s);
> +
> + if (s.flags & DIRTY_BITMAP_MIG_FLAG_START) {
> + ret = dirty_bitmap_load_start(f, &s);
> + } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_COMPLETE) {
> + dirty_bitmap_load_complete(f, &s);
> + } else if (s.flags & DIRTY_BITMAP_MIG_FLAG_BITS) {
> + ret = dirty_bitmap_load_bits(f, &s);
> + }
> +
> + DPRINTF("ret: %d\n", ret);
> + if (!ret) {
> + ret = qemu_file_get_error(f);
> + }
> +
> + DPRINTF("ret: %d\n", ret);
> + if (ret) {
> + return ret;
> + }
> + } while (!(s.flags & DIRTY_BITMAP_MIG_FLAG_EOS));
> +
> + DPRINTF("load finish\n");
> + return 0;
> +}
> +
> +static int dirty_bitmap_save_setup(QEMUFile *f, void *opaque)
> +{
> + DirtyBitmapMigBitmapState *dbms = NULL;
> + init_dirty_bitmap_migration();
> +
> + qemu_mutex_lock_iothread();
> + /* start track dirtiness of dirty bitmaps */
> + set_dirty_tracking();
> + qemu_mutex_unlock_iothread();
> +
> + blk_mig_reset_dirty_cursor();
> +
> + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
> + send_bitmap_start(f, dbms);
> + }
> + qemu_put_flags(f, DIRTY_BITMAP_MIG_FLAG_EOS);
> +
> + return 0;
> +}
> +
> +static bool dirty_bitmap_is_active(void *opaque)
> +{
> + return migrate_dirty_bitmaps();
> +}
> +
> +static bool dirty_bitmap_live_iterate_is_active(void *opaque)
> +{
> + return dirty_bitmap_mig_state.is_live_iterative && migrate_dirty_bitmaps();
> +}
> +
> +static SaveVMHandlers savevm_dirty_bitmap_handlers = {
> + .save_live_setup = dirty_bitmap_save_setup,
> + .save_live_complete = dirty_bitmap_save_complete,
> + .save_live_pending = dirty_bitmap_save_pending,
> + .load_state = dirty_bitmap_load,
> + .cancel = dirty_bitmap_migration_cancel,
> + .is_active = dirty_bitmap_is_active,
> +};
> +
> +static SaveVMHandlers savevm_dirty_bitmap_live_iterate_handlers = {
> + .save_live_iterate = dirty_bitmap_save_iterate,
> + .load_state = dirty_bitmap_load,
> + .is_active = dirty_bitmap_live_iterate_is_active,
> +};
> +
> +void dirty_bitmap_mig_init(void)
> +{
> + QSIMPLEQ_INIT(&dirty_bitmap_mig_state.dbms_list);
> +
> + register_savevm_live(NULL, "dirty-bitmap", 0, 1,
> + &savevm_dirty_bitmap_handlers,
> + &dirty_bitmap_mig_state);
> + register_savevm_live(NULL, "dirty-bitmap-live-iterate", 0, 1,
> + &savevm_dirty_bitmap_live_iterate_handlers,
> + &dirty_bitmap_mig_state);
> +}
> diff --git a/vl.c b/vl.c
> index 0adbbd6..7d745b5 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -4433,6 +4433,7 @@ int main(int argc, char **argv, char **envp)
>
> blk_mig_init();
> ram_mig_init();
> + dirty_bitmap_mig_init();
>
> /* If the currently selected machine wishes to override the units-per-bus
> * property of its default HBA interface type, do so now. */
>
Reviewed-by: John Snow <jsnow@redhat.com>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 09/12] iotests: maintain several vms in test
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (7 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 08/12] migration: add migration/block-dirty-bitmap.c Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 10/12] iotests: add add_incoming_migration to VM class Vladimir Sementsov-Ogievskiy
` (4 subsequent siblings)
13 siblings, 0 replies; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
The only problem with it is the same qmp socket name (which is
vm._monitor_path) for all vms. And because of this second vm couldn't be
lauched (vm.launch() fails because of socket is already in use).
This patch adds a number of vm into vm._monitor_path
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/iotests.py | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 8615b10..8ebed31 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -97,11 +97,12 @@ def event_match(event, match=None):
class VM(object):
'''A QEMU VM'''
+ nb_vms = 0
def __init__(self):
- self._monitor_path = os.path.join(test_dir, 'qemu-mon.%d' % os.getpid())
- self._qemu_log_path = os.path.join(test_dir, 'qemu-log.%d' % os.getpid())
- self._qtest_path = os.path.join(test_dir, 'qemu-qtest.%d' % os.getpid())
+ self._monitor_path = os.path.join(test_dir, 'qemu-mon.%d.%d' % (os.getpid(), VM.nb_vms))
+ self._qemu_log_path = os.path.join(test_dir, 'qemu-log.%d.%d' % (os.getpid(), VM.nb_vms))
+ self._qtest_path = os.path.join(test_dir, 'qemu-qtest.%d.%d' % (os.getpid(), VM.nb_vms))
self._args = qemu_args + ['-chardev',
'socket,id=mon,path=' + self._monitor_path,
'-mon', 'chardev=mon,mode=control',
@@ -110,6 +111,7 @@ class VM(object):
'-display', 'none', '-vga', 'none']
self._num_drives = 0
self._events = []
+ VM.nb_vms += 1
# This can be used to add an unused monitor instance.
def add_monitor_telnet(self, ip, port):
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 10/12] iotests: add add_incoming_migration to VM class
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (8 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 09/12] iotests: maintain several vms in test Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-08-07 9:32 ` [Qemu-devel] [PATCH 11/12] qapi: add md5 checksum of last dirty bitmap level to query-block Vladimir Sementsov-Ogievskiy
` (3 subsequent siblings)
13 siblings, 0 replies; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/iotests.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 8ebed31..85c5a99 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -134,6 +134,12 @@ class VM(object):
self._num_drives += 1
return self
+ def add_incoming_migration(self, desc):
+ '''Add an incoming migration to the VM'''
+ self._args.append('-incoming')
+ self._args.append(desc)
+ return self
+
def pause_drive(self, drive, event=None):
'''Pause drive r/w operations'''
if not event:
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 11/12] qapi: add md5 checksum of last dirty bitmap level to query-block
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (9 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 10/12] iotests: add add_incoming_migration to VM class Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-09-15 16:29 ` Eric Blake
2015-08-07 9:32 ` [Qemu-devel] [PATCH 12/12] iotests: add dirty bitmap migration test Vladimir Sementsov-Ogievskiy
` (2 subsequent siblings)
13 siblings, 1 reply; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block.c | 1 +
include/qemu/hbitmap.h | 8 ++++++++
qapi/block-core.json | 4 +++-
util/hbitmap.c | 8 ++++++++
4 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/block.c b/block.c
index d9da767..4f7fc0d 100644
--- a/block.c
+++ b/block.c
@@ -3543,6 +3543,7 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs)
info->has_name = !!bm->name;
info->name = g_strdup(bm->name);
info->status = bdrv_dirty_bitmap_status(bm);
+ info->md5 = hbitmap_md5(bm->bitmap);
entry->value = info;
*plist = entry;
plist = &entry->next;
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 1da5b35..d8e1c7c 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -208,6 +208,14 @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count);
void hbitmap_deserialize_finish(HBitmap *hb);
/**
+ * hbitmap_md5:
+ * @bitmap: HBitmap to operate on.
+ *
+ * Returns md5 checksum of the last level.
+ */
+char *hbitmap_md5(const HBitmap *bitmap);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 7b2efb8..d1e68fd 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -359,11 +359,13 @@
#
# @status: current status of the dirty bitmap (since 2.4)
#
+# @md5: md5 checksum of the last bitmap level (since 2.4)
+#
# Since: 1.3
##
{ 'struct': 'BlockDirtyInfo',
'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32',
- 'status': 'DirtyBitmapStatus'} }
+ 'status': 'DirtyBitmapStatus', 'md5': 'str'} }
##
# @BlockInfo:
diff --git a/util/hbitmap.c b/util/hbitmap.c
index c7c21fe..def04b0 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -591,3 +591,11 @@ bool hbitmap_merge(HBitmap *a, const HBitmap *b)
return true;
}
+
+char *hbitmap_md5(const HBitmap *bitmap)
+{
+ uint64_t size =
+ MAX((bitmap->size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+ const guchar *data = (const guchar *)bitmap->levels[HBITMAP_LEVELS - 1];
+ return g_compute_checksum_for_data(G_CHECKSUM_MD5, data, size);
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 11/12] qapi: add md5 checksum of last dirty bitmap level to query-block
2015-08-07 9:32 ` [Qemu-devel] [PATCH 11/12] qapi: add md5 checksum of last dirty bitmap level to query-block Vladimir Sementsov-Ogievskiy
@ 2015-09-15 16:29 ` Eric Blake
2015-09-15 19:44 ` John Snow
0 siblings, 1 reply; 22+ messages in thread
From: Eric Blake @ 2015-09-15 16:29 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: kwolf, peter.maydell, quintela, dgilbert, vsementsov, stefanha,
pbonzini, amit.shah, den, jsnow
[-- Attachment #1: Type: text/plain, Size: 898 bytes --]
On 08/07/2015 03:32 AM, Vladimir Sementsov-Ogievskiy wrote:
> From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
>
> Reviewed-by: John Snow <jsnow@redhat.com>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
> block.c | 1 +
> include/qemu/hbitmap.h | 8 ++++++++
> qapi/block-core.json | 4 +++-
> util/hbitmap.c | 8 ++++++++
> 4 files changed, 20 insertions(+), 1 deletion(-)
> +++ b/qapi/block-core.json
> @@ -359,11 +359,13 @@
> #
> # @status: current status of the dirty bitmap (since 2.4)
> #
> +# @md5: md5 checksum of the last bitmap level (since 2.4)
since 2.5, now. Would it help to be explicit that this is a hexadecimal
encoding of the checksum (and not the actual binary value)?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 11/12] qapi: add md5 checksum of last dirty bitmap level to query-block
2015-09-15 16:29 ` Eric Blake
@ 2015-09-15 19:44 ` John Snow
2015-09-16 5:22 ` Markus Armbruster
0 siblings, 1 reply; 22+ messages in thread
From: John Snow @ 2015-09-15 19:44 UTC (permalink / raw)
To: Eric Blake, Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: kwolf, peter.maydell, quintela, dgilbert, vsementsov, stefanha,
pbonzini, amit.shah, den
On 09/15/2015 12:29 PM, Eric Blake wrote:
> On 08/07/2015 03:32 AM, Vladimir Sementsov-Ogievskiy wrote:
>> From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
>>
>> Reviewed-by: John Snow <jsnow@redhat.com>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> block.c | 1 +
>> include/qemu/hbitmap.h | 8 ++++++++
>> qapi/block-core.json | 4 +++-
>> util/hbitmap.c | 8 ++++++++
>> 4 files changed, 20 insertions(+), 1 deletion(-)
>
>> +++ b/qapi/block-core.json
>> @@ -359,11 +359,13 @@
>> #
>> # @status: current status of the dirty bitmap (since 2.4)
>> #
>> +# @md5: md5 checksum of the last bitmap level (since 2.4)
>
> since 2.5, now. Would it help to be explicit that this is a hexadecimal
> encoding of the checksum (and not the actual binary value)?
>
I had reasoned that it was implied so as to maintain a valid QMP
protocol stream.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH 11/12] qapi: add md5 checksum of last dirty bitmap level to query-block
2015-09-15 19:44 ` John Snow
@ 2015-09-16 5:22 ` Markus Armbruster
0 siblings, 0 replies; 22+ messages in thread
From: Markus Armbruster @ 2015-09-16 5:22 UTC (permalink / raw)
To: John Snow
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
qemu-devel, dgilbert, vsementsov, stefanha, den, amit.shah,
pbonzini
John Snow <jsnow@redhat.com> writes:
> On 09/15/2015 12:29 PM, Eric Blake wrote:
>> On 08/07/2015 03:32 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
>>>
>>> Reviewed-by: John Snow <jsnow@redhat.com>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>> block.c | 1 +
>>> include/qemu/hbitmap.h | 8 ++++++++
>>> qapi/block-core.json | 4 +++-
>>> util/hbitmap.c | 8 ++++++++
>>> 4 files changed, 20 insertions(+), 1 deletion(-)
>>
>>> +++ b/qapi/block-core.json
>>> @@ -359,11 +359,13 @@
>>> #
>>> # @status: current status of the dirty bitmap (since 2.4)
>>> #
>>> +# @md5: md5 checksum of the last bitmap level (since 2.4)
>>
>> since 2.5, now. Would it help to be explicit that this is a hexadecimal
>> encoding of the checksum (and not the actual binary value)?
>>
>
> I had reasoned that it was implied so as to maintain a valid QMP
> protocol stream.
There's more ASCII encodings of binary than hex. Hex is the common one
for MD5, but spelling it out won't hurt.
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Qemu-devel] [PATCH 12/12] iotests: add dirty bitmap migration test
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (10 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 11/12] qapi: add md5 checksum of last dirty bitmap level to query-block Vladimir Sementsov-Ogievskiy
@ 2015-08-07 9:32 ` Vladimir Sementsov-Ogievskiy
2015-08-07 10:10 ` [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
2015-09-11 0:11 ` John Snow
13 siblings, 0 replies; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 9:32 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, Vladimir Sementsov-Ogievskiy, quintela,
dgilbert, vsementsov, stefanha, den, amit.shah, pbonzini, jsnow
From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
The test starts two vms (vm_a, vm_b), create dirty bitmap in
the first one, do several writes to corresponding device and
then migrate vm_a to vm_b with dirty bitmaps.
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/117 | 84 ++++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/117.out | 5 +++
tests/qemu-iotests/group | 1 +
3 files changed, 90 insertions(+)
create mode 100755 tests/qemu-iotests/117
create mode 100644 tests/qemu-iotests/117.out
diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117
new file mode 100755
index 0000000..c78f38f
--- /dev/null
+++ b/tests/qemu-iotests/117
@@ -0,0 +1,84 @@
+#!/usr/bin/env python
+#
+# Tests for dirty bitmaps migration.
+#
+# (C) Vladimir Sementsov-Ogievskiy 2015
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk_a = os.path.join(iotests.test_dir, 'disk_a')
+disk_b = os.path.join(iotests.test_dir, 'disk_b')
+fifo = os.path.join(iotests.test_dir, 'fifo')
+
+size = 0x40000000 # 1G
+sector_size = 512
+granularity = 0x10000
+regions = [
+ { 'start': 0, 'count': 0x100000 },
+ { 'start': 0x10000000, 'count': 0x20000 },
+ { 'start': 0x39990000, 'count': 0x10000 }
+ ]
+
+class TestDirtyBitmapMigration(iotests.QMPTestCase):
+
+ def setUp(self):
+ os.mkfifo(fifo)
+ qemu_img('create', '-f', iotests.imgfmt, disk_a, str(size))
+ qemu_img('create', '-f', iotests.imgfmt, disk_b, str(size))
+ self.vm_a = iotests.VM().add_drive(disk_a)
+ self.vm_b = iotests.VM().add_drive(disk_b)
+ self.vm_b.add_incoming_migration("exec: cat " + fifo)
+ self.vm_a.launch()
+ self.vm_b.launch()
+
+ def tearDown(self):
+ self.vm_a.shutdown()
+ self.vm_b.shutdown()
+ os.remove(disk_a)
+ os.remove(disk_b)
+ os.remove(fifo)
+
+ def test_migration(self):
+ result = self.vm_a.qmp('block-dirty-bitmap-add', node='drive0',
+ name='bitmap', granularity=granularity)
+ self.assert_qmp(result, 'return', {});
+
+ for r in regions:
+ self.vm_a.hmp_qemu_io('drive0',
+ 'write %d %d' % (r['start'], r['count']))
+
+ result = self.vm_a.qmp('query-block');
+ md5 = result['return'][0]['dirty-bitmaps'][0]['md5']
+
+ result = self.vm_a.qmp('migrate-set-capabilities',
+ capabilities=[{'capability': 'dirty-bitmaps',
+ 'state': True}])
+ self.assert_qmp(result, 'return', {})
+
+ result = self.vm_a.qmp('migrate', uri='exec:cat>' + fifo)
+ self.assertNotEqual(self.vm_a.event_wait("STOP"), None)
+ self.assertNotEqual(self.vm_b.event_wait("RESUME"), None)
+
+ result = self.vm_b.qmp('query-block');
+ self.assert_qmp(result, 'return[0]/dirty-bitmaps[0]/md5', md5);
+
+
+if __name__ == '__main__':
+ iotests.main()
diff --git a/tests/qemu-iotests/117.out b/tests/qemu-iotests/117.out
new file mode 100644
index 0000000..ae1213e
--- /dev/null
+++ b/tests/qemu-iotests/117.out
@@ -0,0 +1,5 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index c430b6c..3a86575 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -121,6 +121,7 @@
114 rw auto quick
115 rw auto
116 rw auto quick
+117 rw auto quick
119 rw auto quick
120 rw auto quick
121 rw auto
--
2.1.4
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (11 preceding siblings ...)
2015-08-07 9:32 ` [Qemu-devel] [PATCH 12/12] iotests: add dirty bitmap migration test Vladimir Sementsov-Ogievskiy
@ 2015-08-07 10:10 ` Vladimir Sementsov-Ogievskiy
2015-09-11 0:11 ` John Snow
13 siblings, 0 replies; 22+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2015-08-07 10:10 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, peter.maydell, quintela, dgilbert, vsementsov, stefanha,
pbonzini, amit.shah, den, jsnow
Sorry for 'vsementsov@parallels.com' in patches. I've just fixed this
locally and I'm ready to roll v7 with all my emails fixed. I'm waiting
for other comments.
On 07.08.2015 12:32, Vladimir Sementsov-Ogievskiy wrote:
> From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
>
> These patches provide dirty bitmap migration feature. Only named dirty
> bitmaps are to be migrated. Migration may be enabled using migration
> capabilities.
>
> v6:
> - rebase on master
> - function qemu_get_string() deleted, used existed function
> qemu_get_counted_string()
> - function qemu_put_string() renamed appropriately to
> qemu_put_counted_string()
> (and now it is new separate patch)
> =====================================================
> 0004: Reviewed-by: John Snow <jsnow@redhat.com>
> 0006: (since 2.3) -> (since 2.4)
> 0007: new patch, mentioned above
> 0008: printf format fixed, DEBUG definition set back to 0, in
> dirty_bitmap_live_iterate_is_active conditions swapped.
> qemu_{get,put}_string removed, used ..counted.. verisions
> 0011: (since 2.3) -> (since 2.4)
> 0012: assertIsNotNone -> assertNotEqual for compitibility with
> old Python
> Reviewed-by: John Snow <jsnow@redhat.com>
>
>
> v5:
> - rebase on master
> - drop [PATCH RFC v4 10/13] iotests: add event_wait to VM class
> - remove rfc, as incremental backup series by John Snow are in
> upstream
>
>
> v4 significant changes:
> 0001: tiny bugfix: out[i] -> out[i-start], same for 'in'
> 0007: set chunk size to 1kb, disable live iteration for
> migrating data < 1mb size.
>
> tests: only one with md5 sum is here. used function event_wait
> by John Snow. (I hope, you don't mind me just adding this
> function with your 'Signed-off-by')
>
> rfc: This patch set is based on v13 of
> "block: incremental backup series" by John Snow, which are
> not pushed yet.
>
> v3:
> based on v13 of "block: incremental backup series" by John Snow.
>
> changes from v2:
> removed patch for adding dirty parameter (migration capablities used
> instead).
>
> 0001: printf's dropped, qapi used
> 0002: part0 -> zeroes
> 0003: part0 -> zeroes
> 0005: dirty_dirty -> meta
> add comments about meta bitmap
>
> 0006: the format is changed, nodes used instead of devices.
>
> other patches are new.
>
> rfc: there are two tests. They are the same but using different
> interfaces: md5 checksum of the bitmap last layer in query-block or
> separate query-block-dirty-bitmap with dirty bitmap regions.
> The second form is more appropriate for debugging, the first is more
> appropriate for simple regression control. Which should go to
> upstream?
>
> v2:
> 1. bug-fixes, that are already in upstream, and renaming of function
> bdrv_reset_dirty_bitmap (which is already in Snow's series) are
> dropped
> 2. bitmap store/restore: the concept renamed to serialization, added
> function hbitmap_deserialize_part0, to not transfer zero blocks
> 3. migration dirty parameter: added description comment
> 4. Other patches are new.
>
> v2.rfc:
> Actually, in this version of the series I'm trying not use
> migration/block.c at all. Instead a separate migration unit is added
> in the new file migration/dirty-bitmap.c. Now bitmaps are migrated
> like blocks in block migration, they have their "dirty-dirty" bitmaps,
> for tracking set/unset changes during migration.
>
> The advantages are:
> - no complications of migration/block.c
> - separate dirty-dirty bitmaps provide handling of "unset's"
> - more effective meta-data/data ratio - no tiny bitmap-blocks.
>
>
>
> v1:
> These patches provide dirty bitmap migration feature. Only named dirty
> bitmaps are to be migrated. Migration is made as a part of block
> migration in block-migration.c.
>
> Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate
> command. If "blk" and "inc" parameters are false when "dirty" is true
> block migration is actually skipped: no allocatoions, no bdrv_read's,
> no bdrv_write's, only bitmaps are migrated.
>
> The patch set includes two my previous bug fixes, which are necessary
> for it. The patch set is based on Incremental backup series by John
> Snow.
> Vladimir Sementsov-Ogievskiy (12):
> hbitmap: serialization
> block: BdrvDirtyBitmap serialization interface
> block: tiny refactoring: minimize hbitmap_(set/reset) usage
> block: add meta bitmaps
> block: add bdrv_next_dirty_bitmap()
> qapi: add dirty-bitmaps migration capability
> migration: add migration/block-dirty-bitmap.c
> iotests: maintain several vms in test
> iotests: add add_incoming_migration to VM class
> qapi: add md5 checksum of last dirty bitmap level to query-block
> iotests: add dirty bitmap migration test
> migration/qemu-file: make functions qemu_(get/put)_string public
>
> block.c | 98 +++++-
> include/block/block.h | 22 ++
> include/migration/block.h | 1 +
> include/migration/migration.h | 1 +
> include/migration/qemu-file.h | 17 +
> include/qemu/hbitmap.h | 67 ++++
> migration/Makefile.objs | 2 +-
> migration/block-dirty-bitmap.c | 693 +++++++++++++++++++++++++++++++++++++++++
> migration/migration.c | 9 +
> migration/qemu-file.c | 18 ++
> qapi-schema.json | 4 +-
> qapi/block-core.json | 4 +-
> tests/qemu-iotests/117 | 84 +++++
> tests/qemu-iotests/117.out | 5 +
> tests/qemu-iotests/group | 1 +
> tests/qemu-iotests/iotests.py | 14 +-
> util/hbitmap.c | 106 +++++++
> vl.c | 1 +
> 18 files changed, 1137 insertions(+), 10 deletions(-)
> create mode 100644 migration/block-dirty-bitmap.c
> create mode 100755 tests/qemu-iotests/117
> create mode 100644 tests/qemu-iotests/117.out
>
--
Best regards,
Vladimir
* now, @virtuozzo.com instead of @parallels.com. Sorry for this inconvenience.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration
2015-08-07 9:32 [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
` (12 preceding siblings ...)
2015-08-07 10:10 ` [Qemu-devel] [PATCH v6 00/12] Dirty bitmaps migration Vladimir Sementsov-Ogievskiy
@ 2015-09-11 0:11 ` John Snow
13 siblings, 0 replies; 22+ messages in thread
From: John Snow @ 2015-09-11 0:11 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: kwolf, peter.maydell, quintela, dgilbert, vsementsov, stefanha,
pbonzini, amit.shah, den
Ping: Quintela, David Gilbert, Amit Shah -- Looking for an ACK on this
series for 2.5 time-frame.
On 08/07/2015 05:32 AM, Vladimir Sementsov-Ogievskiy wrote:
> From: Vladimir Sementsov-Ogievskiy <vsementsov@parallels.com>
>
> These patches provide dirty bitmap migration feature. Only named dirty
> bitmaps are to be migrated. Migration may be enabled using migration
> capabilities.
>
> v6:
> - rebase on master
> - function qemu_get_string() deleted, used existed function
> qemu_get_counted_string()
> - function qemu_put_string() renamed appropriately to
> qemu_put_counted_string()
> (and now it is new separate patch)
> =====================================================
> 0004: Reviewed-by: John Snow <jsnow@redhat.com>
> 0006: (since 2.3) -> (since 2.4)
> 0007: new patch, mentioned above
> 0008: printf format fixed, DEBUG definition set back to 0, in
> dirty_bitmap_live_iterate_is_active conditions swapped.
> qemu_{get,put}_string removed, used ..counted.. verisions
> 0011: (since 2.3) -> (since 2.4)
> 0012: assertIsNotNone -> assertNotEqual for compitibility with
> old Python
> Reviewed-by: John Snow <jsnow@redhat.com>
>
>
> v5:
> - rebase on master
> - drop [PATCH RFC v4 10/13] iotests: add event_wait to VM class
> - remove rfc, as incremental backup series by John Snow are in
> upstream
>
>
> v4 significant changes:
> 0001: tiny bugfix: out[i] -> out[i-start], same for 'in'
> 0007: set chunk size to 1kb, disable live iteration for
> migrating data < 1mb size.
>
> tests: only one with md5 sum is here. used function event_wait
> by John Snow. (I hope, you don't mind me just adding this
> function with your 'Signed-off-by')
>
> rfc: This patch set is based on v13 of
> "block: incremental backup series" by John Snow, which are
> not pushed yet.
>
> v3:
> based on v13 of "block: incremental backup series" by John Snow.
>
> changes from v2:
> removed patch for adding dirty parameter (migration capablities used
> instead).
>
> 0001: printf's dropped, qapi used
> 0002: part0 -> zeroes
> 0003: part0 -> zeroes
> 0005: dirty_dirty -> meta
> add comments about meta bitmap
>
> 0006: the format is changed, nodes used instead of devices.
>
> other patches are new.
>
> rfc: there are two tests. They are the same but using different
> interfaces: md5 checksum of the bitmap last layer in query-block or
> separate query-block-dirty-bitmap with dirty bitmap regions.
> The second form is more appropriate for debugging, the first is more
> appropriate for simple regression control. Which should go to
> upstream?
>
> v2:
> 1. bug-fixes, that are already in upstream, and renaming of function
> bdrv_reset_dirty_bitmap (which is already in Snow's series) are
> dropped
> 2. bitmap store/restore: the concept renamed to serialization, added
> function hbitmap_deserialize_part0, to not transfer zero blocks
> 3. migration dirty parameter: added description comment
> 4. Other patches are new.
>
> v2.rfc:
> Actually, in this version of the series I'm trying not use
> migration/block.c at all. Instead a separate migration unit is added
> in the new file migration/dirty-bitmap.c. Now bitmaps are migrated
> like blocks in block migration, they have their "dirty-dirty" bitmaps,
> for tracking set/unset changes during migration.
>
> The advantages are:
> - no complications of migration/block.c
> - separate dirty-dirty bitmaps provide handling of "unset's"
> - more effective meta-data/data ratio - no tiny bitmap-blocks.
>
>
>
> v1:
> These patches provide dirty bitmap migration feature. Only named dirty
> bitmaps are to be migrated. Migration is made as a part of block
> migration in block-migration.c.
>
> Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate
> command. If "blk" and "inc" parameters are false when "dirty" is true
> block migration is actually skipped: no allocatoions, no bdrv_read's,
> no bdrv_write's, only bitmaps are migrated.
>
> The patch set includes two my previous bug fixes, which are necessary
> for it. The patch set is based on Incremental backup series by John
> Snow.
> Vladimir Sementsov-Ogievskiy (12):
> hbitmap: serialization
> block: BdrvDirtyBitmap serialization interface
> block: tiny refactoring: minimize hbitmap_(set/reset) usage
> block: add meta bitmaps
> block: add bdrv_next_dirty_bitmap()
> qapi: add dirty-bitmaps migration capability
> migration: add migration/block-dirty-bitmap.c
> iotests: maintain several vms in test
> iotests: add add_incoming_migration to VM class
> qapi: add md5 checksum of last dirty bitmap level to query-block
> iotests: add dirty bitmap migration test
> migration/qemu-file: make functions qemu_(get/put)_string public
>
> block.c | 98 +++++-
> include/block/block.h | 22 ++
> include/migration/block.h | 1 +
> include/migration/migration.h | 1 +
> include/migration/qemu-file.h | 17 +
> include/qemu/hbitmap.h | 67 ++++
> migration/Makefile.objs | 2 +-
> migration/block-dirty-bitmap.c | 693 +++++++++++++++++++++++++++++++++++++++++
> migration/migration.c | 9 +
> migration/qemu-file.c | 18 ++
> qapi-schema.json | 4 +-
> qapi/block-core.json | 4 +-
> tests/qemu-iotests/117 | 84 +++++
> tests/qemu-iotests/117.out | 5 +
> tests/qemu-iotests/group | 1 +
> tests/qemu-iotests/iotests.py | 14 +-
> util/hbitmap.c | 106 +++++++
> vl.c | 1 +
> 18 files changed, 1137 insertions(+), 10 deletions(-)
> create mode 100644 migration/block-dirty-bitmap.c
> create mode 100755 tests/qemu-iotests/117
> create mode 100644 tests/qemu-iotests/117.out
>
--
—js
^ permalink raw reply [flat|nested] 22+ messages in thread