[Qemu-devel] [PATCH v7 00/22] qcow2: persistent dirty bitmaps

[Qemu-devel] [PATCH v7 00/22] qcow2: persistent dirty bitmaps

[Vladimir Sementsov-Ogievskiy]

v7:
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v7
based on block-next (https://github.com/XanClic/qemu/commits/block-next)

- a lot of refactoring and reordering of patches.
- dead code removed (bdrv_dirty_bitmap_load, etc.)
- do not maintain extra data for now
- do not store dirty bitmap directory in memory
  (as we use it seldom, we can just reread if needed)

By Kevin's review:
01 - commit message changed: fix->improvement (as it was not a bug)
03 - r-b
04 - r-b
05 - add 21 patch to fix spec, also, removed all (I hope) mentions of
     "Bitmap Header", switch to one unified name for it - "Bitmap
     Directory Entry", to avoid misunderstanding with Qcow2 header.
     (also, add patch 22, to fix it in spec)
v6.06 - improve for_each_dir_entry loop, reorder patches, other small fixes
v6.07 ~> v7.09 - dead code removed, I've moved to one function 
        .bdrv_store_persistent_bitmaps and have wrapper and callback in one
        patch (with also some other staff. If it not ok, I can split them)
v6.08 - about keeping bitmap directory instead of bitmap list: no I don't keep
        it at all.
v6.16 - old bdrv_ bitmap-storing related functions are removed. The new one is
        bdrv_store_persistent_bitmaps.

v6:
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v6
based on block-next (https://github.com/XanClic/qemu/commits/block-next)

There are a lot of changes, reorderings and additions in comparement with v5.
One principal thing: now bitmaps are removed from image after loading instead
of marking them in_use. It is simpler and we do not need to store superfluous data.
Also, we are no more interested in command line interface to dirty bitmaps.
So it is dropped.  If someone needs it I can add it later.

Vladimir Sementsov-Ogievskiy (22):
  hbitmap: improve dirty iter
  tests: add hbitmap iter test
  block: fix bdrv_dirty_bitmap_granularity signature
  block/dirty-bitmap: add deserialize_ones func
  qcow2-bitmap: structs and consts
  qcow2: add dirty bitmaps extension
  qcow2-bitmap: introduce auto-loading bitmaps
  block/dirty-bitmap: add autoload field to BdrvDirtyBitmap
  block: introduce persistent dirty bitmaps
  block/dirty-bitmap: add bdrv_dirty_bitmap_next()
  qcow2-bitmap: add qcow2_store_persistent_bitmaps()
  qcow2-bitmap: add IN_USE flag
  qcow2-bitmap: check constraints
  qcow2: delete bitmaps on truncate
  qcow2-bitmap: add autoclear bit
  qmp: add persistent flag to block-dirty-bitmap-add
  qmp: add autoload parameter to block-dirty-bitmap-add
  qapi: add md5 checksum of last dirty bitmap level to query-block
  iotests: test qcow2 persistent dirty bitmap
  qcow2-dirty-bitmap: refcounts
  specs/qcow2: fix bitmap granularity qemu-specific note
  specs/qcow2: do not use wording 'bitmap header'

 block.c                      |  30 ++
 block/Makefile.objs          |   2 +-
 block/dirty-bitmap.c         |  59 ++-
 block/qcow2-bitmap.c         | 963 +++++++++++++++++++++++++++++++++++++++++++
 block/qcow2-refcount.c       |  14 +-
 block/qcow2.c                | 105 ++++-
 block/qcow2.h                |  54 +++
 blockdev.c                   |  30 +-
 docs/specs/qcow2.txt         |   8 +-
 include/block/block.h        |   2 +
 include/block/block_int.h    |   2 +
 include/block/dirty-bitmap.h |  16 +-
 include/qemu/hbitmap.h       |  47 ++-
 qapi/block-core.json         |  17 +-
 qmp-commands.hx              |   8 +-
 tests/qemu-iotests/165       |  87 ++++
 tests/qemu-iotests/165.out   |   5 +
 tests/qemu-iotests/group     |   1 +
 tests/test-hbitmap.c         |  19 +
 util/hbitmap.c               |  48 ++-
 20 files changed, 1475 insertions(+), 42 deletions(-)
 create mode 100644 block/qcow2-bitmap.c
 create mode 100755 tests/qemu-iotests/165
 create mode 100644 tests/qemu-iotests/165.out

-- 
1.8.3.1

[Qemu-devel] [PATCH 01/22] hbitmap: improve dirty iter

[Vladimir Sementsov-Ogievskiy]

Make dirty iter resistant to resetting bits in corresponding HBitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 include/qemu/hbitmap.h | 24 ++----------------------
 util/hbitmap.c         | 23 ++++++++++++++++++++++-
 2 files changed, 24 insertions(+), 23 deletions(-)

diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index eb46475..9aa86c1 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -243,10 +243,7 @@ void hbitmap_free(HBitmap *hb);
  * the lowest-numbered bit that is set in @hb, starting at @first.
  *
  * Concurrent setting of bits is acceptable, and will at worst cause the
- * iteration to miss some of those bits.  Resetting bits before the current
- * position of the iterator is also okay.  However, concurrent resetting of
- * bits can lead to unexpected behavior if the iterator has not yet reached
- * those bits.
+ * iteration to miss some of those bits. Concurrent bits resetting is ok too.
  */
 void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
 
@@ -285,24 +282,7 @@ void hbitmap_free_meta(HBitmap *hb);
  * Return the next bit that is set in @hbi's associated HBitmap,
  * or -1 if all remaining bits are zero.
  */
-static inline int64_t hbitmap_iter_next(HBitmapIter *hbi)
-{
-    unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1];
-    int64_t item;
-
-    if (cur == 0) {
-        cur = hbitmap_iter_skip_words(hbi);
-        if (cur == 0) {
-            return -1;
-        }
-    }
-
-    /* The next call will resume work from the next bit.  */
-    hbi->cur[HBITMAP_LEVELS - 1] = cur & (cur - 1);
-    item = ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur);
-
-    return item << hbi->granularity;
-}
+int64_t hbitmap_iter_next(HBitmapIter *hbi);
 
 /**
  * hbitmap_iter_next_word:
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 6a13c12..4f5cf96 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -106,8 +106,9 @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi)
 
     unsigned long cur;
     do {
-        cur = hbi->cur[--i];
+        i--;
         pos >>= BITS_PER_LEVEL;
+        cur = hbi->cur[i] & hb->levels[i][pos];
     } while (cur == 0);
 
     /* Check for end of iteration.  We always use fewer than BITS_PER_LONG
@@ -139,6 +140,26 @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi)
     return cur;
 }
 
+int64_t hbitmap_iter_next(HBitmapIter *hbi)
+{
+    unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1] &
+            hbi->hb->levels[HBITMAP_LEVELS - 1][hbi->pos];
+    int64_t item;
+
+    if (cur == 0) {
+        cur = hbitmap_iter_skip_words(hbi);
+        if (cur == 0) {
+            return -1;
+        }
+    }
+
+    /* The next call will resume work from the next bit.  */
+    hbi->cur[HBITMAP_LEVELS - 1] = cur & (cur - 1);
+    item = ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur);
+
+    return item << hbi->granularity;
+}
+
 void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first)
 {
     unsigned i, bit;
-- 
1.8.3.1

[Qemu-devel] [PATCH 02/22] tests: add hbitmap iter test

[Vladimir Sementsov-Ogievskiy]

Test that hbitmap iter is resistant to bitmap resetting.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 tests/test-hbitmap.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index 0ed918c..e24377f 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -881,6 +881,22 @@ static void hbitmap_test_add(const char *testpath,
                hbitmap_test_teardown);
 }
 
+static void test_hbitmap_iter_and_reset(TestHBitmapData *data,
+                                        const void *unused)
+{
+    HBitmapIter hbi;
+
+    hbitmap_test_init(data, L1 * 2, 0);
+    hbitmap_set(data->hb, 0, data->size);
+
+    hbitmap_iter_init(&hbi, data->hb, BITS_PER_LONG - 1);
+
+    hbitmap_iter_next(&hbi);
+
+    hbitmap_reset_all(data->hb);
+    hbitmap_iter_next(&hbi);
+}
+
 int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);
@@ -938,6 +954,9 @@ int main(int argc, char **argv)
                      test_hbitmap_serialize_part);
     hbitmap_test_add("/hbitmap/serialize/zeroes",
                      test_hbitmap_serialize_zeroes);
+
+    hbitmap_test_add("/hbitmap/iter/iter_and_reset",
+                     test_hbitmap_iter_and_reset);
     g_test_run();
 
     return 0;
-- 
1.8.3.1

[Qemu-devel] [PATCH 03/22] block: fix bdrv_dirty_bitmap_granularity signature

[Vladimir Sementsov-Ogievskiy]

Make getter signature const-correct. This allows other functions with
const dirty bitmap parameter use bdrv_dirty_bitmap_granularity().

Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/dirty-bitmap.c         | 2 +-
 include/block/dirty-bitmap.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 519737c..186941c 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -388,7 +388,7 @@ uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs)
     return granularity;
 }
 
-uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap)
+uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap)
 {
     return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap);
 }
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 9dea14b..7cbe623 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -29,7 +29,7 @@ void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
 void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
 BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs);
 uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs);
-uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap);
+uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap);
 uint32_t bdrv_dirty_bitmap_meta_granularity(BdrvDirtyBitmap *bitmap);
 bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap);
 bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap);
-- 
1.8.3.1

[Qemu-devel] [PATCH 04/22] block/dirty-bitmap: add deserialize_ones func

[Vladimir Sementsov-Ogievskiy]

Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for
qcow2 bitmap loading, to handle unallocated bitmap parts, marked as
all-ones.

Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/dirty-bitmap.c         |  7 +++++++
 include/block/dirty-bitmap.h |  3 +++
 include/qemu/hbitmap.h       | 15 +++++++++++++++
 util/hbitmap.c               | 17 +++++++++++++++++
 4 files changed, 42 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 186941c..90af372 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -499,6 +499,13 @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
     hbitmap_deserialize_zeroes(bitmap->bitmap, start, count, finish);
 }
 
+void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap,
+                                        uint64_t start, uint64_t count,
+                                        bool finish)
+{
+    hbitmap_deserialize_ones(bitmap->bitmap, start, count, finish);
+}
+
 void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap)
 {
     hbitmap_deserialize_finish(bitmap->bitmap);
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 7cbe623..1e17729 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -70,6 +70,9 @@ void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
 void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
                                           uint64_t start, uint64_t count,
                                           bool finish);
+void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap,
+                                        uint64_t start, uint64_t count,
+                                        bool finish);
 void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
 
 #endif
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 9aa86c1..64e5963 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -216,6 +216,21 @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count,
                                 bool finish);
 
 /**
+ * hbitmap_deserialize_ones
+ * @hb: HBitmap to operate on.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ * @finish: Whether to call hbitmap_deserialize_finish automatically.
+ *
+ * Fills the bitmap with ones.
+ *
+ * If @finish is false, caller must call hbitmap_serialize_finish before using
+ * the bitmap.
+ */
+void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count,
+                              bool finish);
+
+/**
  * hbitmap_deserialize_finish
  * @hb: HBitmap to operate on.
  *
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 4f5cf96..f49520f 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -531,6 +531,23 @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count,
     }
 }
 
+void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count,
+                              bool finish)
+{
+    uint64_t el_count;
+    unsigned long *first;
+
+    if (!count) {
+        return;
+    }
+    serialization_chunk(hb, start, count, &first, &el_count);
+
+    memset(first, 0xff, el_count * sizeof(unsigned long));
+    if (finish) {
+        hbitmap_deserialize_finish(hb);
+    }
+}
+
 void hbitmap_deserialize_finish(HBitmap *bitmap)
 {
     int64_t i, size, prev_size;
-- 
1.8.3.1

[Qemu-devel] [PATCH 05/22] qcow2-bitmap: structs and consts

[Vladimir Sementsov-Ogievskiy]

Create block/qcow2-bitmap.c
Add data structures and constraints accordingly to docs/specs/qcow2.txt

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/Makefile.objs  |  2 +-
 block/qcow2-bitmap.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 block/qcow2.h        | 29 +++++++++++++++++++++++++++++
 3 files changed, 77 insertions(+), 1 deletion(-)
 create mode 100644 block/qcow2-bitmap.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index fa4d8b8..0f661bb 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -1,5 +1,5 @@
 block-obj-y += raw_bsd.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o dmg.o
-block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
+block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o
 block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
 block-obj-y += qed-check.o
 block-obj-$(CONFIG_VHDX) += vhdx.o vhdx-endian.o vhdx-log.o
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
new file mode 100644
index 0000000..cd18b07
--- /dev/null
+++ b/block/qcow2-bitmap.c
@@ -0,0 +1,47 @@
+/*
+ * Bitmaps for the QCOW version 2 format
+ *
+ * Copyright (c) 2014-2016 Vladimir Sementsov-Ogievskiy
+ *
+ * This file is derived from qcow2-snapshot.c, original copyright:
+ * Copyright (c) 2004-2006 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+/* NOTICE: BME here means Bitmaps Extension and used as a namespace for
+ * _internal_ constants. Please do not use this _internal_ abbreviation for
+ * other needs and/or outside of this file. */
+
+/* Bitmap directory entry constraints */
+#define BME_MAX_TABLE_SIZE 0x8000000
+#define BME_MAX_PHYS_SIZE 0x20000000 /* 512 mb */
+#define BME_MAX_GRANULARITY_BITS 31
+#define BME_MIN_GRANULARITY_BITS 9
+#define BME_MAX_NAME_SIZE 1023
+
+/* Bitmap directory entry flags */
+#define BME_RESERVED_FLAGS 0xffffffff
+
+/* bits [1, 8] U [56, 63] are reserved */
+#define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001fe
+
+typedef enum BitmapType {
+    BT_DIRTY_TRACKING_BITMAP = 1
+} BitmapType;
diff --git a/block/qcow2.h b/block/qcow2.h
index b36a7bf..0480b8b 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -52,6 +52,10 @@
  * space for snapshot names and IDs */
 #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS)
 
+/* Bitmap header extension constraints */
+#define QCOW_MAX_DIRTY_BITMAPS 65535
+#define QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE (1024 * QCOW_MAX_DIRTY_BITMAPS)
+
 /* indicate that the refcount of the referenced cluster is exactly one. */
 #define QCOW_OFLAG_COPIED     (1ULL << 63)
 /* indicate that the cluster is compressed (they never have the copied flag) */
@@ -142,6 +146,22 @@ typedef struct QEMU_PACKED QCowSnapshotHeader {
     /* name follows  */
 } QCowSnapshotHeader;
 
+/* Qcow2BitmapDirEntry is actually a bitmap directory entry */
+typedef struct QEMU_PACKED Qcow2BitmapDirEntry {
+    /* header is 8 byte aligned */
+    uint64_t bitmap_table_offset;
+
+    uint32_t bitmap_table_size;
+    uint32_t flags;
+
+    uint8_t type;
+    uint8_t granularity_bits;
+    uint16_t name_size;
+    uint32_t extra_data_size;
+    /* extra data follows  */
+    /* name follows  */
+} Qcow2BitmapDirEntry;
+
 typedef struct QEMU_PACKED QCowSnapshotExtraData {
     uint64_t vm_state_size_large;
     uint64_t disk_size;
@@ -222,6 +242,15 @@ typedef uint64_t Qcow2GetRefcountFunc(const void *refcount_array,
 typedef void Qcow2SetRefcountFunc(void *refcount_array,
                                   uint64_t index, uint64_t value);
 
+/* Be careful, Qcow2BitmapHeaderExt is not an extension of Qcow2BitmapDirEntry, it
+ * is Qcow2 header extension */
+typedef struct Qcow2BitmapHeaderExt {
+    uint32_t nb_bitmaps;
+    uint32_t reserved32;
+    uint64_t bitmap_directory_size;
+    uint64_t bitmap_directory_offset;
+} QEMU_PACKED Qcow2BitmapHeaderExt;
+
 typedef struct BDRVQcow2State {
     int cluster_bits;
     int cluster_size;
-- 
1.8.3.1

[Qemu-devel] [PATCH 06/22] qcow2: add dirty bitmaps extension

[Vladimir Sementsov-Ogievskiy]

Add dirty bitmap extension as specified in docs/specs/qcow2.txt.
For now, just mirror extension header into Qcow2 state and check
constraints.

For now, disable image resize if it has bitmaps. It will be fixed later.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 block/qcow2.h |  4 +++
 2 files changed, 87 insertions(+)

diff --git a/block/qcow2.c b/block/qcow2.c
index c079aa8..08c4ef9 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -63,6 +63,7 @@ typedef struct {
 #define  QCOW2_EXT_MAGIC_END 0
 #define  QCOW2_EXT_MAGIC_BACKING_FORMAT 0xE2792ACA
 #define  QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857
+#define  QCOW2_EXT_MAGIC_DIRTY_BITMAPS 0x23852875
 
 static int qcow2_probe(const uint8_t *buf, int buf_size, const char *filename)
 {
@@ -92,6 +93,7 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
     QCowExtension ext;
     uint64_t offset;
     int ret;
+    Qcow2BitmapHeaderExt bitmaps_ext;
 
 #ifdef DEBUG_EXT
     printf("qcow2_read_extensions: start=%ld end=%ld\n", start_offset, end_offset);
@@ -162,6 +164,62 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
             }
             break;
 
+        case QCOW2_EXT_MAGIC_DIRTY_BITMAPS:
+            ret = bdrv_pread(bs->file, offset, &bitmaps_ext, ext.len);
+            if (ret < 0) {
+                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
+                                 "Could not read ext header");
+                return ret;
+            }
+
+            if (bitmaps_ext.reserved32 != 0) {
+                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
+                                 "Reserved field is not zero.");
+                return -EINVAL;
+            }
+
+            be32_to_cpus(&bitmaps_ext.nb_bitmaps);
+            be64_to_cpus(&bitmaps_ext.bitmap_directory_size);
+            be64_to_cpus(&bitmaps_ext.bitmap_directory_offset);
+
+            if (bitmaps_ext.nb_bitmaps > QCOW_MAX_DIRTY_BITMAPS) {
+                error_setg(errp, "ERROR: bitmaps_ext: "
+                                 "too many dirty bitmaps");
+                return -EINVAL;
+            }
+
+            if (bitmaps_ext.nb_bitmaps == 0) {
+                error_setg(errp, "ERROR: bitmaps_ext: "
+                                 "found bitmaps extension with zero bitmaps");
+                return -EINVAL;
+            }
+
+            if (bitmaps_ext.bitmap_directory_offset & (s->cluster_size - 1)) {
+                error_setg(errp, "ERROR: bitmaps_ext: "
+                                 "wrong dirty bitmap directory offset");
+                return -EINVAL;
+            }
+
+            if (bitmaps_ext.bitmap_directory_size >
+                QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE) {
+                error_setg(errp, "ERROR: bitmaps_ext: "
+                                 "too large dirty bitmap directory");
+                return -EINVAL;
+            }
+
+            s->nb_bitmaps = bitmaps_ext.nb_bitmaps;
+            s->bitmap_directory_offset =
+                    bitmaps_ext.bitmap_directory_offset;
+            s->bitmap_directory_size =
+                    bitmaps_ext.bitmap_directory_size;
+
+#ifdef DEBUG_EXT
+            printf("Qcow2: Got dirty bitmaps extension:"
+                   " offset=%" PRIu64 " nb_bitmaps=%" PRIu32 "\n",
+                   s->bitmap_directory_offset, s->nb_bitmaps);
+#endif
+            break;
+
         default:
             /* unknown magic - save it in case we need to rewrite the header */
             {
@@ -1939,6 +1997,24 @@ int qcow2_update_header(BlockDriverState *bs)
         buflen -= ret;
     }
 
+    if (s->nb_bitmaps > 0) {
+        Qcow2BitmapHeaderExt bitmaps_header = {
+            .nb_bitmaps = cpu_to_be32(s->nb_bitmaps),
+            .bitmap_directory_size =
+                    cpu_to_be64(s->bitmap_directory_size),
+            .bitmap_directory_offset =
+                    cpu_to_be64(s->bitmap_directory_offset)
+        };
+        ret = header_ext_add(buf, QCOW2_EXT_MAGIC_DIRTY_BITMAPS,
+                             &bitmaps_header, sizeof(bitmaps_header),
+                             buflen);
+        if (ret < 0) {
+            goto fail;
+        }
+        buf += ret;
+        buflen -= ret;
+    }
+
     /* Keep unknown header extensions */
     QLIST_FOREACH(uext, &s->unknown_header_ext, next) {
         ret = header_ext_add(buf, uext->magic, uext->data, uext->len, buflen);
@@ -2509,6 +2585,13 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset)
         return -ENOTSUP;
     }
 
+    /* cannot proceed if image has bitmaps */
+    if (s->nb_bitmaps) {
+        /* FIXME */
+        error_report("Can't resize an image which has bitmaps");
+        return -ENOTSUP;
+    }
+
     /* shrinking is currently not supported */
     if (offset < bs->total_sectors * 512) {
         error_report("qcow2 doesn't support shrinking images yet");
diff --git a/block/qcow2.h b/block/qcow2.h
index 0480b8b..c068b2c 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -292,6 +292,10 @@ typedef struct BDRVQcow2State {
     unsigned int nb_snapshots;
     QCowSnapshot *snapshots;
 
+    uint64_t bitmap_directory_offset;
+    uint64_t bitmap_directory_size;
+    uint32_t nb_bitmaps;
+
     int flags;
     int qcow_version;
     bool use_lazy_refcounts;
-- 
1.8.3.1

[Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Vladimir Sementsov-Ogievskiy]

Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are
loaded at image open and becomes BdrvDirtyBitmap's for corresponding
drive. These bitmaps are deleted from Qcow2 image after loading to avoid
conflicts.

Extra data in bitmaps is not supported for now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-bitmap.c | 518 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 block/qcow2.c        |   5 +
 block/qcow2.h        |   3 +
 3 files changed, 525 insertions(+), 1 deletion(-)

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index cd18b07..760a047 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -25,6 +25,12 @@
  * THE SOFTWARE.
  */
 
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+
+#include "block/block_int.h"
+#include "block/qcow2.h"
+
 /* NOTICE: BME here means Bitmaps Extension and used as a namespace for
  * _internal_ constants. Please do not use this _internal_ abbreviation for
  * other needs and/or outside of this file. */
@@ -37,11 +43,521 @@
 #define BME_MAX_NAME_SIZE 1023
 
 /* Bitmap directory entry flags */
-#define BME_RESERVED_FLAGS 0xffffffff
+#define BME_RESERVED_FLAGS 0xfffffffd
+#define BME_FLAG_AUTO   (1U << 1)
 
 /* bits [1, 8] U [56, 63] are reserved */
 #define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001fe
 
+#define for_each_bitmap_dir_entry_unsafe(entry, dir, size) \
+    for (entry = (Qcow2BitmapDirEntry *)(dir); \
+         entry < (Qcow2BitmapDirEntry *)((uint8_t *)(dir) + size); \
+         entry = next_dir_entry(entry))
+
+#define for_each_bitmap_dir_entry(entry, dir, size) \
+    for (entry = (Qcow2BitmapDirEntry *)(dir); \
+         assert(check_dir_iter(entry, (uint8_t *)(dir) + size)), \
+             entry < (Qcow2BitmapDirEntry *)((uint8_t *)(dir) + size); \
+         entry = next_dir_entry(entry))
+
 typedef enum BitmapType {
     BT_DIRTY_TRACKING_BITMAP = 1
 } BitmapType;
+
+static inline bool can_write(BlockDriverState *bs)
+{
+    return !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE);
+}
+
+static inline void bitmap_dir_entry_to_cpu(Qcow2BitmapDirEntry *entry)
+{
+    be64_to_cpus(&entry->bitmap_table_offset);
+    be32_to_cpus(&entry->bitmap_table_size);
+    be32_to_cpus(&entry->flags);
+    be16_to_cpus(&entry->name_size);
+    be32_to_cpus(&entry->extra_data_size);
+}
+
+static inline void bitmap_dir_entry_to_be(Qcow2BitmapDirEntry *entry)
+{
+    cpu_to_be64s(&entry->bitmap_table_offset);
+    cpu_to_be32s(&entry->bitmap_table_size);
+    cpu_to_be32s(&entry->flags);
+    cpu_to_be16s(&entry->name_size);
+    cpu_to_be32s(&entry->extra_data_size);
+}
+
+static inline void bitmap_table_to_cpu(uint64_t *bitmap_table, size_t size)
+{
+    size_t i;
+
+    for (i = 0; i < size; ++i) {
+        be64_to_cpus(&bitmap_table[i]);
+    }
+}
+
+static inline int calc_dir_entry_size(size_t name_size, size_t extra_data_size)
+{
+    return align_offset(sizeof(Qcow2BitmapDirEntry) +
+                        name_size + extra_data_size, 8);
+}
+
+static inline int dir_entry_size(Qcow2BitmapDirEntry *entry)
+{
+    return calc_dir_entry_size(entry->name_size, entry->extra_data_size);
+}
+
+static inline const char *dir_entry_name_notcstr(Qcow2BitmapDirEntry *entry)
+{
+    return (const char *)(entry + 1) + entry->extra_data_size;
+}
+
+static inline int copy_dir_entry(void *dest, Qcow2BitmapDirEntry *entry)
+{
+    int sz = dir_entry_size(entry);
+    memcpy(dest, entry, sz);
+    return sz;
+}
+
+static inline Qcow2BitmapDirEntry *next_dir_entry(Qcow2BitmapDirEntry *entry)
+{
+    return (Qcow2BitmapDirEntry *)((uint8_t *)entry + dir_entry_size(entry));
+}
+
+static inline bool check_dir_iter(Qcow2BitmapDirEntry *it, void *directory_end)
+{
+    return ((void *)it == directory_end) ||
+               (((void *)(it + 1) <= directory_end) &&
+                ((void *)next_dir_entry(it) <= directory_end));
+}
+
+static inline void bitmap_directory_to_be(uint8_t *dir, size_t size)
+{
+    uint8_t *end = dir + size;
+    while (dir < end) {
+        Qcow2BitmapDirEntry *e = (Qcow2BitmapDirEntry *)dir;
+        dir += dir_entry_size(e);
+
+        bitmap_dir_entry_to_be(e);
+    }
+}
+
+static void clear_bitmap_table(BlockDriverState *bs, uint64_t *bitmap_table,
+                               uint32_t bitmap_table_size)
+{
+    BDRVQcow2State *s = bs->opaque;
+    int cl_size = s->cluster_size;
+    int i;
+
+    for (i = 0; i < bitmap_table_size; ++i) {
+        uint64_t addr = bitmap_table[i];
+        if (addr <= 1) {
+            continue;
+        }
+
+        qcow2_free_clusters(bs, addr, cl_size, QCOW2_DISCARD_ALWAYS);
+        bitmap_table[i] = 0;
+    }
+}
+
+static int bitmap_table_load(BlockDriverState *bs, Qcow2BitmapDirEntry *entry,
+                             uint64_t **table)
+{
+    int ret;
+
+    *table = g_try_new(uint64_t, entry->bitmap_table_size);
+    if (*table == NULL) {
+        return -ENOMEM;
+    }
+
+    ret = bdrv_pread(bs->file, entry->bitmap_table_offset,
+                     *table, entry->bitmap_table_size * sizeof(uint64_t));
+    if (ret < 0) {
+        g_free(*table);
+        *table = NULL;
+        return ret;
+    }
+
+    bitmap_table_to_cpu(*table, entry->bitmap_table_size);
+
+    return 0;
+}
+
+static void do_free_bitmap_clusters(BlockDriverState *bs,
+                                    uint64_t table_offset,
+                                    uint32_t table_size,
+                                    uint64_t *bitmap_table)
+{
+    clear_bitmap_table(bs, bitmap_table, table_size);
+
+    qcow2_free_clusters(bs, table_offset, table_size * sizeof(uint64_t),
+                        QCOW2_DISCARD_ALWAYS);
+}
+
+static int free_bitmap_clusters(BlockDriverState *bs, Qcow2BitmapDirEntry *entry)
+{
+    int ret;
+    uint64_t *bitmap_table;
+
+    ret = bitmap_table_load(bs, entry, &bitmap_table);
+    if (ret < 0 || bitmap_table == NULL) {
+        return ret;
+    }
+
+    do_free_bitmap_clusters(bs, entry->bitmap_table_offset,
+                            entry->bitmap_table_size, bitmap_table);
+
+    return 0;
+}
+
+/* dirty sectors in cluster is a number of sectors in the image, corresponding
+ * to one cluster of bitmap data */
+static uint64_t dirty_sectors_in_cluster(const BDRVQcow2State *s,
+                                         const BdrvDirtyBitmap *bitmap)
+{
+    uint32_t sector_granularity =
+            bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS;
+
+    return (uint64_t)sector_granularity * (s->cluster_size << 3);
+}
+
+static int load_bitmap_data(BlockDriverState *bs,
+                            const uint64_t *dirty_bitmap_table,
+                            uint32_t dirty_bitmap_table_size,
+                            BdrvDirtyBitmap *bitmap)
+{
+    int ret = 0;
+    BDRVQcow2State *s = bs->opaque;
+    uint64_t sector, dsc;
+    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
+    int cl_size = s->cluster_size;
+    uint8_t *buf = NULL;
+    uint32_t i, tab_size =
+            size_to_clusters(s,
+                bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));
+
+    if (tab_size != dirty_bitmap_table_size) {
+        return -EINVAL;
+    }
+
+    bdrv_clear_dirty_bitmap(bitmap, NULL);
+
+    buf = g_malloc0(cl_size);
+    dsc = dirty_sectors_in_cluster(s, bitmap);
+    for (i = 0, sector = 0; i < tab_size; ++i, sector += dsc) {
+        uint64_t end = MIN(bm_size, sector + dsc);
+        uint64_t offset = dirty_bitmap_table[i];
+
+        if (offset == 1) {
+            bdrv_dirty_bitmap_deserialize_ones(bitmap, sector, end, false);
+        } else if (offset > 1) {
+            ret = bdrv_pread(bs->file, offset, buf, cl_size);
+            if (ret < 0) {
+                goto finish;
+            }
+            bdrv_dirty_bitmap_deserialize_part(bitmap, buf, sector, end, false);
+        }
+    }
+    ret = 0;
+
+    bdrv_dirty_bitmap_deserialize_finish(bitmap);
+
+finish:
+    g_free(buf);
+
+    return ret;
+}
+
+static BdrvDirtyBitmap *load_bitmap(BlockDriverState *bs,
+                                    Qcow2BitmapDirEntry *entry, Error **errp)
+{
+    int ret;
+    uint64_t *bitmap_table = NULL;
+    uint32_t granularity;
+    BdrvDirtyBitmap *bitmap = NULL;
+    char *name = g_strndup(dir_entry_name_notcstr(entry), entry->name_size);
+
+    ret = bitmap_table_load(bs, entry, &bitmap_table);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret,
+                         "Could not read bitmap_table table from image");
+        goto fail;
+    }
+
+    granularity = 1U << entry->granularity_bits;
+    bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp);
+    if (bitmap == NULL) {
+        goto fail;
+    }
+
+    ret = load_bitmap_data(bs, bitmap_table, entry->bitmap_table_size,
+                           bitmap);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Could not read bitmap from image");
+        goto fail;
+    }
+
+    g_free(name);
+    g_free(bitmap_table);
+    return bitmap;
+
+fail:
+    g_free(name);
+    g_free(bitmap_table);
+    if (bitmap != NULL) {
+        bdrv_release_dirty_bitmap(bs, bitmap);
+    }
+
+    return NULL;
+}
+
+/* directory_read
+ * Read bitmaps directory from bs by @offset and @size. Convert it to cpu
+ * format from BE.
+ */
+static uint8_t *directory_read(BlockDriverState *bs,
+                               uint64_t offset, uint64_t size, Error **errp)
+{
+    int ret;
+    uint8_t *dir, *dir_end;
+    Qcow2BitmapDirEntry *e;
+
+    dir = g_try_malloc(size);
+    if (dir == NULL) {
+        error_setg(errp, "Can't allocate space for bitmap directory.");
+        return NULL;
+    }
+    dir_end = dir + size;
+
+    ret = bdrv_pread(bs->file, offset, dir, size);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Can't read bitmap directory.");
+        goto fail;
+    }
+
+    /* loop is safe because next entry offset is calculated after conversion to
+     * cpu format */
+    for_each_bitmap_dir_entry_unsafe(e, dir, size) {
+        if ((uint8_t *)(e + 1) > dir_end) {
+            goto broken_dir;
+        }
+
+        bitmap_dir_entry_to_cpu(e);
+
+        if ((uint8_t *)next_dir_entry(e) > dir_end) {
+            goto broken_dir;
+        }
+
+        if (e->extra_data_size != 0) {
+            error_setg(errp, "Can't load bitmap '%.*s' from '%s':"
+                       "extra data not supported.", e->name_size,
+                       dir_entry_name_notcstr(e),
+                       bdrv_get_device_or_node_name(bs));
+            goto fail;
+        }
+    }
+
+    assert((uint8_t *)e == dir_end);
+
+    return dir;
+
+broken_dir:
+    error_setg(errp, "Broken bitmap directory.");
+
+fail:
+    g_free(dir);
+
+    return NULL;
+}
+
+static int update_header_sync(BlockDriverState *bs)
+{
+    int ret;
+
+    ret = qcow2_update_header(bs);
+    if (ret < 0) {
+        return ret;
+    }
+
+    ret = bdrv_flush(bs);
+    if (ret < 0) {
+        return ret;
+    }
+
+    return 0;
+}
+
+/* write bitmap directory from the state to new allocated clusters */
+static int64_t directory_write(BlockDriverState *bs, const uint8_t *dir,
+                               size_t size)
+{
+    int ret = 0;
+    uint8_t *dir_be = NULL;
+    int64_t dir_offset = 0;
+
+    dir_be = g_try_malloc(size);
+    if (dir_be == NULL) {
+        return -ENOMEM;
+    }
+    memcpy(dir_be, dir, size);
+    bitmap_directory_to_be(dir_be, size);
+
+    /* Allocate space for the new bitmap directory */
+    dir_offset = qcow2_alloc_clusters(bs, size);
+    if (dir_offset < 0) {
+        ret = dir_offset;
+        goto out;
+    }
+
+    /* The bitmap directory position has not yet been updated, so these
+     * clusters must indeed be completely free */
+    ret = qcow2_pre_write_overlap_check(bs, 0, dir_offset, size);
+    if (ret < 0) {
+        goto out;
+    }
+
+    ret = bdrv_pwrite(bs->file, dir_offset, dir_be, size);
+    if (ret < 0) {
+        goto out;
+    }
+
+out:
+    g_free(dir_be);
+
+    if (ret < 0) {
+        if (dir_offset > 0) {
+            qcow2_free_clusters(bs, dir_offset, size, QCOW2_DISCARD_ALWAYS);
+        }
+
+        return ret;
+    }
+
+    return dir_offset;
+}
+
+static int directory_update(BlockDriverState *bs, uint8_t *new_dir,
+                            size_t new_size, uint32_t new_nb_bitmaps)
+{
+    BDRVQcow2State *s = bs->opaque;
+    int ret;
+    int64_t new_offset = 0;
+    uint64_t old_offset = s->bitmap_directory_offset;
+    uint64_t old_size = s->bitmap_directory_size;
+    uint32_t old_nb_bitmaps = s->nb_bitmaps;
+    uint64_t old_autocl = s->autoclear_features;
+
+    if (new_size > QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE) {
+        return -EINVAL;
+    }
+
+    if ((new_size == 0) != (new_nb_bitmaps == 0)) {
+        return -EINVAL;
+    }
+
+    if (new_nb_bitmaps > 0) {
+        new_offset = directory_write(bs, new_dir, new_size);
+        if (new_offset < 0) {
+            return new_offset;
+        }
+
+        ret = bdrv_flush(bs);
+        if (ret < 0) {
+            goto fail;
+        }
+    }
+    s->bitmap_directory_offset = new_offset;
+    s->bitmap_directory_size = new_size;
+    s->nb_bitmaps = new_nb_bitmaps;
+
+    ret = update_header_sync(bs);
+    if (ret < 0) {
+        goto fail;
+    }
+
+    if (old_size) {
+        qcow2_free_clusters(bs, old_offset, old_size, QCOW2_DISCARD_ALWAYS);
+    }
+
+    return 0;
+
+fail:
+    if (new_offset > 0) {
+        qcow2_free_clusters(bs, new_offset, new_size, QCOW2_DISCARD_ALWAYS);
+        s->bitmap_directory_offset = old_offset;
+        s->bitmap_directory_size = old_size;
+        s->nb_bitmaps = old_nb_bitmaps;
+        s->autoclear_features = old_autocl;
+    }
+
+    return ret;
+}
+
+int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp)
+{
+    int ret;
+    BDRVQcow2State *s = bs->opaque;
+    uint8_t *dir, *new_dir, *new_pos;
+    uint64_t dir_size;
+    Qcow2BitmapDirEntry *e;
+    uint32_t new_nb_bitmaps = 0;
+
+    if (s->nb_bitmaps == 0) {
+        /* No bitmaps - nothing to do */
+        return 0;
+    }
+
+    new_pos = new_dir = g_try_malloc(s->bitmap_directory_size);
+    if (new_dir == NULL) {
+        error_setg(errp, "Can't allocate space for bitmap directory.");
+        return -ENOMEM;
+    }
+
+    dir_size = s->bitmap_directory_size;
+    dir = directory_read(bs, s->bitmap_directory_offset,
+                         s->bitmap_directory_size, errp);
+    if (dir == NULL) {
+        ret = -EINVAL;
+        goto out;
+    }
+
+    for_each_bitmap_dir_entry(e, dir, dir_size) {
+        uint32_t fl = e->flags;
+
+        if (fl & BME_FLAG_AUTO) {
+            BdrvDirtyBitmap *bitmap = load_bitmap(bs, e, errp);
+            if (bitmap == NULL) {
+                ret = -EINVAL;
+                goto out;
+            }
+        } else {
+            /* leave bitmap in the image */
+            new_pos += copy_dir_entry(new_pos, e);
+            new_nb_bitmaps++;
+        }
+    }
+
+    if (!can_write(bs)) {
+        goto out;
+    }
+
+    ret = directory_update(bs, new_dir, new_pos - new_dir, new_nb_bitmaps);
+    if (ret < 0) {
+        error_setg(errp, "Can't update bitmap directory.");
+        goto out;
+    }
+
+    /* to be consistent, free bitmap only after successfull directory update */
+    for_each_bitmap_dir_entry(e, dir, dir_size) {
+        uint32_t fl = e->flags;
+
+        if (fl & BME_FLAG_AUTO) {
+            free_bitmap_clusters(bs, e);
+        }
+    }
+
+out:
+    g_free(dir);
+    g_free(new_dir);
+
+    return ret;
+}
diff --git a/block/qcow2.c b/block/qcow2.c
index 08c4ef9..02ec224 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -213,6 +213,11 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
             s->bitmap_directory_size =
                     bitmaps_ext.bitmap_directory_size;
 
+            ret = qcow2_read_bitmaps(bs, errp);
+            if (ret < 0) {
+                return ret;
+            }
+
 #ifdef DEBUG_EXT
             printf("Qcow2: Got dirty bitmaps extension:"
                    " offset=%" PRIu64 " nb_bitmaps=%" PRIu32 "\n",
diff --git a/block/qcow2.h b/block/qcow2.h
index c068b2c..482a29f 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -603,6 +603,9 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs,
 void qcow2_free_snapshots(BlockDriverState *bs);
 int qcow2_read_snapshots(BlockDriverState *bs);
 
+/* qcow2-bitmap.c functions */
+int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp);
+
 /* qcow2-cache.c functions */
 Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables);
 int qcow2_cache_destroy(BlockDriverState* bs, Qcow2Cache *c);
-- 
1.8.3.1

[Qemu-devel] [PATCH 08/22] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

[Vladimir Sementsov-Ogievskiy]

Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be
needed in future, to save this flag back to Qcow2 for persistent
bitmaps.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/dirty-bitmap.c         | 15 +++++++++++++++
 block/qcow2-bitmap.c         |  2 ++
 include/block/dirty-bitmap.h |  2 ++
 3 files changed, 19 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 90af372..623e1d1 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -44,6 +44,7 @@ struct BdrvDirtyBitmap {
     int64_t size;               /* Size of the bitmap (Number of sectors) */
     bool disabled;              /* Bitmap is read-only */
     int active_iterators;       /* How many iterators are active */
+    bool autoload;              /* bitmap must be autoloaded on image opening */
     QLIST_ENTRY(BdrvDirtyBitmap) list;
 };
 
@@ -70,6 +71,8 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
     assert(!bdrv_dirty_bitmap_frozen(bitmap));
     g_free(bitmap->name);
     bitmap->name = NULL;
+
+    bitmap->autoload = false;
 }
 
 BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
@@ -238,6 +241,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
     bitmap->name = NULL;
     successor->name = name;
     bitmap->successor = NULL;
+    successor->autoload = bitmap->autoload;
+    bitmap->autoload = false;
     bdrv_release_dirty_bitmap(bs, bitmap);
 
     return successor;
@@ -540,3 +545,13 @@ int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap)
 {
     return hbitmap_count(bitmap->meta);
 }
+
+void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload)
+{
+    bitmap->autoload = autoload;
+}
+
+bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap)
+{
+    return bitmap->autoload;
+}
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 760a047..c086436 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -529,6 +529,8 @@ int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp)
                 ret = -EINVAL;
                 goto out;
             }
+
+            bdrv_dirty_bitmap_set_autoload(bitmap, true);
         } else {
             /* leave bitmap in the image */
             new_pos += copy_dir_entry(new_pos, e);
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 1e17729..45a389a 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -75,4 +75,6 @@ void bdrv_dirty_bitmap_deserialize_ones(BdrvDirtyBitmap *bitmap,
                                         bool finish);
 void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
 
+void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload);
+bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap);
 #endif
-- 
1.8.3.1

[Qemu-devel] [PATCH 09/22] block: introduce persistent dirty bitmaps

[Vladimir Sementsov-Ogievskiy]

New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
on bdrv_close, using format driver. Format driver should maintain bitmap
storing.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block.c                      | 30 ++++++++++++++++++++++++++++++
 block/dirty-bitmap.c         | 27 +++++++++++++++++++++++++++
 block/qcow2-bitmap.c         |  1 +
 include/block/block.h        |  2 ++
 include/block/block_int.h    |  2 ++
 include/block/dirty-bitmap.h |  6 ++++++
 6 files changed, 68 insertions(+)

diff --git a/block.c b/block.c
index 804e3d4..1cde03a 100644
--- a/block.c
+++ b/block.c
@@ -2196,6 +2196,7 @@ void bdrv_reopen_abort(BDRVReopenState *reopen_state)
 static void bdrv_close(BlockDriverState *bs)
 {
     BdrvAioNotifier *ban, *ban_next;
+    Error *local_err = NULL;
 
     assert(!bs->job);
     assert(!bs->refcnt);
@@ -2204,6 +2205,10 @@ static void bdrv_close(BlockDriverState *bs)
     bdrv_flush(bs);
     bdrv_drain(bs); /* in case flush left pending I/O */
 
+    bdrv_store_persistent_bitmaps(bs, &local_err);
+    if (local_err != NULL) {
+        error_report_err(local_err);
+    }
     bdrv_release_named_dirty_bitmaps(bs);
     assert(QLIST_EMPTY(&bs->dirty_bitmaps));
 
@@ -3969,3 +3974,28 @@ void bdrv_del_child(BlockDriverState *parent_bs, BdrvChild *child, Error **errp)
 
     parent_bs->drv->bdrv_del_child(parent_bs, child, errp);
 }
+
+void bdrv_store_persistent_bitmaps(BlockDriverState *bs, Error **errp)
+{
+    BlockDriver *drv = bs->drv;
+
+    if (!bdrv_has_persistent_bitmaps(bs)) {
+        return;
+    }
+
+    if (!drv) {
+        error_setg_errno(errp, ENOMEDIUM,
+                         "Can't store persistent bitmaps to %s",
+                         bdrv_get_device_or_node_name(bs));
+        return;
+    }
+
+    if (!drv->bdrv_store_persistent_bitmaps) {
+        error_setg_errno(errp, ENOTSUP,
+                         "Can't store persistent bitmaps to %s",
+                         bdrv_get_device_or_node_name(bs));
+        return;
+    }
+
+    drv->bdrv_store_persistent_bitmaps(bs, errp);
+}
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 623e1d1..0314581 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -44,6 +44,7 @@ struct BdrvDirtyBitmap {
     int64_t size;               /* Size of the bitmap (Number of sectors) */
     bool disabled;              /* Bitmap is read-only */
     int active_iterators;       /* How many iterators are active */
+    bool persistent;            /* bitmap must be saved to owner disk image */
     bool autoload;              /* bitmap must be autoloaded on image opening */
     QLIST_ENTRY(BdrvDirtyBitmap) list;
 };
@@ -72,6 +73,7 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
     g_free(bitmap->name);
     bitmap->name = NULL;
 
+    bitmap->persistent = false;
     bitmap->autoload = false;
 }
 
@@ -241,6 +243,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
     bitmap->name = NULL;
     successor->name = name;
     bitmap->successor = NULL;
+    successor->persistent = bitmap->persistent;
+    bitmap->persistent = false;
     successor->autoload = bitmap->autoload;
     bitmap->autoload = false;
     bdrv_release_dirty_bitmap(bs, bitmap);
@@ -555,3 +559,26 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap)
 {
     return bitmap->autoload;
 }
+
+void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
+                                                bool persistent)
+{
+    bitmap->persistent = persistent;
+}
+
+bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap)
+{
+    return bitmap->persistent;
+}
+
+bool bdrv_has_persistent_bitmaps(BlockDriverState *bs)
+{
+    BdrvDirtyBitmap *bm;
+    QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
+        if (bm->persistent) {
+            return true;
+        }
+    }
+
+    return false;
+}
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index c086436..81520cd 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -530,6 +530,7 @@ int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp)
                 goto out;
             }
 
+            bdrv_dirty_bitmap_set_persistance(bitmap, true);
             bdrv_dirty_bitmap_set_autoload(bitmap, true);
         } else {
             /* leave bitmap in the image */
diff --git a/include/block/block.h b/include/block/block.h
index 7edce5c..8457b60 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -532,4 +532,6 @@ void bdrv_add_child(BlockDriverState *parent, BlockDriverState *child,
                     Error **errp);
 void bdrv_del_child(BlockDriverState *parent, BdrvChild *child, Error **errp);
 
+void bdrv_store_persistent_bitmaps(BlockDriverState *bs, Error **errp);
+
 #endif
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 0ca6a78..a4a4816 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -320,6 +320,8 @@ struct BlockDriver {
     void (*bdrv_del_child)(BlockDriverState *parent, BdrvChild *child,
                            Error **errp);
 
+    void (*bdrv_store_persistent_bitmaps)(BlockDriverState *bs, Error **errp);
+
     QLIST_ENTRY(BlockDriver) list;
 };
 
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 45a389a..8dbd16b 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -77,4 +77,10 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
 
 void bdrv_dirty_bitmap_set_autoload(BdrvDirtyBitmap *bitmap, bool autoload);
 bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap);
+void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
+                                                bool persistent);
+bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap);
+
+bool bdrv_has_persistent_bitmaps(BlockDriverState *bs);
+
 #endif
-- 
1.8.3.1

[Qemu-devel] [PATCH 10/22] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/dirty-bitmap.c         | 7 +++++++
 include/block/dirty-bitmap.h | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 0314581..392d660 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -582,3 +582,10 @@ bool bdrv_has_persistent_bitmaps(BlockDriverState *bs)
 
     return false;
 }
+
+BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
+                                        BdrvDirtyBitmap *bitmap)
+{
+    return bitmap == NULL ? QLIST_FIRST(&bs->dirty_bitmaps) :
+                            QLIST_NEXT(bitmap, list);
+}
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 8dbd16b..d71edc4 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -81,6 +81,9 @@ void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
                                                 bool persistent);
 bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap);
 
+BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
+                                        BdrvDirtyBitmap *bitmap);
+
 bool bdrv_has_persistent_bitmaps(BlockDriverState *bs);
 
 #endif
-- 
1.8.3.1

[Qemu-devel] [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Vladimir Sementsov-Ogievskiy]

Realize block bitmap stroing interface, to allow qcow2 images store
persistent bitmaps.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-bitmap.c | 241 +++++++++++++++++++++++++++++++++++++++++++++++++++
 block/qcow2.c        |   2 +
 block/qcow2.h        |   2 +
 3 files changed, 245 insertions(+)

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 81520cd..a5be25a 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -27,6 +27,7 @@
 
 #include "qemu/osdep.h"
 #include "qapi/error.h"
+#include "qemu/cutils.h"
 
 #include "block/block_int.h"
 #include "block/qcow2.h"
@@ -96,6 +97,15 @@ static inline void bitmap_table_to_cpu(uint64_t *bitmap_table, size_t size)
     }
 }
 
+static inline void bitmap_table_to_be(uint64_t *bitmap_table, size_t size)
+{
+    size_t i;
+
+    for (i = 0; i < size; ++i) {
+        cpu_to_be64s(&bitmap_table[i]);
+    }
+}
+
 static inline int calc_dir_entry_size(size_t name_size, size_t extra_data_size)
 {
     return align_offset(sizeof(Qcow2BitmapDirEntry) +
@@ -564,3 +574,234 @@ out:
 
     return ret;
 }
+
+/* store_bitmap_data()
+ * Store bitmap to image, filling bitamp table accordingly.
+ */
+static int store_bitmap_data(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
+                             uint64_t *bitmap_table, uint32_t bitmap_table_size)
+{
+    int ret;
+    BDRVQcow2State *s = bs->opaque;
+    uint64_t sector, dsc;
+    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
+    int cl_size = s->cluster_size;
+    uint8_t *buf = NULL;
+    uint32_t tb_size =
+            size_to_clusters(s,
+                bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));
+
+    BdrvDirtyBitmapIter *dbi;
+
+    if (tb_size != bitmap_table_size) {
+        return -EINVAL;
+    }
+
+    memset(bitmap_table, 0, bitmap_table_size * sizeof(bitmap_table[0]));
+
+    dbi = bdrv_dirty_iter_new(bitmap, 0);
+    buf = g_malloc(cl_size);
+    dsc = dirty_sectors_in_cluster(s, bitmap);
+
+    while ((sector = bdrv_dirty_iter_next(dbi)) != -1) {
+        uint64_t cluster = sector / dsc;
+        sector = cluster * dsc;
+        uint64_t end = MIN(bm_size, sector + dsc);
+        uint64_t write_size =
+            bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sector);
+
+        int64_t off = qcow2_alloc_clusters(bs, cl_size);
+        if (off < 0) {
+            ret = off;
+            goto finish;
+        }
+        bitmap_table[cluster] = off;
+
+        bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end);
+        if (write_size < cl_size) {
+            memset(buf + write_size, 0, cl_size - write_size);
+        }
+
+        ret = bdrv_pwrite(bs->file, off, buf, cl_size);
+        if (ret < 0) {
+            goto finish;
+        }
+
+        if (end >= bm_size) {
+            break;
+        }
+
+        bdrv_set_dirty_iter(dbi, end);
+    }
+    ret = 0; /* writes */
+
+finish:
+    if (ret < 0) {
+        clear_bitmap_table(bs, bitmap_table, bitmap_table_size);
+    }
+    g_free(buf);
+    bdrv_dirty_iter_free(dbi);
+
+    return ret;
+}
+
+/* store_bitmap()
+ * Store bitmap to qcow2 and set bitmap_table. bitmap_table itself is not
+ * stored to qcow2.
+ */
+static int store_bitmap(BlockDriverState *bs,
+                        BdrvDirtyBitmap *bitmap,
+                        Qcow2BitmapDirEntry *entry)
+{
+    int ret;
+    BDRVQcow2State *s = bs->opaque;
+    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
+    const char *bm_name = bdrv_dirty_bitmap_name(bitmap);
+
+    uint64_t *tb;
+    int64_t tb_offset;
+    uint32_t tb_size =
+            size_to_clusters(s,
+                bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));
+
+    tb = g_try_new(uint64_t, tb_size);
+    if (tb == NULL) {
+        return -ENOMEM;
+    }
+
+    ret = store_bitmap_data(bs, bitmap, tb, tb_size);
+    if (ret < 0) {
+        g_free(tb);
+        return ret;
+    }
+
+    tb_offset = qcow2_alloc_clusters(bs, tb_size * sizeof(tb[0]));
+    if (tb_offset < 0) {
+        ret = tb_offset;
+        goto fail;
+    }
+
+    bitmap_table_to_be(tb, tb_size);
+    ret = bdrv_pwrite(bs->file, tb_offset, tb, tb_size * sizeof(tb[0]));
+    if (ret < 0) {
+        goto fail;
+    }
+
+    g_free(tb);
+
+    entry->bitmap_table_offset = tb_offset;
+    entry->bitmap_table_size = tb_size;
+    entry->flags = bdrv_dirty_bitmap_granularity(bitmap) ? BME_FLAG_AUTO : 0;
+    entry->type = BT_DIRTY_TRACKING_BITMAP;
+    entry->granularity_bits = ctz32(bdrv_dirty_bitmap_granularity(bitmap));
+    entry->name_size = strlen(bm_name);
+    entry->extra_data_size = 0;
+    memcpy(entry + 1, bm_name, entry->name_size);
+
+    return 0;
+
+fail:
+    clear_bitmap_table(bs, tb, tb_size);
+
+    if (tb_offset > 0) {
+        qcow2_free_clusters(bs, tb_offset, tb_size, QCOW2_DISCARD_ALWAYS);
+    }
+
+    g_free(tb);
+
+    return ret;
+}
+
+static Qcow2BitmapDirEntry *find_bitmap_by_name(uint8_t *bitmap_directory,
+                                                size_t size, const char *name)
+{
+    Qcow2BitmapDirEntry *e;
+
+    for_each_bitmap_dir_entry(e, bitmap_directory, size) {
+        if (strncmp((char *)(e + 1), name, e->name_size) == 0) {
+            return e;
+        }
+    }
+
+    return NULL;
+}
+
+void qcow2_store_persistent_bitmaps(BlockDriverState *bs, Error **errp)
+{
+    BdrvDirtyBitmap *bm;
+    BDRVQcow2State *s = bs->opaque;
+    uint32_t new_nb_bitmaps = s->nb_bitmaps;
+    uint64_t new_dir_size = s->bitmap_directory_size;
+    uint8_t *dir = NULL, *new_dir = NULL;
+    int ret;
+    Qcow2BitmapDirEntry *new_pos;
+
+    if (s->nb_bitmaps > 0) {
+        dir = directory_read(bs, s->bitmap_directory_offset,
+                             s->bitmap_directory_size, errp);
+        if (dir == NULL) {
+            goto out;
+        }
+    }
+
+    for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm != NULL;
+            bm = bdrv_dirty_bitmap_next(bs, bm)) {
+        const char *name = bdrv_dirty_bitmap_name(bm);
+
+        if (!bdrv_dirty_bitmap_get_persistance(bm)) {
+            continue;
+        }
+
+        if (s->nb_bitmaps > 0 &&
+                find_bitmap_by_name(dir, s->bitmap_directory_size, name)) {
+            error_setg(errp,
+                       "Can't store bitmap '%s' to '%s', as it already exists",
+                       name, bdrv_get_device_or_node_name(bs));
+            goto out;
+        }
+
+        new_nb_bitmaps++;
+        new_dir_size += calc_dir_entry_size(strlen(name), 0);
+    }
+
+    if (s->nb_bitmaps == new_nb_bitmaps) {
+        /* No new bitmaps - nothing to do */
+        goto out;
+    }
+
+    new_dir = g_try_malloc0(new_dir_size);
+    if (new_dir == NULL) {
+        error_setg(errp, "Can't allocate space for bitmap directory.");
+        goto out;
+    }
+
+    memcpy(new_dir, dir, s->bitmap_directory_size);
+    new_pos = (Qcow2BitmapDirEntry *)(new_dir + s->bitmap_directory_size);
+
+    for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm != NULL;
+            bm = bdrv_dirty_bitmap_next(bs, bm)) {
+        if (!bdrv_dirty_bitmap_get_persistance(bm)) {
+            continue;
+        }
+
+        ret = store_bitmap(bs, bm, new_pos);
+        if (ret < 0) {
+            error_setg_errno(errp, -ret, "Can't store bitmap '%s' to '%s'",
+                             bdrv_dirty_bitmap_name(bm),
+                             bdrv_get_device_or_node_name(bs));
+            goto out;
+        }
+        new_pos = next_dir_entry(new_pos);
+    }
+
+    ret = directory_update(bs, new_dir, new_dir_size, new_nb_bitmaps);
+    if (ret < 0) {
+        error_setg_errno(errp, -ret, "Can't update bitmap directory in '%s'",
+                         bdrv_get_device_or_node_name(bs));
+        goto out;
+    }
+
+out:
+    g_free(new_dir);
+    g_free(dir);
+}
diff --git a/block/qcow2.c b/block/qcow2.c
index 02ec224..8238205 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -3493,6 +3493,8 @@ BlockDriver bdrv_qcow2 = {
 
     .bdrv_detach_aio_context  = qcow2_detach_aio_context,
     .bdrv_attach_aio_context  = qcow2_attach_aio_context,
+
+    .bdrv_store_persistent_bitmaps = qcow2_store_persistent_bitmaps,
 };
 
 static void bdrv_qcow2_init(void)
diff --git a/block/qcow2.h b/block/qcow2.h
index 482a29f..dfcf4c6 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -627,4 +627,6 @@ int qcow2_cache_get_empty(BlockDriverState *bs, Qcow2Cache *c, uint64_t offset,
     void **table);
 void qcow2_cache_put(BlockDriverState *bs, Qcow2Cache *c, void **table);
 
+void qcow2_store_persistent_bitmaps(BlockDriverState *bs, Error **errp);
+
 #endif
-- 
1.8.3.1

[Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

This flag means that the bitmap is now in use by the software or was not
successfully saved. In any way, with this flag set the bitmap data must
be considered inconsistent and should not be loaded.

With current implementation this flag is never set, as we just remove
bitmaps from the image after loading. But it defined in qcow2 spec and
must be handled. Also, it can be used in future, if async schemes of
bitmap loading/saving are implemented.

We also remove in-use bitmaps from the image on open.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-bitmap.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index a5be25a..8cf40f0 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -28,6 +28,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "qemu/cutils.h"
+#include "exec/log.h"
 
 #include "block/block_int.h"
 #include "block/qcow2.h"
@@ -44,7 +45,8 @@
 #define BME_MAX_NAME_SIZE 1023
 
 /* Bitmap directory entry flags */
-#define BME_RESERVED_FLAGS 0xfffffffd
+#define BME_RESERVED_FLAGS 0xfffffffc
+#define BME_FLAG_IN_USE 1
 #define BME_FLAG_AUTO   (1U << 1)
 
 /* bits [1, 8] U [56, 63] are reserved */
@@ -287,6 +289,11 @@ static BdrvDirtyBitmap *load_bitmap(BlockDriverState *bs,
     BdrvDirtyBitmap *bitmap = NULL;
     char *name = g_strndup(dir_entry_name_notcstr(entry), entry->name_size);
 
+    if (entry->flags & BME_FLAG_IN_USE) {
+        error_setg(errp, "Bitmap '%s' is in use", name);
+        goto fail;
+    }
+
     ret = bitmap_table_load(bs, entry, &bitmap_table);
     if (ret < 0) {
         error_setg_errno(errp, -ret,
@@ -533,6 +540,14 @@ int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp)
     for_each_bitmap_dir_entry(e, dir, dir_size) {
         uint32_t fl = e->flags;
 
+        if (fl & BME_FLAG_IN_USE) {
+            qemu_log("qcow2 warning: "
+                     "removing in_use bitmap '%.*s' for image %s.\n",
+                     e->name_size, (char *)(e + 1), bdrv_get_device_or_node_name(bs));
+
+            continue;
+        }
+
         if (fl & BME_FLAG_AUTO) {
             BdrvDirtyBitmap *bitmap = load_bitmap(bs, e, errp);
             if (bitmap == NULL) {
-- 
1.8.3.1

[Qemu-devel] [PATCH 13/22] qcow2-bitmap: check constraints

[Vladimir Sementsov-Ogievskiy]

Check bitmap header constraints as specified in docs/specs/qcow2.txt

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-bitmap.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 8cf40f0..1c3abea 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -154,6 +154,34 @@ static inline void bitmap_directory_to_be(uint8_t *dir, size_t size)
     }
 }
 
+static int check_constraints(BlockDriverState *bs, Qcow2BitmapDirEntry *h)
+{
+    BDRVQcow2State *s = bs->opaque;
+    uint64_t phys_bitmap_bytes =
+        (uint64_t)h->bitmap_table_size * s->cluster_size;
+    uint64_t max_virtual_bits = (phys_bitmap_bytes * 8) << h->granularity_bits;
+    int64_t nb_sectors = bdrv_nb_sectors(bs);
+
+    if (nb_sectors < 0) {
+        return nb_sectors;
+    }
+
+    int fail =
+            ((h->bitmap_table_size == 0) != (h->bitmap_table_offset == 0)) ||
+            (h->bitmap_table_offset % s->cluster_size) ||
+            (h->bitmap_table_size > BME_MAX_TABLE_SIZE) ||
+            (phys_bitmap_bytes > BME_MAX_PHYS_SIZE) ||
+            (h->bitmap_table_offset != 0 &&
+                (nb_sectors << BDRV_SECTOR_BITS) > max_virtual_bits) ||
+            (h->granularity_bits > BME_MAX_GRANULARITY_BITS) ||
+            (h->granularity_bits < BME_MIN_GRANULARITY_BITS) ||
+            (h->flags & BME_RESERVED_FLAGS) ||
+            (h->name_size > BME_MAX_NAME_SIZE) ||
+            (h->type != BT_DIRTY_TRACKING_BITMAP);
+
+    return fail ? -EINVAL : 0;
+}
+
 static void clear_bitmap_table(BlockDriverState *bs, uint64_t *bitmap_table,
                                uint32_t bitmap_table_size)
 {
@@ -372,6 +400,12 @@ static uint8_t *directory_read(BlockDriverState *bs,
                        bdrv_get_device_or_node_name(bs));
             goto fail;
         }
+
+        ret = check_constraints(bs, e);
+        if (ret < 0) {
+            error_setg(errp, "Bitmap doesn't satisfy the constraints.");
+            goto fail;
+        }
     }
 
     assert((uint8_t *)e == dir_end);
@@ -713,6 +747,11 @@ static int store_bitmap(BlockDriverState *bs,
     entry->extra_data_size = 0;
     memcpy(entry + 1, bm_name, entry->name_size);
 
+    ret = check_constraints(bs, entry);
+    if (ret < 0) {
+        goto fail;
+    }
+
     return 0;
 
 fail:
-- 
1.8.3.1

[Qemu-devel] [PATCH 14/22] qcow2: delete bitmaps on truncate

[Vladimir Sementsov-Ogievskiy]

This will not touch loaded bitmaps, as they are alredy removed from the
image.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-bitmap.c | 38 ++++++++++++++++++++++++++++++++++++++
 block/qcow2.c        |  9 ++++++---
 block/qcow2.h        |  2 ++
 3 files changed, 46 insertions(+), 3 deletions(-)

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 1c3abea..2642afe 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -624,6 +624,44 @@ out:
     return ret;
 }
 
+
+int qcow2_delete_bitmaps(BlockDriverState *bs)
+{
+    int ret;
+    BDRVQcow2State *s = bs->opaque;
+    uint8_t *dir;
+    uint64_t dir_size;
+    Qcow2BitmapDirEntry *e;
+
+    if (s->nb_bitmaps == 0) {
+        /* No bitmaps - nothing to do */
+        return 0;
+    }
+
+    dir_size = s->bitmap_directory_size;
+    dir = directory_read(bs, s->bitmap_directory_offset,
+                         s->bitmap_directory_size, NULL);
+    if (dir == NULL) {
+        ret = -EINVAL;
+        goto out;
+    }
+
+    ret = directory_update(bs, NULL, 0, 0);
+    if (ret < 0) {
+        goto out;
+    }
+
+    /* to be consistent, free bitmap only after successfull directory update */
+    for_each_bitmap_dir_entry(e, dir, dir_size) {
+        free_bitmap_clusters(bs, e);
+    }
+
+out:
+    g_free(dir);
+
+    return ret;
+}
+
 /* store_bitmap_data()
  * Store bitmap to image, filling bitamp table accordingly.
  */
diff --git a/block/qcow2.c b/block/qcow2.c
index 8238205..aa967ed 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2592,9 +2592,12 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset)
 
     /* cannot proceed if image has bitmaps */
     if (s->nb_bitmaps) {
-        /* FIXME */
-        error_report("Can't resize an image which has bitmaps");
-        return -ENOTSUP;
+        /* FIXME: not loaded bitmaps will be lost */
+        ret = qcow2_delete_bitmaps(bs);
+        if (ret < 0) {
+            error_report("Can't remove bitmaps from qcow2 on truncate");
+            return ret;
+        }
     }
 
     /* shrinking is currently not supported */
diff --git a/block/qcow2.h b/block/qcow2.h
index dfcf4c6..af18efc 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -606,6 +606,8 @@ int qcow2_read_snapshots(BlockDriverState *bs);
 /* qcow2-bitmap.c functions */
 int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp);
 
+int qcow2_delete_bitmaps(BlockDriverState *bs);
+
 /* qcow2-cache.c functions */
 Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables);
 int qcow2_cache_destroy(BlockDriverState* bs, Qcow2Cache *c);
-- 
1.8.3.1

[Qemu-devel] [PATCH 15/22] qcow2-bitmap: add autoclear bit

[Vladimir Sementsov-Ogievskiy]

Add autoclear bit for handling rewriting image by old qemu version.

If autoclear bit is not set, but bitmaps extension is found it
would not be loaded and warning will be generated.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-bitmap.c |  4 ++++
 block/qcow2.c        | 12 ++++++++++--
 block/qcow2.h        |  9 +++++++++
 3 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 2642afe..76f7e2b 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -515,6 +515,10 @@ static int directory_update(BlockDriverState *bs, uint8_t *new_dir,
         if (ret < 0) {
             goto fail;
         }
+
+        s->autoclear_features |= QCOW2_AUTOCLEAR_DIRTY_BITMAPS;
+    } else {
+        s->autoclear_features &= ~(uint64_t)QCOW2_AUTOCLEAR_DIRTY_BITMAPS;
     }
     s->bitmap_directory_offset = new_offset;
     s->bitmap_directory_size = new_size;
diff --git a/block/qcow2.c b/block/qcow2.c
index aa967ed..373cf7e 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -165,6 +165,13 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
             break;
 
         case QCOW2_EXT_MAGIC_DIRTY_BITMAPS:
+            if (!(s->autoclear_features & QCOW2_AUTOCLEAR_DIRTY_BITMAPS)) {
+                fprintf(stderr,
+                        "WARNING: bitmaps_ext: autoclear flag is not "
+                        "set, all bitmaps will be considered as inconsistent");
+                break;
+            }
+
             ret = bdrv_pread(bs->file, offset, &bitmaps_ext, ext.len);
             if (ret < 0) {
                 error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
@@ -1206,8 +1213,9 @@ static int qcow2_open(BlockDriverState *bs, QDict *options, int flags,
     }
 
     /* Clear unknown autoclear feature bits */
-    if (!bs->read_only && !(flags & BDRV_O_INACTIVE) && s->autoclear_features) {
-        s->autoclear_features = 0;
+    if (!bs->read_only && !(flags & BDRV_O_INACTIVE) &&
+        (s->autoclear_features & ~QCOW2_AUTOCLEAR_MASK)) {
+        s->autoclear_features &= QCOW2_AUTOCLEAR_MASK;
         ret = qcow2_update_header(bs);
         if (ret < 0) {
             error_setg_errno(errp, -ret, "Could not update qcow2 header");
diff --git a/block/qcow2.h b/block/qcow2.h
index af18efc..a5e7592 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -215,6 +215,15 @@ enum {
     QCOW2_COMPAT_FEAT_MASK            = QCOW2_COMPAT_LAZY_REFCOUNTS,
 };
 
+/* Autoclear feature bits */
+enum {
+    QCOW2_AUTOCLEAR_DIRTY_BITMAPS_BITNR = 0,
+    QCOW2_AUTOCLEAR_DIRTY_BITMAPS       =
+        1 << QCOW2_AUTOCLEAR_DIRTY_BITMAPS_BITNR,
+
+    QCOW2_AUTOCLEAR_MASK                = QCOW2_AUTOCLEAR_DIRTY_BITMAPS,
+};
+
 enum qcow2_discard_type {
     QCOW2_DISCARD_NEVER = 0,
     QCOW2_DISCARD_ALWAYS,
-- 
1.8.3.1

[Qemu-devel] [PATCH 16/22] qmp: add persistent flag to block-dirty-bitmap-add

[Vladimir Sementsov-Ogievskiy]

Add optional 'persistent' flag to qmp command block-dirty-bitmap-add.
Default is false.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 blockdev.c           | 12 +++++++++++-
 qapi/block-core.json |  7 ++++++-
 qmp-commands.hx      |  5 ++++-
 3 files changed, 21 insertions(+), 3 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 97062e3..ec0ec75 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1991,6 +1991,7 @@ static void block_dirty_bitmap_add_prepare(BlkActionState *common,
     /* AIO context taken and released within qmp_block_dirty_bitmap_add */
     qmp_block_dirty_bitmap_add(action->node, action->name,
                                action->has_granularity, action->granularity,
+                               action->has_persistent, action->persistent,
                                &local_err);
 
     if (!local_err) {
@@ -2694,10 +2695,12 @@ out:
 
 void qmp_block_dirty_bitmap_add(const char *node, const char *name,
                                 bool has_granularity, uint32_t granularity,
+                                bool has_persistent, bool persistent,
                                 Error **errp)
 {
     AioContext *aio_context;
     BlockDriverState *bs;
+    BdrvDirtyBitmap *bitmap;
 
     if (!name || name[0] == '\0') {
         error_setg(errp, "Bitmap name cannot be empty");
@@ -2723,7 +2726,14 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name,
         granularity = bdrv_get_default_bitmap_granularity(bs);
     }
 
-    bdrv_create_dirty_bitmap(bs, granularity, name, errp);
+    if (!has_persistent) {
+        persistent = false;
+    }
+
+    bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp);
+    if (bitmap != NULL) {
+        bdrv_dirty_bitmap_set_persistance(bitmap, persistent);
+    }
 
  out:
     aio_context_release(aio_context);
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 31f9990..2bf56cd 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1235,10 +1235,15 @@
 # @granularity: #optional the bitmap granularity, default is 64k for
 #               block-dirty-bitmap-add
 #
+# @persistent: #optional the bitmap is persistent, i.e. it will be saved to
+#              corresponding block device on it's close. Default is false.
+#              For block-dirty-bitmap-add. (Since 2.8)
+#
 # Since 2.4
 ##
 { 'struct': 'BlockDirtyBitmapAdd',
-  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32' } }
+  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
+  '*persistent': 'bool' } }
 
 ##
 # @block-dirty-bitmap-add
diff --git a/qmp-commands.hx b/qmp-commands.hx
index ba2a916..434b418 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -1441,7 +1441,7 @@ EQMP
 
     {
         .name       = "block-dirty-bitmap-add",
-        .args_type  = "node:B,name:s,granularity:i?",
+        .args_type  = "node:B,name:s,granularity:i?,persistent:b?",
         .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
     },
 
@@ -1458,6 +1458,9 @@ Arguments:
 - "node": device/node on which to create dirty bitmap (json-string)
 - "name": name of the new dirty bitmap (json-string)
 - "granularity": granularity to track writes with (int, optional)
+- "persistent": bitmap will be saved to corresponding block device
+                on it's close. Block driver should maintain persistent bitmaps
+                (json-bool, optional, default false) (Since 2.8)
 
 Example:
 
-- 
1.8.3.1

[Qemu-devel] [PATCH 17/22] qmp: add autoload parameter to block-dirty-bitmap-add

[Vladimir Sementsov-Ogievskiy]

Optional. Default is false.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 blockdev.c           | 22 ++++++++++++++++++++--
 qapi/block-core.json |  7 ++++++-
 qmp-commands.hx      |  5 ++++-
 3 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index ec0ec75..00da7a1 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1992,6 +1992,7 @@ static void block_dirty_bitmap_add_prepare(BlkActionState *common,
     qmp_block_dirty_bitmap_add(action->node, action->name,
                                action->has_granularity, action->granularity,
                                action->has_persistent, action->persistent,
+                               action->has_autoload, action->autoload,
                                &local_err);
 
     if (!local_err) {
@@ -2696,6 +2697,7 @@ out:
 void qmp_block_dirty_bitmap_add(const char *node, const char *name,
                                 bool has_granularity, uint32_t granularity,
                                 bool has_persistent, bool persistent,
+                                bool has_autoload, bool autoload,
                                 Error **errp)
 {
     AioContext *aio_context;
@@ -2729,10 +2731,26 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name,
     if (!has_persistent) {
         persistent = false;
     }
+    if (!has_autoload) {
+        autoload = false;
+    }
+
+    if (autoload && !persistent) {
+        error_setg(errp, "Autoload flag must be used only for persistent"
+                         "bitmaps");
+        goto out;
+    }
 
     bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp);
-    if (bitmap != NULL) {
-        bdrv_dirty_bitmap_set_persistance(bitmap, persistent);
+    if (bitmap == NULL) {
+        goto out;
+    }
+
+    if (persistent) {
+        bdrv_dirty_bitmap_set_persistance(bitmap, true);
+        if (autoload) {
+            bdrv_dirty_bitmap_set_autoload(bitmap, true);
+        }
     }
 
  out:
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 2bf56cd..087a681 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1239,11 +1239,16 @@
 #              corresponding block device on it's close. Default is false.
 #              For block-dirty-bitmap-add. (Since 2.8)
 #
+# @autoload: #optional the bitmap will be autoloaded on it's storage image
+#            open. This flag is only for persistent bitmap and needed to inform
+#            block driver that bitmap should be autoloaded on the next image
+#            open. Default is false. For block-dirty-bitmap-add. (Since 2.8)
+#
 # Since 2.4
 ##
 { 'struct': 'BlockDirtyBitmapAdd',
   'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
-  '*persistent': 'bool' } }
+  '*persistent': 'bool', '*autoload': 'bool' } }
 
 ##
 # @block-dirty-bitmap-add
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 434b418..8f4e841 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -1441,7 +1441,7 @@ EQMP
 
     {
         .name       = "block-dirty-bitmap-add",
-        .args_type  = "node:B,name:s,granularity:i?,persistent:b?",
+        .args_type  = "node:B,name:s,granularity:i?,persistent:b?,autoload:b?",
         .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
     },
 
@@ -1461,6 +1461,9 @@ Arguments:
 - "persistent": bitmap will be saved to corresponding block device
                 on it's close. Block driver should maintain persistent bitmaps
                 (json-bool, optional, default false) (Since 2.8)
+- "autoload": only for persistent bitmaps. Bitmap will be autoloaded on it's
+              storage image open. (json-bool, optional, default false)
+              (Since 2.8)
 
 Example:
 
-- 
1.8.3.1

[Qemu-devel] [PATCH 18/22] qapi: add md5 checksum of last dirty bitmap level to query-block

[Vladimir Sementsov-Ogievskiy]

Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/dirty-bitmap.c   | 1 +
 include/qemu/hbitmap.h | 8 ++++++++
 qapi/block-core.json   | 5 ++++-
 util/hbitmap.c         | 8 ++++++++
 4 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 392d660..d0fe382 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -359,6 +359,7 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs)
         info->has_name = !!bm->name;
         info->name = g_strdup(bm->name);
         info->status = bdrv_dirty_bitmap_status(bm);
+        info->md5 = hbitmap_md5(bm->bitmap);
         entry->value = info;
         *plist = entry;
         plist = &entry->next;
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 64e5963..a47ce6a 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -240,6 +240,14 @@ void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count,
 void hbitmap_deserialize_finish(HBitmap *hb);
 
 /**
+ * hbitmap_md5:
+ * @bitmap: HBitmap to operate on.
+ *
+ * Returns md5 checksum of the last level.
+ */
+char *hbitmap_md5(const HBitmap *bitmap);
+
+/**
  * hbitmap_free:
  * @hb: HBitmap to operate on.
  *
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 087a681..8387e9c 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -419,11 +419,14 @@
 #
 # @status: current status of the dirty bitmap (since 2.4)
 #
+# @md5: md5 checksum (as a hexadecimal string) of the last bitmap level
+#       (since 2.8)
+#
 # Since: 1.3
 ##
 { 'struct': 'BlockDirtyInfo',
   'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32',
-           'status': 'DirtyBitmapStatus'} }
+           'status': 'DirtyBitmapStatus', 'md5': 'str'} }
 
 ##
 # @BlockInfo:
diff --git a/util/hbitmap.c b/util/hbitmap.c
index f49520f..1f0c417 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -707,3 +707,11 @@ void hbitmap_free_meta(HBitmap *hb)
     hbitmap_free(hb->meta);
     hb->meta = NULL;
 }
+
+char *hbitmap_md5(const HBitmap *bitmap)
+{
+    uint64_t size =
+        MAX((bitmap->size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+    const guchar *data = (const guchar *)bitmap->levels[HBITMAP_LEVELS - 1];
+    return g_compute_checksum_for_data(G_CHECKSUM_MD5, data, size);
+}
-- 
1.8.3.1

[Qemu-devel] [PATCH 19/22] iotests: test qcow2 persistent dirty bitmap

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/165     | 87 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/165.out |  5 +++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 93 insertions(+)
 create mode 100755 tests/qemu-iotests/165
 create mode 100644 tests/qemu-iotests/165.out

diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165
new file mode 100755
index 0000000..a69799c
--- /dev/null
+++ b/tests/qemu-iotests/165
@@ -0,0 +1,87 @@
+#!/usr/bin/env python
+#
+# Tests for persistent dirty bitmaps.
+#
+# Copyright: Vladimir Sementsov-Ogievskiy 2015-2016
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+from iotests import qemu_img
+
+disk = os.path.join(iotests.test_dir, 'disk')
+disk_size = 0x40000000 # 1G
+
+# regions for qemu_io: (start, count) in bytes
+regions1 = ((0,        0x100000),
+            (0x200000, 0x100000))
+
+regions2 = ((0x10000000, 0x20000),
+            (0x39990000, 0x10000))
+
+class TestPersistentDirtyBitmap(iotests.QMPTestCase):
+
+    def setUp(self):
+        qemu_img('create', '-f', iotests.imgfmt, disk, str(disk_size))
+
+    def tearDown(self):
+        os.remove(disk)
+
+    def mkVm(self):
+        return iotests.VM().add_drive(disk)
+
+    def getMd5(self):
+        result = self.vm.qmp('query-block');
+        return result['return'][0]['dirty-bitmaps'][0]['md5']
+
+    def checkBitmap(self, md5):
+        result = self.vm.qmp('query-block');
+        self.assert_qmp(result, 'return[0]/dirty-bitmaps[0]/md5', md5);
+
+    def writeRegions(self, regions):
+        for r in regions:
+          self.vm.hmp_qemu_io('drive0',
+                              'write %d %d' % r)
+
+    def qmpAddBitmap(self):
+        self.vm.qmp('block-dirty-bitmap-add', node='drive0',
+                    name='bitmap0', persistent=True, autoload=True)
+
+    def test_persistent(self):
+        self.vm = self.mkVm()
+        self.vm.launch()
+        self.qmpAddBitmap()
+
+        self.writeRegions(regions1)
+        md5 = self.getMd5()
+
+        self.vm.shutdown()
+        self.vm = self.mkVm()
+        self.vm.launch()
+
+        self.checkBitmap(md5)
+        self.writeRegions(regions2)
+        md5 = self.getMd5()
+
+        self.vm.shutdown()
+        self.vm.launch()
+
+        self.checkBitmap(md5)
+
+        self.vm.shutdown()
+
+if __name__ == '__main__':
+    iotests.main(supported_fmts=['qcow2'])
diff --git a/tests/qemu-iotests/165.out b/tests/qemu-iotests/165.out
new file mode 100644
index 0000000..ae1213e
--- /dev/null
+++ b/tests/qemu-iotests/165.out
@@ -0,0 +1,5 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 503eb27..ed14294 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -161,3 +161,4 @@
 159 rw auto quick
 160 rw auto quick
 162 auto quick
+165 rw auto quick
-- 
1.8.3.1

[Qemu-devel] [PATCH 20/22] qcow2-dirty-bitmap: refcounts

[Vladimir Sementsov-Ogievskiy]

Calculate refcounts for dirty bitmaps.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-bitmap.c   | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++
 block/qcow2-refcount.c | 14 +++++++-----
 block/qcow2.h          |  5 +++++
 3 files changed, 74 insertions(+), 5 deletions(-)

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 76f7e2b..6d9394a 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -901,3 +901,63 @@ out:
     g_free(new_dir);
     g_free(dir);
 }
+
+int qcow2_dirty_bitmaps_refcounts(BlockDriverState *bs,
+                                  BdrvCheckResult *res,
+                                  void **refcount_table,
+                                  int64_t *refcount_table_size)
+{
+    BDRVQcow2State *s = bs->opaque;
+    uint8_t *dir;
+    Qcow2BitmapDirEntry *e;
+    Error *local_err = NULL;
+
+    int i;
+    int ret = inc_refcounts(bs, res, refcount_table, refcount_table_size,
+                            s->bitmap_directory_offset,
+                            s->bitmap_directory_size);
+    if (ret < 0) {
+        return ret;
+    }
+
+    dir = directory_read(bs, s->bitmap_directory_offset,
+                         s->bitmap_directory_size, &local_err);
+    if (dir == NULL) {
+        error_report_err(local_err);
+        return -EINVAL;
+    }
+
+    for_each_bitmap_dir_entry(e, dir, s->bitmap_directory_size) {
+        uint64_t *bitmap_table = NULL;
+
+        ret = inc_refcounts(bs, res, refcount_table, refcount_table_size,
+                            e->bitmap_table_offset,
+                            e->bitmap_table_size);
+        if (ret < 0) {
+            return ret;
+        }
+
+        ret = bitmap_table_load(bs, e, &bitmap_table);
+        if (ret < 0) {
+            return ret;
+        }
+
+        for (i = 0; i < e->bitmap_table_size; ++i) {
+            uint64_t off = be64_to_cpu(bitmap_table[i]);
+            if (off <= 1) {
+                continue;
+            }
+
+            ret = inc_refcounts(bs, res, refcount_table, refcount_table_size,
+                                off, s->cluster_size);
+            if (ret < 0) {
+                g_free(bitmap_table);
+                return ret;
+            }
+        }
+
+        g_free(bitmap_table);
+    }
+
+    return 0;
+}
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index cbfb3fe..05bcc57 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -1309,11 +1309,9 @@ static int realloc_refcount_array(BDRVQcow2State *s, void **array,
  *
  * Modifies the number of errors in res.
  */
-static int inc_refcounts(BlockDriverState *bs,
-                         BdrvCheckResult *res,
-                         void **refcount_table,
-                         int64_t *refcount_table_size,
-                         int64_t offset, int64_t size)
+int inc_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
+                  void **refcount_table, int64_t *refcount_table_size,
+                  int64_t offset, int64_t size)
 {
     BDRVQcow2State *s = bs->opaque;
     uint64_t start, last, cluster_offset, k, refcount;
@@ -1843,6 +1841,12 @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
         return ret;
     }
 
+    /* dirty bitmaps */
+    ret = qcow2_dirty_bitmaps_refcounts(bs, res, refcount_table, nb_clusters);
+    if (ret < 0) {
+        return ret;
+    }
+
     return check_refblocks(bs, res, fix, rebuild, refcount_table, nb_clusters);
 }
 
diff --git a/block/qcow2.h b/block/qcow2.h
index a5e7592..0a050ea 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -563,6 +563,9 @@ int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset,
                                  int64_t size);
 int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset,
                                   int64_t size);
+int inc_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
+                  void **refcount_table, int64_t *refcount_table_size,
+                  int64_t offset, int64_t size);
 
 int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,
                                 BlockDriverAmendStatusCB *status_cb,
@@ -616,6 +619,8 @@ int qcow2_read_snapshots(BlockDriverState *bs);
 int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp);
 
 int qcow2_delete_bitmaps(BlockDriverState *bs);
+int qcow2_dirty_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
+    void **refcount_table, int64_t *refcount_table_size);
 
 /* qcow2-cache.c functions */
 Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables);
-- 
1.8.3.1

[Qemu-devel] [PATCH 21/22] specs/qcow2: fix bitmap granularity qemu-specific note

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 docs/specs/qcow2.txt | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
index 80cdfd0..d3e292f 100644
--- a/docs/specs/qcow2.txt
+++ b/docs/specs/qcow2.txt
@@ -472,8 +472,7 @@ Structure of a bitmap directory entry:
              17:    granularity_bits
                     Granularity bits. Valid values: 0 - 63.
 
-                    Note: Qemu currently doesn't support granularity_bits
-                    greater than 31.
+                    Note: Qemu currently support only values 9 - 31.
 
                     Granularity is calculated as
                         granularity = 1 << granularity_bits
-- 
1.8.3.1

[Qemu-devel] [PATCH 22/22] specs/qcow2: do not use wording 'bitmap header'

[Vladimir Sementsov-Ogievskiy]

For now bitmap directory entry some times called 'bitmap header'. This
patch leaves only one name - 'bitmap directory entry'. The name 'bitmap
header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap
header extension' (which is extension of qcow2 header)

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 docs/specs/qcow2.txt | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
index d3e292f..d5f88c1 100644
--- a/docs/specs/qcow2.txt
+++ b/docs/specs/qcow2.txt
@@ -201,7 +201,7 @@ The fields of the bitmaps extension are:
 
           8 - 15:  bitmap_directory_size
                    Size of the bitmap directory in bytes. It is the cumulative
-                   size of all (nb_bitmaps) bitmap headers.
+                   size of all (nb_bitmaps) bitmap directory entries.
 
          16 - 23:  bitmap_directory_offset
                    Offset into the image file at which the bitmap directory
@@ -426,8 +426,7 @@ Each bitmap saved in the image is described in a bitmap directory entry. The
 bitmap directory is a contiguous area in the image file, whose starting offset
 and length are given by the header extension fields bitmap_directory_offset and
 bitmap_directory_size. The entries of the bitmap directory have variable
-length, depending on the lengths of the bitmap name and extra data. These
-entries are also called bitmap headers.
+length, depending on the lengths of the bitmap name and extra data.
 
 Structure of a bitmap directory entry:
 
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v7 00/22] qcow2: persistent dirty bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 796 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> v7:
> https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v7
> based on block-next (https://github.com/XanClic/qemu/commits/block-next)

It should be noted that (at least my) block-next is only valid during
freeze, after that it becomes stale. I assume the patches you require
from my block-next branch are the ones from the "Dirty bitmap changes
for migration/persistence work" series, which I had to drop from my pull
requests, however, due to some issues on Big Endian machines.

The only reason they are still in my block-next branch is, as I said,
that that branch is just stale. I won't overwrite it for the time being,
though, so this series can still be applied on top.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 01/22] hbitmap: improve dirty iter

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 3596 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Make dirty iter resistant to resetting bits in corresponding HBitmap.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  include/qemu/hbitmap.h | 24 ++----------------------
>  util/hbitmap.c         | 23 ++++++++++++++++++++++-
>  2 files changed, 24 insertions(+), 23 deletions(-)
> 
> diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
> index eb46475..9aa86c1 100644
> --- a/include/qemu/hbitmap.h
> +++ b/include/qemu/hbitmap.h
> @@ -243,10 +243,7 @@ void hbitmap_free(HBitmap *hb);
>   * the lowest-numbered bit that is set in @hb, starting at @first.
>   *
>   * Concurrent setting of bits is acceptable, and will at worst cause the
> - * iteration to miss some of those bits.  Resetting bits before the current
> - * position of the iterator is also okay.  However, concurrent resetting of
> - * bits can lead to unexpected behavior if the iterator has not yet reached
> - * those bits.
> + * iteration to miss some of those bits. Concurrent bits resetting is ok too.

I'd prefer:

Concurrent resetting of bits is OK, too.

Or:

Concurrently resetting bits is OK, too.

Or:

Resetting bits concurrently is OK, too.

With or without that change:

Reviewed-by: Max Reitz <mreitz@redhat.com>

>   */
>  void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
>  
> @@ -285,24 +282,7 @@ void hbitmap_free_meta(HBitmap *hb);
>   * Return the next bit that is set in @hbi's associated HBitmap,
>   * or -1 if all remaining bits are zero.
>   */
> -static inline int64_t hbitmap_iter_next(HBitmapIter *hbi)
> -{
> -    unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1];
> -    int64_t item;
> -
> -    if (cur == 0) {
> -        cur = hbitmap_iter_skip_words(hbi);
> -        if (cur == 0) {
> -            return -1;
> -        }
> -    }
> -
> -    /* The next call will resume work from the next bit.  */
> -    hbi->cur[HBITMAP_LEVELS - 1] = cur & (cur - 1);
> -    item = ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur);
> -
> -    return item << hbi->granularity;
> -}
> +int64_t hbitmap_iter_next(HBitmapIter *hbi);
>  
>  /**
>   * hbitmap_iter_next_word:
> diff --git a/util/hbitmap.c b/util/hbitmap.c
> index 6a13c12..4f5cf96 100644
> --- a/util/hbitmap.c
> +++ b/util/hbitmap.c
> @@ -106,8 +106,9 @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi)
>  
>      unsigned long cur;
>      do {
> -        cur = hbi->cur[--i];
> +        i--;
>          pos >>= BITS_PER_LEVEL;
> +        cur = hbi->cur[i] & hb->levels[i][pos];
>      } while (cur == 0);
>  
>      /* Check for end of iteration.  We always use fewer than BITS_PER_LONG
> @@ -139,6 +140,26 @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi)
>      return cur;
>  }
>  
> +int64_t hbitmap_iter_next(HBitmapIter *hbi)
> +{
> +    unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1] &
> +            hbi->hb->levels[HBITMAP_LEVELS - 1][hbi->pos];
> +    int64_t item;
> +
> +    if (cur == 0) {
> +        cur = hbitmap_iter_skip_words(hbi);
> +        if (cur == 0) {
> +            return -1;
> +        }
> +    }
> +
> +    /* The next call will resume work from the next bit.  */
> +    hbi->cur[HBITMAP_LEVELS - 1] = cur & (cur - 1);
> +    item = ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur);
> +
> +    return item << hbi->granularity;
> +}
> +
>  void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first)
>  {
>      unsigned i, bit;
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 02/22] tests: add hbitmap iter test

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Test that hbitmap iter is resistant to bitmap resetting.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> ---
>  tests/test-hbitmap.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)

Reviewed-by: Max Reitz <mreitz@redhat.com>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 05/22] qcow2-bitmap: structs and consts

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 7296 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Create block/qcow2-bitmap.c
> Add data structures and constraints accordingly to docs/specs/qcow2.txt
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/Makefile.objs  |  2 +-
>  block/qcow2-bitmap.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++
>  block/qcow2.h        | 29 +++++++++++++++++++++++++++++
>  3 files changed, 77 insertions(+), 1 deletion(-)
>  create mode 100644 block/qcow2-bitmap.c
> 
> diff --git a/block/Makefile.objs b/block/Makefile.objs
> index fa4d8b8..0f661bb 100644
> --- a/block/Makefile.objs
> +++ b/block/Makefile.objs
> @@ -1,5 +1,5 @@
>  block-obj-y += raw_bsd.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o dmg.o
> -block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
> +block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o
>  block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
>  block-obj-y += qed-check.o
>  block-obj-$(CONFIG_VHDX) += vhdx.o vhdx-endian.o vhdx-log.o
> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> new file mode 100644
> index 0000000..cd18b07
> --- /dev/null
> +++ b/block/qcow2-bitmap.c
> @@ -0,0 +1,47 @@
> +/*
> + * Bitmaps for the QCOW version 2 format
> + *
> + * Copyright (c) 2014-2016 Vladimir Sementsov-Ogievskiy
> + *
> + * This file is derived from qcow2-snapshot.c, original copyright:
> + * Copyright (c) 2004-2006 Fabrice Bellard
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to deal
> + * in the Software without restriction, including without limitation the rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> + * THE SOFTWARE.
> + */
> +
> +/* NOTICE: BME here means Bitmaps Extension and used as a namespace for
> + * _internal_ constants. Please do not use this _internal_ abbreviation for
> + * other needs and/or outside of this file. */
> +
> +/* Bitmap directory entry constraints */
> +#define BME_MAX_TABLE_SIZE 0x8000000
> +#define BME_MAX_PHYS_SIZE 0x20000000 /* 512 mb */

I suppose BME_MAX_TABLE_SIZE (8M) is greater than BME_MAX_PHYS_SIZE (512
MB) divided by the cluster size (>= 512; 512 MB / cluster_size <= 1 MB)
because fully zero or one clusters do not require any physical space?

Makes some sense, but I can see that this might make give some trouble
when trying to serialize overly large bitmaps. But I guess that comes
later in this series, so I'll wait for that point.

Another thing is that 512 MB is rather big. It gets worse: The bitmap
may only require 512 MB on disk, but with a maximum table size of 8 MB,
it can require up to 8M * cluster_size in memory (with just 64 MB of
disk space!) by using the "read as all zeroes" or "read as all ones"
flags. With the default cluster size of 64 kB, this would be 512 GB in
RAM. That sounds bad to me.

Well, it is probably fine as long as the bitmap is not auto-loaded...
But we do have a flag for exactly that. So it seems to me that a
manipulated image can easily consume huge amounts of RAM on the host.

So I think we also need some sane limitation on the in-RAM size of a
bitmap (which is BME_MAX_TABLE_SIZE * cluster_size, as far as I
understand). The question of course is, what is sane? For a server
system with no image manipulation possible from the outside, 1 GB may be
completely fine. But imagine you download some qcow2 image to your
laptop. Then, 1 GB may not be fine, actually.

Maybe it would make sense to use a runtime-adjustable limit here?

> +#define BME_MAX_GRANULARITY_BITS 31
> +#define BME_MIN_GRANULARITY_BITS 9
> +#define BME_MAX_NAME_SIZE 1023
> +
> +/* Bitmap directory entry flags */
> +#define BME_RESERVED_FLAGS 0xffffffff
> +
> +/* bits [1, 8] U [56, 63] are reserved */
> +#define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001fe

ull suffix is missing.

> +
> +typedef enum BitmapType {
> +    BT_DIRTY_TRACKING_BITMAP = 1
> +} BitmapType;
> diff --git a/block/qcow2.h b/block/qcow2.h
> index b36a7bf..0480b8b 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -52,6 +52,10 @@
>   * space for snapshot names and IDs */
>  #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS)
>  
> +/* Bitmap header extension constraints */
> +#define QCOW_MAX_DIRTY_BITMAPS 65535
> +#define QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE (1024 * QCOW_MAX_DIRTY_BITMAPS)
> +
>  /* indicate that the refcount of the referenced cluster is exactly one. */
>  #define QCOW_OFLAG_COPIED     (1ULL << 63)
>  /* indicate that the cluster is compressed (they never have the copied flag) */
> @@ -142,6 +146,22 @@ typedef struct QEMU_PACKED QCowSnapshotHeader {
>      /* name follows  */
>  } QCowSnapshotHeader;
>  
> +/* Qcow2BitmapDirEntry is actually a bitmap directory entry */

This comment doesn't make a whole lot of sense now that this structure
is indeed called what it "actually is". ;-)

> +typedef struct QEMU_PACKED Qcow2BitmapDirEntry {
> +    /* header is 8 byte aligned */
> +    uint64_t bitmap_table_offset;
> +
> +    uint32_t bitmap_table_size;
> +    uint32_t flags;
> +
> +    uint8_t type;
> +    uint8_t granularity_bits;
> +    uint16_t name_size;
> +    uint32_t extra_data_size;
> +    /* extra data follows  */
> +    /* name follows  */
> +} Qcow2BitmapDirEntry;
> +
>  typedef struct QEMU_PACKED QCowSnapshotExtraData {
>      uint64_t vm_state_size_large;
>      uint64_t disk_size;
> @@ -222,6 +242,15 @@ typedef uint64_t Qcow2GetRefcountFunc(const void *refcount_array,
>  typedef void Qcow2SetRefcountFunc(void *refcount_array,
>                                    uint64_t index, uint64_t value);
>  
> +/* Be careful, Qcow2BitmapHeaderExt is not an extension of Qcow2BitmapDirEntry, it
> + * is Qcow2 header extension */

And this makes even less sense now.

(These comments don't stop me from giving an R-b, but I'm not so sure
about the constants...)

Max

> +typedef struct Qcow2BitmapHeaderExt {
> +    uint32_t nb_bitmaps;
> +    uint32_t reserved32;
> +    uint64_t bitmap_directory_size;
> +    uint64_t bitmap_directory_offset;
> +} QEMU_PACKED Qcow2BitmapHeaderExt;
> +
>  typedef struct BDRVQcow2State {
>      int cluster_bits;
>      int cluster_size;
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 06/22] qcow2: add dirty bitmaps extension

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 4147 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Add dirty bitmap extension as specified in docs/specs/qcow2.txt.
> For now, just mirror extension header into Qcow2 state and check
> constraints.
> 
> For now, disable image resize if it has bitmaps. It will be fixed later.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  block/qcow2.h |  4 +++
>  2 files changed, 87 insertions(+)
> 
> diff --git a/block/qcow2.c b/block/qcow2.c
> index c079aa8..08c4ef9 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c

[...]

> @@ -162,6 +164,62 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
>              }
>              break;
>  
> +        case QCOW2_EXT_MAGIC_DIRTY_BITMAPS:
> +            ret = bdrv_pread(bs->file, offset, &bitmaps_ext, ext.len);

Overflows with ext.len > sizeof(bitmaps_ext).

(ext.len < sizeof(bitmaps_ext) is also wrong, but less dramatically so.)

> +            if (ret < 0) {
> +                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
> +                                 "Could not read ext header");
> +                return ret;
> +            }
> +
> +            if (bitmaps_ext.reserved32 != 0) {
> +                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
> +                                 "Reserved field is not zero.");

Please drop the full stop at the end.

> +                return -EINVAL;
> +            }
> +
> +            be32_to_cpus(&bitmaps_ext.nb_bitmaps);
> +            be64_to_cpus(&bitmaps_ext.bitmap_directory_size);
> +            be64_to_cpus(&bitmaps_ext.bitmap_directory_offset);
> +
> +            if (bitmaps_ext.nb_bitmaps > QCOW_MAX_DIRTY_BITMAPS) {
> +                error_setg(errp, "ERROR: bitmaps_ext: "
> +                                 "too many dirty bitmaps");
> +                return -EINVAL;
> +            }
> +
> +            if (bitmaps_ext.nb_bitmaps == 0) {
> +                error_setg(errp, "ERROR: bitmaps_ext: "
> +                                 "found bitmaps extension with zero bitmaps");
> +                return -EINVAL;
> +            }
> +
> +            if (bitmaps_ext.bitmap_directory_offset & (s->cluster_size - 1)) {
> +                error_setg(errp, "ERROR: bitmaps_ext: "
> +                                 "wrong dirty bitmap directory offset");

s/wrong/invalid/

> +                return -EINVAL;
> +            }
> +
> +            if (bitmaps_ext.bitmap_directory_size >
> +                QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE) {
> +                error_setg(errp, "ERROR: bitmaps_ext: "
> +                                 "too large dirty bitmap directory");
> +                return -EINVAL;
> +            }
> +
> +            s->nb_bitmaps = bitmaps_ext.nb_bitmaps;
> +            s->bitmap_directory_offset =
> +                    bitmaps_ext.bitmap_directory_offset;
> +            s->bitmap_directory_size =
> +                    bitmaps_ext.bitmap_directory_size;
> +
> +#ifdef DEBUG_EXT
> +            printf("Qcow2: Got dirty bitmaps extension:"
> +                   " offset=%" PRIu64 " nb_bitmaps=%" PRIu32 "\n",
> +                   s->bitmap_directory_offset, s->nb_bitmaps);
> +#endif
> +            break;
> +
>          default:
>              /* unknown magic - save it in case we need to rewrite the header */
>              {

[...]

> @@ -2509,6 +2585,13 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset)
>          return -ENOTSUP;
>      }
>  
> +    /* cannot proceed if image has bitmaps */
> +    if (s->nb_bitmaps) {
> +        /* FIXME */

I'd call it a TODO, but that's probably a matter of taste.

> +        error_report("Can't resize an image which has bitmaps");
> +        return -ENOTSUP;
> +    }
> +
>      /* shrinking is currently not supported */
>      if (offset < bs->total_sectors * 512) {
>          error_report("qcow2 doesn't support shrinking images yet");

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 05/22] qcow2-bitmap: structs and consts

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 7615 bytes --]

On 01.10.2016 16:34, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Create block/qcow2-bitmap.c
>> Add data structures and constraints accordingly to docs/specs/qcow2.txt
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>  block/Makefile.objs  |  2 +-
>>  block/qcow2-bitmap.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++
>>  block/qcow2.h        | 29 +++++++++++++++++++++++++++++
>>  3 files changed, 77 insertions(+), 1 deletion(-)
>>  create mode 100644 block/qcow2-bitmap.c
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index fa4d8b8..0f661bb 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -1,5 +1,5 @@
>>  block-obj-y += raw_bsd.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o dmg.o
>> -block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
>> +block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o
>>  block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
>>  block-obj-y += qed-check.o
>>  block-obj-$(CONFIG_VHDX) += vhdx.o vhdx-endian.o vhdx-log.o
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> new file mode 100644
>> index 0000000..cd18b07
>> --- /dev/null
>> +++ b/block/qcow2-bitmap.c
>> @@ -0,0 +1,47 @@
>> +/*
>> + * Bitmaps for the QCOW version 2 format
>> + *
>> + * Copyright (c) 2014-2016 Vladimir Sementsov-Ogievskiy
>> + *
>> + * This file is derived from qcow2-snapshot.c, original copyright:
>> + * Copyright (c) 2004-2006 Fabrice Bellard
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to deal
>> + * in the Software without restriction, including without limitation the rights
>> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>> + * copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
>> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>> + * THE SOFTWARE.
>> + */
>> +
>> +/* NOTICE: BME here means Bitmaps Extension and used as a namespace for
>> + * _internal_ constants. Please do not use this _internal_ abbreviation for
>> + * other needs and/or outside of this file. */
>> +
>> +/* Bitmap directory entry constraints */
>> +#define BME_MAX_TABLE_SIZE 0x8000000
>> +#define BME_MAX_PHYS_SIZE 0x20000000 /* 512 mb */
> 
> I suppose BME_MAX_TABLE_SIZE (8M) is greater than BME_MAX_PHYS_SIZE (512
> MB) divided by the cluster size (>= 512; 512 MB / cluster_size <= 1 MB)
> because fully zero or one clusters do not require any physical space?
> 
> Makes some sense, but I can see that this might make give some trouble
> when trying to serialize overly large bitmaps. But I guess that comes
> later in this series, so I'll wait for that point.
> 
> Another thing is that 512 MB is rather big. It gets worse: The bitmap
> may only require 512 MB on disk, but with a maximum table size of 8 MB,
> it can require up to 8M * cluster_size in memory (with just 64 MB of
> disk space!) by using the "read as all zeroes" or "read as all ones"
> flags. With the default cluster size of 64 kB, this would be 512 GB in
> RAM. That sounds bad to me.
> 
> Well, it is probably fine as long as the bitmap is not auto-loaded...
> But we do have a flag for exactly that. So it seems to me that a
> manipulated image can easily consume huge amounts of RAM on the host.
> 
> So I think we also need some sane limitation on the in-RAM size of a
> bitmap (which is BME_MAX_TABLE_SIZE * cluster_size, as far as I
> understand). The question of course is, what is sane? For a server
> system with no image manipulation possible from the outside, 1 GB may be
> completely fine. But imagine you download some qcow2 image to your
> laptop. Then, 1 GB may not be fine, actually.
> 
> Maybe it would make sense to use a runtime-adjustable limit here?
> 
>> +#define BME_MAX_GRANULARITY_BITS 31
>> +#define BME_MIN_GRANULARITY_BITS 9
>> +#define BME_MAX_NAME_SIZE 1023
>> +
>> +/* Bitmap directory entry flags */
>> +#define BME_RESERVED_FLAGS 0xffffffff

Oh, and this should probably get a u suffix.

Max

>> +
>> +/* bits [1, 8] U [56, 63] are reserved */
>> +#define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001fe
> 
> ull suffix is missing.
> 
>> +
>> +typedef enum BitmapType {
>> +    BT_DIRTY_TRACKING_BITMAP = 1
>> +} BitmapType;
>> diff --git a/block/qcow2.h b/block/qcow2.h
>> index b36a7bf..0480b8b 100644
>> --- a/block/qcow2.h
>> +++ b/block/qcow2.h
>> @@ -52,6 +52,10 @@
>>   * space for snapshot names and IDs */
>>  #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS)
>>  
>> +/* Bitmap header extension constraints */
>> +#define QCOW_MAX_DIRTY_BITMAPS 65535
>> +#define QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE (1024 * QCOW_MAX_DIRTY_BITMAPS)
>> +
>>  /* indicate that the refcount of the referenced cluster is exactly one. */
>>  #define QCOW_OFLAG_COPIED     (1ULL << 63)
>>  /* indicate that the cluster is compressed (they never have the copied flag) */
>> @@ -142,6 +146,22 @@ typedef struct QEMU_PACKED QCowSnapshotHeader {
>>      /* name follows  */
>>  } QCowSnapshotHeader;
>>  
>> +/* Qcow2BitmapDirEntry is actually a bitmap directory entry */
> 
> This comment doesn't make a whole lot of sense now that this structure
> is indeed called what it "actually is". ;-)
> 
>> +typedef struct QEMU_PACKED Qcow2BitmapDirEntry {
>> +    /* header is 8 byte aligned */
>> +    uint64_t bitmap_table_offset;
>> +
>> +    uint32_t bitmap_table_size;
>> +    uint32_t flags;
>> +
>> +    uint8_t type;
>> +    uint8_t granularity_bits;
>> +    uint16_t name_size;
>> +    uint32_t extra_data_size;
>> +    /* extra data follows  */
>> +    /* name follows  */
>> +} Qcow2BitmapDirEntry;
>> +
>>  typedef struct QEMU_PACKED QCowSnapshotExtraData {
>>      uint64_t vm_state_size_large;
>>      uint64_t disk_size;
>> @@ -222,6 +242,15 @@ typedef uint64_t Qcow2GetRefcountFunc(const void *refcount_array,
>>  typedef void Qcow2SetRefcountFunc(void *refcount_array,
>>                                    uint64_t index, uint64_t value);
>>  
>> +/* Be careful, Qcow2BitmapHeaderExt is not an extension of Qcow2BitmapDirEntry, it
>> + * is Qcow2 header extension */
> 
> And this makes even less sense now.
> 
> (These comments don't stop me from giving an R-b, but I'm not so sure
> about the constants...)
> 
> Max
> 
>> +typedef struct Qcow2BitmapHeaderExt {
>> +    uint32_t nb_bitmaps;
>> +    uint32_t reserved32;
>> +    uint64_t bitmap_directory_size;
>> +    uint64_t bitmap_directory_offset;
>> +} QEMU_PACKED Qcow2BitmapHeaderExt;
>> +
>>  typedef struct BDRVQcow2State {
>>      int cluster_bits;
>>      int cluster_size;
>>
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 19510 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are

*with the AUTO flag set

> loaded at image open and becomes BdrvDirtyBitmap's for corresponding

"loaded when the image is opened and become BdrvDirtyBitmaps for the
corresponding drive."

> drive. These bitmaps are deleted from Qcow2 image after loading to avoid

*from the Qcow2 image

> conflicts.
> 
> Extra data in bitmaps is not supported for now.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c | 518 ++++++++++++++++++++++++++++++++++++++++++++++++++-
>  block/qcow2.c        |   5 +
>  block/qcow2.h        |   3 +
>  3 files changed, 525 insertions(+), 1 deletion(-)
> 
> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> index cd18b07..760a047 100644
> --- a/block/qcow2-bitmap.c
> +++ b/block/qcow2-bitmap.c
> @@ -25,6 +25,12 @@

[...]

> +static void clear_bitmap_table(BlockDriverState *bs, uint64_t *bitmap_table,
> +                               uint32_t bitmap_table_size)
> +{
> +    BDRVQcow2State *s = bs->opaque;
> +    int cl_size = s->cluster_size;
> +    int i;
> +
> +    for (i = 0; i < bitmap_table_size; ++i) {
> +        uint64_t addr = bitmap_table[i];
> +        if (addr <= 1) {

I'd rather add a BME_TABLE_ENTRY_OFFSET_MASK
(0x00fffffffffffe00ull) and then use:

uint64_t addr = bitmap_table[i] & BME_TABLE_ENTRY_OFFSET_MASK;
if (!addr) {

> +            continue;
> +        }
> +
> +        qcow2_free_clusters(bs, addr, cl_size, QCOW2_DISCARD_ALWAYS);

I think this should rather be QCOW2_DISCARD_OTHER; or you introduce a
new QCOW2_DISCARD_* flag.

(The same applies to all the other QCOW2_DISCARD_ALWAYS users here.)

Also, I don't really see the point of introducing cl_size just for this
place; it doesn't even save you from a line wrap.

> +        bitmap_table[i] = 0;
> +    }
> +}
> +
> +static int bitmap_table_load(BlockDriverState *bs, Qcow2BitmapDirEntry *entry,
> +                             uint64_t **table)
> +{
> +    int ret;
> +
> +    *table = g_try_new(uint64_t, entry->bitmap_table_size);
> +    if (*table == NULL) {

Not that g_try_new() will return NULL if you try to allocate a buffer
with size 0.

Normally, entry->bitmap_table_size shouldn't be 0, but I don't think
you're catching that case anywhere.

> +        return -ENOMEM;
> +    }
> +

I wouldn't mind an

assert(entry->bitmap_table_size <= UINT32_MAX / sizeof(uint64_t));

here because of the following multiplication (which is extended to 64
bit on 64 bit machines, but stays in 32 bit on 32 bit machines).

(Of course,

assert(entry->bitmap_table_size <= BME_MAX_TABLE_SIZE);

would be fine, too.)

> +    ret = bdrv_pread(bs->file, entry->bitmap_table_offset,
> +                     *table, entry->bitmap_table_size * sizeof(uint64_t));
> +    if (ret < 0) {
> +        g_free(*table);
> +        *table = NULL;
> +        return ret;
> +    }
> +
> +    bitmap_table_to_cpu(*table, entry->bitmap_table_size);
> +
> +    return 0;
> +}
> +
> +static void do_free_bitmap_clusters(BlockDriverState *bs,
> +                                    uint64_t table_offset,
> +                                    uint32_t table_size,
> +                                    uint64_t *bitmap_table)
> +{
> +    clear_bitmap_table(bs, bitmap_table, table_size);
> +
> +    qcow2_free_clusters(bs, table_offset, table_size * sizeof(uint64_t),
> +                        QCOW2_DISCARD_ALWAYS);
> +}
> +
> +static int free_bitmap_clusters(BlockDriverState *bs, Qcow2BitmapDirEntry *entry)
> +{
> +    int ret;
> +    uint64_t *bitmap_table;
> +
> +    ret = bitmap_table_load(bs, entry, &bitmap_table);
> +    if (ret < 0 || bitmap_table == NULL) {

That should be:

if (ret < 0) {
    assert(bitmap_table == NULL);

Or the other way around:

if (bitmap_table == NULL) {
    assert(ret < 0);

Otherwise, it looks like bitmap_table may be NULL with ret being 0, and
the next statement would make this function return success even though
it actually failed.

> +        return ret;
> +    }
> +
> +    do_free_bitmap_clusters(bs, entry->bitmap_table_offset,
> +                            entry->bitmap_table_size, bitmap_table);
> +
> +    return 0;

You're leaking bitmap_table here.

> +}
> +
> +/* dirty sectors in cluster is a number of sectors in the image, corresponding
> + * to one cluster of bitmap data */

How about:

This function returns the number of sectors covered by a single cluster
of dirty bitmap data.

> +static uint64_t dirty_sectors_in_cluster(const BDRVQcow2State *s,
> +                                         const BdrvDirtyBitmap *bitmap)
> +{
> +    uint32_t sector_granularity =
> +            bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS;
> +
> +    return (uint64_t)sector_granularity * (s->cluster_size << 3);
> +}
> +
> +static int load_bitmap_data(BlockDriverState *bs,
> +                            const uint64_t *dirty_bitmap_table,
> +                            uint32_t dirty_bitmap_table_size,
> +                            BdrvDirtyBitmap *bitmap)
> +{
> +    int ret = 0;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint64_t sector, dsc;
> +    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
> +    int cl_size = s->cluster_size;

Once more, I don't really see the reason to put this into a dedicated
variable (I feel like it makes the code harder to read because you have
to keep in mind what cl_size is).

> +    uint8_t *buf = NULL;
> +    uint32_t i, tab_size =
> +            size_to_clusters(s,
> +                bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));
> +
> +    if (tab_size != dirty_bitmap_table_size) {
> +        return -EINVAL;
> +    }

You should also check that tab_size does not exceed BME_MAX_TABLE_SIZE.

And maybe you should check that the physical size of the bitmap data
does not exceed BME_MAX_PHYS_SIZE, but I'm not so sure why you have that
constant at all because it's basically superseded by the existence of
BME_MAX_TABLE_SIZE.

> +
> +    bdrv_clear_dirty_bitmap(bitmap, NULL);
> +
> +    buf = g_malloc0(cl_size);

g_malloc() would suffice as well, since you'll be overwriting all of it
anyway.

> +    dsc = dirty_sectors_in_cluster(s, bitmap);
> +    for (i = 0, sector = 0; i < tab_size; ++i, sector += dsc) {
> +        uint64_t end = MIN(bm_size, sector + dsc);
> +        uint64_t offset = dirty_bitmap_table[i];
> +
> +        if (offset == 1) {
> +            bdrv_dirty_bitmap_deserialize_ones(bitmap, sector, end, false);
> +        } else if (offset > 1) {

As before, I'd organize this differently, e.g. like:

uint64_t table_entry = dirty_bitmap_table[i];
uint64_t offset = table_entry & BME_TABLE_ENTRY_OFFSET_MASK;

if (table_entry & BME_TABLE_ENTRY_RESERVED_MASK) {
    ret = -EINVAL;
    goto finish;
}

if (!offset) {
    if (table_entry & 1) {
        bdrv_dirty_bitmap_deserialize_ones(bitmap, sector, end, false);
    } else {
        /* No need to deserialize zeroes because the dirty bitmap is
         * already cleared */
    }
} else {
    ret = bdrv_pread(...);
    ...
}

It's more code, yes, but I find it easier to read because it reflects
the specification better.

> +            ret = bdrv_pread(bs->file, offset, buf, cl_size);
> +            if (ret < 0) {
> +                goto finish;
> +            }
> +            bdrv_dirty_bitmap_deserialize_part(bitmap, buf, sector, end, false);
> +        }
> +    }
> +    ret = 0;
> +
> +    bdrv_dirty_bitmap_deserialize_finish(bitmap);
> +
> +finish:
> +    g_free(buf);
> +
> +    return ret;
> +}
> +
> +static BdrvDirtyBitmap *load_bitmap(BlockDriverState *bs,
> +                                    Qcow2BitmapDirEntry *entry, Error **errp)
> +{
> +    int ret;
> +    uint64_t *bitmap_table = NULL;
> +    uint32_t granularity;
> +    BdrvDirtyBitmap *bitmap = NULL;
> +    char *name = g_strndup(dir_entry_name_notcstr(entry), entry->name_size);

Maybe you should limit entry->name_size to BME_MAX_NAME_SIZE.

> +
> +    ret = bitmap_table_load(bs, entry, &bitmap_table);
> +    if (ret < 0) {
> +        error_setg_errno(errp, -ret,
> +                         "Could not read bitmap_table table from image");
> +        goto fail;
> +    }
> +
> +    granularity = 1U << entry->granularity_bits;

You should be making sure that entry->granularity_bits does not exceed
[BME_MIN_GRANULARITY_BITS, BME_MAX_GRANULARITY_BITS] before this point.

> +    bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp);
> +    if (bitmap == NULL) {
> +        goto fail;
> +    }
> +
> +    ret = load_bitmap_data(bs, bitmap_table, entry->bitmap_table_size,
> +                           bitmap);
> +    if (ret < 0) {
> +        error_setg_errno(errp, -ret, "Could not read bitmap from image");
> +        goto fail;
> +    }
> +
> +    g_free(name);
> +    g_free(bitmap_table);
> +    return bitmap;
> +
> +fail:
> +    g_free(name);
> +    g_free(bitmap_table);
> +    if (bitmap != NULL) {
> +        bdrv_release_dirty_bitmap(bs, bitmap);
> +    }
> +
> +    return NULL;
> +}
> +
> +/* directory_read
> + * Read bitmaps directory from bs by @offset and @size. Convert it to cpu
> + * format from BE.
> + */
> +static uint8_t *directory_read(BlockDriverState *bs,
> +                               uint64_t offset, uint64_t size, Error **errp)
> +{
> +    int ret;
> +    uint8_t *dir, *dir_end;
> +    Qcow2BitmapDirEntry *e;

I think you should return an error if size >
QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE.

> +
> +    dir = g_try_malloc(size);
> +    if (dir == NULL) {
> +        error_setg(errp, "Can't allocate space for bitmap directory.");

Please drop the full stop at the end of the message.

(And I personally don't like contractions ("can't") in user-facing
messages too much, but that is just my taste.)

> +        return NULL;
> +    }
> +    dir_end = dir + size;
> +
> +    ret = bdrv_pread(bs->file, offset, dir, size);
> +    if (ret < 0) {
> +        error_setg_errno(errp, -ret, "Can't read bitmap directory.");

Again, no full stop at the end of error messages, please.

> +        goto fail;
> +    }
> +
> +    /* loop is safe because next entry offset is calculated after conversion to

Should it be s/safe/unsafe/?

> +     * cpu format */
> +    for_each_bitmap_dir_entry_unsafe(e, dir, size) {
> +        if ((uint8_t *)(e + 1) > dir_end) {
> +            goto broken_dir;
> +        }
> +
> +        bitmap_dir_entry_to_cpu(e);
> +
> +        if ((uint8_t *)next_dir_entry(e) > dir_end) {
> +            goto broken_dir;
> +        }
> +
> +        if (e->extra_data_size != 0) {
> +            error_setg(errp, "Can't load bitmap '%.*s' from '%s':"
> +                       "extra data not supported.", e->name_size,

Full stop again.

Also, I'm not quite sure why you give the device/node name here. You
don't do that anywhere else and I think if we want to emit the
information where something failed, it should be added at some previous
point in the call chain.

> +                       dir_entry_name_notcstr(e),
> +                       bdrv_get_device_or_node_name(bs));
> +            goto fail;
> +        }
> +    }

Maybe you should count the number of dirty bitmaps and return an error
if it exceeds QCOW_MAX_DIRTY_BITMAPS.

> +
> +    assert((uint8_t *)e == dir_end);

I think this should rather go to broken_dir if they're not equal.

> +
> +    return dir;
> +
> +broken_dir:
> +    error_setg(errp, "Broken bitmap directory.");

And again.

> +
> +fail:
> +    g_free(dir);
> +
> +    return NULL;
> +}
> +
> +static int update_header_sync(BlockDriverState *bs)
> +{
> +    int ret;
> +
> +    ret = qcow2_update_header(bs);
> +    if (ret < 0) {
> +        return ret;
> +    }
> +
> +    ret = bdrv_flush(bs);
> +    if (ret < 0) {
> +        return ret;
> +    }
> +
> +    return 0;
> +}
> +
> +/* write bitmap directory from the state to new allocated clusters */
> +static int64_t directory_write(BlockDriverState *bs, const uint8_t *dir,
> +                               size_t size)
> +{
> +    int ret = 0;
> +    uint8_t *dir_be = NULL;
> +    int64_t dir_offset = 0;
> +
> +    dir_be = g_try_malloc(size);
> +    if (dir_be == NULL) {
> +        return -ENOMEM;
> +    }
> +    memcpy(dir_be, dir, size);
> +    bitmap_directory_to_be(dir_be, size);
> +
> +    /* Allocate space for the new bitmap directory */
> +    dir_offset = qcow2_alloc_clusters(bs, size);
> +    if (dir_offset < 0) {
> +        ret = dir_offset;
> +        goto out;
> +    }
> +
> +    /* The bitmap directory position has not yet been updated, so these
> +     * clusters must indeed be completely free */
> +    ret = qcow2_pre_write_overlap_check(bs, 0, dir_offset, size);
> +    if (ret < 0) {
> +        goto out;
> +    }
> +
> +    ret = bdrv_pwrite(bs->file, dir_offset, dir_be, size);
> +    if (ret < 0) {
> +        goto out;
> +    }
> +
> +out:
> +    g_free(dir_be);
> +
> +    if (ret < 0) {
> +        if (dir_offset > 0) {
> +            qcow2_free_clusters(bs, dir_offset, size, QCOW2_DISCARD_ALWAYS);
> +        }
> +
> +        return ret;
> +    }
> +
> +    return dir_offset;
> +}
> +
> +static int directory_update(BlockDriverState *bs, uint8_t *new_dir,
> +                            size_t new_size, uint32_t new_nb_bitmaps)
> +{
> +    BDRVQcow2State *s = bs->opaque;
> +    int ret;
> +    int64_t new_offset = 0;
> +    uint64_t old_offset = s->bitmap_directory_offset;
> +    uint64_t old_size = s->bitmap_directory_size;
> +    uint32_t old_nb_bitmaps = s->nb_bitmaps;
> +    uint64_t old_autocl = s->autoclear_features;
> +
> +    if (new_size > QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE) {
> +        return -EINVAL;
> +    }

Also, you should probably check new_nb_bitmaps against
QCOW_MAX_DIRTY_BITMAPS, but maybe you'll be adding that check somewhere
else in a future patch. In that case, an assertion wouldn't be too bad
here, though.

> +
> +    if ((new_size == 0) != (new_nb_bitmaps == 0)) {
> +        return -EINVAL;
> +    }
> +
> +    if (new_nb_bitmaps > 0) {
> +        new_offset = directory_write(bs, new_dir, new_size);
> +        if (new_offset < 0) {
> +            return new_offset;
> +        }
> +
> +        ret = bdrv_flush(bs);
> +        if (ret < 0) {
> +            goto fail;
> +        }
> +    }
> +    s->bitmap_directory_offset = new_offset;
> +    s->bitmap_directory_size = new_size;
> +    s->nb_bitmaps = new_nb_bitmaps;
> +
> +    ret = update_header_sync(bs);
> +    if (ret < 0) {
> +        goto fail;
> +    }
> +
> +    if (old_size) {
> +        qcow2_free_clusters(bs, old_offset, old_size, QCOW2_DISCARD_ALWAYS);
> +    }
> +
> +    return 0;
> +
> +fail:
> +    if (new_offset > 0) {
> +        qcow2_free_clusters(bs, new_offset, new_size, QCOW2_DISCARD_ALWAYS);
> +        s->bitmap_directory_offset = old_offset;
> +        s->bitmap_directory_size = old_size;
> +        s->nb_bitmaps = old_nb_bitmaps;
> +        s->autoclear_features = old_autocl;

Why are you restoring the autoclear features? From a quick glance I
can't see any code path that changes this field here, and if there is
one, it probably has a good reason to do so.

> +    }
> +
> +    return ret;
> +}
> +
> +int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp)
> +{
> +    int ret;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint8_t *dir, *new_dir, *new_pos;
> +    uint64_t dir_size;

I suggest initializing this variable here (to s->bitmap_directory_size)...

> +    Qcow2BitmapDirEntry *e;
> +    uint32_t new_nb_bitmaps = 0;
> +
> +    if (s->nb_bitmaps == 0) {
> +        /* No bitmaps - nothing to do */
> +        return 0;
> +    }
> +
> +    new_pos = new_dir = g_try_malloc(s->bitmap_directory_size);

...and then using it here...

> +    if (new_dir == NULL) {
> +        error_setg(errp, "Can't allocate space for bitmap directory.");
> +        return -ENOMEM;
> +    }
> +
> +    dir_size = s->bitmap_directory_size;
> +    dir = directory_read(bs, s->bitmap_directory_offset,
> +                         s->bitmap_directory_size, errp);

...and here.

Alternatively, of course, you can just drop the variable and use
s->bitmap_directory_size everywhere.

> +    if (dir == NULL) {
> +        ret = -EINVAL;
> +        goto out;
> +    }
> +
> +    for_each_bitmap_dir_entry(e, dir, dir_size) {
> +        uint32_t fl = e->flags;
> +
> +        if (fl & BME_FLAG_AUTO) {
> +            BdrvDirtyBitmap *bitmap = load_bitmap(bs, e, errp);
> +            if (bitmap == NULL) {
> +                ret = -EINVAL;
> +                goto out;
> +            }
> +        } else {
> +            /* leave bitmap in the image */
> +            new_pos += copy_dir_entry(new_pos, e);
> +            new_nb_bitmaps++;
> +        }
> +    }
> +
> +    if (!can_write(bs)) {

You should be setting ret here (gcc complains about that).

> +        goto out;
> +    }
> +
> +    ret = directory_update(bs, new_dir, new_pos - new_dir, new_nb_bitmaps);
> +    if (ret < 0) {
> +        error_setg(errp, "Can't update bitmap directory.");

And once more, please no full stops at the end of error messages.

> +        goto out;
> +    }
> +
> +    /* to be consistent, free bitmap only after successfull directory update */

*successful

> +    for_each_bitmap_dir_entry(e, dir, dir_size) {
> +        uint32_t fl = e->flags;
> +
> +        if (fl & BME_FLAG_AUTO) {
> +            free_bitmap_clusters(bs, e);
> +        }
> +    }
> +
> +out:
> +    g_free(dir);
> +    g_free(new_dir);
> +
> +    return ret;
> +}
> diff --git a/block/qcow2.c b/block/qcow2.c
> index 08c4ef9..02ec224 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c
> @@ -213,6 +213,11 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
>              s->bitmap_directory_size =
>                      bitmaps_ext.bitmap_directory_size;
>  
> +            ret = qcow2_read_bitmaps(bs, errp);
> +            if (ret < 0) {
> +                return ret;
> +            }
> +

I think I'd put this directly into qcow2_open(), just like
qcow2_read_snapshots(); but that's an optional suggestion.

Max

>  #ifdef DEBUG_EXT
>              printf("Qcow2: Got dirty bitmaps extension:"
>                     " offset=%" PRIu64 " nb_bitmaps=%" PRIu32 "\n",
> diff --git a/block/qcow2.h b/block/qcow2.h
> index c068b2c..482a29f 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -603,6 +603,9 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs,
>  void qcow2_free_snapshots(BlockDriverState *bs);
>  int qcow2_read_snapshots(BlockDriverState *bs);
>  
> +/* qcow2-bitmap.c functions */
> +int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp);
> +
>  /* qcow2-cache.c functions */
>  Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables);
>  int qcow2_cache_destroy(BlockDriverState* bs, Qcow2Cache *c);
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 05/22] qcow2-bitmap: structs and consts

[Vladimir Sementsov-Ogievskiy]

On 01.10.2016 17:34, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Create block/qcow2-bitmap.c
>> Add data structures and constraints accordingly to docs/specs/qcow2.txt
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block/Makefile.objs  |  2 +-
>>   block/qcow2-bitmap.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++
>>   block/qcow2.h        | 29 +++++++++++++++++++++++++++++
>>   3 files changed, 77 insertions(+), 1 deletion(-)
>>   create mode 100644 block/qcow2-bitmap.c
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index fa4d8b8..0f661bb 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -1,5 +1,5 @@
>>   block-obj-y += raw_bsd.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o dmg.o
>> -block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
>> +block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o
>>   block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
>>   block-obj-y += qed-check.o
>>   block-obj-$(CONFIG_VHDX) += vhdx.o vhdx-endian.o vhdx-log.o
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> new file mode 100644
>> index 0000000..cd18b07
>> --- /dev/null
>> +++ b/block/qcow2-bitmap.c
>> @@ -0,0 +1,47 @@
>> +/*
>> + * Bitmaps for the QCOW version 2 format
>> + *
>> + * Copyright (c) 2014-2016 Vladimir Sementsov-Ogievskiy
>> + *
>> + * This file is derived from qcow2-snapshot.c, original copyright:
>> + * Copyright (c) 2004-2006 Fabrice Bellard
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to deal
>> + * in the Software without restriction, including without limitation the rights
>> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>> + * copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
>> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>> + * THE SOFTWARE.
>> + */
>> +
>> +/* NOTICE: BME here means Bitmaps Extension and used as a namespace for
>> + * _internal_ constants. Please do not use this _internal_ abbreviation for
>> + * other needs and/or outside of this file. */
>> +
>> +/* Bitmap directory entry constraints */
>> +#define BME_MAX_TABLE_SIZE 0x8000000
>> +#define BME_MAX_PHYS_SIZE 0x20000000 /* 512 mb */
> I suppose BME_MAX_TABLE_SIZE (8M) is greater than BME_MAX_PHYS_SIZE (512
> MB) divided by the cluster size (>= 512; 512 MB / cluster_size <= 1 MB)
> because fully zero or one clusters do not require any physical space?
>
> Makes some sense, but I can see that this might make give some trouble
> when trying to serialize overly large bitmaps. But I guess that comes
> later in this series, so I'll wait for that point.
>
> Another thing is that 512 MB is rather big. It gets worse: The bitmap
> may only require 512 MB on disk, but with a maximum table size of 8 MB,
> it can require up to 8M * cluster_size in memory (with just 64 MB of
> disk space!) by using the "read as all zeroes" or "read as all ones"
> flags. With the default cluster size of 64 kB, this would be 512 GB in
> RAM. That sounds bad to me.

This will not be so much if we introduce sparse bitmaps in qemu. But for 
current realization you are right, this should be restricted more 
carefully and noted in spec as 'Note: Qemu currently supports only...'

>
> Well, it is probably fine as long as the bitmap is not auto-loaded...
> But we do have a flag for exactly that. So it seems to me that a
> manipulated image can easily consume huge amounts of RAM on the host.
>
> So I think we also need some sane limitation on the in-RAM size of a
> bitmap (which is BME_MAX_TABLE_SIZE * cluster_size, as far as I
> understand). The question of course is, what is sane? For a server
> system with no image manipulation possible from the outside, 1 GB may be
> completely fine. But imagine you download some qcow2 image to your
> laptop. Then, 1 GB may not be fine, actually.
>
> Maybe it would make sense to use a runtime-adjustable limit here?
>
>> +#define BME_MAX_GRANULARITY_BITS 31
>> +#define BME_MIN_GRANULARITY_BITS 9
>> +#define BME_MAX_NAME_SIZE 1023
>> +
>> +/* Bitmap directory entry flags */
>> +#define BME_RESERVED_FLAGS 0xffffffff
>> +
>> +/* bits [1, 8] U [56, 63] are reserved */
>> +#define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001fe
> ull suffix is missing.
>
>> +
>> +typedef enum BitmapType {
>> +    BT_DIRTY_TRACKING_BITMAP = 1
>> +} BitmapType;
>> diff --git a/block/qcow2.h b/block/qcow2.h
>> index b36a7bf..0480b8b 100644
>> --- a/block/qcow2.h
>> +++ b/block/qcow2.h
>> @@ -52,6 +52,10 @@
>>    * space for snapshot names and IDs */
>>   #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS)
>>   
>> +/* Bitmap header extension constraints */
>> +#define QCOW_MAX_DIRTY_BITMAPS 65535
>> +#define QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE (1024 * QCOW_MAX_DIRTY_BITMAPS)
>> +
>>   /* indicate that the refcount of the referenced cluster is exactly one. */
>>   #define QCOW_OFLAG_COPIED     (1ULL << 63)
>>   /* indicate that the cluster is compressed (they never have the copied flag) */
>> @@ -142,6 +146,22 @@ typedef struct QEMU_PACKED QCowSnapshotHeader {
>>       /* name follows  */
>>   } QCowSnapshotHeader;
>>   
>> +/* Qcow2BitmapDirEntry is actually a bitmap directory entry */
> This comment doesn't make a whole lot of sense now that this structure
> is indeed called what it "actually is". ;-)
>
>> +typedef struct QEMU_PACKED Qcow2BitmapDirEntry {
>> +    /* header is 8 byte aligned */
>> +    uint64_t bitmap_table_offset;
>> +
>> +    uint32_t bitmap_table_size;
>> +    uint32_t flags;
>> +
>> +    uint8_t type;
>> +    uint8_t granularity_bits;
>> +    uint16_t name_size;
>> +    uint32_t extra_data_size;
>> +    /* extra data follows  */
>> +    /* name follows  */
>> +} Qcow2BitmapDirEntry;
>> +
>>   typedef struct QEMU_PACKED QCowSnapshotExtraData {
>>       uint64_t vm_state_size_large;
>>       uint64_t disk_size;
>> @@ -222,6 +242,15 @@ typedef uint64_t Qcow2GetRefcountFunc(const void *refcount_array,
>>   typedef void Qcow2SetRefcountFunc(void *refcount_array,
>>                                     uint64_t index, uint64_t value);
>>   
>> +/* Be careful, Qcow2BitmapHeaderExt is not an extension of Qcow2BitmapDirEntry, it
>> + * is Qcow2 header extension */
> And this makes even less sense now.
>
> (These comments don't stop me from giving an R-b, but I'm not so sure
> about the constants...)
>
> Max
>
>> +typedef struct Qcow2BitmapHeaderExt {
>> +    uint32_t nb_bitmaps;
>> +    uint32_t reserved32;
>> +    uint64_t bitmap_directory_size;
>> +    uint64_t bitmap_directory_offset;
>> +} QEMU_PACKED Qcow2BitmapHeaderExt;
>> +
>>   typedef struct BDRVQcow2State {
>>       int cluster_bits;
>>       int cluster_size;
>>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 08/22] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be
> needed in future, to save this flag back to Qcow2 for persistent
> bitmaps.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/dirty-bitmap.c         | 15 +++++++++++++++
>  block/qcow2-bitmap.c         |  2 ++
>  include/block/dirty-bitmap.h |  2 ++
>  3 files changed, 19 insertions(+)
> 
> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
> index 90af372..623e1d1 100644
> --- a/block/dirty-bitmap.c
> +++ b/block/dirty-bitmap.c
> @@ -44,6 +44,7 @@ struct BdrvDirtyBitmap {
>      int64_t size;               /* Size of the bitmap (Number of sectors) */
>      bool disabled;              /* Bitmap is read-only */
>      int active_iterators;       /* How many iterators are active */
> +    bool autoload;              /* bitmap must be autoloaded on image opening */

I think a note that this is for persistent dirty bitmaps would be
useful, e.g.:

"/* For persistent bitmaps: Bitmap must be autoloaded [...] */"

Up to you, though, so however you decide:

Reviewed-by: Max Reitz <mreitz@redhat.com>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 09/22] block: introduce persistent dirty bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 5619 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
> on bdrv_close, using format driver. Format driver should maintain bitmap
> storing.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block.c                      | 30 ++++++++++++++++++++++++++++++
>  block/dirty-bitmap.c         | 27 +++++++++++++++++++++++++++
>  block/qcow2-bitmap.c         |  1 +
>  include/block/block.h        |  2 ++
>  include/block/block_int.h    |  2 ++
>  include/block/dirty-bitmap.h |  6 ++++++
>  6 files changed, 68 insertions(+)
> 
> diff --git a/block.c b/block.c
> index 804e3d4..1cde03a 100644
> --- a/block.c
> +++ b/block.c
> @@ -2196,6 +2196,7 @@ void bdrv_reopen_abort(BDRVReopenState *reopen_state)
>  static void bdrv_close(BlockDriverState *bs)
>  {
>      BdrvAioNotifier *ban, *ban_next;
> +    Error *local_err = NULL;
>  
>      assert(!bs->job);
>      assert(!bs->refcnt);
> @@ -2204,6 +2205,10 @@ static void bdrv_close(BlockDriverState *bs)
>      bdrv_flush(bs);
>      bdrv_drain(bs); /* in case flush left pending I/O */
>  
> +    bdrv_store_persistent_bitmaps(bs, &local_err);
> +    if (local_err != NULL) {
> +        error_report_err(local_err);
> +    }

That seems pretty wrong to me. If the persistent bitmaps cannot be
stored, the node should not be closed to avoid loss of data.

>      bdrv_release_named_dirty_bitmaps(bs);

Especially since the next function will just drop all the dirty bitmaps.

I see the issue that bdrv_close() is only called by bdrv_delete() which
in turn is only called by bdrv_unref(); and how are you supposed to
react to bdrv_unref() failing?

So I'm not sure how this issue should be addressed, but this is most
certainly not ideal. You should not just drop supposedly persistent
dirty bitmaps if they cannot be saved.

We really should to have some way to keep the bitmap around if it cannot
be saved, but I don't know how to do that either.

In any case, we should make sure that the node supports saving
persistent dirty bitmaps, because having persistent dirty bitmaps at a
node that does not support them is something we can and must prevent
beforehand.

But I don't know how to handle failure if writing the dirty bitmap
fails. I guess one could argue that it's the same as bdrv_flush()
failing and thus can be handled in the same way, i.e. ignore it. I'm not
happy with that, but I'd accept it if there's no other way.

>      assert(QLIST_EMPTY(&bs->dirty_bitmaps));
>  
> @@ -3969,3 +3974,28 @@ void bdrv_del_child(BlockDriverState *parent_bs, BdrvChild *child, Error **errp)
>  
>      parent_bs->drv->bdrv_del_child(parent_bs, child, errp);
>  }
> +
> +void bdrv_store_persistent_bitmaps(BlockDriverState *bs, Error **errp)
> +{
> +    BlockDriver *drv = bs->drv;
> +
> +    if (!bdrv_has_persistent_bitmaps(bs)) {
> +        return;
> +    }
> +
> +    if (!drv) {
> +        error_setg_errno(errp, ENOMEDIUM,
> +                         "Can't store persistent bitmaps to %s",
> +                         bdrv_get_device_or_node_name(bs));
> +        return;
> +    }
> +
> +    if (!drv->bdrv_store_persistent_bitmaps) {
> +        error_setg_errno(errp, ENOTSUP,
> +                         "Can't store persistent bitmaps to %s",
> +                         bdrv_get_device_or_node_name(bs));
> +        return;
> +    }
> +
> +    drv->bdrv_store_persistent_bitmaps(bs, errp);
> +}
> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
> index 623e1d1..0314581 100644
> --- a/block/dirty-bitmap.c
> +++ b/block/dirty-bitmap.c
> @@ -44,6 +44,7 @@ struct BdrvDirtyBitmap {
>      int64_t size;               /* Size of the bitmap (Number of sectors) */
>      bool disabled;              /* Bitmap is read-only */
>      int active_iterators;       /* How many iterators are active */
> +    bool persistent;            /* bitmap must be saved to owner disk image */
>      bool autoload;              /* bitmap must be autoloaded on image opening */
>      QLIST_ENTRY(BdrvDirtyBitmap) list;
>  };
> @@ -72,6 +73,7 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
>      g_free(bitmap->name);
>      bitmap->name = NULL;
>  
> +    bitmap->persistent = false;
>      bitmap->autoload = false;
>  }
>  
> @@ -241,6 +243,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
>      bitmap->name = NULL;
>      successor->name = name;
>      bitmap->successor = NULL;
> +    successor->persistent = bitmap->persistent;
> +    bitmap->persistent = false;
>      successor->autoload = bitmap->autoload;
>      bitmap->autoload = false;
>      bdrv_release_dirty_bitmap(bs, bitmap);
> @@ -555,3 +559,26 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap)
>  {
>      return bitmap->autoload;
>  }
> +
> +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
> +                                                bool persistent)

The second parameter should be aligned to the opening parenthesis.

Max

> +{
> +    bitmap->persistent = persistent;
> +}
> +
> +bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap)
> +{
> +    return bitmap->persistent;
> +}
> +
> +bool bdrv_has_persistent_bitmaps(BlockDriverState *bs)
> +{
> +    BdrvDirtyBitmap *bm;
> +    QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
> +        if (bm->persistent) {
> +            return true;
> +        }
> +    }
> +
> +    return false;
> +}


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 10/22] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 311 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/dirty-bitmap.c         | 7 +++++++
>  include/block/dirty-bitmap.h | 3 +++
>  2 files changed, 10 insertions(+)

Reviewed-by: Max Reitz <mreitz@redhat.com>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 11599 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Realize block bitmap stroing interface, to allow qcow2 images store
> persistent bitmaps.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c | 241 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  block/qcow2.c        |   2 +
>  block/qcow2.h        |   2 +
>  3 files changed, 245 insertions(+)
> 
> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> index 81520cd..a5be25a 100644
> --- a/block/qcow2-bitmap.c
> +++ b/block/qcow2-bitmap.c
> @@ -27,6 +27,7 @@
>  
>  #include "qemu/osdep.h"
>  #include "qapi/error.h"
> +#include "qemu/cutils.h"
>  
>  #include "block/block_int.h"
>  #include "block/qcow2.h"
> @@ -96,6 +97,15 @@ static inline void bitmap_table_to_cpu(uint64_t *bitmap_table, size_t size)
>      }
>  }
>  
> +static inline void bitmap_table_to_be(uint64_t *bitmap_table, size_t size)
> +{
> +    size_t i;
> +
> +    for (i = 0; i < size; ++i) {
> +        cpu_to_be64s(&bitmap_table[i]);
> +    }
> +}
> +
>  static inline int calc_dir_entry_size(size_t name_size, size_t extra_data_size)
>  {
>      return align_offset(sizeof(Qcow2BitmapDirEntry) +
> @@ -564,3 +574,234 @@ out:
>  
>      return ret;
>  }
> +
> +/* store_bitmap_data()
> + * Store bitmap to image, filling bitamp table accordingly.

s/bitamp/bitmap/

> + */
> +static int store_bitmap_data(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
> +                             uint64_t *bitmap_table, uint32_t bitmap_table_size)
> +{
> +    int ret;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint64_t sector, dsc;
> +    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
> +    int cl_size = s->cluster_size;

Once more, I don't think this variable is necessary, and I feel like it
makes reading the code more difficult for no gain.

> +    uint8_t *buf = NULL;
> +    uint32_t tb_size =
> +            size_to_clusters(s,
> +                bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));

Should be uint64_t or you might accidentally get a truncation here. I'm
not sure how you would get such huge bitmaps but it's trivial to use
uint64_t.

> +
> +    BdrvDirtyBitmapIter *dbi;
> +
> +    if (tb_size != bitmap_table_size) {
> +        return -EINVAL;
> +    }
> +
> +    memset(bitmap_table, 0, bitmap_table_size * sizeof(bitmap_table[0]));

On 32 bit machines, this multiplication can overflow. There should at
least be an assertion to prevent this. Other than that, of course, there
needs to be some place which limits bitmap_table_size to some sane value
and emits a real error if it exceeds that value.

> +
> +    dbi = bdrv_dirty_iter_new(bitmap, 0);
> +    buf = g_malloc(cl_size);
> +    dsc = dirty_sectors_in_cluster(s, bitmap);
> +
> +    while ((sector = bdrv_dirty_iter_next(dbi)) != -1) {

sector should be int64_t instead of uint64_t, then.

> +        uint64_t cluster = sector / dsc;
> +        sector = cluster * dsc;

Our coding style does not allow interleaving declarations and
non-declarations.

> +        uint64_t end = MIN(bm_size, sector + dsc);
> +        uint64_t write_size =
> +            bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sector);
> +
> +        int64_t off = qcow2_alloc_clusters(bs, cl_size);
> +        if (off < 0) {
> +            ret = off;
> +            goto finish;
> +        }
> +        bitmap_table[cluster] = off;
> +
> +        bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end);

s/end/end - sector/?

> +        if (write_size < cl_size) {
> +            memset(buf + write_size, 0, cl_size - write_size);
> +        }
> +

I guess there should be a metadata overlap check here.

> +        ret = bdrv_pwrite(bs->file, off, buf, cl_size);
> +        if (ret < 0) {
> +            goto finish;
> +        }
> +
> +        if (end >= bm_size) {
> +            break;
> +        }
> +
> +        bdrv_set_dirty_iter(dbi, end);
> +    }
> +    ret = 0; /* writes */

What is that comment supposed to mean?

> +
> +finish:
> +    if (ret < 0) {
> +        clear_bitmap_table(bs, bitmap_table, bitmap_table_size);
> +    }
> +    g_free(buf);
> +    bdrv_dirty_iter_free(dbi);
> +
> +    return ret;

In case you decide to keep BME_MAX_PHYS_SIZE, this function should check
somewhere that the physical size of the bitmap does not exceed that value.

> +}
> +
> +/* store_bitmap()
> + * Store bitmap to qcow2 and set bitmap_table. bitmap_table itself is not
> + * stored to qcow2.

First of all, there is no parameter called "bitmap_table", and second,
yes, the bitmap table is written to the qcow2 file.

> + */
> +static int store_bitmap(BlockDriverState *bs,
> +                        BdrvDirtyBitmap *bitmap,
> +                        Qcow2BitmapDirEntry *entry)
> +{
> +    int ret;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
> +    const char *bm_name = bdrv_dirty_bitmap_name(bitmap);
> +
> +    uint64_t *tb;
> +    int64_t tb_offset;
> +    uint32_t tb_size =
> +            size_to_clusters(s,
> +                bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));

As above, this variable should be of type uint64_t.

Also, you have to check that it does not exceed BME_MAX_TABLE_SIZE.

> +
> +    tb = g_try_new(uint64_t, tb_size);
> +    if (tb == NULL) {
> +        return -ENOMEM;
> +    }
> +
> +    ret = store_bitmap_data(bs, bitmap, tb, tb_size);
> +    if (ret < 0) {
> +        g_free(tb);
> +        return ret;
> +    }
> +
> +    tb_offset = qcow2_alloc_clusters(bs, tb_size * sizeof(tb[0]));

If you don't limit tb_size, then this multiplication can overflow on 32
bit machines.

> +    if (tb_offset < 0) {
> +        ret = tb_offset;
> +        goto fail;
> +    }
> +

There should be a metadata overlap check here.

> +    bitmap_table_to_be(tb, tb_size);
> +    ret = bdrv_pwrite(bs->file, tb_offset, tb, tb_size * sizeof(tb[0]));
> +    if (ret < 0) {
> +        goto fail;
> +    }
> +
> +    g_free(tb);
> +
> +    entry->bitmap_table_offset = tb_offset;
> +    entry->bitmap_table_size = tb_size;
> +    entry->flags = bdrv_dirty_bitmap_granularity(bitmap) ? BME_FLAG_AUTO : 0;

s/granularity/get_autoload/

> +    entry->type = BT_DIRTY_TRACKING_BITMAP;
> +    entry->granularity_bits = ctz32(bdrv_dirty_bitmap_granularity(bitmap));

You should probably check somewhere that the resulting value for
entry->granularity_bits is in the BME_{MIN,MAX}_GRANULARITY_BITS range.

> +    entry->name_size = strlen(bm_name);

And that this length does not exceed BME_MAX_NAME_SIZE.

> +    entry->extra_data_size = 0;
> +    memcpy(entry + 1, bm_name, entry->name_size);
> +
> +    return 0;
> +
> +fail:
> +    clear_bitmap_table(bs, tb, tb_size);
> +
> +    if (tb_offset > 0) {
> +        qcow2_free_clusters(bs, tb_offset, tb_size, QCOW2_DISCARD_ALWAYS);

As before, I'd vote for QCOW2_DISCARD_OTHER.

> +    }
> +
> +    g_free(tb);
> +
> +    return ret;
> +}
> +
> +static Qcow2BitmapDirEntry *find_bitmap_by_name(uint8_t *bitmap_directory,
> +                                                size_t size, const char *name)
> +{
> +    Qcow2BitmapDirEntry *e;
> +
> +    for_each_bitmap_dir_entry(e, bitmap_directory, size) {
> +        if (strncmp((char *)(e + 1), name, e->name_size) == 0) {
> +            return e;
> +        }
> +    }
> +
> +    return NULL;
> +}
> +
> +void qcow2_store_persistent_bitmaps(BlockDriverState *bs, Error **errp)
> +{
> +    BdrvDirtyBitmap *bm;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint32_t new_nb_bitmaps = s->nb_bitmaps;
> +    uint64_t new_dir_size = s->bitmap_directory_size;
> +    uint8_t *dir = NULL, *new_dir = NULL;
> +    int ret;
> +    Qcow2BitmapDirEntry *new_pos;
> +
> +    if (s->nb_bitmaps > 0) {
> +        dir = directory_read(bs, s->bitmap_directory_offset,
> +                             s->bitmap_directory_size, errp);
> +        if (dir == NULL) {
> +            goto out;
> +        }
> +    }
> +
> +    for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm != NULL;
> +            bm = bdrv_dirty_bitmap_next(bs, bm)) {
> +        const char *name = bdrv_dirty_bitmap_name(bm);
> +
> +        if (!bdrv_dirty_bitmap_get_persistance(bm)) {
> +            continue;
> +        }
> +
> +        if (s->nb_bitmaps > 0 &&
> +                find_bitmap_by_name(dir, s->bitmap_directory_size, name)) {
> +            error_setg(errp,
> +                       "Can't store bitmap '%s' to '%s', as it already exists",
> +                       name, bdrv_get_device_or_node_name(bs));
> +            goto out;
> +        }
> +
> +        new_nb_bitmaps++;
> +        new_dir_size += calc_dir_entry_size(strlen(name), 0);
> +    }
> +
> +    if (s->nb_bitmaps == new_nb_bitmaps) {
> +        /* No new bitmaps - nothing to do */
> +        goto out;
> +    }
> +
> +    new_dir = g_try_malloc0(new_dir_size);
> +    if (new_dir == NULL) {
> +        error_setg(errp, "Can't allocate space for bitmap directory.");
> +        goto out;
> +    }
> +
> +    memcpy(new_dir, dir, s->bitmap_directory_size);
> +    new_pos = (Qcow2BitmapDirEntry *)(new_dir + s->bitmap_directory_size);
> +
> +    for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm != NULL;
> +            bm = bdrv_dirty_bitmap_next(bs, bm)) {
> +        if (!bdrv_dirty_bitmap_get_persistance(bm)) {
> +            continue;
> +        }
> +
> +        ret = store_bitmap(bs, bm, new_pos);
> +        if (ret < 0) {
> +            error_setg_errno(errp, -ret, "Can't store bitmap '%s' to '%s'",
> +                             bdrv_dirty_bitmap_name(bm),
> +                             bdrv_get_device_or_node_name(bs));
> +            goto out;
> +        }
> +        new_pos = next_dir_entry(new_pos);
> +    }
> +
> +    ret = directory_update(bs, new_dir, new_dir_size, new_nb_bitmaps);
> +    if (ret < 0) {
> +        error_setg_errno(errp, -ret, "Can't update bitmap directory in '%s'",
> +                         bdrv_get_device_or_node_name(bs));
> +        goto out;
> +    }
> +
> +out:
> +    g_free(new_dir);
> +    g_free(dir);

This error path leaks all the bitmaps that have been written
successfully (if any). I guess this is more or less fine if
directory_update() failed (because you can't really tell the state of
the image header after directory_update(), so better be safe) but it's
not so fine if just some store_bitmap() failed.

Max

> +}
> diff --git a/block/qcow2.c b/block/qcow2.c
> index 02ec224..8238205 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c
> @@ -3493,6 +3493,8 @@ BlockDriver bdrv_qcow2 = {
>  
>      .bdrv_detach_aio_context  = qcow2_detach_aio_context,
>      .bdrv_attach_aio_context  = qcow2_attach_aio_context,
> +
> +    .bdrv_store_persistent_bitmaps = qcow2_store_persistent_bitmaps,
>  };
>  
>  static void bdrv_qcow2_init(void)
> diff --git a/block/qcow2.h b/block/qcow2.h
> index 482a29f..dfcf4c6 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -627,4 +627,6 @@ int qcow2_cache_get_empty(BlockDriverState *bs, Qcow2Cache *c, uint64_t offset,
>      void **table);
>  void qcow2_cache_put(BlockDriverState *bs, Qcow2Cache *c, void **table);
>  
> +void qcow2_store_persistent_bitmaps(BlockDriverState *bs, Error **errp);
> +
>  #endif
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 2761 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are
> loaded at image open and becomes BdrvDirtyBitmap's for corresponding
> drive. These bitmaps are deleted from Qcow2 image after loading to avoid
> conflicts.
> 
> Extra data in bitmaps is not supported for now.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c | 518 ++++++++++++++++++++++++++++++++++++++++++++++++++-
>  block/qcow2.c        |   5 +
>  block/qcow2.h        |   3 +
>  3 files changed, 525 insertions(+), 1 deletion(-)
> 
> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> index cd18b07..760a047 100644
> --- a/block/qcow2-bitmap.c
> +++ b/block/qcow2-bitmap.c

[...]

> +static int directory_update(BlockDriverState *bs, uint8_t *new_dir,
> +                            size_t new_size, uint32_t new_nb_bitmaps)
> +{
> +    BDRVQcow2State *s = bs->opaque;
> +    int ret;
> +    int64_t new_offset = 0;
> +    uint64_t old_offset = s->bitmap_directory_offset;
> +    uint64_t old_size = s->bitmap_directory_size;
> +    uint32_t old_nb_bitmaps = s->nb_bitmaps;
> +    uint64_t old_autocl = s->autoclear_features;
> +
> +    if (new_size > QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE) {
> +        return -EINVAL;
> +    }
> +
> +    if ((new_size == 0) != (new_nb_bitmaps == 0)) {
> +        return -EINVAL;
> +    }
> +
> +    if (new_nb_bitmaps > 0) {
> +        new_offset = directory_write(bs, new_dir, new_size);
> +        if (new_offset < 0) {
> +            return new_offset;
> +        }
> +
> +        ret = bdrv_flush(bs);
> +        if (ret < 0) {
> +            goto fail;
> +        }
> +    }
> +    s->bitmap_directory_offset = new_offset;
> +    s->bitmap_directory_size = new_size;
> +    s->nb_bitmaps = new_nb_bitmaps;
> +
> +    ret = update_header_sync(bs);
> +    if (ret < 0) {
> +        goto fail;
> +    }
> +
> +    if (old_size) {
> +        qcow2_free_clusters(bs, old_offset, old_size, QCOW2_DISCARD_ALWAYS);
> +    }
> +
> +    return 0;
> +
> +fail:
> +    if (new_offset > 0) {
> +        qcow2_free_clusters(bs, new_offset, new_size, QCOW2_DISCARD_ALWAYS);
> +        s->bitmap_directory_offset = old_offset;
> +        s->bitmap_directory_size = old_size;
> +        s->nb_bitmaps = old_nb_bitmaps;
> +        s->autoclear_features = old_autocl;

What if bdrv_flush() in update_header_sync() failed? Then you cannot be
sure what bitmap directory the header is actually pointing to. In that
case, it would be wrong to assume it's still pointing to the old one and
freeing the new one.

Max

> +    }
> +
> +    return ret;
> +}


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 09/22] block: introduce persistent dirty bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1906 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
> on bdrv_close, using format driver. Format driver should maintain bitmap
> storing.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block.c                      | 30 ++++++++++++++++++++++++++++++
>  block/dirty-bitmap.c         | 27 +++++++++++++++++++++++++++
>  block/qcow2-bitmap.c         |  1 +
>  include/block/block.h        |  2 ++
>  include/block/block_int.h    |  2 ++
>  include/block/dirty-bitmap.h |  6 ++++++
>  6 files changed, 68 insertions(+)

[...]

> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
> index 623e1d1..0314581 100644
> --- a/block/dirty-bitmap.c
> +++ b/block/dirty-bitmap.c

[...]

> @@ -555,3 +559,26 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap)
>  {
>      return bitmap->autoload;
>  }
> +
> +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
> +                                                bool persistent)
> +{
> +    bitmap->persistent = persistent;

After some thinking, I think this function should be more complex: It
should check whether the node the bitmap is attached to actually can
handle persistent bitmaps and whether it would actually support storing
*this* bitmap.

For instance, a qcow2 node would not support writing overly large
bitmaps (limited by BME_MAX_TABLE_SIZE and BME_MAX_PHYS_SIZE) or bitmaps
with overly large granularities (BME_MAX_GRANULARITY_BITS) or bitmaps
whose name is already occupied by some bitmap that is already stored in
the file but has not been loaded.

Checking this here will trivially prevent users from creating such
bitmaps and will also preempt detection of such failures during
bdrv_close() when they cannot be handled gracefully.

Max

> +}


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 3460 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> This flag means that the bitmap is now in use by the software or was not
> successfully saved. In any way, with this flag set the bitmap data must
> be considered inconsistent and should not be loaded.
> 
> With current implementation this flag is never set, as we just remove
> bitmaps from the image after loading. But it defined in qcow2 spec and
> must be handled. Also, it can be used in future, if async schemes of
> bitmap loading/saving are implemented.
> 
> We also remove in-use bitmaps from the image on open.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c | 17 ++++++++++++++++-
>  1 file changed, 16 insertions(+), 1 deletion(-)

Don't you want to make use of this flag? It would appear useful to me if
you just marked autoload bitmaps as in_use instead of deleting them from
the image when it's opened and then overwrite them when the image is closed.

> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> index a5be25a..8cf40f0 100644
> --- a/block/qcow2-bitmap.c
> +++ b/block/qcow2-bitmap.c
> @@ -28,6 +28,7 @@
>  #include "qemu/osdep.h"
>  #include "qapi/error.h"
>  #include "qemu/cutils.h"
> +#include "exec/log.h"
>  
>  #include "block/block_int.h"
>  #include "block/qcow2.h"
> @@ -44,7 +45,8 @@
>  #define BME_MAX_NAME_SIZE 1023
>  
>  /* Bitmap directory entry flags */
> -#define BME_RESERVED_FLAGS 0xfffffffd
> +#define BME_RESERVED_FLAGS 0xfffffffc
> +#define BME_FLAG_IN_USE 1

This should be (1U << 0) to be consistent with the definition of
BME_FLAG_AUTO.

>  #define BME_FLAG_AUTO   (1U << 1)
>  
>  /* bits [1, 8] U [56, 63] are reserved */
> @@ -287,6 +289,11 @@ static BdrvDirtyBitmap *load_bitmap(BlockDriverState *bs,
>      BdrvDirtyBitmap *bitmap = NULL;
>      char *name = g_strndup(dir_entry_name_notcstr(entry), entry->name_size);
>  
> +    if (entry->flags & BME_FLAG_IN_USE) {
> +        error_setg(errp, "Bitmap '%s' is in use", name);
> +        goto fail;
> +    }
> +
>      ret = bitmap_table_load(bs, entry, &bitmap_table);
>      if (ret < 0) {
>          error_setg_errno(errp, -ret,
> @@ -533,6 +540,14 @@ int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp)
>      for_each_bitmap_dir_entry(e, dir, dir_size) {
>          uint32_t fl = e->flags;
>  
> +        if (fl & BME_FLAG_IN_USE) {
> +            qemu_log("qcow2 warning: "
> +                     "removing in_use bitmap '%.*s' for image %s.\n",
> +                     e->name_size, (char *)(e + 1), bdrv_get_device_or_node_name(bs));

I'm not sure whether qemu_log() is the right way to do this. I think
fprintf() to stderr might actually be more appropriate. qemu_log() looks
like it's mostly used for debugging purposes.

Also, this is not a warning. You are just doing it. You are not warning
someone about a cliff when he's already fallen off.

But you can actually make it a warning: Just leave the bitmap in the
image (maybe someone can do something with it?) and instead let qemu-img
check clean it up. The warning should then just be "Warning: Ignoring
in_use bitmap %.*s, use qemu-img check -r all to delete it".

Max

> +
> +            continue;
> +        }
> +
>          if (fl & BME_FLAG_AUTO) {
>              BdrvDirtyBitmap *bitmap = load_bitmap(bs, e, errp);
>              if (bitmap == NULL) {
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 16/22] qmp: add persistent flag to block-dirty-bitmap-add

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1659 bytes --]

On 09/30/2016 05:53 AM, Vladimir Sementsov-Ogievskiy wrote:
> Add optional 'persistent' flag to qmp command block-dirty-bitmap-add.
> Default is false.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> ---
>  blockdev.c           | 12 +++++++++++-
>  qapi/block-core.json |  7 ++++++-
>  qmp-commands.hx      |  5 ++++-

This and later patches have conflicts with recent changes in the tree
that removed qmp-commands.hx; hopefully shouldn't impact review of this
series too badly.

> +++ b/qapi/block-core.json
> @@ -1235,10 +1235,15 @@
>  # @granularity: #optional the bitmap granularity, default is 64k for
>  #               block-dirty-bitmap-add
>  #
> +# @persistent: #optional the bitmap is persistent, i.e. it will be saved to
> +#              corresponding block device on it's close. Default is false.

s/corresponding/the corresponding/
s/it's/its/

("it's" is only appropriate if you can substitute "it is" or "it has")


> @@ -1458,6 +1458,9 @@ Arguments:
>  - "node": device/node on which to create dirty bitmap (json-string)
>  - "name": name of the new dirty bitmap (json-string)
>  - "granularity": granularity to track writes with (int, optional)
> +- "persistent": bitmap will be saved to corresponding block device
> +                on it's close. Block driver should maintain persistent bitmaps

One nice benefit of rebasing on top of qmp-commands.hx being removed is
that you don't have to repeat yourself :)

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 17/22] qmp: add autoload parameter to block-dirty-bitmap-add

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]

On 09/30/2016 05:53 AM, Vladimir Sementsov-Ogievskiy wrote:
> Optional. Default is false.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> ---
>  blockdev.c           | 22 ++++++++++++++++++++--
>  qapi/block-core.json |  7 ++++++-
>  qmp-commands.hx      |  5 ++++-
>  3 files changed, 30 insertions(+), 4 deletions(-)

> +++ b/qapi/block-core.json
> @@ -1239,11 +1239,16 @@
>  #              corresponding block device on it's close. Default is false.
>  #              For block-dirty-bitmap-add. (Since 2.8)
>  #
> +# @autoload: #optional the bitmap will be autoloaded on it's storage image

s/it's/its/

> +#            open. This flag is only for persistent bitmap and needed to inform

s/bitmap/bitmaps,/
s/needed/is needed/

> +#            block driver that bitmap should be autoloaded on the next image

s/block/the block/
s/bitmap/the bitmap/


-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 13/22] qcow2-bitmap: check constraints

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 3412 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Check bitmap header constraints as specified in docs/specs/qcow2.txt
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c | 39 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 39 insertions(+)

I'd pull this patch to some previous point in the series because the
previous patches would already require you to check these constraints
(which you just haven't done until now).

> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> index 8cf40f0..1c3abea 100644
> --- a/block/qcow2-bitmap.c
> +++ b/block/qcow2-bitmap.c
> @@ -154,6 +154,34 @@ static inline void bitmap_directory_to_be(uint8_t *dir, size_t size)
>      }
>  }
>  
> +static int check_constraints(BlockDriverState *bs, Qcow2BitmapDirEntry *h)
> +{
> +    BDRVQcow2State *s = bs->opaque;
> +    uint64_t phys_bitmap_bytes =
> +        (uint64_t)h->bitmap_table_size * s->cluster_size;
> +    uint64_t max_virtual_bits = (phys_bitmap_bytes * 8) << h->granularity_bits;
> +    int64_t nb_sectors = bdrv_nb_sectors(bs);
> +
> +    if (nb_sectors < 0) {
> +        return nb_sectors;
> +    }
> +
> +    int fail =
> +            ((h->bitmap_table_size == 0) != (h->bitmap_table_offset == 0)) ||
> +            (h->bitmap_table_offset % s->cluster_size) ||
> +            (h->bitmap_table_size > BME_MAX_TABLE_SIZE) ||
> +            (phys_bitmap_bytes > BME_MAX_PHYS_SIZE) ||
> +            (h->bitmap_table_offset != 0 &&
> +                (nb_sectors << BDRV_SECTOR_BITS) > max_virtual_bits) ||
> +            (h->granularity_bits > BME_MAX_GRANULARITY_BITS) ||
> +            (h->granularity_bits < BME_MIN_GRANULARITY_BITS) ||
> +            (h->flags & BME_RESERVED_FLAGS) ||
> +            (h->name_size > BME_MAX_NAME_SIZE) ||
> +            (h->type != BT_DIRTY_TRACKING_BITMAP);
> +
> +    return fail ? -EINVAL : 0;
> +}
> +
>  static void clear_bitmap_table(BlockDriverState *bs, uint64_t *bitmap_table,
>                                 uint32_t bitmap_table_size)
>  {
> @@ -372,6 +400,12 @@ static uint8_t *directory_read(BlockDriverState *bs,
>                         bdrv_get_device_or_node_name(bs));
>              goto fail;
>          }
> +
> +        ret = check_constraints(bs, e);
> +        if (ret < 0) {
> +            error_setg(errp, "Bitmap doesn't satisfy the constraints.");

I think I'd at least mention the name of the bitmap; also, no full stop
at the end of error messages.

> +            goto fail;
> +        }
>      }
>  
>      assert((uint8_t *)e == dir_end);
> @@ -713,6 +747,11 @@ static int store_bitmap(BlockDriverState *bs,
>      entry->extra_data_size = 0;
>      memcpy(entry + 1, bm_name, entry->name_size);
>  
> +    ret = check_constraints(bs, entry);
> +    if (ret < 0) {
> +        goto fail;
> +    }
> +

As I said in my second reply to patch 9, I think it's a bit too late if
we detect that the bitmap is actually invalid at this point. We really
should notice earlier.

Apart from what would actually better for the user, it is actually too
late to check the constraints here, as you have already written the
bitmap data to disk. You should always check the constraints before
reading and also before writing, not afterwards.

Max

>      return 0;
>  
>  fail:
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 14/22] qcow2: delete bitmaps on truncate

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 3338 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> This will not touch loaded bitmaps, as they are alredy removed from the
> image.

Well, yes, and I don't think that's ideal. :-)

I think marking them in_use instead of deleting them would be better.

Also, I'm not so sure whether this patch is an improvement at all.
Wouldn't it be better to have a qemu-img tool (and/or QMP interface) for
bitmap management and then making the user delete all the bitmaps before
the image can be resized?

Max

> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c | 38 ++++++++++++++++++++++++++++++++++++++
>  block/qcow2.c        |  9 ++++++---
>  block/qcow2.h        |  2 ++
>  3 files changed, 46 insertions(+), 3 deletions(-)
> 
> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> index 1c3abea..2642afe 100644
> --- a/block/qcow2-bitmap.c
> +++ b/block/qcow2-bitmap.c
> @@ -624,6 +624,44 @@ out:
>      return ret;
>  }
>  
> +
> +int qcow2_delete_bitmaps(BlockDriverState *bs)
> +{
> +    int ret;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint8_t *dir;
> +    uint64_t dir_size;
> +    Qcow2BitmapDirEntry *e;
> +
> +    if (s->nb_bitmaps == 0) {
> +        /* No bitmaps - nothing to do */
> +        return 0;
> +    }
> +
> +    dir_size = s->bitmap_directory_size;
> +    dir = directory_read(bs, s->bitmap_directory_offset,
> +                         s->bitmap_directory_size, NULL);
> +    if (dir == NULL) {
> +        ret = -EINVAL;
> +        goto out;
> +    }
> +
> +    ret = directory_update(bs, NULL, 0, 0);
> +    if (ret < 0) {
> +        goto out;
> +    }
> +
> +    /* to be consistent, free bitmap only after successfull directory update */
> +    for_each_bitmap_dir_entry(e, dir, dir_size) {
> +        free_bitmap_clusters(bs, e);
> +    }
> +
> +out:
> +    g_free(dir);
> +
> +    return ret;
> +}
> +
>  /* store_bitmap_data()
>   * Store bitmap to image, filling bitamp table accordingly.
>   */
> diff --git a/block/qcow2.c b/block/qcow2.c
> index 8238205..aa967ed 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c
> @@ -2592,9 +2592,12 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset)
>  
>      /* cannot proceed if image has bitmaps */
>      if (s->nb_bitmaps) {
> -        /* FIXME */
> -        error_report("Can't resize an image which has bitmaps");
> -        return -ENOTSUP;
> +        /* FIXME: not loaded bitmaps will be lost */
> +        ret = qcow2_delete_bitmaps(bs);
> +        if (ret < 0) {
> +            error_report("Can't remove bitmaps from qcow2 on truncate");
> +            return ret;
> +        }
>      }
>  
>      /* shrinking is currently not supported */
> diff --git a/block/qcow2.h b/block/qcow2.h
> index dfcf4c6..af18efc 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -606,6 +606,8 @@ int qcow2_read_snapshots(BlockDriverState *bs);
>  /* qcow2-bitmap.c functions */
>  int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp);
>  
> +int qcow2_delete_bitmaps(BlockDriverState *bs);
> +
>  /* qcow2-cache.c functions */
>  Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables);
>  int qcow2_cache_destroy(BlockDriverState* bs, Qcow2Cache *c);
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 15/22] qcow2-bitmap: add autoclear bit

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 876 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Add autoclear bit for handling rewriting image by old qemu version.

"Add support for the autoclear bit [...]"?

> 
> If autoclear bit is not set, but bitmaps extension is found it
> would not be loaded and warning will be generated.

"If the autoclear bit is not set, but the bitmaps extension is found,
the bitmaps will not be loaded and a warning will be generated."

> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c |  4 ++++
>  block/qcow2.c        | 12 ++++++++++--
>  block/qcow2.h        |  9 +++++++++
>  3 files changed, 23 insertions(+), 2 deletions(-)

Apart from the above, the patch looks good, but why does it come so late
in the series?

So with the commit message fixed:

Reviewed-by: Max Reitz <mreitz@redhat.com>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 21/22] specs/qcow2: fix bitmap granularity qemu-specific note

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]

On 09/30/2016 05:53 AM, Vladimir Sementsov-Ogievskiy wrote:
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  docs/specs/qcow2.txt | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
> index 80cdfd0..d3e292f 100644
> --- a/docs/specs/qcow2.txt
> +++ b/docs/specs/qcow2.txt
> @@ -472,8 +472,7 @@ Structure of a bitmap directory entry:
>               17:    granularity_bits
>                      Granularity bits. Valid values: 0 - 63.

Can we really theoretically go as low as 0, or are we constrained by the
fact that cluster_bits must be at least 9?

>  
> -                    Note: Qemu currently doesn't support granularity_bits
> -                    greater than 31.
> +                    Note: Qemu currently support only values 9 - 31.

s/support/supports/

>  
>                      Granularity is calculated as
>                          granularity = 1 << granularity_bits
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 22/22] specs/qcow2: do not use wording 'bitmap header'

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 781 bytes --]

On 09/30/2016 05:53 AM, Vladimir Sementsov-Ogievskiy wrote:
> For now bitmap directory entry some times called 'bitmap header'. This

Awkward wording; maybe:

A bitmap directory entry is sometimes called a 'bitmap header'.

> patch leaves only one name - 'bitmap directory entry'. The name 'bitmap
> header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap
> header extension' (which is extension of qcow2 header)
> 

Doc touchup patches like this are fairly uncontroversial, and could be
moved earlier in the series so that they can be merged rather than
having to be rebased multiple times.

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 16/22] qmp: add persistent flag to block-dirty-bitmap-add

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 4931 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Add optional 'persistent' flag to qmp command block-dirty-bitmap-add.
> Default is false.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> ---
>  blockdev.c           | 12 +++++++++++-
>  qapi/block-core.json |  7 ++++++-
>  qmp-commands.hx      |  5 ++++-
>  3 files changed, 21 insertions(+), 3 deletions(-)
> 
> diff --git a/blockdev.c b/blockdev.c
> index 97062e3..ec0ec75 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -1991,6 +1991,7 @@ static void block_dirty_bitmap_add_prepare(BlkActionState *common,
>      /* AIO context taken and released within qmp_block_dirty_bitmap_add */
>      qmp_block_dirty_bitmap_add(action->node, action->name,
>                                 action->has_granularity, action->granularity,
> +                               action->has_persistent, action->persistent,
>                                 &local_err);
>  
>      if (!local_err) {
> @@ -2694,10 +2695,12 @@ out:
>  
>  void qmp_block_dirty_bitmap_add(const char *node, const char *name,
>                                  bool has_granularity, uint32_t granularity,
> +                                bool has_persistent, bool persistent,
>                                  Error **errp)
>  {
>      AioContext *aio_context;
>      BlockDriverState *bs;
> +    BdrvDirtyBitmap *bitmap;
>  
>      if (!name || name[0] == '\0') {
>          error_setg(errp, "Bitmap name cannot be empty");
> @@ -2723,7 +2726,14 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name,
>          granularity = bdrv_get_default_bitmap_granularity(bs);
>      }
>  
> -    bdrv_create_dirty_bitmap(bs, granularity, name, errp);
> +    if (!has_persistent) {
> +        persistent = false;
> +    }
> +
> +    bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp);
> +    if (bitmap != NULL) {
> +        bdrv_dirty_bitmap_set_persistance(bitmap, persistent);

As I said before, I think this command should be able to return errors
and make use of that to reject making bitmaps persistent when the
respective block driver cannot handle them.

> +    }
>  
>   out:
>      aio_context_release(aio_context);
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 31f9990..2bf56cd 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1235,10 +1235,15 @@
>  # @granularity: #optional the bitmap granularity, default is 64k for
>  #               block-dirty-bitmap-add
>  #
> +# @persistent: #optional the bitmap is persistent, i.e. it will be saved to
> +#              corresponding block device on it's close. Default is false.
> +#              For block-dirty-bitmap-add. (Since 2.8)

I'm not sure what the "For block-dirty-bitmap-add." is supposed to mean,
because this whole struct is for block-dirty-bitmap-add (and for
block-dirty-bitmap-add transactions, to be exact, but @persistent will
surely work there, too, won't it?).

Also, I'd say "will be saved to the corresponding block device image
file" instead of just "block device", because in my understanding a
block device and its image file are two separate things.

> +#
>  # Since 2.4
>  ##
>  { 'struct': 'BlockDirtyBitmapAdd',
> -  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32' } }
> +  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
> +  '*persistent': 'bool' } }

I think normally we'd align the new line so that the opening ' of
'*persistent' is under the opening ' of 'node'.

>  
>  ##
>  # @block-dirty-bitmap-add
> diff --git a/qmp-commands.hx b/qmp-commands.hx
> index ba2a916..434b418 100644
> --- a/qmp-commands.hx
> +++ b/qmp-commands.hx
> @@ -1441,7 +1441,7 @@ EQMP
>  
>      {
>          .name       = "block-dirty-bitmap-add",
> -        .args_type  = "node:B,name:s,granularity:i?",
> +        .args_type  = "node:B,name:s,granularity:i?,persistent:b?",
>          .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
>      },
>  
> @@ -1458,6 +1458,9 @@ Arguments:
>  - "node": device/node on which to create dirty bitmap (json-string)
>  - "name": name of the new dirty bitmap (json-string)
>  - "granularity": granularity to track writes with (int, optional)
> +- "persistent": bitmap will be saved to corresponding block device
> +                on it's close. Block driver should maintain persistent bitmaps
> +                (json-bool, optional, default false) (Since 2.8)

And I don't know what the user is supposed to make of the information
that block drivers will take care of maintaining persistent bitmaps. All
they care about is that it will be stored in the corresponding image
file, so in my opinion it would be better to just omit the last sentence
here.

Max

>  
>  Example:
>  
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 17/22] qmp: add autoload parameter to block-dirty-bitmap-add

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 5668 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Optional. Default is false.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
> ---
>  blockdev.c           | 22 ++++++++++++++++++++--
>  qapi/block-core.json |  7 ++++++-
>  qmp-commands.hx      |  5 ++++-
>  3 files changed, 30 insertions(+), 4 deletions(-)

Design question: I see that being able to specify these flags when
creating bitmaps is useful. However, would about a way for the user to
change these flags on an existing dirty bitmap? Would you consider that
useful?

(Of course, if so, it can always be added later, we don't need it now.)

> diff --git a/blockdev.c b/blockdev.c
> index ec0ec75..00da7a1 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -1992,6 +1992,7 @@ static void block_dirty_bitmap_add_prepare(BlkActionState *common,
>      qmp_block_dirty_bitmap_add(action->node, action->name,
>                                 action->has_granularity, action->granularity,
>                                 action->has_persistent, action->persistent,
> +                               action->has_autoload, action->autoload,
>                                 &local_err);
>  
>      if (!local_err) {
> @@ -2696,6 +2697,7 @@ out:
>  void qmp_block_dirty_bitmap_add(const char *node, const char *name,
>                                  bool has_granularity, uint32_t granularity,
>                                  bool has_persistent, bool persistent,
> +                                bool has_autoload, bool autoload,
>                                  Error **errp)
>  {
>      AioContext *aio_context;
> @@ -2729,10 +2731,26 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name,
>      if (!has_persistent) {
>          persistent = false;
>      }
> +    if (!has_autoload) {
> +        autoload = false;
> +    }
> +
> +    if (autoload && !persistent) {
> +        error_setg(errp, "Autoload flag must be used only for persistent"
> +                         "bitmaps");

Missing space between "persistent" and "bitmaps".

Also, technically I think you should throw this error if has_autoload is
true instead of autoload. I would consider it wrong if a user specified
autoload at all, even autoload=false, without setting persistent=true.
But if you disagree, then please keep the condition the way it is.

> +        goto out;
> +    }
>  
>      bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp);
> -    if (bitmap != NULL) {
> -        bdrv_dirty_bitmap_set_persistance(bitmap, persistent);
> +    if (bitmap == NULL) {
> +        goto out;
> +    }
> +
> +    if (persistent) {
> +        bdrv_dirty_bitmap_set_persistance(bitmap, true);
> +        if (autoload) {
> +            bdrv_dirty_bitmap_set_autoload(bitmap, true);
> +        }

By the way, a simpler way to do the same would be just

bdrv_dirty_bitmap_set_persistance(bitmap, persistent);
bdrv_dirty_bitmap_set_autoload(bitmap, autoload);

But if you prefer this explicit style, that's fine, too.

>      }
>  
>   out:
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 2bf56cd..087a681 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1239,11 +1239,16 @@
>  #              corresponding block device on it's close. Default is false.
>  #              For block-dirty-bitmap-add. (Since 2.8)
>  #
> +# @autoload: #optional the bitmap will be autoloaded on it's storage image
> +#            open.

I'd rephrase this like "The bitmap will be automatically loaded when the
image it is stored in is opened." (Or keep "autoloaded" instead of
"automatically loaded", that doesn't really matter.)

>                     This flag is only for persistent bitmap and needed to inform
> +#            block driver that bitmap should be autoloaded on the next image
> +#            open.

Again, the user doesn't really have to care how the internals work. All
they need to know is that the image will be automatically loaded when
the image is opened the next time, and that is something the first
sentence told them already.

(Of course, "This flag may only be specified for persistent bitmaps"
should stay.)

>                     Default is false. For block-dirty-bitmap-add. (Since 2.8)

Again, I don't see the point of the last sentence.

Max

> +#
>  # Since 2.4
>  ##
>  { 'struct': 'BlockDirtyBitmapAdd',
>    'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
> -  '*persistent': 'bool' } }
> +  '*persistent': 'bool', '*autoload': 'bool' } }
>  
>  ##
>  # @block-dirty-bitmap-add
> diff --git a/qmp-commands.hx b/qmp-commands.hx
> index 434b418..8f4e841 100644
> --- a/qmp-commands.hx
> +++ b/qmp-commands.hx
> @@ -1441,7 +1441,7 @@ EQMP
>  
>      {
>          .name       = "block-dirty-bitmap-add",
> -        .args_type  = "node:B,name:s,granularity:i?,persistent:b?",
> +        .args_type  = "node:B,name:s,granularity:i?,persistent:b?,autoload:b?",
>          .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
>      },
>  
> @@ -1461,6 +1461,9 @@ Arguments:
>  - "persistent": bitmap will be saved to corresponding block device
>                  on it's close. Block driver should maintain persistent bitmaps
>                  (json-bool, optional, default false) (Since 2.8)
> +- "autoload": only for persistent bitmaps. Bitmap will be autoloaded on it's
> +              storage image open. (json-bool, optional, default false)
> +              (Since 2.8)
>  
>  Example:
>  
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 18/22] qapi: add md5 checksum of last dirty bitmap level to query-block

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1061 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Reviewed-by: John Snow <jsnow@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/dirty-bitmap.c   | 1 +
>  include/qemu/hbitmap.h | 8 ++++++++
>  qapi/block-core.json   | 5 ++++-
>  util/hbitmap.c         | 8 ++++++++
>  4 files changed, 21 insertions(+), 1 deletion(-)

Having read John's and Eric's comments, I won't block this patch, but I
won't give an R-b either.

It's probably true that this will not significantly slow down the
query-block call, but doing this only for debugging does not seem right
to me.

I'm not sure what the right way would be to get this information out
(...maybe make it optional and set it only if qtest_enabled() is true?),
but in my opinion this is not the right way.

Since I'm not the maintainer of the bitmap code (Fam and John are, even
though their MAINTAINERS patch is not in master still...), I can't and
won't block this, though.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 18/22] qapi: add md5 checksum of last dirty bitmap level to query-block

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1571 bytes --]

On 10.10.2016 18:44, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Reviewed-by: John Snow <jsnow@redhat.com>
>> Reviewed-by: Eric Blake <eblake@redhat.com>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>  block/dirty-bitmap.c   | 1 +
>>  include/qemu/hbitmap.h | 8 ++++++++
>>  qapi/block-core.json   | 5 ++++-
>>  util/hbitmap.c         | 8 ++++++++
>>  4 files changed, 21 insertions(+), 1 deletion(-)
> 
> Having read John's and Eric's comments, I won't block this patch, but I
> won't give an R-b either.
> 
> It's probably true that this will not significantly slow down the
> query-block call, but doing this only for debugging does not seem right
> to me.
> 
> I'm not sure what the right way would be to get this information out
> (...maybe make it optional and set it only if qtest_enabled() is true?),
> but in my opinion this is not the right way.

By the way, the cleanest way I can come up with (which I didn't write
about in my first reply because it's not so trivial) would be some kind
of debugging QMP command convention. For instance, we could say that all
debugging commands have an x-debug- prefix, and then you could add an
x-debug-get-bitmap-md5 to read the MD5 hash of a named bitmap. That
would appear to be the cleanest way to do this to me.

Max

> Since I'm not the maintainer of the bitmap code (Fam and John are, even
> though their MAINTAINERS patch is not in master still...), I can't and
> won't block this, though.
> 
> Max
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 19/22] iotests: test qcow2 persistent dirty bitmap

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 4197 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  tests/qemu-iotests/165     | 87 ++++++++++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/165.out |  5 +++
>  tests/qemu-iotests/group   |  1 +
>  3 files changed, 93 insertions(+)
>  create mode 100755 tests/qemu-iotests/165
>  create mode 100644 tests/qemu-iotests/165.out
> 
> diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165
> new file mode 100755
> index 0000000..a69799c
> --- /dev/null
> +++ b/tests/qemu-iotests/165
> @@ -0,0 +1,87 @@
> +#!/usr/bin/env python
> +#
> +# Tests for persistent dirty bitmaps.
> +#
> +# Copyright: Vladimir Sementsov-Ogievskiy 2015-2016
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +#
> +
> +import os
> +import iotests
> +from iotests import qemu_img
> +
> +disk = os.path.join(iotests.test_dir, 'disk')
> +disk_size = 0x40000000 # 1G
> +
> +# regions for qemu_io: (start, count) in bytes
> +regions1 = ((0,        0x100000),
> +            (0x200000, 0x100000))
> +
> +regions2 = ((0x10000000, 0x20000),
> +            (0x39990000, 0x10000))

Did you mean 0x3fff0000?

Looks good apart from that (and the fact that I don't like the @md5
field in query-block too much, but that's a matter for patch 18).

Max

> +
> +class TestPersistentDirtyBitmap(iotests.QMPTestCase):
> +
> +    def setUp(self):
> +        qemu_img('create', '-f', iotests.imgfmt, disk, str(disk_size))
> +
> +    def tearDown(self):
> +        os.remove(disk)
> +
> +    def mkVm(self):
> +        return iotests.VM().add_drive(disk)
> +
> +    def getMd5(self):
> +        result = self.vm.qmp('query-block');
> +        return result['return'][0]['dirty-bitmaps'][0]['md5']
> +
> +    def checkBitmap(self, md5):
> +        result = self.vm.qmp('query-block');
> +        self.assert_qmp(result, 'return[0]/dirty-bitmaps[0]/md5', md5);
> +
> +    def writeRegions(self, regions):
> +        for r in regions:
> +          self.vm.hmp_qemu_io('drive0',
> +                              'write %d %d' % r)
> +
> +    def qmpAddBitmap(self):
> +        self.vm.qmp('block-dirty-bitmap-add', node='drive0',
> +                    name='bitmap0', persistent=True, autoload=True)
> +
> +    def test_persistent(self):
> +        self.vm = self.mkVm()
> +        self.vm.launch()
> +        self.qmpAddBitmap()
> +
> +        self.writeRegions(regions1)
> +        md5 = self.getMd5()
> +
> +        self.vm.shutdown()
> +        self.vm = self.mkVm()
> +        self.vm.launch()
> +
> +        self.checkBitmap(md5)
> +        self.writeRegions(regions2)
> +        md5 = self.getMd5()
> +
> +        self.vm.shutdown()
> +        self.vm.launch()
> +
> +        self.checkBitmap(md5)
> +
> +        self.vm.shutdown()
> +
> +if __name__ == '__main__':
> +    iotests.main(supported_fmts=['qcow2'])
> diff --git a/tests/qemu-iotests/165.out b/tests/qemu-iotests/165.out
> new file mode 100644
> index 0000000..ae1213e
> --- /dev/null
> +++ b/tests/qemu-iotests/165.out
> @@ -0,0 +1,5 @@
> +.
> +----------------------------------------------------------------------
> +Ran 1 tests
> +
> +OK
> diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
> index 503eb27..ed14294 100644
> --- a/tests/qemu-iotests/group
> +++ b/tests/qemu-iotests/group
> @@ -161,3 +161,4 @@
>  159 rw auto quick
>  160 rw auto quick
>  162 auto quick
> +165 rw auto quick
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 20/22] qcow2-dirty-bitmap: refcounts

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 6619 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Calculate refcounts for dirty bitmaps.

The commit message should mention that this is for qcow2's qemu-img
check implementation.

> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c   | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  block/qcow2-refcount.c | 14 +++++++-----
>  block/qcow2.h          |  5 +++++
>  3 files changed, 74 insertions(+), 5 deletions(-)
> 
> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> index 76f7e2b..6d9394a 100644
> --- a/block/qcow2-bitmap.c
> +++ b/block/qcow2-bitmap.c
> @@ -901,3 +901,63 @@ out:
>      g_free(new_dir);
>      g_free(dir);
>  }
> +
> +int qcow2_dirty_bitmaps_refcounts(BlockDriverState *bs,
> +                                  BdrvCheckResult *res,
> +                                  void **refcount_table,
> +                                  int64_t *refcount_table_size)

I'd rename this function to make clear that this is for checking the
refcounts, e.g. to "qcow2_check_dirty_bitmaps_refcounts" or
"qcow2_count_dirty_bitmaps_refcounts" or just
"qcow2_check_dirty_bitmaps". Probably the last one is the best because
this function should ideally perform a full check of the dirty bitmaps
extension.

> +{
> +    BDRVQcow2State *s = bs->opaque;
> +    uint8_t *dir;
> +    Qcow2BitmapDirEntry *e;
> +    Error *local_err = NULL;
> +
> +    int i;
> +    int ret = inc_refcounts(bs, res, refcount_table, refcount_table_size,
> +                            s->bitmap_directory_offset,
> +                            s->bitmap_directory_size);
> +    if (ret < 0) {
> +        return ret;
> +    }
> +
> +    dir = directory_read(bs, s->bitmap_directory_offset,
> +                         s->bitmap_directory_size, &local_err);
> +    if (dir == NULL) {
> +        error_report_err(local_err);

I think you should increment res->corruptions here.

> +        return -EINVAL;
> +    }
> +
> +    for_each_bitmap_dir_entry(e, dir, s->bitmap_directory_size) {
> +        uint64_t *bitmap_table = NULL;

I think you should call check_constraints() here.

> +
> +        ret = inc_refcounts(bs, res, refcount_table, refcount_table_size,
> +                            e->bitmap_table_offset,
> +                            e->bitmap_table_size);

Probably rather e->bitmap_table_size * sizeof(uint64_t).

> +        if (ret < 0) {
> +            return ret;
> +        }
> +
> +        ret = bitmap_table_load(bs, e, &bitmap_table);
> +        if (ret < 0) {

Again, it would make sense to increment res->corruptions here.

> +            return ret;
> +        }
> +
> +        for (i = 0; i < e->bitmap_table_size; ++i) {
> +            uint64_t off = be64_to_cpu(bitmap_table[i]);
> +            if (off <= 1) {
> +                continue;
> +            }

As I said in some other patch, I'd write this condition differently
(with an offset mask, etc.).

Also, you should check that the offset is aligned to a cluster boundary
and that no unknown flags are set.

> +
> +            ret = inc_refcounts(bs, res, refcount_table, refcount_table_size,
> +                                off, s->cluster_size);
> +            if (ret < 0) {
> +                g_free(bitmap_table);
> +                return ret;
> +            }
> +        }
> +
> +        g_free(bitmap_table);
> +    }
> +
> +    return 0;

dir is leaked here.

> +}
> diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
> index cbfb3fe..05bcc57 100644
> --- a/block/qcow2-refcount.c
> +++ b/block/qcow2-refcount.c
> @@ -1309,11 +1309,9 @@ static int realloc_refcount_array(BDRVQcow2State *s, void **array,
>   *
>   * Modifies the number of errors in res.
>   */
> -static int inc_refcounts(BlockDriverState *bs,
> -                         BdrvCheckResult *res,
> -                         void **refcount_table,
> -                         int64_t *refcount_table_size,
> -                         int64_t offset, int64_t size)
> +int inc_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
> +                  void **refcount_table, int64_t *refcount_table_size,
> +                  int64_t offset, int64_t size)

First, if you make this function public, you have to give it a qcow2_
prefix.

Second, if this function is public, it should have a name that makes
sense. inc_refcounts() sounds as if it's the same as update_refcount()
with an addend of 1. I'd rename it qcow2_inc_refcounts_imrt(), because
that's probably the shortest name I can come up with (and
qcow2-refcount.c explains what an IMRT is in some comment).

>  {
>      BDRVQcow2State *s = bs->opaque;
>      uint64_t start, last, cluster_offset, k, refcount;
> @@ -1843,6 +1841,12 @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
>          return ret;
>      }
>  
> +    /* dirty bitmaps */
> +    ret = qcow2_dirty_bitmaps_refcounts(bs, res, refcount_table, nb_clusters);
> +    if (ret < 0) {
> +        return ret;
> +    }
> +
>      return check_refblocks(bs, res, fix, rebuild, refcount_table, nb_clusters);
>  }
>  
> diff --git a/block/qcow2.h b/block/qcow2.h
> index a5e7592..0a050ea 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -563,6 +563,9 @@ int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset,
>                                   int64_t size);
>  int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset,
>                                    int64_t size);
> +int inc_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
> +                  void **refcount_table, int64_t *refcount_table_size,
> +                  int64_t offset, int64_t size);
>  
>  int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,
>                                  BlockDriverAmendStatusCB *status_cb,
> @@ -616,6 +619,8 @@ int qcow2_read_snapshots(BlockDriverState *bs);
>  int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp);
>  
>  int qcow2_delete_bitmaps(BlockDriverState *bs);
> +int qcow2_dirty_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
> +    void **refcount_table, int64_t *refcount_table_size);

Normally we try to align parameters along the opening parenthesis as
long as there is enough space, and there is enough space here.

Max

>  
>  /* qcow2-cache.c functions */
>  Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables);
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 18/22] qapi: add md5 checksum of last dirty bitmap level to query-block

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

On 10/10/2016 12:03 PM, Max Reitz wrote:
>> I'm not sure what the right way would be to get this information out
>> (...maybe make it optional and set it only if qtest_enabled() is true?),
>> but in my opinion this is not the right way.
> 
> By the way, the cleanest way I can come up with (which I didn't write
> about in my first reply because it's not so trivial) would be some kind
> of debugging QMP command convention. For instance, we could say that all
> debugging commands have an x-debug- prefix, and then you could add an
> x-debug-get-bitmap-md5 to read the MD5 hash of a named bitmap. That
> would appear to be the cleanest way to do this to me.

Anything named x- is automatically not stable, and therefore free to use
in the testsuite without having to worry about keeping it backwards
compatible (libvirt won't touch x- commands).  The suggestion of
x-debug- to mark it as specifically for debug use is reasonable.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 05/22] qcow2-bitmap: structs and consts

[Vladimir Sementsov-Ogievskiy]

On 01.10.2016 17:34, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Create block/qcow2-bitmap.c
>> Add data structures and constraints accordingly to docs/specs/qcow2.txt
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block/Makefile.objs  |  2 +-
>>   block/qcow2-bitmap.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++
>>   block/qcow2.h        | 29 +++++++++++++++++++++++++++++
>>   3 files changed, 77 insertions(+), 1 deletion(-)
>>   create mode 100644 block/qcow2-bitmap.c
>>
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index fa4d8b8..0f661bb 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -1,5 +1,5 @@
>>   block-obj-y += raw_bsd.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o dmg.o
>> -block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
>> +block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o
>>   block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
>>   block-obj-y += qed-check.o
>>   block-obj-$(CONFIG_VHDX) += vhdx.o vhdx-endian.o vhdx-log.o
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> new file mode 100644
>> index 0000000..cd18b07
>> --- /dev/null
>> +++ b/block/qcow2-bitmap.c
>> @@ -0,0 +1,47 @@
>> +/*
>> + * Bitmaps for the QCOW version 2 format
>> + *
>> + * Copyright (c) 2014-2016 Vladimir Sementsov-Ogievskiy
>> + *
>> + * This file is derived from qcow2-snapshot.c, original copyright:
>> + * Copyright (c) 2004-2006 Fabrice Bellard
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to deal
>> + * in the Software without restriction, including without limitation the rights
>> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>> + * copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
>> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>> + * THE SOFTWARE.
>> + */
>> +
>> +/* NOTICE: BME here means Bitmaps Extension and used as a namespace for
>> + * _internal_ constants. Please do not use this _internal_ abbreviation for
>> + * other needs and/or outside of this file. */
>> +
>> +/* Bitmap directory entry constraints */
>> +#define BME_MAX_TABLE_SIZE 0x8000000
>> +#define BME_MAX_PHYS_SIZE 0x20000000 /* 512 mb */
> I suppose BME_MAX_TABLE_SIZE (8M) is greater than BME_MAX_PHYS_SIZE (512
> MB) divided by the cluster size (>= 512; 512 MB / cluster_size <= 1 MB)
> because fully zero or one clusters do not require any physical space?
>
> Makes some sense, but I can see that this might make give some trouble
> when trying to serialize overly large bitmaps. But I guess that comes
> later in this series, so I'll wait for that point.
>
> Another thing is that 512 MB is rather big. It gets worse: The bitmap
> may only require 512 MB on disk, but with a maximum table size of 8 MB,
> it can require up to 8M * cluster_size in memory (with just 64 MB of
> disk space!) by using the "read as all zeroes" or "read as all ones"
> flags. With the default cluster size of 64 kB, this would be 512 GB in
> RAM. That sounds bad to me.
>
> Well, it is probably fine as long as the bitmap is not auto-loaded...
> But we do have a flag for exactly that. So it seems to me that a
> manipulated image can easily consume huge amounts of RAM on the host.
>
> So I think we also need some sane limitation on the in-RAM size of a
> bitmap (which is BME_MAX_TABLE_SIZE * cluster_size, as far as I
> understand). The question of course is, what is sane? For a server
> system with no image manipulation possible from the outside, 1 GB may be
> completely fine. But imagine you download some qcow2 image to your
> laptop. Then, 1 GB may not be fine, actually.
>
> Maybe it would make sense to use a runtime-adjustable limit here?

Actualy BME_MAX_PHYS_SIZE is this limit:
in check_constraints we have

uint64_t phys_bitmap_bytes =
         (uint64_t)h->bitmap_table_size * s->cluster_size;

...

(phys_bitmap_bytes > BME_MAX_PHYS_SIZE) ||

>
>> +#define BME_MAX_GRANULARITY_BITS 31
>> +#define BME_MIN_GRANULARITY_BITS 9
>> +#define BME_MAX_NAME_SIZE 1023
>> +
>> +/* Bitmap directory entry flags */
>> +#define BME_RESERVED_FLAGS 0xffffffff
>> +
>> +/* bits [1, 8] U [56, 63] are reserved */
>> +#define BME_TABLE_ENTRY_RESERVED_MASK 0xff000000000001fe
> ull suffix is missing.
>
>> +
>> +typedef enum BitmapType {
>> +    BT_DIRTY_TRACKING_BITMAP = 1
>> +} BitmapType;
>> diff --git a/block/qcow2.h b/block/qcow2.h
>> index b36a7bf..0480b8b 100644
>> --- a/block/qcow2.h
>> +++ b/block/qcow2.h
>> @@ -52,6 +52,10 @@
>>    * space for snapshot names and IDs */
>>   #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS)
>>   
>> +/* Bitmap header extension constraints */
>> +#define QCOW_MAX_DIRTY_BITMAPS 65535
>> +#define QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE (1024 * QCOW_MAX_DIRTY_BITMAPS)
>> +
>>   /* indicate that the refcount of the referenced cluster is exactly one. */
>>   #define QCOW_OFLAG_COPIED     (1ULL << 63)
>>   /* indicate that the cluster is compressed (they never have the copied flag) */
>> @@ -142,6 +146,22 @@ typedef struct QEMU_PACKED QCowSnapshotHeader {
>>       /* name follows  */
>>   } QCowSnapshotHeader;
>>   
>> +/* Qcow2BitmapDirEntry is actually a bitmap directory entry */
> This comment doesn't make a whole lot of sense now that this structure
> is indeed called what it "actually is". ;-)
>
>> +typedef struct QEMU_PACKED Qcow2BitmapDirEntry {
>> +    /* header is 8 byte aligned */
>> +    uint64_t bitmap_table_offset;
>> +
>> +    uint32_t bitmap_table_size;
>> +    uint32_t flags;
>> +
>> +    uint8_t type;
>> +    uint8_t granularity_bits;
>> +    uint16_t name_size;
>> +    uint32_t extra_data_size;
>> +    /* extra data follows  */
>> +    /* name follows  */
>> +} Qcow2BitmapDirEntry;
>> +
>>   typedef struct QEMU_PACKED QCowSnapshotExtraData {
>>       uint64_t vm_state_size_large;
>>       uint64_t disk_size;
>> @@ -222,6 +242,15 @@ typedef uint64_t Qcow2GetRefcountFunc(const void *refcount_array,
>>   typedef void Qcow2SetRefcountFunc(void *refcount_array,
>>                                     uint64_t index, uint64_t value);
>>   
>> +/* Be careful, Qcow2BitmapHeaderExt is not an extension of Qcow2BitmapDirEntry, it
>> + * is Qcow2 header extension */
> And this makes even less sense now.
>
> (These comments don't stop me from giving an R-b, but I'm not so sure
> about the constants...)
>
> Max
>
>> +typedef struct Qcow2BitmapHeaderExt {
>> +    uint32_t nb_bitmaps;
>> +    uint32_t reserved32;
>> +    uint64_t bitmap_directory_size;
>> +    uint64_t bitmap_directory_offset;
>> +} QEMU_PACKED Qcow2BitmapHeaderExt;
>> +
>>   typedef struct BDRVQcow2State {
>>       int cluster_bits;
>>       int cluster_size;
>>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 06/22] qcow2: add dirty bitmaps extension

[Vladimir Sementsov-Ogievskiy]

On 01.10.2016 17:46, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Add dirty bitmap extension as specified in docs/specs/qcow2.txt.
>> For now, just mirror extension header into Qcow2 state and check
>> constraints.
>>
>> For now, disable image resize if it has bitmaps. It will be fixed later.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block/qcow2.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>   block/qcow2.h |  4 +++
>>   2 files changed, 87 insertions(+)
>>
>> diff --git a/block/qcow2.c b/block/qcow2.c
>> index c079aa8..08c4ef9 100644
>> --- a/block/qcow2.c
>> +++ b/block/qcow2.c
> [...]
>
>> @@ -162,6 +164,62 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
>>               }
>>               break;
>>   
>> +        case QCOW2_EXT_MAGIC_DIRTY_BITMAPS:
>> +            ret = bdrv_pread(bs->file, offset, &bitmaps_ext, ext.len);
> Overflows with ext.len > sizeof(bitmaps_ext).
>
> (ext.len < sizeof(bitmaps_ext) is also wrong, but less dramatically so.)
>
>> +            if (ret < 0) {
>> +                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
>> +                                 "Could not read ext header");
>> +                return ret;
>> +            }
>> +
>> +            if (bitmaps_ext.reserved32 != 0) {
>> +                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
>> +                                 "Reserved field is not zero.");
> Please drop the full stop at the end.

what do you mean? goto to fail: here? or not stop at all, just print error?

>
>> +                return -EINVAL;
>> +            }
>> +
>> +            be32_to_cpus(&bitmaps_ext.nb_bitmaps);
>> +            be64_to_cpus(&bitmaps_ext.bitmap_directory_size);
>> +            be64_to_cpus(&bitmaps_ext.bitmap_directory_offset);
>> +
>> +            if (bitmaps_ext.nb_bitmaps > QCOW_MAX_DIRTY_BITMAPS) {
>> +                error_setg(errp, "ERROR: bitmaps_ext: "
>> +                                 "too many dirty bitmaps");
>> +                return -EINVAL;
>> +            }
>> +
>> +            if (bitmaps_ext.nb_bitmaps == 0) {
>> +                error_setg(errp, "ERROR: bitmaps_ext: "
>> +                                 "found bitmaps extension with zero bitmaps");
>> +                return -EINVAL;
>> +            }
>> +
>> +            if (bitmaps_ext.bitmap_directory_offset & (s->cluster_size - 1)) {
>> +                error_setg(errp, "ERROR: bitmaps_ext: "
>> +                                 "wrong dirty bitmap directory offset");
> s/wrong/invalid/
>
>> +                return -EINVAL;
>> +            }
>> +
>> +            if (bitmaps_ext.bitmap_directory_size >
>> +                QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE) {
>> +                error_setg(errp, "ERROR: bitmaps_ext: "
>> +                                 "too large dirty bitmap directory");
>> +                return -EINVAL;
>> +            }
>> +
>> +            s->nb_bitmaps = bitmaps_ext.nb_bitmaps;
>> +            s->bitmap_directory_offset =
>> +                    bitmaps_ext.bitmap_directory_offset;
>> +            s->bitmap_directory_size =
>> +                    bitmaps_ext.bitmap_directory_size;
>> +
>> +#ifdef DEBUG_EXT
>> +            printf("Qcow2: Got dirty bitmaps extension:"
>> +                   " offset=%" PRIu64 " nb_bitmaps=%" PRIu32 "\n",
>> +                   s->bitmap_directory_offset, s->nb_bitmaps);
>> +#endif
>> +            break;
>> +
>>           default:
>>               /* unknown magic - save it in case we need to rewrite the header */
>>               {
> [...]
>
>> @@ -2509,6 +2585,13 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset)
>>           return -ENOTSUP;
>>       }
>>   
>> +    /* cannot proceed if image has bitmaps */
>> +    if (s->nb_bitmaps) {
>> +        /* FIXME */
> I'd call it a TODO, but that's probably a matter of taste.
>
>> +        error_report("Can't resize an image which has bitmaps");
>> +        return -ENOTSUP;
>> +    }
>> +
>>       /* shrinking is currently not supported */
>>       if (offset < bs->total_sectors * 512) {
>>           error_report("qcow2 doesn't support shrinking images yet");
> Max
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 09/22] block: introduce persistent dirty bitmaps

[Vladimir Sementsov-Ogievskiy]

On 07.10.2016 20:54, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
>> on bdrv_close, using format driver. Format driver should maintain bitmap
>> storing.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block.c                      | 30 ++++++++++++++++++++++++++++++
>>   block/dirty-bitmap.c         | 27 +++++++++++++++++++++++++++
>>   block/qcow2-bitmap.c         |  1 +
>>   include/block/block.h        |  2 ++
>>   include/block/block_int.h    |  2 ++
>>   include/block/dirty-bitmap.h |  6 ++++++
>>   6 files changed, 68 insertions(+)
>>
>> diff --git a/block.c b/block.c
>> index 804e3d4..1cde03a 100644
>> --- a/block.c
>> +++ b/block.c
>> @@ -2196,6 +2196,7 @@ void bdrv_reopen_abort(BDRVReopenState *reopen_state)
>>   static void bdrv_close(BlockDriverState *bs)
>>   {
>>       BdrvAioNotifier *ban, *ban_next;
>> +    Error *local_err = NULL;
>>   
>>       assert(!bs->job);
>>       assert(!bs->refcnt);
>> @@ -2204,6 +2205,10 @@ static void bdrv_close(BlockDriverState *bs)
>>       bdrv_flush(bs);
>>       bdrv_drain(bs); /* in case flush left pending I/O */
>>   
>> +    bdrv_store_persistent_bitmaps(bs, &local_err);
>> +    if (local_err != NULL) {
>> +        error_report_err(local_err);
>> +    }
> That seems pretty wrong to me. If the persistent bitmaps cannot be
> stored, the node should not be closed to avoid loss of data.
>
>>       bdrv_release_named_dirty_bitmaps(bs);
> Especially since the next function will just drop all the dirty bitmaps.
>
> I see the issue that bdrv_close() is only called by bdrv_delete() which
> in turn is only called by bdrv_unref(); and how are you supposed to
> react to bdrv_unref() failing?
>
> So I'm not sure how this issue should be addressed, but this is most
> certainly not ideal. You should not just drop supposedly persistent
> dirty bitmaps if they cannot be saved.
>
> We really should to have some way to keep the bitmap around if it cannot
> be saved, but I don't know how to do that either.
>
> In any case, we should make sure that the node supports saving
> persistent dirty bitmaps, because having persistent dirty bitmaps at a
> node that does not support them is something we can and must prevent
> beforehand.
>
> But I don't know how to handle failure if writing the dirty bitmap
> fails. I guess one could argue that it's the same as bdrv_flush()
> failing and thus can be handled in the same way, i.e. ignore it. I'm not
> happy with that, but I'd accept it if there's no other way.

For now, the only usage of these bitmaps is incremental backup and 
bitmaps are not critical data. If we lost them we will just do full 
backup. If there will be some critical persistent bdrv dirty bitmaps in 
future, we can introduce a callback BdrvDirtyBitmap.store_failed for 
them, which will somehow handle that case.. Detach bitmap from bs and 
save it in memory, add qmp commands to raw-dump them, etc.. I

>
>>       assert(QLIST_EMPTY(&bs->dirty_bitmaps));
>>   
>> @@ -3969,3 +3974,28 @@ void bdrv_del_child(BlockDriverState *parent_bs, BdrvChild *child, Error **errp)
>>   
>>       parent_bs->drv->bdrv_del_child(parent_bs, child, errp);
>>   }
>> +
>> +void bdrv_store_persistent_bitmaps(BlockDriverState *bs, Error **errp)
>> +{
>> +    BlockDriver *drv = bs->drv;
>> +
>> +    if (!bdrv_has_persistent_bitmaps(bs)) {
>> +        return;
>> +    }
>> +
>> +    if (!drv) {
>> +        error_setg_errno(errp, ENOMEDIUM,
>> +                         "Can't store persistent bitmaps to %s",
>> +                         bdrv_get_device_or_node_name(bs));
>> +        return;
>> +    }
>> +
>> +    if (!drv->bdrv_store_persistent_bitmaps) {
>> +        error_setg_errno(errp, ENOTSUP,
>> +                         "Can't store persistent bitmaps to %s",
>> +                         bdrv_get_device_or_node_name(bs));
>> +        return;
>> +    }
>> +
>> +    drv->bdrv_store_persistent_bitmaps(bs, errp);
>> +}
>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>> index 623e1d1..0314581 100644
>> --- a/block/dirty-bitmap.c
>> +++ b/block/dirty-bitmap.c
>> @@ -44,6 +44,7 @@ struct BdrvDirtyBitmap {
>>       int64_t size;               /* Size of the bitmap (Number of sectors) */
>>       bool disabled;              /* Bitmap is read-only */
>>       int active_iterators;       /* How many iterators are active */
>> +    bool persistent;            /* bitmap must be saved to owner disk image */
>>       bool autoload;              /* bitmap must be autoloaded on image opening */
>>       QLIST_ENTRY(BdrvDirtyBitmap) list;
>>   };
>> @@ -72,6 +73,7 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
>>       g_free(bitmap->name);
>>       bitmap->name = NULL;
>>   
>> +    bitmap->persistent = false;
>>       bitmap->autoload = false;
>>   }
>>   
>> @@ -241,6 +243,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
>>       bitmap->name = NULL;
>>       successor->name = name;
>>       bitmap->successor = NULL;
>> +    successor->persistent = bitmap->persistent;
>> +    bitmap->persistent = false;
>>       successor->autoload = bitmap->autoload;
>>       bitmap->autoload = false;
>>       bdrv_release_dirty_bitmap(bs, bitmap);
>> @@ -555,3 +559,26 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap)
>>   {
>>       return bitmap->autoload;
>>   }
>> +
>> +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
>> +                                                bool persistent)
> The second parameter should be aligned to the opening parenthesis.
>
> Max
>
>> +{
>> +    bitmap->persistent = persistent;
>> +}
>> +
>> +bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap)
>> +{
>> +    return bitmap->persistent;
>> +}
>> +
>> +bool bdrv_has_persistent_bitmaps(BlockDriverState *bs)
>> +{
>> +    BdrvDirtyBitmap *bm;
>> +    QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
>> +        if (bm->persistent) {
>> +            return true;
>> +        }
>> +    }
>> +
>> +    return false;
>> +}


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 09/22] block: introduce persistent dirty bitmaps

[Vladimir Sementsov-Ogievskiy]

On 07.10.2016 22:28, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
>> on bdrv_close, using format driver. Format driver should maintain bitmap
>> storing.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block.c                      | 30 ++++++++++++++++++++++++++++++
>>   block/dirty-bitmap.c         | 27 +++++++++++++++++++++++++++
>>   block/qcow2-bitmap.c         |  1 +
>>   include/block/block.h        |  2 ++
>>   include/block/block_int.h    |  2 ++
>>   include/block/dirty-bitmap.h |  6 ++++++
>>   6 files changed, 68 insertions(+)
> [...]
>
>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>> index 623e1d1..0314581 100644
>> --- a/block/dirty-bitmap.c
>> +++ b/block/dirty-bitmap.c
> [...]
>
>> @@ -555,3 +559,26 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap)
>>   {
>>       return bitmap->autoload;
>>   }
>> +
>> +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
>> +                                                bool persistent)
>> +{
>> +    bitmap->persistent = persistent;
> After some thinking, I think this function should be more complex: It
> should check whether the node the bitmap is attached to actually can
> handle persistent bitmaps and whether it would actually support storing
> *this* bitmap.
>
> For instance, a qcow2 node would not support writing overly large
> bitmaps (limited by BME_MAX_TABLE_SIZE and BME_MAX_PHYS_SIZE) or bitmaps
> with overly large granularities (BME_MAX_GRANULARITY_BITS) or bitmaps
> whose name is already occupied by some bitmap that is already stored in
> the file but has not been loaded.
>
> Checking this here will trivially prevent users from creating such
> bitmaps and will also preempt detection of such failures during
> bdrv_close() when they cannot be handled gracefully.
>
> Max

Good point, but I can't do it exactly as you say, because I call this 
function from qcow2_read_bitmaps, for just created bitmap and it should 
not be checked and of course it's name is occupied..

>
>> +}


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 09/22] block: introduce persistent dirty bitmaps

[Vladimir Sementsov-Ogievskiy]

On 12.10.2016 14:38, Vladimir Sementsov-Ogievskiy wrote:
> On 07.10.2016 22:28, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
>>> on bdrv_close, using format driver. Format driver should maintain 
>>> bitmap
>>> storing.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   block.c                      | 30 ++++++++++++++++++++++++++++++
>>>   block/dirty-bitmap.c         | 27 +++++++++++++++++++++++++++
>>>   block/qcow2-bitmap.c         |  1 +
>>>   include/block/block.h        |  2 ++
>>>   include/block/block_int.h    |  2 ++
>>>   include/block/dirty-bitmap.h |  6 ++++++
>>>   6 files changed, 68 insertions(+)
>> [...]
>>
>>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>>> index 623e1d1..0314581 100644
>>> --- a/block/dirty-bitmap.c
>>> +++ b/block/dirty-bitmap.c
>> [...]
>>
>>> @@ -555,3 +559,26 @@ bool bdrv_dirty_bitmap_get_autoload(const 
>>> BdrvDirtyBitmap *bitmap)
>>>   {
>>>       return bitmap->autoload;
>>>   }
>>> +
>>> +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
>>> +                                                bool persistent)
>>> +{
>>> +    bitmap->persistent = persistent;
>> After some thinking, I think this function should be more complex: It
>> should check whether the node the bitmap is attached to actually can
>> handle persistent bitmaps and whether it would actually support storing
>> *this* bitmap.
>>
>> For instance, a qcow2 node would not support writing overly large
>> bitmaps (limited by BME_MAX_TABLE_SIZE and BME_MAX_PHYS_SIZE) or bitmaps
>> with overly large granularities (BME_MAX_GRANULARITY_BITS) or bitmaps
>> whose name is already occupied by some bitmap that is already stored in
>> the file but has not been loaded.
>>
>> Checking this here will trivially prevent users from creating such
>> bitmaps and will also preempt detection of such failures during
>> bdrv_close() when they cannot be handled gracefully.
>>
>> Max
>
> Good point, but I can't do it exactly as you say, because I call this 
> function from qcow2_read_bitmaps, for just created bitmap and it 
> should not be checked and of course it's name is occupied..

So, I'll add an additional checking function, to call it from 
qmp_block_dirty_bitmap_add, if persistent parameter is set to true.

>
>>
>>> +}
>
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 05/22] qcow2-bitmap: structs and consts

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 5671 bytes --]

On 11.10.2016 13:50, Vladimir Sementsov-Ogievskiy wrote:
> On 01.10.2016 17:34, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> Create block/qcow2-bitmap.c
>>> Add data structures and constraints accordingly to docs/specs/qcow2.txt
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   block/Makefile.objs  |  2 +-
>>>   block/qcow2-bitmap.c | 47
>>> +++++++++++++++++++++++++++++++++++++++++++++++
>>>   block/qcow2.h        | 29 +++++++++++++++++++++++++++++
>>>   3 files changed, 77 insertions(+), 1 deletion(-)
>>>   create mode 100644 block/qcow2-bitmap.c
>>>
>>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>>> index fa4d8b8..0f661bb 100644
>>> --- a/block/Makefile.objs
>>> +++ b/block/Makefile.objs
>>> @@ -1,5 +1,5 @@
>>>   block-obj-y += raw_bsd.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o
>>> vvfat.o dmg.o
>>> -block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o
>>> qcow2-snapshot.o qcow2-cache.o
>>> +block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o
>>> qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o
>>>   block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o
>>> qed-cluster.o
>>>   block-obj-y += qed-check.o
>>>   block-obj-$(CONFIG_VHDX) += vhdx.o vhdx-endian.o vhdx-log.o
>>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>>> new file mode 100644
>>> index 0000000..cd18b07
>>> --- /dev/null
>>> +++ b/block/qcow2-bitmap.c
>>> @@ -0,0 +1,47 @@
>>> +/*
>>> + * Bitmaps for the QCOW version 2 format
>>> + *
>>> + * Copyright (c) 2014-2016 Vladimir Sementsov-Ogievskiy
>>> + *
>>> + * This file is derived from qcow2-snapshot.c, original copyright:
>>> + * Copyright (c) 2004-2006 Fabrice Bellard
>>> + *
>>> + * Permission is hereby granted, free of charge, to any person
>>> obtaining a copy
>>> + * of this software and associated documentation files (the
>>> "Software"), to deal
>>> + * in the Software without restriction, including without limitation
>>> the rights
>>> + * to use, copy, modify, merge, publish, distribute, sublicense,
>>> and/or sell
>>> + * copies of the Software, and to permit persons to whom the
>>> Software is
>>> + * furnished to do so, subject to the following conditions:
>>> + *
>>> + * The above copyright notice and this permission notice shall be
>>> included in
>>> + * all copies or substantial portions of the Software.
>>> + *
>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
>>> EXPRESS OR
>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
>>> MERCHANTABILITY,
>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
>>> SHALL
>>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
>>> OR OTHER
>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
>>> ARISING FROM,
>>> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
>>> DEALINGS IN
>>> + * THE SOFTWARE.
>>> + */
>>> +
>>> +/* NOTICE: BME here means Bitmaps Extension and used as a namespace for
>>> + * _internal_ constants. Please do not use this _internal_
>>> abbreviation for
>>> + * other needs and/or outside of this file. */
>>> +
>>> +/* Bitmap directory entry constraints */
>>> +#define BME_MAX_TABLE_SIZE 0x8000000
>>> +#define BME_MAX_PHYS_SIZE 0x20000000 /* 512 mb */
>> I suppose BME_MAX_TABLE_SIZE (8M) is greater than BME_MAX_PHYS_SIZE (512
>> MB) divided by the cluster size (>= 512; 512 MB / cluster_size <= 1 MB)
>> because fully zero or one clusters do not require any physical space?
>>
>> Makes some sense, but I can see that this might make give some trouble
>> when trying to serialize overly large bitmaps. But I guess that comes
>> later in this series, so I'll wait for that point.
>>
>> Another thing is that 512 MB is rather big. It gets worse: The bitmap
>> may only require 512 MB on disk, but with a maximum table size of 8 MB,
>> it can require up to 8M * cluster_size in memory (with just 64 MB of
>> disk space!) by using the "read as all zeroes" or "read as all ones"
>> flags. With the default cluster size of 64 kB, this would be 512 GB in
>> RAM. That sounds bad to me.
>>
>> Well, it is probably fine as long as the bitmap is not auto-loaded...
>> But we do have a flag for exactly that. So it seems to me that a
>> manipulated image can easily consume huge amounts of RAM on the host.
>>
>> So I think we also need some sane limitation on the in-RAM size of a
>> bitmap (which is BME_MAX_TABLE_SIZE * cluster_size, as far as I
>> understand). The question of course is, what is sane? For a server
>> system with no image manipulation possible from the outside, 1 GB may be
>> completely fine. But imagine you download some qcow2 image to your
>> laptop. Then, 1 GB may not be fine, actually.
>>
>> Maybe it would make sense to use a runtime-adjustable limit here?
> 
> Actualy BME_MAX_PHYS_SIZE is this limit:
> in check_constraints we have
> 
> uint64_t phys_bitmap_bytes =
>         (uint64_t)h->bitmap_table_size * s->cluster_size;
> 
> ...
> 
> (phys_bitmap_bytes > BME_MAX_PHYS_SIZE) ||

OK, so BME_MAX_PHYS_SIZE is actually supposed to be the limit of the
size of the bitmaps in RAM? And I suppose it is going to be calculated
differently in the future once qemu has sparse bitmap support?

My fault, then, I thought BME_MAX_PHYS_SIZE was supposed to be the limit
of the size on disk. OK, makes sense then, but the question whether a
runtime-adjustable limit would make sense still remains. OTOH, this is
something that can always be added later on.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 06/22] qcow2: add dirty bitmaps extension

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1881 bytes --]

On 11.10.2016 14:09, Vladimir Sementsov-Ogievskiy wrote:
> On 01.10.2016 17:46, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> Add dirty bitmap extension as specified in docs/specs/qcow2.txt.
>>> For now, just mirror extension header into Qcow2 state and check
>>> constraints.
>>>
>>> For now, disable image resize if it has bitmaps. It will be fixed later.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   block/qcow2.c | 83
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>   block/qcow2.h |  4 +++
>>>   2 files changed, 87 insertions(+)
>>>
>>> diff --git a/block/qcow2.c b/block/qcow2.c
>>> index c079aa8..08c4ef9 100644
>>> --- a/block/qcow2.c
>>> +++ b/block/qcow2.c
>> [...]
>>
>>> @@ -162,6 +164,62 @@ static int
>>> qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
>>>               }
>>>               break;
>>>   +        case QCOW2_EXT_MAGIC_DIRTY_BITMAPS:
>>> +            ret = bdrv_pread(bs->file, offset, &bitmaps_ext, ext.len);
>> Overflows with ext.len > sizeof(bitmaps_ext).
>>
>> (ext.len < sizeof(bitmaps_ext) is also wrong, but less dramatically so.)
>>
>>> +            if (ret < 0) {
>>> +                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
>>> +                                 "Could not read ext header");
>>> +                return ret;
>>> +            }
>>> +
>>> +            if (bitmaps_ext.reserved32 != 0) {
>>> +                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
>>> +                                 "Reserved field is not zero.");
>> Please drop the full stop at the end.
> 
> what do you mean? goto to fail: here? or not stop at all, just print error?

The "." at the end of the message. :-)

(https://en.wikipedia.org/wiki/Full_stop)

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 09/22] block: introduce persistent dirty bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 3484 bytes --]

On 11.10.2016 15:11, Vladimir Sementsov-Ogievskiy wrote:
> On 07.10.2016 20:54, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
>>> on bdrv_close, using format driver. Format driver should maintain bitmap
>>> storing.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   block.c                      | 30 ++++++++++++++++++++++++++++++
>>>   block/dirty-bitmap.c         | 27 +++++++++++++++++++++++++++
>>>   block/qcow2-bitmap.c         |  1 +
>>>   include/block/block.h        |  2 ++
>>>   include/block/block_int.h    |  2 ++
>>>   include/block/dirty-bitmap.h |  6 ++++++
>>>   6 files changed, 68 insertions(+)
>>>
>>> diff --git a/block.c b/block.c
>>> index 804e3d4..1cde03a 100644
>>> --- a/block.c
>>> +++ b/block.c
>>> @@ -2196,6 +2196,7 @@ void bdrv_reopen_abort(BDRVReopenState
>>> *reopen_state)
>>>   static void bdrv_close(BlockDriverState *bs)
>>>   {
>>>       BdrvAioNotifier *ban, *ban_next;
>>> +    Error *local_err = NULL;
>>>         assert(!bs->job);
>>>       assert(!bs->refcnt);
>>> @@ -2204,6 +2205,10 @@ static void bdrv_close(BlockDriverState *bs)
>>>       bdrv_flush(bs);
>>>       bdrv_drain(bs); /* in case flush left pending I/O */
>>>   +    bdrv_store_persistent_bitmaps(bs, &local_err);
>>> +    if (local_err != NULL) {
>>> +        error_report_err(local_err);
>>> +    }
>> That seems pretty wrong to me. If the persistent bitmaps cannot be
>> stored, the node should not be closed to avoid loss of data.
>>
>>>       bdrv_release_named_dirty_bitmaps(bs);
>> Especially since the next function will just drop all the dirty bitmaps.
>>
>> I see the issue that bdrv_close() is only called by bdrv_delete() which
>> in turn is only called by bdrv_unref(); and how are you supposed to
>> react to bdrv_unref() failing?
>>
>> So I'm not sure how this issue should be addressed, but this is most
>> certainly not ideal. You should not just drop supposedly persistent
>> dirty bitmaps if they cannot be saved.
>>
>> We really should to have some way to keep the bitmap around if it cannot
>> be saved, but I don't know how to do that either.
>>
>> In any case, we should make sure that the node supports saving
>> persistent dirty bitmaps, because having persistent dirty bitmaps at a
>> node that does not support them is something we can and must prevent
>> beforehand.
>>
>> But I don't know how to handle failure if writing the dirty bitmap
>> fails. I guess one could argue that it's the same as bdrv_flush()
>> failing and thus can be handled in the same way, i.e. ignore it. I'm not
>> happy with that, but I'd accept it if there's no other way.
> 
> For now, the only usage of these bitmaps is incremental backup and
> bitmaps are not critical data. If we lost them we will just do full
> backup. If there will be some critical persistent bdrv dirty bitmaps in
> future, we can introduce a callback BdrvDirtyBitmap.store_failed for
> them, which will somehow handle that case.. Detach bitmap from bs and
> save it in memory, add qmp commands to raw-dump them, etc.. I

Yes, fine with me. Still, we should make an effort to detect the case
that some block driver will not be able to store a certain persistent
bitmap attached to one of its nodes as early as possible, ideally
already when the bitmap is created.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 09/22] block: introduce persistent dirty bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 2634 bytes --]

On 12.10.2016 14:30, Vladimir Sementsov-Ogievskiy wrote:
> On 12.10.2016 14:38, Vladimir Sementsov-Ogievskiy wrote:
>> On 07.10.2016 22:28, Max Reitz wrote:
>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>> New field BdrvDirtyBitmap.persistent means, that bitmap should be saved
>>>> on bdrv_close, using format driver. Format driver should maintain
>>>> bitmap
>>>> storing.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>>   block.c                      | 30 ++++++++++++++++++++++++++++++
>>>>   block/dirty-bitmap.c         | 27 +++++++++++++++++++++++++++
>>>>   block/qcow2-bitmap.c         |  1 +
>>>>   include/block/block.h        |  2 ++
>>>>   include/block/block_int.h    |  2 ++
>>>>   include/block/dirty-bitmap.h |  6 ++++++
>>>>   6 files changed, 68 insertions(+)
>>> [...]
>>>
>>>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>>>> index 623e1d1..0314581 100644
>>>> --- a/block/dirty-bitmap.c
>>>> +++ b/block/dirty-bitmap.c
>>> [...]
>>>
>>>> @@ -555,3 +559,26 @@ bool bdrv_dirty_bitmap_get_autoload(const
>>>> BdrvDirtyBitmap *bitmap)
>>>>   {
>>>>       return bitmap->autoload;
>>>>   }
>>>> +
>>>> +void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap,
>>>> +                                                bool persistent)
>>>> +{
>>>> +    bitmap->persistent = persistent;
>>> After some thinking, I think this function should be more complex: It
>>> should check whether the node the bitmap is attached to actually can
>>> handle persistent bitmaps and whether it would actually support storing
>>> *this* bitmap.
>>>
>>> For instance, a qcow2 node would not support writing overly large
>>> bitmaps (limited by BME_MAX_TABLE_SIZE and BME_MAX_PHYS_SIZE) or bitmaps
>>> with overly large granularities (BME_MAX_GRANULARITY_BITS) or bitmaps
>>> whose name is already occupied by some bitmap that is already stored in
>>> the file but has not been loaded.
>>>
>>> Checking this here will trivially prevent users from creating such
>>> bitmaps and will also preempt detection of such failures during
>>> bdrv_close() when they cannot be handled gracefully.
>>>
>>> Max
>>
>> Good point, but I can't do it exactly as you say, because I call this
>> function from qcow2_read_bitmaps, for just created bitmap and it
>> should not be checked and of course it's name is occupied..
> 
> So, I'll add an additional checking function, to call it from
> qmp_block_dirty_bitmap_add, if persistent parameter is set to true.

That would work just as well, yes. Thanks!

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 06/22] qcow2: add dirty bitmaps extension

[Vladimir Sementsov-Ogievskiy]

On 12.10.2016 21:21, Max Reitz wrote:
> On 11.10.2016 14:09, Vladimir Sementsov-Ogievskiy wrote:
>> On 01.10.2016 17:46, Max Reitz wrote:
>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>> Add dirty bitmap extension as specified in docs/specs/qcow2.txt.
>>>> For now, just mirror extension header into Qcow2 state and check
>>>> constraints.
>>>>
>>>> For now, disable image resize if it has bitmaps. It will be fixed later.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>>    block/qcow2.c | 83
>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>    block/qcow2.h |  4 +++
>>>>    2 files changed, 87 insertions(+)
>>>>
>>>> diff --git a/block/qcow2.c b/block/qcow2.c
>>>> index c079aa8..08c4ef9 100644
>>>> --- a/block/qcow2.c
>>>> +++ b/block/qcow2.c
>>> [...]
>>>
>>>> @@ -162,6 +164,62 @@ static int
>>>> qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
>>>>                }
>>>>                break;
>>>>    +        case QCOW2_EXT_MAGIC_DIRTY_BITMAPS:
>>>> +            ret = bdrv_pread(bs->file, offset, &bitmaps_ext, ext.len);
>>> Overflows with ext.len > sizeof(bitmaps_ext).
>>>
>>> (ext.len < sizeof(bitmaps_ext) is also wrong, but less dramatically so.)
>>>
>>>> +            if (ret < 0) {
>>>> +                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
>>>> +                                 "Could not read ext header");
>>>> +                return ret;
>>>> +            }
>>>> +
>>>> +            if (bitmaps_ext.reserved32 != 0) {
>>>> +                error_setg_errno(errp, -ret, "ERROR: bitmaps_ext: "
>>>> +                                 "Reserved field is not zero.");
>>> Please drop the full stop at the end.
>> what do you mean? goto to fail: here? or not stop at all, just print error?
> The "." at the end of the message. :-)
>
> (https://en.wikipedia.org/wiki/Full_stop)
>
> Max
>
aha, cool. didn't know this. )

-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Vladimir Sementsov-Ogievskiy]

On 07.10.2016 22:24, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Realize block bitmap stroing interface, to allow qcow2 images store

[snip]

>> +        uint64_t end = MIN(bm_size, sector + dsc);
>> +        uint64_t write_size =
>> +            bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sector);
>> +
>> +        int64_t off = qcow2_alloc_clusters(bs, cl_size);
>> +        if (off < 0) {
>> +            ret = off;
>> +            goto finish;
>> +        }
>> +        bitmap_table[cluster] = off;
>> +
>> +        bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end);
> s/end/end - sector/?

o_0 terrible mistake, thank you.

>
>> +        if (write_size < cl_size) {
>> +            memset(buf + write_size, 0, cl_size - write_size);
>> +        }
>> +
> I guess there should be a metadata overlap check here.

What is the general rule of checking it? Should I check it before all my 
extension related writes?


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v7 00/22] qcow2: persistent dirty bitmaps

[John Snow]

On 10/01/2016 09:37 AM, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> v7:
>> https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=refs%2Ftags%2Fqcow2-bitmap-v7
>> based on block-next (https://github.com/XanClic/qemu/commits/block-next)
>
> It should be noted that (at least my) block-next is only valid during
> freeze, after that it becomes stale. I assume the patches you require
> from my block-next branch are the ones from the "Dirty bitmap changes
> for migration/persistence work" series, which I had to drop from my pull
> requests, however, due to some issues on Big Endian machines.
>
> The only reason they are still in my block-next branch is, as I said,
> that that branch is just stale. I won't overwrite it for the time being,
> though, so this series can still be applied on top.
>
> Max
>

Lemme fix up the 32bit problems and resend that out right now.
(At least it's not lock correctness.)

--js

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Vladimir Sementsov-Ogievskiy]

On 01.10.2016 19:26, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are

...

> +        goto fail;
> +    }
> +
> +    /* loop is safe because next entry offset is calculated after conversion to
> Should it be s/safe/unsafe/?

loop is safe. _unsafe is related to absence of assert in a loop, as it 
loops through BE data. Bad wording, I'll change it somehow..

>
>> +     * cpu format */
>> +    for_each_bitmap_dir_entry_unsafe(e, dir, size) {
>> +        if ((uint8_t *)(e + 1) > dir_end) {
>> +            goto broken_dir;
>> +        }
>> +
>> +        bitmap_dir_entry_to_cpu(e);
>> +
>> +        if ((uint8_t *)next_dir_entry(e) > dir_end) {
>> +            goto broken_dir;
>> +        }
>> +
>> +        if (e->extra_data_size != 0) {
>> +            error_setg(errp, "Can't load bitmap '%.*s' from '%s':"
>> +                       "extra data not supported.", e->name_size,
> Full stop again.
>
> Also, I'm not quite sure why you give the device/node name here. You
> don't do that anywhere else and I think if we want to emit the
> information where something failed, it should be added at some previous
> point in the call chain.

For example, how? As I understand, I can emit it only by error_setg, so 
actually it would be better to add node and bitmap name to all 
error_setg in the code.. or create helper function for it.

>
>> +                       dir_entry_name_notcstr(e),
>> +                       bdrv_get_device_or_node_name(bs));
>> +            goto fail;
>> +        }
>> +    }

...

>> +        if (ret < 0) {
>> +            goto fail;
>> +        }
>> +    }
>> +    s->bitmap_directory_offset = new_offset;
>> +    s->bitmap_directory_size = new_size;
>> +    s->nb_bitmaps = new_nb_bitmaps;
>> +
>> +    ret = update_header_sync(bs);
>> +    if (ret < 0) {
>> +        goto fail;
>> +    }
>> +
>> +    if (old_size) {
>> +        qcow2_free_clusters(bs, old_offset, old_size, QCOW2_DISCARD_ALWAYS);
>> +    }
>> +
>> +    return 0;
>> +
>> +fail:
>> +    if (new_offset > 0) {
>> +        qcow2_free_clusters(bs, new_offset, new_size, QCOW2_DISCARD_ALWAYS);
>> +        s->bitmap_directory_offset = old_offset;
>> +        s->bitmap_directory_size = old_size;
>> +        s->nb_bitmaps = old_nb_bitmaps;
>> +        s->autoclear_features = old_autocl;
> Why are you restoring the autoclear features? From a quick glance I
> can't see any code path that changes this field here, and if there is
> one, it probably has a good reason to do so.

hmm.. it's an artefact from future). should be moved to later patch.



-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1201 bytes --]

On 13.10.2016 18:48, Vladimir Sementsov-Ogievskiy wrote:
> On 07.10.2016 22:24, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> Realize block bitmap stroing interface, to allow qcow2 images store
> 
> [snip]
> 
>>> +        uint64_t end = MIN(bm_size, sector + dsc);
>>> +        uint64_t write_size =
>>> +            bdrv_dirty_bitmap_serialization_size(bitmap, sector, end
>>> - sector);
>>> +
>>> +        int64_t off = qcow2_alloc_clusters(bs, cl_size);
>>> +        if (off < 0) {
>>> +            ret = off;
>>> +            goto finish;
>>> +        }
>>> +        bitmap_table[cluster] = off;
>>> +
>>> +        bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end);
>> s/end/end - sector/?
> 
> o_0 terrible mistake, thank you.
> 
>>
>>> +        if (write_size < cl_size) {
>>> +            memset(buf + write_size, 0, cl_size - write_size);
>>> +        }
>>> +
>> I guess there should be a metadata overlap check here.
> 
> What is the general rule of checking it? Should I check it before all my
> extension related writes?

The general rule is supposed to be "One check before every write to
bs->file".

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 2991 bytes --]

On 14.10.2016 20:44, Vladimir Sementsov-Ogievskiy wrote:
> On 01.10.2016 19:26, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are
> 
> ...
> 
>> +        goto fail;
>> +    }
>> +
>> +    /* loop is safe because next entry offset is calculated after
>> conversion to
>> Should it be s/safe/unsafe/?
> 
> loop is safe. _unsafe is related to absence of assert in a loop, as it
> loops through BE data. Bad wording, I'll change it somehow..

Yes, I know that the loop is safe in the sense of the word "safe", but I
meant that it's kind of confusing that the loop uses
"for_each_bitmap_dir_entry_unsafe" and thus is "unsafe", too.

Another idea would be to write "This is actually safe" instead of "loop
is safe".

>>> +     * cpu format */
>>> +    for_each_bitmap_dir_entry_unsafe(e, dir, size) {
>>> +        if ((uint8_t *)(e + 1) > dir_end) {
>>> +            goto broken_dir;
>>> +        }
>>> +
>>> +        bitmap_dir_entry_to_cpu(e);
>>> +
>>> +        if ((uint8_t *)next_dir_entry(e) > dir_end) {
>>> +            goto broken_dir;
>>> +        }
>>> +
>>> +        if (e->extra_data_size != 0) {
>>> +            error_setg(errp, "Can't load bitmap '%.*s' from '%s':"
>>> +                       "extra data not supported.", e->name_size,
>> Full stop again.
>>
>> Also, I'm not quite sure why you give the device/node name here. You
>> don't do that anywhere else and I think if we want to emit the
>> information where something failed, it should be added at some previous
>> point in the call chain.
> 
> For example, how? As I understand, I can emit it only by error_setg, so
> actually it would be better to add node and bitmap name to all
> error_setg in the code.. or create helper function for it.

error_prepend() would be the function.

This code will be invoked by any code that is opening an image. There
are of course a couple of places where that can be the case: For
external tools such as qemu-img, it's normally pretty clear which image
is meant (and it additionally uses error_reportf_err() to give you more
information).

For -drive, every error message will at least be prepended by the
corresponding -drive parameter, so that will help a bit.

blockdev-add, unfortunately, doesn't do anything like this. But the user
can of course choose to construct the BDS graph node by node and thus
always know which node an error originates from.

Anyway, if you want to add this information to every error message, you
should probably do so in bdrv_open_inherit(). The issue I'd take with
this is that most users probably won't set the node name themselves
(either they don't at all or it's some management tool that does), so
reporting the node name doesn't actually help them at all (and
management tools are not supposed to parse error messages, so it won't
help in that case either).

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Vladimir Sementsov-Ogievskiy]

On 15.10.2016 20:03, Max Reitz wrote:
> On 14.10.2016 20:44, Vladimir Sementsov-Ogievskiy wrote:
>> On 01.10.2016 19:26, Max Reitz wrote:
>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are
>> ...
>>
>>> +        goto fail;
>>> +    }
>>> +
>>> +    /* loop is safe because next entry offset is calculated after
>>> conversion to
>>> Should it be s/safe/unsafe/?
>> loop is safe. _unsafe is related to absence of assert in a loop, as it
>> loops through BE data. Bad wording, I'll change it somehow..
> Yes, I know that the loop is safe in the sense of the word "safe", but I
> meant that it's kind of confusing that the loop uses
> "for_each_bitmap_dir_entry_unsafe" and thus is "unsafe", too.
>
> Another idea would be to write "This is actually safe" instead of "loop
> is safe".

Finally I've decided to introduce normal list of normal structures like 
in snapshots..

>
>>>> +     * cpu format */
>>>> +    for_each_bitmap_dir_entry_unsafe(e, dir, size) {
>>>> +        if ((uint8_t *)(e + 1) > dir_end) {
>>>> +            goto broken_dir;
>>>> +        }
>>>> +
>>>> +        bitmap_dir_entry_to_cpu(e);
>>>> +
>>>> +        if ((uint8_t *)next_dir_entry(e) > dir_end) {
>>>> +            goto broken_dir;
>>>> +        }
>>>> +
>>>> +        if (e->extra_data_size != 0) {
>>>> +            error_setg(errp, "Can't load bitmap '%.*s' from '%s':"
>>>> +                       "extra data not supported.", e->name_size,
>>> Full stop again.
>>>
>>> Also, I'm not quite sure why you give the device/node name here. You
>>> don't do that anywhere else and I think if we want to emit the
>>> information where something failed, it should be added at some previous
>>> point in the call chain.
>> For example, how? As I understand, I can emit it only by error_setg, so
>> actually it would be better to add node and bitmap name to all
>> error_setg in the code.. or create helper function for it.
> error_prepend() would be the function.
>
> This code will be invoked by any code that is opening an image. There
> are of course a couple of places where that can be the case: For
> external tools such as qemu-img, it's normally pretty clear which image
> is meant (and it additionally uses error_reportf_err() to give you more
> information).
>
> For -drive, every error message will at least be prepended by the
> corresponding -drive parameter, so that will help a bit.
>
> blockdev-add, unfortunately, doesn't do anything like this. But the user
> can of course choose to construct the BDS graph node by node and thus
> always know which node an error originates from.
>
> Anyway, if you want to add this information to every error message, you
> should probably do so in bdrv_open_inherit(). The issue I'd take with
> this is that most users probably won't set the node name themselves
> (either they don't at all or it's some management tool that does), so
> reporting the node name doesn't actually help them at all (and
> management tools are not supposed to parse error messages, so it won't
> help in that case either).
>
> Max
>

Thanks for explanations, and for the whole review, it's great! Sorry for 
my laziness and for spelling(
O! I've discovered vim spell, so one problem less, I hope.

-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Vladimir Sementsov-Ogievskiy]

On 07.10.2016 22:24, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Realize block bitmap stroing interface, to allow qcow2 images store
>> persistent bitmaps.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block/qcow2-bitmap.c | 241 +++++++++++++++++++++++++++++++++++++++++++++++++++

[...]

>>
>> +        ret = bdrv_pwrite(bs->file, off, buf, cl_size);
>> +        if (ret < 0) {
>> +            goto finish;
>> +        }
>> +
>> +        if (end >= bm_size) {
>> +            break;
>> +        }
>> +
>> +        bdrv_set_dirty_iter(dbi, end);
>> +    }
>> +    ret = 0; /* writes */
> What is that comment supposed to mean?
>
>

Now I think I can drop this assignment, as bdrv_aligned_preadv have 
'return ret < 0 ? ret : 0;' in the end...  Am I right? Can bdrv_pwrite 
and friends return positive value on success?

-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Vladimir Sementsov-Ogievskiy]

[-- Attachment #1: Type: text/plain, Size: 10445 bytes --]

On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
> Realize block bitmap stroing interface, to allow qcow2 images store
> persistent bitmaps.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/qcow2-bitmap.c | 241 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  block/qcow2.c        |   2 +
>  block/qcow2.h        |   2 +
>  3 files changed, 245 insertions(+)
>
> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
> index 81520cd..a5be25a 100644
> --- a/block/qcow2-bitmap.c
> +++ b/block/qcow2-bitmap.c
> @@ -27,6 +27,7 @@
>
>  #include "qemu/osdep.h"
>  #include "qapi/error.h"
> +#include "qemu/cutils.h"
>
>  #include "block/block_int.h"
>  #include "block/qcow2.h"
> @@ -96,6 +97,15 @@ static inline void bitmap_table_to_cpu(uint64_t *bitmap_table, size_t size)
>      }
>  }
>
> +static inline void bitmap_table_to_be(uint64_t *bitmap_table, size_t size)
> +{
> +    size_t i;
> +
> +    for (i = 0; i < size; ++i) {
> +        cpu_to_be64s(&bitmap_table[i]);
> +    }
> +}
> +
>  static inline int calc_dir_entry_size(size_t name_size, size_t extra_data_size)
>  {
>      return align_offset(sizeof(Qcow2BitmapDirEntry) +
> @@ -564,3 +574,234 @@ out:
>
>      return ret;
>  }
> +
> +/* store_bitmap_data()
> + * Store bitmap to image, filling bitamp table accordingly.


> + */
> +static int store_bitmap_data(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
> +                             uint64_t *bitmap_table, uint32_t bitmap_table_size)
> +{
> +    int ret;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint64_t sector, dsc;
> +    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
> +    int cl_size = s->cluster_size;



> +    uint8_t *buf = NULL;
> +    uint32_t tb_size =
> +            size_to_clusters(s,
> +                bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));


> +
> +    BdrvDirtyBitmapIter *dbi;
> +
> +    if (tb_size != bitmap_table_size) {
> +        return -EINVAL;
> +    }
> +
> +    memset(bitmap_table, 0, bitmap_table_size * sizeof(bitmap_table[0]));


> +
> +    dbi = bdrv_dirty_iter_new(bitmap, 0);
> +    buf = g_malloc(cl_size);
> +    dsc = dirty_sectors_in_cluster(s, bitmap);
> +
> +    while ((sector = bdrv_dirty_iter_next(dbi)) != -1) {


> +        uint64_t cluster = sector / dsc;
> +        sector = cluster * dsc;



> +        uint64_t end = MIN(bm_size, sector + dsc);
> +        uint64_t write_size =
> +            bdrv_dirty_bitmap_serialization_size(bitmap, sector, end - sector);
> +
> +        int64_t off = qcow2_alloc_clusters(bs, cl_size);
> +        if (off < 0) {
> +            ret = off;
> +            goto finish;
> +        }
> +        bitmap_table[cluster] = off;
> +
> +        bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end);



> +        if (write_size < cl_size) {
> +            memset(buf + write_size, 0, cl_size - write_size);
> +        }
> +


> +        ret = bdrv_pwrite(bs->file, off, buf, cl_size);
> +        if (ret < 0) {
> +            goto finish;
> +        }
> +
> +        if (end >= bm_size) {
> +            break;
> +        }
> +
> +        bdrv_set_dirty_iter(dbi, end);
> +    }
> +    ret = 0; /* writes */

What is that comment supposed to mean?

> +
> +finish:
> +    if (ret < 0) {
> +        clear_bitmap_table(bs, bitmap_table, bitmap_table_size);
> +    }
> +    g_free(buf);
> +    bdrv_dirty_iter_free(dbi);
> +
> +    return ret;

In case you decide to keep BME_MAX_PHYS_SIZE, this function should check
somewhere that the physical size of the bitmap does not exceed that value.

> +}
> +
> +/* store_bitmap()
> + * Store bitmap to qcow2 and set bitmap_table. bitmap_table itself is not
> + * stored to qcow2.

First of all, there is no parameter called "bitmap_table", and second,
yes, the bitmap table is written to the qcow2 file.

> + */
> +static int store_bitmap(BlockDriverState *bs,
> +                        BdrvDirtyBitmap *bitmap,
> +                        Qcow2BitmapDirEntry *entry)
> +{
> +    int ret;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
> +    const char *bm_name = bdrv_dirty_bitmap_name(bitmap);
> +
> +    uint64_t *tb;
> +    int64_t tb_offset;
> +    uint32_t tb_size =
> +            size_to_clusters(s,
> +                bdrv_dirty_bitmap_serialization_size(bitmap, 0, bm_size));

As above, this variable should be of type uint64_t.

Also, you have to check that it does not exceed BME_MAX_TABLE_SIZE.

> +
> +    tb = g_try_new(uint64_t, tb_size);
> +    if (tb == NULL) {
> +        return -ENOMEM;
> +    }
> +
> +    ret = store_bitmap_data(bs, bitmap, tb, tb_size);
> +    if (ret < 0) {
> +        g_free(tb);
> +        return ret;
> +    }
> +
> +    tb_offset = qcow2_alloc_clusters(bs, tb_size * sizeof(tb[0]));

If you don't limit tb_size, then this multiplication can overflow on 32
bit machines.

> +    if (tb_offset < 0) {
> +        ret = tb_offset;
> +        goto fail;
> +    }
> +

There should be a metadata overlap check here.

> +    bitmap_table_to_be(tb, tb_size);
> +    ret = bdrv_pwrite(bs->file, tb_offset, tb, tb_size * sizeof(tb[0]));
> +    if (ret < 0) {
> +        goto fail;
> +    }
> +
> +    g_free(tb);
> +
> +    entry->bitmap_table_offset = tb_offset;
> +    entry->bitmap_table_size = tb_size;
> +    entry->flags = bdrv_dirty_bitmap_granularity(bitmap) ? BME_FLAG_AUTO : 0;

s/granularity/get_autoload/

> +    entry->type = BT_DIRTY_TRACKING_BITMAP;
> +    entry->granularity_bits = ctz32(bdrv_dirty_bitmap_granularity(bitmap));

You should probably check somewhere that the resulting value for
entry->granularity_bits is in the BME_{MIN,MAX}_GRANULARITY_BITS range.

> +    entry->name_size = strlen(bm_name);

And that this length does not exceed BME_MAX_NAME_SIZE.

> +    entry->extra_data_size = 0;
> +    memcpy(entry + 1, bm_name, entry->name_size);
> +
> +    return 0;
> +
> +fail:
> +    clear_bitmap_table(bs, tb, tb_size);
> +
> +    if (tb_offset > 0) {
> +        qcow2_free_clusters(bs, tb_offset, tb_size, QCOW2_DISCARD_ALWAYS);

As before, I'd vote for QCOW2_DISCARD_OTHER.

> +    }
> +
> +    g_free(tb);
> +
> +    return ret;
> +}
> +
> +static Qcow2BitmapDirEntry *find_bitmap_by_name(uint8_t *bitmap_directory,
> +                                                size_t size, const char *name)
> +{
> +    Qcow2BitmapDirEntry *e;
> +
> +    for_each_bitmap_dir_entry(e, bitmap_directory, size) {
> +        if (strncmp((char *)(e + 1), name, e->name_size) == 0) {
> +            return e;
> +        }
> +    }
> +
> +    return NULL;
> +}
> +
> +void qcow2_store_persistent_bitmaps(BlockDriverState *bs, Error **errp)
> +{
> +    BdrvDirtyBitmap *bm;
> +    BDRVQcow2State *s = bs->opaque;
> +    uint32_t new_nb_bitmaps = s->nb_bitmaps;
> +    uint64_t new_dir_size = s->bitmap_directory_size;
> +    uint8_t *dir = NULL, *new_dir = NULL;
> +    int ret;
> +    Qcow2BitmapDirEntry *new_pos;
> +
> +    if (s->nb_bitmaps > 0) {
> +        dir = directory_read(bs, s->bitmap_directory_offset,
> +                             s->bitmap_directory_size, errp);
> +        if (dir == NULL) {
> +            goto out;
> +        }
> +    }
> +
> +    for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm != NULL;
> +            bm = bdrv_dirty_bitmap_next(bs, bm)) {
> +        const char *name = bdrv_dirty_bitmap_name(bm);
> +
> +        if (!bdrv_dirty_bitmap_get_persistance(bm)) {
> +            continue;
> +        }
> +
> +        if (s->nb_bitmaps > 0 &&
> +                find_bitmap_by_name(dir, s->bitmap_directory_size, name)) {
> +            error_setg(errp,
> +                       "Can't store bitmap '%s' to '%s', as it already exists",
> +                       name, bdrv_get_device_or_node_name(bs));
> +            goto out;
> +        }
> +
> +        new_nb_bitmaps++;
> +        new_dir_size += calc_dir_entry_size(strlen(name), 0);
> +    }
> +
> +    if (s->nb_bitmaps == new_nb_bitmaps) {
> +        /* No new bitmaps - nothing to do */
> +        goto out;
> +    }
> +
> +    new_dir = g_try_malloc0(new_dir_size);
> +    if (new_dir == NULL) {
> +        error_setg(errp, "Can't allocate space for bitmap directory.");
> +        goto out;
> +    }
> +
> +    memcpy(new_dir, dir, s->bitmap_directory_size);
> +    new_pos = (Qcow2BitmapDirEntry *)(new_dir + s->bitmap_directory_size);
> +
> +    for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm != NULL;
> +            bm = bdrv_dirty_bitmap_next(bs, bm)) {
> +        if (!bdrv_dirty_bitmap_get_persistance(bm)) {
> +            continue;
> +        }
> +
> +        ret = store_bitmap(bs, bm, new_pos);
> +        if (ret < 0) {
> +            error_setg_errno(errp, -ret, "Can't store bitmap '%s' to '%s'",
> +                             bdrv_dirty_bitmap_name(bm),
> +                             bdrv_get_device_or_node_name(bs));
> +            goto out;
> +        }
> +        new_pos = next_dir_entry(new_pos);
> +    }
> +
> +    ret = directory_update(bs, new_dir, new_dir_size, new_nb_bitmaps);
> +    if (ret < 0) {
> +        error_setg_errno(errp, -ret, "Can't update bitmap directory in '%s'",
> +                         bdrv_get_device_or_node_name(bs));
> +        goto out;
> +    }
> +
> +out:
> +    g_free(new_dir);
> +    g_free(dir);

This error path leaks all the bitmaps that have been written
successfully (if any). I guess this is more or less fine if
directory_update() failed (because you can't really tell the state of
the image header after directory_update(), so better be safe) but it's
not so fine if just some store_bitmap() failed.

Max

> +}
> diff --git a/block/qcow2.c b/block/qcow2.c
> index 02ec224..8238205 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c
> @@ -3493,6 +3493,8 @@ BlockDriver bdrv_qcow2 = {
>
>      .bdrv_detach_aio_context  = qcow2_detach_aio_context,
>      .bdrv_attach_aio_context  = qcow2_attach_aio_context,
> +
> +    .bdrv_store_persistent_bitmaps = qcow2_store_persistent_bitmaps,
>  };
>
>  static void bdrv_qcow2_init(void)
> diff --git a/block/qcow2.h b/block/qcow2.h
> index 482a29f..dfcf4c6 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -627,4 +627,6 @@ int qcow2_cache_get_empty(BlockDriverState *bs, Qcow2Cache *c, uint64_t offset,
>      void **table);
>  void qcow2_cache_put(BlockDriverState *bs, Qcow2Cache *c, void **table);
>
> +void qcow2_store_persistent_bitmaps(BlockDriverState *bs, Error **errp);
> +
>  #endif
>




[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 481 bytes --]

[Qemu-devel] DROP THIS Re: [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Vladimir Sementsov-Ogievskiy]

Sorry, this was an accidental reply.

On 17.10.2016 20:57, Vladimir Sementsov-Ogievskiy wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Realize block bitmap stroing interface, to allow qcow2 images store
>> persistent bitmaps.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>  block/qcow2-bitmap.c | 241 
>> +++++++++++++++++++++++++++++++++++++++++++++++++++
>>  block/qcow2.c        |   2 +
>>  block/qcow2.h        |   2 +
>>  3 files changed, 245 insertions(+)
>>
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> index 81520cd..a5be25a 100644
>> --- a/block/qcow2-bitmap.c
>> +++ b/block/qcow2-bitmap.c
>> @@ -27,6 +27,7 @@
>>
>>  #include "qemu/osdep.h"
>>  #include "qapi/error.h"
>> +#include "qemu/cutils.h"
>>
>>  #include "block/block_int.h"
>>  #include "block/qcow2.h"
>> @@ -96,6 +97,15 @@ static inline void bitmap_table_to_cpu(uint64_t 
>> *bitmap_table, size_t size)
>>      }
>>  }
>>
>> +static inline void bitmap_table_to_be(uint64_t *bitmap_table, size_t 
>> size)
>> +{
>> +    size_t i;
>> +
>> +    for (i = 0; i < size; ++i) {
>> +        cpu_to_be64s(&bitmap_table[i]);
>> +    }
>> +}
>> +
>>  static inline int calc_dir_entry_size(size_t name_size, size_t 
>> extra_data_size)
>>  {
>>      return align_offset(sizeof(Qcow2BitmapDirEntry) +
>> @@ -564,3 +574,234 @@ out:
>>
>>      return ret;
>>  }
>> +
>> +/* store_bitmap_data()
>> + * Store bitmap to image, filling bitamp table accordingly.
>
>
>> + */
>> +static int store_bitmap_data(BlockDriverState *bs, BdrvDirtyBitmap 
>> *bitmap,
>> +                             uint64_t *bitmap_table, uint32_t 
>> bitmap_table_size)
>> +{
>> +    int ret;
>> +    BDRVQcow2State *s = bs->opaque;
>> +    uint64_t sector, dsc;
>> +    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
>> +    int cl_size = s->cluster_size;
>
>
>
>> +    uint8_t *buf = NULL;
>> +    uint32_t tb_size =
>> +            size_to_clusters(s,
>> +                bdrv_dirty_bitmap_serialization_size(bitmap, 0, 
>> bm_size));
>
>
>> +
>> +    BdrvDirtyBitmapIter *dbi;
>> +
>> +    if (tb_size != bitmap_table_size) {
>> +        return -EINVAL;
>> +    }
>> +
>> +    memset(bitmap_table, 0, bitmap_table_size * 
>> sizeof(bitmap_table[0]));
>
>
>> +
>> +    dbi = bdrv_dirty_iter_new(bitmap, 0);
>> +    buf = g_malloc(cl_size);
>> +    dsc = dirty_sectors_in_cluster(s, bitmap);
>> +
>> +    while ((sector = bdrv_dirty_iter_next(dbi)) != -1) {
>
>
>> +        uint64_t cluster = sector / dsc;
>> +        sector = cluster * dsc;
>
>
>
>> +        uint64_t end = MIN(bm_size, sector + dsc);
>> +        uint64_t write_size =
>> +            bdrv_dirty_bitmap_serialization_size(bitmap, sector, end 
>> - sector);
>> +
>> +        int64_t off = qcow2_alloc_clusters(bs, cl_size);
>> +        if (off < 0) {
>> +            ret = off;
>> +            goto finish;
>> +        }
>> +        bitmap_table[cluster] = off;
>> +
>> +        bdrv_dirty_bitmap_serialize_part(bitmap, buf, sector, end);
>
>
>
>> +        if (write_size < cl_size) {
>> +            memset(buf + write_size, 0, cl_size - write_size);
>> +        }
>> +
>
>
>> +        ret = bdrv_pwrite(bs->file, off, buf, cl_size);
>> +        if (ret < 0) {
>> +            goto finish;
>> +        }
>> +
>> +        if (end >= bm_size) {
>> +            break;
>> +        }
>> +
>> +        bdrv_set_dirty_iter(dbi, end);
>> +    }
>> +    ret = 0; /* writes */
>
> What is that comment supposed to mean?
>
>> +
>> +finish:
>> +    if (ret < 0) {
>> +        clear_bitmap_table(bs, bitmap_table, bitmap_table_size);
>> +    }
>> +    g_free(buf);
>> +    bdrv_dirty_iter_free(dbi);
>> +
>> +    return ret;
>
> In case you decide to keep BME_MAX_PHYS_SIZE, this function should check
> somewhere that the physical size of the bitmap does not exceed that 
> value.
>
>> +}
>> +
>> +/* store_bitmap()
>> + * Store bitmap to qcow2 and set bitmap_table. bitmap_table itself 
>> is not
>> + * stored to qcow2.
>
> First of all, there is no parameter called "bitmap_table", and second,
> yes, the bitmap table is written to the qcow2 file.
>
>> + */
>> +static int store_bitmap(BlockDriverState *bs,
>> +                        BdrvDirtyBitmap *bitmap,
>> +                        Qcow2BitmapDirEntry *entry)
>> +{
>> +    int ret;
>> +    BDRVQcow2State *s = bs->opaque;
>> +    uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
>> +    const char *bm_name = bdrv_dirty_bitmap_name(bitmap);
>> +
>> +    uint64_t *tb;
>> +    int64_t tb_offset;
>> +    uint32_t tb_size =
>> +            size_to_clusters(s,
>> +                bdrv_dirty_bitmap_serialization_size(bitmap, 0, 
>> bm_size));
>
> As above, this variable should be of type uint64_t.
>
> Also, you have to check that it does not exceed BME_MAX_TABLE_SIZE.
>
>> +
>> +    tb = g_try_new(uint64_t, tb_size);
>> +    if (tb == NULL) {
>> +        return -ENOMEM;
>> +    }
>> +
>> +    ret = store_bitmap_data(bs, bitmap, tb, tb_size);
>> +    if (ret < 0) {
>> +        g_free(tb);
>> +        return ret;
>> +    }
>> +
>> +    tb_offset = qcow2_alloc_clusters(bs, tb_size * sizeof(tb[0]));
>
> If you don't limit tb_size, then this multiplication can overflow on 32
> bit machines.
>
>> +    if (tb_offset < 0) {
>> +        ret = tb_offset;
>> +        goto fail;
>> +    }
>> +
>
> There should be a metadata overlap check here.
>
>> +    bitmap_table_to_be(tb, tb_size);
>> +    ret = bdrv_pwrite(bs->file, tb_offset, tb, tb_size * 
>> sizeof(tb[0]));
>> +    if (ret < 0) {
>> +        goto fail;
>> +    }
>> +
>> +    g_free(tb);
>> +
>> +    entry->bitmap_table_offset = tb_offset;
>> +    entry->bitmap_table_size = tb_size;
>> +    entry->flags = bdrv_dirty_bitmap_granularity(bitmap) ? 
>> BME_FLAG_AUTO : 0;
>
> s/granularity/get_autoload/
>
>> +    entry->type = BT_DIRTY_TRACKING_BITMAP;
>> +    entry->granularity_bits = 
>> ctz32(bdrv_dirty_bitmap_granularity(bitmap));
>
> You should probably check somewhere that the resulting value for
> entry->granularity_bits is in the BME_{MIN,MAX}_GRANULARITY_BITS range.
>
>> +    entry->name_size = strlen(bm_name);
>
> And that this length does not exceed BME_MAX_NAME_SIZE.
>
>> +    entry->extra_data_size = 0;
>> +    memcpy(entry + 1, bm_name, entry->name_size);
>> +
>> +    return 0;
>> +
>> +fail:
>> +    clear_bitmap_table(bs, tb, tb_size);
>> +
>> +    if (tb_offset > 0) {
>> +        qcow2_free_clusters(bs, tb_offset, tb_size, 
>> QCOW2_DISCARD_ALWAYS);
>
> As before, I'd vote for QCOW2_DISCARD_OTHER.
>
>> +    }
>> +
>> +    g_free(tb);
>> +
>> +    return ret;
>> +}
>> +
>> +static Qcow2BitmapDirEntry *find_bitmap_by_name(uint8_t 
>> *bitmap_directory,
>> +                                                size_t size, const 
>> char *name)
>> +{
>> +    Qcow2BitmapDirEntry *e;
>> +
>> +    for_each_bitmap_dir_entry(e, bitmap_directory, size) {
>> +        if (strncmp((char *)(e + 1), name, e->name_size) == 0) {
>> +            return e;
>> +        }
>> +    }
>> +
>> +    return NULL;
>> +}
>> +
>> +void qcow2_store_persistent_bitmaps(BlockDriverState *bs, Error **errp)
>> +{
>> +    BdrvDirtyBitmap *bm;
>> +    BDRVQcow2State *s = bs->opaque;
>> +    uint32_t new_nb_bitmaps = s->nb_bitmaps;
>> +    uint64_t new_dir_size = s->bitmap_directory_size;
>> +    uint8_t *dir = NULL, *new_dir = NULL;
>> +    int ret;
>> +    Qcow2BitmapDirEntry *new_pos;
>> +
>> +    if (s->nb_bitmaps > 0) {
>> +        dir = directory_read(bs, s->bitmap_directory_offset,
>> +                             s->bitmap_directory_size, errp);
>> +        if (dir == NULL) {
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm != NULL;
>> +            bm = bdrv_dirty_bitmap_next(bs, bm)) {
>> +        const char *name = bdrv_dirty_bitmap_name(bm);
>> +
>> +        if (!bdrv_dirty_bitmap_get_persistance(bm)) {
>> +            continue;
>> +        }
>> +
>> +        if (s->nb_bitmaps > 0 &&
>> +                find_bitmap_by_name(dir, s->bitmap_directory_size, 
>> name)) {
>> +            error_setg(errp,
>> +                       "Can't store bitmap '%s' to '%s', as it 
>> already exists",
>> +                       name, bdrv_get_device_or_node_name(bs));
>> +            goto out;
>> +        }
>> +
>> +        new_nb_bitmaps++;
>> +        new_dir_size += calc_dir_entry_size(strlen(name), 0);
>> +    }
>> +
>> +    if (s->nb_bitmaps == new_nb_bitmaps) {
>> +        /* No new bitmaps - nothing to do */
>> +        goto out;
>> +    }
>> +
>> +    new_dir = g_try_malloc0(new_dir_size);
>> +    if (new_dir == NULL) {
>> +        error_setg(errp, "Can't allocate space for bitmap directory.");
>> +        goto out;
>> +    }
>> +
>> +    memcpy(new_dir, dir, s->bitmap_directory_size);
>> +    new_pos = (Qcow2BitmapDirEntry *)(new_dir + 
>> s->bitmap_directory_size);
>> +
>> +    for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm != NULL;
>> +            bm = bdrv_dirty_bitmap_next(bs, bm)) {
>> +        if (!bdrv_dirty_bitmap_get_persistance(bm)) {
>> +            continue;
>> +        }
>> +
>> +        ret = store_bitmap(bs, bm, new_pos);
>> +        if (ret < 0) {
>> +            error_setg_errno(errp, -ret, "Can't store bitmap '%s' to 
>> '%s'",
>> +                             bdrv_dirty_bitmap_name(bm),
>> +                             bdrv_get_device_or_node_name(bs));
>> +            goto out;
>> +        }
>> +        new_pos = next_dir_entry(new_pos);
>> +    }
>> +
>> +    ret = directory_update(bs, new_dir, new_dir_size, new_nb_bitmaps);
>> +    if (ret < 0) {
>> +        error_setg_errno(errp, -ret, "Can't update bitmap directory 
>> in '%s'",
>> +                         bdrv_get_device_or_node_name(bs));
>> +        goto out;
>> +    }
>> +
>> +out:
>> +    g_free(new_dir);
>> +    g_free(dir);
>
> This error path leaks all the bitmaps that have been written
> successfully (if any). I guess this is more or less fine if
> directory_update() failed (because you can't really tell the state of
> the image header after directory_update(), so better be safe) but it's
> not so fine if just some store_bitmap() failed.
>
> Max
>
>> +}
>> diff --git a/block/qcow2.c b/block/qcow2.c
>> index 02ec224..8238205 100644
>> --- a/block/qcow2.c
>> +++ b/block/qcow2.c
>> @@ -3493,6 +3493,8 @@ BlockDriver bdrv_qcow2 = {
>>
>>      .bdrv_detach_aio_context  = qcow2_detach_aio_context,
>>      .bdrv_attach_aio_context  = qcow2_attach_aio_context,
>> +
>> +    .bdrv_store_persistent_bitmaps = qcow2_store_persistent_bitmaps,
>>  };
>>
>>  static void bdrv_qcow2_init(void)
>> diff --git a/block/qcow2.h b/block/qcow2.h
>> index 482a29f..dfcf4c6 100644
>> --- a/block/qcow2.h
>> +++ b/block/qcow2.h
>> @@ -627,4 +627,6 @@ int qcow2_cache_get_empty(BlockDriverState *bs, 
>> Qcow2Cache *c, uint64_t offset,
>>      void **table);
>>  void qcow2_cache_put(BlockDriverState *bs, Qcow2Cache *c, void 
>> **table);
>>
>> +void qcow2_store_persistent_bitmaps(BlockDriverState *bs, Error 
>> **errp);
>> +
>>  #endif
>>
>
>
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Vladimir Sementsov-Ogievskiy]

On 01.10.2016 19:26, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are

[...]

> diff --git a/block/qcow2.c b/block/qcow2.c
> index 08c4ef9..02ec224 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c
> @@ -213,6 +213,11 @@ static int qcow2_read_extensions(BlockDriverState *bs, uint64_t start_offset,
>               s->bitmap_directory_size =
>                       bitmaps_ext.bitmap_directory_size;
>   
> +            ret = qcow2_read_bitmaps(bs, errp);
> +            if (ret < 0) {
> +                return ret;
> +            }
> +
> I think I'd put this directly into qcow2_open(), just like
> qcow2_read_snapshots(); but that's an optional suggestion.
>
> Max
>
>

Snapshots are not header extension.. so it is not the case. Here 
qcow2_read_bitmaps looks like part of header extension loading, and 
header extension fields describe other parts of the extension.. I think 
this is a good point, isn't it?

-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Vladimir Sementsov-Ogievskiy]

07.10.2016 22:25, Max Reitz пишет:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are
>> loaded at image open and becomes BdrvDirtyBitmap's for corresponding
>> drive. These bitmaps are deleted from Qcow2 image after loading to avoid
>> conflicts.
>>
>> Extra data in bitmaps is not supported for now.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block/qcow2-bitmap.c | 518 ++++++++++++++++++++++++++++++++++++++++++++++++++-
>>   block/qcow2.c        |   5 +
>>   block/qcow2.h        |   3 +
>>   3 files changed, 525 insertions(+), 1 deletion(-)
>>
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> index cd18b07..760a047 100644
>> --- a/block/qcow2-bitmap.c
>> +++ b/block/qcow2-bitmap.c
> [...]
>
>> +static int directory_update(BlockDriverState *bs, uint8_t *new_dir,
>> +                            size_t new_size, uint32_t new_nb_bitmaps)
>> +{
>> +    BDRVQcow2State *s = bs->opaque;
>> +    int ret;
>> +    int64_t new_offset = 0;
>> +    uint64_t old_offset = s->bitmap_directory_offset;
>> +    uint64_t old_size = s->bitmap_directory_size;
>> +    uint32_t old_nb_bitmaps = s->nb_bitmaps;
>> +    uint64_t old_autocl = s->autoclear_features;
>> +
>> +    if (new_size > QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE) {
>> +        return -EINVAL;
>> +    }
>> +
>> +    if ((new_size == 0) != (new_nb_bitmaps == 0)) {
>> +        return -EINVAL;
>> +    }
>> +
>> +    if (new_nb_bitmaps > 0) {
>> +        new_offset = directory_write(bs, new_dir, new_size);
>> +        if (new_offset < 0) {
>> +            return new_offset;
>> +        }
>> +
>> +        ret = bdrv_flush(bs);
>> +        if (ret < 0) {
>> +            goto fail;
>> +        }
>> +    }
>> +    s->bitmap_directory_offset = new_offset;
>> +    s->bitmap_directory_size = new_size;
>> +    s->nb_bitmaps = new_nb_bitmaps;
>> +
>> +    ret = update_header_sync(bs);
>> +    if (ret < 0) {
>> +        goto fail;
>> +    }
>> +
>> +    if (old_size) {
>> +        qcow2_free_clusters(bs, old_offset, old_size, QCOW2_DISCARD_ALWAYS);
>> +    }
>> +
>> +    return 0;
>> +
>> +fail:
>> +    if (new_offset > 0) {
>> +        qcow2_free_clusters(bs, new_offset, new_size, QCOW2_DISCARD_ALWAYS);
>> +        s->bitmap_directory_offset = old_offset;
>> +        s->bitmap_directory_size = old_size;
>> +        s->nb_bitmaps = old_nb_bitmaps;
>> +        s->autoclear_features = old_autocl;
> What if bdrv_flush() in update_header_sync() failed? Then you cannot be
> sure what bitmap directory the header is actually pointing to. In that
> case, it would be wrong to assume it's still pointing to the old one and
> freeing the new one.
>
> Max

Hmm.. Good point. Any suggestions?

I'm trying to achieve some kind of transaction - if function failed file 
is not changed.. But now I understand that because of flush ability to 
fail I can't achieve this..

Has write and flush any guaranties in case of fail? Would it be 
old-or-new state, or some mixed state is possible?

>
>> +    }
>> +
>> +    return ret;
>> +}


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

07.10.2016 22:44, Max Reitz пишет:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> This flag means that the bitmap is now in use by the software or was not
>> successfully saved. In any way, with this flag set the bitmap data must
>> be considered inconsistent and should not be loaded.
>>
>> With current implementation this flag is never set, as we just remove
>> bitmaps from the image after loading. But it defined in qcow2 spec and
>> must be handled. Also, it can be used in future, if async schemes of
>> bitmap loading/saving are implemented.
>>
>> We also remove in-use bitmaps from the image on open.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>   1 file changed, 16 insertions(+), 1 deletion(-)
> Don't you want to make use of this flag? It would appear useful to me if
> you just marked autoload bitmaps as in_use instead of deleting them from
> the image when it's opened and then overwrite them when the image is closed.

And what is the use of it?

Any way, storing an invalid copy of online data is bad I think. 
Therefore I will have to free bitmap data clusters and zero bitmap table 
in file.

>
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> index a5be25a..8cf40f0 100644
>> --- a/block/qcow2-bitmap.c
>> +++ b/block/qcow2-bitmap.c
>> @@ -28,6 +28,7 @@
>>   #include "qemu/osdep.h"
>>   #include "qapi/error.h"
>>   #include "qemu/cutils.h"
>> +#include "exec/log.h"
>>   
>>   #include "block/block_int.h"
>>   #include "block/qcow2.h"
>> @@ -44,7 +45,8 @@
>>   #define BME_MAX_NAME_SIZE 1023
>>   
>>   /* Bitmap directory entry flags */
>> -#define BME_RESERVED_FLAGS 0xfffffffd
>> +#define BME_RESERVED_FLAGS 0xfffffffc
>> +#define BME_FLAG_IN_USE 1
> This should be (1U << 0) to be consistent with the definition of
> BME_FLAG_AUTO.
>
>>   #define BME_FLAG_AUTO   (1U << 1)
>>   
>>   /* bits [1, 8] U [56, 63] are reserved */
>> @@ -287,6 +289,11 @@ static BdrvDirtyBitmap *load_bitmap(BlockDriverState *bs,
>>       BdrvDirtyBitmap *bitmap = NULL;
>>       char *name = g_strndup(dir_entry_name_notcstr(entry), entry->name_size);
>>   
>> +    if (entry->flags & BME_FLAG_IN_USE) {
>> +        error_setg(errp, "Bitmap '%s' is in use", name);
>> +        goto fail;
>> +    }
>> +
>>       ret = bitmap_table_load(bs, entry, &bitmap_table);
>>       if (ret < 0) {
>>           error_setg_errno(errp, -ret,
>> @@ -533,6 +540,14 @@ int qcow2_read_bitmaps(BlockDriverState *bs, Error **errp)
>>       for_each_bitmap_dir_entry(e, dir, dir_size) {
>>           uint32_t fl = e->flags;
>>   
>> +        if (fl & BME_FLAG_IN_USE) {
>> +            qemu_log("qcow2 warning: "
>> +                     "removing in_use bitmap '%.*s' for image %s.\n",
>> +                     e->name_size, (char *)(e + 1), bdrv_get_device_or_node_name(bs));
> I'm not sure whether qemu_log() is the right way to do this. I think
> fprintf() to stderr might actually be more appropriate. qemu_log() looks
> like it's mostly used for debugging purposes.
>
> Also, this is not a warning. You are just doing it. You are not warning
> someone about a cliff when he's already fallen off.
>
> But you can actually make it a warning: Just leave the bitmap in the
> image (maybe someone can do something with it?) and instead let qemu-img
> check clean it up. The warning should then just be "Warning: Ignoring
> in_use bitmap %.*s, use qemu-img check -r all to delete it".
>
> Max
>
>> +
>> +            continue;
>> +        }
>> +
>>           if (fl & BME_FLAG_AUTO) {
>>               BdrvDirtyBitmap *bitmap = load_bitmap(bs, e, errp);
>>               if (bitmap == NULL) {
>>
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1322 bytes --]

On 17.10.2016 19:19, Vladimir Sementsov-Ogievskiy wrote:
> On 07.10.2016 22:24, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> Realize block bitmap stroing interface, to allow qcow2 images store
>>> persistent bitmaps.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   block/qcow2-bitmap.c | 241
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++
> 
> [...]
> 
>>>
>>> +        ret = bdrv_pwrite(bs->file, off, buf, cl_size);
>>> +        if (ret < 0) {
>>> +            goto finish;
>>> +        }
>>> +
>>> +        if (end >= bm_size) {
>>> +            break;
>>> +        }
>>> +
>>> +        bdrv_set_dirty_iter(dbi, end);
>>> +    }
>>> +    ret = 0; /* writes */
>> What is that comment supposed to mean?
>>
>>
> 
> Now I think I can drop this assignment, as bdrv_aligned_preadv have
> 'return ret < 0 ? ret : 0;' in the end...  Am I right? Can bdrv_pwrite
> and friends return positive value on success?

I think so, but if they do return positive value they're supposed to be
equal to the number of bytes written. In practice, though, any
non-negative return value is treated the same (as success).

I was just wondering about the comment, though, not about the assignment
itself.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1621 bytes --]

On 20.10.2016 14:22, Vladimir Sementsov-Ogievskiy wrote:
> On 01.10.2016 19:26, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are
> 
> [...]
> 
>> diff --git a/block/qcow2.c b/block/qcow2.c
>> index 08c4ef9..02ec224 100644
>> --- a/block/qcow2.c
>> +++ b/block/qcow2.c
>> @@ -213,6 +213,11 @@ static int qcow2_read_extensions(BlockDriverState
>> *bs, uint64_t start_offset,
>>               s->bitmap_directory_size =
>>                       bitmaps_ext.bitmap_directory_size;
>>   +            ret = qcow2_read_bitmaps(bs, errp);
>> +            if (ret < 0) {
>> +                return ret;
>> +            }
>> +
>> I think I'd put this directly into qcow2_open(), just like
>> qcow2_read_snapshots(); but that's an optional suggestion.
>>
>> Max
>>
>>
> 
> Snapshots are not header extension.. so it is not the case. Here
> qcow2_read_bitmaps looks like part of header extension loading, and
> header extension fields describe other parts of the extension.. I think
> this is a good point, isn't it?

I said it's optional, so it's optional. :-)

I personally feel like a header extension is just the bit of data in the
qcow2 header. The bitmaps itself are managed by that extension, but not
part of the extension itself. Therefore, I wouldn't load it here but in
the main function qcow2_open().

However, this is a personal opinion and thus it's an optional suggestion
(as I said), so if you disagree (which you apparently do) then don't let
it bother you. :-)

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 07/22] qcow2-bitmap: introduce auto-loading bitmaps

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 3943 bytes --]

On 21.10.2016 13:59, Vladimir Sementsov-Ogievskiy wrote:
> 07.10.2016 22:25, Max Reitz пишет:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> Auto loading bitmaps are bitmaps in Qcow2, with AUTO flag set. They are
>>> loaded at image open and becomes BdrvDirtyBitmap's for corresponding
>>> drive. These bitmaps are deleted from Qcow2 image after loading to avoid
>>> conflicts.
>>>
>>> Extra data in bitmaps is not supported for now.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   block/qcow2-bitmap.c | 518
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++-
>>>   block/qcow2.c        |   5 +
>>>   block/qcow2.h        |   3 +
>>>   3 files changed, 525 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>>> index cd18b07..760a047 100644
>>> --- a/block/qcow2-bitmap.c
>>> +++ b/block/qcow2-bitmap.c
>> [...]
>>
>>> +static int directory_update(BlockDriverState *bs, uint8_t *new_dir,
>>> +                            size_t new_size, uint32_t new_nb_bitmaps)
>>> +{
>>> +    BDRVQcow2State *s = bs->opaque;
>>> +    int ret;
>>> +    int64_t new_offset = 0;
>>> +    uint64_t old_offset = s->bitmap_directory_offset;
>>> +    uint64_t old_size = s->bitmap_directory_size;
>>> +    uint32_t old_nb_bitmaps = s->nb_bitmaps;
>>> +    uint64_t old_autocl = s->autoclear_features;
>>> +
>>> +    if (new_size > QCOW_MAX_DIRTY_BITMAP_DIRECTORY_SIZE) {
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    if ((new_size == 0) != (new_nb_bitmaps == 0)) {
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    if (new_nb_bitmaps > 0) {
>>> +        new_offset = directory_write(bs, new_dir, new_size);
>>> +        if (new_offset < 0) {
>>> +            return new_offset;
>>> +        }
>>> +
>>> +        ret = bdrv_flush(bs);
>>> +        if (ret < 0) {
>>> +            goto fail;
>>> +        }
>>> +    }
>>> +    s->bitmap_directory_offset = new_offset;
>>> +    s->bitmap_directory_size = new_size;
>>> +    s->nb_bitmaps = new_nb_bitmaps;
>>> +
>>> +    ret = update_header_sync(bs);
>>> +    if (ret < 0) {
>>> +        goto fail;
>>> +    }
>>> +
>>> +    if (old_size) {
>>> +        qcow2_free_clusters(bs, old_offset, old_size,
>>> QCOW2_DISCARD_ALWAYS);
>>> +    }
>>> +
>>> +    return 0;
>>> +
>>> +fail:
>>> +    if (new_offset > 0) {
>>> +        qcow2_free_clusters(bs, new_offset, new_size,
>>> QCOW2_DISCARD_ALWAYS);
>>> +        s->bitmap_directory_offset = old_offset;
>>> +        s->bitmap_directory_size = old_size;
>>> +        s->nb_bitmaps = old_nb_bitmaps;
>>> +        s->autoclear_features = old_autocl;
>> What if bdrv_flush() in update_header_sync() failed? Then you cannot be
>> sure what bitmap directory the header is actually pointing to. In that
>> case, it would be wrong to assume it's still pointing to the old one and
>> freeing the new one.
>>
>> Max
> 
> Hmm.. Good point. Any suggestions?
> 
> I'm trying to achieve some kind of transaction - if function failed file
> is not changed.. But now I understand that because of flush ability to
> fail I can't achieve this..
> 
> Has write and flush any guaranties in case of fail? Would it be
> old-or-new state, or some mixed state is possible?

Anything is possible, I think. However, if flushing the data fails, it
still means writing it was successful -- you just don't know whether it
has been written to the disk successfully.

The most correct way out would be to signal image corruption, but that
isn't what we should do, I think.

If the write operation was successful but flushing was not, I'd always
assume the data hopefully gets flushed later and continue as if it had
been written to disk successfully. Therefore, I think it would be best
for update_header_sync() to just ignore bdrv_flush()'s return value.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]

On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
> 07.10.2016 22:44, Max Reitz пишет:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> This flag means that the bitmap is now in use by the software or was not
>>> successfully saved. In any way, with this flag set the bitmap data must
>>> be considered inconsistent and should not be loaded.
>>>
>>> With current implementation this flag is never set, as we just remove
>>> bitmaps from the image after loading. But it defined in qcow2 spec and
>>> must be handled. Also, it can be used in future, if async schemes of
>>> bitmap loading/saving are implemented.
>>>
>>> We also remove in-use bitmaps from the image on open.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>   1 file changed, 16 insertions(+), 1 deletion(-)
>> Don't you want to make use of this flag? It would appear useful to me if
>> you just marked autoload bitmaps as in_use instead of deleting them from
>> the image when it's opened and then overwrite them when the image is
>> closed.
> 
> And what is the use of it?

You don't need to free any bitmaps when opening the file, and you don't
need to allocate any new bitmap space when closing it.

> Any way, storing an invalid copy of online data is bad I think.
> Therefore I will have to free bitmap data clusters and zero bitmap table
> in file.

Why can't you just set this flag to mark it invalid? In case qemu
crashes, in one case you'll have some invalid bitmaps and in the other
you won't have any of them at all. I don't think there's a difference
from that perspective.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 11/22] qcow2-bitmap: add qcow2_store_persistent_bitmaps()

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 967 bytes --]

On 10/21/2016 02:44 PM, Max Reitz wrote:

>> Now I think I can drop this assignment, as bdrv_aligned_preadv have
>> 'return ret < 0 ? ret : 0;' in the end...  Am I right? Can bdrv_pwrite
>> and friends return positive value on success?
> 
> I think so, but if they do return positive value they're supposed to be
> equal to the number of bytes written. In practice, though, any
> non-negative return value is treated the same (as success).

At one point in my conversion to byte access, I ran into test failures
when trying to return number of bytes written; so commit 1a62d0a
hard-codes a return of 0 on success instead.  It will be a long and
painful audit to find which callers expect which semantics, which I did
not want to do.

> 
> I was just wondering about the comment, though, not about the assignment
> itself.
> 
> Max
> 

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

21.10.2016 22:58, Max Reitz пишет:
> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>> 07.10.2016 22:44, Max Reitz пишет:
>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>> This flag means that the bitmap is now in use by the software or was not
>>>> successfully saved. In any way, with this flag set the bitmap data must
>>>> be considered inconsistent and should not be loaded.
>>>>
>>>> With current implementation this flag is never set, as we just remove
>>>> bitmaps from the image after loading. But it defined in qcow2 spec and
>>>> must be handled. Also, it can be used in future, if async schemes of
>>>> bitmap loading/saving are implemented.
>>>>
>>>> We also remove in-use bitmaps from the image on open.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>>    block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>    1 file changed, 16 insertions(+), 1 deletion(-)
>>> Don't you want to make use of this flag? It would appear useful to me if
>>> you just marked autoload bitmaps as in_use instead of deleting them from
>>> the image when it's opened and then overwrite them when the image is
>>> closed.
>> And what is the use of it?
> You don't need to free any bitmaps when opening the file, and you don't
> need to allocate any new bitmap space when closing it.

As bitmaps are sparce in file, I need to allocate new space when 
closing. Or free it...

>
>> Any way, storing an invalid copy of online data is bad I think.
>> Therefore I will have to free bitmap data clusters and zero bitmap table
>> in file.
> Why can't you just set this flag to mark it invalid? In case qemu
> crashes, in one case you'll have some invalid bitmaps and in the other
> you won't have any of them at all. I don't think there's a difference
> from that perspective.
>
> Max
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
> 21.10.2016 22:58, Max Reitz пишет:
>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>> 07.10.2016 22:44, Max Reitz пишет:
>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>> This flag means that the bitmap is now in use by the software or 
>>>>> was not
>>>>> successfully saved. In any way, with this flag set the bitmap data 
>>>>> must
>>>>> be considered inconsistent and should not be loaded.
>>>>>
>>>>> With current implementation this flag is never set, as we just remove
>>>>> bitmaps from the image after loading. But it defined in qcow2 spec 
>>>>> and
>>>>> must be handled. Also, it can be used in future, if async schemes of
>>>>> bitmap loading/saving are implemented.
>>>>>
>>>>> We also remove in-use bitmaps from the image on open.
>>>>>
>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy 
>>>>> <vsementsov@virtuozzo.com>
>>>>> ---
>>>>>    block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>    1 file changed, 16 insertions(+), 1 deletion(-)
>>>> Don't you want to make use of this flag? It would appear useful to 
>>>> me if
>>>> you just marked autoload bitmaps as in_use instead of deleting them 
>>>> from
>>>> the image when it's opened and then overwrite them when the image is
>>>> closed.
>>> And what is the use of it?
>> You don't need to free any bitmaps when opening the file, and you don't
>> need to allocate any new bitmap space when closing it.
>
> As bitmaps are sparce in file, I need to allocate new space when 
> closing. Or free it...


Is it a real problem to reallocate space in qcow2? If so, to minimaze 
allocations, I will have to make a list of clusters of in_use bitmaps on 
close, and then save current bitmaps to these clusters (and allocated 
some additional clusters, or free extra clusters).

Also, if I don't free them on open, I'll have to free them on remove of 
bdrv dirty bitmap..




>
>>
>>> Any way, storing an invalid copy of online data is bad I think.
>>> Therefore I will have to free bitmap data clusters and zero bitmap 
>>> table
>>> in file.
>> Why can't you just set this flag to mark it invalid? In case qemu
>> crashes, in one case you'll have some invalid bitmaps and in the other
>> you won't have any of them at all. I don't think there's a difference
>> from that perspective.
>>
>> Max
>>
>
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 15/22] qcow2-bitmap: add autoclear bit

[Vladimir Sementsov-Ogievskiy]

07.10.2016 23:11, Max Reitz пишет:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Add autoclear bit for handling rewriting image by old qemu version.
> "Add support for the autoclear bit [...]"?

finally "Add support for the autoclear bit to handle rewriting image by old
qemu version" - ok?

Or I can merge this patch into two earlier patches 06 and 07.

>
>> If autoclear bit is not set, but bitmaps extension is found it
>> would not be loaded and warning will be generated.
> "If the autoclear bit is not set, but the bitmaps extension is found,
> the bitmaps will not be loaded and a warning will be generated."
>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block/qcow2-bitmap.c |  4 ++++
>>   block/qcow2.c        | 12 ++++++++++--
>>   block/qcow2.h        |  9 +++++++++
>>   3 files changed, 23 insertions(+), 2 deletions(-)
> Apart from the above, the patch looks good, but why does it come so late
> in the series?
>
> So with the commit message fixed:
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 16/22] qmp: add persistent flag to block-dirty-bitmap-add

[Vladimir Sementsov-Ogievskiy]

07.10.2016 22:52, Eric Blake пишет:
> On 09/30/2016 05:53 AM, Vladimir Sementsov-Ogievskiy wrote:
>> Add optional 'persistent' flag to qmp command block-dirty-bitmap-add.
>> Default is false.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Signed-off-by: Denis V. Lunev <den@openvz.org>
>> ---
>>   blockdev.c           | 12 +++++++++++-
>>   qapi/block-core.json |  7 ++++++-
>>   qmp-commands.hx      |  5 ++++-
> This and later patches have conflicts with recent changes in the tree
> that removed qmp-commands.hx; hopefully shouldn't impact review of this
> series too badly.
>
>> +++ b/qapi/block-core.json
>> @@ -1235,10 +1235,15 @@
>>   # @granularity: #optional the bitmap granularity, default is 64k for
>>   #               block-dirty-bitmap-add
>>   #
>> +# @persistent: #optional the bitmap is persistent, i.e. it will be saved to
>> +#              corresponding block device on it's close. Default is false.
> s/corresponding/the corresponding/
> s/it's/its/
>
> ("it's" is only appropriate if you can substitute "it is" or "it has")
>
>
>> @@ -1458,6 +1458,9 @@ Arguments:
>>   - "node": device/node on which to create dirty bitmap (json-string)
>>   - "name": name of the new dirty bitmap (json-string)
>>   - "granularity": granularity to track writes with (int, optional)
>> +- "persistent": bitmap will be saved to corresponding block device
>> +                on it's close. Block driver should maintain persistent bitmaps
> One nice benefit of rebasing on top of qmp-commands.hx being removed is
> that you don't have to repeat yourself :)

As I understand, duplicating description is still here, in 
docs/qmp-commands.txt. I just don't have to repeat parameters definition 
(which is good, anyway)



-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 16/22] qmp: add persistent flag to block-dirty-bitmap-add

[Vladimir Sementsov-Ogievskiy]

10.10.2016 19:08, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>

[...]

>> +    }
>>   
>>    out:
>>       aio_context_release(aio_context);
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index 31f9990..2bf56cd 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -1235,10 +1235,15 @@
>>   # @granularity: #optional the bitmap granularity, default is 64k for
>>   #               block-dirty-bitmap-add
>>   #
>> +# @persistent: #optional the bitmap is persistent, i.e. it will be saved to
>> +#              corresponding block device on it's close. Default is false.
>> +#              For block-dirty-bitmap-add. (Since 2.8)
> I'm not sure what the "For block-dirty-bitmap-add." is supposed to mean,
> because this whole struct is for block-dirty-bitmap-add (and for
> block-dirty-bitmap-add transactions, to be exact, but @persistent will
> surely work there, too, won't it?).
>
> Also, I'd say "will be saved to the corresponding block device image
> file" instead of just "block device", because in my understanding a
> block device and its image file are two separate things.

Hmm.. But 'its close' is block device close, not file close. And we call 
common bdrv_* function to save it, so we actually save it to device, and 
then the device puzzles out, how to actually save it.

>
>> +#
>>   # Since 2.4
>>   ##
>>   { 'struct': 'BlockDirtyBitmapAdd',
>> -  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32' } }
>> +  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
>> +  '*persistent': 'bool' } }
> I think normally we'd align the new line so that the opening ' of
> '*persistent' is under the opening ' of 'node'.
>
>>   
>>   ##
>>   # @block-dirty-bitmap-add
>> diff --git a/qmp-commands.hx b/qmp-commands.hx
>> index ba2a916..434b418 100644
>> --- a/qmp-commands.hx
>> +++ b/qmp-commands.hx
>> @@ -1441,7 +1441,7 @@ EQMP
>>   
>>       {
>>           .name       = "block-dirty-bitmap-add",
>> -        .args_type  = "node:B,name:s,granularity:i?",
>> +        .args_type  = "node:B,name:s,granularity:i?,persistent:b?",
>>           .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
>>       },
>>   
>> @@ -1458,6 +1458,9 @@ Arguments:
>>   - "node": device/node on which to create dirty bitmap (json-string)
>>   - "name": name of the new dirty bitmap (json-string)
>>   - "granularity": granularity to track writes with (int, optional)
>> +- "persistent": bitmap will be saved to corresponding block device
>> +                on it's close. Block driver should maintain persistent bitmaps
>> +                (json-bool, optional, default false) (Since 2.8)
> And I don't know what the user is supposed to make of the information
> that block drivers will take care of maintaining persistent bitmaps. All
> they care about is that it will be stored in the corresponding image
> file, so in my opinion it would be better to just omit the last sentence
> here.

Users shoud know, that only qcow2 supports persistent bitmaps. Instead 
of last sentence I can write "For now only Qcow2 disks supports 
persistent bitmaps".


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 17/22] qmp: add autoload parameter to block-dirty-bitmap-add

[Vladimir Sementsov-Ogievskiy]

10.10.2016 19:25, Max Reitz wrote:
> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>> Optional. Default is false.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Signed-off-by: Denis V. Lunev <den@openvz.org>
>> ---
>>   blockdev.c           | 22 ++++++++++++++++++++--
>>   qapi/block-core.json |  7 ++++++-
>>   qmp-commands.hx      |  5 ++++-
>>   3 files changed, 30 insertions(+), 4 deletions(-)
> Design question: I see that being able to specify these flags when
> creating bitmaps is useful. However, would about a way for the user to
> change these flags on an existing dirty bitmap? Would you consider that
> useful?
>
> (Of course, if so, it can always be added later, we don't need it now.)

I vote for adding later, if needed. For now we don't have such scenarios.


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 1743 bytes --]

On 24.10.2016 12:32, Vladimir Sementsov-Ogievskiy wrote:
> 21.10.2016 22:58, Max Reitz пишет:
>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>> 07.10.2016 22:44, Max Reitz пишет:
>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>> This flag means that the bitmap is now in use by the software or
>>>>> was not
>>>>> successfully saved. In any way, with this flag set the bitmap data
>>>>> must
>>>>> be considered inconsistent and should not be loaded.
>>>>>
>>>>> With current implementation this flag is never set, as we just remove
>>>>> bitmaps from the image after loading. But it defined in qcow2 spec and
>>>>> must be handled. Also, it can be used in future, if async schemes of
>>>>> bitmap loading/saving are implemented.
>>>>>
>>>>> We also remove in-use bitmaps from the image on open.
>>>>>
>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>> ---
>>>>>    block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>    1 file changed, 16 insertions(+), 1 deletion(-)
>>>> Don't you want to make use of this flag? It would appear useful to
>>>> me if
>>>> you just marked autoload bitmaps as in_use instead of deleting them
>>>> from
>>>> the image when it's opened and then overwrite them when the image is
>>>> closed.
>>> And what is the use of it?
>> You don't need to free any bitmaps when opening the file, and you don't
>> need to allocate any new bitmap space when closing it.
> 
> As bitmaps are sparce in file, I need to allocate new space when
> closing. Or free it...

May happen. But not necessarily, and it will probably still save time as
you can reuse existing allocations and don't have to free everything.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 3331 bytes --]

On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>> 21.10.2016 22:58, Max Reitz пишет:
>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>> This flag means that the bitmap is now in use by the software or
>>>>>> was not
>>>>>> successfully saved. In any way, with this flag set the bitmap data
>>>>>> must
>>>>>> be considered inconsistent and should not be loaded.
>>>>>>
>>>>>> With current implementation this flag is never set, as we just remove
>>>>>> bitmaps from the image after loading. But it defined in qcow2 spec
>>>>>> and
>>>>>> must be handled. Also, it can be used in future, if async schemes of
>>>>>> bitmap loading/saving are implemented.
>>>>>>
>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>
>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>> <vsementsov@virtuozzo.com>
>>>>>> ---
>>>>>>    block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>    1 file changed, 16 insertions(+), 1 deletion(-)
>>>>> Don't you want to make use of this flag? It would appear useful to
>>>>> me if
>>>>> you just marked autoload bitmaps as in_use instead of deleting them
>>>>> from
>>>>> the image when it's opened and then overwrite them when the image is
>>>>> closed.
>>>> And what is the use of it?
>>> You don't need to free any bitmaps when opening the file, and you don't
>>> need to allocate any new bitmap space when closing it.
>>
>> As bitmaps are sparce in file, I need to allocate new space when
>> closing. Or free it...
> 
> 
> Is it a real problem to reallocate space in qcow2? If so, to minimaze
> allocations, I will have to make a list of clusters of in_use bitmaps on
> close, and then save current bitmaps to these clusters (and allocated
> some additional clusters, or free extra clusters).

It's not a real problem, but it does take time, and I maintain that it's
time it doesn't need to take because you can just use the in_use flag.

I wouldn't worry about reusing clusters of other bitmaps. Of course we
can do that later on in some optimization but not now.

I just mean the basic case of some auto-loaded bitmap that is not being
deleted while the VM is running and is just saved back to the image file
once it is closed. I don't expect that users will always consume all of
the auto-loaded bitmaps while the VM is active...

> Also, if I don't free them on open, I'll have to free them on remove of
> bdrv dirty bitmap..

Yes, so? That takes time, but that is something the user will probably
expect. I wouldn't expect opening of the file to take that time.

Overall, it doesn't matter time-wise whether you free the bitmap data
when opening the image file or when the dirty bitmap is actually deleted
by the user or when the image file is closed. Just setting the single
in_use flag for all of the auto-loaded bitmaps will basically not take
any time.

On the other hand, as soon as just a single auto-loaded bitmap survives
a VM (or qemu-img) invocation, you will very, very likely safe at least
some time because writing the bitmap to the disk can reuse at least some
of the existing clusters.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 4205 bytes --]

On 24.10.2016 19:08, Max Reitz wrote:
> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>> 21.10.2016 22:58, Max Reitz пишет:
>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> This flag means that the bitmap is now in use by the software or
>>>>>>> was not
>>>>>>> successfully saved. In any way, with this flag set the bitmap data
>>>>>>> must
>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>
>>>>>>> With current implementation this flag is never set, as we just remove
>>>>>>> bitmaps from the image after loading. But it defined in qcow2 spec
>>>>>>> and
>>>>>>> must be handled. Also, it can be used in future, if async schemes of
>>>>>>> bitmap loading/saving are implemented.
>>>>>>>
>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>
>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>> ---
>>>>>>>    block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>    1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>> Don't you want to make use of this flag? It would appear useful to
>>>>>> me if
>>>>>> you just marked autoload bitmaps as in_use instead of deleting them
>>>>>> from
>>>>>> the image when it's opened and then overwrite them when the image is
>>>>>> closed.
>>>>> And what is the use of it?
>>>> You don't need to free any bitmaps when opening the file, and you don't
>>>> need to allocate any new bitmap space when closing it.
>>>
>>> As bitmaps are sparce in file, I need to allocate new space when
>>> closing. Or free it...
>>
>>
>> Is it a real problem to reallocate space in qcow2? If so, to minimaze
>> allocations, I will have to make a list of clusters of in_use bitmaps on
>> close, and then save current bitmaps to these clusters (and allocated
>> some additional clusters, or free extra clusters).
> 
> It's not a real problem, but it does take time, and I maintain that it's
> time it doesn't need to take because you can just use the in_use flag.
> 
> I wouldn't worry about reusing clusters of other bitmaps. Of course we
> can do that later on in some optimization but not now.
> 
> I just mean the basic case of some auto-loaded bitmap that is not being
> deleted while the VM is running and is just saved back to the image file
> once it is closed. I don't expect that users will always consume all of
> the auto-loaded bitmaps while the VM is active...
> 
>> Also, if I don't free them on open, I'll have to free them on remove of
>> bdrv dirty bitmap..
> 
> Yes, so? That takes time, but that is something the user will probably
> expect. I wouldn't expect opening of the file to take that time.
> 
> Overall, it doesn't matter time-wise whether you free the bitmap data
> when opening the image file or when the dirty bitmap is actually deleted
> by the user or when the image file is closed. Just setting the single
> in_use flag for all of the auto-loaded bitmaps will basically not take
> any time.
> 
> On the other hand, as soon as just a single auto-loaded bitmap survives
> a VM (or qemu-img) invocation, you will very, very likely safe at least
> some time because writing the bitmap to the disk can reuse at least some
> of the existing clusters.

By the way, dealing with removal of bitmaps when they are deleted by the
user is also positive when it comes to migration or read-only disks that
are later reopened R/W: Currently, as far as I can see, you just keep
the auto-load bitmaps in the image if the image is part of an incoming
migration or opened read-only, but then you continue under the
assumption that they are removed from the image. That's not very good.

If you delete the bitmaps only when the user asks you to, then you can
just return an error that the bitmap cannot be removed at this time
because the image file is read-only or used in migration (and I don't
think the latter can even happen when it's the user who has requested
removal of the bitmap).

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 15/22] qcow2-bitmap: add autoclear bit

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 482 bytes --]

On 24.10.2016 16:25, Vladimir Sementsov-Ogievskiy wrote:
> 07.10.2016 23:11, Max Reitz пишет:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>> Add autoclear bit for handling rewriting image by old qemu version.
>> "Add support for the autoclear bit [...]"?
> 
> finally "Add support for the autoclear bit to handle rewriting image by old
> qemu version" - ok?
> 
> Or I can merge this patch into two earlier patches 06 and 07.

Sounds good.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 16/22] qmp: add persistent flag to block-dirty-bitmap-add

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 4740 bytes --]

On 24.10.2016 17:12, Vladimir Sementsov-Ogievskiy wrote:
> 10.10.2016 19:08, Max Reitz wrote:
>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>
> 
> [...]
> 
>>> +    }
>>>      out:
>>>       aio_context_release(aio_context);
>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>> index 31f9990..2bf56cd 100644
>>> --- a/qapi/block-core.json
>>> +++ b/qapi/block-core.json
>>> @@ -1235,10 +1235,15 @@
>>>   # @granularity: #optional the bitmap granularity, default is 64k for
>>>   #               block-dirty-bitmap-add
>>>   #
>>> +# @persistent: #optional the bitmap is persistent, i.e. it will be
>>> saved to
>>> +#              corresponding block device on it's close. Default is
>>> false.
>>> +#              For block-dirty-bitmap-add. (Since 2.8)
>> I'm not sure what the "For block-dirty-bitmap-add." is supposed to mean,
>> because this whole struct is for block-dirty-bitmap-add (and for
>> block-dirty-bitmap-add transactions, to be exact, but @persistent will
>> surely work there, too, won't it?).
>>
>> Also, I'd say "will be saved to the corresponding block device image
>> file" instead of just "block device", because in my understanding a
>> block device and its image file are two separate things.
> 
> Hmm.. But 'its close' is block device close, not file close.

In my understanding, it is the file close.

>                                                              And we call
> common bdrv_* function to save it, so we actually save it to device, and
> then the device puzzles out, how to actually save it.

Well, OK, it depends on what you mean by "block device". There are many
things we call a "block device", but nowadays I think it mostly refers
to either a guest block device or a BlockBackend (and as of lately,
we're more and more trying to hide the BlockBackend from the user, so
you could argue that it's only the guest device now).

The bdrv_* functions operate on block layer BDS nodes, and I don't think
we call them "block devices" (at least not any more).

In any case, I think for users the term "block device" refers to either
the device presented to the guest or to all of the block layer stuff
that's underneath, and it's not quite clear how you could save a bitmap
to that, or what it's supposed to mean to "close" a block device (you
can remove it, you can destroy it, you can delete it, but "close" it?).

But saying that it will be saved to the image file that is attached to
the block device will make it absolutely clear what we mean.

So what you have called a "device" here is neither what I'd call a
device (I'd call it a "node" or "BDS"), nor what I think users would
call a device. Also, it's not the BDS that puzzles out how to save it
but some block driver.

>>> +#
>>>   # Since 2.4
>>>   ##
>>>   { 'struct': 'BlockDirtyBitmapAdd',
>>> -  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32' } }
>>> +  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
>>> +  '*persistent': 'bool' } }
>> I think normally we'd align the new line so that the opening ' of
>> '*persistent' is under the opening ' of 'node'.
>>
>>>     ##
>>>   # @block-dirty-bitmap-add
>>> diff --git a/qmp-commands.hx b/qmp-commands.hx
>>> index ba2a916..434b418 100644
>>> --- a/qmp-commands.hx
>>> +++ b/qmp-commands.hx
>>> @@ -1441,7 +1441,7 @@ EQMP
>>>         {
>>>           .name       = "block-dirty-bitmap-add",
>>> -        .args_type  = "node:B,name:s,granularity:i?",
>>> +        .args_type  = "node:B,name:s,granularity:i?,persistent:b?",
>>>           .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
>>>       },
>>>   @@ -1458,6 +1458,9 @@ Arguments:
>>>   - "node": device/node on which to create dirty bitmap (json-string)
>>>   - "name": name of the new dirty bitmap (json-string)
>>>   - "granularity": granularity to track writes with (int, optional)
>>> +- "persistent": bitmap will be saved to corresponding block device
>>> +                on it's close. Block driver should maintain
>>> persistent bitmaps
>>> +                (json-bool, optional, default false) (Since 2.8)
>> And I don't know what the user is supposed to make of the information
>> that block drivers will take care of maintaining persistent bitmaps. All
>> they care about is that it will be stored in the corresponding image
>> file, so in my opinion it would be better to just omit the last sentence
>> here.
> 
> Users shoud know, that only qcow2 supports persistent bitmaps. Instead
> of last sentence I can write "For now only Qcow2 disks supports
> persistent bitmaps".

s/supports/support/, but yes, that sounds preferable to me.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

24.10.2016 20:18, Max Reitz wrote:
> On 24.10.2016 19:08, Max Reitz wrote:
>> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>>> 21.10.2016 22:58, Max Reitz пишет:
>>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>> This flag means that the bitmap is now in use by the software or
>>>>>>>> was not
>>>>>>>> successfully saved. In any way, with this flag set the bitmap data
>>>>>>>> must
>>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>>
>>>>>>>> With current implementation this flag is never set, as we just remove
>>>>>>>> bitmaps from the image after loading. But it defined in qcow2 spec
>>>>>>>> and
>>>>>>>> must be handled. Also, it can be used in future, if async schemes of
>>>>>>>> bitmap loading/saving are implemented.
>>>>>>>>
>>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>>
>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>>> ---
>>>>>>>>     block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>>     1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>>> Don't you want to make use of this flag? It would appear useful to
>>>>>>> me if
>>>>>>> you just marked autoload bitmaps as in_use instead of deleting them
>>>>>>> from
>>>>>>> the image when it's opened and then overwrite them when the image is
>>>>>>> closed.
>>>>>> And what is the use of it?
>>>>> You don't need to free any bitmaps when opening the file, and you don't
>>>>> need to allocate any new bitmap space when closing it.
>>>> As bitmaps are sparce in file, I need to allocate new space when
>>>> closing. Or free it...
>>>
>>> Is it a real problem to reallocate space in qcow2? If so, to minimaze
>>> allocations, I will have to make a list of clusters of in_use bitmaps on
>>> close, and then save current bitmaps to these clusters (and allocated
>>> some additional clusters, or free extra clusters).
>> It's not a real problem, but it does take time, and I maintain that it's
>> time it doesn't need to take because you can just use the in_use flag.
>>
>> I wouldn't worry about reusing clusters of other bitmaps. Of course we
>> can do that later on in some optimization but not now.
>>
>> I just mean the basic case of some auto-loaded bitmap that is not being
>> deleted while the VM is running and is just saved back to the image file
>> once it is closed. I don't expect that users will always consume all of
>> the auto-loaded bitmaps while the VM is active...
>>
>>> Also, if I don't free them on open, I'll have to free them on remove of
>>> bdrv dirty bitmap..
>> Yes, so? That takes time, but that is something the user will probably
>> expect. I wouldn't expect opening of the file to take that time.
>>
>> Overall, it doesn't matter time-wise whether you free the bitmap data
>> when opening the image file or when the dirty bitmap is actually deleted
>> by the user or when the image file is closed. Just setting the single
>> in_use flag for all of the auto-loaded bitmaps will basically not take
>> any time.
>>
>> On the other hand, as soon as just a single auto-loaded bitmap survives
>> a VM (or qemu-img) invocation, you will very, very likely safe at least
>> some time because writing the bitmap to the disk can reuse at least some
>> of the existing clusters.
> By the way, dealing with removal of bitmaps when they are deleted by the
> user is also positive when it comes to migration or read-only disks that
> are later reopened R/W: Currently, as far as I can see, you just keep
> the auto-load bitmaps in the image if the image is part of an incoming
> migration or opened read-only, but then you continue under the
> assumption that they are removed from the image. That's not very good.

You are right, I need to handle reopening more carefully. In my way, I 
need to delete bitmaps when reopening R/W and in yours - set in_use bit.

>
> If you delete the bitmaps only when the user asks you to, then you can
> just return an error that the bitmap cannot be removed at this time
> because the image file is read-only or used in migration (and I don't
> think the latter can even happen when it's the user who has requested
> removal of the bitmap).
>
> Max
>

Ok, I'll rewrite this way, using 'in_use' bit and trying to avoid 
allocate/free overhead.


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 16/22] qmp: add persistent flag to block-dirty-bitmap-add

[Vladimir Sementsov-Ogievskiy]

24.10.2016 20:30, Max Reitz wrote:
> On 24.10.2016 17:12, Vladimir Sementsov-Ogievskiy wrote:
>> 10.10.2016 19:08, Max Reitz wrote:
>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>
>> [...]
>>
>>>> +    }
>>>>       out:
>>>>        aio_context_release(aio_context);
>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>>> index 31f9990..2bf56cd 100644
>>>> --- a/qapi/block-core.json
>>>> +++ b/qapi/block-core.json
>>>> @@ -1235,10 +1235,15 @@
>>>>    # @granularity: #optional the bitmap granularity, default is 64k for
>>>>    #               block-dirty-bitmap-add
>>>>    #
>>>> +# @persistent: #optional the bitmap is persistent, i.e. it will be
>>>> saved to
>>>> +#              corresponding block device on it's close. Default is
>>>> false.
>>>> +#              For block-dirty-bitmap-add. (Since 2.8)
>>> I'm not sure what the "For block-dirty-bitmap-add." is supposed to mean,
>>> because this whole struct is for block-dirty-bitmap-add (and for
>>> block-dirty-bitmap-add transactions, to be exact, but @persistent will
>>> surely work there, too, won't it?).
>>>
>>> Also, I'd say "will be saved to the corresponding block device image
>>> file" instead of just "block device", because in my understanding a
>>> block device and its image file are two separate things.
>> Hmm.. But 'its close' is block device close, not file close.
> In my understanding, it is the file close.
>
>>                                                               And we call
>> common bdrv_* function to save it, so we actually save it to device, and
>> then the device puzzles out, how to actually save it.
> Well, OK, it depends on what you mean by "block device". There are many
> things we call a "block device", but nowadays I think it mostly refers
> to either a guest block device or a BlockBackend (and as of lately,
> we're more and more trying to hide the BlockBackend from the user, so
> you could argue that it's only the guest device now).
>
> The bdrv_* functions operate on block layer BDS nodes, and I don't think
> we call them "block devices" (at least not any more).
>
> In any case, I think for users the term "block device" refers to either
> the device presented to the guest or to all of the block layer stuff
> that's underneath, and it's not quite clear how you could save a bitmap
> to that, or what it's supposed to mean to "close" a block device (you
> can remove it, you can destroy it, you can delete it, but "close" it?).
>
> But saying that it will be saved to the image file that is attached to
> the block device will make it absolutely clear what we mean.
>
> So what you have called a "device" here is neither what I'd call a
> device (I'd call it a "node" or "BDS"), nor what I think users would
> call a device. Also, it's not the BDS that puzzles out how to save it
> but some block driver.
>

Ok, thank you.

>>>> +#
>>>>    # Since 2.4
>>>>    ##
>>>>    { 'struct': 'BlockDirtyBitmapAdd',
>>>> -  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32' } }
>>>> +  'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
>>>> +  '*persistent': 'bool' } }
>>> I think normally we'd align the new line so that the opening ' of
>>> '*persistent' is under the opening ' of 'node'.
>>>
>>>>      ##
>>>>    # @block-dirty-bitmap-add
>>>> diff --git a/qmp-commands.hx b/qmp-commands.hx
>>>> index ba2a916..434b418 100644
>>>> --- a/qmp-commands.hx
>>>> +++ b/qmp-commands.hx
>>>> @@ -1441,7 +1441,7 @@ EQMP
>>>>          {
>>>>            .name       = "block-dirty-bitmap-add",
>>>> -        .args_type  = "node:B,name:s,granularity:i?",
>>>> +        .args_type  = "node:B,name:s,granularity:i?,persistent:b?",
>>>>            .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
>>>>        },
>>>>    @@ -1458,6 +1458,9 @@ Arguments:
>>>>    - "node": device/node on which to create dirty bitmap (json-string)
>>>>    - "name": name of the new dirty bitmap (json-string)
>>>>    - "granularity": granularity to track writes with (int, optional)
>>>> +- "persistent": bitmap will be saved to corresponding block device
>>>> +                on it's close. Block driver should maintain
>>>> persistent bitmaps
>>>> +                (json-bool, optional, default false) (Since 2.8)
>>> And I don't know what the user is supposed to make of the information
>>> that block drivers will take care of maintaining persistent bitmaps. All
>>> they care about is that it will be stored in the corresponding image
>>> file, so in my opinion it would be better to just omit the last sentence
>>> here.
>> Users shoud know, that only qcow2 supports persistent bitmaps. Instead
>> of last sentence I can write "For now only Qcow2 disks supports
>> persistent bitmaps".
> s/supports/support/, but yes, that sounds preferable to me.
>
> Max
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote:
> 24.10.2016 20:18, Max Reitz wrote:
>> On 24.10.2016 19:08, Max Reitz wrote:
>>> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>>>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>>>> 21.10.2016 22:58, Max Reitz пишет:
>>>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>> This flag means that the bitmap is now in use by the software or
>>>>>>>>> was not
>>>>>>>>> successfully saved. In any way, with this flag set the bitmap 
>>>>>>>>> data
>>>>>>>>> must
>>>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>>>
>>>>>>>>> With current implementation this flag is never set, as we just 
>>>>>>>>> remove
>>>>>>>>> bitmaps from the image after loading. But it defined in qcow2 
>>>>>>>>> spec
>>>>>>>>> and
>>>>>>>>> must be handled. Also, it can be used in future, if async 
>>>>>>>>> schemes of
>>>>>>>>> bitmap loading/saving are implemented.
>>>>>>>>>
>>>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>>>> ---
>>>>>>>>>     block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>>>     1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>>>> Don't you want to make use of this flag? It would appear useful to
>>>>>>>> me if
>>>>>>>> you just marked autoload bitmaps as in_use instead of deleting 
>>>>>>>> them
>>>>>>>> from
>>>>>>>> the image when it's opened and then overwrite them when the 
>>>>>>>> image is
>>>>>>>> closed.
>>>>>>> And what is the use of it?
>>>>>> You don't need to free any bitmaps when opening the file, and you 
>>>>>> don't
>>>>>> need to allocate any new bitmap space when closing it.
>>>>> As bitmaps are sparce in file, I need to allocate new space when
>>>>> closing. Or free it...
>>>>
>>>> Is it a real problem to reallocate space in qcow2? If so, to minimaze
>>>> allocations, I will have to make a list of clusters of in_use 
>>>> bitmaps on
>>>> close, and then save current bitmaps to these clusters (and allocated
>>>> some additional clusters, or free extra clusters).
>>> It's not a real problem, but it does take time, and I maintain that 
>>> it's
>>> time it doesn't need to take because you can just use the in_use flag.
>>>
>>> I wouldn't worry about reusing clusters of other bitmaps. Of course we
>>> can do that later on in some optimization but not now.
>>>
>>> I just mean the basic case of some auto-loaded bitmap that is not being
>>> deleted while the VM is running and is just saved back to the image 
>>> file
>>> once it is closed. I don't expect that users will always consume all of
>>> the auto-loaded bitmaps while the VM is active...
>>>
>>>> Also, if I don't free them on open, I'll have to free them on 
>>>> remove of
>>>> bdrv dirty bitmap..
>>> Yes, so? That takes time, but that is something the user will probably
>>> expect. I wouldn't expect opening of the file to take that time.
>>>
>>> Overall, it doesn't matter time-wise whether you free the bitmap data
>>> when opening the image file or when the dirty bitmap is actually 
>>> deleted
>>> by the user or when the image file is closed. Just setting the single
>>> in_use flag for all of the auto-loaded bitmaps will basically not take
>>> any time.
>>>
>>> On the other hand, as soon as just a single auto-loaded bitmap survives
>>> a VM (or qemu-img) invocation, you will very, very likely safe at least
>>> some time because writing the bitmap to the disk can reuse at least 
>>> some
>>> of the existing clusters.
>> By the way, dealing with removal of bitmaps when they are deleted by the
>> user is also positive when it comes to migration or read-only disks that
>> are later reopened R/W: Currently, as far as I can see, you just keep
>> the auto-load bitmaps in the image if the image is part of an incoming
>> migration or opened read-only, but then you continue under the
>> assumption that they are removed from the image. That's not very good.
>
> You are right, I need to handle reopening more carefully. In my way, I 
> need to delete bitmaps when reopening R/W and in yours - set in_use bit.
>
>>
>> If you delete the bitmaps only when the user asks you to, then you can
>> just return an error that the bitmap cannot be removed at this time
>> because the image file is read-only or used in migration (and I don't
>> think the latter can even happen when it's the user who has requested
>> removal of the bitmap).
>>
>> Max
>>
>
> Ok, I'll rewrite this way, using 'in_use' bit and trying to avoid 
> allocate/free overhead.
>
>

Trying to reuse clusters of in_use bitmaps (including contiguous 
allocations for bitmap tables) will complicate things a lot.. We can use 
extra clusters of one in_use bitmap to save another one, the same is for 
bitmap tables. Extra clusters of old bitmap table (in case of resize) 
can be used for saving other bitmap data, etc..


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

26.10.2016 12:04, Vladimir Sementsov-Ogievskiy wrote:
> 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote:
>> 24.10.2016 20:18, Max Reitz wrote:
>>> On 24.10.2016 19:08, Max Reitz wrote:
>>>> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>>>>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>>>>> 21.10.2016 22:58, Max Reitz пишет:
>>>>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>>> This flag means that the bitmap is now in use by the software or
>>>>>>>>>> was not
>>>>>>>>>> successfully saved. In any way, with this flag set the bitmap 
>>>>>>>>>> data
>>>>>>>>>> must
>>>>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>>>>
>>>>>>>>>> With current implementation this flag is never set, as we 
>>>>>>>>>> just remove
>>>>>>>>>> bitmaps from the image after loading. But it defined in qcow2 
>>>>>>>>>> spec
>>>>>>>>>> and
>>>>>>>>>> must be handled. Also, it can be used in future, if async 
>>>>>>>>>> schemes of
>>>>>>>>>> bitmap loading/saving are implemented.
>>>>>>>>>>
>>>>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>>>>> ---
>>>>>>>>>>     block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>>>>     1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>>>>> Don't you want to make use of this flag? It would appear 
>>>>>>>>> useful to
>>>>>>>>> me if
>>>>>>>>> you just marked autoload bitmaps as in_use instead of deleting 
>>>>>>>>> them
>>>>>>>>> from
>>>>>>>>> the image when it's opened and then overwrite them when the 
>>>>>>>>> image is
>>>>>>>>> closed.
>>>>>>>> And what is the use of it?
>>>>>>> You don't need to free any bitmaps when opening the file, and 
>>>>>>> you don't
>>>>>>> need to allocate any new bitmap space when closing it.
>>>>>> As bitmaps are sparce in file, I need to allocate new space when
>>>>>> closing. Or free it...
>>>>>
>>>>> Is it a real problem to reallocate space in qcow2? If so, to minimaze
>>>>> allocations, I will have to make a list of clusters of in_use 
>>>>> bitmaps on
>>>>> close, and then save current bitmaps to these clusters (and allocated
>>>>> some additional clusters, or free extra clusters).
>>>> It's not a real problem, but it does take time, and I maintain that 
>>>> it's
>>>> time it doesn't need to take because you can just use the in_use flag.
>>>>
>>>> I wouldn't worry about reusing clusters of other bitmaps. Of course we
>>>> can do that later on in some optimization but not now.
>>>>
>>>> I just mean the basic case of some auto-loaded bitmap that is not 
>>>> being
>>>> deleted while the VM is running and is just saved back to the image 
>>>> file
>>>> once it is closed. I don't expect that users will always consume 
>>>> all of
>>>> the auto-loaded bitmaps while the VM is active...
>>>>
>>>>> Also, if I don't free them on open, I'll have to free them on 
>>>>> remove of
>>>>> bdrv dirty bitmap..
>>>> Yes, so? That takes time, but that is something the user will probably
>>>> expect. I wouldn't expect opening of the file to take that time.
>>>>
>>>> Overall, it doesn't matter time-wise whether you free the bitmap data
>>>> when opening the image file or when the dirty bitmap is actually 
>>>> deleted
>>>> by the user or when the image file is closed. Just setting the single
>>>> in_use flag for all of the auto-loaded bitmaps will basically not take
>>>> any time.
>>>>
>>>> On the other hand, as soon as just a single auto-loaded bitmap 
>>>> survives
>>>> a VM (or qemu-img) invocation, you will very, very likely safe at 
>>>> least
>>>> some time because writing the bitmap to the disk can reuse at least 
>>>> some
>>>> of the existing clusters.
>>> By the way, dealing with removal of bitmaps when they are deleted by 
>>> the
>>> user is also positive when it comes to migration or read-only disks 
>>> that
>>> are later reopened R/W: Currently, as far as I can see, you just keep
>>> the auto-load bitmaps in the image if the image is part of an incoming
>>> migration or opened read-only, but then you continue under the
>>> assumption that they are removed from the image. That's not very good.
>>
>> You are right, I need to handle reopening more carefully. In my way, 
>> I need to delete bitmaps when reopening R/W and in yours - set in_use 
>> bit.
>>
>>>
>>> If you delete the bitmaps only when the user asks you to, then you can
>>> just return an error that the bitmap cannot be removed at this time
>>> because the image file is read-only or used in migration (and I don't
>>> think the latter can even happen when it's the user who has requested
>>> removal of the bitmap).
>>>
>>> Max
>>>
>>
>> Ok, I'll rewrite this way, using 'in_use' bit and trying to avoid 
>> allocate/free overhead.
>>
>>
>
> Trying to reuse clusters of in_use bitmaps (including contiguous 
> allocations for bitmap tables) will complicate things a lot.. We can 
> use extra clusters of one in_use bitmap to save another one, the same 
> is for bitmap tables. Extra clusters of old bitmap table (in case of 
> resize) can be used for saving other bitmap data, etc..
>
>

A compromise strategy of reusing:

1. accumulate all data clusters of in_use bitmaps, which was loaded by 
this qcow2 instance (we will not touch other old in_use bitmaps) to 
free_clusters list
2. for each persistent BdrvDirtyBitmap:
        If there is corresponding in_use bitmap in the image, and its 
table size == needed table size (there was no resizes), then let's reuse 
bitmap table.
        else, move old bitmap table clusters to free_clusters and 
allocate new table.
3. for each persistent BdrvDirtyBitmap:
        For bitmap data clusters, take them from free_clusters list, and 
if it is empty - allocate new clusters.
4. free extra clusters from free_clusters list if any.


This strategy is not optimal, but not bad I thing. Is it ok for you? 
(I'm not sure that this all is not premature optimization, and may be 
true way is to just free all old staff and reallocate it, as I do.)


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

26.10.2016 12:21, Vladimir Sementsov-Ogievskiy wrote:
> 26.10.2016 12:04, Vladimir Sementsov-Ogievskiy wrote:
>> 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote:
>>> 24.10.2016 20:18, Max Reitz wrote:
>>>> On 24.10.2016 19:08, Max Reitz wrote:
>>>>> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>>>>>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>>>>>> 21.10.2016 22:58, Max Reitz пишет:
>>>>>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>>>> This flag means that the bitmap is now in use by the 
>>>>>>>>>>> software or
>>>>>>>>>>> was not
>>>>>>>>>>> successfully saved. In any way, with this flag set the 
>>>>>>>>>>> bitmap data
>>>>>>>>>>> must
>>>>>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>>>>>
>>>>>>>>>>> With current implementation this flag is never set, as we 
>>>>>>>>>>> just remove
>>>>>>>>>>> bitmaps from the image after loading. But it defined in 
>>>>>>>>>>> qcow2 spec
>>>>>>>>>>> and
>>>>>>>>>>> must be handled. Also, it can be used in future, if async 
>>>>>>>>>>> schemes of
>>>>>>>>>>> bitmap loading/saving are implemented.
>>>>>>>>>>>
>>>>>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>>>>>> ---
>>>>>>>>>>>     block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>>>>>     1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>>>>>> Don't you want to make use of this flag? It would appear 
>>>>>>>>>> useful to
>>>>>>>>>> me if
>>>>>>>>>> you just marked autoload bitmaps as in_use instead of 
>>>>>>>>>> deleting them
>>>>>>>>>> from
>>>>>>>>>> the image when it's opened and then overwrite them when the 
>>>>>>>>>> image is
>>>>>>>>>> closed.
>>>>>>>>> And what is the use of it?
>>>>>>>> You don't need to free any bitmaps when opening the file, and 
>>>>>>>> you don't
>>>>>>>> need to allocate any new bitmap space when closing it.
>>>>>>> As bitmaps are sparce in file, I need to allocate new space when
>>>>>>> closing. Or free it...
>>>>>>
>>>>>> Is it a real problem to reallocate space in qcow2? If so, to 
>>>>>> minimaze
>>>>>> allocations, I will have to make a list of clusters of in_use 
>>>>>> bitmaps on
>>>>>> close, and then save current bitmaps to these clusters (and 
>>>>>> allocated
>>>>>> some additional clusters, or free extra clusters).
>>>>> It's not a real problem, but it does take time, and I maintain 
>>>>> that it's
>>>>> time it doesn't need to take because you can just use the in_use 
>>>>> flag.
>>>>>
>>>>> I wouldn't worry about reusing clusters of other bitmaps. Of 
>>>>> course we
>>>>> can do that later on in some optimization but not now.
>>>>>
>>>>> I just mean the basic case of some auto-loaded bitmap that is not 
>>>>> being
>>>>> deleted while the VM is running and is just saved back to the 
>>>>> image file
>>>>> once it is closed. I don't expect that users will always consume 
>>>>> all of
>>>>> the auto-loaded bitmaps while the VM is active...
>>>>>
>>>>>> Also, if I don't free them on open, I'll have to free them on 
>>>>>> remove of
>>>>>> bdrv dirty bitmap..
>>>>> Yes, so? That takes time, but that is something the user will 
>>>>> probably
>>>>> expect. I wouldn't expect opening of the file to take that time.
>>>>>
>>>>> Overall, it doesn't matter time-wise whether you free the bitmap data
>>>>> when opening the image file or when the dirty bitmap is actually 
>>>>> deleted
>>>>> by the user or when the image file is closed. Just setting the single
>>>>> in_use flag for all of the auto-loaded bitmaps will basically not 
>>>>> take
>>>>> any time.
>>>>>
>>>>> On the other hand, as soon as just a single auto-loaded bitmap 
>>>>> survives
>>>>> a VM (or qemu-img) invocation, you will very, very likely safe at 
>>>>> least
>>>>> some time because writing the bitmap to the disk can reuse at 
>>>>> least some
>>>>> of the existing clusters.
>>>> By the way, dealing with removal of bitmaps when they are deleted 
>>>> by the
>>>> user is also positive when it comes to migration or read-only disks 
>>>> that
>>>> are later reopened R/W: Currently, as far as I can see, you just keep
>>>> the auto-load bitmaps in the image if the image is part of an incoming
>>>> migration or opened read-only, but then you continue under the
>>>> assumption that they are removed from the image. That's not very good.
>>>
>>> You are right, I need to handle reopening more carefully. In my way, 
>>> I need to delete bitmaps when reopening R/W and in yours - set 
>>> in_use bit.
>>>
>>>>
>>>> If you delete the bitmaps only when the user asks you to, then you can
>>>> just return an error that the bitmap cannot be removed at this time
>>>> because the image file is read-only or used in migration (and I don't
>>>> think the latter can even happen when it's the user who has requested
>>>> removal of the bitmap).
>>>>
>>>> Max
>>>>
>>>
>>> Ok, I'll rewrite this way, using 'in_use' bit and trying to avoid 
>>> allocate/free overhead.
>>>
>>>
>>
>> Trying to reuse clusters of in_use bitmaps (including contiguous 
>> allocations for bitmap tables) will complicate things a lot.. We can 
>> use extra clusters of one in_use bitmap to save another one, the same 
>> is for bitmap tables. Extra clusters of old bitmap table (in case of 
>> resize) can be used for saving other bitmap data, etc..
>>
>>
>
> A compromise strategy of reusing:
>
> 1. accumulate all data clusters of in_use bitmaps, which was loaded by 
> this qcow2 instance (we will not touch other old in_use bitmaps) to 
> free_clusters list
> 2. for each persistent BdrvDirtyBitmap:
>        If there is corresponding in_use bitmap in the image, and its 
> table size == needed table size (there was no resizes), then let's 
> reuse bitmap table.
>        else, move old bitmap table clusters to free_clusters and 
> allocate new table.
> 3. for each persistent BdrvDirtyBitmap:
>        For bitmap data clusters, take them from free_clusters list, 
> and if it is empty - allocate new clusters.
> 4. free extra clusters from free_clusters list if any.
>
>
> This strategy is not optimal, but not bad I thing. Is it ok for you? 
> (I'm not sure that this all is not premature optimization, and may be 
> true way is to just free all old staff and reallocate it, as I do.)
>
>

With something like this it is not trivial to maintain consistency in 
qcow2 image in case of fail on reusing (we can get double-linked 
clusters, or dead links in the image on disk).. Simple solution will be 
to remove all in_use bitmaps from header extension, or reallocate for 
them bitmap tables with all zeroes. So, finally:

target: we want to reuse clusters of in_use bitmaps, which was loaded by 
current qcow2 driver (let's call them our in_use bitmaps.

sequence:
1. save bitmap directory in memory
2. remove our in_use bitmaps from bitmap directory in disk. from this 
point, in case of fail we will finish with some leaking clusters and no 
corruptions.
3. free_clusters list = all data clusters of our in_use bitmaps
4. for each our in_use bitmap: If table size is not appropriate move 
bitmap table clusters to free_clusters
5. save bitmaps, using free_clusters list for data clusters, old 
existing bitmap tables and allocating new space when needed.
6. free extra clusters from free_clusters list if any


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

26.10.2016 15:13, Vladimir Sementsov-Ogievskiy wrote:
> 26.10.2016 12:21, Vladimir Sementsov-Ogievskiy wrote:
>> 26.10.2016 12:04, Vladimir Sementsov-Ogievskiy wrote:
>>> 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote:
>>>> 24.10.2016 20:18, Max Reitz wrote:
>>>>> On 24.10.2016 19:08, Max Reitz wrote:
>>>>>> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>>>>>>> 21.10.2016 22:58, Max Reitz пишет:
>>>>>>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>>>>> This flag means that the bitmap is now in use by the 
>>>>>>>>>>>> software or
>>>>>>>>>>>> was not
>>>>>>>>>>>> successfully saved. In any way, with this flag set the 
>>>>>>>>>>>> bitmap data
>>>>>>>>>>>> must
>>>>>>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>>>>>>
>>>>>>>>>>>> With current implementation this flag is never set, as we 
>>>>>>>>>>>> just remove
>>>>>>>>>>>> bitmaps from the image after loading. But it defined in 
>>>>>>>>>>>> qcow2 spec
>>>>>>>>>>>> and
>>>>>>>>>>>> must be handled. Also, it can be used in future, if async 
>>>>>>>>>>>> schemes of
>>>>>>>>>>>> bitmap loading/saving are implemented.
>>>>>>>>>>>>
>>>>>>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>>>>>>> ---
>>>>>>>>>>>>     block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>>>>>>     1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>>>>>>> Don't you want to make use of this flag? It would appear 
>>>>>>>>>>> useful to
>>>>>>>>>>> me if
>>>>>>>>>>> you just marked autoload bitmaps as in_use instead of 
>>>>>>>>>>> deleting them
>>>>>>>>>>> from
>>>>>>>>>>> the image when it's opened and then overwrite them when the 
>>>>>>>>>>> image is
>>>>>>>>>>> closed.
>>>>>>>>>> And what is the use of it?
>>>>>>>>> You don't need to free any bitmaps when opening the file, and 
>>>>>>>>> you don't
>>>>>>>>> need to allocate any new bitmap space when closing it.
>>>>>>>> As bitmaps are sparce in file, I need to allocate new space when
>>>>>>>> closing. Or free it...
>>>>>>>
>>>>>>> Is it a real problem to reallocate space in qcow2? If so, to 
>>>>>>> minimaze
>>>>>>> allocations, I will have to make a list of clusters of in_use 
>>>>>>> bitmaps on
>>>>>>> close, and then save current bitmaps to these clusters (and 
>>>>>>> allocated
>>>>>>> some additional clusters, or free extra clusters).
>>>>>> It's not a real problem, but it does take time, and I maintain 
>>>>>> that it's
>>>>>> time it doesn't need to take because you can just use the in_use 
>>>>>> flag.
>>>>>>
>>>>>> I wouldn't worry about reusing clusters of other bitmaps. Of 
>>>>>> course we
>>>>>> can do that later on in some optimization but not now.
>>>>>>
>>>>>> I just mean the basic case of some auto-loaded bitmap that is not 
>>>>>> being
>>>>>> deleted while the VM is running and is just saved back to the 
>>>>>> image file
>>>>>> once it is closed. I don't expect that users will always consume 
>>>>>> all of
>>>>>> the auto-loaded bitmaps while the VM is active...
>>>>>>
>>>>>>> Also, if I don't free them on open, I'll have to free them on 
>>>>>>> remove of
>>>>>>> bdrv dirty bitmap..
>>>>>> Yes, so? That takes time, but that is something the user will 
>>>>>> probably
>>>>>> expect. I wouldn't expect opening of the file to take that time.
>>>>>>
>>>>>> Overall, it doesn't matter time-wise whether you free the bitmap 
>>>>>> data
>>>>>> when opening the image file or when the dirty bitmap is actually 
>>>>>> deleted
>>>>>> by the user or when the image file is closed. Just setting the 
>>>>>> single
>>>>>> in_use flag for all of the auto-loaded bitmaps will basically not 
>>>>>> take
>>>>>> any time.
>>>>>>
>>>>>> On the other hand, as soon as just a single auto-loaded bitmap 
>>>>>> survives
>>>>>> a VM (or qemu-img) invocation, you will very, very likely safe at 
>>>>>> least
>>>>>> some time because writing the bitmap to the disk can reuse at 
>>>>>> least some
>>>>>> of the existing clusters.
>>>>> By the way, dealing with removal of bitmaps when they are deleted 
>>>>> by the
>>>>> user is also positive when it comes to migration or read-only 
>>>>> disks that
>>>>> are later reopened R/W: Currently, as far as I can see, you just keep
>>>>> the auto-load bitmaps in the image if the image is part of an 
>>>>> incoming
>>>>> migration or opened read-only, but then you continue under the
>>>>> assumption that they are removed from the image. That's not very 
>>>>> good.
>>>>
>>>> You are right, I need to handle reopening more carefully. In my 
>>>> way, I need to delete bitmaps when reopening R/W and in yours - set 
>>>> in_use bit.
>>>>
>>>>>
>>>>> If you delete the bitmaps only when the user asks you to, then you 
>>>>> can
>>>>> just return an error that the bitmap cannot be removed at this time
>>>>> because the image file is read-only or used in migration (and I don't
>>>>> think the latter can even happen when it's the user who has requested
>>>>> removal of the bitmap).
>>>>>
>>>>> Max
>>>>>
>>>>
>>>> Ok, I'll rewrite this way, using 'in_use' bit and trying to avoid 
>>>> allocate/free overhead.
>>>>
>>>>
>>>
>>> Trying to reuse clusters of in_use bitmaps (including contiguous 
>>> allocations for bitmap tables) will complicate things a lot.. We can 
>>> use extra clusters of one in_use bitmap to save another one, the 
>>> same is for bitmap tables. Extra clusters of old bitmap table (in 
>>> case of resize) can be used for saving other bitmap data, etc..
>>>
>>>
>>
>> A compromise strategy of reusing:
>>
>> 1. accumulate all data clusters of in_use bitmaps, which was loaded 
>> by this qcow2 instance (we will not touch other old in_use bitmaps) 
>> to free_clusters list
>> 2. for each persistent BdrvDirtyBitmap:
>>        If there is corresponding in_use bitmap in the image, and its 
>> table size == needed table size (there was no resizes), then let's 
>> reuse bitmap table.
>>        else, move old bitmap table clusters to free_clusters and 
>> allocate new table.
>> 3. for each persistent BdrvDirtyBitmap:
>>        For bitmap data clusters, take them from free_clusters list, 
>> and if it is empty - allocate new clusters.
>> 4. free extra clusters from free_clusters list if any.
>>
>>
>> This strategy is not optimal, but not bad I thing. Is it ok for you? 
>> (I'm not sure that this all is not premature optimization, and may be 
>> true way is to just free all old staff and reallocate it, as I do.)
>>
>>
>
> With something like this it is not trivial to maintain consistency in 
> qcow2 image in case of fail on reusing (we can get double-linked 
> clusters, or dead links in the image on disk).. Simple solution will 
> be to remove all in_use bitmaps from header extension, or reallocate 
> for them bitmap tables with all zeroes. So, finally:
>
> target: we want to reuse clusters of in_use bitmaps, which was loaded 
> by current qcow2 driver (let's call them our in_use bitmaps.
>
> sequence:
> 1. save bitmap directory in memory
> 2. remove our in_use bitmaps from bitmap directory in disk. from this 
> point, in case of fail we will finish with some leaking clusters and 
> no corruptions.
> 3. free_clusters list = all data clusters of our in_use bitmaps
> 4. for each our in_use bitmap: If table size is not appropriate move 
> bitmap table clusters to free_clusters

better is to free it, to not deceiveqcow2 allocator.. Can I allocate 
several clusters by alloc(size), then assume that I have (size + 
cluster_size - 1)/cluster_size clusters and that I can free these 
clusters separately?

> 5. save bitmaps, using free_clusters list for data clusters, old 
> existing bitmap tables and allocating new space when needed.
> 6. free extra clusters from free_clusters list if any
>
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 7177 bytes --]

On 26.10.2016 11:04, Vladimir Sementsov-Ogievskiy wrote:
> 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote:
>> 24.10.2016 20:18, Max Reitz wrote:
>>> On 24.10.2016 19:08, Max Reitz wrote:
>>>> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>>>>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>>>>> 21.10.2016 22:58, Max Reitz пишет:
>>>>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>>> This flag means that the bitmap is now in use by the software or
>>>>>>>>>> was not
>>>>>>>>>> successfully saved. In any way, with this flag set the bitmap
>>>>>>>>>> data
>>>>>>>>>> must
>>>>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>>>>
>>>>>>>>>> With current implementation this flag is never set, as we just
>>>>>>>>>> remove
>>>>>>>>>> bitmaps from the image after loading. But it defined in qcow2
>>>>>>>>>> spec
>>>>>>>>>> and
>>>>>>>>>> must be handled. Also, it can be used in future, if async
>>>>>>>>>> schemes of
>>>>>>>>>> bitmap loading/saving are implemented.
>>>>>>>>>>
>>>>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>>>>> ---
>>>>>>>>>>     block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>>>>     1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>>>>> Don't you want to make use of this flag? It would appear useful to
>>>>>>>>> me if
>>>>>>>>> you just marked autoload bitmaps as in_use instead of deleting
>>>>>>>>> them
>>>>>>>>> from
>>>>>>>>> the image when it's opened and then overwrite them when the
>>>>>>>>> image is
>>>>>>>>> closed.
>>>>>>>> And what is the use of it?
>>>>>>> You don't need to free any bitmaps when opening the file, and you
>>>>>>> don't
>>>>>>> need to allocate any new bitmap space when closing it.
>>>>>> As bitmaps are sparce in file, I need to allocate new space when
>>>>>> closing. Or free it...
>>>>>
>>>>> Is it a real problem to reallocate space in qcow2? If so, to minimaze
>>>>> allocations, I will have to make a list of clusters of in_use
>>>>> bitmaps on
>>>>> close, and then save current bitmaps to these clusters (and allocated
>>>>> some additional clusters, or free extra clusters).
>>>> It's not a real problem, but it does take time, and I maintain that
>>>> it's
>>>> time it doesn't need to take because you can just use the in_use flag.
>>>>
>>>> I wouldn't worry about reusing clusters of other bitmaps. Of course we
>>>> can do that later on in some optimization but not now.
>>>>
>>>> I just mean the basic case of some auto-loaded bitmap that is not being
>>>> deleted while the VM is running and is just saved back to the image
>>>> file
>>>> once it is closed. I don't expect that users will always consume all of
>>>> the auto-loaded bitmaps while the VM is active...
>>>>
>>>>> Also, if I don't free them on open, I'll have to free them on
>>>>> remove of
>>>>> bdrv dirty bitmap..
>>>> Yes, so? That takes time, but that is something the user will probably
>>>> expect. I wouldn't expect opening of the file to take that time.
>>>>
>>>> Overall, it doesn't matter time-wise whether you free the bitmap data
>>>> when opening the image file or when the dirty bitmap is actually
>>>> deleted
>>>> by the user or when the image file is closed. Just setting the single
>>>> in_use flag for all of the auto-loaded bitmaps will basically not take
>>>> any time.
>>>>
>>>> On the other hand, as soon as just a single auto-loaded bitmap survives
>>>> a VM (or qemu-img) invocation, you will very, very likely safe at least
>>>> some time because writing the bitmap to the disk can reuse at least
>>>> some
>>>> of the existing clusters.
>>> By the way, dealing with removal of bitmaps when they are deleted by the
>>> user is also positive when it comes to migration or read-only disks that
>>> are later reopened R/W: Currently, as far as I can see, you just keep
>>> the auto-load bitmaps in the image if the image is part of an incoming
>>> migration or opened read-only, but then you continue under the
>>> assumption that they are removed from the image. That's not very good.
>>
>> You are right, I need to handle reopening more carefully. In my way, I
>> need to delete bitmaps when reopening R/W and in yours - set in_use bit.
>>
>>>
>>> If you delete the bitmaps only when the user asks you to, then you can
>>> just return an error that the bitmap cannot be removed at this time
>>> because the image file is read-only or used in migration (and I don't
>>> think the latter can even happen when it's the user who has requested
>>> removal of the bitmap).
>>>
>>> Max
>>>
>>
>> Ok, I'll rewrite this way, using 'in_use' bit and trying to avoid
>> allocate/free overhead.
>>
>>
> 
> Trying to reuse clusters of in_use bitmaps (including contiguous
> allocations for bitmap tables) will complicate things a lot.. We can use
> extra clusters of one in_use bitmap to save another one, the same is for
> bitmap tables. Extra clusters of old bitmap table (in case of resize)
> can be used for saving other bitmap data, etc..

I feel like you're trying to optimize too much. When rewriting the
bitmap data, reuse only the clusters you have to write to anyway.

Say the bitmap table looks like this when you open the image:

[sparse 0, cluster 42, cluster 45, sparse 0, cluster 46]

("sparse 0" means no cluster is allocated and the bitmap table entry is
0, thus creating a sparse bitmap where that part is supposed to be read
as 0.)


Now during runtime, you set some bits, maybe you even clear the whole
bitmap at some point because you're doing a backup, and then you set
some other bits, so when you close the image, the bitmap table would
have to look like this:

[sparse 0, sparse 0, data, data, data]

Now, when you want to save that bitmap data, you just walk through the
existing bitmap table. The first entry is a sparse 0, and the bitmap is
sparse there, too, so that can stay as it is.

The second entry points to data but the bitmap is now sparse. You can
then free the cluster (cluster 42) and write "sparse 0" into the bitmap
table entry.

The third entry points to data and the bitmap contains data there, so
you just write the data to cluster 45.

The fourth entry is sparse but your bitmap contains data. So now you
need to allocate a cluster, maybe that will cluster 67, and write your
data there, and make the bitmap table entry point there.

The fifth entry finally is handled just like the third entry.

So afterwards, your bitmap table would look like this:

[sparse 0, sparse 0, cluster 45, cluster 67, cluster 46]


It sounds as if you're trying to reuse all the clusters, i.e. you'd
ideally get:

[sparse 0, sparse 0, cluster 45, cluster 42, cluster 46]

May be possible, but I really wouldn't worry about that. I'd consider it
much too difficult. Just reuse the existing bitmap table.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Vladimir Sementsov-Ogievskiy]

25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote:
> 24.10.2016 20:18, Max Reitz wrote:
>> On 24.10.2016 19:08, Max Reitz wrote:
>>> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>>>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>>>> 21.10.2016 22:58, Max Reitz пишет:
>>>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>> This flag means that the bitmap is now in use by the software or
>>>>>>>>> was not
>>>>>>>>> successfully saved. In any way, with this flag set the bitmap 
>>>>>>>>> data
>>>>>>>>> must
>>>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>>>
>>>>>>>>> With current implementation this flag is never set, as we just 
>>>>>>>>> remove
>>>>>>>>> bitmaps from the image after loading. But it defined in qcow2 
>>>>>>>>> spec
>>>>>>>>> and
>>>>>>>>> must be handled. Also, it can be used in future, if async 
>>>>>>>>> schemes of
>>>>>>>>> bitmap loading/saving are implemented.
>>>>>>>>>
>>>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>>>> ---
>>>>>>>>>     block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>>>     1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>>>> Don't you want to make use of this flag? It would appear useful to
>>>>>>>> me if
>>>>>>>> you just marked autoload bitmaps as in_use instead of deleting 
>>>>>>>> them
>>>>>>>> from
>>>>>>>> the image when it's opened and then overwrite them when the 
>>>>>>>> image is
>>>>>>>> closed.
>>>>>>> And what is the use of it?
>>>>>> You don't need to free any bitmaps when opening the file, and you 
>>>>>> don't
>>>>>> need to allocate any new bitmap space when closing it.
>>>>> As bitmaps are sparce in file, I need to allocate new space when
>>>>> closing. Or free it...
>>>>
>>>> Is it a real problem to reallocate space in qcow2? If so, to minimaze
>>>> allocations, I will have to make a list of clusters of in_use 
>>>> bitmaps on
>>>> close, and then save current bitmaps to these clusters (and allocated
>>>> some additional clusters, or free extra clusters).
>>> It's not a real problem, but it does take time, and I maintain that 
>>> it's
>>> time it doesn't need to take because you can just use the in_use flag.
>>>
>>> I wouldn't worry about reusing clusters of other bitmaps. Of course we
>>> can do that later on in some optimization but not now.
>>>
>>> I just mean the basic case of some auto-loaded bitmap that is not being
>>> deleted while the VM is running and is just saved back to the image 
>>> file
>>> once it is closed. I don't expect that users will always consume all of
>>> the auto-loaded bitmaps while the VM is active...
>>>
>>>> Also, if I don't free them on open, I'll have to free them on 
>>>> remove of
>>>> bdrv dirty bitmap..
>>> Yes, so? That takes time, but that is something the user will probably
>>> expect. I wouldn't expect opening of the file to take that time.
>>>
>>> Overall, it doesn't matter time-wise whether you free the bitmap data
>>> when opening the image file or when the dirty bitmap is actually 
>>> deleted
>>> by the user or when the image file is closed. Just setting the single
>>> in_use flag for all of the auto-loaded bitmaps will basically not take
>>> any time.
>>>
>>> On the other hand, as soon as just a single auto-loaded bitmap survives
>>> a VM (or qemu-img) invocation, you will very, very likely safe at least
>>> some time because writing the bitmap to the disk can reuse at least 
>>> some
>>> of the existing clusters.
>> By the way, dealing with removal of bitmaps when they are deleted by the
>> user is also positive when it comes to migration or read-only disks that
>> are later reopened R/W: Currently, as far as I can see, you just keep
>> the auto-load bitmaps in the image if the image is part of an incoming
>> migration or opened read-only, but then you continue under the
>> assumption that they are removed from the image. That's not very good.
>
> You are right, I need to handle reopening more carefully. In my way, I 
> need to delete bitmaps when reopening R/W and in yours - set in_use bit.

Now I think, that loading auto_loading bitmaps in qcow2_open is wrong. I 
should load them only in bdrv_open, to avoid reloading bitmaps on 
drv->bdrv_open/drv->bdrv_close (they are called from bdrv_snapshot_goto).

So, it would be like this:

on bdrv_open, after drv->bdrv_open call drv->load_autoloading_bitmaps,
which will load bitmaps, mark them in_use in the image (if it is 
writable), create corresponding BdrvDirtyBitmaps. If there _any_ 
conflicts with existing BdrvDirtyBitmaps then fail.

on bdrv_close, before drv->bdrv_close call drv->store_persitstent_bitmaps,
which will store persistent bitmaps, set in_use to false and _delete_ 
corresponding BdrvDirtyBitmaps.

Also, in qcow2_reopen_prepare in case of changing write-ability from 
false to true we need to mark corresponding bitmaps in the image as 
in_use.. And something like this for incoming migration too.

qcow2_open will only load header extension fields to bs->opaque, and 
check these fields. It will not load bitmap directory.


>
>>
>> If you delete the bitmaps only when the user asks you to, then you can
>> just return an error that the bitmap cannot be removed at this time
>> because the image file is read-only or used in migration (and I don't
>> think the latter can even happen when it's the user who has requested
>> removal of the bitmap).
>>
>> Max
>>
>
> Ok, I'll rewrite this way, using 'in_use' bit and trying to avoid 
> allocate/free overhead.
>
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

[Max Reitz]

[-- Attachment #1: Type: text/plain, Size: 5641 bytes --]

On 07.11.2016 17:12, Vladimir Sementsov-Ogievskiy wrote:
> 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote:
>> 24.10.2016 20:18, Max Reitz wrote:
>>> On 24.10.2016 19:08, Max Reitz wrote:
>>>> On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:
>>>>> 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:
>>>>>> 21.10.2016 22:58, Max Reitz пишет:
>>>>>>> On 21.10.2016 17:34, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>> 07.10.2016 22:44, Max Reitz пишет:
>>>>>>>>> On 30.09.2016 12:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>>> This flag means that the bitmap is now in use by the software or
>>>>>>>>>> was not
>>>>>>>>>> successfully saved. In any way, with this flag set the bitmap
>>>>>>>>>> data
>>>>>>>>>> must
>>>>>>>>>> be considered inconsistent and should not be loaded.
>>>>>>>>>>
>>>>>>>>>> With current implementation this flag is never set, as we just
>>>>>>>>>> remove
>>>>>>>>>> bitmaps from the image after loading. But it defined in qcow2
>>>>>>>>>> spec
>>>>>>>>>> and
>>>>>>>>>> must be handled. Also, it can be used in future, if async
>>>>>>>>>> schemes of
>>>>>>>>>> bitmap loading/saving are implemented.
>>>>>>>>>>
>>>>>>>>>> We also remove in-use bitmaps from the image on open.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>>>>>>>> <vsementsov@virtuozzo.com>
>>>>>>>>>> ---
>>>>>>>>>>     block/qcow2-bitmap.c | 17 ++++++++++++++++-
>>>>>>>>>>     1 file changed, 16 insertions(+), 1 deletion(-)
>>>>>>>>> Don't you want to make use of this flag? It would appear useful to
>>>>>>>>> me if
>>>>>>>>> you just marked autoload bitmaps as in_use instead of deleting
>>>>>>>>> them
>>>>>>>>> from
>>>>>>>>> the image when it's opened and then overwrite them when the
>>>>>>>>> image is
>>>>>>>>> closed.
>>>>>>>> And what is the use of it?
>>>>>>> You don't need to free any bitmaps when opening the file, and you
>>>>>>> don't
>>>>>>> need to allocate any new bitmap space when closing it.
>>>>>> As bitmaps are sparce in file, I need to allocate new space when
>>>>>> closing. Or free it...
>>>>>
>>>>> Is it a real problem to reallocate space in qcow2? If so, to minimaze
>>>>> allocations, I will have to make a list of clusters of in_use
>>>>> bitmaps on
>>>>> close, and then save current bitmaps to these clusters (and allocated
>>>>> some additional clusters, or free extra clusters).
>>>> It's not a real problem, but it does take time, and I maintain that
>>>> it's
>>>> time it doesn't need to take because you can just use the in_use flag.
>>>>
>>>> I wouldn't worry about reusing clusters of other bitmaps. Of course we
>>>> can do that later on in some optimization but not now.
>>>>
>>>> I just mean the basic case of some auto-loaded bitmap that is not being
>>>> deleted while the VM is running and is just saved back to the image
>>>> file
>>>> once it is closed. I don't expect that users will always consume all of
>>>> the auto-loaded bitmaps while the VM is active...
>>>>
>>>>> Also, if I don't free them on open, I'll have to free them on
>>>>> remove of
>>>>> bdrv dirty bitmap..
>>>> Yes, so? That takes time, but that is something the user will probably
>>>> expect. I wouldn't expect opening of the file to take that time.
>>>>
>>>> Overall, it doesn't matter time-wise whether you free the bitmap data
>>>> when opening the image file or when the dirty bitmap is actually
>>>> deleted
>>>> by the user or when the image file is closed. Just setting the single
>>>> in_use flag for all of the auto-loaded bitmaps will basically not take
>>>> any time.
>>>>
>>>> On the other hand, as soon as just a single auto-loaded bitmap survives
>>>> a VM (or qemu-img) invocation, you will very, very likely safe at least
>>>> some time because writing the bitmap to the disk can reuse at least
>>>> some
>>>> of the existing clusters.
>>> By the way, dealing with removal of bitmaps when they are deleted by the
>>> user is also positive when it comes to migration or read-only disks that
>>> are later reopened R/W: Currently, as far as I can see, you just keep
>>> the auto-load bitmaps in the image if the image is part of an incoming
>>> migration or opened read-only, but then you continue under the
>>> assumption that they are removed from the image. That's not very good.
>>
>> You are right, I need to handle reopening more carefully. In my way, I
>> need to delete bitmaps when reopening R/W and in yours - set in_use bit.
> 
> Now I think, that loading auto_loading bitmaps in qcow2_open is wrong. I
> should load them only in bdrv_open, to avoid reloading bitmaps on
> drv->bdrv_open/drv->bdrv_close (they are called from bdrv_snapshot_goto).
> 
> So, it would be like this:
> 
> on bdrv_open, after drv->bdrv_open call drv->load_autoloading_bitmaps,
> which will load bitmaps, mark them in_use in the image (if it is
> writable), create corresponding BdrvDirtyBitmaps. If there _any_
> conflicts with existing BdrvDirtyBitmaps then fail.
> 
> on bdrv_close, before drv->bdrv_close call drv->store_persitstent_bitmaps,
> which will store persistent bitmaps, set in_use to false and _delete_
> corresponding BdrvDirtyBitmaps.

Sounds good.

> Also, in qcow2_reopen_prepare in case of changing write-ability from
> false to true we need to mark corresponding bitmaps in the image as
> in_use.. And something like this for incoming migration too.

Right.

> qcow2_open will only load header extension fields to bs->opaque, and
> check these fields. It will not load bitmap directory.

Yes, that sounds good.

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 480 bytes --]

Re: [Qemu-devel] [PATCH 21/22] specs/qcow2: fix bitmap granularity qemu-specific note

[Vladimir Sementsov-Ogievskiy]

07.10.2016 23:18, Eric Blake wrote:
> On 09/30/2016 05:53 AM, Vladimir Sementsov-Ogievskiy wrote:
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   docs/specs/qcow2.txt | 3 +--
>>   1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
>> index 80cdfd0..d3e292f 100644
>> --- a/docs/specs/qcow2.txt
>> +++ b/docs/specs/qcow2.txt
>> @@ -472,8 +472,7 @@ Structure of a bitmap directory entry:
>>                17:    granularity_bits
>>                       Granularity bits. Valid values: 0 - 63.
> Can we really theoretically go as low as 0, or are we constrained by the
> fact that cluster_bits must be at least 9?

Bitmaps in qcow2 are abstract, they are not necessary related to 
BdrvDirtyBitmaps.. We do not know how the will be used in future, so 
there no reasons for hard restriction. However values near zero seems 
not very useful. I do not think we should change this. Note about Qemu 
is enough.


>
>>   
>> -                    Note: Qemu currently doesn't support granularity_bits
>> -                    greater than 31.
>> +                    Note: Qemu currently support only values 9 - 31.
> s/support/supports/
>
>>   
>>                       Granularity is calculated as
>>                           granularity = 1 << granularity_bits
>>


-- 
Best regards,
Vladimir