All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v2 0/6] external backup api
@ 2016-01-30 10:56 Vladimir Sementsov-Ogievskiy
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 1/6] block dirty bitmap: add next_zero function Vladimir Sementsov-Ogievskiy
                   ` (6 more replies)
  0 siblings, 7 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-01-30 10:56 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, Vladimir Sementsov-Ogievskiy, famz, den, armbru, jsnow

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

There are three things are done:
- add query-block-dirty-bitmap-ranges qmp command
- add qmp commands for dirty-bitmap functions: create_successor, abdicate,
  reclaim.
- make create-successor command transaction-able

Then, external backup should be done like this:

1.  qmp transaction {
        external-snapshot
        bitmap-create-successor
    }

2.  qmp query frozen bitmap, not acquiring aio context.

3.  do external backup, using snapshot and bitmap


4.  if (success backup)
        qmp bitmap-abdicate
    else
        qmp bitmap-reclaime

5.  qmp merge snapshot


v2: a lot of additions and changes, no sense to compare with v1


Vladimir Sementsov-Ogievskiy (6):
  block dirty bitmap: add next_zero function
  qmp: add query-block-dirty-bitmap-ranges
  iotests: test query-block-dirty-bitmap-ranges
  qapi: add qmp commands for some dirty bitmap functions
  qapi: make block-dirty-bitmap-create-successor transaction-able
  iotests: test external backup api

 block/dirty-bitmap.c         |  60 ++++++++++++++++++++++
 blockdev.c                   | 113 +++++++++++++++++++++++++++++++++++++++++
 include/block/dirty-bitmap.h |   9 ++++
 include/qemu/hbitmap.h       |   8 +++
 qapi-schema.json             |   4 +-
 qapi/block-core.json         |  90 +++++++++++++++++++++++++++++++++
 qmp-commands.hx              | 118 +++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/150       |  88 ++++++++++++++++++++++++++++++++
 tests/qemu-iotests/150.out   |  21 ++++++++
 tests/qemu-iotests/151       |  77 ++++++++++++++++++++++++++++
 tests/qemu-iotests/151.out   |   7 +++
 tests/qemu-iotests/group     |   2 +
 util/hbitmap.c               |  26 ++++++++++
 13 files changed, 622 insertions(+), 1 deletion(-)
 create mode 100755 tests/qemu-iotests/150
 create mode 100644 tests/qemu-iotests/150.out
 create mode 100755 tests/qemu-iotests/151
 create mode 100644 tests/qemu-iotests/151.out

-- 
1.8.3.1

^ permalink raw reply	[flat|nested] 56+ messages in thread

* [Qemu-devel] [PATCH 1/6] block dirty bitmap: add next_zero function
  2016-01-30 10:56 [Qemu-devel] [PATCH v2 0/6] external backup api Vladimir Sementsov-Ogievskiy
@ 2016-01-30 10:56 ` Vladimir Sementsov-Ogievskiy
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges Vladimir Sementsov-Ogievskiy
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-01-30 10:56 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, Vladimir Sementsov-Ogievskiy, famz, den, armbru, jsnow

The function searches for next zero bit (corresponding to next not-dirty
bit).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/dirty-bitmap.c         |  5 +++++
 include/block/dirty-bitmap.h |  2 ++
 include/qemu/hbitmap.h       |  8 ++++++++
 util/hbitmap.c               | 26 ++++++++++++++++++++++++++
 4 files changed, 41 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 919ce10..6da27d9 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -490,3 +490,8 @@ int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap)
 {
     return hbitmap_count(bitmap->bitmap);
 }
+
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start)
+{
+    return hbitmap_next_zero(bitmap->bitmap, start);
+}
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 6ba5bec..0033942 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -68,4 +68,6 @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
                                           bool finish);
 void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
 
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start);
+
 #endif
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 00dbb60..852e423 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -259,6 +259,14 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
  */
 unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi);
 
+/* hbitmap_next_zero:
+ * @hb: The HBitmap to operate on
+ * @start: The bit to start from.
+ *
+ * Find next not dirty bit.
+ */
+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start);
+
 /* hbitmap_create_meta:
  * Create a "meta" hbitmap to track dirtiness of the bits in this HBitmap.
  * The caller owns the created bitmap and must call hbitmap_free_meta(hb) to
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 1e49ab7..bcddca9 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -169,6 +169,32 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first)
     }
 }
 
+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start)
+{
+    size_t pos = (start >> hb->granularity) >> BITS_PER_LEVEL;
+    unsigned long *last_lev = hb->levels[HBITMAP_LEVELS - 1];
+    uint64_t sz = hb->sizes[HBITMAP_LEVELS - 1];
+    unsigned long cur = last_lev[pos];
+    unsigned start_bit_offset =
+            (start >> hb->granularity) & (BITS_PER_LONG - 1);
+    cur |= (1UL << start_bit_offset) - 1;
+
+    if (cur == (unsigned long)-1) {
+        do {
+            pos++;
+        } while (pos < sz && last_lev[pos] == (unsigned long)-1);
+
+        if (pos >= sz) {
+            return -1;
+        }
+
+        cur = last_lev[pos];
+    }
+
+    return MIN((pos << BITS_PER_LEVEL) + ctol(cur), hb->size - 1)
+            << hb->granularity;
+}
+
 bool hbitmap_empty(const HBitmap *hb)
 {
     return hb->count == 0;
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges
  2016-01-30 10:56 [Qemu-devel] [PATCH v2 0/6] external backup api Vladimir Sementsov-Ogievskiy
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 1/6] block dirty bitmap: add next_zero function Vladimir Sementsov-Ogievskiy
@ 2016-01-30 10:56 ` Vladimir Sementsov-Ogievskiy
  2016-02-10 10:08   ` Stefan Hajnoczi
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 3/6] iotests: test query-block-dirty-bitmap-ranges Vladimir Sementsov-Ogievskiy
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-01-30 10:56 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, Vladimir Sementsov-Ogievskiy, famz, den, armbru, jsnow

Add qmp command to query dirty bitmap contents. This is needed for
external backup.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/dirty-bitmap.c         | 55 +++++++++++++++++++++++++++++++++++++++
 blockdev.c                   | 62 ++++++++++++++++++++++++++++++++++++++++++++
 include/block/dirty-bitmap.h |  7 +++++
 qapi/block-core.json         | 54 ++++++++++++++++++++++++++++++++++++++
 qmp-commands.hx              | 33 +++++++++++++++++++++++
 5 files changed, 211 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 6da27d9..4c108e4 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -495,3 +495,58 @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start)
 {
     return hbitmap_next_zero(bitmap->bitmap, start);
 }
+
+/* bdrv_query_dirty_bitmap_ranges
+ * start, count and resulting ranges are all in bytes
+ */
+BlockDirtyRangeList *bdrv_query_dirty_bitmap_ranges(BlockDriverState *bs,
+        BdrvDirtyBitmap *bitmap, uint64_t start, uint64_t count,
+        uint32_t max_ranges)
+{
+    BlockDirtyRangeList *list = NULL, *last_entry = NULL;
+    BlockDirtyRangeList **plist = &list;
+    uint64_t begin, end;
+    uint64_t start_sector = start >> BDRV_SECTOR_BITS;
+    uint64_t last_sector = (start + count - 1) >> BDRV_SECTOR_BITS;
+    uint32_t nb_ranges = 0;
+
+    BdrvDirtyBitmapIter *it = bdrv_dirty_iter_new(bitmap, start_sector);
+
+    while ((begin = bdrv_dirty_iter_next(it)) != -1 && nb_ranges < max_ranges) {
+        BlockDirtyRange *region;
+        if (begin > last_sector) {
+            break;
+        }
+
+        end = bdrv_dirty_bitmap_next_zero(bitmap, begin + 1);
+
+        region = g_new0(BlockDirtyRange, 1);
+        region->start = begin << BDRV_SECTOR_BITS;
+        region->count = (end - begin) << BDRV_SECTOR_BITS;
+
+        last_entry = g_new0(BlockDirtyRangeList, 1);
+        last_entry->value = region;
+
+        *plist = last_entry;
+        plist = &last_entry->next;
+        nb_ranges++;
+
+        bdrv_set_dirty_iter(it, end + 1);
+    }
+
+    bdrv_dirty_iter_free(it);
+
+    if (list != NULL) {
+        if (list->value->start < start) {
+            list->value->count -= start - list->value->start;
+            list->value->start = start;
+        }
+
+        if (last_entry->value->start +
+            last_entry->value->count > start + count) {
+            last_entry->value->count = start + count - last_entry->value->start;
+        }
+    }
+
+    return list;
+}
diff --git a/blockdev.c b/blockdev.c
index 1392fff..fa34cf6 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2733,6 +2733,68 @@ void qmp_block_dirty_bitmap_clear(const char *node, const char *name,
     aio_context_release(aio_context);
 }
 
+BlockDirtyRangeList *qmp_query_block_dirty_bitmap_ranges(const char *node,
+        const char *name, bool has_start, uint64_t start, bool has_count,
+        uint64_t count, uint32_t max_ranges, bool has_block_io, bool block_io,
+        Error **errp)
+{
+    AioContext *aio_context;
+    BdrvDirtyBitmap *bitmap;
+    BlockDriverState *bs;
+    BlockDirtyRangeList *ret = NULL;
+    int64_t disk_size;
+
+    if (!has_block_io) {
+        block_io = true;
+    }
+
+    bitmap = block_dirty_bitmap_lookup(node, name, &bs,
+                                       block_io ? &aio_context : NULL,
+                                       errp);
+    if (!bitmap) {
+        return NULL;
+    }
+
+    if (!block_io && bdrv_dirty_bitmap_enabled(bitmap)) {
+        error_setg(errp, "The bitmap is enabled, block-io must be on");
+        return NULL;
+    }
+
+    disk_size = bdrv_getlength(bs);
+    if (disk_size < 0) {
+        error_setg_errno(errp, -disk_size, "bdrv_getlength failed");
+        goto out;
+    }
+
+    if (has_start) {
+        if (start >= disk_size) {
+            error_setg(errp, "Start must be less than disk size");
+            goto out;
+        }
+    } else {
+        start = 0;
+    }
+
+    if (has_count) {
+        if (start + count > disk_size) {
+            error_setg(errp,
+                       "(start + count) must not be greater than disk size");
+            goto out;
+        }
+    } else {
+        count = disk_size - start;
+    }
+
+    ret = bdrv_query_dirty_bitmap_ranges(bs, bitmap, start, count, max_ranges);
+
+out:
+    if (block_io) {
+        aio_context_release(aio_context);
+    }
+
+    return ret;
+}
+
 void hmp_drive_del(Monitor *mon, const QDict *qdict)
 {
     const char *id = qdict_get_str(qdict, "id");
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 0033942..7918310 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -70,4 +70,11 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
 
 int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start);
 
+/* bdrv_query_dirty_bitmap_ranges
+ * start, count and resulting ranges are all in bytes
+ */
+BlockDirtyRangeList *bdrv_query_dirty_bitmap_ranges(BlockDriverState *bs,
+        BdrvDirtyBitmap *bitmap, uint64_t start, uint64_t count,
+        uint32_t max_ranges);
+
 #endif
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0a915ed..89bdeaf 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -414,6 +414,60 @@
 ##
 { 'command': 'query-block', 'returns': ['BlockInfo'] }
 
+##
+# @BlockDirtyRegion:
+#
+# Region in bytes.
+#
+# @start: first byte
+#
+# @count: number of bytes in the region
+#
+# Since: 2.6
+##
+{ 'struct': 'BlockDirtyRange',
+  'data': { 'start': 'uint64', 'count': 'uint64' } }
+
+##
+# @BlockDirtyBitmapQueryRanges
+#
+# @node: name of device/node which the bitmap is tracking
+#
+# @name: name of the dirty bitmap
+#
+# @start: #optional start of quered range, default is zero for
+#         query-block-dirty-bitmap-ranges
+#
+# @count: #optional lenght of quered range, default is (disk-size - @start)
+#         for query-block-dirty-bitmap-ranges
+#
+# @max-ranges: max dirty ranges to be returned. If there are more than
+#              @max-ranges dirty ranges within quered range, then first
+#              @max-ranges dirty ranges from quered range will be returned.
+#
+# @block-io: #optional wheter aio-context should be acquired while oparating.
+#            False value is allowed only for frozen or disabled bitmaps.
+#            Default is true for query-block-dirty-bitmap-ranges.
+#
+# Since 2.6
+##
+{ 'struct': 'BlockDirtyBitmapQueryRanges',
+  'data': { 'node': 'str', 'name': 'str', '*start': 'uint64',
+            '*count': 'uint64', 'max-ranges': 'uint32', '*block-io': 'bool'} }
+
+##
+# @query-block-dirty-bitmap-ranges
+#
+# Get a description for specified dirty bitmap including it's dirty regions.
+# This command is in general for testing purposes.
+#
+# Returns: @BlockDirtyBitmapInfo
+#
+# Since: 2.6
+##
+{ 'command': 'query-block-dirty-bitmap-ranges',
+  'data': 'BlockDirtyBitmapQueryRanges',
+  'returns': ['BlockDirtyRange'] }
 
 ##
 # @BlockDeviceTimedStats:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index db072a6..02de44c 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -1457,6 +1457,39 @@ Example:
 EQMP
 
     {
+        .name       = "query-block-dirty-bitmap-ranges",
+        .args_type  = "node:B,name:s,start:i?,count:i?,max-ranges:i,"
+                      "block-io:b?",
+        .mhandler.cmd_new = qmp_marshal_query_block_dirty_bitmap_ranges,
+    },
+
+SQMP
+
+query-block-dirty-bitmap-ranges
+------------------------
+Since 2.6
+
+Get contens of specified dirty bitmap. Bitmap data is returned as array of
+dirty ranges.
+
+Arguments:
+
+- "node": device/node on which to remove dirty bitmap (json-string)
+- "name": name of the dirty bitmap (json-string)
+- "start": start of quered range in bytes (json-int, optional, default 0)
+- "count": lenght of quered range in bytes
+           (json-int, optional, default (disk-size - @start))
+- "max-ranges": max dirty ranges to be returned. If there are more than
+                @max-ranges dirty ranges within quered range, then first
+                @max-ranges dirty ranges from quered range will be returned.
+                (json-int)
+- "block-io": wheter aio-context should be acquired while oparating. False
+              value is allowed only for frozen or disabled bitmaps.
+              (json-bool, optional, default true)
+
+EQMP
+
+    {
         .name       = "blockdev-snapshot-sync",
         .args_type  = "device:s?,node-name:s?,snapshot-file:s,snapshot-node-name:s?,format:s?,mode:s?",
         .mhandler.cmd_new = qmp_marshal_blockdev_snapshot_sync,
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [Qemu-devel] [PATCH 3/6] iotests: test query-block-dirty-bitmap-ranges
  2016-01-30 10:56 [Qemu-devel] [PATCH v2 0/6] external backup api Vladimir Sementsov-Ogievskiy
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 1/6] block dirty bitmap: add next_zero function Vladimir Sementsov-Ogievskiy
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges Vladimir Sementsov-Ogievskiy
@ 2016-01-30 10:56 ` Vladimir Sementsov-Ogievskiy
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 4/6] qapi: add qmp commands for some dirty bitmap functions Vladimir Sementsov-Ogievskiy
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-01-30 10:56 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, Vladimir Sementsov-Ogievskiy, famz, den, armbru, jsnow

Add test for QMP command query-block-dirty-bitmap-ranges.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/150     | 88 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/150.out | 21 +++++++++++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 110 insertions(+)
 create mode 100755 tests/qemu-iotests/150
 create mode 100644 tests/qemu-iotests/150.out

diff --git a/tests/qemu-iotests/150 b/tests/qemu-iotests/150
new file mode 100755
index 0000000..f208050
--- /dev/null
+++ b/tests/qemu-iotests/150
@@ -0,0 +1,88 @@
+#!/usr/bin/env python
+#
+# Test qmp command query-block-dirty-bitmap-ranges
+#
+# (C) Vladimir Sementsov-Ogievskiy 2015
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk = os.path.join(iotests.test_dir, 'disk')
+
+size   = 0x40000000 # 1G
+sector_size = 512
+granularity = 0x10000
+regions = [
+    { 'start': 0,          'count': 0x100000 },
+    { 'start': 0x10000000, 'count': 0x20000  },
+    { 'start': 0x39990000, 'count': 0x10000  }
+    ]
+
+class TestQueryDirtyBitmap(iotests.QMPTestCase):
+
+    def setUp(self):
+        qemu_img('create', '-f', iotests.imgfmt, disk, str(size))
+        self.vm = iotests.VM().add_drive(disk)
+        self.vm.launch()
+
+    def tearDown(self):
+        self.vm.shutdown()
+        os.remove(disk)
+
+    def printRange(self, begin, end):
+        print self.vm.qmp('query-block-dirty-bitmap-ranges', node='drive0',
+                name='bitmap', start=begin, count=(end - begin),
+                max_ranges=100);
+
+    def printWhole(self, max_ranges=100):
+        print self.vm.qmp('query-block-dirty-bitmap-ranges', node='drive0',
+                name='bitmap', max_ranges=max_ranges);
+
+    def test_query_bitmap(self):
+        result = self.vm.qmp('block-dirty-bitmap-add', node='drive0',
+                               name='bitmap', granularity=granularity)
+        self.assert_qmp(result, 'return', {});
+
+        for r in regions:
+          self.vm.hmp_qemu_io('drive0',
+                                'write %d %d' % (r['start'], r['count']))
+
+        self.printWhole()
+        self.printWhole(2)
+        self.printRange(100, 200)
+        self.printRange(0x100000 - 10, 0x10000000 + 10)
+        print
+
+        self.printRange(0x100000 - 10, 0x10000000 + 1)
+        self.printRange(0x100000 - 10, 0x10000000)
+        self.printRange(0x100000 - 10, 0x10000000 - 1)
+        print
+
+        self.printRange(0x100000 + 1, 0x10000000 - 1)
+        self.printRange(0x100000 + 1, 0x10000000)
+        self.printRange(0x100000 + 1, 0x10000000 + 1)
+        print
+
+        self.printRange(0x100000, 0x10000000 - 1)
+        self.printRange(0x100000, 0x10000000)
+        self.printRange(0x100000, 0x10000000 + 1)
+
+
+if __name__ == '__main__':
+    iotests.main()
diff --git a/tests/qemu-iotests/150.out b/tests/qemu-iotests/150.out
new file mode 100644
index 0000000..c137925
--- /dev/null
+++ b/tests/qemu-iotests/150.out
@@ -0,0 +1,21 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK
+{u'return': [{u'count': 1048576, u'start': 0}, {u'count': 131072, u'start': 268435456}, {u'count': 65536, u'start': 966328320}]}
+{u'return': [{u'count': 1048576, u'start': 0}, {u'count': 131072, u'start': 268435456}]}
+{u'return': [{u'count': 100, u'start': 100}]}
+{u'return': [{u'count': 10, u'start': 1048566}, {u'count': 10, u'start': 268435456}]}
+
+{u'return': [{u'count': 10, u'start': 1048566}, {u'count': 1, u'start': 268435456}]}
+{u'return': [{u'count': 10, u'start': 1048566}]}
+{u'return': [{u'count': 10, u'start': 1048566}]}
+
+{u'return': []}
+{u'return': []}
+{u'return': [{u'count': 1, u'start': 268435456}]}
+
+{u'return': []}
+{u'return': []}
+{u'return': [{u'count': 1, u'start': 268435456}]}
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index d6e9219..84db670 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -142,3 +142,4 @@
 138 rw auto quick
 139 rw auto quick
 142 auto
+150 auto
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [Qemu-devel] [PATCH 4/6] qapi: add qmp commands for some dirty bitmap functions
  2016-01-30 10:56 [Qemu-devel] [PATCH v2 0/6] external backup api Vladimir Sementsov-Ogievskiy
                   ` (2 preceding siblings ...)
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 3/6] iotests: test query-block-dirty-bitmap-ranges Vladimir Sementsov-Ogievskiy
@ 2016-01-30 10:56 ` Vladimir Sementsov-Ogievskiy
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 5/6] qapi: make block-dirty-bitmap-create-successor transaction-able Vladimir Sementsov-Ogievskiy
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-01-30 10:56 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, Vladimir Sementsov-Ogievskiy, famz, den, armbru, jsnow

Add access to bdrv_dirty_bitmap_create_successor,
bdrv_dirty_bitmap_abdicate, bdrv_reclaim_dirty_bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 blockdev.c           | 51 +++++++++++++++++++++++++++++++
 qapi/block-core.json | 36 ++++++++++++++++++++++
 qmp-commands.hx      | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 172 insertions(+)

diff --git a/blockdev.c b/blockdev.c
index fa34cf6..f03a415 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2733,6 +2733,57 @@ void qmp_block_dirty_bitmap_clear(const char *node, const char *name,
     aio_context_release(aio_context);
 }
 
+void qmp_block_dirty_bitmap_create_successor(const char *node,
+                                             const char *name, Error **errp)
+{
+    AioContext *aio_context;
+    BdrvDirtyBitmap *bitmap;
+    BlockDriverState *bs;
+
+    bitmap = block_dirty_bitmap_lookup(node, name, &bs, &aio_context, errp);
+    if (!bitmap || !bs) {
+        return;
+    }
+
+    bdrv_dirty_bitmap_create_successor(bs, bitmap, errp);
+
+    aio_context_release(aio_context);
+}
+
+void qmp_block_dirty_bitmap_abdicate(const char *node, const char *name,
+                                     Error **errp)
+{
+    AioContext *aio_context;
+    BdrvDirtyBitmap *bitmap;
+    BlockDriverState *bs;
+
+    bitmap = block_dirty_bitmap_lookup(node, name, &bs, &aio_context, errp);
+    if (!bitmap || !bs) {
+        return;
+    }
+
+    bdrv_dirty_bitmap_abdicate(bs, bitmap, errp);
+
+    aio_context_release(aio_context);
+}
+
+void qmp_block_dirty_bitmap_reclaim(const char *node, const char *name,
+                                     Error **errp)
+{
+    AioContext *aio_context;
+    BdrvDirtyBitmap *bitmap;
+    BlockDriverState *bs;
+
+    bitmap = block_dirty_bitmap_lookup(node, name, &bs, &aio_context, errp);
+    if (!bitmap || !bs) {
+        return;
+    }
+
+    bdrv_reclaim_dirty_bitmap(bs, bitmap, errp);
+
+    aio_context_release(aio_context);
+}
+
 BlockDirtyRangeList *qmp_query_block_dirty_bitmap_ranges(const char *node,
         const char *name, bool has_start, uint64_t start, bool has_count,
         uint64_t count, uint32_t max_ranges, bool has_block_io, bool block_io,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 89bdeaf..a83c286 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1238,6 +1238,42 @@
   'data': 'BlockDirtyBitmap' }
 
 ##
+# @block-dirty-bitmap-create-successor
+#
+# Interface to bdrv_dirty_bitmap_create_successor
+#
+# Returns: nothing on success
+#
+# Since 2.6
+##
+{ 'command': 'block-dirty-bitmap-create-successor',
+  'data': 'BlockDirtyBitmap' }
+
+##
+# @block-dirty-bitmap-abdicate
+#
+# Interface to bdrv_dirty_bitmap_abdicate
+#
+# Returns: nothing on success
+#
+# Since 2.6
+##
+{ 'command': 'block-dirty-bitmap-abdicate',
+  'data': 'BlockDirtyBitmap' }
+
+##
+# @block-dirty-bitmap-reclaim
+#
+# Interface to bdrv_reclaim_dirty_bitmap
+#
+# Returns: nothing on success
+#
+# Since 2.6
+##
+{ 'command': 'block-dirty-bitmap-reclaim',
+  'data': 'BlockDirtyBitmap' }
+
+##
 # @blockdev-mirror
 #
 # Start mirroring a block device's writes to a new destination.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 02de44c..6098371 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -1457,6 +1457,91 @@ Example:
 EQMP
 
     {
+        .name       = "block-dirty-bitmap-create-successor",
+        .args_type  = "node:B,name:s",
+        .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_create_successor,
+    },
+
+SQMP
+
+block-dirty-bitmap-create-successor
+------------------------
+Since 2.6
+
+Create a successor bitmap destined to replace this bitmap after an operation.
+Requires that the bitmap is not frozen and has no successor.
+
+Arguments:
+
+- "node": device/node on which to remove dirty bitmap (json-string)
+- "name": name of the dirty bitmap to remove (json-string)
+
+Example:
+
+-> { "execute": "block-dirty-bitmap-create-successor",
+     "arguments": { "node": "drive0", "name": "bitmap0" } }
+<- { "return": {} }
+
+EQMP
+
+    {
+        .name       = "block-dirty-bitmap-abdicate",
+        .args_type  = "node:B,name:s",
+        .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_abdicate,
+    },
+
+SQMP
+
+block-dirty-bitmap-abdicate
+------------------------
+Since 2.6
+
+For a bitmap with a successor, yield our name to the successor, delete the old
+bitmap, and return a handle to the new bitmap.
+
+Arguments:
+
+- "node": device/node on which to remove dirty bitmap (json-string)
+- "name": name of the dirty bitmap to remove (json-string)
+
+Example:
+
+-> { "execute": "block-dirty-bitmap-abdicate",
+     "arguments": { "node": "drive0", "name": "bitmap0" } }
+<- { "return": {} }
+
+EQMP
+
+    {
+        .name       = "block-dirty-bitmap-reclaim",
+        .args_type  = "node:B,name:s",
+        .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_reclaim,
+    },
+
+SQMP
+
+block-dirty-bitmap-reclaim
+------------------------
+Since 2.6
+
+In cases of failure where we can no longer safely delete the parent, we may
+wish to re-join the parent and child/successor.  The merged parent will be
+un-frozen, but not explicitly re-enabled.
+
+Arguments:
+
+- "node": device/node on which to remove dirty bitmap (json-string)
+- "name": name of the dirty bitmap to remove (json-string)
+
+Example:
+
+-> { "execute": "block-dirty-bitmap-reclaim",
+     "arguments": { "node": "drive0", "name": "bitmap0" } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "query-block-dirty-bitmap-ranges",
         .args_type  = "node:B,name:s,start:i?,count:i?,max-ranges:i,"
                       "block-io:b?",
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [Qemu-devel] [PATCH 5/6] qapi: make block-dirty-bitmap-create-successor transaction-able
  2016-01-30 10:56 [Qemu-devel] [PATCH v2 0/6] external backup api Vladimir Sementsov-Ogievskiy
                   ` (3 preceding siblings ...)
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 4/6] qapi: add qmp commands for some dirty bitmap functions Vladimir Sementsov-Ogievskiy
@ 2016-01-30 10:56 ` Vladimir Sementsov-Ogievskiy
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 6/6] iotests: test external backup api Vladimir Sementsov-Ogievskiy
  2016-02-03  8:14 ` [Qemu-devel] [PATCH v2 0/6] " Fam Zheng
  6 siblings, 0 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-01-30 10:56 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, Vladimir Sementsov-Ogievskiy, famz, den, armbru, jsnow

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 qapi-schema.json | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index b3038b2..a7e97c7 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1650,6 +1650,7 @@
 # blockdev-snapshot since 2.5
 # block-dirty-bitmap-add since 2.5
 # block-dirty-bitmap-clear since 2.5
+# block-dirty-bitmap-create-successor since 2.6
 ##
 { 'union': 'TransactionAction',
   'data': {
@@ -1660,7 +1661,8 @@
        'abort': 'Abort',
        'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal',
        'block-dirty-bitmap-add': 'BlockDirtyBitmapAdd',
-       'block-dirty-bitmap-clear': 'BlockDirtyBitmap'
+       'block-dirty-bitmap-clear': 'BlockDirtyBitmap',
+       'block-dirty-bitmap-create-successor': 'BlockDirtyBitmap'
    } }
 
 ##
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* [Qemu-devel] [PATCH 6/6] iotests: test external backup api
  2016-01-30 10:56 [Qemu-devel] [PATCH v2 0/6] external backup api Vladimir Sementsov-Ogievskiy
                   ` (4 preceding siblings ...)
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 5/6] qapi: make block-dirty-bitmap-create-successor transaction-able Vladimir Sementsov-Ogievskiy
@ 2016-01-30 10:56 ` Vladimir Sementsov-Ogievskiy
  2016-02-03  8:14 ` [Qemu-devel] [PATCH v2 0/6] " Fam Zheng
  6 siblings, 0 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-01-30 10:56 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, Vladimir Sementsov-Ogievskiy, famz, den, armbru, jsnow

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/151     | 77 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/151.out |  7 +++++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 85 insertions(+)
 create mode 100755 tests/qemu-iotests/151
 create mode 100644 tests/qemu-iotests/151.out

diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151
new file mode 100755
index 0000000..d2fc2dc
--- /dev/null
+++ b/tests/qemu-iotests/151
@@ -0,0 +1,77 @@
+#!/usr/bin/env python
+#
+# Test external backup api
+#
+# (C) Vladimir Sementsov-Ogievskiy 2016
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk = os.path.join(iotests.test_dir, 'disk')
+
+size   = 0x40000000 # 1G
+sector_size = 512
+granularity = 0x10000
+regions = [
+    { 'start': 0,          'count': 0x100000 },
+    { 'start': 0x10000000, 'count': 0x20000  },
+    { 'start': 0x39990000, 'count': 0x10000  }
+    ]
+
+class TestExternalBackupApi(iotests.QMPTestCase):
+
+    def setUp(self):
+        qemu_img('create', '-f', iotests.imgfmt, disk, str(size))
+        self.vm = iotests.VM().add_drive(disk)
+        self.vm.launch()
+
+    def tearDown(self):
+        self.vm.shutdown()
+        os.remove(disk)
+
+    def printBitmap(self, block_io):
+        print self.vm.qmp('query-block-dirty-bitmap-ranges', node='drive0',
+                          name='bitmap', block_io=(block_io if 'on' else 'off'),
+                          max_ranges=100);
+
+    def test_api(self):
+        result = self.vm.qmp('block-dirty-bitmap-add', node='drive0',
+                             name='bitmap', granularity=granularity)
+        self.assert_qmp(result, 'return', {});
+
+        for r in regions:
+            self.vm.hmp_qemu_io('drive0',
+                                'write %d %d' % (r['start'], r['count']))
+
+        result = self.vm.qmp('block-dirty-bitmap-create-successor',
+                                node='drive0', name='bitmap')
+        self.assert_qmp(result, 'return', {});
+
+        self.printBitmap(False)
+
+        result = self.vm.qmp('block-dirty-bitmap-abdicate',
+                              node='drive0', name='bitmap')
+        self.assert_qmp(result, 'return', {});
+
+        self.printBitmap(True)
+
+
+
+if __name__ == '__main__':
+    iotests.main()
diff --git a/tests/qemu-iotests/151.out b/tests/qemu-iotests/151.out
new file mode 100644
index 0000000..baed190
--- /dev/null
+++ b/tests/qemu-iotests/151.out
@@ -0,0 +1,7 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK
+{u'return': [{u'count': 1048576, u'start': 0}, {u'count': 131072, u'start': 268435456}, {u'count': 65536, u'start': 966328320}]}
+{u'return': []}
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 84db670..9425a23 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -143,3 +143,4 @@
 139 rw auto quick
 142 auto
 150 auto
+151 auto
-- 
1.8.3.1

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-01-30 10:56 [Qemu-devel] [PATCH v2 0/6] external backup api Vladimir Sementsov-Ogievskiy
                   ` (5 preceding siblings ...)
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 6/6] iotests: test external backup api Vladimir Sementsov-Ogievskiy
@ 2016-02-03  8:14 ` Fam Zheng
  2016-02-03 10:57   ` Vladimir Sementsov-Ogievskiy
  2016-02-05  8:28   ` Denis V. Lunev
  6 siblings, 2 replies; 56+ messages in thread
From: Fam Zheng @ 2016-02-03  8:14 UTC (permalink / raw)
  To: Vladimir Sementsov-Ogievskiy; +Cc: kwolf, den, jsnow, qemu-devel, armbru

On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> Hi all.
> 
> These series which aims to add external backup api. This is needed to allow
> backup software use our dirty bitmaps.
> 
> Vmware and Parallels Cloud Server have this feature.

What is the advantage of this appraoch over "drive-backup sync=incremental
..."?

> 
> There are three things are done:
> - add query-block-dirty-bitmap-ranges qmp command
> - add qmp commands for dirty-bitmap functions: create_successor, abdicate,
>   reclaim.
> - make create-successor command transaction-able
> 
> Then, external backup should be done like this:
> 
> 1.  qmp transaction {
>         external-snapshot
>         bitmap-create-successor
>     }
> 
> 2.  qmp query frozen bitmap, not acquiring aio context.

What do you do with the query response, read the snapshot image with an
external tool?

Fam

> 
> 3.  do external backup, using snapshot and bitmap
> 
> 
> 4.  if (success backup)
>         qmp bitmap-abdicate
>     else
>         qmp bitmap-reclaime
> 
> 5.  qmp merge snapshot
> 
> 
> v2: a lot of additions and changes, no sense to compare with v1
> 
> 
> Vladimir Sementsov-Ogievskiy (6):
>   block dirty bitmap: add next_zero function
>   qmp: add query-block-dirty-bitmap-ranges
>   iotests: test query-block-dirty-bitmap-ranges
>   qapi: add qmp commands for some dirty bitmap functions
>   qapi: make block-dirty-bitmap-create-successor transaction-able
>   iotests: test external backup api
> 
>  block/dirty-bitmap.c         |  60 ++++++++++++++++++++++
>  blockdev.c                   | 113 +++++++++++++++++++++++++++++++++++++++++
>  include/block/dirty-bitmap.h |   9 ++++
>  include/qemu/hbitmap.h       |   8 +++
>  qapi-schema.json             |   4 +-
>  qapi/block-core.json         |  90 +++++++++++++++++++++++++++++++++
>  qmp-commands.hx              | 118 +++++++++++++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/150       |  88 ++++++++++++++++++++++++++++++++
>  tests/qemu-iotests/150.out   |  21 ++++++++
>  tests/qemu-iotests/151       |  77 ++++++++++++++++++++++++++++
>  tests/qemu-iotests/151.out   |   7 +++
>  tests/qemu-iotests/group     |   2 +
>  util/hbitmap.c               |  26 ++++++++++
>  13 files changed, 622 insertions(+), 1 deletion(-)
>  create mode 100755 tests/qemu-iotests/150
>  create mode 100644 tests/qemu-iotests/150.out
>  create mode 100755 tests/qemu-iotests/151
>  create mode 100644 tests/qemu-iotests/151.out
> 
> -- 
> 1.8.3.1
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-03  8:14 ` [Qemu-devel] [PATCH v2 0/6] " Fam Zheng
@ 2016-02-03 10:57   ` Vladimir Sementsov-Ogievskiy
  2016-02-03 11:02     ` Fam Zheng
  2016-02-05  8:28   ` Denis V. Lunev
  1 sibling, 1 reply; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-02-03 10:57 UTC (permalink / raw)
  To: Fam Zheng; +Cc: kwolf, den, jsnow, qemu-devel, armbru

On 03.02.2016 11:14, Fam Zheng wrote:
> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>> Hi all.
>>
>> These series which aims to add external backup api. This is needed to allow
>> backup software use our dirty bitmaps.
>>
>> Vmware and Parallels Cloud Server have this feature.
> What is the advantage of this appraoch over "drive-backup sync=incremental
> ..."?

Hmm, you are asking about advantage of external backup over internal? It 
depends on external backup tool.

>
>> There are three things are done:
>> - add query-block-dirty-bitmap-ranges qmp command
>> - add qmp commands for dirty-bitmap functions: create_successor, abdicate,
>>    reclaim.
>> - make create-successor command transaction-able
>>
>> Then, external backup should be done like this:
>>
>> 1.  qmp transaction {
>>          external-snapshot
>>          bitmap-create-successor
>>      }
>>
>> 2.  qmp query frozen bitmap, not acquiring aio context.
> What do you do with the query response, read the snapshot image with an
> external tool?

Yes. Alternatively, image fleecing may be used instead of snapshot.

>
> Fam
>
>> 3.  do external backup, using snapshot and bitmap
>>
>>
>> 4.  if (success backup)
>>          qmp bitmap-abdicate
>>      else
>>          qmp bitmap-reclaime
>>
>> 5.  qmp merge snapshot
>>
>>
>> v2: a lot of additions and changes, no sense to compare with v1
>>
>>
>> Vladimir Sementsov-Ogievskiy (6):
>>    block dirty bitmap: add next_zero function
>>    qmp: add query-block-dirty-bitmap-ranges
>>    iotests: test query-block-dirty-bitmap-ranges
>>    qapi: add qmp commands for some dirty bitmap functions
>>    qapi: make block-dirty-bitmap-create-successor transaction-able
>>    iotests: test external backup api
>>
>>   block/dirty-bitmap.c         |  60 ++++++++++++++++++++++
>>   blockdev.c                   | 113 +++++++++++++++++++++++++++++++++++++++++
>>   include/block/dirty-bitmap.h |   9 ++++
>>   include/qemu/hbitmap.h       |   8 +++
>>   qapi-schema.json             |   4 +-
>>   qapi/block-core.json         |  90 +++++++++++++++++++++++++++++++++
>>   qmp-commands.hx              | 118 +++++++++++++++++++++++++++++++++++++++++++
>>   tests/qemu-iotests/150       |  88 ++++++++++++++++++++++++++++++++
>>   tests/qemu-iotests/150.out   |  21 ++++++++
>>   tests/qemu-iotests/151       |  77 ++++++++++++++++++++++++++++
>>   tests/qemu-iotests/151.out   |   7 +++
>>   tests/qemu-iotests/group     |   2 +
>>   util/hbitmap.c               |  26 ++++++++++
>>   13 files changed, 622 insertions(+), 1 deletion(-)
>>   create mode 100755 tests/qemu-iotests/150
>>   create mode 100644 tests/qemu-iotests/150.out
>>   create mode 100755 tests/qemu-iotests/151
>>   create mode 100644 tests/qemu-iotests/151.out
>>
>> -- 
>> 1.8.3.1
>>


-- 
Best regards,
Vladimir

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-03 10:57   ` Vladimir Sementsov-Ogievskiy
@ 2016-02-03 11:02     ` Fam Zheng
  2016-02-03 11:24       ` Vladimir Sementsov-Ogievskiy
  0 siblings, 1 reply; 56+ messages in thread
From: Fam Zheng @ 2016-02-03 11:02 UTC (permalink / raw)
  To: Vladimir Sementsov-Ogievskiy; +Cc: kwolf, den, jsnow, qemu-devel, armbru

On Wed, 02/03 13:57, Vladimir Sementsov-Ogievskiy wrote:
> On 03.02.2016 11:14, Fam Zheng wrote:
> >On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>Hi all.
> >>
> >>These series which aims to add external backup api. This is needed to allow
> >>backup software use our dirty bitmaps.
> >>
> >>Vmware and Parallels Cloud Server have this feature.
> >What is the advantage of this appraoch over "drive-backup sync=incremental
> >..."?
> 
> Hmm, you are asking about advantage of external backup over
> internal? It depends on external backup tool.

I'm asking why the tool can't use drive-backup to achieve what you want. They
seem very comparable.

Fam

> 
> >
> >>There are three things are done:
> >>- add query-block-dirty-bitmap-ranges qmp command
> >>- add qmp commands for dirty-bitmap functions: create_successor, abdicate,
> >>   reclaim.
> >>- make create-successor command transaction-able
> >>
> >>Then, external backup should be done like this:
> >>
> >>1.  qmp transaction {
> >>         external-snapshot
> >>         bitmap-create-successor
> >>     }
> >>
> >>2.  qmp query frozen bitmap, not acquiring aio context.
> >What do you do with the query response, read the snapshot image with an
> >external tool?
> 
> Yes. Alternatively, image fleecing may be used instead of snapshot.
> 
> >
> >Fam
> >
> >>3.  do external backup, using snapshot and bitmap
> >>
> >>
> >>4.  if (success backup)
> >>         qmp bitmap-abdicate
> >>     else
> >>         qmp bitmap-reclaime
> >>
> >>5.  qmp merge snapshot
> >>
> >>
> >>v2: a lot of additions and changes, no sense to compare with v1
> >>
> >>
> >>Vladimir Sementsov-Ogievskiy (6):
> >>   block dirty bitmap: add next_zero function
> >>   qmp: add query-block-dirty-bitmap-ranges
> >>   iotests: test query-block-dirty-bitmap-ranges
> >>   qapi: add qmp commands for some dirty bitmap functions
> >>   qapi: make block-dirty-bitmap-create-successor transaction-able
> >>   iotests: test external backup api
> >>
> >>  block/dirty-bitmap.c         |  60 ++++++++++++++++++++++
> >>  blockdev.c                   | 113 +++++++++++++++++++++++++++++++++++++++++
> >>  include/block/dirty-bitmap.h |   9 ++++
> >>  include/qemu/hbitmap.h       |   8 +++
> >>  qapi-schema.json             |   4 +-
> >>  qapi/block-core.json         |  90 +++++++++++++++++++++++++++++++++
> >>  qmp-commands.hx              | 118 +++++++++++++++++++++++++++++++++++++++++++
> >>  tests/qemu-iotests/150       |  88 ++++++++++++++++++++++++++++++++
> >>  tests/qemu-iotests/150.out   |  21 ++++++++
> >>  tests/qemu-iotests/151       |  77 ++++++++++++++++++++++++++++
> >>  tests/qemu-iotests/151.out   |   7 +++
> >>  tests/qemu-iotests/group     |   2 +
> >>  util/hbitmap.c               |  26 ++++++++++
> >>  13 files changed, 622 insertions(+), 1 deletion(-)
> >>  create mode 100755 tests/qemu-iotests/150
> >>  create mode 100644 tests/qemu-iotests/150.out
> >>  create mode 100755 tests/qemu-iotests/151
> >>  create mode 100644 tests/qemu-iotests/151.out
> >>
> >>-- 
> >>1.8.3.1
> >>
> 
> 
> -- 
> Best regards,
> Vladimir
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-03 11:02     ` Fam Zheng
@ 2016-02-03 11:24       ` Vladimir Sementsov-Ogievskiy
  0 siblings, 0 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-02-03 11:24 UTC (permalink / raw)
  To: Fam Zheng; +Cc: kwolf, den, jsnow, qemu-devel, armbru

On 03.02.2016 14:02, Fam Zheng wrote:
> On Wed, 02/03 13:57, Vladimir Sementsov-Ogievskiy wrote:
>> On 03.02.2016 11:14, Fam Zheng wrote:
>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>> Hi all.
>>>>
>>>> These series which aims to add external backup api. This is needed to allow
>>>> backup software use our dirty bitmaps.
>>>>
>>>> Vmware and Parallels Cloud Server have this feature.
>>> What is the advantage of this appraoch over "drive-backup sync=incremental
>>> ..."?
>> Hmm, you are asking about advantage of external backup over
>> internal? It depends on external backup tool.
> I'm asking why the tool can't use drive-backup to achieve what you want. They
> seem very comparable.

drive-backup has limited range of maintained by qemu targets.. Third 
party tool uses its own internal targets. If we use drvie-backup there 
will be two copies instead of one: from qemu to drive-backup produced 
backup and then to internal target in third-party tool.

>
> Fam
>
>>>> There are three things are done:
>>>> - add query-block-dirty-bitmap-ranges qmp command
>>>> - add qmp commands for dirty-bitmap functions: create_successor, abdicate,
>>>>    reclaim.
>>>> - make create-successor command transaction-able
>>>>
>>>> Then, external backup should be done like this:
>>>>
>>>> 1.  qmp transaction {
>>>>          external-snapshot
>>>>          bitmap-create-successor
>>>>      }
>>>>
>>>> 2.  qmp query frozen bitmap, not acquiring aio context.
>>> What do you do with the query response, read the snapshot image with an
>>> external tool?
>> Yes. Alternatively, image fleecing may be used instead of snapshot.
>>
>>> Fam
>>>
>>>> 3.  do external backup, using snapshot and bitmap
>>>>
>>>>
>>>> 4.  if (success backup)
>>>>          qmp bitmap-abdicate
>>>>      else
>>>>          qmp bitmap-reclaime
>>>>
>>>> 5.  qmp merge snapshot
>>>>
>>>>
>>>> v2: a lot of additions and changes, no sense to compare with v1
>>>>
>>>>
>>>> Vladimir Sementsov-Ogievskiy (6):
>>>>    block dirty bitmap: add next_zero function
>>>>    qmp: add query-block-dirty-bitmap-ranges
>>>>    iotests: test query-block-dirty-bitmap-ranges
>>>>    qapi: add qmp commands for some dirty bitmap functions
>>>>    qapi: make block-dirty-bitmap-create-successor transaction-able
>>>>    iotests: test external backup api
>>>>
>>>>   block/dirty-bitmap.c         |  60 ++++++++++++++++++++++
>>>>   blockdev.c                   | 113 +++++++++++++++++++++++++++++++++++++++++
>>>>   include/block/dirty-bitmap.h |   9 ++++
>>>>   include/qemu/hbitmap.h       |   8 +++
>>>>   qapi-schema.json             |   4 +-
>>>>   qapi/block-core.json         |  90 +++++++++++++++++++++++++++++++++
>>>>   qmp-commands.hx              | 118 +++++++++++++++++++++++++++++++++++++++++++
>>>>   tests/qemu-iotests/150       |  88 ++++++++++++++++++++++++++++++++
>>>>   tests/qemu-iotests/150.out   |  21 ++++++++
>>>>   tests/qemu-iotests/151       |  77 ++++++++++++++++++++++++++++
>>>>   tests/qemu-iotests/151.out   |   7 +++
>>>>   tests/qemu-iotests/group     |   2 +
>>>>   util/hbitmap.c               |  26 ++++++++++
>>>>   13 files changed, 622 insertions(+), 1 deletion(-)
>>>>   create mode 100755 tests/qemu-iotests/150
>>>>   create mode 100644 tests/qemu-iotests/150.out
>>>>   create mode 100755 tests/qemu-iotests/151
>>>>   create mode 100644 tests/qemu-iotests/151.out
>>>>
>>>> -- 
>>>> 1.8.3.1
>>>>
>>
>> -- 
>> Best regards,
>> Vladimir
>>


-- 
Best regards,
Vladimir

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-03  8:14 ` [Qemu-devel] [PATCH v2 0/6] " Fam Zheng
  2016-02-03 10:57   ` Vladimir Sementsov-Ogievskiy
@ 2016-02-05  8:28   ` Denis V. Lunev
  2016-02-05  8:44     ` Fam Zheng
                       ` (2 more replies)
  1 sibling, 3 replies; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-05  8:28 UTC (permalink / raw)
  To: Fam Zheng, Vladimir Sementsov-Ogievskiy; +Cc: kwolf, jsnow, qemu-devel, armbru

On 02/03/2016 11:14 AM, Fam Zheng wrote:
> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>> Hi all.
>>
>> These series which aims to add external backup api. This is needed to allow
>> backup software use our dirty bitmaps.
>>
>> Vmware and Parallels Cloud Server have this feature.
> What is the advantage of this appraoch over "drive-backup sync=incremental
> ..."?

This will allow third-party vendors to backup QEMU VMs into
their own formats or to the cloud etc.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-05  8:28   ` Denis V. Lunev
@ 2016-02-05  8:44     ` Fam Zheng
  2016-02-09 14:21     ` Stefan Hajnoczi
  2016-02-09 14:28     ` Stefan Hajnoczi
  2 siblings, 0 replies; 56+ messages in thread
From: Fam Zheng @ 2016-02-05  8:44 UTC (permalink / raw)
  To: Denis V. Lunev
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, jsnow, qemu-devel, armbru

On Fri, 02/05 11:28, Denis V. Lunev wrote:
> On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>Hi all.
> >>
> >>These series which aims to add external backup api. This is needed to allow
> >>backup software use our dirty bitmaps.
> >>
> >>Vmware and Parallels Cloud Server have this feature.
> >What is the advantage of this appraoch over "drive-backup sync=incremental
> >..."?
> 
> This will allow third-party vendors to backup QEMU VMs into
> their own formats or to the cloud etc.

Yes, I see, thanks for explaining.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-05  8:28   ` Denis V. Lunev
  2016-02-05  8:44     ` Fam Zheng
@ 2016-02-09 14:21     ` Stefan Hajnoczi
  2016-02-09 14:37       ` Denis V. Lunev
  2016-02-09 14:28     ` Stefan Hajnoczi
  2 siblings, 1 reply; 56+ messages in thread
From: Stefan Hajnoczi @ 2016-02-09 14:21 UTC (permalink / raw)
  To: Denis V. Lunev
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel,
	armbru, jsnow

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>Hi all.
> >>
> >>These series which aims to add external backup api. This is needed to allow
> >>backup software use our dirty bitmaps.
> >>
> >>Vmware and Parallels Cloud Server have this feature.
> >What is the advantage of this appraoch over "drive-backup sync=incremental
> >..."?
> 
> This will allow third-party vendors to backup QEMU VMs into
> their own formats or to the cloud etc.

Backup software can implement NBD to receive the incremental blocks from
QEMU.  Use drive-backup sync=incremental and the backup appliance as the
NBD target.

It's more complicated to add this QMP command flow to backup software
than to implement NBD.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-05  8:28   ` Denis V. Lunev
  2016-02-05  8:44     ` Fam Zheng
  2016-02-09 14:21     ` Stefan Hajnoczi
@ 2016-02-09 14:28     ` Stefan Hajnoczi
  2016-02-09 14:41       ` Denis V. Lunev
  2 siblings, 1 reply; 56+ messages in thread
From: Stefan Hajnoczi @ 2016-02-09 14:28 UTC (permalink / raw)
  To: Denis V. Lunev
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel,
	armbru, jsnow

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>Hi all.
> >>
> >>These series which aims to add external backup api. This is needed to allow
> >>backup software use our dirty bitmaps.
> >>
> >>Vmware and Parallels Cloud Server have this feature.
> >What is the advantage of this appraoch over "drive-backup sync=incremental
> >..."?
> 
> This will allow third-party vendors to backup QEMU VMs into
> their own formats or to the cloud etc.

As an example, there is a third-party backup format called VMA from
Proxmox.  A few years ago I posted a proof-of-concept external backup
tool in Python:

https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html

It takes a full backup using drive-backup NBD (plus RAM/device state)
but the same can be done with incremental backups.

Does this NBD approach meet your requirements?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-09 14:21     ` Stefan Hajnoczi
@ 2016-02-09 14:37       ` Denis V. Lunev
  2016-02-09 16:49         ` John Snow
  0 siblings, 1 reply; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-09 14:37 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel,
	armbru, jsnow

On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>> Hi all.
>>>>
>>>> These series which aims to add external backup api. This is needed to allow
>>>> backup software use our dirty bitmaps.
>>>>
>>>> Vmware and Parallels Cloud Server have this feature.
>>> What is the advantage of this appraoch over "drive-backup sync=incremental
>>> ..."?
>> This will allow third-party vendors to backup QEMU VMs into
>> their own formats or to the cloud etc.
> Backup software can implement NBD to receive the incremental blocks from
> QEMU.  Use drive-backup sync=incremental and the backup appliance as the
> NBD target.
>
> It's more complicated to add this QMP command flow to backup software
> than to implement NBD.
>
> Stefan
it can, but this is a matter of problem due to the nature of
how this software is implemented. Usually it is written
in a semi-standard way and it uses "plugins" to actually
collect the data, i.e. the code is written in standard
interface/real implementation pattern and interfaces are
basically the same.

With this standard approach backup software is working
as an active side of the process, i.e. it performs operations
and controls the flow.

This means that "non-standard" QEMU technology will be
pain here.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-09 14:28     ` Stefan Hajnoczi
@ 2016-02-09 14:41       ` Denis V. Lunev
  2016-02-10 10:10         ` Stefan Hajnoczi
  0 siblings, 1 reply; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-09 14:41 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel,
	armbru, jsnow

On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>> Hi all.
>>>>
>>>> These series which aims to add external backup api. This is needed to allow
>>>> backup software use our dirty bitmaps.
>>>>
>>>> Vmware and Parallels Cloud Server have this feature.
>>> What is the advantage of this appraoch over "drive-backup sync=incremental
>>> ..."?
>> This will allow third-party vendors to backup QEMU VMs into
>> their own formats or to the cloud etc.
> As an example, there is a third-party backup format called VMA from
> Proxmox.  A few years ago I posted a proof-of-concept external backup
> tool in Python:
>
> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>
> It takes a full backup using drive-backup NBD (plus RAM/device state)
> but the same can be done with incremental backups.
>
> Does this NBD approach meet your requirements?
>
> Stefan
for us we should somehow provide implementation of
calls posted by Vladimir. They are available in Parallels Server
version 6 and should be available in the next QEMU based
release using "Parallels SDK to libvirt" convertor. The problem
for us is that this old approach is used in the other side
of the product - in containers implementation while this
SDK is a universal access tool to both things.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-09 14:37       ` Denis V. Lunev
@ 2016-02-09 16:49         ` John Snow
  2016-02-09 16:58           ` Denis V. Lunev
  0 siblings, 1 reply; 56+ messages in thread
From: John Snow @ 2016-02-09 16:49 UTC (permalink / raw)
  To: Denis V. Lunev, Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel, armbru



On 02/09/2016 09:37 AM, Denis V. Lunev wrote:
> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>> Hi all.
>>>>>
>>>>> These series which aims to add external backup api. This is needed
>>>>> to allow
>>>>> backup software use our dirty bitmaps.
>>>>>
>>>>> Vmware and Parallels Cloud Server have this feature.
>>>> What is the advantage of this appraoch over "drive-backup
>>>> sync=incremental
>>>> ..."?
>>> This will allow third-party vendors to backup QEMU VMs into
>>> their own formats or to the cloud etc.
>> Backup software can implement NBD to receive the incremental blocks from
>> QEMU.  Use drive-backup sync=incremental and the backup appliance as the
>> NBD target.
>>
>> It's more complicated to add this QMP command flow to backup software
>> than to implement NBD.
>>
>> Stefan
> it can, but this is a matter of problem due to the nature of
> how this software is implemented. Usually it is written
> in a semi-standard way and it uses "plugins" to actually
> collect the data, i.e. the code is written in standard
> interface/real implementation pattern and interfaces are
> basically the same.
> 
> With this standard approach backup software is working
> as an active side of the process, i.e. it performs operations
> and controls the flow.
> 
> This means that "non-standard" QEMU technology will be
> pain here.
> 
> Den

Am I to understand that for e.g. VMWare the backup appliance is
literally reading the disk image from storage directly while the VM is
running?

I'd be a bit surprised if that were true.

My biggest concern here is that there is not a safe way, today, to read
from a disk image atomically while the VM is running. I think that'd
take a lot of work to do and you'll not find a lot of support in
implementing it.

Of course, while the VM is paused/off is a different story, but for now
I still feel like NBD is the right answer for getting block data from QEMU.

What am I missing?

--js

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-09 16:49         ` John Snow
@ 2016-02-09 16:58           ` Denis V. Lunev
  2016-02-09 18:12             ` John Snow
  0 siblings, 1 reply; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-09 16:58 UTC (permalink / raw)
  To: John Snow, Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel, armbru

On 02/09/2016 07:49 PM, John Snow wrote:
>
> On 02/09/2016 09:37 AM, Denis V. Lunev wrote:
>> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>> Hi all.
>>>>>>
>>>>>> These series which aims to add external backup api. This is needed
>>>>>> to allow
>>>>>> backup software use our dirty bitmaps.
>>>>>>
>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>> What is the advantage of this appraoch over "drive-backup
>>>>> sync=incremental
>>>>> ..."?
>>>> This will allow third-party vendors to backup QEMU VMs into
>>>> their own formats or to the cloud etc.
>>> Backup software can implement NBD to receive the incremental blocks from
>>> QEMU.  Use drive-backup sync=incremental and the backup appliance as the
>>> NBD target.
>>>
>>> It's more complicated to add this QMP command flow to backup software
>>> than to implement NBD.
>>>
>>> Stefan
>> it can, but this is a matter of problem due to the nature of
>> how this software is implemented. Usually it is written
>> in a semi-standard way and it uses "plugins" to actually
>> collect the data, i.e. the code is written in standard
>> interface/real implementation pattern and interfaces are
>> basically the same.
>>
>> With this standard approach backup software is working
>> as an active side of the process, i.e. it performs operations
>> and controls the flow.
>>
>> This means that "non-standard" QEMU technology will be
>> pain here.
>>
>> Den
> Am I to understand that for e.g. VMWare the backup appliance is
> literally reading the disk image from storage directly while the VM is
> running?
>
> I'd be a bit surprised if that were true.
I think that backup software is asking alive VM about the data.

> My biggest concern here is that there is not a safe way, today, to read
> from a disk image atomically while the VM is running. I think that'd
> take a lot of work to do and you'll not find a lot of support in
> implementing it.
>
> Of course, while the VM is paused/off is a different story, but for now
> I still feel like NBD is the right answer for getting block data from QEMU.
>
> What am I missing?
>
> --js
In general, in Parallels Server the backup was created using the
following approach:
- create external snapshot. In this case the base image (backing store 
in QEMU terminology)
   will be READ-ONLY and could be safely read by any entity
- backup that read-only disk image (any way)
- merge snapshots

In this process backup software is active while PCS is passive.
With QEMU the approach looks the same to me:
- start a backup
- ask QEMU to give a data to be backuped (using NBD server in QEMU
   with Fam's image fleecing)
- finish backup

Important bit here is that dirty bitmap should be provided
by QEMU by request. This dirty bitmap will be read-only at that
moment, current active dirty bitmap should be replaced with
new at backup start operation.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-09 16:58           ` Denis V. Lunev
@ 2016-02-09 18:12             ` John Snow
  2016-02-09 19:25               ` Denis V. Lunev
  0 siblings, 1 reply; 56+ messages in thread
From: John Snow @ 2016-02-09 18:12 UTC (permalink / raw)
  To: Denis V. Lunev, Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel, armbru



On 02/09/2016 11:58 AM, Denis V. Lunev wrote:
> On 02/09/2016 07:49 PM, John Snow wrote:
>>
>> On 02/09/2016 09:37 AM, Denis V. Lunev wrote:
>>> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> Hi all.
>>>>>>>
>>>>>>> These series which aims to add external backup api. This is needed
>>>>>>> to allow
>>>>>>> backup software use our dirty bitmaps.
>>>>>>>
>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>> What is the advantage of this appraoch over "drive-backup
>>>>>> sync=incremental
>>>>>> ..."?
>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>> their own formats or to the cloud etc.
>>>> Backup software can implement NBD to receive the incremental blocks
>>>> from
>>>> QEMU.  Use drive-backup sync=incremental and the backup appliance as
>>>> the
>>>> NBD target.
>>>>
>>>> It's more complicated to add this QMP command flow to backup software
>>>> than to implement NBD.
>>>>
>>>> Stefan
>>> it can, but this is a matter of problem due to the nature of
>>> how this software is implemented. Usually it is written
>>> in a semi-standard way and it uses "plugins" to actually
>>> collect the data, i.e. the code is written in standard
>>> interface/real implementation pattern and interfaces are
>>> basically the same.
>>>
>>> With this standard approach backup software is working
>>> as an active side of the process, i.e. it performs operations
>>> and controls the flow.
>>>
>>> This means that "non-standard" QEMU technology will be
>>> pain here.
>>>
>>> Den
>> Am I to understand that for e.g. VMWare the backup appliance is
>> literally reading the disk image from storage directly while the VM is
>> running?
>>
>> I'd be a bit surprised if that were true.
> I think that backup software is asking alive VM about the data.
> 
>> My biggest concern here is that there is not a safe way, today, to read
>> from a disk image atomically while the VM is running. I think that'd
>> take a lot of work to do and you'll not find a lot of support in
>> implementing it.
>>
>> Of course, while the VM is paused/off is a different story, but for now
>> I still feel like NBD is the right answer for getting block data from
>> QEMU.
>>
>> What am I missing?
>>
>> --js
> In general, in Parallels Server the backup was created using the
> following approach:
> - create external snapshot. In this case the base image (backing store
> in QEMU terminology)
>   will be READ-ONLY and could be safely read by any entity
> - backup that read-only disk image (any way)
> - merge snapshots
> 

I see.

> In this process backup software is active while PCS is passive.

PCS?

> With QEMU the approach looks the same to me:
> - start a backup
> - ask QEMU to give a data to be backuped (using NBD server in QEMU
>   with Fam's image fleecing)
> - finish backup
> 
> Important bit here is that dirty bitmap should be provided
> by QEMU by request. This dirty bitmap will be read-only at that
> moment, current active dirty bitmap should be replaced with
> new at backup start operation.
> 
> Den

I don't have any problems providing the bitmap data through an external
API, but the part I want to be 100% clear on before I ACK it is the API
portion where we allow an external client to split or merge bitmaps
externally -- that's functionality you don't need if you query the data
from QEMU itself.

If using NBD (either through the incremental backup tool or image
fleecing) that is completely fine, too.

My only reservations are over making direct reads to image files outside
of QEMU while the VM is running.

--js

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-09 18:12             ` John Snow
@ 2016-02-09 19:25               ` Denis V. Lunev
  2016-02-10  8:04                 ` Denis V. Lunev
  0 siblings, 1 reply; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-09 19:25 UTC (permalink / raw)
  To: John Snow, Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel, armbru

On 02/09/2016 09:12 PM, John Snow wrote:
>
> On 02/09/2016 11:58 AM, Denis V. Lunev wrote:
>> On 02/09/2016 07:49 PM, John Snow wrote:
>>> On 02/09/2016 09:37 AM, Denis V. Lunev wrote:
>>>> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
>>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>> Hi all.
>>>>>>>>
>>>>>>>> These series which aims to add external backup api. This is needed
>>>>>>>> to allow
>>>>>>>> backup software use our dirty bitmaps.
>>>>>>>>
>>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>>> What is the advantage of this appraoch over "drive-backup
>>>>>>> sync=incremental
>>>>>>> ..."?
>>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>>> their own formats or to the cloud etc.
>>>>> Backup software can implement NBD to receive the incremental blocks
>>>>> from
>>>>> QEMU.  Use drive-backup sync=incremental and the backup appliance as
>>>>> the
>>>>> NBD target.
>>>>>
>>>>> It's more complicated to add this QMP command flow to backup software
>>>>> than to implement NBD.
>>>>>
>>>>> Stefan
>>>> it can, but this is a matter of problem due to the nature of
>>>> how this software is implemented. Usually it is written
>>>> in a semi-standard way and it uses "plugins" to actually
>>>> collect the data, i.e. the code is written in standard
>>>> interface/real implementation pattern and interfaces are
>>>> basically the same.
>>>>
>>>> With this standard approach backup software is working
>>>> as an active side of the process, i.e. it performs operations
>>>> and controls the flow.
>>>>
>>>> This means that "non-standard" QEMU technology will be
>>>> pain here.
>>>>
>>>> Den
>>> Am I to understand that for e.g. VMWare the backup appliance is
>>> literally reading the disk image from storage directly while the VM is
>>> running?
>>>
>>> I'd be a bit surprised if that were true.
>> I think that backup software is asking alive VM about the data.
>>
>>> My biggest concern here is that there is not a safe way, today, to read
>>> from a disk image atomically while the VM is running. I think that'd
>>> take a lot of work to do and you'll not find a lot of support in
>>> implementing it.
>>>
>>> Of course, while the VM is paused/off is a different story, but for now
>>> I still feel like NBD is the right answer for getting block data from
>>> QEMU.
>>>
>>> What am I missing?
>>>
>>> --js
>> In general, in Parallels Server the backup was created using the
>> following approach:
>> - create external snapshot. In this case the base image (backing store
>> in QEMU terminology)
>>    will be READ-ONLY and could be safely read by any entity
>> - backup that read-only disk image (any way)
>> - merge snapshots
>>
> I see.
>
>> In this process backup software is active while PCS is passive.
> PCS?
Parallels Cloud Server. Sorry for abbreviation :(


>> With QEMU the approach looks the same to me:
>> - start a backup
>> - ask QEMU to give a data to be backuped (using NBD server in QEMU
>>    with Fam's image fleecing)
>> - finish backup
>>
>> Important bit here is that dirty bitmap should be provided
>> by QEMU by request. This dirty bitmap will be read-only at that
>> moment, current active dirty bitmap should be replaced with
>> new at backup start operation.
>>
>> Den
> I don't have any problems providing the bitmap data through an external
> API, but the part I want to be 100% clear on before I ACK it is the API
> portion where we allow an external client to split or merge bitmaps
> externally -- that's functionality you don't need if you query the data
> from QEMU itself.
That is fine :)

> If using NBD (either through the incremental backup tool or image
> fleecing) that is completely fine, too.
>
> My only reservations are over making direct reads to image files outside
> of QEMU while the VM is running.

Perfect :) We are on the same page. I would not
going to go something unsafe is this very sensitive
area.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-09 19:25               ` Denis V. Lunev
@ 2016-02-10  8:04                 ` Denis V. Lunev
  0 siblings, 0 replies; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-10  8:04 UTC (permalink / raw)
  To: Denis V. Lunev, John Snow, Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel, armbru

On 02/09/2016 10:25 PM, Denis V. Lunev wrote:
> On 02/09/2016 09:12 PM, John Snow wrote:
>>
>> On 02/09/2016 11:58 AM, Denis V. Lunev wrote:
>>> On 02/09/2016 07:49 PM, John Snow wrote:
>>>> On 02/09/2016 09:37 AM, Denis V. Lunev wrote:
>>>>> On 02/09/2016 05:21 PM, Stefan Hajnoczi wrote:
>>>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>> Hi all.
>>>>>>>>>
>>>>>>>>> These series which aims to add external backup api. This is 
>>>>>>>>> needed
>>>>>>>>> to allow
>>>>>>>>> backup software use our dirty bitmaps.
>>>>>>>>>
>>>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>>>> What is the advantage of this appraoch over "drive-backup
>>>>>>>> sync=incremental
>>>>>>>> ..."?
>>>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>>>> their own formats or to the cloud etc.
>>>>>> Backup software can implement NBD to receive the incremental blocks
>>>>>> from
>>>>>> QEMU.  Use drive-backup sync=incremental and the backup appliance as
>>>>>> the
>>>>>> NBD target.
>>>>>>
>>>>>> It's more complicated to add this QMP command flow to backup 
>>>>>> software
>>>>>> than to implement NBD.
>>>>>>
>>>>>> Stefan
>>>>> it can, but this is a matter of problem due to the nature of
>>>>> how this software is implemented. Usually it is written
>>>>> in a semi-standard way and it uses "plugins" to actually
>>>>> collect the data, i.e. the code is written in standard
>>>>> interface/real implementation pattern and interfaces are
>>>>> basically the same.
>>>>>
>>>>> With this standard approach backup software is working
>>>>> as an active side of the process, i.e. it performs operations
>>>>> and controls the flow.
>>>>>
>>>>> This means that "non-standard" QEMU technology will be
>>>>> pain here.
>>>>>
>>>>> Den
>>>> Am I to understand that for e.g. VMWare the backup appliance is
>>>> literally reading the disk image from storage directly while the VM is
>>>> running?
>>>>
>>>> I'd be a bit surprised if that were true.
>>> I think that backup software is asking alive VM about the data.
>>>
>>>> My biggest concern here is that there is not a safe way, today, to 
>>>> read
>>>> from a disk image atomically while the VM is running. I think that'd
>>>> take a lot of work to do and you'll not find a lot of support in
>>>> implementing it.
>>>>
>>>> Of course, while the VM is paused/off is a different story, but for 
>>>> now
>>>> I still feel like NBD is the right answer for getting block data from
>>>> QEMU.
>>>>
>>>> What am I missing?
>>>>
>>>> --js
>>> In general, in Parallels Server the backup was created using the
>>> following approach:
>>> - create external snapshot. In this case the base image (backing store
>>> in QEMU terminology)
>>>    will be READ-ONLY and could be safely read by any entity
>>> - backup that read-only disk image (any way)
>>> - merge snapshots
>>>
>> I see.
>>
>>> In this process backup software is active while PCS is passive.
>> PCS?
> Parallels Cloud Server. Sorry for abbreviation :(
>
>
>>> With QEMU the approach looks the same to me:
>>> - start a backup
>>> - ask QEMU to give a data to be backuped (using NBD server in QEMU
>>>    with Fam's image fleecing)
>>> - finish backup
>>>
>>> Important bit here is that dirty bitmap should be provided
>>> by QEMU by request. This dirty bitmap will be read-only at that
>>> moment, current active dirty bitmap should be replaced with
>>> new at backup start operation.
>>>
>>> Den
>> I don't have any problems providing the bitmap data through an external
>> API, but the part I want to be 100% clear on before I ACK it is the API
>> portion where we allow an external client to split or merge bitmaps
>> externally -- that's functionality you don't need if you query the data
>> from QEMU itself.
> That is fine :)
OOPS. This could be mistake!

The client should not perform operations on the bitmap manually,
but it should instruct QEMU to do this job for him. Bitmap child
should be created by request of backup software and its existence
should also be controlled by the backup software. This could be either
hidden or not, but the state should be controlled.

Lets again discuss the sequence:
1. call guest-fsfreeze-freeze to make consistent backup. This means
     that the guest ensures that all its journals are committed and
     there are no pending writes. Filesystem is in a good state.

2. start backup. At this moment original dirty bitmap child should
   be created to track new writes. There are no writes from the
   guest thanks to step (1). Thus we could avoid to move QEMU
   to VM_PAUSED state to perform the operation. This requires to
   2.1 start image fleecing
   2.2 create bitmap child (successor)

3. call guest-fsfreeze-thaw to unfreeze the guest

4. get dirty bitmap (parent). It is read-only and could be safely extracted

5. collect data from QEMU using NBD server inside QEMU (image
     fleecing)

6a. finish backup successfully. This implies the following ops:
    6a.1  remove fleecing image
    6a.2 drop old dirty (parent) bitmap (abdicate)

6b. finish backup unsuccessfully:
    6b.1 remove fleecing image
    6b.2(a) merge child to parent (reclaim)
or alternatively
    6b.2(b) drop both child and parent dirty bitmaps and stop tracking
                 to facilitate new full backup next round

Constraints:
- step (4) is slow. We do not want to do it before step (3)
- paired operations aka (6.1)/(6.2) and (2.1)/(2.2) could be
   done either by the single QMP command or by two different
   commands, we do not care much on this

There are also troubles on guest snapshot operations. For now we
drop dirty bitmaps in Parallels Server on switch-to-snapshot
operations as it is unclear how to properly make dirty bitmap for
the case.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges
  2016-01-30 10:56 ` [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges Vladimir Sementsov-Ogievskiy
@ 2016-02-10 10:08   ` Stefan Hajnoczi
  2016-02-10 13:57     ` Denis V. Lunev
  0 siblings, 1 reply; 56+ messages in thread
From: Stefan Hajnoczi @ 2016-02-10 10:08 UTC (permalink / raw)
  To: Vladimir Sementsov-Ogievskiy; +Cc: kwolf, famz, den, armbru, qemu-devel, jsnow

[-- Attachment #1: Type: text/plain, Size: 1109 bytes --]

On Sat, Jan 30, 2016 at 01:56:30PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> Add qmp command to query dirty bitmap contents. This is needed for
> external backup.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/dirty-bitmap.c         | 55 +++++++++++++++++++++++++++++++++++++++
>  blockdev.c                   | 62 ++++++++++++++++++++++++++++++++++++++++++++
>  include/block/dirty-bitmap.h |  7 +++++
>  qapi/block-core.json         | 54 ++++++++++++++++++++++++++++++++++++++
>  qmp-commands.hx              | 33 +++++++++++++++++++++++
>  5 files changed, 211 insertions(+)

This API produces large replies and/or requires many calls to fetch all
bitmap data.  The worst case is a 101010... bitmap.

I consider the dirty bitmap to be data (vs control) and not something
that should be sent over a control channel like the QMP monitor.

How about writing the dirty bitmap to a file?  The new bitmap file
format that Fam is working on could be used.  That way the dirty bitmap
can be saved asynchronously without hogging the QMP monitor.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-09 14:41       ` Denis V. Lunev
@ 2016-02-10 10:10         ` Stefan Hajnoczi
  2016-02-16 17:09           ` Stefan Hajnoczi
  0 siblings, 1 reply; 56+ messages in thread
From: Stefan Hajnoczi @ 2016-02-10 10:10 UTC (permalink / raw)
  To: Denis V. Lunev
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel,
	armbru, jsnow

[-- Attachment #1: Type: text/plain, Size: 1690 bytes --]

On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> >On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> >>On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>>>Hi all.
> >>>>
> >>>>These series which aims to add external backup api. This is needed to allow
> >>>>backup software use our dirty bitmaps.
> >>>>
> >>>>Vmware and Parallels Cloud Server have this feature.
> >>>What is the advantage of this appraoch over "drive-backup sync=incremental
> >>>..."?
> >>This will allow third-party vendors to backup QEMU VMs into
> >>their own formats or to the cloud etc.
> >As an example, there is a third-party backup format called VMA from
> >Proxmox.  A few years ago I posted a proof-of-concept external backup
> >tool in Python:
> >
> >https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> >
> >It takes a full backup using drive-backup NBD (plus RAM/device state)
> >but the same can be done with incremental backups.
> >
> >Does this NBD approach meet your requirements?
> >
> >Stefan
> for us we should somehow provide implementation of
> calls posted by Vladimir. They are available in Parallels Server
> version 6 and should be available in the next QEMU based
> release using "Parallels SDK to libvirt" convertor. The problem
> for us is that this old approach is used in the other side
> of the product - in containers implementation while this
> SDK is a universal access tool to both things.

Point taken.  I think many other backup applications will expect a
similar API, so it's pragmatic to provide something compatible.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges
  2016-02-10 10:08   ` Stefan Hajnoczi
@ 2016-02-10 13:57     ` Denis V. Lunev
  2016-02-10 15:26       ` John Snow
  2016-02-14  5:05       ` Fam Zheng
  0 siblings, 2 replies; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-10 13:57 UTC (permalink / raw)
  To: Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy
  Cc: kwolf, jsnow, famz, qemu-devel, armbru

On 02/10/2016 01:08 PM, Stefan Hajnoczi wrote:
> On Sat, Jan 30, 2016 at 01:56:30PM +0300, Vladimir Sementsov-Ogievskiy wrote:
>> Add qmp command to query dirty bitmap contents. This is needed for
>> external backup.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   block/dirty-bitmap.c         | 55 +++++++++++++++++++++++++++++++++++++++
>>   blockdev.c                   | 62 ++++++++++++++++++++++++++++++++++++++++++++
>>   include/block/dirty-bitmap.h |  7 +++++
>>   qapi/block-core.json         | 54 ++++++++++++++++++++++++++++++++++++++
>>   qmp-commands.hx              | 33 +++++++++++++++++++++++
>>   5 files changed, 211 insertions(+)
> This API produces large replies and/or requires many calls to fetch all
> bitmap data.  The worst case is a 101010... bitmap.
>
> I consider the dirty bitmap to be data (vs control) and not something
> that should be sent over a control channel like the QMP monitor.
>
> How about writing the dirty bitmap to a file?  The new bitmap file
> format that Fam is working on could be used.  That way the dirty bitmap
> can be saved asynchronously without hogging the QMP monitor.
Reasonable point.

May be it would be better to setup "special" NBD server inside
QEMU which will allow to directly "read" bitmap data.

Any opinion?

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges
  2016-02-10 13:57     ` Denis V. Lunev
@ 2016-02-10 15:26       ` John Snow
  2016-02-10 15:36         ` Denis V. Lunev
  2016-02-14  5:05       ` Fam Zheng
  1 sibling, 1 reply; 56+ messages in thread
From: John Snow @ 2016-02-10 15:26 UTC (permalink / raw)
  To: Denis V. Lunev, Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy
  Cc: kwolf, famz, qemu-devel, armbru



On 02/10/2016 08:57 AM, Denis V. Lunev wrote:
> On 02/10/2016 01:08 PM, Stefan Hajnoczi wrote:
>> On Sat, Jan 30, 2016 at 01:56:30PM +0300, Vladimir Sementsov-Ogievskiy
>> wrote:
>>> Add qmp command to query dirty bitmap contents. This is needed for
>>> external backup.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>   block/dirty-bitmap.c         | 55
>>> +++++++++++++++++++++++++++++++++++++++
>>>   blockdev.c                   | 62
>>> ++++++++++++++++++++++++++++++++++++++++++++
>>>   include/block/dirty-bitmap.h |  7 +++++
>>>   qapi/block-core.json         | 54
>>> ++++++++++++++++++++++++++++++++++++++
>>>   qmp-commands.hx              | 33 +++++++++++++++++++++++
>>>   5 files changed, 211 insertions(+)
>> This API produces large replies and/or requires many calls to fetch all
>> bitmap data.  The worst case is a 101010... bitmap.
>>
>> I consider the dirty bitmap to be data (vs control) and not something
>> that should be sent over a control channel like the QMP monitor.
>>
>> How about writing the dirty bitmap to a file?  The new bitmap file
>> format that Fam is working on could be used.  That way the dirty bitmap
>> can be saved asynchronously without hogging the QMP monitor.
> Reasonable point.
> 
> May be it would be better to setup "special" NBD server inside
> QEMU which will allow to directly "read" bitmap data.
> 
> Any opinion?
> 
> Den

Or perhaps something like migration, where the client receiving the data
opens a socket of some sort, and QEMU connects to that socket to send
the data.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges
  2016-02-10 15:26       ` John Snow
@ 2016-02-10 15:36         ` Denis V. Lunev
  2016-02-10 15:37           ` John Snow
  0 siblings, 1 reply; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-10 15:36 UTC (permalink / raw)
  To: John Snow, Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy
  Cc: kwolf, famz, qemu-devel, armbru

On 02/10/2016 06:26 PM, John Snow wrote:
>
> On 02/10/2016 08:57 AM, Denis V. Lunev wrote:
>> On 02/10/2016 01:08 PM, Stefan Hajnoczi wrote:
>>> On Sat, Jan 30, 2016 at 01:56:30PM +0300, Vladimir Sementsov-Ogievskiy
>>> wrote:
>>>> Add qmp command to query dirty bitmap contents. This is needed for
>>>> external backup.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>>    block/dirty-bitmap.c         | 55
>>>> +++++++++++++++++++++++++++++++++++++++
>>>>    blockdev.c                   | 62
>>>> ++++++++++++++++++++++++++++++++++++++++++++
>>>>    include/block/dirty-bitmap.h |  7 +++++
>>>>    qapi/block-core.json         | 54
>>>> ++++++++++++++++++++++++++++++++++++++
>>>>    qmp-commands.hx              | 33 +++++++++++++++++++++++
>>>>    5 files changed, 211 insertions(+)
>>> This API produces large replies and/or requires many calls to fetch all
>>> bitmap data.  The worst case is a 101010... bitmap.
>>>
>>> I consider the dirty bitmap to be data (vs control) and not something
>>> that should be sent over a control channel like the QMP monitor.
>>>
>>> How about writing the dirty bitmap to a file?  The new bitmap file
>>> format that Fam is working on could be used.  That way the dirty bitmap
>>> can be saved asynchronously without hogging the QMP monitor.
>> Reasonable point.
>>
>> May be it would be better to setup "special" NBD server inside
>> QEMU which will allow to directly "read" bitmap data.
>>
>> Any opinion?
>>
>> Den
> Or perhaps something like migration, where the client receiving the data
> opens a socket of some sort, and QEMU connects to that socket to send
> the data.
no. The point is that QEMU should be queried for data.
May be even via several sockets to provide it in
parallel.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges
  2016-02-10 15:36         ` Denis V. Lunev
@ 2016-02-10 15:37           ` John Snow
  2016-02-10 15:40             ` Denis V. Lunev
  0 siblings, 1 reply; 56+ messages in thread
From: John Snow @ 2016-02-10 15:37 UTC (permalink / raw)
  To: Denis V. Lunev, Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy
  Cc: kwolf, famz, qemu-devel, armbru



On 02/10/2016 10:36 AM, Denis V. Lunev wrote:
> On 02/10/2016 06:26 PM, John Snow wrote:
>>
>> On 02/10/2016 08:57 AM, Denis V. Lunev wrote:
>>> On 02/10/2016 01:08 PM, Stefan Hajnoczi wrote:
>>>> On Sat, Jan 30, 2016 at 01:56:30PM +0300, Vladimir Sementsov-Ogievskiy
>>>> wrote:
>>>>> Add qmp command to query dirty bitmap contents. This is needed for
>>>>> external backup.
>>>>>
>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>> ---
>>>>>    block/dirty-bitmap.c         | 55
>>>>> +++++++++++++++++++++++++++++++++++++++
>>>>>    blockdev.c                   | 62
>>>>> ++++++++++++++++++++++++++++++++++++++++++++
>>>>>    include/block/dirty-bitmap.h |  7 +++++
>>>>>    qapi/block-core.json         | 54
>>>>> ++++++++++++++++++++++++++++++++++++++
>>>>>    qmp-commands.hx              | 33 +++++++++++++++++++++++
>>>>>    5 files changed, 211 insertions(+)
>>>> This API produces large replies and/or requires many calls to fetch all
>>>> bitmap data.  The worst case is a 101010... bitmap.
>>>>
>>>> I consider the dirty bitmap to be data (vs control) and not something
>>>> that should be sent over a control channel like the QMP monitor.
>>>>
>>>> How about writing the dirty bitmap to a file?  The new bitmap file
>>>> format that Fam is working on could be used.  That way the dirty bitmap
>>>> can be saved asynchronously without hogging the QMP monitor.
>>> Reasonable point.
>>>
>>> May be it would be better to setup "special" NBD server inside
>>> QEMU which will allow to directly "read" bitmap data.
>>>
>>> Any opinion?
>>>
>>> Den
>> Or perhaps something like migration, where the client receiving the data
>> opens a socket of some sort, and QEMU connects to that socket to send
>> the data.
> no. The point is that QEMU should be queried for data.
> May be even via several sockets to provide it in
> parallel.
> 
> Den

I don't follow.

You'd use a QMP command to tell QEMU where to connect to send the data.
You're still "querying" QEMU, it's just not acting as the server for the
data channel.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges
  2016-02-10 15:37           ` John Snow
@ 2016-02-10 15:40             ` Denis V. Lunev
  0 siblings, 0 replies; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-10 15:40 UTC (permalink / raw)
  To: John Snow, Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy
  Cc: kwolf, famz, qemu-devel, armbru

On 02/10/2016 06:37 PM, John Snow wrote:
>
> On 02/10/2016 10:36 AM, Denis V. Lunev wrote:
>> On 02/10/2016 06:26 PM, John Snow wrote:
>>> On 02/10/2016 08:57 AM, Denis V. Lunev wrote:
>>>> On 02/10/2016 01:08 PM, Stefan Hajnoczi wrote:
>>>>> On Sat, Jan 30, 2016 at 01:56:30PM +0300, Vladimir Sementsov-Ogievskiy
>>>>> wrote:
>>>>>> Add qmp command to query dirty bitmap contents. This is needed for
>>>>>> external backup.
>>>>>>
>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>>> ---
>>>>>>     block/dirty-bitmap.c         | 55
>>>>>> +++++++++++++++++++++++++++++++++++++++
>>>>>>     blockdev.c                   | 62
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++
>>>>>>     include/block/dirty-bitmap.h |  7 +++++
>>>>>>     qapi/block-core.json         | 54
>>>>>> ++++++++++++++++++++++++++++++++++++++
>>>>>>     qmp-commands.hx              | 33 +++++++++++++++++++++++
>>>>>>     5 files changed, 211 insertions(+)
>>>>> This API produces large replies and/or requires many calls to fetch all
>>>>> bitmap data.  The worst case is a 101010... bitmap.
>>>>>
>>>>> I consider the dirty bitmap to be data (vs control) and not something
>>>>> that should be sent over a control channel like the QMP monitor.
>>>>>
>>>>> How about writing the dirty bitmap to a file?  The new bitmap file
>>>>> format that Fam is working on could be used.  That way the dirty bitmap
>>>>> can be saved asynchronously without hogging the QMP monitor.
>>>> Reasonable point.
>>>>
>>>> May be it would be better to setup "special" NBD server inside
>>>> QEMU which will allow to directly "read" bitmap data.
>>>>
>>>> Any opinion?
>>>>
>>>> Den
>>> Or perhaps something like migration, where the client receiving the data
>>> opens a socket of some sort, and QEMU connects to that socket to send
>>> the data.
>> no. The point is that QEMU should be queried for data.
>> May be even via several sockets to provide it in
>> parallel.
>>
>> Den
> I don't follow.
>
> You'd use a QMP command to tell QEMU where to connect to send the data.
> You're still "querying" QEMU, it's just not acting as the server for the
> data channel.
ok, I have understood you wrong. This looks working
at the first glance.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges
  2016-02-10 13:57     ` Denis V. Lunev
  2016-02-10 15:26       ` John Snow
@ 2016-02-14  5:05       ` Fam Zheng
  1 sibling, 0 replies; 56+ messages in thread
From: Fam Zheng @ 2016-02-14  5:05 UTC (permalink / raw)
  To: Denis V. Lunev
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Stefan Hajnoczi, qemu-devel,
	armbru, jsnow

On Wed, 02/10 16:57, Denis V. Lunev wrote:
> On 02/10/2016 01:08 PM, Stefan Hajnoczi wrote:
> >On Sat, Jan 30, 2016 at 01:56:30PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> >>Add qmp command to query dirty bitmap contents. This is needed for
> >>external backup.
> >>
> >>Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> >>---
> >>  block/dirty-bitmap.c         | 55 +++++++++++++++++++++++++++++++++++++++
> >>  blockdev.c                   | 62 ++++++++++++++++++++++++++++++++++++++++++++
> >>  include/block/dirty-bitmap.h |  7 +++++
> >>  qapi/block-core.json         | 54 ++++++++++++++++++++++++++++++++++++++
> >>  qmp-commands.hx              | 33 +++++++++++++++++++++++
> >>  5 files changed, 211 insertions(+)
> >This API produces large replies and/or requires many calls to fetch all
> >bitmap data.  The worst case is a 101010... bitmap.
> >
> >I consider the dirty bitmap to be data (vs control) and not something
> >that should be sent over a control channel like the QMP monitor.
> >
> >How about writing the dirty bitmap to a file?  The new bitmap file
> >format that Fam is working on could be used.  That way the dirty bitmap
> >can be saved asynchronously without hogging the QMP monitor.
> Reasonable point.
> 
> May be it would be better to setup "special" NBD server inside
> QEMU which will allow to directly "read" bitmap data.
> 
> Any opinion?

Since Stefan has mentioned "the format" I'm working on, yes, I think it will be
possible to expose the persistent bitmap through NBD if the driver assigns
node-names to the bitmap BDS.

Let me prototype this on top of my branch.

Fam

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-10 10:10         ` Stefan Hajnoczi
@ 2016-02-16 17:09           ` Stefan Hajnoczi
  2016-02-16 17:17             ` Vladimir Sementsov-Ogievskiy
                               ` (2 more replies)
  0 siblings, 3 replies; 56+ messages in thread
From: Stefan Hajnoczi @ 2016-02-16 17:09 UTC (permalink / raw)
  To: Denis V. Lunev
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel,
	armbru, jsnow

[-- Attachment #1: Type: text/plain, Size: 3262 bytes --]

On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> > On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> > >On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> > >>On 02/03/2016 11:14 AM, Fam Zheng wrote:
> > >>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> > >>>>Hi all.
> > >>>>
> > >>>>These series which aims to add external backup api. This is needed to allow
> > >>>>backup software use our dirty bitmaps.
> > >>>>
> > >>>>Vmware and Parallels Cloud Server have this feature.
> > >>>What is the advantage of this appraoch over "drive-backup sync=incremental
> > >>>..."?
> > >>This will allow third-party vendors to backup QEMU VMs into
> > >>their own formats or to the cloud etc.
> > >As an example, there is a third-party backup format called VMA from
> > >Proxmox.  A few years ago I posted a proof-of-concept external backup
> > >tool in Python:
> > >
> > >https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> > >
> > >It takes a full backup using drive-backup NBD (plus RAM/device state)
> > >but the same can be done with incremental backups.
> > >
> > >Does this NBD approach meet your requirements?
> > >
> > >Stefan
> > for us we should somehow provide implementation of
> > calls posted by Vladimir. They are available in Parallels Server
> > version 6 and should be available in the next QEMU based
> > release using "Parallels SDK to libvirt" convertor. The problem
> > for us is that this old approach is used in the other side
> > of the product - in containers implementation while this
> > SDK is a universal access tool to both things.
> 
> Point taken.  I think many other backup applications will expect a
> similar API, so it's pragmatic to provide something compatible.

Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
concerns about the QMP monitor:

Previously I described incremental backup in "push" mode (already
supported today with drive-backup).  QEMU connects to a remote NBD
server and writes out the contents of all dirty blocks:

  QEMU ---Write dirty blocks--> Backup appliance (server)

This doesn't lend itself well to existing backup applications that
expect to iterate the dirty bitmap manually.

Let's add a "pull" mode where the connection of the NBD connection is
reversed.  The backup application connects to QEMU's NBD server (image
fleecing).  The NBD protocol is extended to support the SCSI Get LBA
Status command for querying block provisioning information.  Now the
backup application can use Get LBA Status to fetch the dirty block
information from QEMU.

  QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance

The dirty block information goes over the same NBD connection used to
read the contents of the dirty blocks.  The QMP monitor is not used to
transfer dirty block information.

It may be necessary to extend the nbd-server-add command so that a
bitmap name can be passed.  This bitmap will be used to answer Get LBA
Status queries instead of using on bdrv_co_get_block_status().  This
would be necessary if image fleecing (point in time snapshot) is used.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-16 17:09           ` Stefan Hajnoczi
@ 2016-02-16 17:17             ` Vladimir Sementsov-Ogievskiy
  2016-02-16 17:20             ` Denis V. Lunev
  2016-02-17 17:47             ` Vladimir Sementsov-Ogievskiy
  2 siblings, 0 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-02-16 17:17 UTC (permalink / raw)
  To: Stefan Hajnoczi, Denis V. Lunev
  Cc: kwolf, jsnow, Fam Zheng, qemu-devel, armbru

On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
>> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
>>> On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> Hi all.
>>>>>>>
>>>>>>> These series which aims to add external backup api. This is needed to allow
>>>>>>> backup software use our dirty bitmaps.
>>>>>>>
>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>> What is the advantage of this appraoch over "drive-backup sync=incremental
>>>>>> ..."?
>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>> their own formats or to the cloud etc.
>>>> As an example, there is a third-party backup format called VMA from
>>>> Proxmox.  A few years ago I posted a proof-of-concept external backup
>>>> tool in Python:
>>>>
>>>> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>>>>
>>>> It takes a full backup using drive-backup NBD (plus RAM/device state)
>>>> but the same can be done with incremental backups.
>>>>
>>>> Does this NBD approach meet your requirements?
>>>>
>>>> Stefan
>>> for us we should somehow provide implementation of
>>> calls posted by Vladimir. They are available in Parallels Server
>>> version 6 and should be available in the next QEMU based
>>> release using "Parallels SDK to libvirt" convertor. The problem
>>> for us is that this old approach is used in the other side
>>> of the product - in containers implementation while this
>>> SDK is a universal access tool to both things.
>> Point taken.  I think many other backup applications will expect a
>> similar API, so it's pragmatic to provide something compatible.
> Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> concerns about the QMP monitor:
>
> Previously I described incremental backup in "push" mode (already
> supported today with drive-backup).  QEMU connects to a remote NBD
> server and writes out the contents of all dirty blocks:
>
>    QEMU ---Write dirty blocks--> Backup appliance (server)
>
> This doesn't lend itself well to existing backup applications that
> expect to iterate the dirty bitmap manually.
>
> Let's add a "pull" mode where the connection of the NBD connection is
> reversed.  The backup application connects to QEMU's NBD server (image
> fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> Status command for querying block provisioning information.  Now the
> backup application can use Get LBA Status to fetch the dirty block
> information from QEMU.
>
>    QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
>
> The dirty block information goes over the same NBD connection used to
> read the contents of the dirty blocks.  The QMP monitor is not used to
> transfer dirty block information.
>
> It may be necessary to extend the nbd-server-add command so that a
> bitmap name can be passed.  This bitmap will be used to answer Get LBA
> Status queries instead of using on bdrv_co_get_block_status().  This
> would be necessary if image fleecing (point in time snapshot) is used.
>
> Stefan

Looks much easier than implement nbd-server for dirty bitmap, thanks.

-- 
Best regards,
Vladimir

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-16 17:09           ` Stefan Hajnoczi
  2016-02-16 17:17             ` Vladimir Sementsov-Ogievskiy
@ 2016-02-16 17:20             ` Denis V. Lunev
  2016-02-18 16:39               ` Stefan Hajnoczi
  2016-02-17 17:47             ` Vladimir Sementsov-Ogievskiy
  2 siblings, 1 reply; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-16 17:20 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel,
	armbru, jsnow

On 02/16/2016 08:09 PM, Stefan Hajnoczi wrote:
> On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
>> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
>>> On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> Hi all.
>>>>>>>
>>>>>>> These series which aims to add external backup api. This is needed to allow
>>>>>>> backup software use our dirty bitmaps.
>>>>>>>
>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>> What is the advantage of this appraoch over "drive-backup sync=incremental
>>>>>> ..."?
>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>> their own formats or to the cloud etc.
>>>> As an example, there is a third-party backup format called VMA from
>>>> Proxmox.  A few years ago I posted a proof-of-concept external backup
>>>> tool in Python:
>>>>
>>>> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>>>>
>>>> It takes a full backup using drive-backup NBD (plus RAM/device state)
>>>> but the same can be done with incremental backups.
>>>>
>>>> Does this NBD approach meet your requirements?
>>>>
>>>> Stefan
>>> for us we should somehow provide implementation of
>>> calls posted by Vladimir. They are available in Parallels Server
>>> version 6 and should be available in the next QEMU based
>>> release using "Parallels SDK to libvirt" convertor. The problem
>>> for us is that this old approach is used in the other side
>>> of the product - in containers implementation while this
>>> SDK is a universal access tool to both things.
>> Point taken.  I think many other backup applications will expect a
>> similar API, so it's pragmatic to provide something compatible.
> Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> concerns about the QMP monitor:
>
> Previously I described incremental backup in "push" mode (already
> supported today with drive-backup).  QEMU connects to a remote NBD
> server and writes out the contents of all dirty blocks:
>
>    QEMU ---Write dirty blocks--> Backup appliance (server)
>
> This doesn't lend itself well to existing backup applications that
> expect to iterate the dirty bitmap manually.
>
> Let's add a "pull" mode where the connection of the NBD connection is
> reversed.  The backup application connects to QEMU's NBD server (image
> fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> Status command for querying block provisioning information.  Now the
> backup application can use Get LBA Status to fetch the dirty block
> information from QEMU.
>
>    QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
>
> The dirty block information goes over the same NBD connection used to
> read the contents of the dirty blocks.  The QMP monitor is not used to
> transfer dirty block information.
>
> It may be necessary to extend the nbd-server-add command so that a
> bitmap name can be passed.  This bitmap will be used to answer Get LBA
> Status queries instead of using on bdrv_co_get_block_status().  This
> would be necessary if image fleecing (point in time snapshot) is used.
>
> Stefan
the idea seems feasible at the first glance. I'll need to check
the amount of the overhead of this approach. If we'll need
to perform an additional request for each <granularity>
block - the overhead is toooooo much.

Do you have the link to the spec/implementation?

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-16 17:09           ` Stefan Hajnoczi
  2016-02-16 17:17             ` Vladimir Sementsov-Ogievskiy
  2016-02-16 17:20             ` Denis V. Lunev
@ 2016-02-17 17:47             ` Vladimir Sementsov-Ogievskiy
  2016-02-18  0:59               ` Fam Zheng
  2016-02-18 12:11               ` Daniel P. Berrange
  2 siblings, 2 replies; 56+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2016-02-17 17:47 UTC (permalink / raw)
  To: Stefan Hajnoczi, Denis V. Lunev
  Cc: kwolf, jsnow, Fam Zheng, qemu-devel, armbru

On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
>> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
>>> On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>> Hi all.
>>>>>>>
>>>>>>> These series which aims to add external backup api. This is needed to allow
>>>>>>> backup software use our dirty bitmaps.
>>>>>>>
>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>> What is the advantage of this appraoch over "drive-backup sync=incremental
>>>>>> ..."?
>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>> their own formats or to the cloud etc.
>>>> As an example, there is a third-party backup format called VMA from
>>>> Proxmox.  A few years ago I posted a proof-of-concept external backup
>>>> tool in Python:
>>>>
>>>> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>>>>
>>>> It takes a full backup using drive-backup NBD (plus RAM/device state)
>>>> but the same can be done with incremental backups.
>>>>
>>>> Does this NBD approach meet your requirements?
>>>>
>>>> Stefan
>>> for us we should somehow provide implementation of
>>> calls posted by Vladimir. They are available in Parallels Server
>>> version 6 and should be available in the next QEMU based
>>> release using "Parallels SDK to libvirt" convertor. The problem
>>> for us is that this old approach is used in the other side
>>> of the product - in containers implementation while this
>>> SDK is a universal access tool to both things.
>> Point taken.  I think many other backup applications will expect a
>> similar API, so it's pragmatic to provide something compatible.
> Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> concerns about the QMP monitor:
>
> Previously I described incremental backup in "push" mode (already
> supported today with drive-backup).  QEMU connects to a remote NBD
> server and writes out the contents of all dirty blocks:
>
>    QEMU ---Write dirty blocks--> Backup appliance (server)
>
> This doesn't lend itself well to existing backup applications that
> expect to iterate the dirty bitmap manually.
>
> Let's add a "pull" mode where the connection of the NBD connection is
> reversed.  The backup application connects to QEMU's NBD server (image
> fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> Status command for querying block provisioning information.  Now the
> backup application can use Get LBA Status to fetch the dirty block
> information from QEMU.
>
>    QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
>
> The dirty block information goes over the same NBD connection used to
> read the contents of the dirty blocks.  The QMP monitor is not used to
> transfer dirty block information.
>
> It may be necessary to extend the nbd-server-add command so that a
> bitmap name can be passed.  This bitmap will be used to answer Get LBA
> Status queries instead of using on bdrv_co_get_block_status().  This
> would be necessary if image fleecing (point in time snapshot) is used.
>
> Stefan

There are no such commands in nbd spec here:

https://github.com/yoe/nbd/blob/master/doc/proto.md


So, I'm not sure, that adding something qemu-specific to this external 
protocol will be simple or even true way. Is Qemu already extending 
original nbd?

What about exporting bitmap as separate nbd entity? Just implement block 
driver, which will read from bitmap? If consider only read access and 
disabled (or frozen) bitmaps it should be simple enough.

-- 
Best regards,
Vladimir

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-17 17:47             ` Vladimir Sementsov-Ogievskiy
@ 2016-02-18  0:59               ` Fam Zheng
  2016-02-18 12:11               ` Daniel P. Berrange
  1 sibling, 0 replies; 56+ messages in thread
From: Fam Zheng @ 2016-02-18  0:59 UTC (permalink / raw)
  To: Vladimir Sementsov-Ogievskiy
  Cc: kwolf, Denis V. Lunev, Stefan Hajnoczi, armbru, qemu-devel, jsnow

On Wed, 02/17 20:47, Vladimir Sementsov-Ogievskiy wrote:
> What about exporting bitmap as separate nbd entity? Just implement
> block driver, which will read from bitmap? If consider only read
> access and disabled (or frozen) bitmaps it should be simple enough.

Yes, I think this idea also makes sense. The driver could implement .bdrv_read
by memcpy from the bitmap.

Fam

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-17 17:47             ` Vladimir Sementsov-Ogievskiy
  2016-02-18  0:59               ` Fam Zheng
@ 2016-02-18 12:11               ` Daniel P. Berrange
  2016-02-18 16:41                 ` Stefan Hajnoczi
  1 sibling, 1 reply; 56+ messages in thread
From: Daniel P. Berrange @ 2016-02-18 12:11 UTC (permalink / raw)
  To: Vladimir Sementsov-Ogievskiy
  Cc: kwolf, Fam Zheng, Denis V. Lunev, Stefan Hajnoczi, qemu-devel,
	armbru, jsnow

On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> >On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
> >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> >>>>On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> >>>>>On 02/03/2016 11:14 AM, Fam Zheng wrote:
> >>>>>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> >>>>>>>Hi all.
> >>>>>>>
> >>>>>>>These series which aims to add external backup api. This is needed to allow
> >>>>>>>backup software use our dirty bitmaps.
> >>>>>>>
> >>>>>>>Vmware and Parallels Cloud Server have this feature.
> >>>>>>What is the advantage of this appraoch over "drive-backup sync=incremental
> >>>>>>..."?
> >>>>>This will allow third-party vendors to backup QEMU VMs into
> >>>>>their own formats or to the cloud etc.
> >>>>As an example, there is a third-party backup format called VMA from
> >>>>Proxmox.  A few years ago I posted a proof-of-concept external backup
> >>>>tool in Python:
> >>>>
> >>>>https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> >>>>
> >>>>It takes a full backup using drive-backup NBD (plus RAM/device state)
> >>>>but the same can be done with incremental backups.
> >>>>
> >>>>Does this NBD approach meet your requirements?
> >>>>
> >>>>Stefan
> >>>for us we should somehow provide implementation of
> >>>calls posted by Vladimir. They are available in Parallels Server
> >>>version 6 and should be available in the next QEMU based
> >>>release using "Parallels SDK to libvirt" convertor. The problem
> >>>for us is that this old approach is used in the other side
> >>>of the product - in containers implementation while this
> >>>SDK is a universal access tool to both things.
> >>Point taken.  I think many other backup applications will expect a
> >>similar API, so it's pragmatic to provide something compatible.
> >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> >concerns about the QMP monitor:
> >
> >Previously I described incremental backup in "push" mode (already
> >supported today with drive-backup).  QEMU connects to a remote NBD
> >server and writes out the contents of all dirty blocks:
> >
> >   QEMU ---Write dirty blocks--> Backup appliance (server)
> >
> >This doesn't lend itself well to existing backup applications that
> >expect to iterate the dirty bitmap manually.
> >
> >Let's add a "pull" mode where the connection of the NBD connection is
> >reversed.  The backup application connects to QEMU's NBD server (image
> >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> >Status command for querying block provisioning information.  Now the
> >backup application can use Get LBA Status to fetch the dirty block
> >information from QEMU.
> >
> >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
> >
> >The dirty block information goes over the same NBD connection used to
> >read the contents of the dirty blocks.  The QMP monitor is not used to
> >transfer dirty block information.
> >
> >It may be necessary to extend the nbd-server-add command so that a
> >bitmap name can be passed.  This bitmap will be used to answer Get LBA
> >Status queries instead of using on bdrv_co_get_block_status().  This
> >would be necessary if image fleecing (point in time snapshot) is used.
> >
> >Stefan
> 
> There are no such commands in nbd spec here:
> 
> https://github.com/yoe/nbd/blob/master/doc/proto.md
> 
> 
> So, I'm not sure, that adding something qemu-specific to this external
> protocol will be simple or even true way. Is Qemu already extending original
> nbd?

No, we don't do any QEMU specific extensions. The point of the approach
Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
concept, it is relevant to any NBD server implementation.

For example, consider you were using a regular NBD server to export a
sparse LVM volume. This proposed extension would be relevant to such
a use case. As such this proposed extension is something that is likely
to be acceptable for the generic NBD specification.

> What about exporting bitmap as separate nbd entity? Just implement block
> driver, which will read from bitmap? If consider only read access and
> disabled (or frozen) bitmaps it should be simple enough.

IMHO that's not a desirable idea, since that *is* creating a QEMU specific
solution to the problem.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-16 17:20             ` Denis V. Lunev
@ 2016-02-18 16:39               ` Stefan Hajnoczi
  2016-02-18 17:07                 ` Markus Armbruster
  0 siblings, 1 reply; 56+ messages in thread
From: Stefan Hajnoczi @ 2016-02-18 16:39 UTC (permalink / raw)
  To: Denis V. Lunev
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, qemu-devel,
	armbru, jsnow

[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]

On Tue, Feb 16, 2016 at 08:20:44PM +0300, Denis V. Lunev wrote:
> the idea seems feasible at the first glance. I'll need to check
> the amount of the overhead of this approach. If we'll need
> to perform an additional request for each <granularity>
> block - the overhead is toooooo much.

Regarding overhead, thankfully it's not one-block-at-a-time :).

> Do you have the link to the spec/implementation?

You can reference SCSI Block Commands (SBC-3) 5.4 GET LBA STATUS
command.  The filename is sbc3r25.pdf and it's available from t10.org
although other sites seem to host copies too.

The approach is similar to the QMP command you have proposed.  Input
parameters are "starting LBA" and "allocation length".  Outputs are a
list of "status descriptors" which consists of an LBA, number of blocks,
and a provisioning status field (allocated/unallocated/etc).

This means a single call can retrieve information for a whole range of
the disk.  It's slightly more efficient than QMP since the data is
binary and not JSON text.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-18 12:11               ` Daniel P. Berrange
@ 2016-02-18 16:41                 ` Stefan Hajnoczi
  2016-02-19  2:08                   ` Fam Zheng
  0 siblings, 1 reply; 56+ messages in thread
From: Stefan Hajnoczi @ 2016-02-18 16:41 UTC (permalink / raw)
  To: Daniel P. Berrange
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, Denis V. Lunev,
	qemu-devel, armbru, jsnow

[-- Attachment #1: Type: text/plain, Size: 4718 bytes --]

On Thu, Feb 18, 2016 at 12:11:14PM +0000, Daniel P. Berrange wrote:
> On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> > On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> > >On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
> > >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> > >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> > >>>>On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> > >>>>>On 02/03/2016 11:14 AM, Fam Zheng wrote:
> > >>>>>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> > >>>>>>>Hi all.
> > >>>>>>>
> > >>>>>>>These series which aims to add external backup api. This is needed to allow
> > >>>>>>>backup software use our dirty bitmaps.
> > >>>>>>>
> > >>>>>>>Vmware and Parallels Cloud Server have this feature.
> > >>>>>>What is the advantage of this appraoch over "drive-backup sync=incremental
> > >>>>>>..."?
> > >>>>>This will allow third-party vendors to backup QEMU VMs into
> > >>>>>their own formats or to the cloud etc.
> > >>>>As an example, there is a third-party backup format called VMA from
> > >>>>Proxmox.  A few years ago I posted a proof-of-concept external backup
> > >>>>tool in Python:
> > >>>>
> > >>>>https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> > >>>>
> > >>>>It takes a full backup using drive-backup NBD (plus RAM/device state)
> > >>>>but the same can be done with incremental backups.
> > >>>>
> > >>>>Does this NBD approach meet your requirements?
> > >>>>
> > >>>>Stefan
> > >>>for us we should somehow provide implementation of
> > >>>calls posted by Vladimir. They are available in Parallels Server
> > >>>version 6 and should be available in the next QEMU based
> > >>>release using "Parallels SDK to libvirt" convertor. The problem
> > >>>for us is that this old approach is used in the other side
> > >>>of the product - in containers implementation while this
> > >>>SDK is a universal access tool to both things.
> > >>Point taken.  I think many other backup applications will expect a
> > >>similar API, so it's pragmatic to provide something compatible.
> > >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> > >concerns about the QMP monitor:
> > >
> > >Previously I described incremental backup in "push" mode (already
> > >supported today with drive-backup).  QEMU connects to a remote NBD
> > >server and writes out the contents of all dirty blocks:
> > >
> > >   QEMU ---Write dirty blocks--> Backup appliance (server)
> > >
> > >This doesn't lend itself well to existing backup applications that
> > >expect to iterate the dirty bitmap manually.
> > >
> > >Let's add a "pull" mode where the connection of the NBD connection is
> > >reversed.  The backup application connects to QEMU's NBD server (image
> > >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> > >Status command for querying block provisioning information.  Now the
> > >backup application can use Get LBA Status to fetch the dirty block
> > >information from QEMU.
> > >
> > >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
> > >
> > >The dirty block information goes over the same NBD connection used to
> > >read the contents of the dirty blocks.  The QMP monitor is not used to
> > >transfer dirty block information.
> > >
> > >It may be necessary to extend the nbd-server-add command so that a
> > >bitmap name can be passed.  This bitmap will be used to answer Get LBA
> > >Status queries instead of using on bdrv_co_get_block_status().  This
> > >would be necessary if image fleecing (point in time snapshot) is used.
> > >
> > >Stefan
> > 
> > There are no such commands in nbd spec here:
> > 
> > https://github.com/yoe/nbd/blob/master/doc/proto.md
> > 
> > 
> > So, I'm not sure, that adding something qemu-specific to this external
> > protocol will be simple or even true way. Is Qemu already extending original
> > nbd?
> 
> No, we don't do any QEMU specific extensions. The point of the approach
> Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
> concept, it is relevant to any NBD server implementation.
> 
> For example, consider you were using a regular NBD server to export a
> sparse LVM volume. This proposed extension would be relevant to such
> a use case. As such this proposed extension is something that is likely
> to be acceptable for the generic NBD specification.

Yes, Get LBA Status could be useful for non-QEMU NBD users too.

NBD already supports a TRIM command so the ability to query the
allocation status is a natural feature to add.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-18 16:39               ` Stefan Hajnoczi
@ 2016-02-18 17:07                 ` Markus Armbruster
  0 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2016-02-18 17:07 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, Denis V. Lunev,
	qemu-devel, jsnow

Stefan Hajnoczi <stefanha@gmail.com> writes:

> On Tue, Feb 16, 2016 at 08:20:44PM +0300, Denis V. Lunev wrote:
>> the idea seems feasible at the first glance. I'll need to check
>> the amount of the overhead of this approach. If we'll need
>> to perform an additional request for each <granularity>
>> block - the overhead is toooooo much.
>
> Regarding overhead, thankfully it's not one-block-at-a-time :).
>
>> Do you have the link to the spec/implementation?
>
> You can reference SCSI Block Commands (SBC-3) 5.4 GET LBA STATUS
> command.  The filename is sbc3r25.pdf and it's available from t10.org
> although other sites seem to host copies too.
>
> The approach is similar to the QMP command you have proposed.  Input
> parameters are "starting LBA" and "allocation length".  Outputs are a
> list of "status descriptors" which consists of an LBA, number of blocks,
> and a provisioning status field (allocated/unallocated/etc).
>
> This means a single call can retrieve information for a whole range of
> the disk.  It's slightly more efficient than QMP since the data is
> binary and not JSON text.

Moreover, it avoids abusing control-plane QMP as data plane.

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-18 16:41                 ` Stefan Hajnoczi
@ 2016-02-19  2:08                   ` Fam Zheng
  2016-02-19  8:51                     ` Markus Armbruster
  0 siblings, 1 reply; 56+ messages in thread
From: Fam Zheng @ 2016-02-19  2:08 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Denis V. Lunev, qemu-devel,
	armbru, jsnow

On Thu, 02/18 16:41, Stefan Hajnoczi wrote:
> On Thu, Feb 18, 2016 at 12:11:14PM +0000, Daniel P. Berrange wrote:
> > On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> > > On 16.02.2016 20:09, Stefan Hajnoczi wrote:
> > > >On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
> > > >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
> > > >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
> > > >>>>On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
> > > >>>>>On 02/03/2016 11:14 AM, Fam Zheng wrote:
> > > >>>>>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
> > > >>>>>>>Hi all.
> > > >>>>>>>
> > > >>>>>>>These series which aims to add external backup api. This is needed to allow
> > > >>>>>>>backup software use our dirty bitmaps.
> > > >>>>>>>
> > > >>>>>>>Vmware and Parallels Cloud Server have this feature.
> > > >>>>>>What is the advantage of this appraoch over "drive-backup sync=incremental
> > > >>>>>>..."?
> > > >>>>>This will allow third-party vendors to backup QEMU VMs into
> > > >>>>>their own formats or to the cloud etc.
> > > >>>>As an example, there is a third-party backup format called VMA from
> > > >>>>Proxmox.  A few years ago I posted a proof-of-concept external backup
> > > >>>>tool in Python:
> > > >>>>
> > > >>>>https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
> > > >>>>
> > > >>>>It takes a full backup using drive-backup NBD (plus RAM/device state)
> > > >>>>but the same can be done with incremental backups.
> > > >>>>
> > > >>>>Does this NBD approach meet your requirements?
> > > >>>>
> > > >>>>Stefan
> > > >>>for us we should somehow provide implementation of
> > > >>>calls posted by Vladimir. They are available in Parallels Server
> > > >>>version 6 and should be available in the next QEMU based
> > > >>>release using "Parallels SDK to libvirt" convertor. The problem
> > > >>>for us is that this old approach is used in the other side
> > > >>>of the product - in containers implementation while this
> > > >>>SDK is a universal access tool to both things.
> > > >>Point taken.  I think many other backup applications will expect a
> > > >>similar API, so it's pragmatic to provide something compatible.
> > > >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
> > > >concerns about the QMP monitor:
> > > >
> > > >Previously I described incremental backup in "push" mode (already
> > > >supported today with drive-backup).  QEMU connects to a remote NBD
> > > >server and writes out the contents of all dirty blocks:
> > > >
> > > >   QEMU ---Write dirty blocks--> Backup appliance (server)
> > > >
> > > >This doesn't lend itself well to existing backup applications that
> > > >expect to iterate the dirty bitmap manually.
> > > >
> > > >Let's add a "pull" mode where the connection of the NBD connection is
> > > >reversed.  The backup application connects to QEMU's NBD server (image
> > > >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
> > > >Status command for querying block provisioning information.  Now the
> > > >backup application can use Get LBA Status to fetch the dirty block
> > > >information from QEMU.
> > > >
> > > >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
> > > >
> > > >The dirty block information goes over the same NBD connection used to
> > > >read the contents of the dirty blocks.  The QMP monitor is not used to
> > > >transfer dirty block information.
> > > >
> > > >It may be necessary to extend the nbd-server-add command so that a
> > > >bitmap name can be passed.  This bitmap will be used to answer Get LBA
> > > >Status queries instead of using on bdrv_co_get_block_status().  This
> > > >would be necessary if image fleecing (point in time snapshot) is used.
> > > >
> > > >Stefan
> > > 
> > > There are no such commands in nbd spec here:
> > > 
> > > https://github.com/yoe/nbd/blob/master/doc/proto.md
> > > 
> > > 
> > > So, I'm not sure, that adding something qemu-specific to this external
> > > protocol will be simple or even true way. Is Qemu already extending original
> > > nbd?
> > 
> > No, we don't do any QEMU specific extensions. The point of the approach
> > Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
> > concept, it is relevant to any NBD server implementation.
> > 
> > For example, consider you were using a regular NBD server to export a
> > sparse LVM volume. This proposed extension would be relevant to such
> > a use case. As such this proposed extension is something that is likely
> > to be acceptable for the generic NBD specification.
> 
> Yes, Get LBA Status could be useful for non-QEMU NBD users too.
> 
> NBD already supports a TRIM command so the ability to query the
> allocation status is a natural feature to add.

Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
the command reports "mapped", "allocated" and "anchored" statuses. Does that
mean NBD will use a different status set?

Fam

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-19  2:08                   ` Fam Zheng
@ 2016-02-19  8:51                     ` Markus Armbruster
  2016-02-24 23:34                       ` John Snow
  2016-02-26 19:55                       ` Paolo Bonzini
  0 siblings, 2 replies; 56+ messages in thread
From: Markus Armbruster @ 2016-02-19  8:51 UTC (permalink / raw)
  To: Fam Zheng
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, jsnow

Fam Zheng <famz@redhat.com> writes:

> On Thu, 02/18 16:41, Stefan Hajnoczi wrote:
>> On Thu, Feb 18, 2016 at 12:11:14PM +0000, Daniel P. Berrange wrote:
>> > On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy wrote:
>> > > On 16.02.2016 20:09, Stefan Hajnoczi wrote:
>> > > >On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
>> > > >>On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
>> > > >>>On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
>> > > >>>>On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>> > > >>>>>On 02/03/2016 11:14 AM, Fam Zheng wrote:
>> > > >>>>>>On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>> > > >>>>>>>Hi all.
>> > > >>>>>>>
>> > > >>>>>>>These series which aims to add external backup api. This is needed to allow
>> > > >>>>>>>backup software use our dirty bitmaps.
>> > > >>>>>>>
>> > > >>>>>>>Vmware and Parallels Cloud Server have this feature.
>> > > >>>>>>What is the advantage of this appraoch over "drive-backup sync=incremental
>> > > >>>>>>..."?
>> > > >>>>>This will allow third-party vendors to backup QEMU VMs into
>> > > >>>>>their own formats or to the cloud etc.
>> > > >>>>As an example, there is a third-party backup format called VMA from
>> > > >>>>Proxmox.  A few years ago I posted a proof-of-concept external backup
>> > > >>>>tool in Python:
>> > > >>>>
>> > > >>>>https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>> > > >>>>
>> > > >>>>It takes a full backup using drive-backup NBD (plus RAM/device state)
>> > > >>>>but the same can be done with incremental backups.
>> > > >>>>
>> > > >>>>Does this NBD approach meet your requirements?
>> > > >>>>
>> > > >>>>Stefan
>> > > >>>for us we should somehow provide implementation of
>> > > >>>calls posted by Vladimir. They are available in Parallels Server
>> > > >>>version 6 and should be available in the next QEMU based
>> > > >>>release using "Parallels SDK to libvirt" convertor. The problem
>> > > >>>for us is that this old approach is used in the other side
>> > > >>>of the product - in containers implementation while this
>> > > >>>SDK is a universal access tool to both things.
>> > > >>Point taken.  I think many other backup applications will expect a
>> > > >>similar API, so it's pragmatic to provide something compatible.
>> > > >Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
>> > > >concerns about the QMP monitor:
>> > > >
>> > > >Previously I described incremental backup in "push" mode (already
>> > > >supported today with drive-backup).  QEMU connects to a remote NBD
>> > > >server and writes out the contents of all dirty blocks:
>> > > >
>> > > >   QEMU ---Write dirty blocks--> Backup appliance (server)
>> > > >
>> > > >This doesn't lend itself well to existing backup applications that
>> > > >expect to iterate the dirty bitmap manually.
>> > > >
>> > > >Let's add a "pull" mode where the connection of the NBD connection is
>> > > >reversed.  The backup application connects to QEMU's NBD server (image
>> > > >fleecing).  The NBD protocol is extended to support the SCSI Get LBA
>> > > >Status command for querying block provisioning information.  Now the
>> > > >backup application can use Get LBA Status to fetch the dirty block
>> > > >information from QEMU.
>> > > >
>> > > >   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
>> > > >
>> > > >The dirty block information goes over the same NBD connection used to
>> > > >read the contents of the dirty blocks.  The QMP monitor is not used to
>> > > >transfer dirty block information.
>> > > >
>> > > >It may be necessary to extend the nbd-server-add command so that a
>> > > >bitmap name can be passed.  This bitmap will be used to answer Get LBA
>> > > >Status queries instead of using on bdrv_co_get_block_status().  This
>> > > >would be necessary if image fleecing (point in time snapshot) is used.
>> > > >
>> > > >Stefan
>> > > 
>> > > There are no such commands in nbd spec here:
>> > > 
>> > > https://github.com/yoe/nbd/blob/master/doc/proto.md
>> > > 
>> > > 
>> > > So, I'm not sure, that adding something qemu-specific to this external
>> > > protocol will be simple or even true way. Is Qemu already extending original
>> > > nbd?
>> > 
>> > No, we don't do any QEMU specific extensions. The point of the approach
>> > Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
>> > concept, it is relevant to any NBD server implementation.
>> > 
>> > For example, consider you were using a regular NBD server to export a
>> > sparse LVM volume. This proposed extension would be relevant to such
>> > a use case. As such this proposed extension is something that is likely
>> > to be acceptable for the generic NBD specification.
>> 
>> Yes, Get LBA Status could be useful for non-QEMU NBD users too.
>> 
>> NBD already supports a TRIM command so the ability to query the
>> allocation status is a natural feature to add.
>
> Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
> the command reports "mapped", "allocated" and "anchored" statuses. Does that
> mean NBD will use a different status set?

Perhaps some conceptual gymnastics can get us to standard semantics.

Incremental backup wants to copy out an image's "dirty" blocks.

We can view this as a bitmap telling us which of the image's blocks are
dirty.

An alternative view would be base image + dirty delta image.  In the the
dirty delta, exactly the dirty blocks are allocated.  The delta image
may be conceptual.

Now, consider the dirty delta *without* its backing image.  You can find
its allocated blocks with Get LBA Status.  As usual, you can read even
unallocated blocks, see SBC3 table 9.

If we NBD-export exactly that, we can use standard semantics, can't we?

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-19  8:51                     ` Markus Armbruster
@ 2016-02-24 23:34                       ` John Snow
  2016-02-26 19:55                       ` Paolo Bonzini
  1 sibling, 0 replies; 56+ messages in thread
From: John Snow @ 2016-02-24 23:34 UTC (permalink / raw)
  To: Markus Armbruster, Fam Zheng
  Cc: kwolf, Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy,
	Denis V. Lunev, qemu-devel



On 02/19/2016 03:51 AM, Markus Armbruster wrote:
> Fam Zheng <famz@redhat.com> writes:
> 
>> On Thu, 02/18 16:41, Stefan Hajnoczi wrote:
>>> On Thu, Feb 18, 2016 at 12:11:14PM +0000, Daniel P. Berrange wrote:
>>>> On Wed, Feb 17, 2016 at 08:47:11PM +0300, Vladimir Sementsov-Ogievskiy wrote:
>>>>> On 16.02.2016 20:09, Stefan Hajnoczi wrote:
>>>>>> On Wed, Feb 10, 2016 at 10:10:04AM +0000, Stefan Hajnoczi wrote:
>>>>>>> On Tue, Feb 09, 2016 at 05:41:50PM +0300, Denis V. Lunev wrote:
>>>>>>>> On 02/09/2016 05:28 PM, Stefan Hajnoczi wrote:
>>>>>>>>> On Fri, Feb 05, 2016 at 11:28:42AM +0300, Denis V. Lunev wrote:
>>>>>>>>>> On 02/03/2016 11:14 AM, Fam Zheng wrote:
>>>>>>>>>>> On Sat, 01/30 13:56, Vladimir Sementsov-Ogievskiy wrote:
>>>>>>>>>>>> Hi all.
>>>>>>>>>>>>
>>>>>>>>>>>> These series which aims to add external backup api. This is needed to allow
>>>>>>>>>>>> backup software use our dirty bitmaps.
>>>>>>>>>>>>
>>>>>>>>>>>> Vmware and Parallels Cloud Server have this feature.
>>>>>>>>>>> What is the advantage of this appraoch over "drive-backup sync=incremental
>>>>>>>>>>> ..."?
>>>>>>>>>> This will allow third-party vendors to backup QEMU VMs into
>>>>>>>>>> their own formats or to the cloud etc.
>>>>>>>>> As an example, there is a third-party backup format called VMA from
>>>>>>>>> Proxmox.  A few years ago I posted a proof-of-concept external backup
>>>>>>>>> tool in Python:
>>>>>>>>>
>>>>>>>>> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01536.html
>>>>>>>>>
>>>>>>>>> It takes a full backup using drive-backup NBD (plus RAM/device state)
>>>>>>>>> but the same can be done with incremental backups.
>>>>>>>>>
>>>>>>>>> Does this NBD approach meet your requirements?
>>>>>>>>>
>>>>>>>>> Stefan
>>>>>>>> for us we should somehow provide implementation of
>>>>>>>> calls posted by Vladimir. They are available in Parallels Server
>>>>>>>> version 6 and should be available in the next QEMU based
>>>>>>>> release using "Parallels SDK to libvirt" convertor. The problem
>>>>>>>> for us is that this old approach is used in the other side
>>>>>>>> of the product - in containers implementation while this
>>>>>>>> SDK is a universal access tool to both things.
>>>>>>> Point taken.  I think many other backup applications will expect a
>>>>>>> similar API, so it's pragmatic to provide something compatible.
>>>>>> Kevin Wolf and Daniel Berrange proposed an elegant way to avoid the
>>>>>> concerns about the QMP monitor:
>>>>>>
>>>>>> Previously I described incremental backup in "push" mode (already
>>>>>> supported today with drive-backup).  QEMU connects to a remote NBD
>>>>>> server and writes out the contents of all dirty blocks:
>>>>>>
>>>>>>   QEMU ---Write dirty blocks--> Backup appliance (server)
>>>>>>
>>>>>> This doesn't lend itself well to existing backup applications that
>>>>>> expect to iterate the dirty bitmap manually.
>>>>>>
>>>>>> Let's add a "pull" mode where the connection of the NBD connection is
>>>>>> reversed.  The backup application connects to QEMU's NBD server (image
>>>>>> fleecing).  The NBD protocol is extended to support the SCSI Get LBA
>>>>>> Status command for querying block provisioning information.  Now the
>>>>>> backup application can use Get LBA Status to fetch the dirty block
>>>>>> information from QEMU.
>>>>>>
>>>>>>   QEMU (server) <--Get LBA Status or Read dirty blocks-- Backup appliance
>>>>>>
>>>>>> The dirty block information goes over the same NBD connection used to
>>>>>> read the contents of the dirty blocks.  The QMP monitor is not used to
>>>>>> transfer dirty block information.
>>>>>>
>>>>>> It may be necessary to extend the nbd-server-add command so that a
>>>>>> bitmap name can be passed.  This bitmap will be used to answer Get LBA
>>>>>> Status queries instead of using on bdrv_co_get_block_status().  This
>>>>>> would be necessary if image fleecing (point in time snapshot) is used.
>>>>>>
>>>>>> Stefan
>>>>>
>>>>> There are no such commands in nbd spec here:
>>>>>
>>>>> https://github.com/yoe/nbd/blob/master/doc/proto.md
>>>>>
>>>>>
>>>>> So, I'm not sure, that adding something qemu-specific to this external
>>>>> protocol will be simple or even true way. Is Qemu already extending original
>>>>> nbd?
>>>>
>>>> No, we don't do any QEMU specific extensions. The point of the approach
>>>> Stefan suggests here though, is that it is *not* an inherantly QEMU-specific
>>>> concept, it is relevant to any NBD server implementation.
>>>>
>>>> For example, consider you were using a regular NBD server to export a
>>>> sparse LVM volume. This proposed extension would be relevant to such
>>>> a use case. As such this proposed extension is something that is likely
>>>> to be acceptable for the generic NBD specification.
>>>
>>> Yes, Get LBA Status could be useful for non-QEMU NBD users too.
>>>
>>> NBD already supports a TRIM command so the ability to query the
>>> allocation status is a natural feature to add.
>>
>> Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>> mean NBD will use a different status set?
> 
> Perhaps some conceptual gymnastics can get us to standard semantics.
> 
> Incremental backup wants to copy out an image's "dirty" blocks.
> 
> We can view this as a bitmap telling us which of the image's blocks are
> dirty.
> 
> An alternative view would be base image + dirty delta image.  In the the
> dirty delta, exactly the dirty blocks are allocated.  The delta image
> may be conceptual.
> 
> Now, consider the dirty delta *without* its backing image.  You can find
> its allocated blocks with Get LBA Status.  As usual, you can read even
> unallocated blocks, see SBC3 table 9.
> 
> If we NBD-export exactly that, we can use standard semantics, can't we?
> 

Sounds exactly appropriate to me, from this perspective. If absolutely
necessary we could disguise this behind a macro QMP command that implied
these exact kind of semantics for the fleecing/NBD export -- one that
accepted the bitmap name to tie to the export for this reason.

--js

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-19  8:51                     ` Markus Armbruster
  2016-02-24 23:34                       ` John Snow
@ 2016-02-26 19:55                       ` Paolo Bonzini
  2016-02-26 20:03                         ` Paolo Bonzini
                                           ` (2 more replies)
  1 sibling, 3 replies; 56+ messages in thread
From: Paolo Bonzini @ 2016-02-26 19:55 UTC (permalink / raw)
  To: Markus Armbruster, Fam Zheng
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, jsnow



On 19/02/2016 09:51, Markus Armbruster wrote:
>> Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>> mean NBD will use a different status set?
> 
> Perhaps some conceptual gymnastics can get us to standard semantics.
> 
> Incremental backup wants to copy out an image's "dirty" blocks.
> 
> We can view this as a bitmap telling us which of the image's blocks are
> dirty.
> 
> An alternative view would be base image + dirty delta image.  In the the
> dirty delta, exactly the dirty blocks are allocated.  The delta image
> may be conceptual.

I see a problem here. On one hand I agree that the "GET LBA STATUS" is
a natural extension to the NBD protocol.

On the other hand, the Get LBA Status command in SCSI reflects the
state over the whole chain, not only the top element.  It is the
equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
bdrv_get_block_status(bs, ...).  My understanding is that the dirty
block information would require the latter, especially in the
"conceptual delta image" model that Markus describes above.

Having NBD implement subtly different semantics compared to SCSI is a
bad idea in my opinion.

Of course if we call it "READ DIRTY BLOCKS" that would work, but I
don't think such a command would be something that it makes sense to
have in the general purpose NBD spec, so you would need a mechanism
for vendor-specific extensions.

Paolo

> Now, consider the dirty delta *without* its backing image.  You can find
> its allocated blocks with Get LBA Status.  As usual, you can read even
> unallocated blocks, see SBC3 table 9.
> 
> If we NBD-export exactly that, we can use standard semantics, can't we?

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-26 19:55                       ` Paolo Bonzini
@ 2016-02-26 20:03                         ` Paolo Bonzini
  2016-02-26 20:29                           ` Denis V. Lunev
  2016-02-26 21:37                           ` John Snow
  2016-02-26 20:40                         ` Denis V. Lunev
  2016-02-29  8:14                         ` Markus Armbruster
  2 siblings, 2 replies; 56+ messages in thread
From: Paolo Bonzini @ 2016-02-26 20:03 UTC (permalink / raw)
  To: Markus Armbruster, Fam Zheng
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, jsnow



On 26/02/2016 20:55, Paolo Bonzini wrote:
> 
> 
> On 19/02/2016 09:51, Markus Armbruster wrote:
>>> Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
>>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>>> mean NBD will use a different status set?
>>
>> Perhaps some conceptual gymnastics can get us to standard semantics.
>>
>> Incremental backup wants to copy out an image's "dirty" blocks.
>>
>> We can view this as a bitmap telling us which of the image's blocks are
>> dirty.
>>
>> An alternative view would be base image + dirty delta image.  In the the
>> dirty delta, exactly the dirty blocks are allocated.  The delta image
>> may be conceptual.
> 
> I see a problem here. On one hand I agree that the "GET LBA STATUS" is
> a natural extension to the NBD protocol.
> 
> On the other hand, the Get LBA Status command in SCSI reflects the
> state over the whole chain, not only the top element.  It is the
> equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
> bdrv_get_block_status(bs, ...).  My understanding is that the dirty
> block information would require the latter, especially in the
> "conceptual delta image" model that Markus describes above.
> 
> Having NBD implement subtly different semantics compared to SCSI is a
> bad idea in my opinion.
> 
> Of course if we call it "READ DIRTY BLOCKS" that would work, but I
> don't think such a command would be something that it makes sense to
> have in the general purpose NBD spec, so you would need a mechanism
> for vendor-specific extensions.

Trying to be constructive: shall we instead have a simple mini-protocol
with commands like "read bitmap slice", "clear bitmap slice", "query
next 0 bit", "query next 1 bit"?

Not necessarily QMP, just a little socket thing.  It could be JSON or
ASCII or binary.  It sucks to implement a new protocol, but perhaps it
could be something compatible with VMware or Parallels.

Sorry if this has already been proposed, I'm late to the game and I only
read this subthread.

Paolo

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-26 20:03                         ` Paolo Bonzini
@ 2016-02-26 20:29                           ` Denis V. Lunev
  2016-02-26 21:37                           ` John Snow
  1 sibling, 0 replies; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-26 20:29 UTC (permalink / raw)
  To: Paolo Bonzini, Markus Armbruster, Fam Zheng
  Cc: kwolf, Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy, jsnow, qemu-devel

On 02/26/2016 11:03 PM, Paolo Bonzini wrote:
>
> On 26/02/2016 20:55, Paolo Bonzini wrote:
>>
>> On 19/02/2016 09:51, Markus Armbruster wrote:
>>>> Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
>>>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>>>> mean NBD will use a different status set?
>>> Perhaps some conceptual gymnastics can get us to standard semantics.
>>>
>>> Incremental backup wants to copy out an image's "dirty" blocks.
>>>
>>> We can view this as a bitmap telling us which of the image's blocks are
>>> dirty.
>>>
>>> An alternative view would be base image + dirty delta image.  In the the
>>> dirty delta, exactly the dirty blocks are allocated.  The delta image
>>> may be conceptual.
>> I see a problem here. On one hand I agree that the "GET LBA STATUS" is
>> a natural extension to the NBD protocol.
>>
>> On the other hand, the Get LBA Status command in SCSI reflects the
>> state over the whole chain, not only the top element.  It is the
>> equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
>> bdrv_get_block_status(bs, ...).  My understanding is that the dirty
>> block information would require the latter, especially in the
>> "conceptual delta image" model that Markus describes above.
>>
>> Having NBD implement subtly different semantics compared to SCSI is a
>> bad idea in my opinion.
>>
>> Of course if we call it "READ DIRTY BLOCKS" that would work, but I
>> don't think such a command would be something that it makes sense to
>> have in the general purpose NBD spec, so you would need a mechanism
>> for vendor-specific extensions.
> Trying to be constructive: shall we instead have a simple mini-protocol
> with commands like "read bitmap slice", "clear bitmap slice", "query
> next 0 bit", "query next 1 bit"?
>
> Not necessarily QMP, just a little socket thing.  It could be JSON or
> ASCII or binary.  It sucks to implement a new protocol, but perhaps it
> could be something compatible with VMware or Parallels.
>
> Sorry if this has already been proposed, I'm late to the game and I only
> read this subthread.
>
> Paolo
there was a proposal in the thread to export bitmap itself as NBD
block device. We have also discuss to transfer entire bitmap over
QMP but this idea is not that good according to Eric/Markus.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-26 19:55                       ` Paolo Bonzini
  2016-02-26 20:03                         ` Paolo Bonzini
@ 2016-02-26 20:40                         ` Denis V. Lunev
  2016-02-27  4:26                           ` Fam Zheng
  2016-02-29  8:14                         ` Markus Armbruster
  2 siblings, 1 reply; 56+ messages in thread
From: Denis V. Lunev @ 2016-02-26 20:40 UTC (permalink / raw)
  To: Paolo Bonzini, Markus Armbruster, Fam Zheng
  Cc: kwolf, Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy, jsnow, qemu-devel

On 02/26/2016 10:55 PM, Paolo Bonzini wrote:
>
> On 19/02/2016 09:51, Markus Armbruster wrote:
>>> Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
>>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>>> mean NBD will use a different status set?
>> Perhaps some conceptual gymnastics can get us to standard semantics.
>>
>> Incremental backup wants to copy out an image's "dirty" blocks.
>>
>> We can view this as a bitmap telling us which of the image's blocks are
>> dirty.
>>
>> An alternative view would be base image + dirty delta image.  In the the
>> dirty delta, exactly the dirty blocks are allocated.  The delta image
>> may be conceptual.
> I see a problem here. On one hand I agree that the "GET LBA STATUS" is
> a natural extension to the NBD protocol.
>
> On the other hand, the Get LBA Status command in SCSI reflects the
> state over the whole chain, not only the top element.  It is the
> equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
> bdrv_get_block_status(bs, ...).  My understanding is that the dirty
> block information would require the latter, especially in the
> "conceptual delta image" model that Markus describes above.
>
> Having NBD implement subtly different semantics compared to SCSI is a
> bad idea in my opinion.
>
> Of course if we call it "READ DIRTY BLOCKS" that would work, but I
> don't think such a command would be something that it makes sense to
> have in the general purpose NBD spec, so you would need a mechanism
> for vendor-specific extensions.
>
> Paolo
>
In general, the idea to bind DIRTY BITMAP to GET STATUS command is not that
bad. First of all, NBD has no relation to the SCSI at all thus we are not
bound to the SCSI protocol. This is good thing.

Next, it is generally good to query state of the data block before reading
to reduce amount of transfers over the network. This is useful even for a
full backup operation to avoid zero block transfers.

Thus, if we have created special NBD share, we can provide STATUS bitmap
in the way we want, f.e. with the proposed semantics. This will not be a
violation of the protocol.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-26 20:03                         ` Paolo Bonzini
  2016-02-26 20:29                           ` Denis V. Lunev
@ 2016-02-26 21:37                           ` John Snow
  1 sibling, 0 replies; 56+ messages in thread
From: John Snow @ 2016-02-26 21:37 UTC (permalink / raw)
  To: Paolo Bonzini, Markus Armbruster, Fam Zheng
  Cc: kwolf, Stefan Hajnoczi, Vladimir Sementsov-Ogievskiy,
	Denis V. Lunev, qemu-devel



On 02/26/2016 03:03 PM, Paolo Bonzini wrote:
> 
> 
> On 26/02/2016 20:55, Paolo Bonzini wrote:
>>
>>
>> On 19/02/2016 09:51, Markus Armbruster wrote:
>>>> Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
>>>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>>>> mean NBD will use a different status set?
>>>
>>> Perhaps some conceptual gymnastics can get us to standard semantics.
>>>
>>> Incremental backup wants to copy out an image's "dirty" blocks.
>>>
>>> We can view this as a bitmap telling us which of the image's blocks are
>>> dirty.
>>>
>>> An alternative view would be base image + dirty delta image.  In the the
>>> dirty delta, exactly the dirty blocks are allocated.  The delta image
>>> may be conceptual.
>>
>> I see a problem here. On one hand I agree that the "GET LBA STATUS" is
>> a natural extension to the NBD protocol.
>>
>> On the other hand, the Get LBA Status command in SCSI reflects the
>> state over the whole chain, not only the top element.  It is the
>> equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
>> bdrv_get_block_status(bs, ...).  My understanding is that the dirty
>> block information would require the latter, especially in the
>> "conceptual delta image" model that Markus describes above.
>>

This is unfortunate.

>> Having NBD implement subtly different semantics compared to SCSI is a
>> bad idea in my opinion.
>>
>> Of course if we call it "READ DIRTY BLOCKS" that would work, but I
>> don't think such a command would be something that it makes sense to
>> have in the general purpose NBD spec, so you would need a mechanism
>> for vendor-specific extensions.
> 
> Trying to be constructive: shall we instead have a simple mini-protocol
> with commands like "read bitmap slice", "clear bitmap slice", "query
> next 0 bit", "query next 1 bit"?
> 
> Not necessarily QMP, just a little socket thing.  It could be JSON or
> ASCII or binary.  It sucks to implement a new protocol, but perhaps it
> could be something compatible with VMware or Parallels.
> 
> Sorry if this has already been proposed, I'm late to the game and I only
> read this subthread.
> 
> Paolo
> 

The solution I proposed prior to the LBA status idea (which I did like,
pending its suitability which looks... compromised, now. sigh.) was a
simple binary protocol.

Something akin to this:

{ "execute": "dirty-bitmap-export",
  "arguments": {
    "node": "drive0",
    "bitmap": "bitmap0",
    "uri": "tcp:127.0.0.1:31337"
  }
}

We'll open up a socket to the URI provided, and push data akin to the
following:

Bytes 0-3: "QBMP" (Qemu Bit Map Protocol)
Bytes 4-11: number of dirty bits (As a lazy checksum)
Bytes 12-19: granularity of bits
Bytes 20+: raw bitmap data
EOF

A simple, non-glorious format that just gets the data out there. Then
the client has the info it needs to poll dirty blocks as necessary.


The other suggestion is to literally fetch the bitmap over NBD. How
would this work -- would the export just be sized to the number of bytes
of the bitmap data, and the client queries it?

I can't help but feel like this is slightly hacky, but I can't think of
any truly solid reasons against it either.

--js

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-26 20:40                         ` Denis V. Lunev
@ 2016-02-27  4:26                           ` Fam Zheng
  0 siblings, 0 replies; 56+ messages in thread
From: Fam Zheng @ 2016-02-27  4:26 UTC (permalink / raw)
  To: Denis V. Lunev
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Stefan Hajnoczi, qemu-devel,
	Markus Armbruster, Paolo Bonzini, jsnow

On Fri, 02/26 23:40, Denis V. Lunev wrote:
> On 02/26/2016 10:55 PM, Paolo Bonzini wrote:
> >
> >On 19/02/2016 09:51, Markus Armbruster wrote:
> >>>Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
> >>>the command reports "mapped", "allocated" and "anchored" statuses. Does that
> >>>mean NBD will use a different status set?
> >>Perhaps some conceptual gymnastics can get us to standard semantics.
> >>
> >>Incremental backup wants to copy out an image's "dirty" blocks.
> >>
> >>We can view this as a bitmap telling us which of the image's blocks are
> >>dirty.
> >>
> >>An alternative view would be base image + dirty delta image.  In the the
> >>dirty delta, exactly the dirty blocks are allocated.  The delta image
> >>may be conceptual.
> >I see a problem here. On one hand I agree that the "GET LBA STATUS" is
> >a natural extension to the NBD protocol.
> >
> >On the other hand, the Get LBA Status command in SCSI reflects the
> >state over the whole chain, not only the top element.  It is the
> >equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
> >bdrv_get_block_status(bs, ...).  My understanding is that the dirty
> >block information would require the latter, especially in the
> >"conceptual delta image" model that Markus describes above.
> >
> >Having NBD implement subtly different semantics compared to SCSI is a
> >bad idea in my opinion.
> >
> >Of course if we call it "READ DIRTY BLOCKS" that would work, but I
> >don't think such a command would be something that it makes sense to
> >have in the general purpose NBD spec, so you would need a mechanism
> >for vendor-specific extensions.
> >
> >Paolo
> >
> In general, the idea to bind DIRTY BITMAP to GET STATUS command is not that
> bad. First of all, NBD has no relation to the SCSI at all thus we are not
> bound to the SCSI protocol. This is good thing.
> 
> Next, it is generally good to query state of the data block before reading
> to reduce amount of transfers over the network. This is useful even for a
> full backup operation to avoid zero block transfers.
> 
> Thus, if we have created special NBD share, we can provide STATUS bitmap
> in the way we want, f.e. with the proposed semantics. This will not be a
> violation of the protocol.

Paolo described what I was trying to say, but way clearer then what I said. :)
And I agree a dedicated bitmap status seems better here.

Fam

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-26 19:55                       ` Paolo Bonzini
  2016-02-26 20:03                         ` Paolo Bonzini
  2016-02-26 20:40                         ` Denis V. Lunev
@ 2016-02-29  8:14                         ` Markus Armbruster
  2016-02-29  8:54                           ` Paolo Bonzini
  2016-02-29 10:22                           ` Markus Armbruster
  2 siblings, 2 replies; 56+ messages in thread
From: Markus Armbruster @ 2016-02-29  8:14 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, jsnow

Paolo Bonzini <pbonzini@redhat.com> writes:

> On 19/02/2016 09:51, Markus Armbruster wrote:
>>> Is it an abuse to "Get LBA Status" to return dirty information? Because in SCSI
>>> the command reports "mapped", "allocated" and "anchored" statuses. Does that
>>> mean NBD will use a different status set?
>> 
>> Perhaps some conceptual gymnastics can get us to standard semantics.
>> 
>> Incremental backup wants to copy out an image's "dirty" blocks.
>> 
>> We can view this as a bitmap telling us which of the image's blocks are
>> dirty.
>> 
>> An alternative view would be base image + dirty delta image.  In the the
>> dirty delta, exactly the dirty blocks are allocated.  The delta image
>> may be conceptual.
>
> I see a problem here. On one hand I agree that the "GET LBA STATUS" is
> a natural extension to the NBD protocol.
>
> On the other hand, the Get LBA Status command in SCSI reflects the
> state over the whole chain, not only the top element.  It is the
> equivalent of bdrv_get_block_status_above(bs, NULL, ...), rather than
> bdrv_get_block_status(bs, ...).  My understanding is that the dirty
> block information would require the latter, especially in the
> "conceptual delta image" model that Markus describes above.
>
> Having NBD implement subtly different semantics compared to SCSI is a
> bad idea in my opinion.

I completely agree with you that Get LBA Status cannot just reflect the
top layer.  But that's not what I meant to propose.  Let me try to
explain myself more clearly.

Consider a QCOW2 image D (for delta) with a backing file B (for base).
If you open it normally, you see "D over B".  Get LBA Status should
certainly claim the "deallocated" state only for blocks that are
allocated neither in D nor B.

However, you can also open D *without* its backing file.  Then you see
"D over nothing".  Here, get LBA Status should claim "deallocated" state
for anything not allocated in D.

My proposal is to expose a "just the dirty blocks" view of a block
backend similarly: it's a *separate* backend that contains *only* the
dirty blocks.  Attempts to read a clean block behave exactly like
reading an unmapped block from any other thinly provisioned backend
(QCOW2 gives you zeroes, if I remember correctly).  I think it's only
natural to make Get LBA Status claim "deallocated" for exactly the clean
blocks then.

> Of course if we call it "READ DIRTY BLOCKS" that would work, but I
> don't think such a command would be something that it makes sense to
> have in the general purpose NBD spec, so you would need a mechanism
> for vendor-specific extensions.
>
> Paolo
>
>> Now, consider the dirty delta *without* its backing image.  You can find
>> its allocated blocks with Get LBA Status.  As usual, you can read even
>> unallocated blocks, see SBC3 table 9.
>> 
>> If we NBD-export exactly that, we can use standard semantics, can't we?

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-29  8:14                         ` Markus Armbruster
@ 2016-02-29  8:54                           ` Paolo Bonzini
  2016-02-29  9:42                             ` Paolo Bonzini
  2016-02-29 10:22                           ` Markus Armbruster
  1 sibling, 1 reply; 56+ messages in thread
From: Paolo Bonzini @ 2016-02-29  8:54 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, jsnow



On 29/02/2016 09:14, Markus Armbruster wrote:
> I completely agree with you that Get LBA Status cannot just reflect the
> top layer.  But that's not what I meant to propose.  Let me try to
> explain myself more clearly.
> 
> Consider a QCOW2 image D (for delta) with a backing file B (for base).
> If you open it normally, you see "D over B".  Get LBA Status should
> certainly claim the "deallocated" state only for blocks that are
> allocated neither in D nor B.
> 
> However, you can also open D *without* its backing file.  Then you see
> "D over nothing".  Here, get LBA Status should claim "deallocated" state
> for anything not allocated in D.

Ok, this makes more sense.

The question then is whether to implement this NBD server inside QEMU,
or outside it as a separate process to which QEMU "pushes" blocks as in
the existing backup job.  I would prefer the latter, so that it is
possible to implement various APIs (get block status, but also VMware or
Parallels or whatever).

Basically the same points made in
https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
still apply.

Paolo

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-29  8:54                           ` Paolo Bonzini
@ 2016-02-29  9:42                             ` Paolo Bonzini
  2016-02-29 10:05                               ` Fam Zheng
  2016-03-10 17:37                               ` Stefan Hajnoczi
  0 siblings, 2 replies; 56+ messages in thread
From: Paolo Bonzini @ 2016-02-29  9:42 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, jsnow



On 29/02/2016 09:54, Paolo Bonzini wrote:
> 
> 
> On 29/02/2016 09:14, Markus Armbruster wrote:
>> I completely agree with you that Get LBA Status cannot just reflect the
>> top layer.  But that's not what I meant to propose.  Let me try to
>> explain myself more clearly.
>>
>> Consider a QCOW2 image D (for delta) with a backing file B (for base).
>> If you open it normally, you see "D over B".  Get LBA Status should
>> certainly claim the "deallocated" state only for blocks that are
>> allocated neither in D nor B.
>>
>> However, you can also open D *without* its backing file.  Then you see
>> "D over nothing".  Here, get LBA Status should claim "deallocated" state
>> for anything not allocated in D.
> 
> Ok, this makes more sense.
> 
> The question then is whether to implement this NBD server inside QEMU,
> or outside it as a separate process to which QEMU "pushes" blocks as in
> the existing backup job.  I would prefer the latter, so that it is
> possible to implement various APIs (get block status, but also VMware or
> Parallels or whatever).
> 
> Basically the same points made in
> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
> still apply.

Talked a bit to Fam now and I noted Denis's observation that QEMU would
still use the backup block job, plus the NBD server as in Fam's
fleecing.  Then the NBD server is already the push->pull adapter.  It's
a bit clearer now.

Opening D without backing file still feels a bit weird, because the NBD
server would provide wrong data for clean blocks.  I would think that a
"stupid" backup software could always ignore the get LBA status command
and get a full backup.  Is this a requirement or not, and if not, why?

I don't have any particular opinion against an NBD get LBA status
command that returns deallocated/allocated _and_ clean/dirty.  But
reusing one as the other feels like the kind of hack that seems clever
and that you regret down the road.

I'm not sure whether the clean/dirty status is data plane or control
plane either.  I don't think the terms have a well-defined meaning in
terms of storage.  In the networking world, stuff like routing protocols
(OSPF, BGP, etc.) is control plane, and based on this analogy dirty
bitmaps seem like control plane to me.

So I wouldn't rule out QMP-based export of the dirty bitmap---either
directly or optimized through a separate socket as in
http://article.gmane.org/gmane.comp.emulators.qemu/397083.  Dirty
bitmaps should also compress well, so perhaps gzip+base64 over JSON
might work as well.  I'm not saying we certainly won't regret it, but it
seems "less different".  Don't really know how to put it better...

Paolo

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-29  9:42                             ` Paolo Bonzini
@ 2016-02-29 10:05                               ` Fam Zheng
  2016-03-10 17:37                               ` Stefan Hajnoczi
  1 sibling, 0 replies; 56+ messages in thread
From: Fam Zheng @ 2016-02-29 10:05 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, Markus Armbruster, jsnow

On Mon, 02/29 10:42, Paolo Bonzini wrote:
> 
> 
> On 29/02/2016 09:54, Paolo Bonzini wrote:
> > 
> > 
> > On 29/02/2016 09:14, Markus Armbruster wrote:
> >> I completely agree with you that Get LBA Status cannot just reflect the
> >> top layer.  But that's not what I meant to propose.  Let me try to
> >> explain myself more clearly.
> >>
> >> Consider a QCOW2 image D (for delta) with a backing file B (for base).
> >> If you open it normally, you see "D over B".  Get LBA Status should
> >> certainly claim the "deallocated" state only for blocks that are
> >> allocated neither in D nor B.
> >>
> >> However, you can also open D *without* its backing file.  Then you see
> >> "D over nothing".  Here, get LBA Status should claim "deallocated" state
> >> for anything not allocated in D.
> > 
> > Ok, this makes more sense.
> > 
> > The question then is whether to implement this NBD server inside QEMU,
> > or outside it as a separate process to which QEMU "pushes" blocks as in
> > the existing backup job.  I would prefer the latter, so that it is
> > possible to implement various APIs (get block status, but also VMware or
> > Parallels or whatever).
> > 
> > Basically the same points made in
> > https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
> > still apply.
> 
> Talked a bit to Fam now and I noted Denis's observation that QEMU would
> still use the backup block job, plus the NBD server as in Fam's
> fleecing.  Then the NBD server is already the push->pull adapter.  It's
> a bit clearer now.
> 
> Opening D without backing file still feels a bit weird, because the NBD
> server would provide wrong data for clean blocks.  I would think that a
> "stupid" backup software could always ignore the get LBA status command
> and get a full backup.  Is this a requirement or not, and if not, why?

Can we return -EIO instead of zero for unallocated/clean sectors?

> 
> I don't have any particular opinion against an NBD get LBA status
> command that returns deallocated/allocated _and_ clean/dirty.  But
> reusing one as the other feels like the kind of hack that seems clever
> and that you regret down the road.

Yes, I second this, my preference is also leaning towards separate states for
allocation and dirtiness in "Get LBA State" command.

> 
> I'm not sure whether the clean/dirty status is data plane or control
> plane either.  I don't think the terms have a well-defined meaning in
> terms of storage.  In the networking world, stuff like routing protocols
> (OSPF, BGP, etc.) is control plane, and based on this analogy dirty
> bitmaps seem like control plane to me.
> 
> So I wouldn't rule out QMP-based export of the dirty bitmap---either
> directly or optimized through a separate socket as in
> http://article.gmane.org/gmane.comp.emulators.qemu/397083.  Dirty
> bitmaps should also compress well, so perhaps gzip+base64 over JSON
> might work as well.  I'm not saying we certainly won't regret it, but it
> seems "less different".  Don't really know how to put it better...

One more side point here: a backup archive is useless until it has fully
completed, so in this particular use case, push or pull model doesn't matter in
sending the dirty bitmap.  But while we are at it, probably just be future
proof and do a pull iterface to allow more potential use cases? For example,
during data inspecting (a fleecing client is checking whether a particular file
in the guest file system has changed), only certain ranges in the dirty bitmap
is interesting.

Fam

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-29  8:14                         ` Markus Armbruster
  2016-02-29  8:54                           ` Paolo Bonzini
@ 2016-02-29 10:22                           ` Markus Armbruster
  1 sibling, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2016-02-29 10:22 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: kwolf, Vladimir Sementsov-Ogievskiy, Fam Zheng, Denis V. Lunev,
	Stefan Hajnoczi, qemu-devel, jsnow

Markus Armbruster <armbru@redhat.com> writes:

[...]
> Consider a QCOW2 image D (for delta) with a backing file B (for base).
> If you open it normally, you see "D over B".  Get LBA Status should
> certainly claim the "deallocated" state only for blocks that are
> allocated neither in D nor B.
>
> However, you can also open D *without* its backing file.  Then you see
> "D over nothing".  Here, get LBA Status should claim "deallocated" state
> for anything not allocated in D.
>
> My proposal is to expose a "just the dirty blocks" view of a block
> backend similarly: it's a *separate* backend that contains *only* the
> dirty blocks.  Attempts to read a clean block behave exactly like
> reading an unmapped block from any other thinly provisioned backend
> (QCOW2 gives you zeroes, if I remember correctly).  I think it's only
> natural to make Get LBA Status claim "deallocated" for exactly the clean
> blocks then.

Regarding implementation: perhaps we can have a "bitmap filter" block
driver that masks out blocks based on a bitmap.  Read-only, at least
until somebody comes up with sane semantics for writing, and a use case
:)

Digression: masking out blocks based on a bitmap is is vaguely similar
to how a COW block driver replaces blocks based on delta.  What makes
such a COW block driver a *format* is persistence.  Same for bitmap
filters.  The question whether we want a bitmap filter *format* is being
discussed elsewhere.  Even if the answer should be "no", we could still
have a non-persistent filter if it's useful.

[...]

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-02-29  9:42                             ` Paolo Bonzini
  2016-02-29 10:05                               ` Fam Zheng
@ 2016-03-10 17:37                               ` Stefan Hajnoczi
  2016-03-10 17:40                                 ` Paolo Bonzini
  1 sibling, 1 reply; 56+ messages in thread
From: Stefan Hajnoczi @ 2016-03-10 17:37 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Kevin Wolf, Vladimir Sementsov-Ogievskiy, Fam Zheng,
	Denis V. Lunev, qemu-devel, Markus Armbruster, John Snow

On Mon, Feb 29, 2016 at 9:42 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 29/02/2016 09:54, Paolo Bonzini wrote:
>>
>>
>> On 29/02/2016 09:14, Markus Armbruster wrote:
>>> I completely agree with you that Get LBA Status cannot just reflect the
>>> top layer.  But that's not what I meant to propose.  Let me try to
>>> explain myself more clearly.
>>>
>>> Consider a QCOW2 image D (for delta) with a backing file B (for base).
>>> If you open it normally, you see "D over B".  Get LBA Status should
>>> certainly claim the "deallocated" state only for blocks that are
>>> allocated neither in D nor B.
>>>
>>> However, you can also open D *without* its backing file.  Then you see
>>> "D over nothing".  Here, get LBA Status should claim "deallocated" state
>>> for anything not allocated in D.
>>
>> Ok, this makes more sense.
>>
>> The question then is whether to implement this NBD server inside QEMU,
>> or outside it as a separate process to which QEMU "pushes" blocks as in
>> the existing backup job.  I would prefer the latter, so that it is
>> possible to implement various APIs (get block status, but also VMware or
>> Parallels or whatever).
>>
>> Basically the same points made in
>> https://lists.gnu.org/archive/html/qemu-devel/2013-03/msg01969.html
>> still apply.
>
> Talked a bit to Fam now and I noted Denis's observation that QEMU would
> still use the backup block job, plus the NBD server as in Fam's
> fleecing.  Then the NBD server is already the push->pull adapter.  It's
> a bit clearer now.
>
> Opening D without backing file still feels a bit weird, because the NBD
> server would provide wrong data for clean blocks.  I would think that a
> "stupid" backup software could always ignore the get LBA status command
> and get a full backup.  Is this a requirement or not, and if not, why?
>
> I don't have any particular opinion against an NBD get LBA status
> command that returns deallocated/allocated _and_ clean/dirty.  But
> reusing one as the other feels like the kind of hack that seems clever
> and that you regret down the road.

I suggest giving the new NBD command a "type" argument:
0 - SCSI mapped/anchored values according to SCSI Get LBA Status
1 - Dirty/clean, useful for incremental backup and other blocking tracking cases

This way we don't impinge on SCSI semantics and the command can be
used for both traditional logical block provisioning and dirty bitmap
info.

When the NBD export is started in QEMU you can optionally associate it
with a bitmap.  This bitmap is used to provide type=1 (dirty/clean)
status information.

Stefan

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-03-10 17:37                               ` Stefan Hajnoczi
@ 2016-03-10 17:40                                 ` Paolo Bonzini
  2016-03-14 16:27                                   ` Denis V. Lunev
  0 siblings, 1 reply; 56+ messages in thread
From: Paolo Bonzini @ 2016-03-10 17:40 UTC (permalink / raw)
  To: Stefan Hajnoczi
  Cc: Kevin Wolf, Vladimir Sementsov-Ogievskiy, Fam Zheng,
	Denis V. Lunev, qemu-devel, Markus Armbruster, John Snow



On 10/03/2016 18:37, Stefan Hajnoczi wrote:
> I suggest giving the new NBD command a "type" argument:
> 0 - SCSI mapped/anchored values according to SCSI Get LBA Status
> 1 - Dirty/clean, useful for incremental backup and other blocking tracking cases
> 
> This way we don't impinge on SCSI semantics and the command can be
> used for both traditional logical block provisioning and dirty bitmap
> info.
> 
> When the NBD export is started in QEMU you can optionally associate it
> with a bitmap.  This bitmap is used to provide type=1 (dirty/clean)
> status information.

Good idea.  The precedent in NBD is to use bits 16..31 of the command
for flags, so it could go there.

Paolo

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: [Qemu-devel] [PATCH v2 0/6] external backup api
  2016-03-10 17:40                                 ` Paolo Bonzini
@ 2016-03-14 16:27                                   ` Denis V. Lunev
  0 siblings, 0 replies; 56+ messages in thread
From: Denis V. Lunev @ 2016-03-14 16:27 UTC (permalink / raw)
  To: Paolo Bonzini, Stefan Hajnoczi
  Cc: Kevin Wolf, Vladimir Sementsov-Ogievskiy, Fam Zheng,
	Markus Armbruster, qemu-devel, John Snow

On 03/10/2016 08:40 PM, Paolo Bonzini wrote:
>
> On 10/03/2016 18:37, Stefan Hajnoczi wrote:
>> I suggest giving the new NBD command a "type" argument:
>> 0 - SCSI mapped/anchored values according to SCSI Get LBA Status
>> 1 - Dirty/clean, useful for incremental backup and other blocking tracking cases
>>
>> This way we don't impinge on SCSI semantics and the command can be
>> used for both traditional logical block provisioning and dirty bitmap
>> info.
>>
>> When the NBD export is started in QEMU you can optionally associate it
>> with a bitmap.  This bitmap is used to provide type=1 (dirty/clean)
>> status information.
> Good idea.  The precedent in NBD is to use bits 16..31 of the command
> for flags, so it could go there.
>
> Paolo
like this too.

Den

^ permalink raw reply	[flat|nested] 56+ messages in thread

end of thread, other threads:[~2016-03-14 16:27 UTC | newest]

Thread overview: 56+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-30 10:56 [Qemu-devel] [PATCH v2 0/6] external backup api Vladimir Sementsov-Ogievskiy
2016-01-30 10:56 ` [Qemu-devel] [PATCH 1/6] block dirty bitmap: add next_zero function Vladimir Sementsov-Ogievskiy
2016-01-30 10:56 ` [Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges Vladimir Sementsov-Ogievskiy
2016-02-10 10:08   ` Stefan Hajnoczi
2016-02-10 13:57     ` Denis V. Lunev
2016-02-10 15:26       ` John Snow
2016-02-10 15:36         ` Denis V. Lunev
2016-02-10 15:37           ` John Snow
2016-02-10 15:40             ` Denis V. Lunev
2016-02-14  5:05       ` Fam Zheng
2016-01-30 10:56 ` [Qemu-devel] [PATCH 3/6] iotests: test query-block-dirty-bitmap-ranges Vladimir Sementsov-Ogievskiy
2016-01-30 10:56 ` [Qemu-devel] [PATCH 4/6] qapi: add qmp commands for some dirty bitmap functions Vladimir Sementsov-Ogievskiy
2016-01-30 10:56 ` [Qemu-devel] [PATCH 5/6] qapi: make block-dirty-bitmap-create-successor transaction-able Vladimir Sementsov-Ogievskiy
2016-01-30 10:56 ` [Qemu-devel] [PATCH 6/6] iotests: test external backup api Vladimir Sementsov-Ogievskiy
2016-02-03  8:14 ` [Qemu-devel] [PATCH v2 0/6] " Fam Zheng
2016-02-03 10:57   ` Vladimir Sementsov-Ogievskiy
2016-02-03 11:02     ` Fam Zheng
2016-02-03 11:24       ` Vladimir Sementsov-Ogievskiy
2016-02-05  8:28   ` Denis V. Lunev
2016-02-05  8:44     ` Fam Zheng
2016-02-09 14:21     ` Stefan Hajnoczi
2016-02-09 14:37       ` Denis V. Lunev
2016-02-09 16:49         ` John Snow
2016-02-09 16:58           ` Denis V. Lunev
2016-02-09 18:12             ` John Snow
2016-02-09 19:25               ` Denis V. Lunev
2016-02-10  8:04                 ` Denis V. Lunev
2016-02-09 14:28     ` Stefan Hajnoczi
2016-02-09 14:41       ` Denis V. Lunev
2016-02-10 10:10         ` Stefan Hajnoczi
2016-02-16 17:09           ` Stefan Hajnoczi
2016-02-16 17:17             ` Vladimir Sementsov-Ogievskiy
2016-02-16 17:20             ` Denis V. Lunev
2016-02-18 16:39               ` Stefan Hajnoczi
2016-02-18 17:07                 ` Markus Armbruster
2016-02-17 17:47             ` Vladimir Sementsov-Ogievskiy
2016-02-18  0:59               ` Fam Zheng
2016-02-18 12:11               ` Daniel P. Berrange
2016-02-18 16:41                 ` Stefan Hajnoczi
2016-02-19  2:08                   ` Fam Zheng
2016-02-19  8:51                     ` Markus Armbruster
2016-02-24 23:34                       ` John Snow
2016-02-26 19:55                       ` Paolo Bonzini
2016-02-26 20:03                         ` Paolo Bonzini
2016-02-26 20:29                           ` Denis V. Lunev
2016-02-26 21:37                           ` John Snow
2016-02-26 20:40                         ` Denis V. Lunev
2016-02-27  4:26                           ` Fam Zheng
2016-02-29  8:14                         ` Markus Armbruster
2016-02-29  8:54                           ` Paolo Bonzini
2016-02-29  9:42                             ` Paolo Bonzini
2016-02-29 10:05                               ` Fam Zheng
2016-03-10 17:37                               ` Stefan Hajnoczi
2016-03-10 17:40                                 ` Paolo Bonzini
2016-03-14 16:27                                   ` Denis V. Lunev
2016-02-29 10:22                           ` Markus Armbruster

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.