[Qemu-devel] [PATCH v2 0/3] fix bitmaps migration through shared storage

[Qemu-devel] [PATCH v2 0/3] fix bitmaps migration through shared storage

[Vladimir Sementsov-Ogievskiy]

Hi all.

This fixes bitmaps migration through shared storage. Look at 02 for
details.

The bug introduced in 2.10 with the whole qcow2 bitmaps feature, so
qemu-stable in CC. However I doubt that someone really suffered from this.

v2:
   John, thank you for reviewing v1.
   changes:
    add John's r-bs, change s/timeout=10/timeout=10.0/ in last patch
    and drop old 03 patch, related to this timeout fix.

Vladimir Sementsov-Ogievskiy (3):
  qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint()
  qcow2: handle reopening bitmaps on bdrv_invalidate_cache
  iotests: add dirty bitmap migration test

 block/qcow2.h              |  2 ++
 block/qcow2-bitmap.c       | 15 ++++++++-
 block/qcow2.c              |  8 ++++-
 tests/qemu-iotests/169     | 82 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/169.out |  5 +++
 tests/qemu-iotests/group   |  1 +
 6 files changed, 111 insertions(+), 2 deletions(-)
 create mode 100755 tests/qemu-iotests/169
 create mode 100644 tests/qemu-iotests/169.out

-- 
2.11.1

[Qemu-devel] [PATCH v2 1/3] qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint()

[Vladimir Sementsov-Ogievskiy]

Add version of qcow2_reopen_bitmaps_rw, which do the same work but
also return a hint about was header updated or not. This will be
used in the following fix for bitmaps reloading after migration.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
---
 block/qcow2.h        |  2 ++
 block/qcow2-bitmap.c | 15 ++++++++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/block/qcow2.h b/block/qcow2.h
index 6f0ff15dd0..40fa5b7cfe 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -667,6 +667,8 @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
                                   void **refcount_table,
                                   int64_t *refcount_table_size);
 bool qcow2_load_autoloading_dirty_bitmaps(BlockDriverState *bs, Error **errp);
+int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
+                                 Error **errp);
 int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp);
 void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
 int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index f45e46cfbd..6c80dcda69 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1002,7 +1002,8 @@ fail:
     return false;
 }
 
-int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp)
+int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
+                                 Error **errp)
 {
     BDRVQcow2State *s = bs->opaque;
     Qcow2BitmapList *bm_list;
@@ -1010,6 +1011,10 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp)
     GSList *ro_dirty_bitmaps = NULL;
     int ret = 0;
 
+    if (header_updated != NULL) {
+        *header_updated = false;
+    }
+
     if (s->nb_bitmaps == 0) {
         /* No bitmaps - nothing to do */
         return 0;
@@ -1053,6 +1058,9 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp)
             error_setg_errno(errp, -ret, "Can't update bitmap directory");
             goto out;
         }
+        if (header_updated != NULL) {
+            *header_updated = true;
+        }
         g_slist_foreach(ro_dirty_bitmaps, set_readonly_helper, false);
     }
 
@@ -1063,6 +1071,11 @@ out:
     return ret;
 }
 
+int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp)
+{
+    return qcow2_reopen_bitmaps_rw_hint(bs, NULL, errp);
+}
+
 /* store_bitmap_data()
  * Store bitmap to image, filling bitmap table accordingly.
  */
-- 
2.11.1

[Qemu-devel] [PATCH v2 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache

[Vladimir Sementsov-Ogievskiy]

Consider migration with shared storage. Persistent bitmaps are stored
on bdrv_inactivate. Then, on destination
process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which
leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps
are already loaded on destination start. In this case we should call
qcow2_reopen_bitmaps_rw instead.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
---
 block/qcow2.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 1914a940e5..f5f205e1b7 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1450,7 +1450,13 @@ static int qcow2_do_open(BlockDriverState *bs, QDict *options, int flags,
         s->autoclear_features &= QCOW2_AUTOCLEAR_MASK;
     }
 
-    if (qcow2_load_autoloading_dirty_bitmaps(bs, &local_err)) {
+    if (bdrv_has_readonly_bitmaps(bs)) {
+        if (!bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE)) {
+            bool header_updated = false;
+            qcow2_reopen_bitmaps_rw_hint(bs, &header_updated, &local_err);
+            update_header = update_header && !header_updated;
+        }
+    } else if (qcow2_load_autoloading_dirty_bitmaps(bs, &local_err)) {
         update_header = false;
     }
     if (local_err != NULL) {
-- 
2.11.1

[Qemu-devel] [PATCH v2 3/3] iotests: add dirty bitmap migration test

[Vladimir Sementsov-Ogievskiy]

The test creates two vms (vm_a, vm_b), create dirty bitmap in
the first one, do several writes to corresponding device and
then migrate vm_a to vm_b with dirty bitmaps.

For now, only migration through shared storage for persistent
bitmaps is available, so it is tested here. Only offline variant
is tested for now (a kind of suspend-resume), as it is needed
to test that this case is successfully fixed by recent patch.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
---
 tests/qemu-iotests/169     | 82 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/169.out |  5 +++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 88 insertions(+)
 create mode 100755 tests/qemu-iotests/169
 create mode 100644 tests/qemu-iotests/169.out

diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169
new file mode 100755
index 0000000000..282970df8a
--- /dev/null
+++ b/tests/qemu-iotests/169
@@ -0,0 +1,82 @@
+#!/usr/bin/env python
+#
+# Tests for dirty bitmaps migration.
+#
+# Copyright (c) 2016-2017 Virtuozzo International GmbH. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk = os.path.join(iotests.test_dir, 'disk')
+migfile = os.path.join(iotests.test_dir, 'migfile')
+
+class TestPersistentDirtyBitmapSuspendResume(iotests.QMPTestCase):
+
+    def tearDown(self):
+        self.vm_a.shutdown()
+        self.vm_b.shutdown()
+        os.remove(disk)
+        os.remove(migfile)
+
+    def setUp(self):
+        qemu_img('create', '-f', iotests.imgfmt, disk, '1M')
+
+        self.vm_a = iotests.VM(path_suffix='a').add_drive(disk)
+        self.vm_a.launch()
+
+        self.vm_b = iotests.VM(path_suffix='b').add_drive(disk)
+        self.vm_b.add_incoming("exec: cat '" + migfile + "'")
+
+    def test_migration_persistent_shared_offline(self):
+        """ A kind of suspend-resume """
+        granularity = 512
+        regions = [
+            { 'start': 0,           'count': 0x10000 },
+            { 'start': 0xf0000,     'count': 0x10000 },
+            { 'start': 0xa0201,     'count': 0x1000  }
+            ]
+
+        result = self.vm_a.qmp('block-dirty-bitmap-add', node='drive0',
+                               name='bitmap0', granularity=granularity,
+                               persistent=True, autoload=True)
+        self.assert_qmp(result, 'return', {});
+
+        for r in regions:
+            self.vm_a.hmp_qemu_io('drive0',
+                                  'write %d %d' % (r['start'], r['count']))
+
+        result = self.vm_a.qmp('x-debug-block-dirty-bitmap-sha256',
+                               node='drive0', name='bitmap0')
+        sha256 = result['return']['sha256']
+
+        result = self.vm_a.qmp('migrate', uri='exec:cat>' + migfile)
+        self.assert_qmp(result, 'return', {});
+        self.assertNotEqual(self.vm_a.event_wait("STOP"), None)
+        self.vm_a.shutdown()
+
+        self.vm_b.launch()
+        self.vm_b.event_wait("RESUME", timeout=10.0)
+
+        result = self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256',
+                               node='drive0', name='bitmap0')
+        self.assert_qmp(result, 'return/sha256', sha256);
+
+
+if __name__ == '__main__':
+    iotests.main(supported_fmts=['qcow2'])
diff --git a/tests/qemu-iotests/169.out b/tests/qemu-iotests/169.out
new file mode 100644
index 0000000000..ae1213e6f8
--- /dev/null
+++ b/tests/qemu-iotests/169.out
@@ -0,0 +1,5 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 3e688678dd..e879c7ebc7 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -169,6 +169,7 @@
 162 auto quick
 163 rw auto quick
 165 rw auto quick
+169 rw auto quick
 170 rw auto quick
 171 rw auto quick
 172 auto
-- 
2.11.1

Re: [Qemu-devel] [PATCH v2 0/3] fix bitmaps migration through shared storage

[Vladimir Sementsov-Ogievskiy]

Should qemu-stable be here?


12.12.2017 19:04, Vladimir Sementsov-Ogievskiy wrote:
> Hi all.
>
> This fixes bitmaps migration through shared storage. Look at 02 for
> details.
>
> The bug introduced in 2.10 with the whole qcow2 bitmaps feature, so
> qemu-stable in CC. However I doubt that someone really suffered from this.
>
> v2:
>     John, thank you for reviewing v1.
>     changes:
>      add John's r-bs, change s/timeout=10/timeout=10.0/ in last patch
>      and drop old 03 patch, related to this timeout fix.
>
> Vladimir Sementsov-Ogievskiy (3):
>    qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint()
>    qcow2: handle reopening bitmaps on bdrv_invalidate_cache
>    iotests: add dirty bitmap migration test
>
>   block/qcow2.h              |  2 ++
>   block/qcow2-bitmap.c       | 15 ++++++++-
>   block/qcow2.c              |  8 ++++-
>   tests/qemu-iotests/169     | 82 ++++++++++++++++++++++++++++++++++++++++++++++
>   tests/qemu-iotests/169.out |  5 +++
>   tests/qemu-iotests/group   |  1 +
>   6 files changed, 111 insertions(+), 2 deletions(-)
>   create mode 100755 tests/qemu-iotests/169
>   create mode 100644 tests/qemu-iotests/169.out
>


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v2 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache

[Kevin Wolf]

Am 12.12.2017 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
> Consider migration with shared storage. Persistent bitmaps are stored
> on bdrv_inactivate. Then, on destination
> process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which
> leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps
> are already loaded on destination start. In this case we should call
> qcow2_reopen_bitmaps_rw instead.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Reviewed-by: John Snow <jsnow@redhat.com>

qcow2_invalidate_cache() calls qcow2_close() first, so why are there
still any bitmaps loaded? Isn't this a bug? Do we leak bitmaps when a
qcow2 image is closed?

Kevin

Re: [Qemu-devel] [PATCH v2 3/3] iotests: add dirty bitmap migration test

[Kevin Wolf]

Am 12.12.2017 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
> The test creates two vms (vm_a, vm_b), create dirty bitmap in
> the first one, do several writes to corresponding device and
> then migrate vm_a to vm_b with dirty bitmaps.
> 
> For now, only migration through shared storage for persistent
> bitmaps is available, so it is tested here. Only offline variant
> is tested for now (a kind of suspend-resume), as it is needed
> to test that this case is successfully fixed by recent patch.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Reviewed-by: John Snow <jsnow@redhat.com>

> [...]
> +        self.vm_b.launch()
> +        self.vm_b.event_wait("RESUME", timeout=10.0)

Wasn't this racy because the RESUME event may be sent before we connect
to the socket?

Kevin

Re: [Qemu-devel] [PATCH v2 3/3] iotests: add dirty bitmap migration test

[Vladimir Sementsov-Ogievskiy]

22.12.2017 16:43, Kevin Wolf wrote:
> Am 12.12.2017 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> The test creates two vms (vm_a, vm_b), create dirty bitmap in
>> the first one, do several writes to corresponding device and
>> then migrate vm_a to vm_b with dirty bitmaps.
>>
>> For now, only migration through shared storage for persistent
>> bitmaps is available, so it is tested here. Only offline variant
>> is tested for now (a kind of suspend-resume), as it is needed
>> to test that this case is successfully fixed by recent patch.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Reviewed-by: John Snow <jsnow@redhat.com>
>> [...]
>> +        self.vm_b.launch()
>> +        self.vm_b.event_wait("RESUME", timeout=10.0)
> Wasn't this racy because the RESUME event may be sent before we connect
> to the socket?

Didn't know (and didn't faced) it. If it is so, it should be changed 
like you've changed similar place in my previous iotest.

>
> Kevin


-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v2 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache

[Vladimir Sementsov-Ogievskiy]

22.12.2017 16:39, Kevin Wolf wrote:
> Am 12.12.2017 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> Consider migration with shared storage. Persistent bitmaps are stored
>> on bdrv_inactivate. Then, on destination
>> process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which
>> leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps
>> are already loaded on destination start. In this case we should call
>> qcow2_reopen_bitmaps_rw instead.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Reviewed-by: John Snow <jsnow@redhat.com>
> qcow2_invalidate_cache() calls qcow2_close() first, so why are there
> still any bitmaps loaded? Isn't this a bug? Do we leak bitmaps when a
> qcow2 image is closed?
>
> Kevin

Interesting point.

Now persistent dirty bitmaps are released at the end of 
bdrv_inactivate_recurse,
which is not called here. It was a separate patch

commit 615b5dcf2decbc5f0abb512d13d7e5db2385fa23
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date:   Wed Jun 28 15:05:30 2017 +0300

     block: release persistent bitmaps on inactivate


May be it is more correct to release them immediately after storing, in
qcow2_inactivete. But it will not fix the issue, because qcow2_close will
call qcow2_inactivate only if (!(s->flags & BDRV_O_INACTIVE)), but it is
not the case.

or we can do like this, it fixes the new test:
   static void qcow2_close(BlockDriverState *bs)
{
       BDRVQcow2State *s = bs->opaque;
qemu_vfree(s->l1_table);
       /* else pre-write overlap checks in cache_destroy may crash */
       s->l1_table = NULL;

       if (!(s->flags & BDRV_O_INACTIVE)) {
qcow2_inactivate(bs);
}
+     bdrv_release_persistent_dirty_bitmaps(bs);


What do you think?

-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v2 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache

[Kevin Wolf]

Am 22.12.2017 um 15:25 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 22.12.2017 16:39, Kevin Wolf wrote:
> > Am 12.12.2017 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > Consider migration with shared storage. Persistent bitmaps are stored
> > > on bdrv_inactivate. Then, on destination
> > > process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which
> > > leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps
> > > are already loaded on destination start. In this case we should call
> > > qcow2_reopen_bitmaps_rw instead.
> > > 
> > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > > Reviewed-by: John Snow <jsnow@redhat.com>
> > qcow2_invalidate_cache() calls qcow2_close() first, so why are there
> > still any bitmaps loaded? Isn't this a bug? Do we leak bitmaps when a
> > qcow2 image is closed?
> > 
> > Kevin
> 
> Interesting point.
> 
> Now persistent dirty bitmaps are released at the end of
> bdrv_inactivate_recurse,
> which is not called here. It was a separate patch
> 
> commit 615b5dcf2decbc5f0abb512d13d7e5db2385fa23
> Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Date:   Wed Jun 28 15:05:30 2017 +0300
> 
>     block: release persistent bitmaps on inactivate
> 
> May be it is more correct to release them immediately after storing, in
> qcow2_inactivete.

I chose the question form because I'm really not deep enough into the
bitmap code to have a solid opinion, but I have a feeling that releasing
the bitmaps from the block driver that provided them would be cleaner
indeed. I suppose the same is true for .bdrv_close.

> But it will not fix the issue, because qcow2_close will
> call qcow2_inactivate only if (!(s->flags & BDRV_O_INACTIVE)), but it is
> not the case.

Yes, good point.

Is there a reason why bitmaps are already loaded in qcow2_do_open() even
if the image is inactive? Can bitmaps be meaningfully used on inactive
images?

Otherwise, we could just make qcow2_load_autoloading_dirty_bitmaps()
conditional on cleared BDRV_O_INACTIVE.

> or we can do like this, it fixes the new test:
>   static void qcow2_close(BlockDriverState *bs)
> {
>       BDRVQcow2State *s = bs->opaque;
> qemu_vfree(s->l1_table);
>       /* else pre-write overlap checks in cache_destroy may crash */
>       s->l1_table = NULL;
> 
>       if (!(s->flags & BDRV_O_INACTIVE)) {
> qcow2_inactivate(bs);
> }
> +     bdrv_release_persistent_dirty_bitmaps(bs);
> 
> What do you think?

Hm, I think I don't like this much.

We just need to decide what the status of inactive images is supposed to
be. If they should have bitmaps, then your patch is probably right. But
if inactive images shouldn't have any, we need to change qcow2_do_open()
and qcow2_inactivate().

Kevin

Re: [Qemu-devel] [PATCH v2 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache

[Vladimir Sementsov-Ogievskiy]

22.12.2017 18:43, Kevin Wolf wrote:
> Am 22.12.2017 um 15:25 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> 22.12.2017 16:39, Kevin Wolf wrote:
>>> Am 12.12.2017 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>> Consider migration with shared storage. Persistent bitmaps are stored
>>>> on bdrv_inactivate. Then, on destination
>>>> process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which
>>>> leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps
>>>> are already loaded on destination start. In this case we should call
>>>> qcow2_reopen_bitmaps_rw instead.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> Reviewed-by: John Snow <jsnow@redhat.com>
>>> qcow2_invalidate_cache() calls qcow2_close() first, so why are there
>>> still any bitmaps loaded? Isn't this a bug? Do we leak bitmaps when a
>>> qcow2 image is closed?
>>>
>>> Kevin
>> Interesting point.
>>
>> Now persistent dirty bitmaps are released at the end of
>> bdrv_inactivate_recurse,
>> which is not called here. It was a separate patch
>>
>> commit 615b5dcf2decbc5f0abb512d13d7e5db2385fa23
>> Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> Date:   Wed Jun 28 15:05:30 2017 +0300
>>
>>      block: release persistent bitmaps on inactivate
>>
>> May be it is more correct to release them immediately after storing, in
>> qcow2_inactivete.
> I chose the question form because I'm really not deep enough into the
> bitmap code to have a solid opinion, but I have a feeling that releasing
> the bitmaps from the block driver that provided them would be cleaner
> indeed. I suppose the same is true for .bdrv_close.
>
>> But it will not fix the issue, because qcow2_close will
>> call qcow2_inactivate only if (!(s->flags & BDRV_O_INACTIVE)), but it is
>> not the case.
> Yes, good point.
>
> Is there a reason why bitmaps are already loaded in qcow2_do_open() even
> if the image is inactive? Can bitmaps be meaningfully used on inactive
> images?
>
> Otherwise, we could just make qcow2_load_autoloading_dirty_bitmaps()
> conditional on cleared BDRV_O_INACTIVE.
>
>> or we can do like this, it fixes the new test:
>>    static void qcow2_close(BlockDriverState *bs)
>> {
>>        BDRVQcow2State *s = bs->opaque;
>> qemu_vfree(s->l1_table);
>>        /* else pre-write overlap checks in cache_destroy may crash */
>>        s->l1_table = NULL;
>>
>>        if (!(s->flags & BDRV_O_INACTIVE)) {
>> qcow2_inactivate(bs);
>> }
>> +     bdrv_release_persistent_dirty_bitmaps(bs);
>>
>> What do you think?
> Hm, I think I don't like this much.
>
> We just need to decide what the status of inactive images is supposed to
> be. If they should have bitmaps, then your patch is probably right. But
> if inactive images shouldn't have any, we need to change qcow2_do_open()
> and qcow2_inactivate().
>
> Kevin

Does Qemu start in inactive mode when and only when it is incoming 
migration? In this case I don't
see any reason of early-load the bitmaps. Backup in inactive mode should 
not be allowed too, yes?

So, it looks like it's ok to just do not autoload bitmaps if we are in 
inactive mode. The difference
would be that user will not see these bitmaps during migration. And he 
even may create bitmaps
with same names, which will lead to fault. But it don't look like real 
problem.

So, if there will not be other thoughts, I'll make another patch.

-- 
Best regards,
Vladimir

Re: [Qemu-devel] [PATCH v2 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache

[Kevin Wolf]

Am 22.12.2017 um 17:12 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 22.12.2017 18:43, Kevin Wolf wrote:
> > Am 22.12.2017 um 15:25 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > 22.12.2017 16:39, Kevin Wolf wrote:
> > > > Am 12.12.2017 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > > > Consider migration with shared storage. Persistent bitmaps are stored
> > > > > on bdrv_inactivate. Then, on destination
> > > > > process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which
> > > > > leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps
> > > > > are already loaded on destination start. In this case we should call
> > > > > qcow2_reopen_bitmaps_rw instead.
> > > > > 
> > > > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > > > > Reviewed-by: John Snow <jsnow@redhat.com>
> > > > qcow2_invalidate_cache() calls qcow2_close() first, so why are there
> > > > still any bitmaps loaded? Isn't this a bug? Do we leak bitmaps when a
> > > > qcow2 image is closed?
> > > > 
> > > > Kevin
> > > Interesting point.
> > > 
> > > Now persistent dirty bitmaps are released at the end of
> > > bdrv_inactivate_recurse,
> > > which is not called here. It was a separate patch
> > > 
> > > commit 615b5dcf2decbc5f0abb512d13d7e5db2385fa23
> > > Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > > Date:   Wed Jun 28 15:05:30 2017 +0300
> > > 
> > >      block: release persistent bitmaps on inactivate
> > > 
> > > May be it is more correct to release them immediately after storing, in
> > > qcow2_inactivete.
> > I chose the question form because I'm really not deep enough into the
> > bitmap code to have a solid opinion, but I have a feeling that releasing
> > the bitmaps from the block driver that provided them would be cleaner
> > indeed. I suppose the same is true for .bdrv_close.
> > 
> > > But it will not fix the issue, because qcow2_close will
> > > call qcow2_inactivate only if (!(s->flags & BDRV_O_INACTIVE)), but it is
> > > not the case.
> > Yes, good point.
> > 
> > Is there a reason why bitmaps are already loaded in qcow2_do_open() even
> > if the image is inactive? Can bitmaps be meaningfully used on inactive
> > images?
> > 
> > Otherwise, we could just make qcow2_load_autoloading_dirty_bitmaps()
> > conditional on cleared BDRV_O_INACTIVE.
> > 
> > > or we can do like this, it fixes the new test:
> > >    static void qcow2_close(BlockDriverState *bs)
> > > {
> > >        BDRVQcow2State *s = bs->opaque;
> > > qemu_vfree(s->l1_table);
> > >        /* else pre-write overlap checks in cache_destroy may crash */
> > >        s->l1_table = NULL;
> > > 
> > >        if (!(s->flags & BDRV_O_INACTIVE)) {
> > > qcow2_inactivate(bs);
> > > }
> > > +     bdrv_release_persistent_dirty_bitmaps(bs);
> > > 
> > > What do you think?
> > Hm, I think I don't like this much.
> > 
> > We just need to decide what the status of inactive images is supposed to
> > be. If they should have bitmaps, then your patch is probably right. But
> > if inactive images shouldn't have any, we need to change qcow2_do_open()
> > and qcow2_inactivate().
> > 
> > Kevin
> 
> Does Qemu start in inactive mode when and only when it is incoming
> migration? In this case I don't see any reason of early-load the
> bitmaps. Backup in inactive mode should not be allowed too, yes?

Yes, inactive images are only for incoming migration at the moment.

I would like to change that into opening all images as inactive and then
immediately activate them (just to unify two code paths, one of which
isn't tested as much as it should be), but that wouldn't be visible
externally, so it should be fine, too.

> So, it looks like it's ok to just do not autoload bitmaps if we are in
> inactive mode. The difference would be that user will not see these
> bitmaps during migration. And he even may create bitmaps with same
> names, which will lead to fault. But it don't look like real problem.

You can't create persistent bitmaps on an inactive image anyway, because
that would involve writes.

Conflicts with temporary bitmaps might be possible, though.

Kevin

Re: [Qemu-devel] [PATCH v2 2/3] qcow2: handle reopening bitmaps on bdrv_invalidate_cache

[Vladimir Sementsov-Ogievskiy]

22.12.2017 19:28, Kevin Wolf wrote:
> Am 22.12.2017 um 17:12 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> 22.12.2017 18:43, Kevin Wolf wrote:
>>> Am 22.12.2017 um 15:25 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>> 22.12.2017 16:39, Kevin Wolf wrote:
>>>>> Am 12.12.2017 um 17:04 hat Vladimir Sementsov-Ogievskiy geschrieben:
>>>>>> Consider migration with shared storage. Persistent bitmaps are stored
>>>>>> on bdrv_inactivate. Then, on destination
>>>>>> process_incoming_migration_bh() calls bdrv_invalidate_cache_all() which
>>>>>> leads to qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps
>>>>>> are already loaded on destination start. In this case we should call
>>>>>> qcow2_reopen_bitmaps_rw instead.
>>>>>>
>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>>> Reviewed-by: John Snow <jsnow@redhat.com>
>>>>> qcow2_invalidate_cache() calls qcow2_close() first, so why are there
>>>>> still any bitmaps loaded? Isn't this a bug? Do we leak bitmaps when a
>>>>> qcow2 image is closed?
>>>>>
>>>>> Kevin
>>>> Interesting point.
>>>>
>>>> Now persistent dirty bitmaps are released at the end of
>>>> bdrv_inactivate_recurse,
>>>> which is not called here. It was a separate patch
>>>>
>>>> commit 615b5dcf2decbc5f0abb512d13d7e5db2385fa23
>>>> Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> Date:   Wed Jun 28 15:05:30 2017 +0300
>>>>
>>>>       block: release persistent bitmaps on inactivate
>>>>
>>>> May be it is more correct to release them immediately after storing, in
>>>> qcow2_inactivete.
>>> I chose the question form because I'm really not deep enough into the
>>> bitmap code to have a solid opinion, but I have a feeling that releasing
>>> the bitmaps from the block driver that provided them would be cleaner
>>> indeed. I suppose the same is true for .bdrv_close.
>>>
>>>> But it will not fix the issue, because qcow2_close will
>>>> call qcow2_inactivate only if (!(s->flags & BDRV_O_INACTIVE)), but it is
>>>> not the case.
>>> Yes, good point.
>>>
>>> Is there a reason why bitmaps are already loaded in qcow2_do_open() even
>>> if the image is inactive? Can bitmaps be meaningfully used on inactive
>>> images?
>>>
>>> Otherwise, we could just make qcow2_load_autoloading_dirty_bitmaps()
>>> conditional on cleared BDRV_O_INACTIVE.
>>>
>>>> or we can do like this, it fixes the new test:
>>>>     static void qcow2_close(BlockDriverState *bs)
>>>> {
>>>>         BDRVQcow2State *s = bs->opaque;
>>>> qemu_vfree(s->l1_table);
>>>>         /* else pre-write overlap checks in cache_destroy may crash */
>>>>         s->l1_table = NULL;
>>>>
>>>>         if (!(s->flags & BDRV_O_INACTIVE)) {
>>>> qcow2_inactivate(bs);
>>>> }
>>>> +     bdrv_release_persistent_dirty_bitmaps(bs);
>>>>
>>>> What do you think?
>>> Hm, I think I don't like this much.
>>>
>>> We just need to decide what the status of inactive images is supposed to
>>> be. If they should have bitmaps, then your patch is probably right. But
>>> if inactive images shouldn't have any, we need to change qcow2_do_open()
>>> and qcow2_inactivate().
>>>
>>> Kevin
>> Does Qemu start in inactive mode when and only when it is incoming
>> migration? In this case I don't see any reason of early-load the
>> bitmaps. Backup in inactive mode should not be allowed too, yes?
> Yes, inactive images are only for incoming migration at the moment.
>
> I would like to change that into opening all images as inactive and then
> immediately activate them (just to unify two code paths, one of which
> isn't tested as much as it should be), but that wouldn't be visible
> externally, so it should be fine, too.

similar approach led to a problem for us:

To make it possible to modify drives through nbd before start (a kind of 
external reatore), I've added simple
@@ -4727,6 +4727,8 @@ int main(int argc, char **argv, char **envp)
          }
      } else if (autostart) {
          vm_start();
+    } else {
+        bdrv_inactivate_all();
      }

and now, we have no dirty bitmaps between paused start and cont. And it 
is not comfortable, we can't query, can't
do something with these bitmaps before vm start.

>
>> So, it looks like it's ok to just do not autoload bitmaps if we are in
>> inactive mode. The difference would be that user will not see these
>> bitmaps during migration. And he even may create bitmaps with same
>> names, which will lead to fault. But it don't look like real problem.
> You can't create persistent bitmaps on an inactive image anyway, because
> that would involve writes.
>
> Conflicts with temporary bitmaps might be possible, though.
>
> Kevin


-- 
Best regards,
Vladimir