qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335
@ 2019-09-13 17:27 Maxim Levitsky
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 1/3] " Maxim Levitsky
                   ` (3 more replies)
  0 siblings, 4 replies; 8+ messages in thread
From: Maxim Levitsky @ 2019-09-13 17:27 UTC (permalink / raw)
  To: qemu-devel
  Cc: Kevin Wolf, Vladimir Sementsov-Ogievskiy,
	Daniel P . Berrangé,
	qemu-block, qemu-stable, Max Reitz, Maxim Levitsky

Commit 8ac0f15f335 accidently broke the COW of non changed areas
of newly allocated clusters, when the write spans multiple clusters,
and needs COW both prior and after the write.
This results in 'after' COW area being encrypted with wrong
sector address, which render it corrupted.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1745922

CC: qemu-stable <qemu-stable@nongnu.org>

V2: grammar, spelling and code style fixes.
V3: more fixes after the review.
V4: addressed review comments from Max Reitz,
    and futher refactored the qcow2_co_encrypt to just take full host and guest offset
    which simplifies everything.

V5: reworked the patches so one of them fixes the bug
    only and other one is just refactoring

V6: removed do_perform_cow_encrypt

Best regards,
	Maxim Levitsky

Maxim Levitsky (3):
  Fix qcow2+luks corruption introduced by commit 8ac0f15f335
  block/qcow2: refactor threaded encryption code
  qemu-iotests: Add test for bz #1745922

 block/qcow2-cluster.c      | 34 ++++----------
 block/qcow2-threads.c      | 81 +++++++++++++++++++++++++++------
 block/qcow2.c              |  5 ++-
 block/qcow2.h              |  8 ++--
 tests/qemu-iotests/263     | 91 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/263.out | 40 +++++++++++++++++
 tests/qemu-iotests/group   |  1 +
 7 files changed, 215 insertions(+), 45 deletions(-)
 create mode 100755 tests/qemu-iotests/263
 create mode 100644 tests/qemu-iotests/263.out

-- 
2.17.2



^ permalink raw reply	[flat|nested] 8+ messages in thread

* [Qemu-devel] [PATCH v6 1/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335
  2019-09-13 17:27 [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335 Maxim Levitsky
@ 2019-09-13 17:27 ` Maxim Levitsky
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code Maxim Levitsky
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 8+ messages in thread
From: Maxim Levitsky @ 2019-09-13 17:27 UTC (permalink / raw)
  To: qemu-devel
  Cc: Kevin Wolf, Vladimir Sementsov-Ogievskiy,
	Daniel P . Berrangé,
	qemu-block, qemu-stable, Max Reitz, Maxim Levitsky

This fixes subtle corruption introduced by luks threaded encryption
in commit 8ac0f15f335

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1745922

The corruption happens when we do a write that
   * writes to two or more unallocated clusters at once
   * doesn't fully cover the first sector
   * doesn't fully cover the last sector

In this case, when allocating the new clusters we COW both areas
prior to the write and after the write, and we encrypt them.

The above mentioned commit accidentally made it so we encrypt the
second COW area using the physical cluster offset of the first area.

The problem is that offset_in_cluster in do_perform_cow_encrypt
can be larger that the cluster size, thus cluster_offset
will no longer point to the start of the cluster at which encrypted
area starts.

Next patch in this series will refactor the code to avoid all these
assumptions.

In the bugreport that was triggered by rebasing a luks image to new,
zero filled base, which lot of such writes, and causes some files
with zero areas to contain garbage there instead.
But as described above it can happen elsewhere as well


Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-cluster.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index dcacd3c450..bfeb0241d7 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -474,9 +474,10 @@ static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
         assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
         assert((bytes & ~BDRV_SECTOR_MASK) == 0);
         assert(s->crypto);
-        if (qcow2_co_encrypt(bs, cluster_offset,
-                             src_cluster_offset + offset_in_cluster,
-                             buffer, bytes) < 0) {
+        if (qcow2_co_encrypt(bs,
+                start_of_cluster(s, cluster_offset + offset_in_cluster),
+                src_cluster_offset + offset_in_cluster,
+                buffer, bytes) < 0) {
             return false;
         }
     }
-- 
2.17.2



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code
  2019-09-13 17:27 [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335 Maxim Levitsky
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 1/3] " Maxim Levitsky
@ 2019-09-13 17:27 ` Maxim Levitsky
  2019-09-13 17:51   ` Vladimir Sementsov-Ogievskiy
  2019-09-13 18:37   ` Vladimir Sementsov-Ogievskiy
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 3/3] qemu-iotests: Add test for bz #1745922 Maxim Levitsky
  2019-09-14  0:28 ` [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335 no-reply
  3 siblings, 2 replies; 8+ messages in thread
From: Maxim Levitsky @ 2019-09-13 17:27 UTC (permalink / raw)
  To: qemu-devel
  Cc: Kevin Wolf, Vladimir Sementsov-Ogievskiy,
	Daniel P . Berrangé,
	qemu-block, qemu-stable, Max Reitz, Maxim Levitsky

Change the qcow2_co_encrypt to just receive full host and
guest offsets and in pariticular remove the
offset_in_cluster parameter of do_perform_cow_encrypt,
since it is misleading, because that offset can be larger than
cluster size currently.

Remove the do_perform_cow_encrypt by merging it with
qcow2_co_encrypt

Also document the qcow2_co_encrypt arguments to prevent
that bug from happening again

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 block/qcow2-cluster.c | 35 +++++--------------
 block/qcow2-threads.c | 81 ++++++++++++++++++++++++++++++++++++-------
 block/qcow2.c         |  5 +--
 block/qcow2.h         |  8 ++---
 4 files changed, 83 insertions(+), 46 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index bfeb0241d7..f42b8a404c 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -462,28 +462,6 @@ static int coroutine_fn do_perform_cow_read(BlockDriverState *bs,
     return 0;
 }
 
-static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
-                                                uint64_t src_cluster_offset,
-                                                uint64_t cluster_offset,
-                                                unsigned offset_in_cluster,
-                                                uint8_t *buffer,
-                                                unsigned bytes)
-{
-    if (bytes && bs->encrypted) {
-        BDRVQcow2State *s = bs->opaque;
-        assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
-        assert((bytes & ~BDRV_SECTOR_MASK) == 0);
-        assert(s->crypto);
-        if (qcow2_co_encrypt(bs,
-                start_of_cluster(s, cluster_offset + offset_in_cluster),
-                src_cluster_offset + offset_in_cluster,
-                buffer, bytes) < 0) {
-            return false;
-        }
-    }
-    return true;
-}
-
 static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
                                              uint64_t cluster_offset,
                                              unsigned offset_in_cluster,
@@ -891,11 +869,14 @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta *m)
 
     /* Encrypt the data if necessary before writing it */
     if (bs->encrypted) {
-        if (!do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
-                                    start->offset, start_buffer,
-                                    start->nb_bytes) ||
-            !do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
-                                    end->offset, end_buffer, end->nb_bytes)) {
+        if (!qcow2_co_encrypt(bs,
+                              m->offset + start->offset,
+                              m->alloc_offset + start->offset,
+                              start_buffer, start->nb_bytes) ||
+            !qcow2_co_encrypt(bs,
+                              m->offset + end->offset,
+                              m->alloc_offset + end->offset,
+                              end_buffer, end->nb_bytes)) {
             ret = -EIO;
             goto fail;
         }
diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
index 3b1e63fe41..b31d45fb2b 100644
--- a/block/qcow2-threads.c
+++ b/block/qcow2-threads.c
@@ -234,15 +234,15 @@ static int qcow2_encdec_pool_func(void *opaque)
 }
 
 static int coroutine_fn
-qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
-                  uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc func)
+qcow2_co_encdec(BlockDriverState *bs, uint64_t host_offset,
+                uint64_t guest_offset, void *buf, size_t len,
+                Qcow2EncDecFunc func)
 {
     BDRVQcow2State *s = bs->opaque;
+
     Qcow2EncDecData arg = {
         .block = s->crypto,
-        .offset = s->crypt_physical_offset ?
-                      file_cluster_offset + offset_into_cluster(s, offset) :
-                      offset,
+        .offset = s->crypt_physical_offset ? host_offset : guest_offset,
         .buf = buf,
         .len = len,
         .func = func,
@@ -251,18 +251,73 @@ qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
     return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
 }
 
+
+/*
+ * qcow2_co_encrypt()
+ *
+ * Encrypts one or more contiguous aligned sectors
+ *
+ * @host_offset - underlying storage offset of the first sector of the
+ * data to be encrypted
+ *
+ * @guest_offset - guest (virtual) offset of the first sector of the
+ * data to be encrypted
+ *
+ * @buf - buffer with the data to encrypt, that after encryption
+ *        will be written to the underlying storage device at
+ *        @host_offset
+ *
+ * @len - length of the buffer (must be a BDRV_SECTOR_SIZE multiple)
+ *
+ * Depending on the encryption method, @host_offset and/or @guest_offset
+ * may be used for generating the initialization vector for
+ * encryption.
+ *
+ * Note that while the whole range must be aligned on sectors, it
+ * does not have to be aligned on clusters and can also cross cluster
+ * boundaries
+ */
 int coroutine_fn
-qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
-                 uint64_t offset, void *buf, size_t len)
+qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
+                 uint64_t guest_offset, void *buf, size_t len)
 {
-    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
-                             qcrypto_block_encrypt);
+
+    BDRVQcow2State *s = bs->opaque;
+    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
+    assert(s->crypto);
+
+    if (!len) {
+        return 0;
+    }
+
+    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
+                           qcrypto_block_encrypt);
 }
 
+
+/*
+ * qcow2_co_decrypt()
+ *
+ * Decrypts one or more contiguous aligned sectors
+ * Similar to qcow2_co_encrypt
+ */
+
 int coroutine_fn
-qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
-                 uint64_t offset, void *buf, size_t len)
+qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
+                 uint64_t guest_offset, void *buf, size_t len)
 {
-    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
-                             qcrypto_block_decrypt);
+    BDRVQcow2State *s = bs->opaque;
+    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
+    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
+    assert(s->crypto);
+
+    if (!len) {
+        return 0;
+    }
+
+    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
+                           qcrypto_block_decrypt);
 }
diff --git a/block/qcow2.c b/block/qcow2.c
index 57734f20cf..ac768092bb 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2069,7 +2069,8 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
 
                 assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
                 assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
-                if (qcow2_co_decrypt(bs, cluster_offset, offset,
+                if (qcow2_co_decrypt(bs, cluster_offset + offset_in_cluster,
+                                     offset,
                                      cluster_data, cur_bytes) < 0) {
                     ret = -EIO;
                     goto fail;
@@ -2288,7 +2289,7 @@ static coroutine_fn int qcow2_co_pwritev_part(
             qemu_iovec_to_buf(qiov, qiov_offset + bytes_done,
                               cluster_data, cur_bytes);
 
-            if (qcow2_co_encrypt(bs, cluster_offset, offset,
+            if (qcow2_co_encrypt(bs, cluster_offset + offset_in_cluster, offset,
                                  cluster_data, cur_bytes) < 0) {
                 ret = -EIO;
                 goto out_unlocked;
diff --git a/block/qcow2.h b/block/qcow2.h
index 998bcdaef1..a488d761ff 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -758,10 +758,10 @@ ssize_t coroutine_fn
 qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size,
                     const void *src, size_t src_size);
 int coroutine_fn
-qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
-                 uint64_t offset, void *buf, size_t len);
+qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
+                 uint64_t guest_offset, void *buf, size_t len);
 int coroutine_fn
-qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
-                 uint64_t offset, void *buf, size_t len);
+qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
+                 uint64_t guest_offset, void *buf, size_t len);
 
 #endif
-- 
2.17.2



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Qemu-devel] [PATCH v6 3/3] qemu-iotests: Add test for bz #1745922
  2019-09-13 17:27 [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335 Maxim Levitsky
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 1/3] " Maxim Levitsky
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code Maxim Levitsky
@ 2019-09-13 17:27 ` Maxim Levitsky
  2019-09-14  0:28 ` [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335 no-reply
  3 siblings, 0 replies; 8+ messages in thread
From: Maxim Levitsky @ 2019-09-13 17:27 UTC (permalink / raw)
  To: qemu-devel
  Cc: Kevin Wolf, Vladimir Sementsov-Ogievskiy,
	Daniel P . Berrangé,
	qemu-block, qemu-stable, Max Reitz, Maxim Levitsky

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/263     | 91 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/263.out | 40 +++++++++++++++++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 132 insertions(+)
 create mode 100755 tests/qemu-iotests/263
 create mode 100644 tests/qemu-iotests/263.out

diff --git a/tests/qemu-iotests/263 b/tests/qemu-iotests/263
new file mode 100755
index 0000000000..d2c030fae9
--- /dev/null
+++ b/tests/qemu-iotests/263
@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+#
+# Test encrypted write that crosses cluster boundary of two unallocated clusters
+# Based on 188
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mlevitsk@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+	_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto generic
+_supported_os Linux
+
+
+size=1M
+
+SECRET="secret,id=sec0,data=astrochicken"
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT
+
+
+_run_test()
+{
+	echo "== reading the whole image =="
+	$QEMU_IO --object $SECRET -c "read -P 0 0 $size" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+
+	echo
+	echo "== write two 512 byte sectors on a cluster boundary =="
+	$QEMU_IO --object $SECRET -c "write -P 0xAA 0xFE00 0x400" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+
+	echo
+	echo "== verify that the rest of the image is not changed =="
+	$QEMU_IO --object $SECRET -c "read -P 0x00 0x00000 0xFE00" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+	$QEMU_IO --object $SECRET -c "read -P 0xAA 0x0FE00 0x400" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+	$QEMU_IO --object $SECRET -c "read -P 0x00 0x10200 0xEFE00" --image-opts "$1" | _filter_qemu_io | _filter_testdir
+
+}
+
+
+echo
+echo "testing LUKS qcow2 encryption"
+echo
+
+_make_test_img --object $SECRET -o "encrypt.format=luks,encrypt.key-secret=sec0,encrypt.iter-time=10,cluster_size=64K" $size
+_run_test "driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
+_cleanup_test_img
+
+echo
+echo "testing legacy AES qcow2 encryption"
+echo
+
+
+_make_test_img --object $SECRET -o "encrypt.format=aes,encrypt.key-secret=sec0,cluster_size=64K" $size
+_run_test "driver=$IMGFMT,encrypt.key-secret=sec0,file.filename=$TEST_IMG"
+_cleanup_test_img
+
+
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/263.out b/tests/qemu-iotests/263.out
new file mode 100644
index 0000000000..0c982c55cb
--- /dev/null
+++ b/tests/qemu-iotests/263.out
@@ -0,0 +1,40 @@
+QA output created by 263
+
+testing LUKS qcow2 encryption
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10
+== reading the whole image ==
+read 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== write two 512 byte sectors on a cluster boundary ==
+wrote 1024/1024 bytes at offset 65024
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== verify that the rest of the image is not changed ==
+read 65024/65024 bytes at offset 0
+63.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 1024/1024 bytes at offset 65024
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 982528/982528 bytes at offset 66048
+959.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+testing legacy AES qcow2 encryption
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=aes encrypt.key-secret=sec0
+== reading the whole image ==
+read 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== write two 512 byte sectors on a cluster boundary ==
+wrote 1024/1024 bytes at offset 65024
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== verify that the rest of the image is not changed ==
+read 65024/65024 bytes at offset 0
+63.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 1024/1024 bytes at offset 65024
+1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 982528/982528 bytes at offset 66048
+959.500 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 6082c74806..5d3da937e4 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -274,5 +274,6 @@
 257 rw
 258 rw quick
 262 rw quick migration
+263 rw quick
 265 rw auto quick
 266 rw quick
-- 
2.17.2



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code Maxim Levitsky
@ 2019-09-13 17:51   ` Vladimir Sementsov-Ogievskiy
  2019-09-13 18:19     ` Maxim Levitsky
  2019-09-13 18:37   ` Vladimir Sementsov-Ogievskiy
  1 sibling, 1 reply; 8+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-09-13 17:51 UTC (permalink / raw)
  To: Maxim Levitsky, qemu-devel
  Cc: Kevin Wolf, Daniel P . Berrangé, qemu-block, qemu-stable, Max Reitz

13.09.2019 20:27, Maxim Levitsky wrote:
> Change the qcow2_co_encrypt to just receive full host and
> guest offsets and in pariticular remove the
> offset_in_cluster parameter of do_perform_cow_encrypt,
> since it is misleading, because that offset can be larger than
> cluster size currently.
> 
> Remove the do_perform_cow_encrypt by merging it with
> qcow2_co_encrypt
> 
> Also document the qcow2_co_encrypt arguments to prevent
> that bug from happening again
> 
> Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>   block/qcow2-cluster.c | 35 +++++--------------
>   block/qcow2-threads.c | 81 ++++++++++++++++++++++++++++++++++++-------
>   block/qcow2.c         |  5 +--
>   block/qcow2.h         |  8 ++---
>   4 files changed, 83 insertions(+), 46 deletions(-)
> 
> diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
> index bfeb0241d7..f42b8a404c 100644
> --- a/block/qcow2-cluster.c
> +++ b/block/qcow2-cluster.c
> @@ -462,28 +462,6 @@ static int coroutine_fn do_perform_cow_read(BlockDriverState *bs,
>       return 0;
>   }
>   
> -static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
> -                                                uint64_t src_cluster_offset,
> -                                                uint64_t cluster_offset,
> -                                                unsigned offset_in_cluster,
> -                                                uint8_t *buffer,
> -                                                unsigned bytes)
> -{
> -    if (bytes && bs->encrypted) {
> -        BDRVQcow2State *s = bs->opaque;
> -        assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
> -        assert((bytes & ~BDRV_SECTOR_MASK) == 0);
> -        assert(s->crypto);
> -        if (qcow2_co_encrypt(bs,
> -                start_of_cluster(s, cluster_offset + offset_in_cluster),
> -                src_cluster_offset + offset_in_cluster,
> -                buffer, bytes) < 0) {
> -            return false;
> -        }
> -    }
> -    return true;
> -}
> -
>   static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
>                                                uint64_t cluster_offset,
>                                                unsigned offset_in_cluster,
> @@ -891,11 +869,14 @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta *m)
>   
>       /* Encrypt the data if necessary before writing it */
>       if (bs->encrypted) {
> -        if (!do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
> -                                    start->offset, start_buffer,
> -                                    start->nb_bytes) ||
> -            !do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
> -                                    end->offset, end_buffer, end->nb_bytes)) {
> +        if (!qcow2_co_encrypt(bs,
> +                              m->offset + start->offset,
> +                              m->alloc_offset + start->offset,
> +                              start_buffer, start->nb_bytes) ||

You got caught)
second argument qcow2_co_encrypt is host_offset and third is guest_offset.

I noticed this difference when reviewing previous version, and it's actually
additional reason to drop the function. Unfortunately, I didn't said.

Also, we can use return value of qcow2_co_encrypt instead of dropping it,

something like classical

ret = qcow2_co_encrypt(..)
if (ret < 0) {
    goto fail;
}

ret = qcow2_co_encrypt(..)
if (ret < 0) {
    goto fail;
}

Also, IMHO, too much changes too keep reviewed-by mark. Still, maybe it's my
"with it least fixed" is misleading, may be interpreted like removing the
function and corresponding refactoring is included..


> +            !qcow2_co_encrypt(bs,
> +                              m->offset + end->offset,
> +                              m->alloc_offset + end->offset,
> +                              end_buffer, end->nb_bytes)) {
>               ret = -EIO;
>               goto fail;
>           }
> diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
> index 3b1e63fe41..b31d45fb2b 100644
> --- a/block/qcow2-threads.c
> +++ b/block/qcow2-threads.c
> @@ -234,15 +234,15 @@ static int qcow2_encdec_pool_func(void *opaque)
>   }
>   
>   static int coroutine_fn
> -qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
> -                  uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc func)
> +qcow2_co_encdec(BlockDriverState *bs, uint64_t host_offset,
> +                uint64_t guest_offset, void *buf, size_t len,
> +                Qcow2EncDecFunc func)
>   {
>       BDRVQcow2State *s = bs->opaque;
> +
>       Qcow2EncDecData arg = {
>           .block = s->crypto,
> -        .offset = s->crypt_physical_offset ?
> -                      file_cluster_offset + offset_into_cluster(s, offset) :
> -                      offset,
> +        .offset = s->crypt_physical_offset ? host_offset : guest_offset,
>           .buf = buf,
>           .len = len,
>           .func = func,
> @@ -251,18 +251,73 @@ qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
>       return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
>   }
>   
> +
> +/*
> + * qcow2_co_encrypt()
> + *
> + * Encrypts one or more contiguous aligned sectors
> + *
> + * @host_offset - underlying storage offset of the first sector of the
> + * data to be encrypted
> + *
> + * @guest_offset - guest (virtual) offset of the first sector of the
> + * data to be encrypted
> + *
> + * @buf - buffer with the data to encrypt, that after encryption
> + *        will be written to the underlying storage device at
> + *        @host_offset
> + *
> + * @len - length of the buffer (must be a BDRV_SECTOR_SIZE multiple)
> + *
> + * Depending on the encryption method, @host_offset and/or @guest_offset
> + * may be used for generating the initialization vector for
> + * encryption.
> + *
> + * Note that while the whole range must be aligned on sectors, it
> + * does not have to be aligned on clusters and can also cross cluster
> + * boundaries
> + */
>   int coroutine_fn
> -qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> -                 uint64_t offset, void *buf, size_t len)
> +qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
> +                 uint64_t guest_offset, void *buf, size_t len)
>   {
> -    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
> -                             qcrypto_block_encrypt);
> +
> +    BDRVQcow2State *s = bs->opaque;
> +    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
> +    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
> +    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
> +    assert(s->crypto);

Hmm, better to move these asertions to _encdec I think, and don't duplicate
them in two functions.

> +
> +    if (!len) {
> +        return 0;
> +    }

Aha, that's why you don't check this in perform_cow. That's OK,
but still, better keep it in _encdec and don't duplicate.

> +
> +    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
> +                           qcrypto_block_encrypt);
>   }
>   
> +
> +/*
> + * qcow2_co_decrypt()
> + *
> + * Decrypts one or more contiguous aligned sectors
> + * Similar to qcow2_co_encrypt
> + */
> +
>   int coroutine_fn
> -qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> -                 uint64_t offset, void *buf, size_t len)
> +qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
> +                 uint64_t guest_offset, void *buf, size_t len)
>   {
> -    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
> -                             qcrypto_block_decrypt);
> +    BDRVQcow2State *s = bs->opaque;
> +    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
> +    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
> +    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
> +    assert(s->crypto);
> +
> +    if (!len) {
> +        return 0;
> +    }
> +
> +    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
> +                           qcrypto_block_decrypt);
>   }
> diff --git a/block/qcow2.c b/block/qcow2.c
> index 57734f20cf..ac768092bb 100644
> --- a/block/qcow2.c
> +++ b/block/qcow2.c
> @@ -2069,7 +2069,8 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
>   
>                   assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
>                   assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
> -                if (qcow2_co_decrypt(bs, cluster_offset, offset,
> +                if (qcow2_co_decrypt(bs, cluster_offset + offset_in_cluster,
> +                                     offset,
>                                        cluster_data, cur_bytes) < 0) {
>                       ret = -EIO;
>                       goto fail;
> @@ -2288,7 +2289,7 @@ static coroutine_fn int qcow2_co_pwritev_part(
>               qemu_iovec_to_buf(qiov, qiov_offset + bytes_done,
>                                 cluster_data, cur_bytes);
>   
> -            if (qcow2_co_encrypt(bs, cluster_offset, offset,
> +            if (qcow2_co_encrypt(bs, cluster_offset + offset_in_cluster, offset,
>                                    cluster_data, cur_bytes) < 0) {
>                   ret = -EIO;
>                   goto out_unlocked;
> diff --git a/block/qcow2.h b/block/qcow2.h
> index 998bcdaef1..a488d761ff 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -758,10 +758,10 @@ ssize_t coroutine_fn
>   qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size,
>                       const void *src, size_t src_size);
>   int coroutine_fn
> -qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> -                 uint64_t offset, void *buf, size_t len);
> +qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
> +                 uint64_t guest_offset, void *buf, size_t len);
>   int coroutine_fn
> -qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> -                 uint64_t offset, void *buf, size_t len);
> +qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
> +                 uint64_t guest_offset, void *buf, size_t len);
>   
>   #endif
> 


-- 
Best regards,
Vladimir

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code
  2019-09-13 17:51   ` Vladimir Sementsov-Ogievskiy
@ 2019-09-13 18:19     ` Maxim Levitsky
  0 siblings, 0 replies; 8+ messages in thread
From: Maxim Levitsky @ 2019-09-13 18:19 UTC (permalink / raw)
  To: Vladimir Sementsov-Ogievskiy, qemu-devel
  Cc: Kevin Wolf, Daniel P . Berrangé, qemu-stable, qemu-block, Max Reitz

On Fri, 2019-09-13 at 17:51 +0000, Vladimir Sementsov-Ogievskiy wrote:
> 13.09.2019 20:27, Maxim Levitsky wrote:
> > Change the qcow2_co_encrypt to just receive full host and
> > guest offsets and in pariticular remove the
> > offset_in_cluster parameter of do_perform_cow_encrypt,
> > since it is misleading, because that offset can be larger than
> > cluster size currently.
> > 
> > Remove the do_perform_cow_encrypt by merging it with
> > qcow2_co_encrypt
> > 
> > Also document the qcow2_co_encrypt arguments to prevent
> > that bug from happening again
> > 
> > Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
> > Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> > ---
> >   block/qcow2-cluster.c | 35 +++++--------------
> >   block/qcow2-threads.c | 81 ++++++++++++++++++++++++++++++++++++-------
> >   block/qcow2.c         |  5 +--
> >   block/qcow2.h         |  8 ++---
> >   4 files changed, 83 insertions(+), 46 deletions(-)
> > 
> > diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
> > index bfeb0241d7..f42b8a404c 100644
> > --- a/block/qcow2-cluster.c
> > +++ b/block/qcow2-cluster.c
> > @@ -462,28 +462,6 @@ static int coroutine_fn do_perform_cow_read(BlockDriverState *bs,
> >       return 0;
> >   }
> >   
> > -static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
> > -                                                uint64_t src_cluster_offset,
> > -                                                uint64_t cluster_offset,
> > -                                                unsigned offset_in_cluster,
> > -                                                uint8_t *buffer,
> > -                                                unsigned bytes)
> > -{
> > -    if (bytes && bs->encrypted) {
> > -        BDRVQcow2State *s = bs->opaque;
> > -        assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
> > -        assert((bytes & ~BDRV_SECTOR_MASK) == 0);
> > -        assert(s->crypto);
> > -        if (qcow2_co_encrypt(bs,
> > -                start_of_cluster(s, cluster_offset + offset_in_cluster),
> > -                src_cluster_offset + offset_in_cluster,
> > -                buffer, bytes) < 0) {
> > -            return false;
> > -        }
> > -    }
> > -    return true;
> > -}
> > -
> >   static int coroutine_fn do_perform_cow_write(BlockDriverState *bs,
> >                                                uint64_t cluster_offset,
> >                                                unsigned offset_in_cluster,
> > @@ -891,11 +869,14 @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta *m)
> >   
> >       /* Encrypt the data if necessary before writing it */
> >       if (bs->encrypted) {
> > -        if (!do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
> > -                                    start->offset, start_buffer,
> > -                                    start->nb_bytes) ||
> > -            !do_perform_cow_encrypt(bs, m->offset, m->alloc_offset,
> > -                                    end->offset, end_buffer, end->nb_bytes)) {
> > +        if (!qcow2_co_encrypt(bs,
> > +                              m->offset + start->offset,
> > +                              m->alloc_offset + start->offset,
> > +                              start_buffer, start->nb_bytes) ||
> 
> You got caught)

Ha!, I am just too tired.

> second argument qcow2_co_encrypt is host_offset and third is guest_offset.
> 
> I noticed this difference when reviewing previous version, and it's actually
> additional reason to drop the function. Unfortunately, I didn't said.
> 
> Also, we can use return value of qcow2_co_encrypt instead of dropping it,
> 
> something like classical
> 
> ret = qcow2_co_encrypt(..)
> if (ret < 0) {
>     goto fail;
> }
> 
> ret = qcow2_co_encrypt(..)
> if (ret < 0) {
>     goto fail;
> }
> 
> Also, IMHO, too much changes too keep reviewed-by mark. Still, maybe it's my
> "with it least fixed" is misleading, may be interpreted like removing the
> function and corresponding refactoring is included..
Yea. So V7 there will be...
Re-factoring is hard (but worth it) :-(

> 
> 
> > +            !qcow2_co_encrypt(bs,
> > +                              m->offset + end->offset,
> > +                              m->alloc_offset + end->offset,
> > +                              end_buffer, end->nb_bytes)) {
> >               ret = -EIO;
> >               goto fail;
> >           }
> > diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
> > index 3b1e63fe41..b31d45fb2b 100644
> > --- a/block/qcow2-threads.c
> > +++ b/block/qcow2-threads.c
> > @@ -234,15 +234,15 @@ static int qcow2_encdec_pool_func(void *opaque)
> >   }
> >   
> >   static int coroutine_fn
> > -qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
> > -                  uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc func)
> > +qcow2_co_encdec(BlockDriverState *bs, uint64_t host_offset,
> > +                uint64_t guest_offset, void *buf, size_t len,
> > +                Qcow2EncDecFunc func)
> >   {
> >       BDRVQcow2State *s = bs->opaque;
> > +
> >       Qcow2EncDecData arg = {
> >           .block = s->crypto,
> > -        .offset = s->crypt_physical_offset ?
> > -                      file_cluster_offset + offset_into_cluster(s, offset) :
> > -                      offset,
> > +        .offset = s->crypt_physical_offset ? host_offset : guest_offset,
> >           .buf = buf,
> >           .len = len,
> >           .func = func,
> > @@ -251,18 +251,73 @@ qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
> >       return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
> >   }
> >   
> > +
> > +/*
> > + * qcow2_co_encrypt()
> > + *
> > + * Encrypts one or more contiguous aligned sectors
> > + *
> > + * @host_offset - underlying storage offset of the first sector of the
> > + * data to be encrypted
> > + *
> > + * @guest_offset - guest (virtual) offset of the first sector of the
> > + * data to be encrypted
> > + *
> > + * @buf - buffer with the data to encrypt, that after encryption
> > + *        will be written to the underlying storage device at
> > + *        @host_offset
> > + *
> > + * @len - length of the buffer (must be a BDRV_SECTOR_SIZE multiple)
> > + *
> > + * Depending on the encryption method, @host_offset and/or @guest_offset
> > + * may be used for generating the initialization vector for
> > + * encryption.
> > + *
> > + * Note that while the whole range must be aligned on sectors, it
> > + * does not have to be aligned on clusters and can also cross cluster
> > + * boundaries
> > + */
> >   int coroutine_fn
> > -qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> > -                 uint64_t offset, void *buf, size_t len)
> > +qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
> > +                 uint64_t guest_offset, void *buf, size_t len)
> >   {
> > -    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
> > -                             qcrypto_block_encrypt);
> > +
> > +    BDRVQcow2State *s = bs->opaque;
> > +    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
> > +    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
> > +    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
> > +    assert(s->crypto);
> 
> Hmm, better to move these asertions to _encdec I think, and don't duplicate
> them in two functions.

OK.

> 
> > +
> > +    if (!len) {
> > +        return 0;
> > +    }
> 
> Aha, that's why you don't check this in perform_cow. That's OK,
> but still, better keep it in _encdec and don't duplicate.


OK.

> 
> > +
> > +    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
> > +                           qcrypto_block_encrypt);
> >   }
> >   
> > +
> > +/*
> > + * qcow2_co_decrypt()
> > + *
> > + * Decrypts one or more contiguous aligned sectors
> > + * Similar to qcow2_co_encrypt
> > + */
> > +
> >   int coroutine_fn
> > -qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> > -                 uint64_t offset, void *buf, size_t len)
> > +qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
> > +                 uint64_t guest_offset, void *buf, size_t len)
> >   {
> > -    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
> > -                             qcrypto_block_decrypt);
> > +    BDRVQcow2State *s = bs->opaque;
> > +    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
> > +    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
> > +    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
> > +    assert(s->crypto);
> > +
> > +    if (!len) {
> > +        return 0;
> > +    }
> > +
> > +    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
> > +                           qcrypto_block_decrypt);
> >   }
> > diff --git a/block/qcow2.c b/block/qcow2.c
> > index 57734f20cf..ac768092bb 100644
> > --- a/block/qcow2.c
> > +++ b/block/qcow2.c
> > @@ -2069,7 +2069,8 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,
> >   
> >                   assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0);
> >                   assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
> > -                if (qcow2_co_decrypt(bs, cluster_offset, offset,
> > +                if (qcow2_co_decrypt(bs, cluster_offset + offset_in_cluster,
> > +                                     offset,
> >                                        cluster_data, cur_bytes) < 0) {
> >                       ret = -EIO;
> >                       goto fail;
> > @@ -2288,7 +2289,7 @@ static coroutine_fn int qcow2_co_pwritev_part(
> >               qemu_iovec_to_buf(qiov, qiov_offset + bytes_done,
> >                                 cluster_data, cur_bytes);
> >   
> > -            if (qcow2_co_encrypt(bs, cluster_offset, offset,
> > +            if (qcow2_co_encrypt(bs, cluster_offset + offset_in_cluster, offset,
> >                                    cluster_data, cur_bytes) < 0) {
> >                   ret = -EIO;
> >                   goto out_unlocked;
> > diff --git a/block/qcow2.h b/block/qcow2.h
> > index 998bcdaef1..a488d761ff 100644
> > --- a/block/qcow2.h
> > +++ b/block/qcow2.h
> > @@ -758,10 +758,10 @@ ssize_t coroutine_fn
> >   qcow2_co_decompress(BlockDriverState *bs, void *dest, size_t dest_size,
> >                       const void *src, size_t src_size);
> >   int coroutine_fn
> > -qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> > -                 uint64_t offset, void *buf, size_t len);
> > +qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
> > +                 uint64_t guest_offset, void *buf, size_t len);
> >   int coroutine_fn
> > -qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> > -                 uint64_t offset, void *buf, size_t len);
> > +qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
> > +                 uint64_t guest_offset, void *buf, size_t len);
> >   
> >   #endif
> > 


Best regards,
	Maxim Levitsky



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code Maxim Levitsky
  2019-09-13 17:51   ` Vladimir Sementsov-Ogievskiy
@ 2019-09-13 18:37   ` Vladimir Sementsov-Ogievskiy
  1 sibling, 0 replies; 8+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2019-09-13 18:37 UTC (permalink / raw)
  To: Maxim Levitsky, qemu-devel
  Cc: Kevin Wolf, Daniel P . Berrangé, qemu-block, qemu-stable, Max Reitz

and a bit about empty lines

13.09.2019 20:27, Maxim Levitsky wrote:
> Change the qcow2_co_encrypt to just receive full host and
> guest offsets and in pariticular remove the
> offset_in_cluster parameter of do_perform_cow_encrypt,
> since it is misleading, because that offset can be larger than
> cluster size currently.
> 
> Remove the do_perform_cow_encrypt by merging it with
> qcow2_co_encrypt
> 
> Also document the qcow2_co_encrypt arguments to prevent
> that bug from happening again
> 
> Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---

[..]

> diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
> index 3b1e63fe41..b31d45fb2b 100644
> --- a/block/qcow2-threads.c
> +++ b/block/qcow2-threads.c
> @@ -234,15 +234,15 @@ static int qcow2_encdec_pool_func(void *opaque)
>   }
>   
>   static int coroutine_fn
> -qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
> -                  uint64_t offset, void *buf, size_t len, Qcow2EncDecFunc func)
> +qcow2_co_encdec(BlockDriverState *bs, uint64_t host_offset,
> +                uint64_t guest_offset, void *buf, size_t len,
> +                Qcow2EncDecFunc func)
>   {
>       BDRVQcow2State *s = bs->opaque;
> +

Don't see real benefit in separating one variable initialization from another one here,
but I don't mind, it's OK of course.

>       Qcow2EncDecData arg = {
>           .block = s->crypto,
> -        .offset = s->crypt_physical_offset ?
> -                      file_cluster_offset + offset_into_cluster(s, offset) :
> -                      offset,
> +        .offset = s->crypt_physical_offset ? host_offset : guest_offset,
>           .buf = buf,
>           .len = len,
>           .func = func,
> @@ -251,18 +251,73 @@ qcow2_co_encdec(BlockDriverState *bs, uint64_t file_cluster_offset,
>       return qcow2_co_process(bs, qcow2_encdec_pool_func, &arg);
>   }
>   
> +

Hmm, in this file doubled empty lines
are used to separate "Compression" and "Cryptography"
sections. And all other splits are one-empty-line.

> +/*
> + * qcow2_co_encrypt()
> + *
> + * Encrypts one or more contiguous aligned sectors
> + *
> + * @host_offset - underlying storage offset of the first sector of the
> + * data to be encrypted
> + *
> + * @guest_offset - guest (virtual) offset of the first sector of the
> + * data to be encrypted
> + *
> + * @buf - buffer with the data to encrypt, that after encryption
> + *        will be written to the underlying storage device at
> + *        @host_offset
> + *
> + * @len - length of the buffer (must be a BDRV_SECTOR_SIZE multiple)
> + *
> + * Depending on the encryption method, @host_offset and/or @guest_offset
> + * may be used for generating the initialization vector for
> + * encryption.
> + *
> + * Note that while the whole range must be aligned on sectors, it
> + * does not have to be aligned on clusters and can also cross cluster
> + * boundaries
> + */
>   int coroutine_fn
> -qcow2_co_encrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> -                 uint64_t offset, void *buf, size_t len)
> +qcow2_co_encrypt(BlockDriverState *bs, uint64_t host_offset,
> +                 uint64_t guest_offset, void *buf, size_t len)
>   {
> -    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
> -                             qcrypto_block_encrypt);
> +

This empty line is not needed for sure

> +    BDRVQcow2State *s = bs->opaque;

But it's common practice to split variables definition from code by empty line here

> +    assert(QEMU_IS_ALIGNED(guest_offset, BDRV_SECTOR_SIZE));
> +    assert(QEMU_IS_ALIGNED(host_offset, BDRV_SECTOR_SIZE));
> +    assert(QEMU_IS_ALIGNED(len, BDRV_SECTOR_SIZE));
> +    assert(s->crypto);
> +
> +    if (!len) {
> +        return 0;
> +    }
> +
> +    return qcow2_co_encdec(bs, host_offset, guest_offset, buf, len,
> +                           qcrypto_block_encrypt);
>   }
>   
> +

IMHO extra one (see above)

> +/*
> + * qcow2_co_decrypt()
> + *
> + * Decrypts one or more contiguous aligned sectors
> + * Similar to qcow2_co_encrypt
> + */
> +

Here you replaced one extra empty line by another

>   int coroutine_fn
> -qcow2_co_decrypt(BlockDriverState *bs, uint64_t file_cluster_offset,
> -                 uint64_t offset, void *buf, size_t len)
> +qcow2_co_decrypt(BlockDriverState *bs, uint64_t host_offset,
> +                 uint64_t guest_offset, void *buf, size_t len)
>   {
> -    return qcow2_co_encdec(bs, file_cluster_offset, offset, buf, len,
> -                             qcrypto_block_decrypt);



-- 
Best regards,
Vladimir

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335
  2019-09-13 17:27 [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335 Maxim Levitsky
                   ` (2 preceding siblings ...)
  2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 3/3] qemu-iotests: Add test for bz #1745922 Maxim Levitsky
@ 2019-09-14  0:28 ` no-reply
  3 siblings, 0 replies; 8+ messages in thread
From: no-reply @ 2019-09-14  0:28 UTC (permalink / raw)
  To: mlevitsk
  Cc: kwolf, mlevitsk, vsementsov, berrange, qemu-block, qemu-devel,
	qemu-stable, mreitz

Patchew URL: https://patchew.org/QEMU/20190913172741.5662-1-mlevitsk@redhat.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

libudev           no
default devices   yes

warning: Python 2 support is deprecated
warning: Python 3 will be required for building future versions of QEMU
cross containers  no

NOTE: guest cross-compilers enabled: cc


The full log is available at
http://patchew.org/logs/20190913172741.5662-1-mlevitsk@redhat.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2019-09-14  0:29 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-13 17:27 [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335 Maxim Levitsky
2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 1/3] " Maxim Levitsky
2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 2/3] block/qcow2: refactor threaded encryption code Maxim Levitsky
2019-09-13 17:51   ` Vladimir Sementsov-Ogievskiy
2019-09-13 18:19     ` Maxim Levitsky
2019-09-13 18:37   ` Vladimir Sementsov-Ogievskiy
2019-09-13 17:27 ` [Qemu-devel] [PATCH v6 3/3] qemu-iotests: Add test for bz #1745922 Maxim Levitsky
2019-09-14  0:28 ` [Qemu-devel] [PATCH v6 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335 no-reply

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).