* [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1
@ 2019-11-07 14:33 Max Reitz
2019-11-07 14:33 ` [PULL 1/3] qcow2-bitmap: Fix uint64_t left-shift overflow Max Reitz
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Max Reitz @ 2019-11-07 14:33 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-stable, qemu-devel, Max Reitz
The following changes since commit d0f90e1423b4f412adc620eee93e8bfef8af4117:
Merge remote-tracking branch 'remotes/kraxel/tags/audio-20191106-pull-request' into staging (2019-11-07 09:21:52 +0000)
are available in the Git repository at:
https://github.com/XanClic/qemu.git tags/pull-block-2019-11-07
for you to fetch changes up to b7cd2c11f76d27930f53d3cf26d7b695c78d613b:
iotests: Add test for 4G+ compressed qcow2 write (2019-11-07 14:37:46 +0100)
----------------------------------------------------------------
Block patches for 4.2.0-rc0/4.1.1:
- Fix writing to compressed qcow2 images > 4 GB
- Fix size sanity check for qcow2 bitmaps
----------------------------------------------------------------
Max Reitz (2):
qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK
iotests: Add test for 4G+ compressed qcow2 write
Tuguoyi (1):
qcow2-bitmap: Fix uint64_t left-shift overflow
block/qcow2-bitmap.c | 14 +++++--
block/qcow2.h | 2 +-
tests/qemu-iotests/272 | 79 ++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/272.out | 10 +++++
tests/qemu-iotests/group | 1 +
5 files changed, 102 insertions(+), 4 deletions(-)
create mode 100755 tests/qemu-iotests/272
create mode 100644 tests/qemu-iotests/272.out
--
2.23.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PULL 1/3] qcow2-bitmap: Fix uint64_t left-shift overflow
2019-11-07 14:33 [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1 Max Reitz
@ 2019-11-07 14:33 ` Max Reitz
2019-11-07 14:33 ` [PULL 2/3] qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK Max Reitz
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Max Reitz @ 2019-11-07 14:33 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-stable, qemu-devel, Max Reitz
From: Tuguoyi <tu.guoyi@h3c.com>
There are two issues in In check_constraints_on_bitmap(),
1) The sanity check on the granularity will cause uint64_t
integer left-shift overflow when cluster_size is 2M and the
granularity is BIGGER than 32K.
2) The way to calculate image size that the maximum bitmap
supported can map to is a bit incorrect.
This patch fix it by add a helper function to calculate the
number of bytes needed by a normal bitmap in image and compare
it to the maximum bitmap bytes supported by qemu.
Fixes: 5f72826e7fc62167cf3a
Signed-off-by: Guoyi Tu <tu.guoyi@h3c.com>
Message-id: 4ba40cd1e7ee4a708b40899952e49f22@h3c.com
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/qcow2-bitmap.c | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 98294a7696..ef9ef628a0 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -142,6 +142,13 @@ static int check_table_entry(uint64_t entry, int cluster_size)
return 0;
}
+static int64_t get_bitmap_bytes_needed(int64_t len, uint32_t granularity)
+{
+ int64_t num_bits = DIV_ROUND_UP(len, granularity);
+
+ return DIV_ROUND_UP(num_bits, 8);
+}
+
static int check_constraints_on_bitmap(BlockDriverState *bs,
const char *name,
uint32_t granularity,
@@ -150,6 +157,7 @@ static int check_constraints_on_bitmap(BlockDriverState *bs,
BDRVQcow2State *s = bs->opaque;
int granularity_bits = ctz32(granularity);
int64_t len = bdrv_getlength(bs);
+ int64_t bitmap_bytes;
assert(granularity > 0);
assert((granularity & (granularity - 1)) == 0);
@@ -171,9 +179,9 @@ static int check_constraints_on_bitmap(BlockDriverState *bs,
return -EINVAL;
}
- if ((len > (uint64_t)BME_MAX_PHYS_SIZE << granularity_bits) ||
- (len > (uint64_t)BME_MAX_TABLE_SIZE * s->cluster_size <<
- granularity_bits))
+ bitmap_bytes = get_bitmap_bytes_needed(len, granularity);
+ if ((bitmap_bytes > (uint64_t)BME_MAX_PHYS_SIZE) ||
+ (bitmap_bytes > (uint64_t)BME_MAX_TABLE_SIZE * s->cluster_size))
{
error_setg(errp, "Too much space will be occupied by the bitmap. "
"Use larger granularity");
--
2.23.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PULL 2/3] qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK
2019-11-07 14:33 [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1 Max Reitz
2019-11-07 14:33 ` [PULL 1/3] qcow2-bitmap: Fix uint64_t left-shift overflow Max Reitz
@ 2019-11-07 14:33 ` Max Reitz
2019-11-07 14:33 ` [PULL 3/3] iotests: Add test for 4G+ compressed qcow2 write Max Reitz
2019-11-07 18:16 ` [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1 Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Max Reitz @ 2019-11-07 14:33 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-stable, qemu-devel, Max Reitz
Masks for L2 table entries should have 64 bit.
Fixes: b6c246942b14d3e0dec46a6c5868ed84e7dbea19
Buglink: https://bugs.launchpad.net/qemu/+bug/1850000
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191028161841.1198-2-mreitz@redhat.com
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/qcow2.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index 601c2e4c82..0942126232 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -83,7 +83,7 @@
/* Defined in the qcow2 spec (compressed cluster descriptor) */
#define QCOW2_COMPRESSED_SECTOR_SIZE 512U
-#define QCOW2_COMPRESSED_SECTOR_MASK (~(QCOW2_COMPRESSED_SECTOR_SIZE - 1))
+#define QCOW2_COMPRESSED_SECTOR_MASK (~(QCOW2_COMPRESSED_SECTOR_SIZE - 1ULL))
/* Must be at least 2 to cover COW */
#define MIN_L2_CACHE_SIZE 2 /* cache entries */
--
2.23.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PULL 3/3] iotests: Add test for 4G+ compressed qcow2 write
2019-11-07 14:33 [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1 Max Reitz
2019-11-07 14:33 ` [PULL 1/3] qcow2-bitmap: Fix uint64_t left-shift overflow Max Reitz
2019-11-07 14:33 ` [PULL 2/3] qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK Max Reitz
@ 2019-11-07 14:33 ` Max Reitz
2019-11-07 18:16 ` [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1 Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Max Reitz @ 2019-11-07 14:33 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, Peter Maydell, qemu-stable, qemu-devel, Max Reitz
Test what qemu-img check says about an image after one has written
compressed data to an offset above 4 GB.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191028161841.1198-3-mreitz@redhat.com
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/272 | 79 ++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/272.out | 10 +++++
tests/qemu-iotests/group | 1 +
3 files changed, 90 insertions(+)
create mode 100755 tests/qemu-iotests/272
create mode 100644 tests/qemu-iotests/272.out
diff --git a/tests/qemu-iotests/272 b/tests/qemu-iotests/272
new file mode 100755
index 0000000000..c2f782d47b
--- /dev/null
+++ b/tests/qemu-iotests/272
@@ -0,0 +1,79 @@
+#!/usr/bin/env bash
+#
+# Test compressed write to a qcow2 image at an offset above 4 GB
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+seq=$(basename "$0")
+echo "QA output created by $seq"
+
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# This is a qcow2 regression test
+_supported_fmt qcow2
+_supported_proto file
+
+# External data files do not support compression;
+# We need an exact cluster size (2M) and refcount width (2) so we can
+# get this test quickly over with; and this in turn require
+# compat=1.1
+_unsupported_imgopts data_file cluster_size refcount_bits 'compat=0.10'
+
+# The idea is: Create an empty file, mark the first 4 GB as used, then
+# do a compressed write that thus must be put beyond 4 GB.
+# (This used to fail because the compressed sector mask was just a
+# 32 bit mask, so qemu-img check will count a cluster before 4 GB as
+# referenced twice.)
+
+# We would like to use refcount_bits=1 here, but then qemu-img check
+# will throw an error when trying to count a cluster as referenced
+# twice.
+_make_test_img -o cluster_size=2M,refcount_bits=2 64M
+
+reft_offs=$(peek_file_be "$TEST_IMG" 48 8)
+refb_offs=$(peek_file_be "$TEST_IMG" $reft_offs 8)
+
+# We want to cover 4 GB, those are 2048 clusters, equivalent to
+# 4096 bit = 512 B.
+truncate -s 4G "$TEST_IMG"
+for ((in_refb_offs = 0; in_refb_offs < 512; in_refb_offs += 8)); do
+ poke_file "$TEST_IMG" $((refb_offs + in_refb_offs)) \
+ '\x55\x55\x55\x55\x55\x55\x55\x55'
+done
+
+$QEMU_IO -c 'write -c -P 42 0 2M' "$TEST_IMG" | _filter_qemu_io
+
+echo
+echo '--- Check ---'
+
+# This should only print the leaked clusters in the first 4 GB
+_check_test_img | grep -v '^Leaked cluster '
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/272.out b/tests/qemu-iotests/272.out
new file mode 100644
index 0000000000..35698b0e73
--- /dev/null
+++ b/tests/qemu-iotests/272.out
@@ -0,0 +1,10 @@
+QA output created by 272
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+wrote 2097152/2097152 bytes at offset 0
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+--- Check ---
+
+2044 leaked clusters were found on the image.
+This means waste of disk space, but no harm to data.
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 095ed1b880..065040398d 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -283,3 +283,4 @@
267 rw auto quick snapshot
268 rw auto quick
270 rw backing quick
+272 rw
--
2.23.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1
2019-11-07 14:33 [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1 Max Reitz
` (2 preceding siblings ...)
2019-11-07 14:33 ` [PULL 3/3] iotests: Add test for 4G+ compressed qcow2 write Max Reitz
@ 2019-11-07 18:16 ` Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Peter Maydell @ 2019-11-07 18:16 UTC (permalink / raw)
To: Max Reitz; +Cc: Kevin Wolf, QEMU Developers, Qemu-block, qemu-stable
On Thu, 7 Nov 2019 at 14:34, Max Reitz <mreitz@redhat.com> wrote:
>
> The following changes since commit d0f90e1423b4f412adc620eee93e8bfef8af4117:
>
> Merge remote-tracking branch 'remotes/kraxel/tags/audio-20191106-pull-request' into staging (2019-11-07 09:21:52 +0000)
>
> are available in the Git repository at:
>
> https://github.com/XanClic/qemu.git tags/pull-block-2019-11-07
>
> for you to fetch changes up to b7cd2c11f76d27930f53d3cf26d7b695c78d613b:
>
> iotests: Add test for 4G+ compressed qcow2 write (2019-11-07 14:37:46 +0100)
>
> ----------------------------------------------------------------
> Block patches for 4.2.0-rc0/4.1.1:
> - Fix writing to compressed qcow2 images > 4 GB
> - Fix size sanity check for qcow2 bitmaps
>
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/4.2
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-11-07 18:19 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-07 14:33 [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1 Max Reitz
2019-11-07 14:33 ` [PULL 1/3] qcow2-bitmap: Fix uint64_t left-shift overflow Max Reitz
2019-11-07 14:33 ` [PULL 2/3] qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK Max Reitz
2019-11-07 14:33 ` [PULL 3/3] iotests: Add test for 4G+ compressed qcow2 write Max Reitz
2019-11-07 18:16 ` [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1 Peter Maydell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).