* [PATCH v3 0/5] qcow2: advanced compression options
@ 2019-10-15 18:20 Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 1/5] qcow2: Allow writing compressed data of multiple clusters Andrey Shinkevich
` (8 more replies)
0 siblings, 9 replies; 10+ messages in thread
From: Andrey Shinkevich @ 2019-10-15 18:20 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: kwolf, fam, vsementsov, armbru, mreitz, stefanha,
andrey.shinkevich, den, jsnow
New enhancements for writing compressed data to QCOW2 image.
The preceding patches have been queued in the Max's block branch:
Based-on: <20190916175324.18478-1-vsementsov@virtuozzo.com>
v2:
Instead of introducing multiple key options for many drivers, the
'compression' option has been introduced on generic block layer
as suggested by Roman Kagan. Discussed on the thread ID
<1570026166-748566-1-git-send-email-andrey.shinkevich@virtuozzo.com>
Andrey Shinkevich (5):
qcow2: Allow writing compressed data of multiple clusters
tests/qemu-iotests: add case to write compressed data of multiple
clusters
block: support compressed write for copy-on-read
block-stream: add compress option
tests/qemu-iotests: add case for block-stream compress
block.c | 12 ++++-
block/io.c | 23 +++++++---
block/qcow2.c | 106 +++++++++++++++++++++++++++++++++------------
block/qcow2.h | 1 +
block/stream.c | 10 ++++-
block/trace-events | 2 +-
blockdev.c | 16 ++++++-
include/block/block.h | 1 +
include/block/block_int.h | 2 +
qapi/block-core.json | 6 ++-
qemu-options.hx | 6 ++-
tests/qemu-iotests/030 | 51 +++++++++++++++++++++-
tests/qemu-iotests/030.out | 4 +-
tests/qemu-iotests/214 | 35 +++++++++++++++
tests/qemu-iotests/214.out | 15 +++++++
15 files changed, 246 insertions(+), 44 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v3 1/5] qcow2: Allow writing compressed data of multiple clusters
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
@ 2019-10-15 18:20 ` Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 2/5] tests/qemu-iotests: add case to write " Andrey Shinkevich
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrey Shinkevich @ 2019-10-15 18:20 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: kwolf, fam, vsementsov, armbru, mreitz, stefanha,
andrey.shinkevich, den, jsnow
QEMU currently supports writing compressed data of the size equal to
one cluster. This patch allows writing QCOW2 compressed data that
exceed one cluster. Now, we split buffered data into separate clusters
and write them compressed using the existing functionality.
To inform the block layer about writing all the data compressed, we
introduce the 'compress' command line option. Based on that option, the
written data will be aligned by the cluster size at the generic layer.
Suggested-by: Pavel Butsykin <pbutsykin@virtuozzo.com>
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
---
block.c | 12 +++++-
block/io.c | 2 +-
block/qcow2.c | 106 ++++++++++++++++++++++++++++++++++------------
block/qcow2.h | 1 +
blockdev.c | 4 ++
include/block/block.h | 1 +
include/block/block_int.h | 2 +
qapi/block-core.json | 6 ++-
qemu-options.hx | 6 ++-
9 files changed, 108 insertions(+), 32 deletions(-)
diff --git a/block.c b/block.c
index 5944124..4cfbea2 100644
--- a/block.c
+++ b/block.c
@@ -1418,6 +1418,11 @@ QemuOptsList bdrv_runtime_opts = {
.type = QEMU_OPT_BOOL,
.help = "always accept other writers (default: off)",
},
+ {
+ .name = BDRV_OPT_COMPRESS,
+ .type = QEMU_OPT_BOOL,
+ .help = "compress all writes to the image (default: off)",
+ },
{ /* end of list */ }
},
};
@@ -2983,6 +2988,11 @@ static BlockDriverState *bdrv_open_inherit(const char *filename,
flags &= ~BDRV_O_RDWR;
}
+ if (!g_strcmp0(qdict_get_try_str(options, BDRV_OPT_COMPRESS), "on") ||
+ qdict_get_try_bool(options, BDRV_OPT_COMPRESS, false)) {
+ bs->all_write_compressed = true;
+ }
+
if (flags & BDRV_O_SNAPSHOT) {
snapshot_options = qdict_new();
bdrv_temp_snapshot_options(&snapshot_flags, snapshot_options,
@@ -3208,7 +3218,7 @@ static int bdrv_reset_options_allowed(BlockDriverState *bs,
* in bdrv_reopen_prepare() so they can be left out of @new_opts */
const char *const common_options[] = {
"node-name", "discard", "cache.direct", "cache.no-flush",
- "read-only", "auto-read-only", "detect-zeroes", NULL
+ "read-only", "auto-read-only", "detect-zeroes", "compress", NULL
};
for (e = qdict_first(bs->options); e; e = qdict_next(bs->options, e)) {
diff --git a/block/io.c b/block/io.c
index f8c3596..6a5509c 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1922,7 +1922,7 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child,
} else if (flags & BDRV_REQ_ZERO_WRITE) {
bdrv_debug_event(bs, BLKDBG_PWRITEV_ZERO);
ret = bdrv_co_do_pwrite_zeroes(bs, offset, bytes, flags);
- } else if (flags & BDRV_REQ_WRITE_COMPRESSED) {
+ } else if (flags & BDRV_REQ_WRITE_COMPRESSED || bs->all_write_compressed) {
ret = bdrv_driver_pwritev_compressed(bs, offset, bytes,
qiov, qiov_offset);
} else if (bytes <= max_transfer) {
diff --git a/block/qcow2.c b/block/qcow2.c
index 7961c05..9a85d73 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1787,6 +1787,10 @@ static void qcow2_refresh_limits(BlockDriverState *bs, Error **errp)
/* Encryption works on a sector granularity */
bs->bl.request_alignment = qcrypto_block_get_sector_size(s->crypto);
}
+ if (bs->all_write_compressed) {
+ bs->bl.request_alignment = MAX(bs->bl.request_alignment,
+ s->cluster_size);
+ }
bs->bl.pwrite_zeroes_alignment = s->cluster_size;
bs->bl.pdiscard_alignment = s->cluster_size;
}
@@ -4152,10 +4156,8 @@ fail:
return ret;
}
-/* XXX: put compressed sectors first, then all the cluster aligned
- tables to avoid losing bytes in alignment */
static coroutine_fn int
-qcow2_co_pwritev_compressed_part(BlockDriverState *bs,
+qcow2_co_pwritev_compressed_task(BlockDriverState *bs,
uint64_t offset, uint64_t bytes,
QEMUIOVector *qiov, size_t qiov_offset)
{
@@ -4165,32 +4167,11 @@ qcow2_co_pwritev_compressed_part(BlockDriverState *bs,
uint8_t *buf, *out_buf;
uint64_t cluster_offset;
- if (has_data_file(bs)) {
- return -ENOTSUP;
- }
-
- if (bytes == 0) {
- /* align end of file to a sector boundary to ease reading with
- sector based I/Os */
- int64_t len = bdrv_getlength(bs->file->bs);
- if (len < 0) {
- return len;
- }
- return bdrv_co_truncate(bs->file, len, PREALLOC_MODE_OFF, NULL);
- }
-
- if (offset_into_cluster(s, offset)) {
- return -EINVAL;
- }
+ assert(bytes == s->cluster_size || (bytes < s->cluster_size &&
+ (offset + bytes == bs->total_sectors << BDRV_SECTOR_BITS)));
buf = qemu_blockalign(bs, s->cluster_size);
- if (bytes != s->cluster_size) {
- if (bytes > s->cluster_size ||
- offset + bytes != bs->total_sectors << BDRV_SECTOR_BITS)
- {
- qemu_vfree(buf);
- return -EINVAL;
- }
+ if (bytes < s->cluster_size) {
/* Zero-pad last write if image size is not cluster aligned */
memset(buf + bytes, 0, s->cluster_size - bytes);
}
@@ -4239,6 +4220,77 @@ fail:
return ret;
}
+static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *task)
+{
+ Qcow2AioTask *t = container_of(task, Qcow2AioTask, task);
+
+ assert(!t->cluster_type && !t->l2meta);
+
+ return qcow2_co_pwritev_compressed_task(t->bs, t->offset, t->bytes, t->qiov,
+ t->qiov_offset);
+}
+
+/*
+ * XXX: put compressed sectors first, then all the cluster aligned
+ tables to avoid losing bytes in alignment
+ */
+static coroutine_fn int
+qcow2_co_pwritev_compressed_part(BlockDriverState *bs,
+ uint64_t offset, uint64_t bytes,
+ QEMUIOVector *qiov, size_t qiov_offset)
+{
+ BDRVQcow2State *s = bs->opaque;
+ AioTaskPool *aio = NULL;
+ int ret;
+
+ if (has_data_file(bs)) {
+ return -ENOTSUP;
+ }
+
+ if (bytes == 0) {
+ /*
+ * align end of file to a sector boundary to ease reading with
+ * sector based I/Os
+ */
+ int64_t len = bdrv_getlength(bs->file->bs);
+ if (len < 0) {
+ return len;
+ }
+ return bdrv_co_truncate(bs->file, len, PREALLOC_MODE_OFF, NULL);
+ }
+
+ if (offset_into_cluster(s, offset)) {
+ return -EINVAL;
+ }
+
+ while (bytes && aio_task_pool_status(aio) == 0) {
+ uint32_t chunk_size = MIN(bytes, s->cluster_size);
+
+ if (!aio && chunk_size != bytes) {
+ aio = aio_task_pool_new(QCOW2_MAX_WORKERS);
+ }
+
+ ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_compressed_task_entry,
+ 0, 0, offset, chunk_size, qiov, qiov_offset, NULL);
+ if (ret < 0) {
+ break;
+ }
+ qiov_offset += chunk_size;
+ offset += chunk_size;
+ bytes -= chunk_size;
+ }
+
+ if (aio) {
+ aio_task_pool_wait_all(aio);
+ if (ret == 0) {
+ ret = aio_task_pool_status(aio);
+ }
+ g_free(aio);
+ }
+
+ return ret;
+}
+
static int coroutine_fn
qcow2_co_preadv_compressed(BlockDriverState *bs,
uint64_t file_cluster_offset,
diff --git a/block/qcow2.h b/block/qcow2.h
index f51f478..2d264c6 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -120,6 +120,7 @@
#define QCOW2_OPT_L2_CACHE_ENTRY_SIZE "l2-cache-entry-size"
#define QCOW2_OPT_REFCOUNT_CACHE_SIZE "refcount-cache-size"
#define QCOW2_OPT_CACHE_CLEAN_INTERVAL "cache-clean-interval"
+#define QCOW2_OPT_COMPRESS "compress"
typedef struct QCowHeader {
uint32_t magic;
diff --git a/blockdev.c b/blockdev.c
index fbef684..2103730 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -4682,6 +4682,10 @@ QemuOptsList qemu_common_drive_opts = {
.name = BDRV_OPT_READ_ONLY,
.type = QEMU_OPT_BOOL,
.help = "open drive file as read-only",
+ },{
+ .name = BDRV_OPT_COMPRESS,
+ .type = QEMU_OPT_BOOL,
+ .help = "compress all writes to image",
},
THROTTLE_OPTS,
diff --git a/include/block/block.h b/include/block/block.h
index 37c9de7..2980293 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -139,6 +139,7 @@ typedef struct HDGeometry {
#define BDRV_OPT_AUTO_READ_ONLY "auto-read-only"
#define BDRV_OPT_DISCARD "discard"
#define BDRV_OPT_FORCE_SHARE "force-share"
+#define BDRV_OPT_COMPRESS "compress"
#define BDRV_SECTOR_BITS 9
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 0422acd..6cb6604 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -922,6 +922,8 @@ struct BlockDriverState {
/* BdrvChild links to this node may never be frozen */
bool never_freeze;
+ /* Compress all writes to the image */
+ bool all_write_compressed;
};
struct BlockBackendRootState {
diff --git a/qapi/block-core.json b/qapi/block-core.json
index e6edd64..14d8ce2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3949,6 +3949,9 @@
# @force-share: force share all permission on added nodes.
# Requires read-only=true. (Since 2.10)
#
+# @compress: compress all writes to the image (Since 4.2)
+# (default: false)
+#
# Remaining options are determined by the block driver.
#
# Since: 2.9
@@ -3961,7 +3964,8 @@
'*read-only': 'bool',
'*auto-read-only': 'bool',
'*force-share': 'bool',
- '*detect-zeroes': 'BlockdevDetectZeroesOptions' },
+ '*detect-zeroes': 'BlockdevDetectZeroesOptions',
+ '*compress': 'bool' },
'discriminator': 'driver',
'data': {
'blkdebug': 'BlockdevOptionsBlkdebug',
diff --git a/qemu-options.hx b/qemu-options.hx
index 2a04ca6..0b0bfb9 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -850,7 +850,7 @@ DEF("blockdev", HAS_ARG, QEMU_OPTION_blockdev,
"-blockdev [driver=]driver[,node-name=N][,discard=ignore|unmap]\n"
" [,cache.direct=on|off][,cache.no-flush=on|off]\n"
" [,read-only=on|off][,detect-zeroes=on|off|unmap]\n"
- " [,driver specific parameters...]\n"
+ " [,compress=on|off][,driver specific parameters...]\n"
" configure a block backend\n", QEMU_ARCH_ALL)
STEXI
@item -blockdev @var{option}[,@var{option}[,@var{option}[,...]]]
@@ -905,6 +905,8 @@ discard requests.
conversion of plain zero writes by the OS to driver specific optimized
zero write commands. You may even choose "unmap" if @var{discard} is set
to "unmap" to allow a zero write to be converted to an @code{unmap} operation.
+@item compress
+Compress all writes to the image.
@end table
@item Driver-specific options for @code{file}
@@ -1026,7 +1028,7 @@ DEF("drive", HAS_ARG, QEMU_OPTION_drive,
" [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n"
" [,snapshot=on|off][,rerror=ignore|stop|report]\n"
" [,werror=ignore|stop|report|enospc][,id=name][,aio=threads|native]\n"
- " [,readonly=on|off][,copy-on-read=on|off]\n"
+ " [,readonly=on|off][,copy-on-read=on|off][,compress=on|off]\n"
" [,discard=ignore|unmap][,detect-zeroes=on|off|unmap]\n"
" [[,bps=b]|[[,bps_rd=r][,bps_wr=w]]]\n"
" [[,iops=i]|[[,iops_rd=r][,iops_wr=w]]]\n"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 2/5] tests/qemu-iotests: add case to write compressed data of multiple clusters
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 1/5] qcow2: Allow writing compressed data of multiple clusters Andrey Shinkevich
@ 2019-10-15 18:20 ` Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 3/5] block: support compressed write for copy-on-read Andrey Shinkevich
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrey Shinkevich @ 2019-10-15 18:20 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: kwolf, fam, vsementsov, armbru, mreitz, stefanha,
andrey.shinkevich, den, jsnow
Add the test case to the iotest #214 that checks possibility of writing
compressed data of more than one cluster size.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
---
tests/qemu-iotests/214 | 35 +++++++++++++++++++++++++++++++++++
tests/qemu-iotests/214.out | 15 +++++++++++++++
2 files changed, 50 insertions(+)
diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214
index 21ec8a2..0003dc2 100755
--- a/tests/qemu-iotests/214
+++ b/tests/qemu-iotests/214
@@ -89,6 +89,41 @@ _check_test_img -r all
$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir
$QEMU_IO -c "read -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir
+echo
+echo "=== Write compressed data of multiple clusters ==="
+echo
+cluster_size=0x10000
+_make_test_img 2M -o cluster_size=$cluster_size
+
+echo "Uncompressed data:"
+let data_size="8 * $cluster_size"
+$QEMU_IO -c "write -P 0xaa 0 $data_size" "$TEST_IMG" \
+ 2>&1 | _filter_qemu_io | _filter_testdir
+$QEMU_IMG info "$TEST_IMG" | sed -n '/disk size:/ s/^ *//p'
+
+_make_test_img 2M -o cluster_size=$cluster_size
+let data_size="3 * $cluster_size + ($cluster_size >> 1)"
+# Set compress=on. That will align the written data
+# by the cluster size and will write them compressed.
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
+$QEMU_IO -c "write -P 0xbb 0 $data_size" --image-opts \
+ driver=$IMGFMT,compress=on,file.filename=$TEST_IMG \
+ 2>&1 | _filter_qemu_io | _filter_testdir
+
+let offset="4 * $cluster_size"
+QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \
+$QEMU_IO -c "write -P 0xcc $offset $data_size" "json:{\
+ 'driver': '$IMGFMT',
+ 'file': {
+ 'driver': 'file',
+ 'filename': '$TEST_IMG'
+ },
+ 'compress': true
+}" | _filter_qemu_io | _filter_testdir
+
+echo "After the multiple cluster data have been written compressed,"
+$QEMU_IMG info "$TEST_IMG" | sed -n '/disk size:/ s/^ *//p'
+
# success, all done
echo '*** done'
rm -f $seq.full
diff --git a/tests/qemu-iotests/214.out b/tests/qemu-iotests/214.out
index 0fcd8dc..09a2e9a 100644
--- a/tests/qemu-iotests/214.out
+++ b/tests/qemu-iotests/214.out
@@ -32,4 +32,19 @@ read 4194304/4194304 bytes at offset 0
4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
read 4194304/4194304 bytes at offset 4194304
4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+=== Write compressed data of multiple clusters ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152
+Uncompressed data:
+wrote 524288/524288 bytes at offset 0
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+disk size: 772 KiB
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152
+wrote 229376/229376 bytes at offset 0
+224 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 229376/229376 bytes at offset 262144
+224 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+After the multiple cluster data have been written compressed,
+disk size: 268 KiB
*** done
--
1.8.3.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 3/5] block: support compressed write for copy-on-read
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 1/5] qcow2: Allow writing compressed data of multiple clusters Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 2/5] tests/qemu-iotests: add case to write " Andrey Shinkevich
@ 2019-10-15 18:20 ` Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 4/5] block-stream: add compress option Andrey Shinkevich
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrey Shinkevich @ 2019-10-15 18:20 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: kwolf, fam, vsementsov, armbru, mreitz, stefanha,
andrey.shinkevich, den, jsnow
Support the data compression during block-stream job over a backup
backing chain implemented in the following patch 'block-stream:
add compress option'.
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
---
block/io.c | 21 ++++++++++++++++-----
block/trace-events | 2 +-
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/block/io.c b/block/io.c
index 6a5509c..fc7f157 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1264,12 +1264,13 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(BdrvChild *child,
* allocating cluster in the image file. Note that this value may exceed
* BDRV_REQUEST_MAX_BYTES (even when the original read did not), which
* is one reason we loop rather than doing it all at once.
+ * Also, this is crucial for compressed copy-on-read.
*/
bdrv_round_to_clusters(bs, offset, bytes, &cluster_offset, &cluster_bytes);
skip_bytes = offset - cluster_offset;
trace_bdrv_co_do_copy_on_readv(bs, offset, bytes,
- cluster_offset, cluster_bytes);
+ cluster_offset, cluster_bytes, flags);
while (cluster_bytes) {
int64_t pnum;
@@ -1328,9 +1329,15 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(BdrvChild *child,
/* This does not change the data on the disk, it is not
* necessary to flush even in cache=writethrough mode.
*/
- ret = bdrv_driver_pwritev(bs, cluster_offset, pnum,
- &local_qiov, 0,
- BDRV_REQ_WRITE_UNCHANGED);
+ if (flags & BDRV_REQ_WRITE_COMPRESSED) {
+ ret = bdrv_driver_pwritev_compressed(bs, cluster_offset,
+ pnum, &local_qiov,
+ qiov_offset);
+ } else {
+ ret = bdrv_driver_pwritev(bs, cluster_offset, pnum,
+ &local_qiov, 0,
+ BDRV_REQ_WRITE_UNCHANGED);
+ }
}
if (ret < 0) {
@@ -1396,7 +1403,11 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
* to pass through to drivers. For now, there aren't any
* passthrough flags. */
assert(!(flags & ~(BDRV_REQ_NO_SERIALISING | BDRV_REQ_COPY_ON_READ |
- BDRV_REQ_PREFETCH)));
+ BDRV_REQ_PREFETCH | BDRV_REQ_WRITE_COMPRESSED)));
+
+ /* write compressed only makes sense with copy on read */
+ assert(!(flags & BDRV_REQ_WRITE_COMPRESSED) ||
+ (flags & BDRV_REQ_COPY_ON_READ));
/* Handle Copy on Read and associated serialisation */
if (flags & BDRV_REQ_COPY_ON_READ) {
diff --git a/block/trace-events b/block/trace-events
index 3aa27e6..f444548 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -14,7 +14,7 @@ blk_root_detach(void *child, void *blk, void *bs) "child %p blk %p bs %p"
bdrv_co_preadv(void *bs, int64_t offset, int64_t nbytes, unsigned int flags) "bs %p offset %"PRId64" nbytes %"PRId64" flags 0x%x"
bdrv_co_pwritev(void *bs, int64_t offset, int64_t nbytes, unsigned int flags) "bs %p offset %"PRId64" nbytes %"PRId64" flags 0x%x"
bdrv_co_pwrite_zeroes(void *bs, int64_t offset, int count, int flags) "bs %p offset %"PRId64" count %d flags 0x%x"
-bdrv_co_do_copy_on_readv(void *bs, int64_t offset, unsigned int bytes, int64_t cluster_offset, int64_t cluster_bytes) "bs %p offset %"PRId64" bytes %u cluster_offset %"PRId64" cluster_bytes %"PRId64
+bdrv_co_do_copy_on_readv(void *bs, int64_t offset, unsigned int bytes, int64_t cluster_offset, int64_t cluster_bytes, int flags) "bs %p offset %"PRId64" bytes %u cluster_offset %"PRId64" cluster_bytes %"PRId64" flags 0x%x"
bdrv_co_copy_range_from(void *src, uint64_t src_offset, void *dst, uint64_t dst_offset, uint64_t bytes, int read_flags, int write_flags) "src %p offset %"PRIu64" dst %p offset %"PRIu64" bytes %"PRIu64" rw flags 0x%x 0x%x"
bdrv_co_copy_range_to(void *src, uint64_t src_offset, void *dst, uint64_t dst_offset, uint64_t bytes, int read_flags, int write_flags) "src %p offset %"PRIu64" dst %p offset %"PRIu64" bytes %"PRIu64" rw flags 0x%x 0x%x"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 4/5] block-stream: add compress option
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
` (2 preceding siblings ...)
2019-10-15 18:20 ` [PATCH v3 3/5] block: support compressed write for copy-on-read Andrey Shinkevich
@ 2019-10-15 18:20 ` Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 5/5] tests/qemu-iotests: add case for block-stream compress Andrey Shinkevich
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrey Shinkevich @ 2019-10-15 18:20 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: kwolf, fam, vsementsov, armbru, mreitz, stefanha,
andrey.shinkevich, den, jsnow
Allow data compression during block-stream job for backup backing chain.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
---
block/stream.c | 10 ++++++++--
blockdev.c | 12 +++++++++++-
2 files changed, 19 insertions(+), 3 deletions(-)
diff --git a/block/stream.c b/block/stream.c
index 5562ccb..25f9324 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -41,10 +41,16 @@ typedef struct StreamBlockJob {
static int coroutine_fn stream_populate(BlockBackend *blk,
int64_t offset, uint64_t bytes)
{
+ BlockDriverState *bs = blk_bs(blk);
+ int flags = BDRV_REQ_COPY_ON_READ | BDRV_REQ_PREFETCH;
+
+ if (bs->all_write_compressed) {
+ flags |= BDRV_REQ_WRITE_COMPRESSED;
+ }
+
assert(bytes < SIZE_MAX);
- return blk_co_preadv(blk, offset, bytes, NULL,
- BDRV_REQ_COPY_ON_READ | BDRV_REQ_PREFETCH);
+ return blk_co_preadv(blk, offset, bytes, NULL, flags);
}
static void stream_abort(Job *job)
diff --git a/blockdev.c b/blockdev.c
index 2103730..fd824da 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -471,7 +471,7 @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts,
int bdrv_flags = 0;
int on_read_error, on_write_error;
bool account_invalid, account_failed;
- bool writethrough, read_only;
+ bool writethrough, read_only, compress;
BlockBackend *blk;
BlockDriverState *bs;
ThrottleConfig cfg;
@@ -570,6 +570,7 @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts,
}
read_only = qemu_opt_get_bool(opts, BDRV_OPT_READ_ONLY, false);
+ compress = qemu_opt_get_bool(opts, BDRV_OPT_COMPRESS, false);
/* init */
if ((!file || !*file) && !qdict_size(bs_opts)) {
@@ -595,6 +596,8 @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts,
qdict_set_default_str(bs_opts, BDRV_OPT_READ_ONLY,
read_only ? "on" : "off");
qdict_set_default_str(bs_opts, BDRV_OPT_AUTO_READ_ONLY, "on");
+ qdict_set_default_str(bs_opts, BDRV_OPT_COMPRESS,
+ compress ? "on" : "off");
assert((bdrv_flags & BDRV_O_CACHE_MASK) == 0);
if (runstate_check(RUN_STATE_INMIGRATE)) {
@@ -3308,6 +3311,13 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
goto out;
}
+ if (bs->all_write_compressed &&
+ bs->drv->bdrv_co_pwritev_compressed_part == NULL) {
+ error_setg(errp, "Compression is not supported for this drive %s",
+ bdrv_get_device_name(bs));
+ goto out;
+ }
+
/* backing_file string overrides base bs filename */
base_name = has_backing_file ? backing_file : base_name;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 5/5] tests/qemu-iotests: add case for block-stream compress
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
` (3 preceding siblings ...)
2019-10-15 18:20 ` [PATCH v3 4/5] block-stream: add compress option Andrey Shinkevich
@ 2019-10-15 18:20 ` Andrey Shinkevich
2019-10-16 5:52 ` [PATCH v3 0/5] qcow2: advanced compression options no-reply
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Andrey Shinkevich @ 2019-10-15 18:20 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: kwolf, fam, vsementsov, armbru, mreitz, stefanha,
andrey.shinkevich, den, jsnow
Add a case to the iotest #030 that tests the 'compress' option for a
block-stream job.
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
---
tests/qemu-iotests/030 | 51 +++++++++++++++++++++++++++++++++++++++++++++-
tests/qemu-iotests/030.out | 4 ++--
2 files changed, 52 insertions(+), 3 deletions(-)
diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030
index f3766f2..f0f0e26 100755
--- a/tests/qemu-iotests/030
+++ b/tests/qemu-iotests/030
@@ -21,7 +21,8 @@
import time
import os
import iotests
-from iotests import qemu_img, qemu_io
+from iotests import qemu_img, qemu_io, qemu_img_pipe
+import json
backing_img = os.path.join(iotests.test_dir, 'backing.img')
mid_img = os.path.join(iotests.test_dir, 'mid.img')
@@ -956,6 +957,54 @@ class TestSetSpeed(iotests.QMPTestCase):
self.cancel_and_wait(resume=True)
+class TestCompressed(iotests.QMPTestCase):
+ test_img_init_size = 0
+
+ def setUp(self):
+ qemu_img('create', '-f', iotests.imgfmt, backing_img, '1M')
+ qemu_img('create', '-f', iotests.imgfmt, '-o',
+ 'backing_file=%s' % backing_img, mid_img)
+ qemu_img('create', '-f', iotests.imgfmt, '-o',
+ 'backing_file=%s' % mid_img, test_img)
+ qemu_io('-c', 'write -P 0x1 0 512k', backing_img)
+ top = json.loads(qemu_img_pipe('info', '--output=json', test_img))
+ self.test_img_init_size = top['actual-size']
+ self.vm = iotests.VM().add_drive(test_img, "backing.node-name=mid," +
+ "backing.backing.node-name=base," +
+ "compress=on")
+ self.vm.launch()
+
+ def tearDown(self):
+ self.vm.shutdown()
+ os.remove(test_img)
+ os.remove(mid_img)
+ os.remove(backing_img)
+
+ def test_stream_compress(self):
+ self.assert_no_active_block_jobs()
+
+ result = self.vm.qmp('block-stream', device='mid', job_id='stream-mid')
+ self.assert_qmp(result, 'return', {})
+
+ self.wait_until_completed(drive='stream-mid')
+ # Remove other 'JOB_STATUS_CHANGE' events for the job 'stream-mid'
+ self.vm.get_qmp_events(wait=True)
+
+ result = self.vm.qmp('block-stream', device='drive0',
+ job_id='stream-top')
+ self.assert_qmp(result, 'return', {})
+
+ self.wait_until_completed(drive='stream-top')
+ self.vm.shutdown()
+
+ top = json.loads(qemu_img_pipe('info', '--output=json', test_img))
+ mid = json.loads(qemu_img_pipe('info', '--output=json', mid_img))
+ base = json.loads(qemu_img_pipe('info', '--output=json', backing_img))
+
+ self.assertEqual(mid['actual-size'], base['actual-size'])
+ self.assertLess(top['actual-size'], mid['actual-size'])
+ self.assertLess(self.test_img_init_size, top['actual-size'])
+
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2', 'qed'],
supported_protocols=['file'])
diff --git a/tests/qemu-iotests/030.out b/tests/qemu-iotests/030.out
index 6d9bee1..af8dac1 100644
--- a/tests/qemu-iotests/030.out
+++ b/tests/qemu-iotests/030.out
@@ -1,5 +1,5 @@
-...........................
+............................
----------------------------------------------------------------------
-Ran 27 tests
+Ran 28 tests
OK
--
1.8.3.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v3 0/5] qcow2: advanced compression options
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
` (4 preceding siblings ...)
2019-10-15 18:20 ` [PATCH v3 5/5] tests/qemu-iotests: add case for block-stream compress Andrey Shinkevich
@ 2019-10-16 5:52 ` no-reply
2019-10-16 5:55 ` no-reply
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: no-reply @ 2019-10-16 5:52 UTC (permalink / raw)
To: andrey.shinkevich
Cc: kwolf, fam, vsementsov, qemu-block, qemu-devel, armbru, stefanha,
den, andrey.shinkevich, mreitz, jsnow
Patchew URL: https://patchew.org/QEMU/1571163625-642312-1-git-send-email-andrey.shinkevich@virtuozzo.com/
Hi,
This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===
--> Processing Dependency: perl(File::Copy) for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: perl(File::Basename) for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: perl(Exporter) for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: perl(Error) for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: openssh-clients for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: less for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: /usr/bin/perl for package: git-1.8.3.1-20.el7.x86_64
---
--> Processing Dependency: perl(Filter::Util::Call) for package: 4:perl-5.16.3-294.el7_6.x86_64
--> Processing Dependency: libperl.so()(64bit) for package: 4:perl-5.16.3-294.el7_6.x86_64
---> Package perl-Carp.noarch 0:1.26-244.el7 will be installed
---> Package perl-Error.noarch 1:0.17020-2.el7 will be installed
---> Package perl-Exporter.noarch 0:5.68-3.el7 will be installed
---> Package perl-File-Path.noarch 0:2.09-2.el7 will be installed
---> Package perl-File-Temp.noarch 0:0.23.01-3.el7 will be installed
---
perl-Carp noarch 1.26-244.el7 base 19 k
perl-Data-Dumper x86_64 2.145-3.el7 base 47 k
perl-Encode x86_64 2.51-7.el7 base 1.5 M
perl-Error noarch 1:0.17020-2.el7 base 32 k
perl-Exporter noarch 5.68-3.el7 base 28 k
perl-File-Path noarch 2.09-2.el7 base 26 k
perl-File-Temp noarch 0.23.01-3.el7 base 56 k
---
Total download size: 146 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
warning: /var/cache/yum/x86_64/7/epel/packages/ccache-3.3.4-1.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Public key for ccache-3.3.4-1.el7.x86_64.rpm is not installed
warning: /var/cache/yum/x86_64/7/centos-virt-xen-48/packages/kernel-headers-4.9.188-35.el7.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 61e8806c: NOKEY
Public key for kernel-headers-4.9.188-35.el7.x86_64.rpm is not installed
--------------------------------------------------------------------------------
Total 7.5 MB/s | 146 MB 00:19
---
Updating : glibc-common-2.17-292.el7.x86_64 2/315
Updating : nss-softokn-freebl-3.44.0-5.el7.x86_64 3/315
Updating : glibc-2.17-292.el7.x86_64 4/315
warning: /etc/nsswitch.conf created as /etc/nsswitch.conf.rpmnew
Updating : nspr-4.21.0-1.el7.x86_64 5/315
Updating : nss-util-3.44.0-3.el7.x86_64 6/315
Updating : libstdc++-4.8.5-39.el7.x86_64 7/315
---
Installing : 1:perl-Pod-Simple-3.28-4.el7.noarch 118/315
Installing : perl-Getopt-Long-2.40-3.el7.noarch 119/315
Installing : 4:perl-5.16.3-294.el7_6.x86_64 120/315
Installing : 1:perl-Error-0.17020-2.el7.noarch 121/315
Installing : perl-TermReadKey-2.30-20.el7.x86_64 122/315
Installing : perl-Data-Dumper-2.145-3.el7.x86_64 123/315
Installing : autoconf-2.69-11.el7.noarch 124/315
---
Installing : python-rpm-macros-3-32.el7.noarch 193/315
Installing : libffi-devel-3.0.13-18.el7.x86_64 194/315
install-info: No such file or directory for /usr/share/info/libffi.info.gz
warning: %post(libffi-devel-3.0.13-18.el7.x86_64) scriptlet failed, exit status 1
Non-fatal POSTIN scriptlet failure in rpm package libffi-devel-3.0.13-18.el7.x86_64
Installing : xml-common-0.6.3-39.el7.noarch 195/315
Installing : iso-codes-3.46-2.el7.noarch 196/315
---
Verifying : flac-libs-1.3.0-5.el7_1.x86_64 261/315
Verifying : opus-1.0.2-6.el7.x86_64 262/315
Verifying : libuuid-2.23.2-61.el7.x86_64 263/315
Verifying : 1:perl-Error-0.17020-2.el7.noarch 264/315
Verifying : libXfixes-5.0.3-1.el7.x86_64 265/315
Verifying : xml-common-0.6.3-39.el7.noarch 266/315
Verifying : perl-Scalar-List-Utils-1.27-248.el7.x86_64 267/315
---
perl-Carp.noarch 0:1.26-244.el7
perl-Data-Dumper.x86_64 0:2.145-3.el7
perl-Encode.x86_64 0:2.51-7.el7
perl-Error.noarch 1:0.17020-2.el7
perl-Exporter.noarch 0:5.68-3.el7
perl-File-Path.noarch 0:2.09-2.el7
perl-File-Temp.noarch 0:0.23.01-3.el7
---
libudev no
default devices yes
warning: Python 2 support is deprecated
warning: Python 3 will be required for building future versions of QEMU
cross containers no
NOTE: guest cross-compilers enabled: cc
---
CC block/qed-check.o
CC block/vhdx.o
/tmp/qemu-test/src/block/qcow2.c: In function 'qcow2_co_pwritev_compressed_part':
/tmp/qemu-test/src/block/qcow2.c:4244:9: error: 'ret' may be used uninitialized in this function [-Werror=maybe-uninitialized]
int ret;
^
cc1: all warnings being treated as errors
make: *** [block/qcow2.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
---
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=dc4bae98efd811e99e7f68b59973b7d0', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-hb38s10j/src/docker-src.2019-10-16-01.50.14.25013:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-hb38s10j/src'
make: *** [docker-run-test-quick@centos7] Error 2
real 2m26.033s
user 0m8.345s
The full log is available at
http://patchew.org/logs/1571163625-642312-1-git-send-email-andrey.shinkevich@virtuozzo.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 0/5] qcow2: advanced compression options
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
` (5 preceding siblings ...)
2019-10-16 5:52 ` [PATCH v3 0/5] qcow2: advanced compression options no-reply
@ 2019-10-16 5:55 ` no-reply
2019-10-16 11:00 ` no-reply
2019-10-16 11:04 ` no-reply
8 siblings, 0 replies; 10+ messages in thread
From: no-reply @ 2019-10-16 5:55 UTC (permalink / raw)
To: andrey.shinkevich
Cc: kwolf, fam, vsementsov, qemu-block, qemu-devel, armbru, stefanha,
den, andrey.shinkevich, mreitz, jsnow
Patchew URL: https://patchew.org/QEMU/1571163625-642312-1-git-send-email-andrey.shinkevich@virtuozzo.com/
Hi,
This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===
CC block/blklogwrites.o
CC block/block-backend.o
/tmp/qemu-test/src/block/qcow2.c: In function 'qcow2_co_pwritev_compressed_part':
/tmp/qemu-test/src/block/qcow2.c:4244:9: error: 'ret' may be used uninitialized in this function [-Werror=maybe-uninitialized]
int ret;
^~~
cc1: all warnings being treated as errors
make: *** [/tmp/qemu-test/src/rules.mak:69: block/qcow2.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
---
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=4299392cefd911e9addb68b59973b7d0', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-r2c14at8/src/docker-src.2019-10-16-01.53.08.3890:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-r2c14at8/src'
make: *** [docker-run-test-mingw@fedora] Error 2
real 2m50.343s
user 0m8.261s
The full log is available at
http://patchew.org/logs/1571163625-642312-1-git-send-email-andrey.shinkevich@virtuozzo.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 0/5] qcow2: advanced compression options
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
` (6 preceding siblings ...)
2019-10-16 5:55 ` no-reply
@ 2019-10-16 11:00 ` no-reply
2019-10-16 11:04 ` no-reply
8 siblings, 0 replies; 10+ messages in thread
From: no-reply @ 2019-10-16 11:00 UTC (permalink / raw)
To: andrey.shinkevich
Cc: kwolf, fam, vsementsov, qemu-block, qemu-devel, armbru, stefanha,
den, andrey.shinkevich, mreitz, jsnow
Patchew URL: https://patchew.org/QEMU/1571163625-642312-1-git-send-email-andrey.shinkevich@virtuozzo.com/
Hi,
This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===
--> Processing Dependency: perl(File::Copy) for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: perl(File::Basename) for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: perl(Exporter) for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: perl(Error) for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: openssh-clients for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: less for package: git-1.8.3.1-20.el7.x86_64
--> Processing Dependency: /usr/bin/perl for package: git-1.8.3.1-20.el7.x86_64
---
--> Processing Dependency: perl(Filter::Util::Call) for package: 4:perl-5.16.3-294.el7_6.x86_64
--> Processing Dependency: libperl.so()(64bit) for package: 4:perl-5.16.3-294.el7_6.x86_64
---> Package perl-Carp.noarch 0:1.26-244.el7 will be installed
---> Package perl-Error.noarch 1:0.17020-2.el7 will be installed
---> Package perl-Exporter.noarch 0:5.68-3.el7 will be installed
---> Package perl-File-Path.noarch 0:2.09-2.el7 will be installed
---> Package perl-File-Temp.noarch 0:0.23.01-3.el7 will be installed
---
perl-Carp noarch 1.26-244.el7 base 19 k
perl-Data-Dumper x86_64 2.145-3.el7 base 47 k
perl-Encode x86_64 2.51-7.el7 base 1.5 M
perl-Error noarch 1:0.17020-2.el7 base 32 k
perl-Exporter noarch 5.68-3.el7 base 28 k
perl-File-Path noarch 2.09-2.el7 base 26 k
perl-File-Temp noarch 0.23.01-3.el7 base 56 k
---
Total download size: 146 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
warning: /var/cache/yum/x86_64/7/epel/packages/ccache-3.3.4-1.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Public key for ccache-3.3.4-1.el7.x86_64.rpm is not installed
warning: /var/cache/yum/x86_64/7/centos-virt-xen-48/packages/kernel-headers-4.9.188-35.el7.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 61e8806c: NOKEY
Public key for kernel-headers-4.9.188-35.el7.x86_64.rpm is not installed
--------------------------------------------------------------------------------
Total 8.3 MB/s | 146 MB 00:17
---
Updating : glibc-common-2.17-292.el7.x86_64 2/315
Updating : nss-softokn-freebl-3.44.0-5.el7.x86_64 3/315
Updating : glibc-2.17-292.el7.x86_64 4/315
warning: /etc/nsswitch.conf created as /etc/nsswitch.conf.rpmnew
Updating : nspr-4.21.0-1.el7.x86_64 5/315
Updating : nss-util-3.44.0-3.el7.x86_64 6/315
Updating : libstdc++-4.8.5-39.el7.x86_64 7/315
---
Installing : 1:perl-Pod-Simple-3.28-4.el7.noarch 118/315
Installing : perl-Getopt-Long-2.40-3.el7.noarch 119/315
Installing : 4:perl-5.16.3-294.el7_6.x86_64 120/315
Installing : 1:perl-Error-0.17020-2.el7.noarch 121/315
Installing : perl-TermReadKey-2.30-20.el7.x86_64 122/315
Installing : perl-Data-Dumper-2.145-3.el7.x86_64 123/315
Installing : autoconf-2.69-11.el7.noarch 124/315
---
Installing : python-rpm-macros-3-32.el7.noarch 193/315
Installing : libffi-devel-3.0.13-18.el7.x86_64 194/315
install-info: No such file or directory for /usr/share/info/libffi.info.gz
warning: %post(libffi-devel-3.0.13-18.el7.x86_64) scriptlet failed, exit status 1
Non-fatal POSTIN scriptlet failure in rpm package libffi-devel-3.0.13-18.el7.x86_64
Installing : xml-common-0.6.3-39.el7.noarch 195/315
Installing : iso-codes-3.46-2.el7.noarch 196/315
---
Verifying : flac-libs-1.3.0-5.el7_1.x86_64 261/315
Verifying : opus-1.0.2-6.el7.x86_64 262/315
Verifying : libuuid-2.23.2-61.el7.x86_64 263/315
Verifying : 1:perl-Error-0.17020-2.el7.noarch 264/315
Verifying : libXfixes-5.0.3-1.el7.x86_64 265/315
Verifying : xml-common-0.6.3-39.el7.noarch 266/315
Verifying : perl-Scalar-List-Utils-1.27-248.el7.x86_64 267/315
---
perl-Carp.noarch 0:1.26-244.el7
perl-Data-Dumper.x86_64 0:2.145-3.el7
perl-Encode.x86_64 0:2.51-7.el7
perl-Error.noarch 1:0.17020-2.el7
perl-Exporter.noarch 0:5.68-3.el7
perl-File-Path.noarch 0:2.09-2.el7
perl-File-Temp.noarch 0:0.23.01-3.el7
---
libudev no
default devices yes
warning: Python 2 support is deprecated
warning: Python 3 will be required for building future versions of QEMU
cross containers no
NOTE: guest cross-compilers enabled: cc
---
CC block/accounting.o
CC block/dirty-bitmap.o
/tmp/qemu-test/src/block/qcow2.c: In function 'qcow2_co_pwritev_compressed_part':
/tmp/qemu-test/src/block/qcow2.c:4244:9: error: 'ret' may be used uninitialized in this function [-Werror=maybe-uninitialized]
int ret;
^
cc1: all warnings being treated as errors
make: *** [block/qcow2.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
---
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=f18e34eef00311e98c0768b59973b7d0', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-9ddg5_qn/src/docker-src.2019-10-16-06.58.38.3939:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-9ddg5_qn/src'
make: *** [docker-run-test-quick@centos7] Error 2
real 2m4.544s
user 0m7.752s
The full log is available at
http://patchew.org/logs/1571163625-642312-1-git-send-email-andrey.shinkevich@virtuozzo.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 0/5] qcow2: advanced compression options
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
` (7 preceding siblings ...)
2019-10-16 11:00 ` no-reply
@ 2019-10-16 11:04 ` no-reply
8 siblings, 0 replies; 10+ messages in thread
From: no-reply @ 2019-10-16 11:04 UTC (permalink / raw)
To: andrey.shinkevich
Cc: kwolf, fam, vsementsov, qemu-block, qemu-devel, armbru, stefanha,
den, andrey.shinkevich, mreitz, jsnow
Patchew URL: https://patchew.org/QEMU/1571163625-642312-1-git-send-email-andrey.shinkevich@virtuozzo.com/
Hi,
This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===
CC block/snapshot.o
CC block/qapi.o
/tmp/qemu-test/src/block/qcow2.c: In function 'qcow2_co_pwritev_compressed_part':
/tmp/qemu-test/src/block/qcow2.c:4244:9: error: 'ret' may be used uninitialized in this function [-Werror=maybe-uninitialized]
int ret;
^~~
cc1: all warnings being treated as errors
make: *** [/tmp/qemu-test/src/rules.mak:69: block/qcow2.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 664, in <module>
---
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=4c58e220f00411e9be7868b59973b7d0', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-eeoq4u_0/src/docker-src.2019-10-16-07.01.11.11056:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-eeoq4u_0/src'
make: *** [docker-run-test-mingw@fedora] Error 2
real 2m57.644s
user 0m8.518s
The full log is available at
http://patchew.org/logs/1571163625-642312-1-git-send-email-andrey.shinkevich@virtuozzo.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2019-10-16 11:06 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-15 18:20 [PATCH v3 0/5] qcow2: advanced compression options Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 1/5] qcow2: Allow writing compressed data of multiple clusters Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 2/5] tests/qemu-iotests: add case to write " Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 3/5] block: support compressed write for copy-on-read Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 4/5] block-stream: add compress option Andrey Shinkevich
2019-10-15 18:20 ` [PATCH v3 5/5] tests/qemu-iotests: add case for block-stream compress Andrey Shinkevich
2019-10-16 5:52 ` [PATCH v3 0/5] qcow2: advanced compression options no-reply
2019-10-16 5:55 ` no-reply
2019-10-16 11:00 ` no-reply
2019-10-16 11:04 ` no-reply
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.