From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60463) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLHQm-00023m-GV for qemu-devel@nongnu.org; Tue, 10 Feb 2015 15:29:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YLHQj-0006pa-8R for qemu-devel@nongnu.org; Tue, 10 Feb 2015 15:29:12 -0500 Received: from mx1.redhat.com ([209.132.183.28]:40909) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLHQj-0006pL-0e for qemu-devel@nongnu.org; Tue, 10 Feb 2015 15:29:09 -0500 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1AKT86t006501 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Tue, 10 Feb 2015 15:29:08 -0500 From: Max Reitz Date: Tue, 10 Feb 2015 15:28:42 -0500 Message-Id: <1423600146-7642-1-git-send-email-mreitz@redhat.com> Subject: [Qemu-devel] [PATCH v6 00/24] qcow2: Support refcount orders != 4 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Kevin Wolf , Stefan Hajnoczi , Max Reitz As of version 3, the qcow2 file format supports different widths for refcount entries, ranging from 1 to 64 bit (only powers of two). Currently, qemu only supports 16 bit, which is the only width supported by version 2 (compat=0.10) images. This series adds support to qemu for all other valid refcount orders. This is mainly done by adding two function pointers into the BDRVQcowState structure for reading and writing refcount values independently of the current refcount entry width; all in-memory refcount arrays (mostly cached refcount blocks) now are void pointers and are accessed through these functions alone. Thanks to previous work of making the qemu code agnostic of e.g. the number of refcount entries per refcount block, the rest is fairly trivial. The most complex patch in this series is patch 21 which implements changing the refcount order through qemu-img amend. To test different refcount widths, simply invoke the qemu-iotests check program with -o refcount_bits=${your_desired_width}. The final test in this series adds some tests for operations which do not work with certain refcount orders and for refcount order amendment. In order for iotest 015 to succeed with refcount_bits=32 and refcount_bits=64, this series depends on my series "qcow2: Respect new_block in alloc_refcount_block()". v6: - Dropped the old patches 7 and 8 in favor of the separate patch "qcow2: Rewrite qcow2_alloc_bytes()" - Patch 1: Due to that patch, there is an additional place where 0xffff has to be replaced by s->refcount_max - Patch 4: Rebase conflict due to the different qcow2_alloc_bytes() - Patch 5: - Introduce the function refcount_diff() to calculate the absolute difference between two refcount values, and use it [Kevin] - Rebase conflict due to the different qcow2_alloc_bytes() - Added an assertion for -1 <= addend <= 1 in qcow2_update_snapshot_refcount() (which means that using abs() to determine its absolute value is correct) - Patch 6: - Rebase conflict due to the different qcow2_alloc_bytes() - Rebase conflicts due to usage of refcount_diff() - No need to use imaxabs() in qcow2_update_snapshot_refcount() [Kevin] - Patch 7: - Simplify refcount_array_byte_size() [Kevin] - Mention "on success 0 is returned" in the comment describing realloc_refcount_array() [Kevin] - Use size_to_clusters(x) * cluster_size instead of ROUND_UP(x, cluster_size) [Kevin] - Patch 8: - Removed superfluous casts in two more places [Kevin] - The offset of the data of a refcount block in the refcount array is refblock_index * cluster_size instead of refblock_index << refcount_block_bits [Kevin] - Patch 24: s/2014/2015/ (kept Eric's R-b) git-backport-diff against v5: Key: [----] : patches are identical [####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/24:[0002] [FC] 'qcow2: Add two new fields to BDRVQcowState' 002/24:[----] [--] 'qcow2: Add refcount_bits to format-specific info' 003/24:[----] [--] 'qcow2: Do not return new value after refcount update' 004/24:[0007] [FC] 'qcow2: Only return status from qcow2_get_refcount' 005/24:[0018] [FC] 'qcow2: Use unsigned addend for update_refcount()' 006/24:[0012] [FC] 'qcow2: Use 64 bits for refcount values' 007/24:[0047] [FC] 'qcow2: Helper for refcount array reallocation' 008/24:[0006] [FC] 'qcow2: Helper function for refcount modification' 009/24:[----] [--] 'qcow2: More helpers for refcount modification' 010/24:[----] [--] 'qcow2: Open images with refcount order != 4' 011/24:[----] [--] 'qcow2: refcount_order parameter for qcow2_create2' 012/24:[----] [--] 'qcow2: Use symbolic macros in qcow2_amend_options' 013/24:[----] [-C] 'iotests: Prepare for refcount_bits option' 014/24:[----] [--] 'qcow2: Allow creation with refcount order != 4' 015/24:[----] [--] 'progress: Allow regressing progress' 016/24:[----] [--] 'block: Add opaque value to the amend CB' 017/24:[----] [--] 'qcow2: Use error_report() in qcow2_amend_options()' 018/24:[----] [--] 'qcow2: Use abort() instead of assert(false)' 019/24:[----] [--] 'qcow2: Split upgrade/downgrade paths for amend' 020/24:[----] [--] 'qcow2: Use intermediate helper CB for amend' 021/24:[----] [--] 'qcow2: Add function for refcount order amendment' 022/24:[----] [--] 'qcow2: Invoke refcount order amendment function' 023/24:[----] [--] 'qcow2: Point to amend function in check' 024/24:[0002] [FC] 'iotests: Add test for different refcount widths' Max Reitz (24): qcow2: Add two new fields to BDRVQcowState qcow2: Add refcount_bits to format-specific info qcow2: Do not return new value after refcount update qcow2: Only return status from qcow2_get_refcount qcow2: Use unsigned addend for update_refcount() qcow2: Use 64 bits for refcount values qcow2: Helper for refcount array reallocation qcow2: Helper function for refcount modification qcow2: More helpers for refcount modification qcow2: Open images with refcount order != 4 qcow2: refcount_order parameter for qcow2_create2 qcow2: Use symbolic macros in qcow2_amend_options iotests: Prepare for refcount_bits option qcow2: Allow creation with refcount order != 4 progress: Allow regressing progress block: Add opaque value to the amend CB qcow2: Use error_report() in qcow2_amend_options() qcow2: Use abort() instead of assert(false) qcow2: Split upgrade/downgrade paths for amend qcow2: Use intermediate helper CB for amend qcow2: Add function for refcount order amendment qcow2: Invoke refcount order amendment function qcow2: Point to amend function in check iotests: Add test for different refcount widths block.c | 4 +- block/qcow2-cluster.c | 25 +- block/qcow2-refcount.c | 1000 +++++++++++++++++++++++++++++++------- block/qcow2.c | 286 ++++++++--- block/qcow2.h | 28 +- include/block/block.h | 4 +- include/block/block_int.h | 4 +- qapi/block-core.json | 5 +- qemu-img.c | 5 +- tests/qemu-iotests/007 | 3 + tests/qemu-iotests/015 | 2 + tests/qemu-iotests/026 | 7 + tests/qemu-iotests/029 | 2 + tests/qemu-iotests/049.out | 108 ++-- tests/qemu-iotests/051 | 3 + tests/qemu-iotests/058 | 2 + tests/qemu-iotests/060.out | 1 + tests/qemu-iotests/061.out | 14 +- tests/qemu-iotests/065 | 23 +- tests/qemu-iotests/067 | 2 + tests/qemu-iotests/067.out | 5 + tests/qemu-iotests/079 | 10 +- tests/qemu-iotests/079.out | 38 +- tests/qemu-iotests/080 | 2 + tests/qemu-iotests/082.out | 48 +- tests/qemu-iotests/085.out | 38 +- tests/qemu-iotests/089 | 2 + tests/qemu-iotests/089.out | 2 + tests/qemu-iotests/108 | 2 + tests/qemu-iotests/112 | 296 +++++++++++ tests/qemu-iotests/112.out | 155 ++++++ tests/qemu-iotests/common.filter | 3 +- tests/qemu-iotests/group | 1 + util/qemu-progress.c | 3 +- 34 files changed, 1740 insertions(+), 393 deletions(-) create mode 100755 tests/qemu-iotests/112 create mode 100644 tests/qemu-iotests/112.out -- 2.1.0