From: Eric Blake <eblake@redhat.com>
To: Alberto Garcia <berto@igalia.com>, qemu-devel@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>,
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>,
qemu-block@nongnu.org, Max Reitz <mreitz@redhat.com>
Subject: Re: [PATCH v5 14/31] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()
Date: Tue, 5 May 2020 16:08:57 -0500 [thread overview]
Message-ID: <e6a65feb-e7f9-b006-8171-014e41f62395@redhat.com> (raw)
In-Reply-To: <5b744e9c0f3d348ea675c7a52aabaeeab6fe7b64.1588699789.git.berto@igalia.com>
On 5/5/20 12:38 PM, Alberto Garcia wrote:
> This patch adds QCow2SubclusterType, which is the subcluster-level
> version of QCow2ClusterType. All QCOW2_SUBCLUSTER_* values have the
> the same meaning as their QCOW2_CLUSTER_* equivalents (when they
> exist). See below for details and caveats.
>
> In images without extended L2 entries clusters are treated as having
> exactly one subcluster so it is possible to replace one data type with
> the other while keeping the exact same semantics.
>
> With extended L2 entries there are new possible values, and every
> subcluster in the same cluster can obviously have a different
> QCow2SubclusterType so functions need to be adapted to work on the
> subcluster level.
>
> There are several things that have to be taken into account:
>
> a) QCOW2_SUBCLUSTER_COMPRESSED means that the whole cluster is
> compressed. We do not support compression at the subcluster
> level.
>
> b) There are two different values for unallocated subclusters:
> QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN which means that the whole
> cluster is unallocated, and QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC
> which means that the cluster is allocated but the subcluster is
> not. The latter can only happen in images with extended L2
> entries.
Or put differently, extents of the qcow2 file are always allocated a
contiguous cluster at a time (so using larger clusters reduces
fragmentation), but because we can now defer to the backing image a
sub-cluster at a time, we have less I/O to perform the first time the
guest touches a subcluster. The two different return values thus tell
us when we need to do a cluster allocation vs. just an in-place
overwrite or a sub-cluster COW.
>
> c) QCOW2_SUBCLUSTER_INVALID is used to detect the cases where an L2
> entry has a value that violates the specification. The caller is
> responsible for handling these situations.
>
> To prevent compatibility problems with images that have invalid
> values but are currently being read by QEMU without causing side
> effects, QCOW2_SUBCLUSTER_INVALID is only returned for images
> with extended L2 entries.
>
> qcow2_cluster_to_subcluster_type() is added as a separate function
> from qcow2_get_subcluster_type(), but this is only temporary and both
> will be merged in a subsequent patch.
>
> Signed-off-by: Alberto Garcia <berto@igalia.com>
> ---
> block/qcow2.h | 127 +++++++++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 126 insertions(+), 1 deletion(-)
>
> diff --git a/block/qcow2.h b/block/qcow2.h
> index 4ad93772b9..be7816a3b8 100644
> --- a/block/qcow2.h
> +++ b/block/qcow2.h
> @@ -80,6 +80,21 @@
>
> #define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32
>
> +/* The subcluster X [0..31] is allocated */
> +#define QCOW_OFLAG_SUB_ALLOC(X) (1ULL << (X))
> +/* The subcluster X [0..31] reads as zeroes */
> +#define QCOW_OFLAG_SUB_ZERO(X) (QCOW_OFLAG_SUB_ALLOC(X) << 32)
> +/* Subclusters X to Y (both included) are allocated */
> +#define QCOW_OFLAG_SUB_ALLOC_RANGE(X, Y) \
> + (QCOW_OFLAG_SUB_ALLOC((Y) + 1) - QCOW_OFLAG_SUB_ALLOC(X))
Nicer than my initial thoughts on getting rid of the bit-wise loop. And
uses 64-bit math to produce a 32-bit answer, so there are no edge cases
where overflow could misbehave even though the intermediate steps may
require 33 bits. Works as long as X <= Y (should that be mentioned in
the contract?)
> +/* Subclusters X to Y (both included) read as zeroes */
> +#define QCOW_OFLAG_SUB_ZERO_RANGE(X, Y) \
> + (QCOW_OFLAG_SUB_ALLOC_RANGE(X, Y) << 32)
Also works (you do the math in the low 33 bits before shifting), again
if X <= Y.
> +/* L2 entry bitmap with all allocation bits set */
> +#define QCOW_L2_BITMAP_ALL_ALLOC (QCOW_OFLAG_SUB_ALLOC_RANGE(0, 31))
> +/* L2 entry bitmap with all "read as zeroes" bits set */
> +#define QCOW_L2_BITMAP_ALL_ZEROES (QCOW_OFLAG_SUB_ZERO_RANGE(0, 31))
More complicated than merely writing 0xffffffffULL and
(0xffffffffULL<<32), but the compiler will constant-fold it to the same
value, and it elegantly expresses the intent. I like it.
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
next prev parent reply other threads:[~2020-05-05 21:10 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-05 17:38 [PATCH v5 00/31] Add subcluster allocation to qcow2 Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 01/31] qcow2: Make Qcow2AioTask store the full host offset Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 02/31] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 03/31] qcow2: Add calculate_l2_meta() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 04/31] qcow2: Split cluster_needs_cow() out of count_cow_clusters() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 05/31] qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied() Alberto Garcia
2020-05-05 19:23 ` Eric Blake
2020-05-05 17:38 ` [PATCH v5 06/31] qcow2: Add get_l2_entry() and set_l2_entry() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 07/31] qcow2: Document the Extended L2 Entries feature Alberto Garcia
2020-05-05 19:35 ` Eric Blake
2020-05-06 15:02 ` Alberto Garcia
2020-05-06 15:24 ` Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 08/31] qcow2: Add dummy has_subclusters() function Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 09/31] qcow2: Add subcluster-related fields to BDRVQcow2State Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 10/31] qcow2: Add offset_to_sc_index() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 11/31] qcow2: Add offset_into_subcluster() and size_to_subclusters() Alberto Garcia
2020-05-05 19:42 ` Eric Blake
2020-05-06 10:18 ` Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 12/31] qcow2: Add l2_entry_size() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 13/31] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap() Alberto Garcia
2020-05-05 20:04 ` Eric Blake
2020-05-06 12:06 ` Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 14/31] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type() Alberto Garcia
2020-05-05 21:08 ` Eric Blake [this message]
2020-05-05 17:38 ` [PATCH v5 15/31] qcow2: Add qcow2_cluster_is_allocated() Alberto Garcia
2020-05-05 21:10 ` Eric Blake
2020-05-05 17:38 ` [PATCH v5 16/31] qcow2: Add cluster type parameter to qcow2_get_host_offset() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 17/31] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_* Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 18/31] qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 19/31] qcow2: Add subcluster support to calculate_l2_meta() Alberto Garcia
2020-05-05 21:59 ` Eric Blake
2020-05-06 17:14 ` Alberto Garcia
2020-05-06 17:39 ` Eric Blake
2020-05-07 15:34 ` Alberto Garcia
2020-05-07 15:50 ` Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 20/31] qcow2: Add subcluster support to qcow2_get_host_offset() Alberto Garcia
2020-05-06 17:55 ` Eric Blake
2020-05-08 11:44 ` Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 21/31] qcow2: Add subcluster support to zero_in_l2_slice() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 22/31] qcow2: Add subcluster support to discard_in_l2_slice() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 23/31] qcow2: Add subcluster support to check_refcounts_l2() Alberto Garcia
2020-05-06 17:58 ` Eric Blake
2020-05-05 17:38 ` [PATCH v5 24/31] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 25/31] qcow2: Clear the L2 bitmap when allocating a compressed cluster Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 26/31] qcow2: Add subcluster support to handle_alloc_space() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 27/31] qcow2: Add subcluster support to qcow2_co_pwrite_zeroes() Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 28/31] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit Alberto Garcia
2020-05-06 18:09 ` Eric Blake
2020-05-07 14:37 ` Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 29/31] qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters Alberto Garcia
2020-05-06 18:11 ` Eric Blake
2020-05-05 17:38 ` [PATCH v5 30/31] qcow2: Add subcluster support to qcow2_measure() Alberto Garcia
2020-05-06 18:13 ` Eric Blake
2020-05-07 15:16 ` Alberto Garcia
2020-05-05 17:38 ` [PATCH v5 31/31] iotests: Add tests for qcow2 images with extended L2 entries Alberto Garcia
2020-05-20 9:35 ` [PATCH v5 00/31] Add subcluster allocation to qcow2 Derek Su
2020-05-20 9:50 ` Alberto Garcia
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e6a65feb-e7f9-b006-8171-014e41f62395@redhat.com \
--to=eblake@redhat.com \
--cc=berto@igalia.com \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=vsementsov@virtuozzo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).