From: Eric Blake <eblake@redhat.com>
To: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>,
qemu-devel@nongnu.org
Cc: david.edmondson@oracle.com, qemu-block@nongnu.org, mreitz@redhat.com
Subject: Re: [PATCH 00/17] Improve qcow2 all-zero detection
Date: Wed, 5 Feb 2020 08:22:29 -0600 [thread overview]
Message-ID: <9f2d8730-23ca-f070-52dd-0cbcbb08913b@redhat.com> (raw)
In-Reply-To: <3bad82d4-4d60-4341-d87e-af37e1dd680e@virtuozzo.com>
On 2/5/20 3:04 AM, Vladimir Sementsov-Ogievskiy wrote:
>> [repo.or.cz appears to be down as I type this; I'll post a link to a
>> repository later when it comes back up]
Now up
https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/qcow2-all-zero-v1
>>
>
> I have several ideas around it.
>
> 1. For generic block layer.
> Did you consider as alternative to BDRV_ZEO_OPEN, to export the
> information through normal block_status? So, if we have the
> information, that disk is all-zero, we can always add _ZERO
> flag to block-status result.
Makes sense.
> And in generic bdrv_is_all_zeroes(),
> we can just call block_status(0, disk_size), which will return
> ZERO and n=disk_size if driver supports all-zero feature and is
> all-zero now.
Less obvious. block_status is not required to visit the entire disk,
even if the entire disk is all zero. For example, qcow2 visits at most
one L2 page in a call (if the request spans L1 entries, it will be
truncated at the boundary, even if the region before and after the
boundary have the same status). I'm also worried if we still have
32-bit limitations in block_status (ideally, we've fixed things to
support 64-bit status where possible, but I'm not counting on it).
> I think block-status is a native way for such information, and I
> think that we anyway want to come to support of 64bit block-status
> for qcow2 and nbd.
Block status requires an O(n) loop over the disk, where n is the number
of distinct extents possible. If you get lucky, and
block_status(0,size) returns a single extent, then yes that can feed the
'is_zeroes' request. Similarly, a single return of non-zero data can
instantly tell you that 'is_zeroes' is false. But given that drivers
may break up their response on convenient boundaries, such as qcow2 on
L1 entry granularity, you cannot blindly assume that a return of zero
data for smaller than the requested size implies non-zero data, only
that there is insufficient information to tell if the disk is all_zeroes
without querying further block_status calls, and that's where you lose
out on the speed compared to just being told up-front from an 'is_zero'
call.
>
> 2. For NBD
> Again, possible alternative is BLOCK_STATUS, but we need 64bit
> commands for it. I plan to send a proposal anyway. Still, nothing
> bad in two possible path of receiving all-zero information.
> And even with your NBD extension, we can export this information
> through block-status [1.]
Yes, having 64-bit BLOCK_STATUS in NBD is orthogonal to this, but both
ideas are independently useful, and as the level of difficulty in
implementing things may vary, it is conceivable to have both a server
that provides 'is_zero' but not BLOCK_STATUS, and a server that provides
64-bit BLOCK_STATUS but not 'is_zero'.
>
> 3. For qcow2
> Hmm. Here, as I understand, than main case is freshly created qcow2,
> which is fully-unallocated. To understand that it is empty, we
> need only to check all L1 entries. And for empty L1 table it is fast.
> So we don't need any qcow2 format improvement to check it.
The benefit of this patch series is that it detects preallocated qcow2
images as all_zero. What's more, scanning all L1 entries is O(n), but
detecting an autoclear all_zero bit is O(1). Your proposed L1 scan is
accurate for fewer cases, and costs more time.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
next prev parent reply other threads:[~2020-02-05 14:24 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-31 17:44 [PATCH 00/17] Improve qcow2 all-zero detection Eric Blake
2020-01-31 17:44 ` [PATCH 01/17] qcow2: Comment typo fixes Eric Blake
2020-02-04 14:12 ` Vladimir Sementsov-Ogievskiy
2020-02-09 19:34 ` Alberto Garcia
2020-01-31 17:44 ` [PATCH 02/17] qcow2: List autoclear bit names in header Eric Blake
2020-02-04 14:26 ` Vladimir Sementsov-Ogievskiy
2020-01-31 17:44 ` [PATCH 03/17] qcow2: Avoid feature name extension on small cluster size Eric Blake
2020-02-04 14:39 ` Vladimir Sementsov-Ogievskiy
2020-02-09 19:28 ` Alberto Garcia
2020-01-31 17:44 ` [PATCH 04/17] block: Improve documentation of .bdrv_has_zero_init Eric Blake
2020-02-04 15:03 ` Vladimir Sementsov-Ogievskiy
2020-02-04 15:16 ` Eric Blake
2020-01-31 17:44 ` [PATCH 05/17] block: Don't advertise zero_init_truncate with encryption Eric Blake
2020-02-10 18:12 ` Alberto Garcia
2020-01-31 17:44 ` [PATCH 06/17] block: Improve bdrv_has_zero_init_truncate with backing file Eric Blake
2020-02-10 18:13 ` Alberto Garcia
2020-01-31 17:44 ` [PATCH 07/17] gluster: Drop useless has_zero_init callback Eric Blake
2020-02-04 15:06 ` Vladimir Sementsov-Ogievskiy
2020-02-10 18:21 ` Alberto Garcia
2020-02-17 8:06 ` [GEDI] " Niels de Vos
2020-02-17 12:03 ` Eric Blake
2020-02-17 12:22 ` Eric Blake
2020-02-17 14:01 ` Niels de Vos
2020-01-31 17:44 ` [PATCH 08/17] sheepdog: Consistently set bdrv_has_zero_init_truncate Eric Blake
2020-02-04 15:09 ` Vladimir Sementsov-Ogievskiy
2020-01-31 17:44 ` [PATCH 09/17] block: Refactor bdrv_has_zero_init{,_truncate} Eric Blake
2020-02-04 15:35 ` Vladimir Sementsov-Ogievskiy
2020-02-04 15:49 ` Eric Blake
2020-02-04 16:07 ` Vladimir Sementsov-Ogievskiy
2020-02-04 17:42 ` Max Reitz
2020-02-04 17:51 ` Eric Blake
2020-02-05 16:43 ` Max Reitz
2020-02-05 7:51 ` Vladimir Sementsov-Ogievskiy
2020-02-05 14:07 ` Eric Blake
2020-02-05 14:25 ` Vladimir Sementsov-Ogievskiy
2020-02-05 14:36 ` Eric Blake
2020-02-05 17:55 ` Max Reitz
2020-02-04 17:53 ` Max Reitz
2020-02-04 19:03 ` Eric Blake
2020-02-05 17:22 ` Max Reitz
2020-02-05 18:39 ` Eric Blake
2020-02-06 9:18 ` Max Reitz
2020-01-31 17:44 ` [PATCH 10/17] block: Add new BDRV_ZERO_OPEN flag Eric Blake
2020-01-31 18:03 ` Eric Blake
2020-02-04 17:34 ` Max Reitz
2020-02-04 17:50 ` Eric Blake
2020-02-05 8:39 ` Vladimir Sementsov-Ogievskiy
2020-02-05 17:26 ` Max Reitz
2020-01-31 17:44 ` [PATCH 11/17] file-posix: Support BDRV_ZERO_OPEN Eric Blake
2020-01-31 17:44 ` [PATCH 12/17] gluster: " Eric Blake
2020-02-17 8:16 ` [GEDI] " Niels de Vos
2020-01-31 17:44 ` [PATCH 13/17] qcow2: Add new autoclear feature for all zero image Eric Blake
2020-02-03 17:45 ` Vladimir Sementsov-Ogievskiy
2020-02-04 13:12 ` Eric Blake
2020-02-04 13:29 ` Vladimir Sementsov-Ogievskiy
2020-01-31 17:44 ` [PATCH 14/17] qcow2: Expose all zero bit through .bdrv_known_zeroes Eric Blake
2020-01-31 17:44 ` [PATCH 15/17] qcow2: Implement all-zero autoclear bit Eric Blake
2020-01-31 17:44 ` [PATCH 16/17] iotests: Add new test for qcow2 all-zero bit Eric Blake
2020-01-31 17:44 ` [PATCH 17/17] qcow2: Let qemu-img check cover " Eric Blake
2020-02-04 17:32 ` [PATCH 00/17] Improve qcow2 all-zero detection Max Reitz
2020-02-04 18:53 ` Eric Blake
2020-02-05 17:04 ` Max Reitz
2020-02-05 19:21 ` Eric Blake
2020-02-06 9:12 ` Max Reitz
2020-02-05 9:04 ` Vladimir Sementsov-Ogievskiy
2020-02-05 9:25 ` Vladimir Sementsov-Ogievskiy
2020-02-05 14:26 ` Eric Blake
2020-02-05 14:47 ` Vladimir Sementsov-Ogievskiy
2020-02-05 15:14 ` Vladimir Sementsov-Ogievskiy
2020-02-05 17:58 ` Max Reitz
2020-02-05 14:22 ` Eric Blake [this message]
2020-02-05 14:43 ` Vladimir Sementsov-Ogievskiy
2020-02-05 14:58 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9f2d8730-23ca-f070-52dd-0cbcbb08913b@redhat.com \
--to=eblake@redhat.com \
--cc=david.edmondson@oracle.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=vsementsov@virtuozzo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).