From: Max Reitz <mreitz@redhat.com>
To: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>,
Eric Blake <eblake@redhat.com>,
qemu-devel@nongnu.org
Cc: david.edmondson@oracle.com, qemu-block@nongnu.org
Subject: Re: [PATCH 00/17] Improve qcow2 all-zero detection
Date: Wed, 5 Feb 2020 18:58:22 +0100 [thread overview]
Message-ID: <51829e78-30d7-f0c6-f191-fc12c8cb37f4@redhat.com> (raw)
In-Reply-To: <d5bed92b-0351-7ae7-6b18-f814fe1533ba@virtuozzo.com>
[-- Attachment #1.1: Type: text/plain, Size: 3269 bytes --]
On 05.02.20 16:14, Vladimir Sementsov-Ogievskiy wrote:
> 05.02.2020 17:47, Vladimir Sementsov-Ogievskiy wrote:
>> 05.02.2020 17:26, Eric Blake wrote:
>>> On 2/5/20 3:25 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>
>>>>> 3. For qcow2
>>>>> Hmm. Here, as I understand, than main case is freshly created qcow2,
>>>>> which is fully-unallocated. To understand that it is empty, we
>>>>> need only to check all L1 entries. And for empty L1 table it is fast.
>>>>> So we don't need any qcow2 format improvement to check it.
>>>>>
>>>>
>>>> Ah yes, I forget about preallocated case. Hmm. For preallocated
>>>> clusters,
>>>> we have zero bits in L2 entries. And with them, we even don't need
>>>> preallocated to be filled by zeros, as we never read them (but just
>>>> return
>>>> zeros on read)..
>>>
>>> Scanning all L2 entries is O(n), while an autoclear bit properly
>>> maintained is O(1).
>>>
>>>>
>>>> Then, may be we want similar flag for L1 entry (this will enable large
>>>> fast write-zero). And may be we want flag which marks the whole image
>>>> as read-zero (it's your flag). So, now I think, my previous idea
>>>> of "all allocated is zero" is worse. As for fully-preallocated images
>>>> we are sure that all clusters are allocated, and it is more native to
>>>> have flags similar to ZERO bit in L2 entry.
>>>
>>> Right now, we don't have any L1 entry flags. Adding one would
>>> require adding an incompatible feature flag (if older qemu would
>>> choke to see unexpected flags in an L1 entry), or at best an
>>> autoclear feature flag (if the autoclear bit gets cleared because an
>>> older qemu opened the image and couldn't maintain L1 entry flags
>>> correctly, then newer qemu knows it cannot trust those L1 entry
>>> flags). But as soon as you are talking about adding a feature bit,
>>> then why add one that still requires O(n) traversal to check (true,
>>> the 'n' in an O(n) traversal of L1 tables is much smaller than the
>>> 'n' in an O(n) traversal of L2 tables), when you can instead just add
>>> an O(1) autoclear bit that maintains all_zero status for the image as
>>> a whole?
>>>
>>
>> My suggestion about L1 entry flag is side thing, I understand
>> difference between O(n) and O(1) :) Still additional L1 entry will
>> help to make efficient large block-status and write-zero requests.
>>
>> And I agree that we need top level flag.. I just try to say, that it
>> seems good to make it similar with existing L2 flag. But yes, it would
>> be incomaptible change, as it marks all clusters as ZERO, and older
>> Qemu can't understand it and may treat all clusters as unallocated.
>>
>
> Still, how long is this O(n) ? We load the whole L1 into memory anyway.
> For example, 16Tb disk with 64K granularity, we'll have 32768 L1
> entries. Will we get sensible performance benefit with an extension? I
> doubt in it now. And anyway, if we have an extension, we should fallback
> to this O(n) if we don't have the flag set.
(Sorry, it’s late and I haven’t followed this particular conversation
too closely, but:)
Keep in mind that the default metadata overlap protection mode causes
all L1 entries to be scanned on each I/O write. So it can’t be that bad.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
next prev parent reply other threads:[~2020-02-05 17:59 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-31 17:44 [PATCH 00/17] Improve qcow2 all-zero detection Eric Blake
2020-01-31 17:44 ` [PATCH 01/17] qcow2: Comment typo fixes Eric Blake
2020-02-04 14:12 ` Vladimir Sementsov-Ogievskiy
2020-02-09 19:34 ` Alberto Garcia
2020-01-31 17:44 ` [PATCH 02/17] qcow2: List autoclear bit names in header Eric Blake
2020-02-04 14:26 ` Vladimir Sementsov-Ogievskiy
2020-01-31 17:44 ` [PATCH 03/17] qcow2: Avoid feature name extension on small cluster size Eric Blake
2020-02-04 14:39 ` Vladimir Sementsov-Ogievskiy
2020-02-09 19:28 ` Alberto Garcia
2020-01-31 17:44 ` [PATCH 04/17] block: Improve documentation of .bdrv_has_zero_init Eric Blake
2020-02-04 15:03 ` Vladimir Sementsov-Ogievskiy
2020-02-04 15:16 ` Eric Blake
2020-01-31 17:44 ` [PATCH 05/17] block: Don't advertise zero_init_truncate with encryption Eric Blake
2020-02-10 18:12 ` Alberto Garcia
2020-01-31 17:44 ` [PATCH 06/17] block: Improve bdrv_has_zero_init_truncate with backing file Eric Blake
2020-02-10 18:13 ` Alberto Garcia
2020-01-31 17:44 ` [PATCH 07/17] gluster: Drop useless has_zero_init callback Eric Blake
2020-02-04 15:06 ` Vladimir Sementsov-Ogievskiy
2020-02-10 18:21 ` Alberto Garcia
2020-02-17 8:06 ` [GEDI] " Niels de Vos
2020-02-17 12:03 ` Eric Blake
2020-02-17 12:22 ` Eric Blake
2020-02-17 14:01 ` Niels de Vos
2020-01-31 17:44 ` [PATCH 08/17] sheepdog: Consistently set bdrv_has_zero_init_truncate Eric Blake
2020-02-04 15:09 ` Vladimir Sementsov-Ogievskiy
2020-01-31 17:44 ` [PATCH 09/17] block: Refactor bdrv_has_zero_init{,_truncate} Eric Blake
2020-02-04 15:35 ` Vladimir Sementsov-Ogievskiy
2020-02-04 15:49 ` Eric Blake
2020-02-04 16:07 ` Vladimir Sementsov-Ogievskiy
2020-02-04 17:42 ` Max Reitz
2020-02-04 17:51 ` Eric Blake
2020-02-05 16:43 ` Max Reitz
2020-02-05 7:51 ` Vladimir Sementsov-Ogievskiy
2020-02-05 14:07 ` Eric Blake
2020-02-05 14:25 ` Vladimir Sementsov-Ogievskiy
2020-02-05 14:36 ` Eric Blake
2020-02-05 17:55 ` Max Reitz
2020-02-04 17:53 ` Max Reitz
2020-02-04 19:03 ` Eric Blake
2020-02-05 17:22 ` Max Reitz
2020-02-05 18:39 ` Eric Blake
2020-02-06 9:18 ` Max Reitz
2020-01-31 17:44 ` [PATCH 10/17] block: Add new BDRV_ZERO_OPEN flag Eric Blake
2020-01-31 18:03 ` Eric Blake
2020-02-04 17:34 ` Max Reitz
2020-02-04 17:50 ` Eric Blake
2020-02-05 8:39 ` Vladimir Sementsov-Ogievskiy
2020-02-05 17:26 ` Max Reitz
2020-01-31 17:44 ` [PATCH 11/17] file-posix: Support BDRV_ZERO_OPEN Eric Blake
2020-01-31 17:44 ` [PATCH 12/17] gluster: " Eric Blake
2020-02-17 8:16 ` [GEDI] " Niels de Vos
2020-01-31 17:44 ` [PATCH 13/17] qcow2: Add new autoclear feature for all zero image Eric Blake
2020-02-03 17:45 ` Vladimir Sementsov-Ogievskiy
2020-02-04 13:12 ` Eric Blake
2020-02-04 13:29 ` Vladimir Sementsov-Ogievskiy
2020-01-31 17:44 ` [PATCH 14/17] qcow2: Expose all zero bit through .bdrv_known_zeroes Eric Blake
2020-01-31 17:44 ` [PATCH 15/17] qcow2: Implement all-zero autoclear bit Eric Blake
2020-01-31 17:44 ` [PATCH 16/17] iotests: Add new test for qcow2 all-zero bit Eric Blake
2020-01-31 17:44 ` [PATCH 17/17] qcow2: Let qemu-img check cover " Eric Blake
2020-02-04 17:32 ` [PATCH 00/17] Improve qcow2 all-zero detection Max Reitz
2020-02-04 18:53 ` Eric Blake
2020-02-05 17:04 ` Max Reitz
2020-02-05 19:21 ` Eric Blake
2020-02-06 9:12 ` Max Reitz
2020-02-05 9:04 ` Vladimir Sementsov-Ogievskiy
2020-02-05 9:25 ` Vladimir Sementsov-Ogievskiy
2020-02-05 14:26 ` Eric Blake
2020-02-05 14:47 ` Vladimir Sementsov-Ogievskiy
2020-02-05 15:14 ` Vladimir Sementsov-Ogievskiy
2020-02-05 17:58 ` Max Reitz [this message]
2020-02-05 14:22 ` Eric Blake
2020-02-05 14:43 ` Vladimir Sementsov-Ogievskiy
2020-02-05 14:58 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51829e78-30d7-f0c6-f191-fc12c8cb37f4@redhat.com \
--to=mreitz@redhat.com \
--cc=david.edmondson@oracle.com \
--cc=eblake@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=vsementsov@virtuozzo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).