Re: [PATCH v5 07/10] block: introduce preallocate filter

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

24.08.2020 20:52, Vladimir Sementsov-Ogievskiy wrote:
> 21.08.2020 17:11, Vladimir Sementsov-Ogievskiy wrote:
>> It's intended to be inserted between format and protocol nodes to
>> preallocate additional space (expanding protocol file) on writes
>> crossing EOF. It improves performance for file-systems with slow
>> allocation.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>   docs/system/qemu-block-drivers.rst.inc |  26 +++
>>   qapi/block-core.json                   |  20 +-
>>   block/preallocate.c                    | 291 +++++++++++++++++++++++++
>>   block/Makefile.objs                    |   1 +
>>   4 files changed, 337 insertions(+), 1 deletion(-)
>>   create mode 100644 block/preallocate.c
>>
>> diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc
>> index b052a6d14e..5e8a35c571 100644
>> --- a/docs/system/qemu-block-drivers.rst.inc
>> +++ b/docs/system/qemu-block-drivers.rst.inc
>> @@ -952,3 +952,29 @@ on host and see if there are locks held by the QEMU process on the image file.
>>   More than one byte could be locked by the QEMU instance, each byte of which
>>   reflects a particular permission that is acquired or protected by the running
>>   block driver.
>> +
> 
> [..]
> 
>> +
>> +static coroutine_fn int preallocate_co_preadv_part(
>> +        BlockDriverState *bs, uint64_t offset, uint64_t bytes,
>> +        QEMUIOVector *qiov, size_t qiov_offset, int flags)
>> +{
>> +    return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset,
>> +                               flags);
>> +}
>> +
>> +static int coroutine_fn preallocate_co_pdiscard(BlockDriverState *bs,
>> +                                               int64_t offset, int bytes)
>> +{
>> +    return bdrv_co_pdiscard(bs->file, offset, bytes);
>> +}
>> +
>> +static bool coroutine_fn do_preallocate(BlockDriverState *bs, int64_t offset,
>> +                                       int64_t bytes, bool write_zero)
>> +{
>> +    BDRVPreallocateState *s = bs->opaque;
>> +    int64_t len, start, end;
> 
> int64_t old_data_end = s->data_end;
> 
>> +
>> +    if (!s->active) {
>> +        return false;
>> +    }
>> +
>> +    if (s->data_end >= 0) {
>> +        s->data_end = MAX(s->data_end,
>> +                          QEMU_ALIGN_UP(offset + bytes, BDRV_SECTOR_SIZE));
>> +    }
>> +
>> +    len = bdrv_getlength(bs->file->bs);
>> +    if (len < 0) {
>> +        return false;
> 
> return old_data_end >= 0 && offset >= old_data_end;
> 
> 
> And with this small improvement we make the following test 20% faster (ssd, ext4):
> 
> ./qemu-img create -f qcow2 $img 16G; ./qemu-img bench -c 150000 -d 64 -f qcow2  -s 16k -t none -n -w $img;
> 
> (assume additional patch which inserts the filter).
> 
> Great! So, preallocation is generally good idea, not only for vstorage.
> 
> What about inserting the filter by default?
> 
> I'll come tomorrow with more complete test results.
> 

Some results:

the two commands to compare:
img=/ssd/x.qcow2; for i in {1..5}; do ./qemu-img create -f qcow2 $img 16G; ./qemu-img bench -c 150000 -d 64 -s 16k -t none -n -w --image-opts driver=qcow2,file.filename=$img; done
img=/ssd/x.qcow2; for i in {1..5}; do ./qemu-img create -f qcow2 $img 16G; ./qemu-img bench -c 150000 -d 64 -s 16k -t none -n -w --image-opts driver=qcow2,file.driver=preallocate,file.file.filename=$img; done


       no-filter  with-filter
ssd
   ext4: 6.94s      5.53s (-20%)
   xfs:  6.9s       25s (+262%)
hdd (with -c 45000)
   ext4: 8.23s      12.75s (+55%)
   xfs:  7.79s      25.4s (+226%)

vstorage (some custom distributed fs), with -c 4000 over ext4 over ssd: 42.9s ~> 0.27s, more than 150 times faster with filter!
same, with -c 2000 over ext4 over hdd: 22.8s ~> 0.53s, ~43 times faster.


So, we do have large improvement for the vstorage (some custom distributed fs), which is our main target. And there is a significant improvement for ext4 over ssd. And (a lot more) significant degradation in other cases. For sure, we can't insert the filter by default, but it is useful.


-- 
Best regards,
Vladimir