From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: "qemu-block@nongnu.org" <qemu-block@nongnu.org>
Cc: "kwolf@redhat.com" <kwolf@redhat.com>,
Denis Lunev <den@virtuozzo.com>,
"armbru@redhat.com" <armbru@redhat.com>,
"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
"mreitz@redhat.com" <mreitz@redhat.com>,
"stefanha@redhat.com" <stefanha@redhat.com>
Subject: Re: [Qemu-devel] [PATCH v4 0/5] qcow2: async handling of fragmented io
Date: Tue, 3 Sep 2019 12:16:11 +0000 [thread overview]
Message-ID: <4d40a211-f926-09ec-3f52-9794124395e4@virtuozzo.com> (raw)
In-Reply-To: <20190816153015.447957-1-vsementsov@virtuozzo.com>
Pinging, as Stefan's branch merged into master and now these series based on master.
16.08.2019 18:30, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
>
> Here is an asynchronous scheme for handling fragmented qcow2
> reads and writes. Both qcow2 read and write functions loops through
> sequential portions of data. The series aim it to parallelize these
> loops iterations.
> It improves performance for fragmented qcow2 images, I've tested it
> as described below.
>
> v4 [perf results not updated]:
> 01: new patch. Unrelated, but need to fix 026 before the series to
> correctly fix it after :)
> 02: - use coroutine_fn where appropriate (i.e. in aio_task_pool_new too)
> - add Max's r-b
> 03,04: add Max's r-b
> 05: fix 026 output
>
> v3 (by Max's comments) [perf results not updated]:
>
> 01: - use coroutine_fn where appropriate !!!!!!!!!!!!!!!!!!!!!!!
> - add aio_task_pool_free
> - add some comments
> - move header to include/block
> - s/wait_done/waiting
> 02: - Rewrite note about decryption in guest buffers [thx to Eric]
> - separate g_assert_not_reached for QCOW2_CLUSTER_ZERO_*
> - drop return after g_assert_not_reached
> 03: - drop bytes_done and correctly use qiov_offset
> - fix comment
> 04: - move QCOW2_MAX_WORKERS to block/qcow2.h
> - initialize ret in qcow2_co_preadv_part
> Based-on: https://github.com/stefanha/qemu/commits/block
>
>
> v2: changed a lot, as
> 1. a lot of preparations around locks, hd_qiovs, threads for encryption
> are done
> 2. I decided to create separate file with async request handling API, to
> reuse it for backup, stream and copy-on-read to improve their performance
> too. Mirror and qemu-img convert has their own async request handling,
> may be we'll be able finally merge all these similar code into one
> feature.
> Note that not all API calls used in qcow2, some will be needed on
> following steps for parallelizing other io loops.
>
> About testing:
>
> I have four 4G qcow2 images (with default 64k block size) on my ssd disk:
> t-seq.qcow2 - sequentially written qcow2 image
> t-reverse.qcow2 - filled by writing 64k portions from end to the start
> t-rand.qcow2 - filled by writing 64k portions (aligned) in random order
> t-part-rand.qcow2 - filled by shuffling order of 64k writes in 1m clusters
> (see source code of image generation in the end for details)
>
> and I've done several runs like the following (sequential io by 1mb chunks):
>
> out=/tmp/block; echo > $out; cat /tmp/files | while read file; do for wr in {"","-w"}; do echo "$file" $wr; ./qemu-img bench -c 4096 -d 1 -f qcow2 -n -s 1m -t none $wr "$file" | grep 'Run completed in' | awk '{print $4}' >> $out; done; done
>
>
> short info about parameters:
> -w - do writes (otherwise do reads)
> -c - count of blocks
> -s - block size
> -t none - disable cache
> -n - native aio
> -d 1 - don't use parallel requests provided by qemu-img bench itself
>
> results:
>
> +---------------------------+---------+---------+
> | file | master | async |
> +---------------------------+---------+---------+
> | /ssd/t-part-rand.qcow2 | 14.671 | 9.193 |
> +---------------------------+---------+---------+
> | /ssd/t-part-rand.qcow2 -w | 11.434 | 8.621 |
> +---------------------------+---------+---------+
> | /ssd/t-rand.qcow2 | 20.421 | 10.05 |
> +---------------------------+---------+---------+
> | /ssd/t-rand.qcow2 -w | 11.097 | 8.915 |
> +---------------------------+---------+---------+
> | /ssd/t-reverse.qcow2 | 17.515 | 9.407 |
> +---------------------------+---------+---------+
> | /ssd/t-reverse.qcow2 -w | 11.255 | 8.649 |
> +---------------------------+---------+---------+
> | /ssd/t-seq.qcow2 | 9.081 | 9.072 |
> +---------------------------+---------+---------+
> | /ssd/t-seq.qcow2 -w | 8.761 | 8.747 |
> +---------------------------+---------+---------+
> | /tmp/t-part-rand.qcow2 | 41.179 | 41.37 |
> +---------------------------+---------+---------+
> | /tmp/t-part-rand.qcow2 -w | 54.097 | 55.323 |
> +---------------------------+---------+---------+
> | /tmp/t-rand.qcow2 | 711.899 | 514.339 |
> +---------------------------+---------+---------+
> | /tmp/t-rand.qcow2 -w | 546.259 | 642.114 |
> +---------------------------+---------+---------+
> | /tmp/t-reverse.qcow2 | 86.065 | 96.522 |
> +---------------------------+---------+---------+
> | /tmp/t-reverse.qcow2 -w | 46.557 | 48.499 |
> +---------------------------+---------+---------+
> | /tmp/t-seq.qcow2 | 33.804 | 33.862 |
> +---------------------------+---------+---------+
> | /tmp/t-seq.qcow2 -w | 34.299 | 34.233 |
> +---------------------------+---------+---------+
>
>
> Performance gain is obvious, especially for read and especially for ssd.
> For hdd there is a degradation for reverse case, but this is the most
> impossible case and seems not critical.
>
> How images are generated:
>
> ==== gen-writes ======
> #!/usr/bin/env python
> import random
> import sys
>
> size = 4 * 1024 * 1024 * 1024
> block = 64 * 1024
> block2 = 1024 * 1024
>
> arg = sys.argv[1]
>
> if arg in ('rand', 'reverse', 'seq'):
> writes = list(range(0, size, block))
>
> if arg == 'rand':
> random.shuffle(writes)
> elif arg == 'reverse':
> writes.reverse()
> elif arg == 'part-rand':
> writes = []
> for off in range(0, size, block2):
> wr = list(range(off, off + block2, block))
> random.shuffle(wr)
> writes.extend(wr)
> elif arg != 'seq':
> sys.exit(1)
>
> for w in writes:
> print 'write -P 0xff {} {}'.format(w, block)
>
> print 'q'
> ==========================
>
> ===== gen-test-images.sh =====
> #!/bin/bash
>
> IMG_PATH=/ssd
>
> for name in seq reverse rand part-rand; do
> IMG=$IMG_PATH/t-$name.qcow2
> echo createing $IMG ...
> rm -f $IMG
> qemu-img create -f qcow2 $IMG 4G
> gen-writes $name | qemu-io $IMG
> done
> ==============================
>
>
> Vladimir Sementsov-Ogievskiy (5):
> tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache
> block: introduce aio task pool
> block/qcow2: refactor qcow2_co_preadv_part
> block/qcow2: refactor qcow2_co_pwritev_part
> block/qcow2: introduce parallel subrequest handling in read and write
>
> qapi/block-core.json | 2 +-
> block/qcow2.h | 3 +
> include/block/aio_task.h | 54 ++++
> block/aio_task.c | 124 ++++++++
> block/qcow2.c | 461 +++++++++++++++++++----------
> block/Makefile.objs | 2 +
> block/trace-events | 1 +
> tests/qemu-iotests/026.out | 18 +-
> tests/qemu-iotests/026.out.nocache | 188 ++++++------
> 9 files changed, 591 insertions(+), 262 deletions(-)
> create mode 100644 include/block/aio_task.h
> create mode 100644 block/aio_task.c
>
--
Best regards,
Vladimir
next prev parent reply other threads:[~2019-09-03 12:18 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-16 15:30 [Qemu-devel] [PATCH v4 0/5] qcow2: async handling of fragmented io Vladimir Sementsov-Ogievskiy
2019-08-16 15:30 ` [Qemu-devel] [PATCH v4 1/5] tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache Vladimir Sementsov-Ogievskiy
2019-09-09 21:58 ` [Qemu-devel] [Qemu-block] " John Snow
2019-08-16 15:30 ` [Qemu-devel] [PATCH v4 2/5] block: introduce aio task pool Vladimir Sementsov-Ogievskiy
2019-08-16 15:30 ` [Qemu-devel] [PATCH v4 3/5] block/qcow2: refactor qcow2_co_preadv_part Vladimir Sementsov-Ogievskiy
2019-09-13 10:01 ` Kevin Wolf
2019-09-13 10:53 ` Vladimir Sementsov-Ogievskiy
2019-09-13 11:06 ` Max Reitz
2019-09-13 11:34 ` Kevin Wolf
2019-09-13 11:55 ` Max Reitz
2019-08-16 15:30 ` [Qemu-devel] [PATCH v4 4/5] block/qcow2: refactor qcow2_co_pwritev_part Vladimir Sementsov-Ogievskiy
2019-08-16 15:30 ` [Qemu-devel] [PATCH v4 5/5] block/qcow2: introduce parallel subrequest handling in read and write Vladimir Sementsov-Ogievskiy
2019-09-03 12:16 ` Vladimir Sementsov-Ogievskiy [this message]
2019-09-13 8:58 ` [Qemu-devel] [PATCH v4 0/5] qcow2: async handling of fragmented io Max Reitz
2019-09-13 9:11 ` Vladimir Sementsov-Ogievskiy
2019-09-16 13:26 ` Max Reitz
2019-09-16 15:29 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4d40a211-f926-09ec-3f52-9794124395e4@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=armbru@redhat.com \
--cc=den@virtuozzo.com \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).