From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: Stefan Hajnoczi <stefanha@redhat.com>
Cc: "fam@euphon.net" <fam@euphon.net>,
"kwolf@redhat.com" <kwolf@redhat.com>,
Denis Lunev <den@virtuozzo.com>,
"qemu-block@nongnu.org" <qemu-block@nongnu.org>,
Stefan Hajnoczi <stefanha@gmail.com>,
"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
"mreitz@redhat.com" <mreitz@redhat.com>,
"jsnow@redhat.com" <jsnow@redhat.com>
Subject: Re: [Qemu-devel] [PATCH v2 00/12] block: qiov_offset parameter for io
Date: Sat, 24 Aug 2019 10:15:35 +0000 [thread overview]
Message-ID: <378a87ca-8b65-02c3-cb5d-626ebf9860c1@virtuozzo.com> (raw)
In-Reply-To: <20190823162107.GC27586@stefanha-x1.localdomain>
23.08.2019 19:21, Stefan Hajnoczi wrote:
> On Thu, Aug 22, 2019 at 05:59:43PM +0000, Vladimir Sementsov-Ogievskiy wrote:
>> 22.08.2019 20:39, Vladimir Sementsov-Ogievskiy wrote:
>>> 22.08.2019 20:24, Vladimir Sementsov-Ogievskiy wrote:
>>>> 22.08.2019 18:50, Stefan Hajnoczi wrote:
>>>>> On Tue, Jun 04, 2019 at 07:15:02PM +0300, Vladimir Sementsov-Ogievskiy wrote:
>>>>>> Hi all!
>>>>>>
>>>>>> Here is new parameter qiov_offset for io path, to avoid
>>>>>> a lot of places with same pattern of creating local_qiov or hd_qiov
>>>>>> variables.
>>>>>>
>>>>>> These series also includes my
>>>>>> "[Qemu-devel] [PATCH 0/2] block/io: refactor padding"
>>>>>> with some changes [described in 01 and 03 emails]
>>>>>>
>>>>>> Vladimir Sementsov-Ogievskiy (12):
>>>>>> util/iov: introduce qemu_iovec_init_extended
>>>>>> util/iov: improve qemu_iovec_is_zero
>>>>>> block/io: refactor padding
>>>>>> block: define .*_part io handlers in BlockDriver
>>>>>> block/io: bdrv_co_do_copy_on_readv: use and support qiov_offset
>>>>>> block/io: bdrv_co_do_copy_on_readv: lazy allocation
>>>>>> block/io: bdrv_aligned_preadv: use and support qiov_offset
>>>>>> block/io: bdrv_aligned_pwritev: use and support qiov_offset
>>>>>> block/io: introduce bdrv_co_p{read,write}v_part
>>>>>> block/qcow2: refactor qcow2_co_preadv to use buffer-based io
>>>>>> block/qcow2: implement .bdrv_co_preadv_part
>>>>>> block/qcow2: implement .bdrv_co_pwritev(_compressed)_part
>>>>>>
>>>>>> block/qcow2.h | 1 +
>>>>>> include/block/block_int.h | 21 ++
>>>>>> include/qemu/iov.h | 10 +-
>>>>>> block/backup.c | 2 +-
>>>>>> block/io.c | 532 ++++++++++++++++++++++----------------
>>>>>> block/qcow2-cluster.c | 14 +-
>>>>>> block/qcow2.c | 131 +++++-----
>>>>>> qemu-img.c | 4 +-
>>>>>> util/iov.c | 153 +++++++++--
>>>>>> 9 files changed, 559 insertions(+), 309 deletions(-)
>>>>>
>>>>> qemu-iotests 077 fails after I apply this series (including your
>>>>> uninitialized variable fix). I'm afraid I can't include it in the block
>>>>> pull request, sorry!
>>>>>
>>>>> Stefan
>>>>>
>>>>
>>>> Hmm, 77 don't work on master for me:
>>>> 077 fail [20:23:57] [20:23:59] output mismatch (see 077.out.bad)
>>>> --- /work/src/qemu/up-block-drop-hd-qiov/tests/qemu-iotests/077.out 2019-04-22 15:06:56.162045432 +0300
>>>> +++ /work/src/qemu/up-block-drop-hd-qiov/tests/qemu-iotests/077.out.bad 2019-08-22 20:23:59.124122307 +0300
>>>> @@ -1,7 +1,15 @@
>>>> QA output created by 077
>>>> +==117186==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
>>>> +==117186==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffefc071000; bottom 0x7fad7277b000; size: 0x0051898f6000 (350200225792)
>>>> +False positive error reports may follow
>>>> +For details see http://code.google.com/p/address-sanitizer/issues/detail?id=189
>>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
>>>>
>>>> == Some concurrent requests involving RMW ==
>>>> +==117197==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
>>>> +==117197==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fffa4fcc000; bottom 0x7fa93a2c1000; size: 0x00566ad0b000 (371159248896)
>>>> +False positive error reports may follow
>>>> +For details see http://code.google.com/p/address-sanitizer/issues/detail?id=189
>>>> wrote XXX/XXX bytes at offset XXX
>>>> XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>>> wrote XXX/XXX bytes at offset XXX
>>>> @@ -66,6 +74,10 @@
>>>> XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>>>
>>>> == Verify image content ==
>>>> +==117219==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
>>>> +==117219==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7ffc722de000; bottom 0x7f0848232000; size: 0x00f42a0ac000 (1048677367808)
>>>> +False positive error reports may follow
>>>> +For details see http://code.google.com/p/address-sanitizer/issues/detail?id=189
>>>> read 512/512 bytes at offset 0
>>>> 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>>> read 512/512 bytes at offset 512
>>>> @@ -156,5 +168,9 @@
>>>> 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>>> read 2048/2048 bytes at offset 71680
>>>> 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>>> +==117229==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
>>>> +==117229==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 0x7fffa3342000; bottom 0x7fd85a275000; size: 0x0027490cd000 (168729300992)
>>>> +False positive error reports may follow
>>>> +For details see http://code.google.com/p/address-sanitizer/issues/detail?id=189
>>>> No errors were found on the image.
>>>> *** done
>>>> Failures: 077
>>>> Failed 1 of 1 tests
>>>>
>>>>
>>>>
>>>
>>> But after "block/io: refactor padding" it hangs instead of this fail.. This is not good
>>>
>>>
>>
>> Aha seems it's because 77 has "break pwritev_rmw_after_tail" which may not fire if we have merge_reads = true.
>>
>> So, for me the following fix for 03 helps (77 fails, but not hangs, so same behavior as on master):
>>
>> diff --git a/block/io.c b/block/io.c
>> index 6448f1c503..04e69400d8 100644
>> --- a/block/io.c
>> +++ b/block/io.c
>> @@ -1493,13 +1493,23 @@ static int bdrv_padding_rmw_read(BdrvChild *child,
>>
>> qemu_iovec_init_buf(&local_qiov, pad->buf, bytes);
>>
>> - bdrv_debug_event(bs, BLKDBG_PWRITEV_RMW_HEAD);
>> + if (pad->head) {
>> + bdrv_debug_event(bs, BLKDBG_PWRITEV_RMW_HEAD);
>> + }
>> + if (pad->merge_reads && pad->tail) {
>> + bdrv_debug_event(bs, BLKDBG_PWRITEV_RMW_TAIL);
>> + }
>> ret = bdrv_aligned_preadv(child, req, req->overlap_offset, bytes,
>> align, &local_qiov, 0);
>> if (ret < 0) {
>> return ret;
>> }
>> - bdrv_debug_event(bs, BLKDBG_PWRITEV_RMW_AFTER_HEAD);
>> + if (pad->head) {
>> + bdrv_debug_event(bs, BLKDBG_PWRITEV_RMW_AFTER_HEAD);
>> + }
>> + if (pad->merge_reads && pad->tail) {
>> + bdrv_debug_event(bs, BLKDBG_PWRITEV_RMW_AFTER_TAIL);
>> + }
>>
>>
>> Does it work for you?
>
> Yes, it works as expected now, thanks. I've squashed in your fix and
> will send these patches in the next block pull request.
>
Thank you!
--
Best regards,
Vladimir
prev parent reply other threads:[~2019-08-24 10:16 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-04 16:15 [Qemu-devel] [PATCH v2 00/12] block: qiov_offset parameter for io Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 01/12] util/iov: introduce qemu_iovec_init_extended Vladimir Sementsov-Ogievskiy
2019-06-05 11:54 ` Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 02/12] util/iov: improve qemu_iovec_is_zero Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 03/12] block/io: refactor padding Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 04/12] block: define .*_part io handlers in BlockDriver Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 05/12] block/io: bdrv_co_do_copy_on_readv: use and support qiov_offset Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 06/12] block/io: bdrv_co_do_copy_on_readv: lazy allocation Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 07/12] block/io: bdrv_aligned_preadv: use and support qiov_offset Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 08/12] block/io: bdrv_aligned_pwritev: " Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 09/12] block/io: introduce bdrv_co_p{read, write}v_part Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 10/12] block/qcow2: refactor qcow2_co_preadv to use buffer-based io Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 11/12] block/qcow2: implement .bdrv_co_preadv_part Vladimir Sementsov-Ogievskiy
2019-06-04 16:15 ` [Qemu-devel] [PATCH v2 12/12] block/qcow2: implement .bdrv_co_pwritev(_compressed)_part Vladimir Sementsov-Ogievskiy
2019-06-04 17:48 ` [Qemu-devel] [PATCH v2 00/12] block: qiov_offset parameter for io no-reply
2019-06-28 8:43 ` Stefan Hajnoczi
2019-07-25 8:28 ` Vladimir Sementsov-Ogievskiy
2019-07-29 15:24 ` Stefan Hajnoczi
2019-07-29 15:34 ` Vladimir Sementsov-Ogievskiy
2019-08-15 11:12 ` Vladimir Sementsov-Ogievskiy
2019-08-22 14:48 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2019-08-22 15:50 ` [Qemu-devel] " Stefan Hajnoczi
2019-08-22 17:24 ` Vladimir Sementsov-Ogievskiy
2019-08-22 17:39 ` Vladimir Sementsov-Ogievskiy
2019-08-22 17:59 ` Vladimir Sementsov-Ogievskiy
2019-08-23 16:21 ` Stefan Hajnoczi
2019-08-24 10:15 ` Vladimir Sementsov-Ogievskiy [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=378a87ca-8b65-02c3-cb5d-626ebf9860c1@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=den@virtuozzo.com \
--cc=fam@euphon.net \
--cc=jsnow@redhat.com \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@gmail.com \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).