* [PATCH 0/4] block: copy-before-write: cbw-timeout
@ 2022-03-02 16:24 Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 1/4] util: add qemu-co-timeout Vladimir Sementsov-Ogievskiy
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2022-03-02 16:24 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, stefanha, hreitz, kwolf, vsementsov, jsnow
Hi all!
There is another new option, that allows to limit cbw operation by some
timeout.
So, for example, using cbw-timeout=60 and on-cbw-error=break-snapshot
you can be sure that guest write will not stuck for more than 60
seconds.
Based-on: <20220301205929.2006041-1-vsementsov@virtuozzo.com>
([PATCH 0/3] block: copy-before-write: on-cbw-error behavior)
Vladimir Sementsov-Ogievskiy (4):
util: add qemu-co-timeout
block/block-copy: block_copy(): add timeout_ns parameter
block/copy-before-write: implement cbw-timeout option
iotests: copy-before-write: add cases for cbw-timeout option
qapi/block-core.json | 5 +-
include/block/block-copy.h | 2 +-
include/qemu/coroutine.h | 13 +++
block/block-copy.c | 28 ++++--
block/copy-before-write.c | 6 +-
util/qemu-co-timeout.c | 89 +++++++++++++++++++
tests/qemu-iotests/tests/copy-before-write | 78 ++++++++++++++++
.../qemu-iotests/tests/copy-before-write.out | 4 +-
util/meson.build | 1 +
9 files changed, 213 insertions(+), 13 deletions(-)
create mode 100644 util/qemu-co-timeout.c
--
2.31.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/4] util: add qemu-co-timeout
2022-03-02 16:24 [PATCH 0/4] block: copy-before-write: cbw-timeout Vladimir Sementsov-Ogievskiy
@ 2022-03-02 16:24 ` Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 2/4] block/block-copy: block_copy(): add timeout_ns parameter Vladimir Sementsov-Ogievskiy
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2022-03-02 16:24 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, stefanha, hreitz, kwolf, vsementsov, jsnow
Add new API, to make a time limited call of the coroutine.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/qemu/coroutine.h | 13 ++++++
util/qemu-co-timeout.c | 89 ++++++++++++++++++++++++++++++++++++++++
util/meson.build | 1 +
3 files changed, 103 insertions(+)
create mode 100644 util/qemu-co-timeout.c
diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h
index c828a95ee0..8704b05da8 100644
--- a/include/qemu/coroutine.h
+++ b/include/qemu/coroutine.h
@@ -316,6 +316,19 @@ static inline void coroutine_fn qemu_co_sleep_ns(QEMUClockType type, int64_t ns)
qemu_co_sleep_ns_wakeable(&w, type, ns);
}
+typedef void CleanupFunc(void *opaque);
+/**
+ * Run entry in a coroutine and start timer. Wait for entry to finish or for
+ * timer to elapse, what happen first. If entry finished, return 0, if timer
+ * elapsed earlier, return -ETIMEDOUT.
+ *
+ * Be careful, entry execution is not canceled, user should handle it somehow.
+ * If @clean is provided, it's called after coroutine finish if timeout
+ * happened.
+ */
+int coroutine_fn qemu_co_timeout(CoroutineEntry *entry, void *opaque,
+ uint64_t timeout_ns, CleanupFunc clean);
+
/**
* Wake a coroutine if it is sleeping in qemu_co_sleep_ns. The timer will be
* deleted. @sleep_state must be the variable whose address was given to
diff --git a/util/qemu-co-timeout.c b/util/qemu-co-timeout.c
new file mode 100644
index 0000000000..00cd335649
--- /dev/null
+++ b/util/qemu-co-timeout.c
@@ -0,0 +1,89 @@
+/*
+ * Helper functionality for distributing a fixed total amount of
+ * an abstract resource among multiple coroutines.
+ *
+ * Copyright (c) 2022 Virtuozzo International GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/coroutine.h"
+#include "block/aio.h"
+
+typedef struct QemuCoTimeoutState {
+ CoroutineEntry *entry;
+ void *opaque;
+ QemuCoSleep sleep_state;
+ bool marker;
+ CleanupFunc *clean;
+} QemuCoTimeoutState;
+
+static void coroutine_fn qemu_co_timeout_entry(void *opaque)
+{
+ QemuCoTimeoutState *s = opaque;
+
+ s->entry(s->opaque);
+
+ if (s->marker) {
+ assert(!s->sleep_state.to_wake);
+ /* .marker set by qemu_co_timeout, it have been failed */
+ if (s->clean) {
+ s->clean(s->opaque);
+ }
+ g_free(s);
+ } else {
+ s->marker = true;
+ qemu_co_sleep_wake(&s->sleep_state);
+ }
+}
+
+int coroutine_fn qemu_co_timeout(CoroutineEntry *entry, void *opaque,
+ uint64_t timeout_ns, CleanupFunc clean)
+{
+ QemuCoTimeoutState *s;
+ Coroutine *co;
+
+ if (timeout_ns == 0) {
+ entry(opaque);
+ return 0;
+ }
+
+ s = g_new(QemuCoTimeoutState, 1);
+ *s = (QemuCoTimeoutState) {
+ .entry = entry,
+ .opaque = opaque,
+ .clean = clean
+ };
+
+ co = qemu_coroutine_create(qemu_co_timeout_entry, s);
+
+ aio_co_enter(qemu_get_current_aio_context(), co);
+ qemu_co_sleep_ns_wakeable(&s->sleep_state, QEMU_CLOCK_REALTIME, timeout_ns);
+
+ if (s->marker) {
+ /* .marker set by qemu_co_timeout_entry, success */
+ g_free(s);
+ return 0;
+ }
+
+ /* Don't free s, as we can't cancel qemu_co_timeout_entry execution */
+ s->marker = true;
+ return -ETIMEDOUT;
+}
diff --git a/util/meson.build b/util/meson.build
index 3736988b9f..44392e89a4 100644
--- a/util/meson.build
+++ b/util/meson.build
@@ -82,6 +82,7 @@ if have_block
util_ss.add(files('block-helpers.c'))
util_ss.add(files('qemu-coroutine-sleep.c'))
util_ss.add(files('qemu-co-shared-resource.c'))
+ util_ss.add(files('qemu-co-timeout.c'))
util_ss.add(files('thread-pool.c', 'qemu-timer.c'))
util_ss.add(files('readline.c'))
util_ss.add(files('throttle.c'))
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/4] block/block-copy: block_copy(): add timeout_ns parameter
2022-03-02 16:24 [PATCH 0/4] block: copy-before-write: cbw-timeout Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 1/4] util: add qemu-co-timeout Vladimir Sementsov-Ogievskiy
@ 2022-03-02 16:24 ` Vladimir Sementsov-Ogievskiy
2022-03-09 17:35 ` Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 3/4] block/copy-before-write: implement cbw-timeout option Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 4/4] iotests: copy-before-write: add cases for " Vladimir Sementsov-Ogievskiy
3 siblings, 1 reply; 6+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2022-03-02 16:24 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, stefanha, hreitz, kwolf, vsementsov, jsnow
Add possibility to limit block_copy() call in time. To be used in the
next commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/block/block-copy.h | 2 +-
block/block-copy.c | 28 ++++++++++++++++++++--------
2 files changed, 21 insertions(+), 9 deletions(-)
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index 68bbd344b2..1c9616cdee 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -40,7 +40,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
int64_t offset, int64_t *count);
int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
- bool ignore_ratelimit);
+ bool ignore_ratelimit, uint64_t timeout_ns);
/*
* Run block-copy in a coroutine, create corresponding BlockCopyCallState
diff --git a/block/block-copy.c b/block/block-copy.c
index ef948dccec..e0f07e64ee 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -882,10 +882,18 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
return ret;
}
-int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
- bool ignore_ratelimit)
+static void coroutine_fn block_copy_async_co_entry(void *opaque)
{
- BlockCopyCallState call_state = {
+ block_copy_common(opaque);
+}
+
+int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
+ bool ignore_ratelimit, uint64_t timeout_ns)
+{
+ int ret;
+ BlockCopyCallState *call_state = g_new(BlockCopyCallState, 1);
+
+ *call_state = (BlockCopyCallState) {
.s = s,
.offset = start,
.bytes = bytes,
@@ -893,12 +901,16 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
.max_workers = BLOCK_COPY_MAX_WORKERS,
};
- return block_copy_common(&call_state);
-}
+ ret = qemu_co_timeout(block_copy_async_co_entry, call_state, timeout_ns,
+ g_free);
+ if (ret < 0) {
+ /* Timeout. call_state will be freed by running coroutine. */
+ return ret;
+ }
-static void coroutine_fn block_copy_async_co_entry(void *opaque)
-{
- block_copy_common(opaque);
+ ret = call_state->ret;
+
+ return ret;
}
BlockCopyCallState *block_copy_async(BlockCopyState *s,
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 3/4] block/copy-before-write: implement cbw-timeout option
2022-03-02 16:24 [PATCH 0/4] block: copy-before-write: cbw-timeout Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 1/4] util: add qemu-co-timeout Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 2/4] block/block-copy: block_copy(): add timeout_ns parameter Vladimir Sementsov-Ogievskiy
@ 2022-03-02 16:24 ` Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 4/4] iotests: copy-before-write: add cases for " Vladimir Sementsov-Ogievskiy
3 siblings, 0 replies; 6+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2022-03-02 16:24 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, stefanha, hreitz, kwolf, vsementsov, jsnow
In some scenarios, when copy-before-write operations lasts too long
time, it's better to cancel it.
Most useful would be to use the new option together with
on-cbw-error=break-snapshot: this way if cbw operation takes too long
time we'll just cancel backup process but do not disturb the guest too
much.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
qapi/block-core.json | 5 ++++-
block/copy-before-write.c | 6 +++++-
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index e5206272aa..78c77049c3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -4204,12 +4204,15 @@
# @on-cbw-error: Behavior on failure of copy-before-write operation.
# Default is @break-guest-write. (Since 7.0)
#
+# @cbw-timeout: Zero means no limit. Non-zero sets the timeout in seconds
+# for copy-before-write operation. Default 0. (Since 7.0)
+#
# Since: 6.2
##
{ 'struct': 'BlockdevOptionsCbw',
'base': 'BlockdevOptionsGenericFormat',
'data': { 'target': 'BlockdevRef', '*bitmap': 'BlockDirtyBitmap',
- '*on-cbw-error': 'OnCbwError' } }
+ '*on-cbw-error': 'OnCbwError', '*cbw-timeout': 'uint32' } }
##
# @BlockdevOptions:
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index e89cc9799c..50681cc711 100644
--- a/block/copy-before-write.c
+++ b/block/copy-before-write.c
@@ -42,6 +42,7 @@ typedef struct BDRVCopyBeforeWriteState {
BlockCopyState *bcs;
BdrvChild *target;
OnCbwError on_cbw_error;
+ uint32_t cbw_timeout_ns;
/*
* @lock: protects access to @access_bitmap, @done_bitmap and
@@ -107,7 +108,7 @@ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs,
off = QEMU_ALIGN_DOWN(offset, cluster_size);
end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
- ret = block_copy(s->bcs, off, end - off, true);
+ ret = block_copy(s->bcs, off, end - off, true, s->cbw_timeout_ns);
if (ret < 0 && s->on_cbw_error == ON_CBW_ERROR_BREAK_GUEST_WRITE) {
return ret;
}
@@ -405,6 +406,7 @@ static BlockdevOptionsCbw *cbw_parse_options(QDict *options, Error **errp)
*/
qdict_extract_subqdict(options, NULL, "bitmap");
qdict_del(options, "on-cbw-error");
+ qdict_del(options, "cbw-timeout");
out:
visit_free(v);
@@ -448,6 +450,8 @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
}
s->on_cbw_error = opts->has_on_cbw_error ? opts->on_cbw_error :
ON_CBW_ERROR_BREAK_GUEST_WRITE;
+ s->cbw_timeout_ns = opts->has_cbw_timeout ?
+ opts->cbw_timeout * NANOSECONDS_PER_SECOND : 0;
bs->total_sectors = bs->file->bs->total_sectors;
bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 4/4] iotests: copy-before-write: add cases for cbw-timeout option
2022-03-02 16:24 [PATCH 0/4] block: copy-before-write: cbw-timeout Vladimir Sementsov-Ogievskiy
` (2 preceding siblings ...)
2022-03-02 16:24 ` [PATCH 3/4] block/copy-before-write: implement cbw-timeout option Vladimir Sementsov-Ogievskiy
@ 2022-03-02 16:24 ` Vladimir Sementsov-Ogievskiy
3 siblings, 0 replies; 6+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2022-03-02 16:24 UTC (permalink / raw)
To: qemu-block
Cc: qemu-devel, armbru, eblake, stefanha, hreitz, kwolf, vsementsov, jsnow
Add two simple test-cases: timeout failure with
break-snapshot-on-cbw-error behavior and similar with
break-guest-write-on-cbw-error behavior.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
tests/qemu-iotests/tests/copy-before-write | 78 +++++++++++++++++++
.../qemu-iotests/tests/copy-before-write.out | 4 +-
2 files changed, 80 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/tests/copy-before-write b/tests/qemu-iotests/tests/copy-before-write
index a32608f597..265299957c 100755
--- a/tests/qemu-iotests/tests/copy-before-write
+++ b/tests/qemu-iotests/tests/copy-before-write
@@ -122,6 +122,84 @@ read 1048576/1048576 bytes at offset 0
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
""")
+ def do_cbw_timeout(self, on_cbw_error):
+ result = self.vm.qmp('object-add', {
+ 'qom-type': 'throttle-group',
+ 'id': 'group0',
+ 'limits': {'bps-write': 1}
+ })
+ self.assert_qmp(result, 'return', {})
+
+ result = self.vm.qmp('blockdev-add', {
+ 'node-name': 'cbw',
+ 'driver': 'copy-before-write',
+ 'on-cbw-error': on_cbw_error,
+ 'cbw-timeout': 1,
+ 'file': {
+ 'driver': iotests.imgfmt,
+ 'file': {
+ 'driver': 'file',
+ 'filename': source_img,
+ }
+ },
+ 'target': {
+ 'driver': 'throttle',
+ 'throttle-group': 'group0',
+ 'file': {
+ 'driver': 'qcow2',
+ 'file': {
+ 'driver': 'file',
+ 'filename': temp_img
+ }
+ }
+ }
+ })
+ self.assert_qmp(result, 'return', {})
+
+ result = self.vm.qmp('blockdev-add', {
+ 'node-name': 'access',
+ 'driver': 'snapshot-access',
+ 'file': 'cbw'
+ })
+ self.assert_qmp(result, 'return', {})
+
+ result = self.vm.hmp_qemu_io('cbw', 'write 0 512K')
+ self.assert_qmp(result, 'return', '')
+
+ # We need second write to trigger throttling
+ result = self.vm.hmp_qemu_io('cbw', 'write 512K 512K')
+ self.assert_qmp(result, 'return', '')
+
+ result = self.vm.hmp_qemu_io('access', 'read 0 1M')
+ self.assert_qmp(result, 'return', '')
+
+ self.vm.shutdown()
+ log = self.vm.get_log()
+ log = re.sub(r'^\[I \d+\.\d+\] OPENED\n', '', log)
+ log = re.sub(r'\[I \+\d+\.\d+\] CLOSED\n?$', '', log)
+ log = iotests.filter_qemu_io(log)
+ return log
+
+ def test_timeout_break_guest(self):
+ log = self.do_cbw_timeout('break-guest-write')
+ self.assertEqual(log, """\
+wrote 524288/524288 bytes at offset 0
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+write failed: Connection timed out
+read 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+""")
+
+ def test_timeout_break_snapshot(self):
+ log = self.do_cbw_timeout('break-snapshot')
+ self.assertEqual(log, """\
+wrote 524288/524288 bytes at offset 0
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 524288/524288 bytes at offset 524288
+512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read failed: Permission denied
+""")
+
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2'],
diff --git a/tests/qemu-iotests/tests/copy-before-write.out b/tests/qemu-iotests/tests/copy-before-write.out
index fbc63e62f8..89968f35d7 100644
--- a/tests/qemu-iotests/tests/copy-before-write.out
+++ b/tests/qemu-iotests/tests/copy-before-write.out
@@ -1,5 +1,5 @@
-..
+....
----------------------------------------------------------------------
-Ran 2 tests
+Ran 4 tests
OK
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/4] block/block-copy: block_copy(): add timeout_ns parameter
2022-03-02 16:24 ` [PATCH 2/4] block/block-copy: block_copy(): add timeout_ns parameter Vladimir Sementsov-Ogievskiy
@ 2022-03-09 17:35 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 6+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2022-03-09 17:35 UTC (permalink / raw)
To: qemu-block; +Cc: qemu-devel, armbru, eblake, stefanha, hreitz, kwolf, jsnow
02.03.2022 19:24, Vladimir Sementsov-Ogievskiy wrote:
> Add possibility to limit block_copy() call in time. To be used in the
> next commit.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy<vsementsov@virtuozzo.com>
> ---
> include/block/block-copy.h | 2 +-
> block/block-copy.c | 28 ++++++++++++++++++++--------
> 2 files changed, 21 insertions(+), 9 deletions(-)
block_copy() call in block/copy-before-write.c should be updated to not break compilation by this commit
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-03-09 17:37 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-02 16:24 [PATCH 0/4] block: copy-before-write: cbw-timeout Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 1/4] util: add qemu-co-timeout Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 2/4] block/block-copy: block_copy(): add timeout_ns parameter Vladimir Sementsov-Ogievskiy
2022-03-09 17:35 ` Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 3/4] block/copy-before-write: implement cbw-timeout option Vladimir Sementsov-Ogievskiy
2022-03-02 16:24 ` [PATCH 4/4] iotests: copy-before-write: add cases for " Vladimir Sementsov-Ogievskiy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.