From: John Snow <jsnow@redhat.com>
To: qemu-block@nongnu.org, qemu-devel@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>,
vsementsov@virtuozzo.com, Markus Armbruster <armbru@redhat.com>,
Max Reitz <mreitz@redhat.com>, John Snow <jsnow@redhat.com>
Subject: [Qemu-devel] [PATCH v2 06/11] block/backup: improve sync=bitmap work estimates
Date: Mon, 15 Jul 2019 20:01:12 -0400 [thread overview]
Message-ID: <20190716000117.25219-7-jsnow@redhat.com> (raw)
In-Reply-To: <20190716000117.25219-1-jsnow@redhat.com>
When making backups based on bitmaps, the work estimate can be more
accurate. Update iotests to reflect the new strategy.
TOP work estimates are broken, but do not get worse with this commit.
That issue is addressed in the following commits instead.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/backup.c | 8 +++-----
tests/qemu-iotests/256.out | 4 ++--
tests/qemu-iotests/257.out | 36 ++++++++++++++++++------------------
3 files changed, 23 insertions(+), 25 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index a64b768e24..22fafbb80f 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -449,9 +449,8 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
NULL, true);
assert(ret);
- /* TODO job_progress_set_remaining() would make more sense */
- job_progress_update(&job->common.job,
- job->len - bdrv_get_dirty_count(job->copy_bitmap));
+ job_progress_set_remaining(&job->common.job,
+ bdrv_get_dirty_count(job->copy_bitmap));
}
static int coroutine_fn backup_run(Job *job, Error **errp)
@@ -463,12 +462,11 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
QLIST_INIT(&s->inflight_reqs);
qemu_co_rwlock_init(&s->flush_rwlock);
- job_progress_set_remaining(job, s->len);
-
if (s->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
backup_incremental_init_copy_bitmap(s);
} else {
bdrv_set_dirty_bitmap(s->copy_bitmap, 0, s->len);
+ job_progress_set_remaining(job, s->len);
}
s->before_write.notify = backup_before_write_notify;
diff --git a/tests/qemu-iotests/256.out b/tests/qemu-iotests/256.out
index eec38614ec..f18ecb0f91 100644
--- a/tests/qemu-iotests/256.out
+++ b/tests/qemu-iotests/256.out
@@ -113,7 +113,7 @@
{
"return": {}
}
-{"data": {"device": "j2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "j3", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "j2", "len": 0, "offset": 0, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "j3", "len": 0, "offset": 0, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
--- Done ---
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index 43f2e0f9c9..811b1b11f1 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -150,7 +150,7 @@ expecting 7 dirty sectors; have 7. OK!
{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
{"return": {}}
{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 393216, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -228,7 +228,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 983040, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -367,7 +367,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -445,7 +445,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 917504, "offset": 917504, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -633,7 +633,7 @@ expecting 7 dirty sectors; have 7. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
{"return": {}}
{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 393216, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -711,7 +711,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 983040, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -899,7 +899,7 @@ expecting 7 dirty sectors; have 7. OK!
{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
{"return": {}}
{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 393216, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -977,7 +977,7 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 983040, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1116,7 +1116,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1194,7 +1194,7 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 917504, "offset": 917504, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1382,7 +1382,7 @@ expecting 7 dirty sectors; have 7. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
{"return": {}}
{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 393216, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1460,7 +1460,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 786432, "offset": 786432, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1648,7 +1648,7 @@ expecting 7 dirty sectors; have 7. OK!
{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
{"return": {}}
{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 393216, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1726,7 +1726,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 786432, "offset": 786432, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1865,7 +1865,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 393216, "offset": 65536, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1943,7 +1943,7 @@ expecting 13 dirty sectors; have 13. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 851968, "offset": 851968, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -2131,7 +2131,7 @@ expecting 7 dirty sectors; have 7. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
{"return": {}}
{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 393216, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -2209,7 +2209,7 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 786432, "offset": 786432, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
--
2.21.0
next prev parent reply other threads:[~2019-07-16 0:04 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-16 0:01 [Qemu-devel] [PATCH v2 00/11] bitmaps: allow bitmaps to be used with full and top John Snow
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 01/11] iotests/257: add Pattern class John Snow
2019-07-16 10:08 ` Max Reitz
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 02/11] iotests/257: add EmulatedBitmap class John Snow
2019-07-16 10:11 ` Max Reitz
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 03/11] iotests/257: Refactor backup helpers John Snow
2019-07-16 10:33 ` Max Reitz
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 04/11] block/backup: hoist bitmap check into QMP interface John Snow
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 05/11] iotests/257: test API failures John Snow
2019-07-16 10:35 ` Max Reitz
2019-07-16 0:01 ` John Snow [this message]
2019-07-16 10:53 ` [Qemu-devel] [PATCH v2 06/11] block/backup: improve sync=bitmap work estimates Max Reitz
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 07/11] block/backup: centralize copy_bitmap initialization John Snow
2019-07-16 10:58 ` Max Reitz
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 08/11] block/backup: add backup_is_cluster_allocated John Snow
2019-07-16 11:07 ` Max Reitz
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 09/11] block/backup: teach TOP to never copy unallocated regions John Snow
2019-07-16 11:43 ` Max Reitz
2019-07-16 16:02 ` John Snow
2019-07-16 16:11 ` Max Reitz
2019-07-17 18:10 ` John Snow
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 10/11] block/backup: support bitmap sync modes for non-bitmap backups John Snow
2019-07-16 5:18 ` Markus Armbruster
2019-07-16 14:49 ` John Snow
2019-07-16 11:45 ` Max Reitz
2019-07-16 0:01 ` [Qemu-devel] [PATCH v2 11/11] iotests/257: test traditional sync modes John Snow
2019-07-16 12:04 ` Max Reitz
2019-07-16 16:58 ` John Snow
2019-07-17 9:50 ` Max Reitz
2019-07-17 17:53 ` John Snow
2019-07-17 19:35 ` [Qemu-devel] [PATCH v2 00/11] bitmaps: allow bitmaps to be used with full and top John Snow
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190716000117.25219-7-jsnow@redhat.com \
--to=jsnow@redhat.com \
--cc=armbru@redhat.com \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=vsementsov@virtuozzo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.