All of lore.kernel.org
 help / color / mirror / Atom feed
From: Yehuda Sadeh <yehuda@hq.newdream.net>
To: qemu-devel@nongnu.org, ceph-devel@vger.kernel.org
Cc: sage@newdream.net, yehudasa@gmail.com,
	Yehuda Sadeh <yehuda@hq.newdream.net>
Subject: [PATCH 2/2] qemu-img: don't skip writing small holes
Date: Wed,  7 Sep 2011 16:06:53 -0700	[thread overview]
Message-ID: <92a5407ccc0617e43a4bf7d2a74fe1887382dd75.1315436097.git.yehuda@hq.newdream.net> (raw)
In-Reply-To: <cover.1315436097.git.yehuda@hq.newdream.net>
In-Reply-To: <cover.1315436097.git.yehuda@hq.newdream.net>

When doing convert, we check that the sectors that are written
are not empty. When holes are small, and interleaved with data
it can lead to a significant performance issue.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
---
 qemu-img.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index 0552746..757fc3a 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -623,6 +623,7 @@ static int compare_sectors(const uint8_t *buf1, const uint8_t *buf2, int n,
 
 #define IO_BUF_SIZE (2 * 1024 * 1024)
 #define IO_WRITE_WINDOW_THRESHOLD (32 * 1024 * 1024)
+#define IO_WRITE_MIN_SIZE (128 * 1024)
 
 static int write_window = 0;
 
@@ -991,6 +992,7 @@ static int img_convert(int argc, char **argv)
                should add a specific call to have the info to go faster */
             buf1 = buf;
             while (n > 0) {
+                int is_allocated = is_allocated_sectors(buf1, n, &n1);
                 while (write_window > IO_WRITE_WINDOW_THRESHOLD / 512) {
                     qemu_aio_wait();
                 }
@@ -1001,8 +1003,8 @@ static int img_convert(int argc, char **argv)
                    If the output is to a host device, we also write out
                    sectors that are entirely 0, since whatever data was
                    already there is garbage, not 0s. */
-                if (!has_zero_init || out_baseimg ||
-                    is_allocated_sectors(buf1, n, &n1)) {
+                if (is_allocated || n != n1 || !has_zero_init || out_baseimg) {
+                    n1 = MAX(n1, MIN(n, IO_WRITE_MIN_SIZE / 512));
                     QEMUIOVector *qiov = qemu_mallocz(sizeof(QEMUIOVector));
 		    qemu_iovec_init(qiov, 1);
 		    qemu_iovec_add(qiov, (void *)buf1, n1 * 512);
-- 
1.7.5.1


WARNING: multiple messages have this Message-ID (diff)
From: Yehuda Sadeh <yehuda@hq.newdream.net>
To: qemu-devel@nongnu.org, ceph-devel@vger.kernel.org
Cc: sage@newdream.net, yehudasa@gmail.com,
	Yehuda Sadeh <yehuda@hq.newdream.net>
Subject: [Qemu-devel] [PATCH 2/2] qemu-img: don't skip writing small holes
Date: Wed,  7 Sep 2011 16:06:53 -0700	[thread overview]
Message-ID: <92a5407ccc0617e43a4bf7d2a74fe1887382dd75.1315436097.git.yehuda@hq.newdream.net> (raw)
In-Reply-To: <cover.1315436097.git.yehuda@hq.newdream.net>
In-Reply-To: <cover.1315436097.git.yehuda@hq.newdream.net>

When doing convert, we check that the sectors that are written
are not empty. When holes are small, and interleaved with data
it can lead to a significant performance issue.

Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net>
---
 qemu-img.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index 0552746..757fc3a 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -623,6 +623,7 @@ static int compare_sectors(const uint8_t *buf1, const uint8_t *buf2, int n,
 
 #define IO_BUF_SIZE (2 * 1024 * 1024)
 #define IO_WRITE_WINDOW_THRESHOLD (32 * 1024 * 1024)
+#define IO_WRITE_MIN_SIZE (128 * 1024)
 
 static int write_window = 0;
 
@@ -991,6 +992,7 @@ static int img_convert(int argc, char **argv)
                should add a specific call to have the info to go faster */
             buf1 = buf;
             while (n > 0) {
+                int is_allocated = is_allocated_sectors(buf1, n, &n1);
                 while (write_window > IO_WRITE_WINDOW_THRESHOLD / 512) {
                     qemu_aio_wait();
                 }
@@ -1001,8 +1003,8 @@ static int img_convert(int argc, char **argv)
                    If the output is to a host device, we also write out
                    sectors that are entirely 0, since whatever data was
                    already there is garbage, not 0s. */
-                if (!has_zero_init || out_baseimg ||
-                    is_allocated_sectors(buf1, n, &n1)) {
+                if (is_allocated || n != n1 || !has_zero_init || out_baseimg) {
+                    n1 = MAX(n1, MIN(n, IO_WRITE_MIN_SIZE / 512));
                     QEMUIOVector *qiov = qemu_mallocz(sizeof(QEMUIOVector));
 		    qemu_iovec_init(qiov, 1);
 		    qemu_iovec_add(qiov, (void *)buf1, n1 * 512);
-- 
1.7.5.1

  parent reply	other threads:[~2011-09-07 22:58 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-09-07 23:06 [PATCH 0/2] improve qemu-img conversion performance Yehuda Sadeh
2011-09-07 23:06 ` [Qemu-devel] " Yehuda Sadeh
2011-09-07 23:06 ` [PATCH 1/2] qemu-img: async write to block device when converting image Yehuda Sadeh
2011-09-07 23:06   ` [Qemu-devel] " Yehuda Sadeh
2011-09-08  4:18   ` Sage Weil
2011-09-08  4:18     ` Sage Weil
2011-09-07 23:06 ` Yehuda Sadeh [this message]
2011-09-07 23:06   ` [Qemu-devel] [PATCH 2/2] qemu-img: don't skip writing small holes Yehuda Sadeh
2011-09-08  7:56 ` [Qemu-devel] [PATCH 0/2] improve qemu-img conversion performance Stefan Hajnoczi
2011-09-08  7:56   ` Stefan Hajnoczi
2011-09-09  4:52   ` Sage Weil
2011-09-09  4:52     ` Sage Weil
2011-09-08 14:13 ` Kevin Wolf
2011-09-08 14:13   ` Kevin Wolf
2011-09-08 16:36   ` Sage Weil
2011-09-08 16:36     ` [Qemu-devel] " Sage Weil
2011-09-09  8:18     ` Kevin Wolf
2011-09-09  8:18       ` Kevin Wolf
2011-09-12  3:14       ` Sage Weil
2011-09-12  3:14         ` Sage Weil
2011-09-12  3:17         ` Yehuda Sadeh Weinraub
2011-09-12  3:17           ` [Qemu-devel] " Yehuda Sadeh Weinraub
2011-09-12  7:42           ` Yehuda Sadeh Weinraub
2011-09-12  7:42             ` Yehuda Sadeh Weinraub
2011-09-12  8:05           ` Kevin Wolf
2011-09-12  8:05             ` Kevin Wolf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=92a5407ccc0617e43a4bf7d2a74fe1887382dd75.1315436097.git.yehuda@hq.newdream.net \
    --to=yehuda@hq.newdream.net \
    --cc=ceph-devel@vger.kernel.org \
    --cc=qemu-devel@nongnu.org \
    --cc=sage@newdream.net \
    --cc=yehudasa@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.