From: Yehuda Sadeh <yehuda@hq.newdream.net> To: qemu-devel@nongnu.org, ceph-devel@vger.kernel.org Cc: sage@newdream.net, yehudasa@gmail.com, Yehuda Sadeh <yehuda@hq.newdream.net> Subject: [PATCH 2/2] qemu-img: don't skip writing small holes Date: Wed, 7 Sep 2011 16:06:53 -0700 [thread overview] Message-ID: <92a5407ccc0617e43a4bf7d2a74fe1887382dd75.1315436097.git.yehuda@hq.newdream.net> (raw) In-Reply-To: <cover.1315436097.git.yehuda@hq.newdream.net> In-Reply-To: <cover.1315436097.git.yehuda@hq.newdream.net> When doing convert, we check that the sectors that are written are not empty. When holes are small, and interleaved with data it can lead to a significant performance issue. Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> --- qemu-img.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 0552746..757fc3a 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -623,6 +623,7 @@ static int compare_sectors(const uint8_t *buf1, const uint8_t *buf2, int n, #define IO_BUF_SIZE (2 * 1024 * 1024) #define IO_WRITE_WINDOW_THRESHOLD (32 * 1024 * 1024) +#define IO_WRITE_MIN_SIZE (128 * 1024) static int write_window = 0; @@ -991,6 +992,7 @@ static int img_convert(int argc, char **argv) should add a specific call to have the info to go faster */ buf1 = buf; while (n > 0) { + int is_allocated = is_allocated_sectors(buf1, n, &n1); while (write_window > IO_WRITE_WINDOW_THRESHOLD / 512) { qemu_aio_wait(); } @@ -1001,8 +1003,8 @@ static int img_convert(int argc, char **argv) If the output is to a host device, we also write out sectors that are entirely 0, since whatever data was already there is garbage, not 0s. */ - if (!has_zero_init || out_baseimg || - is_allocated_sectors(buf1, n, &n1)) { + if (is_allocated || n != n1 || !has_zero_init || out_baseimg) { + n1 = MAX(n1, MIN(n, IO_WRITE_MIN_SIZE / 512)); QEMUIOVector *qiov = qemu_mallocz(sizeof(QEMUIOVector)); qemu_iovec_init(qiov, 1); qemu_iovec_add(qiov, (void *)buf1, n1 * 512); -- 1.7.5.1
WARNING: multiple messages have this Message-ID (diff)
From: Yehuda Sadeh <yehuda@hq.newdream.net> To: qemu-devel@nongnu.org, ceph-devel@vger.kernel.org Cc: sage@newdream.net, yehudasa@gmail.com, Yehuda Sadeh <yehuda@hq.newdream.net> Subject: [Qemu-devel] [PATCH 2/2] qemu-img: don't skip writing small holes Date: Wed, 7 Sep 2011 16:06:53 -0700 [thread overview] Message-ID: <92a5407ccc0617e43a4bf7d2a74fe1887382dd75.1315436097.git.yehuda@hq.newdream.net> (raw) In-Reply-To: <cover.1315436097.git.yehuda@hq.newdream.net> In-Reply-To: <cover.1315436097.git.yehuda@hq.newdream.net> When doing convert, we check that the sectors that are written are not empty. When holes are small, and interleaved with data it can lead to a significant performance issue. Signed-off-by: Yehuda Sadeh <yehuda@hq.newdream.net> --- qemu-img.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 0552746..757fc3a 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -623,6 +623,7 @@ static int compare_sectors(const uint8_t *buf1, const uint8_t *buf2, int n, #define IO_BUF_SIZE (2 * 1024 * 1024) #define IO_WRITE_WINDOW_THRESHOLD (32 * 1024 * 1024) +#define IO_WRITE_MIN_SIZE (128 * 1024) static int write_window = 0; @@ -991,6 +992,7 @@ static int img_convert(int argc, char **argv) should add a specific call to have the info to go faster */ buf1 = buf; while (n > 0) { + int is_allocated = is_allocated_sectors(buf1, n, &n1); while (write_window > IO_WRITE_WINDOW_THRESHOLD / 512) { qemu_aio_wait(); } @@ -1001,8 +1003,8 @@ static int img_convert(int argc, char **argv) If the output is to a host device, we also write out sectors that are entirely 0, since whatever data was already there is garbage, not 0s. */ - if (!has_zero_init || out_baseimg || - is_allocated_sectors(buf1, n, &n1)) { + if (is_allocated || n != n1 || !has_zero_init || out_baseimg) { + n1 = MAX(n1, MIN(n, IO_WRITE_MIN_SIZE / 512)); QEMUIOVector *qiov = qemu_mallocz(sizeof(QEMUIOVector)); qemu_iovec_init(qiov, 1); qemu_iovec_add(qiov, (void *)buf1, n1 * 512); -- 1.7.5.1
next prev parent reply other threads:[~2011-09-07 22:58 UTC|newest] Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top 2011-09-07 23:06 [PATCH 0/2] improve qemu-img conversion performance Yehuda Sadeh 2011-09-07 23:06 ` [Qemu-devel] " Yehuda Sadeh 2011-09-07 23:06 ` [PATCH 1/2] qemu-img: async write to block device when converting image Yehuda Sadeh 2011-09-07 23:06 ` [Qemu-devel] " Yehuda Sadeh 2011-09-08 4:18 ` Sage Weil 2011-09-08 4:18 ` Sage Weil 2011-09-07 23:06 ` Yehuda Sadeh [this message] 2011-09-07 23:06 ` [Qemu-devel] [PATCH 2/2] qemu-img: don't skip writing small holes Yehuda Sadeh 2011-09-08 7:56 ` [Qemu-devel] [PATCH 0/2] improve qemu-img conversion performance Stefan Hajnoczi 2011-09-08 7:56 ` Stefan Hajnoczi 2011-09-09 4:52 ` Sage Weil 2011-09-09 4:52 ` Sage Weil 2011-09-08 14:13 ` Kevin Wolf 2011-09-08 14:13 ` Kevin Wolf 2011-09-08 16:36 ` Sage Weil 2011-09-08 16:36 ` [Qemu-devel] " Sage Weil 2011-09-09 8:18 ` Kevin Wolf 2011-09-09 8:18 ` Kevin Wolf 2011-09-12 3:14 ` Sage Weil 2011-09-12 3:14 ` Sage Weil 2011-09-12 3:17 ` Yehuda Sadeh Weinraub 2011-09-12 3:17 ` [Qemu-devel] " Yehuda Sadeh Weinraub 2011-09-12 7:42 ` Yehuda Sadeh Weinraub 2011-09-12 7:42 ` Yehuda Sadeh Weinraub 2011-09-12 8:05 ` Kevin Wolf 2011-09-12 8:05 ` Kevin Wolf
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=92a5407ccc0617e43a4bf7d2a74fe1887382dd75.1315436097.git.yehuda@hq.newdream.net \ --to=yehuda@hq.newdream.net \ --cc=ceph-devel@vger.kernel.org \ --cc=qemu-devel@nongnu.org \ --cc=sage@newdream.net \ --cc=yehudasa@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.