[Qemu-devel] [PATCH v2 0/5] block: Introduce qemu-img bench

[Qemu-devel] [PATCH v2 0/5] block: Introduce qemu-img bench

[Kevin Wolf]

After merging Den's qcow2 patch to avoid duplicated flushes, I thought I would
be nice to reproduce the problem and I remembered 'qemu-img bench', which I had
posted before as part of more than one RFC series, but which never made it to
master somehow. So here is a rebased and cleaned up version of it, just by
itself, so that it hopefully can be merged finally.

Of course, I failed to actually reproduce the problem on my laptop. Who knows,
something on my system might be more intelligent about useless flushes, or
maybe I just misunderstood what the problematic scenario looks like at the
block level. Doesn't make the tool less useful, though, and I already did the
rebasing.

v2:
- Added --pattern=... option for writes [Den]
- Added --no-drain option for write+flush tests [Den]

Kevin Wolf (5):
  qemu-img bench
  qemu-img bench: Sequential writes
  qemu-img bench: Make start offset configurable
  qemu-img bench: Implement -S (step size)
  qemu-img bench: Add --flush-interval

 qemu-img-cmds.hx |   6 +
 qemu-img.c       | 329 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-img.texi    |  24 ++++
 3 files changed, 359 insertions(+)

-- 
1.8.3.1

[Qemu-devel] [PATCH v2 1/5] qemu-img bench

[Kevin Wolf]

This adds a qemu-img command that allows doing some simple benchmarks
for the block layer without involving guest devices and a real VM.

For the start, this implements only a test of sequential reads.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
---
 qemu-img-cmds.hx |   6 ++
 qemu-img.c       | 190 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-img.texi    |  10 +++
 3 files changed, 206 insertions(+)

diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index e7cded6..f3bd546 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -9,6 +9,12 @@ STEXI
 @table @option
 ETEXI
 
+DEF("bench", img_bench,
+    "bench [-c count] [-d depth] [-f fmt] [-n] [-q] [-s buffer_size] [-t cache] filename")
+STEXI
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-q] [-s @var{buffer_size}] [-t @var{cache}] @var{filename}
+ETEXI
+
 DEF("check", img_check,
     "check [-q] [--object objectdef] [--image-opts] [-f fmt] [--output=ofmt] [-r [leaks | all]] [-T src_cache] filename")
 STEXI
diff --git a/qemu-img.c b/qemu-img.c
index 4b56ad3..d471d10 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -3459,6 +3459,196 @@ out_no_progress:
     return 0;
 }
 
+typedef struct BenchData {
+    BlockBackend *blk;
+    uint64_t image_size;
+    int bufsize;
+    int nrreq;
+    int n;
+    uint8_t *buf;
+    QEMUIOVector *qiov;
+
+    int in_flight;
+    uint64_t offset;
+} BenchData;
+
+static void bench_cb(void *opaque, int ret)
+{
+    BenchData *b = opaque;
+    BlockAIOCB *acb;
+
+    if (ret < 0) {
+        error_report("Failed request: %s\n", strerror(-ret));
+        exit(EXIT_FAILURE);
+    }
+    if (b->in_flight > 0) {
+        b->n--;
+        b->in_flight--;
+    }
+
+    while (b->n > b->in_flight && b->in_flight < b->nrreq) {
+        acb = blk_aio_preadv(b->blk, b->offset, b->qiov, 0,
+                             bench_cb, b);
+        if (!acb) {
+            error_report("Failed to issue request");
+            exit(EXIT_FAILURE);
+        }
+        b->in_flight++;
+        b->offset += b->bufsize;
+        b->offset %= b->image_size;
+    }
+}
+
+static int img_bench(int argc, char **argv)
+{
+    int c, ret = 0;
+    const char *fmt = NULL, *filename;
+    bool quiet = false;
+    bool image_opts = false;
+    int count = 75000;
+    int depth = 64;
+    size_t bufsize = 4096;
+    int64_t image_size;
+    BlockBackend *blk = NULL;
+    BenchData data = {};
+    int flags = 0;
+    bool writethrough;
+    struct timeval t1, t2;
+    int i;
+
+    for (;;) {
+        static const struct option long_options[] = {
+            {"help", no_argument, 0, 'h'},
+            {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
+            {0, 0, 0, 0}
+        };
+        c = getopt_long(argc, argv, "hc:d:f:nqs:t:", long_options, NULL);
+        if (c == -1) {
+            break;
+        }
+
+        switch (c) {
+        case 'h':
+        case '?':
+            help();
+            break;
+        case 'c':
+        {
+            char *end;
+            errno = 0;
+            count = strtoul(optarg, &end, 0);
+            if (errno || *end || count > INT_MAX) {
+                error_report("Invalid request count specified");
+                return 1;
+            }
+            break;
+        }
+        case 'd':
+        {
+            char *end;
+            errno = 0;
+            depth = strtoul(optarg, &end, 0);
+            if (errno || *end || depth > INT_MAX) {
+                error_report("Invalid queue depth specified");
+                return 1;
+            }
+            break;
+        }
+        case 'f':
+            fmt = optarg;
+            break;
+        case 'n':
+            flags |= BDRV_O_NATIVE_AIO;
+            break;
+        case 'q':
+            quiet = true;
+            break;
+        case 's':
+        {
+            int64_t sval;
+            char *end;
+
+            sval = qemu_strtosz_suffix(optarg, &end, QEMU_STRTOSZ_DEFSUFFIX_B);
+            if (sval < 0 || sval > INT_MAX || *end) {
+                error_report("Invalid buffer size specified");
+                return 1;
+            }
+
+            bufsize = sval;
+            break;
+        }
+        case 't':
+            ret = bdrv_parse_cache_mode(optarg, &flags, &writethrough);
+            if (ret < 0) {
+                error_report("Invalid cache mode");
+                ret = -1;
+                goto out;
+            }
+            break;
+        case OPTION_IMAGE_OPTS:
+            image_opts = true;
+            break;
+        }
+    }
+
+    if (optind != argc - 1) {
+        error_exit("Expecting one image file name");
+    }
+    filename = argv[argc - 1];
+
+    blk = img_open(image_opts, filename, fmt, flags, writethrough, quiet);
+    if (!blk) {
+        ret = -1;
+        goto out;
+    }
+
+    image_size = blk_getlength(blk);
+    if (image_size < 0) {
+        ret = image_size;
+        goto out;
+    }
+
+    data = (BenchData) {
+        .blk        = blk,
+        .image_size = image_size,
+        .bufsize    = bufsize,
+        .nrreq      = depth,
+        .n          = count,
+    };
+    printf("Sending %d requests, %d bytes each, %d in parallel\n",
+        data.n, data.bufsize, data.nrreq);
+
+    data.buf = blk_blockalign(blk, data.nrreq * data.bufsize);
+    data.qiov = g_new(QEMUIOVector, data.nrreq);
+    for (i = 0; i < data.nrreq; i++) {
+        qemu_iovec_init(&data.qiov[i], 1);
+        qemu_iovec_add(&data.qiov[i],
+                       data.buf + i * data.bufsize, data.bufsize);
+    }
+
+    gettimeofday(&t1, NULL);
+    bench_cb(&data, 0);
+
+    while (data.n > 0) {
+        main_loop_wait(false);
+    }
+    gettimeofday(&t2, NULL);
+
+    printf("Run completed in %3.3f seconds.\n",
+           (t2.tv_sec - t1.tv_sec)
+           + ((double)(t2.tv_usec - t1.tv_usec) / 1000000));
+
+out:
+    qemu_vfree(data.buf);
+    blk_unref(blk);
+
+    if (ret) {
+        return 1;
+    }
+    return 0;
+}
+
+
 static const img_cmd_t img_cmds[] = {
 #define DEF(option, callback, arg_string)        \
     { option, callback },
diff --git a/qemu-img.texi b/qemu-img.texi
index afaebdd..b6b28e3 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -131,6 +131,16 @@ Skip the creation of the target volume
 Command description:
 
 @table @option
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-q] [-s @var{buffer_size}] [-t @var{cache}] @var{filename}
+
+Run a simple sequential read benchmark on the specified image. A total number
+of @var{count} I/O requests is performed, each @var{buffer_size} bytes in size,
+and with @var{depth} requests in parallel.
+
+If @code{-n} is specified, the native AIO backend is used if possible. On
+Linux, this option only works if @code{-t none} or @code{-t directsync} is
+specified as well.
+
 @item check [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] @var{filename}
 
 Perform a consistency check on the disk image @var{filename}. The command can
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 2/5] qemu-img bench: Sequential writes

[Kevin Wolf]

This extends qemu-img bench with an option that makes it use sequential
writes instead of reads for the test run.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 qemu-img-cmds.hx |  4 ++--
 qemu-img.c       | 38 +++++++++++++++++++++++++++++++++-----
 qemu-img.texi    | 13 +++++++++----
 3 files changed, 44 insertions(+), 11 deletions(-)

diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index f3bd546..baca85e 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -10,9 +10,9 @@ STEXI
 ETEXI
 
 DEF("bench", img_bench,
-    "bench [-c count] [-d depth] [-f fmt] [-n] [-q] [-s buffer_size] [-t cache] filename")
+    "bench [-c count] [-d depth] [-f fmt] [-n] [--pattern=pattern] [-q] [-s buffer_size] [-t cache] [-w] filename")
 STEXI
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-q] [-s @var{buffer_size}] [-t @var{cache}] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
 ETEXI
 
 DEF("check", img_check,
diff --git a/qemu-img.c b/qemu-img.c
index d471d10..85d1353 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -53,6 +53,7 @@ enum {
     OPTION_BACKING_CHAIN = 257,
     OPTION_OBJECT = 258,
     OPTION_IMAGE_OPTS = 259,
+    OPTION_PATTERN = 260,
 };
 
 typedef enum OutputFormat {
@@ -3462,6 +3463,7 @@ out_no_progress:
 typedef struct BenchData {
     BlockBackend *blk;
     uint64_t image_size;
+    bool write;
     int bufsize;
     int nrreq;
     int n;
@@ -3487,8 +3489,13 @@ static void bench_cb(void *opaque, int ret)
     }
 
     while (b->n > b->in_flight && b->in_flight < b->nrreq) {
-        acb = blk_aio_preadv(b->blk, b->offset, b->qiov, 0,
-                             bench_cb, b);
+        if (b->write) {
+            acb = blk_aio_pwritev(b->blk, b->offset, b->qiov, 0,
+                                  bench_cb, b);
+        } else {
+            acb = blk_aio_preadv(b->blk, b->offset, b->qiov, 0,
+                                 bench_cb, b);
+        }
         if (!acb) {
             error_report("Failed to issue request");
             exit(EXIT_FAILURE);
@@ -3505,9 +3512,11 @@ static int img_bench(int argc, char **argv)
     const char *fmt = NULL, *filename;
     bool quiet = false;
     bool image_opts = false;
+    bool is_write = false;
     int count = 75000;
     int depth = 64;
     size_t bufsize = 4096;
+    int pattern = 0;
     int64_t image_size;
     BlockBackend *blk = NULL;
     BenchData data = {};
@@ -3520,9 +3529,10 @@ static int img_bench(int argc, char **argv)
         static const struct option long_options[] = {
             {"help", no_argument, 0, 'h'},
             {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
+            {"pattern", required_argument, 0, OPTION_PATTERN},
             {0, 0, 0, 0}
         };
-        c = getopt_long(argc, argv, "hc:d:f:nqs:t:", long_options, NULL);
+        c = getopt_long(argc, argv, "hc:d:f:nqs:t:w", long_options, NULL);
         if (c == -1) {
             break;
         }
@@ -3585,6 +3595,21 @@ static int img_bench(int argc, char **argv)
                 goto out;
             }
             break;
+        case 'w':
+            flags |= BDRV_O_RDWR;
+            is_write = true;
+            break;
+        case OPTION_PATTERN:
+        {
+            char *end;
+            errno = 0;
+            pattern = strtoul(optarg, &end, 0);
+            if (errno || *end || pattern > 0xff) {
+                error_report("Invalid pattern byte specified");
+                return 1;
+            }
+            break;
+        }
         case OPTION_IMAGE_OPTS:
             image_opts = true;
             break;
@@ -3614,11 +3639,14 @@ static int img_bench(int argc, char **argv)
         .bufsize    = bufsize,
         .nrreq      = depth,
         .n          = count,
+        .write      = is_write,
     };
-    printf("Sending %d requests, %d bytes each, %d in parallel\n",
-        data.n, data.bufsize, data.nrreq);
+    printf("Sending %d %s requests, %d bytes each, %d in parallel\n",
+           data.n, data.write ? "write" : "read", data.bufsize, data.nrreq);
 
     data.buf = blk_blockalign(blk, data.nrreq * data.bufsize);
+    memset(data.buf, pattern, data.nrreq * data.bufsize);
+
     data.qiov = g_new(QEMUIOVector, data.nrreq);
     for (i = 0; i < data.nrreq; i++) {
         qemu_iovec_init(&data.qiov[i], 1);
diff --git a/qemu-img.texi b/qemu-img.texi
index b6b28e3..c477fbf 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -131,16 +131,21 @@ Skip the creation of the target volume
 Command description:
 
 @table @option
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-q] [-s @var{buffer_size}] [-t @var{cache}] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
 
-Run a simple sequential read benchmark on the specified image. A total number
-of @var{count} I/O requests is performed, each @var{buffer_size} bytes in size,
-and with @var{depth} requests in parallel.
+Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
+specified, a write test is performed, otherwise a read test is performed.
+
+A total number of @var{count} I/O requests is performed, each @var{buffer_size}
+bytes in size, and with @var{depth} requests in parallel.
 
 If @code{-n} is specified, the native AIO backend is used if possible. On
 Linux, this option only works if @code{-t none} or @code{-t directsync} is
 specified as well.
 
+For write tests, by default a buffer filled with zeros is written. This can be
+overridden with a pattern byte specified by @var{pattern}.
+
 @item check [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] @var{filename}
 
 Perform a consistency check on the disk image @var{filename}. The command can
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 3/5] qemu-img bench: Make start offset configurable

[Kevin Wolf]

This patch adds an option the specify the offset of the first request
made by qemu-img bench. This allows to benchmark misaligned requests.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
---
 qemu-img-cmds.hx |  4 ++--
 qemu-img.c       | 23 ++++++++++++++++++++---
 qemu-img.texi    |  5 +++--
 3 files changed, 25 insertions(+), 7 deletions(-)

diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index baca85e..117d0f9 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -10,9 +10,9 @@ STEXI
 ETEXI
 
 DEF("bench", img_bench,
-    "bench [-c count] [-d depth] [-f fmt] [-n] [--pattern=pattern] [-q] [-s buffer_size] [-t cache] [-w] filename")
+    "bench [-c count] [-d depth] [-f fmt] [-n] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-t cache] [-w] filename")
 STEXI
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
 ETEXI
 
 DEF("check", img_check,
diff --git a/qemu-img.c b/qemu-img.c
index 85d1353..480ef8d 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -3515,6 +3515,7 @@ static int img_bench(int argc, char **argv)
     bool is_write = false;
     int count = 75000;
     int depth = 64;
+    int64_t offset = 0;
     size_t bufsize = 4096;
     int pattern = 0;
     int64_t image_size;
@@ -3532,7 +3533,7 @@ static int img_bench(int argc, char **argv)
             {"pattern", required_argument, 0, OPTION_PATTERN},
             {0, 0, 0, 0}
         };
-        c = getopt_long(argc, argv, "hc:d:f:nqs:t:w", long_options, NULL);
+        c = getopt_long(argc, argv, "hc:d:f:no:qs:t:w", long_options, NULL);
         if (c == -1) {
             break;
         }
@@ -3570,6 +3571,19 @@ static int img_bench(int argc, char **argv)
         case 'n':
             flags |= BDRV_O_NATIVE_AIO;
             break;
+        case 'o':
+        {
+            char *end;
+            errno = 0;
+            offset = qemu_strtosz_suffix(optarg, &end,
+                                         QEMU_STRTOSZ_DEFSUFFIX_B);
+            if (offset < 0|| *end) {
+                error_report("Invalid offset specified");
+                return 1;
+            }
+            break;
+        }
+            break;
         case 'q':
             quiet = true;
             break;
@@ -3639,10 +3653,13 @@ static int img_bench(int argc, char **argv)
         .bufsize    = bufsize,
         .nrreq      = depth,
         .n          = count,
+        .offset     = offset,
         .write      = is_write,
     };
-    printf("Sending %d %s requests, %d bytes each, %d in parallel\n",
-           data.n, data.write ? "write" : "read", data.bufsize, data.nrreq);
+    printf("Sending %d %s requests, %d bytes each, %d in parallel "
+           "(starting at offset %" PRId64 ")\n",
+           data.n, data.write ? "write" : "read", data.bufsize, data.nrreq,
+           data.offset);
 
     data.buf = blk_blockalign(blk, data.nrreq * data.bufsize);
     memset(data.buf, pattern, data.nrreq * data.bufsize);
diff --git a/qemu-img.texi b/qemu-img.texi
index c477fbf..9bffad2 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -131,13 +131,14 @@ Skip the creation of the target volume
 Command description:
 
 @table @option
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
 
 Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
 specified, a write test is performed, otherwise a read test is performed.
 
 A total number of @var{count} I/O requests is performed, each @var{buffer_size}
-bytes in size, and with @var{depth} requests in parallel.
+bytes in size, and with @var{depth} requests in parallel. The first request
+starts at the position given by @var{offset}.
 
 If @code{-n} is specified, the native AIO backend is used if possible. On
 Linux, this option only works if @code{-t none} or @code{-t directsync} is
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 4/5] qemu-img bench: Implement -S (step size)

[Kevin Wolf]

With this new option, qemu-img bench can be told to advance the current
offset after each request by a different value than the buffer size.
This is useful for controlling the conditions for cluster allocation in
image formats (e.g. qcow2 cluster allocation with COW in front of the
request, or COW areas that aren't overwritten immediately).

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
---
 qemu-img-cmds.hx |  4 ++--
 qemu-img.c       | 25 +++++++++++++++++++++----
 qemu-img.texi    |  6 ++++--
 3 files changed, 27 insertions(+), 8 deletions(-)

diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 117d0f9..05a2991 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -10,9 +10,9 @@ STEXI
 ETEXI
 
 DEF("bench", img_bench,
-    "bench [-c count] [-d depth] [-f fmt] [-n] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-t cache] [-w] filename")
+    "bench [-c count] [-d depth] [-f fmt] [-n] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-w] filename")
 STEXI
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
 ETEXI
 
 DEF("check", img_check,
diff --git a/qemu-img.c b/qemu-img.c
index 480ef8d..c5e2638 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -3465,6 +3465,7 @@ typedef struct BenchData {
     uint64_t image_size;
     bool write;
     int bufsize;
+    int step;
     int nrreq;
     int n;
     uint8_t *buf;
@@ -3501,7 +3502,7 @@ static void bench_cb(void *opaque, int ret)
             exit(EXIT_FAILURE);
         }
         b->in_flight++;
-        b->offset += b->bufsize;
+        b->offset += b->step;
         b->offset %= b->image_size;
     }
 }
@@ -3518,6 +3519,7 @@ static int img_bench(int argc, char **argv)
     int64_t offset = 0;
     size_t bufsize = 4096;
     int pattern = 0;
+    size_t step = 0;
     int64_t image_size;
     BlockBackend *blk = NULL;
     BenchData data = {};
@@ -3533,7 +3535,7 @@ static int img_bench(int argc, char **argv)
             {"pattern", required_argument, 0, OPTION_PATTERN},
             {0, 0, 0, 0}
         };
-        c = getopt_long(argc, argv, "hc:d:f:no:qs:t:w", long_options, NULL);
+        c = getopt_long(argc, argv, "hc:d:f:no:qs:S:t:w", long_options, NULL);
         if (c == -1) {
             break;
         }
@@ -3601,6 +3603,20 @@ static int img_bench(int argc, char **argv)
             bufsize = sval;
             break;
         }
+        case 'S':
+        {
+            int64_t sval;
+            char *end;
+
+            sval = qemu_strtosz_suffix(optarg, &end, QEMU_STRTOSZ_DEFSUFFIX_B);
+            if (sval < 0 || sval > INT_MAX || *end) {
+                error_report("Invalid step size specified");
+                return 1;
+            }
+
+            step = sval;
+            break;
+        }
         case 't':
             ret = bdrv_parse_cache_mode(optarg, &flags, &writethrough);
             if (ret < 0) {
@@ -3651,15 +3667,16 @@ static int img_bench(int argc, char **argv)
         .blk        = blk,
         .image_size = image_size,
         .bufsize    = bufsize,
+        .step       = step ?: bufsize,
         .nrreq      = depth,
         .n          = count,
         .offset     = offset,
         .write      = is_write,
     };
     printf("Sending %d %s requests, %d bytes each, %d in parallel "
-           "(starting at offset %" PRId64 ")\n",
+           "(starting at offset %" PRId64 ", step size %d)\n",
            data.n, data.write ? "write" : "read", data.bufsize, data.nrreq,
-           data.offset);
+           data.offset, data.step);
 
     data.buf = blk_blockalign(blk, data.nrreq * data.bufsize);
     memset(data.buf, pattern, data.nrreq * data.bufsize);
diff --git a/qemu-img.texi b/qemu-img.texi
index 9bffad2..ccc0b51 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -131,14 +131,16 @@ Skip the creation of the target volume
 Command description:
 
 @table @option
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
 
 Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
 specified, a write test is performed, otherwise a read test is performed.
 
 A total number of @var{count} I/O requests is performed, each @var{buffer_size}
 bytes in size, and with @var{depth} requests in parallel. The first request
-starts at the position given by @var{offset}.
+starts at the position given by @var{offset}, each following request increases
+the current position by @var{step_size}. If @var{step_size} is not given,
+@var{buffer_size} is used for its value.
 
 If @code{-n} is specified, the native AIO backend is used if possible. On
 Linux, this option only works if @code{-t none} or @code{-t directsync} is
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 5/5] qemu-img bench: Add --flush-interval

[Kevin Wolf]

This options allows to flush the image periodically during write tests.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
 qemu-img-cmds.hx |  4 +--
 qemu-img.c       | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++------
 qemu-img.texi    |  8 ++++-
 3 files changed, 95 insertions(+), 12 deletions(-)

diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 05a2991..7e95b2d 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -10,9 +10,9 @@ STEXI
 ETEXI
 
 DEF("bench", img_bench,
-    "bench [-c count] [-d depth] [-f fmt] [-n] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-w] filename")
+    "bench [-c count] [-d depth] [-f fmt] [--flush-interval=flush_interval] [-n] [--no-drain] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-w] filename")
 STEXI
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
 ETEXI
 
 DEF("check", img_check,
diff --git a/qemu-img.c b/qemu-img.c
index c5e2638..04cddab 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -54,6 +54,8 @@ enum {
     OPTION_OBJECT = 258,
     OPTION_IMAGE_OPTS = 259,
     OPTION_PATTERN = 260,
+    OPTION_FLUSH_INTERVAL = 261,
+    OPTION_NO_DRAIN = 262,
 };
 
 typedef enum OutputFormat {
@@ -3468,13 +3470,24 @@ typedef struct BenchData {
     int step;
     int nrreq;
     int n;
+    int flush_interval;
+    bool drain_on_flush;
     uint8_t *buf;
     QEMUIOVector *qiov;
 
     int in_flight;
+    bool in_flush;
     uint64_t offset;
 } BenchData;
 
+static void bench_undrained_flush_cb(void *opaque, int ret)
+{
+    if (ret < 0) {
+        error_report("Failed flush request: %s\n", strerror(-ret));
+        exit(EXIT_FAILURE);
+    }
+}
+
 static void bench_cb(void *opaque, int ret)
 {
     BenchData *b = opaque;
@@ -3484,9 +3497,39 @@ static void bench_cb(void *opaque, int ret)
         error_report("Failed request: %s\n", strerror(-ret));
         exit(EXIT_FAILURE);
     }
-    if (b->in_flight > 0) {
+
+    if (b->in_flush) {
+        /* Just finished a flush with drained queue: Start next requests */
+        assert(b->in_flight == 0);
+        b->in_flush = false;
+    } else if (b->in_flight > 0) {
+        int remaining = b->n - b->in_flight;
+
         b->n--;
         b->in_flight--;
+
+        /* Time for flush? Drain queue if requested, then flush */
+        if (b->flush_interval && remaining % b->flush_interval == 0) {
+            if (!b->in_flight || !b->drain_on_flush) {
+                BlockCompletionFunc *cb;
+
+                if (b->drain_on_flush) {
+                    b->in_flush = true;
+                    cb = bench_cb;
+                } else {
+                    cb = bench_undrained_flush_cb;
+                }
+
+                acb = blk_aio_flush(b->blk, cb, b);
+                if (!acb) {
+                    error_report("Failed to issue flush request");
+                    exit(EXIT_FAILURE);
+                }
+            }
+            if (b->drain_on_flush) {
+                return;
+            }
+        }
     }
 
     while (b->n > b->in_flight && b->in_flight < b->nrreq) {
@@ -3520,6 +3563,8 @@ static int img_bench(int argc, char **argv)
     size_t bufsize = 4096;
     int pattern = 0;
     size_t step = 0;
+    int flush_interval = 0;
+    bool drain_on_flush = true;
     int64_t image_size;
     BlockBackend *blk = NULL;
     BenchData data = {};
@@ -3531,8 +3576,10 @@ static int img_bench(int argc, char **argv)
     for (;;) {
         static const struct option long_options[] = {
             {"help", no_argument, 0, 'h'},
+            {"flush-interval", required_argument, 0, OPTION_FLUSH_INTERVAL},
             {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
             {"pattern", required_argument, 0, OPTION_PATTERN},
+            {"no-drain", no_argument, 0, OPTION_NO_DRAIN},
             {0, 0, 0, 0}
         };
         c = getopt_long(argc, argv, "hc:d:f:no:qs:S:t:w", long_options, NULL);
@@ -3640,6 +3687,20 @@ static int img_bench(int argc, char **argv)
             }
             break;
         }
+        case OPTION_FLUSH_INTERVAL:
+        {
+            char *end;
+            errno = 0;
+            flush_interval = strtoul(optarg, &end, 0);
+            if (errno || *end || flush_interval > INT_MAX) {
+                error_report("Invalid flush interval specified");
+                return 1;
+            }
+            break;
+        }
+        case OPTION_NO_DRAIN:
+            drain_on_flush = false;
+            break;
         case OPTION_IMAGE_OPTS:
             image_opts = true;
             break;
@@ -3651,6 +3712,17 @@ static int img_bench(int argc, char **argv)
     }
     filename = argv[argc - 1];
 
+    if (!is_write && flush_interval) {
+        error_report("--flush-interval is only available in write tests");
+        ret = -1;
+        goto out;
+    }
+    if (flush_interval && flush_interval < depth) {
+        error_report("Flush interval can't be smaller than depth");
+        ret = -1;
+        goto out;
+    }
+
     blk = img_open(image_opts, filename, fmt, flags, writethrough, quiet);
     if (!blk) {
         ret = -1;
@@ -3664,19 +3736,24 @@ static int img_bench(int argc, char **argv)
     }
 
     data = (BenchData) {
-        .blk        = blk,
-        .image_size = image_size,
-        .bufsize    = bufsize,
-        .step       = step ?: bufsize,
-        .nrreq      = depth,
-        .n          = count,
-        .offset     = offset,
-        .write      = is_write,
+        .blk            = blk,
+        .image_size     = image_size,
+        .bufsize        = bufsize,
+        .step           = step ?: bufsize,
+        .nrreq          = depth,
+        .n              = count,
+        .offset         = offset,
+        .write          = is_write,
+        .flush_interval = flush_interval,
+        .drain_on_flush = drain_on_flush,
     };
     printf("Sending %d %s requests, %d bytes each, %d in parallel "
            "(starting at offset %" PRId64 ", step size %d)\n",
            data.n, data.write ? "write" : "read", data.bufsize, data.nrreq,
            data.offset, data.step);
+    if (flush_interval) {
+        printf("Sending flush every %d requests\n", flush_interval);
+    }
 
     data.buf = blk_blockalign(blk, data.nrreq * data.bufsize);
     memset(data.buf, pattern, data.nrreq * data.bufsize);
diff --git a/qemu-img.texi b/qemu-img.texi
index ccc0b51..cbe50e9 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -131,7 +131,7 @@ Skip the creation of the target volume
 Command description:
 
 @table @option
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
+@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
 
 Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
 specified, a write test is performed, otherwise a read test is performed.
@@ -142,6 +142,12 @@ starts at the position given by @var{offset}, each following request increases
 the current position by @var{step_size}. If @var{step_size} is not given,
 @var{buffer_size} is used for its value.
 
+If @var{flush_interval} is specified for a write test, the request queue is
+drained and a flush is issued before new writes are made whenever the number of
+remaining requests is a multiple of @var{flush_interval}. If additionally
+@code{--no-drain} is specified, a flush is issued without draining the request
+queue first.
+
 If @code{-n} is specified, the native AIO backend is used if possible. On
 Linux, this option only works if @code{-t none} or @code{-t directsync} is
 specified as well.
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v2 2/5] qemu-img bench: Sequential writes

[Denis V. Lunev]

On 06/06/2016 03:45 PM, Kevin Wolf wrote:
> This extends qemu-img bench with an option that makes it use sequential
> writes instead of reads for the test run.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>   qemu-img-cmds.hx |  4 ++--
>   qemu-img.c       | 38 +++++++++++++++++++++++++++++++++-----
>   qemu-img.texi    | 13 +++++++++----
>   3 files changed, 44 insertions(+), 11 deletions(-)
>
> diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
> index f3bd546..baca85e 100644
> --- a/qemu-img-cmds.hx
> +++ b/qemu-img-cmds.hx
> @@ -10,9 +10,9 @@ STEXI
>   ETEXI
>   
>   DEF("bench", img_bench,
> -    "bench [-c count] [-d depth] [-f fmt] [-n] [-q] [-s buffer_size] [-t cache] filename")
> +    "bench [-c count] [-d depth] [-f fmt] [-n] [--pattern=pattern] [-q] [-s buffer_size] [-t cache] [-w] filename")
>   STEXI
> -@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-q] [-s @var{buffer_size}] [-t @var{cache}] @var{filename}
> +@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
>   ETEXI
>   
>   DEF("check", img_check,
> diff --git a/qemu-img.c b/qemu-img.c
> index d471d10..85d1353 100644
> --- a/qemu-img.c
> +++ b/qemu-img.c
> @@ -53,6 +53,7 @@ enum {
>       OPTION_BACKING_CHAIN = 257,
>       OPTION_OBJECT = 258,
>       OPTION_IMAGE_OPTS = 259,
> +    OPTION_PATTERN = 260,
>   };
>   
>   typedef enum OutputFormat {
> @@ -3462,6 +3463,7 @@ out_no_progress:
>   typedef struct BenchData {
>       BlockBackend *blk;
>       uint64_t image_size;
> +    bool write;
>       int bufsize;
>       int nrreq;
>       int n;
> @@ -3487,8 +3489,13 @@ static void bench_cb(void *opaque, int ret)
>       }
>   
>       while (b->n > b->in_flight && b->in_flight < b->nrreq) {
> -        acb = blk_aio_preadv(b->blk, b->offset, b->qiov, 0,
> -                             bench_cb, b);
> +        if (b->write) {
> +            acb = blk_aio_pwritev(b->blk, b->offset, b->qiov, 0,
> +                                  bench_cb, b);
> +        } else {
> +            acb = blk_aio_preadv(b->blk, b->offset, b->qiov, 0,
> +                                 bench_cb, b);
> +        }
>           if (!acb) {
>               error_report("Failed to issue request");
>               exit(EXIT_FAILURE);
> @@ -3505,9 +3512,11 @@ static int img_bench(int argc, char **argv)
>       const char *fmt = NULL, *filename;
>       bool quiet = false;
>       bool image_opts = false;
> +    bool is_write = false;
>       int count = 75000;
>       int depth = 64;
>       size_t bufsize = 4096;
> +    int pattern = 0;
>       int64_t image_size;
>       BlockBackend *blk = NULL;
>       BenchData data = {};
> @@ -3520,9 +3529,10 @@ static int img_bench(int argc, char **argv)
>           static const struct option long_options[] = {
>               {"help", no_argument, 0, 'h'},
>               {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
> +            {"pattern", required_argument, 0, OPTION_PATTERN},
>               {0, 0, 0, 0}
>           };
> -        c = getopt_long(argc, argv, "hc:d:f:nqs:t:", long_options, NULL);
> +        c = getopt_long(argc, argv, "hc:d:f:nqs:t:w", long_options, NULL);
>           if (c == -1) {
>               break;
>           }
> @@ -3585,6 +3595,21 @@ static int img_bench(int argc, char **argv)
>                   goto out;
>               }
>               break;
> +        case 'w':
> +            flags |= BDRV_O_RDWR;
> +            is_write = true;
> +            break;
> +        case OPTION_PATTERN:
> +        {
> +            char *end;
> +            errno = 0;
> +            pattern = strtoul(optarg, &end, 0);
> +            if (errno || *end || pattern > 0xff) {
> +                error_report("Invalid pattern byte specified");
> +                return 1;
> +            }
> +            break;
> +        }
>           case OPTION_IMAGE_OPTS:
>               image_opts = true;
>               break;
> @@ -3614,11 +3639,14 @@ static int img_bench(int argc, char **argv)
>           .bufsize    = bufsize,
>           .nrreq      = depth,
>           .n          = count,
> +        .write      = is_write,
>       };
> -    printf("Sending %d requests, %d bytes each, %d in parallel\n",
> -        data.n, data.bufsize, data.nrreq);
> +    printf("Sending %d %s requests, %d bytes each, %d in parallel\n",
> +           data.n, data.write ? "write" : "read", data.bufsize, data.nrreq);
>   
>       data.buf = blk_blockalign(blk, data.nrreq * data.bufsize);
> +    memset(data.buf, pattern, data.nrreq * data.bufsize);
> +
>       data.qiov = g_new(QEMUIOVector, data.nrreq);
>       for (i = 0; i < data.nrreq; i++) {
>           qemu_iovec_init(&data.qiov[i], 1);
> diff --git a/qemu-img.texi b/qemu-img.texi
> index b6b28e3..c477fbf 100644
> --- a/qemu-img.texi
> +++ b/qemu-img.texi
> @@ -131,16 +131,21 @@ Skip the creation of the target volume
>   Command description:
>   
>   @table @option
> -@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-q] [-s @var{buffer_size}] [-t @var{cache}] @var{filename}
> +@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-t @var{cache}] [-w] @var{filename}
>   
> -Run a simple sequential read benchmark on the specified image. A total number
> -of @var{count} I/O requests is performed, each @var{buffer_size} bytes in size,
> -and with @var{depth} requests in parallel.
> +Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
> +specified, a write test is performed, otherwise a read test is performed.
> +
> +A total number of @var{count} I/O requests is performed, each @var{buffer_size}
> +bytes in size, and with @var{depth} requests in parallel.
>   
>   If @code{-n} is specified, the native AIO backend is used if possible. On
>   Linux, this option only works if @code{-t none} or @code{-t directsync} is
>   specified as well.
>   
> +For write tests, by default a buffer filled with zeros is written. This can be
> +overridden with a pattern byte specified by @var{pattern}.
> +
>   @item check [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] @var{filename}
>   
>   Perform a consistency check on the disk image @var{filename}. The command can
Reviewed-by: Denis V. Lunev <den@openvz.org>

Re: [Qemu-devel] [PATCH v2 5/5] qemu-img bench: Add --flush-interval

[Denis V. Lunev]

On 06/06/2016 03:46 PM, Kevin Wolf wrote:
> This options allows to flush the image periodically during write tests.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>   qemu-img-cmds.hx |  4 +--
>   qemu-img.c       | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++------
>   qemu-img.texi    |  8 ++++-
>   3 files changed, 95 insertions(+), 12 deletions(-)
>
> diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
> index 05a2991..7e95b2d 100644
> --- a/qemu-img-cmds.hx
> +++ b/qemu-img-cmds.hx
> @@ -10,9 +10,9 @@ STEXI
>   ETEXI
>   
>   DEF("bench", img_bench,
> -    "bench [-c count] [-d depth] [-f fmt] [-n] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-w] filename")
> +    "bench [-c count] [-d depth] [-f fmt] [--flush-interval=flush_interval] [-n] [--no-drain] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-w] filename")
>   STEXI
> -@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
> +@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
>   ETEXI
>   
>   DEF("check", img_check,
> diff --git a/qemu-img.c b/qemu-img.c
> index c5e2638..04cddab 100644
> --- a/qemu-img.c
> +++ b/qemu-img.c
> @@ -54,6 +54,8 @@ enum {
>       OPTION_OBJECT = 258,
>       OPTION_IMAGE_OPTS = 259,
>       OPTION_PATTERN = 260,
> +    OPTION_FLUSH_INTERVAL = 261,
> +    OPTION_NO_DRAIN = 262,
>   };
>   
>   typedef enum OutputFormat {
> @@ -3468,13 +3470,24 @@ typedef struct BenchData {
>       int step;
>       int nrreq;
>       int n;
> +    int flush_interval;
> +    bool drain_on_flush;
>       uint8_t *buf;
>       QEMUIOVector *qiov;
>   
>       int in_flight;
> +    bool in_flush;
>       uint64_t offset;
>   } BenchData;
>   
> +static void bench_undrained_flush_cb(void *opaque, int ret)
> +{
> +    if (ret < 0) {
> +        error_report("Failed flush request: %s\n", strerror(-ret));
> +        exit(EXIT_FAILURE);
> +    }
> +}
> +
>   static void bench_cb(void *opaque, int ret)
>   {
>       BenchData *b = opaque;
> @@ -3484,9 +3497,39 @@ static void bench_cb(void *opaque, int ret)
>           error_report("Failed request: %s\n", strerror(-ret));
>           exit(EXIT_FAILURE);
>       }
> -    if (b->in_flight > 0) {
> +
> +    if (b->in_flush) {
> +        /* Just finished a flush with drained queue: Start next requests */
> +        assert(b->in_flight == 0);
> +        b->in_flush = false;
> +    } else if (b->in_flight > 0) {
> +        int remaining = b->n - b->in_flight;
> +
>           b->n--;
>           b->in_flight--;
> +
> +        /* Time for flush? Drain queue if requested, then flush */
> +        if (b->flush_interval && remaining % b->flush_interval == 0) {
> +            if (!b->in_flight || !b->drain_on_flush) {
> +                BlockCompletionFunc *cb;
> +
> +                if (b->drain_on_flush) {
> +                    b->in_flush = true;
> +                    cb = bench_cb;
> +                } else {
> +                    cb = bench_undrained_flush_cb;
> +                }
> +
> +                acb = blk_aio_flush(b->blk, cb, b);
> +                if (!acb) {
> +                    error_report("Failed to issue flush request");
> +                    exit(EXIT_FAILURE);
> +                }
> +            }
> +            if (b->drain_on_flush) {
> +                return;
> +            }
> +        }
>       }
>   
>       while (b->n > b->in_flight && b->in_flight < b->nrreq) {
> @@ -3520,6 +3563,8 @@ static int img_bench(int argc, char **argv)
>       size_t bufsize = 4096;
>       int pattern = 0;
>       size_t step = 0;
> +    int flush_interval = 0;
> +    bool drain_on_flush = true;
>       int64_t image_size;
>       BlockBackend *blk = NULL;
>       BenchData data = {};
> @@ -3531,8 +3576,10 @@ static int img_bench(int argc, char **argv)
>       for (;;) {
>           static const struct option long_options[] = {
>               {"help", no_argument, 0, 'h'},
> +            {"flush-interval", required_argument, 0, OPTION_FLUSH_INTERVAL},
>               {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS},
>               {"pattern", required_argument, 0, OPTION_PATTERN},
> +            {"no-drain", no_argument, 0, OPTION_NO_DRAIN},
>               {0, 0, 0, 0}
>           };
>           c = getopt_long(argc, argv, "hc:d:f:no:qs:S:t:w", long_options, NULL);
> @@ -3640,6 +3687,20 @@ static int img_bench(int argc, char **argv)
>               }
>               break;
>           }
> +        case OPTION_FLUSH_INTERVAL:
> +        {
> +            char *end;
> +            errno = 0;
> +            flush_interval = strtoul(optarg, &end, 0);
> +            if (errno || *end || flush_interval > INT_MAX) {
> +                error_report("Invalid flush interval specified");
> +                return 1;
> +            }
> +            break;
> +        }
> +        case OPTION_NO_DRAIN:
> +            drain_on_flush = false;
> +            break;
>           case OPTION_IMAGE_OPTS:
>               image_opts = true;
>               break;
> @@ -3651,6 +3712,17 @@ static int img_bench(int argc, char **argv)
>       }
>       filename = argv[argc - 1];
>   
> +    if (!is_write && flush_interval) {
> +        error_report("--flush-interval is only available in write tests");
> +        ret = -1;
> +        goto out;
> +    }
> +    if (flush_interval && flush_interval < depth) {
> +        error_report("Flush interval can't be smaller than depth");
> +        ret = -1;
> +        goto out;
> +    }
> +
>       blk = img_open(image_opts, filename, fmt, flags, writethrough, quiet);
>       if (!blk) {
>           ret = -1;
> @@ -3664,19 +3736,24 @@ static int img_bench(int argc, char **argv)
>       }
>   
>       data = (BenchData) {
> -        .blk        = blk,
> -        .image_size = image_size,
> -        .bufsize    = bufsize,
> -        .step       = step ?: bufsize,
> -        .nrreq      = depth,
> -        .n          = count,
> -        .offset     = offset,
> -        .write      = is_write,
> +        .blk            = blk,
> +        .image_size     = image_size,
> +        .bufsize        = bufsize,
> +        .step           = step ?: bufsize,
> +        .nrreq          = depth,
> +        .n              = count,
> +        .offset         = offset,
> +        .write          = is_write,
> +        .flush_interval = flush_interval,
> +        .drain_on_flush = drain_on_flush,
>       };
>       printf("Sending %d %s requests, %d bytes each, %d in parallel "
>              "(starting at offset %" PRId64 ", step size %d)\n",
>              data.n, data.write ? "write" : "read", data.bufsize, data.nrreq,
>              data.offset, data.step);
> +    if (flush_interval) {
> +        printf("Sending flush every %d requests\n", flush_interval);
> +    }
>   
>       data.buf = blk_blockalign(blk, data.nrreq * data.bufsize);
>       memset(data.buf, pattern, data.nrreq * data.bufsize);
> diff --git a/qemu-img.texi b/qemu-img.texi
> index ccc0b51..cbe50e9 100644
> --- a/qemu-img.texi
> +++ b/qemu-img.texi
> @@ -131,7 +131,7 @@ Skip the creation of the target volume
>   Command description:
>   
>   @table @option
> -@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [-n] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
> +@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] @var{filename}
>   
>   Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
>   specified, a write test is performed, otherwise a read test is performed.
> @@ -142,6 +142,12 @@ starts at the position given by @var{offset}, each following request increases
>   the current position by @var{step_size}. If @var{step_size} is not given,
>   @var{buffer_size} is used for its value.
>   
> +If @var{flush_interval} is specified for a write test, the request queue is
> +drained and a flush is issued before new writes are made whenever the number of
> +remaining requests is a multiple of @var{flush_interval}. If additionally
> +@code{--no-drain} is specified, a flush is issued without draining the request
> +queue first.
> +
>   If @code{-n} is specified, the native AIO backend is used if possible. On
>   Linux, this option only works if @code{-t none} or @code{-t directsync} is
>   specified as well.
Reviewed-by: Denis V. Lunev <den@openvz.org>

Re: [Qemu-devel] [PATCH v2 0/5] block: Introduce qemu-img bench

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1386 bytes --]

On Mon, Jun 06, 2016 at 02:45:57PM +0200, Kevin Wolf wrote:
> After merging Den's qcow2 patch to avoid duplicated flushes, I thought I would
> be nice to reproduce the problem and I remembered 'qemu-img bench', which I had
> posted before as part of more than one RFC series, but which never made it to
> master somehow. So here is a rebased and cleaned up version of it, just by
> itself, so that it hopefully can be merged finally.
> 
> Of course, I failed to actually reproduce the problem on my laptop. Who knows,
> something on my system might be more intelligent about useless flushes, or
> maybe I just misunderstood what the problematic scenario looks like at the
> block level. Doesn't make the tool less useful, though, and I already did the
> rebasing.
> 
> v2:
> - Added --pattern=... option for writes [Den]
> - Added --no-drain option for write+flush tests [Den]
> 
> Kevin Wolf (5):
>   qemu-img bench
>   qemu-img bench: Sequential writes
>   qemu-img bench: Make start offset configurable
>   qemu-img bench: Implement -S (step size)
>   qemu-img bench: Add --flush-interval
> 
>  qemu-img-cmds.hx |   6 +
>  qemu-img.c       | 329 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  qemu-img.texi    |  24 ++++
>  3 files changed, 359 insertions(+)
> 
> -- 
> 1.8.3.1
> 
> 

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] [PATCH v2 0/5] block: Introduce qemu-img bench

[Kevin Wolf]

Am 06.06.2016 um 14:45 hat Kevin Wolf geschrieben:
> After merging Den's qcow2 patch to avoid duplicated flushes, I thought I would
> be nice to reproduce the problem and I remembered 'qemu-img bench', which I had
> posted before as part of more than one RFC series, but which never made it to
> master somehow. So here is a rebased and cleaned up version of it, just by
> itself, so that it hopefully can be merged finally.
> 
> Of course, I failed to actually reproduce the problem on my laptop. Who knows,
> something on my system might be more intelligent about useless flushes, or
> maybe I just misunderstood what the problematic scenario looks like at the
> block level. Doesn't make the tool less useful, though, and I already did the
> rebasing.

Thanks for the review, applied to the block branch.

Kevin