From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from merlin.infradead.org ([205.233.59.134]:55272 "EHLO merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752190AbdJ0MAV (ORCPT ); Fri, 27 Oct 2017 08:00:21 -0400 Received: from [216.160.245.99] (helo=kernel.dk) by merlin.infradead.org with esmtpsa (Exim 4.87 #1 (Red Hat Linux)) id 1e83JE-0005cJ-Ey for fio@vger.kernel.org; Fri, 27 Oct 2017 12:00:20 +0000 Subject: Recent changes (master) From: Jens Axboe Message-Id: <20171027120002.1456B2C0094@kernel.dk> Date: Fri, 27 Oct 2017 06:00:02 -0600 (MDT) Sender: fio-owner@vger.kernel.org List-Id: fio@vger.kernel.org To: fio@vger.kernel.org The following changes since commit ff523a66e5af357e67602caf33de1e2cd0521b08: parse: minimum options values are signed (2017-10-25 13:06:40 -0600) are available in the git repository at: git://git.kernel.dk/fio.git master for you to fetch changes up to 11fd6aa8569c55c8488020e4e315d550d121ff79: Fix 'nice' parameter range: should be -20 to 19, not -19 to 20. (2017-10-26 15:31:35 -0600) ---------------------------------------------------------------- Jeff Furlong (1): Add offset_align option Jens Axboe (1): io_u: re-invalidate cache when looping around without file open/close Rebecca Cran (1): Fix 'nice' parameter range: should be -20 to 19, not -19 to 20. HOWTO | 11 +++++++++-- cconv.c | 2 ++ filesetup.c | 8 +++----- fio.1 | 10 ++++++++-- io_u.c | 14 ++++++++++++++ options.c | 15 +++++++++++++-- thread_options.h | 2 ++ 7 files changed, 51 insertions(+), 11 deletions(-) --- Diff of recent changes: diff --git a/HOWTO b/HOWTO index 22a5849..e7142c5 100644 --- a/HOWTO +++ b/HOWTO @@ -1128,13 +1128,20 @@ I/O type .. option:: offset=int Start I/O at the provided offset in the file, given as either a fixed size in - bytes or a percentage. If a percentage is given, the next ``blockalign``-ed - offset will be used. Data before the given offset will not be touched. This + bytes or a percentage. If a percentage is given, the generated offset will be + aligned to the minimum ``blocksize`` or to the value of ``offset_align`` if + provided. Data before the given offset will not be touched. This effectively caps the file size at `real_size - offset`. Can be combined with :option:`size` to constrain the start and end range of the I/O workload. A percentage can be specified by a number between 1 and 100 followed by '%', for example, ``offset=20%`` to specify 20%. +.. option:: offset_align=int + + If set to non-zero value, the byte offset generated by a percentage ``offset`` + is aligned upwards to this value. Defaults to 0 meaning that a percentage + offset is aligned to the minimum block size. + .. option:: offset_increment=int If this is provided, then the real offset becomes `offset + offset_increment diff --git a/cconv.c b/cconv.c index f809fd5..dc3c4e6 100644 --- a/cconv.c +++ b/cconv.c @@ -105,6 +105,7 @@ void convert_thread_options_to_cpu(struct thread_options *o, o->file_size_low = le64_to_cpu(top->file_size_low); o->file_size_high = le64_to_cpu(top->file_size_high); o->start_offset = le64_to_cpu(top->start_offset); + o->start_offset_align = le64_to_cpu(top->start_offset_align); o->start_offset_percent = le32_to_cpu(top->start_offset_percent); for (i = 0; i < DDIR_RWDIR_CNT; i++) { @@ -548,6 +549,7 @@ void convert_thread_options_to_net(struct thread_options_pack *top, top->file_size_low = __cpu_to_le64(o->file_size_low); top->file_size_high = __cpu_to_le64(o->file_size_high); top->start_offset = __cpu_to_le64(o->start_offset); + top->start_offset_align = __cpu_to_le64(o->start_offset_align); top->start_offset_percent = __cpu_to_le32(o->start_offset_percent); top->trim_backlog = __cpu_to_le64(o->trim_backlog); top->offset_increment = __cpu_to_le64(o->offset_increment); diff --git a/filesetup.c b/filesetup.c index 7a602d4..5d7ea5c 100644 --- a/filesetup.c +++ b/filesetup.c @@ -869,12 +869,10 @@ uint64_t get_start_offset(struct thread_data *td, struct fio_file *f) if (o->start_offset_percent > 0) { /* - * if blockalign is provided, find the min across read, write, - * and trim + * if offset_align is provided, set initial offset */ - if (fio_option_is_set(o, ba)) { - align_bs = (unsigned long long) min(o->ba[DDIR_READ], o->ba[DDIR_WRITE]); - align_bs = min((unsigned long long) o->ba[DDIR_TRIM], align_bs); + if (fio_option_is_set(o, start_offset_align)) { + align_bs = o->start_offset_align; } else { /* else take the minimum block size */ align_bs = td_min_bs(td); diff --git a/fio.1 b/fio.1 index 7787ef2..96d8f11 100644 --- a/fio.1 +++ b/fio.1 @@ -913,13 +913,19 @@ should be associated with them. .TP .BI offset \fR=\fPint Start I/O at the provided offset in the file, given as either a fixed size in -bytes or a percentage. If a percentage is given, the next \fBblockalign\fR\-ed -offset will be used. Data before the given offset will not be touched. This +bytes or a percentage. If a percentage is given, the generated offset will be +aligned to the minimum \fBblocksize\fR or to the value of \fBoffset_align\fR if +provided. Data before the given offset will not be touched. This effectively caps the file size at `real_size \- offset'. Can be combined with \fBsize\fR to constrain the start and end range of the I/O workload. A percentage can be specified by a number between 1 and 100 followed by '%', for example, `offset=20%' to specify 20%. .TP +.BI offset_align \fR=\fPint +If set to non-zero value, the byte offset generated by a percentage \fBoffset\fR +is aligned upwards to this value. Defaults to 0 meaning that a percentage +offset is aligned to the minimum block size. +.TP .BI offset_increment \fR=\fPint If this is provided, then the real offset becomes `\fBoffset\fR + \fBoffset_increment\fR * thread_number', where the thread number is a counter that starts at 0 and diff --git a/io_u.c b/io_u.c index fb4180a..4246edf 100644 --- a/io_u.c +++ b/io_u.c @@ -323,6 +323,17 @@ fetch: goto fetch; } +static void loop_cache_invalidate(struct thread_data *td, struct fio_file *f) +{ + struct thread_options *o = &td->o; + + if (o->invalidate_cache && !o->odirect) { + int fio_unused ret; + + ret = file_invalidate_cache(td, f); + } +} + static int get_next_rand_block(struct thread_data *td, struct fio_file *f, enum fio_ddir ddir, uint64_t *b) { @@ -334,6 +345,7 @@ static int get_next_rand_block(struct thread_data *td, struct fio_file *f, fio_file_reset(td, f); if (!get_next_rand_offset(td, f, ddir, b)) return 0; + loop_cache_invalidate(td, f); } dprint(FD_IO, "%s: rand offset failed, last=%llu, size=%llu\n", @@ -358,6 +370,8 @@ static int get_next_seq_offset(struct thread_data *td, struct fio_file *f, f->last_pos[ddir] = 0; else f->last_pos[ddir] = f->last_pos[ddir] - io_size; + + loop_cache_invalidate(td, f); } if (f->last_pos[ddir] < f->real_file_size) { diff --git a/options.c b/options.c index ddcc4e5..5813a66 100644 --- a/options.c +++ b/options.c @@ -2019,6 +2019,17 @@ struct fio_option fio_options[FIO_MAX_OPTS] = { .group = FIO_OPT_G_INVALID, }, { + .name = "offset_align", + .lname = "IO offset alignment", + .type = FIO_OPT_INT, + .off1 = offsetof(struct thread_options, start_offset_align), + .help = "Start IO from this offset alignment", + .def = "0", + .interval = 512, + .category = FIO_OPT_C_IO, + .group = FIO_OPT_G_INVALID, + }, + { .name = "offset_increment", .lname = "IO offset increment", .type = FIO_OPT_STR_VAL, @@ -3241,8 +3252,8 @@ struct fio_option fio_options[FIO_MAX_OPTS] = { .type = FIO_OPT_INT, .off1 = offsetof(struct thread_options, nice), .help = "Set job CPU nice value", - .minval = -19, - .maxval = 20, + .minval = -20, + .maxval = 19, .def = "0", .interval = 1, .category = FIO_OPT_C_GENERAL, diff --git a/thread_options.h b/thread_options.h index 1813cdc..5a037bf 100644 --- a/thread_options.h +++ b/thread_options.h @@ -78,6 +78,7 @@ struct thread_options { unsigned long long file_size_low; unsigned long long file_size_high; unsigned long long start_offset; + unsigned long long start_offset_align; unsigned int bs[DDIR_RWDIR_CNT]; unsigned int ba[DDIR_RWDIR_CNT]; @@ -355,6 +356,7 @@ struct thread_options_pack { uint64_t file_size_low; uint64_t file_size_high; uint64_t start_offset; + uint64_t start_offset_align; uint32_t bs[DDIR_RWDIR_CNT]; uint32_t ba[DDIR_RWDIR_CNT];