* [PATCH 0/3] three patches
@ 2019-11-11 16:30 vincentfu
2019-11-11 16:30 ` [PATCH 1/3] filesetup: improve LFSR init failure error message vincentfu
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: vincentfu @ 2019-11-11 16:30 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
From: Vincent Fu <vincent.fu@wdc.com>
Jens, please consider this patch series.
The LFSR error message and test script error handling changes I believe
should be uncontroversial.
Regarding the zonemode=strided patch, it will break job files that
depend on the old buggy behavior, but that behavior is so inconsistent
with the documentation that I think this fix is warranted.
Vincent
Vincent Fu (3):
filesetup: improve LFSR init failure error message
io_u: move to next zone even if zoneskip is unset
t/run-fio-tests: improve error handling
filesetup.c | 3 +++
io_u.c | 3 +--
t/run-fio-tests.py | 10 ++++++----
t/strided.py | 3 +--
4 files changed, 11 insertions(+), 8 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/3] filesetup: improve LFSR init failure error message
2019-11-11 16:30 [PATCH 0/3] three patches vincentfu
@ 2019-11-11 16:30 ` vincentfu
2019-11-11 16:30 ` [PATCH 2/3] io_u: move to next zone even if zoneskip is unset vincentfu
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: vincentfu @ 2019-11-11 16:30 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
From: Vincent Fu <vincent.fu@wdc.com>
Especially with small sample spaces, the LFSR random generator
occasionally fails to initialize successfully. When this occurs, the
error message refers to problems allocating a random map. Change the
error message to explicitly mention the LFSR failure.
OLD BEHAVIOR
$ ./fio --name=test --ioengine=null --size=4k --random_generator=lfsr --rw=randread --randrepeat=0
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=null, iodepth=1
fio-3.16-13-g998b-dirty
Starting 1 process
fio: failed allocating random map. If running a large number of jobs, try the 'norandommap' option or set 'softrandommap'. Or give a larger --alloc-size to fio.
NEW BEHAVIOR
$ ./fio --name=test --ioengine=null --size=4k --random_generator=lfsr --rw=randread --randrepeat=0
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=null, iodepth=1
fio-3.16-37-g65ee
Starting 1 process
fio: failed initializing LFSR
---
filesetup.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/filesetup.c b/filesetup.c
index 1d3094c1..7d54c9f1 100644
--- a/filesetup.c
+++ b/filesetup.c
@@ -1354,6 +1354,9 @@ bool init_random_map(struct thread_data *td)
if (!lfsr_init(&f->lfsr, blocks, seed, 0)) {
fio_file_set_lfsr(f);
continue;
+ } else {
+ log_err("fio: failed initializing LFSR\n");
+ return false;
}
} else if (!td->o.norandommap) {
f->io_axmap = axmap_new(blocks);
--
2.17.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/3] io_u: move to next zone even if zoneskip is unset
2019-11-11 16:30 [PATCH 0/3] three patches vincentfu
2019-11-11 16:30 ` [PATCH 1/3] filesetup: improve LFSR init failure error message vincentfu
@ 2019-11-11 16:30 ` vincentfu
2019-11-11 16:30 ` [PATCH 3/3] t/run-fio-tests: improve error handling vincentfu
2019-11-14 21:07 ` [PATCH 0/3] three patches Jens Axboe
3 siblings, 0 replies; 5+ messages in thread
From: vincentfu @ 2019-11-11 16:30 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
From: Vincent Fu <vincent.fu@wdc.com>
This patch makes fio's behavior under zonemode=strided conform to the
documentation:
I/O happens in a single zone until zonesize bytes have been
transferred. After that number of bytes has been transferred
processing of the next zone starts.
With zonemode=strided, before commit 35f561eb, fio would only move to
the next zone when zoneskip > 0 and zonesize bytes were written. There
would always be zoneskip bytes between the end of one zone and the
beginning of the next zone. If zoneskip was not set or set to 0, all IO
would happen in the first zone.
Commit 35f561eb changed this so that fio would move to the next zone
upon writing zonesize bytes if zoneskip was explicitly set to a value >=
0. This option made it possible for zones to be contiguous. The
documentation was not updated to reflect the new behavior.
I originally intended to submit a patch to update fio's documentation,
but upon further reflection it seems better to change fio's behavior and
have a clean user interface than to change the documentation to note
that zoneskip must be explciitly set in order for fio to move to the
next zone.
This patch also updates t/strided.py to reflect the new behavior.
---
io_u.c | 3 +--
t/strided.py | 3 +--
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/io_u.c b/io_u.c
index 5cbbe85a..b5c31335 100644
--- a/io_u.c
+++ b/io_u.c
@@ -850,8 +850,7 @@ static void setup_strided_zone_mode(struct thread_data *td, struct io_u *io_u)
/*
* See if it's time to switch to a new zone
*/
- if (td->zone_bytes >= td->o.zone_size &&
- fio_option_is_set(&td->o, zone_skip)) {
+ if (td->zone_bytes >= td->o.zone_size) {
td->zone_bytes = 0;
f->file_offset += td->o.zone_range + td->o.zone_skip;
diff --git a/t/strided.py b/t/strided.py
index c159dc0b..aac15d10 100755
--- a/t/strided.py
+++ b/t/strided.py
@@ -22,7 +22,7 @@
#
# ===TEST MATRIX===
#
-# --zonemode=strided, zoneskip >= 0
+# --zonemode=strided, zoneskip unset
# w/ randommap and LFSR
# zonesize=zonerange all blocks in zonerange touched
# zonesize>zonerange all blocks touched and roll-over back into zone
@@ -57,7 +57,6 @@ def run_fio(fio, test, index):
"--log_offset=1",
"--randrepeat=0",
"--rw=randread",
- "--zoneskip=0",
"--write_iops_log={0}{1:03d}".format(filename, index),
"--output={0}{1:03d}.out".format(filename, index),
"--zonerange={zonerange}".format(**test),
--
2.17.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/3] t/run-fio-tests: improve error handling
2019-11-11 16:30 [PATCH 0/3] three patches vincentfu
2019-11-11 16:30 ` [PATCH 1/3] filesetup: improve LFSR init failure error message vincentfu
2019-11-11 16:30 ` [PATCH 2/3] io_u: move to next zone even if zoneskip is unset vincentfu
@ 2019-11-11 16:30 ` vincentfu
2019-11-14 21:07 ` [PATCH 0/3] three patches Jens Axboe
3 siblings, 0 replies; 5+ messages in thread
From: vincentfu @ 2019-11-11 16:30 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
From: Vincent Fu <vincent.fu@wdc.com>
More gracefully handle errors when running a test. Also change the
EXAMPLE to refer to fio's canonical git repository.
---
t/run-fio-tests.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/t/run-fio-tests.py b/t/run-fio-tests.py
index 1b8ca0a2..cf8de093 100755
--- a/t/run-fio-tests.py
+++ b/t/run-fio-tests.py
@@ -14,7 +14,7 @@
#
#
# EXAMPLE
-# # git clone [fio-repository]
+# # git clone git://git.kernel.dk/fio.git
# # cd fio
# # make -j
# # python3 t/run-fio-tests.py
@@ -123,6 +123,7 @@ class FioExeTest(FioTest):
stderr_file = open(self.stderr_file, "w+")
exticode_file = open(self.exticode_file, "w+")
try:
+ proc = None
# Avoid using subprocess.run() here because when a timeout occurs,
# fio will be stopped with SIGKILL. This does not give fio a
# chance to clean up and means that child processes may continue
@@ -142,9 +143,10 @@ class FioExeTest(FioTest):
assert proc.poll()
self.output['failure'] = 'timeout'
except Exception:
- if not proc.poll():
- proc.terminate()
- proc.communicate()
+ if proc:
+ if not proc.poll():
+ proc.terminate()
+ proc.communicate()
self.output['failure'] = 'exception'
self.output['exc_info'] = sys.exc_info()
finally:
--
2.17.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 0/3] three patches
2019-11-11 16:30 [PATCH 0/3] three patches vincentfu
` (2 preceding siblings ...)
2019-11-11 16:30 ` [PATCH 3/3] t/run-fio-tests: improve error handling vincentfu
@ 2019-11-14 21:07 ` Jens Axboe
3 siblings, 0 replies; 5+ messages in thread
From: Jens Axboe @ 2019-11-14 21:07 UTC (permalink / raw)
To: vincentfu, fio; +Cc: Vincent Fu
On 11/11/19 9:30 AM, vincentfu@gmail.com wrote:
> From: Vincent Fu <vincent.fu@wdc.com>
>
> Jens, please consider this patch series.
>
> The LFSR error message and test script error handling changes I believe
> should be uncontroversial.
>
> Regarding the zonemode=strided patch, it will break job files that
> depend on the old buggy behavior, but that behavior is so inconsistent
> with the documentation that I think this fix is warranted.
Agree, and I don't think that's a high risk.
Applied, thanks.
--
Jens Axboe
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-11-14 21:07 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-11 16:30 [PATCH 0/3] three patches vincentfu
2019-11-11 16:30 ` [PATCH 1/3] filesetup: improve LFSR init failure error message vincentfu
2019-11-11 16:30 ` [PATCH 2/3] io_u: move to next zone even if zoneskip is unset vincentfu
2019-11-11 16:30 ` [PATCH 3/3] t/run-fio-tests: improve error handling vincentfu
2019-11-14 21:07 ` [PATCH 0/3] three patches Jens Axboe
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.