[PATCH 0/7] Sync fio(1) man page with HOWTO

[PATCH 0/7] Sync fio(1) man page with HOWTO

[kusumi.tomohiro]

From: Tomohiro Kusumi <tkusumi@tuxera.com>

The man page hasn't been in sync with many of the updates made for
HOWTO. These commits update the man page based on what HOWTO has,
where each commit corresponds to a single section (instead of updating
with commit-to-commit basis). The man page being in sync with HOWTO
also makes it easier to make changes, whenever HOWTO is updated in the
future.

The goal is to be in sync with HOWTO without being too verbose for
a man page. Note that they don't include contents that do not exist
in HOWTO (i.e. I didn't newly create anything). Also note these are
only beginning part of the man page.

These commits include
1. Import updates made only for HOWTO (diffs), unless the changes are
   too long/verbose to be mentioned in the man page.
2. Replace the entire section or paragraph with the corresponding one
   in HOWTO (partly as a result of 1), if these two are essencially
   saying the same thing with small modification, so that we don't see
   two different contents in two documents that are almost the same.
3. Make some small changes when above two are imported, such as using
   directives, removal of double white spaces, removal of expressions
   that don't fit in well with a man page (e.g. This section describes...).
4. Backport some small changes from the man page to HOWTO, which seem
   to have been made when the man page was first introduced. These are
   mostly minor fixups.

Tomohiro Kusumi (7):
  man: sync OPTIONS section with HOWTO
  man: sync "JOB FILE FORMAT" section with HOWTO
  man: sync "JOB FILE PARAMETERS" section with HOWTO
  man: sync "PARAMETER TYPES" section with HOWTO
  man: refer to REPORTING-BUGS for bug reporting
  HOWTO: minor backports from the man page
  HOWTO: fix indentation for options and job parameters

 HOWTO  | 159 +++++++++++++++++++++--------------------
 README |   3 +-
 fio.1  | 250 ++++++++++++++++++++++++++++++++++++++++++++---------------------
 3 files changed, 251 insertions(+), 161 deletions(-)

-- 
2.9.4

[PATCH 1/7] man: sync OPTIONS section with HOWTO

[kusumi.tomohiro]

From: Tomohiro Kusumi <tkusumi@tuxera.com>

This commit brings in updates from HOWTO. The exception is --debug
option, which is too verbose yet not that important for all debug types
to be listed in details.

Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
---
 fio.1 | 116 +++++++++++++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 87 insertions(+), 29 deletions(-)

diff --git a/fio.1 b/fio.1
index 9783646..016fb67 100644
--- a/fio.1
+++ b/fio.1
@@ -1,4 +1,4 @@
-.TH fio 1 "June 2017" "User Manual"
+.TH fio 1 "July 2017" "User Manual"
 .SH NAME
 fio \- flexible I/O tester
 .SH SYNOPSIS
@@ -14,8 +14,11 @@ one wants to simulate.
 .TP
 .BI \-\-debug \fR=\fPtype
 Enable verbose tracing of various fio actions. May be `all' for all types
-or individual types separated by a comma (eg \-\-debug=io,file). `help' will
-list all available tracing options.
+or individual types separated by a comma (e.g. \-\-debug=file,mem will enable
+file and memory debugging). `help' will list all available tracing options.
+.TP
+.BI \-\-parse-only
+Parse options only, don't start any I/O.
 .TP
 .BI \-\-output \fR=\fPfilename
 Write output to \fIfilename\fR.
@@ -39,68 +42,123 @@ Print statistics in a terse, semicolon-delimited format.
 Print statistics in selected mode AND terse, semicolon-delimited format.
 Deprecated, use \-\-output-format instead to select multiple formats.
 .TP
-.B \-\-version
-Display version information and exit.
-.TP
 .BI \-\-terse\-version \fR=\fPversion
 Set terse version output format (default 3, or 2, 4, 5)
 .TP
+.B \-\-version
+Print version information and exit.
+.TP
 .B \-\-help
-Display usage information and exit.
+Print a summary of the command line options and exit.
 .TP
 .B \-\-cpuclock-test
-Perform test and validation of internal CPU clock
+Perform test and validation of internal CPU clock.
 .TP
-.BI \-\-crctest[\fR=\fPtest]
-Test the speed of the builtin checksumming functions. If no argument is given,
-all of them are tested. Or a comma separated list can be passed, in which
+.BI \-\-crctest \fR=\fP[test]
+Test the speed of the built-in checksumming functions. If no argument is given,
+all of them are tested. Alternatively, a comma separated list can be passed, in which
 case the given ones are tested.
 .TP
 .BI \-\-cmdhelp \fR=\fPcommand
-Print help information for \fIcommand\fR.  May be `all' for all commands.
+Print help information for \fIcommand\fR. May be `all' for all commands.
 .TP
 .BI \-\-enghelp \fR=\fPioengine[,command]
 List all commands defined by \fIioengine\fR, or print help for \fIcommand\fR defined by \fIioengine\fR.
+If no \fIioengine\fR is given, list all available ioengines.
 .TP
 .BI \-\-showcmd \fR=\fPjobfile
 Convert \fIjobfile\fR to a set of command-line options.
 .TP
+.BI \-\-readonly
+Turn on safety read-only checks, preventing writes. The \-\-readonly
+option is an extra safety guard to prevent users from accidentally starting
+a write workload when that is not desired. Fio will only write if
+`rw=write/randwrite/rw/randrw` is given. This extra safety net can be used
+as an extra precaution as \-\-readonly will also enable a write check in
+the I/O engine core to prevent writes due to unknown user space bug(s).
+.TP
 .BI \-\-eta \fR=\fPwhen
-Specifies when real-time ETA estimate should be printed.  \fIwhen\fR may
-be one of `always', `never' or `auto'.
+Specifies when real-time ETA estimate should be printed. \fIwhen\fR may
+be `always', `never' or `auto'.
 .TP
 .BI \-\-eta\-newline \fR=\fPtime
-Force an ETA newline for every `time` period passed.
+Force a new line for every \fItime\fR period passed. When the unit is omitted,
+the value is interpreted in seconds.
 .TP
 .BI \-\-status\-interval \fR=\fPtime
-Report full output status every `time` period passed.
-.TP
-.BI \-\-readonly
-Turn on safety read-only checks, preventing any attempted write.
-.TP
-.BI \-\-section \fR=\fPsec
-Only run section \fIsec\fR from job file. This option can be used multiple times to add more sections to run.
+Force full status dump every \fItime\fR period passed. When the unit is omitted,
+the value is interpreted in seconds.
+.TP
+.BI \-\-section \fR=\fPname
+Only run specified section \fIname\fR in job file. Multiple sections can be specified.
+The \-\-section option allows one to combine related jobs into one file.
+E.g. one job file could define light, moderate, and heavy sections. Tell
+fio to run only the "heavy" section by giving \-\-section=heavy
+command line option. One can also specify the "write" operations in one
+section and "verify" operation in another section. The \-\-section option
+only applies to job sections. The reserved *global* section is always
+parsed and used.
 .TP
 .BI \-\-alloc\-size \fR=\fPkb
-Set the internal smalloc pool size to \fIkb\fP kilobytes.
+Set the internal smalloc pool size to \fIkb\fP in KiB. The
+\-\-alloc-size switch allows one to use a larger pool size for smalloc.
+If running large jobs with randommap enabled, fio can run out of memory.
+Smalloc is an internal allocator for shared structures from a fixed size
+memory pool and can grow to 16 pools. The pool size defaults to 16MiB.
+NOTE: While running .fio_smalloc.* backing store files are visible
+in /tmp.
 .TP
 .BI \-\-warnings\-fatal
 All fio parser warnings are fatal, causing fio to exit with an error.
 .TP
 .BI \-\-max\-jobs \fR=\fPnr
-Set the maximum allowed number of jobs (threads/processes) to support.
+Set the maximum number of threads/processes to support.
 .TP
 .BI \-\-server \fR=\fPargs
-Start a backend server, with \fIargs\fP specifying what to listen to. See client/server section.
+Start a backend server, with \fIargs\fP specifying what to listen to. See Client/Server section.
 .TP
 .BI \-\-daemonize \fR=\fPpidfile
-Background a fio server, writing the pid to the given pid file.
+Background a fio server, writing the pid to the given \fIpidfile\fP file.
+.TP
+.BI \-\-client \fR=\fPhostname
+Instead of running the jobs locally, send and run them on the given host or set of hosts. See Client/Server section.
 .TP
-.BI \-\-client \fR=\fPhost
-Instead of running the jobs locally, send and run them on the given host or set of hosts.  See client/server section.
+.BI \-\-remote-config \fR=\fPfile
+Tell fio server to load this local file.
 .TP
 .BI \-\-idle\-prof \fR=\fPoption
-Report cpu idleness on a system or percpu basis (\fIoption\fP=system,percpu) or run unit work calibration only (\fIoption\fP=calibrate).
+Report CPU idleness. \fIoption\fP is one of the following:
+.RS
+.RS
+.TP
+.B calibrate
+Run unit work calibration only and exit.
+.TP
+.B system
+Show aggregate system idleness and unit work.
+.TP
+.B percpu
+As "system" but also show per CPU idleness.
+.RE
+.RE
+.TP
+.BI \-\-inflate-log \fR=\fPlog
+Inflate and output compressed log.
+.TP
+.BI \-\-trigger-file \fR=\fPfile
+Execute trigger cmd when file exists.
+.TP
+.BI \-\-trigger-timeout \fR=\fPt
+Execute trigger at this time.
+.TP
+.BI \-\-trigger \fR=\fPcmd
+Set this command as local trigger.
+.TP
+.BI \-\-trigger-remote \fR=\fPcmd
+Set this command as remote trigger.
+.TP
+.BI \-\-aux-path \fR=\fPpath
+Use this path for fio state generated files.
 .SH "JOB FILE FORMAT"
 Job files are in `ini' format. They consist of one or more
 job definitions, which begin with a job name in square brackets and
-- 
2.9.4

[PATCH 2/7] man: sync "JOB FILE FORMAT" section with HOWTO

[kusumi.tomohiro]

From: Tomohiro Kusumi <tkusumi@tuxera.com>

This commit brings in updates from HOWTO to man page. As a result it
replaces some entire paragraphs with the corresponding ones in HOWTO,
though not much difference for the replaced part as it only had small
modification basically saying the same thing.

Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
---
 fio.1 | 39 ++++++++++++++++++++++++---------------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/fio.1 b/fio.1
index 016fb67..b22989a 100644
--- a/fio.1
+++ b/fio.1
@@ -160,21 +160,30 @@ Set this command as remote trigger.
 .BI \-\-aux-path \fR=\fPpath
 Use this path for fio state generated files.
 .SH "JOB FILE FORMAT"
-Job files are in `ini' format. They consist of one or more
-job definitions, which begin with a job name in square brackets and
-extend to the next job name.  The job name can be any ASCII string
-except `global', which has a special meaning.  Following the job name is
-a sequence of zero or more parameters, one per line, that define the
-behavior of the job.  Any line starting with a `;' or `#' character is
-considered a comment and ignored.
-.P
-If \fIjobfile\fR is specified as `-', the job file will be read from
-standard input.
-.SS "Global Section"
-The global section contains default parameters for jobs specified in the
-job file.  A job is only affected by global sections residing above it,
-and there may be any number of global sections.  Specific job definitions
-may override any parameter set in global sections.
+Any parameters following the options will be assumed to be job files, unless
+they match a job file parameter. Multiple job files can be listed and each job
+file will be regarded as a separate group. Fio will `stonewall` execution
+between each group.
+
+Fio accepts one or more job files describing what it is
+supposed to do. The job file format is the classic ini file, where the names
+enclosed in [] brackets define the job name. You are free to use any ASCII name
+you want, except *global* which has special meaning. Following the job name is
+a sequence of zero or more parameters, one per line, that define the behavior of
+the job. If the first character in a line is a ';' or a '#', the entire line is
+discarded as a comment.
+
+A *global* section sets defaults for the jobs described in that file. A job may
+override a *global* section parameter, and a job file may even have several
+*global* sections if so desired. A job is only affected by a *global* section
+residing above it.
+
+The \-\-cmdhelp option also lists all options. If used with an `option`
+argument, \-\-cmdhelp will detail the given `option`.
+
+See the `examples/` directory in the fio source for inspiration on how to write
+job files. Note the copyright and license requirements currently apply to
+`examples/` files.
 .SH "JOB PARAMETERS"
 .SS Types
 Some parameters may take arguments of a specific type.
-- 
2.9.4

[PATCH 3/7] man: sync "JOB FILE PARAMETERS" section with HOWTO

[kusumi.tomohiro]

From: Tomohiro Kusumi <tkusumi@tuxera.com>

This commit replaces the entire section with the corresponding section
in HOWTO, since the man page is only slightly different from HOWTO
regarding this section.

It also renames the following section names to sync with HOWTO.
  "JOB PARAMETERS" to "JOB FILE PARAMETERS"
  "JOB PARAMETERS"'s sub section "Types" to non-sub section "PARAMETER TYPES"
  "JOB PARAMETERS"'s sub section "Parameter List" to non-sub section "JOB DESCRIPTION"

Since section names aren't following the typical man page convention
(man-pages(7) in Linux) to begin with, changing the section names
to conform to HOWTO does no harm unless there's a reason to keep names
that were once made.

Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
---
 fio.1 | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/fio.1 b/fio.1
index b22989a..629ab01 100644
--- a/fio.1
+++ b/fio.1
@@ -184,11 +184,10 @@ argument, \-\-cmdhelp will detail the given `option`.
 See the `examples/` directory in the fio source for inspiration on how to write
 job files. Note the copyright and license requirements currently apply to
 `examples/` files.
-.SH "JOB PARAMETERS"
-.SS Types
-Some parameters may take arguments of a specific type.
-Anywhere a numeric value is required, an arithmetic expression may be used,
-provided it is surrounded by parentheses. Supported operators are:
+.SH "JOB FILE PARAMETERS"
+Some parameters take an option of a given type, such as an integer or a
+string. Anywhere a numeric value is required, an arithmetic expression may be
+used, provided it is surrounded by parentheses. Supported operators are:
 .RS
 .RS
 .TP
@@ -208,7 +207,9 @@ provided it is surrounded by parentheses. Supported operators are:
 .P
 For time values in expressions, units are microseconds by default. This is
 different than for time values not in expressions (not enclosed in
-parentheses). The types used are:
+parentheses).
+.SH "PARAMETER TYPES"
+The following parameter types are used.
 .TP
 .I str
 String: a sequence of alphanumeric characters.
@@ -299,7 +300,8 @@ sets of ranges, they are separated with a `,' or `/' character. For example:
 .I float_list
 List of floating numbers: A list of floating numbers, separated by
 a ':' character.
-.SS "Parameter List"
+.SH "JOB DESCRIPTION"
+With the above in mind, here follows the complete list of fio job parameters.
 .TP
 .BI name \fR=\fPstr
 May be used to override the job name.  On the command line, this parameter
-- 
2.9.4

[PATCH 4/7] man: sync "PARAMETER TYPES" section with HOWTO

[kusumi.tomohiro]

From: Tomohiro Kusumi <tkusumi@tuxera.com>

This commit brings in updates from HOWTO to man page. As a result it
replaces some entire paragraphs with the corresponding ones in HOWTO,
though not much difference for the replaced part as it only had small
modification basically saying the same thing.

Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
---
 fio.1 | 74 ++++++++++++++++++++++++++++++++++++++++---------------------------
 1 file changed, 44 insertions(+), 30 deletions(-)

diff --git a/fio.1 b/fio.1
index 629ab01..1fd928f 100644
--- a/fio.1
+++ b/fio.1
@@ -212,25 +212,32 @@ parentheses).
 The following parameter types are used.
 .TP
 .I str
-String: a sequence of alphanumeric characters.
+String. A sequence of alphanumeric characters.
+.TP
+.I time
+Integer with possible time suffix. Without a unit value is interpreted as
+seconds unless otherwise specified. Accepts a suffix of 'd' for days, 'h' for
+hours, 'm' for minutes, 's' for seconds, 'ms' (or 'msec') for milliseconds and 'us'
+(or 'usec') for microseconds. For example, use 10m for 10 minutes.
 .TP
 .I int
 Integer. A whole number value, which may contain an integer prefix
 and an integer suffix.
 
-[integer prefix]number[integer suffix]
+[*integer prefix*] **number** [*integer suffix*]
+
+The optional *integer prefix* specifies the number's base. The default
+is decimal. *0x* specifies hexadecimal.
 
-The optional integer prefix specifies the number's base. The default
-is decimal. 0x specifies hexadecimal.
+The optional *integer suffix* specifies the number's units, and includes an
+optional unit prefix and an optional unit. For quantities of data, the
+default unit is bytes. For quantities of time, the default unit is seconds
+unless otherwise specified.
 
-The optional integer suffix specifies the number's units, and includes
-an optional unit prefix and an optional unit.  For quantities
-of data, the default unit is bytes. For quantities of time,
-the default unit is seconds.
+With \fBkb_base=1000\fR, fio follows international standards for unit
+prefixes. To specify power-of-10 decimal values defined in the
+International System of Units (SI):
 
-With \fBkb_base=1000\fR, fio follows international standards for unit prefixes.
-To specify power-of-10 decimal values defined in the International
-System of Units (SI):
 .nf
 ki means kilo (K) or 1000
 mi means mega (M) or 1000**2
@@ -240,6 +247,7 @@ pi means peta (P) or 1000**5
 .fi
 
 To specify power-of-2 binary values defined in IEC 80000-13:
+
 .nf
 k means kibi (Ki) or 1024
 m means mebi (Mi) or 1024**2
@@ -248,12 +256,19 @@ t means tebi (Ti) or 1024**4
 p means pebi (Pi) or 1024**5
 .fi
 
-With \fBkb_base=1024\fR (the default), the unit prefixes are opposite from
-those specified in the SI and IEC 80000-13 standards to provide
-compatibility with old scripts.  For example, 4k means 4096.
+With \fBkb_base=1024\fR (the default), the unit prefixes are opposite
+from those specified in the SI and IEC 80000-13 standards to provide
+compatibility with old scripts. For example, 4k means 4096.
+
+For quantities of data, an optional unit of 'B' may be included
+(e.g., 'kB' is the same as 'k').
+
+The *integer suffix* is not case sensitive (e.g., m/mi mean mebi/mega,
+not milli). 'b' and 'B' both mean byte, not bit.
 
-.nf
 Examples with \fBkb_base=1000\fR:
+
+.nf
 4 KiB: 4096, 4096b, 4096B, 4k, 4kb, 4kB, 4K, 4KB
 1 MiB: 1048576, 1m, 1024k
 1 MB: 1000000, 1mi, 1000ki
@@ -261,8 +276,9 @@ Examples with \fBkb_base=1000\fR:
 1 TB: 1000000000, 1ti, 1000mi, 1000000ki
 .fi
 
-.nf
 Examples with \fBkb_base=1024\fR (default):
+
+.nf
 4 KiB: 4096, 4096b, 4096B, 4k, 4kb, 4kB, 4K, 4KB
 1 MiB: 1048576, 1m, 1024k
 1 MB: 1000000, 1mi, 1000ki
@@ -270,13 +286,8 @@ Examples with \fBkb_base=1024\fR (default):
 1 TB: 1000000000, 1ti, 1000mi, 1000000ki
 .fi
 
-For quantities of data, an optional unit of 'B' may be included
-(e.g.,  'kb' is the same as 'k').
-
-The integer suffix is not case sensitive (e.g., m/mi mean mebi/mega,
-not milli). 'b' and 'B' both mean byte, not bit.
-
 To specify times (units are not case sensitive):
+
 .nf
 D means days
 H means hours
@@ -286,20 +297,23 @@ ms or msec means milliseconds
 us or usec means microseconds
 .fi
 
+If the option accepts an upper and lower range, use a colon ':' or
+minus '-' to separate such values. See `irange` parameter type.
+If the lower value specified happens to be larger than the upper value
+the two values are swapped.
 .TP
 .I bool
-Boolean: a true or false value. `0' denotes false, `1' denotes true.
+Boolean. Usually parsed as an integer, however only defined for
+true and false (1 and 0).
 .TP
 .I irange
-Integer range: a range of integers specified in the format
-\fIlower\fR:\fIupper\fR or \fIlower\fR\-\fIupper\fR. \fIlower\fR and
-\fIupper\fR may contain a suffix as described above.  If an option allows two
-sets of ranges, they are separated with a `,' or `/' character. For example:
-`8\-8k/8M\-4G'.
+Integer range with suffix. Allows value range to be given, such as
+1024-4096. A colon may also be used as the separator, e.g. 1k:4k. If the
+option allows two sets of ranges, they can be specified with a ',' or '/'
+delimiter: 1k-4k/8k-32k. Also see `int` parameter type.
 .TP
 .I float_list
-List of floating numbers: A list of floating numbers, separated by
-a ':' character.
+A list of floating point numbers, separated by a ':' character.
 .SH "JOB DESCRIPTION"
 With the above in mind, here follows the complete list of fio job parameters.
 .TP
-- 
2.9.4

[PATCH 5/7] man: refer to REPORTING-BUGS for bug reporting

[kusumi.tomohiro]

From: Tomohiro Kusumi <tkusumi@tuxera.com>

REPORTING-BUGS should be mentioned here rather than README, provided
REPORTING-BUGS also mentions to see README for the mailing list
in addition to details of bug reporting.

Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
---
 README | 3 ++-
 fio.1  | 5 ++++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/README b/README
index ec3e9c0..a6eba8f 100644
--- a/README
+++ b/README
@@ -59,7 +59,8 @@ Mailing list
 ------------
 
 The fio project mailing list is meant for anything related to fio including
-general discussion, bug reporting, questions, and development.
+general discussion, bug reporting, questions, and development. For bug reporting,
+see REPORTING-BUGS.
 
 An automated mail detailing recent commits is automatically sent to the list at
 most daily. The list address is fio@vger.kernel.org, subscribe by sending an
diff --git a/fio.1 b/fio.1
index 1fd928f..bc477a2 100644
--- a/fio.1
+++ b/fio.1
@@ -2713,7 +2713,10 @@ This man page was written by Aaron Carroll <aaronc@cse.unsw.edu.au> based
 on documentation by Jens Axboe.
 .SH "REPORTING BUGS"
 Report bugs to the \fBfio\fR mailing list <fio@vger.kernel.org>.
-See \fBREADME\fR.
+.br
+See \fBREPORTING-BUGS\fR.
+
+\fBREPORTING-BUGS\fR: http://git.kernel.dk/cgit/fio/plain/REPORTING-BUGS
 .SH "SEE ALSO"
 For further documentation see \fBHOWTO\fR and \fBREADME\fR.
 .br
-- 
2.9.4

[PATCH 6/7] HOWTO: minor backports from the man page

[kusumi.tomohiro]

From: Tomohiro Kusumi <tkusumi@tuxera.com>

This commit mainly backports some minor fixups from the man page
sections covered in the several previous commits.

Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
---
 HOWTO | 41 ++++++++++++++++++++++-------------------
 1 file changed, 22 insertions(+), 19 deletions(-)

diff --git a/HOWTO b/HOWTO
index 92c3b73..e982b1a 100644
--- a/HOWTO
+++ b/HOWTO
@@ -104,6 +104,16 @@ Command line options
 
 	Write output to file `filename`.
 
+.. option:: --output-format=type
+
+	Set the reporting format to `normal`, `terse`, `json`, or `json+`.  Multiple
+	formats can be selected, separated by a comma.  `terse` is a CSV based
+	format.  `json+` is like `json`, except it adds a full dump of the latency
+	buckets.
+
+.. option:: --runtime
+	Limit run time to runtime seconds.
+
 .. option:: --bandwidth-log
 
 	Generate aggregate bandwidth logs.
@@ -115,23 +125,16 @@ Command line options
 .. option:: --append-terse
 
     Print statistics in selected mode AND terse, semicolon-delimited format.
-    **deprecated**, use :option:`--output-format` instead to select multiple
+    **Deprecated**, use :option:`--output-format` instead to select multiple
     formats.
 
-.. option:: --output-format=type
-
-	Set the reporting format to `normal`, `terse`, `json`, or `json+`.  Multiple
-	formats can be selected, separated by a comma.  `terse` is a CSV based
-	format.  `json+` is like `json`, except it adds a full dump of the latency
-	buckets.
-
 .. option:: --terse-version=type
 
 	Set terse version output format (default 3, or 2 or 4 or 5).
 
 .. option:: --version
 
-	Print version info and exit.
+	Print version information and exit.
 
 .. option:: --help
 
@@ -144,7 +147,7 @@ Command line options
 .. option:: --crctest=[test]
 
     Test the speed of the built-in checksumming functions. If no argument is
-    given all of them are tested. Alternatively, a comma separated list can be passed, in
+    given, all of them are tested. Alternatively, a comma separated list can be passed, in
     which case the given ones are tested.
 
 .. option:: --cmdhelp=command
@@ -159,7 +162,7 @@ Command line options
 
 .. option:: --showcmd=jobfile
 
-	Turn a job file into command line options.
+	Convert `jobfile` to a set of command-line options.
 
 .. option:: --readonly
 
@@ -172,8 +175,8 @@ Command line options
 
 .. option:: --eta=when
 
-	When real-time ETA estimate should be printed.  May be `always`, `never` or
-	`auto`.
+	Specifies when real-time ETA estimate should be printed.  `when` may be
+	`always`, `never` or `auto`.
 
 .. option:: --eta-newline=time
 
@@ -187,7 +190,7 @@ Command line options
 
 .. option:: --section=name
 
-    Only run specified section in job file.  Multiple sections can be specified.
+    Only run specified section `name` in job file.  Multiple sections can be specified.
     The ``--section`` option allows one to combine related jobs into one file.
     E.g. one job file could define light, moderate, and heavy sections. Tell
     fio to run only the "heavy" section by giving ``--section=heavy``
@@ -198,7 +201,7 @@ Command line options
 
 .. option:: --alloc-size=kb
 
-    Set the internal smalloc pool to this size in KiB.  The
+    Set the internal smalloc pool size to `kb` in KiB.  The
     ``--alloc-size`` switch allows one to use a larger pool size for smalloc.
     If running large jobs with randommap enabled, fio can run out of memory.
     Smalloc is an internal allocator for shared structures from a fixed size
@@ -214,7 +217,7 @@ Command line options
 
 .. option:: --max-jobs=nr
 
-	Maximum number of threads/processes to support.
+	Set the maximum number of threads/processes to support.
 
 .. option:: --server=args
 
@@ -236,7 +239,7 @@ Command line options
 
 .. option:: --idle-prof=option
 
-	Report CPU idleness. *option* is one of the following:
+	Report CPU idleness. `option` is one of the following:
 
 		**calibrate**
 			Run unit work calibration only and exit.
@@ -521,7 +524,7 @@ Parameter types
 	compatibility with old scripts.  For example, 4k means 4096.
 
 	For quantities of data, an optional unit of 'B' may be included
-	(e.g.,  'kB' is the same as 'k').
+	(e.g., 'kB' is the same as 'k').
 
 	The *integer suffix* is not case sensitive (e.g., m/mi mean mebi/mega,
 	not milli). 'b' and 'B' both mean byte, not bit.
@@ -3542,7 +3545,7 @@ If windowed logging is enabled and :option:`log_max_value` is set, then fio logs
 maximum values in that window instead of averages.  Since 'data direction' and
 'offset' are per-I/O values, they aren't applicable if windowed logging is enabled.
 
-Client/server
+Client/Server
 -------------
 
 Normally fio is invoked as a stand-alone application on the machine where the
-- 
2.9.4

[PATCH 7/7] HOWTO: fix indentation for options and job parameters

[kusumi.tomohiro]

From: Tomohiro Kusumi <tkusumi@tuxera.com>

HOWTO mostly uses tab indentations for command line options and job
file parameters, so replace white spaces with tabs.

Signed-off-by: Tomohiro Kusumi <tkusumi@tuxera.com>
---
 HOWTO | 126 +++++++++++++++++++++++++++++++++---------------------------------
 1 file changed, 63 insertions(+), 63 deletions(-)

diff --git a/HOWTO b/HOWTO
index e982b1a..0b80a62 100644
--- a/HOWTO
+++ b/HOWTO
@@ -54,51 +54,51 @@ Command line options
 
 .. option:: --debug=type
 
-    Enable verbose tracing of various fio actions.  May be ``all`` for all types
-    or individual types separated by a comma (e.g. ``--debug=file,mem`` will
-    enable file and memory debugging).  Currently, additional logging is
-    available for:
+	Enable verbose tracing of various fio actions.  May be ``all`` for all types
+	or individual types separated by a comma (e.g. ``--debug=file,mem`` will
+	enable file and memory debugging).  Currently, additional logging is
+	available for:
 
-    *process*
+	*process*
 			Dump info related to processes.
-    *file*
+	*file*
 			Dump info related to file actions.
-    *io*
+	*io*
 			Dump info related to I/O queuing.
-    *mem*
+	*mem*
 			Dump info related to memory allocations.
-    *blktrace*
+	*blktrace*
 			Dump info related to blktrace setup.
-    *verify*
+	*verify*
 			Dump info related to I/O verification.
-    *all*
+	*all*
 			Enable all debug options.
-    *random*
+	*random*
 			Dump info related to random offset generation.
-    *parse*
+	*parse*
 			Dump info related to option matching and parsing.
-    *diskutil*
+	*diskutil*
 			Dump info related to disk utilization updates.
-    *job:x*
+	*job:x*
 			Dump info only related to job number x.
-    *mutex*
+	*mutex*
 			Dump info only related to mutex up/down ops.
-    *profile*
+	*profile*
 			Dump info related to profile extensions.
-    *time*
+	*time*
 			Dump info related to internal time keeping.
-    *net*
+	*net*
 			Dump info related to networking connections.
-    *rate*
+	*rate*
 			Dump info related to I/O rate switching.
-    *compress*
+	*compress*
 			Dump info related to log compress/decompress.
-    *?* or *help*
+	*?* or *help*
 			Show available debug options.
 
 .. option:: --parse-only
 
-    Parse options only, don\'t start any I/O.
+	Parse options only, don\'t start any I/O.
 
 .. option:: --output=filename
 
@@ -124,9 +124,9 @@ Command line options
 
 .. option:: --append-terse
 
-    Print statistics in selected mode AND terse, semicolon-delimited format.
-    **Deprecated**, use :option:`--output-format` instead to select multiple
-    formats.
+	Print statistics in selected mode AND terse, semicolon-delimited format.
+	**Deprecated**, use :option:`--output-format` instead to select multiple
+	formats.
 
 .. option:: --terse-version=type
 
@@ -146,9 +146,9 @@ Command line options
 
 .. option:: --crctest=[test]
 
-    Test the speed of the built-in checksumming functions. If no argument is
-    given, all of them are tested. Alternatively, a comma separated list can be passed, in
-    which case the given ones are tested.
+	Test the speed of the built-in checksumming functions. If no argument is
+	given, all of them are tested. Alternatively, a comma separated list can
+	be passed, in which case the given ones are tested.
 
 .. option:: --cmdhelp=command
 
@@ -156,9 +156,9 @@ Command line options
 
 .. option:: --enghelp=[ioengine[,command]]
 
-    List all commands defined by :option:`ioengine`, or print help for `command`
-    defined by :option:`ioengine`.  If no :option:`ioengine` is given, list all
-    available ioengines.
+	List all commands defined by :option:`ioengine`, or print help for `command`
+	defined by :option:`ioengine`.  If no :option:`ioengine` is given, list all
+	available ioengines.
 
 .. option:: --showcmd=jobfile
 
@@ -166,12 +166,12 @@ Command line options
 
 .. option:: --readonly
 
-    Turn on safety read-only checks, preventing writes.  The ``--readonly``
-    option is an extra safety guard to prevent users from accidentally starting
-    a write workload when that is not desired.  Fio will only write if
-    `rw=write/randwrite/rw/randrw` is given.  This extra safety net can be used
-    as an extra precaution as ``--readonly`` will also enable a write check in
-    the I/O engine core to prevent writes due to unknown user space bug(s).
+	Turn on safety read-only checks, preventing writes.  The ``--readonly``
+	option is an extra safety guard to prevent users from accidentally starting
+	a write workload when that is not desired.  Fio will only write if
+	`rw=write/randwrite/rw/randrw` is given.  This extra safety net can be used
+	as an extra precaution as ``--readonly`` will also enable a write check in
+	the I/O engine core to prevent writes due to unknown user space bug(s).
 
 .. option:: --eta=when
 
@@ -190,30 +190,30 @@ Command line options
 
 .. option:: --section=name
 
-    Only run specified section `name` in job file.  Multiple sections can be specified.
-    The ``--section`` option allows one to combine related jobs into one file.
-    E.g. one job file could define light, moderate, and heavy sections. Tell
-    fio to run only the "heavy" section by giving ``--section=heavy``
-    command line option.  One can also specify the "write" operations in one
-    section and "verify" operation in another section.  The ``--section`` option
-    only applies to job sections.  The reserved *global* section is always
-    parsed and used.
+	Only run specified section `name` in job file.  Multiple sections can be specified.
+	The ``--section`` option allows one to combine related jobs into one file.
+	E.g. one job file could define light, moderate, and heavy sections. Tell
+	fio to run only the "heavy" section by giving ``--section=heavy``
+	command line option.  One can also specify the "write" operations in one
+	section and "verify" operation in another section.  The ``--section`` option
+	only applies to job sections.  The reserved *global* section is always
+	parsed and used.
 
 .. option:: --alloc-size=kb
 
-    Set the internal smalloc pool size to `kb` in KiB.  The
-    ``--alloc-size`` switch allows one to use a larger pool size for smalloc.
-    If running large jobs with randommap enabled, fio can run out of memory.
-    Smalloc is an internal allocator for shared structures from a fixed size
-    memory pool and can grow to 16 pools. The pool size defaults to 16MiB.
+	Set the internal smalloc pool size to `kb` in KiB.  The
+	``--alloc-size`` switch allows one to use a larger pool size for smalloc.
+	If running large jobs with randommap enabled, fio can run out of memory.
+	Smalloc is an internal allocator for shared structures from a fixed size
+	memory pool and can grow to 16 pools. The pool size defaults to 16MiB.
 
-    NOTE: While running :file:`.fio_smalloc.*` backing store files are visible
-    in :file:`/tmp`.
+	NOTE: While running :file:`.fio_smalloc.*` backing store files are visible
+	in :file:`/tmp`.
 
 .. option:: --warnings-fatal
 
-    All fio parser warnings are fatal, causing fio to exit with an
-    error.
+	All fio parser warnings are fatal, causing fio to exit with an
+	error.
 
 .. option:: --max-jobs=nr
 
@@ -221,17 +221,17 @@ Command line options
 
 .. option:: --server=args
 
-    Start a backend server, with `args` specifying what to listen to.
-    See `Client/Server`_ section.
+	Start a backend server, with `args` specifying what to listen to.
+	See `Client/Server`_ section.
 
 .. option:: --daemonize=pidfile
 
-    Background a fio server, writing the pid to the given `pidfile` file.
+	Background a fio server, writing the pid to the given `pidfile` file.
 
 .. option:: --client=hostname
 
-    Instead of running the jobs locally, send and run them on the given host or
-    set of hosts.  See `Client/Server`_ section.
+	Instead of running the jobs locally, send and run them on the given host or
+	set of hosts.  See `Client/Server`_ section.
 
 .. option:: --remote-config=file
 
@@ -448,7 +448,7 @@ automatically substituted with the current system values when the job is
 run. Simple math is also supported on these keywords, so you can perform actions
 like::
 
-        size=8*$mb_memory
+	size=8*$mb_memory
 
 and get that properly expanded to 8 times the size of memory in the machine.
 
@@ -477,7 +477,7 @@ Parameter types
 ~~~~~~~~~~~~~~~
 
 **str**
-    String. This is a sequence of alpha characters.
+	String: A sequence of alphanumeric characters.
 
 **time**
 	Integer with possible time suffix.  Without a unit value is interpreted as
@@ -491,7 +491,7 @@ Parameter types
 	Integer. A whole number value, which may contain an integer prefix
 	and an integer suffix:
 
-        [*integer prefix*] **number** [*integer suffix*]
+	[*integer prefix*] **number** [*integer suffix*]
 
 	The optional *integer prefix* specifies the number's base. The default
 	is decimal. *0x* specifies hexadecimal.
-- 
2.9.4

Re: [PATCH 1/7] man: sync OPTIONS section with HOWTO

[Sitsofe Wheeler]

Hi,

On 4 July 2017 at 19:08,  <kusumi.tomohiro@gmail.com> wrote:
>
> This commit brings in updates from HOWTO. The exception is --debug
> option, which is too verbose yet not that important for all debug types
> to be listed in details.

I've got to take my hat off to you Tomohiro - I thought it was
difficult enough to update just the HOWTO in the first place. Your
Herculean effort here brings the two more into sync!

I'm secretly jealous of your productivity... Plus you've set back my
covert plan to make the current fio man page so useless that we switch
to having it built from HOWTO (using make -C doc man ) via Sphinx :-).

-- 
Sitsofe | http://sucs.org/~sits/

Re: [PATCH 1/7] man: sync OPTIONS section with HOWTO

[Tomohiro Kusumi]

Hi

Thanks.
I think it's better if there's a way to somehow automatically generate
the man page from HOWTO like you mentioned (and update from that).

I simply don't know high-level documentation tools that this was the only way.
(The current man page also seems to have been written by hand)


2017-07-04 22:03 GMT+03:00 Sitsofe Wheeler <sitsofe@gmail.com>:
> Hi,
>
> On 4 July 2017 at 19:08,  <kusumi.tomohiro@gmail.com> wrote:
>>
>> This commit brings in updates from HOWTO. The exception is --debug
>> option, which is too verbose yet not that important for all debug types
>> to be listed in details.
>
> I've got to take my hat off to you Tomohiro - I thought it was
> difficult enough to update just the HOWTO in the first place. Your
> Herculean effort here brings the two more into sync!
>
> I'm secretly jealous of your productivity... Plus you've set back my
> covert plan to make the current fio man page so useless that we switch
> to having it built from HOWTO (using make -C doc man ) via Sphinx :-).
>
> --
> Sitsofe | http://sucs.org/~sits/

Re: [PATCH 1/7] man: sync OPTIONS section with HOWTO

[Jens Axboe]

On Tue, Jul 04 2017, Tomohiro Kusumi wrote:
> Hi
> 
> Thanks.
> I think it's better if there's a way to somehow automatically generate
> the man page from HOWTO like you mentioned (and update from that).

Yes, that should definitely be the long term goal. It's pretty annoying
to have to update both, hence people forget (even me), and then they
drift again.

Thankfully people are working on that :-)

-- 
Jens Axboe

Re: [PATCH 0/7] Sync fio(1) man page with HOWTO

[Jens Axboe]

On Tue, Jul 04 2017, kusumi.tomohiro@gmail.com wrote:
> From: Tomohiro Kusumi <tkusumi@tuxera.com>
> 
> The man page hasn't been in sync with many of the updates made for
> HOWTO. These commits update the man page based on what HOWTO has,
> where each commit corresponds to a single section (instead of updating
> with commit-to-commit basis). The man page being in sync with HOWTO
> also makes it easier to make changes, whenever HOWTO is updated in the
> future.
> 
> The goal is to be in sync with HOWTO without being too verbose for
> a man page. Note that they don't include contents that do not exist
> in HOWTO (i.e. I didn't newly create anything). Also note these are
> only beginning part of the man page.
> 
> These commits include
> 1. Import updates made only for HOWTO (diffs), unless the changes are
>    too long/verbose to be mentioned in the man page.
> 2. Replace the entire section or paragraph with the corresponding one
>    in HOWTO (partly as a result of 1), if these two are essencially
>    saying the same thing with small modification, so that we don't see
>    two different contents in two documents that are almost the same.
> 3. Make some small changes when above two are imported, such as using
>    directives, removal of double white spaces, removal of expressions
>    that don't fit in well with a man page (e.g. This section describes...).
> 4. Backport some small changes from the man page to HOWTO, which seem
>    to have been made when the man page was first introduced. These are
>    mostly minor fixups.

Thanks, applied.

-- 
Jens Axboe

Re: [PATCH 1/7] man: sync OPTIONS section with HOWTO

[Jens Axboe]

On Tue, Jul 04 2017, Tomohiro Kusumi wrote:
> (The current man page also seems to have been written by hand)

It is.

-- 
Jens Axboe

Re: [PATCH 1/7] man: sync OPTIONS section with HOWTO

[Sitsofe Wheeler]

Hi,

I'm very slowly working on improving the Sphinx generated man page.
The latest round of updates was partially to try and add any missing
sections that were only in the man page and weren't in the HOWTO.

There's still work to do as the "generated" man page doesn't have
typical man page section headings, it is missing key sections like
SYNTAX, it repeats the AUTHOR section twice etc. There are also a lot of
options that are missing their default values.

Other improvements need to be made to Sphinx itself: it seems to output
a UTF-8 man page that some platforms (macOS/OSX) struggle
with so it could do with an option to just output plain ASCII man
pages. Additionally some of the rendering looks a bit strange
(underlines only seem to go under letters, meta variable names after
parameters not underlined) and a few other oddities
like that.

On 4 July 2017 at 20:33, Tomohiro Kusumi <kusumi.tomohiro@gmail.com> wrote:
> Hi
>
> Thanks.
> I think it's better if there's a way to somehow automatically generate
> the man page from HOWTO like you mentioned (and update from that).
>
> I simply don't know high-level documentation tools that this was the only way.
> (The current man page also seems to have been written by hand)
>
>
> 2017-07-04 22:03 GMT+03:00 Sitsofe Wheeler <sitsofe@gmail.com>:
>> Hi,
>>
>> On 4 July 2017 at 19:08,  <kusumi.tomohiro@gmail.com> wrote:
>>>
>>> This commit brings in updates from HOWTO. The exception is --debug
>>> option, which is too verbose yet not that important for all debug types
>>> to be listed in details.
>>
>> I've got to take my hat off to you Tomohiro - I thought it was
>> difficult enough to update just the HOWTO in the first place. Your
>> Herculean effort here brings the two more into sync!
>>
>> I'm secretly jealous of your productivity... Plus you've set back my
>> covert plan to make the current fio man page so useless that we switch
>> to having it built from HOWTO (using make -C doc man ) via Sphinx :-).

-- 
Sitsofe | http://sucs.org/~sits/