* [PATCH v2 1/5] steadystate: delete incorrect comment
[not found] ` <CGME20220524142325uscas1p1ac4d3543301cc43d0e5fa8dce8882dcb@uscas1p1.samsung.com>
@ 2022-05-24 14:23 ` Vincent Fu
0 siblings, 0 replies; 7+ messages in thread
From: Vincent Fu @ 2022-05-24 14:23 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
Fio actually does not begin collecting steady state data until the
steady state ramp time has expired. Remove the comment that said steady
state data is collected from the start of the job.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
steadystate.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/steadystate.c b/steadystate.c
index 2e3da1db..ad19318c 100644
--- a/steadystate.c
+++ b/steadystate.c
@@ -250,13 +250,6 @@ int steadystate_check(void)
rate_time = mtime_since(&ss->prev_time, &now);
memcpy(&ss->prev_time, &now, sizeof(now));
- /*
- * Begin monitoring when job starts but don't actually use
- * data in checking stopping criterion until ss->ramp_time is
- * over. This ensures that we will have a sane value in
- * prev_iops/bw the first time through after ss->ramp_time
- * is done.
- */
if (ss->state & FIO_SS_RAMP_OVER) {
group_bw += 1000 * (td_bytes - ss->prev_bytes) / rate_time;
group_iops += 1000 * (td_iops - ss->prev_iops) / rate_time;
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 2/5] configure: refer to zlib1g-dev package for zlib support
[not found] ` <CGME20220524142325uscas1p139f830316e1e51e34a35e1fa64923b2d@uscas1p1.samsung.com>
@ 2022-05-24 14:23 ` Vincent Fu
0 siblings, 0 replies; 7+ messages in thread
From: Vincent Fu @ 2022-05-24 14:23 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
Recent Debian-based distributions provide zlib support in the zlib1g-dev
package.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
configure | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/configure b/configure
index 95b60bb7..4ee536a0 100755
--- a/configure
+++ b/configure
@@ -3142,7 +3142,7 @@ if test "$libzbc" = "yes" ; then
output_sym "CONFIG_LIBZBC"
fi
if test "$zlib" = "no" ; then
- echo "Consider installing zlib-dev (zlib-devel, some fio features depend on it."
+ echo "Consider installing zlib1g-dev (zlib-devel) as some fio features depend on it."
if test "$build_static" = "yes"; then
echo "Note that some distros have separate packages for static libraries."
fi
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 4/5] t/run-fio-tests: improve json data decoding
[not found] ` <CGME20220524142326uscas1p130d78e36a5aa36964068cde26412f4ae@uscas1p1.samsung.com>
@ 2022-05-24 14:23 ` Vincent Fu
0 siblings, 0 replies; 7+ messages in thread
From: Vincent Fu @ 2022-05-24 14:23 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
Instead of skipping up to five lines, just skip everything until the
opening curly brace when trying to decode JSON data.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
t/run-fio-tests.py | 20 +++++++-------------
1 file changed, 7 insertions(+), 13 deletions(-)
diff --git a/t/run-fio-tests.py b/t/run-fio-tests.py
index ecceb67e..32cdbc19 100755
--- a/t/run-fio-tests.py
+++ b/t/run-fio-tests.py
@@ -311,21 +311,15 @@ class FioJobTest(FioExeTest):
#
# Sometimes fio informational messages are included at the top of the
# JSON output, especially under Windows. Try to decode output as JSON
- # data, lopping off up to the first four lines
+ # data, skipping everything until the first {
#
lines = file_data.splitlines()
- for i in range(5):
- file_data = '\n'.join(lines[i:])
- try:
- self.json_data = json.loads(file_data)
- except json.JSONDecodeError:
- continue
- else:
- logging.debug("Test %d: skipped %d lines decoding JSON data", self.testnum, i)
- return
-
- self.failure_reason = "{0} unable to decode JSON data,".format(self.failure_reason)
- self.passed = False
+ file_data = '\n'.join(lines[lines.index("{"):])
+ try:
+ self.json_data = json.loads(file_data)
+ except json.JSONDecodeError:
+ self.failure_reason = "{0} unable to decode JSON data,".format(self.failure_reason)
+ self.passed = False
class FioJobTest_t0005(FioJobTest):
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 5/5] docs: update discussion of huge page sizes
[not found] ` <CGME20220524142326uscas1p27c8cd9fd4f344840227201790f922bb5@uscas1p2.samsung.com>
@ 2022-05-24 14:23 ` Vincent Fu
0 siblings, 0 replies; 7+ messages in thread
From: Vincent Fu @ 2022-05-24 14:23 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
Note that the default huge page size is either 2 or 4 MiB, depending on
the platform. Also mention /sys/kernel/mm/hugepages/ as another place to
see the supported huge page sizes.
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
HOWTO.rst | 25 ++++++++++++++-----------
fio.1 | 19 ++++++++++---------
2 files changed, 24 insertions(+), 20 deletions(-)
diff --git a/HOWTO.rst b/HOWTO.rst
index eee386c1..58d02fa2 100644
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -1823,13 +1823,14 @@ Buffers and memory
**mmaphuge** to work, the system must have free huge pages allocated. This
can normally be checked and set by reading/writing
:file:`/proc/sys/vm/nr_hugepages` on a Linux system. Fio assumes a huge page
- is 4MiB in size. So to calculate the number of huge pages you need for a
- given job file, add up the I/O depth of all jobs (normally one unless
- :option:`iodepth` is used) and multiply by the maximum bs set. Then divide
- that number by the huge page size. You can see the size of the huge pages in
- :file:`/proc/meminfo`. If no huge pages are allocated by having a non-zero
- number in `nr_hugepages`, using **mmaphuge** or **shmhuge** will fail. Also
- see :option:`hugepage-size`.
+ is 2 or 4MiB in size depending on the platform. So to calculate the
+ number of huge pages you need for a given job file, add up the I/O
+ depth of all jobs (normally one unless :option:`iodepth` is used) and
+ multiply by the maximum bs set. Then divide that number by the huge
+ page size. You can see the size of the huge pages in
+ :file:`/proc/meminfo`. If no huge pages are allocated by having a
+ non-zero number in `nr_hugepages`, using **mmaphuge** or **shmhuge**
+ will fail. Also see :option:`hugepage-size`.
**mmaphuge** also needs to have hugetlbfs mounted and the file location
should point there. So if it's mounted in :file:`/huge`, you would use
@@ -1848,10 +1849,12 @@ Buffers and memory
.. option:: hugepage-size=int
- Defines the size of a huge page. Must at least be equal to the system
- setting, see :file:`/proc/meminfo`. Defaults to 4MiB. Should probably
- always be a multiple of megabytes, so using ``hugepage-size=Xm`` is the
- preferred way to set this to avoid setting a non-pow-2 bad value.
+ Defines the size of a huge page. Must at least be equal to the system
+ setting, see :file:`/proc/meminfo` and
+ :file:`/sys/kernel/mm/hugepages/`. Defaults to 2 or 4MiB depending on
+ the platform. Should probably always be a multiple of megabytes, so
+ using ``hugepage-size=Xm`` is the preferred way to set this to avoid
+ setting a non-pow-2 bad value.
.. option:: lockmem=int
diff --git a/fio.1 b/fio.1
index ded7bbfc..5f057574 100644
--- a/fio.1
+++ b/fio.1
@@ -1631,11 +1631,11 @@ multiplied by the I/O depth given. Note that for \fBshmhuge\fR and
\fBmmaphuge\fR to work, the system must have free huge pages allocated. This
can normally be checked and set by reading/writing
`/proc/sys/vm/nr_hugepages' on a Linux system. Fio assumes a huge page
-is 4MiB in size. So to calculate the number of huge pages you need for a
-given job file, add up the I/O depth of all jobs (normally one unless
-\fBiodepth\fR is used) and multiply by the maximum bs set. Then divide
-that number by the huge page size. You can see the size of the huge pages in
-`/proc/meminfo'. If no huge pages are allocated by having a non-zero
+is 2 or 4MiB in size depending on the platform. So to calculate the number of
+huge pages you need for a given job file, add up the I/O depth of all jobs
+(normally one unless \fBiodepth\fR is used) and multiply by the maximum bs set.
+Then divide that number by the huge page size. You can see the size of the huge
+pages in `/proc/meminfo'. If no huge pages are allocated by having a non-zero
number in `nr_hugepages', using \fBmmaphuge\fR or \fBshmhuge\fR will fail. Also
see \fBhugepage\-size\fR.
.P
@@ -1655,10 +1655,11 @@ of subsequent I/O memory buffers is the sum of the \fBiomem_align\fR and
\fBbs\fR used.
.TP
.BI hugepage\-size \fR=\fPint
-Defines the size of a huge page. Must at least be equal to the system
-setting, see `/proc/meminfo'. Defaults to 4MiB. Should probably
-always be a multiple of megabytes, so using `hugepage\-size=Xm' is the
-preferred way to set this to avoid setting a non-pow-2 bad value.
+Defines the size of a huge page. Must at least be equal to the system setting,
+see `/proc/meminfo' and `/sys/kernel/mm/hugepages/'. Defaults to 2 or 4MiB
+depending on the platform. Should probably always be a multiple of megabytes,
+so using `hugepage\-size=Xm' is the preferred way to set this to avoid setting
+a non-pow-2 bad value.
.TP
.BI lockmem \fR=\fPint
Pin the specified amount of memory with \fBmlock\fR\|(2). Can be used to
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 3/5] HOWTO: add blank line for prettier formatting
[not found] ` <CGME20220524142325uscas1p269230921c0498a1c6205a132440d2f30@uscas1p2.samsung.com>
@ 2022-05-24 14:23 ` Vincent Fu
0 siblings, 0 replies; 7+ messages in thread
From: Vincent Fu @ 2022-05-24 14:23 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
There needs to be a blank line between the name of an option and its
description in order for the formatting to look nice at
https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-ignore-zone-limits,
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
HOWTO.rst | 1 +
1 file changed, 1 insertion(+)
diff --git a/HOWTO.rst b/HOWTO.rst
index 84bea5c5..eee386c1 100644
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -1064,6 +1064,7 @@ Target file/device
thread/process.
.. option:: ignore_zone_limits=bool
+
If this option is used, fio will ignore the maximum number of open
zones limit of the zoned block device in use, thus allowing the
option :option:`max_open_zones` value to be larger than the device
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v2 0/5] (now) five cleanups
[not found] <CGME20220524142325uscas1p197b28727cdb38d34f866b97b0a1932e5@uscas1p1.samsung.com>
@ 2022-05-24 14:23 ` Vincent Fu
[not found] ` <CGME20220524142325uscas1p139f830316e1e51e34a35e1fa64923b2d@uscas1p1.samsung.com>
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: Vincent Fu @ 2022-05-24 14:23 UTC (permalink / raw)
To: axboe, fio; +Cc: Vincent Fu
Jens, please consider these five cleanup patches.
Changes from v1 (cleanups, parse /proc/meminfo for huge page size):
- Drop patch that parses /proc/meminfo
- Documentation patch about huge page sizes fixes default size
Vincent Fu (5):
steadystate: delete incorrect comment
configure: refer to zlib1g-dev package for zlib support
HOWTO: add blank line for prettier formatting
t/run-fio-tests: improve json data decoding
docs: update discussion of huge page sizes
HOWTO.rst | 26 +++++++++++++++-----------
configure | 2 +-
fio.1 | 19 ++++++++++---------
steadystate.c | 7 -------
t/run-fio-tests.py | 20 +++++++-------------
5 files changed, 33 insertions(+), 41 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2 0/5] (now) five cleanups
2022-05-24 14:23 ` [PATCH v2 0/5] (now) five cleanups Vincent Fu
` (4 preceding siblings ...)
[not found] ` <CGME20220524142326uscas1p130d78e36a5aa36964068cde26412f4ae@uscas1p1.samsung.com>
@ 2022-05-28 2:37 ` Jens Axboe
5 siblings, 0 replies; 7+ messages in thread
From: Jens Axboe @ 2022-05-28 2:37 UTC (permalink / raw)
To: fio, vincent.fu
On Tue, 24 May 2022 14:23:24 +0000, Vincent Fu wrote:
> Jens, please consider these five cleanup patches.
>
> Changes from v1 (cleanups, parse /proc/meminfo for huge page size):
> - Drop patch that parses /proc/meminfo
> - Documentation patch about huge page sizes fixes default size
>
>
> [...]
Applied, thanks!
[1/5] steadystate: delete incorrect comment
commit: b29472cbdbce66ad9905b58f04ac5ed9e9be0090
[2/5] configure: refer to zlib1g-dev package for zlib support
commit: 95f791d3ad7d0cf3b024d51713d4861663ba8ce5
[3/5] HOWTO: add blank line for prettier formatting
commit: a3a6f1058aa41db20794bee1df0bec4019c14d42
[4/5] t/run-fio-tests: improve json data decoding
commit: db2637ec09786c5610006414f28c2b21a3c3c165
[5/5] docs: update discussion of huge page sizes
commit: cb8dcafa69ef6fddd22d41c60083cb68bce92daa
Best regards,
--
Jens Axboe
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-05-28 2:37 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CGME20220524142325uscas1p197b28727cdb38d34f866b97b0a1932e5@uscas1p1.samsung.com>
2022-05-24 14:23 ` [PATCH v2 0/5] (now) five cleanups Vincent Fu
[not found] ` <CGME20220524142325uscas1p139f830316e1e51e34a35e1fa64923b2d@uscas1p1.samsung.com>
2022-05-24 14:23 ` [PATCH v2 2/5] configure: refer to zlib1g-dev package for zlib support Vincent Fu
[not found] ` <CGME20220524142325uscas1p1ac4d3543301cc43d0e5fa8dce8882dcb@uscas1p1.samsung.com>
2022-05-24 14:23 ` [PATCH v2 1/5] steadystate: delete incorrect comment Vincent Fu
[not found] ` <CGME20220524142325uscas1p269230921c0498a1c6205a132440d2f30@uscas1p2.samsung.com>
2022-05-24 14:23 ` [PATCH v2 3/5] HOWTO: add blank line for prettier formatting Vincent Fu
[not found] ` <CGME20220524142326uscas1p27c8cd9fd4f344840227201790f922bb5@uscas1p2.samsung.com>
2022-05-24 14:23 ` [PATCH v2 5/5] docs: update discussion of huge page sizes Vincent Fu
[not found] ` <CGME20220524142326uscas1p130d78e36a5aa36964068cde26412f4ae@uscas1p1.samsung.com>
2022-05-24 14:23 ` [PATCH v2 4/5] t/run-fio-tests: improve json data decoding Vincent Fu
2022-05-28 2:37 ` [PATCH v2 0/5] (now) five cleanups Jens Axboe
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.