All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 1/2] pmemblk: clean up error logs
@ 2016-12-20 23:56 Robert Elliott
  2016-12-20 23:56 ` [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB Robert Elliott
  0 siblings, 1 reply; 6+ messages in thread
From: Robert Elliott @ 2016-12-20 23:56 UTC (permalink / raw)
  To: fio; +Cc: Robert Elliott

From: Robert Elliott <elliott@hpe.com>

Print the errno string rather than the number, and prefix
all log_err prints with pmemblk rather than fio.
---
 engines/pmemblk.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/engines/pmemblk.c b/engines/pmemblk.c
index ca72697..977a02b 100644
--- a/engines/pmemblk.c
+++ b/engines/pmemblk.c
@@ -119,7 +119,7 @@ int load_libpmemblk(const char *path)
 	return 0;
 
 errorout:
-	log_err("fio: unable to load libpmemblk: %s\n", dlerror());
+	log_err("pmemblk: unable to load libpmemblk: %s\n", dlerror());
 	if (dl)
 		dlclose(dl);
 
@@ -267,9 +267,8 @@ static fio_pmemblk_file_t pmb_open(const char *pathspec, int flags)
 			    pmemblk_create(path, bsize, fsize, 0644);
 		}
 		if (!pmb->pmb_pool) {
-			log_err
-			    ("fio: enable to open pmemblk pool file (errno %d)\n",
-			     errno);
+			log_err("pmemblk: unable to open pmemblk pool file %s (%s)\n",
+			     path, strerror(errno));
 			goto error;
 		}
 
@@ -331,14 +330,14 @@ static int pmb_get_flags(struct thread_data *td, uint64_t *pflags)
 	if (!td->o.use_thread) {
 		if (!thread_warned) {
 			thread_warned = 1;
-			log_err("fio: must set thread=1 for pmemblk engine\n");
+			log_err("pmemblk: must set thread=1 for pmemblk engine\n");
 		}
 		return 1;
 	}
 
 	if (!td->o.odirect && !odirect_warned) {
 		odirect_warned = 1;
-		log_info("fio: direct == 0, but pmemblk is always direct\n");
+		log_info("pmemblk: direct == 0, but pmemblk is always direct\n");
 	}
 
 	if (td->o.allow_create)
-- 
2.9.3


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB
  2016-12-20 23:56 [PATCH 1/2] pmemblk: clean up error logs Robert Elliott
@ 2016-12-20 23:56 ` Robert Elliott
  2016-12-21  6:05   ` Sitsofe Wheeler
  0 siblings, 1 reply; 6+ messages in thread
From: Robert Elliott @ 2016-12-20 23:56 UTC (permalink / raw)
  To: fio; +Cc: Robert Elliott

From: Robert Elliott <elliott@hpe.com>

Clarify that the FSIZE argument is in MiB (power of 2) rather
than MB (power of 10) in comments and variable names.
---
 engines/pmemblk.c | 39 ++++++++++++++++++++++++---------------
 1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/engines/pmemblk.c b/engines/pmemblk.c
index 977a02b..62f3746 100644
--- a/engines/pmemblk.c
+++ b/engines/pmemblk.c
@@ -27,11 +27,11 @@
  *   ioengine=pmemblk
  *
  * Other relevant settings:
+ *   thread=1   REQUIRED
  *   iodepth=1
  *   direct=1
- *   thread=1   REQUIRED
  *   unlink=1
- *   filename=/pmem0/fiotestfile,BSIZE,FSIZEMB
+ *   filename=/mnt/pmem0/fiotestfile,BSIZE,FSIZEMiB
  *
  *   thread must be set to 1 for pmemblk as multiple processes cannot
  *     open the same block pool file.
@@ -39,22 +39,31 @@
  *   iodepth should be set to 1 as pmemblk is always synchronous.
  *   Use numjobs to scale up.
  *
- *   direct=1 is implied as pmemblk is always direct.
+ *   direct=1 is implied as pmemblk is always direct. A warning message
+ *   is printed if this is not specified.
+ *
+ *   unlink=1 removes the block pool file after testing, and is optional.
+ *
+ *   The pmem device must have a DAX-capable filesystem and be mounted
+ *   with DAX enabled.  filename must point to a file on that filesystem.
  *
- *   Can set unlink to 1 to remove the block pool file after testing.
+ *   Example:
+ *     mkfs.xfs /dev/pmem0
+ *     mkdir /mnt/pmem0
+ *     mount -o dax /dev/pmem0 /mnt/pmem0
  *
  *   When specifying the filename, if the block pool file does not already
- *   exist, then the pmemblk engine can create the pool file if you specify
+ *   exist, then the pmemblk engine creates the pool file if you specify
  *   the block and file sizes.  BSIZE is the block size in bytes.
- *   FSIZEMB is the pool file size in MB.
+ *   FSIZEMiB is the pool file size in MiB.
  *
  *   See examples/pmemblk.fio for more.
  *
  * libpmemblk.so
- *   By default, the pmemblk engine will let the system find the libpmemblk.so
- *   that it uses.  You can use an alternative libpmemblk by setting the
- *   FIO_PMEMBLK_LIB environment variable to the full path to the desired
- *   libpmemblk.so.
+ *   By default, the pmemblk engine lets the system find libpmemblk.so
+ *   (e.g., see /etc/ld.conf.so).  You may use an alternative libpmemblk
+ *   by setting the FIO_PMEMBLK_LIB environment variable to the full path
+ *   to the desired libpmemblk.so file.
  *
  */
 
@@ -187,7 +196,7 @@ static void fio_pmemblk_cache_remove(fio_pmemblk_file_t pmb)
  * level, we allow the block size and file size to be appended
  * to the file name:
  *
- *   path[,bsize,fsizemb]
+ *   path[,bsize,fsizemib]
  *
  * note that we do not use the fio option "filesize" to dictate
  * the file size because we can only give libpmemblk the gross
@@ -197,7 +206,7 @@ static void fio_pmemblk_cache_remove(fio_pmemblk_file_t pmb)
  * the final path without the parameters is returned in ppath.
  * the block size and file size are returned in pbsize and fsize.
  *
- * note that the user should specify the file size in MiB, but
+ * note that the user specifies the file size in MiB, but
  * we return bytes from here.
  */
 static void pmb_parse_path(const char *pathspec, char **ppath, uint64_t *pbsize,
@@ -206,7 +215,7 @@ static void pmb_parse_path(const char *pathspec, char **ppath, uint64_t *pbsize,
 	char *path;
 	char *s;
 	uint64_t bsize;
-	uint64_t fsizemb;
+	uint64_t fsizemib;
 
 	path = strdup(pathspec);
 	if (!path) {
@@ -216,14 +225,14 @@ static void pmb_parse_path(const char *pathspec, char **ppath, uint64_t *pbsize,
 
 	/* extract sizes, if given */
 	s = strrchr(path, ',');
-	if (s && (fsizemb = strtoull(s + 1, NULL, 10))) {
+	if (s && (fsizemib = strtoull(s + 1, NULL, 10))) {
 		*s = 0;
 		s = strrchr(path, ',');
 		if (s && (bsize = strtoull(s + 1, NULL, 10))) {
 			*s = 0;
 			*ppath = path;
 			*pbsize = bsize;
-			*pfsize = fsizemb << 20;
+			*pfsize = fsizemib << 20;
 			return;
 		}
 	}
-- 
2.9.3


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB
  2016-12-20 23:56 ` [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB Robert Elliott
@ 2016-12-21  6:05   ` Sitsofe Wheeler
  2016-12-21  7:57     ` Elliott, Robert (Persistent Memory)
  0 siblings, 1 reply; 6+ messages in thread
From: Sitsofe Wheeler @ 2016-12-21  6:05 UTC (permalink / raw)
  To: Robert Elliott; +Cc: fio

On 20 December 2016 at 23:56, Robert Elliott <elliott@hpe.com> wrote:
> From: Robert Elliott <elliott@hpe.com>
>
> Clarify that the FSIZE argument is in MiB (power of 2) rather
> than MB (power of 10) in comments and variable names.
> ---
>  engines/pmemblk.c | 39 ++++++++++++++++++++++++---------------
>  1 file changed, 24 insertions(+), 15 deletions(-)
>
> diff --git a/engines/pmemblk.c b/engines/pmemblk.c
> index 977a02b..62f3746 100644
> --- a/engines/pmemblk.c
> +++ b/engines/pmemblk.c
[...]
> @@ -39,22 +39,31 @@
>   *   iodepth should be set to 1 as pmemblk is always synchronous.
>   *   Use numjobs to scale up.
>   *
> - *   direct=1 is implied as pmemblk is always direct.
> + *   direct=1 is implied as pmemblk is always direct. A warning message
> + *   is printed if this is not specified.
> + *
> + *   unlink=1 removes the block pool file after testing, and is optional.
> + *
> + *   The pmem device must have a DAX-capable filesystem and be mounted
> + *   with DAX enabled.  filename must point to a file on that filesystem.
>   *
> - *   Can set unlink to 1 to remove the block pool file after testing.
> + *   Example:
> + *     mkfs.xfs /dev/pmem0
> + *     mkdir /mnt/pmem0
> + *     mount -o dax /dev/pmem0 /mnt/pmem0
>   *
>   *   When specifying the filename, if the block pool file does not already
> - *   exist, then the pmemblk engine can create the pool file if you specify
> + *   exist, then the pmemblk engine creates the pool file if you specify
>   *   the block and file sizes.  BSIZE is the block size in bytes.
> - *   FSIZEMB is the pool file size in MB.
> + *   FSIZEMiB is the pool file size in MiB.

It might be safer to explicitly say "in megabytes" rather than "in
MiB" here because of fio's infamous reversal of the terms M(B)/MiB
(https://github.com/axboe/fio/issues/103 ). Can the user explicitly
specify units as is done elsewhere in fio?

-- 
Sitsofe | http://sucs.org/~sits/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* RE: [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB
  2016-12-21  6:05   ` Sitsofe Wheeler
@ 2016-12-21  7:57     ` Elliott, Robert (Persistent Memory)
  2016-12-21 10:39       ` Sitsofe Wheeler
  0 siblings, 1 reply; 6+ messages in thread
From: Elliott, Robert (Persistent Memory) @ 2016-12-21  7:57 UTC (permalink / raw)
  To: Sitsofe Wheeler; +Cc: fio


> >   *   When specifying the filename, if the block pool file does not
> already
> > - *   exist, then the pmemblk engine can create the pool file if
> you specify
> > + *   exist, then the pmemblk engine creates the pool file if you
> specify
> >   *   the block and file sizes.  BSIZE is the block size in bytes.
> > - *   FSIZEMB is the pool file size in MB.
> > + *   FSIZEMiB is the pool file size in MiB.
> 
> It might be safer to explicitly say "in megabytes" rather than "in
> MiB" here because of fio's infamous reversal of the terms M(B)/MiB
> (https://github.com/axboe/fio/issues/103 ). Can the user explicitly
> specify units as is done elsewhere in fio?

"mega" is still the SI power of 10 prefix; mebi would be the IEC 
power of 2 prefix.

I'm working on a big patch set to clean up the units throughout the 
whole codebase, and hope to post it tomorrow.

For inputs (e.g., parameters in .fio files), kb_base=1000 will 
cause it to use proper units and kb_base=1024 (the default) will
continue to use wrong units (for compatibility).  I'm guessing that
kb_base=1000 has little use so far (there are several bugs making
it pretty unusable right now) so we can take it over for this
purpose.

For outputs, proper units will always be displayed onscreen,
sometimes with both side-by-side. 

The terse and json output formats are not changed - they'll
continue to report KiB/s values (without units).

Example text output:

Jobs: 576 (f=576), 549MiB/s-80.5GiB/s: [w(288),r(288)] [0.0% done] [r=33200MiB/s,w=25544MiB/s,t=0KiB/s] [r=8499k,w=6539k,t=0 iops] [eta 06h:59m:53s]
...
drive_0: (groupid=0, jobs=36): err= 0: pid=54154: Tue Dec 20 08:17:35 2016
  write: io=75.7TiB, bw=3145MiB/s (3298MB/s), iops=805k, runt=25200023msec
    clat percentiles (usec):
     |  1.00th=[    0],  5.00th=[    0], 10.00th=[    0], 20.00th=[    0],
     | 30.00th=[    0], 40.00th=[    0], 50.00th=[    0], 60.00th=[    0],
     | 70.00th=[    0], 80.00th=[    0], 90.00th=[    0], 95.00th=[    0],
     | 99.00th=[    0], 99.50th=[    0], 99.90th=[    0], 99.95th=[    0],
     | 99.99th=[    0]
  cpu          : usr=1.80%, sys=10.70%, ctx=32257714, majf=0, minf=5992
  IO depths    : 1=15.3%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=15.3%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=15.3%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total r=0,w=20290240440,t=0, short r=0,w=0,t=0, dropped r=0,w=0,t=0
     latency   : target=0, window=0, percentile=100.00%, depth=1
...
Run status group 0 (all jobs):
   READ: io=837GiB (899GB), aggrbw=31.9GiB/s (34.2GB/s), minbw=31.9GiB/s (34.2GB/s), maxbw=31.9GiB/s (34.2GB/s), mint=26287msec, maxt=26287msec
  WRITE: io=640GiB (687GB), aggrbw=24.4GiB/s (26.2GB/s), minbw=24.4GiB/s (26.2GB/s), maxbw=24.4GiB/s (26.2GB/s), mint=26300msec, maxt=26300msec


---
Robert Elliott, HPE Persistent Memory



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB
  2016-12-21  7:57     ` Elliott, Robert (Persistent Memory)
@ 2016-12-21 10:39       ` Sitsofe Wheeler
  2016-12-21 16:46         ` Elliott, Robert (Persistent Memory)
  0 siblings, 1 reply; 6+ messages in thread
From: Sitsofe Wheeler @ 2016-12-21 10:39 UTC (permalink / raw)
  To: Elliott, Robert (Persistent Memory); +Cc: fio

On 21 December 2016 at 07:57, Elliott, Robert (Persistent Memory)
<elliott@hpe.com> wrote:
>
>> >   *   When specifying the filename, if the block pool file does not
>> already
>> > - *   exist, then the pmemblk engine can create the pool file if
>> you specify
>> > + *   exist, then the pmemblk engine creates the pool file if you
>> specify
>> >   *   the block and file sizes.  BSIZE is the block size in bytes.
>> > - *   FSIZEMB is the pool file size in MB.
>> > + *   FSIZEMiB is the pool file size in MiB.
>>
>> It might be safer to explicitly say "in megabytes" rather than "in
>> MiB" here because of fio's infamous reversal of the terms M(B)/MiB
>> (https://github.com/axboe/fio/issues/103 ). Can the user explicitly
>> specify units as is done elsewhere in fio?
>
> "mega" is still the SI power of 10 prefix; mebi would be the IEC
> power of 2 prefix.

OK.

> I'm working on a big patch set to clean up the units throughout the
> whole codebase, and hope to post it tomorrow.
>
> For inputs (e.g., parameters in .fio files), kb_base=1000 will
> cause it to use proper units and kb_base=1024 (the default) will
> continue to use wrong units (for compatibility).  I'm guessing that
> kb_base=1000 has little use so far (there are several bugs making
> it pretty unusable right now) so we can take it over for this
> purpose.

Perhaps it would be better to phase out kb_base and just introduce
something like iec_mode instead (perhaps with an alias iecly_correct
to provide snark for those who love powers of 10 sizes)? That way
anyone depending on the buggy kb_base behaviour would at least be
forced to look up the new option when it disappeared.

> For outputs, proper units will always be displayed onscreen,
> sometimes with both side-by-side.
>
> The terse and json output formats are not changed - they'll
> continue to report KiB/s values (without units).
>
> Example text output:
>
> Jobs: 576 (f=576), 549MiB/s-80.5GiB/s: [w(288),r(288)] [0.0% done] [r=33200MiB/s,w=25544MiB/s,t=0KiB/s] [r=8499k,w=6539k,t=0 iops] [eta 06h:59m:53s]
> ...
> drive_0: (groupid=0, jobs=36): err= 0: pid=54154: Tue Dec 20 08:17:35 2016
>   write: io=75.7TiB, bw=3145MiB/s (3298MB/s), iops=805k, runt=25200023msec
>     clat percentiles (usec):
>      |  1.00th=[    0],  5.00th=[    0], 10.00th=[    0], 20.00th=[    0],
>      | 30.00th=[    0], 40.00th=[    0], 50.00th=[    0], 60.00th=[    0],
>      | 70.00th=[    0], 80.00th=[    0], 90.00th=[    0], 95.00th=[    0],
>      | 99.00th=[    0], 99.50th=[    0], 99.90th=[    0], 99.95th=[    0],
>      | 99.99th=[    0]
>   cpu          : usr=1.80%, sys=10.70%, ctx=32257714, majf=0, minf=5992
>   IO depths    : 1=15.3%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
>      submit    : 0=0.0%, 4=15.3%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
>      complete  : 0=0.0%, 4=15.3%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
>      issued    : total r=0,w=20290240440,t=0, short r=0,w=0,t=0, dropped r=0,w=0,t=0
>      latency   : target=0, window=0, percentile=100.00%, depth=1
> ...
> Run status group 0 (all jobs):
>    READ: io=837GiB (899GB), aggrbw=31.9GiB/s (34.2GB/s), minbw=31.9GiB/s (34.2GB/s), maxbw=31.9GiB/s (34.2GB/s), mint=26287msec, maxt=26287msec
>   WRITE: io=640GiB (687GB), aggrbw=24.4GiB/s (26.2GB/s), minbw=24.4GiB/s (26.2GB/s), maxbw=24.4GiB/s (26.2GB/s), mint=26300msec, maxt=26300msec

Agh... From my perspective fio's interactive output is wide enough as
it stands - it already typically exceeds 80 columns and causes
wrapping until I make the terminal is wider with moderate speed
read/write workloads. The other problem is that people really do
scrape fio's "normal" output today... I know, I know, it's not stable,
use terse/json or etc but it happens. See
https://www.youtube.com/watch?v=vm1GJMp0QN4#t=17m48s for a Brendan
Gregg fio output parsing war story (punchline is at
https://youtu.be/vm1GJMp0QN4?t=1308 ). Perhaps you only get this new
output in iec_mode? I have to admit showing both the SI unit and the
binary unit forms is the way GNU tools like dd wound up going in the
end.

The SI storage unit wars haven't generated winners - only people who
are confused what the numbers mean and who are angry the units aren't
what they want.

-- 
Sitsofe | http://sucs.org/~sits/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* RE: [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB
  2016-12-21 10:39       ` Sitsofe Wheeler
@ 2016-12-21 16:46         ` Elliott, Robert (Persistent Memory)
  0 siblings, 0 replies; 6+ messages in thread
From: Elliott, Robert (Persistent Memory) @ 2016-12-21 16:46 UTC (permalink / raw)
  To: Sitsofe Wheeler; +Cc: fio



> -----Original Message-----
> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On
> Behalf Of Sitsofe Wheeler
> Sent: Wednesday, December 21, 2016 4:40 AM
> To: Elliott, Robert (Persistent Memory) <elliott@hpe.com>
> Cc: fio@vger.kernel.org
> Subject: Re: [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB
> 
> On 21 December 2016 at 07:57, Elliott, Robert (Persistent Memory)
> <elliott@hpe.com> wrote:
> >
> >> >   *   When specifying the filename, if the block pool file does
> not
> >> already
> >> > - *   exist, then the pmemblk engine can create the pool file if
> >> you specify
> >> > + *   exist, then the pmemblk engine creates the pool file if
> you
> >> specify
> >> >   *   the block and file sizes.  BSIZE is the block size in
> bytes.
> >> > - *   FSIZEMB is the pool file size in MB.
> >> > + *   FSIZEMiB is the pool file size in MiB.
> >>
> >> It might be safer to explicitly say "in megabytes" rather than "in
> >> MiB" here because of fio's infamous reversal of the terms M(B)/MiB
> >> (https://github.com/axboe/fio/issues/103 ). Can the user
> explicitly
> >> specify units as is done elsewhere in fio?
> >
> > "mega" is still the SI power of 10 prefix; mebi would be the IEC
> > power of 2 prefix.
> 
> OK.
> 
> > I'm working on a big patch set to clean up the units throughout the
> > whole codebase, and hope to post it tomorrow.
> >
> > For inputs (e.g., parameters in .fio files), kb_base=1000 will
> > cause it to use proper units and kb_base=1024 (the default) will
> > continue to use wrong units (for compatibility).  I'm guessing that
> > kb_base=1000 has little use so far (there are several bugs making
> > it pretty unusable right now) so we can take it over for this
> > purpose.
> 
> Perhaps it would be better to phase out kb_base and just introduce
> something like iec_mode instead (perhaps with an alias iecly_correct
> to provide snark for those who love powers of 10 sizes)? That way
> anyone depending on the buggy kb_base behaviour would at least be
> forced to look up the new option when it disappeared.

I'll post the first version with kb_base, but am open to a new
parameter like that.

> Agh... From my perspective fio's interactive output is wide enough as
> it stands - it already typically exceeds 80 columns and causes
> wrapping until I make the terminal is wider with moderate speed
> read/write workloads.

> > Example text output:
> >
> > Jobs: 576 (f=576), 549MiB/s-80.5GiB/s: [w(288),r(288)] [0.0% done]
> [r=33200MiB/s,w=25544MiB/s,t=0KiB/s] [r=8499k,w=6539k,t=0 iops] [eta
> 06h:59m:53s]

The rate limits (either bandwidth or IOPS) are only shown if they
are set, so it's a bit better if there are none.

I would like to drop the trim rates if trims are not being used.

The real worst case is while all the jobs are in different thread
states, and you get something like:

Jobs: 197 (f=101), rate=197MiB/s-21.7GiB/s: 
[f(2),_(1),f(2),_(1),f(4),_(2),f(1),_(1),f(6),_(1),f(2),_(3),f(1),_(2),
f(1),_(1),f(4),_(2),f(3),_(2),f(3),_(1),f(1),_(1),f(1),E(1),f(7),_(1),f
(1),_(3),f(1),_(1),f(1),_(1),f(5),_(3),f(4),E(2),f(1),_(1),f(1),_(2),f(
3),_(2),f(2),_(1),f(2),_(1),f(6),_(2),f(1),_(2),f(1),_(2),f(1),_(2),f(4
),_(2),f(2),_(1),f(7),_(3),f(3),_(3),f(6),_(2),f(4),_(3),f(16),_(2),f(2
),_(1),f(7),_(2),f(1),E(1),f(1),_(1),f(3),_(2),f(3),_(1),f(6),_(2),f(1)
,_(1),f(4),E(1),f(2),_(1),f(3),E(1),f(3),_(6),f(1),_(1),f(1),_(2),f(1),
_(3),f(1),_(1),f(3),_(1),f(2),_(1),f(6),_(2),f(5),_(1),f(1),_(1),f(1),_
(2),f(1),_(1),E(1),_(2),f(1),E(1),_(2),f(1),_(5),E(1),_(4),f(1),_(1),E(
1),_(1),E(2),_(1),f(5),_(2),f(1),_(2),E(1),_(2),f(2),E(1),_(1),E(1),_(3
),f(2),_(3),E(1),_(1),E(1),_(5),f(1),_(1),E(1),f(2),E(1),_(1),E(1),_(1)
,f(3),E(1),_(2),E(1),_(4),f(1),_(1),f(1),E(3),_(1),f(1),E(2),_(1),f(1),
_(4),E(3),f(1),E(3),_(1),E(1),_(1),f(2),E(2),_(2),E(3),_(1),E(1),_(3),f
(1),E(2),_(1),E(2),_(1),E(1),f(1),_(2),E(1),_(1),E(5),_(5),E(6),_(32),E
(1),_(52),E(1),_(76)] [100.0% done] [25908MiB/s          |19797MiB/s|0K
iB/s] [6632K|5068K|0 iops] [eta 00m:00s]



> > ...
> > drive_0: (groupid=0, jobs=36): err= 0: pid=54154: Tue Dec 20
> 08:17:35 2016
> >   write: io=75.7TiB, bw=3145MiB/s (3298MB/s), iops=805k,
> runt=25200023msec
> > ...
> > Run status group 0 (all jobs):
> >    READ: io=837GiB (899GB), aggrbw=31.9GiB/s (34.2GB/s),
> minbw=31.9GiB/s (34.2GB/s), maxbw=31.9GiB/s (34.2GB/s),
> mint=26287msec, maxt=26287msec
> >   WRITE: io=640GiB (687GB), aggrbw=24.4GiB/s (26.2GB/s),
> minbw=24.4GiB/s (26.2GB/s), maxbw=24.4GiB/s (26.2GB/s),
> mint=26300msec, maxt=26300msec
> 

For reference, the current status group format is:
  WRITE: io=8788KB, aggrb=1075KB/s, minb=358KB/s, maxb=359KB/s, mint=8166msec, maxt=8172msec

> The other problem is that people really do
> scrape fio's "normal" output today... I know, I know, it's not
> stable, use terse/json or etc but it happens. See
> https://www.youtube.com/watch?v=vm1GJMp0QN4#t=17m48s for a Brendan
> Gregg fio output parsing war story (punchline is at
> https://youtu.be/vm1GJMp0QN4?t=1308 ). Perhaps you only get this new
> output in iec_mode? I have to admit showing both the SI unit and the
> binary unit forms is the way GNU tools like dd wound up going in the
> end.

trim values were slipped in fairly recently, even when they're not
being used.

We could change some of the lines more drastically to try to ensure
that parsing fails rather than misinterprets, like:
  write: io=75.7TiB, bw=3145MiB/s (3298MB/s), iops=805k, runt=25200023msec
to
  write: IOPS=805k, BW=3145MiB/s (3298MB/s, 75.7TiB/7H)

> 
> The SI storage unit wars haven't generated winners - only people who
> are confused what the numbers mean and who are angry the units aren't
> what they want.

The difference used to not matter; when I run the new code on a short
benchmark with a slow HDD and get 31.2kB/S (30.4KiB/s), who cares?

With large capacities and fast speeds, though, the difference has
become quite noticeable. NVDIMMs yield results like 31.9 GiB/s 
(34.2GB/s), a 7.2% difference.




^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2016-12-21 16:46 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-20 23:56 [PATCH 1/2] pmemblk: clean up error logs Robert Elliott
2016-12-20 23:56 ` [PATCH 2/2] pmemblk: Clarify fsize is in MiB not MB Robert Elliott
2016-12-21  6:05   ` Sitsofe Wheeler
2016-12-21  7:57     ` Elliott, Robert (Persistent Memory)
2016-12-21 10:39       ` Sitsofe Wheeler
2016-12-21 16:46         ` Elliott, Robert (Persistent Memory)

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.