All of lore.kernel.org
 help / color / mirror / Atom feed
* [BUG] parse.c: val_too_large incorrectly handling overly large FIO_OPT_INT values
@ 2020-05-08 13:47 Stephen  Bates
  2020-05-08 20:48 ` Elliott, Robert (Servers)
  0 siblings, 1 reply; 5+ messages in thread
From: Stephen  Bates @ 2020-05-08 13:47 UTC (permalink / raw)
  To: fio; +Cc: Jens Axboe, David Sloan

Hi All

While trying to do some rate limiting in a high performance NVMe system we noticed that setting --rate to many values over 4G was broken (e.g. 8GB). As we dug into that it seems like the error handling for any FIO_OPTS_INT >=4GiB is not working because val_too_large in parse.c is conflating large positives uints with small negatives units uints and letting large values slide through. Tested against the tip of master (b2ed1c4a07c6 at time of writing).

I *would* submit a patch but it is actually a little tricky to keep the patch contained in val_too_large and separate the small negative number issue from the big positive number issue. So I wondered if anyone had any suggestions. I don't really want to propose something like adding a FIO_OPTS_UINT if there is an easier fix. We could also consider changing val to a signed long long but (I think) breaks the case when a ULL is passed in.

I have also created a GitHub issue here [1]. You can test with something like

static bool val_too_large(const struct fio_option *o, unsigned long long val,
                          bool is_uint)
{
        if (!o->maxval)
                return false;

        if (is_uint) {
                if ((int) val < 0)
                        return (int) val > (int) o->maxval;
                return (unsigned int) val > o->maxval;
        }

	return val > o->maxval;
}

You can test this with something like:

./fio --debug=parse --name=parse_check --rate=8G,8G --size=16k --filename /tmp/parse.dat

Cheers
 
Stephen

[1]: https://github.com/axboe/fio/issues/975
 


^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: [BUG] parse.c: val_too_large incorrectly handling overly large FIO_OPT_INT values
  2020-05-08 13:47 [BUG] parse.c: val_too_large incorrectly handling overly large FIO_OPT_INT values Stephen  Bates
@ 2020-05-08 20:48 ` Elliott, Robert (Servers)
  2020-05-08 20:56   ` Stephen  Bates
  0 siblings, 1 reply; 5+ messages in thread
From: Elliott, Robert (Servers) @ 2020-05-08 20:48 UTC (permalink / raw)
  To: Stephen Bates, fio; +Cc: Jens Axboe, David Sloan



> -----Original Message-----
> From: fio-owner@vger.kernel.org <fio-owner@vger.kernel.org> On Behalf
> Of Stephen Bates
> Sent: Friday, May 08, 2020 8:47 AM
> To: fio@vger.kernel.org
> Cc: Jens Axboe <axboe@kernel.dk>; David Sloan
> <David.Sloan@eideticom.com>
> Subject: [BUG] parse.c: val_too_large incorrectly handling overly
> large FIO_OPT_INT values
> 
> Hi All
> 
> While trying to do some rate limiting in a high performance NVMe
> system we noticed that setting --rate to many values over 4G was
> broken (e.g. 8GB). As we dug into that it seems like the error
> handling for any FIO_OPTS_INT >=4GiB is not working because
> val_too_large in parse.c is conflating large positives uints with
> small negatives units uints and letting large values slide through.
> Tested against the tip of master (b2ed1c4a07c6 at time of writing).
> 
> I *would* submit a patch but it is actually a little tricky to keep
> the patch contained in val_too_large and separate the small negative
> number issue from the big positive number issue. So I wondered if
> anyone had any suggestions. I don't really want to propose something
> like adding a FIO_OPTS_UINT if there is an easier fix. We could also
> consider changing val to a signed long long but (I think) breaks the
> case when a ULL is passed in.
...
> You can test this with something like:
> 
> ./fio --debug=parse --name=parse_check --rate=8G,8G --size=16k --
> filename /tmp/parse.dat

Fundamentally, this argument needs to support larger than
32-bit values. Persistent memory devices easily exceed that
bandwidth too.

Other arguments like "size" already take large values into an
unsigned long long. So, just changing this for "rate" and
"rate_min" might suffice:
                .type   = FIO_OPT_INT,
to
                .type   = FIO_OPT_STR_VAL,





^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [BUG] parse.c: val_too_large incorrectly handling overly large FIO_OPT_INT values
  2020-05-08 20:48 ` Elliott, Robert (Servers)
@ 2020-05-08 20:56   ` Stephen  Bates
  2020-05-22 18:27     ` Elliott, Robert (Servers)
  0 siblings, 1 reply; 5+ messages in thread
From: Stephen  Bates @ 2020-05-08 20:56 UTC (permalink / raw)
  To: Elliott, Robert (Servers), fio; +Cc: Jens Axboe, David Sloan

Robert

> Fundamentally, this argument needs to support larger than
>    32-bit values. Persistent memory devices easily exceed that
>    bandwidth too.
>
>    Other arguments like "size" already take large values into an
>    unsigned long long. So, just changing this for "rate" and
>    "rate_min" might suffice:
>                    .type   = FIO_OPT_INT,
>    to
>                    .type   = FIO_OPT_STR_VAL,

Yup but there are two issues here:

1. (This bug report) The range handling of all FIO_OPT_INT arguments is broken.

2. rate (and min_rate) should support values bigger than 4G. 

I am working on a PR/patch for item 2 [1]. Looking for guidance on item 1.

Cheers
 
Stephen

[1]: https://github.com/axboe/fio/issues/716


^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: [BUG] parse.c: val_too_large incorrectly handling overly large FIO_OPT_INT values
  2020-05-08 20:56   ` Stephen  Bates
@ 2020-05-22 18:27     ` Elliott, Robert (Servers)
  2020-05-22 18:59       ` Jens Axboe
  0 siblings, 1 reply; 5+ messages in thread
From: Elliott, Robert (Servers) @ 2020-05-22 18:27 UTC (permalink / raw)
  To: Stephen Bates, fio; +Cc: Jens Axboe, David Sloan


> > Fundamentally, this argument needs to support larger than
> >    32-bit values. Persistent memory devices easily exceed that
> >    bandwidth too.
> >
> >    Other arguments like "size" already take large values into an
> >    unsigned long long. So, just changing this for "rate" and
> >    "rate_min" might suffice:
> >                    .type   = FIO_OPT_INT,
> >    to
> >                    .type   = FIO_OPT_STR_VAL,
> 
> Yup but there are two issues here:
> 
> 1. (This bug report) The range handling of all FIO_OPT_INT arguments
> is broken.
> 
> 2. rate (and min_rate) should support values bigger than 4G.
> 
> I am working on a PR/patch for item 2 [1]. Looking for guidance on
> item 1.
> 

Should at least item 2 go into 3.20?


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [BUG] parse.c: val_too_large incorrectly handling overly large FIO_OPT_INT values
  2020-05-22 18:27     ` Elliott, Robert (Servers)
@ 2020-05-22 18:59       ` Jens Axboe
  0 siblings, 0 replies; 5+ messages in thread
From: Jens Axboe @ 2020-05-22 18:59 UTC (permalink / raw)
  To: Elliott, Robert (Servers), Stephen Bates, fio; +Cc: David Sloan

On 5/22/20 12:27 PM, Elliott, Robert (Servers) wrote:
> 
>>> Fundamentally, this argument needs to support larger than
>>>    32-bit values. Persistent memory devices easily exceed that
>>>    bandwidth too.
>>>
>>>    Other arguments like "size" already take large values into an
>>>    unsigned long long. So, just changing this for "rate" and
>>>    "rate_min" might suffice:
>>>                    .type   = FIO_OPT_INT,
>>>    to
>>>                    .type   = FIO_OPT_STR_VAL,
>>
>> Yup but there are two issues here:
>>
>> 1. (This bug report) The range handling of all FIO_OPT_INT arguments
>> is broken.
>>
>> 2. rate (and min_rate) should support values bigger than 4G.
>>
>> I am working on a PR/patch for item 2 [1]. Looking for guidance on
>> item 1.
>>
> 
> Should at least item 2 go into 3.20?

Should already be in the repo.

-- 
Jens Axboe



^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-05-22 18:59 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-08 13:47 [BUG] parse.c: val_too_large incorrectly handling overly large FIO_OPT_INT values Stephen  Bates
2020-05-08 20:48 ` Elliott, Robert (Servers)
2020-05-08 20:56   ` Stephen  Bates
2020-05-22 18:27     ` Elliott, Robert (Servers)
2020-05-22 18:59       ` Jens Axboe

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.