From: Peter Rosin <peda@axentia.se>
To: Andy Shevchenko <andy.shevchenko@gmail.com>,
Liam Beguin <liambeguin@gmail.com>
Cc: Jonathan Cameron <jic23@kernel.org>,
Lars-Peter Clausen <lars@metafoo.de>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
linux-iio <linux-iio@vger.kernel.org>,
devicetree <devicetree@vger.kernel.org>,
Rob Herring <robh+dt@kernel.org>
Subject: Re: [PATCH v12 09/16] iio: afe: rescale: fix accuracy for small fractional scales
Date: Sun, 9 Jan 2022 21:07:30 +0100 [thread overview]
Message-ID: <6901bb1b-b701-9760-1d36-89e8127c1953@axentia.se> (raw)
In-Reply-To: <CAHp75Vd-FOV7BL0VjhGLyC5fhYXbW3x-hC5J1VvMT9W3Kfc_0Q@mail.gmail.com>
Hi!
On 2022-01-09 14:02, Andy Shevchenko wrote:
> On Sat, Jan 8, 2022 at 10:53 PM Liam Beguin <liambeguin@gmail.com> wrote:
>>
>> The approximation caused by integer divisions can be costly on smaller
>> scale values since the decimal part is significant compared to the
>> integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
>> cases to maintain accuracy.
>
...
> ...
>
>> + rem2 = *val % (int)tmp;
>> + *val = *val / (int)tmp;
>> +
>> + *val2 = rem / (int)tmp;
>
> Hmm... You divide s64 by 10^9, which means that the maximum value can
> be ~10^10 / 2 (because 2^64-1 ~= 10^19), but this _might_ be bigger
> than 'int' can hold. Can you confirm that tmp can't be so big?
>
If tmp is so big that it does not fit in 32-bits, that is indeed a
problem, and it means that the scale has overflowed. However, the
problem with the scale overflowing very much existed prior to this
series. Doing something about that overflow is not strictly related
to improving the accuracy for small scale factors.
Cheers,
Peter
next prev parent reply other threads:[~2022-01-09 20:07 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-08 20:53 [PATCH v12 00/16] iio: afe: add temperature rescaling support Liam Beguin
2022-01-08 20:53 ` [PATCH v12 01/16] iio: inkern: apply consumer scale on IIO_VAL_INT cases Liam Beguin
2022-01-08 20:53 ` [PATCH v12 02/16] iio: inkern: apply consumer scale when no channel scale is available Liam Beguin
2022-01-08 20:53 ` [PATCH v12 03/16] iio: inkern: make a best effort on offset calculation Liam Beguin
2022-01-08 20:53 ` [PATCH v12 04/16] iio: afe: rescale: use s64 for temporary scale calculations Liam Beguin
2022-01-08 20:53 ` [PATCH v12 05/16] iio: afe: rescale: reorder includes Liam Beguin
2022-01-08 20:53 ` [PATCH v12 06/16] iio: afe: rescale: expose scale processing function Liam Beguin
2022-01-09 13:07 ` Andy Shevchenko
2022-01-09 20:19 ` Peter Rosin
2022-01-09 20:25 ` Peter Rosin
2022-01-11 14:43 ` Liam Beguin
2022-01-11 14:35 ` Liam Beguin
2022-01-08 20:53 ` [PATCH v12 07/16] iio: afe: rescale: add INT_PLUS_{MICRO,NANO} support Liam Beguin
2022-01-09 12:48 ` Andy Shevchenko
2022-01-09 20:20 ` Peter Rosin
2022-01-11 14:33 ` Liam Beguin
2022-01-08 20:53 ` [PATCH v12 08/16] iio: afe: rescale: add offset support Liam Beguin
2022-01-08 20:53 ` [PATCH v12 09/16] iio: afe: rescale: fix accuracy for small fractional scales Liam Beguin
2022-01-09 13:02 ` Andy Shevchenko
2022-01-09 20:07 ` Peter Rosin [this message]
2022-01-08 20:53 ` [PATCH v12 10/16] iio: afe: rescale: reduce risk of integer overflow Liam Beguin
2022-01-08 20:53 ` [PATCH v12 11/16] iio: afe: rescale: make use of units.h Liam Beguin
2022-01-08 20:53 ` [PATCH v12 12/16] iio: test: add basic tests for the iio-rescale driver Liam Beguin
2022-01-08 20:53 ` [PATCH v12 13/16] iio: afe: rescale: add RTD temperature sensor support Liam Beguin
2022-01-08 20:53 ` [PATCH v12 14/16] iio: afe: rescale: add temperature transducers Liam Beguin
2022-01-08 20:53 ` [PATCH v12 15/16] dt-bindings: iio: afe: add bindings for temperature-sense-rtd Liam Beguin
2022-01-08 20:53 ` [PATCH v12 16/16] dt-bindings: iio: afe: add bindings for temperature transducers Liam Beguin
2022-01-09 13:10 ` [PATCH v12 00/16] iio: afe: add temperature rescaling support Andy Shevchenko
2022-01-30 14:39 ` Jonathan Cameron
2022-01-30 14:41 ` Jonathan Cameron
2022-01-30 14:59 ` Liam Beguin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6901bb1b-b701-9760-1d36-89e8127c1953@axentia.se \
--to=peda@axentia.se \
--cc=andy.shevchenko@gmail.com \
--cc=devicetree@vger.kernel.org \
--cc=jic23@kernel.org \
--cc=lars@metafoo.de \
--cc=liambeguin@gmail.com \
--cc=linux-iio@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=robh+dt@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).