From: Liam Beguin <liambeguin@gmail.com>
To: liambeguin@gmail.com, peda@axentia.se, jic23@kernel.org, lars@metafoo.de
Cc: linux-kernel@vger.kernel.org, linux-iio@vger.kernel.org,
devicetree@vger.kernel.org, robh+dt@kernel.org
Subject: [PATCH v8 09/14] iio: afe: rescale: fix accuracy for small fractional scales
Date: Fri, 20 Aug 2021 15:17:09 -0400 [thread overview]
Message-ID: <20210820191714.69898-10-liambeguin@gmail.com> (raw)
In-Reply-To: <20210820191714.69898-1-liambeguin@gmail.com>
From: Liam Beguin <lvb@xiphos.com>
The approximation caused by integer divisions can be costly on smaller
scale values since the decimal part is significant compared to the
integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
cases to maintain accuracy.
Signed-off-by: Liam Beguin <lvb@xiphos.com>
---
drivers/iio/afe/iio-rescale.c | 27 +++++++++++++++++++++++++--
1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
index c408c4057c08..7304306c9806 100644
--- a/drivers/iio/afe/iio-rescale.c
+++ b/drivers/iio/afe/iio-rescale.c
@@ -22,7 +22,7 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
int *val, int *val2)
{
s64 tmp;
- s32 rem;
+ s32 rem, rem2;
u32 mult;
u32 neg;
@@ -38,8 +38,31 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
tmp = (s64)*val * 1000000000LL;
tmp = div_s64(tmp, rescale->denominator);
tmp *= rescale->numerator;
- tmp = div_s64(tmp, 1000000000LL);
+
+ tmp = div_s64_rem(tmp, 1000000000LL, &rem);
*val = tmp;
+
+ /*
+ * For small values, the approximation can be costly,
+ * change scale type to maintain accuracy.
+ *
+ * 100 vs. 10000000 NANO caps the error to about 100 ppm.
+ */
+ if (scale_type == IIO_VAL_FRACTIONAL)
+ tmp = *val2;
+ else
+ tmp = 1 << *val2;
+
+ if (abs(rem) > 10000000 && abs(*val / tmp) < 100) {
+ *val = div_s64_rem(*val, tmp, &rem2);
+
+ *val2 = div_s64(rem, tmp);
+ if (rem2)
+ *val2 += div_s64(rem2 * 1000000000LL, tmp);
+
+ return IIO_VAL_INT_PLUS_NANO;
+ }
+
return scale_type;
case IIO_VAL_INT_PLUS_NANO:
case IIO_VAL_INT_PLUS_MICRO:
--
2.32.0.452.g940fe202adcb
next prev parent reply other threads:[~2021-08-20 19:18 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-20 19:17 [PATCH v8 00/14] iio: afe: add temperature rescaling support Liam Beguin
2021-08-20 19:17 ` [PATCH v8 01/14] iio: inkern: apply consumer scale on IIO_VAL_INT cases Liam Beguin
2021-08-20 19:17 ` [PATCH v8 02/14] iio: inkern: apply consumer scale when no channel scale is available Liam Beguin
2021-08-20 19:17 ` [PATCH v8 03/14] iio: inkern: make a best effort on offset calculation Liam Beguin
2021-08-20 19:17 ` [PATCH v8 04/14] iio: afe: rescale: expose scale processing function Liam Beguin
2021-08-20 19:17 ` [PATCH v8 05/14] iio: afe: rescale: add INT_PLUS_{MICRO,NANO} support Liam Beguin
2021-08-26 8:11 ` Peter Rosin
2021-08-29 2:50 ` Liam Beguin
2021-08-20 19:17 ` [PATCH v8 06/14] iio: afe: rescale: add offset support Liam Beguin
2021-08-20 19:17 ` [PATCH v8 07/14] iio: afe: rescale: use s64 for temporary scale calculations Liam Beguin
2021-08-20 19:17 ` [PATCH v8 08/14] iio: afe: rescale: reduce risk of integer overflow Liam Beguin
2021-08-26 9:13 ` Peter Rosin
2021-08-29 4:01 ` Liam Beguin
2021-08-20 19:17 ` Liam Beguin [this message]
2021-08-20 23:37 ` [PATCH v8 09/14] iio: afe: rescale: fix accuracy for small fractional scales kernel test robot
2021-08-21 1:33 ` Liam Beguin
2021-08-23 6:53 ` Peter Rosin
2021-08-21 2:00 ` kernel test robot
2021-08-21 7:21 ` kernel test robot
2021-08-22 22:18 ` Peter Rosin
2021-08-24 20:28 ` [PATCH v8 09/14] iio: afe: rescale: fix accuracy for small Liam Beguin
2021-08-26 9:53 ` Peter Rosin
2021-08-29 4:41 ` Liam Beguin
2021-08-30 11:27 ` Jonathan Cameron
2021-09-11 23:31 ` Liam Beguin
2021-08-30 13:03 ` Peter Rosin
2021-09-11 23:20 ` Liam Beguin
2021-08-30 11:22 ` [PATCH v8 09/14] iio: afe: rescale: fix accuracy for small fractional scales Jonathan Cameron
2021-08-30 14:30 ` Peter Rosin
2021-08-30 17:03 ` Jonathan Cameron
2021-09-02 2:27 ` Liam Beguin
2021-09-02 21:52 ` Peter Rosin
2021-08-20 19:17 ` [PATCH v8 10/14] iio: test: add basic tests for the iio-rescale driver Liam Beguin
2021-08-20 19:17 ` [PATCH v8 11/14] iio: afe: rescale: add RTD temperature sensor support Liam Beguin
2021-08-20 19:17 ` [PATCH v8 12/14] iio: afe: rescale: add temperature transducers Liam Beguin
2021-08-26 8:56 ` Peter Rosin
2021-08-29 2:33 ` Liam Beguin
2021-08-20 19:17 ` [PATCH v8 13/14] dt-bindings: iio: afe: add bindings for temperature-sense-rtd Liam Beguin
2021-08-20 19:17 ` [PATCH v8 14/14] dt-bindings: iio: afe: add bindings for temperature transducers Liam Beguin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210820191714.69898-10-liambeguin@gmail.com \
--to=liambeguin@gmail.com \
--cc=devicetree@vger.kernel.org \
--cc=jic23@kernel.org \
--cc=lars@metafoo.de \
--cc=linux-iio@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=peda@axentia.se \
--cc=robh+dt@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).