From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932925AbdDJRAr (ORCPT ); Mon, 10 Apr 2017 13:00:47 -0400 Received: from mout.kundenserver.de ([212.227.17.24]:53633 "EHLO mout.kundenserver.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932795AbdDJRAo (ORCPT ); Mon, 10 Apr 2017 13:00:44 -0400 Date: Mon, 10 Apr 2017 19:00:01 +0200 From: Andreas Klinger To: Jonathan Cameron , Matt Ranostay , Linus Walleij , Hartmut Knaack , Lars-Peter Clausen , Peter Meerwald , Vlad Dogaru , Akinobu Mita , Wei Yongjun , Aniroop Mathur Cc: "linux-iio@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2] IIO: bmp280-core.c: fix error in humidity calculation Message-ID: <20170410170001.GA2524@project> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Provags-ID: V03:K0:mtYotcqrRX744j7TsVNr/4eK6Nqf0MKMxEIglPT1iA0MuVFLivL YDnMb9kPQiFEPJQxpAiOYIQ7p0tCjyBD1iMNsX2xj+xh5g46fTBWdf0WnOvXUKzu8ame6e8 3MaePzpDY3fuea8+LgFdTz0vrivYygSib8JYZMdE3yTYW6SxSczbdDIQeGeEtdG/gAoG0gd PWOD0vUhD2v6aIESs8LCw== X-UI-Out-Filterresults: notjunk:1;V01:K0:ffsJX2Wfutk=:aJL3UEZUK4OtU2VC9hOE11 gANj1xun+i1axWyEQ5aStlBl+ivEwJXHxI5go6XnswdH6ffwC7jISycji+5l2aZy0j1XJNFbb fxFtAhFmNQFG7SrXqdypJVJSxJb3y3QB1L4kK911QR/c1iyPNrVGlR867DvyunNiTm68f8Bwo KqekyqbTjKxZz4ONootif1hfaT2Rkce8hRe3w8rVF9GlFaKJe5EJv4Xo9zmTYO3LqdJSRCeFc JZkfWDkoaqK37jPU61O2E7VBGKaSeIzOiacwZS2PDuLKFo+U24Ghz2H80Da55s6cFZirpCqrx gq5W/xfD6yvulq/wY2S9+a3d2IMQmmgGPtjsMy5ebIKkMBEohMJbADjC+jRfeLCKMdsaBYDzH 53v5qv3TLBbie4YelouIFPwxb/XUwymgt3u8k8GKsPQiv3vI9V+4wFLbTb44J28YrU4HxG/Gv WHx+3HZj1GIlZGEvvQ/58WvsFuHgUpnMtRkRk9mDsbiqD5ir5BsONictbmRD/fePVkdSwsHm6 AcxCEib6wH120cYql9y3g4RkpbF7mY28vrDocEi+61UH+yVqmSdEexLbf3un654Ti0UDZll9c c8iYufpkT/n6vnsLFXocx9l73r9IA9vtSiOWTADhlnD/gipRtvGTPb4mkPnbP/RWbsA/k2eHl ym0lPJRwkyrGlXmYgJDTYYVdqKMZ/HVCZO3Fr416enz++Noz1N4N1CIjdP20U/0xd0wY= Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org While calculating the compensation of the humidity there are negative values interpreted as unsigned because of unsigned variables used. These values as well as the constants need to be casted to signed as indicated by the documentation of the sensor. Changes in v2: - cast also constants to signed; otherwise calculation is not correct I removed the Acked-By of Linus Walleij because of another change in the calculation. Don't know how it should be treated. Signed-off-by: Andreas Klinger --- drivers/iio/pressure/bmp280-core.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/iio/pressure/bmp280-core.c b/drivers/iio/pressure/bmp280-core.c index 4d18826ac63c..d82b788374b6 100644 --- a/drivers/iio/pressure/bmp280-core.c +++ b/drivers/iio/pressure/bmp280-core.c @@ -175,11 +175,12 @@ static u32 bmp280_compensate_humidity(struct bmp280_data *data, } H6 = sign_extend32(tmp, 7); - var = ((s32)data->t_fine) - 76800; - var = ((((adc_humidity << 14) - (H4 << 20) - (H5 * var)) + 16384) >> 15) - * (((((((var * H6) >> 10) * (((var * H3) >> 11) + 32768)) >> 10) - + 2097152) * H2 + 8192) >> 14); - var -= ((((var >> 15) * (var >> 15)) >> 7) * H1) >> 4; + var = ((s32)data->t_fine) - (s32)76800; + var = ((((adc_humidity << 14) - (H4 << 20) - (H5 * var)) + + (s32)16384) >> 15) * (((((((var * H6) >> 10) + * (((var * (s32)H3) >> 11) + (s32)32768)) >> 10) + + (s32)2097152) * H2 + 8192) >> 14); + var -= ((((var >> 15) * (var >> 15)) >> 7) * (s32)H1) >> 4; return var >> 12; }; -- 2.1.4 --