From: Chris Packham <Chris.Packham@alliedtelesis.co.nz>
To: Guenter Roeck <linux@roeck-us.net>,
"jdelvare@suse.com" <jdelvare@suse.com>
Cc: "linux-hwmon@vger.kernel.org" <linux-hwmon@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-i2c@vger.kernel.org" <linux-i2c@vger.kernel.org>,
"linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
Subject: Re: Errant readings on LM81 with T2080 SoC
Date: Mon, 8 Mar 2021 04:37:47 +0000 [thread overview]
Message-ID: <a67ea323-634d-d34e-c63e-b1aaa4737b19@alliedtelesis.co.nz> (raw)
In-Reply-To: <4a1b1494-df96-2d8c-9323-beb2c2ba706b@alliedtelesis.co.nz>
On 8/03/21 3:27 pm, Chris Packham wrote:
>
> On 8/03/21 1:31 pm, Guenter Roeck wrote:
>> On 3/7/21 2:52 PM, Chris Packham wrote:
>>> Hi,
>>>
>>> I've got a system using a PowerPC T2080 SoC and among other things has
>>> an LM81 hwmon chip.
>>>
>>> Under a high CPU load we see errant readings from the LM81 as well as
>>> actual failures. It's the errant readings that cause the most concern
>>> since we can easily ignore the read errors in our monitoring
>>> application
>>> (although it would be better if they weren't there at all).
>>>
>>> I'm able to reproduce this with a test application[0] that artificially
>>> creates a high CPU load then by repeatedly checking for the all-1s
>>> values from the LM81 datasheet[1](page 17). The all-1s readings stick
>>> out as they are obviously higher than the voltage rails that are
>>> connected and disagree with measurements taken with a multimeter.
>>>
>>> Here's the output from my device
>>>
>>> [root@linuxbox ~]# cpuload 90&
>>> [root@linuxbox ~]# (while true; do cat
>>> /sys/class/hwmon/hwmon0/in*_input
>>> | grep '3320\|4383\|6641\|15930\|3586'; sleep 1; done)&
>>> 3586
>>> 3586
>>> cat: read error: No such device or address
>>> cat: read error: No such device or address
>>> 3320
>>> 3320
>>> 3586
>>> 3586
>>> 6641
>>> 6641
>>> 4383
>>> 4383
>>>
>>> Fundamentally I think this is a problem with the fact that the LM81 is
>>> an SMBus device but the T2080 (and other Freescale SoCs) uses i2c
>>> and we
>>> emulate SMBus. I suspect the errant readings are when we don't get
>>> round
>>> to completing the read within the timeout specified by the SMBus
>>> specification. Depending on when that happens we either fail the
>>> transfer or interpret the result as all-1s.
>>>
>> That is quite unlikely. Many sensor chips are SMBus chips connected to
>> i2c busses. It is much more likely that there is a bug in the T2080
>> i2c driver,
>> that the chip doesn't like the bulk read command issued through
>> regmap, that
>> the chip has problems with the i2c bus speed, or that the i2c bus is
>> noisy.
> Perhaps something gets upset when interrupt processing is delayed
> because of CPU load. I don't see the problem when there isn't a CPU
> load so I think that eliminates board issues.
>> In this context, the "No such device or address" responses are very
>> suspicious.
>> Those are reported by the i2c driver, not by the hwmon driver, and
>> suggest
>> that the chip did not respond to a read request. Maybe it helps to
>> enable
>> debugging to the i2c driver to see if it reports anything useful. Even
>> better might be to connect an i2c bus analyzer to the i2c bus and check
>> what is going on.
> That's from -ENXIO which is used in only one place in i2c-mpc.c. I'll
> enable some debug and see what we get.
For the errant readings there was nothing abnormal reported by the driver.
For the "No such device or address" I saw "mpc-i2c ffe119000.i2c: No
RXAK" which matches up with the -ENXIO return.
next prev parent reply other threads:[~2021-03-08 4:39 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-07 22:52 Errant readings on LM81 with T2080 SoC Chris Packham
2021-03-08 0:31 ` Guenter Roeck
2021-03-08 2:27 ` Chris Packham
2021-03-08 4:37 ` Chris Packham [this message]
2021-03-08 4:59 ` Guenter Roeck
2021-03-08 20:27 ` Chris Packham
2021-03-08 22:39 ` Guenter Roeck
2021-03-10 2:19 ` Chris Packham
2021-03-10 5:06 ` Guenter Roeck
2021-03-10 21:48 ` Chris Packham
2021-03-11 7:41 ` Guenter Roeck
2021-03-11 8:18 ` Wolfram Sang
2021-03-11 15:19 ` Guenter Roeck
2021-03-11 21:17 ` Chris Packham
2021-03-11 21:34 ` Guenter Roeck
2021-03-11 23:47 ` Chris Packham
2021-03-12 0:07 ` Guenter Roeck
2021-03-12 0:19 ` Chris Packham
2021-03-12 9:25 ` David Laight
2021-03-14 21:26 ` Chris Packham
2021-03-15 9:46 ` David Laight
2021-03-18 5:44 ` Wolfram Sang
2021-03-18 3:46 ` Chris Packham
2021-03-18 4:02 ` Guenter Roeck
2021-03-18 5:39 ` Wolfram Sang
2021-03-08 22:10 ` Chris Packham
2021-03-09 4:36 ` Chris Packham
2021-03-09 5:24 ` Guenter Roeck
2021-03-09 23:35 ` Chris Packham
2021-03-10 3:29 ` Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a67ea323-634d-d34e-c63e-b1aaa4737b19@alliedtelesis.co.nz \
--to=chris.packham@alliedtelesis.co.nz \
--cc=jdelvare@suse.com \
--cc=linux-hwmon@vger.kernel.org \
--cc=linux-i2c@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@roeck-us.net \
--cc=linuxppc-dev@lists.ozlabs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).