From: Eddie James <eajames@linux.ibm.com>
To: openbmc@lists.ozlabs.org
Subject: [PATCH linux dev-5.10 35/35] pmbus: (max31785) Retry enabling fans after writing MFR_FAN_CONFIG
Date: Mon, 8 Mar 2021 16:54:19 -0600 [thread overview]
Message-ID: <20210308225419.46530-36-eajames@linux.ibm.com> (raw)
In-Reply-To: <20210308225419.46530-1-eajames@linux.ibm.com>
From: Andrew Jeffery <andrew@aj.id.au>
It has been observed across large fleets of systems that a small subset
of those systems occasionally loose control of some number of fans
across a BMC reboot (their hwmon fan attributes are missing from sysfs).
>From extensive testing and tracing it was discovered that writes
enabling a fan in FAN_CONFIG_1_2 failed to stick on the system under
test with a frequency of about 1 in 1000 re-binds of the driver.
The MAX31785 datasheet recommends in the documentation for
MFR_FAN_CONFIG that the asssociated fan(s) be disabled before updating
the register. The sequence in question implements this suggestion, and
the observed loss-of-fans symptom occurs when the write to re-enable the
fan in FAN_CONFIG_1_2 fails to stick.
The trace data suggests a one-shot retry is enough to successfully
update FAN_CONFIG_1_2. With the workaround, no loss of fans was observed
in over 20,000 consecutive rebinds of the driver.
OpenBMC-Staging-Count: 1
Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Joel Stanley <joel@jms.id.au>
---
drivers/hwmon/pmbus/max31785.c | 23 +++++++++++++++++++----
1 file changed, 19 insertions(+), 4 deletions(-)
diff --git a/drivers/hwmon/pmbus/max31785.c b/drivers/hwmon/pmbus/max31785.c
index 1e021c38d507..0c2fe22018a3 100644
--- a/drivers/hwmon/pmbus/max31785.c
+++ b/drivers/hwmon/pmbus/max31785.c
@@ -398,6 +398,7 @@ static int max31785_of_fan_config(struct i2c_client *client,
u32 page;
u32 uval;
int ret;
+ int i;
if (!of_device_is_compatible(child, "pmbus-fan"))
return 0;
@@ -574,10 +575,24 @@ static int max31785_of_fan_config(struct i2c_client *client,
if (ret < 0)
return ret;
- ret = max31785_i2c_smbus_write_byte_data(client, PMBUS_FAN_CONFIG_12,
- pb_cfg);
- if (ret < 0)
- return ret;
+ for (i = 0; i < 2; i++) {
+ ret = max31785_i2c_smbus_write_byte_data(client,
+ PMBUS_FAN_CONFIG_12,
+ pb_cfg);
+ if (ret < 0)
+ continue;
+
+ ret = max31785_i2c_smbus_read_byte_data(client,
+ PMBUS_FAN_CONFIG_12);
+ if (ret < 0)
+ continue;
+
+ if (ret == pb_cfg)
+ break;
+ }
+
+ if (i == 2)
+ return -EIO;
/*
* Fans are on pages 0 - 5. If the page property of a fan node is
--
2.27.0
next prev parent reply other threads:[~2021-03-08 23:09 UTC|newest]
Thread overview: 61+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-08 22:53 [PATCH linux dev-5.10 00/35] Rainier and Everest system updates Eddie James
2021-03-08 22:53 ` [PATCH linux dev-5.10 01/35] ARM: dts: aspeed: rainier: Add Operator Panel LEDs Eddie James
2021-03-12 0:05 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 02/35] ARM: dts: aspeed: rainier: Add directly controlled LEDs Eddie James
2021-03-12 0:04 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 03/35] ARM: dts: aspeed: rainier: Add gpio-keys-polled for fans Eddie James
2021-03-12 0:06 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 04/35] ARM: dts: aspeed: rainier: Set MAX31785 config Eddie James
2021-03-12 0:07 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 05/35] ARM: dts: aspeed: rainier: Add additional processor CFAMs Eddie James
2021-03-12 0:07 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 06/35] ARM: dts: aspeed: rainier: Add leds that are off PCA9552 Eddie James
2021-03-12 0:09 ` Joel Stanley
2021-03-12 0:21 ` Milton Miller II
2021-03-12 0:30 ` Joel Stanley
[not found] ` <6ACEC474-8CFD-4BA9-B8FF-CCD41007AA67@linux.vnet.ibm.com>
2021-03-24 23:43 ` Joel Stanley
2021-04-26 5:59 ` vishwanatha subbanna
2021-04-27 21:22 ` Jacek Anaszewski
2021-03-08 22:53 ` [PATCH linux dev-5.10 07/35] ARM: dts: aspeed: rainier: Add leds that are off pic16f882 Eddie James
2021-03-12 0:10 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 08/35] ARM: dts: aspeed: rainier: Add leds on optional DASD cards Eddie James
2021-03-12 0:10 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 09/35] ARM: dts: aspeed: rainier: Add leds that are on optional PCI cable cards Eddie James
2021-03-12 0:11 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 10/35] ARM: dts: aspeed: rainier: Add presence GPIOs Eddie James
2021-03-12 0:14 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 11/35] ARM: dts: aspeed: rainier: Mark controllers as restricted Eddie James
2021-03-12 0:15 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 12/35] ARM: dts: aspeed: rainier 4U: Fix fan configuration Eddie James
2021-03-12 0:17 ` Joel Stanley
2021-03-08 22:53 ` [PATCH linux dev-5.10 13/35] dt: bindings: mmc: Add phase control properties for the Aspeed SDHCI Eddie James
2021-03-12 0:19 ` Joel Stanley
2021-04-12 3:21 ` Andrew Jeffery
2021-03-08 22:53 ` [PATCH linux dev-5.10 14/35] mmc: sdhci: aspeed: Expose data sample phase delay tuning Eddie James
2021-03-08 22:53 ` [PATCH linux dev-5.10 15/35] ARM: dts: aspeed: tacoma: Add data sample phase delay for eMMC Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 16/35] ARM: dts: aspeed: tacoma: Remove CFAM reset GPIO Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 17/35] ARM: dts: aspeed: Everest: Add I2C components Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 18/35] ARM: dts: Aspeed: Everest: Add max31785 fan controller device Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 19/35] ARM: dts: Aspeed: Everest: Add FSI CFAMs and re-number engines Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 20/35] ARM: dts: Aspeed: Everest: Add pca9552 fan presence Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 21/35] ARM: dts: aspeed: everest: Add power supply i2c devices Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 22/35] ARM: dts: aspeed: everest: Add UCD90320 power sequencer Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 23/35] ARM: dts: aspeed: everest: GPIOs support Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 24/35] ARM: dts: Aspeed: Everest: Add RTC Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 25/35] ARM: dts: aspeed: rainier: Support pass 2 planar Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 26/35] fsi: scom: Handle FSI2PIB timeout Eddie James
2021-03-12 0:20 ` Joel Stanley
2021-03-08 22:54 ` [PATCH linux dev-5.10 27/35] net/ncsi: Avoid channel_monitor hrtimer deadlock Eddie James
2021-03-12 0:35 ` Joel Stanley
2021-03-08 22:54 ` [PATCH linux dev-5.10 28/35] ftgmac100: Restart MAC HW once Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 29/35] hwmon: (pmbus) Add a PMBUS_NO_CAPABILITY platform data flag Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 30/35] hwmon: (pmbus/ibm-cffps) Set the PMBUS_NO_CAPABILITY flag Eddie James
2021-03-12 0:23 ` Joel Stanley
2021-03-08 22:54 ` [PATCH linux dev-5.10 31/35] i2c: Allow throttling of transfers to client devices Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 32/35] pmbus: (ucd9000) Throttle SMBus transfers to avoid poor behaviour Eddie James
2021-03-08 22:54 ` [PATCH linux dev-5.10 33/35] pmbus: (core) Add a one-shot retry in pmbus_set_page() Eddie James
2021-03-09 20:21 ` Andrei Kartashev
2021-03-12 0:35 ` Joel Stanley
2021-03-08 22:54 ` [PATCH linux dev-5.10 34/35] pmbus: (max31785) Add a local pmbus_set_page() implementation Eddie James
2021-03-08 22:54 ` Eddie James [this message]
2021-03-12 0:37 ` [PATCH linux dev-5.10 00/35] Rainier and Everest system updates Joel Stanley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210308225419.46530-36-eajames@linux.ibm.com \
--to=eajames@linux.ibm.com \
--cc=openbmc@lists.ozlabs.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).