meta-phosphor/sensors and phosphor-power/phosphor-regulators compatibility

meta-phosphor/sensors and phosphor-power/phosphor-regulators compatibility

[Mike Jones]

Hi,

I think meta-phosphor/sensors and phosphor-power/phosphor-regulators interfere with each other.

When I power on chassis, journalctl shows errors from phosphor-regulators saying the device or resource is busy and mentions the correct i2c. It stops trying after a few attempts. Looking at the code, I see that phosphor-regulators is using /dev/i2c.

At the time of chassis on, the i2c has traffic on the bus, from sensors.

Is there a way to make these play nice?

Mike

Sent from my iPad

Fwd: meta-phosphor/sensors and phosphor-power/phosphor-regulators compatibility

[Shawn McCarney]

[-- Attachment #1: Type: text/plain, Size: 2223 bytes --]

Forgot to cc the list of this reponse

-------- Forwarded Message --------
Subject: 	Re: meta-phosphor/sensors and 
phosphor-power/phosphor-regulators compatibility
Date: 	Fri, 11 Feb 2022 09:46:09 -0600
From: 	Shawn McCarney <shawnmm@linux.ibm.com>
To: 	Mike Jones <proclivis@gmail.com>



Hi,
> I think meta-phosphor/sensors and phosphor-power/phosphor-regulators 
> interfere with each other.
>
> When I power on chassis, journalctl shows errors from 
> phosphor-regulators saying the device or resource is busy and mentions 
> the correct i2c. It stops trying after a few attempts. Looking at the 
> code, I see that phosphor-regulators is using /dev/i2c.
>
> At the time of chassis on, the i2c has traffic on the bus, from sensors.
>
> Is there a way to make these play nice?
>
Hi,

If dbus-sensors requires use of a device driver, I'm not sure if both 
applications can read sensors from the same regulator at the same time.  
As noted in my other response, phosphor-regulators communicates directly 
with the regulator via i2c-dev.  That is because most regulator drivers 
don't support regulator configuration, which is one of the key functions 
of phosphor-regulators.

phosphor-regulators supports the following sensor types: iout, 
iout_peak, iout_valley, pout, temperature, temperature_peak, vout, 
vout_peak, vout_valley.  Does that cover the ones you want to read?  If 
so, could you switch to doing all of your sensor readings from 
phosphor-regulators?

Otherwise, if each application is missing one or two key sensors, 
perhaps dbus-sensors or phosphor-regulators could be enhanced to provide 
the missing ones?  I'm pretty booked right now, but contributions are 
welcome.

Long term, it would be ideal for phosphor-regulators to move to 
driver-based communication.  That would avoid this issue.  I think we 
would need define a new regulator device driver framework that provided 
full configuration capability.  It would need to support not just PMBus 
configuration but use of manufacturer-specific interfaces or registers.  
Last time I read about it, the Linux framework provided only very 
limited configuration and was not intended to be used by user-space 
applications.

Thanks,

Shawn

[-- Attachment #2: Type: text/html, Size: 3667 bytes --]

Re: meta-phosphor/sensors and phosphor-power/phosphor-regulators compatibility

[Shawn McCarney]

> So I was looking for a way to pull a block of data for the fault log, 
> but I was hoping hwmon would still work in parallel, because it is 
> used a lot.
>
There may be support in phosphor-regulators you could use, although it 
may be intended for a different use case.

Regulators may have redundant phases.  For example, an N+1 regulator has 
one redundant phase.  If a phase stops working, the regulator will still 
work but the customer has lost redundancy. So an error can be logged to 
warn the customer.

As part of logging the phase fault error, you have the option of 
capturing an arbitrary set of debug data.  This is stored in the 
resulting error log as additional data.

You could possibly use this support for your use case.  The logic to 
determine if a 'phase fault' occurred is completely defined in the JSON 
file, so it could do whatever I2C registers compares you want.  
However,  the error message would include 'phase fault' which might be 
misleading in your case.

See the following for more info:
* 
https://github.com/openbmc/phosphor-power/blob/master/phosphor-regulators/docs/config_file/phase_fault_detection.md

* 
https://github.com/openbmc/phosphor-power/blob/master/phosphor-regulators/docs/config_file/log_phase_fault.md

* 
https://github.com/openbmc/phosphor-power/blob/master/phosphor-regulators/docs/config_file/i2c_capture_bytes.md 
for more info.

> Also, min/max values are not directly supported, clearing those 
> values, etc.
>
the *_peak and *_valley sensor types are intended for max/min sensor 
values that are calculated by the regulator device itself. The regulator 
usually samples values very frequently and is more accurate than a 
firmware calculated min/max.  Currently the _peak and _valley sensors 
are cleared on each boot.  You are correct that there is not currently 
support to clear it manually.

> But, if phosphor-regulators could consume hwmon values, or some other 
> driver interface, that would have value. Currently, I have ADI 
> customers that want to switch from a PMBus Hot Swap to a I2C Hotswap 
> to get some features, and they are in pain because their software uses 
> PMBus directly via i2c. If they had used hwmon, I could have written 
> them an emulation hwmon driver and all their software would have just 
> worked. The lack of driver abstraction is making their product 
> evolution difficult. This makes me hesitate wrt using 
> phosphor-regulators.
>
I understand your concern here.  I would not claim the 
phosphor-regulators application solves all use cases or is an ideal 
design.  It provides some functionality that was mandatory on the 
systems I work on.  I was hopeful it would be useful to others, but 
there are definitely caveats.

> I will invest more time in understanding phosphor-regulators. Mainly I 
> need to know how telemetry here would get connected to Redfish and 
> WebUI, etc. Does all the telemetry you mentioned below automatically 
> get connected to these?
>
I answered this in another response.

Thanks,

Shawn