openbmc.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
* Re: No dbus objects for phosphor-regulators
@ 2022-01-31 22:25 Mike
  2022-02-01 17:37 ` Shawn McCarney
  0 siblings, 1 reply; 11+ messages in thread
From: Mike @ 2022-01-31 22:25 UTC (permalink / raw)
  To: Shawn McCarney; +Cc: OpenBMC Maillist

[-- Attachment #1: Type: text/plain, Size: 4804 bytes --]

Jason,

Sent from my iPhone

> On Jan 26, 2022, at 3:43 PM, Shawn McCarney <shawnmm@linux.ibm.com> wrote:
> 
> Hi Mike,
> 
>> I made a config file below, and verified that it is on the target and loads without error.
> Just to make sure I understand what you are saying, on the BMC your 'config.json' file is installed in the /usr/share/phosphor-regulators directory
> 
Correct

> And when you run 'journalctl -u phosphor-regulators.service', you see a journal message stating that the phosphor-regulators application has loaded your JSON file successfully?  And no error messages?
> 

Correct

>> When the target boots, using a I2C spy tool, the 0xDD command is being read periodically, suggesting that this service is processing the read command, and a query show the service is up.
> Sensor reading should begin when the chassis is powered on and stop when the chassis is powered off.  That is because some regulators don't have power at standby or report invalid sensor readings.
> 

I am using the SDK, so I have not worked on chassis power, so I don’t know if it is powered or not, but this gives me a good hint about the problem. Even so, the read command in config.json is getting polled. The other config item to modify the voltage did not occur.

Given I am using an Aspeed EVK, is there an example for how to turn on a chassis with a GPIO, or a dbus operation, or an automatic turn on at boot?
> Does the 'journalctl | grep -i Regulators' show that the service files are being executed that enable and disable regulator sensor monitoring?
> 
No, only the daemon starting.
>> Using dbus-monitor –system, I do not see any transactions that place these sensor read values on dbus, nor does the normal probing the chassis and inventory show values.
> Since you are reading a vout sensor for the rail named "vdd", the resulting D-Bus object path for the sensor should be
> 
>   /xyz/openbmc_project/sensors/voltage/vdd_vout
> 
> See https://github.com/openbmc/phosphor-power/blob/master/phosphor-regulators/docs/config_file/pmbus_read_sensor.md#d-bus-sensor for more details.
> 
> Does that object path exist on D-Bus after you've powered on the chassis and sensor monitoring has been enabled?
> 
It does not exist.

> You can see all the properties of the D-Bus sensor using the command
> 
>   busctl introspect xyz.openbmc_project.Power.Regulators /xyz/openbmc_project/sensors/voltage/vdd_vout
> 
> You can see all regulator sensor object paths via the command:
> 
>   busctl tree xyz.openbmc_project.Power.Regulators
> 
> The phosphor-regulators application reads sensors once per second.  However, it only updates D-Bus if the new sensor value has changed more than a hysteresis amount.  This is to avoid generating a bunch of D-Bus traffic for minor value changes.
> 
> You can see the hysteresis values here: https://github.com/openbmc/phosphor-power/blob/master/phosphor-regulators/src/dbus_sensor.cpp#L66
> 
>> What else do I need to so that there are dbus objects?
> For bmcweb and Redfish to work, you need associations between the D-Bus sensor and two D-Bus inventory objects:
> 
> * The FRU that contains the regulator.  This is the replaceable hardware item where the regulator is located.  Some regulators are FRUs by themselves (replaceable).  Some are located on larger FRUs like a motherboard.  This is used to get presence, health info, and if applicable VPD.
> 
Ah, it did not occurs to me that a mother board as FRU is wapping regulators inside an FRU. Seems obvious now.
> * The chassis that contains the regulator.  This is required by Redfish since it groups sensors by chassis.
> 
Ok

> The phosphor-regulators application creates those associations automatically based on information in your JSON file.  The "fru" property of the regulator "device" provides the first inventory object path.  The "inventory_path" property of the "chassis" provides the second inventory path.  Both of those are relative to the "/xyz/openbmc_project/inventory" root path.
> 
> Do the "fru" and "inventory_path" properties in your JSON file match the correct inventory object paths on your system?
> 
I have a psu.json with fruConfigs, and this has

“PsuDevices”: {
  “/xyz/openbmc-project/inventory/system/chassis/motherboard/powersupply0” : “/sys/bus/i2c/devices/11-004f”,
}

Which is the same i2c address as used by phosphor-regulators.

And a power-supply-monitor-0.conf to match.
>> Note I have not done anything to intentionally use open_power. My goal is to only use phosphor-regulators to expose some telemetry that hwmon does not support.
> That's fine.  This application does not have any dependencies on open_power.
> 
> Let me know if you need more info.
> 
> Shawn
> 
Mike


[-- Attachment #2: Type: text/html, Size: 8231 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-01-31 22:25 No dbus objects for phosphor-regulators Mike
@ 2022-02-01 17:37 ` Shawn McCarney
  2022-02-09 22:30   ` Mike Jones
  0 siblings, 1 reply; 11+ messages in thread
From: Shawn McCarney @ 2022-02-01 17:37 UTC (permalink / raw)
  To: Mike; +Cc: OpenBMC Maillist

[-- Attachment #1: Type: text/plain, Size: 4739 bytes --]


>>> When the target boots, using a I2C spy tool, the 0xDD command is 
>>> being read periodically, suggesting that this service is processing 
>>> the read command, and a query show the service is up.
>>>
>> Sensor reading should begin when the chassis is powered on and stop 
>> when the chassis is powered off.  That is because some regulators 
>> don't have power at standby or report invalid sensor readings.
>>
>
> I am using the SDK, so I have not worked on chassis power, so I don’t 
> know if it is powered or not, but this gives me a good hint about the 
> problem. Even so, the read command in config.json is getting polled. 
> The other config item to modify the voltage did not occur.

It sounds like at least part of the problem might be the systemd service 
files not running.  The regulators service files are located in 
https://github.com/openbmc/phosphor-power/tree/master/services.

* phosphor-regulators.service:  This one launches the regulators app.  
This must be happening since you are a getting journal message about it 
loading the JSON config file.

* phosphor-regulators-config.service:  This is what causes the 
configuration entries to be executed in the JSON file.  This should 
happen early in the process of powering on the chassis before the 
regulators have received power (been enabled).

* phosphor-regulators-monitor-enable.service:  This enables sensor and 
phase fault monitoring.  This should happen during the chassis power on 
after the regulators have received power (been enabled).

* phosphor-regulators-monitor-disable.service:  This disables sensor and 
phase fault monitoring.  This should happen early when the chassis is 
being powered off.

Sounds like maybe the last 3 service files are being run?  You can tell 
for sure by looking in the journal (e.g. 'journalctl | grep -i 
regulator').  The Wants/Before/After dependencies in the files determine 
when they are run.  Perhaps the systemd targets they are dependent on 
aren't running on your system?

You can manually cause the regulators application to perform 
configuration and sensor monitoring using the 'regsctl' program. Look at 
the service files to see the proper regsctl command to use.  This is an 
implementation detail and could change in the future, but it may help 
with debugging right now.

Regarding the fact the 0xDD is being read, is it possibly being read by 
another application like hwmon?

Note that the phosphor-regulators application does direct I2C reads and 
writes using the i2c-dev subsystem.  This is the same code path as 
i2cget/i2cset.  This means it should not be used in conjunction with a 
device driver on the regulator.  Otherwise there may be interleaved I2C 
commands going to the device, which can be problematic.

>
> Given I am using an Aspeed EVK, is there an example for how to turn on 
> a chassis with a GPIO, or a dbus operation, or an automatic turn on at 
> boot?
>
Sorry, I'm not very familiar with that.  Others on this list could help 
more with that as a separate question thread.  I use the 'obmcutil 
chassison' command.  Since that is a script, you could check out what it 
is doing and see if that would help.

>
>> The phosphor-regulators application creates those associations 
>> automatically based on information in your JSON file.  The "fru" 
>> property of the regulator "device" provides the first inventory 
>> object path.  The "inventory_path" property of the "chassis" provides 
>> the second inventory path.  Both of those are relative to the 
>> "/xyz/openbmc_project/inventory" root path.
>>
>> Do the "fru" and "inventory_path" properties in your JSON file match 
>> the correct inventory object paths on your system?
>>
> I have a psu.json with fruConfigs, and this has
>
> “PsuDevices”: {
> “/xyz/openbmc-project/inventory/system/chassis/motherboard/powersupply0” 
> : “/sys/bus/i2c/devices/11-004f”,
> }
>
> Which is the same i2c address as used by phosphor-regulators.
>
> And a power-supply-monitor-0.conf to match.
>
It sounds like you are using a power supply application from the same 
repository.  That is no problem, but they are completely separate 
applications that do no share any data/functionality.  So I don't think 
the work you've done with the PSU app would impact the regulators app.

It may be a typo above, but a voltage regulator would not normally be at 
the same I2C address as a power supply.  The term 'power supply' in that 
repo means the device that converts AC/DC wall power to 12V DC to the 
system (e.g. the big brick).  The term 'voltage regulator' means the 
devices that step 12V DC from the power supply down to the levels needed 
by system components (like 3.3V, 1.1V, etc.).

[-- Attachment #2: Type: text/html, Size: 7032 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-02-01 17:37 ` Shawn McCarney
@ 2022-02-09 22:30   ` Mike Jones
  2022-02-11 15:32     ` Shawn McCarney
  0 siblings, 1 reply; 11+ messages in thread
From: Mike Jones @ 2022-02-09 22:30 UTC (permalink / raw)
  To: Shawn McCarney; +Cc: OpenBMC Maillist

[-- Attachment #1: Type: text/plain, Size: 4983 bytes --]

The journal shows an I2C error: Device or resource busy.

Does hwmon lock out phosphor-regulators if it uses the same address?

Sent from my iPad

> On Feb 1, 2022, at 10:38 AM, Shawn McCarney <shawnmm@linux.ibm.com> wrote:
> 
>  
>>>> When the target boots, using a I2C spy tool, the 0xDD command is being read periodically, suggesting that this service is processing the read command, and a query show the service is up.
>>> Sensor reading should begin when the chassis is powered on and stop when the chassis is powered off.  That is because some regulators don't have power at standby or report invalid sensor readings.
>>> 
>> 
>> I am using the SDK, so I have not worked on chassis power, so I don’t know if it is powered or not, but this gives me a good hint about the problem. Even so, the read command in config.json is getting polled. The other config item to modify the voltage did not occur.
> It sounds like at least part of the problem might be the systemd service files not running.  The regulators service files are located in https://github.com/openbmc/phosphor-power/tree/master/services.
> 
> * phosphor-regulators.service:  This one launches the regulators app.  This must be happening since you are a getting journal message about it loading the JSON config file.
> 
> * phosphor-regulators-config.service:  This is what causes the configuration entries to be executed in the JSON file.  This should happen early in the process of powering on the chassis before the regulators have received power (been enabled).
> 
> * phosphor-regulators-monitor-enable.service:  This enables sensor and phase fault monitoring.  This should happen during the chassis power on after the regulators have received power (been enabled).
> 
> * phosphor-regulators-monitor-disable.service:  This disables sensor and phase fault monitoring.  This should happen early when the chassis is being powered off.
> 
> Sounds like maybe the last 3 service files are being run?  You can tell for sure by looking in the journal (e.g. 'journalctl | grep -i regulator').  The Wants/Before/After dependencies in the files determine when they are run.  Perhaps the systemd targets they are dependent on aren't running on your system?
> 
> You can manually cause the regulators application to perform configuration and sensor monitoring using the 'regsctl' program.  Look at the service files to see the proper regsctl command to use.  This is an implementation detail and could change in the future, but it may help with debugging right now.
> 
> Regarding the fact the 0xDD is being read, is it possibly being read by another application like hwmon?
> 
> Note that the phosphor-regulators application does direct I2C reads and writes using the i2c-dev subsystem.  This is the same code path as i2cget/i2cset.  This means it should not be used in conjunction with a device driver on the regulator.  Otherwise there may be interleaved I2C commands going to the device, which can be problematic.
> 
>> 
>> Given I am using an Aspeed EVK, is there an example for how to turn on a chassis with a GPIO, or a dbus operation, or an automatic turn on at boot?
>> 
> Sorry, I'm not very familiar with that.  Others on this list could help more with that as a separate question thread.  I use the 'obmcutil chassison' command.  Since that is a script, you could check out what it is doing and see if that would help.
> 
>> 
>>> The phosphor-regulators application creates those associations automatically based on information in your JSON file.  The "fru" property of the regulator "device" provides the first inventory object path.  The "inventory_path" property of the "chassis" provides the second inventory path.  Both of those are relative to the "/xyz/openbmc_project/inventory" root path.
>>> 
>>> Do the "fru" and "inventory_path" properties in your JSON file match the correct inventory object paths on your system?
>>> 
>> I have a psu.json with fruConfigs, and this has
>> 
>> “PsuDevices”: {
>>   “/xyz/openbmc-project/inventory/system/chassis/motherboard/powersupply0” : “/sys/bus/i2c/devices/11-004f”,
>> }
>> 
>> Which is the same i2c address as used by phosphor-regulators.
>> 
>> And a power-supply-monitor-0.conf to match.
>> 
> It sounds like you are using a power supply application from the same repository.  That is no problem, but they are completely separate applications that do no share any data/functionality.  So I don't think the work you've done with the PSU app would impact the regulators app.
> 
> It may be a typo above, but a voltage regulator would not normally be at the same I2C address as a power supply.  The term 'power supply' in that repo means the device that converts AC/DC wall power to 12V DC to the system (e.g. the big brick).  The term 'voltage regulator' means the devices that step 12V DC from the power supply down to the levels needed by system components (like 3.3V, 1.1V, etc.).

[-- Attachment #2: Type: text/html, Size: 7739 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-02-09 22:30   ` Mike Jones
@ 2022-02-11 15:32     ` Shawn McCarney
  2022-02-11 15:42       ` Mike Jones
  2022-02-11 20:53       ` Patrick Williams
  0 siblings, 2 replies; 11+ messages in thread
From: Shawn McCarney @ 2022-02-11 15:32 UTC (permalink / raw)
  To: Mike Jones; +Cc: openbmc

[-- Attachment #1: Type: text/plain, Size: 1269 bytes --]

On 2/9/2022 4:30 PM, Mike Jones wrote:
> The journal shows an I2C error: Device or resource busy. Does hwmon 
> lock out phosphor-regulators if it uses the same address? Sent from my 
> iPad On Feb 1, 2022, at 10:38 AM, Shawn McCarney 
> <shawnmm@linux.ibm.com> wrote:  ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ 
> 
> The journal shows an I2C error: Device or resource busy.
>
> Does hwmon lock out phosphor-regulators if it uses the same address?
>
phosphor-regulators currently communicates directly with voltage 
regulators using i2c-dev; it does not use device drivers.

The reason is that most regulator drivers provide either no or very 
limited ability to configure regulators.  For the systems I work on, we 
need to do extensive regulator configuration to fine tune things (output 
voltage, pgood min/max, timing, etc.) Redundant phase fault detection is 
also pretty device-specific and on involves communicating with multiple 
devices.

I have not tested using phosphor-regulators when a driver is bound to 
the regulator, but I suspect you may be right.  I assume hwmon is using 
read sensor data from a driver, and the driver being bound is stopping 
the phosphor-regulators read from succeeding.

Thanks,

Shawn

[-- Attachment #2: Type: text/html, Size: 1530 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-02-11 15:32     ` Shawn McCarney
@ 2022-02-11 15:42       ` Mike Jones
  2022-02-11 16:06         ` Shawn McCarney
  2022-02-11 20:53       ` Patrick Williams
  1 sibling, 1 reply; 11+ messages in thread
From: Mike Jones @ 2022-02-11 15:42 UTC (permalink / raw)
  To: Shawn McCarney; +Cc: OpenBMC Maillist

[-- Attachment #1: Type: text/plain, Size: 1719 bytes --]

Shawn,

I was mainly surprised because a conversation I had with Guenter, if I remember correctly, suggested that /dev/i2c calls from user space work with hwmon, because hwmon does not lock the i2c except when using it.

So I assumed that in this case, it was the polling of hwmon that was just keeping it locked enough to conflict with phosphor-regulators and then it gives up.

I could just not use hwmon at all use phosphor-regulators for all telemetry, but this seemed like more work.

Also, I will need to figure out how to connect phosphor-regulators telemetry to Redfish and the WebUI. Are there examples of how to do that?

Mike

> On Feb 11, 2022, at 8:32 AM, Shawn McCarney <shawnmm@linux.ibm.com> wrote:
> 
> On 2/9/2022 4:30 PM, Mike Jones wrote:
>> The journal shows an I2C error: Device or resource busy.
>> 
>> Does hwmon lock out phosphor-regulators if it uses the same address?
>> 
> phosphor-regulators currently communicates directly with voltage regulators using i2c-dev; it does not use device drivers.
> 
> The reason is that most regulator drivers provide either no or very limited ability to configure regulators.  For the systems I work on, we need to do extensive regulator configuration to fine tune things (output voltage, pgood min/max, timing, etc.)  Redundant phase fault detection is also pretty device-specific and on involves communicating with multiple devices.
> 
> I have not tested using phosphor-regulators when a driver is bound to the regulator, but I suspect you may be right.  I assume hwmon is using read sensor data from a driver, and the driver being bound is stopping the phosphor-regulators read from succeeding.
> 
> Thanks,
> 
> Shawn
> 


[-- Attachment #2: Type: text/html, Size: 3068 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-02-11 15:42       ` Mike Jones
@ 2022-02-11 16:06         ` Shawn McCarney
  2022-02-11 20:20           ` Ed Tanous
  0 siblings, 1 reply; 11+ messages in thread
From: Shawn McCarney @ 2022-02-11 16:06 UTC (permalink / raw)
  To: Mike Jones; +Cc: openbmc

[-- Attachment #1: Type: text/plain, Size: 2914 bytes --]

On 2/11/2022 9:42 AM, Mike Jones wrote:
> Shawn, I was mainly surprised because a conversation I had with 
> Guenter, if I remember correctly, suggested that /dev/i2c calls from 
> user space work with hwmon, because hwmon does not lock the i2c except 
> when using it. So I assumed that in this 
> I was mainly surprised because a conversation I had with Guenter, if I 
> remember correctly, suggested that /dev/i2c calls from user space work 
> with hwmon, because hwmon does not lock the i2c except when using it.
>
> So I assumed that in this case, it was the polling of hwmon that was 
> just keeping it locked enough to conflict with phosphor-regulators and 
> then it gives up.

With i2c-dev, you can specify I2C_SLAVE or I2C_SLAVE_FORCE when creating 
the connection.  I believe I2C_SLAVE will cause communication attempts 
to fail if a driver is bound. I2C_SLAVE_FORCE will still try to communicate.

I wanted to take the more conservative approach, because there are 
scenarios where interleaving communication can cause serious problems.  
For example, if one PMBus regulator supports multiple rails, you have to 
set the PMBus PAGE register to the rail you are interested in.  
Subsequent PMBus commands will affect that rail. If phosphor-regulators 
and a driver are both communicating with the regulator, they may both be 
setting PAGE and thus their subsequent commands are going to the wrong rail.

>
> I could just not use hwmon at all use phosphor-regulators for all 
> telemetry, but this seemed like more work.
>
Sorry about that.  I think if you have already done the work of defining 
the regulator and a read sensors rule in your JSON file, adding a few 
more sensors is not much additional work though.  Let me know if you 
have questions.

> Also, I will need to figure out how to connect phosphor-regulators 
> telemetry to Redfish and the WebUI. Are there examples of how to do that?
It should just work with no additional effort as long as you define the 
following correctly in your JSON file:

* The "inventory_path" property of your "chassis" object needs to match 
a real chassis inventory item on D-Bus.

* The "fru" property of your regulator "device" object needs to match a 
real inventory item on D-Bus.  As I mentioned earlier, normally if the 
regulator is pluggable then this is the regulator itself.  Otherwise it 
is the larger hardware item that contains it, such as a motherboard.  
FRU is the acronym for Field Replaceable Unit, referring to a pluggable 
hardware item.

phosphor-regulators will publish your sensors on D-Bus with the standard 
object paths and properties.  The two JSON properties above are used to 
define a "chassis" association and an "inventory_item" association on 
D-Bus that is used by bmcweb. This will cause your sensors to appear in 
Redfish and the Web UI.

Thanks,

Shawn

[-- Attachment #2: Type: text/html, Size: 3760 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-02-11 16:06         ` Shawn McCarney
@ 2022-02-11 20:20           ` Ed Tanous
  2022-02-11 22:56             ` Mike Jones
  0 siblings, 1 reply; 11+ messages in thread
From: Ed Tanous @ 2022-02-11 20:20 UTC (permalink / raw)
  To: Shawn McCarney, Justin Ledford, Gunnar Mills, Andrew Jeffery,
	Adriana Kobylak, Peter Lundgren, Jason Ling
  Cc: openbmc, Mike Jones

On Fri, Feb 11, 2022 at 8:06 AM Shawn McCarney <shawnmm@linux.ibm.com> wrote:
>
> On 2/11/2022 9:42 AM, Mike Jones wrote:
>
> I was mainly surprised because a conversation I had with Guenter, if I remember correctly, suggested that /dev/i2c calls from user space work with hwmon, because hwmon does not lock the i2c except when using it.
>
> So I assumed that in this case, it was the polling of hwmon that was just keeping it locked enough to conflict with phosphor-regulators and then it gives up.
>
> With i2c-dev, you can specify I2C_SLAVE or I2C_SLAVE_FORCE when creating the connection.  I believe I2C_SLAVE will cause communication attempts to fail if a driver is bound.  I2C_SLAVE_FORCE will still try to communicate.
>
> I wanted to take the more conservative approach, because there are scenarios where interleaving communication can cause serious problems.  For example, if one PMBus regulator supports multiple rails, you have to set the PMBus PAGE register to the rail you are interested in.  Subsequent PMBus commands will affect that rail.  If phosphor-regulators and a driver are both communicating with the regulator, they may both be setting PAGE and thus their subsequent commands are going to the wrong rail.
>
>
> I could just not use hwmon at all use phosphor-regulators for all telemetry, but this seemed like more work.
>
> Sorry about that.  I think if you have already done the work of defining the regulator and a read sensors rule in your JSON file, adding a few more sensors is not much additional work though.  Let me know if you have questions.
>
> Also, I will need to figure out how to connect phosphor-regulators telemetry to Redfish and the WebUI. Are there examples of how to do that?
>
> It should just work with no additional effort as long as you define the following correctly in your JSON file:
>
> * The "inventory_path" property of your "chassis" object needs to match a real chassis inventory item on D-Bus.
>
> * The "fru" property of your regulator "device" object needs to match a real inventory item on D-Bus.  As I mentioned earlier, normally if the regulator is pluggable then this is the regulator itself.  Otherwise it is the larger hardware item that contains it, such as a motherboard.  FRU is the acronym for Field Replaceable Unit, referring to a pluggable hardware item.
>
> phosphor-regulators will publish your sensors on D-Bus with the standard object paths and properties.  The two JSON properties above are used to define a "chassis" association and an "inventory_item" association on D-Bus that is used by bmcweb.  This will cause your sensors to appear in Redfish and the Web UI.
>
> Thanks,
>
> Shawn

This same topic is essentially being discussed in a design doc here:
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/50509

I've CCed the relevant parties from that review on this thread, and I
don't have a strong opinion what forum we use to discuss this, but we
should ideally all be talking together on a common solution.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-02-11 15:32     ` Shawn McCarney
  2022-02-11 15:42       ` Mike Jones
@ 2022-02-11 20:53       ` Patrick Williams
  1 sibling, 0 replies; 11+ messages in thread
From: Patrick Williams @ 2022-02-11 20:53 UTC (permalink / raw)
  To: Shawn McCarney; +Cc: openbmc, Mike Jones

[-- Attachment #1: Type: text/plain, Size: 1093 bytes --]

On Fri, Feb 11, 2022 at 09:32:13AM -0600, Shawn McCarney wrote:
> On 2/9/2022 4:30 PM, Mike Jones wrote:
> > iPad On Feb 1, 2022, at 10:38 AM, Shawn McCarney 
> > <shawnmm@linux.ibm.com> wrote:  ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ 
> phosphor-regulators currently communicates directly with voltage 
> regulators using i2c-dev; it does not use device drivers.
> 
> The reason is that most regulator drivers provide either no or very 
> limited ability to configure regulators.  For the systems I work on, we 
> need to do extensive regulator configuration to fine tune things (output 
> voltage, pgood min/max, timing, etc.) Redundant phase fault detection is 
> also pretty device-specific and on involves communicating with multiple 
> devices.

Out of curiosity, had you done any investigation with getting the support you
need upstreamed in drivers?  If you had upstream discussions I'd like some
pointers to them (lore links) since, as Ed pointed out, we're having design
discussions on this same problem generally speaking.

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-02-11 20:20           ` Ed Tanous
@ 2022-02-11 22:56             ` Mike Jones
  0 siblings, 0 replies; 11+ messages in thread
From: Mike Jones @ 2022-02-11 22:56 UTC (permalink / raw)
  To: Ed Tanous
  Cc: Peter Lundgren, Andrew Jeffery, openbmc, Shawn McCarney,
	Justin Ledford, Gunnar Mills, Jason Ling



Sent from my iPad

> On Feb 11, 2022, at 1:20 PM, Ed Tanous <edtanous@google.com> wrote:
> 
> On Fri, Feb 11, 2022 at 8:06 AM Shawn McCarney <shawnmm@linux.ibm.com> wrote:
>> 
>> On 2/11/2022 9:42 AM, Mike Jones wrote:
>> 
>> I was mainly surprised because a conversation I had with Guenter, if I remember correctly, suggested that /dev/i2c calls from user space work with hwmon, because hwmon does not lock the i2c except when using it.
>> 
>> So I assumed that in this case, it was the polling of hwmon that was just keeping it locked enough to conflict with phosphor-regulators and then it gives up.
>> 
>> With i2c-dev, you can specify I2C_SLAVE or I2C_SLAVE_FORCE when creating the connection.  I believe I2C_SLAVE will cause communication attempts to fail if a driver is bound.  I2C_SLAVE_FORCE will still try to communicate.
>> 
>> I wanted to take the more conservative approach, because there are scenarios where interleaving communication can cause serious problems.  For example, if one PMBus regulator supports multiple rails, you have to set the PMBus PAGE register to the rail you are interested in.  Subsequent PMBus commands will affect that rail.  If phosphor-regulators and a driver are both communicating with the regulator, they may both be setting PAGE and thus their subsequent commands are going to the wrong rail.
>> 
>> 
>> I could just not use hwmon at all use phosphor-regulators for all telemetry, but this seemed like more work.
>> 
>> Sorry about that.  I think if you have already done the work of defining the regulator and a read sensors rule in your JSON file, adding a few more sensors is not much additional work though.  Let me know if you have questions.
>> 
>> Also, I will need to figure out how to connect phosphor-regulators telemetry to Redfish and the WebUI. Are there examples of how to do that?
>> 
>> It should just work with no additional effort as long as you define the following correctly in your JSON file:
>> 
>> * The "inventory_path" property of your "chassis" object needs to match a real chassis inventory item on D-Bus.
>> 
>> * The "fru" property of your regulator "device" object needs to match a real inventory item on D-Bus.  As I mentioned earlier, normally if the regulator is pluggable then this is the regulator itself.  Otherwise it is the larger hardware item that contains it, such as a motherboard.  FRU is the acronym for Field Replaceable Unit, referring to a pluggable hardware item.
>> 
>> phosphor-regulators will publish your sensors on D-Bus with the standard object paths and properties.  The two JSON properties above are used to define a "chassis" association and an "inventory_item" association on D-Bus that is used by bmcweb.  This will cause your sensors to appear in Redfish and the Web UI.
>> 
>> Thanks,
>> 
>> Shawn
> 
> This same topic is essentially being discussed in a design doc here:
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/50509

I will move my discussion to Gerrit.

Note that I work for ADI, my team supports ADM1266, ADM1278, LTC297X, LTC388X, the MAX devices in this category, and I am on the SMIF (PMBus) Board and on the PMBus Technical Committee.

> 
> I've CCed the relevant parties from that review on this thread, and I
> don't have a strong opinion what forum we use to discuss this, but we
> should ideally all be talking together on a common solution.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: No dbus objects for phosphor-regulators
  2022-01-25 22:15 Mike Jones
@ 2022-01-26 22:43 ` Shawn McCarney
  0 siblings, 0 replies; 11+ messages in thread
From: Shawn McCarney @ 2022-01-26 22:43 UTC (permalink / raw)
  To: Mike Jones; +Cc: OpenBMC Maillist

[-- Attachment #1: Type: text/plain, Size: 3679 bytes --]

Hi Mike,

> I made a config file below, and verified that it is on the target and 
> loads without error.

Just to make sure I understand what you are saying, on the BMC your 
'config.json' file is installed in the /usr/share/phosphor-regulators 
directory?

And when you run 'journalctl -u phosphor-regulators.service', you see a 
journal message stating that the phosphor-regulators application has 
loaded your JSON file successfully?  And no error messages?

> When the target boots, using a I2C spy tool, the 0xDD command is being 
> read periodically, suggesting that this service is processing the read 
> command, and a query show the service is up.
>
Sensor reading should begin when the chassis is powered on and stop when 
the chassis is powered off.  That is because some regulators don't have 
power at standby or report invalid sensor readings.

Does the 'journalctl | grep -i Regulators' show that the service files 
are being executed that enable and disable regulator sensor monitoring?

> Using dbus-monitor –system, I do not see any transactions that place 
> these sensor read values on dbus, nor does the normal probing the 
> chassis and inventory show values.
>
Since you are reading a vout sensor for the rail named "vdd", the 
resulting D-Bus object path for the sensor should be

   /xyz/openbmc_project/sensors/voltage/vdd_vout

See 
https://github.com/openbmc/phosphor-power/blob/master/phosphor-regulators/docs/config_file/pmbus_read_sensor.md#d-bus-sensor 
for more details.

Does that object path exist on D-Bus after you've powered on the chassis 
and sensor monitoring has been enabled?

You can see all the properties of the D-Bus sensor using the command

   busctl introspect xyz.openbmc_project.Power.Regulators 
/xyz/openbmc_project/sensors/voltage/vdd_vout

You can see all regulator sensor object paths via the command:

   busctl tree xyz.openbmc_project.Power.Regulators

The phosphor-regulators application reads sensors once per second.  
However, it only updates D-Bus if the new sensor value has changed more 
than a hysteresis amount.  This is to avoid generating a bunch of D-Bus 
traffic for minor value changes.

You can see the hysteresis values here: 
https://github.com/openbmc/phosphor-power/blob/master/phosphor-regulators/src/dbus_sensor.cpp#L66

> What else do I need to so that there are dbus objects?

For bmcweb and Redfish to work, you need associations between the D-Bus 
sensor and two D-Bus inventory objects:

* The FRU that contains the regulator.  This is the replaceable hardware 
item where the regulator is located.  Some regulators are FRUs by 
themselves (replaceable).  Some are located on larger FRUs like a 
motherboard.  This is used to get presence, health info, and if 
applicable VPD.

* The chassis that contains the regulator.  This is required by Redfish 
since it groups sensors by chassis.

The phosphor-regulators application creates those associations 
automatically based on information in your JSON file.  The "fru" 
property of the regulator "device" provides the first inventory object 
path.  The "inventory_path" property of the "chassis" provides the 
second inventory path.  Both of those are relative to the 
"/xyz/openbmc_project/inventory" root path.

Do the "fru" and "inventory_path" properties in your JSON file match the 
correct inventory object paths on your system?

> Note I have not done anything to intentionally use open_power. My goal 
> is to only use phosphor-regulators to expose some telemetry that hwmon 
> does not support.
>
That's fine.  This application does not have any dependencies on open_power.

Let me know if you need more info.

Shawn

[-- Attachment #2: Type: text/html, Size: 5858 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* No dbus objects for phosphor-regulators
@ 2022-01-25 22:15 Mike Jones
  2022-01-26 22:43 ` Shawn McCarney
  0 siblings, 1 reply; 11+ messages in thread
From: Mike Jones @ 2022-01-25 22:15 UTC (permalink / raw)
  To: OpenBMC Maillist

[-- Attachment #1: Type: text/plain, Size: 2905 bytes --]

Hi,
 
I need some help getting phosphor-power/phosphor-regulators to add dbus objects.
 
I made a config file below, and verified that it is on the target and loads without error.
 
When the target boots, using a I2C spy tool, the 0xDD command is being read periodically, suggesting that this service is processing the read command, and a query show the service is up.
 
Using dbus-monitor –system, I do not see any transactions that place these sensor read values on dbus, nor does the normal probing the chassis and inventory show values.
 
My bbappend is shown below, it is simple, but I wonder what is missing.
 
What else do I need to so that there are dbus objects?
 
Note I have not done anything to intentionally use open_power. My goal is to only use phosphor-regulators to expose some telemetry that hwmon does not support.
 
Mike
 
*************** phosphor-power_%.bbappend ************
 
FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"
 
inherit obmc-phosphor-systemd
 
SRC_URI += file://config.json
do_install:append(){
    install -D ${WORKDIR}/config.json ${D}${datadir}/phosphor-regulators/config.json
}
FILES:${PN} += "${datadir}/phosphor-regulators/config.json"
 
************* config file ****************
 
{
  "comments": [ "Config file for a ADI one-chassis system" ],
 
  "rules": [
    {
      "comments": [ "Sets output voltage for a PMBus regulator rail" ],
      "id": "set_voltage_rule",
      "actions": [
        {
          "pmbus_write_vout_command": {
            "format": "linear"
          }
        }
      ]
    },
 
    {
      "comments": [ "Reads sensors from a PMBus regulator rail" ],
      "id": "read_sensors_rule",
      "actions": [
        {
          "comments": [ "Read output voltage from READ_VOUT." ],
          "pmbus_read_sensor": {
            "type": "vout",
            "command": "0xDD",
            "format": "linear_16"
          }
        }
      ]
    }
  ],
 
  "chassis": [
    {
      "comments": [ "Chassis number 1 containing CPUs and memory" ],
      "number": 1,
      "inventory_path": "system/chassis",
      "devices": [
        {
          "comments": [ "LTC3888 regulator producing the Vdd rail" ],
          "id": "vdd_regulator",
          "is_regulator": true,
          "fru": "system/chassis/motherboard/regulator1",
          "i2c_interface": {
            "bus": 11,
            "address": "0x4f"
          },
          "rails": [
            {
              "comments": [ "Vdd rail" ],
              "id": "vdd",
              "configuration": {
                "volts": 1.10,
                "rule_id": "set_voltage_rule"
              },
              "sensor_monitoring": {
                "rule_id": "read_sensors_rule"
              }
            }
          ]
        }
      ]
    }
  ]
}
 
Mike

Sent from my iPad

[-- Attachment #2: Type: text/html, Size: 19173 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2022-02-11 22:57 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-31 22:25 No dbus objects for phosphor-regulators Mike
2022-02-01 17:37 ` Shawn McCarney
2022-02-09 22:30   ` Mike Jones
2022-02-11 15:32     ` Shawn McCarney
2022-02-11 15:42       ` Mike Jones
2022-02-11 16:06         ` Shawn McCarney
2022-02-11 20:20           ` Ed Tanous
2022-02-11 22:56             ` Mike Jones
2022-02-11 20:53       ` Patrick Williams
  -- strict thread matches above, loose matches on Subject: below --
2022-01-25 22:15 Mike Jones
2022-01-26 22:43 ` Shawn McCarney

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).