Redfish Software Updates in OpenBMC

Redfish Software Updates in OpenBMC

[Andrew Geissler]

Greetings,

Recently I've started to dig into what it's going to take to fully support
our OpenBMC implementation of code update within Redfish.
Specifically, how to support our REST based upload of the .tar images for
both our BMC and BIOS.

The current flow is the following for both BMC and BIOS:

# Upload image and get hash back identifying image
hash = POST -T <file_to_upload> https://${bmc}/upload/image

# Activate the image (i.e. write it to the flash chip)
PUT -d '{"data":
  "xyz.openbmc_project.Software.Activation.RequestedActivations.Active"}' \
  https://${bmc}/xyz/openbmc_project/software/<hash>/attr/RequestedActivation

# wait for update to complete
GET https://${bmc}/xyz/openbmc_project/software/<hash>/attr/Progress

# Verify "Activation" property is set to "Active"
GET https://${bmc}/xyz/openbmc_project/software/<hash>

# If a BMC update, reboot the BMC

There are then a set of REST api's we support to query the firmware images
loaded, which one's are active, and what the priority of each. We have a
priority concept for systems that support redundant flash chips which allows
a user to quickly switch between the activated firmware image for BMC or BIOS.


Redfish defines a UpdateService schema[a]. An action available on the
UpdateService is a SimpleUpdate interface. I think the flow to use this would
look like this:

# Retrieve the URI to upload an image to
HttpPushUri = GET  /redfish/v1/UpdateService

# Upload firmware image
ImageURI = POST HttpPushUri

# Call SimpleUpdate and point it to the "ImageURI" (returned from POST) Also
# point it to the redfish "Targets" to update(i.e. /redfish/v1/Managers/openbmc)
task = POST redfish/v1/UpdateService/Actions/UpdateService.SimpleUpdate

# Call TaskService (?) to see when operations is completed
GET redfish/v1/TaskService/Tasks/<task>

The main issue with the above is the loss of our ability to have multiple
firmware images of the same type in a system and to be able to change the
priority of them to switch back and forth between them. It's also not obvious
to me what the target is for the BIOS when doing an update of it? There are OEM
options for both Actions and Properties if needed (but would be great to avoid).

I looked around a bit at other implementations of the UpdateService.
SuperMicro[b] and Dell[c] created their own OEM commands for the update
process (although Dell seems to also support the SimpleUpdate as well[d])

If any of you are reading, I'd appreciate some thoughts on why you went the OEM
route. Was it for more fine grained control? Or maybe for issues like I
highlight above with multiple image support?

We already have a good start on our UpdateService support in bmcweb:
https://github.com/openbmc/bmcweb/blob/master/redfish-core/lib/update_service.hpp

So before I dig much deeper, I'd definitely like to get some feedback from
the community on what y'all are thinking with software update and redfish!


[a] https://redfish.dmtf.org/schemas/UpdateService.v1_3_0.json
[b] https://www.supermicro.com/manuals/other/RedfishRefGuide.pdf
[c] https://topics-cdn.dell.com/pdf/idrac9-lifecycle-controller-v3212121_api-guide_en-us.pdf
[d] https://github.com/dell/iDRAC-Redfish-Scripting/blob/master/Redfish%20Python/DeviceFirmwareSimpleUpdateREDFISH.py

SuperMicro OEM Examples of Software Update:
# BMC
POST https://<IP>/redfish/v1/UpdateService/FirmwareInventory/BMC/Actions/Oem/FirmwareInventory.EnterBMCUpdateMode
POST https://<IP>/redfish/v1/UpdateService/FirmwareInventory/BMC/Actions/Oem/FirmwareInventory.UploadBMC
(File)
POST https://<IP>/redfish/v1/UpdateService/FirmwareInventory/BMC/Actions/Oem/FirmwareInventory.UpdateBMC
(preserve flags)
GET https://<IP>redfish/v1/UpdateService/FirmwareInventory/BMC/

# BIOS
POST https://<IP>/redfish/v1/UpdateService/FirmwareInventory/BIOS/Actions/Oem/FirmwareInventory.EnterBIOSUpdateMode
POST https://<IP>/redfish/v1/UpdateService/FirmwareInventory/BIOS/Actions/Oem/FirmwareInventory.UploadBIOS
(File)
POST https://<IP>/redfish/v1/UpdateService/FirmwareInventory/BIOS/Actions/Oem/FirmwareInventory.UpdateBIOS
(preserve flags)
GET https://<IP>redfish/v1/UpdateService/FirmwareInventory/BIOS/

Thanks,
Andrew

Re: Redfish Software Updates in OpenBMC

[Ed Tanous]

On 1/10/19 2:43 PM, Andrew Geissler wrote
> Redfish defines a UpdateService schema[a]. An action available on the
> UpdateService is a SimpleUpdate interface. I think the flow to use this would
> look like this:
>
> # Retrieve the URI to upload an image to
> HttpPushUri = GET  /redfish/v1/UpdateService
>
> # Upload firmware image
> ImageURI = POST HttpPushUri

As redfish is written, this is where the flow stops.  The BMC takes over
from here, and updates itself.  One enhancement I'd really like to see
made is to return a reference to a TaskService task, so we can track the
progress as the update is applied.

My understanding is that SimpleUpdate is a flow where the BMC can pull
down an image itself, so the URI is generally not something local to the
BMC, but a remote server that the BMC can connect to.

> The main issue with the above is the loss of our ability to have multiple
> firmware images of the same type in a system and to be able to change the
> priority of them to switch back and forth between them. It's also not obvious
> to me what the target is for the BIOS when doing an update of it? There are OEM
> options for both Actions and Properties if needed (but would be great to avoid).

The piece I think you're missing is the new property that was added in
redfish UpdateService 1.1.1 named HttpPushUriTargets.  It essentially
allows you to define multiple targets for the push flow you outlined
above, although I don't see anyone using it yet.

https://github.com/openbmc/bmcweb/blob/c0eb9bd95afa900dc808dce7c2c5fa521ba828f5/static/redfish/v1/schema/UpdateService_v1.xml#L224

If you want to implement SimpleUpdate, it also supports the Targets
property, which lets you direct where you would like a given payload to
be updated to:

https://github.com/openbmc/bmcweb/blob/c0eb9bd95afa900dc808dce7c2c5fa521ba828f5/static/redfish/v1/schema/UpdateService_v1.xml#L74

One design conflict I'm seeing is that Redfish wasn't really meant to
separate out the upload and activate interfaces.  Personally, I like the
Redfish approach better, as it simplifies things for the user.  Loading
a target with firmware is a single step (upload), instead of 2 (upload
then activate).  Most of the systems I know of use the single step process.

> I looked around a bit at other implementations of the UpdateService.
> SuperMicro[b] and Dell[c] created their own OEM commands for the update
> process (although Dell seems to also support the SimpleUpdate as well[d])
>
> If any of you are reading, I'd appreciate some thoughts on why you went the OEM
> route. Was it for more fine grained control? Or maybe for issues like I
> highlight above with multiple image support?

I can't speak for those companies, but I know that initially there was
no way to do component level updates using the stock schemas.  To
support their customers, a lot of companies ended up building their own
OEM upload mechanisms.  I've seen a lot of cases where Redfish schemas
initially started as an OEM definition, then were standardized later, as
people saw the utility of them.

Re: Redfish Software Updates in OpenBMC

[Andrew Geissler]

On Mon, Jan 14, 2019 at 11:04 AM Ed Tanous <ed.tanous@intel.com> wrote:

> On 1/10/19 2:43 PM, Andrew Geissler wrote
> > Redfish defines a UpdateService schema[a]. An action available on the
> > UpdateService is a SimpleUpdate interface. I think the flow to use this would
> > look like this:
> >
> > # Retrieve the URI to upload an image to
> > HttpPushUri = GET  /redfish/v1/UpdateService
> >
> > # Upload firmware image
> > ImageURI = POST HttpPushUri
>
> As redfish is written, this is where the flow stops.  The BMC takes over
> from here, and updates itself.  One enhancement I'd really like to see
> made is to return a reference to a TaskService task, so we can track the
> progress as the update is applied.
>
> My understanding is that SimpleUpdate is a flow where the BMC can pull
> down an image itself, so the URI is generally not something local to the
> BMC, but a remote server that the BMC can connect to.

Looked like the Dell implementation was using it as a local pointer but I agree,
the specification seems to only support the remote server concept. This is
a feature we'll want (like for TFTP and such).

>
> > The main issue with the above is the loss of our ability to have multiple
> > firmware images of the same type in a system and to be able to change the
> > priority of them to switch back and forth between them. It's also not obvious
> > to me what the target is for the BIOS when doing an update of it? There are OEM
> > options for both Actions and Properties if needed (but would be great to avoid).
>
> The piece I think you're missing is the new property that was added in
> redfish UpdateService 1.1.1 named HttpPushUriTargets.  It essentially
> allows you to define multiple targets for the push flow you outlined
> above, although I don't see anyone using it yet.
>
> https://github.com/openbmc/bmcweb/blob/c0eb9bd95afa900dc808dce7c2c5fa521ba828f5/static/redfish/v1/schema/UpdateService_v1.xml#L224
>
> If you want to implement SimpleUpdate, it also supports the Targets
> property, which lets you direct where you would like a given payload to
> be updated to:
>
> https://github.com/openbmc/bmcweb/blob/c0eb9bd95afa900dc808dce7c2c5fa521ba828f5/static/redfish/v1/schema/UpdateService_v1.xml#L74
>
> One design conflict I'm seeing is that Redfish wasn't really meant to
> separate out the upload and activate interfaces.  Personally, I like the
> Redfish approach better, as it simplifies things for the user.  Loading
> a target with firmware is a single step (upload), instead of 2 (upload
> then activate).  Most of the systems I know of use the single step process.

Yeah, I'd like to stick with the default interface and I agree that the one
step process is easier. I would propose though that we add an OEM
"Priority" field so in instances where the system supports multiple
images for the same HttpPushUriTargets, the user would have an easy
way to switch back and forth. We use this a lot in development and
I know customers use it when they want to quickly roll back to a
previous update (when they were just testing out a new release or
run into issues with it). Some investigation will be required by me to
determine if the existing API's would support multiple firmware instances
for the same Target.

Re: Redfish Software Updates in OpenBMC

[Andrew Geissler]

On Tue, Jan 22, 2019 at 9:26 AM Andrew Geissler <geissonator@gmail.com> wrote:
>
> On Mon, Jan 14, 2019 at 11:04 AM Ed Tanous <ed.tanous@intel.com> wrote:
>
> > On 1/10/19 2:43 PM, Andrew Geissler wrote
> > > Redfish defines a UpdateService schema[a]. An action available on the
> > > UpdateService is a SimpleUpdate interface. I think the flow to use this would
> > > look like this:
> > >
> > > # Retrieve the URI to upload an image to
> > > HttpPushUri = GET  /redfish/v1/UpdateService
> > >
> > > # Upload firmware image
> > > ImageURI = POST HttpPushUri
> >
> > As redfish is written, this is where the flow stops.  The BMC takes over
> > from here, and updates itself.  One enhancement I'd really like to see
> > made is to return a reference to a TaskService task, so we can track the
> > progress as the update is applied.
> >
> > My understanding is that SimpleUpdate is a flow where the BMC can pull
> > down an image itself, so the URI is generally not something local to the
> > BMC, but a remote server that the BMC can connect to.
>
> Looked like the Dell implementation was using it as a local pointer but I agree,
> the specification seems to only support the remote server concept. This is
> a feature we'll want (like for TFTP and such).
>
> >
> > > The main issue with the above is the loss of our ability to have multiple
> > > firmware images of the same type in a system and to be able to change the
> > > priority of them to switch back and forth between them. It's also not obvious
> > > to me what the target is for the BIOS when doing an update of it? There are OEM
> > > options for both Actions and Properties if needed (but would be great to avoid).
> >
> > The piece I think you're missing is the new property that was added in
> > redfish UpdateService 1.1.1 named HttpPushUriTargets.  It essentially
> > allows you to define multiple targets for the push flow you outlined
> > above, although I don't see anyone using it yet.
> >
> > https://github.com/openbmc/bmcweb/blob/c0eb9bd95afa900dc808dce7c2c5fa521ba828f5/static/redfish/v1/schema/UpdateService_v1.xml#L224
> >
> > If you want to implement SimpleUpdate, it also supports the Targets
> > property, which lets you direct where you would like a given payload to
> > be updated to:
> >
> > https://github.com/openbmc/bmcweb/blob/c0eb9bd95afa900dc808dce7c2c5fa521ba828f5/static/redfish/v1/schema/UpdateService_v1.xml#L74
> >
> > One design conflict I'm seeing is that Redfish wasn't really meant to
> > separate out the upload and activate interfaces.  Personally, I like the
> > Redfish approach better, as it simplifies things for the user.  Loading
> > a target with firmware is a single step (upload), instead of 2 (upload
> > then activate).  Most of the systems I know of use the single step process.
>
> Yeah, I'd like to stick with the default interface and I agree that the one
> step process is easier. I would propose though that we add an OEM
> "Priority" field so in instances where the system supports multiple
> images for the same HttpPushUriTargets, the user would have an easy
> way to switch back and forth. We use this a lot in development and
> I know customers use it when they want to quickly roll back to a
> previous update (when they were just testing out a new release or
> run into issues with it). Some investigation will be required by me to
> determine if the existing API's would support multiple firmware instances
> for the same Target.

I put a design doc up at
https://gerrit.openbmc-project.xyz/#/c/openbmc/docs/+/18186/
I hemmed and hawed a bit on whether it was needed but figured with all
our recent community discussions, it's better to over communicate then
under.

Andrew