Re: [PATCH v2 net-next 1/4] devlink: Add new "allow_fw_live_reset" generic device parameter.

[Vasundhara Volam <vasundhara-v.volam@broadcom.com>]

On Mon, Jun 1, 2020 at 3:20 PM Jiri Pirko <jiri@resnulli.us> wrote:
>
> Mon, Jun 01, 2020 at 10:53:19AM CEST, vasundhara-v.volam@broadcom.com wrote:
> >On Mon, Jun 1, 2020 at 12:48 PM Jiri Pirko <jiri@resnulli.us> wrote:
> >>
> >> Tue, May 26, 2020 at 04:23:48PM CEST, vasundhara-v.volam@broadcom.com wrote:
> >> >On Tue, May 26, 2020 at 7:10 PM Jiri Pirko <jiri@resnulli.us> wrote:
> >> >>
> >> >> Tue, May 26, 2020 at 08:42:28AM CEST, vasundhara-v.volam@broadcom.com wrote:
> >> >> >On Tue, May 26, 2020 at 10:17 AM Jiri Pirko <jiri@resnulli.us> wrote:
> >> >> >>
> >> >> >> Tue, May 26, 2020 at 06:28:59AM CEST, vasundhara-v.volam@broadcom.com wrote:
> >> >> >> >On Mon, May 25, 2020 at 10:56 PM Jiri Pirko <jiri@resnulli.us> wrote:
> >> >> >> >>
> >> >> >> >> Sun, May 24, 2020 at 08:29:56AM CEST, vasundhara-v.volam@broadcom.com wrote:
> >> >> >> >> >On Sun, May 24, 2020 at 10:23 AM Jiri Pirko <jiri@resnulli.us> wrote:
> >> >> >> >> >>
> >> >> >> >> >> Sat, May 23, 2020 at 08:08:22AM CEST, vasundhara-v.volam@broadcom.com wrote:
> >> >> >> >> >> >Add a new "allow_fw_live_reset" generic device bool parameter. When
> >> >> >> >> >> >parameter is set, user is allowed to reset the firmware in real time.
> >> >> >> >> >> >
> >> >> >> >> >> >This parameter is employed to communicate user consent or dissent for
> >> >> >> >> >> >the live reset to happen. A separate command triggers the actual live
> >> >> >> >> >> >reset.
> >> >> >> >> >> >
> >> >> >> >> >> >Cc: Jiri Pirko <jiri@mellanox.com>
> >> >> >> >> >> >Signed-off-by: Vasundhara Volam <vasundhara-v.volam@broadcom.com>
> >> >> >> >> >> >Signed-off-by: Michael Chan <michael.chan@broadcom.com>
> >> >> >> >> >> >---
> >> >> >> >> >> >v2: Rename param name to "allow_fw_live_reset" from
> >> >> >> >> >> >"enable_hot_fw_reset".
> >> >> >> >> >> >Update documentation for the param in devlink-params.rst file.
> >> >> >> >> >> >---
> >> >> >> >> >> > Documentation/networking/devlink/devlink-params.rst | 6 ++++++
> >> >> >> >> >> > include/net/devlink.h                               | 4 ++++
> >> >> >> >> >> > net/core/devlink.c                                  | 5 +++++
> >> >> >> >> >> > 3 files changed, 15 insertions(+)
> >> >> >> >> >> >
> >> >> >> >> >> >diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst
> >> >> >> >> >> >index d075fd0..ad54dfb 100644
> >> >> >> >> >> >--- a/Documentation/networking/devlink/devlink-params.rst
> >> >> >> >> >> >+++ b/Documentation/networking/devlink/devlink-params.rst
> >> >> >> >> >> >@@ -108,3 +108,9 @@ own name.
> >> >> >> >> >> >    * - ``region_snapshot_enable``
> >> >> >> >> >> >      - Boolean
> >> >> >> >> >> >      - Enable capture of ``devlink-region`` snapshots.
> >> >> >> >> >> >+   * - ``allow_fw_live_reset``
> >> >> >> >> >> >+     - Boolean
> >> >> >> >> >> >+     - Firmware live reset allows users to reset the firmware in real time.
> >> >> >> >> >> >+       For example, after firmware upgrade, this feature can immediately reset
> >> >> >> >> >> >+       to run the new firmware without reloading the driver or rebooting the
> >> >> >> >> >>
> >> >> >> >> >> This does not tell me anything about the reset being done on another
> >> >> >> >> >> host. You need to emhasize that, in the name of the param too.
> >> >> >> >> >I am not sure if I completely understand your query.
> >> >> >> >> >
> >> >> >> >> >Reset is actually initiated by one of the PF/host of the device, which
> >> >> >> >> >resets the entire same device.
> >> >> >> >> >
> >> >> >> >> >Reset is not initiated by any other remote device/host.
> >> >> >> >>
> >> >> >> >> Well, in case of multihost system, it might be, right?
> >> >> >> >>
> >> >> >> >In case of multi-host system also, it is one of the host that triggers
> >> >> >> >the reset, which resets the entire same device. I don't think this is
> >> >> >> >remote.
> >> >> >> >
> >> >> >> >As the parameter is a device parameter, it is applicable to the entire
> >> >> >> >device. When a user initiates the reset from any of the host in case
> >> >> >> >of multi-host and any of the PF in case of stand-alone or smartNIC
> >> >> >> >device, the entire device goes for a reset.
> >> >> >> >
> >> >> >> >I will be expanding the description to the following to make it more clear.
> >> >> >> >
> >> >> >> >------------------------
> >> >> >> >- Firmware live reset allows users to reset the firmware in real time.
> >> >> >> >For example, after firmware upgrade, this feature can immediately
> >> >> >> >reset to run the new firmware without reloading the driver or
> >> >> >> >rebooting the system.
> >> >> >> >When a user initiates the reset from any of the host (in case of
> >> >> >> >multi-host system) / PF (in case of stand-alone or smartNIC device),
> >> >> >> >the entire device goes for a reset when the parameter is enabled.
> >> >> >>
> >> >> >> Sorry, this is still not clear. I think that you are mixing up two
> >> >> >> different things:
> >> >> >> 1) option of devlink reload to indicate that user is interested in "live
> >> >> >>    reset" of firmware without reloading driver
> >> >> >
> >> >> >This is the option we are trying to add. If a user is interested in
> >> >> >"live reset", he needs to enable the parameter to enable it in device
> >> >> >capabilities, which is achieved by permanent configuration mode. When
> >> >> >capability is enabled in the device, new firmware which is aware will
> >> >> >allocate the resources and exposes the capability to host drivers.
> >> >> >
> >> >> >But firmware allows the "live reset" only when all the loaded drivers
> >> >> >are aware of/supports the capability. For example, if any of the host
> >> >> >is loaded with an old driver, "live reset" is not allowed until the
> >> >> >driver is upgraded or unloaded. or if the host driver turns it off,
> >> >> >then also "live reset" is not allowed.
> >> >> >
> >> >> >In case of runtime parameter cmode, if any of the host turns off the
> >> >> >capability in the host driver, "live reset" is not allowed until the
> >> >> >driver is unloaded or the user enables it again.
> >> >> >
> >> >> >To make it clear, I can add two parameters.
> >> >> >
> >> >> >1. enable_fw_live_reset - To indicate that the user is interested in
> >> >> >"live reset". This will be a generic param.
> >> >>
> >> >> As I wrote above, I believe this should be an option
> >> >> to "devlink dev reload", not a param.
> >> >I think you are still confused with enabling feature in NVRAM
> >> >configuration of the device and command to trigger reset. This param
> >> >will enable the feature in the device NVRAM configuration and does not
> >> >trigger the actual reset.
> >> >
> >> >Only when the param is set, feature will be enabled in the device and
> >> >firmware supports the "live reset". When the param is disabled,
> >> >firmware cannot support "live reset" and user needs to do PCIe reset
> >> >after flashing the firmware for it to take effect..
> >>
> >> Does that mean that after reboot, when user triggers fw reset, it will
> >> be always "live" is possible? Meaning, user will no have a way to
> >> specify that per-reset?
> >Right now, there is no option for user to mention the type of reset.
> >
> >As you suggested, we need to extend 'devlink dev reload' for users to
> >mention the type of reset.
>
> Does your fw support it? The option of "I want live reset now/I
> don't want live reset now"?

Yes, our firmware supports both. Even when "live reset" is enabled,
users can choose the option of "live reset" / "reset on driver
reload".

>
>
>
> >
> >>
> >>
> >> >
> >> >Once feature is enabled in NVRAM configuration, it will be persistent
> >> >across reboots.
> >> >
> >> >User still needs to use "devlink dev reload" command to do the "live reset".
> >> >>
> >> >>
> >> >> >
> >> >> >2. allow_fw_live_reset - To indicate, if any of the host/PF turns it
> >> >> >off, "live reset" is not allowed. This serves the purpose of what we
> >> >> >are trying to add in runtime cmode.
> >> >>
> >> >> Yeah.
> >> >And this param will enable the feature in the driver for driver to
> >> >allow the firmware to go for "live reset", where as above param will
> >> >enable the feature in NVRAM configuration of the device.
> >> >>
> >> >> >Do you want me to keep it as a driver-specific param?
> >> >>
> >> >> There is nothing driver-specific about this.
> >> >okay.
> >> >>
> >> >>
> >> >> >
> >> >> >Please let me know if this is clear and makes less confusion.
> >> >> >
> >> >> >Thanks,
> >> >> >Vasundhara
> >> >> >
> >> >> >> 2) devlink param that would indicate "I am okay if someone else (not by
> >> >> >>    my devlink instance) resets my firmware".
> >> >> >>
> >> >> >> Could you please split?
> >> >> >>
> >> >> >>
> >> >> >> >------------------------
> >> >> >> >
> >> >> >> >Thanks,
> >> >> >> >Vasundhara
> >> >> >> >>
> >> >> >> >> >
> >> >> >> >> >Thanks,
> >> >> >> >> >Vasundhara
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> >+       system.
> >> >> >> >> >> >diff --git a/include/net/devlink.h b/include/net/devlink.h
> >> >> >> >> >> >index 8ffc1b5c..488b61c 100644
> >> >> >> >> >> >--- a/include/net/devlink.h
> >> >> >> >> >> >+++ b/include/net/devlink.h
> >> >> >> >> >> >@@ -406,6 +406,7 @@ enum devlink_param_generic_id {
> >> >> >> >> >> >       DEVLINK_PARAM_GENERIC_ID_FW_LOAD_POLICY,
> >> >> >> >> >> >       DEVLINK_PARAM_GENERIC_ID_RESET_DEV_ON_DRV_PROBE,
> >> >> >> >> >> >       DEVLINK_PARAM_GENERIC_ID_ENABLE_ROCE,
> >> >> >> >> >> >+      DEVLINK_PARAM_GENERIC_ID_ALLOW_FW_LIVE_RESET,
> >> >> >> >> >> >
> >> >> >> >> >> >       /* add new param generic ids above here*/
> >> >> >> >> >> >       __DEVLINK_PARAM_GENERIC_ID_MAX,
> >> >> >> >> >> >@@ -443,6 +444,9 @@ enum devlink_param_generic_id {
> >> >> >> >> >> > #define DEVLINK_PARAM_GENERIC_ENABLE_ROCE_NAME "enable_roce"
> >> >> >> >> >> > #define DEVLINK_PARAM_GENERIC_ENABLE_ROCE_TYPE DEVLINK_PARAM_TYPE_BOOL
> >> >> >> >> >> >
> >> >> >> >> >> >+#define DEVLINK_PARAM_GENERIC_ALLOW_FW_LIVE_RESET_NAME "allow_fw_live_reset"
> >> >> >> >> >> >+#define DEVLINK_PARAM_GENERIC_ALLOW_FW_LIVE_RESET_TYPE DEVLINK_PARAM_TYPE_BOOL
> >> >> >> >> >> >+
> >> >> >> >> >> > #define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate)    \
> >> >> >> >> >> > {                                                                     \
> >> >> >> >> >> >       .id = DEVLINK_PARAM_GENERIC_ID_##_id,                           \
> >> >> >> >> >> >diff --git a/net/core/devlink.c b/net/core/devlink.c
> >> >> >> >> >> >index 7b76e5f..8544f23 100644
> >> >> >> >> >> >--- a/net/core/devlink.c
> >> >> >> >> >> >+++ b/net/core/devlink.c
> >> >> >> >> >> >@@ -3011,6 +3011,11 @@ static int devlink_nl_cmd_flash_update(struct sk_buff *skb,
> >> >> >> >> >> >               .name = DEVLINK_PARAM_GENERIC_ENABLE_ROCE_NAME,
> >> >> >> >> >> >               .type = DEVLINK_PARAM_GENERIC_ENABLE_ROCE_TYPE,
> >> >> >> >> >> >       },
> >> >> >> >> >> >+      {
> >> >> >> >> >> >+              .id = DEVLINK_PARAM_GENERIC_ID_ALLOW_FW_LIVE_RESET,
> >> >> >> >> >> >+              .name = DEVLINK_PARAM_GENERIC_ALLOW_FW_LIVE_RESET_NAME,
> >> >> >> >> >> >+              .type = DEVLINK_PARAM_GENERIC_ALLOW_FW_LIVE_RESET_TYPE,
> >> >> >> >> >> >+      },
> >> >> >> >> >> > };
> >> >> >> >> >> >
> >> >> >> >> >> > static int devlink_param_generic_verify(const struct devlink_param *param)
> >> >> >> >> >> >--
> >> >> >> >> >> >1.8.3.1
> >> >> >> >> >> >