netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Michael Chan <michael.chan@broadcom.com>
To: davem@davemloft.net
Cc: netdev@vger.kernel.org, vasundhara-v.volam@broadcom.com,
	ray.jui@broadcom.com
Subject: [PATCH net-next v2 21/22] bnxt_en: Add bnxt_fw_exception() to handle fatal firmware errors.
Date: Thu, 29 Aug 2019 23:55:04 -0400	[thread overview]
Message-ID: <1567137305-5853-22-git-send-email-michael.chan@broadcom.com> (raw)
In-Reply-To: <1567137305-5853-1-git-send-email-michael.chan@broadcom.com>

This call will handle fatal firmware errors by forcing a reset on the
firmware.  The master function driver will carry out the forced reset.
The sequence will go through the same bnxt_fw_reset_task() workqueue.
This fatal reset differs from the non-fatal reset at the beginning
stages.  From the BNXT_FW_RESET_STATE_ENABLE_DEV state onwards where
the firmware is coming out of reset, it is practically identical to the
non-fatal reset.

The next patch will add the periodic heartbeat check and the devlink
reporter to report the fatal event and to initiate the bnxt_fw_exception()
call.

Signed-off-by: Vasundhara Volam <vasundhara-v.volam@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 44 +++++++++++++++++++++++++++++++
 drivers/net/ethernet/broadcom/bnxt/bnxt.h |  1 +
 2 files changed, 45 insertions(+)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 51cf679..5c7379e 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -10003,6 +10003,40 @@ static void bnxt_fw_reset_close(struct bnxt *bp)
 	bp->ctx = NULL;
 }
 
+/* rtnl_lock is acquired before calling this function */
+static void bnxt_force_fw_reset(struct bnxt *bp)
+{
+	struct bnxt_fw_health *fw_health = bp->fw_health;
+	u32 wait_dsecs;
+
+	if (!test_bit(BNXT_STATE_OPEN, &bp->state) ||
+	    test_bit(BNXT_STATE_IN_FW_RESET, &bp->state))
+		return;
+
+	set_bit(BNXT_STATE_IN_FW_RESET, &bp->state);
+	bnxt_fw_reset_close(bp);
+	wait_dsecs = fw_health->master_func_wait_dsecs;
+	if (fw_health->master) {
+		if (fw_health->flags & ERROR_RECOVERY_QCFG_RESP_FLAGS_CO_CPU)
+			wait_dsecs = 0;
+		bp->fw_reset_state = BNXT_FW_RESET_STATE_RESET_FW;
+	} else {
+		bp->fw_reset_timestamp = jiffies + wait_dsecs * HZ / 10;
+		wait_dsecs = fw_health->normal_func_wait_dsecs;
+		bp->fw_reset_state = BNXT_FW_RESET_STATE_ENABLE_DEV;
+	}
+	bp->fw_reset_max_dsecs = fw_health->post_reset_max_wait_dsecs;
+	bnxt_queue_fw_reset_work(bp, wait_dsecs * HZ / 10);
+}
+
+void bnxt_fw_exception(struct bnxt *bp)
+{
+	set_bit(BNXT_STATE_FW_FATAL_COND, &bp->state);
+	bnxt_rtnl_lock_sp(bp);
+	bnxt_force_fw_reset(bp);
+	bnxt_rtnl_unlock_sp(bp);
+}
+
 void bnxt_fw_reset(struct bnxt *bp)
 {
 	int rc;
@@ -10506,6 +10540,16 @@ static void bnxt_fw_reset_task(struct work_struct *work)
 		return;
 	}
 	case BNXT_FW_RESET_STATE_ENABLE_DEV:
+		if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state) &&
+		    bp->fw_health) {
+			u32 val;
+
+			val = bnxt_fw_health_readl(bp,
+						   BNXT_FW_RESET_INPROG_REG);
+			if (val)
+				netdev_warn(bp->dev, "FW reset inprog %x after min wait time.\n",
+					    val);
+		}
 		clear_bit(BNXT_STATE_FW_FATAL_COND, &bp->state);
 		if (pci_enable_device(bp->pdev)) {
 			netdev_err(bp->dev, "Cannot re-enable PCI device\n");
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index f3a6aad..3459b2a 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -1982,6 +1982,7 @@ int bnxt_open_nic(struct bnxt *, bool, bool);
 int bnxt_half_open_nic(struct bnxt *bp);
 void bnxt_half_close_nic(struct bnxt *bp);
 int bnxt_close_nic(struct bnxt *, bool, bool);
+void bnxt_fw_exception(struct bnxt *bp);
 void bnxt_fw_reset(struct bnxt *bp);
 int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
 		     int tx_xdp);
-- 
2.5.1


  parent reply	other threads:[~2019-08-30  3:56 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-30  3:54 [PATCH net-next v2 00/22] bnxt_en: health and error recovery Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 01/22] bnxt_en: Use a common function to print the same ethtool -f error message Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 02/22] bnxt_en: Remove the -1 error return code from bnxt_hwrm_do_send_msg() Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 03/22] bnxt_en: Convert error code in firmware message response to standard code Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 04/22] bnxt_en: Simplify error checking in the SR-IOV message forwarding functions Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 05/22] bnxt_en: Suppress all error messages in hwrm_do_send_msg() in silent mode Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 06/22] bnxt_en: Prepare bnxt_init_one() to be called multiple times Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 07/22] bnxt_en: Refactor bnxt_sriov_enable() Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 08/22] bnxt_en: Register buffers for VFs before reserving resources Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 09/22] bnxt_en: Handle firmware reset status during IF_UP Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 10/22] bnxt_en: Discover firmware error recovery capabilities Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 11/22] bnxt_en: Pre-map the firmware health monitoring registers Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 12/22] bnxt_en: Enable health monitoring Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 13/22] bnxt_en: Add BNXT_STATE_IN_FW_RESET state Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 14/22] bnxt_en: Add new FW devlink_health_reporter Michael Chan
2019-10-07  9:56   ` Jiri Pirko
2019-10-09  4:55     ` Vasundhara Volam
2019-10-09  7:04       ` Jiri Pirko
2019-08-30  3:54 ` [PATCH net-next v2 15/22] bnxt_en: Handle RESET_NOTIFY async event from firmware Michael Chan
2019-08-30  3:54 ` [PATCH net-next v2 16/22] bnxt_en: Handle firmware reset Michael Chan
2019-08-30 17:56   ` kbuild test robot
2019-08-30  3:55 ` [PATCH net-next v2 17/22] bnxt_en: Add devlink health reset reporter Michael Chan
2019-08-30  3:55 ` [PATCH net-next v2 18/22] bnxt_en: Retain user settings on a VF after RESET_NOTIFY event Michael Chan
2019-08-30  3:55 ` [PATCH net-next v2 19/22] bnxt_en: Do not send firmware messages if firmware is in error state Michael Chan
2019-08-30  3:55 ` [PATCH net-next v2 20/22] bnxt_en: Add RESET_FW state logic to bnxt_fw_reset_task() Michael Chan
2019-08-30  3:55 ` Michael Chan [this message]
2019-08-30  3:55 ` [PATCH net-next v2 22/22] bnxt_en: Add FW fatal devlink_health_reporter Michael Chan
2019-08-30 21:02 ` [PATCH net-next v2 00/22] bnxt_en: health and error recovery David Miller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1567137305-5853-22-git-send-email-michael.chan@broadcom.com \
    --to=michael.chan@broadcom.com \
    --cc=davem@davemloft.net \
    --cc=netdev@vger.kernel.org \
    --cc=ray.jui@broadcom.com \
    --cc=vasundhara-v.volam@broadcom.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).