All of lore.kernel.org
 help / color / mirror / Atom feed
From: saeed@kernel.org
To: "David S. Miller" <davem@davemloft.net>,
	Jakub Kicinski <kuba@kernel.org>
Cc: netdev@vger.kernel.org, Shay Drory <shayd@mellanox.com>,
	Saeed Mahameed <saeedm@mellanox.com>,
	Saeed Mahameed <saeedm@nvidia.com>
Subject: [net 01/15] net/mlx5: Don't allow health work when device is uninitialized
Date: Wed, 30 Sep 2020 19:05:02 -0700	[thread overview]
Message-ID: <20201001020516.41217-2-saeed@kernel.org> (raw)
In-Reply-To: <20201001020516.41217-1-saeed@kernel.org>

From: Shay Drory <shayd@mellanox.com>

On error flow due to failure on driver load, driver can be
un-initializing while a health work is running in the background,
health work shouldn't be allowed at this point, as it needs resources to
be initialized and there is no point to recover on driver load failures.

Therefore, introducing a new state bit to indicated if device is
initialized, for health work to check before trying to recover the driver.

Fixes: b6e0b6bebe07 ("net/mlx5: Fix fatal error handling during device load")
Signed-off-by: Shay Drory <shayd@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/health.c | 11 +++++++++++
 drivers/net/ethernet/mellanox/mlx5/core/main.c   |  2 ++
 include/linux/mlx5/driver.h                      |  1 +
 3 files changed, 14 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c
index b31f769d2df9..c7a8dfe1ae19 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/health.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c
@@ -190,6 +190,11 @@ static bool reset_fw_if_needed(struct mlx5_core_dev *dev)
 	return true;
 }
 
+static bool mlx5_is_device_initialized(struct mlx5_core_dev *dev)
+{
+	return test_bit(MLX5_INTERFACE_STATE_INITIALIZED, &dev->intf_state);
+}
+
 void mlx5_enter_error_state(struct mlx5_core_dev *dev, bool force)
 {
 	bool err_detected = false;
@@ -201,6 +206,9 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, bool force)
 		err_detected = true;
 	}
 	mutex_lock(&dev->intf_state_mutex);
+	if (!mlx5_is_device_initialized(dev))
+		return;
+
 	if (!err_detected && dev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR)
 		goto unlock;/* a previous error is still being handled */
 	if (dev->state == MLX5_DEVICE_STATE_UNINITIALIZED) {
@@ -609,6 +617,9 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
 	dev = container_of(priv, struct mlx5_core_dev, priv);
 
 	mlx5_enter_error_state(dev, false);
+	if (!mlx5_is_device_initialized(dev))
+		return;
+
 	if (IS_ERR_OR_NULL(health->fw_fatal_reporter)) {
 		if (mlx5_health_try_recover(dev))
 			mlx5_core_err(dev, "health recovery failed\n");
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index ce43e3feccd9..8ebb8d27d3ac 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -878,6 +878,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev)
 	dev->tracer = mlx5_fw_tracer_create(dev);
 	dev->hv_vhca = mlx5_hv_vhca_create(dev);
 	dev->rsc_dump = mlx5_rsc_dump_create(dev);
+	set_bit(MLX5_INTERFACE_STATE_INITIALIZED, &dev->intf_state);
 
 	return 0;
 
@@ -906,6 +907,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev)
 
 static void mlx5_cleanup_once(struct mlx5_core_dev *dev)
 {
+	clear_bit(MLX5_INTERFACE_STATE_INITIALIZED, &dev->intf_state);
 	mlx5_rsc_dump_destroy(dev);
 	mlx5_hv_vhca_destroy(dev->hv_vhca);
 	mlx5_fw_tracer_destroy(dev->tracer);
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index c145de0473bc..223aaaaaf8a6 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -594,6 +594,7 @@ enum mlx5_device_state {
 
 enum mlx5_interface_state {
 	MLX5_INTERFACE_STATE_UP = BIT(0),
+	MLX5_INTERFACE_STATE_INITIALIZED = BIT(1),
 };
 
 enum mlx5_pci_status {
-- 
2.26.2


  reply	other threads:[~2020-10-01  2:05 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-01  2:05 [pull request][net 00/15] mlx5 fixes 2020-09-30 saeed
2020-10-01  2:05 ` saeed [this message]
2020-10-01 19:21   ` [net 01/15] net/mlx5: Don't allow health work when device is uninitialized David Miller
2020-10-01 19:41     ` Saeed Mahameed
2020-10-01  2:05 ` [net 02/15] net/mlx5: Fix a race when moving command interface to polling mode saeed
2020-10-01  2:05 ` [net 03/15] net/mlx5: Avoid possible free of command entry while timeout comp handler saeed
2020-10-01  2:05 ` [net 04/15] net/mlx5: poll cmd EQ in case of command timeout saeed
2020-10-01  2:05 ` [net 05/15] net/mlx5: Add retry mechanism to the command entry index allocation saeed
2020-10-01  2:05 ` [net 06/15] net/mlx5: cmdif, Avoid skipping reclaim pages if FW is not accessible saeed
2020-10-01  2:05 ` [net 07/15] net/mlx5: Fix request_irqs error flow saeed
2020-10-01  2:05 ` [net 08/15] net/mlx5e: Fix error path for RQ alloc saeed
2020-10-01  2:05 ` [net 09/15] net/mlx5e: Add resiliency in Striding RQ mode for packets larger than MTU saeed
2020-10-01  2:05 ` [net 10/15] net/mlx5e: CT, Fix coverity issue saeed
2020-10-01  2:05 ` [net 11/15] net/mlx5e: Fix driver's declaration to support GRE offload saeed
2020-10-01  2:05 ` [net 12/15] net/mlx5e: Fix return status when setting unsupported FEC mode saeed
2020-10-01  2:05 ` [net 13/15] net/mlx5e: Fix VLAN cleanup flow saeed
2020-10-01  2:05 ` [net 14/15] net/mlx5e: Fix VLAN create flow saeed
2020-10-01  2:05 ` [net 15/15] net/mlx5e: Fix race condition on nhe->n pointer in neigh update saeed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201001020516.41217-2-saeed@kernel.org \
    --to=saeed@kernel.org \
    --cc=davem@davemloft.net \
    --cc=kuba@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=saeedm@mellanox.com \
    --cc=saeedm@nvidia.com \
    --cc=shayd@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.