Netdev Archive on lore.kernel.org
 help / color / Atom feed
From: Saeed Mahameed <saeedm@mellanox.com>
To: "David S. Miller" <davem@davemloft.net>, kuba@kernel.org
Cc: netdev@vger.kernel.org, Jiri Pirko <jiri@mellanox.com>,
	Parav Pandit <parav@mellanox.com>,
	Mark Bloch <markb@mellanox.com>,
	Saeed Mahameed <saeedm@mellanox.com>
Subject: [net-next 13/16] devlink: Rely on driver eswitch thread safety instead of devlink
Date: Wed, 25 Mar 2020 23:38:06 -0700
Message-ID: <20200326063809.139919-14-saeedm@mellanox.com> (raw)
In-Reply-To: <20200326063809.139919-1-saeedm@mellanox.com>

From: Parav Pandit <parav@mellanox.com>

devlink_nl_cmd_eswitch_set_doit() doesn't hold devlink->lock mutex while
invoking driver callback. This is likely due to eswitch mode setting
involves adding/remove devlink ports, health reporters or
other devlink objects for a devlink device.

So it is driver responsiblity to ensure thread safe eswitch state
transition happening via either sriov legacy enablement or via devlink
eswitch set callback.

Therefore, get() callback should also be invoked without holding
devlink->lock mutex.
Vendor driver can use same internal lock which it uses during eswitch
mode set() callback.
This makes get() and set() implimentation symmetric in devlink core and
in vendor drivers.

Hence, remove holding devlink->lock mutex during eswitch get() callback.

Failing to do so results into below deadlock scenario when mlx5_core
driver is improved to handle eswitch mode set critical section invoked
by devlink and sriov sysfs interface in subsequent patch.

devlink_nl_cmd_eswitch_set_doit()
   mlx5_eswitch_mode_set()
     mutex_lock(esw->mode_lock) <- Lock A
     [...]
     register_devlink_port()
       mutex_lock(&devlink->lock); <- lock B

mutex_lock(&devlink->lock); <- lock B
devlink_nl_cmd_eswitch_get_doit()
   mlx5_eswitch_mode_get()
   mutex_lock(esw->mode_lock) <- Lock A

In subsequent patch, mlx5_core driver uses its internal lock during
get() and set() eswitch callbacks.

Other drivers have been inspected which returns either constant during
get operations or reads the value from already allocated structure.
Hence it is safe to remove the lock in get( ) callback and let vendor
driver handle it.

Reviewed-by: Jiri Pirko <jiri@mellanox.com>
Reviewed-by: Mark Bloch <markb@mellanox.com>
Signed-off-by: Parav Pandit <parav@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
 net/core/devlink.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index 73bb8fbe3393..a9036af7e002 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -6187,7 +6187,8 @@ static const struct genl_ops devlink_nl_ops[] = {
 		.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
 		.doit = devlink_nl_cmd_eswitch_get_doit,
 		.flags = GENL_ADMIN_PERM,
-		.internal_flags = DEVLINK_NL_FLAG_NEED_DEVLINK,
+		.internal_flags = DEVLINK_NL_FLAG_NEED_DEVLINK |
+				  DEVLINK_NL_FLAG_NO_LOCK,
 	},
 	{
 		.cmd = DEVLINK_CMD_ESWITCH_SET,
-- 
2.25.1


  parent reply index

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-26  6:37 [pull request][net-next 00/16] Mellanox, mlx5 updates 2020-03-25 Saeed Mahameed
2020-03-26  6:37 ` [net-next 01/16] net/mlx5e: Fix actions_match_supported() return Saeed Mahameed
2020-03-26  6:37 ` [net-next 02/16] net/mlx5e: remove duplicated check chain_index in mlx5e_rep_setup_ft_cb Saeed Mahameed
2020-03-26  6:37 ` [net-next 03/16] net/mlx5: E-Switch, Enable restore table only if reg_c1 is supported Saeed Mahameed
2020-03-26  6:37 ` [net-next 04/16] net/mlx5: E-Switch, Enable chains only if regs loopback is enabled Saeed Mahameed
2020-03-26  6:37 ` [net-next 05/16] net/mlx5: E-Switch, free flow_group_in after creating the restore table Saeed Mahameed
2020-03-26  6:37 ` [net-next 06/16] net/mlx5: E-Switch, Use correct type for chain, prio and level values Saeed Mahameed
2020-03-26  6:38 ` [net-next 07/16] net/mlx5: Simplify matching group searches Saeed Mahameed
2020-03-26  6:38 ` [net-next 08/16] net/mlx5: Fix group version management Saeed Mahameed
2020-03-26  6:38 ` [net-next 09/16] net/mlx5: Avoid incrementing FTE version Saeed Mahameed
2020-03-26  6:38 ` [net-next 10/16] net/mlx5: Avoid group version scan when not necessary Saeed Mahameed
2020-03-26  6:38 ` [net-next 11/16] net/mlx5: Simplify mlx5_register_device to return void Saeed Mahameed
2020-03-26  6:38 ` [net-next 12/16] net/mlx5: Simplify mlx5_unload_one() and its callers Saeed Mahameed
2020-03-26  6:38 ` Saeed Mahameed [this message]
2020-03-26  6:38 ` [net-next 14/16] net/mlx5: Split eswitch mode check to different helper function Saeed Mahameed
2020-03-26  6:38 ` [net-next 15/16] net/mlx5: E-switch, Extend eswitch enable to handle num_vfs change Saeed Mahameed
2020-03-26  6:38 ` [net-next 16/16] net/mlx5: E-switch, Protect eswitch mode changes Saeed Mahameed
2020-03-26 18:39 ` [pull request][net-next 00/16] Mellanox, mlx5 updates 2020-03-25 David Miller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200326063809.139919-14-saeedm@mellanox.com \
    --to=saeedm@mellanox.com \
    --cc=davem@davemloft.net \
    --cc=jiri@mellanox.com \
    --cc=kuba@kernel.org \
    --cc=markb@mellanox.com \
    --cc=netdev@vger.kernel.org \
    --cc=parav@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Netdev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/netdev/0 netdev/git/0.git
	git clone --mirror https://lore.kernel.org/netdev/1 netdev/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 netdev netdev/ https://lore.kernel.org/netdev \
		netdev@vger.kernel.org
	public-inbox-index netdev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.netdev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git