From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A3C972 for ; Sat, 25 Sep 2021 11:23:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 853346128B; Sat, 25 Sep 2021 11:23:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1632569003; bh=02xRTwet1c5l8qRgcXs4VCpKQJvqfYPSwFUrWIoc2cM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eBdkOeQZAzQfx/m/yxFbsJIGBvkhRpE2hSfkJbHrHeRhC5LOGgZSfk4qdoFd5jXLC 4cajFeDHROqHcKL6Uxo3t/x66TBMHadOtMGgbKbHUJLVc+EoKzRywI13DAUuwL4Nmy XdaiN+sF1pwhdlQ+uSs0w4bITD8GdZlM7wYUfcvrz8HaBW66RC6ITexIAdOoJUX2tS NdMaGkjmUePmj+YdIJxYmXTz6HRjqnWB052FPKJjeF22nVurS6AbIJA54sOsNoQIvo MozNtLmeX9dcJkGV28/X2nazhnG4nqa89dbG8o8cBxCkZF4jyGYcMvQtjkCR91Gadf 06/PrR1PnbLsA== From: Leon Romanovsky To: "David S . Miller" , Jakub Kicinski Cc: Leon Romanovsky , Alexandre Belloni , Andrew Lunn , Ariel Elior , Bin Luo , Claudiu Manoil , Coiby Xu , Derek Chickles , drivers@pensando.io, Felix Manlunas , Florian Fainelli , Geetha sowjanya , Greg Kroah-Hartman , GR-everest-linux-l2@marvell.com, GR-Linux-NIC-Dev@marvell.com, hariprasad , Ido Schimmel , Intel Corporation , intel-wired-lan@lists.osuosl.org, Ioana Ciornei , Jerin Jacob , Jesse Brandeburg , Jiri Pirko , Jonathan Lemon , Linu Cherian , linux-kernel@vger.kernel.org, linux-omap@vger.kernel.org, linux-rdma@vger.kernel.org, linux-staging@lists.linux.dev, Loic Poulain , Manish Chopra , M Chetan Kumar , Michael Chan , Michael Guralnik , netdev@vger.kernel.org, oss-drivers@corigine.com, Richard Cochran , Saeed Mahameed , Satanand Burla , Sergey Ryazanov , Shannon Nelson , Simon Horman , Subbaraya Sundeep , Sunil Goutham , Taras Chornyi , Tariq Toukan , Tony Nguyen , UNGLinuxDriver@microchip.com, Vadym Kochan , Vivien Didelot , Vladimir Oltean Subject: [PATCH net-next v1 03/21] liquidio: Overcome missing device lock protection in init/remove flows Date: Sat, 25 Sep 2021 14:22:43 +0300 Message-Id: <4e78bd8c6f45eef8093547a37c4d00ef0d1fef56.1632565508.git.leonro@nvidia.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-staging@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Leon Romanovsky The liquidio driver is broken by design. It initialize PCI devices in separate delayed works. It causes to the situation where device lock is dropped during initialize and remove sequences. That lock is part of driver/core and needed to protect from races during init, destroy and bus invocations. In addition to lack of locking protection, it has incorrect order of destroy flows and very questionable synchronization scheme based on atomic_t. This change doesn't fix that driver but makes sure that rest of the netdev subsystem doesn't suffer from such basic protection by adding device_lock over devlink_*() APIs and by moving devlink_register() to be last command in setup_nic_devices(). Signed-off-by: Leon Romanovsky --- .../net/ethernet/cavium/liquidio/lio_main.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c index a34b3bb2dd4f..dafc79bd34f4 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c @@ -1279,6 +1279,14 @@ static int liquidio_stop_nic_module(struct octeon_device *oct) struct lio *lio; dev_dbg(&oct->pci_dev->dev, "Stopping network interfaces\n"); + device_lock(&oct->pci_dev->dev); + if (oct->devlink) { + devlink_unregister(oct->devlink); + devlink_free(oct->devlink); + oct->devlink = NULL; + } + device_unlock(&oct->pci_dev->dev); + if (!oct->ifcount) { dev_err(&oct->pci_dev->dev, "Init for Octeon was not completed\n"); return 1; @@ -1300,12 +1308,6 @@ static int liquidio_stop_nic_module(struct octeon_device *oct) for (i = 0; i < oct->ifcount; i++) liquidio_destroy_nic_device(oct, i); - if (oct->devlink) { - devlink_unregister(oct->devlink); - devlink_free(oct->devlink); - oct->devlink = NULL; - } - dev_dbg(&oct->pci_dev->dev, "Network interfaces stopped\n"); return 0; } @@ -3749,10 +3751,12 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) } } + device_lock(&octeon_dev->pci_dev->dev); devlink = devlink_alloc(&liquidio_devlink_ops, sizeof(struct lio_devlink_priv), &octeon_dev->pci_dev->dev); if (!devlink) { + device_unlock(&octeon_dev->pci_dev->dev); dev_err(&octeon_dev->pci_dev->dev, "devlink alloc failed\n"); goto setup_nic_dev_free; } @@ -3760,9 +3764,10 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) lio_devlink = devlink_priv(devlink); lio_devlink->oct = octeon_dev; - devlink_register(devlink); octeon_dev->devlink = devlink; octeon_dev->eswitch_mode = DEVLINK_ESWITCH_MODE_LEGACY; + devlink_register(devlink); + device_unlock(&octeon_dev->pci_dev->dev); return 0; -- 2.31.1 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Leon Romanovsky Date: Sat, 25 Sep 2021 14:22:43 +0300 Subject: [Intel-wired-lan] [PATCH net-next v1 03/21] liquidio: Overcome missing device lock protection in init/remove flows In-Reply-To: References: Message-ID: <4e78bd8c6f45eef8093547a37c4d00ef0d1fef56.1632565508.git.leonro@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: intel-wired-lan@osuosl.org List-ID: From: Leon Romanovsky The liquidio driver is broken by design. It initialize PCI devices in separate delayed works. It causes to the situation where device lock is dropped during initialize and remove sequences. That lock is part of driver/core and needed to protect from races during init, destroy and bus invocations. In addition to lack of locking protection, it has incorrect order of destroy flows and very questionable synchronization scheme based on atomic_t. This change doesn't fix that driver but makes sure that rest of the netdev subsystem doesn't suffer from such basic protection by adding device_lock over devlink_*() APIs and by moving devlink_register() to be last command in setup_nic_devices(). Signed-off-by: Leon Romanovsky --- .../net/ethernet/cavium/liquidio/lio_main.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c index a34b3bb2dd4f..dafc79bd34f4 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c @@ -1279,6 +1279,14 @@ static int liquidio_stop_nic_module(struct octeon_device *oct) struct lio *lio; dev_dbg(&oct->pci_dev->dev, "Stopping network interfaces\n"); + device_lock(&oct->pci_dev->dev); + if (oct->devlink) { + devlink_unregister(oct->devlink); + devlink_free(oct->devlink); + oct->devlink = NULL; + } + device_unlock(&oct->pci_dev->dev); + if (!oct->ifcount) { dev_err(&oct->pci_dev->dev, "Init for Octeon was not completed\n"); return 1; @@ -1300,12 +1308,6 @@ static int liquidio_stop_nic_module(struct octeon_device *oct) for (i = 0; i < oct->ifcount; i++) liquidio_destroy_nic_device(oct, i); - if (oct->devlink) { - devlink_unregister(oct->devlink); - devlink_free(oct->devlink); - oct->devlink = NULL; - } - dev_dbg(&oct->pci_dev->dev, "Network interfaces stopped\n"); return 0; } @@ -3749,10 +3751,12 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) } } + device_lock(&octeon_dev->pci_dev->dev); devlink = devlink_alloc(&liquidio_devlink_ops, sizeof(struct lio_devlink_priv), &octeon_dev->pci_dev->dev); if (!devlink) { + device_unlock(&octeon_dev->pci_dev->dev); dev_err(&octeon_dev->pci_dev->dev, "devlink alloc failed\n"); goto setup_nic_dev_free; } @@ -3760,9 +3764,10 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) lio_devlink = devlink_priv(devlink); lio_devlink->oct = octeon_dev; - devlink_register(devlink); octeon_dev->devlink = devlink; octeon_dev->eswitch_mode = DEVLINK_ESWITCH_MODE_LEGACY; + devlink_register(devlink); + device_unlock(&octeon_dev->pci_dev->dev); return 0; -- 2.31.1