[RFC PATCH v1 1/3] PCI: Unify pci_reset_function_locked() and __pci_reset_function_locked()

[RFC PATCH v1 1/3] PCI: Unify pci_reset_function_locked() and __pci_reset_function_locked()

[Sinan Kaya]

The difference between pci_reset_function_locked() and
__pci_reset_function_locked() is the saving and restoring of the registers.
Unify these API by adding saverestore argument that caller passes.

Signed-off-by: Sinan Kaya <okaya@kernel.org>
---
 drivers/infiniband/hw/hfi1/pcie.c               | 3 ++-
 drivers/net/ethernet/cavium/liquidio/lio_main.c | 3 ++-
 drivers/pci/pci.c                               | 6 ++++--
 drivers/pci/pci.h                               | 1 +
 drivers/xen/xen-pciback/pci_stub.c              | 9 ++++++---
 include/linux/pci.h                             | 7 +++++--
 6 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/pcie.c b/drivers/infiniband/hw/hfi1/pcie.c
index 38f96192e5f0..a43cd031bfe4 100644
--- a/drivers/infiniband/hw/hfi1/pcie.c
+++ b/drivers/infiniband/hw/hfi1/pcie.c
@@ -897,7 +897,8 @@ static int trigger_sbr(struct hfi1_devdata *dd)
 	 * to be implemented to have cleaner interface but this fixes the
 	 * current brokenness
 	 */
-	return __pci_reset_function_locked(dev, PCI_RESET_LINK);
+	return pci_reset_function_locked(dev, PCI_RESET_LINK |
+					      PCI_RESET_NOSAVERESTORE);
 }
 
 /*
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 0ff76722734d..4ae3935ff10d 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -989,7 +989,8 @@ static void octeon_pci_flr(struct octeon_device *oct)
 	pci_write_config_word(oct->pci_dev, PCI_COMMAND,
 			      PCI_COMMAND_INTX_DISABLE);
 
-	rc = __pci_reset_function_locked(oct->pci_dev, PCI_RESET_ANY);
+	rc = pci_reset_function_locked(oct->pci_dev, PCI_RESET_ANY |
+						     PCI_RESET_NOSAVERESTORE);
 
 	if (rc != 0)
 		dev_err(&oct->pci_dev->dev, "Error %d resetting PCI function %d\n",
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 1feecbb7f85d..ccd69f5d01f3 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -4856,11 +4856,13 @@ int pci_reset_function_locked(struct pci_dev *dev, u32 reset_type)
 	if (!dev->reset_fn)
 		return -ENOTTY;
 
-	pci_dev_save_and_disable(dev);
+	if (!(reset_type & PCI_RESET_NOSAVERESTORE))
+		pci_dev_save_and_disable(dev);
 
 	rc = __pci_reset_function_locked(dev, reset_type);
 
-	pci_dev_restore(dev);
+	if (!(reset_type & PCI_RESET_NOSAVERESTORE))
+		pci_dev_restore(dev);
 
 	return rc;
 }
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 0444bfa51b52..fbfb44fb32b7 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -35,6 +35,7 @@ int pci_mmap_fits(struct pci_dev *pdev, int resno, struct vm_area_struct *vmai,
 
 int pci_probe_reset_function(struct pci_dev *dev, u32 reset_type);
 int pci_bridge_secondary_bus_reset(struct pci_dev *dev);
+int __pci_reset_function_locked(struct pci_dev *dev, u32 reset_type);
 
 /**
  * struct pci_platform_pm_ops - Firmware PM callbacks
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index 6dfb805bcb19..7a27d8fb0528 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -105,7 +105,8 @@ static void pcistub_device_release(struct kref *kref)
 	/* Call the reset function which does not take lock as this
 	 * is called from "unbind" which takes a device_lock mutex.
 	 */
-	__pci_reset_function_locked(dev, PCI_RESET_ANY);
+	pci_reset_function_locked(dev, PCI_RESET_ANY |
+				       PCI_RESET_NOSAVERESTORE);
 	if (pci_load_and_free_saved_state(dev, &dev_data->pci_saved_state))
 		dev_info(&dev->dev, "Could not reload PCI state\n");
 	else
@@ -283,7 +284,8 @@ void pcistub_put_pci_dev(struct pci_dev *dev)
 	 * (so it's ready for the next domain)
 	 */
 	device_lock_assert(&dev->dev);
-	__pci_reset_function_locked(dev, PCI_RESET_ANY);
+	pci_reset_function_locked(dev, PCI_RESET_ANY |
+				       PCI_RESET_NOSAVERESTORE);
 
 	dev_data = pci_get_drvdata(dev);
 	ret = pci_load_saved_state(dev, dev_data->pci_saved_state);
@@ -417,7 +419,8 @@ static int pcistub_init_device(struct pci_dev *dev)
 		dev_err(&dev->dev, "Could not store PCI conf saved state!\n");
 	else {
 		dev_dbg(&dev->dev, "resetting (FLR, D3, etc) the device\n");
-		__pci_reset_function_locked(dev, PCI_RESET_ANY);
+		pci_reset_function_locked(dev, PCI_RESET_ANY |
+					       PCI_RESET_NOSAVERESTORE);
 		pci_restore_state(dev);
 	}
 	/* Now disable the device (this also ensures some private device
diff --git a/include/linux/pci.h b/include/linux/pci.h
index cfb1018d774e..fedd34a5af77 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -893,17 +893,21 @@ enum {
  * recovery. Using this option directly and bypassing hotplug driver may
  * cause a deadlock if platform supports hotplug. Please refer to
  * PCI_RESET_LINK and let the PCI core do the hotplug detection.
+ *
+ * PCI_RESET_NOSAVERESTORE tells the PCI core to not save the card context
+ * before performing a reset and restore the values after reset.
  */
 #define PCI_RESET_DEV_SPECIFIC	(1 << 0)
 #define PCI_RESET_FLR		(1 << 1)
 #define PCI_RESET_PM		(1 << 2)
 #define PCI_RESET_SLOT		(1 << 3)
 #define PCI_RESET_BUS		(1 << 4)
+#define PCI_RESET_NOSAVERESTORE (1 << 5)
 
-#define PCI_RESET_ANY		(~0)
 #define PCI_RESET_FUNC		(PCI_RESET_DEV_SPECIFIC | \
 				 PCI_RESET_FLR | PCI_RESET_PM)
 #define PCI_RESET_LINK		(PCI_RESET_SLOT | PCI_RESET_BUS)
+#define PCI_RESET_ANY		(PCI_RESET_FUNC | PCI_RESET_LINK)
 
 /* These external functions are only available when PCI support is enabled */
 #ifdef CONFIG_PCI
@@ -1167,7 +1171,6 @@ u32 pcie_bandwidth_available(struct pci_dev *dev, struct pci_dev **limiting_dev,
 void pcie_print_link_status(struct pci_dev *dev);
 bool pcie_has_flr(struct pci_dev *dev);
 int pcie_flr(struct pci_dev *dev);
-int __pci_reset_function_locked(struct pci_dev *dev, u32 reset_type);
 int pci_reset_function(struct pci_dev *dev, u32 reset_type);
 int pci_reset_function_locked(struct pci_dev *dev, u32 reset_type);
 int pci_try_reset_function(struct pci_dev *dev, u32 reset_type);
-- 
2.19.0

[RFC PATCH v1 2/3] PCI: Add options to pci_reset_function

[Sinan Kaya]

Getting ready to deprecate pci_reset_function_locked(). Add
saverestore and locked parameters to pci_reset_function() function
and add saverestore = true and locked = false to all existing
callers.

Signed-off-by: Sinan Kaya <okaya@kernel.org>
---
 drivers/pci/pci.c   | 14 ++++++++++----
 include/linux/pci.h |  4 ++++
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index ccd69f5d01f3..17b10c33cb53 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -4819,13 +4819,19 @@ int pci_reset_function(struct pci_dev *dev, u32 reset_type)
 	if (!dev->reset_fn)
 		return -ENOTTY;
 
-	pci_dev_lock(dev);
-	pci_dev_save_and_disable(dev);
+	if (!(reset_type & PCI_RESET_ALREADYLOCKED))
+		pci_dev_lock(dev);
+
+	if (!(reset_type & PCI_RESET_NOSAVERESTORE))
+		pci_dev_save_and_disable(dev);
 
 	rc = __pci_reset_function_locked(dev, reset_type);
 
-	pci_dev_restore(dev);
-	pci_dev_unlock(dev);
+	if (!(reset_type & PCI_RESET_NOSAVERESTORE))
+		pci_dev_restore(dev);
+
+	if (!(reset_type & PCI_RESET_ALREADYLOCKED))
+		pci_dev_unlock(dev);
 
 	return rc;
 }
diff --git a/include/linux/pci.h b/include/linux/pci.h
index fedd34a5af77..ad5996d521eb 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -896,6 +896,9 @@ enum {
  *
  * PCI_RESET_NOSAVERESTORE tells the PCI core to not save the card context
  * before performing a reset and restore the values after reset.
+ *
+ * PCI_RESET_ALREADYLOCKED indicates that caller is holding the device lock and
+ * PCI core should not try to lock again.
  */
 #define PCI_RESET_DEV_SPECIFIC	(1 << 0)
 #define PCI_RESET_FLR		(1 << 1)
@@ -903,6 +906,7 @@ enum {
 #define PCI_RESET_SLOT		(1 << 3)
 #define PCI_RESET_BUS		(1 << 4)
 #define PCI_RESET_NOSAVERESTORE (1 << 5)
+#define PCI_RESET_ALREADYLOCKED (1 << 6)
 
 #define PCI_RESET_FUNC		(PCI_RESET_DEV_SPECIFIC | \
 				 PCI_RESET_FLR | PCI_RESET_PM)
-- 
2.19.0

[RFC PATCH v1 3/3] PCI: Hide pci_reset_function_locked()

[Sinan Kaya]

It is time to hide pci_reset_function_locked() since pci_reset_function()
provides the same functionality. One less API to expose to the users.

Signed-off-by: Sinan Kaya <okaya@kernel.org>
---
 drivers/infiniband/hw/hfi1/pcie.c               |  5 +++--
 drivers/net/ethernet/cavium/liquidio/lio_main.c |  6 +++---
 drivers/pci/pci.h                               |  1 +
 drivers/xen/xen-pciback/pci_stub.c              | 13 +++++++------
 include/linux/pci.h                             |  1 -
 5 files changed, 14 insertions(+), 12 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/pcie.c b/drivers/infiniband/hw/hfi1/pcie.c
index a43cd031bfe4..298086b7936b 100644
--- a/drivers/infiniband/hw/hfi1/pcie.c
+++ b/drivers/infiniband/hw/hfi1/pcie.c
@@ -897,8 +897,9 @@ static int trigger_sbr(struct hfi1_devdata *dd)
 	 * to be implemented to have cleaner interface but this fixes the
 	 * current brokenness
 	 */
-	return pci_reset_function_locked(dev, PCI_RESET_LINK |
-					      PCI_RESET_NOSAVERESTORE);
+	return pci_reset_function(dev, PCI_RESET_LINK |
+				       PCI_RESET_NOSAVERESTORE |
+				       PCI_RESET_ALREADY_LOCKED);
 }
 
 /*
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 4ae3935ff10d..605061b2ee66 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -989,9 +989,9 @@ static void octeon_pci_flr(struct octeon_device *oct)
 	pci_write_config_word(oct->pci_dev, PCI_COMMAND,
 			      PCI_COMMAND_INTX_DISABLE);
 
-	rc = pci_reset_function_locked(oct->pci_dev, PCI_RESET_ANY |
-						     PCI_RESET_NOSAVERESTORE);
-
+	rc = pci_reset_function(oct->pci_dev, PCI_RESET_ANY |
+					      PCI_RESET_NOSAVERESTORE |
+					      PCI_RESET_ALREADYLOCKED);
 	if (rc != 0)
 		dev_err(&oct->pci_dev->dev, "Error %d resetting PCI function %d\n",
 			rc, oct->pf_num);
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index fbfb44fb32b7..e6eefc028ba2 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -36,6 +36,7 @@ int pci_mmap_fits(struct pci_dev *pdev, int resno, struct vm_area_struct *vmai,
 int pci_probe_reset_function(struct pci_dev *dev, u32 reset_type);
 int pci_bridge_secondary_bus_reset(struct pci_dev *dev);
 int __pci_reset_function_locked(struct pci_dev *dev, u32 reset_type);
+int pci_reset_function_locked(struct pci_dev *dev, u32 reset_type);
 
 /**
  * struct pci_platform_pm_ops - Firmware PM callbacks
diff --git a/drivers/xen/xen-pciback/pci_stub.c b/drivers/xen/xen-pciback/pci_stub.c
index 7a27d8fb0528..a43476447809 100644
--- a/drivers/xen/xen-pciback/pci_stub.c
+++ b/drivers/xen/xen-pciback/pci_stub.c
@@ -105,8 +105,8 @@ static void pcistub_device_release(struct kref *kref)
 	/* Call the reset function which does not take lock as this
 	 * is called from "unbind" which takes a device_lock mutex.
 	 */
-	pci_reset_function_locked(dev, PCI_RESET_ANY |
-				       PCI_RESET_NOSAVERESTORE);
+	pci_reset_function(dev, PCI_RESET_ANY | PCI_RESET_NOSAVERESTORE |
+				PCI_RESET_ALREADY_LOCKED);
 	if (pci_load_and_free_saved_state(dev, &dev_data->pci_saved_state))
 		dev_info(&dev->dev, "Could not reload PCI state\n");
 	else
@@ -284,8 +284,8 @@ void pcistub_put_pci_dev(struct pci_dev *dev)
 	 * (so it's ready for the next domain)
 	 */
 	device_lock_assert(&dev->dev);
-	pci_reset_function_locked(dev, PCI_RESET_ANY |
-				       PCI_RESET_NOSAVERESTORE);
+	pci_reset_function(dev, PCI_RESET_ANY | PCI_RESET_NOSAVERESTORE |
+				PCI_RESET_ALREADYLOCKED);
 
 	dev_data = pci_get_drvdata(dev);
 	ret = pci_load_saved_state(dev, dev_data->pci_saved_state);
@@ -419,8 +419,9 @@ static int pcistub_init_device(struct pci_dev *dev)
 		dev_err(&dev->dev, "Could not store PCI conf saved state!\n");
 	else {
 		dev_dbg(&dev->dev, "resetting (FLR, D3, etc) the device\n");
-		pci_reset_function_locked(dev, PCI_RESET_ANY |
-					       PCI_RESET_NOSAVERESTORE);
+		pci_reset_function(dev,	PCI_RESET_ANY |
+					PCI_RESET_NOSAVERESTORE |
+					PCI_RESET_ALREADYLOCKED);
 		pci_restore_state(dev);
 	}
 	/* Now disable the device (this also ensures some private device
diff --git a/include/linux/pci.h b/include/linux/pci.h
index ad5996d521eb..f93007f71ed7 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1176,7 +1176,6 @@ void pcie_print_link_status(struct pci_dev *dev);
 bool pcie_has_flr(struct pci_dev *dev);
 int pcie_flr(struct pci_dev *dev);
 int pci_reset_function(struct pci_dev *dev, u32 reset_type);
-int pci_reset_function_locked(struct pci_dev *dev, u32 reset_type);
 int pci_try_reset_function(struct pci_dev *dev, u32 reset_type);
 int pci_probe_reset_slot(struct pci_slot *slot);
 int pci_probe_reset_bus(struct pci_bus *bus);
-- 
2.19.0

Re: [RFC PATCH v1 2/3] PCI: Add options to pci_reset_function

[Sinan Kaya]

On 10/18/2018 11:23 PM, Sinan Kaya wrote:
> Getting ready to deprecate pci_reset_function_locked(). Add
> saverestore and locked parameters to pci_reset_function() function
> and add saverestore = true and locked = false to all existing
> callers.

I realized that I forgot to update the commit text but this is a follow up
to this conversation per Christoph's request.

https://patchwork.kernel.org/patch/10636071/

Since Alex didn't like the idea much, I also split the API reduction work
out of the original reset API refactoring work and sent as an RFC.

Re: [RFC PATCH v1 3/3] PCI: Hide pci_reset_function_locked()

[Juergen Gross]

On 19/10/2018 05:23, Sinan Kaya wrote:
> It is time to hide pci_reset_function_locked() since pci_reset_function()
> provides the same functionality. One less API to expose to the users.
> 
> Signed-off-by: Sinan Kaya <okaya@kernel.org>

Did you try to compile that?

You are mixing use of PCI_RESET_ALREADY_LOCKED and
PCI_RESET_ALREADYLOCKED, and I can't imagine both are valid.


Juergen

Re: [RFC PATCH v1 3/3] PCI: Hide pci_reset_function_locked()

[Sinan Kaya]

On 10/19/2018 4:12 AM, Juergen Gross wrote:
> On 19/10/2018 05:23, Sinan Kaya wrote:
>> It is time to hide pci_reset_function_locked() since pci_reset_function()
>> provides the same functionality. One less API to expose to the users.
>>
>> Signed-off-by: Sinan Kaya <okaya@kernel.org>
> 
> Did you try to compile that?
> 
> You are mixing use of PCI_RESET_ALREADY_LOCKED and
> PCI_RESET_ALREADYLOCKED, and I can't imagine both are valid.

You are right, I did compile. However, not all kernel options were
enabled apparently. I should have done allyesconfig.

Thanks for the heads up. I can fix it on the next rev.

> 
> 
> Juergen
>