[PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[Manivannan Sadhasivam]

Hello,

During the review of the patch that fixes DBI access in PCI EP, Rob
suggested [1] using a fixed interface for passing the events from EPC to
EPF instead of the in-kernel notifiers.

This series introduces a simple callback based mechanism for passing the
events from EPC to EPF. This interface is chosen for satisfying the below
requirements:

1. The notification has to reach the EPF drivers without any additional
latency.
2. The context of the caller (EPC) needs to be preserved while passing the
notifications.

With the existing notifier mechanism, the 1st case can be satisfied since
notifiers aren't adding any huge overhead. But the 2nd case is clearly not
satisfied, because the current atomic notifiers forces the EPF
notification context to be atomic even though the caller (EPC) may not be
in atomic context. In the notification function, the EPF drivers are
required to call several EPC APIs that might sleep and this triggers a
sleeping in atomic bug during runtime.

The above issue could be fixed by using a blocking notifier instead of
atomic, but that proposal was not accepted either [2].

So instead of working around the issues within the notifiers, let's get rid
of it and use the callback mechanism.

NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
on the real platforms is greatly appreciated.

Thanks,
Mani

[1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
[2] https://lore.kernel.org/all/20220228055240.24774-1-manivannan.sadhasivam@linaro.org

Changes in v4:

* Added check for the presence of event_ops before involing the callbacks (Kishon)
* Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
  handler of tegra194 driver (Vidya)
* Collected review tags

Changes in v3:

* As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
  call the LINK_UP callback in threaded IRQ handler.

Changes in v2:

* Introduced a new "list_lock" for protecting the epc->pci_epf list and
  used it in the callback mechanism.

Manivannan Sadhasivam (5):
  PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
  PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
  PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
  PCI: endpoint: Use callback mechanism for passing events from EPC to
    EPF
  PCI: endpoint: Use link_up() callback in place of LINK_UP notifier

 drivers/pci/controller/dwc/pci-dra7xx.c       |  2 +-
 drivers/pci/controller/dwc/pcie-tegra194.c    |  9 ++++-
 drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
 drivers/pci/endpoint/pci-epc-core.c           | 32 ++++++++++++----
 include/linux/pci-epc.h                       | 10 +----
 include/linux/pci-epf.h                       | 19 ++++++----
 6 files changed, 59 insertions(+), 51 deletions(-)

-- 
2.25.1

[PATCH v4 1/5] PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ

[Manivannan Sadhasivam]

The "dra7xx-pcie-main" hard IRQ handler is just printing the IRQ status
and calling the dw_pcie_ep_linkup() API if LINK_UP status is set. But the
execution of dw_pcie_ep_linkup() depends on the EPF driver and may take
more time depending on the EPF implementation.

In general, hard IRQ handlers are supposed to return quickly and not block
for so long. Moreover, there is no real need of the current IRQ handler to
be a hard IRQ handler. So switch to the threaded IRQ handler for the
"dra7xx-pcie-main" IRQ.

Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/controller/dwc/pci-dra7xx.c b/drivers/pci/controller/dwc/pci-dra7xx.c
index 38462ed11d07..4ae807e7cf79 100644
--- a/drivers/pci/controller/dwc/pci-dra7xx.c
+++ b/drivers/pci/controller/dwc/pci-dra7xx.c
@@ -840,7 +840,7 @@ static int dra7xx_pcie_probe(struct platform_device *pdev)
 	}
 	dra7xx->mode = mode;
 
-	ret = devm_request_irq(dev, irq, dra7xx_pcie_irq_handler,
+	ret = devm_request_threaded_irq(dev, irq, NULL, dra7xx_pcie_irq_handler,
 			       IRQF_SHARED, "dra7xx-pcie-main", dra7xx);
 	if (ret) {
 		dev_err(dev, "failed to request irq\n");
-- 
2.25.1

[PATCH v4 2/5] PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler

[Manivannan Sadhasivam]

dw_pcie_ep_linkup() may take more time to execute depending on the EPF
driver implementation. Calling this API in the hard IRQ handler is not
encouraged since the hard IRQ handlers are supposed to complete quickly.

So move the dw_pcie_ep_linkup() call to threaded IRQ handler.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/pcie-tegra194.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 1b6b437823d2..a0d231b7a435 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -287,6 +287,7 @@ struct tegra_pcie_dw {
 	struct gpio_desc *pex_refclk_sel_gpiod;
 	unsigned int pex_rst_irq;
 	int ep_state;
+	long link_status;
 };
 
 static inline struct tegra_pcie_dw *to_tegra_pcie(struct dw_pcie *pci)
@@ -450,9 +451,13 @@ static void pex_ep_event_hot_rst_done(struct tegra_pcie_dw *pcie)
 static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
 {
 	struct tegra_pcie_dw *pcie = arg;
+	struct dw_pcie_ep *ep = &pcie->pci.ep;
 	struct dw_pcie *pci = &pcie->pci;
 	u32 val, speed;
 
+	if (test_and_clear_bit(0, &pcie->link_status))
+		dw_pcie_ep_linkup(ep);
+
 	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
 		PCI_EXP_LNKSTA_CLS;
 	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
@@ -499,7 +504,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
 static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
 {
 	struct tegra_pcie_dw *pcie = arg;
-	struct dw_pcie_ep *ep = &pcie->pci.ep;
 	int spurious = 1;
 	u32 status_l0, status_l1, link_status;
 
@@ -515,7 +519,8 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
 			link_status = appl_readl(pcie, APPL_LINK_STATUS);
 			if (link_status & APPL_LINK_STATUS_RDLH_LINK_UP) {
 				dev_dbg(pcie->dev, "Link is up with Host\n");
-				dw_pcie_ep_linkup(ep);
+				set_bit(0, &pcie->link_status);
+				return IRQ_WAKE_THREAD;
 			}
 		}
 
-- 
2.25.1

[PATCH v4 3/5] PCI: endpoint: Use a separate lock for protecting epc->pci_epf list

[Manivannan Sadhasivam]

The EPC controller maintains a list of EPF drivers added to it. For
protecting this list against the concurrent accesses, the epc->lock
(used for protecting epc_ops) has been used so far. Since there were
no users trying to use epc_ops and modify the pci_epf list simultaneously,
this was not an issue.

But with the addition of callback mechanism for passing the events, this
will be a problem. Because the pci_epf list needs to be iterated first
for getting hold of the EPF driver and then the relevant event specific
callback needs to be called for the driver.

If the same epc->lock is used, then it will result in a deadlock scenario.

For instance,

...
	mutex_lock(&epc->lock);
	list_for_each_entry(epf, &epc->pci_epf, list) {
		epf->event_ops->core_init(epf);
		|
		|-> pci_epc_set_bar();
			|
			|-> mutex_lock(&epc->lock) # DEADLOCK
...

So to fix this issue, use a separate lock called "list_lock" for
protecting the pci_epf list against the concurrent accesses. This lock
will also be used by the callback mechanism.

Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/pci-epc-core.c | 9 +++++----
 include/linux/pci-epc.h             | 2 ++
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
index 3bc9273d0a08..6cce430d431b 100644
--- a/drivers/pci/endpoint/pci-epc-core.c
+++ b/drivers/pci/endpoint/pci-epc-core.c
@@ -613,7 +613,7 @@ int pci_epc_add_epf(struct pci_epc *epc, struct pci_epf *epf,
 	if (type == SECONDARY_INTERFACE && epf->sec_epc)
 		return -EBUSY;
 
-	mutex_lock(&epc->lock);
+	mutex_lock(&epc->list_lock);
 	func_no = find_first_zero_bit(&epc->function_num_map,
 				      BITS_PER_LONG);
 	if (func_no >= BITS_PER_LONG) {
@@ -640,7 +640,7 @@ int pci_epc_add_epf(struct pci_epc *epc, struct pci_epf *epf,
 
 	list_add_tail(list, &epc->pci_epf);
 ret:
-	mutex_unlock(&epc->lock);
+	mutex_unlock(&epc->list_lock);
 
 	return ret;
 }
@@ -672,11 +672,11 @@ void pci_epc_remove_epf(struct pci_epc *epc, struct pci_epf *epf,
 		list = &epf->sec_epc_list;
 	}
 
-	mutex_lock(&epc->lock);
+	mutex_lock(&epc->list_lock);
 	clear_bit(func_no, &epc->function_num_map);
 	list_del(list);
 	epf->epc = NULL;
-	mutex_unlock(&epc->lock);
+	mutex_unlock(&epc->list_lock);
 }
 EXPORT_SYMBOL_GPL(pci_epc_remove_epf);
 
@@ -773,6 +773,7 @@ __pci_epc_create(struct device *dev, const struct pci_epc_ops *ops,
 	}
 
 	mutex_init(&epc->lock);
+	mutex_init(&epc->list_lock);
 	INIT_LIST_HEAD(&epc->pci_epf);
 	ATOMIC_INIT_NOTIFIER_HEAD(&epc->notifier);
 
diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h
index a48778e1a4ee..fe729dfe509b 100644
--- a/include/linux/pci-epc.h
+++ b/include/linux/pci-epc.h
@@ -122,6 +122,7 @@ struct pci_epc_mem {
  * struct pci_epc - represents the PCI EPC device
  * @dev: PCI EPC device
  * @pci_epf: list of endpoint functions present in this EPC device
+ * list_lock: Mutex for protecting pci_epf list
  * @ops: function pointers for performing endpoint operations
  * @windows: array of address space of the endpoint controller
  * @mem: first window of the endpoint controller, which corresponds to
@@ -139,6 +140,7 @@ struct pci_epc_mem {
 struct pci_epc {
 	struct device			dev;
 	struct list_head		pci_epf;
+	struct mutex			list_lock;
 	const struct pci_epc_ops	*ops;
 	struct pci_epc_mem		**windows;
 	struct pci_epc_mem		*mem;
-- 
2.25.1

[PATCH v4 4/5] PCI: endpoint: Use callback mechanism for passing events from EPC to EPF

[Manivannan Sadhasivam]

Instead of using the notifiers for passing the events from EPC to EPF,
let's introduce a callback based mechanism where the EPF drivers can
populate relevant callbacks for EPC events they want to subscribe.

The use of notifiers in kernel is not recommended if there is a real link
between the sender and receiver, like in this case. Also, the existing
atomic notifier forces the notification functions to be in atomic context
while the caller may be in non-atomic context. For instance, the two
in-kernel users of the notifiers, pcie-qcom and pcie-tegra194, both are
calling the notifier functions in non-atomic context (from threaded IRQ
handlers). This creates a sleeping in atomic context issue with the
existing EPF_TEST driver that calls the EPC APIs that may sleep.

For all these reasons, let's get rid of the notifier chains and use the
simple callback mechanism for signalling the events from EPC to EPF
drivers. This preserves the context of the caller and avoids the latency
of going through a separate interface for triggering the notifications.

As a first step of the transition, the core_init() callback is introduced
in this commit, that'll replace the existing CORE_INIT notifier used for
signalling the init complete event from EPC.

During the occurrence of the event, EPC will go over the list of EPF
drivers attached to it and will call the core_init() callback if available.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/functions/pci-epf-test.c | 13 ++++++-------
 drivers/pci/endpoint/pci-epc-core.c           | 11 ++++++++++-
 include/linux/pci-epf.h                       | 11 ++++++++++-
 3 files changed, 26 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
index a6f906a96669..868de17e1ad2 100644
--- a/drivers/pci/endpoint/functions/pci-epf-test.c
+++ b/drivers/pci/endpoint/functions/pci-epf-test.c
@@ -826,20 +826,17 @@ static int pci_epf_test_core_init(struct pci_epf *epf)
 	return 0;
 }
 
+static const struct pci_epc_event_ops pci_epf_test_event_ops = {
+	.core_init = pci_epf_test_core_init,
+};
+
 static int pci_epf_test_notifier(struct notifier_block *nb, unsigned long val,
 				 void *data)
 {
 	struct pci_epf *epf = container_of(nb, struct pci_epf, nb);
 	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
-	int ret;
 
 	switch (val) {
-	case CORE_INIT:
-		ret = pci_epf_test_core_init(epf);
-		if (ret)
-			return NOTIFY_BAD;
-		break;
-
 	case LINK_UP:
 		queue_delayed_work(kpcitest_workqueue, &epf_test->cmd_handler,
 				   msecs_to_jiffies(1));
@@ -1010,6 +1007,8 @@ static int pci_epf_test_probe(struct pci_epf *epf, const struct pci_epf_device_i
 
 	INIT_DELAYED_WORK(&epf_test->cmd_handler, pci_epf_test_cmd_handler);
 
+	epf->event_ops = &pci_epf_test_event_ops;
+
 	epf_set_drvdata(epf, epf_test);
 	return 0;
 }
diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
index 6cce430d431b..8bb60528f530 100644
--- a/drivers/pci/endpoint/pci-epc-core.c
+++ b/drivers/pci/endpoint/pci-epc-core.c
@@ -707,10 +707,19 @@ EXPORT_SYMBOL_GPL(pci_epc_linkup);
  */
 void pci_epc_init_notify(struct pci_epc *epc)
 {
+	struct pci_epf *epf;
+
 	if (!epc || IS_ERR(epc))
 		return;
 
-	atomic_notifier_call_chain(&epc->notifier, CORE_INIT, NULL);
+	mutex_lock(&epc->list_lock);
+	list_for_each_entry(epf, &epc->pci_epf, list) {
+		mutex_lock(&epf->lock);
+		if (epf->event_ops && epf->event_ops->core_init)
+			epf->event_ops->core_init(epf);
+		mutex_unlock(&epf->lock);
+	}
+	mutex_unlock(&epc->list_lock);
 }
 EXPORT_SYMBOL_GPL(pci_epc_init_notify);
 
diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
index 0c94cc1513bc..a06f3b4c8bee 100644
--- a/include/linux/pci-epf.h
+++ b/include/linux/pci-epf.h
@@ -18,7 +18,6 @@ struct pci_epf;
 enum pci_epc_interface_type;
 
 enum pci_notify_event {
-	CORE_INIT,
 	LINK_UP,
 };
 
@@ -72,6 +71,14 @@ struct pci_epf_ops {
 					struct config_group *group);
 };
 
+/**
+ * struct pci_epf_event_ops - Callbacks for capturing the EPC events
+ * @core_init: Callback for the EPC initialization complete event
+ */
+struct pci_epc_event_ops {
+	int (*core_init)(struct pci_epf *epf);
+};
+
 /**
  * struct pci_epf_driver - represents the PCI EPF driver
  * @probe: ops to perform when a new EPF device has been bound to the EPF driver
@@ -140,6 +147,7 @@ struct pci_epf_bar {
  * @is_vf: true - virtual function, false - physical function
  * @vfunction_num_map: bitmap to manage virtual function number
  * @pci_vepf: list of virtual endpoint functions associated with this function
+ * @event_ops: Callbacks for capturing the EPC events
  */
 struct pci_epf {
 	struct device		dev;
@@ -170,6 +178,7 @@ struct pci_epf {
 	unsigned int		is_vf;
 	unsigned long		vfunction_num_map;
 	struct list_head	pci_vepf;
+	const struct pci_epc_event_ops *event_ops;
 };
 
 /**
-- 
2.25.1

[PATCH v4 5/5] PCI: endpoint: Use link_up() callback in place of LINK_UP notifier

[Manivannan Sadhasivam]

As a part of the transition towards callback mechanism for signalling the
events from EPC to EPF, let's use the link_up() callback in the place of
the LINK_UP notifier. This also removes the notifier support completely
from the PCI endpoint framework.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/functions/pci-epf-test.c | 33 ++++++-------------
 drivers/pci/endpoint/pci-epc-core.c           | 12 +++++--
 include/linux/pci-epc.h                       |  8 -----
 include/linux/pci-epf.h                       |  8 ++---
 4 files changed, 22 insertions(+), 39 deletions(-)

diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
index 868de17e1ad2..f75045f2dee3 100644
--- a/drivers/pci/endpoint/functions/pci-epf-test.c
+++ b/drivers/pci/endpoint/functions/pci-epf-test.c
@@ -826,30 +826,21 @@ static int pci_epf_test_core_init(struct pci_epf *epf)
 	return 0;
 }
 
-static const struct pci_epc_event_ops pci_epf_test_event_ops = {
-	.core_init = pci_epf_test_core_init,
-};
-
-static int pci_epf_test_notifier(struct notifier_block *nb, unsigned long val,
-				 void *data)
+int pci_epf_test_link_up(struct pci_epf *epf)
 {
-	struct pci_epf *epf = container_of(nb, struct pci_epf, nb);
 	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
 
-	switch (val) {
-	case LINK_UP:
-		queue_delayed_work(kpcitest_workqueue, &epf_test->cmd_handler,
-				   msecs_to_jiffies(1));
-		break;
-
-	default:
-		dev_err(&epf->dev, "Invalid EPF test notifier event\n");
-		return NOTIFY_BAD;
-	}
+	queue_delayed_work(kpcitest_workqueue, &epf_test->cmd_handler,
+			   msecs_to_jiffies(1));
 
-	return NOTIFY_OK;
+	return 0;
 }
 
+static const struct pci_epc_event_ops pci_epf_test_event_ops = {
+	.core_init = pci_epf_test_core_init,
+	.link_up = pci_epf_test_link_up,
+};
+
 static int pci_epf_test_alloc_space(struct pci_epf *epf)
 {
 	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
@@ -976,12 +967,8 @@ static int pci_epf_test_bind(struct pci_epf *epf)
 	if (ret)
 		epf_test->dma_supported = false;
 
-	if (linkup_notifier || core_init_notifier) {
-		epf->nb.notifier_call = pci_epf_test_notifier;
-		pci_epc_register_notifier(epc, &epf->nb);
-	} else {
+	if (!linkup_notifier && !core_init_notifier)
 		queue_work(kpcitest_workqueue, &epf_test->cmd_handler.work);
-	}
 
 	return 0;
 }
diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
index 8bb60528f530..c0954fddcc14 100644
--- a/drivers/pci/endpoint/pci-epc-core.c
+++ b/drivers/pci/endpoint/pci-epc-core.c
@@ -690,10 +690,19 @@ EXPORT_SYMBOL_GPL(pci_epc_remove_epf);
  */
 void pci_epc_linkup(struct pci_epc *epc)
 {
+	struct pci_epf *epf;
+
 	if (!epc || IS_ERR(epc))
 		return;
 
-	atomic_notifier_call_chain(&epc->notifier, LINK_UP, NULL);
+	mutex_lock(&epc->list_lock);
+	list_for_each_entry(epf, &epc->pci_epf, list) {
+		mutex_lock(&epf->lock);
+		if (epf->event_ops && epf->event_ops->link_up)
+			epf->event_ops->link_up(epf);
+		mutex_unlock(&epf->lock);
+	}
+	mutex_unlock(&epc->list_lock);
 }
 EXPORT_SYMBOL_GPL(pci_epc_linkup);
 
@@ -784,7 +793,6 @@ __pci_epc_create(struct device *dev, const struct pci_epc_ops *ops,
 	mutex_init(&epc->lock);
 	mutex_init(&epc->list_lock);
 	INIT_LIST_HEAD(&epc->pci_epf);
-	ATOMIC_INIT_NOTIFIER_HEAD(&epc->notifier);
 
 	device_initialize(&epc->dev);
 	epc->dev.class = pci_epc_class;
diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h
index fe729dfe509b..301bb0e53707 100644
--- a/include/linux/pci-epc.h
+++ b/include/linux/pci-epc.h
@@ -135,7 +135,6 @@ struct pci_epc_mem {
  * @group: configfs group representing the PCI EPC device
  * @lock: mutex to protect pci_epc ops
  * @function_num_map: bitmap to manage physical function number
- * @notifier: used to notify EPF of any EPC events (like linkup)
  */
 struct pci_epc {
 	struct device			dev;
@@ -151,7 +150,6 @@ struct pci_epc {
 	/* mutex to protect against concurrent access of EP controller */
 	struct mutex			lock;
 	unsigned long			function_num_map;
-	struct atomic_notifier_head	notifier;
 };
 
 /**
@@ -194,12 +192,6 @@ static inline void *epc_get_drvdata(struct pci_epc *epc)
 	return dev_get_drvdata(&epc->dev);
 }
 
-static inline int
-pci_epc_register_notifier(struct pci_epc *epc, struct notifier_block *nb)
-{
-	return atomic_notifier_chain_register(&epc->notifier, nb);
-}
-
 struct pci_epc *
 __devm_pci_epc_create(struct device *dev, const struct pci_epc_ops *ops,
 		      struct module *owner);
diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
index a06f3b4c8bee..bc613f0df7e3 100644
--- a/include/linux/pci-epf.h
+++ b/include/linux/pci-epf.h
@@ -17,10 +17,6 @@
 struct pci_epf;
 enum pci_epc_interface_type;
 
-enum pci_notify_event {
-	LINK_UP,
-};
-
 enum pci_barno {
 	NO_BAR = -1,
 	BAR_0,
@@ -74,9 +70,11 @@ struct pci_epf_ops {
 /**
  * struct pci_epf_event_ops - Callbacks for capturing the EPC events
  * @core_init: Callback for the EPC initialization complete event
+ * @link_up: Callback for the EPC link up event
  */
 struct pci_epc_event_ops {
 	int (*core_init)(struct pci_epf *epf);
+	int (*link_up)(struct pci_epf *epf);
 };
 
 /**
@@ -135,7 +133,6 @@ struct pci_epf_bar {
  * @driver: the EPF driver to which this EPF device is bound
  * @id: Pointer to the EPF device ID
  * @list: to add pci_epf as a list of PCI endpoint functions to pci_epc
- * @nb: notifier block to notify EPF of any EPC events (like linkup)
  * @lock: mutex to protect pci_epf_ops
  * @sec_epc: the secondary EPC device to which this EPF device is bound
  * @sec_epc_list: to add pci_epf as list of PCI endpoint functions to secondary
@@ -164,7 +161,6 @@ struct pci_epf {
 	struct pci_epf_driver	*driver;
 	const struct pci_epf_device_id *id;
 	struct list_head	list;
-	struct notifier_block   nb;
 	/* mutex to protect against concurrent access of pci_epf_ops */
 	struct mutex		lock;
 
-- 
2.25.1

Re: [PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[Manivannan Sadhasivam]

Hi Kishon,

On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> Hello,
> 
> During the review of the patch that fixes DBI access in PCI EP, Rob
> suggested [1] using a fixed interface for passing the events from EPC to
> EPF instead of the in-kernel notifiers.
> 
> This series introduces a simple callback based mechanism for passing the
> events from EPC to EPF. This interface is chosen for satisfying the below
> requirements:
> 
> 1. The notification has to reach the EPF drivers without any additional
> latency.
> 2. The context of the caller (EPC) needs to be preserved while passing the
> notifications.
> 
> With the existing notifier mechanism, the 1st case can be satisfied since
> notifiers aren't adding any huge overhead. But the 2nd case is clearly not
> satisfied, because the current atomic notifiers forces the EPF
> notification context to be atomic even though the caller (EPC) may not be
> in atomic context. In the notification function, the EPF drivers are
> required to call several EPC APIs that might sleep and this triggers a
> sleeping in atomic bug during runtime.
> 
> The above issue could be fixed by using a blocking notifier instead of
> atomic, but that proposal was not accepted either [2].
> 
> So instead of working around the issues within the notifiers, let's get rid
> of it and use the callback mechanism.
> 
> NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
> on the real platforms is greatly appreciated.
> 

Any feedback on this series?

Thanks,
Mani

> Thanks,
> Mani
> 
> [1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
> [2] https://lore.kernel.org/all/20220228055240.24774-1-manivannan.sadhasivam@linaro.org
> 
> Changes in v4:
> 
> * Added check for the presence of event_ops before involing the callbacks (Kishon)
> * Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
>   handler of tegra194 driver (Vidya)
> * Collected review tags
> 
> Changes in v3:
> 
> * As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
>   call the LINK_UP callback in threaded IRQ handler.
> 
> Changes in v2:
> 
> * Introduced a new "list_lock" for protecting the epc->pci_epf list and
>   used it in the callback mechanism.
> 
> Manivannan Sadhasivam (5):
>   PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
>   PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
>   PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
>   PCI: endpoint: Use callback mechanism for passing events from EPC to
>     EPF
>   PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> 
>  drivers/pci/controller/dwc/pci-dra7xx.c       |  2 +-
>  drivers/pci/controller/dwc/pcie-tegra194.c    |  9 ++++-
>  drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
>  drivers/pci/endpoint/pci-epc-core.c           | 32 ++++++++++++----
>  include/linux/pci-epc.h                       | 10 +----
>  include/linux/pci-epf.h                       | 19 ++++++----
>  6 files changed, 59 insertions(+), 51 deletions(-)
> 
> -- 
> 2.25.1
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[Bjorn Helgaas]

On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> Hello,
> 
> During the review of the patch that fixes DBI access in PCI EP, Rob
> suggested [1] using a fixed interface for passing the events from EPC to
> EPF instead of the in-kernel notifiers.

> Manivannan Sadhasivam (5):
>   PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
>   PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
>   PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
>   PCI: endpoint: Use callback mechanism for passing events from EPC to
>     EPF
>   PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> 
>  drivers/pci/controller/dwc/pci-dra7xx.c       |  2 +-
>  drivers/pci/controller/dwc/pcie-tegra194.c    |  9 ++++-
>  drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
>  drivers/pci/endpoint/pci-epc-core.c           | 32 ++++++++++++----
>  include/linux/pci-epc.h                       | 10 +----
>  include/linux/pci-epf.h                       | 19 ++++++----
>  6 files changed, 59 insertions(+), 51 deletions(-)

Doesn't apply cleanly on v6.1-rc1.  Does it depend on something else?

Re: [PATCH v4 4/5] PCI: endpoint: Use callback mechanism for passing events from EPC to EPF

[Kishon Vijay Abraham I]

On 10/25/2022 8:21 PM, Manivannan Sadhasivam wrote:
> Instead of using the notifiers for passing the events from EPC to EPF,
> let's introduce a callback based mechanism where the EPF drivers can
> populate relevant callbacks for EPC events they want to subscribe.
> 
> The use of notifiers in kernel is not recommended if there is a real link
> between the sender and receiver, like in this case. Also, the existing
> atomic notifier forces the notification functions to be in atomic context
> while the caller may be in non-atomic context. For instance, the two
> in-kernel users of the notifiers, pcie-qcom and pcie-tegra194, both are
> calling the notifier functions in non-atomic context (from threaded IRQ
> handlers). This creates a sleeping in atomic context issue with the
> existing EPF_TEST driver that calls the EPC APIs that may sleep.
> 
> For all these reasons, let's get rid of the notifier chains and use the
> simple callback mechanism for signalling the events from EPC to EPF
> drivers. This preserves the context of the caller and avoids the latency
> of going through a separate interface for triggering the notifications.
> 
> As a first step of the transition, the core_init() callback is introduced
> in this commit, that'll replace the existing CORE_INIT notifier used for
> signalling the init complete event from EPC.
> 
> During the occurrence of the event, EPC will go over the list of EPF
> drivers attached to it and will call the core_init() callback if available.
> 
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Acked-by: Kishon Vijay Abraham I <kishon@kernel.org>
> ---
>  drivers/pci/endpoint/functions/pci-epf-test.c | 13 ++++++-------
>  drivers/pci/endpoint/pci-epc-core.c           | 11 ++++++++++-
>  include/linux/pci-epf.h                       | 11 ++++++++++-
>  3 files changed, 26 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
> index a6f906a96669..868de17e1ad2 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-test.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-test.c
> @@ -826,20 +826,17 @@ static int pci_epf_test_core_init(struct pci_epf *epf)
>  	return 0;
>  }
>  
> +static const struct pci_epc_event_ops pci_epf_test_event_ops = {
> +	.core_init = pci_epf_test_core_init,
> +};
> +
>  static int pci_epf_test_notifier(struct notifier_block *nb, unsigned long val,
>  				 void *data)
>  {
>  	struct pci_epf *epf = container_of(nb, struct pci_epf, nb);
>  	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
> -	int ret;
>  
>  	switch (val) {
> -	case CORE_INIT:
> -		ret = pci_epf_test_core_init(epf);
> -		if (ret)
> -			return NOTIFY_BAD;
> -		break;
> -
>  	case LINK_UP:
>  		queue_delayed_work(kpcitest_workqueue, &epf_test->cmd_handler,
>  				   msecs_to_jiffies(1));
> @@ -1010,6 +1007,8 @@ static int pci_epf_test_probe(struct pci_epf *epf, const struct pci_epf_device_i
>  
>  	INIT_DELAYED_WORK(&epf_test->cmd_handler, pci_epf_test_cmd_handler);
>  
> +	epf->event_ops = &pci_epf_test_event_ops;
> +
>  	epf_set_drvdata(epf, epf_test);
>  	return 0;
>  }
> diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
> index 6cce430d431b..8bb60528f530 100644
> --- a/drivers/pci/endpoint/pci-epc-core.c
> +++ b/drivers/pci/endpoint/pci-epc-core.c
> @@ -707,10 +707,19 @@ EXPORT_SYMBOL_GPL(pci_epc_linkup);
>   */
>  void pci_epc_init_notify(struct pci_epc *epc)
>  {
> +	struct pci_epf *epf;
> +
>  	if (!epc || IS_ERR(epc))
>  		return;
>  
> -	atomic_notifier_call_chain(&epc->notifier, CORE_INIT, NULL);
> +	mutex_lock(&epc->list_lock);
> +	list_for_each_entry(epf, &epc->pci_epf, list) {
> +		mutex_lock(&epf->lock);
> +		if (epf->event_ops && epf->event_ops->core_init)
> +			epf->event_ops->core_init(epf);
> +		mutex_unlock(&epf->lock);
> +	}
> +	mutex_unlock(&epc->list_lock);
>  }
>  EXPORT_SYMBOL_GPL(pci_epc_init_notify);
>  
> diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
> index 0c94cc1513bc..a06f3b4c8bee 100644
> --- a/include/linux/pci-epf.h
> +++ b/include/linux/pci-epf.h
> @@ -18,7 +18,6 @@ struct pci_epf;
>  enum pci_epc_interface_type;
>  
>  enum pci_notify_event {
> -	CORE_INIT,
>  	LINK_UP,
>  };
>  
> @@ -72,6 +71,14 @@ struct pci_epf_ops {
>  					struct config_group *group);
>  };
>  
> +/**
> + * struct pci_epf_event_ops - Callbacks for capturing the EPC events
> + * @core_init: Callback for the EPC initialization complete event
> + */
> +struct pci_epc_event_ops {
> +	int (*core_init)(struct pci_epf *epf);
> +};
> +
>  /**
>   * struct pci_epf_driver - represents the PCI EPF driver
>   * @probe: ops to perform when a new EPF device has been bound to the EPF driver
> @@ -140,6 +147,7 @@ struct pci_epf_bar {
>   * @is_vf: true - virtual function, false - physical function
>   * @vfunction_num_map: bitmap to manage virtual function number
>   * @pci_vepf: list of virtual endpoint functions associated with this function
> + * @event_ops: Callbacks for capturing the EPC events
>   */
>  struct pci_epf {
>  	struct device		dev;
> @@ -170,6 +178,7 @@ struct pci_epf {
>  	unsigned int		is_vf;
>  	unsigned long		vfunction_num_map;
>  	struct list_head	pci_vepf;
> +	const struct pci_epc_event_ops *event_ops;
>  };
>  
>  /**

Re: [PATCH v4 5/5] PCI: endpoint: Use link_up() callback in place of LINK_UP notifier

[Kishon Vijay Abraham I]

On 10/25/2022 8:21 PM, Manivannan Sadhasivam wrote:
> As a part of the transition towards callback mechanism for signalling the
> events from EPC to EPF, let's use the link_up() callback in the place of
> the LINK_UP notifier. This also removes the notifier support completely
> from the PCI endpoint framework.
> 
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Acked-by: Kishon Vijay Abraham I <kishon@kernel.org>
> ---
>  drivers/pci/endpoint/functions/pci-epf-test.c | 33 ++++++-------------
>  drivers/pci/endpoint/pci-epc-core.c           | 12 +++++--
>  include/linux/pci-epc.h                       |  8 -----
>  include/linux/pci-epf.h                       |  8 ++---
>  4 files changed, 22 insertions(+), 39 deletions(-)
> 
> diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
> index 868de17e1ad2..f75045f2dee3 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-test.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-test.c
> @@ -826,30 +826,21 @@ static int pci_epf_test_core_init(struct pci_epf *epf)
>  	return 0;
>  }
>  
> -static const struct pci_epc_event_ops pci_epf_test_event_ops = {
> -	.core_init = pci_epf_test_core_init,
> -};
> -
> -static int pci_epf_test_notifier(struct notifier_block *nb, unsigned long val,
> -				 void *data)
> +int pci_epf_test_link_up(struct pci_epf *epf)
>  {
> -	struct pci_epf *epf = container_of(nb, struct pci_epf, nb);
>  	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
>  
> -	switch (val) {
> -	case LINK_UP:
> -		queue_delayed_work(kpcitest_workqueue, &epf_test->cmd_handler,
> -				   msecs_to_jiffies(1));
> -		break;
> -
> -	default:
> -		dev_err(&epf->dev, "Invalid EPF test notifier event\n");
> -		return NOTIFY_BAD;
> -	}
> +	queue_delayed_work(kpcitest_workqueue, &epf_test->cmd_handler,
> +			   msecs_to_jiffies(1));
>  
> -	return NOTIFY_OK;
> +	return 0;
>  }
>  
> +static const struct pci_epc_event_ops pci_epf_test_event_ops = {
> +	.core_init = pci_epf_test_core_init,
> +	.link_up = pci_epf_test_link_up,
> +};
> +
>  static int pci_epf_test_alloc_space(struct pci_epf *epf)
>  {
>  	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
> @@ -976,12 +967,8 @@ static int pci_epf_test_bind(struct pci_epf *epf)
>  	if (ret)
>  		epf_test->dma_supported = false;
>  
> -	if (linkup_notifier || core_init_notifier) {
> -		epf->nb.notifier_call = pci_epf_test_notifier;
> -		pci_epc_register_notifier(epc, &epf->nb);
> -	} else {
> +	if (!linkup_notifier && !core_init_notifier)
>  		queue_work(kpcitest_workqueue, &epf_test->cmd_handler.work);
> -	}
>  
>  	return 0;
>  }
> diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
> index 8bb60528f530..c0954fddcc14 100644
> --- a/drivers/pci/endpoint/pci-epc-core.c
> +++ b/drivers/pci/endpoint/pci-epc-core.c
> @@ -690,10 +690,19 @@ EXPORT_SYMBOL_GPL(pci_epc_remove_epf);
>   */
>  void pci_epc_linkup(struct pci_epc *epc)
>  {
> +	struct pci_epf *epf;
> +
>  	if (!epc || IS_ERR(epc))
>  		return;
>  
> -	atomic_notifier_call_chain(&epc->notifier, LINK_UP, NULL);
> +	mutex_lock(&epc->list_lock);
> +	list_for_each_entry(epf, &epc->pci_epf, list) {
> +		mutex_lock(&epf->lock);
> +		if (epf->event_ops && epf->event_ops->link_up)
> +			epf->event_ops->link_up(epf);
> +		mutex_unlock(&epf->lock);
> +	}
> +	mutex_unlock(&epc->list_lock);
>  }
>  EXPORT_SYMBOL_GPL(pci_epc_linkup);
>  
> @@ -784,7 +793,6 @@ __pci_epc_create(struct device *dev, const struct pci_epc_ops *ops,
>  	mutex_init(&epc->lock);
>  	mutex_init(&epc->list_lock);
>  	INIT_LIST_HEAD(&epc->pci_epf);
> -	ATOMIC_INIT_NOTIFIER_HEAD(&epc->notifier);
>  
>  	device_initialize(&epc->dev);
>  	epc->dev.class = pci_epc_class;
> diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h
> index fe729dfe509b..301bb0e53707 100644
> --- a/include/linux/pci-epc.h
> +++ b/include/linux/pci-epc.h
> @@ -135,7 +135,6 @@ struct pci_epc_mem {
>   * @group: configfs group representing the PCI EPC device
>   * @lock: mutex to protect pci_epc ops
>   * @function_num_map: bitmap to manage physical function number
> - * @notifier: used to notify EPF of any EPC events (like linkup)
>   */
>  struct pci_epc {
>  	struct device			dev;
> @@ -151,7 +150,6 @@ struct pci_epc {
>  	/* mutex to protect against concurrent access of EP controller */
>  	struct mutex			lock;
>  	unsigned long			function_num_map;
> -	struct atomic_notifier_head	notifier;
>  };
>  
>  /**
> @@ -194,12 +192,6 @@ static inline void *epc_get_drvdata(struct pci_epc *epc)
>  	return dev_get_drvdata(&epc->dev);
>  }
>  
> -static inline int
> -pci_epc_register_notifier(struct pci_epc *epc, struct notifier_block *nb)
> -{
> -	return atomic_notifier_chain_register(&epc->notifier, nb);
> -}
> -
>  struct pci_epc *
>  __devm_pci_epc_create(struct device *dev, const struct pci_epc_ops *ops,
>  		      struct module *owner);
> diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
> index a06f3b4c8bee..bc613f0df7e3 100644
> --- a/include/linux/pci-epf.h
> +++ b/include/linux/pci-epf.h
> @@ -17,10 +17,6 @@
>  struct pci_epf;
>  enum pci_epc_interface_type;
>  
> -enum pci_notify_event {
> -	LINK_UP,
> -};
> -
>  enum pci_barno {
>  	NO_BAR = -1,
>  	BAR_0,
> @@ -74,9 +70,11 @@ struct pci_epf_ops {
>  /**
>   * struct pci_epf_event_ops - Callbacks for capturing the EPC events
>   * @core_init: Callback for the EPC initialization complete event
> + * @link_up: Callback for the EPC link up event
>   */
>  struct pci_epc_event_ops {
>  	int (*core_init)(struct pci_epf *epf);
> +	int (*link_up)(struct pci_epf *epf);
>  };
>  
>  /**
> @@ -135,7 +133,6 @@ struct pci_epf_bar {
>   * @driver: the EPF driver to which this EPF device is bound
>   * @id: Pointer to the EPF device ID
>   * @list: to add pci_epf as a list of PCI endpoint functions to pci_epc
> - * @nb: notifier block to notify EPF of any EPC events (like linkup)
>   * @lock: mutex to protect pci_epf_ops
>   * @sec_epc: the secondary EPC device to which this EPF device is bound
>   * @sec_epc_list: to add pci_epf as list of PCI endpoint functions to secondary
> @@ -164,7 +161,6 @@ struct pci_epf {
>  	struct pci_epf_driver	*driver;
>  	const struct pci_epf_device_id *id;
>  	struct list_head	list;
> -	struct notifier_block   nb;
>  	/* mutex to protect against concurrent access of pci_epf_ops */
>  	struct mutex		lock;
>  

Re: [PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[Manivannan Sadhasivam]

On Mon, Nov 07, 2022 at 02:28:53PM -0600, Bjorn Helgaas wrote:
> On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> > Hello,
> > 
> > During the review of the patch that fixes DBI access in PCI EP, Rob
> > suggested [1] using a fixed interface for passing the events from EPC to
> > EPF instead of the in-kernel notifiers.
> 
> > Manivannan Sadhasivam (5):
> >   PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> >   PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> >   PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> >   PCI: endpoint: Use callback mechanism for passing events from EPC to
> >     EPF
> >   PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> > 
> >  drivers/pci/controller/dwc/pci-dra7xx.c       |  2 +-
> >  drivers/pci/controller/dwc/pcie-tegra194.c    |  9 ++++-
> >  drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> >  drivers/pci/endpoint/pci-epc-core.c           | 32 ++++++++++++----
> >  include/linux/pci-epc.h                       | 10 +----
> >  include/linux/pci-epf.h                       | 19 ++++++----
> >  6 files changed, 59 insertions(+), 51 deletions(-)
> 
> Doesn't apply cleanly on v6.1-rc1.  Does it depend on something else?

Yes, this patch:
https://lore.kernel.org/linux-pci/20220825090101.20474-1-hayashi.kunihiko@socionext.com/

Since this patch is already merged by Lorenzo, I based this series on top of
that. If that's not required, I can send a new version without that patch.

Let me know.

Thanks,
Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[Bjorn Helgaas]

On Tue, Nov 08, 2022 at 05:44:40PM +0530, Manivannan Sadhasivam wrote:
> On Mon, Nov 07, 2022 at 02:28:53PM -0600, Bjorn Helgaas wrote:
> > On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> > > Hello,
> > > 
> > > During the review of the patch that fixes DBI access in PCI EP, Rob
> > > suggested [1] using a fixed interface for passing the events from EPC to
> > > EPF instead of the in-kernel notifiers.
> > 
> > > Manivannan Sadhasivam (5):
> > >   PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> > >   PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> > >   PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> > >   PCI: endpoint: Use callback mechanism for passing events from EPC to
> > >     EPF
> > >   PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> > > 
> > >  drivers/pci/controller/dwc/pci-dra7xx.c       |  2 +-
> > >  drivers/pci/controller/dwc/pcie-tegra194.c    |  9 ++++-
> > >  drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> > >  drivers/pci/endpoint/pci-epc-core.c           | 32 ++++++++++++----
> > >  include/linux/pci-epc.h                       | 10 +----
> > >  include/linux/pci-epf.h                       | 19 ++++++----
> > >  6 files changed, 59 insertions(+), 51 deletions(-)
> > 
> > Doesn't apply cleanly on v6.1-rc1.  Does it depend on something else?
> 
> Yes, this patch:
> https://lore.kernel.org/linux-pci/20220825090101.20474-1-hayashi.kunihiko@socionext.com/
> 
> Since this patch is already merged by Lorenzo, I based this series on top of
> that. If that's not required, I can send a new version without that patch.

I think it's fine as-is.  

I tried applying it on both v6.1-rc1 and my current "next" branch.
Both failed because I haven't merged Lorenzo's branch into "next" yet.
As long as Lorenzo merges this on the correct branch, there's no
problem.  

Mentioning the dependency or what the patch is based on in the cover
letter is the easiest way to make this smoother.

Bjorn

Re: [PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[Manivannan Sadhasivam]

On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> Hello,
> 
> During the review of the patch that fixes DBI access in PCI EP, Rob
> suggested [1] using a fixed interface for passing the events from EPC to
> EPF instead of the in-kernel notifiers.
> 
> This series introduces a simple callback based mechanism for passing the
> events from EPC to EPF. This interface is chosen for satisfying the below
> requirements:
> 
> 1. The notification has to reach the EPF drivers without any additional
> latency.
> 2. The context of the caller (EPC) needs to be preserved while passing the
> notifications.
> 
> With the existing notifier mechanism, the 1st case can be satisfied since
> notifiers aren't adding any huge overhead. But the 2nd case is clearly not
> satisfied, because the current atomic notifiers forces the EPF
> notification context to be atomic even though the caller (EPC) may not be
> in atomic context. In the notification function, the EPF drivers are
> required to call several EPC APIs that might sleep and this triggers a
> sleeping in atomic bug during runtime.
> 
> The above issue could be fixed by using a blocking notifier instead of
> atomic, but that proposal was not accepted either [2].
> 
> So instead of working around the issues within the notifiers, let's get rid
> of it and use the callback mechanism.
> 
> NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
> on the real platforms is greatly appreciated.
> 

Lorenzo, can this series be merged for v6.2 since all the patches are reviewed
now?

Thanks,
Mani

> Thanks,
> Mani
> 
> [1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
> [2] https://lore.kernel.org/all/20220228055240.24774-1-manivannan.sadhasivam@linaro.org
> 
> Changes in v4:
> 
> * Added check for the presence of event_ops before involing the callbacks (Kishon)
> * Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
>   handler of tegra194 driver (Vidya)
> * Collected review tags
> 
> Changes in v3:
> 
> * As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
>   call the LINK_UP callback in threaded IRQ handler.
> 
> Changes in v2:
> 
> * Introduced a new "list_lock" for protecting the epc->pci_epf list and
>   used it in the callback mechanism.
> 
> Manivannan Sadhasivam (5):
>   PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
>   PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
>   PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
>   PCI: endpoint: Use callback mechanism for passing events from EPC to
>     EPF
>   PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> 
>  drivers/pci/controller/dwc/pci-dra7xx.c       |  2 +-
>  drivers/pci/controller/dwc/pcie-tegra194.c    |  9 ++++-
>  drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
>  drivers/pci/endpoint/pci-epc-core.c           | 32 ++++++++++++----
>  include/linux/pci-epc.h                       | 10 +----
>  include/linux/pci-epf.h                       | 19 ++++++----
>  6 files changed, 59 insertions(+), 51 deletions(-)
> 
> -- 
> 2.25.1
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[Lorenzo Pieralisi]

On Mon, Nov 14, 2022 at 01:03:16PM +0530, Manivannan Sadhasivam wrote:
> On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> > Hello,
> > 
> > During the review of the patch that fixes DBI access in PCI EP, Rob
> > suggested [1] using a fixed interface for passing the events from EPC to
> > EPF instead of the in-kernel notifiers.
> > 
> > This series introduces a simple callback based mechanism for passing the
> > events from EPC to EPF. This interface is chosen for satisfying the below
> > requirements:
> > 
> > 1. The notification has to reach the EPF drivers without any additional
> > latency.
> > 2. The context of the caller (EPC) needs to be preserved while passing the
> > notifications.
> > 
> > With the existing notifier mechanism, the 1st case can be satisfied since
> > notifiers aren't adding any huge overhead. But the 2nd case is clearly not
> > satisfied, because the current atomic notifiers forces the EPF
> > notification context to be atomic even though the caller (EPC) may not be
> > in atomic context. In the notification function, the EPF drivers are
> > required to call several EPC APIs that might sleep and this triggers a
> > sleeping in atomic bug during runtime.
> > 
> > The above issue could be fixed by using a blocking notifier instead of
> > atomic, but that proposal was not accepted either [2].
> > 
> > So instead of working around the issues within the notifiers, let's get rid
> > of it and use the callback mechanism.
> > 
> > NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
> > on the real platforms is greatly appreciated.
> > 
> 
> Lorenzo, can this series be merged for v6.2 since all the patches are reviewed
> now?

Patch (2) isn't (or I missed something) - we should be looking for
review/testing on it.

Thanks,
Lorenzo

> Thanks,
> Mani
> 
> > Thanks,
> > Mani
> > 
> > [1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
> > [2] https://lore.kernel.org/all/20220228055240.24774-1-manivannan.sadhasivam@linaro.org
> > 
> > Changes in v4:
> > 
> > * Added check for the presence of event_ops before involing the callbacks (Kishon)
> > * Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
> >   handler of tegra194 driver (Vidya)
> > * Collected review tags
> > 
> > Changes in v3:
> > 
> > * As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
> >   call the LINK_UP callback in threaded IRQ handler.
> > 
> > Changes in v2:
> > 
> > * Introduced a new "list_lock" for protecting the epc->pci_epf list and
> >   used it in the callback mechanism.
> > 
> > Manivannan Sadhasivam (5):
> >   PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> >   PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> >   PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> >   PCI: endpoint: Use callback mechanism for passing events from EPC to
> >     EPF
> >   PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> > 
> >  drivers/pci/controller/dwc/pci-dra7xx.c       |  2 +-
> >  drivers/pci/controller/dwc/pcie-tegra194.c    |  9 ++++-
> >  drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> >  drivers/pci/endpoint/pci-epc-core.c           | 32 ++++++++++++----
> >  include/linux/pci-epc.h                       | 10 +----
> >  include/linux/pci-epf.h                       | 19 ++++++----
> >  6 files changed, 59 insertions(+), 51 deletions(-)
> > 
> > -- 
> > 2.25.1
> > 
> 
> -- 
> மணிவண்ணன் சதாசிவம்

Re: [PATCH v4 0/5] PCI: endpoint: Rework the EPC to EPF notification

[Manivannan Sadhasivam]

On Mon, Nov 14, 2022 at 11:05:44AM +0100, Lorenzo Pieralisi wrote:
> On Mon, Nov 14, 2022 at 01:03:16PM +0530, Manivannan Sadhasivam wrote:
> > On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> > > Hello,
> > > 
> > > During the review of the patch that fixes DBI access in PCI EP, Rob
> > > suggested [1] using a fixed interface for passing the events from EPC to
> > > EPF instead of the in-kernel notifiers.
> > > 
> > > This series introduces a simple callback based mechanism for passing the
> > > events from EPC to EPF. This interface is chosen for satisfying the below
> > > requirements:
> > > 
> > > 1. The notification has to reach the EPF drivers without any additional
> > > latency.
> > > 2. The context of the caller (EPC) needs to be preserved while passing the
> > > notifications.
> > > 
> > > With the existing notifier mechanism, the 1st case can be satisfied since
> > > notifiers aren't adding any huge overhead. But the 2nd case is clearly not
> > > satisfied, because the current atomic notifiers forces the EPF
> > > notification context to be atomic even though the caller (EPC) may not be
> > > in atomic context. In the notification function, the EPF drivers are
> > > required to call several EPC APIs that might sleep and this triggers a
> > > sleeping in atomic bug during runtime.
> > > 
> > > The above issue could be fixed by using a blocking notifier instead of
> > > atomic, but that proposal was not accepted either [2].
> > > 
> > > So instead of working around the issues within the notifiers, let's get rid
> > > of it and use the callback mechanism.
> > > 
> > > NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
> > > on the real platforms is greatly appreciated.
> > > 
> > 
> > Lorenzo, can this series be merged for v6.2 since all the patches are reviewed
> > now?
> 
> Patch (2) isn't (or I missed something) - we should be looking for
> review/testing on it.
> 

Yes, 2/5 doesn't have a review tag yet. But as per comments from Vidya on v3
[1], I believe it is okay to get merged.

But I'll ping on that patch anyway.

Thanks,
Mani

[1] https://lore.kernel.org/lkml/5ec4b46f-2590-bd34-f6fa-e4e2eeb38b7b@nvidia.com/

> Thanks,
> Lorenzo
> 
> > Thanks,
> > Mani
> > 
> > > Thanks,
> > > Mani
> > > 
> > > [1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
> > > [2] https://lore.kernel.org/all/20220228055240.24774-1-manivannan.sadhasivam@linaro.org
> > > 
> > > Changes in v4:
> > > 
> > > * Added check for the presence of event_ops before involing the callbacks (Kishon)
> > > * Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
> > >   handler of tegra194 driver (Vidya)
> > > * Collected review tags
> > > 
> > > Changes in v3:
> > > 
> > > * As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
> > >   call the LINK_UP callback in threaded IRQ handler.
> > > 
> > > Changes in v2:
> > > 
> > > * Introduced a new "list_lock" for protecting the epc->pci_epf list and
> > >   used it in the callback mechanism.
> > > 
> > > Manivannan Sadhasivam (5):
> > >   PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> > >   PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> > >   PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> > >   PCI: endpoint: Use callback mechanism for passing events from EPC to
> > >     EPF
> > >   PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> > > 
> > >  drivers/pci/controller/dwc/pci-dra7xx.c       |  2 +-
> > >  drivers/pci/controller/dwc/pcie-tegra194.c    |  9 ++++-
> > >  drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> > >  drivers/pci/endpoint/pci-epc-core.c           | 32 ++++++++++++----
> > >  include/linux/pci-epc.h                       | 10 +----
> > >  include/linux/pci-epf.h                       | 19 ++++++----
> > >  6 files changed, 59 insertions(+), 51 deletions(-)
> > > 
> > > -- 
> > > 2.25.1
> > > 
> > 
> > -- 
> > மணிவண்ணன் சதாசிவம்

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v4 2/5] PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler

[Manivannan Sadhasivam]

On Tue, Oct 25, 2022 at 08:20:58PM +0530, Manivannan Sadhasivam wrote:
> dw_pcie_ep_linkup() may take more time to execute depending on the EPF
> driver implementation. Calling this API in the hard IRQ handler is not
> encouraged since the hard IRQ handlers are supposed to complete quickly.
> 
> So move the dw_pcie_ep_linkup() call to threaded IRQ handler.
> 
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> ---
>  drivers/pci/controller/dwc/pcie-tegra194.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> index 1b6b437823d2..a0d231b7a435 100644
> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> @@ -287,6 +287,7 @@ struct tegra_pcie_dw {
>  	struct gpio_desc *pex_refclk_sel_gpiod;
>  	unsigned int pex_rst_irq;
>  	int ep_state;
> +	long link_status;
>  };
>  
>  static inline struct tegra_pcie_dw *to_tegra_pcie(struct dw_pcie *pci)
> @@ -450,9 +451,13 @@ static void pex_ep_event_hot_rst_done(struct tegra_pcie_dw *pcie)
>  static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
>  {
>  	struct tegra_pcie_dw *pcie = arg;
> +	struct dw_pcie_ep *ep = &pcie->pci.ep;
>  	struct dw_pcie *pci = &pcie->pci;
>  	u32 val, speed;
>  
> +	if (test_and_clear_bit(0, &pcie->link_status))
> +		dw_pcie_ep_linkup(ep);
> +
>  	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
>  		PCI_EXP_LNKSTA_CLS;
>  	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
> @@ -499,7 +504,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
>  static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
>  {
>  	struct tegra_pcie_dw *pcie = arg;
> -	struct dw_pcie_ep *ep = &pcie->pci.ep;
>  	int spurious = 1;
>  	u32 status_l0, status_l1, link_status;
>  
> @@ -515,7 +519,8 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
>  			link_status = appl_readl(pcie, APPL_LINK_STATUS);
>  			if (link_status & APPL_LINK_STATUS_RDLH_LINK_UP) {
>  				dev_dbg(pcie->dev, "Link is up with Host\n");
> -				dw_pcie_ep_linkup(ep);
> +				set_bit(0, &pcie->link_status);
> +				return IRQ_WAKE_THREAD;
>  			}
>  		}
>  
> -- 
> 2.25.1
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v4 2/5] PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler

[Manivannan Sadhasivam]

On Mon, Nov 14, 2022 at 04:37:00PM +0530, Manivannan Sadhasivam wrote:
> On Tue, Oct 25, 2022 at 08:20:58PM +0530, Manivannan Sadhasivam wrote:
> > dw_pcie_ep_linkup() may take more time to execute depending on the EPF
> > driver implementation. Calling this API in the hard IRQ handler is not
> > encouraged since the hard IRQ handlers are supposed to complete quickly.
> > 
> > So move the dw_pcie_ep_linkup() call to threaded IRQ handler.
> > 
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Sorry for resending it (something messed up with my email client).

Vidya, can you please review this patch?

Thanks,
Mani

> > ---
> >  drivers/pci/controller/dwc/pcie-tegra194.c | 9 +++++++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> > index 1b6b437823d2..a0d231b7a435 100644
> > --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> > @@ -287,6 +287,7 @@ struct tegra_pcie_dw {
> >  	struct gpio_desc *pex_refclk_sel_gpiod;
> >  	unsigned int pex_rst_irq;
> >  	int ep_state;
> > +	long link_status;
> >  };
> >  
> >  static inline struct tegra_pcie_dw *to_tegra_pcie(struct dw_pcie *pci)
> > @@ -450,9 +451,13 @@ static void pex_ep_event_hot_rst_done(struct tegra_pcie_dw *pcie)
> >  static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
> >  {
> >  	struct tegra_pcie_dw *pcie = arg;
> > +	struct dw_pcie_ep *ep = &pcie->pci.ep;
> >  	struct dw_pcie *pci = &pcie->pci;
> >  	u32 val, speed;
> >  
> > +	if (test_and_clear_bit(0, &pcie->link_status))
> > +		dw_pcie_ep_linkup(ep);
> > +
> >  	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
> >  		PCI_EXP_LNKSTA_CLS;
> >  	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
> > @@ -499,7 +504,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
> >  static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
> >  {
> >  	struct tegra_pcie_dw *pcie = arg;
> > -	struct dw_pcie_ep *ep = &pcie->pci.ep;
> >  	int spurious = 1;
> >  	u32 status_l0, status_l1, link_status;
> >  
> > @@ -515,7 +519,8 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
> >  			link_status = appl_readl(pcie, APPL_LINK_STATUS);
> >  			if (link_status & APPL_LINK_STATUS_RDLH_LINK_UP) {
> >  				dev_dbg(pcie->dev, "Link is up with Host\n");
> > -				dw_pcie_ep_linkup(ep);
> > +				set_bit(0, &pcie->link_status);
> > +				return IRQ_WAKE_THREAD;
> >  			}
> >  		}
> >  
> > -- 
> > 2.25.1
> > 
> 
> -- 
> மணிவண்ணன் சதாசிவம்

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v4 2/5] PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler

[Manivannan Sadhasivam]

On Mon, Nov 14, 2022 at 04:38:20PM +0530, Manivannan Sadhasivam wrote:
> On Mon, Nov 14, 2022 at 04:37:00PM +0530, Manivannan Sadhasivam wrote:
> > On Tue, Oct 25, 2022 at 08:20:58PM +0530, Manivannan Sadhasivam wrote:
> > > dw_pcie_ep_linkup() may take more time to execute depending on the EPF
> > > driver implementation. Calling this API in the hard IRQ handler is not
> > > encouraged since the hard IRQ handlers are supposed to complete quickly.
> > > 
> > > So move the dw_pcie_ep_linkup() call to threaded IRQ handler.
> > > 
> > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> 
> Sorry for resending it (something messed up with my email client).
> 
> Vidya, can you please review this patch?
> 

Ping!

Thanks,
Mani

> Thanks,
> Mani
> 
> > > ---
> > >  drivers/pci/controller/dwc/pcie-tegra194.c | 9 +++++++--
> > >  1 file changed, 7 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> > > index 1b6b437823d2..a0d231b7a435 100644
> > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> > > @@ -287,6 +287,7 @@ struct tegra_pcie_dw {
> > >  	struct gpio_desc *pex_refclk_sel_gpiod;
> > >  	unsigned int pex_rst_irq;
> > >  	int ep_state;
> > > +	long link_status;
> > >  };
> > >  
> > >  static inline struct tegra_pcie_dw *to_tegra_pcie(struct dw_pcie *pci)
> > > @@ -450,9 +451,13 @@ static void pex_ep_event_hot_rst_done(struct tegra_pcie_dw *pcie)
> > >  static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
> > >  {
> > >  	struct tegra_pcie_dw *pcie = arg;
> > > +	struct dw_pcie_ep *ep = &pcie->pci.ep;
> > >  	struct dw_pcie *pci = &pcie->pci;
> > >  	u32 val, speed;
> > >  
> > > +	if (test_and_clear_bit(0, &pcie->link_status))
> > > +		dw_pcie_ep_linkup(ep);
> > > +
> > >  	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
> > >  		PCI_EXP_LNKSTA_CLS;
> > >  	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
> > > @@ -499,7 +504,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
> > >  static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
> > >  {
> > >  	struct tegra_pcie_dw *pcie = arg;
> > > -	struct dw_pcie_ep *ep = &pcie->pci.ep;
> > >  	int spurious = 1;
> > >  	u32 status_l0, status_l1, link_status;
> > >  
> > > @@ -515,7 +519,8 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
> > >  			link_status = appl_readl(pcie, APPL_LINK_STATUS);
> > >  			if (link_status & APPL_LINK_STATUS_RDLH_LINK_UP) {
> > >  				dev_dbg(pcie->dev, "Link is up with Host\n");
> > > -				dw_pcie_ep_linkup(ep);
> > > +				set_bit(0, &pcie->link_status);
> > > +				return IRQ_WAKE_THREAD;
> > >  			}
> > >  		}
> > >  
> > > -- 
> > > 2.25.1
> > > 
> > 
> > -- 
> > மணிவண்ணன் சதாசிவம்
> 
> -- 
> மணிவண்ணன் சதாசிவம்

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v4 2/5] PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler

[Manivannan Sadhasivam]

On Tue, Nov 22, 2022 at 07:19:41PM +0530, Manivannan Sadhasivam wrote:
> On Mon, Nov 14, 2022 at 04:38:20PM +0530, Manivannan Sadhasivam wrote:
> > On Mon, Nov 14, 2022 at 04:37:00PM +0530, Manivannan Sadhasivam wrote:
> > > On Tue, Oct 25, 2022 at 08:20:58PM +0530, Manivannan Sadhasivam wrote:
> > > > dw_pcie_ep_linkup() may take more time to execute depending on the EPF
> > > > driver implementation. Calling this API in the hard IRQ handler is not
> > > > encouraged since the hard IRQ handlers are supposed to complete quickly.
> > > > 
> > > > So move the dw_pcie_ep_linkup() call to threaded IRQ handler.
> > > > 
> > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > 
> > Sorry for resending it (something messed up with my email client).
> > 
> > Vidya, can you please review this patch?
> > 
> 
> Ping!
> 

Ping again. Vidya, only your t-b tag is missing to get this series merged. Can
you please test and report back.

There are few series that depends on this including a couple from you.

Thanks,
Mani

> Thanks,
> Mani
> 
> > Thanks,
> > Mani
> > 
> > > > ---
> > > >  drivers/pci/controller/dwc/pcie-tegra194.c | 9 +++++++--
> > > >  1 file changed, 7 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> > > > index 1b6b437823d2..a0d231b7a435 100644
> > > > --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> > > > +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> > > > @@ -287,6 +287,7 @@ struct tegra_pcie_dw {
> > > >  	struct gpio_desc *pex_refclk_sel_gpiod;
> > > >  	unsigned int pex_rst_irq;
> > > >  	int ep_state;
> > > > +	long link_status;
> > > >  };
> > > >  
> > > >  static inline struct tegra_pcie_dw *to_tegra_pcie(struct dw_pcie *pci)
> > > > @@ -450,9 +451,13 @@ static void pex_ep_event_hot_rst_done(struct tegra_pcie_dw *pcie)
> > > >  static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
> > > >  {
> > > >  	struct tegra_pcie_dw *pcie = arg;
> > > > +	struct dw_pcie_ep *ep = &pcie->pci.ep;
> > > >  	struct dw_pcie *pci = &pcie->pci;
> > > >  	u32 val, speed;
> > > >  
> > > > +	if (test_and_clear_bit(0, &pcie->link_status))
> > > > +		dw_pcie_ep_linkup(ep);
> > > > +
> > > >  	speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
> > > >  		PCI_EXP_LNKSTA_CLS;
> > > >  	clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
> > > > @@ -499,7 +504,6 @@ static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
> > > >  static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
> > > >  {
> > > >  	struct tegra_pcie_dw *pcie = arg;
> > > > -	struct dw_pcie_ep *ep = &pcie->pci.ep;
> > > >  	int spurious = 1;
> > > >  	u32 status_l0, status_l1, link_status;
> > > >  
> > > > @@ -515,7 +519,8 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
> > > >  			link_status = appl_readl(pcie, APPL_LINK_STATUS);
> > > >  			if (link_status & APPL_LINK_STATUS_RDLH_LINK_UP) {
> > > >  				dev_dbg(pcie->dev, "Link is up with Host\n");
> > > > -				dw_pcie_ep_linkup(ep);
> > > > +				set_bit(0, &pcie->link_status);
> > > > +				return IRQ_WAKE_THREAD;
> > > >  			}
> > > >  		}
> > > >  
> > > > -- 
> > > > 2.25.1
> > > > 
> > > 
> > > -- 
> > > மணிவண்ணன் சதாசிவம்
> > 
> > -- 
> > மணிவண்ணன் சதாசிவம்
> 
> -- 
> மணிவண்ணன் சதாசிவம்

-- 
மணிவண்ணன் சதாசிவம்