[PATCH v5 0/9] Add support for MHI Endpoint function driver

[PATCH v5 0/9] Add support for MHI Endpoint function driver

[Manivannan Sadhasivam]

Hello,

This series adds support for Modem Host Interface (MHI) Endpoint function
driver and few updates to the PCI endpoint core.

MHI
===

MHI is the communication protocol used by the host machines to control and
communicate with the Qualcomm modems/WLAN devices over any high speed physical
bus like PCIe. In Linux kernel, MHI is modeled as a bus driver [1] and there
are two instances of MHI used in a typical setup.

1. MHI host - MHI implementation for the host machines like x86/ARM64.
2. MHI Endpoint - MHI implementation for the endpoint devices like modems.

MHI EPF
=======

The MHI Endpoint function driver (MHI EPF) is used on the MHI endpoint devices
like modems. The MHI EPF driver sits in between the PCIe EP and MHI EP bus and
carries out all of the PCIe related activities like BAR config, PCIe Event
handling, MMIO read/write etc,... for the MHI EP bus.

Below is the simple representation of the setup:


                 +----------------------------------------------------+
                 |                  Endpoint CPU                      |                   
                 |                                                    |
+------------+   |   +------------+   +-----------+   +-----------+   |
|            |   |   |            |   |           |   |           |   |
|            |   |   |   MHI EP   |   |           |   |           |   | PCIe Bus
|  Modem DSP +---+---+    Bus     +---+  MHI EPF  +---+  PCIe EP  +---+---------
|            |   |   |            |   |           |   |           |   |
|            |   |   |            |   |           |   |           |   |
+------------+   |   +------------+   +-----------+   +-----------+   |
                 |                                                    |
                 |                                                    |
                 +----------------------------------------------------+

The data packets will be read from the Modem DSP by the MHI stack and will be
transmitted to the host machine over PCIe bus with the help of MHI EPF driver.

Test setup
==========

This series has been tested on Snapdragon X55 modem a.k.a SDX55 connected to
the ARM64 host machine.

Thanks,
Mani

[1] https://www.kernel.org/doc/html/latest/mhi/mhi.html

Changes in v5:

* Moved the PCI EPF driver match logic to pci_epf_match_id() function and used
  that to get the matched driver ID for passing to driver probe instead of
  storing the id during match.
* Added a patch to fix the missing documentation about MSI/MSI-X start vector.
* Addressed the review comments on the MHI EPF driver. Most notably, got rid of
  local variable for tracking MHI registration and used the mhi_dev pointer.
  Also, modified the MSI vector increment comment to make it clear.
* Added a patch for adding MHI EPF driver to MAINTAINERS file

Changes in v4:

* Collected review tag from Kishon
* Changed the IP_SW0 channel numbers as per latest MHI spec

Changes in v3:

* Fixed the probe function of EPF_VNTB driver

Changes in v2:

* Rebased on top of v6.3-rc1
* Switched to the new callback interface for passing events from EPC to EPF
* Dropped one patch related to notifier

Manivannan Sadhasivam (9):
  MAINTAINERS: Add entry for MHI networking drivers under MHI bus
  PCI: endpoint: Add missing documentation about the MSI/MSI-X range
  PCI: endpoint: Pass EPF device ID to the probe function
  PCI: endpoint: Warn and return if EPC is started/stopped multiple
    times
  PCI: endpoint: Add linkdown notifier support
  PCI: endpoint: Add BME notifier support
  PCI: qcom-ep: Add support for Link down notification
  PCI: qcom-ep: Add support for BME notification
  PCI: endpoint: Add PCI Endpoint function driver for MHI bus

 MAINTAINERS                                   |   1 +
 drivers/pci/controller/dwc/pcie-qcom-ep.c     |   2 +
 drivers/pci/endpoint/functions/Kconfig        |  10 +
 drivers/pci/endpoint/functions/Makefile       |   1 +
 drivers/pci/endpoint/functions/pci-epf-mhi.c  | 462 ++++++++++++++++++
 drivers/pci/endpoint/functions/pci-epf-ntb.c  |   3 +-
 drivers/pci/endpoint/functions/pci-epf-test.c |   2 +-
 drivers/pci/endpoint/functions/pci-epf-vntb.c |   2 +-
 drivers/pci/endpoint/pci-ep-cfs.c             |   3 +
 drivers/pci/endpoint/pci-epc-core.c           |  56 ++-
 drivers/pci/endpoint/pci-epf-core.c           |  20 +-
 include/linux/pci-epc.h                       |   2 +
 include/linux/pci-epf.h                       |   8 +-
 13 files changed, 559 insertions(+), 13 deletions(-)
 create mode 100644 drivers/pci/endpoint/functions/pci-epf-mhi.c

-- 
2.25.1

[PATCH v5 1/9] MAINTAINERS: Add entry for MHI networking drivers under MHI bus

[Manivannan Sadhasivam]

The host MHI net driver was not listed earlier. So let's add both host and
endpoint MHI net drivers under MHI bus.

Cc: Loic Poulain <loic.poulain@linaro.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 7e0b87d5aa2e..07625a47cf08 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -13629,6 +13629,7 @@ T:	git git://git.kernel.org/pub/scm/linux/kernel/git/mani/mhi.git
 F:	Documentation/ABI/stable/sysfs-bus-mhi
 F:	Documentation/mhi/
 F:	drivers/bus/mhi/
+F:	drivers/net/mhi_*
 F:	include/linux/mhi.h
 
 MICROBLAZE ARCHITECTURE
-- 
2.25.1

[PATCH v5 2/9] PCI: endpoint: Add missing documentation about the MSI/MSI-X range

[Manivannan Sadhasivam]

Both pci_epc_raise_irq() and pci_epc_map_msi_irq() APIs expects the
MSI/MSI-X vectors to start from 1 but it is not documented. Add the
range info to the kdoc of the APIs to make it clear.

Fixes: 5e8cb4033807 ("PCI: endpoint: Add EP core layer to enable EP controller and EP functions")
Fixes: 87d5972e476f ("PCI: endpoint: Add pci_epc_ops to map MSI IRQ")
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/pci-epc-core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
index 46c9a5c3ca14..0cf602c83d4a 100644
--- a/drivers/pci/endpoint/pci-epc-core.c
+++ b/drivers/pci/endpoint/pci-epc-core.c
@@ -213,7 +213,7 @@ EXPORT_SYMBOL_GPL(pci_epc_start);
  * @func_no: the physical endpoint function number in the EPC device
  * @vfunc_no: the virtual endpoint function number in the physical function
  * @type: specify the type of interrupt; legacy, MSI or MSI-X
- * @interrupt_num: the MSI or MSI-X interrupt number
+ * @interrupt_num: the MSI or MSI-X interrupt number with range (1-N)
  *
  * Invoke to raise an legacy, MSI or MSI-X interrupt
  */
@@ -246,7 +246,7 @@ EXPORT_SYMBOL_GPL(pci_epc_raise_irq);
  * @func_no: the physical endpoint function number in the EPC device
  * @vfunc_no: the virtual endpoint function number in the physical function
  * @phys_addr: the physical address of the outbound region
- * @interrupt_num: the MSI interrupt number
+ * @interrupt_num: the MSI interrupt number with range (1-N)
  * @entry_size: Size of Outbound address region for each interrupt
  * @msi_data: the data that should be written in order to raise MSI interrupt
  *            with interrupt number as 'interrupt num'
-- 
2.25.1

Re: [PATCH v5 2/9] PCI: endpoint: Add missing documentation about the MSI/MSI-X range

[Damien Le Moal]

On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> Both pci_epc_raise_irq() and pci_epc_map_msi_irq() APIs expects the
> MSI/MSI-X vectors to start from 1 but it is not documented. Add the
> range info to the kdoc of the APIs to make it clear.
> 
> Fixes: 5e8cb4033807 ("PCI: endpoint: Add EP core layer to enable EP controller and EP functions")
> Fixes: 87d5972e476f ("PCI: endpoint: Add pci_epc_ops to map MSI IRQ")
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research

[PATCH v5 3/9] PCI: endpoint: Pass EPF device ID to the probe function

[Manivannan Sadhasivam]

Currently, the EPF probe function doesn't get the device ID argument needed
to correctly identify the device table ID of the EPF device.

When multiple entries are added to the "struct pci_epf_device_id" table,
the probe function needs to identify the correct one. This is achieved by
modifying the pci_epf_match_id() function to return the match ID pointer
and passing it to the driver's probe function.

pci_epf_device_match() function can return bool based on the return value
of pci_epf_match_id().

Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/functions/pci-epf-ntb.c  |  3 ++-
 drivers/pci/endpoint/functions/pci-epf-test.c |  2 +-
 drivers/pci/endpoint/functions/pci-epf-vntb.c |  2 +-
 drivers/pci/endpoint/pci-epf-core.c           | 20 ++++++++++++-------
 include/linux/pci-epf.h                       |  4 +++-
 5 files changed, 20 insertions(+), 11 deletions(-)

diff --git a/drivers/pci/endpoint/functions/pci-epf-ntb.c b/drivers/pci/endpoint/functions/pci-epf-ntb.c
index 9a00448c7e61..980b4ecf19a2 100644
--- a/drivers/pci/endpoint/functions/pci-epf-ntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-ntb.c
@@ -2075,11 +2075,12 @@ static struct config_group *epf_ntb_add_cfs(struct pci_epf *epf,
 /**
  * epf_ntb_probe() - Probe NTB function driver
  * @epf: NTB endpoint function device
+ * @id: NTB endpoint function device ID
  *
  * Probe NTB function driver when endpoint function bus detects a NTB
  * endpoint function.
  */
-static int epf_ntb_probe(struct pci_epf *epf)
+static int epf_ntb_probe(struct pci_epf *epf, const struct pci_epf_device_id *id)
 {
 	struct epf_ntb *ntb;
 	struct device *dev;
diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
index 0f9d2ec822ac..d5fcc78a5b73 100644
--- a/drivers/pci/endpoint/functions/pci-epf-test.c
+++ b/drivers/pci/endpoint/functions/pci-epf-test.c
@@ -980,7 +980,7 @@ static const struct pci_epf_device_id pci_epf_test_ids[] = {
 	{},
 };
 
-static int pci_epf_test_probe(struct pci_epf *epf)
+static int pci_epf_test_probe(struct pci_epf *epf, const struct pci_epf_device_id *id)
 {
 	struct pci_epf_test *epf_test;
 	struct device *dev = &epf->dev;
diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
index b7c7a8af99f4..122eb7a12028 100644
--- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
+++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
@@ -1401,7 +1401,7 @@ static struct pci_epf_ops epf_ntb_ops = {
  *
  * Returns: Zero for success, or an error code in case of failure
  */
-static int epf_ntb_probe(struct pci_epf *epf)
+static int epf_ntb_probe(struct pci_epf *epf, const struct pci_epf_device_id *id)
 {
 	struct epf_ntb *ntb;
 	struct device *dev;
diff --git a/drivers/pci/endpoint/pci-epf-core.c b/drivers/pci/endpoint/pci-epf-core.c
index 2036e38be093..7307e052136f 100644
--- a/drivers/pci/endpoint/pci-epf-core.c
+++ b/drivers/pci/endpoint/pci-epf-core.c
@@ -493,16 +493,16 @@ static const struct device_type pci_epf_type = {
 	.release	= pci_epf_dev_release,
 };
 
-static int
-pci_epf_match_id(const struct pci_epf_device_id *id, const struct pci_epf *epf)
+static const struct pci_epf_device_id
+*pci_epf_match_id(const struct pci_epf_device_id *id, const struct pci_epf *epf)
 {
 	while (id->name[0]) {
 		if (strcmp(epf->name, id->name) == 0)
-			return true;
+			return id;
 		id++;
 	}
 
-	return false;
+	return NULL;
 }
 
 static int pci_epf_device_match(struct device *dev, struct device_driver *drv)
@@ -510,8 +510,12 @@ static int pci_epf_device_match(struct device *dev, struct device_driver *drv)
 	struct pci_epf *epf = to_pci_epf(dev);
 	struct pci_epf_driver *driver = to_pci_epf_driver(drv);
 
-	if (driver->id_table)
-		return pci_epf_match_id(driver->id_table, epf);
+	if (driver->id_table) {
+		if (pci_epf_match_id(driver->id_table, epf))
+			return true;
+		else
+			return false;
+	}
 
 	return !strcmp(epf->name, drv->name);
 }
@@ -520,13 +524,15 @@ static int pci_epf_device_probe(struct device *dev)
 {
 	struct pci_epf *epf = to_pci_epf(dev);
 	struct pci_epf_driver *driver = to_pci_epf_driver(dev->driver);
+	const struct pci_epf_device_id *id;
 
 	if (!driver->probe)
 		return -ENODEV;
 
 	epf->driver = driver;
+	id = pci_epf_match_id(driver->id_table, epf);
 
-	return driver->probe(epf);
+	return driver->probe(epf, id);
 }
 
 static void pci_epf_device_remove(struct device *dev)
diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
index a215dc8ce693..bc613f0df7e3 100644
--- a/include/linux/pci-epf.h
+++ b/include/linux/pci-epf.h
@@ -89,7 +89,7 @@ struct pci_epc_event_ops {
  * @id_table: identifies EPF devices for probing
  */
 struct pci_epf_driver {
-	int	(*probe)(struct pci_epf *epf);
+	int	(*probe)(struct pci_epf *epf, const struct pci_epf_device_id *id);
 	void	(*remove)(struct pci_epf *epf);
 
 	struct device_driver	driver;
@@ -131,6 +131,7 @@ struct pci_epf_bar {
  * @epc: the EPC device to which this EPF device is bound
  * @epf_pf: the physical EPF device to which this virtual EPF device is bound
  * @driver: the EPF driver to which this EPF device is bound
+ * @id: Pointer to the EPF device ID
  * @list: to add pci_epf as a list of PCI endpoint functions to pci_epc
  * @lock: mutex to protect pci_epf_ops
  * @sec_epc: the secondary EPC device to which this EPF device is bound
@@ -158,6 +159,7 @@ struct pci_epf {
 	struct pci_epc		*epc;
 	struct pci_epf		*epf_pf;
 	struct pci_epf_driver	*driver;
+	const struct pci_epf_device_id *id;
 	struct list_head	list;
 	/* mutex to protect against concurrent access of pci_epf_ops */
 	struct mutex		lock;
-- 
2.25.1

Re: [PATCH v5 3/9] PCI: endpoint: Pass EPF device ID to the probe function

[Damien Le Moal]

On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> Currently, the EPF probe function doesn't get the device ID argument needed
> to correctly identify the device table ID of the EPF device.
> 
> When multiple entries are added to the "struct pci_epf_device_id" table,
> the probe function needs to identify the correct one. This is achieved by
> modifying the pci_epf_match_id() function to return the match ID pointer
> and passing it to the driver's probe function.
> 
> pci_epf_device_match() function can return bool based on the return value
> of pci_epf_match_id().
> 
> Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

[...]

>  static int pci_epf_device_match(struct device *dev, struct device_driver *drv)
> @@ -510,8 +510,12 @@ static int pci_epf_device_match(struct device *dev, struct device_driver *drv)
>  	struct pci_epf *epf = to_pci_epf(dev);
>  	struct pci_epf_driver *driver = to_pci_epf_driver(drv);
>  
> -	if (driver->id_table)
> -		return pci_epf_match_id(driver->id_table, epf);
> +	if (driver->id_table) {
> +		if (pci_epf_match_id(driver->id_table, epf))
> +			return true;
> +		else
> +			return false;

		return pci_epf_match_id(driver->id_table, epf) != NULL;

is simpler. If you do not like this, at least drop the "else" as it is not
necessary at all.

> +	}
>  
>  	return !strcmp(epf->name, drv->name);
>  }
> @@ -520,13 +524,15 @@ static int pci_epf_device_probe(struct device *dev)
>  {
>  	struct pci_epf *epf = to_pci_epf(dev);
>  	struct pci_epf_driver *driver = to_pci_epf_driver(dev->driver);
> +	const struct pci_epf_device_id *id;
>  
>  	if (!driver->probe)
>  		return -ENODEV;
>  
>  	epf->driver = driver;
> +	id = pci_epf_match_id(driver->id_table, epf);

Not sure that the id variable is that useful.

>  
> -	return driver->probe(epf);
> +	return driver->probe(epf, id);
>  }

-- 
Damien Le Moal
Western Digital Research

Re: [PATCH v5 3/9] PCI: endpoint: Pass EPF device ID to the probe function

[Manivannan Sadhasivam]

On Fri, Jun 02, 2023 at 08:16:45AM +0900, Damien Le Moal wrote:
> On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> > Currently, the EPF probe function doesn't get the device ID argument needed
> > to correctly identify the device table ID of the EPF device.
> > 
> > When multiple entries are added to the "struct pci_epf_device_id" table,
> > the probe function needs to identify the correct one. This is achieved by
> > modifying the pci_epf_match_id() function to return the match ID pointer
> > and passing it to the driver's probe function.
> > 
> > pci_epf_device_match() function can return bool based on the return value
> > of pci_epf_match_id().
> > 
> > Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> 
> [...]
> 
> >  static int pci_epf_device_match(struct device *dev, struct device_driver *drv)
> > @@ -510,8 +510,12 @@ static int pci_epf_device_match(struct device *dev, struct device_driver *drv)
> >  	struct pci_epf *epf = to_pci_epf(dev);
> >  	struct pci_epf_driver *driver = to_pci_epf_driver(drv);
> >  
> > -	if (driver->id_table)
> > -		return pci_epf_match_id(driver->id_table, epf);
> > +	if (driver->id_table) {
> > +		if (pci_epf_match_id(driver->id_table, epf))
> > +			return true;
> > +		else
> > +			return false;
> 
> 		return pci_epf_match_id(driver->id_table, epf) != NULL;
> 
> is simpler. If you do not like this, at least drop the "else" as it is not
> necessary at all.
> 

I settle for simplicity :) Also, there is a theoretical possibility of passing
NULL as the id->name, so doesn't want to rule out that.

> > +	}
> >  
> >  	return !strcmp(epf->name, drv->name);
> >  }
> > @@ -520,13 +524,15 @@ static int pci_epf_device_probe(struct device *dev)
> >  {
> >  	struct pci_epf *epf = to_pci_epf(dev);
> >  	struct pci_epf_driver *driver = to_pci_epf_driver(dev->driver);
> > +	const struct pci_epf_device_id *id;
> >  
> >  	if (!driver->probe)
> >  		return -ENODEV;
> >  
> >  	epf->driver = driver;
> > +	id = pci_epf_match_id(driver->id_table, epf);
> 
> Not sure that the id variable is that useful.
> 

Thought it makes the code clear but looking again, it doesn't hurt to call match
directly in probe. Will change it in next iteration.

- Mani

> >  
> > -	return driver->probe(epf);
> > +	return driver->probe(epf, id);
> >  }
> 
> -- 
> Damien Le Moal
> Western Digital Research
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH v5 4/9] PCI: endpoint: Warn and return if EPC is started/stopped multiple times

[Manivannan Sadhasivam]

When the EPC is started or stopped multiple times from configfs, just emit
a once time warning and return. There is no need to call the EPC start/stop
functions in those cases.

Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/pci-ep-cfs.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/pci/endpoint/pci-ep-cfs.c b/drivers/pci/endpoint/pci-ep-cfs.c
index 4b8ac0ac84d5..62c8e09c59f4 100644
--- a/drivers/pci/endpoint/pci-ep-cfs.c
+++ b/drivers/pci/endpoint/pci-ep-cfs.c
@@ -178,6 +178,9 @@ static ssize_t pci_epc_start_store(struct config_item *item, const char *page,
 	if (kstrtobool(page, &start) < 0)
 		return -EINVAL;
 
+	if (WARN_ON_ONCE(start == epc_group->start))
+		return 0;
+
 	if (!start) {
 		pci_epc_stop(epc);
 		epc_group->start = 0;
-- 
2.25.1

Re: [PATCH v5 4/9] PCI: endpoint: Warn and return if EPC is started/stopped multiple times

[Damien Le Moal]

On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> When the EPC is started or stopped multiple times from configfs, just emit
> a once time warning and return. There is no need to call the EPC start/stop
> functions in those cases.
> 
> Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> ---
>  drivers/pci/endpoint/pci-ep-cfs.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/pci/endpoint/pci-ep-cfs.c b/drivers/pci/endpoint/pci-ep-cfs.c
> index 4b8ac0ac84d5..62c8e09c59f4 100644
> --- a/drivers/pci/endpoint/pci-ep-cfs.c
> +++ b/drivers/pci/endpoint/pci-ep-cfs.c
> @@ -178,6 +178,9 @@ static ssize_t pci_epc_start_store(struct config_item *item, const char *page,
>  	if (kstrtobool(page, &start) < 0)
>  		return -EINVAL;
>  
> +	if (WARN_ON_ONCE(start == epc_group->start))
> +		return 0;

WARN will dump a backtrace which is fairly scary for the user. This case is
simply a bad user manipulation of the device, so why not simply add a pr_err()
(optional) and return -EALREADY ?

> +
>  	if (!start) {
>  		pci_epc_stop(epc);
>  		epc_group->start = 0;

-- 
Damien Le Moal
Western Digital Research

Re: [PATCH v5 4/9] PCI: endpoint: Warn and return if EPC is started/stopped multiple times

[Manivannan Sadhasivam]

On Fri, Jun 02, 2023 at 08:18:58AM +0900, Damien Le Moal wrote:
> On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> > When the EPC is started or stopped multiple times from configfs, just emit
> > a once time warning and return. There is no need to call the EPC start/stop
> > functions in those cases.
> > 
> > Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > ---
> >  drivers/pci/endpoint/pci-ep-cfs.c | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/drivers/pci/endpoint/pci-ep-cfs.c b/drivers/pci/endpoint/pci-ep-cfs.c
> > index 4b8ac0ac84d5..62c8e09c59f4 100644
> > --- a/drivers/pci/endpoint/pci-ep-cfs.c
> > +++ b/drivers/pci/endpoint/pci-ep-cfs.c
> > @@ -178,6 +178,9 @@ static ssize_t pci_epc_start_store(struct config_item *item, const char *page,
> >  	if (kstrtobool(page, &start) < 0)
> >  		return -EINVAL;
> >  
> > +	if (WARN_ON_ONCE(start == epc_group->start))
> > +		return 0;
> 
> WARN will dump a backtrace which is fairly scary for the user. This case is
> simply a bad user manipulation of the device, so why not simply add a pr_err()
> (optional) and return -EALREADY ?
> 

There EPF core uses WARN_ON_ONCE in other similar places, so thought of sticking
to that pattern. But I agree, WARN_ON_ONCE is not strictly required here. Will
add a error log and return the appropriate error no.

Moreover, will push a patch later to change other instances as well.

- Mani

> > +
> >  	if (!start) {
> >  		pci_epc_stop(epc);
> >  		epc_group->start = 0;
> 
> -- 
> Damien Le Moal
> Western Digital Research
> 

-- 
மணிவண்ணன் சதாசிவம்

[PATCH v5 5/9] PCI: endpoint: Add linkdown notifier support

[Manivannan Sadhasivam]

Add support to notify the EPF device about the linkdown event from the
EPC device.

Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/pci-epc-core.c | 26 ++++++++++++++++++++++++++
 include/linux/pci-epc.h             |  1 +
 include/linux/pci-epf.h             |  2 ++
 3 files changed, 29 insertions(+)

diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
index 0cf602c83d4a..e0570b52698d 100644
--- a/drivers/pci/endpoint/pci-epc-core.c
+++ b/drivers/pci/endpoint/pci-epc-core.c
@@ -706,6 +706,32 @@ void pci_epc_linkup(struct pci_epc *epc)
 }
 EXPORT_SYMBOL_GPL(pci_epc_linkup);
 
+/**
+ * pci_epc_linkdown() - Notify the EPF device that EPC device has dropped the
+ *			connection with the Root Complex.
+ * @epc: the EPC device which has dropped the link with the host
+ *
+ * Invoke to Notify the EPF device that the EPC device has dropped the
+ * connection with the Root Complex.
+ */
+void pci_epc_linkdown(struct pci_epc *epc)
+{
+	struct pci_epf *epf;
+
+	if (!epc || IS_ERR(epc))
+		return;
+
+	mutex_lock(&epc->list_lock);
+	list_for_each_entry(epf, &epc->pci_epf, list) {
+		mutex_lock(&epf->lock);
+		if (epf->event_ops && epf->event_ops->link_down)
+			epf->event_ops->link_down(epf);
+		mutex_unlock(&epf->lock);
+	}
+	mutex_unlock(&epc->list_lock);
+}
+EXPORT_SYMBOL_GPL(pci_epc_linkdown);
+
 /**
  * pci_epc_init_notify() - Notify the EPF device that EPC device's core
  *			   initialization is completed.
diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h
index 301bb0e53707..63a6cc5e5282 100644
--- a/include/linux/pci-epc.h
+++ b/include/linux/pci-epc.h
@@ -203,6 +203,7 @@ void pci_epc_destroy(struct pci_epc *epc);
 int pci_epc_add_epf(struct pci_epc *epc, struct pci_epf *epf,
 		    enum pci_epc_interface_type type);
 void pci_epc_linkup(struct pci_epc *epc);
+void pci_epc_linkdown(struct pci_epc *epc);
 void pci_epc_init_notify(struct pci_epc *epc);
 void pci_epc_remove_epf(struct pci_epc *epc, struct pci_epf *epf,
 			enum pci_epc_interface_type type);
diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
index bc613f0df7e3..f8e5a63d0c83 100644
--- a/include/linux/pci-epf.h
+++ b/include/linux/pci-epf.h
@@ -71,10 +71,12 @@ struct pci_epf_ops {
  * struct pci_epf_event_ops - Callbacks for capturing the EPC events
  * @core_init: Callback for the EPC initialization complete event
  * @link_up: Callback for the EPC link up event
+ * @link_down: Callback for the EPC link down event
  */
 struct pci_epc_event_ops {
 	int (*core_init)(struct pci_epf *epf);
 	int (*link_up)(struct pci_epf *epf);
+	int (*link_down)(struct pci_epf *epf);
 };
 
 /**
-- 
2.25.1

Re: [PATCH v5 5/9] PCI: endpoint: Add linkdown notifier support

[Damien Le Moal]

On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> Add support to notify the EPF device about the linkdown event from the
> EPC device.
> 
> Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research

[PATCH v5 6/9] PCI: endpoint: Add BME notifier support

[Manivannan Sadhasivam]

Add support to notify the EPF device about the Bus Master Enable (BME)
event received by the EPC device from the Root complex.

Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/pci-epc-core.c | 26 ++++++++++++++++++++++++++
 include/linux/pci-epc.h             |  1 +
 include/linux/pci-epf.h             |  2 ++
 3 files changed, 29 insertions(+)

diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
index e0570b52698d..6c54fa5684d2 100644
--- a/drivers/pci/endpoint/pci-epc-core.c
+++ b/drivers/pci/endpoint/pci-epc-core.c
@@ -758,6 +758,32 @@ void pci_epc_init_notify(struct pci_epc *epc)
 }
 EXPORT_SYMBOL_GPL(pci_epc_init_notify);
 
+/**
+ * pci_epc_bme_notify() - Notify the EPF device that the EPC device has received
+ *			  the BME event from the Root complex
+ * @epc: the EPC device that received the BME event
+ *
+ * Invoke to Notify the EPF device that the EPC device has received the Bus
+ * Master Enable (BME) event from the Root complex
+ */
+void pci_epc_bme_notify(struct pci_epc *epc)
+{
+	struct pci_epf *epf;
+
+	if (!epc || IS_ERR(epc))
+		return;
+
+	mutex_lock(&epc->list_lock);
+	list_for_each_entry(epf, &epc->pci_epf, list) {
+		mutex_lock(&epf->lock);
+		if (epf->event_ops && epf->event_ops->bme)
+			epf->event_ops->bme(epf);
+		mutex_unlock(&epf->lock);
+	}
+	mutex_unlock(&epc->list_lock);
+}
+EXPORT_SYMBOL_GPL(pci_epc_bme_notify);
+
 /**
  * pci_epc_destroy() - destroy the EPC device
  * @epc: the EPC device that has to be destroyed
diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h
index 63a6cc5e5282..5cb694031072 100644
--- a/include/linux/pci-epc.h
+++ b/include/linux/pci-epc.h
@@ -205,6 +205,7 @@ int pci_epc_add_epf(struct pci_epc *epc, struct pci_epf *epf,
 void pci_epc_linkup(struct pci_epc *epc);
 void pci_epc_linkdown(struct pci_epc *epc);
 void pci_epc_init_notify(struct pci_epc *epc);
+void pci_epc_bme_notify(struct pci_epc *epc);
 void pci_epc_remove_epf(struct pci_epc *epc, struct pci_epf *epf,
 			enum pci_epc_interface_type type);
 int pci_epc_write_header(struct pci_epc *epc, u8 func_no, u8 vfunc_no,
diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h
index f8e5a63d0c83..f34b3b32a0e7 100644
--- a/include/linux/pci-epf.h
+++ b/include/linux/pci-epf.h
@@ -72,11 +72,13 @@ struct pci_epf_ops {
  * @core_init: Callback for the EPC initialization complete event
  * @link_up: Callback for the EPC link up event
  * @link_down: Callback for the EPC link down event
+ * @bme: Callback for the EPC BME (Bus Master Enable) event
  */
 struct pci_epc_event_ops {
 	int (*core_init)(struct pci_epf *epf);
 	int (*link_up)(struct pci_epf *epf);
 	int (*link_down)(struct pci_epf *epf);
+	int (*bme)(struct pci_epf *epf);
 };
 
 /**
-- 
2.25.1

Re: [PATCH v5 6/9] PCI: endpoint: Add BME notifier support

[Damien Le Moal]

On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> Add support to notify the EPF device about the Bus Master Enable (BME)
> event received by the EPC device from the Root complex.
> 
> Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research

[PATCH v5 7/9] PCI: qcom-ep: Add support for Link down notification

[Manivannan Sadhasivam]

Add support to pass Link down notification to Endpoint function driver
so that the LINK_DOWN event can be processed by the function.

Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/pcie-qcom-ep.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c
index 19b32839ea26..4ce01ff7527c 100644
--- a/drivers/pci/controller/dwc/pcie-qcom-ep.c
+++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c
@@ -569,6 +569,7 @@ static irqreturn_t qcom_pcie_ep_global_irq_thread(int irq, void *data)
 	if (FIELD_GET(PARF_INT_ALL_LINK_DOWN, status)) {
 		dev_dbg(dev, "Received Linkdown event\n");
 		pcie_ep->link_status = QCOM_PCIE_EP_LINK_DOWN;
+		pci_epc_linkdown(pci->ep.epc);
 	} else if (FIELD_GET(PARF_INT_ALL_BME, status)) {
 		dev_dbg(dev, "Received BME event. Link is enabled!\n");
 		pcie_ep->link_status = QCOM_PCIE_EP_LINK_ENABLED;
-- 
2.25.1

Re: [PATCH v5 7/9] PCI: qcom-ep: Add support for Link down notification

[Damien Le Moal]

On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> Add support to pass Link down notification to Endpoint function driver
> so that the LINK_DOWN event can be processed by the function.
> 
> Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research

[PATCH v5 8/9] PCI: qcom-ep: Add support for BME notification

[Manivannan Sadhasivam]

Add support to pass BME (Bus Master Enable) notification to Endpoint
function driver so that the BME event can be processed by the function.

Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/controller/dwc/pcie-qcom-ep.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c
index 4ce01ff7527c..1435f516d3f7 100644
--- a/drivers/pci/controller/dwc/pcie-qcom-ep.c
+++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c
@@ -573,6 +573,7 @@ static irqreturn_t qcom_pcie_ep_global_irq_thread(int irq, void *data)
 	} else if (FIELD_GET(PARF_INT_ALL_BME, status)) {
 		dev_dbg(dev, "Received BME event. Link is enabled!\n");
 		pcie_ep->link_status = QCOM_PCIE_EP_LINK_ENABLED;
+		pci_epc_bme_notify(pci->ep.epc);
 	} else if (FIELD_GET(PARF_INT_ALL_PM_TURNOFF, status)) {
 		dev_dbg(dev, "Received PM Turn-off event! Entering L23\n");
 		val = readl_relaxed(pcie_ep->parf + PARF_PM_CTRL);
-- 
2.25.1

Re: [PATCH v5 8/9] PCI: qcom-ep: Add support for BME notification

[Damien Le Moal]

On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> Add support to pass BME (Bus Master Enable) notification to Endpoint
> function driver so that the BME event can be processed by the function.
> 
> Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research

[PATCH v5 9/9] PCI: endpoint: Add PCI Endpoint function driver for MHI bus

[Manivannan Sadhasivam]

Add PCI Endpoint driver for the Qualcomm MHI (Modem Host Interface) bus.
The driver implements the MHI function over PCI in the endpoint device
such as SDX55 modem. The MHI endpoint function driver acts as a
controller driver for the MHI Endpoint stack and carries out all PCI
related activities like mapping the host memory using iATU, triggering
MSIs etc...

Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/pci/endpoint/functions/Kconfig       |  10 +
 drivers/pci/endpoint/functions/Makefile      |   1 +
 drivers/pci/endpoint/functions/pci-epf-mhi.c | 462 +++++++++++++++++++
 3 files changed, 473 insertions(+)
 create mode 100644 drivers/pci/endpoint/functions/pci-epf-mhi.c

diff --git a/drivers/pci/endpoint/functions/Kconfig b/drivers/pci/endpoint/functions/Kconfig
index 9fd560886871..f5171b4fabbe 100644
--- a/drivers/pci/endpoint/functions/Kconfig
+++ b/drivers/pci/endpoint/functions/Kconfig
@@ -37,3 +37,13 @@ config PCI_EPF_VNTB
 	  between PCI Root Port and PCIe Endpoint.
 
 	  If in doubt, say "N" to disable Endpoint NTB driver.
+
+config PCI_EPF_MHI
+	tristate "PCI Endpoint driver for MHI bus"
+	depends on PCI_ENDPOINT && MHI_BUS_EP
+	help
+	   Enable this configuration option to enable the PCI Endpoint
+	   driver for Modem Host Interface (MHI) bus in Qualcomm Endpoint
+	   devices such as SDX55.
+
+	   If in doubt, say "N" to disable Endpoint driver for MHI bus.
diff --git a/drivers/pci/endpoint/functions/Makefile b/drivers/pci/endpoint/functions/Makefile
index 5c13001deaba..696473fce50e 100644
--- a/drivers/pci/endpoint/functions/Makefile
+++ b/drivers/pci/endpoint/functions/Makefile
@@ -6,3 +6,4 @@
 obj-$(CONFIG_PCI_EPF_TEST)		+= pci-epf-test.o
 obj-$(CONFIG_PCI_EPF_NTB)		+= pci-epf-ntb.o
 obj-$(CONFIG_PCI_EPF_VNTB) 		+= pci-epf-vntb.o
+obj-$(CONFIG_PCI_EPF_MHI)		+= pci-epf-mhi.o
diff --git a/drivers/pci/endpoint/functions/pci-epf-mhi.c b/drivers/pci/endpoint/functions/pci-epf-mhi.c
new file mode 100644
index 000000000000..98f0d96cfd46
--- /dev/null
+++ b/drivers/pci/endpoint/functions/pci-epf-mhi.c
@@ -0,0 +1,462 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCI EPF driver for MHI Endpoint devices
+ *
+ * Copyright (C) 2023 Linaro Ltd.
+ * Author: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
+ */
+
+#include <linux/mhi_ep.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pci-epc.h>
+#include <linux/pci-epf.h>
+
+#define MHI_VERSION_1_0 0x01000000
+
+#define to_epf_mhi(cntrl) container_of(cntrl, struct pci_epf_mhi, cntrl)
+
+struct pci_epf_mhi_ep_info {
+	const struct mhi_ep_cntrl_config *config;
+	struct pci_epf_header *epf_header;
+	enum pci_barno bar_num;
+	u32 epf_flags;
+	u32 msi_count;
+	u32 mru;
+};
+
+#define MHI_EP_CHANNEL_CONFIG(ch_num, ch_name, direction)	\
+	{							\
+		.num = ch_num,					\
+		.name = ch_name,				\
+		.dir = direction,				\
+	}
+
+#define MHI_EP_CHANNEL_CONFIG_UL(ch_num, ch_name)		\
+	MHI_EP_CHANNEL_CONFIG(ch_num, ch_name, DMA_TO_DEVICE)
+
+#define MHI_EP_CHANNEL_CONFIG_DL(ch_num, ch_name)		\
+	MHI_EP_CHANNEL_CONFIG(ch_num, ch_name, DMA_FROM_DEVICE)
+
+static const struct mhi_ep_channel_config mhi_v1_channels[] = {
+	MHI_EP_CHANNEL_CONFIG_UL(0, "LOOPBACK"),
+	MHI_EP_CHANNEL_CONFIG_DL(1, "LOOPBACK"),
+	MHI_EP_CHANNEL_CONFIG_UL(2, "SAHARA"),
+	MHI_EP_CHANNEL_CONFIG_DL(3, "SAHARA"),
+	MHI_EP_CHANNEL_CONFIG_UL(4, "DIAG"),
+	MHI_EP_CHANNEL_CONFIG_DL(5, "DIAG"),
+	MHI_EP_CHANNEL_CONFIG_UL(6, "SSR"),
+	MHI_EP_CHANNEL_CONFIG_DL(7, "SSR"),
+	MHI_EP_CHANNEL_CONFIG_UL(8, "QDSS"),
+	MHI_EP_CHANNEL_CONFIG_DL(9, "QDSS"),
+	MHI_EP_CHANNEL_CONFIG_UL(10, "EFS"),
+	MHI_EP_CHANNEL_CONFIG_DL(11, "EFS"),
+	MHI_EP_CHANNEL_CONFIG_UL(12, "MBIM"),
+	MHI_EP_CHANNEL_CONFIG_DL(13, "MBIM"),
+	MHI_EP_CHANNEL_CONFIG_UL(14, "QMI"),
+	MHI_EP_CHANNEL_CONFIG_DL(15, "QMI"),
+	MHI_EP_CHANNEL_CONFIG_UL(16, "QMI"),
+	MHI_EP_CHANNEL_CONFIG_DL(17, "QMI"),
+	MHI_EP_CHANNEL_CONFIG_UL(18, "IP-CTRL-1"),
+	MHI_EP_CHANNEL_CONFIG_DL(19, "IP-CTRL-1"),
+	MHI_EP_CHANNEL_CONFIG_UL(20, "IPCR"),
+	MHI_EP_CHANNEL_CONFIG_DL(21, "IPCR"),
+	MHI_EP_CHANNEL_CONFIG_UL(32, "DUN"),
+	MHI_EP_CHANNEL_CONFIG_DL(33, "DUN"),
+	MHI_EP_CHANNEL_CONFIG_UL(46, "IP_SW0"),
+	MHI_EP_CHANNEL_CONFIG_DL(47, "IP_SW0"),
+};
+
+static const struct mhi_ep_cntrl_config mhi_v1_config = {
+	.max_channels = 128,
+	.num_channels = ARRAY_SIZE(mhi_v1_channels),
+	.ch_cfg = mhi_v1_channels,
+	.mhi_version = MHI_VERSION_1_0,
+};
+
+static struct pci_epf_header sdx55_header = {
+	.vendorid = PCI_VENDOR_ID_QCOM,
+	.deviceid = 0x0306,
+	.baseclass_code = PCI_BASE_CLASS_COMMUNICATION,
+	.subclass_code = PCI_CLASS_COMMUNICATION_MODEM & 0xff,
+	.interrupt_pin	= PCI_INTERRUPT_INTA,
+};
+
+static const struct pci_epf_mhi_ep_info sdx55_info = {
+	.config = &mhi_v1_config,
+	.epf_header = &sdx55_header,
+	.bar_num = BAR_0,
+	.epf_flags = PCI_BASE_ADDRESS_MEM_TYPE_32,
+	.msi_count = 32,
+	.mru = 0x8000,
+};
+
+struct pci_epf_mhi {
+	const struct pci_epf_mhi_ep_info *info;
+	struct mhi_ep_cntrl mhi_cntrl;
+	struct pci_epf *epf;
+	struct mutex lock;
+	void __iomem *mmio;
+	resource_size_t mmio_phys;
+	u32 mmio_size;
+	int irq;
+};
+
+static int pci_epf_mhi_alloc_map(struct mhi_ep_cntrl *mhi_cntrl, u64 pci_addr,
+				 phys_addr_t *paddr, void __iomem **vaddr,
+				 size_t size)
+{
+	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
+	struct pci_epf *epf = epf_mhi->epf;
+	struct pci_epc *epc = epf_mhi->epf->epc;
+	size_t offset = pci_addr & (epc->mem->window.page_size - 1);
+	void __iomem *__vaddr;
+	phys_addr_t __paddr;
+	int ret;
+
+	__vaddr = pci_epc_mem_alloc_addr(epc, &__paddr, size + offset);
+	if (!__vaddr)
+		return -ENOMEM;
+
+	ret = pci_epc_map_addr(epc, epf->func_no, epf->vfunc_no, __paddr,
+			       pci_addr - offset, size + offset);
+	if (ret) {
+		pci_epc_mem_free_addr(epc, __paddr, __vaddr, size + offset);
+
+		return ret;
+	}
+
+	*paddr = __paddr + offset;
+	*vaddr = __vaddr + offset;
+
+	return 0;
+}
+
+static void pci_epf_mhi_unmap_free(struct mhi_ep_cntrl *mhi_cntrl, u64 pci_addr,
+				   phys_addr_t paddr, void __iomem *vaddr,
+				   size_t size)
+{
+	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
+	struct pci_epf *epf = epf_mhi->epf;
+	struct pci_epc *epc = epf->epc;
+	size_t offset = pci_addr & (epc->mem->window.page_size - 1);
+
+	pci_epc_unmap_addr(epc, epf->func_no, epf->vfunc_no, paddr - offset);
+	pci_epc_mem_free_addr(epc, paddr - offset, vaddr - offset,
+			      size + offset);
+}
+
+static void pci_epf_mhi_raise_irq(struct mhi_ep_cntrl *mhi_cntrl, u32 vector)
+{
+	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
+	struct pci_epf *epf = epf_mhi->epf;
+	struct pci_epc *epc = epf->epc;
+
+	/*
+	 * MHI supplies 0 based MSI vector but the API expects the vector to be
+	 * 1 based, so we need to increment the vector by 1.
+	 */
+	pci_epc_raise_irq(epc, epf->func_no, epf->vfunc_no, PCI_EPC_IRQ_MSI,
+			  vector + 1);
+}
+
+static int pci_epf_mhi_read_from_host(struct mhi_ep_cntrl *mhi_cntrl, u64 from,
+				      void __iomem *to, size_t size)
+{
+	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
+	struct pci_epf *epf = epf_mhi->epf;
+	struct pci_epc *epc = epf_mhi->epf->epc;
+	void __iomem *tre_buf;
+	phys_addr_t tre_phys;
+	size_t offset = from % SZ_4K;
+	int ret;
+
+	mutex_lock(&epf_mhi->lock);
+
+	tre_buf = pci_epc_mem_alloc_addr(epc, &tre_phys, size + offset);
+	if (!tre_buf) {
+		ret = -ENOMEM;
+		goto err_unlock;
+	}
+
+	ret = pci_epc_map_addr(epc, epf->func_no, epf->vfunc_no, tre_phys,
+			       from - offset, size + offset);
+	if (ret) {
+		pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
+		goto err_unlock;
+	}
+
+	memcpy_fromio(to, tre_buf + offset, size);
+
+	pci_epc_unmap_addr(epc, epf->func_no, epf->vfunc_no, tre_phys);
+	pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
+
+err_unlock:
+	mutex_unlock(&epf_mhi->lock);
+
+	return ret;
+}
+
+static int pci_epf_mhi_write_to_host(struct mhi_ep_cntrl *mhi_cntrl,
+				     void __iomem *from, u64 to, size_t size)
+{
+	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
+	struct pci_epf *epf = epf_mhi->epf;
+	struct pci_epc *epc = epf_mhi->epf->epc;
+	void __iomem *tre_buf;
+	phys_addr_t tre_phys;
+	size_t offset = to % SZ_4K;
+	int ret;
+
+	mutex_lock(&epf_mhi->lock);
+
+	tre_buf = pci_epc_mem_alloc_addr(epc, &tre_phys, size + offset);
+	if (!tre_buf) {
+		ret = -ENOMEM;
+		goto err_unlock;
+	}
+
+	ret = pci_epc_map_addr(epc, epf->func_no, epf->vfunc_no, tre_phys,
+			       to - offset, size + offset);
+	if (ret) {
+		pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
+		goto err_unlock;
+	}
+
+	memcpy_toio(tre_buf + offset, from, size);
+
+	pci_epc_unmap_addr(epc, epf->func_no, epf->vfunc_no, tre_phys);
+	pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
+
+err_unlock:
+	mutex_unlock(&epf_mhi->lock);
+
+	return ret;
+}
+
+static int pci_epf_mhi_core_init(struct pci_epf *epf)
+{
+	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
+	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
+	struct pci_epf_bar *epf_bar = &epf->bar[info->bar_num];
+	struct pci_epc *epc = epf->epc;
+	struct device *dev = &epf->dev;
+	int ret;
+
+	epf_bar->phys_addr = epf_mhi->mmio_phys;
+	epf_bar->size = epf_mhi->mmio_size;
+	epf_bar->barno = info->bar_num;
+	epf_bar->flags = info->epf_flags;
+	ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, epf_bar);
+	if (ret) {
+		dev_err(dev, "Failed to set BAR: %d\n", ret);
+		return ret;
+	}
+
+	ret = pci_epc_set_msi(epc, epf->func_no, epf->vfunc_no,
+			      order_base_2(info->msi_count));
+	if (ret) {
+		dev_err(dev, "Failed to set MSI configuration: %d\n", ret);
+		return ret;
+	}
+
+	ret = pci_epc_write_header(epc, epf->func_no, epf->vfunc_no,
+				   epf->header);
+	if (ret) {
+		dev_err(dev, "Failed to set Configuration header: %d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int pci_epf_mhi_link_up(struct pci_epf *epf)
+{
+	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
+	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
+	struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
+	struct pci_epc *epc = epf->epc;
+	struct device *dev = &epf->dev;
+	int ret;
+
+	mhi_cntrl->mmio = epf_mhi->mmio;
+	mhi_cntrl->irq = epf_mhi->irq;
+	mhi_cntrl->mru = info->mru;
+
+	/* Assign the struct dev of PCI EP as MHI controller device */
+	mhi_cntrl->cntrl_dev = epc->dev.parent;
+	mhi_cntrl->raise_irq = pci_epf_mhi_raise_irq;
+	mhi_cntrl->alloc_map = pci_epf_mhi_alloc_map;
+	mhi_cntrl->unmap_free = pci_epf_mhi_unmap_free;
+	mhi_cntrl->read_from_host = pci_epf_mhi_read_from_host;
+	mhi_cntrl->write_to_host = pci_epf_mhi_write_to_host;
+
+	/* Register the MHI EP controller */
+	ret = mhi_ep_register_controller(mhi_cntrl, info->config);
+	if (ret) {
+		dev_err(dev, "Failed to register MHI EP controller: %d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int pci_epf_mhi_link_down(struct pci_epf *epf)
+{
+	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
+	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
+	struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
+
+	if (mhi_cntrl->mhi_dev) {
+		mhi_ep_power_down(mhi_cntrl);
+		mhi_ep_unregister_controller(mhi_cntrl);
+	}
+
+	return 0;
+}
+
+static int pci_epf_mhi_bme(struct pci_epf *epf)
+{
+	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
+	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
+	struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
+	struct device *dev = &epf->dev;
+	int ret;
+
+	/*
+	 * Power up the MHI EP stack if link is up and stack is in power down
+	 * state.
+	 */
+	if (!mhi_cntrl->enabled && mhi_cntrl->mhi_dev) {
+		ret = mhi_ep_power_up(mhi_cntrl);
+		if (ret) {
+			dev_err(dev, "Failed to power up MHI EP: %d\n", ret);
+			mhi_ep_unregister_controller(mhi_cntrl);
+		}
+	}
+
+	return 0;
+}
+
+static int pci_epf_mhi_bind(struct pci_epf *epf)
+{
+	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
+	struct pci_epc *epc = epf->epc;
+	struct platform_device *pdev = to_platform_device(epc->dev.parent);
+	struct device *dev = &epf->dev;
+	struct resource *res;
+	int ret;
+
+	if (WARN_ON_ONCE(!epc))
+		return -EINVAL;
+
+	/* Get MMIO base address from Endpoint controller */
+	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "mmio");
+	epf_mhi->mmio_phys = res->start;
+	epf_mhi->mmio_size = resource_size(res);
+
+	epf_mhi->mmio = ioremap(epf_mhi->mmio_phys, epf_mhi->mmio_size);
+	if (IS_ERR(epf_mhi->mmio))
+		return PTR_ERR(epf_mhi->mmio);
+
+	ret = platform_get_irq_byname(pdev, "doorbell");
+	if (ret < 0) {
+		dev_err(dev, "Failed to get Doorbell IRQ\n");
+		iounmap(epf_mhi->mmio);
+		return ret;
+	}
+
+	epf_mhi->irq = ret;
+
+	return 0;
+}
+
+static void pci_epf_mhi_unbind(struct pci_epf *epf)
+{
+	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
+	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
+	struct pci_epf_bar *epf_bar = &epf->bar[info->bar_num];
+	struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
+	struct pci_epc *epc = epf->epc;
+
+	/*
+	 * Forcefully power down the MHI EP stack. Only way to bring the MHI EP
+	 * stack back to working state after successive bind is by getting BME
+	 * from host.
+	 */
+	if (mhi_cntrl->mhi_dev) {
+		mhi_ep_power_down(mhi_cntrl);
+		mhi_ep_unregister_controller(mhi_cntrl);
+	}
+
+	iounmap(epf_mhi->mmio);
+	pci_epc_clear_bar(epc, epf->func_no, epf->vfunc_no, epf_bar);
+}
+
+static struct pci_epc_event_ops pci_epf_mhi_event_ops = {
+	.core_init = pci_epf_mhi_core_init,
+	.link_up = pci_epf_mhi_link_up,
+	.link_down = pci_epf_mhi_link_down,
+	.bme = pci_epf_mhi_bme,
+};
+
+static int pci_epf_mhi_probe(struct pci_epf *epf,
+			     const struct pci_epf_device_id *id)
+{
+	struct pci_epf_mhi_ep_info *info =
+			(struct pci_epf_mhi_ep_info *)id->driver_data;
+	struct pci_epf_mhi *epf_mhi;
+	struct device *dev = &epf->dev;
+
+	epf_mhi = devm_kzalloc(dev, sizeof(*epf_mhi), GFP_KERNEL);
+	if (!epf_mhi)
+		return -ENOMEM;
+
+	epf->header = info->epf_header;
+	epf_mhi->info = info;
+	epf_mhi->epf = epf;
+
+	epf->event_ops = &pci_epf_mhi_event_ops;
+
+	mutex_init(&epf_mhi->lock);
+
+	epf_set_drvdata(epf, epf_mhi);
+
+	return 0;
+}
+
+static const struct pci_epf_device_id pci_epf_mhi_ids[] = {
+	{
+		.name = "sdx55", .driver_data = (kernel_ulong_t)&sdx55_info,
+	},
+	{},
+};
+
+static struct pci_epf_ops pci_epf_mhi_ops = {
+	.unbind	= pci_epf_mhi_unbind,
+	.bind	= pci_epf_mhi_bind,
+};
+
+static struct pci_epf_driver pci_epf_mhi_driver = {
+	.driver.name	= "pci_epf_mhi",
+	.probe		= pci_epf_mhi_probe,
+	.id_table	= pci_epf_mhi_ids,
+	.ops		= &pci_epf_mhi_ops,
+	.owner		= THIS_MODULE,
+};
+
+static int __init pci_epf_mhi_init(void)
+{
+	return pci_epf_register_driver(&pci_epf_mhi_driver);
+}
+module_init(pci_epf_mhi_init);
+
+static void __exit pci_epf_mhi_exit(void)
+{
+	pci_epf_unregister_driver(&pci_epf_mhi_driver);
+}
+module_exit(pci_epf_mhi_exit);
+
+MODULE_DESCRIPTION("PCI EPF driver for MHI Endpoint devices");
+MODULE_AUTHOR("Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>");
+MODULE_LICENSE("GPL");
-- 
2.25.1

Re: [PATCH v5 9/9] PCI: endpoint: Add PCI Endpoint function driver for MHI bus

[Damien Le Moal]

On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> Add PCI Endpoint driver for the Qualcomm MHI (Modem Host Interface) bus.
> The driver implements the MHI function over PCI in the endpoint device
> such as SDX55 modem. The MHI endpoint function driver acts as a
> controller driver for the MHI Endpoint stack and carries out all PCI
> related activities like mapping the host memory using iATU, triggering
> MSIs etc...
> 
> Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>

Looks good, but not knowing this hardware, I only did a style review. I added
some nits below.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>


> ---
>  drivers/pci/endpoint/functions/Kconfig       |  10 +
>  drivers/pci/endpoint/functions/Makefile      |   1 +
>  drivers/pci/endpoint/functions/pci-epf-mhi.c | 462 +++++++++++++++++++
>  3 files changed, 473 insertions(+)
>  create mode 100644 drivers/pci/endpoint/functions/pci-epf-mhi.c
> 
> diff --git a/drivers/pci/endpoint/functions/Kconfig b/drivers/pci/endpoint/functions/Kconfig
> index 9fd560886871..f5171b4fabbe 100644
> --- a/drivers/pci/endpoint/functions/Kconfig
> +++ b/drivers/pci/endpoint/functions/Kconfig
> @@ -37,3 +37,13 @@ config PCI_EPF_VNTB
>  	  between PCI Root Port and PCIe Endpoint.
>  
>  	  If in doubt, say "N" to disable Endpoint NTB driver.
> +
> +config PCI_EPF_MHI
> +	tristate "PCI Endpoint driver for MHI bus"
> +	depends on PCI_ENDPOINT && MHI_BUS_EP
> +	help
> +	   Enable this configuration option to enable the PCI Endpoint
> +	   driver for Modem Host Interface (MHI) bus in Qualcomm Endpoint
> +	   devices such as SDX55.
> +
> +	   If in doubt, say "N" to disable Endpoint driver for MHI bus.
> diff --git a/drivers/pci/endpoint/functions/Makefile b/drivers/pci/endpoint/functions/Makefile
> index 5c13001deaba..696473fce50e 100644
> --- a/drivers/pci/endpoint/functions/Makefile
> +++ b/drivers/pci/endpoint/functions/Makefile
> @@ -6,3 +6,4 @@
>  obj-$(CONFIG_PCI_EPF_TEST)		+= pci-epf-test.o
>  obj-$(CONFIG_PCI_EPF_NTB)		+= pci-epf-ntb.o
>  obj-$(CONFIG_PCI_EPF_VNTB) 		+= pci-epf-vntb.o
> +obj-$(CONFIG_PCI_EPF_MHI)		+= pci-epf-mhi.o
> diff --git a/drivers/pci/endpoint/functions/pci-epf-mhi.c b/drivers/pci/endpoint/functions/pci-epf-mhi.c
> new file mode 100644
> index 000000000000..98f0d96cfd46
> --- /dev/null
> +++ b/drivers/pci/endpoint/functions/pci-epf-mhi.c
> @@ -0,0 +1,462 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * PCI EPF driver for MHI Endpoint devices
> + *
> + * Copyright (C) 2023 Linaro Ltd.
> + * Author: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> + */
> +
> +#include <linux/mhi_ep.h>
> +#include <linux/module.h>
> +#include <linux/platform_device.h>
> +#include <linux/pci-epc.h>
> +#include <linux/pci-epf.h>
> +
> +#define MHI_VERSION_1_0 0x01000000
> +
> +#define to_epf_mhi(cntrl) container_of(cntrl, struct pci_epf_mhi, cntrl)
> +
> +struct pci_epf_mhi_ep_info {
> +	const struct mhi_ep_cntrl_config *config;
> +	struct pci_epf_header *epf_header;
> +	enum pci_barno bar_num;
> +	u32 epf_flags;
> +	u32 msi_count;
> +	u32 mru;
> +};
> +
> +#define MHI_EP_CHANNEL_CONFIG(ch_num, ch_name, direction)	\
> +	{							\
> +		.num = ch_num,					\
> +		.name = ch_name,				\
> +		.dir = direction,				\
> +	}
> +
> +#define MHI_EP_CHANNEL_CONFIG_UL(ch_num, ch_name)		\
> +	MHI_EP_CHANNEL_CONFIG(ch_num, ch_name, DMA_TO_DEVICE)
> +
> +#define MHI_EP_CHANNEL_CONFIG_DL(ch_num, ch_name)		\
> +	MHI_EP_CHANNEL_CONFIG(ch_num, ch_name, DMA_FROM_DEVICE)
> +
> +static const struct mhi_ep_channel_config mhi_v1_channels[] = {
> +	MHI_EP_CHANNEL_CONFIG_UL(0, "LOOPBACK"),
> +	MHI_EP_CHANNEL_CONFIG_DL(1, "LOOPBACK"),
> +	MHI_EP_CHANNEL_CONFIG_UL(2, "SAHARA"),
> +	MHI_EP_CHANNEL_CONFIG_DL(3, "SAHARA"),
> +	MHI_EP_CHANNEL_CONFIG_UL(4, "DIAG"),
> +	MHI_EP_CHANNEL_CONFIG_DL(5, "DIAG"),
> +	MHI_EP_CHANNEL_CONFIG_UL(6, "SSR"),
> +	MHI_EP_CHANNEL_CONFIG_DL(7, "SSR"),
> +	MHI_EP_CHANNEL_CONFIG_UL(8, "QDSS"),
> +	MHI_EP_CHANNEL_CONFIG_DL(9, "QDSS"),
> +	MHI_EP_CHANNEL_CONFIG_UL(10, "EFS"),
> +	MHI_EP_CHANNEL_CONFIG_DL(11, "EFS"),
> +	MHI_EP_CHANNEL_CONFIG_UL(12, "MBIM"),
> +	MHI_EP_CHANNEL_CONFIG_DL(13, "MBIM"),
> +	MHI_EP_CHANNEL_CONFIG_UL(14, "QMI"),
> +	MHI_EP_CHANNEL_CONFIG_DL(15, "QMI"),
> +	MHI_EP_CHANNEL_CONFIG_UL(16, "QMI"),
> +	MHI_EP_CHANNEL_CONFIG_DL(17, "QMI"),
> +	MHI_EP_CHANNEL_CONFIG_UL(18, "IP-CTRL-1"),
> +	MHI_EP_CHANNEL_CONFIG_DL(19, "IP-CTRL-1"),
> +	MHI_EP_CHANNEL_CONFIG_UL(20, "IPCR"),
> +	MHI_EP_CHANNEL_CONFIG_DL(21, "IPCR"),
> +	MHI_EP_CHANNEL_CONFIG_UL(32, "DUN"),
> +	MHI_EP_CHANNEL_CONFIG_DL(33, "DUN"),
> +	MHI_EP_CHANNEL_CONFIG_UL(46, "IP_SW0"),
> +	MHI_EP_CHANNEL_CONFIG_DL(47, "IP_SW0"),
> +};
> +
> +static const struct mhi_ep_cntrl_config mhi_v1_config = {
> +	.max_channels = 128,
> +	.num_channels = ARRAY_SIZE(mhi_v1_channels),
> +	.ch_cfg = mhi_v1_channels,
> +	.mhi_version = MHI_VERSION_1_0,
> +};
> +
> +static struct pci_epf_header sdx55_header = {
> +	.vendorid = PCI_VENDOR_ID_QCOM,
> +	.deviceid = 0x0306,
> +	.baseclass_code = PCI_BASE_CLASS_COMMUNICATION,
> +	.subclass_code = PCI_CLASS_COMMUNICATION_MODEM & 0xff,
> +	.interrupt_pin	= PCI_INTERRUPT_INTA,
> +};
> +
> +static const struct pci_epf_mhi_ep_info sdx55_info = {
> +	.config = &mhi_v1_config,
> +	.epf_header = &sdx55_header,
> +	.bar_num = BAR_0,
> +	.epf_flags = PCI_BASE_ADDRESS_MEM_TYPE_32,
> +	.msi_count = 32,
> +	.mru = 0x8000,
> +};
> +
> +struct pci_epf_mhi {
> +	const struct pci_epf_mhi_ep_info *info;
> +	struct mhi_ep_cntrl mhi_cntrl;
> +	struct pci_epf *epf;
> +	struct mutex lock;
> +	void __iomem *mmio;
> +	resource_size_t mmio_phys;
> +	u32 mmio_size;
> +	int irq;
> +};
> +
> +static int pci_epf_mhi_alloc_map(struct mhi_ep_cntrl *mhi_cntrl, u64 pci_addr,
> +				 phys_addr_t *paddr, void __iomem **vaddr,
> +				 size_t size)
> +{
> +	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
> +	struct pci_epf *epf = epf_mhi->epf;
> +	struct pci_epc *epc = epf_mhi->epf->epc;
> +	size_t offset = pci_addr & (epc->mem->window.page_size - 1);
> +	void __iomem *__vaddr;
> +	phys_addr_t __paddr;
> +	int ret;
> +
> +	__vaddr = pci_epc_mem_alloc_addr(epc, &__paddr, size + offset);
> +	if (!__vaddr)
> +		return -ENOMEM;
> +
> +	ret = pci_epc_map_addr(epc, epf->func_no, epf->vfunc_no, __paddr,
> +			       pci_addr - offset, size + offset);
> +	if (ret) {
> +		pci_epc_mem_free_addr(epc, __paddr, __vaddr, size + offset);
> +

Drop the blank line here.

> +		return ret;
> +	}
> +
> +	*paddr = __paddr + offset;
> +	*vaddr = __vaddr + offset;
> +
> +	return 0;
> +}
> +
> +static void pci_epf_mhi_unmap_free(struct mhi_ep_cntrl *mhi_cntrl, u64 pci_addr,
> +				   phys_addr_t paddr, void __iomem *vaddr,
> +				   size_t size)
> +{
> +	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
> +	struct pci_epf *epf = epf_mhi->epf;
> +	struct pci_epc *epc = epf->epc;
> +	size_t offset = pci_addr & (epc->mem->window.page_size - 1);
> +
> +	pci_epc_unmap_addr(epc, epf->func_no, epf->vfunc_no, paddr - offset);
> +	pci_epc_mem_free_addr(epc, paddr - offset, vaddr - offset,
> +			      size + offset);
> +}
> +
> +static void pci_epf_mhi_raise_irq(struct mhi_ep_cntrl *mhi_cntrl, u32 vector)
> +{
> +	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
> +	struct pci_epf *epf = epf_mhi->epf;
> +	struct pci_epc *epc = epf->epc;
> +
> +	/*
> +	 * MHI supplies 0 based MSI vector but the API expects the vector to be
> +	 * 1 based, so we need to increment the vector by 1.
> +	 */
> +	pci_epc_raise_irq(epc, epf->func_no, epf->vfunc_no, PCI_EPC_IRQ_MSI,
> +			  vector + 1);
> +}
> +
> +static int pci_epf_mhi_read_from_host(struct mhi_ep_cntrl *mhi_cntrl, u64 from,
> +				      void __iomem *to, size_t size)
> +{
> +	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
> +	struct pci_epf *epf = epf_mhi->epf;
> +	struct pci_epc *epc = epf_mhi->epf->epc;
> +	void __iomem *tre_buf;
> +	phys_addr_t tre_phys;
> +	size_t offset = from % SZ_4K;
> +	int ret;
> +
> +	mutex_lock(&epf_mhi->lock);
> +
> +	tre_buf = pci_epc_mem_alloc_addr(epc, &tre_phys, size + offset);
> +	if (!tre_buf) {
> +		ret = -ENOMEM;
> +		goto err_unlock;
> +	}
> +
> +	ret = pci_epc_map_addr(epc, epf->func_no, epf->vfunc_no, tre_phys,
> +			       from - offset, size + offset);
> +	if (ret) {
> +		pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
> +		goto err_unlock;
> +	}
> +
> +	memcpy_fromio(to, tre_buf + offset, size);
> +
> +	pci_epc_unmap_addr(epc, epf->func_no, epf->vfunc_no, tre_phys);
> +	pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
> +
> +err_unlock:
> +	mutex_unlock(&epf_mhi->lock);
> +
> +	return ret;
> +}
> +
> +static int pci_epf_mhi_write_to_host(struct mhi_ep_cntrl *mhi_cntrl,
> +				     void __iomem *from, u64 to, size_t size)
> +{
> +	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
> +	struct pci_epf *epf = epf_mhi->epf;
> +	struct pci_epc *epc = epf_mhi->epf->epc;
> +	void __iomem *tre_buf;
> +	phys_addr_t tre_phys;
> +	size_t offset = to % SZ_4K;
> +	int ret;
> +
> +	mutex_lock(&epf_mhi->lock);
> +
> +	tre_buf = pci_epc_mem_alloc_addr(epc, &tre_phys, size + offset);
> +	if (!tre_buf) {
> +		ret = -ENOMEM;
> +		goto err_unlock;
> +	}
> +
> +	ret = pci_epc_map_addr(epc, epf->func_no, epf->vfunc_no, tre_phys,
> +			       to - offset, size + offset);
> +	if (ret) {
> +		pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
> +		goto err_unlock;
> +	}

The above 2 hunks are that same as in pci_epf_mhi_read_from_host(). May be pack
these in a helper function ? E.g. pci_epf_mhi_map() ?
Note that I have a patch series in the work that does that at the pci_epc API
level (pci_epc_map() + pci_epc_unmap()) to simplify function drivers.

> +
> +	memcpy_toio(tre_buf + offset, from, size);
> +
> +	pci_epc_unmap_addr(epc, epf->func_no, epf->vfunc_no, tre_phys);
> +	pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);

Same here: pci_epf_mhi_unmap() ?

> +
> +err_unlock:
> +	mutex_unlock(&epf_mhi->lock);
> +
> +	return ret;
> +}
> +
> +static int pci_epf_mhi_core_init(struct pci_epf *epf)
> +{
> +	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
> +	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
> +	struct pci_epf_bar *epf_bar = &epf->bar[info->bar_num];
> +	struct pci_epc *epc = epf->epc;
> +	struct device *dev = &epf->dev;
> +	int ret;
> +
> +	epf_bar->phys_addr = epf_mhi->mmio_phys;
> +	epf_bar->size = epf_mhi->mmio_size;
> +	epf_bar->barno = info->bar_num;
> +	epf_bar->flags = info->epf_flags;
> +	ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, epf_bar);
> +	if (ret) {
> +		dev_err(dev, "Failed to set BAR: %d\n", ret);
> +		return ret;
> +	}
> +
> +	ret = pci_epc_set_msi(epc, epf->func_no, epf->vfunc_no,
> +			      order_base_2(info->msi_count));
> +	if (ret) {
> +		dev_err(dev, "Failed to set MSI configuration: %d\n", ret);
> +		return ret;
> +	}
> +
> +	ret = pci_epc_write_header(epc, epf->func_no, epf->vfunc_no,
> +				   epf->header);
> +	if (ret) {
> +		dev_err(dev, "Failed to set Configuration header: %d\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int pci_epf_mhi_link_up(struct pci_epf *epf)
> +{
> +	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
> +	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
> +	struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
> +	struct pci_epc *epc = epf->epc;
> +	struct device *dev = &epf->dev;
> +	int ret;
> +
> +	mhi_cntrl->mmio = epf_mhi->mmio;
> +	mhi_cntrl->irq = epf_mhi->irq;
> +	mhi_cntrl->mru = info->mru;
> +
> +	/* Assign the struct dev of PCI EP as MHI controller device */
> +	mhi_cntrl->cntrl_dev = epc->dev.parent;
> +	mhi_cntrl->raise_irq = pci_epf_mhi_raise_irq;
> +	mhi_cntrl->alloc_map = pci_epf_mhi_alloc_map;
> +	mhi_cntrl->unmap_free = pci_epf_mhi_unmap_free;
> +	mhi_cntrl->read_from_host = pci_epf_mhi_read_from_host;
> +	mhi_cntrl->write_to_host = pci_epf_mhi_write_to_host;
> +
> +	/* Register the MHI EP controller */
> +	ret = mhi_ep_register_controller(mhi_cntrl, info->config);
> +	if (ret) {
> +		dev_err(dev, "Failed to register MHI EP controller: %d\n", ret);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static int pci_epf_mhi_link_down(struct pci_epf *epf)
> +{
> +	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
> +	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
> +	struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
> +
> +	if (mhi_cntrl->mhi_dev) {
> +		mhi_ep_power_down(mhi_cntrl);
> +		mhi_ep_unregister_controller(mhi_cntrl);
> +	}
> +
> +	return 0;
> +}
> +
> +static int pci_epf_mhi_bme(struct pci_epf *epf)
> +{
> +	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
> +	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
> +	struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
> +	struct device *dev = &epf->dev;
> +	int ret;
> +
> +	/*
> +	 * Power up the MHI EP stack if link is up and stack is in power down
> +	 * state.
> +	 */
> +	if (!mhi_cntrl->enabled && mhi_cntrl->mhi_dev) {
> +		ret = mhi_ep_power_up(mhi_cntrl);
> +		if (ret) {
> +			dev_err(dev, "Failed to power up MHI EP: %d\n", ret);
> +			mhi_ep_unregister_controller(mhi_cntrl);
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int pci_epf_mhi_bind(struct pci_epf *epf)
> +{
> +	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
> +	struct pci_epc *epc = epf->epc;
> +	struct platform_device *pdev = to_platform_device(epc->dev.parent);
> +	struct device *dev = &epf->dev;
> +	struct resource *res;
> +	int ret;
> +
> +	if (WARN_ON_ONCE(!epc))
> +		return -EINVAL;

This should never happen. If there is a possibility that the EPC core code calls
bind with epc == NULL, we need to fix that there.

> +
> +	/* Get MMIO base address from Endpoint controller */
> +	res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "mmio");
> +	epf_mhi->mmio_phys = res->start;
> +	epf_mhi->mmio_size = resource_size(res);
> +
> +	epf_mhi->mmio = ioremap(epf_mhi->mmio_phys, epf_mhi->mmio_size);
> +	if (IS_ERR(epf_mhi->mmio))
> +		return PTR_ERR(epf_mhi->mmio);
> +
> +	ret = platform_get_irq_byname(pdev, "doorbell");
> +	if (ret < 0) {
> +		dev_err(dev, "Failed to get Doorbell IRQ\n");
> +		iounmap(epf_mhi->mmio);
> +		return ret;
> +	}
> +
> +	epf_mhi->irq = ret;
> +
> +	return 0;
> +}
> +
> +static void pci_epf_mhi_unbind(struct pci_epf *epf)
> +{
> +	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
> +	const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
> +	struct pci_epf_bar *epf_bar = &epf->bar[info->bar_num];
> +	struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
> +	struct pci_epc *epc = epf->epc;
> +
> +	/*
> +	 * Forcefully power down the MHI EP stack. Only way to bring the MHI EP
> +	 * stack back to working state after successive bind is by getting BME
> +	 * from host.
> +	 */
> +	if (mhi_cntrl->mhi_dev) {
> +		mhi_ep_power_down(mhi_cntrl);
> +		mhi_ep_unregister_controller(mhi_cntrl);
> +	}
> +
> +	iounmap(epf_mhi->mmio);
> +	pci_epc_clear_bar(epc, epf->func_no, epf->vfunc_no, epf_bar);
> +}
> +
> +static struct pci_epc_event_ops pci_epf_mhi_event_ops = {
> +	.core_init = pci_epf_mhi_core_init,
> +	.link_up = pci_epf_mhi_link_up,
> +	.link_down = pci_epf_mhi_link_down,
> +	.bme = pci_epf_mhi_bme,
> +};
> +
> +static int pci_epf_mhi_probe(struct pci_epf *epf,
> +			     const struct pci_epf_device_id *id)
> +{
> +	struct pci_epf_mhi_ep_info *info =
> +			(struct pci_epf_mhi_ep_info *)id->driver_data;
> +	struct pci_epf_mhi *epf_mhi;
> +	struct device *dev = &epf->dev;
> +
> +	epf_mhi = devm_kzalloc(dev, sizeof(*epf_mhi), GFP_KERNEL);
> +	if (!epf_mhi)
> +		return -ENOMEM;
> +
> +	epf->header = info->epf_header;
> +	epf_mhi->info = info;
> +	epf_mhi->epf = epf;
> +
> +	epf->event_ops = &pci_epf_mhi_event_ops;
> +
> +	mutex_init(&epf_mhi->lock);
> +
> +	epf_set_drvdata(epf, epf_mhi);
> +
> +	return 0;
> +}
> +
> +static const struct pci_epf_device_id pci_epf_mhi_ids[] = {
> +	{
> +		.name = "sdx55", .driver_data = (kernel_ulong_t)&sdx55_info,
> +	},
> +	{},
> +};
> +
> +static struct pci_epf_ops pci_epf_mhi_ops = {
> +	.unbind	= pci_epf_mhi_unbind,
> +	.bind	= pci_epf_mhi_bind,
> +};
> +
> +static struct pci_epf_driver pci_epf_mhi_driver = {
> +	.driver.name	= "pci_epf_mhi",
> +	.probe		= pci_epf_mhi_probe,
> +	.id_table	= pci_epf_mhi_ids,
> +	.ops		= &pci_epf_mhi_ops,
> +	.owner		= THIS_MODULE,
> +};
> +
> +static int __init pci_epf_mhi_init(void)
> +{
> +	return pci_epf_register_driver(&pci_epf_mhi_driver);
> +}
> +module_init(pci_epf_mhi_init);
> +
> +static void __exit pci_epf_mhi_exit(void)
> +{
> +	pci_epf_unregister_driver(&pci_epf_mhi_driver);
> +}
> +module_exit(pci_epf_mhi_exit);
> +
> +MODULE_DESCRIPTION("PCI EPF driver for MHI Endpoint devices");
> +MODULE_AUTHOR("Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>");
> +MODULE_LICENSE("GPL");

-- 
Damien Le Moal
Western Digital Research

Re: [PATCH v5 9/9] PCI: endpoint: Add PCI Endpoint function driver for MHI bus

[Manivannan Sadhasivam]

On Fri, Jun 02, 2023 at 08:31:31AM +0900, Damien Le Moal wrote:
> On 6/1/23 23:57, Manivannan Sadhasivam wrote:
> > Add PCI Endpoint driver for the Qualcomm MHI (Modem Host Interface) bus.
> > The driver implements the MHI function over PCI in the endpoint device
> > such as SDX55 modem. The MHI endpoint function driver acts as a
> > controller driver for the MHI Endpoint stack and carries out all PCI
> > related activities like mapping the host memory using iATU, triggering
> > MSIs etc...
> > 
> > Reviewed-by: Kishon Vijay Abraham I <kishon@kernel.org>
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> 
> Looks good, but not knowing this hardware, I only did a style review. I added
> some nits below.
> 
> Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
> 
> 
> > ---
> >  drivers/pci/endpoint/functions/Kconfig       |  10 +
> >  drivers/pci/endpoint/functions/Makefile      |   1 +
> >  drivers/pci/endpoint/functions/pci-epf-mhi.c | 462 +++++++++++++++++++
> >  3 files changed, 473 insertions(+)
> >  create mode 100644 drivers/pci/endpoint/functions/pci-epf-mhi.c
> > 
> > diff --git a/drivers/pci/endpoint/functions/Kconfig b/drivers/pci/endpoint/functions/Kconfig
> > index 9fd560886871..f5171b4fabbe 100644
> > --- a/drivers/pci/endpoint/functions/Kconfig
> > +++ b/drivers/pci/endpoint/functions/Kconfig
> > @@ -37,3 +37,13 @@ config PCI_EPF_VNTB
> >  	  between PCI Root Port and PCIe Endpoint.
> >  
> >  	  If in doubt, say "N" to disable Endpoint NTB driver.
> > +
> > +config PCI_EPF_MHI
> > +	tristate "PCI Endpoint driver for MHI bus"
> > +	depends on PCI_ENDPOINT && MHI_BUS_EP
> > +	help
> > +	   Enable this configuration option to enable the PCI Endpoint
> > +	   driver for Modem Host Interface (MHI) bus in Qualcomm Endpoint
> > +	   devices such as SDX55.
> > +
> > +	   If in doubt, say "N" to disable Endpoint driver for MHI bus.
> > diff --git a/drivers/pci/endpoint/functions/Makefile b/drivers/pci/endpoint/functions/Makefile
> > index 5c13001deaba..696473fce50e 100644
> > --- a/drivers/pci/endpoint/functions/Makefile
> > +++ b/drivers/pci/endpoint/functions/Makefile
> > @@ -6,3 +6,4 @@
> >  obj-$(CONFIG_PCI_EPF_TEST)		+= pci-epf-test.o
> >  obj-$(CONFIG_PCI_EPF_NTB)		+= pci-epf-ntb.o
> >  obj-$(CONFIG_PCI_EPF_VNTB) 		+= pci-epf-vntb.o
> > +obj-$(CONFIG_PCI_EPF_MHI)		+= pci-epf-mhi.o
> > diff --git a/drivers/pci/endpoint/functions/pci-epf-mhi.c b/drivers/pci/endpoint/functions/pci-epf-mhi.c
> > new file mode 100644
> > index 000000000000..98f0d96cfd46
> > --- /dev/null
> > +++ b/drivers/pci/endpoint/functions/pci-epf-mhi.c

[...]

> > +static int pci_epf_mhi_write_to_host(struct mhi_ep_cntrl *mhi_cntrl,
> > +				     void __iomem *from, u64 to, size_t size)
> > +{
> > +	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
> > +	struct pci_epf *epf = epf_mhi->epf;
> > +	struct pci_epc *epc = epf_mhi->epf->epc;
> > +	void __iomem *tre_buf;
> > +	phys_addr_t tre_phys;
> > +	size_t offset = to % SZ_4K;
> > +	int ret;
> > +
> > +	mutex_lock(&epf_mhi->lock);
> > +
> > +	tre_buf = pci_epc_mem_alloc_addr(epc, &tre_phys, size + offset);
> > +	if (!tre_buf) {
> > +		ret = -ENOMEM;
> > +		goto err_unlock;
> > +	}
> > +
> > +	ret = pci_epc_map_addr(epc, epf->func_no, epf->vfunc_no, tre_phys,
> > +			       to - offset, size + offset);
> > +	if (ret) {
> > +		pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
> > +		goto err_unlock;
> > +	}
> 
> The above 2 hunks are that same as in pci_epf_mhi_read_from_host(). May be pack
> these in a helper function ? E.g. pci_epf_mhi_map() ?
> Note that I have a patch series in the work that does that at the pci_epc API
> level (pci_epc_map() + pci_epc_unmap()) to simplify function drivers.
> 
> > +
> > +	memcpy_toio(tre_buf + offset, from, size);
> > +
> > +	pci_epc_unmap_addr(epc, epf->func_no, epf->vfunc_no, tre_phys);
> > +	pci_epc_mem_free_addr(epc, tre_phys, tre_buf, size + offset);
> 
> Same here: pci_epf_mhi_unmap() ?
> 

Agree with both of the above comments. At have this change in my local tree but
that's not part of this series. Will merge it to this patch itself.

> > +
> > +err_unlock:
> > +	mutex_unlock(&epf_mhi->lock);
> > +
> > +	return ret;
> > +}
> > +

[...]

> > +static int pci_epf_mhi_bind(struct pci_epf *epf)
> > +{
> > +	struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
> > +	struct pci_epc *epc = epf->epc;
> > +	struct platform_device *pdev = to_platform_device(epc->dev.parent);
> > +	struct device *dev = &epf->dev;
> > +	struct resource *res;
> > +	int ret;
> > +
> > +	if (WARN_ON_ONCE(!epc))
> > +		return -EINVAL;
> 
> This should never happen. If there is a possibility that the EPC core code calls
> bind with epc == NULL, we need to fix that there.
> 

Right, this seems to be redundant.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH v5 9/9] PCI: endpoint: Add PCI Endpoint function driver for MHI bus

[kernel test robot]

Hi Manivannan,

kernel test robot noticed the following build warnings:

[auto build test WARNING on pci/next]
[also build test WARNING on pci/for-linus jonmason-ntb/ntb-next linus/master v6.4-rc4 next-20230601]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Manivannan-Sadhasivam/MAINTAINERS-Add-entry-for-MHI-networking-drivers-under-MHI-bus/20230601-232444
base:   https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git next
patch link:    https://lore.kernel.org/r/20230601145718.12204-10-manivannan.sadhasivam%40linaro.org
patch subject: [PATCH v5 9/9] PCI: endpoint: Add PCI Endpoint function driver for MHI bus
config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20230602/202306021200.ndNlhtdv-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build):
        # https://github.com/intel-lab-lkp/linux/commit/b6931b37d070b86af64eeb92c1d52b4a0c6d61e8
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Manivannan-Sadhasivam/MAINTAINERS-Add-entry-for-MHI-networking-drivers-under-MHI-bus/20230601-232444
        git checkout b6931b37d070b86af64eeb92c1d52b4a0c6d61e8
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        make W=1 O=build_dir ARCH=x86_64 olddefconfig
        make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash drivers/pci/endpoint/functions/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306021200.ndNlhtdv-lkp@intel.com/

All warnings (new ones prefixed by >>):

   drivers/pci/endpoint/functions/pci-epf-mhi.c: In function 'pci_epf_mhi_link_down':
>> drivers/pci/endpoint/functions/pci-epf-mhi.c:307:43: warning: unused variable 'info' [-Wunused-variable]
     307 |         const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
         |                                           ^~~~
   drivers/pci/endpoint/functions/pci-epf-mhi.c: In function 'pci_epf_mhi_bme':
   drivers/pci/endpoint/functions/pci-epf-mhi.c:321:43: warning: unused variable 'info' [-Wunused-variable]
     321 |         const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
         |                                           ^~~~


vim +/info +307 drivers/pci/endpoint/functions/pci-epf-mhi.c

   303	
   304	static int pci_epf_mhi_link_down(struct pci_epf *epf)
   305	{
   306		struct pci_epf_mhi *epf_mhi = epf_get_drvdata(epf);
 > 307		const struct pci_epf_mhi_ep_info *info = epf_mhi->info;
   308		struct mhi_ep_cntrl *mhi_cntrl = &epf_mhi->mhi_cntrl;
   309	
   310		if (mhi_cntrl->mhi_dev) {
   311			mhi_ep_power_down(mhi_cntrl);
   312			mhi_ep_unregister_controller(mhi_cntrl);
   313		}
   314	
   315		return 0;
   316	}
   317	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki