[PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP

[PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

V3:
 Drop the dependency in V2.
 Tested on i.MX8QM/8MM
 Use 'fsl,resource-id' and 'fsl,entry-address' Per dt maintainer
 Drop 'reg' property Per remoteproc maintainer
 Drop mcore self recovery, until we land in common framework support.

V2:
 https://patchwork.kernel.org/project/linux-remoteproc/cover/20220309102118.8131-1-peng.fan@oss.nxp.com/
 Depends on https://patchwork.kernel.org/project/linux-remoteproc/list/?series=621311
 Tested on i.MX8QXP/QM/8MP
 Addressed Mathieu's comments
 Drop V1 patch 5/9, patch 3/9 is replaced with upper dependency patchset
 Move V1 patch 4/9 out to https://patchwork.kernel.org/project/linux-remoteproc/patch/20220308065754.3355-1-peng.fan@oss.nxp.com/
 Update commit log
 Drop magic number to get entry address from device tree in patch 4/6

The V1 patchset:
https://patchwork.kernel.org/project/linux-remoteproc/patch/20220111033333.403448-4-peng.fan@oss.nxp.com/

Peng Fan (6):
  dt-bindings: remoteproc: imx_rproc: support i.MX8QXP
  dt-bindings: remoteproc: imx_rproc: support i.MX8QM
  remoteproc: imx_rproc: support attaching to i.MX8QXP M4
  remoteproc: imx_rproc: support kicking Mcore from Linux for i.MX8QXP
  remoteproc: imx_rproc: support i.MX8QM
  remoteproc: imx_rproc: request mbox channel later

 .../bindings/remoteproc/fsl,imx-rproc.yaml    |  16 ++
 drivers/remoteproc/imx_rproc.c                | 222 +++++++++++++++++-
 2 files changed, 231 insertions(+), 7 deletions(-)

-- 
2.25.1

[PATCH V3 1/6] dt-bindings: remoteproc: imx_rproc: support i.MX8QXP

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Add i.MX8QXP compatible

Add a new property fsl,resource-id for SoC which supports SCFW.
This property is used to check whether remote process is under control
of Linux or not.

Add fsl,entry-address to specify the entry address which used by SCFW
to kick M4.

To i.MX8QM/QXP, when M4 is in the same hardware partition with Cortex-A
cores, need power up M4 through SCFW, then M4 could start. So introduce
power-domains property

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 .../bindings/remoteproc/fsl,imx-rproc.yaml        | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml b/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
index 3a1f59ad79e2..70322e57b6ff 100644
--- a/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
+++ b/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
@@ -22,6 +22,7 @@ properties:
       - fsl,imx8mn-cm7
       - fsl,imx8mp-cm7
       - fsl,imx8mq-cm4
+      - fsl,imx8qxp-cm4
       - fsl,imx8ulp-cm33
       - fsl,imx93-cm33
 
@@ -54,12 +55,26 @@ properties:
     minItems: 1
     maxItems: 32
 
+  power-domains:
+    maxItems: 8
+
   fsl,auto-boot:
     $ref: /schemas/types.yaml#/definitions/flag
     description:
       Indicate whether need to load the default firmware and start the remote
       processor automatically.
 
+  fsl,entry-address:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description:
+      Specify CPU entry address for SCU enabled processor.
+
+  fsl,resource-id:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description:
+      This property is to specify the resource id of the remote processor in SoC
+      which supports SCFW
+
 required:
   - compatible
 
-- 
2.25.1

[PATCH V3 2/6] dt-bindings: remoteproc: imx_rproc: support i.MX8QM

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Add i.MX8QM compatible

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml b/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
index 70322e57b6ff..64e783234e38 100644
--- a/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
+++ b/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
@@ -23,6 +23,7 @@ properties:
       - fsl,imx8mp-cm7
       - fsl,imx8mq-cm4
       - fsl,imx8qxp-cm4
+      - fsl,imx8qm-cm4
       - fsl,imx8ulp-cm33
       - fsl,imx93-cm33
 
-- 
2.25.1

[PATCH V3 3/6] remoteproc: imx_rproc: support attaching to i.MX8QXP M4

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

When M4 is kicked by SCFW, M4 runs in its own hardware partition, Linux
could only do IPC with M4, it could not start, stop, update image.

We disable recovery reboot when M4 is managed by SCFW, because
remoteproc core still not support M4 auto-recovery without loading
image.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/remoteproc/imx_rproc.c | 88 ++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)

diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
index 4a3352821b1d..2e751fc90760 100644
--- a/drivers/remoteproc/imx_rproc.c
+++ b/drivers/remoteproc/imx_rproc.c
@@ -6,6 +6,7 @@
 #include <linux/arm-smccc.h>
 #include <linux/clk.h>
 #include <linux/err.h>
+#include <linux/firmware/imx/sci.h>
 #include <linux/interrupt.h>
 #include <linux/kernel.h>
 #include <linux/mailbox_client.h>
@@ -59,6 +60,8 @@
 #define IMX_SIP_RPROC_STARTED		0x01
 #define IMX_SIP_RPROC_STOP		0x02
 
+#define	IMX_SC_IRQ_GROUP_REBOOTED	5
+
 /**
  * struct imx_rproc_mem - slim internal memory structure
  * @cpu_addr: MPU virtual address of the memory region
@@ -89,6 +92,10 @@ struct imx_rproc {
 	struct work_struct		rproc_work;
 	struct workqueue_struct		*workqueue;
 	void __iomem			*rsc_table;
+	struct imx_sc_ipc		*ipc_handle;
+	struct notifier_block		rproc_nb;
+	u32				rproc_pt;	/* partition id */
+	u32				rsrc_id;	/* resource id */
 };
 
 static const struct imx_rproc_att imx_rproc_att_imx93[] = {
@@ -117,6 +124,18 @@ static const struct imx_rproc_att imx_rproc_att_imx93[] = {
 	{ 0xD0000000, 0xa0000000, 0x10000000, 0 },
 };
 
+static const struct imx_rproc_att imx_rproc_att_imx8qxp[] = {
+	{ 0x08000000, 0x08000000, 0x10000000, 0 },
+	/* TCML/U */
+	{ 0x1FFE0000, 0x34FE0000, 0x00040000, ATT_OWN | ATT_IOMEM },
+	/* OCRAM(Low 96KB) */
+	{ 0x21000000, 0x00100000, 0x00018000, 0 },
+	/* OCRAM */
+	{ 0x21100000, 0x00100000, 0x00040000, 0 },
+	/* DDR (Data) */
+	{ 0x80000000, 0x80000000, 0x60000000, 0 },
+};
+
 static const struct imx_rproc_att imx_rproc_att_imx8mn[] = {
 	/* dev addr , sys addr  , size	    , flags */
 	/* ITCM   */
@@ -255,6 +274,12 @@ static const struct imx_rproc_dcfg imx_rproc_cfg_imx8mq = {
 	.method		= IMX_RPROC_MMIO,
 };
 
+static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qxp = {
+	.att		= imx_rproc_att_imx8qxp,
+	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8qxp),
+	.method		= IMX_RPROC_SCU_API,
+};
+
 static const struct imx_rproc_dcfg imx_rproc_cfg_imx8ulp = {
 	.att		= imx_rproc_att_imx8ulp,
 	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8ulp),
@@ -683,6 +708,22 @@ static void imx_rproc_free_mbox(struct rproc *rproc)
 	mbox_free_channel(priv->rx_ch);
 }
 
+static int imx_rproc_partition_notify(struct notifier_block *nb,
+				      unsigned long event, void *group)
+{
+	struct imx_rproc *priv = container_of(nb, struct imx_rproc, rproc_nb);
+
+	/* Ignore other irqs */
+	if (!((event & BIT(priv->rproc_pt)) && (*(u8 *)group == IMX_SC_IRQ_GROUP_REBOOTED)))
+		return 0;
+
+	rproc_report_crash(priv->rproc, RPROC_WATCHDOG);
+
+	pr_info("Partition%d reset!\n", priv->rproc_pt);
+
+	return 0;
+}
+
 static int imx_rproc_detect_mode(struct imx_rproc *priv)
 {
 	struct regmap_config config = { .name = "imx-rproc" };
@@ -692,6 +733,7 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
 	struct arm_smccc_res res;
 	int ret;
 	u32 val;
+	u8 pt;
 
 	switch (dcfg->method) {
 	case IMX_RPROC_NONE:
@@ -702,6 +744,51 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
 		if (res.a0)
 			priv->rproc->state = RPROC_DETACHED;
 		return 0;
+	case IMX_RPROC_SCU_API:
+		ret = imx_scu_get_handle(&priv->ipc_handle);
+		if (ret)
+			return ret;
+		ret = of_property_read_u32(dev->of_node, "fsl,resource-id", &priv->rsrc_id);
+		if (ret) {
+			dev_err(dev, "No fsl,resource-id property\n");
+			return ret;
+		}
+
+		/*
+		 * If Mcore resource is not owned by Acore partition, It is kicked by ROM,
+		 * and Linux could only do IPC with Mcore and nothing else.
+		 */
+		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id))
+			return 0;
+
+		priv->rproc->state = RPROC_DETACHED;
+		priv->rproc->recovery_disabled = true;
+
+		/* Get partition id and enable irq in SCFW */
+		ret = imx_sc_rm_get_resource_owner(priv->ipc_handle, priv->rsrc_id, &pt);
+		if (ret) {
+			dev_err(dev, "not able to get resource owner\n");
+			return ret;
+		}
+
+		priv->rproc_pt = pt;
+		priv->rproc_nb.notifier_call = imx_rproc_partition_notify;
+
+		ret = imx_scu_irq_register_notifier(&priv->rproc_nb);
+		if (ret) {
+			dev_warn(dev, "register scu notifier failed.\n");
+			return ret;
+		}
+
+		ret = imx_scu_irq_group_enable(IMX_SC_IRQ_GROUP_REBOOTED, BIT(priv->rproc_pt),
+					       true);
+		if (ret) {
+			imx_scu_irq_unregister_notifier(&priv->rproc_nb);
+			dev_warn(dev, "Enable irq failed.\n");
+			return ret;
+		}
+
+		return 0;
 	default:
 		break;
 	}
@@ -855,6 +942,7 @@ static const struct of_device_id imx_rproc_of_match[] = {
 	{ .compatible = "fsl,imx8mm-cm4", .data = &imx_rproc_cfg_imx8mq },
 	{ .compatible = "fsl,imx8mn-cm7", .data = &imx_rproc_cfg_imx8mn },
 	{ .compatible = "fsl,imx8mp-cm7", .data = &imx_rproc_cfg_imx8mn },
+	{ .compatible = "fsl,imx8qxp-cm4", .data = &imx_rproc_cfg_imx8qxp },
 	{ .compatible = "fsl,imx8ulp-cm33", .data = &imx_rproc_cfg_imx8ulp },
 	{ .compatible = "fsl,imx93-cm33", .data = &imx_rproc_cfg_imx93 },
 	{},
-- 
2.25.1

[PATCH V3 4/6] remoteproc: imx_rproc: support kicking Mcore from Linux for i.MX8QXP

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

When M4 is in the same hardware partition with Cortex-A, it
could be start/stop by Linux.

Added power domain to make sure M4 could run, it requires several power
domains to work. Make clk always optional for i.MX8QXP, because
SCFW handles it when power up M4 core.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/remoteproc/imx_rproc.c | 69 +++++++++++++++++++++++++++++++++-
 1 file changed, 67 insertions(+), 2 deletions(-)

diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
index 2e751fc90760..49cce9dd55c7 100644
--- a/drivers/remoteproc/imx_rproc.c
+++ b/drivers/remoteproc/imx_rproc.c
@@ -16,6 +16,7 @@
 #include <linux/of_reserved_mem.h>
 #include <linux/of_device.h>
 #include <linux/platform_device.h>
+#include <linux/pm_domain.h>
 #include <linux/regmap.h>
 #include <linux/remoteproc.h>
 #include <linux/workqueue.h>
@@ -96,6 +97,10 @@ struct imx_rproc {
 	struct notifier_block		rproc_nb;
 	u32				rproc_pt;	/* partition id */
 	u32				rsrc_id;	/* resource id */
+	u32				entry;		/* cpu start address */
+	int                             num_pd;
+	struct device                   **pd_dev;
+	struct device_link              **pd_dev_link;
 };
 
 static const struct imx_rproc_att imx_rproc_att_imx93[] = {
@@ -335,6 +340,9 @@ static int imx_rproc_start(struct rproc *rproc)
 		arm_smccc_smc(IMX_SIP_RPROC, IMX_SIP_RPROC_START, 0, 0, 0, 0, 0, 0, &res);
 		ret = res.a0;
 		break;
+	case IMX_RPROC_SCU_API:
+		ret = imx_sc_pm_cpu_start(priv->ipc_handle, priv->rsrc_id, true, priv->entry);
+		break;
 	default:
 		return -EOPNOTSUPP;
 	}
@@ -364,6 +372,9 @@ static int imx_rproc_stop(struct rproc *rproc)
 		if (res.a1)
 			dev_info(dev, "Not in wfi, force stopped\n");
 		break;
+	case IMX_RPROC_SCU_API:
+		ret = imx_sc_pm_cpu_start(priv->ipc_handle, priv->rsrc_id, false, priv->entry);
+		break;
 	default:
 		return -EOPNOTSUPP;
 	}
@@ -724,6 +735,56 @@ static int imx_rproc_partition_notify(struct notifier_block *nb,
 	return 0;
 }
 
+static int imx_rproc_attach_pd(struct imx_rproc *priv)
+{
+	struct device *dev = priv->dev;
+	int ret, i;
+
+	priv->num_pd = of_count_phandle_with_args(dev->of_node, "power-domains",
+						  "#power-domain-cells");
+	if (priv->num_pd < 0)
+		return priv->num_pd;
+
+	if (!priv->num_pd)
+		return 0;
+
+	priv->pd_dev = devm_kmalloc_array(dev, priv->num_pd, sizeof(*priv->pd_dev), GFP_KERNEL);
+	if (!priv->pd_dev)
+		return -ENOMEM;
+
+	priv->pd_dev_link = devm_kmalloc_array(dev, priv->num_pd, sizeof(*priv->pd_dev_link),
+					       GFP_KERNEL);
+
+	if (!priv->pd_dev_link)
+		return -ENOMEM;
+
+	for (i = 0; i < priv->num_pd; i++) {
+		priv->pd_dev[i] = dev_pm_domain_attach_by_id(dev, i);
+		if (IS_ERR(priv->pd_dev[i])) {
+			ret = PTR_ERR(priv->pd_dev[i]);
+			goto detach_pd;
+		}
+
+		priv->pd_dev_link[i] = device_link_add(dev, priv->pd_dev[i], DL_FLAG_STATELESS |
+						       DL_FLAG_PM_RUNTIME | DL_FLAG_RPM_ACTIVE);
+		if (!priv->pd_dev_link[i]) {
+			dev_pm_domain_detach(priv->pd_dev[i], false);
+			ret = -EINVAL;
+			goto detach_pd;
+		}
+	}
+
+	return 0;
+
+detach_pd:
+	while (--i >= 0) {
+		device_link_del(priv->pd_dev_link[i]);
+		dev_pm_domain_detach(priv->pd_dev[i], false);
+	}
+
+	return ret;
+}
+
 static int imx_rproc_detect_mode(struct imx_rproc *priv)
 {
 	struct regmap_config config = { .name = "imx-rproc" };
@@ -758,8 +819,12 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
 		 * If Mcore resource is not owned by Acore partition, It is kicked by ROM,
 		 * and Linux could only do IPC with Mcore and nothing else.
 		 */
-		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id))
-			return 0;
+		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id)) {
+			if (of_property_read_u32(dev->of_node, "fsl,entry-address", &priv->entry))
+				return -EINVAL;
+
+			return imx_rproc_attach_pd(priv);
+		}
 
 		priv->rproc->state = RPROC_DETACHED;
 		priv->rproc->recovery_disabled = true;
-- 
2.25.1

[PATCH V3 5/6] remoteproc: imx_rproc: support i.MX8QM

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

Most logic are same as i.MX8QXP, but i.MX8QM has two general purpose
M4 cores, the two cores runs independently and they has different resource
id, different start address from SCFW view.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/remoteproc/imx_rproc.c | 41 +++++++++++++++++++++++++++++++---
 1 file changed, 38 insertions(+), 3 deletions(-)

diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
index 49cce9dd55c7..8326193c13d6 100644
--- a/drivers/remoteproc/imx_rproc.c
+++ b/drivers/remoteproc/imx_rproc.c
@@ -3,6 +3,7 @@
  * Copyright (c) 2017 Pengutronix, Oleksij Rempel <kernel@pengutronix.de>
  */
 
+#include <dt-bindings/firmware/imx/rsrc.h>
 #include <linux/arm-smccc.h>
 #include <linux/clk.h>
 #include <linux/err.h>
@@ -75,10 +76,13 @@ struct imx_rproc_mem {
 	size_t size;
 };
 
-/* att flags */
+/* att flags: lower 16 bits specifying core, higher 16 bits for flags  */
 /* M4 own area. Can be mapped at probe */
-#define ATT_OWN		BIT(1)
-#define ATT_IOMEM	BIT(2)
+#define ATT_OWN         BIT(31)
+#define ATT_IOMEM       BIT(30)
+
+#define ATT_CORE_MASK   0xffff
+#define ATT_CORE(I)     BIT((I))
 
 struct imx_rproc {
 	struct device			*dev;
@@ -99,6 +103,7 @@ struct imx_rproc {
 	u32				rsrc_id;	/* resource id */
 	u32				entry;		/* cpu start address */
 	int                             num_pd;
+	u32				core_index;
 	struct device                   **pd_dev;
 	struct device_link              **pd_dev_link;
 };
@@ -129,6 +134,19 @@ static const struct imx_rproc_att imx_rproc_att_imx93[] = {
 	{ 0xD0000000, 0xa0000000, 0x10000000, 0 },
 };
 
+static const struct imx_rproc_att imx_rproc_att_imx8qm[] = {
+	/* dev addr , sys addr  , size      , flags */
+	{ 0x08000000, 0x08000000, 0x10000000, 0},
+	/* TCML */
+	{ 0x1FFE0000, 0x34FE0000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(0)},
+	{ 0x1FFE0000, 0x38FE0000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(1)},
+	/* TCMU */
+	{ 0x20000000, 0x35000000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(0)},
+	{ 0x20000000, 0x39000000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(1)},
+	/* DDR (Data) */
+	{ 0x80000000, 0x80000000, 0x60000000, 0 },
+};
+
 static const struct imx_rproc_att imx_rproc_att_imx8qxp[] = {
 	{ 0x08000000, 0x08000000, 0x10000000, 0 },
 	/* TCML/U */
@@ -279,6 +297,12 @@ static const struct imx_rproc_dcfg imx_rproc_cfg_imx8mq = {
 	.method		= IMX_RPROC_MMIO,
 };
 
+static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qm = {
+	.att            = imx_rproc_att_imx8qm,
+	.att_size       = ARRAY_SIZE(imx_rproc_att_imx8qm),
+	.method         = IMX_RPROC_SCU_API,
+};
+
 static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qxp = {
 	.att		= imx_rproc_att_imx8qxp,
 	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8qxp),
@@ -395,6 +419,11 @@ static int imx_rproc_da_to_sys(struct imx_rproc *priv, u64 da,
 	for (i = 0; i < dcfg->att_size; i++) {
 		const struct imx_rproc_att *att = &dcfg->att[i];
 
+		if (att->flags & ATT_CORE_MASK) {
+			if (!((BIT(priv->core_index)) & (att->flags & ATT_CORE_MASK)))
+				continue;
+		}
+
 		if (da >= att->da && da + len < att->da + att->size) {
 			unsigned int offset = da - att->da;
 
@@ -815,6 +844,11 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
 			return ret;
 		}
 
+		if (priv->rsrc_id == IMX_SC_R_M4_1_PID0)
+			priv->core_index = 1;
+		else
+			priv->core_index = 0;
+
 		/*
 		 * If Mcore resource is not owned by Acore partition, It is kicked by ROM,
 		 * and Linux could only do IPC with Mcore and nothing else.
@@ -1008,6 +1042,7 @@ static const struct of_device_id imx_rproc_of_match[] = {
 	{ .compatible = "fsl,imx8mn-cm7", .data = &imx_rproc_cfg_imx8mn },
 	{ .compatible = "fsl,imx8mp-cm7", .data = &imx_rproc_cfg_imx8mn },
 	{ .compatible = "fsl,imx8qxp-cm4", .data = &imx_rproc_cfg_imx8qxp },
+	{ .compatible = "fsl,imx8qm-cm4", .data = &imx_rproc_cfg_imx8qm },
 	{ .compatible = "fsl,imx8ulp-cm33", .data = &imx_rproc_cfg_imx8ulp },
 	{ .compatible = "fsl,imx93-cm33", .data = &imx_rproc_cfg_imx93 },
 	{},
-- 
2.25.1

[PATCH V3 6/6] remoteproc: imx_rproc: request mbox channel later

[Peng Fan (OSS)]

From: Peng Fan <peng.fan@nxp.com>

It is possible that when remote processor crash, the communication
channel will be broken with garbage value in mailbox, such as
when Linux is issuing a message through mailbox, remote processor
crashes, we need free & rebuild the mailbox channels to make sure
no garbage value in mailbox channels.

So move the request/free to start/stop for managing remote procesosr in
Linux, move to attach/detach for remote processor is out of control of
Linux.

Previous, we just request mbox when attach for CM4 boot early before
Linux, but if mbox defer probe, remoteproc core will do resource cleanup
and corrupt resource table for later probe.

So move request mbox ealier and still keep mbox request when attach
for self recovery case, but keep a check when request/free mbox.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
---
 drivers/remoteproc/imx_rproc.c | 28 ++++++++++++++++++++++++----
 1 file changed, 24 insertions(+), 4 deletions(-)

diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
index 8326193c13d6..8fe139624ecc 100644
--- a/drivers/remoteproc/imx_rproc.c
+++ b/drivers/remoteproc/imx_rproc.c
@@ -84,6 +84,9 @@ struct imx_rproc_mem {
 #define ATT_CORE_MASK   0xffff
 #define ATT_CORE(I)     BIT((I))
 
+static int imx_rproc_xtr_mbox_init(struct rproc *rproc);
+static void imx_rproc_free_mbox(struct rproc *rproc);
+
 struct imx_rproc {
 	struct device			*dev;
 	struct regmap			*regmap;
@@ -355,6 +358,10 @@ static int imx_rproc_start(struct rproc *rproc)
 	struct arm_smccc_res res;
 	int ret;
 
+	ret = imx_rproc_xtr_mbox_init(rproc);
+	if (ret)
+		return ret;
+
 	switch (dcfg->method) {
 	case IMX_RPROC_MMIO:
 		ret = regmap_update_bits(priv->regmap, dcfg->src_reg, dcfg->src_mask,
@@ -405,6 +412,8 @@ static int imx_rproc_stop(struct rproc *rproc)
 
 	if (ret)
 		dev_err(dev, "Failed to stop remote core\n");
+	else
+		imx_rproc_free_mbox(rproc);
 
 	return ret;
 }
@@ -584,7 +593,7 @@ static void imx_rproc_kick(struct rproc *rproc, int vqid)
 
 static int imx_rproc_attach(struct rproc *rproc)
 {
-	return 0;
+	return imx_rproc_xtr_mbox_init(rproc);
 }
 
 static struct resource_table *imx_rproc_get_loaded_rsc_table(struct rproc *rproc, size_t *table_sz)
@@ -712,6 +721,9 @@ static int imx_rproc_xtr_mbox_init(struct rproc *rproc)
 	struct mbox_client *cl;
 	int ret;
 
+	if (priv->tx_ch && priv->rx_ch)
+		return 0;
+
 	if (!of_get_property(dev->of_node, "mbox-names", NULL))
 		return 0;
 
@@ -744,8 +756,15 @@ static void imx_rproc_free_mbox(struct rproc *rproc)
 {
 	struct imx_rproc *priv = rproc->priv;
 
-	mbox_free_channel(priv->tx_ch);
-	mbox_free_channel(priv->rx_ch);
+	if (priv->tx_ch) {
+		mbox_free_channel(priv->tx_ch);
+		priv->tx_ch = NULL;
+	}
+
+	if (priv->rx_ch) {
+		mbox_free_channel(priv->rx_ch);
+		priv->rx_ch = NULL;
+	}
 }
 
 static int imx_rproc_partition_notify(struct notifier_block *nb,
@@ -757,6 +776,8 @@ static int imx_rproc_partition_notify(struct notifier_block *nb,
 	if (!((event & BIT(priv->rproc_pt)) && (*(u8 *)group == IMX_SC_IRQ_GROUP_REBOOTED)))
 		return 0;
 
+	imx_rproc_free_mbox(priv->rproc);
+
 	rproc_report_crash(priv->rproc, RPROC_WATCHDOG);
 
 	pr_info("Partition%d reset!\n", priv->rproc_pt);
@@ -1026,7 +1047,6 @@ static int imx_rproc_remove(struct platform_device *pdev)
 
 	clk_disable_unprepare(priv->clk);
 	rproc_del(rproc);
-	imx_rproc_free_mbox(rproc);
 	destroy_workqueue(priv->workqueue);
 	rproc_free(rproc);
 
-- 
2.25.1

Re: [PATCH V3 1/6] dt-bindings: remoteproc: imx_rproc: support i.MX8QXP

[Rob Herring]

On Tue, 17 May 2022 14:49:32 +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> Add i.MX8QXP compatible
> 
> Add a new property fsl,resource-id for SoC which supports SCFW.
> This property is used to check whether remote process is under control
> of Linux or not.
> 
> Add fsl,entry-address to specify the entry address which used by SCFW
> to kick M4.
> 
> To i.MX8QM/QXP, when M4 is in the same hardware partition with Cortex-A
> cores, need power up M4 through SCFW, then M4 could start. So introduce
> power-domains property
> 
> Signed-off-by: Peng Fan <peng.fan@nxp.com>
> ---
>  .../bindings/remoteproc/fsl,imx-rproc.yaml        | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 

Reviewed-by: Rob Herring <robh@kernel.org>

Re: [PATCH V3 2/6] dt-bindings: remoteproc: imx_rproc: support i.MX8QM

[Rob Herring]

On Tue, 17 May 2022 14:49:33 +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> Add i.MX8QM compatible
> 
> Signed-off-by: Peng Fan <peng.fan@nxp.com>
> ---
>  Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml | 1 +
>  1 file changed, 1 insertion(+)
> 

Reviewed-by: Rob Herring <robh@kernel.org>

RE: [PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP

[Peng Fan]

Bjorn, Mathieu

> Subject: [PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP

Please give a look on this patchset if possible.

Thanks,
Peng.

> 
> From: Peng Fan <peng.fan@nxp.com>
> 
> V3:
>  Drop the dependency in V2.
>  Tested on i.MX8QM/8MM
>  Use 'fsl,resource-id' and 'fsl,entry-address' Per dt maintainer  Drop 'reg'
> property Per remoteproc maintainer  Drop mcore self recovery, until we land
> in common framework support.
> 
> V2:
> 
> https://patchwork.kernel.org/project/linux-remoteproc/cover/202203091021
> 18.8131-1-peng.fan@oss.nxp.com/
>  Depends on
> https://patchwork.kernel.org/project/linux-remoteproc/list/?series=621311
>  Tested on i.MX8QXP/QM/8MP
>  Addressed Mathieu's comments
>  Drop V1 patch 5/9, patch 3/9 is replaced with upper dependency patchset
> Move V1 patch 4/9 out to
> https://patchwork.kernel.org/project/linux-remoteproc/patch/202203080657
> 54.3355-1-peng.fan@oss.nxp.com/
>  Update commit log
>  Drop magic number to get entry address from device tree in patch 4/6
> 
> The V1 patchset:
> https://patchwork.kernel.org/project/linux-remoteproc/patch/202201110333
> 33.403448-4-peng.fan@oss.nxp.com/
> 
> Peng Fan (6):
>   dt-bindings: remoteproc: imx_rproc: support i.MX8QXP
>   dt-bindings: remoteproc: imx_rproc: support i.MX8QM
>   remoteproc: imx_rproc: support attaching to i.MX8QXP M4
>   remoteproc: imx_rproc: support kicking Mcore from Linux for i.MX8QXP
>   remoteproc: imx_rproc: support i.MX8QM
>   remoteproc: imx_rproc: request mbox channel later
> 
>  .../bindings/remoteproc/fsl,imx-rproc.yaml    |  16 ++
>  drivers/remoteproc/imx_rproc.c                | 222
> +++++++++++++++++-
>  2 files changed, 231 insertions(+), 7 deletions(-)
> 
> --
> 2.25.1

Re: [PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP

[Mathieu Poirier]

On Wed, Jun 15, 2022 at 03:24:03AM +0000, Peng Fan wrote:
> Bjorn, Mathieu
> 
> > Subject: [PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP
> 
> Please give a look on this patchset if possible.

Your pathset is currently 3rd from the top in my queue.  As such I will likely
get to it in the next couple of weeks but as usual, I can't make any guarantees
on that front.

Thanks,
Mathieu

> 
> Thanks,
> Peng.
> 
> > 
> > From: Peng Fan <peng.fan@nxp.com>
> > 
> > V3:
> >  Drop the dependency in V2.
> >  Tested on i.MX8QM/8MM
> >  Use 'fsl,resource-id' and 'fsl,entry-address' Per dt maintainer  Drop 'reg'
> > property Per remoteproc maintainer  Drop mcore self recovery, until we land
> > in common framework support.
> > 
> > V2:
> > 
> > https://patchwork.kernel.org/project/linux-remoteproc/cover/202203091021
> > 18.8131-1-peng.fan@oss.nxp.com/
> >  Depends on
> > https://patchwork.kernel.org/project/linux-remoteproc/list/?series=621311
> >  Tested on i.MX8QXP/QM/8MP
> >  Addressed Mathieu's comments
> >  Drop V1 patch 5/9, patch 3/9 is replaced with upper dependency patchset
> > Move V1 patch 4/9 out to
> > https://patchwork.kernel.org/project/linux-remoteproc/patch/202203080657
> > 54.3355-1-peng.fan@oss.nxp.com/
> >  Update commit log
> >  Drop magic number to get entry address from device tree in patch 4/6
> > 
> > The V1 patchset:
> > https://patchwork.kernel.org/project/linux-remoteproc/patch/202201110333
> > 33.403448-4-peng.fan@oss.nxp.com/
> > 
> > Peng Fan (6):
> >   dt-bindings: remoteproc: imx_rproc: support i.MX8QXP
> >   dt-bindings: remoteproc: imx_rproc: support i.MX8QM
> >   remoteproc: imx_rproc: support attaching to i.MX8QXP M4
> >   remoteproc: imx_rproc: support kicking Mcore from Linux for i.MX8QXP
> >   remoteproc: imx_rproc: support i.MX8QM
> >   remoteproc: imx_rproc: request mbox channel later
> > 
> >  .../bindings/remoteproc/fsl,imx-rproc.yaml    |  16 ++
> >  drivers/remoteproc/imx_rproc.c                | 222
> > +++++++++++++++++-
> >  2 files changed, 231 insertions(+), 7 deletions(-)
> > 
> > --
> > 2.25.1
> 

RE: [PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP

[Peng Fan]

> Subject: Re: [PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP
> 
> On Wed, Jun 15, 2022 at 03:24:03AM +0000, Peng Fan wrote:
> > Bjorn, Mathieu
> >
> > > Subject: [PATCH V3 0/6] remoteproc: imx_rproc: support i.MX8QM/QXP
> >
> > Please give a look on this patchset if possible.
> 
> Your pathset is currently 3rd from the top in my queue.  As such I will likely get
> to it in the next couple of weeks but as usual, I can't make any guarantees on
> that front.

Thanks, just hope could land in this in next tree, and catch next release.

Thanks,
Peng.

> 
> Thanks,
> Mathieu
> 
> >
> > Thanks,
> > Peng.
> >
> > >
> > > From: Peng Fan <peng.fan@nxp.com>
> > >
> > > V3:
> > >  Drop the dependency in V2.
> > >  Tested on i.MX8QM/8MM
> > >  Use 'fsl,resource-id' and 'fsl,entry-address' Per dt maintainer  Drop 'reg'
> > > property Per remoteproc maintainer  Drop mcore self recovery, until
> > > we land in common framework support.
> > >
> > > V2:
> > >
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpa
> > >
> tchwork.kernel.org%2Fproject%2Flinux-remoteproc%2Fcover%2F2022030910
> > >
> 21&amp;data=05%7C01%7Cpeng.fan%40nxp.com%7Cff2ee8dad882403ab39f
> 08da4
> > >
> f0fe8fc%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6379092269
> 99668
> > >
> 234%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> iLCJB
> > >
> TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=SEnpWxX1
> mTgDMPe
> > > GSGk%2FP6xJyWaTT529oh2SLTFK%2FjU%3D&amp;reserved=0
> > > 18.8131-1-peng.fan@oss.nxp.com/
> > >  Depends on
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpa
> > > tchwork.kernel.org%2Fproject%2Flinux-remoteproc%2Flist%2F%3Fseries%3
> > >
> D621311&amp;data=05%7C01%7Cpeng.fan%40nxp.com%7Cff2ee8dad882403
> ab39f
> > >
> 08da4f0fe8fc%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63790
> 92269
> > >
> 99668234%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> 2luMzI
> > >
> iLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=%2F0f
> xhWEg
> > > %2FyqMhovh8UCaig%2FnN6FGYR06FeOtMOpGXb0%3D&amp;reserved=0
> > >  Tested on i.MX8QXP/QM/8MP
> > >  Addressed Mathieu's comments
> > >  Drop V1 patch 5/9, patch 3/9 is replaced with upper dependency
> > > patchset Move V1 patch 4/9 out to
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpa
> > >
> tchwork.kernel.org%2Fproject%2Flinux-remoteproc%2Fpatch%2F2022030806
> > >
> 57&amp;data=05%7C01%7Cpeng.fan%40nxp.com%7Cff2ee8dad882403ab39f
> 08da4
> > >
> f0fe8fc%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6379092269
> 99668
> > >
> 234%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> iLCJB
> > >
> TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=25SCYuuYHi
> ZX9Mo
> > > Z%2FSFpyDTEXg8IzIzTNv%2FwQt2h%2BKM%3D&amp;reserved=0
> > > 54.3355-1-peng.fan@oss.nxp.com/
> > >  Update commit log
> > >  Drop magic number to get entry address from device tree in patch
> > > 4/6
> > >
> > > The V1 patchset:
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpa
> > >
> tchwork.kernel.org%2Fproject%2Flinux-remoteproc%2Fpatch%2F2022011103
> > >
> 33&amp;data=05%7C01%7Cpeng.fan%40nxp.com%7Cff2ee8dad882403ab39f
> 08da4
> > >
> f0fe8fc%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6379092269
> 99668
> > >
> 234%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> iLCJB
> > >
> TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=C9LrnuMK7
> xob%2B
> > > zQP%2BifyGQxSH0YRbuJl6dsDc56Bliw%3D&amp;reserved=0
> > > 33.403448-4-peng.fan@oss.nxp.com/
> > >
> > > Peng Fan (6):
> > >   dt-bindings: remoteproc: imx_rproc: support i.MX8QXP
> > >   dt-bindings: remoteproc: imx_rproc: support i.MX8QM
> > >   remoteproc: imx_rproc: support attaching to i.MX8QXP M4
> > >   remoteproc: imx_rproc: support kicking Mcore from Linux for i.MX8QXP
> > >   remoteproc: imx_rproc: support i.MX8QM
> > >   remoteproc: imx_rproc: request mbox channel later
> > >
> > >  .../bindings/remoteproc/fsl,imx-rproc.yaml    |  16 ++
> > >  drivers/remoteproc/imx_rproc.c                | 222
> > > +++++++++++++++++-
> > >  2 files changed, 231 insertions(+), 7 deletions(-)
> > >
> > > --
> > > 2.25.1
> >

Re: [PATCH V3 3/6] remoteproc: imx_rproc: support attaching to i.MX8QXP M4

[Mathieu Poirier]

Hi Peng,

On Tue, May 17, 2022 at 02:49:34PM +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> When M4 is kicked by SCFW, M4 runs in its own hardware partition, Linux
> could only do IPC with M4, it could not start, stop, update image.
> 
> We disable recovery reboot when M4 is managed by SCFW, because
> remoteproc core still not support M4 auto-recovery without loading
> image.
> 
> Signed-off-by: Peng Fan <peng.fan@nxp.com>
> ---
>  drivers/remoteproc/imx_rproc.c | 88 ++++++++++++++++++++++++++++++++++
>  1 file changed, 88 insertions(+)
> 
> diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
> index 4a3352821b1d..2e751fc90760 100644
> --- a/drivers/remoteproc/imx_rproc.c
> +++ b/drivers/remoteproc/imx_rproc.c
> @@ -6,6 +6,7 @@
>  #include <linux/arm-smccc.h>
>  #include <linux/clk.h>
>  #include <linux/err.h>
> +#include <linux/firmware/imx/sci.h>
>  #include <linux/interrupt.h>
>  #include <linux/kernel.h>
>  #include <linux/mailbox_client.h>
> @@ -59,6 +60,8 @@
>  #define IMX_SIP_RPROC_STARTED		0x01
>  #define IMX_SIP_RPROC_STOP		0x02
>  
> +#define	IMX_SC_IRQ_GROUP_REBOOTED	5
> +
>  /**
>   * struct imx_rproc_mem - slim internal memory structure
>   * @cpu_addr: MPU virtual address of the memory region
> @@ -89,6 +92,10 @@ struct imx_rproc {
>  	struct work_struct		rproc_work;
>  	struct workqueue_struct		*workqueue;
>  	void __iomem			*rsc_table;
> +	struct imx_sc_ipc		*ipc_handle;
> +	struct notifier_block		rproc_nb;
> +	u32				rproc_pt;	/* partition id */
> +	u32				rsrc_id;	/* resource id */
>  };
>  
>  static const struct imx_rproc_att imx_rproc_att_imx93[] = {
> @@ -117,6 +124,18 @@ static const struct imx_rproc_att imx_rproc_att_imx93[] = {
>  	{ 0xD0000000, 0xa0000000, 0x10000000, 0 },
>  };
>  
> +static const struct imx_rproc_att imx_rproc_att_imx8qxp[] = {
> +	{ 0x08000000, 0x08000000, 0x10000000, 0 },
> +	/* TCML/U */
> +	{ 0x1FFE0000, 0x34FE0000, 0x00040000, ATT_OWN | ATT_IOMEM },
> +	/* OCRAM(Low 96KB) */
> +	{ 0x21000000, 0x00100000, 0x00018000, 0 },
> +	/* OCRAM */
> +	{ 0x21100000, 0x00100000, 0x00040000, 0 },
> +	/* DDR (Data) */
> +	{ 0x80000000, 0x80000000, 0x60000000, 0 },
> +};
> +
>  static const struct imx_rproc_att imx_rproc_att_imx8mn[] = {
>  	/* dev addr , sys addr  , size	    , flags */
>  	/* ITCM   */
> @@ -255,6 +274,12 @@ static const struct imx_rproc_dcfg imx_rproc_cfg_imx8mq = {
>  	.method		= IMX_RPROC_MMIO,
>  };
>  
> +static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qxp = {
> +	.att		= imx_rproc_att_imx8qxp,
> +	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8qxp),
> +	.method		= IMX_RPROC_SCU_API,
> +};
> +
>  static const struct imx_rproc_dcfg imx_rproc_cfg_imx8ulp = {
>  	.att		= imx_rproc_att_imx8ulp,
>  	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8ulp),
> @@ -683,6 +708,22 @@ static void imx_rproc_free_mbox(struct rproc *rproc)
>  	mbox_free_channel(priv->rx_ch);
>  }
>  
> +static int imx_rproc_partition_notify(struct notifier_block *nb,
> +				      unsigned long event, void *group)
> +{
> +	struct imx_rproc *priv = container_of(nb, struct imx_rproc, rproc_nb);
> +
> +	/* Ignore other irqs */
> +	if (!((event & BIT(priv->rproc_pt)) && (*(u8 *)group == IMX_SC_IRQ_GROUP_REBOOTED)))
> +		return 0;
> +
> +	rproc_report_crash(priv->rproc, RPROC_WATCHDOG);
> +
> +	pr_info("Partition%d reset!\n", priv->rproc_pt);
> +
> +	return 0;
> +}
> +
>  static int imx_rproc_detect_mode(struct imx_rproc *priv)
>  {
>  	struct regmap_config config = { .name = "imx-rproc" };
> @@ -692,6 +733,7 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
>  	struct arm_smccc_res res;
>  	int ret;
>  	u32 val;
> +	u8 pt;
>  
>  	switch (dcfg->method) {
>  	case IMX_RPROC_NONE:
> @@ -702,6 +744,51 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
>  		if (res.a0)
>  			priv->rproc->state = RPROC_DETACHED;
>  		return 0;
> +	case IMX_RPROC_SCU_API:
> +		ret = imx_scu_get_handle(&priv->ipc_handle);
> +		if (ret)
> +			return ret;
> +		ret = of_property_read_u32(dev->of_node, "fsl,resource-id", &priv->rsrc_id);
> +		if (ret) {
> +			dev_err(dev, "No fsl,resource-id property\n");
> +			return ret;
> +		}
> +
> +		/*
> +		 * If Mcore resource is not owned by Acore partition, It is kicked by ROM,
> +		 * and Linux could only do IPC with Mcore and nothing else.
> +		 */
> +		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id))
> +			return 0;
> +
> +		priv->rproc->state = RPROC_DETACHED;
> +		priv->rproc->recovery_disabled = true;
> +
> +		/* Get partition id and enable irq in SCFW */
> +		ret = imx_sc_rm_get_resource_owner(priv->ipc_handle, priv->rsrc_id, &pt);
> +		if (ret) {
> +			dev_err(dev, "not able to get resource owner\n");
> +			return ret;
> +		}
> +
> +		priv->rproc_pt = pt;
> +		priv->rproc_nb.notifier_call = imx_rproc_partition_notify;
> +
> +		ret = imx_scu_irq_register_notifier(&priv->rproc_nb);
> +		if (ret) {
> +			dev_warn(dev, "register scu notifier failed.\n");
> +			return ret;
> +		}

I woul have expected to see imx_scu_irq_register_notifier() being called when
the driver is removed and in the probe() error path.  More comments to come.

Thanks,
Mathieu

> +
> +		ret = imx_scu_irq_group_enable(IMX_SC_IRQ_GROUP_REBOOTED, BIT(priv->rproc_pt),
> +					       true);
> +		if (ret) {
> +			imx_scu_irq_unregister_notifier(&priv->rproc_nb);
> +			dev_warn(dev, "Enable irq failed.\n");
> +			return ret;
> +		}
> +
> +		return 0;
>  	default:
>  		break;
>  	}
> @@ -855,6 +942,7 @@ static const struct of_device_id imx_rproc_of_match[] = {
>  	{ .compatible = "fsl,imx8mm-cm4", .data = &imx_rproc_cfg_imx8mq },
>  	{ .compatible = "fsl,imx8mn-cm7", .data = &imx_rproc_cfg_imx8mn },
>  	{ .compatible = "fsl,imx8mp-cm7", .data = &imx_rproc_cfg_imx8mn },
> +	{ .compatible = "fsl,imx8qxp-cm4", .data = &imx_rproc_cfg_imx8qxp },
>  	{ .compatible = "fsl,imx8ulp-cm33", .data = &imx_rproc_cfg_imx8ulp },
>  	{ .compatible = "fsl,imx93-cm33", .data = &imx_rproc_cfg_imx93 },
>  	{},
> -- 
> 2.25.1
> 

Re: [PATCH V3 4/6] remoteproc: imx_rproc: support kicking Mcore from Linux for i.MX8QXP

[Mathieu Poirier]

On Tue, May 17, 2022 at 02:49:35PM +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> When M4 is in the same hardware partition with Cortex-A, it
> could be start/stop by Linux.
> 
> Added power domain to make sure M4 could run, it requires several power
> domains to work. Make clk always optional for i.MX8QXP, because
> SCFW handles it when power up M4 core.
> 
> Signed-off-by: Peng Fan <peng.fan@nxp.com>
> ---
>  drivers/remoteproc/imx_rproc.c | 69 +++++++++++++++++++++++++++++++++-
>  1 file changed, 67 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
> index 2e751fc90760..49cce9dd55c7 100644
> --- a/drivers/remoteproc/imx_rproc.c
> +++ b/drivers/remoteproc/imx_rproc.c
> @@ -16,6 +16,7 @@
>  #include <linux/of_reserved_mem.h>
>  #include <linux/of_device.h>
>  #include <linux/platform_device.h>
> +#include <linux/pm_domain.h>
>  #include <linux/regmap.h>
>  #include <linux/remoteproc.h>
>  #include <linux/workqueue.h>
> @@ -96,6 +97,10 @@ struct imx_rproc {
>  	struct notifier_block		rproc_nb;
>  	u32				rproc_pt;	/* partition id */
>  	u32				rsrc_id;	/* resource id */
> +	u32				entry;		/* cpu start address */
> +	int                             num_pd;
> +	struct device                   **pd_dev;
> +	struct device_link              **pd_dev_link;
>  };
>  
>  static const struct imx_rproc_att imx_rproc_att_imx93[] = {
> @@ -335,6 +340,9 @@ static int imx_rproc_start(struct rproc *rproc)
>  		arm_smccc_smc(IMX_SIP_RPROC, IMX_SIP_RPROC_START, 0, 0, 0, 0, 0, 0, &res);
>  		ret = res.a0;
>  		break;
> +	case IMX_RPROC_SCU_API:
> +		ret = imx_sc_pm_cpu_start(priv->ipc_handle, priv->rsrc_id, true, priv->entry);
> +		break;
>  	default:
>  		return -EOPNOTSUPP;
>  	}
> @@ -364,6 +372,9 @@ static int imx_rproc_stop(struct rproc *rproc)
>  		if (res.a1)
>  			dev_info(dev, "Not in wfi, force stopped\n");
>  		break;
> +	case IMX_RPROC_SCU_API:
> +		ret = imx_sc_pm_cpu_start(priv->ipc_handle, priv->rsrc_id, false, priv->entry);
> +		break;
>  	default:
>  		return -EOPNOTSUPP;
>  	}
> @@ -724,6 +735,56 @@ static int imx_rproc_partition_notify(struct notifier_block *nb,
>  	return 0;
>  }
>  
> +static int imx_rproc_attach_pd(struct imx_rproc *priv)
> +{
> +	struct device *dev = priv->dev;
> +	int ret, i;
> +
> +	priv->num_pd = of_count_phandle_with_args(dev->of_node, "power-domains",
> +						  "#power-domain-cells");
> +	if (priv->num_pd < 0)
> +		return priv->num_pd;
> +
> +	if (!priv->num_pd)
> +		return 0;
> +
> +	priv->pd_dev = devm_kmalloc_array(dev, priv->num_pd, sizeof(*priv->pd_dev), GFP_KERNEL);
> +	if (!priv->pd_dev)
> +		return -ENOMEM;
> +
> +	priv->pd_dev_link = devm_kmalloc_array(dev, priv->num_pd, sizeof(*priv->pd_dev_link),
> +					       GFP_KERNEL);
> +
> +	if (!priv->pd_dev_link)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < priv->num_pd; i++) {
> +		priv->pd_dev[i] = dev_pm_domain_attach_by_id(dev, i);
> +		if (IS_ERR(priv->pd_dev[i])) {
> +			ret = PTR_ERR(priv->pd_dev[i]);
> +			goto detach_pd;
> +		}
> +
> +		priv->pd_dev_link[i] = device_link_add(dev, priv->pd_dev[i], DL_FLAG_STATELESS |
> +						       DL_FLAG_PM_RUNTIME | DL_FLAG_RPM_ACTIVE);
> +		if (!priv->pd_dev_link[i]) {
> +			dev_pm_domain_detach(priv->pd_dev[i], false);
> +			ret = -EINVAL;
> +			goto detach_pd;
> +		}
> +	}
> +
> +	return 0;
> +
> +detach_pd:
> +	while (--i >= 0) {
> +		device_link_del(priv->pd_dev_link[i]);
> +		dev_pm_domain_detach(priv->pd_dev[i], false);

Same here - why are these not called during driver removal and along the error
path in probe()?

> +	}
> +
> +	return ret;
> +}
> +
>  static int imx_rproc_detect_mode(struct imx_rproc *priv)
>  {
>  	struct regmap_config config = { .name = "imx-rproc" };
> @@ -758,8 +819,12 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
>  		 * If Mcore resource is not owned by Acore partition, It is kicked by ROM,
>  		 * and Linux could only do IPC with Mcore and nothing else.
>  		 */
> -		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id))
> -			return 0;
> +		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id)) {
> +			if (of_property_read_u32(dev->of_node, "fsl,entry-address", &priv->entry))
> +				return -EINVAL;
> +
> +			return imx_rproc_attach_pd(priv);
> +		}
>  
>  		priv->rproc->state = RPROC_DETACHED;
>  		priv->rproc->recovery_disabled = true;
> -- 
> 2.25.1
> 

Re: [PATCH V3 5/6] remoteproc: imx_rproc: support i.MX8QM

[Mathieu Poirier]

On Tue, May 17, 2022 at 02:49:36PM +0800, Peng Fan (OSS) wrote:
> From: Peng Fan <peng.fan@nxp.com>
> 
> Most logic are same as i.MX8QXP, but i.MX8QM has two general purpose
> M4 cores, the two cores runs independently and they has different resource
> id, different start address from SCFW view.
> 
> Signed-off-by: Peng Fan <peng.fan@nxp.com>
> ---
>  drivers/remoteproc/imx_rproc.c | 41 +++++++++++++++++++++++++++++++---
>  1 file changed, 38 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
> index 49cce9dd55c7..8326193c13d6 100644
> --- a/drivers/remoteproc/imx_rproc.c
> +++ b/drivers/remoteproc/imx_rproc.c
> @@ -3,6 +3,7 @@
>   * Copyright (c) 2017 Pengutronix, Oleksij Rempel <kernel@pengutronix.de>
>   */
>  
> +#include <dt-bindings/firmware/imx/rsrc.h>
>  #include <linux/arm-smccc.h>
>  #include <linux/clk.h>
>  #include <linux/err.h>
> @@ -75,10 +76,13 @@ struct imx_rproc_mem {
>  	size_t size;
>  };
>  
> -/* att flags */
> +/* att flags: lower 16 bits specifying core, higher 16 bits for flags  */
>  /* M4 own area. Can be mapped at probe */
> -#define ATT_OWN		BIT(1)
> -#define ATT_IOMEM	BIT(2)
> +#define ATT_OWN         BIT(31)
> +#define ATT_IOMEM       BIT(30)
> +
> +#define ATT_CORE_MASK   0xffff
> +#define ATT_CORE(I)     BIT((I))
>  
>  struct imx_rproc {
>  	struct device			*dev;
> @@ -99,6 +103,7 @@ struct imx_rproc {
>  	u32				rsrc_id;	/* resource id */
>  	u32				entry;		/* cpu start address */
>  	int                             num_pd;
> +	u32				core_index;
>  	struct device                   **pd_dev;
>  	struct device_link              **pd_dev_link;
>  };
> @@ -129,6 +134,19 @@ static const struct imx_rproc_att imx_rproc_att_imx93[] = {
>  	{ 0xD0000000, 0xa0000000, 0x10000000, 0 },
>  };
>  
> +static const struct imx_rproc_att imx_rproc_att_imx8qm[] = {
> +	/* dev addr , sys addr  , size      , flags */
> +	{ 0x08000000, 0x08000000, 0x10000000, 0},
> +	/* TCML */
> +	{ 0x1FFE0000, 0x34FE0000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(0)},
> +	{ 0x1FFE0000, 0x38FE0000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(1)},
> +	/* TCMU */
> +	{ 0x20000000, 0x35000000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(0)},
> +	{ 0x20000000, 0x39000000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(1)},
> +	/* DDR (Data) */
> +	{ 0x80000000, 0x80000000, 0x60000000, 0 },
> +};
> +
>  static const struct imx_rproc_att imx_rproc_att_imx8qxp[] = {
>  	{ 0x08000000, 0x08000000, 0x10000000, 0 },
>  	/* TCML/U */
> @@ -279,6 +297,12 @@ static const struct imx_rproc_dcfg imx_rproc_cfg_imx8mq = {
>  	.method		= IMX_RPROC_MMIO,
>  };
>  
> +static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qm = {
> +	.att            = imx_rproc_att_imx8qm,
> +	.att_size       = ARRAY_SIZE(imx_rproc_att_imx8qm),
> +	.method         = IMX_RPROC_SCU_API,
> +};
> +
>  static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qxp = {
>  	.att		= imx_rproc_att_imx8qxp,
>  	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8qxp),
> @@ -395,6 +419,11 @@ static int imx_rproc_da_to_sys(struct imx_rproc *priv, u64 da,
>  	for (i = 0; i < dcfg->att_size; i++) {
>  		const struct imx_rproc_att *att = &dcfg->att[i];
>  
> +		if (att->flags & ATT_CORE_MASK) {
> +			if (!((BIT(priv->core_index)) & (att->flags & ATT_CORE_MASK)))
> +				continue;
> +		}

This is very cryptic - I just spent 20 minutes looking at it and I'm still not
sure I got the full meaning.  Please add enough comments to make things obvious
on first read.

I am done reviewing this patchset.

Thanks,
Mathieu


> +
>  		if (da >= att->da && da + len < att->da + att->size) {
>  			unsigned int offset = da - att->da;
>  
> @@ -815,6 +844,11 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
>  			return ret;
>  		}
>  
> +		if (priv->rsrc_id == IMX_SC_R_M4_1_PID0)
> +			priv->core_index = 1;
> +		else
> +			priv->core_index = 0;
> +
>  		/*
>  		 * If Mcore resource is not owned by Acore partition, It is kicked by ROM,
>  		 * and Linux could only do IPC with Mcore and nothing else.
> @@ -1008,6 +1042,7 @@ static const struct of_device_id imx_rproc_of_match[] = {
>  	{ .compatible = "fsl,imx8mn-cm7", .data = &imx_rproc_cfg_imx8mn },
>  	{ .compatible = "fsl,imx8mp-cm7", .data = &imx_rproc_cfg_imx8mn },
>  	{ .compatible = "fsl,imx8qxp-cm4", .data = &imx_rproc_cfg_imx8qxp },
> +	{ .compatible = "fsl,imx8qm-cm4", .data = &imx_rproc_cfg_imx8qm },
>  	{ .compatible = "fsl,imx8ulp-cm33", .data = &imx_rproc_cfg_imx8ulp },
>  	{ .compatible = "fsl,imx93-cm33", .data = &imx_rproc_cfg_imx93 },
>  	{},
> -- 
> 2.25.1
> 

RE: [PATCH V3 3/6] remoteproc: imx_rproc: support attaching to i.MX8QXP M4

[Peng Fan]

> Subject: Re: [PATCH V3 3/6] remoteproc: imx_rproc: support attaching to
> i.MX8QXP M4
> 
> Hi Peng,
> 
> On Tue, May 17, 2022 at 02:49:34PM +0800, Peng Fan (OSS) wrote:
> > From: Peng Fan <peng.fan@nxp.com>
> >
> > When M4 is kicked by SCFW, M4 runs in its own hardware partition,
> > Linux could only do IPC with M4, it could not start, stop, update image.
> >
> > We disable recovery reboot when M4 is managed by SCFW, because
> > remoteproc core still not support M4 auto-recovery without loading
> > image.
> >
> > Signed-off-by: Peng Fan <peng.fan@nxp.com>
> > ---
> >  drivers/remoteproc/imx_rproc.c | 88
> > ++++++++++++++++++++++++++++++++++
> >  1 file changed, 88 insertions(+)
> >
> > diff --git a/drivers/remoteproc/imx_rproc.c
> > b/drivers/remoteproc/imx_rproc.c index 4a3352821b1d..2e751fc90760
> > 100644
> > --- a/drivers/remoteproc/imx_rproc.c
> > +++ b/drivers/remoteproc/imx_rproc.c
> > @@ -6,6 +6,7 @@
> >  #include <linux/arm-smccc.h>
> >  #include <linux/clk.h>
> >  #include <linux/err.h>
> > +#include <linux/firmware/imx/sci.h>
> >  #include <linux/interrupt.h>
> >  #include <linux/kernel.h>
> >  #include <linux/mailbox_client.h>
> > @@ -59,6 +60,8 @@
> >  #define IMX_SIP_RPROC_STARTED		0x01
> >  #define IMX_SIP_RPROC_STOP		0x02
> >
> > +#define	IMX_SC_IRQ_GROUP_REBOOTED	5
> > +
> >  /**
> >   * struct imx_rproc_mem - slim internal memory structure
> >   * @cpu_addr: MPU virtual address of the memory region @@ -89,6
> > +92,10 @@ struct imx_rproc {
> >  	struct work_struct		rproc_work;
> >  	struct workqueue_struct		*workqueue;
> >  	void __iomem			*rsc_table;
> > +	struct imx_sc_ipc		*ipc_handle;
> > +	struct notifier_block		rproc_nb;
> > +	u32				rproc_pt;	/* partition id */
> > +	u32				rsrc_id;	/* resource id */
> >  };
> >
> >  static const struct imx_rproc_att imx_rproc_att_imx93[] = { @@ -117,6
> > +124,18 @@ static const struct imx_rproc_att imx_rproc_att_imx93[] = {
> >  	{ 0xD0000000, 0xa0000000, 0x10000000, 0 },  };
> >
> > +static const struct imx_rproc_att imx_rproc_att_imx8qxp[] = {
> > +	{ 0x08000000, 0x08000000, 0x10000000, 0 },
> > +	/* TCML/U */
> > +	{ 0x1FFE0000, 0x34FE0000, 0x00040000, ATT_OWN | ATT_IOMEM },
> > +	/* OCRAM(Low 96KB) */
> > +	{ 0x21000000, 0x00100000, 0x00018000, 0 },
> > +	/* OCRAM */
> > +	{ 0x21100000, 0x00100000, 0x00040000, 0 },
> > +	/* DDR (Data) */
> > +	{ 0x80000000, 0x80000000, 0x60000000, 0 }, };
> > +
> >  static const struct imx_rproc_att imx_rproc_att_imx8mn[] = {
> >  	/* dev addr , sys addr  , size	    , flags */
> >  	/* ITCM   */
> > @@ -255,6 +274,12 @@ static const struct imx_rproc_dcfg
> imx_rproc_cfg_imx8mq = {
> >  	.method		= IMX_RPROC_MMIO,
> >  };
> >
> > +static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qxp = {
> > +	.att		= imx_rproc_att_imx8qxp,
> > +	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8qxp),
> > +	.method		= IMX_RPROC_SCU_API,
> > +};
> > +
> >  static const struct imx_rproc_dcfg imx_rproc_cfg_imx8ulp = {
> >  	.att		= imx_rproc_att_imx8ulp,
> >  	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8ulp),
> > @@ -683,6 +708,22 @@ static void imx_rproc_free_mbox(struct rproc
> *rproc)
> >  	mbox_free_channel(priv->rx_ch);
> >  }
> >
> > +static int imx_rproc_partition_notify(struct notifier_block *nb,
> > +				      unsigned long event, void *group) {
> > +	struct imx_rproc *priv = container_of(nb, struct imx_rproc,
> > +rproc_nb);
> > +
> > +	/* Ignore other irqs */
> > +	if (!((event & BIT(priv->rproc_pt)) && (*(u8 *)group ==
> IMX_SC_IRQ_GROUP_REBOOTED)))
> > +		return 0;
> > +
> > +	rproc_report_crash(priv->rproc, RPROC_WATCHDOG);
> > +
> > +	pr_info("Partition%d reset!\n", priv->rproc_pt);
> > +
> > +	return 0;
> > +}
> > +
> >  static int imx_rproc_detect_mode(struct imx_rproc *priv)  {
> >  	struct regmap_config config = { .name = "imx-rproc" }; @@ -692,6
> > +733,7 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
> >  	struct arm_smccc_res res;
> >  	int ret;
> >  	u32 val;
> > +	u8 pt;
> >
> >  	switch (dcfg->method) {
> >  	case IMX_RPROC_NONE:
> > @@ -702,6 +744,51 @@ static int imx_rproc_detect_mode(struct
> imx_rproc *priv)
> >  		if (res.a0)
> >  			priv->rproc->state = RPROC_DETACHED;
> >  		return 0;
> > +	case IMX_RPROC_SCU_API:
> > +		ret = imx_scu_get_handle(&priv->ipc_handle);
> > +		if (ret)
> > +			return ret;
> > +		ret = of_property_read_u32(dev->of_node, "fsl,resource-id",
> &priv->rsrc_id);
> > +		if (ret) {
> > +			dev_err(dev, "No fsl,resource-id property\n");
> > +			return ret;
> > +		}
> > +
> > +		/*
> > +		 * If Mcore resource is not owned by Acore partition, It is kicked by
> ROM,
> > +		 * and Linux could only do IPC with Mcore and nothing else.
> > +		 */
> > +		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id))
> > +			return 0;
> > +
> > +		priv->rproc->state = RPROC_DETACHED;
> > +		priv->rproc->recovery_disabled = true;
> > +
> > +		/* Get partition id and enable irq in SCFW */
> > +		ret = imx_sc_rm_get_resource_owner(priv->ipc_handle,
> priv->rsrc_id, &pt);
> > +		if (ret) {
> > +			dev_err(dev, "not able to get resource owner\n");
> > +			return ret;
> > +		}
> > +
> > +		priv->rproc_pt = pt;
> > +		priv->rproc_nb.notifier_call = imx_rproc_partition_notify;
> > +
> > +		ret = imx_scu_irq_register_notifier(&priv->rproc_nb);
> > +		if (ret) {
> > +			dev_warn(dev, "register scu notifier failed.\n");
> > +			return ret;
> > +		}
> 
> I woul have expected to see imx_scu_irq_register_notifier() being called when
> the driver is removed and in the probe() error path.  

You mean unregister, right? I not included driver removing in my test case,
will add the test and fix the issues.

Thanks,
Peng.

More comments to
> come.
> 
> Thanks,
> Mathieu
> 
> > +
> > +		ret = imx_scu_irq_group_enable(IMX_SC_IRQ_GROUP_REBOOTED,
> BIT(priv->rproc_pt),
> > +					       true);
> > +		if (ret) {
> > +			imx_scu_irq_unregister_notifier(&priv->rproc_nb);
> > +			dev_warn(dev, "Enable irq failed.\n");
> > +			return ret;
> > +		}
> > +
> > +		return 0;
> >  	default:
> >  		break;
> >  	}
> > @@ -855,6 +942,7 @@ static const struct of_device_id
> imx_rproc_of_match[] = {
> >  	{ .compatible = "fsl,imx8mm-cm4", .data = &imx_rproc_cfg_imx8mq },
> >  	{ .compatible = "fsl,imx8mn-cm7", .data = &imx_rproc_cfg_imx8mn },
> >  	{ .compatible = "fsl,imx8mp-cm7", .data = &imx_rproc_cfg_imx8mn },
> > +	{ .compatible = "fsl,imx8qxp-cm4", .data = &imx_rproc_cfg_imx8qxp },
> >  	{ .compatible = "fsl,imx8ulp-cm33", .data = &imx_rproc_cfg_imx8ulp },
> >  	{ .compatible = "fsl,imx93-cm33", .data = &imx_rproc_cfg_imx93 },
> >  	{},
> > --
> > 2.25.1
> >

RE: [PATCH V3 4/6] remoteproc: imx_rproc: support kicking Mcore from Linux for i.MX8QXP

[Peng Fan]

> Subject: Re: [PATCH V3 4/6] remoteproc: imx_rproc: support kicking Mcore
> from Linux for i.MX8QXP
> 
> On Tue, May 17, 2022 at 02:49:35PM +0800, Peng Fan (OSS) wrote:
> > From: Peng Fan <peng.fan@nxp.com>
> >
> > When M4 is in the same hardware partition with Cortex-A, it could be
> > start/stop by Linux.
> >
> > Added power domain to make sure M4 could run, it requires several
> > power domains to work. Make clk always optional for i.MX8QXP, because
> > SCFW handles it when power up M4 core.
> >
> > Signed-off-by: Peng Fan <peng.fan@nxp.com>
> > ---
> >  drivers/remoteproc/imx_rproc.c | 69
> > +++++++++++++++++++++++++++++++++-
> >  1 file changed, 67 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/remoteproc/imx_rproc.c
> > b/drivers/remoteproc/imx_rproc.c index 2e751fc90760..49cce9dd55c7
> > 100644
> > --- a/drivers/remoteproc/imx_rproc.c
> > +++ b/drivers/remoteproc/imx_rproc.c
> > @@ -16,6 +16,7 @@
> >  #include <linux/of_reserved_mem.h>
> >  #include <linux/of_device.h>
> >  #include <linux/platform_device.h>
> > +#include <linux/pm_domain.h>
> >  #include <linux/regmap.h>
> >  #include <linux/remoteproc.h>
> >  #include <linux/workqueue.h>
> > @@ -96,6 +97,10 @@ struct imx_rproc {
> >  	struct notifier_block		rproc_nb;
> >  	u32				rproc_pt;	/* partition id */
> >  	u32				rsrc_id;	/* resource id */
> > +	u32				entry;		/* cpu start address */
> > +	int                             num_pd;
> > +	struct device                   **pd_dev;
> > +	struct device_link              **pd_dev_link;
> >  };
> >
> >  static const struct imx_rproc_att imx_rproc_att_imx93[] = { @@ -335,6
> > +340,9 @@ static int imx_rproc_start(struct rproc *rproc)
> >  		arm_smccc_smc(IMX_SIP_RPROC, IMX_SIP_RPROC_START, 0, 0, 0, 0,
> 0, 0, &res);
> >  		ret = res.a0;
> >  		break;
> > +	case IMX_RPROC_SCU_API:
> > +		ret = imx_sc_pm_cpu_start(priv->ipc_handle, priv->rsrc_id, true,
> priv->entry);
> > +		break;
> >  	default:
> >  		return -EOPNOTSUPP;
> >  	}
> > @@ -364,6 +372,9 @@ static int imx_rproc_stop(struct rproc *rproc)
> >  		if (res.a1)
> >  			dev_info(dev, "Not in wfi, force stopped\n");
> >  		break;
> > +	case IMX_RPROC_SCU_API:
> > +		ret = imx_sc_pm_cpu_start(priv->ipc_handle, priv->rsrc_id, false,
> priv->entry);
> > +		break;
> >  	default:
> >  		return -EOPNOTSUPP;
> >  	}
> > @@ -724,6 +735,56 @@ static int imx_rproc_partition_notify(struct
> notifier_block *nb,
> >  	return 0;
> >  }
> >
> > +static int imx_rproc_attach_pd(struct imx_rproc *priv) {
> > +	struct device *dev = priv->dev;
> > +	int ret, i;
> > +
> > +	priv->num_pd = of_count_phandle_with_args(dev->of_node,
> "power-domains",
> > +						  "#power-domain-cells");
> > +	if (priv->num_pd < 0)
> > +		return priv->num_pd;
> > +
> > +	if (!priv->num_pd)
> > +		return 0;
> > +
> > +	priv->pd_dev = devm_kmalloc_array(dev, priv->num_pd,
> sizeof(*priv->pd_dev), GFP_KERNEL);
> > +	if (!priv->pd_dev)
> > +		return -ENOMEM;
> > +
> > +	priv->pd_dev_link = devm_kmalloc_array(dev, priv->num_pd,
> sizeof(*priv->pd_dev_link),
> > +					       GFP_KERNEL);
> > +
> > +	if (!priv->pd_dev_link)
> > +		return -ENOMEM;
> > +
> > +	for (i = 0; i < priv->num_pd; i++) {
> > +		priv->pd_dev[i] = dev_pm_domain_attach_by_id(dev, i);
> > +		if (IS_ERR(priv->pd_dev[i])) {
> > +			ret = PTR_ERR(priv->pd_dev[i]);
> > +			goto detach_pd;
> > +		}
> > +
> > +		priv->pd_dev_link[i] = device_link_add(dev, priv->pd_dev[i],
> DL_FLAG_STATELESS |
> > +						       DL_FLAG_PM_RUNTIME |
> DL_FLAG_RPM_ACTIVE);
> > +		if (!priv->pd_dev_link[i]) {
> > +			dev_pm_domain_detach(priv->pd_dev[i], false);
> > +			ret = -EINVAL;
> > +			goto detach_pd;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +
> > +detach_pd:
> > +	while (--i >= 0) {
> > +		device_link_del(priv->pd_dev_link[i]);
> > +		dev_pm_domain_detach(priv->pd_dev[i], false);
> 
> Same here - why are these not called during driver removal and along the error
> path in probe()?

Will add that, with a function imx_rproc_detach_pd

Thanks,
Peng.

> 
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> >  static int imx_rproc_detect_mode(struct imx_rproc *priv)  {
> >  	struct regmap_config config = { .name = "imx-rproc" }; @@ -758,8
> > +819,12 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
> >  		 * If Mcore resource is not owned by Acore partition, It is kicked by
> ROM,
> >  		 * and Linux could only do IPC with Mcore and nothing else.
> >  		 */
> > -		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id))
> > -			return 0;
> > +		if (imx_sc_rm_is_resource_owned(priv->ipc_handle, priv->rsrc_id)) {
> > +			if (of_property_read_u32(dev->of_node, "fsl,entry-address",
> &priv->entry))
> > +				return -EINVAL;
> > +
> > +			return imx_rproc_attach_pd(priv);
> > +		}
> >
> >  		priv->rproc->state = RPROC_DETACHED;
> >  		priv->rproc->recovery_disabled = true;
> > --
> > 2.25.1
> >

RE: [PATCH V3 5/6] remoteproc: imx_rproc: support i.MX8QM

[Peng Fan]

> Subject: Re: [PATCH V3 5/6] remoteproc: imx_rproc: support i.MX8QM
> 
> On Tue, May 17, 2022 at 02:49:36PM +0800, Peng Fan (OSS) wrote:
> > From: Peng Fan <peng.fan@nxp.com>
> >
> > Most logic are same as i.MX8QXP, but i.MX8QM has two general purpose
> > M4 cores, the two cores runs independently and they has different
> > resource id, different start address from SCFW view.
> >
> > Signed-off-by: Peng Fan <peng.fan@nxp.com>
> > ---
> >  drivers/remoteproc/imx_rproc.c | 41
> > +++++++++++++++++++++++++++++++---
> >  1 file changed, 38 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/remoteproc/imx_rproc.c
> > b/drivers/remoteproc/imx_rproc.c index 49cce9dd55c7..8326193c13d6
> > 100644
> > --- a/drivers/remoteproc/imx_rproc.c
> > +++ b/drivers/remoteproc/imx_rproc.c
> > @@ -3,6 +3,7 @@
> >   * Copyright (c) 2017 Pengutronix, Oleksij Rempel
> <kernel@pengutronix.de>
> >   */
> >
> > +#include <dt-bindings/firmware/imx/rsrc.h>
> >  #include <linux/arm-smccc.h>
> >  #include <linux/clk.h>
> >  #include <linux/err.h>
> > @@ -75,10 +76,13 @@ struct imx_rproc_mem {
> >  	size_t size;
> >  };
> >
> > -/* att flags */
> > +/* att flags: lower 16 bits specifying core, higher 16 bits for flags
> > +*/
> >  /* M4 own area. Can be mapped at probe */
> > -#define ATT_OWN		BIT(1)
> > -#define ATT_IOMEM	BIT(2)
> > +#define ATT_OWN         BIT(31)
> > +#define ATT_IOMEM       BIT(30)
> > +
> > +#define ATT_CORE_MASK   0xffff
> > +#define ATT_CORE(I)     BIT((I))
> >
> >  struct imx_rproc {
> >  	struct device			*dev;
> > @@ -99,6 +103,7 @@ struct imx_rproc {
> >  	u32				rsrc_id;	/* resource id */
> >  	u32				entry;		/* cpu start address */
> >  	int                             num_pd;
> > +	u32				core_index;
> >  	struct device                   **pd_dev;
> >  	struct device_link              **pd_dev_link;
> >  };
> > @@ -129,6 +134,19 @@ static const struct imx_rproc_att
> imx_rproc_att_imx93[] = {
> >  	{ 0xD0000000, 0xa0000000, 0x10000000, 0 },  };
> >
> > +static const struct imx_rproc_att imx_rproc_att_imx8qm[] = {
> > +	/* dev addr , sys addr  , size      , flags */
> > +	{ 0x08000000, 0x08000000, 0x10000000, 0},
> > +	/* TCML */
> > +	{ 0x1FFE0000, 0x34FE0000, 0x00020000, ATT_OWN | ATT_IOMEM |
> ATT_CORE(0)},
> > +	{ 0x1FFE0000, 0x38FE0000, 0x00020000, ATT_OWN | ATT_IOMEM |
> ATT_CORE(1)},
> > +	/* TCMU */
> > +	{ 0x20000000, 0x35000000, 0x00020000, ATT_OWN | ATT_IOMEM |
> ATT_CORE(0)},
> > +	{ 0x20000000, 0x39000000, 0x00020000, ATT_OWN | ATT_IOMEM |
> ATT_CORE(1)},
> > +	/* DDR (Data) */
> > +	{ 0x80000000, 0x80000000, 0x60000000, 0 }, };
> > +
> >  static const struct imx_rproc_att imx_rproc_att_imx8qxp[] = {
> >  	{ 0x08000000, 0x08000000, 0x10000000, 0 },
> >  	/* TCML/U */
> > @@ -279,6 +297,12 @@ static const struct imx_rproc_dcfg
> imx_rproc_cfg_imx8mq = {
> >  	.method		= IMX_RPROC_MMIO,
> >  };
> >
> > +static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qm = {
> > +	.att            = imx_rproc_att_imx8qm,
> > +	.att_size       = ARRAY_SIZE(imx_rproc_att_imx8qm),
> > +	.method         = IMX_RPROC_SCU_API,
> > +};
> > +
> >  static const struct imx_rproc_dcfg imx_rproc_cfg_imx8qxp = {
> >  	.att		= imx_rproc_att_imx8qxp,
> >  	.att_size	= ARRAY_SIZE(imx_rproc_att_imx8qxp),
> > @@ -395,6 +419,11 @@ static int imx_rproc_da_to_sys(struct imx_rproc
> *priv, u64 da,
> >  	for (i = 0; i < dcfg->att_size; i++) {
> >  		const struct imx_rproc_att *att = &dcfg->att[i];
> >
> > +		if (att->flags & ATT_CORE_MASK) {
> > +			if (!((BIT(priv->core_index)) & (att->flags & ATT_CORE_MASK)))
> > +				continue;
> > +		}
> 
> This is very cryptic - I just spent 20 minutes looking at it and I'm still not sure I
> got the full meaning.  Please add enough comments to make things obvious
> on first read.

There are two generic M4 cores in i.MX8QM, so core_index is 0 for M4_0,
and 1 for M4_1.

In the memory mapping array:
        { 0x1FFE0000, 0x34FE0000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(0)},                   
        { 0x1FFE0000, 0x38FE0000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(1)},                   
        /* TCMU */                                                                                  
        { 0x20000000, 0x35000000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(0)},                   
        { 0x20000000, 0x39000000, 0x00020000, ATT_OWN | ATT_IOMEM | ATT_CORE(1)},

ATT_CORE(0) means it is for M4_0, ATT_CORE(1) for M4_1.

Back to this piece code:
                if (att->flags & ATT_CORE_MASK) {                                                   
                        if (!((BIT(priv->core_index)) & (att->flags & ATT_CORE_MASK)))              
                                continue;                                                           
                }

Taking M4_1 for example, priv->core_index is 1. So when it need translate an address with
ATT_CORE(X) flag, it should ignore ATT_CORE(0) entries. Hope this is clear. For adding
comments, how do you think:

/* Bypass the entries that not belong to the current remote core */

Thanks,
Peng.

> 
> I am done reviewing this patchset.
> 
> Thanks,
> Mathieu
> 
> 
> > +
> >  		if (da >= att->da && da + len < att->da + att->size) {
> >  			unsigned int offset = da - att->da;
> >
> > @@ -815,6 +844,11 @@ static int imx_rproc_detect_mode(struct
> imx_rproc *priv)
> >  			return ret;
> >  		}
> >
> > +		if (priv->rsrc_id == IMX_SC_R_M4_1_PID0)
> > +			priv->core_index = 1;
> > +		else
> > +			priv->core_index = 0;
> > +
> >  		/*
> >  		 * If Mcore resource is not owned by Acore partition, It is kicked by
> ROM,
> >  		 * and Linux could only do IPC with Mcore and nothing else.
> > @@ -1008,6 +1042,7 @@ static const struct of_device_id
> imx_rproc_of_match[] = {
> >  	{ .compatible = "fsl,imx8mn-cm7", .data = &imx_rproc_cfg_imx8mn },
> >  	{ .compatible = "fsl,imx8mp-cm7", .data = &imx_rproc_cfg_imx8mn },
> >  	{ .compatible = "fsl,imx8qxp-cm4", .data = &imx_rproc_cfg_imx8qxp },
> > +	{ .compatible = "fsl,imx8qm-cm4", .data = &imx_rproc_cfg_imx8qm },
> >  	{ .compatible = "fsl,imx8ulp-cm33", .data = &imx_rproc_cfg_imx8ulp },
> >  	{ .compatible = "fsl,imx93-cm33", .data = &imx_rproc_cfg_imx93 },
> >  	{},
> > --
> > 2.25.1
> >