[PATCH v2 00/17] remoteproc: Add support for synchronisation with MCU

[PATCH v2 00/17] remoteproc: Add support for synchronisation with MCU

[Mathieu Poirier]

This is the second revision of this set that tries to address the
problem of synchronising with an MCU with as much flexibility as
possible.

New in this revision is a fix for a couple of bugs I found while
testing things.  First with the helper macro in patch 09 and the
suppression of a boot message when synchronising with an MCU
in patch 12.  I have completely removed what used to be patch 18,
the example on how to use the new API.  This will be the subject
of an upcoming patchset.

Tested on ST's mp157c platform.  Applies on v5.6-rc7 to keep things
simple.

Comments would be much appreciated.

Thanks,
Mathieu

Mathieu Poirier (17):
  remoteproc: Add new operation and state machine for MCU
    synchronisation
  remoteproc: Introduce function rproc_set_mcu_sync_state()
  remoteproc: Split firmware name allocation from rproc_alloc()
  remoteproc: Split rproc_ops allocation from rproc_alloc()
  remoteproc: Get rid of tedious error path
  remoteproc: Introduce function rproc_alloc_internals()
  remoteproc: Introduce function rproc_alloc_state_machine()
  remoteproc: Allocate synchronisation state machine
  remoteproc: Call the right core function based on synchronisation
    state
  remoteproc: Decouple firmware load and remoteproc booting
  remoteproc: Repurpose function rproc_trigger_auto_boot()
  remoteproc: Rename function rproc_fw_boot()
  remoteproc: Introducting new functions to start and stop an MCU
  remoteproc: Refactor function rproc_trigger_recovery()
  remoteproc: Correctly deal with MCU synchronisation when changing FW
    image
  remoteproc: Correctly deal with MCU synchronisation when changing
    state
  remoteproc: Make MCU synchronisation state changes on stop and crashed

 drivers/remoteproc/remoteproc_core.c     | 387 ++++++++++++++++++-----
 drivers/remoteproc/remoteproc_debugfs.c  |  31 ++
 drivers/remoteproc/remoteproc_internal.h |  91 ++++--
 drivers/remoteproc/remoteproc_sysfs.c    |  57 +++-
 include/linux/remoteproc.h               |  28 +-
 5 files changed, 487 insertions(+), 107 deletions(-)

-- 
2.20.1

[PATCH v2 01/17] remoteproc: Add new operation and state machine for MCU synchronisation

[Mathieu Poirier]

Add a new rproc_ops sync_ops to support use cases where the remoteproc
core is synchronisting with the MCU.  When exactly to use the sync_ops is
directed by the states in struct rproc_sync_states.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_debugfs.c  | 31 ++++++++++++++++++++++++
 drivers/remoteproc/remoteproc_internal.h |  5 ++++
 include/linux/remoteproc.h               | 23 +++++++++++++++++-
 3 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/drivers/remoteproc/remoteproc_debugfs.c b/drivers/remoteproc/remoteproc_debugfs.c
index dd93cf04e17f..187bcc67f997 100644
--- a/drivers/remoteproc/remoteproc_debugfs.c
+++ b/drivers/remoteproc/remoteproc_debugfs.c
@@ -311,6 +311,35 @@ static const struct file_operations rproc_carveouts_ops = {
 	.release	= single_release,
 };
 
+/* Expose synchronisation states via debugfs */
+static int rproc_sync_states_show(struct seq_file *seq, void *p)
+{
+	struct rproc *rproc = seq->private;
+
+	seq_printf(seq, "Sync with MCU: %s\n",
+		   rproc->sync_with_mcu ? "true" : "false");
+	seq_printf(seq, "On init: %s\n",
+		   rproc->sync_states->on_init ? "true" : "false");
+	seq_printf(seq, "After stop: %s\n",
+		   rproc->sync_states->after_stop ? "true" : "false");
+	seq_printf(seq, "After crash: %s\n",
+		   rproc->sync_states->after_crash ? "true" : "false");
+
+	return 0;
+}
+
+static int rproc_sync_states_open(struct inode *inode, struct file *file)
+{
+	return single_open(file, rproc_sync_states_show, inode->i_private);
+}
+
+static const struct file_operations rproc_sync_states_ops = {
+	.open		= rproc_sync_states_open,
+	.read		= seq_read,
+	.llseek		= seq_lseek,
+	.release	= single_release,
+};
+
 void rproc_remove_trace_file(struct dentry *tfile)
 {
 	debugfs_remove(tfile);
@@ -357,6 +386,8 @@ void rproc_create_debug_dir(struct rproc *rproc)
 			    rproc, &rproc_rsc_table_ops);
 	debugfs_create_file("carveout_memories", 0400, rproc->dbg_dir,
 			    rproc, &rproc_carveouts_ops);
+	debugfs_create_file("sync_states", 0400, rproc->dbg_dir,
+			    rproc, &rproc_sync_states_ops);
 }
 
 void __init rproc_init_debugfs(void)
diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
index 493ef9262411..5c93de5e00bb 100644
--- a/drivers/remoteproc/remoteproc_internal.h
+++ b/drivers/remoteproc/remoteproc_internal.h
@@ -63,6 +63,11 @@ struct resource_table *rproc_elf_find_loaded_rsc_table(struct rproc *rproc,
 struct rproc_mem_entry *
 rproc_find_carveout_by_name(struct rproc *rproc, const char *name, ...);
 
+static inline bool rproc_sync_with_mcu(struct rproc *rproc)
+{
+	return rproc->sync_with_mcu;
+}
+
 static inline
 int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
 {
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index 16ad66683ad0..d115e47d702d 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -353,6 +353,21 @@ enum rsc_handling_status {
 	RSC_IGNORED	= 1,
 };
 
+/**
+ * struct rproc_sync_states - platform specific states indicating which
+ *			      rproc_ops to use at specific times during
+ *			      the MCU lifecycle.
+ * @on_init: true if synchronising with MCU at system initialisation time
+ * @after_stop: true if synchronising with MCU after stopped from the
+ *		command line
+ * @after_crash: true if synchonising with MCU after the MCU has crashed
+ */
+struct rproc_sync_states {
+	bool on_init;
+	bool after_stop;
+	bool after_crash;
+};
+
 /**
  * struct rproc_ops - platform-specific device handlers
  * @start:	power on the device and boot it
@@ -456,6 +471,9 @@ struct rproc_dump_segment {
  * @firmware: name of firmware file to be loaded
  * @priv: private data which belongs to the platform-specific rproc module
  * @ops: platform-specific start/stop rproc handlers
+ * @sync_ops: paltform-specific start/stop rproc handlers when
+ *	      synchronising with a remote processor.
+ * @sync_states: Determine the rproc_ops to choose in specific states.
  * @dev: virtual device for refcounting and common remoteproc behavior
  * @power: refcount of users who need this rproc powered up
  * @state: state of the device
@@ -479,6 +497,7 @@ struct rproc_dump_segment {
  * @table_sz: size of @cached_table
  * @has_iommu: flag to indicate if remote processor is behind an MMU
  * @auto_boot: flag to indicate if remote processor should be auto-started
+ * @sync_with_mcu: true if currently synchronising with MCU
  * @dump_segments: list of segments in the firmware
  * @nb_vdev: number of vdev currently handled by rproc
  */
@@ -488,7 +507,8 @@ struct rproc {
 	const char *name;
 	char *firmware;
 	void *priv;
-	struct rproc_ops *ops;
+	struct rproc_ops *ops, *sync_ops;
+	struct rproc_sync_states *sync_states;
 	struct device dev;
 	atomic_t power;
 	unsigned int state;
@@ -512,6 +532,7 @@ struct rproc {
 	size_t table_sz;
 	bool has_iommu;
 	bool auto_boot;
+	bool sync_with_mcu;
 	struct list_head dump_segments;
 	int nb_vdev;
 };
-- 
2.20.1

[PATCH v2 02/17] remoteproc: Introduce function rproc_set_mcu_sync_state()

[Mathieu Poirier]

Introduce function rproc_set_mcu_sync_state() to set the synchronisation
state of the MCU at various stages of the lifecycle process.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_internal.h | 38 ++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
index 5c93de5e00bb..73ea32df0156 100644
--- a/drivers/remoteproc/remoteproc_internal.h
+++ b/drivers/remoteproc/remoteproc_internal.h
@@ -24,6 +24,26 @@ struct rproc_debug_trace {
 	struct rproc_mem_entry trace_mem;
 };
 
+/*
+ * enum rproc_sync_states - remote processsor sync states
+ * @RPROC_SYNC_STATE_INIT	state to use when the remoteproc core
+ *				is initialising.
+ * @RPROC_SYNC_STATE_SHUTDOWN	state to use after the remoteproc core
+ *				has shutdown (rproc_shutdown()) the MCU.
+ * @RPROC_SYNC_STATE_CRASHED	state to use after the MCU has crashed but
+ *				has not been recovered by the remoteproc
+ *				core yet.
+ *
+ * Keeping these separate from the enum rproc_state in order to avoid
+ * introducing coupling between the state of the MCU and the synchronisation
+ * operation to use.
+ */
+enum rproc_mcu_sync_states {
+	RPROC_SYNC_STATE_INIT,
+	RPROC_SYNC_STATE_SHUTDOWN,
+	RPROC_SYNC_STATE_CRASHED,
+};
+
 /* from remoteproc_core.c */
 void rproc_release(struct kref *kref);
 irqreturn_t rproc_vq_interrupt(struct rproc *rproc, int vq_id);
@@ -68,6 +88,24 @@ static inline bool rproc_sync_with_mcu(struct rproc *rproc)
 	return rproc->sync_with_mcu;
 }
 
+static inline void rproc_set_mcu_sync_state(struct rproc *rproc,
+					    unsigned int state)
+{
+	switch (state) {
+	case RPROC_SYNC_STATE_INIT:
+		rproc->sync_with_mcu = rproc->sync_states->on_init;
+		break;
+	case RPROC_SYNC_STATE_SHUTDOWN:
+		rproc->sync_with_mcu = rproc->sync_states->after_stop;
+		break;
+	case RPROC_SYNC_STATE_CRASHED:
+		rproc->sync_with_mcu = rproc->sync_states->after_crash;
+		break;
+	default:
+		break;
+	}
+}
+
 static inline
 int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
 {
-- 
2.20.1

[PATCH v2 03/17] remoteproc: Split firmware name allocation from rproc_alloc()

[Mathieu Poirier]

Make the firmware name allocation a function on its own in order to
introduce more flexibility to function rproc_alloc().

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 62 +++++++++++++++++-----------
 1 file changed, 39 insertions(+), 23 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 097f33e4f1f3..c0871f69929b 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1962,6 +1962,36 @@ static const struct device_type rproc_type = {
 	.release	= rproc_type_release,
 };
 
+static int rproc_alloc_firmware(struct rproc *rproc,
+				const char *name, const char *firmware)
+{
+	char *p, *template = "rproc-%s-fw";
+	int name_len;
+
+	if (!rproc || !name)
+		return -EINVAL;
+
+	if (!firmware) {
+		/*
+		 * If the caller didn't pass in a firmware name then
+		 * construct a default name.
+		 */
+		name_len = strlen(name) + strlen(template) - 2 + 1;
+		p = kmalloc(name_len, GFP_KERNEL);
+		if (!p)
+			return -ENOMEM;
+		snprintf(p, name_len, template, name);
+	} else {
+		p = kstrdup(firmware, GFP_KERNEL);
+		if (!p)
+			return -ENOMEM;
+	}
+
+	rproc->firmware = p;
+
+	return 0;
+}
+
 /**
  * rproc_alloc() - allocate a remote processor handle
  * @dev: the underlying device
@@ -1990,42 +2020,24 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 			  const char *firmware, int len)
 {
 	struct rproc *rproc;
-	char *p, *template = "rproc-%s-fw";
-	int name_len;
 
 	if (!dev || !name || !ops)
 		return NULL;
 
-	if (!firmware) {
-		/*
-		 * If the caller didn't pass in a firmware name then
-		 * construct a default name.
-		 */
-		name_len = strlen(name) + strlen(template) - 2 + 1;
-		p = kmalloc(name_len, GFP_KERNEL);
-		if (!p)
-			return NULL;
-		snprintf(p, name_len, template, name);
-	} else {
-		p = kstrdup(firmware, GFP_KERNEL);
-		if (!p)
-			return NULL;
-	}
-
 	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
-	if (!rproc) {
-		kfree(p);
+	if (!rproc)
 		return NULL;
-	}
+
+	if (rproc_alloc_firmware(rproc, name, firmware))
+		goto free_rproc;
 
 	rproc->ops = kmemdup(ops, sizeof(*ops), GFP_KERNEL);
 	if (!rproc->ops) {
-		kfree(p);
+		kfree(rproc->firmware);
 		kfree(rproc);
 		return NULL;
 	}
 
-	rproc->firmware = p;
 	rproc->name = name;
 	rproc->priv = &rproc[1];
 	rproc->auto_boot = true;
@@ -2073,6 +2085,10 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 	rproc->state = RPROC_OFFLINE;
 
 	return rproc;
+
+free_rproc:
+	kfree(rproc);
+	return NULL;
 }
 EXPORT_SYMBOL(rproc_alloc);
 
-- 
2.20.1

[PATCH v2 04/17] remoteproc: Split rproc_ops allocation from rproc_alloc()

[Mathieu Poirier]

Make the rproc_ops allocation a function on its own in order to
introduce more flexibility to function rproc_alloc().

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 45 ++++++++++++++++++----------
 1 file changed, 30 insertions(+), 15 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index c0871f69929b..d22e557f27ed 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1992,6 +1992,32 @@ static int rproc_alloc_firmware(struct rproc *rproc,
 	return 0;
 }
 
+static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops)
+{
+	if (!rproc)
+		return -EINVAL;
+
+	/* Nothing to do if there isn't and ops to work with */
+	if (!ops)
+		return 0;
+
+	rproc->ops = kmemdup(ops, sizeof(*ops), GFP_KERNEL);
+	if (!rproc->ops)
+		return -ENOMEM;
+
+	/* Default to ELF loader if no load function is specified */
+	if (!rproc->ops->load) {
+		rproc->ops->load = rproc_elf_load_segments;
+		rproc->ops->parse_fw = rproc_elf_load_rsc_table;
+		rproc->ops->find_loaded_rsc_table =
+						rproc_elf_find_loaded_rsc_table;
+		rproc->ops->sanity_check = rproc_elf_sanity_check;
+		rproc->ops->get_boot_addr = rproc_elf_get_boot_addr;
+	}
+
+	return 0;
+}
+
 /**
  * rproc_alloc() - allocate a remote processor handle
  * @dev: the underlying device
@@ -2031,12 +2057,8 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 	if (rproc_alloc_firmware(rproc, name, firmware))
 		goto free_rproc;
 
-	rproc->ops = kmemdup(ops, sizeof(*ops), GFP_KERNEL);
-	if (!rproc->ops) {
-		kfree(rproc->firmware);
-		kfree(rproc);
-		return NULL;
-	}
+	if (rproc_alloc_ops(rproc, ops))
+		goto free_firmware;
 
 	rproc->name = name;
 	rproc->priv = &rproc[1];
@@ -2060,15 +2082,6 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 
 	atomic_set(&rproc->power, 0);
 
-	/* Default to ELF loader if no load function is specified */
-	if (!rproc->ops->load) {
-		rproc->ops->load = rproc_elf_load_segments;
-		rproc->ops->parse_fw = rproc_elf_load_rsc_table;
-		rproc->ops->find_loaded_rsc_table = rproc_elf_find_loaded_rsc_table;
-		rproc->ops->sanity_check = rproc_elf_sanity_check;
-		rproc->ops->get_boot_addr = rproc_elf_get_boot_addr;
-	}
-
 	mutex_init(&rproc->lock);
 
 	idr_init(&rproc->notifyids);
@@ -2086,6 +2099,8 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 
 	return rproc;
 
+free_firmware:
+	kfree(rproc->firmware);
 free_rproc:
 	kfree(rproc);
 	return NULL;
-- 
2.20.1

[PATCH v2 05/17] remoteproc: Get rid of tedious error path

[Mathieu Poirier]

Get rid of tedious error management by moving firmware and operation
allocation after calling device_initialize().  That way we take advantage
of the automatic call to rproc_type_release() to cleanup after ourselves
when put_device() is called.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index d22e557f27ed..ee277bc5556c 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -2054,12 +2054,6 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 	if (!rproc)
 		return NULL;
 
-	if (rproc_alloc_firmware(rproc, name, firmware))
-		goto free_rproc;
-
-	if (rproc_alloc_ops(rproc, ops))
-		goto free_firmware;
-
 	rproc->name = name;
 	rproc->priv = &rproc[1];
 	rproc->auto_boot = true;
@@ -2070,12 +2064,17 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 	rproc->dev.class = &rproc_class;
 	rproc->dev.driver_data = rproc;
 
+	if (rproc_alloc_firmware(rproc, name, firmware))
+		goto out;
+
+	if (rproc_alloc_ops(rproc, ops))
+		goto out;
+
 	/* Assign a unique device index and name */
 	rproc->index = ida_simple_get(&rproc_dev_index, 0, 0, GFP_KERNEL);
 	if (rproc->index < 0) {
 		dev_err(dev, "ida_simple_get failed: %d\n", rproc->index);
-		put_device(&rproc->dev);
-		return NULL;
+		goto out;
 	}
 
 	dev_set_name(&rproc->dev, "remoteproc%d", rproc->index);
@@ -2098,11 +2097,8 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 	rproc->state = RPROC_OFFLINE;
 
 	return rproc;
-
-free_firmware:
-	kfree(rproc->firmware);
-free_rproc:
-	kfree(rproc);
+out:
+	put_device(&rproc->dev);
 	return NULL;
 }
 EXPORT_SYMBOL(rproc_alloc);
-- 
2.20.1

[PATCH v2 06/17] remoteproc: Introduce function rproc_alloc_internals()

[Mathieu Poirier]

In preparation to allocate the synchronisation operation and state
machine, spin off a new function in order to keep rproc_alloc() as
clean as possible.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index ee277bc5556c..9da245734db6 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -2018,6 +2018,26 @@ static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops)
 	return 0;
 }
 
+static int rproc_alloc_internals(struct rproc *rproc, const char *name,
+				 const struct rproc_ops *boot_ops,
+				 const char *firmware, int len)
+{
+	int ret;
+
+	/* We have a boot_ops so allocate firmware name and operations */
+	if (boot_ops) {
+		ret = rproc_alloc_firmware(rproc, name, firmware);
+		if (ret)
+			return ret;
+
+		ret = rproc_alloc_ops(rproc, boot_ops);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
 /**
  * rproc_alloc() - allocate a remote processor handle
  * @dev: the underlying device
@@ -2064,10 +2084,8 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 	rproc->dev.class = &rproc_class;
 	rproc->dev.driver_data = rproc;
 
-	if (rproc_alloc_firmware(rproc, name, firmware))
-		goto out;
-
-	if (rproc_alloc_ops(rproc, ops))
+	if (rproc_alloc_internals(rproc, name, ops,
+				  firmware, len))
 		goto out;
 
 	/* Assign a unique device index and name */
-- 
2.20.1

[PATCH v2 07/17] remoteproc: Introduce function rproc_alloc_state_machine()

[Mathieu Poirier]

Introducing new function rproc_alloc_state_machine() to allocate
the MCU synchronisation operations and position it as the central
remoteproc core allocation function.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 84 +++++++++++++++++++++++++---
 include/linux/remoteproc.h           |  5 ++
 2 files changed, 81 insertions(+), 8 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 9da245734db6..02dbb826aa29 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1954,6 +1954,7 @@ static void rproc_type_release(struct device *dev)
 
 	kfree(rproc->firmware);
 	kfree(rproc->ops);
+	kfree(rproc->sync_ops);
 	kfree(rproc);
 }
 
@@ -2018,12 +2019,34 @@ static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops)
 	return 0;
 }
 
+static int rproc_alloc_sync_ops(struct rproc *rproc,
+				const struct rproc_ops *sync_ops)
+{
+	/*
+	 * Given the unlimited amount of possibilities when
+	 * synchronising with an MCU, no constraints are imposed
+	 * on sync_ops.
+	 */
+	rproc->sync_ops = kmemdup(sync_ops,
+				  sizeof(*sync_ops), GFP_KERNEL);
+	if (!rproc->sync_ops)
+		return -ENOMEM;
+
+	return 0;
+}
+
 static int rproc_alloc_internals(struct rproc *rproc, const char *name,
 				 const struct rproc_ops *boot_ops,
+				 const struct rproc_ops *sync_ops,
+				 struct rproc_sync_states *sync_states,
 				 const char *firmware, int len)
 {
 	int ret;
 
+	/* We need at least a boot or a sync ops. */
+	if (!boot_ops && !sync_ops)
+		return -EINVAL;
+
 	/* We have a boot_ops so allocate firmware name and operations */
 	if (boot_ops) {
 		ret = rproc_alloc_firmware(rproc, name, firmware);
@@ -2035,14 +2058,23 @@ static int rproc_alloc_internals(struct rproc *rproc, const char *name,
 			return ret;
 	}
 
+	/* Allocate a sync_ops if need be */
+	if (sync_ops) {
+		ret = rproc_alloc_sync_ops(rproc, sync_ops);
+		if (ret)
+			return ret;
+	}
+
 	return 0;
 }
 
 /**
- * rproc_alloc() - allocate a remote processor handle
+ * rproc_alloc_state_machine() - allocate a remote processor handle
  * @dev: the underlying device
  * @name: name of this remote processor
  * @ops: platform-specific handlers (mainly start/stop)
+ * @sync_ops: platform-specific handlers for synchronising with MCU
+ * @sync_states: states in which @ops and @sync_ops are to be used
  * @firmware: name of firmware file to load, can be NULL
  * @len: length of private data needed by the rproc driver (in bytes)
  *
@@ -2061,13 +2093,15 @@ static int rproc_alloc_internals(struct rproc *rproc, const char *name,
  * Note: _never_ directly deallocate @rproc, even if it was not registered
  * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
  */
-struct rproc *rproc_alloc(struct device *dev, const char *name,
-			  const struct rproc_ops *ops,
-			  const char *firmware, int len)
+struct rproc *rproc_alloc_state_machine(struct device *dev, const char *name,
+					const struct rproc_ops *ops,
+					const struct rproc_ops *sync_ops,
+					struct rproc_sync_states *sync_states,
+					const char *firmware, int len)
 {
 	struct rproc *rproc;
 
-	if (!dev || !name || !ops)
+	if (!dev || !name)
 		return NULL;
 
 	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
@@ -2084,8 +2118,8 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 	rproc->dev.class = &rproc_class;
 	rproc->dev.driver_data = rproc;
 
-	if (rproc_alloc_internals(rproc, name, ops,
-				  firmware, len))
+	if (rproc_alloc_internals(rproc, name, ops, sync_ops,
+				  sync_states, firmware, len))
 		goto out;
 
 	/* Assign a unique device index and name */
@@ -2119,7 +2153,41 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 	put_device(&rproc->dev);
 	return NULL;
 }
-EXPORT_SYMBOL(rproc_alloc);
+EXPORT_SYMBOL(rproc_alloc_state_machine);
+
+/**
+ * rproc_alloc() - allocate a remote processor handle
+ * @dev: the underlying device
+ * @name: name of this remote processor
+ * @ops: platform-specific handlers (mainly start/stop)
+ * @firmware: name of firmware file to load, can be NULL
+ * @len: length of private data needed by the rproc driver (in bytes)
+ *
+ * Allocates a new remote processor handle, but does not register
+ * it yet. if @firmware is NULL, a default name is used.
+ *
+ * This function should be used by rproc implementations during initialization
+ * of the remote processor.
+ *
+ * After creating an rproc handle using this function, and when ready,
+ * implementations should then call rproc_add() to complete
+ * the registration of the remote processor.
+ *
+ * On success the new rproc is returned, and on failure, NULL.
+ *
+ * Note: _never_ directly deallocate @rproc, even if it was not registered
+ * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
+ */
+struct rproc *rproc_alloc(struct device *dev, const char *name,
+			  const struct rproc_ops *ops,
+			  const char *firmware, int len)
+{
+	if (!name && !firmware)
+		return NULL;
+
+	return rproc_alloc_state_machine(dev, name, ops, NULL, NULL,
+					 firmware, len);
+}
 
 /**
  * rproc_free() - unroll rproc_alloc()
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index d115e47d702d..d1214487daac 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -611,6 +611,11 @@ struct rproc *rproc_get_by_child(struct device *dev);
 struct rproc *rproc_alloc(struct device *dev, const char *name,
 			  const struct rproc_ops *ops,
 			  const char *firmware, int len);
+struct rproc *rproc_alloc_state_machine(struct device *dev, const char *name,
+					const struct rproc_ops *ops,
+					const struct rproc_ops *sync_ops,
+					struct rproc_sync_states *sync_states,
+					const char *firmware, int len);
 void rproc_put(struct rproc *rproc);
 int rproc_add(struct rproc *rproc);
 int rproc_del(struct rproc *rproc);
-- 
2.20.1

[PATCH v2 08/17] remoteproc: Allocate synchronisation state machine

[Mathieu Poirier]

This patch allocates a synchronisation state machine, either provided or
not by users, in order to enact the proper behavior requested by the
platform or specific scenarios.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 59 +++++++++++++++++++++++++++-
 1 file changed, 58 insertions(+), 1 deletion(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 02dbb826aa29..1578a9c70422 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1955,6 +1955,7 @@ static void rproc_type_release(struct device *dev)
 	kfree(rproc->firmware);
 	kfree(rproc->ops);
 	kfree(rproc->sync_ops);
+	kfree(rproc->sync_states);
 	kfree(rproc);
 }
 
@@ -2035,6 +2036,59 @@ static int rproc_alloc_sync_ops(struct rproc *rproc,
 	return 0;
 }
 
+static int rproc_alloc_sync_states(struct rproc *rproc,
+				   const struct rproc_ops *boot_ops,
+				   const struct rproc_ops *sync_ops,
+				   struct rproc_sync_states *sync_states)
+{
+	struct rproc_sync_states *st;
+
+	/* At least one set of operation is needed */
+	if (!boot_ops && !sync_ops)
+		return -EINVAL;
+
+	/* We have a synchronisation state machine, no need to build one */
+	if (sync_states) {
+		st = kmemdup(sync_states, sizeof(*st), GFP_KERNEL);
+		if (!st)
+			return -ENOMEM;
+
+		/* Nothing else to do */
+		goto out;
+	}
+
+	/* Allocate synchronisation state machine */
+	st = kzalloc(sizeof(*st), GFP_KERNEL);
+	if (!st)
+		return -ENOMEM;
+
+	/*
+	 * We have a boot_ops and no sync_ops - build a state machine that
+	 * does _not_ synchronise with an MCU.
+	 */
+	if (boot_ops && !sync_ops) {
+		st->on_init = st->after_stop = st->after_crash = false;
+		goto out;
+	}
+
+	/*
+	 * We have a sync_ops and an no boot_ops - build a state machine that
+	 * _only_ synchronises with an MCU.
+	 */
+	if (sync_ops && !boot_ops) {
+		st->on_init = st->after_stop = st->after_crash = true;
+		goto out;
+	}
+
+out:
+	rproc->sync_with_mcu = st->on_init;
+	/* And the synchronisation state machine to use */
+	rproc->sync_states = st;
+	/* Tell the core what to do when initialising */
+	rproc_set_mcu_sync_state(rproc, RPROC_SYNC_STATE_INIT);
+	return 0;
+}
+
 static int rproc_alloc_internals(struct rproc *rproc, const char *name,
 				 const struct rproc_ops *boot_ops,
 				 const struct rproc_ops *sync_ops,
@@ -2065,7 +2119,10 @@ static int rproc_alloc_internals(struct rproc *rproc, const char *name,
 			return ret;
 	}
 
-	return 0;
+	/* Finally allocate the synchronisation state machine */
+	ret = rproc_alloc_sync_states(rproc, boot_ops, sync_ops, sync_states);
+
+	return ret;
 }
 
 /**
-- 
2.20.1

[PATCH v2 09/17] remoteproc: Call the right core function based on synchronisation state

[Mathieu Poirier]

Call the right core function based on whether we should synchronise
with an MCU or boot it from scratch.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_internal.h | 36 +++++++++++-------------
 1 file changed, 17 insertions(+), 19 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
index 73ea32df0156..5f711ceb97ba 100644
--- a/drivers/remoteproc/remoteproc_internal.h
+++ b/drivers/remoteproc/remoteproc_internal.h
@@ -106,38 +106,41 @@ static inline void rproc_set_mcu_sync_state(struct rproc *rproc,
 	}
 }
 
+#define RPROC_OPS_HELPER(__operation, ...)				\
+	do {								\
+		if (rproc_sync_with_mcu(rproc)) {			\
+			if (!rproc->sync_ops ||				\
+			    !rproc->sync_ops->__operation)		\
+				return 0;				\
+			return rproc->sync_ops->__operation(__VA_ARGS__); \
+		} else if (rproc->ops && rproc->ops->__operation)	\
+			return rproc->ops->__operation(__VA_ARGS__);	\
+	} while (0)							\
+
 static inline
 int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
 {
-	if (rproc->ops->sanity_check)
-		return rproc->ops->sanity_check(rproc, fw);
-
+	RPROC_OPS_HELPER(sanity_check, rproc, fw);
 	return 0;
 }
 
 static inline
 u32 rproc_get_boot_addr(struct rproc *rproc, const struct firmware *fw)
 {
-	if (rproc->ops->get_boot_addr)
-		return rproc->ops->get_boot_addr(rproc, fw);
-
+	RPROC_OPS_HELPER(get_boot_addr, rproc, fw);
 	return 0;
 }
 
 static inline
 int rproc_load_segments(struct rproc *rproc, const struct firmware *fw)
 {
-	if (rproc->ops->load)
-		return rproc->ops->load(rproc, fw);
-
+	RPROC_OPS_HELPER(load, rproc, fw);
 	return -EINVAL;
 }
 
 static inline int rproc_parse_fw(struct rproc *rproc, const struct firmware *fw)
 {
-	if (rproc->ops->parse_fw)
-		return rproc->ops->parse_fw(rproc, fw);
-
+	RPROC_OPS_HELPER(parse_fw, rproc, fw);
 	return 0;
 }
 
@@ -145,10 +148,7 @@ static inline
 int rproc_handle_rsc(struct rproc *rproc, u32 rsc_type, void *rsc, int offset,
 		     int avail)
 {
-	if (rproc->ops->handle_rsc)
-		return rproc->ops->handle_rsc(rproc, rsc_type, rsc, offset,
-					      avail);
-
+	RPROC_OPS_HELPER(handle_rsc, rproc, rsc_type, rsc, offset, avail);
 	return RSC_IGNORED;
 }
 
@@ -156,9 +156,7 @@ static inline
 struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
 						   const struct firmware *fw)
 {
-	if (rproc->ops->find_loaded_rsc_table)
-		return rproc->ops->find_loaded_rsc_table(rproc, fw);
-
+	RPROC_OPS_HELPER(find_loaded_rsc_table, rproc, fw);
 	return NULL;
 }
 
-- 
2.20.1

[PATCH v2 10/17] remoteproc: Decouple firmware load and remoteproc booting

[Mathieu Poirier]

In preparation to support scenarios where MCU firmware is loaded and
the MCU itself booted by another entity, split function rproc_boot()
to decouple the functions of loading the firwmare and starting the
initialisation process.  That way we can reuse the functionality
provided by the current implementation without invariably dealing
with firmware loading.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 63 +++++++++++++++++-----------
 1 file changed, 39 insertions(+), 24 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 1578a9c70422..7faee1396ef7 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1714,21 +1714,10 @@ static void rproc_crash_handler_work(struct work_struct *work)
 		rproc_trigger_recovery(rproc);
 }
 
-/**
- * rproc_boot() - boot a remote processor
- * @rproc: handle of a remote processor
- *
- * Boot a remote processor (i.e. load its firmware, power it on, ...).
- *
- * If the remote processor is already powered on, this function immediately
- * returns (successfully).
- *
- * Returns 0 on success, and an appropriate error value otherwise.
- */
-int rproc_boot(struct rproc *rproc)
+static int rproc_actuate(struct rproc *rproc,
+			 const struct firmware *firmware_p)
 {
-	const struct firmware *firmware_p;
-	struct device *dev;
+	struct device *dev = &rproc->dev;
 	int ret;
 
 	if (!rproc) {
@@ -1736,8 +1725,6 @@ int rproc_boot(struct rproc *rproc)
 		return -EINVAL;
 	}
 
-	dev = &rproc->dev;
-
 	ret = mutex_lock_interruptible(&rproc->lock);
 	if (ret) {
 		dev_err(dev, "can't lock rproc %s: %d\n", rproc->name, ret);
@@ -1756,24 +1743,52 @@ int rproc_boot(struct rproc *rproc)
 		goto unlock_mutex;
 	}
 
-	dev_info(dev, "powering up %s\n", rproc->name);
+	dev_info(dev, "%s %s\n",
+		 firmware_p ? "powering up" : "syncing with",
+		 rproc->name);
+
+	ret = rproc_fw_boot(rproc, firmware_p);
+	if (ret)
+		atomic_dec(&rproc->power);
+
+unlock_mutex:
+	mutex_unlock(&rproc->lock);
+	return ret;
+}
+
+/**
+ * rproc_boot() - boot a remote processor
+ * @rproc: handle of a remote processor
+ *
+ * Boot a remote processor (i.e. load its firmware, power it on, ...).
+ *
+ * If the remote processor is already powered on, this function immediately
+ * returns (successfully).
+ *
+ * Returns 0 on success, and an appropriate error value otherwise.
+ */
+int rproc_boot(struct rproc *rproc)
+{
+	const struct firmware *firmware_p;
+	struct device *dev = &rproc->dev;
+	int ret;
+
+	if (!rproc) {
+		pr_err("invalid rproc handle\n");
+		return -EINVAL;
+	}
 
 	/* load firmware */
 	ret = request_firmware(&firmware_p, rproc->firmware, dev);
 	if (ret < 0) {
 		dev_err(dev, "request_firmware failed: %d\n", ret);
-		goto downref_rproc;
+		return ret;
 	}
 
-	ret = rproc_fw_boot(rproc, firmware_p);
+	ret = rproc_actuate(rproc, firmware_p);
 
 	release_firmware(firmware_p);
 
-downref_rproc:
-	if (ret)
-		atomic_dec(&rproc->power);
-unlock_mutex:
-	mutex_unlock(&rproc->lock);
 	return ret;
 }
 EXPORT_SYMBOL(rproc_boot);
-- 
2.20.1

[PATCH v2 11/17] remoteproc: Repurpose function rproc_trigger_auto_boot()

[Mathieu Poirier]

Repurpose function rproc_trigger_auto_boot() so that it can deal with
scenarios where the MCU is already running.  As such give it a new name
to better represent the capabilities and add a call to rproc_actuate()
if instructed by the platform code to synchronise with the MCU rather
than boot it from scratch.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 7faee1396ef7..d57b47b0d6be 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -51,6 +51,8 @@ static int rproc_alloc_carveout(struct rproc *rproc,
 				struct rproc_mem_entry *mem);
 static int rproc_release_carveout(struct rproc *rproc,
 				  struct rproc_mem_entry *mem);
+static int rproc_actuate(struct rproc *rproc,
+			 const struct firmware *firmware_p);
 
 /* Unique indices for remoteproc devices */
 static DEFINE_IDA(rproc_dev_index);
@@ -1444,10 +1446,17 @@ static void rproc_auto_boot_callback(const struct firmware *fw, void *context)
 	release_firmware(fw);
 }
 
-static int rproc_trigger_auto_boot(struct rproc *rproc)
+static int rproc_trigger_auto_initiate(struct rproc *rproc)
 {
 	int ret;
 
+	/*
+	 * The MCU is already booted, all we need to do is synchronise with it.
+	 * No point dealing with a firmware image.
+	 */
+	if (rproc_sync_with_mcu(rproc))
+		return rproc_actuate(rproc, NULL);
+
 	/*
 	 * We're initiating an asynchronous firmware loading, so we can
 	 * be built-in kernel code, without hanging the boot process.
@@ -1931,9 +1940,12 @@ int rproc_add(struct rproc *rproc)
 	/* create debugfs entries */
 	rproc_create_debug_dir(rproc);
 
-	/* if rproc is marked always-on, request it to boot */
+	/*
+	 * if rproc is marked always-on, request it to boot or synchronise
+	 * with it.
+	 */
 	if (rproc->auto_boot) {
-		ret = rproc_trigger_auto_boot(rproc);
+		ret = rproc_trigger_auto_initiate(rproc);
 		if (ret < 0)
 			return ret;
 	}
-- 
2.20.1

[PATCH v2 12/17] remoteproc: Rename function rproc_fw_boot()

[Mathieu Poirier]

Renaming function rproc_fw_boot() in order to better reflect the work
that is done when supporting scenarios where the remoteproc core is
synchronising with an MCU.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index d57b47b0d6be..488723fcb142 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1363,7 +1363,8 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
 /*
  * take a firmware and boot a remote processor with it.
  */
-static int rproc_fw_boot(struct rproc *rproc, const struct firmware *fw)
+static int rproc_actuate_platform(struct rproc *rproc,
+				  const struct firmware *fw)
 {
 	struct device *dev = &rproc->dev;
 	const char *name = rproc->firmware;
@@ -1373,7 +1374,9 @@ static int rproc_fw_boot(struct rproc *rproc, const struct firmware *fw)
 	if (ret)
 		return ret;
 
-	dev_info(dev, "Booting fw image %s, size %zd\n", name, fw->size);
+	if (!rproc_sync_with_mcu(rproc))
+		dev_info(dev, "Booting fw image %s, size %zd\n",
+			 name, fw->size);
 
 	/*
 	 * if enabling an IOMMU isn't relevant for this rproc, this is
@@ -1756,7 +1759,7 @@ static int rproc_actuate(struct rproc *rproc,
 		 firmware_p ? "powering up" : "syncing with",
 		 rproc->name);
 
-	ret = rproc_fw_boot(rproc, firmware_p);
+	ret = rproc_actuate_platform(rproc, firmware_p);
 	if (ret)
 		atomic_dec(&rproc->power);
 
-- 
2.20.1

[PATCH v2 13/17] remoteproc: Introducting new functions to start and stop an MCU

[Mathieu Poirier]

Add new functions to replace direct calling of rproc->ops->start() and
rproc->ops->stop().  That way different behaviour can be played out
when booting an MCU or synchronising with it.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c     |  6 +++---
 drivers/remoteproc/remoteproc_internal.h | 12 ++++++++++++
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 488723fcb142..d3c4d7e6ca25 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1330,7 +1330,7 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
 	}
 
 	/* power up the remote processor */
-	ret = rproc->ops->start(rproc);
+	ret = rproc_start_hw(rproc);
 	if (ret) {
 		dev_err(dev, "can't start rproc %s: %d\n", rproc->name, ret);
 		goto unprepare_subdevices;
@@ -1351,7 +1351,7 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
 	return 0;
 
 stop_rproc:
-	rproc->ops->stop(rproc);
+	rproc_stop_hw(rproc);
 unprepare_subdevices:
 	rproc_unprepare_subdevices(rproc);
 reset_table_ptr:
@@ -1485,7 +1485,7 @@ static int rproc_stop(struct rproc *rproc, bool crashed)
 	rproc->table_ptr = rproc->cached_table;
 
 	/* power off the remote processor */
-	ret = rproc->ops->stop(rproc);
+	ret = rproc_stop_hw(rproc);
 	if (ret) {
 		dev_err(dev, "can't stop rproc: %d\n", ret);
 		return ret;
diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
index 5f711ceb97ba..7ca23d46dfd4 100644
--- a/drivers/remoteproc/remoteproc_internal.h
+++ b/drivers/remoteproc/remoteproc_internal.h
@@ -160,4 +160,16 @@ struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
 	return NULL;
 }
 
+static inline int rproc_start_hw(struct rproc *rproc)
+{
+	RPROC_OPS_HELPER(start, rproc);
+	return -EINVAL;
+}
+
+static inline int rproc_stop_hw(struct rproc *rproc)
+{
+	RPROC_OPS_HELPER(stop, rproc);
+	return -EINVAL;
+}
+
 #endif /* REMOTEPROC_INTERNAL_H */
-- 
2.20.1

[PATCH v2 14/17] remoteproc: Refactor function rproc_trigger_recovery()

[Mathieu Poirier]

Refactor function rproc_trigger_recovery() in order to avoid
reloading the fw image when synchronising with an MCU rather than
booting it.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index d3c4d7e6ca25..dbb0a8467205 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1661,7 +1661,7 @@ static void rproc_coredump(struct rproc *rproc)
  */
 int rproc_trigger_recovery(struct rproc *rproc)
 {
-	const struct firmware *firmware_p;
+	const struct firmware *firmware_p = NULL;
 	struct device *dev = &rproc->dev;
 	int ret;
 
@@ -1678,14 +1678,16 @@ int rproc_trigger_recovery(struct rproc *rproc)
 	/* generate coredump */
 	rproc_coredump(rproc);
 
-	/* load firmware */
-	ret = request_firmware(&firmware_p, rproc->firmware, dev);
-	if (ret < 0) {
-		dev_err(dev, "request_firmware failed: %d\n", ret);
-		goto unlock_mutex;
+	/* load firmware if need be */
+	if (!rproc_sync_with_mcu(rproc)) {
+		ret = request_firmware(&firmware_p, rproc->firmware, dev);
+		if (ret < 0) {
+			dev_err(dev, "request_firmware failed: %d\n", ret);
+			goto unlock_mutex;
+		}
 	}
 
-	/* boot the remote processor up again */
+	/* boot up or synchronise with the remote processor again */
 	ret = rproc_start(rproc, firmware_p);
 
 	release_firmware(firmware_p);
-- 
2.20.1

[PATCH v2 15/17] remoteproc: Correctly deal with MCU synchronisation when changing FW image

[Mathieu Poirier]

This patch prevents the firmware image from being displayed or changed when
the remoteproc core is synchronising with an MCU. This is needed since
there is no guarantee about the nature of the firmware image that is loaded
by the external entity.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_sysfs.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/drivers/remoteproc/remoteproc_sysfs.c b/drivers/remoteproc/remoteproc_sysfs.c
index 7f8536b73295..4956577ad4b4 100644
--- a/drivers/remoteproc/remoteproc_sysfs.c
+++ b/drivers/remoteproc/remoteproc_sysfs.c
@@ -13,9 +13,20 @@
 static ssize_t firmware_show(struct device *dev, struct device_attribute *attr,
 			  char *buf)
 {
+	ssize_t ret;
 	struct rproc *rproc = to_rproc(dev);
 
-	return sprintf(buf, "%s\n", rproc->firmware);
+	/*
+	 * In most instances there is no guarantee about the firmware
+	 * that was loaded by the external entity.  As such simply don't
+	 * print anything.
+	 */
+	if (rproc_sync_with_mcu(rproc))
+		ret = sprintf(buf, "\n");
+	else
+		ret = sprintf(buf, "%s\n", rproc->firmware);
+
+	return ret;
 }
 
 /* Change firmware name via sysfs */
@@ -33,6 +44,18 @@ static ssize_t firmware_store(struct device *dev,
 		return -EINVAL;
 	}
 
+	/*
+	 * There is no point in trying to change the firmware if the MCU
+	 * is currently running or if loading of the image is done by
+	 * another entity.
+	 */
+	if (rproc_sync_with_mcu(rproc)) {
+		dev_err(dev,
+			"can't change firmware while synchronising with MCU\n");
+		err = -EBUSY;
+		goto out;
+	}
+
 	if (rproc->state != RPROC_OFFLINE) {
 		dev_err(dev, "can't change firmware while running\n");
 		err = -EBUSY;
-- 
2.20.1

[PATCH v2 16/17] remoteproc: Correctly deal with MCU synchronisation when changing state

[Mathieu Poirier]

This patch deals with state changes when synchronising with an MCU. More
specifically it prevents the MCU from being started if it already has been
started by another entity.  Similarly it prevents the AP from stopping the
MCU if it hasn't been given the capability by platform firmware.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_sysfs.c | 32 ++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/drivers/remoteproc/remoteproc_sysfs.c b/drivers/remoteproc/remoteproc_sysfs.c
index 4956577ad4b4..741a3c152b82 100644
--- a/drivers/remoteproc/remoteproc_sysfs.c
+++ b/drivers/remoteproc/remoteproc_sysfs.c
@@ -108,6 +108,29 @@ static ssize_t state_show(struct device *dev, struct device_attribute *attr,
 	return sprintf(buf, "%s\n", rproc_state_string[state]);
 }
 
+static int rproc_can_shutdown(struct rproc *rproc)
+{
+	/* The MCU is not running, obviously an invalid operation. */
+	if (rproc->state != RPROC_RUNNING)
+		return false;
+
+	/*
+	 * The MCU is not running (see above) and the remoteproc core is the
+	 * lifecycle manager, no problem calling for a shutdown.
+	 */
+	if (!rproc_sync_with_mcu(rproc))
+		return true;
+
+	/*
+	 * The MCU has been loaded by another entity (see above) and the
+	 * platform code has _not_ given us the capability of stopping it.
+	 */
+	if (!rproc->sync_ops->stop)
+		return false;
+
+	return true;
+}
+
 /* Change remote processor state via sysfs */
 static ssize_t state_store(struct device *dev,
 			      struct device_attribute *attr,
@@ -120,11 +143,18 @@ static ssize_t state_store(struct device *dev,
 		if (rproc->state == RPROC_RUNNING)
 			return -EBUSY;
 
+		/*
+		 * In synchronisation mode, booting the MCU is the
+		 * responsibility of an external entity.
+		 */
+		if (rproc_sync_with_mcu(rproc))
+			return -EINVAL;
+
 		ret = rproc_boot(rproc);
 		if (ret)
 			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
 	} else if (sysfs_streq(buf, "stop")) {
-		if (rproc->state != RPROC_RUNNING)
+		if (!rproc_can_shutdown(rproc))
 			return -EINVAL;
 
 		rproc_shutdown(rproc);
-- 
2.20.1

[PATCH v2 17/17] remoteproc: Make MCU synchronisation state changes on stop and crashed

[Mathieu Poirier]

Call on the MCU synchronisation state machine to determine the
synchronisation status to enact when the MCU is either stop from sysfs
or has crashed.

Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
---
 drivers/remoteproc/remoteproc_core.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index dbb0a8467205..0608593cccc6 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -1722,6 +1722,14 @@ static void rproc_crash_handler_work(struct work_struct *work)
 	dev_err(dev, "handling crash #%u in %s\n", ++rproc->crash_cnt,
 		rproc->name);
 
+	/*
+	 * The MCU has crashed - tell the core what operation to
+	 * use from hereon, i.e whether an external entity will
+	 * reboot the MCU or it is now the remoteproc core's
+	 * responsability.
+	 */
+	rproc_set_mcu_sync_state(rproc, RPROC_SYNC_STATE_CRASHED);
+
 	mutex_unlock(&rproc->lock);
 
 	if (!rproc->recovery_disabled)
@@ -1856,6 +1864,13 @@ void rproc_shutdown(struct rproc *rproc)
 	kfree(rproc->cached_table);
 	rproc->cached_table = NULL;
 	rproc->table_ptr = NULL;
+
+	/*
+	 * The MCU has been switched off - tell the core what operation to
+	 * use from hereon, i.e whether an external entity will reboot the
+	 * MCU or it is now the remoteproc core's responsability.
+	 */
+	rproc_set_mcu_sync_state(rproc, RPROC_SYNC_STATE_SHUTDOWN);
 out:
 	mutex_unlock(&rproc->lock);
 }
-- 
2.20.1

RE: [PATCH v2 03/17] remoteproc: Split firmware name allocation from rproc_alloc()

[Loic PALLARDY]

Hi Mathieu,

> -----Original Message-----
> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> Sent: mardi 24 mars 2020 22:46
> To: bjorn.andersson@linaro.org
> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> Subject: [PATCH v2 03/17] remoteproc: Split firmware name allocation from
> rproc_alloc()
> 
> Make the firmware name allocation a function on its own in order to
> introduce more flexibility to function rproc_alloc().
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 62 +++++++++++++++++-----------
>  1 file changed, 39 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c
> b/drivers/remoteproc/remoteproc_core.c
> index 097f33e4f1f3..c0871f69929b 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -1962,6 +1962,36 @@ static const struct device_type rproc_type = {
>  	.release	= rproc_type_release,
>  };
> 
> +static int rproc_alloc_firmware(struct rproc *rproc,
> +				const char *name, const char *firmware)
> +{
> +	char *p, *template = "rproc-%s-fw";
> +	int name_len;
> +
> +	if (!rproc || !name)
> +		return -EINVAL;
> +
> +	if (!firmware) {
> +		/*
> +		 * If the caller didn't pass in a firmware name then
> +		 * construct a default name.
> +		 */
> +		name_len = strlen(name) + strlen(template) - 2 + 1;
> +		p = kmalloc(name_len, GFP_KERNEL);
> +		if (!p)
> +			return -ENOMEM;
> +		snprintf(p, name_len, template, name);
> +	} else {
> +		p = kstrdup(firmware, GFP_KERNEL);
> +		if (!p)
> +			return -ENOMEM;
> +	}
> +
> +	rproc->firmware = p;
> +
> +	return 0;
> +}
> +
>  /**
>   * rproc_alloc() - allocate a remote processor handle
>   * @dev: the underlying device
> @@ -1990,42 +2020,24 @@ struct rproc *rproc_alloc(struct device *dev,
> const char *name,
>  			  const char *firmware, int len)
>  {
>  	struct rproc *rproc;
> -	char *p, *template = "rproc-%s-fw";
> -	int name_len;
> 
>  	if (!dev || !name || !ops)
>  		return NULL;
> 
> -	if (!firmware) {
> -		/*
> -		 * If the caller didn't pass in a firmware name then
> -		 * construct a default name.
> -		 */
> -		name_len = strlen(name) + strlen(template) - 2 + 1;
> -		p = kmalloc(name_len, GFP_KERNEL);
> -		if (!p)
> -			return NULL;
> -		snprintf(p, name_len, template, name);
> -	} else {
> -		p = kstrdup(firmware, GFP_KERNEL);
> -		if (!p)
> -			return NULL;
> -	}
> -
>  	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
> -	if (!rproc) {
> -		kfree(p);
> +	if (!rproc)
>  		return NULL;
> -	}
> +
> +	if (rproc_alloc_firmware(rproc, name, firmware))
> +		goto free_rproc;
> 
>  	rproc->ops = kmemdup(ops, sizeof(*ops), GFP_KERNEL);
>  	if (!rproc->ops) {
> -		kfree(p);
> +		kfree(rproc->firmware);
>  		kfree(rproc);
Small remark only for patch coherency, as it is modified in next patches.
Use free_rproc label which is introduced just below here for error management.

Regards,
Loic
>  		return NULL;
>  	}
> 
> -	rproc->firmware = p;
>  	rproc->name = name;
>  	rproc->priv = &rproc[1];
>  	rproc->auto_boot = true;
> @@ -2073,6 +2085,10 @@ struct rproc *rproc_alloc(struct device *dev, const
> char *name,
>  	rproc->state = RPROC_OFFLINE;
> 
>  	return rproc;
> +
> +free_rproc:
> +	kfree(rproc);
> +	return NULL;
>  }
>  EXPORT_SYMBOL(rproc_alloc);
> 
> --
> 2.20.1

RE: [PATCH v2 06/17] remoteproc: Introduce function rproc_alloc_internals()

[Loic PALLARDY]

Hi Mathieu,

> -----Original Message-----
> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> Sent: mardi 24 mars 2020 22:46
> To: bjorn.andersson@linaro.org
> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> Subject: [PATCH v2 06/17] remoteproc: Introduce function
> rproc_alloc_internals()
> 
> In preparation to allocate the synchronisation operation and state
> machine, spin off a new function in order to keep rproc_alloc() as
> clean as possible.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 26 ++++++++++++++++++++++---
> -
>  1 file changed, 22 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c
> b/drivers/remoteproc/remoteproc_core.c
> index ee277bc5556c..9da245734db6 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -2018,6 +2018,26 @@ static int rproc_alloc_ops(struct rproc *rproc,
> const struct rproc_ops *ops)
>  	return 0;
>  }
> 
> +static int rproc_alloc_internals(struct rproc *rproc, const char *name,
> +				 const struct rproc_ops *boot_ops,
> +				 const char *firmware, int len)

len argument is not used in the patch nor in the following, maybe removed from my pov.

Regards,
Loic
> +{
> +	int ret;
> +
> +	/* We have a boot_ops so allocate firmware name and operations */
> +	if (boot_ops) {
> +		ret = rproc_alloc_firmware(rproc, name, firmware);
> +		if (ret)
> +			return ret;
> +
> +		ret = rproc_alloc_ops(rproc, boot_ops);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	return 0;
> +}
> +
>  /**
>   * rproc_alloc() - allocate a remote processor handle
>   * @dev: the underlying device
> @@ -2064,10 +2084,8 @@ struct rproc *rproc_alloc(struct device *dev, const
> char *name,
>  	rproc->dev.class = &rproc_class;
>  	rproc->dev.driver_data = rproc;
> 
> -	if (rproc_alloc_firmware(rproc, name, firmware))
> -		goto out;
> -
> -	if (rproc_alloc_ops(rproc, ops))
> +	if (rproc_alloc_internals(rproc, name, ops,
> +				  firmware, len))
>  		goto out;
> 
>  	/* Assign a unique device index and name */
> --
> 2.20.1

RE: [PATCH v2 07/17] remoteproc: Introduce function rproc_alloc_state_machine()

[Loic PALLARDY]

> -----Original Message-----
> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> Sent: mardi 24 mars 2020 22:46
> To: bjorn.andersson@linaro.org
> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> Subject: [PATCH v2 07/17] remoteproc: Introduce function
> rproc_alloc_state_machine()
> 
> Introducing new function rproc_alloc_state_machine() to allocate
> the MCU synchronisation operations and position it as the central
> remoteproc core allocation function.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 84
> +++++++++++++++++++++++++---
>  include/linux/remoteproc.h           |  5 ++
>  2 files changed, 81 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c
> b/drivers/remoteproc/remoteproc_core.c
> index 9da245734db6..02dbb826aa29 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -1954,6 +1954,7 @@ static void rproc_type_release(struct device *dev)
> 
>  	kfree(rproc->firmware);
>  	kfree(rproc->ops);
> +	kfree(rproc->sync_ops);
>  	kfree(rproc);
>  }
> 
> @@ -2018,12 +2019,34 @@ static int rproc_alloc_ops(struct rproc *rproc,
> const struct rproc_ops *ops)
>  	return 0;
>  }
> 
> +static int rproc_alloc_sync_ops(struct rproc *rproc,
> +				const struct rproc_ops *sync_ops)
> +{
> +	/*
> +	 * Given the unlimited amount of possibilities when
> +	 * synchronising with an MCU, no constraints are imposed
> +	 * on sync_ops.
> +	 */
> +	rproc->sync_ops = kmemdup(sync_ops,
> +				  sizeof(*sync_ops), GFP_KERNEL);
> +	if (!rproc->sync_ops)
> +		return -ENOMEM;
Should we check a minimal set of functions in sync_ops to be required?
Or we should consider all pointers at NULL is ok ?

> +
> +	return 0;
> +}
> +
>  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
>  				 const struct rproc_ops *boot_ops,
> +				 const struct rproc_ops *sync_ops,
> +				 struct rproc_sync_states *sync_states,
sync_states not used in this patch, should be introduced in patch 8

Regards,
Loic

>  				 const char *firmware, int len)
>  {
>  	int ret;
> 
> +	/* We need at least a boot or a sync ops. */
> +	if (!boot_ops && !sync_ops)
> +		return -EINVAL;
> +
>  	/* We have a boot_ops so allocate firmware name and operations */
>  	if (boot_ops) {
>  		ret = rproc_alloc_firmware(rproc, name, firmware);
> @@ -2035,14 +2058,23 @@ static int rproc_alloc_internals(struct rproc
> *rproc, const char *name,
>  			return ret;
>  	}
> 
> +	/* Allocate a sync_ops if need be */
> +	if (sync_ops) {
> +		ret = rproc_alloc_sync_ops(rproc, sync_ops);
> +		if (ret)
> +			return ret;
> +	}
> +
>  	return 0;
>  }
> 
>  /**
> - * rproc_alloc() - allocate a remote processor handle
> + * rproc_alloc_state_machine() - allocate a remote processor handle
>   * @dev: the underlying device
>   * @name: name of this remote processor
>   * @ops: platform-specific handlers (mainly start/stop)
> + * @sync_ops: platform-specific handlers for synchronising with MCU
> + * @sync_states: states in which @ops and @sync_ops are to be used
>   * @firmware: name of firmware file to load, can be NULL
>   * @len: length of private data needed by the rproc driver (in bytes)
>   *
> @@ -2061,13 +2093,15 @@ static int rproc_alloc_internals(struct rproc
> *rproc, const char *name,
>   * Note: _never_ directly deallocate @rproc, even if it was not registered
>   * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
>   */
> -struct rproc *rproc_alloc(struct device *dev, const char *name,
> -			  const struct rproc_ops *ops,
> -			  const char *firmware, int len)
> +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
> *name,
> +					const struct rproc_ops *ops,
> +					const struct rproc_ops *sync_ops,
> +					struct rproc_sync_states
> *sync_states,
> +					const char *firmware, int len)
>  {
>  	struct rproc *rproc;
> 
> -	if (!dev || !name || !ops)
> +	if (!dev || !name)
>  		return NULL;
> 
>  	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
> @@ -2084,8 +2118,8 @@ struct rproc *rproc_alloc(struct device *dev, const
> char *name,
>  	rproc->dev.class = &rproc_class;
>  	rproc->dev.driver_data = rproc;
> 
> -	if (rproc_alloc_internals(rproc, name, ops,
> -				  firmware, len))
> +	if (rproc_alloc_internals(rproc, name, ops, sync_ops,
> +				  sync_states, firmware, len))
>  		goto out;
> 
>  	/* Assign a unique device index and name */
> @@ -2119,7 +2153,41 @@ struct rproc *rproc_alloc(struct device *dev, const
> char *name,
>  	put_device(&rproc->dev);
>  	return NULL;
>  }
> -EXPORT_SYMBOL(rproc_alloc);
> +EXPORT_SYMBOL(rproc_alloc_state_machine);
> +
> +/**
> + * rproc_alloc() - allocate a remote processor handle
> + * @dev: the underlying device
> + * @name: name of this remote processor
> + * @ops: platform-specific handlers (mainly start/stop)
> + * @firmware: name of firmware file to load, can be NULL
> + * @len: length of private data needed by the rproc driver (in bytes)
> + *
> + * Allocates a new remote processor handle, but does not register
> + * it yet. if @firmware is NULL, a default name is used.
> + *
> + * This function should be used by rproc implementations during
> initialization
> + * of the remote processor.
> + *
> + * After creating an rproc handle using this function, and when ready,
> + * implementations should then call rproc_add() to complete
> + * the registration of the remote processor.
> + *
> + * On success the new rproc is returned, and on failure, NULL.
> + *
> + * Note: _never_ directly deallocate @rproc, even if it was not registered
> + * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
> + */
> +struct rproc *rproc_alloc(struct device *dev, const char *name,
> +			  const struct rproc_ops *ops,
> +			  const char *firmware, int len)
> +{
> +	if (!name && !firmware)
> +		return NULL;
> +
> +	return rproc_alloc_state_machine(dev, name, ops, NULL, NULL,
> +					 firmware, len);
> +}
> 
>  /**
>   * rproc_free() - unroll rproc_alloc()
> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
> index d115e47d702d..d1214487daac 100644
> --- a/include/linux/remoteproc.h
> +++ b/include/linux/remoteproc.h
> @@ -611,6 +611,11 @@ struct rproc *rproc_get_by_child(struct device
> *dev);
>  struct rproc *rproc_alloc(struct device *dev, const char *name,
>  			  const struct rproc_ops *ops,
>  			  const char *firmware, int len);
> +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
> *name,
> +					const struct rproc_ops *ops,
> +					const struct rproc_ops *sync_ops,
> +					struct rproc_sync_states
> *sync_states,
> +					const char *firmware, int len);
>  void rproc_put(struct rproc *rproc);
>  int rproc_add(struct rproc *rproc);
>  int rproc_del(struct rproc *rproc);
> --
> 2.20.1

RE: [PATCH v2 08/17] remoteproc: Allocate synchronisation state machine

[Loic PALLARDY]

> -----Original Message-----
> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> Sent: mardi 24 mars 2020 22:46
> To: bjorn.andersson@linaro.org
> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> Subject: [PATCH v2 08/17] remoteproc: Allocate synchronisation state
> machine
> 
> This patch allocates a synchronisation state machine, either provided or
> not by users, in order to enact the proper behavior requested by the
> platform or specific scenarios.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 59
> +++++++++++++++++++++++++++-
>  1 file changed, 58 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c
> b/drivers/remoteproc/remoteproc_core.c
> index 02dbb826aa29..1578a9c70422 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -1955,6 +1955,7 @@ static void rproc_type_release(struct device *dev)
>  	kfree(rproc->firmware);
>  	kfree(rproc->ops);
>  	kfree(rproc->sync_ops);
> +	kfree(rproc->sync_states);
>  	kfree(rproc);
>  }
> 
> @@ -2035,6 +2036,59 @@ static int rproc_alloc_sync_ops(struct rproc *rproc,
>  	return 0;
>  }
> 
> +static int rproc_alloc_sync_states(struct rproc *rproc,
> +				   const struct rproc_ops *boot_ops,
> +				   const struct rproc_ops *sync_ops,
> +				   struct rproc_sync_states *sync_states)
> +{
> +	struct rproc_sync_states *st;
> +
> +	/* At least one set of operation is needed */
> +	if (!boot_ops && !sync_ops)
> +		return -EINVAL;
> +
> +	/* We have a synchronisation state machine, no need to build one */
> +	if (sync_states) {
> +		st = kmemdup(sync_states, sizeof(*st), GFP_KERNEL);
> +		if (!st)
> +			return -ENOMEM;
> +

I think a check between sync_states and boot_ops/sync_ops may be needed here
even if it is platform driver responsibility to provide coherent configuration
As soon as one of the sync_states is set at true, sync_ops must be provided
As soon as one of the sync_states is set at false, boot_ops must be provided

Regards,
Loic

> +		/* Nothing else to do */
> +		goto out;
> +	}
> +
> +	/* Allocate synchronisation state machine */
> +	st = kzalloc(sizeof(*st), GFP_KERNEL);
> +	if (!st)
> +		return -ENOMEM;
> +
> +	/*
> +	 * We have a boot_ops and no sync_ops - build a state machine that
> +	 * does _not_ synchronise with an MCU.
> +	 */
> +	if (boot_ops && !sync_ops) {
> +		st->on_init = st->after_stop = st->after_crash = false;
> +		goto out;
> +	}
> +
> +	/*
> +	 * We have a sync_ops and an no boot_ops - build a state machine
> that
> +	 * _only_ synchronises with an MCU.
> +	 */
> +	if (sync_ops && !boot_ops) {
> +		st->on_init = st->after_stop = st->after_crash = true;
> +		goto out;
> +	}
> +
> +out:
> +	rproc->sync_with_mcu = st->on_init;
> +	/* And the synchronisation state machine to use */
> +	rproc->sync_states = st;
> +	/* Tell the core what to do when initialising */
> +	rproc_set_mcu_sync_state(rproc, RPROC_SYNC_STATE_INIT);
> +	return 0;
> +}
> +
>  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
>  				 const struct rproc_ops *boot_ops,
>  				 const struct rproc_ops *sync_ops,
> @@ -2065,7 +2119,10 @@ static int rproc_alloc_internals(struct rproc *rproc,
> const char *name,
>  			return ret;
>  	}
> 
> -	return 0;
> +	/* Finally allocate the synchronisation state machine */
> +	ret = rproc_alloc_sync_states(rproc, boot_ops, sync_ops,
> sync_states);
> +
> +	return ret;
>  }
> 
>  /**
> --
> 2.20.1

RE: [PATCH v2 15/17] remoteproc: Correctly deal with MCU synchronisation when changing FW image

[Loic PALLARDY]

> -----Original Message-----
> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> Sent: mardi 24 mars 2020 22:46
> To: bjorn.andersson@linaro.org
> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> Subject: [PATCH v2 15/17] remoteproc: Correctly deal with MCU
> synchronisation when changing FW image
> 
> This patch prevents the firmware image from being displayed or changed
> when
> the remoteproc core is synchronising with an MCU. This is needed since
> there is no guarantee about the nature of the firmware image that is loaded
> by the external entity.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_sysfs.c | 25
> ++++++++++++++++++++++++-
>  1 file changed, 24 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> b/drivers/remoteproc/remoteproc_sysfs.c
> index 7f8536b73295..4956577ad4b4 100644
> --- a/drivers/remoteproc/remoteproc_sysfs.c
> +++ b/drivers/remoteproc/remoteproc_sysfs.c
> @@ -13,9 +13,20 @@
>  static ssize_t firmware_show(struct device *dev, struct device_attribute
> *attr,
>  			  char *buf)
>  {
> +	ssize_t ret;
>  	struct rproc *rproc = to_rproc(dev);
> 
> -	return sprintf(buf, "%s\n", rproc->firmware);
> +	/*
> +	 * In most instances there is no guarantee about the firmware
> +	 * that was loaded by the external entity.  As such simply don't
> +	 * print anything.
> +	 */
> +	if (rproc_sync_with_mcu(rproc))
> +		ret = sprintf(buf, "\n");
Is it enough to provide empty name, or should we add a message to indicate that's name is unkown/undefined ?

Regards,
Loic
> +	else
> +		ret = sprintf(buf, "%s\n", rproc->firmware);
> +
> +	return ret;
>  }
> 
>  /* Change firmware name via sysfs */
> @@ -33,6 +44,18 @@ static ssize_t firmware_store(struct device *dev,
>  		return -EINVAL;
>  	}
> 
> +	/*
> +	 * There is no point in trying to change the firmware if the MCU
> +	 * is currently running or if loading of the image is done by
> +	 * another entity.
> +	 */
> +	if (rproc_sync_with_mcu(rproc)) {
> +		dev_err(dev,
> +			"can't change firmware while synchronising with
> MCU\n");
> +		err = -EBUSY;
> +		goto out;
> +	}
> +
>  	if (rproc->state != RPROC_OFFLINE) {
>  		dev_err(dev, "can't change firmware while running\n");
>  		err = -EBUSY;
> --
> 2.20.1

RE: [PATCH v2 16/17] remoteproc: Correctly deal with MCU synchronisation when changing state

[Loic PALLARDY]

> -----Original Message-----
> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> Sent: mardi 24 mars 2020 22:46
> To: bjorn.andersson@linaro.org
> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> Subject: [PATCH v2 16/17] remoteproc: Correctly deal with MCU
> synchronisation when changing state
> 
> This patch deals with state changes when synchronising with an MCU. More
> specifically it prevents the MCU from being started if it already has been
> started by another entity.  Similarly it prevents the AP from stopping the
> MCU if it hasn't been given the capability by platform firmware.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_sysfs.c | 32
> ++++++++++++++++++++++++++-
>  1 file changed, 31 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> b/drivers/remoteproc/remoteproc_sysfs.c
> index 4956577ad4b4..741a3c152b82 100644
> --- a/drivers/remoteproc/remoteproc_sysfs.c
> +++ b/drivers/remoteproc/remoteproc_sysfs.c
> @@ -108,6 +108,29 @@ static ssize_t state_show(struct device *dev, struct
> device_attribute *attr,
>  	return sprintf(buf, "%s\n", rproc_state_string[state]);
>  }
> 
> +static int rproc_can_shutdown(struct rproc *rproc)
> +{
> +	/* The MCU is not running, obviously an invalid operation. */
> +	if (rproc->state != RPROC_RUNNING)
> +		return false;
> +
> +	/*
> +	 * The MCU is not running (see above) and the remoteproc core is
> the
> +	 * lifecycle manager, no problem calling for a shutdown.
> +	 */
> +	if (!rproc_sync_with_mcu(rproc))
> +		return true;
> +
> +	/*
> +	 * The MCU has been loaded by another entity (see above) and the
> +	 * platform code has _not_ given us the capability of stopping it.
> +	 */
> +	if (!rproc->sync_ops->stop)
> +		return false;

Test could be simplified
if (rproc_sync_with_mcu(rproc)) && !rproc->sync_ops->stop)
	return false;

> +
> +	return true;
> +}
> +
>  /* Change remote processor state via sysfs */
>  static ssize_t state_store(struct device *dev,
>  			      struct device_attribute *attr,
> @@ -120,11 +143,18 @@ static ssize_t state_store(struct device *dev,
>  		if (rproc->state == RPROC_RUNNING)
>  			return -EBUSY;
> 
> +		/*
> +		 * In synchronisation mode, booting the MCU is the
> +		 * responsibility of an external entity.
> +		 */
> +		if (rproc_sync_with_mcu(rproc))
> +			return -EINVAL;
> +
I don't understand this restriction, simply because it is preventing to resynchronize with a
coprocessor after a "stop".
In the following configuration which can be configuration for coprocessor with romed/flashed
firmware (no reload needed):
on_init = true
after_stop = true
after_crash = true
Once you stop it via sysfs interface, you can't anymore restart/resync to it.

I think it will be better to modify rproc_boot() to take into account rproc_sync_with_mcu()
as below:

int rproc_boot(struct rproc *rproc)
 {
-	const struct firmware *firmware_p;
+	const struct firmware *firmware_p = NULL;
 	struct device *dev = &rproc->dev;
 	int ret;
 
 	if (!rproc) {
 		pr_err("invalid rproc handle\n");
 		return -EINVAL;
 	}
 
 	/* load firmware */
-	ret = request_firmware(&firmware_p, rproc->firmware, dev);
-	if (ret < 0) {
-		dev_err(dev, "request_firmware failed: %d\n", ret);
-		return ret;
+	if (!rproc_sync_with_mcu(rproc)) {
+		ret = request_firmware(&firmware_p, rproc->firmware, dev);
+		if (ret < 0) {
+			dev_err(dev, "request_firmware failed: %d\n", ret);
+			return ret;
+		}
 	}
 
 	ret = rproc_actuate(rproc, firmware_p);
 
-	release_firmware(firmware_p);
+	if (firmware_p)
+		release_firmware(firmware_p);
 
 	return ret;
 }
 
Thanks to these modifications, I'm able to resync after a stop with coprocessor without reloading firmware.

>  		ret = rproc_boot(rproc);
>  		if (ret)
>  			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
>  	} else if (sysfs_streq(buf, "stop")) {
> -		if (rproc->state != RPROC_RUNNING)
> +		if (!rproc_can_shutdown(rproc))
>  			return -EINVAL;
> 
>  		rproc_shutdown(rproc);
As rproc shutdown is also accessible as kernel API, I propose to move
rproc_can_shutdown() check inside rproc_shutdown() and to test
returned error

Regards,
Loic
> --
> 2.20.1

RE: [PATCH v2 00/17] remoteproc: Add support for synchronisation with MCU

[Loic PALLARDY]

Hi Mathieu,

> -----Original Message-----
> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> Sent: mardi 24 mars 2020 22:46
> To: bjorn.andersson@linaro.org
> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> Subject: [PATCH v2 00/17] remoteproc: Add support for synchronisation with
> MCU
> 
> This is the second revision of this set that tries to address the
> problem of synchronising with an MCU with as much flexibility as
> possible.
> 
> New in this revision is a fix for a couple of bugs I found while
> testing things.  First with the helper macro in patch 09 and the
> suppression of a boot message when synchronising with an MCU
> in patch 12.  I have completely removed what used to be patch 18,
> the example on how to use the new API.  This will be the subject
> of an upcoming patchset.
> 
> Tested on ST's mp157c platform.  Applies on v5.6-rc7 to keep things
> simple.

Thanks Mathieu for the 2 series. I tested on my STM32MP157-DK2 the different
synchronization use cases (on_init, after_stop, after_crash), mixing the values and
I succeed to start/stop/restart M4 coprocessor with or without reloading firmware
according to sync values. (I only applied the correction I proposed in patch 16 review
to allow to resync with a preloaded or an already running coprocessor.

Regards,
Loic
> 
> Comments would be much appreciated.
> 
> Thanks,
> Mathieu
> 
> Mathieu Poirier (17):
>   remoteproc: Add new operation and state machine for MCU
>     synchronisation
>   remoteproc: Introduce function rproc_set_mcu_sync_state()
>   remoteproc: Split firmware name allocation from rproc_alloc()
>   remoteproc: Split rproc_ops allocation from rproc_alloc()
>   remoteproc: Get rid of tedious error path
>   remoteproc: Introduce function rproc_alloc_internals()
>   remoteproc: Introduce function rproc_alloc_state_machine()
>   remoteproc: Allocate synchronisation state machine
>   remoteproc: Call the right core function based on synchronisation
>     state
>   remoteproc: Decouple firmware load and remoteproc booting
>   remoteproc: Repurpose function rproc_trigger_auto_boot()
>   remoteproc: Rename function rproc_fw_boot()
>   remoteproc: Introducting new functions to start and stop an MCU
>   remoteproc: Refactor function rproc_trigger_recovery()
>   remoteproc: Correctly deal with MCU synchronisation when changing FW
>     image
>   remoteproc: Correctly deal with MCU synchronisation when changing
>     state
>   remoteproc: Make MCU synchronisation state changes on stop and crashed
> 
>  drivers/remoteproc/remoteproc_core.c     | 387 ++++++++++++++++++-----
>  drivers/remoteproc/remoteproc_debugfs.c  |  31 ++
>  drivers/remoteproc/remoteproc_internal.h |  91 ++++--
>  drivers/remoteproc/remoteproc_sysfs.c    |  57 +++-
>  include/linux/remoteproc.h               |  28 +-
>  5 files changed, 487 insertions(+), 107 deletions(-)
> 
> --
> 2.20.1

Re: [PATCH v2 03/17] remoteproc: Split firmware name allocation from rproc_alloc()

[Suman Anna]

Hi Mathieu,

On 3/27/20 6:05 AM, Loic PALLARDY wrote:
> Hi Mathieu,
> 
>> -----Original Message-----
>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
>> Sent: mardi 24 mars 2020 22:46
>> To: bjorn.andersson@linaro.org
>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
>> Subject: [PATCH v2 03/17] remoteproc: Split firmware name allocation from
>> rproc_alloc()
>>
>> Make the firmware name allocation a function on its own in order to
>> introduce more flexibility to function rproc_alloc().

I see patches 3 through 5 are generic cleanups, can you post them
separately from this series? Bjorn has commented about using the
put_device() to free the code on one of the remoteproc core patches [1]
in my R5 patch series, and I can do my patch on top of yours. I plan to
split out those 2 core patches for my next version, and can do them on
top of these.

[1] https://patchwork.kernel.org/patch/11456385/#23248321

>>
>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>> ---
>>  drivers/remoteproc/remoteproc_core.c | 62 +++++++++++++++++-----------
>>  1 file changed, 39 insertions(+), 23 deletions(-)
>>
>> diff --git a/drivers/remoteproc/remoteproc_core.c
>> b/drivers/remoteproc/remoteproc_core.c
>> index 097f33e4f1f3..c0871f69929b 100644
>> --- a/drivers/remoteproc/remoteproc_core.c
>> +++ b/drivers/remoteproc/remoteproc_core.c
>> @@ -1962,6 +1962,36 @@ static const struct device_type rproc_type = {
>>  	.release	= rproc_type_release,
>>  };
>>
>> +static int rproc_alloc_firmware(struct rproc *rproc,
>> +				const char *name, const char *firmware)
>> +{
>> +	char *p, *template = "rproc-%s-fw";
>> +	int name_len;
>> +
>> +	if (!rproc || !name)
>> +		return -EINVAL;

This is an internal function, and these are already checked in
rproc_alloc(), so you can drop this.

>> +
>> +	if (!firmware) {
>> +		/*
>> +		 * If the caller didn't pass in a firmware name then
>> +		 * construct a default name.
>> +		 */
>> +		name_len = strlen(name) + strlen(template) - 2 + 1;
>> +		p = kmalloc(name_len, GFP_KERNEL);
>> +		if (!p)
>> +			return -ENOMEM;
>> +		snprintf(p, name_len, template, name);
>> +	} else {
>> +		p = kstrdup(firmware, GFP_KERNEL);
>> +		if (!p)
>> +			return -ENOMEM;
>> +	}
>> +
>> +	rproc->firmware = p;
>> +
>> +	return 0;
>> +}
>> +
>>  /**
>>   * rproc_alloc() - allocate a remote processor handle
>>   * @dev: the underlying device
>> @@ -1990,42 +2020,24 @@ struct rproc *rproc_alloc(struct device *dev,
>> const char *name,
>>  			  const char *firmware, int len)
>>  {
>>  	struct rproc *rproc;
>> -	char *p, *template = "rproc-%s-fw";
>> -	int name_len;
>>
>>  	if (!dev || !name || !ops)
>>  		return NULL;
>>
>> -	if (!firmware) {
>> -		/*
>> -		 * If the caller didn't pass in a firmware name then
>> -		 * construct a default name.
>> -		 */
>> -		name_len = strlen(name) + strlen(template) - 2 + 1;
>> -		p = kmalloc(name_len, GFP_KERNEL);
>> -		if (!p)
>> -			return NULL;
>> -		snprintf(p, name_len, template, name);
>> -	} else {
>> -		p = kstrdup(firmware, GFP_KERNEL);
>> -		if (!p)
>> -			return NULL;
>> -	}
>> -
>>  	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
>> -	if (!rproc) {
>> -		kfree(p);
>> +	if (!rproc)
>>  		return NULL;
>> -	}
>> +
>> +	if (rproc_alloc_firmware(rproc, name, firmware))
>> +		goto free_rproc;

Since you are already moving this after rproc_alloc() here in this
patch, you might as well fold the relevant patch 5 contents here?
Otherwise, retain the existing code as is, and do all the movement in
patch 5.

regards
Suman

>>
>>  	rproc->ops = kmemdup(ops, sizeof(*ops), GFP_KERNEL);
>>  	if (!rproc->ops) {
>> -		kfree(p);
>> +		kfree(rproc->firmware);
>>  		kfree(rproc);
> Small remark only for patch coherency, as it is modified in next patches.
> Use free_rproc label which is introduced just below here for error management.
> 
> Regards,
> Loic
>>  		return NULL;
>>  	}
>>
>> -	rproc->firmware = p;
>>  	rproc->name = name;
>>  	rproc->priv = &rproc[1];
>>  	rproc->auto_boot = true;
>> @@ -2073,6 +2085,10 @@ struct rproc *rproc_alloc(struct device *dev, const
>> char *name,
>>  	rproc->state = RPROC_OFFLINE;
>>
>>  	return rproc;
>> +
>> +free_rproc:
>> +	kfree(rproc);
>> +	return NULL;
>>  }
>>  EXPORT_SYMBOL(rproc_alloc);
>>
>> --
>> 2.20.1
> 

Re: [PATCH v2 04/17] remoteproc: Split rproc_ops allocation from rproc_alloc()

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> Make the rproc_ops allocation a function on its own in order to
> introduce more flexibility to function rproc_alloc().
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 45 ++++++++++++++++++----------
>  1 file changed, 30 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> index c0871f69929b..d22e557f27ed 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -1992,6 +1992,32 @@ static int rproc_alloc_firmware(struct rproc *rproc,
>  	return 0;
>  }
>  
> +static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops)
> +{
> +	if (!rproc)
> +		return -EINVAL;

This is an internal function, and this check is already performed in
rproc_alloc(), so you can drop this.

> +
> +	/* Nothing to do if there isn't and ops to work with */
> +	if (!ops)
> +		return 0;

Hmm, ops at the moment is mandatory, and is one of the first checks done
in rproc_alloc(). So, do drop the concept of optional ops from this
patch. It needs to be moved to a later patch that makes it optional.

> +
> +	rproc->ops = kmemdup(ops, sizeof(*ops), GFP_KERNEL);
> +	if (!rproc->ops)
> +		return -ENOMEM;
> +
> +	/* Default to ELF loader if no load function is specified */
> +	if (!rproc->ops->load) {
> +		rproc->ops->load = rproc_elf_load_segments;
> +		rproc->ops->parse_fw = rproc_elf_load_rsc_table;
> +		rproc->ops->find_loaded_rsc_table =
> +						rproc_elf_find_loaded_rsc_table;
> +		rproc->ops->sanity_check = rproc_elf_sanity_check;

I understand you have done this patch on the latest -rc, but this hunk
has been modified on rproc-next as part of Clement's 64-bit loader
support, and there is one more patch from him that's changing stuff here
again.

regards
Suman

> +		rproc->ops->get_boot_addr = rproc_elf_get_boot_addr;
> +	}
> +
> +	return 0;
> +}
> +
>  /**
>   * rproc_alloc() - allocate a remote processor handle
>   * @dev: the underlying device
> @@ -2031,12 +2057,8 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
>  	if (rproc_alloc_firmware(rproc, name, firmware))
>  		goto free_rproc;
>  
> -	rproc->ops = kmemdup(ops, sizeof(*ops), GFP_KERNEL);
> -	if (!rproc->ops) {
> -		kfree(rproc->firmware);
> -		kfree(rproc);
> -		return NULL;
> -	}
> +	if (rproc_alloc_ops(rproc, ops))
> +		goto free_firmware;
>  
>  	rproc->name = name;
>  	rproc->priv = &rproc[1];
> @@ -2060,15 +2082,6 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
>  
>  	atomic_set(&rproc->power, 0);
>  
> -	/* Default to ELF loader if no load function is specified */
> -	if (!rproc->ops->load) {
> -		rproc->ops->load = rproc_elf_load_segments;
> -		rproc->ops->parse_fw = rproc_elf_load_rsc_table;
> -		rproc->ops->find_loaded_rsc_table = rproc_elf_find_loaded_rsc_table;
> -		rproc->ops->sanity_check = rproc_elf_sanity_check;
> -		rproc->ops->get_boot_addr = rproc_elf_get_boot_addr;
> -	}
> -
>  	mutex_init(&rproc->lock);
>  
>  	idr_init(&rproc->notifyids);
> @@ -2086,6 +2099,8 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
>  
>  	return rproc;
>  
> +free_firmware:
> +	kfree(rproc->firmware);
>  free_rproc:
>  	kfree(rproc);
>  	return NULL;
> 

Re: [PATCH v2 05/17] remoteproc: Get rid of tedious error path

[Suman Anna]

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> Get rid of tedious error management by moving firmware and operation
> allocation after calling device_initialize().  That way we take advantage
> of the automatic call to rproc_type_release() to cleanup after ourselves
> when put_device() is called.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 22 +++++++++-------------
>  1 file changed, 9 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> index d22e557f27ed..ee277bc5556c 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -2054,12 +2054,6 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
>  	if (!rproc)
>  		return NULL;
>  
> -	if (rproc_alloc_firmware(rproc, name, firmware))
> -		goto free_rproc;
> -
> -	if (rproc_alloc_ops(rproc, ops))
> -		goto free_firmware;
> -
>  	rproc->name = name;
>  	rproc->priv = &rproc[1];
>  	rproc->auto_boot = true;
> @@ -2070,12 +2064,17 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
>  	rproc->dev.class = &rproc_class;
>  	rproc->dev.driver_data = rproc;

Unrelated to this patch, but do we need to move the idr_init on
notify_ids here?

regards
Suman

>  
> +	if (rproc_alloc_firmware(rproc, name, firmware))
> +		goto out;
> +
> +	if (rproc_alloc_ops(rproc, ops))
> +		goto out;
> +
>  	/* Assign a unique device index and name */
>  	rproc->index = ida_simple_get(&rproc_dev_index, 0, 0, GFP_KERNEL);
>  	if (rproc->index < 0) {
>  		dev_err(dev, "ida_simple_get failed: %d\n", rproc->index);
> -		put_device(&rproc->dev);
> -		return NULL;
> +		goto out;
>  	}
>  
>  	dev_set_name(&rproc->dev, "remoteproc%d", rproc->index);
> @@ -2098,11 +2097,8 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
>  	rproc->state = RPROC_OFFLINE;
>  
>  	return rproc;
> -
> -free_firmware:
> -	kfree(rproc->firmware);
> -free_rproc:
> -	kfree(rproc);
> +out:
> +	put_device(&rproc->dev);
>  	return NULL;
>  }
>  EXPORT_SYMBOL(rproc_alloc);
> 

Re: [PATCH v2 06/17] remoteproc: Introduce function rproc_alloc_internals()

[Suman Anna]

Hi Mathieu,

On 3/27/20 6:10 AM, Loic PALLARDY wrote:
> Hi Mathieu,
> 
>>
>> In preparation to allocate the synchronisation operation and state
>> machine, spin off a new function in order to keep rproc_alloc() as
>> clean as possible.
>>
>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>> ---
>>  drivers/remoteproc/remoteproc_core.c | 26 ++++++++++++++++++++++---
>> -
>>  1 file changed, 22 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/remoteproc/remoteproc_core.c
>> b/drivers/remoteproc/remoteproc_core.c
>> index ee277bc5556c..9da245734db6 100644
>> --- a/drivers/remoteproc/remoteproc_core.c
>> +++ b/drivers/remoteproc/remoteproc_core.c
>> @@ -2018,6 +2018,26 @@ static int rproc_alloc_ops(struct rproc *rproc,
>> const struct rproc_ops *ops)
>>  	return 0;
>>  }
>>
>> +static int rproc_alloc_internals(struct rproc *rproc, const char *name,
>> +				 const struct rproc_ops *boot_ops,
>> +				 const char *firmware, int len)
> 
> len argument is not used in the patch nor in the following, maybe removed from my pov.

Indeed.

> 
> Regards,
> Loic

>> +{
>> +	int ret;
>> +
>> +	/* We have a boot_ops so allocate firmware name and operations */
>> +	if (boot_ops) {
>> +		ret = rproc_alloc_firmware(rproc, name, firmware);
>> +		if (ret)
>> +			return ret;

So, can you explain why firmware allocation now becomes conditional on
this boot_ops?

Perhaps, continue to call this as ops following the field name in struct
rproc.

regards
Suman

>> +
>> +		ret = rproc_alloc_ops(rproc, boot_ops);
>> +		if (ret)
>> +			return ret;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>>  /**
>>   * rproc_alloc() - allocate a remote processor handle
>>   * @dev: the underlying device
>> @@ -2064,10 +2084,8 @@ struct rproc *rproc_alloc(struct device *dev, const
>> char *name,
>>  	rproc->dev.class = &rproc_class;
>>  	rproc->dev.driver_data = rproc;
>>
>> -	if (rproc_alloc_firmware(rproc, name, firmware))
>> -		goto out;
>> -
>> -	if (rproc_alloc_ops(rproc, ops))
>> +	if (rproc_alloc_internals(rproc, name, ops,
>> +				  firmware, len))
>>  		goto out;
>>
>>  	/* Assign a unique device index and name */
>> --
>> 2.20.1
> 

Re: [PATCH v2 01/17] remoteproc: Add new operation and state machine for MCU synchronisation

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> Add a new rproc_ops sync_ops to support use cases where the remoteproc
> core is synchronisting with the MCU.  When exactly to use the sync_ops is

typo on syschronisting..

> directed by the states in struct rproc_sync_states.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_debugfs.c  | 31 ++++++++++++++++++++++++
>  drivers/remoteproc/remoteproc_internal.h |  5 ++++
>  include/linux/remoteproc.h               | 23 +++++++++++++++++-
>  3 files changed, 58 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_debugfs.c b/drivers/remoteproc/remoteproc_debugfs.c
> index dd93cf04e17f..187bcc67f997 100644
> --- a/drivers/remoteproc/remoteproc_debugfs.c
> +++ b/drivers/remoteproc/remoteproc_debugfs.c
> @@ -311,6 +311,35 @@ static const struct file_operations rproc_carveouts_ops = {
>  	.release	= single_release,
>  };
>  
> +/* Expose synchronisation states via debugfs */
> +static int rproc_sync_states_show(struct seq_file *seq, void *p)
> +{
> +	struct rproc *rproc = seq->private;
> +
> +	seq_printf(seq, "Sync with MCU: %s\n",
> +		   rproc->sync_with_mcu ? "true" : "false");
> +	seq_printf(seq, "On init: %s\n",
> +		   rproc->sync_states->on_init ? "true" : "false");
> +	seq_printf(seq, "After stop: %s\n",
> +		   rproc->sync_states->after_stop ? "true" : "false");
> +	seq_printf(seq, "After crash: %s\n",
> +		   rproc->sync_states->after_crash ? "true" : "false");
> +
> +	return 0;
> +}
> +
> +static int rproc_sync_states_open(struct inode *inode, struct file *file)
> +{
> +	return single_open(file, rproc_sync_states_show, inode->i_private);
> +}
> +
> +static const struct file_operations rproc_sync_states_ops = {
> +	.open		= rproc_sync_states_open,
> +	.read		= seq_read,
> +	.llseek		= seq_lseek,
> +	.release	= single_release,
> +};
> +
>  void rproc_remove_trace_file(struct dentry *tfile)
>  {
>  	debugfs_remove(tfile);
> @@ -357,6 +386,8 @@ void rproc_create_debug_dir(struct rproc *rproc)
>  			    rproc, &rproc_rsc_table_ops);
>  	debugfs_create_file("carveout_memories", 0400, rproc->dbg_dir,
>  			    rproc, &rproc_carveouts_ops);
> +	debugfs_create_file("sync_states", 0400, rproc->dbg_dir,
> +			    rproc, &rproc_sync_states_ops);
>  }
>  
>  void __init rproc_init_debugfs(void)
> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
> index 493ef9262411..5c93de5e00bb 100644
> --- a/drivers/remoteproc/remoteproc_internal.h
> +++ b/drivers/remoteproc/remoteproc_internal.h
> @@ -63,6 +63,11 @@ struct resource_table *rproc_elf_find_loaded_rsc_table(struct rproc *rproc,
>  struct rproc_mem_entry *
>  rproc_find_carveout_by_name(struct rproc *rproc, const char *name, ...);
>  
> +static inline bool rproc_sync_with_mcu(struct rproc *rproc)
> +{
> +	return rproc->sync_with_mcu;
> +}
> +

Since you are using this mostly for checking and as a boolean, I suggest
you rename this appropriately, something like rproc_needs_sync,
rproc_has_sync or rproc_uses_sync().

And I am wondering if it is actually better to introduce the sync state
to check against here, rather than using the stored sync state and
return. The current way makes it confusing to read the state machine.

>  static inline
>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
>  {
> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
> index 16ad66683ad0..d115e47d702d 100644
> --- a/include/linux/remoteproc.h
> +++ b/include/linux/remoteproc.h
> @@ -353,6 +353,21 @@ enum rsc_handling_status {
>  	RSC_IGNORED	= 1,
>  };
>  
> +/**
> + * struct rproc_sync_states - platform specific states indicating which
> + *			      rproc_ops to use at specific times during
> + *			      the MCU lifecycle.
> + * @on_init: true if synchronising with MCU at system initialisation time
> + * @after_stop: true if synchronising with MCU after stopped from the
> + *		command line
> + * @after_crash: true if synchonising with MCU after the MCU has crashed
> + */
> +struct rproc_sync_states {
> +	bool on_init;
> +	bool after_stop;
> +	bool after_crash;
> +};
> +

Overall, this patch can move down the order, and better to add it in
the patches where you actually introduce these code. And the debugfs
pieces can be added as a separate patch by itself.

>  /**
>   * struct rproc_ops - platform-specific device handlers
>   * @start:	power on the device and boot it
> @@ -456,6 +471,9 @@ struct rproc_dump_segment {
>   * @firmware: name of firmware file to be loaded
>   * @priv: private data which belongs to the platform-specific rproc module
>   * @ops: platform-specific start/stop rproc handlers
> + * @sync_ops: paltform-specific start/stop rproc handlers when

typo on platform

> + *	      synchronising with a remote processor.
> + * @sync_states: Determine the rproc_ops to choose in specific states.
>   * @dev: virtual device for refcounting and common remoteproc behavior
>   * @power: refcount of users who need this rproc powered up
>   * @state: state of the device
> @@ -479,6 +497,7 @@ struct rproc_dump_segment {
>   * @table_sz: size of @cached_table
>   * @has_iommu: flag to indicate if remote processor is behind an MMU
>   * @auto_boot: flag to indicate if remote processor should be auto-started
> + * @sync_with_mcu: true if currently synchronising with MCU
>   * @dump_segments: list of segments in the firmware
>   * @nb_vdev: number of vdev currently handled by rproc
>   */
> @@ -488,7 +507,8 @@ struct rproc {
>  	const char *name;
>  	char *firmware;
>  	void *priv;
> -	struct rproc_ops *ops;
> +	struct rproc_ops *ops, *sync_ops;

Nothing wrong with this, but prefer to have the new variable in a new
line for better readability.

regards
Suman

> +	struct rproc_sync_states *sync_states;
>  	struct device dev;
>  	atomic_t power;
>  	unsigned int state;
> @@ -512,6 +532,7 @@ struct rproc {
>  	size_t table_sz;
>  	bool has_iommu;
>  	bool auto_boot;
> +	bool sync_with_mcu;
>  	struct list_head dump_segments;
>  	int nb_vdev;
>  };
> 

Re: [PATCH v2 01/17] remoteproc: Add new operation and state machine for MCU synchronisation

[Suman Anna]

On 3/30/20 5:46 PM, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/24/20 4:45 PM, Mathieu Poirier wrote:
>> Add a new rproc_ops sync_ops to support use cases where the remoteproc
>> core is synchronisting with the MCU.  When exactly to use the sync_ops is
> 
> typo on syschronisting..
> 
>> directed by the states in struct rproc_sync_states.
>>
>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>> ---
>>  drivers/remoteproc/remoteproc_debugfs.c  | 31 ++++++++++++++++++++++++
>>  drivers/remoteproc/remoteproc_internal.h |  5 ++++
>>  include/linux/remoteproc.h               | 23 +++++++++++++++++-
>>  3 files changed, 58 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/remoteproc/remoteproc_debugfs.c b/drivers/remoteproc/remoteproc_debugfs.c
>> index dd93cf04e17f..187bcc67f997 100644
>> --- a/drivers/remoteproc/remoteproc_debugfs.c
>> +++ b/drivers/remoteproc/remoteproc_debugfs.c
>> @@ -311,6 +311,35 @@ static const struct file_operations rproc_carveouts_ops = {
>>  	.release	= single_release,
>>  };
>>  
>> +/* Expose synchronisation states via debugfs */
>> +static int rproc_sync_states_show(struct seq_file *seq, void *p)
>> +{
>> +	struct rproc *rproc = seq->private;
>> +
>> +	seq_printf(seq, "Sync with MCU: %s\n",
>> +		   rproc->sync_with_mcu ? "true" : "false");
>> +	seq_printf(seq, "On init: %s\n",
>> +		   rproc->sync_states->on_init ? "true" : "false");
>> +	seq_printf(seq, "After stop: %s\n",
>> +		   rproc->sync_states->after_stop ? "true" : "false");
>> +	seq_printf(seq, "After crash: %s\n",
>> +		   rproc->sync_states->after_crash ? "true" : "false");
>> +
>> +	return 0;
>> +}
>> +
>> +static int rproc_sync_states_open(struct inode *inode, struct file *file)
>> +{
>> +	return single_open(file, rproc_sync_states_show, inode->i_private);
>> +}
>> +
>> +static const struct file_operations rproc_sync_states_ops = {
>> +	.open		= rproc_sync_states_open,
>> +	.read		= seq_read,
>> +	.llseek		= seq_lseek,
>> +	.release	= single_release,
>> +};
>> +
>>  void rproc_remove_trace_file(struct dentry *tfile)
>>  {
>>  	debugfs_remove(tfile);
>> @@ -357,6 +386,8 @@ void rproc_create_debug_dir(struct rproc *rproc)
>>  			    rproc, &rproc_rsc_table_ops);
>>  	debugfs_create_file("carveout_memories", 0400, rproc->dbg_dir,
>>  			    rproc, &rproc_carveouts_ops);
>> +	debugfs_create_file("sync_states", 0400, rproc->dbg_dir,
>> +			    rproc, &rproc_sync_states_ops);
>>  }
>>  
>>  void __init rproc_init_debugfs(void)
>> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
>> index 493ef9262411..5c93de5e00bb 100644
>> --- a/drivers/remoteproc/remoteproc_internal.h
>> +++ b/drivers/remoteproc/remoteproc_internal.h
>> @@ -63,6 +63,11 @@ struct resource_table *rproc_elf_find_loaded_rsc_table(struct rproc *rproc,
>>  struct rproc_mem_entry *
>>  rproc_find_carveout_by_name(struct rproc *rproc, const char *name, ...);
>>  
>> +static inline bool rproc_sync_with_mcu(struct rproc *rproc)
>> +{
>> +	return rproc->sync_with_mcu;
>> +}
>> +
> 
> Since you are using this mostly for checking and as a boolean, I suggest
> you rename this appropriately, something like rproc_needs_sync,
> rproc_has_sync or rproc_uses_sync().
> 
> And I am wondering if it is actually better to introduce the sync state
> to check against here, rather than using the stored sync state and
> return. The current way makes it confusing to read the state machine.
> 
>>  static inline
>>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
>>  {
>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
>> index 16ad66683ad0..d115e47d702d 100644
>> --- a/include/linux/remoteproc.h
>> +++ b/include/linux/remoteproc.h
>> @@ -353,6 +353,21 @@ enum rsc_handling_status {
>>  	RSC_IGNORED	= 1,
>>  };
>>  
>> +/**
>> + * struct rproc_sync_states - platform specific states indicating which
>> + *			      rproc_ops to use at specific times during
>> + *			      the MCU lifecycle.
>> + * @on_init: true if synchronising with MCU at system initialisation time
>> + * @after_stop: true if synchronising with MCU after stopped from the
>> + *		command line
>> + * @after_crash: true if synchonising with MCU after the MCU has crashed
>> + */
>> +struct rproc_sync_states {
>> +	bool on_init;
>> +	bool after_stop;
>> +	bool after_crash;
>> +};
>> +
> 
> Overall, this patch can move down the order, and better to add it in
> the patches where you actually introduce these code. And the debugfs
> pieces can be added as a separate patch by itself.

Also, actually sounds more like flags than states..

regards
Suman

> 
>>  /**
>>   * struct rproc_ops - platform-specific device handlers
>>   * @start:	power on the device and boot it
>> @@ -456,6 +471,9 @@ struct rproc_dump_segment {
>>   * @firmware: name of firmware file to be loaded
>>   * @priv: private data which belongs to the platform-specific rproc module
>>   * @ops: platform-specific start/stop rproc handlers
>> + * @sync_ops: paltform-specific start/stop rproc handlers when
> 
> typo on platform
> 
>> + *	      synchronising with a remote processor.
>> + * @sync_states: Determine the rproc_ops to choose in specific states.
>>   * @dev: virtual device for refcounting and common remoteproc behavior
>>   * @power: refcount of users who need this rproc powered up
>>   * @state: state of the device
>> @@ -479,6 +497,7 @@ struct rproc_dump_segment {
>>   * @table_sz: size of @cached_table
>>   * @has_iommu: flag to indicate if remote processor is behind an MMU
>>   * @auto_boot: flag to indicate if remote processor should be auto-started
>> + * @sync_with_mcu: true if currently synchronising with MCU
>>   * @dump_segments: list of segments in the firmware
>>   * @nb_vdev: number of vdev currently handled by rproc
>>   */
>> @@ -488,7 +507,8 @@ struct rproc {
>>  	const char *name;
>>  	char *firmware;
>>  	void *priv;
>> -	struct rproc_ops *ops;
>> +	struct rproc_ops *ops, *sync_ops;
> 
> Nothing wrong with this, but prefer to have the new variable in a new
> line for better readability.
> 
> regards
> Suman
> 
>> +	struct rproc_sync_states *sync_states;
>>  	struct device dev;
>>  	atomic_t power;
>>  	unsigned int state;
>> @@ -512,6 +532,7 @@ struct rproc {
>>  	size_t table_sz;
>>  	bool has_iommu;
>>  	bool auto_boot;
>> +	bool sync_with_mcu;
>>  	struct list_head dump_segments;
>>  	int nb_vdev;
>>  };
>>
> 

Re: [PATCH v2 02/17] remoteproc: Introduce function rproc_set_mcu_sync_state()

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> Introduce function rproc_set_mcu_sync_state() to set the synchronisation
> state of the MCU at various stages of the lifecycle process.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_internal.h | 38 ++++++++++++++++++++++++
>  1 file changed, 38 insertions(+)
> 
> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
> index 5c93de5e00bb..73ea32df0156 100644
> --- a/drivers/remoteproc/remoteproc_internal.h
> +++ b/drivers/remoteproc/remoteproc_internal.h
> @@ -24,6 +24,26 @@ struct rproc_debug_trace {
>  	struct rproc_mem_entry trace_mem;
>  };
>  
> +/*
> + * enum rproc_sync_states - remote processsor sync states
> + * @RPROC_SYNC_STATE_INIT	state to use when the remoteproc core
> + *				is initialising.
> + * @RPROC_SYNC_STATE_SHUTDOWN	state to use after the remoteproc core
> + *				has shutdown (rproc_shutdown()) the MCU.
> + * @RPROC_SYNC_STATE_CRASHED	state to use after the MCU has crashed but
> + *				has not been recovered by the remoteproc
> + *				core yet.
> + *
> + * Keeping these separate from the enum rproc_state in order to avoid
> + * introducing coupling between the state of the MCU and the synchronisation
> + * operation to use.
> + */
> +enum rproc_mcu_sync_states {
> +	RPROC_SYNC_STATE_INIT,
> +	RPROC_SYNC_STATE_SHUTDOWN,
> +	RPROC_SYNC_STATE_CRASHED,
> +};
> +

Perhaps rename the enum as rproc_sync_state

>  /* from remoteproc_core.c */
>  void rproc_release(struct kref *kref);
>  irqreturn_t rproc_vq_interrupt(struct rproc *rproc, int vq_id);
> @@ -68,6 +88,24 @@ static inline bool rproc_sync_with_mcu(struct rproc *rproc)
>  	return rproc->sync_with_mcu;
>  }
>  
> +static inline void rproc_set_mcu_sync_state(struct rproc *rproc,
> +					    unsigned int state)

Change the argument type to the above enum, and perhaps rename the
function to use flags instead of state.

regards
Suman


> +{
> +	switch (state) {
> +	case RPROC_SYNC_STATE_INIT:
> +		rproc->sync_with_mcu = rproc->sync_states->on_init;
> +		break;
> +	case RPROC_SYNC_STATE_SHUTDOWN:
> +		rproc->sync_with_mcu = rproc->sync_states->after_stop;
> +		break;
> +	case RPROC_SYNC_STATE_CRASHED:
> +		rproc->sync_with_mcu = rproc->sync_states->after_crash;
> +		break;
> +	default:
> +		break;
> +	}
> +}
> +
>  static inline
>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
>  {
> 

Re: [PATCH v2 06/17] remoteproc: Introduce function rproc_alloc_internals()

[Mathieu Poirier]

On Fri, Mar 27, 2020 at 11:10:20AM +0000, Loic PALLARDY wrote:
> Hi Mathieu,
> 
> > -----Original Message-----
> > From: Mathieu Poirier <mathieu.poirier@linaro.org>
> > Sent: mardi 24 mars 2020 22:46
> > To: bjorn.andersson@linaro.org
> > Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> > anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> > <arnaud.pouliquen@st.com>; Fabien DESSENNE
> > <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> > Subject: [PATCH v2 06/17] remoteproc: Introduce function
> > rproc_alloc_internals()
> > 
> > In preparation to allocate the synchronisation operation and state
> > machine, spin off a new function in order to keep rproc_alloc() as
> > clean as possible.
> > 
> > Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> > ---
> >  drivers/remoteproc/remoteproc_core.c | 26 ++++++++++++++++++++++---
> > -
> >  1 file changed, 22 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/remoteproc/remoteproc_core.c
> > b/drivers/remoteproc/remoteproc_core.c
> > index ee277bc5556c..9da245734db6 100644
> > --- a/drivers/remoteproc/remoteproc_core.c
> > +++ b/drivers/remoteproc/remoteproc_core.c
> > @@ -2018,6 +2018,26 @@ static int rproc_alloc_ops(struct rproc *rproc,
> > const struct rproc_ops *ops)
> >  	return 0;
> >  }
> > 
> > +static int rproc_alloc_internals(struct rproc *rproc, const char *name,
> > +				 const struct rproc_ops *boot_ops,
> > +				 const char *firmware, int len)
> 
> len argument is not used in the patch nor in the following, maybe removed from my pov.

I debated over this one... It is either introduce the function signature as a
whole or incrementally as parameters are needed.  I'm fine with both and will
adopt the latter on the next revision.

> 
> Regards,
> Loic
> > +{
> > +	int ret;
> > +
> > +	/* We have a boot_ops so allocate firmware name and operations */
> > +	if (boot_ops) {
> > +		ret = rproc_alloc_firmware(rproc, name, firmware);
> > +		if (ret)
> > +			return ret;
> > +
> > +		ret = rproc_alloc_ops(rproc, boot_ops);
> > +		if (ret)
> > +			return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> >  /**
> >   * rproc_alloc() - allocate a remote processor handle
> >   * @dev: the underlying device
> > @@ -2064,10 +2084,8 @@ struct rproc *rproc_alloc(struct device *dev, const
> > char *name,
> >  	rproc->dev.class = &rproc_class;
> >  	rproc->dev.driver_data = rproc;
> > 
> > -	if (rproc_alloc_firmware(rproc, name, firmware))
> > -		goto out;
> > -
> > -	if (rproc_alloc_ops(rproc, ops))
> > +	if (rproc_alloc_internals(rproc, name, ops,
> > +				  firmware, len))
> >  		goto out;
> > 
> >  	/* Assign a unique device index and name */
> > --
> > 2.20.1
> 

Re: [PATCH v2 07/17] remoteproc: Introduce function rproc_alloc_state_machine()

[Suman Anna]

Hi Mathieu,
> 
>> -----Original Message-----
>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
>> Sent: mardi 24 mars 2020 22:46
>> To: bjorn.andersson@linaro.org
>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
>> Subject: [PATCH v2 07/17] remoteproc: Introduce function
>> rproc_alloc_state_machine()
>>
>> Introducing new function rproc_alloc_state_machine() to allocate
>> the MCU synchronisation operations and position it as the central
>> remoteproc core allocation function.
>>
>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>> ---
>>  drivers/remoteproc/remoteproc_core.c | 84
>> +++++++++++++++++++++++++---
>>  include/linux/remoteproc.h           |  5 ++
>>  2 files changed, 81 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/remoteproc/remoteproc_core.c
>> b/drivers/remoteproc/remoteproc_core.c
>> index 9da245734db6..02dbb826aa29 100644
>> --- a/drivers/remoteproc/remoteproc_core.c
>> +++ b/drivers/remoteproc/remoteproc_core.c
>> @@ -1954,6 +1954,7 @@ static void rproc_type_release(struct device *dev)
>>
>>  	kfree(rproc->firmware);
>>  	kfree(rproc->ops);
>> +	kfree(rproc->sync_ops);
>>  	kfree(rproc);
>>  }
>>
>> @@ -2018,12 +2019,34 @@ static int rproc_alloc_ops(struct rproc *rproc,
>> const struct rproc_ops *ops)
>>  	return 0;
>>  }
>>
>> +static int rproc_alloc_sync_ops(struct rproc *rproc,
>> +				const struct rproc_ops *sync_ops)
>> +{
>> +	/*
>> +	 * Given the unlimited amount of possibilities when
>> +	 * synchronising with an MCU, no constraints are imposed
>> +	 * on sync_ops.
>> +	 */
>> +	rproc->sync_ops = kmemdup(sync_ops,
>> +				  sizeof(*sync_ops), GFP_KERNEL);
>> +	if (!rproc->sync_ops)
>> +		return -ENOMEM;
> Should we check a minimal set of functions in sync_ops to be required?
> Or we should consider all pointers at NULL is ok ?
> 
>> +
>> +	return 0;
>> +}
>> +
>>  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
>>  				 const struct rproc_ops *boot_ops,
>> +				 const struct rproc_ops *sync_ops,
>> +				 struct rproc_sync_states *sync_states,
> sync_states not used in this patch, should be introduced in patch 8

+1

> 
> Regards,
> Loic
> 
>>  				 const char *firmware, int len)
>>  {
>>  	int ret;
>>
>> +	/* We need at least a boot or a sync ops. */
>> +	if (!boot_ops && !sync_ops)
>> +		return -EINVAL;
>> +
>>  	/* We have a boot_ops so allocate firmware name and operations */
>>  	if (boot_ops) {
>>  		ret = rproc_alloc_firmware(rproc, name, firmware);
>> @@ -2035,14 +2058,23 @@ static int rproc_alloc_internals(struct rproc
>> *rproc, const char *name,
>>  			return ret;
>>  	}
>>
>> +	/* Allocate a sync_ops if need be */
>> +	if (sync_ops) {
>> +		ret = rproc_alloc_sync_ops(rproc, sync_ops);
>> +		if (ret)
>> +			return ret;
>> +	}
>> +
>>  	return 0;
>>  }
>>
>>  /**
>> - * rproc_alloc() - allocate a remote processor handle
>> + * rproc_alloc_state_machine() - allocate a remote processor handle
>>   * @dev: the underlying device
>>   * @name: name of this remote processor
>>   * @ops: platform-specific handlers (mainly start/stop)
>> + * @sync_ops: platform-specific handlers for synchronising with MCU
>> + * @sync_states: states in which @ops and @sync_ops are to be used
>>   * @firmware: name of firmware file to load, can be NULL
>>   * @len: length of private data needed by the rproc driver (in bytes)
>>   *
>> @@ -2061,13 +2093,15 @@ static int rproc_alloc_internals(struct rproc
>> *rproc, const char *name,
>>   * Note: _never_ directly deallocate @rproc, even if it was not registered
>>   * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
>>   */
>> -struct rproc *rproc_alloc(struct device *dev, const char *name,
>> -			  const struct rproc_ops *ops,
>> -			  const char *firmware, int len)
>> +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
>> *name,
>> +					const struct rproc_ops *ops,
>> +					const struct rproc_ops *sync_ops,
>> +					struct rproc_sync_states
>> *sync_states,
>> +					const char *firmware, int len)

Do you foresee the need for sync_ops to be present as long as the rproc
is registered? I am wondering if it is better to introduce an API where
you can set the ops at runtime rather than allocate it upfront, so that
once the initial handling is done, you can reset both the sync_states
and ops.


>>  {
>>  	struct rproc *rproc;
>>
>> -	if (!dev || !name || !ops)
>> +	if (!dev || !name)
>>  		return NULL;
>>
>>  	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
>> @@ -2084,8 +2118,8 @@ struct rproc *rproc_alloc(struct device *dev, const
>> char *name,
>>  	rproc->dev.class = &rproc_class;
>>  	rproc->dev.driver_data = rproc;
>>
>> -	if (rproc_alloc_internals(rproc, name, ops,
>> -				  firmware, len))
>> +	if (rproc_alloc_internals(rproc, name, ops, sync_ops,
>> +				  sync_states, firmware, len))
>>  		goto out;
>>
>>  	/* Assign a unique device index and name */
>> @@ -2119,7 +2153,41 @@ struct rproc *rproc_alloc(struct device *dev, const
>> char *name,
>>  	put_device(&rproc->dev);
>>  	return NULL;
>>  }
>> -EXPORT_SYMBOL(rproc_alloc);
>> +EXPORT_SYMBOL(rproc_alloc_state_machine);
>> +
>> +/**
>> + * rproc_alloc() - allocate a remote processor handle
>> + * @dev: the underlying device
>> + * @name: name of this remote processor
>> + * @ops: platform-specific handlers (mainly start/stop)
>> + * @firmware: name of firmware file to load, can be NULL
>> + * @len: length of private data needed by the rproc driver (in bytes)
>> + *
>> + * Allocates a new remote processor handle, but does not register
>> + * it yet. if @firmware is NULL, a default name is used.
>> + *
>> + * This function should be used by rproc implementations during
>> initialization
>> + * of the remote processor.
>> + *
>> + * After creating an rproc handle using this function, and when ready,
>> + * implementations should then call rproc_add() to complete
>> + * the registration of the remote processor.
>> + *
>> + * On success the new rproc is returned, and on failure, NULL.
>> + *
>> + * Note: _never_ directly deallocate @rproc, even if it was not registered
>> + * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
>> + */
>> +struct rproc *rproc_alloc(struct device *dev, const char *name,
>> +			  const struct rproc_ops *ops,
>> +			  const char *firmware, int len)
>> +{
>> +	if (!name && !firmware)

Retain the original checks on dev, name and ops from the previous
rproc_alloc(). A NULL firmware was perfectly valid before, and the name
is allocated using the default template.

>> +		return NULL;
>> +
>> +	return rproc_alloc_state_machine(dev, name, ops, NULL, NULL,
>> +					 firmware, len);
>> +}

Missing the EXPORT_SYMBOL on rproc_alloc() -> it is an API used by modules.

regards
Suman

>>
>>  /**
>>   * rproc_free() - unroll rproc_alloc()
>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
>> index d115e47d702d..d1214487daac 100644
>> --- a/include/linux/remoteproc.h
>> +++ b/include/linux/remoteproc.h
>> @@ -611,6 +611,11 @@ struct rproc *rproc_get_by_child(struct device
>> *dev);
>>  struct rproc *rproc_alloc(struct device *dev, const char *name,
>>  			  const struct rproc_ops *ops,
>>  			  const char *firmware, int len);
>> +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
>> *name,
>> +					const struct rproc_ops *ops,
>> +					const struct rproc_ops *sync_ops,
>> +					struct rproc_sync_states
>> *sync_states,
>> +					const char *firmware, int len);
>>  void rproc_put(struct rproc *rproc);
>>  int rproc_add(struct rproc *rproc);
>>  int rproc_del(struct rproc *rproc);
>> --
>> 2.20.1
> 

Re: [PATCH v2 07/17] remoteproc: Introduce function rproc_alloc_state_machine()

[Mathieu Poirier]

On Fri, Mar 27, 2020 at 01:12:11PM +0000, Loic PALLARDY wrote:
> 
> 
> > -----Original Message-----
> > From: Mathieu Poirier <mathieu.poirier@linaro.org>
> > Sent: mardi 24 mars 2020 22:46
> > To: bjorn.andersson@linaro.org
> > Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> > anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> > <arnaud.pouliquen@st.com>; Fabien DESSENNE
> > <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> > Subject: [PATCH v2 07/17] remoteproc: Introduce function
> > rproc_alloc_state_machine()
> > 
> > Introducing new function rproc_alloc_state_machine() to allocate
> > the MCU synchronisation operations and position it as the central
> > remoteproc core allocation function.
> > 
> > Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> > ---
> >  drivers/remoteproc/remoteproc_core.c | 84
> > +++++++++++++++++++++++++---
> >  include/linux/remoteproc.h           |  5 ++
> >  2 files changed, 81 insertions(+), 8 deletions(-)
> > 
> > diff --git a/drivers/remoteproc/remoteproc_core.c
> > b/drivers/remoteproc/remoteproc_core.c
> > index 9da245734db6..02dbb826aa29 100644
> > --- a/drivers/remoteproc/remoteproc_core.c
> > +++ b/drivers/remoteproc/remoteproc_core.c
> > @@ -1954,6 +1954,7 @@ static void rproc_type_release(struct device *dev)
> > 
> >  	kfree(rproc->firmware);
> >  	kfree(rproc->ops);
> > +	kfree(rproc->sync_ops);
> >  	kfree(rproc);
> >  }
> > 
> > @@ -2018,12 +2019,34 @@ static int rproc_alloc_ops(struct rproc *rproc,
> > const struct rproc_ops *ops)
> >  	return 0;
> >  }
> > 
> > +static int rproc_alloc_sync_ops(struct rproc *rproc,
> > +				const struct rproc_ops *sync_ops)
> > +{
> > +	/*
> > +	 * Given the unlimited amount of possibilities when
> > +	 * synchronising with an MCU, no constraints are imposed
> > +	 * on sync_ops.
> > +	 */
> > +	rproc->sync_ops = kmemdup(sync_ops,
> > +				  sizeof(*sync_ops), GFP_KERNEL);
> > +	if (!rproc->sync_ops)
> > +		return -ENOMEM;
> Should we check a minimal set of functions in sync_ops to be required?
> Or we should consider all pointers at NULL is ok ?

I spent a fair amount of time thinking about that very question... The problem
is that requirements are all over the place and I did not want to introduce
constraints.  I am highly influenceable on this one - ideas welcomed.

> 
> > +
> > +	return 0;
> > +}
> > +
> >  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
> >  				 const struct rproc_ops *boot_ops,
> > +				 const struct rproc_ops *sync_ops,
> > +				 struct rproc_sync_states *sync_states,
> sync_states not used in this patch, should be introduced in patch 8

Very well.

> 
> Regards,
> Loic
> 
> >  				 const char *firmware, int len)
> >  {
> >  	int ret;
> > 
> > +	/* We need at least a boot or a sync ops. */
> > +	if (!boot_ops && !sync_ops)
> > +		return -EINVAL;
> > +
> >  	/* We have a boot_ops so allocate firmware name and operations */
> >  	if (boot_ops) {
> >  		ret = rproc_alloc_firmware(rproc, name, firmware);
> > @@ -2035,14 +2058,23 @@ static int rproc_alloc_internals(struct rproc
> > *rproc, const char *name,
> >  			return ret;
> >  	}
> > 
> > +	/* Allocate a sync_ops if need be */
> > +	if (sync_ops) {
> > +		ret = rproc_alloc_sync_ops(rproc, sync_ops);
> > +		if (ret)
> > +			return ret;
> > +	}
> > +
> >  	return 0;
> >  }
> > 
> >  /**
> > - * rproc_alloc() - allocate a remote processor handle
> > + * rproc_alloc_state_machine() - allocate a remote processor handle
> >   * @dev: the underlying device
> >   * @name: name of this remote processor
> >   * @ops: platform-specific handlers (mainly start/stop)
> > + * @sync_ops: platform-specific handlers for synchronising with MCU
> > + * @sync_states: states in which @ops and @sync_ops are to be used
> >   * @firmware: name of firmware file to load, can be NULL
> >   * @len: length of private data needed by the rproc driver (in bytes)
> >   *
> > @@ -2061,13 +2093,15 @@ static int rproc_alloc_internals(struct rproc
> > *rproc, const char *name,
> >   * Note: _never_ directly deallocate @rproc, even if it was not registered
> >   * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
> >   */
> > -struct rproc *rproc_alloc(struct device *dev, const char *name,
> > -			  const struct rproc_ops *ops,
> > -			  const char *firmware, int len)
> > +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
> > *name,
> > +					const struct rproc_ops *ops,
> > +					const struct rproc_ops *sync_ops,
> > +					struct rproc_sync_states
> > *sync_states,
> > +					const char *firmware, int len)
> >  {
> >  	struct rproc *rproc;
> > 
> > -	if (!dev || !name || !ops)
> > +	if (!dev || !name)
> >  		return NULL;
> > 
> >  	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
> > @@ -2084,8 +2118,8 @@ struct rproc *rproc_alloc(struct device *dev, const
> > char *name,
> >  	rproc->dev.class = &rproc_class;
> >  	rproc->dev.driver_data = rproc;
> > 
> > -	if (rproc_alloc_internals(rproc, name, ops,
> > -				  firmware, len))
> > +	if (rproc_alloc_internals(rproc, name, ops, sync_ops,
> > +				  sync_states, firmware, len))
> >  		goto out;
> > 
> >  	/* Assign a unique device index and name */
> > @@ -2119,7 +2153,41 @@ struct rproc *rproc_alloc(struct device *dev, const
> > char *name,
> >  	put_device(&rproc->dev);
> >  	return NULL;
> >  }
> > -EXPORT_SYMBOL(rproc_alloc);
> > +EXPORT_SYMBOL(rproc_alloc_state_machine);
> > +
> > +/**
> > + * rproc_alloc() - allocate a remote processor handle
> > + * @dev: the underlying device
> > + * @name: name of this remote processor
> > + * @ops: platform-specific handlers (mainly start/stop)
> > + * @firmware: name of firmware file to load, can be NULL
> > + * @len: length of private data needed by the rproc driver (in bytes)
> > + *
> > + * Allocates a new remote processor handle, but does not register
> > + * it yet. if @firmware is NULL, a default name is used.
> > + *
> > + * This function should be used by rproc implementations during
> > initialization
> > + * of the remote processor.
> > + *
> > + * After creating an rproc handle using this function, and when ready,
> > + * implementations should then call rproc_add() to complete
> > + * the registration of the remote processor.
> > + *
> > + * On success the new rproc is returned, and on failure, NULL.
> > + *
> > + * Note: _never_ directly deallocate @rproc, even if it was not registered
> > + * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
> > + */
> > +struct rproc *rproc_alloc(struct device *dev, const char *name,
> > +			  const struct rproc_ops *ops,
> > +			  const char *firmware, int len)
> > +{
> > +	if (!name && !firmware)
> > +		return NULL;
> > +
> > +	return rproc_alloc_state_machine(dev, name, ops, NULL, NULL,
> > +					 firmware, len);
> > +}
> > 
> >  /**
> >   * rproc_free() - unroll rproc_alloc()
> > diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
> > index d115e47d702d..d1214487daac 100644
> > --- a/include/linux/remoteproc.h
> > +++ b/include/linux/remoteproc.h
> > @@ -611,6 +611,11 @@ struct rproc *rproc_get_by_child(struct device
> > *dev);
> >  struct rproc *rproc_alloc(struct device *dev, const char *name,
> >  			  const struct rproc_ops *ops,
> >  			  const char *firmware, int len);
> > +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
> > *name,
> > +					const struct rproc_ops *ops,
> > +					const struct rproc_ops *sync_ops,
> > +					struct rproc_sync_states
> > *sync_states,
> > +					const char *firmware, int len);
> >  void rproc_put(struct rproc *rproc);
> >  int rproc_add(struct rproc *rproc);
> >  int rproc_del(struct rproc *rproc);
> > --
> > 2.20.1
> 

Re: [PATCH v2 08/17] remoteproc: Allocate synchronisation state machine

[Mathieu Poirier]

On Fri, Mar 27, 2020 at 01:47:21PM +0000, Loic PALLARDY wrote:
> 
> 
> > -----Original Message-----
> > From: Mathieu Poirier <mathieu.poirier@linaro.org>
> > Sent: mardi 24 mars 2020 22:46
> > To: bjorn.andersson@linaro.org
> > Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> > anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> > <arnaud.pouliquen@st.com>; Fabien DESSENNE
> > <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> > Subject: [PATCH v2 08/17] remoteproc: Allocate synchronisation state
> > machine
> > 
> > This patch allocates a synchronisation state machine, either provided or
> > not by users, in order to enact the proper behavior requested by the
> > platform or specific scenarios.
> > 
> > Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> > ---
> >  drivers/remoteproc/remoteproc_core.c | 59
> > +++++++++++++++++++++++++++-
> >  1 file changed, 58 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/remoteproc/remoteproc_core.c
> > b/drivers/remoteproc/remoteproc_core.c
> > index 02dbb826aa29..1578a9c70422 100644
> > --- a/drivers/remoteproc/remoteproc_core.c
> > +++ b/drivers/remoteproc/remoteproc_core.c
> > @@ -1955,6 +1955,7 @@ static void rproc_type_release(struct device *dev)
> >  	kfree(rproc->firmware);
> >  	kfree(rproc->ops);
> >  	kfree(rproc->sync_ops);
> > +	kfree(rproc->sync_states);
> >  	kfree(rproc);
> >  }
> > 
> > @@ -2035,6 +2036,59 @@ static int rproc_alloc_sync_ops(struct rproc *rproc,
> >  	return 0;
> >  }
> > 
> > +static int rproc_alloc_sync_states(struct rproc *rproc,
> > +				   const struct rproc_ops *boot_ops,
> > +				   const struct rproc_ops *sync_ops,
> > +				   struct rproc_sync_states *sync_states)
> > +{
> > +	struct rproc_sync_states *st;
> > +
> > +	/* At least one set of operation is needed */
> > +	if (!boot_ops && !sync_ops)
> > +		return -EINVAL;
> > +
> > +	/* We have a synchronisation state machine, no need to build one */
> > +	if (sync_states) {
> > +		st = kmemdup(sync_states, sizeof(*st), GFP_KERNEL);
> > +		if (!st)
> > +			return -ENOMEM;
> > +
> 
> I think a check between sync_states and boot_ops/sync_ops may be needed here
> even if it is platform driver responsibility to provide coherent configuration
> As soon as one of the sync_states is set at true, sync_ops must be provided
> As soon as one of the sync_states is set at false, boot_ops must be provided

That will help catch errors early - OK.

> 
> Regards,
> Loic
> 
> > +		/* Nothing else to do */
> > +		goto out;
> > +	}
> > +
> > +	/* Allocate synchronisation state machine */
> > +	st = kzalloc(sizeof(*st), GFP_KERNEL);
> > +	if (!st)
> > +		return -ENOMEM;
> > +
> > +	/*
> > +	 * We have a boot_ops and no sync_ops - build a state machine that
> > +	 * does _not_ synchronise with an MCU.
> > +	 */
> > +	if (boot_ops && !sync_ops) {
> > +		st->on_init = st->after_stop = st->after_crash = false;
> > +		goto out;
> > +	}
> > +
> > +	/*
> > +	 * We have a sync_ops and an no boot_ops - build a state machine
> > that
> > +	 * _only_ synchronises with an MCU.
> > +	 */
> > +	if (sync_ops && !boot_ops) {
> > +		st->on_init = st->after_stop = st->after_crash = true;
> > +		goto out;
> > +	}
> > +
> > +out:
> > +	rproc->sync_with_mcu = st->on_init;
> > +	/* And the synchronisation state machine to use */
> > +	rproc->sync_states = st;
> > +	/* Tell the core what to do when initialising */
> > +	rproc_set_mcu_sync_state(rproc, RPROC_SYNC_STATE_INIT);
> > +	return 0;
> > +}
> > +
> >  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
> >  				 const struct rproc_ops *boot_ops,
> >  				 const struct rproc_ops *sync_ops,
> > @@ -2065,7 +2119,10 @@ static int rproc_alloc_internals(struct rproc *rproc,
> > const char *name,
> >  			return ret;
> >  	}
> > 
> > -	return 0;
> > +	/* Finally allocate the synchronisation state machine */
> > +	ret = rproc_alloc_sync_states(rproc, boot_ops, sync_ops,
> > sync_states);
> > +
> > +	return ret;
> >  }
> > 
> >  /**
> > --
> > 2.20.1
> 

Re: [PATCH v2 08/17] remoteproc: Allocate synchronisation state machine

[Suman Anna]

Hi Mathieu,

On 3/27/20 8:47 AM, Loic PALLARDY wrote:
> 
> 
>> -----Original Message-----
>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
>> Sent: mardi 24 mars 2020 22:46
>> To: bjorn.andersson@linaro.org
>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
>> Subject: [PATCH v2 08/17] remoteproc: Allocate synchronisation state
>> machine
>>
>> This patch allocates a synchronisation state machine, either provided or
>> not by users, in order to enact the proper behavior requested by the
>> platform or specific scenarios.
>>
>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>> ---
>>  drivers/remoteproc/remoteproc_core.c | 59
>> +++++++++++++++++++++++++++-
>>  1 file changed, 58 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/remoteproc/remoteproc_core.c
>> b/drivers/remoteproc/remoteproc_core.c
>> index 02dbb826aa29..1578a9c70422 100644
>> --- a/drivers/remoteproc/remoteproc_core.c
>> +++ b/drivers/remoteproc/remoteproc_core.c
>> @@ -1955,6 +1955,7 @@ static void rproc_type_release(struct device *dev)
>>  	kfree(rproc->firmware);
>>  	kfree(rproc->ops);
>>  	kfree(rproc->sync_ops);
>> +	kfree(rproc->sync_states);
>>  	kfree(rproc);
>>  }
>>
>> @@ -2035,6 +2036,59 @@ static int rproc_alloc_sync_ops(struct rproc *rproc,
>>  	return 0;
>>  }
>>
>> +static int rproc_alloc_sync_states(struct rproc *rproc,
>> +				   const struct rproc_ops *boot_ops,
>> +				   const struct rproc_ops *sync_ops,
>> +				   struct rproc_sync_states *sync_states)
>> +{
>> +	struct rproc_sync_states *st;
>> +
>> +	/* At least one set of operation is needed */
>> +	if (!boot_ops && !sync_ops)
>> +		return -EINVAL;
>> +
>> +	/* We have a synchronisation state machine, no need to build one */
>> +	if (sync_states) {
>> +		st = kmemdup(sync_states, sizeof(*st), GFP_KERNEL);
>> +		if (!st)
>> +			return -ENOMEM;
>> +
> 
> I think a check between sync_states and boot_ops/sync_ops may be needed here
> even if it is platform driver responsibility to provide coherent configuration
> As soon as one of the sync_states is set at true, sync_ops must be provided
> As soon as one of the sync_states is set at false, boot_ops must be provided
> 
> Regards,
> Loic
> 
>> +		/* Nothing else to do */
>> +		goto out;
>> +	}
>> +
>> +	/* Allocate synchronisation state machine */
>> +	st = kzalloc(sizeof(*st), GFP_KERNEL);

Hmm, do you really want to allocate these dynamically? You are
allocating/initializing these no matter what, and I see these as no
different from the likes of has_iommu or auto_boot. Why not just add the
struct as a regular member instead of a pointer?

>> +	if (!st)
>> +		return -ENOMEM;
>> +
>> +	/*
>> +	 * We have a boot_ops and no sync_ops - build a state machine that
>> +	 * does _not_ synchronise with an MCU.
>> +	 */
>> +	if (boot_ops && !sync_ops) {
>> +		st->on_init = st->after_stop = st->after_crash = false;
>> +		goto out;
>> +	}
>> +
>> +	/*
>> +	 * We have a sync_ops and an no boot_ops - build a state machine
>> that
>> +	 * _only_ synchronises with an MCU.
>> +	 */
>> +	if (sync_ops && !boot_ops) {
>> +		st->on_init = st->after_stop = st->after_crash = true;
>> +		goto out;
>> +	}
>> +
>> +out:
>> +	rproc->sync_with_mcu = st->on_init;

This is not needed because of the rproc_set_mcu_sync_state call below.

regards
Suman

>> +	/* And the synchronisation state machine to use */
>> +	rproc->sync_states = st;
>> +	/* Tell the core what to do when initialising */
>> +	rproc_set_mcu_sync_state(rproc, RPROC_SYNC_STATE_INIT);
>> +	return 0;
>> +}
>> +
>>  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
>>  				 const struct rproc_ops *boot_ops,
>>  				 const struct rproc_ops *sync_ops,
>> @@ -2065,7 +2119,10 @@ static int rproc_alloc_internals(struct rproc *rproc,
>> const char *name,
>>  			return ret;
>>  	}
>>
>> -	return 0;
>> +	/* Finally allocate the synchronisation state machine */
>> +	ret = rproc_alloc_sync_states(rproc, boot_ops, sync_ops,
>> sync_states);
>> +
>> +	return ret;
>>  }
>>
>>  /**
>> --
>> 2.20.1
> 

Re: [PATCH v2 15/17] remoteproc: Correctly deal with MCU synchronisation when changing FW image

[Mathieu Poirier]

On Fri, Mar 27, 2020 at 01:50:18PM +0000, Loic PALLARDY wrote:
> 
> 
> > -----Original Message-----
> > From: Mathieu Poirier <mathieu.poirier@linaro.org>
> > Sent: mardi 24 mars 2020 22:46
> > To: bjorn.andersson@linaro.org
> > Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> > anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> > <arnaud.pouliquen@st.com>; Fabien DESSENNE
> > <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> > Subject: [PATCH v2 15/17] remoteproc: Correctly deal with MCU
> > synchronisation when changing FW image
> > 
> > This patch prevents the firmware image from being displayed or changed
> > when
> > the remoteproc core is synchronising with an MCU. This is needed since
> > there is no guarantee about the nature of the firmware image that is loaded
> > by the external entity.
> > 
> > Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> > ---
> >  drivers/remoteproc/remoteproc_sysfs.c | 25
> > ++++++++++++++++++++++++-
> >  1 file changed, 24 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> > b/drivers/remoteproc/remoteproc_sysfs.c
> > index 7f8536b73295..4956577ad4b4 100644
> > --- a/drivers/remoteproc/remoteproc_sysfs.c
> > +++ b/drivers/remoteproc/remoteproc_sysfs.c
> > @@ -13,9 +13,20 @@
> >  static ssize_t firmware_show(struct device *dev, struct device_attribute
> > *attr,
> >  			  char *buf)
> >  {
> > +	ssize_t ret;
> >  	struct rproc *rproc = to_rproc(dev);
> > 
> > -	return sprintf(buf, "%s\n", rproc->firmware);
> > +	/*
> > +	 * In most instances there is no guarantee about the firmware
> > +	 * that was loaded by the external entity.  As such simply don't
> > +	 * print anything.
> > +	 */
> > +	if (rproc_sync_with_mcu(rproc))
> > +		ret = sprintf(buf, "\n");
> Is it enough to provide empty name, or should we add a message to indicate that's name is unkown/undefined ?
>

Don't know... It is easy to find plenty of cases in sysfs where null values are
represented with a "\n", and just as many where "unknown", "undefined" or "-1"
are used. I know GKH prefers the least amount of information as possible, hence
going with a "\n".

Again, no strong opinion...

> Regards,
> Loic
> > +	else
> > +		ret = sprintf(buf, "%s\n", rproc->firmware);
> > +
> > +	return ret;
> >  }
> > 
> >  /* Change firmware name via sysfs */
> > @@ -33,6 +44,18 @@ static ssize_t firmware_store(struct device *dev,
> >  		return -EINVAL;
> >  	}
> > 
> > +	/*
> > +	 * There is no point in trying to change the firmware if the MCU
> > +	 * is currently running or if loading of the image is done by
> > +	 * another entity.
> > +	 */
> > +	if (rproc_sync_with_mcu(rproc)) {
> > +		dev_err(dev,
> > +			"can't change firmware while synchronising with
> > MCU\n");
> > +		err = -EBUSY;
> > +		goto out;
> > +	}
> > +
> >  	if (rproc->state != RPROC_OFFLINE) {
> >  		dev_err(dev, "can't change firmware while running\n");
> >  		err = -EBUSY;
> > --
> > 2.20.1
> 

Re: [PATCH v2 16/17] remoteproc: Correctly deal with MCU synchronisation when changing state

[Mathieu Poirier]

On Fri, Mar 27, 2020 at 02:04:36PM +0000, Loic PALLARDY wrote:
> 
> 
> > -----Original Message-----
> > From: Mathieu Poirier <mathieu.poirier@linaro.org>
> > Sent: mardi 24 mars 2020 22:46
> > To: bjorn.andersson@linaro.org
> > Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> > anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> > <arnaud.pouliquen@st.com>; Fabien DESSENNE
> > <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> > Subject: [PATCH v2 16/17] remoteproc: Correctly deal with MCU
> > synchronisation when changing state
> > 
> > This patch deals with state changes when synchronising with an MCU. More
> > specifically it prevents the MCU from being started if it already has been
> > started by another entity.  Similarly it prevents the AP from stopping the
> > MCU if it hasn't been given the capability by platform firmware.
> > 
> > Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> > ---
> >  drivers/remoteproc/remoteproc_sysfs.c | 32
> > ++++++++++++++++++++++++++-
> >  1 file changed, 31 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> > b/drivers/remoteproc/remoteproc_sysfs.c
> > index 4956577ad4b4..741a3c152b82 100644
> > --- a/drivers/remoteproc/remoteproc_sysfs.c
> > +++ b/drivers/remoteproc/remoteproc_sysfs.c
> > @@ -108,6 +108,29 @@ static ssize_t state_show(struct device *dev, struct
> > device_attribute *attr,
> >  	return sprintf(buf, "%s\n", rproc_state_string[state]);
> >  }
> > 
> > +static int rproc_can_shutdown(struct rproc *rproc)
> > +{
> > +	/* The MCU is not running, obviously an invalid operation. */
> > +	if (rproc->state != RPROC_RUNNING)
> > +		return false;
> > +
> > +	/*
> > +	 * The MCU is not running (see above) and the remoteproc core is
> > the
> > +	 * lifecycle manager, no problem calling for a shutdown.
> > +	 */
> > +	if (!rproc_sync_with_mcu(rproc))
> > +		return true;
> > +
> > +	/*
> > +	 * The MCU has been loaded by another entity (see above) and the
> > +	 * platform code has _not_ given us the capability of stopping it.
> > +	 */
> > +	if (!rproc->sync_ops->stop)
> > +		return false;
> 
> Test could be simplified
> if (rproc_sync_with_mcu(rproc)) && !rproc->sync_ops->stop)
> 	return false;

I laid out the test individually on purpose.  That way there is no coupling
between conditions, it is plane to see what is going on and remains maintainable
as we add new tests.

> 
> > +
> > +	return true;
> > +}
> > +
> >  /* Change remote processor state via sysfs */
> >  static ssize_t state_store(struct device *dev,
> >  			      struct device_attribute *attr,
> > @@ -120,11 +143,18 @@ static ssize_t state_store(struct device *dev,
> >  		if (rproc->state == RPROC_RUNNING)
> >  			return -EBUSY;
> > 
> > +		/*
> > +		 * In synchronisation mode, booting the MCU is the
> > +		 * responsibility of an external entity.
> > +		 */
> > +		if (rproc_sync_with_mcu(rproc))
> > +			return -EINVAL;
> > +
> I don't understand this restriction, simply because it is preventing to resynchronize with a
> coprocessor after a "stop".
> In the following configuration which can be configuration for coprocessor with romed/flashed
> firmware (no reload needed):
> on_init = true
> after_stop = true
> after_crash = true
> Once you stop it via sysfs interface, you can't anymore restart/resync to it.

Very true.  The MCU will get restarted by another entity but the AP won't
synchronise with it.  I need more time to think about the best way to deal with
this and may have to get back to you for further discussions.

> 
> I think it will be better to modify rproc_boot() to take into account rproc_sync_with_mcu()
> as below:
> 
> int rproc_boot(struct rproc *rproc)
>  {
> -	const struct firmware *firmware_p;
> +	const struct firmware *firmware_p = NULL;
>  	struct device *dev = &rproc->dev;
>  	int ret;
>  
>  	if (!rproc) {
>  		pr_err("invalid rproc handle\n");
>  		return -EINVAL;
>  	}
>  
>  	/* load firmware */
> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
> -	if (ret < 0) {
> -		dev_err(dev, "request_firmware failed: %d\n", ret);
> -		return ret;
> +	if (!rproc_sync_with_mcu(rproc)) {
> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
> +		if (ret < 0) {
> +			dev_err(dev, "request_firmware failed: %d\n", ret);
> +			return ret;
> +		}
>  	}
>  
>  	ret = rproc_actuate(rproc, firmware_p);
>  
> -	release_firmware(firmware_p);
> +	if (firmware_p)
> +		release_firmware(firmware_p);
>  
>  	return ret;
>  }
>  
> Thanks to these modifications, I'm able to resync after a stop with coprocessor without reloading firmware.
> 
> >  		ret = rproc_boot(rproc);
> >  		if (ret)
> >  			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
> >  	} else if (sysfs_streq(buf, "stop")) {
> > -		if (rproc->state != RPROC_RUNNING)
> > +		if (!rproc_can_shutdown(rproc))
> >  			return -EINVAL;
> > 
> >  		rproc_shutdown(rproc);
> As rproc shutdown is also accessible as kernel API, I propose to move
> rproc_can_shutdown() check inside rproc_shutdown() and to test
> returned error

Ah yes, it is public...  As you point out, I think we'll need to move
rproc_can_shutdown() in rproc_shutdown().

Thank you for taking the time to review this set,
Mathieu

> 
> Regards,
> Loic
> > --
> > 2.20.1
> 

Re: [PATCH v2 09/17] remoteproc: Call the right core function based on synchronisation state

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> Call the right core function based on whether we should synchronise
> with an MCU or boot it from scratch.

This patch does generate some checkpatch warnings.

> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_internal.h | 36 +++++++++++-------------
>  1 file changed, 17 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
> index 73ea32df0156..5f711ceb97ba 100644
> --- a/drivers/remoteproc/remoteproc_internal.h
> +++ b/drivers/remoteproc/remoteproc_internal.h
> @@ -106,38 +106,41 @@ static inline void rproc_set_mcu_sync_state(struct rproc *rproc,
>  	}
>  }
>  
> +#define RPROC_OPS_HELPER(__operation, ...)				\
> +	do {								\
> +		if (rproc_sync_with_mcu(rproc)) {			\

So this does make the logic a bit convoluted, since you have three
different flags for rproc_sync_with_mcu, and you apply them in common
for all the ops. This is what I meant in my comment on Patch 1.

> +			if (!rproc->sync_ops ||				\
> +			    !rproc->sync_ops->__operation)		\
> +				return 0;				\
> +			return rproc->sync_ops->__operation(__VA_ARGS__); \

Use the same semantics as the regular ops instead of two return
statements, the code should fallback to the common return 0 after the
RPROC_OPS_HELPER when neither of them are present.

regards
Suman

> +		} else if (rproc->ops && rproc->ops->__operation)	\
> +			return rproc->ops->__operation(__VA_ARGS__);	\
> +	} while (0)							\
> +
>  static inline
>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
>  {
> -	if (rproc->ops->sanity_check)
> -		return rproc->ops->sanity_check(rproc, fw);
> -
> +	RPROC_OPS_HELPER(sanity_check, rproc, fw);
>  	return 0;
>  }
>  
>  static inline
>  u32 rproc_get_boot_addr(struct rproc *rproc, const struct firmware *fw)
>  {
> -	if (rproc->ops->get_boot_addr)
> -		return rproc->ops->get_boot_addr(rproc, fw);
> -
> +	RPROC_OPS_HELPER(get_boot_addr, rproc, fw);
>  	return 0;
>  }
>  
>  static inline
>  int rproc_load_segments(struct rproc *rproc, const struct firmware *fw)
>  {
> -	if (rproc->ops->load)
> -		return rproc->ops->load(rproc, fw);
> -
> +	RPROC_OPS_HELPER(load, rproc, fw);
>  	return -EINVAL;
>  }
>  
>  static inline int rproc_parse_fw(struct rproc *rproc, const struct firmware *fw)
>  {
> -	if (rproc->ops->parse_fw)
> -		return rproc->ops->parse_fw(rproc, fw);
> -
> +	RPROC_OPS_HELPER(parse_fw, rproc, fw);
>  	return 0;
>  }
>  
> @@ -145,10 +148,7 @@ static inline
>  int rproc_handle_rsc(struct rproc *rproc, u32 rsc_type, void *rsc, int offset,
>  		     int avail)
>  {
> -	if (rproc->ops->handle_rsc)
> -		return rproc->ops->handle_rsc(rproc, rsc_type, rsc, offset,
> -					      avail);
> -
> +	RPROC_OPS_HELPER(handle_rsc, rproc, rsc_type, rsc, offset, avail);
>  	return RSC_IGNORED;
>  }
>  
> @@ -156,9 +156,7 @@ static inline
>  struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
>  						   const struct firmware *fw)
>  {
> -	if (rproc->ops->find_loaded_rsc_table)
> -		return rproc->ops->find_loaded_rsc_table(rproc, fw);
> -
> +	RPROC_OPS_HELPER(find_loaded_rsc_table, rproc, fw);
>  	return NULL;
>  }
>  
> 

Re: [PATCH v2 13/17] remoteproc: Introducting new functions to start and stop an MCU

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> Add new functions to replace direct calling of rproc->ops->start() and
> rproc->ops->stop().  That way different behaviour can be played out
> when booting an MCU or synchronising with it.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c     |  6 +++---
>  drivers/remoteproc/remoteproc_internal.h | 12 ++++++++++++
>  2 files changed, 15 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> index 488723fcb142..d3c4d7e6ca25 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -1330,7 +1330,7 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
>  	}
>  
>  	/* power up the remote processor */
> -	ret = rproc->ops->start(rproc);
> +	ret = rproc_start_hw(rproc);
>  	if (ret) {
>  		dev_err(dev, "can't start rproc %s: %d\n", rproc->name, ret);
>  		goto unprepare_subdevices;
> @@ -1351,7 +1351,7 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
>  	return 0;
>  
>  stop_rproc:
> -	rproc->ops->stop(rproc);
> +	rproc_stop_hw(rproc);
>  unprepare_subdevices:
>  	rproc_unprepare_subdevices(rproc);
>  reset_table_ptr:
> @@ -1485,7 +1485,7 @@ static int rproc_stop(struct rproc *rproc, bool crashed)
>  	rproc->table_ptr = rproc->cached_table;
>  
>  	/* power off the remote processor */
> -	ret = rproc->ops->stop(rproc);
> +	ret = rproc_stop_hw(rproc);
>  	if (ret) {
>  		dev_err(dev, "can't stop rproc: %d\n", ret);
>  		return ret;
> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
> index 5f711ceb97ba..7ca23d46dfd4 100644
> --- a/drivers/remoteproc/remoteproc_internal.h
> +++ b/drivers/remoteproc/remoteproc_internal.h
> @@ -160,4 +160,16 @@ struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
>  	return NULL;
>  }
>  
> +static inline int rproc_start_hw(struct rproc *rproc)
> +{
> +	RPROC_OPS_HELPER(start, rproc);
> +	return -EINVAL;
> +}
> +
> +static inline int rproc_stop_hw(struct rproc *rproc)
> +{
> +	RPROC_OPS_HELPER(stop, rproc);
> +	return -EINVAL;
> +}

Since we already have the concept of subdevices, how about we call these
rproc_{start/stop}_device?

regards
Suman

> +
>  #endif /* REMOTEPROC_INTERNAL_H */
> 

Re: [PATCH v2 10/17] remoteproc: Decouple firmware load and remoteproc booting

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> In preparation to support scenarios where MCU firmware is loaded and
> the MCU itself booted by another entity, split function rproc_boot()
> to decouple the functions of loading the firwmare and starting the
> initialisation process.  That way we can reuse the functionality
> provided by the current implementation without invariably dealing
> with firmware loading.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 63 +++++++++++++++++-----------
>  1 file changed, 39 insertions(+), 24 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> index 1578a9c70422..7faee1396ef7 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -1714,21 +1714,10 @@ static void rproc_crash_handler_work(struct work_struct *work)
>  		rproc_trigger_recovery(rproc);
>  }
>  
> -/**
> - * rproc_boot() - boot a remote processor
> - * @rproc: handle of a remote processor
> - *
> - * Boot a remote processor (i.e. load its firmware, power it on, ...).
> - *
> - * If the remote processor is already powered on, this function immediately
> - * returns (successfully).
> - *
> - * Returns 0 on success, and an appropriate error value otherwise.
> - */
> -int rproc_boot(struct rproc *rproc)
> +static int rproc_actuate(struct rproc *rproc,
> +			 const struct firmware *firmware_p)

I would recommend to add some documentation above this function given
that you are refactoring, and using it from multiple places with
different arguments and expected behaviors based on the sync states,
with certain behaviors from the underlying ops.

regards
Suman

>  {
> -	const struct firmware *firmware_p;
> -	struct device *dev;
> +	struct device *dev = &rproc->dev;
>  	int ret;
>  
>  	if (!rproc) {
> @@ -1736,8 +1725,6 @@ int rproc_boot(struct rproc *rproc)
>  		return -EINVAL;
>  	}
>  
> -	dev = &rproc->dev;
> -
>  	ret = mutex_lock_interruptible(&rproc->lock);
>  	if (ret) {
>  		dev_err(dev, "can't lock rproc %s: %d\n", rproc->name, ret);
> @@ -1756,24 +1743,52 @@ int rproc_boot(struct rproc *rproc)
>  		goto unlock_mutex;
>  	}
>  
> -	dev_info(dev, "powering up %s\n", rproc->name);
> +	dev_info(dev, "%s %s\n",
> +		 firmware_p ? "powering up" : "syncing with",
> +		 rproc->name);
> +
> +	ret = rproc_fw_boot(rproc, firmware_p);
> +	if (ret)
> +		atomic_dec(&rproc->power);
> +
> +unlock_mutex:
> +	mutex_unlock(&rproc->lock);
> +	return ret;
> +}
> +
> +/**
> + * rproc_boot() - boot a remote processor
> + * @rproc: handle of a remote processor
> + *
> + * Boot a remote processor (i.e. load its firmware, power it on, ...).
> + *
> + * If the remote processor is already powered on, this function immediately
> + * returns (successfully).
> + *
> + * Returns 0 on success, and an appropriate error value otherwise.
> + */
> +int rproc_boot(struct rproc *rproc)
> +{
> +	const struct firmware *firmware_p;
> +	struct device *dev = &rproc->dev;
> +	int ret;
> +
> +	if (!rproc) {
> +		pr_err("invalid rproc handle\n");
> +		return -EINVAL;
> +	}
>  
>  	/* load firmware */
>  	ret = request_firmware(&firmware_p, rproc->firmware, dev);
>  	if (ret < 0) {
>  		dev_err(dev, "request_firmware failed: %d\n", ret);
> -		goto downref_rproc;
> +		return ret;
>  	}
>  
> -	ret = rproc_fw_boot(rproc, firmware_p);
> +	ret = rproc_actuate(rproc, firmware_p);
>  
>  	release_firmware(firmware_p);
>  
> -downref_rproc:
> -	if (ret)
> -		atomic_dec(&rproc->power);
> -unlock_mutex:
> -	mutex_unlock(&rproc->lock);
>  	return ret;
>  }
>  EXPORT_SYMBOL(rproc_boot);
> 

Re: [PATCH v2 11/17] remoteproc: Repurpose function rproc_trigger_auto_boot()

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> Repurpose function rproc_trigger_auto_boot() so that it can deal with
> scenarios where the MCU is already running.  As such give it a new name
> to better represent the capabilities and add a call to rproc_actuate()
> if instructed by the platform code to synchronise with the MCU rather
> than boot it from scratch.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 18 +++++++++++++++---
>  1 file changed, 15 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> index 7faee1396ef7..d57b47b0d6be 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -51,6 +51,8 @@ static int rproc_alloc_carveout(struct rproc *rproc,
>  				struct rproc_mem_entry *mem);
>  static int rproc_release_carveout(struct rproc *rproc,
>  				  struct rproc_mem_entry *mem);
> +static int rproc_actuate(struct rproc *rproc,
> +			 const struct firmware *firmware_p);
>  
>  /* Unique indices for remoteproc devices */
>  static DEFINE_IDA(rproc_dev_index);
> @@ -1444,10 +1446,17 @@ static void rproc_auto_boot_callback(const struct firmware *fw, void *context)
>  	release_firmware(fw);
>  }
>  
> -static int rproc_trigger_auto_boot(struct rproc *rproc)
> +static int rproc_trigger_auto_initiate(struct rproc *rproc)
>  {
>  	int ret;
>  
> +	/*
> +	 * The MCU is already booted, all we need to do is synchronise with it.
> +	 * No point dealing with a firmware image.
> +	 */
> +	if (rproc_sync_with_mcu(rproc))
> +		return rproc_actuate(rproc, NULL);
> +
>  	/*
>  	 * We're initiating an asynchronous firmware loading, so we can
>  	 * be built-in kernel code, without hanging the boot process.
> @@ -1931,9 +1940,12 @@ int rproc_add(struct rproc *rproc)
>  	/* create debugfs entries */
>  	rproc_create_debug_dir(rproc);
>  
> -	/* if rproc is marked always-on, request it to boot */
> +	/*
> +	 * if rproc is marked always-on, request it to boot or synchronise
> +	 * with it.

Can you please rephrase the "always-on" here while at this since are
already modifying this comment.

regards
Suman

> +	 */
>  	if (rproc->auto_boot) {
> -		ret = rproc_trigger_auto_boot(rproc);
> +		ret = rproc_trigger_auto_initiate(rproc);
>  		if (ret < 0)
>  			return ret;
>  	}
> 

Re: [PATCH v2 12/17] remoteproc: Rename function rproc_fw_boot()

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> Renaming function rproc_fw_boot() in order to better reflect the work
> that is done when supporting scenarios where the remoteproc core is
> synchronising with an MCU.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> index d57b47b0d6be..488723fcb142 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -1363,7 +1363,8 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
>  /*
>   * take a firmware and boot a remote processor with it.
>   */
> -static int rproc_fw_boot(struct rproc *rproc, const struct firmware *fw)
> +static int rproc_actuate_platform(struct rproc *rproc,
> +				  const struct firmware *fw)

Perhaps rproc_actuate_device instead of using platform and hw in the
other patch.

>  {
>  	struct device *dev = &rproc->dev;
>  	const char *name = rproc->firmware;
> @@ -1373,7 +1374,9 @@ static int rproc_fw_boot(struct rproc *rproc, const struct firmware *fw)
>  	if (ret)
>  		return ret;
>  
> -	dev_info(dev, "Booting fw image %s, size %zd\n", name, fw->size);
> +	if (!rproc_sync_with_mcu(rproc))
> +		dev_info(dev, "Booting fw image %s, size %zd\n",
> +			 name, fw->size);

Better off moving this to patch 11 since that's where you change the
behavior.

>  
>  	/*
>  	 * if enabling an IOMMU isn't relevant for this rproc, this is
> @@ -1756,7 +1759,7 @@ static int rproc_actuate(struct rproc *rproc,
>  		 firmware_p ? "powering up" : "syncing with",
>  		 rproc->name);

Also, you can move this similar "syncing with" trace from patch 10 to
patch 11.

regards
Suman

>  
> -	ret = rproc_fw_boot(rproc, firmware_p);
> +	ret = rproc_actuate_platform(rproc, firmware_p);
>  	if (ret)
>  		atomic_dec(&rproc->power);
>  
> 

Re: [PATCH v2 13/17] remoteproc: Introducting new functions to start and stop an MCU

[Suman Anna]

On 3/31/20 1:08 PM, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/24/20 4:45 PM, Mathieu Poirier wrote:
>> Add new functions to replace direct calling of rproc->ops->start() and
>> rproc->ops->stop().  That way different behaviour can be played out
>> when booting an MCU or synchronising with it.
>>
>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>> ---
>>  drivers/remoteproc/remoteproc_core.c     |  6 +++---
>>  drivers/remoteproc/remoteproc_internal.h | 12 ++++++++++++
>>  2 files changed, 15 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
>> index 488723fcb142..d3c4d7e6ca25 100644
>> --- a/drivers/remoteproc/remoteproc_core.c
>> +++ b/drivers/remoteproc/remoteproc_core.c
>> @@ -1330,7 +1330,7 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
>>  	}
>>  
>>  	/* power up the remote processor */
>> -	ret = rproc->ops->start(rproc);
>> +	ret = rproc_start_hw(rproc);
>>  	if (ret) {
>>  		dev_err(dev, "can't start rproc %s: %d\n", rproc->name, ret);
>>  		goto unprepare_subdevices;
>> @@ -1351,7 +1351,7 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
>>  	return 0;
>>  
>>  stop_rproc:
>> -	rproc->ops->stop(rproc);
>> +	rproc_stop_hw(rproc);
>>  unprepare_subdevices:
>>  	rproc_unprepare_subdevices(rproc);
>>  reset_table_ptr:
>> @@ -1485,7 +1485,7 @@ static int rproc_stop(struct rproc *rproc, bool crashed)
>>  	rproc->table_ptr = rproc->cached_table;
>>  
>>  	/* power off the remote processor */
>> -	ret = rproc->ops->stop(rproc);
>> +	ret = rproc_stop_hw(rproc);
>>  	if (ret) {
>>  		dev_err(dev, "can't stop rproc: %d\n", ret);
>>  		return ret;
>> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
>> index 5f711ceb97ba..7ca23d46dfd4 100644
>> --- a/drivers/remoteproc/remoteproc_internal.h
>> +++ b/drivers/remoteproc/remoteproc_internal.h
>> @@ -160,4 +160,16 @@ struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
>>  	return NULL;
>>  }
>>  
>> +static inline int rproc_start_hw(struct rproc *rproc)
>> +{
>> +	RPROC_OPS_HELPER(start, rproc);
>> +	return -EINVAL;
>> +}
>> +
>> +static inline int rproc_stop_hw(struct rproc *rproc)
>> +{
>> +	RPROC_OPS_HELPER(stop, rproc);
>> +	return -EINVAL;
>> +}
> 
> Since we already have the concept of subdevices, how about we call these
> rproc_{start/stop}_device?

Actually, does this patch needs to be moved up in the order atleast
prior to patch 11, may be after patch 9?

regards
Suman

> 
>> +
>>  #endif /* REMOTEPROC_INTERNAL_H */
>>
> 

Re: [PATCH v2 14/17] remoteproc: Refactor function rproc_trigger_recovery()

[Suman Anna]

Hi Mathieu,

On 3/24/20 4:46 PM, Mathieu Poirier wrote:
> Refactor function rproc_trigger_recovery() in order to avoid
> reloading the fw image when synchronising with an MCU rather than
> booting it.
> 
> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> ---
>  drivers/remoteproc/remoteproc_core.c | 16 +++++++++-------
>  1 file changed, 9 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> index d3c4d7e6ca25..dbb0a8467205 100644
> --- a/drivers/remoteproc/remoteproc_core.c
> +++ b/drivers/remoteproc/remoteproc_core.c
> @@ -1661,7 +1661,7 @@ static void rproc_coredump(struct rproc *rproc)
>   */
>  int rproc_trigger_recovery(struct rproc *rproc)
>  {
> -	const struct firmware *firmware_p;
> +	const struct firmware *firmware_p = NULL;
>  	struct device *dev = &rproc->dev;
>  	int ret;
>  
> @@ -1678,14 +1678,16 @@ int rproc_trigger_recovery(struct rproc *rproc)
>  	/* generate coredump */
>  	rproc_coredump(rproc);
>  
> -	/* load firmware */
> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
> -	if (ret < 0) {
> -		dev_err(dev, "request_firmware failed: %d\n", ret);
> -		goto unlock_mutex;
> +	/* load firmware if need be */
> +	if (!rproc_sync_with_mcu(rproc)) {
> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
> +		if (ret < 0) {
> +			dev_err(dev, "request_firmware failed: %d\n", ret);
> +			goto unlock_mutex;
> +		}

So, I am trying to understand the need for the flag around
RPROC_SYNC_STATE_CRASHED. Can you explain what all usecases that is
covering?

In anycase, you should probably combine this piece with the flag change
for STATE_CRASHED on the last patch.

regards
Suman

>  	}
>  
> -	/* boot the remote processor up again */
> +	/* boot up or synchronise with the remote processor again */
>  	ret = rproc_start(rproc, firmware_p);
>  
>  	release_firmware(firmware_p);
> 

Re: [PATCH v2 15/17] remoteproc: Correctly deal with MCU synchronisation when changing FW image

[Suman Anna]

Hi Mathieu,

On 3/30/20 6:21 PM, Mathieu Poirier wrote:
> On Fri, Mar 27, 2020 at 01:50:18PM +0000, Loic PALLARDY wrote:
>>
>>> This patch prevents the firmware image from being displayed or changed
>>> when
>>> the remoteproc core is synchronising with an MCU. This is needed since
>>> there is no guarantee about the nature of the firmware image that is loaded
>>> by the external entity.
>>>
>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>> ---
>>>  drivers/remoteproc/remoteproc_sysfs.c | 25
>>> ++++++++++++++++++++++++-
>>>  1 file changed, 24 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
>>> b/drivers/remoteproc/remoteproc_sysfs.c
>>> index 7f8536b73295..4956577ad4b4 100644
>>> --- a/drivers/remoteproc/remoteproc_sysfs.c
>>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
>>> @@ -13,9 +13,20 @@
>>>  static ssize_t firmware_show(struct device *dev, struct device_attribute
>>> *attr,
>>>  			  char *buf)
>>>  {
>>> +	ssize_t ret;
>>>  	struct rproc *rproc = to_rproc(dev);
>>>
>>> -	return sprintf(buf, "%s\n", rproc->firmware);
>>> +	/*
>>> +	 * In most instances there is no guarantee about the firmware
>>> +	 * that was loaded by the external entity.  As such simply don't
>>> +	 * print anything.
>>> +	 */
>>> +	if (rproc_sync_with_mcu(rproc))
>>> +		ret = sprintf(buf, "\n");
>> Is it enough to provide empty name, or should we add a message to indicate that's name is unkown/undefined ?
>>
> 
> Don't know... It is easy to find plenty of cases in sysfs where null values are
> represented with a "\n", and just as many where "unknown", "undefined" or "-1"
> are used. I know GKH prefers the least amount of information as possible, hence
> going with a "\n".
> 
> Again, no strong opinion...
> 
>> Regards,
>> Loic
>>> +	else
>>> +		ret = sprintf(buf, "%s\n", rproc->firmware);
>>> +
>>> +	return ret;
>>>  }
>>>
>>>  /* Change firmware name via sysfs */
>>> @@ -33,6 +44,18 @@ static ssize_t firmware_store(struct device *dev,
>>>  		return -EINVAL;
>>>  	}
>>>
>>> +	/*
>>> +	 * There is no point in trying to change the firmware if the MCU
>>> +	 * is currently running or if loading of the image is done by
>>> +	 * another entity.
>>> +	 */
>>> +	if (rproc_sync_with_mcu(rproc)) {
>>> +		dev_err(dev,
>>> +			"can't change firmware while synchronising with
>>> MCU\n");
>>> +		err = -EBUSY;
>>> +		goto out;
>>> +	}
>>> +

So, I have done a patch sometime back to deny sysfs operations [1] (the
primary usecase is for a rproc-client driver driven boot where auto-boot
is not set) which is still a need for me. Do you see that as orthogonal
to that, or can we leverage that here somehow. I cannot use the sync_
conditions for my cases since they are not already booted before.

Also, any reason why you want to do this check before the rproc->state
unlike the logic around the 'state' file in the next patch?

[1] https://patchwork.kernel.org/patch/10601325/

regards
Suman

>>>  	if (rproc->state != RPROC_OFFLINE) {
>>>  		dev_err(dev, "can't change firmware while running\n");
>>>  		err = -EBUSY;
>>> --
>>> 2.20.1
>>

Re: [PATCH v2 16/17] remoteproc: Correctly deal with MCU synchronisation when changing state

[Suman Anna]

Hi Mathieu,

On 3/30/20 6:49 PM, Mathieu Poirier wrote:
> On Fri, Mar 27, 2020 at 02:04:36PM +0000, Loic PALLARDY wrote:
>>
>>
>>> -----Original Message-----
>>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
>>> Sent: mardi 24 mars 2020 22:46
>>> To: bjorn.andersson@linaro.org
>>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
>>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
>>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
>>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
>>> Subject: [PATCH v2 16/17] remoteproc: Correctly deal with MCU
>>> synchronisation when changing state
>>>
>>> This patch deals with state changes when synchronising with an MCU. More
>>> specifically it prevents the MCU from being started if it already has been
>>> started by another entity.  Similarly it prevents the AP from stopping the
>>> MCU if it hasn't been given the capability by platform firmware.
>>>
>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>> ---
>>>  drivers/remoteproc/remoteproc_sysfs.c | 32
>>> ++++++++++++++++++++++++++-
>>>  1 file changed, 31 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
>>> b/drivers/remoteproc/remoteproc_sysfs.c
>>> index 4956577ad4b4..741a3c152b82 100644
>>> --- a/drivers/remoteproc/remoteproc_sysfs.c
>>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
>>> @@ -108,6 +108,29 @@ static ssize_t state_show(struct device *dev, struct
>>> device_attribute *attr,
>>>  	return sprintf(buf, "%s\n", rproc_state_string[state]);
>>>  }
>>>
>>> +static int rproc_can_shutdown(struct rproc *rproc)
>>> +{
>>> +	/* The MCU is not running, obviously an invalid operation. */
>>> +	if (rproc->state != RPROC_RUNNING)
>>> +		return false;
>>> +
>>> +	/*
>>> +	 * The MCU is not running (see above) and the remoteproc core is
>>> the
>>> +	 * lifecycle manager, no problem calling for a shutdown.
>>> +	 */
>>> +	if (!rproc_sync_with_mcu(rproc))
>>> +		return true;
>>> +
>>> +	/*
>>> +	 * The MCU has been loaded by another entity (see above) and the
>>> +	 * platform code has _not_ given us the capability of stopping it.
>>> +	 */
>>> +	if (!rproc->sync_ops->stop)
>>> +		return false;
>>
>> Test could be simplified
>> if (rproc_sync_with_mcu(rproc)) && !rproc->sync_ops->stop)
>> 	return false;
> 
> I laid out the test individually on purpose.  That way there is no coupling
> between conditions, it is plane to see what is going on and remains maintainable
> as we add new tests.
> 
>>
>>> +
>>> +	return true;
>>> +}
>>> +
>>>  /* Change remote processor state via sysfs */
>>>  static ssize_t state_store(struct device *dev,
>>>  			      struct device_attribute *attr,
>>> @@ -120,11 +143,18 @@ static ssize_t state_store(struct device *dev,
>>>  		if (rproc->state == RPROC_RUNNING)
>>>  			return -EBUSY;
>>>
>>> +		/*
>>> +		 * In synchronisation mode, booting the MCU is the
>>> +		 * responsibility of an external entity.
>>> +		 */
>>> +		if (rproc_sync_with_mcu(rproc))
>>> +			return -EINVAL;
>>> +
>> I don't understand this restriction, simply because it is preventing to resynchronize with a
>> coprocessor after a "stop".

There's actually one more scenario even without "stop". If auto_boot is
set to false, then rproc_actuate will never get called.

>> In the following configuration which can be configuration for coprocessor with romed/flashed
>> firmware (no reload needed):
>> on_init = true
>> after_stop = true
>> after_crash = true
>> Once you stop it via sysfs interface, you can't anymore restart/resync to it.
> 
> Very true.  The MCU will get restarted by another entity but the AP won't
> synchronise with it.  I need more time to think about the best way to deal with
> this and may have to get back to you for further discussions.
> 
>>
>> I think it will be better to modify rproc_boot() to take into account rproc_sync_with_mcu()
>> as below:
>>
>> int rproc_boot(struct rproc *rproc)
>>  {
>> -	const struct firmware *firmware_p;
>> +	const struct firmware *firmware_p = NULL;
>>  	struct device *dev = &rproc->dev;
>>  	int ret;
>>  
>>  	if (!rproc) {
>>  		pr_err("invalid rproc handle\n");
>>  		return -EINVAL;
>>  	}
>>  
>>  	/* load firmware */
>> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
>> -	if (ret < 0) {
>> -		dev_err(dev, "request_firmware failed: %d\n", ret);
>> -		return ret;
>> +	if (!rproc_sync_with_mcu(rproc)) {

I guess this is what the original skip_fw_load was doing. And with the
current series, the userspace loading support usecase I have cannot be
achieved. If this is added back, I can try if that works for my usecases.

>> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
>> +		if (ret < 0) {
>> +			dev_err(dev, "request_firmware failed: %d\n", ret);
>> +			return ret;
>> +		}
>>  	}
>>  
>>  	ret = rproc_actuate(rproc, firmware_p);
>>  
>> -	release_firmware(firmware_p);
>> +	if (firmware_p)
>> +		release_firmware(firmware_p);
>>  
>>  	return ret;
>>  }
>>  
>> Thanks to these modifications, I'm able to resync after a stop with coprocessor without reloading firmware.
>>
>>>  		ret = rproc_boot(rproc);
>>>  		if (ret)
>>>  			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
>>>  	} else if (sysfs_streq(buf, "stop")) {
>>> -		if (rproc->state != RPROC_RUNNING)
>>> +		if (!rproc_can_shutdown(rproc))
>>>  			return -EINVAL;
>>>
>>>  		rproc_shutdown(rproc);
>> As rproc shutdown is also accessible as kernel API, I propose to move
>> rproc_can_shutdown() check inside rproc_shutdown() and to test
>> returned error
> 
> Ah yes, it is public...  As you point out, I think we'll need to move
> rproc_can_shutdown() in rproc_shutdown().

I am assuming only the new conditions, right?

regards
Suman

> 
> Thank you for taking the time to review this set,
> Mathieu
> 

Re: [PATCH v2 00/17] remoteproc: Add support for synchronisation with MCU

[Suman Anna]

Hi Mathieu,

On 3/27/20 12:20 PM, Loic PALLARDY wrote:
> Hi Mathieu,
> 
>>
>> This is the second revision of this set that tries to address the
>> problem of synchronising with an MCU with as much flexibility as
>> possible.
>>
>> New in this revision is a fix for a couple of bugs I found while
>> testing things.  First with the helper macro in patch 09 and the
>> suppression of a boot message when synchronising with an MCU
>> in patch 12.  I have completely removed what used to be patch 18,
>> the example on how to use the new API.  This will be the subject
>> of an upcoming patchset.
>>
>> Tested on ST's mp157c platform.  Applies on v5.6-rc7 to keep things
>> simple.
> 
> Thanks Mathieu for the 2 series. I tested on my STM32MP157-DK2 the different
> synchronization use cases (on_init, after_stop, after_crash), mixing the values and
> I succeed to start/stop/restart M4 coprocessor with or without reloading firmware
> according to sync values. (I only applied the correction I proposed in patch 16 review
> to allow to resync with a preloaded or an already running coprocessor.
> 
> Regards,
> Loic
>>
>> Comments would be much appreciated.

Thank you for the enhanced series to implement the logic in remoteproc
core. I have provided my comments on most of the patches.

Overall, I can see my early-boot scenarios work with the series, and the
slightly different userspace-loading support usecase would need some
additional support.

As I commented on patch 1 in v1, I would rather reuse the the generic
"rproc" instead of adding a new "mcu" terminology to code.. Let's just
stick with the rproc

Another thing I would prefer (echoing my comments on patch 7) is to just
use an API for modifying the sync states, that can be used between
rproc_alloc() and rproc_add(). The state-machine really doesn't kick in
until rproc_add() is invoked. The memory for the driver private rproc
structure is allocated using rproc_alloc() normally, and most of the
DT-parsing in platform drivers is generally done directly into this
allocated memory. I see it a bit cumbersome having to maintain a
separate structure, and then do a memcpy, especially given that the
rproc_alloc_state_machine() logic requires that you detect the state
before calling the rproc_alloc().

regards
Suman

>>
>> Thanks,
>> Mathieu
>>
>> Mathieu Poirier (17):
>>   remoteproc: Add new operation and state machine for MCU
>>     synchronisation
>>   remoteproc: Introduce function rproc_set_mcu_sync_state()
>>   remoteproc: Split firmware name allocation from rproc_alloc()
>>   remoteproc: Split rproc_ops allocation from rproc_alloc()
>>   remoteproc: Get rid of tedious error path
>>   remoteproc: Introduce function rproc_alloc_internals()
>>   remoteproc: Introduce function rproc_alloc_state_machine()
>>   remoteproc: Allocate synchronisation state machine
>>   remoteproc: Call the right core function based on synchronisation
>>     state
>>   remoteproc: Decouple firmware load and remoteproc booting
>>   remoteproc: Repurpose function rproc_trigger_auto_boot()
>>   remoteproc: Rename function rproc_fw_boot()
>>   remoteproc: Introducting new functions to start and stop an MCU
>>   remoteproc: Refactor function rproc_trigger_recovery()
>>   remoteproc: Correctly deal with MCU synchronisation when changing FW
>>     image
>>   remoteproc: Correctly deal with MCU synchronisation when changing
>>     state
>>   remoteproc: Make MCU synchronisation state changes on stop and crashed
>>
>>  drivers/remoteproc/remoteproc_core.c     | 387 ++++++++++++++++++-----
>>  drivers/remoteproc/remoteproc_debugfs.c  |  31 ++
>>  drivers/remoteproc/remoteproc_internal.h |  91 ++++--
>>  drivers/remoteproc/remoteproc_sysfs.c    |  57 +++-
>>  include/linux/remoteproc.h               |  28 +-
>>  5 files changed, 487 insertions(+), 107 deletions(-)
>>
>> --
>> 2.20.1
> 

Re: [PATCH v2 06/17] remoteproc: Introduce function rproc_alloc_internals()

[Mathieu Poirier]

Hi Suman,

On Mon, Mar 30, 2020 at 03:38:14PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/27/20 6:10 AM, Loic PALLARDY wrote:
> > Hi Mathieu,
> > 
> >>
> >> In preparation to allocate the synchronisation operation and state
> >> machine, spin off a new function in order to keep rproc_alloc() as
> >> clean as possible.
> >>
> >> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> ---
> >>  drivers/remoteproc/remoteproc_core.c | 26 ++++++++++++++++++++++---
> >> -
> >>  1 file changed, 22 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/drivers/remoteproc/remoteproc_core.c
> >> b/drivers/remoteproc/remoteproc_core.c
> >> index ee277bc5556c..9da245734db6 100644
> >> --- a/drivers/remoteproc/remoteproc_core.c
> >> +++ b/drivers/remoteproc/remoteproc_core.c
> >> @@ -2018,6 +2018,26 @@ static int rproc_alloc_ops(struct rproc *rproc,
> >> const struct rproc_ops *ops)
> >>  	return 0;
> >>  }
> >>
> >> +static int rproc_alloc_internals(struct rproc *rproc, const char *name,
> >> +				 const struct rproc_ops *boot_ops,
> >> +				 const char *firmware, int len)
> > 
> > len argument is not used in the patch nor in the following, maybe removed from my pov.
> 
> Indeed.
> 
> > 
> > Regards,
> > Loic
> 
> >> +{
> >> +	int ret;
> >> +
> >> +	/* We have a boot_ops so allocate firmware name and operations */
> >> +	if (boot_ops) {
> >> +		ret = rproc_alloc_firmware(rproc, name, firmware);
> >> +		if (ret)
> >> +			return ret;
> 
> So, can you explain why firmware allocation now becomes conditional on
> this boot_ops?

There is no point in allocating a firmware name in a scenario where the
remoteproc core is only synchronising with the MCU.  As soon as a boot_ops (to
be renamed ops as per your comment below) is present I assume firmware loading
will be involved at some point.   Do you see a scenario where that wouldn't be
be case?

> 
> Perhaps, continue to call this as ops following the field name in struct
> rproc.

Ok

> 
> regards
> Suman
> 
> >> +
> >> +		ret = rproc_alloc_ops(rproc, boot_ops);
> >> +		if (ret)
> >> +			return ret;
> >> +	}
> >> +
> >> +	return 0;
> >> +}
> >> +
> >>  /**
> >>   * rproc_alloc() - allocate a remote processor handle
> >>   * @dev: the underlying device
> >> @@ -2064,10 +2084,8 @@ struct rproc *rproc_alloc(struct device *dev, const
> >> char *name,
> >>  	rproc->dev.class = &rproc_class;
> >>  	rproc->dev.driver_data = rproc;
> >>
> >> -	if (rproc_alloc_firmware(rproc, name, firmware))
> >> -		goto out;
> >> -
> >> -	if (rproc_alloc_ops(rproc, ops))
> >> +	if (rproc_alloc_internals(rproc, name, ops,
> >> +				  firmware, len))
> >>  		goto out;
> >>
> >>  	/* Assign a unique device index and name */
> >> --
> >> 2.20.1
> > 
> 

Re: [PATCH v2 07/17] remoteproc: Introduce function rproc_alloc_state_machine()

[Mathieu Poirier]

On Mon, Mar 30, 2020 at 06:10:51PM -0500, Suman Anna wrote:
> Hi Mathieu,
> > 
> >> -----Original Message-----
> >> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> Sent: mardi 24 mars 2020 22:46
> >> To: bjorn.andersson@linaro.org
> >> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> >> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> >> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> >> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> >> Subject: [PATCH v2 07/17] remoteproc: Introduce function
> >> rproc_alloc_state_machine()
> >>
> >> Introducing new function rproc_alloc_state_machine() to allocate
> >> the MCU synchronisation operations and position it as the central
> >> remoteproc core allocation function.
> >>
> >> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> ---
> >>  drivers/remoteproc/remoteproc_core.c | 84
> >> +++++++++++++++++++++++++---
> >>  include/linux/remoteproc.h           |  5 ++
> >>  2 files changed, 81 insertions(+), 8 deletions(-)
> >>
> >> diff --git a/drivers/remoteproc/remoteproc_core.c
> >> b/drivers/remoteproc/remoteproc_core.c
> >> index 9da245734db6..02dbb826aa29 100644
> >> --- a/drivers/remoteproc/remoteproc_core.c
> >> +++ b/drivers/remoteproc/remoteproc_core.c
> >> @@ -1954,6 +1954,7 @@ static void rproc_type_release(struct device *dev)
> >>
> >>  	kfree(rproc->firmware);
> >>  	kfree(rproc->ops);
> >> +	kfree(rproc->sync_ops);
> >>  	kfree(rproc);
> >>  }
> >>
> >> @@ -2018,12 +2019,34 @@ static int rproc_alloc_ops(struct rproc *rproc,
> >> const struct rproc_ops *ops)
> >>  	return 0;
> >>  }
> >>
> >> +static int rproc_alloc_sync_ops(struct rproc *rproc,
> >> +				const struct rproc_ops *sync_ops)
> >> +{
> >> +	/*
> >> +	 * Given the unlimited amount of possibilities when
> >> +	 * synchronising with an MCU, no constraints are imposed
> >> +	 * on sync_ops.
> >> +	 */
> >> +	rproc->sync_ops = kmemdup(sync_ops,
> >> +				  sizeof(*sync_ops), GFP_KERNEL);
> >> +	if (!rproc->sync_ops)
> >> +		return -ENOMEM;
> > Should we check a minimal set of functions in sync_ops to be required?
> > Or we should consider all pointers at NULL is ok ?
> > 
> >> +
> >> +	return 0;
> >> +}
> >> +
> >>  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
> >>  				 const struct rproc_ops *boot_ops,
> >> +				 const struct rproc_ops *sync_ops,
> >> +				 struct rproc_sync_states *sync_states,
> > sync_states not used in this patch, should be introduced in patch 8
> 
> +1

I will do that.

> 
> > 
> > Regards,
> > Loic
> > 
> >>  				 const char *firmware, int len)
> >>  {
> >>  	int ret;
> >>
> >> +	/* We need at least a boot or a sync ops. */
> >> +	if (!boot_ops && !sync_ops)
> >> +		return -EINVAL;
> >> +
> >>  	/* We have a boot_ops so allocate firmware name and operations */
> >>  	if (boot_ops) {
> >>  		ret = rproc_alloc_firmware(rproc, name, firmware);
> >> @@ -2035,14 +2058,23 @@ static int rproc_alloc_internals(struct rproc
> >> *rproc, const char *name,
> >>  			return ret;
> >>  	}
> >>
> >> +	/* Allocate a sync_ops if need be */
> >> +	if (sync_ops) {
> >> +		ret = rproc_alloc_sync_ops(rproc, sync_ops);
> >> +		if (ret)
> >> +			return ret;
> >> +	}
> >> +
> >>  	return 0;
> >>  }
> >>
> >>  /**
> >> - * rproc_alloc() - allocate a remote processor handle
> >> + * rproc_alloc_state_machine() - allocate a remote processor handle
> >>   * @dev: the underlying device
> >>   * @name: name of this remote processor
> >>   * @ops: platform-specific handlers (mainly start/stop)
> >> + * @sync_ops: platform-specific handlers for synchronising with MCU
> >> + * @sync_states: states in which @ops and @sync_ops are to be used
> >>   * @firmware: name of firmware file to load, can be NULL
> >>   * @len: length of private data needed by the rproc driver (in bytes)
> >>   *
> >> @@ -2061,13 +2093,15 @@ static int rproc_alloc_internals(struct rproc
> >> *rproc, const char *name,
> >>   * Note: _never_ directly deallocate @rproc, even if it was not registered
> >>   * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
> >>   */
> >> -struct rproc *rproc_alloc(struct device *dev, const char *name,
> >> -			  const struct rproc_ops *ops,
> >> -			  const char *firmware, int len)
> >> +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
> >> *name,
> >> +					const struct rproc_ops *ops,
> >> +					const struct rproc_ops *sync_ops,
> >> +					struct rproc_sync_states
> >> *sync_states,
> >> +					const char *firmware, int len)
> 
> Do you foresee the need for sync_ops to be present as long as the rproc
> is registered? I am wondering if it is better to introduce an API where
> you can set the ops at runtime rather than allocate it upfront, so that
> once the initial handling is done, you can reset both the sync_states
> and ops.

That is something I spent a fair amount of time thinking about.  I decided to
proceed with an upfront allocation scheme and see if people would come up
with needs that would mandate a runtime allocation.  I am willing to provide the
API if there is a need for it but would rather not if someone "may" need it. 

> 
> 
> >>  {
> >>  	struct rproc *rproc;
> >>
> >> -	if (!dev || !name || !ops)
> >> +	if (!dev || !name)
> >>  		return NULL;
> >>
> >>  	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
> >> @@ -2084,8 +2118,8 @@ struct rproc *rproc_alloc(struct device *dev, const
> >> char *name,
> >>  	rproc->dev.class = &rproc_class;
> >>  	rproc->dev.driver_data = rproc;
> >>
> >> -	if (rproc_alloc_internals(rproc, name, ops,
> >> -				  firmware, len))
> >> +	if (rproc_alloc_internals(rproc, name, ops, sync_ops,
> >> +				  sync_states, firmware, len))
> >>  		goto out;
> >>
> >>  	/* Assign a unique device index and name */
> >> @@ -2119,7 +2153,41 @@ struct rproc *rproc_alloc(struct device *dev, const
> >> char *name,
> >>  	put_device(&rproc->dev);
> >>  	return NULL;
> >>  }
> >> -EXPORT_SYMBOL(rproc_alloc);
> >> +EXPORT_SYMBOL(rproc_alloc_state_machine);
> >> +
> >> +/**
> >> + * rproc_alloc() - allocate a remote processor handle
> >> + * @dev: the underlying device
> >> + * @name: name of this remote processor
> >> + * @ops: platform-specific handlers (mainly start/stop)
> >> + * @firmware: name of firmware file to load, can be NULL
> >> + * @len: length of private data needed by the rproc driver (in bytes)
> >> + *
> >> + * Allocates a new remote processor handle, but does not register
> >> + * it yet. if @firmware is NULL, a default name is used.
> >> + *
> >> + * This function should be used by rproc implementations during
> >> initialization
> >> + * of the remote processor.
> >> + *
> >> + * After creating an rproc handle using this function, and when ready,
> >> + * implementations should then call rproc_add() to complete
> >> + * the registration of the remote processor.
> >> + *
> >> + * On success the new rproc is returned, and on failure, NULL.
> >> + *
> >> + * Note: _never_ directly deallocate @rproc, even if it was not registered
> >> + * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
> >> + */
> >> +struct rproc *rproc_alloc(struct device *dev, const char *name,
> >> +			  const struct rproc_ops *ops,
> >> +			  const char *firmware, int len)
> >> +{
> >> +	if (!name && !firmware)
> 
> Retain the original checks on dev, name and ops from the previous
> rproc_alloc(). A NULL firmware was perfectly valid before, and the name
> is allocated using the default template.

Here firmware can be NULL, but we need a name in order to construct the default
one.  On the flip side ops can be NULL for scenarios where the remoteproc core
is never in charge of the MCU lifecycle.  As for dev, it is checked in
rproc_alloc_state_machine().

> 
> >> +		return NULL;
> >> +
> >> +	return rproc_alloc_state_machine(dev, name, ops, NULL, NULL,
> >> +					 firmware, len);
> >> +}
> 
> Missing the EXPORT_SYMBOL on rproc_alloc() -> it is an API used by modules.

Of course yes, good catch.

> 
> regards
> Suman
> 
> >>
> >>  /**
> >>   * rproc_free() - unroll rproc_alloc()
> >> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
> >> index d115e47d702d..d1214487daac 100644
> >> --- a/include/linux/remoteproc.h
> >> +++ b/include/linux/remoteproc.h
> >> @@ -611,6 +611,11 @@ struct rproc *rproc_get_by_child(struct device
> >> *dev);
> >>  struct rproc *rproc_alloc(struct device *dev, const char *name,
> >>  			  const struct rproc_ops *ops,
> >>  			  const char *firmware, int len);
> >> +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
> >> *name,
> >> +					const struct rproc_ops *ops,
> >> +					const struct rproc_ops *sync_ops,
> >> +					struct rproc_sync_states
> >> *sync_states,
> >> +					const char *firmware, int len);
> >>  void rproc_put(struct rproc *rproc);
> >>  int rproc_add(struct rproc *rproc);
> >>  int rproc_del(struct rproc *rproc);
> >> --
> >> 2.20.1
> > 
> 

Re: [PATCH v2 08/17] remoteproc: Allocate synchronisation state machine

[Mathieu Poirier]

On Mon, Mar 30, 2020 at 06:20:37PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/27/20 8:47 AM, Loic PALLARDY wrote:
> > 
> > 
> >> -----Original Message-----
> >> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> Sent: mardi 24 mars 2020 22:46
> >> To: bjorn.andersson@linaro.org
> >> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> >> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> >> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> >> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> >> Subject: [PATCH v2 08/17] remoteproc: Allocate synchronisation state
> >> machine
> >>
> >> This patch allocates a synchronisation state machine, either provided or
> >> not by users, in order to enact the proper behavior requested by the
> >> platform or specific scenarios.
> >>
> >> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> ---
> >>  drivers/remoteproc/remoteproc_core.c | 59
> >> +++++++++++++++++++++++++++-
> >>  1 file changed, 58 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/remoteproc/remoteproc_core.c
> >> b/drivers/remoteproc/remoteproc_core.c
> >> index 02dbb826aa29..1578a9c70422 100644
> >> --- a/drivers/remoteproc/remoteproc_core.c
> >> +++ b/drivers/remoteproc/remoteproc_core.c
> >> @@ -1955,6 +1955,7 @@ static void rproc_type_release(struct device *dev)
> >>  	kfree(rproc->firmware);
> >>  	kfree(rproc->ops);
> >>  	kfree(rproc->sync_ops);
> >> +	kfree(rproc->sync_states);
> >>  	kfree(rproc);
> >>  }
> >>
> >> @@ -2035,6 +2036,59 @@ static int rproc_alloc_sync_ops(struct rproc *rproc,
> >>  	return 0;
> >>  }
> >>
> >> +static int rproc_alloc_sync_states(struct rproc *rproc,
> >> +				   const struct rproc_ops *boot_ops,
> >> +				   const struct rproc_ops *sync_ops,
> >> +				   struct rproc_sync_states *sync_states)
> >> +{
> >> +	struct rproc_sync_states *st;
> >> +
> >> +	/* At least one set of operation is needed */
> >> +	if (!boot_ops && !sync_ops)
> >> +		return -EINVAL;
> >> +
> >> +	/* We have a synchronisation state machine, no need to build one */
> >> +	if (sync_states) {
> >> +		st = kmemdup(sync_states, sizeof(*st), GFP_KERNEL);
> >> +		if (!st)
> >> +			return -ENOMEM;
> >> +
> > 
> > I think a check between sync_states and boot_ops/sync_ops may be needed here
> > even if it is platform driver responsibility to provide coherent configuration
> > As soon as one of the sync_states is set at true, sync_ops must be provided
> > As soon as one of the sync_states is set at false, boot_ops must be provided
> > 
> > Regards,
> > Loic
> > 
> >> +		/* Nothing else to do */
> >> +		goto out;
> >> +	}
> >> +
> >> +	/* Allocate synchronisation state machine */
> >> +	st = kzalloc(sizeof(*st), GFP_KERNEL);
> 
> Hmm, do you really want to allocate these dynamically? You are
> allocating/initializing these no matter what, and I see these as no
> different from the likes of has_iommu or auto_boot. Why not just add the
> struct as a regular member instead of a pointer?


That's a valid point.

> 
> >> +	if (!st)
> >> +		return -ENOMEM;
> >> +
> >> +	/*
> >> +	 * We have a boot_ops and no sync_ops - build a state machine that
> >> +	 * does _not_ synchronise with an MCU.
> >> +	 */
> >> +	if (boot_ops && !sync_ops) {
> >> +		st->on_init = st->after_stop = st->after_crash = false;
> >> +		goto out;
> >> +	}
> >> +
> >> +	/*
> >> +	 * We have a sync_ops and an no boot_ops - build a state machine
> >> that
> >> +	 * _only_ synchronises with an MCU.
> >> +	 */
> >> +	if (sync_ops && !boot_ops) {
> >> +		st->on_init = st->after_stop = st->after_crash = true;
> >> +		goto out;
> >> +	}
> >> +
> >> +out:
> >> +	rproc->sync_with_mcu = st->on_init;
> 
> This is not needed because of the rproc_set_mcu_sync_state call below.

You are correct.

> 
> regards
> Suman
> 
> >> +	/* And the synchronisation state machine to use */
> >> +	rproc->sync_states = st;
> >> +	/* Tell the core what to do when initialising */
> >> +	rproc_set_mcu_sync_state(rproc, RPROC_SYNC_STATE_INIT);
> >> +	return 0;
> >> +}
> >> +
> >>  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
> >>  				 const struct rproc_ops *boot_ops,
> >>  				 const struct rproc_ops *sync_ops,
> >> @@ -2065,7 +2119,10 @@ static int rproc_alloc_internals(struct rproc *rproc,
> >> const char *name,
> >>  			return ret;
> >>  	}
> >>
> >> -	return 0;
> >> +	/* Finally allocate the synchronisation state machine */
> >> +	ret = rproc_alloc_sync_states(rproc, boot_ops, sync_ops,
> >> sync_states);
> >> +
> >> +	return ret;
> >>  }
> >>
> >>  /**
> >> --
> >> 2.20.1
> > 
> 

Re: [PATCH v2 15/17] remoteproc: Correctly deal with MCU synchronisation when changing FW image

[Mathieu Poirier]

On Tue, Mar 31, 2020 at 05:14:18PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/30/20 6:21 PM, Mathieu Poirier wrote:
> > On Fri, Mar 27, 2020 at 01:50:18PM +0000, Loic PALLARDY wrote:
> >>
> >>> This patch prevents the firmware image from being displayed or changed
> >>> when
> >>> the remoteproc core is synchronising with an MCU. This is needed since
> >>> there is no guarantee about the nature of the firmware image that is loaded
> >>> by the external entity.
> >>>
> >>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >>> ---
> >>>  drivers/remoteproc/remoteproc_sysfs.c | 25
> >>> ++++++++++++++++++++++++-
> >>>  1 file changed, 24 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> >>> b/drivers/remoteproc/remoteproc_sysfs.c
> >>> index 7f8536b73295..4956577ad4b4 100644
> >>> --- a/drivers/remoteproc/remoteproc_sysfs.c
> >>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
> >>> @@ -13,9 +13,20 @@
> >>>  static ssize_t firmware_show(struct device *dev, struct device_attribute
> >>> *attr,
> >>>  			  char *buf)
> >>>  {
> >>> +	ssize_t ret;
> >>>  	struct rproc *rproc = to_rproc(dev);
> >>>
> >>> -	return sprintf(buf, "%s\n", rproc->firmware);
> >>> +	/*
> >>> +	 * In most instances there is no guarantee about the firmware
> >>> +	 * that was loaded by the external entity.  As such simply don't
> >>> +	 * print anything.
> >>> +	 */
> >>> +	if (rproc_sync_with_mcu(rproc))
> >>> +		ret = sprintf(buf, "\n");
> >> Is it enough to provide empty name, or should we add a message to indicate that's name is unkown/undefined ?
> >>
> > 
> > Don't know... It is easy to find plenty of cases in sysfs where null values are
> > represented with a "\n", and just as many where "unknown", "undefined" or "-1"
> > are used. I know GKH prefers the least amount of information as possible, hence
> > going with a "\n".
> > 
> > Again, no strong opinion...
> > 
> >> Regards,
> >> Loic
> >>> +	else
> >>> +		ret = sprintf(buf, "%s\n", rproc->firmware);
> >>> +
> >>> +	return ret;
> >>>  }
> >>>
> >>>  /* Change firmware name via sysfs */
> >>> @@ -33,6 +44,18 @@ static ssize_t firmware_store(struct device *dev,
> >>>  		return -EINVAL;
> >>>  	}
> >>>
> >>> +	/*
> >>> +	 * There is no point in trying to change the firmware if the MCU
> >>> +	 * is currently running or if loading of the image is done by
> >>> +	 * another entity.
> >>> +	 */
> >>> +	if (rproc_sync_with_mcu(rproc)) {
> >>> +		dev_err(dev,
> >>> +			"can't change firmware while synchronising with
> >>> MCU\n");
> >>> +		err = -EBUSY;
> >>> +		goto out;
> >>> +	}
> >>> +
> 
> So, I have done a patch sometime back to deny sysfs operations [1] (the
> primary usecase is for a rproc-client driver driven boot where auto-boot
> is not set) which is still a need for me. Do you see that as orthogonal
> to that, or can we leverage that here somehow. I cannot use the sync_
> conditions for my cases since they are not already booted before.

I will look at your patch and see if I there is a way to fit that in.  I will
get back to you...

> 
> Also, any reason why you want to do this check before the rproc->state
> unlike the logic around the 'state' file in the next patch?

No specific reason, I will move the check down to be consistent.

> 
> [1] https://patchwork.kernel.org/patch/10601325/
> 
> regards
> Suman
> 
> >>>  	if (rproc->state != RPROC_OFFLINE) {
> >>>  		dev_err(dev, "can't change firmware while running\n");
> >>>  		err = -EBUSY;
> >>> --
> >>> 2.20.1
> >>
> 

Re: [PATCH v2 16/17] remoteproc: Correctly deal with MCU synchronisation when changing state

[Mathieu Poirier]

On Tue, Mar 31, 2020 at 05:35:58PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/30/20 6:49 PM, Mathieu Poirier wrote:
> > On Fri, Mar 27, 2020 at 02:04:36PM +0000, Loic PALLARDY wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> >>> Sent: mardi 24 mars 2020 22:46
> >>> To: bjorn.andersson@linaro.org
> >>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> >>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> >>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> >>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> >>> Subject: [PATCH v2 16/17] remoteproc: Correctly deal with MCU
> >>> synchronisation when changing state
> >>>
> >>> This patch deals with state changes when synchronising with an MCU. More
> >>> specifically it prevents the MCU from being started if it already has been
> >>> started by another entity.  Similarly it prevents the AP from stopping the
> >>> MCU if it hasn't been given the capability by platform firmware.
> >>>
> >>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >>> ---
> >>>  drivers/remoteproc/remoteproc_sysfs.c | 32
> >>> ++++++++++++++++++++++++++-
> >>>  1 file changed, 31 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> >>> b/drivers/remoteproc/remoteproc_sysfs.c
> >>> index 4956577ad4b4..741a3c152b82 100644
> >>> --- a/drivers/remoteproc/remoteproc_sysfs.c
> >>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
> >>> @@ -108,6 +108,29 @@ static ssize_t state_show(struct device *dev, struct
> >>> device_attribute *attr,
> >>>  	return sprintf(buf, "%s\n", rproc_state_string[state]);
> >>>  }
> >>>
> >>> +static int rproc_can_shutdown(struct rproc *rproc)
> >>> +{
> >>> +	/* The MCU is not running, obviously an invalid operation. */
> >>> +	if (rproc->state != RPROC_RUNNING)
> >>> +		return false;
> >>> +
> >>> +	/*
> >>> +	 * The MCU is not running (see above) and the remoteproc core is
> >>> the
> >>> +	 * lifecycle manager, no problem calling for a shutdown.
> >>> +	 */
> >>> +	if (!rproc_sync_with_mcu(rproc))
> >>> +		return true;
> >>> +
> >>> +	/*
> >>> +	 * The MCU has been loaded by another entity (see above) and the
> >>> +	 * platform code has _not_ given us the capability of stopping it.
> >>> +	 */
> >>> +	if (!rproc->sync_ops->stop)
> >>> +		return false;
> >>
> >> Test could be simplified
> >> if (rproc_sync_with_mcu(rproc)) && !rproc->sync_ops->stop)
> >> 	return false;
> > 
> > I laid out the test individually on purpose.  That way there is no coupling
> > between conditions, it is plane to see what is going on and remains maintainable
> > as we add new tests.
> > 
> >>
> >>> +
> >>> +	return true;
> >>> +}
> >>> +
> >>>  /* Change remote processor state via sysfs */
> >>>  static ssize_t state_store(struct device *dev,
> >>>  			      struct device_attribute *attr,
> >>> @@ -120,11 +143,18 @@ static ssize_t state_store(struct device *dev,
> >>>  		if (rproc->state == RPROC_RUNNING)
> >>>  			return -EBUSY;
> >>>
> >>> +		/*
> >>> +		 * In synchronisation mode, booting the MCU is the
> >>> +		 * responsibility of an external entity.
> >>> +		 */
> >>> +		if (rproc_sync_with_mcu(rproc))
> >>> +			return -EINVAL;
> >>> +
> >> I don't understand this restriction, simply because it is preventing to resynchronize with a
> >> coprocessor after a "stop".
> 
> There's actually one more scenario even without "stop". If auto_boot is
> set to false, then rproc_actuate will never get called.

I fail to fully understand the situation you are describing - can you give me
more information?

> 
> >> In the following configuration which can be configuration for coprocessor with romed/flashed
> >> firmware (no reload needed):
> >> on_init = true
> >> after_stop = true
> >> after_crash = true
> >> Once you stop it via sysfs interface, you can't anymore restart/resync to it.
> > 
> > Very true.  The MCU will get restarted by another entity but the AP won't
> > synchronise with it.  I need more time to think about the best way to deal with
> > this and may have to get back to you for further discussions.
> > 
> >>
> >> I think it will be better to modify rproc_boot() to take into account rproc_sync_with_mcu()
> >> as below:
> >>
> >> int rproc_boot(struct rproc *rproc)
> >>  {
> >> -	const struct firmware *firmware_p;
> >> +	const struct firmware *firmware_p = NULL;
> >>  	struct device *dev = &rproc->dev;
> >>  	int ret;
> >>  
> >>  	if (!rproc) {
> >>  		pr_err("invalid rproc handle\n");
> >>  		return -EINVAL;
> >>  	}
> >>  
> >>  	/* load firmware */
> >> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
> >> -	if (ret < 0) {
> >> -		dev_err(dev, "request_firmware failed: %d\n", ret);
> >> -		return ret;
> >> +	if (!rproc_sync_with_mcu(rproc)) {
> 
> I guess this is what the original skip_fw_load was doing. And with the
> current series, the userspace loading support usecase I have cannot be
> achieved. If this is added back, I can try if that works for my usecases.
> 
> >> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
> >> +		if (ret < 0) {
> >> +			dev_err(dev, "request_firmware failed: %d\n", ret);
> >> +			return ret;
> >> +		}
> >>  	}
> >>  
> >>  	ret = rproc_actuate(rproc, firmware_p);
> >>  
> >> -	release_firmware(firmware_p);
> >> +	if (firmware_p)
> >> +		release_firmware(firmware_p);
> >>  
> >>  	return ret;
> >>  }
> >>  
> >> Thanks to these modifications, I'm able to resync after a stop with coprocessor without reloading firmware.
> >>
> >>>  		ret = rproc_boot(rproc);
> >>>  		if (ret)
> >>>  			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
> >>>  	} else if (sysfs_streq(buf, "stop")) {
> >>> -		if (rproc->state != RPROC_RUNNING)
> >>> +		if (!rproc_can_shutdown(rproc))
> >>>  			return -EINVAL;
> >>>
> >>>  		rproc_shutdown(rproc);
> >> As rproc shutdown is also accessible as kernel API, I propose to move
> >> rproc_can_shutdown() check inside rproc_shutdown() and to test
> >> returned error
> > 
> > Ah yes, it is public...  As you point out, I think we'll need to move
> > rproc_can_shutdown() in rproc_shutdown().
> 
> I am assuming only the new conditions, right?

Once again I fail to grasp the extent of your comment.  Can you be more specific
about the "new conditions"?

Thanks
Mathieu

> 
> regards
> Suman
> 
> > 
> > Thank you for taking the time to review this set,
> > Mathieu
> > 

Re: [PATCH v2 00/17] remoteproc: Add support for synchronisation with MCU

[Mathieu Poirier]

On Tue, Mar 31, 2020 at 05:51:44PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/27/20 12:20 PM, Loic PALLARDY wrote:
> > Hi Mathieu,
> > 
> >>
> >> This is the second revision of this set that tries to address the
> >> problem of synchronising with an MCU with as much flexibility as
> >> possible.
> >>
> >> New in this revision is a fix for a couple of bugs I found while
> >> testing things.  First with the helper macro in patch 09 and the
> >> suppression of a boot message when synchronising with an MCU
> >> in patch 12.  I have completely removed what used to be patch 18,
> >> the example on how to use the new API.  This will be the subject
> >> of an upcoming patchset.
> >>
> >> Tested on ST's mp157c platform.  Applies on v5.6-rc7 to keep things
> >> simple.
> > 
> > Thanks Mathieu for the 2 series. I tested on my STM32MP157-DK2 the different
> > synchronization use cases (on_init, after_stop, after_crash), mixing the values and
> > I succeed to start/stop/restart M4 coprocessor with or without reloading firmware
> > according to sync values. (I only applied the correction I proposed in patch 16 review
> > to allow to resync with a preloaded or an already running coprocessor.
> > 
> > Regards,
> > Loic
> >>
> >> Comments would be much appreciated.
> 
> Thank you for the enhanced series to implement the logic in remoteproc
> core. I have provided my comments on most of the patches.
> 
> Overall, I can see my early-boot scenarios work with the series, and the
> slightly different userspace-loading support usecase would need some
> additional support.
> 
> As I commented on patch 1 in v1, I would rather reuse the the generic
> "rproc" instead of adding a new "mcu" terminology to code.. Let's just
> stick with the rproc

You got it.

> 
> Another thing I would prefer (echoing my comments on patch 7) is to just
> use an API for modifying the sync states, that can be used between
> rproc_alloc() and rproc_add(). The state-machine really doesn't kick in
> until rproc_add() is invoked. The memory for the driver private rproc
> structure is allocated using rproc_alloc() normally, and most of the
> DT-parsing in platform drivers is generally done directly into this
> allocated memory. I see it a bit cumbersome having to maintain a
> separate structure, and then do a memcpy, especially given that the
> rproc_alloc_state_machine() logic requires that you detect the state
> before calling the rproc_alloc().

You raise an interesting point... As my work on the mp157c [1] proved, mandating
to know if the core should sync before calling rproc_alloc_state_machine()
requires a fair amount of refactoring.  I will follow your recommendation for
the next revision.

Thanks,
Mathieu


[1]. https://patchwork.kernel.org/project/linux-remoteproc/list/?series=261083

> 
> regards
> Suman
> 
> >>
> >> Thanks,
> >> Mathieu
> >>
> >> Mathieu Poirier (17):
> >>   remoteproc: Add new operation and state machine for MCU
> >>     synchronisation
> >>   remoteproc: Introduce function rproc_set_mcu_sync_state()
> >>   remoteproc: Split firmware name allocation from rproc_alloc()
> >>   remoteproc: Split rproc_ops allocation from rproc_alloc()
> >>   remoteproc: Get rid of tedious error path
> >>   remoteproc: Introduce function rproc_alloc_internals()
> >>   remoteproc: Introduce function rproc_alloc_state_machine()
> >>   remoteproc: Allocate synchronisation state machine
> >>   remoteproc: Call the right core function based on synchronisation
> >>     state
> >>   remoteproc: Decouple firmware load and remoteproc booting
> >>   remoteproc: Repurpose function rproc_trigger_auto_boot()
> >>   remoteproc: Rename function rproc_fw_boot()
> >>   remoteproc: Introducting new functions to start and stop an MCU
> >>   remoteproc: Refactor function rproc_trigger_recovery()
> >>   remoteproc: Correctly deal with MCU synchronisation when changing FW
> >>     image
> >>   remoteproc: Correctly deal with MCU synchronisation when changing
> >>     state
> >>   remoteproc: Make MCU synchronisation state changes on stop and crashed
> >>
> >>  drivers/remoteproc/remoteproc_core.c     | 387 ++++++++++++++++++-----
> >>  drivers/remoteproc/remoteproc_debugfs.c  |  31 ++
> >>  drivers/remoteproc/remoteproc_internal.h |  91 ++++--
> >>  drivers/remoteproc/remoteproc_sysfs.c    |  57 +++-
> >>  include/linux/remoteproc.h               |  28 +-
> >>  5 files changed, 487 insertions(+), 107 deletions(-)
> >>
> >> --
> >> 2.20.1
> > 
> 

Re: [PATCH v2 01/17] remoteproc: Add new operation and state machine for MCU synchronisation

[Mathieu Poirier]

On Mon, Mar 30, 2020 at 05:49:11PM -0500, Suman Anna wrote:
> On 3/30/20 5:46 PM, Suman Anna wrote:
> > Hi Mathieu,
> > 
> > On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> >> Add a new rproc_ops sync_ops to support use cases where the remoteproc
> >> core is synchronisting with the MCU.  When exactly to use the sync_ops is
> > 
> > typo on syschronisting..
> > 
> >> directed by the states in struct rproc_sync_states.
> >>
> >> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> ---
> >>  drivers/remoteproc/remoteproc_debugfs.c  | 31 ++++++++++++++++++++++++
> >>  drivers/remoteproc/remoteproc_internal.h |  5 ++++
> >>  include/linux/remoteproc.h               | 23 +++++++++++++++++-
> >>  3 files changed, 58 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/remoteproc/remoteproc_debugfs.c b/drivers/remoteproc/remoteproc_debugfs.c
> >> index dd93cf04e17f..187bcc67f997 100644
> >> --- a/drivers/remoteproc/remoteproc_debugfs.c
> >> +++ b/drivers/remoteproc/remoteproc_debugfs.c
> >> @@ -311,6 +311,35 @@ static const struct file_operations rproc_carveouts_ops = {
> >>  	.release	= single_release,
> >>  };
> >>  
> >> +/* Expose synchronisation states via debugfs */
> >> +static int rproc_sync_states_show(struct seq_file *seq, void *p)
> >> +{
> >> +	struct rproc *rproc = seq->private;
> >> +
> >> +	seq_printf(seq, "Sync with MCU: %s\n",
> >> +		   rproc->sync_with_mcu ? "true" : "false");
> >> +	seq_printf(seq, "On init: %s\n",
> >> +		   rproc->sync_states->on_init ? "true" : "false");
> >> +	seq_printf(seq, "After stop: %s\n",
> >> +		   rproc->sync_states->after_stop ? "true" : "false");
> >> +	seq_printf(seq, "After crash: %s\n",
> >> +		   rproc->sync_states->after_crash ? "true" : "false");
> >> +
> >> +	return 0;
> >> +}
> >> +
> >> +static int rproc_sync_states_open(struct inode *inode, struct file *file)
> >> +{
> >> +	return single_open(file, rproc_sync_states_show, inode->i_private);
> >> +}
> >> +
> >> +static const struct file_operations rproc_sync_states_ops = {
> >> +	.open		= rproc_sync_states_open,
> >> +	.read		= seq_read,
> >> +	.llseek		= seq_lseek,
> >> +	.release	= single_release,
> >> +};
> >> +
> >>  void rproc_remove_trace_file(struct dentry *tfile)
> >>  {
> >>  	debugfs_remove(tfile);
> >> @@ -357,6 +386,8 @@ void rproc_create_debug_dir(struct rproc *rproc)
> >>  			    rproc, &rproc_rsc_table_ops);
> >>  	debugfs_create_file("carveout_memories", 0400, rproc->dbg_dir,
> >>  			    rproc, &rproc_carveouts_ops);
> >> +	debugfs_create_file("sync_states", 0400, rproc->dbg_dir,
> >> +			    rproc, &rproc_sync_states_ops);
> >>  }
> >>  
> >>  void __init rproc_init_debugfs(void)
> >> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
> >> index 493ef9262411..5c93de5e00bb 100644
> >> --- a/drivers/remoteproc/remoteproc_internal.h
> >> +++ b/drivers/remoteproc/remoteproc_internal.h
> >> @@ -63,6 +63,11 @@ struct resource_table *rproc_elf_find_loaded_rsc_table(struct rproc *rproc,
> >>  struct rproc_mem_entry *
> >>  rproc_find_carveout_by_name(struct rproc *rproc, const char *name, ...);
> >>  
> >> +static inline bool rproc_sync_with_mcu(struct rproc *rproc)
> >> +{
> >> +	return rproc->sync_with_mcu;
> >> +}
> >> +
> > 
> > Since you are using this mostly for checking and as a boolean, I suggest
> > you rename this appropriately, something like rproc_needs_sync,
> > rproc_has_sync or rproc_uses_sync().

I will rename to rproc_syncing_with_rproc()


> > 
> > And I am wondering if it is actually better to introduce the sync state
> > to check against here, rather than using the stored sync state and
> > return. The current way makes it confusing to read the state machine.

I decided to proceed this way because there may not be a direct correlation
between the current synchronisation state and the location of the check itself.
for instance, in firmware_show(), what sync state should be key on?

> > 
> >>  static inline
> >>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
> >>  {
> >> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
> >> index 16ad66683ad0..d115e47d702d 100644
> >> --- a/include/linux/remoteproc.h
> >> +++ b/include/linux/remoteproc.h
> >> @@ -353,6 +353,21 @@ enum rsc_handling_status {
> >>  	RSC_IGNORED	= 1,
> >>  };
> >>  
> >> +/**
> >> + * struct rproc_sync_states - platform specific states indicating which
> >> + *			      rproc_ops to use at specific times during
> >> + *			      the MCU lifecycle.
> >> + * @on_init: true if synchronising with MCU at system initialisation time
> >> + * @after_stop: true if synchronising with MCU after stopped from the
> >> + *		command line
> >> + * @after_crash: true if synchonising with MCU after the MCU has crashed
> >> + */
> >> +struct rproc_sync_states {
> >> +	bool on_init;
> >> +	bool after_stop;
> >> +	bool after_crash;
> >> +};
> >> +
> > 
> > Overall, this patch can move down the order, and better to add it in
> > the patches where you actually introduce these code. And the debugfs
> > pieces can be added as a separate patch by itself.
> 
> Also, actually sounds more like flags than states..

I thought about this in terms of "states" in which a decision should be made.
I'm not sure those are flags...

> 
> regards
> Suman
> 
> > 
> >>  /**
> >>   * struct rproc_ops - platform-specific device handlers
> >>   * @start:	power on the device and boot it
> >> @@ -456,6 +471,9 @@ struct rproc_dump_segment {
> >>   * @firmware: name of firmware file to be loaded
> >>   * @priv: private data which belongs to the platform-specific rproc module
> >>   * @ops: platform-specific start/stop rproc handlers
> >> + * @sync_ops: paltform-specific start/stop rproc handlers when
> > 
> > typo on platform

No matter how many times you read your own code, there's always something like
this that escapes...

> > 
> >> + *	      synchronising with a remote processor.
> >> + * @sync_states: Determine the rproc_ops to choose in specific states.
> >>   * @dev: virtual device for refcounting and common remoteproc behavior
> >>   * @power: refcount of users who need this rproc powered up
> >>   * @state: state of the device
> >> @@ -479,6 +497,7 @@ struct rproc_dump_segment {
> >>   * @table_sz: size of @cached_table
> >>   * @has_iommu: flag to indicate if remote processor is behind an MMU
> >>   * @auto_boot: flag to indicate if remote processor should be auto-started
> >> + * @sync_with_mcu: true if currently synchronising with MCU
> >>   * @dump_segments: list of segments in the firmware
> >>   * @nb_vdev: number of vdev currently handled by rproc
> >>   */
> >> @@ -488,7 +507,8 @@ struct rproc {
> >>  	const char *name;
> >>  	char *firmware;
> >>  	void *priv;
> >> -	struct rproc_ops *ops;
> >> +	struct rproc_ops *ops, *sync_ops;
> > 
> > Nothing wrong with this, but prefer to have the new variable in a new
> > line for better readability.

Sure thing.

> > 
> > regards
> > Suman
> > 
> >> +	struct rproc_sync_states *sync_states;
> >>  	struct device dev;
> >>  	atomic_t power;
> >>  	unsigned int state;
> >> @@ -512,6 +532,7 @@ struct rproc {
> >>  	size_t table_sz;
> >>  	bool has_iommu;
> >>  	bool auto_boot;
> >> +	bool sync_with_mcu;
> >>  	struct list_head dump_segments;
> >>  	int nb_vdev;
> >>  };
> >>
> > 
> 

Re: [PATCH v2 13/17] remoteproc: Introducting new functions to start and stop an MCU

[Mathieu Poirier]

On Tue, Mar 31, 2020 at 04:46:32PM -0500, Suman Anna wrote:
> On 3/31/20 1:08 PM, Suman Anna wrote:
> > Hi Mathieu,
> > 
> > On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> >> Add new functions to replace direct calling of rproc->ops->start() and
> >> rproc->ops->stop().  That way different behaviour can be played out
> >> when booting an MCU or synchronising with it.
> >>
> >> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> ---
> >>  drivers/remoteproc/remoteproc_core.c     |  6 +++---
> >>  drivers/remoteproc/remoteproc_internal.h | 12 ++++++++++++
> >>  2 files changed, 15 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> >> index 488723fcb142..d3c4d7e6ca25 100644
> >> --- a/drivers/remoteproc/remoteproc_core.c
> >> +++ b/drivers/remoteproc/remoteproc_core.c
> >> @@ -1330,7 +1330,7 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
> >>  	}
> >>  
> >>  	/* power up the remote processor */
> >> -	ret = rproc->ops->start(rproc);
> >> +	ret = rproc_start_hw(rproc);
> >>  	if (ret) {
> >>  		dev_err(dev, "can't start rproc %s: %d\n", rproc->name, ret);
> >>  		goto unprepare_subdevices;
> >> @@ -1351,7 +1351,7 @@ static int rproc_start(struct rproc *rproc, const struct firmware *fw)
> >>  	return 0;
> >>  
> >>  stop_rproc:
> >> -	rproc->ops->stop(rproc);
> >> +	rproc_stop_hw(rproc);
> >>  unprepare_subdevices:
> >>  	rproc_unprepare_subdevices(rproc);
> >>  reset_table_ptr:
> >> @@ -1485,7 +1485,7 @@ static int rproc_stop(struct rproc *rproc, bool crashed)
> >>  	rproc->table_ptr = rproc->cached_table;
> >>  
> >>  	/* power off the remote processor */
> >> -	ret = rproc->ops->stop(rproc);
> >> +	ret = rproc_stop_hw(rproc);
> >>  	if (ret) {
> >>  		dev_err(dev, "can't stop rproc: %d\n", ret);
> >>  		return ret;
> >> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
> >> index 5f711ceb97ba..7ca23d46dfd4 100644
> >> --- a/drivers/remoteproc/remoteproc_internal.h
> >> +++ b/drivers/remoteproc/remoteproc_internal.h
> >> @@ -160,4 +160,16 @@ struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
> >>  	return NULL;
> >>  }
> >>  
> >> +static inline int rproc_start_hw(struct rproc *rproc)
> >> +{
> >> +	RPROC_OPS_HELPER(start, rproc);
> >> +	return -EINVAL;
> >> +}
> >> +
> >> +static inline int rproc_stop_hw(struct rproc *rproc)
> >> +{
> >> +	RPROC_OPS_HELPER(stop, rproc);
> >> +	return -EINVAL;
> >> +}
> > 
> > Since we already have the concept of subdevices, how about we call these
> > rproc_{start/stop}_device?
> 
> Actually, does this patch needs to be moved up in the order atleast
> prior to patch 11, may be after patch 9?

Sure, that can be done...

> 
> regards
> Suman
> 
> > 
> >> +
> >>  #endif /* REMOTEPROC_INTERNAL_H */
> >>
> > 
> 

Re: [PATCH v2 03/17] remoteproc: Split firmware name allocation from rproc_alloc()

[Mathieu Poirier]

On Mon, Mar 30, 2020 at 02:47:54PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/27/20 6:05 AM, Loic PALLARDY wrote:
> > Hi Mathieu,
> > 
> >> -----Original Message-----
> >> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> Sent: mardi 24 mars 2020 22:46
> >> To: bjorn.andersson@linaro.org
> >> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> >> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> >> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> >> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> >> Subject: [PATCH v2 03/17] remoteproc: Split firmware name allocation from
> >> rproc_alloc()
> >>
> >> Make the firmware name allocation a function on its own in order to
> >> introduce more flexibility to function rproc_alloc().
> 
> I see patches 3 through 5 are generic cleanups, can you post them
> separately from this series? Bjorn has commented about using the
> put_device() to free the code on one of the remoteproc core patches [1]
> in my R5 patch series, and I can do my patch on top of yours. I plan to
> split out those 2 core patches for my next version, and can do them on
> top of these.

That shouldn't be a problem.

> 
> [1] https://patchwork.kernel.org/patch/11456385/#23248321
> 
> >>
> >> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> ---
> >>  drivers/remoteproc/remoteproc_core.c | 62 +++++++++++++++++-----------
> >>  1 file changed, 39 insertions(+), 23 deletions(-)
> >>
> >> diff --git a/drivers/remoteproc/remoteproc_core.c
> >> b/drivers/remoteproc/remoteproc_core.c
> >> index 097f33e4f1f3..c0871f69929b 100644
> >> --- a/drivers/remoteproc/remoteproc_core.c
> >> +++ b/drivers/remoteproc/remoteproc_core.c
> >> @@ -1962,6 +1962,36 @@ static const struct device_type rproc_type = {
> >>  	.release	= rproc_type_release,
> >>  };
> >>
> >> +static int rproc_alloc_firmware(struct rproc *rproc,
> >> +				const char *name, const char *firmware)
> >> +{
> >> +	char *p, *template = "rproc-%s-fw";
> >> +	int name_len;
> >> +
> >> +	if (!rproc || !name)
> >> +		return -EINVAL;
> 
> This is an internal function, and these are already checked in
> rproc_alloc(), so you can drop this.
> 
> >> +
> >> +	if (!firmware) {
> >> +		/*
> >> +		 * If the caller didn't pass in a firmware name then
> >> +		 * construct a default name.
> >> +		 */
> >> +		name_len = strlen(name) + strlen(template) - 2 + 1;
> >> +		p = kmalloc(name_len, GFP_KERNEL);
> >> +		if (!p)
> >> +			return -ENOMEM;
> >> +		snprintf(p, name_len, template, name);
> >> +	} else {
> >> +		p = kstrdup(firmware, GFP_KERNEL);
> >> +		if (!p)
> >> +			return -ENOMEM;
> >> +	}
> >> +
> >> +	rproc->firmware = p;
> >> +
> >> +	return 0;
> >> +}
> >> +
> >>  /**
> >>   * rproc_alloc() - allocate a remote processor handle
> >>   * @dev: the underlying device
> >> @@ -1990,42 +2020,24 @@ struct rproc *rproc_alloc(struct device *dev,
> >> const char *name,
> >>  			  const char *firmware, int len)
> >>  {
> >>  	struct rproc *rproc;
> >> -	char *p, *template = "rproc-%s-fw";
> >> -	int name_len;
> >>
> >>  	if (!dev || !name || !ops)
> >>  		return NULL;
> >>
> >> -	if (!firmware) {
> >> -		/*
> >> -		 * If the caller didn't pass in a firmware name then
> >> -		 * construct a default name.
> >> -		 */
> >> -		name_len = strlen(name) + strlen(template) - 2 + 1;
> >> -		p = kmalloc(name_len, GFP_KERNEL);
> >> -		if (!p)
> >> -			return NULL;
> >> -		snprintf(p, name_len, template, name);
> >> -	} else {
> >> -		p = kstrdup(firmware, GFP_KERNEL);
> >> -		if (!p)
> >> -			return NULL;
> >> -	}
> >> -
> >>  	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
> >> -	if (!rproc) {
> >> -		kfree(p);
> >> +	if (!rproc)
> >>  		return NULL;
> >> -	}
> >> +
> >> +	if (rproc_alloc_firmware(rproc, name, firmware))
> >> +		goto free_rproc;
> 
> Since you are already moving this after rproc_alloc() here in this
> patch, you might as well fold the relevant patch 5 contents here?
> Otherwise, retain the existing code as is, and do all the movement in
> patch 5.
> 
> regards
> Suman
> 
> >>
> >>  	rproc->ops = kmemdup(ops, sizeof(*ops), GFP_KERNEL);
> >>  	if (!rproc->ops) {
> >> -		kfree(p);
> >> +		kfree(rproc->firmware);
> >>  		kfree(rproc);
> > Small remark only for patch coherency, as it is modified in next patches.
> > Use free_rproc label which is introduced just below here for error management.
> > 
> > Regards,
> > Loic
> >>  		return NULL;
> >>  	}
> >>
> >> -	rproc->firmware = p;
> >>  	rproc->name = name;
> >>  	rproc->priv = &rproc[1];
> >>  	rproc->auto_boot = true;
> >> @@ -2073,6 +2085,10 @@ struct rproc *rproc_alloc(struct device *dev, const
> >> char *name,
> >>  	rproc->state = RPROC_OFFLINE;
> >>
> >>  	return rproc;
> >> +
> >> +free_rproc:
> >> +	kfree(rproc);
> >> +	return NULL;
> >>  }
> >>  EXPORT_SYMBOL(rproc_alloc);
> >>
> >> --
> >> 2.20.1
> > 
> 

Re: [PATCH v2 09/17] remoteproc: Call the right core function based on synchronisation state

[Mathieu Poirier]

On Tue, Mar 31, 2020 at 10:10:50AM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/24/20 4:45 PM, Mathieu Poirier wrote:
> > Call the right core function based on whether we should synchronise
> > with an MCU or boot it from scratch.
> 
> This patch does generate some checkpatch warnings.

Right, checkpatch is complaining but other than duplicating the same code for
all functions, I don't see another way to do this.

> 
> > 
> > Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> > ---
> >  drivers/remoteproc/remoteproc_internal.h | 36 +++++++++++-------------
> >  1 file changed, 17 insertions(+), 19 deletions(-)
> > 
> > diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
> > index 73ea32df0156..5f711ceb97ba 100644
> > --- a/drivers/remoteproc/remoteproc_internal.h
> > +++ b/drivers/remoteproc/remoteproc_internal.h
> > @@ -106,38 +106,41 @@ static inline void rproc_set_mcu_sync_state(struct rproc *rproc,
> >  	}
> >  }
> >  
> > +#define RPROC_OPS_HELPER(__operation, ...)				\
> > +	do {								\
> > +		if (rproc_sync_with_mcu(rproc)) {			\
> 
> So this does make the logic a bit convoluted, since you have three
> different flags for rproc_sync_with_mcu, and you apply them in common
> for all the ops. This is what I meant in my comment on Patch 1.

There is indeed 3 different flags but they are only valid in a specific state.
What "ops" are you referring to here?  I'm also not sure about the comment in
"patch 1" - which one would that be and how does it relate to the current block
of code.  Apologies, I need more clarifications.  

> 
> > +			if (!rproc->sync_ops ||				\
> > +			    !rproc->sync_ops->__operation)		\
> > +				return 0;				\
> > +			return rproc->sync_ops->__operation(__VA_ARGS__); \
> 
> Use the same semantics as the regular ops instead of two return
> statements, the code should fallback to the common return 0 after the
> RPROC_OPS_HELPER when neither of them are present.

Yes the tests are exactly the same, no reason to proceed differently.

> 
> regards
> Suman
> 
> > +		} else if (rproc->ops && rproc->ops->__operation)	\
> > +			return rproc->ops->__operation(__VA_ARGS__);	\
> > +	} while (0)							\
> > +
> >  static inline
> >  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
> >  {
> > -	if (rproc->ops->sanity_check)
> > -		return rproc->ops->sanity_check(rproc, fw);
> > -
> > +	RPROC_OPS_HELPER(sanity_check, rproc, fw);
> >  	return 0;
> >  }
> >  
> >  static inline
> >  u32 rproc_get_boot_addr(struct rproc *rproc, const struct firmware *fw)
> >  {
> > -	if (rproc->ops->get_boot_addr)
> > -		return rproc->ops->get_boot_addr(rproc, fw);
> > -
> > +	RPROC_OPS_HELPER(get_boot_addr, rproc, fw);
> >  	return 0;
> >  }
> >  
> >  static inline
> >  int rproc_load_segments(struct rproc *rproc, const struct firmware *fw)
> >  {
> > -	if (rproc->ops->load)
> > -		return rproc->ops->load(rproc, fw);
> > -
> > +	RPROC_OPS_HELPER(load, rproc, fw);
> >  	return -EINVAL;
> >  }
> >  
> >  static inline int rproc_parse_fw(struct rproc *rproc, const struct firmware *fw)
> >  {
> > -	if (rproc->ops->parse_fw)
> > -		return rproc->ops->parse_fw(rproc, fw);
> > -
> > +	RPROC_OPS_HELPER(parse_fw, rproc, fw);
> >  	return 0;
> >  }
> >  
> > @@ -145,10 +148,7 @@ static inline
> >  int rproc_handle_rsc(struct rproc *rproc, u32 rsc_type, void *rsc, int offset,
> >  		     int avail)
> >  {
> > -	if (rproc->ops->handle_rsc)
> > -		return rproc->ops->handle_rsc(rproc, rsc_type, rsc, offset,
> > -					      avail);
> > -
> > +	RPROC_OPS_HELPER(handle_rsc, rproc, rsc_type, rsc, offset, avail);
> >  	return RSC_IGNORED;
> >  }
> >  
> > @@ -156,9 +156,7 @@ static inline
> >  struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
> >  						   const struct firmware *fw)
> >  {
> > -	if (rproc->ops->find_loaded_rsc_table)
> > -		return rproc->ops->find_loaded_rsc_table(rproc, fw);
> > -
> > +	RPROC_OPS_HELPER(find_loaded_rsc_table, rproc, fw);
> >  	return NULL;
> >  }
> >  
> > 
> 

Re: [PATCH v2 14/17] remoteproc: Refactor function rproc_trigger_recovery()

[Mathieu Poirier]

On Tue, Mar 31, 2020 at 04:52:12PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/24/20 4:46 PM, Mathieu Poirier wrote:
> > Refactor function rproc_trigger_recovery() in order to avoid
> > reloading the fw image when synchronising with an MCU rather than
> > booting it.
> > 
> > Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> > ---
> >  drivers/remoteproc/remoteproc_core.c | 16 +++++++++-------
> >  1 file changed, 9 insertions(+), 7 deletions(-)
> > 
> > diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
> > index d3c4d7e6ca25..dbb0a8467205 100644
> > --- a/drivers/remoteproc/remoteproc_core.c
> > +++ b/drivers/remoteproc/remoteproc_core.c
> > @@ -1661,7 +1661,7 @@ static void rproc_coredump(struct rproc *rproc)
> >   */
> >  int rproc_trigger_recovery(struct rproc *rproc)
> >  {
> > -	const struct firmware *firmware_p;
> > +	const struct firmware *firmware_p = NULL;
> >  	struct device *dev = &rproc->dev;
> >  	int ret;
> >  
> > @@ -1678,14 +1678,16 @@ int rproc_trigger_recovery(struct rproc *rproc)
> >  	/* generate coredump */
> >  	rproc_coredump(rproc);
> >  
> > -	/* load firmware */
> > -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
> > -	if (ret < 0) {
> > -		dev_err(dev, "request_firmware failed: %d\n", ret);
> > -		goto unlock_mutex;
> > +	/* load firmware if need be */
> > +	if (!rproc_sync_with_mcu(rproc)) {
> > +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
> > +		if (ret < 0) {
> > +			dev_err(dev, "request_firmware failed: %d\n", ret);
> > +			goto unlock_mutex;
> > +		}
> 
> So, I am trying to understand the need for the flag around
> RPROC_SYNC_STATE_CRASHED. Can you explain what all usecases that is
> covering?

There could scenarios where another entity is in charge of the entire MCU
lifecycle.  That entity could be able to recognise the MCU has crashed and
automatically boot it again, in which case all the remoteproc core needs to do
is synchronise with it.

But it could also be that another entity has booted the MCU when the system
started but if the MCU crashes or is manually stopped, then the AP becomes the
MCU lifecycle. 

> 
> In anycase, you should probably combine this piece with the flag change
> for STATE_CRASHED on the last patch.

Sure.

> 
> regards
> Suman
> 
> >  	}
> >  
> > -	/* boot the remote processor up again */
> > +	/* boot up or synchronise with the remote processor again */
> >  	ret = rproc_start(rproc, firmware_p);
> >  
> >  	release_firmware(firmware_p);
> > 
> 

Re: [PATCH v2 16/17] remoteproc: Correctly deal with MCU synchronisation when changing state

[Mathieu Poirier]

Hi Suman,

On Tue, Mar 31, 2020 at 05:35:58PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/30/20 6:49 PM, Mathieu Poirier wrote:
> > On Fri, Mar 27, 2020 at 02:04:36PM +0000, Loic PALLARDY wrote:
> >>
> >>
> >>> -----Original Message-----
> >>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
> >>> Sent: mardi 24 mars 2020 22:46
> >>> To: bjorn.andersson@linaro.org
> >>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
> >>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
> >>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
> >>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
> >>> Subject: [PATCH v2 16/17] remoteproc: Correctly deal with MCU
> >>> synchronisation when changing state
> >>>
> >>> This patch deals with state changes when synchronising with an MCU. More
> >>> specifically it prevents the MCU from being started if it already has been
> >>> started by another entity.  Similarly it prevents the AP from stopping the
> >>> MCU if it hasn't been given the capability by platform firmware.
> >>>
> >>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >>> ---
> >>>  drivers/remoteproc/remoteproc_sysfs.c | 32
> >>> ++++++++++++++++++++++++++-
> >>>  1 file changed, 31 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> >>> b/drivers/remoteproc/remoteproc_sysfs.c
> >>> index 4956577ad4b4..741a3c152b82 100644
> >>> --- a/drivers/remoteproc/remoteproc_sysfs.c
> >>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
> >>> @@ -108,6 +108,29 @@ static ssize_t state_show(struct device *dev, struct
> >>> device_attribute *attr,
> >>>  	return sprintf(buf, "%s\n", rproc_state_string[state]);
> >>>  }
> >>>
> >>> +static int rproc_can_shutdown(struct rproc *rproc)
> >>> +{
> >>> +	/* The MCU is not running, obviously an invalid operation. */
> >>> +	if (rproc->state != RPROC_RUNNING)
> >>> +		return false;
> >>> +
> >>> +	/*
> >>> +	 * The MCU is not running (see above) and the remoteproc core is
> >>> the
> >>> +	 * lifecycle manager, no problem calling for a shutdown.
> >>> +	 */
> >>> +	if (!rproc_sync_with_mcu(rproc))
> >>> +		return true;
> >>> +
> >>> +	/*
> >>> +	 * The MCU has been loaded by another entity (see above) and the
> >>> +	 * platform code has _not_ given us the capability of stopping it.
> >>> +	 */
> >>> +	if (!rproc->sync_ops->stop)
> >>> +		return false;
> >>
> >> Test could be simplified
> >> if (rproc_sync_with_mcu(rproc)) && !rproc->sync_ops->stop)
> >> 	return false;
> > 
> > I laid out the test individually on purpose.  That way there is no coupling
> > between conditions, it is plane to see what is going on and remains maintainable
> > as we add new tests.
> > 
> >>
> >>> +
> >>> +	return true;
> >>> +}
> >>> +
> >>>  /* Change remote processor state via sysfs */
> >>>  static ssize_t state_store(struct device *dev,
> >>>  			      struct device_attribute *attr,
> >>> @@ -120,11 +143,18 @@ static ssize_t state_store(struct device *dev,
> >>>  		if (rproc->state == RPROC_RUNNING)
> >>>  			return -EBUSY;
> >>>
> >>> +		/*
> >>> +		 * In synchronisation mode, booting the MCU is the
> >>> +		 * responsibility of an external entity.
> >>> +		 */
> >>> +		if (rproc_sync_with_mcu(rproc))
> >>> +			return -EINVAL;
> >>> +
> >> I don't understand this restriction, simply because it is preventing to resynchronize with a
> >> coprocessor after a "stop".
> 
> There's actually one more scenario even without "stop". If auto_boot is
> set to false, then rproc_actuate will never get called.
> 
> >> In the following configuration which can be configuration for coprocessor with romed/flashed
> >> firmware (no reload needed):
> >> on_init = true
> >> after_stop = true
> >> after_crash = true
> >> Once you stop it via sysfs interface, you can't anymore restart/resync to it.
> > 
> > Very true.  The MCU will get restarted by another entity but the AP won't
> > synchronise with it.  I need more time to think about the best way to deal with
> > this and may have to get back to you for further discussions.
> > 
> >>
> >> I think it will be better to modify rproc_boot() to take into account rproc_sync_with_mcu()
> >> as below:
> >>
> >> int rproc_boot(struct rproc *rproc)
> >>  {
> >> -	const struct firmware *firmware_p;
> >> +	const struct firmware *firmware_p = NULL;
> >>  	struct device *dev = &rproc->dev;
> >>  	int ret;
> >>  
> >>  	if (!rproc) {
> >>  		pr_err("invalid rproc handle\n");
> >>  		return -EINVAL;
> >>  	}
> >>  
> >>  	/* load firmware */
> >> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
> >> -	if (ret < 0) {
> >> -		dev_err(dev, "request_firmware failed: %d\n", ret);
> >> -		return ret;
> >> +	if (!rproc_sync_with_mcu(rproc)) {
> 
> I guess this is what the original skip_fw_load was doing. And with the
> current series, the userspace loading support usecase I have cannot be
> achieved. If this is added back, I can try if that works for my usecases.

I didn't notice this comment upon first read... Can you give me more details on
what your usecase is order to see how best to deal with it?

Thanks,
Mathieu

> 
> >> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
> >> +		if (ret < 0) {
> >> +			dev_err(dev, "request_firmware failed: %d\n", ret);
> >> +			return ret;
> >> +		}
> >>  	}
> >>  
> >>  	ret = rproc_actuate(rproc, firmware_p);
> >>  
> >> -	release_firmware(firmware_p);
> >> +	if (firmware_p)
> >> +		release_firmware(firmware_p);
> >>  
> >>  	return ret;
> >>  }
> >>  
> >> Thanks to these modifications, I'm able to resync after a stop with coprocessor without reloading firmware.
> >>
> >>>  		ret = rproc_boot(rproc);
> >>>  		if (ret)
> >>>  			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
> >>>  	} else if (sysfs_streq(buf, "stop")) {
> >>> -		if (rproc->state != RPROC_RUNNING)
> >>> +		if (!rproc_can_shutdown(rproc))
> >>>  			return -EINVAL;
> >>>
> >>>  		rproc_shutdown(rproc);
> >> As rproc shutdown is also accessible as kernel API, I propose to move
> >> rproc_can_shutdown() check inside rproc_shutdown() and to test
> >> returned error
> > 
> > Ah yes, it is public...  As you point out, I think we'll need to move
> > rproc_can_shutdown() in rproc_shutdown().
> 
> I am assuming only the new conditions, right?
> 
> regards
> Suman
> 
> > 
> > Thank you for taking the time to review this set,
> > Mathieu
> > 

Re: [PATCH v2 07/17] remoteproc: Introduce function rproc_alloc_state_machine()

[Suman Anna]

On 4/1/20 3:41 PM, Mathieu Poirier wrote:
> On Mon, Mar 30, 2020 at 06:10:51PM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>>
>>>> -----Original Message-----
>>>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>> Sent: mardi 24 mars 2020 22:46
>>>> To: bjorn.andersson@linaro.org
>>>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
>>>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
>>>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
>>>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
>>>> Subject: [PATCH v2 07/17] remoteproc: Introduce function
>>>> rproc_alloc_state_machine()
>>>>
>>>> Introducing new function rproc_alloc_state_machine() to allocate
>>>> the MCU synchronisation operations and position it as the central
>>>> remoteproc core allocation function.
>>>>
>>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>> ---
>>>>  drivers/remoteproc/remoteproc_core.c | 84
>>>> +++++++++++++++++++++++++---
>>>>  include/linux/remoteproc.h           |  5 ++
>>>>  2 files changed, 81 insertions(+), 8 deletions(-)
>>>>
>>>> diff --git a/drivers/remoteproc/remoteproc_core.c
>>>> b/drivers/remoteproc/remoteproc_core.c
>>>> index 9da245734db6..02dbb826aa29 100644
>>>> --- a/drivers/remoteproc/remoteproc_core.c
>>>> +++ b/drivers/remoteproc/remoteproc_core.c
>>>> @@ -1954,6 +1954,7 @@ static void rproc_type_release(struct device *dev)
>>>>
>>>>  	kfree(rproc->firmware);
>>>>  	kfree(rproc->ops);
>>>> +	kfree(rproc->sync_ops);
>>>>  	kfree(rproc);
>>>>  }
>>>>
>>>> @@ -2018,12 +2019,34 @@ static int rproc_alloc_ops(struct rproc *rproc,
>>>> const struct rproc_ops *ops)
>>>>  	return 0;
>>>>  }
>>>>
>>>> +static int rproc_alloc_sync_ops(struct rproc *rproc,
>>>> +				const struct rproc_ops *sync_ops)
>>>> +{
>>>> +	/*
>>>> +	 * Given the unlimited amount of possibilities when
>>>> +	 * synchronising with an MCU, no constraints are imposed
>>>> +	 * on sync_ops.
>>>> +	 */
>>>> +	rproc->sync_ops = kmemdup(sync_ops,
>>>> +				  sizeof(*sync_ops), GFP_KERNEL);
>>>> +	if (!rproc->sync_ops)
>>>> +		return -ENOMEM;
>>> Should we check a minimal set of functions in sync_ops to be required?
>>> Or we should consider all pointers at NULL is ok ?
>>>
>>>> +
>>>> +	return 0;
>>>> +}
>>>> +
>>>>  static int rproc_alloc_internals(struct rproc *rproc, const char *name,
>>>>  				 const struct rproc_ops *boot_ops,
>>>> +				 const struct rproc_ops *sync_ops,
>>>> +				 struct rproc_sync_states *sync_states,
>>> sync_states not used in this patch, should be introduced in patch 8
>>
>> +1
> 
> I will do that.
> 
>>
>>>
>>> Regards,
>>> Loic
>>>
>>>>  				 const char *firmware, int len)
>>>>  {
>>>>  	int ret;
>>>>
>>>> +	/* We need at least a boot or a sync ops. */
>>>> +	if (!boot_ops && !sync_ops)
>>>> +		return -EINVAL;
>>>> +
>>>>  	/* We have a boot_ops so allocate firmware name and operations */
>>>>  	if (boot_ops) {
>>>>  		ret = rproc_alloc_firmware(rproc, name, firmware);
>>>> @@ -2035,14 +2058,23 @@ static int rproc_alloc_internals(struct rproc
>>>> *rproc, const char *name,
>>>>  			return ret;
>>>>  	}
>>>>
>>>> +	/* Allocate a sync_ops if need be */
>>>> +	if (sync_ops) {
>>>> +		ret = rproc_alloc_sync_ops(rproc, sync_ops);
>>>> +		if (ret)
>>>> +			return ret;
>>>> +	}
>>>> +
>>>>  	return 0;
>>>>  }
>>>>
>>>>  /**
>>>> - * rproc_alloc() - allocate a remote processor handle
>>>> + * rproc_alloc_state_machine() - allocate a remote processor handle
>>>>   * @dev: the underlying device
>>>>   * @name: name of this remote processor
>>>>   * @ops: platform-specific handlers (mainly start/stop)
>>>> + * @sync_ops: platform-specific handlers for synchronising with MCU
>>>> + * @sync_states: states in which @ops and @sync_ops are to be used
>>>>   * @firmware: name of firmware file to load, can be NULL
>>>>   * @len: length of private data needed by the rproc driver (in bytes)
>>>>   *
>>>> @@ -2061,13 +2093,15 @@ static int rproc_alloc_internals(struct rproc
>>>> *rproc, const char *name,
>>>>   * Note: _never_ directly deallocate @rproc, even if it was not registered
>>>>   * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
>>>>   */
>>>> -struct rproc *rproc_alloc(struct device *dev, const char *name,
>>>> -			  const struct rproc_ops *ops,
>>>> -			  const char *firmware, int len)
>>>> +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
>>>> *name,
>>>> +					const struct rproc_ops *ops,
>>>> +					const struct rproc_ops *sync_ops,
>>>> +					struct rproc_sync_states
>>>> *sync_states,
>>>> +					const char *firmware, int len)
>>
>> Do you foresee the need for sync_ops to be present as long as the rproc
>> is registered? I am wondering if it is better to introduce an API where
>> you can set the ops at runtime rather than allocate it upfront, so that
>> once the initial handling is done, you can reset both the sync_states
>> and ops.
> 
> That is something I spent a fair amount of time thinking about.  I decided to
> proceed with an upfront allocation scheme and see if people would come up
> with needs that would mandate a runtime allocation.  I am willing to provide the
> API if there is a need for it but would rather not if someone "may" need it. 

I think you will have a need of a variant of this atleast as you rework
the series for the comments on the cover-letter [1]

[1] https://patchwork.kernel.org/comment/23260081/

> 
>>
>>
>>>>  {
>>>>  	struct rproc *rproc;
>>>>
>>>> -	if (!dev || !name || !ops)
>>>> +	if (!dev || !name)
>>>>  		return NULL;
>>>>
>>>>  	rproc = kzalloc(sizeof(struct rproc) + len, GFP_KERNEL);
>>>> @@ -2084,8 +2118,8 @@ struct rproc *rproc_alloc(struct device *dev, const
>>>> char *name,
>>>>  	rproc->dev.class = &rproc_class;
>>>>  	rproc->dev.driver_data = rproc;
>>>>
>>>> -	if (rproc_alloc_internals(rproc, name, ops,
>>>> -				  firmware, len))
>>>> +	if (rproc_alloc_internals(rproc, name, ops, sync_ops,
>>>> +				  sync_states, firmware, len))
>>>>  		goto out;
>>>>
>>>>  	/* Assign a unique device index and name */
>>>> @@ -2119,7 +2153,41 @@ struct rproc *rproc_alloc(struct device *dev, const
>>>> char *name,
>>>>  	put_device(&rproc->dev);
>>>>  	return NULL;
>>>>  }
>>>> -EXPORT_SYMBOL(rproc_alloc);
>>>> +EXPORT_SYMBOL(rproc_alloc_state_machine);
>>>> +
>>>> +/**
>>>> + * rproc_alloc() - allocate a remote processor handle
>>>> + * @dev: the underlying device
>>>> + * @name: name of this remote processor
>>>> + * @ops: platform-specific handlers (mainly start/stop)
>>>> + * @firmware: name of firmware file to load, can be NULL
>>>> + * @len: length of private data needed by the rproc driver (in bytes)
>>>> + *
>>>> + * Allocates a new remote processor handle, but does not register
>>>> + * it yet. if @firmware is NULL, a default name is used.
>>>> + *
>>>> + * This function should be used by rproc implementations during
>>>> initialization
>>>> + * of the remote processor.
>>>> + *
>>>> + * After creating an rproc handle using this function, and when ready,
>>>> + * implementations should then call rproc_add() to complete
>>>> + * the registration of the remote processor.
>>>> + *
>>>> + * On success the new rproc is returned, and on failure, NULL.
>>>> + *
>>>> + * Note: _never_ directly deallocate @rproc, even if it was not registered
>>>> + * yet. Instead, when you need to unroll rproc_alloc(), use rproc_free().
>>>> + */
>>>> +struct rproc *rproc_alloc(struct device *dev, const char *name,
>>>> +			  const struct rproc_ops *ops,
>>>> +			  const char *firmware, int len)
>>>> +{
>>>> +	if (!name && !firmware)
>>
>> Retain the original checks on dev, name and ops from the previous
>> rproc_alloc(). A NULL firmware was perfectly valid before, and the name
>> is allocated using the default template.
> 
> Here firmware can be NULL, but we need a name in order to construct the default
> one.  On the flip side ops can be NULL for scenarios where the remoteproc core
> is never in charge of the MCU lifecycle.  As for dev, it is checked in
> rproc_alloc_state_machine().

So, rproc_alloc() continues to be the API for normal use-cases, and
anyone needing the sync lifecycle would be using
rproc_alloc_state_machine(). rproc_alloc() was originally checking for
dev, name, ops. You are checking for the first two at the beginning of
the new function. The ops check is now not done until
rproc_alloc_internals(). I don't see the advantage of the additional
conditional check here, it will just be simpler to check for NULL ops
here, or retain the original checks as they were for better readability
on the mandatory stuff on this function.

regards
Suman

> 
>>
>>>> +		return NULL;
>>>> +
>>>> +	return rproc_alloc_state_machine(dev, name, ops, NULL, NULL,
>>>> +					 firmware, len);
>>>> +}
>>
>> Missing the EXPORT_SYMBOL on rproc_alloc() -> it is an API used by modules.
> 
> Of course yes, good catch.
> 
>>
>> regards
>> Suman
>>
>>>>
>>>>  /**
>>>>   * rproc_free() - unroll rproc_alloc()
>>>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
>>>> index d115e47d702d..d1214487daac 100644
>>>> --- a/include/linux/remoteproc.h
>>>> +++ b/include/linux/remoteproc.h
>>>> @@ -611,6 +611,11 @@ struct rproc *rproc_get_by_child(struct device
>>>> *dev);
>>>>  struct rproc *rproc_alloc(struct device *dev, const char *name,
>>>>  			  const struct rproc_ops *ops,
>>>>  			  const char *firmware, int len);
>>>> +struct rproc *rproc_alloc_state_machine(struct device *dev, const char
>>>> *name,
>>>> +					const struct rproc_ops *ops,
>>>> +					const struct rproc_ops *sync_ops,
>>>> +					struct rproc_sync_states
>>>> *sync_states,
>>>> +					const char *firmware, int len);
>>>>  void rproc_put(struct rproc *rproc);
>>>>  int rproc_add(struct rproc *rproc);
>>>>  int rproc_del(struct rproc *rproc);
>>>> --
>>>> 2.20.1
>>>
>>

Re: [PATCH v2 09/17] remoteproc: Call the right core function based on synchronisation state

[Suman Anna]

On 4/2/20 3:16 PM, Mathieu Poirier wrote:
> On Tue, Mar 31, 2020 at 10:10:50AM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>
>> On 3/24/20 4:45 PM, Mathieu Poirier wrote:
>>> Call the right core function based on whether we should synchronise
>>> with an MCU or boot it from scratch.
>>
>> This patch does generate some checkpatch warnings.
> 
> Right, checkpatch is complaining but other than duplicating the same code for
> all functions, I don't see another way to do this.
> 
>>
>>>
>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>> ---
>>>  drivers/remoteproc/remoteproc_internal.h | 36 +++++++++++-------------
>>>  1 file changed, 17 insertions(+), 19 deletions(-)
>>>
>>> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
>>> index 73ea32df0156..5f711ceb97ba 100644
>>> --- a/drivers/remoteproc/remoteproc_internal.h
>>> +++ b/drivers/remoteproc/remoteproc_internal.h
>>> @@ -106,38 +106,41 @@ static inline void rproc_set_mcu_sync_state(struct rproc *rproc,
>>>  	}
>>>  }
>>>  
>>> +#define RPROC_OPS_HELPER(__operation, ...)				\
>>> +	do {								\
>>> +		if (rproc_sync_with_mcu(rproc)) {			\
>>
>> So this does make the logic a bit convoluted, since you have three
>> different flags for rproc_sync_with_mcu, and you apply them in common
>> for all the ops. This is what I meant in my comment on Patch 1.
> 
> There is indeed 3 different flags but they are only valid in a specific state.
> What "ops" are you referring to here?

All the rproc_ops callbacks, since only a subset of each might be valid
at each of the three different states. Granted this provides the maximum
flexibility for platform drivers, but it's a bit convoluted. Kinda goes
with Loic's comment on Patch 7 [1].

[1] https://patchwork.kernel.org/comment/23249975/


I'm also not sure about the comment in
> "patch 1" - which one would that be and how does it relate to the current block
> of code.  Apologies, I need more clarifications.  

I meant the following comment,
"And I am wondering if it is actually better to introduce the sync state
to check against here, rather than using the stored sync state and
return. The current way makes it confusing to read the state machine."

regards
Suman

> 
>>
>>> +			if (!rproc->sync_ops ||				\
>>> +			    !rproc->sync_ops->__operation)		\
>>> +				return 0;				\
>>> +			return rproc->sync_ops->__operation(__VA_ARGS__); \
>>
>> Use the same semantics as the regular ops instead of two return
>> statements, the code should fallback to the common return 0 after the
>> RPROC_OPS_HELPER when neither of them are present.
> 
> Yes the tests are exactly the same, no reason to proceed differently.
> 
>>
>> regards
>> Suman
>>
>>> +		} else if (rproc->ops && rproc->ops->__operation)	\
>>> +			return rproc->ops->__operation(__VA_ARGS__);	\
>>> +	} while (0)							\
>>> +
>>>  static inline
>>>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->sanity_check)
>>> -		return rproc->ops->sanity_check(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(sanity_check, rproc, fw);
>>>  	return 0;
>>>  }
>>>  
>>>  static inline
>>>  u32 rproc_get_boot_addr(struct rproc *rproc, const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->get_boot_addr)
>>> -		return rproc->ops->get_boot_addr(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(get_boot_addr, rproc, fw);
>>>  	return 0;
>>>  }
>>>  
>>>  static inline
>>>  int rproc_load_segments(struct rproc *rproc, const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->load)
>>> -		return rproc->ops->load(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(load, rproc, fw);
>>>  	return -EINVAL;
>>>  }
>>>  
>>>  static inline int rproc_parse_fw(struct rproc *rproc, const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->parse_fw)
>>> -		return rproc->ops->parse_fw(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(parse_fw, rproc, fw);
>>>  	return 0;
>>>  }
>>>  
>>> @@ -145,10 +148,7 @@ static inline
>>>  int rproc_handle_rsc(struct rproc *rproc, u32 rsc_type, void *rsc, int offset,
>>>  		     int avail)
>>>  {
>>> -	if (rproc->ops->handle_rsc)
>>> -		return rproc->ops->handle_rsc(rproc, rsc_type, rsc, offset,
>>> -					      avail);
>>> -
>>> +	RPROC_OPS_HELPER(handle_rsc, rproc, rsc_type, rsc, offset, avail);
>>>  	return RSC_IGNORED;
>>>  }
>>>  
>>> @@ -156,9 +156,7 @@ static inline
>>>  struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
>>>  						   const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->find_loaded_rsc_table)
>>> -		return rproc->ops->find_loaded_rsc_table(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(find_loaded_rsc_table, rproc, fw);
>>>  	return NULL;
>>>  }
>>>  
>>>
>>

Re: [PATCH v2 09/17] remoteproc: Call the right core function based on synchronisation state

[Suman Anna]

On 4/2/20 3:16 PM, Mathieu Poirier wrote:
> On Tue, Mar 31, 2020 at 10:10:50AM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>
>> On 3/24/20 4:45 PM, Mathieu Poirier wrote:
>>> Call the right core function based on whether we should synchronise
>>> with an MCU or boot it from scratch.
>>
>> This patch does generate some checkpatch warnings.
> 
> Right, checkpatch is complaining but other than duplicating the same code for
> all functions, I don't see another way to do this.
> 
>>
>>>
>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>> ---
>>>  drivers/remoteproc/remoteproc_internal.h | 36 +++++++++++-------------
>>>  1 file changed, 17 insertions(+), 19 deletions(-)
>>>
>>> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
>>> index 73ea32df0156..5f711ceb97ba 100644
>>> --- a/drivers/remoteproc/remoteproc_internal.h
>>> +++ b/drivers/remoteproc/remoteproc_internal.h
>>> @@ -106,38 +106,41 @@ static inline void rproc_set_mcu_sync_state(struct rproc *rproc,
>>>  	}
>>>  }
>>>  
>>> +#define RPROC_OPS_HELPER(__operation, ...)				\
>>> +	do {								\
>>> +		if (rproc_sync_with_mcu(rproc)) {			\
>>
>> So this does make the logic a bit convoluted, since you have three
>> different flags for rproc_sync_with_mcu, and you apply them in common
>> for all the ops. This is what I meant in my comment on Patch 1.
> 
> There is indeed 3 different flags but they are only valid in a specific state.
> What "ops" are you referring to here?

All the rproc_ops callbacks, since only a subset of each might be valid
at each of the three different states. Granted this provides the maximum
flexibility for platform drivers, but it's a bit convoluted. Kinda goes
with Loic's comment on Patch 7 [1].

[1] https://patchwork.kernel.org/comment/23249975/


I'm also not sure about the comment in
> "patch 1" - which one would that be and how does it relate to the current block
> of code.  Apologies, I need more clarifications.  

I meant the following comment,
"And I am wondering if it is actually better to introduce the sync state
to check against here, rather than using the stored sync state and
return. The current way makes it confusing to read the state machine."

regards
Suman

> 
>>
>>> +			if (!rproc->sync_ops ||				\
>>> +			    !rproc->sync_ops->__operation)		\
>>> +				return 0;				\
>>> +			return rproc->sync_ops->__operation(__VA_ARGS__); \
>>
>> Use the same semantics as the regular ops instead of two return
>> statements, the code should fallback to the common return 0 after the
>> RPROC_OPS_HELPER when neither of them are present.
> 
> Yes the tests are exactly the same, no reason to proceed differently.
> 
>>
>> regards
>> Suman
>>
>>> +		} else if (rproc->ops && rproc->ops->__operation)	\
>>> +			return rproc->ops->__operation(__VA_ARGS__);	\
>>> +	} while (0)							\
>>> +
>>>  static inline
>>>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->sanity_check)
>>> -		return rproc->ops->sanity_check(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(sanity_check, rproc, fw);
>>>  	return 0;
>>>  }
>>>  
>>>  static inline
>>>  u32 rproc_get_boot_addr(struct rproc *rproc, const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->get_boot_addr)
>>> -		return rproc->ops->get_boot_addr(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(get_boot_addr, rproc, fw);
>>>  	return 0;
>>>  }
>>>  
>>>  static inline
>>>  int rproc_load_segments(struct rproc *rproc, const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->load)
>>> -		return rproc->ops->load(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(load, rproc, fw);
>>>  	return -EINVAL;
>>>  }
>>>  
>>>  static inline int rproc_parse_fw(struct rproc *rproc, const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->parse_fw)
>>> -		return rproc->ops->parse_fw(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(parse_fw, rproc, fw);
>>>  	return 0;
>>>  }
>>>  
>>> @@ -145,10 +148,7 @@ static inline
>>>  int rproc_handle_rsc(struct rproc *rproc, u32 rsc_type, void *rsc, int offset,
>>>  		     int avail)
>>>  {
>>> -	if (rproc->ops->handle_rsc)
>>> -		return rproc->ops->handle_rsc(rproc, rsc_type, rsc, offset,
>>> -					      avail);
>>> -
>>> +	RPROC_OPS_HELPER(handle_rsc, rproc, rsc_type, rsc, offset, avail);
>>>  	return RSC_IGNORED;
>>>  }
>>>  
>>> @@ -156,9 +156,7 @@ static inline
>>>  struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
>>>  						   const struct firmware *fw)
>>>  {
>>> -	if (rproc->ops->find_loaded_rsc_table)
>>> -		return rproc->ops->find_loaded_rsc_table(rproc, fw);
>>> -
>>> +	RPROC_OPS_HELPER(find_loaded_rsc_table, rproc, fw);
>>>  	return NULL;
>>>  }
>>>  
>>>
>>

Re: [PATCH v2 14/17] remoteproc: Refactor function rproc_trigger_recovery()

[Suman Anna]

On 4/2/20 3:35 PM, Mathieu Poirier wrote:
> On Tue, Mar 31, 2020 at 04:52:12PM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>
>> On 3/24/20 4:46 PM, Mathieu Poirier wrote:
>>> Refactor function rproc_trigger_recovery() in order to avoid
>>> reloading the fw image when synchronising with an MCU rather than
>>> booting it.
>>>
>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>> ---
>>>  drivers/remoteproc/remoteproc_core.c | 16 +++++++++-------
>>>  1 file changed, 9 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
>>> index d3c4d7e6ca25..dbb0a8467205 100644
>>> --- a/drivers/remoteproc/remoteproc_core.c
>>> +++ b/drivers/remoteproc/remoteproc_core.c
>>> @@ -1661,7 +1661,7 @@ static void rproc_coredump(struct rproc *rproc)
>>>   */
>>>  int rproc_trigger_recovery(struct rproc *rproc)
>>>  {
>>> -	const struct firmware *firmware_p;
>>> +	const struct firmware *firmware_p = NULL;
>>>  	struct device *dev = &rproc->dev;
>>>  	int ret;
>>>  
>>> @@ -1678,14 +1678,16 @@ int rproc_trigger_recovery(struct rproc *rproc)
>>>  	/* generate coredump */
>>>  	rproc_coredump(rproc);
>>>  
>>> -	/* load firmware */
>>> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
>>> -	if (ret < 0) {
>>> -		dev_err(dev, "request_firmware failed: %d\n", ret);
>>> -		goto unlock_mutex;
>>> +	/* load firmware if need be */
>>> +	if (!rproc_sync_with_mcu(rproc)) {
>>> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
>>> +		if (ret < 0) {
>>> +			dev_err(dev, "request_firmware failed: %d\n", ret);
>>> +			goto unlock_mutex;
>>> +		}
>>
>> So, I am trying to understand the need for the flag around
>> RPROC_SYNC_STATE_CRASHED. Can you explain what all usecases that is
>> covering?
> 
> There could scenarios where another entity is in charge of the entire MCU
> lifecycle.  That entity could be able to recognise the MCU has crashed and
> automatically boot it again, in which case all the remoteproc core needs to do
> is synchronise with it.

So, Linux won't be responsible for error recovery in that case, and
wondering why this function would even be called in such a scenario?

> 
> But it could also be that another entity has booted the MCU when the system
> started but if the MCU crashes or is manually stopped, then the AP becomes the
> MCU lifecycle.

Yeah, this is more of a standard early-boot by bootloader scenario which
should be satisfied by just the on_init state.

I mostly still trying to understand the usecase here.

regards
Suman


> 
>>
>> In anycase, you should probably combine this piece with the flag change
>> for STATE_CRASHED on the last patch.
> 
> Sure.
> 
>>
>> regards
>> Suman
>>
>>>  	}
>>>  
>>> -	/* boot the remote processor up again */
>>> +	/* boot up or synchronise with the remote processor again */
>>>  	ret = rproc_start(rproc, firmware_p);
>>>  
>>>  	release_firmware(firmware_p);
>>>
>>

Re: [PATCH v2 14/17] remoteproc: Refactor function rproc_trigger_recovery()

[Suman Anna]

On 4/2/20 3:35 PM, Mathieu Poirier wrote:
> On Tue, Mar 31, 2020 at 04:52:12PM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>
>> On 3/24/20 4:46 PM, Mathieu Poirier wrote:
>>> Refactor function rproc_trigger_recovery() in order to avoid
>>> reloading the fw image when synchronising with an MCU rather than
>>> booting it.
>>>
>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>> ---
>>>  drivers/remoteproc/remoteproc_core.c | 16 +++++++++-------
>>>  1 file changed, 9 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
>>> index d3c4d7e6ca25..dbb0a8467205 100644
>>> --- a/drivers/remoteproc/remoteproc_core.c
>>> +++ b/drivers/remoteproc/remoteproc_core.c
>>> @@ -1661,7 +1661,7 @@ static void rproc_coredump(struct rproc *rproc)
>>>   */
>>>  int rproc_trigger_recovery(struct rproc *rproc)
>>>  {
>>> -	const struct firmware *firmware_p;
>>> +	const struct firmware *firmware_p = NULL;
>>>  	struct device *dev = &rproc->dev;
>>>  	int ret;
>>>  
>>> @@ -1678,14 +1678,16 @@ int rproc_trigger_recovery(struct rproc *rproc)
>>>  	/* generate coredump */
>>>  	rproc_coredump(rproc);
>>>  
>>> -	/* load firmware */
>>> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
>>> -	if (ret < 0) {
>>> -		dev_err(dev, "request_firmware failed: %d\n", ret);
>>> -		goto unlock_mutex;
>>> +	/* load firmware if need be */
>>> +	if (!rproc_sync_with_mcu(rproc)) {
>>> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
>>> +		if (ret < 0) {
>>> +			dev_err(dev, "request_firmware failed: %d\n", ret);
>>> +			goto unlock_mutex;
>>> +		}
>>
>> So, I am trying to understand the need for the flag around
>> RPROC_SYNC_STATE_CRASHED. Can you explain what all usecases that is
>> covering?
> 
> There could scenarios where another entity is in charge of the entire MCU
> lifecycle.  That entity could be able to recognise the MCU has crashed and
> automatically boot it again, in which case all the remoteproc core needs to do
> is synchronise with it.

So, Linux won't be responsible for error recovery in that case, and
wondering why this function would even be called in such a scenario?

> 
> But it could also be that another entity has booted the MCU when the system
> started but if the MCU crashes or is manually stopped, then the AP becomes the
> MCU lifecycle.

Yeah, this is more of a standard early-boot by bootloader scenario which
should be satisfied by just the on_init state.

I mostly still trying to understand the usecase here.

regards
Suman


> 
>>
>> In anycase, you should probably combine this piece with the flag change
>> for STATE_CRASHED on the last patch.
> 
> Sure.
> 
>>
>> regards
>> Suman
>>
>>>  	}
>>>  
>>> -	/* boot the remote processor up again */
>>> +	/* boot up or synchronise with the remote processor again */
>>>  	ret = rproc_start(rproc, firmware_p);
>>>  
>>>  	release_firmware(firmware_p);
>>>
>>

Re: [PATCH v2 16/17] remoteproc: Correctly deal with MCU synchronisation when changing state

[Suman Anna]

On 4/2/20 3:42 PM, Mathieu Poirier wrote:
> Hi Suman,
> 
> On Tue, Mar 31, 2020 at 05:35:58PM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>
>> On 3/30/20 6:49 PM, Mathieu Poirier wrote:
>>> On Fri, Mar 27, 2020 at 02:04:36PM +0000, Loic PALLARDY wrote:
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>>> Sent: mardi 24 mars 2020 22:46
>>>>> To: bjorn.andersson@linaro.org
>>>>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
>>>>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
>>>>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
>>>>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
>>>>> Subject: [PATCH v2 16/17] remoteproc: Correctly deal with MCU
>>>>> synchronisation when changing state
>>>>>
>>>>> This patch deals with state changes when synchronising with an MCU. More
>>>>> specifically it prevents the MCU from being started if it already has been
>>>>> started by another entity.  Similarly it prevents the AP from stopping the
>>>>> MCU if it hasn't been given the capability by platform firmware.
>>>>>
>>>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>>> ---
>>>>>  drivers/remoteproc/remoteproc_sysfs.c | 32
>>>>> ++++++++++++++++++++++++++-
>>>>>  1 file changed, 31 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
>>>>> b/drivers/remoteproc/remoteproc_sysfs.c
>>>>> index 4956577ad4b4..741a3c152b82 100644
>>>>> --- a/drivers/remoteproc/remoteproc_sysfs.c
>>>>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
>>>>> @@ -108,6 +108,29 @@ static ssize_t state_show(struct device *dev, struct
>>>>> device_attribute *attr,
>>>>>  	return sprintf(buf, "%s\n", rproc_state_string[state]);
>>>>>  }
>>>>>
>>>>> +static int rproc_can_shutdown(struct rproc *rproc)
>>>>> +{
>>>>> +	/* The MCU is not running, obviously an invalid operation. */
>>>>> +	if (rproc->state != RPROC_RUNNING)
>>>>> +		return false;
>>>>> +
>>>>> +	/*
>>>>> +	 * The MCU is not running (see above) and the remoteproc core is
>>>>> the
>>>>> +	 * lifecycle manager, no problem calling for a shutdown.
>>>>> +	 */
>>>>> +	if (!rproc_sync_with_mcu(rproc))
>>>>> +		return true;
>>>>> +
>>>>> +	/*
>>>>> +	 * The MCU has been loaded by another entity (see above) and the
>>>>> +	 * platform code has _not_ given us the capability of stopping it.
>>>>> +	 */
>>>>> +	if (!rproc->sync_ops->stop)
>>>>> +		return false;
>>>>
>>>> Test could be simplified
>>>> if (rproc_sync_with_mcu(rproc)) && !rproc->sync_ops->stop)
>>>> 	return false;
>>>
>>> I laid out the test individually on purpose.  That way there is no coupling
>>> between conditions, it is plane to see what is going on and remains maintainable
>>> as we add new tests.
>>>
>>>>
>>>>> +
>>>>> +	return true;
>>>>> +}
>>>>> +
>>>>>  /* Change remote processor state via sysfs */
>>>>>  static ssize_t state_store(struct device *dev,
>>>>>  			      struct device_attribute *attr,
>>>>> @@ -120,11 +143,18 @@ static ssize_t state_store(struct device *dev,
>>>>>  		if (rproc->state == RPROC_RUNNING)
>>>>>  			return -EBUSY;
>>>>>
>>>>> +		/*
>>>>> +		 * In synchronisation mode, booting the MCU is the
>>>>> +		 * responsibility of an external entity.
>>>>> +		 */
>>>>> +		if (rproc_sync_with_mcu(rproc))
>>>>> +			return -EINVAL;
>>>>> +
>>>> I don't understand this restriction, simply because it is preventing to resynchronize with a
>>>> coprocessor after a "stop".
>>
>> There's actually one more scenario even without "stop". If auto_boot is
>> set to false, then rproc_actuate will never get called.
>>
>>>> In the following configuration which can be configuration for coprocessor with romed/flashed
>>>> firmware (no reload needed):
>>>> on_init = true
>>>> after_stop = true
>>>> after_crash = true
>>>> Once you stop it via sysfs interface, you can't anymore restart/resync to it.
>>>
>>> Very true.  The MCU will get restarted by another entity but the AP won't
>>> synchronise with it.  I need more time to think about the best way to deal with
>>> this and may have to get back to you for further discussions.
>>>
>>>>
>>>> I think it will be better to modify rproc_boot() to take into account rproc_sync_with_mcu()
>>>> as below:
>>>>
>>>> int rproc_boot(struct rproc *rproc)
>>>>  {
>>>> -	const struct firmware *firmware_p;
>>>> +	const struct firmware *firmware_p = NULL;
>>>>  	struct device *dev = &rproc->dev;
>>>>  	int ret;
>>>>  
>>>>  	if (!rproc) {
>>>>  		pr_err("invalid rproc handle\n");
>>>>  		return -EINVAL;
>>>>  	}
>>>>  
>>>>  	/* load firmware */
>>>> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
>>>> -	if (ret < 0) {
>>>> -		dev_err(dev, "request_firmware failed: %d\n", ret);
>>>> -		return ret;
>>>> +	if (!rproc_sync_with_mcu(rproc)) {
>>
>> I guess this is what the original skip_fw_load was doing. And with the
>> current series, the userspace loading support usecase I have cannot be
>> achieved. If this is added back, I can try if that works for my usecases.
> 
> I didn't notice this comment upon first read... Can you give me more details on
> what your usecase is order to see how best to deal with it?

We have a userspace loader usecase, where we use a daemon/server that
does the loading, and talks to the keystone remoteproc driver over a
character driver to publish the resource table and boot vectors, and to
start/stop the processor. You can find more details on the downstream
commit [1] we have. We exercise the regular kernel rproc state-machine
and suppress the firmware segment loading portion of it.

regards
Suman

[1]
https://git.ti.com/gitweb?p=rpmsg/remoteproc.git;a=commit;h=c41f591ccaaeef66bd4ccbb883ae72f6b0d173d7

> 
> Thanks,
> Mathieu
> 
>>
>>>> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
>>>> +		if (ret < 0) {
>>>> +			dev_err(dev, "request_firmware failed: %d\n", ret);
>>>> +			return ret;
>>>> +		}
>>>>  	}
>>>>  
>>>>  	ret = rproc_actuate(rproc, firmware_p);
>>>>  
>>>> -	release_firmware(firmware_p);
>>>> +	if (firmware_p)
>>>> +		release_firmware(firmware_p);
>>>>  
>>>>  	return ret;
>>>>  }
>>>>  
>>>> Thanks to these modifications, I'm able to resync after a stop with coprocessor without reloading firmware.
>>>>
>>>>>  		ret = rproc_boot(rproc);
>>>>>  		if (ret)
>>>>>  			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
>>>>>  	} else if (sysfs_streq(buf, "stop")) {
>>>>> -		if (rproc->state != RPROC_RUNNING)
>>>>> +		if (!rproc_can_shutdown(rproc))
>>>>>  			return -EINVAL;
>>>>>
>>>>>  		rproc_shutdown(rproc);
>>>> As rproc shutdown is also accessible as kernel API, I propose to move
>>>> rproc_can_shutdown() check inside rproc_shutdown() and to test
>>>> returned error
>>>
>>> Ah yes, it is public...  As you point out, I think we'll need to move
>>> rproc_can_shutdown() in rproc_shutdown().
>>
>> I am assuming only the new conditions, right?
>>
>> regards
>> Suman
>>
>>>
>>> Thank you for taking the time to review this set,
>>> Mathieu
>>>

Re: [PATCH v2 16/17] remoteproc: Correctly deal with MCU synchronisation when changing state

[Suman Anna]

On 4/1/20 4:29 PM, Mathieu Poirier wrote:
> On Tue, Mar 31, 2020 at 05:35:58PM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>
>> On 3/30/20 6:49 PM, Mathieu Poirier wrote:
>>> On Fri, Mar 27, 2020 at 02:04:36PM +0000, Loic PALLARDY wrote:
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>>> Sent: mardi 24 mars 2020 22:46
>>>>> To: bjorn.andersson@linaro.org
>>>>> Cc: ohad@wizery.com; Loic PALLARDY <loic.pallardy@st.com>; s-
>>>>> anna@ti.com; peng.fan@nxp.com; Arnaud POULIQUEN
>>>>> <arnaud.pouliquen@st.com>; Fabien DESSENNE
>>>>> <fabien.dessenne@st.com>; linux-remoteproc@vger.kernel.org
>>>>> Subject: [PATCH v2 16/17] remoteproc: Correctly deal with MCU
>>>>> synchronisation when changing state
>>>>>
>>>>> This patch deals with state changes when synchronising with an MCU. More
>>>>> specifically it prevents the MCU from being started if it already has been
>>>>> started by another entity.  Similarly it prevents the AP from stopping the
>>>>> MCU if it hasn't been given the capability by platform firmware.
>>>>>
>>>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>>> ---
>>>>>  drivers/remoteproc/remoteproc_sysfs.c | 32
>>>>> ++++++++++++++++++++++++++-
>>>>>  1 file changed, 31 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
>>>>> b/drivers/remoteproc/remoteproc_sysfs.c
>>>>> index 4956577ad4b4..741a3c152b82 100644
>>>>> --- a/drivers/remoteproc/remoteproc_sysfs.c
>>>>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
>>>>> @@ -108,6 +108,29 @@ static ssize_t state_show(struct device *dev, struct
>>>>> device_attribute *attr,
>>>>>  	return sprintf(buf, "%s\n", rproc_state_string[state]);
>>>>>  }
>>>>>
>>>>> +static int rproc_can_shutdown(struct rproc *rproc)
>>>>> +{
>>>>> +	/* The MCU is not running, obviously an invalid operation. */
>>>>> +	if (rproc->state != RPROC_RUNNING)
>>>>> +		return false;
>>>>> +
>>>>> +	/*
>>>>> +	 * The MCU is not running (see above) and the remoteproc core is
>>>>> the
>>>>> +	 * lifecycle manager, no problem calling for a shutdown.
>>>>> +	 */
>>>>> +	if (!rproc_sync_with_mcu(rproc))
>>>>> +		return true;
>>>>> +
>>>>> +	/*
>>>>> +	 * The MCU has been loaded by another entity (see above) and the
>>>>> +	 * platform code has _not_ given us the capability of stopping it.
>>>>> +	 */
>>>>> +	if (!rproc->sync_ops->stop)
>>>>> +		return false;
>>>>
>>>> Test could be simplified
>>>> if (rproc_sync_with_mcu(rproc)) && !rproc->sync_ops->stop)
>>>> 	return false;
>>>
>>> I laid out the test individually on purpose.  That way there is no coupling
>>> between conditions, it is plane to see what is going on and remains maintainable
>>> as we add new tests.
>>>
>>>>
>>>>> +
>>>>> +	return true;
>>>>> +}
>>>>> +
>>>>>  /* Change remote processor state via sysfs */
>>>>>  static ssize_t state_store(struct device *dev,
>>>>>  			      struct device_attribute *attr,
>>>>> @@ -120,11 +143,18 @@ static ssize_t state_store(struct device *dev,
>>>>>  		if (rproc->state == RPROC_RUNNING)
>>>>>  			return -EBUSY;
>>>>>
>>>>> +		/*
>>>>> +		 * In synchronisation mode, booting the MCU is the
>>>>> +		 * responsibility of an external entity.
>>>>> +		 */
>>>>> +		if (rproc_sync_with_mcu(rproc))
>>>>> +			return -EINVAL;
>>>>> +
>>>> I don't understand this restriction, simply because it is preventing to resynchronize with a
>>>> coprocessor after a "stop".
>>
>> There's actually one more scenario even without "stop". If auto_boot is
>> set to false, then rproc_actuate will never get called.
> 
> I fail to fully understand the situation you are describing - can you give me
> more information?

The platform driver has auto_boot set to false and sync_states set, so
rproc_trigger_auto_initiate() never gets called in rproc_add(), and
rproc_actuate() doesn't happen either. rproc->state is not RUNNING, and
you will bail out here, and there is no way to start it either.

> 
>>
>>>> In the following configuration which can be configuration for coprocessor with romed/flashed
>>>> firmware (no reload needed):
>>>> on_init = true
>>>> after_stop = true
>>>> after_crash = true
>>>> Once you stop it via sysfs interface, you can't anymore restart/resync to it.
>>>
>>> Very true.  The MCU will get restarted by another entity but the AP won't
>>> synchronise with it.  I need more time to think about the best way to deal with
>>> this and may have to get back to you for further discussions.
>>>
>>>>
>>>> I think it will be better to modify rproc_boot() to take into account rproc_sync_with_mcu()
>>>> as below:
>>>>
>>>> int rproc_boot(struct rproc *rproc)
>>>>  {
>>>> -	const struct firmware *firmware_p;
>>>> +	const struct firmware *firmware_p = NULL;
>>>>  	struct device *dev = &rproc->dev;
>>>>  	int ret;
>>>>  
>>>>  	if (!rproc) {
>>>>  		pr_err("invalid rproc handle\n");
>>>>  		return -EINVAL;
>>>>  	}
>>>>  
>>>>  	/* load firmware */
>>>> -	ret = request_firmware(&firmware_p, rproc->firmware, dev);
>>>> -	if (ret < 0) {
>>>> -		dev_err(dev, "request_firmware failed: %d\n", ret);
>>>> -		return ret;
>>>> +	if (!rproc_sync_with_mcu(rproc)) {
>>
>> I guess this is what the original skip_fw_load was doing. And with the
>> current series, the userspace loading support usecase I have cannot be
>> achieved. If this is added back, I can try if that works for my usecases.
>>
>>>> +		ret = request_firmware(&firmware_p, rproc->firmware, dev);
>>>> +		if (ret < 0) {
>>>> +			dev_err(dev, "request_firmware failed: %d\n", ret);
>>>> +			return ret;
>>>> +		}
>>>>  	}
>>>>  
>>>>  	ret = rproc_actuate(rproc, firmware_p);
>>>>  
>>>> -	release_firmware(firmware_p);
>>>> +	if (firmware_p)
>>>> +		release_firmware(firmware_p);
>>>>  
>>>>  	return ret;
>>>>  }
>>>>  
>>>> Thanks to these modifications, I'm able to resync after a stop with coprocessor without reloading firmware.
>>>>
>>>>>  		ret = rproc_boot(rproc);
>>>>>  		if (ret)
>>>>>  			dev_err(&rproc->dev, "Boot failed: %d\n", ret);
>>>>>  	} else if (sysfs_streq(buf, "stop")) {
>>>>> -		if (rproc->state != RPROC_RUNNING)
>>>>> +		if (!rproc_can_shutdown(rproc))
>>>>>  			return -EINVAL;
>>>>>
>>>>>  		rproc_shutdown(rproc);
>>>> As rproc shutdown is also accessible as kernel API, I propose to move
>>>> rproc_can_shutdown() check inside rproc_shutdown() and to test
>>>> returned error
>>>
>>> Ah yes, it is public...  As you point out, I think we'll need to move
>>> rproc_can_shutdown() in rproc_shutdown().
>>
>> I am assuming only the new conditions, right?
> 
> Once again I fail to grasp the extent of your comment.  Can you be more specific
> about the "new conditions"?

rproc_can_shutdown() has three checks, where the first check is existing
check that got moved, and the other 2 are the new checks that deal with
the sync states. So, I meant these 2 new sync checks.

regards
Suman

> 
> Thanks
> Mathieu
> 
>>
>> regards
>> Suman
>>
>>>
>>> Thank you for taking the time to review this set,
>>> Mathieu
>>>

Re: [PATCH v2 01/17] remoteproc: Add new operation and state machine for MCU synchronisation

[Suman Anna]

On 4/1/20 4:53 PM, Mathieu Poirier wrote:
> On Mon, Mar 30, 2020 at 05:49:11PM -0500, Suman Anna wrote:
>> On 3/30/20 5:46 PM, Suman Anna wrote:
>>> Hi Mathieu,
>>>
>>> On 3/24/20 4:45 PM, Mathieu Poirier wrote:
>>>> Add a new rproc_ops sync_ops to support use cases where the remoteproc
>>>> core is synchronisting with the MCU.  When exactly to use the sync_ops is
>>>
>>> typo on syschronisting..
>>>
>>>> directed by the states in struct rproc_sync_states.
>>>>
>>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>> ---
>>>>  drivers/remoteproc/remoteproc_debugfs.c  | 31 ++++++++++++++++++++++++
>>>>  drivers/remoteproc/remoteproc_internal.h |  5 ++++
>>>>  include/linux/remoteproc.h               | 23 +++++++++++++++++-
>>>>  3 files changed, 58 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/remoteproc/remoteproc_debugfs.c b/drivers/remoteproc/remoteproc_debugfs.c
>>>> index dd93cf04e17f..187bcc67f997 100644
>>>> --- a/drivers/remoteproc/remoteproc_debugfs.c
>>>> +++ b/drivers/remoteproc/remoteproc_debugfs.c
>>>> @@ -311,6 +311,35 @@ static const struct file_operations rproc_carveouts_ops = {
>>>>  	.release	= single_release,
>>>>  };
>>>>  
>>>> +/* Expose synchronisation states via debugfs */
>>>> +static int rproc_sync_states_show(struct seq_file *seq, void *p)
>>>> +{
>>>> +	struct rproc *rproc = seq->private;
>>>> +
>>>> +	seq_printf(seq, "Sync with MCU: %s\n",
>>>> +		   rproc->sync_with_mcu ? "true" : "false");
>>>> +	seq_printf(seq, "On init: %s\n",
>>>> +		   rproc->sync_states->on_init ? "true" : "false");
>>>> +	seq_printf(seq, "After stop: %s\n",
>>>> +		   rproc->sync_states->after_stop ? "true" : "false");
>>>> +	seq_printf(seq, "After crash: %s\n",
>>>> +		   rproc->sync_states->after_crash ? "true" : "false");
>>>> +
>>>> +	return 0;
>>>> +}
>>>> +
>>>> +static int rproc_sync_states_open(struct inode *inode, struct file *file)
>>>> +{
>>>> +	return single_open(file, rproc_sync_states_show, inode->i_private);
>>>> +}
>>>> +
>>>> +static const struct file_operations rproc_sync_states_ops = {
>>>> +	.open		= rproc_sync_states_open,
>>>> +	.read		= seq_read,
>>>> +	.llseek		= seq_lseek,
>>>> +	.release	= single_release,
>>>> +};
>>>> +
>>>>  void rproc_remove_trace_file(struct dentry *tfile)
>>>>  {
>>>>  	debugfs_remove(tfile);
>>>> @@ -357,6 +386,8 @@ void rproc_create_debug_dir(struct rproc *rproc)
>>>>  			    rproc, &rproc_rsc_table_ops);
>>>>  	debugfs_create_file("carveout_memories", 0400, rproc->dbg_dir,
>>>>  			    rproc, &rproc_carveouts_ops);
>>>> +	debugfs_create_file("sync_states", 0400, rproc->dbg_dir,
>>>> +			    rproc, &rproc_sync_states_ops);
>>>>  }
>>>>  
>>>>  void __init rproc_init_debugfs(void)
>>>> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
>>>> index 493ef9262411..5c93de5e00bb 100644
>>>> --- a/drivers/remoteproc/remoteproc_internal.h
>>>> +++ b/drivers/remoteproc/remoteproc_internal.h
>>>> @@ -63,6 +63,11 @@ struct resource_table *rproc_elf_find_loaded_rsc_table(struct rproc *rproc,
>>>>  struct rproc_mem_entry *
>>>>  rproc_find_carveout_by_name(struct rproc *rproc, const char *name, ...);
>>>>  
>>>> +static inline bool rproc_sync_with_mcu(struct rproc *rproc)
>>>> +{
>>>> +	return rproc->sync_with_mcu;
>>>> +}
>>>> +
>>>
>>> Since you are using this mostly for checking and as a boolean, I suggest
>>> you rename this appropriately, something like rproc_needs_sync,
>>> rproc_has_sync or rproc_uses_sync().
> 
> I will rename to rproc_syncing_with_rproc()

Hmm, I want this to reflect a boolean answer for better code
flow/readability.

> 
> 
>>>
>>> And I am wondering if it is actually better to introduce the sync state
>>> to check against here, rather than using the stored sync state and
>>> return. The current way makes it confusing to read the state machine.
> 
> I decided to proceed this way because there may not be a direct correlation
> between the current synchronisation state and the location of the check itself.
> for instance, in firmware_show(), what sync state should be key on?

Yeah OK. Its the combinations of ops (11 callbacks) plus sync states (3)
that kinda makes it hard to read the state-machine.

> 
>>>
>>>>  static inline
>>>>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
>>>>  {
>>>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
>>>> index 16ad66683ad0..d115e47d702d 100644
>>>> --- a/include/linux/remoteproc.h
>>>> +++ b/include/linux/remoteproc.h
>>>> @@ -353,6 +353,21 @@ enum rsc_handling_status {
>>>>  	RSC_IGNORED	= 1,
>>>>  };
>>>>  
>>>> +/**
>>>> + * struct rproc_sync_states - platform specific states indicating which
>>>> + *			      rproc_ops to use at specific times during
>>>> + *			      the MCU lifecycle.
>>>> + * @on_init: true if synchronising with MCU at system initialisation time
>>>> + * @after_stop: true if synchronising with MCU after stopped from the
>>>> + *		command line
>>>> + * @after_crash: true if synchonising with MCU after the MCU has crashed
>>>> + */
>>>> +struct rproc_sync_states {
>>>> +	bool on_init;
>>>> +	bool after_stop;
>>>> +	bool after_crash;
>>>> +};
>>>> +
>>>
>>> Overall, this patch can move down the order, and better to add it in
>>> the patches where you actually introduce these code. And the debugfs
>>> pieces can be added as a separate patch by itself.
>>
>> Also, actually sounds more like flags than states..
> 
> I thought about this in terms of "states" in which a decision should be made.
> I'm not sure those are flags...

I see them as just decision flags for sync, it is not reflecting a state
like rproc->state. The rproc structure variable holds the current sync
flag state though.

> 
>>
>> regards
>> Suman
>>
>>>
>>>>  /**
>>>>   * struct rproc_ops - platform-specific device handlers
>>>>   * @start:	power on the device and boot it
>>>> @@ -456,6 +471,9 @@ struct rproc_dump_segment {
>>>>   * @firmware: name of firmware file to be loaded
>>>>   * @priv: private data which belongs to the platform-specific rproc module
>>>>   * @ops: platform-specific start/stop rproc handlers
>>>> + * @sync_ops: paltform-specific start/stop rproc handlers when
>>>
>>> typo on platform
> 
> No matter how many times you read your own code, there's always something like
> this that escapes...

he he, indeed :)

> 
>>>
>>>> + *	      synchronising with a remote processor.
>>>> + * @sync_states: Determine the rproc_ops to choose in specific states.
>>>>   * @dev: virtual device for refcounting and common remoteproc behavior
>>>>   * @power: refcount of users who need this rproc powered up
>>>>   * @state: state of the device
>>>> @@ -479,6 +497,7 @@ struct rproc_dump_segment {
>>>>   * @table_sz: size of @cached_table
>>>>   * @has_iommu: flag to indicate if remote processor is behind an MMU
>>>>   * @auto_boot: flag to indicate if remote processor should be auto-started
>>>> + * @sync_with_mcu: true if currently synchronising with MCU
>>>>   * @dump_segments: list of segments in the firmware
>>>>   * @nb_vdev: number of vdev currently handled by rproc
>>>>   */
>>>> @@ -488,7 +507,8 @@ struct rproc {
>>>>  	const char *name;
>>>>  	char *firmware;
>>>>  	void *priv;
>>>> -	struct rproc_ops *ops;
>>>> +	struct rproc_ops *ops, *sync_ops;
>>>
>>> Nothing wrong with this, but prefer to have the new variable in a new
>>> line for better readability.
> 
> Sure thing.

Thanks,
Suman


> 
>>>
>>> regards
>>> Suman
>>>
>>>> +	struct rproc_sync_states *sync_states;
>>>>  	struct device dev;
>>>>  	atomic_t power;
>>>>  	unsigned int state;
>>>> @@ -512,6 +532,7 @@ struct rproc {
>>>>  	size_t table_sz;
>>>>  	bool has_iommu;
>>>>  	bool auto_boot;
>>>> +	bool sync_with_mcu;
>>>>  	struct list_head dump_segments;
>>>>  	int nb_vdev;
>>>>  };
>>>>
>>>
>>

Re: [PATCH v2 01/17] remoteproc: Add new operation and state machine for MCU synchronisation

[Suman Anna]

On 4/1/20 4:53 PM, Mathieu Poirier wrote:
> On Mon, Mar 30, 2020 at 05:49:11PM -0500, Suman Anna wrote:
>> On 3/30/20 5:46 PM, Suman Anna wrote:
>>> Hi Mathieu,
>>>
>>> On 3/24/20 4:45 PM, Mathieu Poirier wrote:
>>>> Add a new rproc_ops sync_ops to support use cases where the remoteproc
>>>> core is synchronisting with the MCU.  When exactly to use the sync_ops is
>>>
>>> typo on syschronisting..
>>>
>>>> directed by the states in struct rproc_sync_states.
>>>>
>>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>> ---
>>>>  drivers/remoteproc/remoteproc_debugfs.c  | 31 ++++++++++++++++++++++++
>>>>  drivers/remoteproc/remoteproc_internal.h |  5 ++++
>>>>  include/linux/remoteproc.h               | 23 +++++++++++++++++-
>>>>  3 files changed, 58 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/remoteproc/remoteproc_debugfs.c b/drivers/remoteproc/remoteproc_debugfs.c
>>>> index dd93cf04e17f..187bcc67f997 100644
>>>> --- a/drivers/remoteproc/remoteproc_debugfs.c
>>>> +++ b/drivers/remoteproc/remoteproc_debugfs.c
>>>> @@ -311,6 +311,35 @@ static const struct file_operations rproc_carveouts_ops = {
>>>>  	.release	= single_release,
>>>>  };
>>>>  
>>>> +/* Expose synchronisation states via debugfs */
>>>> +static int rproc_sync_states_show(struct seq_file *seq, void *p)
>>>> +{
>>>> +	struct rproc *rproc = seq->private;
>>>> +
>>>> +	seq_printf(seq, "Sync with MCU: %s\n",
>>>> +		   rproc->sync_with_mcu ? "true" : "false");
>>>> +	seq_printf(seq, "On init: %s\n",
>>>> +		   rproc->sync_states->on_init ? "true" : "false");
>>>> +	seq_printf(seq, "After stop: %s\n",
>>>> +		   rproc->sync_states->after_stop ? "true" : "false");
>>>> +	seq_printf(seq, "After crash: %s\n",
>>>> +		   rproc->sync_states->after_crash ? "true" : "false");
>>>> +
>>>> +	return 0;
>>>> +}
>>>> +
>>>> +static int rproc_sync_states_open(struct inode *inode, struct file *file)
>>>> +{
>>>> +	return single_open(file, rproc_sync_states_show, inode->i_private);
>>>> +}
>>>> +
>>>> +static const struct file_operations rproc_sync_states_ops = {
>>>> +	.open		= rproc_sync_states_open,
>>>> +	.read		= seq_read,
>>>> +	.llseek		= seq_lseek,
>>>> +	.release	= single_release,
>>>> +};
>>>> +
>>>>  void rproc_remove_trace_file(struct dentry *tfile)
>>>>  {
>>>>  	debugfs_remove(tfile);
>>>> @@ -357,6 +386,8 @@ void rproc_create_debug_dir(struct rproc *rproc)
>>>>  			    rproc, &rproc_rsc_table_ops);
>>>>  	debugfs_create_file("carveout_memories", 0400, rproc->dbg_dir,
>>>>  			    rproc, &rproc_carveouts_ops);
>>>> +	debugfs_create_file("sync_states", 0400, rproc->dbg_dir,
>>>> +			    rproc, &rproc_sync_states_ops);
>>>>  }
>>>>  
>>>>  void __init rproc_init_debugfs(void)
>>>> diff --git a/drivers/remoteproc/remoteproc_internal.h b/drivers/remoteproc/remoteproc_internal.h
>>>> index 493ef9262411..5c93de5e00bb 100644
>>>> --- a/drivers/remoteproc/remoteproc_internal.h
>>>> +++ b/drivers/remoteproc/remoteproc_internal.h
>>>> @@ -63,6 +63,11 @@ struct resource_table *rproc_elf_find_loaded_rsc_table(struct rproc *rproc,
>>>>  struct rproc_mem_entry *
>>>>  rproc_find_carveout_by_name(struct rproc *rproc, const char *name, ...);
>>>>  
>>>> +static inline bool rproc_sync_with_mcu(struct rproc *rproc)
>>>> +{
>>>> +	return rproc->sync_with_mcu;
>>>> +}
>>>> +
>>>
>>> Since you are using this mostly for checking and as a boolean, I suggest
>>> you rename this appropriately, something like rproc_needs_sync,
>>> rproc_has_sync or rproc_uses_sync().
> 
> I will rename to rproc_syncing_with_rproc()

Hmm, I want this to reflect a boolean answer for better code
flow/readability.

> 
> 
>>>
>>> And I am wondering if it is actually better to introduce the sync state
>>> to check against here, rather than using the stored sync state and
>>> return. The current way makes it confusing to read the state machine.
> 
> I decided to proceed this way because there may not be a direct correlation
> between the current synchronisation state and the location of the check itself.
> for instance, in firmware_show(), what sync state should be key on?

Yeah OK. Its the combinations of ops (11 callbacks) plus sync states (3)
that kinda makes it hard to read the state-machine.

> 
>>>
>>>>  static inline
>>>>  int rproc_fw_sanity_check(struct rproc *rproc, const struct firmware *fw)
>>>>  {
>>>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
>>>> index 16ad66683ad0..d115e47d702d 100644
>>>> --- a/include/linux/remoteproc.h
>>>> +++ b/include/linux/remoteproc.h
>>>> @@ -353,6 +353,21 @@ enum rsc_handling_status {
>>>>  	RSC_IGNORED	= 1,
>>>>  };
>>>>  
>>>> +/**
>>>> + * struct rproc_sync_states - platform specific states indicating which
>>>> + *			      rproc_ops to use at specific times during
>>>> + *			      the MCU lifecycle.
>>>> + * @on_init: true if synchronising with MCU at system initialisation time
>>>> + * @after_stop: true if synchronising with MCU after stopped from the
>>>> + *		command line
>>>> + * @after_crash: true if synchonising with MCU after the MCU has crashed
>>>> + */
>>>> +struct rproc_sync_states {
>>>> +	bool on_init;
>>>> +	bool after_stop;
>>>> +	bool after_crash;
>>>> +};
>>>> +
>>>
>>> Overall, this patch can move down the order, and better to add it in
>>> the patches where you actually introduce these code. And the debugfs
>>> pieces can be added as a separate patch by itself.
>>
>> Also, actually sounds more like flags than states..
> 
> I thought about this in terms of "states" in which a decision should be made.
> I'm not sure those are flags...

I see them as just decision flags for sync, it is not reflecting a state
like rproc->state. The rproc structure variable holds the current sync
flag state though.

> 
>>
>> regards
>> Suman
>>
>>>
>>>>  /**
>>>>   * struct rproc_ops - platform-specific device handlers
>>>>   * @start:	power on the device and boot it
>>>> @@ -456,6 +471,9 @@ struct rproc_dump_segment {
>>>>   * @firmware: name of firmware file to be loaded
>>>>   * @priv: private data which belongs to the platform-specific rproc module
>>>>   * @ops: platform-specific start/stop rproc handlers
>>>> + * @sync_ops: paltform-specific start/stop rproc handlers when
>>>
>>> typo on platform
> 
> No matter how many times you read your own code, there's always something like
> this that escapes...

he he, indeed :)

> 
>>>
>>>> + *	      synchronising with a remote processor.
>>>> + * @sync_states: Determine the rproc_ops to choose in specific states.
>>>>   * @dev: virtual device for refcounting and common remoteproc behavior
>>>>   * @power: refcount of users who need this rproc powered up
>>>>   * @state: state of the device
>>>> @@ -479,6 +497,7 @@ struct rproc_dump_segment {
>>>>   * @table_sz: size of @cached_table
>>>>   * @has_iommu: flag to indicate if remote processor is behind an MMU
>>>>   * @auto_boot: flag to indicate if remote processor should be auto-started
>>>> + * @sync_with_mcu: true if currently synchronising with MCU
>>>>   * @dump_segments: list of segments in the firmware
>>>>   * @nb_vdev: number of vdev currently handled by rproc
>>>>   */
>>>> @@ -488,7 +507,8 @@ struct rproc {
>>>>  	const char *name;
>>>>  	char *firmware;
>>>>  	void *priv;
>>>> -	struct rproc_ops *ops;
>>>> +	struct rproc_ops *ops, *sync_ops;
>>>
>>> Nothing wrong with this, but prefer to have the new variable in a new
>>> line for better readability.
> 
> Sure thing.

Thanks,
Suman


> 
>>>
>>> regards
>>> Suman
>>>
>>>> +	struct rproc_sync_states *sync_states;
>>>>  	struct device dev;
>>>>  	atomic_t power;
>>>>  	unsigned int state;
>>>> @@ -512,6 +532,7 @@ struct rproc {
>>>>  	size_t table_sz;
>>>>  	bool has_iommu;
>>>>  	bool auto_boot;
>>>> +	bool sync_with_mcu;
>>>>  	struct list_head dump_segments;
>>>>  	int nb_vdev;
>>>>  };
>>>>
>>>
>>

Re: [PATCH v2 06/17] remoteproc: Introduce function rproc_alloc_internals()

[Suman Anna]

On 4/1/20 3:29 PM, Mathieu Poirier wrote:
> Hi Suman,
> 
> On Mon, Mar 30, 2020 at 03:38:14PM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>
>> On 3/27/20 6:10 AM, Loic PALLARDY wrote:
>>> Hi Mathieu,
>>>
>>>>
>>>> In preparation to allocate the synchronisation operation and state
>>>> machine, spin off a new function in order to keep rproc_alloc() as
>>>> clean as possible.
>>>>
>>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>> ---
>>>>  drivers/remoteproc/remoteproc_core.c | 26 ++++++++++++++++++++++---
>>>> -
>>>>  1 file changed, 22 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/drivers/remoteproc/remoteproc_core.c
>>>> b/drivers/remoteproc/remoteproc_core.c
>>>> index ee277bc5556c..9da245734db6 100644
>>>> --- a/drivers/remoteproc/remoteproc_core.c
>>>> +++ b/drivers/remoteproc/remoteproc_core.c
>>>> @@ -2018,6 +2018,26 @@ static int rproc_alloc_ops(struct rproc *rproc,
>>>> const struct rproc_ops *ops)
>>>>  	return 0;
>>>>  }
>>>>
>>>> +static int rproc_alloc_internals(struct rproc *rproc, const char *name,
>>>> +				 const struct rproc_ops *boot_ops,
>>>> +				 const char *firmware, int len)
>>>
>>> len argument is not used in the patch nor in the following, maybe removed from my pov.
>>
>> Indeed.
>>
>>>
>>> Regards,
>>> Loic
>>
>>>> +{
>>>> +	int ret;
>>>> +
>>>> +	/* We have a boot_ops so allocate firmware name and operations */
>>>> +	if (boot_ops) {
>>>> +		ret = rproc_alloc_firmware(rproc, name, firmware);
>>>> +		if (ret)
>>>> +			return ret;
>>
>> So, can you explain why firmware allocation now becomes conditional on
>> this boot_ops?
> 
> There is no point in allocating a firmware name in a scenario where the
> remoteproc core is only synchronising with the MCU.  As soon as a boot_ops (to
> be renamed ops as per your comment below) is present I assume firmware loading
> will be involved at some point.   Do you see a scenario where that wouldn't be
> be case?

No. But that isn't until the whole sync stuff is introduced. As of this
patch, it it still code refactoring. And I would think that the cases
where only sync_ops will be used will be the minority.

regards
Suman

> 
>>
>> Perhaps, continue to call this as ops following the field name in struct
>> rproc.
> 
> Ok
> 
>>
>> regards
>> Suman
>>
>>>> +
>>>> +		ret = rproc_alloc_ops(rproc, boot_ops);
>>>> +		if (ret)
>>>> +			return ret;
>>>> +	}
>>>> +
>>>> +	return 0;
>>>> +}
>>>> +
>>>>  /**
>>>>   * rproc_alloc() - allocate a remote processor handle
>>>>   * @dev: the underlying device
>>>> @@ -2064,10 +2084,8 @@ struct rproc *rproc_alloc(struct device *dev, const
>>>> char *name,
>>>>  	rproc->dev.class = &rproc_class;
>>>>  	rproc->dev.driver_data = rproc;
>>>>
>>>> -	if (rproc_alloc_firmware(rproc, name, firmware))
>>>> -		goto out;
>>>> -
>>>> -	if (rproc_alloc_ops(rproc, ops))
>>>> +	if (rproc_alloc_internals(rproc, name, ops,
>>>> +				  firmware, len))
>>>>  		goto out;
>>>>
>>>>  	/* Assign a unique device index and name */
>>>> --
>>>> 2.20.1
>>>
>>

Re: [PATCH v2 15/17] remoteproc: Correctly deal with MCU synchronisation when changing FW image

[Mathieu Poirier]

Hi Suman,

On Tue, Mar 31, 2020 at 05:14:18PM -0500, Suman Anna wrote:
> Hi Mathieu,
> 
> On 3/30/20 6:21 PM, Mathieu Poirier wrote:
> > On Fri, Mar 27, 2020 at 01:50:18PM +0000, Loic PALLARDY wrote:
> >>
> >>> This patch prevents the firmware image from being displayed or changed
> >>> when
> >>> the remoteproc core is synchronising with an MCU. This is needed since
> >>> there is no guarantee about the nature of the firmware image that is loaded
> >>> by the external entity.
> >>>
> >>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> >>> ---
> >>>  drivers/remoteproc/remoteproc_sysfs.c | 25
> >>> ++++++++++++++++++++++++-
> >>>  1 file changed, 24 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
> >>> b/drivers/remoteproc/remoteproc_sysfs.c
> >>> index 7f8536b73295..4956577ad4b4 100644
> >>> --- a/drivers/remoteproc/remoteproc_sysfs.c
> >>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
> >>> @@ -13,9 +13,20 @@
> >>>  static ssize_t firmware_show(struct device *dev, struct device_attribute
> >>> *attr,
> >>>  			  char *buf)
> >>>  {
> >>> +	ssize_t ret;
> >>>  	struct rproc *rproc = to_rproc(dev);
> >>>
> >>> -	return sprintf(buf, "%s\n", rproc->firmware);
> >>> +	/*
> >>> +	 * In most instances there is no guarantee about the firmware
> >>> +	 * that was loaded by the external entity.  As such simply don't
> >>> +	 * print anything.
> >>> +	 */
> >>> +	if (rproc_sync_with_mcu(rproc))
> >>> +		ret = sprintf(buf, "\n");
> >> Is it enough to provide empty name, or should we add a message to indicate that's name is unkown/undefined ?
> >>
> > 
> > Don't know... It is easy to find plenty of cases in sysfs where null values are
> > represented with a "\n", and just as many where "unknown", "undefined" or "-1"
> > are used. I know GKH prefers the least amount of information as possible, hence
> > going with a "\n".
> > 
> > Again, no strong opinion...
> > 
> >> Regards,
> >> Loic
> >>> +	else
> >>> +		ret = sprintf(buf, "%s\n", rproc->firmware);
> >>> +
> >>> +	return ret;
> >>>  }
> >>>
> >>>  /* Change firmware name via sysfs */
> >>> @@ -33,6 +44,18 @@ static ssize_t firmware_store(struct device *dev,
> >>>  		return -EINVAL;
> >>>  	}
> >>>
> >>> +	/*
> >>> +	 * There is no point in trying to change the firmware if the MCU
> >>> +	 * is currently running or if loading of the image is done by
> >>> +	 * another entity.
> >>> +	 */
> >>> +	if (rproc_sync_with_mcu(rproc)) {
> >>> +		dev_err(dev,
> >>> +			"can't change firmware while synchronising with
> >>> MCU\n");
> >>> +		err = -EBUSY;
> >>> +		goto out;
> >>> +	}
> >>> +
> 
> So, I have done a patch sometime back to deny sysfs operations [1] (the
> primary usecase is for a rproc-client driver driven boot where auto-boot
> is not set) which is still a need for me. Do you see that as orthogonal
> to that, or can we leverage that here somehow. I cannot use the sync_
> conditions for my cases since they are not already booted before.

No matter how much I try to fit the functionality provided by the
"deny_sysfs_ops" flag in the patch you pointed out, I just can't come up with
something I am happy with.

The only thing the topics of remote processor synchronisation and kernel
initiated remote processor boot have in common is preventing sysfs access under
specific circumstances.  As such it is probably best to keep their implemenation
seperated for the time being.

Thanks,
Mathieu

> 
> Also, any reason why you want to do this check before the rproc->state
> unlike the logic around the 'state' file in the next patch?
> 
> [1] https://patchwork.kernel.org/patch/10601325/
> 
> regards
> Suman
> 
> >>>  	if (rproc->state != RPROC_OFFLINE) {
> >>>  		dev_err(dev, "can't change firmware while running\n");
> >>>  		err = -EBUSY;
> >>> --
> >>> 2.20.1
> >>
> 

Re: [PATCH v2 15/17] remoteproc: Correctly deal with MCU synchronisation when changing FW image

[Suman Anna]

On 4/22/20 4:29 PM, Mathieu Poirier wrote:
> Hi Suman,
> 
> On Tue, Mar 31, 2020 at 05:14:18PM -0500, Suman Anna wrote:
>> Hi Mathieu,
>>
>> On 3/30/20 6:21 PM, Mathieu Poirier wrote:
>>> On Fri, Mar 27, 2020 at 01:50:18PM +0000, Loic PALLARDY wrote:
>>>>
>>>>> This patch prevents the firmware image from being displayed or changed
>>>>> when
>>>>> the remoteproc core is synchronising with an MCU. This is needed since
>>>>> there is no guarantee about the nature of the firmware image that is loaded
>>>>> by the external entity.
>>>>>
>>>>> Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
>>>>> ---
>>>>>   drivers/remoteproc/remoteproc_sysfs.c | 25
>>>>> ++++++++++++++++++++++++-
>>>>>   1 file changed, 24 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/drivers/remoteproc/remoteproc_sysfs.c
>>>>> b/drivers/remoteproc/remoteproc_sysfs.c
>>>>> index 7f8536b73295..4956577ad4b4 100644
>>>>> --- a/drivers/remoteproc/remoteproc_sysfs.c
>>>>> +++ b/drivers/remoteproc/remoteproc_sysfs.c
>>>>> @@ -13,9 +13,20 @@
>>>>>   static ssize_t firmware_show(struct device *dev, struct device_attribute
>>>>> *attr,
>>>>>   			  char *buf)
>>>>>   {
>>>>> +	ssize_t ret;
>>>>>   	struct rproc *rproc = to_rproc(dev);
>>>>>
>>>>> -	return sprintf(buf, "%s\n", rproc->firmware);
>>>>> +	/*
>>>>> +	 * In most instances there is no guarantee about the firmware
>>>>> +	 * that was loaded by the external entity.  As such simply don't
>>>>> +	 * print anything.
>>>>> +	 */
>>>>> +	if (rproc_sync_with_mcu(rproc))
>>>>> +		ret = sprintf(buf, "\n");
>>>> Is it enough to provide empty name, or should we add a message to indicate that's name is unkown/undefined ?
>>>>
>>>
>>> Don't know... It is easy to find plenty of cases in sysfs where null values are
>>> represented with a "\n", and just as many where "unknown", "undefined" or "-1"
>>> are used. I know GKH prefers the least amount of information as possible, hence
>>> going with a "\n".
>>>
>>> Again, no strong opinion...
>>>
>>>> Regards,
>>>> Loic
>>>>> +	else
>>>>> +		ret = sprintf(buf, "%s\n", rproc->firmware);
>>>>> +
>>>>> +	return ret;
>>>>>   }
>>>>>
>>>>>   /* Change firmware name via sysfs */
>>>>> @@ -33,6 +44,18 @@ static ssize_t firmware_store(struct device *dev,
>>>>>   		return -EINVAL;
>>>>>   	}
>>>>>
>>>>> +	/*
>>>>> +	 * There is no point in trying to change the firmware if the MCU
>>>>> +	 * is currently running or if loading of the image is done by
>>>>> +	 * another entity.
>>>>> +	 */
>>>>> +	if (rproc_sync_with_mcu(rproc)) {
>>>>> +		dev_err(dev,
>>>>> +			"can't change firmware while synchronising with
>>>>> MCU\n");
>>>>> +		err = -EBUSY;
>>>>> +		goto out;
>>>>> +	}
>>>>> +
>>
>> So, I have done a patch sometime back to deny sysfs operations [1] (the
>> primary usecase is for a rproc-client driver driven boot where auto-boot
>> is not set) which is still a need for me. Do you see that as orthogonal
>> to that, or can we leverage that here somehow. I cannot use the sync_
>> conditions for my cases since they are not already booted before.
> 
> No matter how much I try to fit the functionality provided by the
> "deny_sysfs_ops" flag in the patch you pointed out, I just can't come up with
> something I am happy with.
> 
> The only thing the topics of remote processor synchronisation and kernel
> initiated remote processor boot have in common is preventing sysfs access under
> specific circumstances. As such it is probably best to keep their implemenation
> seperated for the time being.

Yeah, OK, we can revisit this. We already have an in-kernel user 
wkup_m3_ipc driver that controls the boot and shutdown of the wkup_m3 
rproc driver, which is used for SoC Power Management, and so we really 
do not want any control from userspace for that. I will have the same 
need for the PRU remoteproc client usage perspective as well.

regards
Suman

> 
> Thanks,
> Mathieu
> 
>>
>> Also, any reason why you want to do this check before the rproc->state
>> unlike the logic around the 'state' file in the next patch?
>>
>> [1] https://patchwork.kernel.org/patch/10601325/
>>
>> regards
>> Suman
>>
>>>>>   	if (rproc->state != RPROC_OFFLINE) {
>>>>>   		dev_err(dev, "can't change firmware while running\n");
>>>>>   		err = -EBUSY;
>>>>> --
>>>>> 2.20.1
>>>>
>>