[PATCH 0/7] Add support for cyclic function execution infrastruture

[PATCH 0/7] Add support for cyclic function execution infrastruture

[Stefan Roese]

This patchset adds the basic infrastructure to periodically execute
code, e.g. all 100ms. Examples for such functions might be LED blinking
etc. The functions that are hooked into this cyclic list should be
small timewise as otherwise the execution of the other code that relies
on a high frequent polling (e.g. UART rx char ready check) might be
delayed too much. This patch also adds the Kconfig option
CONFIG_CYCLIC_MAX_CPU_TIME_US, which configures the max allowed time
for such a cyclic function. If it's execution time exceeds this time,
this cyclic function will get removed from the cyclic list.

How is this cyclic functionality executed?
This patchset integrates the main function responsible for calling all
registered cyclic functions cyclic_run() into the common WATCHDOG_RESET
macro. This guarantees that cyclic_run() is executed very often, which
is necessary for the cyclic functions to get scheduled and executed at
their configured periods.

This cyclic infrastructure will be used by a board specific function on
the NIC23 MIPS Octeon board, which needs to check periodically, if a
PCIe FLR has occurred.

Ideas how to continue:
One idea is to rename WATCHDOG_RESET to something like SCHEDULE and
move the watchdog_reset call into this cyclic infrastructure as well.
Or to perhaps move the shell UART RX ready polling to a cyclic
function.

It's also possible to extend the "cyclic" command, to support the
creation of periodically executed shell commands (for testing etc).

Thanks,
Stefan

Aaron Williams (1):
  mips: octeon_nic23: Add PCIe FLR fixup via cyclic infrastructure

Stefan Roese (6):
  time: Import time_after64() and friends from Linux
  cyclic: Add basic support for cyclic function execution infrastruture
  cyclic: Integrate cyclic infrastructure into WATCHDOG_RESET
  cyclic: Integrate cyclic functionality at bootup in board_r/f
  cyclic: Add 'cyclic list' command
  sandbox: Add cyclic demo function

 MAINTAINERS                        |   7 +
 board/Marvell/octeon_nic23/board.c | 197 +++++++++++++++++++++++++++++
 board/sandbox/sandbox.c            |  15 +++
 cmd/Kconfig                        |   6 +
 cmd/Makefile                       |   1 +
 cmd/cyclic.c                       |  40 ++++++
 common/Kconfig                     |  22 ++++
 common/Makefile                    |   1 +
 common/board_f.c                   |   2 +
 common/board_r.c                   |   2 +
 common/cyclic.c                    | 112 ++++++++++++++++
 configs/octeon_nic23_defconfig     |   3 +
 configs/sandbox_defconfig          |   2 +
 fs/cramfs/uncompress.c             |   2 +-
 include/cyclic.h                   |  84 ++++++++++++
 include/time.h                     |  19 +++
 include/watchdog.h                 |  23 +++-
 17 files changed, 534 insertions(+), 4 deletions(-)
 create mode 100644 cmd/cyclic.c
 create mode 100644 common/cyclic.c
 create mode 100644 include/cyclic.h

-- 
2.36.1

[PATCH 1/7] time: Import time_after64() and friends from Linux

[Stefan Roese]

When using us times it makes sense to use 64bit variables for storage.
The currently implemented time_after() and friends functions only handle
32bit variables. This patch now includes the 64bit variants as well
from Linux. This will be used by the upcoming generic cyclic function
infrastructure.

These macros were copied from include/linux/jiffies.h of Linux 5.18.

Signed-off-by: Stefan Roese <sr@denx.de>
---
 include/time.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/include/time.h b/include/time.h
index 9deb2cf61cc4..3b2ba0912470 100644
--- a/include/time.h
+++ b/include/time.h
@@ -83,6 +83,25 @@ uint64_t usec_to_tick(unsigned long usec);
 	(time_after_eq(a,b) && \
 	 time_before(a,c))
 
+/* Same as above, but does so with platform independent 64bit types.
+ * These must be used when utilizing jiffies_64 (i.e. return value of
+ * get_jiffies_64() */
+#define time_after64(a,b)	\
+	(typecheck(__u64, a) &&	\
+	 typecheck(__u64, b) && \
+	 ((__s64)((b) - (a)) < 0))
+#define time_before64(a,b)	time_after64(b,a)
+
+#define time_after_eq64(a,b)	\
+	(typecheck(__u64, a) && \
+	 typecheck(__u64, b) && \
+	 ((__s64)((a) - (b)) >= 0))
+#define time_before_eq64(a,b)	time_after_eq64(b,a)
+
+#define time_in_range64(a, b, c) \
+	(time_after_eq64(a, b) && \
+	 time_before_eq64(a, c))
+
 /**
  * usec2ticks() - Convert microseconds to internal ticks
  *
-- 
2.36.1

[PATCH 2/7] cyclic: Add basic support for cyclic function execution infrastruture

[Stefan Roese]

Add the basic infrastructure to periodically execute code, e.g. all
100ms. Examples for such functions might be LED blinking etc. The
functions that are hooked into this cyclic list should be small timewise
as otherwise the execution of the other code that relies on a high
frequent polling (e.g. UART rx char ready check) might be delayed too
much. This patch also adds the Kconfig option
CONFIG_CYCLIC_MAX_CPU_TIME_US, which configures the max allowed time
for such a cyclic function. If it's execution time exceeds this time,
this cyclic function will get removed from the cyclic list.

How is this cyclic functionality executed?
The following patch integrates the main function responsible for
calling all registered cyclic functions cyclic_run() into the
common WATCHDOG_RESET macro. This guarantees that cyclic_run() is
executed very often, which is necessary for the cyclic functions to
get scheduled and executed at their configured periods.

This cyclic infrastructure will be used by a board specific function on
the NIC23 MIPS Octeon board, which needs to check periodically, if a
PCIe FLR has occurred.

Signed-off-by: Stefan Roese <sr@denx.de>
---
 MAINTAINERS      |   6 +++
 common/Kconfig   |  22 ++++++++++
 common/Makefile  |   1 +
 common/cyclic.c  | 112 +++++++++++++++++++++++++++++++++++++++++++++++
 include/cyclic.h |  84 +++++++++++++++++++++++++++++++++++
 5 files changed, 225 insertions(+)
 create mode 100644 common/cyclic.c
 create mode 100644 include/cyclic.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 56be0bfad00c..920f515dd555 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -752,6 +752,12 @@ T:	git https://source.denx.de/u-boot/custodians/u-boot-coldfire.git
 F:	arch/m68k/
 F:	doc/arch/m68k.rst
 
+CYCLIC
+M:	Stefan Roese <sr@denx.de>
+S:	Maintained
+F:	common/cyclic.c
+F:	include/cyclic.h
+
 DFU
 M:	Lukasz Majewski <lukma@denx.de>
 S:	Maintained
diff --git a/common/Kconfig b/common/Kconfig
index a96842a5c11d..1b04904fd688 100644
--- a/common/Kconfig
+++ b/common/Kconfig
@@ -551,6 +551,28 @@ config DISPLAY_BOARDINFO_LATE
 
 menu "Start-up hooks"
 
+config CYCLIC
+	bool "General-purpose cyclic execution mechanism"
+	help
+	  This enables a general-purpose cyclic execution infrastructure,
+	  to allow "small" (run-time wise) functions to be executed at
+	  a specified frequency. Things like LED blinking or watchdog
+	  triggering are examples for such tasks.
+
+	  See doc/develop/cyclic.rst for more information.
+
+if CYCLIC
+
+config CYCLIC_MAX_CPU_TIME_US
+	int "Sets the max allowed time for a cyclic function in us"
+	default 1000
+	help
+	  The max allowed time for a cyclic function in us. If a functions
+	  takes longer than this duration this function will get unregistered
+	  automatically.
+
+endif # CYCLIC
+
 config EVENT
 	bool "General-purpose event-handling mechanism"
 	default y if SANDBOX
diff --git a/common/Makefile b/common/Makefile
index 75c24e324927..a5f00ec371c1 100644
--- a/common/Makefile
+++ b/common/Makefile
@@ -89,6 +89,7 @@ obj-y += malloc_simple.o
 endif
 endif
 
+obj-$(CONFIG_$(SPL_TPL_)CYCLIC) += cyclic.o
 obj-$(CONFIG_$(SPL_TPL_)EVENT) += event.o
 
 obj-$(CONFIG_$(SPL_TPL_)HASH) += hash.o
diff --git a/common/cyclic.c b/common/cyclic.c
new file mode 100644
index 000000000000..6e7ebbe7b218
--- /dev/null
+++ b/common/cyclic.c
@@ -0,0 +1,112 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * A general-purpose cyclic execution infrastructure, to allow "small"
+ * (run-time wise) functions to be executed at a specified frequency.
+ * Things like LED blinking or watchdog triggering are examples for such
+ * tasks.
+ *
+ * Copyright (C) 2022 Stefan Roese <sr@denx.de>
+ */
+
+#include <cyclic.h>
+#include <log.h>
+#include <linker_lists.h>
+#include <malloc.h>
+#include <time.h>
+#include <linux/errno.h>
+#include <linux/list.h>
+
+struct list_head cyclic_list;
+static bool cyclic_ready;
+static bool cyclic_running;
+
+struct cyclic_struct *cyclic_register(cyclic_func_t func, uint64_t delay_us,
+				      const char *name, void *ctx)
+{
+	struct cyclic_struct *cyclic;
+
+	if (!cyclic_ready) {
+		pr_debug("Cyclic IF not ready yet\n");
+		return NULL;
+	}
+
+	cyclic = calloc(1, sizeof(struct cyclic_struct));
+	if (!cyclic) {
+		pr_debug("Memory allocation error\n");
+		return NULL;
+	}
+
+	/* Store values in struct */
+	cyclic->func = func;
+	cyclic->ctx = ctx;
+	cyclic->name = strdup(name);
+	cyclic->delay_us = delay_us;
+	cyclic->start_time_us = timer_get_us();
+	list_add_tail(&cyclic->list, &cyclic_list);
+
+	return cyclic;
+}
+
+int cyclic_unregister(struct cyclic_struct *cyclic)
+{
+	list_del(&cyclic->list);
+	free(cyclic);
+
+	return 0;
+}
+
+void cyclic_run(void)
+{
+	struct cyclic_struct *cyclic, *tmp;
+	uint64_t now, cpu_time;
+
+	/* Prevent recursion */
+	if (cyclic_running)
+		return;
+
+	cyclic_running = true;
+	list_for_each_entry_safe(cyclic, tmp, &cyclic_list, list) {
+		/*
+		 * Check if this cyclic function needs to get called, e.g.
+		 * do not call the cyclic func too often
+		 */
+		now = timer_get_us();
+		if (time_after_eq64(now, cyclic->next_call)) {
+			/* Call cyclic function and account it's cpu-time */
+			cyclic->next_call = now + cyclic->delay_us;
+			cyclic->func(cyclic->ctx);
+			cyclic->run_cnt++;
+			cpu_time = timer_get_us() - now;
+			cyclic->cpu_time_us += cpu_time;
+
+			/* Check if cpu-time exceeds max allowed time */
+			if (cpu_time > CONFIG_CYCLIC_MAX_CPU_TIME_US) {
+				pr_err("cyclic function %s took too long: %lldus vs %dus max, disabling\n",
+				       cyclic->name, cpu_time,
+				       CONFIG_CYCLIC_MAX_CPU_TIME_US);
+
+				/* Unregister this cyclic function */
+				cyclic_unregister(cyclic);
+			}
+		}
+	}
+	cyclic_running = false;
+}
+
+int cyclic_uninit(void)
+{
+	struct cyclic_struct *cyclic, *tmp;
+
+	list_for_each_entry_safe(cyclic, tmp, &cyclic_list, list)
+		cyclic_unregister(cyclic);
+
+	return 0;
+}
+
+int cyclic_init(void)
+{
+	INIT_LIST_HEAD(&cyclic_list);
+	cyclic_ready = true;
+
+	return 0;
+}
diff --git a/include/cyclic.h b/include/cyclic.h
new file mode 100644
index 000000000000..d07c04458a4f
--- /dev/null
+++ b/include/cyclic.h
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * A general-purpose cyclic execution infrastructure, to allow "small"
+ * (run-time wise) functions to be executed at a specified frequency.
+ * Things like LED blinking or watchdog triggering are examples for such
+ * tasks.
+ *
+ * Copyright (C) 2022 Stefan Roese <sr@denx.de>
+ */
+
+#ifndef __cyclic_h
+#define __cyclic_h
+
+#include <linux/list.h>
+#include <asm/types.h>
+
+struct cyclic_struct {
+	void (*func)(void *ctx);
+	void *ctx;
+	char *name;
+	uint64_t delay_us;
+	uint64_t start_time_us;
+	uint64_t cpu_time_us;
+	uint64_t run_cnt;
+	uint64_t next_call;
+	struct list_head list;
+};
+
+/** Function type for cyclic functions */
+typedef void (*cyclic_func_t)(void *ctx);
+
+/**
+ * cyclic_register - Register a new cyclic function
+ *
+ * @func: Function to call periodically
+ * @delay_us: Delay is us after which this function shall get executed
+ * @name: Cyclic function name/id
+ * @ctx: Context to pass to the function
+ * @return: pointer to cyclic_struct if OK, NULL on error
+ */
+struct cyclic_struct *cyclic_register(cyclic_func_t func, uint64_t delay_us,
+				      const char *name, void *ctx);
+
+/**
+ * cyclic_unregister - Unregister a cyclic function
+ *
+ * @cyclic: Pointer to cyclic_struct of the function that shall be removed
+ * @return: 0 if OK, -ve on error
+ */
+int cyclic_unregister(struct cyclic_struct *cyclic);
+
+#if defined(CONFIG_CYCLIC)
+/**
+ * cyclic_init() - Set up cyclic functions
+ *
+ * Init a list of cyclic functions, so that these can be added as needed
+ */
+int cyclic_init(void);
+
+/**
+ * cyclic_uninit() - Clean up cyclic functions
+ *
+ * This removes all cyclic functions
+ */
+int cyclic_uninit(void);
+
+void cyclic_run(void);
+#else
+static inline void cyclic_run(void)
+{
+}
+
+static inline int cyclic_init(void)
+{
+	return 0;
+}
+
+static inline int cyclic_uninit(void)
+{
+	return 0;
+}
+#endif
+
+#endif
-- 
2.36.1

[PATCH 3/7] cyclic: Integrate cyclic infrastructure into WATCHDOG_RESET

[Stefan Roese]

This patch integrates the main function responsible for calling all
registered cyclic functions cyclic_run() into the common WATCHDOG_RESET
macro. This guarantees that cyclic_run() is executed very often, which
is necessary for the cyclic functions to get scheduled and executed at
their configured periods.

If CONFIG_WATCHDOG is not enabled, only cyclic_run() without calling
watchdog_reset(). This guarantees that the cyclic functionality does not
rely on CONFIG_WATCHDOG being enabled.

Signed-off-by: Stefan Roese <sr@denx.de>
---
 fs/cramfs/uncompress.c |  2 +-
 include/watchdog.h     | 23 ++++++++++++++++++++---
 2 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/fs/cramfs/uncompress.c b/fs/cramfs/uncompress.c
index f431cc46c1f7..38e10e2e4422 100644
--- a/fs/cramfs/uncompress.c
+++ b/fs/cramfs/uncompress.c
@@ -62,7 +62,7 @@ int cramfs_uncompress_init (void)
 	stream.avail_in = 0;
 
 #if defined(CONFIG_HW_WATCHDOG) || defined(CONFIG_WATCHDOG)
-	stream.outcb = (cb_func) WATCHDOG_RESET;
+	stream.outcb = (cb_func)watchdog_reset_func;
 #else
 	stream.outcb = Z_NULL;
 #endif /* CONFIG_HW_WATCHDOG */
diff --git a/include/watchdog.h b/include/watchdog.h
index 14fa5fda259e..c252cabffc2c 100644
--- a/include/watchdog.h
+++ b/include/watchdog.h
@@ -11,6 +11,8 @@
 #define _WATCHDOG_H_
 
 #if !defined(__ASSEMBLY__)
+#include <cyclic.h>
+
 /*
  * Reset the watchdog timer, always returns 0
  *
@@ -54,11 +56,16 @@ int init_func_watchdog_reset(void);
 			/* Don't require the watchdog to be enabled in SPL */
 			#if defined(CONFIG_SPL_BUILD) &&		\
 				!defined(CONFIG_SPL_WATCHDOG)
-				#define WATCHDOG_RESET() {}
+				#define WATCHDOG_RESET() { \
+					cyclic_run(); \
+				}
 			#else
 				extern void watchdog_reset(void);
 
-				#define WATCHDOG_RESET watchdog_reset
+				#define WATCHDOG_RESET() { \
+					watchdog_reset(); \
+					cyclic_run(); \
+				}
 			#endif
 		#endif
 	#else
@@ -68,11 +75,21 @@ int init_func_watchdog_reset(void);
 		#if defined(__ASSEMBLY__)
 			#define WATCHDOG_RESET /*XXX DO_NOT_DEL_THIS_COMMENT*/
 		#else
-			#define WATCHDOG_RESET() {}
+			#define WATCHDOG_RESET() { \
+				cyclic_run(); \
+			}
 		#endif /* __ASSEMBLY__ */
 	#endif /* CONFIG_WATCHDOG && !__ASSEMBLY__ */
 #endif /* CONFIG_HW_WATCHDOG */
 
+#if !defined(__ASSEMBLY__)
+/* Currently only needed for fs/cramfs/uncompress.c */
+static inline void watchdog_reset_func(void)
+{
+	WATCHDOG_RESET();
+}
+#endif
+
 /*
  * Prototypes from $(CPU)/cpu.c.
  */
-- 
2.36.1

[PATCH 4/7] cyclic: Integrate cyclic functionality at bootup in board_r/f

[Stefan Roese]

This patch adds a call to cyclic_init() to board_f/r.c, enabling the
common cyclic infrastructure. After this it's possible to add cyclic
functions via cyclic_register().

Signed-off-by: Stefan Roese <sr@denx.de>
---
 common/board_f.c | 2 ++
 common/board_r.c | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/common/board_f.c b/common/board_f.c
index 51d2f3c365e9..a02f435f477f 100644
--- a/common/board_f.c
+++ b/common/board_f.c
@@ -16,6 +16,7 @@
 #include <console.h>
 #include <cpu.h>
 #include <cpu_func.h>
+#include <cyclic.h>
 #include <dm.h>
 #include <env.h>
 #include <env_internal.h>
@@ -829,6 +830,7 @@ static const init_fnc_t init_sequence_f[] = {
 	initf_malloc,
 	log_init,
 	initf_bootstage,	/* uses its own timer, so does not need DM */
+	cyclic_init,
 	event_init,
 #ifdef CONFIG_BLOBLIST
 	bloblist_init,
diff --git a/common/board_r.c b/common/board_r.c
index 6f4aca2077d6..f8ea2a08ea12 100644
--- a/common/board_r.c
+++ b/common/board_r.c
@@ -13,6 +13,7 @@
 #include <api.h>
 #include <bootstage.h>
 #include <cpu_func.h>
+#include <cyclic.h>
 #include <exports.h>
 #include <flash.h>
 #include <hang.h>
@@ -600,6 +601,7 @@ static int run_main_loop(void)
 static init_fnc_t init_sequence_r[] = {
 	initr_trace,
 	initr_reloc,
+	cyclic_init,
 	event_init,
 	/* TODO: could x86/PPC have this also perhaps? */
 #if defined(CONFIG_ARM) || defined(CONFIG_RISCV)
-- 
2.36.1

[PATCH 5/7] cyclic: Add 'cyclic list' command

[Stefan Roese]

This patch adds the cyclic command, which currently only supports the
'list' subcommand, to list all currently registered cyclic functions.
Here an example:

=> cyclic list
function: cyclic_demo, cpu-time: 7010 us, frequency: 99.80 times/s
function: cyclic_demo2, cpu-time: 1 us, frequency: 1.13 times/s

As you can see, the cpu-time is accounted, so that cyclic functions
that take too long might be discovered. Additionally the frequency is
logged.

Signed-off-by: Stefan Roese <sr@denx.de>
---
 MAINTAINERS  |  1 +
 cmd/Kconfig  |  6 ++++++
 cmd/Makefile |  1 +
 cmd/cyclic.c | 40 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 48 insertions(+)
 create mode 100644 cmd/cyclic.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 920f515dd555..e0f5bf83a953 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -755,6 +755,7 @@ F:	doc/arch/m68k.rst
 CYCLIC
 M:	Stefan Roese <sr@denx.de>
 S:	Maintained
+F:	cmd/cyclic.c
 F:	common/cyclic.c
 F:	include/cyclic.h
 
diff --git a/cmd/Kconfig b/cmd/Kconfig
index 69c1814d24af..cb9bd0d6692a 100644
--- a/cmd/Kconfig
+++ b/cmd/Kconfig
@@ -2456,6 +2456,12 @@ config CMD_CBSYSINFO
 	  memory by coreboot before jumping to U-Boot. It can be useful for
 	  debugging the beaaviour of coreboot or U-Boot.
 
+config CMD_CYCLIC
+	bool "cyclic - Show information about cyclic functions"
+	help
+	  This enables the 'cyclic' command which provides information about
+	  cyclic excution functions.
+
 config CMD_DIAG
 	bool "diag - Board diagnostics"
 	help
diff --git a/cmd/Makefile b/cmd/Makefile
index 5e43a1e022e8..c5a4fc5a5cfc 100644
--- a/cmd/Makefile
+++ b/cmd/Makefile
@@ -56,6 +56,7 @@ obj-$(CONFIG_CMD_DIAG) += diag.o
 endif
 obj-$(CONFIG_CMD_ADTIMG) += adtimg.o
 obj-$(CONFIG_CMD_ABOOTIMG) += abootimg.o
+obj-$(CONFIG_CMD_CYCLIC) += cyclic.o
 obj-$(CONFIG_CMD_EVENT) += event.o
 obj-$(CONFIG_CMD_EXTENSION) += extension_board.o
 obj-$(CONFIG_CMD_ECHO) += echo.o
diff --git a/cmd/cyclic.c b/cmd/cyclic.c
new file mode 100644
index 000000000000..1714db18e480
--- /dev/null
+++ b/cmd/cyclic.c
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * A general-purpose cyclic execution infrastructure, to allow "small"
+ * (run-time wise) functions to be executed at a specified frequency.
+ * Things like LED blinking or watchdog triggering are examples for such
+ * tasks.
+ *
+ * Copyright (C) 2022 Stefan Roese <sr@denx.de>
+ */
+
+#include <common.h>
+#include <command.h>
+#include <cyclic.h>
+
+extern struct list_head cyclic_list;
+
+static int do_cyclic_list(struct cmd_tbl *cmdtp, int flag, int argc,
+			  char *const argv[])
+{
+	struct cyclic_struct *cyclic, *tmp;
+	uint64_t cnt, freq;
+
+	list_for_each_entry_safe(cyclic, tmp, &cyclic_list, list) {
+		cnt = cyclic->run_cnt * 1000000ULL * 100ULL;
+		freq = cnt / (timer_get_us() - cyclic->start_time_us);
+		printf("function: %s, cpu-time: %lld us, frequency: %lld.%02lld times/s\n",
+		       cyclic->name, cyclic->cpu_time_us,
+		       freq / 100, freq % 100);
+	}
+
+	return 0;
+}
+
+#ifdef CONFIG_SYS_LONGHELP
+static char cyclic_help_text[] =
+	"cyclic list   - list cyclic functions";
+#endif
+
+U_BOOT_CMD_WITH_SUBCMDS(cyclic, "Cyclic", cyclic_help_text,
+	U_BOOT_SUBCMD_MKENT(list, 1, 1, do_cyclic_list));
-- 
2.36.1

[PATCH 6/7] sandbox: Add cyclic demo function

[Stefan Roese]

This patch enables the cyclic infrastructure on sandbox and also adds
one simple example/demo functions using this cyclic functionality.

Signed-off-by: Stefan Roese <sr@denx.de>
---
 board/sandbox/sandbox.c   | 15 +++++++++++++++
 configs/sandbox_defconfig |  2 ++
 2 files changed, 17 insertions(+)

diff --git a/board/sandbox/sandbox.c b/board/sandbox/sandbox.c
index e054f300c4a5..3b48c9bf8cbb 100644
--- a/board/sandbox/sandbox.c
+++ b/board/sandbox/sandbox.c
@@ -6,6 +6,7 @@
 #include <common.h>
 #include <cpu_func.h>
 #include <cros_ec.h>
+#include <cyclic.h>
 #include <dm.h>
 #include <efi.h>
 #include <efi_loader.h>
@@ -16,6 +17,7 @@
 #include <asm/global_data.h>
 #include <asm/test.h>
 #include <asm/u-boot-sandbox.h>
+#include <linux/delay.h>
 #include <linux/kernel.h>
 #include <malloc.h>
 
@@ -105,8 +107,21 @@ int dram_init(void)
 	return 0;
 }
 
+static void cyclic_demo(void *ctx)
+{
+	/* Just a small dummy delay here */
+	udelay(10);
+}
+
 int board_init(void)
 {
+	struct cyclic_struct *cyclic;
+
+	/* Register demo cyclic function */
+	cyclic = cyclic_register(cyclic_demo, 10 * 1000, "cyclic_demo", NULL);
+	if (!cyclic)
+		printf("Registering of cyclic_demo failed\n");
+
 	return 0;
 }
 
diff --git a/configs/sandbox_defconfig b/configs/sandbox_defconfig
index c509a924e6b3..601c5dfe5a51 100644
--- a/configs/sandbox_defconfig
+++ b/configs/sandbox_defconfig
@@ -32,6 +32,7 @@ CONFIG_CONSOLE_RECORD_OUT_SIZE=0x6000
 CONFIG_PRE_CONSOLE_BUFFER=y
 CONFIG_LOG=y
 CONFIG_DISPLAY_BOARDINFO_LATE=y
+CONFIG_CYCLIC=y
 CONFIG_STACKPROTECTOR=y
 CONFIG_ANDROID_AB=y
 CONFIG_CMD_CPU=y
@@ -111,6 +112,7 @@ CONFIG_CMD_EROFS=y
 CONFIG_CMD_EXT4_WRITE=y
 CONFIG_CMD_SQUASHFS=y
 CONFIG_CMD_MTDPARTS=y
+CONFIG_CMD_CYCLIC=y
 CONFIG_CMD_STACKPROTECTOR_TEST=y
 CONFIG_MAC_PARTITION=y
 CONFIG_AMIGA_PARTITION=y
-- 
2.36.1

[PATCH 7/7] mips: octeon_nic23: Add PCIe FLR fixup via cyclic infrastructure

[Stefan Roese]

From: Aaron Williams <awilliams@marvell.com>

This patch adds a fixup function related to a PCIe FLR (Function Level
Reset) problem on the NIC23 PCIe board. This function is imported from
the Marvell Octeon 2013 U-Boot version as a (nearly) verbatim copy. It
uses the newly introduced cyclic infrastructure, so that this function
gets called every 100us, which is needed to detect this FLR issue.

Signed-off-by: Aaron Williams <awilliams@marvell.com>
Signed-off-by: Stefan Roese <sr@denx.de>
---
 board/Marvell/octeon_nic23/board.c | 197 +++++++++++++++++++++++++++++
 configs/octeon_nic23_defconfig     |   3 +
 2 files changed, 200 insertions(+)

diff --git a/board/Marvell/octeon_nic23/board.c b/board/Marvell/octeon_nic23/board.c
index 3e2c54444397..446fa5a659a8 100644
--- a/board/Marvell/octeon_nic23/board.c
+++ b/board/Marvell/octeon_nic23/board.c
@@ -3,8 +3,10 @@
  * Copyright (C) 2021-2022 Stefan Roese <sr@denx.de>
  */
 
+#include <cyclic.h>
 #include <dm.h>
 #include <ram.h>
+#include <time.h>
 #include <asm/gpio.h>
 
 #include <mach/octeon_ddr.h>
@@ -15,11 +17,90 @@
 #include <mach/cvmx-helper-cfg.h>
 #include <mach/cvmx-helper-util.h>
 #include <mach/cvmx-bgxx-defs.h>
+#include <mach/cvmx-dtx-defs.h>
 
 #include "board_ddr.h"
 
+/**
+ * cvmx_spem#_cfg_rd
+ *
+ * This register allows read access to the configuration in the PCIe core.
+ *
+ */
+union cvmx_spemx_cfg_rd {
+	u64 u64;
+	struct cvmx_spemx_cfg_rd_s {
+		u64 data                         : 32;
+		u64 addr                         : 32;
+	} s;
+	struct cvmx_spemx_cfg_rd_s            cn73xx;
+};
+
+/**
+ * cvmx_spem#_cfg_wr
+ *
+ * This register allows write access to the configuration in the PCIe core.
+ *
+ */
+union cvmx_spemx_cfg_wr {
+	u64 u64;
+	struct cvmx_spemx_cfg_wr_s {
+		u64 data                         : 32;
+		u64 addr                         : 32;
+	} s;
+	struct cvmx_spemx_cfg_wr_s            cn73xx;
+};
+
+/**
+ * cvmx_spem#_flr_pf_stopreq
+ *
+ * PF function level reset stop outbound requests register.
+ * Hardware automatically sets the STOPREQ bit for the PF when it enters a
+ * function level reset (FLR).  Software is responsible for clearing the STOPREQ
+ * bit but must not do so prior to hardware taking down the FLR, which could be
+ * as long as 100ms.  It may be appropriate for software to wait longer before clearing
+ * STOPREQ, software may need to drain deep DPI queues for example.
+ * Whenever SPEM receives a PF or child VF request mastered by CNXXXX over S2M (i.e. P or NP),
+ * when STOPREQ is set for the function, SPEM will discard the outgoing request
+ * before sending it to the PCIe core.  If a NP, SPEM will schedule an immediate
+ * SWI_RSP_ERROR completion for the request - no timeout is required.
+ * In both cases, SPEM()_DBG_PF()_INFO[P()_BMD_E] will be set and a error
+ * interrupt is generated.
+ *
+ * STOPREQ mimics the behavior of PCIEEP()_CFG001[ME] for outbound requests that will
+ * master the PCIe bus (P and NP).
+ *
+ * STOPREQ will have no effect on completions returned by CNXXXX over the S2M,
+ * nor on M2S traffic.
+ *
+ * When a PF()_STOPREQ is set, none of the associated
+ * PEM()_FLR_PF()_VF_STOPREQ[VF_STOPREQ] will be set.
+ *
+ * STOPREQ is reset when the MAC is reset, and is not reset after a chip soft reset.
+ */
+union cvmx_spemx_flr_pf_stopreq {
+	u64 u64;
+	struct cvmx_spemx_flr_pf_stopreq_s {
+		u64 reserved_3_63                : 61;
+		u64 pf2_stopreq                  : 1;
+		u64 pf1_stopreq                  : 1;
+		u64 pf0_stopreq                  : 1;
+	} s;
+	struct cvmx_spemx_flr_pf_stopreq_s    cn73xx;
+};
+
+#define CVMX_SPEMX_CFG_WR(offset)		0x00011800C0000028ull
+#define CVMX_SPEMX_CFG_RD(offset)		0x00011800C0000030ull
+#define CVMX_SPEMX_FLR_PF_STOPREQ(offset)	0x00011800C0000218ull
+
+#define DTX_SELECT_LTSSM		0x0
+#define DTX_SELECT_LTSSM_ENA		0x3ff
+#define LTSSM_L0			0x11
+
 #define NIC23_DEF_DRAM_FREQ		800
 
+static u32 pci_cfgspace_reg0[2] = { 0, 0 };
+
 static u8 octeon_nic23_cfg0_spd_values[512] = {
 	OCTEON_NIC23_CFG0_SPD_VALUES
 };
@@ -145,8 +226,118 @@ void board_configure_qlms(void)
 	      cvmx_qlm_measure_clock(4), cvmx_qlm_measure_clock(5));
 }
 
+/**
+ * If there is a PF FLR then the PCI EEPROM is not re-read.  In this case
+ * we need to re-program the vendor and device ID immediately after hardware
+ * completes FLR.
+ *
+ * PCI spec requires FLR to be completed within 100ms.  The user who triggered
+ * FLR expects hardware to finish FLR within 100ms, otherwise the user will
+ * end up reading DEVICE_ID incorrectly from the reset value.
+ * CN23XX exits FLR at any point between 66 and 99ms, so U-Boot has to wait
+ * 99ms to let hardware finish its part, then finish reprogramming the
+ * correct device ID before the end of 100ms.
+ *
+ * Note: this solution only works properly when there is no other activity
+ * within U-Boot for 100ms from the time FLR is triggered.
+ *
+ * This function gets called every 100usec.  If FLR happens during any
+ * other activity like bootloader/image update then it is possible that
+ * this function does not get called for more than the FLR period which will
+ * cause the PF device ID restore to happen after whoever initiated the FLR to
+ * read the incorrect device ID 0x9700 (reset value) instead of 0x9702
+ * (restored value).
+ */
+static void octeon_board_restore_pf(void *ctx)
+{
+	union cvmx_spemx_flr_pf_stopreq stopreq;
+	static bool start_initialized[2] = {false, false};
+	bool pf0_flag, pf1_flag;
+	u64 ltssm_bits;
+	const u64 pf_flr_wait_usecs = 99700;
+	u64 elapsed_usecs;
+	union cvmx_spemx_cfg_wr cfg_wr;
+	union cvmx_spemx_cfg_rd cfg_rd;
+	static u64 start_us[2];
+	int pf_num;
+
+	csr_wr(CVMX_DTX_SPEM_SELX(0), DTX_SELECT_LTSSM);
+	csr_rd(CVMX_DTX_SPEM_SELX(0));
+	csr_wr(CVMX_DTX_SPEM_ENAX(0), DTX_SELECT_LTSSM_ENA);
+	csr_rd(CVMX_DTX_SPEM_ENAX(0));
+	ltssm_bits = csr_rd(CVMX_DTX_SPEM_DATX(0));
+	if (((ltssm_bits >> 3) & 0x3f) != LTSSM_L0)
+		return;
+
+	stopreq.u64 = csr_rd(CVMX_SPEMX_FLR_PF_STOPREQ(0));
+	pf0_flag = stopreq.s.pf0_stopreq;
+	pf1_flag = stopreq.s.pf1_stopreq;
+	/* See if PF interrupt happened */
+	if (!(pf0_flag || pf1_flag))
+		return;
+
+	if (pf0_flag && !start_initialized[0]) {
+		start_initialized[0] = true;
+		start_us[0] = get_timer_us(0);
+	}
+
+	/* Store programmed PCIe DevID SPEM0 PF0 */
+	if (pf0_flag && !pci_cfgspace_reg0[0]) {
+		cfg_rd.s.addr = (0 << 24) | 0x0;
+		csr_wr(CVMX_SPEMX_CFG_RD(0), cfg_rd.u64);
+		cfg_rd.u64 = csr_rd(CVMX_SPEMX_CFG_RD(0));
+		pci_cfgspace_reg0[0] = cfg_rd.s.data;
+	}
+
+	if (pf1_flag && !start_initialized[1]) {
+		start_initialized[1] = true;
+		start_us[1] = get_timer_us(0);
+	}
+
+	/* Store programmed PCIe DevID SPEM0 PF1 */
+	if (pf1_flag && !pci_cfgspace_reg0[1]) {
+		cfg_rd.s.addr = (1 << 24) | 0x0;
+		csr_wr(CVMX_SPEMX_CFG_RD(0), cfg_rd.u64);
+		cfg_rd.u64 = csr_rd(CVMX_SPEMX_CFG_RD(0));
+		pci_cfgspace_reg0[1] = cfg_rd.s.data;
+	}
+
+	/* For PF, rewrite pci config space reg 0 */
+	for (pf_num = 0; pf_num < 2; pf_num++) {
+		if (!start_initialized[pf_num])
+			continue;
+
+		elapsed_usecs = get_timer_us(0) - start_us[pf_num];
+
+		if (elapsed_usecs > pf_flr_wait_usecs) {
+			/* Here, our measured FLR duration has passed;
+			 * check if device ID has been reset,
+			 * which indicates FLR completion (per MA team).
+			 */
+			cfg_rd.s.addr = (pf_num << 24) | 0x0;
+			csr_wr(CVMX_SPEMX_CFG_RD(0), cfg_rd.u64);
+			cfg_rd.u64 = csr_rd(CVMX_SPEMX_CFG_RD(0));
+			/* if DevID has NOT been reset, FLR is not yet
+			 * complete
+			 */
+			if (cfg_rd.s.data != pci_cfgspace_reg0[pf_num]) {
+				stopreq.s.pf0_stopreq = (pf_num == 0) ? 1 : 0;
+				stopreq.s.pf1_stopreq = (pf_num == 1) ? 1 : 0;
+				csr_wr(CVMX_SPEMX_FLR_PF_STOPREQ(0), stopreq.u64);
+
+				cfg_wr.u64 = 0;
+				cfg_wr.s.addr = (pf_num << 24) | 0;
+				cfg_wr.s.data = pci_cfgspace_reg0[pf_num];
+				csr_wr(CVMX_SPEMX_CFG_WR(0), cfg_wr.u64);
+				start_initialized[pf_num] = false;
+			}
+		}
+	}
+}
+
 int board_late_init(void)
 {
+	struct cyclic_struct *cyclic;
 	struct gpio_desc gpio = {};
 	ofnode node;
 
@@ -164,6 +355,12 @@ int board_late_init(void)
 
 	board_configure_qlms();
 
+	/* Register cyclic function for PCIe FLR fixup */
+	cyclic = cyclic_register(octeon_board_restore_pf, 100,
+				 "pcie_flr_fix", NULL);
+	if (!cyclic)
+		printf("Registering of cyclic function failed\n");
+
 	return 0;
 }
 
diff --git a/configs/octeon_nic23_defconfig b/configs/octeon_nic23_defconfig
index 3ab5838f03e5..d9ccf079cbd9 100644
--- a/configs/octeon_nic23_defconfig
+++ b/configs/octeon_nic23_defconfig
@@ -20,6 +20,8 @@ CONFIG_AHCI=y
 CONFIG_OF_BOARD_FIXUP=y
 CONFIG_SYS_CONSOLE_ENV_OVERWRITE=y
 # CONFIG_SYS_DEVICE_NULLDEV is not set
+CONFIG_CYCLIC=y
+CONFIG_CYCLIC_MAX_CPU_TIME_US=5000
 CONFIG_ARCH_MISC_INIT=y
 CONFIG_BOARD_EARLY_INIT_F=y
 CONFIG_BOARD_LATE_INIT=y
@@ -38,6 +40,7 @@ CONFIG_CMD_TIME=y
 CONFIG_CMD_EXT4=y
 CONFIG_CMD_FAT=y
 CONFIG_CMD_FS_GENERIC=y
+CONFIG_CMD_CYCLIC=y
 CONFIG_EFI_PARTITION=y
 CONFIG_ENV_IS_IN_SPI_FLASH=y
 CONFIG_TFTP_TSIZE=y
-- 
2.36.1

Re: [PATCH 0/7] Add support for cyclic function execution infrastruture

[Stefan Roese]

Hi Tom,

On 25.05.22 09:40, Stefan Roese wrote:
> This patchset adds the basic infrastructure to periodically execute
> code, e.g. all 100ms. Examples for such functions might be LED blinking
> etc. The functions that are hooked into this cyclic list should be
> small timewise as otherwise the execution of the other code that relies
> on a high frequent polling (e.g. UART rx char ready check) might be
> delayed too much. This patch also adds the Kconfig option
> CONFIG_CYCLIC_MAX_CPU_TIME_US, which configures the max allowed time
> for such a cyclic function. If it's execution time exceeds this time,
> this cyclic function will get removed from the cyclic list.
> 
> How is this cyclic functionality executed?
> This patchset integrates the main function responsible for calling all
> registered cyclic functions cyclic_run() into the common WATCHDOG_RESET
> macro. This guarantees that cyclic_run() is executed very often, which
> is necessary for the cyclic functions to get scheduled and executed at
> their configured periods.
> 
> This cyclic infrastructure will be used by a board specific function on
> the NIC23 MIPS Octeon board, which needs to check periodically, if a
> PCIe FLR has occurred.
> 
> Ideas how to continue:
> One idea is to rename WATCHDOG_RESET to something like SCHEDULE and
> move the watchdog_reset call into this cyclic infrastructure as well.
> Or to perhaps move the shell UART RX ready polling to a cyclic
> function.
> 
> It's also possible to extend the "cyclic" command, to support the
> creation of periodically executed shell commands (for testing etc).
> 
> Thanks,
> Stefan

Tom, did you find the time to take a look at this series? What are
you ideas / plans with it? I would very much like to get it merged
as one of it's pro's is, providing the base for bigger cleanups in
the current WATCHDOG_RESET mess in U-Boot at some time.

Thanks,
Stefan

> Aaron Williams (1):
>    mips: octeon_nic23: Add PCIe FLR fixup via cyclic infrastructure
> 
> Stefan Roese (6):
>    time: Import time_after64() and friends from Linux
>    cyclic: Add basic support for cyclic function execution infrastruture
>    cyclic: Integrate cyclic infrastructure into WATCHDOG_RESET
>    cyclic: Integrate cyclic functionality at bootup in board_r/f
>    cyclic: Add 'cyclic list' command
>    sandbox: Add cyclic demo function
> 
>   MAINTAINERS                        |   7 +
>   board/Marvell/octeon_nic23/board.c | 197 +++++++++++++++++++++++++++++
>   board/sandbox/sandbox.c            |  15 +++
>   cmd/Kconfig                        |   6 +
>   cmd/Makefile                       |   1 +
>   cmd/cyclic.c                       |  40 ++++++
>   common/Kconfig                     |  22 ++++
>   common/Makefile                    |   1 +
>   common/board_f.c                   |   2 +
>   common/board_r.c                   |   2 +
>   common/cyclic.c                    | 112 ++++++++++++++++
>   configs/octeon_nic23_defconfig     |   3 +
>   configs/sandbox_defconfig          |   2 +
>   fs/cramfs/uncompress.c             |   2 +-
>   include/cyclic.h                   |  84 ++++++++++++
>   include/time.h                     |  19 +++
>   include/watchdog.h                 |  23 +++-
>   17 files changed, 534 insertions(+), 4 deletions(-)
>   create mode 100644 cmd/cyclic.c
>   create mode 100644 common/cyclic.c
>   create mode 100644 include/cyclic.h
> 

Viele Grüße,
Stefan Roese

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-51 Fax: (+49)-8142-66989-80 Email: sr@denx.de

Re: [PATCH 0/7] Add support for cyclic function execution infrastruture

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 2298 bytes --]

On Mon, Jul 25, 2022 at 10:39:11AM +0200, Stefan Roese wrote:
> Hi Tom,
> 
> On 25.05.22 09:40, Stefan Roese wrote:
> > This patchset adds the basic infrastructure to periodically execute
> > code, e.g. all 100ms. Examples for such functions might be LED blinking
> > etc. The functions that are hooked into this cyclic list should be
> > small timewise as otherwise the execution of the other code that relies
> > on a high frequent polling (e.g. UART rx char ready check) might be
> > delayed too much. This patch also adds the Kconfig option
> > CONFIG_CYCLIC_MAX_CPU_TIME_US, which configures the max allowed time
> > for such a cyclic function. If it's execution time exceeds this time,
> > this cyclic function will get removed from the cyclic list.
> > 
> > How is this cyclic functionality executed?
> > This patchset integrates the main function responsible for calling all
> > registered cyclic functions cyclic_run() into the common WATCHDOG_RESET
> > macro. This guarantees that cyclic_run() is executed very often, which
> > is necessary for the cyclic functions to get scheduled and executed at
> > their configured periods.
> > 
> > This cyclic infrastructure will be used by a board specific function on
> > the NIC23 MIPS Octeon board, which needs to check periodically, if a
> > PCIe FLR has occurred.
> > 
> > Ideas how to continue:
> > One idea is to rename WATCHDOG_RESET to something like SCHEDULE and
> > move the watchdog_reset call into this cyclic infrastructure as well.
> > Or to perhaps move the shell UART RX ready polling to a cyclic
> > function.
> > 
> > It's also possible to extend the "cyclic" command, to support the
> > creation of periodically executed shell commands (for testing etc).
> > 
> > Thanks,
> > Stefan
> 
> Tom, did you find the time to take a look at this series? What are
> you ideas / plans with it? I would very much like to get it merged
> as one of it's pro's is, providing the base for bigger cleanups in
> the current WATCHDOG_RESET mess in U-Boot at some time.

Sorry, I had intended to chime in sooner.  Conceptually, OK, lets see
where this takes us.  Implementation-wise, this fails to build on
sandbox_spl and maybe others:
https://source.denx.de/u-boot/u-boot/-/jobs/471828

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]