[PATCH] powerpc/opal: Wake up kopald polling thread before waiting for events

[PATCH] powerpc/opal: Wake up kopald polling thread before waiting for events

[Benjamin Herrenschmidt]

On some environments (prototype machines, some simulators, etc...)
there is no functional interrupt source to signal completion, so
we rely on the fairly slow OPAL heartbeat.

In a number of cases, the calls complete very quickly or even
immediately. We've observed that it helps a lot to wakeup the OPAL
heartbeat thread before waiting for event in those cases, it will
call OPAL immediately to collect completions for anything that
finished fast enough.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
 arch/powerpc/include/asm/opal.h             |  2 ++
 arch/powerpc/platforms/powernv/opal-async.c |  5 +++++
 arch/powerpc/platforms/powernv/opal.c       | 12 ++++++++++--
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 6135816..0c76bc0 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -277,6 +277,8 @@ extern int opal_error_code(int rc);
 
 ssize_t opal_msglog_copy(char *to, loff_t pos, size_t count);
 
+void opal_wake_poller(void);
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_POWERPC_OPAL_H */
diff --git a/arch/powerpc/platforms/powernv/opal-async.c b/arch/powerpc/platforms/powernv/opal-async.c
index bdc8c0c..83bebee 100644
--- a/arch/powerpc/platforms/powernv/opal-async.c
+++ b/arch/powerpc/platforms/powernv/opal-async.c
@@ -117,6 +117,11 @@ int opal_async_wait_response(uint64_t token, struct opal_msg *msg)
 		return -EINVAL;
 	}
 
+	/* Wakeup the poller before we wait for events to speed things
+	 * up on platforms or simulators where the interrupts aren't
+	 * functional.
+	 */
+	opal_wake_poller();
 	wait_event(opal_async_wait, test_bit(token, opal_async_complete_map));
 	memcpy(msg, &opal_async_responses[token], sizeof(*msg));
 
diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
index 802f3b7..7f13302 100644
--- a/arch/powerpc/platforms/powernv/opal.c
+++ b/arch/powerpc/platforms/powernv/opal.c
@@ -55,6 +55,7 @@ struct device_node *opal_node;
 static DEFINE_SPINLOCK(opal_write_lock);
 static struct atomic_notifier_head opal_msg_notifier_head[OPAL_MSG_TYPE_MAX];
 static uint32_t opal_heartbeat;
+static struct task_struct *kopald_tsk;
 
 void opal_configure_cores(void)
 {
@@ -650,6 +651,7 @@ static void opal_i2c_create_devs(void)
 
 static int kopald(void *unused)
 {
+	unsigned long timeout = msecs_to_jiffies(opal_heartbeat) + 1;
 	__be64 events;
 
 	set_freezable();
@@ -657,12 +659,18 @@ static int kopald(void *unused)
 		try_to_freeze();
 		opal_poll_events(&events);
 		opal_handle_events(be64_to_cpu(events));
-		msleep_interruptible(opal_heartbeat);
+		schedule_timeout_interruptible(timeout);
 	} while (!kthread_should_stop());
 
 	return 0;
 }
 
+void opal_wake_poller(void)
+{
+	if (kopald_tsk)
+		wake_up_process(kopald_tsk);
+}
+
 static void opal_init_heartbeat(void)
 {
 	/* Old firwmware, we assume the HVC heartbeat is sufficient */
@@ -671,7 +679,7 @@ static void opal_init_heartbeat(void)
 		opal_heartbeat = 0;
 
 	if (opal_heartbeat)
-		kthread_run(kopald, NULL, "kopald");
+		kopald_tsk = kthread_run(kopald, NULL, "kopald");
 }
 
 static int __init opal_init(void)

Re: powerpc/opal: Wake up kopald polling thread before waiting for events

[Michael Ellerman]

On Mon, 2016-04-07 at 04:51:44 UTC, Benjamin Herrenschmidt wrote:
> On some environments (prototype machines, some simulators, etc...)
> there is no functional interrupt source to signal completion, so
> we rely on the fairly slow OPAL heartbeat.
> 
> In a number of cases, the calls complete very quickly or even
> immediately. We've observed that it helps a lot to wakeup the OPAL
> heartbeat thread before waiting for event in those cases, it will
> call OPAL immediately to collect completions for anything that
> finished fast enough.

Any performance/jitter impact on normal systems?

cheers

Re: powerpc/opal: Wake up kopald polling thread before waiting for events

[Benjamin Herrenschmidt]

On Mon, 2016-07-04 at 16:11 +1000, Michael Ellerman wrote:
> On Mon, 2016-04-07 at 04:51:44 UTC, Benjamin Herrenschmidt wrote:
> > On some environments (prototype machines, some simulators, etc...)
> > there is no functional interrupt source to signal completion, so
> > we rely on the fairly slow OPAL heartbeat.
> > 
> > In a number of cases, the calls complete very quickly or even
> > immediately. We've observed that it helps a lot to wakeup the OPAL
> > heartbeat thread before waiting for event in those cases, it will
> > call OPAL immediately to collect completions for anything that
> > finished fast enough.
> 
> Any performance/jitter impact on normal systems?

Not that I can think of.

Cheers,
Ben.

Re: [PATCH] powerpc/opal: Wake up kopald polling thread before waiting for events

[Michael Neuling]

On Mon, 2016-07-04 at 14:51 +1000, Benjamin Herrenschmidt wrote:
> On some environments (prototype machines, some simulators, etc...)
> there is no functional interrupt source to signal completion, so
> we rely on the fairly slow OPAL heartbeat.
>=20
> In a number of cases, the calls complete very quickly or even
> immediately. We've observed that it helps a lot to wakeup the OPAL
> heartbeat thread before waiting for event in those cases, it will
> call OPAL immediately to collect completions for anything that
> finished fast enough.
>=20
> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

Without this using drivers/mtd/devices/powernv_flash.c on mambo is
impossible. =C2=A0Feels like it's at least 1000x faster.

Acked-By: Michael Neuling <mikey@neuling.org>

> ---
> =C2=A0arch/powerpc/include/asm/opal.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0|=C2=A0=C2=A02 ++
> =C2=A0arch/powerpc/platforms/powernv/opal-async.c |=C2=A0=C2=A05 +++++
> =C2=A0arch/powerpc/platforms/powernv/opal.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0| 12 ++++++++++--
> =C2=A03 files changed, 17 insertions(+), 2 deletions(-)
>=20
> diff --git a/arch/powerpc/include/asm/opal.h
> b/arch/powerpc/include/asm/opal.h
> index 6135816..0c76bc0 100644
> --- a/arch/powerpc/include/asm/opal.h
> +++ b/arch/powerpc/include/asm/opal.h
> @@ -277,6 +277,8 @@ extern int opal_error_code(int rc);
> =C2=A0
> =C2=A0ssize_t opal_msglog_copy(char *to, loff_t pos, size_t count);
> =C2=A0
> +void opal_wake_poller(void);
> +
> =C2=A0#endif /* __ASSEMBLY__ */
> =C2=A0
> =C2=A0#endif /* _ASM_POWERPC_OPAL_H */
> diff --git a/arch/powerpc/platforms/powernv/opal-async.c
> b/arch/powerpc/platforms/powernv/opal-async.c
> index bdc8c0c..83bebee 100644
> --- a/arch/powerpc/platforms/powernv/opal-async.c
> +++ b/arch/powerpc/platforms/powernv/opal-async.c
> @@ -117,6 +117,11 @@ int opal_async_wait_response(uint64_t token, struct
> opal_msg *msg)
> =C2=A0		return -EINVAL;
> =C2=A0	}
> =C2=A0
> +	/* Wakeup the poller before we wait for events to speed things
> +	=C2=A0* up on platforms or simulators where the interrupts aren't
> +	=C2=A0* functional.
> +	=C2=A0*/
> +	opal_wake_poller();
> =C2=A0	wait_event(opal_async_wait, test_bit(token,
> opal_async_complete_map));
> =C2=A0	memcpy(msg, &opal_async_responses[token], sizeof(*msg));
> =C2=A0
> diff --git a/arch/powerpc/platforms/powernv/opal.c
> b/arch/powerpc/platforms/powernv/opal.c
> index 802f3b7..7f13302 100644
> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -55,6 +55,7 @@ struct device_node *opal_node;
> =C2=A0static DEFINE_SPINLOCK(opal_write_lock);
> =C2=A0static struct atomic_notifier_head
> opal_msg_notifier_head[OPAL_MSG_TYPE_MAX];
> =C2=A0static uint32_t opal_heartbeat;
> +static struct task_struct *kopald_tsk;
> =C2=A0
> =C2=A0void opal_configure_cores(void)
> =C2=A0{
> @@ -650,6 +651,7 @@ static void opal_i2c_create_devs(void)
> =C2=A0
> =C2=A0static int kopald(void *unused)
> =C2=A0{
> +	unsigned long timeout =3D msecs_to_jiffies(opal_heartbeat) + 1;
> =C2=A0	__be64 events;
> =C2=A0
> =C2=A0	set_freezable();
> @@ -657,12 +659,18 @@ static int kopald(void *unused)
> =C2=A0		try_to_freeze();
> =C2=A0		opal_poll_events(&events);
> =C2=A0		opal_handle_events(be64_to_cpu(events));
> -		msleep_interruptible(opal_heartbeat);
> +		schedule_timeout_interruptible(timeout);
> =C2=A0	} while (!kthread_should_stop());
> =C2=A0
> =C2=A0	return 0;
> =C2=A0}
> =C2=A0
> +void opal_wake_poller(void)
> +{
> +	if (kopald_tsk)
> +		wake_up_process(kopald_tsk);
> +}
> +
> =C2=A0static void opal_init_heartbeat(void)
> =C2=A0{
> =C2=A0	/* Old firwmware, we assume the HVC heartbeat is sufficient */
> @@ -671,7 +679,7 @@ static void opal_init_heartbeat(void)
> =C2=A0		opal_heartbeat =3D 0;
> =C2=A0
> =C2=A0	if (opal_heartbeat)
> -		kthread_run(kopald, NULL, "kopald");
> +		kopald_tsk =3D kthread_run(kopald, NULL, "kopald");
> =C2=A0}
> =C2=A0
> =C2=A0static int __init opal_init(void)
>=20
>=20

Re: powerpc/opal: Wake up kopald polling thread before waiting for events

[Michael Ellerman]

On Mon, 2016-04-07 at 04:51:44 UTC, Benjamin Herrenschmidt wrote:
> On some environments (prototype machines, some simulators, etc...)
> there is no functional interrupt source to signal completion, so
> we rely on the fairly slow OPAL heartbeat.
> 
> In a number of cases, the calls complete very quickly or even
> immediately. We've observed that it helps a lot to wakeup the OPAL
> heartbeat thread before waiting for event in those cases, it will
> call OPAL immediately to collect completions for anything that
> finished fast enough.
> 
> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Acked-By: Michael Neuling <mikey@neuling.org>

Applied to powerpc next, thanks.

https://git.kernel.org/powerpc/c/a203658b5ed37c11e5016d3fbb

cheers