[PATCH 0/2] PSCI: don't call CPU_OFF when it might fail

[PATCH 0/2] PSCI: don't call CPU_OFF when it might fail

[Mark Rutland]

[Apologies to all who received this already, I typo'd the LAKML address]

In certain conditions PSCI's CPU_OFF call may return DENIED (in the presence of
a UP OS on the secure world), or might not exist (in early implementations
prior to PSCI 0.2). Both the arm and arm64 callers fail to handle the former,
and the arm caller doesn't handle the latter.

We can determine wether a CPU_OFF call will succeed by calling
MIGRATE_INFO_TYPE. If we receive 2 (A.K.A. PSCI_0_2_TOS_MP) in response we know
that CPU_OFF should succeed, and in other cases it is necessary to discover
which CPU said OS lives on and attempt migration. As there don't seem to be any
UP implementations out there so far, for now we can limit ourselves to only
supporting CPU hotplug in the PSCI_0_2_TOS_MP case. Hotplug support on such
systems will require additional work, but at least they won't blow up by
default.

These patches add the necessary checks to the arm and arm64 PSCI callers to
prevent the issues described above. MIGRATE_INFO_TYPE is a mandatory part of
PSCI 0.2, so no conforming implementation should be adversely affected. Both
the ARM Trusted Firmware and KVM respond to MIGRATE_INFO_TYPE with
PSCI_0_2_TOS_MP, so should see no loss of functionality (verified on Juno). 

Thanks,
Mark.

Mark Rutland (2):
  arm64: psci: respect MIGRATE_INFO_TYPE
  arm: psci: don't call CPU_OFF blindly

 arch/arm/kernel/psci_smp.c | 36 +++++++++++++++++++++++++++++-------
 arch/arm64/kernel/psci.c   | 14 ++++++++++++++
 2 files changed, 43 insertions(+), 7 deletions(-)

-- 
1.9.1

[PATCH 1/2] arm64: psci: respect MIGRATE_INFO_TYPE

[Mark Rutland]

Commit e71246a23acb (PSCI: Add initial support for PSCIv0.2 functions)
added hooks for MIGRATE_INFO_TYPE, but didn't call it anywhere. This
means that currently a UP Trusted OS can reject CPU_OFF calls, which
will cause things to blow up.

This patch ensures we test MIGRATE_INFO_TYPE before calling CPU_OFF. If
there's a UP trusted OS hotplug is rejected. Full support for PSCI in
the presence of a UP trusted OS will require the use of MIGRATE and will
have to come as a later patch.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Ashwin Chaugule <ashwin.chaugule@linaro.org>
Cc: Rob Herring <robh@kernel.org>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
---
 arch/arm64/kernel/psci.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/arm64/kernel/psci.c b/arch/arm64/kernel/psci.c
index 5539547..a5407af 100644
--- a/arch/arm64/kernel/psci.c
+++ b/arch/arm64/kernel/psci.c
@@ -386,6 +386,20 @@ static int cpu_psci_cpu_disable(unsigned int cpu)
 	/* Fail early if we don't have CPU_OFF support */
 	if (!psci_ops.cpu_off)
 		return -EOPNOTSUPP;
+
+	/*
+	 * In the presence of a UP trusted OS, it might not be possible to
+	 * hotplug certain CPUs, and CPU_OFF may return (which would be bad).
+	 * Supporting a UP trusted OS requires careful use of
+	 * MIGRATE_INFO_UP_CPU and MIGRATE, so for now fail in the presence of
+	 * a UP Trusted OS.
+	 */
+	if (psci_ops.migrate_info_type &&
+		psci_ops.migrate_info_type() != PSCI_0_2_TOS_MP) {
+			pr_warn("Unable to handle UP trusted OS\n");
+			return -EPERM;
+	}
+
 	return 0;
 }
 
-- 
1.9.1

[PATCH 2/2] arm: psci: don't call CPU_OFF blindly

[Mark Rutland]

The generic PSCI operations for arm check the presence of a CPU_OFF ID
far too late, and in the absence of an ID will panic(), rather than
producing a warning.

This patch adds a psci_cpu_disable callback which tests the presence of
a CPU_OFF id. As this is called earlier than psci_cpu_die, the failure
can be handled gracefully without brining down the system. Additionally
a check is added for a UP trusted OS in the presence of PSCI 0.2+. Full
support will require the use of MIGRATE, but for now rejecting hotplug
will prevent psci_cpu_die from brining down the system.

The now redundant check for scpi_ops.cpu_off is removed from
psci_cpu_die. At the same time, the whitespace is corrected from seven
spaces to tabs.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Ashwin Chaugule <ashwin.chaugule@linaro.org>
Cc: Rob Herring <robh@kernel.org>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
---
 arch/arm/kernel/psci_smp.c | 36 +++++++++++++++++++++++++++++-------
 1 file changed, 29 insertions(+), 7 deletions(-)

Stefano, I've followed your lead with the __ref annotation here, but I couldn't
figure out why they exist on cpu_die and cpu_kill; it feels rather dodgy. Do
you know why they were added, or if they are superfluous?

There are some other cleanups that should happen here (static,
CPU_METHOD_OF_DECLARE), but those will come as a later cleanups series.

diff --git a/arch/arm/kernel/psci_smp.c b/arch/arm/kernel/psci_smp.c
index 28a1db4..2b00d3c 100644
--- a/arch/arm/kernel/psci_smp.c
+++ b/arch/arm/kernel/psci_smp.c
@@ -56,17 +56,38 @@ static int psci_boot_secondary(unsigned int cpu, struct task_struct *idle)
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
+int __ref psci_cpu_disable(unsigned int cpu)
+{
+	/* Fail early if we don't have CPU_OFF support */
+	if (!psci_ops.cpu_off)
+		return -EOPNOTSUPP;
+
+	/*
+	 * In the presence of a UP trusted OS, it might not be possible to
+	 * hotplug certain CPUs, and CPU_OFF may return (which would be bad).
+	 * Supporting a UP trusted OS requires careful use of
+	 * MIGRATE_INFO_UP_CPU and MIGRATE, so for now fail in the presence of
+	 * a UP Trusted OS.
+	 */
+	if (psci_ops.migrate_info_type &&
+		psci_ops.migrate_info_type() != PSCI_0_2_TOS_MP) {
+			pr_warn("Unable to handle UP trusted OS\n");
+			return -EPERM;
+	}
+
+	return 0;
+}
+
 void __ref psci_cpu_die(unsigned int cpu)
 {
-       const struct psci_power_state ps = {
-               .type = PSCI_POWER_STATE_TYPE_POWER_DOWN,
-       };
+	const struct psci_power_state ps = {
+		.type = PSCI_POWER_STATE_TYPE_POWER_DOWN,
+	};
 
-       if (psci_ops.cpu_off)
-               psci_ops.cpu_off(ps);
+	psci_ops.cpu_off(ps);
 
-       /* We should never return */
-       panic("psci: cpu %d failed to shutdown\n", cpu);
+	/* We should never return */
+	panic("psci: cpu %d failed to shutdown\n", cpu);
 }
 
 int __ref psci_cpu_kill(unsigned int cpu)
@@ -109,6 +130,7 @@ bool __init psci_smp_available(void)
 struct smp_operations __initdata psci_smp_ops = {
 	.smp_boot_secondary	= psci_boot_secondary,
 #ifdef CONFIG_HOTPLUG_CPU
+	.cpu_disable		= psci_cpu_disable,
 	.cpu_die		= psci_cpu_die,
 	.cpu_kill		= psci_cpu_kill,
 #endif
-- 
1.9.1

[PATCH 1/2] arm64: psci: respect MIGRATE_INFO_TYPE

[Sergei Shtylyov]

Hello.

On 9/5/2014 3:22 PM, Mark Rutland wrote:

> Commit e71246a23acb (PSCI: Add initial support for PSCIv0.2 functions)
> added hooks for MIGRATE_INFO_TYPE, but didn't call it anywhere. This
> means that currently a UP Trusted OS can reject CPU_OFF calls, which
> will cause things to blow up.

> This patch ensures we test MIGRATE_INFO_TYPE before calling CPU_OFF. If
> there's a UP trusted OS hotplug is rejected. Full support for PSCI in
> the presence of a UP trusted OS will require the use of MIGRATE and will
> have to come as a later patch.

> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Cc: Ashwin Chaugule <ashwin.chaugule@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Christoffer Dall <christoffer.dall@linaro.org>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> ---
>   arch/arm64/kernel/psci.c | 14 ++++++++++++++
>   1 file changed, 14 insertions(+)

> diff --git a/arch/arm64/kernel/psci.c b/arch/arm64/kernel/psci.c
> index 5539547..a5407af 100644
> --- a/arch/arm64/kernel/psci.c
> +++ b/arch/arm64/kernel/psci.c
> @@ -386,6 +386,20 @@ static int cpu_psci_cpu_disable(unsigned int cpu)
>   	/* Fail early if we don't have CPU_OFF support */
>   	if (!psci_ops.cpu_off)
>   		return -EOPNOTSUPP;
> +
> +	/*
> +	 * In the presence of a UP trusted OS, it might not be possible to
> +	 * hotplug certain CPUs, and CPU_OFF may return (which would be bad).
> +	 * Supporting a UP trusted OS requires careful use of
> +	 * MIGRATE_INFO_UP_CPU and MIGRATE, so for now fail in the presence of
> +	 * a UP Trusted OS.
> +	 */
> +	if (psci_ops.migrate_info_type &&
> +		psci_ops.migrate_info_type() != PSCI_0_2_TOS_MP) {
> +			pr_warn("Unable to handle UP trusted OS\n");
> +			return -EPERM;

    The above 2 lines are indented too much to the right.

> +	}
> +

WNR, Sergei

[PATCH 1/2] arm64: psci: respect MIGRATE_INFO_TYPE

[Mark Rutland]

[...]

> > +	if (psci_ops.migrate_info_type &&
> > +		psci_ops.migrate_info_type() != PSCI_0_2_TOS_MP) {
> > +			pr_warn("Unable to handle UP trusted OS\n");
> > +			return -EPERM;
> 
>     The above 2 lines are indented too much to the right.

Noted; locally fixed. Thanks.

The logic is the same regardless, so I won't send out a v2 unless
someone has an issue with that logic, rather than the style.

Mark.

[PATCH 2/2] arm: psci: don't call CPU_OFF blindly

[Stefano Stabellini]

On Fri, 5 Sep 2014, Mark Rutland wrote:
> The generic PSCI operations for arm check the presence of a CPU_OFF ID
> far too late, and in the absence of an ID will panic(), rather than
> producing a warning.
> 
> This patch adds a psci_cpu_disable callback which tests the presence of
> a CPU_OFF id. As this is called earlier than psci_cpu_die, the failure
> can be handled gracefully without brining down the system. Additionally
> a check is added for a UP trusted OS in the presence of PSCI 0.2+. Full
> support will require the use of MIGRATE, but for now rejecting hotplug
> will prevent psci_cpu_die from brining down the system.
> 
> The now redundant check for scpi_ops.cpu_off is removed from
> psci_cpu_die. At the same time, the whitespace is corrected from seven
> spaces to tabs.
> 
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Cc: Ashwin Chaugule <ashwin.chaugule@linaro.org>
> Cc: Rob Herring <robh@kernel.org>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Christoffer Dall <christoffer.dall@linaro.org>
> Cc: Will Deacon <will.deacon@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> ---
>  arch/arm/kernel/psci_smp.c | 36 +++++++++++++++++++++++++++++-------
>  1 file changed, 29 insertions(+), 7 deletions(-)
> 
> Stefano, I've followed your lead with the __ref annotation here, but I couldn't
> figure out why they exist on cpu_die and cpu_kill; it feels rather dodgy. Do
> you know why they were added, or if they are superfluous?

I don't think that __ref is needed.
That particular snipped of code came from Rob Herring, maybe he knows
why it was added in the first place.



> There are some other cleanups that should happen here (static,
> CPU_METHOD_OF_DECLARE), but those will come as a later cleanups series.
> 
> diff --git a/arch/arm/kernel/psci_smp.c b/arch/arm/kernel/psci_smp.c
> index 28a1db4..2b00d3c 100644
> --- a/arch/arm/kernel/psci_smp.c
> +++ b/arch/arm/kernel/psci_smp.c
> @@ -56,17 +56,38 @@ static int psci_boot_secondary(unsigned int cpu, struct task_struct *idle)
>  }
>  
>  #ifdef CONFIG_HOTPLUG_CPU
> +int __ref psci_cpu_disable(unsigned int cpu)
> +{
> +	/* Fail early if we don't have CPU_OFF support */
> +	if (!psci_ops.cpu_off)
> +		return -EOPNOTSUPP;
> +
> +	/*
> +	 * In the presence of a UP trusted OS, it might not be possible to
> +	 * hotplug certain CPUs, and CPU_OFF may return (which would be bad).
> +	 * Supporting a UP trusted OS requires careful use of
> +	 * MIGRATE_INFO_UP_CPU and MIGRATE, so for now fail in the presence of
> +	 * a UP Trusted OS.
> +	 */
> +	if (psci_ops.migrate_info_type &&
> +		psci_ops.migrate_info_type() != PSCI_0_2_TOS_MP) {
> +			pr_warn("Unable to handle UP trusted OS\n");
> +			return -EPERM;
> +	}
> +
> +	return 0;
> +}
> +
>  void __ref psci_cpu_die(unsigned int cpu)
>  {
> -       const struct psci_power_state ps = {
> -               .type = PSCI_POWER_STATE_TYPE_POWER_DOWN,
> -       };
> +	const struct psci_power_state ps = {
> +		.type = PSCI_POWER_STATE_TYPE_POWER_DOWN,
> +	};
>  
> -       if (psci_ops.cpu_off)
> -               psci_ops.cpu_off(ps);
> +	psci_ops.cpu_off(ps);
>  
> -       /* We should never return */
> -       panic("psci: cpu %d failed to shutdown\n", cpu);
> +	/* We should never return */
> +	panic("psci: cpu %d failed to shutdown\n", cpu);
>  }
>  
>  int __ref psci_cpu_kill(unsigned int cpu)
> @@ -109,6 +130,7 @@ bool __init psci_smp_available(void)
>  struct smp_operations __initdata psci_smp_ops = {
>  	.smp_boot_secondary	= psci_boot_secondary,
>  #ifdef CONFIG_HOTPLUG_CPU
> +	.cpu_disable		= psci_cpu_disable,
>  	.cpu_die		= psci_cpu_die,
>  	.cpu_kill		= psci_cpu_kill,
>  #endif
> -- 
> 1.9.1
> 

[PATCH 0/2] PSCI: don't call CPU_OFF when it might fail

[Stefano Stabellini]

On Fri, 5 Sep 2014, Mark Rutland wrote:
> [Apologies to all who received this already, I typo'd the LAKML address]
> 
> In certain conditions PSCI's CPU_OFF call may return DENIED (in the presence of
> a UP OS on the secure world), or might not exist (in early implementations
> prior to PSCI 0.2). Both the arm and arm64 callers fail to handle the former,
> and the arm caller doesn't handle the latter.
> 
> We can determine wether a CPU_OFF call will succeed by calling
> MIGRATE_INFO_TYPE. If we receive 2 (A.K.A. PSCI_0_2_TOS_MP) in response we know
> that CPU_OFF should succeed, and in other cases it is necessary to discover
> which CPU said OS lives on and attempt migration. As there don't seem to be any
> UP implementations out there so far, for now we can limit ourselves to only
> supporting CPU hotplug in the PSCI_0_2_TOS_MP case. Hotplug support on such
> systems will require additional work, but at least they won't blow up by
> default.
> 
> These patches add the necessary checks to the arm and arm64 PSCI callers to
> prevent the issues described above. MIGRATE_INFO_TYPE is a mandatory part of
> PSCI 0.2, so no conforming implementation should be adversely affected. Both
> the ARM Trusted Firmware and KVM respond to MIGRATE_INFO_TYPE with
> PSCI_0_2_TOS_MP, so should see no loss of functionality (verified on Juno). 
> 
> Thanks,
> Mark.
> 
> Mark Rutland (2):
>   arm64: psci: respect MIGRATE_INFO_TYPE
>   arm: psci: don't call CPU_OFF blindly
> 
>  arch/arm/kernel/psci_smp.c | 36 +++++++++++++++++++++++++++++-------
>  arch/arm64/kernel/psci.c   | 14 ++++++++++++++
>  2 files changed, 43 insertions(+), 7 deletions(-)
 
The patches look good to me and AFACT they would work fine on Xen 4.4
and the soon to be release 4.5.

[PATCH 2/2] arm: psci: don't call CPU_OFF blindly

[Mark Rutland]

On Fri, Sep 05, 2014 at 09:48:46PM +0100, Stefano Stabellini wrote:
> On Fri, 5 Sep 2014, Mark Rutland wrote:
> > The generic PSCI operations for arm check the presence of a CPU_OFF ID
> > far too late, and in the absence of an ID will panic(), rather than
> > producing a warning.
> > 
> > This patch adds a psci_cpu_disable callback which tests the presence of
> > a CPU_OFF id. As this is called earlier than psci_cpu_die, the failure
> > can be handled gracefully without brining down the system. Additionally
> > a check is added for a UP trusted OS in the presence of PSCI 0.2+. Full
> > support will require the use of MIGRATE, but for now rejecting hotplug
> > will prevent psci_cpu_die from brining down the system.
> > 
> > The now redundant check for scpi_ops.cpu_off is removed from
> > psci_cpu_die. At the same time, the whitespace is corrected from seven
> > spaces to tabs.
> > 
> > Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> > Cc: Ashwin Chaugule <ashwin.chaugule@linaro.org>
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > Cc: Ian Campbell <ian.campbell@citrix.com>
> > Cc: Christoffer Dall <christoffer.dall@linaro.org>
> > Cc: Will Deacon <will.deacon@arm.com>
> > Cc: Catalin Marinas <catalin.marinas@arm.com>
> > ---
> >  arch/arm/kernel/psci_smp.c | 36 +++++++++++++++++++++++++++++-------
> >  1 file changed, 29 insertions(+), 7 deletions(-)
> > 
> > Stefano, I've followed your lead with the __ref annotation here, but I couldn't
> > figure out why they exist on cpu_die and cpu_kill; it feels rather dodgy. Do
> > you know why they were added, or if they are superfluous?
> 
> I don't think that __ref is needed.
> That particular snipped of code came from Rob Herring, maybe he knows
> why it was added in the first place.

I've traced that back, but can't see any rationale. Perhaps that had
something to do with __cpuinit/__cpuexit, but there doesn't seem to be
any reason for them now. I guess the __ref annotation on cpu_die in
arch/arm/kernel/smp.c can go too given __cpuinit and __cpuexit are gone.

Rob, do you have any idea either way?

Thanks,
Mark.

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-March/158570.html

[PATCH 0/2] PSCI: don't call CPU_OFF when it might fail

[Mark Rutland]

On Fri, Sep 05, 2014 at 09:53:05PM +0100, Stefano Stabellini wrote:
> On Fri, 5 Sep 2014, Mark Rutland wrote:
> > [Apologies to all who received this already, I typo'd the LAKML address]
> > 
> > In certain conditions PSCI's CPU_OFF call may return DENIED (in the presence of
> > a UP OS on the secure world), or might not exist (in early implementations
> > prior to PSCI 0.2). Both the arm and arm64 callers fail to handle the former,
> > and the arm caller doesn't handle the latter.
> > 
> > We can determine wether a CPU_OFF call will succeed by calling
> > MIGRATE_INFO_TYPE. If we receive 2 (A.K.A. PSCI_0_2_TOS_MP) in response we know
> > that CPU_OFF should succeed, and in other cases it is necessary to discover
> > which CPU said OS lives on and attempt migration. As there don't seem to be any
> > UP implementations out there so far, for now we can limit ourselves to only
> > supporting CPU hotplug in the PSCI_0_2_TOS_MP case. Hotplug support on such
> > systems will require additional work, but at least they won't blow up by
> > default.
> > 
> > These patches add the necessary checks to the arm and arm64 PSCI callers to
> > prevent the issues described above. MIGRATE_INFO_TYPE is a mandatory part of
> > PSCI 0.2, so no conforming implementation should be adversely affected. Both
> > the ARM Trusted Firmware and KVM respond to MIGRATE_INFO_TYPE with
> > PSCI_0_2_TOS_MP, so should see no loss of functionality (verified on Juno). 
> > 
> > Thanks,
> > Mark.
> > 
> > Mark Rutland (2):
> >   arm64: psci: respect MIGRATE_INFO_TYPE
> >   arm: psci: don't call CPU_OFF blindly
> > 
> >  arch/arm/kernel/psci_smp.c | 36 +++++++++++++++++++++++++++++-------
> >  arch/arm64/kernel/psci.c   | 14 ++++++++++++++
> >  2 files changed, 43 insertions(+), 7 deletions(-)
>  
> The patches look good to me and AFACT they would work fine on Xen 4.4
> and the soon to be release 4.5.

Cheers.

Can I take that as an ack?

Mark.

[PATCH 0/2] PSCI: don't call CPU_OFF when it might fail

[Stefano Stabellini]

On Mon, 8 Sep 2014, Mark Rutland wrote:
> On Fri, Sep 05, 2014 at 09:53:05PM +0100, Stefano Stabellini wrote:
> > On Fri, 5 Sep 2014, Mark Rutland wrote:
> > > [Apologies to all who received this already, I typo'd the LAKML address]
> > > 
> > > In certain conditions PSCI's CPU_OFF call may return DENIED (in the presence of
> > > a UP OS on the secure world), or might not exist (in early implementations
> > > prior to PSCI 0.2). Both the arm and arm64 callers fail to handle the former,
> > > and the arm caller doesn't handle the latter.
> > > 
> > > We can determine wether a CPU_OFF call will succeed by calling
> > > MIGRATE_INFO_TYPE. If we receive 2 (A.K.A. PSCI_0_2_TOS_MP) in response we know
> > > that CPU_OFF should succeed, and in other cases it is necessary to discover
> > > which CPU said OS lives on and attempt migration. As there don't seem to be any
> > > UP implementations out there so far, for now we can limit ourselves to only
> > > supporting CPU hotplug in the PSCI_0_2_TOS_MP case. Hotplug support on such
> > > systems will require additional work, but at least they won't blow up by
> > > default.
> > > 
> > > These patches add the necessary checks to the arm and arm64 PSCI callers to
> > > prevent the issues described above. MIGRATE_INFO_TYPE is a mandatory part of
> > > PSCI 0.2, so no conforming implementation should be adversely affected. Both
> > > the ARM Trusted Firmware and KVM respond to MIGRATE_INFO_TYPE with
> > > PSCI_0_2_TOS_MP, so should see no loss of functionality (verified on Juno). 
> > > 
> > > Thanks,
> > > Mark.
> > > 
> > > Mark Rutland (2):
> > >   arm64: psci: respect MIGRATE_INFO_TYPE
> > >   arm: psci: don't call CPU_OFF blindly
> > > 
> > >  arch/arm/kernel/psci_smp.c | 36 +++++++++++++++++++++++++++++-------
> > >  arch/arm64/kernel/psci.c   | 14 ++++++++++++++
> > >  2 files changed, 43 insertions(+), 7 deletions(-)
> >  
> > The patches look good to me and AFACT they would work fine on Xen 4.4
> > and the soon to be release 4.5.
> 
> Cheers.
> 
> Can I take that as an ack?

Yes