[PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Abhilash Kesavan]

Setup the mcpm entry address again on system resume as the
iRAM contents are lost across an s2r cycle.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
This has been tested after applying the Exynos5420 S2R support series
along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
Pit.

 arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
 1 file changed, 21 insertions(+), 10 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 8c610e2..0bf734d 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -26,6 +27,7 @@
 #define EXYNOS5420_CPUS_PER_CLUSTER	4
 #define EXYNOS5420_NR_CLUSTERS		2
 
+static void __iomem *ns_sram_base_addr;
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	int ret;
 
 	node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
@@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
 
 	pr_info("Exynos MCPM support installed\n");
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
-- 
1.8.3.2

[PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Abhilash Kesavan]

Setup the mcpm entry address again on system resume as the
iRAM contents are lost across an s2r cycle.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
This has been tested after applying the Exynos5420 S2R support series
along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
Pit.

 arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
 1 file changed, 21 insertions(+), 10 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 8c610e2..0bf734d 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -26,6 +27,7 @@
 #define EXYNOS5420_CPUS_PER_CLUSTER	4
 #define EXYNOS5420_NR_CLUSTERS		2
 
+static void __iomem *ns_sram_base_addr;
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	int ret;
 
 	node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
@@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
 
 	pr_info("Exynos MCPM support installed\n");
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
-- 
1.8.3.2

Re: [PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Abhilash Kesavan]

Hi,

On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
> Setup the mcpm entry address again on system resume as the
> iRAM contents are lost across an s2r cycle.
>
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> ---
> This has been tested after applying the Exynos5420 S2R support series
> along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
> Pit.
>
>  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
>  1 file changed, 21 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> index 8c610e2..0bf734d 100644
> --- a/arch/arm/mach-exynos/mcpm-exynos.c
> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> @@ -15,6 +15,7 @@
>  #include <linux/delay.h>
>  #include <linux/io.h>
>  #include <linux/of_address.h>
> +#include <linux/syscore_ops.h>
>
>  #include <asm/cputype.h>
>  #include <asm/cp15.h>
> @@ -26,6 +27,7 @@
>  #define EXYNOS5420_CPUS_PER_CLUSTER    4
>  #define EXYNOS5420_NR_CLUSTERS         2
>
> +static void __iomem *ns_sram_base_addr;
>  /*
>   * The common v7_exit_coherency_flush API could not be used because of the
>   * Erratum 799270 workaround. This macro is the same as the common one (in
> @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>         {},
>  };
>
> +static void exynos_mcpm_setup_entry_point(void)
> +{
> +       /*
> +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> +        * as part of secondary_cpu_start().  Let's redirect it to the
> +        * mcpm_entry_point(). This is done during both secondary boot-up as
> +        * well as system resume.
> +        */
> +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> +}
> +
> +static struct syscore_ops exynos_mcpm_syscore_ops = {
> +       .resume = exynos_mcpm_setup_entry_point,
> +};
> +
>  static int __init exynos_mcpm_init(void)
>  {
>         struct device_node *node;
> -       void __iomem *ns_sram_base_addr;
>         int ret;
>
>         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
> @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
>
>         pr_info("Exynos MCPM support installed\n");
>
> -       /*
> -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> -        * as part of secondary_cpu_start().  Let's redirect it to the
> -        * mcpm_entry_point().
> -        */
> -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> +       exynos_mcpm_setup_entry_point();
>
> -       iounmap(ns_sram_base_addr);
> +       register_syscore_ops(&exynos_mcpm_syscore_ops);
>
>         return ret;
>  }

This patch alone is not enough to bring the 8 cores back up post
resume on exynos5420, We need to enable the boot cluster snoops as
well.

Nicolas, if I add code to enable the boot cluster CCI in
mach-exynos/sleep.S, all 8 cores come up. However, if I use
mcpm_loopback as part of the newly added resume function to do the
same, I get a hang on resume.
Is there anything else that I need to take care of while doing this ?

Regards,
Abhilash
> --
> 1.8.3.2
>

[PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Abhilash Kesavan]

Hi,

On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
> Setup the mcpm entry address again on system resume as the
> iRAM contents are lost across an s2r cycle.
>
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> ---
> This has been tested after applying the Exynos5420 S2R support series
> along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
> Pit.
>
>  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
>  1 file changed, 21 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> index 8c610e2..0bf734d 100644
> --- a/arch/arm/mach-exynos/mcpm-exynos.c
> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> @@ -15,6 +15,7 @@
>  #include <linux/delay.h>
>  #include <linux/io.h>
>  #include <linux/of_address.h>
> +#include <linux/syscore_ops.h>
>
>  #include <asm/cputype.h>
>  #include <asm/cp15.h>
> @@ -26,6 +27,7 @@
>  #define EXYNOS5420_CPUS_PER_CLUSTER    4
>  #define EXYNOS5420_NR_CLUSTERS         2
>
> +static void __iomem *ns_sram_base_addr;
>  /*
>   * The common v7_exit_coherency_flush API could not be used because of the
>   * Erratum 799270 workaround. This macro is the same as the common one (in
> @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>         {},
>  };
>
> +static void exynos_mcpm_setup_entry_point(void)
> +{
> +       /*
> +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> +        * as part of secondary_cpu_start().  Let's redirect it to the
> +        * mcpm_entry_point(). This is done during both secondary boot-up as
> +        * well as system resume.
> +        */
> +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> +}
> +
> +static struct syscore_ops exynos_mcpm_syscore_ops = {
> +       .resume = exynos_mcpm_setup_entry_point,
> +};
> +
>  static int __init exynos_mcpm_init(void)
>  {
>         struct device_node *node;
> -       void __iomem *ns_sram_base_addr;
>         int ret;
>
>         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
> @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
>
>         pr_info("Exynos MCPM support installed\n");
>
> -       /*
> -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> -        * as part of secondary_cpu_start().  Let's redirect it to the
> -        * mcpm_entry_point().
> -        */
> -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> +       exynos_mcpm_setup_entry_point();
>
> -       iounmap(ns_sram_base_addr);
> +       register_syscore_ops(&exynos_mcpm_syscore_ops);
>
>         return ret;
>  }

This patch alone is not enough to bring the 8 cores back up post
resume on exynos5420, We need to enable the boot cluster snoops as
well.

Nicolas, if I add code to enable the boot cluster CCI in
mach-exynos/sleep.S, all 8 cores come up. However, if I use
mcpm_loopback as part of the newly added resume function to do the
same, I get a hang on resume.
Is there anything else that I need to take care of while doing this ?

Regards,
Abhilash
> --
> 1.8.3.2
>

Re: [PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Lorenzo Pieralisi]

On Thu, Jun 26, 2014 at 01:25:30PM +0100, Abhilash Kesavan wrote:
> Hi,
> 
> On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
> > Setup the mcpm entry address again on system resume as the
> > iRAM contents are lost across an s2r cycle.
> >
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > ---
> > This has been tested after applying the Exynos5420 S2R support series
> > along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
> > Pit.
> >
> >  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
> >  1 file changed, 21 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > index 8c610e2..0bf734d 100644
> > --- a/arch/arm/mach-exynos/mcpm-exynos.c
> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > @@ -15,6 +15,7 @@
> >  #include <linux/delay.h>
> >  #include <linux/io.h>
> >  #include <linux/of_address.h>
> > +#include <linux/syscore_ops.h>
> >
> >  #include <asm/cputype.h>
> >  #include <asm/cp15.h>
> > @@ -26,6 +27,7 @@
> >  #define EXYNOS5420_CPUS_PER_CLUSTER    4
> >  #define EXYNOS5420_NR_CLUSTERS         2
> >
> > +static void __iomem *ns_sram_base_addr;
> >  /*
> >   * The common v7_exit_coherency_flush API could not be used because of the
> >   * Erratum 799270 workaround. This macro is the same as the common one (in
> > @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
> >         {},
> >  };
> >
> > +static void exynos_mcpm_setup_entry_point(void)
> > +{
> > +       /*
> > +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > +        * as part of secondary_cpu_start().  Let's redirect it to the
> > +        * mcpm_entry_point(). This is done during both secondary boot-up as
> > +        * well as system resume.
> > +        */
> > +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +}
> > +
> > +static struct syscore_ops exynos_mcpm_syscore_ops = {
> > +       .resume = exynos_mcpm_setup_entry_point,
> > +};
> > +
> >  static int __init exynos_mcpm_init(void)
> >  {
> >         struct device_node *node;
> > -       void __iomem *ns_sram_base_addr;
> >         int ret;
> >
> >         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
> > @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
> >
> >         pr_info("Exynos MCPM support installed\n");
> >
> > -       /*
> > -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > -        * as part of secondary_cpu_start().  Let's redirect it to the
> > -        * mcpm_entry_point().
> > -        */
> > -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +       exynos_mcpm_setup_entry_point();
> >
> > -       iounmap(ns_sram_base_addr);
> > +       register_syscore_ops(&exynos_mcpm_syscore_ops);
> >
> >         return ret;
> >  }
> 
> This patch alone is not enough to bring the 8 cores back up post
> resume on exynos5420, We need to enable the boot cluster snoops as
> well.
> 
> Nicolas, if I add code to enable the boot cluster CCI in
> mach-exynos/sleep.S, all 8 cores come up. However, if I use
> mcpm_loopback as part of the newly added resume function to do the
> same, I get a hang on resume.
> Is there anything else that I need to take care of while doing this ?

Why do not you use MCPM to suspend the CPU for S2R ? Put it differently,
plug the MCPM suspend call in the suspend ops, it should work, right ?

I think you should chuck the MCPM entry point in the register used for
jumping back to the kernel when resuming to the kernel, then by restoring
RAM code through syscore_ops you ensure you can bring secondaries
back up again (done by suspend core), if I understand how firmware
behaves.

I do not see why you need the MCPM loopback interface for that.

Lorenzo

[PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Lorenzo Pieralisi]

On Thu, Jun 26, 2014 at 01:25:30PM +0100, Abhilash Kesavan wrote:
> Hi,
> 
> On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
> > Setup the mcpm entry address again on system resume as the
> > iRAM contents are lost across an s2r cycle.
> >
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > ---
> > This has been tested after applying the Exynos5420 S2R support series
> > along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
> > Pit.
> >
> >  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
> >  1 file changed, 21 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > index 8c610e2..0bf734d 100644
> > --- a/arch/arm/mach-exynos/mcpm-exynos.c
> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > @@ -15,6 +15,7 @@
> >  #include <linux/delay.h>
> >  #include <linux/io.h>
> >  #include <linux/of_address.h>
> > +#include <linux/syscore_ops.h>
> >
> >  #include <asm/cputype.h>
> >  #include <asm/cp15.h>
> > @@ -26,6 +27,7 @@
> >  #define EXYNOS5420_CPUS_PER_CLUSTER    4
> >  #define EXYNOS5420_NR_CLUSTERS         2
> >
> > +static void __iomem *ns_sram_base_addr;
> >  /*
> >   * The common v7_exit_coherency_flush API could not be used because of the
> >   * Erratum 799270 workaround. This macro is the same as the common one (in
> > @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
> >         {},
> >  };
> >
> > +static void exynos_mcpm_setup_entry_point(void)
> > +{
> > +       /*
> > +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > +        * as part of secondary_cpu_start().  Let's redirect it to the
> > +        * mcpm_entry_point(). This is done during both secondary boot-up as
> > +        * well as system resume.
> > +        */
> > +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +}
> > +
> > +static struct syscore_ops exynos_mcpm_syscore_ops = {
> > +       .resume = exynos_mcpm_setup_entry_point,
> > +};
> > +
> >  static int __init exynos_mcpm_init(void)
> >  {
> >         struct device_node *node;
> > -       void __iomem *ns_sram_base_addr;
> >         int ret;
> >
> >         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
> > @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
> >
> >         pr_info("Exynos MCPM support installed\n");
> >
> > -       /*
> > -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > -        * as part of secondary_cpu_start().  Let's redirect it to the
> > -        * mcpm_entry_point().
> > -        */
> > -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +       exynos_mcpm_setup_entry_point();
> >
> > -       iounmap(ns_sram_base_addr);
> > +       register_syscore_ops(&exynos_mcpm_syscore_ops);
> >
> >         return ret;
> >  }
> 
> This patch alone is not enough to bring the 8 cores back up post
> resume on exynos5420, We need to enable the boot cluster snoops as
> well.
> 
> Nicolas, if I add code to enable the boot cluster CCI in
> mach-exynos/sleep.S, all 8 cores come up. However, if I use
> mcpm_loopback as part of the newly added resume function to do the
> same, I get a hang on resume.
> Is there anything else that I need to take care of while doing this ?

Why do not you use MCPM to suspend the CPU for S2R ? Put it differently,
plug the MCPM suspend call in the suspend ops, it should work, right ?

I think you should chuck the MCPM entry point in the register used for
jumping back to the kernel when resuming to the kernel, then by restoring
RAM code through syscore_ops you ensure you can bring secondaries
back up again (done by suspend core), if I understand how firmware
behaves.

I do not see why you need the MCPM loopback interface for that.

Lorenzo

Re: [PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Lorenzo Pieralisi]

On Thu, Jun 26, 2014 at 01:25:30PM +0100, Abhilash Kesavan wrote:
> Hi,
> 
> On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
> > Setup the mcpm entry address again on system resume as the
> > iRAM contents are lost across an s2r cycle.
> >
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > ---
> > This has been tested after applying the Exynos5420 S2R support series
> > along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
> > Pit.
> >
> >  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
> >  1 file changed, 21 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > index 8c610e2..0bf734d 100644
> > --- a/arch/arm/mach-exynos/mcpm-exynos.c
> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > @@ -15,6 +15,7 @@
> >  #include <linux/delay.h>
> >  #include <linux/io.h>
> >  #include <linux/of_address.h>
> > +#include <linux/syscore_ops.h>
> >
> >  #include <asm/cputype.h>
> >  #include <asm/cp15.h>
> > @@ -26,6 +27,7 @@
> >  #define EXYNOS5420_CPUS_PER_CLUSTER    4
> >  #define EXYNOS5420_NR_CLUSTERS         2
> >
> > +static void __iomem *ns_sram_base_addr;
> >  /*
> >   * The common v7_exit_coherency_flush API could not be used because of the
> >   * Erratum 799270 workaround. This macro is the same as the common one (in
> > @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
> >         {},
> >  };
> >
> > +static void exynos_mcpm_setup_entry_point(void)
> > +{
> > +       /*
> > +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > +        * as part of secondary_cpu_start().  Let's redirect it to the
> > +        * mcpm_entry_point(). This is done during both secondary boot-up as
> > +        * well as system resume.
> > +        */
> > +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +}
> > +
> > +static struct syscore_ops exynos_mcpm_syscore_ops = {
> > +       .resume = exynos_mcpm_setup_entry_point,
> > +};
> > +
> >  static int __init exynos_mcpm_init(void)
> >  {
> >         struct device_node *node;
> > -       void __iomem *ns_sram_base_addr;
> >         int ret;
> >
> >         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
> > @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
> >
> >         pr_info("Exynos MCPM support installed\n");
> >
> > -       /*
> > -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > -        * as part of secondary_cpu_start().  Let's redirect it to the
> > -        * mcpm_entry_point().
> > -        */
> > -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +       exynos_mcpm_setup_entry_point();
> >
> > -       iounmap(ns_sram_base_addr);
> > +       register_syscore_ops(&exynos_mcpm_syscore_ops);
> >
> >         return ret;
> >  }
> 
> This patch alone is not enough to bring the 8 cores back up post
> resume on exynos5420, We need to enable the boot cluster snoops as
> well.
> 
> Nicolas, if I add code to enable the boot cluster CCI in
> mach-exynos/sleep.S, all 8 cores come up. However, if I use
> mcpm_loopback as part of the newly added resume function to do the
> same, I get a hang on resume.

BTW mcpm_loopback is __init, and that's correct, so on resume if you
call it kernel goes boom.

Lorenzo

[PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Lorenzo Pieralisi]

On Thu, Jun 26, 2014 at 01:25:30PM +0100, Abhilash Kesavan wrote:
> Hi,
> 
> On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
> > Setup the mcpm entry address again on system resume as the
> > iRAM contents are lost across an s2r cycle.
> >
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > ---
> > This has been tested after applying the Exynos5420 S2R support series
> > along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
> > Pit.
> >
> >  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
> >  1 file changed, 21 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > index 8c610e2..0bf734d 100644
> > --- a/arch/arm/mach-exynos/mcpm-exynos.c
> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > @@ -15,6 +15,7 @@
> >  #include <linux/delay.h>
> >  #include <linux/io.h>
> >  #include <linux/of_address.h>
> > +#include <linux/syscore_ops.h>
> >
> >  #include <asm/cputype.h>
> >  #include <asm/cp15.h>
> > @@ -26,6 +27,7 @@
> >  #define EXYNOS5420_CPUS_PER_CLUSTER    4
> >  #define EXYNOS5420_NR_CLUSTERS         2
> >
> > +static void __iomem *ns_sram_base_addr;
> >  /*
> >   * The common v7_exit_coherency_flush API could not be used because of the
> >   * Erratum 799270 workaround. This macro is the same as the common one (in
> > @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
> >         {},
> >  };
> >
> > +static void exynos_mcpm_setup_entry_point(void)
> > +{
> > +       /*
> > +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > +        * as part of secondary_cpu_start().  Let's redirect it to the
> > +        * mcpm_entry_point(). This is done during both secondary boot-up as
> > +        * well as system resume.
> > +        */
> > +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +}
> > +
> > +static struct syscore_ops exynos_mcpm_syscore_ops = {
> > +       .resume = exynos_mcpm_setup_entry_point,
> > +};
> > +
> >  static int __init exynos_mcpm_init(void)
> >  {
> >         struct device_node *node;
> > -       void __iomem *ns_sram_base_addr;
> >         int ret;
> >
> >         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
> > @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
> >
> >         pr_info("Exynos MCPM support installed\n");
> >
> > -       /*
> > -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > -        * as part of secondary_cpu_start().  Let's redirect it to the
> > -        * mcpm_entry_point().
> > -        */
> > -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +       exynos_mcpm_setup_entry_point();
> >
> > -       iounmap(ns_sram_base_addr);
> > +       register_syscore_ops(&exynos_mcpm_syscore_ops);
> >
> >         return ret;
> >  }
> 
> This patch alone is not enough to bring the 8 cores back up post
> resume on exynos5420, We need to enable the boot cluster snoops as
> well.
> 
> Nicolas, if I add code to enable the boot cluster CCI in
> mach-exynos/sleep.S, all 8 cores come up. However, if I use
> mcpm_loopback as part of the newly added resume function to do the
> same, I get a hang on resume.

BTW mcpm_loopback is __init, and that's correct, so on resume if you
call it kernel goes boom.

Lorenzo

Re: [PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Nicolas Pitre]

On Thu, 26 Jun 2014, Abhilash Kesavan wrote:

> Hi,
> 
> On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
> > Setup the mcpm entry address again on system resume as the
> > iRAM contents are lost across an s2r cycle.
> >
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > ---
> > This has been tested after applying the Exynos5420 S2R support series
> > along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
> > Pit.
> >
> >  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
> >  1 file changed, 21 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > index 8c610e2..0bf734d 100644
> > --- a/arch/arm/mach-exynos/mcpm-exynos.c
> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > @@ -15,6 +15,7 @@
> >  #include <linux/delay.h>
> >  #include <linux/io.h>
> >  #include <linux/of_address.h>
> > +#include <linux/syscore_ops.h>
> >
> >  #include <asm/cputype.h>
> >  #include <asm/cp15.h>
> > @@ -26,6 +27,7 @@
> >  #define EXYNOS5420_CPUS_PER_CLUSTER    4
> >  #define EXYNOS5420_NR_CLUSTERS         2
> >
> > +static void __iomem *ns_sram_base_addr;
> >  /*
> >   * The common v7_exit_coherency_flush API could not be used because of the
> >   * Erratum 799270 workaround. This macro is the same as the common one (in
> > @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
> >         {},
> >  };
> >
> > +static void exynos_mcpm_setup_entry_point(void)
> > +{
> > +       /*
> > +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > +        * as part of secondary_cpu_start().  Let's redirect it to the
> > +        * mcpm_entry_point(). This is done during both secondary boot-up as
> > +        * well as system resume.
> > +        */
> > +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +}
> > +
> > +static struct syscore_ops exynos_mcpm_syscore_ops = {
> > +       .resume = exynos_mcpm_setup_entry_point,
> > +};
> > +
> >  static int __init exynos_mcpm_init(void)
> >  {
> >         struct device_node *node;
> > -       void __iomem *ns_sram_base_addr;
> >         int ret;
> >
> >         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
> > @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
> >
> >         pr_info("Exynos MCPM support installed\n");
> >
> > -       /*
> > -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > -        * as part of secondary_cpu_start().  Let's redirect it to the
> > -        * mcpm_entry_point().
> > -        */
> > -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +       exynos_mcpm_setup_entry_point();
> >
> > -       iounmap(ns_sram_base_addr);
> > +       register_syscore_ops(&exynos_mcpm_syscore_ops);
> >
> >         return ret;
> >  }
> 
> This patch alone is not enough to bring the 8 cores back up post
> resume on exynos5420, We need to enable the boot cluster snoops as
> well.
> 
> Nicolas, if I add code to enable the boot cluster CCI in
> mach-exynos/sleep.S, all 8 cores come up. However, if I use
> mcpm_loopback as part of the newly added resume function to do the
> same, I get a hang on resume.
> Is there anything else that I need to take care of while doing this ?

No no no.

Forget about the MCPM loopback -- that's for boot time setup of the 
booting cluster nothing else.

The snoops are enabled in a ***race free way*** by the low-level MCPM 
code.  But of course you must call it upon resume.  Nothing else than 
MCPM should ever play with the snoops enable/disable.  Certainly not 
mach-exynos/sleep.S.

You must tell your firmware to resume execution after suspend at 
mcpm_entry_point.  Before suspending, you tell MCPM where it should 
branch to after it is done with its resuming business by calling 
mcpm_set_entry_vector(), passing the address for exynos_cpu_resume for 
example.

And of course the CPU shutdown during idle has to go through MCPM too.

When resumed, you also have to call mcpm_cpu_powered_up().

See how it is done in cpuidle-big_little.c for example.


Nicolas

[PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Nicolas Pitre]

On Thu, 26 Jun 2014, Abhilash Kesavan wrote:

> Hi,
> 
> On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
> > Setup the mcpm entry address again on system resume as the
> > iRAM contents are lost across an s2r cycle.
> >
> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > ---
> > This has been tested after applying the Exynos5420 S2R support series
> > along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
> > Pit.
> >
> >  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
> >  1 file changed, 21 insertions(+), 10 deletions(-)
> >
> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> > index 8c610e2..0bf734d 100644
> > --- a/arch/arm/mach-exynos/mcpm-exynos.c
> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> > @@ -15,6 +15,7 @@
> >  #include <linux/delay.h>
> >  #include <linux/io.h>
> >  #include <linux/of_address.h>
> > +#include <linux/syscore_ops.h>
> >
> >  #include <asm/cputype.h>
> >  #include <asm/cp15.h>
> > @@ -26,6 +27,7 @@
> >  #define EXYNOS5420_CPUS_PER_CLUSTER    4
> >  #define EXYNOS5420_NR_CLUSTERS         2
> >
> > +static void __iomem *ns_sram_base_addr;
> >  /*
> >   * The common v7_exit_coherency_flush API could not be used because of the
> >   * Erratum 799270 workaround. This macro is the same as the common one (in
> > @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
> >         {},
> >  };
> >
> > +static void exynos_mcpm_setup_entry_point(void)
> > +{
> > +       /*
> > +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > +        * as part of secondary_cpu_start().  Let's redirect it to the
> > +        * mcpm_entry_point(). This is done during both secondary boot-up as
> > +        * well as system resume.
> > +        */
> > +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +}
> > +
> > +static struct syscore_ops exynos_mcpm_syscore_ops = {
> > +       .resume = exynos_mcpm_setup_entry_point,
> > +};
> > +
> >  static int __init exynos_mcpm_init(void)
> >  {
> >         struct device_node *node;
> > -       void __iomem *ns_sram_base_addr;
> >         int ret;
> >
> >         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
> > @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
> >
> >         pr_info("Exynos MCPM support installed\n");
> >
> > -       /*
> > -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> > -        * as part of secondary_cpu_start().  Let's redirect it to the
> > -        * mcpm_entry_point().
> > -        */
> > -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> > -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> > -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> > +       exynos_mcpm_setup_entry_point();
> >
> > -       iounmap(ns_sram_base_addr);
> > +       register_syscore_ops(&exynos_mcpm_syscore_ops);
> >
> >         return ret;
> >  }
> 
> This patch alone is not enough to bring the 8 cores back up post
> resume on exynos5420, We need to enable the boot cluster snoops as
> well.
> 
> Nicolas, if I add code to enable the boot cluster CCI in
> mach-exynos/sleep.S, all 8 cores come up. However, if I use
> mcpm_loopback as part of the newly added resume function to do the
> same, I get a hang on resume.
> Is there anything else that I need to take care of while doing this ?

No no no.

Forget about the MCPM loopback -- that's for boot time setup of the 
booting cluster nothing else.

The snoops are enabled in a ***race free way*** by the low-level MCPM 
code.  But of course you must call it upon resume.  Nothing else than 
MCPM should ever play with the snoops enable/disable.  Certainly not 
mach-exynos/sleep.S.

You must tell your firmware to resume execution after suspend at 
mcpm_entry_point.  Before suspending, you tell MCPM where it should 
branch to after it is done with its resuming business by calling 
mcpm_set_entry_vector(), passing the address for exynos_cpu_resume for 
example.

And of course the CPU shutdown during idle has to go through MCPM too.

When resumed, you also have to call mcpm_cpu_powered_up().

See how it is done in cpuidle-big_little.c for example.


Nicolas

Re: [PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Abhilash Kesavan]

Hi,

On Fri, Jun 27, 2014 at 1:36 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 26 Jun 2014, Abhilash Kesavan wrote:
>
>> Hi,
>>
>> On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
>> > Setup the mcpm entry address again on system resume as the
>> > iRAM contents are lost across an s2r cycle.
>> >
>> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> > ---
>> > This has been tested after applying the Exynos5420 S2R support series
>> > along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
>> > Pit.
>> >
>> >  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
>> >  1 file changed, 21 insertions(+), 10 deletions(-)
>> >
>> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> > index 8c610e2..0bf734d 100644
>> > --- a/arch/arm/mach-exynos/mcpm-exynos.c
>> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> > @@ -15,6 +15,7 @@
>> >  #include <linux/delay.h>
>> >  #include <linux/io.h>
>> >  #include <linux/of_address.h>
>> > +#include <linux/syscore_ops.h>
>> >
>> >  #include <asm/cputype.h>
>> >  #include <asm/cp15.h>
>> > @@ -26,6 +27,7 @@
>> >  #define EXYNOS5420_CPUS_PER_CLUSTER    4
>> >  #define EXYNOS5420_NR_CLUSTERS         2
>> >
>> > +static void __iomem *ns_sram_base_addr;
>> >  /*
>> >   * The common v7_exit_coherency_flush API could not be used because of the
>> >   * Erratum 799270 workaround. This macro is the same as the common one (in
>> > @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>> >         {},
>> >  };
>> >
>> > +static void exynos_mcpm_setup_entry_point(void)
>> > +{
>> > +       /*
>> > +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> > +        * as part of secondary_cpu_start().  Let's redirect it to the
>> > +        * mcpm_entry_point(). This is done during both secondary boot-up as
>> > +        * well as system resume.
>> > +        */
>> > +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> > +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> > +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> > +}
>> > +
>> > +static struct syscore_ops exynos_mcpm_syscore_ops = {
>> > +       .resume = exynos_mcpm_setup_entry_point,
>> > +};
>> > +
>> >  static int __init exynos_mcpm_init(void)
>> >  {
>> >         struct device_node *node;
>> > -       void __iomem *ns_sram_base_addr;
>> >         int ret;
>> >
>> >         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
>> > @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
>> >
>> >         pr_info("Exynos MCPM support installed\n");
>> >
>> > -       /*
>> > -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> > -        * as part of secondary_cpu_start().  Let's redirect it to the
>> > -        * mcpm_entry_point().
>> > -        */
>> > -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> > -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> > -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> > +       exynos_mcpm_setup_entry_point();
>> >
>> > -       iounmap(ns_sram_base_addr);
>> > +       register_syscore_ops(&exynos_mcpm_syscore_ops);
>> >
>> >         return ret;
>> >  }
>>
>> This patch alone is not enough to bring the 8 cores back up post
>> resume on exynos5420, We need to enable the boot cluster snoops as
>> well.
>>
>> Nicolas, if I add code to enable the boot cluster CCI in
>> mach-exynos/sleep.S, all 8 cores come up. However, if I use
>> mcpm_loopback as part of the newly added resume function to do the
>> same, I get a hang on resume.
>> Is there anything else that I need to take care of while doing this ?
>
> No no no.
>
> Forget about the MCPM loopback -- that's for boot time setup of the
> booting cluster nothing else.
Ok, I thought this is to be used at resume too.
>
> The snoops are enabled in a ***race free way*** by the low-level MCPM
> code.  But of course you must call it upon resume.  Nothing else than
> MCPM should ever play with the snoops enable/disable.  Certainly not
> mach-exynos/sleep.S.
The sleep.S change was something I tried to isolate where I was going
wrong. Of course, in this case my basic premise of using mcpm_loopback
to enable snoops itself was flawed.
>
> You must tell your firmware to resume execution after suspend at
> mcpm_entry_point.  Before suspending, you tell MCPM where it should
> branch to after it is done with its resuming business by calling
> mcpm_set_entry_vector(), passing the address for exynos_cpu_resume for
> example.
Thanks for the explantion, will re-work the patch.
>
> And of course the CPU shutdown during idle has to go through MCPM too.
>
> When resumed, you also have to call mcpm_cpu_powered_up().
>
> See how it is done in cpuidle-big_little.c for example
Will check the cpuidle big.LITTLE driver also.

Regards,
Abhilash

[PATCH] ARM: EXYNOS: Restore the entry address setup code post-resume

[Abhilash Kesavan]

Hi,

On Fri, Jun 27, 2014 at 1:36 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 26 Jun 2014, Abhilash Kesavan wrote:
>
>> Hi,
>>
>> On Thu, Jun 26, 2014 at 4:28 PM, Abhilash Kesavan <a.kesavan@samsung.com> wrote:
>> > Setup the mcpm entry address again on system resume as the
>> > iRAM contents are lost across an s2r cycle.
>> >
>> > Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> > ---
>> > This has been tested after applying the Exynos5420 S2R support series
>> > along with Nicolas Pitre's boot cluster CCI enablement patches on Peach
>> > Pit.
>> >
>> >  arch/arm/mach-exynos/mcpm-exynos.c | 31 +++++++++++++++++++++----------
>> >  1 file changed, 21 insertions(+), 10 deletions(-)
>> >
>> > diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> > index 8c610e2..0bf734d 100644
>> > --- a/arch/arm/mach-exynos/mcpm-exynos.c
>> > +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> > @@ -15,6 +15,7 @@
>> >  #include <linux/delay.h>
>> >  #include <linux/io.h>
>> >  #include <linux/of_address.h>
>> > +#include <linux/syscore_ops.h>
>> >
>> >  #include <asm/cputype.h>
>> >  #include <asm/cp15.h>
>> > @@ -26,6 +27,7 @@
>> >  #define EXYNOS5420_CPUS_PER_CLUSTER    4
>> >  #define EXYNOS5420_NR_CLUSTERS         2
>> >
>> > +static void __iomem *ns_sram_base_addr;
>> >  /*
>> >   * The common v7_exit_coherency_flush API could not be used because of the
>> >   * Erratum 799270 workaround. This macro is the same as the common one (in
>> > @@ -308,10 +310,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>> >         {},
>> >  };
>> >
>> > +static void exynos_mcpm_setup_entry_point(void)
>> > +{
>> > +       /*
>> > +        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> > +        * as part of secondary_cpu_start().  Let's redirect it to the
>> > +        * mcpm_entry_point(). This is done during both secondary boot-up as
>> > +        * well as system resume.
>> > +        */
>> > +       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> > +       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> > +       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> > +}
>> > +
>> > +static struct syscore_ops exynos_mcpm_syscore_ops = {
>> > +       .resume = exynos_mcpm_setup_entry_point,
>> > +};
>> > +
>> >  static int __init exynos_mcpm_init(void)
>> >  {
>> >         struct device_node *node;
>> > -       void __iomem *ns_sram_base_addr;
>> >         int ret;
>> >
>> >         node = of_find_matching_node(NULL, exynos_dt_mcpm_match);
>> > @@ -357,16 +375,9 @@ static int __init exynos_mcpm_init(void)
>> >
>> >         pr_info("Exynos MCPM support installed\n");
>> >
>> > -       /*
>> > -        * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> > -        * as part of secondary_cpu_start().  Let's redirect it to the
>> > -        * mcpm_entry_point().
>> > -        */
>> > -       __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> > -       __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> > -       __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> > +       exynos_mcpm_setup_entry_point();
>> >
>> > -       iounmap(ns_sram_base_addr);
>> > +       register_syscore_ops(&exynos_mcpm_syscore_ops);
>> >
>> >         return ret;
>> >  }
>>
>> This patch alone is not enough to bring the 8 cores back up post
>> resume on exynos5420, We need to enable the boot cluster snoops as
>> well.
>>
>> Nicolas, if I add code to enable the boot cluster CCI in
>> mach-exynos/sleep.S, all 8 cores come up. However, if I use
>> mcpm_loopback as part of the newly added resume function to do the
>> same, I get a hang on resume.
>> Is there anything else that I need to take care of while doing this ?
>
> No no no.
>
> Forget about the MCPM loopback -- that's for boot time setup of the
> booting cluster nothing else.
Ok, I thought this is to be used at resume too.
>
> The snoops are enabled in a ***race free way*** by the low-level MCPM
> code.  But of course you must call it upon resume.  Nothing else than
> MCPM should ever play with the snoops enable/disable.  Certainly not
> mach-exynos/sleep.S.
The sleep.S change was something I tried to isolate where I was going
wrong. Of course, in this case my basic premise of using mcpm_loopback
to enable snoops itself was flawed.
>
> You must tell your firmware to resume execution after suspend at
> mcpm_entry_point.  Before suspending, you tell MCPM where it should
> branch to after it is done with its resuming business by calling
> mcpm_set_entry_vector(), passing the address for exynos_cpu_resume for
> example.
Thanks for the explantion, will re-work the patch.
>
> And of course the CPU shutdown during idle has to go through MCPM too.
>
> When resumed, you also have to call mcpm_cpu_powered_up().
>
> See how it is done in cpuidle-big_little.c for example
Will check the cpuidle big.LITTLE driver also.

Regards,
Abhilash

[PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Hi Lorenzo and Nicolas,

I am now using the suspend/powered_up call-backs that were added as part
of Chander's CPUIdle series for Exynos5420. exynos_mcpm_cpu_suspend()
is based on the code in bL cpuidle driver except for a small addition.
This has been tested both on an SMDK5420 and Peach Pit Chromebook on
3.16-rc3.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) MCPM boot CPU CCI enablement patches
www.spinics.net/lists/linux-samsung-soc/msg32923.html

5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

6) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

7) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html

 arch/arm/mach-exynos/mcpm-exynos.c |   40 ++++++++++++++++++++++++++----------
 arch/arm/mach-exynos/pm.c          |   38 +++++++++++++++++++++++++++++++---
 arch/arm/mach-exynos/regs-pmu.h    |    1 +
 3 files changed, 65 insertions(+), 14 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 037a05e..dd298f3 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -150,7 +153,13 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. This is being
+		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
+		 */
+		if ((cpunr != 0) ||
+		(__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -319,10 +328,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +414,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index bf8564a..a4f4292 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -191,7 +192,6 @@ int exynos_cluster_power_state(int cluster)
 			pmu_base_addr + S5P_INFORM1))
 
 #define S5P_CHECK_AFTR  0xFCBA0D10
-#define S5P_CHECK_SLEEP 0x00000BAD
 
 /* Ext-GIC nIRQ/nFIQ is the only wakeup source in AFTR */
 static void exynos_set_wakeupmask(long mask)
@@ -318,7 +318,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420())
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -490,6 +493,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(0);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +542,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420())
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420())
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index 3cf0454..e8c75db 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -152,6 +152,7 @@
 #define S5P_PAD_RET_EBIB_OPTION			0x31A8
 
 #define S5P_CORE_LOCAL_PWR_EN			0x3
+#define S5P_CHECK_SLEEP				0x00000BAD
 
 /* Only for EXYNOS4210 */
 #define S5P_CMU_CLKSTOP_LCD1_LOWPWR	0x1154
-- 
1.7.9.5

[PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Hi Lorenzo and Nicolas,

I am now using the suspend/powered_up call-backs that were added as part
of Chander's CPUIdle series for Exynos5420. exynos_mcpm_cpu_suspend()
is based on the code in bL cpuidle driver except for a small addition.
This has been tested both on an SMDK5420 and Peach Pit Chromebook on
3.16-rc3.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) MCPM boot CPU CCI enablement patches
www.spinics.net/lists/linux-samsung-soc/msg32923.html

5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

6) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

7) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html

 arch/arm/mach-exynos/mcpm-exynos.c |   40 ++++++++++++++++++++++++++----------
 arch/arm/mach-exynos/pm.c          |   38 +++++++++++++++++++++++++++++++---
 arch/arm/mach-exynos/regs-pmu.h    |    1 +
 3 files changed, 65 insertions(+), 14 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 037a05e..dd298f3 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -150,7 +153,13 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. This is being
+		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
+		 */
+		if ((cpunr != 0) ||
+		(__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -319,10 +328,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +414,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index bf8564a..a4f4292 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -191,7 +192,6 @@ int exynos_cluster_power_state(int cluster)
 			pmu_base_addr + S5P_INFORM1))
 
 #define S5P_CHECK_AFTR  0xFCBA0D10
-#define S5P_CHECK_SLEEP 0x00000BAD
 
 /* Ext-GIC nIRQ/nFIQ is the only wakeup source in AFTR */
 static void exynos_set_wakeupmask(long mask)
@@ -318,7 +318,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420())
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -490,6 +493,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(0);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +542,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420())
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420())
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index 3cf0454..e8c75db 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -152,6 +152,7 @@
 #define S5P_PAD_RET_EBIB_OPTION			0x31A8
 
 #define S5P_CORE_LOCAL_PWR_EN			0x3
+#define S5P_CHECK_SLEEP				0x00000BAD
 
 /* Only for EXYNOS4210 */
 #define S5P_CMU_CLKSTOP_LCD1_LOWPWR	0x1154
-- 
1.7.9.5

Re: [PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Nicolas Pitre]

On Mon, 30 Jun 2014, Abhilash Kesavan wrote:

> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> Also, restore the entry address setup code post-resume.
> 
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> ---
[...]

Could you tell me more about this?

> @@ -150,7 +153,13 @@ static void exynos_power_down(void)
>  	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>  	cpu_use_count[cpu][cluster]--;
>  	if (cpu_use_count[cpu][cluster] == 0) {
> -		exynos_cpu_power_down(cpunr);
> +		/*
> +		 * Bypass power down for CPU0 during suspend. This is being
> +		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> +		 */
> +		if ((cpunr != 0) ||
> +		(__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
> +			exynos_cpu_power_down(cpunr);

What happens if CPU0 is the first in the cluster to be idled?  Will it 
remain powered up until all the others have gone idle too?


Nicolas

[PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Nicolas Pitre]

On Mon, 30 Jun 2014, Abhilash Kesavan wrote:

> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> Also, restore the entry address setup code post-resume.
> 
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> ---
[...]

Could you tell me more about this?

> @@ -150,7 +153,13 @@ static void exynos_power_down(void)
>  	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>  	cpu_use_count[cpu][cluster]--;
>  	if (cpu_use_count[cpu][cluster] == 0) {
> -		exynos_cpu_power_down(cpunr);
> +		/*
> +		 * Bypass power down for CPU0 during suspend. This is being
> +		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> +		 */
> +		if ((cpunr != 0) ||
> +		(__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
> +			exynos_cpu_power_down(cpunr);

What happens if CPU0 is the first in the cluster to be idled?  Will it 
remain powered up until all the others have gone idle too?


Nicolas

Re: [PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Hi Nicolas,

On Tue, Jul 1, 2014 at 9:49 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Mon, 30 Jun 2014, Abhilash Kesavan wrote:
>
>> Use the MCPM layer to handle core suspend/resume on Exynos5420.
>> Also, restore the entry address setup code post-resume.
>>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> ---
> [...]
>
> Could you tell me more about this?
>
>> @@ -150,7 +153,13 @@ static void exynos_power_down(void)
>>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>>       cpu_use_count[cpu][cluster]--;
>>       if (cpu_use_count[cpu][cluster] == 0) {
>> -             exynos_cpu_power_down(cpunr);
>> +             /*
>> +              * Bypass power down for CPU0 during suspend. This is being
>> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> +              */
>> +             if ((cpunr != 0) ||
>> +             (__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
>> +                     exynos_cpu_power_down(cpunr);
>
> What happens if CPU0 is the first in the cluster to be idled?  Will it
> remain powered up until all the others have gone idle too?
This check is only for handling the S2R CPU0 case. In case of
idle/switching the S5P_CHECK_SLEEP flag would not be set and hence
there would be no change in behavior for them.
During suspend, we enable the ARM_USE_STANDBY_WFI0 bit which tells the
PMU when the CPU0 is ready to be suspended. This in conjunction with
the sleep state core configuration (setting SYS_PWR_REG low) causes
the CPU0 to go down. We should not write to the CPU0 power
configuration register (exynos_cpu_power_down) along with this during
suspend.

Regards,
Abhilash
>
>
> Nicolas

[PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Hi Nicolas,

On Tue, Jul 1, 2014 at 9:49 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Mon, 30 Jun 2014, Abhilash Kesavan wrote:
>
>> Use the MCPM layer to handle core suspend/resume on Exynos5420.
>> Also, restore the entry address setup code post-resume.
>>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> ---
> [...]
>
> Could you tell me more about this?
>
>> @@ -150,7 +153,13 @@ static void exynos_power_down(void)
>>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>>       cpu_use_count[cpu][cluster]--;
>>       if (cpu_use_count[cpu][cluster] == 0) {
>> -             exynos_cpu_power_down(cpunr);
>> +             /*
>> +              * Bypass power down for CPU0 during suspend. This is being
>> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> +              */
>> +             if ((cpunr != 0) ||
>> +             (__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
>> +                     exynos_cpu_power_down(cpunr);
>
> What happens if CPU0 is the first in the cluster to be idled?  Will it
> remain powered up until all the others have gone idle too?
This check is only for handling the S2R CPU0 case. In case of
idle/switching the S5P_CHECK_SLEEP flag would not be set and hence
there would be no change in behavior for them.
During suspend, we enable the ARM_USE_STANDBY_WFI0 bit which tells the
PMU when the CPU0 is ready to be suspended. This in conjunction with
the sleep state core configuration (setting SYS_PWR_REG low) causes
the CPU0 to go down. We should not write to the CPU0 power
configuration register (exynos_cpu_power_down) along with this during
suspend.

Regards,
Abhilash
>
>
> Nicolas

Re: [PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Lorenzo Pieralisi]

On Tue, Jul 01, 2014 at 02:14:49PM +0100, Abhilash Kesavan wrote:
> Hi Nicolas,
> 
> On Tue, Jul 1, 2014 at 9:49 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Mon, 30 Jun 2014, Abhilash Kesavan wrote:
> >
> >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> >> Also, restore the entry address setup code post-resume.
> >>
> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> >> ---
> > [...]
> >
> > Could you tell me more about this?
> >
> >> @@ -150,7 +153,13 @@ static void exynos_power_down(void)
> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> >>       cpu_use_count[cpu][cluster]--;
> >>       if (cpu_use_count[cpu][cluster] == 0) {
> >> -             exynos_cpu_power_down(cpunr);
> >> +             /*
> >> +              * Bypass power down for CPU0 during suspend. This is being
> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> >> +              */
> >> +             if ((cpunr != 0) ||
> >> +             (__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
> >> +                     exynos_cpu_power_down(cpunr);
> >
> > What happens if CPU0 is the first in the cluster to be idled?  Will it
> > remain powered up until all the others have gone idle too?
> This check is only for handling the S2R CPU0 case. In case of
> idle/switching the S5P_CHECK_SLEEP flag would not be set and hence
> there would be no change in behavior for them.
> During suspend, we enable the ARM_USE_STANDBY_WFI0 bit which tells the
> PMU when the CPU0 is ready to be suspended. This in conjunction with
> the sleep state core configuration (setting SYS_PWR_REG low) causes
> the CPU0 to go down. We should not write to the CPU0 power
> configuration register (exynos_cpu_power_down) along with this during
> suspend.

I think this should be part of a refactoring that includes the exynos MCPM
suspend call parameters. In particular, at the moment there is no code in
the back-end to detect if the last man should request core gating or cluster
gating (ie last man _always_ request cluster gating, that might not be what
we want), there is the residency value that can be also be used to imply a
S2R request (eg residency = ~0 ?).

The command sent must be implied by the state that is entered, not by
peeking at registers that should contain magic values, and that's true
not only for exynos but for all ARM platforms out there.

What I mean is: we can pass the requested state as a suspend parameter
and the power_down state machine will send the required command
accordingly.

It can be done using the residency value or by passing the power state "index"
as suspend parameter, the power down MCPM code will do a look-up and send
the respective command.

Thoughts appreciated.

Lorenzo

[PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Lorenzo Pieralisi]

On Tue, Jul 01, 2014 at 02:14:49PM +0100, Abhilash Kesavan wrote:
> Hi Nicolas,
> 
> On Tue, Jul 1, 2014 at 9:49 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Mon, 30 Jun 2014, Abhilash Kesavan wrote:
> >
> >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> >> Also, restore the entry address setup code post-resume.
> >>
> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> >> ---
> > [...]
> >
> > Could you tell me more about this?
> >
> >> @@ -150,7 +153,13 @@ static void exynos_power_down(void)
> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> >>       cpu_use_count[cpu][cluster]--;
> >>       if (cpu_use_count[cpu][cluster] == 0) {
> >> -             exynos_cpu_power_down(cpunr);
> >> +             /*
> >> +              * Bypass power down for CPU0 during suspend. This is being
> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> >> +              */
> >> +             if ((cpunr != 0) ||
> >> +             (__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
> >> +                     exynos_cpu_power_down(cpunr);
> >
> > What happens if CPU0 is the first in the cluster to be idled?  Will it
> > remain powered up until all the others have gone idle too?
> This check is only for handling the S2R CPU0 case. In case of
> idle/switching the S5P_CHECK_SLEEP flag would not be set and hence
> there would be no change in behavior for them.
> During suspend, we enable the ARM_USE_STANDBY_WFI0 bit which tells the
> PMU when the CPU0 is ready to be suspended. This in conjunction with
> the sleep state core configuration (setting SYS_PWR_REG low) causes
> the CPU0 to go down. We should not write to the CPU0 power
> configuration register (exynos_cpu_power_down) along with this during
> suspend.

I think this should be part of a refactoring that includes the exynos MCPM
suspend call parameters. In particular, at the moment there is no code in
the back-end to detect if the last man should request core gating or cluster
gating (ie last man _always_ request cluster gating, that might not be what
we want), there is the residency value that can be also be used to imply a
S2R request (eg residency = ~0 ?).

The command sent must be implied by the state that is entered, not by
peeking at registers that should contain magic values, and that's true
not only for exynos but for all ARM platforms out there.

What I mean is: we can pass the requested state as a suspend parameter
and the power_down state machine will send the required command
accordingly.

It can be done using the residency value or by passing the power state "index"
as suspend parameter, the power down MCPM code will do a look-up and send
the respective command.

Thoughts appreciated.

Lorenzo

Re: [PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Nicolas Pitre]

On Tue, 1 Jul 2014, Lorenzo Pieralisi wrote:

> On Tue, Jul 01, 2014 at 02:14:49PM +0100, Abhilash Kesavan wrote:
> > Hi Nicolas,
> > 
> > On Tue, Jul 1, 2014 at 9:49 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > > On Mon, 30 Jun 2014, Abhilash Kesavan wrote:
> > >
> > >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> > >> Also, restore the entry address setup code post-resume.
> > >>
> > >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > >> ---
> > > [...]
> > >
> > > Could you tell me more about this?
> > >
> > >> @@ -150,7 +153,13 @@ static void exynos_power_down(void)
> > >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> > >>       cpu_use_count[cpu][cluster]--;
> > >>       if (cpu_use_count[cpu][cluster] == 0) {
> > >> -             exynos_cpu_power_down(cpunr);
> > >> +             /*
> > >> +              * Bypass power down for CPU0 during suspend. This is being
> > >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> > >> +              */
> > >> +             if ((cpunr != 0) ||
> > >> +             (__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
> > >> +                     exynos_cpu_power_down(cpunr);
> > >
> > > What happens if CPU0 is the first in the cluster to be idled?  Will it
> > > remain powered up until all the others have gone idle too?
> > This check is only for handling the S2R CPU0 case. In case of
> > idle/switching the S5P_CHECK_SLEEP flag would not be set and hence
> > there would be no change in behavior for them.
> > During suspend, we enable the ARM_USE_STANDBY_WFI0 bit which tells the
> > PMU when the CPU0 is ready to be suspended. This in conjunction with
> > the sleep state core configuration (setting SYS_PWR_REG low) causes
> > the CPU0 to go down. We should not write to the CPU0 power
> > configuration register (exynos_cpu_power_down) along with this during
> > suspend.
> 
> I think this should be part of a refactoring that includes the exynos MCPM
> suspend call parameters. In particular, at the moment there is no code in
> the back-end to detect if the last man should request core gating or cluster
> gating (ie last man _always_ request cluster gating, that might not be what
> we want), there is the residency value that can be also be used to imply a
> S2R request (eg residency = ~0 ?).
> 
> The command sent must be implied by the state that is entered, not by
> peeking at registers that should contain magic values, and that's true
> not only for exynos but for all ARM platforms out there.
> 
> What I mean is: we can pass the requested state as a suspend parameter
> and the power_down state machine will send the required command
> accordingly.
> 
> It can be done using the residency value or by passing the power state "index"
> as suspend parameter, the power down MCPM code will do a look-up and send
> the respective command.
> 
> Thoughts appreciated.

I agree.  Having the MCPM abstraction having to rely on some magic value 
to be set in a register beforehand for things to work properly is not 
nice.


Nicolas

[PATCH v2] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Nicolas Pitre]

On Tue, 1 Jul 2014, Lorenzo Pieralisi wrote:

> On Tue, Jul 01, 2014 at 02:14:49PM +0100, Abhilash Kesavan wrote:
> > Hi Nicolas,
> > 
> > On Tue, Jul 1, 2014 at 9:49 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > > On Mon, 30 Jun 2014, Abhilash Kesavan wrote:
> > >
> > >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> > >> Also, restore the entry address setup code post-resume.
> > >>
> > >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> > >> ---
> > > [...]
> > >
> > > Could you tell me more about this?
> > >
> > >> @@ -150,7 +153,13 @@ static void exynos_power_down(void)
> > >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> > >>       cpu_use_count[cpu][cluster]--;
> > >>       if (cpu_use_count[cpu][cluster] == 0) {
> > >> -             exynos_cpu_power_down(cpunr);
> > >> +             /*
> > >> +              * Bypass power down for CPU0 during suspend. This is being
> > >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> > >> +              */
> > >> +             if ((cpunr != 0) ||
> > >> +             (__raw_readl(pmu_base_addr + S5P_INFORM1) != S5P_CHECK_SLEEP))
> > >> +                     exynos_cpu_power_down(cpunr);
> > >
> > > What happens if CPU0 is the first in the cluster to be idled?  Will it
> > > remain powered up until all the others have gone idle too?
> > This check is only for handling the S2R CPU0 case. In case of
> > idle/switching the S5P_CHECK_SLEEP flag would not be set and hence
> > there would be no change in behavior for them.
> > During suspend, we enable the ARM_USE_STANDBY_WFI0 bit which tells the
> > PMU when the CPU0 is ready to be suspended. This in conjunction with
> > the sleep state core configuration (setting SYS_PWR_REG low) causes
> > the CPU0 to go down. We should not write to the CPU0 power
> > configuration register (exynos_cpu_power_down) along with this during
> > suspend.
> 
> I think this should be part of a refactoring that includes the exynos MCPM
> suspend call parameters. In particular, at the moment there is no code in
> the back-end to detect if the last man should request core gating or cluster
> gating (ie last man _always_ request cluster gating, that might not be what
> we want), there is the residency value that can be also be used to imply a
> S2R request (eg residency = ~0 ?).
> 
> The command sent must be implied by the state that is entered, not by
> peeking at registers that should contain magic values, and that's true
> not only for exynos but for all ARM platforms out there.
> 
> What I mean is: we can pass the requested state as a suspend parameter
> and the power_down state machine will send the required command
> accordingly.
> 
> It can be done using the residency value or by passing the power state "index"
> as suspend parameter, the power down MCPM code will do a look-up and send
> the respective command.
> 
> Thoughts appreciated.

I agree.  Having the MCPM abstraction having to rely on some magic value 
to be set in a register beforehand for things to work properly is not 
nice.


Nicolas

[RFC PATCH v3] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Hi Lorenzo and Nicolas,

I have re-worked the patch to use the residency value as an indicator of the
current state. This is untested and I haven't taken care of any changes that
tc2 might require. Posting this as an RFC to check if this is the expected
approach.

Depencies remain the same as v2.

 arch/arm/include/asm/mcpm.h          |  6 +++++
 arch/arm/mach-exynos/mcpm-exynos.c   | 50 ++++++++++++++++++++++++++----------
 arch/arm/mach-exynos/pm.c            | 38 ++++++++++++++++++++++++---
 arch/arm/mach-exynos/regs-pmu.h      |  1 +
 drivers/cpuidle/cpuidle-big_little.c |  2 +-
 5 files changed, 79 insertions(+), 18 deletions(-)

diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
index ff73aff..0848829 100644
--- a/arch/arm/include/asm/mcpm.h
+++ b/arch/arm/include/asm/mcpm.h
@@ -272,4 +272,10 @@ void __init mcpm_smp_set_ops(void);
 #define MCPM_SYNC_CLUSTER_SIZE \
 	(MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE)
 
+/* Definitions for various MCPM scenarios that might need special handling */
+#define MCPM_CPU_IDLE		0x0
+#define MCPM_CPU_SUSPEND	0x1
+#define MCPM_CPU_SWITCH		0x2
+#define MCPM_CPU_HOTPLUG	0x3
+
 #endif
diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 1baca55..c3673bd 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
  * and can only be executed on processors like A15 and A7 that hit the cache
  * with the C bit clear in the SCTLR register.
  */
-static void exynos_power_down(void)
+static void exynos_mcpm_power_down(u64 residency)
 {
 	unsigned int mpidr, cpu, cluster;
 	bool last_man = false, skip_wfi = false;
@@ -150,7 +153,12 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. This is being
+		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
+		 */
+		if ((cpunr != 0) || (residency != MCPM_CPU_SUSPEND))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -209,6 +217,11 @@ static void exynos_power_down(void)
 	/* Not dead at this point?  Let our caller cope. */
 }
 
+static void exynos_power_down(void)
+{
+	exynos_mcpm_power_down(MCPM_CPU_SWITCH | MCPM_CPU_HOTPLUG);
+}
+
 static int exynos_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
 {
 	unsigned int tries = 100;
@@ -250,11 +263,11 @@ static void exynos_suspend(u64 residency)
 {
 	unsigned int mpidr, cpunr;
 
-	exynos_power_down();
+	exynos_mcpm_power_down(residency);
 
 	/*
 	 * Execution reaches here only if cpu did not power down.
-	 * Hence roll back the changes done in exynos_power_down function.
+	 * Hence roll back the changes done in exynos_mcpm_power_down function.
 	 *
 	 * CAUTION: "This function requires the stack data to be visible through
 	 * power down and can only be executed on processors like A15 and A7
@@ -319,10 +332,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +418,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index bf8564a..1dce5d2 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -191,7 +192,6 @@ int exynos_cluster_power_state(int cluster)
 			pmu_base_addr + S5P_INFORM1))
 
 #define S5P_CHECK_AFTR  0xFCBA0D10
-#define S5P_CHECK_SLEEP 0x00000BAD
 
 /* Ext-GIC nIRQ/nFIQ is the only wakeup source in AFTR */
 static void exynos_set_wakeupmask(long mask)
@@ -318,7 +318,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420())
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -490,6 +493,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(MCPM_CPU_SUSPEND);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +542,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420())
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420())
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index 3cf0454..e8c75db 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -152,6 +152,7 @@
 #define S5P_PAD_RET_EBIB_OPTION			0x31A8
 
 #define S5P_CORE_LOCAL_PWR_EN			0x3
+#define S5P_CHECK_SLEEP				0x00000BAD
 
 /* Only for EXYNOS4210 */
 #define S5P_CMU_CLKSTOP_LCD1_LOWPWR	0x1154
diff --git a/drivers/cpuidle/cpuidle-big_little.c b/drivers/cpuidle/cpuidle-big_little.c
index 344d79fa..fab2270 100644
--- a/drivers/cpuidle/cpuidle-big_little.c
+++ b/drivers/cpuidle/cpuidle-big_little.c
@@ -108,7 +108,7 @@ static int notrace bl_powerdown_finisher(unsigned long arg)
 	 * has to be given clear semantics. Set to 0 as a
 	 * temporary value.
 	 */
-	mcpm_cpu_suspend(0);
+	mcpm_cpu_suspend(MCPM_CPU_IDLE);
 
 	/* return value != 0 means failure */
 	return 1;
-- 
2.0.0

[RFC PATCH v3] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Hi Lorenzo and Nicolas,

I have re-worked the patch to use the residency value as an indicator of the
current state. This is untested and I haven't taken care of any changes that
tc2 might require. Posting this as an RFC to check if this is the expected
approach.

Depencies remain the same as v2.

 arch/arm/include/asm/mcpm.h          |  6 +++++
 arch/arm/mach-exynos/mcpm-exynos.c   | 50 ++++++++++++++++++++++++++----------
 arch/arm/mach-exynos/pm.c            | 38 ++++++++++++++++++++++++---
 arch/arm/mach-exynos/regs-pmu.h      |  1 +
 drivers/cpuidle/cpuidle-big_little.c |  2 +-
 5 files changed, 79 insertions(+), 18 deletions(-)

diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
index ff73aff..0848829 100644
--- a/arch/arm/include/asm/mcpm.h
+++ b/arch/arm/include/asm/mcpm.h
@@ -272,4 +272,10 @@ void __init mcpm_smp_set_ops(void);
 #define MCPM_SYNC_CLUSTER_SIZE \
 	(MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE)
 
+/* Definitions for various MCPM scenarios that might need special handling */
+#define MCPM_CPU_IDLE		0x0
+#define MCPM_CPU_SUSPEND	0x1
+#define MCPM_CPU_SWITCH		0x2
+#define MCPM_CPU_HOTPLUG	0x3
+
 #endif
diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 1baca55..c3673bd 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
  * and can only be executed on processors like A15 and A7 that hit the cache
  * with the C bit clear in the SCTLR register.
  */
-static void exynos_power_down(void)
+static void exynos_mcpm_power_down(u64 residency)
 {
 	unsigned int mpidr, cpu, cluster;
 	bool last_man = false, skip_wfi = false;
@@ -150,7 +153,12 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. This is being
+		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
+		 */
+		if ((cpunr != 0) || (residency != MCPM_CPU_SUSPEND))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -209,6 +217,11 @@ static void exynos_power_down(void)
 	/* Not dead at this point?  Let our caller cope. */
 }
 
+static void exynos_power_down(void)
+{
+	exynos_mcpm_power_down(MCPM_CPU_SWITCH | MCPM_CPU_HOTPLUG);
+}
+
 static int exynos_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
 {
 	unsigned int tries = 100;
@@ -250,11 +263,11 @@ static void exynos_suspend(u64 residency)
 {
 	unsigned int mpidr, cpunr;
 
-	exynos_power_down();
+	exynos_mcpm_power_down(residency);
 
 	/*
 	 * Execution reaches here only if cpu did not power down.
-	 * Hence roll back the changes done in exynos_power_down function.
+	 * Hence roll back the changes done in exynos_mcpm_power_down function.
 	 *
 	 * CAUTION: "This function requires the stack data to be visible through
 	 * power down and can only be executed on processors like A15 and A7
@@ -319,10 +332,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +418,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index bf8564a..1dce5d2 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -191,7 +192,6 @@ int exynos_cluster_power_state(int cluster)
 			pmu_base_addr + S5P_INFORM1))
 
 #define S5P_CHECK_AFTR  0xFCBA0D10
-#define S5P_CHECK_SLEEP 0x00000BAD
 
 /* Ext-GIC nIRQ/nFIQ is the only wakeup source in AFTR */
 static void exynos_set_wakeupmask(long mask)
@@ -318,7 +318,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420())
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -490,6 +493,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(MCPM_CPU_SUSPEND);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +542,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420())
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420())
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index 3cf0454..e8c75db 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -152,6 +152,7 @@
 #define S5P_PAD_RET_EBIB_OPTION			0x31A8
 
 #define S5P_CORE_LOCAL_PWR_EN			0x3
+#define S5P_CHECK_SLEEP				0x00000BAD
 
 /* Only for EXYNOS4210 */
 #define S5P_CMU_CLKSTOP_LCD1_LOWPWR	0x1154
diff --git a/drivers/cpuidle/cpuidle-big_little.c b/drivers/cpuidle/cpuidle-big_little.c
index 344d79fa..fab2270 100644
--- a/drivers/cpuidle/cpuidle-big_little.c
+++ b/drivers/cpuidle/cpuidle-big_little.c
@@ -108,7 +108,7 @@ static int notrace bl_powerdown_finisher(unsigned long arg)
 	 * has to be given clear semantics. Set to 0 as a
 	 * temporary value.
 	 */
-	mcpm_cpu_suspend(0);
+	mcpm_cpu_suspend(MCPM_CPU_IDLE);
 
 	/* return value != 0 means failure */
 	return 1;
-- 
2.0.0

[RFC PATCH v4] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Changes in v2:
	- Made use of the MCPM suspend/powered_up call-backs
Changes in v3:
	- Used the residency value to indicate the entered state
Changes in v4:
	- Checked if MCPM has been enabled to prevent build error

This has been tested both on an SMDK5420 and Peach Pit Chromebook on
3.16-rc3/next-20140702.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) MCPM boot CPU CCI enablement patches
www.spinics.net/lists/linux-samsung-soc/msg32923.html

5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

6) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

7) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html

 arch/arm/include/asm/mcpm.h          |    6 ++++
 arch/arm/mach-exynos/mcpm-exynos.c   |   50 ++++++++++++++++++++++++----------
 arch/arm/mach-exynos/pm.c            |   38 ++++++++++++++++++++++++--
 arch/arm/mach-exynos/regs-pmu.h      |    1 +
 drivers/cpuidle/cpuidle-big_little.c |    2 +-
 5 files changed, 79 insertions(+), 18 deletions(-)

diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
index ff73aff..051fbf1 100644
--- a/arch/arm/include/asm/mcpm.h
+++ b/arch/arm/include/asm/mcpm.h
@@ -272,4 +272,10 @@ void __init mcpm_smp_set_ops(void);
 #define MCPM_SYNC_CLUSTER_SIZE \
 	(MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE)
 
+/* Definitions for various MCPM scenarios that might need special handling */
+#define MCPM_CPU_IDLE		0x0
+#define MCPM_CPU_SUSPEND	0x1
+#define MCPM_CPU_SWITCH		0x2
+#define MCPM_CPU_HOTPLUG	0x3
+
 #endif
diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 0315601..9a381f6 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
  * and can only be executed on processors like A15 and A7 that hit the cache
  * with the C bit clear in the SCTLR register.
  */
-static void exynos_power_down(void)
+static void exynos_mcpm_power_down(u64 residency)
 {
 	unsigned int mpidr, cpu, cluster;
 	bool last_man = false, skip_wfi = false;
@@ -150,7 +153,12 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. This is being
+		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
+		 */
+		if ((cpunr != 0) || (residency != MCPM_CPU_SUSPEND))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -209,6 +217,11 @@ static void exynos_power_down(void)
 	/* Not dead at this point?  Let our caller cope. */
 }
 
+static void exynos_power_down(void)
+{
+	exynos_mcpm_power_down(MCPM_CPU_SWITCH | MCPM_CPU_HOTPLUG);
+}
+
 static int exynos_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
 {
 	unsigned int tries = 100;
@@ -250,11 +263,11 @@ static void exynos_suspend(u64 residency)
 {
 	unsigned int mpidr, cpunr;
 
-	exynos_power_down();
+	exynos_mcpm_power_down(residency);
 
 	/*
 	 * Execution reaches here only if cpu did not power down.
-	 * Hence roll back the changes done in exynos_power_down function.
+	 * Hence roll back the changes done in exynos_mcpm_power_down function.
 	 *
 	 * CAUTION: "This function requires the stack data to be visible through
 	 * power down and can only be executed on processors like A15 and A7
@@ -319,10 +332,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +418,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index bf8564a..8b425df 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -191,7 +192,6 @@ int exynos_cluster_power_state(int cluster)
 			pmu_base_addr + S5P_INFORM1))
 
 #define S5P_CHECK_AFTR  0xFCBA0D10
-#define S5P_CHECK_SLEEP 0x00000BAD
 
 /* Ext-GIC nIRQ/nFIQ is the only wakeup source in AFTR */
 static void exynos_set_wakeupmask(long mask)
@@ -318,7 +318,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -490,6 +493,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(MCPM_CPU_SUSPEND);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +542,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index 3cf0454..e8c75db 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -152,6 +152,7 @@
 #define S5P_PAD_RET_EBIB_OPTION			0x31A8
 
 #define S5P_CORE_LOCAL_PWR_EN			0x3
+#define S5P_CHECK_SLEEP				0x00000BAD
 
 /* Only for EXYNOS4210 */
 #define S5P_CMU_CLKSTOP_LCD1_LOWPWR	0x1154
diff --git a/drivers/cpuidle/cpuidle-big_little.c b/drivers/cpuidle/cpuidle-big_little.c
index b45fc62..15f077e 100644
--- a/drivers/cpuidle/cpuidle-big_little.c
+++ b/drivers/cpuidle/cpuidle-big_little.c
@@ -108,7 +108,7 @@ static int notrace bl_powerdown_finisher(unsigned long arg)
 	 * has to be given clear semantics. Set to 0 as a
 	 * temporary value.
 	 */
-	mcpm_cpu_suspend(0);
+	mcpm_cpu_suspend(MCPM_CPU_IDLE);
 
 	/* return value != 0 means failure */
 	return 1;
-- 
1.7.9.5

[RFC PATCH v4] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Changes in v2:
	- Made use of the MCPM suspend/powered_up call-backs
Changes in v3:
	- Used the residency value to indicate the entered state
Changes in v4:
	- Checked if MCPM has been enabled to prevent build error

This has been tested both on an SMDK5420 and Peach Pit Chromebook on
3.16-rc3/next-20140702.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) MCPM boot CPU CCI enablement patches
www.spinics.net/lists/linux-samsung-soc/msg32923.html

5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

6) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

7) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html

 arch/arm/include/asm/mcpm.h          |    6 ++++
 arch/arm/mach-exynos/mcpm-exynos.c   |   50 ++++++++++++++++++++++++----------
 arch/arm/mach-exynos/pm.c            |   38 ++++++++++++++++++++++++--
 arch/arm/mach-exynos/regs-pmu.h      |    1 +
 drivers/cpuidle/cpuidle-big_little.c |    2 +-
 5 files changed, 79 insertions(+), 18 deletions(-)

diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
index ff73aff..051fbf1 100644
--- a/arch/arm/include/asm/mcpm.h
+++ b/arch/arm/include/asm/mcpm.h
@@ -272,4 +272,10 @@ void __init mcpm_smp_set_ops(void);
 #define MCPM_SYNC_CLUSTER_SIZE \
 	(MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE)
 
+/* Definitions for various MCPM scenarios that might need special handling */
+#define MCPM_CPU_IDLE		0x0
+#define MCPM_CPU_SUSPEND	0x1
+#define MCPM_CPU_SWITCH		0x2
+#define MCPM_CPU_HOTPLUG	0x3
+
 #endif
diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 0315601..9a381f6 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
  * and can only be executed on processors like A15 and A7 that hit the cache
  * with the C bit clear in the SCTLR register.
  */
-static void exynos_power_down(void)
+static void exynos_mcpm_power_down(u64 residency)
 {
 	unsigned int mpidr, cpu, cluster;
 	bool last_man = false, skip_wfi = false;
@@ -150,7 +153,12 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. This is being
+		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
+		 */
+		if ((cpunr != 0) || (residency != MCPM_CPU_SUSPEND))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -209,6 +217,11 @@ static void exynos_power_down(void)
 	/* Not dead at this point?  Let our caller cope. */
 }
 
+static void exynos_power_down(void)
+{
+	exynos_mcpm_power_down(MCPM_CPU_SWITCH | MCPM_CPU_HOTPLUG);
+}
+
 static int exynos_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
 {
 	unsigned int tries = 100;
@@ -250,11 +263,11 @@ static void exynos_suspend(u64 residency)
 {
 	unsigned int mpidr, cpunr;
 
-	exynos_power_down();
+	exynos_mcpm_power_down(residency);
 
 	/*
 	 * Execution reaches here only if cpu did not power down.
-	 * Hence roll back the changes done in exynos_power_down function.
+	 * Hence roll back the changes done in exynos_mcpm_power_down function.
 	 *
 	 * CAUTION: "This function requires the stack data to be visible through
 	 * power down and can only be executed on processors like A15 and A7
@@ -319,10 +332,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +418,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index bf8564a..8b425df 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -191,7 +192,6 @@ int exynos_cluster_power_state(int cluster)
 			pmu_base_addr + S5P_INFORM1))
 
 #define S5P_CHECK_AFTR  0xFCBA0D10
-#define S5P_CHECK_SLEEP 0x00000BAD
 
 /* Ext-GIC nIRQ/nFIQ is the only wakeup source in AFTR */
 static void exynos_set_wakeupmask(long mask)
@@ -318,7 +318,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -490,6 +493,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(MCPM_CPU_SUSPEND);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +542,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index 3cf0454..e8c75db 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -152,6 +152,7 @@
 #define S5P_PAD_RET_EBIB_OPTION			0x31A8
 
 #define S5P_CORE_LOCAL_PWR_EN			0x3
+#define S5P_CHECK_SLEEP				0x00000BAD
 
 /* Only for EXYNOS4210 */
 #define S5P_CMU_CLKSTOP_LCD1_LOWPWR	0x1154
diff --git a/drivers/cpuidle/cpuidle-big_little.c b/drivers/cpuidle/cpuidle-big_little.c
index b45fc62..15f077e 100644
--- a/drivers/cpuidle/cpuidle-big_little.c
+++ b/drivers/cpuidle/cpuidle-big_little.c
@@ -108,7 +108,7 @@ static int notrace bl_powerdown_finisher(unsigned long arg)
 	 * has to be given clear semantics. Set to 0 as a
 	 * temporary value.
 	 */
-	mcpm_cpu_suspend(0);
+	mcpm_cpu_suspend(MCPM_CPU_IDLE);
 
 	/* return value != 0 means failure */
 	return 1;
-- 
1.7.9.5

Re: [RFC PATCH v4] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Nicolas Pitre]

On Thu, 3 Jul 2014, Abhilash Kesavan wrote:

> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> Also, restore the entry address setup code post-resume.
> 
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> ---
> Changes in v2:
> 	- Made use of the MCPM suspend/powered_up call-backs
> Changes in v3:
> 	- Used the residency value to indicate the entered state
> Changes in v4:
> 	- Checked if MCPM has been enabled to prevent build error
> 
> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
> 3.16-rc3/next-20140702.
> 
> Here are the dependencies (some of these patches did not apply cleanly):
> 1) Cleanup patches for mach-exynos
> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
> 
> 2) PMU cleanup and refactoring for using DT
> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html
> 
> 3) Exynos5420 PMU/S2R Series
> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
> 
> 4) MCPM boot CPU CCI enablement patches
> www.spinics.net/lists/linux-samsung-soc/msg32923.html
> 
> 5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
> call-backs.
> www.gossamer-threads.com/lists/linux/kernel/1945347
> https://patchwork.kernel.org/patch/4357461/
> 
> 6) Exynos5420 MCPM cluster power down support
> http://www.spinics.net/lists/arm-kernel/msg339988.html
> 
> 7) TPM reset mask patch
> http://www.spinics.net/lists/arm-kernel/msg341884.html
> 
>  arch/arm/include/asm/mcpm.h          |    6 ++++
>  arch/arm/mach-exynos/mcpm-exynos.c   |   50 ++++++++++++++++++++++++----------
>  arch/arm/mach-exynos/pm.c            |   38 ++++++++++++++++++++++++--
>  arch/arm/mach-exynos/regs-pmu.h      |    1 +
>  drivers/cpuidle/cpuidle-big_little.c |    2 +-
>  5 files changed, 79 insertions(+), 18 deletions(-)
> 
> diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
> index ff73aff..051fbf1 100644
> --- a/arch/arm/include/asm/mcpm.h
> +++ b/arch/arm/include/asm/mcpm.h
> @@ -272,4 +272,10 @@ void __init mcpm_smp_set_ops(void);
>  #define MCPM_SYNC_CLUSTER_SIZE \
>  	(MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE)
>  
> +/* Definitions for various MCPM scenarios that might need special handling */
> +#define MCPM_CPU_IDLE		0x0
> +#define MCPM_CPU_SUSPEND	0x1
> +#define MCPM_CPU_SWITCH		0x2
> +#define MCPM_CPU_HOTPLUG	0x3

Please, let's avoid going that route.  There is no such special handling 
needed if the API is sufficient.  And the provided API allows you to 
suspend a CPU or shut it down.  It shouldn't matter at that level if 
this is due to a cluster switch or a hotplug event. Do you really need 
something else?

[...]
> @@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
>   * and can only be executed on processors like A15 and A7 that hit the cache
>   * with the C bit clear in the SCTLR register.
>   */
> -static void exynos_power_down(void)
> +static void exynos_mcpm_power_down(u64 residency)
>  {
>  	unsigned int mpidr, cpu, cluster;
>  	bool last_man = false, skip_wfi = false;
> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>  	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>  	cpu_use_count[cpu][cluster]--;
>  	if (cpu_use_count[cpu][cluster] == 0) {
> -		exynos_cpu_power_down(cpunr);
> +		/*
> +		 * Bypass power down for CPU0 during suspend. This is being
> +		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> +		 */
> +		if ((cpunr != 0) || (residency != MCPM_CPU_SUSPEND))
> +			exynos_cpu_power_down(cpunr);
>  
>  		if (exynos_cluster_unused(cluster)) {
>  			exynos_cluster_power_down(cluster);
> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>  	/* Not dead at this point?  Let our caller cope. */
>  }
>  
> +static void exynos_power_down(void)
> +{
> +	exynos_mcpm_power_down(MCPM_CPU_SWITCH | MCPM_CPU_HOTPLUG);
> +}

To distinguish between a suspend and a power-down, you can simply use 
exynos_power_down() as your common handler, and have 
exynos_mcpm_power_down() and exynos_mcpm_suspend() as wrappers around it 
passing the appropriate private flags with local meanings.


Nicolas

[RFC PATCH v4] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Nicolas Pitre]

On Thu, 3 Jul 2014, Abhilash Kesavan wrote:

> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> Also, restore the entry address setup code post-resume.
> 
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> ---
> Changes in v2:
> 	- Made use of the MCPM suspend/powered_up call-backs
> Changes in v3:
> 	- Used the residency value to indicate the entered state
> Changes in v4:
> 	- Checked if MCPM has been enabled to prevent build error
> 
> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
> 3.16-rc3/next-20140702.
> 
> Here are the dependencies (some of these patches did not apply cleanly):
> 1) Cleanup patches for mach-exynos
> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
> 
> 2) PMU cleanup and refactoring for using DT
> https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html
> 
> 3) Exynos5420 PMU/S2R Series
> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
> 
> 4) MCPM boot CPU CCI enablement patches
> www.spinics.net/lists/linux-samsung-soc/msg32923.html
> 
> 5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
> call-backs.
> www.gossamer-threads.com/lists/linux/kernel/1945347
> https://patchwork.kernel.org/patch/4357461/
> 
> 6) Exynos5420 MCPM cluster power down support
> http://www.spinics.net/lists/arm-kernel/msg339988.html
> 
> 7) TPM reset mask patch
> http://www.spinics.net/lists/arm-kernel/msg341884.html
> 
>  arch/arm/include/asm/mcpm.h          |    6 ++++
>  arch/arm/mach-exynos/mcpm-exynos.c   |   50 ++++++++++++++++++++++++----------
>  arch/arm/mach-exynos/pm.c            |   38 ++++++++++++++++++++++++--
>  arch/arm/mach-exynos/regs-pmu.h      |    1 +
>  drivers/cpuidle/cpuidle-big_little.c |    2 +-
>  5 files changed, 79 insertions(+), 18 deletions(-)
> 
> diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
> index ff73aff..051fbf1 100644
> --- a/arch/arm/include/asm/mcpm.h
> +++ b/arch/arm/include/asm/mcpm.h
> @@ -272,4 +272,10 @@ void __init mcpm_smp_set_ops(void);
>  #define MCPM_SYNC_CLUSTER_SIZE \
>  	(MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE)
>  
> +/* Definitions for various MCPM scenarios that might need special handling */
> +#define MCPM_CPU_IDLE		0x0
> +#define MCPM_CPU_SUSPEND	0x1
> +#define MCPM_CPU_SWITCH		0x2
> +#define MCPM_CPU_HOTPLUG	0x3

Please, let's avoid going that route.  There is no such special handling 
needed if the API is sufficient.  And the provided API allows you to 
suspend a CPU or shut it down.  It shouldn't matter at that level if 
this is due to a cluster switch or a hotplug event. Do you really need 
something else?

[...]
> @@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
>   * and can only be executed on processors like A15 and A7 that hit the cache
>   * with the C bit clear in the SCTLR register.
>   */
> -static void exynos_power_down(void)
> +static void exynos_mcpm_power_down(u64 residency)
>  {
>  	unsigned int mpidr, cpu, cluster;
>  	bool last_man = false, skip_wfi = false;
> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>  	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>  	cpu_use_count[cpu][cluster]--;
>  	if (cpu_use_count[cpu][cluster] == 0) {
> -		exynos_cpu_power_down(cpunr);
> +		/*
> +		 * Bypass power down for CPU0 during suspend. This is being
> +		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> +		 */
> +		if ((cpunr != 0) || (residency != MCPM_CPU_SUSPEND))
> +			exynos_cpu_power_down(cpunr);
>  
>  		if (exynos_cluster_unused(cluster)) {
>  			exynos_cluster_power_down(cluster);
> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>  	/* Not dead at this point?  Let our caller cope. */
>  }
>  
> +static void exynos_power_down(void)
> +{
> +	exynos_mcpm_power_down(MCPM_CPU_SWITCH | MCPM_CPU_HOTPLUG);
> +}

To distinguish between a suspend and a power-down, you can simply use 
exynos_power_down() as your common handler, and have 
exynos_mcpm_power_down() and exynos_mcpm_suspend() as wrappers around it 
passing the appropriate private flags with local meanings.


Nicolas

Re: [RFC PATCH v4] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Hi Nicolas,

On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>
>> Use the MCPM layer to handle core suspend/resume on Exynos5420.
>> Also, restore the entry address setup code post-resume.
>>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> ---
>> Changes in v2:
>>       - Made use of the MCPM suspend/powered_up call-backs
>> Changes in v3:
>>       - Used the residency value to indicate the entered state
>> Changes in v4:
>>       - Checked if MCPM has been enabled to prevent build error
>>
>> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
>> 3.16-rc3/next-20140702.
>>
>> Here are the dependencies (some of these patches did not apply cleanly):
>> 1) Cleanup patches for mach-exynos
>> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
>>
>> 2) PMU cleanup and refactoring for using DT
>> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html
>>
>> 3) Exynos5420 PMU/S2R Series
>> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
>>
>> 4) MCPM boot CPU CCI enablement patches
>> www.spinics.net/lists/linux-samsung-soc/msg32923.html
>>
>> 5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
>> call-backs.
>> www.gossamer-threads.com/lists/linux/kernel/1945347
>> https://patchwork.kernel.org/patch/4357461/
>>
>> 6) Exynos5420 MCPM cluster power down support
>> http://www.spinics.net/lists/arm-kernel/msg339988.html
>>
>> 7) TPM reset mask patch
>> http://www.spinics.net/lists/arm-kernel/msg341884.html
>>
>>  arch/arm/include/asm/mcpm.h          |    6 ++++
>>  arch/arm/mach-exynos/mcpm-exynos.c   |   50 ++++++++++++++++++++++++----------
>>  arch/arm/mach-exynos/pm.c            |   38 ++++++++++++++++++++++++--
>>  arch/arm/mach-exynos/regs-pmu.h      |    1 +
>>  drivers/cpuidle/cpuidle-big_little.c |    2 +-
>>  5 files changed, 79 insertions(+), 18 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
>> index ff73aff..051fbf1 100644
>> --- a/arch/arm/include/asm/mcpm.h
>> +++ b/arch/arm/include/asm/mcpm.h
>> @@ -272,4 +272,10 @@ void __init mcpm_smp_set_ops(void);
>>  #define MCPM_SYNC_CLUSTER_SIZE \
>>       (MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE)
>>
>> +/* Definitions for various MCPM scenarios that might need special handling */
>> +#define MCPM_CPU_IDLE                0x0
>> +#define MCPM_CPU_SUSPEND     0x1
>> +#define MCPM_CPU_SWITCH              0x2
>> +#define MCPM_CPU_HOTPLUG     0x3
>
> Please, let's avoid going that route.  There is no such special handling
> needed if the API is sufficient.  And the provided API allows you to
> suspend a CPU or shut it down.  It shouldn't matter at that level if
> this is due to a cluster switch or a hotplug event. Do you really need
> something else?
No, just one local flag for suspend should be enough for me. Will remove these.
>
> [...]
>> @@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
>>   * and can only be executed on processors like A15 and A7 that hit the cache
>>   * with the C bit clear in the SCTLR register.
>>   */
>> -static void exynos_power_down(void)
>> +static void exynos_mcpm_power_down(u64 residency)
>>  {
>>       unsigned int mpidr, cpu, cluster;
>>       bool last_man = false, skip_wfi = false;
>> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>>       cpu_use_count[cpu][cluster]--;
>>       if (cpu_use_count[cpu][cluster] == 0) {
>> -             exynos_cpu_power_down(cpunr);
>> +             /*
>> +              * Bypass power down for CPU0 during suspend. This is being
>> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> +              */
>> +             if ((cpunr != 0) || (residency != MCPM_CPU_SUSPEND))
>> +                     exynos_cpu_power_down(cpunr);
>>
>>               if (exynos_cluster_unused(cluster)) {
>>                       exynos_cluster_power_down(cluster);
>> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>>       /* Not dead at this point?  Let our caller cope. */
>>  }
>>
>> +static void exynos_power_down(void)
>> +{
>> +     exynos_mcpm_power_down(MCPM_CPU_SWITCH | MCPM_CPU_HOTPLUG);
>> +}
>
> To distinguish between a suspend and a power-down, you can simply use
> exynos_power_down() as your common handler, and have
> exynos_mcpm_power_down() and exynos_mcpm_suspend() as wrappers around it
> passing the appropriate private flags with local meanings.
My patch is in line with your explanation, except it makes use of the
new MCPM flags. I will clean those out and just use one flag for
suspend instead. Please check the v5 patch that I will post in a bit
in case I have missed something.

Regards,
Abhilash
>
>
> Nicolas
>

[RFC PATCH v4] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Hi Nicolas,

On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>
>> Use the MCPM layer to handle core suspend/resume on Exynos5420.
>> Also, restore the entry address setup code post-resume.
>>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> ---
>> Changes in v2:
>>       - Made use of the MCPM suspend/powered_up call-backs
>> Changes in v3:
>>       - Used the residency value to indicate the entered state
>> Changes in v4:
>>       - Checked if MCPM has been enabled to prevent build error
>>
>> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
>> 3.16-rc3/next-20140702.
>>
>> Here are the dependencies (some of these patches did not apply cleanly):
>> 1) Cleanup patches for mach-exynos
>> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
>>
>> 2) PMU cleanup and refactoring for using DT
>> https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html
>>
>> 3) Exynos5420 PMU/S2R Series
>> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
>>
>> 4) MCPM boot CPU CCI enablement patches
>> www.spinics.net/lists/linux-samsung-soc/msg32923.html
>>
>> 5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
>> call-backs.
>> www.gossamer-threads.com/lists/linux/kernel/1945347
>> https://patchwork.kernel.org/patch/4357461/
>>
>> 6) Exynos5420 MCPM cluster power down support
>> http://www.spinics.net/lists/arm-kernel/msg339988.html
>>
>> 7) TPM reset mask patch
>> http://www.spinics.net/lists/arm-kernel/msg341884.html
>>
>>  arch/arm/include/asm/mcpm.h          |    6 ++++
>>  arch/arm/mach-exynos/mcpm-exynos.c   |   50 ++++++++++++++++++++++++----------
>>  arch/arm/mach-exynos/pm.c            |   38 ++++++++++++++++++++++++--
>>  arch/arm/mach-exynos/regs-pmu.h      |    1 +
>>  drivers/cpuidle/cpuidle-big_little.c |    2 +-
>>  5 files changed, 79 insertions(+), 18 deletions(-)
>>
>> diff --git a/arch/arm/include/asm/mcpm.h b/arch/arm/include/asm/mcpm.h
>> index ff73aff..051fbf1 100644
>> --- a/arch/arm/include/asm/mcpm.h
>> +++ b/arch/arm/include/asm/mcpm.h
>> @@ -272,4 +272,10 @@ void __init mcpm_smp_set_ops(void);
>>  #define MCPM_SYNC_CLUSTER_SIZE \
>>       (MCPM_SYNC_CLUSTER_INBOUND + __CACHE_WRITEBACK_GRANULE)
>>
>> +/* Definitions for various MCPM scenarios that might need special handling */
>> +#define MCPM_CPU_IDLE                0x0
>> +#define MCPM_CPU_SUSPEND     0x1
>> +#define MCPM_CPU_SWITCH              0x2
>> +#define MCPM_CPU_HOTPLUG     0x3
>
> Please, let's avoid going that route.  There is no such special handling
> needed if the API is sufficient.  And the provided API allows you to
> suspend a CPU or shut it down.  It shouldn't matter at that level if
> this is due to a cluster switch or a hotplug event. Do you really need
> something else?
No, just one local flag for suspend should be enough for me. Will remove these.
>
> [...]
>> @@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
>>   * and can only be executed on processors like A15 and A7 that hit the cache
>>   * with the C bit clear in the SCTLR register.
>>   */
>> -static void exynos_power_down(void)
>> +static void exynos_mcpm_power_down(u64 residency)
>>  {
>>       unsigned int mpidr, cpu, cluster;
>>       bool last_man = false, skip_wfi = false;
>> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>>       cpu_use_count[cpu][cluster]--;
>>       if (cpu_use_count[cpu][cluster] == 0) {
>> -             exynos_cpu_power_down(cpunr);
>> +             /*
>> +              * Bypass power down for CPU0 during suspend. This is being
>> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> +              */
>> +             if ((cpunr != 0) || (residency != MCPM_CPU_SUSPEND))
>> +                     exynos_cpu_power_down(cpunr);
>>
>>               if (exynos_cluster_unused(cluster)) {
>>                       exynos_cluster_power_down(cluster);
>> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>>       /* Not dead at this point?  Let our caller cope. */
>>  }
>>
>> +static void exynos_power_down(void)
>> +{
>> +     exynos_mcpm_power_down(MCPM_CPU_SWITCH | MCPM_CPU_HOTPLUG);
>> +}
>
> To distinguish between a suspend and a power-down, you can simply use
> exynos_power_down() as your common handler, and have
> exynos_mcpm_power_down() and exynos_mcpm_suspend() as wrappers around it
> passing the appropriate private flags with local meanings.
My patch is in line with your explanation, except it makes use of the
new MCPM flags. I will clean those out and just use one flag for
suspend instead. Please check the v5 patch that I will post in a bit
in case I have missed something.

Regards,
Abhilash
>
>
> Nicolas
>

[PATCH v5] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Changes in v2:
	- Made use of the MCPM suspend/powered_up call-backs
Changes in v3:
	- Used the residency value to indicate the entered state
Changes in v4:
	- Checked if MCPM has been enabled to prevent build error
Changes in v5:
	- Removed the MCPM flags and just used a local flag to
	indicate that we are suspending.

This has been tested both on an SMDK5420 and Peach Pit Chromebook on
3.16-rc3/next-20140702.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) MCPM boot CPU CCI enablement patches
www.spinics.net/lists/linux-samsung-soc/msg32923.html

5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

6) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

7) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html
 
 arch/arm/mach-exynos/mcpm-exynos.c | 50 +++++++++++++++++++++++++++-----------
 arch/arm/mach-exynos/pm.c          | 37 +++++++++++++++++++++++++---
 arch/arm/mach-exynos/regs-pmu.h    |  1 +
 3 files changed, 71 insertions(+), 17 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 2dd51cc..60f84c9 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
  * and can only be executed on processors like A15 and A7 that hit the cache
  * with the C bit clear in the SCTLR register.
  */
-static void exynos_power_down(void)
+static void exynos_mcpm_power_down(u64 residency)
 {
 	unsigned int mpidr, cpu, cluster;
 	bool last_man = false, skip_wfi = false;
@@ -150,7 +153,12 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. This is being
+		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
+		 */
+		if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -209,6 +217,11 @@ static void exynos_power_down(void)
 	/* Not dead at this point?  Let our caller cope. */
 }
 
+static void exynos_power_down(void)
+{
+	exynos_mcpm_power_down(0);
+}
+
 static int exynos_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
 {
 	unsigned int tries = 100;
@@ -250,11 +263,11 @@ static void exynos_suspend(u64 residency)
 {
 	unsigned int mpidr, cpunr;
 
-	exynos_power_down();
+	exynos_mcpm_power_down(residency);
 
 	/*
 	 * Execution reaches here only if cpu did not power down.
-	 * Hence roll back the changes done in exynos_power_down function.
+	 * Hence roll back the changes done in exynos_mcpm_power_down function.
 	 *
 	 * CAUTION: "This function requires the stack data to be visible through
 	 * power down and can only be executed on processors like A15 and A7
@@ -319,10 +332,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +418,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index 69cf678..278f204 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -191,7 +192,6 @@ int exynos_cluster_power_state(int cluster)
 			pmu_base_addr + S5P_INFORM1))
 
 #define S5P_CHECK_AFTR  0xFCBA0D10
-#define S5P_CHECK_SLEEP 0x00000BAD
 
 /* Ext-GIC nIRQ/nFIQ is the only wakeup source in AFTR */
 static void exynos_set_wakeupmask(long mask)
@@ -318,7 +318,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -490,6 +493,27 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
+	 * we are suspending the system and need to skip CPU0 power down.
+	 */
+	mcpm_cpu_suspend(S5P_CHECK_SLEEP);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +541,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index 3cf0454..e8c75db 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -152,6 +152,7 @@
 #define S5P_PAD_RET_EBIB_OPTION			0x31A8
 
 #define S5P_CORE_LOCAL_PWR_EN			0x3
+#define S5P_CHECK_SLEEP				0x00000BAD
 
 /* Only for EXYNOS4210 */
 #define S5P_CMU_CLKSTOP_LCD1_LOWPWR	0x1154
-- 
2.0.0

[PATCH v5] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Changes in v2:
	- Made use of the MCPM suspend/powered_up call-backs
Changes in v3:
	- Used the residency value to indicate the entered state
Changes in v4:
	- Checked if MCPM has been enabled to prevent build error
Changes in v5:
	- Removed the MCPM flags and just used a local flag to
	indicate that we are suspending.

This has been tested both on an SMDK5420 and Peach Pit Chromebook on
3.16-rc3/next-20140702.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) MCPM boot CPU CCI enablement patches
www.spinics.net/lists/linux-samsung-soc/msg32923.html

5) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

6) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

7) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html
 
 arch/arm/mach-exynos/mcpm-exynos.c | 50 +++++++++++++++++++++++++++-----------
 arch/arm/mach-exynos/pm.c          | 37 +++++++++++++++++++++++++---
 arch/arm/mach-exynos/regs-pmu.h    |  1 +
 3 files changed, 71 insertions(+), 17 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 2dd51cc..60f84c9 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -129,7 +132,7 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
  * and can only be executed on processors like A15 and A7 that hit the cache
  * with the C bit clear in the SCTLR register.
  */
-static void exynos_power_down(void)
+static void exynos_mcpm_power_down(u64 residency)
 {
 	unsigned int mpidr, cpu, cluster;
 	bool last_man = false, skip_wfi = false;
@@ -150,7 +153,12 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. This is being
+		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
+		 */
+		if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -209,6 +217,11 @@ static void exynos_power_down(void)
 	/* Not dead at this point?  Let our caller cope. */
 }
 
+static void exynos_power_down(void)
+{
+	exynos_mcpm_power_down(0);
+}
+
 static int exynos_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
 {
 	unsigned int tries = 100;
@@ -250,11 +263,11 @@ static void exynos_suspend(u64 residency)
 {
 	unsigned int mpidr, cpunr;
 
-	exynos_power_down();
+	exynos_mcpm_power_down(residency);
 
 	/*
 	 * Execution reaches here only if cpu did not power down.
-	 * Hence roll back the changes done in exynos_power_down function.
+	 * Hence roll back the changes done in exynos_mcpm_power_down function.
 	 *
 	 * CAUTION: "This function requires the stack data to be visible through
 	 * power down and can only be executed on processors like A15 and A7
@@ -319,10 +332,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +418,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index 69cf678..278f204 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -191,7 +192,6 @@ int exynos_cluster_power_state(int cluster)
 			pmu_base_addr + S5P_INFORM1))
 
 #define S5P_CHECK_AFTR  0xFCBA0D10
-#define S5P_CHECK_SLEEP 0x00000BAD
 
 /* Ext-GIC nIRQ/nFIQ is the only wakeup source in AFTR */
 static void exynos_set_wakeupmask(long mask)
@@ -318,7 +318,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -490,6 +493,27 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
+	 * we are suspending the system and need to skip CPU0 power down.
+	 */
+	mcpm_cpu_suspend(S5P_CHECK_SLEEP);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +541,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
diff --git a/arch/arm/mach-exynos/regs-pmu.h b/arch/arm/mach-exynos/regs-pmu.h
index 3cf0454..e8c75db 100644
--- a/arch/arm/mach-exynos/regs-pmu.h
+++ b/arch/arm/mach-exynos/regs-pmu.h
@@ -152,6 +152,7 @@
 #define S5P_PAD_RET_EBIB_OPTION			0x31A8
 
 #define S5P_CORE_LOCAL_PWR_EN			0x3
+#define S5P_CHECK_SLEEP				0x00000BAD
 
 /* Only for EXYNOS4210 */
 #define S5P_CMU_CLKSTOP_LCD1_LOWPWR	0x1154
-- 
2.0.0

Re: several messages

[Nicolas Pitre]

On Thu, 3 Jul 2014, Abhilash Kesavan wrote:

> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > Please, let's avoid going that route.  There is no such special handling
> > needed if the API is sufficient.  And the provided API allows you to
> > suspend a CPU or shut it down.  It shouldn't matter at that level if
> > this is due to a cluster switch or a hotplug event. Do you really need
> > something else?
> No, just one local flag for suspend should be enough for me. Will remove these.

[...]

> Changes in v5:
> 	- Removed the MCPM flags and just used a local flag to
> 	indicate that we are suspending.

[...]

> -static void exynos_power_down(void)
> +static void exynos_mcpm_power_down(u64 residency)
>  {
>  	unsigned int mpidr, cpu, cluster;
>  	bool last_man = false, skip_wfi = false;
> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>  	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>  	cpu_use_count[cpu][cluster]--;
>  	if (cpu_use_count[cpu][cluster] == 0) {
> -		exynos_cpu_power_down(cpunr);
> +		/*
> +		 * Bypass power down for CPU0 during suspend. This is being
> +		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> +		 */
> +		if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
> +			exynos_cpu_power_down(cpunr);
>  
>  		if (exynos_cluster_unused(cluster)) {
>  			exynos_cluster_power_down(cluster);
> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>  	/* Not dead at this point?  Let our caller cope. */
>  }
>  
> +static void exynos_power_down(void)
> +{
> +	exynos_mcpm_power_down(0);
> +}

[...]

> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> +{
> +	/* MCPM works with HW CPU identifiers */
> +	unsigned int mpidr = read_cpuid_mpidr();
> +	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +
> +	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> +
> +	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> +
> +	/*
> +	 * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
> +	 * we are suspending the system and need to skip CPU0 power down.
> +	 */
> +	mcpm_cpu_suspend(S5P_CHECK_SLEEP);

NAK.

When I say "local flag with local meaning", I don't want you to smuggle 
that flag through a public interface either.  I find it rather inelegant 
to have the notion of special handling for CPU0 being spread around like 
that.

If CPU0 is special, then it should be dealth with in one place only, 
ideally in the MCPM backend, so the rest of the kernel doesn't have to 
care.

Again, here's what I mean:

static void exynos_mcpm_down_handler(int flags)
{
	[...]
	if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
		exynos_cpu_power_down(cpunr);
	[...]
}

static void exynos_mcpm_power_down()
{
	exynos_mcpm_down_handler(0);
}

static void exynos_mcpm_suspend(u64 residency)
{
	/*
	 * Theresidency argument is ignored for now.
	 * However, in the CPU suspend case, we bypass power down for 
	 * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
	 * CORE0_SYS_PWR_REG.
	 */
	exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
}

And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to 
mcpm-exynos.c only.


Nicolas

several messages

[Nicolas Pitre]

On Thu, 3 Jul 2014, Abhilash Kesavan wrote:

> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > Please, let's avoid going that route.  There is no such special handling
> > needed if the API is sufficient.  And the provided API allows you to
> > suspend a CPU or shut it down.  It shouldn't matter at that level if
> > this is due to a cluster switch or a hotplug event. Do you really need
> > something else?
> No, just one local flag for suspend should be enough for me. Will remove these.

[...]

> Changes in v5:
> 	- Removed the MCPM flags and just used a local flag to
> 	indicate that we are suspending.

[...]

> -static void exynos_power_down(void)
> +static void exynos_mcpm_power_down(u64 residency)
>  {
>  	unsigned int mpidr, cpu, cluster;
>  	bool last_man = false, skip_wfi = false;
> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>  	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>  	cpu_use_count[cpu][cluster]--;
>  	if (cpu_use_count[cpu][cluster] == 0) {
> -		exynos_cpu_power_down(cpunr);
> +		/*
> +		 * Bypass power down for CPU0 during suspend. This is being
> +		 * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> +		 */
> +		if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
> +			exynos_cpu_power_down(cpunr);
>  
>  		if (exynos_cluster_unused(cluster)) {
>  			exynos_cluster_power_down(cluster);
> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>  	/* Not dead at this point?  Let our caller cope. */
>  }
>  
> +static void exynos_power_down(void)
> +{
> +	exynos_mcpm_power_down(0);
> +}

[...]

> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> +{
> +	/* MCPM works with HW CPU identifiers */
> +	unsigned int mpidr = read_cpuid_mpidr();
> +	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +
> +	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> +
> +	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> +
> +	/*
> +	 * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
> +	 * we are suspending the system and need to skip CPU0 power down.
> +	 */
> +	mcpm_cpu_suspend(S5P_CHECK_SLEEP);

NAK.

When I say "local flag with local meaning", I don't want you to smuggle 
that flag through a public interface either.  I find it rather inelegant 
to have the notion of special handling for CPU0 being spread around like 
that.

If CPU0 is special, then it should be dealth with in one place only, 
ideally in the MCPM backend, so the rest of the kernel doesn't have to 
care.

Again, here's what I mean:

static void exynos_mcpm_down_handler(int flags)
{
	[...]
	if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
		exynos_cpu_power_down(cpunr);
	[...]
}

static void exynos_mcpm_power_down()
{
	exynos_mcpm_down_handler(0);
}

static void exynos_mcpm_suspend(u64 residency)
{
	/*
	 * Theresidency argument is ignored for now.
	 * However, in the CPU suspend case, we bypass power down for 
	 * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
	 * CORE0_SYS_PWR_REG.
	 */
	exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
}

And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to 
mcpm-exynos.c only.


Nicolas

Re: several messages

[Abhilash Kesavan]

Hi Nicolas,

On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>
>> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > Please, let's avoid going that route.  There is no such special handling
>> > needed if the API is sufficient.  And the provided API allows you to
>> > suspend a CPU or shut it down.  It shouldn't matter at that level if
>> > this is due to a cluster switch or a hotplug event. Do you really need
>> > something else?
>> No, just one local flag for suspend should be enough for me. Will remove these.
>
> [...]
>
>> Changes in v5:
>>       - Removed the MCPM flags and just used a local flag to
>>       indicate that we are suspending.
>
> [...]
>
>> -static void exynos_power_down(void)
>> +static void exynos_mcpm_power_down(u64 residency)
>>  {
>>       unsigned int mpidr, cpu, cluster;
>>       bool last_man = false, skip_wfi = false;
>> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>>       cpu_use_count[cpu][cluster]--;
>>       if (cpu_use_count[cpu][cluster] == 0) {
>> -             exynos_cpu_power_down(cpunr);
>> +             /*
>> +              * Bypass power down for CPU0 during suspend. This is being
>> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> +              */
>> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
>> +                     exynos_cpu_power_down(cpunr);
>>
>>               if (exynos_cluster_unused(cluster)) {
>>                       exynos_cluster_power_down(cluster);
>> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>>       /* Not dead at this point?  Let our caller cope. */
>>  }
>>
>> +static void exynos_power_down(void)
>> +{
>> +     exynos_mcpm_power_down(0);
>> +}
>
> [...]
>
>> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> +{
>> +     /* MCPM works with HW CPU identifiers */
>> +     unsigned int mpidr = read_cpuid_mpidr();
>> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +
>> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> +
>> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> +
>> +     /*
>> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
>> +      * we are suspending the system and need to skip CPU0 power down.
>> +      */
>> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
>
> NAK.
>
> When I say "local flag with local meaning", I don't want you to smuggle
> that flag through a public interface either.  I find it rather inelegant
> to have the notion of special handling for CPU0 being spread around like
> that.
>
> If CPU0 is special, then it should be dealth with in one place only,
> ideally in the MCPM backend, so the rest of the kernel doesn't have to
> care.
>
> Again, here's what I mean:
>
> static void exynos_mcpm_down_handler(int flags)
> {
>         [...]
>         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
>                 exynos_cpu_power_down(cpunr);
>         [...]
> }
>
> static void exynos_mcpm_power_down()
> {
>         exynos_mcpm_down_handler(0);
> }
>
> static void exynos_mcpm_suspend(u64 residency)
> {
>         /*
>          * Theresidency argument is ignored for now.
>          * However, in the CPU suspend case, we bypass power down for
>          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
>          * CORE0_SYS_PWR_REG.
>          */
>         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
> }
>
> And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
> mcpm-exynos.c only.
Sorry if I am being dense, but the exynos_mcpm_suspend function would
get called from both the bL cpuidle driver as well the exynos pm code.
We want to skip CPU0 only in case of the S2R case i.e. the pm code and
keep the CPU0 power down code for all other cases including CPUIdle.
If I call exynos_mcpm_down_handler with the flag in
exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?

Regards,
Abhilash
>
>
> Nicolas

several messages

[Abhilash Kesavan]

Hi Nicolas,

On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>
>> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > Please, let's avoid going that route.  There is no such special handling
>> > needed if the API is sufficient.  And the provided API allows you to
>> > suspend a CPU or shut it down.  It shouldn't matter at that level if
>> > this is due to a cluster switch or a hotplug event. Do you really need
>> > something else?
>> No, just one local flag for suspend should be enough for me. Will remove these.
>
> [...]
>
>> Changes in v5:
>>       - Removed the MCPM flags and just used a local flag to
>>       indicate that we are suspending.
>
> [...]
>
>> -static void exynos_power_down(void)
>> +static void exynos_mcpm_power_down(u64 residency)
>>  {
>>       unsigned int mpidr, cpu, cluster;
>>       bool last_man = false, skip_wfi = false;
>> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>>       cpu_use_count[cpu][cluster]--;
>>       if (cpu_use_count[cpu][cluster] == 0) {
>> -             exynos_cpu_power_down(cpunr);
>> +             /*
>> +              * Bypass power down for CPU0 during suspend. This is being
>> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> +              */
>> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
>> +                     exynos_cpu_power_down(cpunr);
>>
>>               if (exynos_cluster_unused(cluster)) {
>>                       exynos_cluster_power_down(cluster);
>> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>>       /* Not dead at this point?  Let our caller cope. */
>>  }
>>
>> +static void exynos_power_down(void)
>> +{
>> +     exynos_mcpm_power_down(0);
>> +}
>
> [...]
>
>> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> +{
>> +     /* MCPM works with HW CPU identifiers */
>> +     unsigned int mpidr = read_cpuid_mpidr();
>> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +
>> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> +
>> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> +
>> +     /*
>> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
>> +      * we are suspending the system and need to skip CPU0 power down.
>> +      */
>> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
>
> NAK.
>
> When I say "local flag with local meaning", I don't want you to smuggle
> that flag through a public interface either.  I find it rather inelegant
> to have the notion of special handling for CPU0 being spread around like
> that.
>
> If CPU0 is special, then it should be dealth with in one place only,
> ideally in the MCPM backend, so the rest of the kernel doesn't have to
> care.
>
> Again, here's what I mean:
>
> static void exynos_mcpm_down_handler(int flags)
> {
>         [...]
>         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
>                 exynos_cpu_power_down(cpunr);
>         [...]
> }
>
> static void exynos_mcpm_power_down()
> {
>         exynos_mcpm_down_handler(0);
> }
>
> static void exynos_mcpm_suspend(u64 residency)
> {
>         /*
>          * Theresidency argument is ignored for now.
>          * However, in the CPU suspend case, we bypass power down for
>          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
>          * CORE0_SYS_PWR_REG.
>          */
>         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
> }
>
> And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
> mcpm-exynos.c only.
Sorry if I am being dense, but the exynos_mcpm_suspend function would
get called from both the bL cpuidle driver as well the exynos pm code.
We want to skip CPU0 only in case of the S2R case i.e. the pm code and
keep the CPU0 power down code for all other cases including CPUIdle.
If I call exynos_mcpm_down_handler with the flag in
exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?

Regards,
Abhilash
>
>
> Nicolas

Re: several messages

[Nicolas Pitre]

On Thu, 3 Jul 2014, Abhilash Kesavan wrote:

> Hi Nicolas,
> 
> On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
> >
> >> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> >> > Please, let's avoid going that route.  There is no such special handling
> >> > needed if the API is sufficient.  And the provided API allows you to
> >> > suspend a CPU or shut it down.  It shouldn't matter at that level if
> >> > this is due to a cluster switch or a hotplug event. Do you really need
> >> > something else?
> >> No, just one local flag for suspend should be enough for me. Will remove these.
> >
> > [...]
> >
> >> Changes in v5:
> >>       - Removed the MCPM flags and just used a local flag to
> >>       indicate that we are suspending.
> >
> > [...]
> >
> >> -static void exynos_power_down(void)
> >> +static void exynos_mcpm_power_down(u64 residency)
> >>  {
> >>       unsigned int mpidr, cpu, cluster;
> >>       bool last_man = false, skip_wfi = false;
> >> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> >>       cpu_use_count[cpu][cluster]--;
> >>       if (cpu_use_count[cpu][cluster] == 0) {
> >> -             exynos_cpu_power_down(cpunr);
> >> +             /*
> >> +              * Bypass power down for CPU0 during suspend. This is being
> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> >> +              */
> >> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
> >> +                     exynos_cpu_power_down(cpunr);
> >>
> >>               if (exynos_cluster_unused(cluster)) {
> >>                       exynos_cluster_power_down(cluster);
> >> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
> >>       /* Not dead at this point?  Let our caller cope. */
> >>  }
> >>
> >> +static void exynos_power_down(void)
> >> +{
> >> +     exynos_mcpm_power_down(0);
> >> +}
> >
> > [...]
> >
> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> >> +{
> >> +     /* MCPM works with HW CPU identifiers */
> >> +     unsigned int mpidr = read_cpuid_mpidr();
> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> >> +
> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> >> +
> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> >> +
> >> +     /*
> >> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
> >> +      * we are suspending the system and need to skip CPU0 power down.
> >> +      */
> >> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
> >
> > NAK.
> >
> > When I say "local flag with local meaning", I don't want you to smuggle
> > that flag through a public interface either.  I find it rather inelegant
> > to have the notion of special handling for CPU0 being spread around like
> > that.
> >
> > If CPU0 is special, then it should be dealth with in one place only,
> > ideally in the MCPM backend, so the rest of the kernel doesn't have to
> > care.
> >
> > Again, here's what I mean:
> >
> > static void exynos_mcpm_down_handler(int flags)
> > {
> >         [...]
> >         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
> >                 exynos_cpu_power_down(cpunr);
> >         [...]
> > }
> >
> > static void exynos_mcpm_power_down()
> > {
> >         exynos_mcpm_down_handler(0);
> > }
> >
> > static void exynos_mcpm_suspend(u64 residency)
> > {
> >         /*
> >          * Theresidency argument is ignored for now.
> >          * However, in the CPU suspend case, we bypass power down for
> >          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
> >          * CORE0_SYS_PWR_REG.
> >          */
> >         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
> > }
> >
> > And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
> > mcpm-exynos.c only.
> Sorry if I am being dense, but the exynos_mcpm_suspend function would
> get called from both the bL cpuidle driver as well the exynos pm code.

What is that exynos pm code actually doing?

> We want to skip CPU0 only in case of the S2R case i.e. the pm code and
> keep the CPU0 power down code for all other cases including CPUIdle.

OK.  If so I missed that somehow (hint hint).

> If I call exynos_mcpm_down_handler with the flag in
> exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?

As it is, yes.  You could logically use an infinite residency time 
(something like U64_MAX) to distinguish S2RAM from other types of 
suspend.

Yet, why is this SYS_PWR_CFG bit set outside of MCPM?  Couldn't the MCPM 
backend handle it directly instead of expecting some other entity to do 
it?


Nicolas

several messages

[Nicolas Pitre]

On Thu, 3 Jul 2014, Abhilash Kesavan wrote:

> Hi Nicolas,
> 
> On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
> >
> >> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> >> > Please, let's avoid going that route.  There is no such special handling
> >> > needed if the API is sufficient.  And the provided API allows you to
> >> > suspend a CPU or shut it down.  It shouldn't matter at that level if
> >> > this is due to a cluster switch or a hotplug event. Do you really need
> >> > something else?
> >> No, just one local flag for suspend should be enough for me. Will remove these.
> >
> > [...]
> >
> >> Changes in v5:
> >>       - Removed the MCPM flags and just used a local flag to
> >>       indicate that we are suspending.
> >
> > [...]
> >
> >> -static void exynos_power_down(void)
> >> +static void exynos_mcpm_power_down(u64 residency)
> >>  {
> >>       unsigned int mpidr, cpu, cluster;
> >>       bool last_man = false, skip_wfi = false;
> >> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> >>       cpu_use_count[cpu][cluster]--;
> >>       if (cpu_use_count[cpu][cluster] == 0) {
> >> -             exynos_cpu_power_down(cpunr);
> >> +             /*
> >> +              * Bypass power down for CPU0 during suspend. This is being
> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> >> +              */
> >> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
> >> +                     exynos_cpu_power_down(cpunr);
> >>
> >>               if (exynos_cluster_unused(cluster)) {
> >>                       exynos_cluster_power_down(cluster);
> >> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
> >>       /* Not dead at this point?  Let our caller cope. */
> >>  }
> >>
> >> +static void exynos_power_down(void)
> >> +{
> >> +     exynos_mcpm_power_down(0);
> >> +}
> >
> > [...]
> >
> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> >> +{
> >> +     /* MCPM works with HW CPU identifiers */
> >> +     unsigned int mpidr = read_cpuid_mpidr();
> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> >> +
> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> >> +
> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> >> +
> >> +     /*
> >> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
> >> +      * we are suspending the system and need to skip CPU0 power down.
> >> +      */
> >> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
> >
> > NAK.
> >
> > When I say "local flag with local meaning", I don't want you to smuggle
> > that flag through a public interface either.  I find it rather inelegant
> > to have the notion of special handling for CPU0 being spread around like
> > that.
> >
> > If CPU0 is special, then it should be dealth with in one place only,
> > ideally in the MCPM backend, so the rest of the kernel doesn't have to
> > care.
> >
> > Again, here's what I mean:
> >
> > static void exynos_mcpm_down_handler(int flags)
> > {
> >         [...]
> >         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
> >                 exynos_cpu_power_down(cpunr);
> >         [...]
> > }
> >
> > static void exynos_mcpm_power_down()
> > {
> >         exynos_mcpm_down_handler(0);
> > }
> >
> > static void exynos_mcpm_suspend(u64 residency)
> > {
> >         /*
> >          * Theresidency argument is ignored for now.
> >          * However, in the CPU suspend case, we bypass power down for
> >          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
> >          * CORE0_SYS_PWR_REG.
> >          */
> >         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
> > }
> >
> > And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
> > mcpm-exynos.c only.
> Sorry if I am being dense, but the exynos_mcpm_suspend function would
> get called from both the bL cpuidle driver as well the exynos pm code.

What is that exynos pm code actually doing?

> We want to skip CPU0 only in case of the S2R case i.e. the pm code and
> keep the CPU0 power down code for all other cases including CPUIdle.

OK.  If so I missed that somehow (hint hint).

> If I call exynos_mcpm_down_handler with the flag in
> exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?

As it is, yes.  You could logically use an infinite residency time 
(something like U64_MAX) to distinguish S2RAM from other types of 
suspend.

Yet, why is this SYS_PWR_CFG bit set outside of MCPM?  Couldn't the MCPM 
backend handle it directly instead of expecting some other entity to do 
it?


Nicolas

Re: several messages

[Abhilash Kesavan]

Hi Nicolas,

On Fri, Jul 4, 2014 at 12:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>
>> Hi Nicolas,
>>
>> On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>> >
>> >> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> >> > Please, let's avoid going that route.  There is no such special handling
>> >> > needed if the API is sufficient.  And the provided API allows you to
>> >> > suspend a CPU or shut it down.  It shouldn't matter at that level if
>> >> > this is due to a cluster switch or a hotplug event. Do you really need
>> >> > something else?
>> >> No, just one local flag for suspend should be enough for me. Will remove these.
>> >
>> > [...]
>> >
>> >> Changes in v5:
>> >>       - Removed the MCPM flags and just used a local flag to
>> >>       indicate that we are suspending.
>> >
>> > [...]
>> >
>> >> -static void exynos_power_down(void)
>> >> +static void exynos_mcpm_power_down(u64 residency)
>> >>  {
>> >>       unsigned int mpidr, cpu, cluster;
>> >>       bool last_man = false, skip_wfi = false;
>> >> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>> >>       cpu_use_count[cpu][cluster]--;
>> >>       if (cpu_use_count[cpu][cluster] == 0) {
>> >> -             exynos_cpu_power_down(cpunr);
>> >> +             /*
>> >> +              * Bypass power down for CPU0 during suspend. This is being
>> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> >> +              */
>> >> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
>> >> +                     exynos_cpu_power_down(cpunr);
>> >>
>> >>               if (exynos_cluster_unused(cluster)) {
>> >>                       exynos_cluster_power_down(cluster);
>> >> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>> >>       /* Not dead at this point?  Let our caller cope. */
>> >>  }
>> >>
>> >> +static void exynos_power_down(void)
>> >> +{
>> >> +     exynos_mcpm_power_down(0);
>> >> +}
>> >
>> > [...]
>> >
>> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> >> +{
>> >> +     /* MCPM works with HW CPU identifiers */
>> >> +     unsigned int mpidr = read_cpuid_mpidr();
>> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> >> +
>> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> >> +
>> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> >> +
>> >> +     /*
>> >> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
>> >> +      * we are suspending the system and need to skip CPU0 power down.
>> >> +      */
>> >> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
>> >
>> > NAK.
>> >
>> > When I say "local flag with local meaning", I don't want you to smuggle
>> > that flag through a public interface either.  I find it rather inelegant
>> > to have the notion of special handling for CPU0 being spread around like
>> > that.
>> >
>> > If CPU0 is special, then it should be dealth with in one place only,
>> > ideally in the MCPM backend, so the rest of the kernel doesn't have to
>> > care.
>> >
>> > Again, here's what I mean:
>> >
>> > static void exynos_mcpm_down_handler(int flags)
>> > {
>> >         [...]
>> >         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
>> >                 exynos_cpu_power_down(cpunr);
>> >         [...]
>> > }
>> >
>> > static void exynos_mcpm_power_down()
>> > {
>> >         exynos_mcpm_down_handler(0);
>> > }
>> >
>> > static void exynos_mcpm_suspend(u64 residency)
>> > {
>> >         /*
>> >          * Theresidency argument is ignored for now.
>> >          * However, in the CPU suspend case, we bypass power down for
>> >          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
>> >          * CORE0_SYS_PWR_REG.
>> >          */
>> >         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
>> > }
>> >
>> > And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
>> > mcpm-exynos.c only.
>> Sorry if I am being dense, but the exynos_mcpm_suspend function would
>> get called from both the bL cpuidle driver as well the exynos pm code.
>
> What is that exynos pm code actually doing?
exynos pm code is shared across Exynos4 and 5 SoCs. It primarily does
a series of register configurations which are required to put the
system to sleep (some parts of these are SoC specific and others
common). It also populates the suspend_ops for exynos. In the current
patch, exynos_suspend_enter() is where I have plugged in the
mcpm_cpu_suspend call.

This patch is based on the S2R series for 5420
(http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898), you
may also have a look at that for a clearer picture.
>
>> We want to skip CPU0 only in case of the S2R case i.e. the pm code and
>> keep the CPU0 power down code for all other cases including CPUIdle.
>
> OK.  If so I missed that somehow (hint hint).
>
>> If I call exynos_mcpm_down_handler with the flag in
>> exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?
>
> As it is, yes.  You could logically use an infinite residency time
> (something like U64_MAX) to distinguish S2RAM from other types of
> suspend.
OK, I will use this rather than the S5P_CHECK_SLEEP macro.
>
> Yet, why is this SYS_PWR_CFG bit set outside of MCPM?  Couldn't the MCPM
> backend handle it directly instead of expecting some other entity to do
> it?
Low power modes such as Sleep, Low Power Audio, AFTR (ARM Off Top
Running) require a series of register configurations as specified by
the UM to enter/exit them. All the exynos SoCs including 5420, do such
configurations (including sys_pwr_reg setup) as part of the
exynos_pm_prepare function in pm.c and so we just need to skip the cpu
power down.

Regards,
Abhilash
>
>
> Nicolas

several messages

[Abhilash Kesavan]

Hi Nicolas,

On Fri, Jul 4, 2014 at 12:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>
>> Hi Nicolas,
>>
>> On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>> >
>> >> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> >> > Please, let's avoid going that route.  There is no such special handling
>> >> > needed if the API is sufficient.  And the provided API allows you to
>> >> > suspend a CPU or shut it down.  It shouldn't matter at that level if
>> >> > this is due to a cluster switch or a hotplug event. Do you really need
>> >> > something else?
>> >> No, just one local flag for suspend should be enough for me. Will remove these.
>> >
>> > [...]
>> >
>> >> Changes in v5:
>> >>       - Removed the MCPM flags and just used a local flag to
>> >>       indicate that we are suspending.
>> >
>> > [...]
>> >
>> >> -static void exynos_power_down(void)
>> >> +static void exynos_mcpm_power_down(u64 residency)
>> >>  {
>> >>       unsigned int mpidr, cpu, cluster;
>> >>       bool last_man = false, skip_wfi = false;
>> >> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>> >>       cpu_use_count[cpu][cluster]--;
>> >>       if (cpu_use_count[cpu][cluster] == 0) {
>> >> -             exynos_cpu_power_down(cpunr);
>> >> +             /*
>> >> +              * Bypass power down for CPU0 during suspend. This is being
>> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> >> +              */
>> >> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
>> >> +                     exynos_cpu_power_down(cpunr);
>> >>
>> >>               if (exynos_cluster_unused(cluster)) {
>> >>                       exynos_cluster_power_down(cluster);
>> >> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>> >>       /* Not dead at this point?  Let our caller cope. */
>> >>  }
>> >>
>> >> +static void exynos_power_down(void)
>> >> +{
>> >> +     exynos_mcpm_power_down(0);
>> >> +}
>> >
>> > [...]
>> >
>> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> >> +{
>> >> +     /* MCPM works with HW CPU identifiers */
>> >> +     unsigned int mpidr = read_cpuid_mpidr();
>> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> >> +
>> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> >> +
>> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> >> +
>> >> +     /*
>> >> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
>> >> +      * we are suspending the system and need to skip CPU0 power down.
>> >> +      */
>> >> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
>> >
>> > NAK.
>> >
>> > When I say "local flag with local meaning", I don't want you to smuggle
>> > that flag through a public interface either.  I find it rather inelegant
>> > to have the notion of special handling for CPU0 being spread around like
>> > that.
>> >
>> > If CPU0 is special, then it should be dealth with in one place only,
>> > ideally in the MCPM backend, so the rest of the kernel doesn't have to
>> > care.
>> >
>> > Again, here's what I mean:
>> >
>> > static void exynos_mcpm_down_handler(int flags)
>> > {
>> >         [...]
>> >         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
>> >                 exynos_cpu_power_down(cpunr);
>> >         [...]
>> > }
>> >
>> > static void exynos_mcpm_power_down()
>> > {
>> >         exynos_mcpm_down_handler(0);
>> > }
>> >
>> > static void exynos_mcpm_suspend(u64 residency)
>> > {
>> >         /*
>> >          * Theresidency argument is ignored for now.
>> >          * However, in the CPU suspend case, we bypass power down for
>> >          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
>> >          * CORE0_SYS_PWR_REG.
>> >          */
>> >         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
>> > }
>> >
>> > And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
>> > mcpm-exynos.c only.
>> Sorry if I am being dense, but the exynos_mcpm_suspend function would
>> get called from both the bL cpuidle driver as well the exynos pm code.
>
> What is that exynos pm code actually doing?
exynos pm code is shared across Exynos4 and 5 SoCs. It primarily does
a series of register configurations which are required to put the
system to sleep (some parts of these are SoC specific and others
common). It also populates the suspend_ops for exynos. In the current
patch, exynos_suspend_enter() is where I have plugged in the
mcpm_cpu_suspend call.

This patch is based on the S2R series for 5420
(http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898), you
may also have a look at that for a clearer picture.
>
>> We want to skip CPU0 only in case of the S2R case i.e. the pm code and
>> keep the CPU0 power down code for all other cases including CPUIdle.
>
> OK.  If so I missed that somehow (hint hint).
>
>> If I call exynos_mcpm_down_handler with the flag in
>> exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?
>
> As it is, yes.  You could logically use an infinite residency time
> (something like U64_MAX) to distinguish S2RAM from other types of
> suspend.
OK, I will use this rather than the S5P_CHECK_SLEEP macro.
>
> Yet, why is this SYS_PWR_CFG bit set outside of MCPM?  Couldn't the MCPM
> backend handle it directly instead of expecting some other entity to do
> it?
Low power modes such as Sleep, Low Power Audio, AFTR (ARM Off Top
Running) require a series of register configurations as specified by
the UM to enter/exit them. All the exynos SoCs including 5420, do such
configurations (including sys_pwr_reg setup) as part of the
exynos_pm_prepare function in pm.c and so we just need to skip the cpu
power down.

Regards,
Abhilash
>
>
> Nicolas

Re: several messages

[Nicolas Pitre]

On Fri, 4 Jul 2014, Abhilash Kesavan wrote:

> Hi Nicolas,
> 
> On Fri, Jul 4, 2014 at 12:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
> >
> >> Hi Nicolas,
> >>
> >> On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> >> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
> >> >
> >> >> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> >> >> > Please, let's avoid going that route.  There is no such special handling
> >> >> > needed if the API is sufficient.  And the provided API allows you to
> >> >> > suspend a CPU or shut it down.  It shouldn't matter at that level if
> >> >> > this is due to a cluster switch or a hotplug event. Do you really need
> >> >> > something else?
> >> >> No, just one local flag for suspend should be enough for me. Will remove these.
> >> >
> >> > [...]
> >> >
> >> >> Changes in v5:
> >> >>       - Removed the MCPM flags and just used a local flag to
> >> >>       indicate that we are suspending.
> >> >
> >> > [...]
> >> >
> >> >> -static void exynos_power_down(void)
> >> >> +static void exynos_mcpm_power_down(u64 residency)
> >> >>  {
> >> >>       unsigned int mpidr, cpu, cluster;
> >> >>       bool last_man = false, skip_wfi = false;
> >> >> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
> >> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> >> >>       cpu_use_count[cpu][cluster]--;
> >> >>       if (cpu_use_count[cpu][cluster] == 0) {
> >> >> -             exynos_cpu_power_down(cpunr);
> >> >> +             /*
> >> >> +              * Bypass power down for CPU0 during suspend. This is being
> >> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> >> >> +              */
> >> >> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
> >> >> +                     exynos_cpu_power_down(cpunr);
> >> >>
> >> >>               if (exynos_cluster_unused(cluster)) {
> >> >>                       exynos_cluster_power_down(cluster);
> >> >> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
> >> >>       /* Not dead at this point?  Let our caller cope. */
> >> >>  }
> >> >>
> >> >> +static void exynos_power_down(void)
> >> >> +{
> >> >> +     exynos_mcpm_power_down(0);
> >> >> +}
> >> >
> >> > [...]
> >> >
> >> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> >> >> +{
> >> >> +     /* MCPM works with HW CPU identifiers */
> >> >> +     unsigned int mpidr = read_cpuid_mpidr();
> >> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> >> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> >> >> +
> >> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> >> >> +
> >> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> >> >> +
> >> >> +     /*
> >> >> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
> >> >> +      * we are suspending the system and need to skip CPU0 power down.
> >> >> +      */
> >> >> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
> >> >
> >> > NAK.
> >> >
> >> > When I say "local flag with local meaning", I don't want you to smuggle
> >> > that flag through a public interface either.  I find it rather inelegant
> >> > to have the notion of special handling for CPU0 being spread around like
> >> > that.
> >> >
> >> > If CPU0 is special, then it should be dealth with in one place only,
> >> > ideally in the MCPM backend, so the rest of the kernel doesn't have to
> >> > care.
> >> >
> >> > Again, here's what I mean:
> >> >
> >> > static void exynos_mcpm_down_handler(int flags)
> >> > {
> >> >         [...]
> >> >         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
> >> >                 exynos_cpu_power_down(cpunr);
> >> >         [...]
> >> > }
> >> >
> >> > static void exynos_mcpm_power_down()
> >> > {
> >> >         exynos_mcpm_down_handler(0);
> >> > }
> >> >
> >> > static void exynos_mcpm_suspend(u64 residency)
> >> > {
> >> >         /*
> >> >          * Theresidency argument is ignored for now.
> >> >          * However, in the CPU suspend case, we bypass power down for
> >> >          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
> >> >          * CORE0_SYS_PWR_REG.
> >> >          */
> >> >         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
> >> > }
> >> >
> >> > And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
> >> > mcpm-exynos.c only.
> >> Sorry if I am being dense, but the exynos_mcpm_suspend function would
> >> get called from both the bL cpuidle driver as well the exynos pm code.
> >
> > What is that exynos pm code actually doing?
> exynos pm code is shared across Exynos4 and 5 SoCs. It primarily does
> a series of register configurations which are required to put the
> system to sleep (some parts of these are SoC specific and others
> common). It also populates the suspend_ops for exynos. In the current
> patch, exynos_suspend_enter() is where I have plugged in the
> mcpm_cpu_suspend call.
> 
> This patch is based on the S2R series for 5420
> (http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898), you
> may also have a look at that for a clearer picture.
> >
> >> We want to skip CPU0 only in case of the S2R case i.e. the pm code and
> >> keep the CPU0 power down code for all other cases including CPUIdle.
> >
> > OK.  If so I missed that somehow (hint hint).
> >
> >> If I call exynos_mcpm_down_handler with the flag in
> >> exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?
> >
> > As it is, yes.  You could logically use an infinite residency time
> > (something like U64_MAX) to distinguish S2RAM from other types of
> > suspend.
> OK, I will use this rather than the S5P_CHECK_SLEEP macro.

Another suggestion which might possibly be better: why not looking for 
the SYS_PWR_CFG bit in exynos_cpu_power_down() directly?  After all, 
exynos_cpu_power_down() is semantically supposed to do what its name 
suggest and could simply do nothing if the proper conditions are already 
in place.


Nicolas

several messages

[Nicolas Pitre]

On Fri, 4 Jul 2014, Abhilash Kesavan wrote:

> Hi Nicolas,
> 
> On Fri, Jul 4, 2014 at 12:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
> >
> >> Hi Nicolas,
> >>
> >> On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> >> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
> >> >
> >> >> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> >> >> > Please, let's avoid going that route.  There is no such special handling
> >> >> > needed if the API is sufficient.  And the provided API allows you to
> >> >> > suspend a CPU or shut it down.  It shouldn't matter at that level if
> >> >> > this is due to a cluster switch or a hotplug event. Do you really need
> >> >> > something else?
> >> >> No, just one local flag for suspend should be enough for me. Will remove these.
> >> >
> >> > [...]
> >> >
> >> >> Changes in v5:
> >> >>       - Removed the MCPM flags and just used a local flag to
> >> >>       indicate that we are suspending.
> >> >
> >> > [...]
> >> >
> >> >> -static void exynos_power_down(void)
> >> >> +static void exynos_mcpm_power_down(u64 residency)
> >> >>  {
> >> >>       unsigned int mpidr, cpu, cluster;
> >> >>       bool last_man = false, skip_wfi = false;
> >> >> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
> >> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
> >> >>       cpu_use_count[cpu][cluster]--;
> >> >>       if (cpu_use_count[cpu][cluster] == 0) {
> >> >> -             exynos_cpu_power_down(cpunr);
> >> >> +             /*
> >> >> +              * Bypass power down for CPU0 during suspend. This is being
> >> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
> >> >> +              */
> >> >> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
> >> >> +                     exynos_cpu_power_down(cpunr);
> >> >>
> >> >>               if (exynos_cluster_unused(cluster)) {
> >> >>                       exynos_cluster_power_down(cluster);
> >> >> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
> >> >>       /* Not dead at this point?  Let our caller cope. */
> >> >>  }
> >> >>
> >> >> +static void exynos_power_down(void)
> >> >> +{
> >> >> +     exynos_mcpm_power_down(0);
> >> >> +}
> >> >
> >> > [...]
> >> >
> >> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> >> >> +{
> >> >> +     /* MCPM works with HW CPU identifiers */
> >> >> +     unsigned int mpidr = read_cpuid_mpidr();
> >> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> >> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> >> >> +
> >> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> >> >> +
> >> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> >> >> +
> >> >> +     /*
> >> >> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
> >> >> +      * we are suspending the system and need to skip CPU0 power down.
> >> >> +      */
> >> >> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
> >> >
> >> > NAK.
> >> >
> >> > When I say "local flag with local meaning", I don't want you to smuggle
> >> > that flag through a public interface either.  I find it rather inelegant
> >> > to have the notion of special handling for CPU0 being spread around like
> >> > that.
> >> >
> >> > If CPU0 is special, then it should be dealth with in one place only,
> >> > ideally in the MCPM backend, so the rest of the kernel doesn't have to
> >> > care.
> >> >
> >> > Again, here's what I mean:
> >> >
> >> > static void exynos_mcpm_down_handler(int flags)
> >> > {
> >> >         [...]
> >> >         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
> >> >                 exynos_cpu_power_down(cpunr);
> >> >         [...]
> >> > }
> >> >
> >> > static void exynos_mcpm_power_down()
> >> > {
> >> >         exynos_mcpm_down_handler(0);
> >> > }
> >> >
> >> > static void exynos_mcpm_suspend(u64 residency)
> >> > {
> >> >         /*
> >> >          * Theresidency argument is ignored for now.
> >> >          * However, in the CPU suspend case, we bypass power down for
> >> >          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
> >> >          * CORE0_SYS_PWR_REG.
> >> >          */
> >> >         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
> >> > }
> >> >
> >> > And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
> >> > mcpm-exynos.c only.
> >> Sorry if I am being dense, but the exynos_mcpm_suspend function would
> >> get called from both the bL cpuidle driver as well the exynos pm code.
> >
> > What is that exynos pm code actually doing?
> exynos pm code is shared across Exynos4 and 5 SoCs. It primarily does
> a series of register configurations which are required to put the
> system to sleep (some parts of these are SoC specific and others
> common). It also populates the suspend_ops for exynos. In the current
> patch, exynos_suspend_enter() is where I have plugged in the
> mcpm_cpu_suspend call.
> 
> This patch is based on the S2R series for 5420
> (http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898), you
> may also have a look at that for a clearer picture.
> >
> >> We want to skip CPU0 only in case of the S2R case i.e. the pm code and
> >> keep the CPU0 power down code for all other cases including CPUIdle.
> >
> > OK.  If so I missed that somehow (hint hint).
> >
> >> If I call exynos_mcpm_down_handler with the flag in
> >> exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?
> >
> > As it is, yes.  You could logically use an infinite residency time
> > (something like U64_MAX) to distinguish S2RAM from other types of
> > suspend.
> OK, I will use this rather than the S5P_CHECK_SLEEP macro.

Another suggestion which might possibly be better: why not looking for 
the SYS_PWR_CFG bit in exynos_cpu_power_down() directly?  After all, 
exynos_cpu_power_down() is semantically supposed to do what its name 
suggest and could simply do nothing if the proper conditions are already 
in place.


Nicolas

Re: several messages

[Abhilash Kesavan]

Hi Nicolas,

On Fri, Jul 4, 2014 at 9:43 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Fri, 4 Jul 2014, Abhilash Kesavan wrote:
>
>> Hi Nicolas,
>>
>> On Fri, Jul 4, 2014 at 12:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>> >
>> >> Hi Nicolas,
>> >>
>> >> On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> >> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>> >> >
>> >> >> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> >> >> > Please, let's avoid going that route.  There is no such special handling
>> >> >> > needed if the API is sufficient.  And the provided API allows you to
>> >> >> > suspend a CPU or shut it down.  It shouldn't matter at that level if
>> >> >> > this is due to a cluster switch or a hotplug event. Do you really need
>> >> >> > something else?
>> >> >> No, just one local flag for suspend should be enough for me. Will remove these.
>> >> >
>> >> > [...]
>> >> >
>> >> >> Changes in v5:
>> >> >>       - Removed the MCPM flags and just used a local flag to
>> >> >>       indicate that we are suspending.
>> >> >
>> >> > [...]
>> >> >
>> >> >> -static void exynos_power_down(void)
>> >> >> +static void exynos_mcpm_power_down(u64 residency)
>> >> >>  {
>> >> >>       unsigned int mpidr, cpu, cluster;
>> >> >>       bool last_man = false, skip_wfi = false;
>> >> >> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>> >> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>> >> >>       cpu_use_count[cpu][cluster]--;
>> >> >>       if (cpu_use_count[cpu][cluster] == 0) {
>> >> >> -             exynos_cpu_power_down(cpunr);
>> >> >> +             /*
>> >> >> +              * Bypass power down for CPU0 during suspend. This is being
>> >> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> >> >> +              */
>> >> >> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
>> >> >> +                     exynos_cpu_power_down(cpunr);
>> >> >>
>> >> >>               if (exynos_cluster_unused(cluster)) {
>> >> >>                       exynos_cluster_power_down(cluster);
>> >> >> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>> >> >>       /* Not dead at this point?  Let our caller cope. */
>> >> >>  }
>> >> >>
>> >> >> +static void exynos_power_down(void)
>> >> >> +{
>> >> >> +     exynos_mcpm_power_down(0);
>> >> >> +}
>> >> >
>> >> > [...]
>> >> >
>> >> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> >> >> +{
>> >> >> +     /* MCPM works with HW CPU identifiers */
>> >> >> +     unsigned int mpidr = read_cpuid_mpidr();
>> >> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> >> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> >> >> +
>> >> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> >> >> +
>> >> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> >> >> +
>> >> >> +     /*
>> >> >> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
>> >> >> +      * we are suspending the system and need to skip CPU0 power down.
>> >> >> +      */
>> >> >> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
>> >> >
>> >> > NAK.
>> >> >
>> >> > When I say "local flag with local meaning", I don't want you to smuggle
>> >> > that flag through a public interface either.  I find it rather inelegant
>> >> > to have the notion of special handling for CPU0 being spread around like
>> >> > that.
>> >> >
>> >> > If CPU0 is special, then it should be dealth with in one place only,
>> >> > ideally in the MCPM backend, so the rest of the kernel doesn't have to
>> >> > care.
>> >> >
>> >> > Again, here's what I mean:
>> >> >
>> >> > static void exynos_mcpm_down_handler(int flags)
>> >> > {
>> >> >         [...]
>> >> >         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
>> >> >                 exynos_cpu_power_down(cpunr);
>> >> >         [...]
>> >> > }
>> >> >
>> >> > static void exynos_mcpm_power_down()
>> >> > {
>> >> >         exynos_mcpm_down_handler(0);
>> >> > }
>> >> >
>> >> > static void exynos_mcpm_suspend(u64 residency)
>> >> > {
>> >> >         /*
>> >> >          * Theresidency argument is ignored for now.
>> >> >          * However, in the CPU suspend case, we bypass power down for
>> >> >          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
>> >> >          * CORE0_SYS_PWR_REG.
>> >> >          */
>> >> >         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
>> >> > }
>> >> >
>> >> > And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
>> >> > mcpm-exynos.c only.
>> >> Sorry if I am being dense, but the exynos_mcpm_suspend function would
>> >> get called from both the bL cpuidle driver as well the exynos pm code.
>> >
>> > What is that exynos pm code actually doing?
>> exynos pm code is shared across Exynos4 and 5 SoCs. It primarily does
>> a series of register configurations which are required to put the
>> system to sleep (some parts of these are SoC specific and others
>> common). It also populates the suspend_ops for exynos. In the current
>> patch, exynos_suspend_enter() is where I have plugged in the
>> mcpm_cpu_suspend call.
>>
>> This patch is based on the S2R series for 5420
>> (http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898), you
>> may also have a look at that for a clearer picture.
>> >
>> >> We want to skip CPU0 only in case of the S2R case i.e. the pm code and
>> >> keep the CPU0 power down code for all other cases including CPUIdle.
>> >
>> > OK.  If so I missed that somehow (hint hint).
>> >
>> >> If I call exynos_mcpm_down_handler with the flag in
>> >> exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?
>> >
>> > As it is, yes.  You could logically use an infinite residency time
>> > (something like U64_MAX) to distinguish S2RAM from other types of
>> > suspend.
>> OK, I will use this rather than the S5P_CHECK_SLEEP macro.
>
> Another suggestion which might possibly be better: why not looking for
> the SYS_PWR_CFG bit in exynos_cpu_power_down() directly?  After all,
> exynos_cpu_power_down() is semantically supposed to do what its name
> suggest and could simply do nothing if the proper conditions are already
> in place.
I have implemented this and it works fine. Patch coming up.

Regards,
Abhilash
>
>
> Nicolas

several messages

[Abhilash Kesavan]

Hi Nicolas,

On Fri, Jul 4, 2014 at 9:43 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Fri, 4 Jul 2014, Abhilash Kesavan wrote:
>
>> Hi Nicolas,
>>
>> On Fri, Jul 4, 2014 at 12:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>> >
>> >> Hi Nicolas,
>> >>
>> >> On Thu, Jul 3, 2014 at 9:15 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> >> > On Thu, 3 Jul 2014, Abhilash Kesavan wrote:
>> >> >
>> >> >> On Thu, Jul 3, 2014 at 6:59 PM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> >> >> > Please, let's avoid going that route.  There is no such special handling
>> >> >> > needed if the API is sufficient.  And the provided API allows you to
>> >> >> > suspend a CPU or shut it down.  It shouldn't matter at that level if
>> >> >> > this is due to a cluster switch or a hotplug event. Do you really need
>> >> >> > something else?
>> >> >> No, just one local flag for suspend should be enough for me. Will remove these.
>> >> >
>> >> > [...]
>> >> >
>> >> >> Changes in v5:
>> >> >>       - Removed the MCPM flags and just used a local flag to
>> >> >>       indicate that we are suspending.
>> >> >
>> >> > [...]
>> >> >
>> >> >> -static void exynos_power_down(void)
>> >> >> +static void exynos_mcpm_power_down(u64 residency)
>> >> >>  {
>> >> >>       unsigned int mpidr, cpu, cluster;
>> >> >>       bool last_man = false, skip_wfi = false;
>> >> >> @@ -150,7 +153,12 @@ static void exynos_power_down(void)
>> >> >>       BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
>> >> >>       cpu_use_count[cpu][cluster]--;
>> >> >>       if (cpu_use_count[cpu][cluster] == 0) {
>> >> >> -             exynos_cpu_power_down(cpunr);
>> >> >> +             /*
>> >> >> +              * Bypass power down for CPU0 during suspend. This is being
>> >> >> +              * taken care by the SYS_PWR_CFG bit in CORE0_SYS_PWR_REG.
>> >> >> +              */
>> >> >> +             if ((cpunr != 0) || (residency != S5P_CHECK_SLEEP))
>> >> >> +                     exynos_cpu_power_down(cpunr);
>> >> >>
>> >> >>               if (exynos_cluster_unused(cluster)) {
>> >> >>                       exynos_cluster_power_down(cluster);
>> >> >> @@ -209,6 +217,11 @@ static void exynos_power_down(void)
>> >> >>       /* Not dead at this point?  Let our caller cope. */
>> >> >>  }
>> >> >>
>> >> >> +static void exynos_power_down(void)
>> >> >> +{
>> >> >> +     exynos_mcpm_power_down(0);
>> >> >> +}
>> >> >
>> >> > [...]
>> >> >
>> >> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> >> >> +{
>> >> >> +     /* MCPM works with HW CPU identifiers */
>> >> >> +     unsigned int mpidr = read_cpuid_mpidr();
>> >> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> >> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> >> >> +
>> >> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> >> >> +
>> >> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> >> >> +
>> >> >> +     /*
>> >> >> +      * Pass S5P_CHECK_SLEEP flag to the MCPM back-end to indicate that
>> >> >> +      * we are suspending the system and need to skip CPU0 power down.
>> >> >> +      */
>> >> >> +     mcpm_cpu_suspend(S5P_CHECK_SLEEP);
>> >> >
>> >> > NAK.
>> >> >
>> >> > When I say "local flag with local meaning", I don't want you to smuggle
>> >> > that flag through a public interface either.  I find it rather inelegant
>> >> > to have the notion of special handling for CPU0 being spread around like
>> >> > that.
>> >> >
>> >> > If CPU0 is special, then it should be dealth with in one place only,
>> >> > ideally in the MCPM backend, so the rest of the kernel doesn't have to
>> >> > care.
>> >> >
>> >> > Again, here's what I mean:
>> >> >
>> >> > static void exynos_mcpm_down_handler(int flags)
>> >> > {
>> >> >         [...]
>> >> >         if ((cpunr != 0) || !(flags & SKIP_CPU_POWERDOWN_IF_CPU0))
>> >> >                 exynos_cpu_power_down(cpunr);
>> >> >         [...]
>> >> > }
>> >> >
>> >> > static void exynos_mcpm_power_down()
>> >> > {
>> >> >         exynos_mcpm_down_handler(0);
>> >> > }
>> >> >
>> >> > static void exynos_mcpm_suspend(u64 residency)
>> >> > {
>> >> >         /*
>> >> >          * Theresidency argument is ignored for now.
>> >> >          * However, in the CPU suspend case, we bypass power down for
>> >> >          * CPU0 as this is being taken care by the SYS_PWR_CFG bit in
>> >> >          * CORE0_SYS_PWR_REG.
>> >> >          */
>> >> >         exynos_mcpm_down_handler(SKIP_CPU_POWERDOWN_IF_CPU0);
>> >> > }
>> >> >
>> >> > And SKIP_CPU_POWERDOWN_IF_CPU0 is defined in and visible to
>> >> > mcpm-exynos.c only.
>> >> Sorry if I am being dense, but the exynos_mcpm_suspend function would
>> >> get called from both the bL cpuidle driver as well the exynos pm code.
>> >
>> > What is that exynos pm code actually doing?
>> exynos pm code is shared across Exynos4 and 5 SoCs. It primarily does
>> a series of register configurations which are required to put the
>> system to sleep (some parts of these are SoC specific and others
>> common). It also populates the suspend_ops for exynos. In the current
>> patch, exynos_suspend_enter() is where I have plugged in the
>> mcpm_cpu_suspend call.
>>
>> This patch is based on the S2R series for 5420
>> (http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898), you
>> may also have a look at that for a clearer picture.
>> >
>> >> We want to skip CPU0 only in case of the S2R case i.e. the pm code and
>> >> keep the CPU0 power down code for all other cases including CPUIdle.
>> >
>> > OK.  If so I missed that somehow (hint hint).
>> >
>> >> If I call exynos_mcpm_down_handler with the flag in
>> >> exynos_mcpm_suspend(), CPUIdle will also skip CPU0 isn't it ?
>> >
>> > As it is, yes.  You could logically use an infinite residency time
>> > (something like U64_MAX) to distinguish S2RAM from other types of
>> > suspend.
>> OK, I will use this rather than the S5P_CHECK_SLEEP macro.
>
> Another suggestion which might possibly be better: why not looking for
> the SYS_PWR_CFG bit in exynos_cpu_power_down() directly?  After all,
> exynos_cpu_power_down() is semantically supposed to do what its name
> suggest and could simply do nothing if the proper conditions are already
> in place.
I have implemented this and it works fine. Patch coming up.

Regards,
Abhilash
>
>
> Nicolas

[PATCH v6] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Changes in v2:
	- Made use of the MCPM suspend/powered_up call-backs
Changes in v3:
	- Used the residency value to indicate the entered state
Changes in v4:
	- Checked if MCPM has been enabled to prevent build error
Changes in v5:
	- Removed the MCPM flags and just used a local flag to
	indicate that we are suspending.
Changes in v6:
	- Read the SYS_PWR_REG value to decide if we are suspending
	the system.
	- Restore the SYS_PWR_REG value post-resume.
	- Modified the comments to reflect the first change.

This has been tested both on an SMDK5420 and Peach Pit Chromebook on
next-20140704. Nicolas' boot cluster CCI enablement patches are in
linux-next now.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

5) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

6) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html
 
 arch/arm/mach-exynos/mcpm-exynos.c |   55 ++++++++++++++++++++++++++----------
 arch/arm/mach-exynos/pm.c          |   42 +++++++++++++++++++++++++--
 2 files changed, 80 insertions(+), 17 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 2dd51cc..26670e7 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -129,11 +132,11 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
  * and can only be executed on processors like A15 and A7 that hit the cache
  * with the C bit clear in the SCTLR register.
  */
-static void exynos_power_down(void)
+static void exynos_mcpm_power_down(u64 residency)
 {
 	unsigned int mpidr, cpu, cluster;
 	bool last_man = false, skip_wfi = false;
-	unsigned int cpunr;
+	unsigned int cpunr, temp;
 
 	mpidr = read_cpuid_mpidr();
 	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
@@ -150,7 +153,15 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. Check for
+		 * the SYS_PWR_REG value to decide if we are suspending
+		 * the system.
+		 */
+		temp = __raw_readl(pmu_base_addr +
+			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+		if ((cpu != 0) || ((temp & S5P_CORE_LOCAL_PWR_EN) != 0))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -209,6 +220,11 @@ static void exynos_power_down(void)
 	/* Not dead at this point?  Let our caller cope. */
 }
 
+static void exynos_power_down(void)
+{
+	exynos_mcpm_power_down(0);
+}
+
 static int exynos_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
 {
 	unsigned int tries = 100;
@@ -250,11 +266,11 @@ static void exynos_suspend(u64 residency)
 {
 	unsigned int mpidr, cpunr;
 
-	exynos_power_down();
+	exynos_mcpm_power_down(residency);
 
 	/*
 	 * Execution reaches here only if cpu did not power down.
-	 * Hence roll back the changes done in exynos_power_down function.
+	 * Hence roll back the changes done in exynos_mcpm_power_down function.
 	 *
 	 * CAUTION: "This function requires the stack data to be visible through
 	 * power down and can only be executed on processors like A15 and A7
@@ -319,10 +335,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +421,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index 69cf678..d2751f4 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -318,7 +319,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -408,6 +412,11 @@ static void exynos_pm_resume(void)
 	unsigned int tmp;
 
 	if (soc_is_exynos5420()) {
+		/* Restore the CPU0 low power state register */
+		pmu_raw_writel(
+		__raw_readl(pmu_base_addr + EXYNOS5_ARM_CORE0_SYS_PWR_REG) |
+		S5P_CORE_LOCAL_PWR_EN, EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+
 		/* Restore the sysram cpu state register */
 		__raw_writel(exynos5420_cpu_state,
 		sysram_base_addr + EXYNOS5420_CPU_STATE);
@@ -490,6 +499,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(0);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +548,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
-- 
1.7.9.5

[PATCH v6] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Changes in v2:
	- Made use of the MCPM suspend/powered_up call-backs
Changes in v3:
	- Used the residency value to indicate the entered state
Changes in v4:
	- Checked if MCPM has been enabled to prevent build error
Changes in v5:
	- Removed the MCPM flags and just used a local flag to
	indicate that we are suspending.
Changes in v6:
	- Read the SYS_PWR_REG value to decide if we are suspending
	the system.
	- Restore the SYS_PWR_REG value post-resume.
	- Modified the comments to reflect the first change.

This has been tested both on an SMDK5420 and Peach Pit Chromebook on
next-20140704. Nicolas' boot cluster CCI enablement patches are in
linux-next now.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

5) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

6) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html
 
 arch/arm/mach-exynos/mcpm-exynos.c |   55 ++++++++++++++++++++++++++----------
 arch/arm/mach-exynos/pm.c          |   42 +++++++++++++++++++++++++--
 2 files changed, 80 insertions(+), 17 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 2dd51cc..26670e7 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -129,11 +132,11 @@ static int exynos_power_up(unsigned int cpu, unsigned int cluster)
  * and can only be executed on processors like A15 and A7 that hit the cache
  * with the C bit clear in the SCTLR register.
  */
-static void exynos_power_down(void)
+static void exynos_mcpm_power_down(u64 residency)
 {
 	unsigned int mpidr, cpu, cluster;
 	bool last_man = false, skip_wfi = false;
-	unsigned int cpunr;
+	unsigned int cpunr, temp;
 
 	mpidr = read_cpuid_mpidr();
 	cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
@@ -150,7 +153,15 @@ static void exynos_power_down(void)
 	BUG_ON(__mcpm_cluster_state(cluster) != CLUSTER_UP);
 	cpu_use_count[cpu][cluster]--;
 	if (cpu_use_count[cpu][cluster] == 0) {
-		exynos_cpu_power_down(cpunr);
+		/*
+		 * Bypass power down for CPU0 during suspend. Check for
+		 * the SYS_PWR_REG value to decide if we are suspending
+		 * the system.
+		 */
+		temp = __raw_readl(pmu_base_addr +
+			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+		if ((cpu != 0) || ((temp & S5P_CORE_LOCAL_PWR_EN) != 0))
+			exynos_cpu_power_down(cpunr);
 
 		if (exynos_cluster_unused(cluster)) {
 			exynos_cluster_power_down(cluster);
@@ -209,6 +220,11 @@ static void exynos_power_down(void)
 	/* Not dead at this point?  Let our caller cope. */
 }
 
+static void exynos_power_down(void)
+{
+	exynos_mcpm_power_down(0);
+}
+
 static int exynos_wait_for_powerdown(unsigned int cpu, unsigned int cluster)
 {
 	unsigned int tries = 100;
@@ -250,11 +266,11 @@ static void exynos_suspend(u64 residency)
 {
 	unsigned int mpidr, cpunr;
 
-	exynos_power_down();
+	exynos_mcpm_power_down(residency);
 
 	/*
 	 * Execution reaches here only if cpu did not power down.
-	 * Hence roll back the changes done in exynos_power_down function.
+	 * Hence roll back the changes done in exynos_mcpm_power_down function.
 	 *
 	 * CAUTION: "This function requires the stack data to be visible through
 	 * power down and can only be executed on processors like A15 and A7
@@ -319,10 +335,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +421,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index 69cf678..d2751f4 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -318,7 +319,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -408,6 +412,11 @@ static void exynos_pm_resume(void)
 	unsigned int tmp;
 
 	if (soc_is_exynos5420()) {
+		/* Restore the CPU0 low power state register */
+		pmu_raw_writel(
+		__raw_readl(pmu_base_addr + EXYNOS5_ARM_CORE0_SYS_PWR_REG) |
+		S5P_CORE_LOCAL_PWR_EN, EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+
 		/* Restore the sysram cpu state register */
 		__raw_writel(exynos5420_cpu_state,
 		sysram_base_addr + EXYNOS5420_CPU_STATE);
@@ -490,6 +499,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(0);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +548,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
-- 
1.7.9.5

[PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Changes in v2:
	- Made use of the MCPM suspend/powered_up call-backs
Changes in v3:
	- Used the residency value to indicate the entered state
Changes in v4:
	- Checked if MCPM has been enabled to prevent build error
Changes in v5:
	- Removed the MCPM flags and just used a local flag to
	indicate that we are suspending.
Changes in v6:
	- Read the SYS_PWR_REG value to decide if we are suspending
	the system.
	- Restore the SYS_PWR_REG value post-resume.
	- Modified the comments to reflect the first change.
Changes in v7:
	- Add the suspend check in exynos_cpu_power_down() rather
	than the MCPM back-end.
	- Clean-up unnecessary changes related to earlier versions.

This has been tested both on an SMDK5420 and Peach Pit Chromebook on
next-20140704. Nicolas' boot cluster CCI enablement patches are in
linux-next now.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

5) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

6) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html
 
 arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
 arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 74 insertions(+), 12 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 2dd51cc..74ad772 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index 69cf678..e2ba1e5 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
  */
 void exynos_cpu_power_down(int cpu)
 {
+	if (soc_is_exynos5420() && cpu == 0) {
+		/*
+		 * Bypass power down for CPU0 during suspend. Check for
+		 * the SYS_PWR_REG value to decide if we are suspending
+		 * the system.
+		 */
+		int val = __raw_readl(pmu_base_addr +
+				EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+		if (!(val & S5P_CORE_LOCAL_PWR_EN))
+			return;
+	}
 	pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
 }
 
@@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
 	unsigned int tmp;
 
 	if (soc_is_exynos5420()) {
+		/* Restore the CPU0 low power state register */
+		tmp = __raw_readl(pmu_base_addr +
+			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+		pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
+			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+
 		/* Restore the sysram cpu state register */
 		__raw_writel(exynos5420_cpu_state,
 		sysram_base_addr + EXYNOS5420_CPU_STATE);
@@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(0);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
-- 
2.0.0

[PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Use the MCPM layer to handle core suspend/resume on Exynos5420.
Also, restore the entry address setup code post-resume.

Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
---
Changes in v2:
	- Made use of the MCPM suspend/powered_up call-backs
Changes in v3:
	- Used the residency value to indicate the entered state
Changes in v4:
	- Checked if MCPM has been enabled to prevent build error
Changes in v5:
	- Removed the MCPM flags and just used a local flag to
	indicate that we are suspending.
Changes in v6:
	- Read the SYS_PWR_REG value to decide if we are suspending
	the system.
	- Restore the SYS_PWR_REG value post-resume.
	- Modified the comments to reflect the first change.
Changes in v7:
	- Add the suspend check in exynos_cpu_power_down() rather
	than the MCPM back-end.
	- Clean-up unnecessary changes related to earlier versions.

This has been tested both on an SMDK5420 and Peach Pit Chromebook on
next-20140704. Nicolas' boot cluster CCI enablement patches are in
linux-next now.

Here are the dependencies (some of these patches did not apply cleanly):
1) Cleanup patches for mach-exynos
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772

2) PMU cleanup and refactoring for using DT
https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html

3) Exynos5420 PMU/S2R Series
http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898

4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
call-backs.
www.gossamer-threads.com/lists/linux/kernel/1945347
https://patchwork.kernel.org/patch/4357461/

5) Exynos5420 MCPM cluster power down support
http://www.spinics.net/lists/arm-kernel/msg339988.html

6) TPM reset mask patch
http://www.spinics.net/lists/arm-kernel/msg341884.html
 
 arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
 arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 74 insertions(+), 12 deletions(-)

diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
index 2dd51cc..74ad772 100644
--- a/arch/arm/mach-exynos/mcpm-exynos.c
+++ b/arch/arm/mach-exynos/mcpm-exynos.c
@@ -15,6 +15,7 @@
 #include <linux/delay.h>
 #include <linux/io.h>
 #include <linux/of_address.h>
+#include <linux/syscore_ops.h>
 
 #include <asm/cputype.h>
 #include <asm/cp15.h>
@@ -30,6 +31,8 @@
 #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
 #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
 
+static void __iomem *ns_sram_base_addr;
+
 /*
  * The common v7_exit_coherency_flush API could not be used because of the
  * Erratum 799270 workaround. This macro is the same as the common one (in
@@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
 	{},
 };
 
+static void exynos_mcpm_setup_entry_point(void)
+{
+	/*
+	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
+	 * as part of secondary_cpu_start().  Let's redirect it to the
+	 * mcpm_entry_point(). This is done during both secondary boot-up as
+	 * well as system resume.
+	 */
+	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
+	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
+	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+}
+
+static struct syscore_ops exynos_mcpm_syscore_ops = {
+	.resume	= exynos_mcpm_setup_entry_point,
+};
+
 static int __init exynos_mcpm_init(void)
 {
 	struct device_node *node;
-	void __iomem *ns_sram_base_addr;
 	unsigned int value, i;
 	int ret;
 
@@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
 		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
 	}
 
-	/*
-	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
-	 * as part of secondary_cpu_start().  Let's redirect it to the
-	 * mcpm_entry_point().
-	 */
-	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
-	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
-	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
+	exynos_mcpm_setup_entry_point();
 
-	iounmap(ns_sram_base_addr);
+	register_syscore_ops(&exynos_mcpm_syscore_ops);
 
 	return ret;
 }
diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
index 69cf678..e2ba1e5 100644
--- a/arch/arm/mach-exynos/pm.c
+++ b/arch/arm/mach-exynos/pm.c
@@ -24,6 +24,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/hardware/cache-l2x0.h>
+#include <asm/mcpm.h>
 #include <asm/smp_scu.h>
 #include <asm/suspend.h>
 
@@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
  */
 void exynos_cpu_power_down(int cpu)
 {
+	if (soc_is_exynos5420() && cpu == 0) {
+		/*
+		 * Bypass power down for CPU0 during suspend. Check for
+		 * the SYS_PWR_REG value to decide if we are suspending
+		 * the system.
+		 */
+		int val = __raw_readl(pmu_base_addr +
+				EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+		if (!(val & S5P_CORE_LOCAL_PWR_EN))
+			return;
+	}
 	pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
 }
 
@@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
 
 	/* ensure at least INFORM0 has the resume address */
 
-	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
+	else
+		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
 
 	if (soc_is_exynos5420()) {
 		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
@@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
 	unsigned int tmp;
 
 	if (soc_is_exynos5420()) {
+		/* Restore the CPU0 low power state register */
+		tmp = __raw_readl(pmu_base_addr +
+			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+		pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
+			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
+
 		/* Restore the sysram cpu state register */
 		__raw_writel(exynos5420_cpu_state,
 		sysram_base_addr + EXYNOS5420_CPU_STATE);
@@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
 	.resume		= exynos_pm_resume,
 };
 
+static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
+{
+	/* MCPM works with HW CPU identifiers */
+	unsigned int mpidr = read_cpuid_mpidr();
+	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
+
+	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
+
+	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
+
+	/*
+	 * Residency value passed to mcpm_cpu_suspend back-end
+	 * has to be given clear semantics. Set to 0 as a
+	 * temporary value.
+	 */
+	mcpm_cpu_suspend(0);
+
+	/* return value != 0 means failure */
+	return 1;
+}
+
 /*
  * Suspend Ops
  */
@@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
 	flush_cache_all();
 	s3c_pm_check_store();
 
-	ret = cpu_suspend(0, exynos_cpu_suspend);
+	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
+	else
+		ret = cpu_suspend(0, exynos_cpu_suspend);
 	if (ret)
 		return ret;
 
+	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
+		mcpm_cpu_powered_up();
+
 	s3c_pm_restore_uarts();
 
 	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
-- 
2.0.0

Re: [PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Nicolas Pitre]

On Sat, 5 Jul 2014, Abhilash Kesavan wrote:

> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> Also, restore the entry address setup code post-resume.
> 
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>

Acked-by: Nicolas Pitre <nico@linaro.org>


> ---
> Changes in v2:
> 	- Made use of the MCPM suspend/powered_up call-backs
> Changes in v3:
> 	- Used the residency value to indicate the entered state
> Changes in v4:
> 	- Checked if MCPM has been enabled to prevent build error
> Changes in v5:
> 	- Removed the MCPM flags and just used a local flag to
> 	indicate that we are suspending.
> Changes in v6:
> 	- Read the SYS_PWR_REG value to decide if we are suspending
> 	the system.
> 	- Restore the SYS_PWR_REG value post-resume.
> 	- Modified the comments to reflect the first change.
> Changes in v7:
> 	- Add the suspend check in exynos_cpu_power_down() rather
> 	than the MCPM back-end.
> 	- Clean-up unnecessary changes related to earlier versions.
> 
> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
> next-20140704. Nicolas' boot cluster CCI enablement patches are in
> linux-next now.
> 
> Here are the dependencies (some of these patches did not apply cleanly):
> 1) Cleanup patches for mach-exynos
> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
> 
> 2) PMU cleanup and refactoring for using DT
> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html
> 
> 3) Exynos5420 PMU/S2R Series
> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
> 
> 4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
> call-backs.
> www.gossamer-threads.com/lists/linux/kernel/1945347
> https://patchwork.kernel.org/patch/4357461/
> 
> 5) Exynos5420 MCPM cluster power down support
> http://www.spinics.net/lists/arm-kernel/msg339988.html
> 
> 6) TPM reset mask patch
> http://www.spinics.net/lists/arm-kernel/msg341884.html
>  
>  arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
>  arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
>  2 files changed, 74 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> index 2dd51cc..74ad772 100644
> --- a/arch/arm/mach-exynos/mcpm-exynos.c
> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> @@ -15,6 +15,7 @@
>  #include <linux/delay.h>
>  #include <linux/io.h>
>  #include <linux/of_address.h>
> +#include <linux/syscore_ops.h>
>  
>  #include <asm/cputype.h>
>  #include <asm/cp15.h>
> @@ -30,6 +31,8 @@
>  #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
>  #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
>  
> +static void __iomem *ns_sram_base_addr;
> +
>  /*
>   * The common v7_exit_coherency_flush API could not be used because of the
>   * Erratum 799270 workaround. This macro is the same as the common one (in
> @@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>  	{},
>  };
>  
> +static void exynos_mcpm_setup_entry_point(void)
> +{
> +	/*
> +	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> +	 * as part of secondary_cpu_start().  Let's redirect it to the
> +	 * mcpm_entry_point(). This is done during both secondary boot-up as
> +	 * well as system resume.
> +	 */
> +	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> +	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> +	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> +}
> +
> +static struct syscore_ops exynos_mcpm_syscore_ops = {
> +	.resume	= exynos_mcpm_setup_entry_point,
> +};
> +
>  static int __init exynos_mcpm_init(void)
>  {
>  	struct device_node *node;
> -	void __iomem *ns_sram_base_addr;
>  	unsigned int value, i;
>  	int ret;
>  
> @@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
>  		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
>  	}
>  
> -	/*
> -	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> -	 * as part of secondary_cpu_start().  Let's redirect it to the
> -	 * mcpm_entry_point().
> -	 */
> -	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> -	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> -	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> +	exynos_mcpm_setup_entry_point();
>  
> -	iounmap(ns_sram_base_addr);
> +	register_syscore_ops(&exynos_mcpm_syscore_ops);
>  
>  	return ret;
>  }
> diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
> index 69cf678..e2ba1e5 100644
> --- a/arch/arm/mach-exynos/pm.c
> +++ b/arch/arm/mach-exynos/pm.c
> @@ -24,6 +24,7 @@
>  
>  #include <asm/cacheflush.h>
>  #include <asm/hardware/cache-l2x0.h>
> +#include <asm/mcpm.h>
>  #include <asm/smp_scu.h>
>  #include <asm/suspend.h>
>  
> @@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
>   */
>  void exynos_cpu_power_down(int cpu)
>  {
> +	if (soc_is_exynos5420() && cpu == 0) {
> +		/*
> +		 * Bypass power down for CPU0 during suspend. Check for
> +		 * the SYS_PWR_REG value to decide if we are suspending
> +		 * the system.
> +		 */
> +		int val = __raw_readl(pmu_base_addr +
> +				EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> +		if (!(val & S5P_CORE_LOCAL_PWR_EN))
> +			return;
> +	}
>  	pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
>  }
>  
> @@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
>  
>  	/* ensure at least INFORM0 has the resume address */
>  
> -	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
> +	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> +		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
> +	else
> +		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>  
>  	if (soc_is_exynos5420()) {
>  		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
> @@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
>  	unsigned int tmp;
>  
>  	if (soc_is_exynos5420()) {
> +		/* Restore the CPU0 low power state register */
> +		tmp = __raw_readl(pmu_base_addr +
> +			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> +		pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
> +			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> +
>  		/* Restore the sysram cpu state register */
>  		__raw_writel(exynos5420_cpu_state,
>  		sysram_base_addr + EXYNOS5420_CPU_STATE);
> @@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
>  	.resume		= exynos_pm_resume,
>  };
>  
> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> +{
> +	/* MCPM works with HW CPU identifiers */
> +	unsigned int mpidr = read_cpuid_mpidr();
> +	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +
> +	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> +
> +	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> +
> +	/*
> +	 * Residency value passed to mcpm_cpu_suspend back-end
> +	 * has to be given clear semantics. Set to 0 as a
> +	 * temporary value.
> +	 */
> +	mcpm_cpu_suspend(0);
> +
> +	/* return value != 0 means failure */
> +	return 1;
> +}
> +
>  /*
>   * Suspend Ops
>   */
> @@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
>  	flush_cache_all();
>  	s3c_pm_check_store();
>  
> -	ret = cpu_suspend(0, exynos_cpu_suspend);
> +	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
> +	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> +		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
> +	else
> +		ret = cpu_suspend(0, exynos_cpu_suspend);
>  	if (ret)
>  		return ret;
>  
> +	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> +		mcpm_cpu_powered_up();
> +
>  	s3c_pm_restore_uarts();
>  
>  	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
> -- 
> 2.0.0
> 
> 

[PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Nicolas Pitre]

On Sat, 5 Jul 2014, Abhilash Kesavan wrote:

> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> Also, restore the entry address setup code post-resume.
> 
> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>

Acked-by: Nicolas Pitre <nico@linaro.org>


> ---
> Changes in v2:
> 	- Made use of the MCPM suspend/powered_up call-backs
> Changes in v3:
> 	- Used the residency value to indicate the entered state
> Changes in v4:
> 	- Checked if MCPM has been enabled to prevent build error
> Changes in v5:
> 	- Removed the MCPM flags and just used a local flag to
> 	indicate that we are suspending.
> Changes in v6:
> 	- Read the SYS_PWR_REG value to decide if we are suspending
> 	the system.
> 	- Restore the SYS_PWR_REG value post-resume.
> 	- Modified the comments to reflect the first change.
> Changes in v7:
> 	- Add the suspend check in exynos_cpu_power_down() rather
> 	than the MCPM back-end.
> 	- Clean-up unnecessary changes related to earlier versions.
> 
> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
> next-20140704. Nicolas' boot cluster CCI enablement patches are in
> linux-next now.
> 
> Here are the dependencies (some of these patches did not apply cleanly):
> 1) Cleanup patches for mach-exynos
> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
> 
> 2) PMU cleanup and refactoring for using DT
> https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html
> 
> 3) Exynos5420 PMU/S2R Series
> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
> 
> 4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
> call-backs.
> www.gossamer-threads.com/lists/linux/kernel/1945347
> https://patchwork.kernel.org/patch/4357461/
> 
> 5) Exynos5420 MCPM cluster power down support
> http://www.spinics.net/lists/arm-kernel/msg339988.html
> 
> 6) TPM reset mask patch
> http://www.spinics.net/lists/arm-kernel/msg341884.html
>  
>  arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
>  arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
>  2 files changed, 74 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> index 2dd51cc..74ad772 100644
> --- a/arch/arm/mach-exynos/mcpm-exynos.c
> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> @@ -15,6 +15,7 @@
>  #include <linux/delay.h>
>  #include <linux/io.h>
>  #include <linux/of_address.h>
> +#include <linux/syscore_ops.h>
>  
>  #include <asm/cputype.h>
>  #include <asm/cp15.h>
> @@ -30,6 +31,8 @@
>  #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE	BIT(29)
>  #define EXYNOS5420_USE_L2_COMMON_UP_STATE	BIT(30)
>  
> +static void __iomem *ns_sram_base_addr;
> +
>  /*
>   * The common v7_exit_coherency_flush API could not be used because of the
>   * Erratum 799270 workaround. This macro is the same as the common one (in
> @@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>  	{},
>  };
>  
> +static void exynos_mcpm_setup_entry_point(void)
> +{
> +	/*
> +	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> +	 * as part of secondary_cpu_start().  Let's redirect it to the
> +	 * mcpm_entry_point(). This is done during both secondary boot-up as
> +	 * well as system resume.
> +	 */
> +	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> +	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> +	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> +}
> +
> +static struct syscore_ops exynos_mcpm_syscore_ops = {
> +	.resume	= exynos_mcpm_setup_entry_point,
> +};
> +
>  static int __init exynos_mcpm_init(void)
>  {
>  	struct device_node *node;
> -	void __iomem *ns_sram_base_addr;
>  	unsigned int value, i;
>  	int ret;
>  
> @@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
>  		__raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
>  	}
>  
> -	/*
> -	 * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> -	 * as part of secondary_cpu_start().  Let's redirect it to the
> -	 * mcpm_entry_point().
> -	 */
> -	__raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> -	__raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> -	__raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> +	exynos_mcpm_setup_entry_point();
>  
> -	iounmap(ns_sram_base_addr);
> +	register_syscore_ops(&exynos_mcpm_syscore_ops);
>  
>  	return ret;
>  }
> diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
> index 69cf678..e2ba1e5 100644
> --- a/arch/arm/mach-exynos/pm.c
> +++ b/arch/arm/mach-exynos/pm.c
> @@ -24,6 +24,7 @@
>  
>  #include <asm/cacheflush.h>
>  #include <asm/hardware/cache-l2x0.h>
> +#include <asm/mcpm.h>
>  #include <asm/smp_scu.h>
>  #include <asm/suspend.h>
>  
> @@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
>   */
>  void exynos_cpu_power_down(int cpu)
>  {
> +	if (soc_is_exynos5420() && cpu == 0) {
> +		/*
> +		 * Bypass power down for CPU0 during suspend. Check for
> +		 * the SYS_PWR_REG value to decide if we are suspending
> +		 * the system.
> +		 */
> +		int val = __raw_readl(pmu_base_addr +
> +				EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> +		if (!(val & S5P_CORE_LOCAL_PWR_EN))
> +			return;
> +	}
>  	pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
>  }
>  
> @@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
>  
>  	/* ensure at least INFORM0 has the resume address */
>  
> -	pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
> +	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> +		pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
> +	else
> +		pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>  
>  	if (soc_is_exynos5420()) {
>  		tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
> @@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
>  	unsigned int tmp;
>  
>  	if (soc_is_exynos5420()) {
> +		/* Restore the CPU0 low power state register */
> +		tmp = __raw_readl(pmu_base_addr +
> +			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> +		pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
> +			EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> +
>  		/* Restore the sysram cpu state register */
>  		__raw_writel(exynos5420_cpu_state,
>  		sysram_base_addr + EXYNOS5420_CPU_STATE);
> @@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
>  	.resume		= exynos_pm_resume,
>  };
>  
> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> +{
> +	/* MCPM works with HW CPU identifiers */
> +	unsigned int mpidr = read_cpuid_mpidr();
> +	unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> +	unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> +
> +	__raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> +
> +	mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> +
> +	/*
> +	 * Residency value passed to mcpm_cpu_suspend back-end
> +	 * has to be given clear semantics. Set to 0 as a
> +	 * temporary value.
> +	 */
> +	mcpm_cpu_suspend(0);
> +
> +	/* return value != 0 means failure */
> +	return 1;
> +}
> +
>  /*
>   * Suspend Ops
>   */
> @@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
>  	flush_cache_all();
>  	s3c_pm_check_store();
>  
> -	ret = cpu_suspend(0, exynos_cpu_suspend);
> +	/* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
> +	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> +		ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
> +	else
> +		ret = cpu_suspend(0, exynos_cpu_suspend);
>  	if (ret)
>  		return ret;
>  
> +	if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> +		mcpm_cpu_powered_up();
> +
>  	s3c_pm_restore_uarts();
>  
>  	S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
> -- 
> 2.0.0
> 
> 

Re: [PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

On Sat, Jul 5, 2014 at 2:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sat, 5 Jul 2014, Abhilash Kesavan wrote:
>
>> Use the MCPM layer to handle core suspend/resume on Exynos5420.
>> Also, restore the entry address setup code post-resume.
>>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>
> Acked-by: Nicolas Pitre <nico@linaro.org>
This is one of the last missing bits for exynos MCPM. Thanks for your
patient reviews of all the exynos mcpm back-end patches.

Regards,
Abhilash
>
>
>> ---
>> Changes in v2:
>>       - Made use of the MCPM suspend/powered_up call-backs
>> Changes in v3:
>>       - Used the residency value to indicate the entered state
>> Changes in v4:
>>       - Checked if MCPM has been enabled to prevent build error
>> Changes in v5:
>>       - Removed the MCPM flags and just used a local flag to
>>       indicate that we are suspending.
>> Changes in v6:
>>       - Read the SYS_PWR_REG value to decide if we are suspending
>>       the system.
>>       - Restore the SYS_PWR_REG value post-resume.
>>       - Modified the comments to reflect the first change.
>> Changes in v7:
>>       - Add the suspend check in exynos_cpu_power_down() rather
>>       than the MCPM back-end.
>>       - Clean-up unnecessary changes related to earlier versions.
>>
>> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
>> next-20140704. Nicolas' boot cluster CCI enablement patches are in
>> linux-next now.
>>
>> Here are the dependencies (some of these patches did not apply cleanly):
>> 1) Cleanup patches for mach-exynos
>> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
>>
>> 2) PMU cleanup and refactoring for using DT
>> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html
>>
>> 3) Exynos5420 PMU/S2R Series
>> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
>>
>> 4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
>> call-backs.
>> www.gossamer-threads.com/lists/linux/kernel/1945347
>> https://patchwork.kernel.org/patch/4357461/
>>
>> 5) Exynos5420 MCPM cluster power down support
>> http://www.spinics.net/lists/arm-kernel/msg339988.html
>>
>> 6) TPM reset mask patch
>> http://www.spinics.net/lists/arm-kernel/msg341884.html
>>
>>  arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
>>  arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
>>  2 files changed, 74 insertions(+), 12 deletions(-)
>>
>> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> index 2dd51cc..74ad772 100644
>> --- a/arch/arm/mach-exynos/mcpm-exynos.c
>> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> @@ -15,6 +15,7 @@
>>  #include <linux/delay.h>
>>  #include <linux/io.h>
>>  #include <linux/of_address.h>
>> +#include <linux/syscore_ops.h>
>>
>>  #include <asm/cputype.h>
>>  #include <asm/cp15.h>
>> @@ -30,6 +31,8 @@
>>  #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE   BIT(29)
>>  #define EXYNOS5420_USE_L2_COMMON_UP_STATE    BIT(30)
>>
>> +static void __iomem *ns_sram_base_addr;
>> +
>>  /*
>>   * The common v7_exit_coherency_flush API could not be used because of the
>>   * Erratum 799270 workaround. This macro is the same as the common one (in
>> @@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>>       {},
>>  };
>>
>> +static void exynos_mcpm_setup_entry_point(void)
>> +{
>> +     /*
>> +      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> +      * as part of secondary_cpu_start().  Let's redirect it to the
>> +      * mcpm_entry_point(). This is done during both secondary boot-up as
>> +      * well as system resume.
>> +      */
>> +     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> +     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> +     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> +}
>> +
>> +static struct syscore_ops exynos_mcpm_syscore_ops = {
>> +     .resume = exynos_mcpm_setup_entry_point,
>> +};
>> +
>>  static int __init exynos_mcpm_init(void)
>>  {
>>       struct device_node *node;
>> -     void __iomem *ns_sram_base_addr;
>>       unsigned int value, i;
>>       int ret;
>>
>> @@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
>>               __raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
>>       }
>>
>> -     /*
>> -      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> -      * as part of secondary_cpu_start().  Let's redirect it to the
>> -      * mcpm_entry_point().
>> -      */
>> -     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> -     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> -     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> +     exynos_mcpm_setup_entry_point();
>>
>> -     iounmap(ns_sram_base_addr);
>> +     register_syscore_ops(&exynos_mcpm_syscore_ops);
>>
>>       return ret;
>>  }
>> diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
>> index 69cf678..e2ba1e5 100644
>> --- a/arch/arm/mach-exynos/pm.c
>> +++ b/arch/arm/mach-exynos/pm.c
>> @@ -24,6 +24,7 @@
>>
>>  #include <asm/cacheflush.h>
>>  #include <asm/hardware/cache-l2x0.h>
>> +#include <asm/mcpm.h>
>>  #include <asm/smp_scu.h>
>>  #include <asm/suspend.h>
>>
>> @@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
>>   */
>>  void exynos_cpu_power_down(int cpu)
>>  {
>> +     if (soc_is_exynos5420() && cpu == 0) {
>> +             /*
>> +              * Bypass power down for CPU0 during suspend. Check for
>> +              * the SYS_PWR_REG value to decide if we are suspending
>> +              * the system.
>> +              */
>> +             int val = __raw_readl(pmu_base_addr +
>> +                             EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> +             if (!(val & S5P_CORE_LOCAL_PWR_EN))
>> +                     return;
>> +     }
>>       pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
>>  }
>>
>> @@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
>>
>>       /* ensure at least INFORM0 has the resume address */
>>
>> -     pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> +             pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
>> +     else
>> +             pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>>
>>       if (soc_is_exynos5420()) {
>>               tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
>> @@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
>>       unsigned int tmp;
>>
>>       if (soc_is_exynos5420()) {
>> +             /* Restore the CPU0 low power state register */
>> +             tmp = __raw_readl(pmu_base_addr +
>> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> +             pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
>> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> +
>>               /* Restore the sysram cpu state register */
>>               __raw_writel(exynos5420_cpu_state,
>>               sysram_base_addr + EXYNOS5420_CPU_STATE);
>> @@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
>>       .resume         = exynos_pm_resume,
>>  };
>>
>> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> +{
>> +     /* MCPM works with HW CPU identifiers */
>> +     unsigned int mpidr = read_cpuid_mpidr();
>> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +
>> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> +
>> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> +
>> +     /*
>> +      * Residency value passed to mcpm_cpu_suspend back-end
>> +      * has to be given clear semantics. Set to 0 as a
>> +      * temporary value.
>> +      */
>> +     mcpm_cpu_suspend(0);
>> +
>> +     /* return value != 0 means failure */
>> +     return 1;
>> +}
>> +
>>  /*
>>   * Suspend Ops
>>   */
>> @@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
>>       flush_cache_all();
>>       s3c_pm_check_store();
>>
>> -     ret = cpu_suspend(0, exynos_cpu_suspend);
>> +     /* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
>> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> +             ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
>> +     else
>> +             ret = cpu_suspend(0, exynos_cpu_suspend);
>>       if (ret)
>>               return ret;
>>
>> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> +             mcpm_cpu_powered_up();
>> +
>>       s3c_pm_restore_uarts();
>>
>>       S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
>> --
>> 2.0.0
>>
>>

[PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

On Sat, Jul 5, 2014 at 2:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> On Sat, 5 Jul 2014, Abhilash Kesavan wrote:
>
>> Use the MCPM layer to handle core suspend/resume on Exynos5420.
>> Also, restore the entry address setup code post-resume.
>>
>> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>
> Acked-by: Nicolas Pitre <nico@linaro.org>
This is one of the last missing bits for exynos MCPM. Thanks for your
patient reviews of all the exynos mcpm back-end patches.

Regards,
Abhilash
>
>
>> ---
>> Changes in v2:
>>       - Made use of the MCPM suspend/powered_up call-backs
>> Changes in v3:
>>       - Used the residency value to indicate the entered state
>> Changes in v4:
>>       - Checked if MCPM has been enabled to prevent build error
>> Changes in v5:
>>       - Removed the MCPM flags and just used a local flag to
>>       indicate that we are suspending.
>> Changes in v6:
>>       - Read the SYS_PWR_REG value to decide if we are suspending
>>       the system.
>>       - Restore the SYS_PWR_REG value post-resume.
>>       - Modified the comments to reflect the first change.
>> Changes in v7:
>>       - Add the suspend check in exynos_cpu_power_down() rather
>>       than the MCPM back-end.
>>       - Clean-up unnecessary changes related to earlier versions.
>>
>> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
>> next-20140704. Nicolas' boot cluster CCI enablement patches are in
>> linux-next now.
>>
>> Here are the dependencies (some of these patches did not apply cleanly):
>> 1) Cleanup patches for mach-exynos
>> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
>>
>> 2) PMU cleanup and refactoring for using DT
>> https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html
>>
>> 3) Exynos5420 PMU/S2R Series
>> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
>>
>> 4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
>> call-backs.
>> www.gossamer-threads.com/lists/linux/kernel/1945347
>> https://patchwork.kernel.org/patch/4357461/
>>
>> 5) Exynos5420 MCPM cluster power down support
>> http://www.spinics.net/lists/arm-kernel/msg339988.html
>>
>> 6) TPM reset mask patch
>> http://www.spinics.net/lists/arm-kernel/msg341884.html
>>
>>  arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
>>  arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
>>  2 files changed, 74 insertions(+), 12 deletions(-)
>>
>> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> index 2dd51cc..74ad772 100644
>> --- a/arch/arm/mach-exynos/mcpm-exynos.c
>> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> @@ -15,6 +15,7 @@
>>  #include <linux/delay.h>
>>  #include <linux/io.h>
>>  #include <linux/of_address.h>
>> +#include <linux/syscore_ops.h>
>>
>>  #include <asm/cputype.h>
>>  #include <asm/cp15.h>
>> @@ -30,6 +31,8 @@
>>  #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE   BIT(29)
>>  #define EXYNOS5420_USE_L2_COMMON_UP_STATE    BIT(30)
>>
>> +static void __iomem *ns_sram_base_addr;
>> +
>>  /*
>>   * The common v7_exit_coherency_flush API could not be used because of the
>>   * Erratum 799270 workaround. This macro is the same as the common one (in
>> @@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>>       {},
>>  };
>>
>> +static void exynos_mcpm_setup_entry_point(void)
>> +{
>> +     /*
>> +      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> +      * as part of secondary_cpu_start().  Let's redirect it to the
>> +      * mcpm_entry_point(). This is done during both secondary boot-up as
>> +      * well as system resume.
>> +      */
>> +     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> +     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> +     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> +}
>> +
>> +static struct syscore_ops exynos_mcpm_syscore_ops = {
>> +     .resume = exynos_mcpm_setup_entry_point,
>> +};
>> +
>>  static int __init exynos_mcpm_init(void)
>>  {
>>       struct device_node *node;
>> -     void __iomem *ns_sram_base_addr;
>>       unsigned int value, i;
>>       int ret;
>>
>> @@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
>>               __raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
>>       }
>>
>> -     /*
>> -      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> -      * as part of secondary_cpu_start().  Let's redirect it to the
>> -      * mcpm_entry_point().
>> -      */
>> -     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> -     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> -     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> +     exynos_mcpm_setup_entry_point();
>>
>> -     iounmap(ns_sram_base_addr);
>> +     register_syscore_ops(&exynos_mcpm_syscore_ops);
>>
>>       return ret;
>>  }
>> diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
>> index 69cf678..e2ba1e5 100644
>> --- a/arch/arm/mach-exynos/pm.c
>> +++ b/arch/arm/mach-exynos/pm.c
>> @@ -24,6 +24,7 @@
>>
>>  #include <asm/cacheflush.h>
>>  #include <asm/hardware/cache-l2x0.h>
>> +#include <asm/mcpm.h>
>>  #include <asm/smp_scu.h>
>>  #include <asm/suspend.h>
>>
>> @@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
>>   */
>>  void exynos_cpu_power_down(int cpu)
>>  {
>> +     if (soc_is_exynos5420() && cpu == 0) {
>> +             /*
>> +              * Bypass power down for CPU0 during suspend. Check for
>> +              * the SYS_PWR_REG value to decide if we are suspending
>> +              * the system.
>> +              */
>> +             int val = __raw_readl(pmu_base_addr +
>> +                             EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> +             if (!(val & S5P_CORE_LOCAL_PWR_EN))
>> +                     return;
>> +     }
>>       pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
>>  }
>>
>> @@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
>>
>>       /* ensure at least INFORM0 has the resume address */
>>
>> -     pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> +             pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
>> +     else
>> +             pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>>
>>       if (soc_is_exynos5420()) {
>>               tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
>> @@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
>>       unsigned int tmp;
>>
>>       if (soc_is_exynos5420()) {
>> +             /* Restore the CPU0 low power state register */
>> +             tmp = __raw_readl(pmu_base_addr +
>> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> +             pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
>> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> +
>>               /* Restore the sysram cpu state register */
>>               __raw_writel(exynos5420_cpu_state,
>>               sysram_base_addr + EXYNOS5420_CPU_STATE);
>> @@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
>>       .resume         = exynos_pm_resume,
>>  };
>>
>> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> +{
>> +     /* MCPM works with HW CPU identifiers */
>> +     unsigned int mpidr = read_cpuid_mpidr();
>> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> +
>> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> +
>> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> +
>> +     /*
>> +      * Residency value passed to mcpm_cpu_suspend back-end
>> +      * has to be given clear semantics. Set to 0 as a
>> +      * temporary value.
>> +      */
>> +     mcpm_cpu_suspend(0);
>> +
>> +     /* return value != 0 means failure */
>> +     return 1;
>> +}
>> +
>>  /*
>>   * Suspend Ops
>>   */
>> @@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
>>       flush_cache_all();
>>       s3c_pm_check_store();
>>
>> -     ret = cpu_suspend(0, exynos_cpu_suspend);
>> +     /* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
>> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> +             ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
>> +     else
>> +             ret = cpu_suspend(0, exynos_cpu_suspend);
>>       if (ret)
>>               return ret;
>>
>> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> +             mcpm_cpu_powered_up();
>> +
>>       s3c_pm_restore_uarts();
>>
>>       S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
>> --
>> 2.0.0
>>
>>

Re: [PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Lorenzo Pieralisi]

On Fri, Jul 04, 2014 at 10:21:56PM +0100, Abhilash Kesavan wrote:
> On Sat, Jul 5, 2014 at 2:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Sat, 5 Jul 2014, Abhilash Kesavan wrote:
> >
> >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> >> Also, restore the entry address setup code post-resume.
> >>
> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> >
> > Acked-by: Nicolas Pitre <nico@linaro.org>
> This is one of the last missing bits for exynos MCPM. Thanks for your
> patient reviews of all the exynos mcpm back-end patches.

Sorry for the delay in getting back to you.

You still need to handle the residency parameter in the MCPM suspend call
properly to differentiate between core gating and cluster gating.

CPUidle might want to request a state that allows for core gating
residency but not cluster gating one, in that case the last man
standing must not shutdown the cluster.

This must be handled in the power down function, and it can be done
in a separate patch since it is an optimization. Current MCPM interface has
all it is needed to handle this properly, and I am keen on patching the
CPUidle bL driver to pass the actual residency instead of hardcoding it
to 0 (ie on TC2 it was not needed, no core gating).

Lorenzo

> 
> Regards,
> Abhilash
> >
> >
> >> ---
> >> Changes in v2:
> >>       - Made use of the MCPM suspend/powered_up call-backs
> >> Changes in v3:
> >>       - Used the residency value to indicate the entered state
> >> Changes in v4:
> >>       - Checked if MCPM has been enabled to prevent build error
> >> Changes in v5:
> >>       - Removed the MCPM flags and just used a local flag to
> >>       indicate that we are suspending.
> >> Changes in v6:
> >>       - Read the SYS_PWR_REG value to decide if we are suspending
> >>       the system.
> >>       - Restore the SYS_PWR_REG value post-resume.
> >>       - Modified the comments to reflect the first change.
> >> Changes in v7:
> >>       - Add the suspend check in exynos_cpu_power_down() rather
> >>       than the MCPM back-end.
> >>       - Clean-up unnecessary changes related to earlier versions.
> >>
> >> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
> >> next-20140704. Nicolas' boot cluster CCI enablement patches are in
> >> linux-next now.
> >>
> >> Here are the dependencies (some of these patches did not apply cleanly):
> >> 1) Cleanup patches for mach-exynos
> >> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
> >>
> >> 2) PMU cleanup and refactoring for using DT
> >> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html
> >>
> >> 3) Exynos5420 PMU/S2R Series
> >> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
> >>
> >> 4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
> >> call-backs.
> >> www.gossamer-threads.com/lists/linux/kernel/1945347
> >> https://patchwork.kernel.org/patch/4357461/
> >>
> >> 5) Exynos5420 MCPM cluster power down support
> >> http://www.spinics.net/lists/arm-kernel/msg339988.html
> >>
> >> 6) TPM reset mask patch
> >> http://www.spinics.net/lists/arm-kernel/msg341884.html
> >>
> >>  arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
> >>  arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
> >>  2 files changed, 74 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> >> index 2dd51cc..74ad772 100644
> >> --- a/arch/arm/mach-exynos/mcpm-exynos.c
> >> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> >> @@ -15,6 +15,7 @@
> >>  #include <linux/delay.h>
> >>  #include <linux/io.h>
> >>  #include <linux/of_address.h>
> >> +#include <linux/syscore_ops.h>
> >>
> >>  #include <asm/cputype.h>
> >>  #include <asm/cp15.h>
> >> @@ -30,6 +31,8 @@
> >>  #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE   BIT(29)
> >>  #define EXYNOS5420_USE_L2_COMMON_UP_STATE    BIT(30)
> >>
> >> +static void __iomem *ns_sram_base_addr;
> >> +
> >>  /*
> >>   * The common v7_exit_coherency_flush API could not be used because of the
> >>   * Erratum 799270 workaround. This macro is the same as the common one (in
> >> @@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
> >>       {},
> >>  };
> >>
> >> +static void exynos_mcpm_setup_entry_point(void)
> >> +{
> >> +     /*
> >> +      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> >> +      * as part of secondary_cpu_start().  Let's redirect it to the
> >> +      * mcpm_entry_point(). This is done during both secondary boot-up as
> >> +      * well as system resume.
> >> +      */
> >> +     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> >> +     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> >> +     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> >> +}
> >> +
> >> +static struct syscore_ops exynos_mcpm_syscore_ops = {
> >> +     .resume = exynos_mcpm_setup_entry_point,
> >> +};
> >> +
> >>  static int __init exynos_mcpm_init(void)
> >>  {
> >>       struct device_node *node;
> >> -     void __iomem *ns_sram_base_addr;
> >>       unsigned int value, i;
> >>       int ret;
> >>
> >> @@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
> >>               __raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
> >>       }
> >>
> >> -     /*
> >> -      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> >> -      * as part of secondary_cpu_start().  Let's redirect it to the
> >> -      * mcpm_entry_point().
> >> -      */
> >> -     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> >> -     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> >> -     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> >> +     exynos_mcpm_setup_entry_point();
> >>
> >> -     iounmap(ns_sram_base_addr);
> >> +     register_syscore_ops(&exynos_mcpm_syscore_ops);
> >>
> >>       return ret;
> >>  }
> >> diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
> >> index 69cf678..e2ba1e5 100644
> >> --- a/arch/arm/mach-exynos/pm.c
> >> +++ b/arch/arm/mach-exynos/pm.c
> >> @@ -24,6 +24,7 @@
> >>
> >>  #include <asm/cacheflush.h>
> >>  #include <asm/hardware/cache-l2x0.h>
> >> +#include <asm/mcpm.h>
> >>  #include <asm/smp_scu.h>
> >>  #include <asm/suspend.h>
> >>
> >> @@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
> >>   */
> >>  void exynos_cpu_power_down(int cpu)
> >>  {
> >> +     if (soc_is_exynos5420() && cpu == 0) {
> >> +             /*
> >> +              * Bypass power down for CPU0 during suspend. Check for
> >> +              * the SYS_PWR_REG value to decide if we are suspending
> >> +              * the system.
> >> +              */
> >> +             int val = __raw_readl(pmu_base_addr +
> >> +                             EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> >> +             if (!(val & S5P_CORE_LOCAL_PWR_EN))
> >> +                     return;
> >> +     }
> >>       pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
> >>  }
> >>
> >> @@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
> >>
> >>       /* ensure at least INFORM0 has the resume address */
> >>
> >> -     pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> >> +             pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
> >> +     else
> >> +             pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
> >>
> >>       if (soc_is_exynos5420()) {
> >>               tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
> >> @@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
> >>       unsigned int tmp;
> >>
> >>       if (soc_is_exynos5420()) {
> >> +             /* Restore the CPU0 low power state register */
> >> +             tmp = __raw_readl(pmu_base_addr +
> >> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> >> +             pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
> >> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> >> +
> >>               /* Restore the sysram cpu state register */
> >>               __raw_writel(exynos5420_cpu_state,
> >>               sysram_base_addr + EXYNOS5420_CPU_STATE);
> >> @@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
> >>       .resume         = exynos_pm_resume,
> >>  };
> >>
> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> >> +{
> >> +     /* MCPM works with HW CPU identifiers */
> >> +     unsigned int mpidr = read_cpuid_mpidr();
> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> >> +
> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> >> +
> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> >> +
> >> +     /*
> >> +      * Residency value passed to mcpm_cpu_suspend back-end
> >> +      * has to be given clear semantics. Set to 0 as a
> >> +      * temporary value.
> >> +      */
> >> +     mcpm_cpu_suspend(0);
> >> +
> >> +     /* return value != 0 means failure */
> >> +     return 1;
> >> +}
> >> +
> >>  /*
> >>   * Suspend Ops
> >>   */
> >> @@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
> >>       flush_cache_all();
> >>       s3c_pm_check_store();
> >>
> >> -     ret = cpu_suspend(0, exynos_cpu_suspend);
> >> +     /* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> >> +             ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
> >> +     else
> >> +             ret = cpu_suspend(0, exynos_cpu_suspend);
> >>       if (ret)
> >>               return ret;
> >>
> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> >> +             mcpm_cpu_powered_up();
> >> +
> >>       s3c_pm_restore_uarts();
> >>
> >>       S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
> >> --
> >> 2.0.0
> >>
> >>
> 

[PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Lorenzo Pieralisi]

On Fri, Jul 04, 2014 at 10:21:56PM +0100, Abhilash Kesavan wrote:
> On Sat, Jul 5, 2014 at 2:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> > On Sat, 5 Jul 2014, Abhilash Kesavan wrote:
> >
> >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> >> Also, restore the entry address setup code post-resume.
> >>
> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> >
> > Acked-by: Nicolas Pitre <nico@linaro.org>
> This is one of the last missing bits for exynos MCPM. Thanks for your
> patient reviews of all the exynos mcpm back-end patches.

Sorry for the delay in getting back to you.

You still need to handle the residency parameter in the MCPM suspend call
properly to differentiate between core gating and cluster gating.

CPUidle might want to request a state that allows for core gating
residency but not cluster gating one, in that case the last man
standing must not shutdown the cluster.

This must be handled in the power down function, and it can be done
in a separate patch since it is an optimization. Current MCPM interface has
all it is needed to handle this properly, and I am keen on patching the
CPUidle bL driver to pass the actual residency instead of hardcoding it
to 0 (ie on TC2 it was not needed, no core gating).

Lorenzo

> 
> Regards,
> Abhilash
> >
> >
> >> ---
> >> Changes in v2:
> >>       - Made use of the MCPM suspend/powered_up call-backs
> >> Changes in v3:
> >>       - Used the residency value to indicate the entered state
> >> Changes in v4:
> >>       - Checked if MCPM has been enabled to prevent build error
> >> Changes in v5:
> >>       - Removed the MCPM flags and just used a local flag to
> >>       indicate that we are suspending.
> >> Changes in v6:
> >>       - Read the SYS_PWR_REG value to decide if we are suspending
> >>       the system.
> >>       - Restore the SYS_PWR_REG value post-resume.
> >>       - Modified the comments to reflect the first change.
> >> Changes in v7:
> >>       - Add the suspend check in exynos_cpu_power_down() rather
> >>       than the MCPM back-end.
> >>       - Clean-up unnecessary changes related to earlier versions.
> >>
> >> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
> >> next-20140704. Nicolas' boot cluster CCI enablement patches are in
> >> linux-next now.
> >>
> >> Here are the dependencies (some of these patches did not apply cleanly):
> >> 1) Cleanup patches for mach-exynos
> >> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
> >>
> >> 2) PMU cleanup and refactoring for using DT
> >> https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html
> >>
> >> 3) Exynos5420 PMU/S2R Series
> >> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
> >>
> >> 4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
> >> call-backs.
> >> www.gossamer-threads.com/lists/linux/kernel/1945347
> >> https://patchwork.kernel.org/patch/4357461/
> >>
> >> 5) Exynos5420 MCPM cluster power down support
> >> http://www.spinics.net/lists/arm-kernel/msg339988.html
> >>
> >> 6) TPM reset mask patch
> >> http://www.spinics.net/lists/arm-kernel/msg341884.html
> >>
> >>  arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
> >>  arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
> >>  2 files changed, 74 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
> >> index 2dd51cc..74ad772 100644
> >> --- a/arch/arm/mach-exynos/mcpm-exynos.c
> >> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
> >> @@ -15,6 +15,7 @@
> >>  #include <linux/delay.h>
> >>  #include <linux/io.h>
> >>  #include <linux/of_address.h>
> >> +#include <linux/syscore_ops.h>
> >>
> >>  #include <asm/cputype.h>
> >>  #include <asm/cp15.h>
> >> @@ -30,6 +31,8 @@
> >>  #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE   BIT(29)
> >>  #define EXYNOS5420_USE_L2_COMMON_UP_STATE    BIT(30)
> >>
> >> +static void __iomem *ns_sram_base_addr;
> >> +
> >>  /*
> >>   * The common v7_exit_coherency_flush API could not be used because of the
> >>   * Erratum 799270 workaround. This macro is the same as the common one (in
> >> @@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
> >>       {},
> >>  };
> >>
> >> +static void exynos_mcpm_setup_entry_point(void)
> >> +{
> >> +     /*
> >> +      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> >> +      * as part of secondary_cpu_start().  Let's redirect it to the
> >> +      * mcpm_entry_point(). This is done during both secondary boot-up as
> >> +      * well as system resume.
> >> +      */
> >> +     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> >> +     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> >> +     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> >> +}
> >> +
> >> +static struct syscore_ops exynos_mcpm_syscore_ops = {
> >> +     .resume = exynos_mcpm_setup_entry_point,
> >> +};
> >> +
> >>  static int __init exynos_mcpm_init(void)
> >>  {
> >>       struct device_node *node;
> >> -     void __iomem *ns_sram_base_addr;
> >>       unsigned int value, i;
> >>       int ret;
> >>
> >> @@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
> >>               __raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
> >>       }
> >>
> >> -     /*
> >> -      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
> >> -      * as part of secondary_cpu_start().  Let's redirect it to the
> >> -      * mcpm_entry_point().
> >> -      */
> >> -     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
> >> -     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
> >> -     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
> >> +     exynos_mcpm_setup_entry_point();
> >>
> >> -     iounmap(ns_sram_base_addr);
> >> +     register_syscore_ops(&exynos_mcpm_syscore_ops);
> >>
> >>       return ret;
> >>  }
> >> diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
> >> index 69cf678..e2ba1e5 100644
> >> --- a/arch/arm/mach-exynos/pm.c
> >> +++ b/arch/arm/mach-exynos/pm.c
> >> @@ -24,6 +24,7 @@
> >>
> >>  #include <asm/cacheflush.h>
> >>  #include <asm/hardware/cache-l2x0.h>
> >> +#include <asm/mcpm.h>
> >>  #include <asm/smp_scu.h>
> >>  #include <asm/suspend.h>
> >>
> >> @@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
> >>   */
> >>  void exynos_cpu_power_down(int cpu)
> >>  {
> >> +     if (soc_is_exynos5420() && cpu == 0) {
> >> +             /*
> >> +              * Bypass power down for CPU0 during suspend. Check for
> >> +              * the SYS_PWR_REG value to decide if we are suspending
> >> +              * the system.
> >> +              */
> >> +             int val = __raw_readl(pmu_base_addr +
> >> +                             EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> >> +             if (!(val & S5P_CORE_LOCAL_PWR_EN))
> >> +                     return;
> >> +     }
> >>       pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
> >>  }
> >>
> >> @@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
> >>
> >>       /* ensure at least INFORM0 has the resume address */
> >>
> >> -     pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> >> +             pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
> >> +     else
> >> +             pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
> >>
> >>       if (soc_is_exynos5420()) {
> >>               tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
> >> @@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
> >>       unsigned int tmp;
> >>
> >>       if (soc_is_exynos5420()) {
> >> +             /* Restore the CPU0 low power state register */
> >> +             tmp = __raw_readl(pmu_base_addr +
> >> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> >> +             pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
> >> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
> >> +
> >>               /* Restore the sysram cpu state register */
> >>               __raw_writel(exynos5420_cpu_state,
> >>               sysram_base_addr + EXYNOS5420_CPU_STATE);
> >> @@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
> >>       .resume         = exynos_pm_resume,
> >>  };
> >>
> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
> >> +{
> >> +     /* MCPM works with HW CPU identifiers */
> >> +     unsigned int mpidr = read_cpuid_mpidr();
> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
> >> +
> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
> >> +
> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
> >> +
> >> +     /*
> >> +      * Residency value passed to mcpm_cpu_suspend back-end
> >> +      * has to be given clear semantics. Set to 0 as a
> >> +      * temporary value.
> >> +      */
> >> +     mcpm_cpu_suspend(0);
> >> +
> >> +     /* return value != 0 means failure */
> >> +     return 1;
> >> +}
> >> +
> >>  /*
> >>   * Suspend Ops
> >>   */
> >> @@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
> >>       flush_cache_all();
> >>       s3c_pm_check_store();
> >>
> >> -     ret = cpu_suspend(0, exynos_cpu_suspend);
> >> +     /* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> >> +             ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
> >> +     else
> >> +             ret = cpu_suspend(0, exynos_cpu_suspend);
> >>       if (ret)
> >>               return ret;
> >>
> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
> >> +             mcpm_cpu_powered_up();
> >> +
> >>       s3c_pm_restore_uarts();
> >>
> >>       S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
> >> --
> >> 2.0.0
> >>
> >>
> 

Re: [PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Hi Lorenzo,

On Tue, Jul 8, 2014 at 4:23 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
> On Fri, Jul 04, 2014 at 10:21:56PM +0100, Abhilash Kesavan wrote:
>> On Sat, Jul 5, 2014 at 2:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Sat, 5 Jul 2014, Abhilash Kesavan wrote:
>> >
>> >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
>> >> Also, restore the entry address setup code post-resume.
>> >>
>> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> >
>> > Acked-by: Nicolas Pitre <nico@linaro.org>
>> This is one of the last missing bits for exynos MCPM. Thanks for your
>> patient reviews of all the exynos mcpm back-end patches.
>
> Sorry for the delay in getting back to you.
Thanks for your comments.
>
> You still need to handle the residency parameter in the MCPM suspend call
> properly to differentiate between core gating and cluster gating.
I am not too familiar with the cpuidle driver, but I see only cluster
power off state listed in the bL cpuidle driver.
>
> CPUidle might want to request a state that allows for core gating
> residency but not cluster gating one, in that case the last man
> standing must not shutdown the cluster.
For this, I would have to add another state to the bL cpuidle driver
or migrate to using your under-review generic idle state patchset.
Then based on the residency value skip/do cluster power down. Is this
correct ?
>
> This must be handled in the power down function, and it can be done
> in a separate patch since it is an optimization. Current MCPM interface has
> all it is needed to handle this properly, and I am keen on patching the
> CPUidle bL driver to pass the actual residency instead of hardcoding it
> to 0 (ie on TC2 it was not needed, no core gating).
I will look at doing this once the s2r patch gets merged.

Thanks,
Abhilash
>
> Lorenzo
>
>>
>> Regards,
>> Abhilash
>> >
>> >
>> >> ---
>> >> Changes in v2:
>> >>       - Made use of the MCPM suspend/powered_up call-backs
>> >> Changes in v3:
>> >>       - Used the residency value to indicate the entered state
>> >> Changes in v4:
>> >>       - Checked if MCPM has been enabled to prevent build error
>> >> Changes in v5:
>> >>       - Removed the MCPM flags and just used a local flag to
>> >>       indicate that we are suspending.
>> >> Changes in v6:
>> >>       - Read the SYS_PWR_REG value to decide if we are suspending
>> >>       the system.
>> >>       - Restore the SYS_PWR_REG value post-resume.
>> >>       - Modified the comments to reflect the first change.
>> >> Changes in v7:
>> >>       - Add the suspend check in exynos_cpu_power_down() rather
>> >>       than the MCPM back-end.
>> >>       - Clean-up unnecessary changes related to earlier versions.
>> >>
>> >> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
>> >> next-20140704. Nicolas' boot cluster CCI enablement patches are in
>> >> linux-next now.
>> >>
>> >> Here are the dependencies (some of these patches did not apply cleanly):
>> >> 1) Cleanup patches for mach-exynos
>> >> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
>> >>
>> >> 2) PMU cleanup and refactoring for using DT
>> >> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg671625.html
>> >>
>> >> 3) Exynos5420 PMU/S2R Series
>> >> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
>> >>
>> >> 4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
>> >> call-backs.
>> >> www.gossamer-threads.com/lists/linux/kernel/1945347
>> >> https://patchwork.kernel.org/patch/4357461/
>> >>
>> >> 5) Exynos5420 MCPM cluster power down support
>> >> http://www.spinics.net/lists/arm-kernel/msg339988.html
>> >>
>> >> 6) TPM reset mask patch
>> >> http://www.spinics.net/lists/arm-kernel/msg341884.html
>> >>
>> >>  arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
>> >>  arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
>> >>  2 files changed, 74 insertions(+), 12 deletions(-)
>> >>
>> >> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> >> index 2dd51cc..74ad772 100644
>> >> --- a/arch/arm/mach-exynos/mcpm-exynos.c
>> >> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> >> @@ -15,6 +15,7 @@
>> >>  #include <linux/delay.h>
>> >>  #include <linux/io.h>
>> >>  #include <linux/of_address.h>
>> >> +#include <linux/syscore_ops.h>
>> >>
>> >>  #include <asm/cputype.h>
>> >>  #include <asm/cp15.h>
>> >> @@ -30,6 +31,8 @@
>> >>  #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE   BIT(29)
>> >>  #define EXYNOS5420_USE_L2_COMMON_UP_STATE    BIT(30)
>> >>
>> >> +static void __iomem *ns_sram_base_addr;
>> >> +
>> >>  /*
>> >>   * The common v7_exit_coherency_flush API could not be used because of the
>> >>   * Erratum 799270 workaround. This macro is the same as the common one (in
>> >> @@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>> >>       {},
>> >>  };
>> >>
>> >> +static void exynos_mcpm_setup_entry_point(void)
>> >> +{
>> >> +     /*
>> >> +      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> >> +      * as part of secondary_cpu_start().  Let's redirect it to the
>> >> +      * mcpm_entry_point(). This is done during both secondary boot-up as
>> >> +      * well as system resume.
>> >> +      */
>> >> +     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> >> +     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> >> +     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> >> +}
>> >> +
>> >> +static struct syscore_ops exynos_mcpm_syscore_ops = {
>> >> +     .resume = exynos_mcpm_setup_entry_point,
>> >> +};
>> >> +
>> >>  static int __init exynos_mcpm_init(void)
>> >>  {
>> >>       struct device_node *node;
>> >> -     void __iomem *ns_sram_base_addr;
>> >>       unsigned int value, i;
>> >>       int ret;
>> >>
>> >> @@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
>> >>               __raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
>> >>       }
>> >>
>> >> -     /*
>> >> -      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> >> -      * as part of secondary_cpu_start().  Let's redirect it to the
>> >> -      * mcpm_entry_point().
>> >> -      */
>> >> -     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> >> -     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> >> -     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> >> +     exynos_mcpm_setup_entry_point();
>> >>
>> >> -     iounmap(ns_sram_base_addr);
>> >> +     register_syscore_ops(&exynos_mcpm_syscore_ops);
>> >>
>> >>       return ret;
>> >>  }
>> >> diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
>> >> index 69cf678..e2ba1e5 100644
>> >> --- a/arch/arm/mach-exynos/pm.c
>> >> +++ b/arch/arm/mach-exynos/pm.c
>> >> @@ -24,6 +24,7 @@
>> >>
>> >>  #include <asm/cacheflush.h>
>> >>  #include <asm/hardware/cache-l2x0.h>
>> >> +#include <asm/mcpm.h>
>> >>  #include <asm/smp_scu.h>
>> >>  #include <asm/suspend.h>
>> >>
>> >> @@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
>> >>   */
>> >>  void exynos_cpu_power_down(int cpu)
>> >>  {
>> >> +     if (soc_is_exynos5420() && cpu == 0) {
>> >> +             /*
>> >> +              * Bypass power down for CPU0 during suspend. Check for
>> >> +              * the SYS_PWR_REG value to decide if we are suspending
>> >> +              * the system.
>> >> +              */
>> >> +             int val = __raw_readl(pmu_base_addr +
>> >> +                             EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> >> +             if (!(val & S5P_CORE_LOCAL_PWR_EN))
>> >> +                     return;
>> >> +     }
>> >>       pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
>> >>  }
>> >>
>> >> @@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
>> >>
>> >>       /* ensure at least INFORM0 has the resume address */
>> >>
>> >> -     pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> >> +             pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
>> >> +     else
>> >> +             pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>> >>
>> >>       if (soc_is_exynos5420()) {
>> >>               tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
>> >> @@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
>> >>       unsigned int tmp;
>> >>
>> >>       if (soc_is_exynos5420()) {
>> >> +             /* Restore the CPU0 low power state register */
>> >> +             tmp = __raw_readl(pmu_base_addr +
>> >> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> >> +             pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
>> >> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> >> +
>> >>               /* Restore the sysram cpu state register */
>> >>               __raw_writel(exynos5420_cpu_state,
>> >>               sysram_base_addr + EXYNOS5420_CPU_STATE);
>> >> @@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
>> >>       .resume         = exynos_pm_resume,
>> >>  };
>> >>
>> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> >> +{
>> >> +     /* MCPM works with HW CPU identifiers */
>> >> +     unsigned int mpidr = read_cpuid_mpidr();
>> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> >> +
>> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> >> +
>> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> >> +
>> >> +     /*
>> >> +      * Residency value passed to mcpm_cpu_suspend back-end
>> >> +      * has to be given clear semantics. Set to 0 as a
>> >> +      * temporary value.
>> >> +      */
>> >> +     mcpm_cpu_suspend(0);
>> >> +
>> >> +     /* return value != 0 means failure */
>> >> +     return 1;
>> >> +}
>> >> +
>> >>  /*
>> >>   * Suspend Ops
>> >>   */
>> >> @@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
>> >>       flush_cache_all();
>> >>       s3c_pm_check_store();
>> >>
>> >> -     ret = cpu_suspend(0, exynos_cpu_suspend);
>> >> +     /* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
>> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> >> +             ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
>> >> +     else
>> >> +             ret = cpu_suspend(0, exynos_cpu_suspend);
>> >>       if (ret)
>> >>               return ret;
>> >>
>> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> >> +             mcpm_cpu_powered_up();
>> >> +
>> >>       s3c_pm_restore_uarts();
>> >>
>> >>       S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
>> >> --
>> >> 2.0.0
>> >>
>> >>
>>
>

[PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Abhilash Kesavan]

Hi Lorenzo,

On Tue, Jul 8, 2014 at 4:23 PM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
> On Fri, Jul 04, 2014 at 10:21:56PM +0100, Abhilash Kesavan wrote:
>> On Sat, Jul 5, 2014 at 2:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
>> > On Sat, 5 Jul 2014, Abhilash Kesavan wrote:
>> >
>> >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
>> >> Also, restore the entry address setup code post-resume.
>> >>
>> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
>> >
>> > Acked-by: Nicolas Pitre <nico@linaro.org>
>> This is one of the last missing bits for exynos MCPM. Thanks for your
>> patient reviews of all the exynos mcpm back-end patches.
>
> Sorry for the delay in getting back to you.
Thanks for your comments.
>
> You still need to handle the residency parameter in the MCPM suspend call
> properly to differentiate between core gating and cluster gating.
I am not too familiar with the cpuidle driver, but I see only cluster
power off state listed in the bL cpuidle driver.
>
> CPUidle might want to request a state that allows for core gating
> residency but not cluster gating one, in that case the last man
> standing must not shutdown the cluster.
For this, I would have to add another state to the bL cpuidle driver
or migrate to using your under-review generic idle state patchset.
Then based on the residency value skip/do cluster power down. Is this
correct ?
>
> This must be handled in the power down function, and it can be done
> in a separate patch since it is an optimization. Current MCPM interface has
> all it is needed to handle this properly, and I am keen on patching the
> CPUidle bL driver to pass the actual residency instead of hardcoding it
> to 0 (ie on TC2 it was not needed, no core gating).
I will look at doing this once the s2r patch gets merged.

Thanks,
Abhilash
>
> Lorenzo
>
>>
>> Regards,
>> Abhilash
>> >
>> >
>> >> ---
>> >> Changes in v2:
>> >>       - Made use of the MCPM suspend/powered_up call-backs
>> >> Changes in v3:
>> >>       - Used the residency value to indicate the entered state
>> >> Changes in v4:
>> >>       - Checked if MCPM has been enabled to prevent build error
>> >> Changes in v5:
>> >>       - Removed the MCPM flags and just used a local flag to
>> >>       indicate that we are suspending.
>> >> Changes in v6:
>> >>       - Read the SYS_PWR_REG value to decide if we are suspending
>> >>       the system.
>> >>       - Restore the SYS_PWR_REG value post-resume.
>> >>       - Modified the comments to reflect the first change.
>> >> Changes in v7:
>> >>       - Add the suspend check in exynos_cpu_power_down() rather
>> >>       than the MCPM back-end.
>> >>       - Clean-up unnecessary changes related to earlier versions.
>> >>
>> >> This has been tested both on an SMDK5420 and Peach Pit Chromebook on
>> >> next-20140704. Nicolas' boot cluster CCI enablement patches are in
>> >> linux-next now.
>> >>
>> >> Here are the dependencies (some of these patches did not apply cleanly):
>> >> 1) Cleanup patches for mach-exynos
>> >> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33772
>> >>
>> >> 2) PMU cleanup and refactoring for using DT
>> >> https://www.mail-archive.com/linux-kernel at vger.kernel.org/msg671625.html
>> >>
>> >> 3) Exynos5420 PMU/S2R Series
>> >> http://comments.gmane.org/gmane.linux.kernel.samsung-soc/33898
>> >>
>> >> 4) Exynos5420 CPUIdle Series which populates MCPM suspend/powered_up
>> >> call-backs.
>> >> www.gossamer-threads.com/lists/linux/kernel/1945347
>> >> https://patchwork.kernel.org/patch/4357461/
>> >>
>> >> 5) Exynos5420 MCPM cluster power down support
>> >> http://www.spinics.net/lists/arm-kernel/msg339988.html
>> >>
>> >> 6) TPM reset mask patch
>> >> http://www.spinics.net/lists/arm-kernel/msg341884.html
>> >>
>> >>  arch/arm/mach-exynos/mcpm-exynos.c | 32 +++++++++++++++-------
>> >>  arch/arm/mach-exynos/pm.c          | 54 ++++++++++++++++++++++++++++++++++++--
>> >>  2 files changed, 74 insertions(+), 12 deletions(-)
>> >>
>> >> diff --git a/arch/arm/mach-exynos/mcpm-exynos.c b/arch/arm/mach-exynos/mcpm-exynos.c
>> >> index 2dd51cc..74ad772 100644
>> >> --- a/arch/arm/mach-exynos/mcpm-exynos.c
>> >> +++ b/arch/arm/mach-exynos/mcpm-exynos.c
>> >> @@ -15,6 +15,7 @@
>> >>  #include <linux/delay.h>
>> >>  #include <linux/io.h>
>> >>  #include <linux/of_address.h>
>> >> +#include <linux/syscore_ops.h>
>> >>
>> >>  #include <asm/cputype.h>
>> >>  #include <asm/cp15.h>
>> >> @@ -30,6 +31,8 @@
>> >>  #define EXYNOS5420_USE_ARM_CORE_DOWN_STATE   BIT(29)
>> >>  #define EXYNOS5420_USE_L2_COMMON_UP_STATE    BIT(30)
>> >>
>> >> +static void __iomem *ns_sram_base_addr;
>> >> +
>> >>  /*
>> >>   * The common v7_exit_coherency_flush API could not be used because of the
>> >>   * Erratum 799270 workaround. This macro is the same as the common one (in
>> >> @@ -319,10 +322,26 @@ static const struct of_device_id exynos_dt_mcpm_match[] = {
>> >>       {},
>> >>  };
>> >>
>> >> +static void exynos_mcpm_setup_entry_point(void)
>> >> +{
>> >> +     /*
>> >> +      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> >> +      * as part of secondary_cpu_start().  Let's redirect it to the
>> >> +      * mcpm_entry_point(). This is done during both secondary boot-up as
>> >> +      * well as system resume.
>> >> +      */
>> >> +     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> >> +     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> >> +     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> >> +}
>> >> +
>> >> +static struct syscore_ops exynos_mcpm_syscore_ops = {
>> >> +     .resume = exynos_mcpm_setup_entry_point,
>> >> +};
>> >> +
>> >>  static int __init exynos_mcpm_init(void)
>> >>  {
>> >>       struct device_node *node;
>> >> -     void __iomem *ns_sram_base_addr;
>> >>       unsigned int value, i;
>> >>       int ret;
>> >>
>> >> @@ -389,16 +408,9 @@ static int __init exynos_mcpm_init(void)
>> >>               __raw_writel(value, pmu_base_addr + EXYNOS_COMMON_OPTION(i));
>> >>       }
>> >>
>> >> -     /*
>> >> -      * U-Boot SPL is hardcoded to jump to the start of ns_sram_base_addr
>> >> -      * as part of secondary_cpu_start().  Let's redirect it to the
>> >> -      * mcpm_entry_point().
>> >> -      */
>> >> -     __raw_writel(0xe59f0000, ns_sram_base_addr);     /* ldr r0, [pc, #0] */
>> >> -     __raw_writel(0xe12fff10, ns_sram_base_addr + 4); /* bx  r0 */
>> >> -     __raw_writel(virt_to_phys(mcpm_entry_point), ns_sram_base_addr + 8);
>> >> +     exynos_mcpm_setup_entry_point();
>> >>
>> >> -     iounmap(ns_sram_base_addr);
>> >> +     register_syscore_ops(&exynos_mcpm_syscore_ops);
>> >>
>> >>       return ret;
>> >>  }
>> >> diff --git a/arch/arm/mach-exynos/pm.c b/arch/arm/mach-exynos/pm.c
>> >> index 69cf678..e2ba1e5 100644
>> >> --- a/arch/arm/mach-exynos/pm.c
>> >> +++ b/arch/arm/mach-exynos/pm.c
>> >> @@ -24,6 +24,7 @@
>> >>
>> >>  #include <asm/cacheflush.h>
>> >>  #include <asm/hardware/cache-l2x0.h>
>> >> +#include <asm/mcpm.h>
>> >>  #include <asm/smp_scu.h>
>> >>  #include <asm/suspend.h>
>> >>
>> >> @@ -123,6 +124,17 @@ static int exynos_irq_set_wake(struct irq_data *data, unsigned int state)
>> >>   */
>> >>  void exynos_cpu_power_down(int cpu)
>> >>  {
>> >> +     if (soc_is_exynos5420() && cpu == 0) {
>> >> +             /*
>> >> +              * Bypass power down for CPU0 during suspend. Check for
>> >> +              * the SYS_PWR_REG value to decide if we are suspending
>> >> +              * the system.
>> >> +              */
>> >> +             int val = __raw_readl(pmu_base_addr +
>> >> +                             EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> >> +             if (!(val & S5P_CORE_LOCAL_PWR_EN))
>> >> +                     return;
>> >> +     }
>> >>       pmu_raw_writel(0, EXYNOS_ARM_CORE_CONFIGURATION(cpu));
>> >>  }
>> >>
>> >> @@ -318,7 +330,10 @@ static void exynos_pm_prepare(void)
>> >>
>> >>       /* ensure at least INFORM0 has the resume address */
>> >>
>> >> -     pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> >> +             pmu_raw_writel(virt_to_phys(mcpm_entry_point), S5P_INFORM0);
>> >> +     else
>> >> +             pmu_raw_writel(virt_to_phys(exynos_cpu_resume), S5P_INFORM0);
>> >>
>> >>       if (soc_is_exynos5420()) {
>> >>               tmp = __raw_readl(pmu_base_addr + EXYNOS5_ARM_L2_OPTION);
>> >> @@ -408,6 +423,12 @@ static void exynos_pm_resume(void)
>> >>       unsigned int tmp;
>> >>
>> >>       if (soc_is_exynos5420()) {
>> >> +             /* Restore the CPU0 low power state register */
>> >> +             tmp = __raw_readl(pmu_base_addr +
>> >> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> >> +             pmu_raw_writel(tmp | S5P_CORE_LOCAL_PWR_EN,
>> >> +                     EXYNOS5_ARM_CORE0_SYS_PWR_REG);
>> >> +
>> >>               /* Restore the sysram cpu state register */
>> >>               __raw_writel(exynos5420_cpu_state,
>> >>               sysram_base_addr + EXYNOS5420_CPU_STATE);
>> >> @@ -490,6 +511,28 @@ static struct syscore_ops exynos_pm_syscore_ops = {
>> >>       .resume         = exynos_pm_resume,
>> >>  };
>> >>
>> >> +static int notrace exynos_mcpm_cpu_suspend(unsigned long arg)
>> >> +{
>> >> +     /* MCPM works with HW CPU identifiers */
>> >> +     unsigned int mpidr = read_cpuid_mpidr();
>> >> +     unsigned int cluster = MPIDR_AFFINITY_LEVEL(mpidr, 1);
>> >> +     unsigned int cpu = MPIDR_AFFINITY_LEVEL(mpidr, 0);
>> >> +
>> >> +     __raw_writel(0x0, sysram_base_addr + EXYNOS5420_CPU_STATE);
>> >> +
>> >> +     mcpm_set_entry_vector(cpu, cluster, exynos_cpu_resume);
>> >> +
>> >> +     /*
>> >> +      * Residency value passed to mcpm_cpu_suspend back-end
>> >> +      * has to be given clear semantics. Set to 0 as a
>> >> +      * temporary value.
>> >> +      */
>> >> +     mcpm_cpu_suspend(0);
>> >> +
>> >> +     /* return value != 0 means failure */
>> >> +     return 1;
>> >> +}
>> >> +
>> >>  /*
>> >>   * Suspend Ops
>> >>   */
>> >> @@ -517,10 +560,17 @@ static int exynos_suspend_enter(suspend_state_t state)
>> >>       flush_cache_all();
>> >>       s3c_pm_check_store();
>> >>
>> >> -     ret = cpu_suspend(0, exynos_cpu_suspend);
>> >> +     /* Use the MCPM layer to suspend 5420 which is a multi-cluster SoC */
>> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> >> +             ret = cpu_suspend(0, exynos_mcpm_cpu_suspend);
>> >> +     else
>> >> +             ret = cpu_suspend(0, exynos_cpu_suspend);
>> >>       if (ret)
>> >>               return ret;
>> >>
>> >> +     if (soc_is_exynos5420() && IS_ENABLED(CONFIG_MCPM))
>> >> +             mcpm_cpu_powered_up();
>> >> +
>> >>       s3c_pm_restore_uarts();
>> >>
>> >>       S3C_PMDBG("%s: wakeup stat: %08x\n", __func__,
>> >> --
>> >> 2.0.0
>> >>
>> >>
>>
>

Re: [PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Lorenzo Pieralisi]

On Tue, Jul 08, 2014 at 02:39:37PM +0100, Abhilash Kesavan wrote:
> Hi Lorenzo,
> 
> On Tue, Jul 8, 2014 at 4:23 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> > On Fri, Jul 04, 2014 at 10:21:56PM +0100, Abhilash Kesavan wrote:
> >> On Sat, Jul 5, 2014 at 2:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> >> > On Sat, 5 Jul 2014, Abhilash Kesavan wrote:
> >> >
> >> >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> >> >> Also, restore the entry address setup code post-resume.
> >> >>
> >> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> >> >
> >> > Acked-by: Nicolas Pitre <nico@linaro.org>
> >> This is one of the last missing bits for exynos MCPM. Thanks for your
> >> patient reviews of all the exynos mcpm back-end patches.
> >
> > Sorry for the delay in getting back to you.
> Thanks for your comments.
> >
> > You still need to handle the residency parameter in the MCPM suspend call
> > properly to differentiate between core gating and cluster gating.
> I am not too familiar with the cpuidle driver, but I see only cluster
> power off state listed in the bL cpuidle driver.
> >
> > CPUidle might want to request a state that allows for core gating
> > residency but not cluster gating one, in that case the last man
> > standing must not shutdown the cluster.
> For this, I would have to add another state to the bL cpuidle driver
> or migrate to using your under-review generic idle state patchset.
> Then based on the residency value skip/do cluster power down. Is this
> correct ?
> >
> > This must be handled in the power down function, and it can be done
> > in a separate patch since it is an optimization. Current MCPM interface has
> > all it is needed to handle this properly, and I am keen on patching the
> > CPUidle bL driver to pass the actual residency instead of hardcoding it
> > to 0 (ie on TC2 it was not needed, no core gating).
> I will look at doing this once the s2r patch gets merged.

Ok, let's get it done once the CPUidle bL driver DT version gets merged,
I will add a patch that passes the proper residency and you can patch
the MCPM back-end then, until then CPUidle will be suboptimal but I
guess we can live with that for now.

Thanks,
Lorenzo

[PATCH v7] ARM: EXYNOS: Use MCPM call-backs to support S2R on Exynos5420

[Lorenzo Pieralisi]

On Tue, Jul 08, 2014 at 02:39:37PM +0100, Abhilash Kesavan wrote:
> Hi Lorenzo,
> 
> On Tue, Jul 8, 2014 at 4:23 PM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> > On Fri, Jul 04, 2014 at 10:21:56PM +0100, Abhilash Kesavan wrote:
> >> On Sat, Jul 5, 2014 at 2:30 AM, Nicolas Pitre <nicolas.pitre@linaro.org> wrote:
> >> > On Sat, 5 Jul 2014, Abhilash Kesavan wrote:
> >> >
> >> >> Use the MCPM layer to handle core suspend/resume on Exynos5420.
> >> >> Also, restore the entry address setup code post-resume.
> >> >>
> >> >> Signed-off-by: Abhilash Kesavan <a.kesavan@samsung.com>
> >> >
> >> > Acked-by: Nicolas Pitre <nico@linaro.org>
> >> This is one of the last missing bits for exynos MCPM. Thanks for your
> >> patient reviews of all the exynos mcpm back-end patches.
> >
> > Sorry for the delay in getting back to you.
> Thanks for your comments.
> >
> > You still need to handle the residency parameter in the MCPM suspend call
> > properly to differentiate between core gating and cluster gating.
> I am not too familiar with the cpuidle driver, but I see only cluster
> power off state listed in the bL cpuidle driver.
> >
> > CPUidle might want to request a state that allows for core gating
> > residency but not cluster gating one, in that case the last man
> > standing must not shutdown the cluster.
> For this, I would have to add another state to the bL cpuidle driver
> or migrate to using your under-review generic idle state patchset.
> Then based on the residency value skip/do cluster power down. Is this
> correct ?
> >
> > This must be handled in the power down function, and it can be done
> > in a separate patch since it is an optimization. Current MCPM interface has
> > all it is needed to handle this properly, and I am keen on patching the
> > CPUidle bL driver to pass the actual residency instead of hardcoding it
> > to 0 (ie on TC2 it was not needed, no core gating).
> I will look at doing this once the s2r patch gets merged.

Ok, let's get it done once the CPUidle bL driver DT version gets merged,
I will add a patch that passes the proper residency and you can patch
the MCPM back-end then, until then CPUidle will be suboptimal but I
guess we can live with that for now.

Thanks,
Lorenzo