[PATCH v2 00/12] MIPS CPC fixup and CPU Idle for MIPSr6 CPUs

[PATCH v2 00/12] MIPS CPC fixup and CPU Idle for MIPSr6 CPUs

[Matt Redfearn]

This updated series incorporates comments from Peter Zijlstra on v1
around the barriers in pm-cps.c.

This series fixes a small issue with the CPC driver when A CM3 is
present, where a redundant lock was taken.

There are then additions to the pm-cps driver to add support for R6 CPUs
such as the I6400, and additionally the CM3 present in the I6400.

Finally we enable the cpuidle-cps driver for MIPSr6 CPUs.

Applies atop v4.8-rc4


Changes in v2:
Update comments on barriers
Add new patch to define standard MIPS barrier types
Use architecturally standard lightweight sync types rather than
selecting CPU specific ones.

Matt Redfearn (12):
  MIPS: CPC: Convert bare 'unsigned' to 'unsigned int'
  MIPS: CPC: Avoid lock when MIPS CM >= 3 is present
  MIPS: pm-cps: Change FSB workaround to CPU blacklist
  MIPS: pm-cps: Update comments on barrier instructions
  MIPS: Barrier: Add definitions of SYNC stype values
  MIPS: pm-cps: Use MIPS standard lightweight ordering barrier
  MIPS: pm-cps: Use MIPS standard completion barrier
  MIPS: pm-cps: Remove selection of sync types
  MIPS: pm-cps: Add MIPSr6 CPU support
  MIPS: pm-cps: Support CM3 changes to Coherence Enable Register
  MIPS: SMP: Wrap call to mips_cpc_lock_other in mips_cm_lock_other
  cpuidle: cpuidle-cps: Enable use with MIPSr6 CPUs.

 arch/mips/include/asm/barrier.h |  96 +++++++++++++++++++++++++++++++++
 arch/mips/include/asm/mips-cm.h |   1 +
 arch/mips/include/asm/pm-cps.h  |   6 ++-
 arch/mips/kernel/mips-cpc.c     |  17 ++++--
 arch/mips/kernel/pm-cps.c       | 115 +++++++++++++++++++---------------------
 arch/mips/kernel/smp.c          |   2 +
 drivers/cpuidle/Kconfig.mips    |   2 +-
 drivers/cpuidle/cpuidle-cps.c   |   2 +-
 8 files changed, 173 insertions(+), 68 deletions(-)

-- 
2.7.4

[PATCH v2 01/12] MIPS: CPC: Convert bare 'unsigned' to 'unsigned int'

[Matt Redfearn]

Checkpatch complains about use of bare unsigned type.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2: None

 arch/mips/kernel/mips-cpc.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/mips/kernel/mips-cpc.c b/arch/mips/kernel/mips-cpc.c
index 566b8d2c092c..0e337f55ac60 100644
--- a/arch/mips/kernel/mips-cpc.c
+++ b/arch/mips/kernel/mips-cpc.c
@@ -52,7 +52,7 @@ static phys_addr_t mips_cpc_phys_base(void)
 int mips_cpc_probe(void)
 {
 	phys_addr_t addr;
-	unsigned cpu;
+	unsigned int cpu;
 
 	for_each_possible_cpu(cpu)
 		spin_lock_init(&per_cpu(cpc_core_lock, cpu));
@@ -70,7 +70,7 @@ int mips_cpc_probe(void)
 
 void mips_cpc_lock_other(unsigned int core)
 {
-	unsigned curr_core;
+	unsigned int curr_core;
 	preempt_disable();
 	curr_core = current_cpu_data.core;
 	spin_lock_irqsave(&per_cpu(cpc_core_lock, curr_core),
@@ -86,7 +86,7 @@ void mips_cpc_lock_other(unsigned int core)
 
 void mips_cpc_unlock_other(void)
 {
-	unsigned curr_core = current_cpu_data.core;
+	unsigned int curr_core = current_cpu_data.core;
 	spin_unlock_irqrestore(&per_cpu(cpc_core_lock, curr_core),
 			       per_cpu(cpc_core_lock_flags, curr_core));
 	preempt_enable();
-- 
2.7.4

[PATCH v2 02/12] MIPS: CPC: Avoid lock when MIPS CM >= 3 is present

[Matt Redfearn]

MIPS CM version 3 removed the CPC_CL_OTHER register and instead the
CM_CL_OTHER register is used to redirect the CPC_OTHER region. As such,
we should not write the unimplmented register and can avoid the
spinlock as well.
These lock functions should aleady be called within the context of a
mips_cm_{lock,unlock}_other pair ensuring the correct CPC_OTHER region
will be accessed.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2: None

 arch/mips/kernel/mips-cpc.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/arch/mips/kernel/mips-cpc.c b/arch/mips/kernel/mips-cpc.c
index 0e337f55ac60..2a45867d3b4f 100644
--- a/arch/mips/kernel/mips-cpc.c
+++ b/arch/mips/kernel/mips-cpc.c
@@ -71,6 +71,11 @@ int mips_cpc_probe(void)
 void mips_cpc_lock_other(unsigned int core)
 {
 	unsigned int curr_core;
+
+	if (mips_cm_revision() >= CM_REV_CM3)
+		/* Systems with CM >= 3 lock the CPC via mips_cm_lock_other */
+		return;
+
 	preempt_disable();
 	curr_core = current_cpu_data.core;
 	spin_lock_irqsave(&per_cpu(cpc_core_lock, curr_core),
@@ -86,7 +91,13 @@ void mips_cpc_lock_other(unsigned int core)
 
 void mips_cpc_unlock_other(void)
 {
-	unsigned int curr_core = current_cpu_data.core;
+	unsigned int curr_core;
+
+	if (mips_cm_revision() >= CM_REV_CM3)
+		/* Systems with CM >= 3 lock the CPC via mips_cm_lock_other */
+		return;
+
+	curr_core = current_cpu_data.core;
 	spin_unlock_irqrestore(&per_cpu(cpc_core_lock, curr_core),
 			       per_cpu(cpc_core_lock_flags, curr_core));
 	preempt_enable();
-- 
2.7.4

[PATCH v2 03/12] MIPS: pm-cps: Change FSB workaround to CPU blacklist

[Matt Redfearn]

The check for whether a CPU required the FSB flush workaround
previously required every CPU not requiring it to be whitelisted. That
approach does not scale well as new CPUs are introduced so change the
default from a WARN and returning an error to just returning 0. Any CPUs
requiring the workaround can then be added to the blacklist.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2: None

 arch/mips/kernel/pm-cps.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
index 5b31a9405ebc..2faa227a032e 100644
--- a/arch/mips/kernel/pm-cps.c
+++ b/arch/mips/kernel/pm-cps.c
@@ -272,14 +272,9 @@ static int __init cps_gen_flush_fsb(u32 **pp, struct uasm_label **pl,
 		/* On older ones it's unavailable */
 		return -1;
 
-	/* CPUs which do not require the workaround */
-	case CPU_P5600:
-	case CPU_I6400:
-		return 0;
-
 	default:
-		WARN_ONCE(1, "pm-cps: FSB flush unsupported for this CPU\n");
-		return -1;
+		/* Assume that the CPU does not need this workaround */
+		return 0;
 	}
 
 	/*
-- 
2.7.4

[PATCH v2 04/12] MIPS: pm-cps: Update comments on barrier instructions

[Matt Redfearn]

This code makes large use of barriers, which had quite vague
descriptions. Update the comments to make the choice of barrier and
reason for it more clear.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>

---

Changes in v2:
Update comments on barriers

 arch/mips/kernel/pm-cps.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
index 2faa227a032e..7e8d4aa22233 100644
--- a/arch/mips/kernel/pm-cps.c
+++ b/arch/mips/kernel/pm-cps.c
@@ -315,7 +315,7 @@ static int __init cps_gen_flush_fsb(u32 **pp, struct uasm_label **pl,
 			     i * line_size * line_stride, t0);
 	}
 
-	/* Completion barrier */
+	/* Barrier ensuring previous cache invalidates are complete */
 	uasm_i_sync(pp, stype_memory);
 	uasm_i_ehb(pp);
 
@@ -414,7 +414,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 		uasm_il_beqz(&p, &r, t2, lbl_incready);
 		uasm_i_addiu(&p, t1, t1, 1);
 
-		/* Ordering barrier */
+		/* Barrier ensuring all CPUs see the updated r_nc_count value */
 		uasm_i_sync(&p, stype_ordering);
 
 		/*
@@ -467,7 +467,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 	cps_gen_cache_routine(&p, &l, &r, &cpu_data[cpu].dcache,
 			      Index_Writeback_Inv_D, lbl_flushdcache);
 
-	/* Completion barrier */
+	/* Barrier ensuring previous cache invalidates are complete */
 	uasm_i_sync(&p, stype_memory);
 	uasm_i_ehb(&p);
 
@@ -480,7 +480,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 	uasm_i_sw(&p, t0, 0, r_pcohctl);
 	uasm_i_lw(&p, t0, 0, r_pcohctl);
 
-	/* Sync to ensure previous interventions are complete */
+	/* Barrier to ensure write to coherence control is complete */
 	uasm_i_sync(&p, stype_intervention);
 	uasm_i_ehb(&p);
 
@@ -526,7 +526,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 			goto gen_done;
 		}
 
-		/* Completion barrier */
+		/* Barrier to ensure write to CPC command is complete */
 		uasm_i_sync(&p, stype_memory);
 		uasm_i_ehb(&p);
 	}
@@ -561,7 +561,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 	uasm_i_sw(&p, t0, 0, r_pcohctl);
 	uasm_i_lw(&p, t0, 0, r_pcohctl);
 
-	/* Completion barrier */
+	/* Barrier to ensure write to coherence control is complete */
 	uasm_i_sync(&p, stype_memory);
 	uasm_i_ehb(&p);
 
@@ -575,7 +575,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 		uasm_il_beqz(&p, &r, t2, lbl_decready);
 		uasm_i_andi(&p, v0, t1, (1 << fls(smp_num_siblings)) - 1);
 
-		/* Ordering barrier */
+		/* Barrier ensuring all CPUs see the updated r_nc_count value */
 		uasm_i_sync(&p, stype_ordering);
 	}
 
@@ -597,7 +597,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 		 */
 		uasm_build_label(&l, p, lbl_secondary_cont);
 
-		/* Ordering barrier */
+		/* Barrier ensuring all CPUs see the updated r_nc_count value */
 		uasm_i_sync(&p, stype_ordering);
 	}
 
-- 
2.7.4

[PATCH v2 05/12] MIPS: Barrier: Add definitions of SYNC stype values

[Matt Redfearn]

Add the definitions of sync stype 0 (global completion barrier) and sync
stype 0x10 (local ordering barrier) to barrier.h for use with the sync
instruction.

These types are defined by the MIPS Instruction Set since R2 of the
architecture and are documented in document MD00087 table 6.5.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2:
Add new patch to define standard MIPS barrier types

 arch/mips/include/asm/barrier.h | 96 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 96 insertions(+)

diff --git a/arch/mips/include/asm/barrier.h b/arch/mips/include/asm/barrier.h
index d296633d890e..a5eb1bb199a7 100644
--- a/arch/mips/include/asm/barrier.h
+++ b/arch/mips/include/asm/barrier.h
@@ -10,6 +10,102 @@
 
 #include <asm/addrspace.h>
 
+/*
+ * Sync types defined by the MIPS architecture (document MD00087 table 6.5)
+ * These values are used with the sync instruction to perform memory barriers.
+ * Types of ordering guarantees available through the SYNC instruction:
+ * - Completion Barriers
+ * - Ordering Barriers
+ * As compared to the completion barrier, the ordering barrier is a
+ * lighter-weight operation as it does not require the specified instructions
+ * before the SYNC to be already completed. Instead it only requires that those
+ * specified instructions which are subsequent to the SYNC in the instruction
+ * stream are never re-ordered for processing ahead of the specified
+ * instructions which are before the SYNC in the instruction stream.
+ * This potentially reduces how many cycles the barrier instruction must stall
+ * before it completes.
+ * Implementations that do not use any of the non-zero values of stype to define
+ * different barriers, such as ordering barriers, must make those stype values
+ * act the same as stype zero.
+ */
+
+/*
+ * Completion barriers:
+ * - Every synchronizable specified memory instruction (loads or stores or both)
+ *   that occurs in the instruction stream before the SYNC instruction must be
+ *   already globally performed before any synchronizable specified memory
+ *   instructions that occur after the SYNC are allowed to be performed, with
+ *   respect to any other processor or coherent I/O module.
+ *
+ * - The barrier does not guarantee the order in which instruction fetches are
+ *   performed.
+ *
+ * - A stype value of zero will always be defined such that it performs the most
+ *   complete set of synchronization operations that are defined.This means
+ *   stype zero always does a completion barrier that affects both loads and
+ *   stores preceding the SYNC instruction and both loads and stores that are
+ *   subsequent to the SYNC instruction. Non-zero values of stype may be defined
+ *   by the architecture or specific implementations to perform synchronization
+ *   behaviors that are less complete than that of stype zero. If an
+ *   implementation does not use one of these non-zero values to define a
+ *   different synchronization behavior, then that non-zero value of stype must
+ *   act the same as stype zero completion barrier. This allows software written
+ *   for an implementation with a lighter-weight barrier to work on another
+ *   implementation which only implements the stype zero completion barrier.
+ *
+ * - A completion barrier is required, potentially in conjunction with SSNOP (in
+ *   Release 1 of the Architecture) or EHB (in Release 2 of the Architecture),
+ *   to guarantee that memory reference results are visible across operating
+ *   mode changes. For example, a completion barrier is required on some
+ *   implementations on entry to and exit from Debug Mode to guarantee that
+ *   memory effects are handled correctly.
+ */
+
+/*
+ * stype 0 - A completion barrier that affects preceding loads and stores and
+ * subsequent loads and stores.
+ * Older instructions which must reach the load/store ordering point before the
+ * SYNC instruction completes: Loads, Stores
+ * Younger instructions which must reach the load/store ordering point only
+ * after the SYNC instruction completes: Loads, Stores
+ * Older instructions which must be globally performed when the SYNC instruction
+ * completes: Loads, Stores
+ */
+#define STYPE_SYNC 0x0
+
+/*
+ * Ordering barriers:
+ * - Every synchronizable specified memory instruction (loads or stores or both)
+ *   that occurs in the instruction stream before the SYNC instruction must
+ *   reach a stage in the load/store datapath after which no instruction
+ *   re-ordering is possible before any synchronizable specified memory
+ *   instruction which occurs after the SYNC instruction in the instruction
+ *   stream reaches the same stage in the load/store datapath.
+ *
+ * - If any memory instruction before the SYNC instruction in program order,
+ *   generates a memory request to the external memory and any memory
+ *   instruction after the SYNC instruction in program order also generates a
+ *   memory request to external memory, the memory request belonging to the
+ *   older instruction must be globally performed before the time the memory
+ *   request belonging to the younger instruction is globally performed.
+ *
+ * - The barrier does not guarantee the order in which instruction fetches are
+ *   performed.
+ */
+
+/*
+ * stype 0x10 - An ordering barrier that affects preceding loads and stores and
+ * subsequent loads and stores.
+ * Older instructions which must reach the load/store ordering point before the
+ * SYNC instruction completes: Loads, Stores
+ * Younger instructions which must reach the load/store ordering point only
+ * after the SYNC instruction completes: Loads, Stores
+ * Older instructions which must be globally performed when the SYNC instruction
+ * completes: N/A
+ */
+#define STYPE_SYNC_MB 0x10
+
+
 #ifdef CONFIG_CPU_HAS_SYNC
 #define __sync()				\
 	__asm__ __volatile__(			\
-- 
2.7.4

[PATCH v2 06/12] MIPS: pm-cps: Use MIPS standard lightweight ordering barrier

[Matt Redfearn]

Since R2 of the MIPS architecture, SYNC(0x10) has been an optional but
architecturally defined ordering barrier. If a CPU does not implement it,
the arch specifies that it must fall back to SYNC(0).

In places where we require that the instruction stream not be reordered,
but do not require that loads / stores are gloablly completed, use the
defined standard sync stype.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2: None

 arch/mips/kernel/pm-cps.c | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
index 7e8d4aa22233..d7037fe00d1c 100644
--- a/arch/mips/kernel/pm-cps.c
+++ b/arch/mips/kernel/pm-cps.c
@@ -76,7 +76,6 @@ static struct uasm_reloc relocs[32] __initdata;
 /* CPU dependant sync types */
 static unsigned stype_intervention;
 static unsigned stype_memory;
-static unsigned stype_ordering;
 
 enum mips_reg {
 	zero, at, v0, v1, a0, a1, a2, a3,
@@ -406,7 +405,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 
 	if (coupled_coherence) {
 		/* Increment ready_count */
-		uasm_i_sync(&p, stype_ordering);
+		uasm_i_sync(&p, STYPE_SYNC_MB);
 		uasm_build_label(&l, p, lbl_incready);
 		uasm_i_ll(&p, t1, 0, r_nc_count);
 		uasm_i_addiu(&p, t2, t1, 1);
@@ -415,7 +414,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 		uasm_i_addiu(&p, t1, t1, 1);
 
 		/* Barrier ensuring all CPUs see the updated r_nc_count value */
-		uasm_i_sync(&p, stype_ordering);
+		uasm_i_sync(&p, STYPE_SYNC_MB);
 
 		/*
 		 * If this is the last VPE to become ready for non-coherence
@@ -568,7 +567,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 	if (coupled_coherence && (state == CPS_PM_NC_WAIT)) {
 		/* Decrement ready_count */
 		uasm_build_label(&l, p, lbl_decready);
-		uasm_i_sync(&p, stype_ordering);
+		uasm_i_sync(&p, STYPE_SYNC_MB);
 		uasm_i_ll(&p, t1, 0, r_nc_count);
 		uasm_i_addiu(&p, t2, t1, -1);
 		uasm_i_sc(&p, t2, 0, r_nc_count);
@@ -576,7 +575,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 		uasm_i_andi(&p, v0, t1, (1 << fls(smp_num_siblings)) - 1);
 
 		/* Barrier ensuring all CPUs see the updated r_nc_count value */
-		uasm_i_sync(&p, stype_ordering);
+		uasm_i_sync(&p, STYPE_SYNC_MB);
 	}
 
 	if (coupled_coherence && (state == CPS_PM_CLOCK_GATED)) {
@@ -598,7 +597,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 		uasm_build_label(&l, p, lbl_secondary_cont);
 
 		/* Barrier ensuring all CPUs see the updated r_nc_count value */
-		uasm_i_sync(&p, stype_ordering);
+		uasm_i_sync(&p, STYPE_SYNC_MB);
 	}
 
 	/* The core is coherent, time to return to C code */
@@ -677,7 +676,6 @@ static int __init cps_pm_init(void)
 	case CPU_I6400:
 		stype_intervention = 0x2;
 		stype_memory = 0x3;
-		stype_ordering = 0x10;
 		break;
 
 	default:
-- 
2.7.4

[PATCH v2 07/12] MIPS: pm-cps: Use MIPS standard completion barrier

[Matt Redfearn]

SYNC type 0 is defined in the MIPS architecture as a completion barrier
where all loads/stores in the pipeline before the sync instruction must
complete before any loads/stores subsequent to the sync instruction.

In places where we require loads / stores be globally completed, use the
standard completion sync stype.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2: None

 arch/mips/kernel/pm-cps.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
index d7037fe00d1c..953ff0db9061 100644
--- a/arch/mips/kernel/pm-cps.c
+++ b/arch/mips/kernel/pm-cps.c
@@ -315,7 +315,7 @@ static int __init cps_gen_flush_fsb(u32 **pp, struct uasm_label **pl,
 	}
 
 	/* Barrier ensuring previous cache invalidates are complete */
-	uasm_i_sync(pp, stype_memory);
+	uasm_i_sync(pp, STYPE_SYNC);
 	uasm_i_ehb(pp);
 
 	/* Check whether the pipeline stalled due to the FSB being full */
@@ -467,7 +467,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 			      Index_Writeback_Inv_D, lbl_flushdcache);
 
 	/* Barrier ensuring previous cache invalidates are complete */
-	uasm_i_sync(&p, stype_memory);
+	uasm_i_sync(&p, STYPE_SYNC);
 	uasm_i_ehb(&p);
 
 	/*
@@ -480,7 +480,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 	uasm_i_lw(&p, t0, 0, r_pcohctl);
 
 	/* Barrier to ensure write to coherence control is complete */
-	uasm_i_sync(&p, stype_intervention);
+	uasm_i_sync(&p, STYPE_SYNC);
 	uasm_i_ehb(&p);
 
 	/* Disable coherence */
@@ -526,7 +526,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 		}
 
 		/* Barrier to ensure write to CPC command is complete */
-		uasm_i_sync(&p, stype_memory);
+		uasm_i_sync(&p, STYPE_SYNC);
 		uasm_i_ehb(&p);
 	}
 
@@ -561,7 +561,7 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 	uasm_i_lw(&p, t0, 0, r_pcohctl);
 
 	/* Barrier to ensure write to coherence control is complete */
-	uasm_i_sync(&p, stype_memory);
+	uasm_i_sync(&p, STYPE_SYNC);
 	uasm_i_ehb(&p);
 
 	if (coupled_coherence && (state == CPS_PM_NC_WAIT)) {
-- 
2.7.4

[PATCH v2 08/12] MIPS: pm-cps: Remove selection of sync types

[Matt Redfearn]

Instead of selecting an implementation or vendor specific sync type for
the required sync operations, always use the architecturally mandated
sync types which previous patches have put in place. The selection of
special sync types is now redundant an can be removed.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>

---

Changes in v2:
Use architecturally standard lightweight sync types rather than
selecting CPU specific ones.

 arch/mips/kernel/pm-cps.c | 19 -------------------
 1 file changed, 19 deletions(-)

diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
index 953ff0db9061..b3a7d36ada5a 100644
--- a/arch/mips/kernel/pm-cps.c
+++ b/arch/mips/kernel/pm-cps.c
@@ -73,10 +73,6 @@ DEFINE_PER_CPU_ALIGNED(struct mips_static_suspend_state, cps_cpu_state);
 static struct uasm_label labels[32] __initdata;
 static struct uasm_reloc relocs[32] __initdata;
 
-/* CPU dependant sync types */
-static unsigned stype_intervention;
-static unsigned stype_memory;
-
 enum mips_reg {
 	zero, at, v0, v1, a0, a1, a2, a3,
 	t0, t1, t2, t3, t4, t5, t6, t7,
@@ -667,21 +663,6 @@ static int __init cps_pm_init(void)
 	unsigned cpu;
 	int err;
 
-	/* Detect appropriate sync types for the system */
-	switch (current_cpu_data.cputype) {
-	case CPU_INTERAPTIV:
-	case CPU_PROAPTIV:
-	case CPU_M5150:
-	case CPU_P5600:
-	case CPU_I6400:
-		stype_intervention = 0x2;
-		stype_memory = 0x3;
-		break;
-
-	default:
-		pr_warn("Power management is using heavyweight sync 0\n");
-	}
-
 	/* A CM is required for all non-coherent states */
 	if (!mips_cm_present()) {
 		pr_warn("pm-cps: no CM, non-coherent states unavailable\n");
-- 
2.7.4

[PATCH v2 09/12] MIPS: pm-cps: Add MIPSr6 CPU support

[Matt Redfearn]

This patch adds support for CPUs implementing the MIPSr6 ISA to the CPS
power management code. Three changes are necessary:

1. In MIPSr6, coupled coherence is necessary when CPUS implement multiple
   Virtual Processors (VPs).

2. MIPSr6 virtual processors are more like real cores and cannot yield
   to other VPs on the same core, so drop the MT ASE yield instruction.

3. To halt a MIPSr6 VP, the CPC VP_STOP register is used rather than the
   MT ASE TCHalt CP0 register.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2: None

 arch/mips/include/asm/pm-cps.h |  6 ++++--
 arch/mips/kernel/pm-cps.c      | 22 ++++++++++++++++++----
 2 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/arch/mips/include/asm/pm-cps.h b/arch/mips/include/asm/pm-cps.h
index 625eda53d571..89d58d80b77b 100644
--- a/arch/mips/include/asm/pm-cps.h
+++ b/arch/mips/include/asm/pm-cps.h
@@ -13,10 +13,12 @@
 
 /*
  * The CM & CPC can only handle coherence & power control on a per-core basis,
- * thus in an MT system the VPEs within each core are coupled and can only
+ * thus in an MT system the VP(E)s within each core are coupled and can only
  * enter or exit states requiring CM or CPC assistance in unison.
  */
-#ifdef CONFIG_MIPS_MT
+#if defined(CONFIG_CPU_MIPSR6)
+# define coupled_coherence cpu_has_vp
+#elif defined(CONFIG_MIPS_MT)
 # define coupled_coherence cpu_has_mipsmt
 #else
 # define coupled_coherence 0
diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
index b3a7d36ada5a..440e79259566 100644
--- a/arch/mips/kernel/pm-cps.c
+++ b/arch/mips/kernel/pm-cps.c
@@ -129,7 +129,7 @@ int cps_pm_enter_state(enum cps_pm_state state)
 		return -EINVAL;
 
 	/* Calculate which coupled CPUs (VPEs) are online */
-#ifdef CONFIG_MIPS_MT
+#if defined(CONFIG_MIPS_MT) || defined(CONFIG_CPU_MIPSR6)
 	if (cpu_online(cpu)) {
 		cpumask_and(coupled_mask, cpu_online_mask,
 			    &cpu_sibling_map[cpu]);
@@ -431,7 +431,8 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 			uasm_i_lw(&p, t0, 0, r_nc_count);
 			uasm_il_bltz(&p, &r, t0, lbl_secondary_cont);
 			uasm_i_ehb(&p);
-			uasm_i_yield(&p, zero, t1);
+			if (cpu_has_mipsmt)
+				uasm_i_yield(&p, zero, t1);
 			uasm_il_b(&p, &r, lbl_poll_cont);
 			uasm_i_nop(&p);
 		} else {
@@ -439,8 +440,21 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 			 * The core will lose power & this VPE will not continue
 			 * so it can simply halt here.
 			 */
-			uasm_i_addiu(&p, t0, zero, TCHALT_H);
-			uasm_i_mtc0(&p, t0, 2, 4);
+			if (cpu_has_mipsmt) {
+				/* Halt the VPE via C0 tchalt register */
+				uasm_i_addiu(&p, t0, zero, TCHALT_H);
+				uasm_i_mtc0(&p, t0, 2, 4);
+			} else if (cpu_has_vp) {
+				/* Halt the VP via the CPC VP_STOP register */
+				unsigned int vpe_id;
+
+				vpe_id = cpu_vpe_id(&cpu_data[cpu]);
+				uasm_i_addiu(&p, t0, zero, 1 << vpe_id);
+				UASM_i_LA(&p, t1, (long)addr_cpc_cl_vp_stop());
+				uasm_i_sw(&p, t0, 0, t1);
+			} else {
+				BUG();
+			}
 			uasm_build_label(&l, p, lbl_secondary_hang);
 			uasm_il_b(&p, &r, lbl_secondary_hang);
 			uasm_i_nop(&p);
-- 
2.7.4

[PATCH v2 10/12] MIPS: pm-cps: Support CM3 changes to Coherence Enable Register

[Matt Redfearn]

MIPS CM3 changed the management of coherence. Instead of a coherence
control register with a bitmask of coherent domains, CM3 simply has a
coherence enable register with a single bit to enable coherence of the
local core. Support this by clearing and setting this single bit to
disable / enable coherence.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2: None

 arch/mips/include/asm/mips-cm.h |  1 +
 arch/mips/kernel/pm-cps.c       | 31 ++++++++++++++++++-------------
 2 files changed, 19 insertions(+), 13 deletions(-)

diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-cm.h
index 58e7874e9347..ac30981a8360 100644
--- a/arch/mips/include/asm/mips-cm.h
+++ b/arch/mips/include/asm/mips-cm.h
@@ -359,6 +359,7 @@ BUILD_CM_Cx_R_(tcid_8_priority,	0x80)
 /* GCR_Cx_COHERENCE register fields */
 #define CM_GCR_Cx_COHERENCE_COHDOMAINEN_SHF	0
 #define CM_GCR_Cx_COHERENCE_COHDOMAINEN_MSK	(_ULCAST_(0xff) << 0)
+#define CM3_GCR_Cx_COHERENCE_COHEN_MSK		(_ULCAST_(0x1) << 0)
 
 /* GCR_Cx_CONFIG register fields */
 #define CM_GCR_Cx_CONFIG_IOCUTYPE_SHF		10
diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c
index 440e79259566..05bcdedcf9bd 100644
--- a/arch/mips/kernel/pm-cps.c
+++ b/arch/mips/kernel/pm-cps.c
@@ -480,18 +480,20 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 	uasm_i_sync(&p, STYPE_SYNC);
 	uasm_i_ehb(&p);
 
-	/*
-	 * Disable all but self interventions. The load from COHCTL is defined
-	 * by the interAptiv & proAptiv SUMs as ensuring that the operation
-	 * resulting from the preceding store is complete.
-	 */
-	uasm_i_addiu(&p, t0, zero, 1 << cpu_data[cpu].core);
-	uasm_i_sw(&p, t0, 0, r_pcohctl);
-	uasm_i_lw(&p, t0, 0, r_pcohctl);
-
-	/* Barrier to ensure write to coherence control is complete */
-	uasm_i_sync(&p, STYPE_SYNC);
-	uasm_i_ehb(&p);
+	if (mips_cm_revision() < CM_REV_CM3) {
+		/*
+		* Disable all but self interventions. The load from COHCTL is
+		* defined by the interAptiv & proAptiv SUMs as ensuring that the
+		*  operation resulting from the preceding store is complete.
+		*/
+		uasm_i_addiu(&p, t0, zero, 1 << cpu_data[cpu].core);
+		uasm_i_sw(&p, t0, 0, r_pcohctl);
+		uasm_i_lw(&p, t0, 0, r_pcohctl);
+
+		/* Barrier to ensure write to coherence control is complete */
+		uasm_i_sync(&p, STYPE_SYNC);
+		uasm_i_ehb(&p);
+	}
 
 	/* Disable coherence */
 	uasm_i_sw(&p, zero, 0, r_pcohctl);
@@ -566,7 +568,10 @@ static void * __init cps_gen_entry_code(unsigned cpu, enum cps_pm_state state)
 	 * will run this. The first will actually re-enable coherence & the
 	 * rest will just be performing a rather unusual nop.
 	 */
-	uasm_i_addiu(&p, t0, zero, CM_GCR_Cx_COHERENCE_COHDOMAINEN_MSK);
+	uasm_i_addiu(&p, t0, zero, mips_cm_revision() < CM_REV_CM3
+				? CM_GCR_Cx_COHERENCE_COHDOMAINEN_MSK
+				: CM3_GCR_Cx_COHERENCE_COHEN_MSK);
+
 	uasm_i_sw(&p, t0, 0, r_pcohctl);
 	uasm_i_lw(&p, t0, 0, r_pcohctl);
 
-- 
2.7.4

[PATCH v2 11/12] MIPS: SMP: Wrap call to mips_cpc_lock_other in mips_cm_lock_other

[Matt Redfearn]

All calls to mips_cpc_lock_other should be wrapped in
mips_cm_lock_other. This only matters if the system has CM3 and is using
cpu idle, since otherwise a) the CPC lock is sufficent for CM < 3 and b)
any systems with CM > 3 have not been able to use cpu idle until now.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>
---

Changes in v2: None

 arch/mips/kernel/smp.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index f95f094f36e4..78ccd5388654 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -192,9 +192,11 @@ void mips_smp_send_ipi_mask(const struct cpumask *mask, unsigned int action)
 				continue;
 
 			while (!cpumask_test_cpu(cpu, &cpu_coherent_mask)) {
+				mips_cm_lock_other(core, 0);
 				mips_cpc_lock_other(core);
 				write_cpc_co_cmd(CPC_Cx_CMD_PWRUP);
 				mips_cpc_unlock_other();
+				mips_cm_unlock_other();
 			}
 		}
 	}
-- 
2.7.4

[PATCH v2 12/12] cpuidle: cpuidle-cps: Enable use with MIPSr6 CPUs.

[Matt Redfearn]

This patch enables the MIPS CPS driver for MIPSr6 CPUs.

Signed-off-by: Matt Redfearn <matt.redfearn@imgtec.com>
Reviewed-by: Paul Burton <paul.burton@imgtec.com>

---

Changes in v2: None

 drivers/cpuidle/Kconfig.mips  | 2 +-
 drivers/cpuidle/cpuidle-cps.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/cpuidle/Kconfig.mips b/drivers/cpuidle/Kconfig.mips
index 4102be01d06a..512ee37b374b 100644
--- a/drivers/cpuidle/Kconfig.mips
+++ b/drivers/cpuidle/Kconfig.mips
@@ -5,7 +5,7 @@ config MIPS_CPS_CPUIDLE
 	bool "CPU Idle driver for MIPS CPS platforms"
 	depends on CPU_IDLE && MIPS_CPS
 	depends on SYS_SUPPORTS_MIPS_CPS
-	select ARCH_NEEDS_CPU_IDLE_COUPLED if MIPS_MT
+	select ARCH_NEEDS_CPU_IDLE_COUPLED if MIPS_MT || CPU_MIPSR6
 	select GENERIC_CLOCKEVENTS_BROADCAST if SMP
 	select MIPS_CPS_PM
 	default y
diff --git a/drivers/cpuidle/cpuidle-cps.c b/drivers/cpuidle/cpuidle-cps.c
index 1adb6980b707..926ba9871c62 100644
--- a/drivers/cpuidle/cpuidle-cps.c
+++ b/drivers/cpuidle/cpuidle-cps.c
@@ -163,7 +163,7 @@ static int __init cps_cpuidle_init(void)
 		core = cpu_data[cpu].core;
 		device = &per_cpu(cpuidle_dev, cpu);
 		device->cpu = cpu;
-#ifdef CONFIG_MIPS_MT
+#ifdef CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED
 		cpumask_copy(&device->coupled_cpus, &cpu_sibling_map[cpu]);
 #endif
 
-- 
2.7.4

Re: [PATCH v2 05/12] MIPS: Barrier: Add definitions of SYNC stype values

[Peter Zijlstra]

This seems to be verbatim copies of the text from the manual. A few
questions below.

On Wed, Sep 07, 2016 at 10:45:13AM +0100, Matt Redfearn wrote:

> +/*
> + * Completion barriers:
> + * - Every synchronizable specified memory instruction (loads or stores or both)
> + *   that occurs in the instruction stream before the SYNC instruction must be
> + *   already globally performed before any synchronizable specified memory
> + *   instructions that occur after the SYNC are allowed to be performed, with
> + *   respect to any other processor or coherent I/O module.
> + *
> + * - The barrier does not guarantee the order in which instruction fetches are
> + *   performed.
> + *
> + * - A stype value of zero will always be defined such that it performs the most
> + *   complete set of synchronization operations that are defined.This means
> + *   stype zero always does a completion barrier that affects both loads and
> + *   stores preceding the SYNC instruction and both loads and stores that are
> + *   subsequent to the SYNC instruction. Non-zero values of stype may be defined
> + *   by the architecture or specific implementations to perform synchronization
> + *   behaviors that are less complete than that of stype zero. If an
> + *   implementation does not use one of these non-zero values to define a
> + *   different synchronization behavior, then that non-zero value of stype must
> + *   act the same as stype zero completion barrier. This allows software written
> + *   for an implementation with a lighter-weight barrier to work on another
> + *   implementation which only implements the stype zero completion barrier.
> + *
> + * - A completion barrier is required, potentially in conjunction with SSNOP (in
> + *   Release 1 of the Architecture) or EHB (in Release 2 of the Architecture),
> + *   to guarantee that memory reference results are visible across operating
> + *   mode changes. For example, a completion barrier is required on some
> + *   implementations on entry to and exit from Debug Mode to guarantee that
> + *   memory effects are handled correctly.
> + */
> +
> +/*
> + * stype 0 - A completion barrier that affects preceding loads and stores and
> + * subsequent loads and stores.
> + * Older instructions which must reach the load/store ordering point before the
> + * SYNC instruction completes: Loads, Stores
> + * Younger instructions which must reach the load/store ordering point only
> + * after the SYNC instruction completes: Loads, Stores
> + * Older instructions which must be globally performed when the SYNC instruction
> + * completes: Loads, Stores
> + */
> +#define STYPE_SYNC 0x0

So, and I think there was no confusion on this point, "SYNC 0" is fully
transitive. Everything prior to the SYNC must be globally visible before
we continue.

> +/*
> + * Ordering barriers:
> + * - Every synchronizable specified memory instruction (loads or stores or both)
> + *   that occurs in the instruction stream before the SYNC instruction must
> + *   reach a stage in the load/store datapath after which no instruction
> + *   re-ordering is possible before any synchronizable specified memory
> + *   instruction which occurs after the SYNC instruction in the instruction
> + *   stream reaches the same stage in the load/store datapath.
> + *
> + * - If any memory instruction before the SYNC instruction in program order,
> + *   generates a memory request to the external memory and any memory
> + *   instruction after the SYNC instruction in program order also generates a
> + *   memory request to external memory, the memory request belonging to the
> + *   older instruction must be globally performed before the time the memory
> + *   request belonging to the younger instruction is globally performed.
> + *
> + * - The barrier does not guarantee the order in which instruction fetches are
> + *   performed.
> + */
> +
> +/*
> + * stype 0x10 - An ordering barrier that affects preceding loads and stores and
> + * subsequent loads and stores.
> + * Older instructions which must reach the load/store ordering point before the
> + * SYNC instruction completes: Loads, Stores
> + * Younger instructions which must reach the load/store ordering point only
> + * after the SYNC instruction completes: Loads, Stores
> + * Older instructions which must be globally performed when the SYNC instruction
> + * completes: N/A
> + */
> +#define STYPE_SYNC_MB 0x10

This I'm not sure of; it states that things must become globally visible
in the order specified, but the wording leaves a fairly big hole. It
doesn't state that things cannot be less than globally visible at
intermediate times.

To take the example from Documentation/memory-barriers.txt:

        CPU 1                   CPU 2                   CPU 3
        ======================= ======================= =======================
                { X = 0, Y = 0 }
        STORE X=1               LOAD X                  STORE Y=1
                                <general barrier>       <general barrier>
                                LOAD Y                  LOAD X

Suppose that CPU 2's load from X returns 1 and its load from Y returns 0.
This indicates that CPU 2's load from X in some sense follows CPU 1's
store to X and that CPU 2's load from Y in some sense preceded CPU 3's
store to Y.  The question is then "Can CPU 3's load from X return 0?"


Is it ever possible for CPU2 and CPU3 to match "SYNC 10" points but to
disagree on their loads of X?

That is, even though CPU2 and CPU3 agree on their respective past and
future stores, the 'happens before' relation CPU1 and CPU2 have wrt. X
is not included?

Re: [PATCH v2 05/12] MIPS: Barrier: Add definitions of SYNC stype values

[Peter Zijlstra]

On Wed, Sep 07, 2016 at 01:24:23PM +0200, Peter Zijlstra wrote:
> > +/*
> > + * Ordering barriers:
> > + * - Every synchronizable specified memory instruction (loads or stores or both)
> > + *   that occurs in the instruction stream before the SYNC instruction must
> > + *   reach a stage in the load/store datapath after which no instruction
> > + *   re-ordering is possible before any synchronizable specified memory
> > + *   instruction which occurs after the SYNC instruction in the instruction
> > + *   stream reaches the same stage in the load/store datapath.
> > + *
> > + * - If any memory instruction before the SYNC instruction in program order,
> > + *   generates a memory request to the external memory and any memory
> > + *   instruction after the SYNC instruction in program order also generates a
> > + *   memory request to external memory, the memory request belonging to the
> > + *   older instruction must be globally performed before the time the memory
> > + *   request belonging to the younger instruction is globally performed.
> > + *
> > + * - The barrier does not guarantee the order in which instruction fetches are
> > + *   performed.
> > + */
> > +
> > +/*
> > + * stype 0x10 - An ordering barrier that affects preceding loads and stores and
> > + * subsequent loads and stores.
> > + * Older instructions which must reach the load/store ordering point before the
> > + * SYNC instruction completes: Loads, Stores
> > + * Younger instructions which must reach the load/store ordering point only
> > + * after the SYNC instruction completes: Loads, Stores
> > + * Older instructions which must be globally performed when the SYNC instruction
> > + * completes: N/A
> > + */
> > +#define STYPE_SYNC_MB 0x10
> 
> This I'm not sure of; it states that things must become globally visible
> in the order specified, but the wording leaves a fairly big hole. It
> doesn't state that things cannot be less than globally visible at
> intermediate times.
> 
> To take the example from Documentation/memory-barriers.txt:
> 
>         CPU 1                   CPU 2                   CPU 3
>         ======================= ======================= =======================
>                 { X = 0, Y = 0 }
>         STORE X=1               LOAD X                  STORE Y=1
>                                 <general barrier>       <general barrier>
>                                 LOAD Y                  LOAD X
> 
> Suppose that CPU 2's load from X returns 1 and its load from Y returns 0.
> This indicates that CPU 2's load from X in some sense follows CPU 1's
> store to X and that CPU 2's load from Y in some sense preceded CPU 3's
> store to Y.  The question is then "Can CPU 3's load from X return 0?"
> 
> 
> Is it ever possible for CPU2 and CPU3 to match "SYNC 10" points but to
> disagree on their loads of X?
> 
> That is, even though CPU2 and CPU3 agree on their respective past and
> future stores, the 'happens before' relation CPU1 and CPU2 have wrt. X
> is not included?
> 

Now, I suspect it _is_ transitive, because CPU2's "LOAD X" must be
globally performed wrt CPU3's "LOAD X", and my interpretation of that
means that the STORE of X must be globally visible for that to be true.

But, like said, wording... so clarification would be grand.

Also, IFF "SYNC 10" is indeed transitive, you should be able to replace
smp_mb() with it unconditionally.