[PATCH v2 0/7] Enable queued rwlock and queued spinlock for SPARC

[PATCH v2 0/7] Enable queued rwlock and queued spinlock for SPARC

[Babu Moger]

This series of patches enables queued rwlock and queued spinlock support
for SPARC. These features were introduced some time ago in upstream.
Here are some of the earlier discussions.
https://lwn.net/Articles/572765/
https://lwn.net/Articles/582200/
https://lwn.net/Articles/561775/
https://lwn.net/Articles/590243/

Tests: Ran AIM7 benchmark to verify the performance on various workloads.
https://github.com/davidlohr/areaim. Same benchmark was used when this
feature was introduced and enabled on x86. Here are the test results.

Kernel				4.11.0-rc6     4.11.0-rc6 + 	Change
				baseline	queued locks
			      (Avg No.of jobs) (Avg No.of jobs)
Workload
High systime 10-100 user	 17290.48	 17295.18	+0.02
High systime 200-1000 users	109814.95	110248.87	+0.39
High systime 1200-2000 users	107912.40	127923.16	+18.54

Disk IO 10-100 users		168910.16	158834.17	-5.96
Disk IO 200-1000 users		242781.74	281285.80	+15.85
Disk IO 1200-2000 users		228518.23	218421.23	-4.41

Disk IO 10-100 users		183933.77	207928.67	+13.04
Disk IO 200-1000 users		491981.56	500162.33	+1.66
Disk IO 1200-2000 users		463395.66	467312.70	+0.84

fserver 10-100 users		254177.53	270283.08	+6.33
fserver IO 200-1000 users	269017.35	324812.2	+20.74
fserver IO 1200-2000 users	229538.87	284713.77	+24.03

Disk I/O results are little bit in negative territory. But majority of the 
performance changes are in positive and it is significant in some cases.

Changes:
v1->v2:
Addressed the comments from David Miller.
1. Added CPU_BIG_ENDIAN for all SPARC
2. Removed #ifndef __LINUX_SPINLOCK_TYPES_H guard from spinlock_types.h
3. Removed check for CONFIG_QUEUED_RWLOCKS in SPARC64 as it is the 
   default definition for SPARC64 now. Cleaned-up the previous arch_read_xxx
   and arch_write_xxx definitions as it is defined now in qrwlock.h.
4. Removed check for CONFIG_QUEUED_SPINLOCKS in SPARC64 as it is the default
   definition now for SPARC64 now. Cleaned-up the previous arch_spin_xxx
   definitions as it is defined in qspinlock.h. 

v1: Initial version

Babu Moger (7):
  kernel/locking: Fix compile error with qrwlock.c
  arch/sparc: Remove the check #ifndef __LINUX_SPINLOCK_TYPES_H
  arch/sparc: Define config parameter CPU_BIG_ENDIAN
  arch/sparc: Introduce cmpxchg_u8 SPARC
  arch/sparc: Enable queued rwlocks for SPARC
  arch/sparc: Introduce xchg16 for SPARC
  arch/sparc: Enable queued spinlock support for SPARC

 arch/sparc/Kconfig                      |    6 +
 arch/sparc/include/asm/cmpxchg_64.h     |   76 ++++++++++--
 arch/sparc/include/asm/qrwlock.h        |    7 +
 arch/sparc/include/asm/qspinlock.h      |    7 +
 arch/sparc/include/asm/spinlock_64.h    |  208 +------------------------------
 arch/sparc/include/asm/spinlock_types.h |   12 ++-
 include/asm-generic/qrwlock_types.h     |    6 +-
 kernel/locking/qrwlock.c                |    1 +
 8 files changed, 101 insertions(+), 222 deletions(-)
 create mode 100644 arch/sparc/include/asm/qrwlock.h
 create mode 100644 arch/sparc/include/asm/qspinlock.h

[PATCH v2 0/7] Enable queued rwlock and queued spinlock for SPARC

[Babu Moger]

This series of patches enables queued rwlock and queued spinlock support
for SPARC. These features were introduced some time ago in upstream.
Here are some of the earlier discussions.
https://lwn.net/Articles/572765/
https://lwn.net/Articles/582200/
https://lwn.net/Articles/561775/
https://lwn.net/Articles/590243/

Tests: Ran AIM7 benchmark to verify the performance on various workloads.
https://github.com/davidlohr/areaim. Same benchmark was used when this
feature was introduced and enabled on x86. Here are the test results.

Kernel				4.11.0-rc6     4.11.0-rc6 + 	Change
				baseline	queued locks
			      (Avg No.of jobs) (Avg No.of jobs)
Workload
High systime 10-100 user	 17290.48	 17295.18	+0.02
High systime 200-1000 users	109814.95	110248.87	+0.39
High systime 1200-2000 users	107912.40	127923.16	+18.54

Disk IO 10-100 users		168910.16	158834.17	-5.96
Disk IO 200-1000 users		242781.74	281285.80	+15.85
Disk IO 1200-2000 users		228518.23	218421.23	-4.41

Disk IO 10-100 users		183933.77	207928.67	+13.04
Disk IO 200-1000 users		491981.56	500162.33	+1.66
Disk IO 1200-2000 users		463395.66	467312.70	+0.84

fserver 10-100 users		254177.53	270283.08	+6.33
fserver IO 200-1000 users	269017.35	324812.2	+20.74
fserver IO 1200-2000 users	229538.87	284713.77	+24.03

Disk I/O results are little bit in negative territory. But majority of the 
performance changes are in positive and it is significant in some cases.

Changes:
v1->v2:
Addressed the comments from David Miller.
1. Added CPU_BIG_ENDIAN for all SPARC
2. Removed #ifndef __LINUX_SPINLOCK_TYPES_H guard from spinlock_types.h
3. Removed check for CONFIG_QUEUED_RWLOCKS in SPARC64 as it is the 
   default definition for SPARC64 now. Cleaned-up the previous arch_read_xxx
   and arch_write_xxx definitions as it is defined now in qrwlock.h.
4. Removed check for CONFIG_QUEUED_SPINLOCKS in SPARC64 as it is the default
   definition now for SPARC64 now. Cleaned-up the previous arch_spin_xxx
   definitions as it is defined in qspinlock.h. 

v1: Initial version

Babu Moger (7):
  kernel/locking: Fix compile error with qrwlock.c
  arch/sparc: Remove the check #ifndef __LINUX_SPINLOCK_TYPES_H
  arch/sparc: Define config parameter CPU_BIG_ENDIAN
  arch/sparc: Introduce cmpxchg_u8 SPARC
  arch/sparc: Enable queued rwlocks for SPARC
  arch/sparc: Introduce xchg16 for SPARC
  arch/sparc: Enable queued spinlock support for SPARC

 arch/sparc/Kconfig                      |    6 +
 arch/sparc/include/asm/cmpxchg_64.h     |   76 ++++++++++--
 arch/sparc/include/asm/qrwlock.h        |    7 +
 arch/sparc/include/asm/qspinlock.h      |    7 +
 arch/sparc/include/asm/spinlock_64.h    |  208 +------------------------------
 arch/sparc/include/asm/spinlock_types.h |   12 ++-
 include/asm-generic/qrwlock_types.h     |    6 +-
 kernel/locking/qrwlock.c                |    1 +
 8 files changed, 101 insertions(+), 222 deletions(-)
 create mode 100644 arch/sparc/include/asm/qrwlock.h
 create mode 100644 arch/sparc/include/asm/qspinlock.h

[PATCH v2 1/7] kernel/locking: Fix compile error with qrwlock.c

[Babu Moger]

Some architectures use the following guard in include file
 "asm/spinlock_types.h" to discourage including the file directly.

Saw these compile errors on SPARC when queued rwlock feature is enabled.

  CC      kernel/locking/qrwlock.o
In file included from ./include/asm-generic/qrwlock_types.h:5,
                 from ./arch/sparc/include/asm/qrwlock.h:4,
                 from kernel/locking/qrwlock.c:24:
./arch/sparc/include/asm/spinlock_types.h:5:3: error:
        #error "please don't include this file directly"

Re-arrange the includes in qrwlock_types.h and include spinlock.h
in qrwlock.c to fix it.

Also will be removing this stanza from SPARC. Stay tuned.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 include/asm-generic/qrwlock_types.h |    6 +++---
 kernel/locking/qrwlock.c            |    1 +
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/include/asm-generic/qrwlock_types.h b/include/asm-generic/qrwlock_types.h
index 0abc6b6..3988c7d 100644
--- a/include/asm-generic/qrwlock_types.h
+++ b/include/asm-generic/qrwlock_types.h
@@ -1,9 +1,6 @@
 #ifndef __ASM_GENERIC_QRWLOCK_TYPES_H
 #define __ASM_GENERIC_QRWLOCK_TYPES_H
 
-#include <linux/types.h>
-#include <asm/spinlock_types.h>
-
 /*
  * The queue read/write lock data structure
  */
@@ -18,4 +15,7 @@
 	.wait_lock = __ARCH_SPIN_LOCK_UNLOCKED,	\
 }
 
+#include <linux/types.h>
+#include <linux/spinlock_types.h>
+
 #endif /* __ASM_GENERIC_QRWLOCK_TYPES_H */
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index cc3ed0c..2655f26 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -20,6 +20,7 @@
 #include <linux/cpumask.h>
 #include <linux/percpu.h>
 #include <linux/hardirq.h>
+#include <linux/spinlock.h>
 #include <asm/qrwlock.h>
 
 /*
-- 
1.7.1

[PATCH v2 1/7] kernel/locking: Fix compile error with qrwlock.c

[Babu Moger]

Some architectures use the following guard in include file
 "asm/spinlock_types.h" to discourage including the file directly.

Saw these compile errors on SPARC when queued rwlock feature is enabled.

  CC      kernel/locking/qrwlock.o
In file included from ./include/asm-generic/qrwlock_types.h:5,
                 from ./arch/sparc/include/asm/qrwlock.h:4,
                 from kernel/locking/qrwlock.c:24:
./arch/sparc/include/asm/spinlock_types.h:5:3: error:
        #error "please don't include this file directly"

Re-arrange the includes in qrwlock_types.h and include spinlock.h
in qrwlock.c to fix it.

Also will be removing this stanza from SPARC. Stay tuned.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: HÃ¥kon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 include/asm-generic/qrwlock_types.h |    6 +++---
 kernel/locking/qrwlock.c            |    1 +
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/include/asm-generic/qrwlock_types.h b/include/asm-generic/qrwlock_types.h
index 0abc6b6..3988c7d 100644
--- a/include/asm-generic/qrwlock_types.h
+++ b/include/asm-generic/qrwlock_types.h
@@ -1,9 +1,6 @@
 #ifndef __ASM_GENERIC_QRWLOCK_TYPES_H
 #define __ASM_GENERIC_QRWLOCK_TYPES_H
 
-#include <linux/types.h>
-#include <asm/spinlock_types.h>
-
 /*
  * The queue read/write lock data structure
  */
@@ -18,4 +15,7 @@
 	.wait_lock = __ARCH_SPIN_LOCK_UNLOCKED,	\
 }
 
+#include <linux/types.h>
+#include <linux/spinlock_types.h>
+
 #endif /* __ASM_GENERIC_QRWLOCK_TYPES_H */
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index cc3ed0c..2655f26 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -20,6 +20,7 @@
 #include <linux/cpumask.h>
 #include <linux/percpu.h>
 #include <linux/hardirq.h>
+#include <linux/spinlock.h>
 #include <asm/qrwlock.h>
 
 /*
-- 
1.7.1

[PATCH v2 2/7] arch/sparc: Remove the check #ifndef __LINUX_SPINLOCK_TYPES_H

[Babu Moger]

Remove the un-necessary "ifndef __LINUX_SPINLOCK_TYPES_H" stanza from SPARC.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Suggested-by: David Miller <davem@davemloft.net>
---
 arch/sparc/include/asm/spinlock_types.h |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index 9c454fd..019c085 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -1,10 +1,6 @@
 #ifndef __SPARC_SPINLOCK_TYPES_H
 #define __SPARC_SPINLOCK_TYPES_H
 
-#ifndef __LINUX_SPINLOCK_TYPES_H
-# error "please don't include this file directly"
-#endif
-
 typedef struct {
 	volatile unsigned char lock;
 } arch_spinlock_t;
-- 
1.7.1

[PATCH v2 2/7] arch/sparc: Remove the check #ifndef __LINUX_SPINLOCK_TYPES_H

[Babu Moger]

Remove the un-necessary "ifndef __LINUX_SPINLOCK_TYPES_H" stanza from SPARC.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Suggested-by: David Miller <davem@davemloft.net>
---
 arch/sparc/include/asm/spinlock_types.h |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index 9c454fd..019c085 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -1,10 +1,6 @@
 #ifndef __SPARC_SPINLOCK_TYPES_H
 #define __SPARC_SPINLOCK_TYPES_H
 
-#ifndef __LINUX_SPINLOCK_TYPES_H
-# error "please don't include this file directly"
-#endif
-
 typedef struct {
 	volatile unsigned char lock;
 } arch_spinlock_t;
-- 
1.7.1

[PATCH v2 3/7] arch/sparc: Define config parameter CPU_BIG_ENDIAN

[Babu Moger]

Found this problem while enabling queued rwlock on SPARC.
The parameter CONFIG_CPU_BIG_ENDIAN is used to clear the
specific byte in qrwlock structure. Without this parameter,
we clear the wrong byte. Here is the code.

static inline u8 *__qrwlock_write_byte(struct qrwlock *lock)
 {
	return (u8 *)lock + 3 * IS_BUILTIN(CONFIG_CPU_BIG_ENDIAN);
 }

Define CPU_BIG_ENDIAN for SPARC to fix it.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index a2ad946..0f5813b 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -91,6 +91,10 @@ config ARCH_DEFCONFIG
 config ARCH_PROC_KCORE_TEXT
 	def_bool y
 
+config CPU_BIG_ENDIAN
+	bool
+	default y if SPARC
+
 config ARCH_ATU
 	bool
 	default y if SPARC64
-- 
1.7.1

[PATCH v2 3/7] arch/sparc: Define config parameter CPU_BIG_ENDIAN

[Babu Moger]

Found this problem while enabling queued rwlock on SPARC.
The parameter CONFIG_CPU_BIG_ENDIAN is used to clear the
specific byte in qrwlock structure. Without this parameter,
we clear the wrong byte. Here is the code.

static inline u8 *__qrwlock_write_byte(struct qrwlock *lock)
 {
	return (u8 *)lock + 3 * IS_BUILTIN(CONFIG_CPU_BIG_ENDIAN);
 }

Define CPU_BIG_ENDIAN for SPARC to fix it.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: HÃ¥kon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index a2ad946..0f5813b 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -91,6 +91,10 @@ config ARCH_DEFCONFIG
 config ARCH_PROC_KCORE_TEXT
 	def_bool y
 
+config CPU_BIG_ENDIAN
+	bool
+	default y if SPARC
+
 config ARCH_ATU
 	bool
 	default y if SPARC64
-- 
1.7.1

[PATCH v2 4/7] arch/sparc: Introduce cmpxchg_u8 SPARC

[Babu Moger]

SPARC supports 32 bit and 64 bit cmpxchg right now. Add support
for 8 bit (1 byte) cmpxchg. This is required to support queued
rwlocks feature  which uses 1 byte cmpxchg.

The function __cmpxchg_u8 here uses the 4 byte cas instruction with a
byte manipulation to achieve 1 byte cmpxchg.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Steve Sistare <steven.sistare@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/include/asm/cmpxchg_64.h |   29 +++++++++++++++++++++++++++++
 1 files changed, 29 insertions(+), 0 deletions(-)

diff --git a/arch/sparc/include/asm/cmpxchg_64.h b/arch/sparc/include/asm/cmpxchg_64.h
index faa2f61..000f7d7 100644
--- a/arch/sparc/include/asm/cmpxchg_64.h
+++ b/arch/sparc/include/asm/cmpxchg_64.h
@@ -87,6 +87,33 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 	return new;
 }
 
+/*
+ * Use 4 byte cas instruction to achieve 1 byte cmpxchg. Main logic
+ * here is to get the bit shift of the byte we are interested in.
+ * The XOR is handy for reversing the bits for big-endian byte order
+ */
+static inline unsigned long
+__cmpxchg_u8(volatile unsigned char *m, unsigned char old, unsigned char new)
+{
+	unsigned long maddr = (unsigned long)m;
+	int bit_shift = (((unsigned long)m & 3) ^ 3) << 3;
+	unsigned int mask = 0xff << bit_shift;
+	unsigned int *ptr = (unsigned int *) (maddr & ~3);
+	unsigned int old32, new32, load;
+	unsigned int load32 = *ptr;
+
+	do {
+		new32 = (load32 & ~mask) | (new << bit_shift);
+		old32 = (load32 & ~mask) | (old << bit_shift);
+		load32 = __cmpxchg_u32(ptr, old32, new32);
+		if (load32 == old32)
+			return old;
+		load = (load32 & mask) >> bit_shift;
+	} while (load == old);
+
+	return load;
+}
+
 /* This function doesn't exist, so you'll get a linker error
    if something tries to do an invalid cmpxchg().  */
 void __cmpxchg_called_with_bad_pointer(void);
@@ -95,6 +122,8 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 __cmpxchg(volatile void *ptr, unsigned long old, unsigned long new, int size)
 {
 	switch (size) {
+		case 1:
+			return __cmpxchg_u8(ptr, old, new);
 		case 4:
 			return __cmpxchg_u32(ptr, old, new);
 		case 8:
-- 
1.7.1

[PATCH v2 4/7] arch/sparc: Introduce cmpxchg_u8 SPARC

[Babu Moger]

SPARC supports 32 bit and 64 bit cmpxchg right now. Add support
for 8 bit (1 byte) cmpxchg. This is required to support queued
rwlocks feature  which uses 1 byte cmpxchg.

The function __cmpxchg_u8 here uses the 4 byte cas instruction with a
byte manipulation to achieve 1 byte cmpxchg.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: HÃ¥kon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Steve Sistare <steven.sistare@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/include/asm/cmpxchg_64.h |   29 +++++++++++++++++++++++++++++
 1 files changed, 29 insertions(+), 0 deletions(-)

diff --git a/arch/sparc/include/asm/cmpxchg_64.h b/arch/sparc/include/asm/cmpxchg_64.h
index faa2f61..000f7d7 100644
--- a/arch/sparc/include/asm/cmpxchg_64.h
+++ b/arch/sparc/include/asm/cmpxchg_64.h
@@ -87,6 +87,33 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 	return new;
 }
 
+/*
+ * Use 4 byte cas instruction to achieve 1 byte cmpxchg. Main logic
+ * here is to get the bit shift of the byte we are interested in.
+ * The XOR is handy for reversing the bits for big-endian byte order
+ */
+static inline unsigned long
+__cmpxchg_u8(volatile unsigned char *m, unsigned char old, unsigned char new)
+{
+	unsigned long maddr = (unsigned long)m;
+	int bit_shift = (((unsigned long)m & 3) ^ 3) << 3;
+	unsigned int mask = 0xff << bit_shift;
+	unsigned int *ptr = (unsigned int *) (maddr & ~3);
+	unsigned int old32, new32, load;
+	unsigned int load32 = *ptr;
+
+	do {
+		new32 = (load32 & ~mask) | (new << bit_shift);
+		old32 = (load32 & ~mask) | (old << bit_shift);
+		load32 = __cmpxchg_u32(ptr, old32, new32);
+		if (load32 = old32)
+			return old;
+		load = (load32 & mask) >> bit_shift;
+	} while (load = old);
+
+	return load;
+}
+
 /* This function doesn't exist, so you'll get a linker error
    if something tries to do an invalid cmpxchg().  */
 void __cmpxchg_called_with_bad_pointer(void);
@@ -95,6 +122,8 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 __cmpxchg(volatile void *ptr, unsigned long old, unsigned long new, int size)
 {
 	switch (size) {
+		case 1:
+			return __cmpxchg_u8(ptr, old, new);
 		case 4:
 			return __cmpxchg_u32(ptr, old, new);
 		case 8:
-- 
1.7.1

[PATCH v2 5/7] arch/sparc: Enable queued rwlocks for SPARC

[Babu Moger]

Enable queued rwlocks for SPARC. Here are the discussions on this feature
when this was introduced.
https://lwn.net/Articles/572765/
https://lwn.net/Articles/582200/

Cleaned-up the arch_read_xxx and arch_write_xxx definitions in spinlock_64.h.
These routines are replaced by the functions in include/asm-generic/qrwlock.h

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig                      |    1 +
 arch/sparc/include/asm/qrwlock.h        |    7 ++
 arch/sparc/include/asm/spinlock_64.h    |  124 +------------------------------
 arch/sparc/include/asm/spinlock_types.h |    5 +-
 4 files changed, 13 insertions(+), 124 deletions(-)
 create mode 100644 arch/sparc/include/asm/qrwlock.h

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 0f5813b..9ec1d2f 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -82,6 +82,7 @@ config SPARC64
 	select HAVE_ARCH_AUDITSYSCALL
 	select ARCH_SUPPORTS_ATOMIC_RMW
 	select HAVE_NMI
+	select ARCH_USE_QUEUED_RWLOCKS
 
 config ARCH_DEFCONFIG
 	string
diff --git a/arch/sparc/include/asm/qrwlock.h b/arch/sparc/include/asm/qrwlock.h
new file mode 100644
index 0000000..d68a4b1
--- /dev/null
+++ b/arch/sparc/include/asm/qrwlock.h
@@ -0,0 +1,7 @@
+#ifndef _ASM_SPARC_QRWLOCK_H
+#define _ASM_SPARC_QRWLOCK_H
+
+#include <asm-generic/qrwlock_types.h>
+#include <asm-generic/qrwlock.h>
+
+#endif /* _ASM_SPARC_QRWLOCK_H */
diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
index 07c9f2e..8901c2d 100644
--- a/arch/sparc/include/asm/spinlock_64.h
+++ b/arch/sparc/include/asm/spinlock_64.h
@@ -10,6 +10,7 @@
 
 #include <asm/processor.h>
 #include <asm/barrier.h>
+#include <asm/qrwlock.h>
 
 /* To get debugging spinlocks which detect and catch
  * deadlock situations, set CONFIG_DEBUG_SPINLOCK
@@ -94,132 +95,9 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
 	: "memory");
 }
 
-/* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
-
-static inline void arch_read_lock(arch_rwlock_t *lock)
-{
-	unsigned long tmp1, tmp2;
-
-	__asm__ __volatile__ (
-"1:	ldsw		[%2], %0\n"
-"	brlz,pn		%0, 2f\n"
-"4:	 add		%0, 1, %1\n"
-"	cas		[%2], %0, %1\n"
-"	cmp		%0, %1\n"
-"	bne,pn		%%icc, 1b\n"
-"	 nop\n"
-"	.subsection	2\n"
-"2:	ldsw		[%2], %0\n"
-"	brlz,pt		%0, 2b\n"
-"	 nop\n"
-"	ba,a,pt		%%xcc, 4b\n"
-"	.previous"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r" (lock)
-	: "memory");
-}
-
-static inline int arch_read_trylock(arch_rwlock_t *lock)
-{
-	int tmp1, tmp2;
-
-	__asm__ __volatile__ (
-"1:	ldsw		[%2], %0\n"
-"	brlz,a,pn	%0, 2f\n"
-"	 mov		0, %0\n"
-"	add		%0, 1, %1\n"
-"	cas		[%2], %0, %1\n"
-"	cmp		%0, %1\n"
-"	bne,pn		%%icc, 1b\n"
-"	 mov		1, %0\n"
-"2:"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r" (lock)
-	: "memory");
-
-	return tmp1;
-}
-
-static inline void arch_read_unlock(arch_rwlock_t *lock)
-{
-	unsigned long tmp1, tmp2;
-
-	__asm__ __volatile__(
-"1:	lduw	[%2], %0\n"
-"	sub	%0, 1, %1\n"
-"	cas	[%2], %0, %1\n"
-"	cmp	%0, %1\n"
-"	bne,pn	%%xcc, 1b\n"
-"	 nop"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r" (lock)
-	: "memory");
-}
-
-static inline void arch_write_lock(arch_rwlock_t *lock)
-{
-	unsigned long mask, tmp1, tmp2;
-
-	mask = 0x80000000UL;
-
-	__asm__ __volatile__(
-"1:	lduw		[%2], %0\n"
-"	brnz,pn		%0, 2f\n"
-"4:	 or		%0, %3, %1\n"
-"	cas		[%2], %0, %1\n"
-"	cmp		%0, %1\n"
-"	bne,pn		%%icc, 1b\n"
-"	 nop\n"
-"	.subsection	2\n"
-"2:	lduw		[%2], %0\n"
-"	brnz,pt		%0, 2b\n"
-"	 nop\n"
-"	ba,a,pt		%%xcc, 4b\n"
-"	.previous"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r" (lock), "r" (mask)
-	: "memory");
-}
-
-static inline void arch_write_unlock(arch_rwlock_t *lock)
-{
-	__asm__ __volatile__(
-"	stw		%%g0, [%0]"
-	: /* no outputs */
-	: "r" (lock)
-	: "memory");
-}
-
-static inline int arch_write_trylock(arch_rwlock_t *lock)
-{
-	unsigned long mask, tmp1, tmp2, result;
-
-	mask = 0x80000000UL;
-
-	__asm__ __volatile__(
-"	mov		0, %2\n"
-"1:	lduw		[%3], %0\n"
-"	brnz,pn		%0, 2f\n"
-"	 or		%0, %4, %1\n"
-"	cas		[%3], %0, %1\n"
-"	cmp		%0, %1\n"
-"	bne,pn		%%icc, 1b\n"
-"	 nop\n"
-"	mov		1, %2\n"
-"2:"
-	: "=&r" (tmp1), "=&r" (tmp2), "=&r" (result)
-	: "r" (lock), "r" (mask)
-	: "memory");
-
-	return result;
-}
-
 #define arch_read_lock_flags(p, f) arch_read_lock(p)
 #define arch_write_lock_flags(p, f) arch_write_lock(p)
 
-#define arch_read_can_lock(rw)		(!((rw)->lock & 0x80000000UL))
-#define arch_write_can_lock(rw)	(!(rw)->lock)
-
 #define arch_spin_relax(lock)	cpu_relax()
 #define arch_read_relax(lock)	cpu_relax()
 #define arch_write_relax(lock)	cpu_relax()
diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index 019c085..64fce21 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -7,10 +7,13 @@
 
 #define __ARCH_SPIN_LOCK_UNLOCKED	{ 0 }
 
+#ifdef CONFIG_QUEUED_RWLOCKS
+#include <asm-generic/qrwlock_types.h>
+#else
 typedef struct {
 	volatile unsigned int lock;
 } arch_rwlock_t;
 
 #define __ARCH_RW_LOCK_UNLOCKED		{ 0 }
-
+#endif /* CONFIG_QUEUED_RWLOCKS */
 #endif
-- 
1.7.1

[PATCH v2 5/7] arch/sparc: Enable queued rwlocks for SPARC

[Babu Moger]

Enable queued rwlocks for SPARC. Here are the discussions on this feature
when this was introduced.
https://lwn.net/Articles/572765/
https://lwn.net/Articles/582200/

Cleaned-up the arch_read_xxx and arch_write_xxx definitions in spinlock_64.h.
These routines are replaced by the functions in include/asm-generic/qrwlock.h

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: HÃ¥kon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig                      |    1 +
 arch/sparc/include/asm/qrwlock.h        |    7 ++
 arch/sparc/include/asm/spinlock_64.h    |  124 +------------------------------
 arch/sparc/include/asm/spinlock_types.h |    5 +-
 4 files changed, 13 insertions(+), 124 deletions(-)
 create mode 100644 arch/sparc/include/asm/qrwlock.h

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 0f5813b..9ec1d2f 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -82,6 +82,7 @@ config SPARC64
 	select HAVE_ARCH_AUDITSYSCALL
 	select ARCH_SUPPORTS_ATOMIC_RMW
 	select HAVE_NMI
+	select ARCH_USE_QUEUED_RWLOCKS
 
 config ARCH_DEFCONFIG
 	string
diff --git a/arch/sparc/include/asm/qrwlock.h b/arch/sparc/include/asm/qrwlock.h
new file mode 100644
index 0000000..d68a4b1
--- /dev/null
+++ b/arch/sparc/include/asm/qrwlock.h
@@ -0,0 +1,7 @@
+#ifndef _ASM_SPARC_QRWLOCK_H
+#define _ASM_SPARC_QRWLOCK_H
+
+#include <asm-generic/qrwlock_types.h>
+#include <asm-generic/qrwlock.h>
+
+#endif /* _ASM_SPARC_QRWLOCK_H */
diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
index 07c9f2e..8901c2d 100644
--- a/arch/sparc/include/asm/spinlock_64.h
+++ b/arch/sparc/include/asm/spinlock_64.h
@@ -10,6 +10,7 @@
 
 #include <asm/processor.h>
 #include <asm/barrier.h>
+#include <asm/qrwlock.h>
 
 /* To get debugging spinlocks which detect and catch
  * deadlock situations, set CONFIG_DEBUG_SPINLOCK
@@ -94,132 +95,9 @@ static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long fla
 	: "memory");
 }
 
-/* Multi-reader locks, these are much saner than the 32-bit Sparc ones... */
-
-static inline void arch_read_lock(arch_rwlock_t *lock)
-{
-	unsigned long tmp1, tmp2;
-
-	__asm__ __volatile__ (
-"1:	ldsw		[%2], %0\n"
-"	brlz,pn		%0, 2f\n"
-"4:	 add		%0, 1, %1\n"
-"	cas		[%2], %0, %1\n"
-"	cmp		%0, %1\n"
-"	bne,pn		%%icc, 1b\n"
-"	 nop\n"
-"	.subsection	2\n"
-"2:	ldsw		[%2], %0\n"
-"	brlz,pt		%0, 2b\n"
-"	 nop\n"
-"	ba,a,pt		%%xcc, 4b\n"
-"	.previous"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r" (lock)
-	: "memory");
-}
-
-static inline int arch_read_trylock(arch_rwlock_t *lock)
-{
-	int tmp1, tmp2;
-
-	__asm__ __volatile__ (
-"1:	ldsw		[%2], %0\n"
-"	brlz,a,pn	%0, 2f\n"
-"	 mov		0, %0\n"
-"	add		%0, 1, %1\n"
-"	cas		[%2], %0, %1\n"
-"	cmp		%0, %1\n"
-"	bne,pn		%%icc, 1b\n"
-"	 mov		1, %0\n"
-"2:"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r" (lock)
-	: "memory");
-
-	return tmp1;
-}
-
-static inline void arch_read_unlock(arch_rwlock_t *lock)
-{
-	unsigned long tmp1, tmp2;
-
-	__asm__ __volatile__(
-"1:	lduw	[%2], %0\n"
-"	sub	%0, 1, %1\n"
-"	cas	[%2], %0, %1\n"
-"	cmp	%0, %1\n"
-"	bne,pn	%%xcc, 1b\n"
-"	 nop"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r" (lock)
-	: "memory");
-}
-
-static inline void arch_write_lock(arch_rwlock_t *lock)
-{
-	unsigned long mask, tmp1, tmp2;
-
-	mask = 0x80000000UL;
-
-	__asm__ __volatile__(
-"1:	lduw		[%2], %0\n"
-"	brnz,pn		%0, 2f\n"
-"4:	 or		%0, %3, %1\n"
-"	cas		[%2], %0, %1\n"
-"	cmp		%0, %1\n"
-"	bne,pn		%%icc, 1b\n"
-"	 nop\n"
-"	.subsection	2\n"
-"2:	lduw		[%2], %0\n"
-"	brnz,pt		%0, 2b\n"
-"	 nop\n"
-"	ba,a,pt		%%xcc, 4b\n"
-"	.previous"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r" (lock), "r" (mask)
-	: "memory");
-}
-
-static inline void arch_write_unlock(arch_rwlock_t *lock)
-{
-	__asm__ __volatile__(
-"	stw		%%g0, [%0]"
-	: /* no outputs */
-	: "r" (lock)
-	: "memory");
-}
-
-static inline int arch_write_trylock(arch_rwlock_t *lock)
-{
-	unsigned long mask, tmp1, tmp2, result;
-
-	mask = 0x80000000UL;
-
-	__asm__ __volatile__(
-"	mov		0, %2\n"
-"1:	lduw		[%3], %0\n"
-"	brnz,pn		%0, 2f\n"
-"	 or		%0, %4, %1\n"
-"	cas		[%3], %0, %1\n"
-"	cmp		%0, %1\n"
-"	bne,pn		%%icc, 1b\n"
-"	 nop\n"
-"	mov		1, %2\n"
-"2:"
-	: "=&r" (tmp1), "=&r" (tmp2), "=&r" (result)
-	: "r" (lock), "r" (mask)
-	: "memory");
-
-	return result;
-}
-
 #define arch_read_lock_flags(p, f) arch_read_lock(p)
 #define arch_write_lock_flags(p, f) arch_write_lock(p)
 
-#define arch_read_can_lock(rw)		(!((rw)->lock & 0x80000000UL))
-#define arch_write_can_lock(rw)	(!(rw)->lock)
-
 #define arch_spin_relax(lock)	cpu_relax()
 #define arch_read_relax(lock)	cpu_relax()
 #define arch_write_relax(lock)	cpu_relax()
diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index 019c085..64fce21 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -7,10 +7,13 @@
 
 #define __ARCH_SPIN_LOCK_UNLOCKED	{ 0 }
 
+#ifdef CONFIG_QUEUED_RWLOCKS
+#include <asm-generic/qrwlock_types.h>
+#else
 typedef struct {
 	volatile unsigned int lock;
 } arch_rwlock_t;
 
 #define __ARCH_RW_LOCK_UNLOCKED		{ 0 }
-
+#endif /* CONFIG_QUEUED_RWLOCKS */
 #endif
-- 
1.7.1

[PATCH v2 6/7] arch/sparc: Introduce xchg16 for SPARC

[Babu Moger]

SPARC supports 32 bit and 64 bit xchg right now. Add the support
for 16 bit (2 byte) xchg. This is required to support queued spinlock
feature which uses 2 byte xchg. This is achieved using 4 byte cas
instructions with byte manipulations.

Also re-arranged the code to call __cmpxchg_u32 inside xchg16.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Steven Sistare <steven.sistare@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/include/asm/cmpxchg_64.h |   49 +++++++++++++++++++++++++++-------
 1 files changed, 39 insertions(+), 10 deletions(-)

diff --git a/arch/sparc/include/asm/cmpxchg_64.h b/arch/sparc/include/asm/cmpxchg_64.h
index 000f7d7..4028f4f 100644
--- a/arch/sparc/include/asm/cmpxchg_64.h
+++ b/arch/sparc/include/asm/cmpxchg_64.h
@@ -6,6 +6,17 @@
 #ifndef __ARCH_SPARC64_CMPXCHG__
 #define __ARCH_SPARC64_CMPXCHG__
 
+static inline unsigned long
+__cmpxchg_u32(volatile int *m, int old, int new)
+{
+	__asm__ __volatile__("cas [%2], %3, %0"
+			     : "=&r" (new)
+			     : "0" (new), "r" (m), "r" (old)
+			     : "memory");
+
+	return new;
+}
+
 static inline unsigned long xchg32(__volatile__ unsigned int *m, unsigned int val)
 {
 	unsigned long tmp1, tmp2;
@@ -44,10 +55,38 @@ static inline unsigned long xchg64(__volatile__ unsigned long *m, unsigned long 
 
 void __xchg_called_with_bad_pointer(void);
 
+/*
+ * Use 4 byte cas instruction to achieve 2 byte xchg. Main logic
+ * here is to get the bit shift of the byte we are interested in.
+ * The XOR is handy for reversing the bits for big-endian byte order.
+ */
+static inline unsigned long
+xchg16(__volatile__ unsigned short *m, unsigned short val)
+{
+	unsigned long maddr = (unsigned long)m;
+	int bit_shift = (((unsigned long)m & 2) ^ 2) << 3;
+	unsigned int mask = 0xffff << bit_shift;
+	unsigned int *ptr = (unsigned int  *) (maddr & ~2);
+	unsigned int old32, new32, load32;
+
+	/* Read the old value */
+	load32 = *ptr;
+
+	do {
+		old32 = load32;
+		new32 = (load32 & (~mask)) | val << bit_shift;
+		load32 = __cmpxchg_u32(ptr, old32, new32);
+	} while (load32 != old32);
+
+	return (load32 & mask) >> bit_shift;
+}
+
 static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 				       int size)
 {
 	switch (size) {
+	case 2:
+		return xchg16(ptr, x);
 	case 4:
 		return xchg32(ptr, x);
 	case 8:
@@ -65,16 +104,6 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 
 #include <asm-generic/cmpxchg-local.h>
 
-static inline unsigned long
-__cmpxchg_u32(volatile int *m, int old, int new)
-{
-	__asm__ __volatile__("cas [%2], %3, %0"
-			     : "=&r" (new)
-			     : "0" (new), "r" (m), "r" (old)
-			     : "memory");
-
-	return new;
-}
 
 static inline unsigned long
 __cmpxchg_u64(volatile long *m, unsigned long old, unsigned long new)
-- 
1.7.1

[PATCH v2 6/7] arch/sparc: Introduce xchg16 for SPARC

[Babu Moger]

SPARC supports 32 bit and 64 bit xchg right now. Add the support
for 16 bit (2 byte) xchg. This is required to support queued spinlock
feature which uses 2 byte xchg. This is achieved using 4 byte cas
instructions with byte manipulations.

Also re-arranged the code to call __cmpxchg_u32 inside xchg16.

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: HÃ¥kon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Steven Sistare <steven.sistare@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/include/asm/cmpxchg_64.h |   49 +++++++++++++++++++++++++++-------
 1 files changed, 39 insertions(+), 10 deletions(-)

diff --git a/arch/sparc/include/asm/cmpxchg_64.h b/arch/sparc/include/asm/cmpxchg_64.h
index 000f7d7..4028f4f 100644
--- a/arch/sparc/include/asm/cmpxchg_64.h
+++ b/arch/sparc/include/asm/cmpxchg_64.h
@@ -6,6 +6,17 @@
 #ifndef __ARCH_SPARC64_CMPXCHG__
 #define __ARCH_SPARC64_CMPXCHG__
 
+static inline unsigned long
+__cmpxchg_u32(volatile int *m, int old, int new)
+{
+	__asm__ __volatile__("cas [%2], %3, %0"
+			     : "=&r" (new)
+			     : "0" (new), "r" (m), "r" (old)
+			     : "memory");
+
+	return new;
+}
+
 static inline unsigned long xchg32(__volatile__ unsigned int *m, unsigned int val)
 {
 	unsigned long tmp1, tmp2;
@@ -44,10 +55,38 @@ static inline unsigned long xchg64(__volatile__ unsigned long *m, unsigned long 
 
 void __xchg_called_with_bad_pointer(void);
 
+/*
+ * Use 4 byte cas instruction to achieve 2 byte xchg. Main logic
+ * here is to get the bit shift of the byte we are interested in.
+ * The XOR is handy for reversing the bits for big-endian byte order.
+ */
+static inline unsigned long
+xchg16(__volatile__ unsigned short *m, unsigned short val)
+{
+	unsigned long maddr = (unsigned long)m;
+	int bit_shift = (((unsigned long)m & 2) ^ 2) << 3;
+	unsigned int mask = 0xffff << bit_shift;
+	unsigned int *ptr = (unsigned int  *) (maddr & ~2);
+	unsigned int old32, new32, load32;
+
+	/* Read the old value */
+	load32 = *ptr;
+
+	do {
+		old32 = load32;
+		new32 = (load32 & (~mask)) | val << bit_shift;
+		load32 = __cmpxchg_u32(ptr, old32, new32);
+	} while (load32 != old32);
+
+	return (load32 & mask) >> bit_shift;
+}
+
 static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 				       int size)
 {
 	switch (size) {
+	case 2:
+		return xchg16(ptr, x);
 	case 4:
 		return xchg32(ptr, x);
 	case 8:
@@ -65,16 +104,6 @@ static inline unsigned long __xchg(unsigned long x, __volatile__ void * ptr,
 
 #include <asm-generic/cmpxchg-local.h>
 
-static inline unsigned long
-__cmpxchg_u32(volatile int *m, int old, int new)
-{
-	__asm__ __volatile__("cas [%2], %3, %0"
-			     : "=&r" (new)
-			     : "0" (new), "r" (m), "r" (old)
-			     : "memory");
-
-	return new;
-}
 
 static inline unsigned long
 __cmpxchg_u64(volatile long *m, unsigned long old, unsigned long new)
-- 
1.7.1

[PATCH v2 7/7] arch/sparc: Enable queued spinlock support for SPARC

[Babu Moger]

This patch makes the necessary changes in SPARC architecture to enable
queued spinlock support. Here are some of the earlier discussions about
this feature.
https://lwn.net/Articles/561775/
https://lwn.net/Articles/590243/

Cleaned-up the spinlock_64.h. The definitions of arch_spin_xxx are
replaced by the function in <asm-generic/qspinlock.h>

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig                      |    1 +
 arch/sparc/include/asm/qspinlock.h      |    7 +++
 arch/sparc/include/asm/spinlock_64.h    |   84 +------------------------------
 arch/sparc/include/asm/spinlock_types.h |    5 ++
 4 files changed, 14 insertions(+), 83 deletions(-)
 create mode 100644 arch/sparc/include/asm/qspinlock.h

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 9ec1d2f..d4a24ea 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -83,6 +83,7 @@ config SPARC64
 	select ARCH_SUPPORTS_ATOMIC_RMW
 	select HAVE_NMI
 	select ARCH_USE_QUEUED_RWLOCKS
+	select ARCH_USE_QUEUED_SPINLOCKS
 
 config ARCH_DEFCONFIG
 	string
diff --git a/arch/sparc/include/asm/qspinlock.h b/arch/sparc/include/asm/qspinlock.h
new file mode 100644
index 0000000..5ae9a28
--- /dev/null
+++ b/arch/sparc/include/asm/qspinlock.h
@@ -0,0 +1,7 @@
+#ifndef _ASM_SPARC_QSPINLOCK_H
+#define _ASM_SPARC_QSPINLOCK_H
+
+#include <asm-generic/qspinlock_types.h>
+#include <asm-generic/qspinlock.h>
+
+#endif /* _ASM_SPARC_QSPINLOCK_H */
diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
index 8901c2d..f7028f5 100644
--- a/arch/sparc/include/asm/spinlock_64.h
+++ b/arch/sparc/include/asm/spinlock_64.h
@@ -11,89 +11,7 @@
 #include <asm/processor.h>
 #include <asm/barrier.h>
 #include <asm/qrwlock.h>
-
-/* To get debugging spinlocks which detect and catch
- * deadlock situations, set CONFIG_DEBUG_SPINLOCK
- * and rebuild your kernel.
- */
-
-/* Because we play games to save cycles in the non-contention case, we
- * need to be extra careful about branch targets into the "spinning"
- * code.  They live in their own section, but the newer V9 branches
- * have a shorter range than the traditional 32-bit sparc branch
- * variants.  The rule is that the branches that go into and out of
- * the spinner sections must be pre-V9 branches.
- */
-
-#define arch_spin_is_locked(lp)	((lp)->lock != 0)
-
-static inline void arch_spin_unlock_wait(arch_spinlock_t *lock)
-{
-	smp_cond_load_acquire(&lock->lock, !VAL);
-}
-
-static inline void arch_spin_lock(arch_spinlock_t *lock)
-{
-	unsigned long tmp;
-
-	__asm__ __volatile__(
-"1:	ldstub		[%1], %0\n"
-"	brnz,pn		%0, 2f\n"
-"	 nop\n"
-"	.subsection	2\n"
-"2:	ldub		[%1], %0\n"
-"	brnz,pt		%0, 2b\n"
-"	 nop\n"
-"	ba,a,pt		%%xcc, 1b\n"
-"	.previous"
-	: "=&r" (tmp)
-	: "r" (lock)
-	: "memory");
-}
-
-static inline int arch_spin_trylock(arch_spinlock_t *lock)
-{
-	unsigned long result;
-
-	__asm__ __volatile__(
-"	ldstub		[%1], %0\n"
-	: "=r" (result)
-	: "r" (lock)
-	: "memory");
-
-	return (result == 0UL);
-}
-
-static inline void arch_spin_unlock(arch_spinlock_t *lock)
-{
-	__asm__ __volatile__(
-"	stb		%%g0, [%0]"
-	: /* No outputs */
-	: "r" (lock)
-	: "memory");
-}
-
-static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long flags)
-{
-	unsigned long tmp1, tmp2;
-
-	__asm__ __volatile__(
-"1:	ldstub		[%2], %0\n"
-"	brnz,pn		%0, 2f\n"
-"	 nop\n"
-"	.subsection	2\n"
-"2:	rdpr		%%pil, %1\n"
-"	wrpr		%3, %%pil\n"
-"3:	ldub		[%2], %0\n"
-"	brnz,pt		%0, 3b\n"
-"	 nop\n"
-"	ba,pt		%%xcc, 1b\n"
-"	 wrpr		%1, %%pil\n"
-"	.previous"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r"(lock), "r"(flags)
-	: "memory");
-}
+#include <asm/qspinlock.h>
 
 #define arch_read_lock_flags(p, f) arch_read_lock(p)
 #define arch_write_lock_flags(p, f) arch_write_lock(p)
diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index 64fce21..bce8ef4 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -1,11 +1,16 @@
 #ifndef __SPARC_SPINLOCK_TYPES_H
 #define __SPARC_SPINLOCK_TYPES_H
 
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm-generic/qspinlock_types.h>
+#else
+
 typedef struct {
 	volatile unsigned char lock;
 } arch_spinlock_t;
 
 #define __ARCH_SPIN_LOCK_UNLOCKED	{ 0 }
+#endif /* CONFIG_QUEUED_SPINLOCKS */
 
 #ifdef CONFIG_QUEUED_RWLOCKS
 #include <asm-generic/qrwlock_types.h>
-- 
1.7.1

[PATCH v2 7/7] arch/sparc: Enable queued spinlock support for SPARC

[Babu Moger]

This patch makes the necessary changes in SPARC architecture to enable
queued spinlock support. Here are some of the earlier discussions about
this feature.
https://lwn.net/Articles/561775/
https://lwn.net/Articles/590243/

Cleaned-up the spinlock_64.h. The definitions of arch_spin_xxx are
replaced by the function in <asm-generic/qspinlock.h>

Signed-off-by: Babu Moger <babu.moger@oracle.com>
Reviewed-by: HÃ¥kon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
---
 arch/sparc/Kconfig                      |    1 +
 arch/sparc/include/asm/qspinlock.h      |    7 +++
 arch/sparc/include/asm/spinlock_64.h    |   84 +------------------------------
 arch/sparc/include/asm/spinlock_types.h |    5 ++
 4 files changed, 14 insertions(+), 83 deletions(-)
 create mode 100644 arch/sparc/include/asm/qspinlock.h

diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index 9ec1d2f..d4a24ea 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -83,6 +83,7 @@ config SPARC64
 	select ARCH_SUPPORTS_ATOMIC_RMW
 	select HAVE_NMI
 	select ARCH_USE_QUEUED_RWLOCKS
+	select ARCH_USE_QUEUED_SPINLOCKS
 
 config ARCH_DEFCONFIG
 	string
diff --git a/arch/sparc/include/asm/qspinlock.h b/arch/sparc/include/asm/qspinlock.h
new file mode 100644
index 0000000..5ae9a28
--- /dev/null
+++ b/arch/sparc/include/asm/qspinlock.h
@@ -0,0 +1,7 @@
+#ifndef _ASM_SPARC_QSPINLOCK_H
+#define _ASM_SPARC_QSPINLOCK_H
+
+#include <asm-generic/qspinlock_types.h>
+#include <asm-generic/qspinlock.h>
+
+#endif /* _ASM_SPARC_QSPINLOCK_H */
diff --git a/arch/sparc/include/asm/spinlock_64.h b/arch/sparc/include/asm/spinlock_64.h
index 8901c2d..f7028f5 100644
--- a/arch/sparc/include/asm/spinlock_64.h
+++ b/arch/sparc/include/asm/spinlock_64.h
@@ -11,89 +11,7 @@
 #include <asm/processor.h>
 #include <asm/barrier.h>
 #include <asm/qrwlock.h>
-
-/* To get debugging spinlocks which detect and catch
- * deadlock situations, set CONFIG_DEBUG_SPINLOCK
- * and rebuild your kernel.
- */
-
-/* Because we play games to save cycles in the non-contention case, we
- * need to be extra careful about branch targets into the "spinning"
- * code.  They live in their own section, but the newer V9 branches
- * have a shorter range than the traditional 32-bit sparc branch
- * variants.  The rule is that the branches that go into and out of
- * the spinner sections must be pre-V9 branches.
- */
-
-#define arch_spin_is_locked(lp)	((lp)->lock != 0)
-
-static inline void arch_spin_unlock_wait(arch_spinlock_t *lock)
-{
-	smp_cond_load_acquire(&lock->lock, !VAL);
-}
-
-static inline void arch_spin_lock(arch_spinlock_t *lock)
-{
-	unsigned long tmp;
-
-	__asm__ __volatile__(
-"1:	ldstub		[%1], %0\n"
-"	brnz,pn		%0, 2f\n"
-"	 nop\n"
-"	.subsection	2\n"
-"2:	ldub		[%1], %0\n"
-"	brnz,pt		%0, 2b\n"
-"	 nop\n"
-"	ba,a,pt		%%xcc, 1b\n"
-"	.previous"
-	: "=&r" (tmp)
-	: "r" (lock)
-	: "memory");
-}
-
-static inline int arch_spin_trylock(arch_spinlock_t *lock)
-{
-	unsigned long result;
-
-	__asm__ __volatile__(
-"	ldstub		[%1], %0\n"
-	: "=r" (result)
-	: "r" (lock)
-	: "memory");
-
-	return (result = 0UL);
-}
-
-static inline void arch_spin_unlock(arch_spinlock_t *lock)
-{
-	__asm__ __volatile__(
-"	stb		%%g0, [%0]"
-	: /* No outputs */
-	: "r" (lock)
-	: "memory");
-}
-
-static inline void arch_spin_lock_flags(arch_spinlock_t *lock, unsigned long flags)
-{
-	unsigned long tmp1, tmp2;
-
-	__asm__ __volatile__(
-"1:	ldstub		[%2], %0\n"
-"	brnz,pn		%0, 2f\n"
-"	 nop\n"
-"	.subsection	2\n"
-"2:	rdpr		%%pil, %1\n"
-"	wrpr		%3, %%pil\n"
-"3:	ldub		[%2], %0\n"
-"	brnz,pt		%0, 3b\n"
-"	 nop\n"
-"	ba,pt		%%xcc, 1b\n"
-"	 wrpr		%1, %%pil\n"
-"	.previous"
-	: "=&r" (tmp1), "=&r" (tmp2)
-	: "r"(lock), "r"(flags)
-	: "memory");
-}
+#include <asm/qspinlock.h>
 
 #define arch_read_lock_flags(p, f) arch_read_lock(p)
 #define arch_write_lock_flags(p, f) arch_write_lock(p)
diff --git a/arch/sparc/include/asm/spinlock_types.h b/arch/sparc/include/asm/spinlock_types.h
index 64fce21..bce8ef4 100644
--- a/arch/sparc/include/asm/spinlock_types.h
+++ b/arch/sparc/include/asm/spinlock_types.h
@@ -1,11 +1,16 @@
 #ifndef __SPARC_SPINLOCK_TYPES_H
 #define __SPARC_SPINLOCK_TYPES_H
 
+#ifdef CONFIG_QUEUED_SPINLOCKS
+#include <asm-generic/qspinlock_types.h>
+#else
+
 typedef struct {
 	volatile unsigned char lock;
 } arch_spinlock_t;
 
 #define __ARCH_SPIN_LOCK_UNLOCKED	{ 0 }
+#endif /* CONFIG_QUEUED_SPINLOCKS */
 
 #ifdef CONFIG_QUEUED_RWLOCKS
 #include <asm-generic/qrwlock_types.h>
-- 
1.7.1

Re: [PATCH v2 1/7] kernel/locking: Fix compile error with qrwlock.c

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Fri, 19 May 2017 18:16:55 -0600

> Some architectures use the following guard in include file
>  "asm/spinlock_types.h" to discourage including the file directly.
> 
> Saw these compile errors on SPARC when queued rwlock feature is enabled.
> 
>   CC      kernel/locking/qrwlock.o
> In file included from ./include/asm-generic/qrwlock_types.h:5,
>                  from ./arch/sparc/include/asm/qrwlock.h:4,
>                  from kernel/locking/qrwlock.c:24:
> ./arch/sparc/include/asm/spinlock_types.h:5:3: error:
>         #error "please don't include this file directly"
> 
> Re-arrange the includes in qrwlock_types.h and include spinlock.h
> in qrwlock.c to fix it.
> 
> Also will be removing this stanza from SPARC. Stay tuned.
> 
> Signed-off-by: Babu Moger <babu.moger@oracle.com>
> Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
> Reviewed-by: Jane Chu <jane.chu@oracle.com>
> Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
> Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>

In my feedback I said to remove that check from
sparc's asm/spinlock_types.h header.

Which you do in patch #2

But that means this patch #1 here is no longer necessary, since we
conform to the hearder scheme used by other architectures using queued
locks.

So please remove this patch from the series.

Thank you.

Re: [PATCH v2 1/7] kernel/locking: Fix compile error with qrwlock.c

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Fri, 19 May 2017 18:16:55 -0600

> Some architectures use the following guard in include file
>  "asm/spinlock_types.h" to discourage including the file directly.
> 
> Saw these compile errors on SPARC when queued rwlock feature is enabled.
> 
>   CC      kernel/locking/qrwlock.o
> In file included from ./include/asm-generic/qrwlock_types.h:5,
>                  from ./arch/sparc/include/asm/qrwlock.h:4,
>                  from kernel/locking/qrwlock.c:24:
> ./arch/sparc/include/asm/spinlock_types.h:5:3: error:
>         #error "please don't include this file directly"
> 
> Re-arrange the includes in qrwlock_types.h and include spinlock.h
> in qrwlock.c to fix it.
> 
> Also will be removing this stanza from SPARC. Stay tuned.
> 
> Signed-off-by: Babu Moger <babu.moger@oracle.com>
> Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
> Reviewed-by: Jane Chu <jane.chu@oracle.com>
> Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
> Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>

In my feedback I said to remove that check from
sparc's asm/spinlock_types.h header.

Which you do in patch #2

But that means this patch #1 here is no longer necessary, since we
conform to the hearder scheme used by other architectures using queued
locks.

So please remove this patch from the series.

Thank you.

Re: [PATCH v2 1/7] kernel/locking: Fix compile error with qrwlock.c

[Babu Moger]

On 5/22/2017 2:03 PM, David Miller wrote:
> From: Babu Moger <babu.moger@oracle.com>
> Date: Fri, 19 May 2017 18:16:55 -0600
>
>> Some architectures use the following guard in include file
>>   "asm/spinlock_types.h" to discourage including the file directly.
>>
>> Saw these compile errors on SPARC when queued rwlock feature is enabled.
>>
>>    CC      kernel/locking/qrwlock.o
>> In file included from ./include/asm-generic/qrwlock_types.h:5,
>>                   from ./arch/sparc/include/asm/qrwlock.h:4,
>>                   from kernel/locking/qrwlock.c:24:
>> ./arch/sparc/include/asm/spinlock_types.h:5:3: error:
>>          #error "please don't include this file directly"
>>
>> Re-arrange the includes in qrwlock_types.h and include spinlock.h
>> in qrwlock.c to fix it.
>>
>> Also will be removing this stanza from SPARC. Stay tuned.
>>
>> Signed-off-by: Babu Moger <babu.moger@oracle.com>
>> Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
>> Reviewed-by: Jane Chu <jane.chu@oracle.com>
>> Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
>> Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
> In my feedback I said to remove that check from
> sparc's asm/spinlock_types.h header.
>
> Which you do in patch #2
>
> But that means this patch #1 here is no longer necessary, since we
> conform to the hearder scheme used by other architectures using queued
> locks.
>
> So please remove this patch from the series.
Sorry. I did not make it clear. Patch #2 addresses this following problem.

./arch/sparc/include/asm/spinlock_types.h:5:3: error:
#error "please don't include this file directly"

But we still have this following problem. So we needed part of the patch #1.

   CC      kernel/locking/qrwlock.o
kernel/locking/qrwlock.c: In function ‘queued_read_lock_slowpath’:
kernel/locking/qrwlock.c:89: error: implicit declaration of function 
‘arch_spin_lock’
kernel/locking/qrwlock.c:102: error: implicit declaration of function 
‘arch_spin_unlock’
make[4]: *** [kernel/locking/qrwlock.o] Error 1

To address this we need following change.
============================================
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index cc3ed0c..2655f26 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -20,6 +20,7 @@
  #include <linux/cpumask.h>
  #include <linux/percpu.h>
  #include <linux/hardirq.h>
+#include <linux/spinlock.h>
  #include <asm/qrwlock.h>

  /*
I will update the patch set with the above information. Does that work?

> Thank you.

Re: [PATCH v2 1/7] kernel/locking: Fix compile error with qrwlock.c

[Babu Moger]

On 5/22/2017 2:03 PM, David Miller wrote:
> From: Babu Moger <babu.moger@oracle.com>
> Date: Fri, 19 May 2017 18:16:55 -0600
>
>> Some architectures use the following guard in include file
>>   "asm/spinlock_types.h" to discourage including the file directly.
>>
>> Saw these compile errors on SPARC when queued rwlock feature is enabled.
>>
>>    CC      kernel/locking/qrwlock.o
>> In file included from ./include/asm-generic/qrwlock_types.h:5,
>>                   from ./arch/sparc/include/asm/qrwlock.h:4,
>>                   from kernel/locking/qrwlock.c:24:
>> ./arch/sparc/include/asm/spinlock_types.h:5:3: error:
>>          #error "please don't include this file directly"
>>
>> Re-arrange the includes in qrwlock_types.h and include spinlock.h
>> in qrwlock.c to fix it.
>>
>> Also will be removing this stanza from SPARC. Stay tuned.
>>
>> Signed-off-by: Babu Moger <babu.moger@oracle.com>
>> Reviewed-by: Håkon Bugge <haakon.bugge@oracle.com>
>> Reviewed-by: Jane Chu <jane.chu@oracle.com>
>> Reviewed-by: Shannon Nelson <shannon.nelson@oracle.com>
>> Reviewed-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
> In my feedback I said to remove that check from
> sparc's asm/spinlock_types.h header.
>
> Which you do in patch #2
>
> But that means this patch #1 here is no longer necessary, since we
> conform to the hearder scheme used by other architectures using queued
> locks.
>
> So please remove this patch from the series.
Sorry. I did not make it clear. Patch #2 addresses this following problem.

./arch/sparc/include/asm/spinlock_types.h:5:3: error:
#error "please don't include this file directly"

But we still have this following problem. So we needed part of the patch #1.

   CC      kernel/locking/qrwlock.o
kernel/locking/qrwlock.c: In function ‘queued_read_lock_slowpath’:
kernel/locking/qrwlock.c:89: error: implicit declaration of function 
‘arch_spin_lock’
kernel/locking/qrwlock.c:102: error: implicit declaration of function 
‘arch_spin_unlock’
make[4]: *** [kernel/locking/qrwlock.o] Error 1

To address this we need following change.
======================
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index cc3ed0c..2655f26 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -20,6 +20,7 @@
  #include <linux/cpumask.h>
  #include <linux/percpu.h>
  #include <linux/hardirq.h>
+#include <linux/spinlock.h>
  #include <asm/qrwlock.h>

  /*
I will update the patch set with the above information. Does that work?

> Thank you.

Re: [PATCH v2 1/7] kernel/locking: Fix compile error with qrwlock.c

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Mon, 22 May 2017 16:57:39 -0500

> I will update the patch set with the above information. Does that
> work?

Yes it does, I didn't realize there were multiple problems.

Thanks.

Re: [PATCH v2 1/7] kernel/locking: Fix compile error with qrwlock.c

[David Miller]

From: Babu Moger <babu.moger@oracle.com>
Date: Mon, 22 May 2017 16:57:39 -0500

> I will update the patch set with the above information. Does that
> work?

Yes it does, I didn't realize there were multiple problems.

Thanks.