From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754229AbbGIUjL (ORCPT ); Thu, 9 Jul 2015 16:39:11 -0400 Received: from mail-am1on0083.outbound.protection.outlook.com ([157.56.112.83]:14880 "EHLO emea01-am1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753995AbbGIUiq (ORCPT ); Thu, 9 Jul 2015 16:38:46 -0400 Authentication-Results: spf=fail (sender IP is 12.216.194.146) smtp.mailfrom=ezchip.com; kernel.org; dkim=none (message not signed) header.d=none; From: Chris Metcalf To: , , CC: Chris Metcalf , , , , , , , , , , , , , , , , , Subject: [PATCH] tile: Provide atomic_{or,xor,and} Date: Thu, 9 Jul 2015 16:38:17 -0400 Message-ID: <1436474297-32187-1-git-send-email-cmetcalf@ezchip.com> X-Mailer: git-send-email 2.1.2 In-Reply-To: <20150709172855.564686637@infradead.org> X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1;DB3FFO11FD011;1:MFR+SOm9gKCAKnOCVuKW8PfJ6DTHVboKBikNGD2oeki+EBFhssbSwKqjid57qky4dSqxOgwLcJaW5ypaNRlHI/ilcqERjjxb1eFWArMDfxIzroH8fUt1wXQCTp7zJ1GzehWm4njOy0F7X7JQcuLLUcyB9qpkN+6KOOeGrv912ULLpFA+gV/4dwAGkve4WrHhQjAuyLTcrH+8fmWOYRcJgpfAsLvftCiPrEjn8xiSbPs8CZzr++faC8PzPI1rk2hi69HfU1h1Fl4w7YVbnd5NRhoC7FbLkNw9cM0E4JaV41QbK+uacB4lyG6Rg2dA+cezNSsWh6ktlPd/V/Nydx0FkGLQEIcpYvVIp8JPEzz587k1zhvC/pQ+Zr/WWSdah/tx+VHuzDPte0CPKq3gZ906CERuAq84g78KqFEZocW3W7MhD2N8/5XLYKKmR2+J6nIL X-Forefront-Antispam-Report: CIP:12.216.194.146;CTRY:US;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(2980300002)(339900001)(199003)(189002)(5001770100001)(189998001)(104016003)(106466001)(87936001)(33646002)(229853001)(92566002)(5001960100002)(105606002)(2950100001)(50226001)(36756003)(5003940100001)(86362001)(46102003)(77156002)(62966003)(6806004)(2201001)(19580405001)(85426001)(19580395003)(50466002)(50986999)(48376002)(42186005)(47776003)(142933001);DIR:OUT;SFP:1101;SCL:1;SRVR:AM2PR02MB0769;H:ld-1.internal.tilera.com;FPR:;SPF:Fail;MLV:sfv;A:1;MX:1;LANG:en; MIME-Version: 1.0 Content-Type: text/plain X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0769;2:QTqD9gLl7qNA2qkxz6evJaAByXfdqUHkRV7uPV55hEQtg5KRO8IDDfL+b9IVt16S;3:MnxsXR3ivV0lue6A6OUvgPiv2h+ZkPFqusFpkovBz5MdW/Ne3B2QAqSX5V7X1rZgda1B0b4iwm6bssyHUUO3hyxMnbK0NIIWgxZS2gH1OKBEK4yxGNmGcoaJVKKjy/nqnwLs3R6v5O30pWCGVCv669b8h0UBX0+uMPmvef2JgKIiu4yqn5o70FhBHOLicocSDMtj7824oWb/EGmIvi6OTok9KorZIFh3MxG4ZRg9FZe3U1Xm3rXTXReiW9M+wRgm;25:MDIOQ85TgwKEdIzND1aLN5OWrXb/WrX2feX9GoLaTpuhu5OnEhIi1iyaSlncbRCDCORjbI2kCqsXIpzudh2g0G34HO7lALNL25kiE/At1rXcXK34um16DIKIjpRF8RtmdlVa4rZOoeNRR6WgirYyi5bzS2AJuql/VAMF23Wg1deYdLOP/uqx1CxE2IHDll5nbC7jI49y1FwfccKg+bvi2Y667LlTYvi6tT2DT/j+X1BoDXwE0CYs8JHkC4h0g/aSvsDvLO1V3ESGiKoUdzHKbw==;20:aFoJuu3+9NcOGeEjMNfqyvfuKVJcOA+3WdoZsjj/Tz/SXvb9js2H+pRRuMYRPo+OJbM8sE9IhwC1nklVyDJAsgHSv19pSiyvga1chUd9gMDuaWjS7bLMCkEpV9r0R9B5khl9VS1wlyGjSOZgT5Zl17q83IWkCBCllJ6E/f4Bczw= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0769; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(5005006)(3002001);SRVR:AM2PR02MB0769;BCL:0;PCL:0;RULEID:;SRVR:AM2PR02MB0769; X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0769;4:HXAF7M5wkEu0b19ncDWDxDNEl4BThSLy8W41I1MMt2NI8j2IsqIA0dsLv7KQoqpvwOLk1Hz1L1fZWeyDlQ0FK6XLEZOgmah0ou0lBpdIaLkIlY/1ivnD26+VFvB6W902cKqaf9pNxssd3+XO/nVdsydal22PCbQqTb3Z5J6cq6zoLb+Y6nhA31UuofKEFTrfR/wk4sGXxUajOaCCqhFhg5q1PuXgx4k4vfvPdG/2ukOpZvUKwSVzAVO3OKXgzHsctOwq+4K/Q0w7brJdc6UoYlRZdViB5yzq6eWwRR02b0Q= X-Forefront-PRVS: 0632519F33 X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0769;23:DC+eHmQgvxxuPLgCr9LrN33ZEm2/87IZpbaOT3xqIfhLpN3jNeVNghlvlteZVWvrn5pdhmOR1VFWt7AYYCY4fSrW7lR5sR/mzwzo6v5Os/GZeA6Ydv3NjZFIiOiw8fVgCV7T2lzRAS3/wB0pMenSmffTgNtuyTW2qYS2Fo48EuWhi0Bfq/mfFF0LUiGcc1M6UmToMVhUwqtwKgacYjLw/ul65UkxYSz+AGcPn3YfCghV+msCJ2ySUEua2dDB7oQbzqfJtNB7jB+vPPr+CeXDB603lT5FJMTF8nY3BnjEOv7Thw+8kR8lpKl2F2JVD/x96SiViDKs8H7JXlJ29fGhVFU97uUWJpx8AGbVH6VCeqrkvLsc+kZFJ1YraB9ujydmas/nzIlhfSCW9yCUwKoeOd8zLrM1RK9BGkm+14Fv9iweVqF6jFUx4h+BDfi9xX83PAFSbvJSVOpSz7oOK5jBgtVd9aBrJKU+FK8erLmHEmFsiX28ncjTqdj6ln+28cF7OvRYdaOTqxkJ4WGzZ5cYN4rAvruapys5OVGtXl4YV2KpHKevz3jnO3EjY+rjFp1XbtbZ8kPzjdWHDtWTMIYXvDJ3Vse8dBAankKziF5sZ7eMuU/l7dgn3HwLrhnpwRhO7MW3+GEMmopNqqvamwRN7W/ScaFY7Puvw5isQXL3MwbJgcDWRHyMsXsIMr531E5VOXQPWe45xArRRIJkCd8vSh3cB8zb80GVwCouLyWxt/ErdZ/Cr1ALDNBnsqnNojdBr8yh5YYJw3OwditXVGNEmZVCiX0K7XXgKlhBJUgvuKUbzY/5jb7Gw2hL4k6MrERdVznVbzXV/DU/GbcgxQGeWO6hX8lp/piu0b6BJL7O2nbBk9MW6hQ8dkp1w9cARIC36/R2PPTVpdbBNVnEy6zCr7D7JmzZFd36+JBtlAE7A6M= X-Microsoft-Exchange-Diagnostics: 1;AM2PR02MB0769;5:xcPySLVI12x4khndfuGZl2NZSl7Fk03GBt42tKvEAFR/Gg0uLExwXZbrvFtzOKd4udjLZxch1e7gBjY0Geb2RvDqUlvICLwxfx99h2ORmU9M5VpGOrj/pwW86gi52iE74FV5fikfPR00klvpqoPPYw==;24:ESkSBgTJFZUqfb/496jGJsH+h8cE9vFA2Fs4fpIg2Qs6xtQs6zZ33MxOOXHDX56vXOAPnxSD2tLelauf9Slv5BkAQn4Nr1X3emIJdXePMxA=;20:g2S+XqBCQo4+3x29cltJFtdjx57QfVaaAYu6ivg/ZvylYzT/kt0eyAhaZIEcgCbx4H8prhjY1mwm8DA42zNFUQ== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: ezchip.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jul 2015 20:38:40.4933 (UTC) X-MS-Exchange-CrossTenant-Id: 0fc16e0a-3cd3-4092-8b2f-0a42cff122c3 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0fc16e0a-3cd3-4092-8b2f-0a42cff122c3;Ip=[12.216.194.146];Helo=[ld-1.internal.tilera.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR02MB0769 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Implement atomic logic ops -- atomic_{or,xor,and}. For tilegx, these are relatively straightforward; the architecture provides atomic "or" and "and", both 32-bit and 64-bit. To support xor we provide a loop using "cmpexch". For the older 32-bit tilepro architecture, we have to extend the set of low-level assembly routines to include 32-bit "and", as well as all three 64-bit routines. Somewhat confusingly, some 32-bit versions are already used by the bitops inlines, with parameter types appropriate for bitops, so we have to do a bit of casting to match "int" to "unsigned long". Signed-off-by: Chris Metcalf --- Peter, I'm guessing you should just take this into your series, rather than my pushing it through the tile tree. arch/tile/include/asm/atomic_32.h | 28 +++++++++++++++++++++++++++ arch/tile/include/asm/atomic_64.h | 40 +++++++++++++++++++++++++++++++++++++++ arch/tile/lib/atomic_32.c | 23 ++++++++++++++++++++++ arch/tile/lib/atomic_asm_32.S | 4 ++++ 4 files changed, 95 insertions(+) diff --git a/arch/tile/include/asm/atomic_32.h b/arch/tile/include/asm/atomic_32.h index 1b109fad9fff..d320ce253d86 100644 --- a/arch/tile/include/asm/atomic_32.h +++ b/arch/tile/include/asm/atomic_32.h @@ -34,6 +34,19 @@ static inline void atomic_add(int i, atomic_t *v) _atomic_xchg_add(&v->counter, i); } +#define ATOMIC_OP(op) \ +unsigned long _atomic_##op(volatile unsigned long *p, unsigned long mask); \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + _atomic_##op((unsigned long *)&v->counter, i); \ +} + +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + +#undef ATOMIC_OP + /** * atomic_add_return - add integer and return * @v: pointer of type atomic_t @@ -113,6 +126,17 @@ static inline void atomic64_add(long long i, atomic64_t *v) _atomic64_xchg_add(&v->counter, i); } +#define ATOMIC64_OP(op) \ +long long _atomic64_##op(long long *v, long long n); \ +static inline void atomic64_##op(long long i, atomic64_t *v) \ +{ \ + _atomic64_##op(&v->counter, i); \ +} + +ATOMIC64_OP(and) +ATOMIC64_OP(or) +ATOMIC64_OP(xor) + /** * atomic64_add_return - add integer and return * @v: pointer of type atomic64_t @@ -225,6 +249,7 @@ extern struct __get_user __atomic_xchg_add(volatile int *p, int *lock, int n); extern struct __get_user __atomic_xchg_add_unless(volatile int *p, int *lock, int o, int n); extern struct __get_user __atomic_or(volatile int *p, int *lock, int n); +extern struct __get_user __atomic_and(volatile int *p, int *lock, int n); extern struct __get_user __atomic_andn(volatile int *p, int *lock, int n); extern struct __get_user __atomic_xor(volatile int *p, int *lock, int n); extern long long __atomic64_cmpxchg(volatile long long *p, int *lock, @@ -234,6 +259,9 @@ extern long long __atomic64_xchg_add(volatile long long *p, int *lock, long long n); extern long long __atomic64_xchg_add_unless(volatile long long *p, int *lock, long long o, long long n); +extern long long __atomic64_and(volatile long long *p, int *lock, long long n); +extern long long __atomic64_or(volatile long long *p, int *lock, long long n); +extern long long __atomic64_xor(volatile long long *p, int *lock, long long n); /* Return failure from the atomic wrappers. */ struct __get_user __atomic_bad_address(int __user *addr); diff --git a/arch/tile/include/asm/atomic_64.h b/arch/tile/include/asm/atomic_64.h index 0496970cef82..096a56d6ead4 100644 --- a/arch/tile/include/asm/atomic_64.h +++ b/arch/tile/include/asm/atomic_64.h @@ -58,6 +58,26 @@ static inline int __atomic_add_unless(atomic_t *v, int a, int u) return oldval; } +static inline void atomic_and(int i, atomic_t *v) +{ + __insn_fetchand4((void *)&v->counter, i); +} + +static inline void atomic_or(int i, atomic_t *v) +{ + __insn_fetchor4((void *)&v->counter, i); +} + +static inline void atomic_xor(int i, atomic_t *v) +{ + int guess, oldval = v->counter; + do { + guess = oldval; + __insn_mtspr(SPR_CMPEXCH_VALUE, guess); + oldval = __insn_cmpexch4(&v->counter, guess ^ i); + } while (guess != oldval); +} + /* Now the true 64-bit operations. */ #define ATOMIC64_INIT(i) { (i) } @@ -91,6 +111,26 @@ static inline long atomic64_add_unless(atomic64_t *v, long a, long u) return oldval != u; } +static inline void atomic64_and(long i, atomic64_t *v) +{ + __insn_fetchand((void *)&v->counter, i); +} + +static inline void atomic64_or(long i, atomic64_t *v) +{ + __insn_fetchor((void *)&v->counter, i); +} + +static inline void atomic64_xor(long i, atomic64_t *v) +{ + long guess, oldval = v->counter; + do { + guess = oldval; + __insn_mtspr(SPR_CMPEXCH_VALUE, guess); + oldval = __insn_cmpexch(&v->counter, guess ^ i); + } while (guess != oldval); +} + #define atomic64_sub_return(i, v) atomic64_add_return(-(i), (v)) #define atomic64_sub(i, v) atomic64_add(-(i), (v)) #define atomic64_inc_return(v) atomic64_add_return(1, (v)) diff --git a/arch/tile/lib/atomic_32.c b/arch/tile/lib/atomic_32.c index c89b211fd9e7..298df1e9912a 100644 --- a/arch/tile/lib/atomic_32.c +++ b/arch/tile/lib/atomic_32.c @@ -94,6 +94,12 @@ unsigned long _atomic_or(volatile unsigned long *p, unsigned long mask) } EXPORT_SYMBOL(_atomic_or); +unsigned long _atomic_and(volatile unsigned long *p, unsigned long mask) +{ + return __atomic_and((int *)p, __atomic_setup(p), mask).val; +} +EXPORT_SYMBOL(_atomic_and); + unsigned long _atomic_andn(volatile unsigned long *p, unsigned long mask) { return __atomic_andn((int *)p, __atomic_setup(p), mask).val; @@ -136,6 +142,23 @@ long long _atomic64_cmpxchg(long long *v, long long o, long long n) } EXPORT_SYMBOL(_atomic64_cmpxchg); +long long _atomic64_and(long long *v, long long n) +{ + return __atomic64_and(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_and); + +long long _atomic64_or(long long *v, long long n) +{ + return __atomic64_or(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_or); + +long long _atomic64_xor(long long *v, long long n) +{ + return __atomic64_xor(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_xor); /* * If any of the atomic or futex routines hit a bad address (not in diff --git a/arch/tile/lib/atomic_asm_32.S b/arch/tile/lib/atomic_asm_32.S index 6bda3132cd61..f611265633d6 100644 --- a/arch/tile/lib/atomic_asm_32.S +++ b/arch/tile/lib/atomic_asm_32.S @@ -178,6 +178,7 @@ atomic_op _xchg_add, 32, "add r24, r22, r2" atomic_op _xchg_add_unless, 32, \ "sne r26, r22, r2; { bbns r26, 3f; add r24, r22, r3 }" atomic_op _or, 32, "or r24, r22, r2" +atomic_op _and, 32, "and r24, r22, r2" atomic_op _andn, 32, "nor r2, r2, zero; and r24, r22, r2" atomic_op _xor, 32, "xor r24, r22, r2" @@ -191,6 +192,9 @@ atomic_op 64_xchg_add_unless, 64, \ { bbns r26, 3f; add r24, r22, r4 }; \ { bbns r27, 3f; add r25, r23, r5 }; \ slt_u r26, r24, r22; add r25, r25, r26" +atomic_op 64_or, 64, "{ or r24, r22, r2; or r25, r23, r3 }" +atomic_op 64_and, 64, "{ and r24, r22, r2; and r25, r23, r3 }" +atomic_op 64_xor, 64, "{ xor r24, r22, r2; xor r25, r23, r3 }" jrp lr /* happy backtracer */ -- 2.1.2 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Metcalf Subject: [PATCH] tile: Provide atomic_{or,xor,and} Date: Thu, 9 Jul 2015 16:38:17 -0400 Message-ID: <1436474297-32187-1-git-send-email-cmetcalf@ezchip.com> References: <20150709172855.564686637@infradead.org> Mime-Version: 1.0 Content-Type: text/plain Return-path: In-Reply-To: <20150709172855.564686637@infradead.org> Sender: linux-kernel-owner@vger.kernel.org To: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, peterz@infradead.org Cc: Chris Metcalf , rth@twiddle.net, vgupta@synopsys.com, linux@arm.linux.org.uk, will.deacon@arm.com, hskinnemoen@gmail.com, realmz6@gmail.com, dhowells@redhat.com, rkuo@codeaurora.org, tony.luck@intel.com, geert@linux-m68k.org, james.hogan@imgtec.com, ralf@linux-mips.org, jejb@parisc-linux.org, benh@kernel.crashing.org, heiko.carstens@de.ibm.com, davem@davemloft.net, mingo@kernel.org List-Id: linux-arch.vger.kernel.org Implement atomic logic ops -- atomic_{or,xor,and}. For tilegx, these are relatively straightforward; the architecture provides atomic "or" and "and", both 32-bit and 64-bit. To support xor we provide a loop using "cmpexch". For the older 32-bit tilepro architecture, we have to extend the set of low-level assembly routines to include 32-bit "and", as well as all three 64-bit routines. Somewhat confusingly, some 32-bit versions are already used by the bitops inlines, with parameter types appropriate for bitops, so we have to do a bit of casting to match "int" to "unsigned long". Signed-off-by: Chris Metcalf --- Peter, I'm guessing you should just take this into your series, rather than my pushing it through the tile tree. arch/tile/include/asm/atomic_32.h | 28 +++++++++++++++++++++++++++ arch/tile/include/asm/atomic_64.h | 40 +++++++++++++++++++++++++++++++++++++++ arch/tile/lib/atomic_32.c | 23 ++++++++++++++++++++++ arch/tile/lib/atomic_asm_32.S | 4 ++++ 4 files changed, 95 insertions(+) diff --git a/arch/tile/include/asm/atomic_32.h b/arch/tile/include/asm/atomic_32.h index 1b109fad9fff..d320ce253d86 100644 --- a/arch/tile/include/asm/atomic_32.h +++ b/arch/tile/include/asm/atomic_32.h @@ -34,6 +34,19 @@ static inline void atomic_add(int i, atomic_t *v) _atomic_xchg_add(&v->counter, i); } +#define ATOMIC_OP(op) \ +unsigned long _atomic_##op(volatile unsigned long *p, unsigned long mask); \ +static inline void atomic_##op(int i, atomic_t *v) \ +{ \ + _atomic_##op((unsigned long *)&v->counter, i); \ +} + +ATOMIC_OP(and) +ATOMIC_OP(or) +ATOMIC_OP(xor) + +#undef ATOMIC_OP + /** * atomic_add_return - add integer and return * @v: pointer of type atomic_t @@ -113,6 +126,17 @@ static inline void atomic64_add(long long i, atomic64_t *v) _atomic64_xchg_add(&v->counter, i); } +#define ATOMIC64_OP(op) \ +long long _atomic64_##op(long long *v, long long n); \ +static inline void atomic64_##op(long long i, atomic64_t *v) \ +{ \ + _atomic64_##op(&v->counter, i); \ +} + +ATOMIC64_OP(and) +ATOMIC64_OP(or) +ATOMIC64_OP(xor) + /** * atomic64_add_return - add integer and return * @v: pointer of type atomic64_t @@ -225,6 +249,7 @@ extern struct __get_user __atomic_xchg_add(volatile int *p, int *lock, int n); extern struct __get_user __atomic_xchg_add_unless(volatile int *p, int *lock, int o, int n); extern struct __get_user __atomic_or(volatile int *p, int *lock, int n); +extern struct __get_user __atomic_and(volatile int *p, int *lock, int n); extern struct __get_user __atomic_andn(volatile int *p, int *lock, int n); extern struct __get_user __atomic_xor(volatile int *p, int *lock, int n); extern long long __atomic64_cmpxchg(volatile long long *p, int *lock, @@ -234,6 +259,9 @@ extern long long __atomic64_xchg_add(volatile long long *p, int *lock, long long n); extern long long __atomic64_xchg_add_unless(volatile long long *p, int *lock, long long o, long long n); +extern long long __atomic64_and(volatile long long *p, int *lock, long long n); +extern long long __atomic64_or(volatile long long *p, int *lock, long long n); +extern long long __atomic64_xor(volatile long long *p, int *lock, long long n); /* Return failure from the atomic wrappers. */ struct __get_user __atomic_bad_address(int __user *addr); diff --git a/arch/tile/include/asm/atomic_64.h b/arch/tile/include/asm/atomic_64.h index 0496970cef82..096a56d6ead4 100644 --- a/arch/tile/include/asm/atomic_64.h +++ b/arch/tile/include/asm/atomic_64.h @@ -58,6 +58,26 @@ static inline int __atomic_add_unless(atomic_t *v, int a, int u) return oldval; } +static inline void atomic_and(int i, atomic_t *v) +{ + __insn_fetchand4((void *)&v->counter, i); +} + +static inline void atomic_or(int i, atomic_t *v) +{ + __insn_fetchor4((void *)&v->counter, i); +} + +static inline void atomic_xor(int i, atomic_t *v) +{ + int guess, oldval = v->counter; + do { + guess = oldval; + __insn_mtspr(SPR_CMPEXCH_VALUE, guess); + oldval = __insn_cmpexch4(&v->counter, guess ^ i); + } while (guess != oldval); +} + /* Now the true 64-bit operations. */ #define ATOMIC64_INIT(i) { (i) } @@ -91,6 +111,26 @@ static inline long atomic64_add_unless(atomic64_t *v, long a, long u) return oldval != u; } +static inline void atomic64_and(long i, atomic64_t *v) +{ + __insn_fetchand((void *)&v->counter, i); +} + +static inline void atomic64_or(long i, atomic64_t *v) +{ + __insn_fetchor((void *)&v->counter, i); +} + +static inline void atomic64_xor(long i, atomic64_t *v) +{ + long guess, oldval = v->counter; + do { + guess = oldval; + __insn_mtspr(SPR_CMPEXCH_VALUE, guess); + oldval = __insn_cmpexch(&v->counter, guess ^ i); + } while (guess != oldval); +} + #define atomic64_sub_return(i, v) atomic64_add_return(-(i), (v)) #define atomic64_sub(i, v) atomic64_add(-(i), (v)) #define atomic64_inc_return(v) atomic64_add_return(1, (v)) diff --git a/arch/tile/lib/atomic_32.c b/arch/tile/lib/atomic_32.c index c89b211fd9e7..298df1e9912a 100644 --- a/arch/tile/lib/atomic_32.c +++ b/arch/tile/lib/atomic_32.c @@ -94,6 +94,12 @@ unsigned long _atomic_or(volatile unsigned long *p, unsigned long mask) } EXPORT_SYMBOL(_atomic_or); +unsigned long _atomic_and(volatile unsigned long *p, unsigned long mask) +{ + return __atomic_and((int *)p, __atomic_setup(p), mask).val; +} +EXPORT_SYMBOL(_atomic_and); + unsigned long _atomic_andn(volatile unsigned long *p, unsigned long mask) { return __atomic_andn((int *)p, __atomic_setup(p), mask).val; @@ -136,6 +142,23 @@ long long _atomic64_cmpxchg(long long *v, long long o, long long n) } EXPORT_SYMBOL(_atomic64_cmpxchg); +long long _atomic64_and(long long *v, long long n) +{ + return __atomic64_and(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_and); + +long long _atomic64_or(long long *v, long long n) +{ + return __atomic64_or(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_or); + +long long _atomic64_xor(long long *v, long long n) +{ + return __atomic64_xor(v, __atomic_setup(v), n); +} +EXPORT_SYMBOL(_atomic64_xor); /* * If any of the atomic or futex routines hit a bad address (not in diff --git a/arch/tile/lib/atomic_asm_32.S b/arch/tile/lib/atomic_asm_32.S index 6bda3132cd61..f611265633d6 100644 --- a/arch/tile/lib/atomic_asm_32.S +++ b/arch/tile/lib/atomic_asm_32.S @@ -178,6 +178,7 @@ atomic_op _xchg_add, 32, "add r24, r22, r2" atomic_op _xchg_add_unless, 32, \ "sne r26, r22, r2; { bbns r26, 3f; add r24, r22, r3 }" atomic_op _or, 32, "or r24, r22, r2" +atomic_op _and, 32, "and r24, r22, r2" atomic_op _andn, 32, "nor r2, r2, zero; and r24, r22, r2" atomic_op _xor, 32, "xor r24, r22, r2" @@ -191,6 +192,9 @@ atomic_op 64_xchg_add_unless, 64, \ { bbns r26, 3f; add r24, r22, r4 }; \ { bbns r27, 3f; add r25, r23, r5 }; \ slt_u r26, r24, r22; add r25, r25, r26" +atomic_op 64_or, 64, "{ or r24, r22, r2; or r25, r23, r3 }" +atomic_op 64_and, 64, "{ and r24, r22, r2; and r25, r23, r3 }" +atomic_op 64_xor, 64, "{ xor r24, r22, r2; xor r25, r23, r3 }" jrp lr /* happy backtracer */ -- 2.1.2