From mboxrd@z Thu Jan 1 00:00:00 1970 From: Feng Wu Subject: [PATCH v9 02/17] Add cmpxchg16b support for x86-64 Date: Tue, 3 Nov 2015 16:43:12 +0800 Message-ID: <1446540207-4806-3-git-send-email-feng.wu@intel.com> References: <1446540207-4806-1-git-send-email-feng.wu@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1446540207-4806-1-git-send-email-feng.wu@intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org Cc: Keir Fraser , Feng Wu , Jan Beulich , Andrew Cooper List-Id: xen-devel@lists.xenproject.org This patch adds cmpxchg16b support for x86-64, so software can perform 128-bit atomic write/read. CC: Keir Fraser CC: Jan Beulich CC: Andrew Cooper Signed-off-by: Feng Wu --- v9: - Make the *ptr operand an input and output. v8: - Remove pointless cast when assigning 'new_low' - properly parenthesize cmpxchg16b() v7: - Make the last two parameters of __cmpxchg16b() const - Remove memory clobber - Add run-time and build-build check in cmpxchg16b() - Cast the last two parameter to void * when calling __cmpxchg16b() v6: - Fix a typo v5: - Change back the parameters of __cmpxchg16b() to __uint128_t * - Remove pointless cast for 'ptr' - Remove pointless parentheses - Use A constraint for the output v4: - Use pointer as the parameter of __cmpxchg16b(). - Use gcc's __uint128_t built-in type - Make the parameters of __cmpxchg16b() void * v3: - Newly added. xen/include/asm-x86/x86_64/system.h | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/xen/include/asm-x86/x86_64/system.h b/xen/include/asm-x86/x86_64/system.h index 662813a..7026c05 100644 --- a/xen/include/asm-x86/x86_64/system.h +++ b/xen/include/asm-x86/x86_64/system.h @@ -6,6 +6,39 @@ (unsigned long)(n),sizeof(*(ptr)))) /* + * Atomic 16 bytes compare and exchange. Compare OLD with MEM, if + * identical, store NEW in MEM. Return the initial value in MEM. + * Success is indicated by comparing RETURN with OLD. + * + * This function can only be called when cpu_has_cx16 is true. + */ + +static always_inline __uint128_t __cmpxchg16b( + volatile void *ptr, const __uint128_t *old, const __uint128_t *new) +{ + __uint128_t prev; + uint64_t new_high = *new >> 64; + uint64_t new_low = *new; + + ASSERT(cpu_has_cx16); + + asm volatile ( "lock; cmpxchg16b %1" + : "=A" (prev), "+m" (*__xg(ptr)) + : "c" (new_high), "b" (new_low), + "0" (*old) ); + + return prev; +} + +#define cmpxchg16b(ptr, o, n) ({ \ + volatile void *_p = (ptr); \ + ASSERT(!((unsigned long)_p & 0xf)); \ + BUILD_BUG_ON(sizeof(*(o)) != sizeof(__uint128_t)); \ + BUILD_BUG_ON(sizeof(*(n)) != sizeof(__uint128_t)); \ + __cmpxchg16b(_p, (void *)(o), (void *)(n)); \ +}) + +/* * This function causes value _o to be changed to _n at location _p. * If this access causes a fault then we return 1, otherwise we return 0. * If no fault occurs then _o is updated to the value we saw at _p. If this -- 2.1.0