[PATCH] [URCU] Allow forcing the use of sys membarrier

[PATCH] [URCU] Allow forcing the use of sys membarrier

[Duncan Sands]

[-- Attachment #1: Type: text/plain, Size: 864 bytes --]

When using the RCU_MEMBARRIER flavour of userspace-RCU, kernel support for 
sys-membarrier is detected dynamically and stored in the rcu_has_sys_membarrier 
global variable.  Checking the value of this variable adds a small but 
measurable overhead to smp_mb_slave.  I only use userspace-rcu on systems which 
support sys-membarrier, and I'd like a way of avoiding that overhead.

The attached patch makes the following changes: if RCU_FORCE_SYS_MEMBARRIER is 
defined then rcu_has_sys_membarrier is replaced with the constant 1, eliminating 
the overhead in smp_mb_slave.  As a sanity check, support for sys-membarrier is 
still detected at startup and if it isn't supported then the program aborts.

I didn't try to integrate this feature into the build system (eg by adding a 
configure option for it) since I'm using my own build system.

Best wishes, Duncan.

[-- Attachment #2: force-sys-membarrier.diff --]
[-- Type: text/x-patch, Size: 1544 bytes --]

Index: userspace-rcu-0.9.2/urcu/map/urcu.h
===================================================================
--- userspace-rcu-0.9.2/urcu/map/urcu.h	(revision 32603)
+++ userspace-rcu-0.9.2/urcu/map/urcu.h	(working copy)
@@ -80,9 +80,6 @@
 
 #define rcu_flavor			rcu_flavor_memb
 
-/* Specific to MEMBARRIER flavor */
-#define rcu_has_sys_membarrier		rcu_has_sys_membarrier_memb
-
 #elif defined(RCU_SIGNAL)
 
 #define rcu_read_lock			rcu_read_lock_sig
Index: userspace-rcu-0.9.2/urcu/static/urcu.h
===================================================================
--- userspace-rcu-0.9.2/urcu/static/urcu.h	(revision 32603)
+++ userspace-rcu-0.9.2/urcu/static/urcu.h	(working copy)
@@ -90,7 +90,11 @@
  */
 
 #ifdef RCU_MEMBARRIER
+#ifdef RCU_FORCE_SYS_MEMBARRIER
+#define rcu_has_sys_membarrier 1
+#else
 extern int rcu_has_sys_membarrier;
+#endif
 
 static inline void smp_mb_slave(void)
 {
Index: userspace-rcu-0.9.2/urcu.c
===================================================================
--- userspace-rcu-0.9.2/urcu.c	(revision 32603)
+++ userspace-rcu-0.9.2/urcu.c	(working copy)
@@ -78,7 +78,9 @@
 
 #ifdef RCU_MEMBARRIER
 static int init_done;
+#ifndef RCU_FORCE_SYS_MEMBARRIER
 int rcu_has_sys_membarrier;
+#endif
 
 void __attribute__((constructor)) rcu_init(void);
 #endif
@@ -542,7 +544,12 @@
 	init_done = 1;
 	ret = membarrier(MEMBARRIER_CMD_QUERY, 0);
 	if (ret >= 0 && (ret & MEMBARRIER_CMD_SHARED)) {
+#ifndef RCU_FORCE_SYS_MEMBARRIER
 		rcu_has_sys_membarrier = 1;
+#else
+	} else {
+		abort();
+#endif
 	}
 }
 #endif

[-- Attachment #3: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [PATCH] [URCU] Allow forcing the use of sys membarrier

[Mathieu Desnoyers]

----- On Sep 16, 2016, at 3:58 PM, Mathieu Desnoyers mathieu.desnoyers@efficios.com wrote:

> ----- On Sep 16, 2016, at 11:38 AM, Duncan Sands duncan.sands@deepbluecap.com
> wrote:
> 
>> Hi Mathieu,
>> 
>> On 09/12/2016 05:05 PM, Mathieu Desnoyers wrote:
>>> ----- On Sep 6, 2016, at 9:07 AM, Duncan Sands duncan.sands@deepbluecap.com
>>> wrote:
>>>
>>>> When using the RCU_MEMBARRIER flavour of userspace-RCU, kernel support for
>>>> sys-membarrier is detected dynamically and stored in the rcu_has_sys_membarrier
>>>> global variable.  Checking the value of this variable adds a small but
>>>> measurable overhead to smp_mb_slave.  I only use userspace-rcu on systems which
>>>> support sys-membarrier, and I'd like a way of avoiding that overhead.
>>>>
>>>> The attached patch makes the following changes: if RCU_FORCE_SYS_MEMBARRIER is
>>>> defined then rcu_has_sys_membarrier is replaced with the constant 1, eliminating
>>>> the overhead in smp_mb_slave.  As a sanity check, support for sys-membarrier is
>>>> still detected at startup and if it isn't supported then the program aborts.
>>>>
>>>> I didn't try to integrate this feature into the build system (eg by adding a
>>>> configure option for it) since I'm using my own build system.
>>>
>>> Hi Duncan,
>>>
>>> This is an interesting improvement. I'd be interested to merge it if it was
>>> integrated with the liburcu build system. Perhaps a
>>> --disable-dynamic-membarrier-check
>>> at configure ?
>> 
>> how about the attached patch?
> 
> I pushed the following commit into urcu master, inspired from your patch,
> 
> commit d8d9a3405ce46af6d34d2e80e260ad50f3d211a0
> Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Date:   Fri Sep 16 15:15:41 2016 -0400
> 
>    Allow forcing the use of sys membarrier
>    
>    When using the default (liburcu.so) and bulletproof (liburcu-bp.so)
>    flavours of Userspace RCU, kernel support for sys-membarrier is detected
>    dynamically and stored in the rcu_has_sys_membarrier_memb and
>    urcu_bp_has_sys_membarrier global variables.
>    
>    Checking the value of these variables adds a small but measurable overhead
>    to smp_mb_slave. On systems which support sys-membarrier, it would be
>    nice to have a way of avoiding that overhead.
>    
>    Here is the proposed approach: if CONFIG_RCU_FORCE_SYS_MEMBARRIER is
>    defined then rcu_has_sys_membarrier_memb/urcu_bp_has_sys_membarrier are
>    replaced with the constant 1, eliminating the overhead in smp_mb_slave.
>    As a sanity check, support for sys-membarrier is still detected at
>    startup. A program using liburcu or liburcu-bp compiled with this option
>    aborts in the library constructor if the membarrier system call is not
>    supported by the operating system.
>    
>    Suggested-by: Duncan Sands <duncan.sands@deepbluecap.com>
>    Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> 

Along with the following fixup:

commit a8e7c8d9eabfecc5017053754f6b446a95b00010
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date:   Fri Sep 16 16:11:46 2016 -0400

    Fix: add missing CONFIG_RCU_FORCE_SYS_MEMBARRIER to urcu/config.h.in
    
    Expose this configuration define in the installed header.
    
    Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Thanks,

Mathieu

> Thanks !!
> 
> Mathieu
> 
> 
>> 
>> Ciao, Duncan.
>> 
>>>
>>> Thanks,
>>>
>>> Mathieu
>>>
>>>>
>>>> Best wishes, Duncan.
>>>>
>>>> _______________________________________________
>>>> lttng-dev mailing list
>>>> lttng-dev@lists.lttng.org
>>>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [PATCH] [URCU] Allow forcing the use of sys membarrier

[Mathieu Desnoyers]

----- On Sep 16, 2016, at 11:38 AM, Duncan Sands duncan.sands@deepbluecap.com wrote:

> Hi Mathieu,
> 
> On 09/12/2016 05:05 PM, Mathieu Desnoyers wrote:
>> ----- On Sep 6, 2016, at 9:07 AM, Duncan Sands duncan.sands@deepbluecap.com
>> wrote:
>>
>>> When using the RCU_MEMBARRIER flavour of userspace-RCU, kernel support for
>>> sys-membarrier is detected dynamically and stored in the rcu_has_sys_membarrier
>>> global variable.  Checking the value of this variable adds a small but
>>> measurable overhead to smp_mb_slave.  I only use userspace-rcu on systems which
>>> support sys-membarrier, and I'd like a way of avoiding that overhead.
>>>
>>> The attached patch makes the following changes: if RCU_FORCE_SYS_MEMBARRIER is
>>> defined then rcu_has_sys_membarrier is replaced with the constant 1, eliminating
>>> the overhead in smp_mb_slave.  As a sanity check, support for sys-membarrier is
>>> still detected at startup and if it isn't supported then the program aborts.
>>>
>>> I didn't try to integrate this feature into the build system (eg by adding a
>>> configure option for it) since I'm using my own build system.
>>
>> Hi Duncan,
>>
>> This is an interesting improvement. I'd be interested to merge it if it was
>> integrated with the liburcu build system. Perhaps a
>> --disable-dynamic-membarrier-check
>> at configure ?
> 
> how about the attached patch?

I pushed the following commit into urcu master, inspired from your patch,

commit d8d9a3405ce46af6d34d2e80e260ad50f3d211a0
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date:   Fri Sep 16 15:15:41 2016 -0400

    Allow forcing the use of sys membarrier
    
    When using the default (liburcu.so) and bulletproof (liburcu-bp.so)
    flavours of Userspace RCU, kernel support for sys-membarrier is detected
    dynamically and stored in the rcu_has_sys_membarrier_memb and
    urcu_bp_has_sys_membarrier global variables.
    
    Checking the value of these variables adds a small but measurable overhead
    to smp_mb_slave. On systems which support sys-membarrier, it would be
    nice to have a way of avoiding that overhead.
    
    Here is the proposed approach: if CONFIG_RCU_FORCE_SYS_MEMBARRIER is
    defined then rcu_has_sys_membarrier_memb/urcu_bp_has_sys_membarrier are
    replaced with the constant 1, eliminating the overhead in smp_mb_slave.
    As a sanity check, support for sys-membarrier is still detected at
    startup. A program using liburcu or liburcu-bp compiled with this option
    aborts in the library constructor if the membarrier system call is not
    supported by the operating system.
    
    Suggested-by: Duncan Sands <duncan.sands@deepbluecap.com>
    Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Thanks !!

Mathieu


> 
> Ciao, Duncan.
> 
>>
>> Thanks,
>>
>> Mathieu
>>
>>>
>>> Best wishes, Duncan.
>>>
>>> _______________________________________________
>>> lttng-dev mailing list
>>> lttng-dev@lists.lttng.org
>>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [PATCH] [URCU] Allow forcing the use of sys membarrier

[Duncan Sands]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

Hi Mathieu,

On 09/12/2016 05:05 PM, Mathieu Desnoyers wrote:
> ----- On Sep 6, 2016, at 9:07 AM, Duncan Sands duncan.sands@deepbluecap.com wrote:
>
>> When using the RCU_MEMBARRIER flavour of userspace-RCU, kernel support for
>> sys-membarrier is detected dynamically and stored in the rcu_has_sys_membarrier
>> global variable.  Checking the value of this variable adds a small but
>> measurable overhead to smp_mb_slave.  I only use userspace-rcu on systems which
>> support sys-membarrier, and I'd like a way of avoiding that overhead.
>>
>> The attached patch makes the following changes: if RCU_FORCE_SYS_MEMBARRIER is
>> defined then rcu_has_sys_membarrier is replaced with the constant 1, eliminating
>> the overhead in smp_mb_slave.  As a sanity check, support for sys-membarrier is
>> still detected at startup and if it isn't supported then the program aborts.
>>
>> I didn't try to integrate this feature into the build system (eg by adding a
>> configure option for it) since I'm using my own build system.
>
> Hi Duncan,
>
> This is an interesting improvement. I'd be interested to merge it if it was
> integrated with the liburcu build system. Perhaps a --disable-dynamic-membarrier-check
> at configure ?

how about the attached patch?

Ciao, Duncan.

>
> Thanks,
>
> Mathieu
>
>>
>> Best wishes, Duncan.
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>


[-- Attachment #2: force.diff --]
[-- Type: text/x-patch, Size: 3719 bytes --]

diff --git a/README.md b/README.md
index 6fe9c1e..adc432a 100644
--- a/README.md
+++ b/README.md
@@ -159,7 +159,9 @@ This is the preferred version of the library, in terms of
 grace-period detection speed, read-side speed and flexibility.
 Dynamically detects kernel support for `sys_membarrier()`. Falls back
 on `urcu-mb` scheme if support is not present, which has slower
-read-side.
+read-side.  Use the --disable-sys-membarrier-fallback configure option
+to disable falling back.  This gives a small speedup when `sys_membarrier()`
+is supported by the kernel (and a crash when it isn't).
 
 
 ### Usage of `liburcu-qsbr`
diff --git a/configure.ac b/configure.ac
index 7a992ed..5709270 100644
--- a/configure.ac
+++ b/configure.ac
@@ -18,6 +18,7 @@ AM_MAINTAINER_MODE([enable])
 # Enable silent rules if available (Introduced in AM 1.11)
 m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
 
+AH_TEMPLATE([RCU_FORCE_SYS_MEMBARRIER], [Require the O/S to support sys-membarrier (when using the default flavor).])
 AH_TEMPLATE([CONFIG_RCU_SMP], [Enable SMP support. With SMP support enabled, uniprocessors are also supported. With SMP support disabled, UP systems work fine, but the behavior of SMP systems is undefined.])
 AH_TEMPLATE([CONFIG_RCU_HAVE_FENCE], [Defined when on a system that has memory fence instructions.])
 AH_TEMPLATE([CONFIG_RCU_HAVE_FUTEX], [Defined when on a system with futex support.])
@@ -26,6 +27,13 @@ AH_TEMPLATE([CONFIG_RCU_ARM_HAVE_DMB], [Use the dmb instruction if available for
 AH_TEMPLATE([CONFIG_RCU_TLS], [TLS provided by the compiler.])
 AH_TEMPLATE([CONFIG_RCU_HAVE_CLOCK_GETTIME], [clock_gettime() is detected.])
 
+# Allow requiring the O/S to support sys-membarrier (when using the default flavor).
+AC_ARG_ENABLE([sys-membarrier-fallback],
+	AS_HELP_STRING([--disable-sys-membarrier-fallback], [Abort if sys-membarrier is needed but not available rather than using a slow alternative.]),
+	[def_sys_membarrier_fallback=$enableval],
+	[def_sys_membarrier_fallback="yes"])
+AS_IF([test "x$def_sys_membarrier_fallback" != "xyes"], [AC_DEFINE([RCU_FORCE_SYS_MEMBARRIER], [1])])
+
 # Allow overriding storage used for TLS variables.
 AC_ARG_ENABLE([compiler-tls],
 	AS_HELP_STRING([--disable-compiler-tls], [Use pthread_getspecific() to emulate Thread Local Storage (TLS) variables.]),
diff --git a/include/urcu/map/urcu.h b/include/urcu/map/urcu.h
index 9a4bb1a..449513e 100644
--- a/include/urcu/map/urcu.h
+++ b/include/urcu/map/urcu.h
@@ -80,9 +80,6 @@
 
 #define rcu_flavor			rcu_flavor_memb
 
-/* Specific to MEMBARRIER flavor */
-#define rcu_has_sys_membarrier		rcu_has_sys_membarrier_memb
-
 #elif defined(RCU_SIGNAL)
 
 #define rcu_read_lock			rcu_read_lock_sig
diff --git a/include/urcu/static/urcu.h b/include/urcu/static/urcu.h
index 7048f99..5fe1ba5 100644
--- a/include/urcu/static/urcu.h
+++ b/include/urcu/static/urcu.h
@@ -89,7 +89,11 @@ enum rcu_state {
  */
 
 #ifdef RCU_MEMBARRIER
+#ifdef RCU_FORCE_SYS_MEMBARRIER
+#define rcu_has_sys_membarrier 1
+#else
 extern int rcu_has_sys_membarrier;
+#endif
 
 static inline void smp_mb_slave(void)
 {
diff --git a/src/urcu.c b/src/urcu.c
index ccd9706..b2ac166 100644
--- a/src/urcu.c
+++ b/src/urcu.c
@@ -77,7 +77,9 @@ enum membarrier_cmd {
 
 #ifdef RCU_MEMBARRIER
 static int init_done;
+#ifndef RCU_FORCE_SYS_MEMBARRIER
 int rcu_has_sys_membarrier;
+#endif
 
 void __attribute__((constructor)) rcu_init(void);
 #endif
@@ -541,7 +543,12 @@ void rcu_init(void)
 	init_done = 1;
 	ret = membarrier(MEMBARRIER_CMD_QUERY, 0);
 	if (ret >= 0 && (ret & MEMBARRIER_CMD_SHARED)) {
+#ifndef RCU_FORCE_SYS_MEMBARRIER
 		rcu_has_sys_membarrier = 1;
+#else
+       } else {
+               abort();
+#endif
 	}
 }
 #endif

[-- Attachment #3: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [PATCH] [URCU] Allow forcing the use of sys membarrier

[Mathieu Desnoyers]

----- On Sep 6, 2016, at 9:07 AM, Duncan Sands duncan.sands@deepbluecap.com wrote:

> When using the RCU_MEMBARRIER flavour of userspace-RCU, kernel support for
> sys-membarrier is detected dynamically and stored in the rcu_has_sys_membarrier
> global variable.  Checking the value of this variable adds a small but
> measurable overhead to smp_mb_slave.  I only use userspace-rcu on systems which
> support sys-membarrier, and I'd like a way of avoiding that overhead.
> 
> The attached patch makes the following changes: if RCU_FORCE_SYS_MEMBARRIER is
> defined then rcu_has_sys_membarrier is replaced with the constant 1, eliminating
> the overhead in smp_mb_slave.  As a sanity check, support for sys-membarrier is
> still detected at startup and if it isn't supported then the program aborts.
> 
> I didn't try to integrate this feature into the build system (eg by adding a
> configure option for it) since I'm using my own build system.

Hi Duncan,

This is an interesting improvement. I'd be interested to merge it if it was
integrated with the liburcu build system. Perhaps a --disable-dynamic-membarrier-check
at configure ?

Thanks,

Mathieu

> 
> Best wishes, Duncan.
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev