* [PATCH] Fix: disable sys_membarrier when nohz_full is enabled @ 2016-11-03 16:29 Mathieu Desnoyers 2016-11-03 17:49 ` Paul E. McKenney 2016-11-17 6:51 ` Lai Jiangshan 0 siblings, 2 replies; 18+ messages in thread From: Mathieu Desnoyers @ 2016-11-03 16:29 UTC (permalink / raw) To: Paul E. McKenney Cc: linux-kernel, Mathieu Desnoyers, Josh Triplett, Steven Rostedt, Lai Jiangshan, stable Userspace applications should be allowed to expect the membarrier system call with MEMBARRIER_CMD_SHARED command to issue memory barriers on nohz_full CPUs, but synchronize_sched() does not take those into account. Given that we do not want unrelated processes to be able to affect real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier is invoked on a kernel with enabled nohz_full CPUs. Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> CC: Josh Triplett <josh@joshtriplett.org> CC: Steven Rostedt <rostedt@goodmis.org> CC: Lai Jiangshan <jiangshanlai@gmail.com> CC: <stable@vger.kernel.org> [3.10+] --- kernel/membarrier.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/membarrier.c b/kernel/membarrier.c index 536c727..9f9284f 100644 --- a/kernel/membarrier.c +++ b/kernel/membarrier.c @@ -16,6 +16,7 @@ #include <linux/syscalls.h> #include <linux/membarrier.h> +#include <linux/tick.h> /* * Bitmask made from a "or" of all commands within enum membarrier_cmd, @@ -51,6 +52,9 @@ */ SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) { + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ + if (tick_nohz_full_enabled()) + return -ENOSYS; if (unlikely(flags)) return -EINVAL; switch (cmd) { -- 2.1.4 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-03 16:29 [PATCH] Fix: disable sys_membarrier when nohz_full is enabled Mathieu Desnoyers @ 2016-11-03 17:49 ` Paul E. McKenney 2016-11-07 17:08 ` Mathieu Desnoyers 2016-11-17 6:51 ` Lai Jiangshan 1 sibling, 1 reply; 18+ messages in thread From: Paul E. McKenney @ 2016-11-03 17:49 UTC (permalink / raw) To: Mathieu Desnoyers Cc: linux-kernel, Josh Triplett, Steven Rostedt, Lai Jiangshan, stable On Thu, Nov 03, 2016 at 10:29:28AM -0600, Mathieu Desnoyers wrote: > Userspace applications should be allowed to expect the membarrier system > call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > nohz_full CPUs, but synchronize_sched() does not take those into > account. > > Given that we do not want unrelated processes to be able to affect > real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > is invoked on a kernel with enabled nohz_full CPUs. > > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> Acked-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > CC: Josh Triplett <josh@joshtriplett.org> > CC: Steven Rostedt <rostedt@goodmis.org> > CC: Lai Jiangshan <jiangshanlai@gmail.com> > CC: <stable@vger.kernel.org> [3.10+] > --- > kernel/membarrier.c | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/kernel/membarrier.c b/kernel/membarrier.c > index 536c727..9f9284f 100644 > --- a/kernel/membarrier.c > +++ b/kernel/membarrier.c > @@ -16,6 +16,7 @@ > > #include <linux/syscalls.h> > #include <linux/membarrier.h> > +#include <linux/tick.h> > > /* > * Bitmask made from a "or" of all commands within enum membarrier_cmd, > @@ -51,6 +52,9 @@ > */ > SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > { > + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > + if (tick_nohz_full_enabled()) > + return -ENOSYS; > if (unlikely(flags)) > return -EINVAL; > switch (cmd) { > -- > 2.1.4 > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-03 17:49 ` Paul E. McKenney @ 2016-11-07 17:08 ` Mathieu Desnoyers 2016-11-07 18:06 ` Paul E. McKenney 0 siblings, 1 reply; 18+ messages in thread From: Mathieu Desnoyers @ 2016-11-07 17:08 UTC (permalink / raw) To: Paul E. McKenney Cc: linux-kernel, Josh Triplett, rostedt, Lai Jiangshan, stable ----- On Nov 3, 2016, at 1:49 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > On Thu, Nov 03, 2016 at 10:29:28AM -0600, Mathieu Desnoyers wrote: >> Userspace applications should be allowed to expect the membarrier system >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on >> nohz_full CPUs, but synchronize_sched() does not take those into >> account. >> >> Given that we do not want unrelated processes to be able to affect >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier >> is invoked on a kernel with enabled nohz_full CPUs. >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > > Acked-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> Hi Paul, Do you plan to pick it up through your tree, or I should sent it directly to Linus ? Thanks, Mathieu > >> CC: Josh Triplett <josh@joshtriplett.org> >> CC: Steven Rostedt <rostedt@goodmis.org> >> CC: Lai Jiangshan <jiangshanlai@gmail.com> >> CC: <stable@vger.kernel.org> [3.10+] >> --- >> kernel/membarrier.c | 4 ++++ >> 1 file changed, 4 insertions(+) >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c >> index 536c727..9f9284f 100644 >> --- a/kernel/membarrier.c >> +++ b/kernel/membarrier.c >> @@ -16,6 +16,7 @@ >> >> #include <linux/syscalls.h> >> #include <linux/membarrier.h> >> +#include <linux/tick.h> >> >> /* >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, >> @@ -51,6 +52,9 @@ >> */ >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) >> { >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ >> + if (tick_nohz_full_enabled()) >> + return -ENOSYS; >> if (unlikely(flags)) >> return -EINVAL; >> switch (cmd) { >> -- >> 2.1.4 -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-07 17:08 ` Mathieu Desnoyers @ 2016-11-07 18:06 ` Paul E. McKenney 2016-11-07 18:10 ` Mathieu Desnoyers 0 siblings, 1 reply; 18+ messages in thread From: Paul E. McKenney @ 2016-11-07 18:06 UTC (permalink / raw) To: Mathieu Desnoyers Cc: linux-kernel, Josh Triplett, rostedt, Lai Jiangshan, stable On Mon, Nov 07, 2016 at 05:08:59PM +0000, Mathieu Desnoyers wrote: > ----- On Nov 3, 2016, at 1:49 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > > > On Thu, Nov 03, 2016 at 10:29:28AM -0600, Mathieu Desnoyers wrote: > >> Userspace applications should be allowed to expect the membarrier system > >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > >> nohz_full CPUs, but synchronize_sched() does not take those into > >> account. > >> > >> Given that we do not want unrelated processes to be able to affect > >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > >> is invoked on a kernel with enabled nohz_full CPUs. > >> > >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > > > > Acked-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > > Hi Paul, > > Do you plan to pick it up through your tree, or I should sent > it directly to Linus ? Your choice. I believe that the original went some other way, but I would be fine carrying this one. Thanx, Paul > Thanks, > > Mathieu > > > > >> CC: Josh Triplett <josh@joshtriplett.org> > >> CC: Steven Rostedt <rostedt@goodmis.org> > >> CC: Lai Jiangshan <jiangshanlai@gmail.com> > >> CC: <stable@vger.kernel.org> [3.10+] > >> --- > >> kernel/membarrier.c | 4 ++++ > >> 1 file changed, 4 insertions(+) > >> > >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c > >> index 536c727..9f9284f 100644 > >> --- a/kernel/membarrier.c > >> +++ b/kernel/membarrier.c > >> @@ -16,6 +16,7 @@ > >> > >> #include <linux/syscalls.h> > >> #include <linux/membarrier.h> > >> +#include <linux/tick.h> > >> > >> /* > >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, > >> @@ -51,6 +52,9 @@ > >> */ > >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > >> { > >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > >> + if (tick_nohz_full_enabled()) > >> + return -ENOSYS; > >> if (unlikely(flags)) > >> return -EINVAL; > >> switch (cmd) { > >> -- > >> 2.1.4 > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-07 18:06 ` Paul E. McKenney @ 2016-11-07 18:10 ` Mathieu Desnoyers 2016-11-07 20:03 ` Paul E. McKenney 0 siblings, 1 reply; 18+ messages in thread From: Mathieu Desnoyers @ 2016-11-07 18:10 UTC (permalink / raw) To: Paul E. McKenney Cc: linux-kernel, Josh Triplett, rostedt, Lai Jiangshan, stable ----- On Nov 7, 2016, at 1:06 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > On Mon, Nov 07, 2016 at 05:08:59PM +0000, Mathieu Desnoyers wrote: >> ----- On Nov 3, 2016, at 1:49 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com >> wrote: >> >> > On Thu, Nov 03, 2016 at 10:29:28AM -0600, Mathieu Desnoyers wrote: >> >> Userspace applications should be allowed to expect the membarrier system >> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on >> >> nohz_full CPUs, but synchronize_sched() does not take those into >> >> account. >> >> >> >> Given that we do not want unrelated processes to be able to affect >> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier >> >> is invoked on a kernel with enabled nohz_full CPUs. >> >> >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> >> >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> >> > >> > Acked-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> >> >> Hi Paul, >> >> Do you plan to pick it up through your tree, or I should sent >> it directly to Linus ? > > Your choice. I believe that the original went some other way, but I > would be fine carrying this one. Not sure what you mean by "the original" ? And which other way ? I have not been notified about this. If you can carry this patch it would be very much appreciated, Thanks, Mathieu > > Thanx, Paul > >> Thanks, >> >> Mathieu >> >> > >> >> CC: Josh Triplett <josh@joshtriplett.org> >> >> CC: Steven Rostedt <rostedt@goodmis.org> >> >> CC: Lai Jiangshan <jiangshanlai@gmail.com> >> >> CC: <stable@vger.kernel.org> [3.10+] >> >> --- >> >> kernel/membarrier.c | 4 ++++ >> >> 1 file changed, 4 insertions(+) >> >> >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c >> >> index 536c727..9f9284f 100644 >> >> --- a/kernel/membarrier.c >> >> +++ b/kernel/membarrier.c >> >> @@ -16,6 +16,7 @@ >> >> >> >> #include <linux/syscalls.h> >> >> #include <linux/membarrier.h> >> >> +#include <linux/tick.h> >> >> >> >> /* >> >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, >> >> @@ -51,6 +52,9 @@ >> >> */ >> >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) >> >> { >> >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ >> >> + if (tick_nohz_full_enabled()) >> >> + return -ENOSYS; >> >> if (unlikely(flags)) >> >> return -EINVAL; >> >> switch (cmd) { >> >> -- >> >> 2.1.4 >> >> -- >> Mathieu Desnoyers >> EfficiOS Inc. >> http://www.efficios.com -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-07 18:10 ` Mathieu Desnoyers @ 2016-11-07 20:03 ` Paul E. McKenney 2016-11-08 11:15 ` Mathieu Desnoyers 0 siblings, 1 reply; 18+ messages in thread From: Paul E. McKenney @ 2016-11-07 20:03 UTC (permalink / raw) To: Mathieu Desnoyers Cc: linux-kernel, Josh Triplett, rostedt, Lai Jiangshan, stable On Mon, Nov 07, 2016 at 06:10:14PM +0000, Mathieu Desnoyers wrote: > > ----- On Nov 7, 2016, at 1:06 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > > > On Mon, Nov 07, 2016 at 05:08:59PM +0000, Mathieu Desnoyers wrote: > >> ----- On Nov 3, 2016, at 1:49 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com > >> wrote: > >> > >> > On Thu, Nov 03, 2016 at 10:29:28AM -0600, Mathieu Desnoyers wrote: > >> >> Userspace applications should be allowed to expect the membarrier system > >> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > >> >> nohz_full CPUs, but synchronize_sched() does not take those into > >> >> account. > >> >> > >> >> Given that we do not want unrelated processes to be able to affect > >> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > >> >> is invoked on a kernel with enabled nohz_full CPUs. > >> >> > >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > >> >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > >> > > >> > Acked-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > >> > >> Hi Paul, > >> > >> Do you plan to pick it up through your tree, or I should sent > >> it directly to Linus ? > > > > Your choice. I believe that the original went some other way, but I > > would be fine carrying this one. > > Not sure what you mean by "the original" ? And which other way ? > I have not been notified about this. If I remember correctly, you sent the original sys_membarrier() patch through akpm or similar. > If you can carry this patch it would be very much appreciated, Will do! Thanx, Paul > Thanks, > > Mathieu > > > > > Thanx, Paul > > > >> Thanks, > >> > >> Mathieu > >> > >> > > >> >> CC: Josh Triplett <josh@joshtriplett.org> > >> >> CC: Steven Rostedt <rostedt@goodmis.org> > >> >> CC: Lai Jiangshan <jiangshanlai@gmail.com> > >> >> CC: <stable@vger.kernel.org> [3.10+] > >> >> --- > >> >> kernel/membarrier.c | 4 ++++ > >> >> 1 file changed, 4 insertions(+) > >> >> > >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c > >> >> index 536c727..9f9284f 100644 > >> >> --- a/kernel/membarrier.c > >> >> +++ b/kernel/membarrier.c > >> >> @@ -16,6 +16,7 @@ > >> >> > >> >> #include <linux/syscalls.h> > >> >> #include <linux/membarrier.h> > >> >> +#include <linux/tick.h> > >> >> > >> >> /* > >> >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, > >> >> @@ -51,6 +52,9 @@ > >> >> */ > >> >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > >> >> { > >> >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > >> >> + if (tick_nohz_full_enabled()) > >> >> + return -ENOSYS; > >> >> if (unlikely(flags)) > >> >> return -EINVAL; > >> >> switch (cmd) { > >> >> -- > >> >> 2.1.4 > >> > >> -- > >> Mathieu Desnoyers > >> EfficiOS Inc. > >> http://www.efficios.com > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-07 20:03 ` Paul E. McKenney @ 2016-11-08 11:15 ` Mathieu Desnoyers 2016-11-12 17:58 ` Paul E. McKenney 0 siblings, 1 reply; 18+ messages in thread From: Mathieu Desnoyers @ 2016-11-08 11:15 UTC (permalink / raw) To: Paul E. McKenney Cc: linux-kernel, Josh Triplett, rostedt, Lai Jiangshan, stable ----- On Nov 7, 2016, at 3:03 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > On Mon, Nov 07, 2016 at 06:10:14PM +0000, Mathieu Desnoyers wrote: >> >> ----- On Nov 7, 2016, at 1:06 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com >> wrote: >> >> > On Mon, Nov 07, 2016 at 05:08:59PM +0000, Mathieu Desnoyers wrote: >> >> ----- On Nov 3, 2016, at 1:49 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com >> >> wrote: >> >> >> >> > On Thu, Nov 03, 2016 at 10:29:28AM -0600, Mathieu Desnoyers wrote: >> >> >> Userspace applications should be allowed to expect the membarrier system >> >> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on >> >> >> nohz_full CPUs, but synchronize_sched() does not take those into >> >> >> account. >> >> >> >> >> >> Given that we do not want unrelated processes to be able to affect >> >> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier >> >> >> is invoked on a kernel with enabled nohz_full CPUs. >> >> >> >> >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> >> >> >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> >> >> > >> >> > Acked-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> >> >> >> >> Hi Paul, >> >> >> >> Do you plan to pick it up through your tree, or I should sent >> >> it directly to Linus ? >> > >> > Your choice. I believe that the original went some other way, but I >> > would be fine carrying this one. >> >> Not sure what you mean by "the original" ? And which other way ? >> I have not been notified about this. > > If I remember correctly, you sent the original sys_membarrier() > patch through akpm or similar. Ah right, the original implementation, yes. > >> If you can carry this patch it would be very much appreciated, > > Will do! Especially since the regression is somewhat related to RCU behavior wrt nohz_full, getting it through your tree seems relevant. Thanks! Mathieu > > Thanx, Paul > >> Thanks, >> >> Mathieu >> >> > >> > Thanx, Paul >> > >> >> Thanks, >> >> >> >> Mathieu >> >> >> >> > >> >> >> CC: Josh Triplett <josh@joshtriplett.org> >> >> >> CC: Steven Rostedt <rostedt@goodmis.org> >> >> >> CC: Lai Jiangshan <jiangshanlai@gmail.com> >> >> >> CC: <stable@vger.kernel.org> [3.10+] >> >> >> --- >> >> >> kernel/membarrier.c | 4 ++++ >> >> >> 1 file changed, 4 insertions(+) >> >> >> >> >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c >> >> >> index 536c727..9f9284f 100644 >> >> >> --- a/kernel/membarrier.c >> >> >> +++ b/kernel/membarrier.c >> >> >> @@ -16,6 +16,7 @@ >> >> >> >> >> >> #include <linux/syscalls.h> >> >> >> #include <linux/membarrier.h> >> >> >> +#include <linux/tick.h> >> >> >> >> >> >> /* >> >> >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, >> >> >> @@ -51,6 +52,9 @@ >> >> >> */ >> >> >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) >> >> >> { >> >> >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ >> >> >> + if (tick_nohz_full_enabled()) >> >> >> + return -ENOSYS; >> >> >> if (unlikely(flags)) >> >> >> return -EINVAL; >> >> >> switch (cmd) { >> >> >> -- >> >> >> 2.1.4 >> >> >> >> -- >> >> Mathieu Desnoyers >> >> EfficiOS Inc. >> >> http://www.efficios.com >> >> -- >> Mathieu Desnoyers >> EfficiOS Inc. >> http://www.efficios.com -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-08 11:15 ` Mathieu Desnoyers @ 2016-11-12 17:58 ` Paul E. McKenney 0 siblings, 0 replies; 18+ messages in thread From: Paul E. McKenney @ 2016-11-12 17:58 UTC (permalink / raw) To: Mathieu Desnoyers Cc: linux-kernel, Josh Triplett, rostedt, Lai Jiangshan, stable On Tue, Nov 08, 2016 at 11:15:51AM +0000, Mathieu Desnoyers wrote: > ----- On Nov 7, 2016, at 3:03 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > > > On Mon, Nov 07, 2016 at 06:10:14PM +0000, Mathieu Desnoyers wrote: > >> > >> ----- On Nov 7, 2016, at 1:06 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com > >> wrote: > >> > >> > On Mon, Nov 07, 2016 at 05:08:59PM +0000, Mathieu Desnoyers wrote: > >> >> ----- On Nov 3, 2016, at 1:49 PM, Paul E. McKenney paulmck@linux.vnet.ibm.com > >> >> wrote: > >> >> > >> >> > On Thu, Nov 03, 2016 at 10:29:28AM -0600, Mathieu Desnoyers wrote: > >> >> >> Userspace applications should be allowed to expect the membarrier system > >> >> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > >> >> >> nohz_full CPUs, but synchronize_sched() does not take those into > >> >> >> account. > >> >> >> > >> >> >> Given that we do not want unrelated processes to be able to affect > >> >> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > >> >> >> is invoked on a kernel with enabled nohz_full CPUs. > >> >> >> > >> >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > >> >> >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > >> >> > > >> >> > Acked-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > >> >> > >> >> Hi Paul, > >> >> > >> >> Do you plan to pick it up through your tree, or I should sent > >> >> it directly to Linus ? > >> > > >> > Your choice. I believe that the original went some other way, but I > >> > would be fine carrying this one. > >> > >> Not sure what you mean by "the original" ? And which other way ? > >> I have not been notified about this. > > > > If I remember correctly, you sent the original sys_membarrier() > > patch through akpm or similar. > > Ah right, the original implementation, yes. > > > > >> If you can carry this patch it would be very much appreciated, > > > > Will do! > > Especially since the regression is somewhat related to RCU > behavior wrt nohz_full, getting it through your tree seems > relevant. I have it queued for 4.11, thank you! Thanx, Paul ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-03 16:29 [PATCH] Fix: disable sys_membarrier when nohz_full is enabled Mathieu Desnoyers 2016-11-03 17:49 ` Paul E. McKenney @ 2016-11-17 6:51 ` Lai Jiangshan 2016-11-17 11:46 ` Mathieu Desnoyers 2016-11-17 13:38 ` Paul E. McKenney 1 sibling, 2 replies; 18+ messages in thread From: Lai Jiangshan @ 2016-11-17 6:51 UTC (permalink / raw) To: Mathieu Desnoyers Cc: Paul E. McKenney, LKML, Josh Triplett, Steven Rostedt, stable On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote: > Userspace applications should be allowed to expect the membarrier system > call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > nohz_full CPUs, but synchronize_sched() does not take those into > account. > > Given that we do not want unrelated processes to be able to affect > real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > is invoked on a kernel with enabled nohz_full CPUs. > > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > CC: Josh Triplett <josh@joshtriplett.org> > CC: Steven Rostedt <rostedt@goodmis.org> > CC: Lai Jiangshan <jiangshanlai@gmail.com> > CC: <stable@vger.kernel.org> [3.10+] > --- > kernel/membarrier.c | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/kernel/membarrier.c b/kernel/membarrier.c > index 536c727..9f9284f 100644 > --- a/kernel/membarrier.c > +++ b/kernel/membarrier.c > @@ -16,6 +16,7 @@ > > #include <linux/syscalls.h> > #include <linux/membarrier.h> > +#include <linux/tick.h> > > /* > * Bitmask made from a "or" of all commands within enum membarrier_cmd, > @@ -51,6 +52,9 @@ > */ > SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > { > + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > + if (tick_nohz_full_enabled()) > + return -ENOSYS; I guess this code needs to be moved down into the branch of "case MEMBARRIER_CMD_SHARED" to match its comment. Acked-by: Lai Jiangshan <jiangshanlai@gmail.com> But I'm afraid, in the future, tick_nohz_full will become a default y feature. thus it makes sys_membarrier() always disabled. we might need a new MEMBARRIER_CMD_XXX to handle it? thanks, Lai ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 6:51 ` Lai Jiangshan @ 2016-11-17 11:46 ` Mathieu Desnoyers 2016-11-17 13:40 ` Paul E. McKenney 2016-11-17 13:38 ` Paul E. McKenney 1 sibling, 1 reply; 18+ messages in thread From: Mathieu Desnoyers @ 2016-11-17 11:46 UTC (permalink / raw) To: Lai Jiangshan Cc: Paul E. McKenney, linux-kernel, Josh Triplett, rostedt, stable ----- On Nov 17, 2016, at 1:51 AM, Lai Jiangshan jiangshanlai@gmail.com wrote: > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers > <mathieu.desnoyers@efficios.com> wrote: >> Userspace applications should be allowed to expect the membarrier system >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on >> nohz_full CPUs, but synchronize_sched() does not take those into >> account. >> >> Given that we do not want unrelated processes to be able to affect >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier >> is invoked on a kernel with enabled nohz_full CPUs. >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> >> CC: Josh Triplett <josh@joshtriplett.org> >> CC: Steven Rostedt <rostedt@goodmis.org> >> CC: Lai Jiangshan <jiangshanlai@gmail.com> >> CC: <stable@vger.kernel.org> [3.10+] >> --- >> kernel/membarrier.c | 4 ++++ >> 1 file changed, 4 insertions(+) >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c >> index 536c727..9f9284f 100644 >> --- a/kernel/membarrier.c >> +++ b/kernel/membarrier.c >> @@ -16,6 +16,7 @@ >> >> #include <linux/syscalls.h> >> #include <linux/membarrier.h> >> +#include <linux/tick.h> >> >> /* >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, >> @@ -51,6 +52,9 @@ >> */ >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) >> { >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ >> + if (tick_nohz_full_enabled()) >> + return -ENOSYS; > > I guess this code needs to be moved down into the branch of > "case MEMBARRIER_CMD_SHARED" to match its comment. No, that would be unexpected from user-space. Either a system call is implemented or not, not "implemented for some parameters". We also want MEMBARRIER_CMD_QUERY to return -ENOSYS in this case, and all other parameter values to also return -ENOSYS (rather than -EINVAL). If a system call that returns successfully on CMD_QUERY or EINVAL, user-space may assume it will not have to handle ENOSYS in the next calls. > > Acked-by: Lai Jiangshan <jiangshanlai@gmail.com> > > But I'm afraid, in the future, tick_nohz_full will become a default y > feature. thus it makes sys_membarrier() always disabled. we might > need a new MEMBARRIER_CMD_XXX to handle it? This may require that we send an IPI to nohz_full CPUs, which will disturb them real-time wise. Any better ideas ? Thanks, Mathieu > > thanks, > Lai -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 11:46 ` Mathieu Desnoyers @ 2016-11-17 13:40 ` Paul E. McKenney 2016-11-17 13:54 ` Mathieu Desnoyers 0 siblings, 1 reply; 18+ messages in thread From: Paul E. McKenney @ 2016-11-17 13:40 UTC (permalink / raw) To: Mathieu Desnoyers Cc: Lai Jiangshan, linux-kernel, Josh Triplett, rostedt, stable On Thu, Nov 17, 2016 at 11:46:34AM +0000, Mathieu Desnoyers wrote: > ----- On Nov 17, 2016, at 1:51 AM, Lai Jiangshan jiangshanlai@gmail.com wrote: > > > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers > > <mathieu.desnoyers@efficios.com> wrote: > >> Userspace applications should be allowed to expect the membarrier system > >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > >> nohz_full CPUs, but synchronize_sched() does not take those into > >> account. > >> > >> Given that we do not want unrelated processes to be able to affect > >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > >> is invoked on a kernel with enabled nohz_full CPUs. > >> > >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > >> CC: Josh Triplett <josh@joshtriplett.org> > >> CC: Steven Rostedt <rostedt@goodmis.org> > >> CC: Lai Jiangshan <jiangshanlai@gmail.com> > >> CC: <stable@vger.kernel.org> [3.10+] > >> --- > >> kernel/membarrier.c | 4 ++++ > >> 1 file changed, 4 insertions(+) > >> > >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c > >> index 536c727..9f9284f 100644 > >> --- a/kernel/membarrier.c > >> +++ b/kernel/membarrier.c > >> @@ -16,6 +16,7 @@ > >> > >> #include <linux/syscalls.h> > >> #include <linux/membarrier.h> > >> +#include <linux/tick.h> > >> > >> /* > >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, > >> @@ -51,6 +52,9 @@ > >> */ > >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > >> { > >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > >> + if (tick_nohz_full_enabled()) > >> + return -ENOSYS; > > > > I guess this code needs to be moved down into the branch of > > "case MEMBARRIER_CMD_SHARED" to match its comment. > > No, that would be unexpected from user-space. Either a system > call is implemented or not, not "implemented for some parameters". > > We also want MEMBARRIER_CMD_QUERY to return -ENOSYS in this case, > and all other parameter values to also return -ENOSYS (rather than > -EINVAL). > > If a system call that returns successfully on CMD_QUERY or EINVAL, > user-space may assume it will not have to handle ENOSYS in the > next calls. > > > > > > Acked-by: Lai Jiangshan <jiangshanlai@gmail.com> > > > > But I'm afraid, in the future, tick_nohz_full will become a default y > > feature. thus it makes sys_membarrier() always disabled. we might > > need a new MEMBARRIER_CMD_XXX to handle it? > > This may require that we send an IPI to nohz_full CPUs, which will > disturb them real-time wise. Any better ideas ? Restrict the IPIs to CPUs running the process executing the sys_membarrier() system call. This would mean that CPUs only are interrupted by their own application's request. Thanx, Paul ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 13:40 ` Paul E. McKenney @ 2016-11-17 13:54 ` Mathieu Desnoyers 2016-11-17 14:09 ` Paul E. McKenney 2016-11-17 14:50 ` Steven Rostedt 0 siblings, 2 replies; 18+ messages in thread From: Mathieu Desnoyers @ 2016-11-17 13:54 UTC (permalink / raw) To: Paul E. McKenney Cc: Lai Jiangshan, linux-kernel, Josh Triplett, rostedt, stable ----- On Nov 17, 2016, at 8:40 AM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > On Thu, Nov 17, 2016 at 11:46:34AM +0000, Mathieu Desnoyers wrote: >> ----- On Nov 17, 2016, at 1:51 AM, Lai Jiangshan jiangshanlai@gmail.com wrote: >> >> > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers >> > <mathieu.desnoyers@efficios.com> wrote: >> >> Userspace applications should be allowed to expect the membarrier system >> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on >> >> nohz_full CPUs, but synchronize_sched() does not take those into >> >> account. >> >> >> >> Given that we do not want unrelated processes to be able to affect >> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier >> >> is invoked on a kernel with enabled nohz_full CPUs. >> >> >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> >> >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> >> >> CC: Josh Triplett <josh@joshtriplett.org> >> >> CC: Steven Rostedt <rostedt@goodmis.org> >> >> CC: Lai Jiangshan <jiangshanlai@gmail.com> >> >> CC: <stable@vger.kernel.org> [3.10+] >> >> --- >> >> kernel/membarrier.c | 4 ++++ >> >> 1 file changed, 4 insertions(+) >> >> >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c >> >> index 536c727..9f9284f 100644 >> >> --- a/kernel/membarrier.c >> >> +++ b/kernel/membarrier.c >> >> @@ -16,6 +16,7 @@ >> >> >> >> #include <linux/syscalls.h> >> >> #include <linux/membarrier.h> >> >> +#include <linux/tick.h> >> >> >> >> /* >> >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, >> >> @@ -51,6 +52,9 @@ >> >> */ >> >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) >> >> { >> >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ >> >> + if (tick_nohz_full_enabled()) >> >> + return -ENOSYS; >> > >> > I guess this code needs to be moved down into the branch of >> > "case MEMBARRIER_CMD_SHARED" to match its comment. >> >> No, that would be unexpected from user-space. Either a system >> call is implemented or not, not "implemented for some parameters". >> >> We also want MEMBARRIER_CMD_QUERY to return -ENOSYS in this case, >> and all other parameter values to also return -ENOSYS (rather than >> -EINVAL). >> >> If a system call that returns successfully on CMD_QUERY or EINVAL, >> user-space may assume it will not have to handle ENOSYS in the >> next calls. >> >> >> > >> > Acked-by: Lai Jiangshan <jiangshanlai@gmail.com> >> > >> > But I'm afraid, in the future, tick_nohz_full will become a default y >> > feature. thus it makes sys_membarrier() always disabled. we might >> > need a new MEMBARRIER_CMD_XXX to handle it? >> >> This may require that we send an IPI to nohz_full CPUs, which will >> disturb them real-time wise. Any better ideas ? > > Restrict the IPIs to CPUs running the process executing the > sys_membarrier() system call. This would mean that CPUs only > are interrupted by their own application's request. This would break use-cases of cross-process shared memory. :-( Mathieu > > Thanx, Paul -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 13:54 ` Mathieu Desnoyers @ 2016-11-17 14:09 ` Paul E. McKenney 2016-11-17 14:50 ` Steven Rostedt 1 sibling, 0 replies; 18+ messages in thread From: Paul E. McKenney @ 2016-11-17 14:09 UTC (permalink / raw) To: Mathieu Desnoyers Cc: Lai Jiangshan, linux-kernel, Josh Triplett, rostedt, stable On Thu, Nov 17, 2016 at 01:54:27PM +0000, Mathieu Desnoyers wrote: > ----- On Nov 17, 2016, at 8:40 AM, Paul E. McKenney paulmck@linux.vnet.ibm.com wrote: > > > On Thu, Nov 17, 2016 at 11:46:34AM +0000, Mathieu Desnoyers wrote: > >> ----- On Nov 17, 2016, at 1:51 AM, Lai Jiangshan jiangshanlai@gmail.com wrote: > >> > >> > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers > >> > <mathieu.desnoyers@efficios.com> wrote: > >> >> Userspace applications should be allowed to expect the membarrier system > >> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > >> >> nohz_full CPUs, but synchronize_sched() does not take those into > >> >> account. > >> >> > >> >> Given that we do not want unrelated processes to be able to affect > >> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > >> >> is invoked on a kernel with enabled nohz_full CPUs. > >> >> > >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > >> >> CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > >> >> CC: Josh Triplett <josh@joshtriplett.org> > >> >> CC: Steven Rostedt <rostedt@goodmis.org> > >> >> CC: Lai Jiangshan <jiangshanlai@gmail.com> > >> >> CC: <stable@vger.kernel.org> [3.10+] > >> >> --- > >> >> kernel/membarrier.c | 4 ++++ > >> >> 1 file changed, 4 insertions(+) > >> >> > >> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c > >> >> index 536c727..9f9284f 100644 > >> >> --- a/kernel/membarrier.c > >> >> +++ b/kernel/membarrier.c > >> >> @@ -16,6 +16,7 @@ > >> >> > >> >> #include <linux/syscalls.h> > >> >> #include <linux/membarrier.h> > >> >> +#include <linux/tick.h> > >> >> > >> >> /* > >> >> * Bitmask made from a "or" of all commands within enum membarrier_cmd, > >> >> @@ -51,6 +52,9 @@ > >> >> */ > >> >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > >> >> { > >> >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > >> >> + if (tick_nohz_full_enabled()) > >> >> + return -ENOSYS; > >> > > >> > I guess this code needs to be moved down into the branch of > >> > "case MEMBARRIER_CMD_SHARED" to match its comment. > >> > >> No, that would be unexpected from user-space. Either a system > >> call is implemented or not, not "implemented for some parameters". > >> > >> We also want MEMBARRIER_CMD_QUERY to return -ENOSYS in this case, > >> and all other parameter values to also return -ENOSYS (rather than > >> -EINVAL). > >> > >> If a system call that returns successfully on CMD_QUERY or EINVAL, > >> user-space may assume it will not have to handle ENOSYS in the > >> next calls. > >> > >> > >> > > >> > Acked-by: Lai Jiangshan <jiangshanlai@gmail.com> > >> > > >> > But I'm afraid, in the future, tick_nohz_full will become a default y > >> > feature. thus it makes sys_membarrier() always disabled. we might > >> > need a new MEMBARRIER_CMD_XXX to handle it? > >> > >> This may require that we send an IPI to nohz_full CPUs, which will > >> disturb them real-time wise. Any better ideas ? > > > > Restrict the IPIs to CPUs running the process executing the > > sys_membarrier() system call. This would mean that CPUs only > > are interrupted by their own application's request. > > This would break use-cases of cross-process shared memory. :-( Good point -- getting this working does look to be good clean fun... Thanx, Paul ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 13:54 ` Mathieu Desnoyers 2016-11-17 14:09 ` Paul E. McKenney @ 2016-11-17 14:50 ` Steven Rostedt 2016-11-17 15:02 ` Mathieu Desnoyers 1 sibling, 1 reply; 18+ messages in thread From: Steven Rostedt @ 2016-11-17 14:50 UTC (permalink / raw) To: Mathieu Desnoyers Cc: Paul E. McKenney, Lai Jiangshan, linux-kernel, Josh Triplett, stable On Thu, 17 Nov 2016 13:54:27 +0000 (UTC) Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote: > >> > > >> > Acked-by: Lai Jiangshan <jiangshanlai@gmail.com> > >> > > >> > But I'm afraid, in the future, tick_nohz_full will become a default y > >> > feature. thus it makes sys_membarrier() always disabled. we might > >> > need a new MEMBARRIER_CMD_XXX to handle it? > >> > >> This may require that we send an IPI to nohz_full CPUs, which will > >> disturb them real-time wise. Any better ideas ? > > > > Restrict the IPIs to CPUs running the process executing the > > sys_membarrier() system call. This would mean that CPUs only > > are interrupted by their own application's request. > > This would break use-cases of cross-process shared memory. :-( Perhaps make this an opt in. That is, all processes that want to be affected by this can call this function with some flag that sets a flag in tasks struct. And have that process get an IPI even in no-hz-full mode if it asked to do it. -- Steve ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 14:50 ` Steven Rostedt @ 2016-11-17 15:02 ` Mathieu Desnoyers 2016-11-17 15:17 ` Steven Rostedt 0 siblings, 1 reply; 18+ messages in thread From: Mathieu Desnoyers @ 2016-11-17 15:02 UTC (permalink / raw) To: rostedt Cc: Paul E. McKenney, Lai Jiangshan, linux-kernel, Josh Triplett, stable ----- On Nov 17, 2016, at 9:50 AM, rostedt rostedt@goodmis.org wrote: > On Thu, 17 Nov 2016 13:54:27 +0000 (UTC) > Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote: > > >> >> > >> >> > Acked-by: Lai Jiangshan <jiangshanlai@gmail.com> >> >> > >> >> > But I'm afraid, in the future, tick_nohz_full will become a default y >> >> > feature. thus it makes sys_membarrier() always disabled. we might >> >> > need a new MEMBARRIER_CMD_XXX to handle it? >> >> >> >> This may require that we send an IPI to nohz_full CPUs, which will >> >> disturb them real-time wise. Any better ideas ? >> > >> > Restrict the IPIs to CPUs running the process executing the >> > sys_membarrier() system call. This would mean that CPUs only >> > are interrupted by their own application's request. >> >> This would break use-cases of cross-process shared memory. :-( > > Perhaps make this an opt in. That is, all processes that want to be > affected by this can call this function with some flag that sets a flag > in tasks struct. And have that process get an IPI even in no-hz-full > mode if it asked to do it. That's an interesting approach. I would be tempted to give it a per-thread (rather than per-process) scope. E.g., a thread could do the following to ask to be interrupted by IPIs: membarrier(MEMBARRIER_CMD_REGISTER_EXPEDITED, 0) and could unregister with: membarrier(MEMBARRIER_CMD_UNREGISTER_EXPEDITED, 0) We can then keep a per-thread refcount internally. (not sure the "EXPEDITED" is the right word there... do we want it to be "NOHZ_FULL" instead ?) Then in membarrier(MEMBARRIER_CMD_SHARED, 0), for each nohz_full cpu, we grab the rq lock, and only send an IPI if the running thread is registered as "expedited". Thoughts ? Thanks, Mathieu -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 15:02 ` Mathieu Desnoyers @ 2016-11-17 15:17 ` Steven Rostedt 2016-11-17 16:02 ` Paul E. McKenney 0 siblings, 1 reply; 18+ messages in thread From: Steven Rostedt @ 2016-11-17 15:17 UTC (permalink / raw) To: Mathieu Desnoyers Cc: Paul E. McKenney, Lai Jiangshan, linux-kernel, Josh Triplett, stable On Thu, 17 Nov 2016 15:02:18 +0000 (UTC) Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote: > That's an interesting approach. I would be tempted to give it a > per-thread (rather than per-process) scope. Sure, per thread, but have it inherit to child processes. > > E.g., a thread could do the following to ask to be > interrupted by IPIs: > > membarrier(MEMBARRIER_CMD_REGISTER_EXPEDITED, 0) > > and could unregister with: > > membarrier(MEMBARRIER_CMD_UNREGISTER_EXPEDITED, 0) Sure why not ;-) > > We can then keep a per-thread refcount internally. > (not sure the "EXPEDITED" is the right word there... > do we want it to be "NOHZ_FULL" instead ?) No, it shouldn't mention NOHZ_FULL. Perhaps have all tasks do this regardless, even though it will only affect nohz full ones. But in the future it may be other tasks as well. > > Then in membarrier(MEMBARRIER_CMD_SHARED, 0), for each > nohz_full cpu, we grab the rq lock, and only send an IPI > if the running thread is registered as "expedited". Yeah, something like that. That way it wont interrupt tasks that are running in no-hz-full and don't care about this syscall. -- Steve ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 15:17 ` Steven Rostedt @ 2016-11-17 16:02 ` Paul E. McKenney 0 siblings, 0 replies; 18+ messages in thread From: Paul E. McKenney @ 2016-11-17 16:02 UTC (permalink / raw) To: Steven Rostedt Cc: Mathieu Desnoyers, Lai Jiangshan, linux-kernel, Josh Triplett, stable On Thu, Nov 17, 2016 at 10:17:25AM -0500, Steven Rostedt wrote: > On Thu, 17 Nov 2016 15:02:18 +0000 (UTC) > Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote: > > > > That's an interesting approach. I would be tempted to give it a > > per-thread (rather than per-process) scope. > > Sure, per thread, but have it inherit to child processes. > > > > > E.g., a thread could do the following to ask to be > > interrupted by IPIs: > > > > membarrier(MEMBARRIER_CMD_REGISTER_EXPEDITED, 0) > > > > and could unregister with: > > > > membarrier(MEMBARRIER_CMD_UNREGISTER_EXPEDITED, 0) > > Sure why not ;-) Makes a lot of sense to me! > > We can then keep a per-thread refcount internally. > > (not sure the "EXPEDITED" is the right word there... > > do we want it to be "NOHZ_FULL" instead ?) > > No, it shouldn't mention NOHZ_FULL. Perhaps have all tasks do this > regardless, even though it will only affect nohz full ones. But in the > future it may be other tasks as well. > > > > > Then in membarrier(MEMBARRIER_CMD_SHARED, 0), for each > > nohz_full cpu, we grab the rq lock, and only send an IPI > > if the running thread is registered as "expedited". > > Yeah, something like that. That way it wont interrupt tasks that are > running in no-hz-full and don't care about this syscall. And this as well! Thanx, Paul ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] Fix: disable sys_membarrier when nohz_full is enabled 2016-11-17 6:51 ` Lai Jiangshan 2016-11-17 11:46 ` Mathieu Desnoyers @ 2016-11-17 13:38 ` Paul E. McKenney 1 sibling, 0 replies; 18+ messages in thread From: Paul E. McKenney @ 2016-11-17 13:38 UTC (permalink / raw) To: Lai Jiangshan Cc: Mathieu Desnoyers, LKML, Josh Triplett, Steven Rostedt, stable On Thu, Nov 17, 2016 at 02:51:26PM +0800, Lai Jiangshan wrote: > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers > <mathieu.desnoyers@efficios.com> wrote: > > Userspace applications should be allowed to expect the membarrier system > > call with MEMBARRIER_CMD_SHARED command to issue memory barriers on > > nohz_full CPUs, but synchronize_sched() does not take those into > > account. > > > > Given that we do not want unrelated processes to be able to affect > > real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier > > is invoked on a kernel with enabled nohz_full CPUs. > > > > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > > CC: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> > > CC: Josh Triplett <josh@joshtriplett.org> > > CC: Steven Rostedt <rostedt@goodmis.org> > > CC: Lai Jiangshan <jiangshanlai@gmail.com> > > CC: <stable@vger.kernel.org> [3.10+] > > --- > > kernel/membarrier.c | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/kernel/membarrier.c b/kernel/membarrier.c > > index 536c727..9f9284f 100644 > > --- a/kernel/membarrier.c > > +++ b/kernel/membarrier.c > > @@ -16,6 +16,7 @@ > > > > #include <linux/syscalls.h> > > #include <linux/membarrier.h> > > +#include <linux/tick.h> > > > > /* > > * Bitmask made from a "or" of all commands within enum membarrier_cmd, > > @@ -51,6 +52,9 @@ > > */ > > SYSCALL_DEFINE2(membarrier, int, cmd, int, flags) > > { > > + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */ > > + if (tick_nohz_full_enabled()) > > + return -ENOSYS; > > I guess this code needs to be moved down into the branch of > "case MEMBARRIER_CMD_SHARED" to match its comment. > > Acked-by: Lai Jiangshan <jiangshanlai@gmail.com> Added, thank you! > But I'm afraid, in the future, tick_nohz_full will become a default y > feature. thus it makes sys_membarrier() always disabled. we might > need a new MEMBARRIER_CMD_XXX to handle it? Makes a lot of sense to me! Thanx, Paul ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2016-11-17 19:27 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-11-03 16:29 [PATCH] Fix: disable sys_membarrier when nohz_full is enabled Mathieu Desnoyers 2016-11-03 17:49 ` Paul E. McKenney 2016-11-07 17:08 ` Mathieu Desnoyers 2016-11-07 18:06 ` Paul E. McKenney 2016-11-07 18:10 ` Mathieu Desnoyers 2016-11-07 20:03 ` Paul E. McKenney 2016-11-08 11:15 ` Mathieu Desnoyers 2016-11-12 17:58 ` Paul E. McKenney 2016-11-17 6:51 ` Lai Jiangshan 2016-11-17 11:46 ` Mathieu Desnoyers 2016-11-17 13:40 ` Paul E. McKenney 2016-11-17 13:54 ` Mathieu Desnoyers 2016-11-17 14:09 ` Paul E. McKenney 2016-11-17 14:50 ` Steven Rostedt 2016-11-17 15:02 ` Mathieu Desnoyers 2016-11-17 15:17 ` Steven Rostedt 2016-11-17 16:02 ` Paul E. McKenney 2016-11-17 13:38 ` Paul E. McKenney
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).