* [Qestion] Is preempt_disable/enable needed in non-preemption code path @ 2021-04-15 15:04 Xu, Yanfei 2021-04-15 15:43 ` Paul E. McKenney 0 siblings, 1 reply; 9+ messages in thread From: Xu, Yanfei @ 2021-04-15 15:04 UTC (permalink / raw) To: paulmck; +Cc: rcu, LKML Hi experts, I am learning rcu mechanism and its codes. When looking at the rcu_blocking_is_gp(), I found there is a pair preemption disable/enable operation in non-preemption code path. And it has been a long time. I can't understand why we need it? Is there some thing I missed? If not, can we remove the unnecessary operation like blow? diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index da6f5213fb74..c6d95a00715e 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) if (IS_ENABLED(CONFIG_PREEMPTION)) return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; might_sleep(); /* Check for RCU read-side critical section. */ - preempt_disable(); /* * If the rcu_state.n_online_cpus counter is equal to one, * there is only one CPU, and that CPU sees all prior accesses @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) * Those memory barriers are provided by CPU-hotplug code. */ ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; - preempt_enable(); return ret; } Best regards, Yanfei ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [Qestion] Is preempt_disable/enable needed in non-preemption code path 2021-04-15 15:04 [Qestion] Is preempt_disable/enable needed in non-preemption code path Xu, Yanfei @ 2021-04-15 15:43 ` Paul E. McKenney 2021-04-15 16:18 ` Xu, Yanfei 0 siblings, 1 reply; 9+ messages in thread From: Paul E. McKenney @ 2021-04-15 15:43 UTC (permalink / raw) To: Xu, Yanfei; +Cc: rcu, LKML On Thu, Apr 15, 2021 at 11:04:05PM +0800, Xu, Yanfei wrote: > Hi experts, > > I am learning rcu mechanism and its codes. When looking at the > rcu_blocking_is_gp(), I found there is a pair preemption disable/enable > operation in non-preemption code path. And it has been a long time. I can't > understand why we need it? Is there some thing I missed? If not, can we > remove the unnecessary operation like blow? Good point, you are right that preemption is disabled anyway in that block of code. However, preempt_disable() and preempt_enable() also prevent the compiler from moving that READ_ONCE() around. So my question to you is whether it is safe to remove those statements entirely or whether they should instead be replaced by barrier() or similar. Thanx, Paul > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > index da6f5213fb74..c6d95a00715e 100644 > --- a/kernel/rcu/tree.c > +++ b/kernel/rcu/tree.c > @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) > if (IS_ENABLED(CONFIG_PREEMPTION)) > return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; > might_sleep(); /* Check for RCU read-side critical section. */ > - preempt_disable(); > /* > * If the rcu_state.n_online_cpus counter is equal to one, > * there is only one CPU, and that CPU sees all prior accesses > @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) > * Those memory barriers are provided by CPU-hotplug code. > */ > ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; > - preempt_enable(); > return ret; > } > > > > Best regards, > Yanfei ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qestion] Is preempt_disable/enable needed in non-preemption code path 2021-04-15 15:43 ` Paul E. McKenney @ 2021-04-15 16:18 ` Xu, Yanfei 2021-04-15 17:01 ` Xu, Yanfei 2021-04-15 17:07 ` Paul E. McKenney 0 siblings, 2 replies; 9+ messages in thread From: Xu, Yanfei @ 2021-04-15 16:18 UTC (permalink / raw) To: paulmck; +Cc: rcu, LKML On 4/15/21 11:43 PM, Paul E. McKenney wrote: > [Please note: This e-mail is from an EXTERNAL e-mail address] > > On Thu, Apr 15, 2021 at 11:04:05PM +0800, Xu, Yanfei wrote: >> Hi experts, >> >> I am learning rcu mechanism and its codes. When looking at the >> rcu_blocking_is_gp(), I found there is a pair preemption disable/enable >> operation in non-preemption code path. And it has been a long time. I can't >> understand why we need it? Is there some thing I missed? If not, can we >> remove the unnecessary operation like blow? > > Good point, you are right that preemption is disabled anyway in that block > of code. However, preempt_disable() and preempt_enable() also prevent the > compiler from moving that READ_ONCE() around. So my question to you is > whether it is safe to remove those statements entirely or whether they > should instead be replaced by barrier() or similar. Thanks for your reply! :) Yes, preempt_disable() and preempt_enable() defined in !preemption are barrier(). barrier can prevent from reordering that READ_ONCE(), but base on my current understanding, volatile in READ_ONCE can also tell the compiler not to reorder it. So, I think it's safe? Best regards, Yanfei > > Thanx, Paul > >> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c >> index da6f5213fb74..c6d95a00715e 100644 >> --- a/kernel/rcu/tree.c >> +++ b/kernel/rcu/tree.c >> @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) >> if (IS_ENABLED(CONFIG_PREEMPTION)) >> return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; >> might_sleep(); /* Check for RCU read-side critical section. */ >> - preempt_disable(); >> /* >> * If the rcu_state.n_online_cpus counter is equal to one, >> * there is only one CPU, and that CPU sees all prior accesses >> @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) >> * Those memory barriers are provided by CPU-hotplug code. >> */ >> ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; >> - preempt_enable(); >> return ret; >> } >> >> >> >> Best regards, >> Yanfei ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qestion] Is preempt_disable/enable needed in non-preemption code path 2021-04-15 16:18 ` Xu, Yanfei @ 2021-04-15 17:01 ` Xu, Yanfei 2021-04-15 17:15 ` Paul E. McKenney 2021-04-15 17:07 ` Paul E. McKenney 1 sibling, 1 reply; 9+ messages in thread From: Xu, Yanfei @ 2021-04-15 17:01 UTC (permalink / raw) To: paulmck; +Cc: rcu, LKML On 4/16/21 12:18 AM, Xu, Yanfei wrote: > > > On 4/15/21 11:43 PM, Paul E. McKenney wrote: >> [Please note: This e-mail is from an EXTERNAL e-mail address] >> >> On Thu, Apr 15, 2021 at 11:04:05PM +0800, Xu, Yanfei wrote: >>> Hi experts, >>> >>> I am learning rcu mechanism and its codes. When looking at the >>> rcu_blocking_is_gp(), I found there is a pair preemption disable/enable >>> operation in non-preemption code path. And it has been a long time. I >>> can't >>> understand why we need it? Is there some thing I missed? If not, can we >>> remove the unnecessary operation like blow? >> >> Good point, you are right that preemption is disabled anyway in that >> block >> of code. However, preempt_disable() and preempt_enable() also prevent >> the >> compiler from moving that READ_ONCE() around. So my question to you is >> whether it is safe to remove those statements entirely or whether they >> should instead be replaced by barrier() or similar. > > Thanks for your reply! :) > > Yes, preempt_disable() and preempt_enable() defined in !preemption are > barrier(). barrier can prevent from reordering that READ_ONCE(), but > base on my current understanding, volatile in READ_ONCE can also tell > the compiler not to reorder it. So, I think it's safe? > > Best regards, > Yanfei Hi Paul, I objdump the function rcu_blocking_is_gp(): after dropping the barrier(): ffffffff81107c50 <rcu_blocking_is_gp>: ffffffff81107c50: e8 7b 2a f5 ff callq ffffffff8105a6d0 <__fentry__> ffffffff81107c55: 8b 05 41 fe 7c 01 mov 0x17cfe41(%rip),%eax # ffffffff828d7a9c <rcu_state+0x221c> ffffffff81107c5b: 55 push %rbp ffffffff81107c5c: 48 89 e5 mov %rsp,%rbp ffffffff81107c5f: 5d pop %rbp ffffffff81107c60: 83 f8 01 cmp $0x1,%eax ffffffff81107c63: 0f 9e c0 setle %al ffffffff81107c66: 0f b6 c0 movzbl %al,%eax ffffffff81107c69: c3 retq ffffffff81107c6a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) the original codes: ffffffff81107ba0 <rcu_blocking_is_gp>: ffffffff81107ba0: e8 2b 2b f5 ff callq ffffffff8105a6d0 <__fentry__> ffffffff81107ba5: 55 push %rbp ffffffff81107ba6: 48 89 e5 mov %rsp,%rbp ffffffff81107ba9: 8b 05 ed fe 7c 01 mov 0x17cfeed(%rip),%eax # ffffffff828d7a9c <rcu_state+0x221c> ffffffff81107baf: 83 f8 01 cmp $0x1,%eax ffffffff81107bb2: 5d pop %rbp ffffffff81107bb3: 0f 9e c0 setle %al ffffffff81107bb6: 0f b6 c0 movzbl %al,%eax ffffffff81107bb9: c3 retq ffffffff81107bba: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) umm... It did been reordered by compiler after dropping the barrier(), however, I think the result will not be effected. Right? Best regards, Yanfei > >> >> Thanx, Paul >> >>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c >>> index da6f5213fb74..c6d95a00715e 100644 >>> --- a/kernel/rcu/tree.c >>> +++ b/kernel/rcu/tree.c >>> @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) >>> if (IS_ENABLED(CONFIG_PREEMPTION)) >>> return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; >>> might_sleep(); /* Check for RCU read-side critical section. */ >>> - preempt_disable(); >>> /* >>> * If the rcu_state.n_online_cpus counter is equal to one, >>> * there is only one CPU, and that CPU sees all prior accesses >>> @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) >>> * Those memory barriers are provided by CPU-hotplug code. >>> */ >>> ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; >>> - preempt_enable(); >>> return ret; >>> } >>> >>> >>> >>> Best regards, >>> Yanfei ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qestion] Is preempt_disable/enable needed in non-preemption code path 2021-04-15 17:01 ` Xu, Yanfei @ 2021-04-15 17:15 ` Paul E. McKenney 0 siblings, 0 replies; 9+ messages in thread From: Paul E. McKenney @ 2021-04-15 17:15 UTC (permalink / raw) To: Xu, Yanfei; +Cc: rcu, LKML On Fri, Apr 16, 2021 at 01:01:17AM +0800, Xu, Yanfei wrote: > > > On 4/16/21 12:18 AM, Xu, Yanfei wrote: > > > > > > On 4/15/21 11:43 PM, Paul E. McKenney wrote: > > > [Please note: This e-mail is from an EXTERNAL e-mail address] > > > > > > On Thu, Apr 15, 2021 at 11:04:05PM +0800, Xu, Yanfei wrote: > > > > Hi experts, > > > > > > > > I am learning rcu mechanism and its codes. When looking at the > > > > rcu_blocking_is_gp(), I found there is a pair preemption disable/enable > > > > operation in non-preemption code path. And it has been a long > > > > time. I can't > > > > understand why we need it? Is there some thing I missed? If not, can we > > > > remove the unnecessary operation like blow? > > > > > > Good point, you are right that preemption is disabled anyway in that > > > block > > > of code. However, preempt_disable() and preempt_enable() also > > > prevent the > > > compiler from moving that READ_ONCE() around. So my question to you is > > > whether it is safe to remove those statements entirely or whether they > > > should instead be replaced by barrier() or similar. > > > > Thanks for your reply! :) > > > > Yes, preempt_disable() and preempt_enable() defined in !preemption are > > barrier(). barrier can prevent from reordering that READ_ONCE(), but > > base on my current understanding, volatile in READ_ONCE can also tell > > the compiler not to reorder it. So, I think it's safe? > > > > Best regards, > > Yanfei > > Hi Paul, > I objdump the function rcu_blocking_is_gp(): > > after dropping the barrier(): > ffffffff81107c50 <rcu_blocking_is_gp>: > ffffffff81107c50: e8 7b 2a f5 ff callq ffffffff8105a6d0 > <__fentry__> > ffffffff81107c55: 8b 05 41 fe 7c 01 mov 0x17cfe41(%rip),%eax > # ffffffff828d7a9c <rcu_state+0x221c> > ffffffff81107c5b: 55 push %rbp > ffffffff81107c5c: 48 89 e5 mov %rsp,%rbp > ffffffff81107c5f: 5d pop %rbp > ffffffff81107c60: 83 f8 01 cmp $0x1,%eax > ffffffff81107c63: 0f 9e c0 setle %al > ffffffff81107c66: 0f b6 c0 movzbl %al,%eax > ffffffff81107c69: c3 retq > ffffffff81107c6a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) > > the original codes: > ffffffff81107ba0 <rcu_blocking_is_gp>: > ffffffff81107ba0: e8 2b 2b f5 ff callq ffffffff8105a6d0 > <__fentry__> > ffffffff81107ba5: 55 push %rbp > ffffffff81107ba6: 48 89 e5 mov %rsp,%rbp > ffffffff81107ba9: 8b 05 ed fe 7c 01 mov 0x17cfeed(%rip),%eax > # ffffffff828d7a9c <rcu_state+0x221c> > ffffffff81107baf: 83 f8 01 cmp $0x1,%eax > ffffffff81107bb2: 5d pop %rbp > ffffffff81107bb3: 0f 9e c0 setle %al > ffffffff81107bb6: 0f b6 c0 movzbl %al,%eax > ffffffff81107bb9: c3 retq > ffffffff81107bba: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) > > umm... It did been reordered by compiler after dropping the barrier(), > however, I think the result will not be effected. Right? Indeed, the compiler is free to reorder volatile accesses with non-volatile accesses. The result might not be affected by your compiler using your particular settings for optimizations, but is that true for all optimization settings for all compilers, both now and in the future? Thanx, Paul > Best regards, > Yanfei > > > > > > > > > Thanx, Paul > > > > > > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > > > > index da6f5213fb74..c6d95a00715e 100644 > > > > --- a/kernel/rcu/tree.c > > > > +++ b/kernel/rcu/tree.c > > > > @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) > > > > if (IS_ENABLED(CONFIG_PREEMPTION)) > > > > return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; > > > > might_sleep(); /* Check for RCU read-side critical section. */ > > > > - preempt_disable(); > > > > /* > > > > * If the rcu_state.n_online_cpus counter is equal to one, > > > > * there is only one CPU, and that CPU sees all prior accesses > > > > @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) > > > > * Those memory barriers are provided by CPU-hotplug code. > > > > */ > > > > ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; > > > > - preempt_enable(); > > > > return ret; > > > > } > > > > > > > > > > > > > > > > Best regards, > > > > Yanfei ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qestion] Is preempt_disable/enable needed in non-preemption code path 2021-04-15 16:18 ` Xu, Yanfei 2021-04-15 17:01 ` Xu, Yanfei @ 2021-04-15 17:07 ` Paul E. McKenney 2021-04-16 10:51 ` Xu, Yanfei 1 sibling, 1 reply; 9+ messages in thread From: Paul E. McKenney @ 2021-04-15 17:07 UTC (permalink / raw) To: Xu, Yanfei; +Cc: rcu, LKML On Fri, Apr 16, 2021 at 12:18:42AM +0800, Xu, Yanfei wrote: > > > On 4/15/21 11:43 PM, Paul E. McKenney wrote: > > [Please note: This e-mail is from an EXTERNAL e-mail address] > > > > On Thu, Apr 15, 2021 at 11:04:05PM +0800, Xu, Yanfei wrote: > > > Hi experts, > > > > > > I am learning rcu mechanism and its codes. When looking at the > > > rcu_blocking_is_gp(), I found there is a pair preemption disable/enable > > > operation in non-preemption code path. And it has been a long time. I can't > > > understand why we need it? Is there some thing I missed? If not, can we > > > remove the unnecessary operation like blow? > > > > Good point, you are right that preemption is disabled anyway in that block > > of code. However, preempt_disable() and preempt_enable() also prevent the > > compiler from moving that READ_ONCE() around. So my question to you is > > whether it is safe to remove those statements entirely or whether they > > should instead be replaced by barrier() or similar. > > Thanks for your reply! :) > > Yes, preempt_disable() and preempt_enable() defined in !preemption are > barrier(). barrier can prevent from reordering that READ_ONCE(), but base on > my current understanding, volatile in READ_ONCE can also tell the compiler > not to reorder it. So, I think it's safe? Maybe. Please keep in mind that although the compiler is prohibited from reordering volatile accesses with each other, there is nothing stopping it from reordering volatile accesses with non-volatile accesses. Thanx, Paul > Best regards, > Yanfei > > > > > Thanx, Paul > > > > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > > > index da6f5213fb74..c6d95a00715e 100644 > > > --- a/kernel/rcu/tree.c > > > +++ b/kernel/rcu/tree.c > > > @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) > > > if (IS_ENABLED(CONFIG_PREEMPTION)) > > > return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; > > > might_sleep(); /* Check for RCU read-side critical section. */ > > > - preempt_disable(); > > > /* > > > * If the rcu_state.n_online_cpus counter is equal to one, > > > * there is only one CPU, and that CPU sees all prior accesses > > > @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) > > > * Those memory barriers are provided by CPU-hotplug code. > > > */ > > > ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; > > > - preempt_enable(); > > > return ret; > > > } > > > > > > > > > > > > Best regards, > > > Yanfei ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qestion] Is preempt_disable/enable needed in non-preemption code path 2021-04-15 17:07 ` Paul E. McKenney @ 2021-04-16 10:51 ` Xu, Yanfei 2021-04-16 17:26 ` Paul E. McKenney 0 siblings, 1 reply; 9+ messages in thread From: Xu, Yanfei @ 2021-04-16 10:51 UTC (permalink / raw) To: paulmck; +Cc: rcu, LKML On 4/16/21 1:07 AM, Paul E. McKenney wrote: > [Please note: This e-mail is from an EXTERNAL e-mail address] > > On Fri, Apr 16, 2021 at 12:18:42AM +0800, Xu, Yanfei wrote: >> >> >> On 4/15/21 11:43 PM, Paul E. McKenney wrote: >>> [Please note: This e-mail is from an EXTERNAL e-mail address] >>> >>> On Thu, Apr 15, 2021 at 11:04:05PM +0800, Xu, Yanfei wrote: >>>> Hi experts, >>>> >>>> I am learning rcu mechanism and its codes. When looking at the >>>> rcu_blocking_is_gp(), I found there is a pair preemption disable/enable >>>> operation in non-preemption code path. And it has been a long time. I can't >>>> understand why we need it? Is there some thing I missed? If not, can we >>>> remove the unnecessary operation like blow? >>> >>> Good point, you are right that preemption is disabled anyway in that block >>> of code. However, preempt_disable() and preempt_enable() also prevent the >>> compiler from moving that READ_ONCE() around. So my question to you is >>> whether it is safe to remove those statements entirely or whether they >>> should instead be replaced by barrier() or similar. >> >> Thanks for your reply! :) >> >> Yes, preempt_disable() and preempt_enable() defined in !preemption are >> barrier(). barrier can prevent from reordering that READ_ONCE(), but base on >> my current understanding, volatile in READ_ONCE can also tell the compiler >> not to reorder it. So, I think it's safe? > > Maybe. > > Please keep in mind that although the compiler is prohibited from > reordering volatile accesses with each other, there is nothing stopping > it from reordering volatile accesses with non-volatile accesses. Thanks for your patient explanation! I am trying to absorb what you said. Blow are my understanding: 1. "the compiler is prohibited from reordering volatile accesses with each other" means these situations: int a; foo() { for(;;) READ_ONCE(a); } or int a,b; foo() { int c,d; c = READ_ONCE(a); d = READ_ONCE(b); } 2. "volatile accesses with non-volatile accesses" means d=b may happen before c=READ_ONCE(a) : int a; foo() { int b = 2 int c,d; c = READ_ONCE(a); d = b; } if we want to keep the ordering of volatile access "c=READ_ONCE(a)" and non-volatile access "d=b", we should use stronger barrier like barrier(). Hope I didn't misunderstand. Back to rcu_blocking_is_gp(), I find this link today https://www.spinics.net/lists/rcu/msg03985.html With the content in this link, I still haven't got the meaning of these two barrier(). I think I should learn knowledge about cpu-hotplug and things which talked in the link first to make sure if I am missing something, and then consult you. :) Best regards, Yanfei > > Thanx, Paul > >> Best regards, >> Yanfei >> >>> >>> Thanx, Paul >>> >>>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c >>>> index da6f5213fb74..c6d95a00715e 100644 >>>> --- a/kernel/rcu/tree.c >>>> +++ b/kernel/rcu/tree.c >>>> @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) >>>> if (IS_ENABLED(CONFIG_PREEMPTION)) >>>> return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; >>>> might_sleep(); /* Check for RCU read-side critical section. */ >>>> - preempt_disable(); >>>> /* >>>> * If the rcu_state.n_online_cpus counter is equal to one, >>>> * there is only one CPU, and that CPU sees all prior accesses >>>> @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) >>>> * Those memory barriers are provided by CPU-hotplug code. >>>> */ >>>> ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; >>>> - preempt_enable(); >>>> return ret; >>>> } >>>> >>>> >>>> >>>> Best regards, >>>> Yanfei ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qestion] Is preempt_disable/enable needed in non-preemption code path 2021-04-16 10:51 ` Xu, Yanfei @ 2021-04-16 17:26 ` Paul E. McKenney 2021-04-19 4:58 ` Xu, Yanfei 0 siblings, 1 reply; 9+ messages in thread From: Paul E. McKenney @ 2021-04-16 17:26 UTC (permalink / raw) To: Xu, Yanfei; +Cc: rcu, LKML On Fri, Apr 16, 2021 at 06:51:10PM +0800, Xu, Yanfei wrote: > > > On 4/16/21 1:07 AM, Paul E. McKenney wrote: > > [Please note: This e-mail is from an EXTERNAL e-mail address] > > > > On Fri, Apr 16, 2021 at 12:18:42AM +0800, Xu, Yanfei wrote: > > > > > > > > > On 4/15/21 11:43 PM, Paul E. McKenney wrote: > > > > [Please note: This e-mail is from an EXTERNAL e-mail address] > > > > > > > > On Thu, Apr 15, 2021 at 11:04:05PM +0800, Xu, Yanfei wrote: > > > > > Hi experts, > > > > > > > > > > I am learning rcu mechanism and its codes. When looking at the > > > > > rcu_blocking_is_gp(), I found there is a pair preemption disable/enable > > > > > operation in non-preemption code path. And it has been a long time. I can't > > > > > understand why we need it? Is there some thing I missed? If not, can we > > > > > remove the unnecessary operation like blow? > > > > > > > > Good point, you are right that preemption is disabled anyway in that block > > > > of code. However, preempt_disable() and preempt_enable() also prevent the > > > > compiler from moving that READ_ONCE() around. So my question to you is > > > > whether it is safe to remove those statements entirely or whether they > > > > should instead be replaced by barrier() or similar. > > > > > > Thanks for your reply! :) > > > > > > Yes, preempt_disable() and preempt_enable() defined in !preemption are > > > barrier(). barrier can prevent from reordering that READ_ONCE(), but base on > > > my current understanding, volatile in READ_ONCE can also tell the compiler > > > not to reorder it. So, I think it's safe? > > > > Maybe. > > > > Please keep in mind that although the compiler is prohibited from > > reordering volatile accesses with each other, there is nothing stopping > > it from reordering volatile accesses with non-volatile accesses. > > Thanks for your patient explanation! > > I am trying to absorb what you said. Blow are my understanding: > 1. "the compiler is prohibited from reordering volatile accesses with each > other" means these situations: > int a; > foo() > { > for(;;) > READ_ONCE(a); > } > > or > > int a,b; > foo() > { > int c,d; > c = READ_ONCE(a); > d = READ_ONCE(b); > } Yes, in both cases the load instructions emitted for the READ_ONCE() macros must be emitted in order. The underlying hardware is free to reorder. > 2. "volatile accesses with non-volatile accesses" means d=b may happen > before c=READ_ONCE(a) : > int a; > foo() > { > int b = 2 > int c,d; > c = READ_ONCE(a); > d = b; > } > if we want to keep the ordering of volatile access "c=READ_ONCE(a)" and > non-volatile access "d=b", we should use stronger barrier like barrier(). Or an additional READ_ONCE() for b or a WRITE_ONCE() for d. But again, this would constrain only the compiler, not the hardware. But this wouldn't matter in most cases, because both b and d are local variables whose addresses were never taken. So someone would need to be using something crazy to poke into others' stacks for this to matter. > Hope I didn't misunderstand. It looks like you have most of it. > Back to rcu_blocking_is_gp(), I find this link today > https://www.spinics.net/lists/rcu/msg03985.html > With the content in this link, I still haven't got the meaning of these two > barrier(). I think I should learn knowledge about cpu-hotplug and things > which talked in the link first to make sure if I am missing something, and > then consult you. :) That sounds like a very good approach! Keep in mind that I am worried not just about the current state of the code and compilers, but also their possible future states. Thanx, Paul > Best regards, > Yanfei > > > > > Thanx, Paul > > > > > Best regards, > > > Yanfei > > > > > > > > > > > Thanx, Paul > > > > > > > > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c > > > > > index da6f5213fb74..c6d95a00715e 100644 > > > > > --- a/kernel/rcu/tree.c > > > > > +++ b/kernel/rcu/tree.c > > > > > @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) > > > > > if (IS_ENABLED(CONFIG_PREEMPTION)) > > > > > return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; > > > > > might_sleep(); /* Check for RCU read-side critical section. */ > > > > > - preempt_disable(); > > > > > /* > > > > > * If the rcu_state.n_online_cpus counter is equal to one, > > > > > * there is only one CPU, and that CPU sees all prior accesses > > > > > @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) > > > > > * Those memory barriers are provided by CPU-hotplug code. > > > > > */ > > > > > ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; > > > > > - preempt_enable(); > > > > > return ret; > > > > > } > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > Yanfei ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qestion] Is preempt_disable/enable needed in non-preemption code path 2021-04-16 17:26 ` Paul E. McKenney @ 2021-04-19 4:58 ` Xu, Yanfei 0 siblings, 0 replies; 9+ messages in thread From: Xu, Yanfei @ 2021-04-19 4:58 UTC (permalink / raw) To: paulmck; +Cc: rcu, LKML On 4/17/21 1:26 AM, Paul E. McKenney wrote: > [Please note: This e-mail is from an EXTERNAL e-mail address] > > On Fri, Apr 16, 2021 at 06:51:10PM +0800, Xu, Yanfei wrote: >> >> >> On 4/16/21 1:07 AM, Paul E. McKenney wrote: >>> [Please note: This e-mail is from an EXTERNAL e-mail address] >>> >>> On Fri, Apr 16, 2021 at 12:18:42AM +0800, Xu, Yanfei wrote: >>>> >>>> >>>> On 4/15/21 11:43 PM, Paul E. McKenney wrote: >>>>> [Please note: This e-mail is from an EXTERNAL e-mail address] >>>>> >>>>> On Thu, Apr 15, 2021 at 11:04:05PM +0800, Xu, Yanfei wrote: >>>>>> Hi experts, >>>>>> >>>>>> I am learning rcu mechanism and its codes. When looking at the >>>>>> rcu_blocking_is_gp(), I found there is a pair preemption disable/enable >>>>>> operation in non-preemption code path. And it has been a long time. I can't >>>>>> understand why we need it? Is there some thing I missed? If not, can we >>>>>> remove the unnecessary operation like blow? >>>>> >>>>> Good point, you are right that preemption is disabled anyway in that block >>>>> of code. However, preempt_disable() and preempt_enable() also prevent the >>>>> compiler from moving that READ_ONCE() around. So my question to you is >>>>> whether it is safe to remove those statements entirely or whether they >>>>> should instead be replaced by barrier() or similar. >>>> >>>> Thanks for your reply! :) >>>> >>>> Yes, preempt_disable() and preempt_enable() defined in !preemption are >>>> barrier(). barrier can prevent from reordering that READ_ONCE(), but base on >>>> my current understanding, volatile in READ_ONCE can also tell the compiler >>>> not to reorder it. So, I think it's safe? >>> >>> Maybe. >>> >>> Please keep in mind that although the compiler is prohibited from >>> reordering volatile accesses with each other, there is nothing stopping >>> it from reordering volatile accesses with non-volatile accesses. >> >> Thanks for your patient explanation! >> >> I am trying to absorb what you said. Blow are my understanding: >> 1. "the compiler is prohibited from reordering volatile accesses with each >> other" means these situations: >> int a; >> foo() >> { >> for(;;) >> READ_ONCE(a); >> } >> >> or >> >> int a,b; >> foo() >> { >> int c,d; >> c = READ_ONCE(a); >> d = READ_ONCE(b); >> } > > Yes, in both cases the load instructions emitted for the READ_ONCE() > macros must be emitted in order. The underlying hardware is free > to reorder. Got it. > >> 2. "volatile accesses with non-volatile accesses" means d=b may happen >> before c=READ_ONCE(a) : >> int a; >> foo() >> { >> int b = 2 >> int c,d; >> c = READ_ONCE(a); >> d = b; >> } >> if we want to keep the ordering of volatile access "c=READ_ONCE(a)" and >> non-volatile access "d=b", we should use stronger barrier like barrier(). > > Or an additional READ_ONCE() for b or a WRITE_ONCE() for d. But again, > this would constrain only the compiler, not the hardware. > > But this wouldn't matter in most cases, because both b and d are local > variables whose addresses were never taken. So someone would need to > be using something crazy to poke into others' stacks for this to matter. Agree. > >> Hope I didn't misunderstand. > > It looks like you have most of it. > >> Back to rcu_blocking_is_gp(), I find this link today >> https://www.spinics.net/lists/rcu/msg03985.html >> With the content in this link, I still haven't got the meaning of these two >> barrier(). I think I should learn knowledge about cpu-hotplug and things >> which talked in the link first to make sure if I am missing something, and >> then consult you. :) > > That sounds like a very good approach! > > Keep in mind that I am worried not just about the current state of > the code and compilers, but also their possible future states. I see. Thanks again. Best regards, Yanfei > > Thanx, Paul > >> Best regards, >> Yanfei >> >>> >>> Thanx, Paul >>> >>>> Best regards, >>>> Yanfei >>>> >>>>> >>>>> Thanx, Paul >>>>> >>>>>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c >>>>>> index da6f5213fb74..c6d95a00715e 100644 >>>>>> --- a/kernel/rcu/tree.c >>>>>> +++ b/kernel/rcu/tree.c >>>>>> @@ -3703,7 +3703,6 @@ static int rcu_blocking_is_gp(void) >>>>>> if (IS_ENABLED(CONFIG_PREEMPTION)) >>>>>> return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE; >>>>>> might_sleep(); /* Check for RCU read-side critical section. */ >>>>>> - preempt_disable(); >>>>>> /* >>>>>> * If the rcu_state.n_online_cpus counter is equal to one, >>>>>> * there is only one CPU, and that CPU sees all prior accesses >>>>>> @@ -3718,7 +3717,6 @@ static int rcu_blocking_is_gp(void) >>>>>> * Those memory barriers are provided by CPU-hotplug code. >>>>>> */ >>>>>> ret = READ_ONCE(rcu_state.n_online_cpus) <= 1; >>>>>> - preempt_enable(); >>>>>> return ret; >>>>>> } >>>>>> >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Yanfei ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-04-19 4:59 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-04-15 15:04 [Qestion] Is preempt_disable/enable needed in non-preemption code path Xu, Yanfei 2021-04-15 15:43 ` Paul E. McKenney 2021-04-15 16:18 ` Xu, Yanfei 2021-04-15 17:01 ` Xu, Yanfei 2021-04-15 17:15 ` Paul E. McKenney 2021-04-15 17:07 ` Paul E. McKenney 2021-04-16 10:51 ` Xu, Yanfei 2021-04-16 17:26 ` Paul E. McKenney 2021-04-19 4:58 ` Xu, Yanfei
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).