* Local execution of ipi_sync_rq_state() on sync_runqueues_membarrier_state()
@ 2021-02-16 21:35 Nadav Amit
2021-02-17 14:54 ` Mathieu Desnoyers
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Nadav Amit @ 2021-02-16 21:35 UTC (permalink / raw)
To: Mathieu Desnoyers; +Cc: LKML, Peter Zijlstra
Hello Mathieu,
While trying to find some unrelated by, something in
sync_runqueues_membarrier_state() caught my eye:
static int sync_runqueues_membarrier_state(struct mm_struct *mm)
{
if (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1) {
this_cpu_write(runqueues.membarrier_state, membarrier_state);
/*
* For single mm user, we can simply issue a memory barrier
* after setting MEMBARRIER_STATE_GLOBAL_EXPEDITED in the
* mm and in the current runqueue to guarantee that no memory
* access following registration is reordered before
* registration.
*/
smp_mb();
return 0;
}
[ snip ]
smp_call_function_many(tmpmask, ipi_sync_rq_state, mm, 1);
And ipi_sync_rq_state() does:
this_cpu_write(runqueues.membarrier_state,
atomic_read(&mm->membarrier_state));
So my question: are you aware smp_call_function_many() would not run
ipi_sync_rq_state() on the local CPU? Is that the intention of the code?
Thanks,
Nadav
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Local execution of ipi_sync_rq_state() on sync_runqueues_membarrier_state()
2021-02-16 21:35 Local execution of ipi_sync_rq_state() on sync_runqueues_membarrier_state() Nadav Amit
@ 2021-02-17 14:54 ` Mathieu Desnoyers
2021-03-01 10:16 ` [tip: sched/urgent] sched/membarrier: fix missing local execution of ipi_sync_rq_state() tip-bot2 for Mathieu Desnoyers
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Mathieu Desnoyers
2 siblings, 0 replies; 4+ messages in thread
From: Mathieu Desnoyers @ 2021-02-17 14:54 UTC (permalink / raw)
To: Nadav Amit; +Cc: linux-kernel, Peter Zijlstra
----- On Feb 16, 2021, at 4:35 PM, Nadav Amit nadav.amit@gmail.com wrote:
> Hello Mathieu,
>
> While trying to find some unrelated by, something in
> sync_runqueues_membarrier_state() caught my eye:
>
>
> static int sync_runqueues_membarrier_state(struct mm_struct *mm)
> {
> if (atomic_read(&mm->mm_users) == 1 || num_online_cpus() == 1) {
> this_cpu_write(runqueues.membarrier_state, membarrier_state);
>
> /*
> * For single mm user, we can simply issue a memory barrier
> * after setting MEMBARRIER_STATE_GLOBAL_EXPEDITED in the
> * mm and in the current runqueue to guarantee that no memory
> * access following registration is reordered before
> * registration.
> */
> smp_mb();
> return 0;
> }
>
> [ snip ]
>
> smp_call_function_many(tmpmask, ipi_sync_rq_state, mm, 1);
>
>
> And ipi_sync_rq_state() does:
>
> this_cpu_write(runqueues.membarrier_state,
> atomic_read(&mm->membarrier_state));
>
>
> So my question: are you aware smp_call_function_many() would not run
> ipi_sync_rq_state() on the local CPU?
Generally, yes, I am aware of it, but it appears that when I wrote that
code, I missed that important fact. See
commit 227a4aadc75b ("sched/membarrier: Fix p->mm->membarrier_state racy load")
> Is that the intention of the code?
Clearly not! If we look at sync_runqueues_membarrier_state(), there is even a
special-case for mm_users==1 || num online cpus == 1 where it writes the membarrier
state into the current cpu runqueue. I'll prepare a fix, thanks a bunch for spotting
this.
Mathieu
>
> Thanks,
> Nadav
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 4+ messages in thread
* [tip: sched/urgent] sched/membarrier: fix missing local execution of ipi_sync_rq_state()
2021-02-16 21:35 Local execution of ipi_sync_rq_state() on sync_runqueues_membarrier_state() Nadav Amit
2021-02-17 14:54 ` Mathieu Desnoyers
@ 2021-03-01 10:16 ` tip-bot2 for Mathieu Desnoyers
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Mathieu Desnoyers
2 siblings, 0 replies; 4+ messages in thread
From: tip-bot2 for Mathieu Desnoyers @ 2021-03-01 10:16 UTC (permalink / raw)
To: linux-tip-commits
Cc: Nadav Amit, Mathieu Desnoyers, Peter Zijlstra (Intel), stable, #,
5.4.x+,
x86, linux-kernel
The following commit has been merged into the sched/urgent branch of tip:
Commit-ID: fba111913e51a934eaad85734254eab801343836
Gitweb: https://git.kernel.org/tip/fba111913e51a934eaad85734254eab801343836
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
AuthorDate: Wed, 17 Feb 2021 11:56:51 -05:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Mon, 01 Mar 2021 11:02:15 +01:00
sched/membarrier: fix missing local execution of ipi_sync_rq_state()
The function sync_runqueues_membarrier_state() should copy the
membarrier state from the @mm received as parameter to each runqueue
currently running tasks using that mm.
However, the use of smp_call_function_many() skips the current runqueue,
which is unintended. Replace by a call to on_each_cpu_mask().
Fixes: 227a4aadc75b ("sched/membarrier: Fix p->mm->membarrier_state racy load")
Reported-by: Nadav Amit <nadav.amit@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org # 5.4.x+
Link: https://lore.kernel.org/r/74F1E842-4A84-47BF-B6C2-5407DFDD4A4A@gmail.com
---
kernel/sched/membarrier.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c
index acdae62..b5add64 100644
--- a/kernel/sched/membarrier.c
+++ b/kernel/sched/membarrier.c
@@ -471,9 +471,7 @@ static int sync_runqueues_membarrier_state(struct mm_struct *mm)
}
rcu_read_unlock();
- preempt_disable();
- smp_call_function_many(tmpmask, ipi_sync_rq_state, mm, 1);
- preempt_enable();
+ on_each_cpu_mask(tmpmask, ipi_sync_rq_state, mm, true);
free_cpumask_var(tmpmask);
cpus_read_unlock();
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [tip: sched/core] sched/membarrier: fix missing local execution of ipi_sync_rq_state()
2021-02-16 21:35 Local execution of ipi_sync_rq_state() on sync_runqueues_membarrier_state() Nadav Amit
2021-02-17 14:54 ` Mathieu Desnoyers
2021-03-01 10:16 ` [tip: sched/urgent] sched/membarrier: fix missing local execution of ipi_sync_rq_state() tip-bot2 for Mathieu Desnoyers
@ 2021-03-06 11:42 ` tip-bot2 for Mathieu Desnoyers
2 siblings, 0 replies; 4+ messages in thread
From: tip-bot2 for Mathieu Desnoyers @ 2021-03-06 11:42 UTC (permalink / raw)
To: linux-tip-commits
Cc: Nadav Amit, Mathieu Desnoyers, Peter Zijlstra (Intel),
Ingo Molnar, stable, #, 5.4.x+,
x86, linux-kernel
The following commit has been merged into the sched/core branch of tip:
Commit-ID: ce29ddc47b91f97e7f69a0fb7cbb5845f52a9825
Gitweb: https://git.kernel.org/tip/ce29ddc47b91f97e7f69a0fb7cbb5845f52a9825
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
AuthorDate: Wed, 17 Feb 2021 11:56:51 -05:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Sat, 06 Mar 2021 12:40:21 +01:00
sched/membarrier: fix missing local execution of ipi_sync_rq_state()
The function sync_runqueues_membarrier_state() should copy the
membarrier state from the @mm received as parameter to each runqueue
currently running tasks using that mm.
However, the use of smp_call_function_many() skips the current runqueue,
which is unintended. Replace by a call to on_each_cpu_mask().
Fixes: 227a4aadc75b ("sched/membarrier: Fix p->mm->membarrier_state racy load")
Reported-by: Nadav Amit <nadav.amit@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: stable@vger.kernel.org # 5.4.x+
Link: https://lore.kernel.org/r/74F1E842-4A84-47BF-B6C2-5407DFDD4A4A@gmail.com
---
kernel/sched/membarrier.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c
index acdae62..b5add64 100644
--- a/kernel/sched/membarrier.c
+++ b/kernel/sched/membarrier.c
@@ -471,9 +471,7 @@ static int sync_runqueues_membarrier_state(struct mm_struct *mm)
}
rcu_read_unlock();
- preempt_disable();
- smp_call_function_many(tmpmask, ipi_sync_rq_state, mm, 1);
- preempt_enable();
+ on_each_cpu_mask(tmpmask, ipi_sync_rq_state, mm, true);
free_cpumask_var(tmpmask);
cpus_read_unlock();
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-03-06 11:43 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-16 21:35 Local execution of ipi_sync_rq_state() on sync_runqueues_membarrier_state() Nadav Amit
2021-02-17 14:54 ` Mathieu Desnoyers
2021-03-01 10:16 ` [tip: sched/urgent] sched/membarrier: fix missing local execution of ipi_sync_rq_state() tip-bot2 for Mathieu Desnoyers
2021-03-06 11:42 ` [tip: sched/core] " tip-bot2 for Mathieu Desnoyers
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).