From: Nadav Amit <namit@vmware.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>,
Andy Lutomirski <luto@kernel.org>,
Dave Hansen <dave.hansen@linux.intel.com>,
the arch/x86 maintainers <x86@kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Ingo Molnar <mingo@redhat.com>, Rik van Riel <riel@surriel.com>,
Josh Poimboeuf <jpoimboe@redhat.com>
Subject: Re: [PATCH v3 1/9] smp: Run functions concurrently in smp_call_function_many()
Date: Mon, 22 Jul 2019 19:02:36 +0000 [thread overview]
Message-ID: <93A98E8B-764F-4E9F-B0B6-FDAABE822B2D@vmware.com> (raw)
In-Reply-To: <alpine.DEB.2.21.1907222045101.1659@nanos.tec.linutronix.de>
> On Jul 22, 2019, at 11:51 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
>
> On Mon, 22 Jul 2019, Nadav Amit wrote:
>>> On Jul 22, 2019, at 11:37 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
>>>
>>> On Mon, 22 Jul 2019, Peter Zijlstra wrote:
>>>
>>>> On Thu, Jul 18, 2019 at 05:58:29PM -0700, Nadav Amit wrote:
>>>>> +/*
>>>>> + * Call a function on all processors. May be used during early boot while
>>>>> + * early_boot_irqs_disabled is set.
>>>>> + */
>>>>> +static inline void on_each_cpu(smp_call_func_t func, void *info, int wait)
>>>>> +{
>>>>> + on_each_cpu_mask(cpu_online_mask, func, info, wait);
>>>>> +}
>>>>
>>>> I'm thinking that one if buggy, nothing protects online mask here.
>>>
>>> The current implementation has preemption disabled before touching
>>> cpu_online_mask which at least protects against a CPU going away as that
>>> prevents the stomp machine thread from getting on the CPU. But it's not
>>> protected against a CPU coming online concurrently.
>>
>> I still don’t understand. If you called cpu_online_mask() and did not
>> disable preemption before calling it, you are already (today) not protected
>> against another CPU coming online. Disabling preemption in on_each_cpu()
>> will not solve it.
>
> Disabling preemption _cannot_ protect against a CPU coming online. It only
> can protect against a CPU being offlined.
>
> The current implementation of on_each_cpu() disables preemption _before_
> touching cpu_online_mask.
>
> void on_each_cpu(void (*func) (void *info), void *info, int wait)
> {
> unsigned long flags;
>
> preempt_disable();
> smp_call_function(func, info, wait);
>
> smp_call_function() has another preempt_disable as it can be called
> separately and it does:
>
> preempt_disable();
> smp_call_function_many(cpu_online_mask, func, info, wait);
>
> Your new on_each_cpu() implementation does not. So there is a
> difference. Whether it matters or not is a different question, but that
> needs to be explained and documented.
Thanks for explaining - so your concern is for CPUs being offlined.
But unless I am missing something: on_each_cpu() calls __on_each_cpu_mask(),
which disables preemption and calls __smp_call_function_many().
Then __smp_call_function_many() runs:
cpumask_and(cfd->cpumask, mask, cpu_online_mask);
… before choosing which remote CPUs should run the function. So the only
case that I was missing is if the current CPU goes away and the function is
called locally.
Can it happen? I can add documentation and a debug assertion for this case.
next prev parent reply other threads:[~2019-07-22 19:02 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-19 0:58 [PATCH v3 0/9] x86: Concurrent TLB flushes Nadav Amit
2019-07-19 0:58 ` [PATCH v3 1/9] smp: Run functions concurrently in smp_call_function_many() Nadav Amit
2019-07-19 18:23 ` Dave Hansen
2019-07-22 18:16 ` Peter Zijlstra
2019-07-22 18:41 ` Nadav Amit
2019-07-22 19:34 ` Peter Zijlstra
2019-07-22 18:21 ` Peter Zijlstra
2019-07-22 18:34 ` Nadav Amit
2019-07-22 19:19 ` Peter Zijlstra
2019-07-22 18:37 ` Thomas Gleixner
2019-07-22 18:40 ` Nadav Amit
2019-07-22 18:51 ` Thomas Gleixner
2019-07-22 19:02 ` Nadav Amit [this message]
2019-07-25 12:36 ` Thomas Gleixner
2019-07-25 19:10 ` Nadav Amit
2019-07-19 0:58 ` [PATCH v3 2/9] x86/mm/tlb: Remove reason as argument for flush_tlb_func_local() Nadav Amit
2019-07-19 0:58 ` [PATCH v3 3/9] x86/mm/tlb: Open-code on_each_cpu_cond_mask() for tlb_is_not_lazy() Nadav Amit
2019-07-19 18:36 ` Dave Hansen
2019-07-19 18:41 ` Nadav Amit
2019-07-19 22:44 ` Joe Perches
2019-07-19 23:02 ` Nadav Amit
2019-07-22 18:27 ` Peter Zijlstra
2019-07-22 19:47 ` Rasmus Villemoes
2019-07-22 19:51 ` Nadav Amit
2019-07-19 0:58 ` [PATCH v3 4/9] x86/mm/tlb: Flush remote and local TLBs concurrently Nadav Amit
2019-07-22 19:14 ` Peter Zijlstra
2019-07-22 19:27 ` Nadav Amit
2019-07-22 19:32 ` Peter Zijlstra
2019-07-26 7:28 ` Juergen Gross
2019-07-31 0:13 ` Michael Kelley
2019-07-19 0:58 ` [PATCH v3 5/9] x86/mm/tlb: Privatize cpu_tlbstate Nadav Amit
2019-07-19 18:38 ` Dave Hansen
2019-07-19 18:43 ` Nadav Amit
2019-07-19 18:48 ` Dave Hansen
2019-07-19 18:54 ` Nadav Amit
2019-07-20 13:58 ` Andy Lutomirski
2019-07-21 20:21 ` Nadav Amit
2019-07-19 0:58 ` [PATCH v3 6/9] x86/mm/tlb: Do not make is_lazy dirty for no reason Nadav Amit
2019-07-19 0:58 ` [PATCH v3 7/9] cpumask: Mark functions as pure Nadav Amit
2019-07-19 0:58 ` [PATCH v3 8/9] x86/mm/tlb: Remove UV special case Nadav Amit
2019-07-19 2:25 ` Mike Travis
2019-07-19 4:58 ` Nadav Amit
2019-07-31 3:11 ` Nadav Amit
2019-07-19 0:58 ` [PATCH v3 9/9] x86/mm/tlb: Remove unnecessary uses of the inline keyword Nadav Amit
2019-07-19 21:36 ` [PATCH v3 0/9] x86: Concurrent TLB flushes Dave Hansen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=93A98E8B-764F-4E9F-B0B6-FDAABE822B2D@vmware.com \
--to=namit@vmware.com \
--cc=dave.hansen@linux.intel.com \
--cc=jpoimboe@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=riel@surriel.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).