Re: [PATCH v3 1/9] smp: Run functions concurrently in smp_call_function_many()

From: Thomas Gleixner <tglx@linutronix.de>
To: Nadav Amit <namit@vmware.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Andy Lutomirski <luto@kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	the arch/x86 maintainers <x86@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@redhat.com>, Rik van Riel <riel@surriel.com>,
	Josh Poimboeuf <jpoimboe@redhat.com>
Subject: Re: [PATCH v3 1/9] smp: Run functions concurrently in smp_call_function_many()
Date: Mon, 22 Jul 2019 20:51:59 +0200 (CEST)	[thread overview]
Message-ID: <alpine.DEB.2.21.1907222045101.1659@nanos.tec.linutronix.de> (raw)
In-Reply-To: <91940019-826C-4F33-904B-0767D95A5E21@vmware.com>

[-- Attachment #1: Type: text/plain, Size: 1919 bytes --]

On Mon, 22 Jul 2019, Nadav Amit wrote:
> > On Jul 22, 2019, at 11:37 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> > 
> > On Mon, 22 Jul 2019, Peter Zijlstra wrote:
> > 
> >> On Thu, Jul 18, 2019 at 05:58:29PM -0700, Nadav Amit wrote:
> >>> +/*
> >>> + * Call a function on all processors.  May be used during early boot while
> >>> + * early_boot_irqs_disabled is set.
> >>> + */
> >>> +static inline void on_each_cpu(smp_call_func_t func, void *info, int wait)
> >>> +{
> >>> +	on_each_cpu_mask(cpu_online_mask, func, info, wait);
> >>> +}
> >> 
> >> I'm thinking that one if buggy, nothing protects online mask here.
> > 
> > The current implementation has preemption disabled before touching
> > cpu_online_mask which at least protects against a CPU going away as that
> > prevents the stomp machine thread from getting on the CPU. But it's not
> > protected against a CPU coming online concurrently.
> 
> I still don’t understand. If you called cpu_online_mask() and did not
> disable preemption before calling it, you are already (today) not protected
> against another CPU coming online. Disabling preemption in on_each_cpu()
> will not solve it.

Disabling preemption _cannot_ protect against a CPU coming online. It only
can protect against a CPU being offlined.

The current implementation of on_each_cpu() disables preemption _before_
touching cpu_online_mask.

void on_each_cpu(void (*func) (void *info), void *info, int wait)
{
        unsigned long flags;

        preempt_disable();
	smp_call_function(func, info, wait);

smp_call_function() has another preempt_disable as it can be called
separately and it does:

        preempt_disable();
        smp_call_function_many(cpu_online_mask, func, info, wait);

Your new on_each_cpu() implementation does not. So there is a
difference. Whether it matters or not is a different question, but that
needs to be explained and documented.

Thanks,

	tglx