Re: [PATCH v2 7/9] sched: define TIF_ALLOW_RESCHED

From: Peter Zijlstra <peterz@infradead.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Steven Rostedt <rostedt@goodmis.org>,
	Ankur Arora <ankur.a.arora@oracle.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org,
	akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de,
	dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com,
	juri.lelli@redhat.com, vincent.guittot@linaro.org,
	willy@infradead.org, mgorman@suse.de, tglx@linutronix.de,
	jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com,
	boris.ostrovsky@oracle.com, konrad.wilk@oracle.com
Subject: Re: [PATCH v2 7/9] sched: define TIF_ALLOW_RESCHED
Date: Tue, 12 Sep 2023 09:20:22 +0200	[thread overview]
Message-ID: <20230912072022.GA35261@noisy.programming.kicks-ass.net> (raw)
In-Reply-To: <CAHk-=whc0Jsji_h-vLyDLT6Q-NCjSTMOg9qXACmoPZOdWqMovg@mail.gmail.com>

On Mon, Sep 11, 2023 at 02:16:18PM -0700, Linus Torvalds wrote:
> On Mon, 11 Sept 2023 at 13:50, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
> >
> > Except we've actually been *adding* to this whole mess, rather than
> > removing it. So we have actively *expanded* on that preemption choice
> > with PREEMPT_DYNAMIC.
> 
> Actually, that config option makes no sense.
> 
> It makes the sched_cond() behavior conditional with a static call.
> 
> But all the *real* overhead is still there and unconditional (ie all
> the preempt count updates and the "did it go down to zero and we need
> to check" code).
> 
> That just seems stupid. It seems to have all the overhead of a
> preemptible kernel, just not doing the preemption.
> 
> So I must be mis-reading this, or just missing something important.
> 
> The real cost seems to be
> 
>    PREEMPT_BUILD -> PREEMPTION -> PREEMPT_COUNT
> 
> and PREEMPT vs PREEMPT_DYNAMIC makes no difference to that, since both
> will end up with that, and thus both cases will have all the spinlock
> preempt count stuff.
> 
> There must be some non-preempt_count cost that people worry about.
> 
> Or maybe I'm just mis-reading the Kconfig stuff entirely. That's
> possible, because this seems *so* pointless to me.
> 
> Somebody please hit me with a clue-bat to the noggin.

Well, I was about to reply to your previous email explaining this, but
this one time I did read more email..

Yes, PREEMPT_DYNAMIC has all the preempt count twiddling and only nops
out the schedule()/cond_resched() calls where appropriate.

This work was done by a distro (SuSE) and if they're willing to ship
this I'm thinking the overheads are acceptable to them.

For a significant number of workloads the real overhead is the extra
preepmtions themselves more than the counting -- but yes, the counting
is measurable, but probably in the noise compared to other some of the
other horrible things we have done the past years.

Anyway, if distros are fine shipping with PREEMPT_DYNAMIC, then yes,
deleting the other options are definitely an option.