Re: [RFC PATCH v4 1/2] sched/fair: Introduce short duration task check

From: Chen Yu <yu.c.chen@intel.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Tim Chen <tim.c.chen@intel.com>,
	Mel Gorman <mgorman@techsingularity.net>,
	Juri Lelli <juri.lelli@redhat.com>,
	"Rik van Riel" <riel@surriel.com>, Aaron Lu <aaron.lu@intel.com>,
	Abel Wu <wuyun.abel@bytedance.com>,
	K Prateek Nayak <kprateek.nayak@amd.com>,
	"Yicong Yang" <yangyicong@hisilicon.com>,
	"Gautham R . Shenoy" <gautham.shenoy@amd.com>,
	Ingo Molnar <mingo@redhat.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>,
	"Daniel Bristot de Oliveira" <bristot@redhat.com>,
	Valentin Schneider <vschneid@redhat.com>,
	Hillf Danton <hdanton@sina.com>,
	Honglei Wang <wanghonglei@didichuxing.com>,
	Len Brown <len.brown@intel.com>, Chen Yu <yu.chen.surf@gmail.com>,
	"Tianchen Ding" <dtcccc@linux.alibaba.com>,
	Joel Fernandes <joel@joelfernandes.org>,
	Josh Don <joshdon@google.com>, <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH v4 1/2] sched/fair: Introduce short duration task check
Date: Thu, 19 Jan 2023 22:29:01 +0800	[thread overview]
Message-ID: <Y8lTrW/WrFPWCHFo@chenyu5-mobl1> (raw)
In-Reply-To: <Y8Un9nmJauxpuSVE@hirez.programming.kicks-ass.net>

Hi Peter,
On 2023-01-16 at 11:33:26 +0100, Peter Zijlstra wrote:
> On Thu, Jan 05, 2023 at 12:33:16PM +0100, Dietmar Eggemann wrote:
> > On 16/12/2022 07:11, Chen Yu wrote:
> > 
> > [...]
> > 
> > > @@ -5995,6 +6005,18 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
> > >  
> > >  static void set_next_buddy(struct sched_entity *se);
> > >  
> > > +static inline void dur_avg_update(struct task_struct *p, bool task_sleep)
> > > +{
> > > +	u64 dur;
> > > +
> > > +	if (!task_sleep)
> > > +		return;
> > > +
> > > +	dur = p->se.sum_exec_runtime - p->se.prev_sum_exec_runtime_vol;
> > > +	p->se.prev_sum_exec_runtime_vol = p->se.sum_exec_runtime;
> > 
> > Shouldn't se->prev_sum_exec_runtime_vol be set in enqueue_task_fair()
> > and not in dequeue_task_fair()->dur_avg_update()? Otherwise `dur` will
> > contain sleep time.
> > 
> > Like we do for se->prev_sum_exec_runtime in set_next_entity() but for
> > one `set_next_entity()-put_prev_entity()` run section.
> > 
> > AFAICS, you want to measure the exec_runtime sum over all run sections
> > between enqueue and dequeue.
> 
> You were thinking of the dynamic PELT window size thread? (which is what
> I had to think of when I looked at this).
> 
> I think we can still do that with this prev_sum_exec_runtime_vol (can't
> say I love the name though).
I agree that this name is not accurate, maybe prev_sleep_sum_exec_runtime?
I'm open to any other name for this : )

Currently I'm checking Prateek's data on Zen3 and Yicong's data on Arm64,
and their data suggested that: inhibiting the spreading of short wakee is not
always a good idea on a system with small LLC. Meanwhile, according to my
test on a system with large number of CPUs in 1 LLC, short duration wakee become
a trouble maker if spreading them on different CPUs, which could trigger unexpected
race condition. I'm thinking of taking nr_llc_cpu into consideration when defining
a short duration task, and do some experiment on this.

thanks,
Chenyu