Re: [PATCH v6 0/2] sched/fair: Wake short task on current CPU

From: Chen Yu <yu.c.chen@intel.com>
To: Yicong Yang <yangyicong@huawei.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Juri Lelli <juri.lelli@redhat.com>, <yangyicong@hisilicon.com>,
	Mel Gorman <mgorman@techsingularity.net>,
	Tim Chen <tim.c.chen@intel.com>,
	"Dietmar Eggemann" <dietmar.eggemann@arm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>,
	K Prateek Nayak <kprateek.nayak@amd.com>,
	Abel Wu <wuyun.abel@bytedance.com>,
	"Gautham R . Shenoy" <gautham.shenoy@amd.com>,
	Honglei Wang <wanghonglei@didichuxing.com>,
	"Len Brown" <len.brown@intel.com>,
	Chen Yu <yu.chen.surf@gmail.com>,
	Tianchen Ding <dtcccc@linux.alibaba.com>,
	Joel Fernandes <joel@joelfernandes.org>,
	Josh Don <joshdon@google.com>, Hillf Danton <hdanton@sina.com>,
	kernel test robot <yujie.liu@intel.com>,
	Arjan Van De Ven <arjan.van.de.ven@intel.com>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v6 0/2] sched/fair: Wake short task on current CPU
Date: Thu, 16 Mar 2023 19:13:15 +0800	[thread overview]
Message-ID: <ZBL5y9gCJxvI4HW6@chenyu5-mobl1> (raw)
In-Reply-To: <718acf0e-2c98-cff3-98f4-da43d49f776b@huawei.com>

On 2023-03-15 at 17:34:43 +0800, Yicong Yang wrote:
> Hi Chenyu,
> 
> On 2023/2/22 22:09, Chen Yu wrote:
> > The main purpose is to avoid too many cross CPU wake up when it is
> > unnecessary. The frequent cross CPU wake up brings significant damage
> > to some workloads, especially on high core count systems.
> > 
> > Inhibits the cross CPU wake-up by placing the wakee on waking CPU,
> > if both the waker and wakee are short-duration tasks. The short
> > duration task could become a trouble maker on high-load system,
> > because it could bring frequent context switch. This strategy
> > only takes effect when the system is busy. Because it is unreasonable
> > to inhibit the idle CPU scan when there are still idle CPUs.
> > 
> > First, introduce the definition of a short-duration task. Then
> > leverages the first patch to choose a local CPU for wakee.
> > 
> > Overall there is performance improvement on some overloaded case.
> > Such as will-it-scale, netperf. And no noticeable impact on
> > schbench, hackbench, tbench and a OLTP workload with a commercial
> > RDBMS, tested on a Intel Xeon 2 x 56C machine.
> > 
> > Per the test on Zen3 from Prateek, most benchmarks result saw small
> > wins or are comparable to sched:tip. SpecJBB Critical-jOps improved while
> > Max-jOPS saw a small hit, but it might be in the expected range.
> > ycsb-mongodb saw small uplift in NPS1 mode.
> > 
> > Throughput improvement of netperf(localhost) was observed on a
> > Rome 2 x 64C machine, when the number of clients equals the CPUs.
> > 
> > Abel reported against a latency regression from Redis on an overloaded
> > system. Inspired by his description, v5 added the check of wakee_flips
> > to mitigate task stacking.
> > 
> > Changes since v5:
> > 1. Check the wakee_flips of the waker/wakee. If the wakee_flips
> >    of waker/wakee are both 0, it indicates that the waker and the wakee
> >    are waking up each other. In this case, put them together on the
> >    same CPU. This is to avoid that too many wakees are stacked on
> >    one CPU, which might cause regression on redis.
> > 
> 
> The patch looks good to me. And for the v6 version there's no significant
> regression on our server. :)
> 
> Detailed results below. The setup are the same as what used on v4. There're
> some gain for UDP_RR. For mysql no significant regression, there're ~2%
> loss for 128 threads case but the proportion is within the fluctuation
> range so it should be ok.
>
Thanks Yicong for the test!

thanks,
Chenyu