All of lore.kernel.org
 help / color / mirror / Atom feed
From: Aubrey Li <aubrey.li@linux.intel.com>
To: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Mel Gorman <mgorman@techsingularity.net>,
	Rik van Riel <riel@surriel.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Valentin Schneider <valentin.schneider@arm.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Gautham R Shenoy <ego@linux.vnet.ibm.com>,
	Parth Shah <parth@linux.ibm.com>
Subject: Re: [PATCH v2 6/8] sched/idle: Move busy_cpu accounting to idle callback
Date: Fri, 14 May 2021 12:11:50 +0800	[thread overview]
Message-ID: <5823f298-6fae-5a73-3ab8-f708d90a7e52@linux.intel.com> (raw)
In-Reply-To: <20210513073112.GV2633526@linux.vnet.ibm.com>

On 5/13/21 3:31 PM, Srikar Dronamraju wrote:
> * Aubrey Li <aubrey.li@linux.intel.com> [2021-05-12 16:08:24]:
> 
>> On 5/7/21 12:45 AM, Srikar Dronamraju wrote:
>>> Currently we account nr_busy_cpus in no_hz idle functions.
>>> There is no reason why nr_busy_cpus should updated be in NO_HZ_COMMON
>>> configs only. Also scheduler can mark a CPU as non-busy as soon as an
>>> idle class task starts to run. Scheduler can then mark a CPU as busy
>>> as soon as its woken up from idle or a new task is placed on it's
>>> runqueue.
>>
>> IIRC, we discussed this before, if a SCHED_IDLE task is placed on the
>> CPU's runqueue, this CPU should be still taken as a wakeup target.
>>
> 
> Yes, this CPU is still a wakeup target, its only when this CPU is busy, that
> we look at other CPUs
> 
>> Also, for those frequent context-switching tasks with very short idle,
>> it's expensive for scheduler to mark idle/busy every time, that's why
>> my patch only marks idle every time and marks busy ratelimited in
>> scheduler tick.
>>
> 
> I have tried few tasks with very short idle times and updating nr_busy
> everytime, doesnt seem to be impacting. Infact, it seems to help in picking
> the idler-llc more often.
> 

How many CPUs in your LLC?

This is a system with 192 CPUs, 4 nodes and each node has 48 CPUs in LLC
domain.

It looks like for netperf both TCP and UDP cases have the notable change
under 2 x overcommit, it may be not interesting though.


hackbench(48 tasks per group)
=========
case            	load    	baseline(std%)	compare%( std%)
process-pipe    	group-1 	 1.00 (  6.74)	 -4.61 (  8.97)
process-pipe    	group-2 	 1.00 ( 36.84)	+11.53 ( 26.35)
process-pipe    	group-3 	 1.00 ( 24.97)	+12.21 ( 19.05)
process-pipe    	group-4 	 1.00 ( 18.27)	 -2.62 ( 17.60)
process-pipe    	group-8 	 1.00 (  4.33)	 -2.22 (  3.08)
process-sockets 	group-1 	 1.00 (  7.88)	-20.26 ( 15.97)
process-sockets 	group-2 	 1.00 (  5.38)	-19.41 (  9.25)
process-sockets 	group-3 	 1.00 (  4.22)	 -5.70 (  3.00)
process-sockets 	group-4 	 1.00 (  1.44)	 -1.80 (  0.79)
process-sockets 	group-8 	 1.00 (  0.44)	 -2.86 (  0.06)
threads-pipe    	group-1 	 1.00 (  5.43)	 -3.69 (  3.59)
threads-pipe    	group-2 	 1.00 ( 18.00)	 -2.69 ( 16.79)
threads-pipe    	group-3 	 1.00 ( 21.72)	 -9.01 ( 21.34)
threads-pipe    	group-4 	 1.00 ( 21.58)	 -6.43 ( 16.26)
threads-pipe    	group-8 	 1.00 (  3.05)	 -0.15 (  2.31)
threads-sockets 	group-1 	 1.00 ( 14.51)	 -5.35 ( 13.85)
threads-sockets 	group-2 	 1.00 (  3.97)	-24.15 (  4.40)
threads-sockets 	group-3 	 1.00 (  4.97)	 -9.05 (  2.46)
threads-sockets 	group-4 	 1.00 (  1.98)	 -3.44 (  0.49)
threads-sockets 	group-8 	 1.00 (  0.37)	 -2.13 (  0.20)

netperf
=======
case            	load    	baseline(std%)	compare%( std%)
TCP_RR          	thread-48	 1.00 (  3.84)	 -2.20 (  3.83)
TCP_RR          	thread-96	 1.00 (  5.22)	 -4.97 (  3.90)
TCP_RR          	thread-144	 1.00 (  7.97)	 -0.75 (  4.39)
TCP_RR          	thread-192	 1.00 (  3.03)	 -0.67 (  4.40)
TCP_RR          	thread-384	 1.00 ( 22.27)	-14.15 ( 36.28)
UDP_RR          	thread-48	 1.00 (  2.08)	 -0.39 (  2.29)
UDP_RR          	thread-96	 1.00 (  2.48)	 -4.26 ( 16.06)
UDP_RR          	thread-144	 1.00 ( 49.50)	 -3.28 ( 34.86)
UDP_RR          	thread-192	 1.00 (  6.39)	 +8.07 ( 88.15)
UDP_RR          	thread-384	 1.00 ( 31.54)	-12.76 ( 35.98)

  reply	other threads:[~2021-05-14  4:12 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-06 16:45 [PATCH v2 0/8] sched/fair: wake_affine improvements Srikar Dronamraju
2021-05-06 16:45 ` [PATCH v2 1/8] sched/fair: Update affine statistics when needed Srikar Dronamraju
2021-05-07 16:08   ` Valentin Schneider
2021-05-07 17:05     ` Srikar Dronamraju
2021-05-11 11:51       ` Valentin Schneider
2021-05-11 16:22         ` Srikar Dronamraju
2021-05-06 16:45 ` [PATCH v2 2/8] sched/fair: Maintain the identity of idle-core Srikar Dronamraju
2021-05-11 11:51   ` Valentin Schneider
2021-05-11 16:27     ` Srikar Dronamraju
2021-05-06 16:45 ` [PATCH v2 3/8] sched/fair: Update idle-core more often Srikar Dronamraju
2021-05-06 16:45 ` [PATCH v2 4/8] sched/fair: Prefer idle CPU to cache affinity Srikar Dronamraju
2021-05-06 16:45 ` [PATCH v2 5/8] sched/fair: Use affine_idler_llc for wakeups across LLC Srikar Dronamraju
2021-05-06 16:45 ` [PATCH v2 6/8] sched/idle: Move busy_cpu accounting to idle callback Srikar Dronamraju
2021-05-11 11:51   ` Valentin Schneider
2021-05-11 16:55     ` Srikar Dronamraju
2021-05-12  0:32     ` Aubrey Li
2021-05-12  8:08   ` Aubrey Li
2021-05-13  7:31     ` Srikar Dronamraju
2021-05-14  4:11       ` Aubrey Li [this message]
2021-05-17 10:40         ` Srikar Dronamraju
2021-05-17 12:48           ` Aubrey Li
2021-05-17 12:57             ` Srikar Dronamraju
2021-05-18  0:59               ` Aubrey Li
2021-05-18  4:00                 ` Srikar Dronamraju
2021-05-18  6:05                   ` Aubrey Li
2021-05-18  7:18                     ` Srikar Dronamraju
2021-05-19  9:43                       ` Aubrey Li
2021-05-19 17:34                         ` Srikar Dronamraju
2021-05-06 16:45 ` [PATCH v2 7/8] sched/fair: Remove ifdefs in waker_affine_idler_llc Srikar Dronamraju
2021-05-06 16:45 ` [PATCH v2 8/8] sched/fair: Dont iterate if no idle CPUs Srikar Dronamraju
2021-05-06 16:53 ` [PATCH v2 0/8] sched/fair: wake_affine improvements Srikar Dronamraju

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5823f298-6fae-5a73-3ab8-f708d90a7e52@linux.intel.com \
    --to=aubrey.li@linux.intel.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=ego@linux.vnet.ibm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@techsingularity.net \
    --cc=mingo@kernel.org \
    --cc=parth@linux.ibm.com \
    --cc=peterz@infradead.org \
    --cc=riel@surriel.com \
    --cc=srikar@linux.vnet.ibm.com \
    --cc=tglx@linutronix.de \
    --cc=valentin.schneider@arm.com \
    --cc=vincent.guittot@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.