From: Srikar Dronamraju <email@example.com> To: Michal Hocko <firstname.lastname@example.org> Cc: Ingo Molnar <email@example.com>, Peter Zijlstra <firstname.lastname@example.org>, LKML <email@example.com>, Mel Gorman <firstname.lastname@example.org>, Rik van Riel <email@example.com> Subject: Re: [PATCH] sched: Fix numabalancing to work with isolated cpus Date: Wed, 5 Apr 2017 20:52:15 +0530 [thread overview] Message-ID: <20170405152215.GA6019@linux.vnet.ibm.com> (raw) In-Reply-To: <20170405125743.GB7258@dhcp22.suse.cz> * Michal Hocko <firstname.lastname@example.org> [2017-04-05 14:57:43]: > On Tue 04-04-17 22:57:28, Srikar Dronamraju wrote: > [...] > > For example: > > perf bench numa mem --no-data_rand_walk -p 4 -t $THREADS -G 0 -P 3072 -T 0 -l 50 -c -s 1000 > > would call sched_setaffinity that resets the cpus_allowed mask. > > > > Cpus_allowed_list: 0-55,57-63,65-71,73-79,81-87,89-175 > > Cpus_allowed_list: 0,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128,136,144,152,160,168 > > Cpus_allowed_list: 0,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128,136,144,152,160,168 > > Cpus_allowed_list: 0,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128,136,144,152,160,168 > > Cpus_allowed_list: 0,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128,136,144,152,160,168 > > > > The isolated cpus are part of the cpus allowed list. In the above case, > > numabalancing ends up scheduling some of these tasks on isolated cpus. > > Why is this bad? If the task is allowed to run on isolated CPUs then why 1. kernel-parameters.txt states: isolcpus as "Isolate CPUs from the general scheduler." So the expectation that numabalancing can schedule tasks on it is wrong. 2. If numabalancing was disabled, the task would never run on the isolated CPUs. 3. With the faulty behaviour, it was observed that tasks scheduled on the isolated cpus might end up taking more time, because they never get a chance to move back to a node which has local memory. 4. The isolated cpus may be idle at that point, but actual work may be scheduled on isolcpus later (when numabalancing had already scheduled work on to it.) Since scheduler doesnt do any balancing on isolcpus even if they are overloaded and the system is completely free, the isolcpus stay overloaded. > shouldn't its numa balancing be allowed the same? The changelog > describes what but doesn't explain _why_ this change is needed/useful.
next prev parent reply other threads:[~2017-04-05 15:22 UTC|newest] Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-04-04 17:27 Srikar Dronamraju 2017-04-04 18:56 ` Rik van Riel 2017-04-04 20:37 ` Mel Gorman 2017-04-05 1:50 ` Srikar Dronamraju 2017-04-05 8:09 ` Mel Gorman 2017-04-05 12:57 ` Michal Hocko 2017-04-05 15:22 ` Srikar Dronamraju [this message] 2017-04-05 16:44 ` Michal Hocko 2017-04-06 7:19 ` Srikar Dronamraju 2017-04-06 7:34 ` Michal Hocko 2017-04-06 9:23 ` Peter Zijlstra 2017-04-06 10:13 ` Michal Hocko 2017-04-06 10:29 ` Peter Zijlstra 2017-04-06 10:42 ` Michal Hocko 2017-04-06 10:47 ` Peter Zijlstra 2017-04-06 13:44 ` Michal Hocko 2017-04-06 7:36 ` Mike Galbraith 2017-04-06 7:36 ` Peter Zijlstra
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20170405152215.GA6019@linux.vnet.ibm.com \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [PATCH] sched: Fix numabalancing to work with isolated cpus' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.