From: Erich Focht <efocht@ess.nec.de>
To: Ingo Molnar <mingo@elte.hu>
Cc: Michael Hohnbaum <hohnbaum@us.ibm.com>,
"Martin J. Bligh" <mbligh@aracnet.com>,
Matthew Dobson <colpatch@us.ibm.com>,
Christoph Hellwig <hch@infradead.org>,
Robert Love <rml@tech9.net>, Andrew Theurer <habanero@us.ibm.com>,
Linus Torvalds <torvalds@transmeta.com>,
linux-kernel <linux-kernel@vger.kernel.org>,
lse-tech <lse-tech@lists.sourceforge.net>
Subject: Re: [patch] sched-2.5.59-A2
Date: Mon, 20 Jan 2003 13:07:55 +0100 [thread overview]
Message-ID: <200301201307.55515.efocht@ess.nec.de> (raw)
In-Reply-To: <Pine.LNX.4.44.0301201022540.2585-100000@localhost.localdomain>
[-- Attachment #1: Type: text/plain, Size: 1892 bytes --]
On Monday 20 January 2003 10:28, Ingo Molnar wrote:
> On Sun, 19 Jan 2003, Erich Focht wrote:
> > The results:
> > - kernbench UserTime is best for the 2.5.59 scheduler (623s). IngoB0
> > best value 627.33s for idle=20ms, busy=2000ms.
> > - hackbench: 2.5.59 scheduler is significantly better for all
> > measurements.
> >
> > I suppose this comes from the fact that the 2.5.59 version has the
> > chance to load_balance across nodes when a cpu goes idle. No idea what
> > other reason it could be... Maybe anybody else?
>
> this shows that agressive idle-rebalancing is the most important factor. I
> think this means that the unification of the balancing code should go into
> the other direction: ie. applying the ->nr_balanced logic to the SMP
> balancer as well.
Could you please explain your idea? As far as I understand, the SMP
balancer (pre-NUMA) tries a global rebalance at each call. Maybe you
mean something different...
> kernelbench is the kind of benchmark that is most sensitive to over-eager
> global balancing, and since the 2.5.59 ->nr_balanced logic produced the
> best results, it clearly shows it's not over-eager. hackbench is one that
> is quite sensitive to under-balancing. Ie. trying to maximize both will
> lead us to a good balance.
Yes! Actually the currently implemented nr_balanced logic is pretty
dumb: the counter reaches the cross-node balance threshold after a
certain number of calls to intra-node lb, no matter whether these were
successfull or not. I'd like to try incrementing the counter only on
unsuccessfull load balances, this would give a clear priority to
intra-node balancing and a clear and controllable delay for cross-node
balancing. A tiny patch for this (for 2.5.59) is attached. As the name
nr_balanced would be misleading for this kind of usage, I renamed it
to nr_lb_failed.
Regards,
Erich
[-- Attachment #2: failed-lb-ctr-2.5.59 --]
[-- Type: text/x-diff, Size: 1243 bytes --]
diff -urNp 2.5.59-topo/kernel/sched.c 2.5.59-topo-succ/kernel/sched.c
--- 2.5.59-topo/kernel/sched.c 2003-01-17 03:22:29.000000000 +0100
+++ 2.5.59-topo-succ/kernel/sched.c 2003-01-20 13:06:04.000000000 +0100
@@ -156,7 +156,7 @@ struct runqueue {
int prev_nr_running[NR_CPUS];
#ifdef CONFIG_NUMA
atomic_t *node_nr_running;
- unsigned int nr_balanced;
+ unsigned int nr_lb_failed;
int prev_node_load[MAX_NUMNODES];
#endif
task_t *migration_thread;
@@ -770,11 +770,11 @@ static inline unsigned long cpus_to_bala
int this_node = __cpu_to_node(this_cpu);
/*
* Avoid rebalancing between nodes too often.
- * We rebalance globally once every NODE_BALANCE_RATE load balances.
+ * We rebalance globally after NODE_BALANCE_RATE failed load balances.
*/
- if (++(this_rq->nr_balanced) == NODE_BALANCE_RATE) {
+ if (this_rq->nr_lb_failed == NODE_BALANCE_RATE) {
int node = find_busiest_node(this_node);
- this_rq->nr_balanced = 0;
+ this_rq->nr_lb_failed = 0;
if (node >= 0)
return (__node_to_cpu_mask(node) | (1UL << this_cpu));
}
@@ -892,6 +892,10 @@ static inline runqueue_t *find_busiest_q
busiest = NULL;
}
out:
+#ifdef CONFIG_NUMA
+ if (!busiest)
+ this_rq->nr_lb_failed++;
+#endif
return busiest;
}
next prev parent reply other threads:[~2003-01-20 11:58 UTC|newest]
Thread overview: 96+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-01-09 23:54 Minature NUMA scheduler Martin J. Bligh
2003-01-10 5:36 ` [Lse-tech] " Michael Hohnbaum
2003-01-10 16:34 ` Erich Focht
2003-01-10 16:57 ` Martin J. Bligh
2003-01-12 23:35 ` Erich Focht
2003-01-12 23:55 ` NUMA scheduler 2nd approach Erich Focht
2003-01-13 8:02 ` Christoph Hellwig
2003-01-13 11:32 ` Erich Focht
2003-01-13 15:26 ` [Lse-tech] " Christoph Hellwig
2003-01-13 15:46 ` Erich Focht
2003-01-13 19:03 ` Michael Hohnbaum
2003-01-14 1:23 ` Michael Hohnbaum
2003-01-14 4:45 ` [Lse-tech] " Andrew Theurer
2003-01-14 4:56 ` Martin J. Bligh
2003-01-14 11:14 ` Erich Focht
2003-01-14 15:55 ` [PATCH 2.5.58] new NUMA scheduler Erich Focht
2003-01-14 16:07 ` [Lse-tech] " Christoph Hellwig
2003-01-14 16:23 ` [PATCH 2.5.58] new NUMA scheduler: fix Erich Focht
2003-01-14 16:43 ` Erich Focht
2003-01-14 19:02 ` Michael Hohnbaum
2003-01-14 21:56 ` [Lse-tech] " Michael Hohnbaum
2003-01-15 15:10 ` Erich Focht
2003-01-16 0:14 ` Michael Hohnbaum
2003-01-16 6:05 ` Martin J. Bligh
2003-01-16 16:47 ` Erich Focht
2003-01-16 18:07 ` Robert Love
2003-01-16 18:48 ` Martin J. Bligh
2003-01-16 19:07 ` Ingo Molnar
2003-01-16 18:59 ` Martin J. Bligh
2003-01-16 19:10 ` Christoph Hellwig
2003-01-16 19:44 ` Ingo Molnar
2003-01-16 19:43 ` Martin J. Bligh
2003-01-16 20:19 ` Ingo Molnar
2003-01-16 20:29 ` [Lse-tech] " Rick Lindsley
2003-01-16 23:31 ` Martin J. Bligh
2003-01-17 7:23 ` Ingo Molnar
2003-01-17 8:47 ` [patch] sched-2.5.59-A2 Ingo Molnar
2003-01-17 14:35 ` Erich Focht
2003-01-17 15:11 ` Ingo Molnar
2003-01-17 15:30 ` Erich Focht
2003-01-17 16:58 ` Martin J. Bligh
2003-01-18 20:54 ` NUMA sched -> pooling scheduler (inc HT) Martin J. Bligh
2003-01-18 21:34 ` [Lse-tech] " Martin J. Bligh
2003-01-19 0:13 ` Andrew Theurer
2003-01-17 18:19 ` [patch] sched-2.5.59-A2 Michael Hohnbaum
2003-01-18 7:08 ` William Lee Irwin III
2003-01-18 8:12 ` Martin J. Bligh
2003-01-18 8:16 ` William Lee Irwin III
2003-01-19 4:22 ` William Lee Irwin III
2003-01-17 17:21 ` Martin J. Bligh
2003-01-17 17:23 ` Martin J. Bligh
2003-01-17 18:11 ` Erich Focht
2003-01-17 19:04 ` Martin J. Bligh
2003-01-17 19:26 ` [Lse-tech] " Martin J. Bligh
2003-01-18 0:13 ` Michael Hohnbaum
2003-01-18 13:31 ` [patch] tunable rebalance rates for sched-2.5.59-B0 Erich Focht
2003-01-18 23:09 ` [patch] sched-2.5.59-A2 Erich Focht
2003-01-20 9:28 ` Ingo Molnar
2003-01-20 12:07 ` Erich Focht [this message]
2003-01-20 16:56 ` Ingo Molnar
2003-01-20 17:04 ` Ingo Molnar
2003-01-20 17:10 ` Martin J. Bligh
2003-01-20 17:24 ` Ingo Molnar
2003-01-20 19:13 ` Andrew Theurer
2003-01-20 19:33 ` Martin J. Bligh
2003-01-20 19:52 ` Andrew Theurer
2003-01-20 19:52 ` Martin J. Bligh
2003-01-20 21:18 ` [patch] HT scheduler, sched-2.5.59-D7 Ingo Molnar
2003-01-20 22:28 ` Andrew Morton
2003-01-21 1:11 ` Michael Hohnbaum
2003-01-22 3:15 ` Michael Hohnbaum
2003-01-22 16:41 ` Andrew Theurer
2003-01-22 16:17 ` Martin J. Bligh
2003-01-22 16:20 ` Andrew Theurer
2003-01-22 16:35 ` Michael Hohnbaum
2003-02-03 18:23 ` [patch] HT scheduler, sched-2.5.59-E2 Ingo Molnar
2003-02-03 20:47 ` Robert Love
2003-02-04 9:31 ` Erich Focht
2003-01-20 17:04 ` [patch] sched-2.5.59-A2 Martin J. Bligh
2003-01-21 17:44 ` Erich Focht
2003-01-20 16:23 ` Martin J. Bligh
2003-01-20 16:59 ` Ingo Molnar
2003-01-17 23:09 ` Matthew Dobson
2003-01-16 23:45 ` [PATCH 2.5.58] new NUMA scheduler: fix Michael Hohnbaum
2003-01-17 11:10 ` Erich Focht
2003-01-17 14:07 ` Ingo Molnar
2003-01-16 19:44 ` John Bradford
2003-01-14 16:51 ` Christoph Hellwig
2003-01-15 0:05 ` Michael Hohnbaum
2003-01-15 7:47 ` Martin J. Bligh
2003-01-14 5:50 ` [Lse-tech] Re: NUMA scheduler 2nd approach Michael Hohnbaum
2003-01-14 16:52 ` Andrew Theurer
2003-01-14 15:13 ` Erich Focht
2003-01-14 10:56 ` Erich Focht
2003-01-11 14:43 ` [Lse-tech] Minature NUMA scheduler Bill Davidsen
2003-01-12 23:24 ` Erich Focht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=200301201307.55515.efocht@ess.nec.de \
--to=efocht@ess.nec.de \
--cc=colpatch@us.ibm.com \
--cc=habanero@us.ibm.com \
--cc=hch@infradead.org \
--cc=hohnbaum@us.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lse-tech@lists.sourceforge.net \
--cc=mbligh@aracnet.com \
--cc=mingo@elte.hu \
--cc=rml@tech9.net \
--cc=torvalds@transmeta.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).