linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Re: [PATCH] Minor scheduler fix to get rid of skipping in xmms
@ 2003-09-08 22:27 Steven Pratt
  2003-09-08 22:56 ` Andrew Morton
  0 siblings, 1 reply; 58+ messages in thread
From: Steven Pratt @ 2003-09-08 22:27 UTC (permalink / raw)
  To: Andrew Morton, linux-kernel

Andrew Morton wrote:

>That is not clear at this time.  We do know that the reaim regression was
>introduced by sched-2.6.0-test2-mm2-A3, but we don't know why.  Certainly
>that patch did not introduce the problem which Andrew's patch fixed.  And
>we have theorised that Andrew's patch brought back the reaim throughput. 
>And we have extrapolated those observations to possible improvements in
>volanomark throughput.
>
>It's all foggy and I'd like to see a clean rerun of specjbb and volanomark
>by Mark Peloquin and co, confirming that -mm6 is performing OK.
>  
>
For specjbb things are looking good from a throughput point of view. 



  
       2.6.0-test4 2.6.0-test4-mm6
  # of WHs      OPs/sec      OPs/sec    %diff         diff    tolerance
---------- ------------ ------------ -------- ------------ ------------
         1      9783.46     10093.09     3.16       309.63       293.50  *
         4     33783.93     35763.79     5.86      1979.86      1013.52  *
         7     54401.52     54288.06    -0.21      -113.46      1632.05
        10     56861.59     56445.20    -0.73      -416.39      1705.85
        13     56024.86     55720.23    -0.54      -304.63      1680.75
        16     43874.77     48994.63    11.67      5119.86      1316.24  *
        19     32658.83     31248.04    -4.32     -1410.79       979.76  *

But to get these numbers we are using much more CPU. I'll leave it to 
others to decide if this is good or not.

CPU IDLE TIME
           2.6.0-test4 2.6.0-test4-mm6
  # of WHs         %CPU         %CPU    %diff         diff    tolerance
---------- ------------ ------------ -------- ------------ ------------
         1        87.30        87.31     0.01         0.01         3.62
         4        49.53        49.51    -0.04        -0.02         2.49
         7        12.40        12.32    -0.65        -0.08         1.37
        10         0.36         0.40    11.11         0.04         1.01
        13         1.20         0.62   -48.33        -0.58         1.04
        16        15.17         2.79   -81.61       -12.38         1.46  *
        19        30.66         5.29   -82.75       -25.37         1.92  *


Volanomark, on the other hand is still off by quite a bit from test4 stock

Results:Throughput
 
                                tolerance = 0.00 + 3.00% of 2.6.0-test4
            2.6.0-test4 2.6.0-test4-mm6
               Msgs/sec     Msgs/sec    %diff         diff    tolerance
---------- ------------ ------------ -------- ------------ ------------
         1        40757        37223    -8.67     -3534.00      1222.71  *
 

>
>Also, I'm concerned that sched-2.6.0-test2-mm2-A3 caused slowdowns and
>Andrew's patch caused speedups and they just cancelled out.  Let's get
>Andrew's patch into Linus's tree and see if it speeds things up.  If it
>does, we probably still have a problem.
>

If thre is any particular patch/tree combination you would like me to 
try out, please let me know and I will see if I can get the results for 
you. 

Steve


^ permalink raw reply	[flat|nested] 58+ messages in thread
* Re: [PATCH] Minor scheduler fix to get rid of skipping in xmms
@ 2003-09-11 23:32 Craig Thomas
  0 siblings, 0 replies; 58+ messages in thread
From: Craig Thomas @ 2003-09-11 23:32 UTC (permalink / raw)
  To: linux-kernel; +Cc: piggin, kernel, akpm

At the request of Cliff White, I have run DBT-2 tests agains patches
from Nick Piggin.  Below are some test results running an OLTP
transaction database workload with two of Nick Piggin's patches:

PLM ID 2117 - sched_rollup (2.6.0-test5-sched-rollup)
PLM ID 2119 - sched_rollup_nopolicy (2.6.0-tst5-nick.v14)

I believe, Cliff has run or is running tests on these patches
using the reaim-7 tests.

These were run on OSDL's STP framework and the kernel patches are
archived in PLM.  The database tests were configured to run where
the database was entirely cached in memory and to run where
the database was larger than memory, forcing I/O activit.


Cached Runs:
------------
These tests run database transactions of an OLTP variety.  The
test is set up so that the entire database resides in memory
and thus avoids I/O where possible.  This test is useful for
determining the overall capababilities of the CPU and memory
features.

STP4-000

   Kernel                NOTPM         test id
-----------------------  -----  ---------------------------------
linux-2.6.0-test5         2914  http://khack.osdl.org/stp/279496/
linux-2.6.0-test3         2642  http://khack.osdl.org/stp/279430/
2.6.0-test5-sched-rollup  2822  http://khack.osdl.org/stp/279670/
2.6.0-test5-nick.v14      2839  http://khack.osdl.org/stp/279686/

These results show that Nick's patches are not quite up to the
overall throughput capability of the standard Linus kernel.
However, they are better than the last -mm kernel I was able to
get runs on (2.6.0-test3-mm1), so the changes are heading in the
right direction.  Unfortunately, I could not get more runs for
this report, but I could perform more in order to get an average,
if you'd like.


Non Cached (disk intensive) runs:

---------------------------------
These tests run a larger version of the same database, but because
of its larger size and queries over a larger table, I/O is used
heavily.

These runs were taken on two different machines.  One system is
slightly faster all around than the other.  Thus, the runs are broken
down by system, rather than lumped all together.

STP4-001

   Kernel                NOTPM         test id
-----------------------  -----   ---------------------------------
linux-2.6.0-test5         1185   http://khack.osdl.org/stp/279495/
2.6.0-test5-nick.v14      1187   http://khack.osdl.org/stp/279693/
2.6.0-test5-nick.v14      1226   http://khack.osdl.org/stp/279689/
2.6.0-test5-sched-rollup  1214   http://khack.osdl.org/stp/279691/


stp4-002

   Kernel                NOTPM         test id
-----------------------  -----  --------------------------------
linux-2.6.0-test5         1317  http://khack.osdl.org/stp/279500/
linux-2.6.0-test5         1336  http://khack.osdl.org/stp/279494/
2.6.0-test5-nick.v14      1348  http://khack.osdl.org/stp/279692/
2.6.0-test5-sched-rollup  1329  http://khack.osdl.org/stp/279688/
2.6.0-test5-sched-rollup  1333  http://khack.osdl.org/stp/279690/


It appears that for non-cached runs, where I/O us used, the
numbers start looking the same as the Linus kernel.  This implies
that the patches from Andrew and Nick are not intrusive.  I don't
beliefve the difference in the numbers are significant in these
cases.

So, overall, the scheduler changes of each kernel don't seem to
have an impact on OLTP transaction database processes where I/O
is involved.

The test id URL, point to information about the system resources
(vmstat, sar, etc.) if anybody really wants to dig down into the
details.

-- 
Craig Thomas
craiger@osdl.org


^ permalink raw reply	[flat|nested] 58+ messages in thread
* Re: [PATCH] Minor scheduler fix to get rid of skipping in xmms
@ 2003-09-11  2:55 Andrew Theurer
  2003-09-11 11:04 ` Nick Piggin
  0 siblings, 1 reply; 58+ messages in thread
From: Andrew Theurer @ 2003-09-11  2:55 UTC (permalink / raw)
  To: linux-kernel

Robert Love <rml@tech9.net> wrote:
>
>>  There are a _lot_ of scheduler changes in 2.6-mm, and who knows which
>>  ones are an improvement, a detriment, and a noop?

> We know that sched-2.6.0-test2-mm2-A3.patch caused the regression, and
> we now that sched-CAN_MIGRATE_TASK-fix.patch mostly fixed it up.

> What we don't know is whether the thing which
> sched-CAN_MIGRATE_TASK-fix.patch
> fixed was the thing which sched-2.6.0-test2-mm2-A3.patch broke.

Sorry for jumping into this late.  I didn't even know the can_migrate patch 
was being discussed, let alone in -mm :).  And to be fair, this really is 
Ingo's aggressive idle steal patch.

Anyway, these patches are somewhat related.  It would seem that A3's 
shortening the tasks' run time would not only slow performance beacuse of 
cache thrash, but could possibly break CAN_MIGRATE's cache warmth check, 
right?  That in turn would stop load balancing from working well, leading to 
more idle time, which the CAN_MIGRATE patch sort of bypassed for idle cpus.

I see Nick's balance patch as somewhat harmless, at least combined with A3 
patch.  However, one concern is that the "ping-pong" steal interval is not 
really 200ms, but 200ms/(nr_cpus-1), which without A3, could show up as a 
problem, especially on an 8 way box.  In addition, I do think there's a 
problem with num tasks we steal.  It should not be imbalance/2, it should be:  
max_load - (node_nr_running / num_cpus_node).  If we steal any more than 
this, which is quite possible with imbalance/2, then it's likely this_cpu now 
has too many tasks, and some other cpu will steal again.  Using *imbalance/2 
works fine on 2-way smp, but I'm pretty sure we "over steal" tasks on 4 way 
and up.  Anyway, I'm getting off topic here...

But Steve's latest results have me toally stumped.  Why would a patch which 
shortens run time and probbaly thrashes cache improve a cpu bound workload 
like JBB?  And why would a patch that makes sure idle cpus don't stay idle 
reduce performance by so much?

Steve, are you absolutely sure your latest results on test5 are correct?  Any 
possibility the original results were the "good" ones?   

FWIW, I have seen the CAN_MIGRATE patch make a huge difference, not just in 
testing, but a -real- enterprise application used in "production".  And 
unlike JBB and Volano, there's no high rate of sched_yield either.  They do 
have a high rate of cswitches, but only because their workload message 
driven.  This patch made a 40% improvement on 4-way on a 2.4 distro kernel 
that has O(1).

-Andrew Theurer






^ permalink raw reply	[flat|nested] 58+ messages in thread
[parent not found: <tCPY.4xU.1@gated-at.bofh.it>]
* Re: [PATCH] Minor scheduler fix to get rid of skipping in xmms
@ 2003-09-06 15:58 John Yau
  2003-09-06 16:57 ` Michael Buesch
  0 siblings, 1 reply; 58+ messages in thread
From: John Yau @ 2003-09-06 15:58 UTC (permalink / raw)
  To: mbuesch; +Cc: linux-kernel

Hi Michael,

I don't know if really a bug due to xmms, I suspect that's the case.  I'm 
not familiar with xmms internals, but when I gdb'ed the process after it 
froze, all the threads either stopped at poll(), write(), select(), or 
nanosleep().  Some combination of the blocking calls among those is probably 
causing the stall.  I highly doubt it's due to the kernel since I haven't 
been experiencing hangs in any other applications.  It could be the socket 
code though if extensive modifications to it have been made, since I've 
never experienced hangs like this in the 2.4.18 kernel used by RedHat 8.0.


John Yau

>From: Michael Buesch <mbuesch@freenet.de>
>To: "John Yau" <jyau_kernel_dev@hotmail.com>
>CC: linux kernel mailing list <linux-kernel@vger.kernel.org>
>Subject: Re: [PATCH] Minor scheduler fix to get rid of skipping in xmms
>Date: Sat, 6 Sep 2003 12:03:35 +0200
>
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On Saturday 06 September 2003 11:46, John Yau wrote:
> > Hi folks,
>
>Hi John,
>
> > xmms still completely hangs every once in a while for me.  However I
> > suspect it's due to a bug in xmms that deadlocks.  Anyone else 
>experiencing
> > hangs with xmms while tuning into Shoutcast???
>
>Yes, that's (was?) a bug of xmms.
>
>- --
>Regards Michael Buesch  [ http://www.tuxsoft.de.vu ]
>Animals on this machine: some GNUs and Penguin 2.6.0-test4-bk2
>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.2.2 (GNU/Linux)
>
>iD8DBQE/WbD/oxoigfggmSgRAs5qAJ99vZeNeMEXhl72VvVlGFMWh55HVgCeLK0R
>MgWcMSUSdEYL+OeehfDNBCc=
>=TdCG
>-----END PGP SIGNATURE-----
>

_________________________________________________________________
Get 10MB of e-mail storage! Sign up for Hotmail Extra Storage.  
http://join.msn.com/?PAGE=features/es


^ permalink raw reply	[flat|nested] 58+ messages in thread
* [PATCH] Minor scheduler fix to get rid of skipping in xmms
@ 2003-09-06  9:46 John Yau
  2003-09-06 10:03 ` Michael Buesch
  2003-09-06 17:01 ` Robert Love
  0 siblings, 2 replies; 58+ messages in thread
From: John Yau @ 2003-09-06  9:46 UTC (permalink / raw)
  To: linux-kernel

Hi folks,

I'm new to patch submission process, so bear with me.  This little patch I 
wrote seems to get rid of the annoying skipping in xmms except in the most 
extreme cases.  See comments inlined in code for details of the fix.

xmms still completely hangs every once in a while for me.  However I suspect 
it's due to a bug in xmms that deadlocks.  Anyone else experiencing hangs 
with xmms while tuning into Shoutcast???

Try out this patch and let me know if it fixes things for you.  Other than 
the hangs, xmms is behaving normally now for me.  Please CC me in all 
replies to this thread.


John Yau

--- linux-2.6.0-0.test4.1.32/kernel/sched.c	2003-08-22 19:58:43.000000000 
-0400
+++ linux-2.6.0-custom/kernel/sched.c	2003-09-06 04:20:09.000000000 -0400
@@ -1,4 +1,4 @@
-/*
+ /*
  *  kernel/sched.c
  *
  *  Kernel scheduler and related syscalls
@@ -14,6 +14,9 @@
  *		an array-switch method of distributing timeslices
  *		and per-CPU runqueues.  Cleanups and useful suggestions
  *		by Davide Libenzi, preemptible kernel bits by Robert Love.
+ *  2003-9-6    Changed recalculation of effective priority from being
+ *              calculated at exhaustion of time slices or sleep to
+ *              every use of 20 ms on the CPU by John Yau (jyau)
  */

#include <linux/mm.h>
@@ -1183,7 +1186,11 @@
		(STARVATION_LIMIT && ((rq)->expired_timestamp && \
		(jiffies - (rq)->expired_timestamp >= \
			STARVATION_LIMIT * ((rq)->nr_running) + 1)))
-
+/* jyau:
+ * Recalculate priorities every 20 ms that a task has
+ * used up in its time slice
+ */
+#define RECALCULATE_PRIORITY_TICK (HZ/50 ?: 1)
/*
  * This function gets called by the timer code, with HZ frequency.
  * We call it with interrupts disabled.
@@ -1237,6 +1244,26 @@
	 * goes to sleep or uses up its timeslice. This makes
	 * it possible for interactive tasks to use up their
	 * timeslices at their highest priority levels.
+	 *
+	 * jyau:
+	 *   Updating priority later is a bad idea.  Tasks like
+	 *   xmms will occasionally not get rescheduled quickly enough
+	 *   because they are penalized for using CPU quite a bit
+	 *   more heavily than occassional clicks in e.g. Mozilla,
+	 *   which should be considered a CPU hog when rendering
+	 *   webpages after a click because it will tend to completely
+	 *   use up its time slice.
+	 *
+	 *   I suspect xmms gets booted out of interactive status
+	 *   quite easily and thus have to wait on the order of
+	 *   1000+ ms to get rescheduled in some circumstances,
+	 *   especially if a couple interactive tasks are in line
+	 *   before it.
+	 *
+	 *   We fix this by updating priorities periodically every
+	 *   RECALCULATE_PRIORITY_TICKs that the process has
+	 *   been on the CPU. RECALCULATE_PRIORITY_TICKs is currently
+	 *   set to 20 ms.
	 */
	if (p->sleep_avg)
		p->sleep_avg--;
@@ -1258,11 +1285,12 @@
	}
	if (!--p->time_slice) {
		dequeue_task(p, rq->active);
+
		set_tsk_need_resched(p);
		p->prio = effective_prio(p);
		p->time_slice = task_timeslice(p);
		p->first_time_slice = 0;
-
+
		if (!TASK_INTERACTIVE(p) || EXPIRED_STARVING(rq)) {
			if (!rq->expired_timestamp)
				rq->expired_timestamp = jiffies;
@@ -1270,6 +1298,26 @@
		} else
			enqueue_task(p, rq->active);
	}
+	else if (!(p->time_slice % RECALCULATE_PRIORITY_TICK)) {
+	        int prio;
+                prio = effective_prio(p);
+		if (p->prio != prio)
+		  {
+		    /* jyau:
+		     * The priority changed, alter the queue
+		     * to reflect the change and ask the
+		     * scheduler to find the next appropriate
+		     * process if the priority was demoted.
+		     */
+		    dequeue_task(p, rq->active);
+		    if (prio > p->prio)
+		      {
+			set_tsk_need_resched(p);
+		      }
+		    p->prio = prio;
+		    enqueue_task(p, rq->active);
+		  }
+        }
out_unlock:
	spin_unlock(&rq->lock);
out:

_________________________________________________________________
Try MSN Messenger 6.0 with integrated webcam functionality! 
http://www.msnmessenger-download.com/tracking/reach_webcam


^ permalink raw reply	[flat|nested] 58+ messages in thread
end of thread, other threads:[~2003-09-11 23:32 UTC | newest]

Thread overview: 58+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2003-09-08 22:27 [PATCH] Minor scheduler fix to get rid of skipping in xmms Steven Pratt
2003-09-08 22:56 ` Andrew Morton
2003-09-08 23:22   ` William Lee Irwin III
2003-09-09  2:10   ` Con Kolivas
2003-09-09  2:16     ` Con Kolivas
2003-09-09  2:31       ` Con Kolivas
2003-09-09  2:33         ` Andrew Morton
2003-09-09  4:14         ` Nick Piggin
2003-09-09  6:49           ` Con Kolivas
2003-09-09 23:53           ` Cliff White
2003-09-10  2:12             ` Nick Piggin
2003-09-10 19:05               ` Steven Pratt
2003-09-10 20:23                 ` Dave Hansen
     [not found]                 ` <3F5FE385.10204@cyberone.com.au>
     [not found]                   ` <3F607E62.3010903@austin.ibm.com>
     [not found]                     ` <3F60873B.4000005@cyberone.com.au>
2003-09-11 22:57                       ` Steven Pratt
2003-09-11  0:14               ` Cliff White
2003-09-09 22:06   ` Steven Pratt
2003-09-09 22:12     ` Andrew Morton
2003-09-10 13:59       ` Steven Pratt
2003-09-10 18:51   ` Steven Pratt
  -- strict thread matches above, loose matches on Subject: below --
2003-09-11 23:32 Craig Thomas
2003-09-11  2:55 Andrew Theurer
2003-09-11 11:04 ` Nick Piggin
2003-09-11 13:05   ` Andrew Theurer
2003-09-11 13:53     ` Nick Piggin
2003-09-11 14:37       ` Andrew Theurer
     [not found] <tCPY.4xU.1@gated-at.bofh.it>
     [not found] ` <tDsR.5tY.31@gated-at.bofh.it>
     [not found]   ` <tZ0f.49P.5@gated-at.bofh.it>
     [not found]     ` <tZjz.4Bn.7@gated-at.bofh.it>
2003-09-09 23:24       ` David Mosberger-Tang
2003-09-06 15:58 John Yau
2003-09-06 16:57 ` Michael Buesch
2003-09-06  9:46 John Yau
2003-09-06 10:03 ` Michael Buesch
2003-09-06 17:01 ` Robert Love
2003-09-06 17:59   ` John Yau
2003-09-06 18:17   ` John Yau
2003-09-06 19:42     ` Rahul Karnik
2003-09-06 20:04     ` Robert Love
2003-09-06 22:41       ` John Yau
2003-09-07  2:40         ` Martin J. Bligh
2003-09-07  5:13         ` Nick Piggin
2003-09-07  7:48           ` Johnny Yau
2003-09-07  8:10             ` Nick Piggin
2003-09-07  8:35               ` John Yau
2003-09-07  9:26                 ` Nick Piggin
2003-09-07 17:30                   ` John Yau
2003-09-07 17:36                     ` Nick Piggin
2003-09-08  0:22                     ` Con Kolivas
2003-09-08  0:27                       ` David Lang
2003-09-08  0:47                         ` Con Kolivas
2003-09-07  5:08       ` Nick Piggin
2003-09-07  6:18         ` Andrew Morton
2003-09-07  6:29           ` Nick Piggin
2003-09-07  6:45             ` Andrew Morton
2003-09-07  6:59               ` Nick Piggin
2003-09-07  7:02                 ` Nick Piggin
2003-09-07 14:32             ` Martin J. Bligh
2003-09-07 17:02           ` Robert Love
2003-09-07 17:34             ` Andrew Morton
2003-09-07 18:12               ` Nick Piggin
2003-09-07 18:13             ` Nick Piggin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).