* Re: [patch] "HT scheduler", sched-2.5.63-B3 [not found] <5.2.0.9.2.20030307093435.01a8fe88@pop.gmx.net> @ 2003-03-07 9:30 ` Ingo Molnar [not found] ` <Pine.LNX.4.44.0303071003060.6318-100000@localhost.localdom ain> 1 sibling, 0 replies; 21+ messages in thread From: Ingo Molnar @ 2003-03-07 9:30 UTC (permalink / raw) To: Mike Galbraith; +Cc: Andrew Morton, Linus Torvalds, rml, linux-kernel ok, forget the -B0 patch, it's broken - we cannot change the current task's priority without unqueueing it first. -B1 will have the right code. Ingo ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <Pine.LNX.4.44.0303071003060.6318-100000@localhost.localdom ain>]
* Re: [patch] "HT scheduler", sched-2.5.63-B3 [not found] ` <Pine.LNX.4.44.0303071003060.6318-100000@localhost.localdom ain> @ 2003-03-07 9:38 ` Mike Galbraith 2003-03-07 10:03 ` [patch] "interactivity changes", sched-2.5.64-B2 Ingo Molnar [not found] ` <Pine.LNX.4.44.0303071049500.7326-100000@localhost.localdom ain> 0 siblings, 2 replies; 21+ messages in thread From: Mike Galbraith @ 2003-03-07 9:38 UTC (permalink / raw) To: Ingo Molnar; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel At 10:03 AM 3/7/2003 +0100, Ingo Molnar wrote: >here's sched-2.5.64-B0: KABOOM! Major explosions on boot. First time I booted, I got an instant reboot. Second time, I got a couple of double faults and then explosion during fsck (uhoh;). PID is 1684368482 :-/ Time to reboot into .virgin for (hopefully) a couple of fretful moments. (rebooting into .virgin after first explosion would have been a bit smarter;) -Mike ^ permalink raw reply [flat|nested] 21+ messages in thread
* [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 9:38 ` Mike Galbraith @ 2003-03-07 10:03 ` Ingo Molnar 2003-03-07 14:45 ` Ingo Molnar [not found] ` <Pine.LNX.4.44.0303071543480.12493-100000@localhost.localdo main> [not found] ` <Pine.LNX.4.44.0303071049500.7326-100000@localhost.localdom ain> 1 sibling, 2 replies; 21+ messages in thread From: Ingo Molnar @ 2003-03-07 10:03 UTC (permalink / raw) To: Mike Galbraith; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel i've attached the -B2 patch (against vanilla 2.5.64), which, with its default settings, should be equivalent to what -B0 was supposed to be. (ie. the only change is that the priority backboost is immediately propagated into current's priority.) -B2 also has all scheduler tunables exported into /proc/sys/kernel/, for testing/development purposes only. The following values are now tunable: MIN_TIMESLICE, MAX_TIMESLICE, CHILD_PENALTY, PARENT_PENALTY, EXIT_WEIGHT, PRIO_BONUS_RATIO, PRIO_BONUS_RATIO, INTERACTIVE_DELTA, WAKER_BONUS_RATIO, MAX_SLEEP_AVG, STARVATION_LIMIT, NODE_THRESHOLD. NOTE: you really have to know which value means what, some combinations might cause a broken scheduler & cause crashes. Eg. dont even attempt to set a smaller min timeslice than max timeslice, and dont set any of them to 0. NOTE2: when setting these tunables, always make sure to reach an 'idle' scheduling state first. While it's safe to change these values on the fly, it makes little sense to eg. set them while a kernel compile is ongoing - the make processes might have accumulated dynamic priority already. The recommended way to set these tunables is to stop everything in the system, (X can stay around, it should just be mostly idle) and wait 10 seconds (or the timeout you use for MAX_SLEEP_AVG), so that all the sleep-averages get back to an 'idle' state. Then start whatever workload you do to test interactivity, from scratch. i've also implemented a new tunable, WAKER_BONUS_RATIO, which implements Linus' idea of sharing the boost between waker and wakee in a more immediate way. It defaults to 0 currently, but you might want to try to set it to 50%, or 25%, or even 75%. The -B2 patch has the following changes over -A6 (BK-curr): - fix a (now-) bug in kernel/softirq.c, it did a wakeup outside any atomic regions, which falsely identified random processes as a non-atomic wakeup, and which causes random priority boost to be distributed. - reset the initial idle thread's priority back to PRIO_MAX after doing the wakeup_forked_process() - correct preemption relies on this. - introduce WAKER_BONUS_RATIO - update current->prio immediately after a backboost. - clean up effective_prio() & sleep_avg calculations so that there are fewer RT-task special cases. This, besides resulting in a much cleaner WAKER_BONUS_RATIO code, also has the advantage of the sleep_avg being maintained even for RT tasks - this could be advantegous for tasks that briefly enter/exit RT mode. Ingo --- linux/include/linux/sched.h.orig +++ linux/include/linux/sched.h @@ -328,7 +328,7 @@ struct task_struct { prio_array_t *array; unsigned long sleep_avg; - unsigned long sleep_timestamp; + unsigned long last_run; unsigned long policy; unsigned long cpus_allowed; --- linux/kernel/sched.c.orig +++ linux/kernel/sched.c @@ -54,20 +54,21 @@ /* * These are the 'tuning knobs' of the scheduler: * - * Minimum timeslice is 10 msecs, default timeslice is 150 msecs, - * maximum timeslice is 300 msecs. Timeslices get refilled after + * Minimum timeslice is 10 msecs, default timeslice is 100 msecs, + * maximum timeslice is 200 msecs. Timeslices get refilled after * they expire. */ -#define MIN_TIMESLICE ( 10 * HZ / 1000) -#define MAX_TIMESLICE (300 * HZ / 1000) -#define CHILD_PENALTY 95 -#define PARENT_PENALTY 100 -#define EXIT_WEIGHT 3 -#define PRIO_BONUS_RATIO 25 -#define INTERACTIVE_DELTA 2 -#define MAX_SLEEP_AVG (2*HZ) -#define STARVATION_LIMIT (2*HZ) -#define NODE_THRESHOLD 125 +unsigned int MIN_TIMESLICE = ( 10 * HZ / 1000); +unsigned int MAX_TIMESLICE = (200 * HZ / 1000); +unsigned int CHILD_PENALTY = 50; +unsigned int PARENT_PENALTY = 100; +unsigned int EXIT_WEIGHT = 3; +unsigned int PRIO_BONUS_RATIO = 25; +unsigned int INTERACTIVE_DELTA = 2; +unsigned int WAKER_BONUS_RATIO = 0; +unsigned int MAX_SLEEP_AVG = (10*HZ); +unsigned int STARVATION_LIMIT = (10*HZ); +unsigned int NODE_THRESHOLD = 125; /* * If a task is 'interactive' then we reinsert it in the active @@ -302,10 +303,13 @@ static inline void enqueue_task(struct t * * Both properties are important to certain workloads. */ -static inline int effective_prio(task_t *p) +static int effective_prio(task_t *p) { int bonus, prio; + if (rt_task(p)) + return p->prio; + bonus = MAX_USER_PRIO*PRIO_BONUS_RATIO*p->sleep_avg/MAX_SLEEP_AVG/100 - MAX_USER_PRIO*PRIO_BONUS_RATIO/100/2; @@ -323,26 +327,61 @@ static inline int effective_prio(task_t * Also update all the scheduling statistics stuff. (sleep average * calculation, priority modifiers, etc.) */ +static inline void __activate_task(task_t *p, runqueue_t *rq) +{ + enqueue_task(p, rq->active); + rq->nr_running++; +} + static inline void activate_task(task_t *p, runqueue_t *rq) { - unsigned long sleep_time = jiffies - p->sleep_timestamp; - prio_array_t *array = rq->active; + unsigned long sleep_time = jiffies - p->last_run; - if (!rt_task(p) && sleep_time) { + if (sleep_time) { + int waker_bonus = 0; /* - * This code gives a bonus to interactive tasks. We update - * an 'average sleep time' value here, based on - * sleep_timestamp. The more time a task spends sleeping, - * the higher the average gets - and the higher the priority - * boost gets as well. + * This code gives a bonus to interactive tasks. For + * asynchronous wakeups all the bonus goes to the woken up + * task. For non-atomic-context wakeups, the bonus is + * shared between the waker and the woken up task. Via + * this we recognize the waker as being related to the + * 'interactiveness' of the woken up task. + * + * The boost works by updating the 'average sleep time' + * value here, based on ->last_run. The more time a task + * spends sleeping, the higher the average gets - and the + * higher the priority boost gets as well. */ - p->sleep_avg += sleep_time; - if (p->sleep_avg > MAX_SLEEP_AVG) + if (!in_interrupt()) + waker_bonus = sleep_time * WAKER_BONUS_RATIO / 100; + + p->sleep_avg += sleep_time - waker_bonus; + + /* + * 'Overflow' bonus ticks go to the waker as well, so the + * ticks are not lost. This has the effect of further + * boosting tasks that are related to maximum-interactive + * tasks. + */ + if (p->sleep_avg > MAX_SLEEP_AVG) { + waker_bonus += p->sleep_avg - MAX_SLEEP_AVG; p->sleep_avg = MAX_SLEEP_AVG; - p->prio = effective_prio(p); + } + p->prio = effective_prio(p); + + if (!in_interrupt()) { + prio_array_t *array = current->array; + BUG_ON(!array); + + current->sleep_avg += waker_bonus; + if (current->sleep_avg > MAX_SLEEP_AVG) + current->sleep_avg = MAX_SLEEP_AVG; + dequeue_task(current, array); + current->prio = effective_prio(current); + enqueue_task(current, array); + } } - enqueue_task(p, array); - nr_running_inc(rq); + __activate_task(p, rq); } /* @@ -479,10 +518,13 @@ repeat_lock_task: } if (old_state == TASK_UNINTERRUPTIBLE) rq->nr_uninterruptible--; - activate_task(p, rq); - - if (p->prio < rq->curr->prio) - resched_task(rq->curr); + if (sync) + __activate_task(p, rq); + else { + activate_task(p, rq); + if (p->prio < rq->curr->prio) + resched_task(rq->curr); + } success = 1; } p->state = TASK_RUNNING; @@ -514,19 +556,25 @@ void wake_up_forked_process(task_t * p) runqueue_t *rq = task_rq_lock(current, &flags); p->state = TASK_RUNNING; - if (!rt_task(p)) { - /* - * We decrease the sleep average of forking parents - * and children as well, to keep max-interactive tasks - * from forking tasks that are max-interactive. - */ - current->sleep_avg = current->sleep_avg * PARENT_PENALTY / 100; - p->sleep_avg = p->sleep_avg * CHILD_PENALTY / 100; - p->prio = effective_prio(p); - } + /* + * We decrease the sleep average of forking parents + * and children as well, to keep max-interactive tasks + * from forking tasks that are max-interactive. + */ + current->sleep_avg = current->sleep_avg * PARENT_PENALTY / 100; + p->sleep_avg = p->sleep_avg * CHILD_PENALTY / 100; + p->prio = effective_prio(p); set_task_cpu(p, smp_processor_id()); - activate_task(p, rq); + if (unlikely(!current->array)) + __activate_task(p, rq); + else { + p->prio = current->prio; + list_add_tail(&p->run_list, ¤t->run_list); + p->array = current->array; + p->array->nr_active++; + rq->nr_running++; + } task_rq_unlock(rq, &flags); } @@ -953,6 +1001,11 @@ static inline void pull_task(runqueue_t */ if (p->prio < this_rq->curr->prio) set_need_resched(); + else { + if (p->prio == this_rq->curr->prio && + p->time_slice > this_rq->curr->time_slice) + set_need_resched(); + } } /* @@ -1016,7 +1069,7 @@ skip_queue: */ #define CAN_MIGRATE_TASK(p,rq,this_cpu) \ - ((jiffies - (p)->sleep_timestamp > cache_decay_ticks) && \ + ((jiffies - (p)->last_run > cache_decay_ticks) && \ !task_running(rq, p) && \ ((p)->cpus_allowed & (1UL << (this_cpu)))) @@ -1076,9 +1129,9 @@ DEFINE_PER_CPU(struct kernel_stat, kstat * increasing number of running tasks: */ #define EXPIRED_STARVING(rq) \ - ((rq)->expired_timestamp && \ + (STARVATION_LIMIT && ((rq)->expired_timestamp && \ (jiffies - (rq)->expired_timestamp >= \ - STARVATION_LIMIT * ((rq)->nr_running) + 1)) + STARVATION_LIMIT * ((rq)->nr_running) + 1))) /* * This function gets called by the timer code, with HZ frequency. @@ -1121,6 +1174,16 @@ void scheduler_tick(int user_ticks, int return; } spin_lock(&rq->lock); + /* + * The task was running during this tick - update the + * time slice counter and the sleep average. Note: we + * do not update a thread's priority until it either + * goes to sleep or uses up its timeslice. This makes + * it possible for interactive tasks to use up their + * timeslices at their highest priority levels. + */ + if (p->sleep_avg) + p->sleep_avg--; if (unlikely(rt_task(p))) { /* * RR tasks need a special form of timeslice management. @@ -1137,16 +1200,6 @@ void scheduler_tick(int user_ticks, int } goto out; } - /* - * The task was running during this tick - update the - * time slice counter and the sleep average. Note: we - * do not update a thread's priority until it either - * goes to sleep or uses up its timeslice. This makes - * it possible for interactive tasks to use up their - * timeslices at their highest priority levels. - */ - if (p->sleep_avg) - p->sleep_avg--; if (!--p->time_slice) { dequeue_task(p, rq->active); set_tsk_need_resched(p); @@ -1201,7 +1254,7 @@ need_resched: rq = this_rq(); release_kernel_lock(prev); - prev->sleep_timestamp = jiffies; + prev->last_run = jiffies; spin_lock_irq(&rq->lock); /* @@ -1701,7 +1754,7 @@ static int setscheduler(pid_t pid, int p else p->prio = p->static_prio; if (array) - activate_task(p, task_rq(p)); + __activate_task(p, task_rq(p)); out_unlock: task_rq_unlock(rq, &flags); @@ -2442,6 +2495,7 @@ void __init sched_init(void) rq->idle = current; set_task_cpu(current, smp_processor_id()); wake_up_forked_process(current); + current->prio = MAX_PRIO; init_timers(); --- linux/kernel/fork.c.orig +++ linux/kernel/fork.c @@ -916,7 +916,7 @@ static struct task_struct *copy_process( */ p->first_time_slice = 1; current->time_slice >>= 1; - p->sleep_timestamp = jiffies; + p->last_run = jiffies; if (!current->time_slice) { /* * This case is rare, it happens when the parent has only --- linux/kernel/sysctl.c.orig +++ linux/kernel/sysctl.c @@ -43,6 +43,7 @@ /* External variables not in a header file. */ extern int panic_timeout; +extern unsigned int MIN_TIMESLICE, MAX_TIMESLICE, CHILD_PENALTY, PARENT_PENALTY, EXIT_WEIGHT, PRIO_BONUS_RATIO, PRIO_BONUS_RATIO, INTERACTIVE_DELTA, WAKER_BONUS_RATIO, MAX_SLEEP_AVG, STARVATION_LIMIT, NODE_THRESHOLD; extern int C_A_D; extern int sysctl_overcommit_memory; extern int sysctl_overcommit_ratio; @@ -172,6 +173,28 @@ static ctl_table kern_table[] = { 0644, NULL, &proc_doutsstring, &sysctl_string}, {KERN_PANIC, "panic", &panic_timeout, sizeof(int), 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "MIN_TIMESLICE", &MIN_TIMESLICE, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "MAX_TIMESLICE", &MAX_TIMESLICE, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "CHILD_PENALTY", &CHILD_PENALTY, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "PARENT_PENALTY", &PARENT_PENALTY, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "EXIT_WEIGHT", &EXIT_WEIGHT, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "PRIO_BONUS_RATIO", &PRIO_BONUS_RATIO, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "INTERACTIVE_DELTA", &INTERACTIVE_DELTA, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "WAKER_BONUS_RATIO", &WAKER_BONUS_RATIO, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "MAX_SLEEP_AVG", &MAX_SLEEP_AVG, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "STARVATION_LIMIT", &STARVATION_LIMIT, sizeof(int), + 0644, NULL, &proc_dointvec}, + {KERN_PANIC, "NODE_THRESHOLD", &NODE_THRESHOLD, sizeof(int), + 0644, NULL, &proc_dointvec}, {KERN_CORE_USES_PID, "core_uses_pid", &core_uses_pid, sizeof(int), 0644, NULL, &proc_dointvec}, {KERN_CORE_PATTERN, "core_pattern", core_pattern, 64, --- linux/kernel/softirq.c.orig +++ linux/kernel/softirq.c @@ -92,10 +92,9 @@ restart: mask &= ~pending; goto restart; } - __local_bh_enable(); - if (pending) wakeup_softirqd(cpu); + __local_bh_enable(); } local_irq_restore(flags); ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 10:03 ` [patch] "interactivity changes", sched-2.5.64-B2 Ingo Molnar @ 2003-03-07 14:45 ` Ingo Molnar [not found] ` <Pine.LNX.4.44.0303071543480.12493-100000@localhost.localdo main> 1 sibling, 0 replies; 21+ messages in thread From: Ingo Molnar @ 2003-03-07 14:45 UTC (permalink / raw) To: Mike Galbraith; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel btw., could you please revert the kernel/softirq.c change, and re-test -j25 interactivity with that patch? That way we'll know exactly which component of -B2 caused the improvement on your box. Ingo ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <Pine.LNX.4.44.0303071543480.12493-100000@localhost.localdo main>]
* Re: [patch] "interactivity changes", sched-2.5.64-B2 [not found] ` <Pine.LNX.4.44.0303071543480.12493-100000@localhost.localdo main> @ 2003-03-07 15:44 ` Mike Galbraith 2003-03-07 18:50 ` Ingo Molnar 0 siblings, 1 reply; 21+ messages in thread From: Mike Galbraith @ 2003-03-07 15:44 UTC (permalink / raw) To: Ingo Molnar; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel At 03:45 PM 3/7/2003 +0100, Ingo Molnar wrote: >btw., could you please revert the kernel/softirq.c change, and re-test >-j25 interactivity with that patch? That way we'll know exactly which >component of -B2 caused the improvement on your box. Done. If either the softirq.c change or changing WAKER_BONUS_RATIO value (25:50:75) make any difference at all with what I'm doing, it's too close for me to tell. -Mike ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 15:44 ` Mike Galbraith @ 2003-03-07 18:50 ` Ingo Molnar 2003-03-07 18:55 ` Linus Torvalds 2003-03-07 19:10 ` Linus Torvalds 0 siblings, 2 replies; 21+ messages in thread From: Ingo Molnar @ 2003-03-07 18:50 UTC (permalink / raw) To: Mike Galbraith; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel On Fri, 7 Mar 2003, Mike Galbraith wrote: > Done. If either the softirq.c change or changing WAKER_BONUS_RATIO > value (25:50:75) make any difference at all with what I'm doing, it's > too close for me to tell. hm. Could you then please just re-test the -A6 patch with the same base tree again? I'd like to make sure that it _is_ the -A6 => -B0 delta that causes this considerable level of improvement for you. Ingo ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 18:50 ` Ingo Molnar @ 2003-03-07 18:55 ` Linus Torvalds 2003-03-07 19:10 ` Linus Torvalds 1 sibling, 0 replies; 21+ messages in thread From: Linus Torvalds @ 2003-03-07 18:55 UTC (permalink / raw) To: Ingo Molnar; +Cc: Mike Galbraith, Andrew Morton, Robert Love, linux-kernel On Fri, 7 Mar 2003, Ingo Molnar wrote: > > hm. Could you then please just re-test the -A6 patch with the same base > tree again? I'd like to make sure that it _is_ the -A6 => -B0 delta that > causes this considerable level of improvement for you. Side note: current BK has the -B2 thing without the non-core changes (ie no WAKER_BONUS_RATIO, and no sysctl stuff, and some reorg of the code due to the dropping of the WAKER_BONUS_RATIO). Ingo, you might want to double-check that you agree that my reorg is equivalent. Linus ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 18:50 ` Ingo Molnar 2003-03-07 18:55 ` Linus Torvalds @ 2003-03-07 19:10 ` Linus Torvalds 2003-03-07 19:13 ` Ingo Molnar 2003-03-07 19:31 ` Ingo Molnar 1 sibling, 2 replies; 21+ messages in thread From: Linus Torvalds @ 2003-03-07 19:10 UTC (permalink / raw) To: Ingo Molnar; +Cc: Mike Galbraith, Andrew Morton, Robert Love, linux-kernel Ingo, I already merged and pushed out the core changes in -B2, but now that I'm looking more closely at it I'm almost certain that it's buggy. In particular, you do this: .. dequeue_task(current, array); current->prio = effective_prio(current); enqueue_task(current, array); .. without actually holding the rq lock on the array! Yes, we hold _a_ rq_lock - we hold the rq lock for the array that "p" is on, but that's necessarily the same as this_rq(). So you can end up with the priority bitmap and the runqueue being corrupted by another wakeup on another CPU that wakes up something that is on the runqueue for the current CPU. At which point the machine will be basically dead (runqueue corruption is not likely to be a survivable event). Or am I missing something? Linus ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 19:10 ` Linus Torvalds @ 2003-03-07 19:13 ` Ingo Molnar 2003-03-07 19:31 ` Ingo Molnar 1 sibling, 0 replies; 21+ messages in thread From: Ingo Molnar @ 2003-03-07 19:13 UTC (permalink / raw) To: Linus Torvalds; +Cc: Mike Galbraith, Andrew Morton, Robert Love, linux-kernel you are right - i'm working on a fix. Ingo ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 19:10 ` Linus Torvalds 2003-03-07 19:13 ` Ingo Molnar @ 2003-03-07 19:31 ` Ingo Molnar 1 sibling, 0 replies; 21+ messages in thread From: Ingo Molnar @ 2003-03-07 19:31 UTC (permalink / raw) To: Linus Torvalds; +Cc: Mike Galbraith, Andrew Morton, Robert Love, linux-kernel the attached patch (against your tree) fixes the SMP locking bug. The patch also includes some other stuff i already added to my tree by that time: - only update the priority and do a requeueing if the sleep average has changed. (this does not happen for pure CPU hogs or pure interactive tasks, so no need to requeue/recalc-prio in that case.) [All the necessary values are available at that point already, so gcc should have an easy job making this branch really cheap.] - do not do a full task activation in the migration-thread path - that is supposed to be near-atomic anyway. - fix up comments i solved the SMP locking bug by moving the requeueing outside of try_to_wake_up(). It does not matter that the priority update is not atomically done now, since the current process wont do anything inbetween. (well, it could get preempted in a preemptible kernel, but even that wont do any harm.) did a quick testcompile & testboot on an SMP box, appears to work. Ingo --- linux/kernel/sched.c.orig +++ linux/kernel/sched.c @@ -321,10 +321,7 @@ static int effective_prio(task_t *p) } /* - * activate_task - move a task to the runqueue. - - * Also update all the scheduling statistics stuff. (sleep average - * calculation, priority modifiers, etc.) + * __activate_task - move a task to the runqueue. */ static inline void __activate_task(task_t *p, runqueue_t *rq) { @@ -332,9 +329,16 @@ static inline void __activate_task(task_ nr_running_inc(rq); } -static inline void activate_task(task_t *p, runqueue_t *rq) +/* + * activate_task - move a task to the runqueue and do priority recalculation + * + * Update all the scheduling statistics stuff. (sleep average + * calculation, priority modifiers, etc.) + */ +static inline int activate_task(task_t *p, runqueue_t *rq) { unsigned long sleep_time = jiffies - p->last_run; + int requeue_waker = 0; if (sleep_time) { int sleep_avg; @@ -357,23 +361,25 @@ static inline void activate_task(task_t */ if (sleep_avg > MAX_SLEEP_AVG) { if (!in_interrupt()) { - prio_array_t *array = current->array; - BUG_ON(!array); sleep_avg += current->sleep_avg - MAX_SLEEP_AVG; if (sleep_avg > MAX_SLEEP_AVG) sleep_avg = MAX_SLEEP_AVG; - current->sleep_avg = sleep_avg; - dequeue_task(current, array); - current->prio = effective_prio(current); - enqueue_task(current, array); + if (current->sleep_avg != sleep_avg) { + current->sleep_avg = sleep_avg; + requeue_waker = 1; + } } sleep_avg = MAX_SLEEP_AVG; } - p->sleep_avg = sleep_avg; - p->prio = effective_prio(p); + if (p->sleep_avg != sleep_avg) { + p->sleep_avg = sleep_avg; + p->prio = effective_prio(p); + } } __activate_task(p, rq); + + return requeue_waker; } /* @@ -486,8 +492,8 @@ void kick_if_running(task_t * p) */ static int try_to_wake_up(task_t * p, unsigned int state, int sync) { + int success = 0, requeue_waker = 0; unsigned long flags; - int success = 0; long old_state; runqueue_t *rq; @@ -513,7 +519,7 @@ repeat_lock_task: if (sync) __activate_task(p, rq); else { - activate_task(p, rq); + requeue_waker = activate_task(p, rq); if (p->prio < rq->curr->prio) resched_task(rq->curr); } @@ -523,6 +529,21 @@ repeat_lock_task: } task_rq_unlock(rq, &flags); + /* + * We have to do this outside the other spinlock, the two + * runqueues might be different: + */ + if (requeue_waker) { + prio_array_t *array; + + rq = task_rq_lock(current, &flags); + array = current->array; + dequeue_task(current, array); + current->prio = effective_prio(current); + enqueue_task(current, array); + task_rq_unlock(rq, &flags); + } + return success; } @@ -2360,7 +2381,7 @@ repeat: set_task_cpu(p, cpu_dest); if (p->array) { deactivate_task(p, rq_src); - activate_task(p, rq_dest); + __activate_task(p, rq_dest); if (p->prio < rq_dest->curr->prio) resched_task(rq_dest->curr); } ^ permalink raw reply [flat|nested] 21+ messages in thread
[parent not found: <Pine.LNX.4.44.0303071049500.7326-100000@localhost.localdom ain>]
* Re: [patch] "interactivity changes", sched-2.5.64-B2 [not found] ` <Pine.LNX.4.44.0303071049500.7326-100000@localhost.localdom ain> @ 2003-03-07 10:23 ` Mike Galbraith 2003-03-07 12:54 ` Mike Galbraith 0 siblings, 1 reply; 21+ messages in thread From: Mike Galbraith @ 2003-03-07 10:23 UTC (permalink / raw) To: Ingo Molnar; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel At 11:03 AM 3/7/2003 +0100, Ingo Molnar wrote: >i've attached the -B2 patch (against vanilla 2.5.64)..... (I'll test and report back in an hour or so. RL honey-do's are SCHED_RR... if a guy knows what's good for him;) ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 10:23 ` Mike Galbraith @ 2003-03-07 12:54 ` Mike Galbraith 2003-03-07 13:27 ` Mike Galbraith 0 siblings, 1 reply; 21+ messages in thread From: Mike Galbraith @ 2003-03-07 12:54 UTC (permalink / raw) To: Ingo Molnar; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel At 11:23 AM 3/7/2003 +0100, Mike Galbraith wrote: >At 11:03 AM 3/7/2003 +0100, Ingo Molnar wrote: > >>i've attached the -B2 patch (against vanilla 2.5.64)..... > >(I'll test and report back in an hour or so... B2 appears to fix my interactivity woes w. X + make -j5 bzImage. I can now waggle windows to my heart's content, browse around and click on RFC's and such and they pop up with nice response times. Spiffy :) /Me thinks desktop users will like this patch a bunch. -Mike (I'd really like to see a multi-client db load time-to-query comparison though. that make -j30 weirdness might not be a bad thing, but it makes me nervous [worry wart]) ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 12:54 ` Mike Galbraith @ 2003-03-07 13:27 ` Mike Galbraith 2003-03-07 14:43 ` Ingo Molnar 0 siblings, 1 reply; 21+ messages in thread From: Mike Galbraith @ 2003-03-07 13:27 UTC (permalink / raw) To: Ingo Molnar; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel At 01:54 PM 3/7/2003 +0100, Mike Galbraith wrote: >Spiffy :) /Me thinks desktop users will like this patch a bunch. (I can even play asteroids [fly little rocket ship around, shoot at and ram space rocks] with make -j25 bzImage and some swapping [sucks when you hit heavy swap of course, but quite playable as long as swap is light]) ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 13:27 ` Mike Galbraith @ 2003-03-07 14:43 ` Ingo Molnar 2003-03-07 20:34 ` Martin Josefsson 0 siblings, 1 reply; 21+ messages in thread From: Ingo Molnar @ 2003-03-07 14:43 UTC (permalink / raw) To: Mike Galbraith; +Cc: Andrew Morton, Linus Torvalds, Robert Love, linux-kernel On Fri, 7 Mar 2003, Mike Galbraith wrote: > >Spiffy :) /Me thinks desktop users will like this patch a bunch. > > (I can even play asteroids [fly little rocket ship around, shoot at and > ram space rocks] with make -j25 bzImage and some swapping [sucks when > you hit heavy swap of course, but quite playable as long as swap is > light]) cool! Could you please also play a bit with various WAKER_BONUS_RATIO values? If the default '0' value is the best-performing variant then i'll nuke it from the patch altogether. Ingo ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 14:43 ` Ingo Molnar @ 2003-03-07 20:34 ` Martin Josefsson 2003-03-07 20:39 ` Ingo Molnar 2003-03-07 21:03 ` David Lang 0 siblings, 2 replies; 21+ messages in thread From: Martin Josefsson @ 2003-03-07 20:34 UTC (permalink / raw) To: Ingo Molnar Cc: Mike Galbraith, Andrew Morton, Linus Torvalds, Robert Love, linux-kernel On Fri, 2003-03-07 at 15:43, Ingo Molnar wrote: > On Fri, 7 Mar 2003, Mike Galbraith wrote: > > > >Spiffy :) /Me thinks desktop users will like this patch a bunch. > > > > (I can even play asteroids [fly little rocket ship around, shoot at and > > ram space rocks] with make -j25 bzImage and some swapping [sucks when > > you hit heavy swap of course, but quite playable as long as swap is > > light]) > > cool! Could you please also play a bit with various WAKER_BONUS_RATIO > values? If the default '0' value is the best-performing variant then i'll > nuke it from the patch altogether. Hi, I've tested 2.5.64 + sched-2.5.64-B2 on an UP pIII 700 with 704MB ram, here's my results... General impression is that interactivity is good but it seems there's a few cases that gets worse, sometimes a lot worse. First the things that got better... Xmms doesn't skip when switching tabs in galeon anymore during general desktopusage (more on that below) Earlier I had to change some schedulersettings to get it to stop skipping. And it was possible to watch a movie using mplayer while running a make -j10 on a kerneltree, the playback wasn't 100% perfect but I'd say it was ~90% , the jerkyness I saw wasn't very irritating. I tried the window wiggle-test while the compile and movie was playing and it was smooth and only added a little more jerkyness to the playback. Some negative things: Xmms could skip during the compile/movie playback, most notably one rather long skip right after a song-change but sometimes in the middle of a song as well, those skips were quite long as well, 2-3 seconds. Another xmms related problem is that even without any load on the machine the mousepointer gets jerky for 0.1-0.5 seconds right after having used the mouse to search forward or backwards in a song. It gets jerky at the moment you release the mousebutton. Basically no change in the jerkyness even with a make -j10 and movie playing. The biggest problem is the fact that my windowmanager gets unresponsive for ~30 seconds when I perform something that works fine with vanilla 2.4 and 2.5. I have two keybindings that execute 'aumix -v+3' and 'aumix -v-3'. And if I hold one of those buttons down for say 2 seconds then at first it seems like nothing is happening but then the volume starts to inc-/decrease sloowly and the windowmanager (sawfish) is unresponsive for ~30 seconds. With 2.4 or vanilla 2.5 the change in volume is instant and I've never noticed that sawfish got unresponsive. Sure holding the button down for 2 seconds at a repeatrate of 30/s will cause a lot of aumix's to be executed but 30 seconds for 60 aumix's is quite a long time. A few times it has been enough to just press one of those keys a few times to get the exact same effect. It's as if sawfish goes into a weird loop. All this might be a bug in sawfish... This behaviour is the same with or without the compile/movie running. Another problem is that the composer-window in evolution gets unresponsive for 1-2 seconds if you hold down a key, and when it gets responsive again it gets all the keypresses... I've tried changing WAKER_BONUS_RATIO but I didn't see any diffrence. At least it didn't help with the Xmms mouse-jerkyness or the sawfish_goes_out_for_coffee_and_a_cigarette problem. There, that was my impressions of 2 hours usage of the patch. -- /Martin Never argue with an idiot. They drag you down to their level, then beat you with experience. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 20:34 ` Martin Josefsson @ 2003-03-07 20:39 ` Ingo Molnar 2003-03-07 21:05 ` Martin Josefsson 2003-03-07 21:03 ` David Lang 1 sibling, 1 reply; 21+ messages in thread From: Ingo Molnar @ 2003-03-07 20:39 UTC (permalink / raw) To: Martin Josefsson Cc: Mike Galbraith, Andrew Morton, Linus Torvalds, Robert Love, linux-kernel On 7 Mar 2003, Martin Josefsson wrote: > Some negative things: if you have time/interest, could you re-test the negative things with X reniced to -10, to further isolate the problem? Another thing to try is to renice xmms to -10 (or RT priority). Which one makes the larger difference? Ingo ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 20:39 ` Ingo Molnar @ 2003-03-07 21:05 ` Martin Josefsson 2003-03-07 21:47 ` Martin Josefsson 0 siblings, 1 reply; 21+ messages in thread From: Martin Josefsson @ 2003-03-07 21:05 UTC (permalink / raw) To: Ingo Molnar Cc: Mike Galbraith, Andrew Morton, Linus Torvalds, Robert Love, linux-kernel On Fri, 2003-03-07 at 21:39, Ingo Molnar wrote: > On 7 Mar 2003, Martin Josefsson wrote: > > > Some negative things: > > if you have time/interest, could you re-test the negative things with X > reniced to -10, to further isolate the problem? Another thing to try is to > renice xmms to -10 (or RT priority). Which one makes the larger > difference? X niced to -10 makes the xmms mouse-jerkyness disappear. but it can still skip right after a songchange. If I renice xmms to -10 as well I get the mouse-jerkyness again, it I renice it to -5 it seems to behave quite well, no mouse-jerkyness and so far I havn't heard any skips, but it can take some time until a skip occurs. With X reniced to -10 it feels like the wiggle_a_window while a 'make -j10' and a movie is playing gets a bit more sluggish. And sawfish still takes 30 second naps when executing a bunch of aumix's. -- /Martin Never argue with an idiot. They drag you down to their level, then beat you with experience. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 21:05 ` Martin Josefsson @ 2003-03-07 21:47 ` Martin Josefsson 0 siblings, 0 replies; 21+ messages in thread From: Martin Josefsson @ 2003-03-07 21:47 UTC (permalink / raw) To: Ingo Molnar Cc: Mike Galbraith, Andrew Morton, Linus Torvalds, Robert Love, linux-kernel On Fri, 2003-03-07 at 22:05, Martin Josefsson wrote: > And sawfish still takes 30 second naps when executing a bunch of > aumix's. For information: this is 2.5.64 + sched-2.5.64-B2, no preempt or other patches. I changed the command that sawfish executes to only append the current date to a file and then I made it take a nap again. I got a file containing 57 lines, one for each time the command was executed by sawfish. it took 41 seconds to execute the command 57 times and the machine was very idle during that time. According to top sawfish's priority is between 15 and 20 the whole time. And as I said the machine is very idle during the test. The command was a very small bashscript: #!/bin/bash date >> /tmp/result So no aumix executed in the script, didn't want another error factor in the test. I've reproduced the problem three times and this is what I got: executions time ---------- ---- 57 41 seconds 52 39 seconds 28 25 seconds results from the third run Fri Mar 7 22:31:00 CET 2003 Fri Mar 7 22:31:02 CET 2003 Fri Mar 7 22:31:03 CET 2003 Fri Mar 7 22:31:04 CET 2003 Fri Mar 7 22:31:05 CET 2003 Fri Mar 7 22:31:06 CET 2003 Fri Mar 7 22:31:07 CET 2003 Fri Mar 7 22:31:07 CET 2003 Fri Mar 7 22:31:07 CET 2003 Fri Mar 7 22:31:09 CET 2003 Fri Mar 7 22:31:10 CET 2003 Fri Mar 7 22:31:11 CET 2003 Fri Mar 7 22:31:12 CET 2003 Fri Mar 7 22:31:14 CET 2003 Fri Mar 7 22:31:14 CET 2003 Fri Mar 7 22:31:14 CET 2003 Fri Mar 7 22:31:15 CET 2003 Fri Mar 7 22:31:16 CET 2003 Fri Mar 7 22:31:17 CET 2003 Fri Mar 7 22:31:18 CET 2003 Fri Mar 7 22:31:19 CET 2003 Fri Mar 7 22:31:19 CET 2003 Fri Mar 7 22:31:19 CET 2003 Fri Mar 7 22:31:20 CET 2003 Fri Mar 7 22:31:22 CET 2003 Fri Mar 7 22:31:23 CET 2003 Fri Mar 7 22:31:24 CET 2003 Fri Mar 7 22:31:25 CET 2003 Do you have any clues to what might be happening? -- /Martin Never argue with an idiot. They drag you down to their level, then beat you with experience. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 20:34 ` Martin Josefsson 2003-03-07 20:39 ` Ingo Molnar @ 2003-03-07 21:03 ` David Lang 2003-03-07 21:13 ` Martin Josefsson 1 sibling, 1 reply; 21+ messages in thread From: David Lang @ 2003-03-07 21:03 UTC (permalink / raw) To: Martin Josefsson Cc: Ingo Molnar, Mike Galbraith, Andrew Morton, Linus Torvalds, Robert Love, linux-kernel sounds like the code in X that detects a key being held down is running into problems. any chance it's doing a busy loop or something silly like that that's just not running enough? (probably not, but since you have problems in a couple applications that happen when you hold a key down I would look there rather then at the scheduling code itself) David Lang On 7 Mar 2003, Martin Josefsson wrote: > Date: 07 Mar 2003 21:34:38 +0100 > From: Martin Josefsson <gandalf@wlug.westbo.se> > To: Ingo Molnar <mingo@elte.hu> > Cc: Mike Galbraith <efault@gmx.de>, Andrew Morton <akpm@digeo.com>, > Linus Torvalds <torvalds@transmeta.com>, Robert Love <rml@tech9.net>, > linux-kernel@vger.kernel.org > Subject: Re: [patch] "interactivity changes", sched-2.5.64-B2 > > On Fri, 2003-03-07 at 15:43, Ingo Molnar wrote: > > On Fri, 7 Mar 2003, Mike Galbraith wrote: > > > > > >Spiffy :) /Me thinks desktop users will like this patch a bunch. > > > > > > (I can even play asteroids [fly little rocket ship around, shoot at and > > > ram space rocks] with make -j25 bzImage and some swapping [sucks when > > > you hit heavy swap of course, but quite playable as long as swap is > > > light]) > > > > cool! Could you please also play a bit with various WAKER_BONUS_RATIO > > values? If the default '0' value is the best-performing variant then i'll > > nuke it from the patch altogether. > > Hi, > > I've tested 2.5.64 + sched-2.5.64-B2 on an UP pIII 700 with 704MB ram, > here's my results... > > General impression is that interactivity is good but it seems there's a > few cases that gets worse, sometimes a lot worse. > > First the things that got better... > > Xmms doesn't skip when switching tabs in galeon anymore during general > desktopusage (more on that below) > Earlier I had to change some schedulersettings to get it to stop > skipping. > > And it was possible to watch a movie using mplayer while running a make > -j10 on a kerneltree, the playback wasn't 100% perfect but I'd say it > was ~90% , the jerkyness I saw wasn't very irritating. I tried the > window wiggle-test while the compile and movie was playing and it was > smooth and only added a little more jerkyness to the playback. > > > Some negative things: > > Xmms could skip during the compile/movie playback, most notably one > rather long skip right after a song-change but sometimes in the middle > of a song as well, those skips were quite long as well, 2-3 seconds. > > Another xmms related problem is that even without any load on the > machine the mousepointer gets jerky for 0.1-0.5 seconds right after > having used the mouse to search forward or backwards in a song. It gets > jerky at the moment you release the mousebutton. > Basically no change in the jerkyness even with a make -j10 and movie > playing. > > The biggest problem is the fact that my windowmanager gets unresponsive > for ~30 seconds when I perform something that works fine with vanilla > 2.4 and 2.5. I have two keybindings that execute 'aumix -v+3' and > 'aumix -v-3'. And if I hold one of those buttons down for say 2 seconds > then at first it seems like nothing is happening but then the volume > starts to inc-/decrease sloowly and the windowmanager (sawfish) is > unresponsive for ~30 seconds. With 2.4 or vanilla 2.5 the change in > volume is instant and I've never noticed that sawfish got unresponsive. > Sure holding the button down for 2 seconds at a repeatrate of 30/s will > cause a lot of aumix's to be executed but 30 seconds for 60 aumix's is > quite a long time. A few times it has been enough to just press one of > those keys a few times to get the exact same effect. It's as if sawfish > goes into a weird loop. All this might be a bug in sawfish... > This behaviour is the same with or without the compile/movie running. > > Another problem is that the composer-window in evolution gets > unresponsive for 1-2 seconds if you hold down a key, and when it gets > responsive again it gets all the keypresses... > > I've tried changing WAKER_BONUS_RATIO but I didn't see any diffrence. > At least it didn't help with the Xmms mouse-jerkyness or the > sawfish_goes_out_for_coffee_and_a_cigarette problem. > > There, that was my impressions of 2 hours usage of the patch. > > -- > /Martin > > Never argue with an idiot. They drag you down to their level, then beat you with experience. > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 21:03 ` David Lang @ 2003-03-07 21:13 ` Martin Josefsson 2003-03-07 23:05 ` Helge Hafting 0 siblings, 1 reply; 21+ messages in thread From: Martin Josefsson @ 2003-03-07 21:13 UTC (permalink / raw) To: David Lang Cc: Ingo Molnar, Mike Galbraith, Andrew Morton, Linus Torvalds, Robert Love, linux-kernel On Fri, 2003-03-07 at 22:03, David Lang wrote: > sounds like the code in X that detects a key being held down is running > into problems. any chance it's doing a busy loop or something silly like > that that's just not running enough? (probably not, but since you have > problems in a couple applications that happen when you hold a key down I > would look there rather then at the scheduling code itself) Wouldn't surprise me if it's an X problem... I can't say that I feel like going digging into X sources... I can get the same problem in sawfish if I press the key a few times quite rapidly as well, without any background load at all. This problem has never occured before on 2.4 or 2.5, with or without load. It could be that the scheduler changes exposes some bug in X but I'm not really sure how to start investigating... -- /Martin Never argue with an idiot. They drag you down to their level, then beat you with experience. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [patch] "interactivity changes", sched-2.5.64-B2 2003-03-07 21:13 ` Martin Josefsson @ 2003-03-07 23:05 ` Helge Hafting 0 siblings, 0 replies; 21+ messages in thread From: Helge Hafting @ 2003-03-07 23:05 UTC (permalink / raw) To: Martin Josefsson; +Cc: Ingo Molnar, linux-kernel Martin Josefsson wrote: > Wouldn't surprise me if it's an X problem... I can't say that I feel > like going digging into X sources... > > I can get the same problem in sawfish if I press the key a few times > quite rapidly as well, without any background load at all. This problem > has never occured before on 2.4 or 2.5, with or without load. It could > be that the scheduler changes exposes some bug in X but I'm not really > sure how to start investigating... You could perhaps rule out sawfish bugs by testing this with another window manager? Helge Hafting ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2003-03-07 22:53 UTC | newest] Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <5.2.0.9.2.20030307093435.01a8fe88@pop.gmx.net> 2003-03-07 9:30 ` [patch] "HT scheduler", sched-2.5.63-B3 Ingo Molnar [not found] ` <Pine.LNX.4.44.0303071003060.6318-100000@localhost.localdom ain> 2003-03-07 9:38 ` Mike Galbraith 2003-03-07 10:03 ` [patch] "interactivity changes", sched-2.5.64-B2 Ingo Molnar 2003-03-07 14:45 ` Ingo Molnar [not found] ` <Pine.LNX.4.44.0303071543480.12493-100000@localhost.localdo main> 2003-03-07 15:44 ` Mike Galbraith 2003-03-07 18:50 ` Ingo Molnar 2003-03-07 18:55 ` Linus Torvalds 2003-03-07 19:10 ` Linus Torvalds 2003-03-07 19:13 ` Ingo Molnar 2003-03-07 19:31 ` Ingo Molnar [not found] ` <Pine.LNX.4.44.0303071049500.7326-100000@localhost.localdom ain> 2003-03-07 10:23 ` Mike Galbraith 2003-03-07 12:54 ` Mike Galbraith 2003-03-07 13:27 ` Mike Galbraith 2003-03-07 14:43 ` Ingo Molnar 2003-03-07 20:34 ` Martin Josefsson 2003-03-07 20:39 ` Ingo Molnar 2003-03-07 21:05 ` Martin Josefsson 2003-03-07 21:47 ` Martin Josefsson 2003-03-07 21:03 ` David Lang 2003-03-07 21:13 ` Martin Josefsson 2003-03-07 23:05 ` Helge Hafting
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).