* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! [not found] <Pine.LNX.4.44.0401031439060.24942-100000@coffee.psychology.mcmaster.ca> @ 2004-01-03 20:19 ` Soeren Sonnenburg 2004-01-03 21:00 ` Con Kolivas 0 siblings, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-03 20:19 UTC (permalink / raw) To: Mark Hahn; +Cc: Linux Kernel On Sat, 2004-01-03 at 20:40, Mark Hahn wrote: > > yeah, I think so... but as generating output in a shell is a very common > > thing to do there should either be an option to turn that unwanted > > behaviour off or to fix this issue... > > has anyone said it's desired behavior? you probably need to describe > your setup more. for instance, is your X niced to negative? are there > some background processes which would be consuming cycles? freshly booted system with X running at niceness 0 no other processes consume cpu cycles. it is reproducable by creating any kind of output which reads from disk... so i.e. a find ./ in my home directory takes sometimes like 30 minutes on 2.6 (100%cpu load) and sometimes 5 minutes (on 2.4 always 5 minutes ~40%load). dmesg is another candidate... just doing cat <file> seems not to trigger that problem. As Willy Tarreau also oberves this very same weirdness - I now know the problem is there and it is not specific to my setup. Soeren. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 20:19 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Soeren Sonnenburg @ 2004-01-03 21:00 ` Con Kolivas 2004-01-03 21:10 ` Soeren Sonnenburg 2004-01-03 21:18 ` Willy Tarreau 0 siblings, 2 replies; 61+ messages in thread From: Con Kolivas @ 2004-01-03 21:00 UTC (permalink / raw) To: Soeren Sonnenburg, Mark Hahn; +Cc: Linux Kernel On Sun, 4 Jan 2004 07:19, Soeren Sonnenburg wrote: > On Sat, 2004-01-03 at 20:40, Mark Hahn wrote: > > > yeah, I think so... but as generating output in a shell is a very > > > common thing to do there should either be an option to turn that > > > unwanted behaviour off or to fix this issue... > > > > has anyone said it's desired behavior? you probably need to describe > > your setup more. for instance, is your X niced to negative? are there > > some background processes which would be consuming cycles? > > freshly booted system with X running at niceness 0 no other processes > consume cpu cycles. > > it is reproducable by creating any kind of output which reads from disk... > so i.e. a find ./ in my home directory takes sometimes like 30 minutes on > 2.6 (100%cpu load) and sometimes 5 minutes (on 2.4 always 5 minutes > ~40%load). > > dmesg is another candidate... just doing cat <file> seems not to trigger > that problem. > > As Willy Tarreau also oberves this very same weirdness - I now know the > problem is there and it is not specific to my setup. There is a BASH bug that Linus noticed brought out by the more sensitive timing in 2.6. The BASH developer has been informed it is there and it is fixed in the latest version. Perhaps you're both seeing that. Check the lkml archives. Con ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 21:00 ` Con Kolivas @ 2004-01-03 21:10 ` Soeren Sonnenburg 2004-01-03 21:15 ` Con Kolivas 2004-01-03 21:18 ` Willy Tarreau 1 sibling, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-03 21:10 UTC (permalink / raw) To: Con Kolivas; +Cc: Mark Hahn, Linux Kernel On Sat, 2004-01-03 at 22:00, Con Kolivas wrote: > On Sun, 4 Jan 2004 07:19, Soeren Sonnenburg wrote: > > On Sat, 2004-01-03 at 20:40, Mark Hahn wrote: [it is a bash bug] huuhh ? How can this be a bash bug if the output is done by other programs like dmesg or find ? however I just tested - this issue happens with csh/tcsh too... Soeren. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 21:10 ` Soeren Sonnenburg @ 2004-01-03 21:15 ` Con Kolivas 2004-01-03 23:35 ` Willy Tarreau 0 siblings, 1 reply; 61+ messages in thread From: Con Kolivas @ 2004-01-03 21:15 UTC (permalink / raw) To: Soeren Sonnenburg; +Cc: Mark Hahn, Linux Kernel On Sun, 4 Jan 2004 08:10, Soeren Sonnenburg wrote: > On Sat, 2004-01-03 at 22:00, Con Kolivas wrote: > > On Sun, 4 Jan 2004 07:19, Soeren Sonnenburg wrote: > > > On Sat, 2004-01-03 at 20:40, Mark Hahn wrote: > > [it is a bash bug] > huuhh ? How can this be a bash bug if the output is done by other > programs like dmesg or find ? however I just tested - this issue happens > with csh/tcsh too... The bash bug was waiting on a pipe problem; not just bash alone. I was just offering a suggestion. I have no idea what exactly to blame without evidence of what's at fault. Con ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 21:15 ` Con Kolivas @ 2004-01-03 23:35 ` Willy Tarreau 2004-01-04 0:11 ` Soeren Sonnenburg 2004-01-04 1:42 ` Con Kolivas 0 siblings, 2 replies; 61+ messages in thread From: Willy Tarreau @ 2004-01-03 23:35 UTC (permalink / raw) To: Con Kolivas; +Cc: Soeren Sonnenburg, Mark Hahn, Linux Kernel, gillb4 On Sun, Jan 04, 2004 at 08:15:54AM +1100, Con Kolivas wrote: > The bash bug was waiting on a pipe problem; not just bash alone. I was just > offering a suggestion. I have no idea what exactly to blame without evidence > of what's at fault. Well, I did a few tests here with both 2.4.23 and 2.6.1rc1. In fact, it is also partially xterm which is to blame, because it is not really that fast on 2.4. Look below : Context ======= - X not reniced (0) - cd to a directory with about 2500 files - ls -l 1) Simple standard 'ls -l' in an xterm ====================================== 2.4.23$ time ls -l real 0m0.444s user 0m0.090s sys 0m0.020s 2.6.1rc1$ time ls -l real 0m3.995s user 0m0.081s sys 0m0.035s 2) introduction of a pipe : ls -l | cat ======================================= 2.4.23$ time ls -l |cat real 0m0.444s user 0m0.090s sys 0m0.020s 2.6.1rc1$ time ls -l|cat real 0m0.486s user 0m0.048s sys 0m0.029s 3) it is not xterm which gets all the CPU either : ================================================== 2.4.23$ time xterm -e sh -c "ls -l" real 0m0.462s user 0m0.140s sys 0m0.040s 2.6.1rc1$ time xterm -e sh -c "ls -l" real 0m4.010s user 0m0.236s sys 0m0.079s 4) CPU is eaten by the X server itself ! ======================================== It appears that on 2.4.23, xterm quickly switches over to jump scrolling which prevents X from eating all CPU. On 2.6.1rc1, it is not the case. If I disable jump scrolling and list 2 directories totalizing 4000 files : 2.4.23$ time xterm +j -e sh -c "ls -l incoming tmp" real 0m7.137s user 0m0.390s sys 0m0.080s 2.6.1rc1$ time xterm +j -e sh -c "ls -l incoming tmp" real 0m7.113s user 0m0.314s sys 0m0.121s And here is what 'top' says during last command : top - 23:37:54 up 17 min, 5 users, load average: 0.72, 0.62, 0.50 Tasks: 58 total, 4 running, 54 sleeping, 0 stopped, 0 zombie Cpu(s): 94.5% user, 5.5% system, 0.0% nice, 0.0% idle, 0.0% IO-wait Mem: 515292k total, 510688k used, 4604k free, 175784k buffers Swap: 265064k total, 0k used, 265064k free, 19400k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ Command 234 root 16 0 22400 12m 11m S 90.0 2.4 2:20.01 X 417 root 19 0 4764 2660 3844 R 7.3 0.5 0:00.24 xterm 418 willy 20 0 1900 1100 1304 R 2.3 0.2 0:00.10 ls 394 willy 16 0 1692 940 1556 R 0.5 0.2 0:00.94 top 1 root 16 0 348 192 316 S 0.0 0.0 0:05.05 init 2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0 3 root 5 -10 0 0 0 S 0.0 0.0 0:00.23 events/0 4 root 5 -10 0 0 0 S 0.0 0.0 0:00.24 kblockd/0 5 root 25 0 0 0 0 S 0.0 0.0 0:00.00 pdflush 5) Tried again with X reniced to +10 ==================================== 2.6.1rc1$ xterm +j -e sh -c "ls -l incoming tmp" top - 23:48:33 up 27 min, 6 users, load average: 0.53, 0.46, 0.44 Tasks: 60 total, 2 running, 58 sleeping, 0 stopped, 0 zombie Cpu(s): 4.9% user, 4.9% system, 90.2% nice, 0.0% idle, 0.0% IO-wait Mem: 515292k total, 510652k used, 4640k free, 175712k buffers Swap: 265064k total, 0k used, 265064k free, 19624k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ Command 234 root 35 10 22400 12m 11m R 92.0 2.4 3:58.50 X 499 root 15 0 4764 2656 3844 S 4.0 0.5 0:00.24 xterm 500 willy 15 0 2024 1160 1416 S 3.0 0.2 0:00.10 ls 1 root 16 0 348 192 316 S 0.0 0.0 0:05.05 init 2 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0 3 root 5 -10 0 0 0 S 0.0 0.0 0:00.28 events/0 4 root 5 -10 0 0 0 S 0.0 0.0 0:00.24 kblockd/0 5 root 25 0 0 0 0 S 0.0 0.0 0:00.00 pdflush 6 root 15 0 0 0 0 S 0.0 0.0 0:00.00 pdflush And now with jump scrolling enabled again with X still reniced to +10 : 2.4.23$ time xterm -e sh -c "ls -l incoming tmp" real 0m0.648s user 0m0.230s sys 0m0.040s 2.6.1rc1$ time xterm -e sh -c "ls -l incoming tmp" real 0m0.691s user 0m0.230s sys 0m0.063s 6) Conclusion ============= Under 2.4, xterm uses jump scrolling which it does not use by default under 2.6 if X responds fast enough. The first dirty solution which comes to mind is to renice X to >+10 to slow it a bit so that xterm hits the high water level and jumps. But it's not an effect of the scheduler alone, but a side effect of the scheduler and xterm both trying to automatically adjust their behaviour in a different manner. If either the scheduler or xterm was a bit smarter or used different thresholds, the problem would go away. It would also explain why there are people who cannot reproduce it. Perhaps a somewhat faster or slower system makes the problem go away. Honnestly, it's the first time that I notice that my xterms are jump-scrolling, it was so much fluid anyway. BTW, I found an even simplest way to reproduce the problem while ensuring that it is not I/O related. For this, I use the 'seq' utility : 2.4.23$ $ time xterm -e seq 1 4000 real 0m0.142s user 0m0.060s sys 0m0.030s 2.4.23$ time xterm +j -e seq 1 4000 real 0m4.079s user 0m0.100s sys 0m0.080s Cheers, Willy ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 23:35 ` Willy Tarreau @ 2004-01-04 0:11 ` Soeren Sonnenburg 2004-01-04 1:42 ` Con Kolivas 1 sibling, 0 replies; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-04 0:11 UTC (permalink / raw) To: Willy Tarreau; +Cc: Con Kolivas, Mark Hahn, Linux Kernel, gillb4 On Sun, 2004-01-04 at 00:35, Willy Tarreau wrote: > On Sun, Jan 04, 2004 at 08:15:54AM +1100, Con Kolivas wrote: [...] > 6) Conclusion > ============= > > Under 2.4, xterm uses jump scrolling which it does not use by default under > 2.6 if X responds fast enough. The first dirty solution which comes to mind > is to renice X to >+10 to slow it a bit so that xterm hits the high water > level and jumps. > > But it's not an effect of the scheduler alone, but a side effect of the > scheduler and xterm both trying to automatically adjust their behaviour in > a different manner. If either the scheduler or xterm was a bit smarter or > used different thresholds, the problem would go away. It would also explain > why there are people who cannot reproduce it. Perhaps a somewhat faster or > slower system makes the problem go away. Honnestly, it's the first time that > I notice that my xterms are jump-scrolling, it was so much fluid anyway. Hmmhh, indeed I could reproduce that here. However I am using multi-gnome-terminal and I've heard of others observing this issue using gnome-terminal. As they are probably all derived from xterm I now do wonder how to fix them in such a way that it works smoothly with 2.4 and 2.6. Soeren. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 23:35 ` Willy Tarreau 2004-01-04 0:11 ` Soeren Sonnenburg @ 2004-01-04 1:42 ` Con Kolivas 2004-01-04 3:32 ` Tim Connors 2004-01-04 8:09 ` Soeren Sonnenburg 1 sibling, 2 replies; 61+ messages in thread From: Con Kolivas @ 2004-01-04 1:42 UTC (permalink / raw) To: Willy Tarreau; +Cc: Soeren Sonnenburg, Mark Hahn, Linux Kernel, gillb4 On Sun, 4 Jan 2004 10:35, Willy Tarreau wrote: > 6) Conclusion > ============= > > Under 2.4, xterm uses jump scrolling which it does not use by default under > 2.6 if X responds fast enough. The first dirty solution which comes to mind > is to renice X to >+10 to slow it a bit so that xterm hits the high water > level and jumps. > > But it's not an effect of the scheduler alone, but a side effect of the > scheduler and xterm both trying to automatically adjust their behaviour in > a different manner. Not quite. The scheduler retains high priority for X for longer so it's no new dynamic adjustment of any sort, just better cpu usage by X (which is why it's smoother now at nice 0 than previously). > If either the scheduler or xterm was a bit smarter or > used different thresholds, the problem would go away. It would also explain > why there are people who cannot reproduce it. Perhaps a somewhat faster or > slower system makes the problem go away. Honnestly, it's the first time > that I notice that my xterms are jump-scrolling, it was so much fluid > anyway. Very thorough but not a scheduler problem as far as I'm concerned. Can you not disable smooth scrolling and force jump scrolling? Con ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 1:42 ` Con Kolivas @ 2004-01-04 3:32 ` Tim Connors 2004-01-04 5:58 ` Con Kolivas 2004-01-05 22:25 ` Bryan Whitehead 2004-01-04 8:09 ` Soeren Sonnenburg 1 sibling, 2 replies; 61+ messages in thread From: Tim Connors @ 2004-01-04 3:32 UTC (permalink / raw) To: linux-kernel Con Kolivas <kernel@kolivas.org> said on Sun, 4 Jan 2004 12:42:47 +1100: > On Sun, 4 Jan 2004 10:35, Willy Tarreau wrote: > > 6) Conclusion > > ============= > > > > Under 2.4, xterm uses jump scrolling which it does not use by default under > > 2.6 if X responds fast enough. The first dirty solution which comes to mind > > is to renice X to >+10 to slow it a bit so that xterm hits the high water > > level and jumps. > > > > But it's not an effect of the scheduler alone, but a side effect of the > > scheduler and xterm both trying to automatically adjust their behaviour in > > a different manner. > > Not quite. The scheduler retains high priority for X for longer so it's no new > dynamic adjustment of any sort, just better cpu usage by X (which is why it's > smoother now at nice 0 than previously). > > > If either the scheduler or xterm was a bit smarter or > > used different thresholds, the problem would go away. It would also explain > > why there are people who cannot reproduce it. Perhaps a somewhat faster or > > slower system makes the problem go away. Honnestly, it's the first time > > that I notice that my xterms are jump-scrolling, it was so much fluid > > anyway. > > Very thorough but not a scheduler problem as far as I'm concerned. Can you not > disable smooth scrolling and force jump scrolling? AFAIK the definition of jump scrolling is that if xterm is falling behind, it jumps. Jump scrolling is enabled by default. What this slowness means is that xterm is getting CPU at just the right moments that it isn't falling behind, so it doesn't jump - which means X gets all the CPU to redraw, which means your ls/dmesg anything else that reads from disk[1] doesn't get any CPU. Xterm is already functioning as designed - you can't force jump scrolling to jump more - it is at the mercy of how it gets scheduled. If there is nothing more in the pipe to draw, it has to draw. These bloody interactive changes to make X more responsive are at the expense of anything that does *real* work. [1] Others say that it only affects them first time through - after something is cached, it goes back to normal speed. For me - it is slow *all* the time. If I pipe it to cat or tail or something, it is a *lot* quicker. -- TimC -- http://astronomy.swin.edu.au/staff/tconnors/ Beware of Programmers who carry screwdrivers. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 3:32 ` Tim Connors @ 2004-01-04 5:58 ` Con Kolivas 2004-01-06 1:09 ` Peter Osterlund 2004-01-05 22:25 ` Bryan Whitehead 1 sibling, 1 reply; 61+ messages in thread From: Con Kolivas @ 2004-01-04 5:58 UTC (permalink / raw) To: Tim Connors, linux-kernel On Sun, 4 Jan 2004 14:32, Tim Connors wrote: > > Not quite. The scheduler retains high priority for X for longer so it's > > no new dynamic adjustment of any sort, just better cpu usage by X (which > > is why it's smoother now at nice 0 than previously). > > > > > If either the scheduler or xterm was a bit smarter or > > > used different thresholds, the problem would go away. It would also > > > explain why there are people who cannot reproduce it. Perhaps a > > > somewhat faster or slower system makes the problem go away. Honnestly, > > > it's the first time that I notice that my xterms are jump-scrolling, it > > > was so much fluid anyway. > > > > Very thorough but not a scheduler problem as far as I'm concerned. Can > > you not disable smooth scrolling and force jump scrolling? > > AFAIK the definition of jump scrolling is that if xterm is falling > behind, it jumps. Jump scrolling is enabled by default. > > What this slowness means is that xterm is getting CPU at just the > right moments that it isn't falling behind, so it doesn't jump - which > means X gets all the CPU to redraw, which means your ls/dmesg anything > else that reads from disk[1] doesn't get any CPU. > > Xterm is already functioning as designed - you can't force jump > scrolling to jump more - it is at the mercy of how it gets > scheduled. If there is nothing more in the pipe to draw, it has to > draw. > > These bloody interactive changes to make X more responsive are at the > expense of anything that does *real* work. Harsh words considering it is the timing sensitive nature of xterm that relies on X running out of steam in the old scheduler design to appear smooth. Con ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 5:58 ` Con Kolivas @ 2004-01-06 1:09 ` Peter Osterlund 2004-01-06 1:37 ` Nick Piggin 0 siblings, 1 reply; 61+ messages in thread From: Peter Osterlund @ 2004-01-06 1:09 UTC (permalink / raw) To: Con Kolivas; +Cc: Tim Connors, linux-kernel Con Kolivas <kernel@kolivas.org> writes: > On Sun, 4 Jan 2004 14:32, Tim Connors wrote: > > > Not quite. The scheduler retains high priority for X for longer so it's > > > no new dynamic adjustment of any sort, just better cpu usage by X (which > > > is why it's smoother now at nice 0 than previously). > > > > > > > If either the scheduler or xterm was a bit smarter or > > > > used different thresholds, the problem would go away. It would also > > > > explain why there are people who cannot reproduce it. Perhaps a > > > > somewhat faster or slower system makes the problem go away. Honnestly, > > > > it's the first time that I notice that my xterms are jump-scrolling, it > > > > was so much fluid anyway. > > > > > > Very thorough but not a scheduler problem as far as I'm concerned. Can > > > you not disable smooth scrolling and force jump scrolling? > > > > AFAIK the definition of jump scrolling is that if xterm is falling > > behind, it jumps. Jump scrolling is enabled by default. > > > > What this slowness means is that xterm is getting CPU at just the > > right moments that it isn't falling behind, so it doesn't jump - which > > means X gets all the CPU to redraw, which means your ls/dmesg anything > > else that reads from disk[1] doesn't get any CPU. > > > > Xterm is already functioning as designed - you can't force jump > > scrolling to jump more - it is at the mercy of how it gets > > scheduled. If there is nothing more in the pipe to draw, it has to > > draw. > > > > These bloody interactive changes to make X more responsive are at the > > expense of anything that does *real* work. > > Harsh words considering it is the timing sensitive nature of xterm that relies > on X running out of steam in the old scheduler design to appear smooth. But the scheduler is also far from fair in this situation. If I run perl -e 'for(;;){printf("ddddddddddddddddddddddddddd\n");}' in an xterm, the system enters a steady state where top shows: Cpu(s): 78.3% us, 20.9% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.9% si Mem: 61660k total, 60544k used, 1116k free, 4568k buffers Swap: 104824k total, 22064k used, 82760k free, 17596k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2562 root 15 0 17284 7088 7664 S 39.2 11.5 3:21.20 X 2755 petero 15 0 5652 3260 4528 S 31.7 5.3 1:53.89 xterm 2808 petero 25 0 3424 1248 2744 R 3.9 2.0 0:01.71 perl 2807 petero 16 0 1764 940 1636 R 1.6 1.5 0:00.87 top What happens is that X and xterm start with highest possible interactivity credit and CURRENT_BONUS, because they were mostly idle before the test started. The perl program starts at some PR>15 and slowly climbs to 25. Its interactivity credit remains at 0. As soon as the perl process delivers one line of output to xterm, xterm and later X are scheduled, because they have a smaller priority value than perl. When they have finished scrolling one line, perl is scheduled again and produces another line of output. However, since X and xterm both have HIGH_CREDIT and CURRENT_BONUS == MAX_BONUS, they only get charged 1/10th of their runtime, because of this code in schedule(): if (HIGH_CREDIT(prev)) run_time /= (CURRENT_BONUS(prev) ? : 1); Since both processes spend approximately 50% of their time sleeping, the sleep_avg increase in recalc_task_prio() is more than enough to keep the processes at maximum interactivity level forever. The situation for the perl process is that it always loses the cpu a short time before it would voluntarily go to sleep. The perl process gets approximately 4% cpu, but it wants 4+epsilon percent cpu, so it is considered a cpu hog, and eventually gets maximum punishment (PR=25). The end result is that the perl process, which requires very little cpu time, is considered a cpu hog, and the two real cpu hogs (X and xterm) are considered interactive. This is not a transient state. The situation does not go away until I kill the perl process or start some other cpu hog to disturb the scheduler. -- Peter Osterlund - petero2@telia.com http://w1.894.telia.com/~u89404340 ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-06 1:09 ` Peter Osterlund @ 2004-01-06 1:37 ` Nick Piggin 2004-01-06 2:28 ` Peter Osterlund 0 siblings, 1 reply; 61+ messages in thread From: Nick Piggin @ 2004-01-06 1:37 UTC (permalink / raw) To: Peter Osterlund; +Cc: Con Kolivas, Tim Connors, linux-kernel Peter Osterlund wrote: >Con Kolivas <kernel@kolivas.org> writes: > > >>On Sun, 4 Jan 2004 14:32, Tim Connors wrote: >> >>>>Not quite. The scheduler retains high priority for X for longer so it's >>>>no new dynamic adjustment of any sort, just better cpu usage by X (which >>>>is why it's smoother now at nice 0 than previously). >>>> >>>> >>>>>If either the scheduler or xterm was a bit smarter or >>>>>used different thresholds, the problem would go away. It would also >>>>>explain why there are people who cannot reproduce it. Perhaps a >>>>>somewhat faster or slower system makes the problem go away. Honnestly, >>>>>it's the first time that I notice that my xterms are jump-scrolling, it >>>>>was so much fluid anyway. >>>>> >>>>Very thorough but not a scheduler problem as far as I'm concerned. Can >>>>you not disable smooth scrolling and force jump scrolling? >>>> >>>AFAIK the definition of jump scrolling is that if xterm is falling >>>behind, it jumps. Jump scrolling is enabled by default. >>> >>>What this slowness means is that xterm is getting CPU at just the >>>right moments that it isn't falling behind, so it doesn't jump - which >>>means X gets all the CPU to redraw, which means your ls/dmesg anything >>>else that reads from disk[1] doesn't get any CPU. >>> >>>Xterm is already functioning as designed - you can't force jump >>>scrolling to jump more - it is at the mercy of how it gets >>>scheduled. If there is nothing more in the pipe to draw, it has to >>>draw. >>> >>>These bloody interactive changes to make X more responsive are at the >>>expense of anything that does *real* work. >>> >>Harsh words considering it is the timing sensitive nature of xterm that relies >>on X running out of steam in the old scheduler design to appear smooth. >> > >But the scheduler is also far from fair in this situation. If I run > snip a good analysis... ... but fairness is not about a set of numbers the scheduler gives to each process, its about the amount of CPU time processes are given. In this case I don't know if I find it objectionable that X and xterm are considered interactive and perl considered a CPU hog. What is the actual problem? ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-06 1:37 ` Nick Piggin @ 2004-01-06 2:28 ` Peter Osterlund 2004-01-06 2:50 ` Nick Piggin 2004-01-06 6:27 ` Nick Piggin 0 siblings, 2 replies; 61+ messages in thread From: Peter Osterlund @ 2004-01-06 2:28 UTC (permalink / raw) To: Nick Piggin; +Cc: Con Kolivas, Tim Connors, linux-kernel Nick Piggin <piggin@cyberone.com.au> writes: > Peter Osterlund wrote: > > >But the scheduler is also far from fair in this situation. If I run > > snip a good analysis... > > ... but fairness is not about a set of numbers the scheduler gives to > each process, its about the amount of CPU time processes are given. > > In this case I don't know if I find it objectionable that X and xterm > are considered interactive and perl considered a CPU hog. What is the > actual problem? The problem is that if perl would get only slightly more cpu time, it would get ahead of xterm, which would make this test case run something like 10 times faster than it currently does. (Because xterm switches to jump scrolling when it can't keep up.) I guess it would be possible to fix this by introducing a usleep(10000) at some strategic place in the xterm source code, but I still find it strange that two tasks eating 40% cpu time each are considered interactive, while a task eating 4% is considered a cpu hog, especially since the 4% task never got a chance to prove that it didn't want to steal all cpu time. All that was proven was that it wanted more than 4% of the cpu. Also, while my test case runs, other tasks (such as running "ps" from a network login) are very slow, at least until the extra load makes the scheduler realize that the two tasks eating most of the cpu time should not have maximum priority bonus. -- Peter Osterlund - petero2@telia.com http://w1.894.telia.com/~u89404340 ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-06 2:28 ` Peter Osterlund @ 2004-01-06 2:50 ` Nick Piggin 2004-01-06 6:27 ` Nick Piggin 1 sibling, 0 replies; 61+ messages in thread From: Nick Piggin @ 2004-01-06 2:50 UTC (permalink / raw) To: Peter Osterlund; +Cc: Con Kolivas, Tim Connors, linux-kernel Peter Osterlund wrote: >Nick Piggin <piggin@cyberone.com.au> writes: > > >>Peter Osterlund wrote: >> >> >>>But the scheduler is also far from fair in this situation. If I run >>> >>snip a good analysis... >> >>... but fairness is not about a set of numbers the scheduler gives to >>each process, its about the amount of CPU time processes are given. >> >>In this case I don't know if I find it objectionable that X and xterm >>are considered interactive and perl considered a CPU hog. What is the >>actual problem? >> > >The problem is that if perl would get only slightly more cpu time, it >would get ahead of xterm, which would make this test case run >something like 10 times faster than it currently does. (Because xterm >switches to jump scrolling when it can't keep up.) > >I guess it would be possible to fix this by introducing a >usleep(10000) at some strategic place in the xterm source code, but I >still find it strange that two tasks eating 40% cpu time each are >considered interactive, while a task eating 4% is considered a cpu >hog, especially since the 4% task never got a chance to prove that it >didn't want to steal all cpu time. All that was proven was that it >wanted more than 4% of the cpu. > >Also, while my test case runs, other tasks (such as running "ps" from >a network login) are very slow, at least until the extra load makes >the scheduler realize that the two tasks eating most of the cpu time >should not have maximum priority bonus. > OK yeah you are right. I perl should get more CPU time if it wants it. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-06 2:28 ` Peter Osterlund 2004-01-06 2:50 ` Nick Piggin @ 2004-01-06 6:27 ` Nick Piggin 1 sibling, 0 replies; 61+ messages in thread From: Nick Piggin @ 2004-01-06 6:27 UTC (permalink / raw) To: Peter Osterlund; +Cc: Con Kolivas, Tim Connors, linux-kernel, efault Peter Osterlund wrote: >Nick Piggin <piggin@cyberone.com.au> writes: > > >>Peter Osterlund wrote: >> >> >>>But the scheduler is also far from fair in this situation. If I run >>> >>snip a good analysis... >> >>... but fairness is not about a set of numbers the scheduler gives to >>each process, its about the amount of CPU time processes are given. >> >>In this case I don't know if I find it objectionable that X and xterm >>are considered interactive and perl considered a CPU hog. What is the >>actual problem? >> > >The problem is that if perl would get only slightly more cpu time, it >would get ahead of xterm, which would make this test case run >something like 10 times faster than it currently does. (Because xterm >switches to jump scrolling when it can't keep up.) > >I guess it would be possible to fix this by introducing a >usleep(10000) at some strategic place in the xterm source code, but I >still find it strange that two tasks eating 40% cpu time each are >considered interactive, while a task eating 4% is considered a cpu >hog, especially since the 4% task never got a chance to prove that it >didn't want to steal all cpu time. All that was proven was that it >wanted more than 4% of the cpu. > >Also, while my test case runs, other tasks (such as running "ps" from >a network login) are very slow, at least until the extra load makes >the scheduler realize that the two tasks eating most of the cpu time >should not have maximum priority bonus. > This is what top looks like with my (now fixed) scheduler (priorities go from 0 to 59). In theory I guess it looks like what you want. 920 root 21 -10 88156 17m 79m S 50.8 7.0 11:08.47 XFree86 16762 npiggin 40 0 4804 2396 4356 R 37.9 0.9 0:07.76 xterm 16784 npiggin 23 0 3128 1212 2664 S 9.3 0.5 0:01.93 perl xterm is a CPU hog, XFree86 is half and half, perl is at close to highest priority (which is 20 for a nice 0 process). When using xterm without jump scrolling, it seems to mess up X's scheduler by flooding it with so many small requests. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 3:32 ` Tim Connors 2004-01-04 5:58 ` Con Kolivas @ 2004-01-05 22:25 ` Bryan Whitehead 1 sibling, 0 replies; 61+ messages in thread From: Bryan Whitehead @ 2004-01-05 22:25 UTC (permalink / raw) To: Tim Connors; +Cc: linux-kernel > [1] Others say that it only affects them first time through - after > something is cached, it goes back to normal speed. For me - it is slow > *all* the time. If I pipe it to cat or tail or something, it is a > *lot* quicker. I get slow all the time at home with a single CPU system. At work I have a Dual p4 1.7Ghz XEON... Only when I keep both CPU's busy (example: make -j 4) does it happen on the dual machine. So basically if my dual machine is busy (one CPU has to be pegged or both are over 50% usage) I start getting very crappy xterm/gnome-terminal response. I've been running each xterm/gnome-terminal with screen and when it starts grinding I detach/reattach. -- Bryan Whitehead SysAdmin - JPL - Interferometry and Large Optical Systems Phone: 818 354 2903 driver@jpl.nasa.gov ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 1:42 ` Con Kolivas 2004-01-04 3:32 ` Tim Connors @ 2004-01-04 8:09 ` Soeren Sonnenburg 2004-01-04 8:49 ` Con Kolivas 2004-01-04 8:54 ` Lincoln Dale 1 sibling, 2 replies; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-04 8:09 UTC (permalink / raw) To: Con Kolivas; +Cc: Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 On Sun, 2004-01-04 at 02:42, Con Kolivas wrote: > On Sun, 4 Jan 2004 10:35, Willy Tarreau wrote: > > 6) Conclusion > > ============= > > > > Under 2.4, xterm uses jump scrolling which it does not use by default under > > 2.6 if X responds fast enough. The first dirty solution which comes to mind > > is to renice X to >+10 to slow it a bit so that xterm hits the high water > > level and jumps. > > > > But it's not an effect of the scheduler alone, but a side effect of the > > scheduler and xterm both trying to automatically adjust their behaviour in > > a different manner. > > Not quite. The scheduler retains high priority for X for longer so it's no new > dynamic adjustment of any sort, just better cpu usage by X (which is why it's > smoother now at nice 0 than previously). But why is i.e. running the command a second or third time much faster compared with the first run ? Does X get less priority then ? > > If either the scheduler or xterm was a bit smarter or > > used different thresholds, the problem would go away. It would also explain > > why there are people who cannot reproduce it. Perhaps a somewhat faster or > > slower system makes the problem go away. Honnestly, it's the first time > > that I notice that my xterms are jump-scrolling, it was so much fluid > > anyway. > > Very thorough but not a scheduler problem as far as I'm concerned. Can you not > disable smooth scrolling and force jump scrolling? I think it is a schedulers problem as it works if you run the program in question often enough (dmesg/find/whatever). Suddenly it comes back to full scrolling speed. Judging from the xfree4.3 xterm sources this is the function that gets called when there is something to scroll. And it looks perfectly ok to *me* as it scrolls amount lines at once. So the output of (dmesg/...) seems to be received slower or xterm updates more often than in 2.4. leading to fewer lines to scroll (== amount). It cannot be xterm updating more often as it would a) be faster than and b) amount would be >>> 1 on later on which would lead to high scrolling speed again. So IMHO it must be the output of the program that is coming in slowly. I added a fprintf(stderr, "%d\n", amount); to that code and indeed amount was *always* 1 no matter what I did (it even was 1 when the (dmesg/...) output came in fast). And jump scrolling would take place if amount > 59 in my case... can this still be not a schedulers issue ? from the xfree86-4.3.0 build-tree ./xc/programs/xterm/util.c /* * scrolls the screen by amount lines, erases bottom, doesn't alter * cursor position (i.e. cursor moves down amount relative to text). * All done within the scrolling region, of course. * requires: amount > 0 */ void xtermScroll(TScreen * screen, int amount) { int i = screen->bot_marg - screen->top_marg + 1; int shift; int bot; int refreshtop = 0; int refreshheight; int scrolltop; int scrollheight; if (screen->cursor_state) HideCursor(); if (amount > i) amount = i; if (screen->jumpscroll) { if (screen->scroll_amt > 0) { if (screen->refresh_amt + amount > i) FlushScroll(screen); screen->scroll_amt += amount; screen->refresh_amt += amount; } else { if (screen->scroll_amt < 0) FlushScroll(screen); screen->scroll_amt = amount; screen->refresh_amt = amount; } refreshheight = 0; } else { ScrollSelection(screen, -(amount)); if (amount == i) { ClearScreen(screen); return; } shift = -screen->topline; bot = screen->max_row - shift; scrollheight = i - amount; refreshheight = amount; [...] Looking at that how can it not be a scheduling problem .... Soeren ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 8:09 ` Soeren Sonnenburg @ 2004-01-04 8:49 ` Con Kolivas 2004-01-04 11:13 ` Martin Schlemmer 2004-01-04 8:54 ` Lincoln Dale 1 sibling, 1 reply; 61+ messages in thread From: Con Kolivas @ 2004-01-04 8:49 UTC (permalink / raw) To: Soeren Sonnenburg; +Cc: Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 On Sun, 4 Jan 2004 19:09, Soeren Sonnenburg wrote: > On Sun, 2004-01-04 at 02:42, Con Kolivas wrote: > > Not quite. The scheduler retains high priority for X for longer so it's > > no new dynamic adjustment of any sort, just better cpu usage by X (which > > is why it's smoother now at nice 0 than previously). > > But why is i.e. running the command a second or third time much faster > compared with the first run ? Does X get less priority then ? Yes. X retains high priority for longer than previous scheduler designs. > > > If either the scheduler or xterm was a bit smarter or > > > used different thresholds, the problem would go away. It would also > > > explain why there are people who cannot reproduce it. Perhaps a > > > somewhat faster or slower system makes the problem go away. Honnestly, > > > it's the first time that I notice that my xterms are jump-scrolling, it > > > was so much fluid anyway. > > > > Very thorough but not a scheduler problem as far as I'm concerned. Can > > you not disable smooth scrolling and force jump scrolling? > > I think it is a schedulers problem as it works if you run the program in > question often enough (dmesg/find/whatever). Suddenly it comes back to > full scrolling speed. > > Judging from the xfree4.3 xterm sources this is the function that gets > called when there is something to scroll. And it looks perfectly ok to > *me* as it scrolls amount lines at once. So the output of (dmesg/...) > seems to be received slower or xterm updates more often than in 2.4. > leading to fewer lines to scroll (== amount). It cannot be xterm > updating more often as it would a) be faster than and b) amount would be It's a timing issue. X gets more priority for longer than previously so X gets to do more work. > >>> 1 on later on which would lead to high scrolling speed again. > > So IMHO it must be the output of the program that is coming in slowly. > > I added a fprintf(stderr, "%d\n", amount); to that code and indeed > amount was *always* 1 no matter what I did (it even was 1 when the > (dmesg/...) output came in fast). And jump scrolling would take place if > amount > 59 in my case... can this still be not a schedulers issue ? > > Looking at that how can it not be a scheduling problem .... Scheduling problem, yes; of a sort. Solution by altering the scheduler, no. My guess is that turning the xterm graphic candy up or down will change the balance. Trying to be both gui intensive and a console is where it's happening. On some hardware you are falling on both sides of the fence with 2.6 where previously you would be on one side. Con ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 8:49 ` Con Kolivas @ 2004-01-04 11:13 ` Martin Schlemmer 2004-01-04 11:24 ` Soeren Sonnenburg 2004-01-04 12:45 ` Con Kolivas 0 siblings, 2 replies; 61+ messages in thread From: Martin Schlemmer @ 2004-01-04 11:13 UTC (permalink / raw) To: Con Kolivas Cc: Soeren Sonnenburg, Willy Tarreau, Mark Hahn, Linux Kernel Mailing Lists, gillb4 [-- Attachment #1: Type: text/plain, Size: 1011 bytes --] On Sun, 2004-01-04 at 10:49, Con Kolivas wrote: > > I added a fprintf(stderr, "%d\n", amount); to that code and indeed > > amount was *always* 1 no matter what I did (it even was 1 when the > > (dmesg/...) output came in fast). And jump scrolling would take place if > > amount > 59 in my case... can this still be not a schedulers issue ? > > > > > Looking at that how can it not be a scheduling problem .... > > Scheduling problem, yes; of a sort. > > Solution by altering the scheduler, no. > > My guess is that turning the xterm graphic candy up or down will change the > balance. Trying to be both gui intensive and a console is where it's > happening. On some hardware you are falling on both sides of the fence with > 2.6 where previously you would be on one side. > So its Ok for 'eye candy' to 'lag', but xmms should not skip? Anyhow, its xterm that he have issues with, not gnome-terminal or such with transparency. I smell something ... -- Martin Schlemmer [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 11:13 ` Martin Schlemmer @ 2004-01-04 11:24 ` Soeren Sonnenburg 2004-01-04 12:45 ` Con Kolivas 1 sibling, 0 replies; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-04 11:24 UTC (permalink / raw) To: azarah Cc: Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Sun, 2004-01-04 at 12:13, Martin Schlemmer wrote: > On Sun, 2004-01-04 at 10:49, Con Kolivas wrote: > > > > I added a fprintf(stderr, "%d\n", amount); to that code and indeed > > > amount was *always* 1 no matter what I did (it even was 1 when the > > > (dmesg/...) output came in fast). And jump scrolling would take place if > > > amount > 59 in my case... can this still be not a schedulers issue ? > > > > > > > > Looking at that how can it not be a scheduling problem .... > > > > Scheduling problem, yes; of a sort. > > > > Solution by altering the scheduler, no. > > > > My guess is that turning the xterm graphic candy up or down will change the > > balance. Trying to be both gui intensive and a console is where it's > > happening. On some hardware you are falling on both sides of the fence with > > 2.6 where previously you would be on one side. > > > > So its Ok for 'eye candy' to 'lag', but xmms should not skip? Anyhow, > its xterm that he have issues with, not gnome-terminal or such with > transparency. I smell something ... I would not call it 'lag' if an ls of /usr/bin takes 20 secs vs 1 sec before... I mean the scroll speed limits e.g. the compile speed... Well I now tried xterm, gnome-terminal (gtk2), multi-gnome-terminal (gtk1), xvt, aterm, wterm, Eterm, and yes they are all (except for Eterm) plain text on solid background - so no eye candy. Interestingly Eterm, wterm and aterm seem to not be affected by that issue. Soeren ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 11:13 ` Martin Schlemmer 2004-01-04 11:24 ` Soeren Sonnenburg @ 2004-01-04 12:45 ` Con Kolivas 2004-01-04 14:42 ` Martin Schlemmer 1 sibling, 1 reply; 61+ messages in thread From: Con Kolivas @ 2004-01-04 12:45 UTC (permalink / raw) To: azarah Cc: Soeren Sonnenburg, Willy Tarreau, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Sun, 4 Jan 2004 22:13, Martin Schlemmer wrote: > On Sun, 2004-01-04 at 10:49, Con Kolivas wrote: > > > I added a fprintf(stderr, "%d\n", amount); to that code and indeed > > > amount was *always* 1 no matter what I did (it even was 1 when the > > > (dmesg/...) output came in fast). And jump scrolling would take place > > > if amount > 59 in my case... can this still be not a schedulers issue ? > > > > > > > > > Looking at that how can it not be a scheduling problem .... > > > > Scheduling problem, yes; of a sort. > > > > Solution by altering the scheduler, no. > > > > My guess is that turning the xterm graphic candy up or down will change > > the balance. Trying to be both gui intensive and a console is where it's > > happening. On some hardware you are falling on both sides of the fence > > with 2.6 where previously you would be on one side. > > So its Ok for 'eye candy' to 'lag', but xmms should not skip? Anyhow, > its xterm that he have issues with, not gnome-terminal or such with > transparency. I smell something ... Sigh... Xmms was a simple test case long forgotten but most still think all I did was make an xmms scheduler. Deleting one character from sched.c before all of my patches would make the scheduler ideal for xmms. Any braindead idiot can tune a scheduler for just one application. An application that changes it's behaviour dynamically well in the setting of a particular scheduler, though? Should a scheduler be tuned to suit a coding style or quirk? I should go back to lurking before people start calling me names. This thread has gone long enough for that. If I hadn't said anything it would have died out by now. Instead I'm drawing attention to my fundamentally flawed code. Con ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 12:45 ` Con Kolivas @ 2004-01-04 14:42 ` Martin Schlemmer 2004-01-04 18:40 ` mikeg 2004-01-04 22:58 ` szonyi calin 0 siblings, 2 replies; 61+ messages in thread From: Martin Schlemmer @ 2004-01-04 14:42 UTC (permalink / raw) To: Con Kolivas Cc: Soeren Sonnenburg, Willy Tarreau, Mark Hahn, Linux Kernel Mailing Lists, gillb4 [-- Attachment #1: Type: text/plain, Size: 3611 bytes --] On Sun, 2004-01-04 at 14:45, Con Kolivas wrote: > On Sun, 4 Jan 2004 22:13, Martin Schlemmer wrote: > > On Sun, 2004-01-04 at 10:49, Con Kolivas wrote: > > > > I added a fprintf(stderr, "%d\n", amount); to that code and indeed > > > > amount was *always* 1 no matter what I did (it even was 1 when the > > > > (dmesg/...) output came in fast). And jump scrolling would take place > > > > if amount > 59 in my case... can this still be not a schedulers issue ? > > > > > > > > > > > > Looking at that how can it not be a scheduling problem .... > > > > > > Scheduling problem, yes; of a sort. > > > > > > Solution by altering the scheduler, no. > > > > > > My guess is that turning the xterm graphic candy up or down will change > > > the balance. Trying to be both gui intensive and a console is where it's > > > happening. On some hardware you are falling on both sides of the fence > > > with 2.6 where previously you would be on one side. > > > > So its Ok for 'eye candy' to 'lag', but xmms should not skip? Anyhow, > > its xterm that he have issues with, not gnome-terminal or such with > > transparency. I smell something ... > > Sigh... > > Xmms was a simple test case long forgotten but most still think all I did was > make an xmms scheduler. Deleting one character from sched.c before all of my > patches would make the scheduler ideal for xmms. Any braindead idiot can tune > a scheduler for just one application. Well, its the favorite example 8) > An application that changes it's > behaviour dynamically well in the setting of a particular scheduler, though? > Should a scheduler be tuned to suit a coding style or quirk? > But the scheduler changes to a particular application? I still am of opinion that the current scheduler in mainline 'breaks' priorities ... call it dynamic tuning or whatever you like. Now something gets priority while something else starves. > I should go back to lurking before people start calling me names. This thread > has gone long enough for that. If I hadn't said anything it would have died > out by now. Well, I have stayed out of this for months now, as its always 'they' at fault - that app, or piece of code. Sure, I am one of those whining users, and I have no particular interest in the scheduler code - that is if it behaves like it should. But whatever is in now, just do not behave as expected, and call it a feature or whatever you want, if it deviates the definition, then what should we call it? Or if its a feature, can we have the weirdness in priorities disabled by default with a sysctl or sysfs switch? > Instead I'm drawing attention to my fundamentally flawed code. > The scrolling is but one part. Just starting an app, or running 'vim /etc/fstab' for example takes ages some times, even with minimal load. If xterm, gnome-term, aterm, multi-gnome-term, etc is broken, how do we fix it then? What about some of the other issues? If its a problem with those apps, why is it I still wonder what they are doing wrong, and it not fixed? Do not worry, _I_ will go back to lurking about this issue _again_, but after _once_again_ seeing a issue about this being blown off as being something wrong with 'it', and some facts (you did see that the skipping code for the other user _never_ kicked in) were just ignored, I just could not help myself - sorry. At least I will not experience those issues of the others, and hopefully Nick will not stop his work, or things change too much to adapt his patch. Thanks, -- Martin Schlemmer [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 14:42 ` Martin Schlemmer @ 2004-01-04 18:40 ` mikeg 2004-01-04 22:58 ` szonyi calin 1 sibling, 0 replies; 61+ messages in thread From: mikeg @ 2004-01-04 18:40 UTC (permalink / raw) To: azarah, Con Kolivas Cc: Soeren Sonnenburg, Willy Tarreau, Mark Hahn, Linux Kernel Mailing Lists, gillb4 At 04:42 PM 1/4/2004 +0200, Martin Schlemmer wrote: >Do not worry, _I_ will go back to lurking about this issue _again_, Wrong answer. Jump in with both feet instead (much fun, highly educational). The code's not complicated (the problem space is though). >but after _once_again_ seeing a issue about this being blown off By you personally. >as being something wrong with 'it', and some facts (you did see >that the skipping code for the other user _never_ kicked in) >were just ignored, I just could not help myself - sorry. Don't apologize... scratch that itch :) -Mike ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 14:42 ` Martin Schlemmer 2004-01-04 18:40 ` mikeg @ 2004-01-04 22:58 ` szonyi calin 2004-01-04 23:33 ` Willy Tarreau 2004-01-05 8:26 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Soeren Sonnenburg 1 sibling, 2 replies; 61+ messages in thread From: szonyi calin @ 2004-01-04 22:58 UTC (permalink / raw) To: azarah, Con Kolivas Cc: Soeren Sonnenburg, Willy Tarreau, Mark Hahn, Linux Kernel Mailing Lists, gillb4 --- Martin Schlemmer <azarah@nosferatu.za.org> a écrit : > On Sun, 2004-01-04 at 14:45, Con Kolivas wrote: > > On Sun, 4 Jan 2004 22:13, Martin Schlemmer wrote: > > > On Sun, 2004-01-04 at 10:49, Con Kolivas wrote: > > > > > I added a fprintf(stderr, "%d\n", amount); to that > code and indeed > > > > > amount was *always* 1 no matter what I did (it even > was 1 when the > > > > > (dmesg/...) output came in fast). And jump scrolling > would take place > > > > > if amount > 59 in my case... can this still be not a > schedulers issue ? > > > > > > > > > > > > > > > Looking at that how can it not be a scheduling problem > .... > > > > > > > > Scheduling problem, yes; of a sort. > > > > > > > > Solution by altering the scheduler, no. > > > > > > > > My guess is that turning the xterm graphic candy up or > down will change > > > > the balance. Trying to be both gui intensive and a > console is where it's > > > > happening. On some hardware you are falling on both > sides of the fence > > > > with 2.6 where previously you would be on one side. > > > > > > So its Ok for 'eye candy' to 'lag', but xmms should not > skip? Anyhow, > > > its xterm that he have issues with, not gnome-terminal or > such with > > > transparency. I smell something ... > > > > Sigh... > > > > Xmms was a simple test case long forgotten but most still > think all I did was > > make an xmms scheduler. Deleting one character from sched.c > before all of my > > patches would make the scheduler ideal for xmms. Any > braindead idiot can tune > > a scheduler for just one application. > > Well, its the favorite example 8) > > > An application that changes it's > > behaviour dynamically well in the setting of a particular > scheduler, though? > > Should a scheduler be tuned to suit a coding style or quirk? > > > > > But the scheduler changes to a particular application? I > still am of > opinion that the current scheduler in mainline 'breaks' > priorities ... > call it dynamic tuning or whatever you like. Now something > gets > priority while something else starves. > > > I should go back to lurking before people start calling me > names. This thread > > has gone long enough for that. If I hadn't said anything it > would have died > > out by now. > > Well, I have stayed out of this for months now, as its always > 'they' at > fault - that app, or piece of code. Sure, I am one of those > whining > users, and I have no particular interest in the scheduler code > - that > is if it behaves like it should. But whatever is in now, just > do not > behave as expected, and call it a feature or whatever you > want, if it > deviates the definition, then what should we call it? Or if > its a > feature, can we have the weirdness in priorities disabled by > default > with a sysctl or sysfs switch? > > > Instead I'm drawing attention to my fundamentally flawed > code. > > > > The scrolling is but one part. Just starting an app, or > running > 'vim /etc/fstab' for example takes ages some times, even with > minimal load. If xterm, gnome-term, aterm, multi-gnome-term, > etc is broken, how do we fix it then? What about some of the > other issues? If its a problem with those apps, why is it I > still > wonder what they are doing wrong, and it not fixed? > > Do not worry, _I_ will go back to lurking about this issue > _again_, > but after _once_again_ seeing a issue about this being blown > off > as being something wrong with 'it', and some facts (you did > see > that the skipping code for the other user _never_ kicked in) > were just ignored, I just could not help myself - sorry. > > At least I will not experience those issues of the others, and > hopefully Nick will not stop his work, or things change too > much > to adapt his patch. > how much free memory do you have when this happens ? I had a similar problem. It was easily reproducive doing a du -sh / and then trying to do other things. It didn't happend all the time but most of the time Doing a echo 16384 >/proc/sys/vm/min_free_kbytes seems to help the kernel remember that it has some swap and he *has* to use it in some cases > > Thanks, > > -- > Martin Schlemmer > > ATTACHMENT part 2 application/pgp-signature name=signature.asc ===== -- A mouse is a device used to point at the xterm you want to type in. Kim Alm on a.s.r. _________________________________________________________________ Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français ! Yahoo! Mail : http://fr.mail.yahoo.com ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 22:58 ` szonyi calin @ 2004-01-04 23:33 ` Willy Tarreau 2004-01-04 23:44 ` Valdis.Kletnieks 2004-01-04 23:47 ` Mike Fedyk 2004-01-05 8:26 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Soeren Sonnenburg 1 sibling, 2 replies; 61+ messages in thread From: Willy Tarreau @ 2004-01-04 23:33 UTC (permalink / raw) To: szonyi calin Cc: azarah, Con Kolivas, Soeren Sonnenburg, Willy Tarreau, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Sun, Jan 04, 2004 at 11:58:27PM +0100, szonyi calin wrote: > how much free memory do you have when this happens ? > I had > a similar problem. It was easily reproducive doing > a du -sh / and then trying to do other things. > It didn't happend all the time but most of the time It's not the problem here. always between 200 and 400 MB free. BTW, the system is not swapping when this happens. The scrolling is very smooth and relatively fast (about 100 lines/s) which is enough to understand that X eats all the CPU scrolling one line at a time. I have yet to understand why 'ls|cat' behaves differently, but fortunately it works and it has already saved me some useful time. Cheers, Willy ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 23:33 ` Willy Tarreau @ 2004-01-04 23:44 ` Valdis.Kletnieks 2004-01-04 23:47 ` Mike Fedyk 1 sibling, 0 replies; 61+ messages in thread From: Valdis.Kletnieks @ 2004-01-04 23:44 UTC (permalink / raw) To: Willy Tarreau; +Cc: Linux Kernel Mailing Lists [-- Attachment #1: Type: text/plain, Size: 716 bytes --] On Mon, 05 Jan 2004 00:33:12 +0100, Willy Tarreau said: > at a time. I have yet to understand why 'ls|cat' behaves > differently, but fortunately it works and it has already saved I suspect that ls and cat do different buffering to their outputs, and/or the fact there's now 4 processes (ls, cat, xterm, Xserver) rather than just 3. End result is that things wake up in a different order and happen to schedule better. (For instance, ls may be able to make progress while cat is blocked waiting for the xterm to read the next block, etc). I remember at least some versions of 'dd' would fork off a sub-process so there would be a reader side and a writer side with a shared memory buffer, for just that reason. [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 23:33 ` Willy Tarreau 2004-01-04 23:44 ` Valdis.Kletnieks @ 2004-01-04 23:47 ` Mike Fedyk 2004-01-05 8:39 ` Soeren Sonnenburg ` (2 more replies) 1 sibling, 3 replies; 61+ messages in thread From: Mike Fedyk @ 2004-01-04 23:47 UTC (permalink / raw) To: Willy Tarreau Cc: szonyi calin, azarah, Con Kolivas, Soeren Sonnenburg, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > at a time. I have yet to understand why 'ls|cat' behaves > differently, but fortunately it works and it has already saved > me some useful time. cat probably does some buffering for you, and sends the output to xterm in larger blocks. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 23:47 ` Mike Fedyk @ 2004-01-05 8:39 ` Soeren Sonnenburg 2004-01-05 20:38 ` Martin Schlemmer 2004-01-05 9:18 ` Soeren Sonnenburg 2004-01-05 9:50 ` Kenneth Johansson 2 siblings, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-05 8:39 UTC (permalink / raw) To: Mike Fedyk Cc: Willy Tarreau, szonyi calin, azarah, Con Kolivas, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > at a time. I have yet to understand why 'ls|cat' behaves > > differently, but fortunately it works and it has already saved > > me some useful time. > > cat probably does some buffering for you, and sends the output to xterm in > larger blocks. interestingly running ls on a remote machine in a directory with a similiar amount of files (local xterm with ssh connection to that machine) is also as fast as this ls | cat workaround... Soeren ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-05 8:39 ` Soeren Sonnenburg @ 2004-01-05 20:38 ` Martin Schlemmer 0 siblings, 0 replies; 61+ messages in thread From: Martin Schlemmer @ 2004-01-05 20:38 UTC (permalink / raw) To: Soeren Sonnenburg Cc: Mike Fedyk, Willy Tarreau, szonyi calin, Con Kolivas, Mark Hahn, Linux Kernel Mailing Lists, gillb4 [-- Attachment #1: Type: text/plain, Size: 1297 bytes --] On Mon, 2004-01-05 at 10:39, Soeren Sonnenburg wrote: > On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > > at a time. I have yet to understand why 'ls|cat' behaves > > > differently, but fortunately it works and it has already saved > > > me some useful time. > > > > cat probably does some buffering for you, and sends the output to xterm in > > larger blocks. > > interestingly running ls on a remote machine in a directory with a > similiar amount of files (local xterm with ssh connection to that > machine) is also as fast as this ls | cat workaround... > Maybe it is because the process generating the output, and the xterm is not on the same box? The X server gets too much time, so the two childs (xterm and ls/whatever) do not get enough time? And that might be why renice +10 `pidof X` helps? Although that do not seem right, as the process level seems the same: xterm->bash->ssh vs xterm->bash->ls Might be because the startup time is ruled out (maybe that is the big issue - startup of child processes?). Could it be that the 'ls | cat' situation now again influence startup times (now it is xterm->bash->ls->cat) if above could be taken as an reason? -- Martin Schlemmer [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 23:47 ` Mike Fedyk 2004-01-05 8:39 ` Soeren Sonnenburg @ 2004-01-05 9:18 ` Soeren Sonnenburg 2004-01-05 17:20 ` Martin Schlemmer 2004-01-05 9:50 ` Kenneth Johansson 2 siblings, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-05 9:18 UTC (permalink / raw) To: Mike Fedyk Cc: Willy Tarreau, szonyi calin, azarah, Con Kolivas, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > at a time. I have yet to understand why 'ls|cat' behaves > > differently, but fortunately it works and it has already saved > > me some useful time. > > cat probably does some buffering for you, and sends the output to xterm in > larger blocks. yes indeed, judging from the cat source it does chose optimal buffer size, here 1024 byte... so it reads/writes larger chunks... and jump scrolling takes place... Hmmhh, Soeren ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-05 9:18 ` Soeren Sonnenburg @ 2004-01-05 17:20 ` Martin Schlemmer 2004-01-05 17:21 ` Willy Tarreau 0 siblings, 1 reply; 61+ messages in thread From: Martin Schlemmer @ 2004-01-05 17:20 UTC (permalink / raw) To: Soeren Sonnenburg Cc: Mike Fedyk, Willy Tarreau, szonyi calin, Con Kolivas, Mark Hahn, Linux Kernel Mailing Lists, gillb4 [-- Attachment #1: Type: text/plain, Size: 814 bytes --] On Mon, 2004-01-05 at 11:18, Soeren Sonnenburg wrote: > On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > > at a time. I have yet to understand why 'ls|cat' behaves > > > differently, but fortunately it works and it has already saved > > > me some useful time. > > > > cat probably does some buffering for you, and sends the output to xterm in > > larger blocks. > > yes indeed, judging from the cat source it does chose optimal buffer > size, here 1024 byte... so it reads/writes larger chunks... and jump > scrolling takes place... > I cannot reboot right now, so have wrong kernel for testing, but could anyone see what happens if you start X reniced to +10 or such? Maybe some other numbers? -- Martin Schlemmer [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-05 17:20 ` Martin Schlemmer @ 2004-01-05 17:21 ` Willy Tarreau 0 siblings, 0 replies; 61+ messages in thread From: Willy Tarreau @ 2004-01-05 17:21 UTC (permalink / raw) To: Martin Schlemmer Cc: Soeren Sonnenburg, Mike Fedyk, Willy Tarreau, szonyi calin, Con Kolivas, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Mon, Jan 05, 2004 at 07:20:08PM +0200, Martin Schlemmer wrote: > On Mon, 2004-01-05 at 11:18, Soeren Sonnenburg wrote: > > On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > > > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > > > at a time. I have yet to understand why 'ls|cat' behaves > > > > differently, but fortunately it works and it has already saved > > > > me some useful time. > > > > > > cat probably does some buffering for you, and sends the output to xterm in > > > larger blocks. > > > > yes indeed, judging from the cat source it does chose optimal buffer > > size, here 1024 byte... so it reads/writes larger chunks... and jump > > scrolling takes place... > > > > I cannot reboot right now, so have wrong kernel for testing, but could > anyone see what happens if you start X reniced to +10 or such? Maybe > some other numbers? I posted such tests with numbers in a previous mail in this thread. IIRC, renicing X to +10 was indeed a work-around for most cases. Cheers, Willy ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 23:47 ` Mike Fedyk 2004-01-05 8:39 ` Soeren Sonnenburg 2004-01-05 9:18 ` Soeren Sonnenburg @ 2004-01-05 9:50 ` Kenneth Johansson 2004-01-05 10:17 ` Soeren Sonnenburg 2004-04-02 18:22 ` solved (was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!) Soeren Sonnenburg 2 siblings, 2 replies; 61+ messages in thread From: Kenneth Johansson @ 2004-01-05 9:50 UTC (permalink / raw) To: Mike Fedyk Cc: Willy Tarreau, szonyi calin, azarah, Con Kolivas, Soeren Sonnenburg, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > at a time. I have yet to understand why 'ls|cat' behaves > > differently, but fortunately it works and it has already saved > > me some useful time. > > cat probably does some buffering for you, and sends the output to xterm in > larger blocks. you can try with "ls |dd bs=1" I also see this problem but it is not constant. I noticed that "ps ax" sometimes takes like 10 times longer than usual. But I can only get this in a gnome-terminal not in xterm. The problem is that it should really not be that big difference when the load of the system is the same. One thing to run is to start 4 xterm and run this little thing in everyone. while true ;do ls ;done In a perfect world they should all appear to scroll at the same time. This is not what is happening on my system. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-05 9:50 ` Kenneth Johansson @ 2004-01-05 10:17 ` Soeren Sonnenburg 2004-04-02 18:22 ` solved (was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!) Soeren Sonnenburg 1 sibling, 0 replies; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-05 10:17 UTC (permalink / raw) To: Kenneth Johansson Cc: Mike Fedyk, Willy Tarreau, szonyi calin, azarah, Con Kolivas, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Mon, 2004-01-05 at 10:50, Kenneth Johansson wrote: > On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > > at a time. I have yet to understand why 'ls|cat' behaves > > > differently, but fortunately it works and it has already saved > > > me some useful time. > > > > cat probably does some buffering for you, and sends the output to xterm in > > larger blocks. > > you can try with "ls |dd bs=1" yes it is slow as #$%$##@ when incresing to say bs=100 it is as fast as it used to be. > I also see this problem but it is not constant. I noticed that "ps ax" > sometimes takes like 10 times longer than usual. But I can only get this > in a gnome-terminal not in xterm. The problem is that it should really > not be that big difference when the load of the system is the same. well I can get it in both here. just any program that produces output... Soeren. ^ permalink raw reply [flat|nested] 61+ messages in thread
* solved (was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!) 2004-01-05 9:50 ` Kenneth Johansson 2004-01-05 10:17 ` Soeren Sonnenburg @ 2004-04-02 18:22 ` Soeren Sonnenburg 2004-04-03 5:35 ` Tim Connors 1 sibling, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-04-02 18:22 UTC (permalink / raw) To: Kenneth Johansson Cc: Mike Fedyk, Willy Tarreau, szonyi calin, azarah, Con Kolivas, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Mon, 2004-01-05 at 10:50, Kenneth Johansson wrote: > On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > > at a time. I have yet to understand why 'ls|cat' behaves > > > differently, but fortunately it works and it has already saved > > > me some useful time. > > > > cat probably does some buffering for you, and sends the output to xterm in > > larger blocks. > > you can try with "ls |dd bs=1" > > I also see this problem but it is not constant. I noticed that "ps ax" > sometimes takes like 10 times longer than usual. But I can only get this > in a gnome-terminal not in xterm. The problem is that it should really > not be that big difference when the load of the system is the same. Ok, there is indeed an issue in the *terminals. As above pointed out buffering the programs output helps. Also a usleep of 5ms in the read loop of the *terms would help. I fixed this issue in multi-gnome-terminal by using a buffer of 32kb. It is filled as long as there is input comming in within 10ms. If the buffer is full or 10ms passed the buffer is written out to the screen. This makes it also 2-3 times faster on kernel 2.4. So in my eyes it is definitely not a scheduler bug, but in every single terminal out there. However it seems as if kernel 2.6 s scheduling is so fast that data can be continously outputted. And the busy loop in the terminal makes it eat up all cpu time instead of jump scrolling take place. static void zvt_term_readdata (gpointer data, gint fd, GdkInputCondition condition) { [...] + while ( (count>0) && (select_retval==1) && (total_count<32768) ) + { + count=0; + int maxread=32768-total_count; + if (maxread>4096) + maxread=4096; + + count = read (fd, &buffer[total_count], maxread); + saveerrno=errno; [...] + if (count>0) + total_count+=count; + + FD_ZERO(&rfds); + FD_SET(fd, &rfds); + tv.tv_sec = 0; + tv.tv_usec = 10000; + select_retval = select(fd+1, &rfds, NULL, NULL, &tv); [...] + } Soeren. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: solved (was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!) 2004-04-02 18:22 ` solved (was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!) Soeren Sonnenburg @ 2004-04-03 5:35 ` Tim Connors 2004-04-03 6:06 ` Tim Connors 0 siblings, 1 reply; 61+ messages in thread From: Tim Connors @ 2004-04-03 5:35 UTC (permalink / raw) To: linux-kernel Soeren Sonnenburg <kernel@nn7.de> said on Fri, 02 Apr 2004 20:22:12 +0200: > On Mon, 2004-01-05 at 10:50, Kenneth Johansson wrote: > > On Mon, 2004-01-05 at 00:47, Mike Fedyk wrote: > > > On Mon, Jan 05, 2004 at 12:33:12AM +0100, Willy Tarreau wrote: > > > > at a time. I have yet to understand why 'ls|cat' behaves > > > > differently, but fortunately it works and it has already saved > > > > me some useful time. > > > > > > cat probably does some buffering for you, and sends the output to xterm in > > > larger blocks. > > > > you can try with "ls |dd bs=1" > > > > I also see this problem but it is not constant. I noticed that "ps ax" > > sometimes takes like 10 times longer than usual. But I can only get this > > in a gnome-terminal not in xterm. The problem is that it should really > > not be that big difference when the load of the system is the same. > > Ok, there is indeed an issue in the *terminals. As above pointed out > buffering the programs output helps. Also a usleep of 5ms in the read > loop of the *terms would help. > > I fixed this issue in multi-gnome-terminal by using a buffer of 32kb. > It is filled as long as there is input comming in within 10ms. > If the buffer is full or 10ms passed the buffer is written out to the > screen. This makes it also 2-3 times faster on kernel 2.4. A factor of 2 or 3 though? In 2.4, to ls -lA my home directory with its 510 files, took less than 0.5 sec. Currently, buffering via cat in 2.6 takes 0.5 sec. Just straight ls -lA takes 6 seconds or so. Does your factor of 3 bring you up to what you were seeing in 2.4, or do you still have a regression? -- TimC -- http://astronomy.swin.edu.au/staff/tconnors/ Some witty text here, can be any number of lines long ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: solved (was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!) 2004-04-03 5:35 ` Tim Connors @ 2004-04-03 6:06 ` Tim Connors 2004-04-03 14:11 ` Jamie Lokier 0 siblings, 1 reply; 61+ messages in thread From: Tim Connors @ 2004-04-03 6:06 UTC (permalink / raw) To: linux-kernel Tim Connors <tconnors+linuxkernel1080970205@astro.swin.edu.au> said on Sat, 3 Apr 2004 15:35:29 +1000: > Soeren Sonnenburg <kernel@nn7.de> said on Fri, 02 Apr 2004 20:22:12 +0200: > > Ok, there is indeed an issue in the *terminals. As above pointed out > > buffering the programs output helps. Also a usleep of 5ms in the read > > loop of the *terms would help. I forgot to mention that I see other 2.6 scheduler regressions too. I am on ADSL, and run wwwoffle to cache my www feed. In mozilla, running through the proxy will quite often crawl - you can see it displaying tags and words one by one. I have been unable to work out under what circumstances this is triggered, but it is easily understood given that mozilla on this box chews CPU (especially when rendering whilst downloading), as does X. So this is much like the xterm situation - 2 CPU chewing things, interacting with a third short lived low CPU job (ls or some wwwoffle fork) that the other two are relying on (mozilla and xterm directly, X indirectly) This is my reason for not thinking this is an xterm bug. The scheduler looks inherently fragile. To help you work out my datapoint in relation to someone elses, my box is a 500MHz AMD KII, now running 2.6.4. The video card is in no way accelarated (crappy old nvidia card), so X likes to chew CPU easily. -- TimC -- http://astronomy.swin.edu.au/staff/tconnors/ Cult: (n) a small, unpopular religion. Religion: (n) a large, popular cult. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: solved (was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!) 2004-04-03 6:06 ` Tim Connors @ 2004-04-03 14:11 ` Jamie Lokier 0 siblings, 0 replies; 61+ messages in thread From: Jamie Lokier @ 2004-04-03 14:11 UTC (permalink / raw) To: Tim Connors; +Cc: linux-kernel Tim Connors wrote: > So this is much like the xterm situation - 2 CPU chewing things, > interacting with a third short lived low CPU job (ls or some > wwwoffle fork) that the other two are relying on (mozilla and xterm > directly, X indirectly) > > This is my reason for not thinking this is an xterm bug. The scheduler > looks inherently fragile. It looks like scheduler fragility to me too. I see the same with Gnome Terminal. Most of the time, ls or cat are fast, with jump scrolling. But occasionally they are much slower. There are two stable scheduling modes here - rapid switching with a few characters moved, or slow switching with lots of characters moved. We'd like the former to decay spontaneously to the latter, quickly. I like the observation that the terminal program can solve this problem by introducing a short delay after receiving a few bytes. However, that's not feasible for X itself, which must draw things very soon after receiving the commands so that animations are smooth when that's intended, which happens when a client deliberately sleeps rather than being preempted by having just sent a command to X. I think the problem is that the scheduler is aggressively preempting a task which has just written through a pipe/terminal/socket, when the task at the other end becomes ready to run as a result. If the writing task is still runnable, usually you want the writing task to continue for a little while more. The scheduler is getting that right most times on my box with ls+Gnome Terminal+X, but occasionally gets stuck in a mode where it consistently gets it wrong until all the programs become idle again. Then it recovers. This mode is quite rare, once every few days. > To help you work out my datapoint in relation to someone elses, my box > is a 500MHz AMD KII, now running 2.6.4. The video card is in no way > accelarated (crappy old nvidia card), so X likes to chew CPU easily. Dual Athlon 1500 MHz, Matrox Millenium video card. -- Jamie ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 22:58 ` szonyi calin 2004-01-04 23:33 ` Willy Tarreau @ 2004-01-05 8:26 ` Soeren Sonnenburg 1 sibling, 0 replies; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-05 8:26 UTC (permalink / raw) To: szonyi calin Cc: azarah, Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel Mailing Lists, gillb4 On Sun, 2004-01-04 at 23:58, szonyi calin wrote: [...] > how much free memory do you have when this happens ? > I had > a similar problem. It was easily reproducive doing > a du -sh / and then trying to do other things. > It didn't happend all the time but most of the time > > Doing a > echo 16384 >/proc/sys/vm/min_free_kbytes > seems to help the kernel remember that it has some swap and he > *has* to use it in some cases Hmmm... this machine has 1G memory and it happens after fresh reboots.... so memory is not the issue. Soeren. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 8:09 ` Soeren Sonnenburg 2004-01-04 8:49 ` Con Kolivas @ 2004-01-04 8:54 ` Lincoln Dale 2004-01-04 9:17 ` Nick Piggin 2004-01-04 10:11 ` Soeren Sonnenburg 1 sibling, 2 replies; 61+ messages in thread From: Lincoln Dale @ 2004-01-04 8:54 UTC (permalink / raw) To: Soeren Sonnenburg Cc: Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 At 07:09 PM 4/01/2004, Soeren Sonnenburg wrote: [..] >Looking at that how can it not be a scheduling problem .... out of interest, have you tried to see how 2.4.xx compares when compiled with HZ set to 1000? (or conversely, 2.6 compiled with HZ set to 100) cheers, lincoln. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 8:54 ` Lincoln Dale @ 2004-01-04 9:17 ` Nick Piggin 2004-01-04 10:24 ` Soeren Sonnenburg ` (2 more replies) 2004-01-04 10:11 ` Soeren Sonnenburg 1 sibling, 3 replies; 61+ messages in thread From: Nick Piggin @ 2004-01-04 9:17 UTC (permalink / raw) To: Lincoln Dale Cc: Soeren Sonnenburg, Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 Lincoln Dale wrote: > At 07:09 PM 4/01/2004, Soeren Sonnenburg wrote: > [..] > >> Looking at that how can it not be a scheduling problem .... > > > out of interest, have you tried to see how 2.4.xx compares when > compiled with HZ set to 1000? > (or conversely, 2.6 compiled with HZ set to 100) > Or, out of interest, an alternate scheduler? http://www.kerneltrap.org/~npiggin/w29p2.gz (applies 2.6.1-rc1-mm1, please renice X to -10 or so) ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 9:17 ` Nick Piggin @ 2004-01-04 10:24 ` Soeren Sonnenburg 2004-01-04 11:12 ` Mike Fedyk 2004-01-04 11:46 ` Nicks's scheduler's OK [was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!] Willy Tarreau 2004-01-04 12:07 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Willy Tarreau 2 siblings, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-04 10:24 UTC (permalink / raw) To: Nick Piggin Cc: Lincoln Dale, Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 [...] > Or, out of interest, an alternate scheduler? > > http://www.kerneltrap.org/~npiggin/w29p2.gz > (applies 2.6.1-rc1-mm1, please renice X to -10 or so) Thats nothing *I* can try out as I am on the powerpc benh tree. Soeren. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 10:24 ` Soeren Sonnenburg @ 2004-01-04 11:12 ` Mike Fedyk 2004-01-04 11:17 ` Soeren Sonnenburg 2004-01-04 11:19 ` Willy Tarreau 0 siblings, 2 replies; 61+ messages in thread From: Mike Fedyk @ 2004-01-04 11:12 UTC (permalink / raw) To: Soeren Sonnenburg Cc: Nick Piggin, Lincoln Dale, Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 On Sun, Jan 04, 2004 at 11:24:39AM +0100, Soeren Sonnenburg wrote: > [...] > > Or, out of interest, an alternate scheduler? > > > > http://www.kerneltrap.org/~npiggin/w29p2.gz > > (applies 2.6.1-rc1-mm1, please renice X to -10 or so) > > Thats nothing *I* can try out as I am on the powerpc benh tree. > Says who? The scheduler isn't platform specific. Nick, do you have any per arch defines in your patch? ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 11:12 ` Mike Fedyk @ 2004-01-04 11:17 ` Soeren Sonnenburg 2004-01-04 11:20 ` Mike Fedyk 2004-01-04 11:19 ` Willy Tarreau 1 sibling, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-04 11:17 UTC (permalink / raw) To: Mike Fedyk Cc: Nick Piggin, Lincoln Dale, Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 On Sun, 2004-01-04 at 12:12, Mike Fedyk wrote: > On Sun, Jan 04, 2004 at 11:24:39AM +0100, Soeren Sonnenburg wrote: > > [...] > > > Or, out of interest, an alternate scheduler? > > > > > > http://www.kerneltrap.org/~npiggin/w29p2.gz > > > (applies 2.6.1-rc1-mm1, please renice X to -10 or so) > > > > Thats nothing *I* can try out as I am on the powerpc benh tree. > > > > Says who? The scheduler isn't platform specific. Nick, do you have any per > arch defines in your patch? Says me, as quite a lot of stuff does not apply cleanly... and probably same with when applying -mm1. S ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 11:17 ` Soeren Sonnenburg @ 2004-01-04 11:20 ` Mike Fedyk 0 siblings, 0 replies; 61+ messages in thread From: Mike Fedyk @ 2004-01-04 11:20 UTC (permalink / raw) To: Soeren Sonnenburg Cc: Nick Piggin, Lincoln Dale, Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 On Sun, Jan 04, 2004 at 12:17:10PM +0100, Soeren Sonnenburg wrote: > Says me, as quite a lot of stuff does not apply cleanly... and probably > same with when applying -mm1. Does -mm1 compile on ppc for you? If not, then post some compile errors to the list... ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 11:12 ` Mike Fedyk 2004-01-04 11:17 ` Soeren Sonnenburg @ 2004-01-04 11:19 ` Willy Tarreau 2004-01-05 0:48 ` Nick Piggin 1 sibling, 1 reply; 61+ messages in thread From: Willy Tarreau @ 2004-01-04 11:19 UTC (permalink / raw) To: Soeren Sonnenburg, Nick Piggin, Lincoln Dale, Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 On Sun, Jan 04, 2004 at 03:12:57AM -0800, Mike Fedyk wrote: > On Sun, Jan 04, 2004 at 11:24:39AM +0100, Soeren Sonnenburg wrote: > > [...] > > > Or, out of interest, an alternate scheduler? > > > > > > http://www.kerneltrap.org/~npiggin/w29p2.gz > > > (applies 2.6.1-rc1-mm1, please renice X to -10 or so) > > > > Thats nothing *I* can try out as I am on the powerpc benh tree. > > > > Says who? The scheduler isn't platform specific. Nick, do you have any per > arch defines in your patch? I found slight changes to arch specific files, but IMHO this should not be the problem. But AFAIR, benh's ppc patches are somewhat complete and may introduce conflicts. BTW, for Nick, the patch didn't compile, I had to change sched_clock() definition from unsigned long long to unsigned long in arch/i386/kernel/timers/timer_tsc.c. Don't know if it was the right thing to do, but it compiles and boots. Cheers, Willy ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 11:19 ` Willy Tarreau @ 2004-01-05 0:48 ` Nick Piggin 0 siblings, 0 replies; 61+ messages in thread From: Nick Piggin @ 2004-01-05 0:48 UTC (permalink / raw) To: Willy Tarreau Cc: Soeren Sonnenburg, Lincoln Dale, Con Kolivas, Mark Hahn, Linux Kernel, gillb4 Willy Tarreau wrote: >On Sun, Jan 04, 2004 at 03:12:57AM -0800, Mike Fedyk wrote: > >>On Sun, Jan 04, 2004 at 11:24:39AM +0100, Soeren Sonnenburg wrote: >> >>>[...] >>> >>>>Or, out of interest, an alternate scheduler? >>>> >>>>http://www.kerneltrap.org/~npiggin/w29p2.gz >>>>(applies 2.6.1-rc1-mm1, please renice X to -10 or so) >>>> >>>Thats nothing *I* can try out as I am on the powerpc benh tree. >>> >>> >>Says who? The scheduler isn't platform specific. Nick, do you have any per >>arch defines in your patch? >> > >I found slight changes to arch specific files, but IMHO this should not be >the problem. But AFAIR, benh's ppc patches are somewhat complete and may >introduce conflicts. > The problem is just that the mm tree is currently carrying some intrusive (patch wise, not functionality wise) scheduler stuff which will go to linus which is why I am working from there. So the patch will only apply cleanly on the mm tree, but only because of the cleanups + minor fixes. Nothing fundamental. > >BTW, for Nick, the patch didn't compile, I had to change sched_clock() >definition from unsigned long long to unsigned long in >arch/i386/kernel/timers/timer_tsc.c. Don't know if it was the right thing to >do, but it compiles and boots. > Yeah thats right ^ permalink raw reply [flat|nested] 61+ messages in thread
* Nicks's scheduler's OK [was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!] 2004-01-04 9:17 ` Nick Piggin 2004-01-04 10:24 ` Soeren Sonnenburg @ 2004-01-04 11:46 ` Willy Tarreau 2004-01-04 12:07 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Willy Tarreau 2 siblings, 0 replies; 61+ messages in thread From: Willy Tarreau @ 2004-01-04 11:46 UTC (permalink / raw) To: Nick Piggin Cc: Lincoln Dale, Soeren Sonnenburg, Con Kolivas, Mark Hahn, Linux Kernel, gillb4 On Sun, Jan 04, 2004 at 08:17:24PM +1100, Nick Piggin wrote: > > Or, out of interest, an alternate scheduler? > > http://www.kerneltrap.org/~npiggin/w29p2.gz > (applies 2.6.1-rc1-mm1, please renice X to -10 or so) Nick's scheduler seems rather interesting. X is nearly insensible to renice as it was in 2.4. I cannot get a slow scrolling anymore except with xterm +j. And the differences in time between renice -15 and renice +10 are about 5%, which is perfectly acceptable to me. I got back the 2.4 behaviour (= a usable desktop). Now recompiling Con's "noint" patch on 2.6.0 for reference. Cheers, Willy 1) X not reniced. ================= w29p2$ time xterm -e seq 1 5000 real 0m0.487s user 0m0.244s sys 0m0.075s w29p2$ time xterm -e sh -c "ls -l incoming tmp" real 0m1.000s user 0m0.491s sys 0m0.106s 2) Now renicing X to -10 as suggested by Nick ============================================= w29p2$ time xterm -e sh -c "ls -l incoming tmp" real 0m0.998s user 0m0.476s sys 0m0.126s w29p2$ time xterm -e seq 1 5000 real 0m0.420s user 0m0.276s sys 0m0.082s 3) Now renicing X to +10 to compare with my previous tests ========================================================== w29p2$ time xterm -e seq 1 5000 real 0m0.528s user 0m0.282s sys 0m0.056s w29p2$ time xterm -e sh -c "ls -l incoming tmp" real 0m0.967s user 0m0.498s sys 0m0.111s w29p2$ time find incoming tmp |wc -l 204276 real 0m0.937s user 0m0.299s sys 0m0.593s w29p2$ time xterm -e sh -c "find incoming tmp" real 0m21.368s user 0m5.838s sys 0m1.341s top - 12:31:23 up 11 min, 6 users, load average: 0.59, 0.68, 0.39 Tasks: 61 total, 2 running, 59 sleeping, 0 stopped, 0 zombie Cpu(s): 29.1% user, 6.8% system, 64.1% nice, 0.0% idle, 0.0% IO-wait Mem: 515248k total, 231160k used, 284088k free, 99172k buffers Swap: 265064k total, 0k used, 265064k free, 45328k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ Command 302 root 50 10 21500 11m 11m R 63.9 2.3 1:41.67 X 475 root 37 0 4764 2660 3844 S 31.0 0.5 0:01.68 xterm 476 willy 39 0 1420 540 1252 S 3.9 0.1 0:00.23 find 1 root 25 0 348 192 316 S 0.0 0.0 0:05.06 init 2 root 41 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0 3 root 12 -10 0 0 0 S 0.0 0.0 0:00.20 events/0 4 root 16 -10 0 0 0 S 0.0 0.0 0:00.03 kblockd/0 4) Same with renice -15 : ========================= w29p2$ time xterm -e sh -c "find incoming tmp" real 0m19.147s user 0m5.085s sys 0m1.238s top - 12:33:33 up 13 min, 6 users, load average: 0.65, 0.68, 0.42 Tasks: 61 total, 4 running, 57 sleeping, 0 stopped, 0 zombie Cpu(s): 93.1% user, 6.9% system, 0.0% nice, 0.0% idle, 0.0% IO-wait Mem: 515248k total, 233464k used, 281784k free, 99540k buffers Swap: 265064k total, 0k used, 265064k free, 45992k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ Command 302 root 24 -15 21500 11m 11m R 66.2 2.3 2:08.25 X 482 root 40 0 4764 2660 3844 R 29.2 0.5 0:01.50 xterm 483 willy 37 0 1420 540 1252 R 4.9 0.1 0:00.25 find 1 root 26 0 348 192 316 S 0.0 0.0 0:05.06 init 2 root 41 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0 3 root 12 -10 0 0 0 S 0.0 0.0 0:00.21 events/0 4 root 15 -10 0 0 0 S 0.0 0.0 0:00.03 kblockd/0 5 root 39 0 0 0 0 S 0.0 0.0 0:00.00 pdflush 6 root 23 0 0 0 0 S 0.0 0.0 0:00.00 pdflush ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 9:17 ` Nick Piggin 2004-01-04 10:24 ` Soeren Sonnenburg 2004-01-04 11:46 ` Nicks's scheduler's OK [was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!] Willy Tarreau @ 2004-01-04 12:07 ` Willy Tarreau 2004-01-05 0:51 ` Nick Piggin 2 siblings, 1 reply; 61+ messages in thread From: Willy Tarreau @ 2004-01-04 12:07 UTC (permalink / raw) To: Nick Piggin Cc: Lincoln Dale, Soeren Sonnenburg, Con Kolivas, Mark Hahn, Linux Kernel, gillb4 Now testing Con's noint patch against 2.6.0. It returns somewhat simmilar results to Nick's w29p2, and behaves normally. The only noticeable difference is that a simple task like "while :; do :; done&" eats about 100ms each second, so if you start 10 of these, you're able to type only once a second (tested). But I understand that this 'dumbness' was exactly the goal of this patch. I think that I'll try to use 2.6 + Nick's scheduler for some time on my notebook to get an overall idea on how it behaves. BTW, Nick, does your patch rely on -mm1 exclusive features, or would it be possible to back-port it to plain 2.6 ? Cheers, Willy 1) X not reniced. ================= noint$ time xterm -e seq 1 5000 real 0m0.548s user 0m0.267s sys 0m0.052s noint$ time xterm -e sh -c "ls -l incoming tmp" real 0m0.974s user 0m0.481s sys 0m0.091s 2) Now renicing X to -10 ======================== noint$ time xterm -e sh -c "ls -l incoming tmp" real 0m0.997s user 0m0.471s sys 0m0.101s noint$ time xterm -e seq 1 5000 real 0m0.390s user 0m0.261s sys 0m0.058s 3) Now renicing X to +10 to compare with my previous tests ========================================================== noint$ time xterm -e seq 1 5000 real 0m0.452s user 0m0.257s sys 0m0.070s noint$ time xterm -e sh -c "ls -l incoming tmp" real 0m0.956s user 0m0.441s sys 0m0.091s noint$ time find incoming tmp |wc -l 204276 real 0m0.914s user 0m0.257s sys 0m0.653s noint$ time xterm -e sh -c "find incoming tmp" real 0m23.107s user 0m5.652s sys 0m1.631s top - 12:56:42 up 9 min, 5 users, load average: 0.61, 0.54, 0.26 Tasks: 59 total, 3 running, 56 sleeping, 0 stopped, 0 zombie Cpu(s): 24.8% user, 8.9% system, 66.3% nice, 0.0% idle, 0.0% IO-wait Mem: 515292k total, 217816k used, 297476k free, 94344k buffers Swap: 265064k total, 0k used, 265064k free, 45236k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ Command 239 root 30 10 23632 13m 11m R 66.2 2.7 0:11.11 X 374 root 20 0 4764 2660 3844 S 24.7 0.5 0:01.22 xterm 375 willy 20 0 1420 540 1252 S 7.9 0.1 0:00.35 find 373 root 20 0 1692 940 1556 R 1.0 0.2 0:00.08 top 4) Same with renice -15 : ========================= noint$ time xterm -e sh -c "find incoming tmp" real 0m22.622s user 0m5.681s sys 0m1.807s top - 12:57:45 up 10 min, 5 users, load average: 0.33, 0.48, 0.26 Tasks: 59 total, 3 running, 56 sleeping, 0 stopped, 0 zombie Cpu(s): 88.8% user, 11.2% system, 0.0% nice, 0.0% idle, 0.0% IO-wait Mem: 515292k total, 217880k used, 297412k free, 94404k buffers Swap: 265064k total, 0k used, 265064k free, 45236k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ Command 239 root 5 -15 23632 13m 11m S 64.3 2.7 0:25.55 X 380 root 20 0 4764 2660 3844 R 26.1 0.5 0:00.82 xterm 381 willy 20 0 1420 540 1252 S 7.5 0.1 0:00.23 find ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 12:07 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Willy Tarreau @ 2004-01-05 0:51 ` Nick Piggin 2004-01-05 18:37 ` Willy Tarreau 0 siblings, 1 reply; 61+ messages in thread From: Nick Piggin @ 2004-01-05 0:51 UTC (permalink / raw) To: Willy Tarreau Cc: Lincoln Dale, Soeren Sonnenburg, Con Kolivas, Mark Hahn, Linux Kernel, gillb4 Willy Tarreau wrote: >Now testing Con's noint patch against 2.6.0. It returns somewhat simmilar >results to Nick's w29p2, and behaves normally. The only noticeable difference >is that a simple task like "while :; do :; done&" eats about 100ms each second, >so if you start 10 of these, you're able to type only once a second (tested). >But I understand that this 'dumbness' was exactly the goal of this patch. >I think that I'll try to use 2.6 + Nick's scheduler for some time on my >notebook to get an overall idea on how it behaves. > >BTW, Nick, does your patch rely on -mm1 exclusive features, or would it be >possible to back-port it to plain 2.6 ? > No, you should be able to port it to 2.6 quite easily if you just apply the scheduler patches from -mm1 to it first. I'll do a proper release in the next day or so and include patches for both trees. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-05 0:51 ` Nick Piggin @ 2004-01-05 18:37 ` Willy Tarreau 2004-01-06 0:33 ` Nick Piggin 0 siblings, 1 reply; 61+ messages in thread From: Willy Tarreau @ 2004-01-05 18:37 UTC (permalink / raw) To: Nick Piggin Cc: Lincoln Dale, Soeren Sonnenburg, Con Kolivas, Mark Hahn, Linux Kernel, gillb4 Hi Nick, during a kernel compilation today while browsing the net on another virtual screen, I noticed extremely long delays to redraw the screen. My setup is the following : - kernel 2.6.1-rc1-mm1 + w29p2 - X either reniced to -10 or 0 (doesnt make a difference) - window maker - opera open on a fairly simple page (marc.theaimsgroup.com) in VS 1 - 2 xterms on VS2 - 4 xterms in VS3, one of which runs make -j 3 bzImage, another one 'top'. -> going from VS3 to VS2 then VS1 takes up to 5 or 6 seconds redrawing parts of windows on each screen. I also noticed that make -j 8 dep blocked the xterm it was run in for about a third of a second about twice a second. It does not seem to slow the process however since it takes the same amount of time as make -j 8 dep >/dev/null. Cheers, Willy ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-05 18:37 ` Willy Tarreau @ 2004-01-06 0:33 ` Nick Piggin 0 siblings, 0 replies; 61+ messages in thread From: Nick Piggin @ 2004-01-06 0:33 UTC (permalink / raw) To: Willy Tarreau Cc: Lincoln Dale, Soeren Sonnenburg, Con Kolivas, Mark Hahn, Linux Kernel, gillb4 Willy Tarreau wrote: >Hi Nick, > >during a kernel compilation today while browsing the net on another >virtual screen, I noticed extremely long delays to redraw the screen. >My setup is the following : > - kernel 2.6.1-rc1-mm1 + w29p2 > - X either reniced to -10 or 0 (doesnt make a difference) > - window maker > - opera open on a fairly simple page (marc.theaimsgroup.com) in VS 1 > - 2 xterms on VS2 > - 4 xterms in VS3, one of which runs make -j 3 bzImage, another one 'top'. > >-> going from VS3 to VS2 then VS1 takes up to 5 or 6 seconds redrawing > parts of windows on each screen. I also noticed that make -j 8 dep > blocked the xterm it was run in for about a third of a second about > twice a second. It does not seem to slow the process however since > it takes the same amount of time as make -j 8 dep >/dev/null. > Hi Willy, Yeah the w29p2 patch you got is broken. I'll fix it. ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 8:54 ` Lincoln Dale 2004-01-04 9:17 ` Nick Piggin @ 2004-01-04 10:11 ` Soeren Sonnenburg 2004-01-05 10:31 ` venom 1 sibling, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-04 10:11 UTC (permalink / raw) To: Lincoln Dale; +Cc: Con Kolivas, Willy Tarreau, Mark Hahn, Linux Kernel, gillb4 On Sun, 2004-01-04 at 09:54, Lincoln Dale wrote: > At 07:09 PM 4/01/2004, Soeren Sonnenburg wrote: > [..] > >Looking at that how can it not be a scheduling problem .... > > out of interest, have you tried to see how 2.4.xx compares when compiled > with HZ set to 1000? > (or conversely, 2.6 compiled with HZ set to 100) assuming you mean changing the HZ value in include/param.h to 1000/100 yes 2.4 with HZ=1000 is fine and 2.6 with HZ=100 still #%$@$^&!! Soeren ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 10:11 ` Soeren Sonnenburg @ 2004-01-05 10:31 ` venom 0 siblings, 0 replies; 61+ messages in thread From: venom @ 2004-01-05 10:31 UTC (permalink / raw) To: Soeren Sonnenburg; +Cc: Linux Kernel On Sun, 4 Jan 2004, Soeren Sonnenburg wrote: > > out of interest, have you tried to see how 2.4.xx compares when compiled > > with HZ set to 1000? > > (or conversely, 2.6 compiled with HZ set to 100) > > assuming you mean changing the HZ value in include/param.h to 1000/100 > yes 2.4 with HZ=1000 is fine and 2.6 with HZ=100 still #%$@$^&!! > mmhhh! depends... On a DB that has to write often big, long data streams HZ=100 on a 2.6.0 kernel allows a better performance instead of HZ=1000 (no kernel preemption). on a DB that has to read often small, short data streams HZ=1000 is better than HZ=100 with a 2.6.0 kernel. Luigi ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 21:00 ` Con Kolivas 2004-01-03 21:10 ` Soeren Sonnenburg @ 2004-01-03 21:18 ` Willy Tarreau 1 sibling, 0 replies; 61+ messages in thread From: Willy Tarreau @ 2004-01-03 21:18 UTC (permalink / raw) To: Con Kolivas; +Cc: Soeren Sonnenburg, Mark Hahn, Linux Kernel Hi Con, On Sun, Jan 04, 2004 at 08:00:06AM +1100, Con Kolivas wrote: > There is a BASH bug that Linus noticed brought out by the more sensitive > timing in 2.6. The BASH developer has been informed it is there and it is > fixed in the latest version. Perhaps you're both seeing that. Check the lkml > archives. I don't think it has anything to do with the bash bug, because it only involved pipes creation. Bash creates no pipe when you simply launch 'ls -l' in an xterm. It seems that I'll start some new compilations this evening just to refresh my mind on this problem... Cheers, Willy ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! @ 2004-01-03 21:39 Bob Gill 0 siblings, 0 replies; 61+ messages in thread From: Bob Gill @ 2004-01-03 21:39 UTC (permalink / raw) To: Linux Kernel Mailing List Hi. I've seen this too. It happens when I run ls -al or ls -alR on an uncached directory (usually after boot). If the directory is big, I wait 3-4 seconds for a listing. I've updated bash with yum (currently I'm using bash 2.05b on Fedora with 2.6.1-pre1). It probably doesn't help that my drives are slow (5400 rpm). Doing regression performance testing on the drives using hdparm helps, but the problem still exists. ^ permalink raw reply [flat|nested] 61+ messages in thread
[parent not found: <Pine.LNX.4.44.0401031402210.24942-100000@coffee.psychology.mcmaster.ca>]
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! [not found] <Pine.LNX.4.44.0401031402210.24942-100000@coffee.psychology.mcmaster.ca> @ 2004-01-03 19:07 ` Soeren Sonnenburg 0 siblings, 0 replies; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-03 19:07 UTC (permalink / raw) To: Mark Hahn; +Cc: Linux Kernel On Sat, 2004-01-03 at 20:03, Mark Hahn wrote: > > Anyone else with that problem - ideas of the cause ? > > the 2.6 scheduler tries to be much more adaptive. doesn't it look like > it's decided to make some wrong decisions wrt scheduling in your first run? yeah, I think so... but as generating output in a shell is a very common thing to do there should either be an option to turn that unwanted behaviour off or to fix this issue... Soeren ^ permalink raw reply [flat|nested] 61+ messages in thread
* xterm scrolling speed - scheduling weirdness in 2.6 ?! @ 2004-01-03 18:52 Soeren Sonnenburg 2004-01-03 19:19 ` Willy Tarreau 0 siblings, 1 reply; 61+ messages in thread From: Soeren Sonnenburg @ 2004-01-03 18:52 UTC (permalink / raw) To: Linux Kernel [-- Attachment #1: Type: text/plain, Size: 905 bytes --] Hi... I notice a severe speed regression with 2.6 and working in e.g. xterm on X11 (no framebuffer device in use). It is all about scrolling speed. I used to type e.g. just dmesg in a fullscreen xterm (or MGT) in 2.4 and the whole output was there lightning fast, i.e. in 2.4: ======= time dmesg real 0m0.096s user 0m0.007s sys 0m0.005s in 2.6 (first run) ====== time dmesg [...] real 0m3.054s user 0m0.008s sys 0m0.008s in 2.6 (second run) =================== dmesg >d time cat d real 0m0.096s user 0m0.002s sys 0m0.005s in 2.6 (third run) ================== time dmesg real 0m0.095s user 0m0.007s sys 0m0.004s So it is like 30 times slower on 2.6. when running for the first time... this also happens if I do e.g. find ./ and watch the output pass by... This is without preemption on powerpc... Anyone else with that problem - ideas of the cause ? Soeren. [-- Attachment #2: config.gz --] [-- Type: application/x-gzip, Size: 7048 bytes --] ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 18:52 Soeren Sonnenburg @ 2004-01-03 19:19 ` Willy Tarreau 2004-01-04 20:47 ` Peter Chubb 0 siblings, 1 reply; 61+ messages in thread From: Willy Tarreau @ 2004-01-03 19:19 UTC (permalink / raw) To: Soeren Sonnenburg; +Cc: Linux Kernel Hi ! On Sat, Jan 03, 2004 at 07:52:36PM +0100, Soeren Sonnenburg wrote: > So it is like 30 times slower on 2.6. when running for the first time... > this also happens if I do e.g. find ./ and watch the output pass by... > > This is without preemption on powerpc... > > Anyone else with that problem - ideas of the cause ? Yes, already reported a while ago. I was having this problem with ls in large directories where it sometimes took more than 10 seconds to scroll. It is clearly a scheduler problem, but most people don't seem concerned by this one since the most important to their ears is that xmms should not skip during make -j $UINT_MAX :-/ It's a bit of a shame since other than that, it seems stable enough for daily use on a desktop, but this problem is too much annoying for me to definitely switch to 2.6. I wanted to give a try to Nick's scheduler, but didn't have time yet. Regards, Willy ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-03 19:19 ` Willy Tarreau @ 2004-01-04 20:47 ` Peter Chubb 2004-01-04 20:54 ` Willy TARREAU 0 siblings, 1 reply; 61+ messages in thread From: Peter Chubb @ 2004-01-04 20:47 UTC (permalink / raw) To: Willy Tarreau; +Cc: Soeren Sonnenburg, Linux Kernel >>>>> "Willy" == Willy Tarreau <willy@w.ods.org> writes: Willy> Hi ! On Sat, Jan 03, 2004 at 07:52:36PM +0100, Soeren Willy> Sonnenburg wrote: >> So it is like 30 times slower on 2.6. when running for the first >> time... this also happens if I do e.g. find ./ and watch the >> output pass by... >> >> This is without preemption on powerpc... >> >> Anyone else with that problem - ideas of the cause ? I see a very similar problem ... seems like a process doing disc I.O isn't being woken up fast enough after the I/O completes. For some processes, allowing interrupts back on (hdparm -u1) helps; for others, switching to the deadline elevator helps; neither are complete solutions. PeterC ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 20:47 ` Peter Chubb @ 2004-01-04 20:54 ` Willy TARREAU 2004-01-05 3:46 ` Peter Chubb 0 siblings, 1 reply; 61+ messages in thread From: Willy TARREAU @ 2004-01-04 20:54 UTC (permalink / raw) To: Peter Chubb; +Cc: Willy Tarreau, Soeren Sonnenburg, Linux Kernel Hi Peter, On Mon, Jan 05, 2004 at 07:47:43AM +1100, Peter Chubb wrote: > >> So it is like 30 times slower on 2.6. when running for the first > >> time... this also happens if I do e.g. find ./ and watch the > >> output pass by... > >> > >> This is without preemption on powerpc... > >> > >> Anyone else with that problem - ideas of the cause ? > > I see a very similar problem ... seems like a process doing disc I.O > isn't being woken up fast enough after the I/O completes. > > For some processes, allowing interrupts back on (hdparm -u1) helps; > for others, switching to the deadline elevator helps; neither are > complete solutions. This is not I/O related since the problem happens even with simple programs such as dmesg and seq. Cheers, willy ^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: xterm scrolling speed - scheduling weirdness in 2.6 ?! 2004-01-04 20:54 ` Willy TARREAU @ 2004-01-05 3:46 ` Peter Chubb 0 siblings, 0 replies; 61+ messages in thread From: Peter Chubb @ 2004-01-05 3:46 UTC (permalink / raw) To: Willy TARREAU; +Cc: Soeren Sonnenburg, Linux Kernel >>>>> "Willy" == Willy TARREAU <willy@w.ods.org> writes: > >> So it is like 30 times slower on 2.6. when running for the first > >> time... this also happens if I do e.g. find ./ and watch the > >> output pass by... >> >> For some processes, allowing interrupts back on (hdparm -u1) helps; >> for others, switching to the deadline elevator helps; neither are >> complete solutions. Willy> This is not I/O related since the problem happens even with Willy> simple programs such as dmesg and seq. Yes ... the first time you run a program it's fetched from disc and shows a significant slowdown. Second and subsequent times, it runs at normal speed. At least, that's what *I'm* seeing. Peter C ^ permalink raw reply [flat|nested] 61+ messages in thread
end of thread, other threads:[~2004-04-03 14:11 UTC | newest] Thread overview: 61+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <Pine.LNX.4.44.0401031439060.24942-100000@coffee.psychology.mcmaster.ca> 2004-01-03 20:19 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Soeren Sonnenburg 2004-01-03 21:00 ` Con Kolivas 2004-01-03 21:10 ` Soeren Sonnenburg 2004-01-03 21:15 ` Con Kolivas 2004-01-03 23:35 ` Willy Tarreau 2004-01-04 0:11 ` Soeren Sonnenburg 2004-01-04 1:42 ` Con Kolivas 2004-01-04 3:32 ` Tim Connors 2004-01-04 5:58 ` Con Kolivas 2004-01-06 1:09 ` Peter Osterlund 2004-01-06 1:37 ` Nick Piggin 2004-01-06 2:28 ` Peter Osterlund 2004-01-06 2:50 ` Nick Piggin 2004-01-06 6:27 ` Nick Piggin 2004-01-05 22:25 ` Bryan Whitehead 2004-01-04 8:09 ` Soeren Sonnenburg 2004-01-04 8:49 ` Con Kolivas 2004-01-04 11:13 ` Martin Schlemmer 2004-01-04 11:24 ` Soeren Sonnenburg 2004-01-04 12:45 ` Con Kolivas 2004-01-04 14:42 ` Martin Schlemmer 2004-01-04 18:40 ` mikeg 2004-01-04 22:58 ` szonyi calin 2004-01-04 23:33 ` Willy Tarreau 2004-01-04 23:44 ` Valdis.Kletnieks 2004-01-04 23:47 ` Mike Fedyk 2004-01-05 8:39 ` Soeren Sonnenburg 2004-01-05 20:38 ` Martin Schlemmer 2004-01-05 9:18 ` Soeren Sonnenburg 2004-01-05 17:20 ` Martin Schlemmer 2004-01-05 17:21 ` Willy Tarreau 2004-01-05 9:50 ` Kenneth Johansson 2004-01-05 10:17 ` Soeren Sonnenburg 2004-04-02 18:22 ` solved (was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!) Soeren Sonnenburg 2004-04-03 5:35 ` Tim Connors 2004-04-03 6:06 ` Tim Connors 2004-04-03 14:11 ` Jamie Lokier 2004-01-05 8:26 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Soeren Sonnenburg 2004-01-04 8:54 ` Lincoln Dale 2004-01-04 9:17 ` Nick Piggin 2004-01-04 10:24 ` Soeren Sonnenburg 2004-01-04 11:12 ` Mike Fedyk 2004-01-04 11:17 ` Soeren Sonnenburg 2004-01-04 11:20 ` Mike Fedyk 2004-01-04 11:19 ` Willy Tarreau 2004-01-05 0:48 ` Nick Piggin 2004-01-04 11:46 ` Nicks's scheduler's OK [was Re: xterm scrolling speed - scheduling weirdness in 2.6 ?!] Willy Tarreau 2004-01-04 12:07 ` xterm scrolling speed - scheduling weirdness in 2.6 ?! Willy Tarreau 2004-01-05 0:51 ` Nick Piggin 2004-01-05 18:37 ` Willy Tarreau 2004-01-06 0:33 ` Nick Piggin 2004-01-04 10:11 ` Soeren Sonnenburg 2004-01-05 10:31 ` venom 2004-01-03 21:18 ` Willy Tarreau 2004-01-03 21:39 Bob Gill [not found] <Pine.LNX.4.44.0401031402210.24942-100000@coffee.psychology.mcmaster.ca> 2004-01-03 19:07 ` Soeren Sonnenburg -- strict thread matches above, loose matches on Subject: below -- 2004-01-03 18:52 Soeren Sonnenburg 2004-01-03 19:19 ` Willy Tarreau 2004-01-04 20:47 ` Peter Chubb 2004-01-04 20:54 ` Willy TARREAU 2004-01-05 3:46 ` Peter Chubb
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).