RSDL-mm 0.28

RSDL-mm 0.28

[Matt Mackall]

I've tested -mm2 against -mm2+noyield and -mm2+rsdl+noyield. The
noyield patch simply makes the sched_yield syscall return immediately.
Xorg and all tests are run at nice 0.

Loads:
 memload: constant memcpy of 16MB buffer
 execload: constant re-exec of a trivial shell script
 forkload: constant fork and exit of a trivial shell script
 make -j 5: hot-cache kernel build without ccache
 make -j 5 ccache: hot-cache kernel build with ccache

Tests:
 beryl - 3D window manager, wiggle windows, spin desktop, etc.
 galeon - web browser, rapidly scrolling long web pages by grabbing
    the scroll bar
 mp3 - XMMS on a FUSE sshfs over wireless (during all tests)
 terminal - responsiveness of ssh and local terminal sessions
 mouse - responsiveness of mouse pointer

Results:
 great = completely smooth
 good = fully responsive
 ok = visible latency
 bad = becomes difficult to use (or mp3 skips)
 awful = make it stop, please

          -mm2        -mm2+noyield   rsdl+noyield
no load
 beryl    great       great          great
 galeon   good        good           good
 mp3      good        good           good
 terminal good        good           good
 mouse    good        good           good
memload x10
 beryl    awful/bad   great          good
 galeon   good        good           ok/good
 mp3      good        good           good
 terminal good        good           good
 mouse    good        good           good
execload x10
 beryl    awful/bad   bad/good       good
 galeon   good        bad/good       ok/good
 mp3      good        bad            good
 terminal good        bad/good       good
 mouse    good        bad/good       good
forkload x10
 beryl    good        good           great
 galeon   good        good           ok/good
 mp3      good        good           good
 terminal good        good           ok/good
 mouse    good        good           good
make -j 5
 beryl    ok          good           good/great
 galeon   good        good           ok/good
 mp3      good        good           good
 terminal good        good           good
 mouse    good        good           good
make -j 5 ccache
 beryl    ok          good           awful
 galeon   good        good           bad
 mp3      good        good           bad
 terminal good        good           bad/ok
 mouse    good        good           bad/ok

make -j 5
real      8m1.857s    8m50.659s      8m9.282s
user      7m19.127s   8m3.494s       7m30.740s
sys       0m30.910s   0m33.722s      0m29.542s

make -j 5 ccache
real      2m6.182s    2m19.032s      2m1.832s
user      1m39.466s   1m48.787s      1m37.250s
sys       0m19.741s   0m22.993s      0m20.109s

There's a substantial performance hit for not yield, so we probably
want to investigate alternate semantics for it. It seems reasonable
for apps to say "let me not hog the CPU" without completely expiring
them. Imagine you're in the front of the line (aka queue) and you
spend a moment fumbling for your wallet. The polite thing to do is to
let the next guy in front. But with the current sched_yield, you go
all the way to the back of the line.

RSDL makes most of the noyield hit back in normal make and then some
with ccache. Impressive. But ccache is still destroying interactivity
somehow. The ccache effect is fairly visible even with non-parallel
'make'.

Also note I could occassionally trigger nasty multi-second pauses with
-mm2+noyield under exectest that didn't show up elsewhere. That's
probably a bug in the mainline scheduler.

-- 
Mathematics is the supreme nostalgia of our time.

Re: RSDL-mm 0.28

[Con Kolivas]

On 11/03/07, Matt Mackall <mpm@selenic.com> wrote:
> I've tested -mm2 against -mm2+noyield and -mm2+rsdl+noyield. The
> noyield patch simply makes the sched_yield syscall return immediately.
> Xorg and all tests are run at nice 0.
>
> Loads:
>  memload: constant memcpy of 16MB buffer
>  execload: constant re-exec of a trivial shell script
>  forkload: constant fork and exit of a trivial shell script
>  make -j 5: hot-cache kernel build without ccache
>  make -j 5 ccache: hot-cache kernel build with ccache
>
> Tests:
>  beryl - 3D window manager, wiggle windows, spin desktop, etc.
>  galeon - web browser, rapidly scrolling long web pages by grabbing
>     the scroll bar
>  mp3 - XMMS on a FUSE sshfs over wireless (during all tests)
>  terminal - responsiveness of ssh and local terminal sessions
>  mouse - responsiveness of mouse pointer
>
> Results:
>  great = completely smooth
>  good = fully responsive
>  ok = visible latency
>  bad = becomes difficult to use (or mp3 skips)
>  awful = make it stop, please
>
>           -mm2        -mm2+noyield   rsdl+noyield
> no load
>  beryl    great       great          great
>  galeon   good        good           good
>  mp3      good        good           good
>  terminal good        good           good
>  mouse    good        good           good
> memload x10
>  beryl    awful/bad   great          good
>  galeon   good        good           ok/good
>  mp3      good        good           good
>  terminal good        good           good
>  mouse    good        good           good
> execload x10
>  beryl    awful/bad   bad/good       good
>  galeon   good        bad/good       ok/good
>  mp3      good        bad            good
>  terminal good        bad/good       good
>  mouse    good        bad/good       good
> forkload x10
>  beryl    good        good           great
>  galeon   good        good           ok/good
>  mp3      good        good           good
>  terminal good        good           ok/good
>  mouse    good        good           good
> make -j 5
>  beryl    ok          good           good/great
>  galeon   good        good           ok/good
>  mp3      good        good           good
>  terminal good        good           good
>  mouse    good        good           good
> make -j 5 ccache
>  beryl    ok          good           awful
>  galeon   good        good           bad
>  mp3      good        good           bad
>  terminal good        good           bad/ok
>  mouse    good        good           bad/ok
>
> make -j 5
> real      8m1.857s    8m50.659s      8m9.282s
> user      7m19.127s   8m3.494s       7m30.740s
> sys       0m30.910s   0m33.722s      0m29.542s
>
> make -j 5 ccache
> real      2m6.182s    2m19.032s      2m1.832s
> user      1m39.466s   1m48.787s      1m37.250s
> sys       0m19.741s   0m22.993s      0m20.109s

Thanks very much for that comprehensive summary and testing!

> There's a substantial performance hit for not yield, so we probably
> want to investigate alternate semantics for it. It seems reasonable
> for apps to say "let me not hog the CPU" without completely expiring
> them. Imagine you're in the front of the line (aka queue) and you
> spend a moment fumbling for your wallet. The polite thing to do is to
> let the next guy in front. But with the current sched_yield, you go
> all the way to the back of the line.

Well... are you advocating we change sched_yield semantics to a
gentler form? This is a cinch to implement but I know how Ingo feels
about this. It will only encourage more lax coding using sched_yield
instead of proper blocking (see huge arguments with the ldap people on
this one who insist it's impossible not to use yield).

> RSDL makes most of the noyield hit back in normal make and then some
> with ccache. Impressive. But ccache is still destroying interactivity
> somehow. The ccache effect is fairly visible even with non-parallel
> 'make'.

Ok I don't think there's any actual accounting problem here per se
(although I did just recently post a bugfix for rsdl however I think
that's unrelated). What I think is going on in the ccache testcase is
that all the work is being offloaded to kernel threads reading/writing
to/from the filesystem and the make is not getting any actual cpu
time. This is "worked around" in mainline thanks to the testing for
sleeping on uninterruptible sleep in the interactivity estimator. What
I suspect is happening is kernel threads that are running nice -5 are
doing all the work on make's behalf in the setting of ccache since it
is mostly i/o bound. The reason for -nice values on kernel threads is
questionable anyway. Can you try renicing your kernel threads all to
nice 0 and see what effect that has? Obviously this doesn't need a
recompile, but is simple enough to implement in kthread code as a new
default.

> Also note I could occassionally trigger nasty multi-second pauses with
> -mm2+noyield under exectest that didn't show up elsewhere. That's
> probably a bug in the mainline scheduler.

Ew. It's probably not a bug but a good example of some of the
starvation scenarios we're hitting on mainline (hence the need for a
rewrite ;))

Thanks!

---
ck

Re: RSDL-mm 0.28

[Andrew Morton]

> On Sun, 11 Mar 2007 13:28:22 +1100 "Con Kolivas" <kernel@kolivas.org> wrote:
> Well... are you advocating we change sched_yield semantics to a
> gentler form?

>From a practical POV: our present yield() behaviour is so truly awful that
it's basically always a bug to use it.  This probably isn't a good thing.

So yes, I do think that we should have a rethink and try to come up with
behaviour which is more in accord with what application developers expect
yield() to do.

otoh,

a) we should have done this five years ago.  Instead, we've spent that
   time training userspace programmers to not use yield(), so perhaps
   there's little to be gained in changing it now.

b) if we _were_ to change yield(), people would use it more, and their
   applications would of course suck bigtime when run on earlier 2.6
   kernels.


Bottom line: we've had a _lot_ of problems with the new yield() semantics. 
We effectively broke back-compatibility by changing its behaviour a lot,
and we can't really turn around and blame application developers for that.

Re: RSDL-mm 0.28

[Andrew Morton]

> On Sun, 11 Mar 2007 14:59:28 +1100 Con Kolivas <kernel@kolivas.org> wrote:
> > Bottom line: we've had a _lot_ of problems with the new yield() semantics.
> > We effectively broke back-compatibility by changing its behaviour a lot,
> > and we can't really turn around and blame application developers for that.
> 
> So... I would take it that's a yes for a recommendation with respect to 
> implementing a new yield() ? A new scheduler is as good a time as any to do 
> it.

I guess so.  We'd, err, need to gather Ingo's input ;)

Perhaps a suitable way of doing this would be to characterise then emulate
the 2.4 behaviour.  As long as it turns out to be vaguely sensible.

Re: RSDL-mm 0.28

[William Lee Irwin III]

On Sun, 11 Mar 2007 13:28:22 +1100 "Con Kolivas" <kernel@kolivas.org> wrote:
>> Well... are you advocating we change sched_yield semantics to a
>> gentler form?

On Sat, Mar 10, 2007 at 07:16:14PM -0800, Andrew Morton wrote:
> From a practical POV: our present yield() behaviour is so truly awful that
> it's basically always a bug to use it.  This probably isn't a good thing.
> So yes, I do think that we should have a rethink and try to come up with
> behaviour which is more in accord with what application developers expect
> yield() to do.

ISTR that apps varied wrt. their expectations for yield(). Some,
particularly those using it to implement multi-tiered userspace locks,
really did expect to go all the way to the back of the queue. (Rumor has
it that realtime apps break otherwise.) Others wanted a kinder, gentler,
"mistress, please hit me, but not too hard" opportunity to let someone
else have a little cpu time, particularly when userspace is spinning in
some sort of busywait. In both scenarios something very much against
the latest trends of Linux kernel politics is done by userspace.


On Sat, Mar 10, 2007 at 07:16:14PM -0800, Andrew Morton wrote:
> otoh,
> a) we should have done this five years ago.  Instead, we've spent that
>    time training userspace programmers to not use yield(), so perhaps
>    there's little to be gained in changing it now.
> b) if we _were_ to change yield(), people would use it more, and their
>    applications would of course suck bigtime when run on earlier 2.6
>    kernels.
> Bottom line: we've had a _lot_ of problems with the new yield() semantics. 
> We effectively broke back-compatibility by changing its behaviour a lot,
> and we can't really turn around and blame application developers for that.

My dumb idea would be to break out new syscall. One for the kinder,
gentler version, one for the serious version. Or otherwise pass an
argument indicating the expected behavior. Then a dumb app can be
LD_PRELOAD'd into calling whatever makes it run fastest.


-- wli

Re: RSDL-mm 0.28

[Con Kolivas]

On Sunday 11 March 2007 14:39, Andrew Morton wrote:
> > On Sun, 11 Mar 2007 14:59:28 +1100 Con Kolivas <kernel@kolivas.org> wrote:
> > > Bottom line: we've had a _lot_ of problems with the new yield()
> > > semantics. We effectively broke back-compatibility by changing its
> > > behaviour a lot, and we can't really turn around and blame application
> > > developers for that.
> >
> > So... I would take it that's a yes for a recommendation with respect to
> > implementing a new yield() ? A new scheduler is as good a time as any to
> > do it.
>
> I guess so.  We'd, err, need to gather Ingo's input ;)

cc'ed. Don't you hate timezones?

> Perhaps a suitable way of doing this would be to characterise then emulate
> the 2.4 behaviour.  As long as it turns out to be vaguely sensible.

It's really very simple. We just go the end of the current queued priority on 
the same array instead of swapping to the expired array; ie we do what 
realtime tasks currently do. It works fine here locally afaict.

-- 
-ck

Re: RSDL-mm 0.28

[Con Kolivas]

On Sunday 11 March 2007 14:16, Andrew Morton wrote:
> > On Sun, 11 Mar 2007 13:28:22 +1100 "Con Kolivas" <kernel@kolivas.org>
> > wrote: Well... are you advocating we change sched_yield semantics to a
> > gentler form?
> >
> >From a practical POV: our present yield() behaviour is so truly awful that
>
> it's basically always a bug to use it.  This probably isn't a good thing.
>
> So yes, I do think that we should have a rethink and try to come up with
> behaviour which is more in accord with what application developers expect
> yield() to do.
>
> otoh,
>
> a) we should have done this five years ago.  Instead, we've spent that
>    time training userspace programmers to not use yield(), so perhaps
>    there's little to be gained in changing it now.
>
> b) if we _were_ to change yield(), people would use it more, and their
>    applications would of course suck bigtime when run on earlier 2.6
>    kernels.
>
>
> Bottom line: we've had a _lot_ of problems with the new yield() semantics.
> We effectively broke back-compatibility by changing its behaviour a lot,
> and we can't really turn around and blame application developers for that.

So... I would take it that's a yes for a recommendation with respect to 
implementing a new yield() ? A new scheduler is as good a time as any to do 
it.

-- 
-ck

Re: RSDL-mm 0.28

[Matt Mackall]

On Sun, Mar 11, 2007 at 01:28:22PM +1100, Con Kolivas wrote:
> >make -j 5 ccache
> > beryl    ok          good           awful
> > galeon   good        good           bad
> > mp3      good        good           bad
> > terminal good        good           bad/ok
> > mouse    good        good           bad/ok
...
> >RSDL makes most of the noyield hit back in normal make and then some
> >with ccache. Impressive. But ccache is still destroying interactivity
> >somehow. The ccache effect is fairly visible even with non-parallel
> >'make'.
> 
> Ok I don't think there's any actual accounting problem here per se
> (although I did just recently post a bugfix for rsdl however I think
> that's unrelated). What I think is going on in the ccache testcase is
> that all the work is being offloaded to kernel threads reading/writing
> to/from the filesystem and the make is not getting any actual cpu
> time.

I don't see significant system time while this is happening.

> This is "worked around" in mainline thanks to the testing for
> sleeping on uninterruptible sleep in the interactivity estimator. What
> I suspect is happening is kernel threads that are running nice -5 are
> doing all the work on make's behalf in the setting of ccache since it
> is mostly i/o bound. The reason for -nice values on kernel threads is
> questionable anyway. Can you try renicing your kernel threads all to
> nice 0 and see what effect that has? Obviously this doesn't need a
> recompile, but is simple enough to implement in kthread code as a new
> default.

Sorry, little to no benefit.

-- 
Mathematics is the supreme nostalgia of our time.

Re: RSDL-mm 0.28

[Matt Mackall]

On Sat, Mar 10, 2007 at 10:01:32PM -0600, Matt Mackall wrote:
> On Sun, Mar 11, 2007 at 01:28:22PM +1100, Con Kolivas wrote:
> > Ok I don't think there's any actual accounting problem here per se
> > (although I did just recently post a bugfix for rsdl however I think
> > that's unrelated). What I think is going on in the ccache testcase is
> > that all the work is being offloaded to kernel threads reading/writing
> > to/from the filesystem and the make is not getting any actual cpu
> > time.
> 
> I don't see significant system time while this is happening.

Also, it's running pretty much entirely out of page cache so there
wouldn't be a whole lot for kernel threads to do.

-- 
Mathematics is the supreme nostalgia of our time.

Re: RSDL-mm 0.28

[Con Kolivas]

On Sunday 11 March 2007 15:03, Matt Mackall wrote:
> On Sat, Mar 10, 2007 at 10:01:32PM -0600, Matt Mackall wrote:
> > On Sun, Mar 11, 2007 at 01:28:22PM +1100, Con Kolivas wrote:
> > > Ok I don't think there's any actual accounting problem here per se
> > > (although I did just recently post a bugfix for rsdl however I think
> > > that's unrelated). What I think is going on in the ccache testcase is
> > > that all the work is being offloaded to kernel threads reading/writing
> > > to/from the filesystem and the make is not getting any actual cpu
> > > time.
> >
> > I don't see significant system time while this is happening.
>
> Also, it's running pretty much entirely out of page cache so there
> wouldn't be a whole lot for kernel threads to do.

Well I can't reproduce that behaviour here at all whether from disk or the 
pagecache with ccache, so I'm not entirely sure what's different at your end. 
However both you and the other person reporting bad behaviour were using ATI 
drivers. That's about the only commonality? I wonder if they do need to 
yield... somewhat instead of not at all.

-- 
-ck

Re: RSDL-mm 0.28

[Willy Tarreau]

On Sat, Mar 10, 2007 at 07:35:06PM -0600, Matt Mackall wrote:
> I've tested -mm2 against -mm2+noyield and -mm2+rsdl+noyield. The
> noyield patch simply makes the sched_yield syscall return immediately.
> Xorg and all tests are run at nice 0.

[skipped long and precise test report]

> Also note I could occassionally trigger nasty multi-second pauses with
> -mm2+noyield under exectest that didn't show up elsewhere. That's
> probably a bug in the mainline scheduler.

This is not a bug per se, but more a design problem. This is caused by
the interactivity booster which is unfair. Mike Galbraith and others
spent a lot of time trying to get rid of those problems a few versions
ago. In early kernels (around 2.6.11), I could trivially cause pauses
more than 30 seconds long by running a few tasks simulating an interactive
workload. It is much more difficult to achieve this with recent kernels,
and it has absolutely no effect on RSDL, which is one of the reasons
I have to find it great !

Regards,
Willy

Re: RSDL for 2.6.21-rc3- 0.29

[Gene Heskett]

On Sunday 11 March 2007, Con Kolivas wrote:
>On Sunday 11 March 2007 15:03, Matt Mackall wrote:
>> On Sat, Mar 10, 2007 at 10:01:32PM -0600, Matt Mackall wrote:
>> > On Sun, Mar 11, 2007 at 01:28:22PM +1100, Con Kolivas wrote:
>> > > Ok I don't think there's any actual accounting problem here per se
>> > > (although I did just recently post a bugfix for rsdl however I
>> > > think that's unrelated). What I think is going on in the ccache
>> > > testcase is that all the work is being offloaded to kernel threads
>> > > reading/writing to/from the filesystem and the make is not getting
>> > > any actual cpu time.
>> >
>> > I don't see significant system time while this is happening.
>>
>> Also, it's running pretty much entirely out of page cache so there
>> wouldn't be a whole lot for kernel threads to do.
>
>Well I can't reproduce that behaviour here at all whether from disk or
> the pagecache with ccache, so I'm not entirely sure what's different at
> your end. However both you and the other person reporting bad behaviour
> were using ATI drivers. That's about the only commonality? I wonder if
> they do need to yield... somewhat instead of not at all.

I hate to say it Con, but this one seems to have broken the amanda-tar 
symbiosis.

I haven't tried a plain 21-rc3, so the problem may exist there, and in 
fact it did for 21-rc1, but I don't recall if it was true for -rc2.  But 
I will have a plain 21-rc3 running by tomorrow nights amanda run to test.

What happens is that when amanda tells tar to do a level 1 or 2, tar still 
thinks its doing a level 0.  The net result is that the tape is filled 
completely and amanda does an EOT exit in about 10 of my 42 dle's.  This 
is tar-1.15-1 for fedora core 6.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
While it may be true that a watched pot never boils, the one you don't
keep an eye on can make an awful mess of your stove.
		-- Edward Stevenson

Re: RSDL for 2.6.21-rc3- 0.29

[Con Kolivas]

Hi Gene.

On Monday 12 March 2007 16:38, Gene Heskett wrote:
> I hate to say it Con, but this one seems to have broken the amanda-tar
> symbiosis.
>
> I haven't tried a plain 21-rc3, so the problem may exist there, and in
> fact it did for 21-rc1, but I don't recall if it was true for -rc2.  But
> I will have a plain 21-rc3 running by tomorrow nights amanda run to test.
>
> What happens is that when amanda tells tar to do a level 1 or 2, tar still
> thinks its doing a level 0.  The net result is that the tape is filled
> completely and amanda does an EOT exit in about 10 of my 42 dle's.  This
> is tar-1.15-1 for fedora core 6.

I'm sorry but I have to say I have no idea what any of this means. I gather 
you're making an association between some application combination failing and 
RSDL cpu scheduler. Unfortunately the details of what the problem is, or how 
the cpu scheduler is responsible, escape me :(

-- 
-ck

Re: RSDL for 2.6.21-rc3- 0.29

[Gene Heskett]

On Monday 12 March 2007, Con Kolivas wrote:
>Hi Gene.
>
>On Monday 12 March 2007 16:38, Gene Heskett wrote:
>> I hate to say it Con, but this one seems to have broken the amanda-tar
>> symbiosis.
>>
>> I haven't tried a plain 21-rc3, so the problem may exist there, and in
>> fact it did for 21-rc1, but I don't recall if it was true for -rc2. 
>> But I will have a plain 21-rc3 running by tomorrow nights amanda run
>> to test.
>>
>> What happens is that when amanda tells tar to do a level 1 or 2, tar
>> still thinks its doing a level 0.  The net result is that the tape is
>> filled completely and amanda does an EOT exit in about 10 of my 42
>> dle's.  This is tar-1.15-1 for fedora core 6.
>
>I'm sorry but I have to say I have no idea what any of this means. I
> gather you're making an association between some application
> combination failing and RSDL cpu scheduler. Unfortunately the details
> of what the problem is, or how the cpu scheduler is responsible, escape
> me :(

I have another backup running right now, after building a plain 
2.6.21-rc3, and rebooting just now for the test.  I don't think its the 
scheduler itself, but is something post 2.6.20 that is messing with tars 
mind and making it think the files it just read to do the estimate phase, 
are all new, so even a level 2 is in effect a level 0.  I'll have an 
answer in about an hour, but its also 2:36am here and I'm headed for the 
rack to get some zzz's.  So I'll report in the morning as to whether or 
not this backup ran as it was supposed to.  I have a feeling its not 
going to though.


-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
"When it comes to humility, I'm the greatest."
-- Bullwinkle Moose

Re: RSDL for 2.6.21-rc3- 0.29

[Gene Heskett]

On Monday 12 March 2007, Gene Heskett wrote:
>On Monday 12 March 2007, Con Kolivas wrote:
>>Hi Gene.
>>
>>On Monday 12 March 2007 16:38, Gene Heskett wrote:
>>> I hate to say it Con, but this one seems to have broken the
>>> amanda-tar symbiosis.
>>>
>>> I haven't tried a plain 21-rc3, so the problem may exist there, and
>>> in fact it did for 21-rc1, but I don't recall if it was true for
>>> -rc2. But I will have a plain 21-rc3 running by tomorrow nights
>>> amanda run to test.
>>>
>>> What happens is that when amanda tells tar to do a level 1 or 2, tar
>>> still thinks its doing a level 0.  The net result is that the tape is
>>> filled completely and amanda does an EOT exit in about 10 of my 42
>>> dle's.  This is tar-1.15-1 for fedora core 6.
>>
>>I'm sorry but I have to say I have no idea what any of this means. I
>> gather you're making an association between some application
>> combination failing and RSDL cpu scheduler. Unfortunately the details
>> of what the problem is, or how the cpu scheduler is responsible,
>> escape me :(
>
>I have another backup running right now, after building a plain
>2.6.21-rc3, and rebooting just now for the test.  I don't think its the
>scheduler itself, but is something post 2.6.20 that is messing with tars
>mind and making it think the files it just read to do the estimate
> phase, are all new, so even a level 2 is in effect a level 0.  I'll
> have an answer in about an hour, but its also 2:36am here and I'm
> headed for the rack to get some zzz's.  So I'll report in the morning
> as to whether or not this backup ran as it was supposed to.  I have a
> feeling its not going to though.

I can confirm that a plain 21-rc3 still suffers from this problem.  This 
run of amanda terminated after 13 of the 32 dle's after writing just 
short of 12GB to the vtape. 8 were level 0's, 5 were level 1's, all were 
gzipped, achieving a compression ratio of 40% of original size.  There is 
about 45GB of data here to backup, on a 5 day dumpcycle.

If, and I have previously, I revert to a 2.6.20-ck1 patching, this does 
not occur.  So my contention is that someplace in this recent progression 
from 2.6.20 to 2.6.21-rc3, there is a patch which acts to change how 
c-time is being reported to tar.  Or there is a spillage into c-times 
when tar does its estimate scans where the output goes to /dev/null.
Or possibly even this version of tar is doing it differently.  I just 
looked up how to get the c-times out of ls, and they, as far as ls is 
concerned, look sane.  But tars actions while running a 2.6.21-rcX kernel 
certainly are not.  I do have a plain -rc2 I can try, so that will be the 
next test.  If that also fails in this manner, I'll build a later 
2.6.20-2 or whatever to verify that it doesn't so suffer.

I love your patches Con, but I'll leave them out of this next testing.  No 
use pointing fingers at good code.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Mal: "Well, you were right about this being a bad idea."

Zoe: "Thanks for sayin', sir."
				--Episode #1, "Serenity"

Re: RSDL for 2.6.21-rc3- 0.29

[Douglas McNaught]

Gene Heskett <gene.heskett@gmail.com> writes:

> If, and I have previously, I revert to a 2.6.20-ck1 patching, this does 
> not occur.  So my contention is that someplace in this recent progression 
> from 2.6.20 to 2.6.21-rc3, there is a patch which acts to change how 
> c-time is being reported to tar.  Or there is a spillage into c-times 
> when tar does its estimate scans where the output goes to /dev/null.
> Or possibly even this version of tar is doing it differently.  I just 
> looked up how to get the c-times out of ls, and they, as far as ls is 
> concerned, look sane.  But tars actions while running a 2.6.21-rcX kernel 
> certainly are not.  I do have a plain -rc2 I can try, so that will be the 
> next test.  If that also fails in this manner, I'll build a later 
> 2.6.20-2 or whatever to verify that it doesn't so suffer.

You may find 'strace' useful to track down this sort of thing (though
the output can be voluminous).

-Doug

RE: RSDL-mm 0.28

[David Schwartz]

> > There's a substantial performance hit for not yield, so we probably
> > want to investigate alternate semantics for it. It seems reasonable
> > for apps to say "let me not hog the CPU" without completely expiring
> > them. Imagine you're in the front of the line (aka queue) and you
> > spend a moment fumbling for your wallet. The polite thing to do is to
> > let the next guy in front. But with the current sched_yield, you go
> > all the way to the back of the line.

> Well... are you advocating we change sched_yield semantics to a
> gentler form? This is a cinch to implement but I know how Ingo feels
> about this. It will only encourage more lax coding using sched_yield
> instead of proper blocking (see huge arguments with the ldap people on
> this one who insist it's impossible not to use yield).

The basic point of sched_yield is to allow every other process at the same
static priority level a chance to use the CPU before you get it back. It is
generally an error to use sched_yield to be nice. It's nice to get your work
done when the scheduler gives you the CPU, that's why it gave it to you.

It is proper to use sched_yield as an optimization when it more efficient to
allow another process/thread to run than you, for example, when you
encounter a task you cannot do efficiently at that time because another
thread holds a lock.

It's also useful prior to doing something that can most efficiently be done
without interruption. So a thread that returns from 'sched_yield' should
ideally be given a full timeslice if possible. This may not be sensible if
the 'sched_yield' didn't actuall yield, but then again, if nothing else
wants to run, why not give the only task that does a full slice?

In no case is much of anything guaranteed, of course. (What can you do if
there's no other process to yield to?)

Note that processes that call sched_yield should be rewarded for doing so
just as process that block on I/O are, assuming they do in fact wind up
giving up the CPU when they would otherwise have had it.

DS

Re: RSDL-mm 0.28

[Ray Lee]

On 3/12/07, David Schwartz <davids@webmaster.com> wrote:
> In no case is much of anything guaranteed, of course. (What can you do if
> there's no other process to yield to?)

Perhaps if sched_yield()'s effects were cumulative inside a timeslice,
then eventually the calling task would get pushed far enough down that
others would run.

Ray

Re: RSDL for 2.6.21-rc3- 0.29

[Gene Heskett]

On Monday 12 March 2007, Douglas McNaught wrote:
>Gene Heskett <gene.heskett@gmail.com> writes:
>> If, and I have previously, I revert to a 2.6.20-ck1 patching, this
>> does not occur.  So my contention is that someplace in this recent
>> progression from 2.6.20 to 2.6.21-rc3, there is a patch which acts to
>> change how c-time is being reported to tar.  Or there is a spillage
>> into c-times when tar does its estimate scans where the output goes to
>> /dev/null. Or possibly even this version of tar is doing it
>> differently.  I just looked up how to get the c-times out of ls, and
>> they, as far as ls is concerned, look sane.  But tars actions while
>> running a 2.6.21-rcX kernel certainly are not.  I do have a plain -rc2
>> I can try, so that will be the next test.  If that also fails in this
>> manner, I'll build a later 2.6.20-2 or whatever to verify that it
>> doesn't so suffer.
>
>You may find 'strace' useful to track down this sort of thing (though
>the output can be voluminous).
>
>-Doug

I'd considered it, but with 32 dle entries, the whole strace output would 
be terrabytes & I don't have THAT much disk.  Not to mention it traces 
only the parent process, so tar would be merrily marching along to its 
own drummer and not traced I'm  afraid.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
I'm definitely not in Omaha!

Re: RSDL for 2.6.21-rc3- 0.29

[Douglas McNaught]

Gene Heskett <gene.heskett@gmail.com> writes:

> I'd considered it, but with 32 dle entries, the whole strace output would 
> be terrabytes & I don't have THAT much disk.  Not to mention it traces 
> only the parent process, so tar would be merrily marching along to its 
> own drummer and not traced I'm  afraid.

$ strace -ff 

-Doug

Re: RSDL for 2.6.21-rc3- 0.29

[Gene Heskett]

On Monday 12 March 2007, Douglas McNaught wrote:
>Gene Heskett <gene.heskett@gmail.com> writes:
>> I'd considered it, but with 32 dle entries, the whole strace output
>> would be terrabytes & I don't have THAT much disk.  Not to mention it
>> traces only the parent process, so tar would be merrily marching along
>> to its own drummer and not traced I'm  afraid.
>
>$ strace -ff
>
>-Doug

Someone else suggested the single -f, and I tried that, but even with the 
shell history set for 100,000 lines, i can't get back to the start, and I 
think its mucking with the shell arguments numbering as what I can see is 
about 5 reads through /etc/services accompanied by endless complaints 
of -EBADFD, the the logfile it generates says the port it was given was 
rejected when amcheck was run, here is that snip:

running /usr/local/sbin//amcheck Daily
Amanda Tape Server Host Check
-----------------------------
Holding disk /dumps: 72597 MB disk space available, using 72097 MB
slot 5: read label `Dailys-5', date `20070223002502'
NOTE: skipping tape-writable test
Tape Dailys-5 label ok
Server check took 4.556 seconds

Amanda Backup Client Hosts Check
--------------------------------
WARNING: coyote: selfcheck request failed: unable to bind to a reserved 
port (got port 32789)
Client check: 1 host checked in 34.733 seconds, 1 problem found

(brought to you by Amanda 2.5.1p3-20070222)
------------------eofsnip------------

The script quit by request shortly after as I didn't want it to further 
dirty the amanda database system.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Safety Third.

Re: RSDL for 2.6.21-rc3- 0.29

[Lee Revell]

On 3/12/07, Gene Heskett <gene.heskett@gmail.com> wrote:
> On Monday 12 March 2007, Douglas McNaught wrote:
> >Gene Heskett <gene.heskett@gmail.com> writes:
> >> I'd considered it, but with 32 dle entries, the whole strace output
> >> would be terrabytes & I don't have THAT much disk.  Not to mention it
> >> traces only the parent process, so tar would be merrily marching along
> >> to its own drummer and not traced I'm  afraid.
> >
> >$ strace -ff
> >
> >-Doug
>
> Someone else suggested the single -f, and I tried that, but even with the
> shell history set for 100,000 lines, i can't get back to the start, and I
> think its mucking with the shell arguments numbering as what I can see is
> about 5 reads through /etc/services accompanied by endless complaints
> of -EBADFD, the the logfile it generates says the port it was given was
> rejected when amcheck was run, here is that snip:

Can we start a new thread for this amanda issue?  It does not seem to
have anything to do with Con's scheduler...

Lee

Re: RSDL for 2.6.21-rc3- 0.29

[Douglas McNaught]

Gene Heskett <gene.heskett@gmail.com> writes:

> On Monday 12 March 2007, Douglas McNaught wrote:
>>Gene Heskett <gene.heskett@gmail.com> writes:
>>> I'd considered it, but with 32 dle entries, the whole strace output
>>> would be terrabytes & I don't have THAT much disk.  Not to mention it
>>> traces only the parent process, so tar would be merrily marching along
>>> to its own drummer and not traced I'm  afraid.
>>
>>$ strace -ff
>>
>>-Doug
>
> Someone else suggested the single -f, and I tried that, but even with the 
> shell history set for 100,000 lines, i can't get back to the start, and I 
> think its mucking with the shell arguments numbering as what I can see is 
> about 5 reads through /etc/services accompanied by endless complaints 
> of -EBADFD, the the logfile it generates says the port it was given was 
> rejected when amcheck was run, here is that snip:

I'd do 'strace -ff -o /tmp/amanda-strace <command>', which will give
you a set of files in /tmp, one for each PID created by fork().  Then
find the one that has the 'tar' invocation you're looking for.

-Doug

Re: RSDL for 2.6.21-rc3- 0.29

[Patrick Mau]

On Mon, Mar 12, 2007 at 03:43:09PM -0400, Douglas McNaught wrote:
> Gene Heskett <gene.heskett@gmail.com> writes:
> 
> > On Monday 12 March 2007, Douglas McNaught wrote:
> >>Gene Heskett <gene.heskett@gmail.com> writes:
> >>> I'd considered it, but with 32 dle entries, the whole strace output
> >>> would be terrabytes & I don't have THAT much disk.  Not to mention it
> >>> traces only the parent process, so tar would be merrily marching along
> >>> to its own drummer and not traced I'm  afraid.
> >>
> >>$ strace -ff
> >>
> >>-Doug
> >
> > Someone else suggested the single -f, and I tried that, but even with the 
> > shell history set for 100,000 lines, i can't get back to the start, and I 
> > think its mucking with the shell arguments numbering as what I can see is 
> > about 5 reads through /etc/services accompanied by endless complaints 
> > of -EBADFD, the the logfile it generates says the port it was given was 
> > rejected when amcheck was run, here is that snip:
> 
> I'd do 'strace -ff -o /tmp/amanda-strace <command>', which will give
> you a set of files in /tmp, one for each PID created by fork().  Then
> find the one that has the 'tar' invocation you're looking for.

Hi,

I hope you don't mind me jumping in ...

Why not temporarly replace "/bin/tar" with a shell script that does:

#!/bin/sh
exec strace -f -o output /bin/real.tar $@

That should be working, shouldn't it ?

Cheers,
Patrick

Re: RSDL for 2.6.21-rc3- 0.29

[Gene Heskett]

On Monday 12 March 2007, Patrick Mau wrote:
>On Mon, Mar 12, 2007 at 03:43:09PM -0400, Douglas McNaught wrote:
>> Gene Heskett <gene.heskett@gmail.com> writes:
>> > On Monday 12 March 2007, Douglas McNaught wrote:
>> >>Gene Heskett <gene.heskett@gmail.com> writes:
>> >>> I'd considered it, but with 32 dle entries, the whole strace
>> >>> output would be terrabytes & I don't have THAT much disk.  Not to
>> >>> mention it traces only the parent process, so tar would be merrily
>> >>> marching along to its own drummer and not traced I'm  afraid.
>> >>
>> >>$ strace -ff
>> >>
>> >>-Doug
>> >
>> > Someone else suggested the single -f, and I tried that, but even
>> > with the shell history set for 100,000 lines, i can't get back to
>> > the start, and I think its mucking with the shell arguments
>> > numbering as what I can see is about 5 reads through /etc/services
>> > accompanied by endless complaints of -EBADFD, the the logfile it
>> > generates says the port it was given was rejected when amcheck was
>> > run, here is that snip:
>>
>> I'd do 'strace -ff -o /tmp/amanda-strace <command>', which will give
>> you a set of files in /tmp, one for each PID created by fork().  Then
>> find the one that has the 'tar' invocation you're looking for.
>
>Hi,
>
>I hope you don't mind me jumping in ...
>
>Why not temporarly replace "/bin/tar" with a shell script that does:
>
>#!/bin/sh
>exec strace -f -o output /bin/real.tar $@
>
>That should be working, shouldn't it ?
>
>Cheers,
>Patrick

It should I'd think, for those instances where these scripts are run by 
hand and therefore have a shell to output to.  I'll give that a try when 
I get my script problem resolved, its a basic diff in the operation of 
real tapes and vtapes I need to get right for both scenarios.

Good Idea(TM), Thanks.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
"Ignorance is the soil in which belief in miracles grows."
-- Robert G. Ingersoll

Stracing Amanda (was: RSDL for 2.6.21-rc3- 0.29)

[Douglas McNaught]

Patrick Mau <mau@oscar.ping.de> writes:

> Why not temporarly replace "/bin/tar" with a shell script that does:
>
> #!/bin/sh
> exec strace -f -o output /bin/real.tar $@

You beat me to it.  :) I've done that before; it's a great suggestion.

Except that if you expect 'tar' to be invoked multiple times in a run,
you should probably use 'output.$$' for the output filename so things
don't get clobbered.

-Doug

Re: Stracing Amanda (was: RSDL for 2.6.21-rc3- 0.29)

[Gene Heskett]

On Monday 12 March 2007, Douglas McNaught wrote:
>Patrick Mau <mau@oscar.ping.de> writes:
>> Why not temporarly replace "/bin/tar" with a shell script that does:
>>
>> #!/bin/sh
>> exec strace -f -o output /bin/real.tar $@
>
>You beat me to it.  :) I've done that before; it's a great suggestion.
>
>Except that if you expect 'tar' to be invoked multiple times in a run,
>you should probably use 'output.$$' for the output filename so things
>don't get clobbered.
>
>-Doug

In my case, Doug, it will get invoked 64 times, amanda does a dummy run to 
get an estimate, calculates what to do based on that output which is 32 
runs, 1 per disklist entry and I have 32, and then reruns tar with the 
appropriate level options against each individual disklist entry.

But I'm puzzled a bit, what does the double $$ do?, or it buried someplace 
in the bash manpage?  Its not something I've stumbled over yet.

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
rugged, adj.:
	Too heavy to lift.

Re: Stracing Amanda (was: RSDL for 2.6.21-rc3- 0.29)

[Nish Aravamudan]

On 3/12/07, Gene Heskett <gene.heskett@gmail.com> wrote:
> On Monday 12 March 2007, Douglas McNaught wrote:
> >Patrick Mau <mau@oscar.ping.de> writes:
> >> Why not temporarly replace "/bin/tar" with a shell script that does:
> >>
> >> #!/bin/sh
> >> exec strace -f -o output /bin/real.tar $@
> >
> >You beat me to it.  :) I've done that before; it's a great suggestion.
> >
> >Except that if you expect 'tar' to be invoked multiple times in a run,
> >you should probably use 'output.$$' for the output filename so things
> >don't get clobbered.
> >
> >-Doug
>
> In my case, Doug, it will get invoked 64 times, amanda does a dummy run to
> get an estimate, calculates what to do based on that output which is 32
> runs, 1 per disklist entry and I have 32, and then reruns tar with the
> appropriate level options against each individual disklist entry.
>
> But I'm puzzled a bit, what does the double $$ do?, or it buried someplace
> in the bash manpage?  Its not something I've stumbled over yet.

buried indeed:

"Special Parameters:
  ...
       $      Expands to the process ID of the shell.  In a  ()  subshell,  it
              expands  to  the  process  ID of the current shell, not the sub‐
              shell.
"

Thanks,
Nish

Re: Stracing Amanda (was: RSDL for 2.6.21-rc3- 0.29)

[Gene Heskett]

On Monday 12 March 2007, Nish Aravamudan wrote:
>On 3/12/07, Gene Heskett <gene.heskett@gmail.com> wrote:
>> On Monday 12 March 2007, Douglas McNaught wrote:
>> >Patrick Mau <mau@oscar.ping.de> writes:
>> >> Why not temporarly replace "/bin/tar" with a shell script that
>> >> does:
>> >>
>> >> #!/bin/sh
>> >> exec strace -f -o output /bin/real.tar $@
>> >
>> >You beat me to it.  :) I've done that before; it's a great
>> > suggestion.
>> >
>> >Except that if you expect 'tar' to be invoked multiple times in a
>> > run, you should probably use 'output.$$' for the output filename so
>> > things don't get clobbered.
>> >
>> >-Doug
>>
>> In my case, Doug, it will get invoked 64 times, amanda does a dummy
>> run to get an estimate, calculates what to do based on that output
>> which is 32 runs, 1 per disklist entry and I have 32, and then reruns
>> tar with the appropriate level options against each individual
>> disklist entry.
>>
>> But I'm puzzled a bit, what does the double $$ do?, or it buried
>> someplace in the bash manpage?  Its not something I've stumbled over
>> yet.
>
>buried indeed:
>
>"Special Parameters:
>  ...
>       $      Expands to the process ID of the shell.  In a  () 
> subshell,  it expands  to  the  process  ID of the current shell, not
> the sub‐ shell.
>"

Well, that's clear enough, but what of the double $$ case?  Would this 
them make a PID unique to each invocation untill it finally wraps a 16 
bit value, or will the kernel re-use them because they won't all be 
running simultainiously, but limited by the number of unique 'spindle' 
numbers on the system, this to prevent as best as it can, the thrashing 
of a drive by having tar working on 2 separate (or more) partitions at 
the same time.  In my case 2 are possible, as /var is on a separate 
drive.

>Thanks,
>Nish



-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
"Say yur prayers, yuh flea-pickin' varmint!"
-- Yosemite Sam

Re: Stracing Amanda (was: RSDL for 2.6.21-rc3- 0.29)

[Willy Tarreau]

On Tue, Mar 13, 2007 at 12:04:42AM -0400, Gene Heskett wrote:
> On Monday 12 March 2007, Nish Aravamudan wrote:
> >On 3/12/07, Gene Heskett <gene.heskett@gmail.com> wrote:
> >> On Monday 12 March 2007, Douglas McNaught wrote:
> >> >Patrick Mau <mau@oscar.ping.de> writes:
> >> >> Why not temporarly replace "/bin/tar" with a shell script that
> >> >> does:
> >> >>
> >> >> #!/bin/sh
> >> >> exec strace -f -o output /bin/real.tar $@
> >> >
> >> >You beat me to it.  :) I've done that before; it's a great
> >> > suggestion.
> >> >
> >> >Except that if you expect 'tar' to be invoked multiple times in a
> >> > run, you should probably use 'output.$$' for the output filename so
> >> > things don't get clobbered.
> >> >
> >> >-Doug
> >>
> >> In my case, Doug, it will get invoked 64 times, amanda does a dummy
> >> run to get an estimate, calculates what to do based on that output
> >> which is 32 runs, 1 per disklist entry and I have 32, and then reruns
> >> tar with the appropriate level options against each individual
> >> disklist entry.
> >>
> >> But I'm puzzled a bit, what does the double $$ do?, or it buried
> >> someplace in the bash manpage?  Its not something I've stumbled over
> >> yet.
> >
> >buried indeed:
> >
> >"Special Parameters:
> >  ...
> >       $      Expands to the process ID of the shell.  In a  () 
> > subshell,  it expands  to  the  process  ID of the current shell, not
> > the sub?$B!> shell.
> >"
> 
> Well, that's clear enough, but what of the double $$ case?  Would this 
> them make a PID unique to each invocation untill it finally wraps a 16 
> bit value, or will the kernel re-use them because they won't all be 
> running simultainiously, but limited by the number of unique 'spindle' 
> numbers on the system, this to prevent as best as it can, the thrashing 
> of a drive by having tar working on 2 separate (or more) partitions at 
> the same time.  In my case 2 are possible, as /var is on a separate 
> drive.

Yes there a risk of wrapping, but it is very small. You can add the command
line arguments to the file name if you want, like this :

#!/bin/sh
exec strace -f -o "output.$$.${*//\//_}" /bin/real.tar $@

It will name the output file "output.<pid>.<args>", replacing slashes with
underscores. This is very dirty but can help.

Cheers,
Willy

Re: Stracing Amanda (was: RSDL for 2.6.21-rc3- 0.29)

[Gene Heskett]

On Tuesday 13 March 2007, Willy Tarreau wrote:
>On Tue, Mar 13, 2007 at 12:04:42AM -0400, Gene Heskett wrote:
>> On Monday 12 March 2007, Nish Aravamudan wrote:
>> >On 3/12/07, Gene Heskett <gene.heskett@gmail.com> wrote:
>> >> On Monday 12 March 2007, Douglas McNaught wrote:
>> >> >Patrick Mau <mau@oscar.ping.de> writes:
>> >> >> Why not temporarly replace "/bin/tar" with a shell script that
>> >> >> does:
>> >> >>
>> >> >> #!/bin/sh
>> >> >> exec strace -f -o output /bin/real.tar $@
>> >> >
>> >> >You beat me to it.  :) I've done that before; it's a great
>> >> > suggestion.
>> >> >
>> >> >Except that if you expect 'tar' to be invoked multiple times in a
>> >> > run, you should probably use 'output.$$' for the output filename
>> >> > so things don't get clobbered.
>> >> >
>> >> >-Doug
>> >>
>> >> In my case, Doug, it will get invoked 64 times, amanda does a dummy
>> >> run to get an estimate, calculates what to do based on that output
>> >> which is 32 runs, 1 per disklist entry and I have 32, and then
>> >> reruns tar with the appropriate level options against each
>> >> individual disklist entry.
>> >>
>> >> But I'm puzzled a bit, what does the double $$ do?, or it buried
>> >> someplace in the bash manpage?  Its not something I've stumbled
>> >> over yet.
>> >
>> >buried indeed:
>> >
>> >"Special Parameters:
>> >  ...
>> >       $      Expands to the process ID of the shell.  In a  ()
>> > subshell,  it expands  to  the  process  ID of the current shell,
>> > not the sub?$B!> shell.
>> >"
>>
>> Well, that's clear enough, but what of the double $$ case?  Would this
>> them make a PID unique to each invocation untill it finally wraps a 16
>> bit value, or will the kernel re-use them because they won't all be
>> running simultainiously, but limited by the number of unique 'spindle'
>> numbers on the system, this to prevent as best as it can, the
>> thrashing of a drive by having tar working on 2 separate (or more)
>> partitions at the same time.  In my case 2 are possible, as /var is on
>> a separate drive.
>
>Yes there a risk of wrapping, but it is very small. You can add the
> command line arguments to the file name if you want, like this :
>
>#!/bin/sh
>exec strace -f -o "output.$$.${*//\//_}" /bin/real.tar $@
>
>It will name the output file "output.<pid>.<args>", replacing slashes
> with underscores. This is very dirty but can help.
>
Excellent Willy, thanks.

>Cheers,
>Willy



-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Whatever doesn't succeed in two months and a half in California will
never succeed.
		-- Rev. Henry Durant, founder of the University of California

Re: RSDL-mm 0.28

[Nick Piggin]

David Schwartz wrote:
>>>There's a substantial performance hit for not yield, so we probably
>>>want to investigate alternate semantics for it. It seems reasonable
>>>for apps to say "let me not hog the CPU" without completely expiring
>>>them. Imagine you're in the front of the line (aka queue) and you
>>>spend a moment fumbling for your wallet. The polite thing to do is to
>>>let the next guy in front. But with the current sched_yield, you go
>>>all the way to the back of the line.
> 
> 
>>Well... are you advocating we change sched_yield semantics to a
>>gentler form? This is a cinch to implement but I know how Ingo feels
>>about this. It will only encourage more lax coding using sched_yield
>>instead of proper blocking (see huge arguments with the ldap people on
>>this one who insist it's impossible not to use yield).
> 
> 
> The basic point of sched_yield is to allow every other process at the same
> static priority level a chance to use the CPU before you get it back. It is
> generally an error to use sched_yield to be nice. It's nice to get your work
> done when the scheduler gives you the CPU, that's why it gave it to you.
> 
> It is proper to use sched_yield as an optimization when it more efficient to
> allow another process/thread to run than you, for example, when you
> encounter a task you cannot do efficiently at that time because another
> thread holds a lock.
> 
> It's also useful prior to doing something that can most efficiently be done
> without interruption. So a thread that returns from 'sched_yield' should
> ideally be given a full timeslice if possible. This may not be sensible if
> the 'sched_yield' didn't actuall yield, but then again, if nothing else
> wants to run, why not give the only task that does a full slice?
> 
> In no case is much of anything guaranteed, of course. (What can you do if
> there's no other process to yield to?)
> 
> Note that processes that call sched_yield should be rewarded for doing so
> just as process that block on I/O are, assuming they do in fact wind up
> giving up the CPU when they would otherwise have had it.
> 
> DS
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 


-- 
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com