[PATCH] (Updated) Preemptible Kernel

[PATCH] (Updated) Preemptible Kernel

[Robert Love]

This patch enables a preemptible kernel - now userspace programs can be
preempted, even if in kernel land.  This should result in greater system
response.

Updated patches are available at:
http://tech9.net/rml/linux/patch-rml-2.4.9-ac12-preempt-kernel-1 and
http://tech9.net/rml/linux/patch-rml-2.4.10-pre11-preempt-kernel-1
for 2.4.9-ac12 and 2.4.10-pre11, respectively.

ChangeLog is at http://tech9.net/rml/linux/changelog-preempt

The Athlon-optimized problem is fixed and a final solution is merged.

We have had reported success with XFS, although a patch is required to
compile, available at
http://tech9.net/rml/linux/other/patch-rml-2.4.10-pre10-xfs-preempt-fix-1

Finally, I have gotten great feedback from SMP users -- both in that "it
works" and that benchmarks show an improvement.  Thus, SMP and
preemption together are no longer marked experimental.

The only outstanding issue is a possible issue with ReiserFS, although I
am beginning to think this is attributed to VM muck and not ReiserFS. 
If you use ReiserFS, I would appreciate some feedback. (Note that the
issue is _not_ fs corruption or anything of the ilk, just odd syslog
messages and the such).

Please continue to supply feedback and relevant benchmarks.  I really
encourage "regular" users to try this out -- patch your kernel and
enable CONFIG_PREEMPT.  I am not an audio guy, trust me -- this is worth
it for any desktop.

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] (Updated) Preemptible Kernel

[Robert Love]

On Tue, 2001-09-18 at 20:10, Robert Love wrote:
> This patch enables a preemptible kernel - now userspace programs can be
> preempted, even if in kernel land.  This should result in greater system
> response.

2.4.10-pre12-preempt patch is available at
http://tech9.net/rml/linux/patch-rml-2.4.10-pre12-preempt-kernel-1

Also, the XFS fix has been committed to CVS. So subsequent CVS dumps or
releases will not need the extra patch.

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] (Updated) Preemptible Kernel

[Robert Love]

On Mon, 2001-10-01 at 01:34, Steve Kieu wrote:
> Sorry it seems that your patches has not uploaded yet
> :-)

Hm, check again -- I see them.  I've already gotten emails about them,
so I guess others do, too :)

>From http://tech9.net/rml/linux, the patches are at:

http://tech9.net/rml/linux/patch-rml-2.4.10-ac1-preempt-kernel-1
http://tech9.net/rml/linux/patch-rml-2.4.11-pre1-preempt-kernel-1
http://tech9.net/rml/linux/patch-rml-2.4.10-preempt-kernel-5

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] (Updated) Preemptible Kernel

[Steve Kieu]

 --- Robert Love <rml@tech9.net> wrote: > Updated
patches for 2.4.10, 2.4.11-pre1, and
> 2.4.10-ac1 are at
> http://tech9.net/rml/linux

Sorry it seems that your patches has not uploaded yet
:-)

Cheers,



=====
S.KIEU

http://travel.yahoo.com.au - Yahoo! Travel
- Got Itchy feet? Get inspired!

[PATCH] (Updated) Preemptible Kernel

[Robert Love]

Updated patches for 2.4.10, 2.4.11-pre1, and 2.4.10-ac1 are at
http://tech9.net/rml/linux

Note that the 2.4.10 patch is updated over previous patches.

It's been awhile, because a lot has changed.  Please give this a good
once (and twice) -over.  For those of you using preempt-stats, I suggest
not keeping that patch compiled in during regular use and certainly not
during any benchmarking.

What's New:

Most notably, George Anzinger of MontaVista rewrote how we keep track of
whether a task is preempted or not.  The "preempt bit" has been moved
from a task's status flag to the preempt_count (per-task_struct)
variable.  The resulting code is cleaner and simpler, and as a bonus the
race with ptrace and signals (and any undetected similar problems) has
been fixed, so we don't need explicit fixes.

I also renamed our namespace, cleaning things up, and trying to stick to
a prefix of "preempt."  Most important, ctx_sw_on and ctx_sw_off are now
preempt_enable and preempt_disable.

The ChangeLog since the last official release:

20010928
- have spin_lock_prefetch prefetch the preempt_count, too,
  since its incremented on each spin_lock
- some more ifdef and namespace cleanups

20010925
- modify our handling of the "preemption bit": move from the
  task's status to the preempt_count
- the above removes the risk of race in ptrace and SIGSTOP
  signal handling, so we can back out those fixes.
- rearrange our namespace.  most notably, ctx_sw_on/off
  is now preempt_enable/disable.
- cleanup some defines

20010923
- fix strace race: make ptrace preempt safe
- fix job control race: make SIGSTOP signal preempt
  safe
- fix adfs compile bug: add sched.h include to
  fs/adfs/map.c

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] (Updated) Preemptible Kernel

[Robert Love]

Quick update:

pre6-preempt patch is up at:
http://tech9.net/rml/linux/patch-rml-2.4.10-pre6-preempt-kernel-1

pre5 does not diff cleanly against pre6, so use this if you are using
Linus's tree.  this patch contains the ieee1394 fix, too. highmem
updates are still pending (see below).

2.4.9-ac10 is still at:
http://tech9.net/rml/linux/patch-rml-2.4.9-ac10-preempt-kernel-1

note this does not contain the ieee1394 fix or highmem update. as
always, you can find the newest patches there, fairly quickly.

if you are using 2.4.9-ac10 or earlier or 2.4.10-pre5 or earlier and
need the ieee1394 patch (fixes compile error) here it is (this is merged
in 2.4.10-pre6-preempt and will be in the next acXX-preempt:)

diff -urN linux-2.4.9-ac10/drivers/ieee1394/ linux/drivers/ieee1394/csr.c
--- linux-2.4.9-ac10/drivers/ieee1394/csr.c	Fri Sep  7 23:53:41 2001
+++ linux/drivers/ieee1394/csr.c	Sun Sep  9 00:07:21 2001
@@ -10,6 +10,7 @@
  */
 
 #include <linux/string.h>
+#include <linux/sched.h>
 
 #include "ieee1394_types.h"
 #include "hosts.h"

if you are using any patch, and are using highmem, here is an
experimental patch that seems to work.  a final version will be in
future preempt patches:


--- linux-corndog/include/linux/highmem.h Sun Sep  9 08:59:04 2001
+++ linux/include/linux/highmem.h Sun Sep  9 09:00:07 2001
@@ -88,6 +88,7 @@
        if (page < highmem_start_page)
                return page_address(page);

+       ctx_sw_off();
        idx = type + KM_TYPE_NR*smp_processor_id();
        vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
#if HIGHMEM_DEBUG
@@ -119,6 +120,7 @@
        pte_clear(kmap_pte-idx);
        __flush_tlb_one(vaddr);
#endif
+       ctx_sw_on();
}

#endif /* __KERNEL__ */

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] (Updated) Preemptible Kernel

[Robert Love]

with the preemption config set, there is a compile failure with the
ieee1394 drivers.  thank you to Patrick Chase and the others who
reported this.

note: drivers are going to need to explicity include linux/sched.h if
they use (certain) spinlock primitives and want to compile with
CONFIG_PREEMPT set.  its cleaner anyhow.

this patch is against 2.4.9-ac10 but should be fine for 2.4.10-pre5...




diff -urN linux-2.4.9-ac10/drivers/ieee1394/ linux/drivers/ieee1394/csr.c
--- linux-2.4.9-ac10/drivers/ieee1394/csr.c	Fri Sep  7 23:53:41 2001
+++ linux/drivers/ieee1394/csr.c	Sun Sep  9 00:07:21 2001
@@ -10,6 +10,7 @@
  */
 
 #include <linux/string.h>
+#include <linux/sched.h>
 
 #include "ieee1394_types.h"
 #include "hosts.h"




-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] (Updated) Preemptible Kernel

[Daniel Phillips]

On September 6, 2001 10:39 pm, Phillip Susi wrote:
> ... what happens if 
> say, the kernel called from user space is holding a lock, and gets preempted? 

The thread will eventually get rescheduled and release the lock.

>  Is there a mechanism to disable preemption while holding locks or at other 
> resources that need to be freed before another task is run?

IIRC spinlocks inhibit scheduling, the simplest way to handle that.

--
Daniel

Re: [PATCH] (Updated) Preemptible Kernel

[Robert Love]

On Thu, 2001-09-06 at 16:39, Phillip Susi wrote:
> This sounds interesting, but am I correct in assuming that this only allows 
> preemption during code that is called from user space?  For instance, it 
> would be bad to preempt an ISR or BH, right?  Actually... what happens if 
> say, the kernel called from user space is holding a lock, and gets preempted? 
>  Is there a mechanism to disable preemption while holding locks or at other 
> resources that need to be freed before another task is run?
> 
> By the way, are there any other 'modes of execution' for lack of a better 
> word, besides IRS, B, and.... 'called from user space', also for lack of a 
> better word.  Forgive me, I'm not that familiar with the Linux kernel yet.

The SMP locking mechanisms (spinlocks, readlocks, writelocks) are used
to prevent preemption for reentrancy or data validity reasons (this is
the same reason we do this for SMP).  So, there is no worry about
holding locks and whatnot.

We also don't allow preemption during interrupt handling, bottom halves,
or during execution of the scheduler itself.  So, you are correct again,
no preemption during ISRs or BHs.

At all other times, preemption is allowed.  It occurs when another task
comes due for execution -- now, even if in kernel space, we will yield
execution to the new process.  Please give the patch a try, enable
preemption, and see what you think.

Some other changes were made, too.  Certain conditions were prevented
(ie, dont return to user with a pending soft interrupt).  A preemption
counter is maintained, and the locks now modify it.  A new
TASK_PREEMPTING flag was created.

To answer your question, besides system calls (ie, `on behalf of a
process') you have ISRs, BHs, wait-queued routines (a growing number of
these), and other kernel threads (keventd, ksoftirqd, kswapd, etc) which
are typically timer/queued events anyhow.

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

[PATCH] (Updated) Preemptible Kernel

[Robert Love]

Available at (about 29K):

http://tech9.net/rml/linux/patch-rml-2.4.10-pre4-preempt-kernel-1
http://tech9.net/rml/linux/patch-rml-2.4.9-ac9-preempt-kernel-1

for kernel 2.4.10-pre4 and 2.4.9-ac9, respectively.

Changes since previous post:
* update for new kernels
* fix newline/space format buglet

Changes since original patch:
* fix compile bug -- CONFIG_HAVE_DEC_LOCK is set as needed, now.

This patch allows a new config setting, CONFIG_PREEMPT (set in
`Processor Type and Features') that enables a fully preemptible kernel. 
Preemption is controled via SMP lock points.  Control of execution is
yielded to higher processes even if the currently running process is in
kernel space.

This should increase response and decrease latency, and is a highly
recommended patch for real-time, audio, and embedded systems.  However,
it is recommended for anyone.  I use it on my everyday workstation.

An interesting new article on a preemptible kernel in linux is available
at:

http://www.linuxdevices.com/articles/AT5152980814.html

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] (Updated) Preemptible Kernel

[Phillip Susi]

This sounds interesting, but am I correct in assuming that this only allows 
preemption during code that is called from user space?  For instance, it 
would be bad to preempt an ISR or BH, right?  Actually... what happens if 
say, the kernel called from user space is holding a lock, and gets preempted? 
 Is there a mechanism to disable preemption while holding locks or at other 
resources that need to be freed before another task is run?

By the way, are there any other 'modes of execution' for lack of a better 
word, besides IRS, B, and.... 'called from user space', also for lack of a 
better word.  Forgive me, I'm not that familiar with the Linux kernel yet.  

On Thursday 06 September 2001 10:02 pm, Robert Love wrote:
> Available at (about 29K):
>
> http://tech9.net/rml/linux/patch-rml-2.4.10-pre4-preempt-kernel-1
> http://tech9.net/rml/linux/patch-rml-2.4.9-ac9-preempt-kernel-1
>
> for kernel 2.4.10-pre4 and 2.4.9-ac9, respectively.
>
> Changes since previous post:
> * update for new kernels
> * fix newline/space format buglet
>
> Changes since original patch:
> * fix compile bug -- CONFIG_HAVE_DEC_LOCK is set as needed, now.
>
> This patch allows a new config setting, CONFIG_PREEMPT (set in
> `Processor Type and Features') that enables a fully preemptible kernel.
> Preemption is controled via SMP lock points.  Control of execution is
> yielded to higher processes even if the currently running process is in
> kernel space.
>
> This should increase response and decrease latency, and is a highly
> recommended patch for real-time, audio, and embedded systems.  However,
> it is recommended for anyone.  I use it on my everyday workstation.
>
> An interesting new article on a preemptible kernel in linux is available
> at:
>
> http://www.linuxdevices.com/articles/AT5152980814.html

-- 
--> Phill Susi