All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ingo Molnar <mingo@elte.hu>
To: linux-kernel@vger.kernel.org
Cc: Lee Revell <rlrevell@joe-job.com>, "K.R. Foley" <kr@cybsft.com>,
	Rui Nuno Capela <rncbc@rncbc.org>,
	Florian Schmidt <mista.tapas@gmx.net>,
	Mark_H_Johnson@raytheon.com,
	Fernando Pablo Lopez-Lezcano <nando@ccrma.Stanford.EDU>,
	Daniel Walker <dwalker@mvista.com>, Andrew Morton <akpm@osdl.org>
Subject: [patch] CONFIG_PREEMPT_REALTIME, 'Fully Preemptible Kernel', VP-2.6.9-rc4-mm1-T4
Date: Mon, 11 Oct 2004 16:29:53 +0200	[thread overview]
Message-ID: <20041011142953.GA32607@elte.hu> (raw)
In-Reply-To: <20041007105230.GA17411@elte.hu>


i've released the -T4 VP patch:

  http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T4

the big change in this release is the addition of PREEMPT_REALTIME,
which is a new implementation of a fully preemptible kernel model:

 - spinlocks and rwlocks use semaphores and are preemptible by default

 - the _irq variants of these locks do not disable interrupts but rely
   on IRQ threading to exclude against interrupt contexts.

note that this implementation is different from other kernel-preemption
patches, in a number of key areas. Initially i looked at merging the
MontaVista patchset from two days ago but decided to implement a new one
from scratch to cure a number of conceptual problems:

 - this patch auto-detects the 'type' of the lock at compilation time. 

   All fully-preemptible kernel patches i've seen so far suffer from one
   nasty problem: they are very large because they redefine _all_ the
   spinlock APIs to provide separation between 'mutex based' and
   'original' spinlocks. E.g. check out the sheer size of the MontaVista
   patchset: Linux-2.6.9-rc3-RT_spinlock1.patch and
   Linux-2.6.9-rc3-RT_spinlock2.patch are 84K and 92K and they convert
   ~30 core spinlocks to new APIs.

   OTOH this patch converts _90_ spinlocks in roughly half the
   patchsize, which makes a large difference in maintainability.

   How it works: this implementation uses a gcc feature to detect the
   type of the spinlock compile-time and to switch to the mutex or
   raw_spinlock API accordingly. Only one, very isolated change has to
   be done to switch a generic spinlock to a spin-only lock: spinlock_t
   is changed to raw_spinlock_t and the initializer is fixed up. All the
   other code remains untouched - and this even if a single C module
   contains both mutex-based and spinlock-based API calls. This approach
   is quite close to a simple object-oriented lock type - but written in
   C and compatible with the existing spinlock APIs.

 - i used the native Linux semaphores/rwsems to implement
   spinlock/rwlock preemption. E.g. the MontaVista patches use separate
   synchronization objects (kmutex/pmutex) to implement this.

   I believe using native semaphores is the better approach
   architecturally because this means that we have to add priority
   inheritance handling only once and to the native Linux semaphores. 
   This has the additional benefit of fixing all mutex-using
   kernel code's priority inheritance problems. (which kmutex/pmutex
   does not solve.)

   OTOH the MontaVista patches naturally have the advantage of having a
   working priority-inheritance mechanism in the pmutex code, right now. 
   (I did a brief attempt to plug the pmutex code into this patch but it
   didnt look good of a match - but others might want to try to 
   integrate it nevertheless.)

   also, another bad property of the kmutex/pmutex code is that it uses
   assembly which makes it quite hard to port to non-x86 architectures. 
   OTOH, the native Linux semaphores and rwsems work on every
   architecture.

 - the patch converts rwlocks too, while e.g. the MontaVista patchset
   still keeps rwlocks as spinlocks. It is important to convert rwlocks
   to rw-semaphores, most notably this allow the conversion of the
   tasklist and signal spinlocks.

 - finally, i went for correctness primarily, not latencies. I checked
   out the MontaVista patches and they categorize roughly 30 spinlocks
   as the ones that are necessary to be 'raw'. Unfortunately this is
   inadequate, my patch excludes 90 such locks and it's still probably
   not a 100% correct conversion. The core kernel needs changes in the
   locking infrastructure to get rid of most of the these 90 non-mutex
   locks.

it is highly recommended to enable DEBUG_PREEMPT when enabling
PREEMPT_REALTIME. It will warn about all the places that are unsafe. The
patch is x86-only for the time being, but the changes necessary for
other architectures should be relatively low.

NOTE: CONFIG_PREEMPT_REALTIME is default-off and i'd only suggest to
enable it on non-critical systems. It is the first iteration of this
feature and it will sure have rough edges. Not for the faint hearted!

NOTE2: some of the lock-break functionality offered by the -VP patchset
is disabled if PREEMPT_REALTIME is enabled - this is temporary. This
will likely result in an increase of the maximum measured latencies.

NOTE3: since so many spinlocks are still non-mutex, even average
latencies will be well above what we could achieve - but i wanted to
reach a known-correct codebase first. For example, most of the
networking spinlocks had to be made non-mutex because of networking's
use of RCU locking primitives and per-CPU data structures. The same is
true for the VFS - many of its locks are non-mutex still due to RCU. 
Once this infrastructure work is done the size of the patch will
decrease significantly.

to build a -T4 tree from scratch the patching order is:

   http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.8.tar.bz2
 + http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.9-rc4.bz2
 + http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc4/2.6.9-rc4-mm1/2.6.9-rc4-mm1.bz2
 + http://redhat.com/~mingo/voluntary-preempt/voluntary-preempt-2.6.9-rc4-mm1-T4

	Ingo

  parent reply	other threads:[~2004-10-11 14:39 UTC|newest]

Thread overview: 224+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20040903120957.00665413@mango.fruits.de>
     [not found] ` <20040903100946.GA22819@elte.hu>
     [not found]   ` <20040903123139.565c806b@mango.fruits.de>
2004-09-03 10:32     ` lockup with voluntary preempt R0 and VP, KP, etc, disabled Ingo Molnar
2004-09-03 11:59       ` Florian Schmidt
2004-09-03 11:55         ` Ingo Molnar
2004-09-03 13:01           ` Florian Schmidt
2004-09-03 12:04         ` Florian Schmidt
2004-09-03 12:08           ` Florian Schmidt
2004-09-03 18:28             ` Lee Revell
2004-09-03 18:54               ` Florian Schmidt
2004-09-03 18:52                 ` Lee Revell
2004-09-03 19:06                 ` K.R. Foley
2004-09-04 19:51 ` [patch] voluntary-preempt-2.6.9-rc1-bk4-R4 Ingo Molnar
2004-09-05 14:02   ` [patch] voluntary-preempt-2.6.9-rc1-bk12-R5 Ingo Molnar
2004-09-05 16:07     ` Matt Heler
2004-09-05 18:16     ` Lee Revell
2004-09-05 19:12       ` Ingo Molnar
2004-09-05 21:03         ` Lee Revell
2004-09-06  6:30           ` Ingo Molnar
2004-09-06  7:44             ` Lee Revell
2004-09-07  3:17               ` K.R. Foley
2004-09-06 11:06     ` [patch] voluntary-preempt-2.6.9-rc1-bk12-R6 Ingo Molnar
2004-09-06 11:48       ` Rafael J. Wysocki
2004-09-06 12:25         ` Alexander Nyberg
2004-09-06 12:29           ` Ingo Molnar
2004-09-07  9:26             ` [patch] voluntary-preempt-2.6.9-rc1-bk12-R7 Ingo Molnar
2004-09-07 11:57               ` [patch] voluntary-preempt-2.6.9-rc1-bk12-R8 Ingo Molnar
2004-09-07 14:50                 ` Alexander Nyberg
2004-09-07 15:04                   ` Ingo Molnar
2004-09-07 15:29                     ` Alexander Nyberg
2004-09-07 19:56                 ` Rafael J. Wysocki
2004-09-07 22:59                 ` Lee Revell
2004-09-08  8:20                   ` [patch] voluntary-preempt-2.6.9-rc1-bk12-R9 Ingo Molnar
2004-09-08  9:46                     ` Rafael J. Wysocki
2004-09-08  9:56                       ` Ingo Molnar
2004-09-08 22:37                     ` Lee Revell
2004-09-09  6:17                       ` [patch] voluntary-preempt-2.6.9-rc1-bk12-S0 Ingo Molnar
2004-09-09 19:30                         ` Lee Revell
2004-09-09 20:23                           ` Lee Revell
2004-09-19 12:26                         ` [patch] voluntary-preempt-2.6.9-rc2-mm1-S1 Ingo Molnar
2004-09-19 17:05                           ` Felipe Alfaro Solana
2004-09-20 17:14                           ` K.R. Foley
2004-09-20 19:48                             ` Ingo Molnar
2004-09-21  2:25                               ` K.R. Foley
2004-09-20 19:47                           ` Magnus Määttä
2004-09-21  2:07                           ` BKL backtraces - was: " K.R. Foley
2004-09-21  7:18                             ` Ingo Molnar
2004-09-21  7:44                               ` [patch] voluntary-preempt-2.6.9-rc2-mm1-S2 Ingo Molnar
2004-09-21 18:51                                 ` K.R. Foley
2004-09-22 10:33                                 ` [patch] voluntary-preempt-2.6.9-rc2-mm1-S3 Ingo Molnar
2004-09-22 15:07                                   ` K.R. Foley
2004-09-22 17:16                                     ` Lee Revell
2004-09-22 17:40                                       ` K.R. Foley
2004-09-22 19:45                                       ` Ingo Molnar
2004-09-22 17:09                                   ` K.R. Foley
2004-09-22 19:08                                   ` Lee Revell
2004-09-23  1:13                                   ` Lee Revell
2004-09-23 12:28                                   ` [patch] voluntary-preempt-2.6.9-rc2-mm1-S4 Ingo Molnar
2004-09-23 12:57                                     ` Norberto Bensa
2004-09-23 13:01                                       ` Ingo Molnar
2004-09-23 13:07                                         ` Ingo Molnar
2004-09-23 13:09                                         ` Ingo Molnar
2004-09-23 16:46                                           ` Norberto Bensa
2004-09-23 17:13                                             ` Norberto Bensa
2004-09-23 18:20                                               ` Ingo Molnar
2004-09-23 13:35                                     ` Rui Nuno Capela
2004-09-23 13:40                                       ` Ingo Molnar
2004-09-23 16:23                                         ` Rui Nuno Capela
2004-09-23 18:07                                           ` Rui Nuno Capela
2004-09-24 12:43                                             ` OHCI_QUIRK_INITRESET (was: 2.6.9-rc2-mm2 ohci_hcd doesn't work) Rui Nuno Capela
2004-09-24 12:55                                               ` Ingo Molnar
2004-09-24 14:00                                                 ` Rui Nuno Capela
2004-09-24 16:16                                                   ` Bjorn Helgaas
2004-09-25 23:37                                                     ` David Brownell
2004-09-26 13:09                                                       ` Rui Nuno Capela
2004-09-27 15:11                                                       ` Bjorn Helgaas
2004-09-23 21:12                                     ` [patch] voluntary-preempt-2.6.9-rc2-mm3-S5 Ingo Molnar
2004-09-24  0:32                                       ` Rui Nuno Capela
2004-09-24  2:22                                       ` K.R. Foley
2004-09-24  3:30                                         ` K.R. Foley
2004-09-24  7:40                                           ` Ingo Molnar
2004-09-24 11:05                                             ` K.R. Foley
2004-09-24 11:45                                               ` Ingo Molnar
2004-09-24  7:44                                       ` [patch] voluntary-preempt-2.6.9-rc2-mm3-S6 Ingo Molnar
2004-09-28  0:05                                         ` [patch] voluntary-preempt-2.6.9-rc2-mm4-S7 Ingo Molnar
2004-09-28 20:17                                           ` Rui Nuno Capela
2004-09-28 21:03                                             ` Rui Nuno Capela
2004-09-28 21:46                                               ` Rui Nuno Capela
2004-09-28 22:01                                                 ` Matt Heler
2004-09-29 17:43                                           ` Lee Revell
2004-09-29 18:40                                           ` Lee Revell
2004-09-29 20:30                                             ` Ingo Molnar
2004-09-29 20:34                                               ` Lee Revell
2004-10-02  3:02                                           ` Lee Revell
2004-10-02  9:50                                             ` Ingo Molnar
2004-10-03  2:01                                           ` Lee Revell
2004-10-03  2:14                                             ` Lee Revell
2004-10-03  2:19                                               ` Lee Revell
2004-10-03 20:08                                             ` Ingo Molnar
2004-10-03  6:37                                           ` Lee Revell
2004-10-03  6:50                                             ` Lee Revell
2004-10-03  7:06                                               ` Lee Revell
2004-10-03 19:57                                                 ` Ingo Molnar
2004-10-04  0:53                                                   ` Lee Revell
2004-10-04 10:17                                                     ` Ingo Molnar
2004-10-04 17:20                                                       ` Lee Revell
2004-10-03 20:05                                             ` Ingo Molnar
2004-10-03 21:09                                           ` [patch] voluntary-preempt-2.6.9-rc3-mm1-S8 Ingo Molnar
2004-10-04 21:53                                             ` [patch] voluntary-preempt-2.6.9-rc3-mm1-S9 Ingo Molnar
2004-10-05  0:31                                               ` Lee Revell
2004-10-05  0:56                                                 ` Florian Schmidt
2004-10-05  0:45                                                   ` Lee Revell
2004-10-05  0:38                                               ` Andrew Rodland
2004-10-05  1:19                                               ` Rui Nuno Capela
2004-10-05  5:32                                                 ` Peter Williams
2004-10-05  6:38                                                 ` Ingo Molnar
2004-10-05  1:42                                               ` Florian Schmidt
2004-10-05  2:05                                                 ` Florian Schmidt
2004-10-05  3:09                                                 ` Andrew Rodland
2004-10-05 10:24                                                   ` Florian Schmidt
2004-10-05  7:02                                               ` [patch] voluntary-preempt-2.6.9-rc3-mm2-T0 Ingo Molnar
2004-10-05 11:11                                                 ` Rui Nuno Capela
2004-10-05 11:17                                                   ` Ingo Molnar
2004-10-05 12:07                                                     ` Hugh Dickins
2004-10-05 11:12                                                 ` Florian Schmidt
2004-10-05 11:03                                                   ` Ingo Molnar
2004-10-05 11:14                                                     ` Rui Nuno Capela
2004-10-05 11:16                                                       ` Ingo Molnar
2004-10-05 13:47                                               ` [patch] voluntary-preempt-2.6.9-rc3-mm2-T1 Ingo Molnar
2004-10-05 16:37                                                 ` Rui Nuno Capela
2004-10-05 18:42                                                   ` Ingo Molnar
2004-10-05 19:38                                                     ` Rui Nuno Capela
2004-10-05 19:44                                                       ` Ingo Molnar
2004-10-05 20:01                                                         ` Rui Nuno Capela
2004-10-06  0:12                                                         ` Lee Revell
2004-10-06  7:51                                                           ` Ingo Molnar
2004-10-07 10:52                                                 ` [patch] voluntary-preempt-2.6.9-rc3-mm3-T3 Ingo Molnar
     [not found]                                                   ` <20041007134116.3e53b239.h.mth@web.de>
2004-10-07 11:44                                                     ` Ingo Molnar
2004-10-07 12:08                                                       ` Hanno Meyer-Thurow
2004-10-07 12:16                                                   ` Rui Nuno Capela
2004-10-07 13:53                                                   ` Rui Nuno Capela
2004-10-07 14:13                                                     ` voluntary-preempt-2.6.9-rc3-mm3-T3 Con Kolivas
2004-10-07 23:26                                                     ` [patch] voluntary-preempt-2.6.9-rc3-mm3-T3 Rui Nuno Capela
2004-10-08  5:36                                                       ` Lee Revell
2004-10-08  6:49                                                         ` Con Kolivas
2004-10-08 18:05                                                           ` Lee Revell
2004-10-08  7:06                                                         ` Ingo Molnar
2004-10-08  7:36                                                           ` Peter Williams
2004-10-08 17:27                                                             ` Lee Revell
2004-10-07 17:55                                                   ` K.R. Foley
2004-10-07 20:29                                                     ` K.R. Foley
2004-10-07 21:55                                                       ` Ingo Molnar
2004-10-08  1:41                                                         ` K.R. Foley
2004-10-08  7:02                                                           ` Ingo Molnar
2004-10-08 14:03                                                             ` K.R. Foley
2004-10-08 14:28                                                               ` Ingo Molnar
2004-10-08 21:14                                                   ` Lee Revell
2004-10-08 23:11                                                   ` Lee Revell
2004-10-09  4:16                                                   ` Lee Revell
2004-10-09  4:57                                                   ` Lee Revell
2004-10-09  5:09                                                     ` voluntary-preempt-2.6.9-rc3-mm3-T3 Con Kolivas
2004-10-09  5:21                                                       ` voluntary-preempt-2.6.9-rc3-mm3-T3 Lee Revell
2004-10-09  5:23                                                         ` voluntary-preempt-2.6.9-rc3-mm3-T3 Con Kolivas
2004-10-09  5:31                                                           ` voluntary-preempt-2.6.9-rc3-mm3-T3 Lee Revell
2004-10-09  5:34                                                             ` voluntary-preempt-2.6.9-rc3-mm3-T3 Con Kolivas
2004-10-09  5:50                                                               ` Preemption model (was Re: voluntary-preempt-2.6.9-rc3-mm3-T3) Lee Revell
2004-10-09  5:53                                                                 ` Con Kolivas
2004-10-09 10:46                                                         ` voluntary-preempt-2.6.9-rc3-mm3-T3 Ingo Molnar
2004-10-09 13:21                                                         ` voluntary-preempt-2.6.9-rc3-mm3-T3 K.R. Foley
2004-10-09 18:16                                                   ` [patch] voluntary-preempt-2.6.9-rc3-mm3-T3 Florian Schmidt
2004-10-11 14:29                                                   ` Ingo Molnar [this message]
2004-10-11 17:48                                                     ` [patch] CONFIG_PREEMPT_REALTIME, 'Fully Preemptible Kernel', VP-2.6.9-rc4-mm1-T4 Florian Schmidt
2004-10-11 21:22                                                     ` Rui Nuno Capela
2004-10-11 21:37                                                       ` Lee Revell
2004-10-12  4:30                                                   ` [patch] voluntary-preempt-2.6.9-rc3-mm3-T3 Lee Revell
2004-10-12  9:17                                                     ` Ingo Molnar
2004-10-13 16:52                                                       ` Lee Revell
2004-10-13 16:48                                                   ` Lee Revell
2004-10-08 11:16                                                 ` [patch] voluntary-preempt-2.6.9-rc3-mm2-T1 William Lee Irwin III
2004-09-22 17:30                                 ` Oops in __posix_lock_file was:Re: [patch] voluntary-preempt-2.6.9-rc2-mm1-S2 K.R. Foley
2004-09-22 19:43                                   ` Ingo Molnar
2004-09-21 18:24                           ` [patch] voluntary-preempt-2.6.9-rc2-mm1-S1 K.R. Foley
2004-09-21 19:21                             ` Ingo Molnar
2004-09-21 19:37                               ` K.R. Foley
2004-09-08  6:56       ` [patch] voluntary-preempt-2.6.9-rc1-bk12-R6 Lee Revell
2004-09-09 19:29         ` Ingo Molnar
2004-09-09 19:33           ` Lee Revell
2004-09-09 20:43             ` Lee Revell
2004-09-09 20:51               ` Ingo Molnar
2004-09-09 21:03               ` Scott Wood
2004-09-09 20:05           ` Andrew Morton
2004-09-09 20:09             ` Alan Cox
2004-09-09 21:28               ` Andrew Morton
2004-09-09 22:45             ` William Lee Irwin III
2004-09-09 22:11               ` Alan Cox
2004-09-09 23:20                 ` William Lee Irwin III
2004-09-10 13:28             ` Ingo Molnar
2004-09-10 14:28               ` Paolo Ciarrocchi
2004-09-10 16:45                 ` Lee Revell
2004-09-10 22:54               ` Lee Revell
2004-09-11  0:21                 ` K.R. Foley
2004-09-09 20:13           ` Lee Revell
2004-09-22  0:17           ` William Lee Irwin III
2004-09-07 22:55     ` [patch] voluntary-preempt-2.6.9-rc1-bk12-R5 Lee Revell
2004-09-07 23:13       ` Lee Revell
2004-09-07 23:58         ` Rui Nuno Capela
2004-09-08  8:23           ` Ingo Molnar
2004-09-08  8:31             ` Ingo Molnar
2004-09-09 11:09               ` Rui Nuno Capela
2004-09-15  1:16                 ` Lee Revell
2004-09-15  9:29                   ` Rui Nuno Capela
2004-09-15  9:38                     ` Ingo Molnar
2004-09-15  9:53                       ` Rui Nuno Capela
2004-09-15 10:00                         ` Ingo Molnar
2004-09-15 10:35                           ` Rui Nuno Capela
2004-09-25 19:26               ` Lee Revell
2004-09-25 20:38                 ` Ingo Molnar
2004-09-25 20:40                   ` Lee Revell
2004-09-25 20:50                     ` Duncan Sands
2004-09-25 23:54                       ` Lee Revell
2004-09-26 13:42                         ` Rui Nuno Capela
2004-09-08  8:46             ` Rui Nuno Capela
2004-09-08  8:52               ` Ingo Molnar
2004-09-05 14:49   ` [patch] voluntary-preempt-2.6.9-rc1-bk4-R4 Florian Schmidt
2004-09-05 14:53     ` K.R. Foley
2004-10-11 18:23 [patch] CONFIG_PREEMPT_REALTIME, 'Fully Preemptible Kernel', VP-2.6.9-rc4-mm1-T4 Mark_H_Johnson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20041011142953.GA32607@elte.hu \
    --to=mingo@elte.hu \
    --cc=Mark_H_Johnson@raytheon.com \
    --cc=akpm@osdl.org \
    --cc=dwalker@mvista.com \
    --cc=kr@cybsft.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mista.tapas@gmx.net \
    --cc=nando@ccrma.Stanford.EDU \
    --cc=rlrevell@joe-job.com \
    --cc=rncbc@rncbc.org \
    --subject='Re: [patch] CONFIG_PREEMPT_REALTIME, '\''Fully Preemptible Kernel'\'', VP-2.6.9-rc4-mm1-T4' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.