2.5 merge candidate list, final version. (End of "crunch time" series.)

2.5 merge candidate list, final version. (End of "crunch time" series.)

[Rob Landley]

Hi Linus.

This list is the result of a week of scouring linux-kernel and posting
more or less daily versions soliciting feedback from everybody seriously
trying to get a patch into 2.5.  This is the ninth and final posting of
this list.

Previous versions, and the discussion they spawned, were here:
1.0) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7006.html
1.1) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7051.html
1.2) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7363.html
1.3) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7452.html
1.4) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7827.html
1.5) http://lists.insecure.org/lists/linux-kernel/2002/Oct/8174.html
1.6) http://lists.insecure.org/lists/linux-kernel/2002/Oct/8787.html
1.7) http://lists.insecure.org/lists/linux-kernel/2002/Oct/9130.html

The vast majority of the 30 items in this list will probably be
dropped, but if this list results in explicit rejections instead of patches
getting missed or lost in the shuffle, it will have served its purpose.

On a side note, Dave Jones has started a 2.5 series development summary:
http://www.codemonkey.org.uk/post-halloween-2.5.txt

================================= Intro ====================================

The following features aim to be ready for submission to Linus by Monday,
October 28th, to be considered for inclusion (in 2.5.45) before the feature
freeze on Thursday, October 31 (halloween).

This list is just pending features trying to get in before feature freeze.
It's primarily for features that need more testing, or might otherwise get
forgotten in the rush.  If you want to know what's already gone in, or what's
being worked on for the next development cycle, check out:
http://kernelnewbies.org/status

Thanks to Rusty Russell and Guillaume Boissiere, whose respective 2.5 merge
candidate lists have been ruthlessly strip-mined in the process of
assembling this.  And to everybody who's emailed stuff.

============================ Pending features: =============================

1) Andrew Morton's -mm tree. (Andrew Morton, editor.)

Andrew Morton's -mm tree collates several other projects, including:

The ext2/ext3 Extended Attributes and Access Control Lists patch from Ted Tso
and Andreas Gruenbacher (ext23-*.patch), Page Table Sharing from Daniel
Phillips and Dave McCracken (shpte-ng.patch), a bunch of huge page upgrades
from Bill Irwin (hugetlb*.patch), the orlov allocator, Ingo's generic
nonlinear mappings...

Stuff.  Lots of stuff.

You can get Andrew Morton's MM tree from the following URL, including a
broken-out patches directory and a description file.  (The latest version
as of this writing is -mm6.)

http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.44

----------------------------------------------------------------------------

2) Device mapper for Logical Volume Manager (LVM2)  (LVM2 team)  (in -ac)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103536883428443&w=2

Download:
http://people.sistina.com/~thornber/patches/2.5-stable/

Home page:
http://www.sistina.com/products_lvm.htm

Note: this is in the 2.5-ac tree, available at:
http://www.kernel.org/pub/linux/kernel/people/alan/

----------------------------------------------------------------------------

3) EVMS (Enterprise Volume Management System) (IBM, Contact: Kevin Corry)

Fighting with LVM2 for a place in the tree, a bigger solution to a bigger
set of problems:

Home page:
http://sourceforge.net/projects/evms

Home page:
http://evms.sourceforge.net

Download:
http://evms.sourceforge.net/patches/

Some related discussions:
http://marc.theaimsgroup.com/?t=103359686900003&r=1&w=2
http://marc.theaimsgroup.com/?t=103439913000001&r=1&w=2
http://marc.theaimsgroup.com/?w=2&r=1&s=%5Bpatch%5D+evms+core&q=t

----------------------------------------------------------------------------

4) New kernel configuration system (Roman Zippel)

Announcement:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/9043.html

Download:
http://www.xs4all.nl/~zippel/lc/

Linus has actually looked fairly favorably on this one so far:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/3250.html

And an AOL for it:

http://lists.insecure.org/lists/linux-kernel/2002/Oct/8255.html

----------------------------------------------------------------------------

5) Linux Trace Toolkit (LTT) (Karim Yaghmour)

Announce:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/7016.html

Download:
http://opersys.com/ftp/pub/LTT/ExtraPatches/patch-ltt-linux-2.5.44-vanilla-021026-2.2.bz2

User tools:
http://opersys.com/ftp/pub/LTT/TraceToolkit-0.9.6pre2.tgz

----------------------------------------------------------------------------

6) Kernel Probes (IBM, contact: Vamsi Krishna S)

Kprobes announcement:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528410215211&w=2

Download Base Kprobes Patch:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528425615302&q=raw

KProbes->DProbes support patches:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528454215523&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528454015520&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528485415813&q=raw

Download gzipped tarball patches from official IBM site:
http://www-124.ibm.com/linux/patches/?project_id=141

KProbes home page:
http://www-124.ibm.com/linux/projects/kprobes

A good explanation of the difference between kprobes, dprobes,
and kernel hooks:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103532874900445&w=2

Clarification: just kprobes is being submitted for 2.5.45, (and
optionally some basic dprobes support,) but not the whole of dprobes:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103536827928012&w=2
And also:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/9160.html

----------------------------------------------------------------------------

7) Posix support and High Resolution timers (George Anzinger)

George has a patch that provides posix support, and on top of that a
patch to provide high resolution timers.  He talks about it here:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103562196700924&w=2

The project's home page is here:
http://high-res-timers.sourceforge.net/

Downloadable patches may be found here:
http://sourceforge.net/project/showfiles.php?group_id=20460&release_id=118345

And descriptions for each patch (in the order they should be applied) are
available here (although in this case the archive truncates the patches
themselves, use the download link above):
http://marc.theaimsgroup.com/?l=linux-kernel&m=103553654329827&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103557676007653&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103557677207693&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103558349714128&w=2

Linus had concerns with this one a while back:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/3463.html

----------------------------------------------------------------------------

8) Posix timers alternate implementation (Jim Houston)

Jim Houston has an alternate patch to provide posix support (but not
high resolution timers on top of it, yet).

http://marc.theaimsgroup.com/?l=linux-kernel&m=103549000027416&q=raw

----------------------------------------------------------------------------

9) Linux Kernel Crash Dumps (Matt Robinson, LKCD team)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103553563728914&w=2
And again:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103536576625905&w=2

Download:
http://lkcd.sourceforge.net/download/latest/

----------------------------------------------------------------------------

10) Rewrite of the console layer (James Simmons)

Announcement:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103487329526903&w=2

Home page:
http://linuxconsole.sourceforge.net/

Downloadable patch:
http://phoenix.infradead.org/~jsimmons/fbdev.diff.gz

Bitkeeper tree:
bk://fbdev.bkbits.net/fbdev-2.5

----------------------------------------------------------------------------

11) IPv6 upgrades and crypto API (Yoshifuji Hideyaki)

The Usagi ipv6 upgrades have been available for a while, and their
author would like to see them in 2.5:

README:
ftp://ftp.linux-ipv6.org/pub/usagi/patch/ipsec/README.IPSEC

Downloadable patch here:
ftp://ftp.linux-ipv6.org/pub/usagi/patch/ipsec/ipsec-2.5.43-ALL-03.patch.gz

Dave Miller is doing a major ipv6 layer rewrite, but no patch has been sent
to the list yet.  Ironically, James Morris has a Crypto API patch on top of
Dave Miller's tree:

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103559983324080&w=2

Download:
http://samba.org/~jamesm/crypto/

----------------------------------------------------------------------------

12) MMU-less processor support (Greg Ungerer)

Most recent announcement (with links):
http://marc.theaimsgroup.com/?l=linux-kernel&m=103578189421588&w=2

A version of this is in the 2.5-ac tree.  An announcement of a patch against
that is here:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103578338922236&w=2

----------------------------------------------------------------------------

13) sys_epoll (I.E. /dev/poll) (Davide Libenzi)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103542994232004&w=2

Homepage:
http://www.xmailserver.org/linux-patches/nio-improve.html

Download:
http://www.xmailserver.org/linux-patches/sys_epoll-2.5.44-last.diff

Linus participated repeatedly in a thread on this one too, expressing
concerns which (hopefully) have been addressed.  See:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6428.html

----------------------------------------------------------------------------

14) CD Recording/sgio patches (Jens Axboe)

Announce:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8060.html

Download patch:
http://www.kernel.org/pub/linux/kernel/people/axboe/patches/v2.5/2.5.44/sgio-15.bz2

This should be in Alan Cox's tree as of 2.4.44-ac4.

----------------------------------------------------------------------------

15) In-kernel module loader (Rusty Russell.)

Announce:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6214.html

Download patch:
http://www.kernel.org/pub/linux/kernel/people/rusty/patches/module-x86-18-10-2002.2.5.43.diff.gz

----------------------------------------------------------------------------

16) Unlimited groups patch (Tim Hockin.)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524761319825&w=2

Download patch set from:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524717119443&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524761819834&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524761619831&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524761519829&q=raw

----------------------------------------------------------------------------

17) Initramfs (Al Viro)

Way back when, Al said:
http://www.cs.helsinki.fi/linux/linux-kernel/2001-30/0110.html

Download (most recent patch so far):
ftp://ftp.math.psu.edu/pub/viro/N0-initramfs-C40

And Linus recently made happy noises about the idea:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/1110.html

----------------------------------------------------------------------------

18) Kernel Hooks (IBM contact: Vamsi Krishna S.)

Website:
http://www-124.ibm.com/linux/projects/kernelhooks/

Download site:
http://www-124.ibm.com/linux/patches/?patch_id=595

Posted patch:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103364774926440&w=2

----------------------------------------------------------------------------

19) NMI request/release interface (Corey Minyard)

He says:
> Add a request/release mechanism to the kernel (x86 only for now) for NMIs.
...
>I have modified the nmi watchdog to use this interface, and it
>seems to work ok.  Keith Owens is copied to see if he would be
>interested in converting kdb to use this, if it gets put into the kernel.

There was a lot of back and forth, resulting in the latest patch (version 8):
http://home.attbi.com/~minyard/linux-nmi-v8.diff

----------------------------------------------------------------------------

20) DriverFS Topology (Matthew Dobson)

Announcement:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103523702710396&w=2

Patches:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540707113401&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540757613962&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540758013984&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540757513957&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540757813966&q=raw

----------------------------------------------------------------------------

21) Advanced TCA Disk Hotswap (Steven Dake)

Announcement of most recent patch, with links:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103558466315221&w=2

Steven's comments:

> This is a generic feature that provides good hotswap support for SCSI
> and FibreChannel disk devices.  The entire SCSI layer has been properly
> analyzed to provide correct locking and a complete RAMFS filesystem is
> available to control the kernel disk hotswap operations.
>
> Both Alan Cox and Greg KH have looked at the patch for 2.4 and suggested
> if I ported to 2.5 and made some changes (as I have in the latest port)
> this feature would be a good candidate for the 2.5 kernel.
>
> A thread discussing Advanced TCA hotswap (of which this partch is one
> part of) can be found at:
> http://marc.theaimsgroup.com/?t=103462115700001&r=1&w=2

----------------------------------------------------------------------------

22) Mobile IPV6 (contact: Antti Tuominen)

Antti Tuominen says:

> We've been working on an implementation of Mobility Support in IPv6
> specification, called MIPL Mobile IPv6 for Linux.  We are now trying
> to get it included in the kernel.  Mobile IPv6 is an integral part of
> the IPv6 protocol.
>
> We've had discussion with Alexey Kuznetsov and Dave Miller.  Dave says
> he does not know enough about IPv6, and trusts Alexey on this one.
> Alexey requested the patch to be split, which we did, and we are
> currently waiting for additional comments whether he is going to
> recommend inclusion.
>
> This project has nothing to do with USAGI IPv6 Project (though they do
> merge our code from time to time).  However, we would benefit from
> having IPSec support for IPv6 in the kernel.
>
> MIPL Mobile IPv6 for Linux Project site:
> http://www.mipl.mediapoli.com/
>
> Patches:
> http://www.mipl.mediapoli.com/patches/

----------------------------------------------------------------------------

23) SCSI multi-path I/O (Patrick Mansfield)

Announcement with URLs:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8736.html

) VFS Intent Lookup Patch (Peter Braam)

Download:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103552797823568&q=raw

----------------------------------------------------------------------------

24) NUMA Scheduler Upgrade

Erich Focht and Michael Hohnbaum have two different NUMA scheduler
patches.

Michael has a stripped down NUMA scheduler, which he says was created
because the full Node Affine NUMA Scheduler didn't look like it would
be ready for 2.5.  He talks about it here, with links to patches:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103548635122591&w=2

Meanwhile, Erich Focht says the full Node Affine Numa Scheduler is
indeed ready for 2.5, and already in use at customer sites.  He makes
his case here, with links to patches, home page, LWN review, etc:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103549657202782&w=2

Here's Erich's scheduler's home page:
http://home.arcor.de/efocht/sched/

The most current version of the patches are downloadable from:

http://home.arcor.de/efocht/sched/01-numa_sched_core-2.5.44-10a.patch
http://home.arcor.de/efocht/sched/02-numa_sched_ilb-2.5.44-10.patch

Martin J. Bligh has been testing both NUMA schedulers, and wandering
back and forth in his endorsement.  At first he was leaning towards
Erich's patch, now he seems to be leaning towards Michael's.

That thread is here:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8904.html

----------------------------------------------------------------------------

25) ptrace over fork (Daniel Jacobowitz)

At the last possible second, this was submitted:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103574480632057&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103574480232051&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103574511932225&q=raw

----------------------------------------------------------------------------

26) Kexec, launch new linux kernel from Linux (Eric W. Biederman)

Announcement, description, links, and patch.  All in one:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103579334328023&q=raw

P.S.  This thread is just too brazen not to include:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/7952.html

----------------------------------------------------------------------------

27) Nanosecond support in stat.

Discussion thread:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8983.html

Download:
ftp://ftp.firstfloor.org/pub/ak/v2.5/nsec-2.5.44-2.bz2

----------------------------------------------------------------------------

28) Digital Video Broadcasting Layer (LinuxTV team)

Home page:
http://www.linuxtv.org:81/dvb/

Download:
http://www.linuxtv.org:81/download/dvb/

This is also in Alan Cox's 2.5 tree.

----------------------------------------------------------------------------

29) Hotplug CPU Removal (Rusty Russell)

On sunday, Rusty announced the submission of hotplug against 2.5.44:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/9141.html

(Unfortunately, he notes the patch conflicts with the -mm5 tree.)

Download:
http://www.kernel.org/pub/linux/kernel/people/rusty/patches/Hotcpu/hotcpu-cpudown.patch.gz

----------------------------------------------------------------------------

30) Reiser4.

I don't have a patch yet, but Hans Reiser is very insistent that this
will be ready by halloween.  (VERY insistent.)  I'll let him speak for
himself:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8793.html

And again (promises, promises):
http://lists.insecure.org/lists/linux-kernel/2002/Oct/9082.html

Still no patch at the time of this writing, though.  In theory it
should show up here:
http://namesys.com/download.html

Or perhaps here:
ftp://ftp.lugoj.org/pub/reiserfs/devlinux.com/pub/namesys/reiserfs-for-2.5

In the meantime, all I can find on Reiser4 is some kind of hybrid
marketing brochure/design document thing:
http://www.reiserfs.org/v4/v4.html

Did I mention Hans was insistent?  The man can make puppy eyes through email.
It's quite impressive.
-- 
http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?

Re: 2.5 merge candidate list, final version. (End of "crunch time" series.)

[Nikita Danilov]

Rob Landley writes:
 > Hi Linus.
 > 
 > This list is the result of a week of scouring linux-kernel and posting
 > more or less daily versions soliciting feedback from everybody seriously
 > trying to get a patch into 2.5.  This is the ninth and final posting of
 > this list.
 > 
 > Previous versions, and the discussion they spawned, were here:
 > 1.0) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7006.html
 > 1.1) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7051.html
 > 1.2) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7363.html
 > 1.3) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7452.html
 > 1.4) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7827.html
 > 1.5) http://lists.insecure.org/lists/linux-kernel/2002/Oct/8174.html

[...]

 > http://www.kernel.org/pub/linux/kernel/people/rusty/patches/Hotcpu/hotcpu-cpudown.patch.gz
 > 
 > ----------------------------------------------------------------------------
 > 
 > 30) Reiser4.
 > 
 > I don't have a patch yet, but Hans Reiser is very insistent that this
 > will be ready by halloween.  (VERY insistent.)  I'll let him speak for
 > himself:
 > http://lists.insecure.org/lists/linux-kernel/2002/Oct/8793.html
 > 
 > And again (promises, promises):
 > http://lists.insecure.org/lists/linux-kernel/2002/Oct/9082.html
 > 
 > Still no patch at the time of this writing, though.  In theory it
 > should show up here:
 > http://namesys.com/download.html

Snapshot is available at http://www.namesys.com/snapshots/:

reiser4 proper:
http://www.namesys.com/snapshots/reiser4-2002.10.24.tar.gz

necessary changes to the core kernel, plus some UML patches, plus some
patches for debugging:
http://www.namesys.com/snapshots/reiser4-core-2002.10.24.diff

utils:
http://www.namesys.com/snapshots/reiser4progs-2002.10.24.tar.gz

I shall produce newer snapshot to-morrow.

It does work, but code is not production quality yet. Do *NOT* put
anything close to critical data on it.

 > 
 > Or perhaps here:
 > ftp://ftp.lugoj.org/pub/reiserfs/devlinux.com/pub/namesys/reiserfs-for-2.5
 > 
 > In the meantime, all I can find on Reiser4 is some kind of hybrid
 > marketing brochure/design document thing:
 > http://www.reiserfs.org/v4/v4.html
 > 
 > Did I mention Hans was insistent?  The man can make puppy eyes through email.
 > It's quite impressive.

Works without email too. :-)

 > -- 
 > http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
 > CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?

Nikita.

 > -

Re: 2.5 merge candidate list, final version. (End of "crunch time" series.)

[Tomas Szepe]

> Snapshot is available at http://www.namesys.com/snapshots/:
> 
> http://www.namesys.com/snapshots/reiser4-2002.10.24.tar.gz
> http://www.namesys.com/snapshots/reiser4-core-2002.10.24.diff

2.5.44:
Producing a module I get a couple unresolved symbols,
and trying to build directly into the kernel results in

...
ld -m elf_i386 -e stext -T arch/i386/vmlinux.lds.s arch/i386/kernel/head.o arch/i386/kernel/init_task.o  init/built-in.o --start-group  arch/i386/kernel/built-in.o  arch/i386/mm/built-in.o  arch/i386/mach-generic/built-in.o  kernel/built-in.o  mm/built-in.o  fs/built-in.o  ipc/built-in.o  security/built-in.o  lib/lib.a  arch/i386/lib/lib.a  drivers/built-in.o  sound/built-in.o  arch/i386/pci/built-in.o  net/built-in.o --end-group  -o vmlinux
fs/built-in.o(.init.text+0x1381): In function `init_reiser4':
: undefined reference to `local symbols in discarded section .exit.text'
make[1]: *** [vmlinux] Error 1
make: *** [vmlinux] Error 2

> http://www.namesys.com/snapshots/reiser4progs-2002.10.24.tar.gz

... is missing the "configure" script.
Generating one with autoconf 2.53 doesn't seem to work, either.

T.

Re: 2.5 merge candidate list, final version. (End of "crunch time" series.)

[Hans Reiser]

We are going to submit a patch appropriate for inclusion as an 
experimental FS on Halloween.   I hope you will forgive our pushing the 
limit timewise, it is not by choice, but the algorithms we used to more 
than double reiserfs V3 performance were, quite frankly, hard to code.

A description of it can be found at www.namesys.com/v4/fast_reiser4.html.

I am going to ask persons to volunteer to produce independent 
benchmarks.  I am told by Oleg and Zam that it is now faster than in the 
benchmarks in the above draft paper, and I hope to see this verified by 
persons outside Namesys.

I hope you will also find our atomic transactions infrastructure 
interesting.  It allows us to guarantee each system call to reiserfs is 
fully atomic, and in later versions we hope to use it to make atomicity 
available to user space so that multiple fs operations can be guaranteed 
atomic.  This will hopefully help to eliminate a major class of security 
holes, as well as provide significantly better protection against user 
data corruption by crashes than any current filesystem.  As you can see, 
it does not seem to substantially harm performance, or at least we 
assume that our more than doubling performance means that performance 
was not harmed.;-)

Finally, and I think in the longterm most importantly, we have 
implemented a full plugin infrastructure for reiser4.  This will make it 
possible for us to implement database and search engine functionality 
one plugin at a time.  (See www.namesys.com/whitepaper.html for a 
description of those semantics.)  We hope you will find the semantics 
much more fun than tacking SQL onto the FS like OFS is going to do.;-)

We don't know how fast OFS will be, but we hope you will enjoy watching 
us raise the bar a few feet.  We are very excited by more than doubling 
performance.  The read performance improvements are more than I expected.  

(If any of you attempt to replicate the benchmarks, please be sure to 
use reiser4 readdir order for writes to reiser4 (that means don't use 
tarballs made from ext2), and to use the latest hard drives and fast 
processors with udma 5 turned on.  We are quite sensitive to transfer 
speed since we do a good job of avoiding seeks.  We are sensitive to 
readdir order because we sort directory entries (which is necessary for 
having efficient large directory lookups)).   In reiser4.1 we will ship 
a repacker, and then it won't matter what order you do writes in so long 
as the repacker gets a chance to run at night.

Cheers,

Hans

Nikita Danilov wrote:

>Rob Landley writes:
> > Hi Linus.
> > 
> > This list is the result of a week of scouring linux-kernel and posting
> > more or less daily versions soliciting feedback from everybody seriously
> > trying to get a patch into 2.5.  This is the ninth and final posting of
> > this list.
> > 
> > Previous versions, and the discussion they spawned, were here:
> > 1.0) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7006.html
> > 1.1) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7051.html
> > 1.2) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7363.html
> > 1.3) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7452.html
> > 1.4) http://lists.insecure.org/lists/linux-kernel/2002/Oct/7827.html
> > 1.5) http://lists.insecure.org/lists/linux-kernel/2002/Oct/8174.html
>
>[...]
>
> > http://www.kernel.org/pub/linux/kernel/people/rusty/patches/Hotcpu/hotcpu-cpudown.patch.gz
> > 
> > ----------------------------------------------------------------------------
> > 
> > 30) Reiser4.
> > 
> > I don't have a patch yet, but Hans Reiser is very insistent that this
> > will be ready by halloween.  (VERY insistent.)  I'll let him speak for
> > himself:
> > http://lists.insecure.org/lists/linux-kernel/2002/Oct/8793.html
> > 
> > And again (promises, promises):
> > http://lists.insecure.org/lists/linux-kernel/2002/Oct/9082.html
> > 
> > Still no patch at the time of this writing, though.  In theory it
> > should show up here:
> > http://namesys.com/download.html
>
>Snapshot is available at http://www.namesys.com/snapshots/:
>
>reiser4 proper:
>http://www.namesys.com/snapshots/reiser4-2002.10.24.tar.gz
>
>necessary changes to the core kernel, plus some UML patches, plus some
>patches for debugging:
>http://www.namesys.com/snapshots/reiser4-core-2002.10.24.diff
>
>utils:
>http://www.namesys.com/snapshots/reiser4progs-2002.10.24.tar.gz
>
>I shall produce newer snapshot to-morrow.
>
>It does work, but code is not production quality yet. Do *NOT* put
>anything close to critical data on it.
>
> > 
> > Or perhaps here:
> > ftp://ftp.lugoj.org/pub/reiserfs/devlinux.com/pub/namesys/reiserfs-for-2.5
> > 
> > In the meantime, all I can find on Reiser4 is some kind of hybrid
> > marketing brochure/design document thing:
> > http://www.reiserfs.org/v4/v4.html
> > 
> > Did I mention Hans was insistent?  The man can make puppy eyes through email.
> > It's quite impressive.
>
>Works without email too. :-)
>
> > -- 
> > http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
> > CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?
>
>Nikita.
>
> > -
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>
>  
>


-- 
Hans

Re: 2.5 merge candidate list, final version. (End of "crunch time" series.)

[Jeff Garzik]

Hans Reiser wrote:

> We are going to submit a patch appropriate for inclusion as an 
> experimental FS on Halloween.   I hope you will forgive our pushing 
> the limit timewise, it is not by choice, but the algorithms we used to 
> more than double reiserfs V3 performance were, quite frankly, hard to 
> code.



Does your merge change the core code at all?  Does it add new syscalls?

Re: 2.5 merge candidate list, final version. (End of "crunch time"series.)

[Andrew Morton]

Jeff Garzik wrote:
> 
> Hans Reiser wrote:
> 
> > We are going to submit a patch appropriate for inclusion as an
> > experimental FS on Halloween.   I hope you will forgive our pushing
> > the limit timewise, it is not by choice, but the algorithms we used to
> > more than double reiserfs V3 performance were, quite frankly, hard to
> > code.
> 
> Does your merge change the core code at all?  Does it add new syscalls?
> 

Their changes are tiny, and sensible.  See
http://www.namesys.com/snapshots/2002.10.29/

But I'd like to ask about the status of reiser3 support.

Chris had patches *ages* ago to convert it to use direct-to-BIO for
reads, and writes should be done as well.  reiserfs3 is still using
buffer-head-based IO for bulk reads and writes.  That's a 25-30% hit
in CPU cost, and all the old ZONE_NORMAL-full-of-buffer_heads
problems.

Any plans to get that work finished off?

Re: 2.5 merge candidate list, final version. (End of "crunch time" series.)

[Nikita Danilov]

Jeff Garzik writes:
 > Hans Reiser wrote:
 > 
 > > We are going to submit a patch appropriate for inclusion as an 
 > > experimental FS on Halloween.   I hope you will forgive our pushing 
 > > the limit timewise, it is not by choice, but the algorithms we used to 
 > > more than double reiserfs V3 performance were, quite frankly, hard to 
 > > code.
 > 
 > 
 > 
 > Does your merge change the core code at all?  Does it add new syscalls?
 > 

Here is a list of things reiser4 needs from the core:

 (1) export of generic_forget_inode()

 (2) export of page_cache_readahead()

 (3) export of remove_from_page_cache()

 (4) export of fsync_super()

 (5) patch to allow safe sharing of ->journal_info pointer in task_struct
     by several users

(1)-(4) are only necessary to compile reiser4 as module.

(5) is good not only for reiser4. Without it different file systems
using ->journal_info cannot "co-exist peacefully".

No new system calls are introduced (yet).

Nikita.

 > 

Re: 2.5 merge candidate list, final version. (End of "crunch time"series.)

[Nikita Danilov]

Andrew Morton writes:
 > Jeff Garzik wrote:
 > > 
 > > Hans Reiser wrote:
 > > 
 > > > We are going to submit a patch appropriate for inclusion as an
 > > > experimental FS on Halloween.   I hope you will forgive our pushing
 > > > the limit timewise, it is not by choice, but the algorithms we used to
 > > > more than double reiserfs V3 performance were, quite frankly, hard to
 > > > code.
 > > 
 > > Does your merge change the core code at all?  Does it add new syscalls?
 > > 
 > 
 > Their changes are tiny, and sensible.  See
 > http://www.namesys.com/snapshots/2002.10.29/
 > 
 > But I'd like to ask about the status of reiser3 support.
 > 
 > Chris had patches *ages* ago to convert it to use direct-to-BIO for
 > reads, and writes should be done as well.  reiserfs3 is still using
 > buffer-head-based IO for bulk reads and writes.  That's a 25-30% hit
 > in CPU cost, and all the old ZONE_NORMAL-full-of-buffer_heads
 > problems.

Are you talking about buffer_heads_over_limit magic inside vmscan.c? Is
it to avoid deadlock with buffer_head allocation in ->vm_writeback?

 > 
 > Any plans to get that work finished off?

Chris had done great work indeed (plus data logging, plus quotas, plus
ACLs by Jeff Mahoney, etc). Unfortunately we were too busy with reiser4
here lately. Hopefully, a lot of stuff will be merged into reiserfs3.x
after Halloween.

Nikita.

Re: 2.5 merge candidate list, final version. (End of "crunch time"series.)

[Chris Mason]

On Wed, 2002-10-30 at 04:35, Andrew Morton wrote:
> Jeff Garzik wrote:
> > 
> > Hans Reiser wrote:
> > 
> > > We are going to submit a patch appropriate for inclusion as an
> > > experimental FS on Halloween.   I hope you will forgive our pushing
> > > the limit timewise, it is not by choice, but the algorithms we used to
> > > more than double reiserfs V3 performance were, quite frankly, hard to
> > > code.
> > 
> > Does your merge change the core code at all?  Does it add new syscalls?
> > 
> 
> Their changes are tiny, and sensible.  See
> http://www.namesys.com/snapshots/2002.10.29/
> 
> But I'd like to ask about the status of reiser3 support.
> 
> Chris had patches *ages* ago to convert it to use direct-to-BIO for
> reads, and writes should be done as well.  reiserfs3 is still using
> buffer-head-based IO for bulk reads and writes.  That's a 25-30% hit
> in CPU cost, and all the old ZONE_NORMAL-full-of-buffer_heads
> problems.
> 
> Any plans to get that work finished off?

Very much so, but I wanted it on top of the data journaling stuff to
make porting that easier.  But, in the interest in getting something
more stable and faster out the door asap, I'll pull just the
direct-to-BIO stuff out.

-chris