Re: [2.6] Most likely to be merged by Halloween... THE LIST]

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Michael Hohnbaum]

On Thursday 18 July 2002 10:31 am, Martin J. Bligh wrote:
> > Do you think the breakdown is realistic?
>
> Here's a list of other things I am hoping to see merged:
>
> shared pagetables
> large page support
> NUMA aware multipath IO
> NUMA aware scheduler extensions
> ia32 discontigmem support for NUMA machines
> NUMA aware slab allocator
> CONFIG_NONLINEAR (in some form, quite possibly a cut down version)
> shared pagetables
> large page support
> rcu rtcache
> rcu dcache

The work for the mentioned NUMA items is quite active.  Some of
the pieces have already been submitted and others are near completion.
I would hope that the items mentioned by Martin make it into the 2.5
kernel.  The NUMA changes tend to be very easy to isolate such that 
they only affect kernels built with the appropriate NUMA config options.

>From my list of NUMA items:

NUMA memory allocation
NUMA aware scheduler
Topology representation in kernel
Memory binding API
Per-zone swapd
Multipath support

Also, not NUMA specific but beneficial to databases (which tend to
run on NUMA platforms) is a fast user space gettimeofday capability.
This shows up in the AMD-64 port as vsyscalls.

-- 

Michael Hohnbaum                      503-578-5486
hohnbaum@us.ibm.com                   T/L 775-5486

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Hans Reiser]

Forgive a silly question due to the odd phrasing of the subject line of 
this thread:

Is Halloween the deadline for submission of patches, or the deadline for 
inclusion?  If I send in reiser4 on Halloween day according to some 
timezone;-), have I made the deadline for inclusion into 2.6 even if it 
takes Linus a few months to reach my place in the queue of patches sent 
to him on Halloween day?

I understand that earlier is better, and I will send it earlier if I 
can, but even if we do get the reiser4 core (that which does all that V3 
does but faster and on top of a plugin infrastructure) done before 
Halloween, we will inevitably add a few features and tweaks after doing 
the core, and we will want to send those in at the last minute.

-- 
Hans

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Andreas Dilger]

On Jul 20, 2002  01:28 +0400, Hans Reiser wrote:
> Is Halloween the deadline for submission of patches, or the deadline for 
> inclusion?  If I send in reiser4 on Halloween day according to some 
> timezone;-), have I made the deadline for inclusion into 2.6 even if it 
> takes Linus a few months to reach my place in the queue of patches sent 
> to him on Halloween day?
> 
> I understand that earlier is better, and I will send it earlier if I 
> can, but even if we do get the reiser4 core (that which does all that V3 
> does but faster and on top of a plugin infrastructure) done before 
> Halloween, we will inevitably add a few features and tweaks after doing 
> the core, and we will want to send those in at the last minute.

Hans,
my understanding is that core changes that aren't in by Halloween are
not going to be accepted until 2.7.  By pre-announcing the deadline,
it is hoped that people will have lots of time to submit things that are
ready for inclusion, as opposed to rushing to submit when the "freeze"
is announced all of a sudden.

If (as we all hope) the important features are added incrementally to
the development kernel over the next few months, maybe all of the usage
and testing that is going into the development kernel will not be
totally lost when the entire kernel is morphed under a huge weight of
patches.

It may even mean that there will not be an extra year of features (and
bugs) being added to the "frozen" kernel, and we will be able to start
2.7 earlier.

As always, I imagine that as long as you have any core changes in 2.5
before the freeze, it will not be impossible to add self-contained
things like filesystems and drivers after the freeze also.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Hans Reiser]

Andreas Dilger wrote:

>  
>
>Hans,
>my understanding is that core changes that aren't in by Halloween are
>not going to be accepted until 2.7.  By pre-announcing the deadline,
>it is hoped that people will have lots of time to submit things that are
>ready for inclusion, as opposed to rushing to submit when the "freeze"
>is announced all of a sudden.
>
>
>
>  
>
I, in my egocentrism, think it would make more sense to have a deadline 
for submission rather than a deadline for acceptance, as that would make 
things predictable for patch submitters, and avoid unintentional 
overlooking of good patches from obscure persons due to the crush of 
patches in October.

Pre-announcing the deadline is good, but having it be a deadline on 
something the patch submitters control (submission time not acceptance 
time) would be even better.

-- 
Hans

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Rik van Riel]

On Sat, 20 Jul 2002, Hans Reiser wrote:

> I, in my egocentrism, think it would make more sense to have a deadline
> for submission rather than a deadline for acceptance,

It's both.  We all know Linus doesn't have the time to keep
forward-porting our hundreds of patches so he can only include
patches into his kernel that apply to the exact same tree he
has at that day.

This (and the fact that Linus gets far too much email and patches
to look at old ones) is bound to make the Halloween deadline stick
for both submission and acceptance.

I hope.

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Andreas Dilger]

On Jul 20, 2002  03:37 +0400, Hans Reiser wrote:
> >my understanding is that core changes that aren't in by Halloween are
> >not going to be accepted until 2.7.  By pre-announcing the deadline,
> >it is hoped that people will have lots of time to submit things that are
> >ready for inclusion, as opposed to rushing to submit when the "freeze"
> >is announced all of a sudden.
>
> I, in my egocentrism, think it would make more sense to have a deadline 
> for submission rather than a deadline for acceptance, as that would make 
> things predictable for patch submitters, and avoid unintentional 
> overlooking of good patches from obscure persons due to the crush of 
> patches in October.
> 
> Pre-announcing the deadline is good, but having it be a deadline on 
> something the patch submitters control (submission time not acceptance 
> time) would be even better.

I would agree, except that this doesn't put any onus on the submitters
to get their patches in early, and causes the thundering heard of
patches problem the same way that not announcing the patch deadline does.

Note that "accepted" may be a bad term on my part - I can't say if this
means that the patch has been recieved by Linus, or whether it actually
has to be in the kernel tree at that date.

Note that I wasn't at the kernel summit myself, hence this is all just
what I have heard from others.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Hans Reiser]

Rik van Riel wrote:

>On Sat, 20 Jul 2002, Hans Reiser wrote:
>
>  
>
>>I, in my egocentrism, think it would make more sense to have a deadline
>>for submission rather than a deadline for acceptance,
>>    
>>
>
>It's both.  We all know Linus doesn't have the time to keep
>forward-porting our hundreds of patches so he can only include
>patches into his kernel that apply to the exact same tree he
>has at that day.
>
>This (and the fact that Linus gets far too much email and patches
>to look at old ones) is bound to make the Halloween deadline stick
>for both submission and acceptance.
>
>I hope.
>
>regards,
>
>Rik
>  
>
That could be dealt with by letting people resend feature containing 
patches that were first submitted by Halloween (forward porting them as 
things progress) until they get a rejection or Linus announces he has 
taken all that he wants from the queue.  

A thundering herd of patches is an opportunity, not a problem, unless 
they need to get applied by Halloween.;-)

-- 
Hans

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Rik van Riel]

On Sat, 20 Jul 2002, Hans Reiser wrote:

> That could be dealt with by letting people resend feature containing
> patches that were first submitted by Halloween (forward porting them as
> things progress) until they get a rejection or Linus announces he has
> taken all that he wants from the queue.

I hope the Halloween feature freeze really will be a feature
freeze.  Nothing is more frustrating than having a "stable
kernel" broken every second release by yet another feature.

If we all restrain ourselves 2.6 will be stable soon and 2.7
will be started shortly after. Backporting "essential" features
from 2.7 into a _stable_ 2.6 will be so much easier than trying
to stabilise a 2.6-pre that's full to the brim of not-yet-stable
new features.

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Hans Reiser]

What I was advocating was a schedule of:

1) feature submission deadline

2) period of working through feature backlog

3) feature acceptance ending date

Most release management teams in the industry do things this way, 
and.... I'd better say no more, those words will lose me the argument.;-)

I'll admit that in most companies the features to be merged backlog 
period lasts for only a few days, and due to the difference in scale, it 
would probably last a few weeks for Linux, but in my egocentrism I 
really think that providing submitters with certainty as to what they 
need to do to get a patch in is a good thing.

I understand though that maybe the needs of the submitters aren't the 
most important thing for Linux as a whole, and so others will 
legitimately disagree.

Hans

Rik van Riel wrote:

>On Sat, 20 Jul 2002, Hans Reiser wrote:
>
>  
>
>>That could be dealt with by letting people resend feature containing
>>patches that were first submitted by Halloween (forward porting them as
>>things progress) until they get a rejection or Linus announces he has
>>taken all that he wants from the queue.
>>    
>>
>
>I hope the Halloween feature freeze really will be a feature
>freeze.  Nothing is more frustrating than having a "stable
>kernel" broken every second release by yet another feature.
>
>If we all restrain ourselves 2.6 will be stable soon and 2.7
>will be started shortly after. Backporting "essential" features
>from 2.7 into a _stable_ 2.6 will be so much easier than trying
>to stabilise a 2.6-pre that's full to the brim of not-yet-stable
>new features.
>
>regards,
>
>Rik
>  
>


-- 
Hans

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Rik van Riel]

On Sat, 20 Jul 2002, Hans Reiser wrote:

> What I was advocating was a schedule of:
> 1) feature submission deadline
> 2) period of working through feature backlog
> 3) feature acceptance ending date

So, what feature are you trying to smuggle into the kernel
but are afraid isn't ready on time and why do you think it
couldn't be backported into 2.6 later, when 2.6 is stable ?

I can't really think of anything that couldn't be backported
later on, by the time people start actually using 2.6, but
maybe you've got something so fundamental it just has to be
merged before the feature freeze ... ;)

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Hans Reiser]

Rik van Riel wrote:

>On Sat, 20 Jul 2002, Hans Reiser wrote:
>
>  
>
>>What I was advocating was a schedule of:
>>1) feature submission deadline
>>2) period of working through feature backlog
>>3) feature acceptance ending date
>>    
>>
>
>So, what feature are you trying to smuggle into the kernel
>but are afraid isn't ready on time and why do you think it
>couldn't be backported into 2.6 later, when 2.6 is stable ?
>
Ouch.  He sees right though me.;)

The core Reiser4 code should be in time.  Reiser4 will have its core 
code stable in a month I hope, and by core code I mean code that does 
what V3 does but on top of a plugin infrastructure and faster than V3 in 
at least some measures.  

What I am worried about schedule-wise are:

* the API for exporting transactions to user space (the in kernel buffer 
management code to support it is completed but the API is not yet done). 
 Uses the new system call we are adding.

* The traditional file API is designed for efficiency of repeated 
operations to the same file.  As part of our effort to make files able 
to do everything that extended attributes can do, but more flexibly, we 
are creating a new system call.  This new system call can perform 
multiple operations on files in one system call, and is very convenient 
for a bunch of small IOs to different files.

* file inheritance

Some other reiser4 things won't make it, but they seem like they should 
be easier to get in later because they are plugins:

* encryption plugin

* ACL plugin

* audit plugin

In our development we started from the bottom layer and worked upwards, 
which means the new system call is close to last.

So, I am assuming a new system call must go in before feature freeze. 
 One can reasonably argue that because it only affects one experimental 
filesystem named reiser4 (until other FS authors see how nice it is and 
start to use it;-) ) and does not complicate VFS,  it is not core code, 
and should not be subject to the freeze.  I'll make that argument if we 
don't have it ready in time....;-)

>
>I can't really think of anything that couldn't be backported
>later on, by the time people start actually using 2.6, but
>maybe you've got something so fundamental it just has to be
>merged before the feature freeze ... ;)
>
>regards,
>
>Rik
>  
>


-- 
Hans

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Oliver Xymoron]

On Sat, 20 Jul 2002, Hans Reiser wrote:

> Rik van Riel wrote:
>
> >So, what feature are you trying to smuggle into the kernel
> >but are afraid isn't ready on time and why do you think it
> >couldn't be backported into 2.6 later, when 2.6 is stable ?
> >
> Ouch.  He sees right though me.;)
>
> The core Reiser4 code should be in time.  Reiser4 will have its core
> code stable in a month I hope, and by core code I mean code that does
> what V3 does but on top of a plugin infrastructure and faster than V3 in
> at least some measures.
>
> What I am worried about schedule-wise are:
>
> * the API for exporting transactions to user space (the in kernel buffer
> management code to support it is completed but the API is not yet done).
>  Uses the new system call we are adding.

You'd better start hashing that API out on the list yesterday.

> * The traditional file API is designed for efficiency of repeated
> operations to the same file.  As part of our effort to make files able
> to do everything that extended attributes can do, but more flexibly, we
> are creating a new system call.  This new system call can perform
> multiple operations on files in one system call, and is very convenient
> for a bunch of small IOs to different files.

And this one.

> * file inheritance

And possibly this. Is this transparency of some form?.

> So, I am assuming a new system call must go in before feature freeze.
>  One can reasonably argue that because it only affects one experimental
> filesystem named reiser4 (until other FS authors see how nice it is and
> start to use it;-) ) and does not complicate VFS,  it is not core code,
> and should not be subject to the freeze.  I'll make that argument if we
> don't have it ready in time....;-)

It only doesn't complicate VFS because we haven't discussed generalizing
it yet. The desire to do transactional processing is not unique to Reiser
any more than journalling is. See recent thread about fsync and MTAs.

If you hide all your nifty features under a carpet (aka ioctl) where no
one but Viro notices them, then maybe you'll be able to push this stuff
late September and get them in. But FS-specific syscalls are a non-starter
- what happens to the second FS that decides it wants transactions or
multi-file I/O but doesn't quite fit your model?

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Hans Reiser]

Oliver Xymoron wrote:

>  
>
>
>  
>
>>So, I am assuming a new system call must go in before feature freeze.
>> One can reasonably argue that because it only affects one experimental
>>filesystem named reiser4 (until other FS authors see how nice it is and
>>start to use it;-) ) and does not complicate VFS,  it is not core code,
>>and should not be subject to the freeze.  I'll make that argument if we
>>don't have it ready in time....;-)
>>    
>>
>
>It only doesn't complicate VFS because we haven't discussed generalizing
>it yet. The desire to do transactional processing is not unique to Reiser
>any more than journalling is. See recent thread about fsync and MTAs.
>
>
>If you hide all your nifty features under a carpet (aka ioctl) where no
>one but Viro notices them, then maybe you'll be able to push this stuff
>late September and get them in. But FS-specific syscalls are a non-starter
>- what happens to the second FS that decides it wants transactions or
>multi-file I/O but doesn't quite fit your model?
>
>  
>
The way the Linux community works is first you show that it works, then 
you ask others to follow you.  In 2.6 we will show that it works.  In 
2.7 we will ask others to consider adopting it.  In 5-15 years, some of 
them will.  Trust me that there is no queue of other FS authors seeking 
to take advantage of our code, and complaining that we are locking them 
out from our transactions API.  Shoot, some of them are still using 
linear search directories and not packing small files together into one 
block.;-)

So, we are not hiding it under a carpet in ioctl, we are doing something 
clean and powerful and general enough to be usable by any other storage 
layer that chooses to implement the required functions.

We need to compete with Microsoft's OFS.  We have a tight and busy 
schedule if we are to ship a filesystem offering superior semantics for 
semi-structured data queries with a clean and powerful code architecture 
behind it, and ship it before OFS ships.  Reiser4 gets the storage layer 
foundation in place, and reduces the amount of code required for the 
later stages several fold.  It also provides a framework for several 
serious approaches to security problems (like giving apps a transactions 
API), and has some features that are just plain nifty (e.g. inheritance).

All of this squabbling over filesystem (distro) competition within Linux 
is just a distraction from the important job of getting something ready 
for when OFS comes over that hill over yonder there.....  unless you 
want to hear from your friends in 2004+ about the superior namespace 
integration Longhorn offers in Windows compared to Linux, and how it 
makes everything simpler....

-- 
Hans

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Marcin Dalecki]

Hans Reiser wrote:
> Rik van Riel wrote:
> 
>> On Sat, 20 Jul 2002, Hans Reiser wrote:
>>
>>  
>>
>>> What I was advocating was a schedule of:
>>> 1) feature submission deadline
>>> 2) period of working through feature backlog
>>> 3) feature acceptance ending date
>>>   
>>
>>
>> So, what feature are you trying to smuggle into the kernel
>> but are afraid isn't ready on time and why do you think it
>> couldn't be backported into 2.6 later, when 2.6 is stable ?
>>
> Ouch.  He sees right though me.;)
> 
> The core Reiser4 code should be in time.  Reiser4 will have its core 
> code stable in a month I hope, and by core code I mean code that does 
> what V3 does but on top of a plugin infrastructure and faster than V3 in 
> at least some measures. 
> What I am worried about schedule-wise are:
> 
> * the API for exporting transactions to user space (the in kernel buffer 
> management code to support it is completed but the API is not yet done). 
> Uses the new system call we are adding.
> 
> * The traditional file API is designed for efficiency of repeated 
> operations to the same file.  As part of our effort to make files able 
> to do everything that extended attributes can do, but more flexibly, we 
> are creating a new system call.  This new system call can perform 
> multiple operations on files in one system call, and is very convenient 
> for a bunch of small IOs to different files.
> 
> * file inheritance
> 
> Some other reiser4 things won't make it, but they seem like they should 
> be easier to get in later because they are plugins:
> 
> * encryption plugin
> 
> * ACL plugin
> 
> * audit plugin

I strongly oppose OSes with mutating semantics and don't like the
"plugin" idea at all therefore.

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Bill Davidsen]

On Sun, 21 Jul 2002, Marcin Dalecki wrote:

> I strongly oppose OSes with mutating semantics and don't like the
> "plugin" idea at all therefore.

I don't think that's the case, it's a little hard to say modules are a
plus and plugins are evil. This is not the core of the filesystem being
plugged, at least I hope not, but features which might have a new
implementation, or which may not be ready at the freeze. Think PAM for
filesystems.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Marcin Dalecki]

Bill Davidsen wrote:
> On Sun, 21 Jul 2002, Marcin Dalecki wrote:
> 
> 
>>I strongly oppose OSes with mutating semantics and don't like the
>>"plugin" idea at all therefore.
> 
> 
> I don't think that's the case, it's a little hard to say modules are a
> plus and plugins are evil. This is not the core of the filesystem being
> plugged, at least I hope not, but features which might have a new
> implementation, or which may not be ready at the freeze. Think PAM for
> filesystems.

I don't like PAM either.

But a better analogy is: Think flash in HTML.

Re: [2.6] Most likely to be merged by Halloween... THE LIST]

[Hans Reiser]

Marcin Dalecki wrote:

>
> I don't like PAM either.
>
> But a better analogy is: Think flash in HTML.
>
>
>
Try plugins for Photoshop.  Or think of it as being like VFS but more 
so.  You can add good features or bad features, but being able to add 
features with fewer lines of code per feature is a plus.
Oses that do not mutate go extinct.  I don't expect any of this to 
convince you though.  People who don't like change are not swayable by 
argument for the most part.

-- 
Hans